Full Code of KMnO4-zx/TinyRAG for AI

Repository: KMnO4-zx/TinyRAG
Branch: master
Commit: 3af6d24ae99a
Files: 20
Total size: 571.4 KB

Directory structure:
gitextract_93gfmh96/

├── .gitignore
├── Blog/
│   └── Hand-on-RAG.md
├── RAG/
│   ├── Embeddings.py
│   ├── LLM.py
│   ├── Reranker.py
│   ├── VectorBase.py
│   └── utils.py
├── README.md
├── data/
│   ├── 2024-01-23_10-07-23/
│   │   └── Git教程.md
│   ├── 2024-01-23_10-10-05/
│   │   └── Git教程.md
│   ├── 2024-01-23_10-14-41/
│   │   └── Github Api 使用.md
│   ├── 2024-01-23_11-59-03/
│   │   └── Github Api 使用.md
│   └── 2024-01-23_12-01-19/
│       └── Github Api 使用.md
├── example.py
├── example_with_reranker.py
├── paper.md
├── requirements.txt
├── storage/
│   ├── doecment.json
│   └── vectors.json
└── test.ipynb

================================================
FILE CONTENTS
================================================

================================================
FILE: .gitignore
================================================
### Python template
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
.vscode/
# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
#  Usually these files are written by a python script from a template
#  before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
.pybuilder/
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
#   For a library or package, you might want to ignore these files since the code is
#   intended to run in multiple environments; otherwise, check them in:
# .python-version

# pipenv
#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
#   However, in case of collaboration, if having platform-specific dependencies or dependencies
#   having no cross-platform support, pipenv may install dependencies that don't work, or not
#   install all needed dependencies.
#Pipfile.lock

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.conda/
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/

### JetBrains template
# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio, WebStorm and Rider
# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839

# User-specific stuff
.idea/**/workspace.xml
.idea/**/tasks.xml
.idea/**/usage.statistics.xml
.idea/**/dictionaries
.idea/**/shelf

# Generated files
.idea/**/contentModel.xml

# Sensitive or high-churn files
.idea/**/dataSources/
.idea/**/dataSources.ids
.idea/**/dataSources.local.xml
.idea/**/sqlDataSources.xml
.idea/**/dynamic.xml
.idea/**/uiDesigner.xml
.idea/**/dbnavigator.xml

# Gradle
.idea/**/gradle.xml
.idea/**/libraries

# Gradle and Maven with auto-import
# When using Gradle or Maven with auto-import, you should exclude module files,
# since they will be recreated, and may cause churn.  Uncomment if using
# auto-import.
# .idea/artifacts
# .idea/compiler.xml
# .idea/jarRepositories.xml
# .idea/modules.xml
# .idea/*.iml
# .idea/modules
# *.iml
# *.ipr

# CMake
cmake-build-*/

# Mongo Explorer plugin
.idea/**/mongoSettings.xml

# File-based project format
*.iws

# IntelliJ
out/

# mpeltonen/sbt-idea plugin
.idea_modules/

# JIRA plugin
atlassian-ide-plugin.xml

# Cursive Clojure plugin
.idea/replstate.xml

# Crashlytics plugin (for Android Studio and IntelliJ)
com_crashlytics_export_strings.xml
crashlytics.properties
crashlytics-build.properties
fabric.properties

# Editor-based Rest Client
.idea/httpRequests

# Android studio 3.1+ serialized cache file
.idea/caches/build_file_checksums.ser

.idea

# logs
log/result

# model
model/
models/

*.yaml

# scripts
scripts/

================================================
FILE: Blog/Hand-on-RAG.md
================================================
# TinyRAG

接下来我会带领大家一步一步的实现一个简单的RAG模型，这个模型是基于RAG的一个简化版本，我们称之为Tiny-RAG。Tiny-RAG是一个基于RAG的简化版本，它只包含了RAG的核心功能，即Retrieval和Generation。Tiny-RAG的目的是为了帮助大家更好的理解RAG模型的原理和实现。

OK，让我们开始吧！

## 1. RAG 介绍

LLM会产生误导性的 “幻觉”，依赖的信息可能过时，处理特定知识时效率不高，缺乏专业领域的深度洞察，同时在推理能力上也有所欠缺。

正是在这样的背景下，检索增强生成技术（Retrieval-Augmented Generation，RAG）应时而生，成为 AI 时代的一大趋势。

RAG 通过在语言模型生成答案之前，先从广泛的文档数据库中检索相关信息，然后利用这些信息来引导生成过程，极大地提升了内容的准确性和相关性。RAG 有效地缓解了幻觉问题，提高了知识更新的速度，并增强了内容生成的可追溯性，使得大型语言模型在实际应用中变得更加实用和可信。

RAG的基本结构有哪些呢？

- 要有一个向量化模块，用来将文档片段向量化。
- 要有一个文档加载和切分的模块，用来加载文档并切分成文档片段。
- 要有一个数据库来存放文档片段和对应的向量表示。
- 要有一个检索模块，用来根据 Query （问题）检索相关的文档片段。
- 要有一个大模型模块，用来根据检索出来的文档回答用户的问题。

OK，那上述这些也就是 TinyRAG 仓库的所有模块内容。

![](../images/Retrieval-Augmented%20Generation（RAG-Learning）.png)

那接下来，让我们梳理一下 RAG 的流程是什么样的呢？

- **索引**：将文档库分割成较短的 Chunk，并通过编码器构建向量索引。
- **检索**：根据问题和 chunks 的相似度检索相关文档片段。
- **生成**：以检索到的上下文为条件，生成问题的回答。

那也就是下图所示的流程，图片出处 ***[Retrieval-Augmented Generation for Large Language Models: A Survey](https://arxiv.org/pdf/2312.10997.pdf)***

![](../images/RAG.png)

## 2. 向量化

首先让我们来动手实现一个向量化的类，这是RAG架构的基础。向量化的类主要是用来将文档片段向量化，将一段文本映射为一个向量。

那首先我们要设置一个 `Embedding` 基类，这样我们再用其他的模型的时候，只需要继承这个基类，然后在此基础上进行修改即可，方便代码扩展。

```python
class BaseEmbeddings:
    """
    Base class for embeddings
    """
    def __init__(self, path: str, is_api: bool) -> None:
        self.path = path
        self.is_api = is_api
    
    def get_embedding(self, text: str, model: str) -> List[float]:
        raise NotImplementedError
    
    @classmethod
    def cosine_similarity(cls, vector1: List[float], vector2: List[float]) -> float:
        """
        calculate cosine similarity between two vectors
        """
        dot_product = np.dot(vector1, vector2)
        magnitude = np.linalg.norm(vector1) * np.linalg.norm(vector2)
        if not magnitude:
            return 0
        return dot_product / magnitude
```

观察一下`BaseEmbeddings`基类都有什么方法，首先有一个`get_embedding`方法，这个方法是用来获取文本的向量表示的，然后有一个`cosine_similarity`方法，这个方法是用来计算两个向量之间的余弦相似度的。其次在初始化类的时候设置了，模型的路径和是否是API模型。比如使用OpenAI的Embedding API的话就需要设置`self.is_api=Ture`。

继承`BaseEmbeddings`类的话，就只需要编写`get_embedding`方法即可，`cosine_similarity`方法会被继承下来，直接用就行。这就是编写基类的好处。

```python
class OpenAIEmbedding(BaseEmbeddings):
    """
    class for OpenAI embeddings
    """
    def __init__(self, path: str = '', is_api: bool = True) -> None:
        super().__init__(path, is_api)
        if self.is_api:
            from openai import OpenAI
            self.client = OpenAI()
            self.client.api_key = os.getenv("OPENAI_API_KEY")
            self.client.base_url = os.getenv("OPENAI_BASE_URL")
    
    def get_embedding(self, text: str, model: str = "text-embedding-3-large") -> List[float]:
        if self.is_api:
            text = text.replace("\n", " ")
            return self.client.embeddings.create(input=[text], model=model).data[0].embedding
        else:
            raise NotImplementedError
```

## 3. 文档加载和切分

接下来我们来实现一个文档加载和切分的类，这个类主要是用来加载文档并切分成文档片段。

那我们都需要切分什么文档呢？这个文档可以是一篇文章，一本书，一段对话，一段代码等等。这个文档的内容可以是任何的，只要是文本就行。比如：pdf文件、md文件、txt文件等等。

这里只展示一部分内容了，完整的代码可以在 ***[RAG/utils.py](../RAG/utils.py)*** 文件中找到。在这个代码中可以看到，能加载的文件类型有：pdf、md、txt，只需要编写对应的函数即可。

```python
def read_file_content(cls, file_path: str):
    # 根据文件扩展名选择读取方法
    if file_path.endswith('.pdf'):
        return cls.read_pdf(file_path)
    elif file_path.endswith('.md'):
        return cls.read_markdown(file_path)
    elif file_path.endswith('.txt'):
        return cls.read_text(file_path)
    else:
        raise ValueError("Unsupported file type")
```

那我们把文件内容都读取之后，还需要切分呀！那怎么切分呢，OK，接下来咱们就按 Token 的长度来切分文档。我们可以设置一个最大的 Token 长度，然后根据这个最大的 Token 长度来切分文档。这样切分出来的文档片段就是一个一个的差不多相同长度的文档片段了。

不过在切分的时候要注意，片段与片段之间最好要有一些重叠的内容，这样才能保证检索的时候能够检索到相关的文档片段。还有就是切分文档的时候最好以句子为单位，也就是按 `\n` 进行粗切分，这样可以基本保证句子内容是完整的。

```python
def get_chunk(cls, text: str, max_token_len: int = 600, cover_content: int = 150):
    chunk_text = []

    curr_len = 0
    curr_chunk = ''

    lines = text.split('\n')  # 假设以换行符分割文本为行

    for line in lines:
        line = line.replace(' ', '')
        line_len = len(enc.encode(line))
        if line_len > max_token_len:
            print('warning line_len = ', line_len)
        if curr_len + line_len <= max_token_len:
            curr_chunk += line
            curr_chunk += '\n'
            curr_len += line_len
            curr_len += 1
        else:
            chunk_text.append(curr_chunk)
            curr_chunk = curr_chunk[-cover_content:]+line
            curr_len = line_len + cover_content

    if curr_chunk:
        chunk_text.append(curr_chunk)

    return chunk_text
```

## 4. 数据库 && 向量检索

上面，我们做好了文档切分，也做好了 Embedding 模型的加载。那接下来就得设计一个向量数据库用来存放文档片段和对应的向量表示了。

还有就是也要设计一个检索模块，用来根据 Query （问题）检索相关的文档片段。OK，我们冲冲冲！

一个数据库对于最小RAG架构来说，需要实现几个功能呢？

- `persist`：数据库持久化，本地保存
- `load_vector`：从本地加载数据库
- `get_vector`：获得文档的向量表示
- `query`：根据问题检索相关的文档片段

嗯嗯，以上四个模块就是一个最小的RAG结构数据库需要实现的功能，具体代码可以在 ***[RAG/VectorBase.py](../RAG/VectorBase.py)*** 文件中找到。

```python
class VectorStore:
    def __init__(self, document: List[str] = ['']) -> None:
        self.document = document

    def get_vector(self, EmbeddingModel: BaseEmbeddings) -> List[List[float]]:
        # 获得文档的向量表示
        pass

    def persist(self, path: str = 'storage'):
        # 数据库持久化，本地保存
        pass

    def load_vector(self, path: str = 'storage'):
        # 从本地加载数据库
        pass

    def query(self, query: str, EmbeddingModel: BaseEmbeddings, k: int = 1) -> List[str]:
        # 根据问题检索相关的文档片段
        pass
```

那让我们来看一下， `query` 方法具体是怎么实现的呢？

首先先把用户提出的问题向量化，然后去数据库中检索相关的文档片段，最后返回检索到的文档片段。可以看到咱们在向量检索的时候仅使用 `Numpy` 进行加速，代码非常容易理解和修改。

主要是方便改写和大家理解，并没有使用成熟的数据库，这样可以更好的理解RAG的原理。

```python
def query(self, query: str, EmbeddingModel: BaseEmbeddings, k: int = 1) -> List[str]:
    query_vector = EmbeddingModel.get_embedding(query)
    result = np.array([self.get_similarity(query_vector, vector)
                        for vector in self.vectors])
    return np.array(self.document)[result.argsort()[-k:][::-1]].tolist()
```


## 5. 大模型模块

那就来到了最后一个模块了，大模型模块。这个模块主要是用来根据检索出来的文档回答用户的问题。

一样的，我们还是先实现一个基类，这样我们在遇到其他的自己感兴趣的模型就可以快速的扩展了。

```python
class BaseModel:
    def __init__(self, path: str = '') -> None:
        self.path = path

    def chat(self, prompt: str, history: List[dict], content: str) -> str:
        pass

    def load_model(self):
        pass
```

`BaseModel` 包含了两个方法，`chat`和`load_model`，如果使用API模型，比如OpenAI的话，那就不需要`load_model`方法，如果你要本地化运行的话，那还是会选择使用开源模型，那就需要`load_model`方法发啦。

这里咱们以 ***[InternLM2-chat-7B](https://huggingface.co/internlm/internlm2-chat-7b)*** 模型为例，大家可以给 ***[InternLM](https://github.com/InternLM/InternLM)*** 仓库点个star哦，还有也给我点个star！

```python
class InternLMChat(BaseModel):
    def __init__(self, path: str = '') -> None:
        super().__init__(path)
        self.load_model()

    def chat(self, prompt: str, history: List = [], content: str='') -> str:
        prompt = PROMPT_TEMPLATE['InternLM_PROMPT_TEMPALTE'].format(question=prompt, context=content)
        response, history = self.model.chat(self.tokenizer, prompt, history)
        return response


    def load_model(self):
        import torch
        from transformers import AutoTokenizer, AutoModelForCausalLM
        self.tokenizer = AutoTokenizer.from_pretrained(self.path, trust_remote_code=True)
        self.model = AutoModelForCausalLM.from_pretrained(self.path, torch_dtype=torch.float16, trust_remote_code=True).cuda()
```

可以用一个字典来保存所有的prompt，这样比较好维护。

```python
PROMPT_TEMPLATE = dict(
    InternLM_PROMPT_TEMPALTE="""先对上下文进行内容总结,再使用上下文来回答用户的问题。如果你不知道答案，就说你不知道。总是使用中文回答。
        问题: {question}
        可参考的上下文：
        ···
        {context}
        ···
        如果给定的上下文无法让你做出回答，请回答数据库中没有这个内容，你不知道。
        有用的回答:"""
)
```

那这样的话，我们就可以利用InternLM2模型来做RAG啦！

## 6.  LLM Tiny-RAG Demo

那接下来，我们就来看一下Tiny-RAG的Demo吧！

```python
from RAG.VectorBase import VectorStore
from RAG.utils import ReadFiles
from RAG.LLM import OpenAIChat, InternLMChat
from RAG.Embeddings import JinaEmbedding, ZhipuEmbedding


没有保存数据库
docs = ReadFiles('./data').get_content(max_token_len=600, cover_content=150) # 获得data目录下的所有文件内容并分割
vector = VectorStore(docs)
embedding = ZhipuEmbedding() # 创建EmbeddingModel
vector.get_vector(EmbeddingModel=embedding)
vector.persist(path='storage') # 将向量和文档内容保存到storage目录下，下次再用就可以直接加载本地的数据库

question = 'git的原理是什么？'

content = vector.query(question, model='zhipu', k=1)[0]
chat = InternLMChat(path='model_path')
print(chat.chat(question, [], content))
```

当然我们也可以从本地加载已经处理好的数据库，毕竟我们在上面的数据库环节已经写过这个功能啦。

```python
from RAG.VectorBase import VectorStore
from RAG.utils import ReadFiles
from RAG.LLM import OpenAIChat, InternLMChat
from RAG.Embeddings import JinaEmbedding, ZhipuEmbedding

# 保存数据库之后
vector = VectorStore()

vector.load_vector('./storage') # 加载本地的数据库

question = 'git的原理是什么？'

embedding = ZhipuEmbedding() # 创建EmbeddingModel

content = vector.query(question, EmbeddingModel=embedding, k=1)[0]
chat = InternLMChat(path='model_path')
print(chat.chat(question, [], content))
```

## 7. 总结

经过上面的学习，你是否学会了如何搭建一个最小RAG架构呢？相信你一定学会啦，哈哈哈。

那让我们再来复习一下，一个最小RAG应该包含哪些内容叭？（此处默写！）

- 向量化模块
- 文档加载和切分模块
- 数据库
- 向量检索
- 大模型模块

okk，你已经学会了，但别忘了给我的项目点个star哦！

================================================
FILE: RAG/Embeddings.py
================================================
#!/usr/bin/env python
# -*- coding: utf-8 -*-
'''
@File    :   Embeddings.py
@Time    :   2024/02/10 21:55:39
@Author  :   不要葱姜蒜
@Version :   1.0
@Desc    :   None
'''

import os
from copy import copy
from typing import Dict, List, Optional, Tuple, Union
import numpy as np

os.environ['CURL_CA_BUNDLE'] = ''
from dotenv import load_dotenv, find_dotenv
_ = load_dotenv(find_dotenv())


class BaseEmbeddings:
    """
    Base class for embeddings
    """
    def __init__(self, path: str, is_api: bool) -> None:
        self.path = path
        self.is_api = is_api

    def get_embedding(self, text: str, model: str) -> List[float]:
        raise NotImplementedError

    @classmethod
    def cosine_similarity(cls, vector1: List[float], vector2: List[float]) -> float:
        """
        calculate cosine similarity between two vectors
        """
        dot_product = np.dot(vector1, vector2)
        magnitude = np.linalg.norm(vector1) * np.linalg.norm(vector2)
        if not magnitude:
            return 0
        return dot_product / magnitude


class OpenAIEmbedding(BaseEmbeddings):
    """
    class for OpenAI embeddings
    """
    def __init__(self, path: str = '', is_api: bool = True) -> None:
        super().__init__(path, is_api)
        if self.is_api:
            from openai import OpenAI
            self.client = OpenAI()
            self.client.api_key = os.getenv("OPENAI_API_KEY")
            self.client.base_url = os.getenv("OPENAI_BASE_URL")

    def get_embedding(self, text: str, model: str = "text-embedding-3-large") -> List[float]:
        if self.is_api:
            text = text.replace("\n", " ")
            return self.client.embeddings.create(input=[text], model=model).data[0].embedding
        else:
            raise NotImplementedError

class JinaEmbedding(BaseEmbeddings):
    """
    class for Jina embeddings
    """
    def __init__(self, path: str = 'jinaai/jina-embeddings-v2-base-zh', is_api: bool = False) -> None:
        super().__init__(path, is_api)
        self._model = self.load_model()

    def get_embedding(self, text: str) -> List[float]:
        return self._model.encode([text])[0].tolist()

    def load_model(self):
        import torch
        from transformers import AutoModel
        if torch.cuda.is_available():
            device = torch.device("cuda")
        else:
            device = torch.device("cpu")
        model = AutoModel.from_pretrained(self.path, trust_remote_code=True).to(device)
        return model

class ZhipuEmbedding(BaseEmbeddings):
    """
    class for Zhipu embeddings
    """
    def __init__(self, path: str = '', is_api: bool = True) -> None:
        super().__init__(path, is_api)
        if self.is_api:
            from zhipuai import ZhipuAI
            self.client = ZhipuAI(api_key=os.getenv("ZHIPUAI_API_KEY"))

    def get_embedding(self, text: str) -> List[float]:
        response = self.client.embeddings.create(
        model="embedding-2",
        input=text,
        )
        return response.data[0].embedding

class DashscopeEmbedding(BaseEmbeddings):
    """
    class for Dashscope embeddings
    """
    def __init__(self, path: str = '', is_api: bool = True) -> None:
        super().__init__(path, is_api)
        if self.is_api:
            import dashscope
            dashscope.api_key = os.getenv("DASHSCOPE_API_KEY")
            self.client = dashscope.TextEmbedding

    def get_embedding(self, text: str, model: str='text-embedding-v1') -> List[float]:
        response = self.client.call(
            model=model,
            input=text
        )
        return response.output['embeddings'][0]['embedding']


class BgeEmbedding(BaseEmbeddings):
    """
    class for BGE embeddings
    """

    def __init__(self, path: str = 'BAAI/bge-base-zh-v1.5', is_api: bool = False) -> None:
        super().__init__(path, is_api)
        self._model, self._tokenizer = self.load_model(path)

    def get_embedding(self, text: str) -> List[float]:
        import torch
        encoded_input = self._tokenizer([text], padding=True, truncation=True, return_tensors='pt')
        encoded_input = {k: v.to(self._model.device) for k, v in encoded_input.items()}
        with torch.no_grad():
            model_output = self._model(**encoded_input)
            sentence_embeddings = model_output[0][:, 0]
        sentence_embeddings = torch.nn.functional.normalize(sentence_embeddings, p=2, dim=1)
        return sentence_embeddings[0].tolist()

    def load_model(self, path: str):
        import torch
        from transformers import AutoModel, AutoTokenizer
        if torch.cuda.is_available():
            device = torch.device("cuda")
        else:
            device = torch.device("cpu")
        tokenizer = AutoTokenizer.from_pretrained(path)
        model = AutoModel.from_pretrained(path).to(device)
        model.eval()
        return model, tokenizer

# BGE SILICONFLOW API KEY

class BgeWithAPIEmbedding(BaseEmbeddings):
    """
    class for SILICON API embeddings
    """
    def __init__(self, path: str = '', is_api: bool = True) -> None:
        super().__init__(path, is_api)
        if self.is_api:
            from openai import OpenAI
            self.client = OpenAI()
            self.client.api_key = os.getenv("SILICONFLOW_API_KEY")
            self.client.base_url = os.getenv("SILICONFLOW_BASE_URL")

    def get_embedding(self, text: str, model: str = "BAAI/bge-m3") -> List[float]:
        if self.is_api:
            text = text.replace("\n", " ")
            return self.client.embeddings.create(input=[text], model=model).data[0].embedding
        else:
            raise NotImplementedError

================================================
FILE: RAG/LLM.py
================================================
#!/usr/bin/env python
# -*- coding: utf-8 -*-
'''
@File    :   LLM.py
@Time    :   2024/02/12 13:50:47
@Author  :   不要葱姜蒜
@Version :   1.0
@Desc    :   None
'''
import os
from typing import Dict, List, Optional, Tuple, Union

PROMPT_TEMPLATE = dict(
    RAG_PROMPT_TEMPALTE="""使用以上下文来回答用户的问题。如果你不知道答案，就说你不知道。总是使用中文回答。
        问题: {question}
        可参考的上下文：
        ···
        {context}
        ···
        如果给定的上下文无法让你做出回答，请回答数据库中没有这个内容，你不知道。
        有用的回答:""",
    InternLM_PROMPT_TEMPALTE="""先对上下文进行内容总结,再使用上下文来回答用户的问题。如果你不知道答案，就说你不知道。总是使用中文回答。
        问题: {question}
        可参考的上下文：
        ···
        {context}
        ···
        如果给定的上下文无法让你做出回答，请回答数据库中没有这个内容，你不知道。
        有用的回答:"""
)


class BaseModel:
    def __init__(self, path: str = '') -> None:
        self.path = path

    def chat(self, prompt: str, history: List[dict], content: str) -> str:
        pass

    def load_model(self):
        pass

class OpenAIChat(BaseModel):
    def __init__(self, path: str = '', model: str = "gpt-3.5-turbo-1106") -> None:
        super().__init__(path)
        self.model = model

    def chat(self, prompt: str, history: List[dict], content: str) -> str:
        from openai import OpenAI
        client = OpenAI()
        client.api_key = os.getenv("OPENAI_API_KEY")   
        client.base_url = os.getenv("OPENAI_BASE_URL")
        history.append({'role': 'user', 'content': PROMPT_TEMPLATE['RAG_PROMPT_TEMPALTE'].format(question=prompt, context=content)})
        response = client.chat.completions.create(
            model=self.model,
            messages=history,
            max_tokens=150,
            temperature=0.1
        )
        return response.choices[0].message.content

class InternLMChat(BaseModel):
    def __init__(self, path: str = '') -> None:
        super().__init__(path)
        self.load_model()

    def chat(self, prompt: str, history: List = [], content: str='') -> str:
        prompt = PROMPT_TEMPLATE['InternLM_PROMPT_TEMPALTE'].format(question=prompt, context=content)
        response, history = self.model.chat(self.tokenizer, prompt, history)
        return response


    def load_model(self):
        import torch
        from transformers import AutoTokenizer, AutoModelForCausalLM
        self.tokenizer = AutoTokenizer.from_pretrained(self.path, trust_remote_code=True)
        self.model = AutoModelForCausalLM.from_pretrained(self.path, torch_dtype=torch.float16, trust_remote_code=True).cuda()

class DashscopeChat(BaseModel):
    def __init__(self, path: str = '', model: str = "qwen-turbo") -> None:
        super().__init__(path)
        self.model = model

    def chat(self, prompt: str, history: List[Dict], content: str) -> str:
        import dashscope
        dashscope.api_key = os.getenv("DASHSCOPE_API_KEY")
        history.append({'role': 'user', 'content': PROMPT_TEMPLATE['RAG_PROMPT_TEMPALTE'].format(question=prompt, context=content)})
        response = dashscope.Generation.call(
            model=self.model,
            messages=history,
            result_format='message',
            max_tokens=150,
            temperature=0.1
        )
        return response.output.choices[0].message.content
    

class ZhipuChat(BaseModel):
    def __init__(self, path: str = '', model: str = "glm-4") -> None:
        super().__init__(path)
        from zhipuai import ZhipuAI
        self.client = ZhipuAI(api_key=os.getenv("ZHIPUAI_API_KEY"))
        self.model = model

    def chat(self, prompt: str, history: List[Dict], content: str) -> str:
        history.append({'role': 'user', 'content': PROMPT_TEMPLATE['RAG_PROMPT_TEMPALTE'].format(question=prompt, context=content)})
        response = self.client.chat.completions.create(
            model=self.model,
            messages=history,
            max_tokens=150,
            temperature=0.1
        )
        return response.choices[0].message

class SiliconflowChat(BaseModel):
    def __init__(self, path: str = '', model: str = "Qwen/Qwen2.5-7B-Instruct") -> None:
        super().__init__(path)
        self.model = model

    def chat(self, prompt: str, history: List[dict], content: str) -> str:
        from openai import OpenAI
        client = OpenAI()
        client.api_key = os.getenv("SILICONFLOW_API_KEY")   
        client.base_url = os.getenv("SILICONFLOW_BASE_URL")
        final_prompt={'role': 'user', 'content': PROMPT_TEMPLATE['RAG_PROMPT_TEMPALTE'].format(question=prompt, context=content)}
        print("---------------------input---------------------")
        print(final_prompt)
        history.append(final_prompt)
        response = client.chat.completions.create(
            model=self.model,
            messages=history,
            # max_tokens=150,
            temperature=0.1
        )
        return response.choices[0].message.content

================================================
FILE: RAG/Reranker.py
================================================
#!/usr/bin/env python
# -*- coding: utf-8 -*-
'''
@File    :   Reranker.py
@Time    :   2024/05/15 14:27:42
@Author  :   YueZhengMeng
@Version :   1.0
@Desc    :   None
'''

from typing import List
import numpy as np


class BaseReranker:
    """
    Base class for reranker
    """

    def __init__(self, path: str) -> None:
        self.path = path

    def rerank(self, text: str, content: List[str], k: int) -> List[str]:
        raise NotImplementedError


class BgeReranker(BaseReranker):
    """
    class for Bge reranker
    """

    def __init__(self, path: str = 'BAAI/bge-reranker-base') -> None:
        super().__init__(path)
        self._model, self._tokenizer = self.load_model(path)

    def rerank(self, text: str, content: List[str], k: int) -> List[str]:
        import torch
        pairs = [(text, c) for c in content]
        with torch.no_grad():
            inputs = self._tokenizer(pairs, padding=True, truncation=True, return_tensors='pt', max_length=512)
            inputs = {k: v.to(self._model.device) for k, v in inputs.items()}
            scores = self._model(**inputs, return_dict=True).logits.view(-1, ).float()
            index = np.argsort(scores.tolist())[-k:][::-1]
        return [content[i] for i in index]

    def load_model(self, path: str):
        import torch
        from transformers import AutoModelForSequenceClassification, AutoTokenizer
        if torch.cuda.is_available():
            device = torch.device("cuda")
        else:
            device = torch.device("cpu")
        tokenizer = AutoTokenizer.from_pretrained(path)
        model = AutoModelForSequenceClassification.from_pretrained(path).to(device)
        model.eval()
        return model, tokenizer


================================================
FILE: RAG/VectorBase.py
================================================
#!/usr/bin/env python
# -*- coding: utf-8 -*-
'''
@File    :   VectorBase.py
@Time    :   2024/02/12 10:11:13
@Author  :   不要葱姜蒜
@Version :   1.0
@Desc    :   None
'''

import os
from typing import Dict, List, Optional, Tuple, Union
import json
from RAG.Embeddings import BaseEmbeddings, OpenAIEmbedding, JinaEmbedding, ZhipuEmbedding
import numpy as np
from tqdm import tqdm


class VectorStore:
    def __init__(self, document: List[str] = ['']) -> None:
        self.document = document

    def get_vector(self, EmbeddingModel: BaseEmbeddings) -> List[List[float]]:
        
        self.vectors = []
        for doc in tqdm(self.document, desc="Calculating embeddings"):
            self.vectors.append(EmbeddingModel.get_embedding(doc))
        return self.vectors

    def persist(self, path: str = 'storage'):
        if not os.path.exists(path):
            os.makedirs(path)
        with open(f"{path}/doecment.json", 'w', encoding='utf-8') as f:
            json.dump(self.document, f, ensure_ascii=False)
        if self.vectors:
            with open(f"{path}/vectors.json", 'w', encoding='utf-8') as f:
                json.dump(self.vectors, f)

    def load_vector(self, path: str = 'storage'):
        with open(f"{path}/vectors.json", 'r', encoding='utf-8') as f:
            self.vectors = json.load(f)
        with open(f"{path}/doecment.json", 'r', encoding='utf-8') as f:
            self.document = json.load(f)

    def get_similarity(self, vector1: List[float], vector2: List[float]) -> float:
        return BaseEmbeddings.cosine_similarity(vector1, vector2)

    def query(self, query: str, EmbeddingModel: BaseEmbeddings, k: int = 1) -> List[str]:
        query_vector = EmbeddingModel.get_embedding(query)
        result = np.array([self.get_similarity(query_vector, vector)
                          for vector in self.vectors])
        return np.array(self.document)[result.argsort()[-k:][::-1]].tolist()


================================================
FILE: RAG/utils.py
================================================
#!/usr/bin/env python
# -*- coding: utf-8 -*-
'''
@File    :   utils.py
@Time    :   2024/02/11 09:52:26
@Author  :   不要葱姜蒜
@Version :   1.0
@Desc    :   None
'''

import os
from typing import Dict, List, Optional, Tuple, Union

import PyPDF2
import markdown
import html2text
import json
from tqdm import tqdm
import tiktoken
from bs4 import BeautifulSoup
import re

enc = tiktoken.get_encoding("cl100k_base")


class ReadFiles:
    """
    class to read files
    """

    def __init__(self, path: str) -> None:
        self._path = path
        self.file_list = self.get_files()

    def get_files(self):
        # args：dir_path，目标文件夹路径
        file_list = []
        for filepath, dirnames, filenames in os.walk(self._path):
            # os.walk 函数将递归遍历指定文件夹
            for filename in filenames:
                # 通过后缀名判断文件类型是否满足要求
                if filename.endswith(".md"):
                    # 如果满足要求，将其绝对路径加入到结果列表
                    file_list.append(os.path.join(filepath, filename))
                elif filename.endswith(".txt"):
                    file_list.append(os.path.join(filepath, filename))
                elif filename.endswith(".pdf"):
                    file_list.append(os.path.join(filepath, filename))
        return file_list

    def get_content(self, max_token_len: int = 600, cover_content: int = 150):
        docs = []
        # 读取文件内容
        for file in self.file_list:
            content = self.read_file_content(file)
            chunk_content = self.get_chunk(
                content, max_token_len=max_token_len, cover_content=cover_content)
            docs.extend(chunk_content)
        return docs

    @classmethod
    def get_chunk(cls, text: str, max_token_len: int = 600, cover_content: int = 150):
        chunk_text = []

        curr_len = 0
        curr_chunk = ''

        token_len = max_token_len - cover_content
        lines = text.splitlines()  # 假设以换行符分割文本为行

        for line in lines:
            line = line.replace(' ', '')
            line_len = len(enc.encode(line))
            if line_len > max_token_len:
                # 如果单行长度就超过限制，则将其分割成多个块
                num_chunks = (line_len + token_len - 1) // token_len
                for i in range(num_chunks):
                    start = i * token_len
                    end = start + token_len
                    # 避免跨单词分割
                    while not line[start:end].rstrip().isspace():
                        start += 1
                        end += 1
                        if start >= line_len:
                            break
                    curr_chunk = curr_chunk[-cover_content:] + line[start:end]
                    chunk_text.append(curr_chunk)
                # 处理最后一个块
                start = (num_chunks - 1) * token_len
                curr_chunk = curr_chunk[-cover_content:] + line[start:end]
                chunk_text.append(curr_chunk)
                
            if curr_len + line_len <= token_len:
                curr_chunk += line
                curr_chunk += '\n'
                curr_len += line_len
                curr_len += 1
            else:
                chunk_text.append(curr_chunk)
                curr_chunk = curr_chunk[-cover_content:]+line
                curr_len = line_len + cover_content

        if curr_chunk:
            chunk_text.append(curr_chunk)

        return chunk_text

    @classmethod
    def read_file_content(cls, file_path: str):
        # 根据文件扩展名选择读取方法
        if file_path.endswith('.pdf'):
            return cls.read_pdf(file_path)
        elif file_path.endswith('.md'):
            return cls.read_markdown(file_path)
        elif file_path.endswith('.txt'):
            return cls.read_text(file_path)
        else:
            raise ValueError("Unsupported file type")

    @classmethod
    def read_pdf(cls, file_path: str):
        # 读取PDF文件
        with open(file_path, 'rb') as file:
            reader = PyPDF2.PdfReader(file)
            text = ""
            for page_num in range(len(reader.pages)):
                text += reader.pages[page_num].extract_text()
            return text

    @classmethod
    def read_markdown(cls, file_path: str):
        # 读取Markdown文件
        with open(file_path, 'r', encoding='utf-8') as file:
            md_text = file.read()
            html_text = markdown.markdown(md_text)
            # 使用BeautifulSoup从HTML中提取纯文本
            soup = BeautifulSoup(html_text, 'html.parser')
            plain_text = soup.get_text()
            # 使用正则表达式移除网址链接
            text = re.sub(r'http\S+', '', plain_text) 
            return text

    @classmethod
    def read_text(cls, file_path: str):
        # 读取文本文件
        with open(file_path, 'r', encoding='utf-8') as file:
            return file.read()


class Documents:
    """
        获取已分好类的json格式文档
    """
    def __init__(self, path: str = '') -> None:
        self.path = path
    
    def get_content(self):
        with open(self.path, mode='r', encoding='utf-8') as f:
            content = json.load(f)
        return content


================================================
FILE: README.md
================================================
> ***注意：本项目以集成到 [Happy-LLM](https://github.com/datawhalechina/happy-llm) 开源项目 Chapter7.2，本项目的后续更新将在 Happy-LLM 项目中进行。***

# Hands on TinyRAG

## 什么是RAG？

LLM会产生误导性的 “幻觉”，依赖的信息可能过时，处理特定知识时效率不高，缺乏专业领域的深度洞察，同时在推理能力上也有所欠缺。

正是在这样的背景下，检索增强生成技术（Retrieval-Augmented Generation，RAG）应时而生，成为 AI 时代的一大趋势。

RAG 通过在语言模型生成答案之前，先从广泛的文档数据库中检索相关信息，然后利用这些信息来引导生成过程，极大地提升了内容的准确性和相关性。RAG 有效地缓解了幻觉问题，提高了知识更新的速度，并增强了内容生成的可追溯性，使得大型语言模型在实际应用中变得更加实用和可信。

此仓库用于学习大模型RAG的相关内容，目前为手搓实现，主要是llama-index和langchain不太好魔改。此仓库可以方便看论文的时候，实现一些小的实验。以下为本仓库的RAG整体框架图。

![alt text](images/Retrieval-Augmented%20Generation（RAG-Learning）.png)

以下为笔者所构思的RAG实现过程，这里面主要包括包括三个基本步骤：

1. 索引 — 将文档库分割成较短的 Chunk，并通过编码器构建向量索引。

2. 检索 — 根据问题和 chunks 的相似度检索相关文档片段。

3. 生成 — 以检索到的上下文为条件，生成问题的回答。

<div align="center">
    <img src="images/RAG.png" alt="RAG" width="100%">
</div>

# QuickStart

安装依赖，需要 Python 3.10 以上版本。

```bash
pip install -r requirements.txt
```

导入所使用的包

```python
from RAG.VectorBase import VectorStore
from RAG.utils import ReadFiles
from RAG.LLM import OpenAIChat
```

如果没有数据库那就按照如下代码：

> 可以使用`VectorStore.persist()`保存到向量数据库。

```python
# 没有保存数据库
docs = ReadFiles('./data').get_content(max_token_len=600, cover_content=150) # 获得data目录下的所有文件内容并分割
embedding = JinaEmbedding("your model path") # 创建EmbeddingModel
vector = VectorStore(docs)
vector.get_vector(EmbeddingModel=embedding)
vector.persist(path='storage') # 将向量和文档内容保存到storage目录下，下次再用就可以直接加载本地的数据库

question = 'git的分支原理？'

content = vector.query(question, EmbeddingModel=embedding, k=1)[0]
chat = OpenAIChat(model='gpt-3.5-turbo-1106')
print(chat.chat(question, [], content))
```

如果有数据库那就按照如下代码：

```python
vector = VectorStore()

vector.load_vector('./storage') # 加载本地的数据库

embedding = JinaEmbedding("your model path")

question = 'git的分支原理？'

content = vector.query(question, EmbeddingModel=embedding, k=1)[0]

chat = OpenAIChat(model='gpt-3.5-turbo-1106')
print(chat.chat(question, [], content))
```

> 如果大家的文档有中文的话，不建议使用`openai`的向量接口，可以使用智谱AI或者Jina的向量模型或接口

# 实现细节

## 向量化

在这一部分共使用了三种向量化的方法，分别是`zhipu`、`jina`和`openai`。大家可以在`Embedding`文中找到实现的方式。

如果你有兴趣想使用其他的向量模型可以继承`BaseEmbeddings`类，然后实现`get_embedding`方法。

```python
class BaseEmbeddings:
    """
    Base class for embeddings
    """
    def __init__(self, path: str, is_api: bool) -> None:
        self.path = path
        self.is_api = is_api
    
    def get_embedding(self, text: str, model: str) -> List[float]:
        raise NotImplementedError
    
    @classmethod
    def cosine_similarity(cls, vector1: List[float], vector2: List[float]) -> float:
        """
        calculate cosine similarity between two vectors
        """
        dot_product = np.dot(vector1, vector2)
        magnitude = np.linalg.norm(vector1) * np.linalg.norm(vector2)
        if not magnitude:
            return 0
        return dot_product / magnitude
```

## 向量检索

这里未使用任何成熟的数据库，只是简单的使用`Json`保存了文档分割后的片段和对应的向量。大家可以在`VectorBase`中找到实现的方式。

在向量检索的时候仅使用`Numpy`进行加速，代码非常容易理解和修改。

```python
def query(self, query: str, EmbeddingModel: BaseEmbeddings, k: int = 1) -> List[str]:
    query_vector = EmbeddingModel.get_embedding(query)
    result = np.array([self.get_similarity(query_vector, vector)
                        for vector in self.vectors])
    return np.array(self.document)[result.argsort()[-k:][::-1]]
```

> 没有考虑生产环境使用，仅供学习使用

## LLM 模型

这里支持了`openai`模型和`InternLM2`模型，如果想要用其他的模型，大家可以在`LLM`中找到实现的方式。继承以下基类，然后在此基础上进行修改即可。

```python
class BaseModel:
    def __init__(self, path: str = '') -> None:
        self.path = path

    def chat(self, prompt: str, history: List[dict], content: str) -> str:
        pass

    def load_model(self):
        pass
```
 

# 参考文献

| Name                                                         | Paper Link                                |
| ------------------------------------------------------------ | ----------------------------------------- |
| When Large Language Models Meet Vector Databases: A Survey   | [paper](http://arxiv.org/abs/2402.01763)  |
| Retrieval-Augmented Generation for Large Language Models: A Survey | [paper](https://arxiv.org/abs/2312.10997) |
| Learning to Filter Context for Retrieval-Augmented Generation | [paper](http://arxiv.org/abs/2311.08377)  |
| In-Context Retrieval-Augmented Language Models               | [paper](https://arxiv.org/abs/2302.00083) |


================================================
FILE: data/2024-01-23_10-07-23/Git教程.md
================================================
# Git教程


# 基础篇

## Git简介

Git 是一个开源的分布式版本控制系统，用于敏捷高效地处理任何或小或大的项目。Git 由 Linus Torvalds 创建，最初目的是为更好地管理 Linux 内核开发而设计。它允许多个开发者在同一个项目上工作，而不必担心彼此的工作可能会发生冲突。

## 安装Git

在不同的操作系统上安装 Git 的步骤略有不同：

- 在 Windows 上，可以从 [Git 官网](https://git-scm.com/) 下载安装程序并运行。
- 在 Mac OS X 上，可以通过 Homebrew 安装 Git：

  ```bash
  brew install git
  ```

- 在 Linux 上，可以使用包管理器安装 Git，例如在 Ubuntu 上：

  ```bash
  sudo apt-get update
  sudo apt-get install git
  ```

## Git基本配置

配置用户信息是使用 Git 的第一步。在提交时，这些信息会记录在提交历史中。

```bash
git config --global user.name "你的名字"
git config --global user.email "你的邮箱"
```

还可以检查所有配置信息：

```bash
git config --list
```

## 创建仓库

要初始化一个新的 Git 仓库，使用 `git init` 命令：

```bash
git init
```

这将创建一个名为 `.git` 的子目录，其中包含所有的仓库元数据。

## 版本控制流程

Git 的版本控制流程通常包括：工作区的更改、暂存更改以及提交更改到仓库。

1. 查看文件状态：

   ```bash
   git status
   ```

2. 将文件添加到暂存区：

   ```bash
   git add <文件名>
   ```

3. 提交更改：

   ```bash
   git commit -m "提交信息"
   ```

## 文件状态与工作区

Git 中的文件有三种状态：已提交（committed）、已修改（modified）和已暂存（staged）。一个工作区包含了实际的文件，而暂存区是一个准备好下次提交的文件列表。

## 提交更改

提交是 Git 中的基本操作，它会将暂存区的更改记录到仓库中。每次提交都会在仓库中创建一个快照，并允许之后恢复到该状态。

```bash
git commit -m "描述性的提交信息"
```

## 查看提交历史

要查看提交历史，可以使用 `git log` 命令：

```bash
git log
```

还可以使用多种选项来定制显示的日志输出。

## 撤销操作

如果需要撤销操作，Git 提供了几个命令：

- 撤销工作区的修改：

  ```bash
  git checkout -- <文件名>
  ```

- 撤销暂存区的文件：

  ```bash
  git reset HEAD <文件名>
  ```

- 撤销提交（创建一个新的提交来撤销之前的提交）：

  ```bash
  git revert <提交ID>
  ```

## 标签管理

标签是指向特定提交的引用，通常用于版本发布。创建一个新标签：

```bash
git tag <标签名>
```

列出所有标签：

```bash
git tag
```

删除标签：

```bash
git tag -d <标签名>
```

查看标签信息：

```bash
git show <标签名>
```

推送标签到远程仓库：

```bash
git push origin <标签名>
```


# 分支管理

## 分支的概念

在Git中，分支是用来隔离开发工作的。每个分支都是一个独立的开发环境，互不影响。分支可以很方便地被创建和合并，因此许多开发者使用分支来进行特性开发、修复bug或者尝试新想法。

Git的一个核心概念是几乎所有操作都是本地执行的，分支也不例外。这意味着你在本地创建或切换分支，不需要与远程仓库进行通信。

## 创建与合并分支

在Git中创建新分支可以使用`git branch`命令，合并分支则使用`git merge`命令。

```bash
# 创建新分支
git branch <branch-name>

# 切换到新分支
git checkout <branch-name>

# 创建新分支并立即切换到该分支
git checkout -b <branch-name>

# 合并指定分支到当前分支
git merge <branch-name>
```

## 分支策略

合理的分支策略可以帮助团队更有效地协作。一种常见的策略是Git Flow，它定义了一个围绕项目发布的分支模型，包括功能分支、发布分支、维护分支等。

另一种策略是GitHub Flow，它更加简单灵活，适合持续交付的项目。在GitHub Flow中，`master`分支通常是稳定的，并且随时可以部署。所有新的开发都在基于`master`的特性分支上进行，一旦完成就可以合并回`master`。

## 解决冲突

当两个分支修改了同一部分代码并尝试合并时，就可能会出现冲突。Git无法自动合并这些更改，需要人工介入解决。

解决冲突的过程通常包括以下步骤：

1. 运行`git merge`，Git会提示冲突发生。
2. 打开冲突文件，查找所有标记为冲突的地方，这些地方会被特殊的标记符号包围。
3. 编辑文件，解决所有冲突。
4. 使用`git add`命令标记冲突已解决。
5. 完成合并操作，如果需要，可以通过`git commit`命令提交更改。

```bash
# 解决冲突后，添加文件标记冲突已解决
git add <file-name>

# 提交解决冲突后的合并
git commit -m "Resolve merge conflict"
```

## 分支合并策略

Git提供了不同的分支合并策略，例如默认的`merge`策略，它会创建一个新的合并提交来合并两个分支的历史。

另一种策略是`rebase`，它会将一个分支的提交重新应用到另一个分支上。这样可以创建一个更线性的提交历史。

```bash
# 使用rebase合并分支
git checkout <feature-branch>
git rebase <base-branch>

# 如果在rebase过程中遇到冲突，解决冲突后
git add <file-name>
git rebase --continue

# 完成rebase后，切换回基础分支并合并特性分支
git checkout <base-branch>
git merge <feature-branch>
```

使用`rebase`的好处是可以避免不必要的合并提交，但它会改变历史，因此在共享的分支上使用时需要谨慎。


# 远程仓库

## 远程仓库的作用

远程仓库是位于互联网或其他网络中的服务器上的 Git 仓库。它可以让多个开发者共享一个项目，而不必将所有的文件和版本历史存储在本地计算机上。远程仓库的主要作用包括：

- **版本控制**：帮助团队成员之间同步和管理代码变更。
- **备份**：防止本地数据丢失后可以从远程仓库恢复。
- **协作**：多人可以同时工作在同一个项目上，提高开发效率。
- **代码审查**：通过 Pull Requests (PRs) 等机制可以进行代码审查。

## 添加远程仓库

要添加新的远程仓库，可以使用 `git remote add` 命令。该命令需要两个参数：远程仓库的名称和远程仓库的 URL。

```bash
git remote add <remote-name> <remote-url>
```

例如，添加一个名为 `origin` 的远程仓库：

```bash
git remote add origin https://github.com/username/repository.git
```

## 推送到远程仓库

将本地的更改推送到远程仓库，可以使用 `git push` 命令。通常，这个命令后面会跟远程仓库的名称和要推送的分支名称。

```bash
git push <remote-name> <branch-name>
```

例如，将本地的 `master` 分支推送到 `origin` 远程仓库：

```bash
git push origin master
```

## 从远程仓库拉取

从远程仓库获取最新的更改并合并到本地分支，可以使用 `git pull` 命令。这个命令会将远程仓库的指定分支的更改拉取到当前分支。

```bash
git pull <remote-name> <branch-name>
```

例如，从 `origin` 远程仓库的 `master` 分支拉取最新更改：

```bash
git pull origin master
```

## 远程分支管理

查看远程分支，可以使用 `git branch` 命令加上 `-r` 选项。

```bash
git branch -r
```

删除远程分支，可以使用 `git push` 命令加上 `--delete` 选项。

```bash
git push <remote-name> --delete <branch-name>
```

例如，删除 `origin` 远程仓库的 `feature` 分支：

```bash
git push origin --delete feature
```

## 远程仓库的协作与贡献

协作和贡献通常涉及以下步骤：

1. **Fork** 远程仓库。
2. **Clone** Fork 后的仓库到本地。
3. 创建新的**分支**进行开发。
4. 完成开发后，将分支**推送**到自己的 Fork 仓库。
5. 在原仓库发起 **Pull Request** (PR)。
6. 维护者**审查代码**，并将其**合并**到主仓库。

例如，将本地分支 `feature` 推送到自己 Fork 的远程仓库：

```bash
git push origin feature
```

之后，在 GitHub 或其他托管服务上发起 Pull Request。


# 高级篇

## 变基

变基（Rebase）是Git中用于整理提交历史的一种工具。它的主要作用是将一系列的提交按照原有顺序复制到另一个基底上。

### 基本原理

当你进行变基操作时，Git会找到这些提交和目标基底（即你想要变基到的提交）的最近共同祖先，然后将每个提交从这个共同祖先开始重新应用。这样，你的提交历史就会看起来像是从目标基底直接分支出来的。

### 使用变基

```bash
# 将当前分支变基到指定的<base>
git rebase <base>
```

## 暂存区管理

暂存区（Staging Area）是Git中的一个概念，它是一个准备提交的更改列表。

### 基本原理

当你执行`git add`命令时，更改就会被添加到暂存区。然后你可以使用`git commit`将这些更改提交到仓库。

### 操作暂存区

```bash
# 添加文件到暂存区
git add <file>

# 查看暂存区状态
git status

# 取消暂存
git reset HEAD <file>
```

## Git钩子

Git钩子（Hooks）是在Git执行特定事件（如提交和合并）时触发的脚本。

### 基本原理

Git钩子存放在仓库的`.git/hooks`目录下。当触发相应的事件时，Git会执行这个目录下的脚本。

### 使用Git钩子

```bash
# 编辑钩子脚本
vim .git/hooks/<hook-name>

# 使钩子脚本可执行
chmod +x .git/hooks/<hook-name>
```

## 子模块

子模块（Submodules）允许你将一个Git仓库作为另一个Git仓库的子目录。

### 基本原理

使用子模块可以帮助你管理项目中的第三方代码库。

### 使用子模块

```bash
# 添加子模块
git submodule add <repository> <path>

# 初始化子模块
git submodule init

# 更新子模块
git submodule update
```

## Git LFS（大文件存储）

Git LFS（Large File Storage）是一个Git扩展，用于改善大文件的存储和访问。

### 基本原理

Git LFS 通过将大文件的内容替换为指针，实际内容存储在服务器上，从而避免了大文件在仓库中的直接存储。

### 使用Git LFS

```bash
# 安装Git LFS
git lfs install

# 跟踪大文件
git lfs track "*.psd"

# 提交更改
git add .gitattributes
git commit -m "Track .psd files using Git LFS"
```


# 实用技巧

## 撤销与重做

在使用Git时，我们有时会需要撤销之前的操作或者重做操作。以下是一些常用的撤销与重做操作命令：

### 撤销工作目录中的修改

如果你对文件进行了修改，但是还没有进行提交，你可以使用以下命令来撤销工作目录中的修改：

```bash
git checkout -- <file>
```

### 撤销暂存区的文件

如果你已经使用`git add`将文件添加到暂存区，但是想要撤销这一操作，可以使用以下命令：

```bash
git reset HEAD <file>
```

### 撤销提交

如果你已经进行了提交，但是想要撤销这次提交，可以使用以下命令：

```bash
git revert <commit>
```

其中`<commit>`是你想要撤销的提交的哈希值。

### 重做提交

如果你撤销了某次提交，但是后来又决定这次提交是正确的，想要重新应用这次提交，可以使用以下命令：

```bash
git reset --hard <commit>
```

这将会重置当前分支到指定的提交。

## 日志搜索技巧

Git提供了强大的日志搜索功能，可以帮助我们快速找到特定的提交信息。

### 搜索提交日志

使用以下命令可以搜索提交日志：

```bash
git log --grep=<pattern>
```

其中`<pattern>`是你想要搜索的关键词或正则表达式。

### 查看特定文件的变更记录

如果你只对某个特定文件的变更记录感兴趣，可以使用以下命令：

```bash
git log -p <file>
```

这将显示该文件的每次提交差异。

### 查看某个范围内的提交

你也可以指定查看某个时间范围内的提交：

```bash
git log --since="2 weeks ago" --until="3 days ago"
```

这将显示从两周前到三天前的所有提交。

## 搭建Git服务器

搭建Git服务器可以让团队成员共享代码库和协作开发。以下是搭建Git服务器的基本步骤：

### 安装Git

首先确保服务器上安装了Git：

```bash
sudo apt-get install git
```

### 创建一个裸仓库

在服务器上创建一个裸仓库：

```bash
git init --bare <repository.git>
```

### 设置SSH访问

确保团队成员的SSH公钥被添加到服务器的`~/.ssh/authorized_keys`文件中，以便他们可以通过SSH访问仓库。

## Git与持续集成

Git可以与持续集成(CI)系统结合使用，以自动化代码的构建、测试和部署流程。

### 集成CI服务

你可以选择一个CI服务，如Jenkins、Travis CI或CircleCI，并按照服务提供商的指南将其与你的Git仓库集成。

### 配置构建脚本

在你的项目中添加一个构建脚本，例如`.travis.yml`或`Jenkinsfile`，并配置构建、测试和部署的命令。

### 自动触发构建

每当有新的提交推送到仓库时，CI服务会自动触发构建流程，并反馈构建结果。

## Git工作流程

Git工作流程是指团队使用Git进行协作开发的一系列规范流程。以下是一些常见的Git工作流程：

### Feature Branch Workflow

特性分支工作流程中，每个新功能都在独立的分支上开发，完成后再合并到主分支：

```bash
git checkout -b feature_branch
# 开发新功能
git commit -am "Add new feature"
git checkout master
git merge feature_branch
```

### Gitflow Workflow

Gitflow工作流程定义了一个围绕项目发布的严格分支模型，包括功能分支、发布分支和维护分支。

### Forking Workflow

在Forking工作流程中，每个贡献者都有自己的服务器端仓库，他们可以自由地推送提交，然后通过拉取请求来贡献代码。

以上是对实用技巧模块的详细介绍。


# 附录

## 常用Git命令清单

在使用Git进行版本控制时，以下是一些常用的命令：

- `git init`：在当前目录中初始化一个新的Git仓库。
- `git clone [url]`：克隆一个仓库到本地目录。
- `git add [file]`：将文件添加到暂存区。
- `git commit -m "[commit message]"`：将暂存区的内容提交到仓库。
- `git status`：查看仓库当前的状态，显示有变更的文件。
- `git push [alias] [branch]`：将本地分支的更新推送到远程仓库。
- `git pull [alias] [branch]`：从远程仓库获取最新版本并合并到本地。
- `git branch`：列出所有本地分支。
- `git branch -a`：列出所有本地分支和远程分支。
- `git branch [branch-name]`：创建新分支。
- `git checkout [branch-name]`：切换到指定分支。
- `git merge [branch]`：合并指定分支到当前分支。
- `git log`：查看提交历史。
- `git diff`：查看未暂存的文件更新了哪些部分。
- `git reset`：重置当前HEAD到指定状态。

## Git配置文件解析

Git配置文件`.gitconfig`通常位于用户主目录下，用于配置用户级别的Git选项。

```ini
[user]
    name = Your Name
    email = you@example.com
[alias]
    co = checkout
    br = branch
    ci = commit
    st = status
```

- `[user]`部分用于设置提交代码时的用户信息。
- `[alias]`部分可以设置命令的别名，简化命令输入。

## Git错误处理

处理Git错误时，首先应该使用`git status`和`git log`检查当前状态和历史提交。以下是一些常见的Git错误处理方法：

- 当遇到合并冲突时，需要手动编辑文件解决冲突，然后使用`git add [file]`标记为已解决，最后提交。
- 如果需要撤销最近的提交，可以使用`git reset --hard HEAD^`回退到上一个提交状态。
- 当本地分支落后于远程分支，需要合并远程变更时，可以使用`git pull`来更新本地分支。

## 参考资料与进阶阅读

以下是一些推荐的参考资料和进阶阅读：

- Pro Git书籍：深入理解Git的原理和使用方法。
- Git官方文档：提供详细的命令参考和使用场景。
- Git社区论坛：在社区中与其他开发者交流心得和技巧。

================================================
FILE: data/2024-01-23_10-10-05/Git教程.md
================================================
# Git教程


# Git简介

## 什么是Git

Git是一种分布式版本控制系统，用于跟踪文件的变化并协作开发。它可以记录文件的修改历史，并允许多人在同一项目上进行协作开发。

## Git的优势

- 分布式：每个开发者都拥有整个代码库的完整历史记录，无需依赖中央服务器。
- 高效：Git的分支和合并操作非常快速，使得并行开发变得更加容易。
- 强大的支持：Git支持大型项目、快速的存储和高效的处理能力。

```bash
# 创建一个新的Git仓库
git init
```

```bash
# 克隆一个现有的Git仓库
git clone <仓库地址>
```

```bash
# 将文件添加到暂存区
git add <文件名>
```

```bash
# 提交文件
git commit -m "提交说明"
```


# Git基础

## Git的安装

首先，您需要下载适用于您操作系统的Git安装程序。然后，按照安装向导的步骤进行安装。

## 创建仓库

要在Git中创建一个新的仓库，可以使用以下命令：

```bash
git init
```

这将在当前目录中创建一个新的Git仓库。

## 添加文件

要将文件添加到Git仓库，可以使用以下命令：

```bash
git add <文件名>
```

这将把指定的文件添加到暂存区。

## 提交更改

一旦您对文件进行了修改并将其添加到暂存区，您可以使用以下命令提交更改：

```bash
git commit -m "提交说明"
```

这将提交您所做的更改并将其保存到仓库中。

## 查看提交历史

要查看Git仓库的提交历史，可以使用以下命令：

```bash
git log
```

这将显示所有的提交历史记录，包括提交者信息、提交时间和提交说明。

以上就是Git基础模块的内容。


# 分支管理

## 创建分支

要创建一个新的分支，可以使用以下命令：

```bash
git branch <branch_name>
```

这将创建一个名为`<branch_name>`的新分支，但并不会自动切换到这个分支。

## 切换分支

要切换到已经存在的分支，可以使用以下命令：

```bash
git checkout <branch_name>
```

这将使当前工作目录切换到名为`<branch_name>`的分支上。

## 合并分支

要将一个分支的更改合并到当前分支，可以使用以下命令：

```bash
git merge <branch_name>
```

这将把名为`<branch_name>`的分支合并到当前分支上。

## 解决冲突

在合并分支时，如果发生冲突，需要手动解决冲突。可以通过编辑文件来解决冲突，然后使用以下命令标记文件为已解决冲突的状态：

```bash
git add <file_name>
```

解决完所有冲突后，可以继续合并分支。

以上是关于Git分支管理的基本操作。


# 远程仓库

## 添加远程仓库

要将本地仓库与远程仓库关联，可以使用以下命令：

```bash
git remote add origin 远程仓库地址
```

其中，`origin`是远程仓库的别名，可以根据实际情况自行命名。

## 推送到远程仓库

将本地提交推送到远程仓库可以使用以下命令：

```bash
git push origin 分支名
```

例如，将本地的`master`分支推送到远程仓库可以使用：

```bash
git push origin master
```

## 从远程仓库拉取

从远程仓库拉取最新代码到本地可以使用以下命令：

```bash
git pull origin 分支名
```

例如，从远程仓库拉取`master`分支的最新代码可以使用：

```bash
git pull origin master
```

## 解决冲突

在拉取或合并代码时，如果出现冲突，需要手动解决冲突并提交。可以通过编辑文件解决冲突后，再执行以下命令提交解决：

```bash
git add .
git commit -m "解决冲突"
git push origin 分支名
```

以上是关于远程仓库操作的基本内容。


### 高级主题

#### 标签

在Git中，标签是用来标记特定提交的版本号的。通过给提交打上标签，可以方便地在需要的时候找到该提交。创建标签可以使用`git tag`命令，例如：

```bash
git tag v1.0.0
```

#### 忽略文件

有时候我们希望Git在进行版本控制时忽略某些文件，比如临时文件或者编译生成的文件。可以通过在项目根目录下创建一个名为`.gitignore`的文件，并在其中指定需要忽略的文件模式，例如：

```plaintext
*.log
build/
```

#### 撤销操作

在Git中，有时候我们需要撤销之前的操作，比如撤销暂存的文件或者撤销已提交的修改。可以使用`git reset`命令来撤销暂存的文件，使用`git revert`命令来撤销已提交的修改。

#### Git工作流

Git工作流是指团队协作时如何使用Git进行版本控制和代码管理的一种约定。常见的Git工作流包括集中式工作流、特性分支工作流、Gitflow工作流等。选择合适的工作流对团队的协作效率有很大的影响。


# 实用技巧

## 查看差异

要查看文件的差异，可以使用以下命令：

```bash
git diff
```

## 修改最后一次提交

如果需要修改最后一次提交的信息，可以使用以下命令：

```bash
git commit --amend
```

## 临时存储修改

使用以下命令可以将当前工作目录中的修改临时存储起来：

```bash
git stash
```

## 重写提交历史

如果需要重写提交历史，可以使用以下命令：

```bash
git rebase -i HEAD~n
```

以上是关于Git实用技巧的详细内容。

================================================
FILE: data/2024-01-23_10-14-41/Github Api 使用.md
================================================
# Github Api 使用


### Github Api 简介

#### 什么是Github Api

Github Api 是 Github 提供的一组用于访问和操作 Github 数据的接口。通过 Github Api，用户可以实现对仓库、提交、用户等资源的管理和操作。

#### Github Api 的优势

- **灵活性**：Github Api 提供了丰富的接口，可以满足各种不同的需求，用户可以根据自身需求进行定制化开发。
- **整合性**：Github Api 可以与其他服务和工具进行无缝整合，例如持续集成工具、项目管理工具等，提高工作效率。
- **实时性**：通过 Github Api，可以实时获取最新的仓库、提交等信息，帮助用户及时了解和响应变化。

以上是 Github Api 的简要介绍和优势。


# Github Api 的基本用法

## 认证与权限

在使用 Github Api 之前，我们需要进行认证以获取相应的权限。Github Api 使用 OAuth 2.0 进行认证，我们可以通过申请一个个人访问令牌（Personal Access Token）来进行认证。

### 申请个人访问令牌

1. 登录 Github，点击头像进入 Settings。
2. 在左侧菜单中选择 Developer settings，然后点击 Personal access tokens。
3. 点击 Generate new token，填写 Token description，并勾选需要的权限。
4. 点击 Generate token，将生成的访问令牌保存好，之后将用于认证。

### 使用个人访问令牌进行认证

在进行数据获取与操作时，需要在请求的 Header 中添加 Authorization 字段，其值为 "token \<your_personal_access_token\>"。

## 数据获取与操作

Github Api 提供了丰富的接口来获取和操作数据，包括获取仓库信息、提交记录、问题等。

### 获取仓库信息

我们可以使用 Github Api 来获取特定用户或组织的仓库信息，例如：

```python
import requests

url = 'https://api.github.com/users/octocat/repos'
headers = {'Authorization': 'token <your_personal_access_token>'}
response = requests.get(url, headers=headers)

print(response.json())
```

### 提交操作

我们可以使用 Github Api 来创建、更新和删除提交，例如：

```python
import requests

url = 'https://api.github.com/repos/octocat/Hello-World/contents/test.txt'
headers = {'Authorization': 'token <your_personal_access_token>'}
data = {
  "message": "my commit message",
  "content": "bXkgbmV3IGZpbGUgY29udGVudHM="  # base64 encoded content
}
response = requests.put(url, headers=headers, json=data)

print(response.json())
```

以上就是 Github Api 的基本用法，包括认证与权限以及数据获取与操作。通过这些基本用法，我们可以在自己的应用中使用 Github Api 来实现丰富的功能。


# Github Api 的高级应用

## Webhooks

Webhooks 是 Github Api 中的一项高级功能，它允许用户在特定事件发生时自动触发自定义的 HTTP 回调。通过配置 Webhooks，您可以实现对代码仓库中的各种事件进行监控和自动化处理，例如代码提交、Issue 创建等。

### 配置 Webhooks

要配置 Webhooks，您可以通过 Github 仓库的设置页面进行操作。在 Webhooks 页面，您可以添加新的 Webhook，并指定触发 Webhook 的事件类型和回调 URL。

### Webhooks 的工作原理

当配置的事件类型在仓库中发生时，Github 会向指定的回调 URL 发送 HTTP POST 请求，请求中包含有关事件的详细信息。您可以编写自定义的服务器端代码来处理这些请求，实现自动化的业务逻辑。

### 示例代码

以下是一个简单的 Node.js Express 服务器端代码示例，用于处理 Github Webhooks 的 HTTP POST 请求：

```javascript
const express = require('express');
const bodyParser = require('body-parser');

const app = express();
app.use(bodyParser.json());

app.post('/webhook', (req, res) => {
  const event = req.get('X-GitHub-Event');
  // 根据 event 类型处理业务逻辑
  res.sendStatus(200);
});

app.listen(3000, () => {
  console.log('Webhook 服务器已启动');
});
```

## Github Actions

Github Actions 是 Github 提供的持续集成和持续部署（CI/CD）工具，它允许您在代码仓库中配置自定义的工作流程，以实现自动化的构建、测试和部署。

### 配置 Github Actions

要配置 Github Actions，您可以在仓库中创建一个名为 `.github/workflows` 的目录，并在该目录中添加 YAML 格式的工作流程配置文件。在配置文件中，您可以定义工作流程的触发条件、执行步骤和环境。

### Github Actions 的工作原理

当配置的触发条件满足时，Github 会自动执行相应的工作流程。您可以在工作流程中使用 Github 提供的预定义动作（actions），也可以编写自定义的脚本来实现特定的构建、测试和部署操作。

### 示例代码

以下是一个简单的 Github Actions 配置文件示例，用于在代码提交后自动运行测试并部署到服务器：

```yaml
name: CI/CD

on:
  push:
    branches: [ main ]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v2

    - name: Run tests
      run: npm test

    - name: Deploy to server
      run: |
        ssh user@server 'cd /path/to/app && git pull'
```

以上是 Github Api 的高级应用中 Webhooks 和 Github Actions 的详细内容。

================================================
FILE: data/2024-01-23_11-59-03/Github Api 使用.md
================================================
# Github Api 使用


 # Github API 简介

Github API 是 Github 提供的一组接口，允许用户通过编程方式与 Github 进行交互。通过这些接口，用户可以获取 Github 仓库的信息、创建和修改仓库、管理 issue 和 pull request 等。

Github API 支持多种编程语言，包括 Python、Java、Ruby 等。用户可以通过这些语言编写程序，调用 Github API 接口，实现自动化操作。

# Github API 功能

Github API 提供了丰富的功能，包括但不限于：

- 获取仓库信息
- 创建和修改仓库
- 管理 issue 和 pull request
- 获取用户信息
- 管理用户权限
- 搜索仓库和用户

以下是一个简单的示例代码，演示如何使用 Python 调用 Github API 获取仓库信息：

```python
import requests

def get_repo_info(owner, repo):
    url = f"https://api.github.com/repos/{owner}/{repo}"
    response = requests.get(url)
    if response.status_code == 200:
        return response.json()
    else:
        return None

# 调用函数获取仓库信息
repo_info = get_repo_info("octocat", "Hello-World")
if repo_info:
    print(repo_info)
else:
    print("获取仓库信息失败")
```

以上代码中，我们使用 `requests` 库发送 HTTP 请求，获取仓库信息。如果请求成功，则返回 JSON 格式的仓库信息；如果请求失败，则返回 None。

需要注意的是，在使用 Github API 时，需要先进行身份验证，可以通过在请求头中添加 `Authorization` 字段来实现。


 目录 2
======

Github Api 使用方法
------

Github Api 是一种用于


 # Github API 常见问题

在使用 Github API 时，可能会遇到一些常见问题，以下是一些可能遇到的问题及其解决方法。

## 问题 1：无法获取

================================================
FILE: data/2024-01-23_12-01-19/Github Api 使用.md
================================================
# Github Api 使用


 目录 1
======

Github Api 简介
------------

Github API 是 Github 提供的一组接口，用于通过编程方式


 ## Github Api 使用方法

Github Api 是一种基于RESTful架构的API，它允许用户通过HTTP请求与Github进行交互。以下是使用Github Api的基本步骤：

### 1. 获取

================================================
FILE: example.py
================================================
from RAG.VectorBase import VectorStore
from RAG.utils import ReadFiles
from RAG.LLM import OpenAIChat, InternLMChat
from RAG.Embeddings import JinaEmbedding, ZhipuEmbedding


# 没有保存数据库
# docs = ReadFiles('./data').get_content(max_token_len=600, cover_content=150) # 获得data目录下的所有文件内容并分割
# vector = VectorStore(docs)
# embedding = ZhipuEmbedding() # 创建EmbeddingModel
# vector.get_vector(EmbeddingModel=embedding)
# vector.persist(path='storage') # 将向量和文档内容保存到storage目录下，下次再用就可以直接加载本地的数据库

# # vector.load_vector('./storage') # 加载本地的数据库

# question = '正向扫描的原理是什么？'

# content = vector.query(question, EmbeddingModel=embedding, k=1)[0]
# chat = OpenAIChat(model='gpt-3.5-turbo-1106')
# print(chat.chat(question, [], content))


# 保存数据库之后
vector = VectorStore()

vector.load_vector('./storage') # 加载本地的数据库

question = '逆向纠错的原理是什么？'

embedding = ZhipuEmbedding() # 创建EmbeddingModel

content = vector.query(question, EmbeddingModel=embedding, k=1)[0]
chat = OpenAIChat(model='gpt-3.5-turbo-1106')
print(chat.chat(question, [], content))



================================================
FILE: example_with_reranker.py
================================================
import os
from RAG.VectorBase import VectorStore
from RAG.utils import ReadFiles
from RAG.LLM import ZhipuChat
from RAG.Embeddings import BgeEmbedding
from RAG.Reranker import BgeReranker

# 未创建向量数据库、更换数据集、或更换其他Embeddings模型后将have_created_db设置为False
# 运行一次成功创建向量数据库后可以将have_created_db设置为True
have_created_db = True
# 请将*替换为自己的API_KEY
os.environ['ZHIPUAI_API_KEY'] = "*"

# 创建EmbeddingModel
embedding = BgeEmbedding()

# 创建RerankerModel
reranker = BgeReranker()

if have_created_db:
    # 保存数据库之后
    vector = VectorStore()
    vector.load_vector('./storage')  # 加载本地的数据库
else:
    # 没有保存数据库
    docs = ReadFiles('./data').get_content(max_token_len=600, cover_content=150)  # 获得data目录下的所有文件内容并分割
    vector = VectorStore(docs)
    vector.get_vector(EmbeddingModel=embedding)
    vector.persist(path='storage')  # 将向量和文档内容保存到storage目录下，下次再用就可以直接加载本地的数据库

question = '远程仓库的协作与贡献有哪些？'

# 从向量数据库中查询出最相似的3个文档
content = vector.query(question, EmbeddingModel=embedding, k=3)
# 从一阶段查询结果中用Reranker再次筛选出最相似的2个文档
rerank_content = reranker.rerank(question, content, k=2)
# 最后选择最相似的文档, 交给LLM作为可参考上下文
best_content = rerank_content[0]
chat = ZhipuChat()
print(chat.chat(question, [], best_content))


================================================
FILE: paper.md
================================================
# RAG (Retrieved-Augmented Generation) for LLM:A Curated Collection

This Markdown is dedicated to curating high-quality papers, resources, and tools related to RAG in the context of Large Language Models (LLM). RAG bridges the gap between retrieval-based and generation-based methods, offering a promising approach for knowledge-intensive tasks.

| Name                                                         | Paper Link                                |
| ------------------------------------------------------------ | ----------------------------------------- |
| When Large Language Models Meet Vector Databases: A Survey   | [paper](http://arxiv.org/abs/2402.01763)  |
| Retrieval-Augmented Generation for Large Language Models: A Survey | [paper](https://arxiv.org/abs/2312.10997) |
| Learning to Filter Context for Retrieval-Augmented Generation | [paper](http://arxiv.org/abs/2311.08377)  |
| In-Context Retrieval-Augmented Language Models               | [paper](https://arxiv.org/abs/2302.00083) |



================================================
FILE: requirements.txt
================================================
openai
zhipuai
numpy
python-dotenv
torch
torchvision
torchaudio
transformers
tqdm
PyPDF2
markdown
html2text
tiktoken
beautifulsoup4


================================================
FILE: storage/doecment.json
================================================
["Git教程\n基础篇\nGit简介\nGit是一个开源的分布式版本控制系统，用于敏捷高效地处理任何或小或大的项目。Git由LinusTorvalds创建，最初目的是为更好地管理Linux内核开发而设计。它允许多个开发者在同一个项目上工作，而不必担心彼此的工作可能会发生冲突。\n安装Git\n在不同的操作系统上安装Git的步骤略有不同：\n\n在Windows上，可以从Git官网下载安装程序并运行。\n在MacOSX上，可以通过Homebrew安装Git：\n\nbash\nbrewinstallgit\n\n在Linux上，可以使用包管理器安装Git，例如在Ubuntu上：\n\nbash\nsudoapt-getupdate\nsudoapt-getinstallgit\nGit基本配置\n配置用户信息是使用Git的第一步。在提交时，这些信息会记录在提交历史中。\nbash\ngitconfig--globaluser.name\"你的名字\"\ngitconfig--globaluser.email\"你的邮箱\"\n还可以检查所有配置信息：\nbash\ngitconfig--list\n创建仓库\n要初始化一个新的Git仓库，使用gitinit命令：\nbash\ngitinit\n这将创建一个名为.git的子目录，其中包含所有的仓库元数据。\n版本控制流程\nGit的版本控制流程通常包括：工作区的更改、暂存更改以及提交更改到仓库。\n\n查看文件状态：\n\nbash\ngitstatus\n\n将文件添加到暂存区：\n\nbash\ngitadd<文件名>\n\n提交更改：\n\nbash\ngitcommit-m\"提交信息\"\n文件状态与工作区\n", "含所有的仓库元数据。\n版本控制流程\nGit的版本控制流程通常包括：工作区的更改、暂存更改以及提交更改到仓库。\n\n查看文件状态：\n\nbash\ngitstatus\n\n将文件添加到暂存区：\n\nbash\ngitadd<文件名>\n\n提交更改：\n\nbash\ngitcommit-m\"提交信息\"\n文件状态与工作区\nGit中的文件有三种状态：已提交（committed）、已修改（modified）和已暂存（staged）。一个工作区包含了实际的文件，而暂存区是一个准备好下次提交的文件列表。提交更改\n提交是Git中的基本操作，它会将暂存区的更改记录到仓库中。每次提交都会在仓库中创建一个快照，并允许之后恢复到该状态。\nbash\ngitcommit-m\"描述性的提交信息\"\n查看提交历史\n要查看提交历史，可以使用gitlog命令：\nbash\ngitlog\n还可以使用多种选项来定制显示的日志输出。\n撤销操作\n如果需要撤销操作，Git提供了几个命令：\n\n撤销工作区的修改：\n\nbash\ngitcheckout--<文件名>\n\n撤销暂存区的文件：\n\nbash\ngitresetHEAD<文件名>\n\n撤销提交（创建一个新的提交来撤销之前的提交）：\n\nbash\ngitrevert<提交ID>\n标签管理\n", "\n撤销操作\n如果需要撤销操作，Git提供了几个命令：\n\n撤销工作区的修改：\n\nbash\ngitcheckout--<文件名>\n\n撤销暂存区的文件：\n\nbash\ngitresetHEAD<文件名>\n\n撤销提交（创建一个新的提交来撤销之前的提交）：\n\nbash\ngitrevert<提交ID>\n标签管理\n标签是指向特定提交的引用，通常用于版本发布。创建一个新标签：bash\ngittag<标签名>\n列出所有标签：\nbash\ngittag\n删除标签：\nbash\ngittag-d<标签名>\n查看标签信息：\nbash\ngitshow<标签名>\n推送标签到远程仓库：\nbash\ngitpushorigin<标签名>\n分支管理\n分支的概念\n在Git中，分支是用来隔离开发工作的。每个分支都是一个独立的开发环境，互不影响。分支可以很方便地被创建和合并，因此许多开发者使用分支来进行特性开发、修复bug或者尝试新想法。\nGit的一个核心概念是几乎所有操作都是本地执行的，分支也不例外。这意味着你在本地创建或切换分支，不需要与远程仓库进行通信。\n创建与合并分支\n", "是用来隔离开发工作的。每个分支都是一个独立的开发环境，互不影响。分支可以很方便地被创建和合并，因此许多开发者使用分支来进行特性开发、修复bug或者尝试新想法。\nGit的一个核心概念是几乎所有操作都是本地执行的，分支也不例外。这意味着你在本地创建或切换分支，不需要与远程仓库进行通信。\n创建与合并分支\n在Git中创建新分支可以使用gitbranch命令，合并分支则使用gitmerge命令。```bash\n创建新分支\ngitbranch\n切换到新分支\ngitcheckout\n创建新分支并立即切换到该分支\ngitcheckout-b\n合并指定分支到当前分支\ngitmerge\n```\n分支策略\n合理的分支策略可以帮助团队更有效地协作。一种常见的策略是GitFlow，它定义了一个围绕项目发布的分支模型，包括功能分支、发布分支、维护分支等。\n另一种策略是GitHubFlow，它更加简单灵活，适合持续交付的项目。在GitHubFlow中，master分支通常是稳定的，并且随时可以部署。所有新的开发都在基于master的特性分支上进行，一旦完成就可以合并回master。\n解决冲突\n", "绕项目发布的分支模型，包括功能分支、发布分支、维护分支等。\n另一种策略是GitHubFlow，它更加简单灵活，适合持续交付的项目。在GitHubFlow中，master分支通常是稳定的，并且随时可以部署。所有新的开发都在基于master的特性分支上进行，一旦完成就可以合并回master。\n解决冲突\n当两个分支修改了同一部分代码并尝试合并时，就可能会出现冲突。Git无法自动合并这些更改，需要人工介入解决。解决冲突的过程通常包括以下步骤：\n\n运行gitmerge，Git会提示冲突发生。\n打开冲突文件，查找所有标记为冲突的地方，这些地方会被特殊的标记符号包围。\n编辑文件，解决所有冲突。\n使用gitadd命令标记冲突已解决。\n完成合并操作，如果需要，可以通过gitcommit命令提交更改。\n\n```bash\n解决冲突后，添加文件标记冲突已解决\ngitadd\n提交解决冲突后的合并\ngitcommit-m\"Resolvemergeconflict\"\n```\n分支合并策略\nGit提供了不同的分支合并策略，例如默认的merge策略，它会创建一个新的合并提交来合并两个分支的历史。\n", "令提交更改。\n\n```bash\n解决冲突后，添加文件标记冲突已解决\ngitadd\n提交解决冲突后的合并\ngitcommit-m\"Resolvemergeconflict\"\n```\n分支合并策略\nGit提供了不同的分支合并策略，例如默认的merge策略，它会创建一个新的合并提交来合并两个分支的历史。\n另一种策略是rebase，它会将一个分支的提交重新应用到另一个分支上。这样可以创建一个更线性的提交历史。```bash\n使用rebase合并分支\ngitcheckout\ngitrebase\n如果在rebase过程中遇到冲突，解决冲突后\ngitadd\ngitrebase--continue\n完成rebase后，切换回基础分支并合并特性分支\ngitcheckout\ngitmerge\n```\n使用rebase的好处是可以避免不必要的合并提交，但它会改变历史，因此在共享的分支上使用时需要谨慎。\n远程仓库\n远程仓库的作用\n远程仓库是位于互联网或其他网络中的服务器上的Git仓库。它可以让多个开发者共享一个项目，而不必将所有的文件和版本历史存储在本地计算机上。远程仓库的主要作用包括：\n\n版本控制：帮助团队成员之间同步和管理代码变更。\n", "合并提交，但它会改变历史，因此在共享的分支上使用时需要谨慎。\n远程仓库\n远程仓库的作用\n远程仓库是位于互联网或其他网络中的服务器上的Git仓库。它可以让多个开发者共享一个项目，而不必将所有的文件和版本历史存储在本地计算机上。远程仓库的主要作用包括：\n\n版本控制：帮助团队成员之间同步和管理代码变更。\n备份：防止本地数据丢失后可以从远程仓库恢复。协作：多人可以同时工作在同一个项目上，提高开发效率。\n代码审查：通过PullRequests(PRs)等机制可以进行代码审查。\n\n添加远程仓库\n要添加新的远程仓库，可以使用gitremoteadd命令。该命令需要两个参数：远程仓库的名称和远程仓库的URL。\nbash\ngitremoteadd<remote-name><remote-url>\n例如，添加一个名为origin的远程仓库：\nbash\ngitremoteaddorigin\n推送到远程仓库\n将本地的更改推送到远程仓库，可以使用gitpush命令。通常，这个命令后面会跟远程仓库的名称和要推送的分支名称。\nbash\ngitpush<remote-name><branch-name>\n例如，将本地的master分支推送到origin远程仓库：\nbash\ngitpushoriginmaster\n从远程仓库拉取\n", "远程仓库，可以使用gitpush命令。通常，这个命令后面会跟远程仓库的名称和要推送的分支名称。\nbash\ngitpush<remote-name><branch-name>\n例如，将本地的master分支推送到origin远程仓库：\nbash\ngitpushoriginmaster\n从远程仓库拉取\n从远程仓库获取最新的更改并合并到本地分支，可以使用gitpull命令。这个命令会将远程仓库的指定分支的更改拉取到当前分支。bash\ngitpull<remote-name><branch-name>\n例如，从origin远程仓库的master分支拉取最新更改：\nbash\ngitpulloriginmaster\n远程分支管理\n查看远程分支，可以使用gitbranch命令加上-r选项。\nbash\ngitbranch-r\n删除远程分支，可以使用gitpush命令加上--delete选项。\nbash\ngitpush<remote-name>--delete<branch-name>\n例如，删除origin远程仓库的feature分支：\nbash\ngitpushorigin--deletefeature\n远程仓库的协作与贡献\n协作和贡献通常涉及以下步骤：\n\nFork远程仓库。\nCloneFork后的仓库到本地。\n创建新的分支进行开发。\n完成开发后，将分支推送到自己的Fork仓库。\n", "如，删除origin远程仓库的feature分支：\nbash\ngitpushorigin--deletefeature\n远程仓库的协作与贡献\n协作和贡献通常涉及以下步骤：\n\nFork远程仓库。\nCloneFork后的仓库到本地。\n创建新的分支进行开发。\n完成开发后，将分支推送到自己的Fork仓库。\n在原仓库发起PullRequest(PR)。维护者审查代码，并将其合并到主仓库。\n\n例如，将本地分支feature推送到自己Fork的远程仓库：\nbash\ngitpushoriginfeature\n之后，在GitHub或其他托管服务上发起PullRequest。\n高级篇\n变基\n变基（Rebase）是Git中用于整理提交历史的一种工具。它的主要作用是将一系列的提交按照原有顺序复制到另一个基底上。\n基本原理\n当你进行变基操作时，Git会找到这些提交和目标基底（即你想要变基到的提交）的最近共同祖先，然后将每个提交从这个共同祖先开始重新应用。这样，你的提交历史就会看起来像是从目标基底直接分支出来的。\n使用变基\n```bash\n将当前分支变基到指定的\ngitrebase\n```\n暂存区管理\n", "底上。\n基本原理\n当你进行变基操作时，Git会找到这些提交和目标基底（即你想要变基到的提交）的最近共同祖先，然后将每个提交从这个共同祖先开始重新应用。这样，你的提交历史就会看起来像是从目标基底直接分支出来的。\n使用变基\n```bash\n将当前分支变基到指定的\ngitrebase\n```\n暂存区管理\n暂存区（StagingArea）是Git中的一个概念，它是一个准备提交的更改列表。基本原理\n当你执行gitadd命令时，更改就会被添加到暂存区。然后你可以使用gitcommit将这些更改提交到仓库。\n操作暂存区\n```bash\n添加文件到暂存区\ngitadd\n查看暂存区状态\ngitstatus\n取消暂存\ngitresetHEAD\n```\nGit钩子\nGit钩子（Hooks）是在Git执行特定事件（如提交和合并）时触发的脚本。\n基本原理\nGit钩子存放在仓库的.git/hooks目录下。当触发相应的事件时，Git会执行这个目录下的脚本。\n使用Git钩子\n```bash\n编辑钩子脚本\nvim.git/hooks/\n使钩子脚本可执行\nchmod+x.git/hooks/\n```\n子模块\n子模块（Submodules）允许你将一个Git仓库作为另一个Git仓库的子目录。\n基本原理\n", "录下。当触发相应的事件时，Git会执行这个目录下的脚本。\n使用Git钩子\n```bash\n编辑钩子脚本\nvim.git/hooks/\n使钩子脚本可执行\nchmod+x.git/hooks/\n```\n子模块\n子模块（Submodules）允许你将一个Git仓库作为另一个Git仓库的子目录。\n基本原理\n使用子模块可以帮助你管理项目中的第三方代码库。使用子模块\n```bash\n添加子模块\ngitsubmoduleadd\n初始化子模块\ngitsubmoduleinit\n更新子模块\ngitsubmoduleupdate\n```\nGitLFS（大文件存储）\nGitLFS（LargeFileStorage）是一个Git扩展，用于改善大文件的存储和访问。\n基本原理\nGitLFS通过将大文件的内容替换为指针，实际内容存储在服务器上，从而避免了大文件在仓库中的直接存储。\n使用GitLFS\n```bash\n安装GitLFS\ngitlfsinstall\n跟踪大文件\ngitlfstrack\"*.psd\"\n提交更改\ngitadd.gitattributes\ngitcommit-m\"Track.psdfilesusingGitLFS\"\n```\n实用技巧\n撤销与重做\n在使用Git时，我们有时会需要撤销之前的操作或者重做操作。以下是一些常用的撤销与重做操作命令：\n撤销工作目录中的修改\n", "ack\"*.psd\"\n提交更改\ngitadd.gitattributes\ngitcommit-m\"Track.psdfilesusingGitLFS\"\n```\n实用技巧\n撤销与重做\n在使用Git时，我们有时会需要撤销之前的操作或者重做操作。以下是一些常用的撤销与重做操作命令：\n撤销工作目录中的修改\n如果你对文件进行了修改，但是还没有进行提交，你可以使用以下命令来撤销工作目录中的修改：bash\ngitcheckout--<file>\n撤销暂存区的文件\n如果你已经使用gitadd将文件添加到暂存区，但是想要撤销这一操作，可以使用以下命令：\nbash\ngitresetHEAD<file>\n撤销提交\n如果你已经进行了提交，但是想要撤销这次提交，可以使用以下命令：\nbash\ngitrevert<commit>\n其中<commit>是你想要撤销的提交的哈希值。\n重做提交\n如果你撤销了某次提交，但是后来又决定这次提交是正确的，想要重新应用这次提交，可以使用以下命令：\nbash\ngitreset--hard<commit>\n这将会重置当前分支到指定的提交。\n日志搜索技巧\nGit提供了强大的日志搜索功能，可以帮助我们快速找到特定的提交信息。\n搜索提交日志\n使用以下命令可以搜索提交日志：\nbash\ngitlog--grep=<pattern>\n", "交，可以使用以下命令：\nbash\ngitreset--hard<commit>\n这将会重置当前分支到指定的提交。\n日志搜索技巧\nGit提供了强大的日志搜索功能，可以帮助我们快速找到特定的提交信息。\n搜索提交日志\n使用以下命令可以搜索提交日志：\nbash\ngitlog--grep=<pattern>\n其中<pattern>是你想要搜索的关键词或正则表达式。查看特定文件的变更记录\n如果你只对某个特定文件的变更记录感兴趣，可以使用以下命令：\nbash\ngitlog-p<file>\n这将显示该文件的每次提交差异。\n查看某个范围内的提交\n你也可以指定查看某个时间范围内的提交：\nbash\ngitlog--since=\"2weeksago\"--until=\"3daysago\"\n这将显示从两周前到三天前的所有提交。\n搭建Git服务器\n搭建Git服务器可以让团队成员共享代码库和协作开发。以下是搭建Git服务器的基本步骤：\n安装Git\n首先确保服务器上安装了Git：\nbash\nsudoapt-getinstallgit\n创建一个裸仓库\n在服务器上创建一个裸仓库：\nbash\ngitinit--bare<repository.git>\n设置SSH访问\n", "可以让团队成员共享代码库和协作开发。以下是搭建Git服务器的基本步骤：\n安装Git\n首先确保服务器上安装了Git：\nbash\nsudoapt-getinstallgit\n创建一个裸仓库\n在服务器上创建一个裸仓库：\nbash\ngitinit--bare<repository.git>\n设置SSH访问\n确保团队成员的SSH公钥被添加到服务器的~/.ssh/authorized_keys文件中，以便他们可以通过SSH访问仓库。Git与持续集成\nGit可以与持续集成(CI)系统结合使用，以自动化代码的构建、测试和部署流程。\n集成CI服务\n你可以选择一个CI服务，如Jenkins、TravisCI或CircleCI，并按照服务提供商的指南将其与你的Git仓库集成。\n配置构建脚本\n在你的项目中添加一个构建脚本，例如.travis.yml或Jenkinsfile，并配置构建、测试和部署的命令。\n自动触发构建\n每当有新的提交推送到仓库时，CI服务会自动触发构建流程，并反馈构建结果。\nGit工作流程\nGit工作流程是指团队使用Git进行协作开发的一系列规范流程。以下是一些常见的Git工作流程：\nFeatureBranchWorkflow\n", "Jenkinsfile，并配置构建、测试和部署的命令。\n自动触发构建\n每当有新的提交推送到仓库时，CI服务会自动触发构建流程，并反馈构建结果。\nGit工作流程\nGit工作流程是指团队使用Git进行协作开发的一系列规范流程。以下是一些常见的Git工作流程：\nFeatureBranchWorkflow\n特性分支工作流程中，每个新功能都在独立的分支上开发，完成后再合并到主分支：```bash\ngitcheckout-bfeature_branch\n开发新功能\ngitcommit-am\"Addnewfeature\"\ngitcheckoutmaster\ngitmergefeature_branch\n```\nGitflowWorkflow\nGitflow工作流程定义了一个围绕项目发布的严格分支模型，包括功能分支、发布分支和维护分支。\nForkingWorkflow\n在Forking工作流程中，每个贡献者都有自己的服务器端仓库，他们可以自由地推送提交，然后通过拉取请求来贡献代码。\n以上是对实用技巧模块的详细介绍。\n附录\n常用Git命令清单\n在使用Git进行版本控制时，以下是一些常用的命令：\n\ngitinit：在当前目录中初始化一个新的Git仓库。\ngitclone[url]：克隆一个仓库到本地目录。\ngitadd[file]：将文件添加到暂存区。\n", "后通过拉取请求来贡献代码。\n以上是对实用技巧模块的详细介绍。\n附录\n常用Git命令清单\n在使用Git进行版本控制时，以下是一些常用的命令：\n\ngitinit：在当前目录中初始化一个新的Git仓库。\ngitclone[url]：克隆一个仓库到本地目录。\ngitadd[file]：将文件添加到暂存区。\ngitcommit-m\"[commitmessage]\"：将暂存区的内容提交到仓库。gitstatus：查看仓库当前的状态，显示有变更的文件。\ngitpush[alias][branch]：将本地分支的更新推送到远程仓库。\ngitpull[alias][branch]：从远程仓库获取最新版本并合并到本地。\ngitbranch：列出所有本地分支。\ngitbranch-a：列出所有本地分支和远程分支。\ngitbranch[branch-name]：创建新分支。\ngitcheckout[branch-name]：切换到指定分支。\ngitmerge[branch]：合并指定分支到当前分支。\ngitlog：查看提交历史。\ngitdiff：查看未暂存的文件更新了哪些部分。\ngitreset：重置当前HEAD到指定状态。\n\nGit配置文件解析\nGit配置文件.gitconfig通常位于用户主目录下，用于配置用户级别的Git选项。\nini\n[user]\nname=YourName\nemail=you@example.com\n[alias]\nco=checkout\nbr=branch\nci=commit\nst=status\n\n", "t配置文件解析\nGit配置文件.gitconfig通常位于用户主目录下，用于配置用户级别的Git选项。\nini\n[user]\nname=YourName\nemail=you@example.com\n[alias]\nco=checkout\nbr=branch\nci=commit\nst=status\n\n[user]部分用于设置提交代码时的用户信息。[alias]部分可以设置命令的别名，简化命令输入。\n\nGit错误处理\n处理Git错误时，首先应该使用gitstatus和gitlog检查当前状态和历史提交。以下是一些常见的Git错误处理方法：\n\n当遇到合并冲突时，需要手动编辑文件解决冲突，然后使用gitadd[file]标记为已解决，最后提交。\n如果需要撤销最近的提交，可以使用gitreset--hardHEAD^回退到上一个提交状态。\n当本地分支落后于远程分支，需要合并远程变更时，可以使用gitpull来更新本地分支。\n\n参考资料与进阶阅读\n以下是一些推荐的参考资料和进阶阅读：\n\nProGit书籍：深入理解Git的原理和使用方法。\nGit官方文档：提供详细的命令参考和使用场景。\nGit社区论坛：在社区中与其他开发者交流心得和技巧。\n", "Git教程\nGit简介\n什么是Git\nGit是一种分布式版本控制系统，用于跟踪文件的变化并协作开发。它可以记录文件的修改历史，并允许多人在同一项目上进行协作开发。\nGit的优势\n\n分布式：每个开发者都拥有整个代码库的完整历史记录，无需依赖中央服务器。\n高效：Git的分支和合并操作非常快速，使得并行开发变得更加容易。\n强大的支持：Git支持大型项目、快速的存储和高效的处理能力。\n\n```bash\n创建一个新的Git仓库\ngitinit\n```\n```bash\n克隆一个现有的Git仓库\ngitclone<仓库地址>\n```\n```bash\n将文件添加到暂存区\ngitadd<文件名>\n```\n```bash\n提交文件\ngitcommit-m\"提交说明\"\n```\nGit基础\nGit的安装\n首先，您需要下载适用于您操作系统的Git安装程序。然后，按照安装向导的步骤进行安装。\n创建仓库\n要在Git中创建一个新的仓库，可以使用以下命令：\nbash\ngitinit\n这将在当前目录中创建一个新的Git仓库。\n添加文件\n要将文件添加到Git仓库，可以使用以下命令：\nbash\ngitadd<文件名>\n这将把指定的文件添加到暂存区。\n提交更改\n一旦您对文件进行了修改并将其添加到暂存区，您可以使用以下命令提交更改：\nbash\ngitcommit-m\"提交说明\"\n", "nit\n这将在当前目录中创建一个新的Git仓库。\n添加文件\n要将文件添加到Git仓库，可以使用以下命令：\nbash\ngitadd<文件名>\n这将把指定的文件添加到暂存区。\n提交更改\n一旦您对文件进行了修改并将其添加到暂存区，您可以使用以下命令提交更改：\nbash\ngitcommit-m\"提交说明\"\n这将提交您所做的更改并将其保存到仓库中。查看提交历史\n要查看Git仓库的提交历史，可以使用以下命令：\nbash\ngitlog\n这将显示所有的提交历史记录，包括提交者信息、提交时间和提交说明。\n以上就是Git基础模块的内容。\n分支管理\n创建分支\n要创建一个新的分支，可以使用以下命令：\nbash\ngitbranch<branch_name>\n这将创建一个名为<branch_name>的新分支，但并不会自动切换到这个分支。\n切换分支\n要切换到已经存在的分支，可以使用以下命令：\nbash\ngitcheckout<branch_name>\n这将使当前工作目录切换到名为<branch_name>的分支上。\n合并分支\n要将一个分支的更改合并到当前分支，可以使用以下命令：\nbash\ngitmerge<branch_name>\n这将把名为<branch_name>的分支合并到当前分支上。\n解决冲突\n", "checkout<branch_name>\n这将使当前工作目录切换到名为<branch_name>的分支上。\n合并分支\n要将一个分支的更改合并到当前分支，可以使用以下命令：\nbash\ngitmerge<branch_name>\n这将把名为<branch_name>的分支合并到当前分支上。\n解决冲突\n在合并分支时，如果发生冲突，需要手动解决冲突。可以通过编辑文件来解决冲突，然后使用以下命令标记文件为已解决冲突的状态：bash\ngitadd<file_name>\n解决完所有冲突后，可以继续合并分支。\n以上是关于Git分支管理的基本操作。\n远程仓库\n添加远程仓库\n要将本地仓库与远程仓库关联，可以使用以下命令：\nbash\ngitremoteaddorigin远程仓库地址\n其中，origin是远程仓库的别名，可以根据实际情况自行命名。\n推送到远程仓库\n将本地提交推送到远程仓库可以使用以下命令：\nbash\ngitpushorigin分支名\n例如，将本地的master分支推送到远程仓库可以使用：\nbash\ngitpushoriginmaster\n从远程仓库拉取\n从远程仓库拉取最新代码到本地可以使用以下命令：\nbash\ngitpullorigin分支名\n", "本地提交推送到远程仓库可以使用以下命令：\nbash\ngitpushorigin分支名\n例如，将本地的master分支推送到远程仓库可以使用：\nbash\ngitpushoriginmaster\n从远程仓库拉取\n从远程仓库拉取最新代码到本地可以使用以下命令：\nbash\ngitpullorigin分支名\n例如，从远程仓库拉取master分支的最新代码可以使用：bash\ngitpulloriginmaster\n解决冲突\n在拉取或合并代码时，如果出现冲突，需要手动解决冲突并提交。可以通过编辑文件解决冲突后，再执行以下命令提交解决：\nbash\ngitadd.\ngitcommit-m\"解决冲突\"\ngitpushorigin分支名\n以上是关于远程仓库操作的基本内容。\n高级主题\n标签\n在Git中，标签是用来标记特定提交的版本号的。通过给提交打上标签，可以方便地在需要的时候找到该提交。创建标签可以使用gittag命令，例如：\nbash\ngittagv1.0.0\n忽略文件\n有时候我们希望Git在进行版本控制时忽略某些文件，比如临时文件或者编译生成的文件。可以通过在项目根目录下创建一个名为.gitignore的文件，并在其中指定需要忽略的文件模式，例如：\nplaintext\n*.log\n", "标签可以使用gittag命令，例如：\nbash\ngittagv1.0.0\n忽略文件\n有时候我们希望Git在进行版本控制时忽略某些文件，比如临时文件或者编译生成的文件。可以通过在项目根目录下创建一个名为.gitignore的文件，并在其中指定需要忽略的文件模式，例如：\nplaintext\n*.log\nbuild/撤销操作\n在Git中，有时候我们需要撤销之前的操作，比如撤销暂存的文件或者撤销已提交的修改。可以使用gitreset命令来撤销暂存的文件，使用gitrevert命令来撤销已提交的修改。\nGit工作流\nGit工作流是指团队协作时如何使用Git进行版本控制和代码管理的一种约定。常见的Git工作流包括集中式工作流、特性分支工作流、Gitflow工作流等。选择合适的工作流对团队的协作效率有很大的影响。\n实用技巧\n查看差异\n要查看文件的差异，可以使用以下命令：\nbash\ngitdiff\n修改最后一次提交\n如果需要修改最后一次提交的信息，可以使用以下命令：\nbash\ngitcommit--amend\n临时存储修改\n使用以下命令可以将当前工作目录中的修改临时存储起来：\nbash\ngitstash\n重写提交历史\n", "\n查看差异\n要查看文件的差异，可以使用以下命令：\nbash\ngitdiff\n修改最后一次提交\n如果需要修改最后一次提交的信息，可以使用以下命令：\nbash\ngitcommit--amend\n临时存储修改\n使用以下命令可以将当前工作目录中的修改临时存储起来：\nbash\ngitstash\n重写提交历史\n如果需要重写提交历史，可以使用以下命令：bash\ngitrebase-iHEAD~n\n以上是关于Git实用技巧的详细内容。\n", "GithubApi使用\nGithubApi简介\n什么是GithubApi\nGithubApi是Github提供的一组用于访问和操作Github数据的接口。通过GithubApi，用户可以实现对仓库、提交、用户等资源的管理和操作。\nGithubApi的优势\n\n灵活性：GithubApi提供了丰富的接口，可以满足各种不同的需求，用户可以根据自身需求进行定制化开发。\n整合性：GithubApi可以与其他服务和工具进行无缝整合，例如持续集成工具、项目管理工具等，提高工作效率。\n实时性：通过GithubApi，可以实时获取最新的仓库、提交等信息，帮助用户及时了解和响应变化。\n\n以上是GithubApi的简要介绍和优势。\nGithubApi的基本用法\n认证与权限\n在使用GithubApi之前，我们需要进行认证以获取相应的权限。GithubApi使用OAuth2.0进行认证，我们可以通过申请一个个人访问令牌（PersonalAccessToken）来进行认证。\n申请个人访问令牌\n\n登录Github，点击头像进入Settings。\n在左侧菜单中选择Developersettings，然后点击Personalaccesstokens。\n点击Generatenewtoken，填写Tokendescription，并勾选需要的权限。\n点击Generatetoken，将生成的访问令牌保存好，之后将用于认证。\n\n使用个人访问令牌进行认证\n在进行数据获取与操作时，需要在请求的Header中添加Authorization字段，其值为\"token\\<your_personal_access_token>\"。\n数据获取与操作\n", "需要的权限。\n点击Generatetoken，将生成的访问令牌保存好，之后将用于认证。\n\n使用个人访问令牌进行认证\n在进行数据获取与操作时，需要在请求的Header中添加Authorization字段，其值为\"token\\<your_personal_access_token>\"。\n数据获取与操作\nGithubApi提供了丰富的接口来获取和操作数据，包括获取仓库信息、提交记录、问题等。获取仓库信息\n我们可以使用GithubApi来获取特定用户或组织的仓库信息，例如：\n```python\nimportrequests\nurl='\nheaders={'Authorization':'token'}\nresponse=requests.get(url,headers=headers)\nprint(response.json())\n```\n提交操作\n我们可以使用GithubApi来创建、更新和删除提交，例如：\n```python\nimportrequests\nurl='\nheaders={'Authorization':'token'}\ndata={\n\"message\":\"mycommitmessage\",\n\"content\":\"bXkgbmV3IGZpbGUgY29udGVudHM=\"#base64encodedcontent\n}\nresponse=requests.put(url,headers=headers,json=data)\nprint(response.json())\n```\n以上就是GithubApi的基本用法，包括认证与权限以及数据获取与操作。通过这些基本用法，我们可以在自己的应用中使用GithubApi来实现丰富的功能。\nGithubApi的高级应用\nWebhooks\n", "rs=headers,json=data)\nprint(response.json())\n```\n以上就是GithubApi的基本用法，包括认证与权限以及数据获取与操作。通过这些基本用法，我们可以在自己的应用中使用GithubApi来实现丰富的功能。\nGithubApi的高级应用\nWebhooks\nWebhooks是GithubApi中的一项高级功能，它允许用户在特定事件发生时自动触发自定义的HTTP回调。通过配置Webhooks，您可以实现对代码仓库中的各种事件进行监控和自动化处理，例如代码提交、Issue创建等。配置Webhooks\n要配置Webhooks，您可以通过Github仓库的设置页面进行操作。在Webhooks页面，您可以添加新的Webhook，并指定触发Webhook的事件类型和回调URL。\nWebhooks的工作原理\n当配置的事件类型在仓库中发生时，Github会向指定的回调URL发送HTTPPOST请求，请求中包含有关事件的详细信息。您可以编写自定义的服务器端代码来处理这些请求，实现自动化的业务逻辑。\n示例代码\n以下是一个简单的Node.jsExpress服务器端代码示例，用于处理GithubWebhooks的HTTPPOST请求：\n```javascript\nconstexpress=require('express');\nconstbodyParser=require('body-parser');\nconstapp=express();\napp.use(bodyParser.json());\napp.post('/webhook',(req,res)=>{\nconstevent=req.get('X-GitHub-Event');\n", "Parser=require('body-parser');\nconstapp=express();\napp.use(bodyParser.json());\napp.post('/webhook',(req,res)=>{\nconstevent=req.get('X-GitHub-Event');\n//根据event类型处理业务逻辑res.sendStatus(200);\n});\napp.listen(3000,()=>{\nconsole.log('Webhook服务器已启动');\n});\n```\nGithubActions\nGithubActions是Github提供的持续集成和持续部署（CI/CD）工具，它允许您在代码仓库中配置自定义的工作流程，以实现自动化的构建、测试和部署。\n配置GithubActions\n要配置GithubActions，您可以在仓库中创建一个名为.github/workflows的目录，并在该目录中添加YAML格式的工作流程配置文件。在配置文件中，您可以定义工作流程的触发条件、执行步骤和环境。\nGithubActions的工作原理\n当配置的触发条件满足时，Github会自动执行相应的工作流程。您可以在工作流程中使用Github提供的预定义动作（actions），也可以编写自定义的脚本来实现特定的构建、测试和部署操作。\n示例代码\n", "在配置文件中，您可以定义工作流程的触发条件、执行步骤和环境。\nGithubActions的工作原理\n当配置的触发条件满足时，Github会自动执行相应的工作流程。您可以在工作流程中使用Github提供的预定义动作（actions），也可以编写自定义的脚本来实现特定的构建、测试和部署操作。\n示例代码\n以下是一个简单的GithubActions配置文件示例，用于在代码提交后自动运行测试并部署到服务器：```yaml\nname:CI/CD\non:\npush:\nbranches:[main]\njobs:\nbuild:\nruns-on:ubuntu-latest\nsteps:\n-name:Checkoutcode\nuses:actions/checkout@v2\n\n-name:Runtests\nrun:npmtest\n\n-name:Deploytoserver\nrun:|\nsshuser@server'cd/path/to/app&&gitpull'\n\n```\n以上是GithubApi的高级应用中Webhooks和GithubActions的详细内容。\n", "GithubApi使用\n#GithubAPI简介\nGithubAPI是Github提供的一组接口，允许用户通过编程方式与Github进行交互。通过这些接口，用户可以获取Github仓库的信息、创建和修改仓库、管理issue和pullrequest等。\nGithubAPI支持多种编程语言，包括Python、Java、Ruby等。用户可以通过这些语言编写程序，调用GithubAPI接口，实现自动化操作。\nGithubAPI功能\nGithubAPI提供了丰富的功能，包括但不限于：\n\n获取仓库信息\n创建和修改仓库\n管理issue和pullrequest\n获取用户信息\n管理用户权限\n搜索仓库和用户\n\n以下是一个简单的示例代码，演示如何使用Python调用GithubAPI获取仓库信息：\n```python\nimportrequests\ndefget_repo_info(owner,repo):\nurl=f\"\nresponse=requests.get(url)\nifresponse.status_code==200:\nreturnresponse.json()\nelse:\nreturnNone\n调用函数获取仓库信息\nrepo_info=get_repo_info(\"octocat\",\"Hello-World\")\nifrepo_info:\nprint(repo_info)\nelse:\nprint(\"获取仓库信息失败\")\n```\n以上代码中，我们使用requests库发送HTTP请求，获取仓库信息。如果请求成功，则返回JSON格式的仓库信息；如果请求失败，则返回None。\n需要注意的是，在使用GithubAPI时，需要先进行身份验证，可以通过在请求头中添加Authorization字段来实现。\n目录2\nGithubApi使用方法\nGithubApi是一种用于\n#GithubAPI常见问题\n在使用GithubAPI时，可能会遇到一些常见问题，以下是一些可能遇到的问题及其解决方法。\n问题1：无法获取\n", "GithubApi使用\n目录1\nGithubApi简介\nGithubAPI是Github提供的一组接口，用于通过编程方式\n##GithubApi使用方法\nGithubApi是一种基于RESTful架构的API，它允许用户通过HTTP请求与Github进行交互。以下是使用GithubApi的基本步骤：\n1.获取\n"]

================================================
FILE: storage/vectors.json
================================================
[[-0.017197759822010994, -0.0077917021699249744, 0.028354724869132042, -0.012402127496898174, -0.006169886328279972, 0.03085353597998619, 0.02078026905655861, 0.007133822422474623, -0.0074239326640963554, -0.01974455453455448, -0.064917653799057, 0.00984879955649376, -0.013309323228895664, -0.009821664541959763, 0.011840742081403732, 0.014113342389464378, 0.04286569729447365, -0.009373747743666172, -0.03756726160645485, 0.0924081802368164, 0.0028943114448338747, -0.0593484565615654, 0.05980180948972702, 0.002079608151689172, 0.01430167630314827, -0.008000838570296764, -0.044636309146881104, 0.03643488883972168, 0.011983884498476982, 0.05243745818734169, 0.011906508356332779, 0.014417668804526329, -0.004172923043370247, -0.0055344258435070515, -0.004556608386337757, 0.032166387885808945, 0.004920446313917637, -0.018027905374765396, -0.015006162226200104, 0.02221720851957798, -0.0076006450690329075, -0.0014201882295310497, 0.012154582887887955, 0.02462827041745186, -0.05755208432674408, 0.021627409383654594, 0.01199701800942421, -0.02724805660545826, 0.01990363374352455, -0.010063902474939823, 0.005891849752515554, 0.19080398976802826, 0.037033457309007645, 0.037783022969961166, -0.0476749949157238, 0.03892304003238678, 0.01685243658721447, -0.004535602871328592, 0.006152578163892031, -0.04235602915287018, -0.0652112141251564, -0.019565697759389877, -0.001215911004692316, -0.011521072126924992, -0.01083435583859682, 0.0037009441293776035, -0.007398827467113733, -0.06613144278526306, 0.03298018500208855, -0.005656063556671143, 0.04376447573304176, 0.05510375648736954, -0.005345395766198635, -0.009188273921608925, 0.038457076996564865, 0.027689378708600998, -0.05167552828788757, -0.025395933538675308, -0.014983445405960083, -0.05180492252111435, 0.02425229921936989, 0.0002990645880345255, -0.021898984909057617, 0.021893195807933807, -0.025847237557172775, 0.05797016620635986, 0.004472613334655762, 0.046069659292697906, 0.0163776446133852, 0.0035680870059877634, 0.05643460527062416, -0.028300689533352852, 0.01689695566892624, 0.00891279149800539, 0.035876642912626266, 0.0021870958153158426, -0.05668463930487633, -0.012151763774454594, -0.020960675552487373, -0.009337716735899448, 0.016966911032795906, 0.011101522482931614, -0.0030517461709678173, -0.01534917950630188, 0.0015929241199046373, 0.01208888553082943, 0.011025075800716877, -0.016674624755978584, 0.027622368186712265, 0.015205688774585724, 0.0016509497072547674, -0.024263979867100716, 0.013543196953833103, 0.05787285417318344, 0.018303627148270607, 0.02097819373011589, 0.04328088462352753, 0.02025974728167057, 0.016624705865979195, -0.020556388422846794, -0.006847816985100508, -0.05198926851153374, 0.005251508206129074, -0.006298276595771313, 0.043849311769008636, -0.06498997658491135, -0.005839669145643711, -0.002567542949691415, -0.028658786788582802, 0.011945963837206364, -0.0036170969251543283, -0.007439082954078913, 0.05876477435231209, 0.017846815288066864, -0.009409205056726933, 0.03418773412704468, 0.00682337861508131, -0.02740642987191677, -0.017384983599185944, -0.016701364889740944, -0.011251494288444519, 0.0016724063316360116, -0.027177343145012856, 0.018321838229894638, -0.008903608657419682, -0.011392250657081604, -0.02022068202495575, -0.04083165526390076, -0.010606745257973671, 0.013921895995736122, 0.0618089996278286, 0.03206725791096687, 0.014768409542739391, -0.022621406242251396, -0.036327213048934937, 0.016245204955339432, -0.026417890563607216, 0.021124498918652534, -0.015667252242565155, -0.06594038009643555, 0.020826082676649094, 0.007282315753400326, -0.008619331754744053, -0.023136885836720467, -0.06718868762254715, -0.07076102495193481, 0.006760864052921534, -0.032413411885499954, -0.034355174750089645, -0.06773725897073746, 0.07219583541154861, -0.058921895921230316, -0.0049834358505904675, -0.0064957537688314915, 0.01586061529815197, 0.03247473016381264, -0.009202277287840843, 0.005102335475385189, 0.02608192339539528, 0.03998207300901413, 0.024941641837358475, -0.046026818454265594, -0.020483436062932014, 0.0036334851756691933, 0.011575644835829735, -0.02487071044743061, -0.03839641436934471, 0.010197194293141365, 0.007150591351091862, -0.0003789533511735499, 0.05548819154500961, -0.012066314928233624, -0.007153196260333061, 0.011248939670622349, -0.06001079082489014, -0.0104060685262084, 0.02122671902179718, -0.05969717353582382, -0.0016373960534110665, 0.016886405646800995, 0.01718713529407978, -0.014319110661745071, 0.02073168195784092, 0.03727247193455696, -0.04198668897151947, 0.00989068578928709, -0.0041458141058683395, 0.013454102911055088, -0.001953396713361144, 0.019349336624145508, 0.032475415617227554, 0.019144322723150253, 0.029463838785886765, -0.004767466336488724, 0.03174727037549019, -0.04143504053354263, -0.03087613172829151, -0.010723520070314407, -0.008088959380984306, -0.011621175333857536, 0.013199134729802608, -0.06026715040206909, -0.01146214734762907, 0.021496260538697243, -0.03027997724711895, 0.0161273293197155, -0.028564829379320145, 0.009044172242283821, -0.02419089898467064, 0.026029186323285103, -0.02096940018236637, 0.012206373736262321, 0.018473930656909943, 0.0012546012876555324, 0.012401613406836987, -0.014011277817189693, -0.025443032383918762, -0.025304647162556648, -0.005514508578926325, 0.0036139562726020813, 0.048749711364507675, -0.0060511562041938305, 0.018441205844283104, -0.06396528333425522, -0.008473601192235947, 0.009959296323359013, -0.034345801919698715, 0.001989110140129924, 0.011988380923867226, 0.0075074234046041965, 0.015250114724040031, 0.015342864207923412, -0.0014977827668190002, -0.032996486872434616, -0.03646702691912651, 0.009200872853398323, 0.09653843939304352, 0.023533819243311882, 0.014935330487787724, 0.0037554395385086536, -0.020902063697576523, 0.025900699198246002, 0.0334300771355629, 0.027109436690807343, 0.02494490146636963, -0.008224662393331528, -0.037067387253046036, 0.008904856629669666, -0.001784196007065475, 0.03605631738901138, -0.0004488654376473278, -0.004918623715639114, 0.009659882634878159, -0.027402784675359726, -0.0076299733482301235, -0.015078245662152767, -0.02061394788324833, -0.024839535355567932, -0.058603640645742416, -0.041169047355651855, -0.05301741510629654, -0.012085752561688423, 0.04001038894057274, 0.01774289831519127, 0.034783340990543365, -0.020280523225665092, -0.01910609006881714, 0.05047924816608429, 0.024615729227662086, -0.033594779670238495, 0.0038072552997618914, 0.038713373243808746, -0.07179544866085052, -0.010766733437776566, 0.39197900891304016, -0.06567473709583282, 0.018084945157170296, 0.01767931878566742, -0.015155311673879623, 0.010404305532574654, 0.021086540073156357, -0.014216518960893154, -0.04425938427448273, -0.004354054108262062, -0.050891999155282974, 0.028640462085604668, -0.01042365375906229, 0.04790693148970604, -0.003560509067028761, -0.04756522923707962, -0.014649632386863232, -0.033214177936315536, 0.0023004021495580673, -0.054168105125427246, 0.05658922344446182, -0.00914736744016409, -0.00423071626573801, -0.05519673600792885, -0.039638668298721313, -0.03206092491745949, 0.032201677560806274, -0.06510694324970245, -0.028078164905309677, -0.0009808979230001569, 0.06130656227469444, 0.0614372156560421, 0.006128442939370871, 0.023035742342472076, -0.04663081839680672, -0.05755745992064476, 0.028064345940947533, -0.03333276882767677, -0.026595892384648323, -0.009258667938411236, 0.011211169883608818, -0.04254601523280144, -0.004302098415791988, 0.042847782373428345, -0.00017497515364084393, -0.07909055054187775, 0.03453053906559944, -0.00036480542621575296, -0.03583317995071411, -0.010057961568236351, 0.01866205595433712, -0.030042041093111038, 0.05136016756296158, -0.024967500939965248, 0.010408920235931873, -0.022250983864068985, 0.005013930145651102, 0.05107681453227997, 0.04041130468249321, 0.007485001813620329, 0.013141341507434845, -0.009937548078596592, 0.005171061027795076, 0.051320746541023254, -0.05701785162091255, 0.009282534010708332, 0.01761743612587452, 0.01628105342388153, -0.019287172704935074, 0.015207629650831223, -0.04999551922082901, -0.021213890984654427, -0.012778112664818764, 0.04109848290681839, -0.02068006992340088, 0.013867723755538464, 0.06445030868053436, 0.0019182192627340555, 0.010286686010658741, 0.012396854348480701, -0.02594250626862049, 0.04576540365815163, 0.014265310019254684, 0.016460945829749107, -0.006711272988468409, 0.03465590998530388, 0.024138538166880608, 0.03740130364894867, 0.017764363437891006, 0.0017154377419501543, 0.028087664395570755, -0.034668248146772385, 0.016667181625962257, -0.005357222631573677, 0.013924892991781235, -0.014694329351186752, 0.022179046645760536, 0.005883888807147741, -0.01853552833199501, 0.04094506800174713, -0.022403528913855553, -0.011237584985792637, 0.00930582545697689, -0.004014487378299236, 0.03457603231072426, -0.025403251871466637, -0.04505274444818497, -0.04089667275547981, 0.008890916593372822, 0.0237202737480402, -0.0503167025744915, -0.015542318113148212, -0.027856316417455673, -0.012626126408576965, -0.0042140306904911995, 0.05101551115512848, -0.02851204201579094, 0.04153137654066086, -0.02252333238720894, -0.025964537635445595, -0.0025238869711756706, -0.04159992188215256, 0.0030617795418947935, -0.03678913041949272, 0.039334941655397415, -0.010358382016420364, -0.036909397691488266, 0.017129328101873398, 0.014655515551567078, -0.019087262451648712, 0.010279681533575058, -0.018831122666597366, 0.10429035127162933, 0.046496253460645676, 0.06553304195404053, 0.041747670620679855, 0.008799925446510315, 0.05433904379606247, -0.031088558956980705, -0.016487911343574524, 0.0024733247701078653, -0.04002862796187401, -0.012628687545657158, 0.028241939842700958, -0.008953919634222984, 0.005017465446144342, -0.04374249279499054, -0.027479827404022217, 0.00908809993416071, 0.021024765446782112, -0.011759603396058083, -0.02705254592001438, 0.033485934138298035, -0.04840824380517006, 0.024073539301753044, 0.05447784811258316, -0.0456615649163723, -0.05859538912773132, -0.058122582733631134, 0.010257667861878872, 0.0010730453068390489, -0.012356498278677464, 0.018397128209471703, -0.022817227989435196, 0.02889951318502426, -0.01883785054087639, 0.04144801199436188, 0.011008636094629765, 0.05150487273931503, 0.0018333469051867723, -0.025095593184232712, 0.026775803416967392, -0.004078419413417578, -0.015455085784196854, -0.020568618550896645, 0.0017473361222073436, -0.04706459492444992, -0.002549347234889865, -0.02097729966044426, -0.03978775069117546, -0.02048986591398716, 0.01744442991912365, 0.026799220591783524, -0.01979202590882778, -0.00809768121689558, 0.02658071741461754, 0.031651102006435394, 0.028288444504141808, -0.0376533567905426, -0.06970475614070892, -0.007611953187733889, -0.02860584482550621, -0.03044874593615532, 0.0428135059773922, 0.010347187519073486, -0.014781645499169827, -0.04862197861075401, -0.046259552240371704, 0.053978823125362396, -0.0020476465579122305, 0.038288015872240067, -0.020422210916876793, -0.02226530946791172, 0.022418534383177757, -0.026297619566321373, 0.010471013374626637, 0.09597571194171906, -0.017011191695928574, 0.0006232737214304507, -0.031205028295516968, 0.016776492819190025, 0.020747177302837372, -0.0006653852178715169, 0.0036059715785086155, 8.047772098507266e-06, -0.042603008449077606, -0.06242362782359123, -0.0022882665507495403, 0.02880142442882061, 0.0700339823961258, -0.031101008877158165, -0.026904411613941193, 0.03999311104416847, 0.023484930396080017, -0.012416621670126915, 0.0071156821213662624, 0.006847628392279148, -0.03576452285051346, -0.01606222242116928, 0.06387559324502945, 0.07293123751878738, -0.012544244527816772, -0.021488169208168983, -0.004130179062485695, 0.04077289626002312, -0.07802421599626541, 0.00510050542652607, -0.01273306179791689, -0.031814251095056534, -0.08507471531629562, -0.020847130566835403, 0.018796125426888466, 0.01905777119100094, -0.054598286747932434, 0.01919279433786869, 0.030347976833581924, -0.018642162904143333, -0.013093771412968636, -0.00026275007985532284, 0.049048129469156265, 0.05227557197213173, 0.06326280534267426, -0.005200780462473631, -0.021461138501763344, -0.005002039950340986, -0.010028999298810959, -0.03767877444624901, 0.0569961778819561, -0.007158085238188505, -0.0055710868909955025, -0.02556893415749073, -0.029974738135933876, -0.003035860601812601, -0.03398020565509796, 0.015505578368902206, 0.0018115780549123883, -0.014120969921350479, 0.026035422459244728, -0.03193747624754906, 0.022631503641605377, -0.026021841913461685, 0.008546063676476479, 0.05017485097050667, -0.018952077254652977, 0.035757876932621, 0.0016180669190362096, -0.03253287822008133, 0.005099905654788017, -0.009276431053876877, 0.043091803789138794, -0.0038848528638482094, -0.0032471215818077326, 0.027876483276486397, -0.013965738005936146, -0.015962230041623116, -0.10914833843708038, -0.0239760410040617, -0.06702754646539688, -0.00784717034548521, -0.046690020710229874, -0.020232291892170906, -0.0699329599738121, -0.03181205317378044, 0.028770040720701218, -0.003858541836962104, -0.07342002540826797, 0.03148343786597252, -0.046072643250226974, -0.00336886802688241, 0.022837061434984207, -0.04270350933074951, 0.005180213134735823, 0.0033988708164542913, 0.04331617429852486, 0.06371177732944489, 0.05007440969347954, 0.007310229819267988, 0.027806419879198074, 0.05103381723165512, 0.0035299162846058607, -0.033850040286779404, -0.05356078967452049, -0.0013971691951155663, -0.008697779849171638, -0.04401213303208351, -0.03790539130568504, 0.06359554827213287, -0.017628613859415054, 0.009068126790225506, 0.03740775212645531, -0.0512065514922142, -0.03208395838737488, -0.03611520677804947, 0.0053635695949196815, 0.018417082726955414, -0.014430196955800056, 0.07507608830928802, -0.025919688865542412, -0.015463843010365963, 0.0064857834950089455, -0.04936976730823517, -0.002939876401796937, 0.00739178666844964, 0.008618810214102268, -0.02945033460855484, 0.021610282361507416, 0.03489147499203682, 0.028238216415047646, -0.007000762969255447, 0.020943989977240562, -0.007171413395553827, -0.024464888498187065, 0.01161651685833931, 0.026810161769390106, -0.02659139595925808, 0.02745945192873478, -0.013155643828213215, -0.08326303958892822, -0.050209954380989075, -0.009706475771963596, 0.036225996911525726, 0.008743248879909515, -0.04863069951534271, 0.003864339552819729, 0.005945851560682058, -0.031353551894426346, -0.03413528576493263, 0.033400196582078934, 0.05013474076986313, -0.05792296677827835, 0.011999146081507206, 0.04207721725106239, 0.014722572639584541, -0.028349652886390686, -0.06588150560855865, -0.0612073615193367, -0.028971213847398758, 0.009072436019778252, 0.0031015416607260704, -0.0002708710671868175, -0.030384380370378494, -0.013541145250201225, 0.006065284833312035, 0.020052973181009293, -0.040977779775857925, 0.011446505784988403, -0.01565415784716606, -0.034930355846881866, 0.05441528931260109, -0.055847812443971634, 0.017033256590366364, -0.0208435095846653, -0.030797285959124565, 0.053567156195640564, -0.01380468625575304, -0.02372252568602562, -0.06096632406115532, -0.002543663140386343, -0.006638861261308193, -0.0005518398247659206, -0.019993726164102554, 0.0103369215503335, -0.023341532796621323, 0.00015673832967877388, -0.0022583547979593277, 0.015041988343000412, -0.006045388523489237, 0.016499977558851242, -0.058141566812992096, 0.022172724828124046, -0.02380804345011711, 0.08502096682786942, 0.010563051328063011, -0.006472250912338495, 0.014946654438972473, -0.09294815361499786, 0.032355792820453644, 0.0047933426685631275, -0.01530291885137558, 0.015547504648566246, -0.05622822046279907, 0.007224789820611477, 0.02795766294002533, 0.0701465755701065, 0.009339137934148312, -0.0031750588677823544, -0.004421220161020756, 0.002610537689179182, 0.01109218318015337, 0.047512128949165344, 0.014251778833568096, 0.06405499577522278, 0.052357278764247894, 0.06949306279420853, -0.021159427240490913, 0.027302365750074387, 0.019867222756147385, -0.028333479538559914, 0.008727345615625381, 0.0023047050926834345, 0.030112091451883316, 0.040812455117702484, -0.00020837580086663365, 0.011449044570326805, -0.020094573497772217, -0.0013654411304742098, -0.08724428713321686, -0.014960666187107563, -0.0012215633178129792, -0.014895695261657238, -0.031034545972943306, 0.015225263312458992, -0.00016314841923303902, -0.002194405999034643, 0.0032538780942559242, -0.044949352741241455, -0.11479921638965607, 0.010216044262051582, -0.007764692418277264, 0.017501315101981163, -0.03738325089216232, 0.005580661818385124, -0.04432961344718933, -0.04300263896584511, -0.01983843930065632, 0.011901640333235264, -0.04016653075814247, 0.04782819375395775, -0.025946354493498802, -0.016691889613866806, -0.048315152525901794, -0.016596542671322823, -0.041720762848854065, 0.03969439119100571, -0.018673015758395195, -0.021220926195383072, 0.08870260417461395, 0.02483464404940605, -0.025807542726397514, 0.047081027179956436, 0.05618199333548546, -0.0350777767598629, 0.011998357251286507, -0.04201826453208923], [-0.02207491733133793, 0.03778044506907463, 0.05336350202560425, -0.03361589461565018, 0.002468990860506892, 0.027896996587514877, 0.026003289967775345, -0.005428783129900694, -0.016957564279437065, -0.04795029014348984, -0.06229796260595322, -0.014259903691709042, -0.0010318858548998833, 0.011248578317463398, 0.027005165815353394, 0.03534344583749771, 0.01788150519132614, 0.02439705654978752, -0.062372852116823196, 0.047951892018318176, 0.02230185829102993, -0.06834352761507034, 0.003398206317797303, 0.0018549676751717925, 0.010791577398777008, -0.04005752503871918, -0.0002574139798525721, 0.04653661698102951, 0.004590219352394342, 0.04957321658730507, 0.005922086536884308, -0.019536972045898438, -0.01211534719914198, -0.012591713108122349, 0.005723780952394009, 0.06017958000302315, 0.0071941944770514965, 0.012439659796655178, -0.015464327298104763, 0.06758785247802734, -0.013859698548913002, -0.0043373797088861465, -0.0369158573448658, 0.007915017195045948, -0.03454575315117836, 0.010468587279319763, 0.027768410742282867, -0.01104956865310669, -0.00638691196218133, -0.022035157307982445, -0.01705924980342388, 0.20848506689071655, 0.032896626740694046, 0.040388114750385284, -0.03933531418442726, 0.08766362071037292, 0.022169072180986404, -0.025142911821603775, 0.01054071169346571, -0.040715258568525314, -0.03248969838023186, 0.010245055891573429, -0.001206941669806838, -0.039311207830905914, -0.013357508927583694, 0.02796732634305954, -0.03187016397714615, -0.05570551007986069, -0.004809004720300436, 0.021295014768838882, 0.03845376893877983, 0.014646654017269611, -0.001630383892916143, 0.022708749398589134, 0.004582671914249659, 0.02137993648648262, -0.0652918666601181, -0.0395941287279129, -0.014228902757167816, -0.06671649217605591, -0.0017247002106159925, 0.02093246579170227, -0.0052572512067854404, 0.02950982004404068, -0.046643149107694626, 0.04140860214829445, 0.0032189758494496346, 0.04572462663054466, -0.003359065158292651, 0.018130000680685043, 0.03991984948515892, -0.01098703220486641, 0.05188944563269615, 0.026425277814269066, 0.02542378567159176, -0.01179222110658884, -0.012286282144486904, 0.011594077572226524, 0.00417662039399147, 0.006853528320789337, 0.030703850090503693, 0.0041516683995723724, 0.03396746888756752, -0.024947315454483032, 0.007740094792097807, 0.007706627249717712, 0.00947894062846899, -0.034848734736442566, 0.0344090573489666, -0.019722310826182365, 0.029335401952266693, -0.010171872563660145, 0.029175635427236557, 0.028995687142014503, 0.027285359799861908, -0.007701994851231575, 0.010543933138251305, -0.01288539357483387, 0.012378843501210213, -0.03168155625462532, -0.015224715694785118, -0.047213152050971985, -0.00396748585626483, -0.04522503539919853, 0.026793822646141052, -0.07722101360559464, 0.0094269635155797, -0.004080881364643574, -0.0485994927585125, 0.05863016098737717, 0.032867785543203354, -0.009131038561463356, 0.04787365347146988, -0.010801647789776325, -0.0018611799459904432, 0.03472526744008064, -0.005347677040845156, 0.006123791914433241, -0.02646017074584961, -0.009888315573334694, -0.0021399841643869877, -0.009398676455020905, -0.012487830594182014, 0.027331756427884102, 0.01892745867371559, -0.012897346168756485, -0.03231188282370567, -0.023036262020468712, -0.0038352191913872957, 0.028259262442588806, 0.054601650685071945, 0.02355547994375229, 0.016202842816710472, -0.03474675491452217, -0.053756535053253174, 0.0017554349033161998, -0.014847191981971264, 0.04132258892059326, -0.0015049796784296632, -0.05708933249115944, 0.011737572029232979, 0.021511370316147804, -0.022968703880906105, -0.024068303406238556, -0.04506200551986694, -0.020023435354232788, 0.034323956817388535, 0.009021919220685959, -0.038626257330179214, -0.06403999030590057, 0.07419984042644501, -0.04622894152998924, -0.026646124199032784, -0.016251064836978912, 0.022245988249778748, 0.04418621584773064, -0.0016347699565812945, 0.026294130831956863, 0.030157655477523804, -0.0040382519364356995, 0.04434291273355484, 0.0013159456430003047, -0.03629732131958008, -0.001288141473196447, -0.014592201448976994, -0.04276461899280548, -0.04383770748972893, 0.009793419390916824, -0.004487662110477686, -0.030724521726369858, 0.06497699022293091, -0.019996387884020805, -0.022440530359745026, -0.015368933789432049, -0.06094437092542648, -0.027196643874049187, 0.0034491769038140774, -0.04752105847001076, -0.030325541272759438, -0.021428614854812622, 0.015458188951015472, 0.005752956494688988, 0.028113575652241707, 0.03789160028100014, 0.000473115302156657, 0.012583620846271515, -0.0097042266279459, 0.004618683364242315, -0.02252296172082424, 0.0360976904630661, 0.036464329808950424, 0.010585528798401356, 0.016726069152355194, 0.006265947129577398, 0.016680827364325523, -0.021620921790599823, -0.031224695965647697, -0.051004476845264435, -0.023498035967350006, -0.005477783735841513, 0.011568550951778889, -0.05255364626646042, -0.036502983421087265, 0.0005367466364987195, -0.04316350817680359, 0.009071946144104004, 0.00034769889316521585, -0.01178091298788786, -0.009710914455354214, -0.0034064583014696836, -0.018518561497330666, 0.06630069762468338, 0.011054864153265953, 0.018105516210198402, 0.02055262215435505, -0.012685884721577168, 0.012223324738442898, -0.04217004030942917, 0.004849003162235022, -0.013120616786181927, 0.035579144954681396, 0.016384605318307877, 0.016383687034249306, -0.028824131935834885, -0.017989950254559517, 0.02348250523209572, -0.02454885095357895, -0.010665985755622387, -0.004243939183652401, -0.0050256154499948025, 0.025696342810988426, 0.029720334336161613, 0.039695240557193756, -0.06231604143977165, -0.02519194595515728, 0.025600260123610497, 0.09142113476991653, 0.017772456631064415, 0.009385521523654461, -0.024798033758997917, -0.017715126276016235, 0.040867459028959274, 0.03737017139792442, 0.06847386807203293, -0.0027985312044620514, 0.022409148514270782, -0.031206810846924782, 0.0201508030295372, -0.0366954579949379, 0.0019309793133288622, -0.06012892350554466, -0.013398698531091213, -0.01606321521103382, -0.04051269590854645, -0.06310489773750305, -0.002650619251653552, -0.026321403682231903, -0.02966621331870556, -0.07654911279678345, -0.02493596449494362, -0.07453393936157227, -0.03212610259652138, 0.03859276324510574, 0.006605707108974457, 0.006960028316825628, -0.01593484915792942, -0.03323248773813248, 0.06376614421606064, 0.01688816025853157, 0.004886123351752758, -0.012549375183880329, 0.06389680504798889, -0.06708607077598572, 0.01545856986194849, 0.39470261335372925, -0.10012330114841461, 0.0018642789218574762, 0.004118379671126604, -0.02784021757543087, -0.009586823172867298, 0.02607211284339428, 0.010695288889110088, -0.05901964008808136, 0.012321258895099163, -0.04739006608724594, 0.03264539688825607, 0.013365928083658218, 0.016690025106072426, 0.023059887811541557, -0.030089663341641426, 0.001957562053576112, -0.03589113429188728, 0.007249740883708, -0.05299646034836769, 0.05849785357713699, -0.01132817380130291, -0.004648979753255844, -0.04353300482034683, -0.0291073489934206, -0.016371507197618484, -0.0028825101908296347, -0.05261363834142685, -0.004356722813099623, -0.010505144484341145, 0.030524691566824913, 0.08655652403831482, -0.013361997902393341, 0.02886621654033661, 0.008542481809854507, -0.056433577090501785, 0.03050469048321247, -0.024536501616239548, -0.008763594552874565, -0.029114123433828354, 0.03392136096954346, -0.004326258786022663, -0.002118803793564439, 0.013132728636264801, -0.024107106029987335, -0.08131666481494904, 0.03956454247236252, 0.014807861298322678, -0.019793977960944176, -0.03980990871787071, 0.03993668779730797, 0.012515767477452755, 0.040620770305395126, -0.03784852847456932, 0.0076788184233009815, -0.0191438477486372, 0.025759173557162285, 0.07868046313524246, 0.01973218098282814, -0.0167477298527956, 0.0034022117033600807, -0.0019534442108124495, -0.02042330428957939, 0.036541156470775604, -0.027041278779506683, 0.012366348877549171, 0.03731478750705719, -0.011837235651910305, -0.02147364430129528, 0.03112337552011013, -0.02646968886256218, -0.0071073235012590885, 0.004210280254483223, 0.03381143882870674, -0.02785472385585308, 0.02456171065568924, 0.03145741671323776, -0.027996625751256943, 0.02449767291545868, -0.0324702113866806, -0.017289182171225548, 0.06499079614877701, 0.061864860355854034, 0.011149565689265728, 0.02039583958685398, 0.01459736842662096, 0.015139142982661724, 0.028243299573659897, 0.021157164126634598, 0.04174615815281868, 0.010688552632927895, -0.027760984376072884, -0.018281104043126106, -0.01708064042031765, -0.0006540568429045379, 0.004663929343223572, 0.022502180188894272, 0.018182959407567978, -0.012086146511137486, 0.05076342076063156, -0.004485734738409519, 0.009212564677000046, 0.002076652366667986, 0.020895812660455704, 0.02193998359143734, -0.0014234422706067562, -0.025451896712183952, -0.03971229866147041, 0.000715890375431627, 0.017395231872797012, -0.026180889457464218, -0.009217442944645882, -0.02107289619743824, 0.009843810461461544, -0.007275546435266733, 0.03906052187085152, -0.02519945614039898, 0.017678508535027504, -0.041344549506902695, -0.02785404585301876, 0.02297930419445038, -0.044478364288806915, -0.021625874564051628, 0.012522729113698006, 0.018852433189749718, -0.044466592371463776, -0.03000548854470253, 0.009715897962450981, 0.0067186676897108555, -0.01800091750919819, -0.0085225198417902, -0.019547341391444206, 0.06725037842988968, 0.0393124520778656, 0.010921712964773178, 0.022281501442193985, 0.003066515550017357, 0.053152911365032196, -0.02817712351679802, -0.02877512387931347, -0.011324271559715271, -0.04257098585367203, -0.010403760708868504, 0.02024124376475811, -0.0029016691260039806, 0.028941333293914795, -0.0641075149178505, -0.0140945790335536, -0.02559632621705532, 0.03449434041976929, 0.007204963825643063, -0.0239377710968256, 0.04178916662931442, -0.015921615064144135, 0.010400461964309216, 0.05862080678343773, -0.05625854432582855, -0.06288734078407288, -0.05512905865907669, 0.02441575564444065, -0.022523414343595505, -0.005805666092783213, 0.03998985514044762, -0.011696111410856247, 0.020753296092152596, -0.017774270847439766, 0.04288255795836449, 0.019219066947698593, 0.004738088231533766, -0.022671010345220566, -0.04114781692624092, 0.007490412797778845, 0.028950273990631104, -0.017018893733620644, -0.018351197242736816, -0.013524849899113178, -0.05824903026223183, -0.006283402442932129, -0.04216734692454338, -0.016898619011044502, -0.008255058899521828, 0.0038432155270129442, 0.03182315453886986, -0.015543750487267971, 0.006290205288678408, -0.0036525705363601446, 0.009470502845942974, -0.017629414796829224, -0.046813953667879105, -0.0942268967628479, -0.018174629658460617, -0.0042082685977220535, -0.020197732374072075, 0.05282061547040939, -0.012541069649159908, -0.02347211353480816, -0.053994521498680115, -0.043725449591875076, -0.004363293293863535, -0.020149771124124527, 0.0439775325357914, -0.011693726293742657, -0.036251045763492584, 0.03463350236415863, -0.01638956367969513, 0.010586036369204521, 0.10154715180397034, 0.0157792866230011, -0.006909392774105072, -0.06592267006635666, 0.0014194174436852336, 0.023354213684797287, 0.004048009868711233, -0.01851869933307171, 0.0018057667184621096, -0.008427313528954983, -0.04657742381095886, -0.006614028476178646, 0.03154650703072548, 0.05784088745713234, -0.0014024899573996663, -0.03249821439385414, 0.01812722161412239, 0.01049849297851324, -0.0066998242400586605, 0.022871872410178185, 0.042771127074956894, -0.0390176959335804, -0.03194645047187805, 0.055527638643980026, 0.05875362083315849, 0.016704954206943512, -0.00437917048111558, -0.007910593412816525, 0.01369831059128046, -0.05270715430378914, -0.040367212146520615, -0.0349927581846714, 0.013544845394790173, -0.05652346834540367, -0.01758694462478161, 0.012384727597236633, 0.0382315069437027, -0.035357765853405, 0.029169771820306778, 0.05459509417414665, -0.019738292321562767, -0.03362181782722473, 0.0019446539226919413, 0.06532846391201019, 0.05612233653664589, 0.03679762780666351, -0.02837015502154827, -0.01910312846302986, 0.001341912429779768, -0.006694074720144272, -0.01882043667137623, 0.014384809881448746, 0.007774887140840292, 0.024533918127417564, -0.017248252406716347, -0.018278736621141434, 0.005174399819225073, -0.029166843742132187, 0.041395992040634155, -0.013741761445999146, -0.032561734318733215, -0.00182226428296417, -0.07384144514799118, 0.03130773827433586, -0.024418286979198456, 0.004673963412642479, 0.05467233806848526, -0.016443245112895966, 0.0397089421749115, -0.014201744459569454, -0.008252243511378765, -0.016673723235726357, 0.0441608652472496, 0.006296293810009956, -0.013724514283239841, -0.005448297597467899, 0.01947759836912155, -0.021308409050107002, -0.022401422262191772, -0.09539996832609177, -0.015610778704285622, -0.040673770010471344, 0.013501860201358795, -0.013823563233017921, -0.0008030296303331852, -0.07542983442544937, -0.02400965988636017, 0.0001011365675367415, -0.001541225123219192, -0.0661478117108345, 0.02518303319811821, -0.04633980989456177, -0.013657496310770512, 0.026259463280439377, -0.0008092244970612228, 0.01791374385356903, 0.03202012553811073, 0.03443008288741112, 0.0435992032289505, 0.0067512341775000095, 0.0335320308804512, 0.01668708212673664, 0.0422564335167408, 0.04263998568058014, -0.06267701834440231, -0.0661742091178894, 0.01278718188405037, -0.01789390668272972, -0.02553742751479149, -0.021107936277985573, 0.05001327767968178, -0.014651501551270485, -0.01830042153596878, 0.01155078038573265, -0.06255708634853363, -0.052775025367736816, -0.012125776149332523, -0.008270006626844406, 0.016420505940914154, -0.038623273372650146, 0.045220885425806046, -0.06518010795116425, -0.006855912506580353, -0.005604216363281012, -0.04686413332819939, 0.022722294554114342, 0.03993123024702072, -0.016082607209682465, -0.04441791772842407, -0.021101905032992363, 0.08099529892206192, 0.03494594991207123, 0.009946553967893124, 0.02383871003985405, 0.025567656382918358, -0.003573312656953931, 0.02477131225168705, 0.007256696000695229, -0.008861272595822811, 0.031587596982717514, -0.032208383083343506, -0.048624251037836075, -0.04166914150118828, 0.028414029628038406, 0.01888701133430004, 0.02432314306497574, -0.03653004392981529, 0.016341552138328552, 7.16942158760503e-05, -0.03340870887041092, -0.04498599097132683, 0.004731855820864439, 0.007166999392211437, -0.04135027900338173, -0.002260846784338355, 0.05968862771987915, 0.033039554953575134, -0.006835693959146738, -0.04946252703666687, -0.05390949547290802, -0.006569209508597851, 0.0015997191658243537, 0.011768410913646221, -0.006696969270706177, -0.032714761793613434, -0.008493042550981045, 0.010132125578820705, 0.007456188090145588, -0.03190124034881592, -0.003725589718669653, -0.00489306915551424, -0.02960209921002388, 0.02178342454135418, -0.04333103448152542, 0.03454500436782837, -0.0041511170566082, -0.022286495193839073, 0.0426362082362175, -0.003741385880857706, -0.012123474851250648, -0.040574777871370316, -0.020819755271077156, 0.03362612798810005, -0.009696024470031261, -0.004942260682582855, -0.00958671048283577, -0.03161589801311493, 0.02238604426383972, 0.0005395559128373861, 0.0006194398156367242, -0.008026261813938618, 0.01823294162750244, -0.058335352689027786, 0.028065450489521027, -0.0008821625960990787, 0.09991125762462616, -0.016953835263848305, 0.0065302071161568165, -0.009572567418217659, -0.06717424094676971, 0.060852956026792526, 0.026521284133195877, -0.01454852893948555, -0.005965926218777895, -0.054464150220155716, 0.04267401993274689, 0.019806182011961937, 0.06527900695800781, -0.014627750962972641, -0.02351476065814495, -0.0016488154651597142, 0.005583044141530991, 0.021098939701914787, 0.01203303411602974, -0.017055246978998184, 0.04668997600674629, 0.0326644703745842, 0.06362315267324448, -0.004076390992850065, 0.0073812659829854965, 0.05780899152159691, -0.0307595357298851, 0.013506025075912476, -0.003513190895318985, 0.023115765303373337, 0.04403551295399666, -0.009666514582931995, -0.02497272565960884, -0.032489266246557236, -0.02066945657134056, -0.09700801968574524, -0.017656391486525536, -0.02180369757115841, -0.013548699207603931, -0.02152223326265812, 0.0010412686970084906, 0.01314027514308691, 0.00042609713273122907, -0.0038540347013622522, -0.03388579562306404, -0.10281910747289658, -6.769334959244588e-06, 0.008923184126615524, -0.02149849385023117, -0.011358559131622314, 0.0019899692852050066, -0.039379749447107315, -0.006535376887768507, -0.031905971467494965, 0.0005600182339549065, -0.06199656426906586, 0.033538952469825745, -0.044939614832401276, -0.00030303833773359656, -0.022106096148490906, -0.02056300640106201, -0.030442794784903526, 0.012250794097781181, -0.006099621765315533, -0.005566449370235205, 0.06313791126012802, 0.007724859286099672, -0.024542132392525673, 0.033498745411634445, 0.08094602078199387, -0.028834909200668335, 0.03459099680185318, -0.011076862923800945], [-0.023342592641711235, 0.007013416383415461, 0.027393873780965805, -0.025322826579213142, -0.0009632112924009562, 0.021295225247740746, -0.0003774894867092371, 0.01408209279179573, -0.021387580782175064, -0.03825082629919052, -0.09310249239206314, -0.019415343180298805, -0.001430354081094265, 0.008030116558074951, 0.012652484700083733, 0.045589156448841095, 0.051136601716279984, 0.03176643326878548, -0.050397537648677826, 0.011198483407497406, 0.021018752828240395, -0.0583813339471817, 0.015377257019281387, 0.008842463605105877, -0.006264046300202608, -0.008657565340399742, -0.00834862794727087, 0.05477476865053177, -0.0063384114764630795, 0.02828524075448513, -0.03295772522687912, 0.03217929229140282, -0.014275730587542057, -0.028731096535921097, -0.028937989845871925, 0.08925289660692215, -0.000388441258110106, -0.0048840888775885105, 0.0185807216912508, 0.03503111004829407, 0.014989938586950302, -0.0031155843753367662, -0.0007197422091849148, -0.01042104884982109, -0.011289982125163078, -0.019817057996988297, 0.04004472866654396, -0.010894740000367165, 0.02226829156279564, -0.015550014562904835, -0.01945984549820423, 0.20488940179347992, 0.05724667012691498, 0.038182422518730164, -0.017888901755213737, 0.09432511031627655, 0.009773203171789646, -0.02608608454465866, -0.0006776658701710403, -0.027757320553064346, -0.02478284388780594, -0.0008648633374832571, -0.0036730635911226273, -0.054907526820898056, -0.018364261835813522, 0.005123812239617109, -0.004961858503520489, -0.05265474691987038, 0.012777877040207386, 0.00818374752998352, 0.0598311610519886, 0.01730930432677269, 0.006763438228517771, 0.016579436138272285, 0.021944861859083176, 0.05928075686097145, -0.06466300785541534, -0.020114196464419365, -0.008554095402359962, -0.029761822894215584, -0.03461482748389244, 0.031763263046741486, 0.0035614660009741783, 0.027643660083413124, -0.0399644672870636, 0.071408212184906, 0.014892191626131535, 0.057960014790296555, -0.03769015520811081, 0.03632074221968651, 0.029044900089502335, -0.005262965802103281, 0.021128514781594276, 0.008084145374596119, 0.02248525433242321, 0.011073153465986252, 0.005998824257403612, -0.002987213432788849, 0.00684404605999589, -0.02827177755534649, 0.051327839493751526, 0.005745724309235811, -0.004666957538574934, -0.0062580290250480175, -0.0105279590934515, 0.012757920660078526, -0.019742295145988464, -0.04217785224318504, 0.05463411286473274, 0.006520386319607496, 0.02055993303656578, -0.008166166953742504, -0.0012474197428673506, 0.020167581737041473, 0.028866704553365707, 0.0067550139501690865, -0.009367414750158787, -0.011950409039855003, 0.02609972655773163, -0.0328768827021122, -0.024806201457977295, -0.04093864932656288, -0.006843608804047108, -0.03290662169456482, 0.006189201958477497, -0.06285793334245682, 0.007029958534985781, 0.006847623270004988, -0.0396309420466423, 0.026718029752373695, 0.037715405225753784, -0.0265506561845541, 0.025436824187636375, 0.01662714034318924, -0.009165126830339432, 0.05429975688457489, 0.022310832515358925, -0.011238888837397099, -0.015356856398284435, -0.006027951370924711, -0.0004169843450654298, 0.0058736652135849, 0.002141556004062295, 0.03076731227338314, 0.013421920128166676, 0.005148066207766533, -0.024622716009616852, -0.00949941948056221, -0.0034224395640194416, 0.019515443593263626, 0.035743702203035355, 0.014057036489248276, 0.034075140953063965, -0.05624675750732422, -0.052110109478235245, 0.005043579265475273, -0.013995928689837456, 0.055275339633226395, 0.0033951785881072283, -0.0773654580116272, 0.013317358680069447, -0.03388263285160065, 0.015920350328087807, -0.006089163012802601, -0.037784699350595474, -0.03822176158428192, -0.00974674429744482, -0.0019177787471562624, -0.04152245819568634, -0.0401533767580986, 0.04919186607003212, -0.04335799068212509, -0.03582534193992615, -0.024076128378510475, 0.007045007776468992, 0.029925169423222542, 0.0005198677536100149, 0.0650731697678566, 0.03627486899495125, -0.00114472268614918, 0.01618455909192562, -0.024307366460561752, -0.040283218026161194, -0.013159886933863163, -0.03619116172194481, -0.049261752516031265, -0.029680857434868813, 0.00200100801885128, -0.0074235862120985985, -0.01757143810391426, 0.05484959855675697, -0.007444824557751417, -0.015165772289037704, -0.039190344512462616, -0.08751221001148224, -0.02545873261988163, 0.01714591681957245, -0.05041496455669403, -0.023561136797070503, 0.0316777229309082, -0.009458166547119617, -0.017833873629570007, 0.014171152375638485, 0.04846244677901268, -1.0380742423876654e-05, -0.0013976157642900944, -0.03357882797718048, -0.02815914899110794, -0.009509935975074768, 0.0360025130212307, 0.034569159150123596, 0.014959141612052917, 0.027194460853934288, -0.010182186029851437, 0.005329624749720097, -0.013305299915373325, -0.035324983298778534, -0.04524186626076698, -0.002823358867317438, -0.005973357707262039, -0.0012358496896922588, -0.08206862211227417, -0.03180666267871857, -0.013991214334964752, -0.010387123562395573, 0.03645221143960953, 0.009847905486822128, -0.018145782873034477, -0.026061004027724266, -0.009087440557777882, 3.6712208384415135e-05, 0.05315051227807999, -0.031968314200639725, 0.024986354634165764, 0.049045782536268234, -0.0579817108809948, -2.5399902369827032e-05, -0.015701932832598686, -0.005319100804626942, -0.028402922675013542, 0.03460171818733215, 0.021664408966898918, -0.02097693271934986, -0.02154487743973732, 0.0019962668884545565, -0.009222944267094135, -0.04061298072338104, -0.014847780577838421, -0.000930743757635355, 0.009180858731269836, -0.027579281479120255, 0.04696192219853401, 0.029733646661043167, -0.022501694038510323, -0.013740264810621738, 0.02545863203704357, 0.10661831498146057, 0.03286931663751602, 0.005613658111542463, -0.024800699204206467, -0.036716099828481674, 0.03844746947288513, 0.030075322836637497, 0.06294988095760345, -0.008444328792393208, 0.010682273656129837, -0.029131650924682617, 0.023276280611753464, -0.0380166694521904, 0.019148603081703186, -0.05094386637210846, -0.007699369452893734, -0.027421457692980766, -0.03084413893520832, -0.05619172006845474, 9.471345401834697e-05, -0.027366120368242264, -0.020334724336862564, -0.04760701581835747, -0.023844480514526367, -0.03747493401169777, 0.004425837192684412, 0.028000224381685257, -0.0049100820906460285, 0.00859822053462267, -0.005408886354416609, -0.029228225350379944, 0.04838107153773308, 0.020861541852355003, -0.004578001331537962, -0.03820386156439781, 0.049680545926094055, -0.035738710314035416, 0.027500225231051445, 0.40454572439193726, -0.06743281334638596, 0.005369241815060377, 0.017086593434214592, -0.020004844292998314, -0.03772980719804764, 0.01843937113881111, -0.00044122885446995497, -0.025233477354049683, 0.02987697906792164, -0.052247945219278336, 0.06421364843845367, -0.01635591685771942, -0.008257829584181309, -0.01061681192368269, -0.009429681114852428, -0.03380217030644417, -0.0438753180205822, 0.0010634089121595025, -0.03499794751405716, 0.07047475874423981, -0.048672281205654144, -0.01892785355448723, -0.05178334191441536, -0.020962033420801163, -0.021438611671328545, 0.011140033602714539, -0.053412262350320816, -0.028329383581876755, -0.021380649879574776, 0.044147420674562454, 0.08466407656669617, -0.0007514433818869293, 0.04553569108247757, 0.0024970120284706354, -0.0478362999856472, 0.0390603132545948, -0.0287262462079525, -0.018067244440317154, -0.0073635815642774105, 0.028316011652350426, -0.01818900741636753, 0.010830157436430454, 0.007673241663724184, 0.009236108511686325, -0.07869352400302887, 0.015517175197601318, 0.010878280736505985, -0.04460287094116211, -0.047239720821380615, 0.02293470688164234, 0.019256796687841415, 0.06493595242500305, -0.04809468612074852, -0.013965046964585781, -0.02638372778892517, -0.011120131239295006, 0.03183924779295921, 0.02888081967830658, -0.014285621233284473, 0.014296275563538074, 0.01076508592814207, -0.010308882221579552, 0.036496296525001526, -0.02413790673017502, 0.0030628941021859646, 0.025240549817681313, -0.014733785763382912, -0.0013085505925118923, 0.03043527901172638, -0.04504597559571266, 0.0001958450593519956, 0.021459074690937996, 0.012703314423561096, -0.030460070818662643, -0.004545321688055992, 0.028115272521972656, -0.039494290947914124, 0.006516746710985899, -0.039730630815029144, 0.005202922970056534, 0.03431972116231918, 0.02459535375237465, 0.0018447687616571784, 0.018329579383134842, 0.020559098571538925, 0.0052847908809781075, 0.05272165313363075, 0.022180570289492607, 0.02678285352885723, 0.013466224074363708, -0.017942078411579132, -0.0018671414582058787, -0.04888424277305603, 0.01436272356659174, -0.011048056185245514, 0.026297971606254578, 0.003988965414464474, -0.03515666723251343, 0.060038164258003235, 0.03480510786175728, 0.005004203412681818, 0.021613068878650665, 0.022906027734279633, 0.012909836135804653, -0.028874335810542107, -0.023812443017959595, -0.0353168286383152, 0.03022785484790802, 0.013275383971631527, -0.022232316434383392, 0.007522741332650185, -0.04195510223507881, 0.0028231500182300806, -0.03108767233788967, 0.027378862723708153, 0.004333457909524441, 0.02101881615817547, -0.01471082866191864, -0.02127256616950035, 0.048922017216682434, -0.02813688851892948, -0.01719437539577484, 0.02463357150554657, 0.02759389951825142, -0.026123786345124245, -0.029388761147856712, 0.01567338965833187, 0.0037520022597163916, -0.007139828521758318, -0.014704444445669651, -0.018358014523983, 0.07306188344955444, 0.018202241510152817, -0.00423047412186861, 0.017220616340637207, -0.017664479091763496, 0.06353019177913666, -0.029075870290398598, -0.007595530711114407, -0.016186902299523354, -0.060011573135852814, -0.021327750757336617, -0.006844673305749893, 0.000796337379142642, 0.011150325648486614, -0.043480534106492996, -0.04099748656153679, -0.017589498311281204, 0.018843213096261024, 0.017052946612238884, -0.004290774464607239, 0.06991376727819443, -0.006496611982584, 0.017961794510483742, 0.058601319789886475, -0.020611513406038284, -0.07075408101081848, -0.06659995764493942, 0.040957409888505936, -0.003751591546460986, 0.012993922457098961, 0.01852942630648613, -0.015976492315530777, 0.002100481418892741, 0.012602192349731922, 0.06416193395853043, 0.00032084985286928713, -0.004002980887889862, 0.003084268420934677, -0.04264543950557709, -0.00278267590329051, 0.019906247034668922, 0.00746852857992053, -0.008974872529506683, -0.016039563342928886, -0.04313160479068756, -0.013655846938490868, -0.019922783598303795, -0.012674327939748764, 0.007727768737822771, 0.008632014505565166, 0.012023872695863247, 0.020086880773305893, 0.02944258041679859, 0.03024853579699993, 0.03190210089087486, 0.0002513383515179157, -0.004375105258077383, -0.08607181161642075, -0.03752442076802254, -0.0008081201231107116, -0.02668849378824234, -0.00018148813978768885, -0.03149570897221565, -0.026358138769865036, -0.05316959694027901, -0.03595536947250366, 0.008531573228538036, -0.00922729354351759, 0.011013603769242764, -0.0009632249712012708, -0.04433956369757652, 0.02774270437657833, -0.026103945448994637, 0.027209190651774406, 0.1152576208114624, 0.024318214505910873, 0.003719550557434559, -0.06428418308496475, -0.013571267947554588, 0.027029795572161674, -0.005722956731915474, -0.027012169361114502, 0.008780092932283878, -0.006448938045650721, -0.031829312443733215, 0.0009138997993431985, 0.05300162732601166, 0.06939037889242172, 0.008863729424774647, -0.0134451137855649, 0.009257769212126732, 0.006813162937760353, -0.012271597050130367, -0.004395310766994953, 0.03557165712118149, -0.023080559447407722, -0.0011972117936238647, 0.039863716810941696, 0.04363483190536499, 0.0034469820093363523, -0.03296348452568054, 0.015285640023648739, 0.051262520253658295, -0.03837429732084274, -0.006385263055562973, -0.0384291335940361, -0.010856101289391518, -0.054713521152734756, -0.036893948912620544, 0.011653912253677845, 0.010515544563531876, -0.06128525361418724, 0.05008038878440857, 0.05866873636841774, -0.01855696737766266, -0.031395938247442245, 0.016563721001148224, 0.05768561363220215, 0.07642056792974472, 0.03320017829537392, -0.022548627108335495, -0.02289537899196148, 0.004701513797044754, -0.004350673872977495, -0.024684134870767593, 0.02583448216319084, 0.03753281384706497, 0.008984374813735485, -0.02371763065457344, -0.011609119363129139, -0.013692825101315975, -0.035926710814237595, 0.01956804469227791, -0.013166731223464012, -0.06687303632497787, -0.0007655133376829326, -0.08027803152799606, 0.026432102546095848, -0.006236232817173004, 0.016829784959554672, 0.05265679582953453, -0.043141257017850876, 0.005428549367934465, -0.01920582354068756, -0.018157009035348892, 0.0015852832002565265, 0.044450994580984116, 0.005447115749120712, -0.004440802149474621, -0.020816512405872345, 0.01753278076648712, -0.029646314680576324, -0.014972598291933537, -0.04083512723445892, -0.054542433470487595, -0.054506175220012665, 0.032616570591926575, -0.0014112290227785707, -0.026051675900816917, -0.06382809579372406, -0.0407579131424427, -0.010855238884687424, -0.013603306375443935, -0.052746210247278214, 0.04387801140546799, -0.025739893317222595, 0.013029634952545166, 0.019862443208694458, -0.027810221537947655, 0.02864898554980755, 0.05475933849811554, 0.021500008180737495, 0.007142761256545782, 0.018978560343384743, 0.02114521525800228, 0.02750871144235134, 0.036193110048770905, 0.03920804709196091, -0.025676777586340904, -0.039714135229587555, 0.02947889268398285, -0.01709700934588909, -0.02014109492301941, -0.02028128318488598, 0.005540527869015932, -0.01926696114242077, -0.009291601367294788, 0.011965581215918064, -0.059187766164541245, -0.04154679551720619, -0.022761372849345207, -0.021753037348389626, -0.013049209490418434, -0.048598773777484894, 0.00499733118340373, -0.03574066609144211, -0.007250492461025715, -0.003967720083892345, -0.04356333613395691, -0.02480950392782688, 0.019712820649147034, -0.011973143555223942, -0.020474182441830635, -0.01788371615111828, 0.040125057101249695, 0.03940027952194214, -0.0041338615119457245, 0.019210459664463997, -0.004408307373523712, -0.008727014996111393, 0.024138610810041428, 0.018637174740433693, -0.004306972026824951, 0.025016797706484795, -0.03963819891214371, -0.058288875967264175, -0.00692760618403554, 0.025919202715158463, 0.03828740492463112, -0.018109003081917763, -0.0464884489774704, 0.005967595614492893, 0.007819056510925293, -0.03422554209828377, -0.04507263004779816, -0.002861080225557089, 0.03101721964776516, -0.05211510509252548, -0.023761063814163208, 0.04711730778217316, 0.04601084068417549, -0.01517556793987751, -0.07930450886487961, -0.051632098853588104, -0.004972491879016161, 0.014678765088319778, -0.014211371541023254, -0.003487646346911788, -0.015713682398200035, 0.006794071290642023, 0.03228191286325455, -0.005345900543034077, -0.021408524364233017, 0.04213220998644829, -0.011053439229726791, -0.02418520860373974, 0.020172417163848877, -0.06537989526987076, 0.08079174906015396, -0.011286843568086624, -0.031215686351060867, 0.03723786026239395, -0.020485470071434975, -0.03330402821302414, -0.04726647585630417, -0.00243766768835485, 0.027441542595624924, -0.027571963146328926, 0.02647782862186432, 0.010702797211706638, -0.022940024733543396, 0.008650693111121655, -0.010530294850468636, 0.0031114378944039345, 0.014120659790933132, 0.01488540694117546, -0.05457695573568344, 0.04795045405626297, -0.024149522185325623, 0.10835808515548706, -0.0007454889127984643, 0.018032854422926903, -0.018652480095624924, -0.0689639002084732, 0.06809703260660172, 0.04885035753250122, -0.02889508195221424, 0.03348459675908089, -0.0755779966711998, 0.008918101899325848, -0.01324958074837923, 0.08009111136198044, -0.01094079203903675, 0.0038655984681099653, -0.018296265974640846, 0.03358132392168045, 0.027006477117538452, 0.011021296493709087, -0.017706258222460747, 0.03995724394917488, 0.060644667595624924, 0.02347799576818943, 0.015232779085636139, 0.024048693478107452, 0.045468926429748535, -0.03139152005314827, 0.014673003926873207, -0.042890727519989014, 0.007949761115014553, 0.04907204210758209, 0.009554613381624222, 0.005997351836413145, -0.008964002132415771, 0.016308346763253212, -0.058633483946323395, -0.0016596348723396659, -0.014519785530865192, -0.03986898064613342, -0.005677760578691959, -0.005391860380768776, 0.018064992502331734, -0.04929664731025696, -0.008411256596446037, -0.004248903598636389, -0.11780498176813126, -0.02626657672226429, 0.012060622684657574, -0.018589848652482033, -0.012870753183960915, -0.02147584594786167, -0.03643050044775009, -0.0004998242948204279, -0.03599150478839874, 0.0057571628130972385, -0.03410540521144867, 0.02942795120179653, -0.02904071845114231, -0.013356409035623074, -0.0022076331079006195, -0.002636006334796548, -0.029878435656428337, 0.01429714635014534, -0.00570177985355258, 0.02163563296198845, 0.04853642359375954, 0.02664809860289097, -0.03283120319247246, 0.01719130016863346, 0.058850083500146866, 0.00433034123852849, 0.05594922602176666, -0.02542933076620102], [-0.009385433048009872, -0.031895484775304794, 0.015606148168444633, -0.006414168979972601, -0.007167282979935408, 0.012977637350559235, 0.009058017283678055, 0.006793965119868517, -0.014825219288468361, -0.044255875051021576, -0.03326670452952385, 0.011042717844247818, 0.0026368703693151474, -0.02020403929054737, 0.04914598912000656, 0.03216332942247391, 0.03405872732400894, 0.04788804426789284, -0.05354173481464386, 0.05740915238857269, -0.010648961178958416, -0.07718484103679657, 0.06793942302465439, -0.01801823265850544, -0.02325505204498768, 0.04043234884738922, -0.024098265916109085, 0.05512746796011925, 0.006350943353027105, -0.0038313406985253096, -0.03706395998597145, 0.02117396891117096, -0.017025191336870193, -0.03984220698475838, -0.03328005224466324, 0.05232639238238335, -0.035069674253463745, -0.01626298390328884, 0.01896338164806366, 0.003934050910174847, 0.0019917727913707495, -0.02162342146039009, 0.033954426646232605, 0.004106140695512295, -0.042914073914289474, 0.03835710510611534, 0.04880663380026817, -0.015628129243850708, 0.013645772822201252, 0.006961246952414513, 0.009488414041697979, 0.16409516334533691, 0.04174218326807022, 0.06424257159233093, -0.013708017766475677, 0.1138298362493515, 0.003272967180237174, -0.01801358163356781, 0.01667811907827854, -0.0516299270093441, -0.0016883310163393617, 0.0031262938864529133, -0.013346493244171143, -0.0464952290058136, 0.021413462236523628, -0.005694909952580929, -0.010792653076350689, -0.0669364258646965, 0.04018435254693031, -0.04435773938894272, 0.0371147021651268, 0.027419643476605415, 0.014386673457920551, 0.013805435039103031, 0.061294522136449814, 0.029899703338742256, -0.04294212907552719, 0.021280141547322273, 0.00454628374427557, 0.02424563094973564, -0.011971714906394482, -0.026697449386119843, 0.027105635032057762, 0.02518565207719803, -0.05211005359888077, 0.07245708256959915, -0.006518659647554159, 0.046133216470479965, -0.024088790640234947, 0.008156169205904007, 0.04671693593263626, -0.01797487773001194, -0.01744777336716652, 0.01845431700348854, -0.00463033327832818, 0.03269725292921066, -0.01962151564657688, -0.0275893434882164, 0.0005605531041510403, -0.046039946377277374, 0.05064809322357178, -0.0006307732546702027, -0.006656522862613201, -0.017849894240498543, 0.0070138308219611645, 0.03325529769062996, -0.015389724634587765, -0.06591767817735672, 0.05996629223227501, -0.00046281443792395294, 0.008433190174400806, -0.06318878382444382, -0.016385508701205254, 0.043691325932741165, 0.013081633485853672, 0.01912485808134079, 0.013193130493164062, -0.038436565548181534, 0.059645384550094604, -0.061083417385816574, -0.012701901607215405, -0.057643868029117584, -0.02282639592885971, -0.01316430326551199, 0.018500888720154762, -0.08032620698213577, 0.023269368335604668, -0.004052196629345417, -0.022861246019601822, -0.02670195885002613, 0.006597238127142191, -0.006032612640410662, 0.018373392522335052, 0.02733275108039379, 0.0172661654651165, 0.07762926816940308, -0.00978172942996025, -0.03541647642850876, -0.01327283401042223, 0.003952879458665848, 0.01455068401992321, 0.010310359299182892, 0.0018765629502013326, 0.0022073828149586916, 0.04581385850906372, -0.02755122259259224, -0.026740847155451775, 0.022110281512141228, -0.0016539681237190962, -0.0031524773221462965, 0.006758092436939478, -0.00283719296567142, 0.01654389500617981, -0.016901452094316483, -0.013328367844223976, 0.0033859608229249716, -0.07463372498750687, 0.03123895637691021, -0.009237646125257015, -0.06868388503789902, 0.01771768555045128, -0.05675898492336273, -0.006365705747157335, 0.004333234392106533, -0.03679031506180763, -0.05274758115410805, -0.04240768030285835, 0.01471983827650547, -0.03180724382400513, -0.04740941897034645, 0.041697241365909576, -0.025262484326958656, -0.01480739563703537, 0.02323201857507229, 0.05089886859059334, 0.03801461309194565, -0.016876503825187683, 0.03858273848891258, 0.0006896965787746012, 0.019274933263659477, -0.0076745799742639065, -0.0487939827144146, -0.04986603930592537, 0.003989866469055414, -0.034648798406124115, -0.014794979244470596, 0.020166024565696716, -0.014057276770472527, -0.005456030368804932, -0.020668817684054375, 0.05881757661700249, -0.007328493520617485, -0.019732331857085228, -0.026936179026961327, -0.06849357485771179, -0.0327034667134285, 0.024702375754714012, -0.03443653881549835, -0.03576183319091797, 0.05486930161714554, 0.0005992587539367378, 0.021276473999023438, 0.040731608867645264, 0.03385654464364052, -0.019452789798378944, -0.02637266181409359, 0.0010131816379725933, -0.013934374786913395, -0.01850920543074608, 0.012401657178997993, -0.00640078354626894, 0.04712798446416855, 0.0060154227539896965, -0.06434684246778488, 0.013726743869483471, -0.0074227782897651196, -0.02798437513411045, -0.0696135088801384, 0.00854842271655798, -0.011579617857933044, 0.02918410301208496, -0.07572190463542938, 0.008197557181119919, -0.004353250376880169, 0.014265590347349644, 0.0075424970127642155, -0.014869359321892262, -0.025663256645202637, -0.03850497305393219, -0.0015709972940385342, 0.00023904943373054266, 0.05648203194141388, -0.004621682222932577, 0.013462466187775135, 0.04249603673815727, -0.02065880037844181, -0.046151302754879, 0.011507905088365078, 0.012960562482476234, -0.019762704148888588, 0.018986405804753304, -0.011258983053267002, 0.008242007344961166, -0.0476059652864933, 0.005903609562665224, -0.07080543041229248, -0.047890570014715195, -0.025125255808234215, -0.010575452819466591, 0.013671823777258396, -0.008452685549855232, 0.021029382944107056, 0.03688666224479675, -0.029176564887166023, -0.040985073894262314, -0.014099684543907642, 0.08349942415952682, 0.02599073015153408, 0.04556603729724884, 0.007350611500442028, -0.018901502713561058, 0.0030944140162318945, 0.029242096468806267, 0.028508976101875305, -0.007765810005366802, 0.0033983595203608274, -0.013031129725277424, -0.015118412673473358, -0.017029844224452972, 0.05791420862078667, -0.028992919251322746, 0.008383085951209068, -0.015324181877076626, -0.036236248910427094, -0.01395439449697733, -0.023564070463180542, -0.012455174699425697, 0.012977756559848785, -0.022287681698799133, -0.03491206467151642, -0.016955459490418434, 0.03288937360048294, 0.02685566060245037, 0.017927587032318115, 0.02297133579850197, -0.004394894931465387, -0.02364477328956127, 0.019689859822392464, 0.0243887510150671, -0.030914323404431343, -0.005117141641676426, -0.013491074554622173, -0.007365900557488203, 0.05817515403032303, 0.39855092763900757, -0.05282498523592949, 0.03726173937320709, 0.003239869140088558, -0.012263084761798382, -0.02486310712993145, -0.006434249226003885, -0.028087474405765533, 0.007046166807413101, 0.011295506730675697, -0.08665433526039124, 0.04711233451962471, -0.03940262272953987, 0.01622302085161209, 0.0022135847248136997, -0.016504602506756783, -0.020977120846509933, -0.022591866552829742, -0.01722598820924759, -0.04725146293640137, 0.08634680509567261, -0.05859073996543884, -0.02936885505914688, -0.03654032573103905, -0.03470482677221298, -0.032555341720581055, 0.01840108260512352, -0.033596549183130264, -0.017311468720436096, -0.006452989298850298, 0.06142296642065048, 0.062044400721788406, 0.01189848780632019, 0.034520529210567474, -0.03014340065419674, -0.05545012280344963, 0.05345242843031883, -0.025555897504091263, -0.013693553395569324, -0.02162517048418522, 0.012636694125831127, -0.07438362389802933, -0.035887461155653, -0.0023170760832726955, 0.019988199695944786, -0.059163425117731094, 0.021453477442264557, 0.0039044851437211037, -0.05351272597908974, -0.011921731755137444, 0.0036852273624390364, -0.003596261143684387, 0.01355405617505312, -0.01580457203090191, -0.005126520525664091, -6.305815622908995e-05, -0.03365243971347809, 0.02200702205300331, 0.002355950651690364, -0.01325371116399765, 0.015482054091989994, 0.009901641868054867, 0.03876335918903351, 0.025324944406747818, -0.01768738217651844, -0.004138321615755558, 0.031606461852788925, -0.013897976838052273, -0.014268888160586357, -0.0015250417636707425, -0.0325094610452652, -0.02574855461716652, 0.010574962012469769, 0.012250884436070919, -0.03247907757759094, -0.024527309462428093, 0.015249338001012802, -0.016512306407094002, -0.04280088469386101, -0.027265701442956924, 0.014815890230238438, 0.011521355248987675, 0.026519091799855232, -0.019219718873500824, -0.0062500000931322575, 0.033098507672548294, 0.004942654632031918, 0.039525195956230164, 0.02062847651541233, 0.005455861333757639, 0.012746674939990044, -0.027081137523055077, -0.0104978047311306, -0.04482215270400047, -0.0208904966711998, -0.02419615350663662, -0.024383211508393288, 0.014807827770709991, -0.043333958834409714, 0.04029605910181999, 0.008332453668117523, -0.021724741905927658, 0.007439314387738705, 0.0035941980313509703, 0.005946723744273186, -0.023132942616939545, -0.022103315219283104, -0.02797836810350418, -0.005332794040441513, 0.01679682545363903, -0.02752087637782097, 0.016509750857949257, -0.030921978875994682, 0.007435621228069067, -0.0004713617963716388, 0.0004172207845840603, 0.02795747108757496, 0.01786021515727043, 0.043798770755529404, -0.016930822283029556, 0.05470006912946701, -0.03657989948987961, -0.011536735109984875, 0.03622429817914963, 0.025203369557857513, 1.0645409020071384e-05, -0.05024847015738487, 0.030042005702853203, 0.0323418490588665, 0.004826457239687443, 0.004085680469870567, -0.010639363899827003, 0.03768095746636391, 0.015158074907958508, 0.027645407244563103, 0.039636969566345215, -0.02144729346036911, 0.04858939349651337, 0.0045397779904305935, 0.0322754867374897, 0.003069448983296752, -0.00307764345780015, -0.02127068117260933, -0.014249890111386776, 0.007399043068289757, -0.029989631846547127, -0.044316615909338, -0.033162012696266174, -0.012202168814837933, 0.0013076522154733539, 0.017370780929923058, 0.021724265068769455, 0.04147261381149292, -0.005139585584402084, 0.023595547303557396, 0.07132693380117416, 0.03597167879343033, -0.05282239988446236, -0.07421616464853287, 0.033128608018159866, -0.0100846067070961, -0.003680864814668894, -0.071330726146698, -0.008875930681824684, -0.00484068039804697, 0.01862458698451519, 0.06088341772556305, -0.012283618561923504, -0.03008660115301609, 0.017507502809166908, -0.031225457787513733, 0.022236203774809837, 0.011219553649425507, -0.007059658411890268, 0.01773524284362793, -0.01644061878323555, -0.055816225707530975, 0.009885657578706741, 0.0014193792594596744, -0.061817627400159836, 0.0008312796708196402, -0.038952119648456573, 0.02844098024070263, -0.012756587006151676, -0.0037608020938932896, 0.017724519595503807, 0.029206672683358192, -0.0030753789469599724, 0.03444994240999222, -0.055632974952459335, 0.02214323915541172, -0.008154047653079033, -0.013196260668337345, 0.014588641934096813, -0.05299564450979233, -0.024594543501734734, -0.03114033117890358, -0.001730480114929378, 0.009990962222218513, -0.005222476087510586, 0.037801992148160934, -0.012333880178630352, -0.03202565386891365, 0.03379486873745918, -0.01358069945126772, 0.02387303113937378, 0.0964081659913063, -0.0046262736432254314, 0.009539440274238586, -0.018674610182642937, 0.0034125586971640587, -0.01062136609107256, 0.019265128299593925, -0.023777686059474945, 0.012459988705813885, -0.024633223190903664, -0.017619585618376732, 0.046350933611392975, 0.031397271901369095, 0.07799896597862244, 0.0027542610187083483, 0.0013590655289590359, 0.03246307000517845, 0.05506893992424011, -0.023963121697306633, 0.04197651520371437, 0.020911959931254387, -0.026048004627227783, -0.016427943482995033, 0.06121016666293144, 0.05285702273249626, -0.029632333666086197, -0.04321730509400368, 0.00624919543042779, 0.032956771552562714, -0.0012625810923054814, 0.01614643819630146, -0.021601302549242973, -0.06426038593053818, -0.052211832255125046, -0.026151195168495178, -0.0070402673445641994, 0.021096743643283844, -0.09012006968259811, 0.056567683815956116, 0.031425487250089645, -0.052051592618227005, -0.04825172200798988, 0.023000584915280342, 0.033806245774030685, 0.03554842248558998, 0.04884601756930351, -0.03369031473994255, 0.048658158630132675, 0.009680166840553284, -0.022846905514597893, -0.016308272257447243, 0.015582280233502388, 0.016462717205286026, -0.028510984033346176, -0.010684767737984657, -0.006590621545910835, -0.03163078427314758, -0.06452224403619766, 0.008037887513637543, -0.016014838591217995, -0.07670524716377258, 0.02862890623509884, -0.027834143489599228, 0.00791150238364935, -0.014374252408742905, -0.005482922773808241, 0.032551221549510956, -0.022987965494394302, -0.0077245705761015415, -0.013125916942954063, 0.012745246291160583, 0.006856758613139391, 0.01135892141610384, -0.03648795187473297, -0.03898732364177704, -0.010644171386957169, 0.007930116727948189, -0.018816884607076645, -0.01997724175453186, -0.023701336234807968, -0.053983740508556366, -0.04231886938214302, 0.012443233281373978, -0.02356063760817051, -0.021645469591021538, -0.05252286419272423, -0.018153375014662743, 0.03625401854515076, -0.00522639462724328, -0.060672786086797714, 0.027090787887573242, -0.023907402530312538, 0.010797780938446522, 0.020772242918610573, -0.04114414006471634, 0.005683746188879013, 0.03719312325119972, 0.041371867060661316, -0.016874704509973526, 0.05899733304977417, 0.004750577732920647, 0.02509468048810959, 0.009310876950621605, 0.030795229598879814, 0.026184892281889915, -0.044630877673625946, 0.016837608069181442, -0.00508201215416193, -0.010509965009987354, -0.020640965551137924, -0.03900197148323059, 0.02683549001812935, -0.0037847193889319897, 0.008794894441962242, -0.06254959106445312, -0.04868461564183235, -0.01625959575176239, -0.01921760104596615, -0.019536824896931648, -0.03247762843966484, 0.023022526875138283, -0.008322073146700859, -0.006093085743486881, 0.009741772897541523, -0.011546514928340912, -0.0025885473005473614, -0.011654475703835487, 0.023264627903699875, 0.001751268981024623, 0.010315985418856144, 0.03904933109879494, 0.034824442118406296, 0.01802799478173256, 0.00389663758687675, -0.015018037520349026, -0.004154720343649387, 0.005654134321957827, 0.005908592138439417, -0.017202291637659073, 0.019945623353123665, -0.02364332601428032, -0.03809339925646782, 0.011186107061803341, 0.009842227213084698, 0.03611011803150177, -0.011967009864747524, -0.02740272507071495, -0.010860193520784378, -0.0013447196688503027, -0.03504852578043938, -0.02154947817325592, 0.01630093902349472, 0.053015854209661484, -0.08385889232158661, -0.03803049400448799, 0.056130148470401764, 0.0539330318570137, -0.05124674364924431, -0.08401983976364136, -0.04642901197075844, -0.006703904829919338, 0.001887202961370349, -0.015725301578640938, 0.01859247125685215, 0.015920164063572884, -0.029577581211924553, 0.02163366600871086, 0.004158743657171726, -0.019263718277215958, 0.057391662150621414, 0.0004979577497579157, -0.022918280214071274, 0.033241402357816696, -0.05532181262969971, 0.06332176178693771, -0.007114325184375048, -0.01933888904750347, 0.010918287560343742, -0.031720250844955444, -0.05004068836569786, -0.0330917127430439, 0.013567779213190079, 0.03082425892353058, -0.012065939605236053, 0.027089029550552368, -0.004998729098588228, -0.044272344559431076, 0.040688782930374146, 0.030400052666664124, 0.036961913108825684, 0.03559935465455055, 0.009244654327630997, -0.0525328554213047, 0.0644826591014862, -0.061578452587127686, 0.08728901296854019, -0.040878184139728546, 0.028818275779485703, 0.003582561621442437, -0.06156265363097191, 0.049587614834308624, 0.02566860243678093, -0.04974575713276863, 0.08334795385599136, -0.05693211033940315, -0.01249326765537262, -0.03582911193370819, 0.053554899990558624, -0.02478613518178463, 0.025351906195282936, 0.017553631216287613, -0.008606337942183018, 0.036498889327049255, -0.00024216524616349488, -0.005221406929194927, 0.03986015170812607, 0.05964706838130951, 0.03735734149813652, 0.00306659284979105, 0.021682916209101677, 0.039266906678676605, -0.009042748250067234, 0.027454840019345284, -0.06136779114603996, -0.023684946820139885, 0.03235805407166481, -0.010130510665476322, 0.012669116258621216, 0.02304290607571602, 0.03679600730538368, -0.027376167476177216, -0.021223844960331917, -0.009555072523653507, -0.0035176679957658052, 0.002010633470490575, -0.0019017192535102367, 0.0041175964288413525, -0.0395912267267704, -0.038575924932956696, -0.011039171367883682, -0.11592809855937958, -0.0017087410669773817, -0.01209713239222765, -0.005779541563242674, -0.02345600724220276, 0.015411400236189365, -0.04400978237390518, -0.010699235834181309, -0.008112700656056404, 0.005842282436788082, 8.99779042811133e-05, 0.007505942136049271, -0.0031450011301785707, -0.02173207327723503, -0.03378661349415779, 0.012795842252671719, -0.0023295350838452578, 0.03755171597003937, -0.00048430319293402135, 0.05910797789692879, 0.07650420814752579, 0.03067389875650406, -0.043159425258636475, 0.05050352215766907, 0.06947936117649078, -0.0025944409426301718, 0.005721292924135923, -0.05692298337817192], [-0.036581818014383316, 0.010073021054267883, 0.026795921847224236, -0.010766640305519104, -0.0070585026405751705, -0.00899476557970047, 0.003140990389510989, -0.02669595740735531, -0.013472821563482285, -0.03233932703733444, -0.02348964288830757, -0.0011551518691703677, 0.030397947877645493, -0.0365615114569664, 0.045417141169309616, 0.02466903068125248, 0.02257111668586731, 0.036125537008047104, -0.051312778145074844, 0.06028195843100548, -0.024950047954916954, -0.08763647824525833, 0.0555235780775547, 0.0012449290370568633, -0.019855843856930733, 0.03494909778237343, -0.014423291198909283, 0.05808895826339722, -0.003759697312489152, 0.012457340024411678, -0.031203513965010643, 0.008697522804141045, -0.038804758340120316, -0.04351627454161644, -0.011007240042090416, 0.04227812960743904, -0.05366982892155647, -0.02141643315553665, 0.002705102786421776, -0.01155872642993927, -0.026111263781785965, -0.02490147575736046, 0.01556649524718523, 0.011519815772771835, -0.03058806248009205, 0.04391561448574066, 0.0446794256567955, -0.003585163038223982, 0.04957081377506256, -0.009542914107441902, -0.015174170956015587, 0.18869274854660034, 0.015485555864870548, 0.09700492769479752, -0.013247719965875149, 0.08334607630968094, 0.012826496735215187, -0.022270457819104195, 0.022324686869978905, -0.04021719470620155, -0.021333444863557816, 0.007791104726493359, -0.021453192457556725, -0.03812616318464279, 0.011956186033785343, -0.014313576743006706, -0.0324644036591053, -0.0526944175362587, 0.022833462804555893, -0.05091758817434311, 0.010845416225492954, 0.024099770933389664, -0.017179984599351883, 0.014596756547689438, 0.028626395389437675, 0.006473413668572903, -0.04430287703871727, 0.019609911367297173, 0.006622153799980879, 0.018568510189652443, -0.023232046514749527, 0.0002627599169500172, 0.03782006725668907, 0.0238049253821373, -0.06639358401298523, 0.060133252292871475, -0.030964702367782593, 0.04203430935740471, -0.015917744487524033, 0.002035807352513075, 0.04017888382077217, -0.025414085015654564, -0.011763452552258968, 0.027789413928985596, 0.008348939009010792, 0.03745666891336441, -0.03576388955116272, 0.002655617892742157, 0.03168431669473648, -0.041627973318099976, 0.0555587038397789, 0.015183131210505962, 0.0011411166051402688, -0.03459044173359871, -0.0011991446372121572, 0.02158713899552822, -0.014043622650206089, -0.07830765843391418, 0.06541228294372559, -0.008357809856534004, 0.005622108932584524, -0.03906689211726189, -0.013922803103923798, 0.04573836922645569, 0.00108401442412287, -0.018082868307828903, 0.028894590213894844, -0.039322711527347565, 0.042333126068115234, -0.05566353723406792, 0.0028872955590486526, -0.017389832064509392, -0.015173235908150673, -0.0028993519954383373, -0.001887913211248815, -0.07107415795326233, 0.03234519809484482, 0.011081533506512642, -0.01013134978711605, -0.002451522508636117, 0.020929157733917236, -0.023568063974380493, -0.02242664247751236, 0.029573846608400345, -0.012176521122455597, 0.07204768061637878, -0.027660103514790535, -0.001658523571677506, -0.018017495051026344, 0.012078855186700821, 0.036635275930166245, 0.005087900906801224, 0.007758649531751871, 0.012119865976274014, 0.07411061972379684, -0.03321191295981407, -0.008321507833898067, -0.002280710730701685, 0.01337346900254488, -0.004737734328955412, 0.0027556754648685455, 0.010998864658176899, 0.009435719810426235, -0.029121100902557373, 0.0018040512222796679, 0.010107532143592834, -0.06987128406763077, 0.027581866830587387, -0.02713521011173725, -0.035461995750665665, -0.015424016863107681, -0.02653784118592739, 0.011443421244621277, -0.0029412328731268644, -0.0472804456949234, -0.03843669593334198, -0.02327510342001915, 0.020191283896565437, -0.03499628230929375, -0.06030609831213951, 0.041142094880342484, -0.039939671754837036, -0.018050426617264748, 0.030399948358535767, 0.04204346612095833, 0.052162058651447296, -0.016414955258369446, 0.031139202415943146, 0.00797954760491848, 0.024478930979967117, -0.013382080011069775, -0.03586176410317421, -0.03694706782698631, 0.018486782908439636, -0.003653736785054207, -0.00587861705571413, 0.031211426481604576, -0.016862060874700546, -0.012091046199202538, -0.004467340186238289, 0.048107754439115524, 0.0021062083542346954, -0.03619375079870224, -0.023124288767576218, -0.049535758793354034, -0.027603132650256157, 0.01013618241995573, -0.013745260424911976, -0.03361058980226517, 0.045922838151454926, 0.014138354919850826, 0.0046354844234883785, 0.043130140751600266, 0.03430952876806259, -0.003869428299367428, -0.04597063362598419, -0.00821775570511818, 0.004351733718067408, -0.011450571939349174, 0.01965819112956524, -0.02123977616429329, 0.05124661326408386, 0.014432908035814762, -0.06550310552120209, 0.02152146026492119, 0.00861317478120327, -0.011803883127868176, -0.03743867203593254, -0.006206560414284468, -0.02600625716149807, 0.01646239310503006, -0.0822233110666275, -0.004082154482603073, -0.007109582889825106, 0.005254955496639013, 0.018213890492916107, -0.044870708137750626, -0.03776099905371666, -0.019951066002249718, 0.014907129108905792, 0.002454438479617238, 0.0647810846567154, 0.025963887572288513, 0.03595118969678879, 0.043750111013650894, -0.017094317823648453, -0.04398880526423454, -0.00042724376544356346, 0.013821817003190517, -0.017451593652367592, -0.0022978447377681732, -0.015478934161365032, 0.01941470429301262, -0.034798040986061096, 0.010106595233082771, -0.03164811059832573, -0.0201395433396101, -0.03812509402632713, -0.016233930364251137, 0.0010937484912574291, 0.01255244854837656, 0.006130138412117958, 0.02518865093588829, -0.008098027668893337, -0.056686919182538986, -0.01651851460337639, 0.06642758846282959, 0.02303437516093254, 0.07493920624256134, -0.005198766943067312, -0.017158016562461853, 0.01788739673793316, 0.013460499234497547, 0.025210341438651085, -0.00518149696290493, 0.0237666517496109, -0.005679132882505655, -0.012893985956907272, -0.04863579943776131, 0.04970117658376694, -0.028631234541535378, 0.029444662854075432, 0.02585621550679207, -0.05548883602023125, -0.006366283632814884, -0.03450919687747955, -0.011873414739966393, -0.007022402714937925, -0.04048844054341316, -0.012937440536916256, -0.02737436071038246, 0.04128827899694443, 0.03852083906531334, 0.022283067926764488, 0.018818428739905357, -0.010002869181334972, -0.02279437892138958, 0.022069226950407028, 0.02610199525952339, -0.001565901329740882, -0.0038639323320239782, -0.029808184131979942, -0.019709017127752304, 0.05083831772208214, 0.39969944953918457, -0.0661829486489296, -0.0012576482258737087, -0.004297880455851555, -0.008991871029138565, -0.0030302247032523155, 0.009013923816382885, -0.00516571244224906, -0.022353585809469223, 0.014033393934369087, -0.10883153229951859, 0.05390012636780739, -0.045276790857315063, 0.03216258063912392, -0.0009499185252934694, -0.02060519903898239, 0.0030808444134891033, -0.03933585435152054, -0.027875201776623726, -0.05745230242609978, 0.05935656279325485, -0.03683535009622574, -0.03922348842024803, -0.03576996922492981, -0.03600393608212471, -0.05082487687468529, 0.02110414206981659, -0.021178515627980232, -0.019959190860390663, -9.032025263877586e-05, 0.03955037146806717, 0.06154070422053337, -0.0010178270749747753, 0.025036312639713287, -0.056930061429739, -0.08301171660423279, 0.0701964795589447, -0.037482861429452896, -0.008948437869548798, -0.032963044941425323, -0.011260964907705784, -0.053728096187114716, -0.03263739496469498, -0.0060315509326756, 0.014697845093905926, -0.02412898652255535, 0.018686292693018913, 0.005517038982361555, -0.03950466588139534, 0.0211291816085577, 0.00811493955552578, 0.00732596917077899, -0.0015931138768792152, -0.013298275880515575, -0.002227883553132415, -0.02082320675253868, -0.031188951805233955, 0.02560822106897831, -0.004734433256089687, -0.028198447078466415, 0.00013712576765101403, 0.010594532825052738, 0.013493484817445278, -0.02157941460609436, -0.05713809281587601, 0.0001350094680674374, 0.024217650294303894, -0.01472527626901865, -0.012997766025364399, 0.014451044611632824, -0.04556480795145035, -0.043715719133615494, 0.013310893438756466, 0.008253132924437523, -0.039675332605838776, -0.01229166891425848, 0.0071889059618115425, -0.013789482414722443, -0.010690032504498959, -0.05058395117521286, 0.013861093670129776, 0.015941273421049118, 0.015693267807364464, -0.011550191789865494, -0.006415822077542543, 0.021820789203047752, -0.01290894951671362, 0.02915102429687977, 0.014439228922128677, 0.011881710961461067, -0.0038531797472387552, -0.022051343694329262, 0.0038631674833595753, -0.0474851168692112, -0.027542833238840103, -0.031031738966703415, -0.013831613585352898, 0.010237623006105423, -0.030846480280160904, 0.04505244269967079, -0.00180640013422817, -0.010975428856909275, 0.004637952893972397, -0.009562348015606403, 0.010327686555683613, -0.005326542537659407, -0.01662730984389782, -0.0266362726688385, -0.004510846920311451, 0.022453486919403076, -0.01448703370988369, 0.01288733258843422, -0.01830177754163742, 0.031383924186229706, 0.006120034493505955, 0.010788487270474434, 0.01343909278512001, -0.01062127947807312, 0.053345028311014175, -0.018427545204758644, 0.02282584272325039, -0.02018718048930168, -0.00890721008181572, 0.012496957555413246, -0.005240926519036293, -0.016256902366876602, -0.035282496362924576, 0.0361487977206707, 0.015476196072995663, 0.008915271610021591, 0.015274636447429657, -0.04319792613387108, 0.02210477739572525, 0.02339285984635353, 0.0303725004196167, 0.04031582921743393, -0.007133690640330315, 0.061912208795547485, 0.021360183134675026, 0.004933001007884741, -0.011022917926311493, 0.009772446937859058, -0.009332459419965744, 0.013137162663042545, -0.02499273419380188, -0.01919126883149147, -0.05381115525960922, -0.025223873555660248, -0.02618958242237568, -0.00700649106875062, -0.001278976327739656, 0.030521588400006294, 0.026249084621667862, -0.028236960992217064, 0.0002978669654112309, 0.09235946089029312, 0.027865925803780556, -0.05616642162203789, -0.03432060033082962, 0.04388026148080826, -0.012317209504544735, -0.03306518867611885, -0.05957326665520668, -0.0017370887799188495, 0.0223593357950449, 0.032990142703056335, 0.05125713720917702, 0.01120319589972496, -0.026585416868329048, 0.021489793434739113, -0.022104298695921898, 0.02427249774336815, -0.008400735445320606, 0.00549588305875659, 0.02188982255756855, -0.005545992869883776, -0.04384542629122734, 0.013368266634643078, 0.001040358212776482, -0.06356944143772125, 0.005332896951586008, -0.05750076100230217, 0.018938474357128143, -0.02585202269256115, 0.0070988452062010765, 0.0002133474190486595, 0.002409174805507064, -0.042504649609327316, 0.027752190828323364, -0.057633720338344574, 0.007266530301421881, -0.00994928926229477, -0.01885038986802101, 0.03238825127482414, -0.04154543578624725, -0.0019872800912708044, -0.011625858023762703, -0.007837834767997265, -0.03918956592679024, -0.01851225085556507, 0.040464356541633606, -0.0343044139444828, 0.009749566204845905, 0.04221927747130394, -0.018641099333763123, 0.029750706627964973, 0.11501779407262802, 0.021870259195566177, 0.012646336108446121, -0.019703276455402374, -0.02379799447953701, -0.016388505697250366, 0.01860063336789608, 0.0012425398454070091, 0.007428859360516071, -0.013034651055932045, -0.02521556243300438, 0.05205870047211647, 0.008971287868916988, 0.07273448258638382, -0.00016579027578700334, -0.007607368752360344, 0.03547785058617592, 0.03619987890124321, -0.019550368189811707, 0.046987131237983704, -0.007517275866121054, -0.02427954412996769, -0.035658057779073715, 0.05588529258966446, 0.05869385972619057, -0.0176065843552351, -0.019820677116513252, 0.02798854559659958, 0.01210649311542511, -0.001977843465283513, 0.0031801380682736635, -0.010457844473421574, -0.06046414375305176, -0.038067661225795746, -0.025752844288945198, -0.009913919493556023, 0.013238738290965557, -0.07600773870944977, 0.06392673403024673, 0.013340029865503311, -0.022501375526189804, -0.0473896749317646, 0.007996066473424435, 0.029044978320598602, 0.018995393067598343, 0.022630631923675537, -0.019288592040538788, 0.043312959372997284, 0.018586456775665283, -0.015895338729023933, -0.03960658609867096, 0.01299650501459837, 0.019215639680624008, -0.045955829322338104, 0.004014914855360985, -0.0312621109187603, -0.04204612597823143, -0.057944219559431076, 0.029942458495497704, -0.01870732568204403, -0.04818938672542572, 0.03720204904675484, -0.035243161022663116, 0.019536878913640976, -0.007933169603347778, -0.009955435991287231, 0.0387917086482048, 0.019344838336110115, -0.007700328715145588, -0.013245568610727787, 0.027345452457666397, 0.016845691949129105, 0.0252726748585701, -0.025632638484239578, -0.029068876057863235, -0.018643701449036598, -0.0024823120329529047, -0.012676612474024296, -0.012016208842396736, -0.02014079876244068, -0.06189701706171036, -0.04807691648602486, 0.0069807288236916065, -0.02172396332025528, -0.02940819039940834, -0.057253073900938034, -0.008816828019917011, 0.020428571850061417, -0.031781021505594254, -0.05801761522889137, 0.016299281269311905, -0.012872662395238876, 0.036200232803821564, 0.009005541913211346, -0.02292540855705738, -0.002824811963364482, 0.02750042825937271, 0.029063113033771515, 0.0001561590179335326, 0.05417504161596298, 0.004147725645452738, 0.012662319466471672, 0.02899838052690029, 0.04044823348522186, -0.012194736860692501, -0.057837069034576416, 0.017125308513641357, 0.011739104054868221, -0.01980442740023136, -0.032435059547424316, -0.02622162736952305, 0.016552191227674484, -0.0010652715573087335, 0.021956143900752068, -0.06455622613430023, -0.042854174971580505, -0.013050875626504421, -0.024326398968696594, -0.02287129871547222, -0.03776739537715912, 0.0342317558825016, -0.011858971789479256, 0.02160385437309742, 0.007320287171751261, -0.03165094926953316, 0.021378446370363235, 0.005179133266210556, 0.016283035278320312, 0.02209150791168213, 0.017158085480332375, 0.05630536749958992, 0.01923459768295288, 0.039623748511075974, -0.004805913195014, -0.009892004542052746, -0.003364283824339509, 0.022162431851029396, -0.0011409398866817355, 0.00042940719868056476, 0.007422021124511957, -0.022484147921204567, -0.05708448588848114, -0.004882959183305502, 0.0055856043472886086, 0.0351715162396431, 0.022156231105327606, -0.03095836192369461, 0.003619660623371601, 0.011045662686228752, -0.0159778855741024, -0.005079891532659531, 0.014876086264848709, 0.015736017376184464, -0.053042419254779816, -0.0073595005087554455, 0.04494920000433922, 0.08333353698253632, -0.07242065668106079, -0.09981326013803482, -0.01498020626604557, -0.004480006173253059, 0.012611309997737408, -0.003834701841697097, -0.0035226100590080023, 0.04136026278138161, -0.044044747948646545, 0.038589175790548325, 0.01726841740310192, 0.004007398150861263, 0.047402311116456985, -0.00032549796742387116, -0.02997979149222374, 0.02087847888469696, -0.05866853520274162, 0.056705161929130554, 0.0037283285055309534, 0.0029951243195682764, 0.014583532698452473, -0.008424866013228893, -0.08799389004707336, -0.02489367313683033, 0.008993335999548435, 0.04734828323125839, 0.009477037005126476, -0.006156465038657188, -0.005291678011417389, -0.04484803229570389, 0.04313182458281517, 0.01641753874719143, 0.03427095711231232, 0.042461901903152466, 0.008549431338906288, -0.034633249044418335, 0.03546756133437157, -0.021284407004714012, 0.11497361958026886, -0.07216253131628036, 0.06792764365673065, 0.001755041885189712, -0.03381579741835594, 0.057783015072345734, 0.04928850010037422, -0.013935836032032967, 0.10344962030649185, -0.02389408089220524, -0.022448239848017693, -0.05523258075118065, 0.06489995867013931, -0.03252505138516426, 0.023745305836200714, 0.02988157607614994, 0.003206297755241394, 0.03817608207464218, -0.01457166112959385, -0.02218608371913433, 0.036761291325092316, 0.0506906621158123, 0.013323454186320305, 0.010936757549643517, 0.003915935289114714, 0.06587395817041397, -0.0066620949655771255, 0.011536965146660805, -0.050766464322805405, -0.019231636077165604, 0.05779711902141571, 0.009757299907505512, 0.020296547561883926, 0.03003658726811409, 0.016931913793087006, -0.0316842719912529, -0.02868836745619774, -0.012017197906970978, -0.0005971092032268643, -0.005944598466157913, 0.012147638015449047, 0.0024184114299714565, -0.028170667588710785, 0.0027272456791251898, -0.031386878341436386, -0.10901948809623718, 0.0012457467382773757, -0.0019374208059161901, -0.005590231157839298, -0.036873213946819305, 0.02601686865091324, -0.028831282630562782, -0.001518182922154665, -0.017782699316740036, 0.0005263431812636554, -0.03298202157020569, -0.023167233914136887, -0.017876656726002693, -0.03822356089949608, -0.03504818305373192, -0.005824193824082613, -0.008516289293766022, 0.024377519264817238, -0.0018409864278510213, 0.035662394016981125, 0.05490808188915253, 0.006278791464865208, -0.04016740992665291, 0.052969496697187424, 0.07926817983388901, -0.010027901269495487, -0.005593312904238701, -0.08356794714927673], [-0.0028111322317272425, 0.03767164424061775, 0.03137720003724098, -0.05027510225772858, -0.007075621280819178, 0.029877055436372757, 0.010408597998321056, 0.003751307725906372, 0.0002585502516012639, -0.03384077548980713, -0.04194878414273262, -0.000921030412428081, 0.01348236296325922, -0.009975946508347988, 0.025361288338899612, 0.034550923854112625, 0.008366541936993599, 0.029949679970741272, -0.030212409794330597, 0.02707592397928238, -0.013389171101152897, -0.08793088048696518, 0.032566290348768234, 0.006016924977302551, 0.004162135999649763, 0.03758365288376808, 0.003421780187636614, 0.05871320888400078, -0.004127333872020245, 0.029693949967622757, -0.01291804388165474, -0.011352654546499252, -0.03342144191265106, -0.030401162803173065, -0.013946975581347942, 0.04262962564826012, -0.03519277274608612, 0.01863132230937481, 0.003973162267357111, 0.0025789730716496706, -0.009667537175118923, -0.025397099554538727, -0.006517753936350346, -0.005025048274546862, -0.053578611463308334, 0.03267692029476166, 0.037972185760736465, 0.015931261703372, 0.02572198398411274, -0.016122104600071907, 0.007044616155326366, 0.1814655363559723, 0.0489870123565197, 0.07856328785419464, -0.009469123557209969, 0.0781204104423523, 0.0032724414486438036, -0.017978152260184288, 0.002809230238199234, -0.04300633445382118, -0.04072220250964165, 0.025720231235027313, -0.02588687650859356, -0.07388044893741608, 0.013568853959441185, 0.009447171352803707, -0.0412590429186821, -0.051517385989427567, 0.02945597842335701, -0.011993104591965675, 0.033574629575014114, 0.011005121283233166, -0.004185929428786039, 0.01893887110054493, 0.04345390945672989, 0.00019605946727097034, -0.050183944404125214, -0.005341758020222187, -0.012518824078142643, -0.01784956268966198, -0.052434295415878296, 0.01028475072234869, 0.020448388531804085, 0.026080474257469177, -0.027716781944036484, 0.05156273394823074, -0.04118610545992851, 0.059218794107437134, -0.03032209910452366, 0.011722482740879059, 0.04996531456708908, -0.00930941104888916, 0.0282922200858593, 0.03151048719882965, 0.007648618891835213, 0.009022513404488564, -0.012462127953767776, 0.03416745364665985, 0.009380746632814407, -0.04463296756148338, 0.063482366502285, 0.0074979099445044994, 0.007331738248467445, -0.04034360498189926, 0.002277844352647662, 0.0226668082177639, 0.017217624932527542, -0.08429290354251862, 0.05959882214665413, -0.0316859595477581, 0.02969539910554886, -0.01616843231022358, -0.01252202782779932, 0.05022761970758438, 0.013268735259771347, 0.006107226945459843, 0.007385630160570145, -0.026868099346756935, 0.04394705221056938, -0.040025029331445694, 0.0012201175559312105, 0.021327117457985878, -0.033453330397605896, -0.02148125134408474, 0.009015792980790138, -0.09359217435121536, 0.02341354638338089, -0.01212356612086296, 0.016809262335300446, 0.04825916513800621, 0.009568776935338974, -0.009221445769071579, 0.015623773448169231, 0.01807398349046707, -0.00017220887821167707, 0.040493469685316086, -0.012842314317822456, -0.026665227487683296, -0.023987028747797012, 0.010605419054627419, 0.0011105325538665056, 0.0073208934627473354, 0.016329431906342506, 0.007625236175954342, 0.04151858761906624, -0.033279553055763245, -0.040040235966444016, -0.014941301196813583, -0.01281601283699274, -0.002134221140295267, 0.016661621630191803, 0.03477572277188301, -0.0037449831143021584, -0.043120916932821274, -0.013167810626327991, 0.004269592463970184, -0.029065076261758804, 0.04336981475353241, -0.024952437728643417, -0.06211227923631668, 0.0164734348654747, -0.03770710527896881, 0.05391794815659523, 0.008878699503839016, -0.05476943030953407, -0.03152364119887352, 0.0014992879005149007, 0.008590076118707657, -0.02659524232149124, -0.03467632830142975, 0.059064291417598724, -0.033191464841365814, -0.011774909682571888, 0.006356224417686462, 0.02379269152879715, 0.030828189104795456, -0.018971841782331467, 0.01423652097582817, 0.02768038772046566, 0.013172158971428871, 0.039028558880090714, -0.013762927614152431, -0.050492141395807266, 0.01143199484795332, -0.008561016991734505, 0.006448483094573021, -0.0061933440156280994, -0.04464444890618324, -0.013588805682957172, -0.05114399269223213, 0.08094646036624908, -0.002226836048066616, -0.03241150453686714, -0.05235912650823593, -0.07403314113616943, -0.026303362101316452, 0.0032949650194495916, 0.0036620707251131535, -0.04892547056078911, 0.035198625177145004, 0.030499329790472984, -0.0029978270176798105, 0.0417739599943161, 6.414765084628016e-05, -0.008739646524190903, 0.0034272398333996534, -0.01063447818160057, -0.025012915953993797, -0.012140178121626377, 0.04300341010093689, -0.01662445440888405, 0.0368448831140995, 0.033666860312223434, -0.0688236653804779, 0.026319043710827827, 0.014504997991025448, -0.01735372096300125, -0.06886939704418182, 0.014768756926059723, 0.002651514718309045, 0.003559115808457136, -0.03507265821099281, -0.0030195938888937235, -0.015187125653028488, -0.04141173139214516, 0.012485632672905922, -0.013070810586214066, -0.046671684831380844, -0.024407662451267242, 0.004041070118546486, -0.018169322982430458, 0.07005815207958221, 0.01192147471010685, 0.03859415650367737, 0.04847600311040878, -0.029390035197138786, -0.005322398152202368, -0.0017257790314033628, 0.012198054231703281, 0.017628615722060204, 0.002329506678506732, 0.01667092740535736, 0.02557802014052868, -0.013045408762991428, 0.003930860199034214, -0.0036001920234411955, -0.054302629083395004, -0.02065136842429638, -0.006692912429571152, 0.015584718436002731, 0.026879169046878815, 0.013082081452012062, 0.025532837957143784, -0.04788297414779663, -0.04161180183291435, -0.026213334873318672, 0.08343100547790527, 0.026276757940649986, 0.02610228955745697, -0.013247043825685978, -0.03316390886902809, 0.01016933098435402, 0.0424853153526783, 0.01933956891298294, 0.02655298449099064, 0.013356157578527927, -0.013790644705295563, -0.007642168551683426, -0.05437413975596428, 0.029217926785349846, -0.0680149719119072, -0.0075433687306940556, 0.029030291363596916, -0.0515422523021698, -0.0010874220170080662, -0.012289847247302532, -0.018860647454857826, -0.027696067467331886, -0.053144022822380066, -0.02283588983118534, -0.06097141280770302, 0.01093895174562931, 0.0031610054429620504, 0.058818306773900986, 0.024277489632368088, 0.0053327735513448715, -0.012429132126271725, 0.053654979914426804, 0.03968747705221176, -0.009557303041219711, -0.0221087783575058, 0.013538512401282787, -0.03999435529112816, -0.003802536055445671, 0.407995343208313, -0.04602925479412079, 0.01969495601952076, -0.014501235447824001, -0.01938300020992756, -0.017062123864889145, 0.0448111928999424, 0.02554510533809662, -0.014903480187058449, -0.012250572443008423, -0.09322906285524368, 0.029913930222392082, -0.06628278642892838, 0.03371003642678261, -0.019198134541511536, -0.009584829211235046, 0.007902022451162338, -0.032702136784791946, -0.007664993405342102, -0.04408729076385498, 0.08448392897844315, -0.010950272902846336, -0.04119589179754257, -0.04837186262011528, -0.03162529692053795, -0.04805063083767891, 0.011418682523071766, -0.04039141535758972, -0.02340346947312355, 0.0017849728465080261, 0.03264176845550537, 0.06302245706319809, -0.017812224105000496, 0.055589720606803894, -0.060804564505815506, -0.08238276839256287, 0.043129101395606995, -0.03623978793621063, -0.036243416368961334, -0.02889639511704445, 0.007345105055719614, -0.03409069776535034, 0.02353040687739849, 0.02039935067296028, -0.04239664226770401, -0.035241272300481796, 0.02634984254837036, 0.013868582434952259, -0.029113974422216415, -0.001912356587126851, 0.028365900740027428, 0.0025877857115119696, 0.02224465273320675, -0.00820360891520977, -0.0065726242028176785, -0.02935021184384823, -0.01138181984424591, 0.01706298254430294, 0.024817170575261116, -0.02017301693558693, 0.02302248403429985, 0.03302709385752678, -0.005758320447057486, -0.029921697452664375, 0.002675008727237582, 0.01844707876443863, 0.05113762244582176, -0.024836527183651924, -0.03526196628808975, 0.018992522731423378, -0.028176017105579376, -0.033340178430080414, 0.015605999156832695, -0.02460186555981636, -0.04371437802910805, -0.0006939538870938122, 0.006104302126914263, -0.031496312469244, 0.026667650789022446, -0.01695672608911991, 0.004741853103041649, 0.026347845792770386, 0.045148760080337524, 0.017485328018665314, 0.015671825036406517, -0.017458995804190636, 0.011146390810608864, 0.06371954828500748, -0.011784122325479984, 0.05446731671690941, 0.029294176027178764, -0.03337472677230835, -0.012205011211335659, -0.05759688466787338, -0.009226622059941292, -0.04995960742235184, -0.014490886591374874, 0.027475016191601753, -0.01847672462463379, 0.08372612297534943, -0.01812230981886387, 0.004686023574322462, 0.019567610695958138, -0.00058971275575459, 0.014270083978772163, -0.01092453021556139, -0.005041907075792551, -0.030561475083231926, 0.008758398704230785, 0.019345469772815704, -0.01771947368979454, 0.030952701345086098, -0.03697638586163521, -0.008383176289498806, 0.0013802408939227462, 0.0391673669219017, -0.011061016470193863, 0.01775405928492546, 0.008640056475996971, -0.009814225137233734, 0.01909407414495945, -0.027260055765509605, 0.005407712422311306, 0.025348402559757233, -0.0011646616039797664, -0.020045990124344826, -0.05238595977425575, -0.0023056960199028254, 0.014571883715689182, -0.010116726160049438, -0.013123899698257446, -0.05361853912472725, 0.058053936809301376, 0.018628479912877083, 0.00969015434384346, 0.03081478923559189, -0.01854657754302025, 0.07321812212467194, -0.022352617233991623, 0.01861782930791378, -0.03413907811045647, -0.01073489896953106, -0.005071757826954126, 0.002884837333112955, -0.02196737378835678, 0.040229540318250656, -0.02276238426566124, -0.023830898106098175, -0.027359865605831146, 0.0022719120606780052, -0.006374682765454054, -0.03153184428811073, 0.01921144127845764, -0.020520472899079323, 0.01018687803298235, 0.08269081264734268, 0.007584182545542717, -0.06095312908291817, -0.03165921941399574, 0.04017963632941246, -0.05537797510623932, -0.02107316441833973, 0.00872233510017395, -0.036883506923913956, 0.011507967486977577, 0.018553465604782104, 0.059458665549755096, -0.005396200809627771, 0.029442638158798218, 0.008862595073878765, -0.028539318591356277, 0.010901370085775852, 0.017306337133049965, 0.02298695594072342, 0.024683324620127678, -0.021954039111733437, -0.057187557220458984, 0.0042526316829025745, -0.020522259175777435, -0.05836636573076248, 0.035053618252277374, -0.02031255140900612, 0.020528478547930717, -0.008451596833765507, 0.021694103255867958, 0.017930753529071808, 0.014212710782885551, -0.03432881459593773, 0.010870610363781452, -0.05807207524776459, 0.002418399089947343, 0.0025347049813717604, -0.03868451714515686, -0.00843062438070774, -0.07166961580514908, -0.05016816779971123, -0.010027709417045116, -0.005763706751167774, -0.02524358220398426, -0.007568519562482834, 0.04366230592131615, 0.006288360804319382, 0.007679494563490152, 0.034583598375320435, -0.0062622432596981525, 0.03177335858345032, 0.11281222850084305, -0.002397672738879919, -0.0025029433891177177, -0.0182785727083683, -0.028890851885080338, -0.013639133423566818, 0.006812871899455786, 0.0007302063168026507, 0.020201558247208595, 0.003094310173764825, -0.01686738431453705, 0.03574647754430771, 0.01445409469306469, 0.06424038112163544, 0.029897993430495262, 0.004247378557920456, 0.025864040479063988, 0.009759627282619476, -0.007800233084708452, 0.02088671550154686, -0.00833303201943636, 0.00484407227486372, -0.012287477031350136, 0.053012847900390625, 0.033483851701021194, -0.0021900376304984093, -0.012974847108125687, 0.03570505604147911, 0.02964480221271515, -0.033530376851558685, -0.020259637385606766, -0.015522417612373829, -0.024706589058041573, -0.051832813769578934, -0.037954606115818024, 0.032704491168260574, 0.008287092670798302, -0.04523250460624695, 0.047101203352212906, 0.05193405598402023, -0.01689341478049755, -0.027804803103208542, 0.00757856760174036, 0.03922516852617264, 0.02762416936457157, 0.02139149233698845, -0.01949223317205906, -0.007600491400808096, 0.024110926315188408, -0.0014181627193465829, -0.017201106995344162, 0.003983304835855961, 0.012902075424790382, -0.0016261141281574965, -0.027386320754885674, -0.03728077933192253, -0.02809356339275837, -0.05027380958199501, 0.05276196077466011, -0.044836726039648056, -0.035405877977609634, 0.02411959506571293, -0.053758587688207626, 0.004851288627833128, -0.020627722144126892, 0.005518397316336632, 0.043844711035490036, 0.006524977739900351, -0.001953128492459655, -0.006625910755246878, 0.01546039804816246, -0.01317356526851654, 0.023319613188505173, -0.018587637692689896, -0.019078286364674568, 0.0071097263135015965, 0.02219374291598797, -0.011235914193093777, -0.02539128065109253, -0.048127252608537674, -0.03422826528549194, -0.04958198219537735, -0.012682202272117138, -0.009422610513865948, -0.05889014154672623, -0.08096152544021606, -0.014047685079276562, 0.0036402251571416855, -0.015212724916636944, -0.045413728803396225, 0.026861876249313354, -0.019476696848869324, 0.009941218420863152, -0.000221085618250072, -0.03700876981019974, 0.01231512613594532, 0.0071842968463897705, 0.03674906864762306, 0.03545890375971794, 0.05711435154080391, -0.0018083011964336038, 0.0019241945119574666, 0.012229875661432743, 0.03205302730202675, -0.038579992949962616, -0.055173780769109726, -0.008089015260338783, 0.0063807228580117226, -0.02566399797797203, -0.02116805501282215, -0.009975014254450798, 0.034104879945516586, -0.00475328741595149, 0.0002477108791936189, -0.0741274505853653, -0.058571185916662216, -0.020604321733117104, -0.03267468512058258, 0.012492026202380657, -0.057769156992435455, 0.007356674876064062, -0.046442825347185135, 0.0110750338062644, -0.008486892096698284, -0.02331285923719406, 0.017413953319191933, 0.0002859128871932626, -0.006286837160587311, 0.01091700978577137, -0.0003247930435463786, 0.05844960734248161, 0.0011800721986219287, 0.01217859610915184, -0.01393275335431099, -0.012591235339641571, -0.0006400559796020389, -0.0005139016429893672, 0.013903596438467503, -0.003980893176048994, 0.020830979570746422, -0.021170631051063538, -0.03863498941063881, -0.002660718047991395, 0.010446744970977306, 0.004540122114121914, 0.0051650432869791985, -0.01755204237997532, 0.008908847346901894, 0.005769080016762018, -0.004711685236543417, -0.014908239245414734, 0.011548410169780254, 0.02445157617330551, -0.07284558564424515, 0.004930868279188871, 0.06493543833494186, 0.04543459787964821, -0.0406198613345623, -0.11076648533344269, -0.03438787907361984, -0.005685598123818636, 0.016510097309947014, -0.005590430926531553, -0.026484567672014236, -0.003508538706228137, -0.038476523011922836, 0.022036027163267136, 0.03175580874085426, -0.016738800331950188, 0.035869598388671875, -0.020630888640880585, -0.021615751087665558, 0.026069240644574165, -0.047477178275585175, 0.06979675590991974, -4.4104719563620165e-05, -0.004498153459280729, 0.02601802349090576, -0.02372797764837742, -0.03386650234460831, -0.03703104704618454, -0.02061428129673004, 0.029277082532644272, -0.004088818561285734, -0.010080674663186073, 0.0030852595809847116, -0.05028234049677849, 0.044931408017873764, 0.004535885527729988, 0.00317631964571774, 0.00505051901564002, 0.028827890753746033, -0.05079721659421921, 0.017964381724596024, -0.018089422956109047, 0.07322697341442108, -0.0419028177857399, 0.020936163142323494, 0.009874156676232815, -0.046320024877786636, 0.07954224944114685, 0.05365102365612984, 0.015094676986336708, 0.0684940367937088, -0.07808104902505875, 0.015728382393717766, -0.05922840163111687, 0.03928744047880173, -0.0012733007315546274, 0.0027793797198683023, 0.01989072561264038, 0.001452946220524609, 0.02477055788040161, 0.004874392878264189, -0.010004617273807526, 0.04364040493965149, 0.06469804048538208, 0.019343450665473938, -0.006802588235586882, 0.013676210306584835, 0.06258201599121094, -0.02706267684698105, 0.018690116703510284, -0.03648998588323593, 0.011913140304386616, 0.08417779952287674, -0.020178955048322678, 0.015712298452854156, 0.020427841693162918, -0.008411870338022709, -0.04445770010352135, -0.02871600165963173, -0.019534440711140633, -0.04141993820667267, -0.004889948293566704, -0.009336705319583416, 0.011559003964066505, -0.052420757710933685, 0.005464163143187761, -0.02680482529103756, -0.11799542605876923, 0.023411255329847336, -0.008615643717348576, 0.004124920349568129, -0.003589207073673606, 0.0014425712870433927, -0.032276950776576996, 0.017133021727204323, -0.0046063382178545, -0.0010429162066429853, -0.017291227355599403, 0.006486611906439066, -0.015459819696843624, -0.024412354454398155, -0.03282231092453003, 0.0012189984554424882, -0.014692092314362526, 0.033123552799224854, -0.01944602094590664, 0.024681992828845978, 0.025349197909235954, 0.009814783930778503, -0.03822169452905655, 0.028458649292588234, 0.10325401276350021, -0.037958718836307526, 0.004322040360420942, -0.06187188997864723], [-0.029679913073778152, 0.02317550592124462, 0.04152222350239754, -0.027218492701649666, -0.021493256092071533, 0.04490916430950165, 0.06501781940460205, 0.04579024761915207, 0.001358668552711606, -0.044206567108631134, -0.0441136434674263, -0.013913225382566452, 0.009297499433159828, -0.012569332495331764, 0.007959193550050259, 0.019835498183965683, 0.04488895460963249, 0.025569964200258255, -0.02872822992503643, 0.016643064096570015, 0.013474207371473312, -0.09893176704645157, 0.04672812297940254, -0.004196531139314175, -0.015341483056545258, 0.024432487785816193, -0.02562589943408966, 0.021447714418172836, 0.014670771546661854, 0.01811370626091957, -0.005280763376504183, -0.0018120550084859133, 0.00803122017532587, -0.01531946286559105, -0.015942096710205078, 0.07602548599243164, -0.010123951360583305, 0.01849151961505413, -0.011171849444508553, 0.02545401267707348, -0.013485622592270374, -0.012789147906005383, -0.010148135013878345, 0.004084959160536528, -0.032362695783376694, -0.014495844021439552, 0.010173635557293892, -0.004714128561317921, 0.03175830841064453, -0.015630312263965607, 0.026670239865779877, 0.20702247321605682, 0.07213856279850006, 0.04023907706141472, -0.01018853485584259, 0.06296142935752869, -0.018593767657876015, 0.016352402046322823, 0.0055224294774234295, -0.05140230804681778, -0.04721704497933388, 0.011941454373300076, -0.002475507091730833, -0.07118263095617294, 0.024658411741256714, 0.0378037691116333, -0.026977721601724625, -0.037848103791475296, 0.038835618644952774, 0.013611487112939358, 0.048657238483428955, 0.04083961993455887, 0.001562256133183837, 0.006319539155811071, 0.060888923704624176, 0.0005776629550382495, -0.02134302817285061, -0.03826993331313133, -0.03211880102753639, -0.06181468442082405, -0.025552663952112198, -0.020526733249425888, -0.014147420413792133, 0.01532319188117981, -0.0026461791712790728, 0.053414326161146164, -0.0027735610492527485, 0.060389239341020584, -0.04814569652080536, 0.012273160740733147, 0.0746723860502243, -0.01818877086043358, 0.02249807119369507, 0.0024316846393048763, 0.0017833755118772388, -0.004576191771775484, -0.013930832967162132, 0.027358930557966232, -0.01225974690169096, -0.03719290345907211, 0.031005859375, -0.0006205353420227766, 0.012183398008346558, -0.016101650893688202, -0.00403216015547514, 0.015097584575414658, 0.01722877472639084, -0.05959071218967438, 0.05523091182112694, -0.011754854582250118, 0.007894185371696949, -0.00746137835085392, -0.012764349579811096, 0.03342200443148613, 0.022898755967617035, 0.02946610376238823, -0.0012589053949341178, 0.0242815762758255, 0.030932707712054253, -0.008727291598916054, -0.025001199916005135, 0.0021343801636248827, -0.02688763104379177, -0.027467433363199234, 0.03209028020501137, -0.04429585114121437, 0.028250794857740402, 0.012954037636518478, -0.0023677856661379337, 0.02745663747191429, -0.006383722182363272, -0.01011870801448822, 0.04763374477624893, 0.007599302567541599, 0.02727055922150612, 0.04911980777978897, -0.00571383535861969, -0.030695617198944092, -0.051590435206890106, 0.020783526822924614, -0.014921136200428009, -0.013867618516087532, 0.007244100794196129, 0.02118806168437004, 0.01598197966814041, -0.008939525112509727, -0.02098820172250271, -0.010877154767513275, -0.03878540173172951, 0.017764372751116753, 0.036635152995586395, 0.05696933716535568, -0.01664832793176174, -0.06069441884756088, -0.04561945050954819, -0.0013430265244096518, -0.02510242536664009, 0.0892905741930008, 0.003583875484764576, -0.06144550070166588, 0.0226026251912117, -0.037423666566610336, 0.04055739566683769, -0.002413359237834811, -0.05113609507679939, -0.04811454191803932, 0.02979394979774952, -0.016628623008728027, -0.03373750299215317, -0.04119406268000603, 0.07446689903736115, -0.04530259594321251, 0.01183742843568325, 0.0004136793431825936, -0.00697093503549695, 0.009643926285207272, -0.03461060672998428, 0.008340195752680302, 0.029484713450074196, 0.048582933843135834, 0.027933524921536446, -0.006681731436401606, -0.022713277488946915, -0.018710866570472717, -0.0007760109147056937, -0.017211487516760826, 0.00714371120557189, -0.028534643352031708, -0.011255433782935143, -0.04043767973780632, 0.024785276502370834, 0.008889729157090187, -0.009414439089596272, -0.04481633007526398, -0.07668916136026382, -0.008335680700838566, 0.027961770072579384, -0.0200203824788332, -0.04207877814769745, 0.0355730764567852, 0.00784712377935648, -0.02794666402041912, 0.009223268367350101, 0.010087352246046066, -0.037664689123630524, 0.032062459737062454, -0.005361676216125488, 0.014798286370933056, -0.040958985686302185, 0.02420109324157238, -0.0020085987634956837, -0.007081793155521154, 0.022807536646723747, -0.03951246291399002, -0.0023361481726169586, 0.02032516896724701, -0.024896938353776932, -0.0770866796374321, 0.010664986446499825, 0.0014660165179520845, -0.006320903077721596, -0.03714633360505104, -0.033222660422325134, -0.0001611709303688258, -0.05183086916804314, -0.0015211122808977962, -0.00173083390109241, -0.006196264643222094, -0.011224093846976757, -0.010588278993964195, -0.012957511469721794, 0.06643659621477127, 0.0032128070015460253, 0.05512862652540207, 0.01862674579024315, -0.02573343738913536, 0.015011657029390335, -0.013303486630320549, 0.014883238822221756, 0.017848843708634377, 0.01950307935476303, 0.01713668555021286, -0.003867360530421138, -0.03819848597049713, -0.006519611459225416, -0.01449123676866293, -0.06160114333033562, -0.0312688909471035, -0.0011106034507974982, 0.005584459286183119, -0.005082013551145792, -0.0015120726311579347, -0.003414369886741042, -0.06370175629854202, -0.0286682341247797, 0.0008184920880012214, 0.06301262229681015, 0.017249515280127525, -0.0019534830935299397, -0.012559407390654087, -0.031994856894016266, 0.006768819410353899, 0.020115647464990616, 0.03663304075598717, 0.00459257559850812, 0.023813940584659576, -0.028918243944644928, -0.009023180231451988, -0.012220347300171852, 0.04507390409708023, -0.030002828687429428, -0.017914628610014915, 0.02321380376815796, -0.045341651886701584, 0.004202919080853462, 0.008151653222739697, -0.01703028194606304, -0.016951289027929306, -0.03883693739771843, -0.030282864347100258, -0.03787434473633766, 0.02182718552649021, -0.007785518653690815, 0.03972022980451584, 0.013827580027282238, 0.013482149690389633, -0.005980101879686117, 0.05699360743165016, 0.04392210394144058, -0.013368722051382065, 0.004005413502454758, 0.005764923989772797, -0.04731950908899307, -0.014613824896514416, 0.40971124172210693, -0.04784049466252327, -0.0021592839621007442, 0.008834381587803364, -0.012172115966677666, -0.034491099417209625, 0.0577688105404377, -0.007112462073564529, -0.026162436231970787, -0.006635133642703295, -0.08402132242918015, 0.024979254230856895, -0.043105486780405045, 0.013574331998825073, 0.006264515221118927, -0.006462512072175741, -0.011636300012469292, -0.02075713314116001, 0.01833139918744564, -0.05119026079773903, 0.07645390927791595, 0.0006229218561202288, -0.01943487860262394, -0.022603332996368408, -0.015227273106575012, -0.02966911345720291, 0.008342587389051914, -0.04940143600106239, -0.028391985222697258, -0.027897557243704796, 0.030584387481212616, 0.04701803997159004, -0.04445675015449524, 0.0554928332567215, -0.001475749653764069, -0.05761096253991127, 0.018585670739412308, -0.016996433958411217, -0.05609758943319321, -0.007172613870352507, 0.0270200464874506, -0.023607512935996056, 0.05182318016886711, 0.040277350693941116, -0.04072798043489456, -0.06357105076313019, 0.020417822524905205, 0.0022873079869896173, -0.03166447579860687, -0.010450296103954315, 0.02220527082681656, -0.004995735362172127, 0.054236192256212234, -0.05703816935420036, -0.008800389245152473, -0.04614091292023659, 0.006623859517276287, 0.06015787273645401, 0.030542604625225067, -0.01357131265103817, -7.369781815214083e-05, 0.020362602546811104, 0.004206211306154728, -0.003485999768599868, -0.019505981355905533, 0.01855013519525528, 0.0697951689362526, 0.025254644453525543, -0.05605093762278557, -0.005933959968388081, -0.029832957312464714, -0.053011342883110046, 0.021358899772167206, -0.015791721642017365, -0.041037119925022125, 0.012915659695863724, 0.006304730661213398, -0.04509768635034561, 0.01372070424258709, 0.012286081910133362, -0.006789083126932383, 0.02489999122917652, 0.037385325878858566, 0.0075346617959439754, -0.0007170337485149503, -0.0019878658931702375, -0.0005224853521212935, 0.034776121377944946, -0.014704891480505466, 0.022595111280679703, 0.0165715292096138, -0.03626164421439171, 0.0008109979680739343, -0.041148215532302856, -0.00685778446495533, -0.02852751687169075, -0.004339171107858419, 0.046085111796855927, -0.014484455808997154, 0.08139186352491379, -0.025123516097664833, -0.0035578066017478704, 0.0061984919011592865, -0.011217283084988594, 0.027779612690210342, -0.0336921289563179, -0.05181192234158516, -0.04582852125167847, 0.03659486398100853, 0.02048388123512268, -0.05403757467865944, -0.01055081281810999, -0.05499175190925598, -0.01795594021677971, -0.006930962670594454, 0.05901515856385231, -0.0011982989963144064, 0.030573925003409386, -0.015471413731575012, -0.010048596188426018, 0.018976235762238503, -0.018229978159070015, 0.02232435718178749, 0.024598386138677597, -0.008172347210347652, -0.007763177156448364, -0.04846442490816116, -0.0310455784201622, 0.014067055657505989, -0.022121721878647804, -0.006123746745288372, -0.019243834540247917, 0.0532761886715889, 0.02064308151602745, 0.020332587882876396, 0.016192320734262466, -0.012361429631710052, 0.05143510177731514, -0.011860476806759834, 0.014174070209264755, -0.014824626967310905, -0.03390922397375107, -0.019408395513892174, 0.020998800173401833, -0.009003610350191593, 0.023702146485447884, -0.01977621763944626, -0.03545878455042839, -0.011903248727321625, 0.0004078313068021089, -0.012925363145768642, -0.036647770553827286, 0.05610296130180359, -0.014047961682081223, 0.0017482319381088018, 0.05449288338422775, -0.012040892615914345, -0.06996285170316696, -0.02556947059929371, 0.024779891595244408, -0.036349281668663025, 0.005950360558927059, 0.01817156933248043, -0.03308570384979248, 0.014081376604735851, 0.007356755901128054, 0.04091737046837807, -0.02218455635011196, 0.04862228408455849, -0.005052453838288784, -0.007373108062893152, 0.012969182804226875, 0.024348095059394836, -0.012473654001951218, 0.036035727709531784, -0.01614779606461525, -0.020952820777893066, 0.0031335053499788046, -0.034010034054517746, -0.02740374393761158, 0.00044966189307160676, -0.0013038086472079158, 0.015817413106560707, -0.013423123396933079, 0.005116328131407499, 0.003114529652521014, 0.012097799219191074, 0.011656053364276886, -0.014098010025918484, -0.10028591752052307, 0.009903771802783012, -0.012750058434903622, 0.015615318901836872, 0.00738929957151413, -0.0701351910829544, -0.02950056828558445, -0.03424900025129318, -0.020304366946220398, 0.027407202869653702, 0.0030526109039783478, 0.03400273248553276, -0.010817710310220718, -0.028652722015976906, 0.037135593593120575, 0.005289430730044842, 0.0743216946721077, 0.10104995220899582, -0.001088534714654088, -0.001202646642923355, -0.02290107123553753, -0.002220168709754944, 0.01542539894580841, -0.004459184594452381, 0.0070365495048463345, 0.04095980152487755, -0.016511542722582817, -0.014689178206026554, 0.01925579272210598, 0.024916190654039383, 0.055575355887413025, 0.015161942690610886, -0.030664388090372086, 0.029281113296747208, -0.01572193205356598, -0.012200925499200821, 0.013132063671946526, 0.016197990626096725, -0.020012779161334038, -0.007612231653183699, 0.04034316912293434, 0.03657454252243042, -0.0069002434611320496, 0.002811188343912363, 0.008630132302641869, 0.04323962703347206, -0.05229044333100319, -0.006049041170626879, -0.021489787846803665, 0.03688231110572815, -0.0547761470079422, -0.0302205141633749, 0.05219198763370514, 0.009115714579820633, -0.05429770424962044, 0.06130490452051163, 0.031601447612047195, -0.0004682882863562554, -0.001781698432751, 0.01755620911717415, 0.06213051453232765, 0.0364658422768116, 0.03286902233958244, -0.01860176958143711, -0.03372068703174591, 0.017086759209632874, 0.014347276650369167, -0.04490682855248451, 0.02725747600197792, 0.021675603464245796, 0.03705700859427452, -0.02950117737054825, -0.05396212637424469, -0.03995542600750923, -0.03189833089709282, 0.03763875365257263, -0.05594300478696823, -0.05679250881075859, 0.02034713141620159, -0.07056991755962372, 0.01078718900680542, 0.0032138866372406483, -0.010352219454944134, 0.04193021357059479, -0.037454891949892044, -0.03822684288024902, -0.009544488042593002, -0.014555578120052814, -0.024533960968255997, 0.006884365808218718, -0.0023931171745061874, -0.009390834718942642, 0.011470037512481213, 0.03343071788549423, -0.023391947150230408, -0.03788952901959419, -0.06111793965101242, -0.03199627995491028, -0.06439773738384247, -0.009197898209095001, -0.027353104203939438, -0.04805533215403557, -0.05461619794368744, -0.043066833168268204, 0.00541262561455369, -0.036590468138456345, -0.059838127344846725, 0.019856326282024384, -0.025703515857458115, -0.01872192695736885, 0.016437964513897896, -0.041546884924173355, 0.0177886001765728, -0.0065603796392679214, 0.03183799237012863, 0.003442200133576989, 0.0423671193420887, -0.014509836211800575, -0.00199774163775146, 0.04723597690463066, 0.010344650596380234, -0.045917972922325134, -0.06471848487854004, 0.008672096766531467, 0.02236829698085785, -0.015995021909475327, -0.018433628603816032, -0.006774387788027525, 0.02287961170077324, 5.96947647863999e-05, 0.015142840333282948, -0.061256974935531616, -0.037268973886966705, -0.04551703855395317, -0.024271264672279358, 0.008681616745889187, -0.0534110963344574, 0.05020653083920479, -0.048079077154397964, 0.023727450519800186, -0.013824555091559887, -0.001429895986802876, -0.0153202461078763, -0.02399304509162903, 0.006636415142565966, -0.005697803106158972, 0.019770553335547447, 0.06022179126739502, 0.037350185215473175, 0.0032578681129962206, -0.0014331903075799346, -0.018863579258322716, -0.029152434319257736, -0.016987333074212074, 0.03172939643263817, -0.019662201404571533, 0.016702286899089813, -0.015600462444126606, -0.07080844044685364, 0.008489313535392284, -0.004625842906534672, 0.006724584382027388, 0.00388883869163692, -0.007341558113694191, 0.013463687151670456, -0.010788442566990852, -0.025578897446393967, -0.004090932663530111, 0.008450179360806942, 0.05200488120317459, -0.07636716216802597, 0.011575894430279732, 0.06441207230091095, 0.022218147292733192, -0.04495416209101677, -0.0906631350517273, -0.0425896979868412, -0.014725835993885994, 0.01927023194730282, -0.008655558340251446, 0.001266930135898292, -0.02705192379653454, -0.034108299762010574, -0.014742385596036911, 0.004541796166449785, -0.01985146664083004, 0.03510972857475281, -8.093920769169927e-05, -0.04082738980650902, 0.06351906806230545, -0.04436669126152992, 0.0555812232196331, 0.00349042983725667, -0.018187951296567917, 0.02164303883910179, -0.02301366813480854, -0.011523413471877575, -0.06263816356658936, -0.005379426293075085, 0.00646966090425849, -0.005997388157993555, -0.019664226099848747, 0.0174611397087574, -0.037047192454338074, -0.0002022966800723225, -0.001494759926572442, -0.017932593822479248, 0.011289599351584911, 0.016830630600452423, -0.04019498825073242, 0.013726748526096344, -0.0362703911960125, 0.09348749369382858, -0.010221835225820541, 0.015284798108041286, 0.012477771379053593, -0.04084807634353638, 0.04229379817843437, 0.046720679849386215, -0.001177076599560678, 0.013779999688267708, -0.08678381890058517, 0.009630339220166206, -0.009793668985366821, 0.030487902462482452, 0.012030749581754208, -0.016332164406776428, 0.009005093015730381, -0.012841724790632725, 0.023750051856040955, 0.014633073471486568, 0.009047984145581722, 0.05044998601078987, 0.09308436512947083, -0.0003709344018716365, -0.004565458744764328, 0.01847098581492901, 0.0751010999083519, -0.03384862095117569, 0.023497838526964188, -0.03613409027457237, 0.020740807056427002, 0.0651625543832779, -0.014697332866489887, 0.001100473222322762, 0.02127331681549549, 0.027618272230029106, -0.04425487294793129, -0.014524664729833603, 0.02184961922466755, -0.02870306931436062, 0.029182028025388718, 0.00801839493215084, 0.026554327458143234, -0.03482244163751602, -0.002198468428105116, -0.01774786226451397, -0.11462622135877609, 0.016558881849050522, -0.0050979903899133205, -0.009588943794369698, 0.0009911577217280865, 0.0037022982724010944, -0.04255770891904831, -0.0007835967699065804, -0.0014713113196194172, 0.01325923204421997, -0.053120724856853485, 0.03331631422042847, 0.004004330839961767, -0.012934686616063118, -0.03285586088895798, 0.0015219772467389703, -0.027942169457674026, 0.0473434254527092, 0.007832451723515987, -0.012048891745507717, 0.044574279338121414, 0.021907145157456398, -0.034552201628685, 0.029058320447802544, 0.0778767541050911, -0.018768809735774994, 0.02929691970348358, -0.03659270703792572], [-0.005251146387308836, 0.045257117599248886, 0.02084403671324253, -0.023779626935720444, -0.0403536893427372, 0.05032229423522949, 0.06506357342004776, 0.03032001480460167, 0.003250393085181713, -0.03506724163889885, -0.05305257439613342, -0.030331017449498177, 0.014839861541986465, 0.008322990499436855, 0.004971264861524105, 0.04162579029798508, 0.05995183438062668, 0.021539773792028427, -0.032119572162628174, 0.01257606316357851, 0.005976524669677019, -0.11052582412958145, 0.04548659920692444, -0.0028868960216641426, -0.021976199001073837, 0.023989062756299973, -0.020345985889434814, 0.025008300319314003, -0.0234268419444561, 0.011606307700276375, -0.0009663340169936419, 0.005017142742872238, 0.017400935292243958, -0.033708930015563965, -0.021530374884605408, 0.08452871441841125, 0.008175578899681568, -0.018465479835867882, 0.018823767080903053, 0.035131193697452545, -0.020169174298644066, -0.03083634376525879, -0.0032406134996563196, 0.026643725112080574, -0.008213885128498077, -0.012062598019838333, 0.02843136340379715, -0.014044985175132751, 0.01500059012323618, -0.038007207214832306, 0.012011361308395863, 0.1893121749162674, 0.08857284486293793, 0.0352877713739872, -0.0007002476486377418, 0.08536050468683243, -0.026447448879480362, 0.021057525649666786, -0.01590338721871376, -0.04606619104743004, -0.03739668428897858, -0.00836794450879097, 0.005792304873466492, -0.06233590468764305, 0.0185723677277565, 0.005891244392842054, 0.002162802731618285, -0.04660705104470253, 0.03201650083065033, 0.023209745064377785, 0.04232614114880562, 0.041481226682662964, -0.011579195037484169, 0.0032965654972940683, 0.04098419472575188, 0.00498666986823082, -0.027649955824017525, -0.039310380816459656, -0.010817520320415497, -0.05964938551187515, -0.02778533473610878, -0.016455020755529404, 0.015675345435738564, 0.022094648331403732, -0.013149261474609375, 0.05598088726401329, 0.020772354677319527, 0.06625629216432571, -0.03751343861222267, 0.010464784689247608, 0.06856346130371094, -0.0006954967393539846, 0.02494598738849163, -0.003714002436026931, 0.013918068259954453, 0.016446642577648163, -0.013315754942595959, 0.025869542732834816, -0.008143560029566288, -0.03753277659416199, 0.030614931136369705, 0.0038346610963344574, -0.006800713483244181, -9.002042497741058e-05, -0.012937348335981369, 0.0073794214986264706, -0.002733062021434307, -0.05937705188989639, 0.05914061516523361, -0.00642734719440341, -0.0015140542527660728, 0.01573491096496582, -0.01672838069498539, 0.019573194906115532, 0.021914394572377205, 0.013450143858790398, 0.04035366326570511, 0.03826912119984627, 0.034201957285404205, 0.010982214473187923, -0.005685268901288509, -0.018483230844140053, -0.04152562841773033, -0.03523465618491173, 0.04299117252230644, -0.03371472284197807, 0.024689143523573875, 0.030654655769467354, 0.0023014310281723738, 0.0226842500269413, -0.021149354055523872, 0.0016892643179744482, 0.05007404461503029, 0.00961057934910059, 0.03817347064614296, 0.055633991956710815, -0.006158820353448391, -0.05187978595495224, -0.024428358301520348, 0.02121071331202984, -0.004493508953601122, -0.015871906653046608, 0.011169581674039364, 0.019517675042152405, 0.022284939885139465, -0.015657123178243637, -0.02629832923412323, -0.001676480402238667, -0.026201613247394562, 0.002794740255922079, 0.016121989116072655, 0.049081675708293915, 0.004466071724891663, -0.07954241335391998, -0.03174004703760147, 0.015241207554936409, -0.015325943939387798, 0.09235251694917679, 0.006062116473913193, -0.05322694033384323, 0.030063439160585403, -0.04961520433425903, 0.030609367415308952, 0.01394260860979557, -0.052690539509058, -0.031937260180711746, 0.04613324999809265, -0.007682137656956911, -0.045668553560972214, -0.026774536818265915, 0.0630861148238182, -0.016788655892014503, -0.002413056790828705, 0.0031445675995200872, 0.0042673577554523945, 0.022430164739489555, -0.027778789401054382, 0.012402198277413845, 0.0024102614261209965, 0.04651131108403206, 0.001011952175758779, -0.0023569788318127394, -0.024498863145709038, -0.016098976135253906, -0.014112858101725578, -0.0449010506272316, -0.01902177557349205, -0.017830274999141693, -0.019594158977270126, -0.03513452410697937, -0.013329818844795227, -0.007817446254193783, -0.006367162801325321, -0.049132734537124634, -0.07732474058866501, -0.02602541632950306, 0.04075754061341286, -0.044805858284235, -0.04251375049352646, 0.05738096311688423, -0.015435700304806232, -0.053964219987392426, 0.010712625458836555, 0.007956859655678272, -0.019904151558876038, 0.031051471829414368, -0.004908385686576366, 0.045592885464429855, -0.029763922095298767, 0.016331395134329796, -0.008159685879945755, 0.0018373889615759254, 0.01384133193641901, -0.046938613057136536, -0.010477236472070217, 0.006699213758111, -0.018393924459815025, -0.05065871402621269, -0.01656457409262657, -0.029293613508343697, 0.0043622031807899475, -0.034635789692401886, -0.02951727993786335, -0.002722785808146, -0.03393224626779556, 0.030415376648306847, -0.01494503766298294, 0.030212100595235825, -0.021437568590044975, -0.03542435169219971, 0.003791387192904949, 0.07840188592672348, -0.028181711211800575, 0.041673459112644196, 0.05240774154663086, -0.015445766970515251, 0.022177541628479958, -0.011757591739296913, -0.023191576823592186, 0.007593302521854639, 0.037170130759477615, 0.0204259