Showing preview only (261K chars total). Download the full file or copy to clipboard to get everything.
Repository: naklecha/llama3-from-scratch
Branch: main
Commit: 1b866ac638dc
Files: 4
Total size: 254.3 KB
Directory structure:
gitextract_rxv1jkc6/
├── LICENSE
├── README.md
├── llama3-from-scratch.ipynb
└── requirements.txt
================================================
FILE CONTENTS
================================================
================================================
FILE: LICENSE
================================================
MIT License
Copyright (c) 2024 Nishant Aklecha
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: README.md
================================================
# llama3 implemented from scratch
in this file, i implemented llama3 from scratch, one tensor and matrix multiplication at a time.
<br>
also, im going to load tensors directly from the model file that meta provided for llama3, you need to download the weights before running this file.
here is the offical link to download the weights: https://llama.meta.com/llama-downloads/
<div>
<img src="images/archi.png"/>
</div>
## tokenizer
im not going to implement a bpe tokenizer (but andrej karpathy has a really clean implementation)
<br>
link to his implementation: https://github.com/karpathy/minbpe
<div>
<img src="images/karpathyminbpe.png" width="600"/>
</div>
```python
from pathlib import Path
import tiktoken
from tiktoken.load import load_tiktoken_bpe
import torch
import json
import matplotlib.pyplot as plt
tokenizer_path = "Meta-Llama-3-8B/tokenizer.model"
special_tokens = [
"<|begin_of_text|>",
"<|end_of_text|>",
"<|reserved_special_token_0|>",
"<|reserved_special_token_1|>",
"<|reserved_special_token_2|>",
"<|reserved_special_token_3|>",
"<|start_header_id|>",
"<|end_header_id|>",
"<|reserved_special_token_4|>",
"<|eot_id|>", # end of turn
] + [f"<|reserved_special_token_{i}|>" for i in range(5, 256 - 5)]
mergeable_ranks = load_tiktoken_bpe(tokenizer_path)
tokenizer = tiktoken.Encoding(
name=Path(tokenizer_path).name,
pat_str=r"(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\r\n\p{L}\p{N}]?\p{L}+|\p{N}{1,3}| ?[^\s\p{L}\p{N}]+[\r\n]*|\s*[\r\n]+|\s+(?!\S)|\s+",
mergeable_ranks=mergeable_ranks,
special_tokens={token: len(mergeable_ranks) + i for i, token in enumerate(special_tokens)},
)
tokenizer.decode(tokenizer.encode("hello world!"))
```
'hello world!'
## reading the model file
normally, reading this depends on how the model classes are written and the variable names inside them.
<br>
but since we are implementing llama3 from scratch we will read the file one tensor at a time.
<div>
<img src="images/model.png" width="600"/>
</div>
```python
model = torch.load("Meta-Llama-3-8B/consolidated.00.pth")
print(json.dumps(list(model.keys())[:20], indent=4))
```
[
"tok_embeddings.weight",
"layers.0.attention.wq.weight",
"layers.0.attention.wk.weight",
"layers.0.attention.wv.weight",
"layers.0.attention.wo.weight",
"layers.0.feed_forward.w1.weight",
"layers.0.feed_forward.w3.weight",
"layers.0.feed_forward.w2.weight",
"layers.0.attention_norm.weight",
"layers.0.ffn_norm.weight",
"layers.1.attention.wq.weight",
"layers.1.attention.wk.weight",
"layers.1.attention.wv.weight",
"layers.1.attention.wo.weight",
"layers.1.feed_forward.w1.weight",
"layers.1.feed_forward.w3.weight",
"layers.1.feed_forward.w2.weight",
"layers.1.attention_norm.weight",
"layers.1.ffn_norm.weight",
"layers.2.attention.wq.weight"
]
```python
with open("Meta-Llama-3-8B/params.json", "r") as f:
config = json.load(f)
config
```
{'dim': 4096,
'n_layers': 32,
'n_heads': 32,
'n_kv_heads': 8,
'vocab_size': 128256,
'multiple_of': 1024,
'ffn_dim_multiplier': 1.3,
'norm_eps': 1e-05,
'rope_theta': 500000.0}
## we use this config to infer details about the model like
1. the model has 32 transformer layers
2. each multi-head attention block has 32 heads
3. the vocab size and so on
```python
dim = config["dim"]
n_layers = config["n_layers"]
n_heads = config["n_heads"]
n_kv_heads = config["n_kv_heads"]
vocab_size = config["vocab_size"]
multiple_of = config["multiple_of"]
ffn_dim_multiplier = config["ffn_dim_multiplier"]
norm_eps = config["norm_eps"]
rope_theta = torch.tensor(config["rope_theta"])
```
## converting text to tokens
here we use tiktoken (i think an openai library) as the tokenizer
<div>
<img src="images/tokens.png" width="600"/>
</div>
```python
prompt = "the answer to the ultimate question of life, the universe, and everything is "
tokens = [128000] + tokenizer.encode(prompt)
print(tokens)
tokens = torch.tensor(tokens)
prompt_split_as_tokens = [tokenizer.decode([token.item()]) for token in tokens]
print(prompt_split_as_tokens)
```
[128000, 1820, 4320, 311, 279, 17139, 3488, 315, 2324, 11, 279, 15861, 11, 323, 4395, 374, 220]
['<|begin_of_text|>', 'the', ' answer', ' to', ' the', ' ultimate', ' question', ' of', ' life', ',', ' the', ' universe', ',', ' and', ' everything', ' is', ' ']
## converting tokens to their embedding
IM SORRY but this is the only part of the codebase where i use an inbuilt neural network module
<br>
anyway, so our [17x1] tokens are now [17x4096], i.e. 17 embeddings (one for each token) of length 4096
<br>
<br>
note: keep track of the shapes, it makes it much easier to understand everything
<div>
<img src="images/embeddings.png" width="600"/>
</div>
```python
embedding_layer = torch.nn.Embedding(vocab_size, dim)
embedding_layer.weight.data.copy_(model["tok_embeddings.weight"])
token_embeddings_unnormalized = embedding_layer(tokens).to(torch.bfloat16)
token_embeddings_unnormalized.shape
```
torch.Size([17, 4096])
## we then normalize the embedding using rms normalization
please, note after this step the shapes dont change, the values are just normalized
<br>
things to keep in mind, we need a norm_eps (from config) because we dont want to accidently set rms to 0 and divide by 0
<br>
here is the formula:
<div>
<img src="images/rms.png" width="600"/>
</div>
```python
# def rms_norm(tensor, norm_weights):
# rms = (tensor.pow(2).mean(-1, keepdim=True) + norm_eps)**0.5
# return tensor * (norm_weights / rms)
def rms_norm(tensor, norm_weights):
return (tensor * torch.rsqrt(tensor.pow(2).mean(-1, keepdim=True) + norm_eps)) * norm_weights
```
# building the first first layer of the transformer
### normalization
you will see me accessing layer.0 from the model dict (this is the first layer)
<br>
anyway, so after normalizing our shapes are still [17x4096] same as embedding but normalized
<div>
<img src="images/norm.png" width="600"/>
</div>
```python
token_embeddings = rms_norm(token_embeddings_unnormalized, model["layers.0.attention_norm.weight"])
token_embeddings.shape
```
torch.Size([17, 4096])
### attention implemented from scratch
let's load the attention heads of the first layer of the transformer
<div>
<img src="images/qkv.png" width="600"/>
</div>
<br>
> when we load the query, key, value and output vectors from the model we notice the shapes to be [4096x4096], [1024x4096], [1024x4096], [4096x4096]
<br>
> at first glance this is weird because ideally we want each q,k,v and o for each head individually
<br>
> the authors of the code bundled them togeather because its easy it helps parallize attention head multiplication.
<br>
> im going to unwrap everything...
```python
print(
model["layers.0.attention.wq.weight"].shape,
model["layers.0.attention.wk.weight"].shape,
model["layers.0.attention.wv.weight"].shape,
model["layers.0.attention.wo.weight"].shape
)
```
torch.Size([4096, 4096]) torch.Size([1024, 4096]) torch.Size([1024, 4096]) torch.Size([4096, 4096])
### unwrapping query
in the next section we will unwrap the queries from multiple attention heads, the resulting shape is [32x128x4096]
<br><br>
here, 32 is the number of attention heads in llama3, 128 is the size of the query vector and 4096 is the size of the token embedding
```python
q_layer0 = model["layers.0.attention.wq.weight"]
head_dim = q_layer0.shape[0] // n_heads
q_layer0 = q_layer0.view(n_heads, head_dim, dim)
q_layer0.shape
```
torch.Size([32, 128, 4096])
### im going to implement the first head of the first layer
here i access the query weight matrix first head of the first layer, the size of this query weight matrix is [128x4096]
```python
q_layer0_head0 = q_layer0[0]
q_layer0_head0.shape
```
torch.Size([128, 4096])
### we now multiply the query weights with the token embedding, to recive a query for the token
here you can see the resulting shape is [17x128], this is because we have 17 tokens and for each token there is a 128 length query.
<div>
<img src="images/q_per_token.png" width="600"/>
</div>
```python
q_per_token = torch.matmul(token_embeddings, q_layer0_head0.T)
q_per_token.shape
```
torch.Size([17, 128])
## positioning encoding
we are now at a stage where we have a query vector for each token in our prompt, but if you think about it -- the indivitually query vector has no idea about the position in the prompt.
<br><br>
query: "the answer to the ultimate question of life, the universe, and everything is "
<br><br>
in our prompt we have used "the" three times, we need the query vectors of all 3 "the" tokens to have different query vectors (each of size [1x128]) based on their positions in the query. we perform these rotations using RoPE (rotory positional embedding).
<br><br>
### RoPE
watch this video (this is what i watched) to understand the math.
https://www.youtube.com/watch?v=o29P0Kpobz0&t=530s
<div>
<img src="images/rope.png" width="600"/>
</div>
```python
q_per_token_split_into_pairs = q_per_token.float().view(q_per_token.shape[0], -1, 2)
q_per_token_split_into_pairs.shape
```
torch.Size([17, 64, 2])
in the above step, we split the query vectors into pairs, we apply a rotational angle shift to each pair!
<br><br>
we now have a vector of size [17x64x2], this is the 128 length queries split into 64 pairs for each token in the prompt! each of those 64 pairs will be rotated by m*(theta) where m is the position of the token for which we are rotating the query!
<div>
<img src="images/qsplit.png" width="600"/>
</div>
## using dot product of complex numbers to rotate a vector
<div>
<img src="images/freq_cis.png" width="600"/>
</div>
```python
zero_to_one_split_into_64_parts = torch.tensor(range(64))/64
zero_to_one_split_into_64_parts
```
tensor([0.0000, 0.0156, 0.0312, 0.0469, 0.0625, 0.0781, 0.0938, 0.1094, 0.1250,
0.1406, 0.1562, 0.1719, 0.1875, 0.2031, 0.2188, 0.2344, 0.2500, 0.2656,
0.2812, 0.2969, 0.3125, 0.3281, 0.3438, 0.3594, 0.3750, 0.3906, 0.4062,
0.4219, 0.4375, 0.4531, 0.4688, 0.4844, 0.5000, 0.5156, 0.5312, 0.5469,
0.5625, 0.5781, 0.5938, 0.6094, 0.6250, 0.6406, 0.6562, 0.6719, 0.6875,
0.7031, 0.7188, 0.7344, 0.7500, 0.7656, 0.7812, 0.7969, 0.8125, 0.8281,
0.8438, 0.8594, 0.8750, 0.8906, 0.9062, 0.9219, 0.9375, 0.9531, 0.9688,
0.9844])
```python
freqs = 1.0 / (rope_theta ** zero_to_one_split_into_64_parts)
freqs
```
tensor([1.0000e+00, 8.1462e-01, 6.6360e-01, 5.4058e-01, 4.4037e-01, 3.5873e-01,
2.9223e-01, 2.3805e-01, 1.9392e-01, 1.5797e-01, 1.2869e-01, 1.0483e-01,
8.5397e-02, 6.9566e-02, 5.6670e-02, 4.6164e-02, 3.7606e-02, 3.0635e-02,
2.4955e-02, 2.0329e-02, 1.6560e-02, 1.3490e-02, 1.0990e-02, 8.9523e-03,
7.2927e-03, 5.9407e-03, 4.8394e-03, 3.9423e-03, 3.2114e-03, 2.6161e-03,
2.1311e-03, 1.7360e-03, 1.4142e-03, 1.1520e-03, 9.3847e-04, 7.6450e-04,
6.2277e-04, 5.0732e-04, 4.1327e-04, 3.3666e-04, 2.7425e-04, 2.2341e-04,
1.8199e-04, 1.4825e-04, 1.2077e-04, 9.8381e-05, 8.0143e-05, 6.5286e-05,
5.3183e-05, 4.3324e-05, 3.5292e-05, 2.8750e-05, 2.3420e-05, 1.9078e-05,
1.5542e-05, 1.2660e-05, 1.0313e-05, 8.4015e-06, 6.8440e-06, 5.5752e-06,
4.5417e-06, 3.6997e-06, 3.0139e-06, 2.4551e-06])
```python
freqs_for_each_token = torch.outer(torch.arange(17), freqs)
freqs_cis = torch.polar(torch.ones_like(freqs_for_each_token), freqs_for_each_token)
freqs_cis.shape
# viewing tjhe third row of freqs_cis
value = freqs_cis[3]
plt.figure()
for i, element in enumerate(value[:17]):
plt.plot([0, element.real], [0, element.imag], color='blue', linewidth=1, label=f"Index: {i}")
plt.annotate(f"{i}", xy=(element.real, element.imag), color='red')
plt.xlabel('Real')
plt.ylabel('Imaginary')
plt.title('Plot of one row of freqs_cis')
plt.show()
```

### now that we have a complex number (the angle change vector) for every token's query element
we can convert our queries (the one we split into pairs) as complex numbers and then dot product to rotate the query based on the position
<br>
honeslty this is beautiful to think about :)
```python
q_per_token_as_complex_numbers = torch.view_as_complex(q_per_token_split_into_pairs)
q_per_token_as_complex_numbers.shape
```
torch.Size([17, 64])
```python
q_per_token_as_complex_numbers_rotated = q_per_token_as_complex_numbers * freqs_cis
q_per_token_as_complex_numbers_rotated.shape
```
torch.Size([17, 64])
### after rotated vector is obtained
we can get back our the queries as pairs by viewing the complex numbers as real numbers again
```python
q_per_token_split_into_pairs_rotated = torch.view_as_real(q_per_token_as_complex_numbers_rotated)
q_per_token_split_into_pairs_rotated.shape
```
torch.Size([17, 64, 2])
the rotated pairs are now merged, we now have a new query vector (rotated query vector) that is of the shape [17x128] where 17 is the number of tokens and the 128 is the dim of the query vector
```python
q_per_token_rotated = q_per_token_split_into_pairs_rotated.view(q_per_token.shape)
q_per_token_rotated.shape
```
torch.Size([17, 128])
# keys (almost the same as queries)
<div>
<img src="images/keys.png" width="600px"/>
</div>
im lazy as fuck, so im not going to go through the math for keys, the only things you need to keep in mind are:
<br>
> keys generate key vectors also of dimention 128
<br>
> keys have only 1/4th the number of the weights as queries, this is because the weights for keys are shared across 4 heads at a time, to reduce the number of computations need
<br>
> keys are also rotated to add positional info, just like queries because of the same reasons
```python
k_layer0 = model["layers.0.attention.wk.weight"]
k_layer0 = k_layer0.view(n_kv_heads, k_layer0.shape[0] // n_kv_heads, dim)
k_layer0.shape
```
torch.Size([8, 128, 4096])
```python
k_layer0_head0 = k_layer0[0]
k_layer0_head0.shape
```
torch.Size([128, 4096])
```python
k_per_token = torch.matmul(token_embeddings, k_layer0_head0.T)
k_per_token.shape
```
torch.Size([17, 128])
```python
k_per_token_split_into_pairs = k_per_token.float().view(k_per_token.shape[0], -1, 2)
k_per_token_split_into_pairs.shape
```
torch.Size([17, 64, 2])
```python
k_per_token_as_complex_numbers = torch.view_as_complex(k_per_token_split_into_pairs)
k_per_token_as_complex_numbers.shape
```
torch.Size([17, 64])
```python
k_per_token_split_into_pairs_rotated = torch.view_as_real(k_per_token_as_complex_numbers * freqs_cis)
k_per_token_split_into_pairs_rotated.shape
```
torch.Size([17, 64, 2])
```python
k_per_token_rotated = k_per_token_split_into_pairs_rotated.view(k_per_token.shape)
k_per_token_rotated.shape
```
torch.Size([17, 128])
## at this stage now have both the rotated values of queries and keys, for each token.
<div>
<img src="images/keys0.png" width="600px"/>
</div>
each of the queries and keys are now of shape [17x128].
## in the next step we will multiply the queries and key matrices
doing this will give us a score mapping each token with one another
<br>
this score describes how well each token's query relates to the each tokens's key.
THIS IS SELF ATTENTION :)
<br>
the shape of the attention score matrix (qk_per_token) is [17x17] where 17 is the number of tokens in the prompt
<div>
<img src="images/qkmatmul.png" width="600px"/>
</div>
```python
qk_per_token = torch.matmul(q_per_token_rotated, k_per_token_rotated.T)/(head_dim)**0.5
qk_per_token.shape
```
torch.Size([17, 17])
# we now have to mask query key scores
during the training process of llama3, the future token qk scores are masked.
<br>
why? because during training we only learn to predict tokens using past tokens.
<br>
as a result, during inference we set the future tokens to zero.
<div>
<img src="images/mask.png" width="600px"/>
</div>
```python
def display_qk_heatmap(qk_per_token):
_, ax = plt.subplots()
im = ax.imshow(qk_per_token.to(float).detach(), cmap='viridis')
ax.set_xticks(range(len(prompt_split_as_tokens)))
ax.set_yticks(range(len(prompt_split_as_tokens)))
ax.set_xticklabels(prompt_split_as_tokens)
ax.set_yticklabels(prompt_split_as_tokens)
ax.figure.colorbar(im, ax=ax)
display_qk_heatmap(qk_per_token)
```

```python
mask = torch.full((len(tokens), len(tokens)), float("-inf"), device=tokens.device)
mask = torch.triu(mask, diagonal=1)
mask
```
tensor([[0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],
[0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],
[0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],
[0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],
[0., 0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],
[0., 0., 0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],
[0., 0., 0., 0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],
[0., 0., 0., 0., 0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],
[0., 0., 0., 0., 0., 0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., -inf, -inf, -inf, -inf],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., -inf, -inf, -inf],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., -inf, -inf],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., -inf],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]])
```python
qk_per_token_after_masking = qk_per_token + mask
display_qk_heatmap(qk_per_token_after_masking)
```

<div>
<img src="images/softmax.png" width="600px"/>
</div>
```python
qk_per_token_after_masking_after_softmax = torch.nn.functional.softmax(qk_per_token_after_masking, dim=1).to(torch.bfloat16)
display_qk_heatmap(qk_per_token_after_masking_after_softmax)
```

## values (almost the end of attention)
<div>
<img src="images/value.png" width="600px"/>
</div>
these scores (0-1) are used to determine how much of value matrix is used per token
<br>
> just like keys, value weights are also shared acorss every 4 attention heads (to save computation)
<br>
> as a result, the shape of the value weight matrix below is [8x128x4096]
```python
v_layer0 = model["layers.0.attention.wv.weight"]
v_layer0 = v_layer0.view(n_kv_heads, v_layer0.shape[0] // n_kv_heads, dim)
v_layer0.shape
```
torch.Size([8, 128, 4096])
the first layer, first head value weight matrix is given below
```python
v_layer0_head0 = v_layer0[0]
v_layer0_head0.shape
```
torch.Size([128, 4096])
## value vectors
<div>
<img src="images/v0.png" width="600px"/>
</div>
we now use the value weghts to get the attention values per token, this is of size [17x128] where 17 is the number of tokens in the prompt and 128 is the dim of the value vector per token
```python
v_per_token = torch.matmul(token_embeddings, v_layer0_head0.T)
v_per_token.shape
```
torch.Size([17, 128])
## attention
<div>
<img src="images/attention.png" width="600px"/>
</div>
the resultant attention vector after multipying with the values per token is of shape [17*128]
```python
qkv_attention = torch.matmul(qk_per_token_after_masking_after_softmax, v_per_token)
qkv_attention.shape
```
torch.Size([17, 128])
# multi head attention
<div>
<img src="images/heads.png" width="600px"/>
</div>
WE NOW HAVE THE ATTENTION VALUE OF THE FIRST LAYER AND FIRST HEAD
<br>
now im going to run a loop and perform the exact same math as the cells above but for every head in the first layer
```python
qkv_attention_store = []
for head in range(n_heads):
q_layer0_head = q_layer0[head]
k_layer0_head = k_layer0[head//4] # key weights are shared across 4 heads
v_layer0_head = v_layer0[head//4] # value weights are shared across 4 heads
q_per_token = torch.matmul(token_embeddings, q_layer0_head.T)
k_per_token = torch.matmul(token_embeddings, k_layer0_head.T)
v_per_token = torch.matmul(token_embeddings, v_layer0_head.T)
q_per_token_split_into_pairs = q_per_token.float().view(q_per_token.shape[0], -1, 2)
q_per_token_as_complex_numbers = torch.view_as_complex(q_per_token_split_into_pairs)
q_per_token_split_into_pairs_rotated = torch.view_as_real(q_per_token_as_complex_numbers * freqs_cis[:len(tokens)])
q_per_token_rotated = q_per_token_split_into_pairs_rotated.view(q_per_token.shape)
k_per_token_split_into_pairs = k_per_token.float().view(k_per_token.shape[0], -1, 2)
k_per_token_as_complex_numbers = torch.view_as_complex(k_per_token_split_into_pairs)
k_per_token_split_into_pairs_rotated = torch.view_as_real(k_per_token_as_complex_numbers * freqs_cis[:len(tokens)])
k_per_token_rotated = k_per_token_split_into_pairs_rotated.view(k_per_token.shape)
qk_per_token = torch.matmul(q_per_token_rotated, k_per_token_rotated.T)/(128)**0.5
mask = torch.full((len(tokens), len(tokens)), float("-inf"), device=tokens.device)
mask = torch.triu(mask, diagonal=1)
qk_per_token_after_masking = qk_per_token + mask
qk_per_token_after_masking_after_softmax = torch.nn.functional.softmax(qk_per_token_after_masking, dim=1).to(torch.bfloat16)
qkv_attention = torch.matmul(qk_per_token_after_masking_after_softmax, v_per_token)
qkv_attention = torch.matmul(qk_per_token_after_masking_after_softmax, v_per_token)
qkv_attention_store.append(qkv_attention)
len(qkv_attention_store)
```
32
<div>
<img src="images/stacked.png" width="600px"/>
</div>
we now have a the qkv_attention matrix for all 32 heads on the first layer, next im going to merge all attention scores into one large matrix of size [17x4096]
<br>
we are almost at the end :)
```python
stacked_qkv_attention = torch.cat(qkv_attention_store, dim=-1)
stacked_qkv_attention.shape
```
torch.Size([17, 4096])
# weight matrix, one of the final steps
<div>
<img src="images/weightmatrix.png" width="600px"/>
</div>
one of the last things to do for a layer 0 attention is, is to multiply the weight matrix of the
```python
w_layer0 = model["layers.0.attention.wo.weight"]
w_layer0.shape
```
torch.Size([4096, 4096])
### this is a simple linear layer, so we just matmul
```python
embedding_delta = torch.matmul(stacked_qkv_attention, w_layer0.T)
embedding_delta.shape
```
torch.Size([17, 4096])
<div>
<img src="images/afterattention.png" width="600px"/>
</div>
we now have the change in the embedding value after attention, that should be adding to the original token embeddings
```python
embedding_after_edit = token_embeddings_unnormalized + embedding_delta
embedding_after_edit.shape
```
torch.Size([17, 4096])
## we normalize and then run a feed forward neural network through the embedding delta
<div>
<img src="images/norm_after.png" width="600px"/>
</div>
```python
embedding_after_edit_normalized = rms_norm(embedding_after_edit, model["layers.0.ffn_norm.weight"])
embedding_after_edit_normalized.shape
```
torch.Size([17, 4096])
## loading the ff weights and implementing the feed forward network
<div>
<img src="images/swiglu.png" width="600px"/>
</div>
in llama3, they used a SwiGLU feedforward network, this network architecture is really good at adding non linearity when needed by the model.
<br>
its pretty standard to use this feed forward network architecture in llms these days
```python
w1 = model["layers.0.feed_forward.w1.weight"]
w2 = model["layers.0.feed_forward.w2.weight"]
w3 = model["layers.0.feed_forward.w3.weight"]
output_after_feedforward = torch.matmul(torch.functional.F.silu(torch.matmul(embedding_after_edit_normalized, w1.T)) * torch.matmul(embedding_after_edit_normalized, w3.T), w2.T)
output_after_feedforward.shape
```
torch.Size([17, 4096])
# WE FINALLY HAVE NEW EDITED EMBEDDINGS FOR EACH TOKEN AFTER THE FIRST LAYER
just 31 more layers to go before we are done (one for loop away)
<br>
you can imagine this edited embedding as having information about all queries asked on the first layer
<br>
now each layer will encode more and more complex queries on the quesions asked, until we have an embedding that knows everything about the next token that we need.
```python
layer_0_embedding = embedding_after_edit+output_after_feedforward
layer_0_embedding.shape
```
torch.Size([17, 4096])
# god, everything all at once
<div>
<img src="images/god.png" width="600px"/>
</div>
yep, this is it. everything we did before, all at once, for every single layer.
<br>
# have fun reading :)
```python
final_embedding = token_embeddings_unnormalized
for layer in range(n_layers):
qkv_attention_store = []
layer_embedding_norm = rms_norm(final_embedding, model[f"layers.{layer}.attention_norm.weight"])
q_layer = model[f"layers.{layer}.attention.wq.weight"]
q_layer = q_layer.view(n_heads, q_layer.shape[0] // n_heads, dim)
k_layer = model[f"layers.{layer}.attention.wk.weight"]
k_layer = k_layer.view(n_kv_heads, k_layer.shape[0] // n_kv_heads, dim)
v_layer = model[f"layers.{layer}.attention.wv.weight"]
v_layer = v_layer.view(n_kv_heads, v_layer.shape[0] // n_kv_heads, dim)
w_layer = model[f"layers.{layer}.attention.wo.weight"]
for head in range(n_heads):
q_layer_head = q_layer[head]
k_layer_head = k_layer[head//4]
v_layer_head = v_layer[head//4]
q_per_token = torch.matmul(layer_embedding_norm, q_layer_head.T)
k_per_token = torch.matmul(layer_embedding_norm, k_layer_head.T)
v_per_token = torch.matmul(layer_embedding_norm, v_layer_head.T)
q_per_token_split_into_pairs = q_per_token.float().view(q_per_token.shape[0], -1, 2)
q_per_token_as_complex_numbers = torch.view_as_complex(q_per_token_split_into_pairs)
q_per_token_split_into_pairs_rotated = torch.view_as_real(q_per_token_as_complex_numbers * freqs_cis)
q_per_token_rotated = q_per_token_split_into_pairs_rotated.view(q_per_token.shape)
k_per_token_split_into_pairs = k_per_token.float().view(k_per_token.shape[0], -1, 2)
k_per_token_as_complex_numbers = torch.view_as_complex(k_per_token_split_into_pairs)
k_per_token_split_into_pairs_rotated = torch.view_as_real(k_per_token_as_complex_numbers * freqs_cis)
k_per_token_rotated = k_per_token_split_into_pairs_rotated.view(k_per_token.shape)
qk_per_token = torch.matmul(q_per_token_rotated, k_per_token_rotated.T)/(128)**0.5
mask = torch.full((len(token_embeddings_unnormalized), len(token_embeddings_unnormalized)), float("-inf"))
mask = torch.triu(mask, diagonal=1)
qk_per_token_after_masking = qk_per_token + mask
qk_per_token_after_masking_after_softmax = torch.nn.functional.softmax(qk_per_token_after_masking, dim=1).to(torch.bfloat16)
qkv_attention = torch.matmul(qk_per_token_after_masking_after_softmax, v_per_token)
qkv_attention_store.append(qkv_attention)
stacked_qkv_attention = torch.cat(qkv_attention_store, dim=-1)
w_layer = model[f"layers.{layer}.attention.wo.weight"]
embedding_delta = torch.matmul(stacked_qkv_attention, w_layer.T)
embedding_after_edit = final_embedding + embedding_delta
embedding_after_edit_normalized = rms_norm(embedding_after_edit, model[f"layers.{layer}.ffn_norm.weight"])
w1 = model[f"layers.{layer}.feed_forward.w1.weight"]
w2 = model[f"layers.{layer}.feed_forward.w2.weight"]
w3 = model[f"layers.{layer}.feed_forward.w3.weight"]
output_after_feedforward = torch.matmul(torch.functional.F.silu(torch.matmul(embedding_after_edit_normalized, w1.T)) * torch.matmul(embedding_after_edit_normalized, w3.T), w2.T)
final_embedding = embedding_after_edit+output_after_feedforward
```
# we now have the final embedding, the best guess the model could make about the next token
the shape of the embedding is the same as regular token embeddings [17x4096] where 17 is the number of tokens and 4096 is the embedding dim
<div>
<img src="images/last_norm.png" width="600px"/>
</div>
```python
final_embedding = rms_norm(final_embedding, model["norm.weight"])
final_embedding.shape
```
torch.Size([17, 4096])
# finally, lets decode the embedding into the token value
<div>
<img src="images/finallayer.png" width="600px"/>
</div>
we will use the output decoder to convert the final embedding into a token
```python
model["output.weight"].shape
```
torch.Size([128256, 4096])
# we use the embedding of the last token to predict the next value
hopefully in our case, 42 :)
note: 42 is the answer to "the answer to the ultimate question of life, the universe, and everything is ", according to the book "hitchhiker's guide to the galaxy", most mordern llms would answer with 42 here, which should validate our entire code! wish me luck :)
```python
logits = torch.matmul(final_embedding[-1], model["output.weight"].T)
logits.shape
```
torch.Size([128256])
### the model predicted token number 2983 as the next token, is this the token number for 42?
IM HYPING YOU UP, this is the last cell of code, hopefully you had fun :)
```python
next_token = torch.argmax(logits, dim=-1)
next_token
```
tensor(2983)
# lets fucking go
<div>
<img src="images/42.png" width="600px"/>
</div>
```python
tokenizer.decode([next_token.item()])
```
'42'
# thank you, i love you :)
This is the end. Hopefully you enjoyed reading it!
If you want to support my work
1. follow me on twitter https://twitter.com/naklecha
2. or, buy me a coffee [https://www.buymeacoffee.com/naklecha](https://www.buymeacoffee.com/naklecha)
Honestly, if you made it this far you already made my day :)
## what motivates me?
My friends and I are on a mission - to make research more accessible!
We created a research lab called A10 - [AAAAAAAAAA.org](http://aaaaaaaaaa.org/)
A10 twitter - https://twitter.com/aaaaaaaaaaorg
our thesis:
<div>
<img src="images/a10.png" width="600px"/>
</div>
================================================
FILE: llama3-from-scratch.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# llama3 implemented from scratch\n",
"in this file, i implemented llama3 from scratch, one tensor and matrix multiplication at a time.\n",
"<br>\n",
"also, im going to load tensors directly from the model file that meta provided for llama3, you need to download the weights before running this file.\n",
"here is the offical link to download the weights: https://llama.meta.com/llama-downloads/\n",
"\n",
"<div>\n",
" <img src=\"images/archi.png\"/>\n",
"</div>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## tokenizer\n",
"im not going to implement a bpe tokenizer (but andrej karpathy has a really clean implementation)\n",
"<br>\n",
"link to his implementation: https://github.com/karpathy/minbpe\n",
"\n",
"<div>\n",
" <img src=\"images/karpathyminbpe.png\" width=\"600\"/>\n",
"</div>\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'hello world!'"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from pathlib import Path\n",
"import tiktoken\n",
"from tiktoken.load import load_tiktoken_bpe\n",
"import torch\n",
"import json\n",
"import matplotlib.pyplot as plt\n",
"\n",
"tokenizer_path = \"Meta-Llama-3-8B/tokenizer.model\"\n",
"special_tokens = [\n",
" \"<|begin_of_text|>\",\n",
" \"<|end_of_text|>\",\n",
" \"<|reserved_special_token_0|>\",\n",
" \"<|reserved_special_token_1|>\",\n",
" \"<|reserved_special_token_2|>\",\n",
" \"<|reserved_special_token_3|>\",\n",
" \"<|start_header_id|>\",\n",
" \"<|end_header_id|>\",\n",
" \"<|reserved_special_token_4|>\",\n",
" \"<|eot_id|>\", # end of turn\n",
" ] + [f\"<|reserved_special_token_{i}|>\" for i in range(5, 256 - 5)]\n",
"mergeable_ranks = load_tiktoken_bpe(tokenizer_path)\n",
"tokenizer = tiktoken.Encoding(\n",
" name=Path(tokenizer_path).name,\n",
" pat_str=r\"(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?\\p{L}+|\\p{N}{1,3}| ?[^\\s\\p{L}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+\",\n",
" mergeable_ranks=mergeable_ranks,\n",
" special_tokens={token: len(mergeable_ranks) + i for i, token in enumerate(special_tokens)},\n",
")\n",
"\n",
"tokenizer.decode(tokenizer.encode(\"hello world!\"))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## reading the model file\n",
"normally, reading this depends on how the model classes are written and the variable names inside them.\n",
"<br>\n",
"but since we are implementing llama3 from scratch we will read the file one tensor at a time.\n",
"<div>\n",
" <img src=\"images/model.png\" width=\"600\"/>\n",
"</div>"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[\n",
" \"tok_embeddings.weight\",\n",
" \"layers.0.attention.wq.weight\",\n",
" \"layers.0.attention.wk.weight\",\n",
" \"layers.0.attention.wv.weight\",\n",
" \"layers.0.attention.wo.weight\",\n",
" \"layers.0.feed_forward.w1.weight\",\n",
" \"layers.0.feed_forward.w3.weight\",\n",
" \"layers.0.feed_forward.w2.weight\",\n",
" \"layers.0.attention_norm.weight\",\n",
" \"layers.0.ffn_norm.weight\",\n",
" \"layers.1.attention.wq.weight\",\n",
" \"layers.1.attention.wk.weight\",\n",
" \"layers.1.attention.wv.weight\",\n",
" \"layers.1.attention.wo.weight\",\n",
" \"layers.1.feed_forward.w1.weight\",\n",
" \"layers.1.feed_forward.w3.weight\",\n",
" \"layers.1.feed_forward.w2.weight\",\n",
" \"layers.1.attention_norm.weight\",\n",
" \"layers.1.ffn_norm.weight\",\n",
" \"layers.2.attention.wq.weight\"\n",
"]\n"
]
}
],
"source": [
"model = torch.load(\"Meta-Llama-3-8B/consolidated.00.pth\")\n",
"print(json.dumps(list(model.keys())[:20], indent=4))"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'dim': 4096,\n",
" 'n_layers': 32,\n",
" 'n_heads': 32,\n",
" 'n_kv_heads': 8,\n",
" 'vocab_size': 128256,\n",
" 'multiple_of': 1024,\n",
" 'ffn_dim_multiplier': 1.3,\n",
" 'norm_eps': 1e-05,\n",
" 'rope_theta': 500000.0}"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"with open(\"Meta-Llama-3-8B/params.json\", \"r\") as f:\n",
" config = json.load(f)\n",
"config"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## we use this config to infer details about the model like\n",
"1. the model has 32 transformer layers\n",
"2. each multi-head attention block has 32 heads\n",
"3. the vocab size and so on"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"dim = config[\"dim\"]\n",
"n_layers = config[\"n_layers\"]\n",
"n_heads = config[\"n_heads\"]\n",
"n_kv_heads = config[\"n_kv_heads\"]\n",
"vocab_size = config[\"vocab_size\"]\n",
"multiple_of = config[\"multiple_of\"]\n",
"ffn_dim_multiplier = config[\"ffn_dim_multiplier\"]\n",
"norm_eps = config[\"norm_eps\"]\n",
"rope_theta = torch.tensor(config[\"rope_theta\"])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## converting text to tokens\n",
"here we use tiktoken (i think an openai library) as the tokenizer\n",
"<div>\n",
" <img src=\"images/tokens.png\" width=\"600\"/>\n",
"</div>"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[128000, 1820, 4320, 311, 279, 17139, 3488, 315, 2324, 11, 279, 15861, 11, 323, 4395, 374, 220]\n",
"['<|begin_of_text|>', 'the', ' answer', ' to', ' the', ' ultimate', ' question', ' of', ' life', ',', ' the', ' universe', ',', ' and', ' everything', ' is', ' ']\n"
]
}
],
"source": [
"prompt = \"the answer to the ultimate question of life, the universe, and everything is \"\n",
"tokens = [128000] + tokenizer.encode(prompt)\n",
"print(tokens)\n",
"tokens = torch.tensor(tokens)\n",
"prompt_split_as_tokens = [tokenizer.decode([token.item()]) for token in tokens]\n",
"print(prompt_split_as_tokens)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## converting tokens to their embedding\n",
"IM SORRY but this is the only part of the codebase where i use an inbuilt neural network module\n",
"<br>\n",
"anyway, so our [17x1] tokens are now [17x4096], i.e. 17 embeddings (one for each token) of length 4096\n",
"<br>\n",
"<br>\n",
"note: keep track of the shapes, it makes it much easier to understand everything\n",
"\n",
"<div>\n",
" <img src=\"images/embeddings.png\" width=\"600\"/>\n",
"</div>"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([17, 4096])"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"embedding_layer = torch.nn.Embedding(vocab_size, dim)\n",
"embedding_layer.weight.data.copy_(model[\"tok_embeddings.weight\"])\n",
"token_embeddings_unnormalized = embedding_layer(tokens).to(torch.bfloat16)\n",
"token_embeddings_unnormalized.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## we then normalize the embedding using rms normalization\n",
"please, note after this step the shapes dont change, the values are just normalized\n",
"<br>\n",
"things to keep in mind, we need a norm_eps (from config) because we dont want to accidently set rms to 0 and divide by 0\n",
"<br>\n",
"here is the formula:\n",
"<div>\n",
" <img src=\"images/rms.png\" width=\"600\"/>\n",
"</div>"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"# def rms_norm(tensor, norm_weights):\n",
"# rms = (tensor.pow(2).mean(-1, keepdim=True) + norm_eps)**0.5\n",
"# return tensor * (norm_weights / rms)\n",
"def rms_norm(tensor, norm_weights):\n",
" return (tensor * torch.rsqrt(tensor.pow(2).mean(-1, keepdim=True) + norm_eps)) * norm_weights"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# building the first first layer of the transformer\n",
"\n",
"### normalization\n",
"you will see me accessing layer.0 from the model dict (this is the first layer)\n",
"<br>\n",
"anyway, so after normalizing our shapes are still [17x4096] same as embedding but normalized \n",
"\n",
"<div>\n",
" <img src=\"images/norm.png\" width=\"600\"/>\n",
"</div>"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([17, 4096])"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"token_embeddings = rms_norm(token_embeddings_unnormalized, model[\"layers.0.attention_norm.weight\"])\n",
"token_embeddings.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### attention implemented from scratch\n",
"let's load the attention heads of the first layer of the transformer\n",
"<div>\n",
" <img src=\"images/qkv.png\" width=\"600\"/>\n",
"</div>\n",
"\n",
"<br>\n",
"\n",
"> when we load the query, key, value and output vectors from the model we notice the shapes to be [4096x4096], [1024x4096], [1024x4096], [4096x4096]\n",
"<br>\n",
"> at first glance this is weird because ideally we want each q,k,v and o for each head individually\n",
"<br>\n",
"> the authors of the code bundled them togeather because its easy it helps parallize attention head multiplication.\n",
"<br>\n",
"> im going to unwrap everything... "
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch.Size([4096, 4096]) torch.Size([1024, 4096]) torch.Size([1024, 4096]) torch.Size([4096, 4096])\n"
]
}
],
"source": [
"print(\n",
" model[\"layers.0.attention.wq.weight\"].shape,\n",
" model[\"layers.0.attention.wk.weight\"].shape,\n",
" model[\"layers.0.attention.wv.weight\"].shape,\n",
" model[\"layers.0.attention.wo.weight\"].shape\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### unwrapping query\n",
"in the next section we will unwrap the queries from multiple attention heads, the resulting shape is [32x128x4096]\n",
"<br><br>\n",
"here, 32 is the number of attention heads in llama3, 128 is the size of the query vector and 4096 is the size of the token embedding"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([32, 128, 4096])"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"q_layer0 = model[\"layers.0.attention.wq.weight\"]\n",
"head_dim = q_layer0.shape[0] // n_heads\n",
"q_layer0 = q_layer0.view(n_heads, head_dim, dim)\n",
"q_layer0.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### im going to implement the first head of the first layer\n",
"here i access the query weight matrix first head of the first layer, the size of this query weight matrix is [128x4096]"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([128, 4096])"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"q_layer0_head0 = q_layer0[0]\n",
"q_layer0_head0.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### we now multiply the query weights with the token embedding, to recive a query for the token\n",
"here you can see the resulting shape is [17x128], this is because we have 17 tokens and for each token there is a 128 length query.\n",
"<div>\n",
" <img src=\"images/q_per_token.png\" width=\"600\"/>\n",
"</div>"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([17, 128])"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"q_per_token = torch.matmul(token_embeddings, q_layer0_head0.T)\n",
"q_per_token.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## positioning encoding\n",
"we are now at a stage where we have a query vector for each token in our prompt, but if you think about it -- the indivitually query vector has no idea about the position in the prompt.\n",
"<br><br>\n",
"query: \"the answer to the ultimate question of life, the universe, and everything is \"\n",
"<br><br>\n",
"in our prompt we have used \"the\" three times, we need the query vectors of all 3 \"the\" tokens to have different query vectors (each of size [1x128]) based on their positions in the query. we perform these rotations using RoPE (rotory positional embedding).\n",
"<br><br>\n",
"### RoPE\n",
"watch this video (this is what i watched) to understand the math.\n",
"https://www.youtube.com/watch?v=o29P0Kpobz0&t=530s\n",
"\n",
"\n",
"<div>\n",
" <img src=\"images/rope.png\" width=\"600\"/>\n",
"</div>"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([17, 64, 2])"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"q_per_token_split_into_pairs = q_per_token.float().view(q_per_token.shape[0], -1, 2)\n",
"q_per_token_split_into_pairs.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"in the above step, we split the query vectors into pairs, we apply a rotational angle shift to each pair!\n",
"<br><br>\n",
"we now have a vector of size [17x64x2], this is the 128 length queries split into 64 pairs for each token in the prompt! each of those 64 pairs will be rotated by m*(theta) where m is the position of the token for which we are rotating the query!\n",
"\n",
"\n",
"<div>\n",
" <img src=\"images/qsplit.png\" width=\"600\"/>\n",
"</div>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## using dot product of complex numbers to rotate a vector\n",
"<div>\n",
" <img src=\"images/freq_cis.png\" width=\"600\"/>\n",
"</div>"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([0.0000, 0.0156, 0.0312, 0.0469, 0.0625, 0.0781, 0.0938, 0.1094, 0.1250,\n",
" 0.1406, 0.1562, 0.1719, 0.1875, 0.2031, 0.2188, 0.2344, 0.2500, 0.2656,\n",
" 0.2812, 0.2969, 0.3125, 0.3281, 0.3438, 0.3594, 0.3750, 0.3906, 0.4062,\n",
" 0.4219, 0.4375, 0.4531, 0.4688, 0.4844, 0.5000, 0.5156, 0.5312, 0.5469,\n",
" 0.5625, 0.5781, 0.5938, 0.6094, 0.6250, 0.6406, 0.6562, 0.6719, 0.6875,\n",
" 0.7031, 0.7188, 0.7344, 0.7500, 0.7656, 0.7812, 0.7969, 0.8125, 0.8281,\n",
" 0.8438, 0.8594, 0.8750, 0.8906, 0.9062, 0.9219, 0.9375, 0.9531, 0.9688,\n",
" 0.9844])"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"zero_to_one_split_into_64_parts = torch.tensor(range(64))/64\n",
"zero_to_one_split_into_64_parts"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([1.0000e+00, 8.1462e-01, 6.6360e-01, 5.4058e-01, 4.4037e-01, 3.5873e-01,\n",
" 2.9223e-01, 2.3805e-01, 1.9392e-01, 1.5797e-01, 1.2869e-01, 1.0483e-01,\n",
" 8.5397e-02, 6.9566e-02, 5.6670e-02, 4.6164e-02, 3.7606e-02, 3.0635e-02,\n",
" 2.4955e-02, 2.0329e-02, 1.6560e-02, 1.3490e-02, 1.0990e-02, 8.9523e-03,\n",
" 7.2927e-03, 5.9407e-03, 4.8394e-03, 3.9423e-03, 3.2114e-03, 2.6161e-03,\n",
" 2.1311e-03, 1.7360e-03, 1.4142e-03, 1.1520e-03, 9.3847e-04, 7.6450e-04,\n",
" 6.2277e-04, 5.0732e-04, 4.1327e-04, 3.3666e-04, 2.7425e-04, 2.2341e-04,\n",
" 1.8199e-04, 1.4825e-04, 1.2077e-04, 9.8381e-05, 8.0143e-05, 6.5286e-05,\n",
" 5.3183e-05, 4.3324e-05, 3.5292e-05, 2.8750e-05, 2.3420e-05, 1.9078e-05,\n",
" 1.5542e-05, 1.2660e-05, 1.0313e-05, 8.4015e-06, 6.8440e-06, 5.5752e-06,\n",
" 4.5417e-06, 3.6997e-06, 3.0139e-06, 2.4551e-06])"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"freqs = 1.0 / (rope_theta ** zero_to_one_split_into_64_parts)\n",
"freqs"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAHHCAYAAABDUnkqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy80BEi2AAAACXBIWXMAAA9hAAAPYQGoP6dpAADB1ElEQVR4nOydd3hTZRvG7w5oyyp7yt5LNsiWJTKCoLKRISAiQ6YCMkQERAUEZCibArKHDAHZW5lC2XtTdkvpbt7vj/sLHbRpkiY5Sfr8ritXIeOcJyfJee/zTDellIIgCIIgCIKL4K61AYIgCIIgCNZExI0gCIIgCC6FiBtBEARBEFwKETeCIAiCILgUIm4EQRAEQXApRNwIgiAIguBSiLgRBEEQBMGlEHEjCIIgCIJLIeJGEARBEASXQsSNICSTvXv3ws3NDXv37tXalDj4+fmhRIkSSJUqFTJmzKi1OSmCn376CYUKFYKHhwfKly9v9Lkp5fNx1N+H4NqIuBGERFi0aBHc3Nxe37y9vVGsWDH07dsXAQEBVtnH1q1b8e2331plW7G5ePEiunbtisKFC2Pu3Ln4/fffrb4PIS47duzAV199hZo1a2LhwoWYMGFCos+Vz0cQbIun1gYIgqPz3XffoWDBgggLC8PBgwcxe/ZsbN26Ff7+/kiTJk2ytr1161bMnDnT6gJn79690Ov1mDZtGooUKWLVbQsJs3v3bri7u2P+/PlInTq10eempM+nTp06CA0NTfKYCII1EXEjCEnQpEkTVK5cGQDQo0cPZMmSBVOmTMHGjRvRvn17ja1LmEePHgGAy4Q7Xr16hbRp02pthlEePXoEHx8fkxZxUz8fpRTCwsLg4+NjDRM1wd3dHd7e3lqbIaQwJCwlCGZSv359AMCNGzeMPm/16tWoVKkSfHx8kDVrVnTq1An37t17/XjXrl0xc+ZMAIgT/kqKWbNmoXTp0vDy8kLu3LnRp08fvHjx4vXjBQoUwJgxYwAA2bJlg5ubW5Keod27d6N27dpImzYtMmbMiA8++AAXLlyI85xvv/0Wbm5uuHr1Krp27YqMGTPC19cX3bp1Q0hIyBvbXLp06ev3nzlzZrRr1w537txJ8v0Z9nP+/Hl06NABmTJlQq1atQAAUVFRGDduHAoXLgwvLy8UKFAAI0aMQHh4+OvXDxo0CFmyZIFS6vV9/fr1g5ubG6ZPn/76voCAALi5uWH27NlG7TFln25ubli4cCFevXr1+nNctGhRgtsz9vkUKFAAzZs3x/bt21G5cmX4+Pjgt99+AwC8ePECAwYMQN68eeHl5YUiRYpg0qRJ0Ov1cbb/4sULdO3aFb6+vsiYMSO6dOmC06dPv2HTw4cP0a1bN7z11lvw8vJCrly58MEHH+DmzZtGj0d87t27h+7duyN37tzw8vJCwYIF0bt3b0RERABIOOfmypUr+Oijj5AzZ054e3vjrbfeQrt27RAYGGjWvgUhMcRzIwhmcu3aNQBAlixZEn3OokWL0K1bN1SpUgUTJ05EQEAApk2bhkOHDuHUqVPImDEjevXqhfv37+Pvv/+Gn5+fSfv+9ttvMXbsWDRs2BC9e/fGpUuXMHv2bBw7dgyHDh1CqlSp8Msvv2DJkiVYv349Zs+ejXTp0uHtt99OdJs7d+5EkyZNUKhQIXz77bcIDQ3FjBkzULNmTZw8eRIFChSI8/w2bdqgYMGCmDhxIk6ePIl58+Yhe/bsmDRp0uvnjB8/HqNGjUKbNm3Qo0cPPH78GDNmzECdOnVev/+kaN26NYoWLYoJEya8Fio9evTA4sWL8fHHH2Pw4MH4559/MHHiRFy4cAHr168HANSuXRtTp07FuXPnUKZMGQDAgQMH4O7ujgMHDqB///6v7wMYNjGGKfv08/PD77//jn///Rfz5s0DANSoUSPB7SX1+Vy6dAnt27dHr1690LNnTxQvXhwhISGoW7cu7t27h169eiFfvnw4fPgwhg8fjgcPHuCXX34BQE/PBx98gIMHD+Lzzz9HyZIlsX79enTp0uUNOz766COcO3cO/fr1Q4ECBfDo0SP8/fffuH379hufeWLcv38fVatWxYsXL/DZZ5+hRIkSuHfvHtasWYOQkJAEvVgRERFo3LgxwsPD0a9fP+TMmRP37t3D5s2b8eLFC/j6+pq0b0EwihIEIUEWLlyoAKidO3eqx48fqzt37qgVK1aoLFmyKB8fH3X37l2llFJ79uxRANSePXuUUkpFRESo7NmzqzJlyqjQ0NDX29u8ebMCoEaPHv36vj59+ihTf4aPHj1SqVOnVu+9956Kjo5+ff+vv/6qAKgFCxa8vm/MmDEKgHr8+HGS2y1fvrzKnj27evr06ev7/vvvP+Xu7q46d+78xjY//fTTOK9v1aqVypIly+v/37x5U3l4eKjx48fHed7Zs2eVp6fnG/fHx7Cf9u3bx7n/9OnTCoDq0aNHnPuHDBmiAKjdu3crpXicAKhZs2YppZR68eKFcnd3V61bt1Y5cuR4/br+/furzJkzK71en6gtpu5TKaW6dOmi0qZNa/S9xX+P8T+f/PnzKwBq27Ztce4fN26cSps2rbp8+XKc+4cNG6Y8PDzU7du3lVJKbdiwQQFQP/744+vnREVFqdq1aysAauHChUoppZ4/f64AqJ9++skkexOjc+fOyt3dXR07duyNxwzHNf7v49SpUwqAWr16dbL2LQjGkLCUICRBw4YNkS1bNuTNmxft2rVDunTpsH79euTJkyfB5x8/fhyPHj3CF198ESfXoFmzZihRogS2bNlikR07d+5EREQEBgwYAHf3mJ9uz549kSFDBou2++DBA5w+fRpdu3ZF5syZX9//9ttvo1GjRti6desbr/n888/j/L927dp4+vQpgoKCAADr1q2DXq9HmzZt8OTJk9e3nDlzomjRotizZ49JtsXfj8GWQYMGxbl/8ODBAPD6/WfLlg0lSpTA/v37AQCHDh2Ch4cHhg4dioCAAFy5cgUAPTe1atUyGgo0dZ/WpGDBgmjcuHGc+1avXo3atWsjU6ZMcY5pw4YNER0d/fq9bt26FZ6enujdu/fr13p4eKBfv35xtmfIDdq7dy+eP39ukZ16vR4bNmyATqd7nZMWm8SOq8Ezs3379gTDmYJgDSQsJQhJMHPmTBQrVgyenp7IkSMHihcvHkdcxOfWrVsAgOLFi7/xWIkSJXDw4EGL7Ehsu6lTp0ahQoVeP26NbQJAyZIlsX379jeSefPlyxfneZkyZQIAPH/+HBkyZMCVK1eglELRokUT3GeqVKlMsq1gwYJv2Oru7v5GdVHOnDmRMWPGOO+/du3ar4XJgQMHULlyZVSuXBmZM2fGgQMHkCNHDvz333/o0KGDURvM2ae1iP++AeaonDlzBtmyZUvwNYYE5Vu3biFXrlxIly5dnMfjf75eXl6YNGkSBg8ejBw5cuCdd95B8+bN0blzZ+TMmdMkOx8/foygoKDXoT9TKViwIAYNGoQpU6Zg2bJlqF27Nlq0aIFOnTpJSEqwGiJuBCEJqlatmuCVaUrFw8MjwfvV//Ni9Ho93Nzc8NdffyX43PgLb2IkViFkStJ1rVq1MHfuXFy/fh0HDhxA7dq14ebmhlq1auHAgQPInTs39Ho9ateubZItpuzTWiT0vvV6PRo1aoSvvvoqwdcUK1bM7P0MGDAAOp0OGzZswPbt2zFq1ChMnDgRu3fvRoUKFczenjlMnjwZXbt2xcaNG7Fjxw70798fEydOxNGjR/HWW2/ZdN9CykDEjSBYmfz58wNgYqihssrApUuXXj8OmLdoxt5uoUKFXt8fERGBGzduoGHDhsmyNT4XL15E1qxZzS7BLly4MJRSKFiwoEWLbmLkz58fer0eV65cQcmSJV/fHxAQgBcvXsQ5rgbR8vfff+PYsWMYNmwYACYPz549G7lz50batGlRqVIlq+3TlhQuXBjBwcFJfsb58+fHrl27EBwcHEdEJvT5GrY7ePBgDB48GFeuXEH58uUxefJkLF26NEmbsmXLhgwZMsDf39+8N/N/ypYti7Jly2LkyJE4fPgwatasiTlz5uD777+3aHuCEBvJuREEK1O5cmVkz54dc+bMiVMu/Ndff+HChQto1qzZ6/sMwiF2KXdiNGzYEKlTp8b06dPjlDnPnz8fgYGBcbZrKrly5UL58uWxePHiODb4+/tjx44daNq0qdnb/PDDD+Hh4YGxY8fGsROgd+fp06dmbxPAa1sMlUEGpkyZAgBx3n/BggWRJ08eTJ06FZGRkahZsyYAip5r165hzZo1eOedd+Dpafz6zpx92pI2bdrgyJEj2L59+xuPvXjxAlFRUQBob1RUVJzy9ujoaMyYMSPOa0JCQhAWFhbnvsKFCyN9+vRxvrPGcHd3R8uWLbFp0yYcP378jcfjf/YGgoKCXttroGzZsnB3dzd534KQFOK5EQQrkypVKkyaNAndunVD3bp10b59+9el4AUKFMDAgQNfP9fgOejfvz8aN24MDw8PtGvXLsHtZsuWDcOHD8fYsWPx/vvvo0WLFrh06RJmzZqFKlWqoFOnThbZ+9NPP6FJkyaoXr06unfv/roU3NfX16LOyYULF8b333+P4cOH4+bNm2jZsiXSp0+PGzduYP369fjss88wZMgQs7dbrlw5dOnSBb///jtevHiBunXr4t9//8XixYvRsmVL1KtXL87za9eujRUrVqBs2bKv84IqVqyItGnT4vLly0nm21iyT1sxdOhQ/Pnnn2jevDm6du2KSpUq4dWrVzh79izWrFmDmzdvImvWrNDpdKhZsyaGDRuGmzdvolSpUli3bt0b/WMuX76MBg0aoE2bNihVqhQ8PT2xfv16BAQEJPr9S4gJEyZgx44dqFu3Lj777DOULFkSDx48wOrVq3Hw4MEES/53796Nvn37onXr1ihWrBiioqLg5+cHDw8PfPTRR8k9VIJANKzUEgSHxlAKnlCZa2zil7oaWLlypapQoYLy8vJSmTNnVh07dnxdPm4gKipK9evXT2XLlk25ubmZVBb+66+/qhIlSqhUqVKpHDlyqN69e6vnz5/HeY45peBKKbVz505Vs2ZN5ePjozJkyKB0Op06f/68Sds0HKcbN27EuX/t2rWqVq1aKm3atCpt2rSqRIkSqk+fPurSpUtGbTFme2RkpBo7dqwqWLCgSpUqlcqbN68aPny4CgsLe+O5M2fOVABU796949zfsGFDBUDt2rXLqB3m7tNapeDNmjVL8DUvX75Uw4cPV0WKFFGpU6dWWbNmVTVq1FA///yzioiIeP28p0+fqk8++URlyJBB+fr6qk8++eR1+bWhFPzJkyeqT58+qkSJEipt2rTK19dXVatWTa1atcok+2Nz69Yt1blzZ5UtWzbl5eWlChUqpPr06aPCw8OVUm/+Pq5fv64+/fRTVbhwYeXt7a0yZ86s6tWrp3bu3Gn2vgUhMdyUSsR3KAiCILgEN2/eRMGCBbFw4UJ07dpVa3MEweZIzo0gCIIgCC6F5NwIgiAIcQgODkZwcLDR52TLli3RtgCCoDUibgRBEIQ4/Pzzzxg7dqzR59y4ccPkGVSCYG8k50YQBEGIw/Xr13H9+nWjz6lVq1ac8SKC4EiIuBEEQRAEwaWQhGJBEARBEFyKFJdzo9frcf/+faRPn96u82IEQRAEQbAcpRRevnyJ3LlzGx1eDKRAcXP//n3kzZtXazMEQRAEQbCAO3fuJDlgNcWJm/Tp0wPgwcmQIYPG1giCIAiCYApBQUHImzfv63XcGClO3BhCURkyZBBxIwiCIAhOhikpJZJQLAiCIAiCSyHiRhAEQRAEl0LEjSAIgiAILoWIG0EQBEEQXAoRN4IgOD6zZwNvvw1kyMBb9erAX39pbZUgCA6KiBtBEByft94CfvgBOHECOH4cqF8f+OAD4Nw5rS0TBMEBSXGl4IIgOCE6Xdz/jx9Pb87Ro0Dp0trYJAiCwyLiRhAE5yI6Gli9Gnj1iuEpQRCEeIi4EQTBOTh7lmImLAxIlw5Yvx4oVUprqwRBcEA0zbnZv38/dDodcufODTc3N2zYsCHJ1+zduxcVK1aEl5cXihQpgkWLFtncTkEQHIDixYHTp4F//gF69wa6dAHOn9faKkEQHBBNxc2rV69Qrlw5zJw506Tn37hxA82aNUO9evVw+vRpDBgwAD169MD27dttbKkgCJqTOjVQpAhQqRIwcSJQrhwwbZrWVgmC4IBoGpZq0qQJmjRpYvLz58yZg4IFC2Ly5MkAgJIlS+LgwYOYOnUqGjdubCszBUFwRPR6IDxcaysEQXBAnCrn5siRI2jYsGGc+xo3bowBAwYk+prw8HCExzoBBgUF2co8QRBsxfDhQJMmQL58wMuXwPLlwN69gHhtBUFIAKfqc/Pw4UPkyJEjzn05cuRAUFAQQkNDE3zNxIkT4evr+/qWN29ee5gqCIIVWL8eqF0bwKNHQOfOzLtp0AA4dozCplEjrU0UBMEBcSpxYwnDhw9HYGDg69udO3e0NkkQBBN5+hQ4eBBQ8+YDN28yDPXoEbBzpwib2PzwA+DmBhjxYgtCSsKpwlI5c+ZEQEBAnPsCAgKQIUMG+Pj4JPgaLy8veHl52cM8QRCsjOFnHRYW828hHseOAb/9xvEUgiAAcDLPTfXq1bFr16449/3999+oLo28BMElMQiaRKLOQnAw0LEjMHcukCmT1tYIgsOgqbgJDg7G6dOncfr0aQAs9T59+jRu374NgCGlzp07v37+559/juvXr+Orr77CxYsXMWvWLKxatQoDBw7UwnxBEGyMiJsk6NMHaNYMiFdoIQgpHU3DUsePH0e9evVe/3/QoEEAgC5dumDRokV48ODBa6EDAAULFsSWLVswcOBATJs2DW+99RbmzZsnZeCC4KJ4e/OviJsEWLECOHmSYSlBEOKgqbh59913oZRK9PGEug+/++67OHXqlA2tEgTBUYidcyPE4s4d4Msvgb//jlGAgiC8xqkSigVBSFlIWCoRTpxg1VjFijH3RUcD+/cDv/7KqjIPD+3sEwSNEXEjCILDIuImERo04CDR2HTrBpQoAXz9tQgbIcUj4kYQBIdFxE0ipE8PlCkT9760aYEsWd68XxBSIE5VCi4IQspCEorjMnIkU20EQTCOeG4EQXBYxHMTl927gcKFE3lw7157miIIDo14bgRBcFikWioGpQB/f4k6CYIpiLgRBMFh8fAAUqUSzw3A6u+XL0XcCIIpiLgRBMGh8fERcQPQawOIuBEEUxBxIwiCQyPihvj7s0gqXz6tLREEx0fEjSAIDo23t4gbICbfxs1Na0sEwfERcSMIgkPj4yMJxYAkEwuCOYi4EQTBoZGwFCcrnD8v4kYQTEXEjSAIDo2IG+DaNY6LEnEjCKYh4kYQbM3EiUCVKswGzZ4daNkSuHRJa6ucBhE3MWOkRNwIgmmIuBEEW7NvH9CnD3D0KPD330BkJPDee8CrV1pb5hRIQjHzbbJlozYWBCFpZPyCINiabdvi/n/RIq5SJ04AdepoYpIz4eMDBAZqbYW2SDKxIJiHeG4Ewd4YVurMmbW1w0mQaikRN4JgLiJuBMGe6PXAgAFAzZqyWplISs+5CQsDrlwBypbV2hJBcB4kLCUI9qRPH16GHzyotSVOQ0oXN5cusRRctLAgmI6IG0GwF337Aps3A/v3A2+9pbU1TkNKFzeGmVKlS2trhyA4EyJuBMHWKAX06wesXw/s3QsULKi1RU5FSq+W8vfnPKkMGbS2RBCcBxE3gmBDQkMBn8F9gOXLgY0b2evm4UM+6OtLt4RgFPHcSEhKEMxFEooFwUaMHg1UrQpg9mxWSL37LpArV8xt5UqtTXQKUnq11NmzIm4EwVxE3AiCjShXjlfd168phqbi37p21dpEp8AgbpTS2hL7ExQE3Lol4kYQzEXEjSDYiPfeA1KnBjZt0toS58YQuUuJ3pvz5/nX5cXNt98Cbm5xbyVKaG2V4MSIuBEEG5E+PSNRIm6Sh7c3/6bEvBt/f8DdPYWs86VLAw8exNykXYKQDCShWBBsSIsW7NkXGMj8YcF8DJ6blCpuihRJIXnnnp5AzpxaWyG4COK5EQQb0rw5EBUFbN+utSXOS0oOS/n7p6DOxFeuALlzA4UKAR07Ardva22R4MSIuBEEG5I/P/D228Cff2ptifOS0j03Lp9vAwDVqnGg7LZtrC68cQOoXRt4+VJrywQnRcJSgmBjdDpg1ix6cDzlF2c2KVXcPH4MBASkEHHTpEnMv99+m2Inf35g1Sqge3ft7BKcFvHcCIKN0emA58+Bw4e1tsQ5Sani5tw5/k0R4iY+GTMCxYoBV69qbYngpIi4EQQbU6UKkCOHVE1ZSkqtlvL3ZyuBIkW0tkQDgoOBa9fY7FIQLEDEjSDYGHd3JhaLuLGMlOq5OXsWKFnStUOZ0dH//8eQIcC+fcDNm3RxtmoFeHgA7dtraZ7gxIi4EQQ7oNMBly4Bly9rbYnzkVKrpVw9mVgpVoLNmgXg7l0KmeLFgTZtgCxZgKNHgWzZtDZTcFJc+JpAEByHhg0BLy96bwYP1toa5yIlem6UorjR6bS2xHYcPgxcuACUKgXgixVamyO4GOK5EQQ7kDYtBY6EpszHwwNIlSpliZu7dzlXypU9N35+QL58QJ06WlsiuCIibgTBTuh07Cj/7JnWljgf3t4pS9z4+/OvqzbwCw8HVq5krz53WYUEGyBfK0GwE82bM4Hyr7+0tsT58PFJeeImXTp6NlyRzZuBFy+ATz7R2hLBVRFxIwh2Ik8eoGJFCU1Zgo9PykooNiQTu7lpbYlt8PMDKldmNZgg2AIRN4JgR1q0YIf5yEitLXEuUqLnxlXzbZ48AbZuFa+NYFtE3AiCHdHpOCH8wAGtLXEuUpK4iY4Gzp93XXGzciWg1wPt2mltieDKiLgRBDtSoQLDUzJI0zxSkri5do0hOFcVN35+HCWVPbvWlgiujIgbQbAjbm4x3YqV0toa5yElVUsZKqVcUdxcvgz884+EpATbI+JGEOxMixbA9etsYCaYRkry3Pj7A1mzuqZnY+lSIEMG125OKDgGIm4Ewc7Urw+kSSNVU+aQkqqlXLVSSq9nSKp165iu04JgK0TcCIKd8fYGGjWSvBtzSGmeG1ds3nfoEOdidu6stSVCSkDEjSBogE4HHDkCPH6stSXOQUoRN+HhzEtxxXwbPz8gf36gVi2tLRFSAiJuBEEDmjfn361btbXDWUgpCcWXLrEU3NXETVgYsGoV0KmTjFsQ7IN8zQRBA3LkAKpWlbwbU0kpnhtDpVTp0traYW02bWJ/J6mSEuyFiBtB0AidDti+naEIwTgpJaHY3x/Imxfw9dXaEuvi50cxX7y41pYIKQURN4KgETodEBwM7N2rtSWOT0rx3Jw963ohqcePOSxWvDaCPRFxIwgaUbYsEywlNJU0KUXcuOJMqRUr+FfGLQj2RMSNIGiEmxu9N9KtOGkMYSlXPk4vX7JU2tXEjZ8f0LQpGxMKgr0QcSMIGqLTAbdvA2fOaG2JY+Ptzb+unHdz/jz/upK4uXgROHZMQlKC/RFxIwgaUrcukC6dhKaSwtDR1pVDU/7+LJMuWVJrS6yHnx+Tow2tDwTBXoi4EQQN8fIC3n9fxE1SGMSNK3tu/P2BIkVcZzSBXs9ZUm3bxnjeBMFeaC5uZs6ciQIFCsDb2xvVqlXDv//+a/T5v/zyC4oXLw4fHx/kzZsXAwcORJgrn/EEl0enA/79F3j4UGtLHJeU4rlxpZDUgQMMuUpIStACTcXNypUrMWjQIIwZMwYnT55EuXLl0LhxYzx69CjB5y9fvhzDhg3DmDFjcOHCBcyfPx8rV67EiBEj7Gy5IFiPpk0Zjti8WWtLHBcRN86Hnx9QsCBQs6bWlggpEU3FzZQpU9CzZ09069YNpUqVwpw5c5AmTRosWLAgwecfPnwYNWvWRIcOHVCgQAG89957aN++fZLeHkFwZLJmBapXl9CUMQxhDVcVN0+e0HPnKuImNBRYvZrjFlxturngHGgmbiIiInDixAk0bNgwxhh3dzRs2BBHjhxJ8DU1atTAiRMnXouZ69evY+vWrWjatKldbBYEW9GiBfD33667eCcXV/fcGMYuuIq4+fNPIChIQlKCdmgmbp48eYLo6GjkyJEjzv05cuTAw0SSDzp06IDvvvsOtWrVQqpUqVC4cGG8++67RsNS4eHhCAoKinMTBEdDp+PCvXu31pY4JilB3KROzYRiV8DPD3jnHaBoUa0tEVIqmicUm8PevXsxYcIEzJo1CydPnsS6deuwZcsWjBs3LtHXTJw4Eb6+vq9vefPmtaPFgmAaJUoAhQvzild4E1evlvL353cgVSqtLUk+jx4B27aJ10bQFs3ETdasWeHh4YGAgIA49wcEBCBnzpwJvmbUqFH45JNP0KNHD5QtWxatWrXChAkTMHHiROj1+gRfM3z4cAQGBr6+3blzx+rvRRCSi6Fb8ebNrt2F11JSgufGVUJSf/zBBPm2bbW2xArcu8fEoSxZ+CUsWxY4flxrqwQT0EzcpE6dGpUqVcKuXbte36fX67Fr1y5Ur149wdeEhITA3T2uyR4eHgAAlciK4OXlhQwZMsS5CYIj0qIFcP8+cPKk1pY4Hq6cUKwUxU3ZslpbYh38/IBmzagHnJrnz1nqlSoVJ3+ePw9MngxkyqS1ZYIJeGq580GDBqFLly6oXLkyqlatil9++QWvXr1Ct27dAACdO3dGnjx5MHHiRACATqfDlClTUKFCBVSrVg1Xr17FqFGjoNPpXoscQXBWatViN9dNm4BKlbS2xrHw9OTNFcXNvXtAYKBreG7OnwdOnABcojvHpElA3rzAwoUx9xUsqJ09glloKm7atm2Lx48fY/To0Xj48CHKly+Pbdu2vU4yvn37dhxPzciRI+Hm5oaRI0fi3r17yJYtG3Q6HcaPH6/VWxAEq5EqFdCkCcXNt99qbY3j4aqTwV2pUsrPj46NZs20tsQK/Pkn0Lgx0Lo1sG8fkCcP8MUXQM+eWlsmmICbSiye46IEBQXB19cXgYGBEqISHI7ly4GOHYE7d4C33tLaGsciRw6gf3/gm2+0tsS6/PwzMHYsvTfuTlXiERe9HihQgMJm9mytrbEChljooEEUOMeOAV9+CcyZA3Tpoq1tKRRz1m8n/ikJguvRpAng4SHdihPClT03pUs7t7AB6Ny4c8eFqqT0eqBiRWDCBKBCBeCzz+i1mTNHa8sEE3Dyn5MguBaZMgG1a0u34oRwVXFz9qzrhKQKF2a3bZcgVy6gVKm495UsyYFZgsMj4kYQHAydDti1C3j1SmtLHAtvb9cTN9HRTMJ1dnETEsJxC5984kLjFmrWBC5dinvf5ctA/vza2COYhYgbQXAwdDogPJzjGIQYXNFzc/06GxM6u7jZuBEIDmZLGGfn/n1gxQpADRgIHD3KsNTVq0yI+/13oE8frU0UTEDEjSA4GEWLsluthKbi4orixlUqpfz8gBo1GJZydkaOBPr1A16WqAKsX8+uhGXKAOPGAb/8wox/weHRtBRcEISE0emAJUuY0+jsiabWwsfH9cYv+PtzKny8EXtOxcOHwPbtwMyZWluSfE6dAhYtAn79FciQAUDz5rwJToecNgXBAdHpgIAAVp8KxFU9N2XKOHeeyh9/sMFimzZaW5I8lAIGD6bX9LPPtLZGSC4ibgTBAaleHcicWQZpxsaVxY0z4+dH50bmzFpbkjw2bQL27GHfIU+JaTg9Im4EwQHx9ASaNpW8m9i4WrVUeDiLb5xZ3Pj7M5Tj7L1tIiKAIUOARo3Ya0pwfkTcCIKD0qIFe6DcuqW1JY6Bq3luLl8GoqKcW9z4+dFj07Sp1pYkjzlzgGvXOBfTmUOEQgwibgTBQWncmPOmxHtDXC2h+OxZ/i1dWls7LCU6Gli2DGjXDkidWmtrLOfZM85y697ddSazCyJuBMFhyZABqFtX8m4MuJrnxt+f88MyZtTaEsvYu5cTzZ09JPX990BkJPDdd1pbIlgTETeC4MDodFxEgoK0tkR7XFHcOHNIaskS9mSqVk1rSyznyhWWfQ8fDuTMqbU1gjURcSMIDoxOx6vKHTu0tkR7XC2h2JnFzatXwNq1zj9u4euvKWoGDtTaEsHaiLgRBAemYEEugJJ3E5Nzo5TWliSf4GDgxg3nFTcbNlDgOPO4hX372IB44kR+twTXQsSNIDg4Oh2wZQsTOFMyhgXIFZKKz5/nX2dNYPXzA2rVovh2RvR6YNAgoEoVoH17ra0RbIGIG0FwcHQ64OlT4MgRrS3RFlcSN/7+DOeULKm1Jebz4AGHunburLUllrN0KXDyJDBliow3cVXkYxUEB6dqVSB7dglNGcSNK+Td+PsDRYo4Zzhk+XK2KGjdWmtLLOPVK2DECODjj+l9ElwTETeC4OB4eADNmom48fbmX1cRN86ab+PnR2+is5awT54MPH4M/PCD1pYItkTEjSA4ATodcOECcPWq1pZoh6t5bpxR3Jw5A/z3n/P2trl/H5g0CejfHyhcWGtrBFsi4kYQnIBGjdgFNiV7b1xF3Dx9yrwVZxQ3fn5A1qzA++9rbYlljBzJ79E332htiWBrRNwIghOQLh3QoIGIG8D5E4r9/fnX2cRNdDTzbZx13MKpU8CiRcDYsc4bUhNMR8SNIDgJOh2wfz/w/LnWlmiDq3hu/P2ZkFu0qNaWmMfu3QzrOGNISilg8GCgeHHgs8+0tkawByJuBMFJaN6cV8/btmltiTa4krgpUYICx5lYsoTioEoVrS0xn82bgT17gJ9/dr7jLliGiBtBcBLy5gXKl0+5oSlXqZby93e+5n3BwcC6dc45biEyEhgyBGjYEGjaVGtrBHsh4kYQnIgWLYC//uIJO6XhCp4bpZyzUmr9eiAkBOjYUWtLzGfOHA7InDzZ+YSZYDkibgTBidDpgBcvgEOHtLbE/nh68ubM4ub+fX5+ziZuliwB6tQBChTQ2hLzeP4c+PZboHt34O23tbZGsCcibgTBiahYEciVK+WGpgzDM50VZ6yUuncP2LXLOcctfP89EB4OjBuntSWCvRFxIwhOhLs7E4v//NM1pmObi4+Pc3tu/P2BtGmB/Pm1tsR0li8HvLw4rsCZuHoVmDEDGD4cyJlTa2sEeyPiRhCcjBYteOK+dElrS+yPt7dzi5uzZ4HSpZ1nWKNSDEm1aAH4+mptjXl8/TVFzaBBWlsiaIGT/MQEQTDQoAE9GCkxNOUKnhtnCkn99x9tdraQ1P79rO6aONE5h5MKyUfEjSA4GT4+LGsVceNcREcD5887l7jx8wOyZQPee09rS0xHr6e3pkoVoH17ra0RtELEjSA4ITodK6aePtXaEvvizOLmxg3a7iziJiqK+Tbt2ztX47tly4ATJ4ApU5wn/CdYH/noBcEJad6cV6hbt2ptiX1x5mopQ6WUszTw27kTePjQucYthIQwgfjjj4FatbS2xgwKFGATnvi3Pn20tsxpEXEjCE5Irlx0u6e00JQze278/YEsWYAcObS2xDT8/ICSJYFKlbS2xHQmTwYePwZ++EFrS8zk2DGOijfc/v6b97dura1dToyIG0FwUnQ6zpmKiNDaEvvhzNVShmRiZ+iS+/IluxI707iF+/cpavr3BwoX1toaM8mWjaVdhtvmzXwTdetqbZnTIuJGEJwUnY6L0L59WltiP5zdc+Ms+Tbr1vE4O9O4hVGj+P345hutLUkmERHA0qXAp586j7J0QETcCMbZv5+raO7c/KFt2KC1RcL/KVeOwzRTUmjKWcVNRAT7EjmLuFmyBKhXD8iXT2tLTOP0aWDhQo5ayJhRY2OSy4YNnNHRtavGhjg3Im4E47x6xVV05kytLRHi4eZG3blpU8rpVuysCcWXLrH6yBnEzd27wJ49zpNIrBQweDBQrBjQq5fW1liB+fOBJk14QSlYjKfWBggOTpMmvAkOiU4HzJoFnDvnHAtncnFWz42hUqp0aW3tMIVlyzhu4aOPtLbENDZvBnbvpsh3ppL1BLl1i2Vq69ZpbYnTI54bQXBi3n2Xs4r+/FNrS+yDsyYU+/sDefIAmTJpbYlxDOMWWrYEMmTQ2pqkiYwEhgxh1+5mzbS2xgosXAhkz+4ib0ZbRNwIghPj7Q00bpxy8m6c2XPjDJ61U6fYRdlZxi3MmQNcucIScGfMvT1zJlZIWa+nuOnSBfCUoEpyEXEjCE6OTgf88w/w6JHWltgeZxY3ztC8z8+PfXgaNdLakqR5/pwJxJ9+yrRAZ+PwYfYQWrbs/3fs3Ancvs03JCQbETeC4OQ0bcq/W7Zoa4c9MCQUO1MC9atXwPXrju+5iT1uwRkcB99/D4SHA+PGaW2J+QQEsD/fO+8Abdv+/8733uMXu1gxTW1zFUTcCIKTkz07T5IpIe/GMOE5PFxbO8zh/Hn+dXRxs2MHvX/OEJK6ehWYMQMYNozdup2JqCgKGr0eWLXKBZKgHRQRN0KCBAfH+sfp07wBnP53+jTdp4LD0KIFFydnLJM2B4O4cabQlL8/80FKltTaEuP4+bGaq3x5rS1JmmHDGD4bNEhrS8xn+HDg4EEKG2cTZs6EiBvhDR48AIoXp4sax48DFSrwBvBsUqECMHq0pjYKcdHpODRwzx6tLbEt3t7862zipnBhIE0arS1JnKAg9o5zhnELBw4Aa9cCEyc69jFNiLVrgZ9/Bn76CahdW2trXBsRN8Ib5MwJNGwIdO8OnMzwLuPA8W+LFmltphCLUqWAggVdv2rKGT03Z886fkhqzRqG+hx93IJez+urypWBDh20tsY8Ll5k0+E2bYABA7S2xvURcSO8gZsbSyzLlGG/i5RQhePspJRuxc4obpyhDNzPD6hfH3jrLa0tMc7y5XQmT5kCuDvR6hUcDHz4IcelzJ/v+N4xV8CJvh6CPfHxYZPM8HBeaURGam2RkBQ6HVvnG9KjXBGDuHGW3KKnTxnmdWRxc/s2sHev449bCAlhvspHHzlXSEcpesHv3OE5NV06rS1KGYi4ERIlb17GiA8dcs7EvZRGnTrsKuvKoSln89ycO8e/jixuli3jcf3wQ60tMc6UKSyhnjRJa0vM45dfmDy8cCFQooTW1qQcRNwIRqlViyWXv/4KLFigtTWCMVKnBt5/37XFjbMlFPv7s9TXUVuXGMYtfPghkD691tYkzoMHwA8/AP37MznbWThwABg6lCMiPv5Ya2tSFiJuhCTp1Qvo2RPo3ZudcAXHRadjTsL9+1pbYhuczXPj78+rdUftZXLiBBNdHT0kNWoUhe3IkVpbYjoPHjCkX6sWK7sE+yLiRkgSNzd6bypV4hXegwdaWyQkRpMmTLTcvFlrS2yDM4obRw5J+fmxOrJBA60tSZz//qPX+NtvgYwZtbbGNCIjKWzc3YGVK52j47OrIeJGMAkvL+bfAEzoc6YOsSmJLFl4peiqoSlnEjdKOba4iYwE/viDJdWOuvgqxXy/YsXoQXYWvvoKOHoUWL2azQYF+yPiRjCZXLmY7X/iBNC3r2uXHDszOh1n8IWEaG2J9fH05M0ZqqUePOBwR0cVN9u3A48fO/a4hS1bgN272fjOUUN78Vm5kknEU6YANWpobU3KRcSNYBbVqrEHzrx5wG+/aW2NkBA6HRf/nTu1tsQ2OMtk8LNn+ddRxY2fHyeVO+pE7chIJuI2aAA0a6a1NaZx/jzLvjt04AWgoB2ai5uZM2eiQIEC8Pb2RrVq1fDvv/8aff6LFy/Qp08f5MqVC15eXihWrBi2bt1qJ2sFAOjWDejXj7cDB7S2RohP8eJA0aKuG5ry9nYOcePvz/EABQpobcmbvHgBbNzo2InEv/0GXL4MTJ7sHE3vgoKYk1iwIPD7785hsyujqbhZuXIlBg0ahDFjxuDkyZMoV64cGjdujEeJtMSNiIhAo0aNcPPmTaxZswaXLl3C3LlzkSdPHjtbLkyeDNSsyfLGO3e0tkaIT4sWTCrW67W2xPo4i+fG35+DKB2xk+6aNfSMOOq4hefPmUD86aeO61mKjVK86HvwgLmJadNqbZGg6c9uypQp6NmzJ7p164ZSpUphzpw5SJMmDRYk0lBlwYIFePbsGTZs2ICaNWuiQIECqFu3Lso5w7ffxUiVisly3t5Aq1bOsdikJHQ64OFD5ke5Gs4kbhw5JNWgAZA7t9aWJMz48QytjhuntSWmMXky8xEXL3bcnkYpDc3ETUREBE6cOIGGDRvGGOPujoYNG+LIkSMJvubPP/9E9erV0adPH+TIkQNlypTBhAkTEB0dneh+wsPDERQUFOcmWIds2YD16xln/vxzSTB2JGrWBDJlcs3QlI+P4ycU6/XsTly2rNaWvMnNm8D+/Y4bkrp2DZg+Hfj6axYxODp79tDWYcM4i09wDDQTN0+ePEF0dDRyxKuTy5EjBx4+fJjga65fv441a9YgOjoaW7duxahRozB58mR8//33ie5n4sSJ8PX1fX3LmzevVd9HSqdiRSYXL1kCTJumtTWCAU9P9rz580+tLbE+zuC5uXGDNjqi52bpUuYCtWqltSUJ8/XXQPbswODBWluSNPfuAe3aAe++6zxeppSCA0aDE0ev1yN79uz4/fffUalSJbRt2xbffPMN5syZk+hrhg8fjsDAwNe3O5IgYnU6dGBVw5AhwK5dWlsjGGjRgg3Qbt/W2hLr4gwJxf7+/Oto4kYphqQ++sgxBzgeOMCclYkTKcAcmYgIoHVrjj354w/H7RWUUrFI3HTp0gX79+9P1o6zZs0KDw8PBAQExLk/ICAAOXPmTPA1uXLlQrFixeDh4fH6vpIlS+Lhw4eIiIhI8DVeXl7IkCFDnJtgfSZOBOrXB9q25VWroD3vv88Trqt1K3YGz42/P5A5M7v/OhLHjrECyRFDUno9G/ZVquS4ic6xGTyYo07WrKGnSXAsLBI3gYGBaNiwIYoWLYoJEybg3r17Zm8jderUqFSpEnbFutTX6/XYtWsXqlevnuBratasiatXr0IfqwTk8uXLyJUrF1KnTm3+GxGshqcnsGIF4OvLuPOrV1pbJPj6clK4q+XdOIu4KVPG8cqB/fyYRFy/vtaWvMny5RQLU6Y4ZoVZbJYt4zDhadPY+0twPCz6Cm3YsAH37t1D7969sXLlShQoUABNmjTBmjVrEBkZafJ2Bg0ahLlz52Lx4sW4cOECevfujVevXqFbt24AgM6dO2P48OGvn9+7d288e/YMX375JS5fvowtW7ZgwoQJ6NOnjyVvQ7AymTMDGzYwIfDTTyXB2BHQ6djh9eVLrS2xHs4kbhyJiAiGTzp2BGI5vx2CkBBg+HD2ialTR2trjHP2LAcJd+7MQgrBQVFW4MSJE6pv377K29tbZc2aVQ0YMEBdvnzZpNfOmDFD5cuXT6VOnVpVrVpVHT169PVjdevWVV26dInz/MOHD6tq1aopLy8vVahQITV+/HgVFRVlsq2BgYEKgAoMDDT5NYJ5rFmjFKDUDz9obYlw9So/i7VrtbbEevTsqVSVKlpbkTjh4Up5eio1a5bWlsRl40Z+F86c0dqSNxk3TqlUqZS6ckVrS4zz4oVSRYoo9fbbSr16pbU1KQ9z1m83pZJ3ff3gwQMsWbIECxcuxN27d/HRRx/h3r172LdvH3788UcMHDjQOirMSgQFBcHX1xeBgYGSf2NDRo1ir4otW1i1I2hH6dJA1arAwoVaW2IdvvyS3ijDeANHw9+fJeD79wO1a2ttTQytWwNXrgCnT2ttSVwePgSKFOFgzMmTtbYmcfR6epb27mX/qMKFtbYo5WHO+m1RWCoyMhJr165F8+bNkT9/fqxevRoDBgzA/fv3sXjxYuzcuROrVq3Cd999Z9EbEJyfsWM5D6Z9e55QBe3Q6SgyjbSDciocvVrKUClVurS2dsTm+XO2BXDEROJRowAvL2DkSK0tMc6PP3JkxdKlDixsoqN5QAsWZPy2cGHWqKfAHAGLitdy5coFvV6P9u3b499//0X58uXfeE69evWQMWPGZJonOCvu7jwJVKsGfPAB8M8/QPr0WluVMtHpgEmT+Bm4wpRiR8+58fdn0m7mzFpbEsPq1UBUFNs2OBL//QfMn8/E3EyZtLYmcXbuBL75hgKseXOtrTHCpEnA7NlslVy6NDO0u3VjdUH//lpbZ1csEjdTp05F69at4e3tnehzMmbMiBtSE5yi8fVlgnG1aky+W7vW8asgXJF33gGyZmXVlIgb22MISzkSfn5Ao0aO1fFXKZZTFy3q2Im5d+7QA92wIeddOTSHD/Nq0jBGvUABZpEnMZDaFTF7qYmMjES3bt1w9epVW9gjuBglSrBscuNG6eCpFR4ePNe5Skm4M4gbR6qUun4dOHjQ8UJSW7ey6efPP3NWnSMSHs7hwGnS8DzmaFVmb1CjBg/q5cv8/3//8cNPgYmPZntuUqVKhXz58hmd5yQIsWneHPjuO4aCy5fnhYVgX3Q6eqqvXwcKFdLamuRhmC2llOP1kXn1isfYkcTN0qXsRuxIc48iI9nRvH59xw7zDBjABOyDB+n9dHiGDQOCgnhV6eHBHJzx452jK6KVsShI8M0332DEiBF49uyZte0RXJQRI1hp0KkTB20K9uW999gm3hW8N4ZoeHi4tnYkxIULFF2OIm5ij1tIm1Zra2L4/Xfg0iVWRzmaQDWweDEwZw6b9VWporU1JrJqFV1My5cDJ0/yTfz8M/+mMCwqBa9QoQKuXr2KyMhI5M+fH2nj/WpOnjxpNQOtjZSCa0dwMFC9Ohelf/8FJN/cvrz/PpNKd+7U2pLksWYNy5qfPXO8JNRFi9jA8uVLxxATR4/yN7dzJ9CggdbWkBcvWPr9wQdMJnZETp/mcWvfnjY6qgB7g7x56b2J3dj2++/pvrt4UTu7rIQ567dFCcUtHcm/KTgN6dIxwbhyZVZtbNrkBDFsF0Kno5s9MJDJ3s6Kjw//hoY6nrg5e5ZhP0cQNgCwZAmQJw+nVjsK48czrOioOXjPn9PTVbIkMHOmEwkbgK2e41dteHiwSU8KwyJxM2bMGGvbIaQQChcGVq5kftuoUcCECVpblHJo3hzo2xfYvh1o00ZraywntrhxNBwpmTgigr+1nj0d5yLi2jVg+nSWVOfOrbU1b6LXM/H6+XN6uwzfNadBp6N6zJePpeCnTnFY16efam2Z3ZHCXMHuvPce2zFMnMgQsWAf8ucH3n6bzdycGcOCExamrR0J4UjiZutWhu4cqUpq2DAgWzaWgDsiEybwuC1bxj54zkBUFAXs7t0AZsxgedcXX9D1NGQIWz87qpvMhljkuYmOjsbUqVOxatUq3L59GxEREXEel0RjISkGD2a+W7duQPHiQLlyWluUMmjRgq72qChOcndGHNVz8+wZcP++44gbPz+gQgXH6ZR88CDzpZYsYWm1o7F9OzB6NDBmjPNUTkdGshBq3TpeNCJ9euCXX3hL4VjkuRk7diymTJmCtm3bIjAwEIMGDcKHH34Id3d3fOvwXY4ER8DNDZg3DyhWjCWqT59qbVHKQKejy/3wYa0tsRxDtZSjiZtz5/jXERr4PXvGnLbOnbW2hOj1wKBBQKVKjlmVfPMm8wDff5/hcmcgPJzh5Q0bYpLshRgsEjfLli3D3LlzMXjwYHh6eqJ9+/aYN28eRo8ejaNHj1rbRsFFSZOGP8zgYKBtW3oTBNtSuTKQM6dzl4Q7qufG35/N6IoW1doShnv1elb7OAJ//AEcO8b0D0frUh4WxkhOhgwsKnI0+xIiLIytNf76C1i/3rF6GDkKFn2MDx8+RNn/X56kS5cOgYGBAIDmzZtjy5Yt1rNOcHny5+fcm717ga++0toa18fdnd2KnTnvxpHFTfHi7CekNX5+DFPkyKG1JSzgGTaMi3GdOlpb8yb9+vGzW7vWseaBJUZICMPLu3fzd2yYtCDExSJx89Zbb+HBgwcAgMKFC2PHjh0AgGPHjsHLy8t61gkpgnffBaZO5c3PT2trXJ8WLdid3dCh3dlwZHHjCPk2V68y7OgoicRTpwIBASwicDTmz2d4fPZsoGJFra1JmuBgiplDh5j4/N57WlvkuFgkblq1aoVdu3YBAPr164dRo0ahaNGi6Ny5Mz5NgSVnQvLp2xfo2pVZ/8ePa22Na9OwIfNWnDU05YjVUko5jrhZupR5pY4w5uThQ1ZF9u3Lxn2OxIkT7HXXsycLGxydoCDmBJ04weTnevW0tsixsahDcXyOHDmCI0eOoGjRotDpdNawy2ZIh2LHJSwMqFuXFSfHjzuGS91Vad6cV4F792ptiWV4erLqtXdvrS0h9++zWd6GDdqKCqWY81OnDrBggXZ2GOjZk5U8V686VsPFp0+Zf5Y1K3DgQEySuqPy4gWFzcWLFDbVqmltkTbYvENxfKpXr47q1atbY1NCCsbbmyfCSpWY4Ldrl2PkL7giOh2vWp89c448g/g42mRwf3/+1dpzc+QIG+XNnautHQAHUs+fz6pkRxI20dGccRcURHHv6MLm6VOGn27e5DmxUiWtLXIOLBY3V65cwZ49e/Do0SPo47V2Hj16dLINE1ImefJQ4Lz7LjBwIHuyCNaneXPg889ZbeGIpblJ4YjixsdH+8ZvS5ZwvFDdutraoRR7WRUt6jjeNQPjxtH7sW0bCxocmUePgEaN6BncvVv6gZmDReJm7ty56N27N7JmzYqcOXPCLdbwDTc3NxE3QrKoUYOi5rPP2ISsRw+tLXI98uThFeCmTSJurIG/P5vlaVlGHB7OEvDPP9e+nHnrVnoZNm5kebyjsHUrMHYsZ0k6ejLugwfMj3v6lB4mR2nG6CxYJG6+//57jB8/Hl9//bW17REEAIzVnzzJLuKlSlHwCNZFp2MlS0SE84X/fHwcK6HY31/75n1btrBBo9ZVUpGR7Ppfrx6/Y47C9esU8jodMHy41tYY5949oH595sXt28cWA4J5WKTvnz9/jtbSDlGwMdOmAVWrckLv/ftaW+N66HScEH7ggNaWmI8jeW70enYn1jrfZskSeuNKltTWjrlzgUuXgMmTHWeidmgozyNZsvA4ae3ZMsatW0wIDwsD9u8XYWMpFn3ErVu3ft3bRhBsRerUbCvu4cEGYOHhWlvkWlSowPCUM5aEe3s7jri5eZON1bQUN0+fMuSi9biFFy84m6lrV36/HAGl6AG+dImN+jJm1NqixLl+ncJGKQqbwoW1tsh5sSgsVaRIEYwaNQpHjx5F2bJlkSpeULV///5WMU4QcuZke/HatXmCmjfPca4GnR03N3pvNm1ieMqZjqsjeW4coVJq5Up6kNq1084GgFO1Q0KY0+IozJ0LLFoELF7s2Am5ly8zFJUmDZOH33pLa4ucG4v63BQ0UhLg5uaG69evJ8soWyJ9bpyTJUuALl2AX39lCbNgHbZuZcfTc+eY2+QsNGvGRNUNG7S2hAv6zz/Te6KVQKxenSGXzZu12T9Ar0PJksA333C6tiNw7BhQqxbQvTswa5bW1iTO+fNAgwYsmd+1C8iVS2uLHBOb97m5ceOGRYYJgqV07swE4wEDeIWsdamrq2C4Uty0ybnEjY8P8PKl1laQs2f5ndRK2Fy+DBw9Su+NlgwbBmTLxhJwR+DJE+bZVKhAz6SjcuYMq6Jy5gR27gSyZ9faItfAgdOqBCEuP//MeHTr1sDt21pb4xp4e7OPhrMN0nSkaimtxy4sXcqJ1lpWJh06xAG4EyYAadNqZ4eB6GhORA8NpV2OOvLw5ElWlb31FrBnjwgba2Ky52bQoEEYN24c0qZNi0GDBhl97pQpU5JtmCDEx9OTV6eVKwMtWwIHD9LrICSPFi3YS+jxY155OwOOklAcEcGW+Fo1qtPrOWy2deuYmVta2DBoEAdPduqkjQ3xGTOGeSs7drCpoSPy779A48ZAsWJsKOhIXZxdAZPFzalTpxAZGfn634nh5kxZiYLTkTUr8yxq1GCTPz8/50qEdUSaNePfrVuZ1+QMOEpC8ZUrQFSUdp6bQ4dYraVlb5sVK7hQ793rGCXWf/4JjB/PgZ0NGmhtTcIcOgQ0acLeSH/9Rc+bYF1MFjd79uxJ8N+CYG/KlwcWLmRlSIUKjhPjd1Zy5GA/oU2bRNyYi9aVUn5+HCFQu7Y2+w8NZa5Nq1aOkQd39Srz81q2BBy1x+zevRx/UrkyE8DTpdPaItfEAXS2IJhP27Y8eX31FfD331pb4/zodJy34yy9hBxJ3OTOrc3w0bAwjlvo1Ek7j8nUqcDDh8CkSdrsPzYhIeyHlT07S78d0aO7cyfQtCmr27ZuFWFjSyyqlmrVqlWC4Sc3Nzd4e3ujSJEi6NChA4pLa0XBhowfD5w+TaFz/DhQqJDWFjkvLVoAI0fyqrJxY62tSRpHSSjWMpl482Z2mNYqJPXwIUM/fftyQKaWKAX06sWJ6P/8A/j6amtPQmzdSvFVvz6HAzv6NHJnxyK97+vri927d+PkyZNwc3ODm5sbTp06hd27dyMqKgorV65EuXLlcOjQIWvbKwiv8fAA/viD/T1atuQcFsEyypRheMNZuhU7kudGK3GzZAnDiVpdQ44ezS7io0Zps//YzJ7NqrF587Qfg5EQGzfyHPX++2xK6lDC5uVL9tjIn58/rBo12CDIybFI3OTMmRMdOnTA9evXsXbtWqxduxbXrl1Dp06dULhwYVy4cAFdunSRwZqCzcmUiQnGN24A3brxCk4wn9jdip3hGBqqpbS0NSSEngItFtPHj5mIqpXX5swZYP58ViVpXeVz9CjX5n79WP7taKxeDXz8MfDBBw5alt6jB2P7fn5s2vTee2y8c++e1pYlD2UBWbNmVZcuXXrj/kuXLqksWbIopZQ6c+aM8vX1tWTzNiUwMFABUIGBgVqbIliRdeuUApQaP15rS5yX7dt5DE+f1tqSpFmyhLaGhmpnw7FjtOHff+2/7xkzlPL0VOrxY/vvW69XqmFDpYoVUyoiwv77j01AgFJ58ihVo4ZS4eHa2pIQy5Yp5e6uVIcOSkVGam1NAoSEKOXhodTmzXHvr1hRqW++0cYmI5izflvkuYmKisLFixffuP/ixYuIjo4GAHh7e0tZuGA3WrWim3zkSGDLFq2tcU7q1gXSp3eO0JShp4uWoSlDpZQWnZ2XLGEpcdas9t/3X38xMfannzgCQyuiolgxGRnJxOrUqbWzJSEWLWKyd+fO/Lw8LcpwtTFRUex4GD9O5uPDRmJOjEXi5pNPPkH37t0xdepUHDx4EAcPHsTUqVPRvXt3dP7/WNp9+/ahdOnSVjVWEIwxZgxDKx06cAKwYB5eXkwmFnFjGv7+TGK3d0feS5eYEqHFBPDISLZeqFdP247IAC9k9u9nY888ebS1JT6//84wec+eDN95eGhtUSKkT8/SrXHjgPv3KXSWLgWOHAEePNDauuRhiWsoKipKff/99ypnzpzKzc1Nubm5qZw5c6rx48erqKgopZRSt27dUnfu3LFk8zZFwlKuTWCgUiVKKFW8uFIvXmhtjfOxeDFDLQ8eaG2JcXbtop3XrmlnQ+PGSrVoYf/9fvONUr6+2oTkZs5Uys1NqZMn7b/v2BjC0D/9pK0dCTFjBm3r25chPIfn6lWl6tSh0R4eSlWpolTHjjyROhjmrN8WTQWPTVBQEAA4zYRtmQru+ly+zCqSOnWYbOwIXVOdhSdP2NTvt9+YZ+ioHD4M1KxJ74lWDuK33gK6dgW+/95++9Tr6S167z16B+zJixcs+W7enE00teLyZTbAe+89Jug6UvbD5MnAkCH0bv30k2PZliSvXgFBQRxJ3rYty08dLMZvzvqd7NN+hgwZRCQIDkWxYsDy5ewD8u23WlvjXGTNykpQRw9NaR2Wev6cxST2rpQ6cAC4dUubkNSECawQs6eYi09wMHvF5M5NgeVI4mHCBAqbESOcUNgAjK/mysUv9/btLO9yYixOcVqzZg1WrVqF27dvIyIiIs5jJ0+eTLZhgpAcmjZlk78RIziu4cMPtbbIedDpKApDQ7UbxpgUWoubc+f4197ixs8PKFiQXit7cv06MG0af09a5bcoxRyWmzc5yyp9em3siI9SwNixMbdRoxxf2Ny9y4GvDRuCQkYpNky6ehUYOhQoUYJJQ06MRZ6b6dOno1u3bsiRIwdOnTqFqlWrIkuWLLh+/TqaNGlibRsFwSKGDWN/ic6dYypbhKTR6Sgadu/W2pLE0Vrc+Puz+qVYMfvtMzSUYZhOney/eA4bRq/ekCH23W9sZszgkM4FC7SpUEsIpSj4xo5lt+bRox1f2OzYwZl8X37J/GEEBgJ9+lDQdO4M1KpFwaNlKZw1sCSpp3jx4mr58uVKKaXSpUunrv0/q2/UqFGqT58+lmzSbkhCccri5UulypZVqnBhpZ4909oa50Cv5/H67DOtLUmcgADmP27YoM3++/RRqnRp++5zxQq+58uX7bvfgwe530WL7Lvf+DZ4eio1cKB2NsRHr6c9gFJTpmhtTdJERSk1ejQTwhs3VurRI60tMh+b97m5ffs2atSoAQDw8fHBy5cvAbBE/I8//rCW7hKEZJMuHZOKnz9nT4z/t2ESjODmxllTmzc7brdig+dGq/lSZ89qE5J65x37znHS64FBg4CKFbWdYdW6NSuWHWFAJ8Dj0q8fB4f++iswcKDWFhknIIBtHr7/HvjuO865ypZNa6tsi8XjF549ewYAyJcvH44ePQoAuHHjBpSjng2FFEuhQuyFsXMnXchC0uh0bHvhqOlzhp5jWoSllLL/TKlHj4Bt2+wvMFauZH7L5MnaVB1GRrJwRyna4giREr0e+PxzYNYsVqz16aO1RcbZv59hKH9/TlkYOTJlVJBa9Bbr16+PP//8EwDQrVs3DBw4EI0aNULbtm3RqlUrqxooCNagYUNWMPz4I+P2gnFq1eJkZUetmkqVio3RtBA3Dx8Cz57ZV9ysWMEFqW1b++0zNJS5Ni1bAu++a7/9xmb4cJb9r17NQh6tiY4GPv2UjfkWLmSCs6Oi1wM//MCGi8WKAadOcSJ5SsGiaqnff/8der0eANCnTx9kyZIFhw8fRosWLdCrVy+rGigI1mLgQP7AP/2UhQEVKmhtkeOSKhXb+//5p+OW02s1GdyQnG5PcbNkCdCsGZAli/32+csv9N7t3Gm/fcZm9Wp6jKZOpdjWmqgooEsXepCWLnXMIZ0Gnj6lrVu2xCQ8O+T4Bxti0dt1d3eHeyy/Vrt27dCuXTurGSUItsDNjW7k8+d5NXr8uOvHnZNDixYcZXH3LhvWORpaihsfH5Zk24MLF4ATJ+wbUg0IYN+Wvn3tm+Nj4MIFXoS0bcuqHq2JjORvYcMGipuPPtLaosT55x+gTZuYHnxNm2ptkTZYrOXCwsJw5swZPHr06LUXx0CLFi2SbZgg2AIfH2D9enY4bdvWNSoebcX77zP0s3kzcwwcDR8fbRKKDV2R7TUvyM8PyJSJnht7MXo0fxejRtlvnwZevmRfqnz5gHnztC+tDg+nWNi2DVi7lqLfEVGK5fJDhjABfNUqHsOUikXiZtu2bejcuTOePHnyxmNubm6vJ4MLgiOSLx+wZg3QoAFPBNOmaW2RY5IpE1C7NvNuHFHceHtr57mxV0hKr2cIpE0bDja1B2fPUlRMmQJkzmyffRpQCujend2f//2X1Y5aEhpKL83u3cDGjRT8jkhgII/b2rXAgAGsKnO0Ken2xqKE4n79+qF169Z48OAB9Hp9nJsIG8EZqFOHomb6dGDRIq2tcVx0OmDXLo6dcTS0CEvp9exObC9xs28fcOeO/cYtKMW5SIULA71722efsZk6lbk2ixaxp5yWhITQS7N3L8M7jipsTp+mJ/rvvylupk4VYQNYKG4CAgIwaNAg5MiRw9r2CILd6N2bVzuff86rROFNdDq65f/+W2tL3kQLcXPrFoWevcSNnx+FRvXq9tnftm38rH/6yf4L5P79wFdfsfu/1uNSgoOZq3LkCPDXX/TyOhpKAXPnsvdR+vRs26D1cXMkLBI3H3/8Mfbu3WtlUwTBvri5ATNnsmrqww9Z4ivEpWhRXkE7Ykm4FuLGnpVSISEMn9pr3EJUFL02775r/7yS+/cZeqtdm4nMWhIYyIZ3p05xVEHdutrakxCvXrEa6rPPOJn+8GGKYCEGi3Jufv31V7Ru3RoHDhxA2bJlkSpeRmb//v2tYpwg2BovL7pyK1fmHKrdu8WlGx+djqXIer1jNf/SQtycPQtkzMip1LZm40Ym19qrcd/cuRymuHSpfZN4IyMpbDw82M9Hy5Ll588pbK5cYQl8lSra2ZIY58+zY/OtW/ysOnbU2iLHxKKv0R9//IEdO3bA29sbe/fuhVusX4Kbm5uIG8GpyJ0bWLeOV2j9+wNz5mhtkWOh0zFMcewYUK2a1tbE4OPD8IE9MSQT22Px9/MDatSwzxV5YCArpLp0YaWNPRk6lGHhffsALTMdnjwB3nsPuH2bFzmO2Adr6VKgVy+gQAH+HkuW1Noix8Wi67BvvvkGY8eORWBgIG7evIkbN268vl2/ft3aNgqCzXnnHbZT/+033oQYqldn87j/NyV3GLSolrJXpdTDhwyJ2MtrM2ECw2Dff2+f/Rn44w8m9k+ZYr+8ooR49Ijde+/dA/bscTxhExZGUfPJJ6ze+vdfETZJYZG4iYiIQNu2beM08ksOM2fORIECBeDt7Y1q1arhXxOzO1esWAE3Nze0bNnSKnYIKZvu3Tknpl8/4NAhra1xHDw9mVzpaHk39g5LRUYybGMPcfPHHwzTtGlj+33duMFuxF99BeTJY/v9GTh3DujRg2EVLeczPXjAPKMnT1gZVbasdrYkxNWrFH6LFzN0uHgxkDat1lY5Phapky5dumDlypVWMWDlypUYNGgQxowZg5MnT6JcuXJo3LgxHj16ZPR1N2/exJAhQ1C7dm2r2CEIAMsoq1fn1dHdu1pb4zjodMw3uXVLa0tisLe4uXKFAscei5+fH9C8uX36zAwbBmTNyp5P9iIoiEn8hQrRU6pVo767dxmOfvmSYTFH84asXQtUqsTw69GjFINaNzV0FiwSN9HR0fjxxx9Rt25d9OvXD4MGDYpzM4cpU6agZ8+e6NatG0qVKoU5c+YgTZo0WLBggdH9d+zYEWPHjkWhQoUseQuCkCCpUrHPRurUPPlq0QHXEWncmMfGkbw39hY3hkqp0qVtu59z51ipY4+Q1OHD7GQ7YYL9vAFKscLn4UPmumnlhbh5k/2uIiJYhq7FmInEiIhgM76PP2Ye0IkTQPnyGhmzfz+vbnLnprLasCHu40oxYStXLv4oGzbklYDGWCRuzp49iwoVKsDd3R3+/v44depUnJupRERE4MSJE2jYsGGMQe7uaNiwIY4cOZLo67777jtkz54d3bt3T3If4eHhCAoKinMTBGNkz84RDWfPsgeOUlpbpD0ZMtB170h5N/Yev+Dvz/O3rYdX+vnRY2PrmUB6PYfJVqxov9wegMnp69ezAk8rQXHtGj027u5cu+01J8wUbt+m6Jo1i01GV63i708zXr0CypVj34yE+PFHGjpnDgdbpU3LqyGNrwwtqpbas2ePVXb+5MkTREdHv9EMMEeOHLh48WKCrzl48CDmz5+P06dPm7SPiRMnYuzYsck1VUhhVKrE+PYnn/DkLwWAvHgbPJghBU1Ptv/H3gnF9kgmjo5mRUy7drZvSbByJRNT9+yxX4n/7t3A8OG8ffCBffYZn0uXmDycPj27b9szzygptmxhN+p06YADBxykOrFJE94SQikmbI0cGfOBLlnCsrcNG/hF1gizxM2HJrQ/dHNzw9q1ay02yBgvX77EJ598grlz5yJr1qwmvWb48OFxQmVBQUHImzevTewTXItOnRgeGDSIeRb16mltkbbodBR5O3bQXa41WoSlmje37T727mXFjq09KaGhzLX54AN65OzB3btc6+rXB8aNs88+43PuHLsNZ8lCYZMzpzZ2xCcqipGdiRP5HVu82P5zvSzixg3GF2NFX+DrS1V25IjziBtfX1+r7jxr1qzw8PBAQEBAnPsDAgKQM4Fv3bVr13Dz5k3odLrX9xkmknt6euLSpUsoHK8phJeXF7zsNXFOcDkmTQLOnGHTrOPH2V8ipVKgAD0XmzY5lrhRyvZJliEhrFqxtefGz4+hGltfsf/yC7sC22usRkQEf0Pe3sDy5fabqB6b//7jGpwnD993tmz2tyEh7t8H2rdnheakSUzsdqRmmUYxtHWP36AoRw7NW76bJW4WLlxo1Z2nTp0alSpVwq5du16Xc+v1euzatQt9+/Z94/klSpTA2bNn49w3cuRIvHz5EtOmTROPjGB1PD3ZNbVKFaBVK56A0qTR2irtaNGC1S3R0dosULHx8eHf8HAumrbkwgWKKFuKm1evOG7hq69sK9YCAphA3KcPUKyY7fYTm0GDOPvowAFtRMWJE0CjRqzO2rHDcbwiu3YBHTrwt7R7N3NtBOuguT4cNGgQ5s6di8WLF+PChQvo3bs3Xr16hW7dugEAOnfujOHDhwMAvL29UaZMmTi3jBkzIn369ChTpgxSS998wQZkycLw8eXL7IWTkhOMdTrg6VN6nLXGIG7sEZoyVEqVKmW7fWzYQIHTqZPt9gEAY8aw8m30aNvux8DSpcxFnTYNqFrVPvuMzdGjDEUVL86RCo4gbPR6huYaNWLI+/RpJxU2hghLvOgLAgI0j/lpOMWDtG3bFo8fP8bo0aPx8OFDlC9fHtu2bXudZHz79m2rNQsUBEt5+21g0SI2VatYkS3jUyJVq7KabNMmoFYtbW0xiBt7FGX4+7OiJl062+3Dz4/H1JbdLfz9mSg/ebJ9FvkzZzjcsUsXdti1NwcPMhe2fHkm6zpCIvzjxxSwf/9NgTlqlPZeUIspWJAiZteumFr1oCBWTfXuralpUCmMwMBABUAFBgZqbYrghIwYoZS7u1LbtmltiXZ066ZUyZJaW6HUzp1KAUpdu2b7fb3/vlItWthu+/fv83v122+224dSSjVurFSRIkqFh9t2P0op9fy5UoULK1WunFKvXtl+f/HZvVupNGmUqldPqZcv7b//hDh4UKk8eZTKlk2pHTu0tsZEXr5U6tQp3gClpkzhv2/d4uM//KBUxoxKbdyo1JkzSn3wgVIFCyoVGmp1U8xZv8UlIghm8N13wPvvswjg6lWtrdGGFi2Yg6L1+7d3WMqW+TbLlzNU1Lq17faxbRuwfTv7zNg6gq/Xs6T56VN22bV3ntqOHewTVLMmsHmzbT1upqAU8PPP7K1TsCCrMBs10tYmYzx5wnNddDRYSVGhQszArUGD+G9DXPOrrziz5rPPmJwYHMwvm60T4ZLC6tLKwRHPjZBcnj9XqlgxpUqXViooSGtr7E9wsFJeXryA05KTJ3kheeyYbffz/Dn3s3y57fZRrpxSH39su+1HRipVqpRSdesqpdfbbj8Gxo/nMdu0yfb7is/mzUqlTq1Us2Y2cR6YzbNn9PoBSn31lVIREVpblDhRUUrNnq1Upkx0xpw9q7VFcRHPjSDYkIwZmfx5+zZzCf7fjSDFkDYte5VoPYrBXp6bc+f411aemzNnWKZsy9428+bR2zZliu3L5v/+m3kko0bZvi9QfNavZ1Vj06Yc7aC18+D4cebo7d8PbNzIUu9UqbS1KTH+/Rd45x2myrRsyWaH9hgSaytE3AiCBZQsySqQ9euB8eO1tsb+6HQ8YT9/rp0N9hI3/v5sCVC8uG227+fHirz337fN9gMDGUHo3JkLrS25fZs9Wxo2ZFWWPVm1imG9Vq34by2LZ5VihVjNmix9P3WK4VxH5OlTRpTeeYfNBA8dAhYsYOGAMyPiRhAspEULYOxYLhyONHPJHjRvznj8tm3a2WC4Krd1tdTZs+wHY4vFMjqa+Tbt29tuMZ44kWkQthbh4eFs7pgunf0b9S1dymPYvj2wbJm23pGXL2lH376sEDtwwDGbf+r1wO+/87u9ahXHQx07BtSoobVl1kHEjSAkg5Ej6cLt1AlIZByaS5I3L3MKtQxN2dNzYyv3/O7d7FBrq5DUjRvA1KnM+bT1DKUvv2SIbc0a2w8Xjc2CBfRKdenCdg2eGjY4OXMGqFwZ2LqVs7umTwccsUH+8eP01PTqRS/spUsUY1oeO2sj4kYQkoG7O+fE5c3LOT0vXmhtkf3Q6YC//gIiI7XZvz3EjVK2FTd+frxyrlLFNtsfPpxCw9Z9mRYtYufqX3/l4m4v5sxhY81evZhXpGW/mIULOTbD25vioU0b7WxJjGfPmFNTtSo9bQcO8LOLPz3BFRBxIwjJJH16Jgs+egR07Pj/8skUgE5HMXfokDb7T5WKi5ktxU1AAHMSbCFugoNZJt25s22SfI8cofdgwgQmgduKU6e4YHbvDvToYbv9xGf6dO63f39g1izt5jGFhADdugGffsrf/9Gj9htrYSp6PcVfsWIMGf7yC0dSaN2I05aIuBEEK1CkCPDHH8xBsXcipVZUrAjkyqVtvpGtJ4Mbxi6ULWv9ba9fz4WxY0frb1spYOBAhg47d7b+9g08fw589BHHUvz6q+32E5+ffmIYbOhQLtS2rgBLjEuX6K1ZuZIekHnzYjyKjsKJE8yj6dmTVWSXLlEQulIIKiFE3AiClXj/fSZvjh/PvANXx92d3ptNm7Sbt+XjY9uEYn9/7qNgQetv28+P84RskWy6ciU74E+ebDuPhl7PXLPAQHqg7FV2/f33zCEaOZKl1VoJmxUrGIKLimIZdZcu2tiRGM+eAV98wZBnSAiwbx9D6BqPfLIbIm4EwYoMHQq0bcsT3ZkzWltje3Q6diq+dEmb/Xt7295zU6qU9XM57t3jEEdbJBKHhgJff80csHr1rL99A99/z5yrZcvsUw2kVMwspu++4+BJLYRNeDhFQ/v2/P4fO+ZY/WD0eiZZFy/OKrIpUziR3SkHcyYDETeCYEXc3ID584GiRVlF9eyZ1hbZlgYN6NnQqmrKHmEpWyxcy5ez9NsW4xamTWMF1o8/Wn/bBrZtA779ljdb9eeJjVLAsGEUNJMmUeBowfXr7F0zfz6TmZct0360Q2xOnWIeTffuQOPGvOgYMMD1Q1AJIeJGEKxM2rTsYBwUxBlUUVFaW2Q7fHzYsE2rvBtbihu9nt2JbSFu/PzoWfH1te52AwKYQNynj+2SWm/cADp04LTtkSNts4/YGPKHfvwxpqxdCzZuZJ7Z8+dM1u7VS7uQWHxevGApd+XKPO/s3UuvTa5cWlumHSJuBMEGFCjAxli7d/OK05Vp0QI4fJhVRfbGluLm1i1WNFlb3Pz3HxsD2iIkNWYMr9INMw2tTVgYG/VlzEiBZusKJb2eQm3aNFZEDRhg2/0lRGQkMHgwPbH16zNB19adnk1Fr2cic7FiwOLFTLQ+dYoDOlM6Im4EwUbUr8+EzsmT6b52VZo140l261b779uW4sZQKWVtcbNkCVvyN25s3e36+wNz51LYZM5s3W0b6NsXOH+eCcS22oeB6GiOBZgzh1VIvXvbdn8JcecOhcL06cxdWbuWws4R+O8/5tF060bv6aVLHNhtt+7M+/cz6Sh3brqwNmyI+/i6dcB777HRkpsbcPq0nQwjIm4EwYb078/k4h49mNTniuTKxYoMLfJuvL1tVy3l78+wkTU7+0ZFxYxbsPYiNGQIUKgQk11twbx5zDWZPZsl5rYkKoqL9sKF9Eh0727b/SXE9u18n3fucB0fONAxwlAvXrAMvmJF5vTt3s3vVO7cdjbk1SugXDkO0Urs8Vq1mCSlASkwzUgQ7IebG688z5+nW/v4cecfSJcQOh1d4hER9h1Y6OPDc6gtMCQTW3NB27ULePjQ+iGpbdu4GK9bZ5vjf/x4zKykrl2tv/3YREby+KxZw0W7bVvb7i8+0dFMlB4/nt41Pz8ga1b72pAQSjGPZuhQhksnTaLI0WyOVpMmvCWG4Ut+86ZdzImPeG4EwcZ4e3PRCQ9ndYxW4wpsSYsWHBi4b59992vrsJS1m/ctWQKUKAFUqmS9bUZFMSekTh0KaGvz9CnzbN5+m7kvtiQigmJm3TrmrNlb2Dx8CDRqxKTs778HtmxxDGFz5gw/386dgXff5Ry7IUO0HRDq6Ii4EQQ78NZbjNcfOcK4uKvx9tucr2Xv0JStxE1kJBcQa+bbvHzJrsTWHrcwbx49g1OmWD9sEh3NDsrBwfSk2HIIZFgYux1v2UJx8+GHtttXQuzdyzDU+fPsQTRihHYjHQwEBjKJumJF4MkT2rViBc8ngnFE3AiCnahVC5gxg23qFyzQ2hrr4uamTbdiW4mbq1fpRbCmuFm3jrZac9xCUBATiDt3tq43yMDYscCOHRwtki+f9bdvIDSUpfE7d7KtQPPmtttXfPR6emoaNKBX7fRp2zY/NAVDCKp4cYrXCROYQNyggbZ2ORMibgTBjvTqxQqQ3r05YM+V0OkYXj93zn77tNX4BUOlVOnS1tumnx9DCtYUCRMn0qsyfrz1tmlg82Y2zfv+e4ZqbMWrV6y4O3iQXhtrV5EZ4+lTCqlvvqGn5u+/tR9P4O/P78knnzAUdeECe/vYM5fNFRBxIwh2Zvp0Ntv68EPgwQOtrbEe9eqxW6s9G/rZavyCvz8XOWvlW9y9y6oWayYS37zJpnZDh1o/THHtGm3V6Wzbp+nlS+akHjvGpOj69W23r/gcPcow1L//cozEuHHadvINCmLIunx55v7s2MG8o7x5tbPJmRFxIwh2xsuL+QtubswxCA/X2iLr4OXFthb2zLuxVVjq7FnrhqSWLePx+fhj621z+HD2mhk61HrbBDhk8aOP2J5kyRLb5Z28eMHvy3//cSGvXds2+4mPUpwkXrs2ReGpU/YZIWHMnuXLGRL77Td6ys6csa23LDno9f//R3AwY3iG/jU3bvDft2/z/8+e8f/nz/P/ly7x/w8f2sdQlcIIDAxUAFRgYKDWpggpnH/+USp1aqV69FBKr9faGuuwcKFSbm5KBQTYZ3+//KKUj4/1t1u0qFIDBlhnW3q9UqVKKdWunXW2p5RShw8rBSi1YIH1tqkUbe3cmcf0v/+su+3YPH2qVKVKSmXKpNSxY7bbT3xevFDqww957AYNUioiwn77Tgh/f6XefZf2fPSRUrduaWuPMU6eVOrjj5Xq2vX/d+zZQ8Pj37p04eMLFyb8+JgxFttgzvot4kYQNGTBAv7eZ83S2hLrEBBAcWPtRTcxfvuNx8+a4jAkhO9h3jzrbO/ECdq4ZYt1tqfXK/XOO0qVL69UVJR1tmlg9mza6udn3e3G5tEjpcqVUyprVqVOnbLdfuJz8qRShQop5eur1Lp19ttvQgQFKTVkiFKenhTS27Zpa48xDh9WqmlTfi8KFVJq/nztbDFn/ZawlCBoSLduQL9+7GR84IDW1iSf7NmB6tXtl3fj48O/1gztXbjAS0xrhaX8/Hhc3nvPOttbtYr5IpMnAx4e1tkmAPzzD7+HX3wBdOpkve3G5uFD5mY9fAjs2cP8ElujFMM91atzdMLJk0CrVrbfb2K2rFzJENTMmaxGO3vWvknUpqAUG07Wrw/UqMH8rqVLGVn69FOtrTMNETeCoDGTJ7NM/OOP2erd2dHpmENhq7EIsfH25l9r7sualVJRUSyj7tDBOsmqYWHA11+zaaI1k28fP+b3r1IlJinbgnv3WAX07Bl7ythi2np8goMp1D7/nIvyoUMcUaEFFy5wBlS7dkC1avz/iBG27R1kLkqxSq56ddoaGMj+XGfPsoWBlgnX5iLiRhA0JlUqXo17e/OK0lYdd+2FTsek1D17bL8vg+fGmsfM3x8oWJCVX8nl77+BgADrVUlNm0aR8OOP1tkewEZ97dvT+7V6tW1Kjm/f5gDKkBDOaSpRwvr7iM+5c5x5tnEjE7pnzYoRw/YkOJiC9O23OWl+61b2PMqf3/62JEZ0NM9BFSrw9+vpSTuPH2dVp9bNDC3BCU0WBNcjWzYO1T1/nn1w7NkIz9qUKkVxYI+qKVuJG2t5FZYs4fGwxqDJR4/Yz+aLL9jczVqMGkUhaqvOtzduUNhER1PYFCli/X3Ex88PqFqVYbvjx+k5szdKUSyWKMH2D2PG8LtlbByTvYmM5GDS0qU56iJ7dnrVDhygnY4wKNRSRNwIgoNQoQKnLi9dylJVZ8XNjWETe3QrdmRxExREwWqtcQtjxnCxHj06+dsysHEjGwFOnGibHjNXrrARnacnhU2BAtbfR2xCQ4GePXnMW7dmDxt7eInic/Eic6zatGFPqwsXgJEjtfEcJURYGKe7FyvGQajFizPnascOClFnFjUGRNwIggPRvj37lgwZwlb0zopOx8Z1hhYYtsLa4iYwkHlP1hA3a9cy1GONcQvnzgG//05hkyVL8rcHUHh07sxQqLV75QBc0OvWZXhv3z7bN6O7coW5IkuX8iJh0SIgTRrb7jM+r16x6eHbbwPXrzN/ZcMG24s6U3n1ijPIChUC+vQB3nmHPXU2bqSny5UQcSMIDsbEiUzma9uWLn1npHZtIEMG24emDOLGWgnFhmRia4ibJUtYGWSNUM+QITELkjV49YqN+nLmBBYutP6VumGEQJYsDHPkzm3d7cdn9WomQ4eE0ANh74oepdiYs2RJ5kWNHElB2qyZfe1IjBcv2Bwwf37m/7z/Pr1Lf/xh/cn3joKIG0FwMDw8eNLJmBFo2ZILkbOROjVPoLYWNwY3v7U8N/7+PP7JzWm5fZuLeufOybdp2zbefvzROsm+SjGv69o1Jrb6+iZ/m7E5dYrCJndu5vLkyGHd7ccmIoLl623a8Pt2/Di9Jvbk8mXuu3VrlrafO0cPmyOEoB4/5tys/Pkpbtq141DYBQsYknJlRNwIggOSOTNdxdeusReOMyYYt2jBxeb+fdvtw9phKX9/nvSTW567bBlt+/DD5G0nKopemzp1KHStwcyZbPc/b551B4MCnBFVvz4Tynftst5sroS4dYsewjlzgF9/Zf+YDBlst7/4vHpF4VCmDENif/7Jm1al5rG5dw8YOJDhsGnTOLD35k0eJ5tUae3fz1h07tx0A27YEPNYZCTdRWXLAmnT8jmdO9v2xAARN4LgsJQpw9DG6tXApElaW2M+TZrQC7J5s+32YQtxk9yQlFKs1mnVCkifPnnbmj+fnoApU6wTOjp8mIte//7M77Imhw8znFqyJPPFMme27vZjs3kzE/AfPWLvmj597JcEqxSwfj2r4CZPZq+ac+e4tmvNjRvs6VOoEHOOhgyhCPzxRxtPO3/1CihXjso5PiEh7Jw4ahT/rlvHboAtWtjQIMhsKUFwdEaO5DiArVu1tsR86tRRqnlz220/IoJt4RcuTP629HqlsmRRauzY5G3n2DHa9NdfydtOYKBS2bJx1pM1ePhQqdy5lapZU6nwcOts08C+fUqlTcvPOyjIutuOTWSkUl99xeOr0yn17Jnt9pUQly8r9f773H/TpkpdvWrf/SfG+fNKffKJUh4e/M5MnMjvjyYASq1fb/w5//7L55k5TEvGLwiCCzF2LBMT27dnfN+Z0Ol4FR8SYpvtp0pF75A1PDePHgFPnyY/wdLPj1fJDRsmbzsTJ7IB3PjxydsOwPBWu3Yxzdqs2ahv1y7mnFSrxsZvyfVWJca9ewx5TZ5MT8TGjUCmTLbZV3xCQuh4KFOGVWAbN9J7VLiwffafGKdOMdendGnmN02ZwvDTsGH2DdGZTWAgXW0ZM9psFyJuBMHBcXdneWvOnMy7CArS2iLTadGClUy2LGv39rZOtZQ1KqUiI60zbuHmTY5BGDrUOtVWI0awMdvKldatXNq2DWjenCXfmzczpcIW7NzJMNS1a0zUHjrUPmEopShkSpWioPr6azbabNFC214whw/zgqdiRUZ6fvuNicL9+9u//N1sDDNE2re3qQITcSMIToCvL0+y9+4xF0+v19oi0yhWjDdbVk35+FjHc+PvT6GUnITQ7dtZoZLccQvDhzNnxRr9Z9auBX76iYtz3brJ356BTZuADz4AGjVi/qgh/8maREfTc/nee6xEOn2ac9jswbVrFG4tWzKP6Nw54LvvtBMPsYdZ1qwZd5hlz56ONaMqUSIjWdqmFLsI2hARN4LgJBQvziqXP//kSdZZ0Ol4VW8rQWZNcVOqVPImbfv5MaxVrpzl2zhyhKMQvv8++fOtLl1itd3HHzOR2FqsXctKMJ2O/V1ssbA+esSk9LFjgW+/Bf76i2NKbE1oKLtBly7N78T69Qy32WNsREIoRSHp9MMsDcLm1i0OXbNx3EzEjSA4Ec2aAePG8YQfu9rSkdHpgIcPgRMnbLN9a4qb5ISkAgPpXfvkE8tDFkoBgwZRHHXpYrktAPN1PvwQyJOHfU2sFUZZsYINJj/+mP+2xaDNAwcYhvrvP66Do0cnT3SayqZNFLg//MBKowsX6LnRIgRlyI8qX55hsFSpKPCccpilQdhcucIYo7XabBvBmQ6PIAhg/sRHH3ERPX9ea2uSpmZNJn7++adttm8NcaPXJ1/crFnDpnLJGdK4ejVw9CiTZpOzmCsF9OjBZoLr1lkvyXfJEnoLOnZkSMTaXgO9nuGzevXoKTl1CmjQwLr7SIjr1ynCW7RgGPXsWXrOtAhBRUayjNswzDJHDo6vOHCAiduONPcpJOT/TUaDgxkzNMxbuXGD/759m2/o44+pypYto2p7+JC3iAjbGWdWHZYLIKXggivw8qVSZcooVaSIUs+fa21N0nTsqFS5crbZ9jvvKNWtW/K2ceMGK1OTU25fp45SDRta/vrQUKUKFGCJc3KZOpXvZ9Wq5G/LwLx5bEnQo4dS0dHW266Bp0/53gGlhg1j2betCQlR6ttvlfLyUipvXqXWrmVLAC0IDVVq1iyl8ufnMWjRQql//tHGFmPo9Wx30KuXUhky8Lum9uyh0fFvXbrE/LgSuu3ZY9a+zVm/RdwIgpNy9apSmTKx70ZUlNbWGGfFCovaWpjEu+8q1b598raxaRPtu33bstcbzt9Lllhuw6RJSnl6KnXxouXbUEqp/fu5nUGDkred2Mycyff3xRe2ETb//MNFPVMmpTZvtv72E2LzZqUKFVIqVSqlhg9XKjjYPvuNz8uXSv38s1K5cinl7q5Uu3ZKnTmjjS3GePpUqWnTlHr7bX4X8uRR6ptvlLp+3X42iLgxgogbwZXYvp0nxGHDtLbEOC9ecMGdOdP6227SRKmWLZO3jYkTlfL1tfyq/fvvlUqThguVJQQE8Cq4Xz/LXm/g/n2lcuakFykiInnbMmDwAg0caH2vhl6v1PTpFBhVqyp186Z1t58Q16/TKwIo1ahR8sWkpTx/rtS4cWwc6emp1KefskmgIxEdrdTff1NwpU5NOz/8UKktW7S5oBJxYwQRN4Kr8dNPPFGvXKm1JcZp0ECpxo2tv90PP0z+djt2ZOdeS9DrlSpWTKlOnSzff+/eSmXMqNSTJ5ZvIyJCqdq16QF48MDy7cTmhx9iwkTWFjaBgUq1bs3tf/ml9bsmxyc0VKnvvlPK21upt95SavVqbUJQjx7RU5QhA8NhffrYR9SZw+3b7NRdoAA/nxIleJ55+FBbu0TcGEHEjeBq6PVKdehAz8Hp01pbkzi//MKrP2u35+/YkZ6K5FCuHHMILOGff7gA7Nhh2ev9/el9mzzZstcbGDiQV9YHDiRvOwa++47va8wY64uA06eVKlpUqfTpKTJszdatShUuzOPz9deWe9iSw927Sg0YoJSPD0dVDB1qPRFqDcLD+Vm8/z5zq9KmpTfp0CHt8pDiI+LGCCJuBFfk1SulKlTgldbjx1pbkzDXrnGxXLvWutvt0UOpKlUsf31kJEXXjBmWvb5vX85sstRN36QJF96wMMterxS9dgAFZHLR65lLASg1fnzytxd/2/Pm0XtSrpxSV65Yd/vxuXGDIUtAqfr1OYPJ3ly7ptRnn/E7ljGjUqNHJ89DZ23OnWN+VtasPE7vvKPU3Lm2nRFmKSJujCDiRnBVbt7kCap+fftUmlhCqVJKde1q3W327atU2bKWv/7CBYsKN5RSvNrNmlWpIUMs2/e2bckXfOfO8Sq7XbvkX2Hr9XwvAJNcrUlwMIeAAkr17MlKJVsRFsY8KB8fCs+VK+3vfYg9zDJ7dob4HGXZCQqigHnnHX4eWbPS8+fvb4WN79vHabm5ciU8RHPMGKWKF6erOWNGxquPHjVp0yJujCDiRnBl9u7lyXTAAK0tSZivv+aJ1JrJiEOHsiTeUlav5jn40SPzX7txI1/733/mvzYyUqnSpZknY+nCGxjIdaJ06eSHWvR6pfr35/uZPj1524rP+fO0MU2a5FWUmcK2bfw+eHryu2FvD8TJk0p99BFDO2+9xQqjV6/sa0NC6PUMMX36KcWwmxtDUKtXWznfaetWuv7WrUtY3Cxbxizla9eoprp3ZwKSCT9AETdGEHEjuDozZvCcsnix1pa8yaFDtO3QIettc/RolqUm5/U5clj22o8/trx/z2+/8VgcO2bZ6/V6LqLp0yt16ZJl2zAQHc2cI4B2WZNly7iYlixpJc9AIty6xeRyQKl69ejRsieHDinVtCn3X6gQPSPJCTVai4AAeuFKlqRt+fMzWdgWbRneICFxE5/AQD5v584kNyfixggibgRXR69nUzsvL8sXTlsRFUXPjTVL1ydOVCpzZstf/+GH9Iyby7NnPMaWhG8CAxmq+OQT819r4OefuSasW2f5NpTiZ9KtG6/kFyxI3rZiExqq1Oef08aOHW2XxBsWptSECQxB5cql1B9/2C8EpddzTX73Xb7PUqWUWrpU+7BwVBTLtT/8kB6s1KmVatuWDhNb9ClKlKTETXg4y7B8fU1KFhRxYwQRN0JKIDSUfUPeekv78s34dOnCEIW1+OUXLmyWUqwYS5HN5bffWOV0/775rx0+nDZb2jRwzx6GH7/6yrLXG4iMpPBwd+eibC2uXmWCu5cXj5OtxMb27fz8PDyUGjzYfiEovV6pP/9Uqlo1rt+VKlFk2lU4JMD160qNHMnfPcBctGnTNExgTkzcbNoUExvLnVupf/81aXMibowg4kZIKdy9y4ZutWrZvoeIOaxZw3PetWvW2Z4hvGPJAhoSwoV97lzzX1urlmX9dW7e5KI/apT5r1WKn2v27Ay9JMdDEBHBPjOentYd07BuHS/ECxdm/oktuH2bIUFAqbp1lTp71jb7iU9UFLttG7r01qql1F9/aVsqHRqq1PLlLCQAmL7Sqxf1guYl3ImJm+BglsodOcIkoAIFGD9LAhE3RhBxI6QkDh1i99fevbW2JIagILrJrVG2rBRziwDL8htOnuRrTSzWeM3163ydJd6O9u0pOi0J04SHK1WjBnOMTFgLEiUsjCXSqVIptWGD5duJb9vAgTwuH33ErtTWJjycVUdp0vAYLltmnwU8IkKphQvpJTJ0Nt63z/b7Ncbp06wUzJSJNtWuzd+CIyQvv8aUnBulmAE+YUKSTzNn/bbyTFdBEByJGjWAmTOBzz4DKlQAevbU2iJOqK5XD9i0Cfjyy+Rvz8eHf0NDAS8v817r78+/pUqZ97qlS4G0aYGWLc173dGjwB9/APPnA+nSmfdaABgyBDh2jFOis2c3//UAEBbGqfK7dgEbNgBNm1q2ndjcucMJ1seOAb/8AvTvb/3p1Tt3An37AlevcvvffgtkyGDdfcQnLAxYuBCYNAm4dQv44AN+9lWq2Ha/ifHiRcz358QJIGdO/rY//ZTTzJ0WvR4ID7fuNk3QXjbn119/Vfnz51deXl6qatWq6h8jo1B///13VatWLZUxY0aVMWNG1aBBA6PPj494boSUSO/evEq3ZpVScvj1V4ZDrHF1bxh6aUnuy1df0SNuDno9LzQ7dzb/ddWrs7rKklL4Zcv4Pn/91fzXGnj1il4HHx/LOyrHZ+tWzkfKm5dRBmtz545SbdrEeCfsMVQy/jDL9u21G2ap17PFQ6dObH7o4cHp6Rs2WG9+mFV5+VKpU6d4A5SaMoX/vnWL4ajhw/lFuXlTqePHY6ofTCilc6qw1IoVK1Tq1KnVggUL1Llz51TPnj1VxowZVUAiPtcOHTqomTNnqlOnTqkLFy6orl27Kl9fX3X37l2T9ifiRkiJhIczPyBnTuZsaM3Nm9abh7Vzp+U5PE2bst+YORw5YnLlahwMXYTNfZ1SzClJk4YLnKVhmJcvWdWTNq1lDQvjExmp1IgRfE9Nm1o/aTU8XKkff6S9OXIo5edn+xDU8+ccO+EIwyzv3WOkpkgRHuMiRVgZeO+eNvYkRng4ewu9rhrcs4cGx7916cIEoVatmEScOjXVY4sWrplQXLVqVdWnT5/X/4+Ojla5c+dWEydONOn1UVFRKn369GqxiU09RNwIKZWHD1lFUbUqzzFaU64cK3WSi6F3jiU9VPLlM78s/YsvmPNijvclNJQeIp3OvH0pRe9W0aJMYrU0nyIwkINB06dX6uBBy7YRm/v3KZTc3bngWrtKaNcu9mVxd2clmy3yd2LjKMMsIyLokdHp6KHx8WG7gL17HSA5OBaBgUysbteOxwzgd9TWfX2cJucmIiICJ06cwPDhw1/f5+7ujoYNG+LIkSMmbSMkJASRkZHInDmzrcwUBJcgRw5g/XqgVi3giy8Yt7d2XoQ56HTMB4qKAjyTcSby9ubfsDDzXhcYCNy+DZQpY/prIiKAFSuAHj0ADw/TXzd9OnD3LrBtm3k2KgV07Qo8egQcPw6kSWPe6wHmabz/PnDxIvD330C1auZvIzZ79gDt2wPu7sDu3UDdusnbXmzu3WNe0YoVQM2awMmTQLly1tt+Qvv7+Wfgt9/4efbuDQwaxFwWe3L5Mn+PixcDAQFApUrAjBk8zhkz2teWxHj4EPjzT+Zp7drF30KFCsDgwUCrVvwdaXk+eQPb6izj3Lt3TwFQhw8fjnP/0KFDVdWqVU3aRu/evVWhQoVUaCKXomFhYSowMPD17c6dO+K5EVI0huoiSwdFWgvDNO3kVp2cP8/tmDsN2+DxMWeS+vr1fI05pccBAby67dfPPPuUYmUQwDEPlvDkiVIVK7LJ4YkTlm3DQHQ05zW5u7Ps2Jr9kyIiGNZIl45l7osX29ZTEX+Y5ZgxSj19arv9JURwsFKLFjGPCGDVU79+5n0fbc3lywwNVq/OljQeHmxBMG2aNp4tpwlLJVfcTJw4UWXKlEn9Z2Swy5gxYxSAN24iboSUzIABPFFZI/fCUqKjmQNk6dBJAzducHEwN0H2t994DMwJ0X34IZvTmUPv3lxAzc1J2bmTQmLECPNeZyAggKGsbNksm30Vm8ePOYfIzY39eaw5G2zPHnb2dXfn4v78ufW2HZ9z55i3pNUwS72e6SW9esWEcxo0YJ8aRwgV6/Xsaj5iBD8TgKGxli0pxEz+Dic1PDM2hrkfU6cmuVmnETfh4eHKw8NDrY/3xjt37qxatGhh9LU//fST8vX1VceS6C8vnhtBeJPISF59Z81qpxkzidCjB3uHJIeHDy3zbvTrp1SJEqY//+lTXulPmWL6a86d46Jt7oiG27f52TRsaJmQuH+fi1POnMmfsXToEHO1smZl4qi1uHdPqQ4d+NnVqMGCGluh9TDLJ0/Y16lsWb7fPHnYSdhajSyTQ0QExzL06RPT2ThzZub/rl9v4XFKanimgXXrmHyXO7driRulmFDct2/f1/+Pjo5WefLkMZpQPGnSJJUhQwZ1xIK6Q0koFgTy+DGTXCtU0K7xl2GqdnIGPxrm7q1YYd7r6tVjh15TmT2bQuXBA9Nf07Qphyiak2gZFsa2/nnzmjRu5w3u3qVgzJMnecdVr1dq8mRWDdWsyZJsaxARQYGYPj29SgsX2m5sQexhloULsxO1vbp1R0fTm9i2LUWxpycF1tat1vV8WcLLl+wU3qkTvYoAk+v796cnzaqzsRITN3fv8kvq789pnq4mblasWKG8vLzUokWL1Pnz59Vnn32mMmbMqB7+P6D7ySefqGGxyhl++OEHlTp1arVmzRr14MGD17eXJrb7FHEjCDGcPk23c4cO2lRjvHrF3h2WDJ80EBHB8+fChea9Lls2Tkc2lRo1lGrSxPTnb99Ou9asMc+u3r25GJrRvus1N29STOXLx/lOlvL8OUMRAMOG1uqnsm+fUmXKUCT26cPho9ZGr6cnIvYwy2XL7DfM8vZtfq8KFOD+S5bk9zs5HaWtwaNHSs2fz2iRl1fM7KnRo+nZstnvPyFxEx3NqwtDm3JXFDdKKTVjxgyVL18+lTp1alW1alV1NFYv9Lp166ouXbq8/n/+/PkTzKEZM2aMSfsScSMIcVmxguef5AiM5NCsGecDWYpez8Vy9mzTXxMQwPe8dq1pz796lc9fvty050dFcRGvVcu8RcOQ7D1njumvMXDtGkVNoULJS/Y8flypggV5RW+t0QwPHtBLACj1zjvJT25OCMMwy6pV7T/MMiyM87kaN2boK21a9sg5fFjbEu5r1+glq12bvxE3N/578uTkiV+zSEjcTJjAbpKGg+Oq4saeiLgRhDcZNownv+3b7b9vQ2JvcqpV0qY1Lxdm1y7zwmHffsswiqnhu99/5/ZN7E2mlKIXzdtbqa5dzV8QL12ih79YMcvDR3q9UrNm0WtUqRLnZyWXyEhenGfIwJyd+fOtLza0HGbp7895Wlmzct/Vqys1b579ppPHR6+nF2b06Jjj4eVFb828eRp5j+KLm+PH2ZUxdjdCETfJR8SNILxJVBSrYTJlsuMV3f+5d8/yIZQGsmZVavx4058/bRpP+qbkPuj1zNfo1s20bQcFsRKnUyfT7Xn2jB6X8uU5qdwczp1j4nCpUpaNoFCKNrdvz8+hTx/rNGM7cIBhDzc3htqsXWodf5jle+/ZZ5hlUBBzd955h/vNmlWpQYOSn7htKZGRzJP58ktqBIBT2Tt2VGr1assGtFqV+OJm6tSYunLDDeDVVf78RjflNE38BEFwDDw8gOXLgapVOQzyyBHLBjtaQu7cbFq2aRPQsaNl2/Dx4eBMUzl7lsMyTWnEd+QIcO0aMHeuadv+4QcgKAiYMMG05+v1QOfOwLNnbLJnGARqCmfOAA0bsunczp2WDdP09wc+/pgN7Vas4ADM5BAQAHz1FbBkCb9P//4LVK6cvG3GJiwMWLAA+PFH+w2zVIrfg3nzgFWrgJAQoHFjYM0aNqNMndp2+06IkBB+VzZs4O/m6VMgTx7+dlu2ZGPFVKnsa5PJfPIJv7SxadyY93frZrXdiLgRBAEAkCkTsHEjO9h27QqsXm2/jqM6HTB1KrueWrJQmCtu/P1N70zs5wfkzWtaJ95bt4DJk4GhQ/kaU5g4Edi8GdiyBShUyLTXAOze26gRUKAAsGMHkCWL6a81sHgxu/IWKcIOyMWLm78NA1FRwOzZwMiRXFjnzuW0and3y7cZm+BgdhL++Wd2bG7blgt72bLW2X5CPHpEkTZ/Pjs8FygAfP01fx+mfr7W4ulTfkfWrwe2b+f3vVQpoFcvCppKlax3rJNNcDDHtxu4cQM4fRrInBnIl+/NL2uqVFToyfkCxidZ7iYnRMJSgmAcQxfe77+33z5PnrR8qKRSzC+INaLOKHo9O+FOmpT0c8PCGKozdf5Uhw4MEZkaCti+nR760aNNe76Bf/5hwm/VqpY1vXv1igmvAMNtyW0FcPAg25W4ubEnmzWHaBqGWWbObJ9hllFRSm3ZwoaNnp7MQWrXjhVY9khOjs2tWwyh1qsXE72pXp3f3eSU+VubO3eUWrKE36VZs5Tx4ZkJITk3yceu4ubXX/mheXnxLGRJbacgaMCYMVyoNm2yz/70ejYQ+/JLy15frRoXPVMwTCTfsiXp565dy+eakk9x9CifO2+e6XZkzsxcJ3P6nhw8yOTmmjUt66576RJzYXx8lFqwwPzXxyYggAnQgFKVK5uXQJ0UsYdZensr1bevbRtOXrvGvnN58vD9vP22UtOn23csg16v1JkzFHMVK9KOVKn4HZkzx/KcKmvz4AErB3v2jJlabigtnznTdvsVcWMEu4mbFSso+Rcs4JmxZ09eamnd7EAQTCA6WqkPPuDCcvGiffb5+edMqrWkyuXdd5kQawqbN/NEfPt20s9t2ZKVQ0mh17MPzttvmyZUQkMpBvLnN8/LsWcPK8PefdeyRNGVK+m1Kl6ci6ilREXx2s3XlwLtt9+s15juzh2KXB8f2jp0qHmNE80hNJQ9cOrX53ciQwZ+D48ds18Jd1QUk68HD2biOkDx2rYtlxFHCDI8esRS99692dXbIGZKllTqiy+YuPzoUQIvTGoMQ5cub3p3GjdO1A4RN0awm7ipWjWunzw6mi2mjXReFgRHIjCQJ6/ixZV68cL2+9uyxXQvSXyaNKEQMYUffuAiltTi9eQJr5oNfcaMsWoVbf/7b9Ns6NmTDt3jx017vlLcto8P24OYG0YKC6PnA2CIJTmlyocPs6u1mxvfhyVdlBPCMMwyVSrbD7M8dYrHw9Cdt04d9hiyV6fu0FCK7B49WFkHMJzZqxfL2K1RrZYcnj5lj6B+/divyaA7ihblZ/THHyYKzqTGMHTpQrfUgwcxNyOdHUXcGMEu4iY8nAHS+B9k585KJTEzSxAciUuXeHXevLnt8w1CQ5VKk8Yy/f/hh0Yv+OLQqRO9LEkxcyZ/xkk5W0ND2fSueXPT9j9/vnnhK6Uo/Ly8OErA3AGL16/TS5Q6NfMhLPVIPHoUk6dTqRLDcNbAXsMsnz/nZ2oI9+TMyVwqW+bvxN//smVKffwxvW8GsfDVVxSM9s7nic2LF2yAOHAg2xG4udG+ggX5mfv5WWH8RmLi5oMPTN6EiBsj2EXcGBp3xJt2roYOpUdHEJyILVtipkHbmpYtTRMe8enYkVffplC+PK8+k+Kdd9g9OSl+/JEL84ULST/3xAmKlB49kn6ugQ0b6M344APzr+g3bqR3omBB87xEsYmKoijKmJHJ1bNnWycEFX+Y5fTp1vec6PUM5XXqxLwdDw9eX27caJ9xDHfvUlA1asTkZECpKlXYk+ncOe26F798SQ/RV19R+Lq707a33lLqk0+YTXHjhpV3mpi48fXlLJRixRgTNBKnFXFjBBE3gmA+EyeaN67AUubP52KXYPzeCN27m/bTioykuJg+3fjzLl82bRjno0cMccWa/ZsoT59y3lClSqZ7X1at4qLYurV5850iIni6ASiKLJ3hdPRojKeje3fzP5eEOHiQYURbDrO8d48d/g05LEWL8jtsj4Tc8+e5b8MYCE9PTnefOdN6w0fN5dUrhjVHjGC1lUFo5czJXLXff1fqyhUbi62ExM0ff1BpnjnDx0qWpPpLRD2LuDGChKUEwXz0ei6wadMqdfas7fbz8CHFzaJF5r2ub19WaiTFhQs8x+7ebfx5o0ZRtCTVLfiLL3jhmVTeSXQ0F/TMmU2/Il62jFfUHTua52W4e5cjCDw9OUPIkgXr8WN6lwDm18S/TjOX+MMsS5e2/jDLiAiecps353Hz8eEpd98+2y7a0dFKHTmi1NdfMz8N4O/ko4/YddsWw0GTIjSUHqvRozlLKlUq2pUtG3/Hs2bxt2BXz1Fi08Fjc+2a0Z4QIm6MYNeE4tiXc9HRrDGUhGLBSQkOZjVQoUK2LY995x3m0JjD0KEsSU2K1at57jTmgdDrGcbp3t34ts6d4zWMKQNHv/2Wom3btqSfqxTHCri5sczanBDQ9u0cB5AnDz0k5hIVxZLjTJkYhpo5M3khKHsMs7x0ieGVHDliStJnz7ZtEnx4OD/LXr3o/TAIh+7d2T7B3BEa1rDnwAGWkNerxxAcQDHdqhU9lWfPajvE0yRxoxS/wIlMjhVxYwRbipsDB3hVptcr+rO9vHgJev48g/wZM/LSVBCclOvXecJ87z3b5SyMH88SYHPyS0aP5oKeFGPGMGnVGAcO8Dy8d6/x5zVtSqGXlJ1bt1KofPdd0vYpxbJqgAunqSIgKorHwM2NidWWhI/+/ZfCwNDYLzldK+IPs6xdm2LAWotrcDBPrbVqcfuZMrGy5/Rp62w/IQIDWUrfrh29eoaE20GDlNq/33ql8KYQGUlvkWG4dpo0MaXsOh2HyJ46pW2S8huYIm7u3OGXeOPGBB8WcWMEW4qb2Aq+WTOltjadoUKy51P61Kl56WKt8gJB0JCdO+n2HzrUNts/c4a/I1O9HErRIZo5c9LP++gj9jQxxmefKZUvn/GFYft22rhmjfFtXb/OhbdZM9MWmhkzuN1+/UwXAg8fKtWgAT+TcePMX9CePOF7dnNjl+FDh8x7fWwiIpiMaothlno9+6B+9hn7wADMZfnjD/MryEzlwQOKzSZNWG1mCNONHcvvqT174Rw7xuT1Jk0o/gH+bdKE9x87Zl+BlRB6PSNL/v7/v+Ovv5jpbxib3q0bVdetW8xqHjKEH2CDBlRo7u50CiRSwibixgi2FDcPH9IlOWoUr54yZYrpD1C4MBO3pk7lycPebktBsCZTpvB7vXy59bet17O5nanjFJTi78rHJ+nnFS9uvAtyaChzaL75JvHnREWx90etWsYXt5AQLoSFCpmWd/HzzzymQ4aYvmju28f+aDlyKLVrl2mvMRAdzUTSzJn5nmfMsNwbFxrKEJZhKnXLltbrVvzkCXsNlS0bU9EzahSFoy24fJlioUYNCj53d+YKTZvGrtL2IDqaXqipU5mm6evL927oczRhAj035iSZ24J791jN9803XPOyZKGd77///yf88EPiYxhCQvgjMhzkXLnYuXDx4kTdhiJujGDP8Qt6PTPQly3jCbV6dYpSQwZ9hQp0Pc+fz3io1qpbEExFr2d5rY8PS3qtTb9+9J6YusjPmcNzpLHnh4byHDp3buLPMeTkGOvK/PvvfI6xxVuv50WqtzcvVJNi/Hhu85tvTHvP0dFcNzw8lKpb1/wqoGPHYvJgunSxPFr+8iVFWc6cPLbt2yev87GB6GilduxQqk0bekxSpaLX7a+/rH+e1Ot5PL75honOBhHRsiVDX9ZqUpiUDf7+FJgffhgjEry8KKzGjmW41NpVZebw5Am9qePGsQIvd+4YrZIjBxO5x45l64gEw6IJhaXatuWJxERE3BhB68GZERHsdTF7Nk9+pUvHNExKl44nqqFDeZK9dUvjBDBBMEJICBNE8+WzTolwbHbs4G/C1ByKxYv5fGP5L6dO8TlHjiT+nBYtWImaGEFBzNlJ6nxsEEBJVX3p9cwDAkzPyXnyhGEugKW95nhbnj5lKxE3N+bDHDhg+mtjE3uYZapUTKS1RjO8W7eYfG3wAJUqxYova3+/IiIYXu3Th54gQ/Jtly5cf23dqVivp4CePZsCztClOFUqOjNGjWJFn63CbUkRFMScs59+ov4oVChGyGTMyHDg8OFsDXH7tonrVHxxEx3NRe+77xi/zJaNittIXo6IGyNoLW4SIiiIZXuTJvHqJG/euIpYp6Na3r5dm7JCQUiM27d5Tnr3Xeu6yMPDmVcxbpxpzzeMPzA2IdvPj89J7Kf/6BE9qsZ64IwYQW+MsblU//5Lb8Pnnxu3Wa9nh1yAXhhTOHqUYjJzZtMGfxqIjmZH5CxZmHQ6bZplIaiAANqcPr31hlmGhfHza9yYoittWoqlI0ese3EXHMwcqU6dYsYu5MunVP/+FBK2bOqn1yt19Sq9hh06xIxa8vBgdeDw4RT0wcG2syExQkP5vZoxg+KuVKmYC+40aSi2Bg5kBOLy5WR8JvHFzYMHMTsxZEBPnMidJ5LNL+LGCI4obhLi/n0mjH/zDWOshpgrwGS9jh15gjpyRDt1LwhKsVLE05OLhDX5+GPTe15u2sTfhrHwzNdf0yOQGDNm8H0k5iW4dYsLurF8nMePuWBWrWrci6TXc8EAeF5PCr2ev/dUqbgYmiMoTpzg1HSA3WctGUIZf5jlV18lv/DT35/HwBCCqV6dIXpLBoImxqNH3KZOF1MeXbYsPSMnT9rWM37zJkv6O3eOuWB1d2dF2tChrKJLzowvS4iMpDd07lymRFSsGNMDJ1Uq2ta7N4/ZmTNWFnzxxY2h2W38ibc6HUvSEkDEjRGcRdzEJzqa/Rz8/JiPUK1aTPZ+qlQMD/TuHVN57lAlgILLM2sWv4sLFlhvm0uWcJumLMY7d/K5164l/pxmzYyPU6halefVxOjYkZ7UxBakqCheiGTNalx8REczHAIwCTcpXryg0AOUGjDA9LyLZ8/YZNDNjQnQllQtxR5mmSlT8odZBgUxZGcQW9mycRq2JcNSE+P6dQrGOnUoJtzc6H2YPJneE1tx7x6b9n36KUvEAe67fHmKuD//NO5ZtDaGNWPpUgrTGjUoTg0iq0wZ9lGaOZPeRpsP64wvbsLDeTUR3z371VeJzmARcWMEZxU3CREezkS4mTPpTixZMsadmD49S16HDWPTrLt3tbZWcGX0ena0TZ3aeh0PnjxJOgHYwMGD/N4bWyTz5aP3JiEuXuTrV61K+PF//uHjxmwZOZL2GpsMHh3N4+TmZtr7OnWKzQkzZDB99EV0NEVm1qw8D0ydan7IMP4wy0mTLPcy6PX8fLp1i6n2bdKE78caCbJ6PY/TmDExfXW8vJjgOm9e8vr1GOPhQ/by6dUrpvQdoGjo14/nXVs2u4yNXk9BvWYNv+P168f19hcpQmfI5Mn0tFrTO2YyCSUUV6/+ZgJby5ZvenP+j4gbI7iSuEmIFy9YEjpxIjtT5skT8wXPnZvfmwkTeKVryw6eQsojLIwhk9y5LQt9JEStWqZNLDlxgt/xxIZDBgbycT+/hB//5hsuBgmFePV6pWrWZDgjsUqdP//k9idMSNzGqChehLi7MwHaGHo9PRxeXqyqNNXjcPIk1wuAniZzq6hOnGC1Tuxhlpa2rQgIYEJqiRK0p0ABXqRbY75SZCTTMr78Mib52NeX73n1atss3o8fUzz06cO8FMN5tXhxes1XrbKdkIpPQIBSmzdT0DVtGpOQDPCc37IlK/B27NA4T/PlSypPQza/IbfG4Npct45uQcNwqxkzqKgTyXQXcWMEVxc3CXH3LgXz8OHslWTorgnwxNO5s1K//mon16Tg0ty7x2TJGjWs812aNImu9KQW2PPn+X1OrPrn8GE+nlBZdnQ0F8iePRN+raE8PDGPzJUrXFhbtEg8HBwZyYtRDw/2LDNGcDBzYwB6BUzJqXv+nMm97u6swEyqu3J84g+znDfPMq9KZCQX3VatGHHw8uL73rkz+aHykBD2VOnaNSZPJ3duht527LB+mfSzZ9zfl1/GeIQMx6dHDybY3rtn3X0mxPPnPH4TJ1J4xi44yZKFPWVGjaLAtsdg0ITQ65ljtGULewR16cLPRO3Zk3ifGwPz59O15O3NLpIbNiS6HxE3RkiJ4iY+0dEcmrZ4Ma9CqlSJSSozNFPu25c5DxcvSv6OYB5HjvB79Nlnyd+WYdDl5s3Gn3fjBp+3Y0fCj//+O4VFQkJh3z6+dv/+Nx8LC2P+RGK5Oq9eceErUiTxfIrwcFZBenom3dH43Dl6BdKmZa5EUkRHM88ue3Ym+k6ebHoIyprDLK9do/fL0PukXDlehCc3LPP0Kc9TrVrFjBgoWZJVa//+a91zU2Agv2eDBzPR1hDiz5+fgmrxYuNVctbg1SsKzalTWVVVtGiMHkifnp/V0KH0Er0e9WNH9Hoeg7/+Yn+jbt24Xhg6JgP87lapYnoY1RxE3BhBxE3ChIUxr2DGDF41GqbbGty9DRvyhLJhg3ZXB4LzMH8+vzuJzL8zGb2ewiEpofTwIfeXyEga1b8/vZQJ0b07QyYJLZQ//URRdP58wrZ98gk9S4k1rgsLY5Jy6tS8sjbG0qVcwEuVSnh/8Tl9muEyQ8GJqV4EvZ7HydDEr3JlenbNFQohIRRD9epxOxkyMDxz/HjyFt1btxgOq1ePx95QSTVpkvHmiuYSHMymdF9/zWNh2FeePEwDmT/fdl2QlaLoPX6cvW4+/ZQi2WCDlxdDvP368SLT3kUiej3Dh9u2UTB/+intie31T5OGhSydO/Oz2byZgsuWdoq4MYKIG9N5/pxXduPHsyOlYXYWwHj8hx+yP8fu3Yn3DhFSLn360CNoaaM4AwMH0iNgbMF88YLfyxUrEn68fn1WHMUnJIQn7FGj3nzs0SM+ltgYCEOFWGIelpAQhgy8vXmlmxihoRRvhlLtpHqdvHhBsebuTi/G7t3Gn28gKoohMcMYA0uHWZ48yWNi6BVTty4XYEsb3+n17ND+3Xf0mBgqQN9/n+LYWhdTISEM73zzDcOmnp4xvcTateMMqWT1cTFCVBRL3xcuZBitatWYalcPD1ZU9exJD+PJk/Ybq6DXUxTv2EFvUY8eFJKxk5F9fPi5fPIJz/ebNtFTp4VHX8SNEUTcWI5Bza9dy6udevVi3JFubrzi7NqVJ/3jx7VtFS5oT0QEy3Fz5EheEunu3caThZXidw1IvCNw9uzsfBuflSv5uoS66/bpw5N8Qn1vjhzhAty3b8L7Cw5mfpuPDxfUxLhyhQubtzfzXIwtrHo9RUSOHHT9//STab+x+MMsGzdOOARnjOfPWZVpEB+5cjGH78oV87ZjICqKonfwYOawGMIubdtSgFmj2CEsjCHHb7+lADOIiaxZKXRnzqRHxNpixtCw748/ODG8Tp2458kSJSgUpk9nLpg95gzq9RSJf//NfkmffUavn0GgAvwOli/PpOwJE+jdu3rVscYCibgxgogb6xIVxTyBhQvpkq5UKeaKyOBa7d+fV7e2uioSHJeAACZAVq5sebPJiAiKjDFjEn+OXk9PxuzZCdsAJJzv0rw5e67E5/x5XlH/9FPC23vrLV7hJiQugoJiFjRjvWXWrKFnqEiRpMdMnDnDyjGAAsAUsZjcYZZ6PfNBO3bkwufhQQ/un39alpcTGsrQRY8eMdU9OXJwof3rr+QnoEdEcCjx99/HCEuAC3jLllzUz5yxvsfBULAxYgSnCGTOHCMYChRQqnVrJtnaw8Ot1zNEu2sXxVOvXvzexB7i7OXFnKj27Xms1q/nudmRRExiiLgxgogb2xMayivbadN4YoydFJcpE08AI0fyJJncLqeC43P8OBfHLl0sF7ft27Mk2hhp0ybc7dfg+YmfrxEQwAU7oUZ6zZoxkTj+ghsZyRBX9uwJC4wXLyh6MmTgVXlChIezAgegB8HYqejFCzbu8/DgFb8xL5CBly8pygzDLDt0YNjHVO7dYyja4FEpWpThCEvCQ8+fMy+ndesY70XRouzTdvhw8oRGZCTzBH/4gd6otGljPEDNmjHh9cQJ6y7aT55QiI0bx+o4wxgFgMdbp+PwyK1brT8PKz6PHlF8/vorLyzr1ImpIjMUh5Qty5DbuHGsur50ybZjJmyNiBsjiLjRhqdPORtr3DieAGL3ZciXjyf5H39kCasmDaYEm7J0KT/rX36x7PXLl/P1xjwWWbNyUY7PtGm8Wo1/UjeMM3jyJO79hqGdq1e/ua1hwygYEspzefaMVSIZMybuIbl1i56iVKl4ZZ2Y2NPrecxy5mTi5qRJSYegnj3jwhp7mKWpYaOICF7BN2/O9+fjQzG6f7/5gvTuXYam33svxotbpQo/m3PnLBe4UVEUKz//TPFiSG5Nm5bi5ocfKHastXjHHh7Zpk1M12GDN6hRI4bm1q3j99JWXunHj2nHzJnM16lbl52dDbakSsVKtzZt+PmvWcMqQ2cWMYkh4sYIIm4cA0NfhNWrWdpYt27MlZehNfinnzKh0J4JdoLtGDyYHohdu8x/7bNnfG1CYScDefPSIxifnj2ZSxCfypUZZolNVBSvdmvWfHOxWr+e388ff3xzW48fcx9ZsvD7mhCbN1N45M/PRTgxzp7lVThAj0dS5cfxh1n262d6yfLFi/z95cgRI0LmzDE/5+XCBeZpGCqwPD1ZYfnrr5aXT0dHM4z0yy/8nAyhFW9vhp2+/55hKGucG2IPj+zcOW639zRpmHw9cCBF9pUrthEyT59STM6ezVyuevXiXgR6ejKvsXVr5hGtXs3waUo6N4q4MYKIG8clKoon9vnzGYcvXz6mNNLbmxUOAwbwBHP1quTvOBuRkVzwsmRhyai5vPsuu7EmRtGiSg0Z8ub9CXV4NzT9i5+HM3cu748vPi5doqfgww/f/N49fEgxnj17wiXhkZG8wgfoGUms90tgIBNQPTyY/JtYzx4Dlg6zDA5mjpwhhydzZubF/fdf0q81EB3N0PPXX8e0jUiblv18li61rCuuXs/PZeZMenKzZo0Jr9StywV9377k5+bEHh752WcMdxo8TLGHRy5YwPORtXNRnj1jMvWcOTzuDRrErUQ1hCA/+kip0aOZ9O7vLwUaSom4MYqIG+fC0NRqyhTGjgsVijkJZM7MctHRo3lVbOsYt5B8njyhe79cOfNLh6dMYXgpsVLpt99+s2xbr6dH44cf4t4/fDhDC7EXyqAgejA6doz73OBgipdixd7Mj7l3jwtRrlwJ96a5f58Ls4cHQ0sJ5Zjo9RTsuXLRSzBxovEF/OpVeqMMwyy//TbpZnl6PQXbZ5/xeAAMq6xYYXqid3g4S8c//zwm1yRrVnpYN20yv+pHr2ci62+/8bdtWOA9Pek5GzmSXr7kVBNFR9M75edHIVG9esx0cIOHuFs3htGOHbNuh/YXL+hZ+v13itCGDePm6Hh4UBh++CFbEaxYQXEsXeITR8SNEUTcOD+PHzNh79tveSVvuMIzVCe0acO4/P79SfcMEezPmTNcxNu2Nc/7duUKP+P4s/cMVKvGhTY2t27xNVu2xNwXHc08r1694j73m2+48MWe6K3XMyE3TRpePcfm9m1WOr31VsKl5Lt20ZuTK1fiVVPnzsV0CP7oI+PTxM+do/Bydzd9mOXjx+xfUqYM95E3Ly8GTPWcBQbSc9C+fUyOS8GC9DDt32++V+P6dXpmO3WKmXvn4cHPbtgw5uVZ+puNHer+6iuGdWI3nStShO9jyhR6Tqx1bggMZHL0vHkMXb33XtyZfu7u9Cq2bMnv2PLl9JKJiDEfETdGEHHjeuj1PGmuWMGTbq1aMWWg7u68ou/Rg1dQp0+7ZqKds2GY1zRpknmvK1HiTQFj4N133xwmvGUL9xNbNBjG3Rw8GHPfrVsUNt98E/f106fzufHnQd24wUU+f/43u9hGR7MhnZsbr9YTGqYYFMQQmqcnF75t2xJ/z4ZhlobmmTNmGPdmREdTJLRpw5BOqlQM8/z1l2li5MEDelOaNInpDVOhApNV//vPPEF6+zbHFnTtGlOS7ubGfjmDB/PzsfRU/PAhPUajR9PW2Em2b73FkQ0TJrC3izWGRwYFMS9n/nyeZxo35n4M+3Rzo4D64AN6Bpcu5SwzS1sgCG8i4sYIIm5SBoa4+u+/U9i8/TaFjqHjZq1aPEGtWMHFSfJ37M+IEVwQjHXvjc/QofRaJBTeadKEV8exmTSJYZjYn2+3bgxvxr6vY0eGpGJ7Qg4epPgYMCDuNq9coQekcOE3PS2PHvHK3c2NnsX4YkKv53cud25+D8ePT/wKPvYwyyJFkh5meesW95kvH19TujS9FKaEa69cYaJ0jRq03d2dYvGXX+gNMZX791n63aNHTCk5wN/fl19yfIslQsPQLX3CBIqW2MMjs2blcRo9mmInuRPpX75ktdvChRSgTZrEHFODiClUiFWfw4Yx5HXypH2a8aV0RNwYQcRNyiU4mK70n3/mVW2BAnFPkE2bcnHYupXufMG2REWxpDdjRtNLlg8c4Od15Mibj7Vqxavp2HzyCfMsDLx6RbETuyHgP/9wm3Pnxtz34AEFSK1acatRLlzg/cWLs+Q5NgcPMhyRLVvCycDnz7NHDkBbExINej1fW7dujEBZvjxxb2NYGMNGBkGVLh2FxZEjSXc7PnaMnqrSpWOS9j/4gF2eTf3+BwRw/59/HnceXalSzH9as8b831JwMD/nKVPoiYs/PLJePYadVq9O3vDI4GAeg0WLKJqbNo3xLsUOczdrxv0tXsyeTZaOmRCSj4gbI4i4EWITEMBk5NGjmZwcu7tooUJMdJwyhQuXnNSsz/PnTNQtVSrp/BGlKIiyZKHXJz4dOlAUxKZCBSbfGvjjD362BjGl1zN5tWzZGC9LZCS3kzNn3MZ1/v707pQqFdc7oNdTMHt4UAzFFz0vX3Jx9PSkN2Pr1jdtN3eY5dmz9CgZmrbVqMFwibEeURERbALYt29MOCVzZvazWb/etByUp0/Z16Vfv5g8HoCfYa9e9EqZ05gzPJwCY9YshhvLlo3xsHp7U5j278+xExcuWNb079UrhvaWLGF1V/PmDCkaSr0BipqmTSlyFi2iTdJvy/EQcWMEETeCMQxzYZYv5+JRo0ZMdYVhwN1nnzFEcOaMc7Qsd3TOn+cVecuWpi1en3zCRTA+3btTHBiIjGR11bRpMfc1aRLXk2PI/fn775j7hgzhZx17/tLp0/TulSsXN8zz7Bm9HQAFTGwvj16v1KpV9OZ4e7OBZfz8i4SGWW7fnrA3IjCQYVaDAMqWjbYamyAeHMxZcJ06xcwRypePgmH37qTzz168YCfxgQP53TcIgkKFeLyXLn1TzCWGodXDggVsRlelSkxOj6cnhehnn/E9njplfv+WkBCGh/z8GC5q0YJiMraIyZuXFzGDB9OOf/4xTVQLjoGIGyOIuBHMJSKCJ805c3h1WaZMzAkzbVo2XBsyhAvZrVuSv2MJf/7J4zl2bNLPXbWKz41f8dO3b1zRc/Ein2doGvjwYdxGgGFhvIJv1izmNQaxE3uMw/HjLLmuVCluyfWxYwxbZMpE+2Nz4QKTiQGKn/hJx6YOs9TrGaLp2pUVW+7u9DCsXZt4/s2jR/Ti6HQxwrxsWZYbnzxp/PsZFETP0tCh9B4ZvCj58tHDs2iR8Yqu2HZfuRIzPLJ27ZgmnW5ubJLXuTMTto8cMS9fJTSUYnPZMnrwPviAOUkGWw0JxY0bc9/z5nEfcsp3fkTcGEHEjWANDK3Zf/yRlSixEw5z5ODC8t13rIJJqgeJQL77jsdv40bjzwsMjBlfEJuhQ7nIGVizhtszVCtNnUpPgeHzMISSDJ6PCxeYs9KmTYwAOHKEQzvfeYchNKX42MyZ3FaVKnFFVnAwvQapUtG7sXlzXBtDQti11/B9admSIik+Dx/yu2XIYylYkJ6fxMZPXL9OQVanDhd5NzeGyCZPpicyMV69Yo7P8OF8j4ammblyMcl63jylrl1LOn/nzp2Y4ZGNGsUd1FiwII/pjz+yUs3UU29YGKuz/viDPW9ataIYjC1icufm/gYMYM7U4cPWmSguOCYibowg4kawFQ8e8Ap+5EgmeBrCAIZql44dGSI5ckTKQxMiOpoLWPr0xkMtSnFBa9Qo7n2jRjEEZODbbxm6MVCxIrevFJNcfX0ZHlGK+RUlS/JmCFMcOECxU6tWzH1BQczDAph3Yqh00usppvLmpbdk7Ni4n7EpwywjIymGWrZkmMbLi8/btevNcJ1ez9DNmDEMlQF8fvPmFCQJlZ8rRZt27+axqlWLIgxgBVqbNvRqXbxoXMw8fswKt+++o4iP3V3XMDzyu+/4HFOSicPDeSxWrKBdH35IUWcQWgax1aABw2m//cYcOGuUdwvOhYgbI4i4EeyFoQPr0qU8Kb/zDhcgQ45BpUps875wIRu0Sf4OxUPp0qyQMXhKEmL6dC7MsX/GEyYwwdbAxx+zOkkpJgPHbgDYty8bvD16xM+pTRsKmQsX+Pju3QwD1asXk2h75gw9B+nTMzRm4NIlilmA4uLatZjHTBlmefUqPR65c3Mb5cuzl038xTsykt7CAQNiKv18fSmaV69OOHckPJwibexYlnYbvn+ZM1NEzJjBY5OYmAkKorflxx850yh2hWGmTBSYI0bwuN69a1wURUTwe75qFUXZxx9TTBpGHxi8nvXrUzjOmUPbxfMpGBBxYwQRN4KWhIczh2PWLOZRlCoVk79jKHP9+mvmVNhy0rAjc+UKvV5NmyYu+G7c4DGLPbl76lQKEgMlSlBUKsVjmjkzj/+FC/QKGAZgTpkSd1vbt9P78t57MRVyCxbwvrffpphRiqJnxAiGpwoWjJt3k9Qwy5AQit569WJESu/erOqJTUgIw3Rdu8ZURuXOTY/Tjh1v5t1ERNAzOGEChUeaNDHbb9GCx+j06YQTt0ND+drp05m0XaJE3Nyy2rWZw/LHH8Znu0VE0PO2ejW9Z61b83seW8Rkz06x1acPfwv79r05nV0Q4iPixggibgRHIzCQnoIffuDVdOyup7lyMWFy/HhW9BjzZrgS27YxfJNQybeBsmW5CBuYM4eLsV7PhdrDg5U3UVE8pr1783mGUuDQUC6qHh6snlGKYaHUqZlkHBpKcdO1Kz+LHj0oNvR6lkPny0dPyOjRMQmxd+5QUCU2zPLkSQoTQ8jy3XdZ3RO7zcDTp+yp0qpVjDgpWZJ5Mf/8E1eYREUxZ+fHH1kJli5djFBu2pShsOPH3xSJEREMa/3+e8zUdIP4MOQSffEFvYr+/gmLzMhIhrDWrmUYqm1bJtsbQl2G/lF163JbM2fS8yQz4ARLEXFjBBE3gjNw7x67uY4Ywaqb2DNyihfnoj5jBhc7V51RM2kS32/sEFBsRoygN8Ow8C5ezOeHhdE7ATDBdOfOmH///XfMNu/fZ45I3bpcqNet48LcqhU9IhcvcrH28WGVkFIMM77/PrfRtGlMsq6xYZbPnjGJuEKFGME6fHjc8NStW/SY1K8fk2vyzjs8BhcvxjwvOpqiZMoU5rb4+vK5adLQ0zRxIkcExC7xjo6mt2rJEnqQ3nkn7vDIsmVZBTh7NoVQfG9QVBS9VevWMam5fXt6sAxl3IYwV+3abOY3YwbFemJ5P4JgKSJujCDiRnBGDNONlyxhvkjVqjGLS6pUvNLu04cLvKXNzhwNvZ7Ju2nSsGomPkeO8P0fOMD/G0rEnz9nyAegV6xLFyZ0R0ZyUa5Rgwt4zZoM8Tx8yC67Hh70PkREMLk1XTqGZs6epWdl5Ege8/z5KTz1+jeHWf74I/NUoqO5wHfsSCHh4UEP3KZNtEOv53bHjWOis+FzfP99eqAMzQMNz5s+naLL0GTS25tCaNw4JtcaBIlez5DdqlWsHos/PLJoUSYpT5365vDIqCgKrg0b6Cns0IHJyoY8HUOeTa1abNg3bRqF44MHKTN8Ktgfc9ZvN6WUQgoiKCgIvr6+CAwMRIYMGbQ2RxAsJjwcOHMG+PffmNvFi3wsQwagShWgatWYW+7c2tprCSEhQM2aQGAgcOwYkCVLzGN6PZArF9C1KzBpErBpE9CiBXD/PjBtGvDHH8CFC0COHMDQoUCePECPHsDRo8CKFcCvvwL79gHXrwNdugAdOwKzZwNffQXMmgV06ADMmQPs3g18+SXw4AEfGz6cx3n8eGDdOiBvXt7fvTvw7BmwaBGwYAG3W6wY7+/cGciWjfvesAFYvx64dg1Inx5o2hRo2RJo0oSf26VLwJ49vO3dCzx+DKRKBbzzDlCvHlC/PlCtGuDtDTx8yONy7Bhw/Dj/PnnC45M3L78DVaoAlSsDlSoBmTLxuN24AZw/D5w7F3O7cAEIC+NrfX2B0qXfvOXMCbi52fUrIAivMWf9FnEjCC7EixfAiRMxYueff7goA1zcY4udypW5mDo6N29ygS5fHvjrL8DTM+ax7t2BI0e4UO/cCTRqRFHRrx99DR078vbff0DjxhQGOh3Qvj0wfTqQNi0FT7duwLBhvP/sWT7WoAFFzdatwPvv876AAIqabduAIkX4mnbtgB07gPnzaZ+3N9C6NbdbqRLF0YYNwJ9/Ao8eUWx98AEFTb16wL17MWJmzx5+Xp6efM/16vFWowbFrEHAGP7evcvjkC1bXCFTpQrvu3UrroAxiJjQUL4uQwaKllKl4oqY3LlFxAiOh4gbI4i4EVIa9+7F9e4cOwa8fMnFq0SJuILn7beB1Km1tvhN9uyhcPnyS2Dy5Jj7N2wAWrUCrlyh8KhViwt406ZA27YUNa9eAe++C/z8M707LVvSw1O7NvDFF0Dv3tx2t270DPn5Adu30xuUIwfwyy8UQRMm0NNTpgwwYgSP1eLFvD16REHRoweF0MGDtO2vv4DgYKBoUdrZsiW9H/v2xYiZO3cAd3cKIYOYqVABuHw5RsQcOwZcvcr3nCFDjICpXJk3IGFPzKtXfCx9+jcFTOnSFLwiYgRnQcSNEUTcCCkdvZ5hldiC57//gKgoCpsKFeIKniJFuPhqzfTpFDd+fkCnTrzv1SsKkokTgbp1KRD27qWYmT4dGDCAnpaxY4E+fShuUqdmmOirr4C+ffn/KVMoPlq3Br75hoJwyBB6i37+mceocmXeFxwMLFwIHDoEZM4MfPIJ0Lw5BdaGDfTUREVRfLRsCVSvTg/L3r0UMzduUFCUL08hU6sWw0UXLsQImfPn+Tn5+PDzMIiYt95iqO7CBQqY8+d5Cw7m8UibNmERkzeviBjB+RFxYwQRN4LwJmFhwOnTcQXPlSt8LGPGN/N3cua0v41K0buyciU9I5Uq8f6mTWn/zJlc2OfMAT7/HBg0iHk1LVtSVFSvToHRqxfw00/02Pz3H9/r11/z35s304vTpAkFzNmz9PB8/DH/vXIlhUTDhnzOq1cUTP/+y1DSu+8y9JUhA/Oh9uyJOY5ly1KAGcSiQcycOQNERPD1ZctSxBQpQm9LcDCFqEHIvHzJbaVJk7iIcQQhKgi2QMSNEUTcCIJpPHvGsEhswRMQwMfy5o0rdipV4mJsa8LCgDp1mEh7/DiQPTvFTN++/H+FCsDAgUwoLlOGXp09e4CPPgLWrmVOzR9/MFl4+3bmx7z/PrB0KXNUdDrm7ly5Qq9K6dIURP7+9Jo0bszk3j17mPibJg3FTMGC9KgcPkzRAjDkV6kSkDUr7T53Djh5ks9zcwOKF6eN2bNzm0FBMUImKIjb8PEBSpZ8U8Tkzy8iRkh5iLgxgogbQbAMpZgfElvsHD9O74WbGz0JsQVP2bJctK3N3bv0bhQvTiESEECxNWsWc2h0OnpZbt6kTaGh/Hf16hQfdesC+/cDFSsCT59ye/XqUVjcucPk3TRp+By9nq/LkIHC5MEDhqIqVOBzbtyg8AGAAgWYW5MmDfD8OT0yL17wsbx5gUKFWIWk17MC6tKlmMe9veOKGINXpkABwMPD+sdQEJwRETdGEHEjCNYjOpqeitiC58wZ3u/t/Wb+TuHC1sn9OHSIguSzzxh6qliR4mHtWnpDIiJYEh4cTGGSMSNw+zb3f/06hdHFixQjz59T5JQuzb8PHtBLkz07PTgvXzKx+K23+O8rVyj0smdnQq5S3NejR7Qtc2ZWG/n40Etz716MiPHyokcnviemYEERMYKQFCJujCDiRhBsS2gocOpUXMFz7Rofy5z5zfyd7Nkt289vvzG3Zt48Cpdp09gPJ316Cg43N4ZuoqLoXTLcHx5O4aXXM1yUMycFiJcXbXnwgK8xhIsePOBzfX15CwmJ6SXj48PQl5sbBYwhJyZ1agqo+CKmUKG4peyCIJiOiBsjiLgRBPvz9CmTZ2MLnseP+Vj+/HHFTsWKQLp0pm3388+Z+Pvbb0w2dnOjgEkIb28KGw8PPsfDgx6edOliqo0MSbxK8fmenjGPeXqyGikiIqZPTKpUCYuYwoVFxAiCtXE6cTNz5kz89NNPePjwIcqVK4cZM2agatWqiT5/9erVGDVqFG7evImiRYti0qRJaNq0qUn7EnEjCNqjFBvMxRY7J07QK+LuToEQW/CUKZOwWIiIYEKvwTP0+DFDYolhED/u7vTGGLvPw4MeHID7LlbsTRFTpIht8ooEQXgTpxI3K1euROfOnTFnzhxUq1YNv/zyC1avXo1Lly4hewL+6sOHD6NOnTqYOHEimjdvjuXLl2PSpEk4efIkypQpk+T+RNwIgmMSFcVy59iC5+zZmH4vFSvGFTwFC1KEPHzIBOPIyJi8F0txd/9fe/ceFFX5xgH8u9wWUBY0EESJQIksUTADYaqtJKWobGoS7SKUo5VdTNQSS02twGs1amkNoDNdGHVMHa+lk1PaZknkBbWEULSCUgsQDAWe3x/89uSJFXZh2YXD9zPDjPue55x9n3337Hnm+J5zGs8kxcQ0Tog2T+6NiOiYNzck6ko6VXETFxeHW265BcuXLwcANDQ0ICQkBC+88AJmzJjRJD4lJQXV1dXYsmWL0jZs2DBER0dj5cqVLb4fixuizqO6uun8nZKSxmXXXPNvoePn1/j8KPOZlpbodI2Xfg8Y0Hg1VHR0YyFz/fUsYog6KluO3079X+FLly4hPz8fGRkZSpuLiwsSExNhMpksrmMymZCenq5qGzlyJDZu3Ggxvra2FrW1tcrrSvMNJIiow+vWrfEOvrfe+m/bn3+q5+8sX944p+dq3N0br3SKj2+8wio2tnGejF7f/v0nIudwanFz9uxZ1NfXIzAwUNUeGBiI4+bHG/9HWVmZxfiysjKL8ZmZmZg7d659OkxEThcQ0HhXYvM0O5HGszl79zY+3RtovMvwtGmNN/zz9HReX4nIOTR/j8uMjAxUVFQof6dPn3Z2l4jIjnS6xkusx41rLHREGp/SPWIECxuirsqpZ278/f3h6uqKcvM93f+vvLwcQVd5eE1QUJBN8Xq9HnqefyYiIuoynHrmxsPDAzfffDN2796ttDU0NGD37t2Ij4+3uE58fLwqHgC++OKLq8YTERFR1+L020ylp6cjNTUVQ4cORWxsLN555x1UV1fjySefBACMGzcOffr0QWZmJgBg8uTJMBqNWLJkCZKTk5GXl4cDBw7ggw8+cGYaRERE1EE4vbhJSUnBn3/+idmzZ6OsrAzR0dHYsWOHMmm4tLQULlc8/jYhIQGffPIJXnvtNcycORMRERHYuHGjVfe4ISIiIu1z+n1uHI33uSEiIup8bDl+a/5qKSIiIupaWNwQERGRprC4ISIiIk1hcUNERESawuKGiIiINIXFDREREWkKixsiIiLSFBY3REREpCksboiIiEhTnP74BUcz35C5srLSyT0hIiIia5mP29Y8WKHLFTdVVVUAgJCQECf3hIiIiGxVVVUFX1/fZmO63LOlGhoa8Ntvv8HHxwc6nc4u26ysrERISAhOnz6tyedVaT0/gDlqBXPUBq3nqPX8gPbJUURQVVWF4OBg1QO1LelyZ25cXFzQt2/fdtm2wWDQ7BcV0H5+AHPUCuaoDVrPUev5AfbPsaUzNmacUExERESawuKGiIiINIXFjR3o9XrMmTMHer3e2V1pF1rPD2COWsEctUHrOWo9P8D5OXa5CcVERESkbTxzQ0RERJrC4oaIiIg0hcUNERERaQqLGyIiItIUFjdWePPNN5GQkABvb2/4+flZtY6IYPbs2ejduze8vLyQmJiIEydOqGLOnz+Pxx57DAaDAX5+fhg/fjwuXLjQDhm0zNa+nDx5EjqdzuLfunXrlDhLy/Py8hyRUhOt+bzvuOOOJv1/5plnVDGlpaVITk6Gt7c3evXqhenTp6Ourq49U7HI1vzOnz+PF154AZGRkfDy8sK1116LF198ERUVFao4Z47hihUrcN1118HT0xNxcXH47rvvmo1ft24dbrjhBnh6eiIqKgrbtm1TLbdmv3Q0W3L88MMPcdttt6FHjx7o0aMHEhMTm8SnpaU1Ga+kpKT2TqNZtuS4evXqJv339PRUxXT2cbT0u6LT6ZCcnKzEdKRx/Oqrr3D//fcjODgYOp0OGzdubHGdPXv2YMiQIdDr9ejfvz9Wr17dJMbW/dsmQi2aPXu2LF26VNLT08XX19eqdbKyssTX11c2btwoBw8elAceeEDCwsLk4sWLSkxSUpIMHjxYvv32W/n666+lf//+Mnbs2HbKonm29qWurk5+//131d/cuXOle/fuUlVVpcQBkNzcXFXclZ+BI7Xm8zYajTJhwgRV/ysqKpTldXV1MnDgQElMTJSCggLZtm2b+Pv7S0ZGRnun04St+R0+fFgeeugh2bx5sxQVFcnu3bslIiJCHn74YVWcs8YwLy9PPDw8JCcnRwoLC2XChAni5+cn5eXlFuP37dsnrq6usnDhQjl69Ki89tpr4u7uLocPH1ZirNkvHcnWHB999FFZsWKFFBQUyLFjxyQtLU18fX3lzJkzSkxqaqokJSWpxuv8+fOOSqkJW3PMzc0Vg8Gg6n9ZWZkqprOP47lz51T5HTlyRFxdXSU3N1eJ6UjjuG3bNnn11Vdlw4YNAkA+++yzZuN/+eUX8fb2lvT0dDl69KgsW7ZMXF1dZceOHUqMrZ+ZrVjc2CA3N9eq4qahoUGCgoJk0aJFStvff/8ter1ePv30UxEROXr0qACQ77//XonZvn276HQ6+fXXX+3e9+bYqy/R0dHy1FNPqdqs2REcobU5Go1GmTx58lWXb9u2TVxcXFQ/vu+//74YDAapra21S9+tYa8xXLt2rXh4eMjly5eVNmeNYWxsrDz33HPK6/r6egkODpbMzEyL8aNHj5bk5GRVW1xcnDz99NMiYt1+6Wi25vhfdXV14uPjI2vWrFHaUlNTZdSoUfbuaqvZmmNLv7NaHMe3335bfHx85MKFC0pbRxtHM2t+D15++WW56aabVG0pKSkycuRI5XVbP7OW8L+l2kFJSQnKysqQmJiotPn6+iIuLg4mkwkAYDKZ4Ofnh6FDhyoxiYmJcHFxwf79+x3aX3v0JT8/Hz/++CPGjx/fZNlzzz0Hf39/xMbGIicnx6rH1dtbW3L8+OOP4e/vj4EDByIjIwM1NTWq7UZFRSEwMFBpGzlyJCorK1FYWGj/RK7CXt+niooKGAwGuLmpHzvn6DG8dOkS8vPzVfuQi4sLEhMTlX3ov0wmkyoeaBwLc7w1+6UjtSbH/6qpqcHly5fRs2dPVfuePXvQq1cvREZG4tlnn8W5c+fs2ndrtTbHCxcuIDQ0FCEhIRg1apRqX9LiOGZnZ2PMmDHo1q2bqr2jjKOtWtoX7fGZtaTLPTjTEcrKygBAdcAzvzYvKysrQ69evVTL3dzc0LNnTyXGUezRl+zsbAwYMAAJCQmq9nnz5uGuu+6Ct7c3Pv/8c0yaNAkXLlzAiy++aLf+W6O1OT766KMIDQ1FcHAwDh06hFdeeQU//fQTNmzYoGzX0jiblzmKPcbw7NmzmD9/PiZOnKhqd8YYnj17FvX19RY/2+PHj1tc52pjceU+Z267WowjtSbH/3rllVcQHBysOkgkJSXhoYceQlhYGIqLizFz5kzcc889MJlMcHV1tWsOLWlNjpGRkcjJycGgQYNQUVGBxYsXIyEhAYWFhejbt6/mxvG7777DkSNHkJ2drWrvSONoq6vti5WVlbh48SL++uuvNn/3W9Jli5sZM2ZgwYIFzcYcO3YMN9xwg4N6ZH/W5thWFy9exCeffIJZs2Y1WXZlW0xMDKqrq7Fo0SK7HRjbO8crD/RRUVHo3bs3hg8fjuLiYvTr16/V27WWo8awsrISycnJuPHGG/H666+rlrX3GFLrZGVlIS8vD3v27FFNuB0zZozy76ioKAwaNAj9+vXDnj17MHz4cGd01Sbx8fGIj49XXickJGDAgAFYtWoV5s+f78SetY/s7GxERUUhNjZW1d7Zx9HZumxxM3XqVKSlpTUbEx4e3qptBwUFAQDKy8vRu3dvpb28vBzR0dFKzB9//KFar66uDufPn1fWbytrc2xrX9avX4+amhqMGzeuxdi4uDjMnz8ftbW1dnnmiKNyNIuLiwMAFBUVoV+/fggKCmoyw7+8vBwA7DKOjsivqqoKSUlJ8PHxwWeffQZ3d/dm4+09hpb4+/vD1dVV+SzNysvLr5pPUFBQs/HW7JeO1JoczRYvXoysrCzs2rULgwYNajY2PDwc/v7+KCoqcvhBsS05mrm7uyMmJgZFRUUAtDWO1dXVyMvLw7x581p8H2eOo62uti8aDAZ4eXnB1dW1zd+LFtll5k4XYeuE4sWLFyttFRUVFicUHzhwQInZuXOnUycUt7YvRqOxyRU2V/PGG29Ijx49Wt3X1rLX5713714BIAcPHhSRfycUXznDf9WqVWIwGOSff/6xXwItaG1+FRUVMmzYMDEajVJdXW3VezlqDGNjY+X5559XXtfX10ufPn2anVB83333qdri4+ObTChubr90NFtzFBFZsGCBGAwGMZlMVr3H6dOnRafTyaZNm9rc39ZoTY5Xqqurk8jISJkyZYqIaGccRRqPKXq9Xs6ePdviezh7HM1g5YTigQMHqtrGjh3bZEJxW74XLfbTLlvRuFOnTklBQYFyqXNBQYEUFBSoLnmOjIyUDRs2KK+zsrLEz89PNm3aJIcOHZJRo0ZZvBQ8JiZG9u/fL3v37pWIiAinXgreXF/OnDkjkZGRsn//ftV6J06cEJ1OJ9u3b2+yzc2bN8uHH34ohw8flhMnTsh7770n3t7eMnv27HbPxxJbcywqKpJ58+bJgQMHpKSkRDZt2iTh4eFy++23K+uYLwUfMWKE/Pjjj7Jjxw4JCAhw2qXgtuRXUVEhcXFxEhUVJUVFRapLTuvq6kTEuWOYl5cner1eVq9eLUePHpWJEyeKn5+fcmXaE088ITNmzFDi9+3bJ25ubrJ48WI5duyYzJkzx+Kl4C3tl45ka45ZWVni4eEh69evV42X+beoqqpKpk2bJiaTSUpKSmTXrl0yZMgQiYiIcGix3ZYc586dKzt37pTi4mLJz8+XMWPGiKenpxQWFioxnX0czW699VZJSUlp0t7RxrGqqko57gGQpUuXSkFBgZw6dUpERGbMmCFPPPGEEm++FHz69Oly7NgxWbFihcVLwZv7zNqKxY0VUlNTBUCTvy+//FKJwf/vBWLW0NAgs2bNksDAQNHr9TJ8+HD56aefVNs9d+6cjB07Vrp37y4Gg0GefPJJVcHkSC31paSkpEnOIiIZGRkSEhIi9fX1Tba5fft2iY6Olu7du0u3bt1k8ODBsnLlSouxjmBrjqWlpXL77bdLz549Ra/XS//+/WX69Omq+9yIiJw8eVLuuece8fLyEn9/f5k6darqUmpHsTW/L7/80uL3GoCUlJSIiPPHcNmyZXLttdeKh4eHxMbGyrfffqssMxqNkpqaqopfu3atXH/99eLh4SE33XSTbN26VbXcmv3S0WzJMTQ01OJ4zZkzR0REampqZMSIERIQECDu7u4SGhoqEyZMsNsBo7VsyfGll15SYgMDA+Xee++VH374QbW9zj6OIiLHjx8XAPL555832VZHG8er/VaYc0pNTRWj0dhknejoaPHw8JDw8HDV8dGsuc+srXQiTrgul4iIiKid8D43REREpCksboiIiEhTWNwQERGRprC4ISIiIk1hcUNERESawuKGiIiINIXFDREREWkKixsi6tLS0tLw4IMPOrsbRGRHLG6IqMNKS0uDTqeDTqeDu7s7wsLC8PLLL+Off/5xdteIqAPrsk8FJ6LOISkpCbm5ubh8+TLy8/ORmpoKnU6HBQsWOLtrRNRB8cwNEXVoer0eQUFBCAkJwYMPPojExER88cUXAICGhgZkZmYiLCwMXl5eGDx4MNavX6+sW19fj/HjxyvLIyMj8e677zorFSJyEJ65IaJO48iRI/jmm28QGhoKAMjMzMRHH32ElStXIiIiAl999RUef/xxBAQEwGg0oqGhAX379sW6detwzTXX4JtvvsHEiRPRu3dvjB492snZEFF7YXFDRB3ali1b0L17d9TV1aG2thYuLi5Yvnw5amtr8dZbb2HXrl2Ij48HAISHh2Pv3r1YtWoVjEYj3N3dMXfuXGVbYWFhMJlMWLt2LYsbIg1jcUNEHdqdd96J999/H9XV1Xj77bfh5uaGhx9+GIWFhaipqcHdd9+tir906RJiYmKU1ytWrEBOTg5KS0tx8eJFXLp0CdHR0Q7OgogcicUNEXVo3bp1Q//+/QEAOTk5GDx4MLKzszFw4EAAwNatW9GnTx/VOnq9HgCQl5eHadOmYcmSJYiPj4ePjw8WLVqE/fv3OzYJInIoFjdE1Gm4uLhg5syZSE9Px88//wy9Xo/S0lIYjUaL8fv27UNCQgImTZqktBUXFzuqu0TkJLxaiog6lUceeQSurq5YtWoVpk2bhilTpmDNmjUoLi7GDz/8gGXLlmHNmjUAgIiICBw4cAA7d+7Ezz//jFmzZuH77793cgZE1N545oaIOhU3Nzc8//zzWLhwIUpKShAQEIDMzEz88ssv8PPzw5AhQzBz5kwAwNNPP42CggKkpKRAp9Nh7NixmDRpErZv3+7kLIioPelERJzdCSIiIiJ74X9LERERkaawuCEiIiJNYXFDREREmsLihoiIiDSFxQ0RERFpCosbIiIi0hQWN0RERKQpLG6IiIhIU1jcEBERkaawuCEiIiJNYXFDREREmsLihoiIiDTlf/gKjzztjEJzAAAAAElFTkSuQmCC",
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"freqs_for_each_token = torch.outer(torch.arange(17), freqs)\n",
"freqs_cis = torch.polar(torch.ones_like(freqs_for_each_token), freqs_for_each_token)\n",
"freqs_cis.shape\n",
"\n",
"# viewing tjhe third row of freqs_cis\n",
"value = freqs_cis[3]\n",
"plt.figure()\n",
"for i, element in enumerate(value[:17]):\n",
" plt.plot([0, element.real], [0, element.imag], color='blue', linewidth=1, label=f\"Index: {i}\")\n",
" plt.annotate(f\"{i}\", xy=(element.real, element.imag), color='red')\n",
"plt.xlabel('Real')\n",
"plt.ylabel('Imaginary')\n",
"plt.title('Plot of one row of freqs_cis')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### now that we have a complex number (the angle change vector) for every token's query element\n",
"we can convert our queries (the one we split into pairs) as complex numbers and then dot product to rotate the query based on the position\n",
"<br>\n",
"honeslty this is beautiful to think about :)"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([17, 64])"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"q_per_token_as_complex_numbers = torch.view_as_complex(q_per_token_split_into_pairs)\n",
"q_per_token_as_complex_numbers.shape"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([17, 64])"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"q_per_token_as_complex_numbers_rotated = q_per_token_as_complex_numbers * freqs_cis\n",
"q_per_token_as_complex_numbers_rotated.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### after rotated vector is obtained\n",
"we can get back our the queries as pairs by viewing the complex numbers as real numbers again"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([17, 64, 2])"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"q_per_token_split_into_pairs_rotated = torch.view_as_real(q_per_token_as_complex_numbers_rotated)\n",
"q_per_token_split_into_pairs_rotated.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"the rotated pairs are now merged, we now have a new query vector (rotated query vector) that is of the shape [17x128] where 17 is the number of tokens and the 128 is the dim of the query vector"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([17, 128])"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"q_per_token_rotated = q_per_token_split_into_pairs_rotated.view(q_per_token.shape)\n",
"q_per_token_rotated.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# keys (almost the same as queries)\n",
"<div>\n",
" <img src=\"images/keys.png\" width=\"600px\"/>\n",
"</div>\n",
"im lazy as fuck, so im not going to go through the math for keys, the only things you need to keep in mind are:\n",
"<br>\n",
"> keys generate key vectors also of dimention 128\n",
"<br>\n",
"> keys have only 1/4th the number of the weights as queries, this is because the weights for keys are shared across 4 heads at a time, to reduce the number of computations need\n",
"<br>\n",
"> keys are also rotated to add positional info, just like queries because of the same reasons "
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([8, 128, 4096])"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"k_layer0 = model[\"layers.0.attention.wk.weight\"]\n",
"k_layer0 = k_layer0.view(n_kv_heads, k_layer0.shape[0] // n_kv_heads, dim)\n",
"k_layer0.shape"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([128, 4096])"
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"k_layer0_head0 = k_layer0[0]\n",
"k_layer0_head0.shape"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([17, 128])"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"k_per_token = torch.matmul(token_embeddings, k_layer0_head0.T)\n",
"k_per_token.shape"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([17, 64, 2])"
]
},
"execution_count": 24,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"k_per_token_split_into_pairs = k_per_token.float().view(k_per_token.shape[0], -1, 2)\n",
"k_per_token_split_into_pairs.shape"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([17, 64])"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"k_per_token_as_complex_numbers = torch.view_as_complex(k_per_token_split_into_pairs)\n",
"k_per_token_as_complex_numbers.shape"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([17, 64, 2])"
]
},
"execution_count": 26,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"k_per_token_split_into_pairs_rotated = torch.view_as_real(k_per_token_as_complex_numbers * freqs_cis)\n",
"k_per_token_split_into_pairs_rotated.shape"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([17, 128])"
]
},
"execution_count": 27,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"k_per_token_rotated = k_per_token_split_into_pairs_rotated.view(k_per_token.shape)\n",
"k_per_token_rotated.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## at this stage now have both the rotated values of queries and keys, for each token. \n",
"<div>\n",
" <img src=\"images/keys0.png\" width=\"600px\"/>\n",
"</div>\n",
"each of the queries and keys are now of shape [17x128]. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## in the next step we will multiply the queries and key matrices\n",
"doing this will give us a score mapping each token with one another\n",
"<br>\n",
"this score describes how well each token's query relates to the each tokens's key. \n",
"THIS IS SELF ATTENTION :)\n",
"<br>\n",
"the shape of the attention score matrix (qk_per_token) is [17x17] where 17 is the number of tokens in the prompt\n",
"\n",
"<div>\n",
" <img src=\"images/qkmatmul.png\" width=\"600px\"/>\n",
"</div>"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([17, 17])"
]
},
"execution_count": 28,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"qk_per_token = torch.matmul(q_per_token_rotated, k_per_token_rotated.T)/(head_dim)**0.5\n",
"qk_per_token.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# we now have to mask query key scores\n",
"during the training process of llama3, the future token qk scores are masked.\n",
"<br>\n",
"why? because during training we only learn to predict tokens using past tokens.\n",
"<br>\n",
"as a result, during inference we set the future tokens to zero.\n",
"<div>\n",
" <img src=\"images/mask.png\" width=\"600px\"/>\n",
"</div>"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAlcAAAGdCAYAAAA/oFbLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy80BEi2AAAACXBIWXMAAA9hAAAPYQGoP6dpAABrMElEQVR4nO3deVhUZf8/8PdhmQWGAVGURVZXMAEVNbUEezRsMQ23UlNMzRYfcleeSnGlx6XUzCV7fqBlaauVpqWmplQu5ZKJqKhBipobiMiAzP37gy8nR1DEOWfY3q/rOtfFzJzzOfc5c2bmw70dSQghQERERESKsKvsAhARERHVJEyuiIiIiBTE5IqIiIhIQUyuiIiIiBTE5IqIiIhIQUyuiIiIiBTE5IqIiIhIQQ6VXQCq/sxmM86ePQsXFxdIklTZxSEiogoSQuDatWvw9vaGnZ069S75+fkoKChQJJZGo4FOp1MklhqYXJHVzp49C19f38ouBhERWSkzMxMNGzZUPG5+fj4C/Q04d6FIkXienp44depUlU2wmFyR1VxcXAAACT90gs6g7CX17fkWisYDgCs39IrHBACtw01V4tpL6txEwU6luJIKcR3szIrHBIBL151Vievjmq1K3GsmreIxcws0iscEgPrOuarEzStUp7xqxK2jy1M8plpu5hVgV///yd/nSisoKMC5C0X489cAGF2sqxnLuWaGf5vTKCgoYHJFNVdJU6DO4KB4cuWQq/yPib2kfEwAcHC0VyUukyv1kit7oc614OisTgLg4KDC58FBpbI6F6oTV6Xkyl6FJNNBr0wtjS2p3bXD4CLB4GLdPsyo+t1PmFwRERGRTRQJM4qs/P+rSKjzz5aSOFqQiIiISEGsuSIiIiKbMEPADOuqrqzd3haYXBEREZFNmGGGtY161kdQn6rNgrGxsUhISJAfJyQkIDw8XM1dAgC2b98OSZJw9epV1fdV4ujRo3jwwQeh0+lscowVcfr0ac4/RUREla5ICEWWqq5G9rnq2LEjsrKy4OrqarN9Tp06Fc7OzkhLS8PWrVvvum5JsnPgwAFFy6BWXCIiIrp3iidXV65cQW6uOvOb3CuNRgNPT0+b1takp6fjoYcegr+/P+rWrWuz/d6vjIyMyi4CERHVMiV9rqxdqjpFkqubN29iw4YN6Nu3L7y8vJCenn7X9ZcvXw5fX184OTmhX79+yM62nHDv/fffR3BwMHQ6HZo3b44lS5ZYvP7TTz8hPDwcOp0OERERWLdunUWNze3NgsnJyXBzc8N3332H4OBgGAwGdO/eHVlZWfd0fGazGdOnT0fDhg2h1WoRHh6OTZs2ya9LkoRff/0V06dPhyRJFk2hZQkMDAQAtGrVCpIkISoq6p6O/fnnn0doaChMJhOA4knZWrVqhcGDB5cb93ZDhgzBAw88gLlz597zeShhMpmQk5NjsRAREZXHDIEiK5can1z9/vvvGDduHBo2bIjBgwfDw8MD27ZtQ1hY2B23OXHiBD755BN888032LRpE/bv34+XX35Zfn316tWYMmUKZs2ahdTUVMyePRtvvPEGVq5cCQDIyclBjx490LJlS/z222+YMWMGJk2aVG5Z8/LyMG/ePHzwwQf48ccfkZGRgfHjx9/TcS5cuBDz58/HvHnzcOjQIURHR+Opp57C8ePHAQBZWVlo0aIFxo0bh6ysrHLj7tmzBwCwZcsWZGVl4YsvvrinY1+0aBGuX7+OyZMnAwBee+01XL16FYsXL75r3LJ88skneOGFF7B27Vr4+vri8ccfx9q1a5Gfn1/u+UhMTISrq6u88NY3RERE/6jwaMFLly7hww8/xMqVK/HHH3/g8ccfx5IlS/Dkk09Coyl/htv8/HysWrUKPj4+AIB33nkHTzzxBObPnw9PT09MnToV8+fPR0xMDIDi2pgjR45g+fLlGDJkCD766CNIkoQVK1ZAp9MhJCQEZ86cwYgRI+6638LCQixbtgyNGjUCAIwaNQrTp0+/p2OeN28eJk2ahGeeeQYA8N///hfbtm3DggUL8O6778LT0xMODg4wGAzw9PQsN56HhwcAoG7duhbrl3fsBoMBH374ISIjI+Hi4oIFCxZg27ZtMBqNd417pzLExcUhLi4OqampWLlyJcaPH48XX3wR/fv3R2xsLB588MEyt42Pj8fYsWPlxzk5OUywiIioXJyK4Q7eeecdTJs2DQ8//DBOnDhR4R9VPz8/ObECgA4dOsBsNiMtLQ0uLi5IT0/HsGHDLJKlmzdvyp3T09LSEBoaanE/oXbt2pW7XycnJzmxAgAvLy9cuHCh3O1ycnJw9uxZdOrUyeL5Tp064eDBg+Vuf6+uX79e7rEDxedr/Pjxco3dQw89ZPW+g4OD8eabb2L27NmYO3cu3njjDaxZs+aOoy21Wi20WnVuG0JERDWXEqP9qsNowQonVy+88AIcHBywatUqtGjRAr1798Zzzz2HqKgo2NlZ14WrpCP8ihUr0L59e4vX7O2tu2+bo6OjxWNJkiCq0Bt0r8duNpuRkpICe3t7nDhxQpF9Z2ZmYvXq1fjggw9w6tQp9O3bF0OHDlUkNhERUW1T4WzI29sbr7/+Oo4dO4ZNmzZBo9EgJiYG/v7+mDx5Mv7444+7bp+RkYGzZ8/Kj3/55RfY2dmhWbNmaNCgAby9vXHy5Ek0btzYYinprN2sWTP8/vvvcqduANi7d29FD+OeGY1GeHt7IyUlxeL5lJQUhISE3FfMkubToqJ/bup5L8cOAHPnzsXRo0exY8cObNq0CUlJSXeNeyfXrl1DcnIyHnnkEQQEBGDDhg0YO3Yszp07h9WrV6Nr1673dWxERER3YlZoqeqsmqG9Y8eO6NixIxYuXIh169YhOTkZ8+bNw/79+9GyZcsyt9HpdBgyZAjmzZuHnJwcxMXFoV+/fnIfoWnTpiEuLg6urq7o3r07TCYT9u3bhytXrmDs2LEYMGAAXnvtNbzwwguYPHkyMjIyMG/ePADq3c17woQJmDp1Kho1aoTw8HAkJSXhwIEDWL169X3Fq1+/PvR6PTZt2oSGDRtCp9PB1dW13GPfv38/pkyZgs8++wydOnXCW2+9hVdffRWRkZEICgq6Y9yy9OrVCydPnsRzzz2HFStWWDSZEhERqaFkxJ+1Mao6RaZi0Ol0eOaZZ7Bp0yZkZGTA39//jus2btwYMTExePzxx/Hoo48iNDTUYrqB4cOH4/3330dSUhJatmyJyMhIJCcny7U3RqMR33zzDQ4cOIDw8HC89tprmDJlilwONcTFxWHs2LEYN24cWrZsiU2bNuHrr79GkyZN7iueg4MDFi1ahOXLl8Pb2xs9e/YEcPdjz8/Px6BBgxAbG4sePXoAKG6i7dKlC5577jkUFRXdMW5ZlixZgpMnT2L69OlMrIiIiBQkCRU7HsXGxiIgIKDceZ+stXr1agwdOhTZ2dnQ6/Wq7qs6On36NAIDA1XrY5aTkwNXV1e8uScSOoOyt6v8+lyoovEA4HKeOteIzvGmKnHtJXXeNzuV4koqxHWwU6ch4GKusypxfd2uqhI3x6T8P5DXTOWP8r4fDQzqTCadV6hOea8XKB/XXZ+neEy13LxuwvYeS5GdnS2PQFdSye/EoSP14eJiXb3OtWtmhIZcUK2sSqiWN25etWoVgoKC4OPjg4MHD2LSpEno168fEysiIqIqTIk+U9Whz1W1vLfguXPnMGjQIAQHB2PMmDHo27cv3nvvvfuOZzAY7rjs3LmzwvFmz559x3iPPfbYfZeTiIioOjNDQpGVixm2u7Xd/VK15qpXr15wc3NTPO7EiRMxceJExeLd7UbHt87Jda9efPFF9OvXr8zXKqN2zc3NDVOnTrX5fomIiGoj1ZOr6qBx48aKxnN3d4e7u7uiMa3h5uamer83ABjplgmjlW3pt0u5qnxn+5NZ9RSPCQDiijp9QYRT+VNr3A+NoUCVuM56U/krVZAQ6vynei1XnX92rjur09cmr9Cx/JUqKL9A+ZiAen2j/rropkrconzlfw4vORgUjwkAagyWM98o/9ZniuxHFC/WxqjqqmWfKyIiIqp+Spr2rI1R1VXLPldERERE9yIxMRFt27aFi4sL6tevj169eiEtLU3VfTK5IiIiIpuwtjP7/dR87dixA6+88gp++eUXbN68GYWFhXj00Udx/fp1lY6SzYJERERkI2YhwWxlH8qKbr9p0yaLx8nJyahfvz5+/fVXdO7c2aqy3AmTKyIiIqp2cnJyLB5rtVpotdpyt8vOzgYAVQeesVmwBtu+fTskScLVq1cruyhERESKNgv6+vrC1dVVXhITE8vdv9lsxujRo9GpUyc88MADqh0na65qkKioKISHh2PBggWVXRQiIqJSimCHIivrdUomp8nMzLS4/c291Fq98sorOHz4MHbt2mVVGcrD5IqIiIhsQijQ56pk3juj0VihewuOGjUK69evx48//oiGDRtaVYbysFmwhoiNjcWOHTuwcOFCSJIESZJw+vRpAMCvv/6KiIgIODk5oWPHjqWGoH711Vdo3bo1dDodgoKCMG3aNNy8qc5NiImIiGxJCIFRo0bhyy+/xA8//IDAwEDV98nkqoZYuHAhOnTogBEjRiArKwtZWVnw9fUFALz22muYP38+9u3bBwcHBzz//PPydjt37sTgwYPx6quv4siRI1i+fDmSk5Mxa9asO+7LZDIhJyfHYiEiIipPZUzF8Morr+DDDz/ERx99BBcXF5w7dw7nzp3DjRs3VDpKJlc1hqurKzQaDZycnODp6QlPT0/Y29sDAGbNmoXIyEiEhIRg8uTJ+Omnn5CfX3yrg2nTpmHy5MkYMmQIgoKC0K1bN8yYMQPLly+/474SExMtOhGWJHFERER3UyTsFFkqYunSpcjOzkZUVBS8vLzkZe3atSodJftc1QqhoaHy315eXgCACxcuwM/PDwcPHkRKSopFTVVRURHy8/ORl5cHJyenUvHi4+MxduxY+XFOTg4TLCIiqpKEsP3NCJlc1QKOjv/cmFWSiqtTzWYzACA3NxfTpk1DTExMqe10Ol2Z8e51LhEiIqJbmSHBbGWjmVmNO1crjMlVDaLRaFBUVFT+irdo3bo10tLS0LhxY5VKRUREVKy23LiZyVUNEhAQgN27d+P06dMwGAxy7dTdTJkyBU8++ST8/PzQp08f2NnZ4eDBgzh8+DBmzpxpg1ITERHVLOzQXoOMHz8e9vb2CAkJgYeHBzIyMsrdJjo6GuvXr8f333+Ptm3b4sEHH8Tbb78Nf39/G5SYiIhqk8ro0F4ZWHNVgzRt2hQ///yzxXOxsbEWj8PDw0t17ouOjkZ0dLTaxSMiolquuM+VlTdurgbNglU//SMiIiKqRlhzRURERDZhVuDeghwtSERERPR/lOgzVVQJ81ZVFJMrqtKaGc4rHnOvUKezfp3f1Wlld08tUCWuY+oZVeLmtQ1SPGZ2I8fyV7oPOoMqYXHqZj1V4tqfU35+OZ8f1bmP6IXWrqrEbZBW/ijo+1FgUL4fzw0PdfoGFbgqn1yY822TsJhhVyvmuWKfKyIiIiIFseaKiIiIbKJISCgSVk4iauX2tsDkioiIiGyiSIEO7UVsFiQiIiKqXVhzRURERDZhFnYwWzla0MzRgkRERETF2CxIRERERBXGmisiIiKyCTOsH+2nzkxnymJyRXdVWFgIR0d1JnAkIqLaRZlJRKt+o1vVL2E1dunSJTz77LPw8fGBk5MTWrZsiY8//thinaioKMTFxWHixIlwd3eHp6cnEhIS5NeFEEhISICfnx+0Wi28vb0RFxcHAFi8eDEeeOABed1169ZBkiQsW7ZMfq5r1654/fXX5cdfffUVWrduDZ1Oh6CgIEybNg03b/4zQ7MkSVi6dCmeeuopODs7Y9asWUqfFiIiohqNyZWK8vPz0aZNG2zYsAGHDx/GCy+8gOeeew579uyxWG/lypVwdnbG7t27MWfOHEyfPh2bN28GAHz++ed4++23sXz5chw/fhzr1q1Dy5YtAQCRkZE4cuQI/v77bwDAjh07UK9ePWzfvh1Aca3Tzz//jKioKADAzp07MXjwYLz66qs4cuQIli9fjuTk5FIJVEJCAp5++mn8/vvveP7550sdl8lkQk5OjsVCRERUnpJ7C1q7VHVVv4TVmI+PD8aPH4/w8HAEBQXh3//+N7p3745PPvnEYr3Q0FBMnToVTZo0weDBgxEREYGtW7cCADIyMuDp6YmuXbvCz88P7dq1w4gRIwAADzzwANzd3bFjxw4AwPbt2zFu3Dj58Z49e1BYWIiOHTsCAKZNm4bJkydjyJAhCAoKQrdu3TBjxgwsX77cojwDBgzA0KFDERQUBD8/v1LHlZiYCFdXV3nx9fVV9sQREVGNZIakyFLVMblSUVFREWbMmIGWLVvC3d0dBoMB3333HTIyMizWCw0NtXjs5eWFCxcuAAD69u2LGzduICgoCCNGjMCXX34pN+NJkoTOnTtj+/btuHr1Ko4cOYKXX34ZJpMJR48exY4dO9C2bVs4OTkBAA4ePIjp06fDYDDIy4gRI5CVlYW8vDx5/xEREXc9rvj4eGRnZ8tLZmam1eeKiIhqvtpSc8UO7SqaO3cuFi5ciAULFqBly5ZwdnbG6NGjUVBQYLHe7R3GJUmC2Vw8HsLX1xdpaWnYsmULNm/ejJdffhlz587Fjh074OjoiKioKLz33nvYuXMnWrVqBaPRKCdcO3bsQGRkpBw3NzcX06ZNQ0xMTKmy6nQ6+W9nZ+e7HpdWq4VWq63w+SAiIqoNmFypKCUlBT179sSgQYMAAGazGceOHUNISEiF4uj1evTo0QM9evTAK6+8gubNm+P3339H69atERkZidGjR+PTTz+V+1ZFRUVhy5YtSElJwbhx4+Q4rVu3RlpaGho3bqzYMRIREd0rZSYRZc1VrdakSRN89tln+Omnn1CnTh289dZbOH/+fIWSq+TkZBQVFaF9+/ZwcnLChx9+CL1eD39/fwDFTYp16tTBRx99hPXr1wMoTq7Gjx8PSZLQqVMnOdaUKVPw5JNPws/PD3369IGdnR0OHjyIw4cPY+bMmcoePBER0W3MQoLZ2nmurNzeFqp++leNvf7662jdujWio6MRFRUFT09P9OrVq0Ix3NzcsGLFCnTq1AmhoaHYsmULvvnmG9StWxdAcRPiww8/DEmS8NBDDwEoTriMRiMiIiIsmviio6Oxfv16fP/992jbti0efPBBvP3223KiRkRERNZjzZWK3N3dsW7duruuUzJtwq1u3aZXr17lJmS378POzg6XL18uc93o6GhER0ffMZaoBjfEJCKi6smsQLNgdZhElMkVERER2YRZ2MFs5Wg/a7e3hapfQiIiIqJqhDVXREREZBNFkFBk5SSg1m5vC0yuSDF/F11HfpGylaH1HZW/tU6bwIzyV7oPZz1cVYl7+mADVeJ6eDdRJa4a1BocZFdQ/jr3w0FTpErcm14mxWOefUidOetu+uSrEveypCt/pfvgmKt8zII66vRhLdIpH9dsZ5v+tmwWJCIiIqIKY80VERER2UQRrG/WU6deWFlMroiIiMgmakuzIJMrIiIisgklbrxcHW7cXPVLSERERFSNsOaKiIiIbEJAgtnKPleCUzEQERERFWOzINU4ycnJcHNzq+xiEBER1WisuSIiIiKbMAsJZitnBbZ2e1tgzVUtsX37dgwdOhTZ2dmQJAmSJCEhIQEAcOXKFQwePBh16tSBk5MTHnvsMRw/fvyOsUwmE3JyciwWIiKi8hTBTpGlqqv6JSRFdOzYEQsWLIDRaERWVhaysrIwfvx4AEBsbCz27duHr7/+Gj///DOEEHj88cdRWFhYZqzExES4urrKi6+vry0PhYiIqEpjclVLaDQauLq6QpIkeHp6wtPTEwaDAcePH8fXX3+N999/Hw8//DDCwsKwevVqnDlzBuvWrSszVnx8PLKzs+UlMzPTtgdDRETVUkmzoLVLVcc+V7VcamoqHBwc0L59e/m5unXrolmzZkhNTS1zG61WC61WnZu9EhFRzWWGHcxW1utYu70tVP0SEhEREVUjTK5qEY1Gg6Iiy1teBgcH4+bNm9i9e7f83KVLl5CWloaQkBBbF5GIiGqwIiEpslR1TK5qkYCAAOTm5mLr1q24ePEi8vLy0KRJE/Ts2RMjRozArl27cPDgQQwaNAg+Pj7o2bNnZReZiIhqkNrS54rJVS3SsWNHvPjii+jfvz88PDwwZ84cAEBSUhLatGmDJ598Eh06dIAQAt9++y0cHR0rucRERFSTCGEHs5WLqAYztLNDey2zdOlSLF261OK5OnXqYNWqVZVUIiIiopqFyRURERHZRBEkFFl542Vrt7cFJldERERkE2Zh/e1rzEKhwqio6jdcEhEREVUjrLkixZwu1MG5UNl83axCx8XGzn8rHhMA6mmvqxI3o/0NVeKmOvqrElf3t/LvmX2B4iEBAJocdf4FvpatziS79i5l35LKGoUeyscEAEfNTVXimuoVlb/SfbArtFc8pqROUQFzNYlZ1m7+r1O6tTGqOiZXREREZBNmSDBb2WfK2u1toeqnf0RERETVCGuuiIiIyCaUmGG9OszQzuSKiIiIbKK29Lmq+iUkIiIiqkaYXNVwAQEBWLBgQWUXg4iIqLhDu7X3FrzPDu3vvvsuAgICoNPp0L59e+zZs0fho/sHk6saIjk5GW5ubpVdDCIiojsS/zda0JpF3EdytXbtWowdOxZTp07Fb7/9hrCwMERHR+PChQsqHCWTKyIiIrIRq2ut/m8BgJycHIvFZDLdcb9vvfUWRowYgaFDhyIkJATLli2Dk5MT/t//+3+qHCeTqxpg+/btGDp0KLKzsyFJEiRJQkJCgvx6Xl4enn/+ebi4uMDPzw/vvfeexfaZmZno168f3Nzc4O7ujp49e+L06dO2PQgiIqIK8PX1haurq7wkJiaWuV5BQQF+/fVXdO3aVX7Ozs4OXbt2xc8//6xK2Zhc1QAdO3bEggULYDQakZWVhaysLIwfP15+ff78+YiIiMD+/fvx8ssv46WXXkJaWhoAoLCwENHR0XBxccHOnTuRkpICg8GA7t27o6Cg7GmxTSZTqf8YiIiIylMyWtDaBSiuGMjOzpaX+Pj4Mvd58eJFFBUVoUGDBhbPN2jQAOfOnVPlOJlc1QAajQaurq6QJAmenp7w9PSEwWCQX3/88cfx8ssvo3Hjxpg0aRLq1auHbdu2AShuhzabzXj//ffRsmVLBAcHIykpCRkZGdi+fXuZ+0tMTLT4b8HX19cWh0lERNWcks2CRqPRYtFq1bnt1P1gclULhIaGyn+XJGAlnfgOHjyIEydOwMXFBQaDAQaDAe7u7sjPz0d6enqZ8eLj4y3+W8jMzLTJcRAREVVUvXr1YG9vj/Pnz1s8f/78eXh6eqqyT04iWgs4OjpaPJYkCWZz8V06c3Nz0aZNG6xevbrUdh4eHmXG02q1Veo/BCIiqh4q496CGo0Gbdq0wdatW9GrV6/iGGYztm7dilGjRllVljthclVDaDQaFBVV/BbsrVu3xtq1a1G/fn0YjUYVSkZERFTs1mY9a2JU1NixYzFkyBBERESgXbt2WLBgAa5fv46hQ4daVZY7YbNgDREQEIDc3Fxs3boVFy9eRF5e3j1tN3DgQNSrVw89e/bEzp07cerUKWzfvh1xcXH466+/VC41ERGR+vr374958+ZhypQpCA8Px4EDB7Bp06ZSndyVwuSqhujYsSNefPFF9O/fHx4eHpgzZ849befk5IQff/wRfn5+iImJQXBwMIYNG4b8/HzWZBERkaKU7NBeUaNGjcKff/4Jk8mE3bt3o3379gof3T/YLFiDLF26FEuXLrV4rqz5qg4cOGDx2NPTEytXrlSxZERERJXXLGhrrLkiIiIiUhBrroiIiMgmakvNFZMrIiIisgmBik+lUFaMqo7JFREREdkEa66IKujUzXrQFyp7SaXlKT97bpD+b8VjAoCf9pIqcd0c7m1ajYrKauKiStxr+XUVj+l0XPGQAACP3eq8ZwUu9VSJe1Ov/Ff2jcCy7yFqLWFWp0uv0JlViQvJXvGQ9jfUSQLs85SPW2RiF2wlMbkiIiIim2DNFREREZGCaktyxXpAIiIiIgWx5oqIiIhsorbUXDG5IiIiIpsQQoKwMjmydntbYLOgymJjY9GrV6+7rrN9+3ZIkoSrV6/apExERESkHiZXNhYVFYXRo0dbPNexY0dkZWXB1dVV1X0ziSMiospkhqTIUtWxWbAK0Gg08PRUfj4nIiKiqqS29LlizZUVEhISEB4ebvHcggULEBAQUOb6sbGx2LFjBxYuXAhJkiBJEk6fPl2qRik5ORlubm5Yv349mjVrBicnJ/Tp0wd5eXlYuXIlAgICUKdOHcTFxaGoqEiO/8EHHyAiIgIuLi7w9PTEgAEDcOHCBQDA6dOn0aVLFwBAnTp1IEkSYmNjAQBmsxmJiYkIDAyEXq9HWFgYPvvsM0XPFRERUW3BmisbWrhwIY4dO4YHHngA06dPBwB4eHjg9OnTpdbNy8vDokWLsGbNGly7dg0xMTF4+umn4ebmhm+//RYnT55E79690alTJ/Tv3x8AUFhYiBkzZqBZs2a4cOECxo4di9jYWHz77bfw9fXF559/jt69eyMtLQ1GoxF6vR4AkJiYiA8//BDLli1DkyZN8OOPP2LQoEHw8PBAZGRkqbKZTCaYTCb5cU5Ojgpni4iIapra0qGdyZUNubq6QqPRwMnJqdxmwMLCQixduhSNGjUCAPTp0wcffPABzp8/D4PBgJCQEHTp0gXbtm2Tk6vnn39e3j4oKAiLFi1C27ZtkZubC4PBAHd3dwBA/fr14ebmBqA4UZo9eza2bNmCDh06yNvu2rULy5cvLzO5SkxMxLRp06w+H0REVLuwWZAqlZOTk5xYAUCDBg0QEBAAg8Fg8VxJsx8A/Prrr+jRowf8/Pzg4uIiJ0YZGRl33M+JEyeQl5eHbt26wWAwyMuqVauQnp5e5jbx8fHIzs6Wl8zMTGsPl4iIaoGSmitrl6qONVdWsLOzgxDC4rnCwkJFYjs6Olo8liSpzOfM5uKbmF6/fh3R0dGIjo7G6tWr4eHhgYyMDERHR6Og4M43Zs3NzQUAbNiwAT4+PhavabXaMrfRarV3fI2IiKi2Y3JlBQ8PD5w7dw5CCEhScSZ94MCBu26j0WgsOqEr5ejRo7h06RLefPNN+Pr6AgD27dtXat8ALPYfEhICrVaLjIyMMpsAiYiIlCIUaBZkzVUNFxUVhb///htz5sxBnz59sGnTJmzcuBFGo/GO2wQEBGD37t04ffq0RT8oa/n5+UGj0eCdd97Biy++iMOHD2PGjBkW6/j7+0OSJKxfvx6PP/449Ho9XFxcMH78eIwZMwZmsxkPPfQQsrOzkZKSAqPRiCFDhihSPiIiIgHgtgaf+4pR1bHPlRWCg4OxZMkSvPvuuwgLC8OePXswfvz4u24zfvx42NvbIyQkRG66U4KHhweSk5Px6aefIiQkBG+++SbmzZtnsY6Pjw+mTZuGyZMno0GDBhg1ahQAYMaMGXjjjTeQmJiI4OBgdO/eHRs2bEBgYKAiZSMiIqpNJHF7pyGiCsrJyYGrqyuW/9YGeoOylaE/5zRWNB4ABOn/VjwmADjZmcpf6T5kmOqqEve7v5qrEvfa78qX1/W44iEBAB67L6kS96/u9VSJe1OvfMwbgXfuk2kNB6ebqsS9metY/kr3wem08nGFWtUXKvxqF5nycWLuf5CdnX3X1pf7VfI7EfbZONg7WddntyjPhIN95qtWViWwWZCIiIhsorbMc8VmQSIiIiIFseaKiIiIbMIsJEi1YBJRJldERERkE0IoMFqwGvQUZ3JFisk3O0IyK3tJXTAZyl+pguwks+IxAcBTo849Fls4nVElbkZdZaYBud3eRjrFYxb8rfx1AABFR46pEteptTrnVv+38p3EL1/XKB4TAHJaqFS74KDOL6tZhX7yDnnKxwQA7VXlz0FRQTXIWKoRJldERERkE7WlQzuTKyIiIrIJJldERERECqotHdo5FQMRERGRglhzRURERDZRW0YLsuaqhomNjUWvXr0quxhERESlFCdXkpVLZR9F+VhzVU2dPn0agYGB2L9/P8LDw+XnFy5cCN4ukoiIqPIwuaphXF1dK7sIREREZaotowXZLHifkpOT4efnBycnJzz99NOYP38+3Nzc5NfLap4bPXo0oqKi5MdmsxmJiYkIDAyEXq9HWFgYPvvsM/n1K1euYODAgfDw8IBer0eTJk2QlJQEAAgMDAQAtGrVCpIkyXFv36/JZEJcXBzq168PnU6Hhx56CHv37pVf3759OyRJwtatWxEREQEnJyd07NgRaWlpdzx2k8mEnJwci4WIiKg8QqGlqmNydR92796NYcOGYdSoUThw4AC6dOmCmTNnVjhOYmIiVq1ahWXLluGPP/7AmDFjMGjQIOzYsQMA8MYbb+DIkSPYuHEjUlNTsXTpUtSrVw8AsGfPHgDAli1bkJWVhS+++KLMfUycOBGff/45Vq5cid9++w2NGzdGdHQ0Ll++bLHea6+9hvnz52Pfvn1wcHDA888/f9dyu7q6youvr2+Fj52IiKimYrPgfVi4cCG6d++OiRMnAgCaNm2Kn376CZs2bbrnGCaTCbNnz8aWLVvQoUMHAEBQUBB27dqF5cuXIzIyEhkZGWjVqhUiIiIAAAEBAfL2Hh4eAIC6devC09OzzH1cv34dS5cuRXJyMh577DEAwIoVK7B582b873//w4QJE+R1Z82ahcjISADA5MmT8cQTTyA/Px86XelbmcTHx2Ps2LHy45ycHCZYRERUrtrSLMjk6j6kpqbi6aeftniuQ4cOFUquTpw4gby8PHTr1s3i+YKCArRq1QoA8NJLL6F379747bff8Oijj6JXr17o2LHjPe8jPT0dhYWF6NSpk/yco6Mj2rVrh9TUVIt1Q0ND5b+9vLwAABcuXICfn1+puFqtFlqt9p7LQUREBECZdr1q0C7I5EoldnZ2pUbtFRYWyn/n5uYCADZs2AAfHx+L9UoSl8ceewx//vknvv32W2zevBn/+te/8Morr2DevHmKl9fR8Z+7lkpS8X8FZrM6NzgmIiKqydjn6j4EBwdj9+7dFs/98ssvFo89PDyQlZVl8dyBAwfkv0NCQqDVapGRkYHGjRtbLLc2sXl4eGDIkCH48MMPsWDBArz33nsAAI2m+E72RUVFdyxno0aNoNFokJKSIj9XWFiIvXv3IiQkpGIHTUREZC2r57iSADYL1kxxcXHo1KkT5s2bh549e+K7774r1ST4yCOPYO7cuVi1ahU6dOiADz/8EIcPH5ab/FxcXDB+/HiMGTMGZrMZDz30ELKzs5GSkgKj0YghQ4ZgypQpaNOmDVq0aAGTyYT169cjODgYAFC/fn3o9Xps2rQJDRs2hE6nKzUNg7OzM1566SVMmDAB7u7u8PPzw5w5c5CXl4dhw4bZ5mQRERH9H87QTnf04IMPYsWKFVi4cCHCwsLw/fff4/XXX7dYJzo6Gm+88QYmTpyItm3b4tq1axg8eLDFOjNmzMAbb7yBxMREBAcHo3v37tiwYYM8zYJGo0F8fDxCQ0PRuXNn2NvbY82aNQAABwcHLFq0CMuXL4e3tzd69uxZZlnffPNN9O7dG8899xxat26NEydO4LvvvkOdOnVUODNERER3Zv3s7NZ3iLcFSXA6b0UkJydj9OjRuHr1amUXxeZycnLg6uqKhfsehN6gbGXod5daKBoPADx16szL5alRJ66/9qIqcTdeaqlK3L2ZpQdBWEv3i0HxmADg+fZPqsTNHvSgKnH1f99UPOblYI3iMQEgp0Vh+SvdD3t1frJ0fyp/HhzyFA8JANBeVf4cFBXk41Dya8jOzobRaFQ8fsnvRMD/ex12TqVHoVeEOS8fp5+fqVpZlcBmQSIiIrINJfpMVYOaKyZXREREZBPsc0UVEhsbWyubBImIiMgSa65IMYXCHg7CvrKLUa4LJhdV4tZxVKeDhT3UmW+sueGcKnGhwmT9GW7qDMDIsrv3SXkr4loT5ftGAYCTCv2ChEq/ApKjOtetWrUWpqB8xWPmF6lTf3HjsvJvmjnfRtVBnESUiIiISDm15fY3bBYkIiIiUhBrroiIiMh2qkGznrWYXBEREZFNsFmQiIiIiCqMNVdERERkG7VktCBrrshCXl4eevfuDaPRCEmSOHcXEREpSFJoqdpYc0UWVq5ciZ07d+Knn35CvXr14OrqWtlFIiKimoI1V1QbpaenIzg4GA888AA8PT0hSVX/PwQiIiJrnT59GsOGDUNgYCD0ej0aNWqEqVOnoqCgoMKxmFzVMp9//jlatGgBrVaLgIAAzJ8/X34tKioK8+fPx48//ghJkhAVFVVmDJPJhJycHIuFiIioXEKhRQVHjx6F2WzG8uXL8ccff+Dtt9/GsmXL8J///KfCsdgsWIv8+uuv6NevHxISEtC/f3/89NNPePnll1G3bl3Exsbiiy++wOTJk3H48GF88cUX0GjKvtVGYmIipk2bZuPSExFRtSek4sXaGCro3r07unfvLj8OCgpCWloali5dinnz5lUoFpOrWuStt97Cv/71L7zxxhsAgKZNm+LIkSOYO3cuYmNj4e7uDicnJ2g0Gnh6et4xTnx8PMaOHSs/zsnJga+vCjeUIyIiuoPbW020Wi20Wq2i+8jOzoa7u3uFt2OzYC2SmpqKTp06WTzXqVMnHD9+HEVFRfccR6vVwmg0WixERETlEUKZBQB8fX3h6uoqL4mJiYqW9cSJE3jnnXcwcuTICm/L5IqIiIhsQ8E+V5mZmcjOzpaX+Pj4Mnc5efJkSJJ01+Xo0aMW25w5cwbdu3dH3759MWLEiAofJpsFa5Hg4GCkpKRYPJeSkoKmTZvC3t6+kkpFRERUcffacjJu3DjExsbedZ2goCD577Nnz6JLly7o2LEj3nvvvfsqG5OrWmTcuHFo27YtZsyYgf79++Pnn3/G4sWLsWTJksouGhER1QaV0KHdw8MDHh4e97TumTNn0KVLF7Rp0wZJSUmws7u/Bj4mV7VI69at8cknn2DKlCmYMWMGvLy8MH369HIzeiIiIiVIonixNoYazpw5g6ioKPj7+2PevHn4+++/5dfuNsirLEyuapnevXujd+/ed3x9wYIFtisMERFRFbF582acOHECJ06cQMOGDS1eE6JiGR07tBMREZFtVOFJRGNjYyGEKHOpKNZcERERkW1U4UlElcTkioiIiGyjlty4mckVKSb1ug80kqOiMU9crqdoPABw0hQqHhMAzua6qhL3b3cXVeL66y6pEvcht+OKx9xrF6h4TAD4oany1xcA2F1XZ2oTj4PKX7v6M7mKxwSAKy3V+TwUGNSptbgSeu8TKd8zh2qQBZAqmFwRERGRbbDmioiIiEhBtSS54mhBIiIiIgWx5oqIiIhsg6MFiYiIiJRTlWdoVxKbBWuY5ORkuLm5yY8TEhIQHh5usU5CQgIaNGgASZKwbt06m5aPiIiopmPNVQ03fvx4/Pvf/5Yfp6amYtq0afjyyy/x4IMPok6dOpVYOiIiqlVqSYd2Jlc1nMFggMFgkB+np6cDAHr27AlJqvrt1kRERNUNmwVruFubBRMSEtCjRw8AgJ2dnUVy9f777yM4OBg6nQ7NmzfHkiVLKqO4RERE1R5rrmqR8ePHIyAgAEOHDkVWVpb8/OrVqzFlyhQsXrwYrVq1wv79+zFixAg4OztjyJAhpeKYTCaYTCb5cU5Ojk3KT0RE1ZsEBTq0K1ISdTG5qkUMBoPc2d3T01N+furUqZg/fz5iYmIAAIGBgThy5AiWL19eZnKVmJiIadOm2aTMRERUg9SSqRjYLFjLXb9+Henp6Rg2bJjcP8tgMGDmzJly/6zbxcfHIzs7W14yMzNtXGoiIqqWhEJLFceaq1ouN7f4pq0rVqxA+/btLV6zty/75rNarRZarVb1shEREVVHTK5quQYNGsDb2xsnT57EwIEDK7s4RERUk3EqBqotpk2bhri4OLi6uqJ79+4wmUzYt28frly5grFjx1Z28YiIqIaoLTO0M7kiDB8+HE5OTpg7dy4mTJgAZ2dntGzZEqNHj67sohEREVU77NBew8TGxuLq1avy44SEBBw4cEB+3KtXLwhROu0fMGAA9u/fD5PJhMuXL2PHjh14+umnbVBiIiKqNdihnYiIiEhBtaTPFWuuiIiIiBTEmisiIiKyCXZoJyIiIlJSLZmhnckVKSZAfxE6vbKX1G47f0XjAYCdSv/2eBuyVYl7o8hRlbiXbzqrEjffrHx5mxuyyl/pPlxroc5kuOHGv1SJu9o/QvGYBSfcFI8JAGbvfFXiSud0qsRVo5OMU9085YMCQF3lQxblmcpfie4ZkysiIiKyjVrSoZ3JFREREdkE+1wRERERKamW1FxxKgYiIiIiBbHmioiIiGxDgWbB6lBzxeSKiIiIbIPNgkRERERUUay5IiIiItuoJTVXTK6IiIjIJjgVA9EdmEwmmEz/zOabk5NTiaUhIiKqWtjniiosMTERrq6u8uLr61vZRSIiIqoymFxRhcXHxyM7O1teMjMzK7tIRERUHQiFliqOzYJUYVqtFlqtOje8JSIiqu6YXBEREZFNsEM7ERERkdKqQXJkLfa5olKSk5MhSVJlF4OIiGqaWtLniskVlXLq1ClERkZWdjGIiIiqJTYLUikbN27E4sWLK7sYRERUw7DPFdVae/bsqewiEBFRTVRLbn/DZkEiIiIiBbHmioiIiGyCzYJEFXTlphO0Nx0VjWm6aa9oPAC4UaBsGUsUmtWpCPZyVufejdkFOlXi1tHcUDxmA60658BTp05cM9QZbRtY97LiMY+cNSgeEwCQrVElrPG0Oue2wKj8z+ENk4viMQFAGG4qHtN8w0YjxNksSEREREQVxZorIiIiso1aUnPF5IqIiIhsorb0uWKzIBEREZGCWHNFREREtlFLmgVZc1XDBQQEYMGCBZVdDCIiIt5bkKqX5ORkuLm5VXYxiIiI7qikz5W1S1XH5IqIiIhIQUyuaoDt27dj6NChyM7OhiRJkCQJCQkJ8ut5eXl4/vnn4eLiAj8/P7z33nsW22dmZqJfv35wc3ODu7s7evbsidOnT99xfyaTCTk5ORYLERFRudgsSNVFx44dsWDBAhiNRmRlZSErKwvjx4+XX58/fz4iIiKwf/9+vPzyy3jppZeQlpYGACgsLER0dDRcXFywc+dOpKSkwGAwoHv37igoKChzf4mJiXB1dZUXX19fmxwnERFVb2wWpGpDo9HA1dUVkiTB09MTnp6eMBj+uaXF448/jpdffhmNGzfGpEmTUK9ePWzbtg0AsHbtWpjNZrz//vto2bIlgoODkZSUhIyMDGzfvr3M/cXHxyM7O1teMjMzbXGYRERE1QKTq1ogNDRU/rskAbtw4QIA4ODBgzhx4gRcXFxgMBhgMBjg7u6O/Px8pKenlxlPq9XCaDRaLEREROWqJs2CJpMJ4eHhkCQJBw4cqPD2nOeqFnB0tLxRsSRJMJvNAIDc3Fy0adMGq1evLrWdh4eHTcpHRES1RDWZ52rixInw9vbGwYMH72t7Jlc1hEajQVFRUYW3a926NdauXYv69euzBoqIiGq9jRs34vvvv8fnn3+OjRs33lcMNgvWEAEBAcjNzcXWrVtx8eJF5OXl3dN2AwcORL169dCzZ0/s3LkTp06dwvbt2xEXF4e//vpL5VITEVFtIim0ACg1at1kMlldvvPnz2PEiBH44IMP4OTkdN9xmFzVEB07dsSLL76I/v37w8PDA3PmzLmn7ZycnPDjjz/Cz88PMTExCA4OxrBhw5Cfn8+aLCIiUpaCfa58fX0tRq4nJiZaVzQhEBsbixdffBERERFWxWKzYA2ydOlSLF261OK5suarur1znqenJ1auXKliyYiIiJSVmZlpUQmg1WrLXG/y5Mn473//e9dYqamp+P7773Ht2jXEx8dbXTYmV0RERGQTSsxTVbL9vY5WHzduHGJjY++6TlBQEH744Qf8/PPPpZK0iIgIDBw4sEKVEEyuiIiIyDYqYbSgh4fHPY1+X7RoEWbOnCk/Pnv2LKKjo7F27Vq0b9++QvtkckVERES2U0VnWPfz87N4XDIZd6NGjdCwYcMKxWJyRYq5dlMH003H8lesSMwLhvJXqiBHl7Jv62Otq0X3P7Lkbm4W2asS19d4RZW4Znksj3JMZnW+qjw16twXs6Hmkipx81w1isc866/OwJUrF1xUiWs4q0pYaFJvKh7zmq/y7xcAZDdW9nsWAMz5FZ/Kh+6MyRURERHZhJJ9rtQWEBAAIe5vZ0yuiIiIyDaqyQzt1uI8V0REREQKYs0VERER2UR1aha0BpMrIiIisg02CxIRERFRRTG5qkQBAQFYsGBBZReDiIjIJkqaBa1dqjo2C1aivXv3wtnZubKLQUREZBu1pFmQyVUlupfp+K0hhEBRUREcHPg2ExER2QqbBe9DQkICwsPDLZ5bsGABAgIC5MexsbHo1asX5s2bBy8vL9StWxevvPIKCgsL5XVubRYcMGAA+vfvbxGzsLAQ9erVw6pVqwAAZrMZiYmJCAwMhF6vR1hYGD777DN5/e3bt0OSJGzcuBFt2rSBVqvFrl27cPDgQXTp0gUuLi4wGo1o06YN9u3bJ2+3a9cuPPzww9Dr9fD19UVcXByuX79+x+M3mUzIycmxWIiIiMolFFqqOCZXKtq2bRvS09Oxbds2rFy5EsnJyUhOTi5z3YEDB+Kbb75Bbm6u/Nx3332HvLw8PP300wCAxMRErFq1CsuWLcMff/yBMWPGYNCgQdixY4dFrMmTJ+PNN99EamoqQkNDMXDgQDRs2BB79+7Fr7/+ismTJ8PRsfj2Cenp6ejevTt69+6NQ4cOYe3atdi1axdGjRp1x+NKTEyEq6urvPj6+lp5poiIqDZgnyuyWp06dbB48WLY29ujefPmeOKJJ7B161aMGDGi1LrR0dFwdnbGl19+ieeeew4A8NFHH+Gpp56Ci4sLTCYTZs+ejS1btqBDhw4AgKCgIOzatQvLly9HZGSkHGv69Ono1q2b/DgjIwMTJkxA8+bNAQBNmjSRX0tMTMTAgQMxevRo+bVFixYhMjISS5cuhU6nK1XW+Ph4jB07Vn6ck5PDBIuIiMrHPldkrRYtWsDe/p+b7np5eeH3338vc10HBwf069cPq1evxnPPPYfr16/jq6++wpo1awAAJ06cQF5enkXSBAAFBQVo1aqVxXMREREWj8eOHYvhw4fjgw8+QNeuXdG3b180atQIAHDw4EEcOnQIq1evltcXQsBsNuPUqVMIDg4uVVatVgutVluBM0FERFR7MLm6D3Z2dqVu5nhrX6oSJU1vJSRJgtlsvmPcgQMHIjIyEhcuXMDmzZuh1+vRvXt3AJCbCzds2AAfHx+L7W5PdG4fgZiQkIABAwZgw4YN2LhxI6ZOnYo1a9bg6aefRm5uLkaOHIm4uLhS5fHz87tjWYmIiCpKEgLSfd4M+dYYVR2Tq/vg4eGBc+fOQQgBSZIAAAcOHLA6bseOHeHr64u1a9di48aN6Nu3r5yghYSEQKvVIiMjw6IJ8F41bdoUTZs2xZgxY/Dss88iKSkJTz/9NFq3bo0jR46gcePGVpefiIjortgsSHcSFRWFv//+G3PmzEGfPn2wadMmbNy4EUaj0erYAwYMwLJly3Ds2DFs27ZNft7FxQXjx4/HmDFjYDab8dBDDyE7OxspKSkwGo0YMmRImfFu3LiBCRMmoE+fPggMDMRff/2FvXv3onfv3gCASZMm4cEHH8SoUaMwfPhwODs748iRI9i8eTMWL15s9fEQERHVNhwteB+Cg4OxZMkSvPvuuwgLC8OePXswfvx4RWIPHDgQR44cgY+PDzp16mTx2owZM/DGG28gMTERwcHB6N69OzZs2IDAwMA7xrO3t8elS5cwePBgNG3aFP369cNjjz2GadOmAQBCQ0OxY8cOHDt2DA8//DBatWqFKVOmwNvbW5HjISIiKlFbRgtK4vbOQ0QVlJOTA1dXV4zY0Qcag2P5G1TAN7+FKxoPABxdChSPCQD2DnfuT2cNg96kSlxf4xVV4rpq8hWP6e5453nXrFFXpbgNNZdUiXv0hvL/9GzKLD1oRQlXLrioErfhBvvyV7oPmuybise85qtRPCYAZKvQi8Ocn4+TM19Ddna2Iq0wtyv5nWg1YBbsNaVHoVdEUUE+9n+kXlmVwJorIiIiIgWxzxURERHZhBLNetWhWZDJFREREdkGRwsSVYzG7ia0dpKiMe2dle8Hoda/PU46dfpGOdgXqRI3u0CvSlyzUL63gZ1K36bnTer013jAI1OVuF6abMVjdvZJVzwmANQPuKZK3I/rtlEl7o2Tyl8Ldup8JaDISfnPg9nRNhlLbam5Yp8rIiIiIgWx5oqIiIhsg82CRERERMqqDs161mKzIBEREZGCWHNFREREtiFE8WJtjCqOyRURERHZBEcLEhEREVGFseaKiIiIbIOjBYmIiIiUI5mLF2tjVHVMrqjCTCYTTKZ/ph7OycmpxNIQERFVLexzRRWWmJgIV1dXefH19a3sIhERUXUgFFqqOCZXVGHx8fHIzs6Wl8xMde6jRkRENUvJaEFrl6qOzYJUYVqtFlqttrKLQURE1U0tmeeKNVdERERECmLNFREREdkEJxGlWis5ORmSJFV2MYiIqKZhh3aqrU6dOoXIyMjKLgYREVG1xGZBKmXjxo1YvHhxZReDiIhqmNrSLMjkikrZs2dPZReBiIhqIo4WJCIiIqKKYs0VERER2QSbBYkq6FKBMzQFGkVjmm8qP2pRCHvFYwKAqdBRlbiFReqUV4jqMyJU51CoStwr+U6qxD18Q51bQrk75CoeM0B3UfGYavI0XlMl7umbrorHNGQoHhIAYG9S/rNbVCBBpeJaUmK0XzVIrtgsSERERKQg1lwRERGRTbBZkIiIiEhJZlG8WBujimNyRURERLbBPldEREREVFFMrqhMkiRh3bp1lV0MIiKqQST80+/qvpfKPoh7wGZBIiIisg3O0E5EREREFcXkqhq7dOkSnn32Wfj4+MDJyQktW7bExx9/bLFOVFQU4uLiMHHiRLi7u8PT0xMJCQkW6xw/fhydO3eGTqdDSEgINm/ebMOjICKi2sLqJkEFpnKwBTYLVmP5+flo06YNJk2aBKPRiA0bNuC5555Do0aN0K5dO3m9lStXYuzYsdi9ezd+/vlnxMbGolOnTujWrRvMZjNiYmLQoEED7N69G9nZ2Rg9evRd92symWAymeTHOTk5ah0iERHVJBwtSFWdj48Pxo8fj/DwcAQFBeHf//43unfvjk8++cRivdDQUEydOhVNmjTB4MGDERERga1btwIAtmzZgqNHj2LVqlUICwtD586dMXv27LvuNzExEa6urvLi66vOrT6IiIiqIyZX1VhRURFmzJiBli1bwt3dHQaDAd999x0yMizvEBUaGmrx2MvLCxcuXAAApKamwtfXF97e3vLrHTp0uOt+4+PjkZ2dLS+ZmZkKHREREdVkkhCKLFUdmwWrsblz52LhwoVYsGABWrZsCWdnZ4wePRoFBQUW6zk6Wt5QWJIkmM3m+96vVquFVqu97+2JiKiWMv/fYm2MKo7JVTWWkpKCnj17YtCgQQAAs9mMY8eOISQk5J5jBAcHIzMzE1lZWfDy8gIA/PLLL6qUl4iIqDZgs2A11qRJE2zevBk//fQTUlNTMXLkSJw/f75CMbp27YqmTZtiyJAhOHjwIHbu3InXXntNpRITEVFtVluaBZlcVWOvv/46WrdujejoaERFRcHT0xO9evWqUAw7Ozt8+eWXuHHjBtq1a4fhw4dj1qxZ6hSYiIhqN6HQoqINGzagffv20Ov1qFOnToV/VwE2C1Zr7u7u5d6iZvv27aWeu32bpk2bYufOnRbPiWrwnwEREVUzVXyG9s8//xwjRozA7Nmz8cgjj+DmzZs4fPhwheMwuSIiIqJa7+bNm3j11Vcxd+5cDBs2TH6+Iv2YS7BZkIiIiGxCyRnac3JyLJZbJ7e+H7/99hvOnDkDOzs7tGrVCl5eXnjsscfuq+aKyRURERHZRkmzoLULAF9fX4sJrRMTE60q2smTJwEACQkJeP3117F+/XrUqVMHUVFRuHz5coViMbkiIiKiaiczM9NiQuv4+Pgy15s8eTIkSbrrcvToUXn+x9deew29e/dGmzZtkJSUBEmS8Omnn1aobOxzRYr5O98AB3uFJxfNcSx/nQoya9WZge7GRXUmVjXr1CnvdedCVeKajDcUj3ku20XxmACgcShSJe5vGnVuCaWzv6l4TB/9VcVjAkAL/V/qxHXLUiUuIpQP+fcZda4DY4byn92bhcpfW2WRzMWLtTEAwGg0wmg0lrv+uHHjEBsbe9d1goKCkJVVfG3d2sdKq9UiKCio1J1PysPkioiIiGyjEkYLenh4wMPDo9z12rRpA61Wi7S0NDz00EMAgMLCQpw+fRr+/v4V2ieTKyIiIqr1jEYjXnzxRUydOhW+vr7w9/fH3LlzAQB9+/atUCwmV0RERGQbSkwCquI0jHPnzoWDgwOee+453LhxA+3bt8cPP/yAOnXqVCgOkysiIiKyCSVuX6Pm7W8cHR0xb948zJs3z6o4HC1IREREpCAmV9VIbGxsufc4CggIwIIFC2xSHiIiogpRcJ6rqozNglXQ6dOnERgYiP379yM8PLxC2+7duxfOzs7qFIyIiMgaAoC1s8tU/dyKyZWtFRYWwtFR+bmbStzLcFMiIqLKUNX7XCmlVjcL7tq1Cw8//DD0ej18fX0RFxeH69evAwD+85//oH379qW2CQsLw/Tp0+XH77//PoKDg6HT6dC8eXMsWbJEfu306dOQJAlr165FZGQkdDod3nvvPRiNRnz22WcWcdetWwdnZ2dcu3YNgYGBAIBWrVpBkiRERUVZrDtv3jx4eXmhbt26eOWVV1BY+M+Ecrc3C0qShPfffx9PP/00nJyc0KRJE3z99dcW8b7++ms0adIEOp0OXbp0wcqVKyFJEq5evVrmeTOZTKXu6URERETFam1ylZ6eju7du6N37944dOgQ1q5di127dmHUqFEAgIEDB2LPnj1IT0+Xt/njjz9w6NAhDBgwAACwevVqTJkyBbNmzUJqaipmz56NN954AytXrrTY1+TJk/Hqq68iNTUVMTExeOaZZ5CUlGSxTlJSEvr06QMXFxfs2bMHALBlyxZkZWXhiy++kNfbtm0b0tPTsW3bNqxcuRLJyclITk6+67FOmzYN/fr1w6FDh/D4449j4MCB8n2STp06hT59+qBXr144ePAgRo4ciddee+2u8RITEy3u5+Trq84sxEREVMMIKNDnqrIPony1NrlKTEzEwIEDMXr0aDRp0gQdO3bEokWLsGrVKuTn56NFixYICwvDRx99JG+zevVqtG/fHo0bNwYATJ06FfPnz0dMTAwCAwMRExODMWPGYPny5Rb7Gj16tLyOl5cXhg8fju+++06eav/ChQv49ttv8fzzzwP4p2mvbt268PT0hLu7uxyrTp06WLx4MZo3b44nn3wSTzzxBLZu3XrXY42NjcWzzz6Lxo0bY/bs2cjNzZUTuOXLl6NZs2aYO3cumjVrhmeeeabc2wTEx8db3M8pMzPzHs44ERHVerWkQ3utTa4OHjyI5ORkGAwGeYmOjobZbMapU6cAFNdelSRXQgh8/PHHGDhwIADg+vXrSE9Px7BhwyxizJw506K2CwAiIixvWtWuXTu0aNFCruH68MMP4e/vj86dO5db7hYtWsDe3l5+7OXlhQsXLtx1m9DQUPlvZ2dnGI1GeZu0tDS0bdu2VPnuRqvVyvd0utd7OxEREdUWtbZDe25uLkaOHIm4uLhSr/n5+QEAnn32WUyaNAm//fYbbty4gczMTPTv31/eHgBWrFhRqm/WrckPgDJH7w0fPhzvvvsuJk+ejKSkJAwdOhSSJJVb7ts7w0uSJN/JW8ltiIiIFGcGUP5PXfkxqrham1y1bt0aR44ckZv4ytKwYUNERkZi9erVuHHjBrp164b69esDABo0aABvb2+cPHlSrs2qiEGDBmHixIlYtGgRjhw5giFDhsivaTQaAEBRUVGF41ZUs2bN8O2331o8t3fvXtX3S0REtQ9HC9ZwkyZNwk8//YRRo0bhwIEDOH78OL766iu5Q3uJgQMHYs2aNfj0009LJVHTpk1DYmIiFi1ahGPHjuH3339HUlIS3nrrrXL3X6dOHcTExGDChAl49NFH0bBhQ/m1+vXrQ6/XY9OmTTh//jyys7OVOegyjBw5EkePHsWkSZNw7NgxfPLJJ3IH+XupSSMiIiJLtTa5Cg0NxY4dO3Ds2DE8/PDDaNWqFaZMmQJvb2+L9fr06YNLly4hLy+v1Ozow4cPx/vvv4+kpCS0bNkSkZGRSE5OlqdSKM+wYcNQUFAgd2Qv4eDggEWLFmH58uXw9vZGz549rTrWuwkMDMRnn32GL774AqGhoVi6dKk8WlCr1aq2XyIiqoVqSYd2SYhqUMoa6oMPPsCYMWNw9uxZuSmwKpg1axaWLVt2z6MAc3Jy4Orqin9tGAkHZ2UTsqOHlZ/mQWjVabC3y7Mvf6X7YNapVF7nwvJXug9G4w3FYxbcVOfcahzUaXpv5H5Rlbg6+5uKx/TRX1U8JgC00P+lStzfrgeoEvePq16Kx/z7K3WmqXE7ofxn92ZhPn7+fiqys7NVGaQk/06EjIeDvXW/EzeLTNh6ZJ5qZVVCre1zVZny8vKQlZWFN998EyNHjqz0xGrJkiVo27Yt6tati5SUFMydO7dU8ygRERHdm1rbLFiZ5syZg+bNm8PT0xPx8fGVXRwcP34cPXv2REhICGbMmIFx48YhISGhsotFREQ1TS1pFmTNVSVISEioUsnL22+/jbfffruyi0FERDUdp2Igqpg62htw1Crch8WswojFInVGQZpdlO8PAwCSozrfJA4adfobmYXy59folK94TADIL1DnJuqeumuqxC0Uyjc2mMzq/Aw01ZxXJW5dh1xV4jbRK1/etU+oc27P/6x8/7Ci/CLge8XDlsKpGIiIiIiowlhzRURERLahRJ+palBzxeSKiIiIbMMsAMnK5Mhc9ZMrNgsSERERKYg1V0RERGQbtaRZkDVXtVRycjLc3NwquxhERFSrKDHHFZMrqqL69++PY8eOVXYxiIiIahw2C9ZSer0eer2+sotBRES1CZsFqSa7vVnw4MGD6NKlC1xcXGA0GtGmTRvs27ev8gpIREQ1j1kos1RxrLkiAMDAgQPRqlUrLF26FPb29jhw4AAcHcuevdpkMsFkMsmPc3JybFVMIiKiKo/JFQEAMjIyMGHCBDRv3hwA0KRJkzuum5iYiGnTptmqaEREVFMIc/FibYwqjs2CBAAYO3Yshg8fjq5du+LNN99Eenr6HdeNj49Hdna2vGRmZtqwpEREVG1ZO1JQiT5bNsDkigAACQkJ+OOPP/DEE0/ghx9+QEhICL788ssy19VqtTAajRYLERFRuWpJnysmVyRr2rQpxowZg++//x4xMTFISkqq7CIRERFVO0yuCDdu3MCoUaOwfft2/Pnnn0hJScHevXsRHBxc2UUjIqKapJY0C7JDO8He3h6XLl3C4MGDcf78edSrVw8xMTHstE5ERMoSUGCeK0VKoiomV7VUbGwsYmNjAQAajQYff/xx5RaIiIiohmByRURERLZRS2ZoZ3JFREREtmE2A7Bynioz57kiIiIiqlVYc0VERES2wWZBooq5lO8EB3utojEdrkuKxgMAc6E6FbbCpE5cs06dKvCCAnXKW1SkfNzcXJ3iMdX0U1aAKnGdNIWKx9Q7Kh8TAJKKHlIlbl3H66rEraNCXD+XK4rHBICr4cp/HuzyTOWvpIRaklyxWZCIiIhIQay5IiIiItswC1g9UVU1uP0NkysiIiKyCSHMEMK6rg7Wbm8LTK6IiIjINoQCN15mnysiIiKi2oU1V0RERGQbQoE+V9Wg5orJFREREdmG2QxIVvaZqgZ9rtgsSERERKQg1lwRERGRbbBZkKhsJpMJJtM/s/nm5ORUYmmIiKi6EGYzhJXNgtVhKgY2C1KFJSYmwtXVVV58fX0ru0hERERVBpMrqrD4+HhkZ2fLS2ZmZmUXiYiIqoOSewtau1RxbBakCtNqtdBqlb1BMxER1QJmAUg1v88Va66IiIiIFMSaKyIiIrINIQBYO89V1a+5YnJFRERENiHMAsLKZkHB5IqIiIjo/wgzrK+54lQMRERERLUKa66IiIjIJtgsSERERKSkWtIsyOSKrFbyX8TNvALFY5vz85WPaVbnvx6h0qfJrNYXiYM6cSXcVDymMEuKx1RTkYOp/JXuw83CQuVjOiofEwAKJOW/DwDApFJ58x2Vv24Lr6tzDorylL++SmKqXSt0E4VW31rwJtS5BpQkiepQv0ZV2l9//cVb4BAR1QCZmZlo2LCh4nHz8/MRGBiIc+fOKRLP09MTp06dgk6nUySe0phckdXMZjPOnj0LFxcXSNLdaxhycnLg6+uLzMxMGI1GxcqgRtzqVNbqFrc6lbW6xa1OZa1ucatTWSsaVwiBa9euwdvbG3Z26ox1y8/PR0GBMrV5Go2myiZWAJsFSQF2dnYV/k/HaDQq+iWiZtzqVNbqFrc6lbW6xa1OZa1ucatTWSsS19XVVfF930qn01XphEhJnIqBiIiISEFMroiIiIgUxOSKbEqr1WLq1KnQarVVPm51Kmt1i1udylrd4lansla3uNWprGrGpfKxQzsRERGRglhzRURERKQgJldERERECmJyRURERKQgJldU7Wzfvh2SJOHq1auVXZQqISAgAAsWLCjztdjYWPTq1euu21eX83kvx3IneXl56N27N4xG4x2PNTk5GW5ubvLjhIQEhIeHW6yTkJCABg0aQJIkrFu37r7Koqa7XQtqbltTKP2+lnfNJicnw87Ortaf95qIyVU1ERsbi4SEBPlxWV/8aqiMH96jR4/iwQcfhE6nQ3h4OKKiojB69Gib7f9uTp8+Xe4s9GW5/Yf7figRo6xz2bFjR2RlZak+geC9Xksl5/jAgQMWzy9cuBDJycn3te+VK1di586d+Omnn+75WMePH4+tW7fKj1NTUzFt2jQsX74cWVlZeOyxx+6rLEpQ4lq43d69e/HCCy8oGrO2uNM1W57+/fvj8OHDPO81EJMruitb/fDeaurUqXB2dkZaWprFj1tZ7vdLrTxqxb2bQhVuynsvNBoNPD097ytptCVXV9f7TijS09MRHByMBx544J6P1WAwoG7duhYxAKBnz57w9PSsccPbPTw84OTkpFp8IQRu3lT+5shqU/NzqdfrERISoup5p8rB5KoKu3LlCnJzcyu1DJXxw5ueno6HHnoI/v7+GDduHHbs2IGFCxdCkiRIkoTTp08DAH799Vc89dRTAICBAwfiiSeegI+PD5ycnNCyZUuMGzcOrVu3hk6nQ1BQEAICAjBq1ChMnDgR7u7u8PT0tKgNFEIgISEBfn5+aNasGQDgv//9LwBg8eLFeOCBB+R1161bB0mSsGzZMmRkZAAAunbtitdff11e56uvvkLr1q2h0WgwdOhQZGdny8eQkJAASZIwf/58+Pr6QpIkODk54bHHHsPx48dLnZPt27eXihEVFSXXXubl5eH555+HVquFg4MD3nvvPYvtMzMzERAQUOa5vL1Gafjw4bCzs4NWq4XBYICjoyM0Gg3y8vKwcuVK+bm4uDgUFRUBAEaPHo3g4GBERETAxcUFDRo0QFhYGPz8/KDX6xEcHIwuXboAAOrUqQNJkhAUFAQPDw/o9XrUq1cP9erVg16vR2BgIACgVatW8nECpZtYTCYT4uLiUL9+fTg6OsLJyQkajQYBAQGYP3++fFxhYWGYP38+fvzxR0iShHbt2t3TdXhr7XBCQgJ69OgBoPh2T7d+Ht5//30EBwdDp9OhefPmWLJkyT3Fv19lXQuenp7y63l5eWjfvj3s7Ozg5+eH9957Tz538+bNQ/369aHVaqHValGnTh307NkTp0+ftmgWHDBgAPr372+x38LCQtSrVw+rVq0CUHxP0cTERAQGBkKv1yMsLAyfffaZRTklScLGjRvRpk0baLVa7Nq1CwcPHkSXLl3g4uICo9GINm3aYN++ffJ2u3btwsMPPwy9Xg9fX1/ExcXh+vXr8msdOnSAg4MD7O3t4eDggBYtWuDjjz/Gf/7zH7Rv3x5AcQ1tXFwcJk6cCAcHBxgMBvmzXvJ+abVaODk5wdHRESEhIdi8eTMA4Omnn8batWsRGRkJnU6H9957D0aj0eLYgOLvAGdnZ1y7du2O12yJefPmwcvLC3Xr1sUrr7wiJ2y3NwsePHgQkiRBp9PB0dERdnZ28PX1xddff20R7+uvv0aTJk2g0+nQpUsXrFy5slo07dcqgqqUwsJCsX79etGnTx+h1WrFgQMHhBBCDBkyREydOlVeb+rUqSIsLEwsW7ZMNGzYUOj1etG3b19x9epVi3grVqwQzZs3F1qtVjRr1ky8++67Fq+npKSIsLAwodVqRZs2bcSXX34pAIj9+/cLIYTYtm2bACCuXLkihBAiKSlJuLq6ik2bNonmzZsLZ2dnER0dLc6ePXtPx1dUVCSmTZsmfHx8hEajEWFhYWLjxo3y6wAslkmTJokOHTqIESNGiKysLJGVlSW2bNkiAIj27duXWr9NmzYiPT1dvPrqqwKAqFu3rtBoNKJhw4bCwcFBaLVakZCQIGJiYkTDhg2FJEni+++/FyaTSQQEBAhHR0fx7bfflhlXkiSxb98+AUCMHj1a1KtXT/Tv319ERUWJkJAQ4ejoKNasWSOEEOLHH38URqNRJCcni9TUVPHSSy8JSZLEuHHjRFZWlrh27ZoAIDQajfDy8hJr1qwRGzduFNHR0aJx48aioKDA4ryZTCaxYMECYTQa5fMQHx8vwsLChL+/v3B3dxfvvvuu+M9//iPc3NyEnZ2dOHr0qBgyZIjo0aOHCA4OFoMGDRJhYWGib9++4umnnxaNGjUSeXl5Fu/xL7/8IiRJEnZ2dqJjx45iwoQJwmAwCEmSxKOPPir69esnevbsKdq3by80Go18vK+++qpo1qyZ+Pbbb0V6eroYOXKk0Ov1IiIiQqSnp4v//e9/wtHRUQAQaWlpYujQoaJly5Zi7969Yty4ccLX11ckJCSI9PR08cYbbwgAYv78+SIrK0tcunRJ/gz07NlTPidxcXHC29tbLFy4UNjZ2Ynw8HBhNBrFO++8I/R6vZg0aZL83j355JMiPDxctG/fXrRt27bMa7Pk2r79MyaEENeuXRNJSUkCgHz+hRDiww8/FF5eXuLzzz8XJ0+eFJ9//rlwd3cXycnJ9/R5uB+3Xwvjxo0TLVu2FEII+Vro3bu38PHxEYmJicLOzk706tVLGI1G8cILL4igoCDRtWtXodPpxLRp08SAAQNEs2bNhL+/v3j77beFEEKsX79e6PV6ce3aNXm/33zzjdDr9SInJ0cIIcTMmTNF8+bNxaZNm0R6erpISkoSWq1WbN++XQjxz3dHaGio+P7778WJEyfEpUuXRIsWLcSgQYNEamqqOHbsmPjkk0/k77kTJ04IZ2dn8fbbb4tjx46JlJQU0apVKxEbGyu/lpCQICZOnCiSkpJEixYtRPv27YW9vb34+OOPBQBx4sQJERkZKYxGo3j55ZcFADFnzhwhSZKYNGmS8PLyEp9++qlo2rSpaNmypXB1dRXx8fGiVatW8uc9ICBAfk/Pnj0rRowYIR5//HGL9+Gpp54SgwcPFkIIsWfPHgFAbNmypdQ1azQaxYsvvihSU1PFN998I5ycnMR7770nhCi+5iRJks97ixYtBADRoEEDMX/+fLFo0SLx7LPPCoPBIMc8efKkcHR0FOPHjxdHjx4VH3/8sfDx8bH4nqbKx+Sqijh06JAYO3asaNCggXB3dxcvvfSS+Omnn+TXy0qunJ2dxSOPPCL2798vduzYIRo3biwGDBggr1PeF392drZwd3cXgwYNEn/88Yf49ttvRdOmTctNrhwdHUXXrl3F3r17xa+//iqCg4Mt9ns3b731ljAajeLjjz8WR48eFRMnThSOjo7i2LFjQgghsrKyRIsWLSySkMjISPHqq6/KMUrKtGXLFvlLbdasWQKAOHPmjBCi+EvK3t5ePPXUU/Kx29nZCXt7eyFE8Y9lUFCQaNCggZg0aZIYP368qFOnjpzY3P5lefHiRVG3bl3x7rvvCgAiPDxcJCYmCk9PT3HhwgXx6quvyknJY489Jh544AExffp0ucxJSUnCyclJeHl5yc+VfJGnpKTIz128eFHo9XrxySeflDp3d/rx9/f3F4MGDRJCCPH2228Lf39/Ub9+fbF06VIxZMgQ0bp1a9GsWTNhNpvlc2kymYRerxffffedxXv87LPPitDQUPlHSggh+vfvLzQajXBychLXrl2Tk5zo6GgxcuRIIURxchUZGSmEECI/P184OTmJ//3vfwKA/AP9+OOPy/vp0aOHGDp0qLzurdf6qVOnBADRvXt3i+O/NbnKzc0Vjo6OYvXq1WLAgAGiW7duoqCgQHh7e4s5c+aICRMmCH9/f/k9LCnfhg0bBABx48aNez6/JUr+8bhVo0aNxEcffWTx3IwZM0SHDh1KxVfSrWW9tZwl10LJdWA2m0X9+vXFgw8+KPz9/cXKlSvla6Fv376if//+8rXQoEED+Ue+sLBQ1KtXT6xatUre57PPPiv69+8vhBBlvm9CCDFs2DDx7LPPCiH++ZyuW7fOYh0XF5c7Jp/Dhg0TL7zwgsVzO3fuFHZ2diI2NvaOr3Xv3l2MGzdOhIWFienTp4vIyEjx0EMPifj4eNG+fXshhBBt27YVbm5u4qOPPhLfffedcHBwEGfOnJHfr40bN8qfyQULFljsZ/fu3cLe3l7+J/L8+fPCwcFBTiRLrtmS780SQ4YMEf7+/uLmzZvycyXnXYjSyZWLi4sAIF5//XV5/dzcXAFA/id00qRJ4oEHHrDYz2uvvcbkqophs2AlunTpEhYuXIjWrVsjIiICJ0+exJIlS5CVlYUlS5agQ4cOd90+Pz8fq1atQnh4ODp37ox33nkHa9aswblz5wAU912aP38+YmJiEBgYiJiYGIwZMwbLly8HAHz00UeQJAkrVqxASEgIHnvsMUyYMKHcchcWFmLZsmWIiIhA69atMWrUqHL7RpWYN28eJk2ahGeeeQbNmjXDf//7X4SHh8vV4p6ennI1vqenJwwGwx1jhYaGwsPDA0DxSCcAeOSRR+Du7o4//vgDRUVF2LBhA1q2bInBgwfDbDajqKgIeXl5MBgM+PDDD3HhwgVs2rQJCxYswIoVK2AymRAUFIT58+cDKO7n4+npibp166Jz58745ZdfAABHjhzByy+/DJPJhEuXLqF+/fro0KEDDh8+jNDQUBw5cgRTpkyRm6tGjhyJvLw8ZGVlIS8vTz4Ge3t7uSkDAOrWrYtmzZohNTX1ns7nrefiVp6enrhw4QIAICcnBydOnICLiwt27tyJd999F+7u7sjPz5f7EZVITU1FUFAQnJyc0KhRIwBAhw4dIEkSAgICLN6PBg0ayPsAgGvXrqFHjx4ICAhAXl4ehg0bBgCoX78+DAYDvv/+e3ndl156CWvWrEFYWBjy8vLwyCOPwGAwwGAwICQkBEBxU+adpKeno7CwEJ06dUJqaio6deoER0dHtGvXTn585syZUufGy8sLACzKfb+uX7+O9PR0DBs2TC67wWDAzJkzS51XW7r1eEuaDPPz89GiRQv8/vvv8rXw5Zdf4tNPP5WvhVv7Fjk4OKBfv35YvXo1gOJj/eqrrzBw4EAAwIkTJ5CXl4du3bpZHPuqVatKHXtERITF47Fjx2L48OHo2rUr3nzzTYv1Dx48iOTkZIuY0dHRMJvN2Ldvn/yaVquFvb09Hn74YZjNZmzevBkZGRkYOHAgPvroIwBAy5Yt8fHHH8tl9vDwwNWrVzFs2DD06NEDRUVFaNq0qfx+3fp9e3uZ27VrhxYtWmDlypUAgA8//BD+/v7o3Llzue9HixYtYG9vLz/28vK64/U3duxYAMVdCkrOjbOzM4xGo7xNWloa2rZtW6p8VLUwuapE77zzDkaPHg2DwYATJ07gyy+/RExMDDQazT1t7+fnBx8fH/lxhw4dYDabkZaWdk9f/GlpaQgNDYVOp5Nj3MuH9NYfXuDuXxa3ysnJwdmzZ9GpUyeL50t+ICvK0dFR/nv9+vUAgJEjR+Lbb78FUPzDIoSA2WyG2WyGJEnQarXy8Xbo0AGNGzfGwYMHMW7cOPTu3RtpaWlYsmSJvM6wYcPkH52oqCjs3r0bQHHfCqPRiM6dO2P79u3YsWMHIiMjERwcjDfffBMajUYeTWZvb4/p06fDxcUFx48ftzjf1rCzs4P4v7tXlZyLkrJKkgSz2QwAuHnzJtq0aYMDBw4gIiICAwcOxIEDB3Ds2DEMGDCg3HN7+3Ml+711H3l5eTh06BCMRqPct2Xu3LkAgDVr1uDAgQMWI/0ee+wx/Pnnn+jXrx+A4v47zzzzDA4cOCC/f/Pmzbvvc3OnYynpK1VSbmuU9IdcsWIFDhw4IC+HDx+Wk3BbuPU6AIqP99ZEqeRz4OjoiNzcXPlaGDhwICIiIuRrwdnZ2SLuwIEDsXXrVly4cAHr1q2DXq9H9+7dAfxz7Bs2bLA49iNHjpTqm3R73ISEBPzxxx944okn8MMPPyAkJARffvmlHHfkyJEWMQ8ePIjjx4+jsLAQI0eOxIsvvggnJyfMmTMHX3/9NbZs2YJHH30UBQUFePbZZ5GWloZr167h4sWLyMzMlPuOlfQPXLFiBSZMmAAfH587vl+3lxko7otYcg0nJSVh6NCh99QX9fbP0q2fm9uVfHYiIiIszs3dtqGqiclVJXrhhRcwY8YMnDt3Di1atMDQoUPxww8/VPkv/rK+LISKt6jUaDTyF+OdpKWlASjujFrSuVSn0+GRRx7BoUOHcOjQIbRt2xbPPvss7OyKL3uz2YzLly9DkiScOHECQPHonR49eshfcocOHcLvv/8OAIiMjJQ7m5d0WI2KisKWLVuQkpKCqKgoZGZm4s0334QkSfj+++/Rr18/fPnll/D19YUQAo0bN5b3DxR/4ZckbEBxbWZaWppce3O38+Dh4YFz585ZnPuyRje6urri+PHjqF+/PlxdXeHi4oLGjRujcePGpUaBBgcH4+TJkxbP3X69eHh4ICsry+K53bt34+bNm3jzzTcxYMAAaLVa+Zz6+fmhcePG8vtScgweHh6YMGECtFotBgwYgE8++QSNGzdG06ZN5dfvpFGjRtBoNEhJSUFwcDBSUlJQWFiIvXv3IiQkBCkpKWjYsOEdt1dCgwYN4O3tjZMnT8rns2QpOVa13Hot3Ot1AACtW7eWrwU3Nzfo9Xq5zLdel0DxSGFfX1+sXbsWq1evRt++feXPfkhICLRaLTIyMkodu6+vb7nlb9q0KcaMGYPvv/8eMTExSEpKkst35MiRUjEbN26MNm3a4MiRI0hLS0NMTAzGjRuHHj16oEuXLvK11rBhQ0RGRuLChQs4evQounXrhvr16wMo/j7Q6/U4efIkOnfujHPnzsHZ2Vl+v8r7Xhw0aBD+/PNPLFq0CEeOHMGQIUMs3g8A5X5P3aunnnqq1Lkp0axZM4sBAEDxNBpUtTC5qkTe3t54/fXXcezYMWzatAkajQYxMTHw9/fH5MmT8ccff9x1+4yMDJw9e1Z+/Msvv8DOzg7NmjW7py/+Zs2a4ffff4fJZJJjqPkhNRqN8Pb2RkpKisXzKSkpZSYTJQICArB7926cPn0aFy9etEg+S77UGjRoAKB4BOHly5eh1Wpx48YNZGZmwmQyobCwENeuXbM4p3PnzsW1a9fQvXt3bNq0CcOGDcP//vc/HD58WE4etFot/P39ARQ3t5QkI7cmV+vWrUN+fj5mzZqFgIAAbNiwASNHjoQkSWjatKncFJWbm4uBAwfi4sWLctNgu3btMGLECHkU1aBBg+Dj44OePXuWeR5yc3OxdetWXLx4Ee3bt8fff/+NnJwcXLx4Ee+++y42btxYajtfX1/Uq1cPPXv2hE6nw48//og1a9ZgxIgR8kjHEnFxcfI1cfz4cSxevBibNm2yWOeRRx7Bvn37kJ6ejtzcXEydOhWnTp2CJEl455138Pfff+Opp57CihUrABQ37/32229y0/H69esxbtw4rFmzBufPn8eQIUOwevVqeHh4ID09HX/99RccHR0xe/ZsnD9/HtnZ2aWOydnZGS+99BImTJiADh06YOvWrWjXrh2uXbsGJycnLF68uNRoNzVMmzYNiYmJWLRoEY4dO4bff/8dSUlJeOutt1Td763XQlhYGP7++2/MmTMHhYWF2LVrV5nXAVBcG1VyLZw5cwb5+fnYvn27xcjPWw0YMADLli3D5s2b5eY1AHBxccH48eMxZswYrFy5Eunp6fjtt9/wzjvvyE1nZblx4wZGjRqF7du3488//0RKSgr27t2L4OBgAMCkSZPw008/YdSoUThw4ACOHz+Or776CqNGjZJfy8zMxIYNG7B27VosXrwYLVu2xPnz5y2O8cKFCzhx4oRFmQEgPDwciYmJSE1Nhb+/P2JiYjBt2jSMGjUKr7322l3PeZ06dRATE4MJEybg0UcftUje69evD71ej02bNt3xmi1PybkBiputbz83JUaOHImjR49i0qRJOHbsGD755BO5Rq2qT6dSq1Raby8q040bN8THH38soqOjhb29vTh06JAQ4s4d2rt27SoOHDggfvzxR9G0aVPxzDPPyOusWLFC6PV6sXDhQpGWliYOHTok/t//+39i/vz5Qoh/OrQPHjxYHDlyRB4BCEAevXOn0YK3Kquj7528/fbbwmg0ijVr1oijR4+KSZMmWXRoF0KIsLAwi2NNS0sTDz74oNDr9QKAPGrrypUrorCwUOj1ejFs2DABQDg5OYn69euLJ554QtjZ2QknJyeh0+mEwWAQOp1ONGnSRAghxG+//SY0Go1o3769GDJkiFi+fLnQ6XTyiDNnZ2chSZIYOnSoOHfunDwKs1u3bhadtIuKiuRRiG+88YbcCVwIITZt2iQ6duwo9Hq9MBqNon79+sLZ2VkAEFOnThUAxAcffCCee+454erqKvR6vYiOjrY4F7d78cUXRd26deUYS5cuFfb29kKj0YjBgweLWbNmCX9/f/kclnQCz8rKEoMHDxZ16tQRkiQJSZIEAHHo0KFS7/HQoUOFJElCr9eLHj16iHnz5gmtVmvRuXvKlClCp9MJBwcHMWbMGDFq1CgRHBwsAgIChFarFQ8++KAYPny4ACAcHByEh4eHiI6OFs8//7zw9PQUAORjdnd3Fw888IAIDAwUjo6OwsPDQ7Ro0ULUr19f2NnZyR3lbx8teOPGDfHvf/9b1KtXTzg4OAi9Xi8cHByEn5+fmDt3rsVxlXRo379/vwAgTp06Verc3k+HdiGEWL16tQgPDxcajUbUqVNHdO7cWXzxxRd3fA9v3+f9fg3fei088cQTwtfXV0iSJCIiIuTrQIjiz1NYWJh87kquBZ1OJyRJEkFBQWLEiBHCz89P7lhd4siRIwKA3Dn+VmazWSxYsEA0a9ZMft+io6PFjh07hBClvzuEKB7p+MwzzwhfX1+h0WiEt7e3+Ne//mVxDvbs2SO6desmDAaDcHZ2FqGhoWLWrFnya1FRUcLe3l6+tqKiosTgwYPl47ty5YqQJEk4ODhYjHbs2bOnGDJkiPx+OTo6CgcHByFJkvDy8hKbNm2SO7Tf3jG9xNatWwWAMgecrFixQvj6+t71mhXCcvDHrR3aS85NyXF5e3uLUaNGiRs3bghXV1eRlJQkx/jqq69E48aNhVarFVFRUWLp0qV3HKhBlYPJVRV25swZkZ2dLYS481QMS5YsEd7e3kKn04k+ffqIy5cvW8Qo74s/JSVFhIaGCo1GI9q0aSM++ugjAUAcPXpUCKF8clVUVCQSEhKEj4+PcHR0LDUVgxClk6vylPWldrdjv3HjhggJCSk18uipp54SHTt2lEf2lBW3ZFTQrY4ePVrqh6cmKes9J2VMmTLF4pqtjarTOVi1apWoW7euMJlMlV0UCzNnzhQNGzas7GLQLSQhVOwsQ4qJjY1FQECAxaSXali9erU8QaFer1d1X9XR6dOnERgYqGofs6omOTkZo0eP5gSFKmjXrh0WL15cq0d7VYdzUDLS96mnnkKvXr0wa9asSi3PkiVL0LZtW9StWxcpKSn497//jVGjRmHmzJmVWi76h0NlF4Aq16pVqxAUFAQfHx8cPHgQkyZNQr9+/ZhYEdnAnj17KrsIla46nIM5c+Zg1qxZ6Ny5M+Lj4yu7ODh+/DhmzpyJy5cvw8/PD+PGjasS5aJ/sOaqmlCr5mrOnDlYsmQJzp07By8vL/m/svu919Xd5qXauHEjHn744QrFmz17NmbPnl3maw8//PAdO+6qpTbWXBERUcUwuaom1q1bBzc3t1L3rKpqSoZEl8XHx6fCNWKXL1/G5cuXy3xNr9dbzPNlC1evXsWCBQtUb54lIqLqi8kVERERkYI4zxURERGRgphcERERESmIyRURERGRgphcERERESmIyRURERGRgphcERERESmIyRURERGRgv4/jTkF+iHKTAQAAAAASUVORK5CYII=",
"text/plain": [
"<Figure size 640x480 with 2 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"def display_qk_heatmap(qk_per_token):\n",
" _, ax = plt.subplots()\n",
" im = ax.imshow(qk_per_token.to(float).detach(), cmap='viridis')\n",
" ax.set_xticks(range(len(prompt_split_as_tokens)))\n",
" ax.set_yticks(range(len(prompt_split_as_tokens)))\n",
" ax.set_xticklabels(prompt_split_as_tokens)\n",
" ax.set_yticklabels(prompt_split_as_tokens)\n",
" ax.figure.colorbar(im, ax=ax)\n",
" \n",
"display_qk_heatmap(qk_per_token)"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([[0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],\n",
" [0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],\n",
" [0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],\n",
" [0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],\n",
" [0., 0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],\n",
" [0., 0., 0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],\n",
" [0., 0., 0., 0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],\n",
" [0., 0., 0., 0., 0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],\n",
" [0., 0., 0., 0., 0., 0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],\n",
" [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf],\n",
" [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf],\n",
" [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf],\n",
" [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., -inf, -inf, -inf, -inf],\n",
" [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., -inf, -inf, -inf],\n",
" [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., -inf, -inf],\n",
" [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., -inf],\n",
" [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]])"
]
},
"execution_count": 30,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"mask = torch.full((len(tokens), len(tokens)), float(\"-inf\"), device=tokens.device)\n",
"mask = torch.triu(mask, diagonal=1)\n",
"mask"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAlcAAAGdCAYAAAA/oFbLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy80BEi2AAAACXBIWXMAAA9hAAAPYQGoP6dpAABmrklEQVR4nO3deVxU5f4H8M9hmWFYBkRBFkFAXEAFVNTUEuxqaGUabiWmmJotXnJXbqmoKV6XQjPX7g+0LO222GJS6hUXKpdStERU1OAqZm4gIgMyz+8PLidHQMQ5M2yf9+v1vF6cM+d8z3PODMOX53nOcyQhhAARERERKcKipitAREREVJ8wuSIiIiJSEJMrIiIiIgUxuSIiIiJSEJMrIiIiIgUxuSIiIiJSEJMrIiIiIgVZ1XQFqO7T6/W4ePEiHBwcIElSTVeHiIiqSQiBmzdvwsPDAxYWpml3KSwsRFFRkSKxVCoVbGxsFIllCkyuyGgXL16El5dXTVeDiIiMlJ2djWbNmiket7CwEL7N7XHpcoki8dzc3HDu3Llam2AxuSKjOTg4AAC6fPwSrGxVisbeHr5A0XhERFReXl4evLy85O9zpRUVFeHS5RL8/rMPtA7GtYzl3dSjeafzKCoqYnJF9VdZV6CVrQpWdmpFY2u1WkXjERFR5Uw9tMPeQYK9g3HH0KP2Dz9hckVERERmUSL0KDHyicYlQq9MZUyIdwsSERERKYgtV0RERGQWegjoYVzTlbH7mwOTKyIiIjILPfQwtlPP+AimZ9JuwejoaMTFxcnLcXFxCAkJMeUhAQApKSmQJAk3btww+bHKnDx5Eo888ghsbGzMco7Vcf78ec4/RURENa5ECEVKbVcvx1x1794dOTk5cHR0NNsx58yZAzs7O2RkZGDXrl333bYs2Tl69KiidTBVXCIiInpwiidX169fR35+vtJhq0WlUsHNzc2srTWZmZl49NFH0bx5czRu3Nhsx31YWVlZNV0FIiJqYMrGXBlbajtFkqs7d+5g27ZtGDJkCNzd3ZGZmXnf7deuXQsvLy/Y2tpi6NChyM3NNXj9/fffR0BAAGxsbNCmTRusWrXK4PUffvgBISEhsLGxQWhoKLZu3WrQYnNvt2BSUhKcnJzw3XffISAgAPb29ujbty9ycnIe6Pz0ej3mzZuHZs2aQa1WIyQkBMnJyfLrkiTh559/xrx58yBJkkFXaEV8fX0BAB06dIAkSQgPD3+gc3/xxRcRFBQEnU4HoHRStg4dOmDkyJFVxr3XqFGj0K5dOyxZsuSBr0MZnU6HvLw8g0JERFQVPQRKjCz1Prk6fvw4pkyZgmbNmmHkyJFwcXHB7t27ERwcXOk+Z86cwSeffIKvv/4aycnJOHLkCF599VX59U2bNmH27NlYsGAB0tPTsXDhQsyaNQsbNmwAUDqLbP/+/dG+fXv88ssvmD9/PmbMmFFlXQsKCrB06VJ88MEH2Lt3L7KysjB16tQHOs/ly5dj2bJlWLp0KY4dO4aIiAg888wzOH36NAAgJycHbdu2xZQpU5CTk1Nl3IMHDwIAdu7ciZycHHz++ecPdO4rVqzArVu3MHPmTADAG2+8gRs3bmDlypX3jVuRTz75BC+99BK2bNkCLy8vPPnkk9iyZQsKCwurvB7x8fFwdHSUCx99Q0RE9Jdq3y149epVfPjhh9iwYQN+++03PPnkk1i1ahWefvppqFRVP/qksLAQGzduhKenJwDg3XffxVNPPYVly5bBzc0Nc+bMwbJlyxAZGQmgtDXmxIkTWLt2LUaNGoWPPvoIkiRh/fr1sLGxQWBgIC5cuIBx48bd97jFxcVYs2YNWrRoAQCYMGEC5s2b90DnvHTpUsyYMQPPPfccAOCf//wndu/ejYSEBLz33ntwc3ODlZUV7O3t4ebmVmU8FxcXAEDjxo0Ntq/q3O3t7fHhhx8iLCwMDg4OSEhIwO7du+VZzCuLW1kdYmJiEBMTg/T0dGzYsAFTp07Fyy+/jGHDhiE6OhqPPPJIhfvGxsZi8uTJ8nLZYxOIiIjuh1MxVOLdd9/F3Llz8dhjj+HMmTPV/qPq7e0tJ1YA0K1bN+j1emRkZMDBwQGZmZkYM2aMQbJ0584deXB6RkYGgoKCDJ4n1KVLlyqPa2trKydWAODu7o7Lly9XuV9eXh4uXryIHj16GKzv0aMH0tLSqtz/Qd26davKcwdKr9fUqVPlFrtHH33U6GMHBARg0aJFWLhwIZYsWYJZs2Zh8+bNld5tqVaroVYr+5gbIiKq/5S4268u3C1Y7eTqpZdegpWVFTZu3Ii2bdti0KBBeOGFFxAeHg4LC+OGcJUNhF+/fj26du1q8JqlpaVRsa2trQ2WJUmCqEVv0IOeu16vR2pqKiwtLXHmzBlFjp2dnY1Nmzbhgw8+wLlz5zBkyBCMHj1akdhEREQNTbWzIQ8PD7z55ps4deoUkpOToVKpEBkZiebNm2PmzJn47bff7rt/VlYWLl68KC//9NNPsLCwQOvWrdG0aVN4eHjg7Nmz8Pf3Nyhlg7Vbt26N48ePy4O6AeDQoUPVPY0HptVq4eHhgdTUVIP1qampCAwMfKiYZd2nJSUl8roHOXcAWLJkCU6ePIk9e/YgOTkZiYmJ941bmZs3byIpKQmPP/44fHx8sG3bNkyePBmXLl3Cpk2b0Lt374c6NyIiosroFSq1nVEztHfv3h3du3fH8uXLsXXrViQlJWHp0qU4cuQI2rdvX+E+NjY2GDVqFJYuXYq8vDzExMRg6NCh8hihuXPnIiYmBo6Ojujbty90Oh0OHz6M69evY/LkyRg+fDjeeOMNvPTSS5g5cyaysrKwdOlSAKZ7mve0adMwZ84ctGjRAiEhIUhMTMTRo0exadOmh4rn6uoKjUaD5ORkNGvWDDY2NnB0dKzy3I8cOYLZs2fj008/RY8ePfD222/j9ddfR1hYGPz8/CqNW5GBAwfi7NmzeOGFF7B+/XqDLlMiIiJTKLvjz9gYtZ0iUzHY2NjgueeeQ3JyMrKystC8efNKt/X390dkZCSefPJJPPHEEwgKCjKYbmDs2LF4//33kZiYiPbt2yMsLAxJSUly641Wq8XXX3+No0ePIiQkBG+88QZmz54t18MUYmJiMHnyZEyZMgXt27dHcnIyvvrqK7Rs2fKh4llZWWHFihVYu3YtPDw8MGDAAAD3P/fCwkKMGDEC0dHR6N+/P4DSLtpevXrhhRdeQElJSaVxK7Jq1SqcPXsW8+bNY2JFRESkIEmYcOBRdHQ0fHx8qpz3yVibNm3C6NGjkZubC41GY9Jj1UXnz5+Hr6+vycaY5eXlwdHREd2/nAArO2UHuu/521JF4xERUXll3+O5ubnyHeimiH/shCscHIxr17l5U4+gwMsmq6sS6uSDmzdu3Ag/Pz94enoiLS0NM2bMwNChQ5lYERER1WJKjJmqC2Ou6uSzBS9duoQRI0YgICAAkyZNwpAhQ7Bu3bqHjmdvb19p2bdvX7XjLVy4sNJ4/fr1e+h6EhER1WV6SCgxsuhhvkfbPSyTtlwNHDgQTk5OisedPn06pk+frli8+z3o+O45uR7Uyy+/jKFDh1b4Wk20rjk5OWHOnDlmPy4REVFDZNIxV9QwlPWlXz/lB62Rfen3avHJy4rGA4BzMVMUj0lEVJeZa8zV4d+awt7IvxP5N/UIbfsHx1wRERERlXXtGRujtquTY66IiIiIaiu2XBEREZFZNJSWKyZXREREZBZ6IUEvjEuOjN3fHNgtSERERKQgJlf1WEpKCiRJwo0bN2q6KkREREbPcaVEt6I5MLmqR8LDwzFx4sSargYREVGFSmChSKntOOaKiIiIzEIoMOZKcMwVmUt0dDT27NmD5cuXQ5IkSJKE8+fPAwB+/vlnhIaGwtbWFt27d0dGRobBvl9++SU6duwIGxsb+Pn5Ye7cubhz504NnAUREVHdx+Sqnli+fDm6deuGcePGIScnBzk5OfDy8gIAvPHGG1i2bBkOHz4MKysrvPjii/J++/btw8iRI/H666/jxIkTWLt2LZKSkrBgwYJKj6XT6ZCXl2dQiIiIqsIxV1SnODo6QqVSwdbWFm5ubnBzc4OlpSUAYMGCBQgLC0NgYCBmzpyJH374AYWFhQCAuXPnYubMmRg1ahT8/PzQp08fzJ8/H2vXrq30WPHx8XB0dJRLWRJHRER0PyXCQpFS29X+GpLRgoKC5J/d3d0BAJcvXwYApKWlYd68ebC3t5dLWetXQUFBhfFiY2ORm5srl+zsbNOfBBERUR3BAe0NgLW1tfyzJJU2p+r1egBAfn4+5s6di8jIyHL72djYVBhPrVZDrVaboKZERFSf6SFBb2S7jh5CodqYDpOrekSlUqGkpKRa+3Ts2BEZGRnw9/c3Ua2IiIhK8fE3VOf4+PjgwIEDOH/+POzt7eXWqfuZPXs2nn76aXh7e2Pw4MGwsLBAWloafv31V7z11ltmqDUREVH9wjFX9cjUqVNhaWmJwMBAuLi4ICsrq8p9IiIi8M033+D7779H586d8cgjj+Cdd95B8+bNzVBjIiJqSBrKgHa2XNUjrVq1wo8//miwLjo62mA5JCQEQhj2V0dERCAiIsLU1SMiogaudMyVkQ9urgPdgrU//SMiIiKqQ9hyRURERGahV+DZgLxbkIiIiOh/lBgzVSKYXBEZReN9U/GYPZ9arHhMANi7bbpJ4hIR1Rd6WDSIea445oqIiIhIQWy5IiIiIrMoERJKhJGTiBq5vzkwuSIiIiKzKFFgQHsJuwWJiIiIGhYmV0RERGQWemGhSKmO+Ph4dO7cGQ4ODnB1dcXAgQORkZFhojMsxeSKiIiIzKKsW9DYUh179uzBa6+9hp9++gk7duxAcXExnnjiCdy6dctEZ8kxV0RERFQH5eXlGSyr1Wqo1epy2yUnJxssJyUlwdXVFT///DN69uxpkrqx5YqIiIjMQo+/7hh82KL/XywvLy84OjrKJT4+/oHqkJubCwBwdnY2zUmCLVdUheLiYlhbW9d0NYiIqB5QZhLR0v2zs7Oh1Wrl9RW1WpXbV6/HxIkT0aNHD7Rr186oetwPW65M6OrVq3j++efh6ekJW1tbtG/fHh9//LHBNuHh4YiJicH06dPh7OwMNzc3xMXFya8LIRAXFwdvb2+o1Wp4eHggJiYGALBy5UqDD8fWrVshSRLWrFkjr+vduzfefPNNefnLL79Ex44dYWNjAz8/P8ydOxd37tyRX5ckCatXr8YzzzwDOzs7LFiwQOnLQkREZDStVmtQHiS5eu211/Drr79i8+bNJq0bkysTKiwsRKdOnbBt2zb8+uuveOmll/DCCy/g4MGDBttt2LABdnZ2OHDgABYvXox58+Zhx44dAIDPPvsM77zzDtauXYvTp09j69ataN++PQAgLCwMJ06cwJ9//gmgdNBekyZNkJKSAqC01enHH39EeHg4AGDfvn0YOXIkXn/9dZw4cQJr165FUlJSuQQqLi4Ozz77LI4fP44XX3yx3HnpdDrk5eUZFCIioqqUPVvQ2PIwJkyYgG+++Qa7d+9Gs2bNFD4zQ0yuTMjT0xNTp05FSEgI/Pz88Pe//x19+/bFJ598YrBdUFAQ5syZg5YtW2LkyJEIDQ3Frl27AABZWVlwc3ND79694e3tjS5dumDcuHEAgHbt2sHZ2Rl79uwBAKSkpGDKlCny8sGDB1FcXIzu3bsDAObOnYuZM2di1KhR8PPzQ58+fTB//nysXbvWoD7Dhw/H6NGj4efnB29v73LnFR8fb9DP7eXlpeyFIyKiekkPSZFSHUIITJgwAV988QX+85//wNfX10Rn9xcmVyZUUlKC+fPno3379nB2doa9vT2+++47ZGVlGWwXFBRksOzu7o7Lly8DAIYMGYLbt2/Dz88P48aNwxdffCF340mShJ49eyIlJQU3btzAiRMn8Oqrr0Kn0+HkyZPYs2cPOnfuDFtbWwBAWloa5s2bB3t7e7mMGzcOOTk5KCgokI8fGhp63/OKjY1Fbm6uXLKzs42+VkREVP/VRMvVa6+9hg8//BAfffQRHBwccOnSJVy6dAm3b9820VlyQLtJLVmyBMuXL0dCQgLat28POzs7TJw4EUVFRQbb3TtgXJIk6PWl90N4eXkhIyMDO3fuxI4dO/Dqq69iyZIl2LNnD6ytrREeHo5169Zh37596NChA7RarZxw7dmzB2FhYXLc/Px8zJ07F5GRkeXqamNjI/9sZ2d33/Oq7HZXIiKi2mb16tUAIA+RKZOYmIjo6GiTHJPJlQmlpqZiwIABGDFiBIDSuxROnTqFwMDAasXRaDTo378/+vfvj9deew1t2rTB8ePH0bFjR4SFhWHixIn497//LX9wwsPDsXPnTqSmpmLKlClynI4dOyIjIwP+/v6KnSMREdGDUubZgtXbXwjzP4uQyZUJtWzZEp9++il++OEHNGrUCG+//Tb++OOPaiVXSUlJKCkpQdeuXWFra4sPP/wQGo0GzZs3B1DapdioUSN89NFH+OabbwCUJldTp06FJEno0aOHHGv27Nl4+umn4e3tjcGDB8PCwgJpaWn49ddf8dZbbyl78kRERPfQCwl6Ub0xUxXFqO045sqE3nzzTXTs2BEREREIDw+Hm5sbBg4cWK0YTk5OWL9+PXr06IGgoCDs3LkTX3/9NRo3bgygtAvxsccegyRJePTRRwGUJlxarRahoaEGXXwRERH45ptv8P3336Nz58545JFH8M4778iJGhERERlPEjXRXkb1Sl5eHhwdHXH9lB+0Dsrm6+1+ilI0HgA0WW+reEwA2LttukniEhGZWtn3eG5ursHEnErHX3QoDDb2xnWaFebfwczOe0xWVyWwW5CIiIjMQi8soH/IearujlHb1f4aEhEREdUhbLkiIiIisyiBhJJqTgJaUYzajskVKSa96Dbsi5RtDHV3VP7ROmf7OCgeEwAC//GOSeKeWDjJJHGJiMyN3YJEREREVG1suSIiIiKzKIHx3XolylTFpJhcERERkVk0lG5BJldERERkFg/z4OWKYtR2tb+GRERERHUIW66IiIjILAQk6I0ccyU4FQMRERFRKXYLUr2TlJQEJyenmq4GERFRvcaWKyIiIjILvZCgF8Z16xm7vzmw5aqBSElJwejRo5GbmwtJkiBJEuLi4gAA169fx8iRI9GoUSPY2tqiX79+OH36dKWxdDod8vLyDAoREVFVSmChSKntan8NSRHdu3dHQkICtFotcnJykJOTg6lTpwIAoqOjcfjwYXz11Vf48ccfIYTAk08+ieLi4gpjxcfHw9HRUS5eXl7mPBUiIqJajclVA6FSqeDo6AhJkuDm5gY3NzfY29vj9OnT+Oqrr/D+++/jscceQ3BwMDZt2oQLFy5g69atFcaKjY1Fbm6uXLKzs817MkREVCeVdQsaW2o7jrlq4NLT02FlZYWuXbvK6xo3bozWrVsjPT29wn3UajXUarW5qkhERPWEHhbQG9muY+z+5lD7a0hERERUhzC5akBUKhVKSgwfeRkQEIA7d+7gwIED8rqrV68iIyMDgYGB5q4iERHVYyVCUqTUdkyuGhAfHx/k5+dj165duHLlCgoKCtCyZUsMGDAA48aNw/79+5GWloYRI0bA09MTAwYMqOkqExFRPdJQxlwxuWpAunfvjpdffhnDhg2Di4sLFi9eDABITExEp06d8PTTT6Nbt24QQuDbb7+FtbV1DdeYiIjqEyEsoDeyiDowQzsHtDcwq1evxurVqw3WNWrUCBs3bqyhGhEREdUvTK6IiIjILEogocTIBy8bu785MLkiIiIis9AL4x9foxcKVcaEan/HJREREVEdwpYrUszeglawsVD2I3VHr3z+b93sluIxAeCWk2kmVvX/59smiXtmxmSTxCUiqkzZoHRjY9R2TK6IiIjILPSQoDdyzJSx+5tD7U//iIiIiOoQtlwRERGRWSgxw3pdmKGdyRURERGZRUMZc1X7a0hERERUhzC5qud8fHyQkJBQ09UgIiIqHdBu7LMFOaCdzCUpKQlOTk41XQ0iIqJKif/dLWhMEXUgueKYKyIiIjKLstYnY2PUdmy5qgdSUlIwevRo5ObmQpIkSJKEuLg4+fWCggK8+OKLcHBwgLe3N9atW2ewf3Z2NoYOHQonJyc4OztjwIABOH/+vHlPgoiIqJ5gclUPdO/eHQkJCdBqtcjJyUFOTg6mTp0qv75s2TKEhobiyJEjePXVV/HKK68gIyMDAFBcXIyIiAg4ODhg3759SE1Nhb29Pfr27YuioqIKj6fT6ZCXl2dQiIiIqlJ2t6Cxpbar/TWkKqlUKjg6OkKSJLi5ucHNzQ329vby608++SReffVV+Pv7Y8aMGWjSpAl2794NANiyZQv0ej3ef/99tG/fHgEBAUhMTERWVhZSUlIqPF58fDwcHR3l4uXlZY7TJCKiOs7owewKdCuaA5OrBiAoKEj+uSwBu3z5MgAgLS0NZ86cgYODA+zt7WFvbw9nZ2cUFhYiMzOzwnixsbHIzc2VS3Z2tlnOg4iIqC7ggPYGwNra2mBZkiTo9XoAQH5+Pjp16oRNmzaV28/FxaXCeGq1Gmq1aR5STERE9VdDebYgk6t6QqVSoaSkpNr7dezYEVu2bIGrqyu0Wq0JakZERFSKdwtSneLj44P8/Hzs2rULV65cQUFBwQPtFxUVhSZNmmDAgAHYt28fzp07h5SUFMTExOC///2viWtNRERU/zC5qie6d++Ol19+GcOGDYOLiwsWL178QPvZ2tpi79698Pb2RmRkJAICAjBmzBgUFhayJYuIiBTVUAa0s1uwHlm9ejVWr15tsK6i+aqOHj1qsOzm5oYNGzaYsGZERETsFiQiIiKih8CWKyIiIjKLhtJyxeSKiIiIzELA+KkUhDJVMSkmV0RERGQWbLkiqqaDuT6wvqNSNObvF5ooGg8A3N2uKx4TABxdrpok7ln7xiaJG/z3d0wSN+3dSSaJS0RUVzC5IiIiIrNgyxURERGRghpKcsWpGIiIiKje2rt3L/r37w8PDw9IkoStW7ea/JhMroiIiMgsamKG9lu3biE4OBjvvfeeic6qPHYLEhERkVkIIUEY2a1Xtn9eXp7BerVaDbVaXW77fv36oV+/fkYds7rYcmVi0dHRGDhw4H23SUlJgSRJuHHjhlnqREREVNd5eXnB0dFRLvHx8TVdJRlbrswsPDwcISEhSEhIkNd1794dOTk5cHR0NOmxU1JS0KtXL1y/fh1OTk4mPRYREdG99JCMnkS0bP/s7GxotVp5fUWtVjWFyVUtoFKp4ObmVtPVICIiMikl7xbUarUGyVVtwm5BI8TFxSEkJMRgXUJCAnx8fCrcPjo6Gnv27MHy5cshSRIkScL58+fLdQsmJSXByckJ33zzDVq3bg1bW1sMHjwYBQUF2LBhA3x8fNCoUSPExMSgpKREjv/BBx8gNDQUDg4OcHNzw/Dhw3H58mUAwPnz59GrVy8AQKNGjSBJEqKjowEAer0e8fHx8PX1hUajQXBwMD799FNFrxUREVFDwZYrM1q+fDlOnTqFdu3aYd68eQAAFxcXnD9/vty2BQUFWLFiBTZv3oybN28iMjISzz77LJycnPDtt9/i7NmzGDRoEHr06IFhw4YBAIqLizF//ny0bt0aly9fxuTJkxEdHY1vv/0WXl5e+OyzzzBo0CBkZGRAq9VCo9EAAOLj4/Hhhx9izZo1aNmyJfbu3YsRI0bAxcUFYWFh5eqm0+mg0+nk5XsHFRIREVVEyQHttRmTKzNydHSESqWCra1tld2AxcXFWL16NVq0aAEAGDx4MD744AP88ccfsLe3R2BgIHr16oXdu3fLydWLL74o7+/n54cVK1agc+fOyM/Ph729PZydnQEArq6u8pgrnU6HhQsXYufOnejWrZu87/79+7F27doKk6v4+HjMnTvX6OtBREQNS01MIpqfn48zZ87Iy+fOncPRo0fh7OwMb29vo+pSGSZXtZStra2cWAFA06ZN4ePjA3t7e4N1Zd1+APDzzz8jLi4OaWlpuH79OvR6PQAgKysLgYGBFR7nzJkzKCgoQJ8+fQzWFxUVoUOHDhXuExsbi8mTJ8vLeXl58PLyqv5JEhFRg1ITLVeHDx+Wh8UAkP9+jRo1CklJSUbVpTJMroxgYWEBIYTBuuLiYkViW1tbGyxLklThurIE6tatW4iIiEBERAQ2bdoEFxcXZGVlISIiAkVFRZUeJz8/HwCwbds2eHp6GrxW2Z0Xlc0lQkREVNuEh4eX+1ttakyujODi4oJLly5BCAFJKs2kjx49et99VCqVwSB0pZw8eRJXr17FokWL5Fakw4cPlzs2AIPjBwYGQq1WIysrq8IuQCIiIqUIBboFOeaqngsPD8eff/6JxYsXY/DgwUhOTsb27dvve2uoj48PDhw4gPPnzxuMgzKWt7c3VCoV3n33Xbz88sv49ddfMX/+fINtmjdvDkmS8M033+DJJ5+ERqOBg4MDpk6dikmTJkGv1+PRRx9Fbm4uUlNTodVqMWrUKEXqR0REJAAY24hk3jaoh8OpGIwQEBCAVatW4b333kNwcDAOHjyIqVOn3nefqVOnwtLSEoGBgXLXnRJcXFyQlJSEf//73wgMDMSiRYuwdOlSg208PT0xd+5czJw5E02bNsWECRMAAPPnz8esWbMQHx+PgIAA9O3bF9u2bYOvr68idSMiImpIJGHujkiqd/Ly8uDo6IjIHaNgbadSNPZPp/0UjQcA7m7XFY8JAI7qQpPEPftnY5PE1exxMEnctHcnmSQuEZlO2fd4bm6uSSbmLIsf/OkUWNoaN2a3pECHtMHLTFZXJbBbkIiIiMyiocxzxW5BIiIiIgWx5YqIiIjMQi8kSGaeRLQmMLkiIiIisxBCgbsF68BIcSZXpJj8O2pY31F2QLt03brqjarpkqWj4jEBoNDJNL9Ooc2yTRL3h4DWJonb7bllisf8cfMUxWMSEZkKkysiIiIyi4YyoJ3JFREREZkFkysiIiIiBTWUAe2cioGIiIhIQWy5IiIiIrNoKHcLsuWqnomOjsbAgQNruhpERETllCZXkpGlps+iamy5qqPOnz8PX19fHDlyBCEhIfL65cuXg4+LJCIiqjlMruoZR0fTzOFERERkrIZytyC7BR9SUlISvL29YWtri2effRbLli2Dk5OT/HpF3XMTJ05EeHi4vKzX6xEfHw9fX19oNBoEBwfj008/lV+/fv06oqKi4OLiAo1Gg5YtWyIxMREA4OvrCwDo0KEDJEmS4957XJ1Oh5iYGLi6usLGxgaPPvooDh06JL+ekpICSZKwa9cuhIaGwtbWFt27d0dGRkal567T6ZCXl2dQiIiIqiIUKrUdk6uHcODAAYwZMwYTJkzA0aNH0atXL7z11lvVjhMfH4+NGzdizZo1+O233zBp0iSMGDECe/bsAQDMmjULJ06cwPbt25Geno7Vq1ejSZMmAICDBw8CAHbu3ImcnBx8/vnnFR5j+vTp+Oyzz7Bhwwb88ssv8Pf3R0REBK5du2aw3RtvvIFly5bh8OHDsLKywosvvnjfejs6OsrFy8ur2udORERUX7Fb8CEsX74cffv2xfTp0wEArVq1wg8//IDk5OQHjqHT6bBw4ULs3LkT3bp1AwD4+flh//79WLt2LcLCwpCVlYUOHTogNDQUAODj4yPv7+LiAgBo3Lgx3NzcKjzGrVu3sHr1aiQlJaFfv34AgPXr12PHjh3417/+hWnTpsnbLliwAGFhYQCAmTNn4qmnnkJhYSFsbGzKxY2NjcXkyZPl5by8PCZYRERUpYbSLcjk6iGkp6fj2WefNVjXrVu3aiVXZ86cQUFBAfr06WOwvqioCB06dAAAvPLKKxg0aBB++eUXPPHEExg4cCC6d+/+wMfIzMxEcXExevToIa+ztrZGly5dkJ6ebrBtUFCQ/LO7uzsA4PLly/D29i4XV61WQ61WP3A9iIiIACjTr1cH+gWZXJmIhYVFubv2iouL5Z/z8/MBANu2bYOnp6fBdmWJS79+/fD777/j22+/xY4dO/C3v/0Nr732GpYuXap4fa2t/3pAsiSV/leg1+sVPw4REVF9xzFXDyEgIAAHDhwwWPfTTz8ZLLu4uCAnJ8dg3dGjR+WfAwMDoVarkZWVBX9/f4Nydxebi4sLRo0ahQ8//BAJCQlYt24dAEClUgEASkpKKq1nixYtoFKpkJqaKq8rLi7GoUOHEBgYWL2TJiIiMpbRc1xJALsF66eYmBj06NEDS5cuxYABA/Ddd9+V6xJ8/PHHsWTJEmzcuBHdunXDhx9+iF9//VXu8nNwcMDUqVMxadIk6PV6PProo8jNzUVqaiq0Wi1GjRqF2bNno1OnTmjbti10Oh2++eYbBAQEAABcXV2h0WiQnJyMZs2awcbGptw0DHZ2dnjllVcwbdo0ODs7w9vbG4sXL0ZBQQHGjBljnotFRET0P5yhnSr1yCOPYP369Vi+fDmCg4Px/fff48033zTYJiIiArNmzcL06dPRuXNn3Lx5EyNHjjTYZv78+Zg1axbi4+MREBCAvn37Ytu2bfI0CyqVCrGxsQgKCkLPnj1haWmJzZs3AwCsrKywYsUKrF27Fh4eHhgwYECFdV20aBEGDRqEF154AR07dsSZM2fw3XffoVGjRia4MkRERJUzfnZ24wfEm4MkOJ23IpKSkjBx4kTcuHGjpqtidnl5eXB0dMQT21+CtZ1K0di//eKjaDwAQBOd8jEBODndMkncgMaXTRL3h19amySu+17lv/h+3DxF8ZhE9Jey7/Hc3FxotVqTxff5vzdhYVv+LvTq0BcU4vyLb5msrkpgtyARERGZhxJjpupAyxWTKyIiIjILjrmiaomOjm6QXYJERERkiC1XpJjiEguIEsuarkaVxDXTTICqsysySVxri8qn2zCGo1euSeLmhCn/8PCAN95RPCYApC+YZJK4RFQJTiJKREREpJyG8vgbdgsSERERKYgtV0RERGQ+daBbz1hMroiIiMgs2C1IRERERNXGlisiIiIyjwZytyBbrshAQUEBBg0aBK1WC0mSOHcXEREpSFKo1G5suSIDGzZswL59+/DDDz+gSZMmcHRUfs4iIiJqoBpIyxWTKzKQmZmJgIAAtGvXrqarQkREVCexW7CB+eyzz9C2bVuo1Wr4+Phg2bJl8mvh4eFYtmwZ9u7dC0mSEB4eXmEMnU6HvLw8g0JERFQloVCp5ZhcNSA///wzhg4diueeew7Hjx9HXFwcZs2ahaSkJADA559/jnHjxqFbt27IycnB559/XmGc+Ph4ODo6ysXLy8uMZ0FERHWWkJQptRy7BRuQt99+G3/7298wa9YsAECrVq1w4sQJLFmyBNHR0XB2doatrS1UKhXc3NwqjRMbG4vJkyfLy3l5eUywiIiI/octVw1Ieno6evToYbCuR48eOH36NEpKHvzhwGq1Glqt1qAQERFVRQhlSm3HlisiIiIyjwZytyBbrhqQgIAApKamGqxLTU1Fq1atYGlpWUO1IiIiql/YctWATJkyBZ07d8b8+fMxbNgw/Pjjj1i5ciVWrVpV01UjIqKGQIkB6XVgQDtbrhqQjh074pNPPsHmzZvRrl07zJ49G/PmzUN0dHRNV42IiBoASShTaju2XDUwgwYNwqBBgyp9PSEhwXyVISIiqoeYXBEREZF5NJAB7UyuiIiIyDwayJgrJldERERkHmy5IqqeM/9tCguNjaIxnU4p/x9KkdZE//XkOJokbErrViaJ6+JqmmdCtmxzQfGYp6zcFY8JAE90mWuSuN8fnGOSuERUNzC5IiIiIvNoIC1XnIqBiIiIzEMoVB7Ce++9Bx8fH9jY2KBr1644ePCgUadyP0yuiIiIqF7bsmULJk+ejDlz5uCXX35BcHAwIiIicPnyZZMcj8kVERERmUfZ3YLGFgB5eXkGRafTVXrYt99+G+PGjcPo0aMRGBiINWvWwNbWFv/3f/9nktNkckVERERmoeQM7V5eXnB0dJRLfHx8hccsKirCzz//jN69e8vrLCws0Lt3b/z4448mOU8mV/VMUlISnJyc5OW4uDiEhIQYbBMXF4emTZtCkiRs3brVrPUjIiJSQnZ2NnJzc+USGxtb4XZXrlxBSUkJmjZtarC+adOmuHTpkknqxrsF67mpU6fi73//u7ycnp6OuXPn4osvvsAjjzyCRo0a1WDtiIioQVHwbkGtVgutVmtsjUyCyVU9Z29vD3t7e3k5MzMTADBgwABIUu2f5ZaIiMgYTZo0gaWlJf744w+D9X/88Qfc3NxMckx2C9Zzd3cLxsXFoX///gBK+5vvTq7ef/99BAQEwMbGBm3atMGqVatqorpERESKUqlU6NSpE3bt2iWv0+v12LVrF7p162aSY7LlqgGZOnUqfHx8MHr0aOTk5MjrN23ahNmzZ2PlypXo0KEDjhw5gnHjxsHOzg6jRo0qF0en0xnclZGXZ5qZvomIqH6R8NeAdGNiVNfkyZMxatQohIaGokuXLkhISMCtW7cwevRo4ypTCSZXDYi9vb082P3uptA5c+Zg2bJliIyMBAD4+vrixIkTWLt2bYXJVXx8PObONc1jQ4iIqB6roQc3Dxs2DH/++Sdmz56NS5cuISQkBMnJyeUGuSuFyVUDd+vWLWRmZmLMmDEYN26cvP7OnTtwdKz4WXmxsbGYPHmyvJyXlwcvLy+T15WIiOq4Gnz8zYQJEzBhwgQjD/5gmFw1cPn5+QCA9evXo2vXrgavWVpaVriPWq2GWq02ed2IiIjqIiZXDVzTpk3h4eGBs2fPIioqqqarQ0RE9VkDeXAzkyvC3LlzERMTA0dHR/Tt2xc6nQ6HDx/G9evXDbr/iIiIjHH3DOvGxKjtmFwRxo4dC1tbWyxZsgTTpk2DnZ0d2rdvj4kTJ9Z01YiIiOocznNVz0RHR+PGjRvyclxcHI4ePSovDxw4EEKUT/uHDx+OI0eOQKfT4dq1a9izZw+effZZM9SYiIgaDKFQqeXYckVERETm0UDGXLHlioiIiEhBbLkiIiIis+CAdiIiIiIl1dAM7ebG5IoU49H0OqzslJ1cNM/KXdF4ACBM1Ble2FRvmsC6iidzNdaNmxqTxNUVK/+14uZ9TfGYAHA6ysUkcTtse8MkcY88tcAkcYlIWUyuiIiIyDwayIB2JldERERkFhxzRURERKSkBtJyxakYiIiIiBTElisiIiIyDwW6BetCyxWTKyIiIjIPdgsSERERUXWx5YqIiIjMo4G0XDG5IiIiIrPgVAxEldDpdNDpdPJyXl5eDdaGiIioduGYK6q2+Ph4ODo6ysXLy6umq0RERFRrMLmiaouNjUVubq5csrOza7pKRERUFwiFSi3HbkGqNrVaDbVa2Qc0ExER1RdMroiIiMgsOKCdiIiISGl1IDkyFsdcUTlJSUmQJKmmq0FERPVNAxlzxeSKyjl37hzCwsJquhpERER1ErsFqZzt27dj5cqVNV0NIiKqZzjmihqsgwcP1nQViIioPmogj79htyARERGRgthyRURERGbBbkGiarqabwtLvY2iMW0LlP8tss5XPCQAwLLINA3BBW6muXNTf9POJHHzHO8oHrOoUaHiMQFANC4ySVy93jSfBf9/vq14zDMzJisek6hS7BYkIiIioupiyxURERGZRwNpuWJyRURERGbRUMZcsVuQiIiISEFsuSIiIiLzaCDdgmy5qud8fHyQkJBQ09UgIiLiswWpbklKSoKTk1NNV4OIiKhSZWOujC21HZMrIiIiIgUxuaoHUlJSMHr0aOTm5kKSJEiShLi4OPn1goICvPjii3BwcIC3tzfWrVtnsH92djaGDh0KJycnODs7Y8CAATh//nylx9PpdMjLyzMoREREVWK3INUV3bt3R0JCArRaLXJycpCTk4OpU6fKry9btgyhoaE4cuQIXn31VbzyyivIyMgAABQXFyMiIgIODg7Yt28fUlNTYW9vj759+6KoqOLZq+Pj4+Ho6CgXLy8vs5wnERHVbewWpDpDpVLB0dERkiTBzc0Nbm5usLe3l19/8skn8eqrr8Lf3x8zZsxAkyZNsHv3bgDAli1boNfr8f7776N9+/YICAhAYmIisrKykJKSUuHxYmNjkZubK5fs7GxznCYREVGdwKkYGoCgoCD557IE7PLlywCAtLQ0nDlzBg4ODgb7FBYWIjMzs8J4arUaarXadBUmIqL6qYFMxcDkqgGwtrY2WJYkCXq9HgCQn5+PTp06YdOmTeX2c3FxMUv9iIiogWByRXWJSqVCSUlJtffr2LEjtmzZAldXV2i1WhPUjIiIqGHhmKt6wsfHB/n5+di1axeuXLmCgoKCB9ovKioKTZo0wYABA7Bv3z6cO3cOKSkpiImJwX//+18T15qIiBoSSaFS2zG5qie6d++Ol19+GcOGDYOLiwsWL178QPvZ2tpi79698Pb2RmRkJAICAjBmzBgUFhayJYuIiJTVQKZiYLdgPbJ69WqsXr3aYF1F81UdPXrUYNnNzQ0bNmwwYc2IiIgaDiZXREREZBZKzFNVF+a5YnJFRERE5sG7BYmIiIgUVgeSI2MxuSLF6G6rYAGVojG9jucrGg8AbnnZKh4TACyLTHMPi0Wxae47yfcy0TecXvnrUFxsqXhMAHByumWSuP7OV0wS97i/ddUbVVPwhHcUjwkAaSsnmSQuUV3AuwWJiIjILGr7swUXLFiA7t27w9bWFk5OTg8dh8kVERERmUctn4qhqKgIQ4YMwSuvvGJUHHYLEhERUZ2Tl5dnsKzEc2/nzp0LAEhKSjIqDluuiIiIyCyU7Bb08vKCo6OjXOLj42v25O7ClisiIiIyDwWnYsjOzjZ4koixrVZKYssVERER1TlardagVJZczZw5E5Ik3becPHlS0bqx5aoG+fj4YOLEiZg4cWJNV4WIiMjkamKG9ilTpiA6Ovq+2/j5+T18hSrA5KoGHTp0CHZ2djVdDSIiIvOogRnaXVxc4OLiYuRBq4fJVQ0y9ZsthEBJSQmsrPg2ExERVSUrKwvXrl1DVlYWSkpKcPToUQCAv78/7O3tHzgOx1w9hLi4OISEhBisS0hIgI+Pj7wcHR2NgQMHYunSpXB3d0fjxo3x2muvobi4WN7Gx8cHCQkJAIDhw4dj2LBhBjGLi4vRpEkTbNy4EQCg1+sRHx8PX19faDQaBAcH49NPP5W3T0lJgSRJ2L59Ozp16gS1Wo39+/cjLS0NvXr1goODA7RaLTp16oTDhw/L++3fvx+PPfYYNBoNvLy8EBMTg1u3Kp+5WqfTIS8vz6AQERFVqZbPczV79mx06NABc+bMQX5+Pjp06IAOHToY/M18EEyuTGj37t3IzMzE7t27sWHDBiQlJVU6d0ZUVBS+/vpr5Of/9biX7777DgUFBXj22WcBAPHx8di4cSPWrFmD3377DZMmTcKIESOwZ88eg1gzZ87EokWLkJ6ejqCgIERFRaFZs2Y4dOgQfv75Z8ycORPW1qWP0cjMzETfvn0xaNAgHDt2DFu2bMH+/fsxYcKESs8rPj7e4PZXLy8vI68UERE1BLV9hvakpCQIIcqV8PDwasVhf5EJNWrUCCtXroSlpSXatGmDp556Crt27cK4cePKbRsREQE7Ozt88cUXeOGFFwAAH330EZ555hk4ODhAp9Nh4cKF2LlzJ7p16wagdADe/v37sXbtWoSFhcmx5s2bhz59+sjLWVlZmDZtGtq0aQMAaNmypfxafHw8oqKi5EH1LVu2xIoVKxAWFobVq1fDxsamXF1jY2MxefJkeTkvL48JFhERVa0GxlzVBCZXJtS2bVtYWv71wFl3d3ccP368wm2trKwwdOhQbNq0CS+88AJu3bqFL7/8Eps3bwYAnDlzBgUFBQZJE1A6VX+HDh0M1oWGhhosT548GWPHjsUHH3yA3r17Y8iQIWjRogUAIC0tDceOHcOmTZvk7YUQ0Ov1OHfuHAICAsrVVYlZcImIiOorJlcPwcLCAkIYps53j6UqU9b1VkaSJOj1+krjRkVFISwsDJcvX8aOHTug0WjQt29fAJC7C7dt2wZPT0+D/e5NdO69AzEuLg7Dhw/Htm3bsH37dsyZMwebN2/Gs88+i/z8fIwfPx4xMTHl6uPt7V1pXYmIiKpLEgKSMK7pydj9zYHJ1UNwcXHBpUuXIISAJEkAIN9RYIzu3bvDy8sLW7Zswfbt2zFkyBA5QQsMDIRarUZWVpZBF+CDatWqFVq1aoVJkybh+eefR2JiIp599ll07NgRJ06cgL+/v9H1JyIiui92C1JlwsPD8eeff2Lx4sUYPHgwkpOTsX37doNp+B/W8OHDsWbNGpw6dQq7d++W1zs4OGDq1KmYNGkS9Ho9Hn30UeTm5iI1NRVarRajRo2qMN7t27cxbdo0DB48GL6+vvjvf/+LQ4cOYdCgQQCAGTNm4JFHHsGECRMwduxY2NnZ4cSJE9ixYwdWrlxp9PkQERE1NLxb8CEEBARg1apVeO+99xAcHIyDBw9i6tSpisSOiorCiRMn4OnpiR49ehi8Nn/+fMyaNQvx8fEICAhA3759sW3bNvj6+lYaz9LSElevXsXIkSPRqlUrDB06FP369ZOf/B0UFIQ9e/bg1KlTeOyxx9ChQwfMnj0bHh4eipwPERFRmdp+t6BSJHHv4CGiasrLyyudkmHdbFhoyt9daIyWK8uPZTPWLS9bxWMCwB0bySRxCxub5n+gfC/T/OqXOJQoHtNSW6R4TADQ2t82SVx/5ysmiXs8R/l/emx2OygeEwDSVk4ySVwyjbLv8dzcXEV6YSqL32H4AliqjPs7UVJUiCMfvWGyuiqBLVdERERECuKYKyIiIjKLmnhwc01gckVERETmwbsFiarHyvoOLFR3FI15212jaDwAEJamGRulczJNL7veuuptHob1TdNcB0lvWfVG1XQHKsVjAsCNa6aZDNfN46xJ4v7p9OAPjn1Q2T1M82egw7Y3TBL3yFMLTBKXzKOhtFxxzBURERGRgthyRURERObBbkEiIiIiZdWFbj1jsVuQiIiISEFsuSIiIiLzEKK0GBujlmNyRURERGbBuwWJiIiIqNrYckVERETmwbsFiYiIiJQj6UuLsTFqOyZXVG06nQ46nU5ezsvLq8HaEBER1S4cc0XVFh8fD0dHR7l4eXnVdJWIiKguEAqVWo7JFVVbbGwscnNz5ZKdnV3TVSIiojqg7G5BY0ttx25Bqja1Wg212jQPvCUionqsgcxzxZYrIiIiIgWx5YqIiIjMgpOIUoOVlJQESZJquhpERFTfcEA7NVTnzp1DWFhYTVeDiIioTmK3IJWzfft2rFy5sqarQURE9UxD6RZkckXlHDx4sKarQERE9RHvFiQiIiKi6mLLFREREZkFuwWJqqk41wYWRTaKxrS6VaJoPACwuGOa30wrO9PcYWlZZJKwkPSmuiNU+bglGtPUVXXDNI33P17yNUncxra3FI/p0eSG4jFNqfW8d0wSN2P2JJPEpXsocbdfHUiu2C1IREREpCC2XBEREZFZsFuQiIiISEl6UVqMjVHLMbkiIiIi8+CYKyIiIiKqLiZXVCFJkrB169aargYREdUjEv4ad/XQpaZP4gGwW5CIiIjMgzO0ExEREVF1Mbmqw65evYrnn38enp6esLW1Rfv27fHxxx8bbBMeHo6YmBhMnz4dzs7OcHNzQ1xcnME2p0+fRs+ePWFjY4PAwEDs2LHDjGdBREQNhdFdggpM5WAO7BaswwoLC9GpUyfMmDEDWq0W27ZtwwsvvIAWLVqgS5cu8nYbNmzA5MmTceDAAfz444+Ijo5Gjx490KdPH+j1ekRGRqJp06Y4cOAAcnNzMXHixPseV6fTQafTyct5eXmmOkUiIqpPeLcg1Xaenp6YOnUqQkJC4Ofnh7///e/o27cvPvnkE4PtgoKCMGfOHLRs2RIjR45EaGgodu3aBQDYuXMnTp48iY0bNyI4OBg9e/bEwoUL73vc+Ph4ODo6ysXLy8tk50hERFTXMLmqw0pKSjB//ny0b98ezs7OsLe3x3fffYesrCyD7YKCggyW3d3dcfnyZQBAeno6vLy84OHhIb/erVu3+x43NjYWubm5csnOzlbojIiIqD6ThFCk1HbsFqzDlixZguXLlyMhIQHt27eHnZ0dJk6ciKIiwyf9WltbGyxLkgS9Xv/Qx1Wr1VCr1Q+9PxERNVD6/xVjY9RyTK7qsNTUVAwYMAAjRowAAOj1epw6dQqBgYEPHCMgIADZ2dnIycmBu7s7AOCnn34ySX2JiIgaAnYL1mEtW7bEjh078MMPPyA9PR3jx4/HH3/8Ua0YvXv3RqtWrTBq1CikpaVh3759eOONN0xUYyIiasgaSrcgk6s67M0330THjh0RERGB8PBwuLm5YeDAgdWKYWFhgS+++AK3b99Gly5dMHbsWCxYsMA0FSYiooZNKFRqOXYL1mHOzs5VPqImJSWl3Lp792nVqhX27dtnsE7Ugf8MiIiojuEM7URERERUXWy5IiIiIrNQYoZ1ztBOREREVIbdgkREREQNw/nz5zFmzBj4+vpCo9GgRYsWmDNnTrm5Ix8EW65IMaprlrCwsVQ0pubsZUXjAcCdJg6KxwQAu7RrJomrb6w1SdxCd3uTxM31Vf5rxeF30/wfeEdjkrC4budskrhXbRopHlPlUqB4TAAI8bxgkrgXWzqZJG7ree8oHjNj9iTFY9Z1kr60GBvDFE6ePAm9Xo+1a9fC398fv/76K8aNG4dbt25h6dKl1YrF5IqIiIjMQ8Fuwby8PIPVxj49pG/fvujbt6+87Ofnh4yMDKxevbrayRW7BYmIiKjO8fLygqOjo1zi4+MVP0Zubi6cnavfEs2WKyIiIjIPJSYB/d/+2dnZ0Gr/Gjah9DNvz5w5g3fffbfarVYAW66IiIjITJR8/I1WqzUolSVXM2fOhCRJ9y0nT5402OfChQvo27cvhgwZgnHjxlX7PNlyRURERPXWlClTEB0dfd9t/Pz85J8vXryIXr16oXv37li3bt1DHZPJVR0SHR2NGzdu3PeRNz4+Ppg4cSImTpxotnoRERE9kBqY58rFxQUuLi4PtO2FCxfQq1cvdOrUCYmJibCweLgOPiZXtdD58+fh6+uLI0eOICQkpFr7Hjp0CHZ2dqapGBERkTEEAGOnUjDRHKIXLlxAeHg4mjdvjqVLl+LPP/+UX3Nzc6tWLCZXZlZcXAxra2uTxX/Q7JyIiMjc7h4zZUwMU9ixYwfOnDmDM2fOoFmzZgaviWoes0EPaN+/fz8ee+wxaDQaeHl5ISYmBrdu3QIA/OMf/0DXrl3L7RMcHIx58+bJy++//z4CAgJgY2ODNm3aYNWqVfJr58+fhyRJ2LJlC8LCwmBjY4N169ZBq9Xi008/NYi7detW2NnZ4ebNm/D19QUAdOjQAZIkITw83GDbpUuXwt3dHY0bN8Zrr72G4uJi+TUfHx8kJCTIy5Ik4f3338ezzz4LW1tbtGzZEl999ZVBvK+++gotW7aEjY0NevXqhQ0bNkCSJNy4caPC66bT6ZCXl2dQiIiI6rLo6GgIISos1dVgk6vMzEz07dsXgwYNwrFjx7Blyxbs378fEyZMAABERUXh4MGDyMzMlPf57bffcOzYMQwfPhwAsGnTJsyePRsLFixAeno6Fi5ciFmzZmHDhg0Gx5o5cyZef/11pKenIzIyEs899xwSExMNtklMTMTgwYPh4OCAgwcPAgB27tyJnJwcfP755/J2u3fvRmZmJnbv3o0NGzYgKSkJSUlJ9z3XuXPnYujQoTh27BiefPJJREVF4dq10tnEz507h8GDB2PgwIFIS0vD+PHj8cYbb9w3Xnx8vMHcIl5eXvfdnoiICMD/pmIQRpaaPomqNdjkKj4+HlFRUZg4cSJatmyJ7t27Y8WKFdi4cSMKCwvRtm1bBAcH46OPPpL32bRpE7p27Qp/f38AwJw5c7Bs2TJERkbC19cXkZGRmDRpEtauXWtwrIkTJ8rbuLu7Y+zYsfjuu++Qk5MDALh8+TK+/fZbvPjiiwD+6tpr3Lgx3NzcDCYwa9SoEVauXIk2bdrg6aefxlNPPYVdu3bd91yjo6Px/PPPw9/fHwsXLkR+fr6cwK1duxatW7fGkiVL0Lp1azz33HNV3lURGxuL3NxcuWRnZz/AFSciogbP6MRKgQHxZtBgk6u0tDQkJSXB3t5eLhEREdDr9Th37hyA0tarsuRKCIGPP/4YUVFRAIBbt24hMzMTY8aMMYjx1ltvGbR2AUBoaKjBcpcuXdC2bVu5hevDDz9E8+bN0bNnzyrr3bZtW1ha/vX8Pnd3d1y+fP/n7wUFBck/29nZQavVyvtkZGSgc+fO5ep3P2q1utz8IkRERFSqwQ5oz8/Px/jx4xETE1PuNW9vbwDA888/jxkzZuCXX37B7du3kZ2djWHDhsn7A8D69evLjc26O/kBUOHde2PHjsV7772HmTNnIjExEaNHj4YkSVXW+97B8JIkQa+//60XD7MPERGR4vQAqv5TV3WMWq7BJlcdO3bEiRMn5C6+ijRr1gxhYWHYtGkTbt++jT59+sDV1RUA0LRpU3h4eODs2bNya1Z1jBgxAtOnT8eKFStw4sQJjBo1Sn5NpVIBAEpKSqodt7pat26Nb7/91mDdoUOHTH5cIiJqeGrz3YJKarDdgjNmzMAPP/yACRMm4OjRozh9+jS+/PJLeUB7maioKGzevBn//ve/yyVRc+fORXx8PFasWIFTp07h+PHjSExMxNtvv13l8Rs1aoTIyEhMmzYNTzzxhMFtn66urtBoNEhOTsYff/yB3NxcZU66AuPHj8fJkycxY8YMnDp1Cp988ok8QP5BWtKIiIjIUINNroKCgrBnzx6cOnUKjz32GDp06IDZs2fDw8PDYLvBgwfj6tWrKCgowMCBAw1eGzt2LN5//30kJiaiffv2CAsLQ1JSkjyVQlXGjBmDoqIieSB7GSsrK6xYsQJr166Fh4cHBgwYYNS53o+vry8+/fRTfP755wgKCsLq1avluwWVfggmERE1cA1kQLskHmYCB1LEBx98gEmTJuHixYtyV2BtsGDBAqxZs+aB7wLMy8uDo6Mj/GYvgIWNjaJ18f9XjqLxAOBOEwfFYwKA1cVrJomrb2yaGwYK3e1NEjfXV/nRBlYFiocEANzRmCZuXkvTfK3qbZSPq3IxzcUN8bxgkriHzzc3SVzr08p/GDJmT1I8pqmUfY/n5uaa5Calsvh/C5wKK0vj/nG/U6LDrhNLTVZXJTTYMVc1qaCgADk5OVi0aBHGjx9f44nVqlWr0LlzZzRu3BipqalYsmRJue5RIiIiejANtluwJi1evBht2rSBm5sbYmNja7o6OH36NAYMGIDAwEDMnz8fU6ZMQVxcXE1Xi4iI6psG0i3IlqsaEBcXV6uSl3feeQfvvPNOTVeDiIjqO07FQFQ9xVo9LDQKf+pNMB2FRUFx1Rs9hCIf0zw0+46daX5NdY6WVW/0ECQTfPEVNjHNnatWt0w0NqpJkUniijvKdzbcKTbN56uHU2bVGz0Er4DrJol71K1Z1RtVU+A/TPNP64mFdWcs1704FQMRERERVRtbroiIiMg8lBgzVQdarphcERERkXnoBSAZmRzpa39yxW5BIiIiIgWx5YqIiIjMo4F0C7LlqoFKSkqCk5NTTVeDiIgaFCXmuGJyRbXUsGHDcOrUqZquBhERUb3DbsEGSqPRQKMx0YPViIiIKsJuQarP7u0WTEtLQ69eveDg4ACtVotOnTrh8OHDNVdBIiKqf/RCmVLLseWKAABRUVHo0KEDVq9eDUtLSxw9ehTW1tYVbqvT6aDT6eTlvLw8c1WTiIio1mNyRQCArKwsTJs2DW3atAEAtGzZstJt4+PjMXfuXHNVjYiI6guhLy3Gxqjl2C1IAIDJkydj7Nix6N27NxYtWoTMzMqfCxYbG4vc3Fy5ZGdnm7GmRERUZxl7p6ASY7bMgMkVAQDi4uLw22+/4amnnsJ//vMfBAYG4osvvqhwW7VaDa1Wa1CIiIiq1EDGXDG5IlmrVq0wadIkfP/994iMjERiYmJNV4mIiKjOYXJFuH37NiZMmICUlBT8/vvvSE1NxaFDhxAQEFDTVSMiovqkgXQLckA7wdLSElevXsXIkSPxxx9/oEmTJoiMjOSgdSIiUpaAAvNcKVITk2Jy1UBFR0cjOjoaAKBSqfDxxx/XbIWIiIjqCSZXREREZB4NZIZ2JldERERkHno9ACPnqdJznisiIiKiBoUtV0RERGQe7BYkqh7rGxawKFS2MfTO78rP/m7ZqJHiMQFAdcPWJHGtGptmklbrfLVJ4lrpVIrHtPhv7f8yNSCZ5toWaSXFY5bYKB4SAPB2YR+TxNU4FpokrqPdbcVjFjQrUTwmAHR86W3FY5YUmea6ltNAkit2CxIREREpiC1XREREZB56AaMnqqoDj79hckVERERmIYQeQhh3t5+x+5sDkysiIiIyD6HAg5c55oqIiIioYWHLFREREZmHUGDMVR1ouWJyRUREROah1wOSkWOm6sCYK3YLEhERESmILVdERERkHuwWJKqYTqeDTqeTl/Py8mqwNkREVFcIvR7CyG7BujAVA7sFqdri4+Ph6OgoFy8vr5quEhERUa3B5IqqLTY2Frm5uXLJzlb++X9ERFQPlT1b0NhSy7FbkKpNrVZDrTbNg2mJiKge0wtAqv9jrthyRURERKQgtlwRERGReQgBwNh5rmp/yxWTKyIiIjILoRcQRnYLCiZXRERERP8j9DC+5YpTMRARERE1KGy5IiIiIrNgtyARERGRkhpItyCTKzJa2X8Rel2h4rHviGLFYwpRpHhMAJD0liaJqy/RVb3RQyi5Y5r//u4UK//FZ1Fc+/9TvVtJkWm+Wkt0kvIxFY9YSn/bNL9nJdbKf88AQAmU/z3T3zZRXYuUH9FTUlRaV1O3Ct1BsdGPFrwD5f8uKE0SdaF9jWq1//73v3wEDhFRPZCdnY1mzZopHrewsBC+vr64dOmSIvHc3Nxw7tw52NjYKBJPaUyuyGh6vR4XL16Eg4MDJOn+/1nn5eXBy8sL2dnZ0Gq1itXBFHHrUl3rWty6VNe6Frcu1bWuxa1Lda1uXCEEbt68CQ8PD1hYmOZet8LCQhQVKdOiqVKpam1iBbBbkBRgYWFR7f90tFqtol8ipoxbl+pa1+LWpbrWtbh1qa51LW5dqmt14jo6Oip+7LvZ2NjU6oRISZyKgYiIiEhBTK6IiIiIFMTkisxKrVZjzpw5UKvVtT5uXaprXYtbl+pa1+LWpbrWtbh1qa6mjEtV44B2IiIiIgWx5YqIiIhIQUyuiIiIiBTE5IqIiIhIQUyuqM5JSUmBJEm4ceNGTVelVvDx8UFCQkKFr0VHR2PgwIH33b+uXM8HOZfKFBQUYNCgQdBqtZWea1JSEpycnOTluLg4hISEGGwTFxeHpk2bQpIkbN269aHqYkr3+yyYct/6Qun3tarPbFJSEiwsLBr8da+PmFzVEdHR0YiLi5OXK/riN4Wa+MN78uRJPPLII7CxsUFISAjCw8MxceJEsx3/fs6fP1/lLPQVufcP98NQIkZF17J79+7Iyckx+QSCD/pZKrvGR48eNVi/fPlyJCUlPdSxN2zYgH379uGHH3544HOdOnUqdu3aJS+np6dj7ty5WLt2LXJyctCvX7+HqosSlPgs3OvQoUN46aWXFI3ZUFT2ma3KsGHD8Ouvv/K610NMrui+zPWH925z5syBnZ0dMjIyDP64VeRhv9SqYqq491NcXDMPI1WpVHBzc3uopNGcHB0dHzqhyMzMREBAANq1a/fA52pvb4/GjRsbxACAAQMGwM3Nrd7d3u7i4gJbW1uTxRdC4M6dOyaLbyqm/L3UaDQIDAw06XWnmsHkqha7fv068vPza7QONfGHNzMzE48++iiaN2+OKVOmYM+ePVi+fDkkSYIkSTh//jwA4Oeff8YzzzwDAIiKisJTTz0FT09P2Nraon379pgyZQo6duwIGxsb+Pn5wcfHBxMmTMD06dPh7OwMNzc3g9ZAIQTi4uLg7e2N1q1bAwD++c9/AgBWrlyJdu3aydtu3boVkiRhzZo1yMrKAgD07t0bb775przNl19+iY4dO0KlUmH06NHIzc2VzyEuLg6SJGHZsmXw8vKCJEmwtbVFv379cPr06XLXJCUlpVyM8PBwufWyoKAAL774ItRqNaysrLBu3TqD/bOzs+Hj41Phtby3RWns2LGwsLCAWq2Gvb09rK2toVKpUFBQgA0bNsjrYmJiUFJSAgCYOHEiAgICEBoaCgcHBzRt2hTBwcHw9vaGRqNBQEAAevXqBQBo1KgRJEmCn58fXFxcoNFo0KRJEzRp0gQajQa+vr4AgA4dOsjnCZTvYtHpdIiJiYGrqyusra1ha2sLlUoFHx8fLFu2TD6v4OBgLFu2DHv37oUkSejSpcsDfQ7vbh2Oi4tD//79AZQ+7unu34f3338fAQEBsLGxQZs2bbBq1aoHiv+wKvosuLm5ya8XFBSga9eusLCwgLe3N9atWydfu6VLl8LV1RVqtRpqtRqNGjXCgAEDcP78eYNuweHDh2PYsGEGxy0uLkaTJk2wceNGAKXPFI2Pj4evry80Gg2Cg4Px6aefGtRTkiRs374dnTp1glqtxv79+5GWloZevXrBwcEBWq0WnTp1wuHDh+X99u/fj8ceewwajQZeXl6IiYnBrVu35Ne6desGKysrWFpawsrKCm3btsXHH3+Mf/zjH+jatSuA0hbamJgYTJ8+HVZWVrC3t5d/18veL7VaDVtbW1hbWyMwMBA7duwAADz77LPYsmULwsLCYGNjg3Xr1kGr1RqcG1D6HWBnZ4ebN29W+pkts3TpUri7u6Nx48Z47bXX5ITt3m7BtLQ0SJIEGxsbWFtbw8LCAl5eXvjqq68M4n311Vdo2bIlbGxs0KtXL2zYsKFOdO03KIJqleLiYvHNN9+IwYMHC7VaLY4ePSqEEGLUqFFizpw58nZz5swRwcHBYs2aNaJZs2ZCo9GIIUOGiBs3bhjEW79+vWjTpo1Qq9WidevW4r333jN4PTU1VQQHBwu1Wi06deokvvjiCwFAHDlyRAghxO7duwUAcf36dSGEEImJicLR0VEkJyeLNm3aCDs7OxERESEuXrz4QOdXUlIi5s6dKzw9PYVKpRLBwcFi+/bt8usADMqMGTNEt27dxLhx40ROTo7IyckRO3fuFABE165dy23fqVMnkZmZKV5//XUBQDRu3FioVCrRrFkzYWVlJdRqtYiLixORkZGiWbNmQpIk8f333wudTid8fHyEtbW1+PbbbyuMK0mSOHz4sAAgJk6cKJo0aSKGDRsmwsPDRWBgoLC2thabN28WQgixd+9eodVqRVJSkkhPTxevvPKKkCRJTJkyReTk5IibN28KAEKlUgl3d3exefNmsX37dhERESH8/f1FUVGRwXXT6XQiISFBaLVa+TrExsaK4OBg0bx5c+Hs7Czee+898Y9//EM4OTkJCwsLcfLkSTFq1CjRv39/ERAQIEaMGCGCg4PFkCFDxLPPPitatGghCgoKDN7jn376SUiSJCwsLET37t3FtGnThL29vZAkSTzxxBNi6NChYsCAAaJr165CpVLJ5/v666+L1q1bi2+//VZkZmaK8ePHC41GI0JDQ0VmZqb417/+JaytrQUAkZGRIUaPHi3at28vDh06JKZMmSK8vLxEXFycyMzMFLNmzRIAxLJly0ROTo64evWq/DswYMAA+ZrExMQIDw8PsXz5cmFhYSFCQkKEVqsV7777rtBoNGLGjBnye/f000+LkJAQ0bVrV9G5c+cKP5tln+17f8eEEOLmzZsiMTFRAJCvvxBCfPjhh8Ld3V189tln4uzZs+Kzzz4Tzs7OIikp6YF+Hx7GvZ+FKVOmiPbt2wshhPxZGDRokPD09BTx8fHCwsJCDBw4UGi1WvHSSy8JPz8/0bt3b2FjYyPmzp0rhg8fLlq3bi2aN28u3nnnHSGEEN98843QaDTi5s2b8nG//vprodFoRF5enhBCiLfeeku0adNGJCcni8zMTJGYmCjUarVISUkRQvz13REUFCS+//57cebMGXH16lXRtm1bMWLECJGeni5OnTolPvnkE/l77syZM8LOzk6888474tSpUyI1NVV06NBBREdHy6/FxcWJ6dOni8TERNG2bVvRtWtXYWlpKT7++GMBQJw5c0aEhYUJrVYrXn31VQFALF68WEiSJGbMmCHc3d3Fv//9b9GqVSvRvn174ejoKGJjY0WHDh3k33cfHx/5Pb148aIYN26cePLJJw3eh2eeeUaMHDlSCCHEwYMHBQCxc+fOcp9ZrVYrXn75ZZGeni6+/vprYWtrK9atWyeEKP3MSZIkX/e2bdsKAKJp06Zi2bJlYsWKFeL5558X9vb2csyzZ88Ka2trMXXqVHHy5Enx8ccfC09PT4Pvaap5TK5qiWPHjonJkyeLpk2bCmdnZ/HKK6+IH374QX69ouTKzs5OPP744+LIkSNiz549wt/fXwwfPlzepqov/tzcXOHs7CxGjBghfvvtN/Htt9+KVq1aVZlcWVtbi969e4tDhw6Jn3/+WQQEBBgc937efvttodVqxccffyxOnjwppk+fLqytrcWpU6eEEELk5OSItm3bGiQhYWFh4vXXX5djlNVp586d8pfaggULBABx4cIFIUTpl5SlpaV45pln5HO3sLAQlpaWQojSP5Z+fn6iadOmYsaMGWLq1KmiUaNGcmJz75fllStXROPGjcV7770nAIiQkBARHx8v3NzcxOXLl8Xrr78uJyX9+vUT7dq1E/PmzZPrnJiYKGxtbYW7u7u8ruyLPDU1VV535coVodFoxCeffFLu2lX2x7958+ZixIgRQggh3nnnHdG8eXPh6uoqVq9eLUaNGiU6duwoWrduLfR6vXwtdTqd0Gg04rvvvjN4j59//nkRFBQk/5ESQohhw4YJlUolbG1txc2bN+UkJyIiQowfP14IUZpchYWFCSGEKCwsFLa2tuJf//qXACD/gX7yySfl4/Tv31+MHj1a3vbuz/q5c+cEANG3b1+D8787ucrPzxfW1tZi06ZNYvjw4aJPnz6iqKhIeHh4iMWLF4tp06aJ5s2by+9hWf22bdsmAIjbt28/8PUtU/aPx91atGghPvroI4N18+fPF926dSsXX0l31/XuepZ9Fso+B3q9Xri6uopHHnlENG/eXGzYsEH+LAwZMkQMGzZM/iw0bdpU/iNfXFwsmjRpIjZu3Cgf8/nnnxfDhg0TQogK3zchhBgzZox4/vnnhRB//Z5u3brVYBsHB4dKk88xY8aIl156yWDdvn37hIWFhYiOjq70tb59+4opU6aI4OBgMW/ePBEWFiYeffRRERsbK7p27SqEEKJz587CyclJfPTRR+K7774TVlZW4sKFC/L7tX37dvl3MiEhweA4Bw4cEJaWlvI/kX/88YewsrKSE8myz2zZ92aZUaNGiebNm4s7d+7I68quuxDlkysHBwcBQLz55pvy9vn5+QKA/E/ojBkzRLt27QyO88YbbzC5qmXYLViDrl69iuXLl6Njx44IDQ3F2bNnsWrVKuTk5GDVqlXo1q3bffcvLCzExo0bERISgp49e+Ldd9/F5s2bcenSJQClY5eWLVuGyMhI+Pr6IjIyEpMmTcLatWsBAB999BEkScL69esRGBiIfv36Ydq0aVXWu7i4GGvWrEFoaCg6duyICRMmVDk2qszSpUsxY8YMPPfcc2jdujX++c9/IiQkRG4Wd3Nzk5vx3dzcYG9vX2msoKAguLi4ACi90wkAHn/8cTg7O+O3335DSUkJtm3bhvbt22PkyJHQ6/UoKSlBQUEB7O3t8eGHH+Ly5ctITk5GQkIC1q9fD51OBz8/PyxbtgxA6TgfNzc3NG7cGD179sRPP/0EADhx4gReffVV6HQ6XL16Fa6urujWrRt+/fVXBAUF4cSJE5g9e7bcXTV+/HgUFBQgJycHBQUF8jlYWlrKXRkA0LhxY7Ru3Rrp6ekPdD3vvhZ3c3Nzw+XLlwEAeXl5OHPmDBwcHLBv3z689957cHZ2RmFhoTyOqEx6ejr8/Pxga2uLFi1aAAC6desGSZLg4+Nj8H40bdpUPgYA3Lx5E/3794ePjw8KCgowZswYAICrqyvs7e3x/fffy9u+8sor2Lx5M4KDg1FQUIDHH38c9vb2sLe3R2BgIIDSrszKZGZmori4GD169EB6ejp69OgBa2trdOnSRV6+cOFCuWvj7u4OAAb1fli3bt1CZmYmxowZI9fd3t4eb731Vrnrak53n29Zl2FhYSHatm2L48ePy5+FL774Av/+97/lz8LdY4usrKwwdOhQbNq0CUDpuX755ZeIiooCAJw5cwYFBQXo06ePwblv3Lix3LmHhoYaLE+ePBljx45F7969sWjRIoPt09LSkJSUZBAzIiICer0ehw8fll9Tq9WwtLTEY489Br1ejx07diArKwtRUVH46KOPAADt27fHxx9/LNfZxcUFN27cwJgxY9C/f3+UlJSgVatW8vt19/ftvXXu0qUL2rZtiw0bNgAAPvzwQzRv3hw9e/as8v1o27YtLC0t5WV3d/dKP3+TJ08GUDqkoOza2NnZQavVyvtkZGSgc+fO5epHtQuTqxr07rvvYuLEibC3t8eZM2fwxRdfIDIyEiqV6oH29/b2hqenp7zcrVs36PV6ZGRkPNAXf0ZGBoKCgmBjYyPHeJBf0rv/8AL3/7K4W15eHi5evIgePXoYrC/7A1ld1tbW8s/ffPMNAGD8+PH49ttvAZT+YRFCQK/XQ6/XQ5IkqNVq+Xy7desGf39/pKWlYcqUKRg0aBAyMjKwatUqeZsxY8bIf3TCw8Nx4MABAKVjK7RaLXr27ImUlBTs2bMHYWFhCAgIwKJFi6BSqeS7ySwtLTFv3jw4ODjg9OnTBtfbGBYWFhD/e3pV2bUoq6skSdDr9QCAO3fuoFOnTjh69ChCQ0MRFRWFo0eP4tSpUxg+fHiV1/bedWXHvfsYBQUFOHbsGLRarTy2ZcmSJQCAzZs34+jRowZ3+vXr1w+///47hg4dCqB0/M5zzz2Ho0ePyu/f0qVLH/raVHYuZWOlyuptjLLxkOvXr8fRo0fl8uuvv8pJuDnc/TkASs/37kSp7PfA2toa+fn58mchKioKoaGh8mfBzs7OIG5UVBR27dqFy5cvY+vWrdBoNOjbty+Av85927ZtBud+4sSJcmOT7o0bFxeH3377DU899RT+85//IDAwEF988YUcd/z48QYx09LScPr0aRQXF2P8+PF4+eWXYWtri8WLF+Orr77Czp078cQTT6CoqAjPP/88MjIycPPmTVy5cgXZ2dny2LGy8YHr16/HtGnT4OnpWen7dW+dgdKxiGWf4cTERIwePfqBxqLe+7t09+/Nvcp+d0JDQw2uzf32odqJyVUNeumllzB//nxcunQJbdu2xejRo/Gf//yn1n/xV/RlIUz4iEqVSiV/MVYmIyMDQOlg1LLBpTY2Nnj88cdx7NgxHDt2DJ07d8bzzz8PC4vSj71er8e1a9cgSRLOnDkDoPTunf79+8tfcseOHcPx48cBAGFhYfJg87IBq+Hh4di5cydSU1MRHh6O7OxsLFq0CJIk4fvvv8fQoUPxxRdfwMvLC0II+Pv7y8cHSr/wyxI2oLQ1MyMjQ269ud91cHFxwaVLlwyufUV3Nzo6OuL06dNwdXWFo6MjHBwc4O/vD39//3J3gQYEBODs2bMG6+79vLi4uCAnJ8dg3YEDB3Dnzh0sWrQIw4cPh1qtlq+pt7c3/P395fel7BxcXFwwbdo0qNVqDB8+HJ988gn8/f3RqlUr+fXKtGjRAiqVCqmpqQgICEBqaiqKi4tx6NAhBAYGIjU1Fc2aNat0fyU0bdoUHh4eOHv2rHw9y0rZuZrK3Z+FB/0cAEDHjh3lz4KTkxM0Go1c57s/l0DpncJeXl7YsmULNm3ahCFDhsi/+4GBgVCr1cjKyip37l5eXlXWv1WrVpg0aRK+//57REZGIjExUa7fiRMnysX09/dHp06dcOLECWRkZCAyMhJTpkxB//790atXL/mz1qxZM4SFheHy5cs4efIk+vTpA1dXVwCl3wcajQZnz55Fz549cenSJdjZ2cnvV1XfiyNGjMDvv/+OFStW4MSJExg1apTB+wGgyu+pB/XMM8+UuzZlWrdubXADAFA6jQbVLkyuapCHhwfefPNNnDp1CsnJyVCpVIiMjETz5s0xc+ZM/Pbbb/fdPysrCxcvXpSXf/rpJ1hYWKB169YP9MXfunVrHD9+HDqdTo5hyl9SrVYLDw8PpKamGqxPTU2tMJko4+PjgwMHDuD8+fO4cuWKQfJZ9qXWtGlTAKV3EF67dg1qtRq3b99GdnY2dDodiouLcfPmTYNrumTJEty8eRN9+/ZFcnIyxowZg3/961/49ddf5eRBrVajefPmAEq7W8qSkbuTq61bt6KwsBALFiyAj48Ptm3bhvHjx0OSJLRq1UruisrPz0dUVBSuXLkidw126dIF48aNk++iGjFiBDw9PTFgwIAKr0N+fj527dqFK1euoGvXrvjzzz+Rl5eHK1eu4L333sP27dvL7efl5YUmTZpgwIABsLGxwd69e7F582aMGzdOvtOxTExMjPyZOH36NFauXInk5GSDbR5//HEcPnwYmZmZyM/Px5w5c3Du3DlIkoR3330Xf/75J5555hmsX78eQGn33i+//CJ3HX/zzTeYMmUKNm/ejD/++AOjRo3Cpk2b4OLigszMTPz3v/+FtbU1Fi5ciD/++AO5ubnlzsnOzg6vvPIKpk2bhm7dumHXrl3o0qULbt68CVtbW6xcubLc3W6mMHfuXMTHx2PFihU4deoUjh8/jsTERLz99tsmPe7dn4Xg4GD8+eefWLx4MYqLi7F///4KPwdAaWtU2WfhwoULKCwsREpKisGdn3cbPnw41qxZgx07dsjdawDg4OCAqVOnYtKkSdiwYQMyMzPxyy+/4N1335W7zipy+/ZtTJgwASkpKfj999+RmpqKQ4cOISAgAAAwY8YM/PDDD5gwYQKOHj2K06dP48svv8SECRPk17Kzs7Ft2zZs2bIFK1euRPv27fHHH38YnOPly5dx5swZgzoDQEhICOLj45Geno7mzZsjMjISc+fOxYQJE/DGG2/c95o3atQIkZGRmDZtGp544gmD5N3V1RUajQbJycmVfmarUnZtgNJu63uvTZnx
gitextract_rxv1jkc6/ ├── LICENSE ├── README.md ├── llama3-from-scratch.ipynb └── requirements.txt
Condensed preview — 4 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (268K chars).
[
{
"path": "LICENSE",
"chars": 1072,
"preview": "MIT License\n\nCopyright (c) 2024 Nishant Aklecha\n\nPermission is hereby granted, free of charge, to any person obtaining a"
},
{
"path": "README.md",
"chars": 31376,
"preview": "# llama3 implemented from scratch\nin this file, i implemented llama3 from scratch, one tensor and matrix multiplication "
},
{
"path": "llama3-from-scratch.ipynb",
"chars": 227889,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# llama3 implemented from scratch\\n"
},
{
"path": "requirements.txt",
"chars": 48,
"preview": "sentencepiece\ntiktoken\ntorch\nblobfile\nmatplotlib"
}
]
About this extraction
This page contains the full source code of the naklecha/llama3-from-scratch GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 4 files (254.3 KB), approximately 143.2k tokens. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.