Repository: sanjeevanahilan/nanoChatGPT Branch: master Commit: a77c85c2cdfc Files: 36 Total size: 549.4 KB Directory structure: gitextract_q6tdf_kr/ ├── .gitattributes ├── LICENSE ├── README.md ├── bench.py ├── chatgpt_dev_teaching.ipynb ├── config/ │ ├── config.yaml │ ├── config_reward.yaml │ ├── config_rl.yaml │ ├── eval_gpt2.py │ ├── eval_gpt2_large.py │ ├── eval_gpt2_medium.py │ ├── eval_gpt2_xl.py │ ├── finetune_shakespeare.py │ ├── train_gpt2.py │ └── train_shakespeare_char.py ├── configurator.py ├── data/ │ ├── openai_summarize_tldr/ │ │ └── prepare.py │ ├── openwebtext/ │ │ ├── prepare.py │ │ └── readme.md │ ├── shakespeare/ │ │ ├── prepare.py │ │ └── readme.md │ └── shakespeare_char/ │ ├── prepare.py │ └── readme.md ├── model.py ├── requirements.txt ├── sample.py ├── scaling_laws.ipynb ├── train.py ├── train_reward_model.py ├── train_reward_model_simple.py ├── train_rl.py ├── trainers/ │ ├── reward_trainer.py │ ├── rl_trainer.py │ └── trainer.py ├── transformer_sizing.ipynb └── utils.py ================================================ FILE CONTENTS ================================================ ================================================ FILE: .gitattributes ================================================ # Override jupyter in Github language stats for more accurate estimate of repo code languages # reference: https://github.com/github/linguist/blob/master/docs/overrides.md#generated-code *.ipynb linguist-generated ================================================ FILE: LICENSE ================================================ MIT License Copyright (c) 2022 Andrej Karpathy Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ================================================ FILE: README.md ================================================ # nanoChatGPT A crude RLHF (Reinforcement Learing from Human Feedback) layer on top of nanoGPT to test an idea I had that you can backpropagate through the reward function rather than use policy gradient. I have verified it works for a very basic example where you incentivise the network to produce words containing 'and'. The trick is to use the Straight-Through Gumbel-Softmax estimator. Also checkout chatgpt_dev_teaching.ipynb and the YouTube video explaining fine-tuning with RL: https://m.youtube.com/watch?v=soqTT0o1ZKo Prepare data: ``` $ python data/shakespeare/prepare.py ``` Once data is prepared start training. The configs assume cuda, if you don't have a gpu change to cpu in config. ``` $ python train.py # settings in config/config.yaml ``` Once a basic model is trained, can fine tune a reward model for an underlying reward rule. ``` $ python train_reward_model_simple.py # settings in config/config_reward.yaml ``` This creates a multihead model on top of the existing one. Once the reward model is trained sufficiently you can train the RL policy using: ``` $ python train_rl.py # settings in config/config_rl.yaml ``` The default config uses the Gumbel trick but it can be set to PG and it will do policy gradient instead (the latter still needs a critic implementation etc). I have validated that the Gumbel method works given that the preceding steps also worked. I am curious to see if this would scale to large models - let me know if you're able to test this. Model output after a short amount of training produces results like: ``` hand hand thousand the thousand the hand hand hand hand thousand ``` If you're feeling adventorous you can also try: ``` $ python train_reward_model.py ``` Which uses the reddit tldr dataset. The pipes should work but I have not actually finetuned this at all. References: Gumbel https://arxiv.org/pdf/1611.01144.pdf InstructGPT https://arxiv.org/abs/2203.02155 Below is Andrej Karpathy's original README, make sure you have installed the revelevant packages ________ # nanoGPT ![nanoGPT](assets/nanogpt.jpg) The simplest, fastest repository for training/finetuning medium-sized GPTs. It is a rewrite of [minGPT](https://github.com/karpathy/minGPT) that prioritizes teeth over education. Still under active development, but currently the file `train.py` reproduces GPT-2 (124M) on OpenWebText, running on a single 8XA100 40GB node in about 4 days of training. The code itself is plain and readable: `train.py` is a ~300-line boilerplate training loop and `model.py` a ~300-line GPT model definition, which can optionally load the GPT-2 weights from OpenAI. That's it. ![repro124m](assets/gpt2_124M_loss.png) Because the code is so simple, it is very easy to hack to your needs, train new models from scratch, or finetune pretrained checkpoints (e.g. biggest one currently available as a starting point would be the GPT-2 1.3B model from OpenAI). ## install Dependencies: - [pytorch](https://pytorch.org) <3 - [numpy](https://numpy.org/install/) <3 - `pip install transformers` for huggingface transformers <3 (to load GPT-2 checkpoints) - `pip install datasets` for huggingface datasets <3 (if you want to download + preprocess OpenWebText) - `pip install tiktoken` for OpenAI's fast BPE code <3 - `pip install wandb` for optional logging <3 - `pip install tqdm` ## quick start If you are not a deep learning professional and you just want to feel the magic and get your feet wet, the fastest way to get started is to train a character-level GPT on the works of Shakespeare. First, we download it as a single (1MB) file and turn it from raw text into one large stream of integers: ``` $ python data/shakespeare_char/prepare.py ``` This creates a `train.bin` and `val.bin` in that data directory. Now it is time to train your GPT. The size of it very much depends on the computational resources of your system: **I have a GPU**. Great, we can quickly train a baby GPT with the settings provided in the [config/train_shakespeare_char.py](config/train_shakespeare_char.py) config file: ``` $ python train.py config/train_shakespeare_char.py ``` If you peak inside it, you'll see that we're training a GPT with a context size of up to 256 characters, 384 feature channels, and it is a 6-layer Transformer with 6 heads in each layer. On one A100 GPU this training run takes about 3 minutes and the best validation loss is 1.4697. Based on the configuration, the model checkpoints are being written into the `--out_dir` directory `out-shakespeare-char`. So once the training finishes we can sample from the best model by pointing the sampling script at this directory: ``` $ python sample.py --out_dir=out-shakespeare-char ``` This generates a few samples, for example: ``` ANGELO: And cowards it be strawn to my bed, And thrust the gates of my threats, Because he that ale away, and hang'd An one with him. DUKE VINCENTIO: I thank your eyes against it. DUKE VINCENTIO: Then will answer him to save the malm: And what have you tyrannous shall do this? DUKE VINCENTIO: If you have done evils of all disposition To end his power, the day of thrust for a common men That I leave, to fight with over-liking Hasting in a roseman. ``` lol `¯\_(ツ)_/¯`. Not bad for a character-level model after 3 minutes of training on a GPU. Better results are quite likely obtainable by instead finetuning a pretrained GPT-2 model on this dataset (see finetuning section later). **I only have a macbook** (or other cheap computer). No worries, we can still train a GPT but we want to dial things down a notch. I recommend getting the bleeding edge PyTorch nightly ([select it here](https://pytorch.org/get-started/locally/) when installing) as it is currently quite likely to make your code more efficient. But even without it, a simple train run could look as follows: ``` $ python train.py config/train_shakespeare_char.py --device=cpu --compile=False --eval_iters=20 --log_interval=1 --block_size=64 --batch_size=12 --n_layer=4 --n_head=4 --n_embd=128 --max_iters=2000 --lr_decay_iters=2000 --dropout=0.0 ``` Here, since we are running on CPU instead of GPU we must set both `--device=cpu` and also turn off PyTorch 2.0 compile with `--compile=False`. Then when we evaluate we get a bit more noisy but faster estimate (`--eval_iters=20`, down from 200), our context size is only 64 characters instead of 256, and the batch size only 12 examples per iteration, not 64. We'll also use a much smaller Transformer (4 layers, 4 heads, 128 embedding size), and decrease the number of iterations to 2000 (and correspondingly usually decay the learning rate to around max_iters with `--lr_decay_iters`). Because our network is so small we also ease down on regularization (`--dropout=0.0`). This still runs in about ~3 minutes, but gets us a loss of only 1.88 and therefore also worse samples, but it's still good fun: ``` GLEORKEN VINGHARD III: Whell's the couse, the came light gacks, And the for mought you in Aut fries the not high shee bot thou the sought bechive in that to doth groan you, No relving thee post mose the wear ``` Not bad for ~3 minutes on a CPU, for a hint of the right character gestalt. If you're willing to wait longer free to tune the hyperparameters, increase the size of the network, the context length (`--block_size`), the length of training, etc. Finally, on Apple Silicon Macbooks and with a recent PyTorch version make sure to add `--device mps` (short for "Metal Performance Shaders"); PyTorch then uses the on-chip GPU that can *significantly* accelerate training (2-3X) and allow you to use larger networks. See [Issue 28](https://github.com/karpathy/nanoGPT/issues/28) for more. ## reproducing GPT-2 A more serious deep learning professional may be more interested in reproducing GPT-2 results. So here we go - we first tokenize the dataset, in this case the [OpenWebText](https://openwebtext2.readthedocs.io/en/latest/), an open reproduction of OpenAI's (private) WebText: ``` $ python data/openwebtext/prepare.py ``` This downloads and tokenizes the [OpenWebText](https://huggingface.co/datasets/openwebtext) dataset. It will create a `train.bin` and `val.bin` which holds the GPT2 BPE token ids in one sequence, stored as raw uint16 bytes. Then we're ready to kick off training. To reproduce GPT-2 (124M) you'll want at least an 8X A100 40GB node and run: ``` $ torchrun --standalone --nproc_per_node=8 train.py config/train_gpt2.py ``` This will run for about 4 days using PyTorch Distributed Data Parallel (DDP) and go down to loss of ~2.85. Now, a GPT-2 model just evaluated on OWT gets a val loss of about 3.11, but if you finetune it it will come down to ~2.85 territory (due to an apparent domain gap), making the two models ~match. If you're in a cluster environment and you are blessed with multiple GPU nodes you can make GPU go brrrr e.g. across 2 nodes like: ``` Run on the first (master) node with example IP 123.456.123.456: $ torchrun --nproc_per_node=8 --nnodes=2 --node_rank=0 --master_addr=123.456.123.456 --master_port=1234 train.py Run on the worker node: $ torchrun --nproc_per_node=8 --nnodes=2 --node_rank=1 --master_addr=123.456.123.456 --master_port=1234 train.py ``` It is a good idea to benchmark your interconnect (e.g. iperf3). In particular, if you don't have Infiniband then also prepend `NCCL_IB_DISABLE=1` to the above launches. Your multinode training will work, but most likely _crawl_. By default checkpoints are periodically written to the `--out_dir`. We can sample from the model by simply `$ python sample.py`. Finally, to train on a single GPU simply run the `$ python train.py` script. Have a look at all of its args, the script tries to be very readable, hackable and transparent. You'll most likely want to tune a number of those variables depending on your needs. ## baselines OpenAI GPT-2 checkpoints allow us to get some baselines in place for openwebtext. We can get the numbers as follows: ``` $ python train.py eval_gpt2 $ python train.py eval_gpt2_medium $ python train.py eval_gpt2_large $ python train.py eval_gpt2_xl ``` and observe the following losses on train and val: | model | params | train loss | val loss | | ------| ------ | ---------- | -------- | | gpt2 | 124M | 3.11 | 3.12 | | gpt2-medium | 350M | 2.85 | 2.84 | | gpt2-large | 774M | 2.66 | 2.67 | | gpt2-xl | 1558M | 2.56 | 2.54 | However, we have to note that GPT-2 was trained on (closed, never released) WebText, while OpenWebText is just a best-effort open reproduction of this dataset. This means there is a dataset domain gap. Indeed, taking the GPT-2 (124M) checkpoint and finetuning on OWT directly for a while reaches loss down to ~2.85. This then becomes the more appropriate baseline w.r.t. reproduction. ## finetuning Finetuning is no different than training, we just make sure to initialize from a pretrained model and train with a smaller learning rate. For an example of how to finetune a GPT on new text go to `data/shakespeare` and run `prepare.py` to download the tiny shakespeare dataset and render it into a `train.bin` and `val.bin`, using the OpenAI BPE tokenizer from GPT-2. Unlike OpenWebText this will run in seconds. Finetuning can take very little time, e.g. on a single GPU just a few minutes. Run an example finetuning like: ``` $ python train.py config/finetune_shakespeare.py ``` This will load the config parameter overrides in `config/finetune_shakespeare.py` (I didn't tune them much though). Basically, we initialize from a GPT2 checkpoint with `init_from` and train as normal, except shorter and with a small learning rate. If you're running out of memory try decreasing the model size (they are `{'gpt2', 'gpt2-medium', 'gpt2-large', 'gpt2-xl'}`) or possibly decreasing the `block_size` (context length). The best checkpoint (lowest validation loss) will be in the `out_dir` directory, e.g. in `out-shakespeare` by default, per the config file. You can then run the code in `sample.py --out_dir=out-shakespeare`: ``` THEODORE: Thou shalt sell me to the highest bidder: if I die, I sell thee to the first; if I go mad, I sell thee to the second; if I lie, I sell thee to the third; if I slay, I sell thee to the fourth: so buy or sell, I tell thee again, thou shalt not sell my possession. JULIET: And if thou steal, thou shalt not sell thyself. THEODORE: I do not steal; I sell the stolen goods. THEODORE: Thou know'st not what thou sell'st; thou, a woman, Thou art ever a victim, a thing of no worth: Thou hast no right, no right, but to be sold. ``` Whoa there, GPT, entering some dark place over there. I didn't really tune the hyperparameters in the config too much, feel free to try! ## sampling / inference Use the script `sample.py` to sample either from pre-trained GPT-2 models released by OpenAI, or from a model you trained yourself. For example, here is a way to sample from the largest available `gpt2-xl` model: ``` $ python sample.py \ --init_from=gpt2-xl \ --start="What is the answer to life, the universe, and everything?" \ --num_samples=5 --max_new_tokens=100 ``` If you'd like to sample from a model you trained, use the `--out_dir` to point the code appropriately. You can also prompt the model with some text from a file, e.g. `$ python sample.py --start=FILE:prompt.txt`. ## efficiency notes For simple model benchmarking and profiling, `bench.py` might be useful. It's identical to what happens in the meat of the training loop of `train.py`, but omits much of the other complexities. Note that the code by default uses [PyTorch 2.0](https://pytorch.org/get-started/pytorch-2.0/). At the time of writing (Dec 29, 2022) this makes `torch.compile()` available in the nightly release. The improvement from the one line of code is noticeable, e.g. cutting down iteration time from ~250ms / iter to 135ms / iter. Nice work PyTorch team! ## todos - Investigate and add FSDP instead of DDP - Eval zero-shot perplexities on standard evals (e.g. LAMBADA? HELM? etc.) - Finetune the finetuning script, I think the hyperparams are not great - Schedule for linear batch size increase during training - Incorporate other embeddings (rotary, alibi) - Separate out the optim buffers from model params in checkpoints I think - Additional logging around network health (e.g. gradient clip events, magnitudes) - Few more investigations around better init etc. ## troubleshooting Note that by default this repo uses PyTorch 2.0 (i.e. `torch.compile`). This is fairly new and experimental, and not yet available on all platforms (e.g. Windows). If you're running into related error messages try to disable this by adding `--compile=False` flag. This will slow down the code but at least it will run. For some context on this repository, GPT, and language modeling it might be helpful to watch my [Zero To Hero series](https://karpathy.ai/zero-to-hero.html). Specifically, the [GPT video](https://www.youtube.com/watch?v=kCc8FmEb1nY) is popular if you have some prior language modeling context. For more questions/discussions feel free to stop by **#nanoGPT** on Discord: [![](https://dcbadge.vercel.app/api/server/3zy8kqD9Cp?compact=true&style=flat)](https://discord.gg/3zy8kqD9Cp) ## acknowledgements All nanoGPT experiments are powered by GPUs on [Lambda labs](https://lambdalabs.com), my favorite Cloud GPU provider. Thank you Lambda labs for sponsoring nanoGPT! ================================================ FILE: bench.py ================================================ """ A much shorter version of train.py for benchmarking """ import os from contextlib import nullcontext import numpy as np import time import torch from model import GPTConfig, GPT # ----------------------------------------------------------------------------- batch_size = 12 block_size = 1024 bias = False real_data = True seed = 1337 device = 'cuda' # examples: 'cpu', 'cuda', 'cuda:0', 'cuda:1', etc. dtype = 'bfloat16' # 'float32' or 'bfloat16' or 'float16' compile = True # use PyTorch 2.0 to compile the model to be faster profile = False # use pytorch profiler, or just simple benchmarking? exec(open('configurator.py').read()) # overrides from command line or config file # ----------------------------------------------------------------------------- torch.manual_seed(seed) torch.cuda.manual_seed(seed) torch.backends.cuda.matmul.allow_tf32 = True # allow tf32 on matmul torch.backends.cudnn.allow_tf32 = True # allow tf32 on cudnn device_type = 'cuda' if 'cuda' in device else 'cpu' # for later use in torch.autocast ptdtype = {'float32': torch.float32, 'bfloat16': torch.bfloat16, 'float16': torch.float16}[dtype] ctx = nullcontext() if device_type == 'cpu' else torch.amp.autocast(device_type=device_type, dtype=ptdtype) # data loading init if real_data: dataset = 'openwebtext' data_dir = os.path.join('data', dataset) train_data = np.memmap(os.path.join(data_dir, 'train.bin'), dtype=np.uint16, mode='r') def get_batch(split): data = train_data # note ignore split in benchmarking script ix = torch.randint(len(data) - block_size, (batch_size,)) x = torch.stack([torch.from_numpy((data[i:i+block_size]).astype(np.int64)) for i in ix]) y = torch.stack([torch.from_numpy((data[i+1:i+1+block_size]).astype(np.int64)) for i in ix]) x, y = x.pin_memory().to(device, non_blocking=True), y.pin_memory().to(device, non_blocking=True) return x, y else: # alternatively, if fixed data is desired to not care about data loading x = torch.randint(50304, (batch_size, block_size), device=device) y = torch.randint(50304, (batch_size, block_size), device=device) get_batch = lambda split: (x, y) # model init gptconf = GPTConfig( block_size = block_size, # how far back does the model look? i.e. context size n_layer = 12, n_head = 12, n_embd = 768, # size of the model dropout = 0, # for determinism bias = bias, ) model = GPT(gptconf) model.to(device) optimizer = model.configure_optimizers(weight_decay=1e-2, learning_rate=1e-4, betas=(0.9, 0.95), device_type=device_type) if compile: print("Compiling model...") model = torch.compile(model) # pytorch 2.0 if profile: # useful docs on pytorch profiler: # - tutorial https://pytorch.org/tutorials/intermediate/tensorboard_profiler_tutorial.html # - api https://pytorch.org/docs/stable/profiler.html#torch.profiler.profile wait, warmup, active = 5, 5, 5 num_steps = wait + warmup + active with torch.profiler.profile( activities=[torch.profiler.ProfilerActivity.CPU, torch.profiler.ProfilerActivity.CUDA], schedule=torch.profiler.schedule(wait=wait, warmup=warmup, active=active, repeat=1), on_trace_ready=torch.profiler.tensorboard_trace_handler('./bench_log'), record_shapes=False, profile_memory=False, with_stack=False, # incurs an additional overhead, disable if not needed with_flops=True, with_modules=False, # only for torchscript models atm ) as prof: X, Y = get_batch('train') for k in range(num_steps): with ctx: logits, loss = model(X, Y) X, Y = get_batch('train') optimizer.zero_grad(set_to_none=True) loss.backward() optimizer.step() lossf = loss.item() print(f"{k}/{num_steps} loss: {lossf:.4f}") prof.step() # notify the profiler at end of each step else: # simple benchmarking torch.cuda.synchronize() for stage, num_steps in enumerate([10, 20]): # burnin, then benchmark t0 = time.time() X, Y = get_batch('train') for k in range(num_steps): with ctx: logits, loss = model(X, Y) X, Y = get_batch('train') optimizer.zero_grad(set_to_none=True) loss.backward() optimizer.step() lossf = loss.item() print(f"{k}/{num_steps} loss: {lossf:.4f}") torch.cuda.synchronize() t1 = time.time() dt = t1-t0 mfu = model.estimate_mfu(batch_size * 1 * num_steps, dt) if stage == 1: print(f"time per iteration: {dt/num_steps*1000:.4f}ms, MFU: {mfu*100:.2f}%") ================================================ FILE: chatgpt_dev_teaching.ipynb ================================================ { "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "provenance": [], "authorship_tag": "ABX9TyNVW45SCGk3DW7yIfDYKg2T", "include_colab_link": true }, "kernelspec": { "name": "python3", "display_name": "Python 3" }, "language_info": { "name": "python" }, "accelerator": "GPU", "gpuClass": "premium", "widgets": { "application/vnd.jupyter.widget-state+json": { "49506f92d9d042d4a6111f3a1b9f79e1": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_95f4423f202b44f9b5f6fbc250b5a300", "IPY_MODEL_e78ecc50321f42328d47999c01244651", "IPY_MODEL_cc422e07c3ce4ba99c84bda8b7d7c596" ], "layout": "IPY_MODEL_8be555a8bb6e4068a42e4312e75a30c2" } }, "95f4423f202b44f9b5f6fbc250b5a300": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_b7f7646e496849b8bb3239b64b93e193", "placeholder": "​", "style": "IPY_MODEL_b728662de8ac41888e838e47e67d6aa9", "value": "Downloading (…)lve/main/config.json: 100%" } }, "e78ecc50321f42328d47999c01244651": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_70a9186eb9e64d9ea5b456f43622d979", "max": 949, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_7a5dedee422f42b49ecac65d3138c167", "value": 949 } }, "cc422e07c3ce4ba99c84bda8b7d7c596": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_c2e28b80efc74a549af7bdf3eee46eda", "placeholder": "​", "style": "IPY_MODEL_057de1e97cad4a2297ad3db02c2481d6", "value": " 949/949 [00:00<00:00, 45.7kB/s]" } }, "8be555a8bb6e4068a42e4312e75a30c2": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "b7f7646e496849b8bb3239b64b93e193": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "b728662de8ac41888e838e47e67d6aa9": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "70a9186eb9e64d9ea5b456f43622d979": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "7a5dedee422f42b49ecac65d3138c167": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "c2e28b80efc74a549af7bdf3eee46eda": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "057de1e97cad4a2297ad3db02c2481d6": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "daabe15b3af34e1098a7e2f620ae1f36": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_c8fd1744cb744200b0bcc8629d173bf1", "IPY_MODEL_7a52e4742e594e25a28feb18c417c076", "IPY_MODEL_513d02e2c28440d9897d1aa097d1e743" ], "layout": "IPY_MODEL_d147cf4c15ff449598d6b429c90db0ae" } }, "c8fd1744cb744200b0bcc8629d173bf1": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_c74d25684a4d4eafbd3546dcff921cb0", "placeholder": "​", "style": "IPY_MODEL_aa57d59560114d55bede331307257f6b", "value": "Downloading pytorch_model.bin: 100%" } }, "7a52e4742e594e25a28feb18c417c076": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_e5c0bc2de75347c39dc9ca23a8129650", "max": 539679413, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_a18d35c8ff2649e397ed983a6f06e959", "value": 539679413 } }, "513d02e2c28440d9897d1aa097d1e743": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_5e3d4cc0ec9d4c0ca0d0136e97aca2ce", "placeholder": "​", "style": "IPY_MODEL_3d4f1a5c8128451884bd21fef234b89b", "value": " 540M/540M [00:08<00:00, 67.9MB/s]" } }, "d147cf4c15ff449598d6b429c90db0ae": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "c74d25684a4d4eafbd3546dcff921cb0": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "aa57d59560114d55bede331307257f6b": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "e5c0bc2de75347c39dc9ca23a8129650": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a18d35c8ff2649e397ed983a6f06e959": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "5e3d4cc0ec9d4c0ca0d0136e97aca2ce": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "3d4f1a5c8128451884bd21fef234b89b": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "9645d7c59b0546159890f042ca420632": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_8893c2570f464d9697dda12cc42023d5", "IPY_MODEL_0c71d29e93f24885802ad5fa6d240675", "IPY_MODEL_7b859eccf54e41698cf179d68bb109d0" ], "layout": "IPY_MODEL_725944a581244ab6a2a4b61afa29c1c0" } }, "8893c2570f464d9697dda12cc42023d5": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_e536fea041f7441aa8cc93a448492e33", "placeholder": "​", "style": "IPY_MODEL_8e5f96a8ec714284a7f000af96300f67", "value": "Downloading (…)okenizer_config.json: 100%" } }, "0c71d29e93f24885802ad5fa6d240675": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_04984bbb402c4bd4afad349e629e2452", "max": 338, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_4a07648ab35047c8b61e32e3a61d970b", "value": 338 } }, "7b859eccf54e41698cf179d68bb109d0": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_bd5178cf064049678d4efac17e47a5da", "placeholder": "​", "style": "IPY_MODEL_ae235b9add2d41ee9f808a2b4d43568b", "value": " 338/338 [00:00<00:00, 25.9kB/s]" } }, "725944a581244ab6a2a4b61afa29c1c0": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "e536fea041f7441aa8cc93a448492e33": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "8e5f96a8ec714284a7f000af96300f67": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "04984bbb402c4bd4afad349e629e2452": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "4a07648ab35047c8b61e32e3a61d970b": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "bd5178cf064049678d4efac17e47a5da": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "ae235b9add2d41ee9f808a2b4d43568b": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "8fb0a31c3ecf4a4880803e81e1bc581d": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_8bc84935b3964e338d00ea1bb7c3c59f", "IPY_MODEL_d729ac6a87174fbda03175c21859ea82", "IPY_MODEL_b80bd9149ae34878bfeeb4fca815231c" ], "layout": "IPY_MODEL_40007a4900824bd88d3a180fd0a8325a" } }, "8bc84935b3964e338d00ea1bb7c3c59f": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_ce6627b477bc430d968fd186443b34e3", "placeholder": "​", "style": "IPY_MODEL_2f2cf4fd7a434da5be62bd088360a6ba", "value": "Downloading (…)solve/main/vocab.txt: 100%" } }, "d729ac6a87174fbda03175c21859ea82": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_31aa09c7a2ea443b8a177c4f91a7d427", "max": 843438, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_f17f1682313e49df97e399508c15716a", "value": 843438 } }, "b80bd9149ae34878bfeeb4fca815231c": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_dde9d315de174921844ebb72865d1d3f", "placeholder": "​", "style": "IPY_MODEL_386eb97816cd41e188dbfa1fcd299b16", "value": " 843k/843k [00:01<00:00, 753kB/s]" } }, "40007a4900824bd88d3a180fd0a8325a": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "ce6627b477bc430d968fd186443b34e3": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "2f2cf4fd7a434da5be62bd088360a6ba": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "31aa09c7a2ea443b8a177c4f91a7d427": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "f17f1682313e49df97e399508c15716a": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "dde9d315de174921844ebb72865d1d3f": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "386eb97816cd41e188dbfa1fcd299b16": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "d5af658c67874c67b08bd8ef72411ea1": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_aba6fb6586534c1f86ce65b7c8bde51f", "IPY_MODEL_a6878fe333e243dc81ec0d36bb0dce19", "IPY_MODEL_d1efe95c3d55440386db28242a8f38d1" ], "layout": "IPY_MODEL_ecfb1bf7ae8a4328ac982234127e5c2b" } }, "aba6fb6586534c1f86ce65b7c8bde51f": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_4847c36ed46d49429e37f4b9b7862a57", "placeholder": "​", "style": "IPY_MODEL_631e5ee44ccc4e4aaec456af90df7965", "value": "Downloading (…)solve/main/bpe.codes: 100%" } }, "a6878fe333e243dc81ec0d36bb0dce19": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_662fd517bf0540ed9c607d3ac8dda0af", "max": 1078931, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_04d90c9c251a4fae8db96d711568cd64", "value": 1078931 } }, "d1efe95c3d55440386db28242a8f38d1": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_7bd831ca2d354ee7a75c22c700963ea8", "placeholder": "​", "style": "IPY_MODEL_3de7e32fefd54df6a6d3596205ba0338", "value": " 1.08M/1.08M [00:01<00:00, 961kB/s]" } }, "ecfb1bf7ae8a4328ac982234127e5c2b": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "4847c36ed46d49429e37f4b9b7862a57": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "631e5ee44ccc4e4aaec456af90df7965": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "662fd517bf0540ed9c607d3ac8dda0af": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "04d90c9c251a4fae8db96d711568cd64": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "7bd831ca2d354ee7a75c22c700963ea8": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "3de7e32fefd54df6a6d3596205ba0338": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "ddbbcddaba34416881ebb023864b0f2d": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_d0fc35ec11634f8691bf5eefa0505cb6", "IPY_MODEL_598a025b08dd4469a88bd9ebe85ebb94", "IPY_MODEL_9bc45acff2e049d59eef1e8374f03a93" ], "layout": "IPY_MODEL_5a627f08513443e09dac036f3ff2fa1f" } }, "d0fc35ec11634f8691bf5eefa0505cb6": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_397d77d019e748f2b4178bb78ff2d471", "placeholder": "​", "style": "IPY_MODEL_590cb10ff4164d309063faf6baedaf1f", "value": "Downloading (…)in/added_tokens.json: 100%" } }, "598a025b08dd4469a88bd9ebe85ebb94": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_9d5f595908964edba6b6f7e8a6fba5eb", "max": 22, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_0e05ee7e2e934ca983d70addd1c9ce92", "value": 22 } }, "9bc45acff2e049d59eef1e8374f03a93": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_9c56fb6a76414f558c665608a21d39cd", "placeholder": "​", "style": "IPY_MODEL_399aa39b85c14ff8b9cd887343af36be", "value": " 22.0/22.0 [00:00<00:00, 1.66kB/s]" } }, "5a627f08513443e09dac036f3ff2fa1f": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "397d77d019e748f2b4178bb78ff2d471": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "590cb10ff4164d309063faf6baedaf1f": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "9d5f595908964edba6b6f7e8a6fba5eb": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "0e05ee7e2e934ca983d70addd1c9ce92": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "9c56fb6a76414f558c665608a21d39cd": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "399aa39b85c14ff8b9cd887343af36be": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "d4712217939b44f59e58a0f23a8d4df7": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_c4fa3386d2234473ad7a8b6dae7a4d95", "IPY_MODEL_97c0a2c0b079410496ab497f8fa8bd12", "IPY_MODEL_565f60bb19d443f6aacc9b08ff68c7ff" ], "layout": "IPY_MODEL_5556b66847b04b4aabe0aaa029dc3e64" } }, "c4fa3386d2234473ad7a8b6dae7a4d95": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_fe8f0c618d104deaaed3ea138acdc230", "placeholder": "​", "style": "IPY_MODEL_52b516dd11a747099e9e0e808a168e55", "value": "Downloading (…)cial_tokens_map.json: 100%" } }, "97c0a2c0b079410496ab497f8fa8bd12": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_8696eb98489f409a848684b77b2353ac", "max": 167, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_679b6d995fad4a649e055bee8bfe83d1", "value": 167 } }, "565f60bb19d443f6aacc9b08ff68c7ff": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_eff4e772ccd4445ba4b6f624ea3b7e8f", "placeholder": "​", "style": "IPY_MODEL_c7570cd91b564389b59c98a694ab7771", "value": " 167/167 [00:00<00:00, 13.0kB/s]" } }, "5556b66847b04b4aabe0aaa029dc3e64": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "fe8f0c618d104deaaed3ea138acdc230": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "52b516dd11a747099e9e0e808a168e55": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "8696eb98489f409a848684b77b2353ac": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "679b6d995fad4a649e055bee8bfe83d1": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "eff4e772ccd4445ba4b6f624ea3b7e8f": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "c7570cd91b564389b59c98a694ab7771": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } } } } }, "cells": [ { "cell_type": "markdown", "metadata": { "id": "view-in-github", "colab_type": "text" }, "source": [ "\"Open" ] }, { "cell_type": "markdown", "source": [ "Let's train models for generating Shakespeare with different sentiments.\n", "\n", "Happy Shakespeare will say things like: \n", "\n", "*“Nay, thanks, then, I do meet change, this Romeo. \n", "The pleasure of his hair!”*\n", "\n", "Sad Shakespear will say things like:\n", "\n", "\n", "*“The senators's dead, of their world:\n", "Be not for your friends.”*\n", "\n" ], "metadata": { "id": "9-LuRS2wAaBS" } }, { "cell_type": "code", "source": [ "# Notebook created by Sanjeevan Ahilan\n", "# GPT implementation was borrowed from Andrej Karpathy \n", "# https://colab.research.google.com/drive/1JMLa53HDuA-i7ZBmqV7ZnA3c_fvtXnx-?usp=sharing\n", "# My repo: https://github.com/sanjeevanahilan/nanoChatGPT\n", "# My twitter: https://twitter.com/sanjeevanahilan\n", "\n", "import torch\n", "import torch.nn as nn\n", "from torch.nn import functional as F\n", "\n", "# hyperparameters\n", "batch_size = 16 # how many independent sequences will we process in parallel?\n", "block_size = 32 # what is the maximum context length for predictions?\n", "max_iters = 3000\n", "eval_interval = 100\n", "learning_rate = 1e-3\n", "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n", "eval_iters = 200\n", "n_embd = 64\n", "n_head = 4\n", "n_layer = 4\n", "dropout = 0.0\n", "# ------------\n", "\n", "torch.manual_seed(1337)\n" ], "metadata": { "id": "aZGtyx5lAOzu", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "67a0cd77-355e-4c15-f07d-ef3758a87312" }, "execution_count": 1, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": {}, "execution_count": 1 } ] }, { "cell_type": "code", "source": [ "# Download Shakespeare\n", "%time\n", "!wget https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt\n", "with open('input.txt', 'r', encoding='utf-8') as f:\n", " text = f.read()" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "DA-Gox1YiM6l", "outputId": "e04b1540-6e70-41dc-d32f-a8e234a8c87d" }, "execution_count": 2, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "CPU times: user 2 µs, sys: 1e+03 ns, total: 3 µs\n", "Wall time: 7.15 µs\n", "--2023-03-18 11:49:44-- https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt\n", "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n", "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n", "HTTP request sent, awaiting response... 200 OK\n", "Length: 1115394 (1.1M) [text/plain]\n", "Saving to: ‘input.txt’\n", "\n", "input.txt 100%[===================>] 1.06M --.-KB/s in 0.03s \n", "\n", "2023-03-18 11:49:45 (40.0 MB/s) - ‘input.txt’ saved [1115394/1115394]\n", "\n" ] } ] }, { "cell_type": "markdown", "source": [ "Rather than use character level tokenizer lets use tiktoken; Openai's implementation of a Byte Pair Encoding (BPE) tokenizer" ], "metadata": { "id": "SyyJfXbdAr81" } }, { "cell_type": "code", "source": [ "!pip install tiktoken\n", "import tiktoken" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "y7LBbGC9onTu", "outputId": "6a9ef1dd-2e22-4468-d797-d41cb0bc1de2" }, "execution_count": 3, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n", "Collecting tiktoken\n", " Downloading tiktoken-0.3.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m29.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: regex>=2022.1.18 in /usr/local/lib/python3.9/dist-packages (from tiktoken) (2022.10.31)\n", "Requirement already satisfied: requests>=2.26.0 in /usr/local/lib/python3.9/dist-packages (from tiktoken) (2.27.1)\n", "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests>=2.26.0->tiktoken) (1.26.15)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests>=2.26.0->tiktoken) (2022.12.7)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests>=2.26.0->tiktoken) (3.4)\n", "Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests>=2.26.0->tiktoken) (2.0.12)\n", "Installing collected packages: tiktoken\n", "Successfully installed tiktoken-0.3.2\n" ] } ] }, { "cell_type": "code", "source": [ "enc = tiktoken.get_encoding(\"gpt2\")\n", "vocab_size = 50257" ], "metadata": { "id": "btWJmnfapKhn" }, "execution_count": 4, "outputs": [] }, { "cell_type": "code", "source": [ "# You can see that the sequence of characters 'the' has been encoded as a single\n", "# number because it is commonly reoccuring whereas 'thh' requires two numbers to\n", "# encode 'th' and 'h'\n", "\n", "print(enc.encode('the'))\n", "print(enc.encode('thh'))\n", "print(enc.decode([400]))" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "8f9l4JdQCfSd", "outputId": "42ac95f9-96cb-4fd4-e110-a339c0e45fcf" }, "execution_count": 5, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "[1169]\n", "[400, 71]\n", "th\n" ] } ] }, { "cell_type": "code", "source": [ "# Train and test splits\n", "data = torch.tensor(enc.encode(text), dtype=torch.long)\n", "n = int(0.9*len(data)) # first 90% will be train, rest val\n", "train_data = data[:n]\n", "val_data = data[n:]\n", "\n", "# data loading\n", "def get_batch(split):\n", " # generate a small batch of data of inputs x and targets y\n", " data = train_data if split == 'train' else val_data\n", " ix = torch.randint(len(data) - block_size, (batch_size,))\n", " x = torch.stack([data[i:i+block_size] for i in ix])\n", " y = torch.stack([data[i+1:i+block_size+1] for i in ix])\n", " x, y = x.to(device), y.to(device)\n", " return x, y\n", "\n", "@torch.no_grad()\n", "def estimate_loss():\n", " out = {}\n", " model.eval()\n", " for split in ['train', 'val']:\n", " losses = torch.zeros(eval_iters)\n", " for k in range(eval_iters):\n", " X, Y = get_batch(split)\n", " logits, loss = model(X, Y)\n", " losses[k] = loss.item()\n", " out[split] = losses.mean()\n", " model.train()\n", " return out\n", "\n", "class Head(nn.Module):\n", " \"\"\" one head of self-attention \"\"\"\n", "\n", " def __init__(self, head_size):\n", " super().__init__()\n", " self.key = nn.Linear(n_embd, head_size, bias=False)\n", " self.query = nn.Linear(n_embd, head_size, bias=False)\n", " self.value = nn.Linear(n_embd, head_size, bias=False)\n", " self.register_buffer('tril', torch.tril(torch.ones(block_size, block_size)))\n", "\n", " self.dropout = nn.Dropout(dropout)\n", "\n", " def forward(self, x):\n", " B,T,C = x.shape\n", " k = self.key(x) # (B,T,C)\n", " q = self.query(x) # (B,T,C)\n", " # compute attention scores (\"affinities\")\n", " wei = q @ k.transpose(-2,-1) * C**-0.5 # (B, T, C) @ (B, C, T) -> (B, T, T)\n", " wei = wei.masked_fill(self.tril[:T, :T] == 0, float('-inf')) # (B, T, T)\n", " wei = F.softmax(wei, dim=-1) # (B, T, T)\n", " wei = self.dropout(wei)\n", " # perform the weighted aggregation of the values\n", " v = self.value(x) # (B,T,C)\n", " out = wei @ v # (B, T, T) @ (B, T, C) -> (B, T, C)\n", " return out\n", "\n", "class MultiHeadAttention(nn.Module):\n", " \"\"\" multiple heads of self-attention in parallel \"\"\"\n", "\n", " def __init__(self, num_heads, head_size):\n", " super().__init__()\n", " self.heads = nn.ModuleList([Head(head_size) for _ in range(num_heads)])\n", " self.proj = nn.Linear(n_embd, n_embd)\n", " self.dropout = nn.Dropout(dropout)\n", "\n", " def forward(self, x):\n", " out = torch.cat([h(x) for h in self.heads], dim=-1)\n", " out = self.dropout(self.proj(out))\n", " return out\n", "\n", "class FeedFoward(nn.Module):\n", " \"\"\" a simple linear layer followed by a non-linearity \"\"\"\n", "\n", " def __init__(self, n_embd):\n", " super().__init__()\n", " self.net = nn.Sequential(\n", " nn.Linear(n_embd, 4 * n_embd),\n", " nn.ReLU(),\n", " nn.Linear(4 * n_embd, n_embd),\n", " nn.Dropout(dropout),\n", " )\n", "\n", " def forward(self, x):\n", " return self.net(x)\n", "\n", "class Block(nn.Module):\n", " \"\"\" Transformer block: communication followed by computation \"\"\"\n", "\n", " def __init__(self, n_embd, n_head):\n", " # n_embd: embedding dimension, n_head: the number of heads we'd like\n", " super().__init__()\n", " head_size = n_embd // n_head\n", " self.sa = MultiHeadAttention(n_head, head_size)\n", " self.ffwd = FeedFoward(n_embd)\n", " self.ln1 = nn.LayerNorm(n_embd)\n", " self.ln2 = nn.LayerNorm(n_embd)\n", "\n", " def forward(self, x):\n", " x = x + self.sa(self.ln1(x))\n", " x = x + self.ffwd(self.ln2(x))\n", " return x\n", "\n", "class GPT(nn.Module):\n", "\n", " def __init__(self):\n", " super().__init__()\n", " # each token directly reads off the logits for the next token from a lookup table\n", " self.token_embedding_table = nn.Embedding(vocab_size, n_embd)\n", " self.position_embedding_table = nn.Embedding(block_size, n_embd)\n", " self.blocks = nn.Sequential(*[Block(n_embd, n_head=n_head) for _ in range(n_layer)])\n", " self.ln_f = nn.LayerNorm(n_embd) # final layer norm\n", " self.lm_head = nn.Linear(n_embd, vocab_size)\n", "\n", " def forward(self, idx, targets=None):\n", " B, T = idx.shape\n", "\n", " # idx and targets are both (B,T) tensor of integers\n", " tok_emb = self.token_embedding_table(idx) # (B,T,C)\n", " pos_emb = self.position_embedding_table(torch.arange(T, device=device)) # (T,C)\n", " x = tok_emb + pos_emb # (B,T,C)\n", " x = self.blocks(x) # (B,T,C)\n", " x = self.ln_f(x) # (B,T,C)\n", " logits = self.lm_head(x) # (B,T,vocab_size)\n", "\n", " if targets is None:\n", " loss = None\n", " else:\n", " B, T, C = logits.shape\n", " logits = logits.view(B*T, C)\n", " targets = targets.view(B*T)\n", " loss = F.cross_entropy(logits, targets)\n", "\n", " return logits, loss\n", "\n", " def generate(self, idx, max_new_tokens):\n", " # idx is (B, T) array of indices in the current context\n", " for _ in range(max_new_tokens):\n", " # crop idx to the last block_size tokens\n", " idx_cond = idx[:, -block_size:]\n", " # get the predictions\n", " logits, loss = self(idx_cond)\n", " # focus only on the last time step\n", " logits = logits[:, -1, :] # becomes (B, C)\n", " # apply softmax to get probabilities\n", " probs = F.softmax(logits, dim=-1) # (B, C)\n", " # sample from the distribution\n", " idx_next = torch.multinomial(probs, num_samples=1) # (B, 1)\n", " # append sampled index to the running sequence\n", " idx = torch.cat((idx, idx_next), dim=1) # (B, T+1)\n", " return idx\n", "\n", "model = GPT()\n", "m = model.to(device)\n", "# print the number of parameters in the model\n", "print(sum(p.numel() for p in m.parameters())/1e6, 'M parameters')\n", "\n", "# create a PyTorch optimizer\n", "optimizer = torch.optim.AdamW(model.parameters(), lr=learning_rate)\n", "\n", "for iter in range(max_iters):\n", "\n", " # every once in a while evaluate the loss on train and val sets\n", " if iter % eval_interval == 0 or iter == max_iters - 1:\n", " losses = estimate_loss()\n", " print(f\"step {iter}: train loss {losses['train']:.4f}, val loss {losses['val']:.4f}\")\n", "\n", " # sample a batch of data\n", " xb, yb = get_batch('train')\n", "\n", " # evaluate the loss\n", " logits, loss = model(xb, yb)\n", " optimizer.zero_grad(set_to_none=True)\n", " loss.backward()\n", " optimizer.step()" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "ngS-jQschyJc", "outputId": "8dedfa53-bdf7-446c-d335-887bf740efab" }, "execution_count": 6, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "6.684497 M parameters\n", "step 0: train loss 10.9982, val loss 11.0105\n", "step 100: train loss 6.3955, val loss 6.4657\n", "step 200: train loss 6.0255, val loss 6.1584\n", "step 300: train loss 5.7608, val loss 5.9578\n", "step 400: train loss 5.4809, val loss 5.7327\n", "step 500: train loss 5.2543, val loss 5.5400\n", "step 600: train loss 5.1017, val loss 5.4008\n", "step 700: train loss 4.9718, val loss 5.3382\n", "step 800: train loss 4.8850, val loss 5.2539\n", "step 900: train loss 4.8084, val loss 5.1674\n", "step 1000: train loss 4.7187, val loss 5.0770\n", "step 1100: train loss 4.6621, val loss 5.1166\n", "step 1200: train loss 4.5916, val loss 5.0159\n", "step 1300: train loss 4.5599, val loss 5.0297\n", "step 1400: train loss 4.4998, val loss 4.9929\n", "step 1500: train loss 4.4481, val loss 4.9933\n", "step 1600: train loss 4.4182, val loss 4.9204\n", "step 1700: train loss 4.4202, val loss 4.9445\n", "step 1800: train loss 4.3644, val loss 4.9217\n", "step 1900: train loss 4.3479, val loss 4.9221\n", "step 2000: train loss 4.2925, val loss 4.8603\n", "step 2100: train loss 4.2446, val loss 4.8765\n", "step 2200: train loss 4.2491, val loss 4.8753\n", "step 2300: train loss 4.1900, val loss 4.8271\n", "step 2400: train loss 4.1705, val loss 4.8660\n", "step 2500: train loss 4.1608, val loss 4.8778\n", "step 2600: train loss 4.1379, val loss 4.8898\n", "step 2700: train loss 4.1021, val loss 4.8513\n", "step 2800: train loss 4.1099, val loss 4.8742\n", "step 2900: train loss 4.0874, val loss 4.8534\n", "step 2999: train loss 4.0720, val loss 4.8296\n" ] } ] }, { "cell_type": "code", "source": [ "# generate from the model\n", "context = torch.zeros((1, 1), dtype=torch.long, device=device)\n", "print(enc.decode(m.generate(context, max_new_tokens=100)[0].tolist()))" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "9C93Lk1biRLE", "outputId": "94a58a5d-a9fb-438f-ec29-ab231b145951" }, "execution_count": 7, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "! what any consul\n", "matter they have we be for so dishonour\n", "Redpt'd in our courtesy,\n", "Or his highness i' the embracements, nor feeds had an err:\n", "I have the hallissign daughter to pawn'd.\n", "and stout see me: but God they forget\n", "And meet the tribranch man,\n", "Of a brace art'd; I'll plant show't\n", "From whom I a joyful with a chair access with callat.\n", "\n", "HENRY\n" ] } ] }, { "cell_type": "code", "source": [ "!pip install transformers\n", "from transformers import pipeline\n", "sentiment_pipeline = pipeline(model=\"finiteautomata/bertweet-base-sentiment-analysis\")" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 675, "referenced_widgets": [ "49506f92d9d042d4a6111f3a1b9f79e1", "95f4423f202b44f9b5f6fbc250b5a300", "e78ecc50321f42328d47999c01244651", "cc422e07c3ce4ba99c84bda8b7d7c596", "8be555a8bb6e4068a42e4312e75a30c2", "b7f7646e496849b8bb3239b64b93e193", "b728662de8ac41888e838e47e67d6aa9", "70a9186eb9e64d9ea5b456f43622d979", "7a5dedee422f42b49ecac65d3138c167", "c2e28b80efc74a549af7bdf3eee46eda", "057de1e97cad4a2297ad3db02c2481d6", "daabe15b3af34e1098a7e2f620ae1f36", "c8fd1744cb744200b0bcc8629d173bf1", "7a52e4742e594e25a28feb18c417c076", "513d02e2c28440d9897d1aa097d1e743", "d147cf4c15ff449598d6b429c90db0ae", "c74d25684a4d4eafbd3546dcff921cb0", "aa57d59560114d55bede331307257f6b", "e5c0bc2de75347c39dc9ca23a8129650", "a18d35c8ff2649e397ed983a6f06e959", "5e3d4cc0ec9d4c0ca0d0136e97aca2ce", "3d4f1a5c8128451884bd21fef234b89b", "9645d7c59b0546159890f042ca420632", "8893c2570f464d9697dda12cc42023d5", "0c71d29e93f24885802ad5fa6d240675", "7b859eccf54e41698cf179d68bb109d0", "725944a581244ab6a2a4b61afa29c1c0", "e536fea041f7441aa8cc93a448492e33", "8e5f96a8ec714284a7f000af96300f67", "04984bbb402c4bd4afad349e629e2452", "4a07648ab35047c8b61e32e3a61d970b", "bd5178cf064049678d4efac17e47a5da", "ae235b9add2d41ee9f808a2b4d43568b", "8fb0a31c3ecf4a4880803e81e1bc581d", "8bc84935b3964e338d00ea1bb7c3c59f", "d729ac6a87174fbda03175c21859ea82", "b80bd9149ae34878bfeeb4fca815231c", "40007a4900824bd88d3a180fd0a8325a", "ce6627b477bc430d968fd186443b34e3", "2f2cf4fd7a434da5be62bd088360a6ba", "31aa09c7a2ea443b8a177c4f91a7d427", "f17f1682313e49df97e399508c15716a", "dde9d315de174921844ebb72865d1d3f", "386eb97816cd41e188dbfa1fcd299b16", "d5af658c67874c67b08bd8ef72411ea1", "aba6fb6586534c1f86ce65b7c8bde51f", "a6878fe333e243dc81ec0d36bb0dce19", "d1efe95c3d55440386db28242a8f38d1", "ecfb1bf7ae8a4328ac982234127e5c2b", "4847c36ed46d49429e37f4b9b7862a57", "631e5ee44ccc4e4aaec456af90df7965", "662fd517bf0540ed9c607d3ac8dda0af", "04d90c9c251a4fae8db96d711568cd64", "7bd831ca2d354ee7a75c22c700963ea8", "3de7e32fefd54df6a6d3596205ba0338", "ddbbcddaba34416881ebb023864b0f2d", "d0fc35ec11634f8691bf5eefa0505cb6", "598a025b08dd4469a88bd9ebe85ebb94", "9bc45acff2e049d59eef1e8374f03a93", "5a627f08513443e09dac036f3ff2fa1f", "397d77d019e748f2b4178bb78ff2d471", "590cb10ff4164d309063faf6baedaf1f", "9d5f595908964edba6b6f7e8a6fba5eb", "0e05ee7e2e934ca983d70addd1c9ce92", "9c56fb6a76414f558c665608a21d39cd", "399aa39b85c14ff8b9cd887343af36be", "d4712217939b44f59e58a0f23a8d4df7", "c4fa3386d2234473ad7a8b6dae7a4d95", "97c0a2c0b079410496ab497f8fa8bd12", "565f60bb19d443f6aacc9b08ff68c7ff", "5556b66847b04b4aabe0aaa029dc3e64", "fe8f0c618d104deaaed3ea138acdc230", "52b516dd11a747099e9e0e808a168e55", "8696eb98489f409a848684b77b2353ac", "679b6d995fad4a649e055bee8bfe83d1", "eff4e772ccd4445ba4b6f624ea3b7e8f", "c7570cd91b564389b59c98a694ab7771" ] }, "id": "f93j9jAXuK8S", "outputId": "d585128a-3fad-4415-e564-fe9d338c4726" }, "execution_count": 8, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n", "Collecting transformers\n", " Downloading transformers-4.27.1-py3-none-any.whl (6.7 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.7/6.7 MB\u001b[0m \u001b[31m90.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.9/dist-packages (from transformers) (4.65.0)\n", "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.9/dist-packages (from transformers) (2022.10.31)\n", "Collecting huggingface-hub<1.0,>=0.11.0\n", " Downloading huggingface_hub-0.13.2-py3-none-any.whl (199 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m199.2/199.2 KB\u001b[0m \u001b[31m23.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.9/dist-packages (from transformers) (6.0)\n", "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.9/dist-packages (from transformers) (1.22.4)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.9/dist-packages (from transformers) (2.27.1)\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.9/dist-packages (from transformers) (3.10.0)\n", "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.9/dist-packages (from transformers) (23.0)\n", "Collecting tokenizers!=0.11.3,<0.14,>=0.11.1\n", " Downloading tokenizers-0.13.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.6 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.6/7.6 MB\u001b[0m \u001b[31m93.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.9/dist-packages (from huggingface-hub<1.0,>=0.11.0->transformers) (4.5.0)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests->transformers) (2022.12.7)\n", "Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests->transformers) (2.0.12)\n", "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests->transformers) (1.26.15)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests->transformers) (3.4)\n", "Installing collected packages: tokenizers, huggingface-hub, transformers\n", "Successfully installed huggingface-hub-0.13.2 tokenizers-0.13.2 transformers-4.27.1\n" ] }, { "output_type": "display_data", "data": { "text/plain": [ "Downloading (…)lve/main/config.json: 0%| | 0.00/949 [00:00 1, this is the micro-batch size block_size: 32 model: n_layer: 2 n_head: 2 n_embd: 32 dropout: 0.0 # for pretraining 0 is good, for finetuning try 0.1+ bias: False # do we use bias inside LayerNorm and Linear layers? optimizer: # adamw learning_rate: 6.0e-4 # max learning rate max_iters: 600000 # total number of training iterations weight_decay: 1.0e-2 beta1: 0.9 beta2: 0.95 grad_clip: 1.0 # clip gradients at this value, or disable if == 0.0 decay_lr: True # whether to decay the learning rate warmup_iters: 2000 # how many steps to warm up for lr_decay_iters: 600000 # should be ~= max_iters per Chinchilla min_lr: 6.0e-5 # minimum learning rate, should be ~= learning_rate/10 per Chinchilla DDP: backend: nccl # 'nccl', 'gloo', etc. system: device: cuda # examples: 'cpu', 'cuda', 'cuda:0', 'cuda:1' etc., or try 'mps' on macbooks dtype: float16 # 'float32', 'bfloat16', or 'float16', the latter will auto implement a GradScaler compile: False # use PyTorch 2.0 to compile the model to be faster ================================================ FILE: config/config_reward.yaml ================================================ IO: out_dir: out eval_interval: 500 log_interval: 1 eval_iters: 100 eval_only: False # if True, script exits right after the first eval always_save_checkpoint: True # if True, always save a checkpoint after each eval init_from: resume # 'scratch' or 'resume' or 'gpt2*' init_multihead_from: scratch out_dir_multihead: out_reward # used if restoring multihead wandb: wandb_log: True # disabled by default wandb_project: rlhf # 'gpt2' wandb_run_name: gpt2 # 'run' + str(time.time()) data: dataset: 'shakespeare' # 'openwebtext', 'shakespeare', 'openai_summarize_tldr' gradient_accumulation_steps: 1 # used to simulate larger batch sizes batch_size: 64 # if gradient_accumulation_steps > 1, this is the micro-batch size block_size: 32 model: n_layer: 2 n_head: 2 n_embd: 32 dropout: 0.0 # for pretraining 0 is good, for finetuning try 0.1+ bias: False # do we use bias inside LayerNorm and Linear layers? optimizer: # adamw learning_rate: 6.0e-4 # max learning rate max_iters: 600000 # total number of training iterations weight_decay: 1.0e-2 beta1: 0.9 beta2: 0.95 grad_clip: 1.0 # clip gradients at this value, or disable if == 0.0 decay_lr: True # whether to decay the learning rate warmup_iters: 2000 # how many steps to warm up for lr_decay_iters: 600000 # should be ~= max_iters per Chinchilla min_lr: 6.0e-5 # minimum learning rate, should be ~= learning_rate/10 per Chinchilla DDP: backend: nccl # 'nccl', 'gloo', etc. system: device: cuda # examples: 'cpu', 'cuda', 'cuda:0', 'cuda:1' etc., or try 'mps' on macbooks dtype: float16 # 'float32', 'bfloat16', or 'float16', the latter will auto implement a GradScaler compile: False # use PyTorch 2.0 to compile the model to be faster ================================================ FILE: config/config_rl.yaml ================================================ algorithm: method: gumbel # pg or gumbel hard_code_reward: False # use a learned reward model or hard code reward (latter does not work with Gumbel) separate_reward_model: True # when using a reward model, instantiate it separately rather than share params with LM discrete_reward: True # reward output is 0 or 1 sample if True, otherwise reward is continuous episode_length: 32 IO: out_dir: out eval_interval: 100 log_interval: 1 eval_iters: 200 eval_only: False # if True, script exits right after the first eval always_save_checkpoint: True # if True, always save a checkpoint after each eval init_from: scratch # 'scratch' or 'resume' or 'gpt2*' init_multihead_from: scratch out_dir_multihead: out_reward # used if restoring multihead wandb: wandb_log: False # disabled by default wandb_project: rlhf # 'gpt2' wandb_run_name: gpt2 # 'run' + str(time.time()) data: dataset: shakespeare # 'openwebtext', 'shakespeare', 'openai_summarize_tldr' gradient_accumulation_steps: 1 # used to simulate larger batch sizes batch_size: 12 # if gradient_accumulation_steps > 1, this is the micro-batch size block_size: 32 model: n_layer: 2 n_head: 2 n_embd: 32 dropout: 0.0 # for pretraining 0 is good, for finetuning try 0.1+ bias: False # do we use bias inside LayerNorm and Linear layers? optimizer: # adamw learning_rate: 6.0e-4 # max learning rate max_iters: 600000 # total number of training iterations weight_decay: 1.0e-2 beta1: 0.9 beta2: 0.95 grad_clip: 1.0 # clip gradients at this value, or disable if == 0.0 decay_lr: True # whether to decay the learning rate warmup_iters: 2000 # how many steps to warm up for lr_decay_iters: 600000 # should be ~= max_iters per Chinchilla min_lr: 6.0e-5 # minimum learning rate, should be ~= learning_rate/10 per Chinchilla DDP: backend: nccl # 'nccl', 'gloo', etc. system: device: cuda # examples: 'cpu', 'cuda', 'cuda:0', 'cuda:1' etc., or try 'mps' on macbooks dtype: float16 # 'float32', 'bfloat16', or 'float16', the latter will auto implement a GradScaler compile: False # use PyTorch 2.0 to compile the model to be faster ================================================ FILE: config/eval_gpt2.py ================================================ # evaluate the base gpt2 # n_layer=12, n_head=12, n_embd=768 # 124M parameters batch_size = 8 eval_iters = 500 # use more iterations to get good estimate eval_only = True wandb_log = False init_from = 'gpt2' ================================================ FILE: config/eval_gpt2_large.py ================================================ # evaluate the base gpt2 # n_layer=36, n_head=20, n_embd=1280 # 774M parameters batch_size = 8 eval_iters = 500 # use more iterations to get good estimate eval_only = True wandb_log = False init_from = 'gpt2-large' ================================================ FILE: config/eval_gpt2_medium.py ================================================ # evaluate the base gpt2 # n_layer=24, n_head=16, n_embd=1024 # 350M parameters batch_size = 8 eval_iters = 500 # use more iterations to get good estimate eval_only = True wandb_log = False init_from = 'gpt2-medium' ================================================ FILE: config/eval_gpt2_xl.py ================================================ # evaluate the base gpt2 # n_layer=48, n_head=25, n_embd=1600 # 1558M parameters batch_size = 8 eval_iters = 500 # use more iterations to get good estimate eval_only = True wandb_log = False init_from = 'gpt2-xl' ================================================ FILE: config/finetune_shakespeare.py ================================================ import time out_dir = 'out-shakespeare' eval_interval = 5 eval_iters = 40 wandb_log = False # feel free to turn on wandb_project = 'shakespeare' wandb_run_name = 'ft-' + str(time.time()) dataset = 'shakespeare' init_from = 'gpt2-xl' # this is the largest GPT-2 model # only save checkpoints if the validation loss improves always_save_checkpoint = False # the number of examples per iter: # 1 batch_size * 32 grad_accum * 1024 tokens = 32,768 tokens/iter # shakespeare has 301,966 tokens, so 1 epoch ~= 9.2 iters batch_size = 1 gradient_accumulation_steps = 32 max_iters = 20 # finetune at constant LR learning_rate = 3e-5 decay_lr = False ================================================ FILE: config/train_gpt2.py ================================================ # config for training GPT-2 (124M) down to very nice loss of ~2.85 on 1 node of 8X A100 40GB # launch as the following (e.g. in a screen session) and wait ~5 days: # $ torchrun --standalone --nproc_per_node=8 train.py config/train_gpt2.py wandb_log = True wandb_project = 'owt' wandb_run_name='gpt2-124M' # these make the total batch size be ~0.5M # 12 batch size * 1024 block size * 5 gradaccum * 8 GPUs = 491,520 batch_size = 12 block_size = 1024 gradient_accumulation_steps = 5 # this makes total number of tokens be 300B max_iters = 600000 lr_decay_iters = 600000 # eval stuff eval_interval = 1000 eval_iters = 200 log_interval = 10 # weight decay weight_decay = 1e-1 ================================================ FILE: config/train_shakespeare_char.py ================================================ # train a miniature character-level shakespeare model # good for debugging and playing on macbooks and such out_dir = 'out-shakespeare-char' eval_interval = 250 # keep frequent because we'll overfit eval_iters = 200 log_interval = 10 # don't print too too often # we expect to overfit on this small dataset, so only save when val improves always_save_checkpoint = False wandb_log = False # override via command line if you like wandb_project = 'shakespeare-char' wandb_run_name = 'mini-gpt' dataset = 'shakespeare_char' batch_size = 64 block_size = 256 # context of up to 256 previous characters # baby GPT model :) n_layer = 6 n_head = 6 n_embd = 384 dropout = 0.2 learning_rate = 1e-3 # with baby networks can afford to go a bit higher max_iters = 5000 lr_decay_iters = 5000 # make equal to max_iters usually min_lr = 1e-4 # learning_rate / 10 usually beta2 = 0.99 # make a bit bigger because number of tokens per iter is small warmup_iters = 100 # not super necessary potentially # on macbook also add # device = 'cpu' # run on cpu only # compile = False # do not torch compile the model ================================================ FILE: configurator.py ================================================ """ Poor Man's Configurator. Probably a terrible idea. Example usage: $ python train.py config/override_file.py --batch_size=32 this will first run config/override_file.py, then override batch_size to 32 The code in this file will be run as follows from e.g. train.py: >>> exec(open('configurator.py').read()) So it's not a Python module, it's just shuttling this code away from train.py The code in this script then overrides the globals() I know people are not going to love this, I just really dislike configuration complexity and having to prepend config. to every single variable. If someone comes up with a better simple Python solution I am all ears. """ import sys from ast import literal_eval for arg in sys.argv[1:]: if '=' not in arg: # assume it's the name of a config file assert not arg.startswith('--') config_file = arg print(f"Overriding config with {config_file}:") with open(config_file) as f: print(f.read()) exec(open(config_file).read()) else: # assume it's a --key=value argument assert arg.startswith('--') key, val = arg.split('=') key = key[2:] if key in globals(): try: # attempt to eval it it (e.g. if bool, number, or etc) attempt = literal_eval(val) except (SyntaxError, ValueError): # if that goes wrong, just use the string attempt = val # ensure the types match ok assert type(attempt) == type(globals()[key]) # cross fingers print(f"Overriding: {key} = {attempt}") globals()[key] = attempt else: raise ValueError(f"Unknown config key: {key}") ================================================ FILE: data/openai_summarize_tldr/prepare.py ================================================ # saves the openwebtext dataset to a binary file for training. following was helpful: # https://github.com/HazyResearch/flash-attention/blob/main/training/src/datamodules/language_modeling_hf.py import os from tqdm import tqdm import numpy as np import tiktoken from datasets import load_dataset # huggingface datasets # number of workers in .map() call # good number to use is ~order number of cpu cores // 2 num_proc = 16 # takes 54GB in huggingface .cache dir, about 8M documents (8,013,769) dataset = load_dataset("CarperAI/openai_summarize_tldr") # class TLDRDataset(Dataset): # def __init__(self, split): # self.text = [] # dataset = load_dataset(train_path, split=split) # for sample in dataset: # self.text.append(sample["prompt"] + sample["label"]) # # if "valid" in train_path: # # self.post_list = self.post_list[0:2000] # # self.tokenizer = tokenizer # # self.max_length = max_length # # self.input_ids = [] # # self.attn_masks = [] # def __len__(self): # return len(self.text) # def __getitem__(self, idx): # txt = self.text[idx] # # encodings_dict = self.tokenizer(txt, truncation=True, max_length=self.max_length, padding="max_length") # # input_ids = torch.tensor(encodings_dict["input_ids"]) # # attn_masks = torch.tensor(encodings_dict["attention_mask"]) # return { # # "input_ids": input_ids, # # "attention_mask": attn_masks, # # "labels": input_ids, # } # dataset = TLDRDataset(split="train") train_text_list = [] for sample in dataset['train']: train_text_list.append(sample['prompt'] + sample['label']) dataset['train'] = dataset['train'].add_column('text', train_text_list) # add the text column to the train dataset dataset['val'] = dataset.pop('valid') # rename the valid dataset to val val_text_list = [] for sample in dataset['val']: val_text_list.append(sample['prompt'] + sample['label']) dataset['val'] = dataset['val'].add_column('text', val_text_list) # add the text column to the val dataset dataset.pop('test') # remove the test dataset split_dataset = dataset # this results in: # >>> split_dataset # DatasetDict({ # train: Dataset({ # features: ['prompt', 'label', 'text'], # num_rows: 116722 # }) # val: Dataset({ # features: ['prompt', 'label', 'text'], # num_rows: 6447 # }) # }) # we now want to tokenize the dataset. first define the encoding function (gpt2 bpe) enc = tiktoken.get_encoding("gpt2") def process(example): ids = enc.encode_ordinary(example['text']) # encode_ordinary ignores any special tokens ids.append(enc.eot_token) # add the end of text token, e.g. 50256 for gpt2 bpe # note: I think eot should be prepended not appended... hmm. it's called "eot" though... out = {'ids': ids, 'len': len(ids)} return out # tokenize the dataset tokenized = split_dataset.map( process, remove_columns=['text','prompt','label'], desc="tokenizing the splits", num_proc=num_proc, ) # concatenate all the ids in each dataset into one large file we can use for training for split, dset in tokenized.items(): arr_len = np.sum(dset['len']) filename = os.path.join(os.path.dirname(__file__), f'{split}.bin') dtype = np.uint16 # (can do since enc.max_token_value == 50256 is < 2**16) arr = np.memmap(filename, dtype=dtype, mode='w+', shape=(arr_len,)) print(f"writing {filename}...") idx = 0 for example in tqdm(dset): arr[idx : idx + example['len']] = example['ids'] idx += example['len'] arr.flush() # train.bin is ~17GB, val.bin ~8.5MB # train has ~9B tokens (9,035,582,198) # val has ~4M tokens (4,434,897) # to read the bin files later, e.g. with numpy: # m = np.memmap('train.bin', dtype=np.uint16, mode='r') ================================================ FILE: data/openwebtext/prepare.py ================================================ # saves the openwebtext dataset to a binary file for training. following was helpful: # https://github.com/HazyResearch/flash-attention/blob/main/training/src/datamodules/language_modeling_hf.py import os from tqdm import tqdm import numpy as np import tiktoken from datasets import load_dataset # huggingface datasets # number of workers in .map() call # good number to use is ~order number of cpu cores // 2 num_proc = 8 # takes 54GB in huggingface .cache dir, about 8M documents (8,013,769) dataset = load_dataset("openwebtext") # owt by default only contains the 'train' split, so create a test split split_dataset = dataset["train"].train_test_split(test_size=0.0005, seed=2357, shuffle=True) split_dataset['val'] = split_dataset.pop('test') # rename the test split to val # this results in: # >>> split_dataset # DatasetDict({ # train: Dataset({ # features: ['text'], # num_rows: 8009762 # }) # val: Dataset({ # features: ['text'], # num_rows: 4007 # }) # }) # we now want to tokenize the dataset. first define the encoding function (gpt2 bpe) enc = tiktoken.get_encoding("gpt2") def process(example): ids = enc.encode_ordinary(example['text']) # encode_ordinary ignores any special tokens ids.append(enc.eot_token) # add the end of text token, e.g. 50256 for gpt2 bpe # note: I think eot should be prepended not appended... hmm. it's called "eot" though... out = {'ids': ids, 'len': len(ids)} return out # tokenize the dataset tokenized = split_dataset.map( process, remove_columns=['text'], desc="tokenizing the splits", num_proc=num_proc, ) # concatenate all the ids in each dataset into one large file we can use for training for split, dset in tokenized.items(): arr_len = np.sum(dset['len']) filename = os.path.join(os.path.dirname(__file__), f'{split}.bin') dtype = np.uint16 # (can do since enc.max_token_value == 50256 is < 2**16) arr = np.memmap(filename, dtype=dtype, mode='w+', shape=(arr_len,)) print(f"writing {filename}...") idx = 0 for example in tqdm(dset): arr[idx : idx + example['len']] = example['ids'] idx += example['len'] arr.flush() # train.bin is ~17GB, val.bin ~8.5MB # train has ~9B tokens (9,035,582,198) # val has ~4M tokens (4,434,897) # to read the bin files later, e.g. with numpy: # m = np.memmap('train.bin', dtype=np.uint16, mode='r') ================================================ FILE: data/openwebtext/readme.md ================================================ ## openwebtext dataset after running `prepare.py` (preprocess) we get: - train.bin is ~17GB, val.bin ~8.5MB - train has ~9B tokens (9,035,582,198) - val has ~4M tokens (4,434,897) this came from 8,013,769 documents in total. references: - OpenAI's WebText dataset is discussed in [GPT-2 paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) - [OpenWebText](https://skylion007.github.io/OpenWebTextCorpus/) dataset ================================================ FILE: data/shakespeare/prepare.py ================================================ import os import requests import tiktoken import numpy as np # download the tiny shakespeare dataset input_file_path = os.path.join(os.path.dirname(__file__), 'input.txt') if not os.path.exists(input_file_path): data_url = 'https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt' with open(input_file_path, 'w') as f: f.write(requests.get(data_url).text) with open(input_file_path, 'r') as f: data = f.read() n = len(data) train_data = data[:int(n*0.9)] val_data = data[int(n*0.9):] # encode with tiktoken gpt2 bpe enc = tiktoken.get_encoding("gpt2") train_ids = enc.encode_ordinary(train_data) val_ids = enc.encode_ordinary(val_data) print(f"train has {len(train_ids):,} tokens") print(f"val has {len(val_ids):,} tokens") # export to bin files train_ids = np.array(train_ids, dtype=np.uint16) val_ids = np.array(val_ids, dtype=np.uint16) train_ids.tofile(os.path.join(os.path.dirname(__file__), 'train.bin')) val_ids.tofile(os.path.join(os.path.dirname(__file__), 'val.bin')) # train.bin has 301,966 tokens # val.bin has 36,059 tokens ================================================ FILE: data/shakespeare/readme.md ================================================ # tiny shakespeare Tiny shakespeare, of the good old char-rnn fame :) After running `prepare.py`: - train.bin has 301,966 tokens - val.bin has 36,059 tokens ================================================ FILE: data/shakespeare_char/prepare.py ================================================ """ Prepare the Shakespeare dataset for character-level language modeling. So instead of encoding with GPT-2 BPE tokens, we just map characters to ints. Will save train.bin, val.bin containing the ids, and meta.pkl containing the encoder and decoder and some other related info. """ import os import pickle import requests import numpy as np # download the tiny shakespeare dataset input_file_path = os.path.join(os.path.dirname(__file__), 'input.txt') if not os.path.exists(input_file_path): data_url = 'https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt' with open(input_file_path, 'w') as f: f.write(requests.get(data_url).text) with open(input_file_path, 'r') as f: data = f.read() print(f"length of dataset in characters: {len(data):,}") # get all the unique characters that occur in this text chars = sorted(list(set(data))) vocab_size = len(chars) print("all the unique characters:", ''.join(chars)) print(f"vocab size: {vocab_size:,}") # create a mapping from characters to integers stoi = { ch:i for i,ch in enumerate(chars) } itos = { i:ch for i,ch in enumerate(chars) } def encode(s): return [stoi[c] for c in s] # encoder: take a string, output a list of integers def decode(l): ''.join([itos[i] for i in l]) # decoder: take a list of integers, output a string # create the train and test splits n = len(data) train_data = data[:int(n*0.9)] val_data = data[int(n*0.9):] # encode both to integers train_ids = encode(train_data) val_ids = encode(val_data) print(f"train has {len(train_ids):,} tokens") print(f"val has {len(val_ids):,} tokens") # export to bin files train_ids = np.array(train_ids, dtype=np.uint16) val_ids = np.array(val_ids, dtype=np.uint16) train_ids.tofile(os.path.join(os.path.dirname(__file__), 'train.bin')) val_ids.tofile(os.path.join(os.path.dirname(__file__), 'val.bin')) # save the meta information as well, to help us encode/decode later meta = { 'vocab_size': vocab_size, 'itos': itos, 'stoi': stoi, } with open(os.path.join(os.path.dirname(__file__), 'meta.pkl'), 'wb') as f: pickle.dump(meta, f) # length of dataset in characters: 1115394 # all the unique characters: # !$&',-.3:;?ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz # vocab size: 65 # train has 1003854 tokens # val has 111540 tokens ================================================ FILE: data/shakespeare_char/readme.md ================================================ # tiny shakespeare, character-level Tiny shakespeare, of the good old char-rnn fame :) Treated on character-level. After running `prepare.py`: - train.bin has 1,003,854 tokens - val.bin has 111,540 tokens ================================================ FILE: model.py ================================================ """ Full definition of a GPT Language Model, all of it in this single file. References: 1) the official GPT-2 TensorFlow implementation released by OpenAI: https://github.com/openai/gpt-2/blob/master/src/model.py 2) huggingface/transformers PyTorch implementation: https://github.com/huggingface/transformers/blob/main/src/transformers/models/gpt2/modeling_gpt2.py """ import math import inspect from dataclasses import dataclass import torch import torch.nn as nn from torch.nn import functional as F # @torch.jit.script # good to enable when not using torch.compile, disable when using (our default) def new_gelu(x): """ Implementation of the GELU activation function currently in Google BERT repo (identical to OpenAI GPT). Reference: Gaussian Error Linear Units (GELU) paper: https://arxiv.org/abs/1606.08415 """ return 0.5 * x * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (x + 0.044715 * torch.pow(x, 3.0)))) class LayerNorm(nn.Module): """ LayerNorm but with an optional bias. PyTorch doesn't support simply bias=False """ def __init__(self, ndim, bias): super().__init__() self.weight = nn.Parameter(torch.ones(ndim)) self.bias = nn.Parameter(torch.zeros(ndim)) if bias else None def forward(self, input): return F.layer_norm(input, self.weight.shape, self.weight, self.bias, 1e-5) class CausalSelfAttention(nn.Module): def __init__(self, config): super().__init__() assert config.n_embd % config.n_head == 0 # key, query, value projections for all heads, but in a batch self.c_attn = nn.Linear(config.n_embd, 3 * config.n_embd, bias=config.bias) # output projection self.c_proj = nn.Linear(config.n_embd, config.n_embd, bias=config.bias) # regularization self.attn_dropout = nn.Dropout(config.dropout) self.resid_dropout = nn.Dropout(config.dropout) self.n_head = config.n_head self.n_embd = config.n_embd self.dropout = config.dropout # flash attention make GPU go brrrrr but support is only in PyTorch nightly and still a bit scary self.flash = hasattr(torch.nn.functional, 'scaled_dot_product_attention') and self.dropout == 0.0 if not self.flash: print("WARNING: using slow attention. Flash Attention atm needs PyTorch nightly and dropout=0.0") # causal mask to ensure that attention is only applied to the left in the input sequence self.register_buffer("bias", torch.tril(torch.ones(config.block_size, config.block_size)) .view(1, 1, config.block_size, config.block_size)) def forward(self, x): B, T, C = x.size() # batch size, sequence length, embedding dimensionality (n_embd) # calculate query, key, values for all heads in batch and move head forward to be the batch dim q, k ,v = self.c_attn(x).split(self.n_embd, dim=2) k = k.view(B, T, self.n_head, C // self.n_head).transpose(1, 2) # (B, nh, T, hs) q = q.view(B, T, self.n_head, C // self.n_head).transpose(1, 2) # (B, nh, T, hs) v = v.view(B, T, self.n_head, C // self.n_head).transpose(1, 2) # (B, nh, T, hs) # causal self-attention; Self-attend: (B, nh, T, hs) x (B, nh, hs, T) -> (B, nh, T, T) if self.flash: # efficient attention using Flash Attention CUDA kernels y = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=None, dropout_p=self.dropout, is_causal=True) else: # manual implementation of attention att = (q @ k.transpose(-2, -1)) * (1.0 / math.sqrt(k.size(-1))) att = att.masked_fill(self.bias[:,:,:T,:T] == 0, float('-inf')) att = F.softmax(att, dim=-1) att = self.attn_dropout(att) y = att @ v # (B, nh, T, T) x (B, nh, T, hs) -> (B, nh, T, hs) y = y.transpose(1, 2).contiguous().view(B, T, C) # re-assemble all head outputs side by side # output projection y = self.resid_dropout(self.c_proj(y)) return y class MLP(nn.Module): def __init__(self, config): super().__init__() self.c_fc = nn.Linear(config.n_embd, 4 * config.n_embd, bias=config.bias) self.c_proj = nn.Linear(4 * config.n_embd, config.n_embd, bias=config.bias) self.dropout = nn.Dropout(config.dropout) def forward(self, x): x = self.c_fc(x) x = new_gelu(x) x = self.c_proj(x) x = self.dropout(x) return x class Block(nn.Module): def __init__(self, config): super().__init__() self.ln_1 = LayerNorm(config.n_embd, bias=config.bias) self.attn = CausalSelfAttention(config) self.ln_2 = LayerNorm(config.n_embd, bias=config.bias) self.mlp = MLP(config) def forward(self, x): x = x + self.attn(self.ln_1(x)) x = x + self.mlp(self.ln_2(x)) return x @dataclass class GPTConfig: block_size: int = 1024 vocab_size: int = 50304 # GPT-2 vocab_size of 50257, padded up to nearest multiple of 64 for efficiency n_layer: int = 12 n_head: int = 12 n_embd: int = 768 dropout: float = 0.0 bias: bool = True # True: bias in Linears and LayerNorms, like GPT-2. False: a bit better and faster class GPT(nn.Module): def __init__(self, config): super().__init__() assert config.vocab_size is not None assert config.block_size is not None self.config = config self.transformer = nn.ModuleDict(dict( wte = nn.Embedding(config.vocab_size, config.n_embd), wpe = nn.Embedding(config.block_size, config.n_embd), drop = nn.Dropout(config.dropout), h = nn.ModuleList([Block(config) for _ in range(config.n_layer)]), ln_f = LayerNorm(config.n_embd, bias=config.bias), )) self.lm_head = nn.Linear(config.n_embd, config.vocab_size, bias=False) # with weight tying when using torch.compile() some warnings get generated: # "UserWarning: functional_call was passed multiple values for tied weights. # This behavior is deprecated and will be an error in future versions" # not 100% sure what this is, so far seems to be harmless. TODO investigate self.transformer.wte.weight = self.lm_head.weight # https://paperswithcode.com/method/weight-tying # init all weights self.apply(self._init_weights) # apply special scaled init to the residual projections, per GPT-2 paper for pn, p in self.named_parameters(): if pn.endswith('c_proj.weight'): torch.nn.init.normal_(p, mean=0.0, std=0.02/math.sqrt(2 * config.n_layer)) # report number of parameters print("number of parameters: %.2fM" % (self.get_num_params()/1e6,)) def get_num_params(self, non_embedding=True): """ Return the number of parameters in the model. For non-embedding count (default), the position embeddings get subtracted. The token embeddings would too, except due to the parameter sharing these params are actually used as weights in the final layer, so we include them. """ n_params = sum(p.numel() for p in self.parameters()) if non_embedding: n_params -= self.transformer.wpe.weight.numel() return n_params def _init_weights(self, module): if isinstance(module, nn.Linear): torch.nn.init.normal_(module.weight, mean=0.0, std=0.02) if module.bias is not None: torch.nn.init.zeros_(module.bias) elif isinstance(module, nn.Embedding): torch.nn.init.normal_(module.weight, mean=0.0, std=0.02) def forward(self, idx, targets=None): device = idx.device b, t = idx.size() assert t <= self.config.block_size, f"Cannot forward sequence of length {t}, block size is only {self.config.block_size}" pos = torch.arange(0, t, dtype=torch.long, device=device).unsqueeze(0) # shape (1, t) # forward the GPT model itself tok_emb = self.transformer.wte(idx) # token embeddings of shape (b, t, n_embd) pos_emb = self.transformer.wpe(pos) # position embeddings of shape (1, t, n_embd) x = self.transformer.drop(tok_emb + pos_emb) for block in self.transformer.h: x = block(x) x = self.transformer.ln_f(x) if targets is not None: # if we are given some desired targets also calculate the loss logits = self.lm_head(x) loss = F.cross_entropy(logits.view(-1, logits.size(-1)), targets.view(-1), ignore_index=-1) else: # inference-time mini-optimization: only forward the lm_head on the very last position logits = self.lm_head(x[:, [-1], :]) # note: using list [-1] to preserve the time dim loss = None return logits, loss def crop_block_size(self, block_size): # model surgery to decrease the block size if necessary # e.g. we may load the GPT2 pretrained model checkpoint (block size 1024) # but want to use a smaller block size for some smaller, simpler model assert block_size <= self.config.block_size self.config.block_size = block_size self.transformer.wpe.weight = nn.Parameter(self.transformer.wpe.weight[:block_size]) for block in self.transformer.h: block.attn.bias = block.attn.bias[:,:,:block_size,:block_size] @classmethod def from_pretrained(cls, model_type, override_args=None): assert model_type in {'gpt2', 'gpt2-medium', 'gpt2-large', 'gpt2-xl'} override_args = override_args or {} # default to empty dict # only dropout can be overridden see more notes below assert all(k == 'dropout' for k in override_args) from transformers import GPT2LMHeadModel print("loading weights from pretrained gpt: %s" % model_type) # n_layer, n_head and n_embd are determined from model_type config_args = { 'gpt2': dict(n_layer=12, n_head=12, n_embd=768), # 124M params 'gpt2-medium': dict(n_layer=24, n_head=16, n_embd=1024), # 350M params 'gpt2-large': dict(n_layer=36, n_head=20, n_embd=1280), # 774M params 'gpt2-xl': dict(n_layer=48, n_head=25, n_embd=1600), # 1558M params }[model_type] print("forcing vocab_size=50257, block_size=1024, bias=True") config_args['vocab_size'] = 50257 # always 50257 for GPT model checkpoints config_args['block_size'] = 1024 # always 1024 for GPT model checkpoints config_args['bias'] = True # always True for GPT model checkpoints # we can override the dropout rate, if desired if 'dropout' in override_args: print(f"overriding dropout rate to {override_args['dropout']}") config_args['dropout'] = override_args['dropout'] # create a from-scratch initialized minGPT model config = GPTConfig(**config_args) model = GPT(config) sd = model.state_dict() sd_keys = sd.keys() sd_keys = [k for k in sd_keys if not k.endswith('.attn.bias')] # discard this mask / buffer, not a param # init a huggingface/transformers model model_hf = GPT2LMHeadModel.from_pretrained(model_type) sd_hf = model_hf.state_dict() # copy while ensuring all of the parameters are aligned and match in names and shapes sd_keys_hf = sd_hf.keys() sd_keys_hf = [k for k in sd_keys_hf if not k.endswith('.attn.masked_bias')] # ignore these, just a buffer sd_keys_hf = [k for k in sd_keys_hf if not k.endswith('.attn.bias')] # same, just the mask (buffer) transposed = ['attn.c_attn.weight', 'attn.c_proj.weight', 'mlp.c_fc.weight', 'mlp.c_proj.weight'] # basically the openai checkpoints use a "Conv1D" module, but we only want to use a vanilla Linear # this means that we have to transpose these weights when we import them assert len(sd_keys_hf) == len(sd_keys), f"mismatched keys: {len(sd_keys_hf)} != {len(sd_keys)}" for k in sd_keys_hf: if any(k.endswith(w) for w in transposed): # special treatment for the Conv1D weights we need to transpose assert sd_hf[k].shape[::-1] == sd[k].shape with torch.no_grad(): sd[k].copy_(sd_hf[k].t()) else: # vanilla copy over the other parameters assert sd_hf[k].shape == sd[k].shape with torch.no_grad(): sd[k].copy_(sd_hf[k]) return model def configure_optimizers(self, weight_decay, learning_rate, betas, device_type): """ This long function is unfortunately doing something very simple and is being very defensive: We are separating out all parameters of the model into two buckets: those that will experience weight decay for regularization and those that won't (biases, and layernorm/embedding weights). We are then returning the PyTorch optimizer object. """ # separate out all parameters to those that will and won't experience regularizing weight decay decay = set() no_decay = set() whitelist_weight_modules = (torch.nn.Linear, ) blacklist_weight_modules = (torch.nn.LayerNorm, LayerNorm, torch.nn.Embedding) for mn, m in self.named_modules(): for pn, p in m.named_parameters(): fpn = '%s.%s' % (mn, pn) if mn else pn # full param name # random note: because named_modules and named_parameters are recursive # we will see the same tensors p many many times. but doing it this way # allows us to know which parent module any tensor p belongs to... if pn.endswith('bias'): # all biases will not be decayed no_decay.add(fpn) elif pn.endswith('weight') and isinstance(m, whitelist_weight_modules): # weights of whitelist modules will be weight decayed decay.add(fpn) elif pn.endswith('weight') and isinstance(m, blacklist_weight_modules): # weights of blacklist modules will NOT be weight decayed no_decay.add(fpn) # subtle: 'transformer.wte.weight' and 'lm_head.weight' are tied, so they # will appear in the no_decay and decay sets respectively after the above. # In addition, because named_parameters() doesn't return duplicates, it # will only return the first occurence, key'd by 'transformer.wte.weight', below. # so let's manually remove 'lm_head.weight' from decay set. This will include # this tensor into optimization via transformer.wte.weight only, and not decayed. decay.remove('lm_head.weight') # validate that we considered every parameter param_dict = {pn: p for pn, p in self.named_parameters()} inter_params = decay & no_decay union_params = decay | no_decay assert len(inter_params) == 0, "parameters %s made it into both decay/no_decay sets!" % (str(inter_params), ) assert len(param_dict.keys() - union_params) == 0, "parameters %s were not separated into either decay/no_decay set!" \ % (str(param_dict.keys() - union_params), ) # create the pytorch optimizer object optim_groups = [ {"params": [param_dict[pn] for pn in sorted(list(decay))], "weight_decay": weight_decay}, {"params": [param_dict[pn] for pn in sorted(list(no_decay))], "weight_decay": 0.0}, ] # new PyTorch nightly has a new 'fused' option for AdamW that is much faster use_fused = (device_type == 'cuda') and ('fused' in inspect.signature(torch.optim.AdamW).parameters) print(f"using fused AdamW: {use_fused}") extra_args = dict(fused=True) if use_fused else dict() optimizer = torch.optim.AdamW(optim_groups, lr=learning_rate, betas=betas, **extra_args) return optimizer def estimate_mfu(self, fwdbwd_per_iter, dt): """ estimate model flops utilization (MFU) in units of A100 bfloat16 peak FLOPS """ # first estimate the number of flops we do per iteration. # see PaLM paper Appendix B as ref: https://arxiv.org/abs/2204.02311 N = self.get_num_params() cfg = self.config L, H, Q, T = cfg.n_layer, cfg.n_head, cfg.n_embd//cfg.n_head, cfg.block_size flops_per_token = 6*N + 12*L*H*Q*T flops_per_fwdbwd = flops_per_token * T flops_per_iter = flops_per_fwdbwd * fwdbwd_per_iter # express our flops throughput as ratio of A100 bfloat16 peak flops flops_achieved = flops_per_iter * (1.0/dt) # per second flops_promised = 312e12 # A100 GPU bfloat16 peak flops is 312 TFLOPS mfu = flops_achieved / flops_promised return mfu @torch.no_grad() def generate(self, idx, max_new_tokens, temperature=1.0, top_k=None): """ Take a conditioning sequence of indices idx (LongTensor of shape (b,t)) and complete the sequence max_new_tokens times, feeding the predictions back into the model each time. Most likely you'll want to make sure to be in model.eval() mode of operation for this. """ for _ in range(max_new_tokens): # if the sequence context is growing too long we must crop it at block_size idx_cond = idx if idx.size(1) <= self.config.block_size else idx[:, -self.config.block_size:] # forward the model to get the logits for the index in the sequence logits, _ = self(idx_cond) # pluck the logits at the final step and scale by desired temperature logits = logits[:, -1, :] / temperature # optionally crop the logits to only the top k options if top_k is not None: v, _ = torch.topk(logits, min(top_k, logits.size(-1))) logits[logits < v[:, [-1]]] = -float('Inf') # apply softmax to convert logits to (normalized) probabilities probs = F.softmax(logits, dim=-1) # sample from the distribution idx_next = torch.multinomial(probs, num_samples=1) # append sampled index to the running sequence and continue idx = torch.cat((idx, idx_next), dim=1) return idx class RLHF(nn.Module): def __init__(self, model, mode, discrete_reward=False): super().__init__() self.model = model self.config = model.config # reward model self.n_embd = model.lm_head.in_features self.block_size = model.config.block_size model.policy_head = nn.Linear(model.lm_head.in_features, model.lm_head.out_features, bias=False) self.mode = mode self.discrete_reward = discrete_reward if discrete_reward: model.reward_head = nn.Linear(model.lm_head.in_features, 2, bias=False) else: model.reward_head = nn.Linear(self.n_embd*self.block_size, 1, bias=False) def forward_reward(self, idx, targets=None): device = idx.device b, t = idx.size() assert t <= self.config.block_size, f"Cannot forward sequence of length {t}, block size is only {self.config.block_size}" pos = torch.arange(0, t, dtype=torch.long, device=device).unsqueeze(0) # shape (1, t) # forward the GPT model itself tok_emb = self.model.transformer.wte(idx) # token embeddings of shape (b, t, n_embd) pos_emb = self.model.transformer.wpe(pos) # position embeddings of shape (1, t, n_embd) x = self.model.transformer.drop(tok_emb + pos_emb) for block in self.model.transformer.h: x = block(x) x = self.model.transformer.ln_f(x) rewards = self.model.reward_head(x[:, -1, :]) if self.discrete_reward: probs = torch.softmax(rewards,1) if targets is not None: # if we are given some desired targets also calculate the loss loss = F.cross_entropy(probs, targets, ignore_index=-1) else: loss = None return probs, loss else: return rewards def forward(self, idx, targets=None): if self.mode == 'reward': return self.forward_reward(idx, targets) else: return self.model(idx, targets) def generate(self, idx, max_new_tokens, device, block_size, use_reference=True, reward_model=None, hard_code_reward=True): # idx is (B, T) array of indices in the current context log_probs = torch.tensor([]).to(device) log_probs_ref = torch.tensor([]).to(device) values = torch.tensor([]).to(device) idx_cond_all = torch.zeros((idx.shape[0], block_size, max_new_tokens)).to(device) values_all = torch.zeros((idx.shape[0], max_new_tokens)).to(device) actions_all = torch.zeros((idx.shape[0], max_new_tokens)).to(device) rewards_all = torch.zeros((idx.shape[0],)).to(device) log_probs_all = torch.zeros((idx.shape[0], max_new_tokens)).to(device) advantages_all = torch.zeros((idx.shape[0], max_new_tokens)).to(device) returns_all = torch.zeros((idx.shape[0], max_new_tokens)).to(device) gamma = 1 lam = 1 # TODO: Critic, PPO for i in range(max_new_tokens): # crop idx to the last block_size tokens # block_size = 256 idx_cond = idx[:, -block_size:] # get the predictions logits, loss = self(idx_cond) # focus only on the last time step logits = logits[:, -1, :] # becomes (B, C) # apply softmax to get probabilities probs_next = F.softmax(logits, dim=-1) # (B, C) # sample from the distribution idx_next = torch.multinomial(probs_next, num_samples=1) # (B, 1) probs_idx_next = torch.gather(probs_next, 1, idx_next) log_probs_idx_next = torch.log(probs_idx_next) log_probs = torch.cat((log_probs, log_probs_idx_next), dim=1) if use_reference: logits_ref, _ = self.model(idx_cond) logits_ref = logits_ref[:, -1, :] # becomes (B, C) probs_ref_next = F.softmax(logits_ref, dim=-1) # (B, C) probs_ref_idx_next = torch.gather(probs_ref_next, 1, idx_next) log_probs_ref_idx_next = torch.log(probs_ref_idx_next) log_probs_ref = torch.cat((log_probs_ref, log_probs_ref_idx_next), dim=1) # append sampled index to the running sequence idx = torch.cat((idx, idx_next), dim=1) # (B, T+1) if i == max_new_tokens-1: states = idx[:,-max_new_tokens:] if hard_code_reward: # simple test where reward for outputting the letter 'z' (89) rewards = torch.zeros_like(states, dtype=torch.float16) rewards[states==89] = 1.0 rewards = torch.sum(rewards, 1, keepdim=True) rewards[rewards > 1] = 1 else: if self.discrete_reward: rewards = reward_model.forward_reward(torch.tensor(states))[0][:,1].unsqueeze(-1) else: rewards = reward_model.forward_reward(torch.tensor(states)) for t in reversed(range(max_new_tokens)): if t == max_new_tokens - 1: # value at last state is 0 delta = rewards[:].squeeze() - values_all[:, t] advantages_all[:, t] = delta # returns_all[:, t] = rewards[:] else: # rewards can only be non-zero at the last state delta = gamma * values_all[:, t + 1] - values_all[:, t] advantages_all[:, t] = delta + gamma * lam * advantages_all[:, t + 1] # returns_all[:, t] += gamma * returns_all[:, t + 1] return idx, log_probs[:,-max_new_tokens:], log_probs_ref[:,-max_new_tokens:], rewards, advantages_all def generate_gumbel(self, idx, max_new_tokens, device, block_size, reward_model, use_reference=True): onehot = torch.tensor([]).to(device) for i in range(max_new_tokens): # crop idx to the last block_size tokens # block_size = 256 idx_cond = idx[:, -block_size:] # get the predictions logits, loss = self(idx_cond) # focus only on the last time step logits = logits[:, -1, :] # becomes (B, C) #gumbel sample idx_next, onehot_next = self.gumbel_softmax(logits, tau=1, device=idx.device) # append sampled index to the running sequence idx = torch.cat((idx, idx_next), dim=1) # (B, T+1) onehot = torch.cat((onehot, onehot_next), dim=2) # (B, T+1) if i == max_new_tokens-1: if self.discrete_reward: rewards = reward_model.forward_reward_gumbel(onehot)[0][:,1].unsqueeze(-1) else: rewards = reward_model.forward_reward_gumbel(onehot) return idx[:,-max_new_tokens:], rewards # modified for PyTorch from https://github.com/ericjang/gumbel-softmax/blob/master/Categorical%20VAE.ipynb def sample_gumbel(self, shape, eps=1e-20): """Sample from Gumbel(0, 1)""" U = torch.distributions.Uniform(0,1).sample(shape) return -torch.log(-torch.log(U + eps) + eps) # modified for PyTorch from https://github.com/ericjang/gumbel-softmax/blob/master/Categorical%20VAE.ipynb def gumbel_softmax_sample(self, logits, tau, device, dim=1): """ Draw a sample from the Gumbel-Softmax distribution""" y = logits + self.sample_gumbel(logits.shape).to(device) return F.softmax(y / tau, dim=dim) def gumbel_softmax(self, logits, tau, device): gumbel_sample = self.gumbel_softmax_sample(logits, tau, device) # Alternatively could try # probs = F.softmax(gumbel_sample, dim=-1) # idx_next = torch.multinomial(probs, num_samples=1) idx_next = gumbel_sample.max(-1, keepdim=True)[1] onehot_idx_next = torch.nn.functional.one_hot(idx_next, num_classes=logits.shape[1]).squeeze() y = (onehot_idx_next-gumbel_sample).detach() + gumbel_sample idx_next_from_y = torch.argmax(y, dim=1).unsqueeze(-1) return idx_next_from_y, y.unsqueeze(-1) def forward_reward_gumbel(self, onehots, idx=None, targets=None): # (embd, vocab) @ (vocab, embd) = (embd,embd) device = onehots.device t = onehots.shape[2] b = onehots.shape[0] # b, t = idx.size() # assert t <= self.config.block_size, f"Cannot forward sequence of length {t}, block size is only {self.config.block_size}" pos = torch.arange(0, t, dtype=torch.long, device=device).unsqueeze(0) # shape (1, t) # forward the GPT model itself # tok_emb = self.model.transformer.wte(idx) # token embeddings of shape (b, t, n_embd) tok_emb = (self.model.transformer.wte.weight.T @ onehots) tok_emb = torch.transpose(tok_emb, 1, 2) if idx is not None: tok_emb2 = self.model.transformer.wte(idx) # token embeddings of shape (b, t, n_embd) assert torch.all(tok_emb == tok_emb2) pos_emb = self.model.transformer.wpe(pos) # position embeddings of shape (1, t, n_embd) x = self.model.transformer.drop(tok_emb + pos_emb) for block in self.model.transformer.h: x = block(x) x = self.model.transformer.ln_f(x) rewards = self.model.reward_head(x[:, -1, :]) if self.discrete_reward: probs = torch.softmax(rewards,1) if targets is not None: # if we are given some desired targets also calculate the loss loss = F.cross_entropy(rewards, targets, ignore_index=-1) else: loss = None return probs, loss else: return rewards ================================================ FILE: requirements.txt ================================================ tiktoken numpy contextlib torch wandb ================================================ FILE: sample.py ================================================ """ Sample from a trained model """ import os import pickle from contextlib import nullcontext import torch import tiktoken from model import GPTConfig, GPT # ----------------------------------------------------------------------------- init_from = 'resume' # either 'resume' (from an out_dir) or a gpt2 variant (e.g. 'gpt2-xl') out_dir = 'out' # ignored if init_from is not 'resume' start = "TITLE" # or "<|endoftext|>" or etc. Can also specify a file, use as: "FILE:prompt.txt" num_samples = 5 # number of samples to draw max_new_tokens = 500 # number of tokens generated in each sample temperature = 0.8 # 1.0 = no change, < 1.0 = less random, > 1.0 = more random, in predictions top_k = 200 # retain only the top_k most likely tokens, clamp others to have 0 probability seed = 1337 device = 'cpu' # examples: 'cpu', 'cuda', 'cuda:0', 'cuda:1', etc. dtype = 'float16' # 'float32' or 'bfloat16' or 'float16' compile = False # use PyTorch 2.0 to compile the model to be faster exec(open('configurator.py').read()) # overrides from command line or config file # ----------------------------------------------------------------------------- torch.manual_seed(seed) torch.cuda.manual_seed(seed) torch.backends.cuda.matmul.allow_tf32 = True # allow tf32 on matmul torch.backends.cudnn.allow_tf32 = True # allow tf32 on cudnn device_type = 'cuda' if 'cuda' in device else 'cpu' # for later use in torch.autocast ptdtype = {'float32': torch.float32, 'bfloat16': torch.bfloat16, 'float16': torch.float16}[dtype] ctx = nullcontext() if device_type == 'cpu' else torch.amp.autocast(device_type=device_type, dtype=ptdtype) # model if init_from == 'resume': # init from a model saved in a specific directory ckpt_path = os.path.join(out_dir, 'ckpt.pt') checkpoint = torch.load(ckpt_path, map_location=device) gptconf = GPTConfig(**checkpoint['model_args']) model = GPT(gptconf) state_dict = checkpoint['model'] unwanted_prefix = '_orig_mod.' for k,v in list(state_dict.items()): if k.startswith(unwanted_prefix): state_dict[k[len(unwanted_prefix):]] = state_dict.pop(k) model.load_state_dict(state_dict) elif init_from.startswith('gpt2'): # init from a given GPT-2 model model = GPT.from_pretrained(init_from, dict(dropout=0.0)) model.eval() model.to(device) if compile: model = torch.compile(model) # requires PyTorch 2.0 (optional) # look for the meta pickle in case it is available in the dataset folder load_meta = False if init_from == 'resume' and 'config' in checkpoint and 'dataset' in checkpoint['config']: # older checkpoints might not have these... meta_path = os.path.join('data', checkpoint['config']['dataset'], 'meta.pkl') load_meta = os.path.exists(meta_path) if load_meta: print(f"Loading meta from {meta_path}...") with open(meta_path, 'rb') as f: meta = pickle.load(f) # TODO want to make this more general to arbitrary encoder/decoder schemes stoi, itos = meta['stoi'], meta['itos'] encode = lambda s: [stoi[c] for c in s] decode = lambda l: ''.join([itos[i] for i in l]) else: # ok let's assume gpt-2 encodings by default print("No meta.pkl found, assuming GPT-2 encodings...") enc = tiktoken.get_encoding("gpt2") encode = lambda s: enc.encode(s, allowed_special={"<|endoftext|>"}) decode = lambda l: enc.decode(l) # encode the beginning of the prompt if start.startswith('FILE:'): with open(start[5:], 'r', encoding='utf-8') as f: start = f.read() start_ids = encode(start) x = (torch.tensor(start_ids, dtype=torch.long, device=device)[None, ...]) # run generation with torch.no_grad(): with ctx: for k in range(num_samples): y = model.generate(x, max_new_tokens, temperature=temperature, top_k=top_k) print(decode(y[0].tolist())) print('---------------') ================================================ FILE: scaling_laws.ipynb ================================================ { "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Reproducing some scaling laws results from [Chinchilla](https://arxiv.org/pdf/2203.15556.pdf). Can't get the numbers to match exactly, but can still be used as a rough guide to help determine compute-optimal models. Also contains related utilities for calculating flops and param counts." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import pandas as pd\n", "%matplotlib inline" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## params\n", "\n", "First some parameter calculations:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "123.653376" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "def gpt_params(seq_len, vocab_size, d_model, num_heads, num_layers):\n", " \"\"\" Given GPT config calculate total number of parameters \"\"\"\n", " ffw_size = 4*d_model # in GPT the number of intermediate features is always 4*d_model\n", " # token and position embeddings\n", " embeddings = d_model * vocab_size + d_model * seq_len\n", " # transformer blocks\n", " attention = 3*d_model**2 + 3*d_model # weights and biases\n", " attproj = d_model**2 + d_model\n", " ffw = d_model*(ffw_size) + ffw_size\n", " ffwproj = ffw_size*d_model + d_model\n", " layernorms = 2*2*d_model\n", " # dense\n", " ln_f = 2*d_model\n", " dense = d_model*vocab_size # note: no bias here\n", " # note: embeddings are not included in the param count!\n", " total_params = num_layers*(attention + attproj + ffw + ffwproj + layernorms) + ln_f + dense\n", " return total_params\n", "\n", "gpt2 = dict(seq_len = 1024, vocab_size = 50257, d_model = 768, num_heads = 12, num_layers = 12)\n", "gpt_params(**gpt2)/1e6" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "OpenAI reports gpt2 (small) as having 124M params, so this is a match. Also, loading the OpenAI weights into nanoGPT and then calling `model.parameters()` exactly matches the above number and verifies the implementation. Now Chinchilla parameters:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "def chinchilla_params(seq_len, vocab_size, d_model, num_heads, num_layers, ffw_size):\n", " \"\"\" Parameters in the Chinchilla models. Unlike GPT they use relative positional embeddings. \"\"\"\n", " # token embeddings only\n", " embeddings = d_model * vocab_size\n", " # transformer blocks\n", " attention = 3*d_model**2 + 3*d_model # weights and biases\n", " relative_pos = d_model**2 + 2*d_model # relative keys, content bias, relative bias\n", " attproj = d_model**2 + d_model\n", " ffw = d_model*ffw_size + ffw_size\n", " ffwproj = ffw_size*d_model + d_model\n", " layernorms = 2*2*d_model\n", " # dense\n", " ln_f = 2*d_model\n", " dense = d_model*vocab_size # note: no bias here\n", " # note: embeddings are not included in the param count!\n", " total_params = num_layers*(attention + relative_pos + attproj + ffw + ffwproj + layernorms) + ln_f + dense\n", " return total_params\n", "\n" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[44000000.0, 512, 2048, 64, 8, 8]" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Load in all the 50 Chinchilla models on the last page of the paper\n", "import json\n", "chinchilla_models_txt = '[[44000000.0, 512, 2048, 64, 8, 8], [57000000.0, 576, 2304, 64, 9, 9], [74000000.0, 640, 2560, 64, 10, 10], [90000000.0, 640, 2560, 64, 10, 13], [106000000.0, 640, 2560, 64, 10, 16], [117000000.0, 768, 3072, 64, 12, 12], [140000000.0, 768, 3072, 64, 12, 15], [163000000.0, 768, 3072, 64, 12, 18], [175000000.0, 896, 3584, 64, 14, 14], [196000000.0, 896, 3584, 64, 14, 16], [217000000.0, 896, 3584, 64, 14, 18], [251000000.0, 1024, 4096, 64, 16, 16], [278000000.0, 1024, 4096, 64, 16, 18], [306000000.0, 1024, 4096, 64, 16, 20], [425000000.0, 1280, 5120, 128, 10, 18], [489000000.0, 1280, 5120, 128, 10, 21], [509000000.0, 1408, 5632, 128, 11, 18], [552000000.0, 1280, 5120, 128, 10, 24], [587000000.0, 1408, 5632, 128, 11, 21], [632000000.0, 1536, 6144, 128, 12, 19], [664000000.0, 1408, 5632, 128, 11, 24], [724000000.0, 1536, 6144, 128, 12, 22], [816000000.0, 1536, 6144, 128, 12, 25], [893000000.0, 1792, 7168, 128, 14, 20], [1018000000.0, 1792, 7168, 128, 14, 23], [1143000000.0, 1792, 7168, 128, 14, 26], [1266000000.0, 2048, 8192, 128, 16, 22], [1424000000.0, 2176, 8704, 128, 17, 22], [1429000000.0, 2048, 8192, 128, 16, 25], [1593000000.0, 2048, 8192, 128, 16, 28], [1609000000.0, 2176, 8704, 128, 17, 25], [1731000000.0, 2304, 9216, 128, 18, 24], [1794000000.0, 2176, 8704, 128, 17, 28], [2007000000.0, 2304, 9216, 128, 18, 28], [2283000000.0, 2304, 9216, 128, 18, 32], [2298000000.0, 2560, 10240, 128, 20, 26], [2639000000.0, 2560, 10240, 128, 20, 30], [2980000000.0, 2560, 10240, 128, 20, 34], [3530000000.0, 2688, 10752, 128, 22, 36], [3802000000.0, 2816, 11264, 128, 22, 36], [4084000000.0, 2944, 11776, 128, 22, 36], [4516000000.0, 3072, 12288, 128, 24, 36], [6796000000.0, 3584, 14336, 128, 28, 40], [9293000000.0, 4096, 16384, 128, 32, 42], [11452000000.0, 4352, 17408, 128, 32, 47], [12295000000.0, 4608, 18432, 128, 36, 44], [12569000000.0, 4608, 18432, 128, 32, 47], [13735000000.0, 4864, 19456, 128, 32, 47], [14940000000.0, 4992, 19968, 128, 32, 49], [16183000000.0, 5120, 20480, 128, 40, 47]]'\n", "chilchilla_models = json.loads(chinchilla_models_txt) # all 50 models\n", "chilchilla_models[0] # tuples of params, d_model, ffw_size, kv_size, n_heads, n_layers from Table A9" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "our estimated params: 12296.1623M, chinchilla params: 12295.0000M, d_model: 4608, n_heads: 36, n_layers: 44\n", "our estimated params: 13124.4826M, chinchilla params: 12569.0000M, d_model: 4608, n_heads: 32, n_layers: 47\n", "our estimated params: 14614.4279M, chinchilla params: 13735.0000M, d_model: 4864, n_heads: 32, n_layers: 47\n", "our estimated params: 16037.5039M, chinchilla params: 14940.0000M, d_model: 4992, n_heads: 32, n_layers: 49\n", "our estimated params: 16184.4582M, chinchilla params: 16183.0000M, d_model: 5120, n_heads: 40, n_layers: 47\n" ] } ], "source": [ "for m in chilchilla_models[-5:]: # only print last 5 models of the table\n", " p, d, f, k, h, l = m\n", " nparams = chinchilla_params(seq_len = 1024, vocab_size = 32000, d_model = d, num_heads = h, num_layers = l, ffw_size=f)\n", " print(f\"our estimated params: {nparams/1e6:.4f}M, chinchilla params: {p/1e6:.4f}M, d_model: {d}, n_heads: {h}, n_layers: {l}\")" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "We are almost able to reproduce the parameter counts for the Chinchilla models.\n", "\n", "Now turning to FLOPs:" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## flops" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "def chinchilla_flops(seq_len, vocab_size, d_model, num_heads, num_layers, ffw_size):\n", " \"\"\" \n", " Calculate total number of FLOPs, see Chinchilla \n", " paper Appendix F as reference: https://arxiv.org/pdf/2203.15556.pdf\n", " \"\"\" \n", " key_size = d_model // num_heads\n", "\n", " # embeddings\n", " embeddings = 2 * seq_len * vocab_size * d_model\n", "\n", " # attention\n", " # key, query, value projections\n", " attention = 2 * 3 * seq_len * d_model * (key_size * num_heads)\n", " # key @ query logits\n", " attlogits = 2 * seq_len * seq_len * (key_size * num_heads)\n", " # softmax\n", " attsoftmax = 3 * num_heads * seq_len * seq_len # 3* is for subtract (max), exp, divide (?)\n", " # softmax @ value reductions\n", " attvalue = 2 * seq_len * seq_len * (key_size * num_heads)\n", " # final linear\n", " attlinear = 2 * seq_len * (key_size * num_heads) * d_model\n", " att = attention + attlogits + attsoftmax + attvalue + attlinear\n", " # feed forward\n", " dense = 2 * seq_len * (d_model * ffw_size + d_model * ffw_size)\n", "\n", " # logits\n", " logits = 2 * seq_len * d_model * vocab_size\n", " \n", " # this is what you'd expect:\n", " # forward_flops = embeddings + num_layers * (att + dense) + logits\n", " # but:\n", " # per author correspondence apparently there is typo in the paper,\n", " # they do not count embeddings and logits to repro table 4. So instead:\n", " forward_flops = num_layers * (att + dense)\n", " backward_flops = 2 * forward_flops # as in Kaplan et al. 2020\n", " total_flops = forward_flops + backward_flops\n", "\n", " return total_flops\n" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
seq_lenvocab_sized_modelnum_headsnum_layersffw_sizeNFapprox_flopschinch_flopsratio
020483200064010102560738252809298771968009071650406409.298772e+111.025036
1204832000102416204096305707008413524819968037565277143044.135248e+121.100817
2204832000128010245120552604160735345377280067903999180807.353454e+121.082919
3204832000179214267168114345369614670316437504140507590164481.467032e+131.044094
4204832000204816288192159312691220220437594112195763434946562.022044e+131.032902
52048320003584284014336679627468883021046743040835126233661448.302105e+130.994114
\n", "
" ], "text/plain": [ " seq_len vocab_size d_model num_heads num_layers ffw_size N \\\n", "0 2048 32000 640 10 10 2560 73825280 \n", "1 2048 32000 1024 16 20 4096 305707008 \n", "2 2048 32000 1280 10 24 5120 552604160 \n", "3 2048 32000 1792 14 26 7168 1143453696 \n", "4 2048 32000 2048 16 28 8192 1593126912 \n", "5 2048 32000 3584 28 40 14336 6796274688 \n", "\n", " F approx_flops chinch_flops ratio \n", "0 929877196800 907165040640 9.298772e+11 1.025036 \n", "1 4135248199680 3756527714304 4.135248e+12 1.100817 \n", "2 7353453772800 6790399918080 7.353454e+12 1.082919 \n", "3 14670316437504 14050759016448 1.467032e+13 1.044094 \n", "4 20220437594112 19576343494656 2.022044e+13 1.032902 \n", "5 83021046743040 83512623366144 8.302105e+13 0.994114 " ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Now try reproduce Table A4 from Chinchilla paper Appendix, \n", "# comparing accurate flops above to approximate flops F = 6*N*D\n", "# note Chinchilla mentions using vocab_size = 32K\n", "\n", "chilchilla_models_table4 = [\n", " [10, 640, 2560, 10, 64],\n", " [20, 1024, 4096, 16, 64],\n", " [24, 1280, 5120, 10, 128 ],\n", " [26, 1792, 7168, 14, 128 ],\n", " [28, 2048, 8192, 16, 128],\n", " [40, 3584, 14336, 28, 128]\n", "]\n", "\n", "rows = []\n", "for num_layers, d_model, ffw_size, num_heads, _ in chilchilla_models_table4:\n", "\n", " args = dict(seq_len = 2048, vocab_size = 32000, d_model = d_model, \n", " num_heads = num_heads, num_layers = num_layers, ffw_size=ffw_size)\n", "\n", " D = args['seq_len'] # dataset size (cancels anyway, for the purposes of the ratio calculation below)\n", " N = chinchilla_params(**args)\n", " F = chinchilla_flops(**args)\n", "\n", " approx_flops = 6*D*N # approximate flops\n", " chinch_flops = F * (float(D) / args['seq_len']) # exact flops according to Chinchilla paper calculations\n", "\n", " # print('---')\n", " # print(f\"params: {N/1e6:.2f}M\")\n", " # print(f\"approx flops: {approx_flops/1e9:.2f}B\")\n", " # print(f\"chinchilla flops: {chinch_flops/1e9:.2f}B\")\n", " # print(f\"ratio (chinchilla / approx): {chinch_flops / approx_flops:.2f}\")\n", "\n", " # first copy all keyvalues from args into out\n", " out = {k:v for k,v in args.items()}\n", " # then add the calculated values\n", " out['N'] = N\n", " out['F'] = F\n", " out['approx_flops'] = approx_flops\n", " out['chinch_flops'] = chinch_flops\n", " out['ratio'] = chinch_flops / approx_flops\n", " rows.append(out)\n", "\n", "# make a pandas dataframe from rows\n", "df = pd.DataFrame(rows)\n", "df" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Pretty good match! Except the param counts are still not perfectly accurate." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Scaling Laws: Approach 3\n", "\n", "In their \"Aproach 3\", Chinchilla paper fits a function L(N,D) to approximate the final loss gives the model size and the data size. Here is the final fit:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA8UAAAHWCAYAAABe7ytwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOy9eXwb9bnv//mOdslavMu7HTv74jgLIZAWKCkJUJYWKPTQS+FwoJcboJD7u72Hvs4BSin0dDvcnkPLgVOgO5SwUxpKA4QtqxMncRYn3ld5k619saT5/TGSLNkzskYayUu+b17fVxtp5pnvjO35zjPP83wewrIsCwqFQqFQKBQKhUKhUM5DmNmeAIVCoVAoFAqFQqFQKLMFdYopFAqFQqFQKBQKhXLeQp1iCoVCoVAoFAqFQqGct1CnmEKhUCgUCoVCoVAo5y3UKaZQKBQKhUKhUCgUynkLdYopFAqFQqFQKBQKhXLeQp1iCoVCoVAoFAqFQqGct1CnmEKhUCgUCoVCoVAo5y3UKaZQKBQKhUKhUCgUynkLdYoplAS8+OKLIISgs7NztqdCoVAoFMqCZiGuuT/5yU+waNEiyGQyrF27FgBQXV2N22+/fVbnRaFQ4qFOMYVCoVAoFArlvOFXv/oVbrrpJlRWVoIQktBBHR8fx913343CwkLodDpcdtllOHLkSFLH+dvf/obvfve7uPjii/HCCy/giSeekOgMKBSK1MhnewIUCoVCoVAoFEq2+Ld/+zc4HA5ccMEFGBgYENwuFArh6quvxrFjx/B//s//QUFBAX75y1/i0ksvRWNjIxYvXpzwOB988AEYhsGvf/1rKJVKqU+DQqFICHWKKRQKhUKhUCjnDXv37o1GiXNycgS327VrFz7//HO88soruPHGGwEAX//617FkyRI88sgj+OMf/5jwOENDQ9BoNNQhplDmATR9mkIRyS9/+UusXLkSKpUKpaWl2LFjB8bHx+O2OXfuHG644QaYzWao1WqUl5fjlltugc1mi27z/vvvY8uWLTCZTMjJycHSpUvxve99L8tnQ6FQKBTK3CaZdRcAnn76aSxatAgajQYXXHABPvnkE1x66aW49NJL47arqqoCIWTG4+7atQvFxcX42te+Fv2ssLAQX//61/Hmm2/C5/MJ7ksIwQsvvACXywVCCAghePHFFwW3b29vx0033YS8vDxotVpceOGF+Mtf/hK3zUcffQRCCF5++WV873vfg9lshk6nw7XXXouenp64bZN5DqFQKJPQSDGFIoJHH30U3//+97F161bcc889aGlpwa9+9SscOnQIn332GRQKBfx+P7Zt2wafz4f77rsPZrMZfX19eOeddzA+Pg6j0YiTJ0/iK1/5CtasWYPHHnsMKpUKra2t+Oyzz2b7FCkUCoVCmTMks+4CXJ3wvffeiy984Qt48MEH0dnZieuvvx65ubkoLy9P6dhHjx7FunXrwDDxMaQLLrgAzz77LM6ePYvVq1fz7vu73/0Ozz77LA4ePIj//u//BgBcdNFFvNsODg7ioosugtvtxv3334/8/Hz85je/wbXXXotdu3bhq1/9atz2P/zhD0EIwf/9v/8XQ0NDeOqpp7B161Y0NTVBo9Ek9RxCoVCmwFIoFEFeeOEFFgDb0dHBDg0NsUqlkr3iiivYYDAY3eY///M/WQDs888/z7Isyx49epQFwL7yyiuCdv/93/+dBcAODw9n/BwoFAqFQpkPxK65LMsmve76fD42Pz+f3bhxIzsxMRHd7sUXX2QBsJdccongMXU6Hfutb31L8Lt//Md/nPb5X/7yFxYAu3v37oTn861vfYvV6XTTPq+qqoo75gMPPMACYD/55JPoZw6Hg62pqWGrq6uj5/7hhx+yANiysjLWbrdHt/3zn//MAmD/3//7fyzLJvccQqFQ4qHp0xRKkvz973+H3+/HAw88EPfW+K677oLBYIimOUXewL733ntwu928tkwmEwDgzTffRCgUyuzEKRQKhUKZhyS77h4+fBijo6O46667IJdPJkHeeuutyM3NTfn4Ho8HKpVq2udqtTr6vRS8++67uOCCC7Bly5boZzk5Obj77rvR2dmJU6dOxW1/2223Qa/XR/994403oqSkBO+++y6A5J5DKBRKPNQpplCSpKurCwCwdOnSuM+VSiUWLVoU/b6mpgY7d+7Ef//3f6OgoADbtm3D008/HVfHc/PNN+Piiy/GP/3TP6G4uBi33HIL/vznP1MHmUKhUCiUMMmuu5H/rauri9tOLpejuro65eNrNBreumGv1xv9Xgq6urqmnSMALF++PPp9LFNVrwkhqKuri/Z3TuY5hEKhxEOdYgolA/zsZz/D8ePH8b3vfQ8ejwf3338/Vq5cid7eXgDcQvrxxx/j73//O/7H//gfOH78OG6++WZ8+ctfRjAYnOXZUygUCoVCKSkp4W3ZFPmstLQ021NKmpmeQygUSjzUKaZQkqSqqgoA0NLSEve53+9HR0dH9PsIq1evxr/8y7/g448/xieffIK+vj4888wz0e8ZhsHll1+On//85zh16hR++MMf4oMPPsCHH36Y+ZOhUCgUCmWOk+y6G/nf1tbWuO0CgUA0epoKa9euxZEjR6ZlcR04cABarRZLlixJ2XYsVVVV084RAM6cORP9PpZz587F/ZtlWbS2tk6Lis/0HEKhUCahTjGFkiRbt26FUqnEL37xC7AsG/3817/+NWw2G66++moAgN1uRyAQiNt39erVYBgmmoZltVqn2V+7di0AJGzxQKFQKBTK+UKy6+6GDRuQn5+P5557Lm79/cMf/oCxsbGUj3/jjTdicHAQr732WvSzkZERvPLKK7jmmmt4641T4aqrrsLBgwexb9++6GculwvPPvssqqursWLFirjtf/vb38LhcET/vWvXLgwMDODKK68EkNxzCIVCiYe2ZKJQkqSwsBAPPfQQvv/972P79u249tpr0dLSgl/+8pfYuHEjvvnNbwIAPvjgA9x777246aabsGTJEgQCAfzud7+DTCbDDTfcAAB47LHH8PHHH+Pqq69GVVUVhoaG8Mtf/hLl5eVxQhsUCoVCoZyvJLvuKpVKPProo7jvvvvwpS99CV//+tfR2dmJF198EbW1tdN6Er/99ts4duwYAGBiYgLHjx/H448/DgC49tprsWbNGgCcU3zhhRfijjvuwKlTp1BQUIBf/vKXCAaD+P73vy/Zef7zP/8z/vSnP+HKK6/E/fffj7y8PPzmN79BR0cHXn311WktofLy8rBlyxbccccdGBwcxFNPPYW6ujrcddddAJJ7DqFQKFOYZfVrCmVOM7U9BMtyrSCWLVvGKhQKtri4mL3nnnvYsbGx6Pft7e3sP/7jP7K1tbWsWq1m8/Ly2Msuu4z9+9//Ht1mz5497HXXXceWlpaySqWSLS0tZb/xjW+wZ8+ezeLZUSgUCoUyd+Bbc1l25nU3wi9+8Qu2qqqKValU7AUXXMB+9tln7Pr169nt27fHbfetb32LBcA7XnjhhbhtrVYre+edd7L5+fmsVqtlL7nkEvbQoUNJnU+yLZlYlmXb2trYG2+8kTWZTKxarWYvuOAC9p133onbJtKS6U9/+hP70EMPsUVFRaxGo2GvvvpqtqurK7pdMs8hFAolHsKyMfkoFAqFQqFQKBTKAiAUCqGwsBBf+9rX8Nxzz832dNLmo48+wmWXXYZXXnkFN95442xPh0JZUNCaYgqFQqFQKBTKvMbr9WJqnOe3v/0trFYrLr300tmZFIVCmTfQmmIKhUKhUCgUyrxm//79ePDBB3HTTTchPz8fR44cwa9//WusWrUKN91002xPj0KhzHGoU0yhUCgUCoVCmddUV1ejoqICv/jFL2C1WpGXl4fbbrsNP/rRj6BUKmd7ehQKZY4zq+nTH3/8Ma655hqUlpaCEII33ngj7vvXXnsNV1xxBfLz80EIQVNTU1J2X3nlFSxbtgxqtRqrV6/Gu+++K/3kKRQKhUKhzAhd6ynZoLq6Gm+99RYsFgv8fj8sFguef/55FBUVzfbUJOPSSy8Fy7K0nphCyQCz6hS7XC7U19fj6aefFvx+y5Yt+Ld/+7ekbX7++ef4xje+gTvvvBNHjx7F9ddfj+uvvx7Nzc1STZtCoVAoFEqS0LWeQqFQKHOdOaM+TQjB66+/juuvv37ad52dnaipqcHRo0exdu3ahHZuvvlmuFwuvPPOO9HPLrzwQqxduxbPPPOMxLOmUCiU+QMbcktmizBayWxRzh/oWk+hUCiZRcq1Hjh/1vsFV1O8b98+7Ny5M+6zbdu2TUvXisXn88Hn80X/HQqFYLVao6lcFAqFkk1YloXD4UBpaSkYRrqEHk/3GvgnpHkPqq44CbVaLYktCkUsdK2nUCjznfmw1gPnz3q/4Jxii8WC4uLiuM+Ki4thsVgE93nyySfx/e9/P9NTo1AoFFH09PSgvLxcElterxe1mzphGQpKYs9srkFHR8d5sVBS5h50radQKAuFubzWA+fPer/gnOJUeOihh+LeONtsNlRWVuLRN1+HWqeL25aEBN4mhwSMC/xOEoHtpfocAi+IJDuuoH3+L8TPUyI7AtUBwuclYEhgPmLnn8iW+GNLtL3QSUhlX/CwQtsL/HCE7ISEfilEbi8wH8EKE7HnK3idp3/k8XnwwL/fA71eL2RMNH6/H5ahILq7zsJgSM+u3e5AZdUS+P3+Bb9IUhYOQmv9I++9Om2tB2HBsuy0CLJQQJkILooC93sCAftC2/N/ziTYXsz8ZQKLolD8nBGwL2hH5HWTkRCvfaHzFfpcaD6C8xe4TzOE/2GOSZBgwCDIm4EgF5qTwIOkTOAYDAK89hmBcxC6FnKBB1WCEAjPb4BMcJ78doSvKf/PQC5oX8CO0PmC5dbpab+jvJtDxv8xZAK/vDIw/L+jAn81fHbsDi8qlz0yZ9d64Pxa7xecU2w2mzE4OBj32eDgIMxms+A+KpUKKpVq2udqnU6UU+zw+jDm8UCrVKAgvJ/APSLqlAVDIQw5XHD7J1BbmCfKOR11uOFw+1Bk0EGnmtJuYAY7drcXY04PDFo1cnM0op3iCX8AIzYXQiEW5UWmmO3FOaFOlw9jNjdytCrkGrUx24t3iicmghgdcyIYDKGsJDclO3yO6diYC9ZRFyor8qBQTPmTkcApttvcGB5yoKIiFyqVYsbtJ48t3mkNBkMYGbTD551A5aJCbq5inT5B+8k5s37vBHrbh5BbqIepgOeGPctOcTAQRFfLALQ5apirCmbFKY6QiZROo6kYBoMhLRvnS30RZe4i+VqfE7/Wj/nd+Ht3K7ZVLYZBOfkgKJVT7Ar4sLv3DLaWLkauKmbtk8gp9gR9eK//FLYU1aFQnRNjh3+aMkacUzwR8mPPYDPW59WgWG2atCPoFIubf4j14+Ph41huqEKpJn/G7cU6xSwCOGg9ikptGco0JZPbCzqU4pziEBvCKfsh5KuKUK6pivtOvFM8fU4sy6LddRA6mQFl2iXx20vgFLMsiwHPYTBEgVJt/RT7AnZEOsVj3sMIsj4Uai6MW+ukcoo9viZMBIeRq7k8zr5UTnHAfxoTgXboNFeDkMnUZ2GnWDg9eq6u9cD5td7Pqvp0Jti8eTP27NkT99n777+PzZs3Z/zYpwaH8ObJMzgxMDjzxmGCIRavHj2Jv548C39AXKrDR6fa8e6xFlhsTrFTRVN7P/7a2IJz/SOi9wWAEZsL73x+Cp+d6Ehp/wjHW/rwt8/OoK07tXnEMjRix18/OIkDRzvTthXLnj2n8Oln52C1uiS1G+Gj90/h870tGB60Z8R+LMMDNvz9zaM4/MlZYWcvwxzeewaf/60ZbSf7ZuX4M3HyYBs+/0sTjn92dranQqFQBMj0Wr+npxV9Ljt2tTbD7vdKYjOWvQNt6Hfb8WrnCYz5pBXFAYB9w+3o99jwZs8xDHvFPyPMxJGxDvR5rPhrfxMGveOS2z9l70KfZwQfDh1Fv2dUcvvtzi70eyw4MHoEfZ4Bye0PeLvR7+3BCVsjej1dktu3+nsx4GlFq/MI+tzSr1XOwCAs3mb0e46i331McvvewDCGvQdh9R3DsGe/5M8jgeA4xr0fwTXRjDHPHsnth0JOuLzvwR84DZfnbbBCL/Mp84pZdYqdTieampqiPQk7OjrQ1NSE7u5uAIDVakVTUxNOnToFAGhpaUFTU1NczdBtt92Ghx56KPrv73znO9i9ezd+9rOf4cyZM3j00Udx+PBh3HvvvRk/H42Ci/J5JgJJ76OUy6AIF9e7/ROijheJDrt8flH7AUCOhntb7vT6Zthyhv09vrRuNgYd9wbe7kr/ocNo4N5mOZxehMRGDxOQm8tFEKxjmXGKC4q4aOnwkCMj9mMpNBsgkzHwuP2wj0n/IJYMFXVcHWB369CsOeaJqFtdAcIQDPVaYR20zfZ0KJR5z3xc67dVLoFRqYbd7ws7xqmtlUJ8qbQO+SotXAF/RhzjLUV1KFLr4QsFMuIYb8yrRYk6FxNsMCOO8SpjDco1hQiyoYw4xnU5NajQlIIFmxHHuFRdhUpNDQBkxDHOU5ajQrscADLiGOsVZpRp1wFARhxjtbwQxdovAEBGHGO5zIQ8zTYAJCOOMcPkIEdzDQAG/kALdYwXCLPqFB8+fBgNDQ1oaGgAAOzcuRMNDQ14+OGHAQBvvfUWGhoacPXVVwMAbrnlFjQ0NMS1W+ju7sbAwOTN7KKLLsIf//hHPPvss6ivr8euXbvwxhtvYNWqVRk/n0mnWJxzq1Vy+7n94pxbnToNpziyr1f8vgCQo1GCgIt0e3zizjcWQ07YKXam7xTrtErIZQxCIRYOp3QPMHl5nFM8ljGnmEtvGRnOvFMsk8tQWGIEAAz0jmX8eHyUVOVDrpDB7fRiNAvRcbFo9RpULuHS6VqOdM7uZCiUBcB8XOv1ShVurFsV4xifkNQx1sqV+Fr16ow5xiqZHNeWr8mYYyxnZNheUp8xx1hGGFxSVJ8xx5gQgo15DRlzjAkhWGFYG+cY90noGBNCUKNbk1HHuESzJqOOca5qZUYdY51yWUYdY6ViMXI012HSMX6HOsbznFmtKb700ksT/oLefvvtuP322xPa+Oijj6Z9dtNNN+Gmm25Kc3bi0YTrTd2xTnESZQJapRI2ry9xpJjHjjZcf+pOwSmOONROzwz7Cgk8MAx0GiWcHj8cbh+0aiX/hmGEfsqxTjGfYIEYCCEwGjQYHXNh3O6B0aBJ2VYsEadYsvTpKacYiRSPDDsQCrFgEil3pGB/KuaKXFh6x2Dps2JZfXnCmlZx9gW+mPI3LpfLUFZTiK6zFnS3DqLAbEzyAEI1e8kdV8z2y9bXoOtMPzpO9WHtJcug1k6vQxRkDka/KZTZZM6v9Sx4bi8EeoUaN9auxq62E7D5vdjVegI31q2GQcl3P+C/rwjdzUJgoZYp8dWq1Xi96wRGfW682nkCN1SvjqsxngmhR3AGgJJR4JqyerzddwxDXgfe7DmG6yrq42qMIwRD/DESoVrjEEvAEDmuMK/F3yxNGPCO4a/9TbiydG1cjXHUjsBMhV0IBgCDLYUN+GT4KPo8w/hw6CguK2qIqzGeGaHYTwgAwfrcdWBB0Ovpw4HRI0D+urga4yisUMVpgrI3IsNS/TqEQNDracdxWyNCLIMKbQXv5nLBpU9IBEWOSm0DWJag13MKrc4jCIFBhbZOwI7AtRA4rpwNolhdD5Yl6Pc0ot9zFCyAsik1xpP2BeYvJK7DMjAqV4NlCYY8H8PqOwYWBEWaTfzPgoLXgf/jAACVcjlMAMY978E10YwQgPwpNcYz2RF+lghBpqiFFtfC7XkL/sAZsB4WOZpr4mqME80/SJ8X5hQLrqZ4NolEir0i0qcBQBOJFIuMME+mT4uP1OaoJ9OnU31zpg87Cg5P6m/Pc8Lp0xOBILx+cdeNj4gjbLNL98Y9kj49Nu5GKJGAVoqYcnWQyxlMTARht2U+pbmkPA8AMNg3jlBwdt5qVi7mUqh7WgfnZAp1QakJ+WYjQsEQWo91z/Z0KBTKLKFXqnBj7eqMRYw1cs4xzmTE+Jqy+oxGjK8wr81oxPgLhQ0oy2DEeENuA8o1ZRmLGC/XN6BcswgA0Gw/JHnEuEq3FuWaFQCAdudhySPGZs0alGrWAwAGMhAxNqlWoUjzRQDAmK9J8oixRrkcpnDE2JOBiLFCUQet5loADCZoxHheQ51iCYk6xYEAgiJqWnURp1ikc6sLR4qdaaRPB0Nsys5opK7Y4U79AUEu4yLOACdNny6mcF3xuN2Ttq0Ier0acjmXlm2T0G4EhiHIL8xeXXFuQQ6UKjkCE0GMDM1O+nJZdQFkcgZOmwdjWUgbFwshBEvXc2lvZ492IjhLLw8oFMrsQx3jxFDHeGb7sY7x8QykUsc6xplIpY51jDORSh3rGGcilTrWMc5EKnWsY+wPnKGO8TyFOsUSoo5p1+MNJO9opltTnEr6tEzGRNOvUxXb0mu5KK/DnZ4zO5lCnb7DaTJykeJxCSOuhBDkRcS2rNKreAJAYTFXVzycBSeVEIKSCi5aPNBjzfjx+JAr5Cir5lpCdZ21zLD17FC1rBSaHBU8Th+6TvfP9nQoFMosslAc4+IYx3gkA45xqSazjnEma4w35DagQksdYyGoY5wY6hjPf6hTLCEMIZN1xSKUpKNp0CmqT/sCQUwExbVzAiajxTPWFQtgiKRPu9J7MDDqwynPEohtRXodj9s9kqY65+VxNViZassUcYqHsiQ8FXWKu2fHKQaAqiVcCnXXWcucTKGWyRgsC0eLTx9qm5NzpFAo2SPT4luaLIhvXRMjvvVGBhzjbeZ5Lr6Vuzaj4lvL9Q1R8a1MOcaZFN8yZ1h8y6RalVHxLY1yeUbFtxSKuhjxrTPnnSr1k08+iY0bN0Kv16OoqAjXX389Wlpa4rb59re/jdraWmg0GhQWFuK6667DmTNnEtq9/fbbQQiJG9u3b4/bxmq14tZbb4XBYIDJZMKdd94Jp1Pc/Y06xRKjTUGBOtVIsVIug1zG/QhTUZGOpj+nWBOs10kTKTZKqECdo1NDJmMQDIbgkKDNU4T8fC5SPJqpSHFYgdpu88DrTV3NO1lKKjmneHTIAW+KL0XSpbS6EHKFDC6HF6OWudn6aHF9FeQKGcaHHRjoHJ7t6VAolFlmoalSZ8IxpqrUie3HqlJnwjGmqtSJOV9UqUMht2QjWfbu3YsdO3Zg//79eP/99zExMYErrrgCLtdkQGn9+vV44YUXcPr0abz33ntgWRZXXHEFgjME97Zv346BgYHo+NOf/hT3/a233oqTJ0/i/fffxzvvvIOPP/4Yd999t6hrNqvq0/MRQXG98P9ydcWemUWzYlTuUo0UE0KQo1Ji3O2F0++HMUcT893Mx9XH9ioWKfgLxApt+RFiWTCEzHh9+DBEIsWOmPRpoRMgAkcIf8wwBKaIArXNDVPY4U52QkL3xdhIcboq2Xyo1AoYTRrYxj0YHrKjojJGXTNNtWc+tDoVTPk6jI+6MNA7hpqw8FV6iPuZyZUylC8qRGeLBZ1nLSgoNXFfiP0lkmot4/mZKjVK1NVX4szhDpw+2I6ymqKZDys4TxHXR9pfLwqFkogQ4UYsjPCNaKGpUr/RcwzXV9SjgKpSQzpVakBQmZqqUoftU1XqkGQPMNMZtqyF15V+7NPh4K6v3R6fyahSqaBSxd/zdu/eHffvF198EUVFRWhsbMQXv8ilxsc6qtXV1Xj88cdRX1+Pzs5O1NbWCs5DpVLBbDbzfnf69Gns3r0bhw4dwoYNGwAA//Ef/4GrrroKP/3pT1FaWprUudJIscRMRn3FR4o9/gmERL6tirZWSitSnFqkUKdWgmEIWJaFK41oo3FKW6Z0iaRQj41L93bdZNSAYQgmJoJwSCAIxkckWpz1FOpZqisGgKql3A2u66wlI8reUrBsfQ0IIbB0jcA6ODcj2mLI9ptjCmUhslBqjDMZMabiW4ntU1XqxFBV6rlFRUUFjEZjdDz55JMz7mOzcc9MeXl5vN+7XC688MILqKmpQUUF/0uhCB999BGKioqwdOlS3HPPPRgdnfx737dvH0wmU9QhBoCtW7eCYRgcOHAgmdMDQCPFkhNJnxbTXkmjVIAQ7qWTxz8RjRwnw2RbplSc4khNcWoLOSEEeo0KNpcXDrc3GjkWi16nAiEEgWAIbo8fuhTtRDCZwk6xhGJbDMMgN1eH0VEnRq1OGCTqgRxLUbEBrWcHMZwlp7i0Mh+nm3rQ32PNSPQ7GUqqCqBUK+B1+zHYM4qSqoKsz2EmcoxaVC0rQefpfpw62IYt16yb7SmlhRRvjyNvjimU85mIY5xcxFg8Ecc4nYhxIiKOcWzEWKiPcSpEHONkIsapEHGM04sYCxNxjAFEI8abhCLGKdpfro/Y5yLGLIByTZVk9qt0a8P2uYgxAJRpl0hiH+AcYwDRiDEAlApFjFPApFoFANGIMQAUai6U7HlFo+TSzCMRYwDIFYoYp4BCURcXMYYH0Gm+wt/HWGIKzU0wGAxp21Hr7ABK0NPTE2dvapR4KqFQCA888AAuvvhirFq1Ku67X/7yl/jud78Ll8uFpUuX4v3334dSKez7bN++HV/72tdQU1ODtrY2fO9738OVV16Jffv2QSaTwWKxoKioKG4fuVyOvLw8WCzJi7nSSLHEaMM/VDGRYoaQqDPtEllXnBN2ilNpyxRJn3akqD4NAIZwXbEtjfpdhmFg0HFzsUkQhc0zSu8UA0B+PvegMDqaGbGtomIjAGBk2IFgIPNOR2GJEXKFDF7PBKwjmamVngmZjEHVEi5a3H5aurfwUrPiAi6lp+tMP+wZqiunUCjzj4USMaaq1PxEVakzLL4ViRifoKrU06Cq1KnBMFrJBgAYDIa4MZNTvGPHDjQ3N+Oll16a9t2tt96Ko0ePYu/evViyZAm+/vWvw+sVfv6/5ZZbcO2112L16tW4/vrr8c477+DQoUP46KOP0rpGU6FOscSkEikGUq8rzkkjfTriFHv9AUwExKtXA5NOsT1NUauIAvW4I/3FPtfEiWLZHR4EUjwvPgqiTnFm+urqDWpoNAqEQixGstC7VyZjUFKeCwDo65LuIUIstSu4Wo+e1kH4RfbqzhZ5xUaU1RaBZYHmfa2zPZ20KDQ3oaikNa1RaG6a7dOgUOYMVJU6MQtClTrD4ltUlToxC02V2u19XzLbc5F7770X77zzDj788EOUl5dP+95oNGLx4sX44he/iF27duHMmTN4/fXXk7a/aNEiFBQUoLWVex4zm80YGhqK2yYQCMBqtQrWIfNBnWKJ0aXaczjFiG+OmnNsU1GfVinkUCk4kQh7iinU0jvF6fcq1qgVUKvkYFmuNZNURCPFYbEtqSGEoMjMRYsHs6TGXFrFpZnNplOcV2yAIU+HYCCE7nODszaPmVhzEZdy1nGqD46xzGQLZAMp3xxTKBQOqkqdGKpKPbN9qkqdmIWiSk2IFirlGsnsziVYlsW9996L119/HR988AFqamqS2odlWfh8yd8ve3t7MTo6ipISrpRh8+bNGB8fR2NjY3SbDz74AKFQCJs2bUraLnWKE0F4xgxE0qdF9xyO7ucXddzJSHGKbZVmaMvEzjBi06cT3jr4zinmvEzhGl0pnFhCSDRabJVKbIsQmEw6yGQMJiaCsDu8CSS+IXy+CXZhY53iQRtYQsCmUtdCCP/gmUxZuIZ3dNgBt2ciekxWyEaiIXaa4f8YwqB2RRkAoP1Uv3j7Um0/g538EhNKFxWBZVmc2HdOvB2Rvw8UCiVL8C1uEUXqqUNwQZxUpY51jG0+H1gWPIPwDgiMEIuoKnWsY2z1egRt8Y2QwGBZElWljnWMhz1O3vkHQwzvELo8sarUsY5xv8eOIMtMG2LnH2QZAHJsmSK+1eu2Cu4jZIdvhFgCFgzW566LE9/q9lgQBOEfrIx3hFjwD3Cq1JFU6uO2RvS4exBgGd4RAv8IsoR3hMCpUsemUve4WxPMn/9aBCDjHSGWoFhdH5dK3ec+hiAY3hFgZbxDaD6hsCp1bCr1kOcAJliCAJhpQ/g68I8ACKdKHeMYj3r2IBBiEWQxfUBgsCz/AKdKrc/5JxBZMYIIIZhAZ30+smPHDvz+97/HH//4R+j1elgsFlgsFng83LN9e3s7nnzySTQ2NqK7uxuff/45brrpJmg0Glx11VVRO8uWLYtGjp1OJ/7P//k/2L9/Pzo7O7Fnzx5cd911qKurw7Zt2wAAy5cvx/bt23HXXXfh4MGD+Oyzz3DvvffilltuSVp5GqBOseREIsXeQADBUPK/7DpVejXFnokAAkHxf1wzOcUzERspTueNmomvLVMa5IXFtqzj0kX0GIYgL49ztkdGM1NXWlxiAsApUIdE/P6kikanQl6hHgDQ3z170eKaZSUgBBjuH4dDQtVwqVlz8WIAQMfJ+R0tplAomWGh1BhTVWp+qCr1zFBV6sQQkryY7nzjV7/6FWw2Gy699FKUlJREx8svvwwAUKvV+OSTT3DVVVehrq4ON998M/R6PT7//PM4oayWlpaocrVMJsPx48dx7bXXYsmSJbjzzjuxfv16fPLJJ3F1zX/4wx+wbNkyXH755bjqqquwZcsWPPvss6LmT53iGTjQ24u/njuHUXdyi45aLgcTji55RNQVp5o+rVLIIWe4H2MqCtTpOsV6rQoE4JSj06gHjUSK3d4J+CcCKduJEI0USyy2VVDAOZAjI5mp+TXlaqFSKxAMhLJSVwwAZeGeyH2ds+cUa3PUKKnkotZtp/pmbR4zUVCSi9KaQrAsi+Z952Z7OhQKZQ6yEB3jYeoYR8m2Y3zc1oheKr4Vx3wX31qoRFKhp47bb78dAFBaWop3330Xg4OD8Pv96OnpwR/+8AcsXbp0mp3IPhqNBu+99x6Ghobg9/vR2dmJZ599FsXFxXH75OXl4Y9//CMcDgdsNhuef/555OSIU9KnTvEMdIyN4ezoKMYTqKLFQgiBJqokLcIpjk2fFgEhJK0Uar02HOn1pFYTLGMY5IRbKNmdqUd5lQo5tOEWUWMSpFDnRZziMWnrfyNO8XCGHFZCCIpLuBRqS/94Ro4xlbJqzike6LEiKKEwmVhqV06mUGcjSp4qay4O1xaf7IU9QxkDFAplfrNQHGOqSs0PVaWeGeoYU+Yb1CmegVTUpCMp1GIc3Khjm0q/YXWktZL4fQ1hh9bhTn2xNuWEU5/TFNvKjdYVp7+wmwwaMAyBfyIIZxrnNpWicKqx1eqSVNk6lpJSEwDA0jeeEftTySvUQ6NTIRAIZu2YfJQvKoJKo4DH5UN/58iszWMmCkpzUV5XDJYFjn3aMtvToVAoEsGyLLyB9DOVIlBV6sRQVeqZ7VNV6sTMd1VqytyCOsUzEO07LMopFi+2FakN9gWCmAiKc7b0aUSKDdFIsS/lP3RjTlhsK41IMQCYDJH+wulHimUyJmpvVMLaT51OBY1GAZZlMZqhKKG5lGuTNDxkz5jjHQshBOXhaHHvLDqjMjmDRcu5aHHbybmbQg0A9Vu4VJ/ulgFYB7OjFE6hUDIHy7L4sKcDf245AdeE+BfMQlBV6sRQVeqZ7VNV6sTMd1VqytyBOsUzkHSkOEZNNhopFrGwKmUyKGTcj4MvWiyk6AikFymO1BT7A0H4UqzlNepiIsVpKOxGIsVjCSLFQuKcfMfMz42kULunfSl4PWeYPyEEhYUGAOEU6lTOdwaVY71BDa1OhVCIxZDFJpnac6K5VtQUAgB6O0YQCqV4sxeyL2L+das4p7ivYwTuqb9PQgeQ4LiJz2u6jdxiI6qXc4qGxz9pSVNMOo0/GgqFIgnuwATax62wej149exJLtOLqlJTVeoUVal73VSVmqpSJ6dKTZk7UKd4BnQppU+LjxQTQqJiW2IFsyKObSqRYrmMgTasfJ1qr+LJSHG66dNcZFeK9GkAUaXoUQkVqAGgMFJXnCGxLULIZAp1luqKi0tNUCrl8Hr8GJnFyKcxLweFpSawLMu1Z5rDrLl4CQgh6GsfwnDf2GxPZ17w9NNPo7q6Gmq1Gps2bcLBgwcTbj8+Po4dO3agpKQEKpUKS5Yswbvvvsu77Y9+9CMQQvDAAw/EfX7ppZeCEBI3/uf//J9SnRJlgaBTKHHjkpXIUSgnHWOpI8YLoMaYqlLzM1V866C1kapST4GqUmeXYMgt2ThfoE7xDEQixS4RTrE2EikW6dxGUqgdYp3icPp0ym2VIinU7tSc2kik2O7yph5lRLwCtc+ffl1XRGxLyvRpACgM1xUPDdkzdkM0h53igSzV+DIyBmXVnPpzT/twVo4pRCRa3NbcN6cXHENeDhatKgcAHN17ek7PdS7w8ssvY+fOnXjkkUdw5MgR1NfXY9u2bRgaGuLd3u/348tf/jI6Ozuxa9cutLS04LnnnkNZWdm0bQ8dOoT/+q//wpo1a3ht3XXXXRgYGIiOH//4x5KeG2VhYFJrqGOcAKpKnRiqSj0zVHwre5zqW4/m3qVpj1N962f7VLIGdYpnIJo+LUI0KxLxFbMPMOkUixXbmkyfTq0uOCK2ZUtRkEqnUUIuYxBiWThSdKwBToFap40oUKe/kEfSp90ePzwppJYLUVCgB8MQeL0TcKQZHReipIyrK7aOOuH1SDf3RFQu4lKou9qGZ/UmX7nYDKVKDqfdM6cFtwAuWiyTMxjqtaLnnGW2p5MUs/Xm+Oc//znuuusu3HHHHVixYgWeeeYZaLVaPP/887zbP//887BarXjjjTdw8cUXo7q6Gpdccgnq6+vjtnM6nbj11lvx3HPPITc3l9eWVquF2WyODoPBIHr+lPMD6hgnhqpSJybbjjFVpZ4OdYwpqUKd4hnQpSC0lRPexykifRoAcsJNqJ0+cQtkjloJAiAYYuEReUwAMKYZKSaEwBhWoB5P00nMC6dQS9FfWKGQwajn5jUiYbRYJmOQn8/1PhsaykwKtUarRG44/bs/SynUJZV5UChl8Lh8GLbMXgq1XCHDohVcve7Z4z2zNo9k0Bk0WL6xFgBw5KPTs9rSKllO9G3Asd5laY0TfRsAAHa7PW74BO5dfr8fjY2N2Lp1a/QzhmGwdetW7Nu3j3eft956C5s3b8aOHTtQXFyMVatW4YknnkBwihDhjh07cPXVV8fZnsof/vAHFBQUYNWqVXjooYfgTrLvPOX8JCuO8TTxLelesFJV6sRkQ3wrG+2aqCq1MFSVGlhR1ohV5S1pjxVljbN9KlmDOsUzEIkUT4RC8CepCh1xij0TEwiK6LeaalsmGcNAF0mhTkOB2pZGlNcUriseT1OBOtconVMMAAWRumKrtErRxUVcpGloyC6p3Vgi0eKBLNWrymQMyqu5aHF32+ymUC9eUwEA6O8YgWN8bjswKzfVQqNTwTnuxpnGztmeTlapqKiA0WiMjieffJJ3u5GREQSDQRQXF8d9XlxcDIuFP8Le3t6OXbt2IRgM4t1338W//uu/4mc/+xkef/zx6DYvvfQSjhw5InhcAPiHf/gH/P73v8eHH36Ihx56CL/73e/wzW9+M4WzpZxPZN8xbpbUMaaq1ImhqtQz26eq1ImZ66rUMkYr2ThfkM/2BOYyLACFTAYFw2AiFILL74dSJRPYeFItVq2QgyEEIZaFOzABfTgCPBP6mJriqX9WM2nR6tUqOL1+2D0+FBv1SR0vgkGXIFIsdOApn5vCEdkxp0dgJ4EbxZRNI3XAVonEtvJzc9DWNSIiUixwwiR+/kXFBuBkn6BTzCZQOSZCN80pu5RW5OLUiV70942DZVmQVJSTkyHGbFVdITrOWtDdNoT1m2vBMFIcM7lrGoUFDLk6lFYXoL9zBGeP9WD9JUuFlaMFr6dE2wsRtqNQyrH2kmXY9+4xNO87h0WryqHWJfc3HzbEMxdxUxHD6rLDaacP2+12ACXo6emJs6VK8l6XDKFQCEVFRXj22Wchk8mwfv169PX14Sc/+QkeeeQR9PT04Dvf+Q7ef/99qNVqQTt333139P+vXr0aJSUluPzyy9HW1oba2lrJ5kuZx0QUpGNh2KhjvOvsyahjfMOSldAplPw2+GCEHpInValfbTuBcb8Xu1qbcWPdKhiUfL/P/PaFbhUhsFFV6te7TmDU58arnSdwQ/Vq5KqSf8gVeq3PAFFV6rf7jmHI68AbPcdwfUU9CtQ507YPhvhjMDKG/wixqtR/szRhwDuGv/Y34crStShWm6bbEZipcFiCAcBgS2EDPhk+ij7PMD4cOorLihpQqskX3IvfDj8yAqzPXQeAoMfThwOjR7Axbz0qtGb+HViB50sIBGQIp0odAkGvpx3HbY0IsQwqtBW8m8uFfllYgatEOFVqliXo9ZxCq/MIQmBQoa0TsCNwLQSOK2eDKFbXg2UJ+j2N6PccBQugTFvPv4Pgn5LA9QmrUrMswZDnY1h9x8CCoEizif95SvA68H8cADhVagDjnvfgmmhGCEC+5nJ++zwfzf3csvMLGilOgqiadJIp1ISQaFsmMVHfVNOngUkF6lTEtozhmmKXb0J0j+SojXD6dLq9iiOR4jGbW5I3bgW5mYkUR9oy2eweeL3iU9aTochshEzGwOP2Y3wsO9FSc0UeFEo5PG7/rKZQA8DStZUAuJ7FExIIr2WSRSvLkVdsxIQ/gGOftsz2dBIi5Ztjg8EQN4Sc4oKCAshkMgwODsZ9Pjg4CLOZ/wGxpKQES5YsgUw2+aC4fPlyWCyWaDr20NAQ1q1bB7lcDrlcjr179+IXv/gF5HL5tDTrCJs2bQIAtLa2ir52lPOPbESMb4irMZY2YkxVqRPDL74lnZYFIQTrY2qMD1FV6mlQVWrKXIE6xUkQVaAWIZwVFc0SsU9ERdoXCCadqh3BEBbbSqWtkkohh1LOPXjaUxTbMuVMRorTuRmY9GowhGAiEIQzxbnEkp/LvbF2SSy2pVYrYDRy55ypFGqZjEFxqREA0NebvRTqikWcCnVXK78qcLYoqcqHPleLCX8A7afndnsmQgg2fIl7aGg73g3rLLa1mosolUqsX78ee/bsiX4WCoWwZ88ebN68mXefiy++GK2trQjFlKCcPXsWJSUlUCqVuPzyy3HixAk0NTVFx4YNG3DrrbeiqakpzpmOpampCQDndFMoyZB98a357xjPb1XqJskdY6pKnRgqvkWZC1CnOAnERorj9hHhFCvlcijDD3Jiew5HFKRTcYoJITDp0qsrNuaoQQD4J4Lw+FKPnDIME23NJJXYVsTeiNR1xcWcwzqYwbrisvI8AEB/rzVjx5hKVV0RAKCrfRihYPI18VJDCIlGi88c6Uqr3Vc2KKrIR9XSErAscPjvzXRBnMLOnTvx3HPP4Te/+Q1Onz6Ne+65By6XC3fccQcA4LbbbsNDDz0U3f6ee+6B1WrFd77zHZw9exZ/+ctf8MQTT2DHjh0AAL1ej1WrVsUNnU6H/Px8rFq1CgDQ1taGH/zgB2hsbERnZyfeeust3HbbbfjiF78o2L6JQuGDOsaJWXiq1PPbMaaq1NOhjjFlJqhTnAS6lCLF4XRm0T2HU9vPoAnXBXtSdGojYluu1NKf5TIZ9GHHeizNFOpof2GbNIrRBXlctHg4Q06xJYNRwdIKziketNjhz1IKsbk8F2qNAj7vBAayFKEWonZFKdeeyeZBX/vsRq6TYd2XVkCukGG4bwwdJ/tmezpziptvvhk//elP8fDDD2Pt2rVoamrC7t27o+Jb3d3dGBiYfEisqKjAe++9h0OHDmHNmjW4//778Z3vfAf//M//nPQxlUol/v73v+OKK67AsmXL8L//9//GDTfcgLffflvy86MsfGZHfEvadk1UlVqY6eJbTVSVeop9qkqdmPmgSk0RhjrFSZCTQqQ4lfRpIEZsS2ykOFxT7PT6U4qoGXXhlkppKFDnRsS2HOktsvkmrl5xVCLV4cKIUzwqrVNsNnN1xWNjrow5rAajBgajBizLYqBvPCPHmArDMNFocce5wRm2zixyhTyqRH3qiHSLe6bQ6TVYddFiAMDRvafhTyNrYiFy7733oqurCz6fDwcOHIjW9wLARx99hBdffDFu+82bN2P//v3wer1oa2vD9773PcG06IiNp556KvrviooK7N27F6Ojo/B6vTh37hx+/OMf0z7FlJSZnXZN0jnGVJU6MVSVemb7VJU6MXNdlZoiDHWKk0CbQip0tFexUMSX8I9opHiqUyywfWTo1ErIGE7x2unzzbj91GGMpE+7vGAJokMMEQXqcQdPpFjo2DxEFahtrri5iJ0PwO1TUMCpcY9YnQiB5eyImA/AiR5OHRqtCnq9Giyb2dZMZeVca6a+qSnUgudBxA0eI9VLOPGjns4RTARCYAmJDvH2BRDYfup/S9dWgZERjAyMY8QyzvMzE/nDFDtPkXaWb1gEfa4OXpcPxz9tiZmmRNeNQqFIC98NPqJIPXVAwDH2+8XZ4duWBcBOqlLHOsY2nw8sC55BeMe0xTM8QiyiqtSxjrHV6xG0xTdCAoNlSVSVOq7G2OPknX8wxPAOocsTq0od6xj3e+wIssy0IXb+QZYBIMeWKeJbvW6r4D6JbPHOCQzW566LS6XudVsQBOEfrIx3hFjwD3Cq1LE1xj3uHgRYhneEwD+CLOEdIXCq1LGp1D3u1gTz578OAch4R4glKFbXx6VS97mPIQiGdwRYGe8Qmk8orEodm0o95DmACZYgAGbaEL4O/CMAwqlSxzjGo549CIRYBFlMG5S5A3WKkyCV9Gl9NFIs7g1vqunThJCoAnUqdcXGNGuKASBXH1aOTjd9OqxA7XD54J9IPwKba9SCYQh8/gAcTuneuAOAuZiLOGUyhbosnELd12PN2tvG/CIDcgwaBAMh9HRIV1eVClqdCjVLSwEAp+dBtFgmY7BxK1fTevZIJ0ZnWcWbQqFIT/ZrjKWNGFNV6sTwq1JLn0odcYwPUlXqaVBVakq2oU5xEkSjvimoT7v8EwiJ+CNINX0amKwrtqVQVxypKXb7JlJ2RE0SpU+rVQroNNx1sEqQQi2TMdHo8/CoI217sUzWFWcuUlxsNkIuZ+DxTMA6Kk2d9UwQQlCzmKv17DxnycoxE7F8XRUAoKd1EA6J0uozSWlNYVR068B7x+MUlCkUysKAOsaJWXiq1Jl1jKkq9XSo+BYlm1CnOAkiStIToVDSrZK0SiUIgBDLwu1Pvq5wMlKcSr/hsNhWCtFelUIOTbi3cqp1xbnhtkxefyAtBWoAyI+IbY1L4wQWhlOopXaKzWGn2Gp1ZqyuWCZjUFLGpVD3dEu3IM9E9RLOKR7oGYPHJW2EXSymAj1KqwvAssCpw52zOpdkWX/5SihVClgHbTh9qH22p0OhUDIAdYwTs/BUqTPjGFfMe1XqlQCoY8zHbDnGgZBbsnG+QJ3iJFDKZJOtkpKMFjOEQBeJ+opYwCL9hp0+P4Iio0tRBelUew2HU6jHnak5xXK5DIawDas9TbGtXGmd4qKwUzw0Iq1TrNOpYDBwdcWZTKGuqORSqHu6sucUG0xaFBQbwLLsrAtuAcDKjTUAgPbTfXCn+DuaTbQ5aqwP9y4+9ulZ2CUWeqNQKHOD+S6+RVWpE5MN8a31816Vup6qUidgqir1uHevZLaF+Kj7Euzp2pD2+Kj7kozPda5AneIk0aWQQq1XTTq4yaJRKCBnGNH7AYAxXFOcal2wKRzpTbUtEwDkhVOorWmmUBeEI8UjY9IsnEX5nFNsHXdhYiK5aH+ymM0mAIAlg7Wj5ZX5IAQYs7rgcGTPIVy0rAQA0H5mYNZTforKclFYakIoyOLMPKgtBoBFq8pRUl2IUDCE/e8dn/VrSKFQMsN8d4ypKnViqCr1zPapKnViIqrUBEpoFHWS2aVIh3y2JzDnCYvA5iiVGPN4RDrFSgxAXCo0YQj0ahXG3B7Yfb5o9BdJ/F1ORoq9YFkWRKSCrSnclmksHafYoEWnZYyrK449vMD8hU4rEiket3sQCAQhl4fbsAidE0l8gBydCjqtEi63H8NWJ8qKBFqyCF0yoYkSghKzCWfPWjBgsU3OL9GNVOQxWEKg0ihRZDZicMCG7q5RrFhdLqyvLPYmnuDXpKquCI2fnYNtzI3RITsKzMakfheTsi94TYW3X7mxBh+9eRRnj/dgxYYaqLXKBErNIg8gdN3SUIImhGDT9jV45/mPMNxrxdmjnVi6riZle1LApUOld+s/n9KpKOcJyd7XQgL3A4aNOsa7zp6MOsY3LFkJnUIpyg4/k6rUu9pOwOb3YlfrCdxYtxoGpYp3e34r/ITARlWpX+86gVGfG692nsAN1auRq9IK7MVnhx8GiKpSv913DENeB97oOYbrK+pRoM6Ztn0wxB+zkTH8R4hVpf6bpQkD3jH8tb8JV5auRbHaNN2OwEyF8/MYAAy2FDbgk+Gj6PMM48Oho7isqAGlmnzBvYRtTUdGgPW568CCoNfThwOjR3BB3nqUa838Zlih9nQCL/4Jp0odAkGvpx3HbY0IsQwqtBW8m8sFl1aBq0Q4VWqWJej1nEKr8whCYFChFXAAWYG4nMBx5WwQxep6sCxBv6cR/Z6jYAGUaesF7AvMnwhcn7AqNcsSDHk+htV3DCwIijSb+J+nBa8D/8cBACrlchTJq8EwGgQABBM9gKXJpZV7JWk/aLfbAZSkP6F5AHWKkyQqtiXizW+0V7HIiK8h4hR7fEBu8vvpNSoQAgRCIbj9E9H07WQxxbRlSpWIAnW6kWKtWgm1SgGvbwJWmzsa6U2HogI9OrpHMTRiF3aKU8Bs5uqKx8fd8Hj80GjEXfdkqagqwOCADT1dI1ixujwjx5iKUiVHZW0ROlosaD09wDnFs0hpdQHyigywDtlx+kgXGrYsntX5JEOOUYuGS5bj0PvNaNp7BmW1xcgxJv+QKTXvd10OrV64128yuB3SZltQKAsFUY5xCkRqjJNzjMUTqTFOxzFORKTGOBnHOBUiNcbJOMapEKkxTt8x5idSYwwAvZ4+HLQ2gpB1KNNI45REUqk5++1oth8CQ0Io01RJZr9KtzZs/xTanYfBIIQy7RJJ7ANcKjUA9HsaMeA5CgKgVMgxTgGTiuseMeT5GGO+JhCwKNRcKDrQJATDaCSxMxNyRgs5k/7frZzJjF7OXGRW06c//vhjXHPNNSgtLQUhBG+88Ubc9yzL4uGHH0ZJSQk0Gg22bt2Kc+fOJbT56KOPghASN5YtW5b2XHNS6FUcSZ8WK5oVEduye8U5pzKGie6bilhWJFI87vKIUsyOJc8Qbstkd6eVdkIImUyhlqyumHOEpa4rVqsVyA1HtgcymEJdWc0tukMWOzwe6dLyZqJ2ObcYd7UOYSJDYmLJQgjB6k2cYMjZY93wZvE6pMOShmoUluchMBHEgd00jZpy/jGf1vt0oeJbiaGq1ImhqtQzM9/Ftyhzk1l1il0uF+rr6/H000/zfv/jH/8Yv/jFL/DMM8/gwIED0Ol02LZtG7wzOIsrV67EwMBAdHz66adpzzWVtkyTTrH4SDGQWlum2BRqsei1KsgYgmCIhTOFXscAYNRpwDAEE8EQHCkKfkWIim1JVVccI7YVCkl7cyspMQEA+vvHJLUbiy5HjfwC7m16T2f2egcXlRihN2kQmAii89xQ1o4rRNmiQuQV6RGYCM4bJWpCCC7cvgYyOQNL1wjOHumctbl8uWoPrqzen9b4ctWeWZs/ZX4yn9Z7KaCOcWKoKnViFo4qNXWMKfOHWXWKr7zySjz++OP46le/Ou07lmXx1FNP4V/+5V9w3XXXYc2aNfjtb3+L/v7+aW+YpyKXy2E2m6OjoKAg7bnmpKAkrVdz+9hFRooNEcGsFJziaLTXLb4umCFksq7YmVpdMcOQaAr1qD29CG9hHucADo9JEynOM+mgkMswMRHEmE3amsiyUi7Pvb9/PKM3zaqaQgBAV0f2nGJCCBavKAUAnDvZN+uLAiEEazZzNUpnj3XPeruoZDHk5aDhUu4B4eje0xiXOGMhWSIpVekOCkUM82G9Z1kWgZB0pQHzXXyLqlInhqpSz2y/SreWqlJT5g1zVn26o6MDFosFW7dujX5mNBqxadMm7Nu3L+G+586dQ2lpKRYtWoRbb70V3d3dCbf3+Xyw2+1xYyrpRIq9gQAmkuxvDABGdbjfcArRWlM4Upxur2Frik4xAOSHU6hH03Q8CyJiWzY3JgLpP6gwDEFRIRcttgxP/xmnQ3GxAXI5A4/Hj7HxzIkQVS7iHvgGB8azmkK9aFkJGBmDsREnRodmx5mLpbS6AAUlRgQDITQfnD89gJc0VKGkphDBQAifvX0EQQl+rymU+U621vtEaz3LstjX24Ndp07CG5CuTGS+O8ZUlToxVJV6ZvtUlZoyX5izTrHFYgEAFBcXx31eXFwc/Y6PTZs24cUXX8Tu3bvxq1/9Ch0dHfjCF74Ah0P4Qf7JJ5+E0WiMjoqK6Up8+rCIhScQQIANcepysYMHlVwe7W88NVrMCgxgMlLsmZiAX+RDs3EGpzjRcYFJp3jMOcOiN/X8Y65DvoFzZq0xkWLB4yawo9OooFUrwIJrpSQF5kKurnhQwClOZZ4AIJMxMBdzIlT9fWPC2yfSaSBEYEzuqzdokFeQA5bNbs9ilVqBqroiAMDZ0/1gCZk2hOcvMIQukMD2sf8xhMHaiziRrdbmXjgdHp7rLGRf6CuheQogcnsCLhvjoivrodIqMT7sQNPHZ8Qfl0JZYGRrvRda6wkLePwTODk0iCGXC2+cOQXvREB4QeAbIcI/IOAY+/3i7Agee1KVOtYxtvl8YFnwDMI7IDBCLKKq1LGOsdXrEbTFN0ICg2VJVJU6rsbY4+SdfzDE8A7BH0uMKnWsY9zvsSPIMtOG2PkHWQaAHFum1Bj3uq2C+ySyxTsnMFifuy6uxrjXbUEQhH+wMt4RYsE/wKlSx9YY97h7EGAZ3hEC/wiyhHeEwKlSx6ZS97hbE8yf/zoEIOMdIZagWF0fl0rd5z6GIBjeEWBlvENoPqGwKnVsKvWQ5wAmWIIAmGlD+DrwD8rcYc46xaly5ZVX4qabbsKaNWuwbds2vPvuuxgfH8ef//xnwX0eeugh2Gy26Ojp6Zm2jUoum+wfLCJabFCJrw9WyeVQyeWi9wNiFKTd3pTEsnJzwkJZ6USKjZH06fTfJhfkhlOordK8OS4OO8WWYbvkb/rKyrgU6r6+zNUVA5Mp1J1ZTKEGgMUruRTqrtYh+HwTWT02H+aKfJgr8hAKsWg+MH+ixZocNTZv55QyzxzuwEDH8CzPiEKZn4hd7xOt9TqlEl9btgIauXzSMZ5vEeMFUGOcyYhxNsW39g4fyaj41kFrY0ZrjJvthzJaY9zuPJzRGuOBDNcYj/maaMR4ATJnnWKzmevLNjg4GPf54OBg9LtkMJlMWLJkCVpbWwW3UalUMBgMcWMqhBDowynUYtSkIynUYuuKjZG6Yo+4NOgcNSeWFWJZOFJIv87LmawpTvWPPaJA7XD74J9I76GiIFpXLM3iWJCXAxlD4PUFYHOk3nqKj9KwUzw4ZIc/gyrNVbOUQl1QbIApX4dgIISOM8LRm2xSv5mLFref6ofdKk02QTYoryvGkgauBcbn7zbB45T2d5FCmU9ka72faa0v0OqoY5wAqkqdGKpKPbN9Kr5FmcvMWae4pqYGZrMZe/ZMqpza7XYcOHAAmzdvTtqO0+lEW1sbSkrS7/GWE4n6iokUq1NryxSpKx4X6RQzhMSkUIuP9hq0asgYgkAwBHuK6tFqpQK6sMhYutHiorBTPDQqzcIokzEoDKtQW4ZsktiMYNBrYNCrwbJsRlsz6Q0a5IdTqLMuuLWyDABwdg4IbgFAYakJZTUFYFkWTZ8lbt8y11h36QqYCvTwunz4/C9NkiuiUyjzhbm03kccYzV1jHmhqtSJyZZjTFWphaGO8ezx5JNPYuPGjdDr9SgqKsL111+PlpaW6PdWqxX33Xcfli5dCo1Gg8rKStx///2w2RI/M09tvRcZP/nJT6LbVFdXT/v+Rz/6kaj5z6pT7HQ60dTUhKamJgCc2EZTUxO6u7tBCMEDDzyAxx9/HG+99RZOnDiB2267DaWlpbj++uujNi6//HL853/+Z/Tf/9//9/9h79696OzsxOeff46vfvWrkMlk+MY3vpH2fKORYhELTCR9WqyStFETToMW6RQDQG5UgVr8vgwzqUBtnamuOAH5xnCPYZs0CtROtw9uiaKiJUVc7e/AkLRiWwBQXp4HAOjttUpuO5bqWq6+t6M9u6m3NUuLoVDK4bB50N+d2XNMlrVbloAQoKdtCMP947M9naSRK2T4wnXrIVfIYOkawcl988upp1DEMJ/W+wKtDl9bPs8dY6pKLUg2VKkz3a6JqlInhqpSzw579+7Fjh07sH//frz//vuYmJjAFVdcAZeL8wX6+/vR39+Pn/70p2hubo5qQtx5550J7ca23RsYGMDzzz8PQghuuOGGuO0ee+yxuO3uu+8+UfOfVaf48OHDaGhoQENDAwBg586daGhowMMPPwwA+O53v4v77rsPd999NzZu3Ain04ndu3dDHY6iAkBbWxtGRiajZb29vfjGN76BpUuX4utf/zry8/Oxf/9+FBYWpj3fVPoOpxwp1qbuFEfbKrlSqwvOD7dUsjpSX+QKIk5xmgJZSoUcueEa5SGrNKrH5iIuZW5gSPq64qhT3DeW0RtkdS33+zw8aIdT4jTwRCgUctQu56IwLSd6s3bcRJjyc7BoBRfBPvLJ2Xm1MBnzc3DBFasBAMc/OwtLV3brxCmUbDHf1nvqGCeGqlInhqpSz2yfqlJnlomQR7KRLLt378btt9+OlStXor6+Hi+++CK6u7vR2NgIAFi1ahVeffVVXHPNNaitrcWXvvQl/PCHP8Tbb7+NQIL7a2zbPbPZjDfffBOXXXYZFi1aFLedXq+P206n04m6ZnJRW0vMpZdemvAXiBCCxx57DI899pjgNp2dnXH/fumll6Sa3jT0KfUqnowUszEiczPpzZk0SaZP8xiKiG2NuT1xxwQSiNrGfJ5n0AL94UixwPYz/d0XmCKRYifYRMdN4v5RlJeDMZsbQ6MO1JTm85sRsM933MICPWQyBl7fBMbt3qjTnXhCQgeI376o2ACFQgavdwIjo85oqnb8XPltERE3U61OheISIwYHbOhsH8aq+hjFdMFfLpEqhwLTWbKqDGeO9WCgxwr7mBuG3BR71or9nUiw/ZrNtehsGcDIwDi6zw2iaolZ+JdO6DqLVXyWyM6ileUY7B5F99kBBCeCM4qUUyjzkTm93k9twwDub7BQo8MNy1bgtTOnoo7x9eHU6qQJCfw1M2zUMd519mTUMb5hyUroFEpRdviZVKXe1XYCNr8Xu1pP4Ma61TCEu2lM3Z7fCj8hsFFV6te7TmDU58arnSdwQ/Vq5KqSXxNCAp8zQFSV+u2+YxjyOvBGzzFcX1GPAnXOtO2DIf4Yj4zhP0KsKvXfLE0Y8I7hr/1NuLJ0LYrVpul2BGYqNH8AYAinSv3J8FH0eYbx4dBRXFK4DuXavAR78Vri/VRGgPW568CCoNfThwOjR3BB3nqUawXq8FmZgH2BLieEU6UOgaDX047jtkaEWAYV2ukdWgBALrhGC1wlwqlSsyxBr+cUWp1HEAKDCm2dgB2BOJ7AceVsEMXqerAsQb+nEf2eo2ABlGnrBewLzJ8IXJ+wKjXLEgx5PobVdwwsCIo0m0D4ngN4rkNQ6OFVAl7puAaanPTdPI+Tc1antqtVqVRQqfjuJZNE0qLz8oR/5202GwwGA+RJ3lcHBwfxl7/8Bb/5zW+mffejH/0IP/jBD1BZWYl/+Id/wIMPPpi0XWAO1xTPRSYjxSLSp9WTvYr9YnoVh51ip8+PQDDRbXc6udFIcWoRxIjYltWRugJ1JFI87vAgkGYv1uJ8zrGUsq64KOysDkhcVyyTMSgt5QS3Mp1CXbOIi4ZkO4Vab9CgrIp7OdHSPDeixdocNVZs4N5oH/307Lzr/7tx6ypc/a0voryueOaNKRRK1qDiW4mhqtSJoarUM9unqtTzg4qKiriWdk8++WTC7UOhEB544AFcfPHFWLVqFe82IyMj+MEPfoC777476Xn85je/gV6vx9e+9rW4z++//3689NJL+PDDD/Htb38bTzzxBL773e8mbReY5UjxfCPSq9jh94NlWf43QVPg2ivJ4AsEYff6UKBL7g2qRsH1OPYHg7B5vSjQJP/mNZI+7fFPwDsRgFoh7secF06fHnd5EAyFIGPEvzvRqpXQqBTw+CYwanfDnDv9zW6yFIWd4pExZ8rzmUpJkREDgzZYhmxYsTh9EbZYKipy0dU1gt7eMTSsrZLUdixVNQU4uK8NY1YXxsdcMOWKSxNJh6VrytHXNYr2MxasuaAGKpUia8cWYsX6KrQ298Jl9+L0kS6s2rho5p3mCHKFDDmmFCPuFAolo0Qc47QixgkQFTFOgYhjnFzEWDwRxzidiHEiIo5xbMT4uop6FPJEjFMh4hgnEzFOhYhjHBsxvqyoAaUa/sw3sUQcYwDRiPGm/HUo00jzbBNxjDn7XMSYBVCukeb5JuIYc/a5iDEAlGmXSGIf4BxjANGIMQCUCkWMU8Ck4py+SMQYAAo1FyblJ2SKm2re5u2mIxa73Y57YEZPT0+cvZmixDt27EBzczM+/fRTQbtXX301VqxYgUcffTTp+Tz//PO49dZb40prAK4kJ8KaNWugVCrx7W9/G08++eSMc41AI8UiyAmnTwdCIXhEvCmOKEnbvclHbgkhMEVVpMVFfJVyGXThuaZSV6xTK6FSyMCygDXFfsWEkGgK9fB4em91DTlqqJVyBEMsRsekabtTGiO2JbXqb1lZHggBrGMuODPYakelUqC0nItKt50bythx+DCX5cKUr0MgEMS55r6sHlsIuUKOhou5Fk3NB9vhzmKt9Xwi2zVGFMpCgKpSJ2ahqFLP93ZNVJVamPNNlVrBaCQbAKa1s0vkaN57771455138OGHH6K8vHza9w6HA9u3b4der8frr78OhSK5wMonn3yClpYW/NM//dOM227atAmBQGBa2U0iqFMsAjnDQBf+waWSQi1WgXqyrlj8A2hsv2GxEEKQb+Ac2lF76k5ooSncYzhNp5gQguJIG6URaRSjC/JyoFDI4PMHYE1TDGwqarUCRWExr56ezKZQ1y7m0m3b24ay2tKHEIIVaysBAGeO96adIi8V1ctKUFhqQjAQwpFPpV1QFwq/a/sanj+3Pa3xu7avzXwgCmWBQcW3ErMQVKkzLb5FVakT26eq1PMblmVx77334vXXX8cHH3yAmpqaadvY7XZcccUVUCqVeOutt6ZFfBPx61//GuvXr0d9/cxR/qamJjAMg6KioqTtU6dYJJEWS3YRTnEkUmwTESkGAJM29dZKk3XFqS1IEQXqkTQUqKVyigGguIBzMi2j0ihQMwyJtmbqHxyXxGYslRVcWlR3hp3i8oo8qNRyeNx+9PeOZfRYU6mqK4JOr4bPO4H2M5asHlsIQgg2XLoMhABdZy2wZPj6UyiU8wvqGCeGqlInhqpSz2yfqlLPX3bs2IHf//73+OMf/wi9Xg+LxQKLxQJPOLgXcYhdLhd+/etfw263R7cJxuguLVu2DK+//nqcbbvdjldeeYU3Srxv3z489dRTOHbsGNrb2/GHP/wBDz74IL75zW8iNzc36fnTmmKR6FUqDDid0xWoEyjjRiLF9thIcRIq0NH06RTaMuVFeg2n2pbJwDnFo2Kd4pj5F4briG1OL/wTASjF1DZPuT6RSPHgqAMhTK/nJiLvNywBSoqN6O6zom/QhtUruPQOsSrZQoetKM/DocMdGByyweubgDqZmluxxyYEjFyGmrpinGnuQ+u5QZRV5QsrF4u9Kc9QCsPIGCxfW4HDn5zD6WPdWLzcDIav3lu0qrO4+Uz92ecXGbF4TQXOHuvB4Y9O46pbN4ORxcxLUKpc5IGlUrHOlA0B/kfta2nXGdntdjwIAYVTCmWeQVgCMuW+wArciKgqdaJPqSp1UudAVanDx6Wq1IEFFpv81a9+BYDrNhDLCy+8gNtvvx1HjhzBgQMHAAB1dfE/z46ODlRXVwMAWlpaosrVEV566SWwLMvbh16lUuGll17Co48+Cp/Ph5qaGjz44INxdcbJsLB+GlkgsmhkI1KcG44Uj7nFO7a5Odzik2qv4gJ9OH3a4Ur5DZdGpUCOlrtew2mmKOfn6iCTMfD5A7CloYodS6mZixQPDdslT//V69XIzdWBZYG+DEdw68Ip1L3do/B6JzJ6rKnULiuBSq2A0+5Fd5ZVsBNRv3kxVBoFbFYXWo71zPZ05hRS1hhRKOcrVJU6MVSVOjELUZW6l6pSx3G+qFLHwrIs77j99tsBTLbm4xsRhzhiJ7JPhLvvvhtutxtGo3HacdetW4f9+/djfHwcHo8Hp06dwkMPPZS0wFYE6hSLJLX06clIcUjEH0SkLZN3IgDvhLjFNpI+7fT64U/B4cvN0YAhBL6JIJze1BdiqVKoZQyDojzOlmVEmhRqo14DrUaJYIjFoEQ2Y6ms4N76dvVIt9DxkZufg7yCHIRCLDpaBzN6rKnIFTIsXV0GADh5tHvO3PBVagXWhkW3ju9vhTuDgmcUCuX8hDrGiZkNx3h4HjvGmRLfKs+i+FYmHWMqvkXJNNQpFkkqTnGOSgWGEIRYFk5f8guaUi6LKl6PiRTbUivk0Cq5lN1U6oplMga5YbGukTTEtorCKdRDEtQVmwvDdcUSiW0RQlBmNgEA+gakj+ZG6or7+8cxMZFZIaq6JVxa1LkzlqzfcJesLodcIcPYqAt9XZl9ASCG2pVlyDcbEfAHcfijM7M9HQqFsgBZKKrUpnnuGM93VepyTdGCUaWmjnE81DGeP1CnWCRRJWmfL+lfaoaQ6H7jWUyhzsuJ1AWnmEIdVqAelsApHhxzpn0TiDjFA8N2yW4oZSUmAEDvwLgk9mLJzdVCr1cjGAyhty+zgk81dUWQyRnYxt0YGpTmpUGyqNQKLFnFRYuPN3bNmZs9IQSbLl8BQgh6WofQneUoOoVCOT9YCOJbN1BVakGyIb61pXDtglGlzpRjvJBUqUe8ByW1T5EG6hSLRB+uKQ6EQnAHkq/fNEXqikWKZqXjFOeHnWJrigrUBUbOKR6xp754FZh0IITA45uA05PeIl6UlwOGELg9fjhc0izYpWYTCAFsdg8cEqfYEkJQVclFizszHEFVKuWoWcTJzp+bBSXo5WsrIJMzsA470D+HFJ9zC/VYsaEaAHDog9PwZbnmmkKhnB8sBMeYqlILQ1WpZ7Yfq0qdiT7GC0WVmkAGjZwKVc5FqFOcCDJ9yBkGOUoupTlWTZoVGBFS7Tmcq+P2s7o9vPNJdNxIr2KrM0Wn2BBxil1gCeIG31z4lP7kMhnyjZxzPjTmnL6DgB2+c5LJZSgM1xUPDMer0gkywzxVSnm03VPvwJjw9qIHAQhBdXUhAKCvbwwTgWD48+Su3eQ5EIERv//i5dxNtrNzGD4fj/M3w1yTHjxG1BoVlqwMR4sPdyEETh2bTUVFWfD6CHwh+DH335pNtTDk6uB1+3H0k7OiziuRfWE7Qucl9jpTKJSswLPgRBSppw6hRZewk6rUcTXGEwHhhZpvhAj/gIBj7PeLsyN47ElV6ljHmMuKA88gvGPaw0J4hFhEValjHWOr1yNoi2+EBAbLkqgqdVyNscfJO/9giOEdgj+WGFXqWMe432NHkGV4h9hzADhV6tga4163VXB7oSE4HzBYn7sursa4121BEIR/sDLeEWLBP8CpUkdSqY/bGtHj7kGAZXhHCPwjyBLeEQKnSh2bSt3jbk0wf/7rEICMd4RYgmJ1fVwqdZ/7GIJgeEeAlfEOofmEwqrU1YZ/gFZegxDLhH/umcEf8ko2zhdoS6YUMKpUcPr9sPt8KNHrk9pn0imehfRpZ6rp09z+Lp8fbp8fWhVPO4gkKMrVY2TchcExJ2rLClKyEcFcaMDgqAOWYTuW1hSnZStCeWkuLMN29PaPYUWdtG/v8vJ0MOjVsDu86OmxYtGi5JuIiyW/UI/cfB3GRl1oOzeEFeGU5myxfG0Fzjb3YXTIDkvvGEoqxLaXyAwyuQwXfnkl/vbng2g72YeqxWaUVOXP9rQoFMoCJFJjnFa7pgSIateUApEa4+TaNYknUmOcTrumRERqjJNp15QKkRrjZNo1pUKkxjjSrmnv8BFcVtSAUo00a1akxhgAej19OGhtBCHrUKYpkcz+cn3Efjua7YfAkBDKNFWS2a/SrQ3bP4V252EwCKFMu0QS+wCXSg0A/Z5GDHiOggAoFWrXlAIKJjm/IV1+efZWqHPSv+94ndJlvMx1aKQ4BSJiWzYxCtQazrlNtabY7vUhEBLuh8dHJFLs8U/A7RefNqqQy2AKR6rTEdsqjohtjaWv8FySgbri8nBd8cCQ9K2ZSEy0uKMzsy2LCCFYvIxb2M6dGch6ba9Gq0LdilIAwPHDHXOmthgAikpzsbS+EgBwYM9JTPjPn5s8hULJLlSVOjFUlToxC02V+ngGUqnnu/gWZW5CI8UpYFSF64N9yTu4ueFIsd3rQzAUgjzJ9xFapQIquQy+QBDjbg8KcnRJH1Mhk8GoUcPm8cLqcEObP72310wUGHQYd3kxbHOhsjBX9P4AUJTHvRUbsbkQCAYhlwk1ik/CVr4eDCFwefxwuLww5KTfL9Vk1EKnVcLl9qN/yI7K0tTOU4iamkIcP9GDvr5xeL0TUKsy92dXU1eEIwc7YLN5YBmwoaTUlLFj8bGioRKtp/oxYrGjr2sU5dXpZQZIydqLF6O3YxguuweNH7fgwq0rZ3tKFAplgRJxjF+d5xHjV9tOYHweR4zf6TuGQa8Db87TiPGnw03o9Qzhw6GjGYkYEwA9nj4cGD2CTfmZixgftzUCQMYixq3OI5z9DEWM+z1HAUgbMc40/2vJH2AwGNK2Y7fb8Qikycyc69BIcQoYI6JZIqK+OUol5AyDEMuKaudECIlGi62piG3puQVmJMW64kJjuM+wLfVIsV6rgkalQIhlMTyeuh2Ai14X5nNz6h+SrjVTZRmX6tudAZVok0mLvFwdWJZFV9eI5PZjUSrlqK3jUrRbTvVn9Fh8aHUqLF1TDgBo2t+OUGjuRIsVSjk2f3kVAKDtZB/6OjIbuadM5+mnn0Z1dTXUajU2bdqEgwcTK3COj49jx44dKCkpgUqlwpIlS/Duu++Ksun1erFjxw7k5+cjJycHN9xwAwYHqRI5hZ+QhBkuC0F8i6pSC0NVqWe2H6tKfZyqUmcVJaOWbJwvUKc4BUyRtkxecc5tpK5YbH1wni5cG+wS7xQXRJxiR2rOaFFYgTodp5gQAnM4WjxoTT+FurSIi3j3D9nSthWhIuwU9/SPZSTtd9EiLoW6vT3zjtjS5dyb3p7uUTglVtROhhUNlVAq5bCNudB5NvtK2IkwV+Rh2dpIGvUpqkadRV5++WXs3LkTjzzyCI4cOYL6+nps27YNQ0NDvNv7/X58+ctfRmdnJ3bt2oWWlhY899xzKCsrE2XzwQcfxNtvv41XXnkFe/fuRX9/P772ta9l/Hwp84/Gnn68ceIUJoLSldEsBMeYqlILQ1WpZ7ZPVakp8wXqFIuFTEaKnRN+BNjQDMrBkyM3xbrifF0kUuyerjKdQDQXJEZB2uGecVuh/Qk4sS2XN/WFtjjsFFvGHHH2Z1Lt5iPiFA8M2SYdWNFKwPHDXGSAQi6DxzuBEatr2gbC80xOsbgm7BQPDdvhcPD//CNqzVOHWIx5OTCXmsCyQMsZy6QdqVSOZ/idUakVWLGeczyPHepAUKhOW6was+AxxW1fv2UxDLk6eFw+HPrwNFiwie0I/oxFDsHrKdHPJUlmS43y5z//Oe666y7ccccdWLFiBZ555hlotVo8//zzvNs///zzsFqteOONN3DxxRejuroal1xyCerr65O2abPZ8Otf/xo///nP8aUvfQnr16/HCy+8gM8//xz79+9P7QJSFh4st8Yd7ulDn82Bt5rPcN0CqCo1VaVOUZV6wGOjqtRUlTopVWrK3IE6xSmglsuhDNfFikmhNmlTixTnRpziFCLFkfTpcZcHgaA4oS4gLLYVrttNJ1ocGylONxJbmJcDuYyB1x/AmE2aN8oyGYOycP1td/+YJDZj0WpVKCnhnPlMC24BwNKw4FXrmQHJxcOSOv7qcmh0KridPrSczH4adyLkchku2rYKhBB0nbWg44x0b8XnAz87czueOHVzWuNnZ24HwNUaxQ6fQGmI3+9HY2Mjtm7dGv2MYRhs3boV+/bt493nrbfewubNm7Fjxw4UFxdj1apVeOKJJxAMR/GSsdnY2IiJiYm4bZYtW4bKykrB41LOT3RKJa5btQxKmQz99rBjLHXEmIpvCTIb4ltSR4xjHeP3Bo5mVHxr7/CRjIpvHbQ2ZlR8q9l+KKPiW+3OwxkV3xqgEeMFCXWKU4AQAlMKdcWRSPGYyLZMkfRpm8crWoE6R6WESiFHiGVhdaVaVxxJoU59Ack36iBjCHwTAYyn2CIqgoxhYA73Fu6TMIU6k3XFAKLtmNrbhzOuzFxemQ9djgo+XwDt57JfPymXy7BmYzUAoPlIF/x8fZNnkXyzEasvrAUAHPrwNBzj0j18nU9UVFTAaDRGx5NPPsm73cjICILBIIqL48U6iouLYbHwp9i3t7dj165dCAaDePfdd/Gv//qv+NnPfobHH388aZsWiwVKpRImkynp41LOX8x6PXWME7AQHWOqSj0JVaWeGapKvbCh6tMpYlKrMeRyYVyEaFaqPYd1MQrUY24PCkUoUBNCUKjXotdqx4jdjeIc8cqLRcYcnO0bwVAaTrGMYVCUq8fAqB2WUQdy9ekpTJYWG9E7OI6+wXGsXlKalq0I5aW5IIRgzOaGzeGBUZ++snUslZX52L+/DTa7ByMjThQWZq5XHcMQLF9VhsP723G6uQ91S0sgy3KWzqJlZpw+1gP7mBsnjnRj/eba7E5gBlZurMFA9wiG+8bx6bvHcMVNmyCTL/z3hP972YtpK1La7Xb8CMXo6emJs6VSSaNMCwChUAhFRUV49tlnIZPJsH79evT19eEnP/kJHnnkEcmOQ6HEEnGM32w+E3WMr121DIo0uibEQlWpE0NVqRNDValntk9VqSmpsvCfADNEJFI85knewY20ZfIGAnBPJB85I4QgPyq2Jf6taaSueDjFXsNFpnCfYZszrQhnSX64x/Bo+qrRFWYTAMAyLF1vYZVSjpIibo5dvdJHi5VKOaqquIWrtZ1fXEhK6paWQKmUw27zoLdburfJycIwDNZdVAcAaDnRC4ctvQwBqWEYgi3b10ClVsA65MCRT1tme0pZQUo1SoPBEDeEnOKCggLIZLJpqs+Dg4Mwm828+5SUlGDJkiWQxTgjy5cvh8Vigd/vT8qm2WyG3+/H+Ph40selUPgixn4qvhWFqlInJhsRY6pKndg+VaWmpAJ1ilMkN+wUixHNUshkMIQfGsVGi6NOcQqtlQqjTnFqN/18vTac+hyEzZ26mnGsU5xu+rBRr4FOq0QwxGJgWJrWTABQXc45rV29mXEi6+q4VM+OjuGM1/oqFDIsCStRnzzWk/GUbT5KK/NQUp6LUIjF0QPtWT/+TGj1amzexrVpOnusB92zkGp+PqBUKrF+/Xrs2bMn+lkoFMKePXuwefNm3n0uvvhitLa2IhRTMnL27FmUlJRAqVQmZXP9+vVQKBRx27S0tKC7u1vwuBQKAJgN8Y7x25lIpZ7njjFVpRaGqlLPbJ+qUlPmGtQpFklEcTCiQD3m9YZVoFn+MYVoz2GvZ0YV39gRdYrdU1Skk6BQzznFo043ggI1yYlUoGUME402D46HFw0Rc4+oTBfm5oAQApfXD4fHl1hlWshW5GtCUF5sAgD0Do4LmhESvRSiopyrKx62OuH0+Ca3n2E+044rMIrNRuh0KkxMBNHTO5akOrHQEFI6ntxm2aoyMDKCkWEHhgYFXh6Iti+kljx9EMJFiwkButuHMWixpayqLZU69NT/ymuKsGJ9NQBg/99PwmGfqtQ+83VO77qdH+zcuRPPPfccfvOb3+D06dO455574HK5cMcddwAAbrvtNjz00EPR7e+55x5YrVZ85zvfwdmzZ/GXv/wFTzzxBHbs2JG0TaPRiDvvvBM7d+7Ehx9+iMbGRtxxxx3YvHkzLrzwwuxeAMrchW/pZgVqjKkqNVWlTkKVmgWoKjVVpZ5RlTrISlOWQZEG6hSniCksmuXy+0WlVaVaV5yfE4kUi09BNWrVUMplCIZYjKWgYA0AxaawevR46m9SFXIZCk2ccz0wkn50t9ycCwDotYynbSuCVqNEUQF3rplIoSaEoK6WE9xqbc18VFKjVWLRYi46ffJEb8aPx4cpPwe1y7m678bPWhEKZT9iPRNrL1qMghITJvwBfPbucQRTUGqnJObmm2/GT3/6Uzz88MNYu3YtmpqasHv37qhQVnd3NwYGJiMRFRUVeO+993Do0CGsWbMG999/P77zne/gn//5n5O2CQD//u//jq985Su44YYb8MUvfhFmsxmvvfZa9k6cMq+h4luJoeJbiaGq1DPbp6rUlLkCdYpTRC2XR4UxxKRQ54Wd4lERtcgAkKebdMK9E+IWTEIICsLCVsOO1OqKi8N1xYNjjpT2jyBlXXFpkQGEENidXtidqad1T6UqwynUtbXcA3v/wDhcLukeHoRYsbocANDbbcX4WOpttdKhfmMNFEo5rMMOtM3BFkiMjMGWK1dDqZJjdNCOIx+fH/XF2ebee+9FV1cXfD4fDhw4gE2bNkW/++ijj/Diiy/Gbb9582bs378fXq8XbW1t+N73vhdXYzyTTQBQq9V4+umnYbVa4XK58Nprr9F6YoooqGOcmIXmGL9JVanjoKrUMzMXVal9Qa9k43yBqk+nQa5GgwGHA+MeD4o1ySlCR5xiq8hIsUouh16lhMPnx6jLjTKTOPXYQkMO+sccGLK7sDwFsebiXC56anW44Q8EoUpRibOkwIhjrf3oG7GBZdm02pYrFXKYC/QYGLaj2zKGVXXSqCNWlefhUFMnBoftcHv80GqkUe2MoNerYS42wjJow7nWQaytr5TU/lSMJi0qqwvQ3TmCE8d68IVLl2X0eHyotUqs2VCNxs9b0bS/DRU1BdCoFVmfRyJ0Bg02b1uNvW8dxdljPcgrMqB2ZdlsT4tCocwBqCp1YhaKKvXbfccwFFalvq6iHoVUlRrApGNMQFWphZhrqtSPnfqfUOWk/5zlc86tlpqZhEaK0yA3pq44WSJOscvvh0/km+CCcCumEaf4aF9RuCZ4KEWxrRy1EnqNCiyAwfHUo8XmPD0YQuDy+GF3pf/2qaKES6HuGRhL21YEfY4ahfk5YNnMRYsXL+EiVefOWbKSTryqoQIA0Nk+DLt9dlSgl6wuQ25+Dvy+AI7ua5uVOcxE+aJCrAn3Lz74wSkM94/P7oQoFMqcgapSJ2YhqFLHim/Nx4gxVaVObJ+qUlMSQSPFaZAbrisW05ZJJZcjR6mE0+/HqNuDUkPyvWoLcrToGB3DSAoK1EVG7m3niIMT25Ix4t+HmHP1cHh8sIw5UJlvEr0/wNUVF+dx/Yr7RmwwVapTshOhsiQXB493YWDYDv9EAEqFNL/SNZUFGB51or1rBMsXS/MmNJaqynwcUivgdvvR22tFZaU0b3OFyC/Qo6w8F329Y2g+1ouLvrA4o8fjg2EYXPDFJXjv9SNob7GgdqkZxaWmrM9jJlZtWoSxEQd6Wofw8TtNuPKWC6HVp/d7SqFQFgYRVepIxPjtSMRYLmHEePkKvHZ6SsRYorVNMGKslDBiXLcKu1qbYYuJGBsl6mEeUaV+rTM+Ypyn1khiP+IYv9V7PBoxvr6iHoUa6SLG20vqsXvgWMYixpcU1WPv0DH0eoYzEjHemNcAWIEeT39GIsYrDGsBAN2eDpywNYJA2ohxjY6L6Pa4T2ckYlwSjhj3uY/MasT44RXPwGAQl1XKh91ux3/idQlmNPehkeJEzKAmHXGKrSnWB1vd4pzbyUixeKfYoFFBpZAjxLL8bZ0SiPhGRkke58BbEtQVC6pYx9gpLTQCAPpHbMLHE2LKdkaDBoYcNUIsi94hW9Iq0zNRU1kAABgaccDp8iZ1fcQMmZyJtmdqOTuQcNuIWvPUIZbVDdyi0t46CKfLN2lHKlXkJM67oMSIuhVc/v6BT84iEGKnn5fY+Qj+Dgl8Ifgx9x9DGFx0xWqYCnLgdfux950mBILBtO1TVWoKZY4ioA7N+whAVampKnUSqtSJlKmpKjVVpY4dmUIlU0s2zheoU5wGeTG9isX0gM3XRtoriXOmCyIK1C7h1kpCEELSTqE2504qUKeT8lsWcYqH0+9XDACVpbkApE2h1mqUMBdxb9jau0cksxvLknAKdX//eFZSmguLDTCXmhAKsTh5fHaUqAFg7YWLoNIoYB9z48yxnlmbRyIUSjkuvaYBKrUC1iE7Drx/clb6PFMolLkJFd9KzEIT38pEH2OqSp3YPlWlpmQb6hSngUGtBkMIAqEQHCJu9vnhSPGoyEixQa2CQiZDiGUx7hZfj1tkCCtI21JTIM7N0UAplyEQDGEkRRVrACg06aCQy+CbCGAkxbnEEq0rtowhJKHjsqiqEADQ3pUZp1ivV6OsjJt7S4slI8eYyuq1nKjXuTMDcEmo2C0GlVqBdRfVAQBONHbCMUs1zjORY9TiC1evBWEIOlssOHmoY7anRKFQ5hDUMU4MdYwTQ1WpZ7ZPVakp2YQ6xWnAEAJTOFosJoU6LxwpHnG5RUWfCCHRaPFwKmJbRi5SPJhipJgQEk2hHrCmLrbFMAxKCrgobN+wLWU7EcwFeigVMnh9AQyPptcyKpbq8jwwDMHYuBvW8cy0Mlq2jKvDaW21YGJCuocpIcylJhSXGBEKsTh+pDvjxxOiZkkxiktNCAZCOPhRy5yNwpor8rDhEk6t+9jnreg6m52XF5mEtmigUKQjW47xfBbfurF2NUzUMeYlW45xuaaIOsYC9qljTIlAhbbSJE+jgdXjgdXjRbUpyX20GhAA3kAA7okJUQIXRTk6DNgcGHK4sMxcKGquxWGxrTGXB75AAKoUWj2U5BnQNTSO/jE76mtSF1YoLzSh2zKGnmEb1i5Or+0NwzCoKMlFW/cIuvqtKC5IX1gAAFQqBSpKc9HVa0Vr5zAuWJtc2y0xlJXlQq9Xw+Hwor1jCEuXSC/qNZWGDTXY/XYT2s5ZsGJNOUxGaQRKxEAIwaZLl+IvLx+CpW8MbWcGUJdKr7AssKS+AvYxF1qauvH5e83Q6FQoCkf45yP/9/h9UOakJ6rjd0r30EyhzHf42jVdE3aUpUBQfGsetWu6gUd8S8p2TXziW1K2a4oV33pjHrZr2lK4Fp8OH82Y+NaG3AYQAvS4M9euiQGLbk8HjtsawQIol7hdEyFsxsS3zJo1IISNim8REJRo10hmnyINNFKcJnkRsS1v8m8mFTIZTBouwjziEvdGs1DPOWbDKaQva1VKGDTcIjSUYtpySR7ncA6MpVcPXF7E1RUPWh2cQEmaVJXmAQA6+6ySRh1rq4sAAG1dIxlpnUQIwfKwM3j6zEBWIqaFxQaUV+aBZYFjjZ0ZP54QeqMWazbWAACOfN4K9yylcyfDui8uRfmiQoSCIex96yhso9JFCigUyvwnokodiRi/nYmI8Txv13RjBts1aWehXZPUEePtJfUZjRhfUlSf0XZNG3PXZrRd0wrD2mi7phMZiBjX6NZktF1TSbRdE4FKlnznGUr2oJFisUwRhY2kQotVoM7XajHm8WLE7UZlninpwxZFnGKnCyGWhUxIqlng42JjDuweHyx2B8oLjJObJ+mLFRp0kMsY+CaCGHW4UWCYEj0VEs2dYt+Yo4Feq4LD7UP/iB1V5rwpdvgnJOQzlptNkDEEDpcPY3YP8ozhN8RCKr5JnnB5qQkqpRwe7wT6B20oL4mPELJTTyxiPoGE9tR9FtUW4ejRLthsHvQPjKOsNMkopOAhZv4hrN1Yg95uK7o6RjA64kR+Ac8bb9H2ZzzsFDMslq0tR3f7EEaHHNi/twWXXb0mwWHFKjILHljU9oQlkDEEW66qx55XD2N4YBwfvHEE275+AX+rJiENPMHfOZ75ZFB9+t/W/EfabRrsdjtewJ8lmhGFMruQEDdiYQVCBonuHnwR42tXLYOCJ2IstEYIrymTqtSvnUkjYhwSOAOGFRcxTmCHn0lV6l1tJ5KIGPPbF7r+IbBRVerXu1KPGAvdvhkgqkr9dt+xaMT4+op6FAhEjIMh/l8iGcN/lIgqdWzEeFtJA0o0Rt7tZQKzFTwHwqlSfzJ8FH3hiPElhetQrs0T2EMIgfMiwPrcdWBB0OvhIsYX5K1HudbMb4YVyqQQeJlEOFXqEAh6Pe04bmtEiGVQoa3g3Vwu+McqcIUIp0rNsgS9nlNodR5BCAwqtHUCdsTdJORsEMXqehgVVVDLTGGlbdppYi5BI8Vpkq8Ni2Z5POIUqHURBWpxbzJNWg0UDINAKIRxkerVwGQK9aAttTecDEOiKtT9VntKNiKUF5kAAD3D42nZAbj+x6XFnL2ufmva9iLIZAwWVXHtmVo7hyWzG4tSKUdtHReRPnNGujericjN06EmfMyjhzuzckw+GIbB5i8tByNjMNBtRdvp7Jx/KsjlMlx6bQMMuTq4HV588MYR+L0Tsz0t0dAWDRRK5qDiW4mh4luJoarUM9uf76rUaplJUnsU6ZhVp/jjjz/GNddcg9LSUhBC8MYbb8R9z7IsHn74YZSUlECj0WDr1q04d+7cjHaffvppVFdXQ61WY9OmTTh48GCGzgAwqtUgAPzBIFwTyT8gF+gmxbbEwBCCgnC0eCiFFOpiY7itks2ZcqpuaTiFuj9Bv+JkiDjFvUPjadmJUBWOsHb1SecUA0BtTWHUrn9CuoePWJYt41Koe/vGYMuSGnP9+ioQQtDfNwZL/3hWjsmHMU+HtZu4lKjGz1rhnKNq1ACg0ijxpa+uh0angm3Uib1vNyEgQfo/hbLQWQjrfbJQxzgx1DFODFWlntn+fBffWqg8+eST2LhxI/R6PYqKinD99dejpaUl+r3VasV9992HpUuXQqPRoLKyEvfffz9stsSiu8msD1arFbfeeisMBgNMJhPuvPNOOJ3i/i5n1Sl2uVyor6/H008/zfv9j3/8Y/ziF7/AM888gwMHDkCn02Hbtm3weoVrD19++WXs3LkTjzzyCI4cOYL6+nps27YNQ0NDGTkHOcNEFahHPcnfdCORYqvbI7rncFEaTnGBXgsZQ+CdCKTU1gkASvPDTrE1vbri0gIjGEJgd/tgc6bvCFWW5oEAGB13weGSrj61IC8HJoMGwWAoYz2LDQYNysPiTadO92fkGFPRGzRYvJxLa2o81DGrCtBL11Sg0GxEYCKIzz44k5H6banIMWhw2XXroFDKMdQ3hk/+cgyhoLi/YQrlfGMhrPdioKrUiaGq1ImhqtQz26eO8dxj79692LFjB/bv34/3338fExMTuOKKK+Bycf5Kf38/+vv78dOf/hTNzc148cUXsXv3btx5550J7SazPtx66604efIk3n//fbzzzjv4+OOPcffdd4ua/6w6xVdeeSUef/xxfPWrX532HcuyeOqpp/Av//IvuO6667BmzRr89re/RX9//7Q3zLH8/Oc/x1133YU77rgDK1aswDPPPAOtVovnn38+Y+eRr42kQifv2BlUKijDPYfHRNYjF+vDKdAptFaSMUy0X7FlPLVIb6FRB4WMgW8igFFH6ouMUiGDOZ+LXHdLEC3WqBQwF3IOe0evtDf3xTVcqvHZ9sw9bK1cyalwt7UNwZultNw166qgUMgwOuJER1tm0sOTgWEILtq6HHKFDMMWG04enb12UcmQV2TApdc1QCZj0N8xgs/fa57TjjyFMtvMl/VeypeDfI6xn4pvRYmoUmcyYpxN8a03eo5heJ45xlsK12ZUfGtDbgMqtJl1jCPiW5lyjDMpvpVJfEGfZCNZdu/ejdtvvx0rV65EfX09XnzxRXR3d6OxsREAsGrVKrz66qu45pprUFtbiy996Uv44Q9/iLfffhsBgXtXMuvD6dOnsXv3bvz3f/83Nm3ahC1btuA//uM/8NJLL6G/P/lA05ytKe7o6IDFYsHWrVujnxmNRmzatAn79u3j3cfv96OxsTFuH4ZhsHXrVsF9AMDn88Fut8cNMUSdYhGRYkJINIV62CnuJl0cdmqHnS7RUWYAMJs4RzRVp1jGMFEV6r7R9OqKK4u46Gj34FhadiLUlHMtBqR0igGgrroQhBCMWJ0Yy1DP4uIiA/LzcxAMhnCmJTu1tRqNEqvqOZGKI4c7ZzUVOMegwcYvcC0Qjjd2YmQovd+tTFNUlosvfKUehCHoOmvB/veb52y/ZQplLpOt9X6mtb65fxB/aW5JaV0VgqpSJ4aqUieGqlLPbD9WlToTjnGmVakzxf1Hd+Luxv+V9rj/6E4AmHbv9Plm/juNpEXn5QmLudlsNhgMBsgFBAOTWR/27dsHk8mEDRs2RLfZunUrGIbBgQMHZr5YYeasU2yxWAAAxcXFcZ8XFxdHv5vKyMgIgsGgqH0ALgfeaDRGR0VFWMmO8AweJsW23Pz7CNgpzOHSoEdcMU5WEvsaNCqo5HKEWBYjbv5jsuAfAFASdooHbEk4xQJzKAunUPdabWAJokMslWbOKR6w2rkHhZnmn+B6AkB1eT4IAUbGXLAnSMmOnXMy81drlKgMpze3dAxNbi/hIAyDVSvLAXCCWxOBEKdCLLA9SwjvEMuyVWXQ6lRwu3w4dbJ/0o7QEAtD+AfPSdUsNaOytghsiMVne07DHwimfF6C8xc9eKcKAoLymiJ84ap6EELQcXoAB/ecCv+e8l03QUNJ32solIVIttZ7wbWeBVw+Pz5t7ULn6DjebT6LUCAUVaWOHUKLExEYYAVSqQNBATuEdyQ6bkSVOq7GeCIgvJDyjRDhHxBwjP1+cXYEjz2pSh3rGNt8PrAseAbhHUKLeohFVJU61jG2ej2CtvhGSGCwLImqUsdHjF0C8+dUqfmG0CWKqFLHOsYDHhuCLMM7xJ4DwKlSx9YY97qtgtsLDcH5gMH63HVxqdS9bguCIPyDlfGOEAv+AU6VOjaVusfdgwDL8I4Q+EeQJbwjBE6VOjaVusfdmmD+/NchABnvmC9UVFTE3T+ffPLJhNuHQiE88MADuPjii7Fq1SrebUZGRvCDH/wgYZpzMuuDxWJBUVFR3PdyuRx5eXkJ/b+p0JZMAB566CHs3Lkz+m+73T65WCbBZKSYU6AmST7AF6YotkUIQbFBh26rDUMOZzSdOlkiCtTjLi+8/gmolQpR+wNAWR7XImDAakcwFIKMSe39ilGnjrZm6hu2obpEbGuAeDQqBUqLjOgbtKG9ZxQNy8rTshfL4kXF6Oq1oq1zGBvqqyCTSf9OqbIyH3q9Gg6HF2fPWbByRZnkx5iKXC5Dw4ZqfLa3Bc3HelC7pBhaLV+LjMxDCMEFlyzBiMUGh82Dxs9aceGly2ZlLslSWVeMi7evxme7j6O1uY97M37Z8qTvAxQKJTskWut1SiW+smop3mluQZeVc4yvWrUk5bVtKmLaNaVCpMY4rXZNCRDVrikFIjXGybVrEk+kxjiddk2JiNQYR9o1vdnTlLBdk1giqdSRdk3vDRzFlaVrUaw2SWI/UmMcade0d/gILitqQKkmXxL7kVRqAOj19OGgtRGErEOZpkQy+8v1EfvtaLYfAiEhlGuqJLNfpVsbtn8K7c7DYBBCmXaJJPYzwS8afp52+0WAu0++hN+jp6cnzp5Klfhvc8eOHWhubsann34qaPfqq6/GihUr8Oijj6Y9TymYs5Fis5kTABocHIz7fHBwMPrdVAoKCiCTyUTtA3A/WIPBEDfEYFSrISMEgVAItiTSCaLz1YV7DrtcotMuI46wJYW6Yo1SAZOWEwezpNiaKd+ghUohx0QwhGFb6unEhBBUFnMRWKlSqBeVcy2U2nukFcUqM5ug1Sjh8wfQLbHCdQSGIdFo8alTfQhmScCppq4IBYV6BCaCOHqoIyvHFEKlUmDz5VyqUtvpAXSeG5xhj9mnemkJNl+xGgBw7kQvDu89Q1OpKZQkydZ6P9NaX55rxFdWLYWcYaKOsaSp1FSVOiFUlToxVJV6Zvux4lsnbI3oPY/Ft1QylWQDwLR7ZyKn+N5778U777yDDz/8EOXl04NTDocD27dvh16vx+uvvw6FQjg4l8z6YDabpwksBgIBWK3WhP7fVOasU1xTUwOz2Yw9e/ZEP7Pb7Thw4AA2b97Mu49SqcT69evj9gmFQtizZ4/gPlLAEII8DZdCPSKi73CeVgOGEPgCQdhFONMAJsWyUnCKAaAkN9JWKbW6TULIZAr1iC0lGxGqwk5x1+CYJI5EVVkeGIZgzO7BaBoO+1QYhmDxonA/4XPJp2OIpba2CFqtEm63H62t2XEICSHYsLkWANB+bghDlvR+puliLsvFqvXcG979H53BuDUzddxSsmh5KS788koAwNljPTj4wWnqGFMoSTCX1vuF4hjP6xpjqkotCFWlntk+dYxnD5Zlce+99+L111/HBx98gJqammnb2O12XHHFFVAqlXjrrbegDnfwESKZ9WHz5s0YHx+PCnoBwAcffIBQKIRNmzYlPf9ZdYqdTieamprQ1NQEgCumbmpqQnd3NwgheOCBB/D444/jrbfewokTJ3DbbbehtLQU119/fdTG5Zdfjv/8z/+M/nvnzp147rnn8Jvf/AanT5/GPffcA5fLhTvuuCOj51KgDdcHi3CKZQwTFdsacop76DeHneJxj5erHRJJaS5XV9yXolMMAOX5XAp172h6DlRJgQEKuQwe3wSGx9NfPFRKOSrCtcptErdQWlpbDEIAy7Ad1gwJbslkk7XFJ5p7sxYtLiwyoG4p90btwGfnEJLwQTAVVm+ogbksF8FACJ+814wJf2Z6REtJ3cryqGPc2tyL/X8/SVWpKRTMr/W+PNeIr6zOrmNMVaknWQiq1NNrjOeXY5wVVeoMim/FqlJnyjGej+JbmWbHjh34/e9/jz/+8Y/Q6/WwWCywWCzwhLvsRBxil8uFX//617Db7dFtgjH3wGXLluH1118HgKTWh+XLl2P79u246667cPDgQXz22We49957ccstt6C0tDTp+c+qU3z48GE0NDSgoYGrA9i5cycaGhrw8MMPAwC++93v4r777sPdd9+NjRs3wul0Yvfu3XFvFdra2jAyMun43HzzzfjpT3+Khx9+GGvXrkVTUxN27949rUBbagrCdcUjbnGOUlFYbEusU6xRKmDUhFOg7eJVpEvDkeIRuwu+FJxqACgv4JziwXEH/CnaALiXAxVFJgBA54A0acl1VVwKdWvPiKTROp1Whcoyru45k9HiJUvM0GiUcLl8aGvLXs/NdRtroFLJMT7mxpmT2emXLATDEFy8dQU0OhXs427s39syLyKvtSvKcNG21SAEaD/Vj892H8/ai41k8AZ9kgwKRQzzbb3PuGNMVakTQlWpE7MgVKnzMusYx6pSn6Cq1FnhV7/6FWw2Gy699FKUlJREx8svvwwAOHLkCA4cOIATJ06grq4ubpuenp6onZaWlqhyNZDc+vCHP/wBy5Ytw+WXX46rrroKW7ZswbPPPitq/oSdD0+ZWcZut3PKah+9B3XYaY0S4hfP6bXa8NqZ0zCqVLh9bcOM25Pw2nrSMoQPWttRbjTgq6tXRD8X2j6W90+1omVwBBsry7CpJl4YjAj8VGPt/P6To7B7fLi6YSmq83P5d5hhPn/c2wS724tt65agpjgvqePy2W/rG8EHjedg1Klx05fWCr6tEbw+U44bCIbwp7cPwz8RxJVfXIHSIuMUO/wTFZp/7HUYGLRh94cnIZcxuPna9VApeYRMEjw7CV6jKX+Kp0714dChDuh0Knz1unXThb2StDPT9piy/bkzA9j/yTnIFTJcf8N6aHWqhNuLtS/WzrDFhvffOAI2xGL9xXVYtmbq77pI+0IIRnRTm3936yA+/esxhIIsSqsK8IWv1EOu4BHV4Tmux+vG3Q/fHG1VIAWR+9p1f/tHKHTpieNMuPx484rnJZ0fhZJtIn8TP37rfWh0U9b68NLdO2bDO80tCIRCqMozCYpvsYKLFv/HkW4HFocDb4YjxaUGvbD4lqAdoYWA+58RtwuvnTkFTyCAIp1OWHxLSBdQ6HOGO+6414NdZ0/COeFHnlojLL41gx2h7R1+X1R8y6BUCYpvCekaEuEFFwDgCfij4ls6uVJQfEvIjqB9AAxh4QsGouJbKkaO6yrWolCt491e6BxkDP8DBQEQCAWj4lsKIsO2kgaUaIy828sEHqCEzoEhLIJsKCq+JSMMLilch3ItvyAqk8CO0HxYlsXhsaPo9fSBgOCCvPUo1/LXgMoE1mIZ4X+ZxBAunfe04yh6Pe0AgFWGjajQ8gvpygWuDyPwQCcjLFiWRZerCb2eUwCARTkbUKGtS3r+TocHly36XxlZ66WyKbW9ucycrSmeb0QixTafT1QaVGykWOz7CXNYRdriSO3tY1mk13AaKdQV4WhxT5p1xRVFJjAMgc3lxXiCVkrJIpcx0Z7Fbd3DaduLxVxkgMmoQSAYQmuntLZj4aLFCi5a3J69aHHdUjMKijjRrcMH2rN2XCEKzUasv4hbZI7sa8PQwPjsTihJKuuKcdm16yCTy9DfNYIPXm+E3zsx29OiUCgiWCg1xlR8i59s1xi/2dOU0Rrj9waOZlR8a+/wkYzWGB+0Nma0xrjZfkjyiHFsjXG78zCNGM9jqFMsERqFArqweppYsS0ZIfAHg7B5vaKOaTZwdcGDdidCKQT8IynU/dbUneLKQhMAoGd4PK3UVqVCjrKwg90hUQp1bVUhZ6/XioCEDxmEECyv49oInD5nyVhKr1wuw8pIbfGJ7NUWE0Kw6eLFIATo7BhBf580quDpsGR1OarquP7Fn75/Eh7X/EjfLakqwNavrYdSJcdw/zj+vuvwrM/92Q0/wu82PZXWeHbDj2b1HCiUbEId48RQxzgxC0V8a6GoUh/PQCo1Fd9aGFCnWEIKIy2WRNQVyxgGBeFo8aBDXF1xvk4LhYyBPxiEVWSvY2AyUjycRl1xaZ4BMobA4fFh3CXOqZ9KpEdxR780N1tzgR45WhUmAkF0SdxCqba6EEqFDHanFz0ZjFwuXWqGWq2A0+VDa1v2WhPlFeRg6XJOnGD/Z60IBKR7QEsFQgg2XboUxlwtPC4/Pn6vGcFZnlOyFJbm4ss3boRaq8TYiAPvvXQAtllU01bLVJIMCuV8YqE4xvO6xpiqUgtCValntk8dY8pMUKdYQgrDKdTDIh1Ucw6XBj0oMg2aYQjMei5aPJCC2FaOWgWTVg0WqadQK+QylIad666h9CKK1SV5IASw2t2wSZBCTQjB4nC0+KzEac4KhQxLajkxl5MtmROkkstlWLOaq385drwnq85pw/oqaHUqOB1eNB2RbvFIFYVCji9uXw2lSo6RQTv2fTh/egHnFhpwxc0XQG/SwuXw4m9/PojB3sz0uqZQKJmBqlInhqpSJ2YhOMYLSZX6OFWlpkyBOsUSUqgVHykGgGJ92Cl2ir85lhjDTrFNvFMMAGV54bZKadQVVxWF+wwPp+cUq5WKaAp1u0Qp1IurOae4f8gGp1vatNXli0tACDAwZMfoWOYif0sWF0OnU8HjmcCZFukWiJlQKOW4MFzLe7q5D8NDqf+OSIXBpMUXrlgFwhB0tQ6hubFztqeUNHqjFld8/QLkm43weyfwwWuNaD89uwrfFApFHFSVOjFUlToxU1WpM+EYU1XqxPapKjVFCOoUSwUBCsNp0KNuN4JsiJMGJKzAQHQUh3sODztdMfslN0pMMU5x7HdJUp7PRXl7rQJCWUnMoTLcTsky5hBMw2YFxlRbNWWcOFZ7/yj/wQTmIGRfr1OjpJA7x3NdMdFiQngHS8A7+I6Zk6NCdQXX+qn57ABYQqJDzM8weg4CQyaXYW19JQDgRHMffP5geL5CdgjvEAtLCMqq8lFTVwSWBT7/5BwCQVbw2gmfm9D2Yu1ww1yRiwu+uAQAcPxQJzpah/jPV2gIwRD+ITQRkfMnINBoVfjyjRtRubgYoRCLfe814/iBNu53bNpxKRRKNuBbogUXFRYoN01PpQ4FQiAhTBtCNoQeDcAKpFIHggJ2CO9IdNxCjQ43TK0xnggkPOdpI0T4BwQcY79fnB3BYxPoFeppNcY2nw8sC55BeIfQghtiAbVseiq11esRtCU0QgJDySh4+hi7BOYPBEMM7xC6RAyRT4sYD3hsCLIM7xA7f0COLVNqjHvdVsHthYbgfMBgfe66uFTqXrcFQRD+wcp4R4gF/4AMS/Xr4lKpe9w9CLAM7wiBfwRZwjtCkKNS2xCXSt3jbuWde0jMA7tIpGq/eD61YOTR5aekilGlglIm42p8PZ5ojfGM+6lVUMvl8AYCGHG5o5HjZCjW54AAcPj8cHh90KvF1fpFIsVjLg9cPj90KvGtWgxaNXJzNBhzetAzMo66kgLRNiJUl+Th02PtGA2nUBtzNCnbirC4qhADw3ac6xzC2mVlICk4iEKsXFqCju4RdHSPYEN9FbSa9FrdCLFoURFOnurD+Lgbzc09WL++JiPH4WPjhbUY6BuDbdyNY0e7sH5DddaOLUTdilLYx9043dSDfR+eQY5ejQIzfxuKuYZcLsMXrqpH02fncPJwB04caIfD5sGFW1dCJqfvKSmU+UCkxvid5paoYyzUrikVIo7xm81noo6xYLumFIjUGL925lTUMRZs15QCEcd419mTUcdYsF1TCkRqjCPtmna1nhBs15QKkRrjSLumVztPCLZrSoVIKnWkXdObPU24vqIeBerkn/8SEUmljrRrem/gKK4sXYtitUkS+5Ea40i7pr3DR3BZ0VqUalJ//oslkkoNAL2ePhy0NoKQdSjTlEhmf7k+Yr8dzfZDYEgIZZoqyexX6daG7Z9Cu/MwGIRQpl0iif1k+Ob+f027/SLAtWA8X6BPYBJCCImpK04+nZYQEnWELSJTqJVyWVSoK5UUarVCjkIDt79gtDgJqsLR4s6h8ZRtAOEU6kLOuWmTSHCrujwfCjkDh8sHy4i0KcCF+XoUFegRCrE4fS5zqc0MQ7BuXTUA4NTpfjid6YmaiUGlVmDTxYu5Yx/vwchwaqn6UrP2wlqUVxcgFAxh7+4TcNjTr0PPFoQQNGxZgk1bV4IwBJ1nBrDntdlXpqZQKMmzUMS3qCo1P7MhvjU8z2qM41Wpm9DvGZHMPhXfomQb6hRLTFFENEuEUwwAZn3qPYdLwynU/bbUHL7ycLS4Jy2nOBcA0D0ylvZDwaJS7k1jW780N1eFXIaacs5mS4f0/X5XLeNUms+cs2BiInNCWOVluSguNiAUYnHkqHQ37mSorC5AdW0hWBb45KOWjJ5nsjAMwUVfXo7cghx4PRP48J1j8Hrm1xvNxavKcdn166BQyjFqscFpmz+OPYVCWTiO8byuMV4AqtTF0T7G8098izrGie3PlmP8+wt/gFcv/kna4/cX/iAr850LUKdYYorDKdNiRbPM4briAYf4KFypkauZ7RtPLYJXkR92ikdtKav5FufqoVbK4ZsIwjKWXiSxujQPDEMw5vDAapdGwGp5WCm6o3cUHt+EJDYjVJTmwaBXwz8RREubRVLbsRBCsHEDd+Pu6BjGcJYjthdcVAetVgmH3YPGgx1ZPbYQCoUcl161BtocFRw2Dz569wQmUmwvNluUVOZj+y2bcNH21SgsNc32dCgUikioKnViqCp1YlQyOb4yz1WpuXZNsY4xVaWOtT8bqtRStV88n1owUqdYYorDkeIRt1vUohiJFNu9Pk6QQgSlYQXqMbcHbr94h68kVw85w8Dtn8CoM7UbPUNINFrckWZrJpVCjsqwrXN90rxxLMjNQUGuDqEQi7MSR4sZhmD1sjIAQPOZfgSC0j0MTSU/Pwd1dZyDf/Bwe1ZbEqlUClx0yVIAwNkzA+jrmRsthbQ6Fb50dT2UKjlGh+z45L1mBDP4M8gEhjwdqpaYZ3saFAolRagqdWKoKnViqCp1YhaaKjVDpNEGoEgLdYrFIiAZGVH9M6g40awgy2LEnfwNUymXIz9cj8yXQi2o3gxAo1QgT8cJUvWPz5BCzaOKK5MxKAv3Gu622uJVl0VQUxx2igetCIGdUb05keheXTjdua1vBCwm1bqTVbHms7+8lnM6WjoGEZwyv1TON5ba6kLotEp4vBNobR9KqJ4s/LPk32HqHNc2VEEul2FkxImOThEvDQTnlLyKckl5Lpat5F4AfP7JOXh8gdTVnkWrUgvP05ivw2VXr4FMzmCgZwz7PjiNIMuvxC2ZKrVEqtqE57+EfxzzmKeffhrV1dVQq9XYtGkTDh48KLjtiy++CEJI3FCr1XHb3H777dO22b59e9w21dXV07b50Y9+lJHzo8xTRKpDU1XqBIOqUlNVaqpKnVCVut60DUXqJdH5UOYO1CmWGEIIiiIp1CLriksiKdR28WmxZaZICnVqdcWTKdTjKe0PAOUFJshlDJxeP0Yd6b1BrSw2QSGXwenxY9AqTZrwovJ8KBUyOFw+9FnGJbEZQSZjsHo55yweP92LkIQRgqlotUqsXl0OAGg80olAILv1vQ0bq2E0aeHx+LH/k7NZjVYnosBsxCXbV4MJ9zA+PIfmRgFefvll7Ny5E4888giOHDmC+vp6bNu2DUNDwpkbBoMBAwMD0dHV1TVtm+3bt8dt86c//WnaNo899ljcNvfdd5+k50ahRFgoNcZUfIufbItvvdnTlNFU6vcGjma0xnjv8JGMpFJHHOOD1saM1hg32w9JHjHWK6RR6KZID3WKZ+DYsAXvtJ3BoCv5m1IkhVp8XXFYMCuFuuJyE+fUpu4Um7hjjzlSXmDlMgblBdw8OgbTS62Vy2SoKckDAJzrlSaFWi6XYXF1EQDgdAZqfxfXFEGjVsDl9qNVTAQ3BZavKIVOp4Lb7Ufzyb6MHmsqcrkMWy5dBoYh6OkaxdnTmVPdFktJZR4u2soJWpw71Y9jB7KbYj4f8AR9kgyx/PznP8ddd92FO+64AytWrMAzzzwDrVaL559/XnAfQgjMZnN0FBcXT9tGpVLFbZObmzttG71eH7eNLsl2eRRKKlDHODHUMU7MwlOllj6Vej6Lb1HmLtQpnoEu2xhax63odybvbKYqtlUSdoqHnC4ERC6gEQVqq5vrNywWk1YNvVqFEMuiz5p626KaYs6R7bCkX29aV8a9TWvvH5GsRnT5Iu6huscyDrtTWqVfuVwWVaJuOpXZaLFcLsOGcIum5pN9cDiy16IJAPIKctCwsQYAcHh/G0bnSJsmAKiqK8IFl3C9AE8e7UZzY+fsTmiOceOn38dVe7+X1rjx0+8DAOx2e9zw+fgfPP1+PxobG7F169boZwzDYOvWrdi3b5/gXJ1OJ6qqqlBRUYHrrrsOJ0+enLbNRx99hKKiIixduhT33HMPRkenP3z96Ec/Qn5+PhoaGvCTn/wEAQkf8CkUPhaKYzyva4ypKrUg1DGe2T51jM8/qFM8A0XacNTXnXwqtDkcKR71eOALJr+IGNUqaBUKhFgWgyJbM2kUChSG+xX3jtlE7QtwN4CqAhMAoGskdaGs6qJcMITA6vRgLE2ns7TQCJ1aCd9EEF2D6Yl3RTDqNSg3mwAAJ1uljxYvqzNDo1bA6fLhXMew5PZjqarKh9lsRDAYwsFD2Y+ILl9VhoqqfIRCLPbuOQWfxKre6bB4ZRnWXVQHADh+qBMnj9DFLBNUVFTAaDRGx5NPPsm73cjICILB4LRIb3FxMSwW/r/DpUuX4vnnn8ebb76J3//+9wiFQrjooovQ29sb3Wb79u347W9/iz179uDf/u3fsHfvXlx55ZUIxjgH999/P1566SV8+OGH+Pa3v40nnngC3/3udyU4ewolMVSVOjFUlToxC0+VOjOO8XxVpabMPeSzPYG5TrGOc3CH3MnfiLRKJQwqFew+HwadLlQajUntRwhBqUGP1lEr+u0OlIVbLSVLRa4Rw04XesZsWFZUKGpfAKgsMKG5dxBdI+NgWRYkkfiQACqFHGUFRvQMj6PdYsX6ujLRNiIwhGBxeQGaWvtxtmcIi0rzU7YVy8q6EvRaxnGucxjrV1ZAqZDuz0Aul2H18jIcPNqJplO9qKsuhEyWmXdPhBBsumAR3n6nCb19Y+jusaKqUpprlOzxL/riUvzljSNwOrz47OOzuGzripR+bzLB8voKBIMhHDvQjqYD7WABrFpXNdvTmnV2bXkEBoO4e8tU7HY7zHgaPT09cbZUKulaN2zevBmbN2+O/vuiiy7C8uXL8V//9V/4wQ+4vom33HJL9PvVq1djzZo1qK2txUcffYTLL78cALBz587oNmvWrIFSqcS3v/1tPPnkk5LOlzL/OTcwgs6RcVy+shaMTJr7WMQxfudES9QxvmrVEjCMNOtCRJX6zeYzUVXqa1ctg0IujYBPxDF+7fSpqGN8/bIVUEu0bkYc411nT0Yd4xuWrIROqZTEfkSVeldrM2x+L3a1nsCNdathlOhvP6JK/VrnCYz63Hi18wRuqF6NXJVWEvsRVeq3eo9jyOvAGz3HcH1FPQrUOZLYj6hS7x44hgHvGP7a34QrS9eiWG2SxH5ElXrv0DH0eobx4dBRXFbUgFKNNM8qEVVqWIEeTz8OjB7Bpvx1KNOUSGZ/hWEtAKDb04ETtkYQAGUa+iyxEKGR4kQQREWzrF4P/KEEb2CnKMpGosUWEbXIwGR7pT6HPTn15phRkRcWyxqzxSk2J6d8DJTlGSBjCJxeP6wuD+95zTgA1IZTqNstMW8ERdqJzGtxJVcD3DM4DqfXn/C4fPAJSZaajTDqNZgIBHG2c0o0V0A5mGXAO/jmsnSxGVqNEi63Hy0dQzzqxylcV77BEJhydVi5khPdOnionUufE7qmPErMiefDfy1it1Gq5fji1uVgZAS93VacPMFT3yyk3iwWQdVogUGAVeursOaCGgDAsQPtaG7sTHAdhOyLnI/gYASGwClkCI1MJckAOCGs2CHkZBYUFEAmk2FwcDDu88HBQZjNybWiUigUaGhoQGtrq+A2ixYtQkFBQcJtNm3ahEAggM7OzqSOS1n4EBZwe/344FQbzllG8PfmVoSCLFWlpqrUKatSj3rdVJWaqlInoUotxR2MIhXUKZ4BnUKJHAX3xnJYTAp1uO+wRWRdcWk46mKxOxESmQ5bYtRDxhC4/BMYc4uvMVXIZCjL5RzrruE0UqiLc0EIMOpww+ZKr9bVlKNBcZ4eLIBzPdKkIxNCsLKOexA/1Tog+jrPhFzGYM2KsBL1qb6Mq0OvWVMOvV4Nt9uPo0e7M3osPvIL9Ni4uRYAcPRwBwYttqzPIRGrN1SjflPYMT7YgROHO2Z5RucfSqUS69evx549e6KfhUIh7NmzJy4anIhgMIgTJ06gpEQ4AtDb24vR0dGE2zQ1NYFhGBQVFSV/ApQFj06lxLbVi8EQgtbBUc4xDkm3NiyUGmMqvsXP1Brj17uoKnUsVJU6+3iCfsnG+QJNn06CIp0OznE/Bt1OlOmSSzuMKFBbnA5Rqcj5Oi0UMhn8wSBGXe5onXAyyGUMSo0G9IzZ0DM2Hu1dLIbqQhO6R8fROTyGdTWppT6rlQqU5RnRO2pDm2UU62pTT6EGgKWVRRi0OtDSPYS1tSWSpOfWVRXicHM3HC4fegfGUFmal7bNWJYsKsaJ031ctLhtECuXlkpqPxa5XIZNm2rx97+fxJkz/VhUU4CCfH3GjsfH4mUlGBqwoaNtGB9/eAZfua4BGq006W9SsGp9NQghaNrfjuOHOhEKsVizsWbOpHqfD+zcuRPf+ta3sGHDBlxwwQV46qmn4HK5cMcddwAAbrvtNpSVlUXrkh977DFceOGFqKurw/j4OH7yk5+gq6sL//RP/wSAE+H6/ve/jxtuuAFmsxltbW347ne/i7q6Omzbtg0AsG/fPhw4cACXXXYZ9Ho99u3bhwcffBDf/OY3eVWqKec3NYV52L5mMXYfP4fWQe6BfeuqOjCpZLbwEHGM32mOT6WWSZVKrY9PpX4rkkotkzCVetkKvHZmSiq1PMOp1AoJU6lrV2NX24m4VGqDUppU6ohj/HpX5lKprymrx9t9x6Kp1NdV1KNQwlTqK8xr8TdLU8ZSqb9Q2IBPho+iL0Op1BtyGwAAvZ6+jKRSL9dH7LfjuK0RwNxNpb527+OQ69L/3Q64pHt5NNehkeIkKI6IbYlIhS7U6cAQAk8gAJuAIisfDCHRfsV9NvEq0BXhSG93CmJbAFBdyD0oWmxOuFNQsY5QW8Ld5Fr7029NVFOaD4WMgd3lxcBo6srYsSjkMiyt4UR/TpyVvqWQTMagfmUFAODYqV74JzKrdltW9v+z999xUlxX/jf+rs6Tc86BicwMzBAFAgQIkFCWg/xdry05rb3S7k+2v4/T4yCv5dWuH3vtdVh7vQ5aW84ICQkJEBJJZBgYJuecc+5J3f37o6aHCV09Xd09wwD1fr3uS6j61qlb1T1169Q593MCSEgIwWKB8+er3abW7SiCILBpyyr8/D0wjk5w6r2yZR/DYmTmxrF2OqJdnN9A/rlqpVzTMvLhD3+Y73//+3zzm99kzZo1FBQUcPTo0RnxrcbGRtrabv4t9vX18elPf5r09HQefPBBBgcHOX/+PBkZYskttVpNYWEhjzzyCCkpKXzyk58kLy+P999/fyaNW6/X8+c//5nt27eTmZnJd7/7XT7/+c/zy1/+cvkvgMJtgdUxViLGtlFUqe2jqFLbR1GlVljJCBblqXABg4ODopLq6WMYvL1oGOjntapS/PUGns7Mtb2TeeGb5L8VF9M2NMz9CUlkhMwVvhJs9AfAAvnNLZyvbyIhMICHMlKn+9vuPn979/AIf75ShEal4tNb1qGZ/wbaATt/u1RE1+AIO9ITyIxaWBfU3nis9scnp/jf9/IxWyx8cEsWwT62I96L2bHy/o0ayhtEsa3deasctyPx6xYsMDI6zl+OXMdisfDIzixCAr3FNVo27dj+wN5xzWYLrx25zuDQGDmZ0eRmxU7vI2FL7rHnbR4bm+T11/MZH59iTU4MOdmxC8Zk247EB7KvhYXBgVHeeqOAyQkTicmhbNmWIh2NlXrQlHtLknxgtb29oqiZq+9XAZCYGs7GHamoVCo710HmeGRfz4U/otGxUT7z/36QgYEBl0WxrFjva+6w6U5bCgq3Cuvv+PuvHcdjXu3q2q5ejhVWYbZYSA4Luhkxlpq6pYLJEtub+wc4XFTBlNlMXKD/TMTYIhWqkHnc9qEhDk2rUUf6+tyMGEvakZogbG/uMo5wsEx0iEO9vG5GjGVeB1S2j9s/ZuRAZQnDkxMEGjxuRoxl2pHqPzQ5NiO+5avTz0SMpaYrQfL62N5unE6h7hkfxUujmxMxlrIltV1lY/u4aWomYqxXaeaIb0mdg1pl+4HFVvcps2kmYqwV1HMixmqJBx854zdZzJztuk6zsQu1oJoTMbbV3952W+OxWCzk912jydiKgDAnYqyWmIzVgu2XR7YSRSwWCxVD12g0isuxsvzyiJ6OGGskro/KxgP4yNAoDyR/Zknm+vbeLrfYHBwcJDww5K6Y75VIsQNY6w73j48xNuV42ZkIHzGFtW1YXg1Xq+p06+Cg7EhWkJcnnjotU2YzrQPO1Y5NDJ0Wyup0fl2xXqshLtQfgCo3RIvT48U1wPVtvRjdVPrHy1NPUqxYC7mwwoZAlIuoVAJ52eJNsri8lZHRpU1BMRi0bNwoRkILi5rp63N8Dby78PXzZPvOdAQBaqs7KSlqXnynZSZ1dTSbrWOsaOfM0WKmJpd23beCgsLtRUJoIHuXOmK8lOWafJchYnybl2v6wG1erumR6OwlLde0LyJnSSPG20NzlrRc0/rApS3XlOG7ZqZcU9EKjBh7qHVua3cLilPsAAaNFn+9AYB2GaWZ5jjFdkRy5yvOhnp7o1WrGJ8y0T0i7yYqCAKxgf4ANPT2L1SZdmAMCdNOcXPvABMSk5yUiuFsO6siRYezqq1H0rmXEnWcP6Zgfy9C/L0wWyxUNHUt6OConfnKvtkp4lrf+pZe+gddq6tsi7joQEKDfTCZzFwvahI3SilcI3FNJU7CVt+4+GBiogMxmy2cu1AtBlFtqEY7pEotE4tKwKISiIgJZN10mvK1K/U0NUhMdJLfj+3rI4mkurVEEyAxLZxt+7JQq1W0NPTw3psFjI1PLbEqtdSQbClSy7/+CgoKTmLjZipYINFWKrWiSj1zXEWVWuLBY7opqtSKKrUjqtQKKwfl23CQcC/564qtTnGP0SjrDapKEGZUqJudWFccN12aqbGvX/a+AAFeBvw9DZgtFhq6nbMBEBsagE6jZmRsgtZe19cCp8WLqdzlDR1uWwsa4OdJbEQAAIWV7o8WC4LA+jXxAFTVddLbv7TRW0EQ2LQpCa1WTU/PMKWl7j8nR0jNiCQlXUxVev90BX29yx+1XozohGB2PpKDTq+hu2OQd16/xsiwa2rpCgoKdxbKGmP7KKrU9lFUqe2jqFIrrCQUp9hBwrxEB1dO3WFPrRZ/gxhhlptCHT2dQt3cPyBrP4DoQD8EoHfEyNCY/Ju7IAgz0eJaF0ozadQqEsNFO1Vtrt/kkqKC0WrUDI6O09It/7pIkZMmqmNXN3QzvAQpzqHBPiTEimtlrlyvX3JxJ09PPevXizfo6wUNtySNWhAE1t+TRHikP1OTJk4cL8E4uvJk/UMj/Ln/8Vw8vPQM9o3yzsFr9PW474FCQUHh9udOdIwnFMd4BtuOsftekC6X+NbciLEivmVFEd9ScBTFKXYQa6S4fXRIllMTOR0tbh2U6RT7i9HelsEh2ZOjh1ZL6HSd5Ibefln7WkkME53Z+u4+plxQEV4VJaZQ17T3uGQHRMXoVdGivZK6dpdszSY0yIeIEF8sFgs3lmBtMUBedhwqlUBrxwCNrc6/aHCU5KRQoqMCMJstnHm/4pYoQatUKrbtSsfH18DI8DgnjpcwuQLX7voHerH3iVx8/T0ZHRnnndeu0drovglZQUHh9udOc4zfVFSp57BQlbr4tnSMFVVq2yiOsYIjKE6xg4R4eqGeKbHk+I0yatopbhmSlz4c4uWJQaNh0mSiY0j+jS0hSEwJrnfSKQ719cLboGPKZKaxxzkbAJGBvvh46JmYMlHX0eu0HSuZ04JbjR19DI26b8JamxENQEV955IIYvl4G1idJq5fvnS9zuUXBIshCAL33LMKg0FLf/8o+fl1S3o8KfQGLbv2rkZv0NDTPcyZk+WY3fig5y68fAzseTyX0OnI9qm3CykvbFZKNikoKMyQEHKbi28tRyr1bS6+9eQc8a3i265c00PzIsa3o2O8lOJb6wKWVnwr3WftjPhW4UA+zYpjfFuhOMUOolGpCJ2uV9w24njUN8pHTIPuGBmRNfkIgkDMdLS40YkU6rggfwCa+waccsAEQSApbLrWcIfzNyVBEEiZjhZXtHQ5bcdKgI8nUcF+WIDS+g6X7VmJCPEjPNgXs3nposXZ6dF4eugYHhmnpKJ1SY4xGw8PHVu2iOWrysrbaG5Z+gi1LXx9Pdi5O1MUtWrq5cLZqhXpbOoNWnY+lENiajgWC+Sfq+Ly6VsTZV9KjFMTbmkKCncjiiq1fe4Ex1hRpZZGUaVe3P5KV6VWkEbjys7j4+Po9Xp3jWXlMSPjKBLu7U3byBBto0OkB4fO62vbhK9Bj7dOx/DEBO0jw8T4iY6u1BQ6W3Q2NsCPqu4emvoH2BwbI7GD7c3B3p546XSMTEzQMjBA3HTkWPrACzclhwdyo6GN+u4+Js1mNOqb71AkxXFt2E+NDiG/uoWmngGGxsbx9rj5m5GszSth3yJAZmI4Ld0DlDd2kpsWjUatlixdKOd812ZGc+R0KeX1nWSnR+E1e5wSR7BI1Ea0VaZOq1Ozfk0cpy9UcaO0maSEELw8Zx1D6hWVC89D0dGBpKdHUFbWxrnzVTzy0Bo8PByT17fYKs6H7XMTd7B9LSyCQHC4H9t2pXPqeAk1VZ3oDTrWbUiQMiSxXepHIdFfYvyS5rGg1qjYtDMN/yAvrp2vobqsjcH+Ue7duxrDvOvmNn1oW4akxu4G9pz4d9Sert23TUtcXkxhZXGnz/XzpnrA/l3Iqkp9tLBq5qXxTB1jG/1tYc+NtqpSHy6umHGMrXWMF9iRmDfsHdcaMT5UXD7jGM/UMV5gR2LukzgDgZuq1AfLS2cc45k6xo5iljgDlWXGMT5QWTLjGM/UMZZhxzY3VakP1BQxMDHGgeqimTrGtvpLIfWJVZXaWsf41foiMYJs8JS0ZQupqdiqSm2tY/x60w0ejVlDiMHLZn+T2faPSKqusVWV2lrH+EhrAXsj1hLh4WfbjsRIpcavEkRV6ve7rtNi7OJk53W2h+QS7RkosYcUEuclQF5ALhYEmo0tXOq5xobAPKI9w22bsSz8uxCReJkkiKrUZgSajbUUDuRjtqiI8Vz4HD8lWZhc4VYg69s4cuQIH//4x0lMTESr1eLp6Ymvry/bt2/nu9/9Lq2tSx/9upVEeFtLLDn+1k0QhJvriofkrSuOCRBvMB1Dw4zLfNMqCALx09HieifTn0P9vPE26Jh0MYXa19NARIB4DSrdULM4JiwAb08945NTVDe7bs9KRIgv4cE+mM0WCsuX5recEBdMWLAPUyYzVwuW5+1hXl4C/v6ejI1Ncu5C9S2L0kbHBrH53hQASouaV2QNY5hOgVoTy4792Wh1ajrbBjj2aj79igCXwl3C3T7XO8KdtsZYUaWei6JKbR9FlXpx+3ejKvVLL73E+vXr8fHxITQ0lMcee4yKioo5fX75y1+yY8cOfH19EQSB/v5+t9jdsWMHgiDMaZ/97Gdljd8hp/i1114jJSWFT3ziE2g0Gr785S9z8OBBjh07xq9+9Su2b9/Ou+++S2JiIp/97Gfp6nI9TXYlEuEtpk93G0dkKTda1xU3D8pbV+yj1+PvYcACNDlRmineuq64p88pR2h2CnWNCynUIEaLASqau1x2ylSCMLO2uLi2zW1OniAIrM0Q3+RV1HUwYnR/NEwQBDbmiRHS2sZu2jvdp6IthVqtYtu2VFQqgZaWPsrK3Xfjl0tSSji568Xzz79cR3Wl+1Lg3U1UXBB7n8jD29eD4aExjr12jaba2//e9s7OL3Nuzzddau/s/PKtPg2FJUCZ6+VxJzrGiir1TZbbMVZUqeeiiG/dfpw+fZpnn32Wixcvcvz4cSYnJ9mzZw8jIzeroIyOjrJv3z6+9rWvudUuwKc//Wna2tpm2ve+9z1Z4xcsDngUmzdv5utf/zoPPPAAKhvpO1ZaWlr4yU9+QlhYGJ///OdlDWQlMTg4iJ+fHy+dOYrBe266ya8L8hmanODJVZnE+M5KFZFI0RHM0DM6yiuFhagFgc+uX49GpQKTRP9538bpmnoKW9vJDAtlV3KiTfs2McOkycSvz+UzZTbz1Losgr29JPtLpTF39A3z6uViNGoVz2zPm0mvkrQjsX1i0sTvTuQzZTLzyMYMIgN97duR+lVO9x+fnOJP7+QzaTKzb2MasaH+ssYjZd9itvDW6RI6uodISwxjS26i/fFI/PlIfi8AFjh/pYaKmg78fD14dG8OarUKQa4tqf4SYy0rbeXylVpUKoEH9mYRHOwzMx45CFIPfVK3kvlpiRYL+ZdqKStuQRDg3h1pxCeGyLazaH8pJO3b3j5unOT9Y8V0tPYDkJYTw9qNiajlpjjLGOfo2Aif+cqTDAwM4Dtds9xVrPc1d9h0py2FlcPdOtf/4NXjeHjNnestUn/eNrbXdfVytLAKs8VCcljQ3FRqO0uBHLXf3DfA4eIKpsxm4gL956RSS2Zfyjhu+9AQh6Yd4khfn7mp1JJ2JO5nNvp3j47w6rRDHOrlNTeVWsZ1AGymPfePGTlQWcLw5ASBBo+5qdQy7Ej1H5oYn0ml9tXp56RSSy4lAwTJazR3u3E6UtwzPoqXRseT8VkE6G+mUkvZkdqumrd93DTF4ZYbdIwNoVdpeCwmh2CD9yw7tocplT49v/uU2TSTSq0V1DwQuYYwg/9NOxIPMY6O32Qxz6RSqwUV94WuJdIjSLL/Ytvnj8disXC17zrNxhYEBDYG5RLlEXGzv8TDh1qw/QJp/qOBxWKhbOg6zcZaALL98ojyiANgZGiUR1d9Yknm+raeLrfYHBwcJCIoxKkxdnV1ERoayunTp9m2bducz06dOsV9991HX18f/v7+LtvdsWMHa9as4Uc/+pEsW7NxaIHHhQsXHDIWFRXFv/3bvzk9mNuBSG9fKvq6aRkenOsU2yHQwwNPrZbRyUk6hoeJkvGjigvwo7C1nYa+fiwWC4K9O/A8tGo1MQF+1PX0UdvdR7C37fUk9gj19cLXQ8+gcZz6rj5WhQfLtgGg06hJjgimvLmTsqbOGafYWfRaDWlxYRTVtnGjulXSKZaLIAisy4zlrdMlVNR1kpUSia+3wS22Z5OXE0djSy8Dg0YKS5tZmxXr9mPMJy01nPb2fhqbejl1poKH9udg0GuX/LjzEQSBvI2JTE6YqK5s5/1TFag1KmJigxbf+RZgFeAquFRL2Y0mym800d0+wL33Z+C1BL8NBYVbhTLXO4dVlfqYA2uMncEqvnW4aPE1xs4gZ42xM1jFtw6WubDG2A6y1hg7gVV860B1sQNrjOVjjRjPXmM83zF2Basq9ew1xvMdY1ewRoyPdxTQahTXGM93jF3BGjE+23Wd5uk1xvMdY1ewRowFLDQZW7nUc22BY+yq/XSftaiw0Giso3AgH2DGMV4qdh3//1zWD4GbGiKD87Jd9Xr9oloTAwNiNmRgoNz14PaRsvuHP/yBV155hfDwcB5++GG+8Y1v4Onp+N+R03fUiYkJKioqmHJjKsztQOT0uuLWYcfTmQVBIHraEW4ckJcuG+3nh1olMDwxQe+oUda+AEkh4g+mpsu5ckiCIMw4wlVtrqWtpMeI4mS17T2MTbj+u1mdFIEgCLT1DNLZ576UoPAQX6LD/bFYLFwraXKb3dnodRo25k7L9pe10DfgvpQpKQRBYMs9q/DxMTAyMs77ZyvdmuondyybtiSTkBSCxWLh9HtlNDe6XrJrqVCpVeTek8y2favR6jR0dwzy9oGrtCj1jBXucO7WuV4uiiq1fRRVavvcCarUe8MVVWp79merUk+Yb78KDjExMfj5+c20l156yW5/s9nM888/z5YtW1i9erXbxiFl9//8n//DK6+8wsmTJ/nqV7/K73//ez760Y/Ksu1Q+vRsRkdH+ad/+if+93//F4DKykoSExP5p3/6J6KiovjKV74iawArEXvp013Do/yh7AZalYrPrdmIyhq5lVI4nJ6zijs7OVFbS4SPDx/MzLSTfrzQzqHiMhr7BtgSF0tedKRN+wvtiP8dm5ziN+fzMVssfHRDDgEeHnbHactO7/Aof75QiEoQeHp7HgatZtH0Zlt2LBYLB84V0TM0yj3pcWTHR8i3M6//qWvVVDV1kRARyP3rUm30l5eKa71uPf0jvP5uIQCP7soiOMD2G1WpVGLJ84KZc7NYLLz3fjlNrX2EBvuwf+dq25kAMtOkF+vf2zvC22/fwGQyk50dw9ociSi1ZOaXm9KbLWA2Wzh7soyGum5UKoEd92cSHR0g244sZKZPz7c/NGDk7DvF9HaJwnmZuXFkr4+fSTeVvD4yxqOkTyvcau6muf4HBxamTzub9mw3ldoN9u2lUs+x42Ratd1Uaofs2E9L7h4d4WB5KUZbqdQO2F8sHdpuKrUMO1L97aVSL9hF4hj20qrnp1LbU6V2Jq163DQ1EzHWqzR2VamdSauen0ptV5XaibTq+anU9lSpnUmrnp9KbU+V2pm0aovFQu9EJ0H6MEBMn34i5enbIn26qalpjr3FIsWf+9znOHLkCGfPniU6OnrB586mTy9m18qJEyfYtWsX1dXVJCUlOWRbdqT4q1/9Kjdu3ODUqVMYDDdTB3fv3s1f/vIXueZuO4I8PNGp1UyazXQbRxbfYZqY6R9Sx/CwbCGL+AB/ABocUGibj0GrIcpfPHZtt3N1agO9PQny9sRssbgkuCUIwky0uKyx0y0CWdnJ4kuCurZeBkbkR9KlCPL3IilWjJBfKWpcEsVmQRDYlJeIRqOis3uI8up2tx/DFoGBXmzenAxAYWETTc23LkKrUglsvS+N2PhgzGYLp46X0Ny0ciPGAD5+Hux5Io+U1VEAlFxr4L03ChgdHrvFI1NQcB93+1zvLHei+JaiSn0TRZXaPooq9eL2rQ7xUuOp0bmtAfj6+s5p9hzi5557jsOHD3Py5Em7jqtc5NjduHEjANXV1Q7bl+0Uv/766/z0pz9l69atc6JamZmZ1NTUyDW3KENDQzz//PPExcXh4eHBPffcw5UrVyT7nzp1aoEktyAItLe7x+FQCQKRXmIKdcuQ4ynUfgYDvno9ZouFFpkq1HGBYuSsdXBIdmkmgKRgawq18zeOlAjRQaxsc60EUkpkMBq1ir4RI6298hW15xPo60lMmD8AN6rdWyYkLzMGlUqgtXOAlo5+t9q24u2lZ12OuK7kamEjw8tU/zUpKZTUVHG9zPtnKxkcdN8LBbmoVCru3TnLMX63lOYVnpasVqtYf28KW+7PQKMVyza99dcrNNR03uqhKSi4heWe62HlzffOcic6xooq9U0UVWr7KKrUdy8Wi4XnnnuO1157jRMnTpCQkHDL7BYUFAAQEeH42nDZTrFV8Ws+IyMjskSgHOVTn/oUx48f5/e//z1FRUXs2bOH3bt309LSYne/ioqKObLctsbsLJHeYuS1Rca6YoAYPzGFpEnmumJ/DwP+BgNmi4WmfvklfBKCRae6Y2iEwTHnbtyrwoMRgLb+IfpdiMjqtBpSIkUHu6TBPeV4claJEbvK5i63llHy8TKQniSmzVwqbFiy9bdpyeGEBvswOWXi3JWaZasjvH59AiEhPkxOmjhxsowJN6zzdpb5jvHJd8uor1v55V7ik8N48IPrCAzxYWJ8irPvlHDuvTImxidv9dAUFFxiued6WJnzvbPcaY7xm0sUMb6t1xjfAY5x2EzE2P1rjBXH+O7j2Wef5ZVXXuGPf/wjPj4+tLe3097ejtF4029ob2+noKBgJoJbVFREQUEBvb03swR37drFT3/6U4ft1tTU8J3vfIf8/Hzq6+t54403+NjHPsa2bdvIzs52ePyyneJ169bx1ltvzfy/dXL81a9+xebNm+Was4vRaOTVV1/le9/7Htu2bSM5OZkXXniB5ORkfv7zn9vdNzQ0lPDw8Jlmr7yEXKJ9bjrFchyYuGmn2Jk06PhAfwDq+uSnQHvpdUT6idHtaiejxd4GHTFB4vjL21xzVlbHiY5mXUcvQ25wYiOCfAkP9MFstnCjxr3R4rXp0eh1GvoHjZTXLU1NXUEQ2LohGbVaRUt7P5W1yxNtVKtV7NiRjqenjoFBI2duofAW3HSM4xNF8a33T5ZTU7Vy6xhb8fHzZM/juazOi0MQoK6qg8N/u0p7i3PLFRQUVgLLOdfDyp3vXcGqSn3bim8tRyq1Ir4lyXI4xg/Nixjfjo7xUopvrQtYOvGtO5Gf//znDAwMsGPHDiIiImba7CU3v/jFL1i7di2f/vSnAdi2bRtr167ljTfemOlTU1NDd3e3w3Z1Oh3vvvsue/bsIS0tjS9+8Ys8+eSTvPnmm7LGL3vm+Nd//Ve+9rWv8bnPfY6pqSn+8z//kz179vDb3/6W7373u3LN2WVqagqTyTRnPROAh4cHZ8+etbvvmjVriIiI4P777+fcuXN2+46PjzM4ODin2SPM0xuNoMI4NUXvmONR02g/PwSgb2yMwXF5N86EADHaW9fbj9mJSOKqUFG6vsqFFOq0KPHte0Vrt1NjsBLo40lUkC8WoLjRPWluuSni2oKyhg5Gx9036el1GnIzYwC4VtLE+BJFU/18PcibLst0qaCO4ZHlWZ/q6anjvh1pokPe0se16/XLclwpVCoVW7enkpwShsUC585UUlpsP0q0ElCrVeRsSOT+x3Lx8fVgdHicd9+8wdVzVUxNuu8hUkFhuVjOuR6WZ75fbK5v6Ozj/ZI6t2brKKrU9lEcY/vcCarU+yIUVeq7BYvFYrM9/fTTM31eeOGFRfvU19fzwgsvOGw3JiaG06dP09PTw9jYGFVVVXzve9+TLTQm2yneunUrBQUFTE1NkZWVxTvvvENoaCgXLlwgLy9Prjm7+Pj4sHnzZr7zne/Q2tqKyWTilVde4cKFC7S12f5RRkRE8Itf/IJXX32VV199lZiYGHbs2MG1a9ckj/PSSy/NkRmPiRGdIATbTa1WETFdmqlpeGB6u0Wi3dzPoNUQ5i2qGDcODti2L0GEnw86tZqxqSnah4exCNNqkQ62pNBABKBzaIR+4xgWmNMcsZEQGoBeq2FkfIKmHok0bgfHkxUvRovLmjuZMJtmzmfR85IgMsSP0ABvTGYLhTVtC+wtsC+Brb6piWH4+3owPjHF9dLmeecr2Gx2jy3R0lMiCAv2YWrKzJlL1Zix2O0//zuc+S4ldpDqHxziy5YtqwAoKW2lsrpDlEmUOq4g2GzS37ftayTVX1Cr2HRvCunTQlZXL9Vy7Wr99O/Uhh2VRJNCcjzyxmmrb0iEPw98eB3JGaIAXHlRC28duEpHx4Cd6ybRFBRuIcs518PyzPdSc71gAaNxguPXqyhp7OBMcR0Ws8XmDVNqqpe6wQoWSLSVSm1yj30sEO2/MJXaPGVGMLOg2Run1HFtRoynTBJ2BJvN3nFDPLx4cv4a48kpu+e8oJkF2w0Jx3hiQp4dyWML+GgNC1Opx8exWJBogs0m9dBgUC+MGPeMjUrakWpmiaZTaW2sMR6RHL/JrLLZpC6RStAsiBi3GQcwWVQ2m9zxg4at81Kpm0d7JftLNcnxoCIvIHdOKnXzaDsmBNvNorbZzBZsNoWVg+ySTMtNTU0Nn/jEJzhz5gxqtZrc3FxSUlLIz8+nrKzMIRvbt28nNjaW3//+9zY/Hx8fZ3xW5HZwcJCYmBheen9hSSbrTfZyazPnWxtJ9g/koeQ0yRJC80s1XWxq4nJzC8mBgexflbKgu2CSeBi2wNHyKqq6e8iLjuSeeDGqKF3aaeG21wtKae4bZHNCDHmxUXP7S/wK5tt5v7yeosZ2ksMC2Zu1cPyLlTqyYrZY+NOpAoaM42xbnUBGTJjd/ottxwIN7X28c7kcjVrFR+7PxaDTSpbHsWfHFi1t/Rw7W4YgCDyxJwd/H7G0ldyySIsde3BojEPHCpiaMrN+TRyr06KkS/w4WLZqsTFZf3EFBY3cuNGIIMDu3auJDPeTHKft40p8IPcaWawfWyi+0UTB1XoAVqWGs/EeifImtpA727hYqmn+By0NPVw6VYFxRLy3pGZFs2ZjIhrt3PImtq7b6NgIn/nyE0pJJoW7iqWe76Xm+v/423E8PL2obO3mRFE1FiA9JpRtmQkL109LTdFSt6VZ2x0q1+SCfUfKNTlbqgkcLNckaUdq4rj5T4fKNTlwHeYwq8SSQ+WaHLAj1d/Rck3OlGoCFpRrejI+iwD9wnJNzpRqAhaUa3osJodgw8JylM6UaoKF5ZoeiFxDmMF/oR0nSjUBC8o13Re6lkiPIMn+jm63jmd+uaaNQblEeSwUcZJTqmlkaJQPpC5NSSZ32byb5nvZkeKdO3fy7W9/e8H2vr4+du7c6ZZBzSYpKYnTp08zPDxMU1MTly9fZnJyksTERIdtbNiwwa4kt16vXyA1vhjRvqLD0Dwkc13xdC2upoEB2SnICdMq1HW9fbL2s5LsjhTqyBAAajv7GJtwXkxIJQgza4uL6tvdkq4WG+ZPkK8nUyYzRTXuTW+JDvcnJiIAi8XCpRv1brU9G18fAxvWJgCQX9hIb5/jZb9cJScnhsTEECwWOHWqjL5lPLYtBEEga00sm7auQhCgqqKdMyfKmJq6PdKRo+KCeOip9SSliZNmRVEzb/31Cu23sAQWwOjUhFuawp3Ncs/1sPTz/WJzfUpkMDuzkhGAsqZOzrg7lfoOE99SVKnncqeIbymq1LZRxLfufGQ7xadOneKnP/0pjz32GCMjNx+aJyYmOH36tFsHNxsvLy8iIiLo6+vj2LFjPProow7vW1BQIEuS2xHCPL3QqlSMmaboklGvOMzbG71azbjJRPuwvJtNXKA/AtA7aqTfKH/NaVJwICpBoHt4lL5R5xSkQ3y9CPYRaxZXtLtWniktJkQszzRspLl7wCVbIN6wclPFdLji2jZGx9z74L4hOw6VINDc3k9D69I5NimJocREBYjlic5Ximlqy4AgCNxzzypCQ32ZnDTx7olSRkaWp0SUPValRbBtZwYqlUBjQw/HjxQxNnZ7qDvr9Fo27Uzjvoey8fTSMzxo5L03b3DhRBljo7fGsdzy1g9Zc+jfXWpb3vrhLRm7wvJxq+Z6uLXzveIY20dRpbbPneIYK6rUtlEc4zsbpyQa3333Xdrb29m0aRP19fVuHtJcjh07xtGjR6mrq+P48ePcd999pKWl8cwzzwDw1a9+lY997GMz/X/0ox9x6NAhqqurKS4u5vnnn+fEiRM8++yzbh2XWqUi2keMFjcOOu7QqQSB2Olocb1MFWqDRkO0v3jMmh75TpmHTktMgLh/RYfzDm1GtJjqXNLc4dLDgl6rIT1aFO+6Ueeem0pceAAh/t5Mmcxcr3KvQJO/jwerU8SHrQsFdUvmrFrVqD09dAwMGbl4rW5JjmMLtVrFzp3p+Pl5MDo6wfH3ShhfAeWFYhOCuX9fFjqdhq7OIY68UcBAv/seBJaayNgg9n94A6syxWULtRXtvPmnS1SWtNxSxW8FBXss51wPK2e+T4kM5r7sm47x6WL3O8a3tfiWokptlzvBMVZUqaVRVKnvXJxyiiMiIjh9+jRZWVmsX7+eU6dOuXlYNxkYGODZZ58lLS2Nj33sY2zdupVjx46h1WoBaGtro7Gxcab/xMQEX/ziF8nKymL79u3cuHGDd999l127drl9bLG+8p1igPgAfwDq++WnQScFBQJQ3e3cH3hqmFgjuLKz2+lJPiUiSIzwjo7R1j/klA0rWfHhCEBzzwDdg66n6wqCwIYMcb11eX0Hg25WcV6THo23p56R0QmulTS51fZsDHot2zdPpw3XdVFTv3w1e/V6Lbt3Z4qlmgaMnDi5MlKWwyL82PdwDt7eeoaGxnj7zQJab6OyRzq9hg3bUtj7eC4Bwd5MTExx5Uwlx16/Rk+Xa39Hcji3//MUPPpll9q5/Z9ftvEq3DqWc66HlTXfp0TddIzLm5cgYqyoUtvljnCMFVVqSRRV6qXHXUul7qblUrKFttRqNW1tbYSGihG+F198kRdffJEvf/nLvPjii5jceFO8VVgXlb901obQ1iwhrB7jKL8vKUAtqPhczgY0tmojmhcqEoxOTvKrq/kAfHJtLt46nd3+cFM8aXRigl9fEpU1P75+LX42RBxAWsxpctLEb87nM2ky8+TaTCKm6xdLCltJ/DpOFtdS1tLJqvAg7s9atehxJcWfzHC8oIqath4xbS0nedH+jth/+3wpLV0DJEcFszN31cIdpESn7OtcANDU1sc7Z8sRBHh0VzZB/l52+y/4TOrBx8bm60WNFBQ3o9GoeHRvDr7TAl/iMZZWgKu/d4QjRwuZnDQRHR3IfTvSxPqfMoWn3CbANY3ROMHp46V0dQwiCLB+UxJp02rPDuHm8Thj32w2U1Xcwo3LtUxOmBAEWJUZRc7GBHR6LUbjCJ/50uMrVnzjbhLeuFu5m+b6//irKLQ1h+mpeI74VnQo21bbEN+a1X8+iwlkOSS+5YJ9R8S3wHkBLofEt+zasS9g5ZD4lh37iwlnOSS+5YAdqWMvEN9KysJXL/Hc5oQA13zxrcfjsggyLBTfsmfHnoDVfPGtR2PWEGJY+Mxjb/z2BLjmi2/tjVhLhIdtkU9nBLjmi29tD8kl2jNQsr+c7WrBvEB8a0NgHtGe4bb723jIGB0a5UNpf78kc33iy/8vKk/D4jssgnl0jNqnv3tXzPeyI8Xzfeivf/3r/OEPf+AHP/iB2wZ1uxBo8MBLq8VkMdM24nikx1OrJcxLvKnITaH21OmI9BUdWWdSqLVqNYnB4g2h0oUU6szpmsU1Hb0uCW4B5CSIKcnVbT0MG93zJnX9dLS4uqWbXjdEoGcTExFAfHSgWEc3v9alms2LkZMZQ1iIL1NTZk6dr8Rkct+b/sUICPBi5/Ra3ubmXs6dr3JrpMRZPDx03L8/m8TkUCwWuHyhhssXam6rNGSVSkVqdgwPf2QT8avEmszVpa0Yb9E6YwWF+ShzvcicNcZLETG+w9YYL0nE+E4S36pZ2lTq1xqWVnzrUFPBkqZSH2u7vqRrjE93XVvSNcaXe/NXXMRYwXFkR4obGhqIiYkRI0azKC4uJj8/n49//ONuHeCtwNFIMcCxuirKerpYFxbF1qi4hcYkIr+XG5u52NJMUkAAD6WkLtp/9ouqGy3tnKmtJ8LXmw9mrbbdX7JEFDT09vNmYTkGjYZn7slFrVI5FeH928UiuoZGuCcljjVxEXaPu1jk941LpbT2DpIdH8E96XEuR4oB3rtSSW1rD7FhAezbkDb3QxcixQAjxglePVrA5JSJzWviyUiOsNt/zmcyIsUAIyPjHDp6g/GJKdJTItiUmzB9jCUu1TS9uamph5OnyrBYIDU1go3rJSIlyxQpvtltumTTlXoAIqL82XZfGnq9drEdl2Q8TtvHQntzHwN9I6RmRQMokWKFW87dNNfbixRbWbRckwullMCBiLGL9heLGLtSqgkWRowfnnaUF7ezeKkjcCBi7EKpJnAgYuxkpNiKI+WanC3VBI6Va3K2VBMsLNf0aEwOIfPKNTlbqgkcK9fkbKkmcKxck7OlmsCxck3LHSlu6+lyW0mmiKCQu2K+lx0pjouLWzBJAqxevfqOmCTlcnNdcb+s/azrihsHBpiS+dY2aTrS2zY4zNC4/DeOMf5+eGq1jE1N0dDbL3t/KxnTIlklTa4JbgGsSRRvHqVNHRjdJOyUlx6DIEBjRx+tblC3no2Xh451WWI0+kpRI0NuXrs851ieeu7dKKaVl1W2UV3XuWTHskVMTBBbt4g1qSsq2rhy1b2REmexlmzavisdjUZFW0s/bx0qoKfbfW+xl4vw6IAZh1hBYSWgzPVzUVSp7aOoUtvnThDfUlSppVmJqtSeGp3b2t2CQ07xE088weDg4My/7bW7jVhffwA6jSOybqChnl54abVMms00yRTq8tbfTKGuckJwS6USSA0XBbdK25wXcEqJCEanUTNgHKOxp99pOwAxwf6E+HkxZTJzo949NxJ/bw/S40Sl7AslDW535NKTwggL8mHKZOZsfu2SOooxUYHkZIpO0/mrtXT3Lq/jl5gYyj2bpx3z8jbyr9WvCMcYIC4+mH0P5eDtY2B4aIwjhwuoLG9bMeNTULhdUOZ6+yiq1PZRVKntcyc4xooqtTSKKvXtj0NOsZ+f30yakJ+fn912t+Gl1RE6nXYlJ1osCAKJAQEA1PbJV9BNCRHTPiqdVKHOCBejvA09fYyMO3fT16rVpEeGAFDY2O6UDSuCIJCXJDp9xQ3tGF1cp2wlLyUGrUZNz+AIVc3uVXAWBIF71yWhVgm0dg5QtcQK0WtXxxATGYDJZObE2XKMy1yrd9WqcDZtSgKgpLSV6wWNK8bxDAzyZv+ja4iODcRssnDxXDXn369cEarZCgq3C8pcvziKKrV9FFVq+yiq1PZRVKkVbiWy1xTfDdxcU3xk4ZpiG2t+zzU3cqWthdTAYB5ISFm0P4hrb+v7+zlUXo6XVssnc3PFhxGp9aDz7IxOTPKbS/lYgI/lrsHfY57CnAPrSl+9VkLbwBCbE2NYFxNle4dF7AyOjvGHswVYgKfuySbI07bqoeR601n2LRYLr54rontwlNzESDamxDpux84a5BtVLVwua8TLoONDO9eg0agl17k6s5a5sKKFK0WN6LRqntiTg5eH3u6aYrnrmWef88TEFG++U8jg0BjhIb7svS9jQYrjUq81rihr5dLlWgCysmJYuyZW/O3eYlVqsYuFkhtNFFytx2KBgEAvtu3KwNfPAdVu2WuBZY5Thn1xTfFjyppiBYUlxPo7/uFfFq4pXmytrqJKbf+4d70q9SxbtvZRVKkVVWoQ1xR/JO2jK3auXwp7KxnZa4qNRiOjozffOjU0NPCjH/2Id955x60Du52I9/MHoGGgX5YScbSvL1qVipHJSTpG5Ckke+q0xPiLNwhnUqgB0iPEKG9pW5fTb7p9PQ3Eh4oRb7dEi5PFaHFRQwdjE+55+5uZGIG3h56RsQlu1LS6xeZsVq+KJDjAi4lJ05KnUet0Gnbdm4ZWo6a9a5DLBQ1Ldiwp0tIiWb9OFPsqKmri2jX3p6Y7iyAIrF4Ty+4HszEYtPT1jvD269doqFu+Os8KCncCylxvH0WV2j6KKrV9FFVq+yiq1Aq3AtlO8aOPPsrvfvc7APr7+9mwYQM/+MEPePTRR/n5z3/u9gHeDoR7eaNTqRkzTdE56vhNQaNSEefvD0BNr/zySqumU6grup0rrZQcEoRWrWLAOEbrgOMlpeaTEyuKZFW2drtcnik+LIAgH08mTSYKG9xzA9GoVWyYLtF0o7qV4VH3TTwgrtHetk58i9/c3k/FEgth+ft5sm2zWHu5rLKNqtqOJT2eLTIyomYc4+KS5hUjvmUlPNKf/Y/nEhruy+SkiTPvlXHpbJWSTq2g4CDKXL84iviWfWw5xhOKYzzDnbDG+OF5a4y7bsM1xneT+JaCfWQ7xdeuXePee+8F4MCBA4SHh9PQ0MDvfvc7fvzjH7t9gLcDapVqRoW6fqBf1r7JgWK6RnVvr+zJNCk4EJUg0DtqpEtmpBlAp1GzKlQU3Cpuc96xigjwIdjHkymzmaJm1xw0MVospnIX1re5bW1xYmQQ4UE+mExmLpTUu8XmbAL8PFmXKTrel27U0z9kdPsxZhMbHcia1TGAKLzV3uledW1HyMiIYuOGRADKylo5f8G9D2yu4uml5/4Hs8nMFq9TZXkbb712je4u518AKbjGz372M+Lj4zEYDGzcuJHLly9L9n355ZcRBGFOMxjmLhM5ePAge/bsISgoCEEQKCgoWGBnbGyMZ599lqCgILy9vXnyySfp6Fj+F0m3G8pc7xiKY2wfRZXaPlbH2P8OcYwPKY7xHBTH+PZCtlM8OjqKj4+ofPzOO+/wxBNPoFKp2LRpEw0Ny5/KuVJI8BNTiOsG5EV8EwICUAsC/WNjdI/Ku1HpNRoSAsXjlnc6Fy1eHSUKblV39TrtgAqCwNr4SACKmtqZMrk2oSaEBxLs68mkycz1WvekOwuCwJasRAQB6tt6aeyQL262GKtTIogI8WXKZObU5Sq3PljYYk1mNPExQZjNFt47W0H/oPsmOkdJS4tkyz2rEASorunk9PsVmFz8/t2JSqUid0MCux/IwtNTx+CAkSNv3qCooHFFOfB3A3/5y1/4whe+wLe+9S2uXbtGTk4Oe/fupbNTOrPC19eXtra2mTZ/jhkZGWHr1q38+7//u6SNz3/+87z55pv87W9/4/Tp07S2tt616slyUOZ6x1FUqe2jqFLbx0en58k7IGIcdps7xooqtYJspzg5OZnXX3+dpqYmjh07xp49ewDo7Oy84xdg2yN+2inuGB1heMLxm6VOrSZ+OoW62okU6rQQMdJb2dUtaz2zlVAfb0J9vDBbLJS2O7/uMiksCB+DDuPkFOUulHkC8QayYZUY3StubGdkzD2TT6CvJ1nT9ZDPFde77LzPRxAEtm9IRq/T0N0/Qn5Jk1vt2zrevRuTCQnyZmJiiuOnyzC66VrJITk5jO3b01CpBBobe3jvZBmTkysrTTkiKoCHnsgjNiEYi8XC9fwG3nm7kKGhpasvvVIZnZxwS5PLf/zHf/DpT3+aZ555hoyMDH7xi1/g6enJb37zG8l9BEEgPDx8poWFhc35/O///u/55je/ye7du23uPzAwwK9//Wv+4z/+g507d5KXl8dvf/tbzp8/z8WLF2Wfw92EMtfLQ1Glto+iSm2fO0GV+uFZqtRLUcf4TlKlHpgYdJttBfch2yn+5je/yf/9v/+X+Ph4Nm7cyObNmwHxTfLatWvdPsBbimCjSeCl1RHm6Q1A/eCsKKRgsdksMNOSg6bLK/X0yJ5E4wL9MWg0jExO0jQwgEWYVn+UajZYHSU+aBa3dWC2zB2bpJ15TaUWyIkXHc7rDa2YsMyMRVIF0479mBB/wvy9MZktXK1tWfS8Zo/Z3vhz02LwMugYGh2noLpFlEWc1Ry1I9W8PPVsXSemFBdWttLc2T/nOojnINhs8/steu0AjUbN7nvT8fE2MDwyzrtnypicMtsenNS1kzqujHHGxgWze1cmGo2KtrZ+3nm3mLHxqel9pI4r2G4q201qPNJt7vH0Hlq27Urnnu2paLVqOjsGefO1a1RUtIvC3FJ2VBJN8ncgc5xS9peIDa/+lIw//4dLbcOrPwVEVcrZbXzc9kPcxMQE+fn5c5xXlUrF7t27uXDhguRYh4eHiYuLIyYmhkcffZSSkhJZ55qfn8/k5OSc46alpREbG2v3uAp32Vxv46YvMXVLTjaCGVIj5qVSF9dhMVvcY98CibZSqU3usY8Fov0XplKbp8wIZhY0e+OUOq7NiPGUScKOYLPZO26IhxdPzl9jPDll95wXNLNguyHhGE9MyLcl0d9Ha1i4xnh8HIsFiSbYbFKTukG9MGLcMzYqaUeqmSWaTqW1scZ4RHL8JrPKZpO6nCpBsyBi3GYcwGRR2Wxyxw8ats5LpW4e7ZXsL9Ukx4OKvIBcNgTmkeqTOuu4CisF2U7xBz7wARobG7l69SpHjx6d2b5r1y5++MMfunVwtxuJ/gEA1Pb3ydpvTgq1Ud6bO7VKxapg0al2NoV6VWgQOrWawbFxGvsGnLIBkB4Vil6rYdA4Tl2n/Kj3bARBYEOKGC0ub+pkcNQ9ET2tRs3mrHgACqpb6R92/9rf+Kgg0hLFFw1nrlQzNr609YQNBi33b09Hr9fQ3TvCqQuVtyQ1OCLCnz33Z4mR8u5hjhy5seIisYIgkLQqjP1PiCJcU5MmLp2t4r2jxYwMu1eA7W4gJiZmTu3al156yWa/7u5uTCbTgkhvWFgY7e22VetTU1P5zW9+w6FDh3jllVcwm83cc889NDc3Ozy+9vZ2dDod/tPZOI4cV0FEmeudQ1Glto+iSm0fRZXaPneCKnWUR6Tt8m1uxl1ZYc5kht2u2CjqtjjWVLbZbNiwwS0Dup1J8AvgQmsTjUP9TJnNaGzU/LOFTq0mzt+f2r4+qnt7CfG0XetNitTQYIraO6jt6WXCZEJnqxagHbRqNWnhIRS2tFPc2kFcoL+s/WfsaNRkRYdxta6F6/WtJIYGuvSHHxXkR3SQH809A1ypamZXTrLTtmYTHxFITKg/TZ39nCus48HN6W6/QW3MjqOta5CBISPvX61h9z2pS3oT9PPxYPe96Rw9WUJTax8X8mu5Z13istx4ZxMS4sMDD2Rz/N0SBofGePvIDXbtTCc4yGdZx7EYPj4e3P9gDuUlLRRcraetpY83DuazbmMiySlhy37dlpPLTz7nljrFEf/wbZqamubY0kvU2XSGzZs3z0QnAe655x7S09P57//+b77zne+47TgK0ihzvXOkRIrLmk4UVVPWJK6Z35YpUcfYCayO8dHCKqo7xAd2yTrGTmB1jA8XV8w4xlJ1jJ3B6hgfKi6fcYwfnnaU3YHVMT5YXjrjGEvWMXYCq2N8oLJkxjGWrGPsBFbH2FrH+EB1ER9IzsJX5577q9UxttYxfrW+iCfjswjQ265jLBerY2ytY/x60w0ei8kh2ODtFvtWx9hax/hIawEPRK4hzODvFvtWx9hax/hk53XuC11LpEeQW+wvFxsO/BSVh2HxjotgNq6s4MZS4p47nAIgpu54a3VMmc00DfXL2nfVdAp1hRMp1OE+3vgZDEyazdQ4WbN4dYQYwanv6WNwzPk/gNWx4ahVAp2DIzT3ur5mYmOqGC2uau2ms989bxtF0a0E1CqBlu4Bqpqdi7DbQ6NRc9/GVeI627Y+iquWXlQhNNiH7dOlmipqO7hWvLRrmqXw8/PkwQeyCQjwYmxskqPHimlodN+bVnehUglkZEWz//FcgkN8mJw0ceFsFcePFDE4uLTq4bcST63OLQ1EIazZTcopDg4ORq1WL1B97ujoWOB0SaHValm7di3V1dUOn2t4eDgTExP09/c7fVwFBWdQVKnto6hS2+dOU6V+fQnWGN/OqtQKKxPFKXYjgiCQ5D9dYqlPvgq1RqViYHyMDpnllQRBICMsBICSdudq5AZ6eRAT4IcFKGpxvlyJp05LxvQa5fw6x9McpQjx8555636+vMFtDxW+Xgbyph3uC8X1GJcgxTnI34uNOfEAXClqoK3L+dR0R4mLDuKePHFN843SZoor3KPeLRdPTz379mYRFRmAyWTm1OlyikuaV1QtYyt+/p7sfXgN6zYkoFaraG8b4I2D+RReb1xRStq3Mzqdjry8PN57772ZbWazmffee29ONNgeJpOJoqIiIiIiHD5uXl4eWq12znErKipobGx0+LgKCs6iqFLbR1Glts+dokp9u9cxXirxraXm8geeo/QjX3C5Xf7Ac7f6VJYNxSl2M8kBYsS3pr9Xlhq0Tq0mMUBck1zZIz9ymR4WggC0DQ3TM+LcTS07SoyclLZ3ujQxrY2PQCUItPYN0drnerR4Q0oMGpWK9r4hajtcW6s8m+ykCIJ8PRmfnOJcUZ3b7M4mPTGMpNhgLBY4cbGKEePSr1tNSw4nL0usmXy5oJ7ymluzdlKn07BzZwapKeLvKv9aAxcuVq9IR9MaNX7kiVwiIv0xmywUXGvgzdeu0d7af6uHd0fwhS98gf/5n//hf//3fykrK+Nzn/scIyMjPPPMMwB87GMf46tf/epM/3/5l3/hnXfeoba2lmvXrvHRj36UhoYGPvWpT8306e3tpaCggNLSUkB0eAsKCmbWC/v5+fHJT36SL3zhC5w8eZL8/HyeeeYZNm/ezKZNm5bx7BVuB1q7B7hS3uRWx1VRpbaPokptnztBlfoRRZX6luCurDBrZtjdgOIU28GWsOxi6sNRPr4YNBrGTFO0DNtxCG3smzItmFXZ24MZi4MKwWLz0uuIDxKd6tLOTieUhiE+yB8/g57xKRMVHd12z1NKHRABvD30pEWJkeurtc327SxyXgjg7aknZ7qU0oWKRibNZrv9HWkWAQS1im1rkxAEqG3toaatx45KsHPjF1QCW/MSCfTzZGx8khMXK5myLBy/XbVnFTabvTFlZUSxOk2sHX3+ai1V9Z1uPDeJcdroK6gFNm5KYsMGsUZ0VXUn77xbIkbm5X5nkqrUUvvIPF+VgLe/J7sezGLrzjQ8PMS6xu8cKeLs6QpGxybnqGTLtS9blfoO48Mf/jDf//73+eY3v8maNWsoKCjg6NGjM+JbjY2NtLXdXGbQ19fHpz/9adLT03nwwQcZHBzk/PnzZGRkzPR54403WLt2Lfv37wfgqaeeYu3atfziF7+Y6fPDH/6Qhx56iCeffJJt27YRHh7OwYMHl+msFW4HBAsYjRMcvVzB9aoWLpU2uk81WlGlXvS4d6Qq9aSiSq2oUjumSq2wchAsDryyfOONNxw2+Mgjj7g0oJXA4OAgfn5+/Nu5Ixi854peWcwSD6uzth+vr6aku5OckHDui0lctL8Vk9nMr67mM24y8URaOjF+fnb7w/SkMk1dbx+HSyowaDR8Yn2ubaEviRe8wvT2G81tvF/dgL+Hgb/bkIMKqfOVsDM9niHjOH84W4DZYuGx9RlE+UkI+0j8+oR59ienTPzpdAGj45NsTIlhbVKU3f7zx2PvuFfLGrle2YJBp+GDO9bgodfasCNhSGr887YPDhs59G4RE5MmMpLC2bw2QcZYbX8gdc7WMVksFi5dq6Osqh1BgB2bU0iICXbYvqPntug4p//b0tLH6dPlTE6a8PLSc9+ONIICZQhvSI5H3vglz3ceE+NTFFytp6JUTEHX6TSsXR9PcmoEKpVg57gyx2MDo3GEz3zxEQYGBtxWE9Z6X3OHTXfaUlg53K1z/Y/+dBwPTy9K6ts5W1wPiJlEG9NjF4pjSUyJkpVVZm2vbO3mRFE1FiA9JtS2+JYL9uu6ejlaWIXZYiE5LMi2+JYL9pv7BjhcXMGU2UxcoL9N8S3JZ3wHjts+NMSh4nImTCYifX14ZHUa2vniW5J2pCaIm//sHh3hYHkpxqkpQr28bItvOXAd5qC6edz+MSMHKksYnpwg0OAhLb7lgC1b/Yem1agHJsbw1eklxbek3qkKktdI3G6cVqPuGR/FS6OTFN+SsiO1XTW9fdw0NSO+pVdpJMW3pMavVtl+6LF2nzKbZsS3tIJaUnxLLfHwtNj4TRbzjPiWWlBJim+pFrEzm9GhUT6W8ZEVO9cvhb2VjENOscpBxUFBEDC5Me3lVjHfKTZbLFgsFtQqlUNOcV1/H4eqy/DSavnU6nW2FScl7JyorqW4q5PMkBB2JyYt2n/235jZYuHly9cZmZhgX0oyKSE2HKBFnOKJKRMvX7jGhMnE/tWpJE5Hnx22M2s8p0prKW3uJCrAl8fyMmzv4KBTDFDR3MXJwhq0ajUf2Z6Dp15nt//88Ugd12Qy89rpQvqGjCREBLI7L2XBd+aqUwzQ2NrH8XPlAGxbn8yquBAHx+qcUyzuauHc5Rqq6joRBIFdW1KJjQp0yL67nWKAgYFRTpwoY3DQiFqt4p7NySQmhNjcz/HxLI1TbKW7c4hLZ6vo7RFTvgKDvFm3KZHwcD/bOyhOscJtyt0611udYmBxx9gFpxIccIxdtL+oY+yi/cUcY1ecYljoGC9QpXbBKQYHHGMXnGJw0DF20ikGxxxjZ51icMwxdtYpBsccY2edYnDMMXbWKQbHHGPFKb59cWgGNJvNDrU7YZKcz/n2Rv675DKVA46v843x9UOnVjMyOUnryJCs46VOp1BX9fYyJXPdjkoQyAgXHYyidufEsnQaNZmRoQBcb3JNpCkvIQqVINDSN0hT74BLtkBcmxXi58WkycTFikaX7VlRq1Vsz01GEATq2nqpbnG/GjVAbGQAa9LFCPe5/Bo6e+T9NpxBEATuWZ9EQmwwFouFE+craGxx37psufj5ebJ/fw5RUaIA1/tnK7l8pXZFrjO2EhziwwOPrmX95iS0OjW9PcO881Yhp94rZegOVqlWuPu4m+d6K5nx4WxdHQ9AYU0bl8oa3bvGWFGltouiSm0fRZXaPooqtYIruJTMPuZC6Z7bBYvFwphpipoBxx0JjUo1o0Jd2SfPwYry8cVHp2PCZKJWpoI1wOrwUASgZXDIacGtnOhpoayBIdoGnHfcfDz0ZMaI6wUvVrv+YCEIAlsz4gGobOmmvc99TmWIvze5qdEAnC2qY2h0aQSxcjNjiI0IwGS28O6FCkaW6DizUakEtm1aRUJMEGaz6Bg3NN+6m7hVgCtrtXi9y8rbOHa8mOGRpb8WzqJSCaRlRvHoB9eTkhaBIEBjfQ+HXs3n6uVaJsbd91CloLDSuBvm+tlkxoezNSsemHaMS93vGCuq1NIoqtT2UVSp7aOoUis4i2yn2GQy8Z3vfIeoqCi8vb2pra0F4Bvf+Aa//vWv3T7AW02Sn+jc1g/1yYrcpkyrUFf19chSoRYEgbRgMdpb2iU/Yumt15MYJI650MlosbdeR1qYmHqd74ZosUatonNwhNquPpdsAYT5+5AWLV6f90vqZF3bxVizKoqwAG8mp0ycul7lVttWBEFg+4ZVBPh6YByb5Pj5ClFQZIlRqQS2b0ohITZ4xjGuaeha8uPaG0/u2jju25GGVqumq2uINw8X0NR866LYjuDhoWPj1lU89EQekVEBmM0WSotaeO1vVygtblnREW8FBTncbXP9fDJmO8a1SxAxVlSp7aKoUttHUaW2j6JKreAMsp3i7373u7z88st873vfQ6e7uU5i9erV/OpXv3Lr4G45AoR5euOl1TFpNtM0MmBHWdYyp8X6+aFXqxmdmqRlZNDOfgtb+vRa4MaBfvEmasP+TenGhWRFiNHZ8q4uxk0mp1Sa18aJysV1PX10j4zaVJl2pHkatOTEicrRl2qaMGGZOx47195mAzamxKLTqOkZGqWkocNuf0kFZRuoVAI7clehVato6x2isKZVcv/Frqe9fXRaNfdvScOg19DTP8KpS6IDLlvtWaYqtaAWI8ZJ8SFYLHD6YhUVtR2S9lFhs7lDldoiACqB2LhgHn54LUFB3kxMTHHiZBlX8+vFFxIOq2QLNpvbVKBt2PAP9GLXg1ns3LcaP39PxsenuHqpltcPXKW2pmv6b2T+9ZRodn7rCgq3irtqrrchaStYIDNuXiq1oko9pymq1EgrSZsFRZVaUaV2qCmsHGQ7xb/73e/45S9/yd/93d+hniV+kJOTQ3l5uVsHtxIQBIEkXzHyWjPg+FsgtUpFcsB0CnWvvIhvgIcH4d7eWIDybvnR4mh/X/w9DEyazFR0ORcNDPD0IDFEHL+ra4vXxEeg12roGzFS2eZ6dNJDr2VDSgwAlyubGBlz39tXXy8Dm7MSALha3kT3wIjbbM/Gx8vA7nvSUKsEGtv6uHyjfkmOMx+VSuDejcmkJosvTs5dqaGk0rXv11V8fAw88EA26enii5iS0haOHi1ieHjlp2xGxQTy0JN5bLp3FR6eOkaGxzl7uoLDr1+ntbnvVg/PJqOTE25pCnc2d9tcL8WcNcZLETGevcZ4KSLGd9ga4yWJGM93jG+3iPFsx7hmaVOpX2tY2lTqQ00FS7rG+Fjb9SVdY3y669pdHTF+6aWXWL9+PT4+PoSGhvLYY49RUVExp8/Y2BjPPvssQUFBeHt78+STT9LRYT+z9emnn0YQhDlt3759c/r09vbyd3/3d/j6+uLv788nP/lJhofl/ZZkO8UtLS0kJycv2G42m5mcnJRr7rYg2c/qFPfKmqxSAsWIb3W/vBRqgMwQawp1l+wJUhAEsqejxYVtHU5PsHmxopNS2dnD0JjzN1m9VkNuvGjrck0zU25IMc2IDSN0WnTrXFm9y/ZmkxITQnx4IGaLhRP5lUuW3hwW5MO29eLfUkl1OyXVbYvs4R4EQWBzXuJMHeNL1+spKGly64OYXNRqFRs2JLLDmk7dLaZTNzQsjeiZO1GpBFalRfDYh9ezZn08Wq2avt4R3j1WzDtvF9LRPnCrhziHDa/8nIzf/qdLbcMrP7/Vp6GwxNyNc70UiviWfW6FYzyhOMYzLHCM74A1xor4luu46wW4nJfgp0+f5tlnn+XixYscP36cyclJ9uzZw8jIzQDT5z//ed58803+9re/cfr0aVpbW3niiScWtb1v3z7a2tpm2p/+9Kc5n//d3/0dJSUlHD9+nMOHD3PmzBk+85nPOH7BAM3iXeaSkZHB+++/T1xc3JztBw4cYO3atXLN3RZEe/uhV4mp0G2jQ0R6OSZJHuPrh4dGg3FqisbBfuL9Ahw+5qqgIE43NNBrNNI2PEykl4+sMaeFhnChvoneUSPNA4PE+PvJ2h8gzNebaH9fmvsHudbUyvZVCbJtWMmKCaewsZ3hsQmKmtpZO+0kO4tKENi2OpFXzxdR295LXUcvCWGBi+/oAIIgcO+aRLpODTMwPMb7N2rYuTbZdmktF0mMCWZoZJyrxY1cLKjH06AjIXph3Tt3IwgC63Li0GjUFBQ3ca24CePYJJtybdTNXEbi4oIJDPDm9JlyenqGOXW6nMTEEDZuSEKnVS9u4Bai0ajJWhNLSko4RQWNVJS10d42QPtbhYRH+JGzNo6wCPl/hwoKt4K7ca63R2Z8OABni+sprBFfYNqsY+wkKZHiS/QTRdWUNXUC2K5j7CSiY5zC0cJKqjvEB3abdYydxOoYHy6umHGMbdUxdharY3youHxGldpmHWMnsTrGr5aXzjjGNusYO4nVMT5QWTLjGEvWMXYCq2P8ak0R/RNjHKgukqxj7AxWx9harunV+iLJOsbOYHWMreWaXm+6IVnH2BmsjrG1XNOR1gLJOsbOYHWMreWaTnZel6xjvFxseOXnqDwMLtsxG8WsvcHBwTnb9Xo9ev3c39fRo0fn/P/LL79MaGgo+fn5bNu2jYGBAX7961/zxz/+kZ07dwLw29/+lvT0dC5evMimTZskx6HX6wkPD7f5WVlZGUePHuXKlSusW7cOgJ/85Cc8+OCDfP/73ycy0jGfQ/Zf+ze/+U0+/vGP09LSgtls5uDBg1RUVPC73/2Ow4cPyzV3W6BWqUj0C6Ssr4uqgW6HnWKVIJASEMyNrnbKe7tlOcV6jYaUoCBKu7oo6uggMlGeU6zXaEgLC6GorYMbre1OOcUA6+KiaO4fpKStk7zYKLz1zt3ANWoVG5OjOVFSS35dC2mRIXjotE7ZshLs68WahEiu17ZyprSOyEBf9Fr3TGAGnZadeas4fL6UmpYewgN9Zh6K3E12aiTDo+OU13Zw6nIVBp2GiNCld54EQWDt6hgMWg0Xr9dRVt2OcWySbZtWoVG750HGGXx8DDywL5sbhY0UFzdTW9tFe/sAWzavIjLS/5aNy1EMBi3rNyWRnhlF0Y0maqo6ROe4rZCwCD9y1sYSHuF/y8Z3+aOfc0ud4ojnv+mmESmsRO7GuX4xMuPDQYCzRdOOsQU2ZrjXMbYIcLJQdIwtFti+2p2OcQB7s1dxrLBq6RzjrFQOFy2PY/zGUjjG6RkcLLt9HeMnk7Jm6hjfCY7xozE5hNxmjvHZrus0rxDH2J3ExMTM+f9vfetbvPDCC3b3GRgQM+UCA8WgVX5+PpOTk+zevXumT1paGrGxsVy4cMGuU3zq1ClCQ0MJCAhg586dvPjiiwQFidf2woUL+Pv7zzjEALt370alUnHp0iUef/xxh85RsDiRo/P+++/zL//yL9y4cYPh4WFyc3P55je/yZ49e+SaWpFYC1X/2/kjGLy9AHE98Rt15XhrdXwybf3CSUoiU6htcJi/VBShVan4TPb6mzdvs+1JaHZN8fahIf5SUoJaEPjU2jybN2ZBwg4W6DMaeSX/BgB/n5eDv4cHEjXLJbdbTBZeu1FK68AQ2VHhbEuOF/tL/GqktmMGs8XCgUtFdA+NsjomjG1pCdLjsWNnNlMmMwfOFdI/MkZaVAj3ZSXNOwEJ+1KZXfP6F9W0crGkAZUg8MiWTEID5hWZlxqnnb8qW8c2WyycvFhJfUsvWo2a/TsyCfL3WuQYtj9w9Nxm+lugtrGbM5eqMJsthAb7sHurKARmE7nfmcSBHbl2nV2DnDtbyeCQ+KYyNTWcvLViivIi5hGkbm1y73hyly/MS0scHhqjuLCJ6sqOmZTFsHBfVufEEhnlb1NTy2gc4TNfeISBgQGXHVgr1vuaO2y605bCyuVumet/9MfjeHh6zf1QamoVoLS+nbNF9QBkJ0WIEWMJx1JSR0fKvkosOXiysBoLkB4TKkaM3WVfgLrOXo4VigKPyWFBomOsdo99BGjuG+BwUQVTZjNxgf48uDoFlUbCMbYzTqn+7YNDHJpOoY709REdY41tx9gi+bAifdzu0REOlokp1KFeXqJjLPXC3d77BJXtY/ePGTlQWcLw5ASBBo+bjrGULQk7Uv2HJsc4UF3MwMQYvjr9jGMs9W5FkPlAN2aa4GC96Bh7aXQzjrGUHantKontE+ZJ3mgupHNsCL1KMxMxlhq/WmX7oUTqcpotUxxtu0HbWB9aQT3jGKslHp7kjh+mON15g2ZjF2pBNeMY2+o/OjTK05lPLclc39bd5Rabg4ODRASH0NTUNMeerUjxbMxmM4888gj9/f2cPXsWgD/+8Y8888wzjI/PTe/fsGED9913H//+7/9u09af//xnPD09SUhIoKamhq997Wt4e3tz4cIF1Go1//qv/8r//u//Lli/HBoayre//W0+97nPOXSuTr2+u/feezl+/DidnZ2Mjo5y9uzZO2aSnMMsKcQ4X3+0KhXDkxO0G4dsSCXaJtzLGz+9gUmzeW6tYwnJxdnif6E+3gR7emKyWCjr6bKjpmujIQp2xQf4A1DQ2j59XHlNUAmsjxPryZa0dTAyMWG3v5QKIIK49nJLqpiKV9LUQc/wqLQdiTa/n0ajYvu0I1ze0kVjT79L6tDz+61OiiA+Qlxf/G5+pahM6YiyssxjCyqxVFN4sA+TUyaOvV/GwJARsDdWwWaTq0ptESAhLpg92zPQadV0dg9x+L0iBobHbas6L7Eq9ewWEurL/ofXkpoqKphXVLRz+K0COruHZtmSOC+VIKu5S616vl0vPw823pvCYx9eT0p6BCqVQEf7IO8dK+atNwqob+jBLAhYVKo5TUHhVnO3zPVy1ZIVVWpp+4oq9aymqFIrqtQOqFIvFZ5andsagK+v75xmzyEGePbZZykuLubPf/6zy+fy1FNP8cgjj5CVlcVjjz3G4cOHuXLlCqdOnXLZ9myUJy8H0UynUANUDTgu/iMIAmnTglvlPfKUlwVBYHVYKADFHZ1OCW+sjRKdibKOLoxOiqNEB/gS7uuNyWzhWqNrSsVRgX4khgZiAc5V1LtFTCQi0IfVcWJq8+niWrcKYwmCwPa1Sfh66hk2TnDyWvWSCVJp1Cp235NGoJ8nxvFJjrxfysio+8QyFiMizI/9u7Pw9tQzODTG4XcLRefzFqPVqtm4KYn7d2fi6aljcGiMI0cKuXK1lqllqPHsLry8DWzcuorHPryB9NVRqDUqeruHOfNeGW8cuEpNZTtmN4rTKCgoLB2KKrV9FFVq+yiq1PZRVKlvb5577jkOHz7MyZMniY6OntkeHh7OxMQE/f39c/p3dHRIrhe2RWJiIsHBwVRXV8/Y7ezsnNNnamqK3t5eWXYdcooDAgIIDAx0qN3JrPIXc9er+ntkTU5pgaKSdMNgv+ybXmpwMBqVit4xI61D8h2UKD9fQry8mDKbKWqzL3kuhSAIbIgXf9TFrR0Mu6BEDbA5JRa1SqC5d5C6rj6XbFnZmBKDj4fouF6qbHSLTSs6rYbd61NRqwSaOvu5XtXiVvuz0es07Ls3HV9vAyOjExx5vwyjG0tOLYa/nyf7788iKMCL8fEpjp4sob5pZShAR0YG8MjDuSQliS+KSktbeePN67S19d/agcnEy1vPus1JPPGRjWStjUWn0zA4YOTC+5WMDC/fSxAFhfkoc708FFVq+yiq1PZRVKnto6hS335YLBaee+45XnvtNU6cOEFCQsKcz/Py8tBqtbz33nsz2yoqKmhsbGTz5s0OH6e5uZmenh4iIsTA3+bNm+nv7yc/P3+mz4kTJzCbzWzcuNFhuw45xT/60Y/44Q9/yA9/+EO+/vWvA7B3715eeOEFXnjhBfbu3QvAN77xDYcPfDsSP51CPTQ5Tvuo43+YAQYPwjzFusMVMmsW6zUaUoJFZ7ywU75TKwgCudHij+ZGa7vTb1JjAvyI9PPBZLFwucE1p9DP00BOnDims5X1bnm7q9Wo2bZa/OMrbuiguXvAZZuzCfbzYkt2IiDWL27scI8zbwsPg44HtmXg5aFjYMjIkTOlGMeXrwSKp4eOB3auJjoyAJPJzMlzlVwvcu/DnrPo9Rq2bklh164MPD11DA2N8c67JZw9V8nY2O1VJsZg0LJmXTyPP7WB3PUJpGZG4ePrcauHpXAXo8z18rkzHOOUO8YxfnOJIsaG29wx9lccY5sojrF7efbZZ3nllVf44x//iI+PD+3t7bS3t2M0issB/fz8+OQnP8kXvvAFTp48SX5+Ps888wybN2+eI7KVlpbGa6+9BsDw8DD/z//z/3Dx4kXq6+t57733ePTRR0lOTp6Zk9LT09m3bx+f/vSnuXz5MufOneO5557jqaeeclh5GpAvtPXkk09y33338dxzz83Z/tOf/pR3332X119/XY65FcmM0NaFt2eEtqy8XVdJRX83a4Ij2BGVePMDk8S6gGkhrMKudk401hJo8ODvM9YgSEkA2BDO6hwZ4c+FRagEgWfWrMVbd1OpUFJoa9acY7ZY+H1+AYNj42xLiGNNZMSC7pLiTLO2tw0M8er1EgTg/6zPIdDTxgO8gyJMk1Mm/nT+BsNjE+TFR7IpOXZuB4lf5WIiUmeKaylt6sRLr+ODW7Pw0EooXMsUi7Juf/9GLWUNHWg1ah7duppAH9tOjJ1l5g4fe2DIyNunSxgdmyTQz5MHtmVg0Gsl+8/gJgEus9nC1Rv1lFSI5UdiIgPYtmkVeqnSSFLnJXFYSQGrRb4DKxMTU1y/3kD59Pj0eg15ufEkJ4XaV2uVtC9vPNL2Ze5go7vROMI/PP+QIrSlcMu4m+b6/3xlodCWRSpkIHFrKWmYJb6VGHFTlVqiv1wBq4q2m+JbadGhN1Wp3WS/tsuG+JbKffab+xeKb6lVKtnXWeq47UM2xLfUajt2pCYC25u7jDbEtzQaOxOctC1bwlmS4lsy7Uj1H5pOoZ4vvgW4RYDLOJ1CPV98y54dOQJW46apGVVqvUozR5XaHQJcU2bTjCr1bPEtwC0CXCaLeUaVerb41ujQKJ9Y/eEVO9fLtSf17PXb3/6Wp59+GoCxsTG++MUv8qc//Ynx8XH27t3Lf/3Xf81JcxYEYWYfo9HIY489xvXr1+nv7ycyMpI9e/bwne98h7CwsJl9ent7ee6553jzzTdRqVQ8+eST/PjHP8bb23H1ctlrio8dO8a+ffsWbN+3bx/vvvuuXHO3Han+Yip0VX83ZhkPv6mBwWgEMQ26bUTeW65QLy8ivH0wWywUOxEtVgkCedHim5JrLW1Ov6WN8PMhISgAC3ChzrUUZa1Gzda0eACuN7TRN2J0yZ6VzWlx+HkZGBmfEMVJ3BzdvCcrnoggX1EM63L5kkZw/Xw8eGB7Jh56Lb0Doxx5v5TxieWLhqpUAhvWJrB1Y7KYOt7ax5vHC+kfdN9bYFfQ6TRs3JjEg/uyCQjwZHx8ivMXqjlyrIieXve9SVZQuBu52+d6uWTEh7M1Kx5YojXGUcHcly1GjMuXYo1xaCB7lzqVOmsJI8a+y7DGOP02jxgnr17SVOon4pc2YvxIdPasNcbujxjvi8hZ0ojx9tAcoqcjxq3GlbEszd1YLBabzeoQAxgMBn72s5/R29vLyMgIBw8eXLDud/Y+Hh4eHDt2jM7OTiYmJqivr+eXv/zlHIcYxLJPf/zjHxkaGmJgYIDf/OY3shxicMIpDgoK4tChQwu2Hzp0aKZe1J2CLSHaeB9/9GoNI1OTtI4M3BR7llSuFZteo2FVoHh9Sno6Fu0/v+WEi19+UWcnJot5ZrtFsNhs8/dPDwvBS6dleGKC8q5uh5R/bakob06KQQBqu/toHRiyqTLtaEsIDSA22B+zxcKZijrMWCSP66hqtFarZveaZFSCQG1HL+UtXbLUoRe7Hiq1it0bUvD1MjA0Os47VysxmS3IUVaWc2w/Xw8e2C5GiHv7RzlypoyxaUfcbarUUm16XKsSQ3lwdxZe0yJXbx4vpr6ld8lUqeWqVYeE+fLQQ2tYty4ejUZNV9cQb719g0uXa5mYNNmwJXX9FyptW2yoSbtbrVrSjoLCLeRumuttSdraUkpWVKmdt6+oUs9qiir1Xa1KvTEwkzX+aTN2FFYOsr+Nb3/723z5y1/m4Ycf5sUXX+TFF1/k4Ycf5itf+Qrf/va3l2KMKwq1SsUqP/GBoLxP3pue1cGiY1vZ2y1bDCI5MBBPrZbRyUmqe3sX32EeapWKtVFitDi/uVVWlHs2gV6epIWL0fILda69CRcEgXtT42dEt6o73LPOIsTPm/UpYpHxs2X19LspCm3FoNOyd2MaOo2ajt4hztyoXdL1tgG+N1One/pHePtMKcZlXj8bHOTNw3uzCQ8Vo+QnzlZwbYWsMwZQqVRkZkbz2GO5xMcHY7FAeUUbr72eT3V1x4oZp4LC7cLdPtc7i6JKbR9Fldo+iiq1fe4EVepknxj7S7wUbhmyneKnn36ac+fO4evry8GDBzl48CC+vr6cPXt2TnjcXQwNDfH8888TFxeHh4cH99xzD1euXLG7z6lTp8jNzUWv15OcnMzLL7/s1jGl+osllqoGemTdzCO9fQiYrllcKdOhVqtUZE2nCtxob5e1r5XMiFAMGg39Y2NUdzvvgG5IiEYtCLQODFHf2++0HRBFt3LjowA4V9HAhJsmlzWJEUQG+TJlMvPejWq3TroA/j4e7FqfgiBAVXMXBdWulapajEA/T/Zvz8DDoKVvYJS3T5cwYlw+VWoQBcD23pdBRsq0cFtJM8fPLK869mJ4eenZvj2NPXtW4+fnwdjYJOfOV3HkSCGdXYO3engKCrcNyz3Xw8qc753hzhDfurMcY0WV+iaKKrV9llp8S2Hl4lTcfuPGjfzhD3/g2rVrXLt2jT/84Q+yJK/l8KlPfYrjx4/z+9//nqKiIvbs2cPu3btpabGtgFxXV8f+/fu57777KCgo4Pnnn+dTn/oUx44dc9uYor398NJoGTdNUTfU5/B+giCQOR0tLuqSvzY4KzQUlSDQNjxM+7D8G4BOrSYnUszbv9LU4vQk6mPQkx0l2jlf2+jyZLY2PhI/DwOjE5Ocd1M5JUEQ2JmdhF6rpmtghEsVTW6xO5voUH/umVa8vlLWSHWzvDrUcvH39WT/9kw8PXT0Dxk5fLKYweGxJT3mfFQqFRtzE9i2aRVqtYqWtn4OHb1Ba3v/so5jMSIi/Hn4obXk5caj0ajo6h7iyJFCTp0qY3DQvZkDCgp3Kss518PKnO+d5c5wjBVVaikUVWr7KI6xwu2IbPVpAJPJxOuvv05ZWRkAmZmZPPLII6jVEqq0TmI0GvHx8eHQoUPs379/ZnteXh4PPPAAL7744oJ9vvzlL/PWW29RXFw8s+2pp56iv7+fo0ePOnRcq9Lav9tQn7ZMqz2faa0nv6uFJN9AHklIn9m+gHnbRycn+FVhPmaLhY+kZRPm5W23vxWr+N071dWUdXeTHBjI/pQUacVfCTvjk1O8fOU6EyYT+9JWsWq63JOUMrHU9vGJKX5/qYCxqSm2JcfPOMkyxApFpu239A5wKF/8PT2Sm05MoJ/d/o7ar2/v5ei1SgD25qaQECbW15SrxGzvvC4U11NU24ZKEHhwczqRwX7YUyB29dhDI2McOVPK0Mg4HgYt+7amE+jvJdn/pn3bH0j3t7+9t3+EU+crGZh2MrPTo8hdHY1KpbLZ3/XjSoxforvVzujoOAUFjVTXdGCxiC9M0lLDyc6KwWDQLujvKEupVm00jvAP/7+lUZ9u6+52i/p0RHCwoj59h7Nccz3cmvl+Rn369+8sUJ+WkrRVVKkVVerZSKpS29lHUaVWVKmtjA6N8umsD67Yud5q726Z7zVyd6iurmb//v00NzeTmpoKwEsvvURMTAxvvfUWSUlJbhvc1NQUJpMJg8EwZ7uHhwdnz561uc+FCxfYvXv3nG179+7l+eeflzzO+Pg44+M335ANDi6eZpkRGEp+Vwt1g32MTk3iodItug+Ap1bHqoAgKnq7KepuJ8wr2aH9rORGRlLW3U11by/9Y2P46wyL7zQLvUbDmqgILjc2c7mxmeSgQKfWNug1GjbER3Omup7L9c2khAZj0Mr+Oc0QFejH6ugwips7OFlay1ObstFpXH/wig8LJDs+gsL6Nk4W1hC0xRNfT3nXbDE2ZcYxMjZBbWsPxy5X8PCWTIJ9Pd16jNn4eBl46L7VHHu/jN6BUd46XcKeLemEBfss2TFtEejvxSN7srl0rZ7K2g4Ky1po7xxg++ZV+Hi59xq7gqennnvuWUV6eiT51+ppaemjrLyN6ppOslZHk54WgcYNv7XbhU2/+m9UBte+H/PY8mYoKCw/yznXw/LM94vN9Z29w7T3DJK9yvG6louRES++MD5bVE9hbRsIsDE91m1rClOixOVcJwurKW/uRBBgW2aC2+xbVamPFVbN6H7sXp2MSu0e+1ZV6sNFFTMR4wdXpyx8ueokVlXqQ8XlM6nUj6xOQ+ume75VlfpgWelMxHiOY+wi1ojxgcqSmYjxHMfYRayq1AeqixmYGONAddEcx9hVrKrUB+tFx/jV+qI5jrGrWFWp32gupHNsiENNN3gsJodggzzVYSmsqtRH227QNtbHkdaCOY7xSmbTr3/h8lwPd9d8L/uu88///M8kJibS1NQ0k1LV2NhIQkIC//zP/+zWwfn4+LB582a+853v0Nraislk4pVXXuHChQu0tbXZ3Ke9vX2BTHdYWBiDg4MzxaPn89JLL+Hn5zfTYmJiFh1bsMGTMA9vzFgo75OXNpsdIk6S5b3djJvkpdsEe3oS7+8PwDWJa7AYayLD0anV9I4aqXJhbfHqyDACPD0Ym5riSkOz03asbF4Vi49Bx9DYOBer3ZNGDbAxNYZQPy8mpkwcv17FlMm964sFQWDH2uSZUk1HLpQxMLy0KbqeBh0Pbs8kNMiHiUkTR86U0NAqX4DNVTQaNVs2JLHjnhS0WjWdPcMcOlZIXdPKK04fEODF7l2Z7NmdSWCAF5OTJq5db+Dg6/mUV7RhcvPvQkHhdmY553pYnvne3lw/OjbB0fNlXC5u5Hq56/PZbOaUa1qKVOpZ5ZqWJJVaKddkF6Vck308l7lc01KkUi9luSaFlYPs9GkvLy8uXrxIVlbWnO03btxgy5YtDDux1tUeNTU1fOITn+DMmTOo1Wpyc3NJSUkhPz9/JqVrNikpKTzzzDN89atfndn29ttvs3//fkZHR/Hw8Fiwj623xzExMfz7xbdspE/ffI9wo7uNE821hHh48Xer1tg+ARtpzBaLhVeKb9AzZmRHTAJrQiPs9he33/xn88AAB8vKUAsCn1ibi6dWu6C7VPq01c7lxmYuNTbj72Hg73JzUEvk4Uim+k5vb+zt543CcgTgI+uzCfKUePsnlW0zz35zzwBvTKdRP5qXTtS8NGrJNO9FUnGHjOMcOFvE+OQU6TGh7MhMlDdOB1J9JyanOHyulJ6BEXw89TyydTVeBhtvcyUeJuSmEwsWmJwycfJSFU1tfQjAPbmJpCWGSfa3bd894xkaGuP0hUq6esR7QFJ8CJtyE9BLZhBIGJL6jiWsSKaq28kgs1gs1NZ2UVDQwPCw+Lfv7W1gzZpYEuJDxPRAJ+1LISfd2mgc4R/+ef+KTam6m9Kp7laWe66HpZ/vpeb6H/9OTJ8uqGzhSqmoP5GXHs3atOiFg3Qhrbqkvp2zxfUAZCdF2I4Yu5CWXNnazYkiMZU6PSbUdsTYBft1Xb0ctZVK7Sb7zX0DHC5emEo9x44LadWSqdQO2Vk8Jbl7dISD5aUYbaVSO3CMxdKh7aZSy7Aj1d9eKvWc7k6m+CXsuAABAABJREFUVdtLpXbEzmJpyfNTqaUixs6mVdtLpZ5jR0ZatXFolM9kf2DFzvVWe3fLfC87UqzX6xkaGlqwfXh4GJ3OPekcs0lKSuL06dMMDw/T1NTE5cuXmZycJDHRtlMTHh5OR8dcEauOjg58fX1tOsQgnpOvr++c5gip/sGoBYEu4widRscfEARBmIkWF3a1y36jG+XrS5iXFyaLxWkl6jWREaIStXGM8g7nBaJiA/1JCArAApypqnf57XR0kB8Z0aEAnCytFWsMugEfDz2714ip6mVNnZQ1d7rF7mx0Wg37NqXh66VnaHScIxfKGJtY2tJJWo2a3ZtTSYkPxQKcu1bL1eJbUyrJx9vAg7tWk5MRjSBATX0Xh47eoK1jYNnHshiCIJCUFMpjj+WxcWMiBoOW4eExzp6t5PDh6zQ0dN+xZZw8tVq3NIU7m+We62Hp5/vF5vo1KVGszxCjx/llzW6PGN8Z4luKKrUUiiq1fRTxreXFXXP93TTfy3aKH3roIT7zmc9w6dIlLBYLFouFixcv8tnPfpZHHnlkKcYIiG+tIyIi6Ovr49ixYzz66KM2+23evJn33ntvzrbjx4+zefNmt4/JoNGS5CcKN5X0ynOy0oJC0KpU9I4ZaR6SVypGEARyI8U1Tzc62hl34qar06jJixZtXGpsdimleGtyHGpBoLl/kKou19Nm70mJxdugY9A4zrnKBpftWYkJ8Wd9ivjm/0xJHe19Cx/4XMXToOOBzRl46LX0Do1y5GIZE5PumxRtoVIJbM1LZG26eG43yls4eamKKTdO9o6PRUVudiwP7FqNj5ee4dFxjpwq4Xx+LZOTyz+exVCrVaSlRfLEE+vIXRuHVqumr3+UU6fLOfTGdWprO9360KegcLtwq+Z6uLXzveIY20dRpbbPneIYK6rUtrndHGMFech2in/84x+TlJTE5s2bMRgMGAwGtmzZQnJyMv/5n//p9gEeO3aMo0ePUldXx/Hjx7nvvvtIS0vjmWeeAeCrX/0qH/vYx2b6f/azn6W2tpYvfelLlJeX81//9V/89a9/5fOf/7zbxwawOkiM+Jb1dTJpdvzGqldryAgSI6LXO+WvDU4KDCTAYGDcZKKwQ355J4DsyHB89DqGJyYocHJ9MoCfh4F1cWKt4fdrGpxy0mej02jYmSmKuJS2dFLf5XjZq8XITYoiISwAs8XC0euVDBvdd7O34utlYP/mDPQ6DV39Ixy9VO62iLcUgiCQmxnDveuSEASBuuYe3j5destqCIcF+/LovjWkJoup3OXV7bx2rGBFRo0BtFo1WVkxPPnEerKzY9Bq1QwMjPL+2Upef+MaVdUdyppjhbuK5Z7rYeXM98viGM9eY1zqfsd4SdcYhwTc3muMbUSMlXJNN/HR6XnyDosYdymOsYIDyHaK/f39OXToEBUVFRw4cIADBw5QUVHBa6+9hp+fRBkdFxgYGODZZ58lLS2Nj33sY2zdupVjx46hnQ7nt7W10dh4U5QpISGBt956i+PHj5OTk8MPfvADfvWrX7F37163jw0g1tsPX52eCbOJqgF5UdKcUNGhrh3oZWBcnrqbShBYFyU6otfb25y6oWtUKjbFiRP/1eZWjJPOp/rmxkbiP11r+EKd6zWBowP9yIkV11qfLK1l1E1pyIIgsDMnmUBvT4wTkxy9XunWydBKoK8n+zelo9Ooae8d4tjlpXeMAVLiQ9l3bzo6rZqu3mEOvVdEd9/Ikh/XFlqtmnvWJbFvRwZenjqGR8So8dnL1YxPLG303Fn0eg1r18TxgSfXs3ZtHHq9hqGhMc5fqOa1Q9cor2hjahm+RwWFW81yz/Wwsub7nNSldYzniG/VKuJb81HEt+xzu4tveSyz+NYhRXxLwQGcqlN8pzNTp3gRoS0rlzuaONfWSKSnDx9Kzp774SL1i1+rKqVhsJ81oRHsiElwSGhrZpPFwu8LChgYH+fe2DhyI24Kdi0mtDVzPhYLfy4oontklDUR4WxLjJ/z+WJCW7Np7hvg9RuiGMqH1q4mzHeWwIGDQluz+0+ZzBy4XETvsJG4YH8eXJOKSkKtQ7Yo1PAYr14oZmxyiuTwIHbnJIuCJG6qqWvt39E7xNsXy5icMhEZ7MveDWloJUpNOHsMW/QPGXn3XDkDw2Oo1Sq2rUsiKTpYwr576xfbsjM5aeLKjXrKa8SsBg+Dlk25CcRH2ygJtgxCW472n5w0UVnZRklpC0aj+GJGr9eQnhZBakoEBr3ja21WitCWO2y605aCwq3C+ju2Cm3NxjrV3KiYJ76Vbrs6hbMCUKX1s+oYJ82qYyzR32H70+OpbLlZxzg9JpRtq22Xa5Jtf3p7Xee8OsZZNsS3nLBv3d7cN7eO8QPZC8W3HBmnVP/2wXniW1k2xLeQX7/YeuzuURt1jKXEJ+UKZCEhviW15l9G/WJxu0UU35ou12QV3/LT2y7XJKd+MYBKgNGpiZlyTVbxrUCDbf0fOfV/rf3HTVMz5Zqs4lshHrbLNcmpX2w97pTZNFOuySq+Felhe05cbqEtd9m8m+Z72U6xxWLhwIEDnDx5ks7OTszz3twdPHjQrQO8Fcw4xZfewmOBU7zwz2N4coL/Kb6KBfj71LUEGWap6S3iFDcM9PNaVSlalYpPZq/DoJK4WUrYKWnv5L26Wry0Wp5esxaNdbKQdEIX2mns6+dQcTkqQeCjuTn4eRhm9ZeyY3v78dJqKjq6CfH25IO5WTOToz3n16b96e09Q6McuFSEyWxha2ocOTERtneQ6ziaobVnkMNXyjBbLKxfFUNecpT7HNNZ2zt6hzhyQXSMI4J92bdBokailHMq8+W4dUzjE1OculRFc3s/ADlpUeRlxix4IJKrSi15XAe+446uQc5dqWFgUCyXEhsVyKa8BLw8b06ysscj+6WBfCfaZDJTVdVBSUkLw8NiVodGo2JVchgZGVF4e8+qBSj3PaOEU/zZ5x5csRPl3TRJ3q3cTXP9T/5X2ikGFFXqRewrqtT2j62oUiuq1GBbldo4NMI/5ChO8UpBdvr0888/z9///d9TV1eHt7f3nJp/S5VStdLx1upI9BUFt4p65KlBx/r6EeThyaTZTHGX/LXB6cHBeOt0jExOUtLlnKJybIA/sf5+mC0Wzje4Vh94S1Iseo2aruFRilqdU8aeTZCPJ/ekxAFwvrKRrkH3pQJHBvmyJSMegCtVTVS3dbvN9mzCAn14YHM6Wo2atu7BZVljDKDXabh/axpZqdOibOUtHD9fcUtTl8NCfHlkbw45mdGoVAKNLb28dqSA0sq2FS1oJQpyRfD443ls25ZKYKAXU1NmysrbOPjaVc6cqaCza/COVaxWuPtQ5vqb3GniW6eLFVXq2Siq1PZRVKnto6wxvnOQHSkODAzklVde4cEHH1yqMd1y5EaKAWr6+zlUV4pereZTGevRqqbfMi4SKQYo6e7geH0NXlodz2Tm3oz2SvSfjWCGG+3tnGqox1ur4+Nr1oj7y4gUA3QNjfDngiIAnszOJNLXZ8a+1HFtYoHi1g5OVdahVan4yPpsfD0MTkeKQYxYHCmopL6rD18PPR/amIV+/ltWJyLFVs6V1VNU345KEHhoXRpRQTYe+NyQVj07YhwW4M2+Telza/i6OVI8m+qGLs5ercFktuDjpWfX5lSC/L0WOQf3R4pn09c/wrkrNTN1jQP8PNmYm0BkqMQD9y2MFC8cioW21n6KS1poa+uf2R4U5E1aagQJ8cGo1Q6+c1QixQorkLtprl8sUmzFbsTYhUixFbsRYxcirVbsRozdYL+uq4+jhZW2I8ZusG8vYuxKpNjK/Ijxw9OOsn07i0eKrSwaMXYyUmxl0Yixk5FiK0MT47xaU0S/RMTY2UixlcUixs5Giq0sFjF2NlJsZbGIsRIpXvnIjhT7+flJ1gy8m4nz8cdPJ6pBV/TJq/ubGhiCt1bHyOQE5b3yawZnhobirdMxPDlBkZNK1CHeXmSEhQDwfq1r9YYzI0KJ9PNh0mzmREWty2+kBUFgZ2biTJmmEyWu25zN5rQ4EsMDZxSpe4aWRpgqLNCH/fdkoNeq6egb5vC5EozjS1vH2EpyXAgP3bcab089QyPjvHmiiPLajlsa2Qzw9+LBXVlsXpeIXqehb2CUoydLOHm+guER96uCuxNBEIiMDGDP/at5aP8akpNCUakEenqGOXe+igOvXuXa9QZGVvh5KChIocz1C1FUqe1zp6lSv7lEEePbWXxLUaWWRokY3/7IdopfeOEFvv3tb2M0GpdiPLctKkEge7o8043udlkTjUalIjdMTHHN72jBLHOS0qhUbJhWor7S2uL0TXxzXAxatZrO4RHKOuQ751YEQWBnahIalYrm/kGKWp1z1Gdj0GnZmyPWRqzt6qWwyfXUbCsqQWBndjIRAT5MTJl462o5Q0tQqgkgJMCbh+7JxEOnpWdwlDfOFjM0ujyOU3CAN4/uyiIm3B+T2cK5a7Wculy15HWU7aFSCaQlh/Pk/rWkJYcjCFDX1MOrR65TUNLkUv3s5SIoyJstW1L44Ac2kLs2Dk9PHWPjkxQVN/Pqa1c5cbKMltY+JbUa+NnPfkZ8fDwGg4GNGzdy+fJlh/b785//jCAIPPbYYws+Kysr45FHHsHPzw8vLy/Wr18/R6F4x44dCIIwp332s5911yndsShzvW0UVWr73Gmq1EtSx1hRpZZEUaVWuJXIdoo/9KEP0dfXR2hoKFlZWeTm5s5pdzOZgaGoBRVdYyO0jQ7J2nd1SBh6tYa+8TFq+ntlHzsjOAR/gwHj1BTXnaw57KnTsSFGdK7PNzS6dKP29zRwT2KsaKumkQGjvJJTtgjz82bLqun1xVWNtPfLu8b20KhV7MtLJcDbg5HxSQ5fLWPMTWWg5hPk58XDWzPx9tAxMDLGobNF9A6676ZvD4Ney/1b0tiQFYcgCNQ29fD6e0V099+ask1W9Hotm9cl8sieHMJDfDGZzFwrbuLg29epbey+LRxKg0Er1jp+fB07tqUSFuaLxQJNzb28+14pB1+/RlFxM2Njy5MdsNL4y1/+whe+8AW+9a1vce3aNXJycti7dy+dnfa1EOrr6/m///f/cu+99y74rKamhq1bt5KWlsapU6coLCzkG9/4BgaDYU6/T3/607S1tc20733ve249tzsRZa6XZlkd46VYY6w4xnZRyjXZZ6kdY89ldoyXYo3xSnCMRycm3dbuFmSvKf7Qhz7EyZMn+cAHPkBYWNgCBcNvfetbbh3grcCaP/+9y4cXrCk2m2y/R7BML1B5p7GKkt5OUvyD2R+fKrkG2VbZmQstjVxqbSHM04un0rLnXlsH7FR2d3O0uhqtWs0zOWvw0C4sFyNZqmn6V2Aym/nTtSL6jEayI8LYkZjg8Phh4Vrg1wvKaOkfJNLPh8dzMhxWPpbabjFZeKeoipqOXrz0Oj60KQsPnVa+arTE+IdHx3ntYgkjYxOE+Xvz8IYMNGqVW8slWfcZMY5z5EIZfUNGdFo1+zakER5kY72Gm0omze/f0TPEyYuVjIxOoFIJbMyJJz1x4d+048d1zzgtZgt1jT1cKahn1ChO5CFB3mxYm0BosI8N+/LWCMsv1eT8GuT+/lEqK9uoru5kclJ8qFKpBGJjg0hZFU54uJ/N4xqNI3z22aVZU9zW1e2WNcURIcGyxrdx40bWr1/PT3/6UwDMZjMxMTH80z/9E1/5ylds7mMymdi2bRuf+MQneP/99+nv7+f111+f+fypp55Cq9Xy+9//XvK4O3bsYM2aNfzoRz9y+PwU7q65/icvL1xT7MhaVEWV2r79u1mVWrRlf5JQVKkVVWrj0Aj/uOaJJZnrE7/1r6jmvSB2BvPYGLXf/tpdsaZYtlPs5eXFsWPH2Lp161KN6ZbjilPcZRzhlYoCBOCTGevw1kj8IG38LY1OTvKbG9eYsph5YlUGsb7+s/ov7hRbLBb+XFRE1+goa8Mj2BYXt6D7Yk4xQFP/AK8XlSEAT+VkETLvGkiNHxY6QIPGMf50pZBJk5mtSXGsiY6w23+x7ZhhYsrEgUtF9I+OER3ox0Nr01BL3bVkOsVYoHdolNcvljAxZSIuxJ89uSloBImZ1QWnGGBsYpJjF8vp7BtGrVaxO28VceGB8/ovjVMMMD4xyZnLNTS29QEQFxnIlrxEPOzU311qp9i6fWrKRHFFK0VlLUxNiV9YfEwQedmx+PrcrGO4kp1iK1NTJupqu6iobKen5+YbaW9vA8lJoSQlhswp67SUTrE7JkrrJNnU1DRnfHq9Hr2NGpYTExN4enpy4MCBOSnQH//4x+nv7+fQoUM2j/Otb32LwsJCXnvtNZ5++uk5TrHZbMbPz48vfelLnD17luvXr5OQkMBXv/rVOcfYsWMHJSUlWCwWwsPDefjhh/nGN76Bp+fChyuFm9xNc72zTjE44Bi7KMC1qGPsooDVbMc4LTqU7fPrGLtof1HH2EX7iznGrgpwLeoYu+AUg4OOsQsCXA45xi4IcC10jFfjq5s7v7giwOWIY+yKAJcjjrErAlyLOcaKU7yykO0Up6Wl8de//pXs7OylGtMtxxWnGOBvVUU0jwyyLjSKreHyIq2nGuop6GwjytuXD6auntXfsYhzQ38/h8rLUQsCf5+dg9+8PwhHnGKAI2WVVHf3EubtxQezV6Oaf1dw0CkGKG4R1ajVgsAHc7MI9va029/edutxe4ZHefVyMVMmM1kx4WxLjbfd3wmnGKCtV6xhbDJbSAgL5P6c5AVvoO3ad9ApBtFhevdqJU0d/QjAPVkJZCaEz+q/dE4xiJHZkqo2rhQ1YrZYMOi1bM1LJC4y0Gb/5XKKrYwaJ7hW1EhVrZhmKwgCqUmhZGdE4+Wpvy2c4tn9e3qGqapqp7auayZ6DBAe5se2e1Pw8NDdNk7xfL71rW/xwgsvLNje2tpKVFQU58+fZ/PmzTPbv/SlL3H69GkuXbq0YJ+zZ8/y1FNPUVBQQHBw8AKnuL29nYiICDw9PXnxxRe57777OHr0KF/72tc4efIk27dvB+CXv/wlcXFxREZGUlhYyJe//GU2bNhwR9TZXUruprneFacYFFXqxezfaarUcxxjF51iuDNUqe05xooqtbRjvJROsTuywqz25GaG3a7IdorfeustfvKTn/CLX/yC+Pj4JRrWrcVVp7hmoJc36srQqdR8Mn09erWNdBgJp2xobIKXi69hslj4QEom0T5+0/0dc4otFguvl5fTNDBASlAQDySvmvO5o07x8PgEf8i/wYTJxPbEeHIiwud2kOEUWywWDhdW0NDbT5CXJx/MXT1TdspZpxigpqOXY4WVAGxPS2B1dJiNg0vYd6B8UFNXP0fyK8SJPDyIXdk2Ur/c4BSDGPU6e6OOikbR8ctKjGBjZpz4MmKJnWJr/+6+EU5fqaJ/UBTWWRUXwqY18ei0c3+/y+0UW+ntG+FqYQMt0yWQ1CqBtFXhZKdF4WGwEdleoU6xlakpEw0N3VTXdNLePoCHh44PPLEOlUq4bdKnHY0Uy3WKh4aGyM7O5r/+67944IEHABY4xVabH/nIR/jjH/84s+8jjzyCl5cXf/rTn2yO/cSJE+zatYvq6mqSkpKcvgZ3OnfTXO+qUwx2HGM3OMUAJQ3tnC2qByA7MYKNGdOOsRucSoCKtm5OFtpwjN1kv7arl2O2IsZust/cP8DhooWOsTucYrBTrskNTjGIjvGr02uLFzjGLjrFsIhj7KJTDPZTqV11isG+Y+yqUwwLHeNHY3IImXaMXXWKQdoxXkqnWCnJJB/ZQlsf/ehHOXnyJElJSfj4+BAYGDin3YkMTIxhnHJ8oXmibwCBeg8mzCZKeuUpL/vo9GQGhwJwsa1J1r4gRtG2xooCV5U9PbQPOydG5a3XsTleFBI539DE8LjzIg2CILArNREPrYaekVEu1DYuvpMDJIUFsiFJHOP7FfU09w64xa6VmBB/9uaKitfV7T2cLK6RrQzuKCqVim05iaxLE8+nqLaNY5fKl1UZOjjAi0d3ZZOVIiqhVzV0cfD4DVo73XtdnSUwwIs92zN4YGcmYcE+mMwWSira+Ntb17ha2Mj4MpW3chcajZqkpDD23r+aJx/P494tqxa+dFkCPHVatzQAX1/fOc2WQwwQHByMWq2mY17JuI6ODsLDwxf0r6mpob6+nocffhiNRoNGo+F3v/sdb7zxBhqNhpqaGoKDg9FoNGRkZMzZNz09fY769Hw2btwIQHV1tazrdrdxN871rqCoUtvnThPfUlSp56KoUttnvvhW9ZD7KqgouA8bIUz73G1iJec66sjvaWZTSBwbQmId2kcQBHJDI3m3qYbr3a3kBEegllqTaoN14dEUd3fSPDRI89DAzWixg4R4eZEREkJpVxdnGhr4YEbmouJJtsiKCKOis5v2oWFO19axPz1Vtg0rnjodu1KTOFxcwY2WdmID/YkL9HfanpW8hEj6Rkapau/haGElH9iQhb+n62sorMSFBnD/2lUcv15FZWs3giBw3+pEp67nYgiCQG5KNH5eBk5dr6aps59DZ4vZuz4VXy/3nZM9NGoVG7LjiI0M4MyVaoZGxjlyppT0pDDWZ8Wh1agXN7LEhIf68cCu1bS2D3CtqJHu3mEKy1ooq24nMyWCzJQI9DrZt7Zbire3Yc6a4jsNnU5HXl4e77333sx6X7PZzHvvvcdzzz23oH9aWhpFRUVztn39619naGiI//zP/yQmJgadTsf69eupqKiY06+yspI4G3oKVgoKCgCIiIiQ7KNw98317iAndbo0YmkT+WWiU7w2PcZt9jPixRdIZ4vqKawRq0zMRIzdQEpUMAAnC6spaxKzlrbNX2PsAlbH+FhhFdUdPQDszrKRgeUkVsf4cFHFjGP8QPZC8S1nsTrGh4rLZ1SpH8myIb7lJFbH+GBZ6Yxj/FhaBgate+Yzq2N8oLJkxjF+MiUTL50N8S0nsDrGB6qLGZgY40B1ER9IzsJP4mWpXKyq1AfrxYjxq/VFPBmfRaDBY/GdHcDqGL/RXEjn2BCvN93gsZgcQjwWim85g9UxLh1sZrWfY/6EwvIiO336bsCaKvD/XTlMw9QIx1oq8NboeCZlA5ht31zN83Jupsxmfl2Sz+jUJPtiU0gLCJm3g1Q6tLj9vYYairo6iPL25QOpmQhSOT0SdkbGJ/jfggKmzGYeXLWKVUFB0/1tm5FKq+4eHuUvBUWYLRb2p6eQGBQ43V/CziJpz6cr6yhq7cBDq+Gpddl4S92MZaQAT5nMHLpSSsfAMP5eBp5cvxr99CTiLlXqmtYe3r1RhcUyN7VMdmqwg2Pq6hvmnUvljI5PYtBpuH99KhHzlamXOF15atLE5cIGymvF6J63p56teYlEhfnLPK68W4yc8VssFppa+rhW1Ehfv/jWWKtVk2F1jm0Ihi39GmSZH9jYbDSO8LnPPbBiU6qcsfWXv/yFj3/84/z3f/83GzZs4Ec/+hF//etfKS8vJywsjI997GNERUXx0ksv2dx/fvo0wGuvvcaHP/xhfvazn82sKX7++ec5deoUW7dupaamhj/+8Y88+OCDBAUFUVhYyOc//3mio6M5ffq0S9dA4fbH+jv+6W8Xpk/LTd9VVKkdt39XqFLbtWV/UlFUqe0/AN7uqtSzMQ6N8Nzax1fsXL8U9lYyDr0+GxmRV8NUbv+VTLJvMB5qLcNTE9QM9Ti8n0alYk2IGInI72yRnYa0ISIatSDQMjxI05D89FVvnY686UjIucZGppxMIwr28mRtlGjnVE09Ey6m82xJiiPIyxPj5BTHy6rdko6sUat4ICcFL72O/pEx3imqcmvaFEBSRBA7lzC1bD4hAd48tj2LYD8vxiameOt8KeUN8lLxXUWrUbMlN5F996bj7alneHSco++X8f7VasZXSN06QRCIjQrk0b057LgnBX8/TyYnTdwoaeZvb17j6o0GjGPuSxFTcJ4Pf/jDfP/73+eb3/wma9asoaCggKNHjxIWJmoBNDY20iazxvrjjz/OL37xC773ve+RlZXFr371K1599dUZxWSdTse7777Lnj17SEtL44tf/CJPPvkkb775ptvP707gbp7r3cmalKVNpc6MD2fr6nhgieoYRwazM+vmfHe62M2p1CGB7FvqVOrVS5hK7bMMdYzTMvC4nVOpk7KWNJX68bilTaV+OCpnyeoYK6xcHHKKk5OT+bd/+ze7DywWi4Xjx4/zwAMP8OMf/9htA7zVaFQqVgeIKUs3elpk7ZsdHIZGpaJrbISGoX5Z+/ro9GSFiA+L55qdm/ByIyPx0moZGB/nmsyHzdlsiInGz6BnZGKC9+tcWw+sUavYl7EKjUpFc/8gl+rd87Dgqdfx4BpxEmzqGeB0mfud1lWRwezIFoV5ypo6ee9Gtdud79l4eeh5eGsmCRGBmC0Wztyo5cyNGqZMS3dMW0SF+fPEnhwyksS/g8r6Lg4cK6C6sWtJXwzIQRAEEmKDeWxfDvdtSSXA35PJKRNFZS387c1rnL9aw8CQ8VYP867nueeeo6GhgfHxcS5dujSzvhfg1KlTvPzyy5L7vvzyy3OixFY+8YlPUFVVhdFopKCggEcffXTms5iYGE6fPk1PTw9jY2NUVVXxve99745/2+0sd/Nc727uJMe4vHkJ1hgrjrFdFMfYPopjrLAUOOQUnzp1iitXrpCQkMDGjRt59tln+e53v8sPfvADvv71r/PEE08QGRnJJz7xCR5++GG+9KUvLfW4l5XswAhUCLSMDtI55rhwlYdGS1ag6Ehc6ZQ/Ia6PiEarUtExOkx1v+NRais6tZqt02vrrrS0MDg2JtsGiI7srlWiM1ja0Uldb59TdqwEeHlwX2oiAFcbW6jrcc2elRBfL/Zkr0IQoLy1i4vV8oXKFiM1KoTda5LFibyth2PXK5fUSdVo1OxelzIjwFXe0MkbZ4sZHHHuu3QWrUbN5rUJ7N+Rib+vB2PjU5y+XM3R98voH3TfROQqgiAQHxPEo3tz2Lk1jZAgb0wmMxXVHRx86zrvnS2no9s58TkFhTudu32udzfL4hhbxbdq2rhUugSOcc4Sim+FBLIvJ+X2Fd+y4RhPuN0xzry9xbfuMMe4S3GM72gccopTU1N59dVXqays5EMf+hAtLS0cOHCA//mf/+HUqVNERUXxP//zP9TX1/OP//iPqN0kOrBS8NbqWeUnClAU9MqLFueFRopp0CODtAzLS4P20urIDROVgM+1NmKyyL+ZpwYFEe3ry5TZzOmGBtn7W4ny82VNpJhGfaKqFuOka+mzqWHBZEWJkfDj5dUMGN3j5MWHBLAjXXS4r9e3UtDgfIRciuSIYPbmpqBWCTR09fPW1TKX08rtYRXgenBTOnqdhu6BEQ6eLqS+vXfJjilFeLAvj+3OJi8zBrVKoLVzgIPHC7lc2MDklPseBlxFEATiogPZvzuLffdlEhMZAEBjcy9vvVfM4XeLqG/ucesDmILC7c7dPtf39Y9QUePeZSq3uyr1qsilVaWODwlQVKntEOzpqahS2+F2V6VWWFnIkuSLjY3li1/8Iq+//jrXr1+nvLycs2fP8pOf/ISHHnrojpsgZ7M2SFSVrBrsYnjS8T9ob62e9ACxxNJlJ6LFeeFReGg09I+PUdLdKXt/QRDYER+PShCo7eujts/5qOzm+BgCPDwYnZzkZK3rE+PWpDjCfbwZnzLxdqn7Iq7pUaFsTBYfQs5VNVDR1u0Wu7OJCw3gofXp6DRqWvuGeONyGcYlXmcbHerPk9uzCQvwZmLKxDtXKrlU2rDsjp1apWJNejRP7FlDbEQAFouFospW/vZOAbXN3SsmpRrE339EmB+7t6Xz+ANrWJUYikol0NkzzIlzlbz69nWKK1qZmFi+0lcKCiudu3GuHzVOcORUCeeu1lBa5d6XqcvqGC9FKrVSrsku8x3jJUmlVhxjSTyX2TFWUqntc+bMGR5++GEiIyMRBGHBsqeOjg6efvppIiMj8fT0ZN++fVRVVdm1uWPHDlHcdl7bv3//TJ+nn356wef79u2TNXb36NTfoQiCZaaFe3oT6emLGQvF/a0IAnMbEk2A9WHRCEDDUD8dxiFRsU6yWeY0nUbFxkhRufJiWxMTZpOdfW82y6wW6OXJ2ggxjft0Yz2TNmxYJNrsPhq1ij2pSeLE1dNLZW8PFhVzmwN2rE2tVrEvcxUeWg3dw6Ocqq7DjEWyv6PjRIDchEiyY8VzPlFaQ0NP/6LjQWDh+Uw3W30jgnx5eINYLqFrcIRDl0oZGhu3a9/eOTjSvDz17N+ayepEMWp/o6aNwxdKGR6fxCIIizTHvxtHroWvj4H7t6Zx/5Y0fLz0jBonOHGpirffL6V3aPTmMVSCzbbwj0hskucvNR6pNu98/P092boxmQ8+nEd2RjQ6nYahkXEuFzTw5zfzuXC9jv7hsYXjlGhS45FsEt+LlH0FBYVlwmLBQ68hJUF8gX3xWh1llW3zp2NR/Fai2eo7uy1IpS5rEpX55zXBjM222HEz4+atMS5txGK2ODxOSfvTx0+NmCu+daa4zj32p1vi/DXGRdWYTa7bt7Zo/7lrjI8UVmKeMsu+zlLHtbnGeMokYUuw2ewdO8TDiyfnrzGenFr0vOc0s2C7IeEYT0zIsyN5bAEfrWFBKvXA+LitPwEsFsFmk3o4MFvAoF6YSt07ZpS0ZauZJZrFIqBTaRemUhuHbY7fZFbZbFKX505jZGSEnJwcfvazny34zGKx8Nhjj1FbW8uhQ4e4fv06cXFx7N69265w48GDB2lra5tpxcXFqNVqPvjBD87pt2/fvjn9/vSnP8kauyynuLS0lH/8x39k7dq1REREEBERwdq1a/nHf/xHSktLZR34dmRtkJjKXNTfJjqWDuKvN5A6XZLpUrv8da5ZIWH46Q2MTk1yraNV9v4A66Oj8dbpGBwf50qrczYAQn28WRcjRs1P1dQxPO7a20Rvg569GavEiba9i5JW+dFwWwiCwJbUOFaFB2G2WDh6o5L2fvevJQ3x8+LRTZl4GXT0jRh5/WIJvUNLu8ZWrVKxOSue3etS0GrUtPcOcfDUDRpuQTo1QGxkAE/sXUNuRjRqlUBb1yCvHb/Bhet1GMdXhkr1bDw9dORlx/KhR/LYvC4Rf18PpqbMlFW1c/Dt6xw/XUpTa6+SWq1w13I3zvWCIJCXFUt2mjjPX7xW5/aI8Z0kvqWoUi9EEd+yz524xnglR4xHJybd1uTwwAMP8OKLL/L4448v+KyqqoqLFy/y85//nPXr15OamsrPf/5zjEajXQc2MDCQ8PDwmXb8+HE8PT0XOMV6vX5Ov4CAAFljd7hO8ZEjR3jsscfIzc1l7969M2U0Ojo6OH78OPn5+Rw6dIi9e/fKGsBKxFqT6/tX38TD+2btQrPFwsuVVxmcHGdHWDLZgZE3P5Oo82uZLm7XOzbK78qvYwE+siqbMA8J9VOJ+3NFdw9H6irRqlQ8vTr3Zk05qXrHNr7V6p5e3q6sRCUIfGR1FsGes+q6SdixVc7NZDZz4EYJncMjRPv58ujqdFTTBdsk6xdLzTvT9vMbW7hQ24RKEHg4O41Yfz+Hz8uefZPJzNsFFTT1DKDXqHkkL4MQXy+31S+29h8yjvPWlTL6R8bQadTsWbOKmGB/WceQU5vZ2n9g2Mh7VyvpmRa7SosLZXNGPFqNjfRGuXWNJbA3nsHhMS4V1tPY2geIAl05aVFkrgpHMy/l8lbUNba13WKx0NYxQGllG03T4wbw8tSxKjGMlIRQvD0X1liUPIDc79fGNqNxhM/9w74VW7vwbqpbeLdxN871P/3NsZk6xRaLhfyiRgrLxZfHm3ITyFgVMbOPK/WLrditY+xC/WIrdusYyxinVH+7dYzdYN9uHWM32LdXx9iV+sVWFq1jLGnLsUli0TrGMn+j8+sOL1rH2EE7Uv0Xq2PsbP1iK4vVMXa2frGVxeoYy6lfbBwa4Z+WqE5x8lf+FbXe4LI90/gY1f/2NZqamuaMUa/Xo9fbeja6iSAIvPbaazz22GMAFBUVkZ2dTXV1NUlJSTP9YmJi2LVrl90KFLPJyspi8+bN/PKXv5zZ9vTTT/P666+j0+kICAhg586dvPjiiwQFBTl8rg47xTk5OTz66KP8y7/8i83PX3jhBQ4ePEhhYaHDB1+pSDnFANe72jjdUYOv1sDHktbPOIOLOcUAxxqqKO3rJNbHnycSVts+uITzZTHBXyqKaB8ZJjMolPvjk6f7O+4UWywW3iqvpLa/j3Avbz6YmTkzfjlOMUDfqJE/Xy9iymxmc1zMTPTYWafYYrFwvKyGys5udGo1H8xdTaCnh0PntZj9SZOJN6+V094/hF6r4dG8dELmfa+u2LcyNjHJsWuVtPWJKfLbMhLIiAlz+BjOOMUgOv5XyhspqhGjGr5eBnauTSY0wGde/6V3iq20dg5wubCBnn4xHcbLQ0deZgzJcSEzD08rxSmezcCQkYrqdqrruhifXmcsCBAdEUBqYhjREQE3H85uU6e4ravbLU5xREiw4hTfgdyNc/1spximHePiJgrLRGHNTWsTyEgRHWN3OMVgxzF2g1MMUNLQztmiemCeY+wGpxKgsr2bEzdsOMZusl/X3cfRG5ULHWM32W/uH+Bw0ULH2B1OMSx0jB+ejiDbt+X4JNE9Osqr5SWM2XKMXXSKYRHH2EWnGOw7xq46xWDfMXbVKQb7jvGd6hTP51vf+hYvvPCC3X3nO8WTk5MkJyezceNG/vu//xsvLy9++MMf8pWvfIU9e/Zw7NixRcdz+fJlNm7cyKVLl9iwYcPM9j//+c94enqSkJBATU0NX/va1/D29ubChQsO62A47BR7eHhQUFBAamqqzc8rKipYs2YNRuPtXwvUnlM8Pmnht9WXGDNNsS8qjRRfcQ2SI07xwPgYL5dfw2yx8GTiamK8/RfuIOV8mQVah4f4a0URAP8nPZtQT29ZTjHAsHGcVwoLmTCb2B4Xz5rw8Bn7trDncJS2d/JedS0qQeDJrAzCfX2cdooBpkxmXr9RSvvgMH4GPR/MXY2HVuvQeS1mf2JqijevldMxMCw6xrnphPjYcIxdcIpBdFBPFddS1SqKe+XER7A5ddZbejvHcNYpttLSNfD/Z++9w9u6rnTv30EhwAoSBHvvVRLVbclFsuReYztxycR24kzKpFwnM8mN5xt7Zm48cfpkJsmdJL6ZiRPXuNuyLRfZkq1eKYq9dwLsBSABopzvDxAQCyoJSpR03ufhI/tgn7U3QBAb737Xehf7TjVjMk8jCLC2IJ11BelnSdw5JMXO6URaOgc5XtOJadKZghWniWBDWQYZKXHIvH4zOH+k2AWb3UFn9xANLQb0/ePu6xHhYeRnJ1CQk4gm2stms8JJcf7/XvpGabeYaf7JP0qk+CLEpbjXzyfF4PxzPXGmcwExDhUpFgU43eCBGIeIFIsC1LZ7IMZe/AqCfl4yaOwZ5OOqecQ4VPEFaOsf5r35irE8NPERZhTjecRYpvDyQgdLxvFBjENAigEGpky8Wle7kBiHgBSDD2IcAlIMMGE183Jz9QJiHApSDGC2T/Nq+0JiHApSDDDtsPJmd9UCYrxSSHEoDsBd8VISdCFRigFOnDjBww8/zOnTp5HL5ezcuROZTIYoirz77rt+1/PVr36VQ4cO+T2YbW1tJS8vjw8//JAdO3b4jQtB1BRnZ2fz9ttve3387bffJmumJ+7FDKVMzpo4pyp6YrA7qHoajUrNqnincnhQ3xF0LU5qVDRFcc7WUB93Lq6WJ1qlYmums6bpYFcn45bF13OUJCVQoJup2W1ocpo+LAEKuYybyouIVqsYM1t4pyZ09T5hCgW3rC0mSROFxWrjjZN1DE54L+pfLORyGdeszmNjvvPU/3R7H+9VNoa0rsgb0hI03LVtDXlp8YginGzs5o39ZxhZ5hpnbxAEgfysBO6+fi0bV2USppQzMjbJBwcb2LW3hr6BsfOyrkCgkMvIzUrgxmvKuevGtZQXpaJWKZicmqaqrodX3jnF23uqaWztx2pdOa2oJEhYKqS93gl3jXGJc78/fKqN2kbJlXo2JFdq3/DUrinkfYwlV2qvuNRdqSPClCH7AYiJiZnz448Qe8P69euprKxkdHSUvr4+du/ezdDQELm5uX7vNZlMvPDCCzz88MN+x+bm5qLT6Whubg54bQErxS+99BL3338/N954Izt37pxTZ7Rnzx52797Nc889x1133RXw5CsVrlOWXx5/c4FSbHfImLJZ+e+mo9hEB3dklpMVFedVKXbMO0Y0Wqf575qT2EUHt+eUkBOjnXeDlyOmmesT0xaerj6FzeHg+pwCSrQJPscvvO5U8F6praV3YoJMjYbbi4uReTnuFLweczv/sdhsvHjqDGNmCznaOG4pLpyris4bvyC+h/1n2DTJyydqmLbbKUrSsbM472zKbZD71fzxFquNt07U0T9uQq1UcNv6EnSzFOOga429jXdAU+8gH1e14BBFdDGR3LCukKhwVciUTV8Kb3P3IAeqWpm22pHJBNYXZ7AmN3VW6u/s+EEqyItUQi3TNqoaeqhp0mOfab+VlqRhQ3kmurgozzcHtJ5lVpZnYLc76OodoanVQE/fqHtahUJGdkY8+dmJJCfGeFfAg0i3npoy8XdfkdKnJZx7XIp7/W//nweleEYyWFBjPCuVes74JSjIPmuM3XEWryD7rDEOYp3exvusMQ5BfJ81xiGI76vG2B0nSKV+9tw+U6kDeg6+N1e/Ncb+1upH+fVbYxxgHG/j/dUYu4cvUkH2V2PsL44/Bdlit7Gr5zQGLzXGvtY/ZTTxyLrbV6x/yFLjeVKK56OpqYni4mLeffddrrvuOp/x/vSnP/G1r32Nnp4ev7XC3d3dZGZm8vrrr3PbbbcFtN6AleLPfvaz7Nu3j4iICH7xi1/wwAMP8MADD/CLX/yC8PBw9u7de1FskoEgXKGkPM6ZdnxssDOoe6OUYVQkOO892Bf8qW10mIpNKc4Nc393+6JOHQVBYEduLnJBoHNsjJqBgaBjuKBSKLihpBCZINA2PMKpXv2iY7mgjYzg+jKnI3WDYZATnYt3y54PlVLBretLSIyJxGy18eaJ5VGMAQpSddy6qQR1mILBcROvHKymb3jc/40hQH66jru3ryEjKRaHQ+RYbSdvHqhmePz8qMYAqjAFG1dl8bkb11KSl4QgCPQYxnhjzxn2HGpgcGR5fg+hglzuJL/XXl3KZ29bz/rVmcREq7HZHDS3DbD74xpeeusEx053MDxqWlH9mmcjlCfHEi4+SHv9XCxwpV4Gxficu1LXSq7Us3GuXamXRTGWXKm94ly4Ut9yAblSLzeMRiOVlZVUVlYC0NbWRmVlJZ2dTr700ksvsXfvXndbpmuvvZY77rhjDiF+4IEHePTRRxfE/uMf/8gdd9yxgBAbjUa+973vcfjwYdrb29mzZw+33347+fn5QZlCBqwUX0rwpxQDGK0W/tR8DLsocmfWKtLC4zzGmq8UA06lueYk0w47N2QWUByXOOsG30oxgM3h4C81lYxZzKxNTOHqjByf4+deP/ufJ3t72d/ZiVIm4/Plq9GoF9YZ+lOKXajq1bOvpR0B+Ex5KemaGJ/j3fF91Oqe6dGzr6kdgB1FuZSkJC5ZKXbBMm3jrZNOxVilkHPLumKSNNEhVYpdmJiysPtEA0MTk8gEgcuLMlmVlbzwJD2ESrH7VlGkqWuAg2fasdrsyASBioI01hakIZe7pZDg4oegZhZgfMLMydouWjoH3dcykmNZU5JOUnz0gvHnWyn2NF4URfoHJ2huH6C9c5DpWanUsTHh5GbqyMnUoYkO9z7BOVaKJfdpCRKcCEQpdv+/5ErtN77kSu17bsmV2ve8l6Ir9cWoFO/du5ft27cvuP7ggw/ypz/9if/8z//kZz/7GQaDgZSUFB544AEee+wxwsLOvpe2bdtGdnb2HDfqhoYGiouLef/997n22mvnxJ6amuKOO+7g1KlTjI6OkpqaynXXXccPf/hDd7ZTIJBIsQcEQooBPu5rpmqkj9SIGO7MWOMxbdgTKQY40tfNQX0n0UoVDxavQ+FK1QmAFAO0j43welMdAnCfy3TLx/iz12evTeTVmTTqlKho7i4pPetGPYNASbEoirzf0EzjwBDhSiX3rVlFlCrM63h3fD8GVgdaOjjV1YcA3FheSJ5W6+WG4ONbrDZ2nXKabynkMm5YU0iWNtbnehbED4AUA1htdvZVt9LcNwRAQUo8V5flzm2dtAyk2AXjlIUDp9voNIwAoIlSc8WqXNISNOeNFLvGj4xPcrq+h9bOQXeIlIQYKkrSSUmIOf9u1d4wb7zN7qCrZ5i2jkG6+kbmKBna2EhyMuLJydASEx3uMw5IpFiChHOBYEgxeHClnkWMQ+ZK3dTDsZrldKU2sP9MGyC5UntCqFypvc2tnzDyRnWdZ2IcAgMuyZV69gMLr59rV+rbM9aQ4MOV+mIkxRcyAk6f9oe6urqAiqQvJmzUZSAXBHonx+meHA3q3rUJqUQpw5iwWqgcDD49OFsTR0FcPCKwp6MVxyLONmSCwHV5eSjlcvqMExzvW3yasiAIXFOQiy4igimrlXcaGrGFIP1oS24mJckJiMB7NU10j4bOnEmlVHDbuhLStRpsdgfvnGqgyTAUsvizoVTI2bEmny0lWQgCNPUN8erhGkZN58bBNSpcxfWbiti5oZBwlZIxo5m3D9Xy0YkmJs2hS4NaDOJiIti2qYC7b6igMDsRQRDoGxjn3U9qeevjatp7hlZsKvJsKOQycjJ17LiimPtu38gVG/NIS45FEGB41MSJM528/E4lb7xfxem6HsYmLnz3XgmXHi7Fvd6FBeZbJ9uobQqx+dYyp1KXZictq/lWQerymm9lJ8Rd2OZb0VFzUqnfrK4PqRGnLiJCMt/ygfBzbL71xiWeSn2hIWSkeHp6mo6OjlCFuyAQpVRRHuc8JT4yGJybtFIm5/LkTACOGrqZslmDnn9bRg5hMjmGSSNnBhZXyxujVrMtOxuAIz3dGEyL/+NVyuXcVFyISi5HP2Hk07b2RcdyQRAEthfmkquLwy6K7KpppH8idB8wSoWcm9cWkZekxSGKvH+miTNdS6+L9gRBEFidncJtG0sJD1MybJzk5YPVNPcN+r85RPPnpsbzuWsqKM12ppM09wzy149PU9OmX9TBSigRExXOlRvy3DXHcpnAwLCRPYcaeeW9SurbDG6DrpUOVZiCwtwkrr+6lPtu38jWDXmkJmkQBBgaMXGiqpNX3qnktd2VnKzuYmgF1yBLkDAbl+JePxvnhBhLrtQ+cbG5UoeeGEuu1L5wqbtSS/COgNOnv/vd7/p8fGBggOeeew77OWg9s9xwpQr8+4k3fKZPg7O2+H+aTjjdpDPKyYqam97rq3+xQxR5tqGSQfMkFboUtqfnIgaQ9jwbp/V6Pu5qI0wm54GytUS5cvK9pvUujC+KIu82NtE8PEycWs19q1adTefx2nfYe1p1+/AIb9U2AHBNfi5lyYkBpxm743tITX2rqp6e0XHUSgV3VZQRF3k2DXWptcYOUeTTujZquvsB2JibzobctABSdwNb//x7TOZpPqxsom94AoDSzES2FGejkC88pwrZ3POuD4wY2X+6lcExp8GVLjaSK1bnkhAb5ec5nJt06ynzNDVNeupa9O5a3XC1krL8FIpzk1CFeXDXDGQ93hBsGnaw35FEMJutdHQP0d49TJ9hbM4XxegoNbfsXIVarWRq0sQ3/vb6FZtSdSmlU11quBT3+t8+tTB92ld/XvBfY+weL7lSX5qu1AHM7bfG2G8cyZUaVrYrta8aY3CmT39nvZQ+vVIQMCmWy+VUVFR4fUGMRiMnT568qDbKQEgxwL6+Vk4N95IcHs1nsyrmfOj7IsUAnROjvNJSgwB8oXgt2rBIj+O9kUeHHV6sP4Nh0khBXDw35xbNTODlyXlZj3nayrNVVZisVlYnJbE9J8fnvL5IMcCxzm4Od3YjEwTuKC8h3dsfUoCkGGDaZuO1U3UMGE1EqcK4c20ZMeqZWpMQGHCJosixlm6OtzrrxVZlJLG1KBuZIIScFAM4HCLHm7o52eKcLz46gmsrCoiNmltzulykGJyHAXVtBo7VdWK1Of92izIT2VCcQaTKw+bmY+LlqkGettppaDNQ09iHacp5Gq1UyCjITqQkL5nY+TW6/tbjDeeAFM+GZdpGV88wHd3D9OhHiYwI486b1iIIgkSKJZw3XIp7/WJIMQTWrmmptcZ+ifESa43nEOPcFDaXziPGS6zVnU2Mi9MTubo8J6Tx/RLjJcb3R4yXQoohwHZNS6g1DogYL6HWOCBivIRa40CI8VJqjQMhxkupNfZHjCVSvLIQMCkuKiriscce42/+5m88Pl5ZWcn69esvqo0yUFJssk7zP03HsYkObkkvIzf6rFW4P1IM8EZrHa3jw2RGafhMTrnnPr/evoA7BPonjTxfV4UI3JpXTF6sNmhSLIjQMTrK6/X1ANxcWEi+VrtoUiyKIrsbmmgeHEatUPC51eXEhi90tw6GFANMWay8cqqG0SkzMWoVd64tI0oVFjJXakSo6tSzv6EdgGxdHNeuyvfcU3BmvMf4AZBiF7oGRtlzuhnztA2FXMbWkmyK0xNCr1L7uD5pnuZIbQfN3c5UbqVcRkVBGqtyU1AseO7nx5jLbnfQ2jXEmYYeRsbP1uOmJWkozUshIyV2zt/OSifFs2G12jGazMTFOj9vJFIs4XzhUtzrF0uKQXKlDiT+JelKHcTckiu173kvZlfq5STFfQODISPFKQm6S2K/D5gUf/7znycxMZF///d/9/j46dOnWbt2LY4Q1l2cLwRLigE+1XdwYqgLrSqC+3PWu12cAyHFoxYzf64/iV0UuTmzmIJY3cIbfJBigP3dHRw39BChUPKFsgrC5V76iPogxQCfdnRwsq+PMLmc+1c5+8Z5HO+HFANY7XZePVNLv9FErFrNZ1eXEa6ct64gSTEiGM0WXq2sZdxsQaNWcUdFqceUGl/w53rdpB/ko5oW7A6R+KgIbq4oIlrtYY4QkGJwplN/dLqZniFnH+PsxDiuLs8lXKU8J6TYBcPwBIeq2xkYdda/RKrD2FySSV6abhbhPD+kePaBS2//GLXNejr7RtwPR0eqKMlNpiA7AbWv181r/PNHiudDIsUSzhcuxb1+KaQYJFfqQOJfcq7UPmJJrtTe43gbf7G5UruI8XKS4qL/9SPkKs/f4YOB3WKm4T/+8ZLY7wM22vrFL37BI4884vXxNWvWXBSb5GKxPj4dlUzBsGWShrH+oO6NVanZmOjc4Pb1tS6qqftlqRlo1eFM2qx83Nka9P0ubMnIICUqimm7nbcbl+YgrZTLuaW0iGhVGKNmM2/Xh8aROspFhNUqxswWXqusZcIcOhMGgIJkHbevdxpiDRkneeloNfqxiZDOMRuR6jBu2VjCZUWZyASB9v4RXtx/mjb98LLN6QlJ2mhuv7Kc7evyiVSHOcn6qWZe//QMPYNj53Qt3iAIAmlJsVy7tZjP3rCW8oIUwpRyJkwWjp7p4IW3T7D3aBP6wXHJvEqChCAh7fXBQ3Kl9g/Jldo3JFdq37jYXKmrRntCFltC6CD1KfaAxSjFDoeME0NdHOhvI1qh4gt5G1HIZAEpxQA2h52n608xPm1hQ0I6V6Rkz5vAy2JnxTeYjLxQ70yjvimnkMI4T4qzb6UYYMJi4fkzZ5iy2ShPSGRHzsL2G4EoxS4MmSZ5uaqGabudIl081xXmn1UdF6EUu9c5Q4jHzRZiVCo+s6aEGHVgp2L+lGL3HFMW3qlsYMg4iVwmsKM0j4Jkndfxftfv4x7X9cFxEx9VtTA84fxALkzVcUVJNiqlwuP4gOcO8rrdZudMax+VTT1YZ1yf0xM0bCrJRKdZWPt+rpRiT7Da7LR0DlLfamBo1OS+HhcTQUluEvmZOsLmv34L4ktK8fmIJUHC+UKolGLneOFsKvUsxbikcKH5ls/4fq6fbpiXSl2SEeQ6fcevbdez/0w7MKMYz68x9rNOf69bY88gH1fNUozn1xgvNv7M9bb+Yd6bnUq9yoP51iLiu653j8xVjG9cvdB8K9C1ehqvH/eQSq3wXL4VjFLsmndw0sSrdbVzFWNve2OwCi9eFOMwL/4kQSjFzuuiUzFurp6jGGtUnjMFg1GKAWQCTNqmebV9rmKsVXvxLQlCKXaNt9htnBruYqMuC7kgk9KnVxgkUuwBrjfUr04uJMXeSK7dIXMS2+bjGG3TXJmUw7r4dK/jHR4+FVvGhnmztR6ZIPCFwgq06rM1Dd5dqedeP9jTydG+btRyBV8oq1hoeBAgme0YHeWNOmd98fV5+RTrdD7Hu+CNLHeNjPFmTT0OUWRjRhqXZTk38qW6UhvNFl6rrGNsyky0Kow7KkrRzKpdDrrW2MN6pm12PjjdRMfgKAAbctPYmJuOsAgDLl+PzY5ltzs41tRNZauzTi1KHca2VXmk6zQ+1+pz7kWS0CmLlVMN3dS1G9xtm/LSdGwoziAmctZr7fU5hyjd2gtmxxFFkcERI3UtBlq7htwtnORyGbnp8RTmJJIUH+25bj+A+AEhWHLtYfjUpIlvPLw8pDgUG+WltElKuHjh+pv4v3/YvYAUi15zMj1fllypF67T2/hLwZUaFn8wIblS+17/he5KPRtTRhPf3XDbij0AX454KxlBk+K4uDiPXygFQUCtVpOfn89DDz3EF7/4xSUvzm638y//8i8888wz6PV6UlNTeeihh/inf/onr19q9+7dy/bt2xdc7+vrIzk5OaB5F0uKAWpG9HzY14RKpuChgg2ECZ5PyDyRYlEUeaOlnraJETKjNNyZW+Z+noGSYrvDwfN1VQxOTZKr0XJrXtHc1yoIhfdwVxdHu3tQyGTcW1ZOfESEz/HgW0GuNfSzp8mZ2h2qVk0ARss0r5+qZXTKTJQqjDvWlBIboZ5Zj5f4XuBtPQ67yOGmTio7nClxeYlatpfloQrSgMvXY57m7hse5+PTLYxPOdOEyjKT2FyYQZhScc6V2XGTmeN1nbT0DAEgEwSKsxJZU5BGVLhqRZDi2bBYbDR3DFDfamB04qwxlyZaTWF2InmZCUSGe3PYDmCd3rDCSXHxt5ZeZ2S3mKn/9aVRY3Sp4lzu9XB+9vvlIMXggRhLrtQLxl/srtSweFIMkiu1v3kvdFdq9zwSKV5RCLim2IXHH38cmUzGzTffzL/+67/yr//6r9x8883IZDK+8Y1vUFhYyNe//nWeeuqpJS/uJz/5Cf/1X//Fb37zG+rq6vjJT37CT3/6U37961/7vbehoYG+vj73T2Ji4pLXEwhKYpOIV0Vgcdg4MtAZ1L2CILAtLQe5INBpHKNhdDDo+eUyGdfnOOtqWseGqRkKrr55Njalp5MRo8HmcLCrqRHLEutPSpMS2ZjhrLn6uLmVpsGhJcVzIUoVxmcqSokNV2O0TPPqqRoGjSb/NwYBmSCwpTCLbaW5yASBlv5hXj5yhiFj6GpOPCFFG8Nnr1hNaYbz/VvTaeDFT6vOea0xQEykmms2FPKZq1aRlqDBIYrUtht4cc8pDlS1YpwKbV33UqEKU1BWkMKd163hlm3lFGYnoJDLGJswc+xMJy++fYLdn9bS1DHgbkclQYIEJ87lXg8X5n7vDe4a4+JUAA6fCn2NccUy1xiXZSdzRXk2AFWtoa8xLkzVcc0qZ41xfXfoa4xzErTcsNw1xuXLWWMcPafG+K2Q1xhHcmdxKeEXco1x3qplrTH+TNby1RhLWJkIWim+6667uPbaa/na17425/rvf/973n//fV555RV+/etf84c//IEzZ84saXG33HILSUlJ/PGPf5wzf3h4OM8884zHe1wnxyMjI8TGxi5q3qUoxQCdxhFe66xGAO7LWU+8amH9pSel2PmAwBFDFwf1nYQrlDxUtBa1QhmwUuzC8b4e9vd0oJDJ+HzJGuJcNRFBKMXgTJ19ofoME9PTZGliua2oyOmsvQilGJwn6B83t1Fj6EcmCNxSUkh2XJyH5+Ulvg91cXJ6mjdO1zNkmiRMLufmVUXe+yN7QSDKtX50gveqmjBZplHIZGwryaUoJbD0cl+P+VNOuwfH+KSmjfFJM+B0qL6yNJuo+a7Yy13DO3O9Z2CMk41d6IecBmQyQaAoM5GK/DSiI1QLb5g/7zIrxZ7iTFvttHUP0tQ+gGHorHGaQi4jK1VLXqaOtCQNslkn/hebUiylT0sIBOdyr4fzs98vl1Lsvia5UvuNf7G6UsPSlGIXJFdq3/Ne6K7UklK8shC0Uvzee++xc+fOBdd37NjBe++9B8BNN91Ea+viHZBd2LJlC3v27KGxsRFwtoLYv38/N954o997KyoqSElJ4dprr+XAgQM+x1osFsbHx+f8zIYoikGdYGZGxZEXHY8IfGpoCfr0c0NCGvHqCKZsVvb2tgV1rwvrk1JJj47B5nCwu23xJ5gRSiW3FM6cho6NcrArOPV7PgRBYFt+DgW6eByiyNv1jfSMjfu/MZC1hjkV41RNNNN2O2+erqN5MPSKanJsNJ+9bBXpWufr+2FNM/vq20J6SuwJ6ToNn7tiNWvzUt0O1S98epqq9j53re+5RFqChlu3lnPzllJSdDE4RJG6DgMvfHSKfZXNjBqn/Ac5xwhTyinKSeKW7eV89oYK1pamExOlxmZ30NI1yPsH6nn+7RMcPNV60bpXR4QpQ/Ij4eLGudzr4dzs9/72+vGJKdraB5b2RGZBcqX2D8mV2jckV2rfuNBdqSWsLARNirVaLW+99daC62+99RZarRYAk8lEdHT0khf3gx/8gHvvvZfi4mKUSiVr167lkUce4fOf/7zXe1JSUvjd737HK6+8wiuvvEJGRgbbtm3j5MmTXu958skn0Wg07p+MjLNujv3mcd7qqaRxQh/U2q9McqZBd02O0moMLk1YLpNxbXo+AHUjA7SNB0/sBEHg+uwCVHIFhkkTB3sXT2YTIyPZOeNAfaKvj9qBpX1pkAkC1xbmkR0Xi90h8mZdA4YJ45JiuqBWKrhtdQm5ujjsosi7tY2c6TWEJPZsRIQpuWVdCRtynF92qrsNvHq8xl37u1xQyGVsLszk7q2rSIqNwmp3cKC+g1cPVWMYDc1rGCxSdRpu2VLGrVvKSNNpEEWRhq4B/vpxJR8cb2BwNLSp7KFCTFQ460ozuPv6Cm67ppzS/GTUKgVmi426FgNv763hxXdOcqSqnYER40VJkCVI8IZzudfDudnvfe31U1PT7H6/mn2fNtLcsviyo/k4J8S4aLmJcfKyEuPCtOUlxjmJ2gubGMdELzMxjpSIsQ9ESMT4kkHQ6dNPPfUUX//617npppvYtGkTAMeOHeOdd97hd7/7HQ8//DC/+MUvOHr0KC+++OKSFvfCCy/wve99j5/97GeUlZVRWVnJI488wi9/+UsefPDBgONcffXVZGZm8pe//MXj4xaLBYvl7B/Q+Pg4GRkZ/OfJ16m3DnFosJUohYrPZ28GPBsr2T3kyRwwtHNsoJsYpZq/yVuPYlZKptdWTbPSsPf1tHFyoJcoZRhfKFyLSu7BHdBHGjZA88gQu1oaALi9oIScGA+pyrPGL7x+9j8PdXVxrKcHmSBwV0kpqZ6+DAXQ8skFm93BWzX1dI+No1YouHNVKfGREV7Hz1+Pr/gOUWRfYxs1vc4vNxuz0tiUne428gg2ddfX+I6BET6sbsFitaFSyNlelkduotbLDaGbW3SI1Hb2c6Shk+mZmtiSjEQ2F2ag9qDknStjLsPwBJVNPXTqR9wPpSVoqChIIyU+xv/vwNvEwf5uggwvAA6Hgx7DGK1dQ3T0DM+pNY6OVJGbriM7XUt8bGRQDtbBrnNq0sQ3v3jdik2pupTSqS5VnMu9Hs7Nfu9tr/+/v9+NWh3B4WOtNDQ5D8Cv2FJAft7C2mTJlbodkFypPY2XXKm9XJdcqb1enzKa+IeNt67YvX454q1kLKol04EDB/jNb35DQ4OTbBUVFfGtb32LLVu2hHRxGRkZ/OAHP+Ab3/iG+9oTTzzBM888Q319fcBxvve977F//34OHToU0HjXG+A/T76OMlLNs+1HMNmm2ZqQR7km0+M9nkjxtN3O000nMNmmuTwhi40JZ+8NhBRbHXb+Ul/J2LSZcm0SO2fU47k3+K81/rijldMDesIVCj5fsoYoDx8SgZBiURR5p6mJluFhwhUK7ikrRzO/L3AQpBicNZ6vV9diMJqIDFNy56oyYsPVIXGlFkWRo23dHOtw1nOVpiSyrSAHmSz4Vkr+xo9PmfmgqhnDmFOtLc9IYktBFgr5wvdFqOeetExzuL6Txh6nMZtKqWBzYQbFGYnO+u9FzrvUGuThcROnm3pp6Rl0l9kmxkVRnptCTooWueDtm8H5I8WzYbM76NaP0No5RGffiLu9EzgJcnaalqzUeBLjowIiyBIplnCh4Vzt9XB+9nt3TfHvdxMeHokoin6JseRKLblS+4ovuVJ7gORK7fW6RIpXFoJOnwbYunUrzz//PCdPnuTkyZM8//zzy7JJTk5OzjG8AZDL5TiCTEuprKwkJWXhiWwgUMjkbIzPBuD4UAcWuzXge8PkcrYm5QBwbLCL8WlzUHMrZXKuzXAS4ephA+0TI37u8IwrM7JJCI9kymbjndZG7OLi0noEQeC6vDwSIp2x3mioX3KKTZhCzm1lJcRHhGOatvLqmRqGJ0NTiyoIAptzMthW4Pwd1Pb189aZeszW0KUFuRATruaOjaVUZDvfZ9VdBl46cobBieVPHY5QhXHNmnxuv6wUbXQEFquNT2raeOXAGboHx5Z9fm/QxkSyfX0Bn7tmLSXZSchlAv0jRj460cQLH56isqkH83Tgf0/nGgq5jOy0eK65rJDP37qBbZsLyE7TIpfLmDBZONPYx6691Ty/6wT7T7TQ2TuCLYQpbRIknG+cq70eVsZ+LwgCl23MpajA2c5p/8Gm5UmlllypvUJypfYNyZXaNyRXaglLwaJIsd1u55VXXuGJJ57giSee4LXXXsO+DF8Gb731Vv7t3/6Nt99+m/b2dl577TV++ctf8pnPfMY95tFHH+WBBx5w//+vfvUr3njjDZqbm6muruaRRx7ho48+mnP6HCyKY5LRhjnbLJ0Ybg/q3qKYBFIjYrCJDj4xBG9IkhGtoULn3OA/6GrCbAueRChkMm7OK3R+iJom+LS7I+gYLijlcm4tKiIqLIwRs5m3Gp0nokuBWqngjvLZxLiWIVPoPmTK05K4qbwQpUxG18gYL52sDhnxng25TMaWwixuWVdMRJiSEdMULx+p5lR77zmpR03RxnD31lVsLckiTCFnaGKSXcfqeOd4PcMT5+9DOyZSzRWrc7l35zrWFaYTHqbEZJ7maH0nz35wkk9OtzA8vrI3FaVCTl6Gjh2XF/H5WzdwzWWF5GXoUCrkTFmsNLT188HBep558zgfHKinoc3ApDl0G70ECecD52qvh5Wz37uJceEyE+NlrDGuKExjY9kyE+NVzsPmZakxTtVxzZplrDFO0HLDmsILnBiXLG+NcXHZhV1jvIAYBydK+YJEjC9eBJ0+3dzczE033URPTw9FRUWAs0dgRkYGb7/9Nnl5eSFb3MTEBI899hivvfYa/f39pKamct999/H4448TFuZMt3jooYdob29n7969APz0pz/lD3/4Az09PURERLB69Woef/xxtm/fHvC8s9Onw6OdbRo6TcPs6qlChsDnMjehCZubLuEpfRqcadJDZhPPt57CgcgtGaXkRscHlD7tgtVh59mG04xYpijU6Lgpq2jWDYG3amoZHeatZmca2o05hRRpdT7HO697vjxkmuSl2hqm7XYKtfHckJ/vTHEKMn16dirrlNXK69V1DJomUSsU3FFWQuK8lliLadXkwqDRxNtnGpiwTBMml3NDaQFZ2liv44ONPxtTFit7a1tpG3Cq+ymx0VxTlocmwkdqeIjmRoSpaSsnmrup7ezHIYoIOFPRNuanE6kOWzA+qPV4GR5oHLvdQUvPINWteobGzyrpKdpoSrOTyU6J81iHda7Tp/2NB7A7HPT1j9PZN0xn7wimqbkbuy4ukozkODJSYkmIDSzNGqT0aQnnH+dyr4fzs9/PT5+eDVEQOXy0lYbGuanUoWjVBM6PlRNnOhe0awpVqyZRgNMNHlKpQ9SqSRSgtl3P/jPtwKwa4/k1un7W6et1a+wZ5OOqeTXGoYovQFv/MO/NT6WWhyY+eG/XFIpWTeCjxngJ6dOzMTBl4tW62oXtmpaQPj0bXlOpl5A+PRsTVjMvN1cvSKUORasmALN9mlfbF6ZSr5T06b7+pbdfdMVLSbw0WjAGTYpvuukmRFHk2WefdTtQDg0N8Td/8zfIZDLefvvtZVnouYQnUgywq7uKzslhsiN1XJ+yas49vkgxwAFDGyeGuolShPH5vPUoBc8tTTyRYoA+4wQvNlchAjdmFlIUmzBzQ+CkGOBAdwfH9D0oZTLuLV5NfHiEz/G+SGjX2BivN9TjEEUqkpO5KjMLwcvrECixM9tsvFFdR7/RhEou5/ayEpKjowJaTyDxJ6etvFvTSN/YBAKwNS+LirTks2QlhMRUFEXqegc40NCO1e5AIZOxpTCT8rQkz+QohHO7MGqa4khDF20Gp4O5Qi6jIieViuwUlAp5wHHmrMfL8GDj4BDRD09Q3dZHu37YXXccrlJSkpVISWYSkeGzCPwKJMVzhokiw2OTdPaN0Nk7zODI3NR5tUpBelIsGclxpCVpPJqhuSCRYgnnG5fSXu+JFCNz/k3PJ8Z5+UmegwVJihGEszXGs4hxSaGX1O9FkGLwQIxLMjyPXyRRW0CM59cY+1mnv9dtATGeX2O82Pgz1xcQ41UezLcWEd/1WPfIQmIsU3h5sRdRz6wf90CMFZ4NYYMlxaIwU2M8nxgrPZhv+YjjlczihRiHeTDf8hXH6+sjOmuM5xFjjcqDrw7Bk2KZAJO2hcRYqw4POP5ykuKyr/4IuUrt/wY/sFvM1Pz+Hy+J/T5oUhwZGcnhw4dZtWouKTx9+jRbt27FaDw/bWFCCdcb6tenXptDioctJl5oP4GIyG1pFaRGxLkf86b8usiyzWHnmZaTjE2bWaNN5aokD6ZZgOhNQRYFDvV1ctjQhUou54GitUSFqbyO90YeHQ54rbGWrokx4tTh3Fuyyulq7S2zxyvpdv5TPzjI+83NAGzJyGBjapqX8YEryNM2G2/WNNA3PoFSLue2smJSY6K9jp+9nkDi2x0O9ja0Uad3tpYqSU5gW2EOcpksdGZUs66PT5n5qLqV3hFnT8w0bQzbS3OJCVd7vSdUc7vQNzzOofpO+mfaNoWHKVmfn0ZJZqJXw6tzacxlmrJQ39FPXbuBKYuzREAQICtJS3F2ImkJsci9He96XY+X09oA1hMQ/DzfyalpuvWjdOlH6NGPzXGyFgCdNor0pFjSkmJJ0EbN+SI2NWniWw9JpFjC+cOltNf/1+88KcUz/wZgvuUcvzgFWXKl9h//gnSlnnWP5Ert5brkSs2U0cT3Nt0ikeIVgqBJsVarZdeuXQvMNg4cOMCtt97K8HDwPXVXGryRYoCP9U3UjvWiU0VxV8YG9wezP1IM0Gkc4bWOagA+m11BcvjCN5cvUmwXHbzYeAbDlJGMKA135pV5VWZ9kdxJq5Xnak9jtE6TGxvHrXnFCP6Om33EP9XXx6cdzjrlHTm5lCcu/MIQtCu1zc6u2gZ6xsZRymTcXFpERqwmJKQYQHSInO7Wc6ClAxFIionixtICor2cIC6VmIqiSHWXgUONndgcDpRyZ/1xaVriottEBd3CSRRp1Q9zpKGT8Umn8UR0uIoN+ekUpOoWfAk4127V4DywaO8dprZdj35own09KjyMosxEijISiYqY9ztaoaR4NhwOB4bBCbr6nCR5dHxuTXuYUk5qooYrN+QRplRctKT4t7/9LT/72c/Q6/WsWbOGX//61+52P77wwgsvcN9993H77bfz+uuvA2C1Wvmnf/on3nnnHVpbW9FoNOzcuZMf//jHpKamuu8dHh7mW9/6Fm+99RYymYy77rqL//iP/yAqKsrLbBLg0trrfZFiCIwYS67Ukiu1rzkkV2oPkFypl5UUS+nTwSNoo61bbrmFr3zlKxw5cgRRFJ2bxeHDfO1rX+O2225bjjWuKGzU5hAmkzNoMVI/Hpw5RmZUHCUa50b6YW/wBlVyQcaNWYUoZDK6jGMcNSzOPCNCqeSW/CLkgkDr6Aj7ezoWFceFtSkprJ/5ErqnrZWGwcElxQOnc/etpUVkxmqwOhy8WVNPQ//S47ogCAIVGSncsqoYlUKOYdzICyfO0DE8GrI55s+3KjOZe7asJjk2Gqvdwb66Nl4/Xsuw8dwYNAiCQF5KPPdctYYry7KJUCmZmLLw8ZkWXvz0NI09AzjOgSGYL8hlMvLSdNy6tZy7tq2hPDcZlVKOcWqaEw3dPPfhSd45VEtz9yA224Xj9CyTyUhJ0LBpdRZ3XVfBvTet48r1eeSkxxOmlDNttdM/NOE17S2UmLJYQ/ITLF588UW++93v8s///M+cPHmSNWvWcP3119Pf79vEqL29nX/4h3/gyiuvnHN9cnKSkydP8thjj3Hy5EleffVVGhoaFuxDn//856mpqeGDDz5g165dfPLJJ3zlK18Jev2XGi71vX42JFdq/5BcqX1DcqX2DcmVOvSIUClD9nOpIGileHR0lAcffJC33noLpdL5QtlsNm677Tb+9Kc/odFolmWh5xK+lGK7Q8bpkS4ODTajliu5L2szKrkyIKUYYMpm5ZmWE0zarKyLT2drYu6cx30pxS7UDPfzfmcTAnBnbjkZUbELbwggHbp+aIDdbU0A7MzMo1znoVYqAKXYuT6Rj9vaqO7vRwBuLigkb6YOzTk+OKXYpcLZHQ7eb2ymedCpSmzNymRdWsrC1Kkl1BqPTZnZXdPIwAw53ZiZxqbs9Dk9fkOl1gI4HCJnOvUcaenCZncgEwTWZKWwMSfNc8pTCOeeDavdTnW7gdOtve42VZoINWvzUilI1aHw1kf4HNcg2+x22nqHaezqp3dw3H1dqZCTm6qlIC2BlPiYhe+JFaQU+xrvEEUGh41Mmq1kpzn/ZpZTKS7/2x8hD1taSpV92kz1U/9IV1fXnPWpVCpUXrItNm/ezMaNG/nNb34DONXzjIwMvvWtb/GDH/zA8zx2O1dddRVf+tKX+PTTTxkdHXUrxZ5w7NgxNm3aREdHB5mZmdTV1VFaWsqxY8fYsGEDALt37+amm26iu7t7jqIsYS4upb3en1LsviaKHD6+0HzLOX7xSvHs+CequxaYb3lbj8/4Xq5XNvVwrMaDYhwiA66aDgP7z7QB81Kpl6jkutCoH+Sj0x5SqUMUv21whN2nGxcqxktUil3wpRiHwoBLP2Hkjeo6z6nUS1CKXRicnOSV+pqF5ls+4gej8PpUjJegFLuwUDEuJ2ZmPwyFAZcvxfhcK8VSn+LgEbRSHBsbyxtvvEFDQwMvv/wyL7/8Mg0NDbz22msXxSYZCMpj04gLi8Rst3JkKLg2S+EKJdekFABwaqgb/dS4nzsWokybSJk2ERF4t7Nh0adpxfEJbE5xbogfdbbSNTG2qDjgPOnenpNDsU7nXFdzE+2jo4uO54JcJuOGogIqUp0n9Ac6OvmkrSOkiqYmXM1da8spT3UeChzr7OH103WYLMvTUsdFgu+7fA05CXE4RJFT7b08f6iK9sGRZZnTE5RyOWvzUrl/21o2FWagUioYmzSz90wrz39SSU2nAZs9dCfZi4VCLqcgI4FbtpRx7461rCtMJzpChdVmp6FzgF2Hanl+zymO1XcyOhH6VlvLDZkgkBgf7SbEFxIyMjLQaDTunyeffNLjuOnpaU6cOMHOnTvd12QyGTt37uTQoUNe4/+f//N/SExM5OGHHw5oPWNjYwiCQGxsLACHDh0iNjbWTYgBdu7ciUwm48iRIwHFvFQh7fULIQgCl226sNs1rVlmxbg0O4krVmUDy9OuqSBVx/bVy9euKTshjuuXWzFetZyKcdQcxTj07ZoiuLOk9MJu15RfPksxrg65Ynxn9spSjCUEjqCV4ksB/pRigN6pUd7sPgXAZzLWkRAW6zGWN1fq3V0NNIz3ExcWzr0561G4TgoDUIrB2abp+cYqhsyTZERp+ExO+VxlM0DjLFEU2d3WRMPwICq5nHuKVqFVR3gd7y++6BB5t7mJ5uFh5ILA7UXFZGg0S2rV5MKpnj72tzlTvfPitVxfmO9+3ZbqSu1Ck2GQjxpasTocRCiVXFuST2acJrRq7bzH2vqH+bShHeNMX9u8RC1XFGYRpVb5nCPUbsxWm52aTgOn2/qYmnamx0aqlKzJSaU0PfG8u1XPjiOKTufqpq4BWnqH5phYxcdEkp+uIy9FS1S4BzONYNcT5PhQxF9OpVgfgjqj8fFxkhN1ASvFvb29pKWlcfDgQS6//HL39e9///vs27fPI0Hdv38/9957L5WVleh0Oh566CGfSrHZbGbr1q0UFxfz7LPPAvCjH/2Ip59+moaGhjljExMT+dd//Ve+/vWvL+bpS7hIEKxSDEiu1AHOe8m7UvuYQ3KlnoHkSi0pxSsMXt7Bc/Hd73434IC//OUvF72YlQa5ICKf/6aXOdlXRmQMxTFJ1I8b+LS/kTsz1iPzkG4qejlh3JaSS9fkCCPTUxwdbOeKpBzAe2ek+QiTy7glp5DnGqqc9cX9nVyenHl2Xi+9/Ji3HkGAa3PyGLdY6DNN8EZLPfcWryJc4aoh8PJh4CW+gMD1+fnYGhtpHx3lzcYGbi0qIjPGs7Lg7RDA09W16SlEhin5oLGFlqFhXqup45aSItRKBV4zfYMkjgVJOnRRkbxb28iwaYo3qupYl5HK5ux0j26R3n5dvjbK+XPnJGtJ12k41tLN6Y4+WvqH6RgcZX1uGhXZKZ579vp4Dl6PufzsG0qlnIq8VMqzk6nr6qeytReTeZqD9R2cbOmhPDuZ8qwk1ErP9SVBr8frixfAdUEgWRdDsi6Gy1dn06EfoalrgO7+MYbGTQzVmjhS20FyfDR5aTpyU+JR+6uL8bo/e3tivsMtZbzXL6IhQLhKSfgSa4SsM/fHxMQsyyY5MTHBF77wBZ566il0Op3f8Varlc997nOIosh//dd/hXw9lwou1b1eEEWEBR9UXg6oHc597rINuSBCQ5Oe/QedZUgeXam9xBG8HSzLnHesL88AUaSqvpfDJ53pyB5dqT2H8dkpCJw1xgDHars4UedUiz25UgveDur98LeyrGQQYX91O1UtTrXbkyt1sNuA63UrStEhiPDRmWbqupxqvSdX6qDjz/ybO1NjvLuqiWbDEIBHV2pfr7Ovj/z0WGeN8a7qBrdiHKwrta/n5qoxfqO63q0Ye3OlFry91708AwFICI/kruJSXq2vdSvGXl2pvcHbF16Z6FaMX26scSvGXl2pfcTxDIFopZq781a5U6lfbj7j1ZXa2yvt7fV3IKKWO2uMXanUr7Sf8Wi+NV/wknB+EdC799SpUwEF89h79SLGloRc2oxDDFqMVI/2sDrO80mrJ6jlSnakFPBWVy0nh7rJjY4nNSK4L5fx6gh2pOexu7OJw4YuUiNjyIqODfJZgEIm49a8Yl6or2LMYuatlnruLCg7q8IGCblMxk2Fhbzd2EjH6ChvNTRwW2GRUzFeIgoTdEQolbxd10jf+AQvna7mlrIir33hFoO4yHA+u66c/c0d1PT1c7Krl+7RMa4rzic2InTzzIZSIWdLYRaFKTo+qWtHPzrBkeYu6nsH2FqYRZYu9pz9fSnkMlZlJ1OakUhDzwCVrb2MT1o43tRNZWsvxemJrM5OJiZi6Vb/oYBCLicvTUdemg6zxUpr7xAtPYPohybcPwfPtJGm05CTGk92snbJpFBCcNDpdMjlcgwGw5zrBoOB5OTkBeNbWlpob2/n1ltvdV9zzKQYKhQKGhoayMvLA84S4o6ODj766KM5JD05OXmBkZfNZmN4eNjjvJc6pL0+cLjMt8A/MV5s/PWrnAfdbmIsssCVeikIlBgvFmXZM2nmLmIs4lUxXgwKU50HZi5iLIosdKVeAnICJMaLhct8KxBivBh4IsYeXakXCZf51pKIsQ8ERYwXAZf5VmDEOHi4zLf8EWMJKwdS+rQHuFIF/m/lqwvTp+ed6tSM9rLX0IRSkPO5rE1EK+cSBVe69Xy4jLne62mgfqyfGKWK+3PXoRQ8f1n3dpokOgQ+6GqmethAuFzB/YVriAlTez998pZW7RAYmprkxfozTDvs5MVquTm3CFmweTuz4tscDjcxlgsCNxcWkhMbt2BeT/CX9jxkmuSt2nomLNOEyeXcUFRAdlys1/EBx5/3+rQMDPNRQwsWmx2FTMYVeVmUpQTQRskH/D03URRp7BvkUGMnkzNpzOnaGLYUZqGb9X5c7hZOLjgcIq36ISpbexkcd9bGCEB2kpbV2ckkx0UjCELw6/GGEKWFGycttPQM0tIzxNCY6WwcIDk+hpwULdmzU6xDpBSHKn362w9cXC2ZNm/ezKZNm/j1r38NOEluZmYm3/zmNxcYbZnNZppn+p+78E//9E9MTEzwH//xHxQWFhIWFuYmxE1NTXz88cckJCTMucdltHX8+HHWr18PwPvvv88NN9wgGW1JcL+Pf/df73pIn/ay5y7w8/PdrmmpBlz++hiHwoDLZ7umEBhw+exjHAKDLJ99jEMQ32e7Jh/8ONA5/LVrWqoBl98+xks04PLbrmmJBlx+2zUt0YDLX7umpRpw+TLfmjKa+P7mm1fsXr8c8VYyljFJ79JAqSaFlPAYrKKdT/obgjZ82JacR4xSzbjVwkd9TYsyjNielkNieCRTdhtvtjk/+BaD+PAIbs0vRi4ItIwO82FHy5IMLBQymZMIx8VhF0V2NTbSNDy06Hhz1hoZwWfXlJMSHcW03c5btfWc6O4NqeEGQF6Clns3rCY9Ngabw8HepjZ2VTdgXCYTLnAqBEWpCdx/xRrWZqciEwS6h8d56fAZPq5pcdcenyvIZAL5qTru2rqKmzcWk6HTIAJthmHeOFLLywfOUNfVvyJMuWYjKlzFmvw07rx6NZ+7poKNJZnoNJGIQN/QOAer23nug5O8uq+Kkw3dDI2bQv7+kXAW3/3ud3nqqad4+umnqaur4+tf/zomk4kvfvGLADzwwAM8+uijAKjVasrLy+f8xMbGEh0dTXl5uZsQ33333Rw/fpxnn30Wu92OXq9Hr9czPe38GykpKeGGG27gb//2bzl69CgHDhzgm9/8Jvfee69EiCWEBO52TRew+VZFYRoby5a5XdMqZ4nYcphvFabquGbN8plv5SRouWFN4QXerqlkGc23nIrxBW2+taBdkzlk8VdiuyYJniGR4iVCEAS2JxchF2R0TQ7TOGHwf9MsqOQKbkxzKrKN44PUjumDXoNCJufW7BIiFEoGzCbe61ocuQbIiNZwY24hAlA71M+nPe1LJsY3FRRQqI3HIYq829RE3cDAouPNRmRYGJ9ZVUppUgIiTmfq95taQk7OotUqbl9dwhV5WcgFgY7hUZ4/fpo6/cCykqgwhYLLCzO5f+sa8pK0iEBd7wDPHajkcFMnlhBuOoFAEAQyEmK5eWMJn7tiNSUZiShkMoYmJtlX3cqf957kcEMnE1Ohc3IMFTRR4VQUpPGZq1dz3851XFaWRZI2GoDBMRPHG7p4ZV8VL+w5xcHqNnoGx0L6pUQC3HPPPfz85z/n8ccfp6KigsrKSnbv3k1SktOgqLOzk76+wL/s9/T08Oabb9Ld3U1FRQUpKSnun4MHD7rHPfvssxQXF7Njxw5uuukmrrjiCv7whz+E/PlJuHQhuVL7h+RK7RsXvit1pORK7QPzXamPDnSFLPZKwyeffMKtt95KamoqgiAsMMc0GAw89NBDpKamEhERwQ033EBTU5PfuC+99BLFxcWo1WpWrVrFO++8M+dxURR5/PHHSUlJITw8nJ07dwYUdzak9GkPCCZ92oXjg10cGWpFJVNwT9ZmIhTO1A5/6dOz7z/Q345CkHFP9jq0ARbjzzaq6jWN81JLNQ5R5LKkTC5Lylx4g4/06dmoGezngw5n+uKW1Ew2Jc+rM1qEK/WetlZqZwjxtuxs1iQlh8SVWhRFzvQZ+KS1HRFIjIzk5pJColWqJadPz5932DTJhw0t9E8403GztLFcU5BDlBfXwqDm9nO9b3SCQ02d6EcnAFApFWzITmNVRtLcGqRz5FYNYJ62Ud/dT3WnHuOUc4MSgOzEOEozksjQabyaeAQSP1TrnBNn1n9PWax0GEbo0A/T3T+KfdaXHKVCTkZCLJlJsWQkxi6sQ5bSpy+JdCoJFy9CkT7thuRKHdC8l4or9WLmkFypZ3CRu1IfGejkyqQcFDL5RZk+/e6773LgwAHWr1/PnXfeyWuvvcYdd9wBOD8jt2zZglKp5Be/+AUxMTH88pe/ZPfu3dTW1hIZGekx5sGDB7nqqqt48sknueWWW3juuef4yU9+wsmTJykvLwfgJz/5CU8++SRPP/00OTk5PPbYY5w5c4ba2lrU6sA8cCRS7AHujfL0KwtIsc0LyZ12iLzaeZJBi5G8qASuSy0DfJBiD62RXmuvodM0ik4VyedyKuYYXc0n0Wfvm0dmhwy839kCwM1ZRRTEznNu9ebS5+FdcFLfxydd7QBsz8xlTUKyz/HO697JsiiKfNLRwWm98wvDprQ0LktL92yK4e2zxoc9d/foGO/WNWG22VArFFxfXECWN3OvJRBHh0PkVFcvR9q6cYgiYXI5V+RnUZKcEFit8RLmFkWR9oERDjd1MWJy9uSNVqvYmJ9OYYrO2ZYrWFK5SLfq2XCIIh2GEarb9fQMne29HROhoiQjieK0hAWkMlQk1xuCfb5Wm52egTE6+obp6h9lymKd83hiXBQZiXFkJMWi00TObYEW0DoDr02emjTxv74gkWIJEpYT7r3+t+8sIMXeOiz4I8v+aoz9xfFH+vzVGM9fT8DxZ133WWPsjrP4WmOfNcZBrNPbeJ81xiGI77PGOARz+KsxdsdZ5MGE3xpjv3F8fznwW2PsJ74/suy3xjjAON7G+6sxdg9fYq0xgNlo4n9ftjykOBTtF13xkhN1i1qjIAhzSHFjYyNFRUVUV1dTVubkSQ6Hg+TkZH70ox/x5S9/2WOce+65B5PJxK5du9zXLrvsMioqKvjd736HKIqkpqby93//9/zDP/wDAGNjYyQlJfGnP/2Je++9N6D1hsYiTgJyQca2pCJe6TxBi3GA1okBcqMT/N84A0EQuD69kGeaTzFoMbHf0Mq2lPyg11EWn8Tg1CQnB/p4r6sJTZiaxIiooOMArEtOwWy1cVTfzcedrajkcoq1gT+n+RAEgauyslArFBzp7uZoTw9TVhvbsrO9E4wgkB6r4Z615bxT28iAaZI3q+u4LDODDempIXVLlckE1melkaOLY09dC4YJEx81tNLcP8S2whxiwpfPlVkQBHIStWTp4qjvHeBYSxcTZgsfVbdwqq2XjXnp5CVqz7k7rEwQyEnSkpOkZXhiktqufhq7BxiftHCkoZNjjV3kJmspyUgkVRuzIt1rlQo52TPmW6IoMjBqpFM/QqdhlKExE/0jRvpHjJxo6EIdpiA9MZb0hFjSEzREqEPjhilBgoQLG5IrtX9IrtS+IblS+8aF7kp9rnD99/6APGzp30ftM/XV4+Pjc66rVCpUQWRJAlgszpT02cqtTCZDpVKxf/9+r6T40KFDC1oGXn/99e7U7La2NvR6PTt37nQ/rtFo2Lx5M4cOHQqYFEs1xSFEgjqatVrnZrWvv5FJW3D1CJHKMK5NLQSgaqSPpvHF1d5emZZNVnQsNoeDN9vrmFhCXcTlqRmsnlGI32tronFkcNGxwLmhb05P5+rsbADO9Bt4u6kxZLUtMWo1d68pd9cZH+rs4o3aereLcyihjYzgrrXlbMnNRC4IdI6M8dyxKo539Cx7PapMJlCansj9V1RwWUEGKoWcEdMU71c18eKhKpr0gzjOUxKINjqCK0qz+cI169i2KpcETSQOUaS5b4i3jtbx7L5THGvqYnwydEYWoYYgCCTGRbOhJJM7r17N/deu44rVuWQna1Eq5JinbTR3D7L3VDPPvH+Clz6u5GB1Gx36Yaat57bWW4IECSsLbvOtgmWuMS52msUdPrVM5lvLWGNclp3MFeXZAFS1LpP51ipnjXF99zKZby13jfGym2+drTF+a5nMt8Iv5BrjBeZbK88v5VwiIyMDjUbj/nnyySeDjlFcXExmZiaPPvooIyMjTE9P85Of/ITu7m6fniJ6vd7tP+JCUlIS+pnMU9e/vsYEAkkpDjE2aLPpNA0zaDHykb6BG1LWBHU6mR2tZX18OieGuvmwt4l4VeSC+mJ/kAkCN2YV8demKoYtU7zRVstn81ehkgf/6xYEge0ZOVgdduqGBni3tRExB4ridP5v9oE1yclEKJW839xM68gIr9TVcltRMRHKpfePVchl7CjMIzkmmk9a2ukcHeP5yiquL8onPQS9kmdDJhNYl5lKTnwce5va6Bkd53BbF/X6AbYV5JARF9r55kMpl7MuJ42y9CROd/RR1aln2DTFB2eaOd7aw/qcNPKT4kN2gh3U2hRyijMSKU5PpH/MSH1XP819QxinpjnR3MOJ5h5S4qIpTksgNzmeMC91TSsBkeEqSrKTKMlOwu5w0D88QXf/KN0DYwyOmRiZmGJkYorqVj2CAAmxUaTGa0hL0JAUF4UiRCfwEiRIuDDgVowFaGhcRsVYEKiq63EqxuAxlXqxqChMAwGO1SyjYiwI7D/T5lSMwXMq9SJRmKoDGXx02qkYA55TqRcJlyv17tONF7BiXMIb1XVuxdhrKvUi4CLGr1xUinE5MSFQX88F3vvZV0KXPv30P9LV1TUnXrAqMYBSqeTVV1/l4YcfRqvVIpfL2blzJzfeeOOK6P4hKcUhhlwmY0dyiduNumYs+NPVyxOzSYvQYHXYeburFos9+NM1tVzB7bmlRCiUDJon2dVev+hTRkEQuDYrn9J4p/q6u62R+uGlO0gXxMe73QoNJhMv1lQzPDW15LgulCUncs/qcrTh4ZisVl6rruNwR9eyqLhxkeHcsaaEa0vyiVAqGZ0y83pVHe/VNWFaxvZNLqiUCjblZ/CFK9eyMTedsBnl+MPqZp47WElNt+G8uiknaqK4qjyXB65Zz841+WTonIcFfSMTfFzdytMfneCD0010DIyseNdnuUxGik7DptIs7rx6NV+4fgM71hdQkpVETKQaUYT+ESOVzT28faiWP+0+xpsHqjle30XP4NiKa10lQYIEJyZNFrq7hkMWT3Kl9g/Jldo3JFdq37jQXamXE+EqZch+AGJiYub8LIYUA6xfv57KykpGR0fp6+tj9+7dDA0NkZub6/We5ORkDIa53X0MBgPJycnux13XvI0JBBIpXgZoVZFcrnP+co8MtTBsMQZ1v0wQuDG9mEhFGCPTU3zQ27ioD3FNmJrbc0pRymR0Gcd4fwmtmmQzxLhcl+gkxu1N1A4tfXNPjY7mc2VlaFQqxi0WXqyppmN0dMlxXYiPjOCeNeWUJjoJ/dHuHl4+U8uoOfSpu4IgUJSk4/Ob17A6LRkBaOwf4i/HTnOqq++ckD2VUsHGvHS+cMVaNuWlo1YqGJ+ysK+ujb8cOEVlRx/TttBteMFCIZeRn6rj5o0lfGHbWjYXZKCJUGNzOGjuG+KdEw38ee9J9te2Yxg1roiTQ38IVynJS9Nx5Zpc7t2xlvt2rOPqNXnkp+mIUCtxOET0wxOcbOrh7UN1MyS5hqN1nXT1j57zdOspizUkPxIkXEwwm6d5790qPt5TS2fH0sqEZuOiIMZFy02Mk5eVGBemLS8xzknUXtjEOCZaIsY+sJAYS6nUoYBGoyEhIYGmpiaOHz/O7bff7nXs5Zdfzp49e+Zc++CDD7j88ssByMnJITk5ec6Y8fFxjhw54h4TCCT3aQ9wObf9oeplhAgFR4YaSY/QkReVjN2L3d98V2pRFNnVU02naZj4sEjuzNgwJ+XFm5v0bFdq/eQEL7VXYRdFLkvIYpMuy8N4z89BnLWejvERXm+tw4HIWl0qV6VmL0wfCtCVWhRFPupo48yA8zTm2qw8ynRJXsefve67hdOk1co7jY30TkwgAFdnZbPG0+mOV1M//+tvHBjk46Y2pu12lHIZV+fmUJyom/NaBOtY7Gt8/4SJfQ2tGGbaN8VGqLkyP4ssbVzQsRa7JqvNTm1PP5XtfW7FWqVUsCojiVWZyYSHKUPnVu0F/uKIosjAmImm3kGae4eYmlX/HROhIj9FR35qPNooz2UEy+1ivZTWS6IoMm4y0zc0Tt/gOL2DY0ya5xJKAYjXRJIcH83G4kwUCjlTkyYe+fy1y+JIufbz/7Zk8w37tJlTz/5/kvu0hAsabvfp37yDShXBgf0NtLUOIAgCV28vJjPLQ5mQ5EoNSK7UnsZLrtSSKzUE50ptNpr435fftGI7TSwmntFopLnZ2dJ17dq1/PKXv2T79u1otVoyMzN56aWXSEhIIDMzkzNnzvC//tf/Yv369bzyyivuGA888ABpaWnuuuWDBw9y9dVX8+Mf/5ibb76ZF154gR/96EcLWjL9+Mc/ntOSqaqqKqiWTJJS7AcNE700G/UcHmxgyh74yZMgCFyTXIRarmRo2sTRodag506OiOaalDwADg900DYxFHQMgKyYOK7LdDpZnxrs5cRAz6LiwMzzyspxt2f6oKOFMwOBF7F7Q4RSyR0lJZTodIjA3o52PmprC+lpaGGCjvvXrSY1Jhqr3cGHTS3sbmjCvExKXWJ0JHevL+ea4lzClUpGJ828VdXAW1X17nZKyw2lQs6arBT+5soKtpXmoolQY7HaON7aw58/Ocm+2tZzthZvEASBxNgots6Yc920sZiCVB0KuYzxSQsnW3r466dVvPjpaU40dzNiPL/rDQaCIKCJCqc4K4nt6wu4/7r1fG5HBVdV5FGYkUB0hAoRGBwz0dw9iFwufSRLkHA+IJMJbL2iiJzcBERRZN/H9XS0h1gxPpfmWyfbqG28gM23Wvo4Urt85lt1Xf3sq5bMt2ZjvvnWmzMEOVSQzLcuDRw/fpy1a9eydu1aAL773e+ydu1aHn/8cQD6+vr4whe+QHFxMd/+9rf5whe+wPPPPz8nRmdn5xzjrS1btvDcc8/xhz/8gTVr1vDyyy/z+uuvuwkxwPe//32+9a1v8ZWvfIWNGzdiNBrZvXt3wIQYJKXYI2YrxWFRat7oOcrItJHsyESuTvBsnOWtf3HL+DC7+84AcFPqajIj44HAlGIXPuptpmpEj1Im53PZFcSrImeN9/wcRA/rOd7fw6e97QDsSM9jVfwsJTaI/sUAoh32dbdT2e98025JzWRjchpCsEeg8z7PRVHkZF8fBzo7AWd69U0FBWdP+5agFLunFEVOdPVytNPZZzgyTMn2vFxy4uNCqhTPhsVm41h7D1XdehyiiACUpyaxMTudiDBlULEWvSacz73VMMyp9l4Gxk3u65nxsazOSiZDq/Hfa3mZ+wW7YLXZ6egfoblviM6B0TlfLuKiwmdaQMWREB0ZnHHKOVSKA4Fpahr90DjTVjsl2c6si+VUikPRu3ApfQslSFgpmK0Uu/oUOxyib8V4kUqx+//9KMaLVYpnx/elGC9FKXbBp2K8BKXYBZ+K8RKUXBd8KsYhiO9XMV7iHP4U48UqxS74VYwXqRS74FcxXqRS7IJfxXiRSrEL/hTjS10pvpAhkWIPmE2Kw6MjGLSM82bPMURErkpYRW7UwpQkb6TY7pDxaX8jNWM9qOVK7s7YQJRSHRQptjscvNJRQ+/kGDFKNZ/LriBCETYz3vNz8ESKAT7paXcrxTdmFlIUN9N3OEhSjENAFEUO9HZyXO+MV5GQzNXpXpwdAyTFLrQNj/BeSzPTdjtRyjBuLiwgOSo6JKTYdd0wYeSDxmZGppz1xYUJ8VyVk+3ZATtEKcYjk1McaO6kfWgEAKVcRkV6CmszUlF5c3wMISl23yqK9I6MU9Whp21gxH09NlJNeXoyxak6727l54gUz4bFaqNNP0yrfpjuwbE57aai1GFOgpwYR0pcjH/nzxVGij2NX05SHIqYl9ImKeHihSdSDH6I8RJJMcwQ4+OtNDQuJMZLJcWu+Cequ6iqc+7Ns4lxKEgxQGVTD8dqPBDjEJBigJoOA/vPOB215xDjEJBWgEb9IB+d9kCMQxS/bWCE3VWNnolxCObwRYyXSooB9BNG3qiu80yMl0iKwUmMX5lRihcQ4yWSYvBDjJdIisE3MZZI8YULKVcvAOhUMayNywHg8FAdJltwJk2X6/LQqaIw2618oK/BLgaX7iKXybg5rZQYpZpxq5l3umuxLTJl5oqULLdC/F5nE81ji0vJBme61hVpWVyVng1A5YCed9oaF7222ciNi+PesnLi1GqM1mleqq2lUq8PaapTUnQU91asZl1aitMUa2CIZ06epnFgcNkMnuIiwrllVRF3rCkhISoSq93BsY4e/nzkFJXd58aMC5y/uzSthhvXFvH5rRWszkxGKZcxajKzv6Gdpz85yb76NoaMk+dkPf6gUiooTk/kpg3FPLhjPTvW5JObpEUhl2E0T3OmQ8+bx+r408cn2FPVTIt+iOkQpmVJkCDh0oFMJrD1yrmp1JL51lxIrtS+kSO5UvuEZL4lYSVCIsUBYk1sNjpVDNMOGwcGa4L68FTI5FyXUk6YTIHBPM7hwZag5w9XKLkto5wwmZzeqXH29DUs6gNcEASuSculOC4BByLvdDTQOr60FhTrklK5MacQmSDQNDrE682LayM1H3Hh4dxTVk6BVotDFNnX0c67LU0hrXFRyGVszcnis2vKiY+IwGyzsbuxmV31jRiXsZVSepyGz60v54bSAjThasxWG5+2dPCXo5XU6QfmKKHLDU2EmiuKsnnwqnVcWZxNXGQ4VruD6m4DLxyu4vUTtTQZhlZMqySVUkFBqo7r1hXy0I4N3LCukKK0BFRKBRarjcbeQd6vbOJ/9pxg17E6znToGZ8Mvdu4BAkSLl5IxNg/JFdq35BcqX1DIsYSVhqk9GkPcKUK/L8zLxERfdbxdnTaxCvdR7GLDrboiimOyXA/ZveWPj0rL6XNOMg7PTUA7EwuIz96ofOkt7Rql+t1h3GENztqcCCyUZfB5Qk5Hsd7iyPOrMchirzb0Ujj6CByQeCW7BJyYjy4IgeRVt05PsaupgamHXZ04RF8pqDUby2w13yheQ6+p/V69nd04hBF4tRqbiosRBcxy43Y2z7gJb6nLB+7w8Hxrl6Od/XgEEXC5HK25mRSlpSIzGu+kLd5vVz3MLfd4aBOP8DRtm4mZ5yXtRHhbMpOJy9B67VOO+j03QAzg0RRpHd4nDOdBtoGhnF9QoQrFRSnJVCaloQmYqFxQchcoBeZruxwiOhHJ+gwjNDeP8KYaS4RjosKJzMhlsyEWJLjoj26dnqc9zy4W09NmvjOfVL6tAQJywnX+/j3//n2nPRpAHEm1dVvjbELkis1ILlSe7pHcqWWXKnB85+A2WjiB1tuXLF7/XLEW8mQSLEHeCPFAKdHuzky1IBCkHFH+uXEKJ2PB0KKAQ4NtHJyuAulIOfOzPXEhc3diP2RYoCaET0f9jYBcE1yAeVxCzc4f6QYnMT4nfYGmsaGkAsCt+WUkBU9jxgHWWvcbzTxelMdkzYrMWEq7igoQauOWBIpdqFvYoJ3G5swTk+jkMnYnpNDaYKrJjq4+L7qXIdMk3zY1EK/0WlElRIdxfa8HHSRkR7HB7p+f3NbbXaqevSc7OzFMtNLOD4ygk3Z6eTGxy3YpJeLFM8ebzRbqO3up66nH9Os3rTpWg0laQnkJDhTmOH8k+L5GDVO0d4/QodhBP3oBLM/6ZRyOek6DRk6Dek6DTEeSL57XokUL2ssCRLOFwIhxbCQGF+1rZis7HnEeAm1xoEQ46XUGi8gxmtzKC2c+71hqbXGfonxEmuN5xDj3BQ2l84jxkus051NjIvTE7m6PGdp8efdExAxXsJzCIQYL6XWeD4xvnVGQQ4sjv9NPSBivIRa44CI8RJqjQMhxhIpXvmQSLEH+CLFVofA7r4T9JlH0KliuCV1IzJBFjApdogib3ZX0TM5ikYZzp0Z61HJzxo7BUKKAQ71t3N0oAsBuCm9lLzouRt0IKTYGdfB2+0NtIwNIxcEbs4uJjdGOytQkAqpQ2DMYua1plpGLWbC5HJuyikk25MKDUGRYgDztJXdzc10jo0BUBgfz/acHNQyb6ZQwZNicP6eqnr1HO7swmp3IABrUpLZnJmOavYHdQhJsesei81GZVcfp7v17lRxbWQ4GzOdyrHM5Q7tbYJlIJsOh0j74Ai13f10Do66r6sUcvKS4ilKTSAlJio0LtDLYGxlsdroGhyjs3+EroGxOb2QwdkPOT3eSZDT4jWolGd/xxIpXt5YEiScLwRKikFypfa3fpBcqf3NIblS+573UnSlXk5SrDcsvdOEK15y0qXRbUIixR7gixTbRBlG2xSvdx9m2mFjlSaLjfGFAZNigEnbNC91nMRos5ARoeXG1NVuohMoKRZFkQ97m6gdNSAXBG7PWEV6ZKz78UBJMTjTd9/ucBJjmSBwU2YR+bHxM4GCJ8UAk1Yru1rr6TVOIABXpmWzNjFlIWkKkhQLopOwHu/p4XB3NyIQFRbGdbl5ZMRoPMRZHCl2YcJi4dPWDlqGnHXXEUolW7MzKU7QOZ/LMpBiF8xWJzmu6jlLjmPD1azLSKUoSYfCW/rvMpPN8Ukz9T0D1PcNYDSfrc/RhKspStFRlKIjJjyAvnDnkBTPGe4QGRgz0TkwSvfQGIZ5KrIA6DSRpGpjSIvXkBobjVLhxRk8BOuRSLEECecHwZBikFyp/a0fJFdqf3NIrtS+573UXKmXkxSv/9y/IQ8LvEevN9inzZz46/93Sez3Ein2AH+kGKDd1M9HhtMAXJtcQao6yWMsT6QYQD9p4vXuk9hFBxVxmVymywMCJ8XgJIe7OutoMw4RJpNzV9YaEtRRPuN4IsUANoeD9zqbaBwdRACuyyigRJu4aFIMTrL9UWcrNUNO85BSbQLXZObNJXOLIMUu6Ccm2N3SwpjZWTu6LjmFy9MzAoofbJugzuFR9rW2MzozV3J0FFflZDnbRAURZzFzm6dtVPXoOd3T506rjgxTUpGeQnlKEmHzCdtyk82Z66Io0jMyTkPvAC39w9jsZ3PYU2KjKUzWkZ+oRR3mocXVuVhngOOnrTZ6hyfoHhqjZ3CMEdPUnMdlgkCiJtJJkLUxJMVGea6nCtF6QCLFEiScCwRLisG57x741AMxDgEpdl4XOXx0ITEOBSkG58fNiTOdC4hxqEixKMDpBg+KcYhIsShAbbue/WfagVnE2OvrH2R8GTT2DPJx1TxiHGx8H3O0DgzznifFOETEvnt0jF1nFhLjUJBi8KEYh4AUAwxMmXi1zgMxDgEpBh/EOASkGGDCaubl5uoFxFgixSsfEin2gEBIMcDhwXpqx7tQyZTcmno5kYrwBbG8kWK7Q0bThIE9+loArkkqoTAmOShSDGC1OXi9q5reyTEi5Eruzq4gNiw8aFIsziiwH3Y1UzviJLHbUnOoiE/zOD4QUuyMK1LZ38cn3e2IQEpkNLfkFs0y4Fo8KQaYttv5tKOD6n7nmnXhEVyfl3/WhCtEpFgQnQcHlb19HOvqwTrj8FiUoGNrZiZRqnm1KSEkxa7r0zY7NX39VHb3YppJ/1Up5KxOTWZ1WjIRLvJ5jkjxbFhtdloNw9T3DdAzMu6+LhMEMuM1FCbryE6Im0smVwgpng+jeZreoTF6hsbpGR7DODXXrdJFklO0MaTGxZASN09JlkixBAkXBBZDihFmFOP5xDgnwePwYEkxshnFeB4xzsv3fPAeLClGEM6mUs8ixiWFC71JfMb3c30BMS7J8Dx+kURtATGeX2PsZ53+XrcFxHh+jbG/+L7mEKCt3wMxloeG2CPMKMbziLFM4eXFXgQZ1497IMZeMqqCJcWiMJNKPZ8YK72UyQVLZvFCjMM8mG/5iuP19RGdivE8YqxRLTTfktKnVxYkUuwBrjfU/9S8uJAUz0qTtosO3uw5zqBlgkRVLDelrEcmzP3QsXn5xHelWx8aaOXUSCcyQeD29AqS1LE+x8+HQxSw2G283F7FoNlEtFLF3dmrifZA0MG/giyKInt72qgcdLZuuCwpg81JGQs3gyAV5I7RMd5pbcBitxOlDOPW/GKSIqOCTz/2Mm/r0DAftrZittmQCwKXZ2RQkZKCPEjXaCEAkm60THOovZP6fmd7DoVMxvr0VNampbhJn8861EUQ8tmwOxw0GAY52dHL6JRTuZYLAkXJCVSkp6CN9Py7D5qMe4OfdU6YLTT3DdGkH2Rw4myfY4VcRk5CHPnJ8WTqYpELnt/T56OGF7zvb+Mms5MgD43RNzyByTyXJAsC6GIiSYmLJlkbQ0psNOEqL+q4p3m9keJ7JFIsQcJywk2K/2PXAlLsXdl0Xpdcqb3En3X9UnalDmQOyZXa92Z8KbhSm40mHt0qGW2tFEik2AMCJcUA49ZJXu0+itVhZ5Umm43agrnj/ZBiURTZ3VtDm2mQcLmSz2SsJ0bpQXH2QYoBTLZpXmmrYmR6Ck2Ymrsz1xCpXHgqFYiCLIoiRwxdHNI7N7MKXQpXp87bDIJNqxYFRsxTvNlcz4h5CrkgcE1WHmXxCzd5n3F8zGuanmZPayvto6MApERFcW1uHnHhHkjiEkixC4YJI5+2dtA3PgFAlCqMy7MyKEzQeSfjPuYOlBS74HCItA2OcLKzB8OEyX09SxvLmvRkMuI0c35n54oUz8awcZKmGYI8PnW2f1+YQk5OQhx5SfGkazVuB2uf6wxyPcGOD0SJFkWRiSkLvUPj9A6P0zc8wcTUwr6Emkg1ybHRJMdFkxQbRWxUuNs3YMG8EimWIOG8YCmkGCRXan/rh0vXlTrQOSRXat/zXuyu1BIpXlmQSLEHBEOKAVqNBvYYqgHYkbiGrMizm5Y/Ugxgddh5vesUAxYjscoI7shYh1qu9Dp+NhyzPp0mrBZebjvNuNVCXFg4d2auXkCMg0mrPjXQy94ep6FFgSae6zMLz9brLoIUA5htNt5ra6JtbASA0vhEtmfmoJQFWBfrZ15RFKkZGODTjg6sdjtyQWBTWjrrU1LmbgQhIMUAokOkaXCIg+2dTFicCqI2IpzLMzPI1S5so+QrVrCkePZz7huboLK7j9bBEffDcRHhrEpLojhJR5hCcV5IsfsWUcQwZqRZP0SLYRiT5azaqpTLyNTFkpugJVMX6/kkeBHrCXb8YtOzJ6Ys6Icn6BuZQD8ywfAsddwFpVxOYmwkibFRJGmiSIyNImIm5f5ck+JQpFRdSulUEi5eLJUUg+RK7W/9cGm6Ugczx8XmSr2AGC+BFMPF7UotkeKVBYkUe0CwpBjgwEATteOdKAUFt6VtQqN0brCBkGIAk83CK51OR+okdQy3plWgmEUUAyHFAGPTZl5pr2LCCzEOtta4bniA97ucH9ZpkTHcml2MWqFcNCl2ziVytK+bw71diEC8OoKb8wqd/Yz9xQlw3nGLhY9aW92tm3QREezIySU5Kspn/GBJseu6ze6sNz7Z3Ytlxik6KSqKLVkZZMRqgoq1cE2Bjx+dNFPVo6dOP4B1Zh1KmYzCJB2rUpNIiAqi17I3LIIUz7ldFNGPOglya/9cgiwTBNLiYshJiCMnQUuU2kuNTwDrCXb8YknxfFimbehnCLJh1MjAmBHrLBMyF6LUYSRookjSRJKgiSIhJtLdCmo5SfHGzzyBQrk08w2b1cyx1/7pktgkJVy8CAUpBsmV2t/64dJzpQ52jovNlXoOMV4iKYaL15VaIsUrCxIp9oDFkGKrHd7tO4HBMkqsMpJbUzehlCkCJsUAwxYTr3adYtphIztSx3Up5e6Uy0BJMcDY9BSvtJ1hwraQGC/GgKtrYpS32uuZdtiJU6m5PaeUWGWEx/GBkGIXusbHeLe1kUmbFaVMxo7MPIrjE3zHCYKMi6JI4+Ag+zo6MNtsCMCapGQuS09HJfesRi6WFLtgsdk42d1LZa8e24wZV4ZGw2VZ6aRERwcV6+yaghsPMG2zUacfpLrXwMjkWTfllJgoylOTyE+IP6v6n2NSPDuOS0FuGxihbWCYUZN5zpCE6EiyE+LI1sWSEB0ZnPLuY15PCBUpnj/eIYqMGKfoHzViGHUS5RHj1ILbZILAwzs3opDLJFIsQcI5QKhIMUiu1P7Xf2m5Ui9mjovNldpNjENAiuHicqUuitVxY3aRRIpXGCRS7AGLIcV2h4xJm4U3eg8zZZ8mMyKBHYlrsOPZjc8bye2eHGNXTyUOUaRMk8YVCQUIghAUKQYYNZt5taPKTYw/k7maKKVqUaQYYHDKxBtttUxYp1HLFdyaVUJapMbDDR7DeP0UNU1P825bI90TTsfi0vhEtmXkEDY/ndqFIBVqQXT2TP6ko52GoSEAIpVKrsrMpkCrXUCylkqKXZi0THO8u5czegOOmRcxK1bD5sz0oNs4LSXtWRRFekbHqe410Do44l6LWqGgOElHWUoi2ggvBxxegwa5ziDijJimaO93EmT9mHHOYxFhSjdBTtdqzhp6rHBS7AnTNjuDYyb6x5xKcv+YiTCFnM9tXQ1I6dMSJJwLhJIUS67Uvtd/qblSL2oOQXKl9jfvxeBKfaCvg+3puajkCokUrzBIpNgDXG+AP9c+HzAptonOP/p+8xi7eo7jQGRtXA4VsQWex/sguc0T/bzf52zVtCk+mw3x2di9kEGHl53DIQpOxbj9DBNWC5owNXdmrfLuSu2tZnbWp5/JOs0brfUYJo3IBIFr0/Mp1SZ6HT93As+XEQUcosiR3i6O9HUDEKdSc2NuIYkRUR7Ge4/j+frZ/+wYHWVvW7u7r3GmRsP2nBxi1WqP4+eu33N8f0Rw3GzmWGcPdYYBd+isuFg2ZaaTHD33+S22VdOCNXkZbjJPU9vXT01vP8ZZKcvJMVGUpSSSnxgfUMuk5SaPrtdh0jJNx+AoHQOjdA2NzklBlgkCqXExZOpiyYyPJS5S7fWLibf4S11nsOP9vW42u8NtODY1aeK7n5WMtiRIWE64SfGvPJFiLzdJrtRz1hNw/FnXLwlX6iXMIblS+95EL3RX6tkwG008esUNK3avX454KxkSKfaApZBigMbxXj4ZcJLabYmryYlMXjjej/JbNdLN/oFmAK5KLKAkxsOmgW9SDDA+bebV9jOMWc1EKVXcmbma2LCFxDgQUgxOU7D3OpppGnWqrpsS09mSfHbDWQwpdqF7YozdrU0YrdPIBIGtaZmsS0ydu9ksgRSDs9fw8Z4eTvT0YhdF5ILAxrQ01qemOtOJQ0yKXRibMnOsq4f6eeR4Y0YaKTHRvmOFiBS7xjtEkc7hUWp7+2kbGnFnA4TJ5RQkxlOcnEByTBSCl0jnihTPht3hoGd4nI7BUdr7R5gwz3V8jlaHkamLJV0bS5o2xvvJsZf4i1lnsOODed0kUixBwvJjOUgxSK7U/tYPl4Ar9RLnuNhcqT0S4yWkVV/ortQuSKR4ZcHbW35FwG6389hjj5GTk0N4eDh5eXn88Ic/xB+P37t3L+vWrUOlUpGfn8+f/vSnc7PgGRTGpFKuyQTg04FqBixjQcdYHZfOBm0WAJ/0N9Ew3reotcSEqbk7ZzVxYeEYrRZeaq+kf2piUbEAlDI5N2cXsinRuYEd7e/mzfZ6LHbbomO6kB6t4W/K1pAXq8Uhinza3cErjTWMW8z+bw4QCpmMyzIy+Pzq1WRqNNhFkcPd3fy5spKGwUG/763FQhOuZmdhHl/YUEFJYgIC0DEyystVNbxcVTNDTs/N+ZRMEMiOj+OmVUU8dPk6LsvJIEatYtpup6avn1dO1fDMkUqOtHe5+yCfb8hlTofqK4uz+ZsrK7hv6xq2FmWRHq9BJghMmKep6e7nvapG/mfvcV46coZDTZ10DY1h82ByJUGChJWFC3W/9wSZTGDrFUXk5CYgiiKf7K2ns2MwZPEFQeCyjbkUFTgP3PcfbKK5pT+k8devymR1cSoAh0+1Udu0uO8g3lBRmMbGUmfq9Im6bk7Vd4c0fll2MleUZwNQ1drHkbrOkO6xhak6rlmVjwDUd/fzSU1bSOPnJGi5YXUBMkGg2TDEh9XNOLypF4tAepyGW8qLUMhkdAyP8k51I3ZH6PbK5Ohobp+pKe4dn+DN6nq3+WcooIuI5M7iUsIVCvpNJl6fMeEKFWLV4dxdWEaUMoxh8xSvNNZgsk77v1HCBY0VrRT/6Ec/4pe//CVPP/00ZWVlHD9+nC9+8Yv827/9G9/+9rc93tPW1kZ5eTlf+9rX+PKXv8yePXt45JFHePvtt7n++usDmne+UjwyPY5aHka4XB2QUgzgEB18oD9N1+QQalkYt6RuJnpW/2F/SjE4T2v3DzRzZrQHAbgmqZT86KR54wOrNTbZpnmjo5oBswmlTM4t6WVkRMbOGu8xjA/lV6B2uJ8Pu5uxiyJxKjW3ZJcQr/LgbgwBKcVn5xSpHjDwSXc7VoeDMJmcqzKyKYtP9KpeBqoUuyDMmDw1DQ3xaWcnxmnnh11yVBRXZWadNcVyr39pSvH8NY1OmTnR3UN9/6C7zlcbHs66tBSKEnQBtY8KWXqzeLb2uE4/QOvAMNZZm2NyTBSFiTryE+KJCFOeF6XYF6w2Oz3D43QNjdI9PM6Iaa6JlVwmkKSJJi0uhjRtDMkxUR5TxYJdZ7DjJaVYggTvOB/7/XIpxS5IrtS+1w8XsSu1j3suLVdqH4rxEpRiFy50V+rlVIr1hoGQKcXJSQmXxH6/oknxLbfcQlJSEn/84x/d1+666y7Cw8N55plnPN7zv//3/+btt9+murrafe3ee+9ldHSU3bt3e7zHYrFgsZxNxxwfHycjI4M/1z7PiHyCg0OVxIVp2J64CVH0nJY5nxQDTDts7Oo5wfD0BLHKSG5K2YRqpv9wIKQYnJvSvv5Gasf6EBDYmVxKblTirPGBG3BZ7Dbe6qylZ3IMmSBwQ2oJ+TG6mfEew/gkxQD6yQl2tdczYZ1GIZOxM62A4jgPxiJBkGLndeeH0XvtzfSZnMp2ZkwsOzPz5jQ+DySOJ8z+zLXa7Zzq6+N4b6+bDBbGx7M1I5MY1cxcISbFLhgt05zu01PdZ2B65hQ1MiyMNSlJlCcnOT/czwEpng2r3U7r4AgN+gG6RsbcDws4T5cLE+PJ02lRzU9VOo8u1rNhNE/TMzxG98yPyWKd87hCJiM5Noq0OA2pcdEk+SPJEile1lgSJMC52e+97fUuUmyemmZsZJKk1NiQkGKQXKn9r/8idaUOIfGWXKl9z3shu1IvJynefOsPl9xpApzdJo689dglsd+v6PTpLVu2sGfPHhobGwE4ffo0+/fv58Ybb/R6z6FDh9i5c+eca9dffz2HDh3yes+TTz6JRqNx/2RknHVDjFZGIUPGoGWEypG6oNYfJlOwM2ktEXIVo1YTH/efxi4Gl54iCAJXJxZSGJ2MiMgefS3tpsWlYankCm7PWEVetA6HKPJuTy2nh3sXFcuF5Iho7i+sIDNKg83hYHdXA3t7WkKShhOrDuezReVcmZ6FXJDROT7KM7WVVA3oQ5qmpJTL2ZSezoMVFZQmOAl949AQfz5dyf7OjpCm5MxHlCqMrdmZPLRxLVuzMolUKjFNT3Owo4v/PnaSj1vaGJla2L5nOaGUyylK0nHb6hIeumwdV+RlkRgdiQh0jYyxp6GV/3fwBG9XN9DYP+gm8ysFUeowilIT2FGezwNXruO+LWu4qjiHvCQt4WFKbA4H3cPjHGnp4rXjtTy19xivHa/hSEsXXUOjTNtW1vORIOFSwLnY733t9RazlQ93VfLRO6fp6x4OwTNyQiYT2Hrl2VTqfR8vQyr1plyKCpc5lbokDYDDJ0OfSr2maHlTqUuzk7liVTYAVS3LkEqdpmP7amcqdV3XMqRSJ2q5frlTqVctYyp1zNxU6reWI5W6xEmEpVRqCUvBilaKHQ4H//iP/8hPf/pT5HI5drudf/u3f+PRRx/1ek9hYSFf/OIX54x55513uPnmm5mcnCQ8fKHJlLfT42frniUiOoLuyX72DZwCYJN2NTmRC02vvPUjtjlkDFkm2NV7HJtoJz8qmasSyhC9tGrymp7tgD36Opom+pEJAjemlJMVFb8oV2qHKPJxXzPVI86T5fXx6WxJ8OyeGKiC7BBFDvV1cdTg3MxSIqK5JbuIqBlV1+tZQIAK74h5ig/aWug1OlXjjGgNO7Py0Kj8nIIFaZyFCP0mE592dNAz7mwTpZLL2Ziaxprk5LO9ff3F93pU61sxtDscNA4McqpHz5Bp0v1QdlwsFWkppGti3L+nkBlGBajYjk6aaeofpMkwxPCs3sdymUCWNpb8hHiy4+MIm99+4Xy5WHsaOtMzuHd4nJ6RcXpHJpianqskCwLooiNJiY0mOTaaFE00kWovLRk8zRuC9U+ZTPz93ZJSLOHSwbnY773t9X/45ZuEhYXzyYe1dHcOIZfL2HbDKlLStQsnXaSCLLlSe4k/6/pF5Uqd7qNd0yLnkFypLz5XarPRxKNXSunTKwXeLVpXAP7617/y7LPP8txzz1FWVkZlZSWPPPIIqampPPjggyGbR6VSoVJ5SMmdQXpEIqs0eZwZa+HESDUaZTTaME3A8eNV0VyTtIoP9KdpNuqJkKtYry0Kao0yQWBHcjF20UGrcZB3+6q5IaWMjAjPPRD9xbomJZ9opYpD/R2cGOpm3Grh2pSihaQviJhbUzNJjojmvY5G+iYneLbxNDdkFZIVHbuomLMRpw7n7uIyTuv1HOjppGtijGdqK7k8NZOKxBRkAbbjCQSJkZHcWVJC++goBzs7GZqaYn9XJ5UGPZelpVOSkBDS+WZDLpNRkpRIcWIC3WPjVPb00T48SvuI8yc+IpxVKckUJehQedpMlhGxEWo2ZqezISuNIdMUTYZBmgeGGZsy0zo4QuvgiJsg5+q0ZMfH+XSBPh8QBAFtVATaqAjKM5MRRZGxSTO9IxP0jozTNzLOhHmagXETA+MmqjqdXzBjwlUkx0aTpIkiKSaK+OiI4OqSJUiQ4BPnYr/3tdfL5TKu2lnqJsZ7d5/xTowXAZf5FkBb6wD7Pq737Eq9SLjMtwAamvTsP9gE4JEYLzb++lVOA9Gq+l4On2wDkQWu1EtBRaFTjT5W28WJOucBu0divEiUZc+o6dXtVLX0gYjXPsaLQWGq83f50Zlm6rr7EcGzK/Ui4TLf2l3VRLPB2QHEKzFeBFzmW7uqG9yKsTdivBi4zLfeqK53m295JcaLgMt869X6Wrdi7JUYLwIuxfjlxhq3YuyVGK8AhKvDCA/iQN8brNMr8/ktB1a0UpyRkcEPfvADvvGNb7ivPfHEEzzzzDPU19d7vOeqq65i3bp1/OpXv3Jf+5//+R8eeeQRxsbGAprXdcriUophprZ34BQ9UwOEy9Vcm7SFcPlZldKXUuxC40Qvn860atoQV8Cq2Byf42fDVSNsFx180FdLq3EQGQLbk0oWmG85xwdWa1w/2s8HvU4ThyR1NLdklBGpCJs13mMY77XGosCoZYpdbQ0MmE0ArEtIZUtSlmfCHWQtMA6BUYuZD9qb6TE6lVxdeATXZOaSGuXhBGsRSvGcYaJIw+Agh7q63GZcsWo1m9PSKIzXIQv6SDZ4hXRkcorTvXrqDAPYZlKalHIZxQk6ypOSSIjyYm4WYPyl1PaKosigcZLmgSGa+50E2X0/kKKJJlenJSc+Dk34XFX/fCjFgcSfmLKgH52gb+ZneGJywS1ymYAuOtJNkhM1UcSEqxAEQVKKJUhYBM7Hfu96H//hl2+6jbbsdodvxXiJtcZ+FeMl1hr7U4yXWmvsTzEORa2xT8U4BLXGPhXjENTp+u1jvMQ5/CrGS4zvTzFeaq2xX8V4ibXGfhXjJdYa+1WMV4hSLLVkCh4rWuqYnJxENo9IyeVyHD5qHS6//HL27Nkz59oHH3zA5ZdfvqS1CILAFt0qohWRTNnN7B88iV0MriaiMDqVjdp8AI6PNFE/3hX0OuSCjGtTSimITsSByB5DLfXji68LLo5N5I6sclQyBQbzBC+2nWLAbFx0PIBYVTj3Fq5idbzzVPbkQC8vNlcxbJ70c2eg8dXcXVjGjsw8VHIFg1OT/LWhmg/am5myWf0HCAIyQaAkIYEH11RwZWYm4QoFo2Yz77W08Jeq09QNDrjdo5cLceHhbMvL4Uub1nFlThax4Wqsdgdn9P08f/oML56upsbQf15qewVBICE6kstzM/mbTWu4d8MqNmSlER8ZgQj0jk2wv6WDvxyt5LljpznY2knv6Piyv2ZLQXS4ioIUHVeV5HDP5at5ePsGbllXzIbcNDLjNagUcuwOEcOYkapOPR9UN/PsgUr+e+9x3jxRx6HmTlr6h5kwW85Zm60LBb/97W/Jzs5GrVazefNmjh496nXsq6++yoYNG4iNjSUyMpKKigr+8pe/zBkjCILHn5/97GfuMdnZ2Qse//GPf7xsz1HC4rBS9nuXYpyepcNud7B395nQ1xhfscw1xhsv7BrjisI0NpYtc7umVU5RYllqjGe1a1qWGuMELTesLpTaNXmBSzGWaowlBIsVrRQ/9NBDfPjhh/z+97+nrKyMU6dO8ZWvfIUvfelL/OQnPwHg0Ucfpaenhz//+c/A2RYN3/jGN/jSl77ERx99xLe//e1FtWSarRS7MGIx82H/QaYdVjIjUrlMuwZBEAJSil04PtzM6dF2AK5OWEVuVIrP8bBQ4XWIIp/MuFIDXK7LZ3VsxqzHA3elBhgxm3mzq5rR6SmUgoxrU4vJj9EtSimejZaxYT7obGLKbkMuyLgqNZvV2uSzp6aLUIpnY9Jq5UBPBzVDzk1fLVewNS2Tcl2Sc44lKsUuuA4op+12Tuv1nNT3uT9kNSo1m1JTKdbNSqsOoVI8/7ooivSMjVOt76dlaNhNMJUyGYUJOkqTEkiOilqYsrUMSrGvOONTZtqGRmgbHKFndHzOwyqFnCxtLNnaODK1GsKVSv/rDHBeF5ZLiXalXPePmzCMGTGMTTA4MemR6IcrFSTERKKLjiQxOpKEmEii1aq5v5tLRCl+8cUXeeCBB/jd737H5s2b+dWvfsVLL71EQ0MDiYkLUzz37t3LyMgIxcXFhIWFsWvXLv7+7/9+zme5Xq+fc8+7777Lww8/THNzM7m5zlTS7OxsHn74Yf72b//WPS46OprIyAAzLCScE5yP/d6TUuyCTYRPPqimp2OeYrxEpdgFyZXa3/ovbFdqgIY+yZXa17ySK7WkFK80rGhSPDExwWOPPcZrr71Gf38/qamp3HfffTz++OOEhTlTFR566CHa29vZu3ev+769e/fyne98h9raWtLT03nsscd46KGHAp7XFym2OeQYzEPsGziKiEhZTD7lmsKgSLEoihwcbKR+ogsBgR1JFe7a4EBJsSvOgYFWqkZnNg1tNuvjnKpIsKRYdAiY7Vbe7amjyzQKwMb4DDbpsj3WwwRKigGM1mne62ii0+iMmxsdx86MfCIUYUsmxS70Gsf5qLOVwSmnGp0YEcm2jBxSI738AS+SFLswbbdTZdBzsq+PKTc5VrEhxUmOFYL3GpmlkuLZa5qcnqa2f4AawwBj5rOpy9qIcMoSEylK1BHhIpznmBTPvm622ugYHqVjeISO4TEss05tBSApJoqsuFgytbEkRkUGV7N9noy8ZsPucDBsnKJ/3MjAuIn+MSNDpkk8fbqqFHJ00ZHooiPQRUWii4ogLjJ8TnracpLiUJhvLMZ4Y/PmzWzcuJHf/OY3gNNYKSMjg29961v84Ac/CCjGunXruPnmm/nhD3/o8fE77riDiYmJOephdnY2jzzyCI888khAc0g4Pzgf+70vUizKZM5U6vnEOMNLjXGQpBhhJpV6PjHO8ewTEiwpRjaTSj2PGOflLyy3cq3H8/q9xBeEs6nUs4hxibca40WQYvBAjEsyPI9fJFFbQIy91BgvhhR7bdcUqnZQArT1eyDG8hARe2EmlXoeMZYpvLzYiyDj+nEPqdTzzTrdcYLb7EVhJpV6PjH25nUSJCkGL8Q4zEsNrkSKVzxWNCk+X3C9AV6s/8sCUmyd6UfcPNHF0eEaAC6LX0VmhOcParuXT2qrXWDfQA0tRr0zJTppDWkR8V7JrDeybHfAieEOjg61A7AqNo0rEvIRvWTGO7y6VQsz/4p8amjj1JAzJTs7Ko7r04pRyed+iIhe4ngjyw4HnBroY39fO3ZRJFyhZGd6HnmaeI/jvX6K+nCxdogip/v7ONTb5U4lLtLquCIti+j5vY2DdY32Mn7aZueMwcDJ3l43OY5UKlmXkkp5YqLz1DNQeCWn/tckiiK94xPU6PtpHhzCPiPxywSn8VVJYgLZcbEeDTO8kt9l6jvscIgYxo20DY3QMTQ6x2UbQK1QkKHVkKmNJTNOQ6TKj8lDqEh9kPG9wfXbstkdDE1MMjBhYnDGuGvIi6IsEwTioyK4c3MZcpmMKZOJf/jM8pDiy2/4P0vuXWizmjm0+3G6urrmrM+bkdH09DQRERG8/PLL3HHHHe7rDz74IKOjo7zxxhs+5xNFkY8++ojbbruN119/nWuvvXbBGIPBQHp6Ok8//TT333+/+3p2djZmsxmr1UpmZib3338/3/nOd1CEyHxFwoULNyn+xUJS7PpD9ltjPG/8wuu+ybLkSu0l/qzrF6Qr9ax7JFdq3/Neyq7UZqOJR6++XiLFKwQruqZ4JSM/OoPSGGd63pGhagzm4GqCBEHgqoRSsiISnOZZhtP0TQVftyQIAhvis7kiwVmrfGa0h/f7ahdd/yETBK5OzuX6tELkgox24wgvtlUyNGOatVgIgsC6xFTuK1yDTh3BlM3KW+317O5sxByiOmCZILA2KZUHy9dSrnNu+g3Dgzxdc4ojfV1YHaGvuQ2Ty1mfmspDa9dyZVYWkWFhmKxWPu3s4H8qT3G4u4tJa2jrnD1BEATSNDFcV5TPw5vXsy0vm8SoSByiSNvQCO/UNfLfR0+yr6UN/YTxvNa6ymQCKbHRbMnL5L5Nq3nw8rVsL8olL0FLmFyO2WajqX+IPfUt/M+hkzx39DSfNrfTNjjC9DL2jA41FHIZSbFRlGcksa0sl89etoq/3bGRz162iu1luazKTCY1LpowhRyHKGKx2S44R+uMjIw5fV+ffPJJj+MGBwex2+0kJc1VqZKSkhakQM/G2NgYUVFRhIWFcfPNN/PrX//aIyEGePrpp4mOjubOO++cc/3b3/42L7zwAh9//DFf/epX+dGPfsT3v//9IJ+phEsV7hrjzPhzVmPc0b4MNcYFy1xjXJwKzNQYNy5DjfEy9jEuy07mivJsYKbGuHYZa4y7+9lXvRw1xsvcx/giqDEOl2qMJfiBpBR7QCBKMcykLw+epnNSj1JQsC3xMuLC5p6ieFOKXcqvXXTwof403VNDyAUZO5PWkhq+UD0NJK26adzAHn09DkRS1BquT12FWq6cO96PUjwb/VNG3uqqY8JqQSHIuCa1gGKNk2wGqxTPvm5zODis7+J4fzciEKFQck1aHvmxs573IpTiBes3Gdnb2UavydnbOEoZxmWpGZTGJwbvGh2gmmpzOGgYHOR4Ty9jFmc6s3zGrKsiKZn4iAgPQXzPEYhS7A1Dpknq+wdo6B/ENKsfr0atokAXT0GCDl14uOdT62VSin3B7nBgGDPSOTxGx/AoA8a5hzECkBgTRVpsDGmxMaTERHtV48+3UhxoHFEUmZiyMDltJTk2GmBZleJQpk8HqhT39vaSlpbGwYMH55ggff/732ffvn0cOXLE4zwOh4PW1laMRiN79uzhhz/8Ia+//jrbtm1bMLa4uJhrr72WX//61z7X/t///d989atfxWg0+mzFJ+HiRyBKsQuSK/X89S+ML7lS+55DcqX2Pe+l6EotKcUrCxeWLLHCIAgCl+tWkaiKwyra+HTgKEZbcIqqXJCxI2k16eHxToJsOEX35OJOiQtikrg5bRVhMjl95jFe6zrB2PTiHZ8Tw6O4J6eCjMhYbKKD93sa2NPbtGTFVSGTcUVqFvcUrEarCmfSZmVXRz272utDerqWGBnFZ4vKuTGnkOgwFUbrNB92tPCX2kqaR4aWRS1VyGSUJSbywJo13JhfQGJkJHZRpLq/n2fOVPFaXR1toyPnTKmNj4xga04WD21ax21lxRQmxKOQyRgzWzje3cvzp6p45lQVRzq7GZmcOidr8gW5TEZqbAyX5WZwz4ZVPLx1PTeUFlCakogmXI0IGMaNnOzs5a2qep7af4yXTlZzsLWTjuHRC0pJdkEQBGIi1G5CvNxw9S5c6g9ATEzMnB9vJFOn0yGXyzEYDHOuGwwGkpOTva5VJpORn59PRUUFf//3f8/dd9/tUY3+9NNPaWho4Mtf/rLf579582ZsNhvt7e1+x0qQ4ILkSu0/vuRK7RuSK7VvSK7UFwc++eQTbr31VlJTUxEEgddff33O40ajkW9+85ukp6cTHh5OaWkpv/vd73zGfOqpp7jyyiuJi4sjLi6OnTt3Luhe8dBDDy3oNHHDDTcEtXaJFC8RckHOVYnriFVGY3ZMs6//GFN2S1AxFDI5O5PXkBmhwy462GM4Refk4jarjEgtd2asI0qhYsw6xWvdJ9FPjS0qFkCEIozbM8vZpHNuRDWjel5oPcXgEtOpAVIio7m/sIJNiekIQPPYEE/Xn+T0YF/INgpBECjS6niwbC1XpWejlisYMTv7KL/QcIauicW/Nr4gEwQK4+O5t6ycu0tKyY/TIgCd42O82dDAn6tOc1qvP2dtlGSCQFZcLNcXFfDlzeu5viifXG0cMkFgZGqKI13d/OXUaZ47VcXRrm6GVwBBBghXKslPjOeaoly+sLmCBy9by46iXIqTE4hWq5wkecLIya5e3jpTz1MHjvPiiSo+aW6neWCIyelLb0NaiQgLC2P9+vVzDLAcDgd79uwJqn2Ow+HAYln4+frHP/6R9evXs2bNGr8xKisrkclkHh2vJUjwBblcxpXXlpGWtYyp1FcuMzHedGET4zXLnEpdmp3EFauygeUjxttXLycxjuP65U6lXnVuiXEovyfpIiK5s0QixssJk8nEmjVr+O1vf+vx8e9+97vs3r2bZ555hrq6Oh555BG++c1v8uabb3qNuXfvXu677z4+/vhjDh06REZGBtdddx09PT1zxt1www309fW5f55//vmg1i6lT3tAoOnTszFhtfKR4RAm+ySxyhi2JW4mTKb0mz49G3bRwceGajom+xEQ2Ja4muzIJK/jwbubtNE6zbu9ZxiwTCAXZGxLLKYgJimo9On517tMo7zf04DJNo1ckHFlYi7lsSlz0n8CSZ+ee935b/+UkT1dLRimnD2SkyOi2ZGWT8L8lLYg0qed1+f+r8Vu46Shl5OGXqwzH+RZMbFsSckkKTJqyenTLnjK5hm3mDmtN1A9cLancJhMTnGCjtWJScSHe06tXkr6tL/xFpuNtqERGgeH6Bodm2MCpQ0PJz9eS358PPERXlKsA1hPyFysPWDcbKF3ZJyesXF6RscZNy8kTLHhalJiomd+oogL5Lkscj0QGtfr5UyfPp8tmR588EF+//vfs2nTJn71q1/x17/+lfr6epKSknjggQdIS0tzK8FPPvkkGzZsIC8vD4vFwjvvvMMPfvAD/uu//muOIjw+Pk5KSgq/+MUv+NrXvjZnzkOHDnHkyBG2b99OdHQ0hw4d4jvf+Q433ngjTz/99JJeAwkXPoJJn3ZBcqV2rd9LfMmV2v8ckiu133kvFVfq5UyfDkWplCtesN0mXBAEgddee22OwWZ5eTn33HMPjz32mPva+vXrufHGG3niiScCimu324mLi+M3v/kNDzzwAOBUikdHRxco08FAst/0AbngQC7MZWEOL38d0Uol1yRt4AP9YUat4xwYPMb2xA3IBW/W7AsvKYBrksrZ119Dq8nA3v4qrkooJT86BW+fNXYvnypRyjDuyKjgw75a2kxD7DHUMmI1slGb65kQeCObs4ZmRWv4fH4F73c30W4cYa+hme7JEXamFrrdqb0eSHo7e5lZf1JkJPcWreL0oJ6DvZ3oJyd4rukU6xPTuCwpw11X4vUjKEDSqpLJuTw9g9WJyRzt7ebMoIGO8VE6xkfJjY3jspRMEiM89C8N0q3a0/DoCDVX5GaxKTOduoEBqvr0jJjNVBkMVBkMpMfEsDopidy4uLl1Ol5fu+DW5OnXrgpTUJycQHFyAlNWK21DIzQPDtM1Osbw1BRHu3s42t2DRq0mTxdHbrzWcw9kvJPfpf7O3PE9XIuJUBETnkBxqvMLo9EyTd/oOL1jE/SNTTBonGR0yszolJk6w4DzOSvkJMdEk6KJJjkmisToKMK8bLbBrge8v9V9te5YEOMizOG55557GBgY4PHHH0ev11NRUcHu3bvd5ludnZ3IZr3vTSYTf/d3f0d3dzfh4eEUFxfzzDPPcM8998yJ+8ILLyCKIvfdd9+COVUqFS+88AL/8i//gsViIScnh+985zt897vfXd4nK+HCgih6+MP18jnqcKAQ4Oodpe4a4727z3h3pfbyASJ42XNFmYBMcKZSA7S1DrDv43quBo+u1ILXzwove5MDBAQu25ALIjQ06dl/sAnAsyu119fB2/qdd6wvzwBRpKq+l8Mn2wA8u1IHtfqzqCh0qtHHars4UedUiz25Ugve/E/88LeyrGQQYX91O1UtTrXbkyu1r3V6fW4OKErRIYjw0Zlm6rr6QcSrK7XXvcZbfCB3xnxrd1UTzYYhAK+u1MHGB0iPdaZS76pucCvGwbpS+5rXpRi/UV3vVoy9uVIL3t7r3v72gITwSO4qLuXV+lq3YuzVldobvLX2lIluxfjlxhq3YuzRlXoZZcm77v8tCsXSOk0A2GxOf5zx8fE51715iPjDli1bePPNN/nSl75Eamoqe/fupbGxkX//938POMbk5CRWqxWtdu5n7t69e0lMTCQuLo5rrrmGJ554gvh4L11uPEBSij3AdcrycsPTASvFLkV4ZHqcDw1HsTpsJKvj2arbgNxDz1pfCrJDFNk/UEuT0flBfHl8EcUxWV7m9fxHaZ9RlkVR5PBgG6dGOgHIjtRxTXIJYbK5f/heFWQPHzaiKHJysJcDhnYciEQrVVyfWkRapMZ7H2SvJkMLxxunLXzc3U7zmPODPCZMxba0HHJjtP6Pjxdc93zZNX7MYuZwbxf1QwPuofmxWjanZJAwmxwH28LJF2ZiiaJI19g4Zwx6WodH3FNEKpWUJSZSlpBAjFodNHn0Sor9rGc2XApy8+AwnSOj2Gf9AiOUSnK0ceTEx5GuiXFvUqHsa+wJi1FgLVYbfeMT6MeM9I1NYBg3YvOQ6qWNDCc5Jpqk6EiSYqLQRkSc/fKw3H2QPWDKZOJ7t19cSrEECSsNbqX45294UIr9G08F1K7JqxrmX0Geb7511bZisrLnEeMlGHAF0q5pKQZcC8y31uZQOk8xXqoBl992TUs04JpjvpXrQTH28RUgkOc223yrOD2Rqz0R4yUYZAXUrmkJ8eebb91YVohCPvfFXYoBV0DtmrzG8b95B9SuKVglfZaC7M98y2w08ei25VGKt+74l5CR4gN7/mXB9X/+53/mX/5l4fXZ8KQUWywWvvKVr/DnP/8ZhUKBTCbjqaeeciu+geDv/u7veO+996ipqUGtdj7HF154gYiICHJycmhpaeEf//EfiYqK4tChQ8gDbI8qkWIPWAopBhiwjPCx4Tg20U6KOoEtunULiLG/tGpRFDk81EjtuPPDfrUmh3Vx+Qs+LP2RYhcaxvXsNTRgF0XiwiK4PmUVsWFnn1swpBhAdAjopybY3V3PmNV5irRWm8ZlCdkoZAtfo2BIsfMBgdaxYT7qbmXC6kyJzYzScFVqDjq1JyV3caTYheGpSQ73ddM4fLZ+K1ejZVNKGsmR0ctCimdjwmKh2tBPdX8/U7NaOGXExFCakEi+Voti/gnsMpLi2Zi22ekYGaV1aJj24dE59T1ymUC6RkNWXCw5cbFo1EF8AJ8DUjwfdoeDIeMkfWNG9GMT6CeMTHhIuVbIZMRHRZAYHUliVBSJ0ZHERYQjm/X3J5FiiRRLuLCxVFIMkiv1wvUvjH8pulJ7W6uney42V+r5xFhypfZOjJeTFIc6fTrQbhOz4YkU//znP+epp57i5z//OVlZWXzyySc8+uijvPbaa+zcudPven784x/z05/+lL1797J69Wqv41pbW8nLy+PDDz9kx44d/p8oEin2iKWSYgCDeYi9/Sewiw6PxDiQWmNRFKkcbePkSCsAeVEpXKErQzYrXypQUgxgmBpnd18NJpsFpUzONUkl5EQ5004XQ4rBWaP7iaGV2lGnq2xsWDjXphaRHD73D3ExpBjAardzxNDNyYEe7KIzLWtVfDKXJ2USrlAuGL8wjufL3sYPTpo40tdN08iQ+1pGtIYNSWlkRmsWnuCGiBSDU2m1Oxw0Dw9TMzBA19iY+7EwuZyieB2lCQkkRUY613GOSPFs2B0OekbHaR0eoW14BKNlroFEXLiarLhYsmJjSdPELCTygcy73MrsvPEmyzSGCSP6cSP940b6J0wejT0UMhm6qAgSoiJJiIokMSoSbWT4wnQxiRRLkHBBIBSkGGaI8Z46ujsGFxLjJZJi8EOMl0iKYYYYH59bY+wixkslxa74J6q75tQYu4hxKEgxQGVTD8dqPBDjEJBigJoOA/vPONPA5xDjEJBi8EOMl0haAdoGRthd1eiZGIcgvi9ivFRSDH6I8RJJMTi/+71SP6/G2EWMl0iKwTsxXk5SvBJaMs0nxVNTU2g0Gl577TVuvvlm97gvf/nLdHd3s3v3bp/xfv7zn/PEE0/w4YcfsmHDBr/zJyQk8MQTT/DVr341sPVKpHgh5pNig9lAhDyCaGV0wKQYQD81xL4Bz8Q4GAOuhvEeDgzWIyKSFh7P9sQ1KGfSn4MhxQBGq5UP9NX0zThSr4nLYFN8LoKX5+WPFLvQNjHMnr4mTLZpBGBtfDqbddluUrRYUuzCmMXMJ71nU6pVcjmbEzNZE5/sJCUhIsWu8cP/P3vvHSbJVd77f6pzTpPzbM7aVdhd5bhCEjJIGNtCcI1A4AhcC2yC7r1GwuCfCDbIBmywL5Z8wQIkm2ALLEBIi0A57Gpznp2cejqH6li/P2q6Z3q6emZ6pmd3dvd8n6efnjl96q23e3rq9KfPOd83meDVkUGOBvzkJxsbrHa2N7ex2lM3NWNYYyierogsc8jv59DYGNFpLspei4X19Q2sr6vHpfUt3RJC8fT+iqIQSCQ5HQzRGwgyFImWhNLrJNpcLro8Hjq9bnwz6yEvEygue1hRCCVlxqJxxqNxxqIxxmNxMrnyZdc6ScJnt9LgsFNvt1HvsFFvs1c27JinBBQLCS29agXFAFmFcvOtdl9NoBggr2iYb3XV1wSK1fZy863VqxprAsWgXnZnmm9tXNNSMyhWJA3zrfXtNYNiRdIw39rQWdEca7ZcK53j6LCG+dYs4F1t/FPjGuZbutrFHwiVm2/pdbqaQDGUg/HbJl2qawHFAONJDfMtg6EmUAylYLzWW89bV6694KC4EOunP/0pt912W7HfH/3RH9HT08PPf/7zirG++MUv8td//df87Gc/4/LLL5/z3AMDA3R2dvKjH/2It7/97fPLV0BxuaZDccQY4ZXAK5h1Zq6pvwajXruWaCXIHUwE+I3/NXJKnlZLI1fUX4xe0lcFxQC98Ql2j71JVslTZ3Kyq+kSbAZz1VCcVyRySp6X/CfZH1LNKZotbm5s2oRDY+/BfKEYQM5l2D18iqMRtcSDz2Tjpta1NFtdi4bigvojYX411MP4ZEkot8nCVc1drHHVaxuILRCKC4qkU7wxOsQB/2hxL6rHbOHixlY21jVglBYAP1W6NCt5hf5IhEPjY5wMBkv2xLY7XWyor2eVz1c0OztTUDxT6UyWvlCY3mCI3lCIeDpT8rjDZKLT46bD46bD7cI203BijvhnCoq1+k8HZX8szvjkLZXVLhXhNJuot9upd9ios6s3j9VSsvx6NgkoFhJaetUSioUrdSH/CvEvMFfq2XIVrtRz51mp//ngSv2rgdO8pXs1VoPxvITiWCzGiRMnALj44ov58pe/zA033IDP56Ozs5Prr78ev9/P1772Nbq6uvjVr37Fn/zJn/DlL3+ZP/mTPwEoq0bxhS98gU9/+tM89thjXHXVVcVzORwOHA4HsViMz3zmM7zzne+kubmZkydP8olPfIJoNMr+/fvnbQgmoFhDhTfAj449gsFmYLf/eaLZGE6DnavqrsGsL39xZ5tBLp0xrueahosBY8X+ldrH5TA/G96LnM9gN5i5pfli3EbtN2glWJ5uhHUyOs4zI0dJ53NY9EZuatpAh73Upa3SuyNfIc+8InEy4ueXQydI5lQo2uZr5fLGbkxae40rOUPOUtoprygcDIzywnAfiax6jmabg2tauml3uOeVf7Uzy8lMlr2jw7w5NoKcU2vaWfQGLmpoZmtjc7mj4GxaRE6pbJaTgQCHx/0MTnMC1EsSK7xe1tXX0+3xaC9bPoOGXYVZ5N5giL5gmKFwpMSsC6DOZqPD46LD46bV5ZrdAZoaGnlV2b/S81XyClE5xXgsgT8Wxx9L4I8lNPcog/o38tmt+OwFUFZ/dphNZR+qkvE4n3ibgGIhoaVUEYq/+MNyKK609WMOWJ6X+da0/vONX4DlOfcYF/Nf2AzyfMy3ZoszFyzPtcd4Zj7zjj+tfU7zLVjUDPKse4yrzFWrf8lS6vbGiq7UC40/L/OtRcSfuZS6Wlfquc47L/OtWePMPtjPy3xrlvjVwPL5CMW7d+/mhhtuKGu/5557ePTRRxkZGeH+++/n5z//OYFAgK6uLv7wD/+Qj370o8X3+fXXX093dzePPvooAN3d3fT29pbFLJh9JZNJ7rzzTvbs2UMoFKK1tZW3vOUtfPazny1WuJiPBBRraDoU2502Etkkz/p/QzIn4zF6uLLuSoy6Uqida1m1CsZvkFNyNJl9XFl/WXEJtFb/Su2RTIKfDe8lnElglPRc33gR7bbyb4rnA8UAoXSSnw0dwp9S6wNf4u3isrru4r7lhUAxQDKb4bmRUxwJq7PGTqOZG1pW0+0o/XCwECguKJ3L8cb4IK+NDRZrDq90+bi6pYs6i23W/Be63Dqdy3HIP8aesSHCKRV+dJLEOl89Fze2apdzqvIc8+0fkVMc9fs5Mj5OUJaL7Sa9ntU+H2vr6mh3uaYGo7PgYl1QJpdjKBylPxSmPxTGH0+UxWxyOmhzu2hzu2hxOssgeblBcaX+qUwWf1wF5YlYkol4golYQtP1GsCo1+OzW6mzWbl+7Up0OklAsZDQGdBSQDEIV2rt/Evjn/eu1FXkqtX/fHOl1gLjC9mVuqDzEYrPZQko1tBMKAaIZKLs9j9POp+hzlTH5b7LMUyD2vnsNR6TA+wee52sksNncnNN/XbMelPF/pXaU7kMT4/sY1gOIgGX+tay2dVVcsGcLxQDZPN5fj16kkMRdYBqtri5sXkDLqN1wVBc0OlogGeGTxQdpFe76rm2aSUOozrbvhgoLiieSfPSSD/7J0aKnLLeU8/Opg68ZltZfzXQwqC4oLyicCoU4I2RIYbi0WJ7i93J1oZmVnvrKptM1QzUJ+8UhfFEgmN+P0cnJohP239sMRhY6fWyyuej0+Wucga5dlA8U8l0hoFQpAjJkVTp7KoENDjsKiC7nLS6nNgM2qsrKuosQbFmV0UhkkwxEUvgjycIxBME4klCSZn85D+ZzWTk3isvBZZ2pnhkZPGOlJFIhObmhgtikBQ6f7VUUAzClbo8//L457UrdZW5avU/31ypZ4Lxhe5KDQKKl5sEFGtIC4oBAukQz/lfJKtkaTA3sNO3s2icNV8DLn8qxO6x10nnM7gMDq5t2I7NYK3Yv1J7TsnzwvgRjkbVAWWVvYUr6zcWyyFVA8Wg7kE+Hh3lubGjZPI5jDo9VzWsYa2jWXPZznyhGNTZ1RfHTvNmYAgFMOr07GzoZKuvFV2F160aKFYfgICc4PmRvqIZlwSs8zRMwrF1Rv/FQfH0/sPxKHtGhzkRnCiaclkNBjbVNbGloQm32VJ2TLXnmG+7oigMR6Mc8fs5GQiQzGaLj5l0erq9Hlb7fHS7PVODx1mA4pn9I3KKwXCYgXCEoXC0DJIBPBYLrZOA3OJy4rFYKu7lWmw+07UUz7egXD5PKCETSCTI5vJsaFE/dC4lFF97zacXXbswm5V57td/dUEMkkLnr5YSikG4Upfmrx3/vHWlXkCuF4Ir9XQwFq7UAoqXmwQUa6gSFAOMyEFeDLxITsnRbG5mu287OklXlSt1OBPjmdHXSOZkbHoL1zbswGV0VOxfqV1RFA6GB3l54igKCvUmFzc0bcNhsCwIigEimSTPjBxmRA4D0G2v59rGdVhnzGhXA8VqrjAux3hm+ASjSXVm1Wuycm3TajrtXo3+1UNxQWOJGC+O9nMqEgDU69N6bwM7GqfBcY3dqkGdsT7gH2X/+CixzNRsbbfLw5aGZla4varR0hJCMUxdj/OKwmAkwslAgBOBAPFp9Y/1kkSn281Kr48VHo/2nuizZNgFat3moXCUoUiE4UiUiUSyrI/FYKDZ6aDF6aTF5aDJ4Sj9lvgcgOJKElAsJLT0mgnF6VSWWDiBr9FVEygG4Uo9lX+Fds5TV+oaukYLV+rZz3suu1ILKF5eElCsodmgOKPoGE+N89LES+TJ02xpZrt3O7kqjbPC6RTPjb9CNBvHrDNxdf1l1Jk9VUExqDA7nAzw7NibpPIZzDoj1zdeRJNFw3iDuaFY7aPwZrCPVyd6yKNg1Ru5rnE9Xfb6aX2qh2L1XuFQaJQXxk4XjbhWOeq5unElLpNlWv+FQ3FBo4kYL4700RMNAlMzx9sb26kzV9j7WwNgzSsKp8IB9o2P0hcJFdvtRiMb6xrZXKcxe1zlOWZr14qiKAoj0Rgngiogz5yJbXE4WOn1sdLjxVuYgT2LUDxTcibLSCTK0ORtLBYrM+6SgDq7jWaHg2anCsllZaAWkM/5BsVi+bSQkKrpUGzQm3nmx3sITcS58Y5tNLRV5xotXKmn8tHOv0L889WVepZyTcKVWrhSFySgeHlJQLGGCm+A/zr+rTIoLhpnyWO8MPEqefK0WJq4zLujaE41XbPNIMu5NLvHXiOQjqCXdFxVv40WS7Nm/+wcsBzNJHl6ZB8T6ai6z9i7mi3u7jIgqAzd5f/F43KMp4ePEEir5Y82uFq4qnEVJp2hIvzmK+wRnlnaSc5leWmslzcnhlEAvaTjsvp2Lq1rx6jTzwLFms2zQvRIIspLI/30RILF9tXuOnY0ttNkc8wr/kJnloNykgPjoxyaGCtZytzhdLO5oYlVHt/UPt9qjbAqaY5cFUVhIpHkVDDAqUCQsXi8pJvLbGaF18sKj4c2l6t8H/IS5zkfCM3l84zH4oxEYwxHooxEYsSm7aUuyKTX0+i00+R00Ohw0OS04zCWuz3PprNh8JWMx/nk7cJoS0hoKVWE4i/8AJPewrM/eZPRwRBGo54b7thGQ4un/CDhSg0IV2qt887LfGueuWr1F67Us5/3XHSllmNx7r/xLct2rF+KeMtZAoo1NB8ohlIwbjY3c5nvsuIe44LmWladyWf5zfhehmU/EnCxZxNrnF1l/eeCYoBsPsfz/iMcjw4D0GVr5JqGjZimOWVXA8UAmVyelyd6eDOo1jR2GMxc37SOdludZv/5QnFB44kEvxo5yUAiPBnfxFWNK1jratS82C8EigsaSUR5dXSAE+FAsa3b6WF7YzttdheSJNUcigvK5fOcCgXY7x8rmT026w2s99Wzoa6BJpujulrLlVRlrlE5RU8wyKlgkIFIpGj8BGDU6eh0u+n2eul0u3GZzcsCirUUS6UYicYYjcYYicQYi8U1HZ9tRiONDhWUGyZrCTtMlUFZQPHSxhISOluaDsU2i51sJlcEY4NRr84YzwRj4UotXKlnOa9wpZ49vnClLtVSQnEtVoUV4l0oK8MEFGtovlAMMCqP8/zEK+TJ02huZLt3e9Wu1Hklz6uBQ5yMqeC5xtHNNs8Gdf/ppOYDxaAOKkejg7wwfpQ8Ck6DlRsat1Bvdmv2n4ozO1QOJoI8O3qUSEYt/bPW2cQVDavL9xpXCcVKXl06dTzi5zdjPUWX6nqznasaV9Bp95Zc8BcDxQX5kwleHR3gaGi8yCNNVgeXNLSy2l2HXmPGv2Z7kIFwSuaQf4yD/rGSvcdes4X1dQ2s9zWULq9eYiie3p7O5egPh+kJBjkdCpGYtg8ZwGux0On20Ol20+5yqft2apznQqF4pvKKwkQ8wWg0xmgszlg0xkQ8oRnGYjDQ4LDTYLdRb7fT4LDhsVrRSZKA4iWOJSR0tjQTioG5wVi4UgtX6jnOK1ypZ48vXKmntJRQfN0V/2fR/iGgeoj86sXPXRDjvYBiDVUDxaCC8QuBV8kpOepMdez07SzWMZ6vAZeiKByMnGJf6DgALZYGrqjbVowzXygu5pSM8OzYPmJZGR0Sl/pWs9nVRZ5K+cwNlZl8lpf8PewPqft+LHojV9avZo2zqXhBXggUF5TN59gTGOI1fz/pfA6ADpuHqxpX0Gh1TuajGWZBe5BDqSSvjQ9yKDBW3J/qMJrYVtfCZl9z6UWuhlBcUF5R6I+EOTQxxslQoGRWs8XuZJ2vnjXeOuwGDROs2VSjvckoCmPxOD2hEH2hECOxWElXnSTR7HDQ4XLT4XbRbHNoLoWqNs9aQbGWsjl12fVYLM5oNIY/HieQSGqG1usk6mw26m026u2TN5u98j6iheYpoFhI6KxIC4phEox/uo/RgWA5GAtXauFKPY/zHuod5dfClbpif+FKrUpA8fKSgGINVQvFAKOpEC9OqOWaPEYPl/sux6w3V+VKDXA6NsrLgTfJKXlcBgfXNFyGw2CrGorzeYlULsNv/IfoTYwB0GrxcVX9RdgMZo0484fK0WSEZ0ePFvcat9u8XNOwFrfJtigoLiiZzfCKv599waHiUt7Vznp2NnThM2kbZC3GmCuRSbNvYoQ3J0ZIZNWZUaNOx0ZvE1vrmvFZbEsCxdOVzuU4EZzgyMQ4fdFwsV0C2p1u1nnrWe2p096/Mt9zL9KwK5XN0h+J0BcK0Rsury1s1OlodbrodLlod7mpt9lKVjvMN8+lhGKtONlcnolEgvF4HH9MvZ+IJ8hoLL0GsJuM1NlsKjDb1Xuf1YpBP8fe8HnmAwKKhYTOhCpBMahbh3Y/+WY5GAtX6sl24Uo923kBDvYKV+rZ4gtXarF8erlJQLGGFgLFGUVHKB3ixcCLpPNpHAYHV/iuwKh3avafzU06kA7x6/HXkfMpTDojV9RdTL253NBitjgFOC0sp345cJScksesM3Jl/WY6baXxqoFigExedah+PdBLTsmjl3Rc7O3kIndXsVZyST5VQHHhvJG0zIvjpzkaUaFeAta6GtlZ34XbZC3rX03+WiCSzec5Ehxnj38Iv5wotnc6PGyra6Hb6SuHvBpB8fRYsXSaY0E/x4J+RuKx4sM6SaLL6WGNt46Vbl9lQF5CF+vp/UOyTH8kTH84TH8kgjzNSAzArNfT5nTR5nLR7nSVQ/IygWLNLopCWE7hj8eZiCXwJxL44wnN+skFuS1mfDYbdVYrvklQLoHlKvIRUCwktPSaDYoVnUQ2kysHY+FKrUq4Us963sI5hCt15fjClVoYbS03CSjWUOEN8NTJf8LuLIWvTF77zV6A00gmyq8nXiKZk7HqLFxedwUuY/mbqBLMFmaEE1mZ58b3MpEOIwFbPetZ6yg3VKgEszPjB9Nxnh09wERahaz1rnZ2+NYWAbYSPM5V7ziUTvDc6An6E6qzs8to4erGNXTZS424Ks0gVzrvdHdrvxznxbFeTkZVgywdEhs9Texo6MRpNM8ap+rl1qgDfX8szJ6xYU5Nc6x2mcxsrW9mk68Jq8E4a/xZobgKOA3LMkcDExwN+JlIToG6TpLocs0DkOejGpiLKYqCP5GgPxxhIBxmMBolk8uV9DHr9bQ4nbS5XLQ6nTTZ7drLrZd8D+/CYTydzTKRSDIRTzCRSBKIq7A88wuB6XJZzPhsVryTkOy1qbfZ/mZyPM4nbxVQLCS0lCpC8ef/owyKCxA6fY+xcKWe3j55L1ypK2vyEOFKPXv8C9mVWo7Fuf8mAcXLRQKKNbQYKAZI5JL8xv8SkWwMo2Rkp28ndea6iv2na/oy6ZyS45WJQ5yKq4NBl62Vy7xbSmZi5wvFalueVyZOcSDcC4DHaOfaxk3Um90LhmJQB62TMT+/GTtBPKsaR61w1HNVw2qcRnU/w2KguKDRZJQXRvvojaugqpckNnmaubS+HafBWtZfja/ZPCsUT1c4JfOmf4QDE2OkctnJ8+pY56lnS30TzVZnBcfo2kDx9FgTyQTHAn6OBycIyMniwzpJosPpZpXHx0q3D4ep2j3Is5933v2nKa8ojMfjDIQjDEYimpBs0OlodjhoczppcTppdjgwGwzLGoo1H1YUkpksgcQUKAcSSQKJ5KywbDUa8FqteKxWvFYLHquFFT7VWE5AsZDQ0ms+UAzzMN8C4UotXKlnPbdwpZ49/oXqSi2geHlJQLGGFgvFAOl8mt9MvEIgHUSHjku9l9Jqba3Yv6CZe4fV5c99vBE8ioKC2+jkqrpLcBrtk3HmD8WF9sHEBM+NHyCZSyMhscXTxVb3Ks1lz/OB4qnnnOUVfy/7ggPF2sNbve1c7OvEgLE8CNVBcaH/YCLMi2O9DE6WcdIhsd7dyKV1HXjNthn9NcPMG4oLyuTyHA362esfZjw5VdfXZ7ayua6J9d5GbIZpz3EJoHi6JpIJjgcmOB6cYGLaUm+AJpuDlR4vqzw+6iy2uWvyLgEUz+xfgOTBSIShaJShaFQTGOusVpodDpodTlocDnxW68Lzr9i/tlBc8bBJWA4mkgSSSQLxJMGkCsvxCjWV//DyywQUCwmdIc0XikG4Upe3z/hduFLPeW7hSj17/AvRlVpA8fKSgGINzYTiwWQPNr0Dr6lh3lAMkM1neSmwh5GUOkhsdG1ktX01kiTNG4oLGkqEeGFiD6l8GqNkYLtvCx22lgVBMYCcS/Oi/wg98VEA3EYbV9VvpsnimdG/OmjNKxITqRi/GTvOUFKFVqveyGW+lax3NaOTZkJ/9VCs3isMJMK8Mt5XrHEMqiHXZfWdNFock/00w1QNxUw773Aiyn7/KMdCfrKKasakkyRWuXxs9DXS5fSiY5aRsgZQPL1/QE5yMjTByVCgZA8ygNNkZoXbwwq3l3anG6PGFx9nAorLmhSFYDLJ4CQgj0SjhDX265r0eprsdpodDprsDpodDuwzZ8KXKRTPFiedzRGSkwQTMsGkCst6Scdb1q0GxPJpIaEzoWqgGIQrdWm7RptwpZ7z3MKVevb4F5ortYDi5SUBxRqaDsWyKcye0K/RoeMiz5W4jdomDJUgNJ2X2B/ZT09cvQh22jrZ6t6KgjZcV4LibF5PIivz4sQe/Gl1+fAqewcXeTahl7RmeOfnVn06PsaL/sMkc+rM1UZXJ5d4V2OcrLW8ECgGdfDqifl5yX+KcEZd6us12bm8bhWd0/YbLxSKp2s4EeFV/wA9sYliW4fdw6W+DtptHs2ZxoVC8XSlclmOBP0cnBhlNDkFozaDkQ2eRjb4Gqmf+UFrtpPUAE7jmTQ94SCnQgF6I2FyypSDsl7S0eF0scLtpdvtnaqFfBagGMrHh3g6zUgsxnA0xkgsymg8XlKmqiCHyUST3UGT3U6Tw0GjzV7dnuplAMVzaSmhuBaOlBeSG6XQ+atqoRiEK/VUu3Y6wpV67nMLV+rZ419IrtQCipeXBBRraDoUWxwm9odexJ8eRkJivfNKGi2dZcfM5kqtKAqn4qc4EDkAQJ2pjku9OzHpyksjzQbFAHklz4HwcQ5HTwLgNjrZ6bsYl7HU5Xq+UAyQymV4eeIYJ2LqsiSnwcqV9RtptdYtGIqnzpfnYGiI1yZOk8qry2XbrV521q+i3uysCRQX2v1ynNcn+jkWGSvyR53ZziW+dta4GtBPm6WuBRSrcdT78WScg4FRjgbHSeamlgU3Wh1s9Day1tMwtbx6CaF4ujL5HAPRMD3hID3hINEZS3a9ZgtdLg9dLi/tTlf5LPIZhuKZ/XP5PIFkkpF4jJFYjNFYjIlkUvMQt9lMk91Bo91Og81Og82G1ai9ZP9Ch+IbLrkfg35xtQuzOZln33joghgkhc5fLQSKhSt1oV07vHClnt+5hSt15fgXkiu1gOLlJQHFGiq8AX556h+xO63klTz7wq8xLA8AsMF1GW3WFSXH5CrUI87kp9qH5VFeDrxBVsli09u43LejzJm6UpyZsDyc9PO8fz9yPo1e0nOpdwMr7W3FWdFqoLjQ3p/w85vxw8Sz6jLW1Y5mtvvWYdWXGzdVKrGUy2vHT+YyvDbRx/7gIPlJOljjbOQy38qy8kpQ/bLn6RAdTsvsmRjiYGikWGvWYTCxzdfGZm8zZr2hZm7VM5XL5+mJhDgYGKMnHCw+VwnodnlZ721gldOn6XxYLZBXA9eKojCRTNITCnI6HGI4Hi3WgAbVtKzN4aLT7aHT5aHBakOardZENflU7F99/HQux1gsxlg8zmgszlgsprnsGsBpMqmQPHmrt9lwmEyVayefJYMvLcnxOJ96i4BiIaGlVBGK/79/L4fiilCptgtX6tkB6EJ3pZ6ea6VzC1fq2eNfCK7UclxA8XKSgGINzYRiUC/AByN76E+eBmCNYytd9rXFY+YDxQDhTJQXJl4hnktgkAxc5r2UZsvUN6jzhWKAZC7F8/79jMjqsuEuWwuX+TZi0hkXBMWgmmW9HjjJwbA6EJh1Rrb71rDG0VpyMa0WigtvsnA6ycv+Ho5HxwHVJGuDu5VLfF3YDVMz54uB4oLkXJZ9gRH2BgZJZDMAGCUdGz1NXORtKzPlWsh5K0lRJJLZDEeC4xwOjJcsrzbodKx2+VjvbaDT4ZnaL1Mx2OKheGacVDZLfzTM6XCI3kj5LLLVYKDD6aHT5abD6Z5aaj2bzgAUa0nOZFVIjscYj8cZj8UrgrLFYKDeZlNB2WajbrKesEGnu2CgWCyfFhJStRgoBuFKrd0+7ecL2JV6Zq6Vzi1cqWePf767UgsoXl4SUKwhLSgG9QJ8OHqQ3sQxALps61jt2DJpnDU/KAZI5VK8EHidibQKsxuc61nrWDNrnMrLquFwpId94eMoKNj0FnbWbabBXP6NLMx/BnlcDvOb8cPFusaNZjdX1K2nzqz+QywUiqfiR3nJ30NfsbySjk3uNrZ5O7EZTDWB4kL/bD7P0cgYeyYGmUhNOTV32D1c5G1lhaOuOHtYSyieroCc4EhwnCNBP+G0XGw36/WsctWxxlNHh8OjwllZsNpDcWmuCkE5yelIiP5ImIFouDjDXpDTZKbd6aLd4aLd6cZlMpcPrGcJirWUymYZj8UZTyQYi8fxx+MEkknNEBLgsVios9qos1kn7224zWbtOsoLyEftvzygWBhtCQmpWiwUg3ClLm+f8fsF6kqtlWulc59PrtSrmnzcvHmNcKWeptnAWEDx8pKAYg1VgmKAdF7H6cQRTsbU/cFN5g42urcD2rVhtaBYjaOwL7yf04neyThNXOq9GL2kXW+3EhQX9vz6U0FenNhPLKtC3yp7Jxd51hcNs6b6z38GOa/kORAa4I3gSbJKDglY62zjEu9qzPry/dAwfyguqC8W5pWJU4zKEUCF443uVrZ6OktmjotxFrEHWVEU+uMh9gaG6IkFiu0Og5nNnmY2epo1zznbeSup8vJshZF4jCPBcY6F/cUZbACTTs9Kl4/Vnjq6nZ6pEllLDMUzlcvnGY5H6QuH6Y+GGY3HisvAC3IaTbQ53bQ5XLQ5nXjN1gUst146KNbqn53co+yPq7Dsj8eZSFauJayTJLwWCz6rFZ/VRp3Vis9qxW2xLHBmWUCxkNByUi2gGIQrdWm7RtsF6EpdKddK5z6fXKnLwLgG8c9XV+qlhOLRGqwKK8RrukBWhgko1tBsUJyZnMkdSp7mcOS1ydrBdWx2X4NJV77EtBIUZyf/K0/He9kX3k+ePFa9lUs9O/CayvcNzQXFoJaA2hs6xvFYHwB2vY0ddRfRYPZN61/dsup8XiKelXklcKxYvsmkM7DVs5L1rs4S8yqoHopzedWIrD8R4LXAacamwfE6VwvbPJ04jVOva62MucKpFPtDwxwMDSNPGmNJQLejjs2eFjrt3pK9p7WCYvXBQs4KQ/EIx8MTnAhPEMtMLWE26HR0OT2sctWx0unDYtAwjVoiKJ4ZJ53LMRyPMhANMxCNaEKy1WCg1e6izeGi1eGkwWYve2/UKp/F9p9+VkVRSGQy+BMJJhJJJpIJJhIJAslk2Wz59OPdFgteixWf1YLPYsVrseK1Wmd3wRZQLCS0rFQrKAbhSj3Vrp3OheZKPVuuF4IrdQkY1yj++eRKvbaujltXr11SKL5x6ycxVJjAqkbZXIpn3vzCBTHeCyjW0HygGCCQHmNf6AWySgaLzs5Wz3XYDKVvmLmgGCCUCfNq4FXiuQQSEptcW+i2rSxZ3jIfKC5oRJ7g5Yn9JHLqMt21jm42u9dh0OkXBMVTcYO8PHGUiXQUAJfRxnbfOtqt9VMGXwuA4mIfRWEgEeT1wGlG5DCg7jle62pmm7cLt9FaU7dqUGcPT0THORAcZigZKT7uNJjZ5Glmg7sZh9G8JFBc2l9hKBHlRHiC46EJopmpPbES0O5wq4Ds8uEyzVVKqVJ7bSA0U4TkCIOxCCPxWEnpJ1ChvtnmoMXupNXhpNnuxDoT7JcBFFfqrygK0XSaQFIF5IlkkkAiSUBOks7lKp7SajDgsVjxWix4LRY8FiseiwWPxYJBo3RaJQkoFhJaetUSioUrdaFdO/yF5ko9e66VzyFcqSvHP19cqZ/t6eG2NWtxmEwCipeZBBRrqPCGeq7nazhmLp+esec3lo3wWvAFkrkEBsnINs9V+ExTg0hmnnuEM/kMrwbfZDCpDhZt1lYu9mybVi+4wp7iCu1yLs8bwSOcjKmO2Q6DjZ11m6g3aQ9wlUsv6Wb8rnAsOswrgZPIk7WN26x17Khbg8/kqAiDFaFbo7+iKAwmw7w60ctgIgSo15hVzga2ebqotzg1jtEMXxmiNa5aE3KC/YERDofHSE2bPe50eNnoaWKFo658z2+VLtaz5TT1uMJ4Ms7xcICToQB+OVHyeL3Fxkq3j5VOH802R9ne3sqGXZXaFwen2XyesUScoUiUwViEoVi0+PpNl9dipcXuoHkSkusttspO0IvIZ+7+C3++iqIQz2QIJpMEkkmCicl7WSY+w6xsppwmUxGQ3VYrHrMZdwGYZ7yv5HicTy3jfUYCioXOBxWh+K+fwGaZYbxYaaWLcKWezH9hM8gXjCs1zPJaz35u4Uo9e/xz3ZVaUZTi6y3H43xql1g+vVwkoFhD1UAxQCon83roJcKZwORM72W0WruB+UMxqP8oR2OnORA+hIKCw2Bnu/cyPCZ31VBc6D+UHOeViYPFWeOV9g62etZj0hln9J8fFBeUzOXYG+zhYLiPPAoSsMbZysWeVdgN5cvIq4FimOKQoUSYNwJ99Man9gB32Hxs83bSavUULyy1gOJC/2w+x4nIBPuDwwwlpmaPLXoD69yNbHA30WCxq+deAiieqaAsczIc4GQ4wFA8UsKANoORFU4v3S4vnQ4PFoPhjEPxzDgF466hWJShWJTheJSgXF5f2KDT0WRz0Gx30Gx30mizaxt4LTSfiv2XJn46lyMky4SSSYJJmWAyqf4uy7POLgM4JoH5tzdsQJIkAcVCQmdASwHFIFyptdun/XwhuFLDgqEYhCv1XPHPdVfqgpYSioXRVvVa9lDc3d1Nb29vWfuf/umf8vWvf72s/dFHH+X9739/SZvZbEaW5bK+lVQtFAOk83Ag/AqjKXVmttO2hrWOi8ihsReUWdykFR0TqQCvBl8jmZORkNjoWs9K+1pNWJgLigEy+Sx7g1N7jS06E9s8G+i0TZVZqhaKC/0jmQSvBk5wOj4GqHuBNzg7uMjTjWVafeOFQnFBfjnGG4F+TkTHio/Vmx1s8bSz2tGErsIHmIVA8XQFU0kOh0Y5FBolnp2aCfSarKx3N7LW1VhVneXZcppP/2Q2w+lIkFORAKcjIdL5KdiSgGabky6nhy6nlyabo3Qm9gxBsZaSmQzD8Sgj8RgjsSgjiZgmKFr0BhrtDppsdppsDhptdpxzgfIygeKKhysKyWyWUFIF5pAsE5ZlQqkU4WnA7DSZuPeSSwAxUyx04elsjvW1hmIQrtTl7TN+P99dqWFRUAznlyu1JhgvMv657koNAoqXm5Y9FI+Pj5Ob9uH5wIED3HzzzTz77LNcf/31Zf0fffRR/uzP/oyjR48W2yRJoqmpwr4VDS0EinOKHkVROBE/SE/8MABeYz0bXFdh1pdD02xQDJDKpdkT2suwrA4YdaY6LvZcgs1g1+xfnk95+5gc4OWJg0SzcQCazHVc4t2Ey+hYMBQXNCqHeDVwglE5BIBR0rPZ3cUmdxcmnWHRUFxQMJXizWA/RyPDZCf3sVr1Rja4Wtnobitzj14sFBeUyyv0xYIcCo/SE50gN+3fptnqZJ2rkTWuBmwG02ScCk9glnNU2z+XzzMYi9ATCXI6GiSQKp2NtegNdDjcar1hhxu30aINl2cAisu6KgqBZJKReJTheIyxRAx/MkFe44Uz6w002uw02Ow02uw0Wu14LNYp4F/mUDxbHEVRSGWzxdnkLo8HEFAsdOHpbI71SwHFIFypS9s12s5nV2pYNBTD+eBKPQsY1yD+ue5KLaB4eWnZQ/FM3XfffTz55JMcP35c8wP+o48+yn333UcoFFrwORYKxQWNyoMciLxCTsli0lnZ7L4Kt7F06dFcUAzqBb030c/+8H6ySg6DZGCL+yLarR3F514NFAOkcwpHoz0cipwgp+RVIyvnCtY515SVb4L5Q3Eh3/5EgNeDJwhMmnGZdUY2u7tY5+yqEL86KC4Yc8m5DIfCQxwMDxLPqqZUOiRWOhrZ7GmjyeKezKk2UDz9vySVy3Iy6udoeJyBeKiYqwS02z2scTaw0lmPVa+9SqBmpl0zXqRoOsXpaJDeaIi+WIjUjJlYl9FMp9OjgrLDXQT4swHFWnGy+TwTyQSjCRWSR+NxJmRtUDZIOuqsNhpsdhqsNhqsduqtdtUpcs7zLh8oBu1xVUCx0IWuMznWF6A4m8kRjyZx+xw1gWIQrtRT7drpnLeu1FATKAbhSj1X/HPZlVpA8fLSHDVTlpfS6TTf+c53uPfee2ddThmLxejq6qKjo4M77riDgwcPzho3lUoRiURKbotRk6WNy303Ydc7SeeT7Ak+w2DiONV+/yBJEt32Tm5svB6v0UdWybIn9AavBV8llUvNHUBDeknPRtdqbm2+hlZLI3kUjkRP8bOR3fQlBqvOcWa+HbZ67mjdyfUNW3AbbaTyGV4PnuA/Bp5jf/gUmbx2TdhqZdEbucTXxXu6L+fm5k00W9zkUTgRG+VHA2/wH/2vcSQyRDY/+17OhcisN7DR08w7urZw7+qdXNO0kiaLEwXoj4d4ZuQ4/3L8JX7cv59DoRHkXGbOmLWQ02RmS10zt3ev54827eT3Vm/hiuZO2uwudJJEJJPiQGCU/+47xj8depVvH93Ds4OnOBH2k8yemRxnk0Gno8nu4KKGZnZ1reY9G7fyp9t28u4NF7GraxVbG5ppsTsx6HRklTyjiRgH/KM829/D48cO8A9vvsy/HHid/zx5mBeG+jga8DNRYfZZ6Ozo61//Ot3d3VgsFnbu3Mkrr7xSse8PfvADLrvsMjweD3a7nW3btvHtb3+7pM+DDz7I+vXrsdvteL1edu3axcsvv1zSJxAI8J73vAeXy4XH4+EDH/gAsVhsSZ6fUG10Nsb6bCbH7h+/wc++9zLB8cV9Bpgug1HP9b+1laZ2L9lMjmd+vJfx4VDN4uv1Oq69eTNtXXXkcnl2P7Wf4YHA3AfOUzqdxFXXrGPFygYUReFXzx6hr9dfs/iSJHH5jpWsW9sMwG9eOM6Jk2M1jX/plk4u2tAGwEtv9HDo+HDN4gNsXdfG9o2qC/XrhwfYc2SgpvE3djdz9ZZuAPadHOblw32L+qw2U2vb6rnhotVIwOH+MZ472FPT+Csafdxy0Rp0ksTJ0QBPHzhBPl+7+O1eN7+1ZR0GnY7eQIifHjhGrkJJxYWo2eXkjkkQHopE+a8DR8jM4RMidG7qnJopfvzxx3n3u99NX18fra2tmn1efPFFjh8/zkUXXUQ4HOZv/uZveO655zh48CDt7doOgQ8++CCf+cxnymP1PIzDZSWRjWDSWTDoTGQrzBRrGWpl8xn2hvcwmlK/oWy1dLHRdQl6SV9xhreSMVcmr+No7DiHI0dRUDDrTGzzXESzpU2z/3xLLw0kxngteIRYVl1622D2cql3I16Tc9Y4lWZ4p/fPKwonYyO8EewhklHjW3RGtni62OBqx6gz1MStGtRvg8flKPuCgxyLjhVByKTTs87VzCZPK17T1NLzmpV2mvZzKJXkWNjPsYgfvxwvtuskiQ67h9WuelY6fVin7bUuDTb3LHWlc88VJ53LMRCL0BcN0RcNlzlag+pq3e5w0+5w0eZwY6s0013xvJUemN9M99xxVOUVhXBKZjwRZzyRwJ9IMJ6Il9R5ni69JOG1WKmz2qZuFhtu84y9yks8I1yN5Fic+2+4Zdl+e7yQWN///vd573vfyze+8Q127tzJww8/zBNPPMHRo0dpbCx3n929ezfBYJD169djMpl48skn+fM//3N+8pOfcMsttwDw2GOP0djYyMqVK0kmk3zlK1/hiSee4MSJEzQ0qE77t912G8PDw3zzm98kk8nw/ve/n+3bt/PYY48t6jUQWjqd6bH+nz73OHrJyDM/eJ2JkTAmi5Gbf3c73gaN97ZwpRau1BX6C1fq2eMLV+ryDwdyPM6nbhYzxctF5xQU33LLLZhMJv7rv/5r3sdkMhk2bNjA3XffzWc/+1nNPqlUilRqauY1EonQ0dHBiz0PY7LDm6Gn0EtGNrqvx6Bza59nFpg9nTjGsdh+ABwGN9vcl2PWVxenAOPBdIjXg3uIZNXlyW3WNra4tmCeUYusmnrEOSXHwXAvByMnySnqwuJVjg62uFdj1JXvh4b5QfFU3zzHo6PsCfYQnYRvs87IJncHG5ydmDXgazF7kJPZDIcjIxwMDRHJTJmutFo9bHS3sMJRj14qX8o9a/wql3kH5CTHI36OhceZSE0BqAS02txFQHYapzl1LyEUz+yfzGYYiEXoj4bpj4UJaLhD+8xW2hwu2u1uWuxOXEYVIs82FJf3V+Mnsxn8iQQTyQT+ZBx/Uv05U+EbY4NOh89ixWex4bNa8Zmt+CxW3GaL5kA6/3wWfmhBSwnFo8OLL9MQiURoaqmuRMPOnTvZvn07X/va1wDI5/N0dHTwkY98hE996lPzinHJJZdw++23V7yWF57j008/zU033cThw4fZuHEjr776KpdddhkATz31FG9961sZGBioCFxCZ1dneqz/p889jtViI53KlIDxrt/Zjq9xxvtbuFILV+pZ+gtX6tnjC1fqUgkoXl46Z6C4t7eXlStX8oMf/IA77rijqmN/93d/F4PBwHe/+9159S+8AV7seRjJJnMo/CzpfBKDZGaN8zpcpuayY+aC2YnUKPsir5DOp9CjZ63zYlos5RebueKACrFHIsc4FjuBgoJJZ2KzazPt1vZpbtLV7TXOKRLxbJI9wSP0J0cBMEh61jlXss65AsOM/cDVQHGhf17JcyI2wpvBHiKTcGyQ9Kx3trPJ3VlSyqkWxlyKotAbD3IwNERvfKLYx6QzsMbZxDpXMw1mZ8nfoFZQPL1/IJXgRMTPycgEY3Lpss16i50VDh8rHD6aLC7Nb2WXAopn9o9n0gzEwgzGIwzGIpozyXaDkRa7ixabkxa7k0aro7S+7lmGYs2HFIVIOjUJygkCyQQTySQBOVFiljYzW7fZgtdixTt577GowGwzGOcuGbXMoXjX2o9hmPElWrXK5lI8fezL9Pf3l+RnNpsxm8tjp9NpbDYb//7v/86dd95ZbL/nnnsIhUL8+Mc/nvV8iqLwzDPP8Pa3v50f/ehH3HzzzZrn+Pu//3s+97nPceLECerr6/mXf/kX/vzP/5xgMDiVezaLxWLhiSee4B3veMcCnr3QUupsjPUFKAbKwLhsxli4UgtX6lnyB+FKPVd84Uo9JQHFy0vnzJ7iRx55hMbGRm6//faqjsvlcuzfv5+WlvJlMvOR3eDlIs+t2A0+skqKw5FfMJI8UvV+izpzE1f4duEzNpAjx+HoaxyIvEQmr73kczbpJT2b3Bu4ruFqXAYX6XyaN0Jv8FLgJeLZ+NwBKshusHJ1w8Xc1LgDn8lFVslxMHKcnw7/ipOxPvLK4vZo6CQda52tvLPjCq5v3IzX5CCr5DgQ6eWJ/t/w6/GDBNO12+snSRKd9jpua9vCe1ZczqW+LhwGM+l8loPhQX7Q/zpP9L3Km8E+Etnq/w7zlc9sY0dDJ3evupj3r9nO1U0raLWqFxa/HOdVfz+Pn36T/3v8JZ4eOsqJiJ9UrjZ7r+cru9HEOm8DN7av4vfXX8wfb97B27vXc0lDK802Bzok4tkMJ8IT/Hr4NI+f2M8/HniJ7x1/k92DpzgSHCeckmu6D6kWkiQJt9nCSo+PHS3t3LpyLe/ZtJUPXXI592y+mN9atY4r2zpZ72ugyWbHqNOhAKGUTE84yBtjw/yy7xT/cewg/7zvNf5h78v826E3+cmpozw/2MtB/xhDsQjxTHrZPfczoY6ODtxud/H20EMPafbz+/3kcrkyZ+CmpiZGRkYqxg+HwzgcDkwmE7fffjtf/epXy4D4ySefxOFwYLFY+MpXvsIvfvEL6uvVD+kjIyNlS7MNBgM+n2/W8wqdPZ2tsb4gk9nIjb99KfXNbtJyhl888WrN9xjfcPsS7zHetZH2rvql22N89RLvMd5+bu8x3rZ2afcYb+pu5potK4Al2mPcWs+NW5Zwj3GDj1sn9xifGJ1Ymj3Gm5dwj7GzdI/xf4o9xiV67rnneNvb3kZrq1r29Uc/+lHJ45Ikad6+9KUvVYz54IMPlvVfv359SR9ZlvnQhz5EXV0dDoeDd77znYyOjlaVu/Ya0mWmfD7PI488wj333IPBUJrye9/7Xtra2oofxv7qr/6Kyy+/nNWrVxMKhfjSl75Eb28vH/zgBxd8frPexhbPzZyIvow/dZre+CsksgG6HTvRSfNwu52URW/lMu+19CSOciJ2kLFUP5HMBJtcO/GYGqrOy2fycl3DdZyIneBo9ChjqTGeGXuGtc61rLCvRV9FbtPVaPHxlqYr6EuM8GboOPFcgteDBzgW7WGLey1t1mYqf908t3SSjlWOZlbam+hPBNgfPs2IHOR4bIjjsSHarHVscHbRaq2fe1ZunnIaLeyoX8Fldd0MJoIcCY/QE/cTSMd50X+Sl/ynaLd5WeNsptvegEG3sNduLrlMFi6pa+eSunYS2TS9sSA9sQB9sSDJXIZD4VEOhUfRIdFic9Ht8NFp81FnttXstZiPrAYjq9x1rHLXAZDN5xhJxBhORBmKRxlORElmM4wkYowkYoD6ocKqN9Bsc9Jkc9BkddBkc065XC8j6Sb3GXst07YHKOosQiyTJiTLBFJJQnKSoJwkmJKJpGQy+TzjyTjjyfIvn4w6HW6zBY/ZgttkmfrZbMFhMqGvNMN0hvVvv7q/RsuntWeKaymn08nevXuJxWL88pe/5GMf+xgrV64sKdFzww03sHfvXvx+P//8z//M7/3e7/Hyyy9r7lMWWt4622N9QSazkRvfeRm//I/XmBgJ84snXq28x3gBKphvFVypn/nxXu2l1AuUXq/jmps3FV2pdz+1v/Ie4wWoYL4F0HNqnF89e6TyHuMFqGC+BXD02Ai/eeE4gOYe44XGv3RLJwD7Dg/y0htqySOtPcYL1ba1KnS/eqif1w+rUFxxj/ECtLGrCQWF3+w/zb6T6vhbNmO8CK1trUeR4Nl9Jzjcr34pUTZjvAitaFDNt3627zgnRidQUKZcqWuggvnWk/uPFsG40h7jhagAxj8+cKQIxkVX6mUiOZHGZFj8hI+cqC5GPB5n69at3Hvvvfz2b/922ePDw6VfQv33f/83H/jAB3jnO985a9xNmzbx9NNPF3+fOUZ89KMf5Sc/+QlPPPEEbrebD3/4w/z2b/82zz///LxzPyeg+Omnn6avr49777237LG+vj50097kwWCQP/iDP2BkZASv18ull17KCy+8wMaNGxeVg14ysNZ5JVa9j/7EG4ynTpDMhVnjug6TzjZ3gElJksRK+3rcxiYOhl8imY/zemg33bb1rLBvBKr7h1JnX9fSYm1hX2gf/rSfI9Ej9CcG2OzeSoN5YYOIJEl02VtosbZwMtbH4cgJotk4L0zswWt0sdG1jmZLw6IukAW36g5bPWNyiAORPnrjowwmJxhMTuA22tno6mKlvbVmkKqaXvlot9WRymU4ER3jaHSEMTlCfyJAfyKAUdKzwtHAGmczLVbvVD3cGstmMLHB08QGTxM5Jc9QPEJPLEBPdIJwRmYwEWYwEeZ5enAYTHTafXTZvbTbPVgqmGAtlQw6/aQJlxsFFR7DaZmRRLQIy+PJOMlclp5okJ7o1HJVp9FMs81Bo9VJk9VBo9WOxXBm85+vJEnCaTLjNJnpoHTffzafJ5KWCckyoZRMUE4SSsmEUkmi6TSZfB7/5DLtsrio7uDuSWB2mc3Fe5fJPL9l2TWSxWbCYlvcFxXprHq8y+WaF2DX19ej1+vLvrUdHR2lubl8O0pBOp2O1atXA7Bt2zYOHz7MQw89VALFdrud1atXs3r1ai6//HLWrFnDt771Le6//36am5sZGyudZcpmswQCgVnPK3R2tBzG+oJMZiM3aYFxo6cm8SuCcVttwLXgSl0Gxh1LDMYrqv+CX0uVwHjV6vnXoZ4rvhYYb1hbOzDeum6Jwbh7cjZ9qcC4Tf2SY8nAuHEKjE+OBpA4oS6l1i8tGOtqBcauUjD+rwNH1KXUhuUBxu+5+q8XvVUK1O1SQFllnkrbpW677TZuu+22ivFmjr0//vGPueGGG1i5cuWseRgMhorjdjgc5lvf+haPPfYYN954I6CuOtqwYQMvvfQSl19++ayxCzpn9hSfSRXWz792+m9xuEqNprKKnmB6iCOR35BTMhglC+tdV2M3al9IM4r29w5ZRUc2n+FQdC9Dch8AToObTe6dOAzlJly5inuNS+sa9yeHeDN0CDmvvonbrW1scW/CordMxqlur24hfjqf4Uikl8OR02QVdZlIvdnDVvdaGi1Tg2y18WfWQY5mkhwI93MkMkRm8jwWnZF1rjbWu9qx6ysYf1WYua583tL2UDrB0cgoxyJjJeZcNr2J1c4GVjkbNff8VutirR6k3Tz9OYRSSU7HgpyOBumPh8lNW7ouAU1WJ512D512L802Z8ks5Hz2Oc8nn2r3LBdmUUfi6uzxaCJGMFVu4AXgMplptDposqmQ3Gi1YzMW6iZXm0+lB6rcy1x1/Cll83kiqRThlArM4UlwDqdSRFJyxf3LBeklHV6Lhfds3IokSeqe4uvOL/fpnTt3smPHDr761a8C6qxgZ2cnH/7wh+dttHXvvfdy6tQpdu/eXbHPqlWr+P3f/30efPDBotHWa6+9xqWXXgrAz3/+c2699VZhtCU0taf4r75X3FNc1OQH5zn3GBckXKmFK3WFY4Qr9ezxL2RXajke55NvuXlp/ENW3VczKH765MNl7Q888AAPPvjgrMdKksQPf/jDEi+R6RodHaW9vZ1//dd/5d3vfnfFOA8++CBf+tKXcLvdWCwWrrjiCh566CE6O9Uvtp555hluuukmgsEgHo+neFxXVxf33XcfH/3oR+d6mmq+AorLNRcUAySzEQ5HniORCwMSXfZLaLZs0DDOqgzFBY3IAxyM7CGjpJHQsdqxmU7b2pJY84Hi4jnzGfaFj3Eqrn7zaZAMrHeuY5VjBUqFxQFzQXFBci7NoUgPx6J9RVBrttSx2b2aBrN30VBcUDKX41hkkIORfmJZFVIlJLpsDWxwddBs8ZYaZC0SiovteRiRwxyNjHIyOk5qWl1lp9HCamcjqx0N1JkdqhPzEkHxdGVzOQYSEU5Hg/TFQwRSpTORRp2eNpubDruHDrubOrO9gmHX0kKxVv9ULstYPF6E5LFknHBa1uip7mtutNppsNhpsKo3t8lSnKlfzlA86+GKQjyTJiynCadkwimZSDpFJJUikpaJptWlSS6TmXsvUsHtfITi73//+9xzzz1885vfZMeOHTz88MM8/vjjHDlyhKamprLlsQ899BCXXXYZq1atIpVK8dOf/pRPfepT/OM//iMf/OAHicfj/PVf/zVvf/vbaWlpwe/38/Wvf53HHnuM119/nU2bNgHqt9ajo6N84xvfKJZkuuyyy0RJJqF5QTGUg7FwpRau1HPFn3mMcKWePf6F6kq9lFBci0oThXhNLQ3zNtacrrmg+Itf/CKf//znGRoawmKxaPYBdYl1LBZj3bp1DA8P85nPfIbBwUEOHDiA0+nkscce4/3vf39JdQGAHTt2cMMNN/CFL3xhXs/1nFg+vRxlNbjY6r2VE9GXGU+dpjf+OpHMKKscV2LQVffNTLOlHa+xngOR1xlPj3A8to+x1CAbXZdhN1T/hjbqjGz1bKHL1sHe0D6CmRAHIgfpiZ9mo2sTzZaWBS9/sehNXOJdx1pHN4cipzgZ62dEnmBEnqDR7GW9azVN5sXvBTbpDGz2dLHR3UFfws+hcD/DcpDTiTFOJ8ZwG22sd7azytGCpVLd3wVIkiRarB5arB6uaVxDfzzA8egYPTE/0YzMnkAfewJ9uIwWVjoaWGFvoLGCa3StZNDp6XZ46XZ4AYhmUvTFQvTGQvTHgyRzWU7HApyOqWYqZp2BNrubdpubtklIXqol4HPJrDfQ4fTQ4fQU2+RslrGkCshjiRijyRihlEw8k6Ynk6YnMrX02iDpqLPYqLcU6gvbqbfYzuhy48VKkiQcJjMOo4U2Z/n/cy6fJ5pOkz7D5mpnWnfddRfj4+N8+tOfZmRkhG3btvHUU08VzbdmLo+Nx+P86Z/+KQMDA1itVtavX893vvMd7rrrLgD0ej1HjhzhX//1X/H7/dTV1bF9+3Z+/etfF4EY4N/+7d/48Ic/zE033YROp+Od73wnf//3f39mn7zQOa2C+VYBjJ/+99rvMb7h9q1FMF6KPcbX7tpYBOMl2WN89dRS6ud2L8Ee4+2TS6mPL/Ee4yNDvLSnB6Rza4/xpsJS6gOn2XdqGKTa7zEGeGb/CY4MjCFJtd9jfOtFa3hqco8xUHnGeAEqmG89eeDM7TGuOGN8hlSLrVJQ/XapavQv//IvvOc975kViIGS5dgXXXQRO3fupKuri8cff5wPfOADNctHzBRraD4zxQUpisKwfIye2Bso5DHp7KxxXoPTqO6rmc9M8fRYfclejsXeJKdk0aGj276Bbvt6FLT3YGrFmd6uKAq9iX4ORQ6TmlxSXWeqY5NrCx6Tp9h/vjPFM/vHsgkOhU/REx8kPzml5jO52eBcRau1qXjBrHamOKfRP5COcTg8wInYcHEJt17S0WVrZI2zjWaLr+wCXe1McaXZ1HQ+R298ghPRMfrigZLlzDa9iRWOBlY6GmixutFJuprOFFecyVUkFEVhXI7THw/RnwgxlIiQyZe6IJp1BlpsLlptbtpsbhosjlLTpzNQ8mkupXNZ/HKCsUS8aGTllxMlr/N0WfQG6iw29Wa24Zv82VayV3l5zBRPxZn/4H4+zhQLCS03zXemuKB0KsOzP3gdv9ZS6kWWaoLJGeOf7mN0IFg+Y7zIUk0wOWP8y8MM9PrLZ4wXMVNc0KxLqRdZqgkmZ4xfO8XRY+Uzxost1VSI//qBfvYdHgRKl1Ivdqa4oFlnjBdZqgngUO8ov96vrhIsK9e0iJnWgmYt11SD+LPOGNcg/mwzxost1QRzzBif4Zni5VCSabaZ4l//+tdce+217N27l61bt1ad1/bt29m1axcPPfSQWD69lKoGigsKZ4Ici/yaVD4KSHTYttJq3UQW7W9pKsIsOuRcgsOR15lIqxd+u97FOtcO3Mbyb13nguKCMvksx2PHOR49SR4VNNqtHax3bsBmsC0YigtKZGUOR3s4GesvgozL4GC9ayUdtlakCgZi1UAxqCCYzmc5FRvmSHSAwLQSTk6DlTXONlY7WrFN1jyuFRRP/yfJ5LP0xQOcivnpjU+UQKhZZ6DTXkenrZ4Oex0mncaXIjWE4pnKKXnGkjEGEmEG4iGGk9EySDZIOpqsTlqsLlpsblosLsz68jzPJBRrBcpPmnn5JwHZLyfwJ+OEKiy/BtX92mex4TVb8ZlVUPaZbTiMpqmBW0CxgGIhIaqHYoBMOls03yoB4xpAMUAmly+ab5WAcQ2gGCCrUDTfKgHjGkAxqNft53+tAcY1gGK1XeGlV8rBuBZQDOrl/vX9fWVgXCsohlnAuAZQDHCwd4Tf7D8NzADjGkAlwNFhP8/u0wDjGsU/NR7gZ5NgvKrJN+VKXaP4A6EwT+4vB+NaQDGUg3HRlVpAcYne9773ceDAAV577bWqc4rFYnR2dvLggw/yP//n/yQcDtPQ0MB3v/vdoov10aNHWb9+PS+++KIw2lqMFgLFqnFWmp7Yy0ykTwPgNDTS5bgOs96h2V8zDlMzvKNyP0eje8ko6gxvq3U1q+wXYdCZ5o5ToT2WlTkcOcRAUl2+o0PHCvtKVjnWYtJY9j1fKC4ons1wLHqak7FeMoq6HNSqt7Da0c1KeydGXemM90KgeOpnhYl0lGPRQU7GRornk4A2az2rHK20WxvRazhXLwaKpyubUxhIBumJjXM65kfOZ4qP6ZBotXrpstfTaa/DabTOGqwWUDyze15RGJdjDCUiDMbDDCXDyBrLdOvMNlqsbpqtLpqtTjxGa+UPW2cIiis1Z/I5gnKSCTkxdUsliKRT2geilkvymKwqMJuseM1TN+NczuYCipc0lpDQ2dJCoFiSJNKpTDkYV3KlrhKKFZ1ENpMrB+NKrtRVw2hnMssAAE5KSURBVKBOnTGeCcaVXKmrhGKkyRnjmWBcwZW6WihGNzljPAOMK7pSVwnFSNLUHuNpYFzRlXoBUKxI8OZRDTCuERQrEhw6rQHGFb+YqDK+Do4NaoBxreJL0DM2BcbFGeMKrtRVf2EhTc4YzwBjnaHSF0+V86zUfySiMWOs4Up9PkJxLBbjxIkTAFx88cV8+ctf5oYbbsDn8xWNsSKRCC0tLfzt3/4tf/zHf1wW46abbuId73gHH/7whwH4i7/4C972trfR1dXF0NAQDzzwAHv37uXQoUM0NKjXlj/5kz/hpz/9KY8++igul4uPfOQjALzwwgvzfq4CijVUeAPs6/0CzhlQnKkIxWq7oiiMpXo4EXuNnJJFLxlZ5dhJvbm7tH+FmdOZEJrOpzgcPchAshcAs87CBufFNJrVotiVoVg7fsEIK5AOsT98iPG0unfDIBlY61zNKvtKDNNmNxfjVn082s+RaC9yLj15Dj2rHR2sc3ZPm8mtEL/CVSiX1+6fyufpiY1yNDrEqBwqtpt0Blbam1jjbKXBPLX3t6JB1iIMu/KKwkgywumYn57YBKFMqfOyz2Sjy1FHp62OJqurrG5tRcitCInVg72iKARSCYYSUYYSEYYSEU3jK4veQPPkbHKz1UmT1VmcTa7dDHJVzXNCZSaXI5BKEpDV24ScIDBZNik/C9k6jCa8Zises2Xy3orXbMFlsmDQ6aqG4loYgsmxOPdffeuyHSgFFAudDypC8YPf1YDiCtcb4Uo9a3zhSj2Pc0y2C1fq2eNfCK7UcjzOJ285v6B49+7d3HDDDWXt99xzD48++igA//RP/8R9993H8PAwbnd5xZ3u7m7e9773Fd2t3/Wud/Hcc88xMTFBQ0MDV199NX/913/NqlWrisfIssyf//mf893vfpdUKsUtt9zCP/zDP1RVflFAsYYWA8UFJXNRjkReIJpVobPBvIIV9u3FWd75QnFBo6kJDkXeIJFTlwvXm5pZ79yGWa/9Bp0LimFyNjo1zv7IYcIZtf6YRWdmvWsdXbZOdJJuwVA8db48p+NDHAqfJpKNA6qLdKetmXXObrwmr3b8KqF4ev9wOs7x6DDHY8PEs1Ozh26jjVWOZlY5mnEa7FWdt9pl2ADBdIKe2ASnY35GkpESJjLp9HTYfHTafbTbfDiM5jMCxVqKp9MMJ1VAHk5EGZNjmiWEfCYrTZOA3GR1Ume2q9BYPMHZheJKcXJKnnAqRVBOEkglS+61Zs2ny2U04zFb8JiteEyWqTrDJrPmkvPZ86/0gIBiIaGzocVAMQhXai0JV+p5nGNau3Clnj3++e5KfT5C8bksAcUamgnF6VwUg86KTjLMG4oB8kqe0/GDDCQPAAomnZ1Vjp14Ta1VQ3FW0ZNTcpyKH6EnfhSF/CRcrqXbthHDjGXJ84HighRFoTc5xKHIERI5tdyPVW9lnXMNbdYu9NL8lx9Xyj+XhyF5nCORHsZSU+7CPpOb1Y4uOm0tJedZDBRPf15DySDHo0Ocjo+RnWbaVG9ysdLRzAp7E3aDZdY4sDAoni45l6E/HuR0fIK+WAA5XwpjXpONdpsKyK1WT8mS3qWG4pkP5PJ5xuU4w4kow8koI8kIkUz50mQdEnUWGw0WB40WB40WJ/VmO4YZy5HPNhTPFieZzRBKyQRTSUKpJMFUkmBKJpRKkslrm3wVZNEbcJssuE1mXJOg7DZZcE7+bpj5YVVAsZDQstJioRjmMWO8yL3Gc4LxIvcazwnGi4BimMeM8SL3Gs8FxovdazzXjPFioRjmAcaLXFZdAsYzzbeqyLNS/znBeJHx5wTjRcafC4wXu9d4NjAWULy8JKBYQ9Oh2O40cCL8IyRJR4fjRvQ67RIDs+01jmTGOB59gVReneVtNK+i3b5ds3TTfJZDx7NRDkf3MpEeBcCks7DafhHNlq7ihagaKAbIIZFX8vTET3M0erzoVG3RWVnjXEOnrRSOF2PMFUiHORrppS8xXFzWataZWGnvYJWjA7vBVhMohimgTOeznI6PcTI2wnAyUMInTWYPKxxNdNsasRismnEWC8XTlc3DmByhLx6gPxFgTI6WPK5Dosnqos3qpc3mpcFcvtQalg6KteIksmlGkzFGklFGk1FG5ajmLKsE+MwqKNebHTRY7NRbHFj0Gu7pywCKK3ZRFBKTwBxKyYRTSYLpQo1hmeQ8yifZDUack7DsNJpxGc0qME/eT800CygWEjobqgUUg3Clni7hSj2Pc2i0n0+u1GVgXIP456srtYDi5SUBxRqaDsUGW4LT0f8mqySRMNBkuxq3aV3Z8pDZoBggp2Toi+9lWD4KgFGy0u24HK+5U7N/eZzyUlDj6WGORPeRnFxS7TL4WOPYisfUsCAonuqT43S8l2PRE8h5db+pRWdhtUOFY4POsGi3agA5l+JEbJCTsT4Sual9rc2WBlbaO2mxNqKb8YFioVA8XclsilPxcXpiI4ymQiWPNZo9dNub6Jo5g1xDKJ55jJzLMJgI0h8PMZAIEM2W7vE1SDqarR7arB5arV7qzQ50ku6MQvHM7oqiEM2kGJdjjBVuyRjJXEbzGIfBrAKy2aHWGjbbcRutmvWTlwMUzxUoncsSTqcIp2XCaZlIOkVk8j6cluecZQZ1Cb1zEpAdRjMuk5ntDe1IkrSkUDw6PFYTKG5qabwgBkmh81e1gmIQrtQFCVfqeZyjQvv55EpdAsY1in8+uVKva6jnLetXCyheZhJQrKGZy6cz+QQDsd3EsurF0GVaTZPtWvTSdBfo+S2HjmTGOBl7iWRO3cPrM3XT5diBUWfV7D9X/HReoT9xnJ7EIXKT7ssN5ja6bRdjMzjL+s8Hiqf65uiJ93M8ehw5r5pGmXQmVtpX0WVbiVFXXm6qWrfqvKIjr+QZSo5xItbLaGqi+JhFb2aFvYOV9k5skzO4tYBitV3tH8vK9MRH6I2PMZYKl/RpMLvpsjXSaW/EYSh3EJ/tec2m2ZyvFUUhkpEZTAYZTAQZTAaRZ4CmUdLTbHXTYvXQYvHSYHGWfHlwJqBYsz0P8WyaMTmGX44xnoozLseIZLRLKBkkHT6zjXqznTqzHd/kvVVv1DbrWEZQPJvyioKcyxJJy0TTaSIZFZaj6ZR6n0lpzrLbDEb+cONOYGlnine1/pHmKpVqlM2neHromxfEICl0/qqWUCxcqafyrHTeC8qVerZzzNIuXKlnz/98cKV+5vgpbt+4DrfVIqB4mUlAsYYKb4DDvQ/hdKmzhWqJpH0MJd9A3R/sotNxIzaDugSocomlcpjNKTl6YgcZSB4GFAySmRWOS2g0ryBXca9xhfbJEk6pnMzx2CH6k6cB1cyqw7qKlY6NJaWWKu75naU9p+Q4nejnaPQEiZwKxwbJwAp7F6sdq7Do555RnS3+dEUzcY7HBjgZGyKVTxfbWyz1rHK002JtrGqP82wwPlOxrMyp2Dg98bESB2sAj9FOl72BbnsjdSbngl2sZz1Go10tO5VgIBFkIBFiOBEmNWM/sjqT7FIh2eqm0eLSLDNUKyOvysuetfvLuSx+Oc64HC/eT8iJkj3e02XRG6gz26kz2/BZbOq92YZNr13z+6y5YVd53unK5HJEMyki6fTkfQoJuKJFXTkix+J86oq3CigWElpCFaH4gcfKobgSVApX6pL+841/oblST89p3ueYbBeu1LPHP9ddqRVFKb4ecjzOJ29dGiiuxaqwQrwLZWWYgGINaUFxQeHMGH3xZ8nk40joaLJeRoNlMzm0nWgrG2rpiWUmOBZ9iUQuBIDb2ES3fSdWQ7k9+VxQXFA0E+FobD/jKXW/sQqvG+i0rUYvGRYExQXllTz9ySGORk8Qyar7YHXo6LR1sNqxCqfRsWgoLiijKAwkRjkR62dUDhTbTToj3fZWVtjb8Zqm/jlrAcUwBbPxbIre+Bin4+MMJ4Mo09DIrjfTaW+g01ZPk9lXZiw1PY6WqoFiKIWyvKIwkYozlAwxmAgzlAiVQbIOiXqLQ607bHHTZHVhN9TO3bpaKNbqnlcUwikZf0oF5UKt4VA6WRFCrXoDvklA9plseM1qzWGH0VzVzPJygOK54i8lFIvl00JCqpYCikG4UmvpQnOlnpnTvM4xrf18cqUuzhgLV+oyLSUU72r5o2LFm8Uom0/z9PCF8SW4gGINzQbFWXRk8zID8V8TyfQCYDM00Wq7HpNeA2bnmPlVlw4fpi++nzw5JHS0WDfQZtuCXjKW9S+Pr/1fOSb7ORbbRzQbAlQzrhX29TRbVmvOtFYDrYqiMCiPcSx2nEB6ykm6ydzISvsqGswNZRe/aqF4OlRGM3FOxQY5FR8kmZtyQfYYnXTZW+mytWDR27TCLBiKpyuVy9AX99ObGGcg4S+Z4dRLOlotPjps9bTb6nFMLvNeKiieGUdRFALpOMPJMMPJMEPJcEkZqoIcBjNNFnUWucniot7sKML82YDiSg9k8zkmUjITKRWUAyn1Fq6wBBvUv4HHZMFrsql1hk2TN6MNq6Hc4OtCh2JhtCUkpGqpoBiEK/VMXWiu1Fo5zXmOGe3nkyu1JhgvMv657koNAoqXmwQUa2guKAb1ghhIHWU48TJ5MqoJl3UHXvOmkn/6uaC4IDkX42TsVYJp9QJr0tnosl+Gz6RepKqF4pyiR1EUhuVeTsQOIufVUktmnZUu20ZarSvQTYPjaqE1hzS5tDfA8dgJRuTR4mMOg4OV9pV0WDsw6AwLiq8FlXlFYSg5QU9sgMHkaNG5GqDB7KPL1ka7rRnTtPJUtYBimALHbD7HUDJAX8JPf8JPIlcKoB6jnTZrHa3WeposXs1Z5FpCsZYiGZnhZIjhZJhROUIgFS+LoUPCZ7bTaHHRYHbRaHbhMdlKjK/OBhRXipPJ5ybrC6szysF0gmA6STid1KypXJBZZ5iCZJN1soSSFbfRikVvKB2gBRSflVhCQmdLSwnFIFypp+tCc6WulNOs59BoP59cqcvAuAbxz3VXarF8enlJQLGG5gPFBaVzUfrjzxHPDgNgM7TSarsOk9452X/+e4QVRWE8NURv/FVS+TgATkMT3Y7LMOu1zShmg+KC8kqewWQPp+KHSU0aZll0Nrrtm2i2dKOTdAuC4umKZWOcivXQm+gnO2n4ZZSMdNo66bZ3Y9Vr/yNVA8Xqc1HbU7k0/YkRehNDjE+re6xDR4u1gU5bKy2WBnSSRikgFg7FpW3qUuaBpArI46lQCeDoJR1NFi9t1nparT48RgeSJC05FM/sn8lnGZOjjCSjjMkRRuRwmXkXqAZedWYHDWYnDRYn9WYXbqOtfMnTWYDiSv3zikI4o9YZDqbV5dehVJJQOkksmy4/YJrMOrXOsMtowWWy4jZaJm/qkuzCFwQCipc2lpDQ2dJMKM7l8iRjMg63rSZQDMKVuqALzZV6tpyEK7VwpS5oKaFYGG1VLwHFGqoGimESZuXDjCZfQSGLDiON1h14zRvJSRX2Gs/iVp1XsgwmDjKUPIhCDoB68xrabJcUXapnywdKobigvJKjL3Ga04lDpIullmx02jbQZFlVMnM8FWf+btUAqVyWvkQfPfEe4rl4sb3B1EiXfQWN5uYSp+SFQvF0xbNJTseH6UsMEs7Eiu16SU+LpZEOWwvNlsYZdZZr62IN6jLrITnAYMLPQHKibBbZqjfRYqlTb9a6kpJPlZ4bLB6KZ/ZXFIVYNsWYHCnexlMxskqu7BijpKfe7KTe7KTO7KDe7MRjtJWVylLPe+aheLb4mVyecDpZnFEOZ2RCkz/PBcw6JLVMktFSvDmNFrXGsNGC3WDS3scMAoqFhM4RTYdik9HCr3/4KhPDQW5+99W46surNwDClVq4Us/or+1KPWu5piqHDuFKPXf+56ortYDi5SUBxRoqvAFO9X1uXlAMkFUkUrkIvfHniGfHALAbGmm1X4dF7ynrn5lHHWE5F+dUfC/jqT4A9JKBdttmWq3riwBbGa4rxEcip+ToS5ziVPwY6bwKbmadhW7bOtptK9FPA/lcxXrE2vELUKkoCiPyGCfivYzIY8XHrXorK+xddNk6seot5Cu8+7SgvpC/5nkn8wymo5OAPEIsmyw+bpD0tNsa6bA102KpR1fhy4pqDbsqAVwuD8FMnIFEgMHEBMNyiNwMt2W30Uar1UeL1UuL1Yu5kiNwhdeoMsBXB6d5RSKvKATTccbkGONylDE5ij8VL8sZ1BnwOrNac7hwqzPbi0vl55tPJZ2JPc7ZfI5wWiZUrDM87eeMPOuSbAC9NB2aVVB2mszqzwa17rB+xhcH1Vxok7E4n9jxW8t2oLyQBkmh81dFKP4/30anGPjFd18k5I9itZu5+T1X46rTKMUnXKmFK3VJ/lPxS823utm4tnXWnOZ9jsl24Uo9e/xz0ZU6GY/zydsEFC8XCSjW0EKhGNQLoz91mKHEa+TJIqGj0Xox9ZatJTOx84HigsKZcU7E9hDLqjV8LToHXfZt1Jk7K7tezwMqc0qOgcRpTsWPFesQGyUz3fa1tFtXYdQZFwzF0xXLxjkZ6+d0oo/0ZJklCYlmSxNdti6azI0axlwLg+KCVAOqCL2JEXrjIyRyU0ZNeklPq6WBdlsTrdYGjPPag1wdFM/sn1PyjMphBhMBBpMB/KlIGSR5jQ5arF6arT6aLR4shRJEZwCKtdung3IMfyqKX46T0ZhRBnAbrZOllBzUmR34THZcRgtU+J+ppLNt/KUoCvFsehKQU0VQLvwey6TmBbh2gwmHwYzTaMIxCc4Oo9rmMJqxGYxl4FyQgGIhoaXXdCi2WmzIiRRPf+8lFYwdZna9+yrcdTNmjIUrtXClLsm/NL5wpZ49vnClLpWA4uUlAcUaWgwUF5TOxehPvEAkMwCAWe+lzXYVdqN6gawGitX+EuOpHnrje0lPAqzd4KPddjFuY2vZRWU+UDx1zhwDiQF6EkdITi551ksG2qwraLeuxaq3azzf+UNx4XnllBwDySF64qdLXKstOgudtg46bR04DI7J/ouD4oKUoiFYmN74MP2J0RJA1iHRaKmjzdpIq7URa5Uu1vOF4pnHpHIZhuUgw8kgQ8kgwXSsrJ/baKfJ4qHJ7KHJ4sFpsJb8nZcairVjK4TSMuOpGH45ykQqjj8VI5HTXo5skHR4TXZ8Jjs+sx2vSb05DBXKKHH2oXgu5RWFWCZFOJOarDEsEy38PHmvNcOuJZvBiMNgxm4w4TSaua55FZIkCSgWEjoDmgnFQBkY3/zuGTPGwpVauFKX5F8eX7hSzx5fuFJPSUDx8pKAYg3NhOJsPoZesiJJ+nlDMUzOGqdPM5x4kZyigpjHtJpm206QtPcrVYLiwnlzSpbBxGGGkofITRpaOQ1NdNgvxmmcurBXA8VqXD15Jc+w3Mfp+FHiucjkIxKN5nY6bWtxG+vmjD8bFE9XJBOhJ95Hf3KgOHsM4DN66bB10GzpwKRhJb8QKC75fXIGuS8xymBilEg2XvK42+ik1dpIq6UJn8ldvFDXGopnKplLM5QIMSwHGEkGCWXiZX2sehNNZg+NFg+NZg9es0tzpnEpobhS/GQ2jT8Vw5+KM5GKEUjHCabjFZchGyS9WkLJZC/ee0w2XEYLUoX/seUCxXPGzyskc5kiKMcyKaLZws9pYtkU8Uy6xD0d1JnlD6zdCYiZYiGhMyEtKIZJMP7+y4TGI+VgLFyphSt1Sf7a8WcDY+FKLVypCxJQvLwkoFhD06HY4TQyEvs+oFBnews6vbY5gxYUg+o+nc3LjCZfJZA6AoAOI3XWy/CZNyPNMLeaC4oLyuRlBhIHGU4eQ0GdlfIY22m3b8VuqFsQFBekzqyO0ps4xkR6qtSS21hPh3UNDeY28lRwdZ4nFE89jxwj8ih9iT5GU1N7j3XoaLI0027tKDHnWiwUz+wfycQYTI4ymBxnIhUswRSLzkSzpYFmawON5saSUk/zjb+QnADkXJoxOcyoHGRUDuFPRcogSifpqDe5aLR4aDC7qTe7sRssZwWKteLnlTzhjExAjhNIxydBOUE4nSh7LgXpkHAZrXhMNvVmVO/dRhtmfYV6e8sMiudjCKYoKjhPh2QFhYt86h40AcVCQkuvSlAMIKeyPP3Y84TGZ8wYC1dqQLhST+VfoX2Wck3ClVqVcKUWULzcJKBYQ9Oh2GyPMRb/D/KKDOhwmHfgNO8og9nZoLigRHacofjzJHPjAJh0HppsV2E3dhT7zBeKC4pnZQYTbzKeOknhY7fX1EGz9WJshrqy/vOB4ukKpcP0JY8yKvcX4duss9JqXUOLZTXGGeZQ1ULxdKMtOSfTnxygPzFAJBspthslE63WNtqs7biN9ZrLbmsxk5vKpRlM+hlKjjIi+4ulpQqqM3lVSLY04jG61PJKSwTFMw4gm88xngozlgozlgoxJodJ5cvLKtn0ZhrMHuonIbnO5MI4aYB1pqG4Uv+ckieSSRJKJ1RQTsUJZRKE0gmysyw7tuiMuCcB2W20Tv5sxWWwFp9jaT7LF4rnkoBiIaGl12xQjF6vzhjPBGPhSj0ZX7hSq/lXiA8VyzUJV+qpfC50V2oBxctLAoo1VHgD9Pd/DpfLQjYfZzz5LPHMSQCMunrqbW8pqR2crQiDpe3qLOwJBhKvkZ1cUu02dtNq245Z76rsJl1x2bbaP5GN0Jc4wFiql8LHb5+pnQ77FhwGX1n/SnHK8p98XnIuSV/iFP3J00XHah06mi0ddNlW4zJ6J+NUKnVUqXSU9usWSEfpTQzQnxhEzk+VNrLqrbRZW+mwtuOeBFOYzSyq0lJcbRXgPafkGZeDDMp+hpLjhGcsZ7boTDRb62gy19NsqcNWVl6p8khZLZxqwZeiKATTsjqbnAoxJkcIpmNlsSXAY7RTb3FRb3LRYHbhMztLll1XhMczVCJq6ne1VFQgnSCYThBMJQhNwnN8jhJKNr0Jj8mKy2hVQdk0eW+0YNZrr2qYb/6V+1fVvSqITsbifOyyty/bgfJCGiSFzl8V3sff/N//rwyKC2NLyR5j4Uo9rV29E67U2s3qc9BYSi1cqcv6X8iu1Ml4nE++VUDxcpGAYg3NhGKY/MCeOc548tnirLHbfBke83YkyTBvKC5IzmcYSe7BnzqE+nFZR4NlE3WWS9FL5ctE54LighLZML2JA4yneottPlM7HbbNOIyVl1XPd7l1TskxIg9wOn6KSHbKLMtt9NFhXUm9ubOkpFNB1UJxASrzSp6xlJ/+5BCDyeGS2VuHwUG7tZVWaysOvavCDPLCoHimolmZoeQ4w0k/I/JEWU1ft9FBs0UF5AazV/NvONe5Fzvbncln8aeijMoRdVZZjpTVSgZ1ibLX5KDO7KTe7KLO5MJrcmDQlb4HzjQUz9Y/nc8RTicIpVVIDmUKtYeTyLmsxhFTMusMuI3WyTrD1ml1h60lpZMuFCgeHRqrCRQ3tTZeEIOk0Pmr+UAxlJtvCVdqZnWlvu6WLbTOnDFeIBTDuelKrT6HqfjClXr2+BeqK7WA4uUlAcUa0oLiglK5BBPJZ0lkTwBg0Hmps16P0dClGatiSaNJSExmAwwlXiaaVS+WeslMg+USfOaNJSWc5gvFBUWzEfoT+/FPg2O3sZlm62bcxpZFuVUD5PIS4UyA3uQJRuUBlMlP9gbJSLOli1bLSpxGT7H/QqG4tG+OQXmMgcQgI/IoeaaW2tr1dlqsLbRaWvEYPdNmkGsDxdOXheeUPBOpEMNJP8PyBIF0uKSvhITX5KbJ7KPRUke9yVtSw3epoFirfyKbYjwVYTwVwS+r91rLriUk3EYbPrMTn8mJz+TAa3RhM5TXTj4bUDybktlsEZDDmSSRacCczJU/15myG9Tawk6jFYfBotYcNphxGC04DBaMOu3/jXMVim9yvgfDLF/azEdZJc0vo/92QQySQuev5gvFIFypy9tLfxWu1FrPoTS+cKWePf6F6Eq9lFBciy/AC/EulC/BBRRraDYozirqxS2ROU5A/hU5JQGA1bgOj+Va9LrS8kVzQTGo8SKZfoYSr5LKhwAw6pw0WbfjNqolWqqF4gLMJrJhBoozx+qf2qb30mLbTJ2pC2lyoK0WiqfDZionMyj3MJA8XSzpBOAy+GixdtNk7kSvs2iFqQqKp+eTyWcYlkcYTA4zJo+VALJVb6XZ0kyzpRmfsbFo0jVdi4HikjiKuhd5VJ5gWJ5gLDVBLJss6SMh4TO5aTD7aDD7qDN7NU27lgKKZ+aqLlGW8aciTKSj6n0qiqwByqAuE/eZnHhNjsmbE7fRXjarrMY/O1A8W5x0PkskIxPNJIlkZCJp9T6aVWsPz6d0kkVnxGE04zBYJssnqcBs16ttNoOpYr3h0nwEFAsJLSdVA8UgXKlL28ubhCv1zOdQHl+4Us8e/0JzpV5KKK7FWA8X1ngvoFhDc0FxQXklRVB+kWj6TQAkTLgtV2I3XVSEzflAcUGKkmc8dZKx5GtkJ2HbrPfRZLkMq1F7j8V8YVbOxRhKHmE0eZI86nJTs85Bk3U9DebVIFnnFacgLWhVFIWx9DhDyVOMp4aKxlw6dNSb22m2rMBnaiq+NrBwKJ6uTD7LiDzGUHKI0dQouWlLmw2SkSZzE02WFhrNTRgnYbSWUDxT8WySETnI2CQkT6+LDOp11G100WD2UW/2Um/2YtVbzggUa8eBZC7FRDpKIBUjkI4SSEcJZxKa/SXAabThNaqw7DHZ8RgdOA32qr6AOBNQPPtxhdJJk5CclollU0QzSfU+K5PJ5+aIosqmN2GfBGa7wYzDYMZmMGPXF9pMGDS+CKmUqFg+LSS09KoWikG4Uk+1azcLV+rpz0GjTbhSzxn/QnKlFlC8vCSgWEPzheKCUrlR/MlnyeTU8kVGXSMe6/WYDa1VQbEaX09eyeCX9+NP7SOvqAZDFn0DDdYd2AztJYN1tTO8yVyGUfkoo8kjZJVJwyzJSL15LY2WDZj1pfukqoFiNZ9J85C8zIjcy3DyNLFcuPi4WWelydJNs6Ubu8FdEyhW+6vtOSXHeGqcYXmYUXmU1DSTLgmJOlM9TZZmGszNOAzlZim1gOLp+SuKQjyXZDwVKN5i2XLYtOtt1Ju91JnUmWSXwVH8Oy81FFfqn8nlCWZiBFJRgpkYwXSUYDqmufwaCuWUVED2mOy4jQ7cRjtOo70m9ZQraSnjpHJZYlmZWFYmmkkRz6rgrJZRUn+uVF5qpow6fRGSbQYzNr2pCM6Fn11G9cspYbQlJLT0WggUC1fqQrt2s3Clnv4cKrQjXKlnjX8BuVKL5dPLSwKKNVR4Q40N/H8aUKy93DKj5Imk9xNIvkAeFWQdxvW4LVdh0JXDVyVjrunuzdl8ilF5P+PyoeLsrt3QTLP1MuzGZmAWOJ1juXVOyTIq9zCYPEoyVyh/JFFnaqfVth6noQFJkiqXiKoA4zPhUVEUItkwfYk+huV+MsqUi7DL4KHF0kWzpQOzvnSmuiIsV7E3Oa8o+FNhhuQRhuURotlSB2mHwU6zuYlmSxP15jp0kq6kRNR8zlsJyCrBO0A8k2IsFWRMDjKeChHMRMv6GHUG6k0e6szuyXtP2ZLrswHLiqKQyGYIpGOTtzjBdJxQOk5G0Z5VlZisPWy04zbZivdug13TGbr6UkfVQXEtIbpYbzibIpZJEc+miGZTxLOT9YcnaxBXem2my6TT8wdrrgYgGY3zkYvfIaBYSGgJVYTi+/+1vCRThQ/gwpW6NE55u3p3wbtSw6zQJ1yp535eF4IrdTIe5xO/JYy2losEFGtoIVBc8L/N5uME5BeIpg8BIGHEbdmBy7QNaZor83yguKBMPslwch8TqcPFJckOQytN1kuwGrQvovPdg6yW9hlmIHmEUGak2G43eGmxrsNrWqHpJj1fKC4op+jIKznGUsMMJnvxp0eL5lwAdaYmWixdNJpbMeiMNYFiKP3SIJqJMZwaZUQeYzw1UXJ+vaSn0dwweWvEbijdG15LKJ55TDqfwZ9SSyv55SD+dEhzr6vLYKfO7KHO5KbO7MZldGnOwC71DLLWc1YUhWhWJliA5IwKyqFMgnS+sju0RWdU6w6b7LiMag1ip9GGy2Ar37e8DKF4vnHS+SzxTFqF5OzkfS5NIjvVZtYZ+L3uSwEBxUJCZ0KLgWIQrtTa7VM/XtCu1DArFBfiC1fq2eOf767UAoqXlwQUa2gxUFyQnB3Bn/wVqZx6MTVILjzWq7EZVk8aZ80fitX4OtK5GGPyHgKpYxQ+etsMLTRYtuGYuax6AcZc8WyIoeQR/PJp8qgzW3rJRIN5JU2WtVgN7jnjzAbF05XOpxiRBxiU+wlnJortOnTUmZtpNHdQb2ot24dZCxdrgFQ+y1hqnBFZheTpy6wBbHobjeYGGswNNJjrMVQwCqsFFBdznTwmr+QJpaP402H8qRD+VEhzybUOHR6TE5/Jjc/kwmty4zY6gApuyUsIxWp/rb4K8VxmEpDjhCdBOZyOE9coFzVddr1ZBWSjFafBhtNgxWVU76fPMJ8LUFytBBQLCS29FgvFIFypy9tLf71gXalhTiguxD+vXKm1wHgRUAnntyu1gOLlJQHFGpoJxYqSLc7yzheKQb3YhdNHCMnPk1PUpbsmfRNey9UYDB3aceaxHDqdizIm7yWYOl6cObbo62mwbMNl7EaSdAt2qwbI5FOMyicYSZ4glY8V252GJposq/GZO8lTXqoH5g/FU/11JLIxhuU+huU+ErmppcQ6dPhMzTRaOqg3tWDUmWoGxdPzVBSFcCbCaGqMEXmMQDpQMosM4Da6qTepkOwz1RVLLC0FFJf3BzmXZiIVYiIdZiIdIpCOkNbY21sAZY9RhWSvyYXb6EQv6c8KFM8WJ53LEs4kirfItPvZZpcBzDrjZPkkFZIdBmvxZ7vBgl7SCSieIQHFQkKlqgUUg3ClLm0vb7ogXalhXlBciH9euVLPBONFQiWcv67UAoqXlwQUa2gmFMcSPwAM2Cw3kJfsmsdU+gifVSCvZIikXieSegMFFWTMhm7clqsw6htm9J//HuF0Ps548gCB1GGUyQxMOjf1li04TOvRVbHsWQviFEUhkB5hVD5GMD1I4eO+XjLhM62kwbIGm6Gu5JiFQPH088WyEUZT/YzIAyWALKHDa2qk3txOvalt/nuQqzTOyiuqm/VE2s9YapzxlJ9otnTPr1qH2EudqZ46UwNeo6+kDjHUHorL2xSi2SSBdJhAOkwwHSGYjpBRyt+JEhJOgx2PyY3H6MJjcuExujDrTRXjw9JD8Wxu2Kl8hkgmQSSTJJqdvM8kiGaTJHNp7QOnyaY345iEZcdk/WHHJDDbdVb0WiWl5ow6M38BxRfCICl0/qpWUAzClXqqXbv5gnOlhnlDMXD+uVJPB+MaQCWcX67U65vq2bVhtYDiZSYBxRqaDsV2e4xI/FHUj7pGzOYrMBovQZJKP1TPBsUF5fJxwqlXiKYPwOQMr9W4HpflCgw692T/6oyz8oqObD7JROoAgdQhcpOO0nrJgte8GY95EwbdFEBW61ZdgNlULq6Wi5JPkM5PGVbZ9HXUW9bgM63AoDMvCoqnK5uHeC7CqNzPWGqARNEMTJXbWF8EZJvBWVMonik5JzOWmsCfHmc8NU4yV16H2GMsQHI9XpMPna6yDX4toBjKYVP9UiFBIB0lmA4TTIcJZaKk8toQadGZcZtcuA1O3EYnbqMLl9FRLKt0tqB4LqOtTD476QatQnIkKxOb/DmaTc6z9rBpCpINFuwGKza9pfi7RWeq7D5bzF9A8YUwSAqdv6olFAtX6kK7dvMF50o9mZP2c6jQznnmSl0A4yVypS6C8RK5UhfBuMau1L88eorf2rKOOrtNQPEyk4BiDRXeAKHBL+ByWcnkRokmf04mNwCAXleH3bILo6GreEwW7Q/iOY2XN50LMS6/SDxzbLJFh8u0CY9lB5Kk4V7J/Iy5ckqGidQxxuSDpCeXPUvoqTOvpcGyCbPeXTUUaxpzZUYYlk8xkRooLt+W0FFnbqfevBKvsbWkFrEav9Ky5/lBaywbZUQeYkQeJpwNljxm1ztpMLfSYG7FY6wr+dBSCZYrgdd88oll45PllSYYT02QmAHJUFhu7aPOXEedyYdFP7UnuVoYrAT2lf5xZy4NT+ZSxdrDwXSUQDpCLFueMxScogvllJx4Ju/tBiu6OUpEnTVYLjlWQc5l1LrDGZloJkk0KxPLyESzSWJZueIWiOnSIU0Cslm916v3xRJKBgtWvan4mswrzyqutMlonD/a+jvLdqC8kAZJofNXRSj+1KPlUFzlDK9wpS6NU96u3l0ortSzxRKu1PPLs1L/kqXUbY1cu2UpXal97CrMGNcofi6fL91T/DYBxctFAoo1NBOKYfLDdmY/UflZFEU1PTIZNmC33oxOslQFxaDOLKeyowTkF0hm+wAVYB2mLXgsV5Utfa7GmEtR8gTSpxmT95PMTZlYuYztNFh2YDWUDxDzheKp/joyeZkxuYdR+RSJXKj4mFGyUG9eQbttM0adudhfM06VM7k5RUcyl2AsNcRoaohg2l+y/9comWgwt9BmXYnXVL8kUFwSR5GIZxMqIKf9+FMB4jmtOsR2fCYvjZYGOqza+8mXAopn5lpQJp8llIkRSkcJTN6H0lHN5dcAekmHy+DAZbTjMjrpsDXhMpZ+0FsOUDxXPoqiIOczRVCeqjus1iKOZ1Mk5jAAK0hCwqo3qeCsL9QeNmMzWFntLHcQFVAsJLS8tBRQDMKVWrt96sd5gfECoRiWhyv1bLHmmkEWrtRzx58OxiVLqWsUfzoYlyylrlH8ggQULy9VemsvG3V3dyNJUtntQx/6UMVjnnjiCdavX4/FYmHLli389Kc/XXQekiRhNV2Ex/FBzMaLAcjmhpEo37c7X5kNTbQ43kGL451Y9K0o5EhlB5EquAfPP1cdXvMq1rruYJXzNlxGFcIimYFiveNayKiz0GbbwMXet7LNcxut1nUYJDMZRWY0dRydtLjnUUlWvY0u22p2eK/lxobfYqt7B82WDgySkYySZkjuJZoNLcm5tWQ32Oi2d7DdezG3Nd/E7c03s917KSvt3bgM6gUknovTnxygN95/xvKaTUadgQazhzXODrb7NnJz805+p+Mm7mi7jusbL2WbZx3dtla8Rhc6dOSUPMFMhN7EMPvDxwhp1FY+FyRJEha9iQazixWORjZ7Orm8fg27mrdwR/sO3t19De9feSN3dV7Fb7Vexg2Nm9nuW81GVwddtgYazC5sejMSoKCQyKUYT0U4nRjnUGSA14IneS1w/Gw/zWWrr3/963R3d2OxWNi5cyevvPJKxb7//M//zDXXXIPX68Xr9bJr166y/rFYjA9/+MO0t7djtVrZuHEj3/jGN0r6XH/99WXjxx//8R8vyfMTWriWy1hfC1lsZna963I89U50eh16fW3HQpPZyI2/fSl1zW50Ogm9vrYf5QxGPTfcvpWmNs+SxNfrdVy7ayPtnerKrlrH1+kkrrp6HStWNiDpwGCobXxJkrh8+0rWrWlGkpYm/qVbOrlovTpDXOv4ANvWtrF9Y8dk/Np/VtvU3czVm7sn4+vm3I5Urda21nPjltVIgLHG7x+AFQ0+br1oDTpJwqjXV/yuR6hczz33HG9729tobW1FkiR+9KMflfU5fPgwb3/723G73djtdrZv305fX1/FmFrjuCRJ3H777cU+73vf+8oev/XWW6vKfeFEd4b06quvksvlir8fOHCAm2++md/93d/V7P/CCy9w991389BDD/Fbv/VbPPbYY9x555288cYbbN68edH56CQLDuvNWExbUJRMSe3hhcpqaMfi+B2S2T4UDDW7eEiShNPYitPYSioXJpQ+jU0/y/6XRZzHYfThMProtF1KKD1EKh/XrG9caxl1JlosHTRZutRSRhk/46khGswaM3VnSFa9hXZbG+22NkCtQxxMB5lIB7DrbXMcffYkSRJ2gxW7wUqrtaE4Y5tXFOLZBOFMbPIWx2s8f78t1Es6nEYrTmNhlUh5n7yikMylSWRl4rkUickZ5kQ2VWa6JqTq+9//Ph/72Mf4xje+wc6dO3n44Ye55ZZbOHr0KI2N5bMsu3fv5u677+bKK6/EYrHwhS98gbe85S0cPHiQtjb1f+tjH/sYzzzzDN/5znfo7u7m5z//OX/6p39Ka2srb3/724ux/uAP/oC/+qu/Kv5usy3f/8MLVcttrF+sCmCczYHDU/v3WwGMU8kMLq+2AehiVADjaEzGo7X8e5EqgHE4IuNdgvgFMN64uZ26JYhfAOM1a5qoX6L4l27ppLu9jvqZqwxqpG1r22hrcFPvq33+oIJxo8dBvXdp4q9trcdjt1DvsdccukEF49/Zvpk6l21J4p+visfjbN26lXvvvZff/u3fLnv85MmTXH311XzgAx/gM5/5DC6Xi4MHD2KxaJc+BfjBD35AOj3ljzMxMcHWrVvLxodbb72VRx55pPi72axdKaeSzrnl0/fddx9PPvkkx48f13yT3nXXXcTjcZ588sli2+WXX862bdvKZhAqSWv5dEFZJad5zEKWT2v3126vtq5xpSW0i91TPNW/kvFXhWXJNVw+rd2/ulJNtVw+rRlnVvfpCu1ncPn0/Pprx1/Oe4rPZD4Vz7vM9xSPDI3WZPl0c2tTVfnt3LmT7du387WvfQ2AfD5PR0cHH/nIR/jUpz415/G5XA6v18vXvvY13vve9wKwefNm7rrrLv7yL/+y2O/SSy/ltttu43Of+xygfsO8bds2Hn744SqfpdDZ1Jkc62u9fLpElWaJa+RKXfG8NXKlrmhsVSNXaqViHO3mhRhwaapGrtSzrbdc6PLp8v5V5lTlst7KcRa+fHp+8WvUf7nls0yWT9dirC/Eq3a8L0iSJH74wx9y5513Ftve9a53YTQa+fa3v73gnB5++GE+/elPMzw8jN2ufin4vve9j1AopDkzPV+dU1Ma6XSa73znO3zsYx+rOBC8+OKLfOxjHytpu+WWW2Z9kVKpFKnU1D7CcDgMQCQql/WtXKe4NlCs5X6s9q8VFFeAx4qQWCUUV4pTIyiu7CZdKb5m87KE4kpXWAHFs/evpFrlU62ddMXu1UBxTN2XvhTfWf5O6/sxYFxUjOxkablIpNQV3mw2a34zm06nef3117n//vuLbTqdjl27dvHiiy/O65yJRIJMJoPPN7X38Morr+Q///M/uffee2ltbWX37t0cO3aMr3zlKyXH/tu//Rvf+c53aG5u5m1vext/+Zd/KWaLl7HO9FifTGkYD54jUFwZQquMX3UcAcWzxp8lloBi0b8gOaFWc1muYz1UP97Ppnw+z09+8hM+8YlPcMstt7Bnzx5WrFjB/fffXwLOc+lb3/oW73rXu4pAXNDu3btpbGzE6/Vy44038rnPfY66uroKUTSknEP6/ve/r+j1emVwcLBiH6PRqDz22GMlbV//+teVxsbGisc88MADCupHVnETN3ETt2VzO3nyZM2un8lkUjFhqVluDoejrO2BBx7QPPfg4KACKC+88EJJ+8c//nFlx44d88r/T/7kT5SVK1cqyWSy2CbLsvLe975XARSDwaCYTCblX//1X0uO++Y3v6k89dRTyr59+5TvfOc7Sltbm/KOd7yjuhdP6IxKjPXiJm7idiHdlvNYD9WN99MFKD/84Q+Lvw8PDyuAYrPZlC9/+cvKnj17lIceekiRJEnZvXv3vJ7fyy+/rADKyy+/XNL+3e9+V/nxj3+s7Nu3T/nhD3+obNiwQdm+fbuSzWbn/dqdUzPF3/rWt7jttttobdW2p1+o7r///pJvnEOhEF1dXfT19eF2u2t6rjOhSCRCR0cH/f3956RT3LmeP5z7z0Hkf3YVDofp7OwsmRVdrCwWCyP+4ZJ9OYuKZzOXzY5V+63xfPX5z3+e733ve+zevbtk39FXv/pVXnrpJf7zP/+Trq4unnvuOT70oQ/R2trKrl27APjDP/zDYv8tW7bQ0tLCTTfdxMmTJ1m1atWS5Cu0OImxfv461691Iv+zK5H/2dW5MNZD7cb7fF5dVXvHHXfw0Y9+FIBt27bxwgsv8I1vfIPrrrtuzhjf+ta32LJlCzt27Chpf9e73lX8ecuWLVx00UWsWrWK3bt3c9NNN80rv3MGint7e3n66af5wQ9+MGu/5uZmRkdHS9pGR0dpbm6ueEylJQBut/uc/CcryOVyifzPss715yDyP7vSVVpauEB56zw1jTdf1dfXo9frq742A/zN3/wNn//853n66ae56KKLiu3JZJL/9b/+Fz/84Q+LDpQXXXQRe/fu5W/+5m+KUDxTO3fuBODEiRMCipehxFi/MJ3r1zqR/9mVyP/s6nwZ6+dSfX09BoOBjRs3lrRv2LCB3/zmN3MeH4/H+d73vldinFlJK1eupL6+nhMnTswbipd9SaaCHnnkERobG0vst7V0xRVX8Mtf/rKk7Re/+AVXXHHFUqYnJCQkJFRBJpOJSy+9tOTanM/n+eUvfznrtfmLX/win/3sZ3nqqae47LLLSh7LZDJkMpmyDxN6vb74bbSW9u7dC0BLy9lzqBeqLDHWCwkJCZ2fMplMbN++naNHj5a0Hzt2jK6urjmPf+KJJ0ilUvyP//E/5uw7MDDAxMREVWP9OTFTnM/neeSRR7jnnnswGEpTfu9730tbWxsPPfQQAH/2Z3/Gddddx9/+7d9y++23873vfY/XXnuNf/qnfzobqQsJCQkJoZZPuueee7jsssvYsWMHDz/8MPF4nPe///1A+bX8C1/4Ap/+9Kd57LHH6O7uZmRkBACHw4HD4cDlcnHdddfx8Y9/HKvVSldXF7/61a/4f//v//HlL38ZUEs/PPbYY7z1rW+lrq6Offv28dGPfpRrr722ZNZZaHlIjPVCQkJC57ZisRgnTpwo/t7T08PevXvx+Xx0dnby8Y9/nLvuuotrr72WG264gaeeeor/+q//Yvfu3cVjZl7vC/rWt77FnXfeWWaeFYvF+MxnPsM73/lOmpubOXnyJJ/4xCdYvXo1t9xyy/yTn/fu47Oon/3sZwqgHD16tOyx6667TrnnnntK2h5//HFl7dq1islkUjZt2qT85Cc/qep8siwrDzzwgCLL8mLSPmsS+Z99nevPQeR/dnWu519JX/3qV5XOzk7FZDIpO3bsUF566aXiYzOv5V1dXZqGH9PNPYaHh5X3ve99Smtrq2KxWJR169Ypf/u3f6vk83lFURSlr69PufbaaxWfz6eYzWZl9erVysc//nElHA6fqacsVIXEWF+9zvXnIPI/uxL5n12d6/lr6dlnn9Ucu6dfv7/1rW8pq1evViwWi7J161blRz/6UUkMrev9kSNHFED5+c9/XnbORCKhvOUtb1EaGhoUo9GodHV1KX/wB3+gjIyMVJX7OVenWEhISEhISEhISEhISEioVjpn9hQLCQkJCQkJCQkJCQkJCdVaAoqFhISEhISEhISEhISELlgJKBYSEhISEhISEhISEhK6YCWgWEhISEhISEhISEhISOiC1QUJxdFolPvuu4+uri6sVitXXnklr7766qzH7N69m0suuQSz2czq1at59NFHz0yyGqo2/927dyNJUtmtUOJkqfXcc8/xtre9jdbWViRJ4kc/+lHJ44qi8OlPf5qWlhasViu7du3i+PHjc8b9+te/Tnd3NxaLhZ07d/LKK6+cM/k/+OCDZX+P9evXn5X8f/CDH/CWt7yFuro6JEkq1nGdS0888QTr16/HYrGwZcsWfvrTn9Y+eZYm/0cffbTs9bdYLGc8/0wmwyc/+Um2bNmC3W6ntbWV9773vQwNDc0Z90y9/4WEzlWJsV6M9Wc7fzHWz19irNeWGOvPnC5IKP7gBz/IL37xC7797W+zf/9+3vKWt7Br1y4GBwc1+/f09HD77bdzww03sHfvXu677z4++MEP8rOf/ewMZ66q2vwLOnr0KMPDw8VbY2PjGck3Ho+zdetWvv71r2s+/sUvfpG///u/5xvf+AYvv/wydrudW265BVmWK8b8/ve/z8c+9jEeeOAB3njjDbZu3cott9zC2NjYOZE/wKZNm0r+Hr/5zW9qnjvMnX88Hufqq6/mC1/4wrxjvvDCC9x999184AMfYM+ePdx5553ceeedHDhwoFZpl+RX6/wBXC5Xyevf29tbi3Q186uUfyKR4I033uAv//IveeONN/jBD37A0aNHefvb3z5rzDP5/hcSOlclxnox1p/t/EGM9fOVGOvLJcb6M6yqCjidB0okEoper1eefPLJkvZLLrlE+d//+39rHvOJT3xC2bRpU0nbXXfdpdxyyy1LlmclLST/Qs2wYDB4BjKcXYDywx/+sPh7Pp9XmpublS996UvFtlAopJjNZuW73/1uxTg7duxQPvShDxV/z+VySmtrq/LQQw8tSd4F1Sr/Bx54QNm6desSZqqtmflPV09PjwIoe/bsmTPO7/3e7ym33357SdvOnTuVP/qjP6pBlpVVq/wfeeQRxe121zS3+Wi2/At65ZVXFEDp7e2t2Odsvf+FhM4VibH+7EqM9arEWL8wibFelRjrz6wuuJnibDZLLpcrWz5htVorfnv34osvsmvXrpK2W265hRdffHHJ8qykheRf0LZt22hpaeHmm2/m+eefX8o0562enh5GRkZKXl+3283OnTsrvr7pdJrXX3+95BidTseuXbvO+N9kIfkXdPz4cVpbW1m5ciXvec976OvrW+p0a6bl9D+xUMViMbq6uujo6OCOO+7g4MGDZzslAMLhMJIk4fF4NB9fTu9/IaHlKjHWi7G+lhJj/ZTEWF8bibF++emCg2Kn08kVV1zBZz/7WYaGhsjlcnznO9/hxRdfZHh4WPOYkZERmpqaStqampqIRCIkk8kzkXZRC8m/paWFb3zjG/zHf/wH//Ef/0FHRwfXX389b7zxxhnNXUuFvU5ar2+lfVB+v59cLlfVMUulheQPsHPnTh599FGeeuop/vEf/5Genh6uueYaotHokuZbK1X6nzjTr/9CtW7dOv7lX/6FH//4x3znO98hn89z5ZVXMjAwcFbzkmWZT37yk9x99924XC7NPsvp/S8ktFwlxnox1tdSYqyf0rk01oixXqgaGc52AmdD3/72t7n33ntpa2tDr9dzySWXcPfdd/P666+f7dTmpWrzX7duHevWrSv+fuWVV3Ly5Em+8pWv8O1vf/tMpS00Tbfddlvx54suuoidO3fS1dXF448/zgc+8IGzmNmFoSuuuIIrrrii+PuVV17Jhg0b+OY3v8lnP/vZs5JTJpPh937v91AUhX/8x388KzkICZ1PEmO9GOvPtsRYf3YlxnqhanTBzRQDrFq1il/96lfEYjH6+/t55ZVXyGQyrFy5UrN/c3Mzo6OjJW2jo6O4XC6sVuuZSLlE1eavpR07dnDixIklzHJ+am5uBtB8fQuPzVR9fT16vb6qY5ZKC8lfSx6Ph7Vr1y6Lv8l8VOl/4ky//rWS0Wjk4osvPmuvf2GQ7O3t5Re/+EXFb45heb3/hYSWs8RYL8b6WkmM9VM6l8caMdYLzaYLEooLstvttLS0EAwG+dnPfsYdd9yh2e+KK67gl7/8ZUnbL37xi5Jvn86G5pu/lvbu3UtLS8sSZjc/rVixgubm5pLXNxKJ8PLLL1d8fU0mE5deemnJMfl8nl/+8pdn/G+ykPy1FIvFOHny5LL4m8xHy/V/YqHK5XLs37//rLz+hUHy+PHjPP3009TV1c3afzm9/4WEzgWJsf7sjytirFclxvqzKzHWC82qs+30dTb01FNPKf/93/+tnDp1Svn5z3+ubN26Vdm5c6eSTqcVRVGUT33qU8rv//7vF/ufOnVKsdlsysc//nHl8OHDyte//nVFr9crTz311DmR/1e+8hXlRz/6kXL8+HFl//79yp/92Z8pOp1Oefrpp89IvtFoVNmzZ4+yZ88eBVC+/OUvK3v27Ck67n3+859XPB6P8uMf/1jZt2+fcscddygrVqxQkslkMcaNN96ofPWrXy3+/r3vfU8xm83Ko48+qhw6dEj5wz/8Q8Xj8SgjIyPnRP5//ud/ruzevVvp6elRnn/+eWXXrl1KfX29MjY2dsbzn5iYUPbs2aP85Cc/UQDle9/7nrJnzx5leHi4GOP3f//3lU996lPF359//nnFYDAof/M3f6McPnxYeeCBBxSj0ajs37//nMj/M5/5jPKzn/1MOXnypPL6668r73rXuxSLxaIcPHjwjOafTqeVt7/97Up7e7uyd+9eZXh4uHhLpVLFGGfz/S8kdK5KjPVirD/b+Yux/uzmL8Z6oWp0QULx97//fWXlypWKyWRSmpublQ996ENKKBQqPn7PPfco1113Xckxzz77rLJt2zbFZDIpK1euVB555JEzm/Q0VZv/F77wBWXVqlWKxWJRfD6fcv311yvPPPPMGcu3UCZi5u2ee+5RFEUtdfCXf/mXSlNTk2I2m5WbbrpJOXr0aEmMrq4u5YEHHihp++pXv6p0dnYqJpNJ2bFjh/LSSy+dM/nfddddSktLi2IymZS2tjblrrvuUk6cOHFW8n/kkUc0H5+e73XXXVfsX9Djjz+urF27VjGZTMqmTZuUn/zkJ+dM/vfdd1/xvdPU1KS89a1vVd54440znn+htITW7dlnny3GOJvvfyGhc1VirBdj/dnOX4z1Zzd/MdYLVSNJURRl7vlkISEhISEhISEhISEhIaHzTxf0nmIhISEhISEhISEhISGhC1sCioWEhISEhISEhISEhIQuWAkoFhISEhISEhISEhISErpgJaBYSEhISEhISEhISEhI6IKVgGIhISEhISEhISEhISGhC1YCioWEhISEhISEhISEhIQuWAkoFhISEhISEhISEhISErpgJaBYSEhISEhISEhISEhI6IKVgGKhmuv666/nvvvuO9tpVNTRo0dpbm4mGo1W7PPoo4/i8XjOXFJnUadPn0aSJPbu3VuTeE899RTbtm0jn8/XJJ6QkJCQ0PKTGOvPLYmxXkhodgkoFjonNDw8zLvf/W7Wrl2LTqerOBA/8cQTrF+/HovFwpYtW/jpT39a1uf+++/nIx/5CE6ns6Y5dnd38/DDD9c05lyqxSDX0dHB8PAwmzdvrklOt956K0ajkX/7t3+rSTwhISEhoQtDYqzXlhjrhYSWXgKKhc4JpVIpGhoa+D//5/+wdetWzT4vvPACd999Nx/4wAfYs2cPd955J3feeScHDhwo9unr6+PJJ5/kfe973xnKfPlLr9fT3NyMwWCoWcz3ve99/P3f/33N4gkJCQkJnf8SY/3SSYz1QkKzS0Cx0JIrGAzy3ve+F6/Xi81m47bbbuP48eMlff75n/+Zjo4ObDYb73jHO/jyl79csqSpu7ubv/u7v+O9730vbrdb8zx/93d/x6233srHP/5xNmzYwGc/+1kuueQSvva1rxX7PP7442zdupW2traSYx999FE6OzuL55+YmCh5/OTJk9xxxx00NTXhcDjYvn07Tz/9dPHx66+/nt7eXj760Y8iSRKSJAEwMTHB3XffTVtbGzabjS1btvDd7363JPa///u/s2XLFqxWK3V1dezatYt4PF58/P/+3//Lhg0bsFgsrF+/nn/4h38oPrZixQoALr74YiRJ4vrrr6/4N3jPe95DQ0MDVquVNWvW8MgjjwDl30C/733vKz6H6bfdu3cD6oeWv/iLv6CtrQ273c7OnTuLjxX0tre9jddee42TJ09q5iMkJCT0/7d39zE1xX8cwN+JW4drkbqFyDyUtModY61Ns1Bq18U8jjyFmJLRjFFD8Y8rzNMwf9iwZmQzVHI913XJaliXNipmSXnYhDzcfX5/+Dlz3OTnN0b1fm33j+/3fL/n+zn37u5zv/ec8z3UtjDXM9cTtWacFNMfN2/ePJSWluL06dOw2WwQEcTHx+PTp08AgOLiYixZsgRpaWkoLy/H2LFjsXnz5l8ex2azYcyYMZq62NhY2Gw2tXzt2jUMHz5c08ZutyMpKQkpKSkoLy/H6NGjkZ2drWnT2NiI+Ph4WK1WlJWVIS4uDiaTCY8fPwYA5OXlISAgAJs2bUJtbS1qa2sBAE1NTRg2bBjOnj2Le/fuYfHixUhMTMTNmzcBfLlUbObMmViwYAEcDgcuX76MyZMnQ0QAAEePHkVmZiY2b94Mh8OBLVu2ICMjA4cPHwYAdT8XLlxAbW0t8vLymn1vMjIyUFFRgfz8fDgcDuzbtw8+Pj7Ntt25c6d6DLW1tUhLS4PBYMDgwYMBACkpKbDZbMjNzcWdO3cwdepUxMXFaX789O3bF35+frh27dqPPi4iImpDmOuZ64laNSH6zaKjoyUtLU1ERCorKwWAFBcXq9sbGhpEURQ5fvy4iIhMnz5dEhISNPuYNWuWeHl5/XT/3+rUqZMcO3ZMU7dnzx4xGAxqOSIiQjZt2qRpM3PmTImPj9fUTZ8+/YfjfxUaGiq7du1Sy4GBgbJ9+/YW+4iIJCQkyKpVq0RE5Pbt2wJAqqurm207YMAAl2PKysqSyMhIERGpqqoSAFJWVtbimCaTSebPn9/stpb2cfLkSfH09JTr16+LiEhNTY24u7vL06dPNe1iYmJk7dq1mjqj0SgbNmxoMS4iImqdmOtbxlxP1LrwTDH9UQ6HAx07dsTIkSPVuh49eiA4OBgOhwPAlxUiR4wYoen3ffl3ef/+PTw9PV1i/DY+AIiMjNSUGxsbkZ6ejpCQEHTr1g16vR4Oh0P99/hHnE4nsrKyEBYWBm9vb+j1ehQWFqr9IiIiEBMTg7CwMEydOhUHDx7Eq1evAABv377Fw4cPkZSUBL1er76ys7N/+VKlpUuXIjc3F0OHDsXq1atRUlLy0z5lZWVITEzE7t27ERUVBQC4e/cunE4ngoKCNDFduXLFJSZFUfDu3btfipOIiFof5nrmeqLW7vfdbU/0l/n7+6Ourk5TV1dXB39/f7Xs4+OjJqJfkZ6ejqKiIlgsFgwcOBCKomDKlCn4+PFji/22bt2KnTt3YseOHQgLC0OXLl2wYsUKtZ+7uzuKiopQUlKC8+fPY9euXVi3bh3sdjs6d+4M4Ms9WN8ncnd391+Kf/z48aipqcG5c+dQVFSEmJgYLFu2DBaLpdn2z549w4QJE7Bw4UIkJSWp9Y2NjXB3d8ft27ddYtDr9Zryy5cv4evr+0txEhERtYS5/seY64n+fzxTTH9USEgIPn/+DLvdrta9ePECDx48wJAhQwAAwcHBuHXrlqbf9+X/RWRkJKxWq6auqKhI80+w0WhERUWFS4zfxgcAN27c0JSLi4sxb948TJo0CWFhYfD390d1dbWmjU6ng9PpdOlnNpsxe/ZsREREoH///qisrNS0cXNzQ1RUFDZu3IiysjLodDqcOnUKfn5+6NWrFx49eoSBAwdqXl8X3dDpdADgMm5zfH19MXfuXBw5cgQ7duzAgQMHmm3X1NQEs9mMwYMHIycnR7PNaDTC6XTi+fPnLjF9+4OkqakJDx8+hNFo/GlcRETUujHXM9cTtXY8U0x/1KBBg2A2m7Fo0SLs378fXbt2xZo1a9C7d2+YzWYAQGpqKkaNGoWcnByYTCZcvHgR+fn56qqOX31dMbGxsRH19fUoLy+HTqdTE25aWhqio6Oxbds2JCQkIDc3F6WlpZqEEBsbi4ULF8LpdKr/fi5fvhxRUVGwWCwwm80oLCxEQUGBy3Hk5eXBZDLBzc0NGRkZLg+s79evH65evYoZM2bAw8MDPj4+GDRoEE6cOIGSkhJ0794dOTk5qKurU2O22+2wWq0YN24cDAYD7HY76uvrERISAgDYuHEjli9fDi8vL8TFxeHDhw8oLS3Fq1evsHLlShgMBiiKgoKCAgQEBMDT07PZFTszMzMxbNgwhIaG4sOHDzhz5ow6xveSk5Px5MkTWK1W1NfXq/Xe3t4ICgrCrFmzMGfOHGzbtg1GoxH19fWwWq0IDw9HQkICgC8/NDw8PFwuTSMioraHuZ65nqjV+9s3NVPb8/3iGC9fvpTExETx8vISRVEkNjZWKisrNX0OHDggvXv3FkVRZOLEiZKdnS3+/v6aNgBcXoGBgZo2x48fl6CgINHpdBIaGipnz57VbP/06ZP06tVLCgoKNPWHDh2SgIAAURRFTCaTWCwWzeIbVVVVMnr0aFEURfr06SO7d+92OU6bzSbh4eHi4eEhX79aL168ELPZLHq9XgwGg6xfv17mzJkjZrNZREQqKiokNjZWfH19xcPDQ4KCgjQLeoiIHD16VIYOHSo6nU66d+8uo0aNkry8PHX7wYMHpU+fPtKhQweJjo5u9jPJysqSkJAQURRFvL29xWw2y6NHj9RjwzeLbwQGBjb7Xl+6dElERD5+/CiZmZnSr18/6dSpk/Ts2VMmTZokd+7cUcdbvHixJCcnNxsLERG1fsz1zPXM9dSWuIn8dz14on/IokWLcP/+/T+yzP+ePXtw+vRpFBYW/vZ9E9DQ0IDg4GCUlpaql34RERF9j7m+9WKup7aGl0/TP8FisWDs2LHo0qUL8vPzcfjwYc2D63+n5ORkvH79Gm/evEHXrl3/yBjtWXV1Nfbu3cskSUREGsz1bQdzPbU1PFNM/4Rp06bh8uXLePPmDfr374/U1FQsWbLkb4dFREREvwlzPRH9qzgpJiIiIiIionaLj2QiIiIiIiKidouTYiIiIiIiImq3OCkmIiIiIiKidouTYiIiIiIiImq3OCkmIiIiIiKidouTYiIiIiIiImq3OCkmIiIiIiKidouTYiIiIiIiImq3/gOG9SaDLikyzQAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def L(N, D):\n", " \"\"\" \n", " Approximates loss given N parameters and D dataset size (in tokens),\n", " per Chinchilla paper.\n", " \"\"\"\n", " E = 1.69 # entropy of natural language, limit of infinite model on infinite data\n", " A = 406.4\n", " B = 410.7\n", " alpha = 0.34\n", " beta = 0.28\n", " return A / (N ** alpha) + B / (D ** beta) + E\n", "\n", "ns = 10 ** np.arange(7, 11, step=2**-4) # model sizes from 10M to 100B\n", "ds = 10 ** np.arange(9, 12, step=2**-4) # dataset sizes from 1B to 1T\n", "plt.figure(figsize=(12, 5))\n", "plt.subplot(121)\n", "# create a 2D countour plot of loss L as a function of model size and dataset size in ns,ds\n", "loss2d = np.log10(np.array([[L(n, d) for d in ds] for n in ns]))\n", "plt.imshow(loss2d, extent=[9, 12, 7, 11], origin='lower', alpha=0.5)\n", "plt.contour(loss2d, levels=30, extent=[9, 12, 7, 11], origin='lower')\n", "plt.xlabel('log10(dataset size)')\n", "plt.ylabel('log10(model size)')\n", "plt.title('loss')\n", "plt.colorbar()\n", "# plot the compute for each point, which is a deterministic function: flops = 6*N*D\n", "plt.subplot(122)\n", "compute2d = np.log10(np.array([[6*n*d for d in ds] for n in ns]))\n", "plt.imshow(compute2d, extent=[9, 12, 7, 11], origin='lower', alpha=0.5)\n", "plt.contour(compute2d, levels=30, extent=[9, 12, 7, 11], origin='lower')\n", "plt.xlabel('log10(dataset size)')\n", "plt.ylabel('log10(model size)')\n", "plt.title('log10 flops')\n", "plt.colorbar()" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Ok so given any N,D we can estimate both: 1) the loss, and 2) the total flops. Now we want to solve the following problem: Given a specific budget of flops C, find: N_opt, D_opt = argmin_{FLOPs(N,D) = C} L(N, D). i.e. how big of a model should we train and for how many tokens?" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "best model size: 316.23M\n", "best dataset size: 11.65B\n" ] }, { "data": { "text/plain": [ "Text(0, 0.5, 'loss')" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAS8AAAEqCAYAAABEE9ZrAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAuRElEQVR4nO3dd3xUVf7/8dekTdpMGpCEFEACCQETQhRIQEAFqVJ2XV0WBV3Erx1FccV1v7qiht8itoW14E/UXVlAFNgfRdpSFEJvoQiElkIKhJDJJKTN3N8fQ0aiBEKYzJ3yeT4e9/FI7tyZ+SRh3px77rnnaBRFURBCCCfjoXYBQgjRHBJeQginJOElhHBKEl5CCKck4SWEcEoSXkIIpyThJYRwSl5qF2BvZrOZs2fPotPp0Gg0apcjhPgFRVEoLy+nbdu2eHg03r5yu/A6e/YsMTExapchhLiO3NxcoqOjG33c7cJLp9MBll+MXq9XuRoXVlEBbdtavj57FgIC1K1HOA2DwUBMTIz1s9oYtwuv+lNFvV4v4dWSPD1//lqvl/ASN+x63TrSYS+EcEoSXkIIpyThJYRwShJeQginJOElhHBKEl5CCKck4SWEaHGf/3iKZfvyMVTV2uw1JbyEEC2q1mRm1pqjTF6wj5ySSpu9roSXEKJF7cu9SEWNidAAHxIjbTcwXMJLCNGifjh+HoD0jmF4eNhuMgQJLyFEi/rx+DkA7ujUyqavK+ElhGgxhqpa9ueVAdC3U2ubvraElxCixWSeKMFkVrilVQBRwX42fW0JLyFEi/nxcn9XXxufMoKElxCiBf2YbQmvPnESXkIIJ5FXWsmp8xV4emhI6xhm89eX8BJCtIgtl1tdydFB6H29bf76El5CiBbxg7W/y7ZXGetJeAkhbM5sVth6ogSw/fiuehJeQgibO1xg4EJFDYFaL7rHBLfIe0h4CSFsbtMxy6j63reE4u3ZMjEj4SWEsLn68Oof36bF3kPCSwhhU+VVtew5UwpA/xbqrAcJLyGEjW09UULd5VuCYsP8W+x9JLyEEDZVf8rYr3PLtbpAwksIYUOKorDpaH1/l4SXEMJJnDhXQf7FS/h4edC7g+1vCbqShJcQwmbqTxl7dQjFz8ezRd9LwksIYTPWIRIt3N8FEl5CCBupqjWx/aTllqABLdzfBRJeQggb2XayhOo6M1HBfnRsHdji7yfhJYSwiSuHSGg0tlslqDESXkIIm9jwUzFgn/4ucKDwmjFjBhqNhueee67RY7744gs0Gk2DzdfX135FCiGu6uQ5I6dLKvH21LTIfPVX42WXd7mOnTt38sknn5CUlHTdY/V6PUePHrV+b4/mqRDi2v57udXV+5YwArX2iRXVW15Go5Fx48Yxd+5cQkJCrnu8RqMhIiLCuoWHh9uhSiHEtdSH150tOIvEL6keXk899RTDhw9n4MCBTTreaDTSrl07YmJiGDVqFIcOHbrm8dXV1RgMhgabEMJ2yqtq2XHqAgB3JbhJeC1YsIA9e/aQkZHRpOPj4+P5/PPPWbZsGf/6178wm82kp6eTl5fX6HMyMjIICgqybjExMbYqXwiBZW3G+lkk2rcKsNv7qhZeubm5TJ48ma+//rrJne5paWmMHz+e7t27079/f7777jtat27NJ5980uhzpk2bRllZmXXLzc211Y8ghODnU0Z7trpAxQ773bt3U1xcTI8ePaz7TCYTmzdvZvbs2VRXV+Ppee17o7y9vUlJSSE7O7vRY7RaLVqt1mZ1CyF+ZjYrbDjqZuF19913k5WV1WDfI488QkJCAn/605+uG1xgCbusrCyGDRvWUmUKIa4hK7+M80bLQhu3tQ+163urFl46nY5u3bo12BcQEEBYWJh1//jx44mKirL2ib3xxhv07t2buLg4Ll68yMyZMzlz5gyPPvqo3esXQvx8ynhHp1b4eNm3F8ohxnk1JicnBw+Pn38hpaWlTJo0icLCQkJCQkhNTWXr1q0kJiaqWKUQ7kutU0YAjaIoit3fVUUGg4GgoCDKysrQ6/Vql+O6Kiog8PLNuUYjBNjvKpSwj2JDFT3fXo9GAzteGUhrnW36lpv6GVV9nJcQwjmtO2JpdXWPCbZZcN0ICS8hRLOsPVwIwMAu6tzlIuElhLhhFdV1bDlhmXjwnkQJLyGEk/jh+Dlq6sy0C/Mnrk3LTzx4NRJeQogbtuZwEQCDuoSrNrOLhJcQ4obUmczWiQcHqnTKCBJeQogbtPtMKaWVtQT7e3Nbu+tPY9VSJLyEEDdk7eVTxrvi2+DlqV6ESHgJIZpMURTWHrnc36XiKSNIeAkhbkB2sZEzJZX4eHpwh50W2miMhJcQosnqrzKmdbTfXPWNkfASQjTZ9wcto+qHdItQuRIJLyFEE+WVVpKVX4aHRv3+LpDwEkI00epDllPG29uH0ipQ/dmJJbyEEE3y/cECwDFOGUHCSwjRBMXlVew6UwrA4K4SXkIIJ7H2cBGKAskxwbQN9lO7HEDCSwjRBNarjA7S6gIJLyHEdZRV1pJ5ee6uwV3Vv8pYT8JLCHFN644UUWdWiA/XcUtrdebuuhoJLyHENa26fMo42EGuMtaT8BJCNKq8qpbNx84BMFTCSwjhLNYdKaLGZKZj6wASInRql9OAhJcQolErDlgGpg5PaqvadM+NkfASQlxV2aVaNh87D8CIpEiVq/k1CS8hxFWtO2w5ZezUJpDO4Y51yggSXkKIRqzIqj9ldLxWF0h4CSGuoqyylh+OW64yDr9VwksI4STWHC6k1mQZmNrJAU8ZQcJLCHEV9aeMwxy01QUSXkKIX7hYWcOPxy1XGYcnOdbA1CtJeAkhGliZVUidWSEhQkdcG8c8ZQQJLyHELyzblw/A6JQolSu5NgkvIYTV2YuX2HH6AgD3JrdVuZprk/ASQlgtP3AWRYGe7UOJcpAZUxsj4SWEsPrP/rMAjOzu2K0ukPASQlyWXWzkYL4BLw+NQw+RqCfhJYQAfm519evcmtAAH5WruT4JLyEEiqLwn8tXGUc5wSkjSHgJIYADeWWcLqnEz9uTgV0cZ5GNa5HwEkKwZK+l1TUoMZwArZfK1TSNhJcQbq7WZLb2d43p4dgDU68k4SWEm9t49BwXKmpordNyR1wrtctpMgkvIdzcd3vyABjdvS1ens4TCQ5T6YwZM9BoNDz33HPXPO6bb74hISEBX19fbr31VlauXGmfAoVwQRcra1h/pBiA3/SIVrmaG+MQ4bVz504++eQTkpKSrnnc1q1bGTt2LBMnTmTv3r2MHj2a0aNHc/DgQTtVKoRrWX6ggBqTmS6RerpE6tUu54aoHl5Go5Fx48Yxd+5cQkJCrnnsBx98wJAhQ5g6dSpdunRh+vTp9OjRg9mzZzf6nOrqagwGQ4NNCGFRf8r4WyfqqK+neng99dRTDB8+nIEDB1732MzMzF8dN3jwYDIzMxt9TkZGBkFBQdYtJibmpmsWwhWcPGdkT85FPD00TnEv4y+pGl4LFixgz549ZGRkNOn4wsJCwsMbDqALDw+nsLCw0edMmzaNsrIy65abm3tTNQvhKurHdvXr1Io2Ol+Vq7lxqo1Gy83NZfLkyaxduxZf35b7xWm1WrRabYu9vhDOyGRW+G6PJbzGOFlHfT3Vwmv37t0UFxfTo0cP6z6TycTmzZuZPXs21dXVeHp6NnhOREQERUVFDfYVFRUREeG482wL4Yi2ZJ8n/+Il9L5e3JPoHLcD/ZJqp4133303WVlZ7Nu3z7rddtttjBs3jn379v0quADS0tJYv359g31r164lLS3NXmUL4RIW7bJ0n4xOicLX+9efNWegWstLp9PRrVu3BvsCAgIICwuz7h8/fjxRUVHWPrHJkyfTv39/Zs2axfDhw1mwYAG7du3i008/tXv9Qjir0ooa1hyynMHcf5vzXsBS/WrjteTk5FBQUGD9Pj09nfnz5/Ppp5+SnJzM4sWLWbp06a9CUAjRuGX78qkxmUmM1NMtKkjtcppNoyiKonYR9mQwGAgKCqKsrAy93rkG5TmVigoIDLR8bTRCQIC69QjAMm/XsA9/5EiBgb+O7MqE9PZql/QrTf2MOnTLSwhhW4fOGjhSYMDHy8NpJh1sjISXEG5k4U5LR/3grhEE+zv+VM/XIuElhJuoqjVZF5R9wIk76utJeAnhJlYcKMBQVUdUsB/pHcPULuemSXgJ4Sbm78gB4A+9YvHw0Khczc1rVnh9+eWXrFixwvr9Sy+9RHBwMOnp6Zw5c8ZmxQkhbOOnQgO7z5Ti5aHhd7c55+1Av9Ss8Hr77bfx87MsBZ6ZmcmcOXP429/+RqtWrXj++edtWqAQ4ubN325pdQ1KDHfKm7Cvplkj7HNzc4mLiwNg6dKl/Pa3v+Wxxx6jT58+DBgwwJb1CSFuUmVNHUsu34Q9rlc7lauxnWa1vAIDAykpKQFgzZo1DBo0CABfX18uXbpku+qEEDft/+0/S3l1He3C/F2io75es1pegwYN4tFHHyUlJYVjx44xbNgwAA4dOkT79u1tWZ8Q4iZ9ffmU8Q89XaOjvl6zWl5z5swhLS2Nc+fO8e233xIWZknz3bt3M3bsWJsWKIRovqy8Mg7kleHj6cF9qa7RUV+vWS2v4ODgq84b/9e//vWmCxJC2M6/tlmu/g/uFkFYoGtNytmsltf333/Pjz/+aP1+zpw5dO/enT/84Q+UlpbarDghRPNdrKxh6eUR9ePTXKejvl6zwmvq1KnWVXiysrJ44YUXGDZsGKdOnWLKlCk2LVAI0TyLduVSXWeZ+ua2dtdemcsZNeu08dSpUyQmJgLw7bffMmLECN5++2327Nlj7bwXQqjHZFb4KtNyyjghvR0ajet01NdrVsvLx8eHyspKANatW8c999wDQGhoqKyLKIQD2PBTMXmllwjy82ZksvOtydgUzWp59e3blylTptCnTx927NjBwoULATh27BjR0a51RUMIZ/Rl5mkAfn97DH4+zjlH/fU0q+U1e/ZsvLy8WLx4MR999BFRUZZkX7VqFUOGDLFpgUKIG5NdbOSH4+fRaODB3q7XUV+vWS2v2NhYli9f/qv977333k0XJIS4OfXDI+5OaENMqL/K1bScZq8eZDKZWLp0KUeOHAGga9eujBw58qpLlgkh7MNQVcs3l5c1G5/WXt1iWlizwis7O5thw4aRn59PfHw8ABkZGcTExLBixQo6duxo0yKFEE2zcEcuFTUmOrUJ5I5OrdQup0U1q8/r2WefpWPHjuTm5rJnzx727NlDTk4OHTp04Nlnn7V1jUKIJqgzmZm35RQAj97RwSWHR1ypWS2vTZs2sW3bNkJDQ637wsLCmDFjBn369LFZcUKIplt1sJCzZVWEBfgwqrtrDo+4UrNaXlqtlvLy8l/tNxqN+Pg494okQjgjRVH47EdLq+vB3u3w9Xb9vudmhdeIESN47LHH2L59O4qioCgK27Zt4/HHH2fkyJG2rlEIcR17ckrZn3sRHy8Plx4ecaVmhdeHH35Ix44dSUtLw9fXF19fX9LT04mLi+P999+3cYlCiOv57AdLq2tM9yha61xr9ojGNHtKnGXLlpGdnW0dKtGlSxfr1NBCCPvJKalk9aFCACbe0UHlauynyeF1vdkiNmzYYP363XffbX5FQogbMveHk5gV6Ne5NZ3DdWqXYzdNDq+9e/c26ThXvzwrhCM5b6xm0eVBqU/0d6/xlU0OrytbVkIIx/DFltNU15npHhNM71tCr/8EFyIrZgvhpIzVdXx1efaIx/t3dLuzHgkvIZzUv7fnYKiq45bWAdyTGK52OXYn4dWI7GIj0747YP2fTQhHUl1n4rMfTwLweL+OLrWkWVNJeDVi1+kL/HtHLp9uPonJrKhdjhANLN2bT5Ghmgi9L6NS2qpdjiokvBoxOiWKEH9v8kovsfZwkdrlCGFVZzLzj40nAJjYtwNaL9e/FehqJLwa4evtydiesQDWO/WFcAT/2X+WMyWVhAb4MK53rNrlqEbC6xoeSmuHp4eG7acucOhsmdrlCIHJrDD7v9mAZdobf59mzyfq9CS8riEyyI+h3SIAmLfltLrFCAEsP3CWk+crCPb3dvmZUq9Hwus6HuljuVfsP/vOct5YrXI1wp2Zr2h1TezTgUCt+7a6QMLrunrEBpMcHUSNycz87TlqlyPc2PeHCjlebETn68WEPu3VLkd1El7XodFo+GNfS+vrn9vOUFNnVrki4Y7MZoUP1x8HLGcDel9vlStSn4RXEwztFkm4Xsu58mqW7ctXuxzhhlZkFfBTYTk6rRd/lFYXIOHVJD5eHvzxct/Xp5tPYpZBq8KO6kxm3lt3DIBH77iFYH+Zah1UDq+PPvqIpKQk9Ho9er2etLQ0Vq1a1ejxX3zxBRqNpsHm6+trl1rH9opFp/XieLGRDUeL7fKeQgAs3XeWk+csVxj/2Le92uU4DFXDKzo6mhkzZrB792527drFXXfdxahRozh06FCjz9Hr9RQUFFi3M2fO2KVWva83f7g8IPCTTSft8p5C1NSZ+WC9pdX1eP+O6KSvy0rV8Lr33nsZNmwYnTp1onPnzrz11lsEBgaybdu2Rp+j0WiIiIiwbuHh9rub/o99OuDtqWHH6QvsPlNqt/cV7mvRrlxyL1yiVaCWCW4+ruuXHKbPy2QysWDBAioqKkhLS2v0OKPRSLt27YiJibluKw2guroag8HQYGuucL0vY1Is6+F9sulEs19HiKaoqjVZx3U9fWdH/Hzc8x7GxqgeXllZWQQGBqLVann88cdZsmQJiYmJVz02Pj6ezz//nGXLlvGvf/0Ls9lMeno6eXl5jb5+RkYGQUFB1i0mJuam6n2s3y0ArD1SRHax8aZeS4hr+SrzNIWGKtoG+TK2l/vew9gYjaIoql46q6mpIScnh7KyMhYvXsxnn33Gpk2bGg2wK9XW1tKlSxfGjh3L9OnTr3pMdXU11dU/j4w3GAzExMRQVlaGXq9vVs2TvtrF2sNF3JcazTu/S27Wa7i8igoIDLR8bTRCQIC69TiZi5U19PvbBgxVdcy8L4nf3XZz/+k6E4PBQFBQ0HU/o6q3vHx8fIiLiyM1NZWMjAySk5P54IMPmvRcb29vUlJSyM7ObvQYrVZrvZpZv92sp+60LPG2ZG8+OSWVN/16QvzSPzaewFBVR0KEjt/0iFa7HIekenj9ktlsbtBSuhaTyURWVhaRkZEtXFVD3WOC6de5NSazwj82Nh6cQjRHXmklX1yeCOBPQxPwdMNZUptC1fCaNm0amzdv5vTp02RlZTFt2jQ2btzIuHHjABg/fjzTpk2zHv/GG2+wZs0aTp48yZ49e3jwwQc5c+YMjz76qN1rn3y3pfW1eHceeaXS+hK28+6aY9SYzKR3DGNA59Zql+OwVL0tvbi4mPHjx1NQUEBQUBBJSUmsXr2aQYMGAZCTk4OHx8/5WlpayqRJkygsLCQkJITU1FS2bt3apP4xW0ttF0qfuDC2ZJfw0cYTvDXmVrvXIFzPobNlLLl8C9q0oV3cbkWgG6F6h729NbUzsCm2nyzhgU+34e2pYdPUO2kb7GejKl2AdNjfMEVRGPfZdraeKGFkcls+HJuidkmqcJoOe2fW65YwenUIpdak8LGM+xI3ac3hIraeKMHHy4Opg+PVLsfhSXjdpMl3dwJgwY5c6fsSzVZdZ+LtlUcAmHRHB2JC/VWuyPFJeN2ktI5hpN0SRo3JbJ1vSYgbNW/Lac6UVNJGp+XJAXFql+MUJLxukkajYeoQSxN/8e48TpyTUffixpwrr7beBvSnIQkEuPn0zk0l4WUDPWJDGNglHLMC7649pnY5wsm8s/ooxuo6kmOCrffOiuuT8LKRFwd3RqOBFQcKOJgvy6SJptmbU8qi3bkA/O+IRDxkQGqTSXjZSEKEnlHJlmXX31lzVOVqhDMwmRX+suwgigK/7RFNarsQtUtyKhJeNvT8oM54eWjYePQcmSdK1C5HOLivt5/hYL4Bva8X04YlqF2O05HwsqF2YQH84fLUJW+tPCxz3YtGnSuvZuZqSwt96pAEWgVqVa7I+Uh42djkuzuh03pxMN/Akr2y0pC4uoyVRyivqiMpOog/9JS5uppDwsvGwgK1PH2XZZzOzNVHqaypU7ki4WgyT5Tw3d58NBqYPqqbzBrRTBJeLWBCenuiQ/woNFQxd/MptcsRDqSq1sS07w4AMK5XLMkxweoW5MQkvFqAr7cnLw+1dMB+vOkERYYqlSsSjuL9dcc5XVJJhN6Xl4ZIJ/3NkPBqIcNvjaRHbDCXak38n1U/qV2OcAAH88uY+4Nl2bzpo7uhl2XMboqEVwvRaDT8771d0Wjgu7357Dx9Qe2ShIrqTGZe/u4AJrPC8KRIBiXab8k+VyXh1YK6xwTz+9stCyf8ZelB6kxmlSsSapn7wykO5hsI8vPm9Xu7ql2OS5DwamFTBycQ7O/NT4Xl/HObfVb3Fo7lWFE5712+5/XV4V1orZMxXbYg4dXCQgN8rBPLvbvmGMXl0nnvTmpNZqYs2keNycxdCW24L1VWArIVCS87+P3tsSRFB1FeXceMldJ5705m/zebg/kGgv29mfGbW2VOehuS8LIDTw8Nb4zqZu2833zsnNolCTs4kHeR2Rss83RNH9WNNnpflStyLRJedtI9JpgJae0BeGVJFhXVMvLelV2qMTFl0X5MZoURSZHce3nGEWE7El52NHVwPFHBfuSVXmLWGpm00JW9ueIw2cVGWuu0TB/VTe1yXJKElx0FaL14a4zlH/K8rafYm1OqckWiJXx/sJCvt+eg0cB793cnJMBH7ZJckoSXnQ2Ib8NvUqJQFHj52yxq6mTslys5e/ESf/rWcu/iY/1uoW+nVipX5LokvFTwlxGJhAX4cLSonPfXyemjqzCZFZ5buI+yS7UkRwfxwiBZe7ElSXipICTAx3r6+PGmE+ySW4dcwgfrjrHj1AUCfDz5cGwKPl7y8WpJ8ttVyZBukfy2RzRmBaYs2o9Rrj46tQ0/FfPh5eXL3hpzK+3CAlSuyPVJeKnotZGJRAX7kXOhkjeXH1a7HNFMuRcqeW7hPgAe6t2O0bJ8mV1IeKlI7+vNrPuT0Whgwc5c1hwqVLskcYOqak08+fUeSz9XTDCvjuiidkluQ8JLZb1vCWPSHbcA8NK3B8i/eEnlikRTKYrC6/85RFZ+GSH+3vxjXA+0Xp5ql+U2JLwcwAv3dCYpOoiLlbU8M38PtTJ1jlP4cutpFuzMRaOB93+fQlSwn9oluRUJLweg9fJk9tge6Hy92JNzkXdWy6K1ju7H4+eZvuIIANOGJtC/c2uVK3I/El4OIjbMn5n3JQHwyeaT/PenIpUrEo05db6CJ7/ejcms8JseUdbTfmFfEl4OZEi3SB5Obw/A8wv3c6akQt2CxK9crKxh4pc7MVTVkRIbzNtjZJobtUh4OZhpwxJIjgmm7FItk77aJeO/HEhVrYnHvtrNyXMVRAb58slDqfh6Swe9WiS8HIzWy5NPH0qljU7LsSIjLyzah9msqF2W2zObFV5YtJ8dpy+g03ox75HbaaOT+bnUJOHlgML1vnz8UCo+nh6sPlTEh/89rnZJbu+tlUdYkVWAt6eGT8ankhChV7sktyfh5aB6xIbw5uX7H99fd5zlB86qXJH7+mTTCf7vj5aVz9/5XTLpHWWmCEcg4eXA7r8thj/26QDAlIX72X6yROWK3M8/t50h4/KiwS8PTWBUd7n1x1FIeDm4Pw/vwuCu4dSYzEz6ahfZxeVql+Q2vt2dx1+WHgTgyQEdebx/R5UrEleS8HJwnh4aPvh9Cj1igzFU1THh850UG2T5tJa2KquAqYv3A/Bwenvr8nXCcUh4OQFfb08+m3A7HVoFkH/xEuM/30FpRY3aZbms5QfO8vS/92JW4Hep0fzviEQZy+WAJLycRGiAD18+0pM2Oi0/FZbz0OfbKbtUq3ZZLmfJ3jye/fdeTGaFMSlRzPhtEh4eElyOSNXw+uijj0hKSkKv16PX60lLS2PVqlXXfM4333xDQkICvr6+3HrrraxcudJO1aovNsyf+ZN6ERbgw8F8Aw/P2yGDWG1o0a5cpizaj1mB+2+L5p3fJeMpweWwVA2v6OhoZsyYwe7du9m1axd33XUXo0aN4tChQ1c9fuvWrYwdO5aJEyeyd+9eRo8ezejRozl48KCdK1dPXBsd/5zYiyA/b/bmXOSP83ZKgNnA3M0neWnxARQFHuwdy4zfJElwOTiNoigONXw7NDSUmTNnMnHixF899sADD1BRUcHy5cut+3r37k337t35+OOPm/T6BoOBoKAgysrK0Oudd6DhgbyLjJu7nfLqOpJjgvnykdsJ9negJbYqKiAw0PK10QgBjjktstms8NbKI9ZxXBP7duDV4V2kj0tFTf2MOkyfl8lkYsGCBVRUVJCWlnbVYzIzMxk4cGCDfYMHDyYzM7PR162ursZgMDTYXEFSdDBfT+pFiL83+3Mv8sAn2ygul6uQN6K6zsRzC/dZg+uVYQkSXE5E9fDKysoiMDAQrVbL448/zpIlS0hMTLzqsYWFhYSHhzfYFx4eTmFh49MnZ2RkEBQUZN1iYmJsWr+akqKDWfg/abTRaTlaVM79H2eSU1KpdllO4byxmgc/285/9p/Fy0PDew8k81i/jhJcTkT18IqPj2ffvn1s376dJ554ggkTJnD4sO0Wo5g2bRplZWXWLTc312av7Qg6h+tY/Hg60SF+nC6pZMw/trD7jCyldi2HzpYxavYWdp4uRaf14vOHb2dMSrTaZYkbpHp4+fj4EBcXR2pqKhkZGSQnJ/PBBx9c9diIiAiKihpO0ldUVERERESjr6/Vaq1XM+s3VxMb5s+3T6TTLUpPSUUNY+duZ9m+fLXLckgrDhRw30eZ5F+8RIdWASx5qg/9ZBZUp6R6eP2S2Wymurr6qo+lpaWxfv36BvvWrl3baB+ZOwnX+7Lof9IYlBhOTZ2ZyQv28e6ao5hkOh3A0r/12rKDPDV/D5dqTdzRqRVLn+xDXJtAtUsTzeSl5ptPmzaNoUOHEhsbS3l5OfPnz2fjxo2sXr0agPHjxxMVFUVGRgYAkydPpn///syaNYvhw4ezYMECdu3axaeffqrmj+Ew/H28+PjBVGasOsLcH07x4X+z2Zt7kfcf6E5YoFbt8lRz+nwFT/97DwfzLRdr/qffLUwdHI+Xp8P93y1ugKrhVVxczPjx4ykoKCAoKIikpCRWr17NoEGDAMjJycHD4+d/YOnp6cyfP59XX32VV155hU6dOrF06VK6deum1o/gcDw9NPx5eCJdIvW8siSLH46fZ/iHPzJnXAqp7ULVLs+uFEVh0a5cpi8/grG6jhB/b969vzt3JrRRuzRhAw43zquluco4r6Y4WljOE19bpi329NDw1ICOPH1XJ3y87NDiUHmcV0HZJV7+NotNx84BcHv7ED4cm0JkkCxP5uicbpyXsL34CB3/ebovo7q3xWRW+PC/2Yyes4WfCl1jrNvVmM0K/96Rwz3vbWbTsXP4eHnwyrAEFjyWJsHlYqTl5SZWHCjg1aVZlFbW4u2p4bF+t/DUnXH4+7RQz4EKLa/9uRf532UH2Z9XBkByTDCzfpdEXBtdi7+3sJ2mfkYlvNxIcXkVr3x3kHVHLMNN2gb58pcRiQzpFmH7wZl2DK+Csku8v/Y4i3bnoiig03rx/KDOjE9rJ53yTkjCqxHuHF5g6cRefaiI6csPk3/xEgA9O4Ty4j3x9Oxgww59O4RXibGajzae4KttZ6ipMwPwm5QoXh6WICv7ODEJr0a4e3jVu1Rj4qNNJ/h40wnrB/+OTq2YMqgzKbEhN/8GLRheeaWVzNtymgU7cqioMQGWAP7TkHi3u6LqiiS8GiHh1dDZi5eYvSGbRTtzqbs8oDW1XQiP9GnP4K4ReDf3tMvG4aUoCntySvkq8wzLDxRYB992i9IzdXAC/Tq1kvsSXYSEVyMkvK4up6SSv//3OEv35VNrsvyTiAzyZUxKFKNTougcfoOd3jYKryJDFUv25rNoVy4nz1VY9/eJC2PSHbfQv3NrCS0XI+HVCAmvays2VPGv7TnM336G88af58nvEqlnWLcI+se3plvboOtPjdzM8FIUhePFRtYeLmLN4SL25160Pubn7cmwWyN5pE97ukUF3eiPJpyEhFcjJLyapqrWxNrDRSzbl8/Go+esp5RgmU8/rWMYydFBdIuybHpf74Yv0MTwulBRw/GicrLyy9h5+gK7TpdS8ovFRVLbhfC71GiGJ0Wi++X7CJcj4dUICa8bV1pRw/eHCtnwUzFbT5RcddrpVoE+RIf4Ex3iR6tALSFKDZNH9wBg3vdZVPr4UlNn5kJFDefKqzlnrOb0+YpfBRWAj5cH6R3DGJQYzsAu4YTr5cqhO5HwaoSE182pNZnZm3ORnacvkJVXRlZ+mXXIxZX8aqo48t59AHR5fjGXfBoPoJhQPzq30ZHaPoSe7UO5NToIrZdni/0MwrE19TOq6o3Zwvl4e3rQs0NogzFhZZW15JZWkldaSV7pJUora6i++PMtSCOT22L298fby4OwAB9a67S0DtQSFeJHXJvAlhvlL1yatLxEy3CSBTiE45Ebs4UQLk3CSwjhlCS8hBBOScJLCOGUJLyEEE5JwksI4ZQkvIQQTsntRgfWD2szGFx3HneHUPHzDBAYDGAyqVeLcCr1n83rDUF1u/AqLy8HICYmRuVK3EjbtmpXIJxQeXk5QUGNzx7idiPszWYzZ8+eRafTueU8UAaDgZiYGHJzc+UOAyfgjn8vRVEoLy+nbdu2DdZt/SW3a3l5eHgQHR2tdhmq0+v1bvNhcAXu9ve6VournnTYCyGckoSXEMIpSXi5Ga1Wy2uvvYZWq1W7FNEE8vdqnNt12AshXIO0vIQQTknCSwjhlCS8hBBOScJLCOGUJLyEEE5Jwks08N5779G1a1cSExN59tlnr3tzrLCvMWPGEBISwn333ddg//Lly4mPj6dTp0589tlnKlVnXzJUQlidO3eO3r17c+jQIby9venXrx/vvPMOaWlpapcmLtu4cSPl5eV8+eWXLF68GIC6ujoSExPZsGEDQUFBpKamsnXrVsLCwlSutmVJy0s0UFdXR1VVFbW1tdTW1tKmTRu1SxJXGDBgADqdrsG+HTt20LVrV6KioggMDGTo0KGsWbNGpQrtR8LLhWzevJl7772Xtm3botFoWLp06a+OmTNnDu3bt8fX15devXqxY8cO62OtW7fmxRdfJDY2lrZt2zJw4EA6duxox5/Atd3s36cxZ8+eJSoqyvp9VFQU+fn5tizdIUl4uZCKigqSk5OZM2fOVR9fuHAhU6ZM4bXXXmPPnj0kJyczePBgiouLASgtLWX58uWcPn2a/Px8tm7dyubNm+35I7i0m/37iIYkvFzI0KFDefPNNxkzZsxVH3/33XeZNGkSjzzyCImJiXz88cf4+/vz+eefA7Bu3Tri4uIIDQ3Fz8+P4cOHs23bNnv+CC7tZv8+jWnbtm2DllZ+fj5t3WACSAkvN1FTU8Pu3bsZOHCgdZ+HhwcDBw4kMzMTsMwuu3XrVqqqqjCZTGzcuJH4+Hi1SnYrTfn7NKZnz54cPHiQ/Px8jEYjq1atYvDgwS1dsurcbjJCd3X+/HlMJhPh4eEN9oeHh/PTTz8B0Lt3b4YNG0ZKSgoeHh7cfffdjBw5Uo1y3U5T/j4AAwcOZP/+/VRUVBAdHc0333xDWloas2bN4s4778RsNvPSSy+5/JVGkPASv/DWW2/x1ltvqV2GaMS6deuuun/kyJFu9x+NnDa6iVatWuHp6UlRUVGD/UVFRURERKhUlagnf58bJ+HlJnx8fEhNTWX9+vXWfWazmfXr18sgVAcgf58bJ6eNLsRoNJKdnW39/tSpU+zbt4/Q0FBiY2OZMmUKEyZM4LbbbqNnz568//77VFRU8Mgjj6hYtfuQv4+NKcJlbNiwQQF+tU2YMMF6zN///nclNjZW8fHxUXr27Kls27ZNvYLdjPx9bEvubRRCOCXp8xJCOCUJLyGEU5LwEkI4JQkvIYRTkvASQjglCS8hhFOS8BJCOCUJLyGEU5LwEkI4JQkv4dAGDBjAc8891+Tjv/jiC4KDg5v9fo3NLS8cj9yYLcQVCgoKCAkJUbsM0QQSXkJcQebOch5y2ihu2IABA3jmmWd47rnnCAkJITw8nLlz51qnb9HpdMTFxbFq1aoGz9u0aRM9e/ZEq9USGRnJyy+/TF1dnfXxiooKxo8fT2BgIJGRkcyaNetX711dXc2LL75IVFQUAQEB9OrVi40bNza59pqaGp5++mkiIyPx9fWlXbt2ZGRkWB+/8rTx9ddfR6PR/Gr74osvAMt8WxkZGXTo0AE/Pz+Sk5OtC8EKO1B7WgvhfPr376/odDpl+vTpyrFjx5Tp06crnp6eytChQ5VPP/1UOXbsmPLEE08oYWFhSkVFhaIoipKXl6f4+/srTz75pHLkyBFlyZIlSqtWrZTXXnvN+rpPPPGEEhsbq6xbt045cOCAMmLECEWn0ymTJ0+2HvPoo48q6enpyubNm5Xs7Gxl5syZilarVY4dO6YoiqLMmzdPCQoKarT2mTNnKjExMcrmzZuV06dPKz/88IMyf/586+OAsmTJEkVRFKW8vFwpKCiwbu+8847i7++vZGVlKYqiKG+++aaSkJCgfP/998qJEyeUefPmKVqtVtm4caNtftHimiS8xA3r37+/0rdvX+v3dXV1SkBAgPLQQw9Z9xUUFCiAkpmZqSiKorzyyitKfHy8YjabrcfMmTNHCQwMVEwmk1JeXq74+PgoixYtsj5eUlKi+Pn5WcPrzJkziqenp5Kfn9+gnrvvvluZNm2aoijXD69nnnlGueuuuxrUcaUrw+tKmZmZiq+vr7Jw4UJFURSlqqpK8ff3V7Zu3drguIkTJypjx45t9P2F7Uifl2iWpKQk69eenp6EhYVx6623WvfVr4JTv2DqkSNHSEtLQ6PRWI/p06cPRqORvLw8SktLqampoVevXtbHQ0NDGyy9lpWVhclkonPnzg1qqa6ubvJqOQ8//DCDBg0iPj6eIUOGMGLECO65555rPicnJ4fRo0fz4osvcv/99wOQnZ1NZWUlgwYNanBsTU0NKSkpTapF3BwJL9Es3t7eDb7XaDQN9tWHlNlsttl7Go1GPD092b17N56eng0eCwwMbNJr9OjRg1OnTrFq1SrWrVvH/fffz8CBAxvtq6qoqGDkyJGkpaXxxhtvNKgFYMWKFURFRTV4jlarvZEfSzSThJewiy5duvDtt9+iKIo12LZs2YJOpyM6OprQ0FC8vb3Zvn07sbGxAJSWlnLs2DH69+8PQEpKCiaTieLiYu64445m16LX63nggQd44IEHuO+++xgyZAgXLlwgNDS0wXGKovDggw9iNpv55z//2aDVmJiYiFarJScnx1qfsC8JL2EXTz75JO+//z7PPPMMTz/9NEePHuW1115jypQpeHh4EBgYyMSJE5k6dSphYWG0adOGP//5z3h4/HxBvHPnzowbN47x48cza9YsUlJSOHfuHOvXrycpKYnhw4dft453332XyMhI68K633zzDREREVcd2Pr666+zbt061qxZg9FotLa2goKC0Ol0vPjiizz//POYzWb69u1LWVkZW7ZsQa/XM2HCBJv97sTVSXgJu4iKimLlypVMnTqV5ORkQkNDmThxIq+++qr1mJkzZ2I0Grn33nvR6XS88MILlJWVNXidefPm8eabb/LCCy+Qn59Pq1at6N27NyNGjGhSHTqdjr/97W8cP34cT09Pbr/9dlauXNkgJOtt2rQJo9FIenr6r2p4+OGHmT59Oq1btyYjI4OTJ08SHBxMjx49eOWVV5rxGxI3ShbgEEI4JRmkKoRwShJeQginJOElhHBKEl5CCKck4SWEcEoSXkIIpyThJYRwShJeQginJOElhHBKEl5CCKck4SWEcEr/HzrztTTmmeQlAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "c = 2.21e19 # target compute budget (usually know this because we know how many GPU for how long go brrr)\n", "# (I got this flop number from row 1 of Table A3)\n", "# sweep model sizes from 10M to 100B\n", "ns = 10 ** np.arange(7, 11, step=2**-4)\n", "# using C = 6*N*D, solve for D that maintains the compute budget c\n", "ds = c / (6 * ns)\n", "# evaluate the loss in each case\n", "losses = L(ns, ds)\n", "# find the argmin\n", "best = np.argmin(losses)\n", "print(f\"best model size: {ns[best]/1e6:.2f}M\")\n", "print(f\"best dataset size: {ds[best]/1e9:.2f}B\")\n", "# plot the loss\n", "plt.figure(figsize=(3,3))\n", "plt.plot(ns, losses)\n", "plt.xscale('log')\n", "# plot a vertical bar at the best model size\n", "plt.axvline(ns[best], color='red')\n", "plt.xlabel('model size')\n", "plt.ylabel('loss')" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "In the plot above, basically the models on the left of best are too small and trained for too long. The models on the right of best are way too large and trained for too little. The model at the red line is just right.\n", "\n", "Now, the Chinchilla paper says that best model size for this flop budget is 400M params and 9.2B tokens (instead of 316M params and 11.65B tokens) so there is some unresolved disagreement here too..." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "2304" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Calculate the Chinchilla optimal models for a range of compute budgets\n", "\n", "# sweep over compute budgets from 1e17 to 1e26\n", "cs = 10 ** np.arange(17, 26, step=2**-8)\n", "models = []\n", "for c in cs:\n", " # sweep over model sizes\n", " ns = 10 ** np.arange(7, 14, step=2**-8)\n", " # the dataset sizes that would maintain the given compute budget\n", " ds = c / (6 * ns)\n", " # losses at each point\n", " losses = L(ns, ds)\n", " # n,d for the best model\n", " best = np.argmin(losses)\n", " models.append((c, ns[best], ds[best])) # c, n, d tuple log\n", "\n", "len(models)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "closest model found:\n", "model size: 399.54M\n", "dataset size: 14.43B\n", "flops: 3.459892e+19\n", "loss: 2.76\n" ] } ], "source": [ "query_model_size = 400e6\n", "ns = np.array([n for c, n, d in models])\n", "ds = np.array([d for c, n, d in models])\n", "# find the index of the closest model size in ns\n", "ix = np.argmin(np.abs(ns - query_model_size))\n", "# retrieve the corresponding params, flops, and data size\n", "print(\"closest model found:\")\n", "print(f\"model size: {ns[ix]/1e6:.2f}M\")\n", "print(f\"dataset size: {ds[ix]/1e9:.2f}B\")\n", "print(f\"flops: {6*ns[ix]*ds[ix]:e}\")\n", "print(f\"loss: {L(ns[ix], ds[ix]):.2f}\")" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "This should have come out as 9.2B according to Table A3 in Chinchilla paper, per my understanding of it." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Scaling Laws: Approach 2\n", "\n", "Approach 2 is probably my favorite one because it fixes a flop budget and runs a number of model/dataset sizes, measures the loss, fits a parabolla, and gets the minimum. So it's a fairly direct measurement of what we're after. The best way to then calculate the compute-optimal number of tokens for any given model size, as an example, is via simple interpolation." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "# Approach 1 numbers\n", "# # parameters, tokens\n", "# raw = [\n", "# [400e6, 8e9],\n", "# [1e9, 20.2e9],\n", "# [10e9, 205.1e9],\n", "# [67e9, 1.5e12],\n", "# [175e9, 3.7e12],\n", "# [280e9, 5.9e12],\n", "# [520e9, 11e12],\n", "# [1e12, 21.2e12],\n", "# [10e12, 216.2e12],\n", "# ]\n", "\n", "# Approach 2 numbers\n", "# parameters, tokens\n", "raw = [\n", " [400e6, 7.7e9],\n", " [1e9, 20.0e9],\n", " [10e9, 219.5e9],\n", " [67e9, 1.7e12],\n", " [175e9, 4.3e12],\n", " [280e9, 7.1e12],\n", " [520e9, 13.4e12],\n", " [1e12, 26.5e12],\n", " [10e12, 292.0e12],\n", "]\n" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "y = 1.0409573169995892x + 0.9353887152390791\n" ] } ], "source": [ "# fit a line by linear regression to the raw data\n", "import numpy as np\n", "x = np.array([np.log10(x[0]) for x in raw])\n", "y = np.array([np.log10(x[1]) for x in raw])\n", "A = np.vstack([x, np.ones(len(x))]).T\n", "m, c = np.linalg.lstsq(A, y, rcond=None)[0]\n", "print(f\"y = {m}x + {c}\")" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAATkAAAFBCAYAAAAMkNhdAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABCEElEQVR4nO3deVxU5f4H8M8MwozIJrHL5q6IgIIQWVdUFslISculEs2fdQ3SIm9JpYQt2oaWUmY3Nc2VNNyJRUwtlFwgvYpbaIoIIsLAINvM8/vjNKPDADIwcIaZ7/v18gXnmTNnvucBPp71OQLGGAMhhOgpId8FEEJIR6KQI4ToNQo5Qoheo5AjhOg1CjlCiF6jkCOE6DUKOUKIXqOQI4ToNQo5Qoheo5AjBisoKAhBQUEG99mamDlzJtzd3dv0Xl1ZRwo50iYff/wxUlJS+C7joc6dO4f3338fV69e5bsUwhMKOdImXSnkEhISmgy5tLQ0pKWldX5RpFN147sAQvhiYmLCdwmkE9CWXAcpLCzE7Nmz4eTkBJFIhN69e2Pu3Lmoq6tTzvPXX3/h2WefhbW1NUxNTfHoo49i3759Kss5dOgQBAIBtm/fjoSEBPTq1Qvm5uaYPHkyKioqUFtbi9dffx12dnYwMzPDrFmzUFtbq7IMgUCAmJgYbNq0CQMHDoRYLIavry8OHz6sMl9zx1/ef/99CAQCleVJpVL88MMPEAgEEAgEmDlzpsq6v/TSS7C3t4dIJMKQIUOwdu3aVvVbQ0MDPvjgA/Tt2xcikQju7u5455131NbJ3d0dTz31FNLS0uDj4wOxWAwPDw/s3LlTOc/69evx7LPPAgBGjx6trPXQoUMA1I8ZaaOv161bhzFjxsDOzg4ikQgeHh745ptvWrXuTVH87JKTk+Hh4YHu3bsjMDAQZ86cAQB8++236NevH8RiMYKCgprcYk1OToavry+6d+8OGxsbvPDCCygsLFSbLyUlBZ6enhCLxfD09MTPP//cZE1yuRwrVqzAkCFDIBaLYW9vj1deeQV379596PqsXLkSQ4YMgampKXr27Ak/Pz9s3rxZs07RFCNaV1hYyJycnJipqSl7/fXX2erVq9miRYvY4MGD2d27dxljjN26dYvZ29szc3Nz9u6777LExETm7e3NhEIh27lzp3JZWVlZDADz8fFhgYGB7KuvvmLz5s1jAoGATZ06lU2fPp2Fh4ezpKQk9uKLLzIALCEhQaUeAMzT05PZ2NiwJUuWsE8++YS5ubmx7t27szNnzijni4qKYm5ubmrrEx8fzx78Vdm4cSMTiUTsiSeeYBs3bmQbN25kv//+u3K9nJ2dmYuLC1uyZAn75ptv2NNPP80AsOXLlz+076KiohgANnnyZJaUlMRmzJjBALCJEyeqzOfm5sYGDBjArKys2MKFC1liYiIbOnQoEwqFLC0tjTHG2JUrV9i8efMYAPbOO+8oa7116xZjjLFRo0axUaNGabWvR4wYwWbOnMmWL1/OVq5cyUJDQxkAtmrVKpX5Gn92cwAwLy8v5uLiwpYtW8aWLVvGLC0tmaurK1u1ahXz8PBgX3zxBXvvvfeYiYkJGz16tMr7161bxwCwESNGsOXLl7OFCxey7t27M3d3d+XvImOM/fLLL0woFDJPT0+WmJjI3n33XWZpacmGDBmi9jvxf//3f6xbt25szpw5bPXq1eztt99mPXr0YCNGjGB1dXXNruOaNWuUP9tvv/2Wffnll2z27Nls3rx5D+2H9qCQ6wAzZsxgQqGQ/fHHH2qvyeVyxhhjr7/+OgPAjhw5onytsrKS9e7dm7m7uzOZTMYYu/+H5+npqfILNG3aNCYQCFh4eLjK8gMDA9V+KQEwAOzEiRPKtmvXrjGxWMwiIyOVba0NOcYY69GjB4uKilKbd/bs2czR0ZGVlpaqtE+dOpVZWlqy6upqtfco5ObmMgDs//7v/1TaFyxYwACwgwcPKtvc3NwYALZjxw5lW0VFBXN0dGTDhg1TtiUnJzMALCsrS+3zmgu59vR1U+sXFhbG+vTp0+JnNwcAE4lErKCgQNn27bffMgDMwcGBSSQSZXtcXBwDoJy3rq6O2dnZMU9PT3bv3j3lfHv37mUA2OLFi5VtPj4+zNHRkZWXlyvb0tLSGACVdTxy5AgDwDZt2qRSZ2pqqlp743WcMGECGzJkyEPXWdtod1XL5HI5UlJSEBERAT8/P7XXFbt9+/fvh7+/Px5//HHla2ZmZnj55Zdx9epVnDt3TuV9M2bMgLGxsXI6ICAAjDG89NJLKvMFBATg+vXraGhoUGkPDAyEr6+vctrV1RUTJkzAL7/8AplM1vYVfgBjDDt27EBERAQYYygtLVX+CwsLQ0VFBU6dOtXs+/fv3w8AiI2NVWl/8803AUBtV97JyQmRkZHKaQsLC8yYMQOnT5/GrVu32rwe7enr7t27K7+vqKhAaWkpRo0ahb/++gsVFRVtqmfs2LEqhxECAgIAAJMmTYK5ubla+19//QUAOHHiBEpKSvDqq69CLBYr5xs/fjwGDRqk7M+ioiLk5uYiKioKlpaWyvlCQkLg4eGhUktycjIsLS0REhKi8vP19fWFmZkZsrKyml0PKysr3LhxA3/88Ueb+qGtKOS07Pbt25BIJPD09GxxvmvXrmHgwIFq7YMHD1a+/iBXV1eVacUvo4uLi1q7XC5X+4Pq37+/2mcNGDAA1dXVuH37dou1ttbt27dRXl6ONWvWwNbWVuXfrFmzAAAlJSXNvv/atWsQCoXo16+fSruDgwOsrKzU+qRfv34qxwoV6wSgXZeMtKevf/vtNwQHB6NHjx6wsrKCra0t3nnnHQBoc8hpUg8A5bExRX819Xs2aNAg5euKr039jjR+76VLl1BRUQE7Ozu1n3FVVVWLP9+3334bZmZm8Pf3R//+/REdHY3ffvut+RXXEjq72kUYGRlp1M7aMKp948BQaO2WnlwuBwC88MILiIqKanIeLy+vNtfRWdra11euXMHYsWMxaNAgJCYmwsXFBSYmJti/fz+WL1+u7J/OqqcjyOVy2NnZYdOmTU2+bmtr2+x7Bw8ejAsXLmDv3r1ITU3Fjh078PXXX2Px4sVISEjoqJIp5LTN1tYWFhYWOHv2bIvzubm54cKFC2rt+fn5yte16dKlS2ptFy9ehKmpqfIXs2fPnigvL1ebr/EWFNB0ENna2sLc3BwymQzBwcEa1+jm5ga5XI5Lly4pt2gBoLi4GOXl5Wp9cvnyZTDGVGq5ePEiACh37zozMPfs2YPa2lrs3r1bZeurpV24jqTorwsXLmDMmDEqr124cEH5uuJrU78jjX9H+/bti4yMDIwcOVJl17y1evTogSlTpmDKlCmoq6vDM888g48++ghxcXEqu9TaRLurWiYUCjFx4kTs2bMHJ06cUHtd8b/sk08+iZycHGRnZytfk0qlWLNmDdzd3dWOhbRXdna2yvGw69evY9euXQgNDVVuEfTt2xcVFRX4888/lfMVFRU1eSlBjx491ALRyMgIkyZNwo4dO5oM+YftFj/55JMAgBUrVqi0JyYmAuCOJT3o5s2bKrVJJBJs2LABPj4+cHBwUNYJoMnw1jZFPz64JVVRUYF169Z1+Gc3xc/PD3Z2dli9erXKpS4HDhzA+fPnlf3p6OgIHx8f/PDDDyq71Onp6WrHhp977jnIZDJ88MEHap/X0NDQYj/fuXNHZdrExAQeHh5gjKG+vr4tq9gqtCXXAT7++GOkpaVh1KhRePnllzF48GAUFRUhOTkZR48ehZWVFRYuXIgtW7YgPDwc8+bNg7W1NX744QcUFBRgx44dEAq1+/+Pp6cnwsLCMG/ePIhEInz99dcAoLKbMHXqVLz99tuIjIzEvHnzUF1djW+++QYDBgxQO2Hg6+uLjIwMJCYmwsnJCb1790ZAQACWLVuGrKwsBAQEYM6cOfDw8EBZWRlOnTqFjIwMlJWVNVujt7c3oqKisGbNGpSXl2PUqFHIycnBDz/8gIkTJ2L06NEq8w8YMACzZ8/GH3/8AXt7e6xduxbFxcUqoeLj4wMjIyN88sknqKiogEgkUl7Hpm2hoaEwMTFBREQEXnnlFVRVVeG7776DnZ0dioqKtP55D2NsbIxPPvkEs2bNwqhRozBt2jQUFxfjyy+/hLu7O9544w3lvEuXLsX48ePx+OOP46WXXkJZWZnymraqqirlfKNGjcIrr7yCpUuXIjc3F6GhoTA2NsalS5eQnJyML7/8EpMnT26yntDQUDg4OGDkyJGwt7fH+fPnsWrVKowfP17lBIrWdfr5XANx7do1NmPGDGZra8tEIhHr06cPi46OZrW1tcp5rly5wiZPnsysrKyYWCxm/v7+bO/evSrLUVzWkJycrNKuuP6p8WUqiss9bt++rWwDwKKjo9mPP/7I+vfvz0QiERs2bFiTl1WkpaUxT09PZmJiwgYOHMh+/PHHJi8hyc/PZ//6179Y9+7dGQCVy0mKi4tZdHQ0c3FxYcbGxszBwYGNHTuWrVmz5qH9Vl9fzxISEljv3r2ZsbExc3FxYXFxcaympkZlPjc3NzZ+/Hj2yy+/MC8vLyYSidigQYPU+okxxr777jvWp08fZmRkpHI5SXOXkLSnr3fv3s28vLyYWCxm7u7u7JNPPmFr165VubSjqc9ujuJn96CCggIGgH322Wcq7c3Vv23bNjZs2DAmEomYtbU1e/7559mNGzfUPmvHjh1s8ODBTCQSMQ8PD7Zz585mLytas2YN8/X1Zd27d2fm5uZs6NCh7K233mI3b95sdh2//fZb9q9//Ys98sgjTCQSsb59+7L//Oc/rKKi4qH90B4Cxui5q/pOIBAgOjoaq1at4rsUrXF3d4enpyf27t3LdylEx9ExOUKIXqOQI4ToNQo5Qoheo2NyhBC9RltyhBC9RiFHCNFrdDFwM+RyOW7evAlzc3Pe76UkhKhijKGyshJOTk4PvXCeQq4ZN2/eVBvlgRCiW65fvw5nZ+cW56GQa4biNpPr16/DwsKC52q6jvr6eqSlpSlv9yGdy1D6XyKRwMXFpVW3g1HINUOxi2phYUEhp4H6+nqYmprCwsJCr//IdJWh9X9rDiXRiQdCiF6jkGskKSkJHh4eGDFiBN+lEEK0gEKukejoaJw7d67Tx6EnhHQMOiZHCNEJMjlDTkEZSiprYGcuhn9vaxgJ23/5FoUcIYR3qWeLkLDnHIoqapRtjpZixEd4YJynY7uWTburhBBepZ4twtwfT6kEHADcqqjB3B9PIfVs+0ZVppAjhPBGJmdI2HMODICAyRGXtRaDS7jnxipGDknYcw4yedvHEaGQa4TOrhLSeXIKylBUUQMBk+Pj1FV4JWcnNmxfjB611QC4oCuqqEFOQfPPBnkYCrlG6OwqIZ2npJILuGUHVmLan2mQCYT4cPRsSEWmavO1FZ14IITwxs7UGJ/u/wrPns2ATCDEG0+9id0eo9TnM2/7M1kp5Agh/JDJEPDRWwg8m4EGgRCvRyzA3sH/UplFAMDBkrucpK0o5AghnU8mA2bNgnDjRsiNjPD6+AXYN/gJlVkUV8jFR3i063o5OiZHCOlcMhkwcyawcSNgZAThli146qP5cLBU3SV1sBTjmxeGt/s6OdqSI4R0noYGICoK2LwZ6NYN2LoVmDQJ4wCEeDjQHQ+EkC6soQGYMQPYsoULuG3bgGeeUb5sJBQgsO8jWv9Y2l1thK6TI6QDNDQAL7xwP+CSk1UCriNRyDVC18kRomX19cD06dyWm7Ex8NNPwMSJnfbxtLtKCOk4ioD76Scu4HbsACIiOrUECjlCSMeorwemTeOCzcSE+/rUU51eBoUcIUT76uqAqVOBn3/mAm7nTmD8eF5KoZAjhGhXXR0wZQqQkgKIRFzQhYfzVg6FHCFEe+rqgGefBXbv5gIuJQUYN47XkijkCCHaUVvLBdyePYBYDOzaBYSG8l2VflxCEhkZiZ49e2Ly5Mlqr1VXV8PNzQ0LFizgoTJCDERtLTBp0v2A271bJwIO0JOQmz9/PjZs2NDkax999BEeffTRTq6IEANSU8Nd2LtvHxdwe/YAISF8V6WkFyEXFBQEc3NztfZLly4hPz8f4Twe9CREr9XUAJGRwP79QPfuwN69QHAw31Wp4D3kDh8+jIiICDg5OUEgECAlJUVtnqSkJLi7u0MsFiMgIAA5OTmtWvaCBQuwdOlSLVdMCAHABdzEiUBqKhdw+/YBY8fyXZUa3kNOKpXC29sbSUlJTb6+bds2xMbGIj4+HqdOnYK3tzfCwsJQUlLS4nJ37dqFAQMGYMCAAR1RNiGG7d49YMIE4JdfAFNTbktu9Gi+q2oS72dXw8PDW9ydTExMxJw5czBr1iwAwOrVq7Fv3z6sXbsWCxcubPZ9x44dw9atW5GcnIyqqirU19fDwsICixcvbnL+2tpa1NbWKqclEgkAoL6+HvX19W1ZNYOk6CvqM350Sv9XV8No0iQIMzPBevSAbPdusJEjuTscOokm68d7yLWkrq4OJ0+eRFxcnLJNKBQiODgY2dnZLb536dKlyl3V9evX4+zZs80GnGL+hIQEtfa0tDSYmpo28Q7SkvT0dL5LMGgd1f9GtbUI+Ogj2P75JxrEYmS/8w7KKiu5LblOVF1d3ep5dTrkSktLIZPJYG9vr9Jub2+P/Px85XRwcDDy8vIglUrh7OyM5ORkBAYGavRZcXFxiI2NVU5LJBK4uLggNDQUFhYW7VsRA1JfX4/09HSEhITA2NiY73IMTof2v1QKo8hICP/8E8zMDNizB4+OHKndz2glxZ5Wa+h0yLVWRkZGi6/PnDnzocsQiUQQiURISkpCUlISZDIZAMDY2Jj+WNuA+o1fWu9/qZQ7i3roEGBmBkFqKrrxFHAANFo33k88tMTGxgZGRkYoLi5WaS8uLoaDg0OHfCaNJ0dII1Ipd3P9oUOAuTl3soHHgNOUToeciYkJfH19kZmZqWyTy+XIzMzUeHeUENIGVVXAk08Cv/4KWFgAaWnAY4/xXZVGeN9draqqwuXLl5XTBQUFyM3NhbW1NVxdXREbG4uoqCj4+fnB398fK1asgFQqVZ5t1bbGu6uEGKzKSi7gjh69H3ABAXxXpTnGs6ysLAZA7V9UVJRynpUrVzJXV1dmYmLC/P392bFjxzq8roqKCgaAVVRUdPhn6ZO6ujqWkpLC6urq+C7FIGmt/yUSxkaOZAxgzNKSsePHtVKftmjy98n7llxQUBAYYy3OExMTg5iYmE6ph7bkiMGTSLjx337/HbC0BNLTgS78YCedPibHBzrxQAxaRQUQFsYFnJUVkJHRpQMO0IFjcoQQHaEIuOPHgZ49uYAbPpzvqtqNtuQaoeeuEoNUXs6N/3b8OGBtDWRm6kXAARRyamh3lRgcRcDl5NwPuGHD+K5KayjkCDFkd+9yA1z+8QfwyCPAwYOAjw/fVWkVHZMjxFCVlXEBd+oUYGPDbcF5efFdldbRllwjdEyOGISyMm4EX0XAHTyolwEHUMipoWNyRO/ducON4Hv6NGBrC2RlAUOH8l1Vh6GQI8SQlJZyAZebC9jZcQHn6cl3VR2KjskRYigUAffnn4C9PbeL6uHBd1UdjrbkGqFjckQv3b4NjBnDBZyDAzdskgEEHEAhp4aOyRG9U1LCBdyZM1zAZWUBgwbxXVWnod1VQvRZcTEXcOfOAY6OXMANHMh3VZ2KtuQI0Ve3bnGPCTx3DnBy4nZRDSzgAAo5QvSGTM6QU1AGADh9/DzY6NHA+fNAr15cwBnoM4hpd5UQPZB6tggJe86hrOoevuxXBpuXoiEoK8Q9e0d0P3QI6NeP7xJ5Q1tyjdDZVdLVpJ4twtwfT6GoogZ2lXfw+HvvoU9ZIQrNbTFuwhKk1vTgu0ReUcg1QmdXSVcikzMk7DkHBsC+shQbNr0Ls5s3UWhhi6nTl+Lvno5I2HMOMnnLo2/rMwo5QrqwnIIyFFXUwEFSiq1b4tC7rBDVtraY8cLHuG7lAAagqKJGeazOEFHIEdKFlVTWwFFymwu4u0W4YWmHox9+iBtWDmrzGSoKOUK6MOdKbgvOvbwI1y3t8eILS3HP3l5tPjtzMQ/V6QY6u0pIV3XtGoZHRUJQfgt/W9pj6vSluGNpC+D+k+YEABwsxfDvbc1bmXyjkCOkK7p2DQgKguDqVVQ7u2FqxPsosrCFCe6fYBD88zU+wgNGQkHTyzEAtLtKSFdz9SoQFMR97dsXptlHsfjVMDhYqu6SOliK8c0LwzHO05GPKnUGbck1Qg+XJjqtoIC7VevaNe4C30OHgF69MM4ZCPFwwLHLJSg9fwxro0bg0X52Br0Fp0Bbco3QdXJEZ/31F7cFd+0a0L+/MuAUjIQC5bE3/97WFHD/oJAjpCu4coULuL//5u5BbRRwpHm0u0qIjlHcaF9SWQM7czH8ZWUwGjsGuHGDG0UkK4sbNom0CoUcITpEcaN9UQV38a57WSG2b3sXdpJSbqDLrCxu4EvSahRyhOgIxY32iotAepcVYsuWONhVleHiI6648c02jKGA0xgdkyNEBzx4oz0A9LlzA1u3xMHhn4B7ftpHeDf7tkHfaN9WFHKE6ADFjfYA0PfOdWzZ+g7sq8pwwcYV06Z9jNs9ehr8jfZtpRchFxkZiZ49e2Ly5MnKtvLycvj5+cHHxweenp747rvveKyQkJYpbqDvW3odW7ZwAZdv44Zp05biTg8rtflI6+lFyM2fPx8bNmxQaTM3N8fhw4eRm5uL48eP4+OPP8adO3d4qpCQltmZi9Gv9G9s3RoHO+ldnLd1x/RpH6PM1FJtPqIZvQi5oKAgmJubq7QZGRnB1NQUAFBbWwvGGBij4xlEN/lXF2H71ndgKy3HObvemD71I5WAEwBwNPAb7duK95A7fPgwIiIi4OTkBIFAgJSUFLV5kpKS4O7uDrFYjICAAOTk5LRq2eXl5fD29oazszP+85//wMbGRsvVE6IFZ8/CaOwYWEvL8T+7Pnh+6ke42yjgALrRvq14DzmpVApvb28kJSU1+fq2bdsQGxuL+Ph4nDp1Ct7e3ggLC0NJSclDl21lZYW8vDwUFBRg8+bNKC4u1nb5hLTPmTPcvai3bwPDhuHWzj0QO9ipzEI32rcP79fJhYeHIzw8vNnXExMTMWfOHMyaNQsAsHr1auzbtw9r167FwoULW/UZ9vb28Pb2xpEjR1ROTjyotrYWtbW1ymmJRAIAqK+vR319fWtXx+Ap+or6rBX+/BPdxo2DoLQU8uHDIdu/H/+ytkaWXz+cvHYXpVW1sDETwdetJ4yEglb1qaH0vybrx3vItaSurg4nT55EXFycsk0oFCI4OBjZ2dktvre4uBimpqYwNzdHRUUFDh8+jLlz5zY7/9KlS5GQkKDWnpaWpjy2R1ovPT2d7xJ0mkVBAR5bvBiCykrc7dcP2bGxqD92TG2+UgC/nNd8+fre/9XV1a2eV6dDrrS0FDKZDPaNhnO2t7dHfn6+cjo4OBh5eXmQSqVwdnZGcnIyjIyM8PLLLytPOLz22msYOnRos58VFxeH2NhY5bREIoGLiwtCQ0NhYWGh/ZXTU/X19UhPT0dISAiMjY35Lkc35eai20svQVBZCbmfH8z270eIlZVWFm0o/a/Y02oNnQ651srIyGiyPTc3t9XLEIlEEIlEauPJGRsb6/UvS0ehfuOo3Wx/9yqMwsKAu3cBf38If/kFQi0F3IP0vf81WTedDjkbGxsYGRmpnTAoLi6GQwfdwxcdHY3o6GhIJBJYWlo+/A2ENKPxzfaety5j8/ZFsLhXCQQEAL/8AtDvWIfj/exqS0xMTODr64vMzExlm1wuR2ZmJgIDAzvkM5OSkuDh4YERI0Z0yPKJYXjwqfYAMLToEjZtfRcW9ypx0mkQ0ldsoIDrJLxvyVVVVeHy5cvK6YKCAuTm5sLa2hqurq6IjY1FVFQU/Pz84O/vjxUrVkAqlSrPtmobbcmR9mp8s71X0UX8uG0RLGqlONFrMGY9mwCzrOsY49+frnvrBLyH3IkTJzB69GjltOLgf1RUFNavX48pU6bg9u3bWLx4MW7dugUfHx+kpqaqnYwgRFc8eLO9980L2LhtESzqqvFHLw/MfPZ9SEWmqPznZvvAvo/wXK3+4z3kgoKCHnq7VUxMDGJiYjqlHnqQDWkvxU30PjcvYMM/AZfj7IFZk7mAazwf6Vg6fUyOD/QgG9JeduZiDC88j43b3oNFXTWOu3hi5rMJKgGnmI90PN635AjRN/5F+diYHI8edfdwzMUTL02OR7VJd+Xr9FT7zkVbco3Q2VXSLr/9BqPwcehRW41s16F4afL7agEH0M32nYlCrhHaXSVtdvQoMG4cUFUFjBmDyp9+hqWtlcosdLN956PdVULaQO1Ohhv/g9H4JwGpFBg7Fti9G6Gmphjr20d1Pnroc6ejkCNEQ43vZAj4+wzW7UiAaV0NEBIC7NoFdOd2UY2EArpMhGe0u9oIHZMjLWl8J8Ojf/+JdT+9D9O6Ghx2H4a0j1crA47oBgq5RuiYHGlO4zsZAq/9ibU/JcC0vha/9h6Ol595D/HpBfTYQB1DIUdIKz14J0PgtTxlwB3q7YuXn3kPNcYiemygDmp3yEkkEqSkpOD8+TaM7EdIF6K4Q2Hk1Vys+ykB3RtqcbCPH1555l3UdjNRm4/oBo1D7rnnnsOqVasAAPfu3YOfnx+ee+45eHl5YceOHVovsLPRMTnSHDtzMR4vOI3vdyyBuKEOmX1H4N+RqgGnmI/oDo1D7vDhw3jiiScAAD///DMYYygvL8dXX32FDz/8UOsFdjY6Jkea43/pBL7fyQVcej9/zJ34Duq63R+8kR4bqJs0DrmKigpYW3M/xNTUVEyaNAmmpqYYP348Ll26pPUCCdEJv/wCo4kTIGqoR1r/RxE9MU4t4AC6k0EXaRxyLi4uyM7OhlQqRWpqKkJDQwEAd+/ehVhMm+lEDx04AEyYANTWAhMngm3bikesVR9mTncy6C6NLwZ+/fXX8fzzz8PMzAxubm4ICgoCwO3GtvSgGEK6pP37gchIoK6O+7p1K8JMTBDs7Up3MnQRGofcq6++Cn9/f1y/fh0hISEQCrmNwT59+ujFMTlClPbuBSZN4gJu0iRgyxbgnweo0J0MXUebbuvy8/ODn5+fStv48eO1UhDfaNBMAgDYs4cLtvp6YPJkYPNmZcCRrkXjkJPJZFi/fj0yMzNRUlICuVyu8vrBgwe1Vhwf6BkPBLt2Ac8+ywXcc88BP/5IAdeFaRxy8+fPx/r16zF+/Hh4enpCIKDjEESPpKRwAdfQAEydCmzcCHSjcSy6Mo1/elu3bsX27dvx5JNPdkQ9hPDn55+5LbeGBmDaNGDDBgo4PaDxJSQmJibo169fR9RCCH927LgfcNOnU8DpEY1D7s0338SXX3750CdsEdJlJCcDU6ZwAffCCxRwekbjn+TRo0eRlZWFAwcOYMiQITBudEB2586dWiuOEG1RG8lXcV3b9u3clptMBrz4IrBuHWBkxHe5RIs0DjkrKytERkZ2RC2EdIjGI/kC3D2m33S7CJ+417iAi4oCvv+eAk4PaRxy69at64g6dAZdJ6dfFCP5Nj644p+diqF7EwEmB2bNAr77jgJOT7VpPLmGhgZkZGTg22+/RWVlJQDg5s2bqKqq0mpxfKBRSPRH45F8FSb8LwuJexNhxOTY7RcO2RoKOH2m8ZbctWvXMG7cOPz999+ora1FSEgIzM3N8cknn6C2tharV6/uiDoJ0diDI/kqTPxfFr7YtxxGTI4tXqF4Z8xc2F4rp1u09JjGW3Lz58+Hn58f7t69i+4PPLAjMjISmZmZWi2OkPZoPELvM2czlVtwm73H4Z1xMWACIY3kq+c03pI7cuQIfv/9d5iYqI6G6u7ujsLCQq0VRkh7PThC76Qzmfhs/woIwbDJZxzeC30VTCBUm4/oH41DTi6XN3lQ/saNGzA3N2/iHYTww7+3NRwtxXj8yB58cuArCMGwcdiTWBzybzCBEAJw48DRSL76TePd1dDQUKxYsUI5LRAIUFVVhfj4eLrVi+gUI6EAa+pzlQH3w/DxWBQyVxlwAI3kawg03pL74osvEBYWBg8PD9TU1GD69Om4dOkSbGxssGXLlo6okZC2+e47DI1fAADYHjgR8U/MBv4ZUMLBUoz4CA8aydcAaBxyzs7OyMvLw7Zt25CXl4eqqirMnj0bzz//vMqJiM4UGRmJQ4cOYezYsfjpp58AANevX8eLL76IkpISdOvWDYsWLcKzzz7LS32EB2vWAK+8wn0/fz4mfZEIl6t3aSRfQ8Q0tHnz5mZfW7BggaaL04qsrCy2e/duNmnSJGXbzZs32enTpxljjBUVFTEnJydWVVXV6mVWVFQwAKyiokLb5eq1uro6lpKSwurq6vgrYvVqxgDu3+uvMyaX81dLJ9OJ/u8Emvx9anxMbu7cuThw4IBa+xtvvIEff/xRC7GruaCgILWTHo6OjvDx8QEAODg4wMbGBmVl9GRzvffNN8C//819HxsLJCYqd1GJYdI45DZt2oRp06bh6NGjyrbXXnsN27dvR1ZWlsYFHD58GBEREXBycoJAIEBKSoraPElJSXB3d4dYLEZAQABycnI0+oyTJ09CJpPBxcVF4/pIF5KUBLz6Kvf9m28Cn39OAUc0D7nx48fj66+/xtNPP42TJ0/i1Vdfxc6dO5GVlYVBgwZpXIBUKoW3tzeSkpKafH3btm2IjY1FfHw8Tp06BW9vb4SFhaGkpKRVyy8rK8OMGTOwZs0ajWsjXcjKlUBMDPf9f/4DfPYZBRwB0MYH2UyfPh3l5eUYOXIkbG1t8euvv7Z5IM3w8HCEh4c3+3piYiLmzJmDWbNmAQBWr16Nffv2Ye3atVi4cGGLy66trcXEiROxcOFCPPbYYw+dt7a2VjktkUgAAPX19aivr2/t6hg8RV91Zp8JV66E0ZtvAgBkCxZA/uGH3NhwBoiP/ueDJuvXqpCLjY1tst3W1hbDhw/H119/rWxLTExs9Yc/TF1dHU6ePIm4uDhlm1AoRHBwMLKzs1t8L2MMM2fOxJgxY/Diiy8+9LOWLl2KhIQEtfa0tDSYmppqXryBS09P75TP6bN7N4auXQsAuDhpEs6PHMk9DNrAdVb/86W6urrV87Yq5E6fPt1ke79+/SCRSJSva/uhNqWlpZDJZLC3t1dpt7e3R35+vnI6ODgYeXl5kEqlcHZ2RnJyMmQyGbZt2wYvLy/lcb6NGzc2+wDsuLg4lTCXSCRwcXFBaGgoLCwstLpe+qy+vh7p6ekICQlRG1BV24QrVsDon4CTLVyI3gkJ6G3gu6id2f98UuxptUarQq4tJxQ6U0ZGRpPtjR+X2BKRSASRSKQ2npyxsbFe/7J0lA7vty++AN56i/t+0SIYJSTAyMAD7kH6/nurybq1aTw5hRs3buDGjRvtWUSLbGxsYGRkhOLiYpX24uJiODg4dMhn0nhyXcBnnwELuDsZEB8PLFlCJxlIszQOOblcjiVLlsDS0hJubm5wc3ODlZUVPvjgA422nFrDxMQEvr6+KkM4yeVyZGZmIjAwUKufpZCUlAQPDw+MGDGiQ5ZP2umTT+5vwb3/PvePkBZofHb13Xffxffff49ly5Zh5MiRALiH27z//vuoqanBRx99pNHyqqqqcPnyZeV0QUEBcnNzYW1tDVdXV8TGxiIqKgp+fn7w9/fHihUrIJVKlWdbtS06OhrR0dGQSCSwtLTskM8gbbR0KfDOO9z3CQnA4sX81kO6Bk1vp3B0dGS7du1Sa09JSWFOTk6aLo5lZWUxAGr/oqKilPOsXLmSubq6MhMTE+bv78+OHTum8edoim7rapsOu63oww/v36r1wQfaXbYeodu61Gm8JVdWVtbkRb+DBg1q021TQUFBD32Ga0xMDGIUF3p2MHqQjQ764IP7W20ffXR/a46QVtD4mJy3tzdWrVql1r5q1Sp4e3trpSg+0YkHHfPgbumDu6uEtJLGW3Kffvopxo8fj4yMDOXB/+zsbFy/fh379+/XeoHEgL3/PhdygOoJB0I0oPGWXO/evXHx4kVERkaivLwc5eXleOaZZ3DhwgW4ubl1RI2dis6u6gDGuEtDFAH32WcUcKTNNN6S6927N4qKitTOot65cwcuLi5d/lgWnV3lGWPc7umHH3LTn3/OjShCSBtpHHLNnSSoqqqCWExPPSLtwBjw3nvAxx9z04mJwBtv8FsT6fJaHXKK+zoFAgEWL16sctO6TCbD8ePHlYNUEqIxxriTCsuWcdMrVgDz5/NaEtEPrQ45xU34jDGcOXNG5bmrJiYm8Pb2xgLFrTZdGF1CwgPGgIULgU8/5aa/+gp47TV+ayJ6o9Uhp7hJf9asWfjyyy/1dmQOOibXyRjjTip8/jk3vWoVEB3Nb01Er2h8TG7dunUdUQcxRIxxo/h+8QU3/eDw5YRoSZtGBiak3RjjzpouX85Nf/01MHcuvzURvdSuoZb0EV0n1wkY486aKgJu9WoKONJhKOQaodu6Ohhj3FnTL7/kph98CDQhHYB2V0nnYYw7a5qUxA1y+d13wOzZfFdF9ByFHOkcjHGPDPz6ay7g/vtf4KWX+K6KGAAKOdLx5HIu4L75hgu4tWuBmTP5rooYCAo50rHkcu6ykG+/5QJu3TogKorvqogBoRMPjdDZVS2Sy4F///t+wP3wAwUc6XQUco3Q2dW2k8kZcgq40aFzrpRCPmcOd3JBKAQ2bABa8ZBvQrSNdleJVqSeLULCnnMoq7qHT/3kuDVlBoR/ZoAJhRBs3AhMn853icRAUciRdks9W4S5P54CA9BdIMOwVavg+udByARCvD7+TYz3Go1xfBdJDBbtrpJ2kckZEvacAwMglMvw8b6v4HrwIBoEQsyPWIC9HqOQsOccZPKWH1ZESEehkCPtklNQhqKKGgjlMny2fwUizxyEXCjEmxP/g72D/wUGoKiiRnmsjpDORrurpF1KKrmA+2LfckSeO4QGgRCnFixAarfHAZnqfITwgbbkSLvYde+GxH2JiDx3CPVCI7wR+RaKHntMfT5zGhqf8INCrhG6Tk4DDQ0IiH8dE8/9inqhEWImvI20QSNVZhEAcLQUw7+3NT81EoNHIdcIXSfXSg0NwPPPQ7htK+TdjBE9MQ5pA1S34AT/fI2P8ICRUKC+DEI6AYUc0Vx9PXfd2/btgLExhDt+wjMJr8LBUnWX1MFSjG9eGI5xno48FUoInXggmqqvB6ZNA3bsAExMuK9PPYVxAEI8HHDscglKzx/D2qgReLSfHW3BEd7Rlhxpvfp6YOrU+wG3cyfw1FPKl42EAuWxN//e1hRwRCfQlhxpnbo6LuB+/pkLuJ9/Bp58ku+qCHkoCjnycHV1wHPPAbt2ASIRkJICjKMbtUjXQCFHWlZbCzz7LLBnDxdwu3YBYWF8V0VIq+nFMbnIyEj07NkTkydPblU7aaXaWmDyZC7gxGJg924KONLl6EXIzZ8/Hxs2bGh1O2mF2lpg0iRg714u4PbsAUJD+a6KEI3pRcgFBQXB3Ny81e3kIWpqgGeeAfbtA7p354IuOJjvqghpE95D7vDhw4iIiICTkxMEAgFSUlLU5klKSoK7uzvEYjECAgKQk5PT+YUaipoaIDIS2L//fsCNHct3VYS0Ge8hJ5VK4e3tjaSkpCZf37ZtG2JjYxEfH49Tp07B29sbYWFhKCkp6eRKDcC9e8CECUBqKmBqygXdmDF8V0VIu/B+djU8PBzh4eHNvp6YmIg5c+Zg1qxZAIDVq1dj3759WLt2LRYuXKi1Ompra1FbW6uclkgkAID6+nrU19dr7XN01r17MJo0CcKMDDBTU8h27wYbOZK7AFgDir4yiD7TQYbS/5qsH+8h15K6ujqcPHkScXFxyjahUIjg4GBkZ2dr9bOWLl2KhIQEtfa0tDSYmppq9bN0jVFtLfw//hh2eXloEItx7N13caeqituSa6P09HQtVkg0pe/9X11d3ep5dTrkSktLIZPJYG9vr9Jub2+P/Px85XRwcDDy8vIglUrh7OyM5ORkBAYGNtvelLi4OMTGxiqnJRIJXFxcEBoaCgsLi45ZQV1QXQ2jyEgI8/LAzMyAPXsQMHLkw9/XjPr6eqSnpyMkJATGxsZaLJS0hqH0v2JPqzV0OuRaKyMjQ6P2pohEIohEIiQlJSEpKQkyGTesrbGxsf7+skil3EmGrCzAzAyC1FR0a0fAPUiv+60L0Pf+12TdeD/x0BIbGxsYGRmhuLhYpb24uBgODg4d8pkGM56cVMrdXJ+VBZibA7/8Amgp4AjRJTodciYmJvD19UVmZqayTS6XIzMzs9ndzvYyiJGBq6q4m+sPHbofcE0MWU6IPuB9d7WqqgqXL19WThcUFCA3NxfW1tZwdXVFbGwsoqKi4OfnB39/f6xYsQJSqVR5tlXboqOjER0dDYlEAktLyw75DF4pAu7IEcDCggu4Rx/luypCOgzvIXfixAmMHj1aOa04+B8VFYX169djypQpuH37NhYvXoxbt27Bx8cHqampaicjSCtUVnIBd/QoYGkJpKUB/v58V0VIh+I95IKCgsBYyw8ejomJQUxMTKfU0/jEg96QSIDwcOD337mAS08H9HmXnJB/6PQxOT7o5YkHiYQb/+333wErKyAjgwKOGAzet+RIB6uo4ALu2DGgZ08u4IYP57sqQjoNbck1oldnVysquPHfKOCIAaOQa0RvdlfLy7nx344fB6ytgcxMCjhikGh3VR/dvcsF3IkTwCOPcAHn7c13VYTwgrbkGunyu6t37wIhIVzA2dgABw9SwBGDRiHXSJfeXS0r40bwPXnyfsB5efFdFSG8ot1VfaEIuNOnAVtbLuA8PfmuihDe0ZacPrhzhxui/PRpwM6Ou+meAo4QABRyXV9pKRdwubmAvT0XcEOG8F0VITqDQq6RLnXi4fZt7hkMeXn3A87Dg++qCNEpFHKNdJkTDyUlXMCdOQM4OHDDJg0ezHdVhOgcOvHQFSkC7n//AxwduS24gQP5rooQnURbcl1NcTEwejQXcE5O3BYcBRwhzaKQ60pu3eIC7tw5oFcvLuAGDOC7KkJ0GoVcV1FUxAXc+fOAszMXcP37810VITqPQq4RnTy7qgi4/HzAxYULuH79+K6KkC6BQq4RnTu7evMmEBQEXLgAuLpyAde3L99VEdJlUMjpssJCLuAuXgTc3LiA69OH76oI6VLoEhIdIpMz5BSUoaSyBs5VZRg+6xkILl++H3Du7nyXSEiXQyGnI1LPFiFhzzkUVdTAUXIbW7a8A0F5Eap7ucD011+5oCOEaIx2V3VA6tkizP3xFIoqauAkKcHWLXFwLy/C35b2CHk6AamVJnyXSEiXRSHHM5mcIWHPOTCAC7jNcXArv4VrVg6YOn0pblrYIWHPOcjkLT+2kRDSNAo5nuUUlKGoogbOFcXYtjkOrhXFuGrliKnTuIBjAIoqapBTUMZ3qYR0SRRyjXT2dXIllVzAbd0cB5eKYhT05AKuyMJWbT5CiOYo5Brp7OvkXMqLsXXzQjhLSvBXTydMnbYUtyxs1OazMxd3Sj2E6Bs6u8qnv/7CsBkTIZDcxl/WvTB16scoMX9EZRYBAAdLMfx7W/NTIyFdHIUcX65cAUaPhuD6dVS598XU8fG4baYaZIJ/vsZHeMBIKFBfBiHkoWh3lQ+XL3N3Mly/DgwaBLPfj2DJv4PhYKm6S+pgKcY3LwzHOE9HfuokRA/Qllxnu3SJu9m+sJAbyffgQcDBAeMcgRAPB+UdD3bm3C4qbcER0j4Ucp3p4kUu4G7e5J7FcPAg92yGfxgJBQjs+0gLCyCEaEovdlcjIyPRs2dPTJ48WaV97969GDhwIPr374///ve/PFX3jwsXuF3Umze5p2llZakEHCGkY+hFyM2fPx8bNmxQaWtoaEBsbCwOHjyI06dP47PPPsOdO3f4KTA/nwu4oiJg6FAu4Ozs+KmFEAOjFyEXFBQEc3NzlbacnBwMGTIEvXr1gpmZGcLDw5GWltb5xZ0/zwXcrVtcwGVmck+4J4R0Ct5D7vDhw4iIiICTkxMEAgFSUlLU5klKSoK7uzvEYjECAgKQk5Pz0OXevHkTvXr1Uk736tULhYWF2iz94c6d4wKuuBjw8uKOwVHAEdKpeA85qVQKb29vJCUlNfn6tm3bEBsbi/j4eJw6dQre3t4ICwtDSUlJJ1eqof/9jzvJUFIC+PhwAWejficDIaRj8X52NTw8HOHh4c2+npiYiDlz5mDWrFkAgNWrV2Pfvn1Yu3YtFi5c2Oz7nJycVLbcCgsL4e/v3+z8tbW1qK2tVU5LJBIAQH19Perr61u9PgCAs2fRLSwMgtu3wXx80HDgAGBhAWi6nC5I0Vca9xnRCkPpf03Wj/eQa0ldXR1OnjyJuLg4ZZtQKERwcDCys7NbfK+/vz/Onj2LwsJCWFpa4sCBA1i0aFGz8y9duhQJCQlq7WlpaTA1NW11zeZXr2Lk4sUQSCQo79MHv7/5JuqPH2/1+/VFeno63yUYNH3v/+rq6lbPq9MhV1paCplMBvtGl1rY29sjPz9fOR0cHIy8vDxIpVI4OzsjOTkZgYGB+OKLLzB69GjI5XK89dZbeOSR5q9Bi4uLQ2xsrHJaIpHAxcUFoaGhsLCwaPI9MjnDyWt3UVpVCxszEfwqrsNk9mwIJBLIhw9HjwMHENKzZzt7oWupr69Heno6QkJCYGxszHc5BsdQ+l+xp9UaOh1yrZWRkdFk+9NPP42nn366VcsQiUQQiURq7cbGxk3+sjw4XDkAeBT/hc3b34OoWgL4+UGYlgahgQXcg5rrN9I59L3/NVk33k88tMTGxgZGRkYoLi5WaS8uLoaDg0OHfGZrxpN7cLhyABhSfAWbtr4Lq2oJ8hz7I+PLjYABBxwhukSnQ87ExAS+vr7IzMxUtsnlcmRmZiIwMLBDPvNh48k9OFw5AAy5dRmbtr6LnjWVOO04EC9O+RCLfi2k4coJ0RG8765WVVXh8uXLyumCggLk5ubC2toarq6uiI2NRVRUFPz8/ODv748VK1ZAKpUqz7ZqW1JSEpKSkiCTyZp8XTFcOQAI5TKs2PsFrGqqcMppIKKeW4JKUQ9I/hmunO5DJYR/vIfciRMnMHr0aOW04uB/VFQU1q9fjylTpuD27dtYvHgxbt26BR8fH6SmpqqdjNCW6OhoREdHQyKRwNLSUu31B4chlwuN8O+J7+Ctwz/gzfGxqBKZNjkfIYQ/vIdcUFAQGGt51y4mJgYxMTGdVFHLGg9DfsXGBa88895D5yOE8EOnj8nx4WEnHvx7W8PRUozmRnkTAHCk4coJ0RkUco087MSDkVCA+AgPAFALOhqunBDdQyHXBuM8HfHNC8NpuHJCugDej8npmoedXVUY5+lIw5UT0gVQyDXysLOrD6LhygnRfbS7SgjRaxRyjbTmti5CSNdBu6uNKHZXKyoqYGVlpdFoB4QbBaO6uhoSiUSvbxDXVYbS/4q/y4ddYwtQyDWrsrISAODi4sJzJYSQ5lRWVj702LmAtSYKDZBcLsfNmzdhbm4OgYDOmLaWYhy+69evNzsOH+k4htL/jDFUVlbCyckJQmHLR91oS64ZQqEQzs7OfJfRZVlYWOj1H5muM4T+f9gWnAKdeCCE6DUKOUKIXqOQI1olEokQHx/f5FDypONR/6ujEw+EEL1GW3KEEL1GIUcI0WsUcoQQvUYhRwjRaxRyhBC9RiFHOlRkZCR69uyJyZMnq7Tv3bsXAwcORP/+/fHf//6Xp+r0X1P9f/36dQQFBcHDwwNeXl5ITk7mscKOR5eQkA516NAhVFZW4ocffsBPP/0EAGhoaICHhweysrJgaWkJX19f/P7773jkERqAVNua6v+ioiIUFxfDx8cHt27dgq+vLy5evIgePXrwXG3HoC050qGCgoJgbm6u0paTk4MhQ4agV69eMDMzQ3h4ONLS0niqUL811f+Ojo7w8fEBADg4OMDGxgZlZWU8VNc5KORIsw4fPoyIiAg4OTlBIBAgJSVFbZ6kpCS4u7tDLBYjICAAOTk5D13uzZs30atXL+V0r169UFhYqM3S9UJH9f+DTp48CZlMptdDilHIkWZJpVJ4e3sjKSmpyde3bduG2NhYxMfH49SpU/D29kZYWBhKSko6uVL91NH9X1ZWhhkzZmDNmjXaLFv3MEJaAQD7+eefVdr8/f1ZdHS0clomkzEnJye2dOlSlfmysrLYpEmTlNO//fYbmzhxonJ6/vz5bNOmTR1TuJ7QZv8zxlhNTQ174okn2IYNGzqsZl1BW3KkTerq6nDy5EkEBwcr24RCIYKDg5Gdnd3ie/39/XH27FkUFhaiqqoKBw4cQFhYWEeXrFfa0/+MMcycORNjxozBiy++2NGl8o4GzSRtUlpaCplMBnt7e5V2e3t75OfnK6eDg4ORl5cHqVQKZ2dnJCcnIzAwEF988QVGjx4NuVyOt956i86saqg9/S+TybBt2zZ4eXkpj/Nt3LgRQ4cO7cxV6DQUcqRDZWRkNNn+9NNP4+mnn+7kagxPc/0vl8s7uRL+0O4qaRMbGxsYGRmhuLhYpb24uBgODg48VWU4qP9bj0KOtImJiQl8fX2RmZmpbJPL5cjMzERgYCCPlRkG6v/Wo91V0qyqqipcvnxZOV1QUIDc3FxYW1vD1dUVsbGxiIqKgp+fH/z9/bFixQpIpVLMmjWLx6r1B/W/lvB9epforqysLAZA7V9UVJRynpUrVzJXV1dmYmLC/P392bFjx/grWM9Q/2sH3btKCNFrdEyOEKLXKOQIIXqNQo4Qotco5Agheo1CjhCi1yjkCCF6jUKOEKLXKOQIIXqNQo4Qotco5Ah5CHd3d6xYsYLvMkgbUcgRncYYQ0NDA99laEVdXR3fJRgkCjmiVUFBQYiJiUFMTAwsLS1hY2ODRYsWQXGL9MaNG+Hn5wdzc3M4ODhg+vTpKg9eOXToEAQCAQ4cOABfX1+IRCIcPXoUV65cwYQJE2Bvbw8zMzOMGDFCbUBId3d3fPjhh5gxYwbMzMzg5uaG3bt34/bt25gwYQLMzMzg5eWFEydOqLzv6NGjeOKJJ9C9e3e4uLhg3rx5kEqlyvW5du0a3njjDQgEAggEgla9T1HPBx98gBkzZsDCwgIvv/wy6urqEBMTA0dHR4jFYri5uWHp0qVa/zmQB/A7PgDRN6NGjWJmZmZs/vz5LD8/n/3444/M1NSUrVmzhjHG2Pfff8/279/Prly5wrKzs1lgYCALDw9Xvl8x8oaXlxdLS0tjly9fZnfu3GG5ubls9erV7MyZM+zixYvsvffeY2KxmF27dk35Xjc3N2Ztbc1Wr17NLl68yObOncssLCzYuHHj2Pbt29mFCxfYxIkT2eDBg5lcLmeMMXb58mXWo0cPtnz5cnbx4kX222+/sWHDhrGZM2cyxhi7c+cOc3Z2ZkuWLGFFRUWsqKioVe9T1GNhYcE+//xzdvnyZXb58mX22WefMRcXF3b48GF29epVduTIEbZ58+YO/7kYMgo5olWjRo1SCRHGGHv77bfZ4MGDm5z/jz/+YABYZWUlY+x+yKWkpDz0s4YMGcJWrlypnHZzc2MvvPCCcrqoqIgBYIsWLVK2ZWdnMwDKsJo9ezZ7+eWXVZZ75MgRJhQK2b1795TLXb58uco8rX3fg08lY4yx1157jY0ZM0alf0jHot1VonWPPvqoym5dYGAgLl26BJlMhpMnTyIiIgKurq4wNzfHqFGjAAB///23yjL8/PxUpquqqrBgwQIMHjwYVlZWMDMzw/nz59Xe5+Xlpfxe8ZCXBx/QomhT7CLn5eVh/fr1MDMzU/4LCwuDXC5HQUFBs+vY2vc1Xo+ZM2ciNzcXAwcOxLx585CWltbsZxDtoJGBSaepqalBWFgYwsLCsGnTJtja2uLvv/9GWFiY2kH5Hj16qEwvWLAA6enp+Pzzz9GvXz90794dkydPVnufsbGx8ntF0DbVpniQS1VVFV555RXMmzdPrV5XV9dm16W172u8HsOHD0dBQQEOHDiAjIwMPPfccwgODsZPP/3U7GeR9qGQI1p3/Phxleljx46hf//+yM/Px507d7Bs2TK4uLgAgNpJgOb89ttvmDlzJiIjIwFwIXP16tV21zp8+HCcO3cO/fr1a3YeExMTyGQyjd/XHAsLC0yZMgVTpkzB5MmTMW7cOJSVlcHa2lrjZZGHo91VonV///03YmNjceHCBWzZsgUrV67E/Pnz4erqChMTE6xcuRJ//fUXdu/ejQ8++KBVy+zfvz927tyJ3Nxc5OXlYfr06Vp5rN7bb7+N33//HTExMcjNzcWlS5ewa9cuxMTEKOdxd3fH4cOHUVhYiNLS0la/rymJiYnYsmUL8vPzcfHiRSQnJ8PBwQFWVlbtXhfSNAo5onUzZszAvXv34O/vj+joaMyfPx8vv/wybG1tsX79eiQnJ8PDwwPLli3D559/3qplJiYmomfPnnjssccQERGBsLAwDB8+vN21enl54ddff8XFixfxxBNPYNiwYVi8eDGcnJyU8yxZsgRXr15F3759YWtr2+r3NcXc3Byffvop/Pz8MGLECFy9ehX79++HUEh/ih2FnvFAtCooKAg+Pj50hwDRGfTfByFEr1HIEUL0Gu2uEkL0Gm3JEUL0GoUcIUSvUcgRQvQahRwhRK9RyBFC9BqFHCFEr1HIEUL0GoUcIUSvUcgRQvTa/wM4VWbglJ1cKAAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.figure(figsize=(3, 3))\n", "# plot the line\n", "plt.plot([q[0] for q in raw], [10**(m*np.log10(q[0]) + c) for q in raw], label='linear regression', color='r')\n", "# plot the raw data\n", "plt.scatter([q[0] for q in raw], [q[1] for q in raw], label='raw data')\n", "plt.xscale('log')\n", "plt.yscale('log')\n", "plt.xlabel('parameters')\n", "plt.ylabel('tokens')\n", "plt.title('compute optimal models')\n", "plt.grid()\n" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "predicted parameters for 1.240000e+08 tokens: 2.292426e+09\n" ] } ], "source": [ "xquery = 124e6 # query model size here (e.g. GPT-2 small is 124M)\n", "yquery = 10**(m*np.log10(xquery) + c)\n", "print(f\"predicted parameters for {xquery:e} tokens: {yquery:e}\")" ] } ], "metadata": { "kernelspec": { "display_name": "MLexperiments", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.8" }, "orig_nbformat": 4, "vscode": { "interpreter": { "hash": "37637b3a7f3e35bfec4c00b744758bb7c354a471edaa8e8e5b028b90b494f3c9" } } }, "nbformat": 4, "nbformat_minor": 2 } ================================================ FILE: train.py ================================================ """ This training script can be run both on a single gpu in debug mode, and also in a larger training run with distributed data parallel (ddp). To run on a single GPU, example: $ python train.py --batch_size=32 --compile=False To run with DDP on 4 gpus on 1 node, example: $ torchrun --standalone --nproc_per_node=4 train.py To run with DDP on 4 gpus across 2 nodes, example: - Run on the first (master) node with example IP 123.456.123.456: $ torchrun --nproc_per_node=8 --nnodes=2 --node_rank=0 --master_addr=123.456.123.456 --master_port=1234 train.py - Run on the worker node: $ torchrun --nproc_per_node=8 --nnodes=2 --node_rank=1 --master_addr=123.456.123.456 --master_port=1234 train.py (If your cluster does not have Infiniband interconnect prepend NCCL_IB_DISABLE=1) """ import os import time import math import pickle from contextlib import nullcontext import numpy as np import torch from model import GPTConfig, GPT import yaml from torch.nn.parallel import DistributedDataParallel as DDP from trainers.trainer import Trainer # load config.yaml from current directory with open('config/config.yaml') as f: conf = yaml.load(f, Loader=yaml.FullLoader) # nested dictionary structure config = {} for k, v in conf.items(): for k2, v2 in v.items(): config[k2] = v2 # convert to dotdict print(config) trainer = Trainer(config) trainer.train() ================================================ FILE: train_reward_model.py ================================================ from trainers.reward_trainer import RewardModelTrainer import yaml from datasets import load_dataset from torch.utils.data import Dataset from tqdm import tqdm import tiktoken import torch # with inspiration from CarperAI's trlx library with open('config/config_reward.yaml') as f: conf = yaml.load(f, Loader=yaml.FullLoader) # nested dictionary structure config = {} for k, v in conf.items(): for k2, v2 in v.items(): config[k2] = v2 print(config) def create_comparison_dataset(path="CarperAI/openai_summarize_comparisons", split="train"): dataset = load_dataset(path, split=split) pairs = [] for sample in tqdm(dataset): pair = {} prompt = sample["prompt"] chosen_summary = sample["chosen"] rejected_summary = sample["rejected"] if chosen_summary == rejected_summary: continue if len(chosen_summary.split()) < 5 or len(rejected_summary.split()) < 5: continue pair["chosen"] = prompt + "\n" + chosen_summary pair["rejected"] = prompt + "\n" + rejected_summary pairs.append(pair) return pairs class PairwiseDataset(Dataset): def __init__(self, pairs, max_length): # self.chosen_input_ids = [] # self.chosen_attn_masks = [] # self.rejected_input_ids = [] # self.rejected_attn_masks = [] self.chosens = [] self.rejecteds = [] self.enc = tiktoken.get_encoding("gpt2") for pair in tqdm(pairs): chosen_enc = self.enc.encode("<|startoftext|>" + pair['chosen'] + "<|endoftext|>", allowed_special="all")[-max_length:] rejected_enc = self.enc.encode("<|startoftext|>" + pair['rejected'] + "<|endoftext|>", allowed_special="all")[-max_length:] self.chosens.append(chosen_enc) self.rejecteds.append(rejected_enc) def __len__(self): return len(self.chosens) def __getitem__(self, idx): return ( self.chosens[idx], self.rejecteds[idx], ) class DataCollatorReward: def __call__(self, data): batch = {} batch["chosen_ids"] = torch.tensor([f[0] for f in data]) batch["rejected_ids"] = torch.tensor([f[1] for f in data]) return batch def collate_fn(data): batch = {} batch["chosen_ids"] = torch.tensor([f[0] for f in data]) batch["rejected_ids"] = torch.tensor([f[1] for f in data]) return batch data_path = "CarperAI/openai_summarize_comparisons" train_pairs = create_comparison_dataset(data_path, "train") val_pairs = create_comparison_dataset(data_path, "test") # Make pairwise datasets for training print("Creating pairwise datasets") train_dataset = PairwiseDataset(train_pairs, max_length=config['block_size']) val_dataset = PairwiseDataset(val_pairs, max_length=config['block_size']) trainer = RewardModelTrainer(config, train_dataset, val_dataset, collate_fn=collate_fn) trainer.train() ================================================ FILE: train_reward_model_simple.py ================================================ from trainers.reward_trainer import ProbRewardModelTrainer import yaml from tqdm import tqdm import tiktoken import torch with open('config/config_reward.yaml') as f: conf = yaml.load(f, Loader=yaml.FullLoader) # nested dictionary structure config = {} for k, v in conf.items(): for k2, v2 in v.items(): config[k2] = v2 print(config) trainer = ProbRewardModelTrainer(config, discrete_reward=True) trainer.train() ================================================ FILE: train_rl.py ================================================ import yaml from torch.nn.parallel import DistributedDataParallel as DDP from trainers.rl_trainer import PolicyGradientTrainer, GumbelTrainer # load config.yaml from current directory with open('config/config_rl.yaml') as f: conf = yaml.load(f, Loader=yaml.FullLoader) # nested dictionary structure config = {} for k, v in conf.items(): for k2, v2 in v.items(): config[k2] = v2 # convert to dotdict if config['method'] == 'gumbel': print('Using Gumbel method') assert config['hard_code_reward'] == False, 'hard_code_reward must be False for Gumbel method' trainer = GumbelTrainer(config) elif config['method'] == 'pg': print('Using Policy Gradient method') trainer = PolicyGradientTrainer(config) else: raise NotImplementedError trainer.train() ================================================ FILE: trainers/reward_trainer.py ================================================ import torch import numpy as np from torch.nn.parallel import DistributedDataParallel as DDP from torch.distributed import destroy_process_group import wandb import time, os from model import RLHF from trainers.trainer import Trainer # This one for reward models similar to InstructGPT paper (rewards based on comparisons) class RewardModelTrainer(Trainer): def __init__(self, config, train_data, val_data, collate_fn): super().__init__(config) import tiktoken self.enc = tiktoken.get_encoding("gpt2") self.mode = 'reward' from torch.utils.data import DataLoader train_dataloader = DataLoader(train_data, batch_size=self.batch_size, shuffle=True, collate_fn=collate_fn) val_dataloader = DataLoader(val_data, batch_size=self.batch_size, shuffle=True, collate_fn=collate_fn) self.train_dataloader = train_dataloader self.val_dataloader = val_dataloader def get_batch(self, split): dataloader = self.train_dataloader if split == 'train' else self.val_dataloader batch = next(iter(dataloader)) x, y = batch['chosen_ids'], batch['rejected_ids'] if self.device_type == 'cuda': # pin arrays x,y, which allows us to move them to GPU asynchronously (non_blocking=True) x, y = x.pin_memory().to(self.device, non_blocking=True), y.pin_memory().to(self.device, non_blocking=True) else: x, y = x.to(self.device), y.to(self.device) return x, y # helps estimate an arbitrarily accurate loss over either split using many batches @torch.no_grad() def estimate_loss(self, model, ctx): out = {} model.eval() for split in ['train', 'val']: losses = torch.zeros(self.eval_iters) for k in range(self.eval_iters): chosen, rejected = self.get_batch(split) with ctx: reward_chosen = model(chosen) reward_rejected = model(rejected) loss = -torch.log(torch.sigmoid(reward_chosen - reward_rejected)).mean() losses[k] = loss.item() out[split] = losses.mean() model.train() return out def evaluate(self, model, ctx): losses = self.estimate_loss(model, ctx) print(f"step {self.iter_num}: train loss {losses['train']:.4f}, val loss {losses['val']:.4f}") if self.wandb_log: wandb.log({ "iter": self.iter_num, "train/loss": losses['train'], "val/loss": losses['val'], "lr": self.lr, "mfu": self.running_mfu*100, # convert to percentage }) if losses['val'] < self.best_val_loss or self.always_save_checkpoint: self.best_val_loss = losses['val'] raw_model = model.module if self.ddp else model if self.iter_num > 0: checkpoint = { 'model': raw_model.state_dict(), 'optimizer': self.optimizer.state_dict(), 'model_args': self.model_args, 'iter_num': self.iter_num, 'best_val_loss': self.best_val_loss, 'config': self.config, } print(f"saving checkpoint to {self.config['out_dir_multihead']}") torch.save(checkpoint, os.path.join(self.config['out_dir_multihead'], 'ckpt.pt')) def train(self): # set up distributed training self.setup_ddp() ctx, meta_vocab_size = self.setup() # model init model = self.init_model() model = RLHF(model, self.mode) print('Config of model: ', model.config) if self.config['init_multihead_from'] == 'scratch': print("initializing multihead from scratch") else: if self.config['init_multihead_from'] == 'resume': print(f"Resuming training from {self.config['out_dir_multihead']}") # resume training from a checkpoint. ckpt_path = os.path.join(self.config['out_dir_multihead'], 'ckpt.pt') checkpoint = torch.load(ckpt_path, map_location=self.device) state_dict = checkpoint['model'] # fix the keys of the state dictionary :( # honestly no idea how checkpoints sometimes get this prefix, have to debug more unwanted_prefix = '_orig_mod.' for k,v in list(state_dict.items()): if k.startswith(unwanted_prefix): state_dict[k[len(unwanted_prefix):]] = state_dict.pop(k) model.load_state_dict(state_dict) model.to(self.device) # self.optimizer = torch.optim.AdamW(model.model.reward_head.parameters(), lr=1e-3) self.optimizer = torch.optim.AdamW(model.model.parameters(), lr=1e-4) model = self.setup_model(model) # logging if self.wandb_log and self.master_process: wandb.init(project=self.wandb_project, name=self.wandb_run_name, config=self.config) # training loop chosen, rejected = self.get_batch('train') # fetch the very first batch t0 = time.time() local_iter_num = 0 # number of iterations in the lifetime of this process self.running_mfu = -1.0 loss = None while True: # determine and set the learning rate for this iteration lr = self.get_lr(self.iter_num) if self.decay_lr else self.learning_rate for param_group in self.optimizer.param_groups: param_group['lr'] = lr # # every once in a while evaluate the loss on train and val sets if self.iter_num % self.eval_interval == 0 and self.master_process: self.evaluate(model, ctx) if self.iter_num == 0 and self.eval_only: break # sample a batch of data chosen, rejected = self.get_batch('train') # evaluate the loss reward_chosen = model(chosen) reward_rejected = model(rejected) loss = -torch.log(torch.sigmoid(reward_chosen - reward_rejected)).mean() self.optimizer.zero_grad(set_to_none=True) loss.backward() self.optimizer.step() # timing and logging t1 = time.time() # dt = t1 - t0 t0 = t1 self.iter_num += 1 local_iter_num += 1 # termination conditions if self.iter_num > self.max_iters: break if self.ddp: destroy_process_group() # This one is for reward models which output a probability of reward directly from a given text (no comparison) class ProbRewardModelTrainer(Trainer): def __init__(self, config, discrete_reward=False): super().__init__(config) import tiktoken self.enc = tiktoken.get_encoding("gpt2") self.mode = 'reward' self.discrete_reward = discrete_reward def get_batch(self, split): # generate a small batch of data of inputs x and targets y data = self.train_data if split == 'train' else self.val_data ix = torch.randint(len(data) - self.block_size, (self.batch_size,)) x = torch.stack([torch.from_numpy((data[i:i+self.block_size]).astype(np.int64)) for i in ix]) y = torch.stack([self.reward(torch.from_numpy((data[i+1:i+1+self.block_size]).astype(np.int64))) for i in ix]) if self.device_type == 'cuda': # pin arrays x,y, which allows us to move them to GPU asynchronously (non_blocking=True) x, y = x.pin_memory().to(self.device, non_blocking=True), y.pin_memory().to(self.device, non_blocking=True) else: x, y = x.to(self.device), y.to(self.device) return x, y def reward(self, sequence, t='and'): if t in self.enc.decode(sequence.tolist()): # print('hello') return torch.tensor([0.0,1.0]) else: return torch.tensor([1.0, 0.0]) def evaluate(self, model, ctx, X, lr): losses = self.estimate_loss(model, ctx) print(f"step {self.iter_num}: train loss {losses['train']:.4f}, val loss {losses['val']:.4f}") text = self.enc.decode(X[self.iter_num % self.eval_interval].tolist()) try: reward_probs, _ = model(X[self.iter_num % self.eval_interval].unsqueeze(0)) actual_reward_probs = self.reward(X[self.iter_num % self.eval_interval])[1] print("input: ", text[:30], f"expect {actual_reward_probs}, reward: {reward_probs[0][-1]} \n") except: pass # test_text = text[:4] + 'z' + text[4 + 1:-1] test_text = list(text) test_text[3] = ' ' test_text[4] = 'a' test_text[5] = 'n' test_text[6] = 'd' test_text[7] = ' ' test_text = ''.join(test_text) try: test_text_enc = torch.tensor(self.enc.encode(test_text)[:self.block_size]).unsqueeze(0) test_reward_probs, _ = model(test_text_enc.to(self.device)) actual_reward_probs = self.reward(test_text_enc[0].to(self.device))[1] print("input: ", test_text[:30], f"expect {actual_reward_probs}, reward: {test_reward_probs[0][-1]} \n") except: pass if self.wandb_log: wandb.log({ "iter": self.iter_num, "train/loss": losses['train'], "val/loss": losses['val'], "lr": lr, # "mfu": self.running_mfu*100, # convert to percentage }) if losses['val'] < self.best_val_loss or self.always_save_checkpoint: self.best_val_loss = losses['val'] raw_model = model.module if self.ddp else model if self.iter_num > 0: checkpoint = { 'model': raw_model.state_dict(), 'optimizer': self.optimizer.state_dict(), 'model_args': self.model_args, 'iter_num': self.iter_num, 'best_val_loss': self.best_val_loss, 'config': self.config, } print(f"saving checkpoint to {self.config['out_dir_multihead']}") torch.save(checkpoint, os.path.join(self.config['out_dir_multihead'], 'ckpt.pt')) def train(self): # set up distributed training self.setup_ddp() ctx, meta_vocab_size = self.setup() # model init if self.master_process: os.makedirs(self.config['out_dir_multihead'], exist_ok=True) model = self.init_model() model = RLHF(model, self.mode, discrete_reward=self.discrete_reward) if self.config['init_multihead_from'] == 'scratch': print("initializing multihead from scratch") else: if self.config['init_multihead_from'] == 'resume': print(f"Resuming training from {self.config['out_dir_multihead']}") # resume training from a checkpoint. ckpt_path = os.path.join(self.config['out_dir_multihead'], 'ckpt.pt') checkpoint = torch.load(ckpt_path, map_location=self.device) state_dict = checkpoint['model'] # fix the keys of the state dictionary :( # honestly no idea how checkpoints sometimes get this prefix, have to debug more unwanted_prefix = '_orig_mod.' for k,v in list(state_dict.items()): if k.startswith(unwanted_prefix): state_dict[k[len(unwanted_prefix):]] = state_dict.pop(k) model.load_state_dict(state_dict) model.to(self.device) # self.optimizer = torch.optim.AdamW(model.model.reward_head.parameters(), lr=1e-3) self.optimizer = torch.optim.AdamW(model.model.parameters(), lr=1e-3) model = self.setup_model(model) # logging if self.wandb_log and self.master_process: wandb.init(project=self.wandb_project, name=self.wandb_run_name, config=self.config) # training loop X, Y = self.get_batch('train') # fetch the very first batch t0 = time.time() local_iter_num = 0 # number of iterations in the lifetime of this process self.running_mfu = -1.0 while True: # determine and set the learning rate for this iteration lr = self.get_lr(self.iter_num) if self.decay_lr else self.learning_rate for param_group in self.optimizer.param_groups: param_group['lr'] = lr # every once in a while evaluate the loss on train and val sets if self.iter_num % self.eval_interval == 0 and self.master_process: self.evaluate(model, ctx, X, lr) if self.iter_num == 0 and self.eval_only: break # sample a batch of data X, Y = self.get_batch('train') # evaluate the loss logits, loss = model(X, Y) self.optimizer.zero_grad(set_to_none=True) loss.backward() self.optimizer.step() # timing and logging t1 = time.time() # dt = t1 - t0 t0 = t1 self.iter_num += 1 local_iter_num += 1 # termination conditions if self.iter_num > self.max_iters: break if self.ddp: destroy_process_group() ================================================ FILE: trainers/rl_trainer.py ================================================ import torch import numpy as np from torch.nn.parallel import DistributedDataParallel as DDP from torch.distributed import destroy_process_group import time, os from model import RLHF from trainers.trainer import Trainer # TODO: this works but is currently crude and incomplete, critic implementation plus PPO are obvious next steps class PolicyGradientTrainer(Trainer): def __init__(self, config): super().__init__(config) import tiktoken self.enc = tiktoken.get_encoding("gpt2") self.mode = 'RL' def train(self): self.setup_ddp() ctx, meta_vocab_size = self.setup() # model init model = self.init_model() model = RLHF(model, self.mode, discrete_reward=self.config['discrete_reward']) if self.config['init_multihead_from'] == 'scratch': print("initializing multihead from scratch") else: if self.config['init_multihead_from'] == 'resume': print(f"Resuming training from {self.config['out_dir_multihead']}") # resume training from a checkpoint. ckpt_path = os.path.join(self.config['out_dir_multihead'], 'ckpt.pt') checkpoint = torch.load(ckpt_path, map_location=self.device) state_dict = checkpoint['model'] # fix the keys of the state dictionary :( # honestly no idea how checkpoints sometimes get this prefix, have to debug more unwanted_prefix = '_orig_mod.' for k,v in list(state_dict.items()): if k.startswith(unwanted_prefix): state_dict[k[len(unwanted_prefix):]] = state_dict.pop(k) model.load_state_dict(state_dict) if self.config['hard_code_reward']: reward_model = None print('Using hard-coded reward') else: print('Using learned reward model') if self.config['separate_reward_model']: import copy reward_model = copy.deepcopy(model) print('Reward model instantiated separately') else: reward_model = model print('Reward model and actor model share backbone') reward_model.to(self.device) model.to(self.device) # actor_optimizer = torch.optim.AdamW(model.model.policy_head.parameters(), lr=1e-2) actor_optimizer = torch.optim.AdamW(model.parameters(), lr=1e-3) last_time = time.time() rews_all = [] max_iters = 100000 X, Y = self.get_batch('train') # fetch the very first batch t0 = time.time() for iter in range(max_iters): states, log_probs, log_probs_reference, rewards, advantages = model.generate( X, self.block_size, self.device, self.block_size, reward_model=reward_model, hard_code_reward=self.config['hard_code_reward']) # minus KL divergence rets = advantages * log_probs.squeeze() #- 1*(log_probs-log_probs_reference) #- 0.05*log_probs actor_loss = -rets.sum() actor_optimizer.zero_grad(set_to_none=True) actor_loss.backward() actor_optimizer.step() torch.mean(rewards) rews_all.append(rewards.mean().detach().cpu().numpy()) if iter % 1000 == 0: t1 = time.time() print(f'iter: {iter}, time: {t1-t0}') # print(actor_loss, critic_loss) print(f'Actor loss: {actor_loss}, iter: {iter}') print(f'rets: {np.mean(rews_all[-1000:])}') current_time = time.time() # print(current_time - last_time) last_time = current_time text = model.generate(X, self.block_size, self.device, self.block_size, reward_model=reward_model)[0] for i in range(1): text_i = text[i,:] # print(reward(text_i)) try: print(self.enc.decode(text_i.tolist())) except: continue class GumbelTrainer(Trainer): def __init__(self, config): super().__init__(config) import tiktoken self.enc = tiktoken.get_encoding("gpt2") self.mode = 'RL' def train(self): self.setup_ddp() ctx, meta_vocab_size = self.setup() # model init model = self.init_model() rl_model = RLHF(model, self.mode, discrete_reward=self.config['discrete_reward']) # The current approach is to use a separate reward model because otherwise optimisation of the reward model changes upstream parameters impacting performance of the multihead # I therefore load the language model from 'out_dir' and the reward model from 'out_dir_multihead' if self.config['init_multihead_from'] == 'scratch': print("initializing multihead from scratch") else: if self.config['init_multihead_from'] == 'resume': print(f"Resuming training from {self.config['out_dir']}") # resume training from a checkpoint. ckpt_path = os.path.join(self.config['out_dir'], 'ckpt.pt') checkpoint = torch.load(ckpt_path, map_location=self.device) state_dict = checkpoint['model'] # fix the keys of the state dictionary :( # honestly no idea how checkpoints sometimes get this prefix, have to debug more unwanted_prefix = '_orig_mod.' for k,v in list(state_dict.items()): if k.startswith(unwanted_prefix): state_dict[k[len(unwanted_prefix):]] = state_dict.pop(k) model.load_state_dict(state_dict) separate_reward_model = True if separate_reward_model: print('Reward model instantiated as copy') import copy reward_model = copy.deepcopy(model) print(f"Resuming reward model from {self.config['out_dir_multihead']}") reward_model = RLHF(reward_model, self.mode, discrete_reward=self.config['discrete_reward']) # resume training from a checkpoint. ckpt_path = os.path.join(self.config['out_dir_multihead'], 'ckpt.pt') checkpoint = torch.load(ckpt_path, map_location=self.device) state_dict = checkpoint['model'] # fix the keys of the state dictionary :( # honestly no idea how checkpoints sometimes get this prefix, have to debug more unwanted_prefix = '_orig_mod.' for k,v in list(state_dict.items()): if k.startswith(unwanted_prefix): state_dict[k[len(unwanted_prefix):]] = state_dict.pop(k) reward_model.load_state_dict(state_dict) else: reward_model = rl_model rl_model.to(self.device) reward_model.to(self.device) gumbel_optimizer = torch.optim.AdamW(rl_model.parameters(), lr=1e-3) # initialize a GradScaler. If enabled=False scaler is a no-op scaler = torch.cuda.amp.GradScaler(enabled=(self.dtype == 'float16')) last_time = time.time() rews_all = [] max_iters = 100000 X, Y = self.get_batch('train') # fetch the very first batch X = torch.zeros((X.shape[0], 1), dtype=torch.long).to(self.device) # for now there is no prompt t0 = time.time() for iter in range(max_iters): for micro_step in range(self.gradient_accumulation_steps): if self.ddp: # in DDP training we only need to sync gradients at the last micro step. # the official way to do this is with model.no_sync() context manager, but # I really dislike that this bloats the code and forces us to repeat code # looking at the source of that context manager, it just toggles this variable rl_model.require_backward_grad_sync = (micro_step == self.gradient_accumulation_steps - 1) with ctx: states, rewards = rl_model.generate_gumbel(X, self.config['episode_length'], self.device, self.block_size, reward_model=reward_model) mean_reward = rewards.mean() loss = -mean_reward # # immediately async prefetch next batch while model is doing the forward pass on the GPU # X, Y = self.get_batch('train') # backward pass, with gradient scaling if training in fp16 scaler.scale(loss).backward() # clip the gradient if self.grad_clip != 0.0: scaler.unscale_(gumbel_optimizer) torch.nn.utils.clip_grad_norm_(rl_model.parameters(), self.grad_clip) # step the optimizer and scaler if training in fp16 scaler.step(gumbel_optimizer) scaler.update() # flush the gradients as soon as we can, no need for this memory anymore gumbel_optimizer.zero_grad(set_to_none=True) rews_all.append(mean_reward.detach().cpu().numpy()) eval_interval = self.config['eval_interval'] if iter % eval_interval == 0: t1 = time.time() print(f'iter: {iter}, time: {t1-t0}') print(f'rets: {np.mean(rews_all[-eval_interval:])}') current_time = time.time() # print(current_time - last_time) last_time = current_time text = rl_model.generate(X, self.config['episode_length'], self.device, self.block_size, reward_model=reward_model)[0] for i in range(1): text_i = text[i,:] # print(reward(text_i)) try: print(self.enc.decode(text_i.tolist())) except: continue ================================================ FILE: trainers/trainer.py ================================================ import os import time import math import pickle from contextlib import nullcontext import numpy as np import torch from model import GPTConfig, GPT, RLHF import yaml from torch.nn.parallel import DistributedDataParallel as DDP from torch.distributed import init_process_group, destroy_process_group from utils import dotdict import wandb from torch.nn import functional as F class Trainer(): def __init__(self, config): self.config = config self.from_config(config) self.model_args = dict(n_layer=self.n_layer, n_head=self.n_head, n_embd=self.n_embd, block_size=self.block_size, bias=self.bias, vocab_size=None, dropout=self.dropout) # start with model_args from command line self.meta_vocab_size = None self.iter_num = 0 self.best_val_loss = float('inf') def from_config(self, config): config = dotdict(config) # IO self.out_dir = config.out_dir self.eval_interval = config.eval_interval self.log_interval = config.log_interval self.eval_iters = config.eval_iters self.eval_only = config.eval_only self.always_save_checkpoint = config.always_save_checkpoint self.init_from = config.init_from # wandb self.wandb_log = config.wandb_log self.wandb_project = config.wandb_project self.wandb_run_name = config.wandb_run_name # data self.dataset = config.dataset self.gradient_accumulation_steps = config.gradient_accumulation_steps self.batch_size = config.batch_size self.block_size = config.block_size # model self.n_layer = config.n_layer self.n_head = config.n_head self.n_embd = config.n_embd self.dropout = config.dropout self.bias = config.bias # optimizer self.learning_rate = config.learning_rate self.max_iters = config.max_iters self.weight_decay = config.weight_decay self.beta1 = config.beta1 self.beta2 = config.beta2 self.grad_clip = config.grad_clip self.decay_lr = config.decay_lr self.warmup_iters = config.warmup_iters self.lr_decay_iters = config.lr_decay_iters self.min_lr = config.min_lr # DDP self.backend = config.backend # system self.device = config.device self.dtype = config.dtype self.compile = config.compile print(self.out_dir) def setup_ddp(self): self.ddp = int(os.environ.get('RANK', -1)) != -1 # is this a ddp run? if self.ddp: init_process_group(backend=self.backend) self.ddp_rank = int(os.environ['RANK']) self.ddp_local_rank = int(os.environ['LOCAL_RANK']) self.world_size = int(os.environ['WORLD_SIZE']) # total number of training processes self.device = f'cuda:{self.ddp_local_rank}' torch.cuda.set_device(self.device) self.master_process = self.ddp_rank == 0 # this process will do logging, checkpointing etc. self.seed_offset = self.ddp_rank # each process gets a different seed else: # if not ddp, we are running on a single gpu, and one process self.world_size = 1 self.master_process = True self.seed_offset = 0 self.ddp_local_rank = None def get_batch(self, split): data = self.train_data if split == 'train' else self.val_data ix = torch.randint(len(data) - self.block_size, (self.batch_size,)) x = torch.stack([torch.from_numpy((data[i:i+self.block_size]).astype(np.int64)) for i in ix]) y = torch.stack([torch.from_numpy((data[i+1:i+1+self.block_size]).astype(np.int64)) for i in ix]) if self.device_type == 'cuda': # pin arrays x,y, which allows us to move them to GPU asynchronously (non_blocking=True) x, y = x.pin_memory().to(self.device, non_blocking=True), y.pin_memory().to(self.device, non_blocking=True) else: x, y = x.to(self.device), y.to(self.device) return x, y def get_lr(self, it): # learning rate decay scheduler (cosine with warmup) # 1) linear warmup for warmup_iters steps if it < self.warmup_iters: return self.learning_rate * it / self.warmup_iters # 2) if it > lr_decay_iters, return min learning rate if it > self.lr_decay_iters: return self.min_lr # 3) in between, use cosine decay down to min learning rate decay_ratio = (it - self.warmup_iters) / (self.lr_decay_iters - self.warmup_iters) assert 0 <= decay_ratio <= 1 coeff = 0.5 * (1.0 + math.cos(math.pi * decay_ratio)) # coeff ranges 0..1 return self.min_lr + coeff * (self.learning_rate - self.min_lr) def init_model(self): if self.init_from == 'scratch': # init a new model from scratch print("Initializing a new model from scratch") # determine the vocab size we'll use for from-scratch training if self.meta_vocab_size is None: print("defaulting to vocab_size of GPT-2 to 50304 (50257 rounded up for efficiency)") self.model_args['vocab_size'] = self.meta_vocab_size if self.meta_vocab_size is not None else 50304 gptconf = GPTConfig(**self.model_args) model = GPT(gptconf) elif self.init_from == 'resume': print(f"Resuming training from {self.out_dir}") # resume training from a checkpoint. ckpt_path = os.path.join(self.out_dir, 'ckpt.pt') checkpoint = torch.load(ckpt_path, map_location=self.device) checkpoint_model_args = checkpoint['model_args'] # force these config attributes to be equal otherwise we can't even resume training # the rest of the attributes (e.g. dropout) can stay as desired from command line for k in ['n_layer', 'n_head', 'n_embd', 'block_size', 'bias', 'vocab_size']: self.model_args[k] = checkpoint_model_args[k] # create the model gptconf = GPTConfig(**self.model_args) model = GPT(gptconf) state_dict = checkpoint['model'] # fix the keys of the state dictionary :( # honestly no idea how checkpoints sometimes get this prefix, have to debug more unwanted_prefix = '_orig_mod.' for k,v in list(state_dict.items()): if k.startswith(unwanted_prefix): state_dict[k[len(unwanted_prefix):]] = state_dict.pop(k) model.load_state_dict(state_dict) self.iter_num = checkpoint['iter_num'] self.best_val_loss = checkpoint['best_val_loss'] self.checkpoint = checkpoint elif self.init_from.startswith('gpt2'): print(f"Initializing from OpenAI GPT-2 weights: {self.init_from}") # initialize from OpenAI GPT-2 weights override_args = dict(dropout=self.dropout) model = GPT.from_pretrained(init_from, override_args) # read off the created config params, so we can store them into checkpoint correctly for k in ['n_layer', 'n_head', 'n_embd', 'block_size', 'bias', 'vocab_size']: self.model_args[k] = getattr(model.config, k) # crop down the model block size if desired, using model surgery if self.block_size < model.config.block_size: model.crop_block_size(self.block_size) self.model_args['block_size'] = self.block_size # so that the checkpoint will have the right value return model def setup(self): if self.master_process: os.makedirs(self.out_dir, exist_ok=True) torch.manual_seed(1337 + self.seed_offset) torch.backends.cuda.matmul.allow_tf32 = True # allow tf32 on matmul torch.backends.cudnn.allow_tf32 = True # allow tf32 on cudnn self.device_type = 'cuda' if 'cuda' in self.device else 'cpu' # for later use in torch.autocast # note: float16 data type will automatically use a GradScaler ptdtype = {'float32': torch.float32, 'bfloat16': torch.bfloat16, 'float16': torch.float16}[self.dtype] ctx = nullcontext() if self.device_type == 'cpu' else torch.amp.autocast(device_type=self.device_type, dtype=ptdtype) # poor man's data loader data_dir = os.path.join('data', self.dataset) self.train_data = np.memmap(os.path.join(data_dir, 'train.bin'), dtype=np.uint16, mode='r') self.val_data = np.memmap(os.path.join(data_dir, 'val.bin'), dtype=np.uint16, mode='r') # attempt to derive vocab_size from the dataset meta_path = os.path.join(data_dir, 'meta.pkl') meta_vocab_size = None if os.path.exists(meta_path): with open(meta_path, 'rb') as f: meta = pickle.load(f) meta_vocab_size = meta['vocab_size'] print(f"found vocab_size = {meta_vocab_size} (inside {meta_path})") return ctx, meta_vocab_size def setup_model(self, model): # compile the model if self.compile: print("compiling the model... (takes a ~minute)") unoptimized_model = model model = torch.compile(model) # requires PyTorch 2.0 # wrap model into DDP container if self.ddp: model = DDP(model, device_ids=[self.ddp_local_rank]) return model def evaluate(self, model, ctx, lr): losses = self.estimate_loss(model, ctx) print(f"step {self.iter_num}: train loss {losses['train']:.4f}, val loss {losses['val']:.4f}") if self.wandb_log: wandb.log({ "iter": self.iter_num, "train/loss": losses['train'], "val/loss": losses['val'], "lr": lr, # "mfu": running_mfu*100, # convert to percentage }) if losses['val'] < self.best_val_loss or self.always_save_checkpoint: self.best_val_loss = losses['val'] raw_model = model.module if self.ddp else model if self.iter_num > 0: checkpoint = { 'model': raw_model.state_dict(), 'optimizer': self.optimizer.state_dict(), 'model_args': self.model_args, 'iter_num': self.iter_num, 'best_val_loss': self.best_val_loss, 'config': self.config, } print(f"saving checkpoint to {self.out_dir}") torch.save(checkpoint, os.path.join(self.out_dir, 'ckpt.pt')) def train(self): # set up distributed training self.setup_ddp() ctx, meta_vocab_size = self.setup() # model init model = self.init_model() model.to(self.device) # initialize a GradScaler. If enabled=False scaler is a no-op scaler = torch.cuda.amp.GradScaler(enabled=(self.dtype == 'float16')) # optimizer self.optimizer = model.configure_optimizers(self.weight_decay, self.learning_rate, \ (self.beta1, self.beta2), self.device_type) if self.init_from == 'resume': self.optimizer.load_state_dict(self.checkpoint['optimizer']) model = self.setup_model(model) # logging if self.wandb_log and self.master_process: wandb.init(project=self.wandb_project, name=self.wandb_run_name, config=self.config) # training loop X, Y = self.get_batch('train') # fetch the very first batch t0 = time.time() local_iter_num = 0 # number of iterations in the lifetime of this process running_mfu = -1.0 while True: # determine and set the learning rate for this iteration lr = self.get_lr(self.iter_num) if self.decay_lr else self.learning_rate for param_group in self.optimizer.param_groups: param_group['lr'] = lr # evaluate the loss on train/val sets and write checkpoints if self.iter_num % self.eval_interval == 0 and self.master_process: self.evaluate(model, ctx, lr) if self.iter_num == 0 and self.eval_only: break # forward backward update, with optional gradient accumulation to simulate larger batch size # and using the GradScaler if data type is float16 for micro_step in range(self.gradient_accumulation_steps): if self.ddp: # in DDP training we only need to sync gradients at the last micro step. # the official way to do this is with model.no_sync() context manager, but # I really dislike that this bloats the code and forces us to repeat code # looking at the source of that context manager, it just toggles this variable model.require_backward_grad_sync = (micro_step == self.gradient_accumulation_steps - 1) with ctx: logits, loss = model(X, Y) # immediately async prefetch next batch while model is doing the forward pass on the GPU X, Y = self.get_batch('train') # backward pass, with gradient scaling if training in fp16 scaler.scale(loss).backward() # clip the gradient if self.grad_clip != 0.0: scaler.unscale_(self.optimizer) torch.nn.utils.clip_grad_norm_(model.parameters(), self.grad_clip) # step the optimizer and scaler if training in fp16 scaler.step(self.optimizer) scaler.update() # flush the gradients as soon as we can, no need for this memory anymore self.optimizer.zero_grad(set_to_none=True) # timing and logging t1 = time.time() dt = t1 - t0 t0 = t1 if self.iter_num % self.log_interval == 0 and self.master_process: lossf = loss.item() # loss as float. note: this is a CPU-GPU sync point if local_iter_num >= 5: # let the training loop settle a bit mfu = model.estimate_mfu(self.batch_size * self.world_size * self.gradient_accumulation_steps, dt) running_mfu = mfu if running_mfu == -1.0 else 0.9*running_mfu + 0.1*mfu print(f"iter {self.iter_num}: loss {lossf:.4f}, time {dt*1000:.2f}ms, mfu {running_mfu*100:.2f}%") self.iter_num += 1 local_iter_num += 1 # termination conditions if self.iter_num > self.max_iters: break if self.ddp: destroy_process_group() # helps estimate an arbitrarily accurate loss over either split using many batches @torch.no_grad() def estimate_loss(self, model, ctx): out = {} model.eval() for split in ['train', 'val']: losses = torch.zeros(self.eval_iters) for k in range(self.eval_iters): X, Y = self.get_batch(split) with ctx: logits, loss = model(X, Y) losses[k] = loss.item() out[split] = losses.mean() model.train() return out ================================================ FILE: transformer_sizing.ipynb ================================================ { "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Transformer Theoretical Model\n", "\n", "This notebook stores a bunch of analysis about a Transformer, e.g. estimates the number of FLOPs, parameters, peak memory footprint, checkpoint size, etc." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "from collections import OrderedDict" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# config_args = {\n", "# 'gpt2': dict(n_layer=12, n_head=12, n_embd=768), # 124M params\n", "# 'gpt2-medium': dict(n_layer=24, n_head=16, n_embd=1024), # 350M params\n", "# 'gpt2-large': dict(n_layer=36, n_head=20, n_embd=1280), # 774M params\n", "# 'gpt2-xl': dict(n_layer=48, n_head=25, n_embd=1600), # 1558M params\n", "# }[model_type]\n", "\n", "block_size = 1024\n", "vocab_size = 50257\n", "n_layer = 12\n", "n_head = 12\n", "n_embd = 768\n", "bias = False\n", "assert not bias, \"this notebook assumes bias=False just for simplicity\"" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "we see: 124337664, expected: 124337664, match: True\n", "name params ratio (%) \n", "emebedding/position 786432 0.6325\n", "embedding/token 38597376 31.0424\n", "embedding 39383808 31.6749\n", "attention/ln 768 0.0006\n", "attention/kqv 1769472 1.4231\n", "attention/proj 589824 0.4744\n", "attention 2360064 1.8981\n", "mlp/ln 768 0.0006\n", "mlp/ffw 2359296 1.8975\n", "mlp/proj 2359296 1.8975\n", "mlp 4719360 3.7956\n", "block 7079424 5.6937\n", "transformer 84953088 68.3245\n", "ln_f 768 0.0006\n", "dense 0 0.0000\n", "total 124337664 100.0000\n" ] } ], "source": [ "def params():\n", " \"\"\" estimates the number of parameters in the model\"\"\"\n", " out = OrderedDict()\n", "\n", " # token and position embeddings\n", " out['emebedding/position'] = n_embd * block_size\n", " out['embedding/token'] = n_embd * vocab_size\n", " out['embedding'] = out['emebedding/position'] + out['embedding/token']\n", "\n", " # attention blocks\n", " out['attention/ln'] = n_embd # note, bias=False in our LN\n", " out['attention/kqv'] = n_embd * 3*n_embd\n", " out['attention/proj'] = n_embd**2\n", " out['attention'] = out['attention/ln'] + out['attention/kqv'] + out['attention/proj']\n", "\n", " # MLP blocks\n", " ffw_size = 4*n_embd # feed forward size\n", " out['mlp/ln'] = n_embd\n", " out['mlp/ffw'] = n_embd * ffw_size\n", " out['mlp/proj'] = ffw_size * n_embd\n", " out['mlp'] = out['mlp/ln'] + out['mlp/ffw'] + out['mlp/proj']\n", " \n", " # the transformer and the rest of it\n", " out['block'] = out['attention'] + out['mlp']\n", " out['transformer'] = n_layer * out['block']\n", " out['ln_f'] = n_embd # final layernorm\n", " out['dense'] = 0 # 0 because of parameter sharing. This layer uses the weights from the embedding layer\n", "\n", " # total\n", " out['total'] = out['embedding'] + out['transformer'] + out['ln_f'] + out['dense']\n", "\n", " return out\n", "\n", "# compare our param count to that reported by PyTorch\n", "p = params()\n", "params_total = p['total']\n", "print(f\"we see: {params_total}, expected: {124337664}, match: {params_total == 124337664}\")\n", "# create a header\n", "print(f\"{'name':20s} {'params':10s} {'ratio (%)':10s}\")\n", "for k,v in p.items():\n", " print(f\"{k:20s} {v:10d} {v/params_total*100:10.4f}\")\n", " " ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "est checkpoint size: 1.49 GB\n", "measured with wc -c ckpt.pt: 1542470366\n", "fluff ratio: 103.38%\n" ] } ], "source": [ "# we can now calculate the size of each checkpoint\n", "# params are stored in fp32, and the AdamW optimizer has 2 additional buffers per param for statistics\n", "params_bytes = params_total*4\n", "params_and_buffers_bytes = params_bytes + 2*params_bytes\n", "print(f\"est checkpoint size: {params_and_buffers_bytes/1e9:.2f} GB\")\n", "measured_bytes = 1542470366 # from wc -c ckpt.pt\n", "print(f\"measured with wc -c ckpt.pt: {measured_bytes}\")\n", "print(f\"fluff ratio: {measured_bytes/params_and_buffers_bytes*100:.2f}%\")" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "We can also estimate the ratio of our GPU memory that will be taken up just by the weights and the buffers inside the AdamW optimizer" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "memory ratio taken up just for parameters: 3.73%\n" ] } ], "source": [ "gpu_memory = 40e9 # 40 GB A100 GPU, roughly\n", "print(f\"memory ratio taken up just for parameters: {params_and_buffers_bytes / gpu_memory * 100:.2f}%\")" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "i.e. not that much of the memory for this tiny model, most of the memory is activations (forward and backward). This of course changes dramatically for larger and larger models." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Let's estimate FLOPs for a single forward pass." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "name flops ratio (%) \n", "attention/kqv 3623878656 1.2426\n", "attention/scores 1610612736 0.5522\n", "attention/reduce 1610612736 0.5522\n", "attention/proj 1207959552 0.4142\n", "attention 8053063680 2.7612\n", "mlp/ffw1 4831838208 1.6567\n", "mlp/ffw2 4831838208 1.6567\n", "mlp 9663676416 3.3135\n", "block 17716740096 6.0747\n", "transformer 212600881152 72.8963\n", "dense 79047426048 27.1037\n", "forward_total 291648307200 100.0000\n", "backward_total 583296614400 200.0000\n", "total 874944921600 300.0000\n" ] } ], "source": [ "def flops():\n", " # we only count Weight FLOPs, all other layers (LayerNorm, Softmax, etc) are effectively irrelevant\n", " # we count actual FLOPs, not MACs. Hence 2* all over the place\n", " # basically for any matrix multiply A (BxC) @ B (CxD) -> (BxD) flops are 2*B*C*D\n", "\n", " out = OrderedDict()\n", " head_size = n_embd // n_head\n", "\n", " # attention blocks\n", " # 1) the projection to key, query, values\n", " out['attention/kqv'] = 2 * block_size * (n_embd * 3*n_embd)\n", " # 2) calculating the attention scores\n", " out['attention/scores'] = 2 * block_size * block_size * n_embd\n", " # 3) the reduction of the values (B, nh, T, T) x (B, nh, T, hs) -> (B, nh, T, hs)\n", " out['attention/reduce'] = 2 * n_head * (block_size * block_size * head_size)\n", " # 4) the final linear projection\n", " out['attention/proj'] = 2 * block_size * (n_embd * n_embd)\n", " out['attention'] = sum(out['attention/'+k] for k in ['kqv', 'scores', 'reduce', 'proj'])\n", "\n", " # MLP blocks\n", " ffw_size = 4*n_embd # feed forward size\n", " out['mlp/ffw1'] = 2 * block_size * (n_embd * ffw_size)\n", " out['mlp/ffw2'] = 2 * block_size * (ffw_size * n_embd)\n", " out['mlp'] = out['mlp/ffw1'] + out['mlp/ffw2']\n", "\n", " # the transformer and the rest of it\n", " out['block'] = out['attention'] + out['mlp']\n", " out['transformer'] = n_layer * out['block']\n", " out['dense'] = 2 * block_size * (n_embd * vocab_size)\n", "\n", " # forward,backward,total\n", " out['forward_total'] = out['transformer'] + out['dense']\n", " out['backward_total'] = 2 * out['forward_total'] # use common estimate of bwd = 2*fwd\n", " out['total'] = out['forward_total'] + out['backward_total']\n", "\n", " return out\n", " \n", "# compare our param count to that reported by PyTorch\n", "f = flops()\n", "flops_total = f['forward_total']\n", "print(f\"{'name':20s} {'flops':14s} {'ratio (%)':10s}\")\n", "for k,v in f.items():\n", " print(f\"{k:20s} {v:14d} {v/flops_total*100:10.4f}\")\n", " " ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "palm_flops: 875062886400, flops: 874944921600, ratio: 1.0001\n" ] } ], "source": [ "# now here is an estimate copy pasted from the PaLM paper\n", "# this formula is often used to calculate MFU (model flops utilization)\n", "def palm_flops():\n", " \"\"\"estimate of the model flops following PaLM paper formula\"\"\"\n", " # non-embedding model parameters. note that we do not subtract the\n", " # embedding/token params because those are tied and get used in the last layer.\n", " N = params()['total'] - params()['emebedding/position']\n", " L, H, Q, T = n_layer, n_head, n_embd//n_head, block_size\n", " mf_per_token = 6*N + 12*L*H*Q*T\n", " mf = mf_per_token * block_size\n", " return mf\n", "\n", "print(f\"palm_flops: {palm_flops():d}, flops: {flops()['total']:d}, ratio: {palm_flops()/flops()['total']:.4f}\")" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Ok they are quite similar, giving some confidence that my math in flops() function was ~ok. Now, A100 is cited at 312TFLOPS bfloat16 on tensor cores. So what is our model flops utilization (MFU)? I trained the model above with a batch_size of 20 and grad_accum of 5, which runs in about 755ms on a single A100 GPU. We get:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "fraction of A100 used: 37.14%\n" ] } ], "source": [ "# here is what we currently roughly measure\n", "batch_size = 20 * 5 # 5 is grad_accum, so total batch size is 100\n", "measured_time = 0.755 # in seconds per iteration\n", "measured_throughput = batch_size / measured_time\n", "flops_achieved = f['total'] * measured_throughput\n", "\n", "# A100 is cited to be 312 TFLOPS of bloat16 running on tensor cores\n", "a100_flops_promised = 312e12\n", "\n", "# the fraction of the A100 that we are using:\n", "print(f\"fraction of A100 used: {flops_achieved / a100_flops_promised * 100:.2f}%\")" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "For reference, we'd prefer to be somewhere around 50%+, and not just for a single GPU but for an entire DDP run. So we still have some work to do, but at least we're within a factor of ~2X of what is achievable with this GPU." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "time needed to train the model: 3.46 days\n" ] } ], "source": [ "# Finally let's check out the 6ND approximation as total cost of training in FLOPs\n", "model_size = params()['total'] # this is number of parameters, N\n", "tokens_num = 300e9 # 300B tokens, this is dataset size in tokens, D\n", "a100_flops = 312e12 # 312 TFLOPS\n", "assumed_mfu = 0.3 # assume this model flops utilization (take the current 37% from above and add some DDP overhead)\n", "flops_throughput = a100_flops * 8 * assumed_mfu # assume an 8XA100 node at 30% utilization\n", "flops_needed = 6 * model_size * tokens_num # 6ND\n", "time_needed_s = flops_needed / flops_throughput # in seconds\n", "print(f\"time needed to train the model: {time_needed_s/3600/24:.2f} days\")" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "This is not a bad estimate at all. I trained this model and it converged in roughly 4 days. Btw as a good reference for where 6ND comes from and some intuition around it I recommend [Dzmitry's post](https://medium.com/@dzmitrybahdanau/the-flops-calculus-of-language-model-training-3b19c1f025e4)." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Now, FLOPs are just one constraint, the other that we have to keep a close track of is the memory bandwidth. TODO estimate LOAD/STORE costs of our model later." ] } ], "metadata": { "kernelspec": { "display_name": "pytorch2", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.8" }, "orig_nbformat": 4, "vscode": { "interpreter": { "hash": "7f5833218766b48e6e35e4452ee875aac0e2188d05bbe5298f2c62b79f08b222" } } }, "nbformat": 4, "nbformat_minor": 2 } ================================================ FILE: utils.py ================================================ import torch class dotdict(dict): """dot.notation access to dictionary attributes""" __getattr__ = dict.get __setattr__ = dict.__setitem__ __delattr__ = dict.__delitem__