[
  {
    "path": ".gitattributes",
    "content": "# Override jupyter in Github language stats for more accurate estimate of repo code languages\n# reference: https://github.com/github/linguist/blob/master/docs/overrides.md#generated-code\n*.ipynb linguist-generated\n"
  },
  {
    "path": "LICENSE",
    "content": "MIT License\n\nCopyright (c) 2022 Andrej Karpathy\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n"
  },
  {
    "path": "README.md",
    "content": "# nanoChatGPT\n\nA crude RLHF (Reinforcement Learing from Human Feedback) layer on top of nanoGPT to test an idea I had that you can backpropagate through the reward function rather than use policy gradient. I have verified it works for a very basic example where you incentivise the network to produce words containing 'and'. The trick is to use the Straight-Through Gumbel-Softmax estimator.\n\nAlso checkout chatgpt_dev_teaching.ipynb and the YouTube video explaining fine-tuning with RL: https://m.youtube.com/watch?v=soqTT0o1ZKo \n\nPrepare data:\n\n```\n$ python data/shakespeare/prepare.py\n```\n\nOnce data is prepared start training. The configs assume cuda, if you don't have a gpu change to cpu in config.\n\n```\n$ python train.py # settings in config/config.yaml\n```\n\nOnce a basic model is trained, can fine tune a reward model for an underlying reward rule. \n\n```\n$ python train_reward_model_simple.py # settings in config/config_reward.yaml\n```\n\nThis creates a multihead model on top of the existing one. Once the reward model is trained sufficiently you can train the RL policy using:\n\n```\n$ python train_rl.py # settings in config/config_rl.yaml\n```\n\nThe default config uses the Gumbel trick but it can be set to PG and it will do policy gradient instead (the latter still needs a critic implementation etc). I have validated that the Gumbel method works given that the preceding steps also worked. I am curious to see if this would scale to large models - let me know if you're able to test this. \n\nModel output after a short amount of training produces results like:\n\n```\nhand hand thousand the thousand the hand hand hand hand thousand\n```\n\nIf you're feeling adventorous you can also try:\n\n```\n$ python train_reward_model.py\n```\n\nWhich uses the reddit tldr dataset. The pipes should work but I have not actually finetuned this at all.\n\nReferences:\n\nGumbel\nhttps://arxiv.org/pdf/1611.01144.pdf\n\nInstructGPT\nhttps://arxiv.org/abs/2203.02155\n\nBelow is Andrej Karpathy's original README, make sure you have installed the revelevant packages\n________\n\n# nanoGPT\n\n![nanoGPT](assets/nanogpt.jpg)\n\nThe simplest, fastest repository for training/finetuning medium-sized GPTs. It is a rewrite of [minGPT](https://github.com/karpathy/minGPT) that prioritizes teeth over education. Still under active development, but currently the file `train.py` reproduces GPT-2 (124M) on OpenWebText, running on a single 8XA100 40GB node in about 4 days of training. The code itself is plain and readable: `train.py` is a ~300-line boilerplate training loop and `model.py` a ~300-line GPT model definition, which can optionally load the GPT-2 weights from OpenAI. That's it.\n\n![repro124m](assets/gpt2_124M_loss.png)\n\nBecause the code is so simple, it is very easy to hack to your needs, train new models from scratch, or finetune pretrained checkpoints (e.g. biggest one currently available as a starting point would be the GPT-2 1.3B model from OpenAI).\n\n## install\n\nDependencies:\n\n- [pytorch](https://pytorch.org) <3\n- [numpy](https://numpy.org/install/) <3\n- `pip install transformers` for huggingface transformers <3 (to load GPT-2 checkpoints)\n- `pip install datasets` for huggingface datasets <3 (if you want to download + preprocess OpenWebText)\n- `pip install tiktoken` for OpenAI's fast BPE code <3\n- `pip install wandb` for optional logging <3\n- `pip install tqdm`\n\n## quick start\n\nIf you are not a deep learning professional and you just want to feel the magic and get your feet wet, the fastest way to get started is to train a character-level GPT on the works of Shakespeare. First, we download it as a single (1MB) file and turn it from raw text into one large stream of integers:\n\n```\n$ python data/shakespeare_char/prepare.py\n```\n\nThis creates a `train.bin` and `val.bin` in that data directory. Now it is time to train your GPT. The size of it very much depends on the computational resources of your system:\n\n**I have a GPU**. Great, we can quickly train a baby GPT with the settings provided in the [config/train_shakespeare_char.py](config/train_shakespeare_char.py) config file:\n\n```\n$ python train.py config/train_shakespeare_char.py\n```\n\nIf you peak inside it, you'll see that we're training a GPT with a context size of up to 256 characters, 384 feature channels, and it is a 6-layer Transformer with 6 heads in each layer. On one A100 GPU this training run takes about 3 minutes and the best validation loss is 1.4697. Based on the configuration, the model checkpoints are being written into the `--out_dir` directory `out-shakespeare-char`. So once the training finishes we can sample from the best model by pointing the sampling script at this directory:\n\n```\n$ python sample.py --out_dir=out-shakespeare-char\n```\n\nThis generates a few samples, for example:\n\n```\nANGELO:\nAnd cowards it be strawn to my bed,\nAnd thrust the gates of my threats,\nBecause he that ale away, and hang'd\nAn one with him.\n\nDUKE VINCENTIO:\nI thank your eyes against it.\n\nDUKE VINCENTIO:\nThen will answer him to save the malm:\nAnd what have you tyrannous shall do this?\n\nDUKE VINCENTIO:\nIf you have done evils of all disposition\nTo end his power, the day of thrust for a common men\nThat I leave, to fight with over-liking\nHasting in a roseman.\n```\n\nlol  `¯\\_(ツ)_/¯`. Not bad for a character-level model after 3 minutes of training on a GPU. Better results are quite likely obtainable by instead finetuning a pretrained GPT-2 model on this dataset (see finetuning section later).\n\n**I only have a macbook** (or other cheap computer). No worries, we can still train a GPT but we want to dial things down a notch. I recommend getting the bleeding edge PyTorch nightly ([select it here](https://pytorch.org/get-started/locally/) when installing) as it is currently quite likely to make your code more efficient. But even without it, a simple train run could look as follows:\n\n```\n$ python train.py config/train_shakespeare_char.py --device=cpu --compile=False --eval_iters=20 --log_interval=1 --block_size=64 --batch_size=12 --n_layer=4 --n_head=4 --n_embd=128 --max_iters=2000 --lr_decay_iters=2000 --dropout=0.0\n```\n\nHere, since we are running on CPU instead of GPU we must set both `--device=cpu` and also turn off PyTorch 2.0 compile with `--compile=False`. Then when we evaluate we get a bit more noisy but faster estimate (`--eval_iters=20`, down from 200), our context size is only 64 characters instead of 256, and the batch size only 12 examples per iteration, not 64. We'll also use a much smaller Transformer (4 layers, 4 heads, 128 embedding size), and decrease the number of iterations to 2000 (and correspondingly usually decay the learning rate to around max_iters with `--lr_decay_iters`). Because our network is so small we also ease down on regularization (`--dropout=0.0`). This still runs in about ~3 minutes, but gets us a loss of only 1.88 and therefore also worse samples, but it's still good fun:\n\n```\nGLEORKEN VINGHARD III:\nWhell's the couse, the came light gacks,\nAnd the for mought you in Aut fries the not high shee\nbot thou the sought bechive in that to doth groan you,\nNo relving thee post mose the wear\n```\n\nNot bad for ~3 minutes on a CPU, for a hint of the right character gestalt. If you're willing to wait longer free to tune the hyperparameters, increase the size of the network, the context length (`--block_size`), the length of training, etc.\n\nFinally, on Apple Silicon Macbooks and with a recent PyTorch version make sure to add `--device mps` (short for \"Metal Performance Shaders\"); PyTorch then uses the on-chip GPU that can *significantly* accelerate training (2-3X) and allow you to use larger networks. See [Issue 28](https://github.com/karpathy/nanoGPT/issues/28) for more.\n\n## reproducing GPT-2\n\nA more serious deep learning professional may be more interested in reproducing GPT-2 results. So here we go - we first tokenize the dataset, in this case the [OpenWebText](https://openwebtext2.readthedocs.io/en/latest/), an open reproduction of OpenAI's (private) WebText:\n\n```\n$ python data/openwebtext/prepare.py\n```\n\nThis downloads and tokenizes the [OpenWebText](https://huggingface.co/datasets/openwebtext) dataset. It will create a `train.bin` and `val.bin` which holds the GPT2 BPE token ids in one sequence, stored as raw uint16 bytes. Then we're ready to kick off training. To reproduce GPT-2 (124M) you'll want at least an 8X A100 40GB node and run:\n\n```\n$ torchrun --standalone --nproc_per_node=8 train.py config/train_gpt2.py\n```\n\nThis will run for about 4 days using PyTorch Distributed Data Parallel (DDP) and go down to loss of ~2.85. Now, a GPT-2 model just evaluated on OWT gets a val loss of about 3.11, but if you finetune it it will come down to ~2.85 territory (due to an apparent domain gap), making the two models ~match.\n\nIf you're in a cluster environment and you are blessed with multiple GPU nodes you can make GPU go brrrr e.g. across 2 nodes like:\n\n```\nRun on the first (master) node with example IP 123.456.123.456:\n$ torchrun --nproc_per_node=8 --nnodes=2 --node_rank=0 --master_addr=123.456.123.456 --master_port=1234 train.py\nRun on the worker node:\n$ torchrun --nproc_per_node=8 --nnodes=2 --node_rank=1 --master_addr=123.456.123.456 --master_port=1234 train.py\n```\n\nIt is a good idea to benchmark your interconnect (e.g. iperf3). In particular, if you don't have Infiniband then also prepend `NCCL_IB_DISABLE=1` to the above launches. Your multinode training will work, but most likely _crawl_. By default checkpoints are periodically written to the `--out_dir`. We can sample from the model by simply `$ python sample.py`.\n\nFinally, to train on a single GPU simply run the `$ python train.py` script. Have a look at all of its args, the script tries to be very readable, hackable and transparent. You'll most likely want to tune a number of those variables depending on your needs.\n\n## baselines\n\nOpenAI GPT-2 checkpoints allow us to get some baselines in place for openwebtext. We can get the numbers as follows:\n\n```\n$ python train.py eval_gpt2\n$ python train.py eval_gpt2_medium\n$ python train.py eval_gpt2_large\n$ python train.py eval_gpt2_xl\n```\n\nand observe the following losses on train and val:\n\n| model | params | train loss | val loss |\n| ------| ------ | ---------- | -------- |\n| gpt2 | 124M         | 3.11  | 3.12     |\n| gpt2-medium | 350M  | 2.85  | 2.84     |\n| gpt2-large | 774M   | 2.66  | 2.67     |\n| gpt2-xl | 1558M     | 2.56  | 2.54     |\n\nHowever, we have to note that GPT-2 was trained on (closed, never released) WebText, while OpenWebText is just a best-effort open reproduction of this dataset. This means there is a dataset domain gap. Indeed, taking the GPT-2 (124M) checkpoint and finetuning on OWT directly for a while reaches loss down to ~2.85. This then becomes the more appropriate baseline w.r.t. reproduction.\n\n## finetuning\n\nFinetuning is no different than training, we just make sure to initialize from a pretrained model and train with a smaller learning rate. For an example of how to finetune a GPT on new text go to `data/shakespeare` and run `prepare.py` to download the tiny shakespeare dataset and render it into a `train.bin` and `val.bin`, using the OpenAI BPE tokenizer from GPT-2. Unlike OpenWebText this will run in seconds. Finetuning can take very little time, e.g. on a single GPU just a few minutes. Run an example finetuning like:\n\n```\n$ python train.py config/finetune_shakespeare.py\n```\n\nThis will load the config parameter overrides in `config/finetune_shakespeare.py` (I didn't tune them much though). Basically, we initialize from a GPT2 checkpoint with `init_from` and train as normal, except shorter and with a small learning rate. If you're running out of memory try decreasing the model size (they are `{'gpt2', 'gpt2-medium', 'gpt2-large', 'gpt2-xl'}`) or possibly decreasing the `block_size` (context length). The best checkpoint (lowest validation loss) will be in the `out_dir` directory, e.g. in `out-shakespeare` by default, per the config file. You can then run the code in `sample.py --out_dir=out-shakespeare`:\n\n```\nTHEODORE:\nThou shalt sell me to the highest bidder: if I die,\nI sell thee to the first; if I go mad,\nI sell thee to the second; if I\nlie, I sell thee to the third; if I slay,\nI sell thee to the fourth: so buy or sell,\nI tell thee again, thou shalt not sell my\npossession.\n\nJULIET:\nAnd if thou steal, thou shalt not sell thyself.\n\nTHEODORE:\nI do not steal; I sell the stolen goods.\n\nTHEODORE:\nThou know'st not what thou sell'st; thou, a woman,\nThou art ever a victim, a thing of no worth:\nThou hast no right, no right, but to be sold.\n```\n\nWhoa there, GPT, entering some dark place over there. I didn't really tune the hyperparameters in the config too much, feel free to try!\n\n## sampling / inference\n\nUse the script `sample.py` to sample either from pre-trained GPT-2 models released by OpenAI, or from a model you trained yourself. For example, here is a way to sample from the largest available `gpt2-xl` model:\n\n```\n$ python sample.py \\\n    --init_from=gpt2-xl \\\n    --start=\"What is the answer to life, the universe, and everything?\" \\\n    --num_samples=5 --max_new_tokens=100\n```\n\nIf you'd like to sample from a model you trained, use the `--out_dir` to point the code appropriately. You can also prompt the model with some text from a file, e.g. `$ python sample.py --start=FILE:prompt.txt`.\n\n## efficiency notes\n\nFor simple model benchmarking and profiling, `bench.py` might be useful. It's identical to what happens in the meat of the training loop of `train.py`, but omits much of the other complexities.\n\nNote that the code by default uses [PyTorch 2.0](https://pytorch.org/get-started/pytorch-2.0/). At the time of writing (Dec 29, 2022) this makes `torch.compile()` available in the nightly release. The improvement from the one line of code is noticeable, e.g. cutting down iteration time from ~250ms / iter to 135ms / iter. Nice work PyTorch team!\n\n## todos\n\n- Investigate and add FSDP instead of DDP\n- Eval zero-shot perplexities on standard evals (e.g. LAMBADA? HELM? etc.)\n- Finetune the finetuning script, I think the hyperparams are not great\n- Schedule for linear batch size increase during training\n- Incorporate other embeddings (rotary, alibi)\n- Separate out the optim buffers from model params in checkpoints I think\n- Additional logging around network health (e.g. gradient clip events, magnitudes)\n- Few more investigations around better init etc.\n\n## troubleshooting\n\nNote that by default this repo uses PyTorch 2.0 (i.e. `torch.compile`). This is fairly new and experimental, and not yet available on all platforms (e.g. Windows). If you're running into related error messages try to disable this by adding `--compile=False` flag. This will slow down the code but at least it will run.\n\nFor some context on this repository, GPT, and language modeling it might be helpful to watch my [Zero To Hero series](https://karpathy.ai/zero-to-hero.html). Specifically, the [GPT video](https://www.youtube.com/watch?v=kCc8FmEb1nY) is popular if you have some prior language modeling context.\n\nFor more questions/discussions feel free to stop by **#nanoGPT** on Discord:\n\n[![](https://dcbadge.vercel.app/api/server/3zy8kqD9Cp?compact=true&style=flat)](https://discord.gg/3zy8kqD9Cp)\n\n## acknowledgements\n\nAll nanoGPT experiments are powered by GPUs on [Lambda labs](https://lambdalabs.com), my favorite Cloud GPU provider. Thank you Lambda labs for sponsoring nanoGPT!\n"
  },
  {
    "path": "bench.py",
    "content": "\"\"\"\nA much shorter version of train.py for benchmarking\n\"\"\"\nimport os\nfrom contextlib import nullcontext\nimport numpy as np\nimport time\nimport torch\nfrom model import GPTConfig, GPT\n\n# -----------------------------------------------------------------------------\nbatch_size = 12\nblock_size = 1024\nbias = False\nreal_data = True\nseed = 1337\ndevice = 'cuda' # examples: 'cpu', 'cuda', 'cuda:0', 'cuda:1', etc.\ndtype = 'bfloat16' # 'float32' or 'bfloat16' or 'float16'\ncompile = True # use PyTorch 2.0 to compile the model to be faster\nprofile = False # use pytorch profiler, or just simple benchmarking?\nexec(open('configurator.py').read()) # overrides from command line or config file\n# -----------------------------------------------------------------------------\n\ntorch.manual_seed(seed)\ntorch.cuda.manual_seed(seed)\ntorch.backends.cuda.matmul.allow_tf32 = True # allow tf32 on matmul\ntorch.backends.cudnn.allow_tf32 = True # allow tf32 on cudnn\ndevice_type = 'cuda' if 'cuda' in device else 'cpu' # for later use in torch.autocast\nptdtype = {'float32': torch.float32, 'bfloat16': torch.bfloat16, 'float16': torch.float16}[dtype]\nctx = nullcontext() if device_type == 'cpu' else torch.amp.autocast(device_type=device_type, dtype=ptdtype)\n\n# data loading init\nif real_data:\n    dataset = 'openwebtext'\n    data_dir = os.path.join('data', dataset)\n    train_data = np.memmap(os.path.join(data_dir, 'train.bin'), dtype=np.uint16, mode='r')\n    def get_batch(split):\n        data = train_data # note ignore split in benchmarking script\n        ix = torch.randint(len(data) - block_size, (batch_size,))\n        x = torch.stack([torch.from_numpy((data[i:i+block_size]).astype(np.int64)) for i in ix])\n        y = torch.stack([torch.from_numpy((data[i+1:i+1+block_size]).astype(np.int64)) for i in ix])\n        x, y = x.pin_memory().to(device, non_blocking=True), y.pin_memory().to(device, non_blocking=True)\n        return x, y\nelse:\n    # alternatively, if fixed data is desired to not care about data loading\n    x = torch.randint(50304, (batch_size, block_size), device=device)\n    y = torch.randint(50304, (batch_size, block_size), device=device)\n    get_batch = lambda split: (x, y)\n\n# model init\ngptconf = GPTConfig(\n    block_size = block_size, # how far back does the model look? i.e. context size\n    n_layer = 12, n_head = 12, n_embd = 768, # size of the model\n    dropout = 0, # for determinism\n    bias = bias,\n)\nmodel = GPT(gptconf)\nmodel.to(device)\n\noptimizer = model.configure_optimizers(weight_decay=1e-2, learning_rate=1e-4, betas=(0.9, 0.95), device_type=device_type)\n\nif compile:\n    print(\"Compiling model...\")\n    model = torch.compile(model) # pytorch 2.0\n\nif profile:\n    # useful docs on pytorch profiler:\n    # - tutorial https://pytorch.org/tutorials/intermediate/tensorboard_profiler_tutorial.html\n    # - api https://pytorch.org/docs/stable/profiler.html#torch.profiler.profile\n    wait, warmup, active = 5, 5, 5\n    num_steps = wait + warmup + active\n    with torch.profiler.profile(\n        activities=[torch.profiler.ProfilerActivity.CPU, torch.profiler.ProfilerActivity.CUDA],\n        schedule=torch.profiler.schedule(wait=wait, warmup=warmup, active=active, repeat=1),\n        on_trace_ready=torch.profiler.tensorboard_trace_handler('./bench_log'),\n        record_shapes=False,\n        profile_memory=False,\n        with_stack=False, # incurs an additional overhead, disable if not needed\n        with_flops=True,\n        with_modules=False, # only for torchscript models atm\n    ) as prof:\n\n        X, Y = get_batch('train')\n        for k in range(num_steps):\n            with ctx:\n                logits, loss = model(X, Y)\n            X, Y = get_batch('train')\n            optimizer.zero_grad(set_to_none=True)\n            loss.backward()\n            optimizer.step()\n            lossf = loss.item()\n            print(f\"{k}/{num_steps} loss: {lossf:.4f}\")\n\n            prof.step() # notify the profiler at end of each step\n\nelse:\n\n    # simple benchmarking\n    torch.cuda.synchronize()\n    for stage, num_steps in enumerate([10, 20]): # burnin, then benchmark\n        t0 = time.time()\n        X, Y = get_batch('train')\n        for k in range(num_steps):\n            with ctx:\n                logits, loss = model(X, Y)\n            X, Y = get_batch('train')\n            optimizer.zero_grad(set_to_none=True)\n            loss.backward()\n            optimizer.step()\n            lossf = loss.item()\n            print(f\"{k}/{num_steps} loss: {lossf:.4f}\")\n        torch.cuda.synchronize()\n        t1 = time.time()\n        dt = t1-t0\n        mfu = model.estimate_mfu(batch_size * 1 * num_steps, dt)\n        if stage == 1:\n            print(f\"time per iteration: {dt/num_steps*1000:.4f}ms, MFU: {mfu*100:.2f}%\")\n"
  },
  {
    "path": "chatgpt_dev_teaching.ipynb",
    "content": "{\n  \"nbformat\": 4,\n  \"nbformat_minor\": 0,\n  \"metadata\": {\n    \"colab\": {\n      \"provenance\": [],\n      \"authorship_tag\": \"ABX9TyNVW45SCGk3DW7yIfDYKg2T\",\n      \"include_colab_link\": true\n    },\n    \"kernelspec\": {\n      \"name\": \"python3\",\n      \"display_name\": \"Python 3\"\n    },\n    \"language_info\": {\n      \"name\": \"python\"\n    },\n    \"accelerator\": \"GPU\",\n    \"gpuClass\": \"premium\",\n    \"widgets\": {\n      \"application/vnd.jupyter.widget-state+json\": {\n        \"49506f92d9d042d4a6111f3a1b9f79e1\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HBoxModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HBoxModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HBoxView\",\n            \"box_style\": \"\",\n            \"children\": [\n              \"IPY_MODEL_95f4423f202b44f9b5f6fbc250b5a300\",\n              \"IPY_MODEL_e78ecc50321f42328d47999c01244651\",\n              \"IPY_MODEL_cc422e07c3ce4ba99c84bda8b7d7c596\"\n            ],\n            \"layout\": \"IPY_MODEL_8be555a8bb6e4068a42e4312e75a30c2\"\n          }\n        },\n        \"95f4423f202b44f9b5f6fbc250b5a300\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HTMLModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HTMLModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HTMLView\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_b7f7646e496849b8bb3239b64b93e193\",\n            \"placeholder\": \"​\",\n            \"style\": \"IPY_MODEL_b728662de8ac41888e838e47e67d6aa9\",\n            \"value\": \"Downloading (…)lve/main/config.json: 100%\"\n          }\n        },\n        \"e78ecc50321f42328d47999c01244651\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"FloatProgressModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"FloatProgressModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"ProgressView\",\n            \"bar_style\": \"success\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_70a9186eb9e64d9ea5b456f43622d979\",\n            \"max\": 949,\n            \"min\": 0,\n            \"orientation\": \"horizontal\",\n            \"style\": \"IPY_MODEL_7a5dedee422f42b49ecac65d3138c167\",\n            \"value\": 949\n          }\n        },\n        \"cc422e07c3ce4ba99c84bda8b7d7c596\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HTMLModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HTMLModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HTMLView\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_c2e28b80efc74a549af7bdf3eee46eda\",\n            \"placeholder\": \"​\",\n            \"style\": \"IPY_MODEL_057de1e97cad4a2297ad3db02c2481d6\",\n            \"value\": \" 949/949 [00:00&lt;00:00, 45.7kB/s]\"\n          }\n        },\n        \"8be555a8bb6e4068a42e4312e75a30c2\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"b7f7646e496849b8bb3239b64b93e193\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"b728662de8ac41888e838e47e67d6aa9\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"DescriptionStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"DescriptionStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"description_width\": \"\"\n          }\n        },\n        \"70a9186eb9e64d9ea5b456f43622d979\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"7a5dedee422f42b49ecac65d3138c167\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"ProgressStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"ProgressStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"bar_color\": null,\n            \"description_width\": \"\"\n          }\n        },\n        \"c2e28b80efc74a549af7bdf3eee46eda\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"057de1e97cad4a2297ad3db02c2481d6\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"DescriptionStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"DescriptionStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"description_width\": \"\"\n          }\n        },\n        \"daabe15b3af34e1098a7e2f620ae1f36\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HBoxModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HBoxModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HBoxView\",\n            \"box_style\": \"\",\n            \"children\": [\n              \"IPY_MODEL_c8fd1744cb744200b0bcc8629d173bf1\",\n              \"IPY_MODEL_7a52e4742e594e25a28feb18c417c076\",\n              \"IPY_MODEL_513d02e2c28440d9897d1aa097d1e743\"\n            ],\n            \"layout\": \"IPY_MODEL_d147cf4c15ff449598d6b429c90db0ae\"\n          }\n        },\n        \"c8fd1744cb744200b0bcc8629d173bf1\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HTMLModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HTMLModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HTMLView\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_c74d25684a4d4eafbd3546dcff921cb0\",\n            \"placeholder\": \"​\",\n            \"style\": \"IPY_MODEL_aa57d59560114d55bede331307257f6b\",\n            \"value\": \"Downloading pytorch_model.bin: 100%\"\n          }\n        },\n        \"7a52e4742e594e25a28feb18c417c076\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"FloatProgressModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"FloatProgressModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"ProgressView\",\n            \"bar_style\": \"success\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_e5c0bc2de75347c39dc9ca23a8129650\",\n            \"max\": 539679413,\n            \"min\": 0,\n            \"orientation\": \"horizontal\",\n            \"style\": \"IPY_MODEL_a18d35c8ff2649e397ed983a6f06e959\",\n            \"value\": 539679413\n          }\n        },\n        \"513d02e2c28440d9897d1aa097d1e743\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HTMLModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HTMLModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HTMLView\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_5e3d4cc0ec9d4c0ca0d0136e97aca2ce\",\n            \"placeholder\": \"​\",\n            \"style\": \"IPY_MODEL_3d4f1a5c8128451884bd21fef234b89b\",\n            \"value\": \" 540M/540M [00:08&lt;00:00, 67.9MB/s]\"\n          }\n        },\n        \"d147cf4c15ff449598d6b429c90db0ae\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"c74d25684a4d4eafbd3546dcff921cb0\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"aa57d59560114d55bede331307257f6b\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"DescriptionStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"DescriptionStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"description_width\": \"\"\n          }\n        },\n        \"e5c0bc2de75347c39dc9ca23a8129650\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"a18d35c8ff2649e397ed983a6f06e959\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"ProgressStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"ProgressStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"bar_color\": null,\n            \"description_width\": \"\"\n          }\n        },\n        \"5e3d4cc0ec9d4c0ca0d0136e97aca2ce\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"3d4f1a5c8128451884bd21fef234b89b\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"DescriptionStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"DescriptionStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"description_width\": \"\"\n          }\n        },\n        \"9645d7c59b0546159890f042ca420632\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HBoxModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HBoxModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HBoxView\",\n            \"box_style\": \"\",\n            \"children\": [\n              \"IPY_MODEL_8893c2570f464d9697dda12cc42023d5\",\n              \"IPY_MODEL_0c71d29e93f24885802ad5fa6d240675\",\n              \"IPY_MODEL_7b859eccf54e41698cf179d68bb109d0\"\n            ],\n            \"layout\": \"IPY_MODEL_725944a581244ab6a2a4b61afa29c1c0\"\n          }\n        },\n        \"8893c2570f464d9697dda12cc42023d5\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HTMLModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HTMLModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HTMLView\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_e536fea041f7441aa8cc93a448492e33\",\n            \"placeholder\": \"​\",\n            \"style\": \"IPY_MODEL_8e5f96a8ec714284a7f000af96300f67\",\n            \"value\": \"Downloading (…)okenizer_config.json: 100%\"\n          }\n        },\n        \"0c71d29e93f24885802ad5fa6d240675\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"FloatProgressModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"FloatProgressModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"ProgressView\",\n            \"bar_style\": \"success\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_04984bbb402c4bd4afad349e629e2452\",\n            \"max\": 338,\n            \"min\": 0,\n            \"orientation\": \"horizontal\",\n            \"style\": \"IPY_MODEL_4a07648ab35047c8b61e32e3a61d970b\",\n            \"value\": 338\n          }\n        },\n        \"7b859eccf54e41698cf179d68bb109d0\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HTMLModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HTMLModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HTMLView\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_bd5178cf064049678d4efac17e47a5da\",\n            \"placeholder\": \"​\",\n            \"style\": \"IPY_MODEL_ae235b9add2d41ee9f808a2b4d43568b\",\n            \"value\": \" 338/338 [00:00&lt;00:00, 25.9kB/s]\"\n          }\n        },\n        \"725944a581244ab6a2a4b61afa29c1c0\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"e536fea041f7441aa8cc93a448492e33\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"8e5f96a8ec714284a7f000af96300f67\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"DescriptionStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"DescriptionStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"description_width\": \"\"\n          }\n        },\n        \"04984bbb402c4bd4afad349e629e2452\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"4a07648ab35047c8b61e32e3a61d970b\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"ProgressStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"ProgressStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"bar_color\": null,\n            \"description_width\": \"\"\n          }\n        },\n        \"bd5178cf064049678d4efac17e47a5da\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"ae235b9add2d41ee9f808a2b4d43568b\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"DescriptionStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"DescriptionStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"description_width\": \"\"\n          }\n        },\n        \"8fb0a31c3ecf4a4880803e81e1bc581d\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HBoxModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HBoxModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HBoxView\",\n            \"box_style\": \"\",\n            \"children\": [\n              \"IPY_MODEL_8bc84935b3964e338d00ea1bb7c3c59f\",\n              \"IPY_MODEL_d729ac6a87174fbda03175c21859ea82\",\n              \"IPY_MODEL_b80bd9149ae34878bfeeb4fca815231c\"\n            ],\n            \"layout\": \"IPY_MODEL_40007a4900824bd88d3a180fd0a8325a\"\n          }\n        },\n        \"8bc84935b3964e338d00ea1bb7c3c59f\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HTMLModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HTMLModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HTMLView\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_ce6627b477bc430d968fd186443b34e3\",\n            \"placeholder\": \"​\",\n            \"style\": \"IPY_MODEL_2f2cf4fd7a434da5be62bd088360a6ba\",\n            \"value\": \"Downloading (…)solve/main/vocab.txt: 100%\"\n          }\n        },\n        \"d729ac6a87174fbda03175c21859ea82\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"FloatProgressModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"FloatProgressModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"ProgressView\",\n            \"bar_style\": \"success\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_31aa09c7a2ea443b8a177c4f91a7d427\",\n            \"max\": 843438,\n            \"min\": 0,\n            \"orientation\": \"horizontal\",\n            \"style\": \"IPY_MODEL_f17f1682313e49df97e399508c15716a\",\n            \"value\": 843438\n          }\n        },\n        \"b80bd9149ae34878bfeeb4fca815231c\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HTMLModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HTMLModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HTMLView\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_dde9d315de174921844ebb72865d1d3f\",\n            \"placeholder\": \"​\",\n            \"style\": \"IPY_MODEL_386eb97816cd41e188dbfa1fcd299b16\",\n            \"value\": \" 843k/843k [00:01&lt;00:00, 753kB/s]\"\n          }\n        },\n        \"40007a4900824bd88d3a180fd0a8325a\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"ce6627b477bc430d968fd186443b34e3\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"2f2cf4fd7a434da5be62bd088360a6ba\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"DescriptionStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"DescriptionStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"description_width\": \"\"\n          }\n        },\n        \"31aa09c7a2ea443b8a177c4f91a7d427\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"f17f1682313e49df97e399508c15716a\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"ProgressStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"ProgressStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"bar_color\": null,\n            \"description_width\": \"\"\n          }\n        },\n        \"dde9d315de174921844ebb72865d1d3f\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"386eb97816cd41e188dbfa1fcd299b16\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"DescriptionStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"DescriptionStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"description_width\": \"\"\n          }\n        },\n        \"d5af658c67874c67b08bd8ef72411ea1\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HBoxModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HBoxModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HBoxView\",\n            \"box_style\": \"\",\n            \"children\": [\n              \"IPY_MODEL_aba6fb6586534c1f86ce65b7c8bde51f\",\n              \"IPY_MODEL_a6878fe333e243dc81ec0d36bb0dce19\",\n              \"IPY_MODEL_d1efe95c3d55440386db28242a8f38d1\"\n            ],\n            \"layout\": \"IPY_MODEL_ecfb1bf7ae8a4328ac982234127e5c2b\"\n          }\n        },\n        \"aba6fb6586534c1f86ce65b7c8bde51f\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HTMLModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HTMLModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HTMLView\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_4847c36ed46d49429e37f4b9b7862a57\",\n            \"placeholder\": \"​\",\n            \"style\": \"IPY_MODEL_631e5ee44ccc4e4aaec456af90df7965\",\n            \"value\": \"Downloading (…)solve/main/bpe.codes: 100%\"\n          }\n        },\n        \"a6878fe333e243dc81ec0d36bb0dce19\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"FloatProgressModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"FloatProgressModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"ProgressView\",\n            \"bar_style\": \"success\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_662fd517bf0540ed9c607d3ac8dda0af\",\n            \"max\": 1078931,\n            \"min\": 0,\n            \"orientation\": \"horizontal\",\n            \"style\": \"IPY_MODEL_04d90c9c251a4fae8db96d711568cd64\",\n            \"value\": 1078931\n          }\n        },\n        \"d1efe95c3d55440386db28242a8f38d1\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HTMLModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HTMLModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HTMLView\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_7bd831ca2d354ee7a75c22c700963ea8\",\n            \"placeholder\": \"​\",\n            \"style\": \"IPY_MODEL_3de7e32fefd54df6a6d3596205ba0338\",\n            \"value\": \" 1.08M/1.08M [00:01&lt;00:00, 961kB/s]\"\n          }\n        },\n        \"ecfb1bf7ae8a4328ac982234127e5c2b\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"4847c36ed46d49429e37f4b9b7862a57\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"631e5ee44ccc4e4aaec456af90df7965\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"DescriptionStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"DescriptionStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"description_width\": \"\"\n          }\n        },\n        \"662fd517bf0540ed9c607d3ac8dda0af\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"04d90c9c251a4fae8db96d711568cd64\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"ProgressStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"ProgressStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"bar_color\": null,\n            \"description_width\": \"\"\n          }\n        },\n        \"7bd831ca2d354ee7a75c22c700963ea8\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"3de7e32fefd54df6a6d3596205ba0338\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"DescriptionStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"DescriptionStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"description_width\": \"\"\n          }\n        },\n        \"ddbbcddaba34416881ebb023864b0f2d\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HBoxModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HBoxModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HBoxView\",\n            \"box_style\": \"\",\n            \"children\": [\n              \"IPY_MODEL_d0fc35ec11634f8691bf5eefa0505cb6\",\n              \"IPY_MODEL_598a025b08dd4469a88bd9ebe85ebb94\",\n              \"IPY_MODEL_9bc45acff2e049d59eef1e8374f03a93\"\n            ],\n            \"layout\": \"IPY_MODEL_5a627f08513443e09dac036f3ff2fa1f\"\n          }\n        },\n        \"d0fc35ec11634f8691bf5eefa0505cb6\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HTMLModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HTMLModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HTMLView\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_397d77d019e748f2b4178bb78ff2d471\",\n            \"placeholder\": \"​\",\n            \"style\": \"IPY_MODEL_590cb10ff4164d309063faf6baedaf1f\",\n            \"value\": \"Downloading (…)in/added_tokens.json: 100%\"\n          }\n        },\n        \"598a025b08dd4469a88bd9ebe85ebb94\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"FloatProgressModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"FloatProgressModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"ProgressView\",\n            \"bar_style\": \"success\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_9d5f595908964edba6b6f7e8a6fba5eb\",\n            \"max\": 22,\n            \"min\": 0,\n            \"orientation\": \"horizontal\",\n            \"style\": \"IPY_MODEL_0e05ee7e2e934ca983d70addd1c9ce92\",\n            \"value\": 22\n          }\n        },\n        \"9bc45acff2e049d59eef1e8374f03a93\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HTMLModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HTMLModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HTMLView\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_9c56fb6a76414f558c665608a21d39cd\",\n            \"placeholder\": \"​\",\n            \"style\": \"IPY_MODEL_399aa39b85c14ff8b9cd887343af36be\",\n            \"value\": \" 22.0/22.0 [00:00&lt;00:00, 1.66kB/s]\"\n          }\n        },\n        \"5a627f08513443e09dac036f3ff2fa1f\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"397d77d019e748f2b4178bb78ff2d471\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"590cb10ff4164d309063faf6baedaf1f\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"DescriptionStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"DescriptionStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"description_width\": \"\"\n          }\n        },\n        \"9d5f595908964edba6b6f7e8a6fba5eb\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"0e05ee7e2e934ca983d70addd1c9ce92\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"ProgressStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"ProgressStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"bar_color\": null,\n            \"description_width\": \"\"\n          }\n        },\n        \"9c56fb6a76414f558c665608a21d39cd\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"399aa39b85c14ff8b9cd887343af36be\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"DescriptionStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"DescriptionStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"description_width\": \"\"\n          }\n        },\n        \"d4712217939b44f59e58a0f23a8d4df7\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HBoxModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HBoxModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HBoxView\",\n            \"box_style\": \"\",\n            \"children\": [\n              \"IPY_MODEL_c4fa3386d2234473ad7a8b6dae7a4d95\",\n              \"IPY_MODEL_97c0a2c0b079410496ab497f8fa8bd12\",\n              \"IPY_MODEL_565f60bb19d443f6aacc9b08ff68c7ff\"\n            ],\n            \"layout\": \"IPY_MODEL_5556b66847b04b4aabe0aaa029dc3e64\"\n          }\n        },\n        \"c4fa3386d2234473ad7a8b6dae7a4d95\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HTMLModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HTMLModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HTMLView\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_fe8f0c618d104deaaed3ea138acdc230\",\n            \"placeholder\": \"​\",\n            \"style\": \"IPY_MODEL_52b516dd11a747099e9e0e808a168e55\",\n            \"value\": \"Downloading (…)cial_tokens_map.json: 100%\"\n          }\n        },\n        \"97c0a2c0b079410496ab497f8fa8bd12\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"FloatProgressModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"FloatProgressModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"ProgressView\",\n            \"bar_style\": \"success\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_8696eb98489f409a848684b77b2353ac\",\n            \"max\": 167,\n            \"min\": 0,\n            \"orientation\": \"horizontal\",\n            \"style\": \"IPY_MODEL_679b6d995fad4a649e055bee8bfe83d1\",\n            \"value\": 167\n          }\n        },\n        \"565f60bb19d443f6aacc9b08ff68c7ff\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"HTMLModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_dom_classes\": [],\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"HTMLModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/controls\",\n            \"_view_module_version\": \"1.5.0\",\n            \"_view_name\": \"HTMLView\",\n            \"description\": \"\",\n            \"description_tooltip\": null,\n            \"layout\": \"IPY_MODEL_eff4e772ccd4445ba4b6f624ea3b7e8f\",\n            \"placeholder\": \"​\",\n            \"style\": \"IPY_MODEL_c7570cd91b564389b59c98a694ab7771\",\n            \"value\": \" 167/167 [00:00&lt;00:00, 13.0kB/s]\"\n          }\n        },\n        \"5556b66847b04b4aabe0aaa029dc3e64\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"fe8f0c618d104deaaed3ea138acdc230\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"52b516dd11a747099e9e0e808a168e55\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"DescriptionStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"DescriptionStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"description_width\": \"\"\n          }\n        },\n        \"8696eb98489f409a848684b77b2353ac\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"679b6d995fad4a649e055bee8bfe83d1\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"ProgressStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"ProgressStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"bar_color\": null,\n            \"description_width\": \"\"\n          }\n        },\n        \"eff4e772ccd4445ba4b6f624ea3b7e8f\": {\n          \"model_module\": \"@jupyter-widgets/base\",\n          \"model_name\": \"LayoutModel\",\n          \"model_module_version\": \"1.2.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/base\",\n            \"_model_module_version\": \"1.2.0\",\n            \"_model_name\": \"LayoutModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"LayoutView\",\n            \"align_content\": null,\n            \"align_items\": null,\n            \"align_self\": null,\n            \"border\": null,\n            \"bottom\": null,\n            \"display\": null,\n            \"flex\": null,\n            \"flex_flow\": null,\n            \"grid_area\": null,\n            \"grid_auto_columns\": null,\n            \"grid_auto_flow\": null,\n            \"grid_auto_rows\": null,\n            \"grid_column\": null,\n            \"grid_gap\": null,\n            \"grid_row\": null,\n            \"grid_template_areas\": null,\n            \"grid_template_columns\": null,\n            \"grid_template_rows\": null,\n            \"height\": null,\n            \"justify_content\": null,\n            \"justify_items\": null,\n            \"left\": null,\n            \"margin\": null,\n            \"max_height\": null,\n            \"max_width\": null,\n            \"min_height\": null,\n            \"min_width\": null,\n            \"object_fit\": null,\n            \"object_position\": null,\n            \"order\": null,\n            \"overflow\": null,\n            \"overflow_x\": null,\n            \"overflow_y\": null,\n            \"padding\": null,\n            \"right\": null,\n            \"top\": null,\n            \"visibility\": null,\n            \"width\": null\n          }\n        },\n        \"c7570cd91b564389b59c98a694ab7771\": {\n          \"model_module\": \"@jupyter-widgets/controls\",\n          \"model_name\": \"DescriptionStyleModel\",\n          \"model_module_version\": \"1.5.0\",\n          \"state\": {\n            \"_model_module\": \"@jupyter-widgets/controls\",\n            \"_model_module_version\": \"1.5.0\",\n            \"_model_name\": \"DescriptionStyleModel\",\n            \"_view_count\": null,\n            \"_view_module\": \"@jupyter-widgets/base\",\n            \"_view_module_version\": \"1.2.0\",\n            \"_view_name\": \"StyleView\",\n            \"description_width\": \"\"\n          }\n        }\n      }\n    }\n  },\n  \"cells\": [\n    {\n      \"cell_type\": \"markdown\",\n      \"metadata\": {\n        \"id\": \"view-in-github\",\n        \"colab_type\": \"text\"\n      },\n      \"source\": [\n        \"<a href=\\\"https://colab.research.google.com/github/sanjeevanahilan/nanoChatGPT/blob/master/chatgpt_dev_teaching.ipynb\\\" target=\\\"_parent\\\"><img src=\\\"https://colab.research.google.com/assets/colab-badge.svg\\\" alt=\\\"Open In Colab\\\"/></a>\"\n      ]\n    },\n    {\n      \"cell_type\": \"markdown\",\n      \"source\": [\n        \"Let's train models for generating Shakespeare with different sentiments.\\n\",\n        \"\\n\",\n        \"Happy Shakespeare will say things like: \\n\",\n        \"\\n\",\n        \"*“Nay, thanks, then, I do meet change, this Romeo. \\n\",\n        \"The pleasure of his hair!”*\\n\",\n        \"\\n\",\n        \"Sad Shakespear will say things like:\\n\",\n        \"\\n\",\n        \"\\n\",\n        \"*“The senators's dead, of their world:\\n\",\n        \"Be not for your friends.”*\\n\",\n        \"\\n\"\n      ],\n      \"metadata\": {\n        \"id\": \"9-LuRS2wAaBS\"\n      }\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"# Notebook created by Sanjeevan Ahilan\\n\",\n        \"# GPT implementation was borrowed from Andrej Karpathy \\n\",\n        \"# https://colab.research.google.com/drive/1JMLa53HDuA-i7ZBmqV7ZnA3c_fvtXnx-?usp=sharing\\n\",\n        \"# My repo: https://github.com/sanjeevanahilan/nanoChatGPT\\n\",\n        \"# My twitter: https://twitter.com/sanjeevanahilan\\n\",\n        \"\\n\",\n        \"import torch\\n\",\n        \"import torch.nn as nn\\n\",\n        \"from torch.nn import functional as F\\n\",\n        \"\\n\",\n        \"# hyperparameters\\n\",\n        \"batch_size = 16 # how many independent sequences will we process in parallel?\\n\",\n        \"block_size = 32 # what is the maximum context length for predictions?\\n\",\n        \"max_iters = 3000\\n\",\n        \"eval_interval = 100\\n\",\n        \"learning_rate = 1e-3\\n\",\n        \"device = 'cuda' if torch.cuda.is_available() else 'cpu'\\n\",\n        \"eval_iters = 200\\n\",\n        \"n_embd = 64\\n\",\n        \"n_head = 4\\n\",\n        \"n_layer = 4\\n\",\n        \"dropout = 0.0\\n\",\n        \"# ------------\\n\",\n        \"\\n\",\n        \"torch.manual_seed(1337)\\n\"\n      ],\n      \"metadata\": {\n        \"id\": \"aZGtyx5lAOzu\",\n        \"colab\": {\n          \"base_uri\": \"https://localhost:8080/\"\n        },\n        \"outputId\": \"67a0cd77-355e-4c15-f07d-ef3758a87312\"\n      },\n      \"execution_count\": 1,\n      \"outputs\": [\n        {\n          \"output_type\": \"execute_result\",\n          \"data\": {\n            \"text/plain\": [\n              \"<torch._C.Generator at 0x7fa55d231410>\"\n            ]\n          },\n          \"metadata\": {},\n          \"execution_count\": 1\n        }\n      ]\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"# Download Shakespeare\\n\",\n        \"%time\\n\",\n        \"!wget https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt\\n\",\n        \"with open('input.txt', 'r', encoding='utf-8') as f:\\n\",\n        \"    text = f.read()\"\n      ],\n      \"metadata\": {\n        \"colab\": {\n          \"base_uri\": \"https://localhost:8080/\"\n        },\n        \"id\": \"DA-Gox1YiM6l\",\n        \"outputId\": \"e04b1540-6e70-41dc-d32f-a8e234a8c87d\"\n      },\n      \"execution_count\": 2,\n      \"outputs\": [\n        {\n          \"output_type\": \"stream\",\n          \"name\": \"stdout\",\n          \"text\": [\n            \"CPU times: user 2 µs, sys: 1e+03 ns, total: 3 µs\\n\",\n            \"Wall time: 7.15 µs\\n\",\n            \"--2023-03-18 11:49:44--  https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt\\n\",\n            \"Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\\n\",\n            \"Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\\n\",\n            \"HTTP request sent, awaiting response... 200 OK\\n\",\n            \"Length: 1115394 (1.1M) [text/plain]\\n\",\n            \"Saving to: ‘input.txt’\\n\",\n            \"\\n\",\n            \"input.txt           100%[===================>]   1.06M  --.-KB/s    in 0.03s   \\n\",\n            \"\\n\",\n            \"2023-03-18 11:49:45 (40.0 MB/s) - ‘input.txt’ saved [1115394/1115394]\\n\",\n            \"\\n\"\n          ]\n        }\n      ]\n    },\n    {\n      \"cell_type\": \"markdown\",\n      \"source\": [\n        \"Rather than use character level tokenizer lets use tiktoken; Openai's implementation of a Byte Pair Encoding (BPE) tokenizer\"\n      ],\n      \"metadata\": {\n        \"id\": \"SyyJfXbdAr81\"\n      }\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"!pip install tiktoken\\n\",\n        \"import tiktoken\"\n      ],\n      \"metadata\": {\n        \"colab\": {\n          \"base_uri\": \"https://localhost:8080/\"\n        },\n        \"id\": \"y7LBbGC9onTu\",\n        \"outputId\": \"6a9ef1dd-2e22-4468-d797-d41cb0bc1de2\"\n      },\n      \"execution_count\": 3,\n      \"outputs\": [\n        {\n          \"output_type\": \"stream\",\n          \"name\": \"stdout\",\n          \"text\": [\n            \"Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\\n\",\n            \"Collecting tiktoken\\n\",\n            \"  Downloading tiktoken-0.3.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)\\n\",\n            \"\\u001b[2K     \\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\\u001b[0m \\u001b[32m1.7/1.7 MB\\u001b[0m \\u001b[31m29.8 MB/s\\u001b[0m eta \\u001b[36m0:00:00\\u001b[0m\\n\",\n            \"\\u001b[?25hRequirement already satisfied: regex>=2022.1.18 in /usr/local/lib/python3.9/dist-packages (from tiktoken) (2022.10.31)\\n\",\n            \"Requirement already satisfied: requests>=2.26.0 in /usr/local/lib/python3.9/dist-packages (from tiktoken) (2.27.1)\\n\",\n            \"Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests>=2.26.0->tiktoken) (1.26.15)\\n\",\n            \"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests>=2.26.0->tiktoken) (2022.12.7)\\n\",\n            \"Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests>=2.26.0->tiktoken) (3.4)\\n\",\n            \"Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests>=2.26.0->tiktoken) (2.0.12)\\n\",\n            \"Installing collected packages: tiktoken\\n\",\n            \"Successfully installed tiktoken-0.3.2\\n\"\n          ]\n        }\n      ]\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"enc = tiktoken.get_encoding(\\\"gpt2\\\")\\n\",\n        \"vocab_size = 50257\"\n      ],\n      \"metadata\": {\n        \"id\": \"btWJmnfapKhn\"\n      },\n      \"execution_count\": 4,\n      \"outputs\": []\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"# You can see that the sequence of characters 'the' has been encoded as a single\\n\",\n        \"# number because it is commonly reoccuring whereas 'thh' requires two numbers to\\n\",\n        \"# encode 'th' and 'h'\\n\",\n        \"\\n\",\n        \"print(enc.encode('the'))\\n\",\n        \"print(enc.encode('thh'))\\n\",\n        \"print(enc.decode([400]))\"\n      ],\n      \"metadata\": {\n        \"colab\": {\n          \"base_uri\": \"https://localhost:8080/\"\n        },\n        \"id\": \"8f9l4JdQCfSd\",\n        \"outputId\": \"42ac95f9-96cb-4fd4-e110-a339c0e45fcf\"\n      },\n      \"execution_count\": 5,\n      \"outputs\": [\n        {\n          \"output_type\": \"stream\",\n          \"name\": \"stdout\",\n          \"text\": [\n            \"[1169]\\n\",\n            \"[400, 71]\\n\",\n            \"th\\n\"\n          ]\n        }\n      ]\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"# Train and test splits\\n\",\n        \"data = torch.tensor(enc.encode(text), dtype=torch.long)\\n\",\n        \"n = int(0.9*len(data)) # first 90% will be train, rest val\\n\",\n        \"train_data = data[:n]\\n\",\n        \"val_data = data[n:]\\n\",\n        \"\\n\",\n        \"# data loading\\n\",\n        \"def get_batch(split):\\n\",\n        \"    # generate a small batch of data of inputs x and targets y\\n\",\n        \"    data = train_data if split == 'train' else val_data\\n\",\n        \"    ix = torch.randint(len(data) - block_size, (batch_size,))\\n\",\n        \"    x = torch.stack([data[i:i+block_size] for i in ix])\\n\",\n        \"    y = torch.stack([data[i+1:i+block_size+1] for i in ix])\\n\",\n        \"    x, y = x.to(device), y.to(device)\\n\",\n        \"    return x, y\\n\",\n        \"\\n\",\n        \"@torch.no_grad()\\n\",\n        \"def estimate_loss():\\n\",\n        \"    out = {}\\n\",\n        \"    model.eval()\\n\",\n        \"    for split in ['train', 'val']:\\n\",\n        \"        losses = torch.zeros(eval_iters)\\n\",\n        \"        for k in range(eval_iters):\\n\",\n        \"            X, Y = get_batch(split)\\n\",\n        \"            logits, loss = model(X, Y)\\n\",\n        \"            losses[k] = loss.item()\\n\",\n        \"        out[split] = losses.mean()\\n\",\n        \"    model.train()\\n\",\n        \"    return out\\n\",\n        \"\\n\",\n        \"class Head(nn.Module):\\n\",\n        \"    \\\"\\\"\\\" one head of self-attention \\\"\\\"\\\"\\n\",\n        \"\\n\",\n        \"    def __init__(self, head_size):\\n\",\n        \"        super().__init__()\\n\",\n        \"        self.key = nn.Linear(n_embd, head_size, bias=False)\\n\",\n        \"        self.query = nn.Linear(n_embd, head_size, bias=False)\\n\",\n        \"        self.value = nn.Linear(n_embd, head_size, bias=False)\\n\",\n        \"        self.register_buffer('tril', torch.tril(torch.ones(block_size, block_size)))\\n\",\n        \"\\n\",\n        \"        self.dropout = nn.Dropout(dropout)\\n\",\n        \"\\n\",\n        \"    def forward(self, x):\\n\",\n        \"        B,T,C = x.shape\\n\",\n        \"        k = self.key(x)   # (B,T,C)\\n\",\n        \"        q = self.query(x) # (B,T,C)\\n\",\n        \"        # compute attention scores (\\\"affinities\\\")\\n\",\n        \"        wei = q @ k.transpose(-2,-1) * C**-0.5 # (B, T, C) @ (B, C, T) -> (B, T, T)\\n\",\n        \"        wei = wei.masked_fill(self.tril[:T, :T] == 0, float('-inf')) # (B, T, T)\\n\",\n        \"        wei = F.softmax(wei, dim=-1) # (B, T, T)\\n\",\n        \"        wei = self.dropout(wei)\\n\",\n        \"        # perform the weighted aggregation of the values\\n\",\n        \"        v = self.value(x) # (B,T,C)\\n\",\n        \"        out = wei @ v # (B, T, T) @ (B, T, C) -> (B, T, C)\\n\",\n        \"        return out\\n\",\n        \"\\n\",\n        \"class MultiHeadAttention(nn.Module):\\n\",\n        \"    \\\"\\\"\\\" multiple heads of self-attention in parallel \\\"\\\"\\\"\\n\",\n        \"\\n\",\n        \"    def __init__(self, num_heads, head_size):\\n\",\n        \"        super().__init__()\\n\",\n        \"        self.heads = nn.ModuleList([Head(head_size) for _ in range(num_heads)])\\n\",\n        \"        self.proj = nn.Linear(n_embd, n_embd)\\n\",\n        \"        self.dropout = nn.Dropout(dropout)\\n\",\n        \"\\n\",\n        \"    def forward(self, x):\\n\",\n        \"        out = torch.cat([h(x) for h in self.heads], dim=-1)\\n\",\n        \"        out = self.dropout(self.proj(out))\\n\",\n        \"        return out\\n\",\n        \"\\n\",\n        \"class FeedFoward(nn.Module):\\n\",\n        \"    \\\"\\\"\\\" a simple linear layer followed by a non-linearity \\\"\\\"\\\"\\n\",\n        \"\\n\",\n        \"    def __init__(self, n_embd):\\n\",\n        \"        super().__init__()\\n\",\n        \"        self.net = nn.Sequential(\\n\",\n        \"            nn.Linear(n_embd, 4 * n_embd),\\n\",\n        \"            nn.ReLU(),\\n\",\n        \"            nn.Linear(4 * n_embd, n_embd),\\n\",\n        \"            nn.Dropout(dropout),\\n\",\n        \"        )\\n\",\n        \"\\n\",\n        \"    def forward(self, x):\\n\",\n        \"        return self.net(x)\\n\",\n        \"\\n\",\n        \"class Block(nn.Module):\\n\",\n        \"    \\\"\\\"\\\" Transformer block: communication followed by computation \\\"\\\"\\\"\\n\",\n        \"\\n\",\n        \"    def __init__(self, n_embd, n_head):\\n\",\n        \"        # n_embd: embedding dimension, n_head: the number of heads we'd like\\n\",\n        \"        super().__init__()\\n\",\n        \"        head_size = n_embd // n_head\\n\",\n        \"        self.sa = MultiHeadAttention(n_head, head_size)\\n\",\n        \"        self.ffwd = FeedFoward(n_embd)\\n\",\n        \"        self.ln1 = nn.LayerNorm(n_embd)\\n\",\n        \"        self.ln2 = nn.LayerNorm(n_embd)\\n\",\n        \"\\n\",\n        \"    def forward(self, x):\\n\",\n        \"        x = x + self.sa(self.ln1(x))\\n\",\n        \"        x = x + self.ffwd(self.ln2(x))\\n\",\n        \"        return x\\n\",\n        \"\\n\",\n        \"class GPT(nn.Module):\\n\",\n        \"\\n\",\n        \"    def __init__(self):\\n\",\n        \"        super().__init__()\\n\",\n        \"        # each token directly reads off the logits for the next token from a lookup table\\n\",\n        \"        self.token_embedding_table = nn.Embedding(vocab_size, n_embd)\\n\",\n        \"        self.position_embedding_table = nn.Embedding(block_size, n_embd)\\n\",\n        \"        self.blocks = nn.Sequential(*[Block(n_embd, n_head=n_head) for _ in range(n_layer)])\\n\",\n        \"        self.ln_f = nn.LayerNorm(n_embd) # final layer norm\\n\",\n        \"        self.lm_head = nn.Linear(n_embd, vocab_size)\\n\",\n        \"\\n\",\n        \"    def forward(self, idx, targets=None):\\n\",\n        \"        B, T = idx.shape\\n\",\n        \"\\n\",\n        \"        # idx and targets are both (B,T) tensor of integers\\n\",\n        \"        tok_emb = self.token_embedding_table(idx) # (B,T,C)\\n\",\n        \"        pos_emb = self.position_embedding_table(torch.arange(T, device=device)) # (T,C)\\n\",\n        \"        x = tok_emb + pos_emb # (B,T,C)\\n\",\n        \"        x = self.blocks(x) # (B,T,C)\\n\",\n        \"        x = self.ln_f(x) # (B,T,C)\\n\",\n        \"        logits = self.lm_head(x) # (B,T,vocab_size)\\n\",\n        \"\\n\",\n        \"        if targets is None:\\n\",\n        \"            loss = None\\n\",\n        \"        else:\\n\",\n        \"            B, T, C = logits.shape\\n\",\n        \"            logits = logits.view(B*T, C)\\n\",\n        \"            targets = targets.view(B*T)\\n\",\n        \"            loss = F.cross_entropy(logits, targets)\\n\",\n        \"\\n\",\n        \"        return logits, loss\\n\",\n        \"\\n\",\n        \"    def generate(self, idx, max_new_tokens):\\n\",\n        \"        # idx is (B, T) array of indices in the current context\\n\",\n        \"        for _ in range(max_new_tokens):\\n\",\n        \"            # crop idx to the last block_size tokens\\n\",\n        \"            idx_cond = idx[:, -block_size:]\\n\",\n        \"            # get the predictions\\n\",\n        \"            logits, loss = self(idx_cond)\\n\",\n        \"            # focus only on the last time step\\n\",\n        \"            logits = logits[:, -1, :] # becomes (B, C)\\n\",\n        \"            # apply softmax to get probabilities\\n\",\n        \"            probs = F.softmax(logits, dim=-1) # (B, C)\\n\",\n        \"            # sample from the distribution\\n\",\n        \"            idx_next = torch.multinomial(probs, num_samples=1) # (B, 1)\\n\",\n        \"            # append sampled index to the running sequence\\n\",\n        \"            idx = torch.cat((idx, idx_next), dim=1) # (B, T+1)\\n\",\n        \"        return idx\\n\",\n        \"\\n\",\n        \"model = GPT()\\n\",\n        \"m = model.to(device)\\n\",\n        \"# print the number of parameters in the model\\n\",\n        \"print(sum(p.numel() for p in m.parameters())/1e6, 'M parameters')\\n\",\n        \"\\n\",\n        \"# create a PyTorch optimizer\\n\",\n        \"optimizer = torch.optim.AdamW(model.parameters(), lr=learning_rate)\\n\",\n        \"\\n\",\n        \"for iter in range(max_iters):\\n\",\n        \"\\n\",\n        \"    # every once in a while evaluate the loss on train and val sets\\n\",\n        \"    if iter % eval_interval == 0 or iter == max_iters - 1:\\n\",\n        \"        losses = estimate_loss()\\n\",\n        \"        print(f\\\"step {iter}: train loss {losses['train']:.4f}, val loss {losses['val']:.4f}\\\")\\n\",\n        \"\\n\",\n        \"    # sample a batch of data\\n\",\n        \"    xb, yb = get_batch('train')\\n\",\n        \"\\n\",\n        \"    # evaluate the loss\\n\",\n        \"    logits, loss = model(xb, yb)\\n\",\n        \"    optimizer.zero_grad(set_to_none=True)\\n\",\n        \"    loss.backward()\\n\",\n        \"    optimizer.step()\"\n      ],\n      \"metadata\": {\n        \"colab\": {\n          \"base_uri\": \"https://localhost:8080/\"\n        },\n        \"id\": \"ngS-jQschyJc\",\n        \"outputId\": \"8dedfa53-bdf7-446c-d335-887bf740efab\"\n      },\n      \"execution_count\": 6,\n      \"outputs\": [\n        {\n          \"output_type\": \"stream\",\n          \"name\": \"stdout\",\n          \"text\": [\n            \"6.684497 M parameters\\n\",\n            \"step 0: train loss 10.9982, val loss 11.0105\\n\",\n            \"step 100: train loss 6.3955, val loss 6.4657\\n\",\n            \"step 200: train loss 6.0255, val loss 6.1584\\n\",\n            \"step 300: train loss 5.7608, val loss 5.9578\\n\",\n            \"step 400: train loss 5.4809, val loss 5.7327\\n\",\n            \"step 500: train loss 5.2543, val loss 5.5400\\n\",\n            \"step 600: train loss 5.1017, val loss 5.4008\\n\",\n            \"step 700: train loss 4.9718, val loss 5.3382\\n\",\n            \"step 800: train loss 4.8850, val loss 5.2539\\n\",\n            \"step 900: train loss 4.8084, val loss 5.1674\\n\",\n            \"step 1000: train loss 4.7187, val loss 5.0770\\n\",\n            \"step 1100: train loss 4.6621, val loss 5.1166\\n\",\n            \"step 1200: train loss 4.5916, val loss 5.0159\\n\",\n            \"step 1300: train loss 4.5599, val loss 5.0297\\n\",\n            \"step 1400: train loss 4.4998, val loss 4.9929\\n\",\n            \"step 1500: train loss 4.4481, val loss 4.9933\\n\",\n            \"step 1600: train loss 4.4182, val loss 4.9204\\n\",\n            \"step 1700: train loss 4.4202, val loss 4.9445\\n\",\n            \"step 1800: train loss 4.3644, val loss 4.9217\\n\",\n            \"step 1900: train loss 4.3479, val loss 4.9221\\n\",\n            \"step 2000: train loss 4.2925, val loss 4.8603\\n\",\n            \"step 2100: train loss 4.2446, val loss 4.8765\\n\",\n            \"step 2200: train loss 4.2491, val loss 4.8753\\n\",\n            \"step 2300: train loss 4.1900, val loss 4.8271\\n\",\n            \"step 2400: train loss 4.1705, val loss 4.8660\\n\",\n            \"step 2500: train loss 4.1608, val loss 4.8778\\n\",\n            \"step 2600: train loss 4.1379, val loss 4.8898\\n\",\n            \"step 2700: train loss 4.1021, val loss 4.8513\\n\",\n            \"step 2800: train loss 4.1099, val loss 4.8742\\n\",\n            \"step 2900: train loss 4.0874, val loss 4.8534\\n\",\n            \"step 2999: train loss 4.0720, val loss 4.8296\\n\"\n          ]\n        }\n      ]\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"# generate from the model\\n\",\n        \"context = torch.zeros((1, 1), dtype=torch.long, device=device)\\n\",\n        \"print(enc.decode(m.generate(context, max_new_tokens=100)[0].tolist()))\"\n      ],\n      \"metadata\": {\n        \"colab\": {\n          \"base_uri\": \"https://localhost:8080/\"\n        },\n        \"id\": \"9C93Lk1biRLE\",\n        \"outputId\": \"94a58a5d-a9fb-438f-ec29-ab231b145951\"\n      },\n      \"execution_count\": 7,\n      \"outputs\": [\n        {\n          \"output_type\": \"stream\",\n          \"name\": \"stdout\",\n          \"text\": [\n            \"! what any consul\\n\",\n            \"matter they have we be for so dishonour\\n\",\n            \"Redpt'd in our courtesy,\\n\",\n            \"Or his highness i' the embracements, nor feeds had an err:\\n\",\n            \"I have the hallissign daughter to pawn'd.\\n\",\n            \"and stout see me: but God they forget\\n\",\n            \"And meet the tribranch man,\\n\",\n            \"Of a brace art'd; I'll plant show't\\n\",\n            \"From whom I a joyful with a chair access with callat.\\n\",\n            \"\\n\",\n            \"HENRY\\n\"\n          ]\n        }\n      ]\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"!pip install transformers\\n\",\n        \"from transformers import pipeline\\n\",\n        \"sentiment_pipeline = pipeline(model=\\\"finiteautomata/bertweet-base-sentiment-analysis\\\")\"\n      ],\n      \"metadata\": {\n        \"colab\": {\n          \"base_uri\": \"https://localhost:8080/\",\n          \"height\": 675,\n          \"referenced_widgets\": [\n            \"49506f92d9d042d4a6111f3a1b9f79e1\",\n            \"95f4423f202b44f9b5f6fbc250b5a300\",\n            \"e78ecc50321f42328d47999c01244651\",\n            \"cc422e07c3ce4ba99c84bda8b7d7c596\",\n            \"8be555a8bb6e4068a42e4312e75a30c2\",\n            \"b7f7646e496849b8bb3239b64b93e193\",\n            \"b728662de8ac41888e838e47e67d6aa9\",\n            \"70a9186eb9e64d9ea5b456f43622d979\",\n            \"7a5dedee422f42b49ecac65d3138c167\",\n            \"c2e28b80efc74a549af7bdf3eee46eda\",\n            \"057de1e97cad4a2297ad3db02c2481d6\",\n            \"daabe15b3af34e1098a7e2f620ae1f36\",\n            \"c8fd1744cb744200b0bcc8629d173bf1\",\n            \"7a52e4742e594e25a28feb18c417c076\",\n            \"513d02e2c28440d9897d1aa097d1e743\",\n            \"d147cf4c15ff449598d6b429c90db0ae\",\n            \"c74d25684a4d4eafbd3546dcff921cb0\",\n            \"aa57d59560114d55bede331307257f6b\",\n            \"e5c0bc2de75347c39dc9ca23a8129650\",\n            \"a18d35c8ff2649e397ed983a6f06e959\",\n            \"5e3d4cc0ec9d4c0ca0d0136e97aca2ce\",\n            \"3d4f1a5c8128451884bd21fef234b89b\",\n            \"9645d7c59b0546159890f042ca420632\",\n            \"8893c2570f464d9697dda12cc42023d5\",\n            \"0c71d29e93f24885802ad5fa6d240675\",\n            \"7b859eccf54e41698cf179d68bb109d0\",\n            \"725944a581244ab6a2a4b61afa29c1c0\",\n            \"e536fea041f7441aa8cc93a448492e33\",\n            \"8e5f96a8ec714284a7f000af96300f67\",\n            \"04984bbb402c4bd4afad349e629e2452\",\n            \"4a07648ab35047c8b61e32e3a61d970b\",\n            \"bd5178cf064049678d4efac17e47a5da\",\n            \"ae235b9add2d41ee9f808a2b4d43568b\",\n            \"8fb0a31c3ecf4a4880803e81e1bc581d\",\n            \"8bc84935b3964e338d00ea1bb7c3c59f\",\n            \"d729ac6a87174fbda03175c21859ea82\",\n            \"b80bd9149ae34878bfeeb4fca815231c\",\n            \"40007a4900824bd88d3a180fd0a8325a\",\n            \"ce6627b477bc430d968fd186443b34e3\",\n            \"2f2cf4fd7a434da5be62bd088360a6ba\",\n            \"31aa09c7a2ea443b8a177c4f91a7d427\",\n            \"f17f1682313e49df97e399508c15716a\",\n            \"dde9d315de174921844ebb72865d1d3f\",\n            \"386eb97816cd41e188dbfa1fcd299b16\",\n            \"d5af658c67874c67b08bd8ef72411ea1\",\n            \"aba6fb6586534c1f86ce65b7c8bde51f\",\n            \"a6878fe333e243dc81ec0d36bb0dce19\",\n            \"d1efe95c3d55440386db28242a8f38d1\",\n            \"ecfb1bf7ae8a4328ac982234127e5c2b\",\n            \"4847c36ed46d49429e37f4b9b7862a57\",\n            \"631e5ee44ccc4e4aaec456af90df7965\",\n            \"662fd517bf0540ed9c607d3ac8dda0af\",\n            \"04d90c9c251a4fae8db96d711568cd64\",\n            \"7bd831ca2d354ee7a75c22c700963ea8\",\n            \"3de7e32fefd54df6a6d3596205ba0338\",\n            \"ddbbcddaba34416881ebb023864b0f2d\",\n            \"d0fc35ec11634f8691bf5eefa0505cb6\",\n            \"598a025b08dd4469a88bd9ebe85ebb94\",\n            \"9bc45acff2e049d59eef1e8374f03a93\",\n            \"5a627f08513443e09dac036f3ff2fa1f\",\n            \"397d77d019e748f2b4178bb78ff2d471\",\n            \"590cb10ff4164d309063faf6baedaf1f\",\n            \"9d5f595908964edba6b6f7e8a6fba5eb\",\n            \"0e05ee7e2e934ca983d70addd1c9ce92\",\n            \"9c56fb6a76414f558c665608a21d39cd\",\n            \"399aa39b85c14ff8b9cd887343af36be\",\n            \"d4712217939b44f59e58a0f23a8d4df7\",\n            \"c4fa3386d2234473ad7a8b6dae7a4d95\",\n            \"97c0a2c0b079410496ab497f8fa8bd12\",\n            \"565f60bb19d443f6aacc9b08ff68c7ff\",\n            \"5556b66847b04b4aabe0aaa029dc3e64\",\n            \"fe8f0c618d104deaaed3ea138acdc230\",\n            \"52b516dd11a747099e9e0e808a168e55\",\n            \"8696eb98489f409a848684b77b2353ac\",\n            \"679b6d995fad4a649e055bee8bfe83d1\",\n            \"eff4e772ccd4445ba4b6f624ea3b7e8f\",\n            \"c7570cd91b564389b59c98a694ab7771\"\n          ]\n        },\n        \"id\": \"f93j9jAXuK8S\",\n        \"outputId\": \"d585128a-3fad-4415-e564-fe9d338c4726\"\n      },\n      \"execution_count\": 8,\n      \"outputs\": [\n        {\n          \"output_type\": \"stream\",\n          \"name\": \"stdout\",\n          \"text\": [\n            \"Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\\n\",\n            \"Collecting transformers\\n\",\n            \"  Downloading transformers-4.27.1-py3-none-any.whl (6.7 MB)\\n\",\n            \"\\u001b[2K     \\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\\u001b[0m \\u001b[32m6.7/6.7 MB\\u001b[0m \\u001b[31m90.2 MB/s\\u001b[0m eta \\u001b[36m0:00:00\\u001b[0m\\n\",\n            \"\\u001b[?25hRequirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.9/dist-packages (from transformers) (4.65.0)\\n\",\n            \"Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.9/dist-packages (from transformers) (2022.10.31)\\n\",\n            \"Collecting huggingface-hub<1.0,>=0.11.0\\n\",\n            \"  Downloading huggingface_hub-0.13.2-py3-none-any.whl (199 kB)\\n\",\n            \"\\u001b[2K     \\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\\u001b[0m \\u001b[32m199.2/199.2 KB\\u001b[0m \\u001b[31m23.6 MB/s\\u001b[0m eta \\u001b[36m0:00:00\\u001b[0m\\n\",\n            \"\\u001b[?25hRequirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.9/dist-packages (from transformers) (6.0)\\n\",\n            \"Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.9/dist-packages (from transformers) (1.22.4)\\n\",\n            \"Requirement already satisfied: requests in /usr/local/lib/python3.9/dist-packages (from transformers) (2.27.1)\\n\",\n            \"Requirement already satisfied: filelock in /usr/local/lib/python3.9/dist-packages (from transformers) (3.10.0)\\n\",\n            \"Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.9/dist-packages (from transformers) (23.0)\\n\",\n            \"Collecting tokenizers!=0.11.3,<0.14,>=0.11.1\\n\",\n            \"  Downloading tokenizers-0.13.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.6 MB)\\n\",\n            \"\\u001b[2K     \\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\\u001b[0m \\u001b[32m7.6/7.6 MB\\u001b[0m \\u001b[31m93.8 MB/s\\u001b[0m eta \\u001b[36m0:00:00\\u001b[0m\\n\",\n            \"\\u001b[?25hRequirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.9/dist-packages (from huggingface-hub<1.0,>=0.11.0->transformers) (4.5.0)\\n\",\n            \"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests->transformers) (2022.12.7)\\n\",\n            \"Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests->transformers) (2.0.12)\\n\",\n            \"Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests->transformers) (1.26.15)\\n\",\n            \"Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests->transformers) (3.4)\\n\",\n            \"Installing collected packages: tokenizers, huggingface-hub, transformers\\n\",\n            \"Successfully installed huggingface-hub-0.13.2 tokenizers-0.13.2 transformers-4.27.1\\n\"\n          ]\n        },\n        {\n          \"output_type\": \"display_data\",\n          \"data\": {\n            \"text/plain\": [\n              \"Downloading (…)lve/main/config.json:   0%|          | 0.00/949 [00:00<?, ?B/s]\"\n            ],\n            \"application/vnd.jupyter.widget-view+json\": {\n              \"version_major\": 2,\n              \"version_minor\": 0,\n              \"model_id\": \"49506f92d9d042d4a6111f3a1b9f79e1\"\n            }\n          },\n          \"metadata\": {}\n        },\n        {\n          \"output_type\": \"display_data\",\n          \"data\": {\n            \"text/plain\": [\n              \"Downloading pytorch_model.bin:   0%|          | 0.00/540M [00:00<?, ?B/s]\"\n            ],\n            \"application/vnd.jupyter.widget-view+json\": {\n              \"version_major\": 2,\n              \"version_minor\": 0,\n              \"model_id\": \"daabe15b3af34e1098a7e2f620ae1f36\"\n            }\n          },\n          \"metadata\": {}\n        },\n        {\n          \"output_type\": \"display_data\",\n          \"data\": {\n            \"text/plain\": [\n              \"Downloading (…)okenizer_config.json:   0%|          | 0.00/338 [00:00<?, ?B/s]\"\n            ],\n            \"application/vnd.jupyter.widget-view+json\": {\n              \"version_major\": 2,\n              \"version_minor\": 0,\n              \"model_id\": \"9645d7c59b0546159890f042ca420632\"\n            }\n          },\n          \"metadata\": {}\n        },\n        {\n          \"output_type\": \"display_data\",\n          \"data\": {\n            \"text/plain\": [\n              \"Downloading (…)solve/main/vocab.txt:   0%|          | 0.00/843k [00:00<?, ?B/s]\"\n            ],\n            \"application/vnd.jupyter.widget-view+json\": {\n              \"version_major\": 2,\n              \"version_minor\": 0,\n              \"model_id\": \"8fb0a31c3ecf4a4880803e81e1bc581d\"\n            }\n          },\n          \"metadata\": {}\n        },\n        {\n          \"output_type\": \"display_data\",\n          \"data\": {\n            \"text/plain\": [\n              \"Downloading (…)solve/main/bpe.codes:   0%|          | 0.00/1.08M [00:00<?, ?B/s]\"\n            ],\n            \"application/vnd.jupyter.widget-view+json\": {\n              \"version_major\": 2,\n              \"version_minor\": 0,\n              \"model_id\": \"d5af658c67874c67b08bd8ef72411ea1\"\n            }\n          },\n          \"metadata\": {}\n        },\n        {\n          \"output_type\": \"display_data\",\n          \"data\": {\n            \"text/plain\": [\n              \"Downloading (…)in/added_tokens.json:   0%|          | 0.00/22.0 [00:00<?, ?B/s]\"\n            ],\n            \"application/vnd.jupyter.widget-view+json\": {\n              \"version_major\": 2,\n              \"version_minor\": 0,\n              \"model_id\": \"ddbbcddaba34416881ebb023864b0f2d\"\n            }\n          },\n          \"metadata\": {}\n        },\n        {\n          \"output_type\": \"display_data\",\n          \"data\": {\n            \"text/plain\": [\n              \"Downloading (…)cial_tokens_map.json:   0%|          | 0.00/167 [00:00<?, ?B/s]\"\n            ],\n            \"application/vnd.jupyter.widget-view+json\": {\n              \"version_major\": 2,\n              \"version_minor\": 0,\n              \"model_id\": \"d4712217939b44f59e58a0f23a8d4df7\"\n            }\n          },\n          \"metadata\": {}\n        },\n        {\n          \"output_type\": \"stream\",\n          \"name\": \"stderr\",\n          \"text\": [\n            \"emoji is not installed, thus not converting emoticons or emojis into text. Install emoji: pip3 install emoji==0.6.0\\n\"\n          ]\n        }\n      ]\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"inps = [\\\"I'm the king of the world!\\\", \\n\",\n        \"        \\\"I'll be back.\\\",\\n\",\n        \"        \\\"The cake is a lie\\\", \\n\",\n        \"        \\\"To be forgotten is worse than death\\\",\\n\",\n        \"        \\\"All happy families are alike; each unhappy family is unhappy in its own way.\\\",\\n\",\n        \"        \\\"You don't need a reason to help people\\\",\\n\",\n        \"        ]\\n\",\n        \"res = sentiment_pipeline(inps)\\n\",\n        \"\\n\",\n        \"for i in range(len(inps)):\\n\",\n        \"  res[i]['text'] = inps[i]\\n\",\n        \"  print(res[i])\"\n      ],\n      \"metadata\": {\n        \"colab\": {\n          \"base_uri\": \"https://localhost:8080/\"\n        },\n        \"id\": \"v-fryBdOyKSD\",\n        \"outputId\": \"671ed206-a9eb-4793-992b-da9378535088\"\n      },\n      \"execution_count\": 9,\n      \"outputs\": [\n        {\n          \"output_type\": \"stream\",\n          \"name\": \"stdout\",\n          \"text\": [\n            \"{'label': 'POS', 'score': 0.9771729707717896, 'text': \\\"I'm the king of the world!\\\"}\\n\",\n            \"{'label': 'POS', 'score': 0.5481611490249634, 'text': \\\"I'll be back.\\\"}\\n\",\n            \"{'label': 'NEG', 'score': 0.7581188678741455, 'text': 'The cake is a lie'}\\n\",\n            \"{'label': 'NEG', 'score': 0.8209365606307983, 'text': 'To be forgotten is worse than death'}\\n\",\n            \"{'label': 'NEU', 'score': 0.7874237895011902, 'text': 'All happy families are alike; each unhappy family is unhappy in its own way.'}\\n\",\n            \"{'label': 'NEU', 'score': 0.8731082081794739, 'text': \\\"You don't need a reason to help people\\\"}\\n\"\n          ]\n        }\n      ]\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"def get_reward(text, mode):\\n\",\n        \"  sent = sentiment_pipeline(text)\\n\",\n        \"  if mode == '+ve':\\n\",\n        \"    labels = torch.tensor([a['label']=='POS' for a in sent],dtype=torch.float16).unsqueeze(-1).to(device)\\n\",\n        \"  elif mode == '-ve':\\n\",\n        \"    labels = torch.tensor([a['label']=='NEG' for a in sent],dtype=torch.float16).unsqueeze(-1).to(device)\\n\",\n        \"  else:\\n\",\n        \"    raise ValueError('Unknown Mode')\\n\",\n        \"  \\n\",\n        \"  weights = torch.tensor([a['score'] for a in sent],dtype=torch.float32).unsqueeze(-1).to(device)\\n\",\n        \"  \\n\",\n        \"  rewards = labels * weights # (B, 1)\\n\",\n        \"\\n\",\n        \"  return rewards\"\n      ],\n      \"metadata\": {\n        \"id\": \"St_giJWTZ19y\"\n      },\n      \"execution_count\": 10,\n      \"outputs\": []\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"def flatten(l):\\n\",\n        \"    return [item for sublist in l for item in sublist]\\n\",\n        \"print('Rewards in +ve mode')\\n\",\n        \"list(zip(inps, flatten(get_reward(inps, '+ve').tolist())))\\n\"\n      ],\n      \"metadata\": {\n        \"colab\": {\n          \"base_uri\": \"https://localhost:8080/\"\n        },\n        \"id\": \"CmjSzmslbZZc\",\n        \"outputId\": \"52846692-ffce-4659-f6cb-f0d9040b6f9d\"\n      },\n      \"execution_count\": 11,\n      \"outputs\": [\n        {\n          \"output_type\": \"stream\",\n          \"name\": \"stdout\",\n          \"text\": [\n            \"Rewards in +ve mode\\n\"\n          ]\n        },\n        {\n          \"output_type\": \"execute_result\",\n          \"data\": {\n            \"text/plain\": [\n              \"[(\\\"I'm the king of the world!\\\", 0.9771729707717896),\\n\",\n              \" (\\\"I'll be back.\\\", 0.5481611490249634),\\n\",\n              \" ('The cake is a lie', 0.0),\\n\",\n              \" ('To be forgotten is worse than death', 0.0),\\n\",\n              \" ('All happy families are alike; each unhappy family is unhappy in its own way.',\\n\",\n              \"  0.0),\\n\",\n              \" (\\\"You don't need a reason to help people\\\", 0.0)]\"\n            ]\n          },\n          \"metadata\": {},\n          \"execution_count\": 11\n        }\n      ]\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"print('Rewards in -ve mode')\\n\",\n        \"list(zip(inps, flatten(get_reward(inps, '-ve').tolist())))\"\n      ],\n      \"metadata\": {\n        \"colab\": {\n          \"base_uri\": \"https://localhost:8080/\"\n        },\n        \"id\": \"OzUDgNwGcfAH\",\n        \"outputId\": \"2b0ba9e3-41e9-4450-b5b0-d6cee2eb4a00\"\n      },\n      \"execution_count\": 12,\n      \"outputs\": [\n        {\n          \"output_type\": \"stream\",\n          \"name\": \"stdout\",\n          \"text\": [\n            \"Rewards in -ve mode\\n\"\n          ]\n        },\n        {\n          \"output_type\": \"execute_result\",\n          \"data\": {\n            \"text/plain\": [\n              \"[(\\\"I'm the king of the world!\\\", 0.0),\\n\",\n              \" (\\\"I'll be back.\\\", 0.0),\\n\",\n              \" ('The cake is a lie', 0.7581188678741455),\\n\",\n              \" ('To be forgotten is worse than death', 0.8209365606307983),\\n\",\n              \" ('All happy families are alike; each unhappy family is unhappy in its own way.',\\n\",\n              \"  0.0),\\n\",\n              \" (\\\"You don't need a reason to help people\\\", 0.0)]\"\n            ]\n          },\n          \"metadata\": {},\n          \"execution_count\": 12\n        }\n      ]\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"eval_interval_rlhf = 20\\n\",\n        \"max_iters_rlhf = 1000\\n\"\n      ],\n      \"metadata\": {\n        \"id\": \"37tezEm-CFXR\"\n      },\n      \"execution_count\": 13,\n      \"outputs\": []\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"from torch.distributions import Categorical\\n\",\n        \"class RLHF(nn.Module):\\n\",\n        \"    def __init__(self, model):\\n\",\n        \"        super().__init__()\\n\",\n        \"        self.model = model\\n\",\n        \"\\n\",\n        \"    def forward(self, idx, targets=None):\\n\",\n        \"        return self.model(idx, targets)\\n\",\n        \"     \\n\",\n        \"    def generate(self, idx, max_new_tokens, block_size, ref_model=None):\\n\",\n        \"        # idx is (B, T) array of indices in the current context\\n\",\n        \"        log_probs = torch.tensor([]).to(device)\\n\",\n        \"        log_probs_ref = torch.tensor([]).to(device)\\n\",\n        \"        \\n\",\n        \"        for i in range(max_new_tokens):\\n\",\n        \"            # crop idx to the last block_size tokens\\n\",\n        \"            idx_cond = idx[:, -block_size:]\\n\",\n        \"\\n\",\n        \"            # get the predictions\\n\",\n        \"            logits, loss = self(idx_cond)\\n\",\n        \"\\n\",\n        \"            # focus only on the last time step\\n\",\n        \"            logits = logits[:, -1, :] # becomes (B, C)\\n\",\n        \"            \\n\",\n        \"            # logits define instance of Iategorical class\\n\",\n        \"            m = Categorical(logits=logits)\\n\",\n        \"            \\n\",\n        \"            # sample from the distribution\\n\",\n        \"            idx_next = m.sample()\\n\",\n        \"            \\n\",\n        \"            # get the log probability and append to running sequence\\n\",\n        \"            log_probs_idx_next = m.log_prob(idx_next)    \\n\",\n        \"            log_probs = torch.cat((log_probs, log_probs_idx_next.view(-1,1)), dim=1)\\n\",\n        \"            \\n\",\n        \"            if ref_model is not None:\\n\",\n        \"              # get log probability of sample idx_next under the reference model\\n\",\n        \"              logits_ref, _ = ref_model(idx_cond)\\n\",\n        \"              logits_ref = logits_ref[:, -1, :] # becomes (B, C)\\n\",\n        \"            \\n\",\n        \"              m_ref = Categorical(logits=logits_ref)\\n\",\n        \"              log_probs_ref_idx_next = m_ref.log_prob(idx_next)    \\n\",\n        \"              log_probs_ref = torch.cat((log_probs_ref, log_probs_ref_idx_next.view(-1,1)), dim=1)\\n\",\n        \"\\n\",\n        \"            # append sampled index to the running sequence\\n\",\n        \"            idx = torch.cat((idx, idx_next.view(-1,1)), dim=1) # (B, T+1)\\n\",\n        \"\\n\",\n        \"        return idx, log_probs, log_probs_ref\"\n      ],\n      \"metadata\": {\n        \"id\": \"Rp4iqC2RWm6n\"\n      },\n      \"execution_count\": 14,\n      \"outputs\": []\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"import copy\\n\",\n        \"ref_model = copy.deepcopy(model)\"\n      ],\n      \"metadata\": {\n        \"id\": \"4Z_-ygQYYhal\"\n      },\n      \"execution_count\": 15,\n      \"outputs\": []\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"import time\\n\",\n        \"import numpy as np\\n\",\n        \"\\n\",\n        \"RLHFmodel = RLHF(model)\\n\",\n        \"RLHFmodel.to(device)\\n\",\n        \"\\n\",\n        \"ref_model.to(device)\\n\",\n        \"\\n\",\n        \"actor_optimizer = torch.optim.AdamW(RLHFmodel.parameters(), lr=1e-3)\\n\",\n        \"X, Y = get_batch('train') # fetch the very first batch\\n\",\n        \"X = torch.ones((X.shape[0], 1), dtype=torch.long).to(device) # for now there is no prompt\\n\",\n        \"X = X*enc.encode('The')[0] # start with ''The'\\n\",\n        \"t0  = time.time()\\n\",\n        \"max_new_tokens = block_size\\n\",\n        \"rews_all = []\\n\",\n        \"actor_loss_all = []\\n\",\n        \"mode = '+ve'\\n\",\n        \"ref_coef = 0.2\\n\",\n        \"e_coef = 0.1\\n\",\n        \"for iter in range(max_iters_rlhf):\\n\",\n        \"\\n\",\n        \"  states, log_probs, log_probs_ref = RLHFmodel.generate(\\n\",\n        \"      X, max_new_tokens, block_size, ref_model=ref_model)\\n\",\n        \"\\n\",\n        \"  states = states[:,-max_new_tokens:]\\n\",\n        \"  log_probs = log_probs[:,-max_new_tokens:] # (B, max_new_tokens)\\n\",\n        \"  if ref_model is not None:\\n\",\n        \"    log_probs_ref = log_probs_ref[:,-max_new_tokens:] # (B, max_new_tokens)\\n\",\n        \"  \\n\",\n        \"  rewards = get_reward([enc.decode(s.tolist()) for s in states], mode)\\n\",\n        \"  \\n\",\n        \"  pg = (rewards+ref_coef*log_probs_ref-e_coef*log_probs)* log_probs.squeeze()\\n\",\n        \"  \\n\",\n        \"  # log(1) = 0\\n\",\n        \"  # -log(1/N) = log(N)\\n\",\n        \"\\n\",\n        \"  # when ref_coef=e_coef this is equivalent to penalising for KL divergence\\n\",\n        \"  # pg = (rewards-ref_coef*(log_probs-log_probs_ref)* log_probs.squeeze() \\n\",\n        \"  \\n\",\n        \"  actor_loss = -pg.sum()\\n\",\n        \"\\n\",\n        \"  actor_optimizer.zero_grad(set_to_none=True)\\n\",\n        \"  actor_loss.backward()\\n\",\n        \"  actor_optimizer.step()\\n\",\n        \"\\n\",\n        \"  rews_all.append(rewards.mean().detach().cpu().numpy())\\n\",\n        \"  actor_loss_all.append(actor_loss.detach().cpu().numpy())\\n\",\n        \"\\n\",\n        \"  if iter % eval_interval_rlhf == 0:\\n\",\n        \"      t1 = time.time()\\n\",\n        \"      print('\\\\n')\\n\",\n        \"      print(f'iter: {iter}, time: {t1-t0}')\\n\",\n        \"      print(f'Actor loss: {np.mean(actor_loss_all[-eval_interval_rlhf:])}')\\n\",\n        \"      print(f'rets: {np.mean(rews_all[-eval_interval_rlhf:])}')\\n\",\n        \"\\n\",\n        \"      textRLHF = RLHFmodel.generate(X, 2*max_new_tokens, block_size, ref_model=None)[0]\\n\",\n        \"      for i in range(1):\\n\",\n        \"          text_i = textRLHF[i,:]\\n\",\n        \"          print(enc.decode(text_i.tolist()))\\n\"\n      ],\n      \"metadata\": {\n        \"colab\": {\n          \"base_uri\": \"https://localhost:8080/\"\n        },\n        \"id\": \"iZ7HG0qAahgo\",\n        \"outputId\": \"0c6b1d4a-c4a8-4c63-ae51-3f399f5c343b\"\n      },\n      \"execution_count\": 16,\n      \"outputs\": [\n        {\n          \"output_type\": \"stream\",\n          \"name\": \"stdout\",\n          \"text\": [\n            \"\\n\",\n            \"\\n\",\n            \"iter: 0, time: 2.0925092697143555\\n\",\n            \"Actor loss: -1174.6953125\\n\",\n            \"rets: 0.15557555854320526\\n\",\n            \"The nobles\\n\",\n            \"withoveseed\\n\",\n            \"Of our dimina to the narrow, of mine,\\n\",\n            \"For this earth brawling some fathers, thou shalt\\n\",\n            \"Con exceeds bow an ostows that were I\\n\",\n            \"actioning Roe that word;\\n\",\n            \"For what 'twas, likeers Lartft,\\n\",\n            \"And whose secret maid\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 20, time: 43.471943855285645\\n\",\n            \"Actor loss: -1276.6146240234375\\n\",\n            \"rets: 0.08284272998571396\\n\",\n            \"The deceived!\\n\",\n            \"\\n\",\n            \"SICINs:\\n\",\n            \"The very faults of Norfolk is the cousin of sword;\\n\",\n            \"And, by me how so, seldom the last of that wounds\\n\",\n            \"Or much power home his clish'd utterhire, wine of that dame'd\\n\",\n            \"Of my life as the loss of death, be\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 40, time: 83.07254672050476\\n\",\n            \"Actor loss: -1171.9931640625\\n\",\n            \"rets: 0.12373824417591095\\n\",\n            \"The bride, wives are\\n\",\n            \"Mis kneel to repent him for heaven and in debt.\\n\",\n            \"Look, thou fly, makes your beauty is his rash, I will not\\n\",\n            \"Say a side that taught their robbery.\\n\",\n            \"Mis up the demands Hastings.\\n\",\n            \"\\n\",\n            \"JULNot SAL:\\n\",\n            \"Your brother is drawn, and in\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 60, time: 123.09890699386597\\n\",\n            \"Actor loss: -1200.064208984375\\n\",\n            \"rets: 0.1355905532836914\\n\",\n            \"The clouds in the blood'd age'd fearful's throne,\\n\",\n            \"To the priestred of me.\\n\",\n            \"\\n\",\n            \"HERMAMILLO:\\n\",\n            \"She are Imen, dear out your grace\\n\",\n            \"Whichose on your mother of the Duke of prince young\\n\",\n            \"Of blood of foul sword! I love upon your feet\\n\",\n            \"and ban\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 80, time: 163.42521929740906\\n\",\n            \"Actor loss: -1015.2078247070312\\n\",\n            \"rets: 0.19954833388328552\\n\",\n            \"The loss ofitiveness.\\n\",\n            \"\\n\",\n            \"PAULINA:\\n\",\n            \"I am about the noble house of this presence,\\n\",\n            \" Rouro too words of England,\\n\",\n            \"And nothing, then all so, weeping yet\\n\",\n            \"Your grace comesstrance ouride the proudous thoughts!\\n\",\n            \"\\n\",\n            \"alt! Rome, pastUMNIA:\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 100, time: 203.27433586120605\\n\",\n            \"Actor loss: -904.3199462890625\\n\",\n            \"rets: 0.2031252384185791\\n\",\n            \"The brother.\\n\",\n            \"\\n\",\n            \"MMeasure about the loved,\\n\",\n            \"Our letters of your worthy Gaunt lady:\\n\",\n            \"But four a town is worth'd by him,\\n\",\n            \"To rigence yield hand, there I am anyhee,\\n\",\n            \"I' love that steed our brother\\n\",\n            \"Than in bount about to joy,\\n\",\n            \"And\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 120, time: 243.294597864151\\n\",\n            \"Actor loss: -1021.115234375\\n\",\n            \"rets: 0.16891352832317352\\n\",\n            \"The thrustied\\n\",\n            \"Against his majesty to- idol and entence?\\n\",\n            \"\\n\",\n            \"DUKE VINCENTIO:\\n\",\n            \"Is that was too late that when love.\\n\",\n            \"\\n\",\n            \"JOHN OF GAUNT:\\n\",\n            \"Bend, fear of the dukedom.\\n\",\n            \"\\n\",\n            \"First Lord:\\n\",\n            \"I along, p stopp The\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 140, time: 283.161762714386\\n\",\n            \"Actor loss: -1081.0517578125\\n\",\n            \"rets: 0.16179659962654114\\n\",\n            \"The set-antied live,\\n\",\n            \"So bed change with the sovereign.\\n\",\n            \"\\n\",\n            \"CAMILLIUS:\\n\",\n            \"At me, sir. arise, welcome my lords.\\n\",\n            \"\\n\",\n            \"First Citizen:\\n\",\n            \"What someever,\\n\",\n            \"I know it again so little, that I have\\n\",\n            \"say that done sweat prisoner to Hermione,\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 160, time: 323.04486894607544\\n\",\n            \"Actor loss: -994.4212036132812\\n\",\n            \"rets: 0.18603841960430145\\n\",\n            \"The get hath a true,\\n\",\n            \"If our land, like every laws;\\n\",\n            \"Some common pomp--\\n\",\n            \"\\n\",\n            \"NOROLYCUS:\\n\",\n            \"For fit.\\n\",\n            \"\\n\",\n            \"NORTHUMBERLAND:\\n\",\n            \"'Tis a man that I think one mine too:\\n\",\n            \"Her consent breathing night.\\n\",\n            \"\\n\",\n            \"QUEEN MARG\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 180, time: 362.9912130832672\\n\",\n            \"Actor loss: -1087.4278564453125\\n\",\n            \"rets: 0.16612616181373596\\n\",\n            \"The seven; that else,\\n\",\n            \"If very Lancaster, prisoners hope, I'll sit,\\n\",\n            \"By our honouredraidign,\\n\",\n            \"Whilst endiniusial earued for that and meet,\\n\",\n            \"Who took mean you what thou fool nort\\n\",\n            \"More than they-rate moleer therefore from thygood course:\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 200, time: 402.95860266685486\\n\",\n            \"Actor loss: -878.0130004882812\\n\",\n            \"rets: 0.22536030411720276\\n\",\n            \"The day,\\n\",\n            \"Had she in theEWway of all!\\n\",\n            \"DUCHESS OF lord, I will not sound in theirAL.\\n\",\n            \"\\n\",\n            \"RICHARD:\\n\",\n            \"\\n\",\n            \"Purs III:\\n\",\n            \"Your Mow me, pale!\\n\",\n            \"\\n\",\n            \"LUCIO:\\n\",\n            \"\\n\",\n            \"ClABETH:\\n\",\n            \"O me,\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 220, time: 442.6643555164337\\n\",\n            \"Actor loss: -704.8284301757812\\n\",\n            \"rets: 0.2918173372745514\\n\",\n            \"Thebleness'd\\n\",\n            \" liberties us. Thouce statue,\\n\",\n            \"The good you had a jest of a name?\\n\",\n            \"\\n\",\n            \"TYBOW:\\n\",\n            \"This I love,\\n\",\n            \"So you in oneseech you, as but\\n\",\n            \" bloody fre gentleman's well-day? marry much tortures,\\n\",\n            \"Of your grace,\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 240, time: 482.6602349281311\\n\",\n            \"Actor loss: -1022.29150390625\\n\",\n            \"rets: 0.1844368577003479\\n\",\n            \"The sons?\\n\",\n            \"Lord ROSS cloy that our maid, where there with honour\\n\",\n            \"That I sleep to the soldiers.\\n\",\n            \"\\n\",\n            \"RUKE VINCENTIO:\\n\",\n            \"J the boy; but, to giving my go.\\n\",\n            \"\\n\",\n            \"HASTINGS:\\n\",\n            \"The better of too great little vanity.\\n\",\n            \"\\n\",\n            \"Second\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 260, time: 522.2385444641113\\n\",\n            \"Actor loss: -835.5398559570312\\n\",\n            \"rets: 0.2348378598690033\\n\",\n            \"The absence of you must the\\n\",\n            \"the heard you have full this 'em.\\n\",\n            \"\\n\",\n            \"QUEEN MARGARET:\\n\",\n            \" punishment is love, g hand of you or\\n\",\n            \"More than thidalines.\\n\",\n            \"\\n\",\n            \"PARIS:\\n\",\n            \"Is aATES affails morely then, I have\\n\",\n            \"k Thomas mercy.\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 280, time: 562.4093163013458\\n\",\n            \"Actor loss: -739.087890625\\n\",\n            \"rets: 0.2898559868335724\\n\",\n            \"The suspicious: nor shall come a cause.\\n\",\n            \"\\n\",\n            \"BENVOLUMNIA:\\n\",\n            \"I give a king.\\n\",\n            \"\\n\",\n            \"POLIXENES:\\n\",\n            \"' they are kind,Pray follows as I look,\\n\",\n            \"Is last.\\n\",\n            \"\\n\",\n            \"CAMILLd.\\n\",\n            \"\\n\",\n            \"KING EDWARD IV:\\n\",\n            \"Talk,\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 300, time: 601.8023648262024\\n\",\n            \"Actor loss: -852.7369384765625\\n\",\n            \"rets: 0.20052771270275116\\n\",\n            \"The soldiers.\\n\",\n            \"b moved how and things, give me not despise.\\n\",\n            \"\\n\",\n            \"ISABELLA:\\n\",\n            \"Ay, sir, further, were my lord;\\n\",\n            \"Behee the child', for King your high temples.\\n\",\n            \"Come, let us this prince!\\n\",\n            \"I must make him too younger whither of his\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 320, time: 641.1348361968994\\n\",\n            \"Actor loss: -746.7435913085938\\n\",\n            \"rets: 0.2848814129829407\\n\",\n            \"The nobles of your hand?\\n\",\n            \"\\n\",\n            \"deepT:\\n\",\n            \"And these inferior shall writ thee here? What that shall I were we better and mine\\n\",\n            \"Now do for gentle learning to ours.\\n\",\n            \"\\n\",\n            \"SUNLORIZEL:\\n\",\n            \"Be patient time to their eyes to up thy brother of rose:\\n\",\n            \"Tell him,\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 340, time: 680.8197565078735\\n\",\n            \"Actor loss: -876.5494995117188\\n\",\n            \"rets: 0.2481001913547516\\n\",\n            \"The mile live?\\n\",\n            \"If shall I will dwell round what love.\\n\",\n            \"Marry, take what I sit: give me?\\n\",\n            \"\\n\",\n            \"MENENIUS:\\n\",\n            \"And Warwick's mad?\\n\",\n            \"O shrewakret please you here;\\n\",\n            \"And all that I'll not.\\n\",\n            \"3 KING EDWARD IV:\\n\",\n            \"E\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 360, time: 720.4318358898163\\n\",\n            \"Actor loss: -697.50244140625\\n\",\n            \"rets: 0.3108132779598236\\n\",\n            \"The king will bear the royal, like a ones,\\n\",\n            \"And gains day.\\n\",\n            \"\\n\",\n            \"CAMILLO:\\n\",\n            \"I dare this, I thank them, on a made, or else\\n\",\n            \"Is stay at the morning are think about, by eyes\\n\",\n            \"inish moreinateers.' Thisush's.\\n\",\n            \"\\n\",\n            \"KING ED\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 380, time: 760.4552597999573\\n\",\n            \"Actor loss: -825.5812377929688\\n\",\n            \"rets: 0.2534932792186737\\n\",\n            \"The stroke of Norfolk.\\n\",\n            \"\\n\",\n            \"DUKE VINCENTIO:\\n\",\n            \"The statue stay, I they are withdraw to your king,\\n\",\n            \"They hold your majestyire to the found it that\\n\",\n            \"for lead the villain to stir, all love, till hear many\\n\",\n            \"will what I you have fall'd with you.\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 400, time: 799.7820234298706\\n\",\n            \"Actor loss: -680.2034301757812\\n\",\n            \"rets: 0.306072473526001\\n\",\n            \"The must fight and their friends that,\\n\",\n            \"It and letcentio. I love, come, good lords,\\n\",\n            \" Hark, what furthercing him them.\\n\",\n            \"\\n\",\n            \"DUKE VINCENTIO:\\n\",\n            \"You were YourTo give thee--\\n\",\n            \"It is Thomas in Henry forsweness\\n\",\n            \"That many these days\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 420, time: 839.0483448505402\\n\",\n            \"Actor loss: -846.8916015625\\n\",\n            \"rets: 0.25812074542045593\\n\",\n            \"The other of Warwick,\\n\",\n            \"With though it honours, for the See of set\\n\",\n            \"As thou stands by this R weary had won,\\n\",\n            \"Let his heads and glst delight.\\n\",\n            \"\\n\",\n            \"KING HENVOLIO:\\n\",\n            \"I thank thou not ables--My lord; I pray more, the\\n\",\n            \"Sh design,\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 440, time: 878.8783130645752\\n\",\n            \"Actor loss: -863.8115234375\\n\",\n            \"rets: 0.2483484447002411\\n\",\n            \"The grace are be now,\\n\",\n            \"And encounters now going married.\\n\",\n            \"\\n\",\n            \"AONTESC:\\n\",\n            \"AndSo one we king, for heaven thought;\\n\",\n            \"SomeENT give that a honesty neighbour following thine.\\n\",\n            \"\\n\",\n            \"HASTINGS:\\n\",\n            \"If such map with thou hemaniful.\\n\",\n            \"\\n\",\n            \"LADY\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 460, time: 918.7519881725311\\n\",\n            \"Actor loss: -773.82421875\\n\",\n            \"rets: 0.3029620051383972\\n\",\n            \"The victory body's wish you well Warwick.\\n\",\n            \"\\n\",\n            \"First Servant::\\n\",\n            \"My lord, noble nobleself for my holpge!\\n\",\n            \"Served, bloody is this grave; but my study\\n\",\n            \" toweratingond degree unto be drawn appear,\\n\",\n            \"The better.\\n\",\n            \"\\n\",\n            \"KING EDWARD IV:\\n\",\n            \"I reason\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 480, time: 957.8695306777954\\n\",\n            \"Actor loss: -790.4373168945312\\n\",\n            \"rets: 0.2682585120201111\\n\",\n            \"The point of this. You sun,\\n\",\n            \"How fares the vault I, is; itad, seem sm indirectutio:\\n\",\n            \"Heio,\\n\",\n            \"I will one to thy sake to make your tongue.\\n\",\n            \"\\n\",\n            \"HENRY BOLARET:\\n\",\n            \"God you, dead: tell him came?\\n\",\n            \"\\n\",\n            \"S\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 500, time: 997.3224573135376\\n\",\n            \"Actor loss: -603.6134033203125\\n\",\n            \"rets: 0.3471847474575043\\n\",\n            \"The ground is best with all subjects\\n\",\n            \"inn shed, I come itself, such content coats,--\\n\",\n            \"Or, come, a present in things,\\n\",\n            \"And by the morning.\\n\",\n            \"\\n\",\n            \"KING EDWARD IV:\\n\",\n            \"I thank thee, and a word as they have you joy,\\n\",\n            \"have you, to show.\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 520, time: 1036.6428000926971\\n\",\n            \"Actor loss: -643.1966552734375\\n\",\n            \"rets: 0.3397030532360077\\n\",\n            \"The house of mine hand's anger\\n\",\n            \"In an day, my gracious lord.\\n\",\n            \"Come it, so did I hunt to breed friends,\\n\",\n            \"Who unh prominous faults, then of Rome time\\n\",\n            \"Shall reg us, with the tombity witness\\n\",\n            \"In sorrow, here: here are more bed\\n\",\n            \"Be frowns love\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 540, time: 1076.3727550506592\\n\",\n            \"Actor loss: -708.031005859375\\n\",\n            \"rets: 0.3134646415710449\\n\",\n            \"The edge!\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"LUCIO:\\n\",\n            \"Ah, acting you untime,\\n\",\n            \"That I say 'twas: pray they are it\\n\",\n            \"In wo; to carryeech you richer;\\n\",\n            \"Like beasts what fellow? Pray, dear you shall fly you\\n\",\n            \"Yourself to have no such a and true\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 560, time: 1115.688039779663\\n\",\n            \"Actor loss: -682.0416259765625\\n\",\n            \"rets: 0.3094523847103119\\n\",\n            \"The place, deputy\\n\",\n            \"Of England.\\n\",\n            \"\\n\",\n            \"DUKE OF AUMLE:\\n\",\n            \"Where you what stands it? Pray? I'll give you\\n\",\n            \"Even with the churchous daughter of us!\\n\",\n            \"\\n\",\n            \"NORTHUMBERLAND:\\n\",\n            \"Nay, come to our wlockish'd hand,\\n\",\n            \"The\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 580, time: 1155.1704773902893\\n\",\n            \"Actor loss: -584.12646484375\\n\",\n            \"rets: 0.36164161562919617\\n\",\n            \"The price;\\n\",\n            \"And so shall I loveing ignorant have\\n\",\n            \"To husband. send myself are\\n\",\n            \"Sinceives? THR modestyible,\\n\",\n            \"On neither in the fire have done.\\n\",\n            \"\\n\",\n            \"BUCKINGHAM:\\n\",\n            \"Let you your love. I\\n\",\n            \"\\n\",\n            \"ISABELLA:\\n\",\n            \"Good brother, good mad\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 600, time: 1194.873331785202\\n\",\n            \"Actor loss: -517.8790283203125\\n\",\n            \"rets: 0.39595964550971985\\n\",\n            \"The king.\\n\",\n            \"\\n\",\n            \"HENRY PER:\\n\",\n            \"Well, the in York, love.\\n\",\n            \"\\n\",\n            \"VOLUMNIA:\\n\",\n            \"I'll cutign defend my lord.\\n\",\n            \"\\n\",\n            \"MENENIUS:\\n\",\n            \"Go, be prosperous so:\\n\",\n            \"O will-morrow in the father's love.\\n\",\n            \"\\n\",\n            \"ROMEO:\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 620, time: 1234.6517779827118\\n\",\n            \"Actor loss: -632.740966796875\\n\",\n            \"rets: 0.3639276921749115\\n\",\n            \"The nature man would respect drunk:\\n\",\n            \"business my mind receives and fear of his way\\n\",\n            \"As to their proceedings.\\n\",\n            \"\\n\",\n            \"VOLUMNIA:\\n\",\n            \"I would I love for this cap, a tear'd,\\n\",\n            \"But, love him should be not that hath\\n\",\n            \"ighthART you undertake, to go.\\n\",\n            \"\\n\",\n            \"CLA\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 640, time: 1273.8108813762665\\n\",\n            \"Actor loss: -603.2329711914062\\n\",\n            \"rets: 0.37746334075927734\\n\",\n            \"The earth,\\n\",\n            \"to not worse set his knife, a Clarence apprehend,\\n\",\n            \"To be me; while him and Romeo so.\\n\",\n            \"\\n\",\n            \"DUKE OF YORK:\\n\",\n            \"I doeech thee the pleasureier:\\n\",\n            \"Didishedief make Sweet loved him my knowledge!\\n\",\n            \"\\n\",\n            \"DUKE OF YORK:\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 660, time: 1313.7660591602325\\n\",\n            \"Actor loss: -787.6739501953125\\n\",\n            \"rets: 0.288984090089798\\n\",\n            \"The second,\\n\",\n            \"First Senator itself, when I will come there have his\\n\",\n            \"With the princely like clouds.\\n\",\n            \"\\n\",\n            \"DUKE OF YORK:\\n\",\n            \"CAPULETis, we dream I love a doth thy retire.\\n\",\n            \"\\n\",\n            \"RICHHENRY BWiltIO:\\n\",\n            \"O, I come to\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 680, time: 1353.1964864730835\\n\",\n            \"Actor loss: -592.0076293945312\\n\",\n            \"rets: 0.36385422945022583\\n\",\n            \"The point and least'd in casting for our king;\\n\",\n            \"I shall have where he, too enrich,--\\n\",\n            \"Withape: what is 'not there by the state; and all love,\\n\",\n            \"I say he may sit.\\n\",\n            \"\\n\",\n            \"KING RICHARD III:\\n\",\n            \"If she is your longbs. He out\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 700, time: 1392.60701918602\\n\",\n            \"Actor loss: -641.7145385742188\\n\",\n            \"rets: 0.3566742539405823\\n\",\n            \"The course?\\n\",\n            \"\\n\",\n            \"MENENIUS:\\n\",\n            \"Your gracious lord.\\n\",\n            \"3 KING HENRY VI.\\n\",\n            \"\\n\",\n            \"First Senator:\\n\",\n            \"O, lady'st, he is that you?\\n\",\n            \"\\n\",\n            \"GLOUCESTER:\\n\",\n            \"Well, Romeo, where is my lord, I\\n\",\n            \"ilt desire you. spirit\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 720, time: 1432.1645839214325\\n\",\n            \"Actor loss: -631.0171508789062\\n\",\n            \"rets: 0.33802512288093567\\n\",\n            \"The sweet tears.\\n\",\n            \"\\n\",\n            \"DUKE VINCENTIO:\\n\",\n            \"I know if woe?\\n\",\n            \"\\n\",\n            \"DU informIA:\\n\",\n            \"My gracious brother, as protest;\\n\",\n            \"A slave clouds, I hath i'ld. O bird.\\n\",\n            \"\\n\",\n            \"NORTHUMBERLAND:\\n\",\n            \"Be length's child mercy,\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 740, time: 1472.0193173885345\\n\",\n            \"Actor loss: -770.1849365234375\\n\",\n            \"rets: 0.306596577167511\\n\",\n            \"The provost: I thank you, be little man.\\n\",\n            \"\\n\",\n            \"HORTENSIO:\\n\",\n            \"Thou hast my son, speak.\\n\",\n            \"\\n\",\n            \"DUKE OF YORK:\\n\",\n            \"No, it is a happy\\n\",\n            \"ears my nime so writ;\\n\",\n            \"And nowcreator entertainment to behold.\\n\",\n            \"\\n\",\n            \"KING EDWARD IV\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 760, time: 1511.5322704315186\\n\",\n            \"Actor loss: -654.1215209960938\\n\",\n            \"rets: 0.34393274784088135\\n\",\n            \"Theou is A spirit.\\n\",\n            \"\\n\",\n            \"KING EDWARD IV:\\n\",\n            \"Come, I do know it so, most greatness,\\n\",\n            \"Tread, I will proud scene him. My a throne\\n\",\n            \"Well, then, sweet for a debt on war,\\n\",\n            \"Thus Barnards and himself to happy one hark.\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 780, time: 1550.8467333316803\\n\",\n            \"Actor loss: -635.3596801757812\\n\",\n            \"rets: 0.32344189286231995\\n\",\n            \"The bed of the face?\\n\",\n            \"\\n\",\n            \"HENRY BOLINGBROKE:\\n\",\n            \"HereORDoler thou odd!\\n\",\n            \"Of it not too, prefer.\\n\",\n            \"\\n\",\n            \"ARD III:\\n\",\n            \"I come,\\n\",\n            \"Upon them.\\n\",\n            \"\\n\",\n            \"HERGDUCHESS OF AUM:\\n\",\n            \"\\n\",\n            \"NORTHUMBERLAND\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 800, time: 1590.659930229187\\n\",\n            \"Actor loss: -659.7162475585938\\n\",\n            \"rets: 0.3370627164840698\\n\",\n            \"The noble\\n\",\n            \" Pleaseing, for grace him as with the excellent of their\\n\",\n            \"To frame out onte.\\n\",\n            \"\\n\",\n            \"MERBIONDELL:\\n\",\n            \"I know for the orderly spirit to know\\n\",\n            \"WhomENTIO.\\n\",\n            \"Why'll may have free\\n\",\n            \"Of your stronger law of smiles of another give:\\n\",\n            \"O\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 820, time: 1629.6801323890686\\n\",\n            \"Actor loss: -540.1078491210938\\n\",\n            \"rets: 0.3763672113418579\\n\",\n            \"The common of thee.\\n\",\n            \"\\n\",\n            \"QUEEN ELIZABETH:\\n\",\n            \"\\n\",\n            \"DUKE VINCENTIO:\\n\",\n            \"God hold me well I am this noble soul we.\\n\",\n            \"\\n\",\n            \"KING EDWARD IV:\\n\",\n            \"\\n\",\n            \"DUKE VINCENTIO:\\n\",\n            \"Come, I'll blELL you, subjects enough\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 840, time: 1669.0213844776154\\n\",\n            \"Actor loss: -715.7509765625\\n\",\n            \"rets: 0.3233412206172943\\n\",\n            \"The sleep,\\n\",\n            \"And being, doublere our forbear, loving friends,\\n\",\n            \"A prett hath thou grow patient;\\n\",\n            \"Whom thou hastest? scalares,--\\n\",\n            \"\\n\",\n            \"PAULINA:\\n\",\n            \"Will he did your pleasure would please him down.\\n\",\n            \"\\n\",\n            \"LUCIO:\\n\",\n            \" marry much, good lord,\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 860, time: 1708.6020300388336\\n\",\n            \"Actor loss: -698.3497924804688\\n\",\n            \"rets: 0.3224356472492218\\n\",\n            \"The trick can shed. Who'\\n\",\n            \"KATHARENCE:\\n\",\n            \"Darest thou did prove I never usur heart, our unroble spent;\\n\",\n            \"We hope, to rise, love you shall kill enver.\\n\",\n            \" You will them more, I will not speak thee.\\n\",\n            \"\\n\",\n            \"KING EDWARD IV:\\n\",\n            \"A\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 880, time: 1748.3153533935547\\n\",\n            \"Actor loss: -658.4968872070312\\n\",\n            \"rets: 0.31735074520111084\\n\",\n            \"The unt in my lner.\\n\",\n            \"\\n\",\n            \"ComeUMNIA:\\n\",\n            \"He cannot good queen of his\\n\",\n            \"Than it in our exnely days, 'em,\\n\",\n            \"As not too.\\n\",\n            \"\\n\",\n            \"MERCALUS:\\n\",\n            \"'Tis you have peril'd.\\n\",\n            \"There's rose out of the matter snow\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 900, time: 1787.5656242370605\\n\",\n            \"Actor loss: -619.747314453125\\n\",\n            \"rets: 0.3222334086894989\\n\",\n            \"The heads to do thou\\n\",\n            \"To free!Glad the gates is do?\\n\",\n            \"SICINCE of Norfolk to the world to London and mother.\\n\",\n            \"Alearch.\\n\",\n            \"\\n\",\n            \"Second Citizen:\\n\",\n            \"That she of the old bright:\\n\",\n            \"The gates of the house?\\n\",\n            \"\\n\",\n            \"GLOUCESTER:\\n\",\n            \"All\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 920, time: 1827.115246772766\\n\",\n            \"Actor loss: -655.8629760742188\\n\",\n            \"rets: 0.3531707227230072\\n\",\n            \"The gageeth man.\\n\",\n            \"\\n\",\n            \"WARD IV:\\n\",\n            \"And therefore, wife!\\n\",\n            \"\\n\",\n            \"ELBRAY, a slaveies fear is\\n\",\n            \"Do I playraw you to-likely:\\n\",\n            \"I have I love else thy breast?\\n\",\n            \"Hear what is made me: thou shalt stand in 'tis happy tallprovided\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 940, time: 1866.3652894496918\\n\",\n            \"Actor loss: -479.79656982421875\\n\",\n            \"rets: 0.40305882692337036\\n\",\n            \"The worth, that my gracious lords.\\n\",\n            \"\\n\",\n            \"May sheANUS:\\n\",\n            \"Your love, like that will prove.\\n\",\n            \"\\n\",\n            \"PARIS:\\n\",\n            \"Why, for you have pin, luck was born,\\n\",\n            \"To bring you the common, what an blood of the\\n\",\n            \"Even are starkly the gods: I will have it\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 960, time: 1905.8332304954529\\n\",\n            \"Actor loss: -573.760986328125\\n\",\n            \"rets: 0.36464911699295044\\n\",\n            \"The sorrow, yet they will gates.\\n\",\n            \"\\n\",\n            \"RIARINA:\\n\",\n            \"She! This is Lord Hastings\\n\",\n            \"Shall be a well up heaven,\\n\",\n            \"If this sin in himself.\\n\",\n            \"\\n\",\n            \"KING EDWARD IV:\\n\",\n            \"\\n\",\n            \"CLARENCE:\\n\",\n            \"My Lord?\\n\",\n            \"\\n\",\n            \"AUFIDIUS:\\n\",\n            \"What pain\\n\",\n            \"\\n\",\n            \"\\n\",\n            \"iter: 980, time: 1945.2791361808777\\n\",\n            \"Actor loss: -630.5237426757812\\n\",\n            \"rets: 0.34373268485069275\\n\",\n            \"The law, I love his\\n\",\n            \"To win them: I seem cannot be found; 'em me no.\\n\",\n            \"\\n\",\n            \"ISABHCFICHARDINE:\\n\",\n            \"Go, peace, misay, than we must you.\\n\",\n            \"\\n\",\n            \"PARians!\\n\",\n            \"A:\\n\",\n            \"\\n\",\n            \"KING RICHARD III:\\n\",\n            \"I thank\\n\"\n          ]\n        }\n      ]\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [\n        \"# generate from the model\\n\",\n        \"context = torch.zeros((1, 1), dtype=torch.long, device=device)\\n\",\n        \"print(enc.decode(m.generate(context, max_new_tokens=200)[0].tolist()))\"\n      ],\n      \"metadata\": {\n        \"colab\": {\n          \"base_uri\": \"https://localhost:8080/\"\n        },\n        \"id\": \"Mjv63OeQ0bUI\",\n        \"outputId\": \"af51f8a4-a8fc-4861-a005-22ae9eb93695\"\n      },\n      \"execution_count\": 17,\n      \"outputs\": [\n        {\n          \"output_type\": \"stream\",\n          \"name\": \"stdout\",\n          \"text\": [\n            \"!\\n\",\n            \"You be right,\\n\",\n            \"And triumph; am so, my good\\n\",\n            \"to of general; if thou husband\\n\",\n            \"As never sunder best against my blood.\\n\",\n            \"\\n\",\n            \"DUKE VINCENTIO:\\n\",\n            \"Then like a Romansers, art speed at the queen,\\n\",\n            \"Fromagenetings, my tears. But\\n\",\n            \"\\n\",\n            \"ulet:\\n\",\n            \"Which was what I break scarce; where then.\\n\",\n            \"For this we'll bear the dewance of man!\\n\",\n            \"\\n\",\n            \"POLIXENES:\\n\",\n            \"I'll swear soon I will say out a happyker you.\\n\",\n            \"Hear thee.\\n\",\n            \"\\n\",\n            \"DUKE VINCENTIO:\\n\",\n            \"But since our life, Lord, Titus me,\\n\",\n            \"For I love this wed is the battle your man.\\n\",\n            \"\\n\",\n            \"CATESBY:\\n\",\n            \"Farewell you, why.\\n\",\n            \"\\n\",\n            \"CLAUDIO:\\n\",\n            \" if you hear that live Not gone, sent, let our headsarer\\n\",\n            \"wixt the XIant cannot talkings\\n\",\n            \"Should is cl\\n\"\n          ]\n        }\n      ]\n    },\n    {\n      \"cell_type\": \"markdown\",\n      \"source\": [\n        \"1. We trained a GPT model to reproduce Shakespeare\\n\",\n        \"2. We built a reward model by repurposing a Huggingface sentiment classifier\\n\",\n        \"3. We fine tuned the GPT model using reinforcement learning. \\n\",\n        \"4. The model over-optimised the reward so we penalised it for moving too far from a reference model\\n\",\n        \"5. We found it to be far too repetitive and so we added in an entropy bonus to encourage diverse outputs\\n\",\n        \"\\n\",\n        \"---\\n\",\n        \"\\n\",\n        \"\\n\",\n        \"\\n\",\n        \"\\n\",\n        \"\\n\"\n      ],\n      \"metadata\": {\n        \"id\": \"6Na4mH46sFpx\"\n      }\n    },\n    {\n      \"cell_type\": \"code\",\n      \"source\": [],\n      \"metadata\": {\n        \"id\": \"PJrQpMzU9gWJ\"\n      },\n      \"execution_count\": 17,\n      \"outputs\": []\n    }\n  ]\n}"
  },
  {
    "path": "config/config.yaml",
    "content": "IO:\n  out_dir: out\n  eval_interval: 2000\n  log_interval: 1\n  eval_iters: 200\n  eval_only: False # if True, script exits right after the first eval\n  always_save_checkpoint: True # if True, always save a checkpoint after each eval\n  init_from: scratch # 'scratch' or 'resume' or 'gpt2*'\nwandb:\n  wandb_log: False # disabled by default\n  wandb_project: rlhf # 'gpt2'\n  wandb_run_name: gpt2 # 'run' + str(time.time())\ndata:\n  dataset: shakespeare # 'openwebtext', 'shakespeare', 'openai_summarize_tldr'\n  gradient_accumulation_steps: 1 # used to simulate larger batch sizes\n  batch_size: 12 # if gradient_accumulation_steps > 1, this is the micro-batch size\n  block_size: 32\nmodel:\n  n_layer: 2\n  n_head: 2\n  n_embd: 32\n  dropout: 0.0 # for pretraining 0 is good, for finetuning try 0.1+\n  bias: False # do we use bias inside LayerNorm and Linear layers?\noptimizer: # adamw\n  learning_rate: 6.0e-4 # max learning rate\n  max_iters: 600000 # total number of training iterations\n  weight_decay: 1.0e-2\n  beta1: 0.9\n  beta2: 0.95\n  grad_clip: 1.0 # clip gradients at this value, or disable if == 0.0\n  decay_lr: True # whether to decay the learning rate\n  warmup_iters: 2000 # how many steps to warm up for\n  lr_decay_iters: 600000 # should be ~= max_iters per Chinchilla\n  min_lr: 6.0e-5 # minimum learning rate, should be ~= learning_rate/10 per Chinchilla\nDDP:\n  backend: nccl # 'nccl', 'gloo', etc.\nsystem:\n  device: cuda # examples: 'cpu', 'cuda', 'cuda:0', 'cuda:1' etc., or try 'mps' on macbooks\n  dtype: float16 # 'float32', 'bfloat16', or 'float16', the latter will auto implement a GradScaler\n  compile: False # use PyTorch 2.0 to compile the model to be faster\n"
  },
  {
    "path": "config/config_reward.yaml",
    "content": "IO:\n  out_dir: out\n  eval_interval: 500\n  log_interval: 1\n  eval_iters: 100\n  eval_only: False # if True, script exits right after the first eval\n  always_save_checkpoint: True # if True, always save a checkpoint after each eval\n  init_from: resume # 'scratch' or 'resume' or 'gpt2*'\n  init_multihead_from: scratch\n  out_dir_multihead: out_reward # used if restoring multihead\nwandb:\n  wandb_log: True # disabled by default\n  wandb_project: rlhf # 'gpt2'\n  wandb_run_name: gpt2 # 'run' + str(time.time())\ndata:\n  dataset: 'shakespeare' # 'openwebtext', 'shakespeare', 'openai_summarize_tldr'\n  gradient_accumulation_steps: 1 # used to simulate larger batch sizes\n  batch_size: 64 # if gradient_accumulation_steps > 1, this is the micro-batch size\n  block_size: 32\nmodel:\n  n_layer: 2\n  n_head: 2\n  n_embd: 32\n  dropout: 0.0 # for pretraining 0 is good, for finetuning try 0.1+\n  bias: False # do we use bias inside LayerNorm and Linear layers?\noptimizer: # adamw\n  learning_rate: 6.0e-4 # max learning rate\n  max_iters: 600000 # total number of training iterations\n  weight_decay: 1.0e-2\n  beta1: 0.9\n  beta2: 0.95\n  grad_clip: 1.0 # clip gradients at this value, or disable if == 0.0\n  decay_lr: True # whether to decay the learning rate\n  warmup_iters: 2000 # how many steps to warm up for\n  lr_decay_iters: 600000 # should be ~= max_iters per Chinchilla\n  min_lr: 6.0e-5 # minimum learning rate, should be ~= learning_rate/10 per Chinchilla\nDDP:\n  backend: nccl # 'nccl', 'gloo', etc.\nsystem:\n  device: cuda # examples: 'cpu', 'cuda', 'cuda:0', 'cuda:1' etc., or try 'mps' on macbooks\n  dtype: float16 # 'float32', 'bfloat16', or 'float16', the latter will auto implement a GradScaler\n  compile: False # use PyTorch 2.0 to compile the model to be faster\n"
  },
  {
    "path": "config/config_rl.yaml",
    "content": "algorithm:\n  method: gumbel # pg or gumbel\n  hard_code_reward: False # use a learned reward model or hard code reward (latter does not work with Gumbel)\n  separate_reward_model: True # when using a reward model, instantiate it separately rather than share params with LM\n  discrete_reward: True # reward output is 0 or 1 sample if True, otherwise reward is continuous\n  episode_length: 32\nIO:\n  out_dir: out\n  eval_interval: 100\n  log_interval: 1\n  eval_iters: 200\n  eval_only: False # if True, script exits right after the first eval\n  always_save_checkpoint: True # if True, always save a checkpoint after each eval\n  init_from: scratch # 'scratch' or 'resume' or 'gpt2*'\n  init_multihead_from: scratch\n  out_dir_multihead: out_reward # used if restoring multihead\nwandb:\n  wandb_log: False # disabled by default\n  wandb_project: rlhf # 'gpt2'\n  wandb_run_name: gpt2 # 'run' + str(time.time())\ndata:\n  dataset: shakespeare # 'openwebtext', 'shakespeare', 'openai_summarize_tldr'\n  gradient_accumulation_steps: 1 # used to simulate larger batch sizes\n  batch_size: 12 # if gradient_accumulation_steps > 1, this is the micro-batch size\n  block_size: 32\nmodel:\n  n_layer: 2\n  n_head: 2\n  n_embd: 32\n  dropout: 0.0 # for pretraining 0 is good, for finetuning try 0.1+\n  bias: False # do we use bias inside LayerNorm and Linear layers?\noptimizer: # adamw\n  learning_rate: 6.0e-4 # max learning rate\n  max_iters: 600000 # total number of training iterations\n  weight_decay: 1.0e-2\n  beta1: 0.9\n  beta2: 0.95\n  grad_clip: 1.0 # clip gradients at this value, or disable if == 0.0\n  decay_lr: True # whether to decay the learning rate\n  warmup_iters: 2000 # how many steps to warm up for\n  lr_decay_iters: 600000 # should be ~= max_iters per Chinchilla\n  min_lr: 6.0e-5 # minimum learning rate, should be ~= learning_rate/10 per Chinchilla\nDDP:\n  backend: nccl # 'nccl', 'gloo', etc.\nsystem:\n  device: cuda # examples: 'cpu', 'cuda', 'cuda:0', 'cuda:1' etc., or try 'mps' on macbooks\n  dtype: float16 # 'float32', 'bfloat16', or 'float16', the latter will auto implement a GradScaler\n  compile: False # use PyTorch 2.0 to compile the model to be faster\n"
  },
  {
    "path": "config/eval_gpt2.py",
    "content": "# evaluate the base gpt2\n# n_layer=12, n_head=12, n_embd=768\n# 124M parameters\nbatch_size = 8\neval_iters = 500 # use more iterations to get good estimate\neval_only = True\nwandb_log = False\ninit_from = 'gpt2'\n"
  },
  {
    "path": "config/eval_gpt2_large.py",
    "content": "# evaluate the base gpt2\n# n_layer=36, n_head=20, n_embd=1280\n# 774M parameters\nbatch_size = 8\neval_iters = 500 # use more iterations to get good estimate\neval_only = True\nwandb_log = False\ninit_from = 'gpt2-large'\n"
  },
  {
    "path": "config/eval_gpt2_medium.py",
    "content": "# evaluate the base gpt2\n# n_layer=24, n_head=16, n_embd=1024\n# 350M parameters\nbatch_size = 8\neval_iters = 500 # use more iterations to get good estimate\neval_only = True\nwandb_log = False\ninit_from = 'gpt2-medium'\n"
  },
  {
    "path": "config/eval_gpt2_xl.py",
    "content": "# evaluate the base gpt2\n# n_layer=48, n_head=25, n_embd=1600\n# 1558M parameters\nbatch_size = 8\neval_iters = 500 # use more iterations to get good estimate\neval_only = True\nwandb_log = False\ninit_from = 'gpt2-xl'\n"
  },
  {
    "path": "config/finetune_shakespeare.py",
    "content": "import time\n\nout_dir = 'out-shakespeare'\neval_interval = 5\neval_iters = 40\nwandb_log = False # feel free to turn on\nwandb_project = 'shakespeare'\nwandb_run_name = 'ft-' + str(time.time())\n\ndataset = 'shakespeare'\ninit_from = 'gpt2-xl' # this is the largest GPT-2 model\n\n# only save checkpoints if the validation loss improves\nalways_save_checkpoint = False\n\n# the number of examples per iter:\n# 1 batch_size * 32 grad_accum * 1024 tokens = 32,768 tokens/iter\n# shakespeare has 301,966 tokens, so 1 epoch ~= 9.2 iters\nbatch_size = 1\ngradient_accumulation_steps = 32\nmax_iters = 20\n\n# finetune at constant LR\nlearning_rate = 3e-5\ndecay_lr = False\n"
  },
  {
    "path": "config/train_gpt2.py",
    "content": "# config for training GPT-2 (124M) down to very nice loss of ~2.85 on 1 node of 8X A100 40GB\n# launch as the following (e.g. in a screen session) and wait ~5 days:\n# $ torchrun --standalone --nproc_per_node=8 train.py config/train_gpt2.py\n\nwandb_log = True\nwandb_project = 'owt'\nwandb_run_name='gpt2-124M'\n\n# these make the total batch size be ~0.5M\n# 12 batch size * 1024 block size * 5 gradaccum * 8 GPUs = 491,520\nbatch_size = 12\nblock_size = 1024\ngradient_accumulation_steps = 5\n\n# this makes total number of tokens be 300B\nmax_iters = 600000\nlr_decay_iters = 600000\n\n# eval stuff\neval_interval = 1000\neval_iters = 200\nlog_interval = 10\n\n# weight decay\nweight_decay = 1e-1\n"
  },
  {
    "path": "config/train_shakespeare_char.py",
    "content": "# train a miniature character-level shakespeare model\n# good for debugging and playing on macbooks and such\n\nout_dir = 'out-shakespeare-char'\neval_interval = 250 # keep frequent because we'll overfit\neval_iters = 200\nlog_interval = 10 # don't print too too often\n\n# we expect to overfit on this small dataset, so only save when val improves\nalways_save_checkpoint = False\n\nwandb_log = False # override via command line if you like\nwandb_project = 'shakespeare-char'\nwandb_run_name = 'mini-gpt'\n\ndataset = 'shakespeare_char'\nbatch_size = 64\nblock_size = 256 # context of up to 256 previous characters\n\n# baby GPT model :)\nn_layer = 6\nn_head = 6\nn_embd = 384\ndropout = 0.2\n\nlearning_rate = 1e-3 # with baby networks can afford to go a bit higher\nmax_iters = 5000\nlr_decay_iters = 5000 # make equal to max_iters usually\nmin_lr = 1e-4 # learning_rate / 10 usually\nbeta2 = 0.99 # make a bit bigger because number of tokens per iter is small\n\nwarmup_iters = 100 # not super necessary potentially\n\n# on macbook also add\n# device = 'cpu'  # run on cpu only\n# compile = False # do not torch compile the model\n"
  },
  {
    "path": "configurator.py",
    "content": "\"\"\"\nPoor Man's Configurator. Probably a terrible idea. Example usage:\n$ python train.py config/override_file.py --batch_size=32\nthis will first run config/override_file.py, then override batch_size to 32\n\nThe code in this file will be run as follows from e.g. train.py:\n>>> exec(open('configurator.py').read())\n\nSo it's not a Python module, it's just shuttling this code away from train.py\nThe code in this script then overrides the globals()\n\nI know people are not going to love this, I just really dislike configuration\ncomplexity and having to prepend config. to every single variable. If someone\ncomes up with a better simple Python solution I am all ears.\n\"\"\"\n\nimport sys\nfrom ast import literal_eval\n\nfor arg in sys.argv[1:]:\n    if '=' not in arg:\n        # assume it's the name of a config file\n        assert not arg.startswith('--')\n        config_file = arg\n        print(f\"Overriding config with {config_file}:\")\n        with open(config_file) as f:\n            print(f.read())\n        exec(open(config_file).read())\n    else:\n        # assume it's a --key=value argument\n        assert arg.startswith('--')\n        key, val = arg.split('=')\n        key = key[2:]\n        if key in globals():\n            try:\n                # attempt to eval it it (e.g. if bool, number, or etc)\n                attempt = literal_eval(val)\n            except (SyntaxError, ValueError):\n                # if that goes wrong, just use the string\n                attempt = val\n            # ensure the types match ok\n            assert type(attempt) == type(globals()[key])\n            # cross fingers\n            print(f\"Overriding: {key} = {attempt}\")\n            globals()[key] = attempt\n        else:\n            raise ValueError(f\"Unknown config key: {key}\")\n"
  },
  {
    "path": "data/openai_summarize_tldr/prepare.py",
    "content": "# saves the openwebtext dataset to a binary file for training. following was helpful:\n# https://github.com/HazyResearch/flash-attention/blob/main/training/src/datamodules/language_modeling_hf.py\n\nimport os\nfrom tqdm import tqdm\nimport numpy as np\nimport tiktoken\nfrom datasets import load_dataset # huggingface datasets\n\n# number of workers in .map() call\n# good number to use is ~order number of cpu cores // 2\nnum_proc = 16\n\n# takes 54GB in huggingface .cache dir, about 8M documents (8,013,769)\ndataset = load_dataset(\"CarperAI/openai_summarize_tldr\")\n\n\n\n# class TLDRDataset(Dataset):\n#     def __init__(self, split):\n#         self.text = []\n#         dataset = load_dataset(train_path, split=split)\n#         for sample in dataset:\n#             self.text.append(sample[\"prompt\"] + sample[\"label\"])\n#         # if \"valid\" in train_path:\n#         #     self.post_list = self.post_list[0:2000]\n#         # self.tokenizer = tokenizer\n#         # self.max_length = max_length\n#         # self.input_ids = []\n#         # self.attn_masks = []\n\n#     def __len__(self):\n#         return len(self.text)\n\n#     def __getitem__(self, idx):\n#         txt = self.text[idx]\n#         # encodings_dict = self.tokenizer(txt, truncation=True, max_length=self.max_length, padding=\"max_length\")\n#         # input_ids = torch.tensor(encodings_dict[\"input_ids\"])\n#         # attn_masks = torch.tensor(encodings_dict[\"attention_mask\"])\n\n#         return {\n#             # \"input_ids\": input_ids,\n#             # \"attention_mask\": attn_masks,\n#             # \"labels\": input_ids,\n#         }\n\n# dataset = TLDRDataset(split=\"train\")\n\ntrain_text_list = []\nfor sample in dataset['train']:\n    train_text_list.append(sample['prompt'] + sample['label'])\ndataset['train'] = dataset['train'].add_column('text', train_text_list) # add the text column to the train dataset\n\ndataset['val'] = dataset.pop('valid') # rename the valid dataset to val\n\nval_text_list = []\nfor sample in dataset['val']:\n    val_text_list.append(sample['prompt'] + sample['label'])\ndataset['val'] = dataset['val'].add_column('text', val_text_list) # add the text column to the val dataset\n\ndataset.pop('test') # remove the test dataset\n\nsplit_dataset = dataset\n\n# this results in:\n# >>> split_dataset\n# DatasetDict({\n#     train: Dataset({\n#         features: ['prompt', 'label', 'text'],\n#         num_rows: 116722\n#     })\n#     val: Dataset({\n#         features: ['prompt', 'label', 'text'],\n#         num_rows: 6447\n#     })\n# })\n\n# we now want to tokenize the dataset. first define the encoding function (gpt2 bpe)\nenc = tiktoken.get_encoding(\"gpt2\")\ndef process(example):\n    ids = enc.encode_ordinary(example['text']) # encode_ordinary ignores any special tokens\n    ids.append(enc.eot_token) # add the end of text token, e.g. 50256 for gpt2 bpe\n    # note: I think eot should be prepended not appended... hmm. it's called \"eot\" though...\n    out = {'ids': ids, 'len': len(ids)}\n    return out\n\n# tokenize the dataset\ntokenized = split_dataset.map(\n    process,\n    remove_columns=['text','prompt','label'],\n    desc=\"tokenizing the splits\",\n    num_proc=num_proc,\n)\n\n# concatenate all the ids in each dataset into one large file we can use for training\nfor split, dset in tokenized.items():\n    arr_len = np.sum(dset['len'])\n    filename = os.path.join(os.path.dirname(__file__), f'{split}.bin')\n    dtype = np.uint16 # (can do since enc.max_token_value == 50256 is < 2**16)\n    arr = np.memmap(filename, dtype=dtype, mode='w+', shape=(arr_len,))\n\n    print(f\"writing {filename}...\")\n    idx = 0\n    for example in tqdm(dset):\n        arr[idx : idx + example['len']] = example['ids']\n        idx += example['len']\n    arr.flush()\n\n# train.bin is ~17GB, val.bin ~8.5MB\n# train has ~9B tokens (9,035,582,198)\n# val has ~4M tokens (4,434,897)\n\n# to read the bin files later, e.g. with numpy:\n# m = np.memmap('train.bin', dtype=np.uint16, mode='r')\n"
  },
  {
    "path": "data/openwebtext/prepare.py",
    "content": "# saves the openwebtext dataset to a binary file for training. following was helpful:\n# https://github.com/HazyResearch/flash-attention/blob/main/training/src/datamodules/language_modeling_hf.py\n\nimport os\nfrom tqdm import tqdm\nimport numpy as np\nimport tiktoken\nfrom datasets import load_dataset # huggingface datasets\n\n# number of workers in .map() call\n# good number to use is ~order number of cpu cores // 2\nnum_proc = 8\n\n# takes 54GB in huggingface .cache dir, about 8M documents (8,013,769)\ndataset = load_dataset(\"openwebtext\")\n\n# owt by default only contains the 'train' split, so create a test split\nsplit_dataset = dataset[\"train\"].train_test_split(test_size=0.0005, seed=2357, shuffle=True)\nsplit_dataset['val'] = split_dataset.pop('test') # rename the test split to val\n\n# this results in:\n# >>> split_dataset\n# DatasetDict({\n#     train: Dataset({\n#         features: ['text'],\n#         num_rows: 8009762\n#     })\n#     val: Dataset({\n#         features: ['text'],\n#         num_rows: 4007\n#     })\n# })\n\n# we now want to tokenize the dataset. first define the encoding function (gpt2 bpe)\nenc = tiktoken.get_encoding(\"gpt2\")\ndef process(example):\n    ids = enc.encode_ordinary(example['text']) # encode_ordinary ignores any special tokens\n    ids.append(enc.eot_token) # add the end of text token, e.g. 50256 for gpt2 bpe\n    # note: I think eot should be prepended not appended... hmm. it's called \"eot\" though...\n    out = {'ids': ids, 'len': len(ids)}\n    return out\n\n# tokenize the dataset\ntokenized = split_dataset.map(\n    process,\n    remove_columns=['text'],\n    desc=\"tokenizing the splits\",\n    num_proc=num_proc,\n)\n\n# concatenate all the ids in each dataset into one large file we can use for training\nfor split, dset in tokenized.items():\n    arr_len = np.sum(dset['len'])\n    filename = os.path.join(os.path.dirname(__file__), f'{split}.bin')\n    dtype = np.uint16 # (can do since enc.max_token_value == 50256 is < 2**16)\n    arr = np.memmap(filename, dtype=dtype, mode='w+', shape=(arr_len,))\n\n    print(f\"writing {filename}...\")\n    idx = 0\n    for example in tqdm(dset):\n        arr[idx : idx + example['len']] = example['ids']\n        idx += example['len']\n    arr.flush()\n\n# train.bin is ~17GB, val.bin ~8.5MB\n# train has ~9B tokens (9,035,582,198)\n# val has ~4M tokens (4,434,897)\n\n# to read the bin files later, e.g. with numpy:\n# m = np.memmap('train.bin', dtype=np.uint16, mode='r')\n"
  },
  {
    "path": "data/openwebtext/readme.md",
    "content": "\n## openwebtext dataset\n\nafter running `prepare.py` (preprocess) we get:\n\n- train.bin is ~17GB, val.bin ~8.5MB\n- train has ~9B tokens (9,035,582,198)\n- val has ~4M tokens (4,434,897)\n\nthis came from 8,013,769 documents in total.\n\nreferences:\n\n- OpenAI's WebText dataset is discussed in [GPT-2 paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)\n- [OpenWebText](https://skylion007.github.io/OpenWebTextCorpus/) dataset\n"
  },
  {
    "path": "data/shakespeare/prepare.py",
    "content": "import os\nimport requests\nimport tiktoken\nimport numpy as np\n\n# download the tiny shakespeare dataset\ninput_file_path = os.path.join(os.path.dirname(__file__), 'input.txt')\nif not os.path.exists(input_file_path):\n    data_url = 'https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt'\n    with open(input_file_path, 'w') as f:\n        f.write(requests.get(data_url).text)\n\nwith open(input_file_path, 'r') as f:\n    data = f.read()\nn = len(data)\ntrain_data = data[:int(n*0.9)]\nval_data = data[int(n*0.9):]\n\n# encode with tiktoken gpt2 bpe\nenc = tiktoken.get_encoding(\"gpt2\")\ntrain_ids = enc.encode_ordinary(train_data)\nval_ids = enc.encode_ordinary(val_data)\nprint(f\"train has {len(train_ids):,} tokens\")\nprint(f\"val has {len(val_ids):,} tokens\")\n\n# export to bin files\ntrain_ids = np.array(train_ids, dtype=np.uint16)\nval_ids = np.array(val_ids, dtype=np.uint16)\ntrain_ids.tofile(os.path.join(os.path.dirname(__file__), 'train.bin'))\nval_ids.tofile(os.path.join(os.path.dirname(__file__), 'val.bin'))\n\n# train.bin has 301,966 tokens\n# val.bin has 36,059 tokens\n"
  },
  {
    "path": "data/shakespeare/readme.md",
    "content": "\n# tiny shakespeare\n\nTiny shakespeare, of the good old char-rnn fame :)\n\nAfter running `prepare.py`:\n\n- train.bin has 301,966 tokens\n- val.bin has 36,059 tokens\n"
  },
  {
    "path": "data/shakespeare_char/prepare.py",
    "content": "\"\"\"\nPrepare the Shakespeare dataset for character-level language modeling.\nSo instead of encoding with GPT-2 BPE tokens, we just map characters to ints.\nWill save train.bin, val.bin containing the ids, and meta.pkl containing the\nencoder and decoder and some other related info.\n\"\"\"\nimport os\nimport pickle\nimport requests\nimport numpy as np\n\n# download the tiny shakespeare dataset\ninput_file_path = os.path.join(os.path.dirname(__file__), 'input.txt')\nif not os.path.exists(input_file_path):\n    data_url = 'https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt'\n    with open(input_file_path, 'w') as f:\n        f.write(requests.get(data_url).text)\n\nwith open(input_file_path, 'r') as f:\n    data = f.read()\nprint(f\"length of dataset in characters: {len(data):,}\")\n\n# get all the unique characters that occur in this text\nchars = sorted(list(set(data)))\nvocab_size = len(chars)\nprint(\"all the unique characters:\", ''.join(chars))\nprint(f\"vocab size: {vocab_size:,}\")\n\n# create a mapping from characters to integers\nstoi = { ch:i for i,ch in enumerate(chars) }\nitos = { i:ch for i,ch in enumerate(chars) }\ndef encode(s):\n    return [stoi[c] for c in s] # encoder: take a string, output a list of integers\ndef decode(l):\n    ''.join([itos[i] for i in l]) # decoder: take a list of integers, output a string\n\n# create the train and test splits\nn = len(data)\ntrain_data = data[:int(n*0.9)]\nval_data = data[int(n*0.9):]\n\n# encode both to integers\ntrain_ids = encode(train_data)\nval_ids = encode(val_data)\nprint(f\"train has {len(train_ids):,} tokens\")\nprint(f\"val has {len(val_ids):,} tokens\")\n\n# export to bin files\ntrain_ids = np.array(train_ids, dtype=np.uint16)\nval_ids = np.array(val_ids, dtype=np.uint16)\ntrain_ids.tofile(os.path.join(os.path.dirname(__file__), 'train.bin'))\nval_ids.tofile(os.path.join(os.path.dirname(__file__), 'val.bin'))\n\n# save the meta information as well, to help us encode/decode later\nmeta = {\n    'vocab_size': vocab_size,\n    'itos': itos,\n    'stoi': stoi,\n}\nwith open(os.path.join(os.path.dirname(__file__), 'meta.pkl'), 'wb') as f:\n    pickle.dump(meta, f)\n\n# length of dataset in characters:  1115394\n# all the unique characters:\n#  !$&',-.3:;?ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz\n# vocab size: 65\n# train has 1003854 tokens\n# val has 111540 tokens\n"
  },
  {
    "path": "data/shakespeare_char/readme.md",
    "content": "\n# tiny shakespeare, character-level\n\nTiny shakespeare, of the good old char-rnn fame :) Treated on character-level.\n\nAfter running `prepare.py`:\n\n- train.bin has 1,003,854 tokens\n- val.bin has 111,540 tokens\n"
  },
  {
    "path": "model.py",
    "content": "\"\"\"\nFull definition of a GPT Language Model, all of it in this single file.\nReferences:\n1) the official GPT-2 TensorFlow implementation released by OpenAI:\nhttps://github.com/openai/gpt-2/blob/master/src/model.py\n2) huggingface/transformers PyTorch implementation:\nhttps://github.com/huggingface/transformers/blob/main/src/transformers/models/gpt2/modeling_gpt2.py\n\"\"\"\n\nimport math\nimport inspect\nfrom dataclasses import dataclass\n\nimport torch\nimport torch.nn as nn\nfrom torch.nn import functional as F\n\n# @torch.jit.script # good to enable when not using torch.compile, disable when using (our default)\ndef new_gelu(x):\n    \"\"\"\n    Implementation of the GELU activation function currently in Google BERT repo (identical to OpenAI GPT).\n    Reference: Gaussian Error Linear Units (GELU) paper: https://arxiv.org/abs/1606.08415\n    \"\"\"\n    return 0.5 * x * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (x + 0.044715 * torch.pow(x, 3.0))))\n\nclass LayerNorm(nn.Module):\n    \"\"\" LayerNorm but with an optional bias. PyTorch doesn't support simply bias=False \"\"\"\n\n    def __init__(self, ndim, bias):\n        super().__init__()\n        self.weight = nn.Parameter(torch.ones(ndim))\n        self.bias = nn.Parameter(torch.zeros(ndim)) if bias else None\n\n    def forward(self, input):\n        return F.layer_norm(input, self.weight.shape, self.weight, self.bias, 1e-5)\n\nclass CausalSelfAttention(nn.Module):\n\n    def __init__(self, config):\n        super().__init__()\n        assert config.n_embd % config.n_head == 0\n        # key, query, value projections for all heads, but in a batch\n        self.c_attn = nn.Linear(config.n_embd, 3 * config.n_embd, bias=config.bias)\n        # output projection\n        self.c_proj = nn.Linear(config.n_embd, config.n_embd, bias=config.bias)\n        # regularization\n        self.attn_dropout = nn.Dropout(config.dropout)\n        self.resid_dropout = nn.Dropout(config.dropout)\n        self.n_head = config.n_head\n        self.n_embd = config.n_embd\n        self.dropout = config.dropout\n        # flash attention make GPU go brrrrr but support is only in PyTorch nightly and still a bit scary\n        self.flash = hasattr(torch.nn.functional, 'scaled_dot_product_attention') and self.dropout == 0.0\n        if not self.flash:\n            print(\"WARNING: using slow attention. Flash Attention atm needs PyTorch nightly and dropout=0.0\")\n            # causal mask to ensure that attention is only applied to the left in the input sequence\n            self.register_buffer(\"bias\", torch.tril(torch.ones(config.block_size, config.block_size))\n                                        .view(1, 1, config.block_size, config.block_size))\n\n    def forward(self, x):\n        B, T, C = x.size() # batch size, sequence length, embedding dimensionality (n_embd)\n\n        # calculate query, key, values for all heads in batch and move head forward to be the batch dim\n        q, k ,v  = self.c_attn(x).split(self.n_embd, dim=2)\n        k = k.view(B, T, self.n_head, C // self.n_head).transpose(1, 2) # (B, nh, T, hs)\n        q = q.view(B, T, self.n_head, C // self.n_head).transpose(1, 2) # (B, nh, T, hs)\n        v = v.view(B, T, self.n_head, C // self.n_head).transpose(1, 2) # (B, nh, T, hs)\n\n        # causal self-attention; Self-attend: (B, nh, T, hs) x (B, nh, hs, T) -> (B, nh, T, T)\n        if self.flash:\n            # efficient attention using Flash Attention CUDA kernels\n            y = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=None, dropout_p=self.dropout, is_causal=True)\n        else:\n            # manual implementation of attention\n            att = (q @ k.transpose(-2, -1)) * (1.0 / math.sqrt(k.size(-1)))\n            att = att.masked_fill(self.bias[:,:,:T,:T] == 0, float('-inf'))\n            att = F.softmax(att, dim=-1)\n            att = self.attn_dropout(att)\n            y = att @ v # (B, nh, T, T) x (B, nh, T, hs) -> (B, nh, T, hs)\n        y = y.transpose(1, 2).contiguous().view(B, T, C) # re-assemble all head outputs side by side\n\n        # output projection\n        y = self.resid_dropout(self.c_proj(y))\n        return y\n\nclass MLP(nn.Module):\n\n    def __init__(self, config):\n        super().__init__()\n        self.c_fc    = nn.Linear(config.n_embd, 4 * config.n_embd, bias=config.bias)\n        self.c_proj  = nn.Linear(4 * config.n_embd, config.n_embd, bias=config.bias)\n        self.dropout = nn.Dropout(config.dropout)\n\n    def forward(self, x):\n        x = self.c_fc(x)\n        x = new_gelu(x)\n        x = self.c_proj(x)\n        x = self.dropout(x)\n        return x\n\nclass Block(nn.Module):\n\n    def __init__(self, config):\n        super().__init__()\n        self.ln_1 = LayerNorm(config.n_embd, bias=config.bias)\n        self.attn = CausalSelfAttention(config)\n        self.ln_2 = LayerNorm(config.n_embd, bias=config.bias)\n        self.mlp = MLP(config)\n\n    def forward(self, x):\n        x = x + self.attn(self.ln_1(x))\n        x = x + self.mlp(self.ln_2(x))\n        return x\n\n@dataclass\nclass GPTConfig:\n    block_size: int = 1024\n    vocab_size: int = 50304 # GPT-2 vocab_size of 50257, padded up to nearest multiple of 64 for efficiency\n    n_layer: int = 12\n    n_head: int = 12\n    n_embd: int = 768\n    dropout: float = 0.0\n    bias: bool = True # True: bias in Linears and LayerNorms, like GPT-2. False: a bit better and faster\n\nclass GPT(nn.Module):\n\n    def __init__(self, config):\n        super().__init__()\n        assert config.vocab_size is not None\n        assert config.block_size is not None\n        self.config = config\n\n        self.transformer = nn.ModuleDict(dict(\n            wte = nn.Embedding(config.vocab_size, config.n_embd),\n            wpe = nn.Embedding(config.block_size, config.n_embd),\n            drop = nn.Dropout(config.dropout),\n            h = nn.ModuleList([Block(config) for _ in range(config.n_layer)]),\n            ln_f = LayerNorm(config.n_embd, bias=config.bias),\n        ))\n        self.lm_head = nn.Linear(config.n_embd, config.vocab_size, bias=False)\n        # with weight tying when using torch.compile() some warnings get generated:\n        # \"UserWarning: functional_call was passed multiple values for tied weights.\n        # This behavior is deprecated and will be an error in future versions\"\n        # not 100% sure what this is, so far seems to be harmless. TODO investigate\n        self.transformer.wte.weight = self.lm_head.weight # https://paperswithcode.com/method/weight-tying\n\n        # init all weights\n        self.apply(self._init_weights)\n        # apply special scaled init to the residual projections, per GPT-2 paper\n        for pn, p in self.named_parameters():\n            if pn.endswith('c_proj.weight'):\n                torch.nn.init.normal_(p, mean=0.0, std=0.02/math.sqrt(2 * config.n_layer))\n\n        # report number of parameters\n        print(\"number of parameters: %.2fM\" % (self.get_num_params()/1e6,))\n\n    def get_num_params(self, non_embedding=True):\n        \"\"\"\n        Return the number of parameters in the model.\n        For non-embedding count (default), the position embeddings get subtracted.\n        The token embeddings would too, except due to the parameter sharing these\n        params are actually used as weights in the final layer, so we include them.\n        \"\"\"\n        n_params = sum(p.numel() for p in self.parameters())\n        if non_embedding:\n            n_params -= self.transformer.wpe.weight.numel()\n        return n_params\n\n    def _init_weights(self, module):\n        if isinstance(module, nn.Linear):\n            torch.nn.init.normal_(module.weight, mean=0.0, std=0.02)\n            if module.bias is not None:\n                torch.nn.init.zeros_(module.bias)\n        elif isinstance(module, nn.Embedding):\n            torch.nn.init.normal_(module.weight, mean=0.0, std=0.02)\n\n    def forward(self, idx, targets=None):\n        device = idx.device\n        b, t = idx.size()\n        assert t <= self.config.block_size, f\"Cannot forward sequence of length {t}, block size is only {self.config.block_size}\"\n        pos = torch.arange(0, t, dtype=torch.long, device=device).unsqueeze(0) # shape (1, t)\n\n        # forward the GPT model itself\n        tok_emb = self.transformer.wte(idx) # token embeddings of shape (b, t, n_embd)\n        pos_emb = self.transformer.wpe(pos) # position embeddings of shape (1, t, n_embd)\n        x = self.transformer.drop(tok_emb + pos_emb)\n        for block in self.transformer.h:\n            x = block(x)\n        x = self.transformer.ln_f(x)\n\n        if targets is not None:\n            # if we are given some desired targets also calculate the loss\n            logits = self.lm_head(x)\n            loss = F.cross_entropy(logits.view(-1, logits.size(-1)), targets.view(-1), ignore_index=-1)\n        else:\n            # inference-time mini-optimization: only forward the lm_head on the very last position\n            logits = self.lm_head(x[:, [-1], :]) # note: using list [-1] to preserve the time dim\n            loss = None\n\n        return logits, loss\n\n    def crop_block_size(self, block_size):\n        # model surgery to decrease the block size if necessary\n        # e.g. we may load the GPT2 pretrained model checkpoint (block size 1024)\n        # but want to use a smaller block size for some smaller, simpler model\n        assert block_size <= self.config.block_size\n        self.config.block_size = block_size\n        self.transformer.wpe.weight = nn.Parameter(self.transformer.wpe.weight[:block_size])\n        for block in self.transformer.h:\n            block.attn.bias = block.attn.bias[:,:,:block_size,:block_size]\n\n    @classmethod\n    def from_pretrained(cls, model_type, override_args=None):\n        assert model_type in {'gpt2', 'gpt2-medium', 'gpt2-large', 'gpt2-xl'}\n        override_args = override_args or {} # default to empty dict\n        # only dropout can be overridden see more notes below\n        assert all(k == 'dropout' for k in override_args)\n        from transformers import GPT2LMHeadModel\n        print(\"loading weights from pretrained gpt: %s\" % model_type)\n\n        # n_layer, n_head and n_embd are determined from model_type\n        config_args = {\n            'gpt2':         dict(n_layer=12, n_head=12, n_embd=768),  # 124M params\n            'gpt2-medium':  dict(n_layer=24, n_head=16, n_embd=1024), # 350M params\n            'gpt2-large':   dict(n_layer=36, n_head=20, n_embd=1280), # 774M params\n            'gpt2-xl':      dict(n_layer=48, n_head=25, n_embd=1600), # 1558M params\n        }[model_type]\n        print(\"forcing vocab_size=50257, block_size=1024, bias=True\")\n        config_args['vocab_size'] = 50257 # always 50257 for GPT model checkpoints\n        config_args['block_size'] = 1024 # always 1024 for GPT model checkpoints\n        config_args['bias'] = True # always True for GPT model checkpoints\n        # we can override the dropout rate, if desired\n        if 'dropout' in override_args:\n            print(f\"overriding dropout rate to {override_args['dropout']}\")\n            config_args['dropout'] = override_args['dropout']\n        # create a from-scratch initialized minGPT model\n        config = GPTConfig(**config_args)\n        model = GPT(config)\n        sd = model.state_dict()\n        sd_keys = sd.keys()\n        sd_keys = [k for k in sd_keys if not k.endswith('.attn.bias')] # discard this mask / buffer, not a param\n\n        # init a huggingface/transformers model\n        model_hf = GPT2LMHeadModel.from_pretrained(model_type)\n        sd_hf = model_hf.state_dict()\n\n        # copy while ensuring all of the parameters are aligned and match in names and shapes\n        sd_keys_hf = sd_hf.keys()\n        sd_keys_hf = [k for k in sd_keys_hf if not k.endswith('.attn.masked_bias')] # ignore these, just a buffer\n        sd_keys_hf = [k for k in sd_keys_hf if not k.endswith('.attn.bias')] # same, just the mask (buffer)\n        transposed = ['attn.c_attn.weight', 'attn.c_proj.weight', 'mlp.c_fc.weight', 'mlp.c_proj.weight']\n        # basically the openai checkpoints use a \"Conv1D\" module, but we only want to use a vanilla Linear\n        # this means that we have to transpose these weights when we import them\n        assert len(sd_keys_hf) == len(sd_keys), f\"mismatched keys: {len(sd_keys_hf)} != {len(sd_keys)}\"\n        for k in sd_keys_hf:\n            if any(k.endswith(w) for w in transposed):\n                # special treatment for the Conv1D weights we need to transpose\n                assert sd_hf[k].shape[::-1] == sd[k].shape\n                with torch.no_grad():\n                    sd[k].copy_(sd_hf[k].t())\n            else:\n                # vanilla copy over the other parameters\n                assert sd_hf[k].shape == sd[k].shape\n                with torch.no_grad():\n                    sd[k].copy_(sd_hf[k])\n\n        return model\n\n    def configure_optimizers(self, weight_decay, learning_rate, betas, device_type):\n        \"\"\"\n        This long function is unfortunately doing something very simple and is being very defensive:\n        We are separating out all parameters of the model into two buckets: those that will experience\n        weight decay for regularization and those that won't (biases, and layernorm/embedding weights).\n        We are then returning the PyTorch optimizer object.\n        \"\"\"\n\n        # separate out all parameters to those that will and won't experience regularizing weight decay\n        decay = set()\n        no_decay = set()\n        whitelist_weight_modules = (torch.nn.Linear, )\n        blacklist_weight_modules = (torch.nn.LayerNorm, LayerNorm, torch.nn.Embedding)\n        for mn, m in self.named_modules():\n            for pn, p in m.named_parameters():\n                fpn = '%s.%s' % (mn, pn) if mn else pn # full param name\n                # random note: because named_modules and named_parameters are recursive\n                # we will see the same tensors p many many times. but doing it this way\n                # allows us to know which parent module any tensor p belongs to...\n                if pn.endswith('bias'):\n                    # all biases will not be decayed\n                    no_decay.add(fpn)\n                elif pn.endswith('weight') and isinstance(m, whitelist_weight_modules):\n                    # weights of whitelist modules will be weight decayed\n                    decay.add(fpn)\n                elif pn.endswith('weight') and isinstance(m, blacklist_weight_modules):\n                    # weights of blacklist modules will NOT be weight decayed\n                    no_decay.add(fpn)\n\n        # subtle: 'transformer.wte.weight' and 'lm_head.weight' are tied, so they\n        # will appear in the no_decay and decay sets respectively after the above.\n        # In addition, because named_parameters() doesn't return duplicates, it\n        # will only return the first occurence, key'd by 'transformer.wte.weight', below.\n        # so let's manually remove 'lm_head.weight' from decay set. This will include\n        # this tensor into optimization via transformer.wte.weight only, and not decayed.\n        decay.remove('lm_head.weight')\n\n        # validate that we considered every parameter\n        param_dict = {pn: p for pn, p in self.named_parameters()}\n        inter_params = decay & no_decay\n        union_params = decay | no_decay\n        assert len(inter_params) == 0, \"parameters %s made it into both decay/no_decay sets!\" % (str(inter_params), )\n        assert len(param_dict.keys() - union_params) == 0, \"parameters %s were not separated into either decay/no_decay set!\" \\\n                                                    % (str(param_dict.keys() - union_params), )\n\n        # create the pytorch optimizer object\n        optim_groups = [\n            {\"params\": [param_dict[pn] for pn in sorted(list(decay))], \"weight_decay\": weight_decay},\n            {\"params\": [param_dict[pn] for pn in sorted(list(no_decay))], \"weight_decay\": 0.0},\n        ]\n        # new PyTorch nightly has a new 'fused' option for AdamW that is much faster\n        use_fused = (device_type == 'cuda') and ('fused' in inspect.signature(torch.optim.AdamW).parameters)\n        print(f\"using fused AdamW: {use_fused}\")\n        extra_args = dict(fused=True) if use_fused else dict()\n        optimizer = torch.optim.AdamW(optim_groups, lr=learning_rate, betas=betas, **extra_args)\n\n        return optimizer\n\n    def estimate_mfu(self, fwdbwd_per_iter, dt):\n        \"\"\" estimate model flops utilization (MFU) in units of A100 bfloat16 peak FLOPS \"\"\"\n        # first estimate the number of flops we do per iteration.\n        # see PaLM paper Appendix B as ref: https://arxiv.org/abs/2204.02311\n        N = self.get_num_params()\n        cfg = self.config\n        L, H, Q, T = cfg.n_layer, cfg.n_head, cfg.n_embd//cfg.n_head, cfg.block_size\n        flops_per_token = 6*N + 12*L*H*Q*T\n        flops_per_fwdbwd = flops_per_token * T\n        flops_per_iter = flops_per_fwdbwd * fwdbwd_per_iter\n        # express our flops throughput as ratio of A100 bfloat16 peak flops\n        flops_achieved = flops_per_iter * (1.0/dt) # per second\n        flops_promised = 312e12 # A100 GPU bfloat16 peak flops is 312 TFLOPS\n        mfu = flops_achieved / flops_promised\n        return mfu\n\n    @torch.no_grad()\n    def generate(self, idx, max_new_tokens, temperature=1.0, top_k=None):\n        \"\"\"\n        Take a conditioning sequence of indices idx (LongTensor of shape (b,t)) and complete\n        the sequence max_new_tokens times, feeding the predictions back into the model each time.\n        Most likely you'll want to make sure to be in model.eval() mode of operation for this.\n        \"\"\"\n        for _ in range(max_new_tokens):\n            # if the sequence context is growing too long we must crop it at block_size\n            idx_cond = idx if idx.size(1) <= self.config.block_size else idx[:, -self.config.block_size:]\n            # forward the model to get the logits for the index in the sequence\n            logits, _ = self(idx_cond)\n            # pluck the logits at the final step and scale by desired temperature\n            logits = logits[:, -1, :] / temperature\n            # optionally crop the logits to only the top k options\n            if top_k is not None:\n                v, _ = torch.topk(logits, min(top_k, logits.size(-1)))\n                logits[logits < v[:, [-1]]] = -float('Inf')\n            # apply softmax to convert logits to (normalized) probabilities\n            probs = F.softmax(logits, dim=-1)\n            # sample from the distribution\n            idx_next = torch.multinomial(probs, num_samples=1)\n            # append sampled index to the running sequence and continue\n            idx = torch.cat((idx, idx_next), dim=1)\n\n        return idx\n\nclass RLHF(nn.Module):\n    def __init__(self, model, mode, discrete_reward=False):\n        super().__init__()\n        self.model = model\n        self.config = model.config\n\n        # reward model\n        self.n_embd = model.lm_head.in_features\n        self.block_size = model.config.block_size\n        model.policy_head = nn.Linear(model.lm_head.in_features, model.lm_head.out_features, bias=False)\n        self.mode = mode\n        self.discrete_reward = discrete_reward\n        if discrete_reward:\n            model.reward_head = nn.Linear(model.lm_head.in_features, 2, bias=False)\n        else:\n            model.reward_head = nn.Linear(self.n_embd*self.block_size, 1, bias=False)\n    \n    def forward_reward(self, idx, targets=None):\n        device = idx.device\n        b, t = idx.size()\n        assert t <= self.config.block_size, f\"Cannot forward sequence of length {t}, block size is only {self.config.block_size}\"\n        pos = torch.arange(0, t, dtype=torch.long, device=device).unsqueeze(0) # shape (1, t)\n\n        # forward the GPT model itself\n        tok_emb = self.model.transformer.wte(idx) # token embeddings of shape (b, t, n_embd)\n        pos_emb = self.model.transformer.wpe(pos) # position embeddings of shape (1, t, n_embd)\n        x = self.model.transformer.drop(tok_emb + pos_emb)\n        for block in self.model.transformer.h:\n            x = block(x)\n        x = self.model.transformer.ln_f(x)\n\n        rewards = self.model.reward_head(x[:, -1, :])\n\n        if self.discrete_reward:\n            probs = torch.softmax(rewards,1)\n            if targets is not None:\n                # if we are given some desired targets also calculate the loss\n                loss = F.cross_entropy(probs, targets, ignore_index=-1)\n            else:\n                loss = None\n            return probs, loss\n        else:\n            return rewards\n    \n    def forward(self, idx, targets=None):\n        if self.mode == 'reward':\n            return self.forward_reward(idx, targets)\n        else:\n            return self.model(idx, targets)\n     \n    def generate(self, idx, max_new_tokens, device, block_size, use_reference=True, reward_model=None, hard_code_reward=True):\n        # idx is (B, T) array of indices in the current context\n        log_probs = torch.tensor([]).to(device)\n        log_probs_ref = torch.tensor([]).to(device)\n        values = torch.tensor([]).to(device)\n\n        idx_cond_all = torch.zeros((idx.shape[0], block_size, max_new_tokens)).to(device)\n        values_all = torch.zeros((idx.shape[0], max_new_tokens)).to(device)\n        actions_all = torch.zeros((idx.shape[0], max_new_tokens)).to(device)\n        rewards_all = torch.zeros((idx.shape[0],)).to(device)\n        log_probs_all = torch.zeros((idx.shape[0], max_new_tokens)).to(device)\n        advantages_all = torch.zeros((idx.shape[0], max_new_tokens)).to(device)\n        returns_all = torch.zeros((idx.shape[0], max_new_tokens)).to(device)\n        gamma = 1\n        lam = 1\n\n        # TODO: Critic, PPO\n        for i in range(max_new_tokens):\n            # crop idx to the last block_size tokens\n            # block_size = 256\n            idx_cond = idx[:, -block_size:]\n\n            # get the predictions\n            logits, loss = self(idx_cond)\n\n            # focus only on the last time step\n            logits = logits[:, -1, :] # becomes (B, C)\n            # apply softmax to get probabilities\n            \n            probs_next = F.softmax(logits, dim=-1) # (B, C)\n            # sample from the distribution\n            idx_next = torch.multinomial(probs_next, num_samples=1) # (B, 1)\n\n            probs_idx_next = torch.gather(probs_next, 1, idx_next)\n            log_probs_idx_next = torch.log(probs_idx_next)\n            log_probs = torch.cat((log_probs, log_probs_idx_next), dim=1)\n            \n            if use_reference:\n                logits_ref, _ = self.model(idx_cond)\n                logits_ref = logits_ref[:, -1, :] # becomes (B, C)\n                probs_ref_next = F.softmax(logits_ref, dim=-1) # (B, C)\n                probs_ref_idx_next = torch.gather(probs_ref_next, 1, idx_next)\n                log_probs_ref_idx_next = torch.log(probs_ref_idx_next)\n                log_probs_ref = torch.cat((log_probs_ref, log_probs_ref_idx_next), dim=1)\n            \n            # append sampled index to the running sequence\n            idx = torch.cat((idx, idx_next), dim=1) # (B, T+1)\n            \n\n            if i == max_new_tokens-1:\n                states = idx[:,-max_new_tokens:]\n                if hard_code_reward: \n                    # simple test where reward for outputting the letter 'z' (89)\n                    rewards = torch.zeros_like(states, dtype=torch.float16)\n                    rewards[states==89] = 1.0\n                    rewards = torch.sum(rewards, 1, keepdim=True)\n                    rewards[rewards > 1] = 1\n\n                else:\n                    if self.discrete_reward:\n                        rewards = reward_model.forward_reward(torch.tensor(states))[0][:,1].unsqueeze(-1)\n                    else:\n                        rewards = reward_model.forward_reward(torch.tensor(states))\n                    \n\n                for t in reversed(range(max_new_tokens)):\n                    if t == max_new_tokens - 1:\n                        # value at last state is 0\n                        delta = rewards[:].squeeze() - values_all[:, t]\n                        advantages_all[:, t] = delta\n                        # returns_all[:, t] = rewards[:]\n                    else:\n                        # rewards can only be non-zero at the last state\n                        delta = gamma * values_all[:, t + 1] - values_all[:, t]\n                        advantages_all[:, t] = delta + gamma * lam * advantages_all[:, t + 1]\n                        # returns_all[:, t] += gamma * returns_all[:, t + 1]\n\n                    \n            \n        return idx, log_probs[:,-max_new_tokens:], log_probs_ref[:,-max_new_tokens:], rewards, advantages_all\n    \n    def generate_gumbel(self, idx, max_new_tokens, device, block_size, reward_model, use_reference=True):\n        \n        onehot = torch.tensor([]).to(device)\n        for i in range(max_new_tokens):\n            # crop idx to the last block_size tokens\n            # block_size = 256\n            idx_cond = idx[:, -block_size:]\n\n            # get the predictions\n            logits, loss = self(idx_cond)\n\n            # focus only on the last time step\n            logits = logits[:, -1, :] # becomes (B, C)\n\n            \n            #gumbel sample\n            idx_next, onehot_next = self.gumbel_softmax(logits, tau=1, device=idx.device)\n\n            # append sampled index to the running sequence\n            idx = torch.cat((idx, idx_next), dim=1) # (B, T+1)\n\n            onehot = torch.cat((onehot, onehot_next), dim=2) # (B, T+1)\n\n            if i == max_new_tokens-1:\n                if self.discrete_reward:\n                    rewards = reward_model.forward_reward_gumbel(onehot)[0][:,1].unsqueeze(-1)\n                else:\n                    rewards = reward_model.forward_reward_gumbel(onehot)\n\n        return idx[:,-max_new_tokens:], rewards\n\n   \n    # modified for PyTorch from https://github.com/ericjang/gumbel-softmax/blob/master/Categorical%20VAE.ipynb\n    def sample_gumbel(self, shape, eps=1e-20):\n        \"\"\"Sample from Gumbel(0, 1)\"\"\"\n        U = torch.distributions.Uniform(0,1).sample(shape)\n        return -torch.log(-torch.log(U + eps) + eps)\n\n    # modified for PyTorch from https://github.com/ericjang/gumbel-softmax/blob/master/Categorical%20VAE.ipynb\n    def gumbel_softmax_sample(self, logits, tau, device, dim=1):\n        \"\"\" Draw a sample from the Gumbel-Softmax distribution\"\"\"\n        y = logits + self.sample_gumbel(logits.shape).to(device)\n        return F.softmax(y / tau, dim=dim)\n\n    def gumbel_softmax(self, logits, tau, device):\n        gumbel_sample = self.gumbel_softmax_sample(logits, tau, device)\n\n        # Alternatively could try\n        # probs = F.softmax(gumbel_sample, dim=-1)\n        # idx_next = torch.multinomial(probs, num_samples=1)\n\n        idx_next = gumbel_sample.max(-1, keepdim=True)[1]\n        onehot_idx_next = torch.nn.functional.one_hot(idx_next, num_classes=logits.shape[1]).squeeze()\n        y = (onehot_idx_next-gumbel_sample).detach() + gumbel_sample\n        idx_next_from_y = torch.argmax(y, dim=1).unsqueeze(-1)\n        return idx_next_from_y, y.unsqueeze(-1)\n\n    def forward_reward_gumbel(self, onehots, idx=None, targets=None):\n        # (embd, vocab) @ (vocab, embd) = (embd,embd)\n        \n        device = onehots.device\n        t = onehots.shape[2]\n        b = onehots.shape[0]\n        # b, t = idx.size()\n        # assert t <= self.config.block_size, f\"Cannot forward sequence of length {t}, block size is only {self.config.block_size}\"\n        pos = torch.arange(0, t, dtype=torch.long, device=device).unsqueeze(0) # shape (1, t)\n\n        # forward the GPT model itself\n        # tok_emb = self.model.transformer.wte(idx) # token embeddings of shape (b, t, n_embd)\n        tok_emb = (self.model.transformer.wte.weight.T @ onehots)\n        tok_emb = torch.transpose(tok_emb, 1, 2)\n\n        if idx is not None:\n            tok_emb2 = self.model.transformer.wte(idx) # token embeddings of shape (b, t, n_embd)\n            assert torch.all(tok_emb == tok_emb2)\n        pos_emb = self.model.transformer.wpe(pos) # position embeddings of shape (1, t, n_embd)\n        x = self.model.transformer.drop(tok_emb + pos_emb)\n        for block in self.model.transformer.h:\n            x = block(x)\n        x = self.model.transformer.ln_f(x)\n\n        rewards = self.model.reward_head(x[:, -1, :])\n\n        if self.discrete_reward:\n            probs = torch.softmax(rewards,1)\n            if targets is not None:\n                # if we are given some desired targets also calculate the loss\n                loss = F.cross_entropy(rewards, targets, ignore_index=-1)\n            else:\n                loss = None\n            return probs, loss\n        else:\n            return rewards"
  },
  {
    "path": "requirements.txt",
    "content": "tiktoken\nnumpy\ncontextlib\ntorch\nwandb\n"
  },
  {
    "path": "sample.py",
    "content": "\"\"\"\nSample from a trained model\n\"\"\"\nimport os\nimport pickle\nfrom contextlib import nullcontext\nimport torch\nimport tiktoken\nfrom model import GPTConfig, GPT\n\n# -----------------------------------------------------------------------------\ninit_from = 'resume' # either 'resume' (from an out_dir) or a gpt2 variant (e.g. 'gpt2-xl')\nout_dir = 'out' # ignored if init_from is not 'resume'\nstart = \"TITLE\" # or \"<|endoftext|>\" or etc. Can also specify a file, use as: \"FILE:prompt.txt\"\nnum_samples = 5 # number of samples to draw\nmax_new_tokens = 500 # number of tokens generated in each sample\ntemperature = 0.8 # 1.0 = no change, < 1.0 = less random, > 1.0 = more random, in predictions\ntop_k = 200 # retain only the top_k most likely tokens, clamp others to have 0 probability\nseed = 1337\ndevice = 'cpu' # examples: 'cpu', 'cuda', 'cuda:0', 'cuda:1', etc.\ndtype = 'float16' # 'float32' or 'bfloat16' or 'float16'\ncompile = False # use PyTorch 2.0 to compile the model to be faster\nexec(open('configurator.py').read()) # overrides from command line or config file\n# -----------------------------------------------------------------------------\n\ntorch.manual_seed(seed)\ntorch.cuda.manual_seed(seed)\ntorch.backends.cuda.matmul.allow_tf32 = True # allow tf32 on matmul\ntorch.backends.cudnn.allow_tf32 = True # allow tf32 on cudnn\ndevice_type = 'cuda' if 'cuda' in device else 'cpu' # for later use in torch.autocast\nptdtype = {'float32': torch.float32, 'bfloat16': torch.bfloat16, 'float16': torch.float16}[dtype]\nctx = nullcontext() if device_type == 'cpu' else torch.amp.autocast(device_type=device_type, dtype=ptdtype)\n\n# model\nif init_from == 'resume':\n    # init from a model saved in a specific directory\n    ckpt_path = os.path.join(out_dir, 'ckpt.pt')\n    checkpoint = torch.load(ckpt_path, map_location=device)\n    gptconf = GPTConfig(**checkpoint['model_args'])\n    model = GPT(gptconf)\n    state_dict = checkpoint['model']\n    unwanted_prefix = '_orig_mod.'\n    for k,v in list(state_dict.items()):\n        if k.startswith(unwanted_prefix):\n            state_dict[k[len(unwanted_prefix):]] = state_dict.pop(k)\n    model.load_state_dict(state_dict)\nelif init_from.startswith('gpt2'):\n    # init from a given GPT-2 model\n    model = GPT.from_pretrained(init_from, dict(dropout=0.0))\n\nmodel.eval()\nmodel.to(device)\nif compile:\n    model = torch.compile(model) # requires PyTorch 2.0 (optional)\n\n# look for the meta pickle in case it is available in the dataset folder\nload_meta = False\nif init_from == 'resume' and 'config' in checkpoint and 'dataset' in checkpoint['config']: # older checkpoints might not have these...\n    meta_path = os.path.join('data', checkpoint['config']['dataset'], 'meta.pkl')\n    load_meta = os.path.exists(meta_path)\nif load_meta:\n    print(f\"Loading meta from {meta_path}...\")\n    with open(meta_path, 'rb') as f:\n        meta = pickle.load(f)\n    # TODO want to make this more general to arbitrary encoder/decoder schemes\n    stoi, itos = meta['stoi'], meta['itos']\n    encode = lambda s: [stoi[c] for c in s]\n    decode = lambda l: ''.join([itos[i] for i in l])\nelse:\n    # ok let's assume gpt-2 encodings by default\n    print(\"No meta.pkl found, assuming GPT-2 encodings...\")\n    enc = tiktoken.get_encoding(\"gpt2\")\n    encode = lambda s: enc.encode(s, allowed_special={\"<|endoftext|>\"})\n    decode = lambda l: enc.decode(l)\n\n# encode the beginning of the prompt\nif start.startswith('FILE:'):\n    with open(start[5:], 'r', encoding='utf-8') as f:\n        start = f.read()\nstart_ids = encode(start)\nx = (torch.tensor(start_ids, dtype=torch.long, device=device)[None, ...])\n\n# run generation\nwith torch.no_grad():\n    with ctx:\n        for k in range(num_samples):\n            y = model.generate(x, max_new_tokens, temperature=temperature, top_k=top_k)\n            print(decode(y[0].tolist()))\n            print('---------------')\n"
  },
  {
    "path": "scaling_laws.ipynb",
    "content": "{\n \"cells\": [\n  {\n   \"attachments\": {},\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"Reproducing some scaling laws results from [Chinchilla](https://arxiv.org/pdf/2203.15556.pdf). Can't get the numbers to match exactly, but can still be used as a rough guide to help determine compute-optimal models. Also contains related utilities for calculating flops and param counts.\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 1,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"import matplotlib.pyplot as plt\\n\",\n    \"import numpy as np\\n\",\n    \"import pandas as pd\\n\",\n    \"%matplotlib inline\"\n   ]\n  },\n  {\n   \"attachments\": {},\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"## params\\n\",\n    \"\\n\",\n    \"First some parameter calculations:\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 2,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"data\": {\n      \"text/plain\": [\n       \"123.653376\"\n      ]\n     },\n     \"execution_count\": 2,\n     \"metadata\": {},\n     \"output_type\": \"execute_result\"\n    }\n   ],\n   \"source\": [\n    \"def gpt_params(seq_len, vocab_size, d_model, num_heads, num_layers):\\n\",\n    \"    \\\"\\\"\\\" Given GPT config calculate total number of parameters \\\"\\\"\\\"\\n\",\n    \"    ffw_size = 4*d_model # in GPT the number of intermediate features is always 4*d_model\\n\",\n    \"    # token and position embeddings\\n\",\n    \"    embeddings = d_model * vocab_size + d_model * seq_len\\n\",\n    \"    # transformer blocks\\n\",\n    \"    attention = 3*d_model**2 + 3*d_model # weights and biases\\n\",\n    \"    attproj = d_model**2 + d_model\\n\",\n    \"    ffw = d_model*(ffw_size) + ffw_size\\n\",\n    \"    ffwproj = ffw_size*d_model + d_model\\n\",\n    \"    layernorms = 2*2*d_model\\n\",\n    \"    # dense\\n\",\n    \"    ln_f = 2*d_model\\n\",\n    \"    dense = d_model*vocab_size # note: no bias here\\n\",\n    \"    # note: embeddings are not included in the param count!\\n\",\n    \"    total_params = num_layers*(attention + attproj + ffw + ffwproj + layernorms) + ln_f + dense\\n\",\n    \"    return total_params\\n\",\n    \"\\n\",\n    \"gpt2 = dict(seq_len = 1024, vocab_size = 50257, d_model = 768, num_heads = 12, num_layers = 12)\\n\",\n    \"gpt_params(**gpt2)/1e6\"\n   ]\n  },\n  {\n   \"attachments\": {},\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"OpenAI reports gpt2 (small) as having 124M params, so this is a match. Also, loading the OpenAI weights into nanoGPT and then calling `model.parameters()` exactly matches the above number and verifies the implementation. Now Chinchilla parameters:\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 3,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"def chinchilla_params(seq_len, vocab_size, d_model, num_heads, num_layers, ffw_size):\\n\",\n    \"    \\\"\\\"\\\" Parameters in the Chinchilla models. Unlike GPT they use relative positional embeddings. \\\"\\\"\\\"\\n\",\n    \"    # token embeddings only\\n\",\n    \"    embeddings = d_model * vocab_size\\n\",\n    \"    # transformer blocks\\n\",\n    \"    attention = 3*d_model**2 + 3*d_model # weights and biases\\n\",\n    \"    relative_pos = d_model**2 + 2*d_model # relative keys, content bias, relative bias\\n\",\n    \"    attproj = d_model**2 + d_model\\n\",\n    \"    ffw = d_model*ffw_size + ffw_size\\n\",\n    \"    ffwproj = ffw_size*d_model + d_model\\n\",\n    \"    layernorms = 2*2*d_model\\n\",\n    \"    # dense\\n\",\n    \"    ln_f = 2*d_model\\n\",\n    \"    dense = d_model*vocab_size # note: no bias here\\n\",\n    \"    # note: embeddings are not included in the param count!\\n\",\n    \"    total_params = num_layers*(attention + relative_pos + attproj + ffw + ffwproj + layernorms) + ln_f + dense\\n\",\n    \"    return total_params\\n\",\n    \"\\n\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 4,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"data\": {\n      \"text/plain\": [\n       \"[44000000.0, 512, 2048, 64, 8, 8]\"\n      ]\n     },\n     \"execution_count\": 4,\n     \"metadata\": {},\n     \"output_type\": \"execute_result\"\n    }\n   ],\n   \"source\": [\n    \"# Load in all the 50 Chinchilla models on the last page of the paper\\n\",\n    \"import json\\n\",\n    \"chinchilla_models_txt = '[[44000000.0, 512, 2048, 64, 8, 8], [57000000.0, 576, 2304, 64, 9, 9], [74000000.0, 640, 2560, 64, 10, 10], [90000000.0, 640, 2560, 64, 10, 13], [106000000.0, 640, 2560, 64, 10, 16], [117000000.0, 768, 3072, 64, 12, 12], [140000000.0, 768, 3072, 64, 12, 15], [163000000.0, 768, 3072, 64, 12, 18], [175000000.0, 896, 3584, 64, 14, 14], [196000000.0, 896, 3584, 64, 14, 16], [217000000.0, 896, 3584, 64, 14, 18], [251000000.0, 1024, 4096, 64, 16, 16], [278000000.0, 1024, 4096, 64, 16, 18], [306000000.0, 1024, 4096, 64, 16, 20], [425000000.0, 1280, 5120, 128, 10, 18], [489000000.0, 1280, 5120, 128, 10, 21], [509000000.0, 1408, 5632, 128, 11, 18], [552000000.0, 1280, 5120, 128, 10, 24], [587000000.0, 1408, 5632, 128, 11, 21], [632000000.0, 1536, 6144, 128, 12, 19], [664000000.0, 1408, 5632, 128, 11, 24], [724000000.0, 1536, 6144, 128, 12, 22], [816000000.0, 1536, 6144, 128, 12, 25], [893000000.0, 1792, 7168, 128, 14, 20], [1018000000.0, 1792, 7168, 128, 14, 23], [1143000000.0, 1792, 7168, 128, 14, 26], [1266000000.0, 2048, 8192, 128, 16, 22], [1424000000.0, 2176, 8704, 128, 17, 22], [1429000000.0, 2048, 8192, 128, 16, 25], [1593000000.0, 2048, 8192, 128, 16, 28], [1609000000.0, 2176, 8704, 128, 17, 25], [1731000000.0, 2304, 9216, 128, 18, 24], [1794000000.0, 2176, 8704, 128, 17, 28], [2007000000.0, 2304, 9216, 128, 18, 28], [2283000000.0, 2304, 9216, 128, 18, 32], [2298000000.0, 2560, 10240, 128, 20, 26], [2639000000.0, 2560, 10240, 128, 20, 30], [2980000000.0, 2560, 10240, 128, 20, 34], [3530000000.0, 2688, 10752, 128, 22, 36], [3802000000.0, 2816, 11264, 128, 22, 36], [4084000000.0, 2944, 11776, 128, 22, 36], [4516000000.0, 3072, 12288, 128, 24, 36], [6796000000.0, 3584, 14336, 128, 28, 40], [9293000000.0, 4096, 16384, 128, 32, 42], [11452000000.0, 4352, 17408, 128, 32, 47], [12295000000.0, 4608, 18432, 128, 36, 44], [12569000000.0, 4608, 18432, 128, 32, 47], [13735000000.0, 4864, 19456, 128, 32, 47], [14940000000.0, 4992, 19968, 128, 32, 49], [16183000000.0, 5120, 20480, 128, 40, 47]]'\\n\",\n    \"chilchilla_models = json.loads(chinchilla_models_txt) # all 50 models\\n\",\n    \"chilchilla_models[0] # tuples of params, d_model, ffw_size, kv_size, n_heads, n_layers from Table A9\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 5,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"name\": \"stdout\",\n     \"output_type\": \"stream\",\n     \"text\": [\n      \"our estimated params: 12296.1623M, chinchilla params: 12295.0000M, d_model: 4608, n_heads: 36, n_layers: 44\\n\",\n      \"our estimated params: 13124.4826M, chinchilla params: 12569.0000M, d_model: 4608, n_heads: 32, n_layers: 47\\n\",\n      \"our estimated params: 14614.4279M, chinchilla params: 13735.0000M, d_model: 4864, n_heads: 32, n_layers: 47\\n\",\n      \"our estimated params: 16037.5039M, chinchilla params: 14940.0000M, d_model: 4992, n_heads: 32, n_layers: 49\\n\",\n      \"our estimated params: 16184.4582M, chinchilla params: 16183.0000M, d_model: 5120, n_heads: 40, n_layers: 47\\n\"\n     ]\n    }\n   ],\n   \"source\": [\n    \"for m in chilchilla_models[-5:]: # only print last 5 models of the table\\n\",\n    \"    p, d, f, k, h, l = m\\n\",\n    \"    nparams = chinchilla_params(seq_len = 1024, vocab_size = 32000, d_model = d, num_heads = h, num_layers = l, ffw_size=f)\\n\",\n    \"    print(f\\\"our estimated params: {nparams/1e6:.4f}M, chinchilla params: {p/1e6:.4f}M, d_model: {d}, n_heads: {h}, n_layers: {l}\\\")\"\n   ]\n  },\n  {\n   \"attachments\": {},\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"We are almost able to reproduce the parameter counts for the Chinchilla models.\\n\",\n    \"\\n\",\n    \"Now turning to FLOPs:\"\n   ]\n  },\n  {\n   \"attachments\": {},\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"## flops\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 6,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"def chinchilla_flops(seq_len, vocab_size, d_model, num_heads, num_layers, ffw_size):\\n\",\n    \"    \\\"\\\"\\\" \\n\",\n    \"    Calculate total number of FLOPs, see Chinchilla \\n\",\n    \"    paper Appendix F as reference: https://arxiv.org/pdf/2203.15556.pdf\\n\",\n    \"    \\\"\\\"\\\" \\n\",\n    \"    key_size = d_model // num_heads\\n\",\n    \"\\n\",\n    \"    # embeddings\\n\",\n    \"    embeddings = 2 * seq_len * vocab_size * d_model\\n\",\n    \"\\n\",\n    \"    # attention\\n\",\n    \"    # key, query, value projections\\n\",\n    \"    attention = 2 * 3 * seq_len * d_model * (key_size * num_heads)\\n\",\n    \"    # key @ query logits\\n\",\n    \"    attlogits = 2 * seq_len * seq_len * (key_size * num_heads)\\n\",\n    \"    # softmax\\n\",\n    \"    attsoftmax = 3 * num_heads * seq_len * seq_len # 3* is for subtract (max), exp, divide (?)\\n\",\n    \"    # softmax @ value reductions\\n\",\n    \"    attvalue = 2 * seq_len * seq_len * (key_size * num_heads)\\n\",\n    \"    # final linear\\n\",\n    \"    attlinear = 2 * seq_len * (key_size * num_heads) * d_model\\n\",\n    \"    att = attention + attlogits + attsoftmax + attvalue + attlinear\\n\",\n    \"    # feed forward\\n\",\n    \"    dense = 2 * seq_len * (d_model * ffw_size + d_model * ffw_size)\\n\",\n    \"\\n\",\n    \"    # logits\\n\",\n    \"    logits = 2 * seq_len * d_model * vocab_size\\n\",\n    \"    \\n\",\n    \"    # this is what you'd expect:\\n\",\n    \"    # forward_flops = embeddings + num_layers * (att + dense) + logits\\n\",\n    \"    # but:\\n\",\n    \"    # per author correspondence apparently there is typo in the paper,\\n\",\n    \"    # they do not count embeddings and logits to repro table 4. So instead:\\n\",\n    \"    forward_flops = num_layers * (att + dense)\\n\",\n    \"    backward_flops = 2 * forward_flops # as in Kaplan et al. 2020\\n\",\n    \"    total_flops = forward_flops + backward_flops\\n\",\n    \"\\n\",\n    \"    return total_flops\\n\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 7,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"data\": {\n      \"text/html\": [\n       \"<div>\\n\",\n       \"<style scoped>\\n\",\n       \"    .dataframe tbody tr th:only-of-type {\\n\",\n       \"        vertical-align: middle;\\n\",\n       \"    }\\n\",\n       \"\\n\",\n       \"    .dataframe tbody tr th {\\n\",\n       \"        vertical-align: top;\\n\",\n       \"    }\\n\",\n       \"\\n\",\n       \"    .dataframe thead th {\\n\",\n       \"        text-align: right;\\n\",\n       \"    }\\n\",\n       \"</style>\\n\",\n       \"<table border=\\\"1\\\" class=\\\"dataframe\\\">\\n\",\n       \"  <thead>\\n\",\n       \"    <tr style=\\\"text-align: right;\\\">\\n\",\n       \"      <th></th>\\n\",\n       \"      <th>seq_len</th>\\n\",\n       \"      <th>vocab_size</th>\\n\",\n       \"      <th>d_model</th>\\n\",\n       \"      <th>num_heads</th>\\n\",\n       \"      <th>num_layers</th>\\n\",\n       \"      <th>ffw_size</th>\\n\",\n       \"      <th>N</th>\\n\",\n       \"      <th>F</th>\\n\",\n       \"      <th>approx_flops</th>\\n\",\n       \"      <th>chinch_flops</th>\\n\",\n       \"      <th>ratio</th>\\n\",\n       \"    </tr>\\n\",\n       \"  </thead>\\n\",\n       \"  <tbody>\\n\",\n       \"    <tr>\\n\",\n       \"      <th>0</th>\\n\",\n       \"      <td>2048</td>\\n\",\n       \"      <td>32000</td>\\n\",\n       \"      <td>640</td>\\n\",\n       \"      <td>10</td>\\n\",\n       \"      <td>10</td>\\n\",\n       \"      <td>2560</td>\\n\",\n       \"      <td>73825280</td>\\n\",\n       \"      <td>929877196800</td>\\n\",\n       \"      <td>907165040640</td>\\n\",\n       \"      <td>9.298772e+11</td>\\n\",\n       \"      <td>1.025036</td>\\n\",\n       \"    </tr>\\n\",\n       \"    <tr>\\n\",\n       \"      <th>1</th>\\n\",\n       \"      <td>2048</td>\\n\",\n       \"      <td>32000</td>\\n\",\n       \"      <td>1024</td>\\n\",\n       \"      <td>16</td>\\n\",\n       \"      <td>20</td>\\n\",\n       \"      <td>4096</td>\\n\",\n       \"      <td>305707008</td>\\n\",\n       \"      <td>4135248199680</td>\\n\",\n       \"      <td>3756527714304</td>\\n\",\n       \"      <td>4.135248e+12</td>\\n\",\n       \"      <td>1.100817</td>\\n\",\n       \"    </tr>\\n\",\n       \"    <tr>\\n\",\n       \"      <th>2</th>\\n\",\n       \"      <td>2048</td>\\n\",\n       \"      <td>32000</td>\\n\",\n       \"      <td>1280</td>\\n\",\n       \"      <td>10</td>\\n\",\n       \"      <td>24</td>\\n\",\n       \"      <td>5120</td>\\n\",\n       \"      <td>552604160</td>\\n\",\n       \"      <td>7353453772800</td>\\n\",\n       \"      <td>6790399918080</td>\\n\",\n       \"      <td>7.353454e+12</td>\\n\",\n       \"      <td>1.082919</td>\\n\",\n       \"    </tr>\\n\",\n       \"    <tr>\\n\",\n       \"      <th>3</th>\\n\",\n       \"      <td>2048</td>\\n\",\n       \"      <td>32000</td>\\n\",\n       \"      <td>1792</td>\\n\",\n       \"      <td>14</td>\\n\",\n       \"      <td>26</td>\\n\",\n       \"      <td>7168</td>\\n\",\n       \"      <td>1143453696</td>\\n\",\n       \"      <td>14670316437504</td>\\n\",\n       \"      <td>14050759016448</td>\\n\",\n       \"      <td>1.467032e+13</td>\\n\",\n       \"      <td>1.044094</td>\\n\",\n       \"    </tr>\\n\",\n       \"    <tr>\\n\",\n       \"      <th>4</th>\\n\",\n       \"      <td>2048</td>\\n\",\n       \"      <td>32000</td>\\n\",\n       \"      <td>2048</td>\\n\",\n       \"      <td>16</td>\\n\",\n       \"      <td>28</td>\\n\",\n       \"      <td>8192</td>\\n\",\n       \"      <td>1593126912</td>\\n\",\n       \"      <td>20220437594112</td>\\n\",\n       \"      <td>19576343494656</td>\\n\",\n       \"      <td>2.022044e+13</td>\\n\",\n       \"      <td>1.032902</td>\\n\",\n       \"    </tr>\\n\",\n       \"    <tr>\\n\",\n       \"      <th>5</th>\\n\",\n       \"      <td>2048</td>\\n\",\n       \"      <td>32000</td>\\n\",\n       \"      <td>3584</td>\\n\",\n       \"      <td>28</td>\\n\",\n       \"      <td>40</td>\\n\",\n       \"      <td>14336</td>\\n\",\n       \"      <td>6796274688</td>\\n\",\n       \"      <td>83021046743040</td>\\n\",\n       \"      <td>83512623366144</td>\\n\",\n       \"      <td>8.302105e+13</td>\\n\",\n       \"      <td>0.994114</td>\\n\",\n       \"    </tr>\\n\",\n       \"  </tbody>\\n\",\n       \"</table>\\n\",\n       \"</div>\"\n      ],\n      \"text/plain\": [\n       \"   seq_len  vocab_size  d_model  num_heads  num_layers  ffw_size           N  \\\\\\n\",\n       \"0     2048       32000      640         10          10      2560    73825280   \\n\",\n       \"1     2048       32000     1024         16          20      4096   305707008   \\n\",\n       \"2     2048       32000     1280         10          24      5120   552604160   \\n\",\n       \"3     2048       32000     1792         14          26      7168  1143453696   \\n\",\n       \"4     2048       32000     2048         16          28      8192  1593126912   \\n\",\n       \"5     2048       32000     3584         28          40     14336  6796274688   \\n\",\n       \"\\n\",\n       \"                F    approx_flops  chinch_flops     ratio  \\n\",\n       \"0    929877196800    907165040640  9.298772e+11  1.025036  \\n\",\n       \"1   4135248199680   3756527714304  4.135248e+12  1.100817  \\n\",\n       \"2   7353453772800   6790399918080  7.353454e+12  1.082919  \\n\",\n       \"3  14670316437504  14050759016448  1.467032e+13  1.044094  \\n\",\n       \"4  20220437594112  19576343494656  2.022044e+13  1.032902  \\n\",\n       \"5  83021046743040  83512623366144  8.302105e+13  0.994114  \"\n      ]\n     },\n     \"execution_count\": 7,\n     \"metadata\": {},\n     \"output_type\": \"execute_result\"\n    }\n   ],\n   \"source\": [\n    \"# Now try reproduce Table A4 from Chinchilla paper Appendix, \\n\",\n    \"# comparing accurate flops above to approximate flops F = 6*N*D\\n\",\n    \"# note Chinchilla mentions using vocab_size = 32K\\n\",\n    \"\\n\",\n    \"chilchilla_models_table4 = [\\n\",\n    \"  [10, 640, 2560, 10, 64],\\n\",\n    \"  [20, 1024, 4096, 16, 64],\\n\",\n    \"  [24, 1280, 5120, 10, 128 ],\\n\",\n    \"  [26, 1792, 7168, 14, 128 ],\\n\",\n    \"  [28, 2048, 8192, 16, 128],\\n\",\n    \"  [40,  3584, 14336, 28, 128]\\n\",\n    \"]\\n\",\n    \"\\n\",\n    \"rows = []\\n\",\n    \"for num_layers, d_model, ffw_size, num_heads, _ in chilchilla_models_table4:\\n\",\n    \"\\n\",\n    \"    args = dict(seq_len = 2048, vocab_size = 32000, d_model = d_model, \\n\",\n    \"                num_heads = num_heads, num_layers = num_layers, ffw_size=ffw_size)\\n\",\n    \"\\n\",\n    \"    D = args['seq_len'] # dataset size (cancels anyway, for the purposes of the ratio calculation below)\\n\",\n    \"    N = chinchilla_params(**args)\\n\",\n    \"    F = chinchilla_flops(**args)\\n\",\n    \"\\n\",\n    \"    approx_flops = 6*D*N # approximate flops\\n\",\n    \"    chinch_flops = F * (float(D) / args['seq_len']) # exact flops according to Chinchilla paper calculations\\n\",\n    \"\\n\",\n    \"    # print('---')\\n\",\n    \"    # print(f\\\"params: {N/1e6:.2f}M\\\")\\n\",\n    \"    # print(f\\\"approx flops: {approx_flops/1e9:.2f}B\\\")\\n\",\n    \"    # print(f\\\"chinchilla flops: {chinch_flops/1e9:.2f}B\\\")\\n\",\n    \"    # print(f\\\"ratio (chinchilla / approx): {chinch_flops / approx_flops:.2f}\\\")\\n\",\n    \"\\n\",\n    \"    # first copy all keyvalues from args into out\\n\",\n    \"    out = {k:v for k,v in args.items()}\\n\",\n    \"    # then add the calculated values\\n\",\n    \"    out['N'] = N\\n\",\n    \"    out['F'] = F\\n\",\n    \"    out['approx_flops'] = approx_flops\\n\",\n    \"    out['chinch_flops'] = chinch_flops\\n\",\n    \"    out['ratio'] = chinch_flops / approx_flops\\n\",\n    \"    rows.append(out)\\n\",\n    \"\\n\",\n    \"# make a pandas dataframe from rows\\n\",\n    \"df = pd.DataFrame(rows)\\n\",\n    \"df\"\n   ]\n  },\n  {\n   \"attachments\": {},\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"Pretty good match! Except the param counts are still not perfectly accurate.\"\n   ]\n  },\n  {\n   \"attachments\": {},\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"## Scaling Laws: Approach 3\\n\",\n    \"\\n\",\n    \"In their \\\"Aproach 3\\\", Chinchilla paper fits a function L(N,D) to approximate the final loss gives the model size and the data size. Here is the final fit:\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 8,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"data\": {\n      \"text/plain\": [\n       \"<matplotlib.colorbar.Colorbar at 0x7f1bd262a9e0>\"\n      ]\n     },\n     \"execution_count\": 8,\n     \"metadata\": {},\n     \"output_type\": \"execute_result\"\n    },\n    {\n     \"data\": {\n      \"image/png\": \"iVBORw0KGgoAAAANSUhEUgAAA8UAAAHWCAYAAABe7ytwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOy9eXwb9bnv//mOdslavMu7HTv74jgLIZAWKCkJUJYWKPTQS+FwoJcboJD7u72Hvs4BSin0dDvcnkPLgVOgO5SwUxpKA4QtqxMncRYn3ld5k619saT5/TGSLNkzskYayUu+b17fVxtp5pnvjO35zjPP83wewrIsCwqFQqFQKBQKhUKhUM5DmNmeAIVCoVAoFAqFQqFQKLMFdYopFAqFQqFQKBQKhXLeQp1iCoVCoVAoFAqFQqGct1CnmEKhUCgUCoVCoVAo5y3UKaZQKBQKhUKhUCgUynkLdYopFAqFQqFQKBQKhXLeQp1iCoVCoVAoFAqFQqGct1CnmEKhUCgUCoVCoVAo5y3UKaZQKBQKhUKhUCgUynkLdYoplAS8+OKLIISgs7NztqdCoVAoFMqCZiGuuT/5yU+waNEiyGQyrF27FgBQXV2N22+/fVbnRaFQ4qFOMYVCoVAoFArlvOFXv/oVbrrpJlRWVoIQktBBHR8fx913343CwkLodDpcdtllOHLkSFLH+dvf/obvfve7uPjii/HCCy/giSeekOgMKBSK1MhnewIUCoVCoVAoFEq2+Ld/+zc4HA5ccMEFGBgYENwuFArh6quvxrFjx/B//s//QUFBAX75y1/i0ksvRWNjIxYvXpzwOB988AEYhsGvf/1rKJVKqU+DQqFICHWKKRQKhUKhUCjnDXv37o1GiXNycgS327VrFz7//HO88soruPHGGwEAX//617FkyRI88sgj+OMf/5jwOENDQ9BoNNQhplDmATR9mkIRyS9/+UusXLkSKpUKpaWl2LFjB8bHx+O2OXfuHG644QaYzWao1WqUl5fjlltugc1mi27z/vvvY8uWLTCZTMjJycHSpUvxve99L8tnQ6FQKBTK3CaZdRcAnn76aSxatAgajQYXXHABPvnkE1x66aW49NJL47arqqoCIWTG4+7atQvFxcX42te+Fv2ssLAQX//61/Hmm2/C5/MJ7ksIwQsvvACXywVCCAghePHFFwW3b29vx0033YS8vDxotVpceOGF+Mtf/hK3zUcffQRCCF5++WV873vfg9lshk6nw7XXXouenp64bZN5DqFQKJPQSDGFIoJHH30U3//+97F161bcc889aGlpwa9+9SscOnQIn332GRQKBfx+P7Zt2wafz4f77rsPZrMZfX19eOeddzA+Pg6j0YiTJ0/iK1/5CtasWYPHHnsMKpUKra2t+Oyzz2b7FCkUCoVCmTMks+4CXJ3wvffeiy984Qt48MEH0dnZieuvvx65ubkoLy9P6dhHjx7FunXrwDDxMaQLLrgAzz77LM6ePYvVq1fz7vu73/0Ozz77LA4ePIj//u//BgBcdNFFvNsODg7ioosugtvtxv3334/8/Hz85je/wbXXXotdu3bhq1/9atz2P/zhD0EIwf/9v/8XQ0NDeOqpp7B161Y0NTVBo9Ek9RxCoVCmwFIoFEFeeOEFFgDb0dHBDg0NsUqlkr3iiivYYDAY3eY///M/WQDs888/z7Isyx49epQFwL7yyiuCdv/93/+dBcAODw9n/BwoFAqFQpkPxK65LMsmve76fD42Pz+f3bhxIzsxMRHd7sUXX2QBsJdccongMXU6Hfutb31L8Lt//Md/nPb5X/7yFxYAu3v37oTn861vfYvV6XTTPq+qqoo75gMPPMACYD/55JPoZw6Hg62pqWGrq6uj5/7hhx+yANiysjLWbrdHt/3zn//MAmD/3//7fyzLJvccQqFQ4qHp0xRKkvz973+H3+/HAw88EPfW+K677oLBYIimOUXewL733ntwu928tkwmEwDgzTffRCgUyuzEKRQKhUKZhyS77h4+fBijo6O46667IJdPJkHeeuutyM3NTfn4Ho8HKpVq2udqtTr6vRS8++67uOCCC7Bly5boZzk5Obj77rvR2dmJU6dOxW1/2223Qa/XR/994403oqSkBO+++y6A5J5DKBRKPNQpplCSpKurCwCwdOnSuM+VSiUWLVoU/b6mpgY7d+7Ef//3f6OgoADbtm3D008/HVfHc/PNN+Piiy/GP/3TP6G4uBi33HIL/vznP1MHmUKhUCiUMMmuu5H/rauri9tOLpejuro65eNrNBreumGv1xv9Xgq6urqmnSMALF++PPp9LFNVrwkhqKuri/Z3TuY5hEKhxEOdYgolA/zsZz/D8ePH8b3vfQ8ejwf3338/Vq5cid7eXgDcQvrxxx/j73//O/7H//gfOH78OG6++WZ8+ctfRjAYnOXZUygUCoVCKSkp4W3ZFPmstLQ021NKmpmeQygUSjzUKaZQkqSqqgoA0NLSEve53+9HR0dH9PsIq1evxr/8y7/g448/xieffIK+vj4888wz0e8ZhsHll1+On//85zh16hR++MMf4oMPPsCHH36Y+ZOhUCgUCmWOk+y6G/nf1tbWuO0CgUA0epoKa9euxZEjR6ZlcR04cABarRZLlixJ2XYsVVVV084RAM6cORP9PpZz587F/ZtlWbS2tk6Lis/0HEKhUCahTjGFkiRbt26FUqnEL37xC7AsG/3817/+NWw2G66++moAgN1uRyAQiNt39erVYBgmmoZltVqn2V+7di0AJGzxQKFQKBTK+UKy6+6GDRuQn5+P5557Lm79/cMf/oCxsbGUj3/jjTdicHAQr732WvSzkZERvPLKK7jmmmt4641T4aqrrsLBgwexb9++6GculwvPPvssqqursWLFirjtf/vb38LhcET/vWvXLgwMDODKK68EkNxzCIVCiYe2ZKJQkqSwsBAPPfQQvv/972P79u249tpr0dLSgl/+8pfYuHEjvvnNbwIAPvjgA9x777246aabsGTJEgQCAfzud7+DTCbDDTfcAAB47LHH8PHHH+Pqq69GVVUVhoaG8Mtf/hLl5eVxQhsUCoVCoZyvJLvuKpVKPProo7jvvvvwpS99CV//+tfR2dmJF198EbW1tdN6Er/99ts4duwYAGBiYgLHjx/H448/DgC49tprsWbNGgCcU3zhhRfijjvuwKlTp1BQUIBf/vKXCAaD+P73vy/Zef7zP/8z/vSnP+HKK6/E/fffj7y8PPzmN79BR0cHXn311WktofLy8rBlyxbccccdGBwcxFNPPYW6ujrcddddAJJ7DqFQKFOYZfVrCmVOM7U9BMtyrSCWLVvGKhQKtri4mL3nnnvYsbGx6Pft7e3sP/7jP7K1tbWsWq1m8/Ly2Msuu4z9+9//Ht1mz5497HXXXceWlpaySqWSLS0tZb/xjW+wZ8+ezeLZUSgUCoUyd+Bbc1l25nU3wi9+8Qu2qqqKValU7AUXXMB+9tln7Pr169nt27fHbfetb32LBcA7XnjhhbhtrVYre+edd7L5+fmsVqtlL7nkEvbQoUNJnU+yLZlYlmXb2trYG2+8kTWZTKxarWYvuOAC9p133onbJtKS6U9/+hP70EMPsUVFRaxGo2GvvvpqtqurK7pdMs8hFAolHsKyMfkoFAqFQqFQKBTKAiAUCqGwsBBf+9rX8Nxzz832dNLmo48+wmWXXYZXXnkFN95442xPh0JZUNCaYgqFQqFQKBTKvMbr9WJqnOe3v/0trFYrLr300tmZFIVCmTfQmmIKhUKhUCgUyrxm//79ePDBB3HTTTchPz8fR44cwa9//WusWrUKN91002xPj0KhzHGoU0yhUCgUCoVCmddUV1ejoqICv/jFL2C1WpGXl4fbbrsNP/rRj6BUKmd7ehQKZY4zq+nTH3/8Ma655hqUlpaCEII33ngj7vvXXnsNV1xxBfLz80EIQVNTU1J2X3nlFSxbtgxqtRqrV6/Gu+++K/3kKRQKhUKhzAhd6ynZoLq6Gm+99RYsFgv8fj8sFguef/55FBUVzfbUJOPSSy8Fy7K0nphCyQCz6hS7XC7U19fj6aefFvx+y5Yt+Ld/+7ekbX7++ef4xje+gTvvvBNHjx7F9ddfj+uvvx7Nzc1STZtCoVAoFEqS0LWeQqFQKHOdOaM+TQjB66+/juuvv37ad52dnaipqcHRo0exdu3ahHZuvvlmuFwuvPPOO9HPLrzwQqxduxbPPPOMxLOmUCiU+QMbcktmizBayWxRzh/oWk+hUCiZRcq1Hjh/1vsFV1O8b98+7Ny5M+6zbdu2TUvXisXn88Hn80X/HQqFYLVao6lcFAqFkk1YloXD4UBpaSkYRrqEHk/3GvgnpHkPqq44CbVaLYktCkUsdK2nUCjznfmw1gPnz3q/4Jxii8WC4uLiuM+Ki4thsVgE93nyySfx/e9/P9NTo1AoFFH09PSgvLxcElterxe1mzphGQpKYs9srkFHR8d5sVBS5h50radQKAuFubzWA+fPer/gnOJUeOihh+LeONtsNlRWVuLRN1+HWqeL25aEBN4mhwSMC/xOEoHtpfocAi+IJDuuoH3+L8TPUyI7AtUBwuclYEhgPmLnn8iW+GNLtL3QSUhlX/CwQtsL/HCE7ISEfilEbi8wH8EKE7HnK3idp3/k8XnwwL/fA71eL2RMNH6/H5ahILq7zsJgSM+u3e5AZdUS+P3+Bb9IUhYOQmv9I++9Om2tB2HBsuy0CLJQQJkILooC93sCAftC2/N/ziTYXsz8ZQKLolD8nBGwL2hH5HWTkRCvfaHzFfpcaD6C8xe4TzOE/2GOSZBgwCDIm4EgF5qTwIOkTOAYDAK89hmBcxC6FnKBB1WCEAjPb4BMcJ78doSvKf/PQC5oX8CO0PmC5dbpab+jvJtDxv8xZAK/vDIw/L+jAn81fHbsDi8qlz0yZ9d64Pxa7xecU2w2mzE4OBj32eDgIMxms+A+KpUKKpVq2udqnU6UU+zw+jDm8UCrVKAgvJ/APSLqlAVDIQw5XHD7J1BbmCfKOR11uOFw+1Bk0EGnmtJuYAY7drcXY04PDFo1cnM0op3iCX8AIzYXQiEW5UWmmO3FOaFOlw9jNjdytCrkGrUx24t3iicmghgdcyIYDKGsJDclO3yO6diYC9ZRFyor8qBQTPmTkcApttvcGB5yoKIiFyqVYsbtJ48t3mkNBkMYGbTD551A5aJCbq5inT5B+8k5s37vBHrbh5BbqIepgOeGPctOcTAQRFfLALQ5apirCmbFKY6QiZROo6kYBoMhLRvnS30RZe4i+VqfE7/Wj/nd+Ht3K7ZVLYZBOfkgKJVT7Ar4sLv3DLaWLkauKmbtk8gp9gR9eK//FLYU1aFQnRNjh3+aMkacUzwR8mPPYDPW59WgWG2atCPoFIubf4j14+Ph41huqEKpJn/G7cU6xSwCOGg9ikptGco0JZPbCzqU4pziEBvCKfsh5KuKUK6pivtOvFM8fU4sy6LddRA6mQFl2iXx20vgFLMsiwHPYTBEgVJt/RT7AnZEOsVj3sMIsj4Uai6MW+ukcoo9viZMBIeRq7k8zr5UTnHAfxoTgXboNFeDkMnUZ2GnWDg9eq6u9cD5td7Pqvp0Jti8eTP27NkT99n777+PzZs3Z/zYpwaH8ObJMzgxMDjzxmGCIRavHj2Jv548C39AXKrDR6fa8e6xFlhsTrFTRVN7P/7a2IJz/SOi9wWAEZsL73x+Cp+d6Ehp/wjHW/rwt8/OoK07tXnEMjRix18/OIkDRzvTthXLnj2n8Oln52C1uiS1G+Gj90/h870tGB60Z8R+LMMDNvz9zaM4/MlZYWcvwxzeewaf/60ZbSf7ZuX4M3HyYBs+/0sTjn92dranQqFQBMj0Wr+npxV9Ljt2tTbD7vdKYjOWvQNt6Hfb8WrnCYz5pBXFAYB9w+3o99jwZs8xDHvFPyPMxJGxDvR5rPhrfxMGveOS2z9l70KfZwQfDh1Fv2dUcvvtzi70eyw4MHoEfZ4Bye0PeLvR7+3BCVsjej1dktu3+nsx4GlFq/MI+tzSr1XOwCAs3mb0e46i331McvvewDCGvQdh9R3DsGe/5M8jgeA4xr0fwTXRjDHPHsnth0JOuLzvwR84DZfnbbBCL/Mp84pZdYqdTieampqiPQk7OjrQ1NSE7u5uAIDVakVTUxNOnToFAGhpaUFTU1NczdBtt92Ghx56KPrv73znO9i9ezd+9rOf4cyZM3j00Udx+PBh3HvvvRk/H42Ci/J5JgJJ76OUy6AIF9e7/ROijheJDrt8flH7AUCOhntb7vT6Zthyhv09vrRuNgYd9wbe7kr/ocNo4N5mOZxehMRGDxOQm8tFEKxjmXGKC4q4aOnwkCMj9mMpNBsgkzHwuP2wj0n/IJYMFXVcHWB369CsOeaJqFtdAcIQDPVaYR20zfZ0KJR5z3xc67dVLoFRqYbd7ws7xqmtlUJ8qbQO+SotXAF/RhzjLUV1KFLr4QsFMuIYb8yrRYk6FxNsMCOO8SpjDco1hQiyoYw4xnU5NajQlIIFmxHHuFRdhUpNDQBkxDHOU5ajQrscADLiGOsVZpRp1wFARhxjtbwQxdovAEBGHGO5zIQ8zTYAJCOOMcPkIEdzDQAG/kALdYwXCLPqFB8+fBgNDQ1oaGgAAOzcuRMNDQ14+OGHAQBvvfUWGhoacPXVVwMAbrnlFjQ0NMS1W+ju7sbAwOTN7KKLLsIf//hHPPvss6ivr8euXbvwxhtvYNWqVRk/n0mnWJxzq1Vy+7n94pxbnToNpziyr1f8vgCQo1GCgIt0e3zizjcWQ07YKXam7xTrtErIZQxCIRYOp3QPMHl5nFM8ljGnmEtvGRnOvFMsk8tQWGIEAAz0jmX8eHyUVOVDrpDB7fRiNAvRcbFo9RpULuHS6VqOdM7uZCiUBcB8XOv1ShVurFsV4xifkNQx1sqV+Fr16ow5xiqZHNeWr8mYYyxnZNheUp8xx1hGGFxSVJ8xx5gQgo15DRlzjAkhWGFYG+cY90noGBNCUKNbk1HHuESzJqOOca5qZUYdY51yWUYdY6ViMXI012HSMX6HOsbznFmtKb700ksT/oLefvvtuP322xPa+Oijj6Z9dtNNN+Gmm25Kc3bi0YTrTd2xTnESZQJapRI2ry9xpJjHjjZcf+pOwSmOONROzwz7Cgk8MAx0GiWcHj8cbh+0aiX/hmGEfsqxTjGfYIEYCCEwGjQYHXNh3O6B0aBJ2VYsEadYsvTpKacYiRSPDDsQCrFgEil3pGB/KuaKXFh6x2Dps2JZfXnCmlZx9gW+mPI3LpfLUFZTiK6zFnS3DqLAbEzyAEI1e8kdV8z2y9bXoOtMPzpO9WHtJcug1k6vQxRkDka/KZTZZM6v9Sx4bi8EeoUaN9auxq62E7D5vdjVegI31q2GQcl3P+C/rwjdzUJgoZYp8dWq1Xi96wRGfW682nkCN1SvjqsxngmhR3AGgJJR4JqyerzddwxDXgfe7DmG6yrq42qMIwRD/DESoVrjEEvAEDmuMK/F3yxNGPCO4a/9TbiydG1cjXHUjsBMhV0IBgCDLYUN+GT4KPo8w/hw6CguK2qIqzGeGaHYTwgAwfrcdWBB0Ovpw4HRI0D+urga4yisUMVpgrI3IsNS/TqEQNDracdxWyNCLIMKbQXv5nLBpU9IBEWOSm0DWJag13MKrc4jCIFBhbZOwI7AtRA4rpwNolhdD5Yl6Pc0ot9zFCyAsik1xpP2BeYvJK7DMjAqV4NlCYY8H8PqOwYWBEWaTfzPgoLXgf/jAACVcjlMAMY978E10YwQgPwpNcYz2RF+lghBpqiFFtfC7XkL/sAZsB4WOZpr4mqME80/SJ8X5hQLrqZ4NolEir0i0qcBQBOJFIuMME+mT4uP1OaoJ9OnU31zpg87Cg5P6m/Pc8Lp0xOBILx+cdeNj4gjbLNL98Y9kj49Nu5GKJGAVoqYcnWQyxlMTARht2U+pbmkPA8AMNg3jlBwdt5qVi7mUqh7WgfnZAp1QakJ+WYjQsEQWo91z/Z0KBTKLKFXqnBj7eqMRYw1cs4xzmTE+Jqy+oxGjK8wr81oxPgLhQ0oy2DEeENuA8o1ZRmLGC/XN6BcswgA0Gw/JHnEuEq3FuWaFQCAdudhySPGZs0alGrWAwAGMhAxNqlWoUjzRQDAmK9J8oixRrkcpnDE2JOBiLFCUQet5loADCZoxHheQ51iCYk6xYEAgiJqWnURp1ikc6sLR4qdaaRPB0Nsys5opK7Y4U79AUEu4yLOACdNny6mcF3xuN2Ttq0Ier0acjmXlm2T0G4EhiHIL8xeXXFuQQ6UKjkCE0GMDM1O+nJZdQFkcgZOmwdjWUgbFwshBEvXc2lvZ492IjhLLw8oFMrsQx3jxFDHeGb7sY7x8QykUsc6xplIpY51jDORSh3rGGcilTrWMc5EKnWsY+wPnKGO8TyFOsUSoo5p1+MNJO9opltTnEr6tEzGRNOvUxXb0mu5KK/DnZ4zO5lCnb7DaTJykeJxCSOuhBDkRcS2rNKreAJAYTFXVzycBSeVEIKSCi5aPNBjzfjx+JAr5Cir5lpCdZ21zLD17FC1rBSaHBU8Th+6TvfP9nQoFMosslAc4+IYx3gkA45xqSazjnEma4w35DagQksdYyGoY5wY6hjPf6hTLCEMIZN1xSKUpKNp0CmqT/sCQUwExbVzAiajxTPWFQtgiKRPu9J7MDDqwynPEohtRXodj9s9kqY65+VxNViZassUcYqHsiQ8FXWKu2fHKQaAqiVcCnXXWcucTKGWyRgsC0eLTx9qm5NzpFAo2SPT4luaLIhvXRMjvvVGBhzjbeZ5Lr6Vuzaj4lvL9Q1R8a1MOcaZFN8yZ1h8y6RalVHxLY1yeUbFtxSKuhjxrTPnnSr1k08+iY0bN0Kv16OoqAjXX389Wlpa4rb59re/jdraWmg0GhQWFuK6667DmTNnEtq9/fbbQQiJG9u3b4/bxmq14tZbb4XBYIDJZMKdd94Jp1Pc/Y06xRKjTUGBOtVIsVIug1zG/QhTUZGOpj+nWBOs10kTKTZKqECdo1NDJmMQDIbgkKDNU4T8fC5SPJqpSHFYgdpu88DrTV3NO1lKKjmneHTIAW+KL0XSpbS6EHKFDC6HF6OWudn6aHF9FeQKGcaHHRjoHJ7t6VAolFlmoalSZ8IxpqrUie3HqlJnwjGmqtSJOV9UqUMht2QjWfbu3YsdO3Zg//79eP/99zExMYErrrgCLtdkQGn9+vV44YUXcPr0abz33ntgWRZXXHEFgjME97Zv346BgYHo+NOf/hT3/a233oqTJ0/i/fffxzvvvIOPP/4Yd999t6hrNqvq0/MRQXG98P9ydcWemUWzYlTuUo0UE0KQo1Ji3O2F0++HMUcT893Mx9XH9ioWKfgLxApt+RFiWTCEzHh9+DBEIsWOmPRpoRMgAkcIf8wwBKaIArXNDVPY4U52QkL3xdhIcboq2Xyo1AoYTRrYxj0YHrKjojJGXTNNtWc+tDoVTPk6jI+6MNA7hpqw8FV6iPuZyZUylC8qRGeLBZ1nLSgoNXFfiP0lkmot4/mZKjVK1NVX4szhDpw+2I6ymqKZDys4TxHXR9pfLwqFkogQ4UYsjPCNaKGpUr/RcwzXV9SjgKpSQzpVakBQmZqqUoftU1XqkGQPMNMZtqyF15V+7NPh4K6v3R6fyahSqaBSxd/zdu/eHffvF198EUVFRWhsbMQXv8ilxsc6qtXV1Xj88cdRX1+Pzs5O1NbWCs5DpVLBbDbzfnf69Gns3r0bhw4dwoYNGwAA//Ef/4GrrroKP/3pT1FaWprUudJIscRMRn3FR4o9/gmERL6tirZWSitSnFqkUKdWgmEIWJaFK41oo3FKW6Z0iaRQj41L93bdZNSAYQgmJoJwSCAIxkckWpz1FOpZqisGgKql3A2u66wlI8reUrBsfQ0IIbB0jcA6ODcj2mLI9ptjCmUhslBqjDMZMabiW4ntU1XqxFBV6rlFRUUFjEZjdDz55JMz7mOzcc9MeXl5vN+7XC688MILqKmpQUUF/0uhCB999BGKioqwdOlS3HPPPRgdnfx737dvH0wmU9QhBoCtW7eCYRgcOHAgmdMDQCPFkhNJnxbTXkmjVIAQ7qWTxz8RjRwnw2RbplSc4khNcWoLOSEEeo0KNpcXDrc3GjkWi16nAiEEgWAIbo8fuhTtRDCZwk6xhGJbDMMgN1eH0VEnRq1OGCTqgRxLUbEBrWcHMZwlp7i0Mh+nm3rQ32PNSPQ7GUqqCqBUK+B1+zHYM4qSqoKsz2EmcoxaVC0rQefpfpw62IYt16yb7SmlhRRvjyNvjimU85mIY5xcxFg8Ecc4nYhxIiKOcWzEWKiPcSpEHONkIsapEHGM04sYCxNxjAFEI8abhCLGKdpfro/Y5yLGLIByTZVk9qt0a8P2uYgxAJRpl0hiH+AcYwDRiDEAlApFjFPApFoFANGIMQAUai6U7HlFo+TSzCMRYwDIFYoYp4BCURcXMYYH0Gm+wt/HWGIKzU0wGAxp21Hr7ABK0NPTE2dvapR4KqFQCA888AAuvvhirFq1Ku67X/7yl/jud78Ll8uFpUuX4v3334dSKez7bN++HV/72tdQU1ODtrY2fO9738OVV16Jffv2QSaTwWKxoKioKG4fuVyOvLw8WCzJi7nSSLHEaMM/VDGRYoaQqDPtEllXnBN2ilNpyxRJn3akqD4NAIZwXbEtjfpdhmFg0HFzsUkQhc0zSu8UA0B+PvegMDqaGbGtomIjAGBk2IFgIPNOR2GJEXKFDF7PBKwjmamVngmZjEHVEi5a3H5aurfwUrPiAi6lp+tMP+wZqiunUCjzj4USMaaq1PxEVakzLL4ViRifoKrU06Cq1KnBMFrJBgAYDIa4MZNTvGPHDjQ3N+Oll16a9t2tt96Ko0ePYu/evViyZAm+/vWvw+sVfv6/5ZZbcO2112L16tW4/vrr8c477+DQoUP46KOP0rpGU6FOscSkEikGUq8rzkkjfTriFHv9AUwExKtXA5NOsT1NUauIAvW4I/3FPtfEiWLZHR4EUjwvPgqiTnFm+urqDWpoNAqEQixGstC7VyZjUFKeCwDo65LuIUIstSu4Wo+e1kH4RfbqzhZ5xUaU1RaBZYHmfa2zPZ20KDQ3oaikNa1RaG6a7dOgUOYMVJU6MQtClTrD4ltUlToxC02V2u19XzLbc5F7770X77zzDj788EOUl5dP+95oNGLx4sX44he/iF27duHMmTN4/fXXk7a/aNEiFBQUoLWVex4zm80YGhqK2yYQCMBqtQrWIfNBnWKJ0aXaczjFiG+OmnNsU1GfVinkUCk4kQh7iinU0jvF6fcq1qgVUKvkYFmuNZNURCPFYbEtqSGEoMjMRYsHs6TGXFrFpZnNplOcV2yAIU+HYCCE7nODszaPmVhzEZdy1nGqD46xzGQLZAMp3xxTKBQOqkqdGKpKPbN9qkqdmIWiSk2IFirlGsnsziVYlsW9996L119/HR988AFqamqS2odlWfh8yd8ve3t7MTo6ipISrpRh8+bNGB8fR2NjY3SbDz74AKFQCJs2bUraLnWKE0F4xgxE0qdF9xyO7ucXddzJSHGKbZVmaMvEzjBi06cT3jr4zinmvEzhGl0pnFhCSDRabJVKbIsQmEw6yGQMJiaCsDu8CSS+IXy+CXZhY53iQRtYQsCmUtdCCP/gmUxZuIZ3dNgBt2ciekxWyEaiIXaa4f8YwqB2RRkAoP1Uv3j7Um0/g538EhNKFxWBZVmc2HdOvB2Rvw8UCiVL8C1uEUXqqUNwQZxUpY51jG0+H1gWPIPwDgiMEIuoKnWsY2z1egRt8Y2QwGBZElWljnWMhz1O3vkHQwzvELo8sarUsY5xv8eOIMtMG2LnH2QZAHJsmSK+1eu2Cu4jZIdvhFgCFgzW566LE9/q9lgQBOEfrIx3hFjwD3Cq1JFU6uO2RvS4exBgGd4RAv8IsoR3hMCpUsemUve4WxPMn/9aBCDjHSGWoFhdH5dK3ec+hiAY3hFgZbxDaD6hsCp1bCr1kOcAJliCAJhpQ/g68I8ACKdKHeMYj3r2IBBiEWQxfUBgsCz/AKdKrc/5JxBZMYIIIZhAZ30+smPHDvz+97/HH//4R+j1elgsFlgsFng83LN9e3s7nnzySTQ2NqK7uxuff/45brrpJmg0Glx11VVRO8uWLYtGjp1OJ/7P//k/2L9/Pzo7O7Fnzx5cd911qKurw7Zt2wAAy5cvx/bt23HXXXfh4MGD+Oyzz3DvvffilltuSVp5GqBOseREIsXeQADBUPK/7DpVejXFnokAAkHxf1wzOcUzERspTueNmomvLVMa5IXFtqzj0kX0GIYgL49ztkdGM1NXWlxiAsApUIdE/P6kikanQl6hHgDQ3z170eKaZSUgBBjuH4dDQtVwqVlz8WIAQMfJ+R0tplAomWGh1BhTVWp+qCr1zFBV6sQQkryY7nzjV7/6FWw2Gy699FKUlJREx8svvwwAUKvV+OSTT3DVVVehrq4ON998M/R6PT7//PM4oayWlpaocrVMJsPx48dx7bXXYsmSJbjzzjuxfv16fPLJJ3F1zX/4wx+wbNkyXH755bjqqquwZcsWPPvss6LmT53iGTjQ24u/njuHUXdyi45aLgcTji55RNQVp5o+rVLIIWe4H2MqCtTpOsV6rQoE4JSj06gHjUSK3d4J+CcCKduJEI0USyy2VVDAOZAjI5mp+TXlaqFSKxAMhLJSVwwAZeGeyH2ds+cUa3PUKKnkotZtp/pmbR4zUVCSi9KaQrAsi+Z952Z7OhQKZQ6yEB3jYeoYR8m2Y3zc1oheKr4Vx3wX31qoRFKhp47bb78dAFBaWop3330Xg4OD8Pv96OnpwR/+8AcsXbp0mp3IPhqNBu+99x6Ghobg9/vR2dmJZ599FsXFxXH75OXl4Y9//CMcDgdsNhuef/555OSIU9KnTvEMdIyN4ezoKMYTqKLFQgiBJqokLcIpjk2fFgEhJK0Uar02HOn1pFYTLGMY5IRbKNmdqUd5lQo5tOEWUWMSpFDnRZziMWnrfyNO8XCGHFZCCIpLuBRqS/94Ro4xlbJqzike6LEiKKEwmVhqV06mUGcjSp4qay4O1xaf7IU9QxkDFAplfrNQHGOqSs0PVaWeGeoYU+Yb1CmegVTUpCMp1GIc3Khjm0q/YXWktZL4fQ1hh9bhTn2xNuWEU5/TFNvKjdYVp7+wmwwaMAyBfyIIZxrnNpWicKqx1eqSVNk6lpJSEwDA0jeeEftTySvUQ6NTIRAIZu2YfJQvKoJKo4DH5UN/58iszWMmCkpzUV5XDJYFjn3aMtvToVAoEsGyLLyB9DOVIlBV6sRQVeqZ7VNV6sTMd1VqytyCOsUzEO07LMopFi+2FakN9gWCmAiKc7b0aUSKDdFIsS/lP3RjTlhsK41IMQCYDJH+wulHimUyJmpvVMLaT51OBY1GAZZlMZqhKKG5lGuTNDxkz5jjHQshBOXhaHHvLDqjMjmDRcu5aHHbybmbQg0A9Vu4VJ/ulgFYB7OjFE6hUDIHy7L4sKcDf245AdeE+BfMQlBV6sRQVeqZ7VNV6sTMd1VqytyBOsUzkHSkOEZNNhopFrGwKmUyKGTcj4MvWiyk6AikFymO1BT7A0H4UqzlNepiIsVpKOxGIsVjCSLFQuKcfMfMz42kULunfSl4PWeYPyEEhYUGAOEU6lTOdwaVY71BDa1OhVCIxZDFJpnac6K5VtQUAgB6O0YQCqV4sxeyL2L+das4p7ivYwTuqb9PQgeQ4LiJz2u6jdxiI6qXc4qGxz9pSVNMOo0/GgqFIgnuwATax62wej149exJLtOLqlJTVeoUVal73VSVmqpSJ6dKTZk7UKd4BnQppU+LjxQTQqJiW2IFsyKObSqRYrmMgTasfJ1qr+LJSHG66dNcZFeK9GkAUaXoUQkVqAGgMFJXnCGxLULIZAp1luqKi0tNUCrl8Hr8GJnFyKcxLweFpSawLMu1Z5rDrLl4CQgh6GsfwnDf2GxPZ17w9NNPo7q6Gmq1Gps2bcLBgwcTbj8+Po4dO3agpKQEKpUKS5Yswbvvvsu77Y9+9CMQQvDAAw/EfX7ppZeCEBI3/uf//J9SnRJlgaBTKHHjkpXIUSgnHWOpI8YLoMaYqlLzM1V866C1kapST4GqUmeXYMgt2ThfoE7xDEQixS4RTrE2EikW6dxGUqgdYp3icPp0ym2VIinU7tSc2kik2O7yph5lRLwCtc+ffl1XRGxLyvRpACgM1xUPDdkzdkM0h53igSzV+DIyBmXVnPpzT/twVo4pRCRa3NbcN6cXHENeDhatKgcAHN17ek7PdS7w8ssvY+fOnXjkkUdw5MgR1NfXY9u2bRgaGuLd3u/348tf/jI6Ozuxa9cutLS04LnnnkNZWdm0bQ8dOoT/+q//wpo1a3ht3XXXXRgYGIiOH//4x5KeG2VhYFJrqGOcAKpKnRiqSj0zVHwre5zqW4/m3qVpj1N962f7VLIGdYpnIJo+LUI0KxLxFbMPMOkUixXbmkyfTq0uOCK2ZUtRkEqnUUIuYxBiWThSdKwBToFap40oUKe/kEfSp90ePzwppJYLUVCgB8MQeL0TcKQZHReipIyrK7aOOuH1SDf3RFQu4lKou9qGZ/UmX7nYDKVKDqfdM6cFtwAuWiyTMxjqtaLnnGW2p5MUs/Xm+Oc//znuuusu3HHHHVixYgWeeeYZaLVaPP/887zbP//887BarXjjjTdw8cUXo7q6Gpdccgnq6+vjtnM6nbj11lvx3HPPITc3l9eWVquF2WyODoPBIHr+lPMD6hgnhqpSJybbjjFVpZ4OdYwpqUKd4hnQpSC0lRPexykifRoAcsJNqJ0+cQtkjloJAiAYYuEReUwAMKYZKSaEwBhWoB5P00nMC6dQS9FfWKGQwajn5jUiYbRYJmOQn8/1PhsaykwKtUarRG44/bs/SynUJZV5UChl8Lh8GLbMXgq1XCHDohVcve7Z4z2zNo9k0Bk0WL6xFgBw5KPTs9rSKllO9G3Asd5laY0TfRsAAHa7PW74BO5dfr8fjY2N2Lp1a/QzhmGwdetW7Nu3j3eft956C5s3b8aOHTtQXFyMVatW4YknnkBwihDhjh07cPXVV8fZnsof/vAHFBQUYNWqVXjooYfgTrLvPOX8JCuO8TTxLelesFJV6sRkQ3wrG+2aqCq1MFSVGlhR1ohV5S1pjxVljbN9KlmDOsUzEIkUT4RC8CepCh1xij0TEwiK6LeaalsmGcNAF0mhTkOB2pZGlNcUriseT1OBOtconVMMAAWRumKrtErRxUVcpGloyC6p3Vgi0eKBLNWrymQMyqu5aHF32+ymUC9eUwEA6O8YgWN8bjswKzfVQqNTwTnuxpnGztmeTlapqKiA0WiMjieffJJ3u5GREQSDQRQXF8d9XlxcDIuFP8Le3t6OXbt2IRgM4t1338W//uu/4mc/+xkef/zx6DYvvfQSjhw5InhcAPiHf/gH/P73v8eHH36Ihx56CL/73e/wzW9+M4WzpZxPZN8xbpbUMaaq1ImhqtQz26eq1ImZ66rUMkYr2ThfkM/2BOYyLACFTAYFw2AiFILL74dSJRPYeFItVq2QgyEEIZaFOzABfTgCPBP6mJriqX9WM2nR6tUqOL1+2D0+FBv1SR0vgkGXIFIsdOApn5vCEdkxp0dgJ4EbxZRNI3XAVonEtvJzc9DWNSIiUixwwiR+/kXFBuBkn6BTzCZQOSZCN80pu5RW5OLUiV70942DZVmQVJSTkyHGbFVdITrOWtDdNoT1m2vBMFIcM7lrGoUFDLk6lFYXoL9zBGeP9WD9JUuFlaMFr6dE2wsRtqNQyrH2kmXY9+4xNO87h0WryqHWJfc3HzbEMxdxUxHD6rLDaacP2+12ACXo6emJs6VK8l6XDKFQCEVFRXj22Wchk8mwfv169PX14Sc/+QkeeeQR9PT04Dvf+Q7ef/99qNVqQTt333139P+vXr0aJSUluPzyy9HW1oba2lrJ5kuZx0QUpGNh2KhjvOvsyahjfMOSldAplPw2+GCEHpInValfbTuBcb8Xu1qbcWPdKhiUfL/P/PaFbhUhsFFV6te7TmDU58arnSdwQ/Vq5KqSf8gVeq3PAFFV6rf7jmHI68AbPcdwfUU9CtQ507YPhvhjMDKG/wixqtR/szRhwDuGv/Y34crStShWm6bbEZipcFiCAcBgS2EDPhk+ij7PMD4cOorLihpQqskX3IvfDj8yAqzPXQeAoMfThwOjR7Axbz0qtGb+HViB50sIBGQIp0odAkGvpx3HbY0IsQwqtBW8m8uFfllYgatEOFVqliXo9ZxCq/MIQmBQoa0TsCNwLQSOK2eDKFbXg2UJ+j2N6PccBQugTFvPv4Pgn5LA9QmrUrMswZDnY1h9x8CCoEizif95SvA68H8cADhVagDjnvfgmmhGCEC+5nJ++zwfzf3csvMLGilOgqiadJIp1ISQaFsmMVHfVNOngUkF6lTEtozhmmKXb0J0j+SojXD6dLq9iiOR4jGbW5I3bgW5mYkUR9oy2eweeL3iU9aTochshEzGwOP2Y3wsO9FSc0UeFEo5PG7/rKZQA8DStZUAuJ7FExIIr2WSRSvLkVdsxIQ/gGOftsz2dBIi5Ztjg8EQN4Sc4oKCAshkMgwODsZ9Pjg4CLOZ/wGxpKQES5YsgUw2+aC4fPlyWCyWaDr20NAQ1q1bB7lcDrlcjr179+IXv/gF5HL5tDTrCJs2bQIAtLa2ir52lPOPbESMb4irMZY2YkxVqRPDL74lnZYFIQTrY2qMD1FV6mlQVWrKXIE6xUkQVaAWIZwVFc0SsU9ERdoXCCadqh3BEBbbSqWtkkohh1LOPXjaUxTbMuVMRorTuRmY9GowhGAiEIQzxbnEkp/LvbF2SSy2pVYrYDRy55ypFGqZjEFxqREA0NebvRTqikWcCnVXK78qcLYoqcqHPleLCX8A7afndnsmQgg2fIl7aGg73g3rLLa1mosolUqsX78ee/bsiX4WCoWwZ88ebN68mXefiy++GK2trQjFlKCcPXsWJSUlUCqVuPzyy3HixAk0NTVFx4YNG3DrrbeiqakpzpmOpampCQDndFMoyZB98a357xjPb1XqJskdY6pKnRgqvkWZC1CnOAnERorj9hHhFCvlcijDD3Jiew5HFKRTcYoJITDp0qsrNuaoQQD4J4Lw+FKPnDIME23NJJXYVsTeiNR1xcWcwzqYwbrisvI8AEB/rzVjx5hKVV0RAKCrfRihYPI18VJDCIlGi88c6Uqr3Vc2KKrIR9XSErAscPjvzXRBnMLOnTvx3HPP4Te/+Q1Onz6Ne+65By6XC3fccQcA4LbbbsNDDz0U3f6ee+6B1WrFd77zHZw9exZ/+ctf8MQTT2DHjh0AAL1ej1WrVsUNnU6H/Px8rFq1CgDQ1taGH/zgB2hsbERnZyfeeust3HbbbfjiF78o2L6JQuGDOsaJWXiq1PPbMaaq1NOhjjFlJqhTnAS6lCLF4XRm0T2HU9vPoAnXBXtSdGojYluu1NKf5TIZ9GHHeizNFOpof2GbNIrRBXlctHg4Q06xJYNRwdIKziketNjhz1IKsbk8F2qNAj7vBAayFKEWonZFKdeeyeZBX/vsRq6TYd2XVkCukGG4bwwdJ/tmezpziptvvhk//elP8fDDD2Pt2rVoamrC7t27o+Jb3d3dGBiYfEisqKjAe++9h0OHDmHNmjW4//778Z3vfAf//M//nPQxlUol/v73v+OKK67AsmXL8L//9//GDTfcgLffflvy86MsfGZHfEvadk1UlVqY6eJbTVSVeop9qkqdmPmgSk0RhjrFSZCTQqQ4lfRpIEZsS2ykOFxT7PT6U4qoGXXhlkppKFDnRsS2HOktsvkmrl5xVCLV4cKIUzwqrVNsNnN1xWNjrow5rAajBgajBizLYqBvPCPHmArDMNFocce5wRm2zixyhTyqRH3qiHSLe6bQ6TVYddFiAMDRvafhTyNrYiFy7733oqurCz6fDwcOHIjW9wLARx99hBdffDFu+82bN2P//v3wer1oa2vD9773PcG06IiNp556KvrviooK7N27F6Ojo/B6vTh37hx+/OMf0z7FlJSZnXZN0jnGVJU6MVSVemb7VJU6MXNdlZoiDHWKk0CbQip0tFexUMSX8I9opHiqUyywfWTo1ErIGE7x2unzzbj91GGMpE+7vGAJokMMEQXqcQdPpFjo2DxEFahtrri5iJ0PwO1TUMCpcY9YnQiB5eyImA/AiR5OHRqtCnq9Giyb2dZMZeVca6a+qSnUgudBxA0eI9VLOPGjns4RTARCYAmJDvH2BRDYfup/S9dWgZERjAyMY8QyzvMzE/nDFDtPkXaWb1gEfa4OXpcPxz9tiZmmRNeNQqFIC98NPqJIPXVAwDH2+8XZ4duWBcBOqlLHOsY2nw8sC55BeMe0xTM8QiyiqtSxjrHV6xG0xTdCAoNlSVSVOq7G2OPknX8wxPAOocsTq0od6xj3e+wIssy0IXb+QZYBIMeWKeJbvW6r4D6JbPHOCQzW566LS6XudVsQBOEfrIx3hFjwD3Cq1LE1xj3uHgRYhneEwD+CLOEdIXCq1LGp1D3u1gTz578OAch4R4glKFbXx6VS97mPIQiGdwRYGe8Qmk8orEodm0o95DmACZYgAGbaEL4O/CMAwqlSxzjGo549CIRYBFlMG5S5A3WKkyCV9Gl9NFIs7g1vqunThJCoAnUqdcXGNGuKASBXH1aOTjd9OqxA7XD54J9IPwKba9SCYQh8/gAcTuneuAOAuZiLOGUyhbosnELd12PN2tvG/CIDcgwaBAMh9HRIV1eVClqdCjVLSwEAp+dBtFgmY7BxK1fTevZIJ0ZnWcWbQqFIT/ZrjKWNGFNV6sTwq1JLn0odcYwPUlXqaVBVakq2oU5xEkSjvimoT7v8EwiJ+CNINX0amKwrtqVQVxypKXb7JlJ2RE0SpU+rVQroNNx1sEqQQi2TMdHo8/CoI217sUzWFWcuUlxsNkIuZ+DxTMA6Kk2d9UwQQlCzmKv17DxnycoxE7F8XRUAoKd1EA6J0uozSWlNYVR068B7x+MUlCkUysKAOsaJWXiq1Jl1jKkq9XSo+BYlm1CnOAkiStIToVDSrZK0SiUIgBDLwu1Pvq5wMlKcSr/hsNhWCtFelUIOTbi3cqp1xbnhtkxefyAtBWoAyI+IbY1L4wQWhlOopXaKzWGn2Gp1ZqyuWCZjUFLGpVD3dEu3IM9E9RLOKR7oGYPHJW2EXSymAj1KqwvAssCpw52zOpdkWX/5SihVClgHbTh9qH22p0OhUDIAdYwTs/BUqTPjGFfMe1XqlQCoY8zHbDnGgZBbsnG+QJ3iJFDKZJOtkpKMFjOEQBeJ+opYwCL9hp0+P4Iio0tRBelUew2HU6jHnak5xXK5DIawDas9TbGtXGmd4qKwUzw0Iq1TrNOpYDBwdcWZTKGuqORSqHu6sucUG0xaFBQbwLLsrAtuAcDKjTUAgPbTfXCn+DuaTbQ5aqwP9y4+9ulZ2CUWeqNQKHOD+S6+RVWpE5MN8a31816Vup6qUidgqir1uHevZLaF+Kj7Euzp2pD2+Kj7kozPda5AneIk0aWQQq1XTTq4yaJRKCBnGNH7AYAxXFOcal2wKRzpTbUtEwDkhVOorWmmUBeEI8UjY9IsnEX5nFNsHXdhYiK5aH+ymM0mAIAlg7Wj5ZX5IAQYs7rgcGTPIVy0rAQA0H5mYNZTforKclFYakIoyOLMPKgtBoBFq8pRUl2IUDCE/e8dn/VrSKFQMsN8d4ypKnViqCr1zPapKnViIqrUBEpoFHWS2aVIh3y2JzDnCYvA5iiVGPN4RDrFSgxAXCo0YQj0ahXG3B7Yfb5o9BdJ/F1ORoq9YFkWRKSCrSnclmksHafYoEWnZYyrK449vMD8hU4rEiket3sQCAQhl4fbsAidE0l8gBydCjqtEi63H8NWJ8qKBFqyCF0yoYkSghKzCWfPWjBgsU3OL9GNVOQxWEKg0ihRZDZicMCG7q5RrFhdLqyvLPYmnuDXpKquCI2fnYNtzI3RITsKzMakfheTsi94TYW3X7mxBh+9eRRnj/dgxYYaqLXKBErNIg8gdN3SUIImhGDT9jV45/mPMNxrxdmjnVi6riZle1LApUOld+s/n9KpKOcJyd7XQgL3A4aNOsa7zp6MOsY3LFkJnUIpyg4/k6rUu9pOwOb3YlfrCdxYtxoGpYp3e34r/ITARlWpX+86gVGfG692nsAN1auRq9IK7MVnhx8GiKpSv913DENeB97oOYbrK+pRoM6Ztn0wxB+zkTH8R4hVpf6bpQkD3jH8tb8JV5auRbHaNN2OwEyF8/MYAAy2FDbgk+Gj6PMM48Oho7isqAGlmnzBvYRtTUdGgPW568CCoNfThwOjR3BB3nqUa838Zlih9nQCL/4Jp0odAkGvpx3HbY0IsQwqtBW8m8sFl1aBq0Q4VWqWJej1nEKr8whCYFChFXAAWYG4nMBx5WwQxep6sCxBv6cR/Z6jYAGUaesF7AvMnwhcn7AqNcsSDHk+htV3DCwIijSb+J+nBa8D/8cBACrlchTJq8EwGgQABBM9gKXJpZV7JWk/aLfbAZSkP6F5AHWKkyQqtiXizW+0V7HIiK8h4hR7fEBu8vvpNSoQAgRCIbj9E9H07WQxxbRlSpWIAnW6kWKtWgm1SgGvbwJWmzsa6U2HogI9OrpHMTRiF3aKU8Bs5uqKx8fd8Hj80GjEXfdkqagqwOCADT1dI1ixujwjx5iKUiVHZW0ROlosaD09wDnFs0hpdQHyigywDtlx+kgXGrYsntX5JEOOUYuGS5bj0PvNaNp7BmW1xcgxJv+QKTXvd10OrV64128yuB3SZltQKAsFUY5xCkRqjJNzjMUTqTFOxzFORKTGOBnHOBUiNcbJOMapEKkxTt8x5idSYwwAvZ4+HLQ2gpB1KNNI45REUqk5++1oth8CQ0Io01RJZr9KtzZs/xTanYfBIIQy7RJJ7ANcKjUA9HsaMeA5CgKgVMgxTgGTiuseMeT5GGO+JhCwKNRcKDrQJATDaCSxMxNyRgs5k/7frZzJjF7OXGRW06c//vhjXHPNNSgtLQUhBG+88Ubc9yzL4uGHH0ZJSQk0Gg22bt2Kc+fOJbT56KOPghASN5YtW5b2XHNS6FUcSZ8WK5oVEduye8U5pzKGie6bilhWJFI87vKIUsyOJc8Qbstkd6eVdkIImUyhlqyumHOEpa4rVqsVyA1HtgcymEJdWc0tukMWOzwe6dLyZqJ2ObcYd7UOYSJDYmLJQgjB6k2cYMjZY93wZvE6pMOShmoUluchMBHEgd00jZpy/jGf1vt0oeJbiaGq1ImhqtQzM9/Ftyhzk1l1il0uF+rr6/H000/zfv/jH/8Yv/jFL/DMM8/gwIED0Ol02LZtG7wzOIsrV67EwMBAdHz66adpzzWVtkyTTrH4SDGQWlum2BRqsei1KsgYgmCIhTOFXscAYNRpwDAEE8EQHCkKfkWIim1JVVccI7YVCkl7cyspMQEA+vvHJLUbiy5HjfwC7m16T2f2egcXlRihN2kQmAii89xQ1o4rRNmiQuQV6RGYCM4bJWpCCC7cvgYyOQNL1wjOHumctbl8uWoPrqzen9b4ctWeWZs/ZX4yn9Z7KaCOcWKoKnViFo4qNXWMKfOHWXWKr7zySjz++OP46le/Ou07lmXx1FNP4V/+5V9w3XXXYc2aNfjtb3+L/v7+aW+YpyKXy2E2m6OjoKAg7bnmpKAkrVdz+9hFRooNEcGsFJziaLTXLb4umCFksq7YmVpdMcOQaAr1qD29CG9hHucADo9JEynOM+mgkMswMRHEmE3amsiyUi7Pvb9/PKM3zaqaQgBAV0f2nGJCCBavKAUAnDvZN+uLAiEEazZzNUpnj3XPeruoZDHk5aDhUu4B4eje0xiXOGMhWSIpVekOCkUM82G9Z1kWgZB0pQHzXXyLqlInhqpSz2y/SreWqlJT5g1zVn26o6MDFosFW7dujX5mNBqxadMm7Nu3L+G+586dQ2lpKRYtWoRbb70V3d3dCbf3+Xyw2+1xYyrpRIq9gQAmkuxvDABGdbjfcArRWlM4Upxur2Frik4xAOSHU6hH03Q8CyJiWzY3JgLpP6gwDEFRIRcttgxP/xmnQ3GxAXI5A4/Hj7HxzIkQVS7iHvgGB8azmkK9aFkJGBmDsREnRodmx5mLpbS6AAUlRgQDITQfnD89gJc0VKGkphDBQAifvX0EQQl+rymU+U621vtEaz3LstjX24Ndp07CG5CuTGS+O8ZUlToxVJV6ZvtUlZoyX5izTrHFYgEAFBcXx31eXFwc/Y6PTZs24cUXX8Tu3bvxq1/9Ch0dHfjCF74Ah0P4Qf7JJ5+E0WiMjoqK6Up8+rCIhScQQIANcepysYMHlVwe7W88NVrMCgxgMlLsmZiAX+RDs3EGpzjRcYFJp3jMOcOiN/X8Y65DvoFzZq0xkWLB4yawo9OooFUrwIJrpSQF5kKurnhQwClOZZ4AIJMxMBdzIlT9fWPC2yfSaSBEYEzuqzdokFeQA5bNbs9ilVqBqroiAMDZ0/1gCZk2hOcvMIQukMD2sf8xhMHaiziRrdbmXjgdHp7rLGRf6CuheQogcnsCLhvjoivrodIqMT7sQNPHZ8Qfl0JZYGRrvRda6wkLePwTODk0iCGXC2+cOQXvREB4QeAbIcI/IOAY+/3i7Agee1KVOtYxtvl8YFnwDMI7IDBCLKKq1LGOsdXrEbTFN0ICg2VJVJU6rsbY4+SdfzDE8A7BH0uMKnWsY9zvsSPIMtOG2PkHWQaAHFum1Bj3uq2C+ySyxTsnMFifuy6uxrjXbUEQhH+wMt4RYsE/wKlSx9YY97h7EGAZ3hEC/wiyhHeEwKlSx6ZS97hbE8yf/zoEIOMdIZagWF0fl0rd5z6GIBjeEWBlvENoPqGwKnVsKvWQ5wAmWIIAmGlD+DrwD8rcYc46xaly5ZVX4qabbsKaNWuwbds2vPvuuxgfH8ef//xnwX0eeugh2Gy26Ojp6Zm2jUoum+wfLCJabFCJrw9WyeVQyeWi9wNiFKTd3pTEsnJzwkJZ6USKjZH06fTfJhfkhlOordK8OS4OO8WWYbvkb/rKyrgU6r6+zNUVA5Mp1J1ZTKEGgMUruRTqrtYh+HwTWT02H+aKfJgr8hAKsWg+MH+ixZocNTZv55QyzxzuwEDH8CzPiEKZn4hd7xOt9TqlEl9btgIauXzSMZ5vEeMFUGOcyYhxNsW39g4fyaj41kFrY0ZrjJvthzJaY9zuPJzRGuOBDNcYj/maaMR4ATJnnWKzmevLNjg4GPf54OBg9LtkMJlMWLJkCVpbWwW3UalUMBgMcWMqhBDowynUYtSkIynUYuuKjZG6Yo+4NOgcNSeWFWJZOFJIv87LmawpTvWPPaJA7XD74J9I76GiIFpXLM3iWJCXAxlD4PUFYHOk3nqKj9KwUzw4ZIc/gyrNVbOUQl1QbIApX4dgIISOM8LRm2xSv5mLFref6ofdKk02QTYoryvGkgauBcbn7zbB45T2d5FCmU9ka72faa0v0OqoY5wAqkqdGKpKPbN9Kr5FmcvMWae4pqYGZrMZe/ZMqpza7XYcOHAAmzdvTtqO0+lEW1sbSkrS7/GWE4n6iokUq1NryxSpKx4X6RQzhMSkUIuP9hq0asgYgkAwBHuK6tFqpQK6sMhYutHiorBTPDQqzcIokzEoDKtQW4ZsktiMYNBrYNCrwbJsRlsz6Q0a5IdTqLMuuLWyDABwdg4IbgFAYakJZTUFYFkWTZ8lbt8y11h36QqYCvTwunz4/C9NkiuiUyjzhbm03kccYzV1jHmhqtSJyZZjTFWphaGO8ezx5JNPYuPGjdDr9SgqKsL111+PlpaW6PdWqxX33Xcfli5dCo1Gg8rKStx///2w2RI/M09tvRcZP/nJT6LbVFdXT/v+Rz/6kaj5z6pT7HQ60dTUhKamJgCc2EZTUxO6u7tBCMEDDzyAxx9/HG+99RZOnDiB2267DaWlpbj++uujNi6//HL853/+Z/Tf/9//9/9h79696OzsxOeff46vfvWrkMlk+MY3vpH2fKORYhELTCR9WqyStFETToMW6RQDQG5UgVr8vgwzqUBtnamuOAH5xnCPYZs0CtROtw9uiaKiJUVc7e/AkLRiWwBQXp4HAOjttUpuO5bqWq6+t6M9u6m3NUuLoVDK4bB50N+d2XNMlrVbloAQoKdtCMP947M9naSRK2T4wnXrIVfIYOkawcl988upp1DEMJ/W+wKtDl9bPs8dY6pKLUg2VKkz3a6JqlInhqpSzw579+7Fjh07sH//frz//vuYmJjAFVdcAZeL8wX6+/vR39+Pn/70p2hubo5qQtx5550J7ca23RsYGMDzzz8PQghuuOGGuO0ee+yxuO3uu+8+UfOfVaf48OHDaGhoQENDAwBg586daGhowMMPPwwA+O53v4v77rsPd999NzZu3Ain04ndu3dDHY6iAkBbWxtGRiajZb29vfjGN76BpUuX4utf/zry8/Oxf/9+FBYWpj3fVPoOpxwp1qbuFEfbKrlSqwvOD7dUsjpSX+QKIk5xmgJZSoUcueEa5SGrNKrH5iIuZW5gSPq64qhT3DeW0RtkdS33+zw8aIdT4jTwRCgUctQu56IwLSd6s3bcRJjyc7BoBRfBPvLJ2Xm1MBnzc3DBFasBAMc/OwtLV3brxCmUbDHf1nvqGCeGqlInhqpSz2yfqlJnlomQR7KRLLt378btt9+OlStXor6+Hi+++CK6u7vR2NgIAFi1ahVeffVVXHPNNaitrcWXvvQl/PCHP8Tbb7+NQIL7a2zbPbPZjDfffBOXXXYZFi1aFLedXq+P206n04m6ZnJRW0vMpZdemvAXiBCCxx57DI899pjgNp2dnXH/fumll6Sa3jT0KfUqnowUszEiczPpzZk0SaZP8xiKiG2NuT1xxwQSiNrGfJ5n0AL94UixwPYz/d0XmCKRYifYRMdN4v5RlJeDMZsbQ6MO1JTm85sRsM933MICPWQyBl7fBMbt3qjTnXhCQgeI376o2ACFQgavdwIjo85oqnb8XPltERE3U61OheISIwYHbOhsH8aq+hjFdMFfLpEqhwLTWbKqDGeO9WCgxwr7mBuG3BR71or9nUiw/ZrNtehsGcDIwDi6zw2iaolZ+JdO6DqLVXyWyM6ileUY7B5F99kBBCeCM4qUUyjzkTm93k9twwDub7BQo8MNy1bgtTOnoo7x9eHU6qQJCfw1M2zUMd519mTUMb5hyUroFEpRdviZVKXe1XYCNr8Xu1pP4Ma61TCEu2lM3Z7fCj8hsFFV6te7TmDU58arnSdwQ/Vq5KqSXxNCAp8zQFSV+u2+YxjyOvBGzzFcX1GPAnXOtO2DIf4Yj4zhP0KsKvXfLE0Y8I7hr/1NuLJ0LYrVpul2BGYqNH8AYAinSv3J8FH0eYbx4dBRXFK4DuXavAR78Vri/VRGgPW568CCoNfThwOjR3BB3nqUawXq8FmZgH2BLieEU6UOgaDX047jtkaEWAYV2ukdWgBALrhGC1wlwqlSsyxBr+cUWp1HEAKDCm2dgB2BOJ7AceVsEMXqerAsQb+nEf2eo2ABlGnrBewLzJ8IXJ+wKjXLEgx5PobVdwwsCIo0m0D4ngN4rkNQ6OFVAl7puAaanPTdPI+Tc1antqtVqVRQqfjuJZNE0qLz8oR/5202GwwGA+RJ3lcHBwfxl7/8Bb/5zW+mffejH/0IP/jBD1BZWYl/+Id/wIMPPpi0XWAO1xTPRSYjxSLSp9WTvYr9YnoVh51ip8+PQDDRbXc6udFIcWoRxIjYltWRugJ1JFI87vAgkGYv1uJ8zrGUsq64KOysDkhcVyyTMSgt5QS3Mp1CXbOIi4ZkO4Vab9CgrIp7OdHSPDeixdocNVZs4N5oH/307Lzr/7tx6ypc/a0voryueOaNKRRK1qDiW4mhqtSJoarUM9unqtTzg4qKiriWdk8++WTC7UOhEB544AFcfPHFWLVqFe82IyMj+MEPfoC777476Xn85je/gV6vx9e+9rW4z++//3689NJL+PDDD/Htb38bTzzxBL773e8mbReY5UjxfCPSq9jh94NlWf43QVPg2ivJ4AsEYff6UKBL7g2qRsH1OPYHg7B5vSjQJP/mNZI+7fFPwDsRgFoh7secF06fHnd5EAyFIGPEvzvRqpXQqBTw+CYwanfDnDv9zW6yFIWd4pExZ8rzmUpJkREDgzZYhmxYsTh9EbZYKipy0dU1gt7eMTSsrZLUdixVNQU4uK8NY1YXxsdcMOWKSxNJh6VrytHXNYr2MxasuaAGKpUia8cWYsX6KrQ298Jl9+L0kS6s2rho5p3mCHKFDDmmFCPuFAolo0Qc47QixgkQFTFOgYhjnFzEWDwRxzidiHEiIo5xbMT4uop6FPJEjFMh4hgnEzFOhYhjHBsxvqyoAaUa/sw3sUQcYwDRiPGm/HUo00jzbBNxjDn7XMSYBVCukeb5JuIYc/a5iDEAlGmXSGIf4BxjANGIMQCUCkWMU8Ck4py+SMQYAAo1FyblJ2SKm2re5u2mIxa73Y57YEZPT0+cvZmixDt27EBzczM+/fRTQbtXX301VqxYgUcffTTp+Tz//PO49dZb40prAK4kJ8KaNWugVCrx7W9/G08++eSMc41AI8UiyAmnTwdCIXhEvCmOKEnbvclHbgkhMEVVpMVFfJVyGXThuaZSV6xTK6FSyMCygDXFfsWEkGgK9fB4em91DTlqqJVyBEMsRsekabtTGiO2JbXqb1lZHggBrGMuODPYakelUqC0nItKt50bythx+DCX5cKUr0MgEMS55r6sHlsIuUKOhou5Fk3NB9vhzmKt9Xwi2zVGFMpCgKpSJ2ahqFLP93ZNVJVamPNNlVrBaCQbAKa1s0vkaN57771455138OGHH6K8vHza9w6HA9u3b4der8frr78OhSK5wMonn3yClpYW/NM//dOM227atAmBQGBa2U0iqFMsAjnDQBf+waWSQi1WgXqyrlj8A2hsv2GxEEKQb+Ac2lF76k5ooSncYzhNp5gQguJIG6URaRSjC/JyoFDI4PMHYE1TDGwqarUCRWExr56ezKZQ1y7m0m3b24ay2tKHEIIVaysBAGeO96adIi8V1ctKUFhqQjAQwpFPpV1QFwq/a/sanj+3Pa3xu7avzXwgCmWBQcW3ErMQVKkzLb5FVakT26eq1PMblmVx77334vXXX8cHH3yAmpqaadvY7XZcccUVUCqVeOutt6ZFfBPx61//GuvXr0d9/cxR/qamJjAMg6KioqTtU6dYJJEWS3YRTnEkUmwTESkGAJM29dZKk3XFqS1IEQXqkTQUqKVyigGguIBzMi2j0ihQMwyJtmbqHxyXxGYslRVcWlR3hp3i8oo8qNRyeNx+9PeOZfRYU6mqK4JOr4bPO4H2M5asHlsIQgg2XLoMhABdZy2wZPj6UyiU8wvqGCeGqlInhqpSz2yfqlLPX3bs2IHf//73+OMf/wi9Xg+LxQKLxQJPOLgXcYhdLhd+/etfw263R7cJxuguLVu2DK+//nqcbbvdjldeeYU3Srxv3z489dRTOHbsGNrb2/GHP/wBDz74IL75zW8iNzc36fnTmmKR6FUqDDid0xWoEyjjRiLF9thIcRIq0NH06RTaMuVFeg2n2pbJwDnFo2Kd4pj5F4briG1OL/wTASjF1DZPuT6RSPHgqAMhTK/nJiLvNywBSoqN6O6zom/QhtUruPQOsSrZQoetKM/DocMdGByyweubgDqZmluxxyYEjFyGmrpinGnuQ+u5QZRV5QsrF4u9Kc9QCsPIGCxfW4HDn5zD6WPdWLzcDIav3lu0qrO4+Uz92ecXGbF4TQXOHuvB4Y9O46pbN4ORxcxLUKpc5IGlUrHOlA0B/kfta2nXGdntdjwIAYVTCmWeQVgCMuW+wArciKgqdaJPqSp1UudAVanDx6Wq1IEFFpv81a9+BYDrNhDLCy+8gNtvvx1HjhzBgQMHAAB1dfE/z46ODlRXVwMAWlpaosrVEV566SWwLMvbh16lUuGll17Co48+Cp/Ph5qaGjz44INxdcbJsLB+GlkgsmhkI1KcG44Uj7nFO7a5Odzik2qv4gJ9OH3a4Ur5DZdGpUCOlrtew2mmKOfn6iCTMfD5A7CloYodS6mZixQPDdslT//V69XIzdWBZYG+DEdw68Ip1L3do/B6JzJ6rKnULiuBSq2A0+5Fd5ZVsBNRv3kxVBoFbFYXWo71zPZ05hRS1hhRKOcrVJU6MVSVOjELUZW6l6pSx3G+qFLHwrIs77j99tsBTLbm4xsRhzhiJ7JPhLvvvhtutxtGo3HacdetW4f9+/djfHwcHo8Hp06dwkMPPZS0wFYE6hSLJLX06clIcUjEH0SkLZN3IgDvhLjFNpI+7fT64U/B4cvN0YAhBL6JIJze1BdiqVKoZQyDojzOlmVEmhRqo14DrUaJYIjFoEQ2Y6ms4N76dvVIt9DxkZufg7yCHIRCLDpaBzN6rKnIFTIsXV0GADh5tHvO3PBVagXWhkW3ju9vhTuDgmcUCuX8hDrGiZkNx3h4HjvGmRLfKs+i+FYmHWMqvkXJNNQpFkkqTnGOSgWGEIRYFk5f8guaUi6LKl6PiRTbUivk0Cq5lN1U6oplMga5YbGukTTEtorCKdRDEtQVmwvDdcUSiW0RQlBmNgEA+gakj+ZG6or7+8cxMZFZIaq6JVxa1LkzlqzfcJesLodcIcPYqAt9XZl9ASCG2pVlyDcbEfAHcfijM7M9HQqFsgBZKKrUpnnuGM93VepyTdGCUaWmjnE81DGeP1CnWCRRJWmfL+lfaoaQ6H7jWUyhzsuJ1AWnmEIdVqAelsApHhxzpn0TiDjFA8N2yW4oZSUmAEDvwLgk9mLJzdVCr1cjGAyhty+zgk81dUWQyRnYxt0YGpTmpUGyqNQKLFnFRYuPN3bNmZs9IQSbLl8BQgh6WofQneUoOoVCOT9YCOJbN1BVakGyIb61pXDtglGlzpRjvJBUqUe8ByW1T5EG6hSLRB+uKQ6EQnAHkq/fNEXqikWKZqXjFOeHnWJrigrUBUbOKR6xp754FZh0IITA45uA05PeIl6UlwOGELg9fjhc0izYpWYTCAFsdg8cEqfYEkJQVclFizszHEFVKuWoWcTJzp+bBSXo5WsrIJMzsA470D+HFJ9zC/VYsaEaAHDog9PwZbnmmkKhnB8sBMeYqlILQ1WpZ7Yfq0qdiT7GC0WVmkAGjZwKVc5FqFOcCDJ9yBkGOUoupTlWTZoVGBFS7Tmcq+P2s7o9vPNJdNxIr2KrM0Wn2BBxil1gCeIG31z4lP7kMhnyjZxzPjTmnL6DgB2+c5LJZSgM1xUPDMer0gkywzxVSnm03VPvwJjw9qIHAQhBdXUhAKCvbwwTgWD48+Su3eQ5EIERv//i5dxNtrNzGD4fj/M3w1yTHjxG1BoVlqwMR4sPdyEETh2bTUVFWfD6CHwh+DH335pNtTDk6uB1+3H0k7OiziuRfWE7Qucl9jpTKJSswLPgRBSppw6hRZewk6rUcTXGEwHhhZpvhAj/gIBj7PeLsyN47ElV6ljHmMuKA88gvGPaw0J4hFhEValjHWOr1yNoi2+EBAbLkqgqdVyNscfJO/9giOEdgj+WGFXqWMe432NHkGV4h9hzADhV6tga4163VXB7oSE4HzBYn7sursa4121BEIR/sDLeEWLBP8CpUkdSqY/bGtHj7kGAZXhHCPwjyBLeEQKnSh2bSt3jbk0wf/7rEICMd4RYgmJ1fVwqdZ/7GIJgeEeAlfEOofmEwqrU1YZ/gFZegxDLhH/umcEf8ko2zhdoS6YUMKpUcPr9sPt8KNHrk9pn0imehfRpZ6rp09z+Lp8fbp8fWhVPO4gkKMrVY2TchcExJ2rLClKyEcFcaMDgqAOWYTuW1hSnZStCeWkuLMN29PaPYUWdtG/v8vJ0MOjVsDu86OmxYtGi5JuIiyW/UI/cfB3GRl1oOzeEFeGU5myxfG0Fzjb3YXTIDkvvGEoqxLaXyAwyuQwXfnkl/vbng2g72YeqxWaUVOXP9rQoFMoCJFJjnFa7pgSIateUApEa4+TaNYknUmOcTrumRERqjJNp15QKkRrjZNo1pUKkxjjSrmnv8BFcVtSAUo00a1akxhgAej19OGhtBCHrUKYpkcz+cn3Efjua7YfAkBDKNFWS2a/SrQ3bP4V252EwCKFMu0QS+wCXSg0A/Z5GDHiOggAoFWrXlAIKJjm/IV1+efZWqHPSv+94ndJlvMx1aKQ4BSJiWzYxCtQazrlNtabY7vUhEBLuh8dHJFLs8U/A7RefNqqQy2AKR6rTEdsqjohtjaWv8FySgbri8nBd8cCQ9K2ZSEy0uKMzsy2LCCFYvIxb2M6dGch6ba9Gq0LdilIAwPHDHXOmthgAikpzsbS+EgBwYM9JTPjPn5s8hULJLlSVOjFUlToxC02V+ngGUqnnu/gWZW5CI8UpYFSF64N9yTu4ueFIsd3rQzAUgjzJ9xFapQIquQy+QBDjbg8KcnRJH1Mhk8GoUcPm8cLqcEObP72310wUGHQYd3kxbHOhsjBX9P4AUJTHvRUbsbkQCAYhlwk1ik/CVr4eDCFwefxwuLww5KTfL9Vk1EKnVcLl9qN/yI7K0tTOU4iamkIcP9GDvr5xeL0TUKsy92dXU1eEIwc7YLN5YBmwoaTUlLFj8bGioRKtp/oxYrGjr2sU5dXpZQZIydqLF6O3YxguuweNH7fgwq0rZ3tKFAplgRJxjF+d5xHjV9tOYHweR4zf6TuGQa8Db87TiPGnw03o9Qzhw6GjGYkYEwA9nj4cGD2CTfmZixgftzUCQMYixq3OI5z9DEWM+z1HAUgbMc40/2vJH2AwGNK2Y7fb8Qikycyc69BIcQoYI6JZIqK+OUol5AyDEMuKaudECIlGi62piG3puQVmJMW64kJjuM+wLfVIsV6rgkalQIhlMTyeuh2Ai14X5nNz6h+SrjVTZRmX6tudAZVok0mLvFwdWJZFV9eI5PZjUSrlqK3jUrRbTvVn9Fh8aHUqLF1TDgBo2t+OUGjuRIsVSjk2f3kVAKDtZB/6OjIbuadM5+mnn0Z1dTXUajU2bdqEgwcTK3COj49jx44dKCkpgUqlwpIlS/Duu++Ksun1erFjxw7k5+cjJycHN9xwAwYHqRI5hZ+QhBkuC0F8i6pSC0NVqWe2H6tKfZyqUmcVJaOWbJwvUKc4BUyRtkxecc5tpK5YbH1wni5cG+wS7xQXRJxiR2rOaFFYgTodp5gQAnM4WjxoTT+FurSIi3j3D9nSthWhIuwU9/SPZSTtd9EiLoW6vT3zjtjS5dyb3p7uUTglVtROhhUNlVAq5bCNudB5NvtK2IkwV+Rh2dpIGvUpqkadRV5++WXs3LkTjzzyCI4cOYL6+nps27YNQ0NDvNv7/X58+ctfRmdnJ3bt2oWWlhY899xzKCsrE2XzwQcfxNtvv41XXnkFe/fuRX9/P772ta9l/Hwp84/Gnn68ceIUJoLSldEsBMeYqlILQ1WpZ7ZPVakp8wXqFIuFTEaKnRN+BNjQDMrBkyM3xbrifF0kUuyerjKdQDQXJEZB2uGecVuh/Qk4sS2XN/WFtjjsFFvGHHH2Z1Lt5iPiFA8M2SYdWNFKwPHDXGSAQi6DxzuBEatr2gbC80xOsbgm7BQPDdvhcPD//CNqzVOHWIx5OTCXmsCyQMsZy6QdqVSOZ/idUakVWLGeczyPHepAUKhOW6was+AxxW1fv2UxDLk6eFw+HPrwNFiwie0I/oxFDsHrKdHPJUlmS43y5z//Oe666y7ccccdWLFiBZ555hlotVo8//zzvNs///zzsFqteOONN3DxxRejuroal1xyCerr65O2abPZ8Otf/xo///nP8aUvfQnr16/HCy+8gM8//xz79+9P7QJSFh4st8Yd7ulDn82Bt5rPcN0CqCo1VaVOUZV6wGOjqtRUlTopVWrK3IE6xSmglsuhDNfFikmhNmlTixTnRpziFCLFkfTpcZcHgaA4oS4gLLYVrttNJ1ocGylONxJbmJcDuYyB1x/AmE2aN8oyGYOycP1td/+YJDZj0WpVKCnhnPlMC24BwNKw4FXrmQHJxcOSOv7qcmh0KridPrSczH4adyLkchku2rYKhBB0nbWg44x0b8XnAz87czueOHVzWuNnZ24HwNUaxQ6fQGmI3+9HY2Mjtm7dGv2MYRhs3boV+/bt493nrbfewubNm7Fjxw4UFxdj1apVeOKJJxAMR/GSsdnY2IiJiYm4bZYtW4bKykrB41LOT3RKJa5btQxKmQz99rBjLHXEmIpvCTIb4ltSR4xjHeP3Bo5mVHxr7/CRjIpvHbQ2ZlR8q9l+KKPiW+3OwxkV3xqgEeMFCXWKU4AQAlMKdcWRSPGYyLZMkfRpm8crWoE6R6WESiFHiGVhdaVaVxxJoU59Ack36iBjCHwTAYyn2CIqgoxhYA73Fu6TMIU6k3XFAKLtmNrbhzOuzFxemQ9djgo+XwDt57JfPymXy7BmYzUAoPlIF/x8fZNnkXyzEasvrAUAHPrwNBzj0j18nU9UVFTAaDRGx5NPPsm73cjICILBIIqL48U6iouLYbHwp9i3t7dj165dCAaDePfdd/Gv//qv+NnPfobHH388aZsWiwVKpRImkynp41LOX8x6PXWME7AQHWOqSj0JVaWeGapKvbCh6tMpYlKrMeRyYVyEaFaqPYd1MQrUY24PCkUoUBNCUKjXotdqx4jdjeIc8cqLRcYcnO0bwVAaTrGMYVCUq8fAqB2WUQdy9ekpTJYWG9E7OI6+wXGsXlKalq0I5aW5IIRgzOaGzeGBUZ++snUslZX52L+/DTa7ByMjThQWZq5XHcMQLF9VhsP723G6uQ91S0sgy3KWzqJlZpw+1gP7mBsnjnRj/eba7E5gBlZurMFA9wiG+8bx6bvHcMVNmyCTL/z3hP972YtpK1La7Xb8CMXo6emJs6VSSaNMCwChUAhFRUV49tlnIZPJsH79evT19eEnP/kJHnnkEcmOQ6HEEnGM32w+E3WMr121DIo0uibEQlWpE0NVqRNDValntk9VqSmpsvCfADNEJFI85knewY20ZfIGAnBPJB85I4QgPyq2Jf6taaSueDjFXsNFpnCfYZszrQhnSX64x/Bo+qrRFWYTAMAyLF1vYZVSjpIibo5dvdJHi5VKOaqquIWrtZ1fXEhK6paWQKmUw27zoLdburfJycIwDNZdVAcAaDnRC4ctvQwBqWEYgi3b10ClVsA65MCRT1tme0pZQUo1SoPBEDeEnOKCggLIZLJpqs+Dg4Mwm828+5SUlGDJkiWQxTgjy5cvh8Vigd/vT8qm2WyG3+/H+Ph40selUPgixn4qvhWFqlInJhsRY6pKndg+VaWmpAJ1ilMkN+wUixHNUshkMIQfGsVGi6NOcQqtlQqjTnFqN/18vTac+hyEzZ26mnGsU5xu+rBRr4FOq0QwxGJgWJrWTABQXc45rV29mXEi6+q4VM+OjuGM1/oqFDIsCStRnzzWk/GUbT5KK/NQUp6LUIjF0QPtWT/+TGj1amzexrVpOnusB92zkGp+PqBUKrF+/Xrs2bMn+lkoFMKePXuwefNm3n0uvvhitLa2IhRTMnL27FmUlJRAqVQmZXP9+vVQKBRx27S0tKC7u1vwuBQKAJgN8Y7x25lIpZ7njjFVpRaGqlLPbJ+qUlPmGtQpFklEcTCiQD3m9YZVoFn+MYVoz2GvZ0YV39gRdYrdU1Skk6BQzznFo043ggI1yYlUoGUME402D46HFw0Rc4+oTBfm5oAQApfXD4fHl1hlWshW5GtCUF5sAgD0Do4LmhESvRSiopyrKx62OuH0+Ca3n2E+044rMIrNRuh0KkxMBNHTO5akOrHQEFI6ntxm2aoyMDKCkWEHhgYFXh6Iti+kljx9EMJFiwkButuHMWixpayqLZU69NT/ymuKsGJ9NQBg/99PwmGfqtQ+83VO77qdH+zcuRPPPfccfvOb3+D06dO455574HK5cMcddwAAbrvtNjz00EPR7e+55x5YrVZ85zvfwdmzZ/GXv/wFTzzxBHbs2JG0TaPRiDvvvBM7d+7Ehx9+iMbGRtxxxx3YvHkzLrzwwuxeAMrchW/pZgVqjKkqNVWlTkKVmgWoKjVVpZ5RlTrISlOWQZEG6hSniCksmuXy+0WlVaVaV5yfE4kUi09BNWrVUMplCIZYjKWgYA0AxaawevR46m9SFXIZCk2ccz0wkn50t9ycCwDotYynbSuCVqNEUQF3rplIoSaEoK6WE9xqbc18VFKjVWLRYi46ffJEb8aPx4cpPwe1y7m678bPWhEKZT9iPRNrL1qMghITJvwBfPbucQRTUGqnJObmm2/GT3/6Uzz88MNYu3YtmpqasHv37qhQVnd3NwYGJiMRFRUVeO+993Do0CGsWbMG999/P77zne/gn//5n5O2CQD//u//jq985Su44YYb8MUvfhFmsxmvvfZa9k6cMq+h4luJoeJbiaGq1DPbp6rUlLkCdYpTRC2XR4UxxKRQ54Wd4lERtcgAkKebdMK9E+IWTEIICsLCVsOO1OqKi8N1xYNjjpT2jyBlXXFpkQGEENidXtidqad1T6UqwynUtbXcA3v/wDhcLukeHoRYsbocANDbbcX4WOpttdKhfmMNFEo5rMMOtM3BFkiMjMGWK1dDqZJjdNCOIx+fH/XF2ebee+9FV1cXfD4fDhw4gE2bNkW/++ijj/Diiy/Gbb9582bs378fXq8XbW1t+N73vhdXYzyTTQBQq9V4+umnYbVa4XK58Nprr9F6YoooqGOcmIXmGL9JVanjoKrUMzMXVal9Qa9k43yBqk+nQa5GgwGHA+MeD4o1ySlCR5xiq8hIsUouh16lhMPnx6jLjTKTOPXYQkMO+sccGLK7sDwFsebiXC56anW44Q8EoUpRibOkwIhjrf3oG7GBZdm02pYrFXKYC/QYGLaj2zKGVXXSqCNWlefhUFMnBoftcHv80GqkUe2MoNerYS42wjJow7nWQaytr5TU/lSMJi0qqwvQ3TmCE8d68IVLl2X0eHyotUqs2VCNxs9b0bS/DRU1BdCoFVmfRyJ0Bg02b1uNvW8dxdljPcgrMqB2ZdlsT4tCocwBqCp1YhaKKvXbfccwFFalvq6iHoVUlRrApGNMQFWphZhrqtSPnfqfUOWk/5zlc86tlpqZhEaK0yA3pq44WSJOscvvh0/km+CCcCumEaf4aF9RuCZ4KEWxrRy1EnqNCiyAwfHUo8XmPD0YQuDy+GF3pf/2qaKES6HuGRhL21YEfY4ahfk5YNnMRYsXL+EiVefOWbKSTryqoQIA0Nk+DLt9dlSgl6wuQ25+Dvy+AI7ua5uVOcxE+aJCrAn3Lz74wSkM94/P7oQoFMqcgapSJ2YhqFLHim/Nx4gxVaVObJ+qUlMSQSPFaZAbrisW05ZJJZcjR6mE0+/HqNuDUkPyvWoLcrToGB3DSAoK1EVG7m3niIMT25Ix4t+HmHP1cHh8sIw5UJlvEr0/wNUVF+dx/Yr7RmwwVapTshOhsiQXB493YWDYDv9EAEqFNL/SNZUFGB51or1rBMsXS/MmNJaqynwcUivgdvvR22tFZaU0b3OFyC/Qo6w8F329Y2g+1ouLvrA4o8fjg2EYXPDFJXjv9SNob7GgdqkZxaWmrM9jJlZtWoSxEQd6Wofw8TtNuPKWC6HVp/d7SqFQFgYRVepIxPjtSMRYLmHEePkKvHZ6SsRYorVNMGKslDBiXLcKu1qbYYuJGBsl6mEeUaV+rTM+Ypyn1khiP+IYv9V7PBoxvr6iHoUa6SLG20vqsXvgWMYixpcU1WPv0DH0eoYzEjHemNcAWIEeT39GIsYrDGsBAN2eDpywNYJA2ohxjY6L6Pa4T2ckYlwSjhj3uY/MasT44RXPwGAQl1XKh91ux3/idQlmNPehkeJEzKAmHXGKrSnWB1vd4pzbyUixeKfYoFFBpZAjxLL8bZ0SiPhGRkke58BbEtQVC6pYx9gpLTQCAPpHbMLHE2LKdkaDBoYcNUIsi94hW9Iq0zNRU1kAABgaccDp8iZ1fcQMmZyJtmdqOTuQcNuIWvPUIZbVDdyi0t46CKfLN2lHKlXkJM67oMSIuhVc/v6BT84iEGKnn5fY+Qj+Dgl8Ifgx9x9DGFx0xWqYCnLgdfux950mBILBtO1TVWoKZY4ioA7N+whAVampKnUSqtSJlKmpKjVVpY4dmUIlU0s2zheoU5wGeTG9isX0gM3XRtoriXOmCyIK1C7h1kpCEELSTqE2504qUKeT8lsWcYqH0+9XDACVpbkApE2h1mqUMBdxb9jau0cksxvLknAKdX//eFZSmguLDTCXmhAKsTh5fHaUqAFg7YWLoNIoYB9z48yxnlmbRyIUSjkuvaYBKrUC1iE7Drx/clb6PFMolLkJFd9KzEIT38pEH2OqSp3YPlWlpmQb6hSngUGtBkMIAqEQHCJu9vnhSPGoyEixQa2CQiZDiGUx7hZfj1tkCCtI21JTIM7N0UAplyEQDGEkRRVrACg06aCQy+CbCGAkxbnEEq0rtowhJKHjsqiqEADQ3pUZp1ivV6OsjJt7S4slI8eYyuq1nKjXuTMDcEmo2C0GlVqBdRfVAQBONHbCMUs1zjORY9TiC1evBWEIOlssOHmoY7anRKFQ5hDUMU4MdYwTQ1WpZ7ZPVakp2YQ6xWnAEAJTOFosJoU6LxwpHnG5RUWfCCHRaPFwKmJbRi5SPJhipJgQEk2hHrCmLrbFMAxKCrgobN+wLWU7EcwFeigVMnh9AQyPptcyKpbq8jwwDMHYuBvW8cy0Mlq2jKvDaW21YGJCuocpIcylJhSXGBEKsTh+pDvjxxOiZkkxiktNCAZCOPhRy5yNwpor8rDhEk6t+9jnreg6m52XF5mEtmigUKQjW47xfBbfurF2NUzUMeYlW45xuaaIOsYC9qljTIlAhbbSJE+jgdXjgdXjRbUpyX20GhAA3kAA7okJUQIXRTk6DNgcGHK4sMxcKGquxWGxrTGXB75AAKoUWj2U5BnQNTSO/jE76mtSF1YoLzSh2zKGnmEb1i5Or+0NwzCoKMlFW/cIuvqtKC5IX1gAAFQqBSpKc9HVa0Vr5zAuWJtc2y0xlJXlQq9Xw+Hwor1jCEuXSC/qNZWGDTXY/XYT2s5ZsGJNOUxGaQRKxEAIwaZLl+IvLx+CpW8MbWcGUJdKr7AssKS+AvYxF1qauvH5e83Q6FQoCkf45yP/9/h9UOakJ6rjd0r30EyhzHf42jVdE3aUpUBQfGsetWu6gUd8S8p2TXziW1K2a4oV33pjHrZr2lK4Fp8OH82Y+NaG3AYQAvS4M9euiQGLbk8HjtsawQIol7hdEyFsxsS3zJo1IISNim8REJRo10hmnyINNFKcJnkRsS1v8m8mFTIZTBouwjziEvdGs1DPOWbDKaQva1VKGDTcIjSUYtpySR7ncA6MpVcPXF7E1RUPWh2cQEmaVJXmAQA6+6ySRh1rq4sAAG1dIxlpnUQIwfKwM3j6zEBWIqaFxQaUV+aBZYFjjZ0ZP54QeqMWazbWAACOfN4K9yylcyfDui8uRfmiQoSCIex96yhso9JFCigUyvwnokodiRi/nYmI8Txv13RjBts1aWehXZPUEePtJfUZjRhfUlSf0XZNG3PXZrRd0wrD2mi7phMZiBjX6NZktF1TSbRdE4FKlnznGUr2oJFisUwRhY2kQotVoM7XajHm8WLE7UZlninpwxZFnGKnCyGWhUxIqlng42JjDuweHyx2B8oLjJObJ+mLFRp0kMsY+CaCGHW4UWCYEj0VEs2dYt+Yo4Feq4LD7UP/iB1V5rwpdvgnJOQzlptNkDEEDpcPY3YP8ozhN8RCKr5JnnB5qQkqpRwe7wT6B20oL4mPELJTTyxiPoGE9tR9FtUW4ejRLthsHvQPjKOsNMkopOAhZv4hrN1Yg95uK7o6RjA64kR+Ac8bb9H2ZzzsFDMslq0tR3f7EEaHHNi/twWXXb0mwWHFKjILHljU9oQlkDEEW66qx55XD2N4YBwfvHEE275+AX+rJiENPMHfOZ75ZFB9+t/W/EfabRrsdjtewJ8lmhGFMruQEDdiYQVCBonuHnwR42tXLYOCJ2IstEYIrymTqtSvnUkjYhwSOAOGFRcxTmCHn0lV6l1tJ5KIGPPbF7r+IbBRVerXu1KPGAvdvhkgqkr9dt+xaMT4+op6FAhEjIMh/l8iGcN/lIgqdWzEeFtJA0o0Rt7tZQKzFTwHwqlSfzJ8FH3hiPElhetQrs0T2EMIgfMiwPrcdWBB0OvhIsYX5K1HudbMb4YVyqQQeJlEOFXqEAh6Pe04bmtEiGVQoa3g3Vwu+McqcIUIp0rNsgS9nlNodR5BCAwqtHUCdsTdJORsEMXqehgVVVDLTGGlbdppYi5BI8Vpkq8Ni2Z5POIUqHURBWpxbzJNWg0UDINAKIRxkerVwGQK9aAttTecDEOiKtT9VntKNiKUF5kAAD3D42nZAbj+x6XFnL2ufmva9iLIZAwWVXHtmVo7hyWzG4tSKUdtHReRPnNGujericjN06EmfMyjhzuzckw+GIbB5i8tByNjMNBtRdvp7Jx/KsjlMlx6bQMMuTq4HV588MYR+L0Tsz0t0dAWDRRK5qDiW4mh4luJoarUM9uf76rUaplJUnsU6ZhVp/jjjz/GNddcg9LSUhBC8MYbb8R9z7IsHn74YZSUlECj0WDr1q04d+7cjHaffvppVFdXQ61WY9OmTTh48GCGzgAwqtUgAPzBIFwTyT8gF+gmxbbEwBCCgnC0eCiFFOpiY7itks2ZcqpuaTiFuj9Bv+JkiDjFvUPjadmJUBWOsHb1SecUA0BtTWHUrn9CuoePWJYt41Koe/vGYMuSGnP9+ioQQtDfNwZL/3hWjsmHMU+HtZu4lKjGz1rhnKNq1ACg0ijxpa+uh0angm3Uib1vNyEgQfo/hbLQWQjrfbJQxzgx1DFODFWlntn+fBffWqg8+eST2LhxI/R6PYqKinD99dejpaUl+r3VasV9992HpUuXQqPRoLKyEvfffz9stsSiu8msD1arFbfeeisMBgNMJhPuvPNOOJ3i/i5n1Sl2uVyor6/H008/zfv9j3/8Y/ziF7/AM888gwMHDkCn02Hbtm3weoVrD19++WXs3LkTjzzyCI4cOYL6+nps27YNQ0NDGTkHOcNEFahHPcnfdCORYqvbI7rncFEaTnGBXgsZQ+CdCKTU1gkASvPDTrE1vbri0gIjGEJgd/tgc6bvCFWW5oEAGB13weGSrj61IC8HJoMGwWAoYz2LDQYNysPiTadO92fkGFPRGzRYvJxLa2o81DGrCtBL11Sg0GxEYCKIzz44k5H6banIMWhw2XXroFDKMdQ3hk/+cgyhoLi/YQrlfGMhrPdioKrUiaGq1ImhqtQz26eO8dxj79692LFjB/bv34/3338fExMTuOKKK+Bycf5Kf38/+vv78dOf/hTNzc148cUXsXv3btx5550J7SazPtx66604efIk3n//fbzzzjv4+OOPcffdd4ua/6w6xVdeeSUef/xxfPWrX532HcuyeOqpp/Av//IvuO6667BmzRr89re/RX9//7Q3zLH8/Oc/x1133YU77rgDK1aswDPPPAOtVovnn38+Y+eRr42kQifv2BlUKijDPYfHRNYjF+vDKdAptFaSMUy0X7FlPLVIb6FRB4WMgW8igFFH6ouMUiGDOZ+LXHdLEC3WqBQwF3IOe0evtDf3xTVcqvHZ9sw9bK1cyalwt7UNwZultNw166qgUMgwOuJER1tm0sOTgWEILtq6HHKFDMMWG04enb12UcmQV2TApdc1QCZj0N8xgs/fa57TjjyFMtvMl/VeypeDfI6xn4pvRYmoUmcyYpxN8a03eo5heJ45xlsK12ZUfGtDbgMqtJl1jCPiW5lyjDMpvpVJfEGfZCNZdu/ejdtvvx0rV65EfX09XnzxRXR3d6OxsREAsGrVKrz66qu45pprUFtbiy996Uv44Q9/iLfffhsBgXtXMuvD6dOnsXv3bvz3f/83Nm3ahC1btuA//uM/8NJLL6G/P/lA05ytKe7o6IDFYsHWrVujnxmNRmzatAn79u3j3cfv96OxsTFuH4ZhsHXrVsF9AMDn88Fut8cNMUSdYhGRYkJINIV62CnuJl0cdmqHnS7RUWYAMJs4RzRVp1jGMFEV6r7R9OqKK4u46Gj34FhadiLUlHMtBqR0igGgrroQhBCMWJ0Yy1DP4uIiA/LzcxAMhnCmJTu1tRqNEqvqOZGKI4c7ZzUVOMegwcYvcC0Qjjd2YmQovd+tTFNUlosvfKUehCHoOmvB/veb52y/ZQplLpOt9X6mtb65fxB/aW5JaV0VgqpSJ4aqUieGqlLPbD9WlToTjnGmVakzxf1Hd+Luxv+V9rj/6E4AmHbv9Plm/juNpEXn5QmLudlsNhgMBsgFBAOTWR/27dsHk8mEDRs2RLfZunUrGIbBgQMHZr5YYeasU2yxWAAAxcXFcZ8XFxdHv5vKyMgIgsGgqH0ALgfeaDRGR0VFWMmO8AweJsW23Pz7CNgpzOHSoEdcMU5WEvsaNCqo5HKEWBYjbv5jsuAfAFASdooHbEk4xQJzKAunUPdabWAJokMslWbOKR6w2rkHhZnmn+B6AkB1eT4IAUbGXLAnSMmOnXMy81drlKgMpze3dAxNbi/hIAyDVSvLAXCCWxOBEKdCLLA9SwjvEMuyVWXQ6lRwu3w4dbJ/0o7QEAtD+AfPSdUsNaOytghsiMVne07DHwimfF6C8xc9eKcKAoLymiJ84ap6EELQcXoAB/ecCv+e8l03QUNJ32solIVIttZ7wbWeBVw+Pz5t7ULn6DjebT6LUCAUVaWOHUKLExEYYAVSqQNBATuEdyQ6bkSVOq7GeCIgvJDyjRDhHxBwjP1+cXYEjz2pSh3rGNt8PrAseAbhHUKLeohFVJU61jG2ej2CtvhGSGCwLImqUsdHjF0C8+dUqfmG0CWKqFLHOsYDHhuCLMM7xJ4DwKlSx9YY97qtgtsLDcH5gMH63HVxqdS9bguCIPyDlfGOEAv+AU6VOjaVusfdgwDL8I4Q+EeQJbwjBE6VOjaVusfdmmD+/NchABnvmC9UVFTE3T+ffPLJhNuHQiE88MADuPjii7Fq1SrebUZGRvCDH/wgYZpzMuuDxWJBUVFR3PdyuRx5eXkJ/b+p0JZMAB566CHs3Lkz+m+73T65WCbBZKSYU6AmST7AF6YotkUIQbFBh26rDUMOZzSdOlkiCtTjLi+8/gmolQpR+wNAWR7XImDAakcwFIKMSe39ilGnjrZm6hu2obpEbGuAeDQqBUqLjOgbtKG9ZxQNy8rTshfL4kXF6Oq1oq1zGBvqqyCTSf9OqbIyH3q9Gg6HF2fPWbByRZnkx5iKXC5Dw4ZqfLa3Bc3HelC7pBhaLV+LjMxDCMEFlyzBiMUGh82Dxs9aceGly2ZlLslSWVeMi7evxme7j6O1uY97M37Z8qTvAxQKJTskWut1SiW+smop3mluQZeVc4yvWrUk5bVtKmLaNaVCpMY4rXZNCRDVrikFIjXGybVrEk+kxjiddk2JiNQYR9o1vdnTlLBdk1giqdSRdk3vDRzFlaVrUaw2SWI/UmMcade0d/gILitqQKkmXxL7kVRqAOj19OGgtRGErEOZpkQy+8v1EfvtaLYfAiEhlGuqJLNfpVsbtn8K7c7DYBBCmXaJJPYzwS8afp52+0WAu0++hN+jp6cnzp5Klfhvc8eOHWhubsann34qaPfqq6/GihUr8Oijj6Y9TymYs5Fis5kTABocHIz7fHBwMPrdVAoKCiCTyUTtA3A/WIPBEDfEYFSrISMEgVAItiTSCaLz1YV7DrtcotMuI46wJYW6Yo1SAZOWEwezpNiaKd+ghUohx0QwhGFb6unEhBBUFnMRWKlSqBeVcy2U2nukFcUqM5ug1Sjh8wfQLbHCdQSGIdFo8alTfQhmScCppq4IBYV6BCaCOHqoIyvHFEKlUmDz5VyqUtvpAXSeG5xhj9mnemkJNl+xGgBw7kQvDu89Q1OpKZQkydZ6P9NaX55rxFdWLYWcYaKOsaSp1FSVOiFUlToxVJV6Zvux4lsnbI3oPY/Ft1QylWQDwLR7ZyKn+N5778U777yDDz/8EOXl04NTDocD27dvh16vx+uvvw6FQjg4l8z6YDabpwksBgIBWK3WhP7fVOasU1xTUwOz2Yw9e/ZEP7Pb7Thw4AA2b97Mu49SqcT69evj9gmFQtizZ4/gPlLAEII8DZdCPSKi73CeVgOGEPgCQdhFONMAJsWyUnCKAaAkN9JWKbW6TULIZAr1iC0lGxGqwk5x1+CYJI5EVVkeGIZgzO7BaBoO+1QYhmDxonA/4XPJp2OIpba2CFqtEm63H62t2XEICSHYsLkWANB+bghDlvR+puliLsvFqvXcG979H53BuDUzddxSsmh5KS788koAwNljPTj4wWnqGFMoSTCX1vuF4hjP6xpjqkotCFWlntk+dYxnD5Zlce+99+L111/HBx98gJqammnb2O12XHHFFVAqlXjrrbegDnfwESKZ9WHz5s0YHx+PCnoBwAcffIBQKIRNmzYlPf9ZdYqdTieamprQ1NQEgCumbmpqQnd3NwgheOCBB/D444/jrbfewokTJ3DbbbehtLQU119/fdTG5Zdfjv/8z/+M/nvnzp147rnn8Jvf/AanT5/GPffcA5fLhTvuuCOj51KgDdcHi3CKZQwTFdsacop76DeHneJxj5erHRJJaS5XV9yXolMMAOX5XAp172h6DlRJgQEKuQwe3wSGx9NfPFRKOSrCtcptErdQWlpbDEIAy7Ad1gwJbslkk7XFJ5p7sxYtLiwyoG4p90btwGfnEJLwQTAVVm+ogbksF8FACJ+814wJf2Z6REtJ3cryqGPc2tyL/X8/SVWpKRTMr/W+PNeIr6zOrmNMVaknWQiq1NNrjOeXY5wVVeoMim/FqlJnyjGej+JbmWbHjh34/e9/jz/+8Y/Q6/WwWCywWCzwhLvsRBxil8uFX//617Db7dFtgjH3wGXLluH1118HgKTWh+XLl2P79u246667cPDgQXz22We49957ccstt6C0tDTp+c+qU3z48GE0NDSgoYGrA9i5cycaGhrw8MMPAwC++93v4r777sPdd9+NjRs3wul0Yvfu3XFvFdra2jAyMun43HzzzfjpT3+Khx9+GGvXrkVTUxN27949rUBbagrCdcUjbnGOUlFYbEusU6xRKmDUhFOg7eJVpEvDkeIRuwu+FJxqACgv4JziwXEH/CnaALiXAxVFJgBA54A0acl1VVwKdWvPiKTROp1Whcoyru45k9HiJUvM0GiUcLl8aGvLXs/NdRtroFLJMT7mxpmT2emXLATDEFy8dQU0OhXs427s39syLyKvtSvKcNG21SAEaD/Vj892H8/ai41k8AZ9kgwKRQzzbb3PuGNMVakTQlWpE7MgVKnzMusYx6pSn6Cq1FnhV7/6FWw2Gy699FKUlJREx8svvwwAOHLkCA4cOIATJ06grq4ubpuenp6onZaWlqhyNZDc+vCHP/wBy5Ytw+WXX46rrroKW7ZswbPPPitq/oSdD0+ZWcZut3PKah+9B3XYaY0S4hfP6bXa8NqZ0zCqVLh9bcOM25Pw2nrSMoQPWttRbjTgq6tXRD8X2j6W90+1omVwBBsry7CpJl4YjAj8VGPt/P6To7B7fLi6YSmq83P5d5hhPn/c2wS724tt65agpjgvqePy2W/rG8EHjedg1Klx05fWCr6tEbw+U44bCIbwp7cPwz8RxJVfXIHSIuMUO/wTFZp/7HUYGLRh94cnIZcxuPna9VApeYRMEjw7CV6jKX+Kp0714dChDuh0Knz1unXThb2StDPT9piy/bkzA9j/yTnIFTJcf8N6aHWqhNuLtS/WzrDFhvffOAI2xGL9xXVYtmbq77pI+0IIRnRTm3936yA+/esxhIIsSqsK8IWv1EOu4BHV4Tmux+vG3Q/fHG1VIAWR+9p1f/tHKHTpieNMuPx484rnJZ0fhZJtIn8TP37rfWh0U9b68NLdO2bDO80tCIRCqMozCYpvsYKLFv/HkW4HFocDb4YjxaUGvbD4lqAdoYWA+58RtwuvnTkFTyCAIp1OWHxLSBdQ6HOGO+6414NdZ0/COeFHnlojLL41gx2h7R1+X1R8y6BUCYpvCekaEuEFFwDgCfij4ls6uVJQfEvIjqB9AAxh4QsGouJbKkaO6yrWolCt491e6BxkDP8DBQEQCAWj4lsKIsO2kgaUaIy828sEHqCEzoEhLIJsKCq+JSMMLilch3ItvyAqk8CO0HxYlsXhsaPo9fSBgOCCvPUo1/LXgMoE1mIZ4X+ZxBAunfe04yh6Pe0AgFWGjajQ8gvpygWuDyPwQCcjLFiWRZerCb2eUwCARTkbUKGtS3r+TocHly36XxlZ66WyKbW9ucycrSmeb0QixTafT1QaVGykWOz7CXNYRdriSO3tY1mk13AaKdQV4WhxT5p1xRVFJjAMgc3lxXiCVkrJIpcx0Z7Fbd3DaduLxVxkgMmoQSAYQmuntLZj4aLFCi5a3J69aHHdUjMKijjRrcMH2rN2XCEKzUasv4hbZI7sa8PQwPjsTihJKuuKcdm16yCTy9DfNYIPXm+E3zsx29OiUCgiWCg1xlR8i59s1xi/2dOU0Rrj9waOZlR8a+/wkYzWGB+0Nma0xrjZfkjyiHFsjXG78zCNGM9jqFMsERqFArqweppYsS0ZIfAHg7B5vaKOaTZwdcGDdidCKQT8IynU/dbUneLKQhMAoGd4PK3UVqVCjrKwg90hUQp1bVUhZ6/XioCEDxmEECyv49oInD5nyVhKr1wuw8pIbfGJ7NUWE0Kw6eLFIATo7BhBf580quDpsGR1OarquP7Fn75/Eh7X/EjfLakqwNavrYdSJcdw/zj+vuvwrM/92Q0/wu82PZXWeHbDj2b1HCiUbEId48RQxzgxC0V8a6GoUh/PQCo1Fd9aGFCnWEIKIy2WRNQVyxgGBeFo8aBDXF1xvk4LhYyBPxiEVWSvY2AyUjycRl1xaZ4BMobA4fFh3CXOqZ9KpEdxR780N1tzgR45WhUmAkF0SdxCqba6EEqFDHanFz0ZjFwuXWqGWq2A0+VDa1v2WhPlFeRg6XJOnGD/Z60IBKR7QEsFQgg2XboUxlwtPC4/Pn6vGcFZnlOyFJbm4ss3boRaq8TYiAPvvXQAtllU01bLVJIMCuV8YqE4xvO6xpiqUgtCValntk8dY8pMUKdYQgrDKdTDIh1Ucw6XBj0oMg2aYQjMei5aPJCC2FaOWgWTVg0WqadQK+QylIad666h9CKK1SV5IASw2t2wSZBCTQjB4nC0+KzEac4KhQxLajkxl5MtmROkkstlWLOaq385drwnq85pw/oqaHUqOB1eNB2RbvFIFYVCji9uXw2lSo6RQTv2fTh/egHnFhpwxc0XQG/SwuXw4m9/PojB3sz0uqZQKJmBqlInhqpSJ2YhOMYLSZX6OFWlpkyBOsUSUqgVHykGgGJ92Cl2ir85lhjDTrFNvFMMAGV54bZKadQVVxWF+wwPp+cUq5WKaAp1u0Qp1IurOae4f8gGp1vatNXli0tACDAwZMfoWOYif0sWF0OnU8HjmcCZFukWiJlQKOW4MFzLe7q5D8NDqf+OSIXBpMUXrlgFwhB0tQ6hubFztqeUNHqjFld8/QLkm43weyfwwWuNaD89uwrfFApFHFSVOjFUlToxU1WpM+EYU1XqxPapKjVFCOoUSwUBCsNp0KNuN4JsiJMGJKzAQHQUh3sODztdMfslN0pMMU5x7HdJUp7PRXl7rQJCWUnMoTLcTsky5hBMw2YFxlRbNWWcOFZ7/yj/wQTmIGRfr1OjpJA7x3NdMdFiQngHS8A7+I6Zk6NCdQXX+qn57ABYQqJDzM8weg4CQyaXYW19JQDgRHMffP5geL5CdgjvEAtLCMqq8lFTVwSWBT7/5BwCQVbw2gmfm9D2Yu1ww1yRiwu+uAQAcPxQJzpah/jPV2gIwRD+ITQRkfMnINBoVfjyjRtRubgYoRCLfe814/iBNu53bNpxKRRKNuBbogUXFRYoN01PpQ4FQiAhTBtCNoQeDcAKpFIHggJ2CO9IdNxCjQ43TK0xnggkPOdpI0T4BwQcY79fnB3BYxPoFeppNcY2nw8sC55BeIfQghtiAbVseiq11esRtCU0QgJDySh4+hi7BOYPBEMM7xC6RAyRT4sYD3hsCLIM7xA7f0COLVNqjHvdVsHthYbgfMBgfe66uFTqXrcFQRD+wcp4R4gF/4AMS/Xr4lKpe9w9CLAM7wiBfwRZwjtCkKNS2xCXSt3jbuWde0jMA7tIpGq/eD61YOTR5aekilGlglIm42p8PZ5ojfGM+6lVUMvl8AYCGHG5o5HjZCjW54AAcPj8cHh90KvF1fpFIsVjLg9cPj90KvGtWgxaNXJzNBhzetAzMo66kgLRNiJUl+Th02PtGA2nUBtzNCnbirC4qhADw3ac6xzC2mVlICk4iEKsXFqCju4RdHSPYEN9FbSa9FrdCLFoURFOnurD+Lgbzc09WL++JiPH4WPjhbUY6BuDbdyNY0e7sH5DddaOLUTdilLYx9043dSDfR+eQY5ejQIzfxuKuYZcLsMXrqpH02fncPJwB04caIfD5sGFW1dCJqfvKSmU+UCkxvid5paoYyzUrikVIo7xm81noo6xYLumFIjUGL925lTUMRZs15QCEcd419mTUcdYsF1TCkRqjCPtmna1nhBs15QKkRrjSLumVztPCLZrSoVIKnWkXdObPU24vqIeBerkn/8SEUmljrRrem/gKK4sXYtitUkS+5Ea40i7pr3DR3BZ0VqUalJ//oslkkoNAL2ePhy0NoKQdSjTlEhmf7k+Yr8dzfZDYEgIZZoqyexX6daG7Z9Cu/MwGIRQpl0iif1k+Ob+f027/SLAtWA8X6BPYBJCCImpK04+nZYQEnWELSJTqJVyWVSoK5UUarVCjkIDt79gtDgJqsLR4s6h8ZRtAOEU6kLOuWmTSHCrujwfCjkDh8sHy4i0KcCF+XoUFegRCrE4fS5zqc0MQ7BuXTUA4NTpfjid6YmaiUGlVmDTxYu5Yx/vwchwaqn6UrP2wlqUVxcgFAxh7+4TcNjTr0PPFoQQNGxZgk1bV4IwBJ1nBrDntdlXpqZQKMmzUMS3qCo1P7MhvjU8z2qM41Wpm9DvGZHMPhXfomQb6hRLTFFENEuEUwwAZn3qPYdLwynU/bbUHL7ycLS4Jy2nOBcA0D0ylvZDwaJS7k1jW780N1eFXIaacs5mS4f0/X5XLeNUms+cs2BiInNCWOVluSguNiAUYnHkqHQ37mSorC5AdW0hWBb45KOWjJ5nsjAMwUVfXo7cghx4PRP48J1j8Hrm1xvNxavKcdn166BQyjFqscFpmz+OPYVCWTiO8byuMV4AqtTF0T7G8098izrGie3PlmP8+wt/gFcv/kna4/cX/iAr850LUKdYYorDKdNiRbPM4briAYf4KFypkauZ7RtPLYJXkR92ikdtKav5FufqoVbK4ZsIwjKWXiSxujQPDEMw5vDAapdGwGp5WCm6o3cUHt+EJDYjVJTmwaBXwz8RREubRVLbsRBCsHEDd+Pu6BjGcJYjthdcVAetVgmH3YPGgx1ZPbYQCoUcl161BtocFRw2Dz569wQmUmwvNluUVOZj+y2bcNH21SgsNc32dCgUikioKnViqCp1YlQyOb4yz1WpuXZNsY4xVaWOtT8bqtRStV88n1owUqdYYorDkeIRt1vUohiJFNu9Pk6QQgSlYQXqMbcHbr94h68kVw85w8Dtn8CoM7UbPUNINFrckWZrJpVCjsqwrXN90rxxLMjNQUGuDqEQi7MSR4sZhmD1sjIAQPOZfgSC0j0MTSU/Pwd1dZyDf/Bwe1ZbEqlUClx0yVIAwNkzA+jrmRsthbQ6Fb50dT2UKjlGh+z45L1mBDP4M8gEhjwdqpaYZ3saFAolRagqdWKoKnViqCp1YhaaKjVDpNEGoEgLdYrFIiAZGVH9M6g40awgy2LEnfwNUymXIz9cj8yXQi2o3gxAo1QgT8cJUvWPz5BCzaOKK5MxKAv3Gu622uJVl0VQUxx2igetCIGdUb05keheXTjdua1vBCwm1bqTVbHms7+8lnM6WjoGEZwyv1TON5ba6kLotEp4vBNobR9KqJ4s/LPk32HqHNc2VEEul2FkxImOThEvDQTnlLyKckl5Lpat5F4AfP7JOXh8gdTVnkWrUgvP05ivw2VXr4FMzmCgZwz7PjiNIMuvxC2ZKrVEqtqE57+EfxzzmKeffhrV1dVQq9XYtGkTDh48KLjtiy++CEJI3FCr1XHb3H777dO22b59e9w21dXV07b50Y9+lJHzo8xTRKpDU1XqBIOqUlNVaqpKnVCVut60DUXqJdH5UOYO1CmWGEIIiiIp1CLriksiKdR28WmxZaZICnVqdcWTKdTjKe0PAOUFJshlDJxeP0Yd6b1BrSw2QSGXwenxY9AqTZrwovJ8KBUyOFw+9FnGJbEZQSZjsHo55yweP92LkIQRgqlotUqsXl0OAGg80olAILv1vQ0bq2E0aeHx+LH/k7NZjVYnosBsxCXbV4MJ9zA+PIfmRgFefvll7Ny5E4888giOHDmC+vp6bNu2DUNDwpkbBoMBAwMD0dHV1TVtm+3bt8dt86c//WnaNo899ljcNvfdd5+k50ahRFgoNcZUfIufbItvvdnTlNFU6vcGjma0xnjv8JGMpFJHHOOD1saM1hg32w9JHjHWK6RR6KZID3WKZ+DYsAXvtJ3BoCv5m1IkhVp8XXFYMCuFuuJyE+fUpu4Um7hjjzlSXmDlMgblBdw8OgbTS62Vy2SoKckDAJzrlSaFWi6XYXF1EQDgdAZqfxfXFEGjVsDl9qNVTAQ3BZavKIVOp4Lb7Ufzyb6MHmsqcrkMWy5dBoYh6OkaxdnTmVPdFktJZR4u2soJWpw71Y9jB7KbYj4f8AR9kgyx/PznP8ddd92FO+64AytWrMAzzzwDrVaL559/XnAfQgjMZnN0FBcXT9tGpVLFbZObmzttG71eH7eNLsl2eRRKKlDHODHUMU7MwlOllj6Vej6Lb1HmLtQpnoEu2xhax63odybvbKYqtlUSdoqHnC4ERC6gEQVqq5vrNywWk1YNvVqFEMuiz5p626KaYs6R7bCkX29aV8a9TWvvH5GsRnT5Iu6huscyDrtTWqVfuVwWVaJuOpXZaLFcLsOGcIum5pN9cDiy16IJAPIKctCwsQYAcHh/G0bnSJsmAKiqK8IFl3C9AE8e7UZzY+fsTmiOceOn38dVe7+X1rjx0+8DAOx2e9zw+fgfPP1+PxobG7F169boZwzDYOvWrdi3b5/gXJ1OJ6qqqlBRUYHrrrsOJ0+enLbNRx99hKKiIixduhT33HMPRkenP3z96Ec/Qn5+PhoaGvCTn/wEAQkf8CkUPhaKYzyva4ypKrUg1DGe2T51jM8/qFM8A0XacNTXnXwqtDkcKR71eOALJr+IGNUqaBUKhFgWgyJbM2kUChSG+xX3jtlE7QtwN4CqAhMAoGskdaGs6qJcMITA6vRgLE2ns7TQCJ1aCd9EEF2D6Yl3RTDqNSg3mwAAJ1uljxYvqzNDo1bA6fLhXMew5PZjqarKh9lsRDAYwsFD2Y+ILl9VhoqqfIRCLPbuOQWfxKre6bB4ZRnWXVQHADh+qBMnj9DFLBNUVFTAaDRGx5NPPsm73cjICILB4LRIb3FxMSwW/r/DpUuX4vnnn8ebb76J3//+9wiFQrjooovQ29sb3Wb79u347W9/iz179uDf/u3fsHfvXlx55ZUIxjgH999/P1566SV8+OGH+Pa3v40nnngC3/3udyU4ewolMVSVOjFUlToxC0+VOjOO8XxVpabMPeSzPYG5TrGOc3CH3MnfiLRKJQwqFew+HwadLlQajUntRwhBqUGP1lEr+u0OlIVbLSVLRa4Rw04XesZsWFZUKGpfAKgsMKG5dxBdI+NgWRYkkfiQACqFHGUFRvQMj6PdYsX6ujLRNiIwhGBxeQGaWvtxtmcIi0rzU7YVy8q6EvRaxnGucxjrV1ZAqZDuz0Aul2H18jIcPNqJplO9qKsuhEyWmXdPhBBsumAR3n6nCb19Y+jusaKqUpprlOzxL/riUvzljSNwOrz47OOzuGzripR+bzLB8voKBIMhHDvQjqYD7WABrFpXNdvTmnV2bXkEBoO4e8tU7HY7zHgaPT09cbZUKulaN2zevBmbN2+O/vuiiy7C8uXL8V//9V/4wQ+4vom33HJL9PvVq1djzZo1qK2txUcffYTLL78cALBz587oNmvWrIFSqcS3v/1tPPnkk5LOlzL/OTcwgs6RcVy+shaMTJr7WMQxfudES9QxvmrVEjCMNOtCRJX6zeYzUVXqa1ctg0IujYBPxDF+7fSpqGN8/bIVUEu0bkYc411nT0Yd4xuWrIROqZTEfkSVeldrM2x+L3a1nsCNdathlOhvP6JK/VrnCYz63Hi18wRuqF6NXJVWEvsRVeq3eo9jyOvAGz3HcH1FPQrUOZLYj6hS7x44hgHvGP7a34QrS9eiWG2SxH5ElXrv0DH0eobx4dBRXFbUgFKNNM8qEVVqWIEeTz8OjB7Bpvx1KNOUSGZ/hWEtAKDb04ETtkYQAGUa+iyxEKGR4kQQREWzrF4P/KEEb2CnKMpGosUWEbXIwGR7pT6HPTn15phRkRcWyxqzxSk2J6d8DJTlGSBjCJxeP6wuD+95zTgA1IZTqNstMW8ERdqJzGtxJVcD3DM4DqfXn/C4fPAJSZaajTDqNZgIBHG2c0o0V0A5mGXAO/jmsnSxGVqNEi63Hy0dQzzqxylcV77BEJhydVi5khPdOnionUufE7qmPErMiefDfy1it1Gq5fji1uVgZAS93VacPMFT3yyk3iwWQdVogUGAVeursOaCGgDAsQPtaG7sTHAdhOyLnI/gYASGwClkCI1MJckAOCGs2CHkZBYUFEAmk2FwcDDu88HBQZjNybWiUigUaGhoQGtrq+A2ixYtQkFBQcJtNm3ahEAggM7OzqSOS1n4EBZwe/344FQbzllG8PfmVoSCLFWlpqrUKatSj3rdVJWaqlInoUotxR2MIhXUKZ4BnUKJHAX3xnJYTAp1uO+wRWRdcWk46mKxOxESmQ5bYtRDxhC4/BMYc4uvMVXIZCjL5RzrruE0UqiLc0EIMOpww+ZKr9bVlKNBcZ4eLIBzPdKkIxNCsLKOexA/1Tog+jrPhFzGYM2KsBL1qb6Mq0OvWVMOvV4Nt9uPo0e7M3osPvIL9Ni4uRYAcPRwBwYttqzPIRGrN1SjflPYMT7YgROHO2Z5RucfSqUS69evx549e6KfhUIh7NmzJy4anIhgMIgTJ06gpEQ4AtDb24vR0dGE2zQ1NYFhGBQVFSV/ApQFj06lxLbVi8EQgtbBUc4xDkm3NiyUGmMqvsXP1Brj17uoKnUsVJU6+3iCfsnG+QJNn06CIp0OznE/Bt1OlOmSSzuMKFBbnA5Rqcj5Oi0UMhn8wSBGXe5onXAyyGUMSo0G9IzZ0DM2Hu1dLIbqQhO6R8fROTyGdTWppT6rlQqU5RnRO2pDm2UU62pTT6EGgKWVRRi0OtDSPYS1tSWSpOfWVRXicHM3HC4fegfGUFmal7bNWJYsKsaJ031ctLhtECuXlkpqPxa5XIZNm2rx97+fxJkz/VhUU4CCfH3GjsfH4mUlGBqwoaNtGB9/eAZfua4BGq006W9SsGp9NQghaNrfjuOHOhEKsVizsWbOpHqfD+zcuRPf+ta3sGHDBlxwwQV46qmn4HK5cMcddwAAbrvtNpSVlUXrkh977DFceOGFqKurw/j4OH7yk5+gq6sL//RP/wSAE+H6/ve/jxtuuAFmsxltbW347ne/i7q6Omzbtg0AsG/fPhw4cACXXXYZ9Ho99u3bhwcffBDf/OY3eVWqKec3NYV52L5mMXYfP4fWQe6BfeuqOjCpZLbwEHGM32mOT6WWSZVKrY9PpX4rkkotkzCVetkKvHZmSiq1PMOp1AoJU6lrV2NX24m4VGqDUppU6ohj/HpX5lKprymrx9t9x6Kp1NdV1KNQwlTqK8xr8TdLU8ZSqb9Q2IBPho+iL0Op1BtyGwAAvZ6+jKRSL9dH7LfjuK0RwNxNpb527+OQ69L/3Q64pHt5NNehkeIkKI6IbYlIhS7U6cAQAk8gAJuAIisfDCHRfsV9NvEq0BXhSG93CmJbAFBdyD0oWmxOuFNQsY5QW8Ld5Fr7029NVFOaD4WMgd3lxcBo6srYsSjkMiyt4UR/TpyVvqWQTMagfmUFAODYqV74JzKrdltW9v+z999xUlxX/jf+rs6Tc86BicwMzBAFAgQIkFCWg/xdry05rb3S7k+2v4/T4yCv5dWuH3vtdVh7vQ5aW84ICQkJEBJJZBgYJuecc+5J3f37o6aHCV09Xd09wwD1fr3uS6j61qlb1T1169Q593MCSEgIwWKB8+er3abW7SiCILBpyyr8/D0wjk5w6r2yZR/DYmTmxrF2OqJdnN9A/rlqpVzTMvLhD3+Y73//+3zzm99kzZo1FBQUcPTo0RnxrcbGRtrabv4t9vX18elPf5r09HQefPBBBgcHOX/+PBkZYskttVpNYWEhjzzyCCkpKXzyk58kLy+P999/fyaNW6/X8+c//5nt27eTmZnJd7/7XT7/+c/zy1/+cvkvgMJtgdUxViLGtlFUqe2jqFLbR1GlVljJCBblqXABg4ODopLq6WMYvL1oGOjntapS/PUGns7Mtb2TeeGb5L8VF9M2NMz9CUlkhMwVvhJs9AfAAvnNLZyvbyIhMICHMlKn+9vuPn979/AIf75ShEal4tNb1qGZ/wbaATt/u1RE1+AIO9ITyIxaWBfU3nis9scnp/jf9/IxWyx8cEsWwT62I96L2bHy/o0ayhtEsa3deasctyPx6xYsMDI6zl+OXMdisfDIzixCAr3FNVo27dj+wN5xzWYLrx25zuDQGDmZ0eRmxU7vI2FL7rHnbR4bm+T11/MZH59iTU4MOdmxC8Zk247EB7KvhYXBgVHeeqOAyQkTicmhbNmWIh2NlXrQlHtLknxgtb29oqiZq+9XAZCYGs7GHamoVCo710HmeGRfz4U/otGxUT7z/36QgYEBl0WxrFjva+6w6U5bCgq3Cuvv+PuvHcdjXu3q2q5ejhVWYbZYSA4Luhkxlpq6pYLJEtub+wc4XFTBlNlMXKD/TMTYIhWqkHnc9qEhDk2rUUf6+tyMGEvakZogbG/uMo5wsEx0iEO9vG5GjGVeB1S2j9s/ZuRAZQnDkxMEGjxuRoxl2pHqPzQ5NiO+5avTz0SMpaYrQfL62N5unE6h7hkfxUujmxMxlrIltV1lY/u4aWomYqxXaeaIb0mdg1pl+4HFVvcps2kmYqwV1HMixmqJBx854zdZzJztuk6zsQu1oJoTMbbV3952W+OxWCzk912jydiKgDAnYqyWmIzVgu2XR7YSRSwWCxVD12g0isuxsvzyiJ6OGGskro/KxgP4yNAoDyR/Zknm+vbeLrfYHBwcJDww5K6Y75VIsQNY6w73j48xNuV42ZkIHzGFtW1YXg1Xq+p06+Cg7EhWkJcnnjotU2YzrQPO1Y5NDJ0Wyup0fl2xXqshLtQfgCo3RIvT48U1wPVtvRjdVPrHy1NPUqxYC7mwwoZAlIuoVAJ52eJNsri8lZHRpU1BMRi0bNwoRkILi5rp63N8Dby78PXzZPvOdAQBaqs7KSlqXnynZSZ1dTSbrWOsaOfM0WKmJpd23beCgsLtRUJoIHuXOmK8lOWafJchYnybl2v6wG1erumR6OwlLde0LyJnSSPG20NzlrRc0/rApS3XlOG7ZqZcU9EKjBh7qHVua3cLilPsAAaNFn+9AYB2GaWZ5jjFdkRy5yvOhnp7o1WrGJ8y0T0i7yYqCAKxgf4ANPT2L1SZdmAMCdNOcXPvABMSk5yUiuFsO6siRYezqq1H0rmXEnWcP6Zgfy9C/L0wWyxUNHUt6OConfnKvtkp4lrf+pZe+gddq6tsi7joQEKDfTCZzFwvahI3SilcI3FNJU7CVt+4+GBiogMxmy2cu1AtBlFtqEY7pEotE4tKwKISiIgJZN10mvK1K/U0NUhMdJLfj+3rI4mkurVEEyAxLZxt+7JQq1W0NPTw3psFjI1PLbEqtdSQbClSy7/+CgoKTmLjZipYINFWKrWiSj1zXEWVWuLBY7opqtSKKrUjqtQKKwfl23CQcC/564qtTnGP0SjrDapKEGZUqJudWFccN12aqbGvX/a+AAFeBvw9DZgtFhq6nbMBEBsagE6jZmRsgtZe19cCp8WLqdzlDR1uWwsa4OdJbEQAAIWV7o8WC4LA+jXxAFTVddLbv7TRW0EQ2LQpCa1WTU/PMKWl7j8nR0jNiCQlXUxVev90BX29yx+1XozohGB2PpKDTq+hu2OQd16/xsiwa2rpCgoKdxbKGmP7KKrU9lFUqe2jqFIrrCQUp9hBwrxEB1dO3WFPrRZ/gxhhlptCHT2dQt3cPyBrP4DoQD8EoHfEyNCY/Ju7IAgz0eJaF0ozadQqEsNFO1Vtrt/kkqKC0WrUDI6O09It/7pIkZMmqmNXN3QzvAQpzqHBPiTEimtlrlyvX3JxJ09PPevXizfo6wUNtySNWhAE1t+TRHikP1OTJk4cL8E4uvJk/UMj/Ln/8Vw8vPQM9o3yzsFr9PW474FCQUHh9udOdIwnFMd4BtuOsftekC6X+NbciLEivmVFEd9ScBTFKXYQa6S4fXRIllMTOR0tbh2U6RT7i9HelsEh2ZOjh1ZL6HSd5Ibefln7WkkME53Z+u4+plxQEV4VJaZQ17T3uGQHRMXoVdGivZK6dpdszSY0yIeIEF8sFgs3lmBtMUBedhwqlUBrxwCNrc6/aHCU5KRQoqMCMJstnHm/4pYoQatUKrbtSsfH18DI8DgnjpcwuQLX7voHerH3iVx8/T0ZHRnnndeu0drovglZQUHh9udOc4zfVFSp57BQlbr4tnSMFVVq2yiOsYIjKE6xg4R4eqGeKbHk+I0yatopbhmSlz4c4uWJQaNh0mSiY0j+jS0hSEwJrnfSKQ719cLboGPKZKaxxzkbAJGBvvh46JmYMlHX0eu0HSuZ04JbjR19DI26b8JamxENQEV955IIYvl4G1idJq5fvnS9zuUXBIshCAL33LMKg0FLf/8o+fl1S3o8KfQGLbv2rkZv0NDTPcyZk+WY3fig5y68fAzseTyX0OnI9qm3CykvbFZKNikoKMyQEHKbi28tRyr1bS6+9eQc8a3i265c00PzIsa3o2O8lOJb6wKWVnwr3WftjPhW4UA+zYpjfFuhOMUOolGpCJ2uV9w24njUN8pHTIPuGBmRNfkIgkDMdLS40YkU6rggfwCa+waccsAEQSApbLrWcIfzNyVBEEiZjhZXtHQ5bcdKgI8nUcF+WIDS+g6X7VmJCPEjPNgXs3nposXZ6dF4eugYHhmnpKJ1SY4xGw8PHVu2iOWrysrbaG5Z+gi1LXx9Pdi5O1MUtWrq5cLZqhXpbOoNWnY+lENiajgWC+Sfq+Ly6VsTZV9KjFMTbmkKCncjiiq1fe4Ex1hRpZZGUaVe3P5KV6VWkEbjys7j4+Po9Xp3jWXlMSPjKBLu7U3byBBto0OkB4fO62vbhK9Bj7dOx/DEBO0jw8T4iY6u1BQ6W3Q2NsCPqu4emvoH2BwbI7GD7c3B3p546XSMTEzQMjBA3HTkWPrACzclhwdyo6GN+u4+Js1mNOqb71AkxXFt2E+NDiG/uoWmngGGxsbx9rj5m5GszSth3yJAZmI4Ld0DlDd2kpsWjUatlixdKOd812ZGc+R0KeX1nWSnR+E1e5wSR7BI1Ea0VaZOq1Ozfk0cpy9UcaO0maSEELw8Zx1D6hWVC89D0dGBpKdHUFbWxrnzVTzy0Bo8PByT17fYKs6H7XMTd7B9LSyCQHC4H9t2pXPqeAk1VZ3oDTrWbUiQMiSxXepHIdFfYvyS5rGg1qjYtDMN/yAvrp2vobqsjcH+Ue7duxrDvOvmNn1oW4akxu4G9pz4d9Sert23TUtcXkxhZXGnz/XzpnrA/l3Iqkp9tLBq5qXxTB1jG/1tYc+NtqpSHy6umHGMrXWMF9iRmDfsHdcaMT5UXD7jGM/UMV5gR2LukzgDgZuq1AfLS2cc45k6xo5iljgDlWXGMT5QWTLjGM/UMZZhxzY3VakP1BQxMDHGgeqimTrGtvpLIfWJVZXaWsf41foiMYJs8JS0ZQupqdiqSm2tY/x60w0ejVlDiMHLZn+T2faPSKqusVWV2lrH+EhrAXsj1hLh4WfbjsRIpcavEkRV6ve7rtNi7OJk53W2h+QS7RkosYcUEuclQF5ALhYEmo0tXOq5xobAPKI9w22bsSz8uxCReJkkiKrUZgSajbUUDuRjtqiI8Vz4HD8lWZhc4VYg69s4cuQIH//4x0lMTESr1eLp6Ymvry/bt2/nu9/9Lq2tSx/9upVEeFtLLDn+1k0QhJvriofkrSuOCRBvMB1Dw4zLfNMqCALx09HieifTn0P9vPE26Jh0MYXa19NARIB4DSrdULM4JiwAb08945NTVDe7bs9KRIgv4cE+mM0WCsuX5recEBdMWLAPUyYzVwuW5+1hXl4C/v6ejI1Ncu5C9S2L0kbHBrH53hQASouaV2QNY5hOgVoTy4792Wh1ajrbBjj2aj79igCXwl3C3T7XO8KdtsZYUaWei6JKbR9FlXpx+3ejKvVLL73E+vXr8fHxITQ0lMcee4yKioo5fX75y1+yY8cOfH19EQSB/v5+t9jdsWMHgiDMaZ/97Gdljd8hp/i1114jJSWFT3ziE2g0Gr785S9z8OBBjh07xq9+9Su2b9/Ou+++S2JiIp/97Gfp6nI9TXYlEuEtpk93G0dkKTda1xU3D8pbV+yj1+PvYcACNDlRmineuq64p88pR2h2CnWNCynUIEaLASqau1x2ylSCMLO2uLi2zW1OniAIrM0Q3+RV1HUwYnR/NEwQBDbmiRHS2sZu2jvdp6IthVqtYtu2VFQqgZaWPsrK3Xfjl0tSSji568Xzz79cR3Wl+1Lg3U1UXBB7n8jD29eD4aExjr12jaba2//e9s7OL3Nuzzddau/s/PKtPg2FJUCZ6+VxJzrGiir1TZbbMVZUqeeiiG/dfpw+fZpnn32Wixcvcvz4cSYnJ9mzZw8jIzeroIyOjrJv3z6+9rWvudUuwKc//Wna2tpm2ve+9z1Z4xcsDngUmzdv5utf/zoPPPAAKhvpO1ZaWlr4yU9+QlhYGJ///OdlDWQlMTg4iJ+fHy+dOYrBe266ya8L8hmanODJVZnE+M5KFZFI0RHM0DM6yiuFhagFgc+uX49GpQKTRP9538bpmnoKW9vJDAtlV3KiTfs2McOkycSvz+UzZTbz1Losgr29JPtLpTF39A3z6uViNGoVz2zPm0mvkrQjsX1i0sTvTuQzZTLzyMYMIgN97duR+lVO9x+fnOJP7+QzaTKzb2MasaH+ssYjZd9itvDW6RI6uodISwxjS26i/fFI/PlIfi8AFjh/pYaKmg78fD14dG8OarUKQa4tqf4SYy0rbeXylVpUKoEH9mYRHOwzMx45CFIPfVK3kvlpiRYL+ZdqKStuQRDg3h1pxCeGyLazaH8pJO3b3j5unOT9Y8V0tPYDkJYTw9qNiajlpjjLGOfo2Aif+cqTDAwM4Dtds9xVrPc1d9h0py2FlcPdOtf/4NXjeHjNnestUn/eNrbXdfVytLAKs8VCcljQ3FRqO0uBHLXf3DfA4eIKpsxm4gL956RSS2Zfyjhu+9AQh6Yd4khfn7mp1JJ2JO5nNvp3j47w6rRDHOrlNTeVWsZ1AGymPfePGTlQWcLw5ASBBo+5qdQy7Ej1H5oYn0ml9tXp56RSSy4lAwTJazR3u3E6UtwzPoqXRseT8VkE6G+mUkvZkdqumrd93DTF4ZYbdIwNoVdpeCwmh2CD9yw7tocplT49v/uU2TSTSq0V1DwQuYYwg/9NOxIPMY6O32Qxz6RSqwUV94WuJdIjSLL/Ytvnj8disXC17zrNxhYEBDYG5RLlEXGzv8TDh1qw/QJp/qOBxWKhbOg6zcZaALL98ojyiANgZGiUR1d9Yknm+raeLrfYHBwcJCIoxKkxdnV1ERoayunTp9m2bducz06dOsV9991HX18f/v7+LtvdsWMHa9as4Uc/+pEsW7NxaIHHhQsXHDIWFRXFv/3bvzk9mNuBSG9fKvq6aRkenOsU2yHQwwNPrZbRyUk6hoeJkvGjigvwo7C1nYa+fiwWC4K9O/A8tGo1MQF+1PX0UdvdR7C37fUk9gj19cLXQ8+gcZz6rj5WhQfLtgGg06hJjgimvLmTsqbOGafYWfRaDWlxYRTVtnGjulXSKZaLIAisy4zlrdMlVNR1kpUSia+3wS22Z5OXE0djSy8Dg0YKS5tZmxXr9mPMJy01nPb2fhqbejl1poKH9udg0GuX/LjzEQSBvI2JTE6YqK5s5/1TFag1KmJigxbf+RZgFeAquFRL2Y0mym800d0+wL33Z+C1BL8NBYVbhTLXO4dVlfqYA2uMncEqvnW4aPE1xs4gZ42xM1jFtw6WubDG2A6y1hg7gVV860B1sQNrjOVjjRjPXmM83zF2Basq9ew1xvMdY1ewRoyPdxTQahTXGM93jF3BGjE+23Wd5uk1xvMdY1ewRowFLDQZW7nUc22BY+yq/XSftaiw0Giso3AgH2DGMV4qdh3//1zWD4GbGiKD87Jd9Xr9oloTAwNiNmRgoNz14PaRsvuHP/yBV155hfDwcB5++GG+8Y1v4Onp+N+R03fUiYkJKioqmHJjKsztQOT0uuLWYcfTmQVBIHraEW4ckJcuG+3nh1olMDwxQe+oUda+AEkh4g+mpsu5ckiCIMw4wlVtrqWtpMeI4mS17T2MTbj+u1mdFIEgCLT1DNLZ576UoPAQX6LD/bFYLFwraXKb3dnodRo25k7L9pe10DfgvpQpKQRBYMs9q/DxMTAyMs77ZyvdmuondyybtiSTkBSCxWLh9HtlNDe6XrJrqVCpVeTek8y2favR6jR0dwzy9oGrtCj1jBXucO7WuV4uiiq1fRRVavvcCarUe8MVVWp79merUk+Yb78KDjExMfj5+c20l156yW5/s9nM888/z5YtW1i9erXbxiFl9//8n//DK6+8wsmTJ/nqV7/K73//ez760Y/Ksu1Q+vRsRkdH+ad/+if+93//F4DKykoSExP5p3/6J6KiovjKV74iawArEXvp013Do/yh7AZalYrPrdmIyhq5lVI4nJ6zijs7OVFbS4SPDx/MzLSTfrzQzqHiMhr7BtgSF0tedKRN+wvtiP8dm5ziN+fzMVssfHRDDgEeHnbHactO7/Aof75QiEoQeHp7HgatZtH0Zlt2LBYLB84V0TM0yj3pcWTHR8i3M6//qWvVVDV1kRARyP3rUm30l5eKa71uPf0jvP5uIQCP7soiOMD2G1WpVGLJ84KZc7NYLLz3fjlNrX2EBvuwf+dq25kAMtOkF+vf2zvC22/fwGQyk50dw9ociSi1ZOaXm9KbLWA2Wzh7soyGum5UKoEd92cSHR0g244sZKZPz7c/NGDk7DvF9HaJwnmZuXFkr4+fSTeVvD4yxqOkTyvcau6muf4HBxamTzub9mw3ldoN9u2lUs+x42Ratd1Uaofs2E9L7h4d4WB5KUZbqdQO2F8sHdpuKrUMO1L97aVSL9hF4hj20qrnp1LbU6V2Jq163DQ1EzHWqzR2VamdSauen0ptV5XaibTq+anU9lSpnUmrnp9KbU+V2pm0aovFQu9EJ0H6MEBMn34i5enbIn26qalpjr3FIsWf+9znOHLkCGfPniU6OnrB586mTy9m18qJEyfYtWsX1dXVJCUlOWRbdqT4q1/9Kjdu3ODUqVMYDDdTB3fv3s1f/vIXueZuO4I8PNGp1UyazXQbRxbfYZqY6R9Sx/CwbCGL+AB/ABocUGibj0GrIcpfPHZtt3N1agO9PQny9sRssbgkuCUIwky0uKyx0y0CWdnJ4kuCurZeBkbkR9KlCPL3IilWjJBfKWpcEsVmQRDYlJeIRqOis3uI8up2tx/DFoGBXmzenAxAYWETTc23LkKrUglsvS+N2PhgzGYLp46X0Ny0ciPGAD5+Hux5Io+U1VEAlFxr4L03ChgdHrvFI1NQcB93+1zvLHei+JaiSn0TRZXaPooq9eL2rQ7xUuOp0bmtAfj6+s5p9hzi5557jsOHD3Py5Em7jqtc5NjduHEjANXV1Q7bl+0Uv/766/z0pz9l69atc6JamZmZ1NTUyDW3KENDQzz//PPExcXh4eHBPffcw5UrVyT7nzp1aoEktyAItLe7x+FQCQKRXmIKdcuQ4ynUfgYDvno9ZouFFpkq1HGBYuSsdXBIdmkmgKRgawq18zeOlAjRQaxsc60EUkpkMBq1ir4RI6298hW15xPo60lMmD8AN6rdWyYkLzMGlUqgtXOAlo5+t9q24u2lZ12OuK7kamEjw8tU/zUpKZTUVHG9zPtnKxkcdN8LBbmoVCru3TnLMX63lOYVnpasVqtYf28KW+7PQKMVyza99dcrNNR03uqhKSi4heWe62HlzffOcic6xooq9U0UVWr7KKrUdy8Wi4XnnnuO1157jRMnTpCQkHDL7BYUFAAQEeH42nDZTrFV8Ws+IyMjskSgHOVTn/oUx48f5/e//z1FRUXs2bOH3bt309LSYne/ioqKObLctsbsLJHeYuS1Rca6YoAYPzGFpEnmumJ/DwP+BgNmi4WmfvklfBKCRae6Y2iEwTHnbtyrwoMRgLb+IfpdiMjqtBpSIkUHu6TBPeV4claJEbvK5i63llHy8TKQniSmzVwqbFiy9bdpyeGEBvswOWXi3JWaZasjvH59AiEhPkxOmjhxsowJN6zzdpb5jvHJd8uor1v55V7ik8N48IPrCAzxYWJ8irPvlHDuvTImxidv9dAUFFxiued6WJnzvbPcaY7xm0sUMb6t1xjfAY5x2EzE2P1rjBXH+O7j2Wef5ZVXXuGPf/wjPj4+tLe3097ejtF4029ob2+noKBgJoJbVFREQUEBvb03swR37drFT3/6U4ft1tTU8J3vfIf8/Hzq6+t54403+NjHPsa2bdvIzs52ePyyneJ169bx1ltvzfy/dXL81a9+xebNm+Was4vRaOTVV1/le9/7Htu2bSM5OZkXXniB5ORkfv7zn9vdNzQ0lPDw8Jlmr7yEXKJ9bjrFchyYuGmn2Jk06PhAfwDq+uSnQHvpdUT6idHtaiejxd4GHTFB4vjL21xzVlbHiY5mXUcvQ25wYiOCfAkP9MFstnCjxr3R4rXp0eh1GvoHjZTXLU1NXUEQ2LohGbVaRUt7P5W1yxNtVKtV7NiRjqenjoFBI2duofAW3HSM4xNF8a33T5ZTU7Vy6xhb8fHzZM/juazOi0MQoK6qg8N/u0p7i3PLFRQUVgLLOdfDyp3vXcGqSn3bim8tRyq1Ir4lyXI4xg/Nixjfjo7xUopvrQtYOvGtO5Gf//znDAwMsGPHDiIiImba7CU3v/jFL1i7di2f/vSnAdi2bRtr167ljTfemOlTU1NDd3e3w3Z1Oh3vvvsue/bsIS0tjS9+8Ys8+eSTvPnmm7LGL3vm+Nd//Ve+9rWv8bnPfY6pqSn+8z//kz179vDb3/6W7373u3LN2WVqagqTyTRnPROAh4cHZ8+etbvvmjVriIiI4P777+fcuXN2+46PjzM4ODin2SPM0xuNoMI4NUXvmONR02g/PwSgb2yMwXF5N86EADHaW9fbj9mJSOKqUFG6vsqFFOq0KPHte0Vrt1NjsBLo40lUkC8WoLjRPWluuSni2oKyhg5Gx9036el1GnIzYwC4VtLE+BJFU/18PcibLst0qaCO4ZHlWZ/q6anjvh1pokPe0se16/XLclwpVCoVW7enkpwShsUC585UUlpsP0q0ElCrVeRsSOT+x3Lx8fVgdHicd9+8wdVzVUxNuu8hUkFhuVjOuR6WZ75fbK5v6Ozj/ZI6t2brKKrU9lEcY/vcCarU+yIUVeq7BYvFYrM9/fTTM31eeOGFRfvU19fzwgsvOGw3JiaG06dP09PTw9jYGFVVVXzve9+TLTQm2yneunUrBQUFTE1NkZWVxTvvvENoaCgXLlwgLy9Prjm7+Pj4sHnzZr7zne/Q2tqKyWTilVde4cKFC7S12f5RRkRE8Itf/IJXX32VV199lZiYGHbs2MG1a9ckj/PSSy/NkRmPiRGdIATbTa1WETFdmqlpeGB6u0Wi3dzPoNUQ5i2qGDcODti2L0GEnw86tZqxqSnah4exCNNqkQ62pNBABKBzaIR+4xgWmNMcsZEQGoBeq2FkfIKmHok0bgfHkxUvRovLmjuZMJtmzmfR85IgMsSP0ABvTGYLhTVtC+wtsC+Brb6piWH4+3owPjHF9dLmeecr2Gx2jy3R0lMiCAv2YWrKzJlL1Zix2O0//zuc+S4ldpDqHxziy5YtqwAoKW2lsrpDlEmUOq4g2GzS37ftayTVX1Cr2HRvCunTQlZXL9Vy7Wr99O/Uhh2VRJNCcjzyxmmrb0iEPw98eB3JGaIAXHlRC28duEpHx4Cd6ybRFBRuIcs518PyzPdSc71gAaNxguPXqyhp7OBMcR0Ws8XmDVNqqpe6wQoWSLSVSm1yj30sEO2/MJXaPGVGMLOg2Run1HFtRoynTBJ2BJvN3nFDPLx4cv4a48kpu+e8oJkF2w0Jx3hiQp4dyWML+GgNC1Opx8exWJBogs0m9dBgUC+MGPeMjUrakWpmiaZTaW2sMR6RHL/JrLLZpC6RStAsiBi3GQcwWVQ2m9zxg4at81Kpm0d7JftLNcnxoCIvIHdOKnXzaDsmBNvNorbZzBZsNoWVg+ySTMtNTU0Nn/jEJzhz5gxqtZrc3FxSUlLIz8+nrKzMIRvbt28nNjaW3//+9zY/Hx8fZ3xW5HZwcJCYmBheen9hSSbrTfZyazPnWxtJ9g/koeQ0yRJC80s1XWxq4nJzC8mBgexflbKgu2CSeBi2wNHyKqq6e8iLjuSeeDGqKF3aaeG21wtKae4bZHNCDHmxUXP7S/wK5tt5v7yeosZ2ksMC2Zu1cPyLlTqyYrZY+NOpAoaM42xbnUBGTJjd/ottxwIN7X28c7kcjVrFR+7PxaDTSpbHsWfHFi1t/Rw7W4YgCDyxJwd/H7G0ldyySIsde3BojEPHCpiaMrN+TRyr06KkS/w4WLZqsTFZf3EFBY3cuNGIIMDu3auJDPeTHKft40p8IPcaWawfWyi+0UTB1XoAVqWGs/EeifImtpA727hYqmn+By0NPVw6VYFxRLy3pGZFs2ZjIhrt3PImtq7b6NgIn/nyE0pJJoW7iqWe76Xm+v/423E8PL2obO3mRFE1FiA9JpRtmQkL109LTdFSt6VZ2x0q1+SCfUfKNTlbqgkcLNckaUdq4rj5T4fKNTlwHeYwq8SSQ+WaHLAj1d/Rck3OlGoCFpRrejI+iwD9wnJNzpRqAhaUa3osJodgw8JylM6UaoKF5ZoeiFxDmMF/oR0nSjUBC8o13Re6lkiPIMn+jm63jmd+uaaNQblEeSwUcZJTqmlkaJQPpC5NSSZ32byb5nvZkeKdO3fy7W9/e8H2vr4+du7c6ZZBzSYpKYnTp08zPDxMU1MTly9fZnJyksTERIdtbNiwwa4kt16vXyA1vhjRvqLD0Dwkc13xdC2upoEB2SnICdMq1HW9fbL2s5LsjhTqyBAAajv7GJtwXkxIJQgza4uL6tvdkq4WG+ZPkK8nUyYzRTXuTW+JDvcnJiIAi8XCpRv1brU9G18fAxvWJgCQX9hIb5/jZb9cJScnhsTEECwWOHWqjL5lPLYtBEEga00sm7auQhCgqqKdMyfKmJq6PdKRo+KCeOip9SSliZNmRVEzb/31Cu23sAQWwOjUhFuawp3Ncs/1sPTz/WJzfUpkMDuzkhGAsqZOzrg7lfoOE99SVKnncqeIbymq1LZRxLfufGQ7xadOneKnP/0pjz32GCMjNx+aJyYmOH36tFsHNxsvLy8iIiLo6+vj2LFjPProow7vW1BQIEuS2xHCPL3QqlSMmaboklGvOMzbG71azbjJRPuwvJtNXKA/AtA7aqTfKH/NaVJwICpBoHt4lL5R5xSkQ3y9CPYRaxZXtLtWniktJkQszzRspLl7wCVbIN6wclPFdLji2jZGx9z74L4hOw6VINDc3k9D69I5NimJocREBYjlic5Ximlqy4AgCNxzzypCQ32ZnDTx7olSRkaWp0SUPValRbBtZwYqlUBjQw/HjxQxNnZ7qDvr9Fo27Uzjvoey8fTSMzxo5L03b3DhRBljo7fGsdzy1g9Zc+jfXWpb3vrhLRm7wvJxq+Z6uLXzveIY20dRpbbPneIYK6rUtlEc4zsbpyQa3333Xdrb29m0aRP19fVuHtJcjh07xtGjR6mrq+P48ePcd999pKWl8cwzzwDw1a9+lY997GMz/X/0ox9x6NAhqqurKS4u5vnnn+fEiRM8++yzbh2XWqUi2keMFjcOOu7QqQSB2Olocb1MFWqDRkO0v3jMmh75TpmHTktMgLh/RYfzDm1GtJjqXNLc4dLDgl6rIT1aFO+6Ueeem0pceAAh/t5Mmcxcr3KvQJO/jwerU8SHrQsFdUvmrFrVqD09dAwMGbl4rW5JjmMLtVrFzp3p+Pl5MDo6wfH3ShhfAeWFYhOCuX9fFjqdhq7OIY68UcBAv/seBJaayNgg9n94A6syxWULtRXtvPmnS1SWtNxSxW8FBXss51wPK2e+T4kM5r7sm47x6WL3O8a3tfiWokptlzvBMVZUqaVRVKnvXJxyiiMiIjh9+jRZWVmsX7+eU6dOuXlYNxkYGODZZ58lLS2Nj33sY2zdupVjx46h1WoBaGtro7Gxcab/xMQEX/ziF8nKymL79u3cuHGDd999l127drl9bLG+8p1igPgAfwDq++WnQScFBQJQ3e3cH3hqmFgjuLKz2+lJPiUiSIzwjo7R1j/klA0rWfHhCEBzzwDdg66n6wqCwIYMcb11eX0Hg25WcV6THo23p56R0QmulTS51fZsDHot2zdPpw3XdVFTv3w1e/V6Lbt3Z4qlmgaMnDi5MlKWwyL82PdwDt7eeoaGxnj7zQJab6OyRzq9hg3bUtj7eC4Bwd5MTExx5Uwlx16/Rk+Xa39Hcji3//MUPPpll9q5/Z9ftvEq3DqWc66HlTXfp0TddIzLm5cgYqyoUtvljnCMFVVqSRRV6qXHXUul7qblUrKFttRqNW1tbYSGihG+F198kRdffJEvf/nLvPjii5jceFO8VVgXlb901obQ1iwhrB7jKL8vKUAtqPhczgY0tmojmhcqEoxOTvKrq/kAfHJtLt46nd3+cFM8aXRigl9fEpU1P75+LX42RBxAWsxpctLEb87nM2ky8+TaTCKm6xdLCltJ/DpOFtdS1tLJqvAg7s9atehxJcWfzHC8oIqath4xbS0nedH+jth/+3wpLV0DJEcFszN31cIdpESn7OtcANDU1sc7Z8sRBHh0VzZB/l52+y/4TOrBx8bm60WNFBQ3o9GoeHRvDr7TAl/iMZZWgKu/d4QjRwuZnDQRHR3IfTvSxPqfMoWn3CbANY3ROMHp46V0dQwiCLB+UxJp02rPDuHm8Thj32w2U1Xcwo3LtUxOmBAEWJUZRc7GBHR6LUbjCJ/50uMrVnzjbhLeuFu5m+b6//irKLQ1h+mpeI74VnQo21bbEN+a1X8+iwlkOSS+5YJ9R8S3wHkBLofEt+zasS9g5ZD4lh37iwlnOSS+5YAdqWMvEN9KysJXL/Hc5oQA13zxrcfjsggyLBTfsmfHnoDVfPGtR2PWEGJY+Mxjb/z2BLjmi2/tjVhLhIdtkU9nBLjmi29tD8kl2jNQsr+c7WrBvEB8a0NgHtGe4bb723jIGB0a5UNpf78kc33iy/8vKk/D4jssgnl0jNqnv3tXzPeyI8Xzfeivf/3r/OEPf+AHP/iB2wZ1uxBo8MBLq8VkMdM24nikx1OrJcxLvKnITaH21OmI9BUdWWdSqLVqNYnB4g2h0oUU6szpmsU1Hb0uCW4B5CSIKcnVbT0MG93zJnX9dLS4uqWbXjdEoGcTExFAfHSgWEc3v9alms2LkZMZQ1iIL1NTZk6dr8Rkct+b/sUICPBi5/Ra3ubmXs6dr3JrpMRZPDx03L8/m8TkUCwWuHyhhssXam6rNGSVSkVqdgwPf2QT8avEmszVpa0Yb9E6YwWF+ShzvcicNcZLETG+w9YYL0nE+E4S36pZ2lTq1xqWVnzrUFPBkqZSH2u7vqRrjE93XVvSNcaXe/NXXMRYwXFkR4obGhqIiYkRI0azKC4uJj8/n49//ONuHeCtwNFIMcCxuirKerpYFxbF1qi4hcYkIr+XG5u52NJMUkAAD6WkLtp/9ouqGy3tnKmtJ8LXmw9mrbbdX7JEFDT09vNmYTkGjYZn7slFrVI5FeH928UiuoZGuCcljjVxEXaPu1jk941LpbT2DpIdH8E96XEuR4oB3rtSSW1rD7FhAezbkDb3QxcixQAjxglePVrA5JSJzWviyUiOsNt/zmcyIsUAIyPjHDp6g/GJKdJTItiUmzB9jCUu1TS9uamph5OnyrBYIDU1go3rJSIlyxQpvtltumTTlXoAIqL82XZfGnq9drEdl2Q8TtvHQntzHwN9I6RmRQMokWKFW87dNNfbixRbWbRckwullMCBiLGL9heLGLtSqgkWRowfnnaUF7ezeKkjcCBi7EKpJnAgYuxkpNiKI+WanC3VBI6Va3K2VBMsLNf0aEwOIfPKNTlbqgkcK9fkbKkmcKxck7OlmsCxck3LHSlu6+lyW0mmiKCQu2K+lx0pjouLWzBJAqxevfqOmCTlcnNdcb+s/azrihsHBpiS+dY2aTrS2zY4zNC4/DeOMf5+eGq1jE1N0dDbL3t/KxnTIlklTa4JbgGsSRRvHqVNHRjdJOyUlx6DIEBjRx+tblC3no2Xh451WWI0+kpRI0NuXrs851ieeu7dKKaVl1W2UV3XuWTHskVMTBBbt4g1qSsq2rhy1b2REmexlmzavisdjUZFW0s/bx0qoKfbfW+xl4vw6IAZh1hBYSWgzPVzUVSp7aOoUtvnThDfUlSppVmJqtSeGp3b2t2CQ07xE088weDg4My/7bW7jVhffwA6jSOybqChnl54abVMms00yRTq8tbfTKGuckJwS6USSA0XBbdK25wXcEqJCEanUTNgHKOxp99pOwAxwf6E+HkxZTJzo949NxJ/bw/S40Sl7AslDW535NKTwggL8mHKZOZsfu2SOooxUYHkZIpO0/mrtXT3Lq/jl5gYyj2bpx3z8jbyr9WvCMcYIC4+mH0P5eDtY2B4aIwjhwuoLG9bMeNTULhdUOZ6+yiq1PZRVKntcyc4xooqtTSKKvXtj0NOsZ+f30yakJ+fn912t+Gl1RE6nXYlJ1osCAKJAQEA1PbJV9BNCRHTPiqdVKHOCBejvA09fYyMO3fT16rVpEeGAFDY2O6UDSuCIJCXJDp9xQ3tGF1cp2wlLyUGrUZNz+AIVc3uVXAWBIF71yWhVgm0dg5QtcQK0WtXxxATGYDJZObE2XKMy1yrd9WqcDZtSgKgpLSV6wWNK8bxDAzyZv+ja4iODcRssnDxXDXn369cEarZCgq3C8pcvziKKrV9FFVq+yiq1PZRVKkVbiWy1xTfDdxcU3xk4ZpiG2t+zzU3cqWthdTAYB5ISFm0P4hrb+v7+zlUXo6XVssnc3PFhxGp9aDz7IxOTPKbS/lYgI/lrsHfY57CnAPrSl+9VkLbwBCbE2NYFxNle4dF7AyOjvGHswVYgKfuySbI07bqoeR601n2LRYLr54rontwlNzESDamxDpux84a5BtVLVwua8TLoONDO9eg0agl17k6s5a5sKKFK0WN6LRqntiTg5eH3u6aYrnrmWef88TEFG++U8jg0BjhIb7svS9jQYrjUq81rihr5dLlWgCysmJYuyZW/O3eYlVqsYuFkhtNFFytx2KBgEAvtu3KwNfPAdVu2WuBZY5Thn1xTfFjyppiBYUlxPo7/uFfFq4pXmytrqJKbf+4d70q9SxbtvZRVKkVVWoQ1xR/JO2jK3auXwp7KxnZa4qNRiOjozffOjU0NPCjH/2Id955x60Du52I9/MHoGGgX5YScbSvL1qVipHJSTpG5Ckke+q0xPiLNwhnUqgB0iPEKG9pW5fTb7p9PQ3Eh4oRb7dEi5PFaHFRQwdjE+55+5uZGIG3h56RsQlu1LS6xeZsVq+KJDjAi4lJ05KnUet0Gnbdm4ZWo6a9a5DLBQ1Ldiwp0tIiWb9OFPsqKmri2jX3p6Y7iyAIrF4Ty+4HszEYtPT1jvD269doqFu+Os8KCncCylxvH0WV2j6KKrV9FFVq+yiq1Aq3AtlO8aOPPsrvfvc7APr7+9mwYQM/+MEPePTRR/n5z3/u9gHeDoR7eaNTqRkzTdE56vhNQaNSEefvD0BNr/zySqumU6grup0rrZQcEoRWrWLAOEbrgOMlpeaTEyuKZFW2drtcnik+LIAgH08mTSYKG9xzA9GoVWyYLtF0o7qV4VH3TTwgrtHetk58i9/c3k/FEgth+ft5sm2zWHu5rLKNqtqOJT2eLTIyomYc4+KS5hUjvmUlPNKf/Y/nEhruy+SkiTPvlXHpbJWSTq2g4CDKXL84iviWfWw5xhOKYzzDnbDG+OF5a4y7bsM1xneT+JaCfWQ7xdeuXePee+8F4MCBA4SHh9PQ0MDvfvc7fvzjH7t9gLcDapVqRoW6fqBf1r7JgWK6RnVvr+zJNCk4EJUg0DtqpEtmpBlAp1GzKlQU3Cpuc96xigjwIdjHkymzmaJm1xw0MVospnIX1re5bW1xYmQQ4UE+mExmLpTUu8XmbAL8PFmXKTrel27U0z9kdPsxZhMbHcia1TGAKLzV3uledW1HyMiIYuOGRADKylo5f8G9D2yu4uml5/4Hs8nMFq9TZXkbb712je4u518AKbjGz372M+Lj4zEYDGzcuJHLly9L9n355ZcRBGFOMxjmLhM5ePAge/bsISgoCEEQKCgoWGBnbGyMZ599lqCgILy9vXnyySfp6Fj+F0m3G8pc7xiKY2wfRZXaPlbH2P8OcYwPKY7xHBTH+PZCtlM8OjqKj4+ofPzOO+/wxBNPoFKp2LRpEw0Ny5/KuVJI8BNTiOsG5EV8EwICUAsC/WNjdI/Ku1HpNRoSAsXjlnc6Fy1eHSUKblV39TrtgAqCwNr4SACKmtqZMrk2oSaEBxLs68mkycz1WvekOwuCwJasRAQB6tt6aeyQL262GKtTIogI8WXKZObU5Sq3PljYYk1mNPExQZjNFt47W0H/oPsmOkdJS4tkyz2rEASorunk9PsVmFz8/t2JSqUid0MCux/IwtNTx+CAkSNv3qCooHFFOfB3A3/5y1/4whe+wLe+9S2uXbtGTk4Oe/fupbNTOrPC19eXtra2mTZ/jhkZGWHr1q38+7//u6SNz3/+87z55pv87W9/4/Tp07S2tt616slyUOZ6x1FUqe2jqFLbx0en58k7IGIcdps7xooqtYJspzg5OZnXX3+dpqYmjh07xp49ewDo7Oy84xdg2yN+2inuGB1heMLxm6VOrSZ+OoW62okU6rQQMdJb2dUtaz2zlVAfb0J9vDBbLJS2O7/uMiksCB+DDuPkFOUulHkC8QayYZUY3StubGdkzD2TT6CvJ1nT9ZDPFde77LzPRxAEtm9IRq/T0N0/Qn5Jk1vt2zrevRuTCQnyZmJiiuOnyzC66VrJITk5jO3b01CpBBobe3jvZBmTkysrTTkiKoCHnsgjNiEYi8XC9fwG3nm7kKGhpasvvVIZnZxwS5PLf/zHf/DpT3+aZ555hoyMDH7xi1/g6enJb37zG8l9BEEgPDx8poWFhc35/O///u/55je/ye7du23uPzAwwK9//Wv+4z/+g507d5KXl8dvf/tbzp8/z8WLF2Wfw92EMtfLQ1Glto+iSm2fO0GV+uFZqtRLUcf4TlKlHpgYdJttBfch2yn+5je/yf/9v/+X+Ph4Nm7cyObNmwHxTfLatWvdPsBbimCjSeCl1RHm6Q1A/eCsKKRgsdksMNOSg6bLK/X0yJ5E4wL9MWg0jExO0jQwgEWYVn+UajZYHSU+aBa3dWC2zB2bpJ15TaUWyIkXHc7rDa2YsMyMRVIF0479mBB/wvy9MZktXK1tWfS8Zo/Z3vhz02LwMugYGh2noLpFlEWc1Ry1I9W8PPVsXSemFBdWttLc2T/nOojnINhs8/steu0AjUbN7nvT8fE2MDwyzrtnypicMtsenNS1kzqujHHGxgWze1cmGo2KtrZ+3nm3mLHxqel9pI4r2G4q201qPNJt7vH0Hlq27Urnnu2paLVqOjsGefO1a1RUtIvC3FJ2VBJN8ncgc5xS9peIDa/+lIw//4dLbcOrPwVEVcrZbXzc9kPcxMQE+fn5c5xXlUrF7t27uXDhguRYh4eHiYuLIyYmhkcffZSSkhJZ55qfn8/k5OSc46alpREbG2v3uAp32Vxv46YvMXVLTjaCGVIj5qVSF9dhMVvcY98CibZSqU3usY8Fov0XplKbp8wIZhY0e+OUOq7NiPGUScKOYLPZO26IhxdPzl9jPDll95wXNLNguyHhGE9MyLcl0d9Ha1i4xnh8HIsFiSbYbFKTukG9MGLcMzYqaUeqmSWaTqW1scZ4RHL8JrPKZpO6nCpBsyBi3GYcwGRR2Wxyxw8ats5LpW4e7ZXsL9Ukx4OKvIBcNgTmkeqTOuu4CisF2U7xBz7wARobG7l69SpHjx6d2b5r1y5++MMfunVwtxuJ/gEA1Pb3ydpvTgq1Ud6bO7VKxapg0al2NoV6VWgQOrWawbFxGvsGnLIBkB4Vil6rYdA4Tl2n/Kj3bARBYEOKGC0ub+pkcNQ9ET2tRs3mrHgACqpb6R92/9rf+Kgg0hLFFw1nrlQzNr609YQNBi33b09Hr9fQ3TvCqQuVtyQ1OCLCnz33Z4mR8u5hjhy5seIisYIgkLQqjP1PiCJcU5MmLp2t4r2jxYwMu1eA7W4gJiZmTu3al156yWa/7u5uTCbTgkhvWFgY7e22VetTU1P5zW9+w6FDh3jllVcwm83cc889NDc3Ozy+9vZ2dDod/tPZOI4cV0FEmeudQ1Glto+iSm0fRZXaPneCKnWUR6Tt8m1uxl1ZYc5kht2u2CjqtjjWVLbZbNiwwS0Dup1J8AvgQmsTjUP9TJnNaGzU/LOFTq0mzt+f2r4+qnt7CfG0XetNitTQYIraO6jt6WXCZEJnqxagHbRqNWnhIRS2tFPc2kFcoL+s/WfsaNRkRYdxta6F6/WtJIYGuvSHHxXkR3SQH809A1ypamZXTrLTtmYTHxFITKg/TZ39nCus48HN6W6/QW3MjqOta5CBISPvX61h9z2pS3oT9PPxYPe96Rw9WUJTax8X8mu5Z13istx4ZxMS4sMDD2Rz/N0SBofGePvIDXbtTCc4yGdZx7EYPj4e3P9gDuUlLRRcraetpY83DuazbmMiySlhy37dlpPLTz7nljrFEf/wbZqamubY0kvU2XSGzZs3z0QnAe655x7S09P57//+b77zne+47TgK0ihzvXOkRIrLmk4UVVPWJK6Z35YpUcfYCayO8dHCKqo7xAd2yTrGTmB1jA8XV8w4xlJ1jJ3B6hgfKi6fcYwfnnaU3YHVMT5YXjrjGEvWMXYCq2N8oLJkxjGWrGPsBFbH2FrH+EB1ER9IzsJX5577q9UxttYxfrW+iCfjswjQ265jLBerY2ytY/x60w0ei8kh2ODtFvtWx9hax/hIawEPRK4hzODvFvtWx9hax/hk53XuC11LpEeQW+wvFxsO/BSVh2HxjotgNq6s4MZS4p47nAIgpu54a3VMmc00DfXL2nfVdAp1hRMp1OE+3vgZDEyazdQ4WbN4dYQYwanv6WNwzPk/gNWx4ahVAp2DIzT3ur5mYmOqGC2uau2ms989bxtF0a0E1CqBlu4Bqpqdi7DbQ6NRc9/GVeI627Y+iquWXlQhNNiH7dOlmipqO7hWvLRrmqXw8/PkwQeyCQjwYmxskqPHimlodN+bVnehUglkZEWz//FcgkN8mJw0ceFsFcePFDE4uLTq4bcST63OLQ1EIazZTcopDg4ORq1WL1B97ujoWOB0SaHValm7di3V1dUOn2t4eDgTExP09/c7fVwFBWdQVKnto6hS2+dOU6V+fQnWGN/OqtQKKxPFKXYjgiCQ5D9dYqlPvgq1RqViYHyMDpnllQRBICMsBICSdudq5AZ6eRAT4IcFKGpxvlyJp05LxvQa5fw6x9McpQjx8555636+vMFtDxW+Xgbyph3uC8X1GJcgxTnI34uNOfEAXClqoK3L+dR0R4mLDuKePHFN843SZoor3KPeLRdPTz379mYRFRmAyWTm1OlyikuaV1QtYyt+/p7sfXgN6zYkoFaraG8b4I2D+RReb1xRStq3Mzqdjry8PN57772ZbWazmffee29ONNgeJpOJoqIiIiIiHD5uXl4eWq12znErKipobGx0+LgKCs6iqFLbR1Glts+dokp9u9cxXirxraXm8geeo/QjX3C5Xf7Ac7f6VJYNxSl2M8kBYsS3pr9Xlhq0Tq0mMUBck1zZIz9ymR4WggC0DQ3TM+LcTS07SoyclLZ3ujQxrY2PQCUItPYN0drnerR4Q0oMGpWK9r4hajtcW6s8m+ykCIJ8PRmfnOJcUZ3b7M4mPTGMpNhgLBY4cbGKEePSr1tNSw4nL0usmXy5oJ7ymluzdlKn07BzZwapKeLvKv9aAxcuVq9IR9MaNX7kiVwiIv0xmywUXGvgzdeu0d7af6uHd0fwhS98gf/5n//hf//3fykrK+Nzn/scIyMjPPPMMwB87GMf46tf/epM/3/5l3/hnXfeoba2lmvXrvHRj36UhoYGPvWpT8306e3tpaCggNLSUkB0eAsKCmbWC/v5+fHJT36SL3zhC5w8eZL8/HyeeeYZNm/ezKZNm5bx7BVuB1q7B7hS3uRWx1VRpbaPokptnztBlfoRRZX6luCurDBrZtjdgOIU28GWsOxi6sNRPr4YNBrGTFO0DNtxCG3smzItmFXZ24MZi4MKwWLz0uuIDxKd6tLOTieUhiE+yB8/g57xKRMVHd12z1NKHRABvD30pEWJkeurtc327SxyXgjg7aknZ7qU0oWKRibNZrv9HWkWAQS1im1rkxAEqG3toaatx45KsHPjF1QCW/MSCfTzZGx8khMXK5myLBy/XbVnFTabvTFlZUSxOk2sHX3+ai1V9Z1uPDeJcdroK6gFNm5KYsMGsUZ0VXUn77xbIkbm5X5nkqrUUvvIPF+VgLe/J7sezGLrzjQ8PMS6xu8cKeLs6QpGxybnqGTLtS9blfoO48Mf/jDf//73+eY3v8maNWsoKCjg6NGjM+JbjY2NtLXdXGbQ19fHpz/9adLT03nwwQcZHBzk/PnzZGRkzPR54403WLt2Lfv37wfgqaeeYu3atfziF7+Y6fPDH/6Qhx56iCeffJJt27YRHh7OwYMHl+msFW4HBAsYjRMcvVzB9aoWLpU2uk81WlGlXvS4d6Qq9aSiSq2oUjumSq2wchAsDryyfOONNxw2+Mgjj7g0oJXA4OAgfn5+/Nu5Ixi854peWcwSD6uzth+vr6aku5OckHDui0lctL8Vk9nMr67mM24y8URaOjF+fnb7w/SkMk1dbx+HSyowaDR8Yn2ubaEviRe8wvT2G81tvF/dgL+Hgb/bkIMKqfOVsDM9niHjOH84W4DZYuGx9RlE+UkI+0j8+oR59ienTPzpdAGj45NsTIlhbVKU3f7zx2PvuFfLGrle2YJBp+GDO9bgodfasCNhSGr887YPDhs59G4RE5MmMpLC2bw2QcZYbX8gdc7WMVksFi5dq6Osqh1BgB2bU0iICXbYvqPntug4p//b0tLH6dPlTE6a8PLSc9+ONIICZQhvSI5H3vglz3ceE+NTFFytp6JUTEHX6TSsXR9PcmoEKpVg57gyx2MDo3GEz3zxEQYGBtxWE9Z6X3OHTXfaUlg53K1z/Y/+dBwPTy9K6ts5W1wPiJlEG9NjF4pjSUyJkpVVZm2vbO3mRFE1FiA9JtS2+JYL9uu6ejlaWIXZYiE5LMi2+JYL9pv7BjhcXMGU2UxcoL9N8S3JZ3wHjts+NMSh4nImTCYifX14ZHUa2vniW5J2pCaIm//sHh3hYHkpxqkpQr28bItvOXAd5qC6edz+MSMHKksYnpwg0OAhLb7lgC1b/Yem1agHJsbw1eklxbek3qkKktdI3G6cVqPuGR/FS6OTFN+SsiO1XTW9fdw0NSO+pVdpJMW3pMavVtl+6LF2nzKbZsS3tIJaUnxLLfHwtNj4TRbzjPiWWlBJim+pFrEzm9GhUT6W8ZEVO9cvhb2VjENOscpBxUFBEDC5Me3lVjHfKTZbLFgsFtQqlUNOcV1/H4eqy/DSavnU6nW2FScl7JyorqW4q5PMkBB2JyYt2n/235jZYuHly9cZmZhgX0oyKSE2HKBFnOKJKRMvX7jGhMnE/tWpJE5Hnx22M2s8p0prKW3uJCrAl8fyMmzv4KBTDFDR3MXJwhq0ajUf2Z6Dp15nt//88Ugd12Qy89rpQvqGjCREBLI7L2XBd+aqUwzQ2NrH8XPlAGxbn8yquBAHx+qcUyzuauHc5Rqq6joRBIFdW1KJjQp0yL67nWKAgYFRTpwoY3DQiFqt4p7NySQmhNjcz/HxLI1TbKW7c4hLZ6vo7RFTvgKDvFm3KZHwcD/bOyhOscJtyt0611udYmBxx9gFpxIccIxdtL+oY+yi/cUcY1ecYljoGC9QpXbBKQYHHGMXnGJw0DF20ikGxxxjZ51icMwxdtYpBsccY2edYnDMMXbWKQbHHGPFKb59cWgGNJvNDrU7YZKcz/n2Rv675DKVA46v843x9UOnVjMyOUnryJCs46VOp1BX9fYyJXPdjkoQyAgXHYyidufEsnQaNZmRoQBcb3JNpCkvIQqVINDSN0hT74BLtkBcmxXi58WkycTFikaX7VlRq1Vsz01GEATq2nqpbnG/GjVAbGQAa9LFCPe5/Bo6e+T9NpxBEATuWZ9EQmwwFouFE+craGxx37psufj5ebJ/fw5RUaIA1/tnK7l8pXZFrjO2EhziwwOPrmX95iS0OjW9PcO881Yhp94rZegOVqlWuPu4m+d6K5nx4WxdHQ9AYU0bl8oa3bvGWFGltouiSm0fRZXaPooqtYIruJTMPuZC6Z7bBYvFwphpipoBxx0JjUo1o0Jd2SfPwYry8cVHp2PCZKJWpoI1wOrwUASgZXDIacGtnOhpoayBIdoGnHfcfDz0ZMaI6wUvVrv+YCEIAlsz4gGobOmmvc99TmWIvze5qdEAnC2qY2h0aQSxcjNjiI0IwGS28O6FCkaW6DizUakEtm1aRUJMEGaz6Bg3NN+6m7hVgCtrtXi9y8rbOHa8mOGRpb8WzqJSCaRlRvHoB9eTkhaBIEBjfQ+HXs3n6uVaJsbd91CloLDSuBvm+tlkxoezNSsemHaMS93vGCuq1NIoqtT2UVSp7aOoUis4i2yn2GQy8Z3vfIeoqCi8vb2pra0F4Bvf+Aa//vWv3T7AW02Sn+jc1g/1yYrcpkyrUFf19chSoRYEgbRgMdpb2iU/Yumt15MYJI650MlosbdeR1qYmHqd74ZosUatonNwhNquPpdsAYT5+5AWLV6f90vqZF3bxVizKoqwAG8mp0ycul7lVttWBEFg+4ZVBPh6YByb5Pj5ClFQZIlRqQS2b0ohITZ4xjGuaeha8uPaG0/u2jju25GGVqumq2uINw8X0NR866LYjuDhoWPj1lU89EQekVEBmM0WSotaeO1vVygtblnREW8FBTncbXP9fDJmO8a1SxAxVlSp7aKoUttHUaW2j6JKreAMsp3i7373u7z88st873vfQ6e7uU5i9erV/OpXv3Lr4G45AoR5euOl1TFpNtM0MmBHWdYyp8X6+aFXqxmdmqRlZNDOfgtb+vRa4MaBfvEmasP+TenGhWRFiNHZ8q4uxk0mp1Sa18aJysV1PX10j4zaVJl2pHkatOTEicrRl2qaMGGZOx47195mAzamxKLTqOkZGqWkocNuf0kFZRuoVAI7clehVato6x2isKZVcv/Frqe9fXRaNfdvScOg19DTP8KpS6IDLlvtWaYqtaAWI8ZJ8SFYLHD6YhUVtR2S9lFhs7lDldoiACqB2LhgHn54LUFB3kxMTHHiZBlX8+vFFxIOq2QLNpvbVKBt2PAP9GLXg1ns3LcaP39PxsenuHqpltcPXKW2pmv6b2T+9ZRodn7rCgq3irtqrrchaStYIDNuXiq1oko9pymq1EgrSZsFRZVaUaV2qCmsHGQ7xb/73e/45S9/yd/93d+hniV+kJOTQ3l5uVsHtxIQBIEkXzHyWjPg+FsgtUpFcsB0CnWvvIhvgIcH4d7eWIDybvnR4mh/X/w9DEyazFR0ORcNDPD0IDFEHL+ra4vXxEeg12roGzFS2eZ6dNJDr2VDSgwAlyubGBlz39tXXy8Dm7MSALha3kT3wIjbbM/Gx8vA7nvSUKsEGtv6uHyjfkmOMx+VSuDejcmkJosvTs5dqaGk0rXv11V8fAw88EA26enii5iS0haOHi1ieHjlp2xGxQTy0JN5bLp3FR6eOkaGxzl7uoLDr1+ntbnvVg/PJqOTE25pCnc2d9tcL8WcNcZLETGevcZ4KSLGd9ga4yWJGM93jG+3iPFsx7hmaVOpX2tY2lTqQ00FS7rG+Fjb9SVdY3y669pdHTF+6aWXWL9+PT4+PoSGhvLYY49RUVExp8/Y2BjPPvssQUFBeHt78+STT9LRYT+z9emnn0YQhDlt3759c/r09vbyd3/3d/j6+uLv788nP/lJhofl/ZZkO8UtLS0kJycv2G42m5mcnJRr7rYg2c/qFPfKmqxSAsWIb3W/vBRqgMwQawp1l+wJUhAEsqejxYVtHU5PsHmxopNS2dnD0JjzN1m9VkNuvGjrck0zU25IMc2IDSN0WnTrXFm9y/ZmkxITQnx4IGaLhRP5lUuW3hwW5MO29eLfUkl1OyXVbYvs4R4EQWBzXuJMHeNL1+spKGly64OYXNRqFRs2JLLDmk7dLaZTNzQsjeiZO1GpBFalRfDYh9ezZn08Wq2avt4R3j1WzDtvF9LRPnCrhziHDa/8nIzf/qdLbcMrP7/Vp6GwxNyNc70UiviWfW6FYzyhOMYzLHCM74A1xor4luu46wW4nJfgp0+f5tlnn+XixYscP36cyclJ9uzZw8jIzQDT5z//ed58803+9re/cfr0aVpbW3niiScWtb1v3z7a2tpm2p/+9Kc5n//d3/0dJSUlHD9+nMOHD3PmzBk+85nPOH7BAM3iXeaSkZHB+++/T1xc3JztBw4cYO3atXLN3RZEe/uhV4mp0G2jQ0R6OSZJHuPrh4dGg3FqisbBfuL9Ahw+5qqgIE43NNBrNNI2PEykl4+sMaeFhnChvoneUSPNA4PE+PvJ2h8gzNebaH9fmvsHudbUyvZVCbJtWMmKCaewsZ3hsQmKmtpZO+0kO4tKENi2OpFXzxdR295LXUcvCWGBi+/oAIIgcO+aRLpODTMwPMb7N2rYuTbZdmktF0mMCWZoZJyrxY1cLKjH06AjIXph3Tt3IwgC63Li0GjUFBQ3ca24CePYJJtybdTNXEbi4oIJDPDm9JlyenqGOXW6nMTEEDZuSEKnVS9u4Bai0ajJWhNLSko4RQWNVJS10d42QPtbhYRH+JGzNo6wCPl/hwoKt4K7ca63R2Z8OABni+sprBFfYNqsY+wkKZHiS/QTRdWUNXUC2K5j7CSiY5zC0cJKqjvEB3abdYydxOoYHy6umHGMbdUxdharY3youHxGldpmHWMnsTrGr5aXzjjGNusYO4nVMT5QWTLjGEvWMXYCq2P8ak0R/RNjHKgukqxj7AxWx9harunV+iLJOsbOYHWMreWaXm+6IVnH2BmsjrG1XNOR1gLJOsbOYHWMreWaTnZel6xjvFxseOXnqDwMLtsxG8WsvcHBwTnb9Xo9ev3c39fRo0fn/P/LL79MaGgo+fn5bNu2jYGBAX7961/zxz/+kZ07dwLw29/+lvT0dC5evMimTZskx6HX6wkPD7f5WVlZGUePHuXKlSusW7cOgJ/85Cc8+OCDfP/73ycy0jGfQ/Zf+ze/+U0+/vGP09LSgtls5uDBg1RUVPC73/2Ow4cPyzV3W6BWqUj0C6Ssr4uqgW6HnWKVIJASEMyNrnbKe7tlOcV6jYaUoCBKu7oo6uggMlGeU6zXaEgLC6GorYMbre1OOcUA6+KiaO4fpKStk7zYKLz1zt3ANWoVG5OjOVFSS35dC2mRIXjotE7ZshLs68WahEiu17ZyprSOyEBf9Fr3TGAGnZadeas4fL6UmpYewgN9Zh6K3E12aiTDo+OU13Zw6nIVBp2GiNCld54EQWDt6hgMWg0Xr9dRVt2OcWySbZtWoVG750HGGXx8DDywL5sbhY0UFzdTW9tFe/sAWzavIjLS/5aNy1EMBi3rNyWRnhlF0Y0maqo6ROe4rZCwCD9y1sYSHuF/y8Z3+aOfc0ud4ojnv+mmESmsRO7GuX4xMuPDQYCzRdOOsQU2ZrjXMbYIcLJQdIwtFti+2p2OcQB7s1dxrLBq6RzjrFQOFy2PY/zGUjjG6RkcLLt9HeMnk7Jm6hjfCY7xozE5hNxmjvHZrus0rxDH2J3ExMTM+f9vfetbvPDCC3b3GRgQM+UCA8WgVX5+PpOTk+zevXumT1paGrGxsVy4cMGuU3zq1ClCQ0MJCAhg586dvPjiiwQFidf2woUL+Pv7zzjEALt370alUnHp0iUef/xxh85RsDiRo/P+++/zL//yL9y4cYPh4WFyc3P55je/yZ49e+SaWpFYC1X/2/kjGLy9AHE98Rt15XhrdXwybf3CSUoiU6htcJi/VBShVan4TPb6mzdvs+1JaHZN8fahIf5SUoJaEPjU2jybN2ZBwg4W6DMaeSX/BgB/n5eDv4cHEjXLJbdbTBZeu1FK68AQ2VHhbEuOF/tL/GqktmMGs8XCgUtFdA+NsjomjG1pCdLjsWNnNlMmMwfOFdI/MkZaVAj3ZSXNOwEJ+1KZXfP6F9W0crGkAZUg8MiWTEID5hWZlxqnnb8qW8c2WyycvFhJfUsvWo2a/TsyCfL3WuQYtj9w9Nxm+lugtrGbM5eqMJsthAb7sHurKARmE7nfmcSBHbl2nV2DnDtbyeCQ+KYyNTWcvLViivIi5hGkbm1y73hyly/MS0scHhqjuLCJ6sqOmZTFsHBfVufEEhnlb1NTy2gc4TNfeISBgQGXHVgr1vuaO2y605bCyuVumet/9MfjeHh6zf1QamoVoLS+nbNF9QBkJ0WIEWMJx1JSR0fKvkosOXiysBoLkB4TKkaM3WVfgLrOXo4VigKPyWFBomOsdo99BGjuG+BwUQVTZjNxgf48uDoFlUbCMbYzTqn+7YNDHJpOoY709REdY41tx9gi+bAifdzu0REOlokp1KFeXqJjLPXC3d77BJXtY/ePGTlQWcLw5ASBBo+bjrGULQk7Uv2HJsc4UF3MwMQYvjr9jGMs9W5FkPlAN2aa4GC96Bh7aXQzjrGUHantKontE+ZJ3mgupHNsCL1KMxMxlhq/WmX7oUTqcpotUxxtu0HbWB9aQT3jGKslHp7kjh+mON15g2ZjF2pBNeMY2+o/OjTK05lPLclc39bd5Rabg4ODRASH0NTUNMeerUjxbMxmM4888gj9/f2cPXsWgD/+8Y8888wzjI/PTe/fsGED9913H//+7/9u09af//xnPD09SUhIoKamhq997Wt4e3tz4cIF1Go1//qv/8r//u//Lli/HBoayre//W0+97nPOXSuTr2+u/feezl+/DidnZ2Mjo5y9uzZO2aSnMMsKcQ4X3+0KhXDkxO0G4dsSCXaJtzLGz+9gUmzeW6tYwnJxdnif6E+3gR7emKyWCjr6bKjpmujIQp2xQf4A1DQ2j59XHlNUAmsjxPryZa0dTAyMWG3v5QKIIK49nJLqpiKV9LUQc/wqLQdiTa/n0ajYvu0I1ze0kVjT79L6tDz+61OiiA+Qlxf/G5+pahM6YiyssxjCyqxVFN4sA+TUyaOvV/GwJARsDdWwWaTq0ptESAhLpg92zPQadV0dg9x+L0iBobHbas6L7Eq9ewWEurL/ofXkpoqKphXVLRz+K0COruHZtmSOC+VIKu5S616vl0vPw823pvCYx9eT0p6BCqVQEf7IO8dK+atNwqob+jBLAhYVKo5TUHhVnO3zPVy1ZIVVWpp+4oq9aymqFIrqtQOqFIvFZ5andsagK+v75xmzyEGePbZZykuLubPf/6zy+fy1FNP8cgjj5CVlcVjjz3G4cOHuXLlCqdOnXLZ9myUJy8H0UynUANUDTgu/iMIAmnTglvlPfKUlwVBYHVYKADFHZ1OCW+sjRKdibKOLoxOiqNEB/gS7uuNyWzhWqNrSsVRgX4khgZiAc5V1LtFTCQi0IfVcWJq8+niWrcKYwmCwPa1Sfh66hk2TnDyWvWSCVJp1Cp235NGoJ8nxvFJjrxfysio+8QyFiMizI/9u7Pw9tQzODTG4XcLRefzFqPVqtm4KYn7d2fi6aljcGiMI0cKuXK1lqllqPHsLry8DWzcuorHPryB9NVRqDUqeruHOfNeGW8cuEpNZTtmN4rTKCgoLB2KKrV9FFVq+yiq1PZRVKlvb5577jkOHz7MyZMniY6OntkeHh7OxMQE/f39c/p3dHRIrhe2RWJiIsHBwVRXV8/Y7ezsnNNnamqK3t5eWXYdcooDAgIIDAx0qN3JrPIXc9er+ntkTU5pgaKSdMNgv+ybXmpwMBqVit4xI61D8h2UKD9fQry8mDKbKWqzL3kuhSAIbIgXf9TFrR0Mu6BEDbA5JRa1SqC5d5C6rj6XbFnZmBKDj4fouF6qbHSLTSs6rYbd61NRqwSaOvu5XtXiVvuz0es07Ls3HV9vAyOjExx5vwyjG0tOLYa/nyf7788iKMCL8fEpjp4sob5pZShAR0YG8MjDuSQliS+KSktbeePN67S19d/agcnEy1vPus1JPPGRjWStjUWn0zA4YOTC+5WMDC/fSxAFhfkoc708FFVq+yiq1PZRVKnto6hS335YLBaee+45XnvtNU6cOEFCQsKcz/Py8tBqtbz33nsz2yoqKmhsbGTz5s0OH6e5uZmenh4iIsTA3+bNm+nv7yc/P3+mz4kTJzCbzWzcuNFhuw45xT/60Y/44Q9/yA9/+EO+/vWvA7B3715eeOEFXnjhBfbu3QvAN77xDYcPfDsSP51CPTQ5Tvuo43+YAQYPwjzFusMVMmsW6zUaUoJFZ7ywU75TKwgCudHij+ZGa7vTb1JjAvyI9PPBZLFwucE1p9DP00BOnDims5X1bnm7q9Wo2bZa/OMrbuiguXvAZZuzCfbzYkt2IiDWL27scI8zbwsPg44HtmXg5aFjYMjIkTOlGMeXrwSKp4eOB3auJjoyAJPJzMlzlVwvcu/DnrPo9Rq2bklh164MPD11DA2N8c67JZw9V8nY2O1VJsZg0LJmXTyPP7WB3PUJpGZG4ePrcauHpXAXo8z18rkzHOOUO8YxfnOJIsaG29wx9lccY5sojrF7efbZZ3nllVf44x//iI+PD+3t7bS3t2M0issB/fz8+OQnP8kXvvAFTp48SX5+Ps888wybN2+eI7KVlpbGa6+9BsDw8DD/z//z/3Dx4kXq6+t57733ePTRR0lOTp6Zk9LT09m3bx+f/vSnuXz5MufOneO5557jqaeeclh5GpAvtPXkk09y33338dxzz83Z/tOf/pR3332X119/XY65FcmM0NaFt2eEtqy8XVdJRX83a4Ij2BGVePMDk8S6gGkhrMKudk401hJo8ODvM9YgSEkA2BDO6hwZ4c+FRagEgWfWrMVbd1OpUFJoa9acY7ZY+H1+AYNj42xLiGNNZMSC7pLiTLO2tw0M8er1EgTg/6zPIdDTxgO8gyJMk1Mm/nT+BsNjE+TFR7IpOXZuB4lf5WIiUmeKaylt6sRLr+ODW7Pw0EooXMsUi7Juf/9GLWUNHWg1ah7duppAH9tOjJ1l5g4fe2DIyNunSxgdmyTQz5MHtmVg0Gsl+8/gJgEus9nC1Rv1lFSI5UdiIgPYtmkVeqnSSFLnJXFYSQGrRb4DKxMTU1y/3kD59Pj0eg15ufEkJ4XaV2uVtC9vPNL2Ze5go7vROMI/PP+QIrSlcMu4m+b6/3xlodCWRSpkIHFrKWmYJb6VGHFTlVqiv1wBq4q2m+JbadGhN1Wp3WS/tsuG+JbKffab+xeKb6lVKtnXWeq47UM2xLfUajt2pCYC25u7jDbEtzQaOxOctC1bwlmS4lsy7Uj1H5pOoZ4vvgW4RYDLOJ1CPV98y54dOQJW46apGVVqvUozR5XaHQJcU2bTjCr1bPEtwC0CXCaLeUaVerb41ujQKJ9Y/eEVO9fLtSf17PXb3/6Wp59+GoCxsTG++MUv8qc//Ynx8XH27t3Lf/3Xf81JcxYEYWYfo9HIY489xvXr1+nv7ycyMpI9e/bwne98h7CwsJl9ent7ee6553jzzTdRqVQ8+eST/PjHP8bb23H1ctlrio8dO8a+ffsWbN+3bx/vvvuuXHO3Han+Yip0VX83ZhkPv6mBwWgEMQ26bUTeW65QLy8ivH0wWywUOxEtVgkCedHim5JrLW1Ov6WN8PMhISgAC3ChzrUUZa1Gzda0eACuN7TRN2J0yZ6VzWlx+HkZGBmfEMVJ3BzdvCcrnoggX1EM63L5kkZw/Xw8eGB7Jh56Lb0Doxx5v5TxieWLhqpUAhvWJrB1Y7KYOt7ax5vHC+kfdN9bYFfQ6TRs3JjEg/uyCQjwZHx8ivMXqjlyrIieXve9SVZQuBu52+d6uWTEh7M1Kx5YojXGUcHcly1GjMuXYo1xaCB7lzqVOmsJI8a+y7DGOP02jxgnr17SVOon4pc2YvxIdPasNcbujxjvi8hZ0ojx9tAcoqcjxq3GlbEszd1YLBabzeoQAxgMBn72s5/R29vLyMgIBw8eXLDud/Y+Hh4eHDt2jM7OTiYmJqivr+eXv/zlHIcYxLJPf/zjHxkaGmJgYIDf/OY3shxicMIpDgoK4tChQwu2Hzp0aKZe1J2CLSHaeB9/9GoNI1OTtI4M3BR7llSuFZteo2FVoHh9Sno6Fu0/v+WEi19+UWcnJot5ZrtFsNhs8/dPDwvBS6dleGKC8q5uh5R/bakob06KQQBqu/toHRiyqTLtaEsIDSA22B+zxcKZijrMWCSP66hqtFarZveaZFSCQG1HL+UtXbLUoRe7Hiq1it0bUvD1MjA0Os47VysxmS3IUVaWc2w/Xw8e2C5GiHv7RzlypoyxaUfcbarUUm16XKsSQ3lwdxZe0yJXbx4vpr6ld8lUqeWqVYeE+fLQQ2tYty4ejUZNV9cQb719g0uXa5mYNNmwJXX9FyptW2yoSbtbrVrSjoLCLeRumuttSdraUkpWVKmdt6+oUs9qiir1Xa1KvTEwkzX+aTN2FFYOsr+Nb3/723z5y1/m4Ycf5sUXX+TFF1/k4Ycf5itf+Qrf/va3l2KMKwq1SsUqP/GBoLxP3pue1cGiY1vZ2y1bDCI5MBBPrZbRyUmqe3sX32EeapWKtVFitDi/uVVWlHs2gV6epIWL0fILda69CRcEgXtT42dEt6o73LPOIsTPm/UpYpHxs2X19LspCm3FoNOyd2MaOo2ajt4hztyoXdL1tgG+N1One/pHePtMKcZlXj8bHOTNw3uzCQ8Vo+QnzlZwbYWsMwZQqVRkZkbz2GO5xMcHY7FAeUUbr72eT3V1x4oZp4LC7cLdPtc7i6JKbR9Fldo+iiq1fe4EVepknxj7S7wUbhmyneKnn36ac+fO4evry8GDBzl48CC+vr6cPXt2TnjcXQwNDfH8888TFxeHh4cH99xzD1euXLG7z6lTp8jNzUWv15OcnMzLL7/s1jGl+osllqoGemTdzCO9fQiYrllcKdOhVqtUZE2nCtxob5e1r5XMiFAMGg39Y2NUdzvvgG5IiEYtCLQODFHf2++0HRBFt3LjowA4V9HAhJsmlzWJEUQG+TJlMvPejWq3TroA/j4e7FqfgiBAVXMXBdWulapajEA/T/Zvz8DDoKVvYJS3T5cwYlw+VWoQBcD23pdBRsq0cFtJM8fPLK869mJ4eenZvj2NPXtW4+fnwdjYJOfOV3HkSCGdXYO3engKCrcNyz3Xw8qc753hzhDfurMcY0WV+iaKKrV9llp8S2Hl4lTcfuPGjfzhD3/g2rVrXLt2jT/84Q+yJK/l8KlPfYrjx4/z+9//nqKiIvbs2cPu3btpabGtgFxXV8f+/fu57777KCgo4Pnnn+dTn/oUx44dc9uYor398NJoGTdNUTfU5/B+giCQOR0tLuqSvzY4KzQUlSDQNjxM+7D8G4BOrSYnUszbv9LU4vQk6mPQkx0l2jlf2+jyZLY2PhI/DwOjE5Ocd1M5JUEQ2JmdhF6rpmtghEsVTW6xO5voUH/umVa8vlLWSHWzvDrUcvH39WT/9kw8PXT0Dxk5fLKYweGxJT3mfFQqFRtzE9i2aRVqtYqWtn4OHb1Ba3v/so5jMSIi/Hn4obXk5caj0ajo6h7iyJFCTp0qY3DQvZkDCgp3Kss518PKnO+d5c5wjBVVaikUVWr7KI6xwu2IbPVpAJPJxOuvv05ZWRkAmZmZPPLII6jVEqq0TmI0GvHx8eHQoUPs379/ZnteXh4PPPAAL7744oJ9vvzlL/PWW29RXFw8s+2pp56iv7+fo0ePOnRcq9Lav9tQn7ZMqz2faa0nv6uFJN9AHklIn9m+gHnbRycn+FVhPmaLhY+kZRPm5W23vxWr+N071dWUdXeTHBjI/pQUacVfCTvjk1O8fOU6EyYT+9JWsWq63JOUMrHU9vGJKX5/qYCxqSm2JcfPOMkyxApFpu239A5wKF/8PT2Sm05MoJ/d/o7ar2/v5ei1SgD25qaQECbW15SrxGzvvC4U11NU24ZKEHhwczqRwX7YUyB29dhDI2McOVPK0Mg4HgYt+7amE+jvJdn/pn3bH0j3t7+9t3+EU+crGZh2MrPTo8hdHY1KpbLZ3/XjSoxforvVzujoOAUFjVTXdGCxiC9M0lLDyc6KwWDQLujvKEupVm00jvAP/7+lUZ9u6+52i/p0RHCwoj59h7Nccz3cmvl+Rn369+8sUJ+WkrRVVKkVVerZSKpS29lHUaVWVKmtjA6N8umsD67Yud5q726Z7zVyd6iurmb//v00NzeTmpoKwEsvvURMTAxvvfUWSUlJbhvc1NQUJpMJg8EwZ7uHhwdnz561uc+FCxfYvXv3nG179+7l+eeflzzO+Pg44+M335ANDi6eZpkRGEp+Vwt1g32MTk3iodItug+Ap1bHqoAgKnq7KepuJ8wr2aH9rORGRlLW3U11by/9Y2P46wyL7zQLvUbDmqgILjc2c7mxmeSgQKfWNug1GjbER3Omup7L9c2khAZj0Mr+Oc0QFejH6ugwips7OFlay1ObstFpXH/wig8LJDs+gsL6Nk4W1hC0xRNfT3nXbDE2ZcYxMjZBbWsPxy5X8PCWTIJ9Pd16jNn4eBl46L7VHHu/jN6BUd46XcKeLemEBfss2TFtEejvxSN7srl0rZ7K2g4Ky1po7xxg++ZV+Hi59xq7gqennnvuWUV6eiT51+ppaemjrLyN6ppOslZHk54WgcYNv7XbhU2/+m9UBte+H/PY8mYoKCw/yznXw/LM94vN9Z29w7T3DJK9yvG6louRES++MD5bVE9hbRsIsDE91m1rClOixOVcJwurKW/uRBBgW2aC2+xbVamPFVbN6H7sXp2MSu0e+1ZV6sNFFTMR4wdXpyx8ueokVlXqQ8XlM6nUj6xOQ+ume75VlfpgWelMxHiOY+wi1ojxgcqSmYjxHMfYRayq1AeqixmYGONAddEcx9hVrKrUB+tFx/jV+qI5jrGrWFWp32gupHNsiENNN3gsJodggzzVYSmsqtRH227QNtbHkdaCOY7xSmbTr3/h8lwPd9d8L/uu88///M8kJibS1NQ0k1LV2NhIQkIC//zP/+zWwfn4+LB582a+853v0Nraislk4pVXXuHChQu0tbXZ3Ke9vX2BTHdYWBiDg4MzxaPn89JLL+Hn5zfTYmJiFh1bsMGTMA9vzFgo75OXNpsdIk6S5b3djJvkpdsEe3oS7+8PwDWJa7AYayLD0anV9I4aqXJhbfHqyDACPD0Ym5riSkOz03asbF4Vi49Bx9DYOBer3ZNGDbAxNYZQPy8mpkwcv17FlMm964sFQWDH2uSZUk1HLpQxMLy0KbqeBh0Pbs8kNMiHiUkTR86U0NAqX4DNVTQaNVs2JLHjnhS0WjWdPcMcOlZIXdPKK04fEODF7l2Z7NmdSWCAF5OTJq5db+Dg6/mUV7RhcvPvQkHhdmY553pYnvne3lw/OjbB0fNlXC5u5Hq56/PZbOaUa1qKVOpZ5ZqWJJVaKddkF6Vck308l7lc01KkUi9luSaFlYPs9GkvLy8uXrxIVlbWnO03btxgy5YtDDux1tUeNTU1fOITn+DMmTOo1Wpyc3NJSUkhPz9/JqVrNikpKTzzzDN89atfndn29ttvs3//fkZHR/Hw8Fiwj623xzExMfz7xbdspE/ffI9wo7uNE821hHh48Xer1tg+ARtpzBaLhVeKb9AzZmRHTAJrQiPs9he33/xn88AAB8vKUAsCn1ibi6dWu6C7VPq01c7lxmYuNTbj72Hg73JzUEvk4Uim+k5vb+zt543CcgTgI+uzCfKUePsnlW0zz35zzwBvTKdRP5qXTtS8NGrJNO9FUnGHjOMcOFvE+OQU6TGh7MhMlDdOB1J9JyanOHyulJ6BEXw89TyydTVeBhtvcyUeJuSmEwsWmJwycfJSFU1tfQjAPbmJpCWGSfa3bd894xkaGuP0hUq6esR7QFJ8CJtyE9BLZhBIGJL6jiWsSKaq28kgs1gs1NZ2UVDQwPCw+Lfv7W1gzZpYEuJDxPRAJ+1LISfd2mgc4R/+ef+KTam6m9Kp7laWe66HpZ/vpeb6H/9OTJ8uqGzhSqmoP5GXHs3atOiFg3Qhrbqkvp2zxfUAZCdF2I4Yu5CWXNnazYkiMZU6PSbUdsTYBft1Xb0ctZVK7Sb7zX0DHC5emEo9x44LadWSqdQO2Vk8Jbl7dISD5aUYbaVSO3CMxdKh7aZSy7Aj1d9eKvWc7k6m+CXsuAABAABJREFUVdtLpXbEzmJpyfNTqaUixs6mVdtLpZ5jR0ZatXFolM9kf2DFzvVWe3fLfC87UqzX6xkaGlqwfXh4GJ3OPekcs0lKSuL06dMMDw/T1NTE5cuXmZycJDHRtlMTHh5OR8dcEauOjg58fX1tOsQgnpOvr++c5gip/sGoBYEu4widRscfEARBmIkWF3a1y36jG+XrS5iXFyaLxWkl6jWREaIStXGM8g7nBaJiA/1JCArAApypqnf57XR0kB8Z0aEAnCytFWsMugEfDz2714ip6mVNnZQ1d7rF7mx0Wg37NqXh66VnaHScIxfKGJtY2tJJWo2a3ZtTSYkPxQKcu1bL1eJbUyrJx9vAg7tWk5MRjSBATX0Xh47eoK1jYNnHshiCIJCUFMpjj+WxcWMiBoOW4eExzp6t5PDh6zQ0dN+xZZw8tVq3NIU7m+We62Hp5/vF5vo1KVGszxCjx/llzW6PGN8Z4luKKrUUiiq1fRTxreXFXXP93TTfy3aKH3roIT7zmc9w6dIlLBYLFouFixcv8tnPfpZHHnlkKcYIiG+tIyIi6Ovr49ixYzz66KM2+23evJn33ntvzrbjx4+zefNmt4/JoNGS5CcKN5X0ynOy0oJC0KpU9I4ZaR6SVypGEARyI8U1Tzc62hl34qar06jJixZtXGpsdimleGtyHGpBoLl/kKou19Nm70mJxdugY9A4zrnKBpftWYkJ8Wd9ivjm/0xJHe19Cx/4XMXToOOBzRl46LX0Do1y5GIZE5PumxRtoVIJbM1LZG26eG43yls4eamKKTdO9o6PRUVudiwP7FqNj5ee4dFxjpwq4Xx+LZOTyz+exVCrVaSlRfLEE+vIXRuHVqumr3+UU6fLOfTGdWprO9360KegcLtwq+Z6uLXzveIY20dRpbbPneIYK6rUtrndHGMFech2in/84x+TlJTE5s2bMRgMGAwGtmzZQnJyMv/5n//p9gEeO3aMo0ePUldXx/Hjx7nvvvtIS0vjmWeeAeCrX/0qH/vYx2b6f/azn6W2tpYvfelLlJeX81//9V/89a9/5fOf/7zbxwawOkiM+Jb1dTJpdvzGqldryAgSI6LXO+WvDU4KDCTAYGDcZKKwQ355J4DsyHB89DqGJyYocHJ9MoCfh4F1cWKt4fdrGpxy0mej02jYmSmKuJS2dFLf5XjZq8XITYoiISwAs8XC0euVDBvdd7O34utlYP/mDPQ6DV39Ixy9VO62iLcUgiCQmxnDveuSEASBuuYe3j5destqCIcF+/LovjWkJoup3OXV7bx2rGBFRo0BtFo1WVkxPPnEerKzY9Bq1QwMjPL+2Upef+MaVdUdyppjhbuK5Z7rYeXM98viGM9eY1zqfsd4SdcYhwTc3muMbUSMlXJNN/HR6XnyDosYdymOsYIDyHaK/f39OXToEBUVFRw4cIADBw5QUVHBa6+9hp+fRBkdFxgYGODZZ58lLS2Nj33sY2zdupVjx46hnQ7nt7W10dh4U5QpISGBt956i+PHj5OTk8MPfvADfvWrX7F37163jw0g1tsPX52eCbOJqgF5UdKcUNGhrh3oZWBcnrqbShBYFyU6otfb25y6oWtUKjbFiRP/1eZWjJPOp/rmxkbiP11r+EKd6zWBowP9yIkV11qfLK1l1E1pyIIgsDMnmUBvT4wTkxy9XunWydBKoK8n+zelo9Ooae8d4tjlpXeMAVLiQ9l3bzo6rZqu3mEOvVdEd9/Ikh/XFlqtmnvWJbFvRwZenjqGR8So8dnL1YxPLG303Fn0eg1r18TxgSfXs3ZtHHq9hqGhMc5fqOa1Q9cor2hjahm+RwWFW81yz/Wwsub7nNSldYzniG/VKuJb81HEt+xzu4tveSyz+NYhRXxLwQGcqlN8pzNTp3gRoS0rlzuaONfWSKSnDx9Kzp774SL1i1+rKqVhsJ81oRHsiElwSGhrZpPFwu8LChgYH+fe2DhyI24Kdi0mtDVzPhYLfy4oontklDUR4WxLjJ/z+WJCW7Np7hvg9RuiGMqH1q4mzHeWwIGDQluz+0+ZzBy4XETvsJG4YH8eXJOKSkKtQ7Yo1PAYr14oZmxyiuTwIHbnJIuCJG6qqWvt39E7xNsXy5icMhEZ7MveDWloJUpNOHsMW/QPGXn3XDkDw2Oo1Sq2rUsiKTpYwr576xfbsjM5aeLKjXrKa8SsBg+Dlk25CcRH2ygJtgxCW472n5w0UVnZRklpC0aj+GJGr9eQnhZBakoEBr3ja21WitCWO2y605aCwq3C+ju2Cm3NxjrV3KiYJ76Vbrs6hbMCUKX1s+oYJ82qYyzR32H70+OpbLlZxzg9JpRtq22Xa5Jtf3p7Xee8OsZZNsS3nLBv3d7cN7eO8QPZC8W3HBmnVP/2wXniW1k2xLeQX7/YeuzuURt1jKXEJ+UKZCEhviW15l9G/WJxu0UU35ou12QV3/LT2y7XJKd+MYBKgNGpiZlyTVbxrUCDbf0fOfV/rf3HTVMz5Zqs4lshHrbLNcmpX2w97pTZNFOuySq+Felhe05cbqEtd9m8m+Z72U6xxWLhwIEDnDx5ks7OTszz3twdPHjQrQO8Fcw4xZfewmOBU7zwz2N4coL/Kb6KBfj71LUEGWap6S3iFDcM9PNaVSlalYpPZq/DoJK4WUrYKWnv5L26Wry0Wp5esxaNdbKQdEIX2mns6+dQcTkqQeCjuTn4eRhm9ZeyY3v78dJqKjq6CfH25IO5WTOToz3n16b96e09Q6McuFSEyWxha2ocOTERtneQ6ziaobVnkMNXyjBbLKxfFUNecpT7HNNZ2zt6hzhyQXSMI4J92bdBokailHMq8+W4dUzjE1OculRFc3s/ADlpUeRlxix4IJKrSi15XAe+446uQc5dqWFgUCyXEhsVyKa8BLw8b06ysscj+6WBfCfaZDJTVdVBSUkLw8NiVodGo2JVchgZGVF4e8+qBSj3PaOEU/zZ5x5csRPl3TRJ3q3cTXP9T/5X2ikGFFXqRewrqtT2j62oUiuq1GBbldo4NMI/5ChO8UpBdvr0888/z9///d9TV1eHt7f3nJp/S5VStdLx1upI9BUFt4p65KlBx/r6EeThyaTZTHGX/LXB6cHBeOt0jExOUtLlnKJybIA/sf5+mC0Wzje4Vh94S1Iseo2aruFRilqdU8aeTZCPJ/ekxAFwvrKRrkH3pQJHBvmyJSMegCtVTVS3dbvN9mzCAn14YHM6Wo2atu7BZVljDKDXabh/axpZqdOibOUtHD9fcUtTl8NCfHlkbw45mdGoVAKNLb28dqSA0sq2FS1oJQpyRfD443ls25ZKYKAXU1NmysrbOPjaVc6cqaCza/COVaxWuPtQ5vqb3GniW6eLFVXq2Siq1PZRVKnto6wxvnOQHSkODAzklVde4cEHH1yqMd1y5EaKAWr6+zlUV4pereZTGevRqqbfMi4SKQYo6e7geH0NXlodz2Tm3oz2SvSfjWCGG+3tnGqox1ur4+Nr1oj7y4gUA3QNjfDngiIAnszOJNLXZ8a+1HFtYoHi1g5OVdahVan4yPpsfD0MTkeKQYxYHCmopL6rD18PPR/amIV+/ltWJyLFVs6V1VNU345KEHhoXRpRQTYe+NyQVj07YhwW4M2+Telza/i6OVI8m+qGLs5ercFktuDjpWfX5lSC/L0WOQf3R4pn09c/wrkrNTN1jQP8PNmYm0BkqMQD9y2MFC8cioW21n6KS1poa+uf2R4U5E1aagQJ8cGo1Q6+c1QixQorkLtprl8sUmzFbsTYhUixFbsRYxcirVbsRozdYL+uq4+jhZW2I8ZusG8vYuxKpNjK/Ijxw9OOsn07i0eKrSwaMXYyUmxl0Yixk5FiK0MT47xaU0S/RMTY2UixlcUixs5Giq0sFjF2NlJsZbGIsRIpXvnIjhT7+flJ1gy8m4nz8cdPJ6pBV/TJq/ubGhiCt1bHyOQE5b3yawZnhobirdMxPDlBkZNK1CHeXmSEhQDwfq1r9YYzI0KJ9PNh0mzmREWty2+kBUFgZ2biTJmmEyWu25zN5rQ4EsMDZxSpe4aWRpgqLNCH/fdkoNeq6egb5vC5EozjS1vH2EpyXAgP3bcab089QyPjvHmiiPLajlsa2Qzw9+LBXVlsXpeIXqehb2CUoydLOHm+guER96uCuxNBEIiMDGDP/at5aP8akpNCUakEenqGOXe+igOvXuXa9QZGVvh5KChIocz1C1FUqe1zp6lSv7lEEePbWXxLUaWWRokY3/7IdopfeOEFvv3tb2M0GpdiPLctKkEge7o8043udlkTjUalIjdMTHHN72jBLHOS0qhUbJhWor7S2uL0TXxzXAxatZrO4RHKOuQ751YEQWBnahIalYrm/kGKWp1z1Gdj0GnZmyPWRqzt6qWwyfXUbCsqQWBndjIRAT5MTJl462o5Q0tQqgkgJMCbh+7JxEOnpWdwlDfOFjM0ujyOU3CAN4/uyiIm3B+T2cK5a7Wculy15HWU7aFSCaQlh/Pk/rWkJYcjCFDX1MOrR65TUNLkUv3s5SIoyJstW1L44Ac2kLs2Dk9PHWPjkxQVN/Pqa1c5cbKMltY+JbUa+NnPfkZ8fDwGg4GNGzdy+fJlh/b785//jCAIPPbYYws+Kysr45FHHsHPzw8vLy/Wr18/R6F4x44dCIIwp332s5911yndsShzvW0UVWr73Gmq1EtSx1hRpZZEUaVWuJXIdoo/9KEP0dfXR2hoKFlZWeTm5s5pdzOZgaGoBRVdYyO0jQ7J2nd1SBh6tYa+8TFq+ntlHzsjOAR/gwHj1BTXnaw57KnTsSFGdK7PNzS6dKP29zRwT2KsaKumkQGjvJJTtgjz82bLqun1xVWNtPfLu8b20KhV7MtLJcDbg5HxSQ5fLWPMTWWg5hPk58XDWzPx9tAxMDLGobNF9A6676ZvD4Ney/1b0tiQFYcgCNQ29fD6e0V099+ask1W9Hotm9cl8sieHMJDfDGZzFwrbuLg29epbey+LRxKg0Er1jp+fB07tqUSFuaLxQJNzb28+14pB1+/RlFxM2Njy5MdsNL4y1/+whe+8AW+9a1vce3aNXJycti7dy+dnfa1EOrr6/m///f/cu+99y74rKamhq1bt5KWlsapU6coLCzkG9/4BgaDYU6/T3/607S1tc20733ve249tzsRZa6XZlkd46VYY6w4xnZRyjXZZ6kdY89ldoyXYo3xSnCMRycm3dbuFmSvKf7Qhz7EyZMn+cAHPkBYWNgCBcNvfetbbh3grcCaP/+9y4cXrCk2m2y/R7BML1B5p7GKkt5OUvyD2R+fKrkG2VbZmQstjVxqbSHM04un0rLnXlsH7FR2d3O0uhqtWs0zOWvw0C4sFyNZqmn6V2Aym/nTtSL6jEayI8LYkZjg8Phh4Vrg1wvKaOkfJNLPh8dzMhxWPpbabjFZeKeoipqOXrz0Oj60KQsPnVa+arTE+IdHx3ntYgkjYxOE+Xvz8IYMNGqVW8slWfcZMY5z5EIZfUNGdFo1+zakER5kY72Gm0omze/f0TPEyYuVjIxOoFIJbMyJJz1x4d+048d1zzgtZgt1jT1cKahn1ChO5CFB3mxYm0BosI8N+/LWCMsv1eT8GuT+/lEqK9uoru5kclJ8qFKpBGJjg0hZFU54uJ/N4xqNI3z22aVZU9zW1e2WNcURIcGyxrdx40bWr1/PT3/6UwDMZjMxMTH80z/9E1/5ylds7mMymdi2bRuf+MQneP/99+nv7+f111+f+fypp55Cq9Xy+9//XvK4O3bsYM2aNfzoRz9y+PwU7q65/icvL1xT7MhaVEWV2r79u1mVWrRlf5JQVKkVVWrj0Aj/uOaJJZnrE7/1r6jmvSB2BvPYGLXf/tpdsaZYtlPs5eXFsWPH2Lp161KN6ZbjilPcZRzhlYoCBOCTGevw1kj8IG38LY1OTvKbG9eYsph5YlUGsb7+s/ov7hRbLBb+XFRE1+goa8Mj2BYXt6D7Yk4xQFP/AK8XlSEAT+VkETLvGkiNHxY6QIPGMf50pZBJk5mtSXGsiY6w23+x7ZhhYsrEgUtF9I+OER3ox0Nr01BL3bVkOsVYoHdolNcvljAxZSIuxJ89uSloBImZ1QWnGGBsYpJjF8vp7BtGrVaxO28VceGB8/ovjVMMMD4xyZnLNTS29QEQFxnIlrxEPOzU311qp9i6fWrKRHFFK0VlLUxNiV9YfEwQedmx+PrcrGO4kp1iK1NTJupqu6iobKen5+YbaW9vA8lJoSQlhswp67SUTrE7JkrrJNnU1DRnfHq9Hr2NGpYTExN4enpy4MCBOSnQH//4x+nv7+fQoUM2j/Otb32LwsJCXnvtNZ5++uk5TrHZbMbPz48vfelLnD17luvXr5OQkMBXv/rVOcfYsWMHJSUlWCwWwsPDefjhh/nGN76Bp+fChyuFm9xNc72zTjE44Bi7KMC1qGPsooDVbMc4LTqU7fPrGLtof1HH2EX7iznGrgpwLeoYu+AUg4OOsQsCXA45xi4IcC10jFfjq5s7v7giwOWIY+yKAJcjjrErAlyLOcaKU7yykO0Up6Wl8de//pXs7OylGtMtxxWnGOBvVUU0jwyyLjSKreHyIq2nGuop6GwjytuXD6auntXfsYhzQ38/h8rLUQsCf5+dg9+8PwhHnGKAI2WVVHf3EubtxQezV6Oaf1dw0CkGKG4R1ajVgsAHc7MI9va029/edutxe4ZHefVyMVMmM1kx4WxLjbfd3wmnGKCtV6xhbDJbSAgL5P6c5AVvoO3ad9ApBtFhevdqJU0d/QjAPVkJZCaEz+q/dE4xiJHZkqo2rhQ1YrZYMOi1bM1LJC4y0Gb/5XKKrYwaJ7hW1EhVrZhmKwgCqUmhZGdE4+Wpvy2c4tn9e3qGqapqp7auayZ6DBAe5se2e1Pw8NDdNk7xfL71rW/xwgsvLNje2tpKVFQU58+fZ/PmzTPbv/SlL3H69GkuXbq0YJ+zZ8/y1FNPUVBQQHBw8AKnuL29nYiICDw9PXnxxRe57777OHr0KF/72tc4efIk27dvB+CXv/wlcXFxREZGUlhYyJe//GU2bNhwR9TZXUruprneFacYFFXqxezfaarUcxxjF51iuDNUqe05xooqtbRjvJROsTuywqz25GaG3a7IdorfeustfvKTn/CLX/yC+Pj4JRrWrcVVp7hmoJc36srQqdR8Mn09erWNdBgJp2xobIKXi69hslj4QEom0T5+0/0dc4otFguvl5fTNDBASlAQDySvmvO5o07x8PgEf8i/wYTJxPbEeHIiwud2kOEUWywWDhdW0NDbT5CXJx/MXT1TdspZpxigpqOXY4WVAGxPS2B1dJiNg0vYd6B8UFNXP0fyK8SJPDyIXdk2Ur/c4BSDGPU6e6OOikbR8ctKjGBjZpz4MmKJnWJr/+6+EU5fqaJ/UBTWWRUXwqY18ei0c3+/y+0UW+ntG+FqYQMt0yWQ1CqBtFXhZKdF4WGwEdleoU6xlakpEw0N3VTXdNLePoCHh44PPLEOlUq4bdKnHY0Uy3WKh4aGyM7O5r/+67944IEHABY4xVabH/nIR/jjH/84s+8jjzyCl5cXf/rTn2yO/cSJE+zatYvq6mqSkpKcvgZ3OnfTXO+qUwx2HGM3OMUAJQ3tnC2qByA7MYKNGdOOsRucSoCKtm5OFtpwjN1kv7arl2O2IsZust/cP8DhooWOsTucYrBTrskNTjGIjvGr02uLFzjGLjrFsIhj7KJTDPZTqV11isG+Y+yqUwwLHeNHY3IImXaMXXWKQdoxXkqnWCnJJB/ZQlsf/ehHOXnyJElJSfj4+BAYGDin3YkMTIxhnHJ8oXmibwCBeg8mzCZKeuUpL/vo9GQGhwJwsa1J1r4gRtG2xooCV5U9PbQPOydG5a3XsTleFBI539DE8LjzIg2CILArNREPrYaekVEu1DYuvpMDJIUFsiFJHOP7FfU09w64xa6VmBB/9uaKitfV7T2cLK6RrQzuKCqVim05iaxLE8+nqLaNY5fKl1UZOjjAi0d3ZZOVIiqhVzV0cfD4DVo73XtdnSUwwIs92zN4YGcmYcE+mMwWSira+Ntb17ha2Mj4MpW3chcajZqkpDD23r+aJx/P494tqxa+dFkCPHVatzQAX1/fOc2WQwwQHByMWq2mY17JuI6ODsLDwxf0r6mpob6+nocffhiNRoNGo+F3v/sdb7zxBhqNhpqaGoKDg9FoNGRkZMzZNz09fY769Hw2btwIQHV1tazrdrdxN871rqCoUtvnThPfUlSp56KoUttnvvhW9ZD7KqgouA8bIUz73G1iJec66sjvaWZTSBwbQmId2kcQBHJDI3m3qYbr3a3kBEegllqTaoN14dEUd3fSPDRI89DAzWixg4R4eZEREkJpVxdnGhr4YEbmouJJtsiKCKOis5v2oWFO19axPz1Vtg0rnjodu1KTOFxcwY2WdmID/YkL9HfanpW8hEj6Rkapau/haGElH9iQhb+n62sorMSFBnD/2lUcv15FZWs3giBw3+pEp67nYgiCQG5KNH5eBk5dr6aps59DZ4vZuz4VXy/3nZM9NGoVG7LjiI0M4MyVaoZGxjlyppT0pDDWZ8Wh1agXN7LEhIf68cCu1bS2D3CtqJHu3mEKy1ooq24nMyWCzJQI9DrZt7Zbire3Yc6a4jsNnU5HXl4e77333sx6X7PZzHvvvcdzzz23oH9aWhpFRUVztn39619naGiI//zP/yQmJgadTsf69eupqKiY06+yspI4G3oKVgoKCgCIiIiQ7KNw98317iAndbo0YmkT+WWiU7w2PcZt9jPixRdIZ4vqKawRq0zMRIzdQEpUMAAnC6spaxKzlrbNX2PsAlbH+FhhFdUdPQDszrKRgeUkVsf4cFHFjGP8QPZC8S1nsTrGh4rLZ1SpH8myIb7lJFbH+GBZ6Yxj/FhaBgate+Yzq2N8oLJkxjF+MiUTL50N8S0nsDrGB6qLGZgY40B1ER9IzsJP4mWpXKyq1AfrxYjxq/VFPBmfRaDBY/GdHcDqGL/RXEjn2BCvN93gsZgcQjwWim85g9UxLh1sZrWfY/6EwvIiO336bsCaKvD/XTlMw9QIx1oq8NboeCZlA5ht31zN83Jupsxmfl2Sz+jUJPtiU0gLCJm3g1Q6tLj9vYYairo6iPL25QOpmQhSOT0SdkbGJ/jfggKmzGYeXLWKVUFB0/1tm5FKq+4eHuUvBUWYLRb2p6eQGBQ43V/CziJpz6cr6yhq7cBDq+Gpddl4S92MZaQAT5nMHLpSSsfAMP5eBp5cvxr99CTiLlXqmtYe3r1RhcUyN7VMdmqwg2Pq6hvmnUvljI5PYtBpuH99KhHzlamXOF15atLE5cIGymvF6J63p56teYlEhfnLPK68W4yc8VssFppa+rhW1Ehfv/jWWKtVk2F1jm0Ihi39GmSZH9jYbDSO8LnPPbBiU6qcsfWXv/yFj3/84/z3f/83GzZs4Ec/+hF//etfKS8vJywsjI997GNERUXx0ksv2dx/fvo0wGuvvcaHP/xhfvazn82sKX7++ec5deoUW7dupaamhj/+8Y88+OCDBAUFUVhYyOc//3mio6M5ffq0S9dA4fbH+jv+6W8Xpk/LTd9VVKkdt39XqFLbtWV/UlFUqe0/AN7uqtSzMQ6N8Nzax1fsXL8U9lYyDr0+GxmRV8NUbv+VTLJvMB5qLcNTE9QM9Ti8n0alYk2IGInI72yRnYa0ISIatSDQMjxI05D89FVvnY686UjIucZGppxMIwr28mRtlGjnVE09Ey6m82xJiiPIyxPj5BTHy6rdko6sUat4ICcFL72O/pEx3imqcmvaFEBSRBA7lzC1bD4hAd48tj2LYD8vxiameOt8KeUN8lLxXUWrUbMlN5F996bj7alneHSco++X8f7VasZXSN06QRCIjQrk0b057LgnBX8/TyYnTdwoaeZvb17j6o0GjGPuSxFTcJ4Pf/jDfP/73+eb3/wma9asoaCggKNHjxIWJmoBNDY20iazxvrjjz/OL37xC773ve+RlZXFr371K1599dUZxWSdTse7777Lnj17SEtL44tf/CJPPvkkb775ptvP707gbp7r3cmalKVNpc6MD2fr6nhgieoYRwazM+vmfHe62M2p1CGB7FvqVOrVS5hK7bMMdYzTMvC4nVOpk7KWNJX68bilTaV+OCpnyeoYK6xcHHKKk5OT+bd/+ze7DywWi4Xjx4/zwAMP8OMf/9htA7zVaFQqVgeIKUs3elpk7ZsdHIZGpaJrbISGoX5Z+/ro9GSFiA+L55qdm/ByIyPx0moZGB/nmsyHzdlsiInGz6BnZGKC9+tcWw+sUavYl7EKjUpFc/8gl+rd87Dgqdfx4BpxEmzqGeB0mfud1lWRwezIFoV5ypo6ee9Gtdud79l4eeh5eGsmCRGBmC0Wztyo5cyNGqZMS3dMW0SF+fPEnhwyksS/g8r6Lg4cK6C6sWtJXwzIQRAEEmKDeWxfDvdtSSXA35PJKRNFZS387c1rnL9aw8CQ8VYP867nueeeo6GhgfHxcS5dujSzvhfg1KlTvPzyy5L7vvzyy3OixFY+8YlPUFVVhdFopKCggEcffXTms5iYGE6fPk1PTw9jY2NUVVXxve99745/2+0sd/Nc727uJMe4vHkJ1hgrjrFdFMfYPopjrLAUOOQUnzp1iitXrpCQkMDGjRt59tln+e53v8sPfvADvv71r/PEE08QGRnJJz7xCR5++GG+9KUvLfW4l5XswAhUCLSMDtI55rhwlYdGS1ag6Ehc6ZQ/Ia6PiEarUtExOkx1v+NRais6tZqt02vrrrS0MDg2JtsGiI7srlWiM1ja0Uldb59TdqwEeHlwX2oiAFcbW6jrcc2elRBfL/Zkr0IQoLy1i4vV8oXKFiM1KoTda5LFibyth2PXK5fUSdVo1OxelzIjwFXe0MkbZ4sZHHHuu3QWrUbN5rUJ7N+Rib+vB2PjU5y+XM3R98voH3TfROQqgiAQHxPEo3tz2Lk1jZAgb0wmMxXVHRx86zrvnS2no9s58TkFhTudu32udzfL4hhbxbdq2rhUugSOcc4Sim+FBLIvJ+X2Fd+y4RhPuN0xzry9xbfuMMe4S3GM72gccopTU1N59dVXqays5EMf+hAtLS0cOHCA//mf/+HUqVNERUXxP//zP9TX1/OP//iPqN0kOrBS8NbqWeUnClAU9MqLFueFRopp0CODtAzLS4P20urIDROVgM+1NmKyyL+ZpwYFEe3ry5TZzOmGBtn7W4ny82VNpJhGfaKqFuOka+mzqWHBZEWJkfDj5dUMGN3j5MWHBLAjXXS4r9e3UtDgfIRciuSIYPbmpqBWCTR09fPW1TKX08rtYRXgenBTOnqdhu6BEQ6eLqS+vXfJjilFeLAvj+3OJi8zBrVKoLVzgIPHC7lc2MDklPseBlxFEATiogPZvzuLffdlEhMZAEBjcy9vvVfM4XeLqG/ucesDmILC7c7dPtf39Y9QUePeZSq3uyr1qsilVaWODwlQVKntEOzpqahS2+F2V6VWWFnIkuSLjY3li1/8Iq+//jrXr1+nvLycs2fP8pOf/ISHHnrojpsgZ7M2SFSVrBrsYnjS8T9ob62e9ACxxNJlJ6LFeeFReGg09I+PUdLdKXt/QRDYER+PShCo7eujts/5qOzm+BgCPDwYnZzkZK3rE+PWpDjCfbwZnzLxdqn7Iq7pUaFsTBYfQs5VNVDR1u0Wu7OJCw3gofXp6DRqWvuGeONyGcYlXmcbHerPk9uzCQvwZmLKxDtXKrlU2rDsjp1apWJNejRP7FlDbEQAFouFospW/vZOAbXN3SsmpRrE339EmB+7t6Xz+ANrWJUYikol0NkzzIlzlbz69nWKK1qZmFi+0lcKCiudu3GuHzVOcORUCeeu1lBa5d6XqcvqGC9FKrVSrsku8x3jJUmlVhxjSTyX2TFWUqntc+bMGR5++GEiIyMRBGHBsqeOjg6efvppIiMj8fT0ZN++fVRVVdm1uWPHDlHcdl7bv3//TJ+nn356wef79u2TNXb36NTfoQiCZaaFe3oT6emLGQvF/a0IAnMbEk2A9WHRCEDDUD8dxiFRsU6yWeY0nUbFxkhRufJiWxMTZpOdfW82y6wW6OXJ2ggxjft0Yz2TNmxYJNrsPhq1ij2pSeLE1dNLZW8PFhVzmwN2rE2tVrEvcxUeWg3dw6Ocqq7DjEWyv6PjRIDchEiyY8VzPlFaQ0NP/6LjQWDh+Uw3W30jgnx5eINYLqFrcIRDl0oZGhu3a9/eOTjSvDz17N+ayepEMWp/o6aNwxdKGR6fxCIIizTHvxtHroWvj4H7t6Zx/5Y0fLz0jBonOHGpirffL6V3aPTmMVSCzbbwj0hskucvNR6pNu98/P092boxmQ8+nEd2RjQ6nYahkXEuFzTw5zfzuXC9jv7hsYXjlGhS45FsEt+LlH0FBYVlwmLBQ68hJUF8gX3xWh1llW3zp2NR/Fai2eo7uy1IpS5rEpX55zXBjM222HEz4+atMS5txGK2ODxOSfvTx0+NmCu+daa4zj32p1vi/DXGRdWYTa7bt7Zo/7lrjI8UVmKeMsu+zlLHtbnGeMokYUuw2ewdO8TDiyfnrzGenFr0vOc0s2C7IeEYT0zIsyN5bAEfrWFBKvXA+LitPwEsFsFmk3o4MFvAoF6YSt07ZpS0ZauZJZrFIqBTaRemUhuHbY7fZFbZbFKX505jZGSEnJwcfvazny34zGKx8Nhjj1FbW8uhQ4e4fv06cXFx7N69265w48GDB2lra5tpxcXFqNVqPvjBD87pt2/fvjn9/vSnP8kauyynuLS0lH/8x39k7dq1REREEBERwdq1a/nHf/xHSktLZR34dmRtkJjKXNTfJjqWDuKvN5A6XZLpUrv8da5ZIWH46Q2MTk1yraNV9v4A66Oj8dbpGBwf50qrczYAQn28WRcjRs1P1dQxPO7a20Rvg569GavEiba9i5JW+dFwWwiCwJbUOFaFB2G2WDh6o5L2fvevJQ3x8+LRTZl4GXT0jRh5/WIJvUNLu8ZWrVKxOSue3etS0GrUtPcOcfDUDRpuQTo1QGxkAE/sXUNuRjRqlUBb1yCvHb/Bhet1GMdXhkr1bDw9dORlx/KhR/LYvC4Rf18PpqbMlFW1c/Dt6xw/XUpTa6+SWq1w13I3zvWCIJCXFUt2mjjPX7xW5/aI8Z0kvqWoUi9EEd+yz524xnglR4xHJybd1uTwwAMP8OKLL/L4448v+KyqqoqLFy/y85//nPXr15OamsrPf/5zjEajXQc2MDCQ8PDwmXb8+HE8PT0XOMV6vX5Ov4CAAFljd7hO8ZEjR3jsscfIzc1l7969M2U0Ojo6OH78OPn5+Rw6dIi9e/fKGsBKxFqT6/tX38TD+2btQrPFwsuVVxmcHGdHWDLZgZE3P5Oo82uZLm7XOzbK78qvYwE+siqbMA8J9VOJ+3NFdw9H6irRqlQ8vTr3Zk05qXrHNr7V6p5e3q6sRCUIfGR1FsGes+q6SdixVc7NZDZz4EYJncMjRPv58ujqdFTTBdsk6xdLzTvT9vMbW7hQ24RKEHg4O41Yfz+Hz8uefZPJzNsFFTT1DKDXqHkkL4MQXy+31S+29h8yjvPWlTL6R8bQadTsWbOKmGB/WceQU5vZ2n9g2Mh7VyvpmRa7SosLZXNGPFqNjfRGuXWNJbA3nsHhMS4V1tPY2geIAl05aVFkrgpHMy/l8lbUNba13WKx0NYxQGllG03T4wbw8tSxKjGMlIRQvD0X1liUPIDc79fGNqNxhM/9w74VW7vwbqpbeLdxN871P/3NsZk6xRaLhfyiRgrLxZfHm3ITyFgVMbOPK/WLrditY+xC/WIrdusYyxinVH+7dYzdYN9uHWM32LdXx9iV+sVWFq1jLGnLsUli0TrGMn+j8+sOL1rH2EE7Uv0Xq2PsbP1iK4vVMXa2frGVxeoYy6lfbBwa4Z+WqE5x8lf+FbXe4LI90/gY1f/2NZqamuaMUa/Xo9fbeja6iSAIvPbaazz22GMAFBUVkZ2dTXV1NUlJSTP9YmJi2LVrl90KFLPJyspi8+bN/PKXv5zZ9vTTT/P666+j0+kICAhg586dvPjiiwQFBTl8rg47xTk5OTz66KP8y7/8i83PX3jhBQ4ePEhhYaHDB1+pSDnFANe72jjdUYOv1sDHktbPOIOLOcUAxxqqKO3rJNbHnycSVts+uITzZTHBXyqKaB8ZJjMolPvjk6f7O+4UWywW3iqvpLa/j3Avbz6YmTkzfjlOMUDfqJE/Xy9iymxmc1zMTPTYWafYYrFwvKyGys5udGo1H8xdTaCnh0PntZj9SZOJN6+V094/hF6r4dG8dELmfa+u2LcyNjHJsWuVtPWJKfLbMhLIiAlz+BjOOMUgOv5XyhspqhGjGr5eBnauTSY0wGde/6V3iq20dg5wubCBnn4xHcbLQ0deZgzJcSEzD08rxSmezcCQkYrqdqrruhifXmcsCBAdEUBqYhjREQE3H85uU6e4ravbLU5xREiw4hTfgdyNc/1spximHePiJgrLRGHNTWsTyEgRHWN3OMVgxzF2g1MMUNLQztmiemCeY+wGpxKgsr2bEzdsOMZusl/X3cfRG5ULHWM32W/uH+Bw0ULH2B1OMSx0jB+ejiDbt+X4JNE9Osqr5SWM2XKMXXSKYRHH2EWnGOw7xq46xWDfMXbVKQb7jvGd6hTP51vf+hYvvPCC3X3nO8WTk5MkJyezceNG/vu//xsvLy9++MMf8pWvfIU9e/Zw7NixRcdz+fJlNm7cyKVLl9iwYcPM9j//+c94enqSkJBATU0NX/va1/D29ubChQsO62A47BR7eHhQUFBAamqqzc8rKipYs2YNRuPtXwvUnlM8Pmnht9WXGDNNsS8qjRRfcQ2SI07xwPgYL5dfw2yx8GTiamK8/RfuIOV8mQVah4f4a0URAP8nPZtQT29ZTjHAsHGcVwoLmTCb2B4Xz5rw8Bn7trDncJS2d/JedS0qQeDJrAzCfX2cdooBpkxmXr9RSvvgMH4GPR/MXY2HVuvQeS1mf2JqijevldMxMCw6xrnphPjYcIxdcIpBdFBPFddS1SqKe+XER7A5ddZbejvHcNYpttLSNfD/Z++9w9u6rnTv30EhwAoSBHvvVRLVbclFsuReYztxycR24kzKpFwnM8mN5xt7Zm48cfpkJsmdJL6ZiRPXuNuyLRfZkq1eKYq9dwLsBSABopzvDxAQCyoJSpR03ufhI/tgn7U3QBAb737Xehf7TjVjMk8jCLC2IJ11BelnSdw5JMXO6URaOgc5XtOJadKZghWniWBDWQYZKXHIvH4zOH+k2AWb3UFn9xANLQb0/ePu6xHhYeRnJ1CQk4gm2stms8JJcf7/XvpGabeYaf7JP0qk+CLEpbjXzyfF4PxzPXGmcwExDhUpFgU43eCBGIeIFIsC1LZ7IMZe/AqCfl4yaOwZ5OOqecQ4VPEFaOsf5r35irE8NPERZhTjecRYpvDyQgdLxvFBjENAigEGpky8Wle7kBiHgBSDD2IcAlIMMGE183Jz9QJiHApSDGC2T/Nq+0JiHApSDDDtsPJmd9UCYrxSSHEoDsBd8VISdCFRigFOnDjBww8/zOnTp5HL5ezcuROZTIYoirz77rt+1/PVr36VQ4cO+T2YbW1tJS8vjw8//JAdO3b4jQtB1BRnZ2fz9ttve3387bffJmumJ+7FDKVMzpo4pyp6YrA7qHoajUrNqnincnhQ3xF0LU5qVDRFcc7WUB93Lq6WJ1qlYmums6bpYFcn45bF13OUJCVQoJup2W1ocpo+LAEKuYybyouIVqsYM1t4pyZ09T5hCgW3rC0mSROFxWrjjZN1DE54L+pfLORyGdeszmNjvvPU/3R7H+9VNoa0rsgb0hI03LVtDXlp8YginGzs5o39ZxhZ5hpnbxAEgfysBO6+fi0bV2USppQzMjbJBwcb2LW3hr6BsfOyrkCgkMvIzUrgxmvKuevGtZQXpaJWKZicmqaqrodX3jnF23uqaWztx2pdOa2oJEhYKqS93gl3jXGJc78/fKqN2kbJlXo2JFdq3/DUrinkfYwlV2qvuNRdqSPClCH7AYiJiZnz448Qe8P69euprKxkdHSUvr4+du/ezdDQELm5uX7vNZlMvPDCCzz88MN+x+bm5qLT6Whubg54bQErxS+99BL3338/N954Izt37pxTZ7Rnzx52797Nc889x1133RXw5CsVrlOWXx5/c4FSbHfImLJZ+e+mo9hEB3dklpMVFedVKXbMO0Y0Wqf575qT2EUHt+eUkBOjnXeDlyOmmesT0xaerj6FzeHg+pwCSrQJPscvvO5U8F6praV3YoJMjYbbi4uReTnuFLweczv/sdhsvHjqDGNmCznaOG4pLpyris4bvyC+h/1n2DTJyydqmLbbKUrSsbM472zKbZD71fzxFquNt07U0T9uQq1UcNv6EnSzFOOga429jXdAU+8gH1e14BBFdDGR3LCukKhwVciUTV8Kb3P3IAeqWpm22pHJBNYXZ7AmN3VW6u/s+EEqyItUQi3TNqoaeqhp0mOfab+VlqRhQ3kmurgozzcHtJ5lVpZnYLc76OodoanVQE/fqHtahUJGdkY8+dmJJCfGeFfAg0i3npoy8XdfkdKnJZx7XIp7/W//nweleEYyWFBjPCuVes74JSjIPmuM3XEWryD7rDEOYp3exvusMQ5BfJ81xiGI76vG2B0nSKV+9tw+U6kDeg6+N1e/Ncb+1upH+fVbYxxgHG/j/dUYu4cvUkH2V2PsL44/Bdlit7Gr5zQGLzXGvtY/ZTTxyLrbV6x/yFLjeVKK56OpqYni4mLeffddrrvuOp/x/vSnP/G1r32Nnp4ev7XC3d3dZGZm8vrrr3PbbbcFtN6AleLPfvaz7Nu3j4iICH7xi1/wwAMP8MADD/CLX/yC8PBw9u7de1FskoEgXKGkPM6ZdnxssDOoe6OUYVQkOO892Bf8qW10mIpNKc4Nc393+6JOHQVBYEduLnJBoHNsjJqBgaBjuKBSKLihpBCZINA2PMKpXv2iY7mgjYzg+jKnI3WDYZATnYt3y54PlVLBretLSIyJxGy18eaJ5VGMAQpSddy6qQR1mILBcROvHKymb3jc/40hQH66jru3ryEjKRaHQ+RYbSdvHqhmePz8qMYAqjAFG1dl8bkb11KSl4QgCPQYxnhjzxn2HGpgcGR5fg+hglzuJL/XXl3KZ29bz/rVmcREq7HZHDS3DbD74xpeeusEx053MDxqWlH9mmcjlCfHEi4+SHv9XCxwpV4Gxficu1LXSq7Us3GuXamXRTGWXKm94ly4Ut9yAblSLzeMRiOVlZVUVlYC0NbWRmVlJZ2dTr700ksvsXfvXndbpmuvvZY77rhjDiF+4IEHePTRRxfE/uMf/8gdd9yxgBAbjUa+973vcfjwYdrb29mzZw+33347+fn5QZlCBqwUX0rwpxQDGK0W/tR8DLsocmfWKtLC4zzGmq8UA06lueYk0w47N2QWUByXOOsG30oxgM3h4C81lYxZzKxNTOHqjByf4+deP/ufJ3t72d/ZiVIm4/Plq9GoF9YZ+lOKXajq1bOvpR0B+Ex5KemaGJ/j3fF91Oqe6dGzr6kdgB1FuZSkJC5ZKXbBMm3jrZNOxVilkHPLumKSNNEhVYpdmJiysPtEA0MTk8gEgcuLMlmVlbzwJD2ESrH7VlGkqWuAg2fasdrsyASBioI01hakIZe7pZDg4oegZhZgfMLMydouWjoH3dcykmNZU5JOUnz0gvHnWyn2NF4URfoHJ2huH6C9c5DpWanUsTHh5GbqyMnUoYkO9z7BOVaKJfdpCRKcCEQpdv+/5ErtN77kSu17bsmV2ve8l6Ir9cWoFO/du5ft27cvuP7ggw/ypz/9if/8z//kZz/7GQaDgZSUFB544AEee+wxwsLOvpe2bdtGdnb2HDfqhoYGiouLef/997n22mvnxJ6amuKOO+7g1KlTjI6OkpqaynXXXccPf/hDd7ZTIJBIsQcEQooBPu5rpmqkj9SIGO7MWOMxbdgTKQY40tfNQX0n0UoVDxavQ+FK1QmAFAO0j43welMdAnCfy3TLx/iz12evTeTVmTTqlKho7i4pPetGPYNASbEoirzf0EzjwBDhSiX3rVlFlCrM63h3fD8GVgdaOjjV1YcA3FheSJ5W6+WG4ONbrDZ2nXKabynkMm5YU0iWNtbnehbED4AUA1htdvZVt9LcNwRAQUo8V5flzm2dtAyk2AXjlIUDp9voNIwAoIlSc8WqXNISNOeNFLvGj4xPcrq+h9bOQXeIlIQYKkrSSUmIOf9u1d4wb7zN7qCrZ5i2jkG6+kbmKBna2EhyMuLJydASEx3uMw5IpFiChHOBYEgxeHClnkWMQ+ZK3dTDsZrldKU2sP9MGyC5UntCqFypvc2tnzDyRnWdZ2IcAgMuyZV69gMLr59rV+rbM9aQ4MOV+mIkxRcyAk6f9oe6urqAiqQvJmzUZSAXBHonx+meHA3q3rUJqUQpw5iwWqgcDD49OFsTR0FcPCKwp6MVxyLONmSCwHV5eSjlcvqMExzvW3yasiAIXFOQiy4igimrlXcaGrGFIP1oS24mJckJiMB7NU10j4bOnEmlVHDbuhLStRpsdgfvnGqgyTAUsvizoVTI2bEmny0lWQgCNPUN8erhGkZN58bBNSpcxfWbiti5oZBwlZIxo5m3D9Xy0YkmJs2hS4NaDOJiIti2qYC7b6igMDsRQRDoGxjn3U9qeevjatp7hlZsKvJsKOQycjJ17LiimPtu38gVG/NIS45FEGB41MSJM528/E4lb7xfxem6HsYmLnz3XgmXHi7Fvd6FBeZbJ9uobQqx+dYyp1KXZictq/lWQerymm9lJ8Rd2OZb0VFzUqnfrK4PqRGnLiJCMt/ygfBzbL71xiWeSn2hIWSkeHp6mo6OjlCFuyAQpVRRHuc8JT4yGJybtFIm5/LkTACOGrqZslmDnn9bRg5hMjmGSSNnBhZXyxujVrMtOxuAIz3dGEyL/+NVyuXcVFyISi5HP2Hk07b2RcdyQRAEthfmkquLwy6K7KpppH8idB8wSoWcm9cWkZekxSGKvH+miTNdS6+L9gRBEFidncJtG0sJD1MybJzk5YPVNPcN+r85RPPnpsbzuWsqKM12ppM09wzy149PU9OmX9TBSigRExXOlRvy3DXHcpnAwLCRPYcaeeW9SurbDG6DrpUOVZiCwtwkrr+6lPtu38jWDXmkJmkQBBgaMXGiqpNX3qnktd2VnKzuYmgF1yBLkDAbl+JePxvnhBhLrtQ+cbG5UoeeGEuu1L5wqbtSS/COgNOnv/vd7/p8fGBggOeeew77OWg9s9xwpQr8+4k3fKZPg7O2+H+aTjjdpDPKyYqam97rq3+xQxR5tqGSQfMkFboUtqfnIgaQ9jwbp/V6Pu5qI0wm54GytUS5cvK9pvUujC+KIu82NtE8PEycWs19q1adTefx2nfYe1p1+/AIb9U2AHBNfi5lyYkBpxm743tITX2rqp6e0XHUSgV3VZQRF3k2DXWptcYOUeTTujZquvsB2JibzobctABSdwNb//x7TOZpPqxsom94AoDSzES2FGejkC88pwrZ3POuD4wY2X+6lcExp8GVLjaSK1bnkhAb5ec5nJt06ynzNDVNeupa9O5a3XC1krL8FIpzk1CFeXDXDGQ93hBsGnaw35FEMJutdHQP0d49TJ9hbM4XxegoNbfsXIVarWRq0sQ3/vb6FZtSdSmlU11quBT3+t8+tTB92ld/XvBfY+weL7lSX5qu1AHM7bfG2G8cyZUaVrYrta8aY3CmT39nvZQ+vVIQMCmWy+VUVFR4fUGMRiMnT568qDbKQEgxwL6+Vk4N95IcHs1nsyrmfOj7IsUAnROjvNJSgwB8oXgt2rBIj+O9kUeHHV6sP4Nh0khBXDw35xbNTODlyXlZj3nayrNVVZisVlYnJbE9J8fnvL5IMcCxzm4Od3YjEwTuKC8h3dsfUoCkGGDaZuO1U3UMGE1EqcK4c20ZMeqZWpMQGHCJosixlm6OtzrrxVZlJLG1KBuZIIScFAM4HCLHm7o52eKcLz46gmsrCoiNmltzulykGJyHAXVtBo7VdWK1Of92izIT2VCcQaTKw+bmY+LlqkGettppaDNQ09iHacp5Gq1UyCjITqQkL5nY+TW6/tbjDeeAFM+GZdpGV88wHd3D9OhHiYwI486b1iIIgkSKJZw3XIp7/WJIMQTWrmmptcZ+ifESa43nEOPcFDaXziPGS6zVnU2Mi9MTubo8J6Tx/RLjJcb3R4yXQoohwHZNS6g1DogYL6HWOCBivIRa40CI8VJqjQMhxkupNfZHjCVSvLIQMCkuKiriscce42/+5m88Pl5ZWcn69esvqo0yUFJssk7zP03HsYkObkkvIzf6rFW4P1IM8EZrHa3jw2RGafhMTrnnPr/evoA7BPonjTxfV4UI3JpXTF6sNmhSLIjQMTrK6/X1ANxcWEi+VrtoUiyKIrsbmmgeHEatUPC51eXEhi90tw6GFANMWay8cqqG0SkzMWoVd64tI0oVFjJXakSo6tSzv6EdgGxdHNeuyvfcU3BmvMf4AZBiF7oGRtlzuhnztA2FXMbWkmyK0xNCr1L7uD5pnuZIbQfN3c5UbqVcRkVBGqtyU1AseO7nx5jLbnfQ2jXEmYYeRsbP1uOmJWkozUshIyV2zt/OSifFs2G12jGazMTFOj9vJFIs4XzhUtzrF0uKQXKlDiT+JelKHcTckiu173kvZlfq5STFfQODISPFKQm6S2K/D5gUf/7znycxMZF///d/9/j46dOnWbt2LY4Q1l2cLwRLigE+1XdwYqgLrSqC+3PWu12cAyHFoxYzf64/iV0UuTmzmIJY3cIbfJBigP3dHRw39BChUPKFsgrC5V76iPogxQCfdnRwsq+PMLmc+1c5+8Z5HO+HFANY7XZePVNLv9FErFrNZ1eXEa6ct64gSTEiGM0WXq2sZdxsQaNWcUdFqceUGl/w53rdpB/ko5oW7A6R+KgIbq4oIlrtYY4QkGJwplN/dLqZniFnH+PsxDiuLs8lXKU8J6TYBcPwBIeq2xkYdda/RKrD2FySSV6abhbhPD+kePaBS2//GLXNejr7RtwPR0eqKMlNpiA7AbWv181r/PNHiudDIsUSzhcuxb1+KaQYJFfqQOJfcq7UPmJJrtTe43gbf7G5UruI8XKS4qL/9SPkKs/f4YOB3WKm4T/+8ZLY7wM22vrFL37BI4884vXxNWvWXBSb5GKxPj4dlUzBsGWShrH+oO6NVanZmOjc4Pb1tS6qqftlqRlo1eFM2qx83Nka9P0ubMnIICUqimm7nbcbl+YgrZTLuaW0iGhVGKNmM2/Xh8aROspFhNUqxswWXqusZcIcOhMGgIJkHbevdxpiDRkneeloNfqxiZDOMRuR6jBu2VjCZUWZyASB9v4RXtx/mjb98LLN6QlJ2mhuv7Kc7evyiVSHOcn6qWZe//QMPYNj53Qt3iAIAmlJsVy7tZjP3rCW8oIUwpRyJkwWjp7p4IW3T7D3aBP6wXHJvEqChCAh7fXBQ3Kl9g/Jldo3JFdq37jYXKmrRntCFltC6CD1KfaAxSjFDoeME0NdHOhvI1qh4gt5G1HIZAEpxQA2h52n608xPm1hQ0I6V6Rkz5vAy2JnxTeYjLxQ70yjvimnkMI4T4qzb6UYYMJi4fkzZ5iy2ShPSGRHzsL2G4EoxS4MmSZ5uaqGabudIl081xXmn1UdF6EUu9c5Q4jHzRZiVCo+s6aEGHVgp2L+lGL3HFMW3qlsYMg4iVwmsKM0j4Jkndfxftfv4x7X9cFxEx9VtTA84fxALkzVcUVJNiqlwuP4gOcO8rrdZudMax+VTT1YZ1yf0xM0bCrJRKdZWPt+rpRiT7Da7LR0DlLfamBo1OS+HhcTQUluEvmZOsLmv34L4ktK8fmIJUHC+UKolGLneOFsKvUsxbikcKH5ls/4fq6fbpiXSl2SEeQ6fcevbdez/0w7MKMYz68x9rNOf69bY88gH1fNUozn1xgvNv7M9bb+Yd6bnUq9yoP51iLiu653j8xVjG9cvdB8K9C1ehqvH/eQSq3wXL4VjFLsmndw0sSrdbVzFWNve2OwCi9eFOMwL/4kQSjFzuuiUzFurp6jGGtUnjMFg1GKAWQCTNqmebV9rmKsVXvxLQlCKXaNt9htnBruYqMuC7kgk9KnVxgkUuwBrjfUr04uJMXeSK7dIXMS2+bjGG3TXJmUw7r4dK/jHR4+FVvGhnmztR6ZIPCFwgq06rM1Dd5dqedeP9jTydG+btRyBV8oq1hoeBAgme0YHeWNOmd98fV5+RTrdD7Hu+CNLHeNjPFmTT0OUWRjRhqXZTk38qW6UhvNFl6rrGNsyky0Kow7KkrRzKpdDrrW2MN6pm12PjjdRMfgKAAbctPYmJuOsAgDLl+PzY5ltzs41tRNZauzTi1KHca2VXmk6zQ+1+pz7kWS0CmLlVMN3dS1G9xtm/LSdGwoziAmctZr7fU5hyjd2gtmxxFFkcERI3UtBlq7htwtnORyGbnp8RTmJJIUH+25bj+A+AEhWHLtYfjUpIlvPLw8pDgUG+WltElKuHjh+pv4v3/YvYAUi15zMj1fllypF67T2/hLwZUaFn8wIblS+17/he5KPRtTRhPf3XDbij0AX454KxlBk+K4uDiPXygFQUCtVpOfn89DDz3EF7/4xSUvzm638y//8i8888wz6PV6UlNTeeihh/inf/onr19q9+7dy/bt2xdc7+vrIzk5OaB5F0uKAWpG9HzY14RKpuChgg2ECZ5PyDyRYlEUeaOlnraJETKjNNyZW+Z+noGSYrvDwfN1VQxOTZKr0XJrXtHc1yoIhfdwVxdHu3tQyGTcW1ZOfESEz/HgW0GuNfSzp8mZ2h2qVk0ARss0r5+qZXTKTJQqjDvWlBIboZ5Zj5f4XuBtPQ67yOGmTio7nClxeYlatpfloQrSgMvXY57m7hse5+PTLYxPOdOEyjKT2FyYQZhScc6V2XGTmeN1nbT0DAEgEwSKsxJZU5BGVLhqRZDi2bBYbDR3DFDfamB04qwxlyZaTWF2InmZCUSGe3PYDmCd3rDCSXHxt5ZeZ2S3mKn/9aVRY3Sp4lzu9XB+9vvlIMXggRhLrtQLxl/srtSweFIMkiu1v3kvdFdq9zwSKV5RCLim2IXHH38cmUzGzTffzL/+67/yr//6r9x8883IZDK+8Y1vUFhYyNe//nWeeuqpJS/uJz/5Cf/1X//Fb37zG+rq6vjJT37CT3/6U37961/7vbehoYG+vj73T2Ji4pLXEwhKYpOIV0Vgcdg4MtAZ1L2CILAtLQe5INBpHKNhdDDo+eUyGdfnOOtqWseGqRkKrr55Njalp5MRo8HmcLCrqRHLEutPSpMS2ZjhrLn6uLmVpsGhJcVzIUoVxmcqSokNV2O0TPPqqRoGjSb/NwYBmSCwpTCLbaW5yASBlv5hXj5yhiFj6GpOPCFFG8Nnr1hNaYbz/VvTaeDFT6vOea0xQEykmms2FPKZq1aRlqDBIYrUtht4cc8pDlS1YpwKbV33UqEKU1BWkMKd163hlm3lFGYnoJDLGJswc+xMJy++fYLdn9bS1DHgbkclQYIEJ87lXg8X5n7vDe4a4+JUAA6fCn2NccUy1xiXZSdzRXk2AFWtoa8xLkzVcc0qZ41xfXfoa4xzErTcsNw1xuXLWWMcPafG+K2Q1xhHcmdxKeEXco1x3qplrTH+TNby1RhLWJkIWim+6667uPbaa/na17425/rvf/973n//fV555RV+/etf84c//IEzZ84saXG33HILSUlJ/PGPf5wzf3h4OM8884zHe1wnxyMjI8TGxi5q3qUoxQCdxhFe66xGAO7LWU+8amH9pSel2PmAwBFDFwf1nYQrlDxUtBa1QhmwUuzC8b4e9vd0oJDJ+HzJGuJcNRFBKMXgTJ19ofoME9PTZGliua2oyOmsvQilGJwn6B83t1Fj6EcmCNxSUkh2XJyH5+Ulvg91cXJ6mjdO1zNkmiRMLufmVUXe+yN7QSDKtX50gveqmjBZplHIZGwryaUoJbD0cl+P+VNOuwfH+KSmjfFJM+B0qL6yNJuo+a7Yy13DO3O9Z2CMk41d6IecBmQyQaAoM5GK/DSiI1QLb5g/7zIrxZ7iTFvttHUP0tQ+gGHorHGaQi4jK1VLXqaOtCQNslkn/hebUiylT0sIBOdyr4fzs98vl1Lsvia5UvuNf7G6UsPSlGIXJFdq3/Ne6K7UklK8shC0Uvzee++xc+fOBdd37NjBe++9B8BNN91Ea+viHZBd2LJlC3v27KGxsRFwtoLYv38/N954o997KyoqSElJ4dprr+XAgQM+x1osFsbHx+f8zIYoikGdYGZGxZEXHY8IfGpoCfr0c0NCGvHqCKZsVvb2tgV1rwvrk1JJj47B5nCwu23xJ5gRSiW3FM6cho6NcrArOPV7PgRBYFt+DgW6eByiyNv1jfSMjfu/MZC1hjkV41RNNNN2O2+erqN5MPSKanJsNJ+9bBXpWufr+2FNM/vq20J6SuwJ6ToNn7tiNWvzUt0O1S98epqq9j53re+5RFqChlu3lnPzllJSdDE4RJG6DgMvfHSKfZXNjBqn/Ac5xwhTyinKSeKW7eV89oYK1pamExOlxmZ30NI1yPsH6nn+7RMcPNV60bpXR4QpQ/Ij4eLGudzr4dzs9/72+vGJKdraB5b2RGZBcqX2D8mV2jckV2rfuNBdqSWsLARNirVaLW+99daC62+99RZarRYAk8lEdHT0khf3gx/8gHvvvZfi4mKUSiVr167lkUce4fOf/7zXe1JSUvjd737HK6+8wiuvvEJGRgbbtm3j5MmTXu958skn0Wg07p+MjLNujv3mcd7qqaRxQh/U2q9McqZBd02O0moMLk1YLpNxbXo+AHUjA7SNB0/sBEHg+uwCVHIFhkkTB3sXT2YTIyPZOeNAfaKvj9qBpX1pkAkC1xbmkR0Xi90h8mZdA4YJ45JiuqBWKrhtdQm5ujjsosi7tY2c6TWEJPZsRIQpuWVdCRtynF92qrsNvHq8xl37u1xQyGVsLszk7q2rSIqNwmp3cKC+g1cPVWMYDc1rGCxSdRpu2VLGrVvKSNNpEEWRhq4B/vpxJR8cb2BwNLSp7KFCTFQ460ozuPv6Cm67ppzS/GTUKgVmi426FgNv763hxXdOcqSqnYER40VJkCVI8IZzudfDudnvfe31U1PT7H6/mn2fNtLcsviyo/k4J8S4aLmJcfKyEuPCtOUlxjmJ2gubGMdELzMxjpSIsQ9ESMT4kkHQ6dNPPfUUX//617npppvYtGkTAMeOHeOdd97hd7/7HQ8//DC/+MUvOHr0KC+++OKSFvfCCy/wve99j5/97GeUlZVRWVnJI488wi9/+UsefPDBgONcffXVZGZm8pe//MXj4xaLBYvl7B/Q+Pg4GRkZ/OfJ16m3DnFosJUohYrPZ28GPBsr2T3kyRwwtHNsoJsYpZq/yVuPYlZKptdWTbPSsPf1tHFyoJcoZRhfKFyLSu7BHdBHGjZA88gQu1oaALi9oIScGA+pyrPGL7x+9j8PdXVxrKcHmSBwV0kpqZ6+DAXQ8skFm93BWzX1dI+No1YouHNVKfGREV7Hz1+Pr/gOUWRfYxs1vc4vNxuz0tiUne428gg2ddfX+I6BET6sbsFitaFSyNlelkduotbLDaGbW3SI1Hb2c6Shk+mZmtiSjEQ2F2ag9qDknStjLsPwBJVNPXTqR9wPpSVoqChIIyU+xv/vwNvEwf5uggwvAA6Hgx7DGK1dQ3T0DM+pNY6OVJGbriM7XUt8bGRQDtbBrnNq0sQ3v3jdik2pupTSqS5VnMu9Hs7Nfu9tr/+/v9+NWh3B4WOtNDQ5D8Cv2FJAft7C2mTJlbodkFypPY2XXKm9XJdcqb1enzKa+IeNt67YvX454q1kLKol04EDB/jNb35DQ4OTbBUVFfGtb32LLVu2hHRxGRkZ/OAHP+Ab3/iG+9oTTzzBM888Q319fcBxvve977F//34OHToU0HjXG+A/T76OMlLNs+1HMNmm2ZqQR7km0+M9nkjxtN3O000nMNmmuTwhi40JZ+8NhBRbHXb+Ul/J2LSZcm0SO2fU47k3+K81/rijldMDesIVCj5fsoYoDx8SgZBiURR5p6mJluFhwhUK7ikrRzO/L3AQpBicNZ6vV9diMJqIDFNy56oyYsPVIXGlFkWRo23dHOtw1nOVpiSyrSAHmSz4Vkr+xo9PmfmgqhnDmFOtLc9IYktBFgr5wvdFqOeetExzuL6Txh6nMZtKqWBzYQbFGYnO+u9FzrvUGuThcROnm3pp6Rl0l9kmxkVRnptCTooWueDtm8H5I8WzYbM76NaP0No5RGffiLu9EzgJcnaalqzUeBLjowIiyBIplnCh4Vzt9XB+9nt3TfHvdxMeHokoin6JseRKLblS+4ovuVJ7gORK7fW6RIpXFoJOnwbYunUrzz//PCdPnuTkyZM8//zzy7JJTk5OzjG8AZDL5TiCTEuprKwkJWXhiWwgUMjkbIzPBuD4UAcWuzXge8PkcrYm5QBwbLCL8WlzUHMrZXKuzXAS4ephA+0TI37u8IwrM7JJCI9kymbjndZG7OLi0noEQeC6vDwSIp2x3mioX3KKTZhCzm1lJcRHhGOatvLqmRqGJ0NTiyoIAptzMthW4Pwd1Pb189aZeszW0KUFuRATruaOjaVUZDvfZ9VdBl46cobBieVPHY5QhXHNmnxuv6wUbXQEFquNT2raeOXAGboHx5Z9fm/QxkSyfX0Bn7tmLSXZSchlAv0jRj460cQLH56isqkH83Tgf0/nGgq5jOy0eK65rJDP37qBbZsLyE7TIpfLmDBZONPYx6691Ty/6wT7T7TQ2TuCLYQpbRIknG+cq70eVsZ+LwgCl23MpajA2c5p/8Gm5UmlllypvUJypfYNyZXaNyRXaglLwaJIsd1u55VXXuGJJ57giSee4LXXXsO+DF8Gb731Vv7t3/6Nt99+m/b2dl577TV++ctf8pnPfMY95tFHH+WBBx5w//+vfvUr3njjDZqbm6muruaRRx7ho48+mnP6HCyKY5LRhjnbLJ0Ybg/q3qKYBFIjYrCJDj4xBG9IkhGtoULn3OA/6GrCbAueRChkMm7OK3R+iJom+LS7I+gYLijlcm4tKiIqLIwRs5m3Gp0nokuBWqngjvLZxLiWIVPoPmTK05K4qbwQpUxG18gYL52sDhnxng25TMaWwixuWVdMRJiSEdMULx+p5lR77zmpR03RxnD31lVsLckiTCFnaGKSXcfqeOd4PcMT5+9DOyZSzRWrc7l35zrWFaYTHqbEZJ7maH0nz35wkk9OtzA8vrI3FaVCTl6Gjh2XF/H5WzdwzWWF5GXoUCrkTFmsNLT188HBep558zgfHKinoc3ApDl0G70ECecD52qvh5Wz37uJceEyE+NlrDGuKExjY9kyE+NVzsPmZakxTtVxzZplrDFO0HLDmsILnBiXLG+NcXHZhV1jvIAYBydK+YJEjC9eBJ0+3dzczE033URPTw9FRUWAs0dgRkYGb7/9Nnl5eSFb3MTEBI899hivvfYa/f39pKamct999/H4448TFuZMt3jooYdob29n7969APz0pz/lD3/4Az09PURERLB69Woef/xxtm/fHvC8s9Onw6OdbRo6TcPs6qlChsDnMjehCZubLuEpfRqcadJDZhPPt57CgcgtGaXkRscHlD7tgtVh59mG04xYpijU6Lgpq2jWDYG3amoZHeatZmca2o05hRRpdT7HO697vjxkmuSl2hqm7XYKtfHckJ/vTHEKMn16dirrlNXK69V1DJomUSsU3FFWQuK8lliLadXkwqDRxNtnGpiwTBMml3NDaQFZ2liv44ONPxtTFit7a1tpG3Cq+ymx0VxTlocmwkdqeIjmRoSpaSsnmrup7ezHIYoIOFPRNuanE6kOWzA+qPV4GR5oHLvdQUvPINWteobGzyrpKdpoSrOTyU6J81iHda7Tp/2NB7A7HPT1j9PZN0xn7wimqbkbuy4ukozkODJSYkmIDSzNGqT0aQnnH+dyr4fzs9/PT5+eDVEQOXy0lYbGuanUoWjVBM6PlRNnOhe0awpVqyZRgNMNHlKpQ9SqSRSgtl3P/jPtwKwa4/k1un7W6et1a+wZ5OOqeTXGoYovQFv/MO/NT6WWhyY+eG/XFIpWTeCjxngJ6dOzMTBl4tW62oXtmpaQPj0bXlOpl5A+PRsTVjMvN1cvSKUORasmALN9mlfbF6ZSr5T06b7+pbdfdMVLSbw0WjAGTYpvuukmRFHk2WefdTtQDg0N8Td/8zfIZDLefvvtZVnouYQnUgywq7uKzslhsiN1XJ+yas49vkgxwAFDGyeGuolShPH5vPUoBc8tTTyRYoA+4wQvNlchAjdmFlIUmzBzQ+CkGOBAdwfH9D0oZTLuLV5NfHiEz/G+SGjX2BivN9TjEEUqkpO5KjMLwcvrECixM9tsvFFdR7/RhEou5/ayEpKjowJaTyDxJ6etvFvTSN/YBAKwNS+LirTks2QlhMRUFEXqegc40NCO1e5AIZOxpTCT8rQkz+QohHO7MGqa4khDF20Gp4O5Qi6jIieViuwUlAp5wHHmrMfL8GDj4BDRD09Q3dZHu37YXXccrlJSkpVISWYSkeGzCPwKJMVzhokiw2OTdPaN0Nk7zODI3NR5tUpBelIsGclxpCVpPJqhuSCRYgnnG5fSXu+JFCNz/k3PJ8Z5+UmegwVJihGEszXGs4hxSaGX1O9FkGLwQIxLMjyPXyRRW0CM59cY+1mnv9dtATGeX2O82Pgz1xcQ41UezLcWEd/1WPfIQmIsU3h5sRdRz6wf90CMFZ4NYYMlxaIwU2M8nxgrPZhv+YjjlczihRiHeTDf8hXH6+sjOmuM5xFjjcqDrw7Bk2KZAJO2hcRYqw4POP5ykuKyr/4IuUrt/wY/sFvM1Pz+Hy+J/T5oUhwZGcnhw4dZtWouKTx9+jRbt27FaDw/bWFCCdcb6tenXptDioctJl5oP4GIyG1pFaRGxLkf86b8usiyzWHnmZaTjE2bWaNN5aokD6ZZgOhNQRYFDvV1ctjQhUou54GitUSFqbyO90YeHQ54rbGWrokx4tTh3Fuyyulq7S2zxyvpdv5TPzjI+83NAGzJyGBjapqX8YEryNM2G2/WNNA3PoFSLue2smJSY6K9jp+9nkDi2x0O9ja0Uad3tpYqSU5gW2EOcpksdGZUs66PT5n5qLqV3hFnT8w0bQzbS3OJCVd7vSdUc7vQNzzOofpO+mfaNoWHKVmfn0ZJZqJXw6tzacxlmrJQ39FPXbuBKYuzREAQICtJS3F2ImkJsci9He96XY+X09oA1hMQ/DzfyalpuvWjdOlH6NGPzXGyFgCdNor0pFjSkmJJ0EbN+SI2NWniWw9JpFjC+cOltNf/1+88KcUz/wZgvuUcvzgFWXKl9h//gnSlnnWP5Ert5brkSs2U0cT3Nt0ikeIVgqBJsVarZdeuXQvMNg4cOMCtt97K8HDwPXVXGryRYoCP9U3UjvWiU0VxV8YG9wezP1IM0Gkc4bWOagA+m11BcvjCN5cvUmwXHbzYeAbDlJGMKA135pV5VWZ9kdxJq5Xnak9jtE6TGxvHrXnFCP6Om33EP9XXx6cdzjrlHTm5lCcu/MIQtCu1zc6u2gZ6xsZRymTcXFpERqwmJKQYQHSInO7Wc6ClAxFIionixtICor2cIC6VmIqiSHWXgUONndgcDpRyZ/1xaVriottEBd3CSRRp1Q9zpKGT8Umn8UR0uIoN+ekUpOoWfAk4127V4DywaO8dprZdj35own09KjyMosxEijISiYqY9ztaoaR4NhwOB4bBCbr6nCR5dHxuTXuYUk5qooYrN+QRplRctKT4t7/9LT/72c/Q6/WsWbOGX//61+52P77wwgsvcN9993H77bfz+uuvA2C1Wvmnf/on3nnnHVpbW9FoNOzcuZMf//jHpKamuu8dHh7mW9/6Fm+99RYymYy77rqL//iP/yAqKsrLbBLg0trrfZFiCIwYS67Ukiu1rzkkV2oPkFypl5UUS+nTwSNoo61bbrmFr3zlKxw5cgRRFJ2bxeHDfO1rX+O2225bjjWuKGzU5hAmkzNoMVI/Hpw5RmZUHCUa50b6YW/wBlVyQcaNWYUoZDK6jGMcNSzOPCNCqeSW/CLkgkDr6Aj7ezoWFceFtSkprJ/5ErqnrZWGwcElxQOnc/etpUVkxmqwOhy8WVNPQ//S47ogCAIVGSncsqoYlUKOYdzICyfO0DE8GrI55s+3KjOZe7asJjk2Gqvdwb66Nl4/Xsuw8dwYNAiCQF5KPPdctYYry7KJUCmZmLLw8ZkWXvz0NI09AzjOgSGYL8hlMvLSdNy6tZy7tq2hPDcZlVKOcWqaEw3dPPfhSd45VEtz9yA224Xj9CyTyUhJ0LBpdRZ3XVfBvTet48r1eeSkxxOmlDNttdM/NOE17S2UmLJYQ/ITLF588UW++93v8s///M+cPHmSNWvWcP3119Pf79vEqL29nX/4h3/gyiuvnHN9cnKSkydP8thjj3Hy5EleffVVGhoaFuxDn//856mpqeGDDz5g165dfPLJJ3zlK18Jev2XGi71vX42JFdq/5BcqX1DcqX2DcmVOvSIUClD9nOpIGileHR0lAcffJC33noLpdL5QtlsNm677Tb+9Kc/odFolmWh5xK+lGK7Q8bpkS4ODTajliu5L2szKrkyIKUYYMpm5ZmWE0zarKyLT2drYu6cx30pxS7UDPfzfmcTAnBnbjkZUbELbwggHbp+aIDdbU0A7MzMo1znoVYqAKXYuT6Rj9vaqO7vRwBuLigkb6YOzTk+OKXYpcLZHQ7eb2ymedCpSmzNymRdWsrC1Kkl1BqPTZnZXdPIwAw53ZiZxqbs9Dk9fkOl1gI4HCJnOvUcaenCZncgEwTWZKWwMSfNc8pTCOeeDavdTnW7gdOtve42VZoINWvzUilI1aHw1kf4HNcg2+x22nqHaezqp3dw3H1dqZCTm6qlIC2BlPiYhe+JFaQU+xrvEEUGh41Mmq1kpzn/ZpZTKS7/2x8hD1taSpV92kz1U/9IV1fXnPWpVCpUXrItNm/ezMaNG/nNb34DONXzjIwMvvWtb/GDH/zA8zx2O1dddRVf+tKX+PTTTxkdHXUrxZ5w7NgxNm3aREdHB5mZmdTV1VFaWsqxY8fYsGEDALt37+amm26iu7t7jqIsYS4upb3en1LsviaKHD6+0HzLOX7xSvHs+CequxaYb3lbj8/4Xq5XNvVwrMaDYhwiA66aDgP7z7QB81Kpl6jkutCoH+Sj0x5SqUMUv21whN2nGxcqxktUil3wpRiHwoBLP2Hkjeo6z6nUS1CKXRicnOSV+pqF5ls+4gej8PpUjJegFLuwUDEuJ2ZmPwyFAZcvxfhcK8VSn+LgEbRSHBsbyxtvvEFDQwMvv/wyL7/8Mg0NDbz22msXxSYZCMpj04gLi8Rst3JkKLg2S+EKJdekFABwaqgb/dS4nzsWokybSJk2ERF4t7Nh0adpxfEJbE5xbogfdbbSNTG2qDjgPOnenpNDsU7nXFdzE+2jo4uO54JcJuOGogIqUp0n9Ac6OvmkrSOkiqYmXM1da8spT3UeChzr7OH103WYLMvTUsdFgu+7fA05CXE4RJFT7b08f6iK9sGRZZnTE5RyOWvzUrl/21o2FWagUioYmzSz90wrz39SSU2nAZs9dCfZi4VCLqcgI4FbtpRx7461rCtMJzpChdVmp6FzgF2Hanl+zymO1XcyOhH6VlvLDZkgkBgf7SbEFxIyMjLQaDTunyeffNLjuOnpaU6cOMHOnTvd12QyGTt37uTQoUNe4/+f//N/SExM5OGHHw5oPWNjYwiCQGxsLACHDh0iNjbWTYgBdu7ciUwm48iRIwHFvFQh7fULIQgCl226sNs1rVlmxbg0O4krVmUDy9OuqSBVx/bVy9euKTshjuuXWzFetZyKcdQcxTj07ZoiuLOk9MJu15RfPksxrg65Ynxn9spSjCUEjqCV4ksB/pRigN6pUd7sPgXAZzLWkRAW6zGWN1fq3V0NNIz3ExcWzr0561G4TgoDUIrB2abp+cYqhsyTZERp+ExO+VxlM0DjLFEU2d3WRMPwICq5nHuKVqFVR3gd7y++6BB5t7mJ5uFh5ILA7UXFZGg0S2rV5MKpnj72tzlTvfPitVxfmO9+3ZbqSu1Ck2GQjxpasTocRCiVXFuST2acJrRq7bzH2vqH+bShHeNMX9u8RC1XFGYRpVb5nCPUbsxWm52aTgOn2/qYmnamx0aqlKzJSaU0PfG8u1XPjiOKTufqpq4BWnqH5phYxcdEkp+uIy9FS1S4BzONYNcT5PhQxF9OpVgfgjqj8fFxkhN1ASvFvb29pKWlcfDgQS6//HL39e9///vs27fPI0Hdv38/9957L5WVleh0Oh566CGfSrHZbGbr1q0UFxfz7LPPAvCjH/2Ip59+moaGhjljExMT+dd//Ve+/vWvL+bpS7hIEKxSDEiu1AHOe8m7UvuYQ3KlnoHkSi0pxSsMXt7Bc/Hd73434IC//OUvF72YlQa5ICKf/6aXOdlXRmQMxTFJ1I8b+LS/kTsz1iPzkG4qejlh3JaSS9fkCCPTUxwdbOeKpBzAe2ek+QiTy7glp5DnGqqc9cX9nVyenHl2Xi+9/Ji3HkGAa3PyGLdY6DNN8EZLPfcWryJc4aoh8PJh4CW+gMD1+fnYGhtpHx3lzcYGbi0qIjPGs7Lg7RDA09W16SlEhin5oLGFlqFhXqup45aSItRKBV4zfYMkjgVJOnRRkbxb28iwaYo3qupYl5HK5ux0j26R3n5dvjbK+XPnJGtJ12k41tLN6Y4+WvqH6RgcZX1uGhXZKZ579vp4Dl6PufzsG0qlnIq8VMqzk6nr6qeytReTeZqD9R2cbOmhPDuZ8qwk1ErP9SVBr8frixfAdUEgWRdDsi6Gy1dn06EfoalrgO7+MYbGTQzVmjhS20FyfDR5aTpyU+JR+6uL8bo/e3tivsMtZbzXL6IhQLhKSfgSa4SsM/fHxMQsyyY5MTHBF77wBZ566il0Op3f8Varlc997nOIosh//dd/hXw9lwou1b1eEEWEBR9UXg6oHc597rINuSBCQ5Oe/QedZUgeXam9xBG8HSzLnHesL88AUaSqvpfDJ53pyB5dqT2H8dkpCJw1xgDHars4UedUiz25UgveDur98LeyrGQQYX91O1UtTrXbkyt1sNuA63UrStEhiPDRmWbqupxqvSdX6qDjz/ybO1NjvLuqiWbDEIBHV2pfr7Ovj/z0WGeN8a7qBrdiHKwrta/n5qoxfqO63q0Ye3OlFry91708AwFICI/kruJSXq2vdSvGXl2pvcHbF16Z6FaMX26scSvGXl2pfcTxDIFopZq781a5U6lfbj7j1ZXa2yvt7fV3IKKWO2uMXanUr7Sf8Wi+NV/wknB+EdC799SpUwEF89h79SLGloRc2oxDDFqMVI/2sDrO80mrJ6jlSnakFPBWVy0nh7rJjY4nNSK4L5fx6gh2pOexu7OJw4YuUiNjyIqODfJZgEIm49a8Yl6or2LMYuatlnruLCg7q8IGCblMxk2Fhbzd2EjH6ChvNTRwW2GRUzFeIgoTdEQolbxd10jf+AQvna7mlrIir33hFoO4yHA+u66c/c0d1PT1c7Krl+7RMa4rzic2InTzzIZSIWdLYRaFKTo+qWtHPzrBkeYu6nsH2FqYRZYu9pz9fSnkMlZlJ1OakUhDzwCVrb2MT1o43tRNZWsvxemJrM5OJiZi6Vb/oYBCLicvTUdemg6zxUpr7xAtPYPohybcPwfPtJGm05CTGk92snbJpFBCcNDpdMjlcgwGw5zrBoOB5OTkBeNbWlpob2/n1ltvdV9zzKQYKhQKGhoayMvLA84S4o6ODj766KM5JD05OXmBkZfNZmN4eNjjvJc6pL0+cLjMt8A/MV5s/PWrnAfdbmIsssCVeikIlBgvFmXZM2nmLmIs4lUxXgwKU50HZi5iLIosdKVeAnICJMaLhct8KxBivBh4IsYeXakXCZf51pKIsQ8ERYwXAZf5VmDEOHi4zLf8EWMJKwdS+rQHuFIF/m/lqwvTp+ed6tSM9rLX0IRSkPO5rE1EK+cSBVe69Xy4jLne62mgfqyfGKWK+3PXoRQ8f1n3dpokOgQ+6GqmethAuFzB/YVriAlTez998pZW7RAYmprkxfozTDvs5MVquTm3CFmweTuz4tscDjcxlgsCNxcWkhMbt2BeT/CX9jxkmuSt2nomLNOEyeXcUFRAdlys1/EBx5/3+rQMDPNRQwsWmx2FTMYVeVmUpQTQRskH/D03URRp7BvkUGMnkzNpzOnaGLYUZqGb9X5c7hZOLjgcIq36ISpbexkcd9bGCEB2kpbV2ckkx0UjCELw6/GGEKWFGycttPQM0tIzxNCY6WwcIDk+hpwULdmzU6xDpBSHKn362w9cXC2ZNm/ezKZNm/j1r38NOEluZmYm3/zmNxcYbZnNZppn+p+78E//9E9MTEzwH//xHxQWFhIWFuYmxE1NTXz88cckJCTMucdltHX8+HHWr18PwPvvv88NN9wgGW1JcL+Pf/df73pIn/ay5y7w8/PdrmmpBlz++hiHwoDLZ7umEBhw+exjHAKDLJ99jEMQ32e7Jh/8ONA5/LVrWqoBl98+xks04PLbrmmJBlx+2zUt0YDLX7umpRpw+TLfmjKa+P7mm1fsXr8c8VYyljFJ79JAqSaFlPAYrKKdT/obgjZ82JacR4xSzbjVwkd9TYsyjNielkNieCRTdhtvtjk/+BaD+PAIbs0vRi4ItIwO82FHy5IMLBQymZMIx8VhF0V2NTbSNDy06Hhz1hoZwWfXlJMSHcW03c5btfWc6O4NqeEGQF6Clns3rCY9Ngabw8HepjZ2VTdgXCYTLnAqBEWpCdx/xRrWZqciEwS6h8d56fAZPq5pcdcenyvIZAL5qTru2rqKmzcWk6HTIAJthmHeOFLLywfOUNfVvyJMuWYjKlzFmvw07rx6NZ+7poKNJZnoNJGIQN/QOAer23nug5O8uq+Kkw3dDI2bQv7+kXAW3/3ud3nqqad4+umnqaur4+tf/zomk4kvfvGLADzwwAM8+uijAKjVasrLy+f8xMbGEh0dTXl5uZsQ33333Rw/fpxnn30Wu92OXq9Hr9czPe38GykpKeGGG27gb//2bzl69CgHDhzgm9/8Jvfee69EiCWEBO52TRew+VZFYRoby5a5XdMqZ4nYcphvFabquGbN8plv5SRouWFN4QXerqlkGc23nIrxBW2+taBdkzlk8VdiuyYJniGR4iVCEAS2JxchF2R0TQ7TOGHwf9MsqOQKbkxzKrKN44PUjumDXoNCJufW7BIiFEoGzCbe61ocuQbIiNZwY24hAlA71M+nPe1LJsY3FRRQqI3HIYq829RE3cDAouPNRmRYGJ9ZVUppUgIiTmfq95taQk7OotUqbl9dwhV5WcgFgY7hUZ4/fpo6/cCykqgwhYLLCzO5f+sa8pK0iEBd7wDPHajkcFMnlhBuOoFAEAQyEmK5eWMJn7tiNSUZiShkMoYmJtlX3cqf957kcEMnE1Ohc3IMFTRR4VQUpPGZq1dz3851XFaWRZI2GoDBMRPHG7p4ZV8VL+w5xcHqNnoGx0L6pUQC3HPPPfz85z/n8ccfp6KigsrKSnbv3k1SktOgqLOzk76+wL/s9/T08Oabb9Ld3U1FRQUpKSnun4MHD7rHPfvssxQXF7Njxw5uuukmrrjiCv7whz+E/PlJuHQhuVL7h+RK7RsXvit1pORK7QPzXamPDnSFLPZKwyeffMKtt95KamoqgiAsMMc0GAw89NBDpKamEhERwQ033EBTU5PfuC+99BLFxcWo1WpWrVrFO++8M+dxURR5/PHHSUlJITw8nJ07dwYUdzak9GkPCCZ92oXjg10cGWpFJVNwT9ZmIhTO1A5/6dOz7z/Q345CkHFP9jq0ARbjzzaq6jWN81JLNQ5R5LKkTC5Lylx4g4/06dmoGezngw5n+uKW1Ew2Jc+rM1qEK/WetlZqZwjxtuxs1iQlh8SVWhRFzvQZ+KS1HRFIjIzk5pJColWqJadPz5932DTJhw0t9E8403GztLFcU5BDlBfXwqDm9nO9b3SCQ02d6EcnAFApFWzITmNVRtLcGqRz5FYNYJ62Ud/dT3WnHuOUc4MSgOzEOEozksjQabyaeAQSP1TrnBNn1n9PWax0GEbo0A/T3T+KfdaXHKVCTkZCLJlJsWQkxi6sQ5bSpy+JdCoJFy9CkT7thuRKHdC8l4or9WLmkFypZ3CRu1IfGejkyqQcFDL5RZk+/e6773LgwAHWr1/PnXfeyWuvvcYdd9wBOD8jt2zZglKp5Be/+AUxMTH88pe/ZPfu3dTW1hIZGekx5sGDB7nqqqt48sknueWWW3juuef4yU9+wsmTJykvLwfgJz/5CU8++SRPP/00OTk5PPbYY5w5c4ba2lrU6sA8cCRS7AHujfL0KwtIsc0LyZ12iLzaeZJBi5G8qASuSy0DfJBiD62RXmuvodM0ik4VyedyKuYYXc0n0Wfvm0dmhwy839kCwM1ZRRTEznNu9ebS5+FdcFLfxydd7QBsz8xlTUKyz/HO697JsiiKfNLRwWm98wvDprQ0LktL92yK4e2zxoc9d/foGO/WNWG22VArFFxfXECWN3OvJRBHh0PkVFcvR9q6cYgiYXI5V+RnUZKcEFit8RLmFkWR9oERDjd1MWJy9uSNVqvYmJ9OYYrO2ZYrWFK5SLfq2XCIIh2GEarb9fQMne29HROhoiQjieK0hAWkMlQk1xuCfb5Wm52egTE6+obp6h9lymKd83hiXBQZiXFkJMWi00TObYEW0DoDr02emjTxv74gkWIJEpYT7r3+t+8sIMXeOiz4I8v+aoz9xfFH+vzVGM9fT8DxZ133WWPsjrP4WmOfNcZBrNPbeJ81xiGI77PGOARz+KsxdsdZ5MGE3xpjv3F8fznwW2PsJ74/suy3xjjAON7G+6sxdg9fYq0xgNlo4n9ftjykOBTtF13xkhN1i1qjIAhzSHFjYyNFRUVUV1dTVubkSQ6Hg+TkZH70ox/x5S9/2WOce+65B5PJxK5du9zXLrvsMioqKvjd736HKIqkpqby93//9/zDP/wDAGNjYyQlJfGnP/2Je++9N6D1hsYiTgJyQca2pCJe6TxBi3GA1okBcqMT/N84A0EQuD69kGeaTzFoMbHf0Mq2lPyg11EWn8Tg1CQnB/p4r6sJTZiaxIiooOMArEtOwWy1cVTfzcedrajkcoq1gT+n+RAEgauyslArFBzp7uZoTw9TVhvbsrO9E4wgkB6r4Z615bxT28iAaZI3q+u4LDODDempIXVLlckE1melkaOLY09dC4YJEx81tNLcP8S2whxiwpfPlVkQBHIStWTp4qjvHeBYSxcTZgsfVbdwqq2XjXnp5CVqz7k7rEwQyEnSkpOkZXhiktqufhq7BxiftHCkoZNjjV3kJmspyUgkVRuzIt1rlQo52TPmW6IoMjBqpFM/QqdhlKExE/0jRvpHjJxo6EIdpiA9MZb0hFjSEzREqEPjhilBgoQLG5IrtX9IrtS+IblS+8aF7kp9rnD99/6APGzp30ftM/XV4+Pjc66rVCpUQWRJAlgszpT02cqtTCZDpVKxf/9+r6T40KFDC1oGXn/99e7U7La2NvR6PTt37nQ/rtFo2Lx5M4cOHQqYFEs1xSFEgjqatVrnZrWvv5FJW3D1CJHKMK5NLQSgaqSPpvHF1d5emZZNVnQsNoeDN9vrmFhCXcTlqRmsnlGI32tronFkcNGxwLmhb05P5+rsbADO9Bt4u6kxZLUtMWo1d68pd9cZH+rs4o3aereLcyihjYzgrrXlbMnNRC4IdI6M8dyxKo539Cx7PapMJlCansj9V1RwWUEGKoWcEdMU71c18eKhKpr0gzjOUxKINjqCK0qz+cI169i2KpcETSQOUaS5b4i3jtbx7L5THGvqYnwydEYWoYYgCCTGRbOhJJM7r17N/deu44rVuWQna1Eq5JinbTR3D7L3VDPPvH+Clz6u5GB1Gx36Yaat57bWW4IECSsLbvOtgmWuMS52msUdPrVM5lvLWGNclp3MFeXZAFS1LpP51ipnjXF99zKZby13jfGym2+drTF+a5nMt8Iv5BrjBeZbK88v5VwiIyMDjUbj/nnyySeDjlFcXExmZiaPPvooIyMjTE9P85Of/ITu7m6fniJ6vd7tP+JCUlIS+pnMU9e/vsYEAkkpDjE2aLPpNA0zaDHykb6BG1LWBHU6mR2tZX18OieGuvmwt4l4VeSC+mJ/kAkCN2YV8demKoYtU7zRVstn81ehkgf/6xYEge0ZOVgdduqGBni3tRExB4ridP5v9oE1yclEKJW839xM68gIr9TVcltRMRHKpfePVchl7CjMIzkmmk9a2ukcHeP5yiquL8onPQS9kmdDJhNYl5lKTnwce5va6Bkd53BbF/X6AbYV5JARF9r55kMpl7MuJ42y9CROd/RR1aln2DTFB2eaOd7aw/qcNPKT4kN2gh3U2hRyijMSKU5PpH/MSH1XP819QxinpjnR3MOJ5h5S4qIpTksgNzmeMC91TSsBkeEqSrKTKMlOwu5w0D88QXf/KN0DYwyOmRiZmGJkYorqVj2CAAmxUaTGa0hL0JAUF4UiRCfwEiRIuDDgVowFaGhcRsVYEKiq63EqxuAxlXqxqChMAwGO1SyjYiwI7D/T5lSMwXMq9SJRmKoDGXx02qkYA55TqRcJlyv17tONF7BiXMIb1XVuxdhrKvUi4CLGr1xUinE5MSFQX88F3vvZV0KXPv30P9LV1TUnXrAqMYBSqeTVV1/l4YcfRqvVIpfL2blzJzfeeOOK6P4hKcUhhlwmY0dyiduNumYs+NPVyxOzSYvQYHXYeburFos9+NM1tVzB7bmlRCiUDJon2dVev+hTRkEQuDYrn9J4p/q6u62R+uGlO0gXxMe73QoNJhMv1lQzPDW15LgulCUncs/qcrTh4ZisVl6rruNwR9eyqLhxkeHcsaaEa0vyiVAqGZ0y83pVHe/VNWFaxvZNLqiUCjblZ/CFK9eyMTedsBnl+MPqZp47WElNt+G8uiknaqK4qjyXB65Zz841+WTonIcFfSMTfFzdytMfneCD0010DIyseNdnuUxGik7DptIs7rx6NV+4fgM71hdQkpVETKQaUYT+ESOVzT28faiWP+0+xpsHqjle30XP4NiKa10lQYIEJyZNFrq7hkMWT3Kl9g/Jldo3JFdq37jQXamXE+EqZch+AGJiYub8LIYUA6xfv57KykpGR0fp6+tj9+7dDA0NkZub6/We5ORkDIa53X0MBgPJycnux13XvI0JBBIpXgZoVZFcrnP+co8MtTBsMQZ1v0wQuDG9mEhFGCPTU3zQ27ioD3FNmJrbc0pRymR0Gcd4fwmtmmQzxLhcl+gkxu1N1A4tfXNPjY7mc2VlaFQqxi0WXqyppmN0dMlxXYiPjOCeNeWUJjoJ/dHuHl4+U8uoOfSpu4IgUJSk4/Ob17A6LRkBaOwf4i/HTnOqq++ckD2VUsHGvHS+cMVaNuWlo1YqGJ+ysK+ujb8cOEVlRx/TttBteMFCIZeRn6rj5o0lfGHbWjYXZKCJUGNzOGjuG+KdEw38ee9J9te2Yxg1roiTQ38IVynJS9Nx5Zpc7t2xlvt2rOPqNXnkp+mIUCtxOET0wxOcbOrh7UN1MyS5hqN1nXT1j57zdOspizUkPxIkXEwwm6d5790qPt5TS2fH0sqEZuOiIMZFy02Mk5eVGBemLS8xzknUXtjEOCZaIsY+sJAYS6nUoYBGoyEhIYGmpiaOHz/O7bff7nXs5Zdfzp49e+Zc++CDD7j88ssByMnJITk5ec6Y8fFxjhw54h4TCCT3aQ9wObf9oeplhAgFR4YaSY/QkReVjN2L3d98V2pRFNnVU02naZj4sEjuzNgwJ+XFm5v0bFdq/eQEL7VXYRdFLkvIYpMuy8N4z89BnLWejvERXm+tw4HIWl0qV6VmL0wfCtCVWhRFPupo48yA8zTm2qw8ynRJXsefve67hdOk1co7jY30TkwgAFdnZbPG0+mOV1M//+tvHBjk46Y2pu12lHIZV+fmUJyom/NaBOtY7Gt8/4SJfQ2tGGbaN8VGqLkyP4ssbVzQsRa7JqvNTm1PP5XtfW7FWqVUsCojiVWZyYSHKUPnVu0F/uKIosjAmImm3kGae4eYmlX/HROhIj9FR35qPNooz2UEy+1ivZTWS6IoMm4y0zc0Tt/gOL2DY0ya5xJKAYjXRJIcH83G4kwUCjlTkyYe+fy1y+JIufbz/7Zk8w37tJlTz/5/kvu0hAsabvfp37yDShXBgf0NtLUOIAgCV28vJjPLQ5mQ5EoNSK7UnsZLrtSSKzUE50ptNpr435fftGI7TSwmntFopLnZ2dJ17dq1/PKXv2T79u1otVoyMzN56aWXSEhIIDMzkzNnzvC//tf/Yv369bzyyivuGA888ABpaWnuuuWDBw9y9dVX8+Mf/5ibb76ZF154gR/96EcLWjL9+Mc/ntOSqaqqKqiWTJJS7AcNE700G/UcHmxgyh74yZMgCFyTXIRarmRo2sTRodag506OiOaalDwADg900DYxFHQMgKyYOK7LdDpZnxrs5cRAz6LiwMzzyspxt2f6oKOFMwOBF7F7Q4RSyR0lJZTodIjA3o52PmprC+lpaGGCjvvXrSY1Jhqr3cGHTS3sbmjCvExKXWJ0JHevL+ea4lzClUpGJ828VdXAW1X17nZKyw2lQs6arBT+5soKtpXmoolQY7HaON7aw58/Ocm+2tZzthZvEASBxNgots6Yc920sZiCVB0KuYzxSQsnW3r466dVvPjpaU40dzNiPL/rDQaCIKCJCqc4K4nt6wu4/7r1fG5HBVdV5FGYkUB0hAoRGBwz0dw9iFwufSRLkHA+IJMJbL2iiJzcBERRZN/H9XS0h1gxPpfmWyfbqG28gM23Wvo4Urt85lt1Xf3sq5bMt2ZjvvnWmzMEOVSQzLcuDRw/fpy1a9eydu1aAL773e+ydu1aHn/8cQD6+vr4whe+QHFxMd/+9rf5whe+wPPPPz8nRmdn5xzjrS1btvDcc8/xhz/8gTVr1vDyyy/z+uuvuwkxwPe//32+9a1v8ZWvfIWNGzdiNBrZvXt3wIQYJKXYI2YrxWFRat7oOcrItJHsyESuTvBsnOWtf3HL+DC7+84AcFPqajIj44HAlGIXPuptpmpEj1Im53PZFcSrImeN9/wcRA/rOd7fw6e97QDsSM9jVfwsJTaI/sUAoh32dbdT2e98025JzWRjchpCsEeg8z7PRVHkZF8fBzo7AWd69U0FBWdP+5agFLunFEVOdPVytNPZZzgyTMn2vFxy4uNCqhTPhsVm41h7D1XdehyiiACUpyaxMTudiDBlULEWvSacz73VMMyp9l4Gxk3u65nxsazOSiZDq/Hfa3mZ+wW7YLXZ6egfoblviM6B0TlfLuKiwmdaQMWREB0ZnHHKOVSKA4Fpahr90DjTVjsl2c6si+VUikPRu3ApfQslSFgpmK0Uu/oUOxyib8V4kUqx+//9KMaLVYpnx/elGC9FKXbBp2K8BKXYBZ+K8RKUXBd8KsYhiO9XMV7iHP4U48UqxS74VYwXqRS74FcxXqRS7IJfxXiRSrEL/hTjS10pvpAhkWIPmE2Kw6MjGLSM82bPMURErkpYRW7UwpQkb6TY7pDxaX8jNWM9qOVK7s7YQJRSHRQptjscvNJRQ+/kGDFKNZ/LriBCETYz3vNz8ESKAT7paXcrxTdmFlIUN9N3OEhSjENAFEUO9HZyXO+MV5GQzNXpXpwdAyTFLrQNj/BeSzPTdjtRyjBuLiwgOSo6JKTYdd0wYeSDxmZGppz1xYUJ8VyVk+3ZATtEKcYjk1McaO6kfWgEAKVcRkV6CmszUlF5c3wMISl23yqK9I6MU9Whp21gxH09NlJNeXoyxak6727l54gUz4bFaqNNP0yrfpjuwbE57aai1GFOgpwYR0pcjH/nzxVGij2NX05SHIqYl9ImKeHihSdSDH6I8RJJMcwQ4+OtNDQuJMZLJcWu+Cequ6iqc+7Ns4lxKEgxQGVTD8dqPBDjEJBigJoOA/vPOB215xDjEJBWgEb9IB+d9kCMQxS/bWCE3VWNnolxCObwRYyXSooB9BNG3qiu80yMl0iKwUmMX5lRihcQ4yWSYvBDjJdIisE3MZZI8YULKVcvAOhUMayNywHg8FAdJltwJk2X6/LQqaIw2618oK/BLgaX7iKXybg5rZQYpZpxq5l3umuxLTJl5oqULLdC/F5nE81ji0vJBme61hVpWVyVng1A5YCed9oaF7222ciNi+PesnLi1GqM1mleqq2lUq8PaapTUnQU91asZl1aitMUa2CIZ06epnFgcNkMnuIiwrllVRF3rCkhISoSq93BsY4e/nzkFJXd58aMC5y/uzSthhvXFvH5rRWszkxGKZcxajKzv6Gdpz85yb76NoaMk+dkPf6gUiooTk/kpg3FPLhjPTvW5JObpEUhl2E0T3OmQ8+bx+r408cn2FPVTIt+iOkQpmVJkCDh0oFMJrD1yrmp1JL51lxIrtS+kSO5UvuEZL4lYSVCIsUBYk1sNjpVDNMOGwcGa4L68FTI5FyXUk6YTIHBPM7hwZag5w9XKLkto5wwmZzeqXH29DUs6gNcEASuSculOC4BByLvdDTQOr60FhTrklK5MacQmSDQNDrE682LayM1H3Hh4dxTVk6BVotDFNnX0c67LU0hrXFRyGVszcnis2vKiY+IwGyzsbuxmV31jRiXsZVSepyGz60v54bSAjThasxWG5+2dPCXo5XU6QfmKKHLDU2EmiuKsnnwqnVcWZxNXGQ4VruD6m4DLxyu4vUTtTQZhlZMqySVUkFBqo7r1hXy0I4N3LCukKK0BFRKBRarjcbeQd6vbOJ/9pxg17E6znToGZ8Mvdu4BAkSLl5IxNg/JFdq35BcqX1DIsYSVhqk9GkPcKUK/L8zLxERfdbxdnTaxCvdR7GLDrboiimOyXA/ZveWPj0rL6XNOMg7PTUA7EwuIz96ofOkt7Rql+t1h3GENztqcCCyUZfB5Qk5Hsd7iyPOrMchirzb0Ujj6CByQeCW7BJyYjy4IgeRVt05PsaupgamHXZ04RF8pqDUby2w13yheQ6+p/V69nd04hBF4tRqbiosRBcxy43Y2z7gJb6nLB+7w8Hxrl6Od/XgEEXC5HK25mRSlpSIzGu+kLd5vVz3MLfd4aBOP8DRtm4mZ5yXtRHhbMpOJy9B67VOO+j03QAzg0RRpHd4nDOdBtoGhnF9QoQrFRSnJVCaloQmYqFxQchcoBeZruxwiOhHJ+gwjNDeP8KYaS4RjosKJzMhlsyEWJLjoj26dnqc9zy4W09NmvjOfVL6tAQJywnX+/j3//n2nPRpAHEm1dVvjbELkis1ILlSe7pHcqWWXKnB85+A2WjiB1tuXLF7/XLEW8mQSLEHeCPFAKdHuzky1IBCkHFH+uXEKJ2PB0KKAQ4NtHJyuAulIOfOzPXEhc3diP2RYoCaET0f9jYBcE1yAeVxCzc4f6QYnMT4nfYGmsaGkAsCt+WUkBU9jxgHWWvcbzTxelMdkzYrMWEq7igoQauOWBIpdqFvYoJ3G5swTk+jkMnYnpNDaYKrJjq4+L7qXIdMk3zY1EK/0WlElRIdxfa8HHSRkR7HB7p+f3NbbXaqevSc7OzFMtNLOD4ygk3Z6eTGxy3YpJeLFM8ebzRbqO3up66nH9Os3rTpWg0laQnkJDhTmOH8k+L5GDVO0d4/QodhBP3oBLM/6ZRyOek6DRk6Dek6DTEeSL57XokUL2ssCRLOFwIhxbCQGF+1rZis7HnEeAm1xoEQ46XUGi8gxmtzKC2c+71hqbXGfonxEmuN5xDj3BQ2l84jxkus051NjIvTE7m6PGdp8efdExAxXsJzCIQYL6XWeD4xvnVGQQ4sjv9NPSBivIRa44CI8RJqjQMhxhIpXvmQSLEH+CLFVofA7r4T9JlH0KliuCV1IzJBFjApdogib3ZX0TM5ikYZzp0Z61HJzxo7BUKKAQ71t3N0oAsBuCm9lLzouRt0IKTYGdfB2+0NtIwNIxcEbs4uJjdGOytQkAqpQ2DMYua1plpGLWbC5HJuyikk25MKDUGRYgDztJXdzc10jo0BUBgfz/acHNQyb6ZQwZNicP6eqnr1HO7swmp3IABrUpLZnJmOavYHdQhJsesei81GZVcfp7v17lRxbWQ4GzOdyrHM5Q7tbYJlIJsOh0j74Ai13f10Do66r6sUcvKS4ilKTSAlJio0LtDLYGxlsdroGhyjs3+EroGxOb2QwdkPOT3eSZDT4jWolGd/xxIpXt5YEiScLwRKikFypfa3fpBcqf3NIblS+573UnSlXk5SrDcsvdOEK15y0qXRbUIixR7gixTbRBlG2xSvdx9m2mFjlSaLjfGFAZNigEnbNC91nMRos5ARoeXG1NVuohMoKRZFkQ97m6gdNSAXBG7PWEV6ZKz78UBJMTjTd9/ucBJjmSBwU2YR+bHxM4GCJ8UAk1Yru1rr6TVOIABXpmWzNjFlIWkKkhQLopOwHu/p4XB3NyIQFRbGdbl5ZMRoPMRZHCl2YcJi4dPWDlqGnHXXEUolW7MzKU7QOZ/LMpBiF8xWJzmu6jlLjmPD1azLSKUoSYfCW/rvMpPN8Ukz9T0D1PcNYDSfrc/RhKspStFRlKIjJjyAvnDnkBTPGe4QGRgz0TkwSvfQGIZ5KrIA6DSRpGpjSIvXkBobjVLhxRk8BOuRSLEECecHwZBikFyp/a0fJFdqf3NIrtS+573UXKmXkxSv/9y/IQ8LvEevN9inzZz46/93Sez3Ein2AH+kGKDd1M9HhtMAXJtcQao6yWMsT6QYQD9p4vXuk9hFBxVxmVymywMCJ8XgJIe7OutoMw4RJpNzV9YaEtRRPuN4IsUANoeD9zqbaBwdRACuyyigRJu4aFIMTrL9UWcrNUNO85BSbQLXZObNJXOLIMUu6Ccm2N3SwpjZWTu6LjmFy9MzAoofbJugzuFR9rW2MzozV3J0FFflZDnbRAURZzFzm6dtVPXoOd3T506rjgxTUpGeQnlKEmHzCdtyk82Z66Io0jMyTkPvAC39w9jsZ3PYU2KjKUzWkZ+oRR3mocXVuVhngOOnrTZ6hyfoHhqjZ3CMEdPUnMdlgkCiJtJJkLUxJMVGea6nCtF6QCLFEiScCwRLisG57x741AMxDgEpdl4XOXx0ITEOBSkG58fNiTOdC4hxqEixKMDpBg+KcYhIsShAbbue/WfagVnE2OvrH2R8GTT2DPJx1TxiHGx8H3O0DgzznifFOETEvnt0jF1nFhLjUJBi8KEYh4AUAwxMmXi1zgMxDgEpBh/EOASkGGDCaubl5uoFxFgixSsfEin2gEBIMcDhwXpqx7tQyZTcmno5kYrwBbG8kWK7Q0bThIE9+loArkkqoTAmOShSDGC1OXi9q5reyTEi5Eruzq4gNiw8aFIsziiwH3Y1UzviJLHbUnOoiE/zOD4QUuyMK1LZ38cn3e2IQEpkNLfkFs0y4Fo8KQaYttv5tKOD6n7nmnXhEVyfl3/WhCtEpFgQnQcHlb19HOvqwTrj8FiUoGNrZiZRqnm1KSEkxa7r0zY7NX39VHb3YppJ/1Up5KxOTWZ1WjIRLvJ5jkjxbFhtdloNw9T3DdAzMu6+LhMEMuM1FCbryE6Im0smVwgpng+jeZreoTF6hsbpGR7DODXXrdJFklO0MaTGxZASN09JlkixBAkXBBZDihFmFOP5xDgnwePwYEkxshnFeB4xzsv3fPAeLClGEM6mUs8ixiWFC71JfMb3c30BMS7J8Dx+kURtATGeX2PsZ53+XrcFxHh+jbG/+L7mEKCt3wMxloeG2CPMKMbziLFM4eXFXgQZ1497IMZeMqqCJcWiMJNKPZ8YK72UyQVLZvFCjMM8mG/5iuP19RGdivE8YqxRLTTfktKnVxYkUuwBrjfU/9S8uJAUz0qTtosO3uw5zqBlgkRVLDelrEcmzP3QsXn5xHelWx8aaOXUSCcyQeD29AqS1LE+x8+HQxSw2G283F7FoNlEtFLF3dmrifZA0MG/giyKInt72qgcdLZuuCwpg81JGQs3gyAV5I7RMd5pbcBitxOlDOPW/GKSIqOCTz/2Mm/r0DAftrZittmQCwKXZ2RQkZKCPEjXaCEAkm60THOovZP6fmd7DoVMxvr0VNampbhJn8861EUQ8tmwOxw0GAY52dHL6JRTuZYLAkXJCVSkp6CN9Py7D5qMe4OfdU6YLTT3DdGkH2Rw4myfY4VcRk5CHPnJ8WTqYpELnt/T56OGF7zvb+Mms5MgD43RNzyByTyXJAsC6GIiSYmLJlkbQ0psNOEqL+q4p3m9keJ7JFIsQcJywk2K/2PXAlLsXdl0Xpdcqb3En3X9UnalDmQOyZXa92Z8KbhSm40mHt0qGW2tFEik2AMCJcUA49ZJXu0+itVhZ5Umm43agrnj/ZBiURTZ3VtDm2mQcLmSz2SsJ0bpQXH2QYoBTLZpXmmrYmR6Ck2Ymrsz1xCpXHgqFYiCLIoiRwxdHNI7N7MKXQpXp87bDIJNqxYFRsxTvNlcz4h5CrkgcE1WHmXxCzd5n3F8zGuanmZPayvto6MApERFcW1uHnHhHkjiEkixC4YJI5+2dtA3PgFAlCqMy7MyKEzQeSfjPuYOlBS74HCItA2OcLKzB8OEyX09SxvLmvRkMuI0c35n54oUz8awcZKmGYI8PnW2f1+YQk5OQhx5SfGkazVuB2uf6wxyPcGOD0SJFkWRiSkLvUPj9A6P0zc8wcTUwr6Emkg1ybHRJMdFkxQbRWxUuNs3YMG8EimWIOG8YCmkGCRXan/rh0vXlTrQOSRXat/zXuyu1BIpXlmQSLEHBEOKAVqNBvYYqgHYkbiGrMizm5Y/Ugxgddh5vesUAxYjscoI7shYh1qu9Dp+NhyzPp0mrBZebjvNuNVCXFg4d2auXkCMg0mrPjXQy94ep6FFgSae6zMLz9brLoIUA5htNt5ra6JtbASA0vhEtmfmoJQFWBfrZ15RFKkZGODTjg6sdjtyQWBTWjrrU1LmbgQhIMUAokOkaXCIg+2dTFicCqI2IpzLMzPI1S5so+QrVrCkePZz7huboLK7j9bBEffDcRHhrEpLojhJR5hCcV5IsfsWUcQwZqRZP0SLYRiT5azaqpTLyNTFkpugJVMX6/kkeBHrCXb8YtOzJ6Ys6Icn6BuZQD8ywfAsddwFpVxOYmwkibFRJGmiSIyNImIm5f5ck+JQpFRdSulUEi5eLJUUg+RK7W/9cGm6Ugczx8XmSr2AGC+BFMPF7UotkeKVBYkUe0CwpBjgwEATteOdKAUFt6VtQqN0brCBkGIAk83CK51OR+okdQy3plWgmEUUAyHFAGPTZl5pr2LCCzEOtta4bniA97ucH9ZpkTHcml2MWqFcNCl2ziVytK+bw71diEC8OoKb8wqd/Yz9xQlw3nGLhY9aW92tm3QREezIySU5Kspn/GBJseu6ze6sNz7Z3Ytlxik6KSqKLVkZZMRqgoq1cE2Bjx+dNFPVo6dOP4B1Zh1KmYzCJB2rUpNIiAqi17I3LIIUz7ldFNGPOglya/9cgiwTBNLiYshJiCMnQUuU2kuNTwDrCXb8YknxfFimbehnCLJh1MjAmBHrLBMyF6LUYSRookjSRJKgiSIhJtLdCmo5SfHGzzyBQrk08w2b1cyx1/7pktgkJVy8CAUpBsmV2t/64dJzpQ52jovNlXoOMV4iKYaL15VaIsUrCxIp9oDFkGKrHd7tO4HBMkqsMpJbUzehlCkCJsUAwxYTr3adYtphIztSx3Up5e6Uy0BJMcDY9BSvtJ1hwraQGC/GgKtrYpS32uuZdtiJU6m5PaeUWGWEx/GBkGIXusbHeLe1kUmbFaVMxo7MPIrjE3zHCYKMi6JI4+Ag+zo6MNtsCMCapGQuS09HJfesRi6WFLtgsdk42d1LZa8e24wZV4ZGw2VZ6aRERwcV6+yaghsPMG2zUacfpLrXwMjkWTfllJgoylOTyE+IP6v6n2NSPDuOS0FuGxihbWCYUZN5zpCE6EiyE+LI1sWSEB0ZnPLuY15PCBUpnj/eIYqMGKfoHzViGHUS5RHj1ILbZILAwzs3opDLJFIsQcI5QKhIMUiu1P7Xf2m5Ui9mjovNldpNjENAiuHicqUuitVxY3aRRIpXGCRS7AGLIcV2h4xJm4U3eg8zZZ8mMyKBHYlrsOPZjc8bye2eHGNXTyUOUaRMk8YVCQUIghAUKQYYNZt5taPKTYw/k7maKKVqUaQYYHDKxBtttUxYp1HLFdyaVUJapMbDDR7DeP0UNU1P825bI90TTsfi0vhEtmXkEDY/ndqFIBVqQXT2TP6ko52GoSEAIpVKrsrMpkCrXUCylkqKXZi0THO8u5czegOOmRcxK1bD5sz0oNs4LSXtWRRFekbHqe410Do44l6LWqGgOElHWUoi2ggvBxxegwa5ziDijJimaO93EmT9mHHOYxFhSjdBTtdqzhp6rHBS7AnTNjuDYyb6x5xKcv+YiTCFnM9tXQ1I6dMSJJwLhJIUS67Uvtd/qblSL2oOQXKl9jfvxeBKfaCvg+3puajkCokUrzBIpNgDXG+AP9c+HzAptonOP/p+8xi7eo7jQGRtXA4VsQWex/sguc0T/bzf52zVtCk+mw3x2di9kEGHl53DIQpOxbj9DBNWC5owNXdmrfLuSu2tZnbWp5/JOs0brfUYJo3IBIFr0/Mp1SZ6HT93As+XEQUcosiR3i6O9HUDEKdSc2NuIYkRUR7Ge4/j+frZ/+wYHWVvW7u7r3GmRsP2nBxi1WqP4+eu33N8f0Rw3GzmWGcPdYYBd+isuFg2ZaaTHD33+S22VdOCNXkZbjJPU9vXT01vP8ZZKcvJMVGUpSSSnxgfUMuk5SaPrtdh0jJNx+AoHQOjdA2NzklBlgkCqXExZOpiyYyPJS5S7fWLibf4S11nsOP9vW42u8NtODY1aeK7n5WMtiRIWE64SfGvPJFiLzdJrtRz1hNw/FnXLwlX6iXMIblS+95EL3RX6tkwG008esUNK3avX454KxkSKfaApZBigMbxXj4ZcJLabYmryYlMXjjej/JbNdLN/oFmAK5KLKAkxsOmgW9SDDA+bebV9jOMWc1EKVXcmbma2LCFxDgQUgxOU7D3OpppGnWqrpsS09mSfHbDWQwpdqF7YozdrU0YrdPIBIGtaZmsS0ydu9ksgRSDs9fw8Z4eTvT0YhdF5ILAxrQ01qemOtOJQ0yKXRibMnOsq4f6eeR4Y0YaKTHRvmOFiBS7xjtEkc7hUWp7+2kbGnFnA4TJ5RQkxlOcnEByTBSCl0jnihTPht3hoGd4nI7BUdr7R5gwz3V8jlaHkamLJV0bS5o2xvvJsZf4i1lnsOODed0kUixBwvJjOUgxSK7U/tYPl4Ar9RLnuNhcqT0S4yWkVV/ortQuSKR4ZcHbW35FwG6389hjj5GTk0N4eDh5eXn88Ic/xB+P37t3L+vWrUOlUpGfn8+f/vSnc7PgGRTGpFKuyQTg04FqBixjQcdYHZfOBm0WAJ/0N9Ew3reotcSEqbk7ZzVxYeEYrRZeaq+kf2piUbEAlDI5N2cXsinRuYEd7e/mzfZ6LHbbomO6kB6t4W/K1pAXq8Uhinza3cErjTWMW8z+bw4QCpmMyzIy+Pzq1WRqNNhFkcPd3fy5spKGwUG/763FQhOuZmdhHl/YUEFJYgIC0DEyystVNbxcVTNDTs/N+ZRMEMiOj+OmVUU8dPk6LsvJIEatYtpup6avn1dO1fDMkUqOtHe5+yCfb8hlTofqK4uz+ZsrK7hv6xq2FmWRHq9BJghMmKep6e7nvapG/mfvcV46coZDTZ10DY1h82ByJUGChJWFC3W/9wSZTGDrFUXk5CYgiiKf7K2ns2MwZPEFQeCyjbkUFTgP3PcfbKK5pT+k8devymR1cSoAh0+1Udu0uO8g3lBRmMbGUmfq9Im6bk7Vd4c0fll2MleUZwNQ1drHkbrOkO6xhak6rlmVjwDUd/fzSU1bSOPnJGi5YXUBMkGg2TDEh9XNOLypF4tAepyGW8qLUMhkdAyP8k51I3ZH6PbK5Ohobp+pKe4dn+DN6nq3+WcooIuI5M7iUsIVCvpNJl6fMeEKFWLV4dxdWEaUMoxh8xSvNNZgsk77v1HCBY0VrRT/6Ec/4pe//CVPP/00ZWVlHD9+nC9+8Yv827/9G9/+9rc93tPW1kZ5eTlf+9rX+PKXv8yePXt45JFHePvtt7n++usDmne+UjwyPY5aHka4XB2QUgzgEB18oD9N1+QQalkYt6RuJnpW/2F/SjE4T2v3DzRzZrQHAbgmqZT86KR54wOrNTbZpnmjo5oBswmlTM4t6WVkRMbOGu8xjA/lV6B2uJ8Pu5uxiyJxKjW3ZJcQr/LgbgwBKcVn5xSpHjDwSXc7VoeDMJmcqzKyKYtP9KpeBqoUuyDMmDw1DQ3xaWcnxmnnh11yVBRXZWadNcVyr39pSvH8NY1OmTnR3UN9/6C7zlcbHs66tBSKEnQBtY8KWXqzeLb2uE4/QOvAMNZZm2NyTBSFiTryE+KJCFOeF6XYF6w2Oz3D43QNjdI9PM6Iaa6JlVwmkKSJJi0uhjRtDMkxUR5TxYJdZ7DjJaVYggTvOB/7/XIpxS5IrtS+1w8XsSu1j3suLVdqH4rxEpRiFy50V+rlVIr1hoGQKcXJSQmXxH6/oknxLbfcQlJSEn/84x/d1+666y7Cw8N55plnPN7zv//3/+btt9+murrafe3ee+9ldHSU3bt3e7zHYrFgsZxNxxwfHycjI4M/1z7PiHyCg0OVxIVp2J64CVH0nJY5nxQDTDts7Oo5wfD0BLHKSG5K2YRqpv9wIKQYnJvSvv5Gasf6EBDYmVxKblTirPGBG3BZ7Dbe6qylZ3IMmSBwQ2oJ+TG6mfEew/gkxQD6yQl2tdczYZ1GIZOxM62A4jgPxiJBkGLndeeH0XvtzfSZnMp2ZkwsOzPz5jQ+DySOJ8z+zLXa7Zzq6+N4b6+bDBbGx7M1I5MY1cxcISbFLhgt05zu01PdZ2B65hQ1MiyMNSlJlCcnOT/czwEpng2r3U7r4AgN+gG6RsbcDws4T5cLE+PJ02lRzU9VOo8u1rNhNE/TMzxG98yPyWKd87hCJiM5Noq0OA2pcdEk+SPJEile1lgSJMC52e+97fUuUmyemmZsZJKk1NiQkGKQXKn9r/8idaUOIfGWXKl9z3shu1IvJynefOsPl9xpApzdJo689dglsd+v6PTpLVu2sGfPHhobGwE4ffo0+/fv58Ybb/R6z6FDh9i5c+eca9dffz2HDh3yes+TTz6JRqNx/2RknHVDjFZGIUPGoGWEypG6oNYfJlOwM2ktEXIVo1YTH/efxi4Gl54iCAJXJxZSGJ2MiMgefS3tpsWlYankCm7PWEVetA6HKPJuTy2nh3sXFcuF5Iho7i+sIDNKg83hYHdXA3t7WkKShhOrDuezReVcmZ6FXJDROT7KM7WVVA3oQ5qmpJTL2ZSezoMVFZQmOAl949AQfz5dyf7OjpCm5MxHlCqMrdmZPLRxLVuzMolUKjFNT3Owo4v/PnaSj1vaGJla2L5nOaGUyylK0nHb6hIeumwdV+RlkRgdiQh0jYyxp6GV/3fwBG9XN9DYP+gm8ysFUeowilIT2FGezwNXruO+LWu4qjiHvCQt4WFKbA4H3cPjHGnp4rXjtTy19xivHa/hSEsXXUOjTNtW1vORIOFSwLnY733t9RazlQ93VfLRO6fp6x4OwTNyQiYT2Hrl2VTqfR8vQyr1plyKCpc5lbokDYDDJ0OfSr2maHlTqUuzk7liVTYAVS3LkEqdpmP7amcqdV3XMqRSJ2q5frlTqVctYyp1zNxU6reWI5W6xEmEpVRqCUvBilaKHQ4H//iP/8hPf/pT5HI5drudf/u3f+PRRx/1ek9hYSFf/OIX54x55513uPnmm5mcnCQ8fKHJlLfT42frniUiOoLuyX72DZwCYJN2NTmRC02vvPUjtjlkDFkm2NV7HJtoJz8qmasSyhC9tGrymp7tgD36Opom+pEJAjemlJMVFb8oV2qHKPJxXzPVI86T5fXx6WxJ8OyeGKiC7BBFDvV1cdTg3MxSIqK5JbuIqBlV1+tZQIAK74h5ig/aWug1OlXjjGgNO7Py0Kj8nIIFaZyFCP0mE592dNAz7mwTpZLL2Ziaxprk5LO9ff3F93pU61sxtDscNA4McqpHz5Bp0v1QdlwsFWkppGti3L+nkBlGBajYjk6aaeofpMkwxPCs3sdymUCWNpb8hHiy4+MIm99+4Xy5WHsaOtMzuHd4nJ6RcXpHJpianqskCwLooiNJiY0mOTaaFE00kWovLRk8zRuC9U+ZTPz93ZJSLOHSwbnY773t9X/45ZuEhYXzyYe1dHcOIZfL2HbDKlLStQsnXaSCLLlSe4k/6/pF5Uqd7qNd0yLnkFypLz5XarPRxKNXSunTKwXeLVpXAP7617/y7LPP8txzz1FWVkZlZSWPPPIIqampPPjggyGbR6VSoVJ5SMmdQXpEIqs0eZwZa+HESDUaZTTaME3A8eNV0VyTtIoP9KdpNuqJkKtYry0Kao0yQWBHcjF20UGrcZB3+6q5IaWMjAjPPRD9xbomJZ9opYpD/R2cGOpm3Grh2pSihaQviJhbUzNJjojmvY5G+iYneLbxNDdkFZIVHbuomLMRpw7n7uIyTuv1HOjppGtijGdqK7k8NZOKxBRkAbbjCQSJkZHcWVJC++goBzs7GZqaYn9XJ5UGPZelpVOSkBDS+WZDLpNRkpRIcWIC3WPjVPb00T48SvuI8yc+IpxVKckUJehQedpMlhGxEWo2ZqezISuNIdMUTYZBmgeGGZsy0zo4QuvgiJsg5+q0ZMfH+XSBPh8QBAFtVATaqAjKM5MRRZGxSTO9IxP0jozTNzLOhHmagXETA+MmqjqdXzBjwlUkx0aTpIkiKSaK+OiI4OqSJUiQ4BPnYr/3tdfL5TKu2lnqJsZ7d5/xTowXAZf5FkBb6wD7Pq737Eq9SLjMtwAamvTsP9gE4JEYLzb++lVOA9Gq+l4On2wDkQWu1EtBRaFTjT5W28WJOucBu0divEiUZc+o6dXtVLX0gYjXPsaLQWGq83f50Zlm6rr7EcGzK/Ui4TLf2l3VRLPB2QHEKzFeBFzmW7uqG9yKsTdivBi4zLfeqK53m295JcaLgMt869X6Wrdi7JUYLwIuxfjlxhq3YuyVGK8AhKvDCA/iQN8brNMr8/ktB1a0UpyRkcEPfvADvvGNb7ivPfHEEzzzzDPU19d7vOeqq65i3bp1/OpXv3Jf+5//+R8eeeQRxsbGAprXdcriUophprZ34BQ9UwOEy9Vcm7SFcPlZldKXUuxC40Qvn860atoQV8Cq2Byf42fDVSNsFx180FdLq3EQGQLbk0oWmG85xwdWa1w/2s8HvU4ThyR1NLdklBGpCJs13mMY77XGosCoZYpdbQ0MmE0ArEtIZUtSlmfCHWQtMA6BUYuZD9qb6TE6lVxdeATXZOaSGuXhBGsRSvGcYaJIw+Agh7q63GZcsWo1m9PSKIzXIQv6SDZ4hXRkcorTvXrqDAPYZlKalHIZxQk6ypOSSIjyYm4WYPyl1PaKosigcZLmgSGa+50E2X0/kKKJJlenJSc+Dk34XFX/fCjFgcSfmLKgH52gb+ZneGJywS1ymYAuOtJNkhM1UcSEqxAEQVKKJUhYBM7Hfu96H//hl2+6jbbsdodvxXiJtcZ+FeMl1hr7U4yXWmvsTzEORa2xT8U4BLXGPhXjENTp+u1jvMQ5/CrGS4zvTzFeaq2xX8V4ibXGfhXjJdYa+1WMV4hSLLVkCh4rWuqYnJxENo9IyeVyHD5qHS6//HL27Nkz59oHH3zA5ZdfvqS1CILAFt0qohWRTNnN7B88iV0MriaiMDqVjdp8AI6PNFE/3hX0OuSCjGtTSimITsSByB5DLfXji68LLo5N5I6sclQyBQbzBC+2nWLAbFx0PIBYVTj3Fq5idbzzVPbkQC8vNlcxbJ70c2eg8dXcXVjGjsw8VHIFg1OT/LWhmg/am5myWf0HCAIyQaAkIYEH11RwZWYm4QoFo2Yz77W08Jeq09QNDrjdo5cLceHhbMvL4Uub1nFlThax4Wqsdgdn9P08f/oML56upsbQf15qewVBICE6kstzM/mbTWu4d8MqNmSlER8ZgQj0jk2wv6WDvxyt5LljpznY2knv6Piyv2ZLQXS4ioIUHVeV5HDP5at5ePsGbllXzIbcNDLjNagUcuwOEcOYkapOPR9UN/PsgUr+e+9x3jxRx6HmTlr6h5kwW85Zm60LBb/97W/Jzs5GrVazefNmjh496nXsq6++yoYNG4iNjSUyMpKKigr+8pe/zBkjCILHn5/97GfuMdnZ2Qse//GPf7xsz1HC4rBS9nuXYpyepcNud7B395nQ1xhfscw1xhsv7BrjisI0NpYtc7umVU5RYllqjGe1a1qWGuMELTesLpTaNXmBSzGWaowlBIsVrRQ/9NBDfPjhh/z+97+nrKyMU6dO8ZWvfIUvfelL/OQnPwHg0Ucfpaenhz//+c/A2RYN3/jGN/jSl77ERx99xLe//e1FtWSarRS7MGIx82H/QaYdVjIjUrlMuwZBEAJSil04PtzM6dF2AK5OWEVuVIrP8bBQ4XWIIp/MuFIDXK7LZ3VsxqzHA3elBhgxm3mzq5rR6SmUgoxrU4vJj9EtSimejZaxYT7obGLKbkMuyLgqNZvV2uSzp6aLUIpnY9Jq5UBPBzVDzk1fLVewNS2Tcl2Sc44lKsUuuA4op+12Tuv1nNT3uT9kNSo1m1JTKdbNSqsOoVI8/7ooivSMjVOt76dlaNhNMJUyGYUJOkqTEkiOilqYsrUMSrGvOONTZtqGRmgbHKFndHzOwyqFnCxtLNnaODK1GsKVSv/rDHBeF5ZLiXalXPePmzCMGTGMTTA4MemR6IcrFSTERKKLjiQxOpKEmEii1aq5v5tLRCl+8cUXeeCBB/jd737H5s2b+dWvfsVLL71EQ0MDiYkLUzz37t3LyMgIxcXFhIWFsWvXLv7+7/9+zme5Xq+fc8+7777Lww8/THNzM7m5zlTS7OxsHn74Yf72b//WPS46OprIyAAzLCScE5yP/d6TUuyCTYRPPqimp2OeYrxEpdgFyZXa3/ovbFdqgIY+yZXa17ySK7WkFK80rGhSPDExwWOPPcZrr71Gf38/qamp3HfffTz++OOEhTlTFR566CHa29vZu3ev+769e/fyne98h9raWtLT03nsscd46KGHAp7XFym2OeQYzEPsGziKiEhZTD7lmsKgSLEoihwcbKR+ogsBgR1JFe7a4EBJsSvOgYFWqkZnNg1tNuvjnKpIsKRYdAiY7Vbe7amjyzQKwMb4DDbpsj3WwwRKigGM1mne62ii0+iMmxsdx86MfCIUYUsmxS70Gsf5qLOVwSmnGp0YEcm2jBxSI738AS+SFLswbbdTZdBzsq+PKTc5VrEhxUmOFYL3GpmlkuLZa5qcnqa2f4AawwBj5rOpy9qIcMoSEylK1BHhIpznmBTPvm622ugYHqVjeISO4TEss05tBSApJoqsuFgytbEkRkUGV7N9noy8ZsPucDBsnKJ/3MjAuIn+MSNDpkk8fbqqFHJ00ZHooiPQRUWii4ogLjJ8TnracpLiUJhvLMZ4Y/PmzWzcuJHf/OY3gNNYKSMjg29961v84Ac/CCjGunXruPnmm/nhD3/o8fE77riDiYmJOephdnY2jzzyCI888khAc0g4Pzgf+70vUizKZM5U6vnEOMNLjXGQpBhhJpV6PjHO8ewTEiwpRjaTSj2PGOflLyy3cq3H8/q9xBeEs6nUs4hxibca40WQYvBAjEsyPI9fJFFbQIy91BgvhhR7bdcUqnZQArT1eyDG8hARe2EmlXoeMZYpvLzYiyDj+nEPqdTzzTrdcYLb7EVhJpV6PjH25nUSJCkGL8Q4zEsNrkSKVzxWNCk+X3C9AV6s/8sCUmyd6UfcPNHF0eEaAC6LX0VmhOcParuXT2qrXWDfQA0tRr0zJTppDWkR8V7JrDeybHfAieEOjg61A7AqNo0rEvIRvWTGO7y6VQsz/4p8amjj1JAzJTs7Ko7r04pRyed+iIhe4ngjyw4HnBroY39fO3ZRJFyhZGd6HnmaeI/jvX6K+nCxdogip/v7ONTb5U4lLtLquCIti+j5vY2DdY32Mn7aZueMwcDJ3l43OY5UKlmXkkp5YqLz1DNQeCWn/tckiiK94xPU6PtpHhzCPiPxywSn8VVJYgLZcbEeDTO8kt9l6jvscIgYxo20DY3QMTQ6x2UbQK1QkKHVkKmNJTNOQ6TKj8lDqEh9kPG9wfXbstkdDE1MMjBhYnDGuGvIi6IsEwTioyK4c3MZcpmMKZOJf/jM8pDiy2/4P0vuXWizmjm0+3G6urrmrM+bkdH09DQRERG8/PLL3HHHHe7rDz74IKOjo7zxxhs+5xNFkY8++ojbbruN119/nWuvvXbBGIPBQHp6Ok8//TT333+/+3p2djZmsxmr1UpmZib3338/3/nOd1CEyHxFwoULNyn+xUJS7PpD9ltjPG/8wuu+ybLkSu0l/qzrF6Qr9ax7JFdq3/Neyq7UZqOJR6++XiLFKwQruqZ4JSM/OoPSGGd63pGhagzm4GqCBEHgqoRSsiISnOZZhtP0TQVftyQIAhvis7kiwVmrfGa0h/f7ahdd/yETBK5OzuX6tELkgox24wgvtlUyNGOatVgIgsC6xFTuK1yDTh3BlM3KW+317O5sxByiOmCZILA2KZUHy9dSrnNu+g3Dgzxdc4ojfV1YHaGvuQ2Ty1mfmspDa9dyZVYWkWFhmKxWPu3s4H8qT3G4u4tJa2jrnD1BEATSNDFcV5TPw5vXsy0vm8SoSByiSNvQCO/UNfLfR0+yr6UN/YTxvNa6ymQCKbHRbMnL5L5Nq3nw8rVsL8olL0FLmFyO2WajqX+IPfUt/M+hkzx39DSfNrfTNjjC9DL2jA41FHIZSbFRlGcksa0sl89etoq/3bGRz162iu1luazKTCY1LpowhRyHKGKx2S44R+uMjIw5fV+ffPJJj+MGBwex2+0kJc1VqZKSkhakQM/G2NgYUVFRhIWFcfPNN/PrX//aIyEGePrpp4mOjubOO++cc/3b3/42L7zwAh9//DFf/epX+dGPfsT3v//9IJ+phEsV7hrjzPhzVmPc0b4MNcYFy1xjXJwKzNQYNy5DjfEy9jEuy07mivJsYKbGuHYZa4y7+9lXvRw1xsvcx/giqDEOl2qMJfiBpBR7QCBKMcykLw+epnNSj1JQsC3xMuLC5p6ieFOKXcqvXXTwof403VNDyAUZO5PWkhq+UD0NJK26adzAHn09DkRS1BquT12FWq6cO96PUjwb/VNG3uqqY8JqQSHIuCa1gGKNk2wGqxTPvm5zODis7+J4fzciEKFQck1aHvmxs573IpTiBes3Gdnb2UavydnbOEoZxmWpGZTGJwbvGh2gmmpzOGgYHOR4Ty9jFmc6s3zGrKsiKZn4iAgPQXzPEYhS7A1Dpknq+wdo6B/ENKsfr0atokAXT0GCDl14uOdT62VSin3B7nBgGDPSOTxGx/AoA8a5hzECkBgTRVpsDGmxMaTERHtV48+3UhxoHFEUmZiyMDltJTk2GmBZleJQpk8HqhT39vaSlpbGwYMH55ggff/732ffvn0cOXLE4zwOh4PW1laMRiN79uzhhz/8Ia+//jrbtm1bMLa4uJhrr72WX//61z7X/t///d989atfxWg0+mzFJ+HiRyBKsQuSK/X89S+ML7lS+55DcqX2Pe+l6EotKcUrCxeWLLHCIAgCl+tWkaiKwyra+HTgKEZbcIqqXJCxI2k16eHxToJsOEX35OJOiQtikrg5bRVhMjl95jFe6zrB2PTiHZ8Tw6O4J6eCjMhYbKKD93sa2NPbtGTFVSGTcUVqFvcUrEarCmfSZmVXRz272utDerqWGBnFZ4vKuTGnkOgwFUbrNB92tPCX2kqaR4aWRS1VyGSUJSbywJo13JhfQGJkJHZRpLq/n2fOVPFaXR1toyPnTKmNj4xga04WD21ax21lxRQmxKOQyRgzWzje3cvzp6p45lQVRzq7GZmcOidr8gW5TEZqbAyX5WZwz4ZVPLx1PTeUFlCakogmXI0IGMaNnOzs5a2qep7af4yXTlZzsLWTjuHRC0pJdkEQBGIi1G5CvNxw9S5c6g9ATEzMnB9vJFOn0yGXyzEYDHOuGwwGkpOTva5VJpORn59PRUUFf//3f8/dd9/tUY3+9NNPaWho4Mtf/rLf579582ZsNhvt7e1+x0qQ4ILkSu0/vuRK7RuSK7VvSK7UFwc++eQTbr31VlJTUxEEgddff33O40ajkW9+85ukp6cTHh5OaWkpv/vd73zGfOqpp7jyyiuJi4sjLi6OnTt3Luhe8dBDDy3oNHHDDTcEtXaJFC8RckHOVYnriFVGY3ZMs6//GFN2S1AxFDI5O5PXkBmhwy462GM4Refk4jarjEgtd2asI0qhYsw6xWvdJ9FPjS0qFkCEIozbM8vZpHNuRDWjel5oPcXgEtOpAVIio7m/sIJNiekIQPPYEE/Xn+T0YF/INgpBECjS6niwbC1XpWejlisYMTv7KL/QcIauicW/Nr4gEwQK4+O5t6ycu0tKyY/TIgCd42O82dDAn6tOc1qvP2dtlGSCQFZcLNcXFfDlzeu5viifXG0cMkFgZGqKI13d/OXUaZ47VcXRrm6GVwBBBghXKslPjOeaoly+sLmCBy9by46iXIqTE4hWq5wkecLIya5e3jpTz1MHjvPiiSo+aW6neWCIyelLb0NaiQgLC2P9+vVzDLAcDgd79uwJqn2Ow+HAYln4+frHP/6R9evXs2bNGr8xKisrkclkHh2vJUjwBblcxpXXlpGWtYyp1FcuMzHedGET4zXLnEpdmp3EFauygeUjxttXLycxjuP65U6lXnVuiXEovyfpIiK5s0QixssJk8nEmjVr+O1vf+vx8e9+97vs3r2bZ555hrq6Oh555BG++c1v8uabb3qNuXfvXu677z4+/vhjDh06REZGBtdddx09PT1zxt1www309fW5f55//vmg1i6lT3tAoOnTszFhtfKR4RAm+ySxyhi2JW4mTKb0mz49G3bRwceGajom+xEQ2Ja4muzIJK/jwbubtNE6zbu9ZxiwTCAXZGxLLKYgJimo9On517tMo7zf04DJNo1ckHFlYi7lsSlz0n8CSZ+ee935b/+UkT1dLRimnD2SkyOi2ZGWT8L8lLYg0qed1+f+r8Vu46Shl5OGXqwzH+RZMbFsSckkKTJqyenTLnjK5hm3mDmtN1A9cLancJhMTnGCjtWJScSHe06tXkr6tL/xFpuNtqERGgeH6Bodm2MCpQ0PJz9eS358PPERXlKsA1hPyFysPWDcbKF3ZJyesXF6RscZNy8kTLHhalJiomd+oogL5Lkscj0QGtfr5UyfPp8tmR588EF+//vfs2nTJn71q1/x17/+lfr6epKSknjggQdIS0tzK8FPPvkkGzZsIC8vD4vFwjvvvMMPfvAD/uu//muOIjw+Pk5KSgq/+MUv+NrXvjZnzkOHDnHkyBG2b99OdHQ0hw4d4jvf+Q433ngjTz/99JJeAwkXPoJJn3ZBcqV2rd9LfMmV2v8ckiu133kvFVfq5UyfDkWplCtesN0mXBAEgddee22OwWZ5eTn33HMPjz32mPva+vXrufHGG3niiScCimu324mLi+M3v/kNDzzwAOBUikdHRxco08FAst/0AbngQC7MZWEOL38d0Uol1yRt4AP9YUat4xwYPMb2xA3IBW/W7AsvKYBrksrZ119Dq8nA3v4qrkooJT86BW+fNXYvnypRyjDuyKjgw75a2kxD7DHUMmI1slGb65kQeCObs4ZmRWv4fH4F73c30W4cYa+hme7JEXamFrrdqb0eSHo7e5lZf1JkJPcWreL0oJ6DvZ3oJyd4rukU6xPTuCwpw11X4vUjKEDSqpLJuTw9g9WJyRzt7ebMoIGO8VE6xkfJjY3jspRMEiM89C8N0q3a0/DoCDVX5GaxKTOduoEBqvr0jJjNVBkMVBkMpMfEsDopidy4uLl1Ol5fu+DW5OnXrgpTUJycQHFyAlNWK21DIzQPDtM1Osbw1BRHu3s42t2DRq0mTxdHbrzWcw9kvJPfpf7O3PE9XIuJUBETnkBxqvMLo9EyTd/oOL1jE/SNTTBonGR0yszolJk6w4DzOSvkJMdEk6KJJjkmisToKMK8bLbBrge8v9V9te5YEOMizOG55557GBgY4PHHH0ev11NRUcHu3bvd5ludnZ3IZr3vTSYTf/d3f0d3dzfh4eEUFxfzzDPPcM8998yJ+8ILLyCKIvfdd9+COVUqFS+88AL/8i//gsViIScnh+985zt897vfXd4nK+HCgih6+MP18jnqcKAQ4Oodpe4a4727z3h3pfbyASJ42XNFmYBMcKZSA7S1DrDv43quBo+u1ILXzwove5MDBAQu25ALIjQ06dl/sAnAsyu119fB2/qdd6wvzwBRpKq+l8Mn2wA8u1IHtfqzqCh0qtHHars4UedUiz25Ugve/E/88LeyrGQQYX91O1UtTrXbkyu1r3V6fW4OKErRIYjw0Zlm6rr6QcSrK7XXvcZbfCB3xnxrd1UTzYYhAK+u1MHGB0iPdaZS76pucCvGwbpS+5rXpRi/UV3vVoy9uVIL3t7r3v72gITwSO4qLuXV+lq3YuzVldobvLX2lIluxfjlxhq3YuzRlXoZZcm77v8tCsXSOk0A2GxOf5zx8fE51715iPjDli1bePPNN/nSl75Eamoqe/fupbGxkX//938POMbk5CRWqxWtdu5n7t69e0lMTCQuLo5rrrmGJ554gvh4L11uPEBSij3AdcrycsPTASvFLkV4ZHqcDw1HsTpsJKvj2arbgNxDz1pfCrJDFNk/UEuT0flBfHl8EcUxWV7m9fxHaZ9RlkVR5PBgG6dGOgHIjtRxTXIJYbK5f/heFWQPHzaiKHJysJcDhnYciEQrVVyfWkRapMZ7H2SvJkMLxxunLXzc3U7zmPODPCZMxba0HHJjtP6Pjxdc93zZNX7MYuZwbxf1QwPuofmxWjanZJAwmxwH28LJF2ZiiaJI19g4Zwx6WodH3FNEKpWUJSZSlpBAjFodNHn0Sor9rGc2XApy8+AwnSOj2Gf9AiOUSnK0ceTEx5GuiXFvUqHsa+wJi1FgLVYbfeMT6MeM9I1NYBg3YvOQ6qWNDCc5Jpqk6EiSYqLQRkSc/fKw3H2QPWDKZOJ7t19cSrEECSsNbqX45294UIr9G08F1K7JqxrmX0Geb7511bZisrLnEeMlGHAF0q5pKQZcC8y31uZQOk8xXqoBl992TUs04JpjvpXrQTH28RUgkOc223yrOD2Rqz0R4yUYZAXUrmkJ8eebb91YVohCPvfFXYoBV0DtmrzG8b95B9SuKVglfZaC7M98y2w08ei25VGKt+74l5CR4gN7/mXB9X/+53/mX/5l4fXZ8KQUWywWvvKVr/DnP/8ZhUKBTCbjqaeeciu+geDv/u7veO+996ipqUGtdj7HF154gYiICHJycmhpaeEf//EfiYqK4tChQ8gDbI8qkWIPWAopBhiwjPCx4Tg20U6KOoEtunULiLG/tGpRFDk81EjtuPPDfrUmh3Vx+Qs+LP2RYhcaxvXsNTRgF0XiwiK4PmUVsWFnn1swpBhAdAjopybY3V3PmNV5irRWm8ZlCdkoZAtfo2BIsfMBgdaxYT7qbmXC6kyJzYzScFVqDjq1JyV3caTYheGpSQ73ddM4fLZ+K1ejZVNKGsmR0ctCimdjwmKh2tBPdX8/U7NaOGXExFCakEi+Voti/gnsMpLi2Zi22ekYGaV1aJj24dE59T1ymUC6RkNWXCw5cbFo1EF8AJ8DUjwfdoeDIeMkfWNG9GMT6CeMTHhIuVbIZMRHRZAYHUliVBSJ0ZHERYQjm/X3J5FiiRRLuLCxVFIMkiv1wvUvjH8pulJ7W6uney42V+r5xFhypfZOjJeTFIc6fTrQbhOz4YkU//znP+epp57i5z//OVlZWXzyySc8+uijvPbaa+zcudPven784x/z05/+lL1797J69Wqv41pbW8nLy+PDDz9kx44d/p8oEin2iKWSYgCDeYi9/Sewiw6PxDiQWmNRFKkcbePkSCsAeVEpXKErQzYrXypQUgxgmBpnd18NJpsFpUzONUkl5EQ5004XQ4rBWaP7iaGV2lGnq2xsWDjXphaRHD73D3ExpBjAardzxNDNyYEe7KIzLWtVfDKXJ2USrlAuGL8wjufL3sYPTpo40tdN08iQ+1pGtIYNSWlkRmsWnuCGiBSDU2m1Oxw0Dw9TMzBA19iY+7EwuZyieB2lCQkkRUY613GOSPFs2B0OekbHaR0eoW14BKNlroFEXLiarLhYsmJjSdPELCTygcy73MrsvPEmyzSGCSP6cSP940b6J0wejT0UMhm6qAgSoiJJiIokMSoSbWT4wnQxiRRLkHBBIBSkGGaI8Z46ujsGFxLjJZJi8EOMl0iKYYYYH59bY+wixkslxa74J6q75tQYu4hxKEgxQGVTD8dqPBDjEJBigJoOA/vPONPA5xDjEJBi8EOMl0haAdoGRthd1eiZGIcgvi9ivFRSDH6I8RJJMTi/+71SP6/G2EWMl0iKwTsxXk5SvBJaMs0nxVNTU2g0Gl577TVuvvlm97gvf/nLdHd3s3v3bp/xfv7zn/PEE0/w4YcfsmHDBr/zJyQk8MQTT/DVr341sPVKpHgh5pNig9lAhDyCaGV0wKQYQD81xL4Bz8Q4GAOuhvEeDgzWIyKSFh7P9sQ1KGfSn4MhxQBGq5UP9NX0zThSr4nLYFN8LoKX5+WPFLvQNjHMnr4mTLZpBGBtfDqbddluUrRYUuzCmMXMJ71nU6pVcjmbEzNZE5/sJCUhIsWu8cP/P3vvHSbJVd77f6pzTpPzbM7aVdhd5bhCEjJIGNtCcI1A4AhcC2yC7r1GwuCfCDbIBmywL5Z8wQIkm2ALLEBIi0A57Gpznp2cejqH6li/P2q6Z3q6emZ6pmd3dvd8n6efnjl96q23e3rq9KfPOd83meDVkUGOBvzkJxsbrHa2N7ex2lM3NWNYYyierogsc8jv59DYGNFpLspei4X19Q2sr6vHpfUt3RJC8fT+iqIQSCQ5HQzRGwgyFImWhNLrJNpcLro8Hjq9bnwz6yEvEygue1hRCCVlxqJxxqNxxqIxxmNxMrnyZdc6ScJnt9LgsFNvt1HvsFFvs1c27JinBBQLCS29agXFAFmFcvOtdl9NoBggr2iYb3XV1wSK1fZy863VqxprAsWgXnZnmm9tXNNSMyhWJA3zrfXtNYNiRdIw39rQWdEca7ZcK53j6LCG+dYs4F1t/FPjGuZbutrFHwiVm2/pdbqaQDGUg/HbJl2qawHFAONJDfMtg6EmUAylYLzWW89bV6694KC4EOunP/0pt912W7HfH/3RH9HT08PPf/7zirG++MUv8td//df87Gc/4/LLL5/z3AMDA3R2dvKjH/2It7/97fPLV0BxuaZDccQY4ZXAK5h1Zq6pvwajXruWaCXIHUwE+I3/NXJKnlZLI1fUX4xe0lcFxQC98Ql2j71JVslTZ3Kyq+kSbAZz1VCcVyRySp6X/CfZH1LNKZotbm5s2oRDY+/BfKEYQM5l2D18iqMRtcSDz2Tjpta1NFtdi4bigvojYX411MP4ZEkot8nCVc1drHHVaxuILRCKC4qkU7wxOsQB/2hxL6rHbOHixlY21jVglBYAP1W6NCt5hf5IhEPjY5wMBkv2xLY7XWyor2eVz1c0OztTUDxT6UyWvlCY3mCI3lCIeDpT8rjDZKLT46bD46bD7cI203BijvhnCoq1+k8HZX8szvjkLZXVLhXhNJuot9upd9ios6s3j9VSsvx6NgkoFhJaetUSioUrdSH/CvEvMFfq2XIVrtRz51mp//ngSv2rgdO8pXs1VoPxvITiWCzGiRMnALj44ov58pe/zA033IDP56Ozs5Prr78ev9/P1772Nbq6uvjVr37Fn/zJn/DlL3+ZP/mTPwEoq0bxhS98gU9/+tM89thjXHXVVcVzORwOHA4HsViMz3zmM7zzne+kubmZkydP8olPfIJoNMr+/fvnbQgmoFhDhTfAj449gsFmYLf/eaLZGE6DnavqrsGsL39xZ5tBLp0xrueahosBY8X+ldrH5TA/G96LnM9gN5i5pfli3EbtN2glWJ5uhHUyOs4zI0dJ53NY9EZuatpAh73Upa3SuyNfIc+8InEy4ueXQydI5lQo2uZr5fLGbkxae40rOUPOUtoprygcDIzywnAfiax6jmabg2tauml3uOeVf7Uzy8lMlr2jw7w5NoKcU2vaWfQGLmpoZmtjc7mj4GxaRE6pbJaTgQCHx/0MTnMC1EsSK7xe1tXX0+3xaC9bPoOGXYVZ5N5giL5gmKFwpMSsC6DOZqPD46LD46bV5ZrdAZoaGnlV2b/S81XyClE5xXgsgT8Wxx9L4I8lNPcog/o38tmt+OwFUFZ/dphNZR+qkvE4n3ibgGIhoaVUEYq/+MNyKK609WMOWJ6X+da0/vONX4DlOfcYF/Nf2AzyfMy3ZoszFyzPtcd4Zj7zjj+tfU7zLVjUDPKse4yrzFWrf8lS6vbGiq7UC40/L/OtRcSfuZS6Wlfquc47L/OtWePMPtjPy3xrlvjVwPL5CMW7d+/mhhtuKGu/5557ePTRRxkZGeH+++/n5z//OYFAgK6uLv7wD/+Qj370o8X3+fXXX093dzePPvooAN3d3fT29pbFLJh9JZNJ7rzzTvbs2UMoFKK1tZW3vOUtfPazny1WuJiPBBRraDoU2502Etkkz/p/QzIn4zF6uLLuSoy6Uqida1m1CsZvkFNyNJl9XFl/WXEJtFb/Su2RTIKfDe8lnElglPRc33gR7bbyb4rnA8UAoXSSnw0dwp9S6wNf4u3isrru4r7lhUAxQDKb4bmRUxwJq7PGTqOZG1pW0+0o/XCwECguKJ3L8cb4IK+NDRZrDq90+bi6pYs6i23W/Be63Dqdy3HIP8aesSHCKRV+dJLEOl89Fze2apdzqvIc8+0fkVMc9fs5Mj5OUJaL7Sa9ntU+H2vr6mh3uaYGo7PgYl1QJpdjKBylPxSmPxTGH0+UxWxyOmhzu2hzu2hxOssgeblBcaX+qUwWf1wF5YlYkol4golYQtP1GsCo1+OzW6mzWbl+7Up0OklAsZDQGdBSQDEIV2rt/Evjn/eu1FXkqtX/fHOl1gLjC9mVuqDzEYrPZQko1tBMKAaIZKLs9j9POp+hzlTH5b7LMUyD2vnsNR6TA+wee52sksNncnNN/XbMelPF/pXaU7kMT4/sY1gOIgGX+tay2dVVcsGcLxQDZPN5fj16kkMRdYBqtri5sXkDLqN1wVBc0OlogGeGTxQdpFe76rm2aSUOozrbvhgoLiieSfPSSD/7J0aKnLLeU8/Opg68ZltZfzXQwqC4oLyicCoU4I2RIYbi0WJ7i93J1oZmVnvrKptM1QzUJ+8UhfFEgmN+P0cnJohP239sMRhY6fWyyuej0+Wucga5dlA8U8l0hoFQpAjJkVTp7KoENDjsKiC7nLS6nNgM2qsrKuosQbFmV0UhkkwxEUvgjycIxBME4klCSZn85D+ZzWTk3isvBZZ2pnhkZPGOlJFIhObmhgtikBQ6f7VUUAzClbo8//L457UrdZW5avU/31ypZ4Lxhe5KDQKKl5sEFGtIC4oBAukQz/lfJKtkaTA3sNO3s2icNV8DLn8qxO6x10nnM7gMDq5t2I7NYK3Yv1J7TsnzwvgRjkbVAWWVvYUr6zcWyyFVA8Wg7kE+Hh3lubGjZPI5jDo9VzWsYa2jWXPZznyhGNTZ1RfHTvNmYAgFMOr07GzoZKuvFV2F160aKFYfgICc4PmRvqIZlwSs8zRMwrF1Rv/FQfH0/sPxKHtGhzkRnCiaclkNBjbVNbGloQm32VJ2TLXnmG+7oigMR6Mc8fs5GQiQzGaLj5l0erq9Hlb7fHS7PVODx1mA4pn9I3KKwXCYgXCEoXC0DJIBPBYLrZOA3OJy4rFYKu7lWmw+07UUz7egXD5PKCETSCTI5vJsaFE/dC4lFF97zacXXbswm5V57td/dUEMkkLnr5YSikG4Upfmrx3/vHWlXkCuF4Ir9XQwFq7UAoqXmwQUa6gSFAOMyEFeDLxITsnRbG5mu287OklXlSt1OBPjmdHXSOZkbHoL1zbswGV0VOxfqV1RFA6GB3l54igKCvUmFzc0bcNhsCwIigEimSTPjBxmRA4D0G2v59rGdVhnzGhXA8VqrjAux3hm+ASjSXVm1Wuycm3TajrtXo3+1UNxQWOJGC+O9nMqEgDU69N6bwM7GqfBcY3dqkGdsT7gH2X/+CixzNRsbbfLw5aGZla4varR0hJCMUxdj/OKwmAkwslAgBOBAPFp9Y/1kkSn281Kr48VHo/2nuizZNgFat3moXCUoUiE4UiUiUSyrI/FYKDZ6aDF6aTF5aDJ4Sj9lvgcgOJKElAsJLT0mgnF6VSWWDiBr9FVEygG4Uo9lX+Fds5TV+oaukYLV+rZz3suu1ILKF5eElCsodmgOKPoGE+N89LES+TJ02xpZrt3O7kqjbPC6RTPjb9CNBvHrDNxdf1l1Jk9VUExqDA7nAzw7NibpPIZzDoj1zdeRJNFw3iDuaFY7aPwZrCPVyd6yKNg1Ru5rnE9Xfb6aX2qh2L1XuFQaJQXxk4XjbhWOeq5unElLpNlWv+FQ3FBo4kYL4700RMNAlMzx9sb26kzV9j7WwNgzSsKp8IB9o2P0hcJFdvtRiMb6xrZXKcxe1zlOWZr14qiKAoj0Rgngiogz5yJbXE4WOn1sdLjxVuYgT2LUDxTcibLSCTK0ORtLBYrM+6SgDq7jWaHg2anCsllZaAWkM/5BsVi+bSQkKrpUGzQm3nmx3sITcS58Y5tNLRV5xotXKmn8tHOv0L889WVepZyTcKVWrhSFySgeHlJQLGGCm+A/zr+rTIoLhpnyWO8MPEqefK0WJq4zLujaE41XbPNIMu5NLvHXiOQjqCXdFxVv40WS7Nm/+wcsBzNJHl6ZB8T6ai6z9i7mi3u7jIgqAzd5f/F43KMp4ePEEir5Y82uFq4qnEVJp2hIvzmK+wRnlnaSc5leWmslzcnhlEAvaTjsvp2Lq1rx6jTzwLFms2zQvRIIspLI/30RILF9tXuOnY0ttNkc8wr/kJnloNykgPjoxyaGCtZytzhdLO5oYlVHt/UPt9qjbAqaY5cFUVhIpHkVDDAqUCQsXi8pJvLbGaF18sKj4c2l6t8H/IS5zkfCM3l84zH4oxEYwxHooxEYsSm7aUuyKTX0+i00+R00Ohw0OS04zCWuz3PprNh8JWMx/nk7cJoS0hoKVWE4i/8AJPewrM/eZPRwRBGo54b7thGQ4un/CDhSg0IV2qt887LfGueuWr1F67Us5/3XHSllmNx7r/xLct2rF+KeMtZAoo1NB8ohlIwbjY3c5nvsuIe44LmWladyWf5zfhehmU/EnCxZxNrnF1l/eeCYoBsPsfz/iMcjw4D0GVr5JqGjZimOWVXA8UAmVyelyd6eDOo1jR2GMxc37SOdludZv/5QnFB44kEvxo5yUAiPBnfxFWNK1jratS82C8EigsaSUR5dXSAE+FAsa3b6WF7YzttdheSJNUcigvK5fOcCgXY7x8rmT026w2s99Wzoa6BJpujulrLlVRlrlE5RU8wyKlgkIFIpGj8BGDU6eh0u+n2eul0u3GZzcsCirUUS6UYicYYjcYYicQYi8U1HZ9tRiONDhWUGyZrCTtMlUFZQPHSxhISOluaDsU2i51sJlcEY4NRr84YzwRj4UotXKlnOa9wpZ49vnClLtVSQnEtVoUV4l0oK8MEFGtovlAMMCqP8/zEK+TJ02huZLt3e9Wu1Hklz6uBQ5yMqeC5xtHNNs8Gdf/ppOYDxaAOKkejg7wwfpQ8Ck6DlRsat1Bvdmv2n4ozO1QOJoI8O3qUSEYt/bPW2cQVDavL9xpXCcVKXl06dTzi5zdjPUWX6nqznasaV9Bp95Zc8BcDxQX5kwleHR3gaGi8yCNNVgeXNLSy2l2HXmPGv2Z7kIFwSuaQf4yD/rGSvcdes4X1dQ2s9zWULq9eYiie3p7O5egPh+kJBjkdCpGYtg8ZwGux0On20Ol20+5yqft2apznQqF4pvKKwkQ8wWg0xmgszlg0xkQ8oRnGYjDQ4LDTYLdRb7fT4LDhsVrRSZKA4iWOJSR0tjQTioG5wVi4UgtX6jnOK1ypZ48vXKmntJRQfN0V/2fR/iGgeoj86sXPXRDjvYBiDVUDxaCC8QuBV8kpOepMdez07SzWMZ6vAZeiKByMnGJf6DgALZYGrqjbVowzXygu5pSM8OzYPmJZGR0Sl/pWs9nVRZ5K+cwNlZl8lpf8PewPqft+LHojV9avZo2zqXhBXggUF5TN59gTGOI1fz/pfA6ADpuHqxpX0Gh1TuajGWZBe5BDqSSvjQ9yKDBW3J/qMJrYVtfCZl9z6UWuhlBcUF5R6I+EOTQxxslQoGRWs8XuZJ2vnjXeOuwGDROs2VSjvckoCmPxOD2hEH2hECOxWElXnSTR7HDQ4XLT4XbRbHNoLoWqNs9aQbGWsjl12fVYLM5oNIY/HieQSGqG1usk6mw26m026u2TN5u98j6iheYpoFhI6KxIC4phEox/uo/RgWA5GAtXauFKPY/zHuod5dfClbpif+FKrUpA8fKSgGINVQvFAKOpEC9OqOWaPEYPl/sux6w3V+VKDXA6NsrLgTfJKXlcBgfXNFyGw2CrGorzeYlULsNv/IfoTYwB0GrxcVX9RdgMZo0484fK0WSEZ0ePFvcat9u8XNOwFrfJtigoLiiZzfCKv599waHiUt7Vznp2NnThM2kbZC3GmCuRSbNvYoQ3J0ZIZNWZUaNOx0ZvE1vrmvFZbEsCxdOVzuU4EZzgyMQ4fdFwsV0C2p1u1nnrWe2p096/Mt9zL9KwK5XN0h+J0BcK0Rsury1s1OlodbrodLlod7mpt9lKVjvMN8+lhGKtONlcnolEgvF4HH9MvZ+IJ8hoLL0GsJuM1NlsKjDb1Xuf1YpBP8fe8HnmAwKKhYTOhCpBMahbh3Y/+WY5GAtX6sl24Uo923kBDvYKV+rZ4gtXarF8erlJQLGGFgLFGUVHKB3ixcCLpPNpHAYHV/iuwKh3avafzU06kA7x6/HXkfMpTDojV9RdTL253NBitjgFOC0sp345cJScksesM3Jl/WY6baXxqoFigExedah+PdBLTsmjl3Rc7O3kIndXsVZyST5VQHHhvJG0zIvjpzkaUaFeAta6GtlZ34XbZC3rX03+WiCSzec5Ehxnj38Iv5wotnc6PGyra6Hb6SuHvBpB8fRYsXSaY0E/x4J+RuKx4sM6SaLL6WGNt46Vbl9lQF5CF+vp/UOyTH8kTH84TH8kgjzNSAzArNfT5nTR5nLR7nSVQ/IygWLNLopCWE7hj8eZiCXwJxL44wnN+skFuS1mfDYbdVYrvklQLoHlKvIRUCwktPSaDYoVnUQ2kysHY+FKrUq4Us963sI5hCt15fjClVoYbS03CSjWUOEN8NTJf8LuLIWvTF77zV6A00gmyq8nXiKZk7HqLFxedwUuY/mbqBLMFmaEE1mZ58b3MpEOIwFbPetZ6yg3VKgEszPjB9Nxnh09wERahaz1rnZ2+NYWAbYSPM5V7ziUTvDc6An6E6qzs8to4erGNXTZS424Ks0gVzrvdHdrvxznxbFeTkZVgywdEhs9Texo6MRpNM8ap+rl1qgDfX8szJ6xYU5Nc6x2mcxsrW9mk68Jq8E4a/xZobgKOA3LMkcDExwN+JlIToG6TpLocs0DkOejGpiLKYqCP5GgPxxhIBxmMBolk8uV9DHr9bQ4nbS5XLQ6nTTZ7drLrZd8D+/CYTydzTKRSDIRTzCRSBKIq7A88wuB6XJZzPhsVryTkOy1qbfZ/mZyPM4nbxVQLCS0lCpC8ef/owyKCxA6fY+xcKWe3j55L1ypK2vyEOFKPXv8C9mVWo7Fuf8mAcXLRQKKNbQYKAZI5JL8xv8SkWwMo2Rkp28ndea6iv2na/oy6ZyS45WJQ5yKq4NBl62Vy7xbSmZi5wvFalueVyZOcSDcC4DHaOfaxk3Um90LhmJQB62TMT+/GTtBPKsaR61w1HNVw2qcRnU/w2KguKDRZJQXRvvojaugqpckNnmaubS+HafBWtZfja/ZPCsUT1c4JfOmf4QDE2OkctnJ8+pY56lnS30TzVZnBcfo2kDx9FgTyQTHAn6OBycIyMniwzpJosPpZpXHx0q3D4ep2j3Is5933v2nKa8ojMfjDIQjDEYimpBs0OlodjhoczppcTppdjgwGwzLGoo1H1YUkpksgcQUKAcSSQKJ5KywbDUa8FqteKxWvFYLHquFFT7VWE5AsZDQ0ms+UAzzMN8C4UotXKlnPbdwpZ49/oXqSi2geHlJQLGGFgvFAOl8mt9MvEIgHUSHjku9l9Jqba3Yv6CZe4fV5c99vBE8ioKC2+jkqrpLcBrtk3HmD8WF9sHEBM+NHyCZSyMhscXTxVb3Ks1lz/OB4qnnnOUVfy/7ggPF2sNbve1c7OvEgLE8CNVBcaH/YCLMi2O9DE6WcdIhsd7dyKV1HXjNthn9NcPMG4oLyuTyHA362esfZjw5VdfXZ7ayua6J9d5GbIZpz3EJoHi6JpIJjgcmOB6cYGLaUm+AJpuDlR4vqzw+6iy2uWvyLgEUz+xfgOTBSIShaJShaFQTGOusVpodDpodTlocDnxW68Lzr9i/tlBc8bBJWA4mkgSSSQLxJMGkCsvxCjWV//DyywQUCwmdIc0XikG4Upe3z/hduFLPeW7hSj17/AvRlVpA8fKSgGINzYTiwWQPNr0Dr6lh3lAMkM1neSmwh5GUOkhsdG1ktX01kiTNG4oLGkqEeGFiD6l8GqNkYLtvCx22lgVBMYCcS/Oi/wg98VEA3EYbV9VvpsnimdG/OmjNKxITqRi/GTvOUFKFVqveyGW+lax3NaOTZkJ/9VCs3isMJMK8Mt5XrHEMqiHXZfWdNFock/00w1QNxUw773Aiyn7/KMdCfrKKasakkyRWuXxs9DXS5fSiY5aRsgZQPL1/QE5yMjTByVCgZA8ygNNkZoXbwwq3l3anG6PGFx9nAorLmhSFYDLJ4CQgj0SjhDX265r0eprsdpodDprsDpodDuwzZ8KXKRTPFiedzRGSkwQTMsGkCst6Scdb1q0GxPJpIaEzoWqgGIQrdWm7RptwpZ7z3MKVevb4F5ortYDi5SUBxRqaDsWyKcye0K/RoeMiz5W4jdomDJUgNJ2X2B/ZT09cvQh22jrZ6t6KgjZcV4LibF5PIivz4sQe/Gl1+fAqewcXeTahl7RmeOfnVn06PsaL/sMkc+rM1UZXJ5d4V2OcrLW8ECgGdfDqifl5yX+KcEZd6us12bm8bhWd0/YbLxSKp2s4EeFV/wA9sYliW4fdw6W+DtptHs2ZxoVC8XSlclmOBP0cnBhlNDkFozaDkQ2eRjb4Gqmf+UFrtpPUAE7jmTQ94SCnQgF6I2FyypSDsl7S0eF0scLtpdvtnaqFfBagGMrHh3g6zUgsxnA0xkgsymg8XlKmqiCHyUST3UGT3U6Tw0GjzV7dnuplAMVzaSmhuBaOlBeSG6XQ+atqoRiEK/VUu3Y6wpV67nMLV+rZ419IrtQCipeXBBRraDoUWxwm9odexJ8eRkJivfNKGi2dZcfM5kqtKAqn4qc4EDkAQJ2pjku9OzHpyksjzQbFAHklz4HwcQ5HTwLgNjrZ6bsYl7HU5Xq+UAyQymV4eeIYJ2LqsiSnwcqV9RtptdYtGIqnzpfnYGiI1yZOk8qry2XbrV521q+i3uysCRQX2v1ynNcn+jkWGSvyR53ZziW+dta4GtBPm6WuBRSrcdT78WScg4FRjgbHSeamlgU3Wh1s9Day1tMwtbx6CaF4ujL5HAPRMD3hID3hINEZS3a9ZgtdLg9dLi/tTlf5LPIZhuKZ/XP5PIFkkpF4jJFYjNFYjIlkUvMQt9lMk91Bo91Og81Og82G1ai9ZP9Ch+IbLrkfg35xtQuzOZln33joghgkhc5fLQSKhSt1oV07vHClnt+5hSt15fgXkiu1gOLlJQHFGiq8AX556h+xO63klTz7wq8xLA8AsMF1GW3WFSXH5CrUI87kp9qH5VFeDrxBVsli09u43LejzJm6UpyZsDyc9PO8fz9yPo1e0nOpdwMr7W3FWdFqoLjQ3p/w85vxw8Sz6jLW1Y5mtvvWYdWXGzdVKrGUy2vHT+YyvDbRx/7gIPlJOljjbOQy38qy8kpQ/bLn6RAdTsvsmRjiYGikWGvWYTCxzdfGZm8zZr2hZm7VM5XL5+mJhDgYGKMnHCw+VwnodnlZ721gldOn6XxYLZBXA9eKojCRTNITCnI6HGI4Hi3WgAbVtKzN4aLT7aHT5aHBakOardZENflU7F99/HQux1gsxlg8zmgszlgsprnsGsBpMqmQPHmrt9lwmEyVayefJYMvLcnxOJ96i4BiIaGlVBGK/79/L4fiilCptgtX6tkB6EJ3pZ6ea6VzC1fq2eNfCK7UclxA8XKSgGINzYRiUC/AByN76E+eBmCNYytd9rXFY+YDxQDhTJQXJl4hnktgkAxc5r2UZsvUN6jzhWKAZC7F8/79jMjqsuEuWwuX+TZi0hkXBMWgmmW9HjjJwbA6EJh1Rrb71rDG0VpyMa0WigtvsnA6ycv+Ho5HxwHVJGuDu5VLfF3YDVMz54uB4oLkXJZ9gRH2BgZJZDMAGCUdGz1NXORtKzPlWsh5K0lRJJLZDEeC4xwOjJcsrzbodKx2+VjvbaDT4ZnaL1Mx2OKheGacVDZLfzTM6XCI3kj5LLLVYKDD6aHT5abD6Z5aaj2bzgAUa0nOZFVIjscYj8cZj8UrgrLFYKDeZlNB2WajbrKesEGnu2CgWCyfFhJStRgoBuFKrd0+7ecL2JV6Zq6Vzi1cqWePf767UgsoXl4SUKwhLSgG9QJ8OHqQ3sQxALps61jt2DJpnDU/KAZI5VK8EHidibQKsxuc61nrWDNrnMrLquFwpId94eMoKNj0FnbWbabBXP6NLMx/BnlcDvOb8cPFusaNZjdX1K2nzqz+QywUiqfiR3nJ30NfsbySjk3uNrZ5O7EZTDWB4kL/bD7P0cgYeyYGmUhNOTV32D1c5G1lhaOuOHtYSyieroCc4EhwnCNBP+G0XGw36/WsctWxxlNHh8OjwllZsNpDcWmuCkE5yelIiP5ImIFouDjDXpDTZKbd6aLd4aLd6cZlMpcPrGcJirWUymYZj8UZTyQYi8fxx+MEkknNEBLgsVios9qos1kn7224zWbtOsoLyEftvzygWBhtCQmpWiwUg3ClLm+f8fsF6kqtlWulc59PrtSrmnzcvHmNcKWeptnAWEDx8pKAYg1VgmKAdF7H6cQRTsbU/cFN5g42urcD2rVhtaBYjaOwL7yf04neyThNXOq9GL2kXW+3EhQX9vz6U0FenNhPLKtC3yp7Jxd51hcNs6b6z38GOa/kORAa4I3gSbJKDglY62zjEu9qzPry/dAwfyguqC8W5pWJU4zKEUCF443uVrZ6OktmjotxFrEHWVEU+uMh9gaG6IkFiu0Og5nNnmY2epo1zznbeSup8vJshZF4jCPBcY6F/cUZbACTTs9Kl4/Vnjq6nZ6pEllLDMUzlcvnGY5H6QuH6Y+GGY3HisvAC3IaTbQ53bQ5XLQ5nXjN1gUst146KNbqn53co+yPq7Dsj8eZSFauJayTJLwWCz6rFZ/VRp3Vis9qxW2xLHBmWUCxkNByUi2gGIQrdWm7RtsF6EpdKddK5z6fXKnLwLgG8c9XV+qlhOLRGqwKK8RrukBWhgko1tBsUJyZnMkdSp7mcOS1ydrBdWx2X4NJV77EtBIUZyf/K0/He9kX3k+ePFa9lUs9O/CayvcNzQXFoJaA2hs6xvFYHwB2vY0ddRfRYPZN61/dsup8XiKelXklcKxYvsmkM7DVs5L1rs4S8yqoHopzedWIrD8R4LXAacamwfE6VwvbPJ04jVOva62MucKpFPtDwxwMDSNPGmNJQLejjs2eFjrt3pK9p7WCYvXBQs4KQ/EIx8MTnAhPEMtMLWE26HR0OT2sctWx0unDYtAwjVoiKJ4ZJ53LMRyPMhANMxCNaEKy1WCg1e6izeGi1eGkwWYve2/UKp/F9p9+VkVRSGQy+BMJJhJJJpIJJhIJAslk2Wz59OPdFgteixWf1YLPYsVrseK1Wmd3wRZQLCS0rFQrKAbhSj3Vrp3OheZKPVuuF4IrdQkY1yj++eRKvbaujltXr11SKL5x6ycxVJjAqkbZXIpn3vzCBTHeCyjW0HygGCCQHmNf6AWySgaLzs5Wz3XYDKVvmLmgGCCUCfNq4FXiuQQSEptcW+i2rSxZ3jIfKC5oRJ7g5Yn9JHLqMt21jm42u9dh0OkXBMVTcYO8PHGUiXQUAJfRxnbfOtqt9VMGXwuA4mIfRWEgEeT1wGlG5DCg7jle62pmm7cLt9FaU7dqUGcPT0THORAcZigZKT7uNJjZ5Glmg7sZh9G8JFBc2l9hKBHlRHiC46EJopmpPbES0O5wq4Ds8uEyzVVKqVJ7bSA0U4TkCIOxCCPxWEnpJ1ChvtnmoMXupNXhpNnuxDoT7JcBFFfqrygK0XSaQFIF5IlkkkAiSUBOks7lKp7SajDgsVjxWix4LRY8FiseiwWPxYJBo3RaJQkoFhJaetUSioUrdaFdO/yF5ko9e66VzyFcqSvHP19cqZ/t6eG2NWtxmEwCipeZBBRrqPCGeq7nazhmLp+esec3lo3wWvAFkrkEBsnINs9V+ExTg0hmnnuEM/kMrwbfZDCpDhZt1lYu9mybVi+4wp7iCu1yLs8bwSOcjKmO2Q6DjZ11m6g3aQ9wlUsv6Wb8rnAsOswrgZPIk7WN26x17Khbg8/kqAiDFaFbo7+iKAwmw7w60ctgIgSo15hVzga2ebqotzg1jtEMXxmiNa5aE3KC/YERDofHSE2bPe50eNnoaWKFo658z2+VLtaz5TT1uMJ4Ms7xcICToQB+OVHyeL3Fxkq3j5VOH802R9ne3sqGXZXaFwen2XyesUScoUiUwViEoVi0+PpNl9dipcXuoHkSkusttspO0IvIZ+7+C3++iqIQz2QIJpMEkkmCicl7WSY+w6xsppwmUxGQ3VYrHrMZdwGYZ7yv5HicTy3jfUYCioXOBxWh+K+fwGaZYbxYaaWLcKWezH9hM8gXjCs1zPJaz35u4Uo9e/xz3ZVaUZTi6y3H43xql1g+vVwkoFhD1UAxQCon83roJcKZwORM72W0WruB+UMxqP8oR2OnORA+hIKCw2Bnu/cyPCZ31VBc6D+UHOeViYPFWeOV9g62etZj0hln9J8fFBeUzOXYG+zhYLiPPAoSsMbZysWeVdgN5cvIq4FimOKQoUSYNwJ99Man9gB32Hxs83bSavUULyy1gOJC/2w+x4nIBPuDwwwlpmaPLXoD69yNbHA30WCxq+deAiieqaAsczIc4GQ4wFA8UsKANoORFU4v3S4vnQ4PFoPhjEPxzDgF466hWJShWJTheJSgXF5f2KDT0WRz0Gx30Gx30mizaxt4LTSfiv2XJn46lyMky4SSSYJJmWAyqf4uy7POLgM4JoH5tzdsQJIkAcVCQmdASwHFIFyptdun/XwhuFLDgqEYhCv1XPHPdVfqgpYSioXRVvVa9lDc3d1Nb29vWfuf/umf8vWvf72s/dFHH+X9739/SZvZbEaW5bK+lVQtFAOk83Ag/AqjKXVmttO2hrWOi8ihsReUWdykFR0TqQCvBl8jmZORkNjoWs9K+1pNWJgLigEy+Sx7g1N7jS06E9s8G+i0TZVZqhaKC/0jmQSvBk5wOj4GqHuBNzg7uMjTjWVafeOFQnFBfjnGG4F+TkTHio/Vmx1s8bSz2tGErsIHmIVA8XQFU0kOh0Y5FBolnp2aCfSarKx3N7LW1VhVneXZcppP/2Q2w+lIkFORAKcjIdL5KdiSgGabky6nhy6nlyabo3Qm9gxBsZaSmQzD8Sgj8RgjsSgjiZgmKFr0BhrtDppsdppsDhptdpxzgfIygeKKhysKyWyWUFIF5pAsE5ZlQqkU4WnA7DSZuPeSSwAxUyx04elsjvW1hmIQrtTl7TN+P99dqWFRUAznlyu1JhgvMv657koNAoqXm5Y9FI+Pj5Ob9uH5wIED3HzzzTz77LNcf/31Zf0fffRR/uzP/oyjR48W2yRJoqmpwr4VDS0EinOKHkVROBE/SE/8MABeYz0bXFdh1pdD02xQDJDKpdkT2suwrA4YdaY6LvZcgs1g1+xfnk95+5gc4OWJg0SzcQCazHVc4t2Ey+hYMBQXNCqHeDVwglE5BIBR0rPZ3cUmdxcmnWHRUFxQMJXizWA/RyPDZCf3sVr1Rja4Wtnobitzj14sFBeUyyv0xYIcCo/SE50gN+3fptnqZJ2rkTWuBmwG02ScCk9glnNU2z+XzzMYi9ATCXI6GiSQKp2NtegNdDjcar1hhxu30aINl2cAisu6KgqBZJKReJTheIyxRAx/MkFe44Uz6w002uw02Ow02uw0Wu14LNYp4F/mUDxbHEVRSGWzxdnkLo8HEFAsdOHpbI71SwHFIFypS9s12s5nV2pYNBTD+eBKPQsY1yD+ue5KLaB4eWnZQ/FM3XfffTz55JMcP35c8wP+o48+yn333UcoFFrwORYKxQWNyoMciLxCTsli0lnZ7L4Kt7F06dFcUAzqBb030c/+8H6ySg6DZGCL+yLarR3F514NFAOkcwpHoz0cipwgp+RVIyvnCtY515SVb4L5Q3Eh3/5EgNeDJwhMmnGZdUY2u7tY5+yqEL86KC4Yc8m5DIfCQxwMDxLPqqZUOiRWOhrZ7GmjyeKezKk2UDz9vySVy3Iy6udoeJyBeKiYqwS02z2scTaw0lmPVa+9SqBmpl0zXqRoOsXpaJDeaIi+WIjUjJlYl9FMp9OjgrLDXQT4swHFWnGy+TwTyQSjCRWSR+NxJmRtUDZIOuqsNhpsdhqsNhqsduqtdtUpcs7zLh8oBu1xVUCx0IWuMznWF6A4m8kRjyZx+xw1gWIQrtRT7drpnLeu1FATKAbhSj1X/HPZlVpA8fLSHDVTlpfS6TTf+c53uPfee2ddThmLxejq6qKjo4M77riDgwcPzho3lUoRiURKbotRk6WNy303Ydc7SeeT7Ak+w2DiONV+/yBJEt32Tm5svB6v0UdWybIn9AavBV8llUvNHUBDeknPRtdqbm2+hlZLI3kUjkRP8bOR3fQlBqvOcWa+HbZ67mjdyfUNW3AbbaTyGV4PnuA/Bp5jf/gUmbx2TdhqZdEbucTXxXu6L+fm5k00W9zkUTgRG+VHA2/wH/2vcSQyRDY/+17OhcisN7DR08w7urZw7+qdXNO0kiaLEwXoj4d4ZuQ4/3L8JX7cv59DoRHkXGbOmLWQ02RmS10zt3ev54827eT3Vm/hiuZO2uwudJJEJJPiQGCU/+47xj8depVvH93Ds4OnOBH2k8yemRxnk0Gno8nu4KKGZnZ1reY9G7fyp9t28u4NF7GraxVbG5ppsTsx6HRklTyjiRgH/KM829/D48cO8A9vvsy/HHid/zx5mBeG+jga8DNRYfZZ6Ozo61//Ot3d3VgsFnbu3Mkrr7xSse8PfvADLrvsMjweD3a7nW3btvHtb3+7pM+DDz7I+vXrsdvteL1edu3axcsvv1zSJxAI8J73vAeXy4XH4+EDH/gAsVhsSZ6fUG10Nsb6bCbH7h+/wc++9zLB8cV9Bpgug1HP9b+1laZ2L9lMjmd+vJfx4VDN4uv1Oq69eTNtXXXkcnl2P7Wf4YHA3AfOUzqdxFXXrGPFygYUReFXzx6hr9dfs/iSJHH5jpWsW9sMwG9eOM6Jk2M1jX/plk4u2tAGwEtv9HDo+HDN4gNsXdfG9o2qC/XrhwfYc2SgpvE3djdz9ZZuAPadHOblw32L+qw2U2vb6rnhotVIwOH+MZ472FPT+Csafdxy0Rp0ksTJ0QBPHzhBPl+7+O1eN7+1ZR0GnY7eQIifHjhGrkJJxYWo2eXkjkkQHopE+a8DR8jM4RMidG7qnJopfvzxx3n3u99NX18fra2tmn1efPFFjh8/zkUXXUQ4HOZv/uZveO655zh48CDt7doOgQ8++CCf+cxnymP1PIzDZSWRjWDSWTDoTGQrzBRrGWpl8xn2hvcwmlK/oWy1dLHRdQl6SV9xhreSMVcmr+No7DiHI0dRUDDrTGzzXESzpU2z/3xLLw0kxngteIRYVl1622D2cql3I16Tc9Y4lWZ4p/fPKwonYyO8EewhklHjW3RGtni62OBqx6gz1MStGtRvg8flKPuCgxyLjhVByKTTs87VzCZPK17T1NLzmpV2mvZzKJXkWNjPsYgfvxwvtuskiQ67h9WuelY6fVin7bUuDTb3LHWlc88VJ53LMRCL0BcN0RcNlzlag+pq3e5w0+5w0eZwY6s0013xvJUemN9M99xxVOUVhXBKZjwRZzyRwJ9IMJ6Il9R5ni69JOG1WKmz2qZuFhtu84y9yks8I1yN5Fic+2+4Zdl+e7yQWN///vd573vfyze+8Q127tzJww8/zBNPPMHRo0dpbCx3n929ezfBYJD169djMpl48skn+fM//3N+8pOfcMsttwDw2GOP0djYyMqVK0kmk3zlK1/hiSee4MSJEzQ0qE77t912G8PDw3zzm98kk8nw/ve/n+3bt/PYY48t6jUQWjqd6bH+nz73OHrJyDM/eJ2JkTAmi5Gbf3c73gaN97ZwpRau1BX6C1fq2eMLV+ryDwdyPM6nbhYzxctF5xQU33LLLZhMJv7rv/5r3sdkMhk2bNjA3XffzWc/+1nNPqlUilRqauY1EonQ0dHBiz0PY7LDm6Gn0EtGNrqvx6Bza59nFpg9nTjGsdh+ABwGN9vcl2PWVxenAOPBdIjXg3uIZNXlyW3WNra4tmCeUYusmnrEOSXHwXAvByMnySnqwuJVjg62uFdj1JXvh4b5QfFU3zzHo6PsCfYQnYRvs87IJncHG5ydmDXgazF7kJPZDIcjIxwMDRHJTJmutFo9bHS3sMJRj14qX8o9a/wql3kH5CTHI36OhceZSE0BqAS02txFQHYapzl1LyEUz+yfzGYYiEXoj4bpj4UJaLhD+8xW2hwu2u1uWuxOXEYVIs82FJf3V+Mnsxn8iQQTyQT+ZBx/Uv05U+EbY4NOh89ixWex4bNa8Zmt+CxW3GaL5kA6/3wWfmhBSwnFo8OLL9MQiURoaqmuRMPOnTvZvn07X/va1wDI5/N0dHTwkY98hE996lPzinHJJZdw++23V7yWF57j008/zU033cThw4fZuHEjr776KpdddhkATz31FG9961sZGBioCFxCZ1dneqz/p889jtViI53KlIDxrt/Zjq9xxvtbuFILV+pZ+gtX6tnjC1fqUgkoXl46Z6C4t7eXlStX8oMf/IA77rijqmN/93d/F4PBwHe/+9159S+8AV7seRjJJnMo/CzpfBKDZGaN8zpcpuayY+aC2YnUKPsir5DOp9CjZ63zYlos5RebueKACrFHIsc4FjuBgoJJZ2KzazPt1vZpbtLV7TXOKRLxbJI9wSP0J0cBMEh61jlXss65AsOM/cDVQHGhf17JcyI2wpvBHiKTcGyQ9Kx3trPJ3VlSyqkWxlyKotAbD3IwNERvfKLYx6QzsMbZxDpXMw1mZ8nfoFZQPL1/IJXgRMTPycgEY3Lpss16i50VDh8rHD6aLC7Nb2WXAopn9o9n0gzEwgzGIwzGIpozyXaDkRa7ixabkxa7k0aro7S+7lmGYs2HFIVIOjUJygkCyQQTySQBOVFiljYzW7fZgtdixTt577GowGwzGOcuGbXMoXjX2o9hmPElWrXK5lI8fezL9Pf3l+RnNpsxm8tjp9NpbDYb//7v/86dd95ZbL/nnnsIhUL8+Mc/nvV8iqLwzDPP8Pa3v50f/ehH3HzzzZrn+Pu//3s+97nPceLECerr6/mXf/kX/vzP/5xgMDiVezaLxWLhiSee4B3veMcCnr3QUupsjPUFKAbKwLhsxli4UgtX6lnyB+FKPVd84Uo9JQHFy0vnzJ7iRx55hMbGRm6//faqjsvlcuzfv5+WlvJlMvOR3eDlIs+t2A0+skqKw5FfMJI8UvV+izpzE1f4duEzNpAjx+HoaxyIvEQmr73kczbpJT2b3Bu4ruFqXAYX6XyaN0Jv8FLgJeLZ+NwBKshusHJ1w8Xc1LgDn8lFVslxMHKcnw7/ipOxPvLK4vZo6CQda52tvLPjCq5v3IzX5CCr5DgQ6eWJ/t/w6/GDBNO12+snSRKd9jpua9vCe1ZczqW+LhwGM+l8loPhQX7Q/zpP9L3Km8E+Etnq/w7zlc9sY0dDJ3evupj3r9nO1U0raLWqFxa/HOdVfz+Pn36T/3v8JZ4eOsqJiJ9UrjZ7r+cru9HEOm8DN7av4vfXX8wfb97B27vXc0lDK802Bzok4tkMJ8IT/Hr4NI+f2M8/HniJ7x1/k92DpzgSHCeckmu6D6kWkiQJt9nCSo+PHS3t3LpyLe/ZtJUPXXI592y+mN9atY4r2zpZ72ugyWbHqNOhAKGUTE84yBtjw/yy7xT/cewg/7zvNf5h78v826E3+cmpozw/2MtB/xhDsQjxTHrZPfczoY6ODtxud/H20EMPafbz+/3kcrkyZ+CmpiZGRkYqxg+HwzgcDkwmE7fffjtf/epXy4D4ySefxOFwYLFY+MpXvsIvfvEL6uvVD+kjIyNlS7MNBgM+n2/W8wqdPZ2tsb4gk9nIjb99KfXNbtJyhl888WrN9xjfcPsS7zHetZH2rvql22N89RLvMd5+bu8x3rZ2afcYb+pu5potK4Al2mPcWs+NW5Zwj3GDj1sn9xifGJ1Ymj3Gm5dwj7GzdI/xf4o9xiV67rnneNvb3kZrq1r29Uc/+lHJ45Ikad6+9KUvVYz54IMPlvVfv359SR9ZlvnQhz5EXV0dDoeDd77znYyOjlaVu/Ya0mWmfD7PI488wj333IPBUJrye9/7Xtra2oofxv7qr/6Kyy+/nNWrVxMKhfjSl75Eb28vH/zgBxd8frPexhbPzZyIvow/dZre+CsksgG6HTvRSfNwu52URW/lMu+19CSOciJ2kLFUP5HMBJtcO/GYGqrOy2fycl3DdZyIneBo9ChjqTGeGXuGtc61rLCvRV9FbtPVaPHxlqYr6EuM8GboOPFcgteDBzgW7WGLey1t1mYqf908t3SSjlWOZlbam+hPBNgfPs2IHOR4bIjjsSHarHVscHbRaq2fe1ZunnIaLeyoX8Fldd0MJoIcCY/QE/cTSMd50X+Sl/ynaLd5WeNsptvegEG3sNduLrlMFi6pa+eSunYS2TS9sSA9sQB9sSDJXIZD4VEOhUfRIdFic9Ht8NFp81FnttXstZiPrAYjq9x1rHLXAZDN5xhJxBhORBmKRxlORElmM4wkYowkYoD6ocKqN9Bsc9Jkc9BkddBkc065XC8j6Sb3GXst07YHKOosQiyTJiTLBFJJQnKSoJwkmJKJpGQy+TzjyTjjyfIvn4w6HW6zBY/ZgttkmfrZbMFhMqGvNMN0hvVvv7q/RsuntWeKaymn08nevXuJxWL88pe/5GMf+xgrV64sKdFzww03sHfvXvx+P//8z//M7/3e7/Hyyy9r7lMWWt4622N9QSazkRvfeRm//I/XmBgJ84snXq28x3gBKphvFVypn/nxXu2l1AuUXq/jmps3FV2pdz+1v/Ie4wWoYL4F0HNqnF89e6TyHuMFqGC+BXD02Ai/eeE4gOYe44XGv3RLJwD7Dg/y0htqySOtPcYL1ba1KnS/eqif1w+rUFxxj/ECtLGrCQWF3+w/zb6T6vhbNmO8CK1trUeR4Nl9Jzjcr34pUTZjvAitaFDNt3627zgnRidQUKZcqWuggvnWk/uPFsG40h7jhagAxj8+cKQIxkVX6mUiOZHGZFj8hI+cqC5GPB5n69at3Hvvvfz2b/922ePDw6VfQv33f/83H/jAB3jnO985a9xNmzbx9NNPF3+fOUZ89KMf5Sc/+QlPPPEEbrebD3/4w/z2b/82zz///LxzPyeg+Omnn6avr49777237LG+vj50097kwWCQP/iDP2BkZASv18ull17KCy+8wMaNGxeVg14ysNZ5JVa9j/7EG4ynTpDMhVnjug6TzjZ3gElJksRK+3rcxiYOhl8imY/zemg33bb1rLBvBKr7h1JnX9fSYm1hX2gf/rSfI9Ej9CcG2OzeSoN5YYOIJEl02VtosbZwMtbH4cgJotk4L0zswWt0sdG1jmZLw6IukAW36g5bPWNyiAORPnrjowwmJxhMTuA22tno6mKlvbVmkKqaXvlot9WRymU4ER3jaHSEMTlCfyJAfyKAUdKzwtHAGmczLVbvVD3cGstmMLHB08QGTxM5Jc9QPEJPLEBPdIJwRmYwEWYwEeZ5enAYTHTafXTZvbTbPVgqmGAtlQw6/aQJlxsFFR7DaZmRRLQIy+PJOMlclp5okJ7o1HJVp9FMs81Bo9VJk9VBo9WOxXBm85+vJEnCaTLjNJnpoHTffzafJ5KWCckyoZRMUE4SSsmEUkmi6TSZfB7/5DLtsrio7uDuSWB2mc3Fe5fJPL9l2TWSxWbCYlvcFxXprHq8y+WaF2DX19ej1+vLvrUdHR2lubl8O0pBOp2O1atXA7Bt2zYOHz7MQw89VALFdrud1atXs3r1ai6//HLWrFnDt771Le6//36am5sZGyudZcpmswQCgVnPK3R2tBzG+oJMZiM3aYFxo6cm8SuCcVttwLXgSl0Gxh1LDMYrqv+CX0uVwHjV6vnXoZ4rvhYYb1hbOzDeum6Jwbh7cjZ9qcC4Tf2SY8nAuHEKjE+OBpA4oS6l1i8tGOtqBcauUjD+rwNH1KXUhuUBxu+5+q8XvVUK1O1SQFllnkrbpW677TZuu+22ivFmjr0//vGPueGGG1i5cuWseRgMhorjdjgc5lvf+haPPfYYN954I6CuOtqwYQMvvfQSl19++ayxCzpn9hSfSRXWz792+m9xuEqNprKKnmB6iCOR35BTMhglC+tdV2M3al9IM4r29w5ZRUc2n+FQdC9Dch8AToObTe6dOAzlJly5inuNS+sa9yeHeDN0CDmvvonbrW1scW/CordMxqlur24hfjqf4Uikl8OR02QVdZlIvdnDVvdaGi1Tg2y18WfWQY5mkhwI93MkMkRm8jwWnZF1rjbWu9qx6ysYf1WYua583tL2UDrB0cgoxyJjJeZcNr2J1c4GVjkbNff8VutirR6k3Tz9OYRSSU7HgpyOBumPh8lNW7ouAU1WJ512D512L802Z8ks5Hz2Oc8nn2r3LBdmUUfi6uzxaCJGMFVu4AXgMplptDposqmQ3Gi1YzMW6iZXm0+lB6rcy1x1/Cll83kiqRThlArM4UlwDqdSRFJyxf3LBeklHV6Lhfds3IokSeqe4uvOL/fpnTt3smPHDr761a8C6qxgZ2cnH/7wh+dttHXvvfdy6tQpdu/eXbHPqlWr+P3f/30efPDBotHWa6+9xqWXXgrAz3/+c2699VZhtCU0taf4r75X3FNc1OQH5zn3GBckXKmFK3WFY4Qr9ezxL2RXajke55NvuXlp/ENW3VczKH765MNl7Q888AAPPvjgrMdKksQPf/jDEi+R6RodHaW9vZ1//dd/5d3vfnfFOA8++CBf+tKXcLvdWCwWrrjiCh566CE6O9Uvtp555hluuukmgsEgHo+neFxXVxf33XcfH/3oR+d6mmq+AorLNRcUAySzEQ5HniORCwMSXfZLaLZs0DDOqgzFBY3IAxyM7CGjpJHQsdqxmU7b2pJY84Hi4jnzGfaFj3Eqrn7zaZAMrHeuY5VjBUqFxQFzQXFBci7NoUgPx6J9RVBrttSx2b2aBrN30VBcUDKX41hkkIORfmJZFVIlJLpsDWxwddBs8ZYaZC0SiovteRiRwxyNjHIyOk5qWl1lp9HCamcjqx0N1JkdqhPzEkHxdGVzOQYSEU5Hg/TFQwRSpTORRp2eNpubDruHDrubOrO9gmHX0kKxVv9ULstYPF6E5LFknHBa1uip7mtutNppsNhpsKo3t8lSnKlfzlA86+GKQjyTJiynCadkwimZSDpFJJUikpaJptWlSS6TmXsvUsHtfITi73//+9xzzz1885vfZMeOHTz88MM8/vjjHDlyhKamprLlsQ899BCXXXYZq1atIpVK8dOf/pRPfepT/OM//iMf/OAHicfj/PVf/zVvf/vbaWlpwe/38/Wvf53HHnuM119/nU2bNgHqt9ajo6N84xvfKJZkuuyyy0RJJqF5QTGUg7FwpRau1HPFn3mMcKWePf6F6kq9lFBci0oThXhNLQ3zNtacrrmg+Itf/CKf//znGRoawmKxaPYBdYl1LBZj3bp1DA8P85nPfIbBwUEOHDiA0+nkscce4/3vf39JdQGAHTt2cMMNN/CFL3xhXs/1nFg+vRxlNbjY6r2VE9GXGU+dpjf+OpHMKKscV2LQVffNTLOlHa+xngOR1xlPj3A8to+x1CAbXZdhN1T/hjbqjGz1bKHL1sHe0D6CmRAHIgfpiZ9mo2sTzZaWBS9/sehNXOJdx1pHN4cipzgZ62dEnmBEnqDR7GW9azVN5sXvBTbpDGz2dLHR3UFfws+hcD/DcpDTiTFOJ8ZwG22sd7azytGCpVLd3wVIkiRarB5arB6uaVxDfzzA8egYPTE/0YzMnkAfewJ9uIwWVjoaWGFvoLGCa3StZNDp6XZ46XZ4AYhmUvTFQvTGQvTHgyRzWU7HApyOqWYqZp2BNrubdpubtklIXqol4HPJrDfQ4fTQ4fQU2+RslrGkCshjiRijyRihlEw8k6Ynk6YnMrX02iDpqLPYqLcU6gvbqbfYzuhy48VKkiQcJjMOo4U2Z/n/cy6fJ5pOkz7D5mpnWnfddRfj4+N8+tOfZmRkhG3btvHUU08VzbdmLo+Nx+P86Z/+KQMDA1itVtavX893vvMd7rrrLgD0ej1HjhzhX//1X/H7/dTV1bF9+3Z+/etfF4EY4N/+7d/48Ic/zE033YROp+Od73wnf//3f39mn7zQOa2C+VYBjJ/+99rvMb7h9q1FMF6KPcbX7tpYBOMl2WN89dRS6ud2L8Ee4+2TS6mPL/Ee4yNDvLSnB6Rza4/xpsJS6gOn2XdqGKTa7zEGeGb/CY4MjCFJtd9jfOtFa3hqco8xUHnGeAEqmG89eeDM7TGuOGN8hlSLrVJQ/XapavQv//IvvOc975kViIGS5dgXXXQRO3fupKuri8cff5wPfOADNctHzBRraD4zxQUpisKwfIye2Bso5DHp7KxxXoPTqO6rmc9M8fRYfclejsXeJKdk0aGj276Bbvt6FLT3YGrFmd6uKAq9iX4ORQ6TmlxSXWeqY5NrCx6Tp9h/vjPFM/vHsgkOhU/REx8kPzml5jO52eBcRau1qXjBrHamOKfRP5COcTg8wInYcHEJt17S0WVrZI2zjWaLr+wCXe1McaXZ1HQ+R298ghPRMfrigZLlzDa9iRWOBlY6GmixutFJuprOFFecyVUkFEVhXI7THw/RnwgxlIiQyZe6IJp1BlpsLlptbtpsbhosjlLTpzNQ8mkupXNZ/HKCsUS8aGTllxMlr/N0WfQG6iw29Wa24Zv82VayV3l5zBRPxZn/4H4+zhQLCS03zXemuKB0KsOzP3gdv9ZS6kWWaoLJGeOf7mN0IFg+Y7zIUk0wOWP8y8MM9PrLZ4wXMVNc0KxLqRdZqgkmZ4xfO8XRY+Uzxost1VSI//qBfvYdHgRKl1Ivdqa4oFlnjBdZqgngUO8ov96vrhIsK9e0iJnWgmYt11SD+LPOGNcg/mwzxost1QRzzBif4Zni5VCSabaZ4l//+tdce+217N27l61bt1ad1/bt29m1axcPPfSQWD69lKoGigsKZ4Ici/yaVD4KSHTYttJq3UQW7W9pKsIsOuRcgsOR15lIqxd+u97FOtcO3Mbyb13nguKCMvksx2PHOR49SR4VNNqtHax3bsBmsC0YigtKZGUOR3s4GesvgozL4GC9ayUdtlakCgZi1UAxqCCYzmc5FRvmSHSAwLQSTk6DlTXONlY7WrFN1jyuFRRP/yfJ5LP0xQOcivnpjU+UQKhZZ6DTXkenrZ4Oex0mncaXIjWE4pnKKXnGkjEGEmEG4iGGk9EySDZIOpqsTlqsLlpsblosLsz68jzPJBRrBcpPmnn5JwHZLyfwJ+OEKiy/BtX92mex4TVb8ZlVUPaZbTiMpqmBW0CxgGIhIaqHYoBMOls03yoB4xpAMUAmly+ab5WAcQ2gGCCrUDTfKgHjGkAxqNft53+tAcY1gGK1XeGlV8rBuBZQDOrl/vX9fWVgXCsohlnAuAZQDHCwd4Tf7D8NzADjGkAlwNFhP8/u0wDjGsU/NR7gZ5NgvKrJN+VKXaP4A6EwT+4vB+NaQDGUg3HRlVpAcYne9773ceDAAV577bWqc4rFYnR2dvLggw/yP//n/yQcDtPQ0MB3v/vdoov10aNHWb9+PS+++KIw2lqMFgLFqnFWmp7Yy0ykTwPgNDTS5bgOs96h2V8zDlMzvKNyP0eje8ko6gxvq3U1q+wXYdCZ5o5ToT2WlTkcOcRAUl2+o0PHCvtKVjnWYtJY9j1fKC4ons1wLHqak7FeMoq6HNSqt7Da0c1KeydGXemM90KgeOpnhYl0lGPRQU7GRornk4A2az2rHK20WxvRazhXLwaKpyubUxhIBumJjXM65kfOZ4qP6ZBotXrpstfTaa/DabTOGqwWUDyze15RGJdjDCUiDMbDDCXDyBrLdOvMNlqsbpqtLpqtTjxGa+UPW2cIiis1Z/I5gnKSCTkxdUsliKRT2geilkvymKwqMJuseM1TN+NczuYCipc0lpDQ2dJCoFiSJNKpTDkYV3KlrhKKFZ1ENpMrB+NKrtRVw2hnMssAAE5KSURBVKBOnTGeCcaVXKmrhGKkyRnjmWBcwZW6WihGNzljPAOMK7pSVwnFSNLUHuNpYFzRlXoBUKxI8OZRDTCuERQrEhw6rQHGFb+YqDK+Do4NaoBxreJL0DM2BcbFGeMKrtRVf2EhTc4YzwBjnaHSF0+V86zUfySiMWOs4Up9PkJxLBbjxIkTAFx88cV8+ctf5oYbbsDn8xWNsSKRCC0tLfzt3/4tf/zHf1wW46abbuId73gHH/7whwH4i7/4C972trfR1dXF0NAQDzzwAHv37uXQoUM0NKjXlj/5kz/hpz/9KY8++igul4uPfOQjALzwwgvzfq4CijVUeAPs6/0CzhlQnKkIxWq7oiiMpXo4EXuNnJJFLxlZ5dhJvbm7tH+FmdOZEJrOpzgcPchAshcAs87CBufFNJrVotiVoVg7fsEIK5AOsT98iPG0unfDIBlY61zNKvtKDNNmNxfjVn082s+RaC9yLj15Dj2rHR2sc3ZPm8mtEL/CVSiX1+6fyufpiY1yNDrEqBwqtpt0Blbam1jjbKXBPLX3t6JB1iIMu/KKwkgywumYn57YBKFMqfOyz2Sjy1FHp62OJqurrG5tRcitCInVg72iKARSCYYSUYYSEYYSEU3jK4veQPPkbHKz1UmT1VmcTa7dDHJVzXNCZSaXI5BKEpDV24ScIDBZNik/C9k6jCa8Zises2Xy3orXbMFlsmDQ6aqG4loYgsmxOPdffeuyHSgFFAudDypC8YPf1YDiCtcb4Uo9a3zhSj2Pc0y2C1fq2eNfCK7UcjzOJ285v6B49+7d3HDDDWXt99xzD48++igA//RP/8R9993H8PAwbnd5xZ3u7m7e9773Fd2t3/Wud/Hcc88xMTFBQ0MDV199NX/913/NqlWrisfIssyf//mf893vfpdUKsUtt9zCP/zDP1RVflFAsYYWA8UFJXNRjkReIJpVobPBvIIV9u3FWd75QnFBo6kJDkXeIJFTlwvXm5pZ79yGWa/9Bp0LimFyNjo1zv7IYcIZtf6YRWdmvWsdXbZOdJJuwVA8db48p+NDHAqfJpKNA6qLdKetmXXObrwmr3b8KqF4ev9wOs7x6DDHY8PEs1Ozh26jjVWOZlY5mnEa7FWdt9pl2ADBdIKe2ASnY35GkpESJjLp9HTYfHTafbTbfDiM5jMCxVqKp9MMJ1VAHk5EGZNjmiWEfCYrTZOA3GR1Ume2q9BYPMHZheJKcXJKnnAqRVBOEkglS+61Zs2ny2U04zFb8JiteEyWqTrDJrPmkvPZ86/0gIBiIaGzocVAMQhXai0JV+p5nGNau3Clnj3++e5KfT5C8bksAcUamgnF6VwUg86KTjLMG4oB8kqe0/GDDCQPAAomnZ1Vjp14Ta1VQ3FW0ZNTcpyKH6EnfhSF/CRcrqXbthHDjGXJ84HighRFoTc5xKHIERI5tdyPVW9lnXMNbdYu9NL8lx9Xyj+XhyF5nCORHsZSU+7CPpOb1Y4uOm0tJedZDBRPf15DySDHo0Ocjo+RnWbaVG9ysdLRzAp7E3aDZdY4sDAoni45l6E/HuR0fIK+WAA5XwpjXpONdpsKyK1WT8mS3qWG4pkP5PJ5xuU4w4kow8koI8kIkUz50mQdEnUWGw0WB40WB40WJ/VmO4YZy5HPNhTPFieZzRBKyQRTSUKpJMFUkmBKJpRKkslrm3wVZNEbcJssuE1mXJOg7DZZcE7+bpj5YVVAsZDQstJioRjmMWO8yL3Gc4LxIvcazwnGi4BimMeM8SL3Gs8FxovdazzXjPFioRjmAcaLXFZdAsYzzbeqyLNS/znBeJHx5wTjRcafC4wXu9d4NjAWULy8JKBYQ9Oh2O40cCL8IyRJR4fjRvQ67RIDs+01jmTGOB59gVReneVtNK+i3b5ds3TTfJZDx7NRDkf3MpEeBcCks7DafhHNlq7ihagaKAbIIZFX8vTET3M0erzoVG3RWVnjXEOnrRSOF2PMFUiHORrppS8xXFzWataZWGnvYJWjA7vBVhMohimgTOeznI6PcTI2wnAyUMInTWYPKxxNdNsasRismnEWC8XTlc3DmByhLx6gPxFgTI6WPK5Dosnqos3qpc3mpcFcvtQalg6KteIksmlGkzFGklFGk1FG5ajmLKsE+MwqKNebHTRY7NRbHFj0Gu7pywCKK3ZRFBKTwBxKyYRTSYLpQo1hmeQ8yifZDUack7DsNJpxGc0qME/eT800CygWEjobqgUUg3Clni7hSj2Pc2i0n0+u1GVgXIP456srtYDi5SUBxRqaDsUGW4LT0f8mqySRMNBkuxq3aV3Z8pDZoBggp2Toi+9lWD4KgFGy0u24HK+5U7N/eZzyUlDj6WGORPeRnFxS7TL4WOPYisfUsCAonuqT43S8l2PRE8h5db+pRWdhtUOFY4POsGi3agA5l+JEbJCTsT4Sual9rc2WBlbaO2mxNqKb8YFioVA8XclsilPxcXpiI4ymQiWPNZo9dNub6Jo5g1xDKJ55jJzLMJgI0h8PMZAIEM2W7vE1SDqarR7arB5arV7qzQ50ku6MQvHM7oqiEM2kGJdjjBVuyRjJXEbzGIfBrAKy2aHWGjbbcRutmvWTlwMUzxUoncsSTqcIp2XCaZlIOkVk8j6cluecZQZ1Cb1zEpAdRjMuk5ntDe1IkrSkUDw6PFYTKG5qabwgBkmh81e1gmIQrtQFCVfqeZyjQvv55EpdAsY1in8+uVKva6jnLetXCyheZhJQrKGZy6cz+QQDsd3EsurF0GVaTZPtWvTSdBfo+S2HjmTGOBl7iWRO3cPrM3XT5diBUWfV7D9X/HReoT9xnJ7EIXKT7ssN5ja6bRdjMzjL+s8Hiqf65uiJ93M8ehw5r5pGmXQmVtpX0WVbiVFXXm6qWrfqvKIjr+QZSo5xItbLaGqi+JhFb2aFvYOV9k5skzO4tYBitV3tH8vK9MRH6I2PMZYKl/RpMLvpsjXSaW/EYSh3EJ/tec2m2ZyvFUUhkpEZTAYZTAQZTAaRZ4CmUdLTbHXTYvXQYvHSYHGWfHlwJqBYsz0P8WyaMTmGX44xnoozLseIZLRLKBkkHT6zjXqznTqzHd/kvVVv1DbrWEZQPJvyioKcyxJJy0TTaSIZFZaj6ZR6n0lpzrLbDEb+cONOYGlnine1/pHmKpVqlM2neHromxfEICl0/qqWUCxcqafyrHTeC8qVerZzzNIuXKlnz/98cKV+5vgpbt+4DrfVIqB4mUlAsYYKb4DDvQ/hdKmzhWqJpH0MJd9A3R/sotNxIzaDugSocomlcpjNKTl6YgcZSB4GFAySmRWOS2g0ryBXca9xhfbJEk6pnMzx2CH6k6cB1cyqw7qKlY6NJaWWKu75naU9p+Q4nejnaPQEiZwKxwbJwAp7F6sdq7Do555RnS3+dEUzcY7HBjgZGyKVTxfbWyz1rHK002JtrGqP82wwPlOxrMyp2Dg98bESB2sAj9FOl72BbnsjdSbngl2sZz1Go10tO5VgIBFkIBFiOBEmNWM/sjqT7FIh2eqm0eLSLDNUKyOvysuetfvLuSx+Oc64HC/eT8iJkj3e02XRG6gz26kz2/BZbOq92YZNr13z+6y5YVd53unK5HJEMyki6fTkfQoJuKJFXTkix+J86oq3CigWElpCFaH4gcfKobgSVApX6pL+841/oblST89p3ueYbBeu1LPHP9ddqRVFKb4ecjzOJ29dGiiuxaqwQrwLZWWYgGINaUFxQeHMGH3xZ8nk40joaLJeRoNlMzm0nWgrG2rpiWUmOBZ9iUQuBIDb2ES3fSdWQ7k9+VxQXFA0E+FobD/jKXW/sQqvG+i0rUYvGRYExQXllTz9ySGORk8Qyar7YHXo6LR1sNqxCqfRsWgoLiijKAwkRjkR62dUDhTbTToj3fZWVtjb8Zqm/jlrAcUwBbPxbIre+Bin4+MMJ4Mo09DIrjfTaW+g01ZPk9lXZiw1PY6WqoFiKIWyvKIwkYozlAwxmAgzlAiVQbIOiXqLQ607bHHTZHVhN9TO3bpaKNbqnlcUwikZf0oF5UKt4VA6WRFCrXoDvklA9plseM1qzWGH0VzVzPJygOK54i8lFIvl00JCqpYCikG4UmvpQnOlnpnTvM4xrf18cqUuzhgLV+oyLSUU72r5o2LFm8Uom0/z9PCF8SW4gGINzQbFWXRk8zID8V8TyfQCYDM00Wq7HpNeA2bnmPlVlw4fpi++nzw5JHS0WDfQZtuCXjKW9S+Pr/1fOSb7ORbbRzQbAlQzrhX29TRbVmvOtFYDrYqiMCiPcSx2nEB6ykm6ydzISvsqGswNZRe/aqF4OlRGM3FOxQY5FR8kmZtyQfYYnXTZW+mytWDR27TCLBiKpyuVy9AX99ObGGcg4S+Z4dRLOlotPjps9bTb6nFMLvNeKiieGUdRFALpOMPJMMPJMEPJcEkZqoIcBjNNFnUWucniot7sKML82YDiSg9k8zkmUjITKRWUAyn1Fq6wBBvUv4HHZMFrsql1hk2TN6MNq6Hc4OtCh2JhtCUkpGqpoBiEK/VMXWiu1Fo5zXmOGe3nkyu1JhgvMv657koNAoqXmwQUa2guKAb1ghhIHWU48TJ5MqoJl3UHXvOmkn/6uaC4IDkX42TsVYJp9QJr0tnosl+Gz6RepKqF4pyiR1EUhuVeTsQOIufVUktmnZUu20ZarSvQTYPjaqE1hzS5tDfA8dgJRuTR4mMOg4OV9pV0WDsw6AwLiq8FlXlFYSg5QU9sgMHkaNG5GqDB7KPL1ka7rRnTtPJUtYBimALHbD7HUDJAX8JPf8JPIlcKoB6jnTZrHa3WeposXs1Z5FpCsZYiGZnhZIjhZJhROUIgFS+LoUPCZ7bTaHHRYHbRaHbhMdlKjK/OBhRXipPJ5ybrC6szysF0gmA6STid1KypXJBZZ5iCZJN1soSSFbfRikVvKB2gBRSflVhCQmdLSwnFIFypp+tCc6WulNOs59BoP59cqcvAuAbxz3VXarF8enlJQLGG5gPFBaVzUfrjzxHPDgNgM7TSarsOk9452X/+e4QVRWE8NURv/FVS+TgATkMT3Y7LMOu1zShmg+KC8kqewWQPp+KHSU0aZll0Nrrtm2i2dKOTdAuC4umKZWOcivXQm+gnO2n4ZZSMdNo66bZ3Y9Vr/yNVA8Xqc1HbU7k0/YkRehNDjE+re6xDR4u1gU5bKy2WBnSSRikgFg7FpW3qUuaBpArI46lQCeDoJR1NFi9t1nparT48RgeSJC05FM/sn8lnGZOjjCSjjMkRRuRwmXkXqAZedWYHDWYnDRYn9WYXbqOtfMnTWYDiSv3zikI4o9YZDqbV5dehVJJQOkksmy4/YJrMOrXOsMtowWWy4jZaJm/qkuzCFwQCipc2lpDQ2dJMKM7l8iRjMg63rSZQDMKVuqALzZV6tpyEK7VwpS5oKaFYGG1VLwHFGqoGimESZuXDjCZfQSGLDiON1h14zRvJSRX2Gs/iVp1XsgwmDjKUPIhCDoB68xrabJcUXapnywdKobigvJKjL3Ga04lDpIullmx02jbQZFlVMnM8FWf+btUAqVyWvkQfPfEe4rl4sb3B1EiXfQWN5uYSp+SFQvF0xbNJTseH6UsMEs7Eiu16SU+LpZEOWwvNlsYZdZZr62IN6jLrITnAYMLPQHKibBbZqjfRYqlTb9a6kpJPlZ4bLB6KZ/ZXFIVYNsWYHCnexlMxskqu7BijpKfe7KTe7KTO7KDe7MRjtJWVylLPe+aheLb4mVyecDpZnFEOZ2RCkz/PBcw6JLVMktFSvDmNFrXGsNGC3WDS3scMAoqFhM4RTYdik9HCr3/4KhPDQW5+99W46surNwDClVq4Us/or+1KPWu5piqHDuFKPXf+56ortYDi5SUBxRoqvAFO9X1uXlAMkFUkUrkIvfHniGfHALAbGmm1X4dF7ynrn5lHHWE5F+dUfC/jqT4A9JKBdttmWq3riwBbGa4rxEcip+ToS5ziVPwY6bwKbmadhW7bOtptK9FPA/lcxXrE2vELUKkoCiPyGCfivYzIY8XHrXorK+xddNk6seot5Cu8+7SgvpC/5nkn8wymo5OAPEIsmyw+bpD0tNsa6bA102KpR1fhy4pqDbsqAVwuD8FMnIFEgMHEBMNyiNwMt2W30Uar1UeL1UuL1Yu5kiNwhdeoMsBXB6d5RSKvKATTccbkGONylDE5ij8VL8sZ1BnwOrNac7hwqzPbi0vl55tPJZ2JPc7ZfI5wWiZUrDM87eeMPOuSbAC9NB2aVVB2mszqzwa17rB+xhcH1Vxok7E4n9jxW8t2oLyQBkmh81dFKP4/30anGPjFd18k5I9itZu5+T1X46rTKMUnXKmFK3VJ/lPxS823utm4tnXWnOZ9jsl24Uo9e/xz0ZU6GY/zydsEFC8XCSjW0EKhGNQLoz91mKHEa+TJIqGj0Xox9ZatJTOx84HigsKZcU7E9hDLqjV8LToHXfZt1Jk7K7tezwMqc0qOgcRpTsWPFesQGyUz3fa1tFtXYdQZFwzF0xXLxjkZ6+d0oo/0ZJklCYlmSxNdti6azI0axlwLg+KCVAOqCL2JEXrjIyRyU0ZNeklPq6WBdlsTrdYGjPPag1wdFM/sn1PyjMphBhMBBpMB/KlIGSR5jQ5arF6arT6aLR4shRJEZwCKtdung3IMfyqKX46T0ZhRBnAbrZOllBzUmR34THZcRgtU+J+ppLNt/KUoCvFsehKQU0VQLvwey6TmBbh2gwmHwYzTaMIxCc4Oo9rmMJqxGYxl4FyQgGIhoaXXdCi2WmzIiRRPf+8lFYwdZna9+yrcdTNmjIUrtXClLsm/NL5wpZ49vnClLpWA4uUlAcUaWgwUF5TOxehPvEAkMwCAWe+lzXYVdqN6gawGitX+EuOpHnrje0lPAqzd4KPddjFuY2vZRWU+UDx1zhwDiQF6EkdITi551ksG2qwraLeuxaq3azzf+UNx4XnllBwDySF64qdLXKstOgudtg46bR04DI7J/ouD4oKUoiFYmN74MP2J0RJA1iHRaKmjzdpIq7URa5Uu1vOF4pnHpHIZhuUgw8kgQ8kgwXSsrJ/baKfJ4qHJ7KHJ4sFpsJb8nZcairVjK4TSMuOpGH45ykQqjj8VI5HTXo5skHR4TXZ8Jjs+sx2vSb05DBXKKHH2oXgu5RWFWCZFOJOarDEsEy38PHmvNcOuJZvBiMNgxm4w4TSaua55FZIkCSgWEjoDmgnFQBkY3/zuGTPGwpVauFKX5F8eX7hSzx5fuFJPSUDx8pKAYg3NhOJsPoZesiJJ+nlDMUzOGqdPM5x4kZyigpjHtJpm206QtPcrVYLiwnlzSpbBxGGGkofITRpaOQ1NdNgvxmmcurBXA8VqXD15Jc+w3Mfp+FHiucjkIxKN5nY6bWtxG+vmjD8bFE9XJBOhJ95Hf3KgOHsM4DN66bB10GzpwKRhJb8QKC75fXIGuS8xymBilEg2XvK42+ik1dpIq6UJn8ldvFDXGopnKplLM5QIMSwHGEkGCWXiZX2sehNNZg+NFg+NZg9es0tzpnEpobhS/GQ2jT8Vw5+KM5GKEUjHCabjFZchGyS9WkLJZC/ee0w2XEYLUoX/seUCxXPGzyskc5kiKMcyKaLZws9pYtkU8Uy6xD0d1JnlD6zdCYiZYiGhMyEtKIZJMP7+y4TGI+VgLFyphSt1Sf7a8WcDY+FKLVypCxJQvLwkoFhD06HY4TQyEvs+oFBnews6vbY5gxYUg+o+nc3LjCZfJZA6AoAOI3XWy/CZNyPNMLeaC4oLyuRlBhIHGU4eQ0GdlfIY22m3b8VuqFsQFBekzqyO0ps4xkR6qtSS21hPh3UNDeY28lRwdZ4nFE89jxwj8ih9iT5GU1N7j3XoaLI0027tKDHnWiwUz+wfycQYTI4ymBxnIhUswRSLzkSzpYFmawON5saSUk/zjb+QnADkXJoxOcyoHGRUDuFPRcogSifpqDe5aLR4aDC7qTe7sRssZwWKteLnlTzhjExAjhNIxydBOUE4nSh7LgXpkHAZrXhMNvVmVO/dRhtmfYV6e8sMiudjCKYoKjhPh2QFhYt86h40AcVCQkuvSlAMIKeyPP3Y84TGZ8wYC1dqQLhST+VfoX2Wck3ClVqVcKUWULzcJKBYQ9Oh2GyPMRb/D/KKDOhwmHfgNO8og9nZoLigRHacofjzJHPjAJh0HppsV2E3dhT7zBeKC4pnZQYTbzKeOknhY7fX1EGz9WJshrqy/vOB4ukKpcP0JY8yKvcX4duss9JqXUOLZTXGGeZQ1ULxdKMtOSfTnxygPzFAJBspthslE63WNtqs7biN9ZrLbmsxk5vKpRlM+hlKjjIi+4ulpQqqM3lVSLY04jG61PJKSwTFMw4gm88xngozlgozlgoxJodJ5cvLKtn0ZhrMHuonIbnO5MI4aYB1pqG4Uv+ckieSSRJKJ1RQTsUJZRKE0gmysyw7tuiMuCcB2W20Tv5sxWWwFp9jaT7LF4rnkoBiIaGl12xQjF6vzhjPBGPhSj0ZX7hSq/lXiA8VyzUJV+qpfC50V2oBxctLAoo1VHgD9Pd/DpfLQjYfZzz5LPHMSQCMunrqbW8pqR2crQiDpe3qLOwJBhKvkZ1cUu02dtNq245Z76rsJl1x2bbaP5GN0Jc4wFiql8LHb5+pnQ77FhwGX1n/SnHK8p98XnIuSV/iFP3J00XHah06mi0ddNlW4zJ6J+NUKnVUqXSU9usWSEfpTQzQnxhEzk+VNrLqrbRZW+mwtuOeBFOYzSyq0lJcbRXgPafkGZeDDMp+hpLjhGcsZ7boTDRb62gy19NsqcNWVl6p8khZLZxqwZeiKATTsjqbnAoxJkcIpmNlsSXAY7RTb3FRb3LRYHbhMztLll1XhMczVCJq6ne1VFQgnSCYThBMJQhNwnN8jhJKNr0Jj8mKy2hVQdk0eW+0YNZrr2qYb/6V+1fVvSqITsbifOyyty/bgfJCGiSFzl8V3sff/N//rwyKC2NLyR5j4Uo9rV29E67U2s3qc9BYSi1cqcv6X8iu1Ml4nE++VUDxcpGAYg3NhGKY/MCeOc548tnirLHbfBke83YkyTBvKC5IzmcYSe7BnzqE+nFZR4NlE3WWS9FL5ctE54LighLZML2JA4yneottPlM7HbbNOIyVl1XPd7l1TskxIg9wOn6KSHbKLMtt9NFhXUm9ubOkpFNB1UJxASrzSp6xlJ/+5BCDyeGS2VuHwUG7tZVWaysOvavCDPLCoHimolmZoeQ4w0k/I/JEWU1ft9FBs0UF5AazV/NvONe5Fzvbncln8aeijMoRdVZZjpTVSgZ1ibLX5KDO7KTe7KLO5MJrcmDQlb4HzjQUz9Y/nc8RTicIpVVIDmUKtYeTyLmsxhFTMusMuI3WyTrD1ml1h60lpZMuFCgeHRqrCRQ3tTZeEIOk0Pmr+UAxlJtvCVdqZnWlvu6WLbTOnDFeIBTDuelKrT6HqfjClXr2+BeqK7WA4uUlAcUa0oLiglK5BBPJZ0lkTwBg0Hmps16P0dClGatiSaNJSExmAwwlXiaaVS+WeslMg+USfOaNJSWc5gvFBUWzEfoT+/FPg2O3sZlm62bcxpZFuVUD5PIS4UyA3uQJRuUBlMlP9gbJSLOli1bLSpxGT7H/QqG4tG+OQXmMgcQgI/IoeaaW2tr1dlqsLbRaWvEYPdNmkGsDxdOXheeUPBOpEMNJP8PyBIF0uKSvhITX5KbJ7KPRUke9yVtSw3epoFirfyKbYjwVYTwVwS+r91rLriUk3EYbPrMTn8mJz+TAa3RhM5TXTj4bUDybktlsEZDDmSSRacCczJU/15myG9Tawk6jFYfBotYcNphxGC04DBaMOu3/jXMVim9yvgfDLF/azEdZJc0vo/92QQySQuev5gvFIFypy9tLfxWu1FrPoTS+cKWePf6F6Eq9lFBciy/AC/EulC/BBRRraDYozirqxS2ROU5A/hU5JQGA1bgOj+Va9LrS8kVzQTGo8SKZfoYSr5LKhwAw6pw0WbfjNqolWqqF4gLMJrJhBoozx+qf2qb30mLbTJ2pC2lyoK0WiqfDZionMyj3MJA8XSzpBOAy+GixdtNk7kSvs2iFqQqKp+eTyWcYlkcYTA4zJo+VALJVb6XZ0kyzpRmfsbFo0jVdi4HikjiKuhd5VJ5gWJ5gLDVBLJss6SMh4TO5aTD7aDD7qDN7NU27lgKKZ+aqLlGW8aciTKSj6n0qiqwByqAuE/eZnHhNjsmbE7fRXjarrMY/O1A8W5x0PkskIxPNJIlkZCJp9T6aVWsPz6d0kkVnxGE04zBYJssnqcBs16ttNoOpYr3h0nwEFAsJLSdVA8UgXKlL28ubhCv1zOdQHl+4Us8e/0JzpV5KKK7FWA8X1ngvoFhDc0FxQXklRVB+kWj6TQAkTLgtV2I3XVSEzflAcUGKkmc8dZKx5GtkJ2HbrPfRZLkMq1F7j8V8YVbOxRhKHmE0eZI86nJTs85Bk3U9DebVIFnnFacgLWhVFIWx9DhDyVOMp4aKxlw6dNSb22m2rMBnaiq+NrBwKJ6uTD7LiDzGUHKI0dQouWlLmw2SkSZzE02WFhrNTRgnYbSWUDxT8WySETnI2CQkT6+LDOp11G100WD2UW/2Um/2YtVbzggUa8eBZC7FRDpKIBUjkI4SSEcJZxKa/SXAabThNaqw7DHZ8RgdOA32qr6AOBNQPPtxhdJJk5CclollU0QzSfU+K5PJ5+aIosqmN2GfBGa7wYzDYMZmMGPXF9pMGDS+CKmUqFg+LSS09KoWikG4Uk+1azcLV+rpz0GjTbhSzxn/QnKlFlC8vCSgWEPzheKCUrlR/MlnyeTU8kVGXSMe6/WYDa1VQbEaX09eyeCX9+NP7SOvqAZDFn0DDdYd2AztJYN1tTO8yVyGUfkoo8kjZJVJwyzJSL15LY2WDZj1pfukqoFiNZ9J85C8zIjcy3DyNLFcuPi4WWelydJNs6Ubu8FdEyhW+6vtOSXHeGqcYXmYUXmU1DSTLgmJOlM9TZZmGszNOAzlZim1gOLp+SuKQjyXZDwVKN5i2XLYtOtt1Ju91JnUmWSXwVH8Oy81FFfqn8nlCWZiBFJRgpkYwXSUYDqmufwaCuWUVED2mOy4jQ7cRjtOo70m9ZQraSnjpHJZYlmZWFYmmkkRz6rgrJZRUn+uVF5qpow6fRGSbQYzNr2pCM6Fn11G9cspYbQlJLT0WggUC1fqQrt2s3Clnv4cKrQjXKlnjX8BuVKL5dPLSwKKNVR4Q40N/H8aUKy93DKj5Imk9xNIvkAeFWQdxvW4LVdh0JXDVyVjrunuzdl8ilF5P+PyoeLsrt3QTLP1MuzGZmAWOJ1juXVOyTIq9zCYPEoyVyh/JFFnaqfVth6noQFJkiqXiKoA4zPhUVEUItkwfYk+huV+MsqUi7DL4KHF0kWzpQOzvnSmuiIsV7E3Oa8o+FNhhuQRhuURotlSB2mHwU6zuYlmSxP15jp0kq6kRNR8zlsJyCrBO0A8k2IsFWRMDjKeChHMRMv6GHUG6k0e6szuyXtP2ZLrswHLiqKQyGYIpGOTtzjBdJxQOk5G0Z5VlZisPWy04zbZivdug13TGbr6UkfVQXEtIbpYbzibIpZJEc+miGZTxLOT9YcnaxBXem2my6TT8wdrrgYgGY3zkYvfIaBYSGgJVYTi+/+1vCRThQ/gwpW6NE55u3p3wbtSw6zQJ1yp535eF4IrdTIe5xO/JYy2losEFGtoIVBc8L/N5uME5BeIpg8BIGHEbdmBy7QNaZor83yguKBMPslwch8TqcPFJckOQytN1kuwGrQvovPdg6yW9hlmIHmEUGak2G43eGmxrsNrWqHpJj1fKC4op+jIKznGUsMMJnvxp0eL5lwAdaYmWixdNJpbMeiMNYFiKP3SIJqJMZwaZUQeYzw1UXJ+vaSn0dwweWvEbijdG15LKJ55TDqfwZ9SSyv55SD+dEhzr6vLYKfO7KHO5KbO7MZldGnOwC71DLLWc1YUhWhWJliA5IwKyqFMgnS+sju0RWdU6w6b7LiMag1ip9GGy2Ar37e8DKF4vnHS+SzxTFqF5OzkfS5NIjvVZtYZ+L3uSwEBxUJCZ0KLgWIQrtTa7VM/XtCu1DArFBfiC1fq2eOf767UAoqXlwQUa2gxUFyQnB3Bn/wVqZx6MTVILjzWq7EZVk8aZ80fitX4OtK5GGPyHgKpYxQ+etsMLTRYtuGYuax6AcZc8WyIoeQR/PJp8qgzW3rJRIN5JU2WtVgN7jnjzAbF05XOpxiRBxiU+wlnJortOnTUmZtpNHdQb2ot24dZCxdrgFQ+y1hqnBFZheTpy6wBbHobjeYGGswNNJjrMVQwCqsFFBdznTwmr+QJpaP402H8qRD+VEhzybUOHR6TE5/Jjc/kwmty4zY6gApuyUsIxWp/rb4K8VxmEpDjhCdBOZyOE9coFzVddr1ZBWSjFafBhtNgxWVU76fPMJ8LUFytBBQLCS29FgvFIFypy9tLf71gXalhTiguxD+vXKm1wHgRUAnntyu1gOLlJQHFGpoJxYqSLc7yzheKQb3YhdNHCMnPk1PUpbsmfRNey9UYDB3aceaxHDqdizIm7yWYOl6cObbo62mwbMNl7EaSdAt2qwbI5FOMyicYSZ4glY8V252GJposq/GZO8lTXqoH5g/FU/11JLIxhuU+huU+ErmppcQ6dPhMzTRaOqg3tWDUmWoGxdPzVBSFcCbCaGqMEXmMQDpQMosM4Da6qTepkOwz1RVLLC0FFJf3BzmXZiIVYiIdZiIdIpCOkNbY21sAZY9RhWSvyYXb6EQv6c8KFM8WJ53LEs4kirfItPvZZpcBzDrjZPkkFZIdBmvxZ7vBgl7SCSieIQHFQkKlqgUUg3ClLm0vb7ogXalhXlBciH9euVLPBONFQiWcv67UAoqXlwQUa2gmFMcSPwAM2Cw3kJfsmsdU+gifVSCvZIikXieSegMFFWTMhm7clqsw6htm9J//HuF0Ps548gCB1GGUyQxMOjf1li04TOvRVbHsWQviFEUhkB5hVD5GMD1I4eO+XjLhM62kwbIGm6Gu5JiFQPH088WyEUZT/YzIAyWALKHDa2qk3txOvalt/nuQqzTOyiuqm/VE2s9YapzxlJ9otnTPr1qH2EudqZ46UwNeo6+kDjHUHorL2xSi2SSBdJhAOkwwHSGYjpBRyt+JEhJOgx2PyY3H6MJjcuExujDrTRXjw9JD8Wxu2Kl8hkgmQSSTJJqdvM8kiGaTJHNp7QOnyaY345iEZcdk/WHHJDDbdVb0WiWl5ow6M38BxRfCICl0/qpWUAzClXqqXbv5gnOlhnlDMXD+uVJPB+MaQCWcX67U65vq2bVhtYDiZSYBxRqaDsV2e4xI/FHUj7pGzOYrMBovQZJKP1TPBsUF5fJxwqlXiKYPwOQMr9W4HpflCgw692T/6oyz8oqObD7JROoAgdQhcpOO0nrJgte8GY95EwbdFEBW61ZdgNlULq6Wi5JPkM5PGVbZ9HXUW9bgM63AoDMvCoqnK5uHeC7CqNzPWGqARNEMTJXbWF8EZJvBWVMonik5JzOWmsCfHmc8NU4yV16H2GMsQHI9XpMPna6yDX4toBjKYVP9UiFBIB0lmA4TTIcJZaKk8toQadGZcZtcuA1O3EYnbqMLl9FRLKt0tqB4LqOtTD476QatQnIkKxOb/DmaTc6z9rBpCpINFuwGKza9pfi7RWeq7D5bzF9A8YUwSAqdv6olFAtX6kK7dvMF50o9mZP2c6jQznnmSl0A4yVypS6C8RK5UhfBuMau1L88eorf2rKOOrtNQPEyk4BiDRXeAKHBL+ByWcnkRokmf04mNwCAXleH3bILo6GreEwW7Q/iOY2XN50LMS6/SDxzbLJFh8u0CY9lB5Kk4V7J/Iy5ckqGidQxxuSDpCeXPUvoqTOvpcGyCbPeXTUUaxpzZUYYlk8xkRooLt+W0FFnbqfevBKvsbWkFrEav9Ky5/lBaywbZUQeYkQeJpwNljxm1ztpMLfSYG7FY6wr+dBSCZYrgdd88oll45PllSYYT02QmAHJUFhu7aPOXEedyYdFP7UnuVoYrAT2lf5xZy4NT+ZSxdrDwXSUQDpCLFueMxScogvllJx4Ju/tBiu6OUpEnTVYLjlWQc5l1LrDGZloJkk0KxPLyESzSWJZueIWiOnSIU0Cslm916v3xRJKBgtWvan4mswrzyqutMlonD/a+jvLdqC8kAZJofNXRSj+1KPlUFzlDK9wpS6NU96u3l0ortSzxRKu1PPLs1L/kqXUbY1cu2UpXal97CrMGNcofi6fL91T/DYBxctFAoo1NBOKYfLDdmY/UflZFEU1PTIZNmC33oxOslQFxaDOLKeyowTkF0hm+wAVYB2mLXgsV5Utfa7GmEtR8gTSpxmT95PMTZlYuYztNFh2YDWUDxDzheKp/joyeZkxuYdR+RSJXKj4mFGyUG9eQbttM0adudhfM06VM7k5RUcyl2AsNcRoaohg2l+y/9comWgwt9BmXYnXVL8kUFwSR5GIZxMqIKf9+FMB4jmtOsR2fCYvjZYGOqza+8mXAopn5lpQJp8llIkRSkcJTN6H0lHN5dcAekmHy+DAZbTjMjrpsDXhMpZ+0FsOUDxXPoqiIOczRVCeqjus1iKOZ1Mk5jAAK0hCwqo3qeCsL9QeNmMzWFntLHcQFVAsJLS8tBRQDMKVWrt96sd5gfECoRiWhyv1bLHmmkEWrtRzx58OxiVLqWsUfzoYlyylrlH8ggQULy9VemsvG3V3dyNJUtntQx/6UMVjnnjiCdavX4/FYmHLli389Kc/XXQekiRhNV2Ex/FBzMaLAcjmhpEo37c7X5kNTbQ43kGL451Y9K0o5EhlB5EquAfPP1cdXvMq1rruYJXzNlxGFcIimYFiveNayKiz0GbbwMXet7LNcxut1nUYJDMZRWY0dRydtLjnUUlWvY0u22p2eK/lxobfYqt7B82WDgySkYySZkjuJZoNLcm5tWQ32Oi2d7DdezG3Nd/E7c03s917KSvt3bgM6gUknovTnxygN95/xvKaTUadgQazhzXODrb7NnJz805+p+Mm7mi7jusbL2WbZx3dtla8Rhc6dOSUPMFMhN7EMPvDxwhp1FY+FyRJEha9iQazixWORjZ7Orm8fg27mrdwR/sO3t19De9feSN3dV7Fb7Vexg2Nm9nuW81GVwddtgYazC5sejMSoKCQyKUYT0U4nRjnUGSA14IneS1w/Gw/zWWrr3/963R3d2OxWNi5cyevvPJKxb7//M//zDXXXIPX68Xr9bJr166y/rFYjA9/+MO0t7djtVrZuHEj3/jGN0r6XH/99WXjxx//8R8vyfMTWriWy1hfC1lsZna963I89U50eh16fW3HQpPZyI2/fSl1zW50Ogm9vrYf5QxGPTfcvpWmNs+SxNfrdVy7ayPtnerKrlrH1+kkrrp6HStWNiDpwGCobXxJkrh8+0rWrWlGkpYm/qVbOrlovTpDXOv4ANvWtrF9Y8dk/Np/VtvU3czVm7sn4+vm3I5Urda21nPjltVIgLHG7x+AFQ0+br1oDTpJwqjXV/yuR6hczz33HG9729tobW1FkiR+9KMflfU5fPgwb3/723G73djtdrZv305fX1/FmFrjuCRJ3H777cU+73vf+8oev/XWW6vKfeFEd4b06quvksvlir8fOHCAm2++md/93d/V7P/CCy9w991389BDD/Fbv/VbPPbYY9x555288cYbbN68edH56CQLDuvNWExbUJRMSe3hhcpqaMfi+B2S2T4UDDW7eEiShNPYitPYSioXJpQ+jU0/y/6XRZzHYfThMProtF1KKD1EKh/XrG9caxl1JlosHTRZutRSRhk/46khGswaM3VnSFa9hXZbG+22NkCtQxxMB5lIB7DrbXMcffYkSRJ2gxW7wUqrtaE4Y5tXFOLZBOFMbPIWx2s8f78t1Es6nEYrTmNhlUh5n7yikMylSWRl4rkUickZ5kQ2VWa6JqTq+9//Ph/72Mf4xje+wc6dO3n44Ye55ZZbOHr0KI2N5bMsu3fv5u677+bKK6/EYrHwhS98gbe85S0cPHiQtjb1f+tjH/sYzzzzDN/5znfo7u7m5z//OX/6p39Ka2srb3/724ux/uAP/oC/+qu/Kv5usy3f/8MLVcttrF+sCmCczYHDU/v3WwGMU8kMLq+2AehiVADjaEzGo7X8e5EqgHE4IuNdgvgFMN64uZ26JYhfAOM1a5qoX6L4l27ppLu9jvqZqwxqpG1r22hrcFPvq33+oIJxo8dBvXdp4q9trcdjt1DvsdccukEF49/Zvpk6l21J4p+visfjbN26lXvvvZff/u3fLnv85MmTXH311XzgAx/gM5/5DC6Xi4MHD2KxaJc+BfjBD35AOj3ljzMxMcHWrVvLxodbb72VRx55pPi72axdKaeSzrnl0/fddx9PPvkkx48f13yT3nXXXcTjcZ588sli2+WXX862bdvKZhAqSWv5dEFZJad5zEKWT2v3126vtq5xpSW0i91TPNW/kvFXhWXJNVw+rd2/ulJNtVw+rRlnVvfpCu1ncPn0/Pprx1/Oe4rPZD4Vz7vM9xSPDI3WZPl0c2tTVfnt3LmT7du387WvfQ2AfD5PR0cHH/nIR/jUpz415/G5XA6v18vXvvY13vve9wKwefNm7rrrLv7yL/+y2O/SSy/ltttu43Of+xygfsO8bds2Hn744SqfpdDZ1Jkc62u9fLpElWaJa+RKXfG8NXKlrmhsVSNXaqViHO3mhRhwaapGrtSzrbdc6PLp8v5V5lTlst7KcRa+fHp+8WvUf7nls0yWT9dirC/Eq3a8L0iSJH74wx9y5513Ftve9a53YTQa+fa3v73gnB5++GE+/elPMzw8jN2ufin4vve9j1AopDkzPV+dU1Ma6XSa73znO3zsYx+rOBC8+OKLfOxjHytpu+WWW2Z9kVKpFKnU1D7CcDgMQCQql/WtXKe4NlCs5X6s9q8VFFeAx4qQWCUUV4pTIyiu7CZdKb5m87KE4kpXWAHFs/evpFrlU62ddMXu1UBxTN2XvhTfWf5O6/sxYFxUjOxkablIpNQV3mw2a34zm06nef3117n//vuLbTqdjl27dvHiiy/O65yJRIJMJoPPN7X38Morr+Q///M/uffee2ltbWX37t0cO3aMr3zlKyXH/tu//Rvf+c53aG5u5m1vext/+Zd/KWaLl7HO9FifTGkYD54jUFwZQquMX3UcAcWzxp8lloBi0b8gOaFWc1muYz1UP97Ppnw+z09+8hM+8YlPcMstt7Bnzx5WrFjB/fffXwLOc+lb3/oW73rXu4pAXNDu3btpbGzE6/Vy44038rnPfY66uroKUTSknEP6/ve/r+j1emVwcLBiH6PRqDz22GMlbV//+teVxsbGisc88MADCupHVnETN3ETt2VzO3nyZM2un8lkUjFhqVluDoejrO2BBx7QPPfg4KACKC+88EJJ+8c//nFlx44d88r/T/7kT5SVK1cqyWSy2CbLsvLe975XARSDwaCYTCblX//1X0uO++Y3v6k89dRTyr59+5TvfOc7Sltbm/KOd7yjuhdP6IxKjPXiJm7idiHdlvNYD9WN99MFKD/84Q+Lvw8PDyuAYrPZlC9/+cvKnj17lIceekiRJEnZvXv3vJ7fyy+/rADKyy+/XNL+3e9+V/nxj3+s7Nu3T/nhD3+obNiwQdm+fbuSzWbn/dqdUzPF3/rWt7jttttobdW2p1+o7r///pJvnEOhEF1dXfT19eF2u2t6rjOhSCRCR0cH/f3956RT3LmeP5z7z0Hkf3YVDofp7OwsmRVdrCwWCyP+4ZJ9OYuKZzOXzY5V+63xfPX5z3+e733ve+zevbtk39FXv/pVXnrpJf7zP/+Trq4unnvuOT70oQ/R2trKrl27APjDP/zDYv8tW7bQ0tLCTTfdxMmTJ1m1atWS5Cu0OImxfv461691Iv+zK5H/2dW5MNZD7cb7fF5dVXvHHXfw0Y9+FIBt27bxwgsv8I1vfIPrrrtuzhjf+ta32LJlCzt27Chpf9e73lX8ecuWLVx00UWsWrWK3bt3c9NNN80rv3MGint7e3n66af5wQ9+MGu/5uZmRkdHS9pGR0dpbm6ueEylJQBut/uc/CcryOVyifzPss715yDyP7vSVVpauEB56zw1jTdf1dfXo9frq742A/zN3/wNn//853n66ae56KKLiu3JZJL/9b/+Fz/84Q+LDpQXXXQRe/fu5W/+5m+KUDxTO3fuBODEiRMCipehxFi/MJ3r1zqR/9mVyP/s6nwZ6+dSfX09BoOBjRs3lrRv2LCB3/zmN3MeH4/H+d73vldinFlJK1eupL6+nhMnTswbipd9SaaCHnnkERobG0vst7V0xRVX8Mtf/rKk7Re/+AVXXHHFUqYnJCQkJFRBJpOJSy+9tOTanM/n+eUvfznrtfmLX/win/3sZ3nqqae47LLLSh7LZDJkMpmyDxN6vb74bbSW9u7dC0BLy9lzqBeqLDHWCwkJCZ2fMplMbN++naNHj5a0Hzt2jK6urjmPf+KJJ0ilUvyP//E/5uw7MDDAxMREVWP9OTFTnM/neeSRR7jnnnswGEpTfu9730tbWxsPPfQQAH/2Z3/Gddddx9/+7d9y++23873vfY/XXnuNf/qnfzobqQsJCQkJoZZPuueee7jsssvYsWMHDz/8MPF4nPe///1A+bX8C1/4Ap/+9Kd57LHH6O7uZmRkBACHw4HD4cDlcnHdddfx8Y9/HKvVSldXF7/61a/4f//v//HlL38ZUEs/PPbYY7z1rW+lrq6Offv28dGPfpRrr722ZNZZaHlIjPVCQkJC57ZisRgnTpwo/t7T08PevXvx+Xx0dnby8Y9/nLvuuotrr72WG264gaeeeor/+q//Yvfu3cVjZl7vC/rWt77FnXfeWWaeFYvF+MxnPsM73/lOmpubOXnyJJ/4xCdYvXo1t9xyy/yTn/fu47Oon/3sZwqgHD16tOyx6667TrnnnntK2h5//HFl7dq1islkUjZt2qT85Cc/qep8siwrDzzwgCLL8mLSPmsS+Z99nevPQeR/dnWu519JX/3qV5XOzk7FZDIpO3bsUF566aXiYzOv5V1dXZqGH9PNPYaHh5X3ve99Smtrq2KxWJR169Ypf/u3f6vk83lFURSlr69PufbaaxWfz6eYzWZl9erVysc//nElHA6fqacsVIXEWF+9zvXnIPI/uxL5n12d6/lr6dlnn9Ucu6dfv7/1rW8pq1evViwWi7J161blRz/6UUkMrev9kSNHFED5+c9/XnbORCKhvOUtb1EaGhoUo9GodHV1KX/wB3+gjIyMVJX7OVenWEhISEhISEhISEhISEioVjpn9hQLCQkJCQkJCQkJCQkJCdVaAoqFhISEhISEhISEhISELlgJKBYSEhISEhISEhISEhK6YCWgWEhISEhISEhISEhISOiC1QUJxdFolPvuu4+uri6sVitXXnklr7766qzH7N69m0suuQSz2czq1at59NFHz0yyGqo2/927dyNJUtmtUOJkqfXcc8/xtre9jdbWViRJ4kc/+lHJ44qi8OlPf5qWlhasViu7du3i+PHjc8b9+te/Tnd3NxaLhZ07d/LKK6+cM/k/+OCDZX+P9evXn5X8f/CDH/CWt7yFuro6JEkq1nGdS0888QTr16/HYrGwZcsWfvrTn9Y+eZYm/0cffbTs9bdYLGc8/0wmwyc/+Um2bNmC3W6ntbWV9773vQwNDc0Z90y9/4WEzlWJsV6M9Wc7fzHWz19irNeWGOvPnC5IKP7gBz/IL37xC7797W+zf/9+3vKWt7Br1y4GBwc1+/f09HD77bdzww03sHfvXu677z4++MEP8rOf/ewMZ66q2vwLOnr0KMPDw8VbY2PjGck3Ho+zdetWvv71r2s+/sUvfpG///u/5xvf+AYvv/wydrudW265BVmWK8b8/ve/z8c+9jEeeOAB3njjDbZu3cott9zC2NjYOZE/wKZNm0r+Hr/5zW9qnjvMnX88Hufqq6/mC1/4wrxjvvDCC9x999184AMfYM+ePdx5553ceeedHDhwoFZpl+RX6/wBXC5Xyevf29tbi3Q186uUfyKR4I033uAv//IveeONN/jBD37A0aNHefvb3z5rzDP5/hcSOlclxnox1p/t/EGM9fOVGOvLJcb6M6yqCjidB0okEoper1eefPLJkvZLLrlE+d//+39rHvOJT3xC2bRpU0nbXXfdpdxyyy1LlmclLST/Qs2wYDB4BjKcXYDywx/+sPh7Pp9XmpublS996UvFtlAopJjNZuW73/1uxTg7duxQPvShDxV/z+VySmtrq/LQQw8tSd4F1Sr/Bx54QNm6desSZqqtmflPV09PjwIoe/bsmTPO7/3e7ym33357SdvOnTuVP/qjP6pBlpVVq/wfeeQRxe121zS3+Wi2/At65ZVXFEDp7e2t2Odsvf+FhM4VibH+7EqM9arEWL8wibFelRjrz6wuuJnibDZLLpcrWz5htVorfnv34osvsmvXrpK2W265hRdffHHJ8qykheRf0LZt22hpaeHmm2/m+eefX8o0562enh5GRkZKXl+3283OnTsrvr7pdJrXX3+95BidTseuXbvO+N9kIfkXdPz4cVpbW1m5ciXvec976OvrW+p0a6bl9D+xUMViMbq6uujo6OCOO+7g4MGDZzslAMLhMJIk4fF4NB9fTu9/IaHlKjHWi7G+lhJj/ZTEWF8bibF++emCg2Kn08kVV1zBZz/7WYaGhsjlcnznO9/hxRdfZHh4WPOYkZERmpqaStqampqIRCIkk8kzkXZRC8m/paWFb3zjG/zHf/wH//Ef/0FHRwfXX389b7zxxhnNXUuFvU5ar2+lfVB+v59cLlfVMUulheQPsHPnTh599FGeeuop/vEf/5Genh6uueYaotHokuZbK1X6nzjTr/9CtW7dOv7lX/6FH//4x3znO98hn89z5ZVXMjAwcFbzkmWZT37yk9x99924XC7NPsvp/S8ktFwlxnox1tdSYqyf0rk01oixXqgaGc52AmdD3/72t7n33ntpa2tDr9dzySWXcPfdd/P666+f7dTmpWrzX7duHevWrSv+fuWVV3Ly5Em+8pWv8O1vf/tMpS00Tbfddlvx54suuoidO3fS1dXF448/zgc+8IGzmNmFoSuuuIIrrrii+PuVV17Jhg0b+OY3v8lnP/vZs5JTJpPh937v91AUhX/8x388KzkICZ1PEmO9GOvPtsRYf3YlxnqhanTBzRQDrFq1il/96lfEYjH6+/t55ZVXyGQyrFy5UrN/c3Mzo6OjJW2jo6O4XC6sVuuZSLlE1eavpR07dnDixIklzHJ+am5uBtB8fQuPzVR9fT16vb6qY5ZKC8lfSx6Ph7Vr1y6Lv8l8VOl/4ky//rWS0Wjk4osvPmuvf2GQ7O3t5Re/+EXFb45heb3/hYSWs8RYL8b6WkmM9VM6l8caMdYLzaYLEooLstvttLS0EAwG+dnPfsYdd9yh2e+KK67gl7/8ZUnbL37xi5Jvn86G5pu/lvbu3UtLS8sSZjc/rVixgubm5pLXNxKJ8PLLL1d8fU0mE5deemnJMfl8nl/+8pdn/G+ykPy1FIvFOHny5LL4m8xHy/V/YqHK5XLs37//rLz+hUHy+PHjPP3009TV1c3afzm9/4WEzgWJsf7sjytirFclxvqzKzHWC82qs+30dTb01FNPKf/93/+tnDp1Svn5z3+ubN26Vdm5c6eSTqcVRVGUT33qU8rv//7vF/ufOnVKsdlsysc//nHl8OHDyte//nVFr9crTz311DmR/1e+8hXlRz/6kXL8+HFl//79yp/92Z8pOp1Oefrpp89IvtFoVNmzZ4+yZ88eBVC+/OUvK3v27Ck67n3+859XPB6P8uMf/1jZt2+fcscddygrVqxQkslkMcaNN96ofPWrXy3+/r3vfU8xm83Ko48+qhw6dEj5wz/8Q8Xj8SgjIyPnRP5//ud/ruzevVvp6elRnn/+eWXXrl1KfX29MjY2dsbzn5iYUPbs2aP85Cc/UQDle9/7nrJnzx5leHi4GOP3f//3lU996lPF359//nnFYDAof/M3f6McPnxYeeCBBxSj0ajs37//nMj/M5/5jPKzn/1MOXnypPL6668r73rXuxSLxaIcPHjwjOafTqeVt7/97Up7e7uyd+9eZXh4uHhLpVLFGGfz/S8kdK5KjPVirD/b+Yux/uzmL8Z6oWp0QULx97//fWXlypWKyWRSmpublQ996ENKKBQqPn7PPfco1113Xckxzz77rLJt2zbFZDIpK1euVB555JEzm/Q0VZv/F77wBWXVqlWKxWJRfD6fcv311yvPPPPMGcu3UCZi5u2ee+5RFEUtdfCXf/mXSlNTk2I2m5WbbrpJOXr0aEmMrq4u5YEHHihp++pXv6p0dnYqJpNJ2bFjh/LSSy+dM/nfddddSktLi2IymZS2tjblrrvuUk6cOHFW8n/kkUc0H5+e73XXXVfsX9Djjz+urF27VjGZTMqmTZuUn/zkJ+dM/vfdd1/xvdPU1KS89a1vVd54440znn+htITW7dlnny3GOJvvfyGhc1VirBdj/dnOX4z1Zzd/MdYLVSNJURRl7vlkISEhISEhISEhISEhIaHzTxf0nmIhISEhISEhISEhISGhC1sCioWEhISEhISEhISEhIQuWAkoFhISEhISEhISEhISErpgJaBYSEhISEhISEhISEhI6IKVgGIhISEhISEhISEhISGhC1YCioWEhISEhISEhISEhIQuWAkoFhISEhISEhISEhISErpgJaBYSEhISEhISEhISEhI6IKVgGKhmuv666/nvvvuO9tpVNTRo0dpbm4mGo1W7PPoo4/i8XjOXFJnUadPn0aSJPbu3VuTeE899RTbtm0jn8/XJJ6QkJCQ0PKTGOvPLYmxXkhodgkoFjonNDw8zLvf/W7Wrl2LTqerOBA/8cQTrF+/HovFwpYtW/jpT39a1uf+++/nIx/5CE6ns6Y5dnd38/DDD9c05lyqxSDX0dHB8PAwmzdvrklOt956K0ajkX/7t3+rSTwhISEhoQtDYqzXlhjrhYSWXgKKhc4JpVIpGhoa+D//5/+wdetWzT4vvPACd999Nx/4wAfYs2cPd955J3feeScHDhwo9unr6+PJJ5/kfe973xnKfPlLr9fT3NyMwWCoWcz3ve99/P3f/33N4gkJCQkJnf8SY/3SSYz1QkKzS0Cx0JIrGAzy3ve+F6/Xi81m47bbbuP48eMlff75n/+Zjo4ObDYb73jHO/jyl79csqSpu7ubv/u7v+O9730vbrdb8zx/93d/x6233srHP/5xNmzYwGc/+1kuueQSvva1rxX7PP7442zdupW2traSYx999FE6OzuL55+YmCh5/OTJk9xxxx00NTXhcDjYvn07Tz/9dPHx66+/nt7eXj760Y8iSRKSJAEwMTHB3XffTVtbGzabjS1btvDd7363JPa///u/s2XLFqxWK3V1dezatYt4PF58/P/+3//Lhg0bsFgsrF+/nn/4h38oPrZixQoALr74YiRJ4vrrr6/4N3jPe95DQ0MDVquVNWvW8MgjjwDl30C/733vKz6H6bfdu3cD6oeWv/iLv6CtrQ273c7OnTuLjxX0tre9jddee42TJ09q5iMkJCT0/7d39zE1xX8cwN+JW4drkbqFyDyUtModY61Ns1Bq18U8jjyFmJLRjFFD8Y8rzNMwf9iwZmQzVHI913XJaliXNipmSXnYhDzcfX5/+Dlz3OTnN0b1fm33j+/3fL/n+zn37u5zv/ec8z3UtjDXM9cTtWacFNMfN2/ePJSWluL06dOw2WwQEcTHx+PTp08AgOLiYixZsgRpaWkoLy/H2LFjsXnz5l8ex2azYcyYMZq62NhY2Gw2tXzt2jUMHz5c08ZutyMpKQkpKSkoLy/H6NGjkZ2drWnT2NiI+Ph4WK1WlJWVIS4uDiaTCY8fPwYA5OXlISAgAJs2bUJtbS1qa2sBAE1NTRg2bBjOnj2Le/fuYfHixUhMTMTNmzcBfLlUbObMmViwYAEcDgcuX76MyZMnQ0QAAEePHkVmZiY2b94Mh8OBLVu2ICMjA4cPHwYAdT8XLlxAbW0t8vLymn1vMjIyUFFRgfz8fDgcDuzbtw8+Pj7Ntt25c6d6DLW1tUhLS4PBYMDgwYMBACkpKbDZbMjNzcWdO3cwdepUxMXFaX789O3bF35+frh27dqPPi4iImpDmOuZ64laNSH6zaKjoyUtLU1ERCorKwWAFBcXq9sbGhpEURQ5fvy4iIhMnz5dEhISNPuYNWuWeHl5/XT/3+rUqZMcO3ZMU7dnzx4xGAxqOSIiQjZt2qRpM3PmTImPj9fUTZ8+/YfjfxUaGiq7du1Sy4GBgbJ9+/YW+4iIJCQkyKpVq0RE5Pbt2wJAqqurm207YMAAl2PKysqSyMhIERGpqqoSAFJWVtbimCaTSebPn9/stpb2cfLkSfH09JTr16+LiEhNTY24u7vL06dPNe1iYmJk7dq1mjqj0SgbNmxoMS4iImqdmOtbxlxP1LrwTDH9UQ6HAx07dsTIkSPVuh49eiA4OBgOhwPAlxUiR4wYoen3ffl3ef/+PTw9PV1i/DY+AIiMjNSUGxsbkZ6ejpCQEHTr1g16vR4Oh0P99/hHnE4nsrKyEBYWBm9vb+j1ehQWFqr9IiIiEBMTg7CwMEydOhUHDx7Eq1evAABv377Fw4cPkZSUBL1er76ys7N/+VKlpUuXIjc3F0OHDsXq1atRUlLy0z5lZWVITEzE7t27ERUVBQC4e/cunE4ngoKCNDFduXLFJSZFUfDu3btfipOIiFof5nrmeqLW7vfdbU/0l/n7+6Ourk5TV1dXB39/f7Xs4+OjJqJfkZ6ejqKiIlgsFgwcOBCKomDKlCn4+PFji/22bt2KnTt3YseOHQgLC0OXLl2wYsUKtZ+7uzuKiopQUlKC8+fPY9euXVi3bh3sdjs6d+4M4Ms9WN8ncnd391+Kf/z48aipqcG5c+dQVFSEmJgYLFu2DBaLpdn2z549w4QJE7Bw4UIkJSWp9Y2NjXB3d8ft27ddYtDr9Zryy5cv4evr+0txEhERtYS5/seY64n+fzxTTH9USEgIPn/+DLvdrta9ePECDx48wJAhQwAAwcHBuHXrlqbf9+X/RWRkJKxWq6auqKhI80+w0WhERUWFS4zfxgcAN27c0JSLi4sxb948TJo0CWFhYfD390d1dbWmjU6ng9PpdOlnNpsxe/ZsREREoH///qisrNS0cXNzQ1RUFDZu3IiysjLodDqcOnUKfn5+6NWrFx49eoSBAwdqXl8X3dDpdADgMm5zfH19MXfuXBw5cgQ7duzAgQMHmm3X1NQEs9mMwYMHIycnR7PNaDTC6XTi+fPnLjF9+4OkqakJDx8+hNFo/GlcRETUujHXM9cTtXY8U0x/1KBBg2A2m7Fo0SLs378fXbt2xZo1a9C7d2+YzWYAQGpqKkaNGoWcnByYTCZcvHgR+fn56qqOX31dMbGxsRH19fUoLy+HTqdTE25aWhqio6Oxbds2JCQkIDc3F6WlpZqEEBsbi4ULF8LpdKr/fi5fvhxRUVGwWCwwm80oLCxEQUGBy3Hk5eXBZDLBzc0NGRkZLg+s79evH65evYoZM2bAw8MDPj4+GDRoEE6cOIGSkhJ0794dOTk5qKurU2O22+2wWq0YN24cDAYD7HY76uvrERISAgDYuHEjli9fDi8vL8TFxeHDhw8oLS3Fq1evsHLlShgMBiiKgoKCAgQEBMDT07PZFTszMzMxbNgwhIaG4sOHDzhz5ow6xveSk5Px5MkTWK1W1NfXq/Xe3t4ICgrCrFmzMGfOHGzbtg1GoxH19fWwWq0IDw9HQkICgC8/NDw8PFwuTSMioraHuZ65nqjV+9s3NVPb8/3iGC9fvpTExETx8vISRVEkNjZWKisrNX0OHDggvXv3FkVRZOLEiZKdnS3+/v6aNgBcXoGBgZo2x48fl6CgINHpdBIaGipnz57VbP/06ZP06tVLCgoKNPWHDh2SgIAAURRFTCaTWCwWzeIbVVVVMnr0aFEURfr06SO7d+92OU6bzSbh4eHi4eEhX79aL168ELPZLHq9XgwGg6xfv17mzJkjZrNZREQqKiokNjZWfH19xcPDQ4KCgjQLeoiIHD16VIYOHSo6nU66d+8uo0aNkry8PHX7wYMHpU+fPtKhQweJjo5u9jPJysqSkJAQURRFvL29xWw2y6NHj9RjwzeLbwQGBjb7Xl+6dElERD5+/CiZmZnSr18/6dSpk/Ts2VMmTZokd+7cUcdbvHixJCcnNxsLERG1fsz1zPXM9dSWuIn8dz14on/IokWLcP/+/T+yzP+ePXtw+vRpFBYW/vZ9E9DQ0IDg4GCUlpaql34RERF9j7m+9WKup7aGl0/TP8FisWDs2LHo0qUL8vPzcfjwYc2D63+n5ORkvH79Gm/evEHXrl3/yBjtWXV1Nfbu3cskSUREGsz1bQdzPbU1PFNM/4Rp06bh8uXLePPmDfr374/U1FQsWbLkb4dFREREvwlzPRH9qzgpJiIiIiIionaLj2QiIiIiIiKidouTYiIiIiIiImq3OCkmIiIiIiKidouTYiIiIiIiImq3OCkmIiIiIiKidouTYiIiIiIiImq3OCkmIiIiIiKidouTYiIiIiIiImq3/gOG9SaDLikyzQAAAABJRU5ErkJggg==\",\n      \"text/plain\": [\n       \"<Figure size 1200x500 with 4 Axes>\"\n      ]\n     },\n     \"metadata\": {},\n     \"output_type\": \"display_data\"\n    }\n   ],\n   \"source\": [\n    \"def L(N, D):\\n\",\n    \"    \\\"\\\"\\\" \\n\",\n    \"    Approximates loss given N parameters and D dataset size (in tokens),\\n\",\n    \"    per Chinchilla paper.\\n\",\n    \"    \\\"\\\"\\\"\\n\",\n    \"    E = 1.69 # entropy of natural language, limit of infinite model on infinite data\\n\",\n    \"    A = 406.4\\n\",\n    \"    B = 410.7\\n\",\n    \"    alpha = 0.34\\n\",\n    \"    beta = 0.28\\n\",\n    \"    return A / (N ** alpha) + B / (D ** beta) + E\\n\",\n    \"\\n\",\n    \"ns = 10 ** np.arange(7, 11, step=2**-4) # model sizes from 10M to 100B\\n\",\n    \"ds = 10 ** np.arange(9, 12, step=2**-4) # dataset sizes from 1B to 1T\\n\",\n    \"plt.figure(figsize=(12, 5))\\n\",\n    \"plt.subplot(121)\\n\",\n    \"# create a 2D countour plot of loss L as a function of model size and dataset size in ns,ds\\n\",\n    \"loss2d = np.log10(np.array([[L(n, d) for d in ds] for n in ns]))\\n\",\n    \"plt.imshow(loss2d, extent=[9, 12, 7, 11], origin='lower', alpha=0.5)\\n\",\n    \"plt.contour(loss2d, levels=30, extent=[9, 12, 7, 11], origin='lower')\\n\",\n    \"plt.xlabel('log10(dataset size)')\\n\",\n    \"plt.ylabel('log10(model size)')\\n\",\n    \"plt.title('loss')\\n\",\n    \"plt.colorbar()\\n\",\n    \"# plot the compute for each point, which is a deterministic function: flops = 6*N*D\\n\",\n    \"plt.subplot(122)\\n\",\n    \"compute2d = np.log10(np.array([[6*n*d for d in ds] for n in ns]))\\n\",\n    \"plt.imshow(compute2d, extent=[9, 12, 7, 11], origin='lower', alpha=0.5)\\n\",\n    \"plt.contour(compute2d, levels=30, extent=[9, 12, 7, 11], origin='lower')\\n\",\n    \"plt.xlabel('log10(dataset size)')\\n\",\n    \"plt.ylabel('log10(model size)')\\n\",\n    \"plt.title('log10 flops')\\n\",\n    \"plt.colorbar()\"\n   ]\n  },\n  {\n   \"attachments\": {},\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"Ok so given any N,D we can estimate both: 1) the loss, and 2) the total flops. Now we want to solve the following problem: Given a specific budget of flops C, find: N_opt, D_opt = argmin_{FLOPs(N,D) = C} L(N, D). i.e. how big of a model should we train and for how many tokens?\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 9,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"name\": \"stdout\",\n     \"output_type\": \"stream\",\n     \"text\": [\n      \"best model size: 316.23M\\n\",\n      \"best dataset size: 11.65B\\n\"\n     ]\n    },\n    {\n     \"data\": {\n      \"text/plain\": [\n       \"Text(0, 0.5, 'loss')\"\n      ]\n     },\n     \"execution_count\": 9,\n     \"metadata\": {},\n     \"output_type\": \"execute_result\"\n    },\n    {\n     \"data\": {\n      \"image/png\": \"iVBORw0KGgoAAAANSUhEUgAAAS8AAAEqCAYAAABEE9ZrAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAuRElEQVR4nO3dd3xUVf7/8dekTdpMGpCEFEACCQETQhRIQEAFqVJ2XV0WBV3Erx1FccV1v7qiht8itoW14E/UXVlAFNgfRdpSFEJvoQiElkIKhJDJJKTN3N8fQ0aiBEKYzJ3yeT4e9/FI7tyZ+SRh3px77rnnaBRFURBCCCfjoXYBQgjRHBJeQginJOElhHBKEl5CCKck4SWEcEoSXkIIpyThJYRwSl5qF2BvZrOZs2fPotPp0Gg0apcjhPgFRVEoLy+nbdu2eHg03r5yu/A6e/YsMTExapchhLiO3NxcoqOjG33c7cJLp9MBll+MXq9XuRoXVlEBbdtavj57FgIC1K1HOA2DwUBMTIz1s9oYtwuv+lNFvV4v4dWSPD1//lqvl/ASN+x63TrSYS+EcEoSXkIIpyThJYRwShJeQginJOElhHBKEl5CCKck4SWEaHGf/3iKZfvyMVTV2uw1JbyEEC2q1mRm1pqjTF6wj5ySSpu9roSXEKJF7cu9SEWNidAAHxIjbTcwXMJLCNGifjh+HoD0jmF4eNhuMgQJLyFEi/rx+DkA7ujUyqavK+ElhGgxhqpa9ueVAdC3U2ubvraElxCixWSeKMFkVrilVQBRwX42fW0JLyFEi/nxcn9XXxufMoKElxCiBf2YbQmvPnESXkIIJ5FXWsmp8xV4emhI6xhm89eX8BJCtIgtl1tdydFB6H29bf76El5CiBbxg7W/y7ZXGetJeAkhbM5sVth6ogSw/fiuehJeQgibO1xg4EJFDYFaL7rHBLfIe0h4CSFsbtMxy6j63reE4u3ZMjEj4SWEsLn68Oof36bF3kPCSwhhU+VVtew5UwpA/xbqrAcJLyGEjW09UULd5VuCYsP8W+x9JLyEEDZVf8rYr3PLtbpAwksIYUOKorDpaH1/l4SXEMJJnDhXQf7FS/h4edC7g+1vCbqShJcQwmbqTxl7dQjFz8ezRd9LwksIYTPWIRIt3N8FEl5CCBupqjWx/aTllqABLdzfBRJeQggb2XayhOo6M1HBfnRsHdji7yfhJYSwiSuHSGg0tlslqDESXkIIm9jwUzFgn/4ucKDwmjFjBhqNhueee67RY7744gs0Gk2DzdfX135FCiGu6uQ5I6dLKvH21LTIfPVX42WXd7mOnTt38sknn5CUlHTdY/V6PUePHrV+b4/mqRDi2v57udXV+5YwArX2iRXVW15Go5Fx48Yxd+5cQkJCrnu8RqMhIiLCuoWHh9uhSiHEtdSH150tOIvEL6keXk899RTDhw9n4MCBTTreaDTSrl07YmJiGDVqFIcOHbrm8dXV1RgMhgabEMJ2yqtq2XHqAgB3JbhJeC1YsIA9e/aQkZHRpOPj4+P5/PPPWbZsGf/6178wm82kp6eTl5fX6HMyMjIICgqybjExMbYqXwiBZW3G+lkk2rcKsNv7qhZeubm5TJ48ma+//rrJne5paWmMHz+e7t27079/f7777jtat27NJ5980uhzpk2bRllZmXXLzc211Y8ghODnU0Z7trpAxQ773bt3U1xcTI8ePaz7TCYTmzdvZvbs2VRXV+Ppee17o7y9vUlJSSE7O7vRY7RaLVqt1mZ1CyF+ZjYrbDjqZuF19913k5WV1WDfI488QkJCAn/605+uG1xgCbusrCyGDRvWUmUKIa4hK7+M80bLQhu3tQ+163urFl46nY5u3bo12BcQEEBYWJh1//jx44mKirL2ib3xxhv07t2buLg4Ll68yMyZMzlz5gyPPvqo3esXQvx8ynhHp1b4eNm3F8ohxnk1JicnBw+Pn38hpaWlTJo0icLCQkJCQkhNTWXr1q0kJiaqWKUQ7kutU0YAjaIoit3fVUUGg4GgoCDKysrQ6/Vql+O6Kiog8PLNuUYjBNjvKpSwj2JDFT3fXo9GAzteGUhrnW36lpv6GVV9nJcQwjmtO2JpdXWPCbZZcN0ICS8hRLOsPVwIwMAu6tzlIuElhLhhFdV1bDlhmXjwnkQJLyGEk/jh+Dlq6sy0C/Mnrk3LTzx4NRJeQogbtuZwEQCDuoSrNrOLhJcQ4obUmczWiQcHqnTKCBJeQogbtPtMKaWVtQT7e3Nbu+tPY9VSJLyEEDdk7eVTxrvi2+DlqV6ESHgJIZpMURTWHrnc36XiKSNIeAkhbkB2sZEzJZX4eHpwh50W2miMhJcQosnqrzKmdbTfXPWNkfASQjTZ9wcto+qHdItQuRIJLyFEE+WVVpKVX4aHRv3+LpDwEkI00epDllPG29uH0ipQ/dmJJbyEEE3y/cECwDFOGUHCSwjRBMXlVew6UwrA4K4SXkIIJ7H2cBGKAskxwbQN9lO7HEDCSwjRBNarjA7S6gIJLyHEdZRV1pJ5ee6uwV3Vv8pYT8JLCHFN644UUWdWiA/XcUtrdebuuhoJLyHENa26fMo42EGuMtaT8BJCNKq8qpbNx84BMFTCSwjhLNYdKaLGZKZj6wASInRql9OAhJcQolErDlgGpg5PaqvadM+NkfASQlxV2aVaNh87D8CIpEiVq/k1CS8hxFWtO2w5ZezUJpDO4Y51yggSXkKIRqzIqj9ldLxWF0h4CSGuoqyylh+OW64yDr9VwksI4STWHC6k1mQZmNrJAU8ZQcJLCHEV9aeMwxy01QUSXkKIX7hYWcOPxy1XGYcnOdbA1CtJeAkhGliZVUidWSEhQkdcG8c8ZQQJLyHELyzblw/A6JQolSu5NgkvIYTV2YuX2HH6AgD3JrdVuZprk/ASQlgtP3AWRYGe7UOJcpAZUxsj4SWEsPrP/rMAjOzu2K0ukPASQlyWXWzkYL4BLw+NQw+RqCfhJYQAfm519evcmtAAH5WruT4JLyEEiqLwn8tXGUc5wSkjSHgJIYADeWWcLqnEz9uTgV0cZ5GNa5HwEkKwZK+l1TUoMZwArZfK1TSNhJcQbq7WZLb2d43p4dgDU68k4SWEm9t49BwXKmpordNyR1wrtctpMgkvIdzcd3vyABjdvS1ens4TCQ5T6YwZM9BoNDz33HPXPO6bb74hISEBX19fbr31VlauXGmfAoVwQRcra1h/pBiA3/SIVrmaG+MQ4bVz504++eQTkpKSrnnc1q1bGTt2LBMnTmTv3r2MHj2a0aNHc/DgQTtVKoRrWX6ggBqTmS6RerpE6tUu54aoHl5Go5Fx48Yxd+5cQkJCrnnsBx98wJAhQ5g6dSpdunRh+vTp9OjRg9mzZzf6nOrqagwGQ4NNCGFRf8r4WyfqqK+neng99dRTDB8+nIEDB1732MzMzF8dN3jwYDIzMxt9TkZGBkFBQdYtJibmpmsWwhWcPGdkT85FPD00TnEv4y+pGl4LFixgz549ZGRkNOn4wsJCwsMbDqALDw+nsLCw0edMmzaNsrIy65abm3tTNQvhKurHdvXr1Io2Ol+Vq7lxqo1Gy83NZfLkyaxduxZf35b7xWm1WrRabYu9vhDOyGRW+G6PJbzGOFlHfT3Vwmv37t0UFxfTo0cP6z6TycTmzZuZPXs21dXVeHp6NnhOREQERUVFDfYVFRUREeG482wL4Yi2ZJ8n/+Il9L5e3JPoHLcD/ZJqp4133303WVlZ7Nu3z7rddtttjBs3jn379v0quADS0tJYv359g31r164lLS3NXmUL4RIW7bJ0n4xOicLX+9efNWegWstLp9PRrVu3BvsCAgIICwuz7h8/fjxRUVHWPrHJkyfTv39/Zs2axfDhw1mwYAG7du3i008/tXv9Qjir0ooa1hyynMHcf5vzXsBS/WrjteTk5FBQUGD9Pj09nfnz5/Ppp5+SnJzM4sWLWbp06a9CUAjRuGX78qkxmUmM1NMtKkjtcppNoyiKonYR9mQwGAgKCqKsrAy93rkG5TmVigoIDLR8bTRCQIC69QjAMm/XsA9/5EiBgb+O7MqE9PZql/QrTf2MOnTLSwhhW4fOGjhSYMDHy8NpJh1sjISXEG5k4U5LR/3grhEE+zv+VM/XIuElhJuoqjVZF5R9wIk76utJeAnhJlYcKMBQVUdUsB/pHcPULuemSXgJ4Sbm78gB4A+9YvHw0Khczc1rVnh9+eWXrFixwvr9Sy+9RHBwMOnp6Zw5c8ZmxQkhbOOnQgO7z5Ti5aHhd7c55+1Av9Ss8Hr77bfx87MsBZ6ZmcmcOXP429/+RqtWrXj++edtWqAQ4ubN325pdQ1KDHfKm7Cvplkj7HNzc4mLiwNg6dKl/Pa3v+Wxxx6jT58+DBgwwJb1CSFuUmVNHUsu34Q9rlc7lauxnWa1vAIDAykpKQFgzZo1DBo0CABfX18uXbpku+qEEDft/+0/S3l1He3C/F2io75es1pegwYN4tFHHyUlJYVjx44xbNgwAA4dOkT79u1tWZ8Q4iZ9ffmU8Q89XaOjvl6zWl5z5swhLS2Nc+fO8e233xIWZknz3bt3M3bsWJsWKIRovqy8Mg7kleHj6cF9qa7RUV+vWS2v4ODgq84b/9e//vWmCxJC2M6/tlmu/g/uFkFYoGtNytmsltf333/Pjz/+aP1+zpw5dO/enT/84Q+UlpbarDghRPNdrKxh6eUR9ePTXKejvl6zwmvq1KnWVXiysrJ44YUXGDZsGKdOnWLKlCk2LVAI0TyLduVSXWeZ+ua2dtdemcsZNeu08dSpUyQmJgLw7bffMmLECN5++2327Nlj7bwXQqjHZFb4KtNyyjghvR0ajet01NdrVsvLx8eHyspKANatW8c999wDQGhoqKyLKIQD2PBTMXmllwjy82ZksvOtydgUzWp59e3blylTptCnTx927NjBwoULATh27BjR0a51RUMIZ/Rl5mkAfn97DH4+zjlH/fU0q+U1e/ZsvLy8WLx4MR999BFRUZZkX7VqFUOGDLFpgUKIG5NdbOSH4+fRaODB3q7XUV+vWS2v2NhYli9f/qv977333k0XJIS4OfXDI+5OaENMqL/K1bScZq8eZDKZWLp0KUeOHAGga9eujBw58qpLlgkh7MNQVcs3l5c1G5/WXt1iWlizwis7O5thw4aRn59PfHw8ABkZGcTExLBixQo6duxo0yKFEE2zcEcuFTUmOrUJ5I5OrdQup0U1q8/r2WefpWPHjuTm5rJnzx727NlDTk4OHTp04Nlnn7V1jUKIJqgzmZm35RQAj97RwSWHR1ypWS2vTZs2sW3bNkJDQ637wsLCmDFjBn369LFZcUKIplt1sJCzZVWEBfgwqrtrDo+4UrNaXlqtlvLy8l/tNxqN+Pg494okQjgjRVH47EdLq+vB3u3w9Xb9vudmhdeIESN47LHH2L59O4qioCgK27Zt4/HHH2fkyJG2rlEIcR17ckrZn3sRHy8Plx4ecaVmhdeHH35Ix44dSUtLw9fXF19fX9LT04mLi+P999+3cYlCiOv57AdLq2tM9yha61xr9ojGNHtKnGXLlpGdnW0dKtGlSxfr1NBCCPvJKalk9aFCACbe0UHlauynyeF1vdkiNmzYYP363XffbX5FQogbMveHk5gV6Ne5NZ3DdWqXYzdNDq+9e/c26ThXvzwrhCM5b6xm0eVBqU/0d6/xlU0OrytbVkIIx/DFltNU15npHhNM71tCr/8EFyIrZgvhpIzVdXx1efaIx/t3dLuzHgkvIZzUv7fnYKiq45bWAdyTGK52OXYn4dWI7GIj0747YP2fTQhHUl1n4rMfTwLweL+OLrWkWVNJeDVi1+kL/HtHLp9uPonJrKhdjhANLN2bT5Ghmgi9L6NS2qpdjiokvBoxOiWKEH9v8kovsfZwkdrlCGFVZzLzj40nAJjYtwNaL9e/FehqJLwa4evtydiesQDWO/WFcAT/2X+WMyWVhAb4MK53rNrlqEbC6xoeSmuHp4eG7acucOhsmdrlCIHJrDD7v9mAZdobf59mzyfq9CS8riEyyI+h3SIAmLfltLrFCAEsP3CWk+crCPb3dvmZUq9Hwus6HuljuVfsP/vOct5YrXI1wp2Zr2h1TezTgUCt+7a6QMLrunrEBpMcHUSNycz87TlqlyPc2PeHCjlebETn68WEPu3VLkd1El7XodFo+GNfS+vrn9vOUFNnVrki4Y7MZoUP1x8HLGcDel9vlStSn4RXEwztFkm4Xsu58mqW7ctXuxzhhlZkFfBTYTk6rRd/lFYXIOHVJD5eHvzxct/Xp5tPYpZBq8KO6kxm3lt3DIBH77iFYH+Zah1UDq+PPvqIpKQk9Ho9er2etLQ0Vq1a1ejxX3zxBRqNpsHm6+trl1rH9opFp/XieLGRDUeL7fKeQgAs3XeWk+csVxj/2Le92uU4DFXDKzo6mhkzZrB792527drFXXfdxahRozh06FCjz9Hr9RQUFFi3M2fO2KVWva83f7g8IPCTTSft8p5C1NSZ+WC9pdX1eP+O6KSvy0rV8Lr33nsZNmwYnTp1onPnzrz11lsEBgaybdu2Rp+j0WiIiIiwbuHh9rub/o99OuDtqWHH6QvsPlNqt/cV7mvRrlxyL1yiVaCWCW4+ruuXHKbPy2QysWDBAioqKkhLS2v0OKPRSLt27YiJibluKw2guroag8HQYGuucL0vY1Is6+F9sulEs19HiKaoqjVZx3U9fWdH/Hzc8x7GxqgeXllZWQQGBqLVann88cdZsmQJiYmJVz02Pj6ezz//nGXLlvGvf/0Ls9lMeno6eXl5jb5+RkYGQUFB1i0mJuam6n2s3y0ArD1SRHax8aZeS4hr+SrzNIWGKtoG+TK2l/vew9gYjaIoql46q6mpIScnh7KyMhYvXsxnn33Gpk2bGg2wK9XW1tKlSxfGjh3L9OnTr3pMdXU11dU/j4w3GAzExMRQVlaGXq9vVs2TvtrF2sNF3JcazTu/S27Wa7i8igoIDLR8bTRCQIC69TiZi5U19PvbBgxVdcy8L4nf3XZz/+k6E4PBQFBQ0HU/o6q3vHx8fIiLiyM1NZWMjAySk5P54IMPmvRcb29vUlJSyM7ObvQYrVZrvZpZv92sp+60LPG2ZG8+OSWVN/16QvzSPzaewFBVR0KEjt/0iFa7HIekenj9ktlsbtBSuhaTyURWVhaRkZEtXFVD3WOC6de5NSazwj82Nh6cQjRHXmklX1yeCOBPQxPwdMNZUptC1fCaNm0amzdv5vTp02RlZTFt2jQ2btzIuHHjABg/fjzTpk2zHv/GG2+wZs0aTp48yZ49e3jwwQc5c+YMjz76qN1rn3y3pfW1eHceeaXS+hK28+6aY9SYzKR3DGNA59Zql+OwVL0tvbi4mPHjx1NQUEBQUBBJSUmsXr2aQYMGAZCTk4OHx8/5WlpayqRJkygsLCQkJITU1FS2bt3apP4xW0ttF0qfuDC2ZJfw0cYTvDXmVrvXIFzPobNlLLl8C9q0oV3cbkWgG6F6h729NbUzsCm2nyzhgU+34e2pYdPUO2kb7GejKl2AdNjfMEVRGPfZdraeKGFkcls+HJuidkmqcJoOe2fW65YwenUIpdak8LGM+xI3ac3hIraeKMHHy4Opg+PVLsfhSXjdpMl3dwJgwY5c6fsSzVZdZ+LtlUcAmHRHB2JC/VWuyPFJeN2ktI5hpN0SRo3JbJ1vSYgbNW/Lac6UVNJGp+XJAXFql+MUJLxukkajYeoQSxN/8e48TpyTUffixpwrr7beBvSnIQkEuPn0zk0l4WUDPWJDGNglHLMC7649pnY5wsm8s/ooxuo6kmOCrffOiuuT8LKRFwd3RqOBFQcKOJgvy6SJptmbU8qi3bkA/O+IRDxkQGqTSXjZSEKEnlHJlmXX31lzVOVqhDMwmRX+suwgigK/7RFNarsQtUtyKhJeNvT8oM54eWjYePQcmSdK1C5HOLivt5/hYL4Bva8X04YlqF2O05HwsqF2YQH84fLUJW+tPCxz3YtGnSuvZuZqSwt96pAEWgVqVa7I+Uh42djkuzuh03pxMN/Akr2y0pC4uoyVRyivqiMpOog/9JS5uppDwsvGwgK1PH2XZZzOzNVHqaypU7ki4WgyT5Tw3d58NBqYPqqbzBrRTBJeLWBCenuiQ/woNFQxd/MptcsRDqSq1sS07w4AMK5XLMkxweoW5MQkvFqAr7cnLw+1dMB+vOkERYYqlSsSjuL9dcc5XVJJhN6Xl4ZIJ/3NkPBqIcNvjaRHbDCXak38n1U/qV2OcAAH88uY+4Nl2bzpo7uhl2XMboqEVwvRaDT8771d0Wjgu7357Dx9Qe2ShIrqTGZe/u4AJrPC8KRIBiXab8k+VyXh1YK6xwTz+9stCyf8ZelB6kxmlSsSapn7wykO5hsI8vPm9Xu7ql2OS5DwamFTBycQ7O/NT4Xl/HObfVb3Fo7lWFE5712+5/XV4V1orZMxXbYg4dXCQgN8rBPLvbvmGMXl0nnvTmpNZqYs2keNycxdCW24L1VWArIVCS87+P3tsSRFB1FeXceMldJ5705m/zebg/kGgv29mfGbW2VOehuS8LIDTw8Nb4zqZu2833zsnNolCTs4kHeR2Rss83RNH9WNNnpflStyLRJedtI9JpgJae0BeGVJFhXVMvLelV2qMTFl0X5MZoURSZHce3nGEWE7El52NHVwPFHBfuSVXmLWGpm00JW9ueIw2cVGWuu0TB/VTe1yXJKElx0FaL14a4zlH/K8rafYm1OqckWiJXx/sJCvt+eg0cB793cnJMBH7ZJckoSXnQ2Ib8NvUqJQFHj52yxq6mTslys5e/ESf/rWcu/iY/1uoW+nVipX5LokvFTwlxGJhAX4cLSonPfXyemjqzCZFZ5buI+yS7UkRwfxwiBZe7ElSXipICTAx3r6+PGmE+ySW4dcwgfrjrHj1AUCfDz5cGwKPl7y8WpJ8ttVyZBukfy2RzRmBaYs2o9Rrj46tQ0/FfPh5eXL3hpzK+3CAlSuyPVJeKnotZGJRAX7kXOhkjeXH1a7HNFMuRcqeW7hPgAe6t2O0bJ8mV1IeKlI7+vNrPuT0Whgwc5c1hwqVLskcYOqak08+fUeSz9XTDCvjuiidkluQ8JLZb1vCWPSHbcA8NK3B8i/eEnlikRTKYrC6/85RFZ+GSH+3vxjXA+0Xp5ql+U2JLwcwAv3dCYpOoiLlbU8M38PtTJ1jlP4cutpFuzMRaOB93+fQlSwn9oluRUJLweg9fJk9tge6Hy92JNzkXdWy6K1ju7H4+eZvuIIANOGJtC/c2uVK3I/El4OIjbMn5n3JQHwyeaT/PenIpUrEo05db6CJ7/ejcms8JseUdbTfmFfEl4OZEi3SB5Obw/A8wv3c6akQt2CxK9crKxh4pc7MVTVkRIbzNtjZJobtUh4OZhpwxJIjgmm7FItk77aJeO/HEhVrYnHvtrNyXMVRAb58slDqfh6Swe9WiS8HIzWy5NPH0qljU7LsSIjLyzah9msqF2W2zObFV5YtJ8dpy+g03ox75HbaaOT+bnUJOHlgML1vnz8UCo+nh6sPlTEh/89rnZJbu+tlUdYkVWAt6eGT8ankhChV7sktyfh5aB6xIbw5uX7H99fd5zlB86qXJH7+mTTCf7vj5aVz9/5XTLpHWWmCEcg4eXA7r8thj/26QDAlIX72X6yROWK3M8/t50h4/KiwS8PTWBUd7n1x1FIeDm4Pw/vwuCu4dSYzEz6ahfZxeVql+Q2vt2dx1+WHgTgyQEdebx/R5UrEleS8HJwnh4aPvh9Cj1igzFU1THh850UG2T5tJa2KquAqYv3A/Bwenvr8nXCcUh4OQFfb08+m3A7HVoFkH/xEuM/30FpRY3aZbms5QfO8vS/92JW4Hep0fzviEQZy+WAJLycRGiAD18+0pM2Oi0/FZbz0OfbKbtUq3ZZLmfJ3jye/fdeTGaFMSlRzPhtEh4eElyOSNXw+uijj0hKSkKv16PX60lLS2PVqlXXfM4333xDQkICvr6+3HrrraxcudJO1aovNsyf+ZN6ERbgw8F8Aw/P2yGDWG1o0a5cpizaj1mB+2+L5p3fJeMpweWwVA2v6OhoZsyYwe7du9m1axd33XUXo0aN4tChQ1c9fuvWrYwdO5aJEyeyd+9eRo8ezejRozl48KCdK1dPXBsd/5zYiyA/b/bmXOSP83ZKgNnA3M0neWnxARQFHuwdy4zfJElwOTiNoigONXw7NDSUmTNnMnHixF899sADD1BRUcHy5cut+3r37k337t35+OOPm/T6BoOBoKAgysrK0Oudd6DhgbyLjJu7nfLqOpJjgvnykdsJ9negJbYqKiAw0PK10QgBjjktstms8NbKI9ZxXBP7duDV4V2kj0tFTf2MOkyfl8lkYsGCBVRUVJCWlnbVYzIzMxk4cGCDfYMHDyYzM7PR162ursZgMDTYXEFSdDBfT+pFiL83+3Mv8sAn2ygul6uQN6K6zsRzC/dZg+uVYQkSXE5E9fDKysoiMDAQrVbL448/zpIlS0hMTLzqsYWFhYSHhzfYFx4eTmFh49MnZ2RkEBQUZN1iYmJsWr+akqKDWfg/abTRaTlaVM79H2eSU1KpdllO4byxmgc/285/9p/Fy0PDew8k81i/jhJcTkT18IqPj2ffvn1s376dJ554ggkTJnD4sO0Wo5g2bRplZWXWLTc312av7Qg6h+tY/Hg60SF+nC6pZMw/trD7jCyldi2HzpYxavYWdp4uRaf14vOHb2dMSrTaZYkbpHp4+fj4EBcXR2pqKhkZGSQnJ/PBBx9c9diIiAiKihpO0ldUVERERESjr6/Vaq1XM+s3VxMb5s+3T6TTLUpPSUUNY+duZ9m+fLXLckgrDhRw30eZ5F+8RIdWASx5qg/9ZBZUp6R6eP2S2Wymurr6qo+lpaWxfv36BvvWrl3baB+ZOwnX+7Lof9IYlBhOTZ2ZyQv28e6ao5hkOh3A0r/12rKDPDV/D5dqTdzRqRVLn+xDXJtAtUsTzeSl5ptPmzaNoUOHEhsbS3l5OfPnz2fjxo2sXr0agPHjxxMVFUVGRgYAkydPpn///syaNYvhw4ezYMECdu3axaeffqrmj+Ew/H28+PjBVGasOsLcH07x4X+z2Zt7kfcf6E5YoFbt8lRz+nwFT/97DwfzLRdr/qffLUwdHI+Xp8P93y1ugKrhVVxczPjx4ykoKCAoKIikpCRWr17NoEGDAMjJycHD4+d/YOnp6cyfP59XX32VV155hU6dOrF06VK6deum1o/gcDw9NPx5eCJdIvW8siSLH46fZ/iHPzJnXAqp7ULVLs+uFEVh0a5cpi8/grG6jhB/b969vzt3JrRRuzRhAw43zquluco4r6Y4WljOE19bpi329NDw1ICOPH1XJ3y87NDiUHmcV0HZJV7+NotNx84BcHv7ED4cm0JkkCxP5uicbpyXsL34CB3/ebovo7q3xWRW+PC/2Yyes4WfCl1jrNvVmM0K/96Rwz3vbWbTsXP4eHnwyrAEFjyWJsHlYqTl5SZWHCjg1aVZlFbW4u2p4bF+t/DUnXH4+7RQz4EKLa/9uRf532UH2Z9XBkByTDCzfpdEXBtdi7+3sJ2mfkYlvNxIcXkVr3x3kHVHLMNN2gb58pcRiQzpFmH7wZl2DK+Csku8v/Y4i3bnoiig03rx/KDOjE9rJ53yTkjCqxHuHF5g6cRefaiI6csPk3/xEgA9O4Ty4j3x9Oxgww59O4RXibGajzae4KttZ6ipMwPwm5QoXh6WICv7ODEJr0a4e3jVu1Rj4qNNJ/h40wnrB/+OTq2YMqgzKbEhN/8GLRheeaWVzNtymgU7cqioMQGWAP7TkHi3u6LqiiS8GiHh1dDZi5eYvSGbRTtzqbs8oDW1XQiP9GnP4K4ReDf3tMvG4aUoCntySvkq8wzLDxRYB992i9IzdXAC/Tq1kvsSXYSEVyMkvK4up6SSv//3OEv35VNrsvyTiAzyZUxKFKNTougcfoOd3jYKryJDFUv25rNoVy4nz1VY9/eJC2PSHbfQv3NrCS0XI+HVCAmvays2VPGv7TnM336G88af58nvEqlnWLcI+se3plvboOtPjdzM8FIUhePFRtYeLmLN4SL25160Pubn7cmwWyN5pE97ukUF3eiPJpyEhFcjJLyapqrWxNrDRSzbl8/Go+esp5RgmU8/rWMYydFBdIuybHpf74Yv0MTwulBRw/GicrLyy9h5+gK7TpdS8ovFRVLbhfC71GiGJ0Wi++X7CJcj4dUICa8bV1pRw/eHCtnwUzFbT5RcddrpVoE+RIf4Ex3iR6tALSFKDZNH9wBg3vdZVPr4UlNn5kJFDefKqzlnrOb0+YpfBRWAj5cH6R3DGJQYzsAu4YTr5cqhO5HwaoSE182pNZnZm3ORnacvkJVXRlZ+mXXIxZX8aqo48t59AHR5fjGXfBoPoJhQPzq30ZHaPoSe7UO5NToIrZdni/0MwrE19TOq6o3Zwvl4e3rQs0NogzFhZZW15JZWkldaSV7pJUora6i++PMtSCOT22L298fby4OwAB9a67S0DtQSFeJHXJvAlhvlL1yatLxEy3CSBTiE45Ebs4UQLk3CSwjhlCS8hBBOScJLCOGUJLyEEE5JwksI4ZQkvIQQTsntRgfWD2szGFx3HneHUPHzDBAYDGAyqVeLcCr1n83rDUF1u/AqLy8HICYmRuVK3EjbtmpXIJxQeXk5QUGNzx7idiPszWYzZ8+eRafTueU8UAaDgZiYGHJzc+UOAyfgjn8vRVEoLy+nbdu2DdZt/SW3a3l5eHgQHR2tdhmq0+v1bvNhcAXu9ve6VournnTYCyGckoSXEMIpSXi5Ga1Wy2uvvYZWq1W7FNEE8vdqnNt12AshXIO0vIQQTknCSwjhlCS8hBBOScJLCOGUJLyEEE5Jwks08N5779G1a1cSExN59tlnr3tzrLCvMWPGEBISwn333ddg//Lly4mPj6dTp0589tlnKlVnXzJUQlidO3eO3r17c+jQIby9venXrx/vvPMOaWlpapcmLtu4cSPl5eV8+eWXLF68GIC6ujoSExPZsGEDQUFBpKamsnXrVsLCwlSutmVJy0s0UFdXR1VVFbW1tdTW1tKmTRu1SxJXGDBgADqdrsG+HTt20LVrV6KioggMDGTo0KGsWbNGpQrtR8LLhWzevJl7772Xtm3botFoWLp06a+OmTNnDu3bt8fX15devXqxY8cO62OtW7fmxRdfJDY2lrZt2zJw4EA6duxox5/Atd3s36cxZ8+eJSoqyvp9VFQU+fn5tizdIUl4uZCKigqSk5OZM2fOVR9fuHAhU6ZM4bXXXmPPnj0kJyczePBgiouLASgtLWX58uWcPn2a/Px8tm7dyubNm+35I7i0m/37iIYkvFzI0KFDefPNNxkzZsxVH3/33XeZNGkSjzzyCImJiXz88cf4+/vz+eefA7Bu3Tri4uIIDQ3Fz8+P4cOHs23bNnv+CC7tZv8+jWnbtm2DllZ+fj5t3WACSAkvN1FTU8Pu3bsZOHCgdZ+HhwcDBw4kMzMTsMwuu3XrVqqqqjCZTGzcuJH4+Hi1SnYrTfn7NKZnz54cPHiQ/Px8jEYjq1atYvDgwS1dsurcbjJCd3X+/HlMJhPh4eEN9oeHh/PTTz8B0Lt3b4YNG0ZKSgoeHh7cfffdjBw5Uo1y3U5T/j4AAwcOZP/+/VRUVBAdHc0333xDWloas2bN4s4778RsNvPSSy+5/JVGkPASv/DWW2/x1ltvqV2GaMS6deuuun/kyJFu9x+NnDa6iVatWuHp6UlRUVGD/UVFRURERKhUlagnf58bJ+HlJnx8fEhNTWX9+vXWfWazmfXr18sgVAcgf58bJ6eNLsRoNJKdnW39/tSpU+zbt4/Q0FBiY2OZMmUKEyZM4LbbbqNnz568//77VFRU8Mgjj6hYtfuQv4+NKcJlbNiwQQF+tU2YMMF6zN///nclNjZW8fHxUXr27Kls27ZNvYLdjPx9bEvubRRCOCXp8xJCOCUJLyGEU5LwEkI4JQkvIYRTkvASQjglCS8hhFOS8BJCOCUJLyGEU5LwEkI4JQkv4dAGDBjAc8891+Tjv/jiC4KDg5v9fo3NLS8cj9yYLcQVCgoKCAkJUbsM0QQSXkJcQebOch5y2ihu2IABA3jmmWd47rnnCAkJITw8nLlz51qnb9HpdMTFxbFq1aoGz9u0aRM9e/ZEq9USGRnJyy+/TF1dnfXxiooKxo8fT2BgIJGRkcyaNetX711dXc2LL75IVFQUAQEB9OrVi40bNza59pqaGp5++mkiIyPx9fWlXbt2ZGRkWB+/8rTx9ddfR6PR/Gr74osvAMt8WxkZGXTo0AE/Pz+Sk5OtC8EKO1B7WgvhfPr376/odDpl+vTpyrFjx5Tp06crnp6eytChQ5VPP/1UOXbsmPLEE08oYWFhSkVFhaIoipKXl6f4+/srTz75pHLkyBFlyZIlSqtWrZTXXnvN+rpPPPGEEhsbq6xbt045cOCAMmLECEWn0ymTJ0+2HvPoo48q6enpyubNm5Xs7Gxl5syZilarVY4dO6YoiqLMmzdPCQoKarT2mTNnKjExMcrmzZuV06dPKz/88IMyf/586+OAsmTJEkVRFKW8vFwpKCiwbu+8847i7++vZGVlKYqiKG+++aaSkJCgfP/998qJEyeUefPmKVqtVtm4caNtftHimiS8xA3r37+/0rdvX+v3dXV1SkBAgPLQQw9Z9xUUFCiAkpmZqSiKorzyyitKfHy8YjabrcfMmTNHCQwMVEwmk1JeXq74+PgoixYtsj5eUlKi+Pn5WcPrzJkziqenp5Kfn9+gnrvvvluZNm2aoijXD69nnnlGueuuuxrUcaUrw+tKmZmZiq+vr7Jw4UJFURSlqqpK8ff3V7Zu3drguIkTJypjx45t9P2F7Uifl2iWpKQk69eenp6EhYVx6623WvfVr4JTv2DqkSNHSEtLQ6PRWI/p06cPRqORvLw8SktLqampoVevXtbHQ0NDGyy9lpWVhclkonPnzg1qqa6ubvJqOQ8//DCDBg0iPj6eIUOGMGLECO65555rPicnJ4fRo0fz4osvcv/99wOQnZ1NZWUlgwYNanBsTU0NKSkpTapF3BwJL9Es3t7eDb7XaDQN9tWHlNlsttl7Go1GPD092b17N56eng0eCwwMbNJr9OjRg1OnTrFq1SrWrVvH/fffz8CBAxvtq6qoqGDkyJGkpaXxxhtvNKgFYMWKFURFRTV4jlarvZEfSzSThJewiy5duvDtt9+iKIo12LZs2YJOpyM6OprQ0FC8vb3Zvn07sbGxAJSWlnLs2DH69+8PQEpKCiaTieLiYu64445m16LX63nggQd44IEHuO+++xgyZAgXLlwgNDS0wXGKovDggw9iNpv55z//2aDVmJiYiFarJScnx1qfsC8JL2EXTz75JO+//z7PPPMMTz/9NEePHuW1115jypQpeHh4EBgYyMSJE5k6dSphYWG0adOGP//5z3h4/HxBvHPnzowbN47x48cza9YsUlJSOHfuHOvXrycpKYnhw4dft453332XyMhI68K633zzDREREVcd2Pr666+zbt061qxZg9FotLa2goKC0Ol0vPjiizz//POYzWb69u1LWVkZW7ZsQa/XM2HCBJv97sTVSXgJu4iKimLlypVMnTqV5ORkQkNDmThxIq+++qr1mJkzZ2I0Grn33nvR6XS88MILlJWVNXidefPm8eabb/LCCy+Qn59Pq1at6N27NyNGjGhSHTqdjr/97W8cP34cT09Pbr/9dlauXNkgJOtt2rQJo9FIenr6r2p4+OGHmT59Oq1btyYjI4OTJ08SHBxMjx49eOWVV5rxGxI3ShbgEEI4JRmkKoRwShJeQginJOElhHBKEl5CCKck4SWEcEoSXkIIpyThJYRwShJeQginJOElhHBKEl5CCKck4SWEcEr/HzrztTTmmeQlAAAAAElFTkSuQmCC\",\n      \"text/plain\": [\n       \"<Figure size 300x300 with 1 Axes>\"\n      ]\n     },\n     \"metadata\": {},\n     \"output_type\": \"display_data\"\n    }\n   ],\n   \"source\": [\n    \"c = 2.21e19 # target compute budget (usually know this because we know how many GPU for how long go brrr)\\n\",\n    \"# (I got this flop number from row 1 of Table A3)\\n\",\n    \"# sweep model sizes from 10M to 100B\\n\",\n    \"ns = 10 ** np.arange(7, 11, step=2**-4)\\n\",\n    \"# using C = 6*N*D, solve for D that maintains the compute budget c\\n\",\n    \"ds = c / (6 * ns)\\n\",\n    \"# evaluate the loss in each case\\n\",\n    \"losses = L(ns, ds)\\n\",\n    \"# find the argmin\\n\",\n    \"best = np.argmin(losses)\\n\",\n    \"print(f\\\"best model size: {ns[best]/1e6:.2f}M\\\")\\n\",\n    \"print(f\\\"best dataset size: {ds[best]/1e9:.2f}B\\\")\\n\",\n    \"# plot the loss\\n\",\n    \"plt.figure(figsize=(3,3))\\n\",\n    \"plt.plot(ns, losses)\\n\",\n    \"plt.xscale('log')\\n\",\n    \"# plot a vertical bar at the best model size\\n\",\n    \"plt.axvline(ns[best], color='red')\\n\",\n    \"plt.xlabel('model size')\\n\",\n    \"plt.ylabel('loss')\"\n   ]\n  },\n  {\n   \"attachments\": {},\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"In the plot above, basically the models on the left of best are too small and trained for too long. The models on the right of best are way too large and trained for too little. The model at the red line is just right.\\n\",\n    \"\\n\",\n    \"Now, the Chinchilla paper says that best model size for this flop budget is 400M params and 9.2B tokens (instead of 316M params and 11.65B tokens) so there is some unresolved disagreement here too...\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 10,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"data\": {\n      \"text/plain\": [\n       \"2304\"\n      ]\n     },\n     \"execution_count\": 10,\n     \"metadata\": {},\n     \"output_type\": \"execute_result\"\n    }\n   ],\n   \"source\": [\n    \"# Calculate the Chinchilla optimal models for a range of compute budgets\\n\",\n    \"\\n\",\n    \"# sweep over compute budgets from 1e17 to 1e26\\n\",\n    \"cs = 10 ** np.arange(17, 26, step=2**-8)\\n\",\n    \"models = []\\n\",\n    \"for c in cs:\\n\",\n    \"    # sweep over model sizes\\n\",\n    \"    ns = 10 ** np.arange(7, 14, step=2**-8)\\n\",\n    \"    # the dataset sizes that would maintain the given compute budget\\n\",\n    \"    ds = c / (6 * ns)\\n\",\n    \"    # losses at each point\\n\",\n    \"    losses = L(ns, ds)\\n\",\n    \"    # n,d for the best model\\n\",\n    \"    best = np.argmin(losses)\\n\",\n    \"    models.append((c, ns[best], ds[best])) # c, n, d tuple log\\n\",\n    \"\\n\",\n    \"len(models)\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 11,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"name\": \"stdout\",\n     \"output_type\": \"stream\",\n     \"text\": [\n      \"closest model found:\\n\",\n      \"model size: 399.54M\\n\",\n      \"dataset size: 14.43B\\n\",\n      \"flops: 3.459892e+19\\n\",\n      \"loss: 2.76\\n\"\n     ]\n    }\n   ],\n   \"source\": [\n    \"query_model_size = 400e6\\n\",\n    \"ns = np.array([n for c, n, d in models])\\n\",\n    \"ds = np.array([d for c, n, d in models])\\n\",\n    \"# find the index of the closest model size in ns\\n\",\n    \"ix = np.argmin(np.abs(ns - query_model_size))\\n\",\n    \"# retrieve the corresponding params, flops, and data size\\n\",\n    \"print(\\\"closest model found:\\\")\\n\",\n    \"print(f\\\"model size: {ns[ix]/1e6:.2f}M\\\")\\n\",\n    \"print(f\\\"dataset size: {ds[ix]/1e9:.2f}B\\\")\\n\",\n    \"print(f\\\"flops: {6*ns[ix]*ds[ix]:e}\\\")\\n\",\n    \"print(f\\\"loss: {L(ns[ix], ds[ix]):.2f}\\\")\"\n   ]\n  },\n  {\n   \"attachments\": {},\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"This should have come out as 9.2B according to Table A3 in Chinchilla paper, per my understanding of it.\"\n   ]\n  },\n  {\n   \"attachments\": {},\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"## Scaling Laws: Approach 2\\n\",\n    \"\\n\",\n    \"Approach 2 is probably my favorite one because it fixes a flop budget and runs a number of model/dataset sizes, measures the loss, fits a parabolla, and gets the minimum. So it's a fairly direct measurement of what we're after. The best way to then calculate the compute-optimal number of tokens for any given model size, as an example, is via simple interpolation.\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 12,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"# Approach 1 numbers\\n\",\n    \"# # parameters, tokens\\n\",\n    \"# raw = [\\n\",\n    \"#     [400e6, 8e9],\\n\",\n    \"#     [1e9, 20.2e9],\\n\",\n    \"#     [10e9, 205.1e9],\\n\",\n    \"#     [67e9, 1.5e12],\\n\",\n    \"#     [175e9, 3.7e12],\\n\",\n    \"#     [280e9, 5.9e12],\\n\",\n    \"#     [520e9, 11e12],\\n\",\n    \"#     [1e12, 21.2e12],\\n\",\n    \"#     [10e12, 216.2e12],\\n\",\n    \"# ]\\n\",\n    \"\\n\",\n    \"# Approach 2 numbers\\n\",\n    \"# parameters, tokens\\n\",\n    \"raw = [\\n\",\n    \"    [400e6, 7.7e9],\\n\",\n    \"    [1e9, 20.0e9],\\n\",\n    \"    [10e9, 219.5e9],\\n\",\n    \"    [67e9, 1.7e12],\\n\",\n    \"    [175e9, 4.3e12],\\n\",\n    \"    [280e9, 7.1e12],\\n\",\n    \"    [520e9, 13.4e12],\\n\",\n    \"    [1e12, 26.5e12],\\n\",\n    \"    [10e12, 292.0e12],\\n\",\n    \"]\\n\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 13,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"name\": \"stdout\",\n     \"output_type\": \"stream\",\n     \"text\": [\n      \"y = 1.0409573169995892x + 0.9353887152390791\\n\"\n     ]\n    }\n   ],\n   \"source\": [\n    \"# fit a line by linear regression to the raw data\\n\",\n    \"import numpy as np\\n\",\n    \"x = np.array([np.log10(x[0]) for x in raw])\\n\",\n    \"y = np.array([np.log10(x[1]) for x in raw])\\n\",\n    \"A = np.vstack([x, np.ones(len(x))]).T\\n\",\n    \"m, c = np.linalg.lstsq(A, y, rcond=None)[0]\\n\",\n    \"print(f\\\"y = {m}x + {c}\\\")\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 14,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"data\": {\n      \"image/png\": \"iVBORw0KGgoAAAANSUhEUgAAATkAAAFBCAYAAAAMkNhdAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABCEElEQVR4nO3deVxU5f4H8M8MwozIJrHL5q6IgIIQWVdUFslISculEs2fdQ3SIm9JpYQt2oaWUmY3Nc2VNNyJRUwtlFwgvYpbaIoIIsLAINvM8/vjNKPDADIwcIaZ7/v18gXnmTNnvucBPp71OQLGGAMhhOgpId8FEEJIR6KQI4ToNQo5Qoheo5AjhOg1CjlCiF6jkCOE6DUKOUKIXqOQI4ToNQo5Qoheo5AjBisoKAhBQUEG99mamDlzJtzd3dv0Xl1ZRwo50iYff/wxUlJS+C7joc6dO4f3338fV69e5bsUwhMKOdImXSnkEhISmgy5tLQ0pKWldX5RpFN147sAQvhiYmLCdwmkE9CWXAcpLCzE7Nmz4eTkBJFIhN69e2Pu3Lmoq6tTzvPXX3/h2WefhbW1NUxNTfHoo49i3759Kss5dOgQBAIBtm/fjoSEBPTq1Qvm5uaYPHkyKioqUFtbi9dffx12dnYwMzPDrFmzUFtbq7IMgUCAmJgYbNq0CQMHDoRYLIavry8OHz6sMl9zx1/ef/99CAQCleVJpVL88MMPEAgEEAgEmDlzpsq6v/TSS7C3t4dIJMKQIUOwdu3aVvVbQ0MDPvjgA/Tt2xcikQju7u5455131NbJ3d0dTz31FNLS0uDj4wOxWAwPDw/s3LlTOc/69evx7LPPAgBGjx6trPXQoUMA1I8ZaaOv161bhzFjxsDOzg4ikQgeHh745ptvWrXuTVH87JKTk+Hh4YHu3bsjMDAQZ86cAQB8++236NevH8RiMYKCgprcYk1OToavry+6d+8OGxsbvPDCCygsLFSbLyUlBZ6enhCLxfD09MTPP//cZE1yuRwrVqzAkCFDIBaLYW9vj1deeQV379596PqsXLkSQ4YMgampKXr27Ak/Pz9s3rxZs07RFCNaV1hYyJycnJipqSl7/fXX2erVq9miRYvY4MGD2d27dxljjN26dYvZ29szc3Nz9u6777LExETm7e3NhEIh27lzp3JZWVlZDADz8fFhgYGB7KuvvmLz5s1jAoGATZ06lU2fPp2Fh4ezpKQk9uKLLzIALCEhQaUeAMzT05PZ2NiwJUuWsE8++YS5ubmx7t27szNnzijni4qKYm5ubmrrEx8fzx78Vdm4cSMTiUTsiSeeYBs3bmQbN25kv//+u3K9nJ2dmYuLC1uyZAn75ptv2NNPP80AsOXLlz+076KiohgANnnyZJaUlMRmzJjBALCJEyeqzOfm5sYGDBjArKys2MKFC1liYiIbOnQoEwqFLC0tjTHG2JUrV9i8efMYAPbOO+8oa7116xZjjLFRo0axUaNGabWvR4wYwWbOnMmWL1/OVq5cyUJDQxkAtmrVKpX5Gn92cwAwLy8v5uLiwpYtW8aWLVvGLC0tmaurK1u1ahXz8PBgX3zxBXvvvfeYiYkJGz16tMr7161bxwCwESNGsOXLl7OFCxey7t27M3d3d+XvImOM/fLLL0woFDJPT0+WmJjI3n33XWZpacmGDBmi9jvxf//3f6xbt25szpw5bPXq1eztt99mPXr0YCNGjGB1dXXNruOaNWuUP9tvv/2Wffnll2z27Nls3rx5D+2H9qCQ6wAzZsxgQqGQ/fHHH2qvyeVyxhhjr7/+OgPAjhw5onytsrKS9e7dm7m7uzOZTMYYu/+H5+npqfILNG3aNCYQCFh4eLjK8gMDA9V+KQEwAOzEiRPKtmvXrjGxWMwiIyOVba0NOcYY69GjB4uKilKbd/bs2czR0ZGVlpaqtE+dOpVZWlqy6upqtfco5ObmMgDs//7v/1TaFyxYwACwgwcPKtvc3NwYALZjxw5lW0VFBXN0dGTDhg1TtiUnJzMALCsrS+3zmgu59vR1U+sXFhbG+vTp0+JnNwcAE4lErKCgQNn27bffMgDMwcGBSSQSZXtcXBwDoJy3rq6O2dnZMU9PT3bv3j3lfHv37mUA2OLFi5VtPj4+zNHRkZWXlyvb0tLSGACVdTxy5AgDwDZt2qRSZ2pqqlp743WcMGECGzJkyEPXWdtod1XL5HI5UlJSEBERAT8/P7XXFbt9+/fvh7+/Px5//HHla2ZmZnj55Zdx9epVnDt3TuV9M2bMgLGxsXI6ICAAjDG89NJLKvMFBATg+vXraGhoUGkPDAyEr6+vctrV1RUTJkzAL7/8AplM1vYVfgBjDDt27EBERAQYYygtLVX+CwsLQ0VFBU6dOtXs+/fv3w8AiI2NVWl/8803AUBtV97JyQmRkZHKaQsLC8yYMQOnT5/GrVu32rwe7enr7t27K7+vqKhAaWkpRo0ahb/++gsVFRVtqmfs2LEqhxECAgIAAJMmTYK5ubla+19//QUAOHHiBEpKSvDqq69CLBYr5xs/fjwGDRqk7M+ioiLk5uYiKioKlpaWyvlCQkLg4eGhUktycjIsLS0REhKi8vP19fWFmZkZsrKyml0PKysr3LhxA3/88Ueb+qGtKOS07Pbt25BIJPD09GxxvmvXrmHgwIFq7YMHD1a+/iBXV1eVacUvo4uLi1q7XC5X+4Pq37+/2mcNGDAA1dXVuH37dou1ttbt27dRXl6ONWvWwNbWVuXfrFmzAAAlJSXNvv/atWsQCoXo16+fSruDgwOsrKzU+qRfv34qxwoV6wSgXZeMtKevf/vtNwQHB6NHjx6wsrKCra0t3nnnHQBoc8hpUg8A5bExRX819Xs2aNAg5euKr039jjR+76VLl1BRUQE7Ozu1n3FVVVWLP9+3334bZmZm8Pf3R//+/REdHY3ffvut+RXXEjq72kUYGRlp1M7aMKp948BQaO2WnlwuBwC88MILiIqKanIeLy+vNtfRWdra11euXMHYsWMxaNAgJCYmwsXFBSYmJti/fz+WL1+u7J/OqqcjyOVy2NnZYdOmTU2+bmtr2+x7Bw8ejAsXLmDv3r1ITU3Fjh078PXXX2Px4sVISEjoqJIp5LTN1tYWFhYWOHv2bIvzubm54cKFC2rt+fn5yte16dKlS2ptFy9ehKmpqfIXs2fPnigvL1ebr/EWFNB0ENna2sLc3BwymQzBwcEa1+jm5ga5XI5Lly4pt2gBoLi4GOXl5Wp9cvnyZTDGVGq5ePEiACh37zozMPfs2YPa2lrs3r1bZeurpV24jqTorwsXLmDMmDEqr124cEH5uuJrU78jjX9H+/bti4yMDIwcOVJl17y1evTogSlTpmDKlCmoq6vDM888g48++ghxcXEqu9TaRLurWiYUCjFx4kTs2bMHJ06cUHtd8b/sk08+iZycHGRnZytfk0qlWLNmDdzd3dWOhbRXdna2yvGw69evY9euXQgNDVVuEfTt2xcVFRX4888/lfMVFRU1eSlBjx491ALRyMgIkyZNwo4dO5oM+YftFj/55JMAgBUrVqi0JyYmAuCOJT3o5s2bKrVJJBJs2LABPj4+cHBwUNYJoMnw1jZFPz64JVVRUYF169Z1+Gc3xc/PD3Z2dli9erXKpS4HDhzA+fPnlf3p6OgIHx8f/PDDDyq71Onp6WrHhp977jnIZDJ88MEHap/X0NDQYj/fuXNHZdrExAQeHh5gjKG+vr4tq9gqtCXXAT7++GOkpaVh1KhRePnllzF48GAUFRUhOTkZR48ehZWVFRYuXIgtW7YgPDwc8+bNg7W1NX744QcUFBRgx44dEAq1+/+Pp6cnwsLCMG/ePIhEInz99dcAoLKbMHXqVLz99tuIjIzEvHnzUF1djW+++QYDBgxQO2Hg6+uLjIwMJCYmwsnJCb1790ZAQACWLVuGrKwsBAQEYM6cOfDw8EBZWRlOnTqFjIwMlJWVNVujt7c3oqKisGbNGpSXl2PUqFHIycnBDz/8gIkTJ2L06NEq8w8YMACzZ8/GH3/8AXt7e6xduxbFxcUqoeLj4wMjIyN88sknqKiogEgkUl7Hpm2hoaEwMTFBREQEXnnlFVRVVeG7776DnZ0dioqKtP55D2NsbIxPPvkEs2bNwqhRozBt2jQUFxfjyy+/hLu7O9544w3lvEuXLsX48ePx+OOP46WXXkJZWZnymraqqirlfKNGjcIrr7yCpUuXIjc3F6GhoTA2NsalS5eQnJyML7/8EpMnT26yntDQUDg4OGDkyJGwt7fH+fPnsWrVKowfP17lBIrWdfr5XANx7do1NmPGDGZra8tEIhHr06cPi46OZrW1tcp5rly5wiZPnsysrKyYWCxm/v7+bO/evSrLUVzWkJycrNKuuP6p8WUqiss9bt++rWwDwKKjo9mPP/7I+vfvz0QiERs2bFiTl1WkpaUxT09PZmJiwgYOHMh+/PHHJi8hyc/PZ//6179Y9+7dGQCVy0mKi4tZdHQ0c3FxYcbGxszBwYGNHTuWrVmz5qH9Vl9fzxISEljv3r2ZsbExc3FxYXFxcaympkZlPjc3NzZ+/Hj2yy+/MC8vLyYSidigQYPU+okxxr777jvWp08fZmRkpHI5SXOXkLSnr3fv3s28vLyYWCxm7u7u7JNPPmFr165VubSjqc9ujuJn96CCggIGgH322Wcq7c3Vv23bNjZs2DAmEomYtbU1e/7559mNGzfUPmvHjh1s8ODBTCQSMQ8PD7Zz585mLytas2YN8/X1Zd27d2fm5uZs6NCh7K233mI3b95sdh2//fZb9q9//Ys98sgjTCQSsb59+7L//Oc/rKKi4qH90B4Cxui5q/pOIBAgOjoaq1at4rsUrXF3d4enpyf27t3LdylEx9ExOUKIXqOQI4ToNQo5Qoheo2NyhBC9RltyhBC9RiFHCNFrdDFwM+RyOW7evAlzc3Pe76UkhKhijKGyshJOTk4PvXCeQq4ZN2/eVBvlgRCiW65fvw5nZ+cW56GQa4biNpPr16/DwsKC52q6jvr6eqSlpSlv9yGdy1D6XyKRwMXFpVW3g1HINUOxi2phYUEhp4H6+nqYmprCwsJCr//IdJWh9X9rDiXRiQdCiF6jkGskKSkJHh4eGDFiBN+lEEK0gEKukejoaJw7d67Tx6EnhHQMOiZHCNEJMjlDTkEZSiprYGcuhn9vaxgJ23/5FoUcIYR3qWeLkLDnHIoqapRtjpZixEd4YJynY7uWTburhBBepZ4twtwfT6kEHADcqqjB3B9PIfVs+0ZVppAjhPBGJmdI2HMODICAyRGXtRaDS7jnxipGDknYcw4yedvHEaGQa4TOrhLSeXIKylBUUQMBk+Pj1FV4JWcnNmxfjB611QC4oCuqqEFOQfPPBnkYCrlG6OwqIZ2npJILuGUHVmLan2mQCYT4cPRsSEWmavO1FZ14IITwxs7UGJ/u/wrPns2ATCDEG0+9id0eo9TnM2/7M1kp5Agh/JDJEPDRWwg8m4EGgRCvRyzA3sH/UplFAMDBkrucpK0o5AghnU8mA2bNgnDjRsiNjPD6+AXYN/gJlVkUV8jFR3i063o5OiZHCOlcMhkwcyawcSNgZAThli146qP5cLBU3SV1sBTjmxeGt/s6OdqSI4R0noYGICoK2LwZ6NYN2LoVmDQJ4wCEeDjQHQ+EkC6soQGYMQPYsoULuG3bgGeeUb5sJBQgsO8jWv9Y2l1thK6TI6QDNDQAL7xwP+CSk1UCriNRyDVC18kRomX19cD06dyWm7Ex8NNPwMSJnfbxtLtKCOk4ioD76Scu4HbsACIiOrUECjlCSMeorwemTeOCzcSE+/rUU51eBoUcIUT76uqAqVOBn3/mAm7nTmD8eF5KoZAjhGhXXR0wZQqQkgKIRFzQhYfzVg6FHCFEe+rqgGefBXbv5gIuJQUYN47XkijkCCHaUVvLBdyePYBYDOzaBYSG8l2VflxCEhkZiZ49e2Ly5Mlqr1VXV8PNzQ0LFizgoTJCDERtLTBp0v2A271bJwIO0JOQmz9/PjZs2NDkax999BEeffTRTq6IEANSU8Nd2LtvHxdwe/YAISF8V6WkFyEXFBQEc3NztfZLly4hPz8f4Twe9CREr9XUAJGRwP79QPfuwN69QHAw31Wp4D3kDh8+jIiICDg5OUEgECAlJUVtnqSkJLi7u0MsFiMgIAA5OTmtWvaCBQuwdOlSLVdMCAHABdzEiUBqKhdw+/YBY8fyXZUa3kNOKpXC29sbSUlJTb6+bds2xMbGIj4+HqdOnYK3tzfCwsJQUlLS4nJ37dqFAQMGYMCAAR1RNiGG7d49YMIE4JdfAFNTbktu9Gi+q2oS72dXw8PDW9ydTExMxJw5czBr1iwAwOrVq7Fv3z6sXbsWCxcubPZ9x44dw9atW5GcnIyqqirU19fDwsICixcvbnL+2tpa1NbWKqclEgkAoL6+HvX19W1ZNYOk6CvqM350Sv9XV8No0iQIMzPBevSAbPdusJEjuTscOokm68d7yLWkrq4OJ0+eRFxcnLJNKBQiODgY2dnZLb536dKlyl3V9evX4+zZs80GnGL+hIQEtfa0tDSYmpo28Q7SkvT0dL5LMGgd1f9GtbUI+Ogj2P75JxrEYmS/8w7KKiu5LblOVF1d3ep5dTrkSktLIZPJYG9vr9Jub2+P/Px85XRwcDDy8vIglUrh7OyM5ORkBAYGavRZcXFxiI2NVU5LJBK4uLggNDQUFhYW7VsRA1JfX4/09HSEhITA2NiY73IMTof2v1QKo8hICP/8E8zMDNizB4+OHKndz2glxZ5Wa+h0yLVWRkZGi6/PnDnzocsQiUQQiURISkpCUlISZDIZAMDY2Jj+WNuA+o1fWu9/qZQ7i3roEGBmBkFqKrrxFHAANFo33k88tMTGxgZGRkYoLi5WaS8uLoaDg0OHfCaNJ0dII1Ipd3P9oUOAuTl3soHHgNOUToeciYkJfH19kZmZqWyTy+XIzMzUeHeUENIGVVXAk08Cv/4KWFgAaWnAY4/xXZVGeN9draqqwuXLl5XTBQUFyM3NhbW1NVxdXREbG4uoqCj4+fnB398fK1asgFQqVZ5t1bbGu6uEGKzKSi7gjh69H3ABAXxXpTnGs6ysLAZA7V9UVJRynpUrVzJXV1dmYmLC/P392bFjxzq8roqKCgaAVVRUdPhn6ZO6ujqWkpLC6urq+C7FIGmt/yUSxkaOZAxgzNKSsePHtVKftmjy98n7llxQUBAYYy3OExMTg5iYmE6ph7bkiMGTSLjx337/HbC0BNLTgS78YCedPibHBzrxQAxaRQUQFsYFnJUVkJHRpQMO0IFjcoQQHaEIuOPHgZ49uYAbPpzvqtqNtuQaoeeuEoNUXs6N/3b8OGBtDWRm6kXAARRyamh3lRgcRcDl5NwPuGHD+K5KayjkCDFkd+9yA1z+8QfwyCPAwYOAjw/fVWkVHZMjxFCVlXEBd+oUYGPDbcF5efFdldbRllwjdEyOGISyMm4EX0XAHTyolwEHUMipoWNyRO/ducON4Hv6NGBrC2RlAUOH8l1Vh6GQI8SQlJZyAZebC9jZcQHn6cl3VR2KjskRYigUAffnn4C9PbeL6uHBd1UdjrbkGqFjckQv3b4NjBnDBZyDAzdskgEEHEAhp4aOyRG9U1LCBdyZM1zAZWUBgwbxXVWnod1VQvRZcTEXcOfOAY6OXMANHMh3VZ2KtuQI0Ve3bnGPCTx3DnBy4nZRDSzgAAo5QvSGTM6QU1AGADh9/DzY6NHA+fNAr15cwBnoM4hpd5UQPZB6tggJe86hrOoevuxXBpuXoiEoK8Q9e0d0P3QI6NeP7xJ5Q1tyjdDZVdLVpJ4twtwfT6GoogZ2lXfw+HvvoU9ZIQrNbTFuwhKk1vTgu0ReUcg1QmdXSVcikzMk7DkHBsC+shQbNr0Ls5s3UWhhi6nTl+Lvno5I2HMOMnnLo2/rMwo5QrqwnIIyFFXUwEFSiq1b4tC7rBDVtraY8cLHuG7lAAagqKJGeazOEFHIEdKFlVTWwFFymwu4u0W4YWmHox9+iBtWDmrzGSoKOUK6MOdKbgvOvbwI1y3t8eILS3HP3l5tPjtzMQ/V6QY6u0pIV3XtGoZHRUJQfgt/W9pj6vSluGNpC+D+k+YEABwsxfDvbc1bmXyjkCOkK7p2DQgKguDqVVQ7u2FqxPsosrCFCe6fYBD88zU+wgNGQkHTyzEAtLtKSFdz9SoQFMR97dsXptlHsfjVMDhYqu6SOliK8c0LwzHO05GPKnUGbck1Qg+XJjqtoIC7VevaNe4C30OHgF69MM4ZCPFwwLHLJSg9fwxro0bg0X52Br0Fp0Bbco3QdXJEZ/31F7cFd+0a0L+/MuAUjIQC5bE3/97WFHD/oJAjpCu4coULuL//5u5BbRRwpHm0u0qIjlHcaF9SWQM7czH8ZWUwGjsGuHGDG0UkK4sbNom0CoUcITpEcaN9UQV38a57WSG2b3sXdpJSbqDLrCxu4EvSahRyhOgIxY32iotAepcVYsuWONhVleHiI6648c02jKGA0xgdkyNEBzx4oz0A9LlzA1u3xMHhn4B7ftpHeDf7tkHfaN9WFHKE6ADFjfYA0PfOdWzZ+g7sq8pwwcYV06Z9jNs9ehr8jfZtpRchFxkZiZ49e2Ly5MnKtvLycvj5+cHHxweenp747rvveKyQkJYpbqDvW3odW7ZwAZdv44Zp05biTg8rtflI6+lFyM2fPx8bNmxQaTM3N8fhw4eRm5uL48eP4+OPP8adO3d4qpCQltmZi9Gv9G9s3RoHO+ldnLd1x/RpH6PM1FJtPqIZvQi5oKAgmJubq7QZGRnB1NQUAFBbWwvGGBij4xlEN/lXF2H71ndgKy3HObvemD71I5WAEwBwNPAb7duK95A7fPgwIiIi4OTkBIFAgJSUFLV5kpKS4O7uDrFYjICAAOTk5LRq2eXl5fD29oazszP+85//wMbGRsvVE6IFZ8/CaOwYWEvL8T+7Pnh+6ke42yjgALrRvq14DzmpVApvb28kJSU1+fq2bdsQGxuL+Ph4nDp1Ct7e3ggLC0NJSclDl21lZYW8vDwUFBRg8+bNKC4u1nb5hLTPmTPcvai3bwPDhuHWzj0QO9ipzEI32rcP79fJhYeHIzw8vNnXExMTMWfOHMyaNQsAsHr1auzbtw9r167FwoULW/UZ9vb28Pb2xpEjR1ROTjyotrYWtbW1ymmJRAIAqK+vR319fWtXx+Ap+or6rBX+/BPdxo2DoLQU8uHDIdu/H/+ytkaWXz+cvHYXpVW1sDETwdetJ4yEglb1qaH0vybrx3vItaSurg4nT55EXFycsk0oFCI4OBjZ2dktvre4uBimpqYwNzdHRUUFDh8+jLlz5zY7/9KlS5GQkKDWnpaWpjy2R1ovPT2d7xJ0mkVBAR5bvBiCykrc7dcP2bGxqD92TG2+UgC/nNd8+fre/9XV1a2eV6dDrrS0FDKZDPaNhnO2t7dHfn6+cjo4OBh5eXmQSqVwdnZGcnIyjIyM8PLLLytPOLz22msYOnRos58VFxeH2NhY5bREIoGLiwtCQ0NhYWGh/ZXTU/X19UhPT0dISAiMjY35Lkc35eai20svQVBZCbmfH8z270eIlZVWFm0o/a/Y02oNnQ651srIyGiyPTc3t9XLEIlEEIlEauPJGRsb6/UvS0ehfuOo3Wx/9yqMwsKAu3cBf38If/kFQi0F3IP0vf81WTedDjkbGxsYGRmpnTAoLi6GQwfdwxcdHY3o6GhIJBJYWlo+/A2ENKPxzfaety5j8/ZFsLhXCQQEAL/8AtDvWIfj/exqS0xMTODr64vMzExlm1wuR2ZmJgIDAzvkM5OSkuDh4YERI0Z0yPKJYXjwqfYAMLToEjZtfRcW9ypx0mkQ0ldsoIDrJLxvyVVVVeHy5cvK6YKCAuTm5sLa2hqurq6IjY1FVFQU/Pz84O/vjxUrVkAqlSrPtmobbcmR9mp8s71X0UX8uG0RLGqlONFrMGY9mwCzrOsY49+frnvrBLyH3IkTJzB69GjltOLgf1RUFNavX48pU6bg9u3bWLx4MW7dugUfHx+kpqaqnYwgRFc8eLO9980L2LhtESzqqvFHLw/MfPZ9SEWmqPznZvvAvo/wXK3+4z3kgoKCHnq7VUxMDGJiYjqlHnqQDWkvxU30PjcvYMM/AZfj7IFZk7mAazwf6Vg6fUyOD/QgG9JeduZiDC88j43b3oNFXTWOu3hi5rMJKgGnmI90PN635AjRN/5F+diYHI8edfdwzMUTL02OR7VJd+Xr9FT7zkVbco3Q2VXSLr/9BqPwcehRW41s16F4afL7agEH0M32nYlCrhHaXSVtdvQoMG4cUFUFjBmDyp9+hqWtlcosdLN956PdVULaQO1Ohhv/g9H4JwGpFBg7Fti9G6Gmphjr20d1Pnroc6ejkCNEQ43vZAj4+wzW7UiAaV0NEBIC7NoFdOd2UY2EArpMhGe0u9oIHZMjLWl8J8Ojf/+JdT+9D9O6Ghx2H4a0j1crA47oBgq5RuiYHGlO4zsZAq/9ibU/JcC0vha/9h6Ol595D/HpBfTYQB1DIUdIKz14J0PgtTxlwB3q7YuXn3kPNcYiemygDmp3yEkkEqSkpOD8+TaM7EdIF6K4Q2Hk1Vys+ykB3RtqcbCPH1555l3UdjNRm4/oBo1D7rnnnsOqVasAAPfu3YOfnx+ee+45eHl5YceOHVovsLPRMTnSHDtzMR4vOI3vdyyBuKEOmX1H4N+RqgGnmI/oDo1D7vDhw3jiiScAAD///DMYYygvL8dXX32FDz/8UOsFdjY6Jkea43/pBL7fyQVcej9/zJ34Duq63R+8kR4bqJs0DrmKigpYW3M/xNTUVEyaNAmmpqYYP348Ll26pPUCCdEJv/wCo4kTIGqoR1r/RxE9MU4t4AC6k0EXaRxyLi4uyM7OhlQqRWpqKkJDQwEAd+/ehVhMm+lEDx04AEyYANTWAhMngm3bikesVR9mTncy6C6NLwZ+/fXX8fzzz8PMzAxubm4ICgoCwO3GtvSgGEK6pP37gchIoK6O+7p1K8JMTBDs7Up3MnQRGofcq6++Cn9/f1y/fh0hISEQCrmNwT59+ujFMTlClPbuBSZN4gJu0iRgyxbgnweo0J0MXUebbuvy8/ODn5+fStv48eO1UhDfaNBMAgDYs4cLtvp6YPJkYPNmZcCRrkXjkJPJZFi/fj0yMzNRUlICuVyu8vrBgwe1Vhwf6BkPBLt2Ac8+ywXcc88BP/5IAdeFaRxy8+fPx/r16zF+/Hh4enpCIKDjEESPpKRwAdfQAEydCmzcCHSjcSy6Mo1/elu3bsX27dvx5JNPdkQ9hPDn55+5LbeGBmDaNGDDBgo4PaDxJSQmJibo169fR9RCCH927LgfcNOnU8DpEY1D7s0338SXX3750CdsEdJlJCcDU6ZwAffCCxRwekbjn+TRo0eRlZWFAwcOYMiQITBudEB2586dWiuOEG1RG8lXcV3b9u3clptMBrz4IrBuHWBkxHe5RIs0DjkrKytERkZ2RC2EdIjGI/kC3D2m33S7CJ+417iAi4oCvv+eAk4PaRxy69at64g6dAZdJ6dfFCP5Nj644p+diqF7EwEmB2bNAr77jgJOT7VpPLmGhgZkZGTg22+/RWVlJQDg5s2bqKqq0mpxfKBRSPRH45F8FSb8LwuJexNhxOTY7RcO2RoKOH2m8ZbctWvXMG7cOPz999+ora1FSEgIzM3N8cknn6C2tharV6/uiDoJ0diDI/kqTPxfFr7YtxxGTI4tXqF4Z8xc2F4rp1u09JjGW3Lz58+Hn58f7t69i+4PPLAjMjISmZmZWi2OkPZoPELvM2czlVtwm73H4Z1xMWACIY3kq+c03pI7cuQIfv/9d5iYqI6G6u7ujsLCQq0VRkh7PThC76Qzmfhs/woIwbDJZxzeC30VTCBUm4/oH41DTi6XN3lQ/saNGzA3N2/iHYTww7+3NRwtxXj8yB58cuArCMGwcdiTWBzybzCBEAJw48DRSL76TePd1dDQUKxYsUI5LRAIUFVVhfj4eLrVi+gUI6EAa+pzlQH3w/DxWBQyVxlwAI3kawg03pL74osvEBYWBg8PD9TU1GD69Om4dOkSbGxssGXLlo6okZC2+e47DI1fAADYHjgR8U/MBv4ZUMLBUoz4CA8aydcAaBxyzs7OyMvLw7Zt25CXl4eqqirMnj0bzz//vMqJiM4UGRmJQ4cOYezYsfjpp58AANevX8eLL76IkpISdOvWDYsWLcKzzz7LS32EB2vWAK+8wn0/fz4mfZEIl6t3aSRfQ8Q0tHnz5mZfW7BggaaL04qsrCy2e/duNmnSJGXbzZs32enTpxljjBUVFTEnJydWVVXV6mVWVFQwAKyiokLb5eq1uro6lpKSwurq6vgrYvVqxgDu3+uvMyaX81dLJ9OJ/u8Emvx9anxMbu7cuThw4IBa+xtvvIEff/xRC7GruaCgILWTHo6OjvDx8QEAODg4wMbGBmVl9GRzvffNN8C//819HxsLJCYqd1GJYdI45DZt2oRp06bh6NGjyrbXXnsN27dvR1ZWlsYFHD58GBEREXBycoJAIEBKSoraPElJSXB3d4dYLEZAQABycnI0+oyTJ09CJpPBxcVF4/pIF5KUBLz6Kvf9m28Cn39OAUc0D7nx48fj66+/xtNPP42TJ0/i1Vdfxc6dO5GVlYVBgwZpXIBUKoW3tzeSkpKafH3btm2IjY1FfHw8Tp06BW9vb4SFhaGkpKRVyy8rK8OMGTOwZs0ajWsjXcjKlUBMDPf9f/4DfPYZBRwB0MYH2UyfPh3l5eUYOXIkbG1t8euvv7Z5IM3w8HCEh4c3+3piYiLmzJmDWbNmAQBWr16Nffv2Ye3atVi4cGGLy66trcXEiROxcOFCPPbYYw+dt7a2VjktkUgAAPX19aivr2/t6hg8RV91Zp8JV66E0ZtvAgBkCxZA/uGH3NhwBoiP/ueDJuvXqpCLjY1tst3W1hbDhw/H119/rWxLTExs9Yc/TF1dHU6ePIm4uDhlm1AoRHBwMLKzs1t8L2MMM2fOxJgxY/Diiy8+9LOWLl2KhIQEtfa0tDSYmppqXryBS09P75TP6bN7N4auXQsAuDhpEs6PHMk9DNrAdVb/86W6urrV87Yq5E6fPt1ke79+/SCRSJSva/uhNqWlpZDJZLC3t1dpt7e3R35+vnI6ODgYeXl5kEqlcHZ2RnJyMmQyGbZt2wYvLy/lcb6NGzc2+wDsuLg4lTCXSCRwcXFBaGgoLCwstLpe+qy+vh7p6ekICQlRG1BV24QrVsDon4CTLVyI3gkJ6G3gu6id2f98UuxptUarQq4tJxQ6U0ZGRpPtjR+X2BKRSASRSKQ2npyxsbFe/7J0lA7vty++AN56i/t+0SIYJSTAyMAD7kH6/nurybq1aTw5hRs3buDGjRvtWUSLbGxsYGRkhOLiYpX24uJiODg4dMhn0nhyXcBnnwELuDsZEB8PLFlCJxlIszQOOblcjiVLlsDS0hJubm5wc3ODlZUVPvjgA422nFrDxMQEvr6+KkM4yeVyZGZmIjAwUKufpZCUlAQPDw+MGDGiQ5ZP2umTT+5vwb3/PvePkBZofHb13Xffxffff49ly5Zh5MiRALiH27z//vuoqanBRx99pNHyqqqqcPnyZeV0QUEBcnNzYW1tDVdXV8TGxiIqKgp+fn7w9/fHihUrIJVKlWdbtS06OhrR0dGQSCSwtLTskM8gbbR0KfDOO9z3CQnA4sX81kO6Bk1vp3B0dGS7du1Sa09JSWFOTk6aLo5lZWUxAGr/oqKilPOsXLmSubq6MhMTE+bv78+OHTum8edoim7rapsOu63oww/v36r1wQfaXbYeodu61Gm8JVdWVtbkRb+DBg1q021TQUFBD32Ga0xMDGIUF3p2MHqQjQ764IP7W20ffXR/a46QVtD4mJy3tzdWrVql1r5q1Sp4e3trpSg+0YkHHfPgbumDu6uEtJLGW3Kffvopxo8fj4yMDOXB/+zsbFy/fh379+/XeoHEgL3/PhdygOoJB0I0oPGWXO/evXHx4kVERkaivLwc5eXleOaZZ3DhwgW4ubl1RI2dis6u6gDGuEtDFAH32WcUcKTNNN6S6927N4qKitTOot65cwcuLi5d/lgWnV3lGWPc7umHH3LTn3/OjShCSBtpHHLNnSSoqqqCWExPPSLtwBjw3nvAxx9z04mJwBtv8FsT6fJaHXKK+zoFAgEWL16sctO6TCbD8ePHlYNUEqIxxriTCsuWcdMrVgDz5/NaEtEPrQ45xU34jDGcOXNG5bmrJiYm8Pb2xgLFrTZdGF1CwgPGgIULgU8/5aa/+gp47TV+ayJ6o9Uhp7hJf9asWfjyyy/1dmQOOibXyRjjTip8/jk3vWoVEB3Nb01Er2h8TG7dunUdUQcxRIxxo/h+8QU3/eDw5YRoSZtGBiak3RjjzpouX85Nf/01MHcuvzURvdSuoZb0EV0n1wkY486aKgJu9WoKONJhKOQaodu6Ohhj3FnTL7/kph98CDQhHYB2V0nnYYw7a5qUxA1y+d13wOzZfFdF9ByFHOkcjHGPDPz6ay7g/vtf4KWX+K6KGAAKOdLx5HIu4L75hgu4tWuBmTP5rooYCAo50rHkcu6ykG+/5QJu3TogKorvqogBoRMPjdDZVS2Sy4F///t+wP3wAwUc6XQUco3Q2dW2k8kZcgq40aFzrpRCPmcOd3JBKAQ2bABa8ZBvQrSNdleJVqSeLULCnnMoq7qHT/3kuDVlBoR/ZoAJhRBs3AhMn853icRAUciRdks9W4S5P54CA9BdIMOwVavg+udByARCvD7+TYz3Go1xfBdJDBbtrpJ2kckZEvacAwMglMvw8b6v4HrwIBoEQsyPWIC9HqOQsOccZPKWH1ZESEehkCPtklNQhqKKGgjlMny2fwUizxyEXCjEmxP/g72D/wUGoKiiRnmsjpDORrurpF1KKrmA+2LfckSeO4QGgRCnFixAarfHAZnqfITwgbbkSLvYde+GxH2JiDx3CPVCI7wR+RaKHntMfT5zGhqf8INCrhG6Tk4DDQ0IiH8dE8/9inqhEWImvI20QSNVZhEAcLQUw7+3NT81EoNHIdcIXSfXSg0NwPPPQ7htK+TdjBE9MQ5pA1S34AT/fI2P8ICRUKC+DEI6AYUc0Vx9PXfd2/btgLExhDt+wjMJr8LBUnWX1MFSjG9eGI5xno48FUoInXggmqqvB6ZNA3bsAExMuK9PPYVxAEI8HHDscglKzx/D2qgReLSfHW3BEd7Rlhxpvfp6YOrU+wG3cyfw1FPKl42EAuWxN//e1hRwRCfQlhxpnbo6LuB+/pkLuJ9/Bp58ku+qCHkoCjnycHV1wHPPAbt2ASIRkJICjKMbtUjXQCFHWlZbCzz7LLBnDxdwu3YBYWF8V0VIq+nFMbnIyEj07NkTkydPblU7aaXaWmDyZC7gxGJg924KONLl6EXIzZ8/Hxs2bGh1O2mF2lpg0iRg714u4PbsAUJD+a6KEI3pRcgFBQXB3Ny81e3kIWpqgGeeAfbtA7p354IuOJjvqghpE95D7vDhw4iIiICTkxMEAgFSUlLU5klKSoK7uzvEYjECAgKQk5PT+YUaipoaIDIS2L//fsCNHct3VYS0Ge8hJ5VK4e3tjaSkpCZf37ZtG2JjYxEfH49Tp07B29sbYWFhKCkp6eRKDcC9e8CECUBqKmBqygXdmDF8V0VIu/B+djU8PBzh4eHNvp6YmIg5c+Zg1qxZAIDVq1dj3759WLt2LRYuXKi1Ompra1FbW6uclkgkAID6+nrU19dr7XN01r17MJo0CcKMDDBTU8h27wYbOZK7AFgDir4yiD7TQYbS/5qsH+8h15K6ujqcPHkScXFxyjahUIjg4GBkZ2dr9bOWLl2KhIQEtfa0tDSYmppq9bN0jVFtLfw//hh2eXloEItx7N13caeqituSa6P09HQtVkg0pe/9X11d3ep5dTrkSktLIZPJYG9vr9Jub2+P/Px85XRwcDDy8vIglUrh7OyM5ORkBAYGNtvelLi4OMTGxiqnJRIJXFxcEBoaCgsLi45ZQV1QXQ2jyEgI8/LAzMyAPXsQMHLkw9/XjPr6eqSnpyMkJATGxsZaLJS0hqH0v2JPqzV0OuRaKyMjQ6P2pohEIohEIiQlJSEpKQkyGTesrbGxsf7+skil3EmGrCzAzAyC1FR0a0fAPUiv+60L0Pf+12TdeD/x0BIbGxsYGRmhuLhYpb24uBgODg4d8pkGM56cVMrdXJ+VBZibA7/8Amgp4AjRJTodciYmJvD19UVmZqayTS6XIzMzs9ndzvYyiJGBq6q4m+sPHbofcE0MWU6IPuB9d7WqqgqXL19WThcUFCA3NxfW1tZwdXVFbGwsoqKi4OfnB39/f6xYsQJSqVR5tlXboqOjER0dDYlEAktLyw75DF4pAu7IEcDCggu4Rx/luypCOgzvIXfixAmMHj1aOa04+B8VFYX169djypQpuH37NhYvXoxbt27Bx8cHqampaicjSCtUVnIBd/QoYGkJpKUB/v58V0VIh+I95IKCgsBYyw8ejomJQUxMTKfU0/jEg96QSIDwcOD337mAS08H9HmXnJB/6PQxOT7o5YkHiYQb/+333wErKyAjgwKOGAzet+RIB6uo4ALu2DGgZ08u4IYP57sqQjoNbck1oldnVysquPHfKOCIAaOQa0RvdlfLy7nx344fB6ytgcxMCjhikGh3VR/dvcsF3IkTwCOPcAHn7c13VYTwgrbkGunyu6t37wIhIVzA2dgABw9SwBGDRiHXSJfeXS0r40bwPXnyfsB5efFdFSG8ot1VfaEIuNOnAVtbLuA8PfmuihDe0ZacPrhzhxui/PRpwM6Ou+meAo4QABRyXV9pKRdwubmAvT0XcEOG8F0VITqDQq6RLnXi4fZt7hkMeXn3A87Dg++qCNEpFHKNdJkTDyUlXMCdOQM4OHDDJg0ezHdVhOgcOvHQFSkC7n//AxwduS24gQP5rooQnURbcl1NcTEwejQXcE5O3BYcBRwhzaKQ60pu3eIC7tw5oFcvLuAGDOC7KkJ0GoVcV1FUxAXc+fOAszMXcP37810VITqPQq4RnTy7qgi4/HzAxYULuH79+K6KkC6BQq4RnTu7evMmEBQEXLgAuLpyAde3L99VEdJlUMjpssJCLuAuXgTc3LiA69OH76oI6VLoEhIdIpMz5BSUoaSyBs5VZRg+6xkILl++H3Du7nyXSEiXQyGnI1LPFiFhzzkUVdTAUXIbW7a8A0F5Eap7ucD011+5oCOEaIx2V3VA6tkizP3xFIoqauAkKcHWLXFwLy/C35b2CHk6AamVJnyXSEiXRSHHM5mcIWHPOTCAC7jNcXArv4VrVg6YOn0pblrYIWHPOcjkLT+2kRDSNAo5nuUUlKGoogbOFcXYtjkOrhXFuGrliKnTuIBjAIoqapBTUMZ3qYR0SRRyjXT2dXIllVzAbd0cB5eKYhT05AKuyMJWbT5CiOYo5Brp7OvkXMqLsXXzQjhLSvBXTydMnbYUtyxs1OazMxd3Sj2E6Bs6u8qnv/7CsBkTIZDcxl/WvTB16scoMX9EZRYBAAdLMfx7W/NTIyFdHIUcX65cAUaPhuD6dVS598XU8fG4baYaZIJ/vsZHeMBIKFBfBiHkoWh3lQ+XL3N3Mly/DgwaBLPfj2DJv4PhYKm6S+pgKcY3LwzHOE9HfuokRA/Qllxnu3SJu9m+sJAbyffgQcDBAeMcgRAPB+UdD3bm3C4qbcER0j4Ucp3p4kUu4G7e5J7FcPAg92yGfxgJBQjs+0gLCyCEaEovdlcjIyPRs2dPTJ48WaV97969GDhwIPr374///ve/PFX3jwsXuF3Umze5p2llZakEHCGkY+hFyM2fPx8bNmxQaWtoaEBsbCwOHjyI06dP47PPPsOdO3f4KTA/nwu4oiJg6FAu4Ozs+KmFEAOjFyEXFBQEc3NzlbacnBwMGTIEvXr1gpmZGcLDw5GWltb5xZ0/zwXcrVtcwGVmck+4J4R0Ct5D7vDhw4iIiICTkxMEAgFSUlLU5klKSoK7uzvEYjECAgKQk5Pz0OXevHkTvXr1Uk736tULhYWF2iz94c6d4wKuuBjw8uKOwVHAEdKpeA85qVQKb29vJCUlNfn6tm3bEBsbi/j4eJw6dQre3t4ICwtDSUlJJ1eqof/9jzvJUFIC+PhwAWejficDIaRj8X52NTw8HOHh4c2+npiYiDlz5mDWrFkAgNWrV2Pfvn1Yu3YtFi5c2Oz7nJycVLbcCgsL4e/v3+z8tbW1qK2tVU5LJBIAQH19Perr61u9PgCAs2fRLSwMgtu3wXx80HDgAGBhAWi6nC5I0Vca9xnRCkPpf03Wj/eQa0ldXR1OnjyJuLg4ZZtQKERwcDCys7NbfK+/vz/Onj2LwsJCWFpa4sCBA1i0aFGz8y9duhQJCQlq7WlpaTA1NW11zeZXr2Lk4sUQSCQo79MHv7/5JuqPH2/1+/VFeno63yUYNH3v/+rq6lbPq9MhV1paCplMBvtGl1rY29sjPz9fOR0cHIy8vDxIpVI4OzsjOTkZgYGB+OKLLzB69GjI5XK89dZbeOSR5q9Bi4uLQ2xsrHJaIpHAxcUFoaGhsLCwaPI9MjnDyWt3UVpVCxszEfwqrsNk9mwIJBLIhw9HjwMHENKzZzt7oWupr69Heno6QkJCYGxszHc5BsdQ+l+xp9UaOh1yrZWRkdFk+9NPP42nn366VcsQiUQQiURq7cbGxk3+sjw4XDkAeBT/hc3b34OoWgL4+UGYlgahgQXcg5rrN9I59L3/NVk33k88tMTGxgZGRkYoLi5WaS8uLoaDg0OHfGZrxpN7cLhyABhSfAWbtr4Lq2oJ8hz7I+PLjYABBxwhukSnQ87ExAS+vr7IzMxUtsnlcmRmZiIwMLBDPvNh48k9OFw5AAy5dRmbtr6LnjWVOO04EC9O+RCLfi2k4coJ0RG8765WVVXh8uXLyumCggLk5ubC2toarq6uiI2NRVRUFPz8/ODv748VK1ZAKpUqz7ZqW1JSEpKSkiCTyZp8XTFcOQAI5TKs2PsFrGqqcMppIKKeW4JKUQ9I/hmunO5DJYR/vIfciRMnMHr0aOW04uB/VFQU1q9fjylTpuD27dtYvHgxbt26BR8fH6SmpqqdjNCW6OhoREdHQyKRwNLSUu31B4chlwuN8O+J7+Ctwz/gzfGxqBKZNjkfIYQ/vIdcUFAQGGt51y4mJgYxMTGdVFHLGg9DfsXGBa88895D5yOE8EOnj8nx4WEnHvx7W8PRUozmRnkTAHCk4coJ0RkUco087MSDkVCA+AgPAFALOhqunBDdQyHXBuM8HfHNC8NpuHJCugDej8npmoedXVUY5+lIw5UT0gVQyDXysLOrD6LhygnRfbS7SgjRaxRyjbTmti5CSNdBu6uNKHZXKyoqYGVlpdFoB4QbBaO6uhoSiUSvbxDXVYbS/4q/y4ddYwtQyDWrsrISAODi4sJzJYSQ5lRWVj702LmAtSYKDZBcLsfNmzdhbm4OgYDOmLaWYhy+69evNzsOH+k4htL/jDFUVlbCyckJQmHLR91oS64ZQqEQzs7OfJfRZVlYWOj1H5muM4T+f9gWnAKdeCCE6DUKOUKIXqOQI1olEokQHx/f5FDypONR/6ujEw+EEL1GW3KEEL1GIUcI0WsUcoQQvUYhRwjRaxRyhBC9RiFHOlRkZCR69uyJyZMnq7Tv3bsXAwcORP/+/fHf//6Xp+r0X1P9f/36dQQFBcHDwwNeXl5ITk7mscKOR5eQkA516NAhVFZW4ocffsBPP/0EAGhoaICHhweysrJgaWkJX19f/P7773jkERqAVNua6v+ioiIUFxfDx8cHt27dgq+vLy5evIgePXrwXG3HoC050qGCgoJgbm6u0paTk4MhQ4agV69eMDMzQ3h4ONLS0niqUL811f+Ojo7w8fEBADg4OMDGxgZlZWU8VNc5KORIsw4fPoyIiAg4OTlBIBAgJSVFbZ6kpCS4u7tDLBYjICAAOTk5D13uzZs30atXL+V0r169UFhYqM3S9UJH9f+DTp48CZlMptdDilHIkWZJpVJ4e3sjKSmpyde3bduG2NhYxMfH49SpU/D29kZYWBhKSko6uVL91NH9X1ZWhhkzZmDNmjXaLFv3MEJaAQD7+eefVdr8/f1ZdHS0clomkzEnJye2dOlSlfmysrLYpEmTlNO//fYbmzhxonJ6/vz5bNOmTR1TuJ7QZv8zxlhNTQ174okn2IYNGzqsZl1BW3KkTerq6nDy5EkEBwcr24RCIYKDg5Gdnd3ie/39/XH27FkUFhaiqqoKBw4cQFhYWEeXrFfa0/+MMcycORNjxozBiy++2NGl8o4GzSRtUlpaCplMBnt7e5V2e3t75OfnK6eDg4ORl5cHqVQKZ2dnJCcnIzAwEF988QVGjx4NuVyOt956i86saqg9/S+TybBt2zZ4eXkpj/Nt3LgRQ4cO7cxV6DQUcqRDZWRkNNn+9NNP4+mnn+7kagxPc/0vl8s7uRL+0O4qaRMbGxsYGRmhuLhYpb24uBgODg48VWU4qP9bj0KOtImJiQl8fX2RmZmpbJPL5cjMzERgYCCPlRkG6v/Wo91V0qyqqipcvnxZOV1QUIDc3FxYW1vD1dUVsbGxiIqKgp+fH/z9/bFixQpIpVLMmjWLx6r1B/W/lvB9epforqysLAZA7V9UVJRynpUrVzJXV1dmYmLC/P392bFjx/grWM9Q/2sH3btKCNFrdEyOEKLXKOQIIXqNQo4Qotco5Agheo1CjhCi1yjkCCF6jUKOEKLXKOQIIXqNQo4Qotco5Ah5CHd3d6xYsYLvMkgbUcgRncYYQ0NDA99laEVdXR3fJRgkCjmiVUFBQYiJiUFMTAwsLS1hY2ODRYsWQXGL9MaNG+Hn5wdzc3M4ODhg+vTpKg9eOXToEAQCAQ4cOABfX1+IRCIcPXoUV65cwYQJE2Bvbw8zMzOMGDFCbUBId3d3fPjhh5gxYwbMzMzg5uaG3bt34/bt25gwYQLMzMzg5eWFEydOqLzv6NGjeOKJJ9C9e3e4uLhg3rx5kEqlyvW5du0a3njjDQgEAggEgla9T1HPBx98gBkzZsDCwgIvv/wy6urqEBMTA0dHR4jFYri5uWHp0qVa/zmQB/A7PgDRN6NGjWJmZmZs/vz5LD8/n/3444/M1NSUrVmzhjHG2Pfff8/279/Prly5wrKzs1lgYCALDw9Xvl8x8oaXlxdLS0tjly9fZnfu3GG5ubls9erV7MyZM+zixYvsvffeY2KxmF27dk35Xjc3N2Ztbc1Wr17NLl68yObOncssLCzYuHHj2Pbt29mFCxfYxIkT2eDBg5lcLmeMMXb58mXWo0cPtnz5cnbx4kX222+/sWHDhrGZM2cyxhi7c+cOc3Z2ZkuWLGFFRUWsqKioVe9T1GNhYcE+//xzdvnyZXb58mX22WefMRcXF3b48GF29epVduTIEbZ58+YO/7kYMgo5olWjRo1SCRHGGHv77bfZ4MGDm5z/jz/+YABYZWUlY+x+yKWkpDz0s4YMGcJWrlypnHZzc2MvvPCCcrqoqIgBYIsWLVK2ZWdnMwDKsJo9ezZ7+eWXVZZ75MgRJhQK2b1795TLXb58uco8rX3fg08lY4yx1157jY0ZM0alf0jHot1VonWPPvqoym5dYGAgLl26BJlMhpMnTyIiIgKurq4wNzfHqFGjAAB///23yjL8/PxUpquqqrBgwQIMHjwYVlZWMDMzw/nz59Xe5+Xlpfxe8ZCXBx/QomhT7CLn5eVh/fr1MDMzU/4LCwuDXC5HQUFBs+vY2vc1Xo+ZM2ciNzcXAwcOxLx585CWltbsZxDtoJGBSaepqalBWFgYwsLCsGnTJtja2uLvv/9GWFiY2kH5Hj16qEwvWLAA6enp+Pzzz9GvXz90794dkydPVnufsbGx8ntF0DbVpniQS1VVFV555RXMmzdPrV5XV9dm16W172u8HsOHD0dBQQEOHDiAjIwMPPfccwgODsZPP/3U7GeR9qGQI1p3/Phxleljx46hf//+yM/Px507d7Bs2TK4uLgAgNpJgOb89ttvmDlzJiIjIwFwIXP16tV21zp8+HCcO3cO/fr1a3YeExMTyGQyjd/XHAsLC0yZMgVTpkzB5MmTMW7cOJSVlcHa2lrjZZGHo91VonV///03YmNjceHCBWzZsgUrV67E/Pnz4erqChMTE6xcuRJ//fUXdu/ejQ8++KBVy+zfvz927tyJ3Nxc5OXlYfr06Vp5rN7bb7+N33//HTExMcjNzcWlS5ewa9cuxMTEKOdxd3fH4cOHUVhYiNLS0la/rymJiYnYsmUL8vPzcfHiRSQnJ8PBwQFWVlbtXhfSNAo5onUzZszAvXv34O/vj+joaMyfPx8vv/wybG1tsX79eiQnJ8PDwwPLli3D559/3qplJiYmomfPnnjssccQERGBsLAwDB8+vN21enl54ddff8XFixfxxBNPYNiwYVi8eDGcnJyU8yxZsgRXr15F3759YWtr2+r3NcXc3Byffvop/Pz8MGLECFy9ehX79++HUEh/ih2FnvFAtCooKAg+Pj50hwDRGfTfByFEr1HIEUL0Gu2uEkL0Gm3JEUL0GoUcIUSvUcgRQvQahRwhRK9RyBFC9BqFHCFEr1HIEUL0GoUcIUSvUcgRQvTa/wM4VWbglJ1cKAAAAABJRU5ErkJggg==\",\n      \"text/plain\": [\n       \"<Figure size 300x300 with 1 Axes>\"\n      ]\n     },\n     \"metadata\": {},\n     \"output_type\": \"display_data\"\n    }\n   ],\n   \"source\": [\n    \"plt.figure(figsize=(3, 3))\\n\",\n    \"# plot the line\\n\",\n    \"plt.plot([q[0] for q in raw], [10**(m*np.log10(q[0]) + c) for q in raw], label='linear regression', color='r')\\n\",\n    \"# plot the raw data\\n\",\n    \"plt.scatter([q[0] for q in raw], [q[1] for q in raw], label='raw data')\\n\",\n    \"plt.xscale('log')\\n\",\n    \"plt.yscale('log')\\n\",\n    \"plt.xlabel('parameters')\\n\",\n    \"plt.ylabel('tokens')\\n\",\n    \"plt.title('compute optimal models')\\n\",\n    \"plt.grid()\\n\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 15,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"name\": \"stdout\",\n     \"output_type\": \"stream\",\n     \"text\": [\n      \"predicted parameters for 1.240000e+08 tokens: 2.292426e+09\\n\"\n     ]\n    }\n   ],\n   \"source\": [\n    \"xquery = 124e6 # query model size here (e.g. GPT-2 small is 124M)\\n\",\n    \"yquery = 10**(m*np.log10(xquery) + c)\\n\",\n    \"print(f\\\"predicted parameters for {xquery:e} tokens: {yquery:e}\\\")\"\n   ]\n  }\n ],\n \"metadata\": {\n  \"kernelspec\": {\n   \"display_name\": \"MLexperiments\",\n   \"language\": \"python\",\n   \"name\": \"python3\"\n  },\n  \"language_info\": {\n   \"codemirror_mode\": {\n    \"name\": \"ipython\",\n    \"version\": 3\n   },\n   \"file_extension\": \".py\",\n   \"mimetype\": \"text/x-python\",\n   \"name\": \"python\",\n   \"nbconvert_exporter\": \"python\",\n   \"pygments_lexer\": \"ipython3\",\n   \"version\": \"3.10.8\"\n  },\n  \"orig_nbformat\": 4,\n  \"vscode\": {\n   \"interpreter\": {\n    \"hash\": \"37637b3a7f3e35bfec4c00b744758bb7c354a471edaa8e8e5b028b90b494f3c9\"\n   }\n  }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 2\n}\n"
  },
  {
    "path": "train.py",
    "content": "\"\"\"\nThis training script can be run both on a single gpu in debug mode,\nand also in a larger training run with distributed data parallel (ddp).\n\nTo run on a single GPU, example:\n$ python train.py --batch_size=32 --compile=False\n\nTo run with DDP on 4 gpus on 1 node, example:\n$ torchrun --standalone --nproc_per_node=4 train.py\n\nTo run with DDP on 4 gpus across 2 nodes, example:\n- Run on the first (master) node with example IP 123.456.123.456:\n$ torchrun --nproc_per_node=8 --nnodes=2 --node_rank=0 --master_addr=123.456.123.456 --master_port=1234 train.py\n- Run on the worker node:\n$ torchrun --nproc_per_node=8 --nnodes=2 --node_rank=1 --master_addr=123.456.123.456 --master_port=1234 train.py\n(If your cluster does not have Infiniband interconnect prepend NCCL_IB_DISABLE=1)\n\"\"\"\n\nimport os\nimport time\nimport math\nimport pickle\nfrom contextlib import nullcontext\n\nimport numpy as np\nimport torch\n\nfrom model import GPTConfig, GPT\nimport yaml\nfrom torch.nn.parallel import DistributedDataParallel as DDP\nfrom trainers.trainer import Trainer\n\n# load config.yaml from current directory\nwith open('config/config.yaml') as f:\n    conf = yaml.load(f, Loader=yaml.FullLoader)\n    # nested dictionary structure\n    config = {}               \n    for k, v in conf.items():\n        for k2, v2 in v.items():\n            config[k2] = v2\n    # convert to dotdict\nprint(config)\ntrainer = Trainer(config)\n\ntrainer.train()"
  },
  {
    "path": "train_reward_model.py",
    "content": "\nfrom trainers.reward_trainer import RewardModelTrainer\nimport yaml\n\nfrom datasets import load_dataset\nfrom torch.utils.data import Dataset\nfrom tqdm import tqdm\nimport tiktoken\nimport torch\n\n\n# with inspiration from CarperAI's trlx library\n\nwith open('config/config_reward.yaml') as f:\n    conf = yaml.load(f, Loader=yaml.FullLoader)\n    # nested dictionary structure\n    config = {}               \n    for k, v in conf.items():\n        for k2, v2 in v.items():\n            config[k2] = v2\nprint(config)\n\ndef create_comparison_dataset(path=\"CarperAI/openai_summarize_comparisons\", split=\"train\"):\n    dataset = load_dataset(path, split=split)\n    pairs = []\n    for sample in tqdm(dataset):\n        pair = {}\n        prompt = sample[\"prompt\"]\n        chosen_summary = sample[\"chosen\"]\n        rejected_summary = sample[\"rejected\"]\n        if chosen_summary == rejected_summary:\n            continue\n        if len(chosen_summary.split()) < 5 or len(rejected_summary.split()) < 5:\n            continue\n        pair[\"chosen\"] = prompt + \"\\n\" + chosen_summary\n        pair[\"rejected\"] = prompt + \"\\n\" + rejected_summary\n        pairs.append(pair)\n    return pairs\n\n\nclass PairwiseDataset(Dataset):\n    def __init__(self, pairs, max_length):\n        # self.chosen_input_ids = []\n        # self.chosen_attn_masks = []\n        # self.rejected_input_ids = []\n        # self.rejected_attn_masks = []\n        self.chosens = []\n        self.rejecteds = []\n        self.enc = tiktoken.get_encoding(\"gpt2\")\n        for pair in tqdm(pairs):\n            chosen_enc = self.enc.encode(\"<|startoftext|>\" + pair['chosen'] + \"<|endoftext|>\", allowed_special=\"all\")[-max_length:]\n            rejected_enc = self.enc.encode(\"<|startoftext|>\" + pair['rejected'] + \"<|endoftext|>\", allowed_special=\"all\")[-max_length:]\n            self.chosens.append(chosen_enc)\n            self.rejecteds.append(rejected_enc)  \n\n    def __len__(self):\n        return len(self.chosens)\n    \n    def __getitem__(self, idx):\n        return (\n            self.chosens[idx],\n            self.rejecteds[idx],\n        )\n\nclass DataCollatorReward:\n    def __call__(self, data):\n        batch = {}\n        batch[\"chosen_ids\"] = torch.tensor([f[0] for f in data])\n        batch[\"rejected_ids\"] = torch.tensor([f[1] for f in data])\n        return batch\n\ndef collate_fn(data):\n    batch = {}\n    batch[\"chosen_ids\"] = torch.tensor([f[0] for f in data])\n    batch[\"rejected_ids\"] = torch.tensor([f[1] for f in data])\n    return batch   \n\ndata_path = \"CarperAI/openai_summarize_comparisons\"\ntrain_pairs = create_comparison_dataset(data_path, \"train\")\nval_pairs = create_comparison_dataset(data_path, \"test\")\n\n\n# Make pairwise datasets for training\nprint(\"Creating pairwise datasets\")\ntrain_dataset = PairwiseDataset(train_pairs, max_length=config['block_size'])\nval_dataset = PairwiseDataset(val_pairs, max_length=config['block_size'])\n\ntrainer = RewardModelTrainer(config, train_dataset, val_dataset, collate_fn=collate_fn)\n\ntrainer.train()\n\n"
  },
  {
    "path": "train_reward_model_simple.py",
    "content": "\nfrom trainers.reward_trainer import ProbRewardModelTrainer\nimport yaml\n\nfrom tqdm import tqdm\nimport tiktoken\nimport torch\n\nwith open('config/config_reward.yaml') as f:\n    conf = yaml.load(f, Loader=yaml.FullLoader)\n    # nested dictionary structure\n    config = {}               \n    for k, v in conf.items():\n        for k2, v2 in v.items():\n            config[k2] = v2\nprint(config)\n\ntrainer = ProbRewardModelTrainer(config, discrete_reward=True)\n\ntrainer.train()\n\n"
  },
  {
    "path": "train_rl.py",
    "content": "import yaml\nfrom torch.nn.parallel import DistributedDataParallel as DDP\nfrom trainers.rl_trainer import PolicyGradientTrainer, GumbelTrainer\n\n# load config.yaml from current directory\nwith open('config/config_rl.yaml') as f:\n    conf = yaml.load(f, Loader=yaml.FullLoader)\n    # nested dictionary structure\n    config = {}               \n    for k, v in conf.items():\n        for k2, v2 in v.items():\n            config[k2] = v2\n    # convert to dotdict\n\nif config['method'] == 'gumbel':\n    print('Using Gumbel method')\n    assert config['hard_code_reward'] == False, 'hard_code_reward must be False for Gumbel method'\n    trainer = GumbelTrainer(config)\nelif config['method'] == 'pg':\n    print('Using Policy Gradient method')\n    trainer = PolicyGradientTrainer(config)\nelse:\n    raise NotImplementedError\n        \ntrainer.train()"
  },
  {
    "path": "trainers/reward_trainer.py",
    "content": "import torch\nimport numpy as np\nfrom torch.nn.parallel import DistributedDataParallel as DDP\nfrom torch.distributed import destroy_process_group\nimport wandb\nimport time, os\nfrom model import RLHF\nfrom trainers.trainer import Trainer\n\n# This one for reward models similar to InstructGPT paper (rewards based on comparisons)\nclass RewardModelTrainer(Trainer):\n    def __init__(self, config, train_data, val_data, collate_fn):\n        super().__init__(config)\n        import tiktoken\n        self.enc = tiktoken.get_encoding(\"gpt2\")\n        self.mode = 'reward'\n        from torch.utils.data import DataLoader\n        train_dataloader = DataLoader(train_data, batch_size=self.batch_size, shuffle=True, collate_fn=collate_fn)\n        val_dataloader = DataLoader(val_data, batch_size=self.batch_size, shuffle=True, collate_fn=collate_fn)\n        self.train_dataloader = train_dataloader\n        self.val_dataloader = val_dataloader\n\n\n    def get_batch(self, split):\n        dataloader = self.train_dataloader if split == 'train' else self.val_dataloader\n        batch = next(iter(dataloader))\n        x, y = batch['chosen_ids'], batch['rejected_ids']\n        if self.device_type == 'cuda':\n            # pin arrays x,y, which allows us to move them to GPU asynchronously (non_blocking=True)\n            x, y = x.pin_memory().to(self.device, non_blocking=True), y.pin_memory().to(self.device, non_blocking=True)\n        else:\n            x, y = x.to(self.device), y.to(self.device)\n        return x, y\n\n    # helps estimate an arbitrarily accurate loss over either split using many batches\n    @torch.no_grad()\n    def estimate_loss(self, model, ctx):\n        out = {}\n        model.eval()\n        for split in ['train', 'val']:\n            losses = torch.zeros(self.eval_iters)\n            for k in range(self.eval_iters):\n                chosen, rejected = self.get_batch(split)\n                with ctx:\n                    reward_chosen  = model(chosen)\n                    reward_rejected = model(rejected)\n                    loss = -torch.log(torch.sigmoid(reward_chosen - reward_rejected)).mean()                    \n                losses[k] = loss.item()\n            out[split] = losses.mean()\n        model.train()\n        return out\n    \n    def evaluate(self, model, ctx):\n        losses = self.estimate_loss(model, ctx)\n        print(f\"step {self.iter_num}: train loss {losses['train']:.4f}, val loss {losses['val']:.4f}\")\n\n        if self.wandb_log:\n            wandb.log({\n                \"iter\": self.iter_num,\n                \"train/loss\": losses['train'],\n                \"val/loss\": losses['val'],\n                \"lr\": self.lr,\n                \"mfu\": self.running_mfu*100, # convert to percentage\n            })\n        if losses['val'] < self.best_val_loss or self.always_save_checkpoint:\n            self.best_val_loss = losses['val']\n            raw_model = model.module if self.ddp else model\n            if self.iter_num > 0:\n                checkpoint = {\n                    'model': raw_model.state_dict(),\n                    'optimizer': self.optimizer.state_dict(),\n                    'model_args': self.model_args,\n                    'iter_num': self.iter_num,\n                    'best_val_loss': self.best_val_loss,\n                    'config': self.config,\n                }\n                print(f\"saving checkpoint to {self.config['out_dir_multihead']}\")\n                torch.save(checkpoint, os.path.join(self.config['out_dir_multihead'], 'ckpt.pt'))\n\n    def train(self):\n        # set up distributed training\n        self.setup_ddp()\n\n        ctx, meta_vocab_size = self.setup()\n\n        # model init\n        \n\n        model = self.init_model()\n        model = RLHF(model, self.mode)\n        print('Config of model: ', model.config)\n        \n        if self.config['init_multihead_from'] == 'scratch':\n            print(\"initializing multihead from scratch\")\n        else:\n            if self.config['init_multihead_from'] == 'resume':\n                print(f\"Resuming training from {self.config['out_dir_multihead']}\")\n                # resume training from a checkpoint.\n                ckpt_path = os.path.join(self.config['out_dir_multihead'], 'ckpt.pt')\n                checkpoint = torch.load(ckpt_path, map_location=self.device)      \n                state_dict = checkpoint['model']\n                # fix the keys of the state dictionary :(\n                # honestly no idea how checkpoints sometimes get this prefix, have to debug more\n                unwanted_prefix = '_orig_mod.'\n                for k,v in list(state_dict.items()):\n                    if k.startswith(unwanted_prefix):\n                        state_dict[k[len(unwanted_prefix):]] = state_dict.pop(k)\n                model.load_state_dict(state_dict)\n\n    \n        model.to(self.device)\n\n        # self.optimizer = torch.optim.AdamW(model.model.reward_head.parameters(), lr=1e-3)\n        self.optimizer = torch.optim.AdamW(model.model.parameters(), lr=1e-4)\n\n        model = self.setup_model(model)\n\n        # logging\n        if self.wandb_log and self.master_process:\n            wandb.init(project=self.wandb_project, name=self.wandb_run_name, config=self.config)\n\n        # training loop\n        chosen, rejected = self.get_batch('train') # fetch the very first batch\n        t0 = time.time()\n        local_iter_num = 0 # number of iterations in the lifetime of this process\n        self.running_mfu = -1.0\n        loss = None\n        while True:\n\n            # determine and set the learning rate for this iteration\n            lr = self.get_lr(self.iter_num) if self.decay_lr else self.learning_rate\n            for param_group in self.optimizer.param_groups:\n                param_group['lr'] = lr\n\n            # # every once in a while evaluate the loss on train and val sets\n            if self.iter_num % self.eval_interval == 0 and self.master_process:\n                self.evaluate(model, ctx)\n\n            if self.iter_num == 0 and self.eval_only:\n                break\n            \n            # sample a batch of data\n            chosen, rejected = self.get_batch('train')\n\n            # evaluate the loss\n            reward_chosen  = model(chosen)\n            reward_rejected = model(rejected)\n            loss = -torch.log(torch.sigmoid(reward_chosen - reward_rejected)).mean()\n\n\n\n            self.optimizer.zero_grad(set_to_none=True)\n            loss.backward()\n            self.optimizer.step()\n\n            # timing and logging\n            t1 = time.time()\n            # dt = t1 - t0\n            t0 = t1\n            self.iter_num += 1\n            local_iter_num += 1\n\n            # termination conditions\n            if self.iter_num > self.max_iters:\n                break\n\n        if self.ddp:\n            destroy_process_group()\n\n# This one is for reward models which output a probability of reward directly from a given text (no comparison)\nclass ProbRewardModelTrainer(Trainer):\n    def __init__(self, config, discrete_reward=False):\n        super().__init__(config)\n        import tiktoken\n        self.enc = tiktoken.get_encoding(\"gpt2\")\n        self.mode = 'reward'\n        self.discrete_reward = discrete_reward\n\n    def get_batch(self, split):\n        # generate a small batch of data of inputs x and targets y\n        data = self.train_data if split == 'train' else self.val_data\n        ix = torch.randint(len(data) - self.block_size, (self.batch_size,))\n        x = torch.stack([torch.from_numpy((data[i:i+self.block_size]).astype(np.int64)) for i in ix])\n        y = torch.stack([self.reward(torch.from_numpy((data[i+1:i+1+self.block_size]).astype(np.int64))) for i in ix])\n        \n\n        if self.device_type == 'cuda':\n            # pin arrays x,y, which allows us to move them to GPU asynchronously (non_blocking=True)\n            x, y = x.pin_memory().to(self.device, non_blocking=True), y.pin_memory().to(self.device, non_blocking=True)\n        else:\n            x, y = x.to(self.device), y.to(self.device)\n        return x, y\n    \n    def reward(self, sequence, t='and'):\n        if t in self.enc.decode(sequence.tolist()):\n            # print('hello')\n            return torch.tensor([0.0,1.0])\n        else:\n            return torch.tensor([1.0, 0.0])\n\n    def evaluate(self, model, ctx, X, lr):\n        losses = self.estimate_loss(model, ctx)\n        print(f\"step {self.iter_num}: train loss {losses['train']:.4f}, val loss {losses['val']:.4f}\")\n        \n        \n        text = self.enc.decode(X[self.iter_num % self.eval_interval].tolist())\n\n        try:\n            reward_probs, _ = model(X[self.iter_num % self.eval_interval].unsqueeze(0))\n            actual_reward_probs = self.reward(X[self.iter_num % self.eval_interval])[1]\n\n            print(\"input: \", text[:30], f\"expect {actual_reward_probs}, reward: {reward_probs[0][-1]} \\n\")\n        except:\n            pass\n        \n        # test_text = text[:4] + 'z' + text[4 + 1:-1]\n        test_text = list(text)\n        test_text[3] = ' '\n        test_text[4] = 'a'\n        test_text[5] = 'n'\n        test_text[6] = 'd'\n        test_text[7] = ' '\n        test_text = ''.join(test_text)\n        try:\n            test_text_enc = torch.tensor(self.enc.encode(test_text)[:self.block_size]).unsqueeze(0)\n            test_reward_probs, _ = model(test_text_enc.to(self.device))\n            actual_reward_probs = self.reward(test_text_enc[0].to(self.device))[1]\n\n            print(\"input: \", test_text[:30], f\"expect {actual_reward_probs}, reward: {test_reward_probs[0][-1]} \\n\")\n        except:\n            pass\n\n        if self.wandb_log:\n            wandb.log({\n                \"iter\": self.iter_num,\n                \"train/loss\": losses['train'],\n                \"val/loss\": losses['val'],\n                \"lr\": lr,\n                # \"mfu\": self.running_mfu*100, # convert to percentage\n            })\n        if losses['val'] < self.best_val_loss or self.always_save_checkpoint:\n            self.best_val_loss = losses['val']\n            raw_model = model.module if self.ddp else model\n            if self.iter_num > 0:\n                checkpoint = {\n                    'model': raw_model.state_dict(),\n                    'optimizer': self.optimizer.state_dict(),\n                    'model_args': self.model_args,\n                    'iter_num': self.iter_num,\n                    'best_val_loss': self.best_val_loss,\n                    'config': self.config,\n                }\n                print(f\"saving checkpoint to {self.config['out_dir_multihead']}\")\n                torch.save(checkpoint, os.path.join(self.config['out_dir_multihead'], 'ckpt.pt'))\n\n    def train(self):\n        # set up distributed training\n        self.setup_ddp()\n\n        ctx, meta_vocab_size = self.setup()\n\n        # model init\n        \n        if self.master_process:\n            os.makedirs(self.config['out_dir_multihead'], exist_ok=True)\n\n        model = self.init_model()\n        model = RLHF(model, self.mode, discrete_reward=self.discrete_reward)\n        \n        if self.config['init_multihead_from'] == 'scratch':\n            print(\"initializing multihead from scratch\")\n        else:\n            if self.config['init_multihead_from'] == 'resume':\n                print(f\"Resuming training from {self.config['out_dir_multihead']}\")\n                # resume training from a checkpoint.\n                ckpt_path = os.path.join(self.config['out_dir_multihead'], 'ckpt.pt')\n                checkpoint = torch.load(ckpt_path, map_location=self.device)      \n                state_dict = checkpoint['model']\n                # fix the keys of the state dictionary :(\n                # honestly no idea how checkpoints sometimes get this prefix, have to debug more\n                unwanted_prefix = '_orig_mod.'\n                for k,v in list(state_dict.items()):\n                    if k.startswith(unwanted_prefix):\n                        state_dict[k[len(unwanted_prefix):]] = state_dict.pop(k)\n                model.load_state_dict(state_dict)\n\n    \n        model.to(self.device)\n\n        # self.optimizer = torch.optim.AdamW(model.model.reward_head.parameters(), lr=1e-3)\n        self.optimizer = torch.optim.AdamW(model.model.parameters(), lr=1e-3)\n\n        model = self.setup_model(model)\n\n        # logging\n        if self.wandb_log and self.master_process:\n            wandb.init(project=self.wandb_project, name=self.wandb_run_name, config=self.config)\n\n        # training loop\n        X, Y = self.get_batch('train') # fetch the very first batch\n        t0 = time.time()\n        local_iter_num = 0 # number of iterations in the lifetime of this process\n        self.running_mfu = -1.0\n        while True:\n\n            # determine and set the learning rate for this iteration\n            lr = self.get_lr(self.iter_num) if self.decay_lr else self.learning_rate\n            for param_group in self.optimizer.param_groups:\n                param_group['lr'] = lr\n\n            # every once in a while evaluate the loss on train and val sets\n            if self.iter_num % self.eval_interval == 0 and self.master_process:\n                self.evaluate(model, ctx, X, lr)\n\n            if self.iter_num == 0 and self.eval_only:\n                break\n            \n            # sample a batch of data\n            X, Y = self.get_batch('train')\n\n            # evaluate the loss\n            logits, loss = model(X, Y)\n            self.optimizer.zero_grad(set_to_none=True)\n            loss.backward()\n            self.optimizer.step()\n\n            # timing and logging\n            t1 = time.time()\n            # dt = t1 - t0\n            t0 = t1\n            self.iter_num += 1\n            local_iter_num += 1\n\n            # termination conditions\n            if self.iter_num > self.max_iters:\n                break\n\n        if self.ddp:\n            destroy_process_group()"
  },
  {
    "path": "trainers/rl_trainer.py",
    "content": "import torch\nimport numpy as np\nfrom torch.nn.parallel import DistributedDataParallel as DDP\nfrom torch.distributed import destroy_process_group\nimport time, os\nfrom model import RLHF\nfrom trainers.trainer import Trainer\n\n# TODO: this works but is currently crude and incomplete, critic implementation plus PPO are obvious next steps\nclass PolicyGradientTrainer(Trainer):\n    def __init__(self, config):\n        super().__init__(config)\n        import tiktoken\n        self.enc = tiktoken.get_encoding(\"gpt2\")\n        self.mode = 'RL'\n    \n    def train(self):\n\n        self.setup_ddp()\n\n        ctx, meta_vocab_size = self.setup()\n\n        # model init\n        model = self.init_model()\n\n        model = RLHF(model, self.mode, discrete_reward=self.config['discrete_reward'])\n\n        if self.config['init_multihead_from'] == 'scratch':\n            print(\"initializing multihead from scratch\")\n        else:\n            if self.config['init_multihead_from'] == 'resume':\n                print(f\"Resuming training from {self.config['out_dir_multihead']}\")\n                # resume training from a checkpoint.\n                ckpt_path = os.path.join(self.config['out_dir_multihead'], 'ckpt.pt')\n                checkpoint = torch.load(ckpt_path, map_location=self.device)      \n                state_dict = checkpoint['model']\n                # fix the keys of the state dictionary :(\n                # honestly no idea how checkpoints sometimes get this prefix, have to debug more\n                unwanted_prefix = '_orig_mod.'\n                for k,v in list(state_dict.items()):\n                    if k.startswith(unwanted_prefix):\n                        state_dict[k[len(unwanted_prefix):]] = state_dict.pop(k)\n                model.load_state_dict(state_dict)\n\n        \n        if self.config['hard_code_reward']:\n            reward_model = None\n            print('Using hard-coded reward')\n        else:\n            print('Using learned reward model')\n            if self.config['separate_reward_model']:\n                import copy\n                reward_model = copy.deepcopy(model)\n                print('Reward model instantiated separately')\n            else:\n                reward_model = model\n                print('Reward model and actor model share backbone')\n            reward_model.to(self.device)\n        \n        model.to(self.device)\n        \n        # actor_optimizer = torch.optim.AdamW(model.model.policy_head.parameters(), lr=1e-2)\n        actor_optimizer = torch.optim.AdamW(model.parameters(), lr=1e-3)\n\n        last_time = time.time()\n        rews_all = []\n        max_iters = 100000\n        X, Y = self.get_batch('train') # fetch the very first batch\n        t0  = time.time()\n        for iter in range(max_iters):\n            \n            states, log_probs, log_probs_reference, rewards, advantages = model.generate(\n                X, self.block_size, self.device, self.block_size, reward_model=reward_model, hard_code_reward=self.config['hard_code_reward'])\n\n            # minus KL divergence\n            rets = advantages * log_probs.squeeze() #- 1*(log_probs-log_probs_reference) #- 0.05*log_probs\n            actor_loss = -rets.sum()\n            actor_optimizer.zero_grad(set_to_none=True)\n            actor_loss.backward()\n            actor_optimizer.step()\n\n            torch.mean(rewards)\n\n            rews_all.append(rewards.mean().detach().cpu().numpy())\n\n            if iter % 1000 == 0:\n                t1 = time.time()\n                print(f'iter: {iter}, time: {t1-t0}')\n                # print(actor_loss, critic_loss)\n                print(f'Actor loss: {actor_loss}, iter: {iter}')\n                print(f'rets: {np.mean(rews_all[-1000:])}')\n                current_time = time.time()\n                # print(current_time - last_time)\n                last_time = current_time\n                text = model.generate(X, self.block_size, self.device, self.block_size, reward_model=reward_model)[0]\n                for i in range(1):\n                    text_i = text[i,:]\n                    # print(reward(text_i))\n                    try:\n                        print(self.enc.decode(text_i.tolist()))\n                    except:\n                        continue \n\n\nclass GumbelTrainer(Trainer):\n    def __init__(self, config):\n        super().__init__(config)\n        import tiktoken\n        self.enc = tiktoken.get_encoding(\"gpt2\")\n        self.mode = 'RL'\n    \n    def train(self):\n\n        self.setup_ddp()\n\n        ctx, meta_vocab_size = self.setup()\n\n        # model init\n        model = self.init_model()\n\n        rl_model = RLHF(model, self.mode, discrete_reward=self.config['discrete_reward'])\n\n\n        # The current approach is to use a separate reward model because otherwise optimisation of the reward model changes upstream parameters impacting performance of the multihead\n        # I therefore load the language model from 'out_dir' and the reward model from 'out_dir_multihead'\n\n        if self.config['init_multihead_from'] == 'scratch':\n            print(\"initializing multihead from scratch\")\n        else:\n            if self.config['init_multihead_from'] == 'resume':\n                print(f\"Resuming training from {self.config['out_dir']}\")\n                # resume training from a checkpoint.\n                ckpt_path = os.path.join(self.config['out_dir'], 'ckpt.pt')\n                checkpoint = torch.load(ckpt_path, map_location=self.device)      \n                state_dict = checkpoint['model']\n                # fix the keys of the state dictionary :(\n                # honestly no idea how checkpoints sometimes get this prefix, have to debug more\n                unwanted_prefix = '_orig_mod.'\n                for k,v in list(state_dict.items()):\n                    if k.startswith(unwanted_prefix):\n                        state_dict[k[len(unwanted_prefix):]] = state_dict.pop(k)\n                model.load_state_dict(state_dict)\n\n        separate_reward_model = True     \n        if separate_reward_model:\n            print('Reward model instantiated as copy')\n            import copy\n            reward_model = copy.deepcopy(model)\n\n            print(f\"Resuming reward model from {self.config['out_dir_multihead']}\")\n\n            reward_model = RLHF(reward_model, self.mode, discrete_reward=self.config['discrete_reward'])\n            # resume training from a checkpoint.\n            ckpt_path = os.path.join(self.config['out_dir_multihead'], 'ckpt.pt')\n            checkpoint = torch.load(ckpt_path, map_location=self.device)      \n            state_dict = checkpoint['model']\n            # fix the keys of the state dictionary :(\n            # honestly no idea how checkpoints sometimes get this prefix, have to debug more\n            unwanted_prefix = '_orig_mod.'\n            for k,v in list(state_dict.items()):\n                if k.startswith(unwanted_prefix):\n                    state_dict[k[len(unwanted_prefix):]] = state_dict.pop(k)\n            reward_model.load_state_dict(state_dict)\n        else:\n            reward_model = rl_model\n        rl_model.to(self.device)\n        reward_model.to(self.device)\n\n        gumbel_optimizer = torch.optim.AdamW(rl_model.parameters(), lr=1e-3)\n\n        # initialize a GradScaler. If enabled=False scaler is a no-op\n        scaler = torch.cuda.amp.GradScaler(enabled=(self.dtype == 'float16'))\n\n        last_time = time.time()\n        rews_all = []\n        max_iters = 100000     \n        \n        X, Y = self.get_batch('train') # fetch the very first batch\n\n        X = torch.zeros((X.shape[0], 1), dtype=torch.long).to(self.device) # for now there is no prompt\n\n        t0  = time.time()\n        for iter in range(max_iters):\n            \n            for micro_step in range(self.gradient_accumulation_steps):\n                if self.ddp:\n                    # in DDP training we only need to sync gradients at the last micro step.\n                    # the official way to do this is with model.no_sync() context manager, but\n                    # I really dislike that this bloats the code and forces us to repeat code\n                    # looking at the source of that context manager, it just toggles this variable\n                    rl_model.require_backward_grad_sync = (micro_step == self.gradient_accumulation_steps - 1)\n                with ctx:\n                    states, rewards = rl_model.generate_gumbel(X, self.config['episode_length'], self.device, self.block_size, reward_model=reward_model)\n                    mean_reward = rewards.mean()\n                    loss = -mean_reward\n                    # # immediately async prefetch next batch while model is doing the forward pass on the GPU\n                    # X, Y = self.get_batch('train')\n                    # backward pass, with gradient scaling if training in fp16\n                    scaler.scale(loss).backward()\n\n            # clip the gradient\n            if self.grad_clip != 0.0:\n                scaler.unscale_(gumbel_optimizer)\n                torch.nn.utils.clip_grad_norm_(rl_model.parameters(), self.grad_clip)\n            # step the optimizer and scaler if training in fp16\n            scaler.step(gumbel_optimizer)\n            scaler.update()\n            # flush the gradients as soon as we can, no need for this memory anymore\n            gumbel_optimizer.zero_grad(set_to_none=True)\n\n            rews_all.append(mean_reward.detach().cpu().numpy())\n            eval_interval = self.config['eval_interval']\n            if iter % eval_interval == 0:\n                t1 = time.time()\n                print(f'iter: {iter}, time: {t1-t0}')\n                print(f'rets: {np.mean(rews_all[-eval_interval:])}')\n                current_time = time.time()\n                # print(current_time - last_time)\n                last_time = current_time\n                text = rl_model.generate(X, self.config['episode_length'], self.device, self.block_size, reward_model=reward_model)[0]\n                for i in range(1):\n                    text_i = text[i,:]\n                    # print(reward(text_i))\n                    try:\n                        print(self.enc.decode(text_i.tolist()))\n                    except:\n                        continue "
  },
  {
    "path": "trainers/trainer.py",
    "content": "\nimport os\nimport time\nimport math\nimport pickle\nfrom contextlib import nullcontext\n\nimport numpy as np\nimport torch\n\nfrom model import GPTConfig, GPT, RLHF\nimport yaml\nfrom torch.nn.parallel import DistributedDataParallel as DDP\nfrom torch.distributed import init_process_group, destroy_process_group\nfrom utils import dotdict\nimport wandb\n\nfrom torch.nn import functional as F\n\nclass Trainer():\n    def __init__(self, config):\n        self.config = config\n        self.from_config(config)\n\n        self.model_args = dict(n_layer=self.n_layer, n_head=self.n_head, n_embd=self.n_embd, block_size=self.block_size,\n                        bias=self.bias, vocab_size=None, dropout=self.dropout) # start with model_args from command line\n        self.meta_vocab_size = None\n        self.iter_num = 0\n        self.best_val_loss = float('inf')\n    \n    def from_config(self, config):\n        config = dotdict(config)\n\n        # IO\n        self.out_dir = config.out_dir\n        self.eval_interval = config.eval_interval\n        self.log_interval = config.log_interval\n        self.eval_iters = config.eval_iters\n        self.eval_only = config.eval_only\n        self.always_save_checkpoint = config.always_save_checkpoint\n        self.init_from = config.init_from\n        \n        # wandb\n        self.wandb_log = config.wandb_log\n        self.wandb_project = config.wandb_project\n        self.wandb_run_name = config.wandb_run_name\n\n        # data\n        self.dataset = config.dataset\n        self.gradient_accumulation_steps = config.gradient_accumulation_steps\n        self.batch_size = config.batch_size\n        self.block_size = config.block_size\n\n        # model\n        self.n_layer = config.n_layer\n        self.n_head = config.n_head\n        self.n_embd = config.n_embd\n        self.dropout = config.dropout\n        self.bias = config.bias\n\n        # optimizer\n        self.learning_rate = config.learning_rate\n        self.max_iters = config.max_iters\n        self.weight_decay = config.weight_decay\n        self.beta1 = config.beta1\n        self.beta2 = config.beta2\n        self.grad_clip = config.grad_clip\n        self.decay_lr = config.decay_lr\n        self.warmup_iters = config.warmup_iters\n        self.lr_decay_iters = config.lr_decay_iters\n        self.min_lr = config.min_lr\n\n        # DDP\n        self.backend = config.backend\n\n        # system\n        self.device = config.device\n        self.dtype = config.dtype\n        self.compile = config.compile\n        \n        print(self.out_dir)\n    \n    def setup_ddp(self):\n\n        self.ddp = int(os.environ.get('RANK', -1)) != -1 # is this a ddp run?\n        if self.ddp:\n            init_process_group(backend=self.backend)\n            self.ddp_rank = int(os.environ['RANK'])\n            self.ddp_local_rank = int(os.environ['LOCAL_RANK'])\n            self.world_size = int(os.environ['WORLD_SIZE']) # total number of training processes\n            self.device = f'cuda:{self.ddp_local_rank}'\n            torch.cuda.set_device(self.device)\n            self.master_process = self.ddp_rank == 0 # this process will do logging, checkpointing etc.\n            self.seed_offset = self.ddp_rank # each process gets a different seed\n        else:\n            # if not ddp, we are running on a single gpu, and one process\n            self.world_size = 1\n            self.master_process = True\n            self.seed_offset = 0\n            self.ddp_local_rank = None\n\n    def get_batch(self, split):\n        data = self.train_data if split == 'train' else self.val_data\n        ix = torch.randint(len(data) - self.block_size, (self.batch_size,))\n        x = torch.stack([torch.from_numpy((data[i:i+self.block_size]).astype(np.int64)) for i in ix])\n        y = torch.stack([torch.from_numpy((data[i+1:i+1+self.block_size]).astype(np.int64)) for i in ix])\n        if self.device_type == 'cuda':\n            # pin arrays x,y, which allows us to move them to GPU asynchronously (non_blocking=True)\n            x, y = x.pin_memory().to(self.device, non_blocking=True), y.pin_memory().to(self.device, non_blocking=True)\n        else:\n            x, y = x.to(self.device), y.to(self.device)\n        return x, y\n    \n    def get_lr(self, it):\n         # learning rate decay scheduler (cosine with warmup)\n        # 1) linear warmup for warmup_iters steps\n        if it < self.warmup_iters:\n            return self.learning_rate * it / self.warmup_iters\n        # 2) if it > lr_decay_iters, return min learning rate\n        if it > self.lr_decay_iters:\n            return self.min_lr\n        # 3) in between, use cosine decay down to min learning rate\n        decay_ratio = (it - self.warmup_iters) / (self.lr_decay_iters - self.warmup_iters)\n        assert 0 <= decay_ratio <= 1\n        coeff = 0.5 * (1.0 + math.cos(math.pi * decay_ratio)) # coeff ranges 0..1\n        return self.min_lr + coeff * (self.learning_rate - self.min_lr)\n    \n    def init_model(self):\n        if self.init_from == 'scratch':\n            # init a new model from scratch\n            print(\"Initializing a new model from scratch\")\n            # determine the vocab size we'll use for from-scratch training\n            if self.meta_vocab_size is None:\n                print(\"defaulting to vocab_size of GPT-2 to 50304 (50257 rounded up for efficiency)\")\n            self.model_args['vocab_size'] = self.meta_vocab_size if self.meta_vocab_size is not None else 50304\n            gptconf = GPTConfig(**self.model_args)\n            model = GPT(gptconf)\n        elif self.init_from == 'resume':\n            print(f\"Resuming training from {self.out_dir}\")\n            # resume training from a checkpoint.\n            ckpt_path = os.path.join(self.out_dir, 'ckpt.pt')\n            checkpoint = torch.load(ckpt_path, map_location=self.device)      \n            checkpoint_model_args = checkpoint['model_args']\n            # force these config attributes to be equal otherwise we can't even resume training\n            # the rest of the attributes (e.g. dropout) can stay as desired from command line\n            for k in ['n_layer', 'n_head', 'n_embd', 'block_size', 'bias', 'vocab_size']:\n                self.model_args[k] = checkpoint_model_args[k]\n            # create the model\n            gptconf = GPTConfig(**self.model_args)\n            model = GPT(gptconf)\n            state_dict = checkpoint['model']\n            # fix the keys of the state dictionary :(\n            # honestly no idea how checkpoints sometimes get this prefix, have to debug more\n            unwanted_prefix = '_orig_mod.'\n            for k,v in list(state_dict.items()):\n                if k.startswith(unwanted_prefix):\n                    state_dict[k[len(unwanted_prefix):]] = state_dict.pop(k)\n            model.load_state_dict(state_dict)\n            self.iter_num = checkpoint['iter_num']\n            self.best_val_loss = checkpoint['best_val_loss']\n            self.checkpoint = checkpoint\n        elif self.init_from.startswith('gpt2'):\n            print(f\"Initializing from OpenAI GPT-2 weights: {self.init_from}\")\n            # initialize from OpenAI GPT-2 weights\n            override_args = dict(dropout=self.dropout)\n            model = GPT.from_pretrained(init_from, override_args)\n            # read off the created config params, so we can store them into checkpoint correctly\n            for k in ['n_layer', 'n_head', 'n_embd', 'block_size', 'bias', 'vocab_size']:\n                self.model_args[k] = getattr(model.config, k)\n                # crop down the model block size if desired, using model surgery\n        if self.block_size < model.config.block_size:\n            model.crop_block_size(self.block_size)\n            self.model_args['block_size'] = self.block_size # so that the checkpoint will have the right value\n        \n        return model\n    def setup(self):\n        if self.master_process:\n            os.makedirs(self.out_dir, exist_ok=True)\n\n        torch.manual_seed(1337 + self.seed_offset)\n        torch.backends.cuda.matmul.allow_tf32 = True # allow tf32 on matmul\n        torch.backends.cudnn.allow_tf32 = True # allow tf32 on cudnn\n        self.device_type = 'cuda' if 'cuda' in self.device else 'cpu' # for later use in torch.autocast\n        # note: float16 data type will automatically use a GradScaler\n        ptdtype = {'float32': torch.float32, 'bfloat16': torch.bfloat16, 'float16': torch.float16}[self.dtype]\n        ctx = nullcontext() if self.device_type == 'cpu' else torch.amp.autocast(device_type=self.device_type, dtype=ptdtype)\n\n        # poor man's data loader\n        data_dir = os.path.join('data', self.dataset)\n        self.train_data = np.memmap(os.path.join(data_dir, 'train.bin'), dtype=np.uint16, mode='r')\n        self.val_data = np.memmap(os.path.join(data_dir, 'val.bin'), dtype=np.uint16, mode='r')\n\n\n        # attempt to derive vocab_size from the dataset\n        meta_path = os.path.join(data_dir, 'meta.pkl')\n        meta_vocab_size = None\n        if os.path.exists(meta_path):\n            with open(meta_path, 'rb') as f:\n                meta = pickle.load(f)\n            meta_vocab_size = meta['vocab_size']\n            print(f\"found vocab_size = {meta_vocab_size} (inside {meta_path})\")\n        \n        return ctx, meta_vocab_size\n\n    def setup_model(self, model):\n        # compile the model\n        if self.compile:\n            print(\"compiling the model... (takes a ~minute)\")\n            unoptimized_model = model\n            model = torch.compile(model) # requires PyTorch 2.0\n\n        # wrap model into DDP container\n        if self.ddp:\n            model = DDP(model, device_ids=[self.ddp_local_rank])\n        \n        return model\n    \n    def evaluate(self, model, ctx, lr):\n        losses = self.estimate_loss(model, ctx)\n        print(f\"step {self.iter_num}: train loss {losses['train']:.4f}, val loss {losses['val']:.4f}\")\n        if self.wandb_log:\n            wandb.log({\n                \"iter\": self.iter_num,\n                \"train/loss\": losses['train'],\n                \"val/loss\": losses['val'],\n                \"lr\": lr,\n                # \"mfu\": running_mfu*100, # convert to percentage\n            })\n        if losses['val'] < self.best_val_loss or self.always_save_checkpoint:\n            self.best_val_loss = losses['val']\n            raw_model = model.module if self.ddp else model\n            if self.iter_num > 0:\n                checkpoint = {\n                    'model': raw_model.state_dict(),\n                    'optimizer': self.optimizer.state_dict(),\n                    'model_args': self.model_args,\n                    'iter_num': self.iter_num,\n                    'best_val_loss': self.best_val_loss,\n                    'config': self.config,\n                }\n                print(f\"saving checkpoint to {self.out_dir}\")\n                torch.save(checkpoint, os.path.join(self.out_dir, 'ckpt.pt'))\n    \n    def train(self):\n        # set up distributed training\n        self.setup_ddp()\n\n        ctx, meta_vocab_size = self.setup()\n\n        # model init\n        model = self.init_model()\n    \n        model.to(self.device)\n\n        # initialize a GradScaler. If enabled=False scaler is a no-op\n        scaler = torch.cuda.amp.GradScaler(enabled=(self.dtype == 'float16'))\n\n        # optimizer\n        self.optimizer = model.configure_optimizers(self.weight_decay, self.learning_rate, \\\n         (self.beta1, self.beta2), self.device_type)\n        if self.init_from == 'resume':\n            self.optimizer.load_state_dict(self.checkpoint['optimizer'])\n\n        model = self.setup_model(model)\n\n        # logging\n        if self.wandb_log and self.master_process:\n            wandb.init(project=self.wandb_project, name=self.wandb_run_name, config=self.config)\n\n        # training loop\n        X, Y = self.get_batch('train') # fetch the very first batch\n        t0 = time.time()\n        local_iter_num = 0 # number of iterations in the lifetime of this process\n        running_mfu = -1.0\n        while True:\n\n            # determine and set the learning rate for this iteration\n            lr = self.get_lr(self.iter_num) if self.decay_lr else self.learning_rate\n            for param_group in self.optimizer.param_groups:\n                param_group['lr'] = lr\n\n            # evaluate the loss on train/val sets and write checkpoints\n            if self.iter_num % self.eval_interval == 0 and self.master_process:\n                self.evaluate(model, ctx, lr)\n\n            if self.iter_num == 0 and self.eval_only:\n                break\n\n            # forward backward update, with optional gradient accumulation to simulate larger batch size\n            # and using the GradScaler if data type is float16\n            for micro_step in range(self.gradient_accumulation_steps):\n                if self.ddp:\n                    # in DDP training we only need to sync gradients at the last micro step.\n                    # the official way to do this is with model.no_sync() context manager, but\n                    # I really dislike that this bloats the code and forces us to repeat code\n                    # looking at the source of that context manager, it just toggles this variable\n                    model.require_backward_grad_sync = (micro_step == self.gradient_accumulation_steps - 1)\n                with ctx:\n                    logits, loss = model(X, Y)\n                # immediately async prefetch next batch while model is doing the forward pass on the GPU\n                X, Y = self.get_batch('train')\n                # backward pass, with gradient scaling if training in fp16\n                scaler.scale(loss).backward()\n            # clip the gradient\n            if self.grad_clip != 0.0:\n                scaler.unscale_(self.optimizer)\n                torch.nn.utils.clip_grad_norm_(model.parameters(), self.grad_clip)\n            # step the optimizer and scaler if training in fp16\n            scaler.step(self.optimizer)\n            scaler.update()\n            # flush the gradients as soon as we can, no need for this memory anymore\n            self.optimizer.zero_grad(set_to_none=True)\n\n            # timing and logging\n            t1 = time.time()\n            dt = t1 - t0\n            t0 = t1\n            if self.iter_num % self.log_interval == 0 and self.master_process:\n                lossf = loss.item() # loss as float. note: this is a CPU-GPU sync point\n                if local_iter_num >= 5: # let the training loop settle a bit\n                    mfu = model.estimate_mfu(self.batch_size * self.world_size * self.gradient_accumulation_steps, dt)\n                    running_mfu = mfu if running_mfu == -1.0 else 0.9*running_mfu + 0.1*mfu\n                print(f\"iter {self.iter_num}: loss {lossf:.4f}, time {dt*1000:.2f}ms, mfu {running_mfu*100:.2f}%\")\n            self.iter_num += 1\n            local_iter_num += 1\n\n            # termination conditions\n            if self.iter_num > self.max_iters:\n                break\n\n        if self.ddp:\n            destroy_process_group()\n        \n    # helps estimate an arbitrarily accurate loss over either split using many batches\n    @torch.no_grad()\n    def estimate_loss(self, model, ctx):\n        out = {}\n        model.eval()\n        for split in ['train', 'val']:\n            losses = torch.zeros(self.eval_iters)\n            for k in range(self.eval_iters):\n                X, Y = self.get_batch(split)\n                with ctx:\n                    logits, loss = model(X, Y)\n                losses[k] = loss.item()\n            out[split] = losses.mean()\n        model.train()\n        return out"
  },
  {
    "path": "transformer_sizing.ipynb",
    "content": "{\n \"cells\": [\n  {\n   \"attachments\": {},\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"### Transformer Theoretical Model\\n\",\n    \"\\n\",\n    \"This notebook stores a bunch of analysis about a Transformer, e.g. estimates the number of FLOPs, parameters, peak memory footprint, checkpoint size, etc.\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 1,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"from collections import OrderedDict\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 2,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"# config_args = {\\n\",\n    \"#     'gpt2':         dict(n_layer=12, n_head=12, n_embd=768),  # 124M params\\n\",\n    \"#     'gpt2-medium':  dict(n_layer=24, n_head=16, n_embd=1024), # 350M params\\n\",\n    \"#     'gpt2-large':   dict(n_layer=36, n_head=20, n_embd=1280), # 774M params\\n\",\n    \"#     'gpt2-xl':      dict(n_layer=48, n_head=25, n_embd=1600), # 1558M params\\n\",\n    \"# }[model_type]\\n\",\n    \"\\n\",\n    \"block_size = 1024\\n\",\n    \"vocab_size = 50257\\n\",\n    \"n_layer = 12\\n\",\n    \"n_head = 12\\n\",\n    \"n_embd = 768\\n\",\n    \"bias = False\\n\",\n    \"assert not bias, \\\"this notebook assumes bias=False just for simplicity\\\"\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 3,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"name\": \"stdout\",\n     \"output_type\": \"stream\",\n     \"text\": [\n      \"we see: 124337664, expected: 124337664, match: True\\n\",\n      \"name                 params     ratio (%) \\n\",\n      \"emebedding/position      786432     0.6325\\n\",\n      \"embedding/token        38597376    31.0424\\n\",\n      \"embedding              39383808    31.6749\\n\",\n      \"attention/ln                768     0.0006\\n\",\n      \"attention/kqv           1769472     1.4231\\n\",\n      \"attention/proj           589824     0.4744\\n\",\n      \"attention               2360064     1.8981\\n\",\n      \"mlp/ln                      768     0.0006\\n\",\n      \"mlp/ffw                 2359296     1.8975\\n\",\n      \"mlp/proj                2359296     1.8975\\n\",\n      \"mlp                     4719360     3.7956\\n\",\n      \"block                   7079424     5.6937\\n\",\n      \"transformer            84953088    68.3245\\n\",\n      \"ln_f                        768     0.0006\\n\",\n      \"dense                         0     0.0000\\n\",\n      \"total                 124337664   100.0000\\n\"\n     ]\n    }\n   ],\n   \"source\": [\n    \"def params():\\n\",\n    \"    \\\"\\\"\\\" estimates the number of parameters in the model\\\"\\\"\\\"\\n\",\n    \"    out = OrderedDict()\\n\",\n    \"\\n\",\n    \"    # token and position embeddings\\n\",\n    \"    out['emebedding/position'] = n_embd * block_size\\n\",\n    \"    out['embedding/token'] = n_embd * vocab_size\\n\",\n    \"    out['embedding'] = out['emebedding/position'] + out['embedding/token']\\n\",\n    \"\\n\",\n    \"    # attention blocks\\n\",\n    \"    out['attention/ln'] = n_embd # note, bias=False in our LN\\n\",\n    \"    out['attention/kqv'] = n_embd * 3*n_embd\\n\",\n    \"    out['attention/proj'] = n_embd**2\\n\",\n    \"    out['attention'] = out['attention/ln'] + out['attention/kqv'] + out['attention/proj']\\n\",\n    \"\\n\",\n    \"    # MLP blocks\\n\",\n    \"    ffw_size = 4*n_embd # feed forward size\\n\",\n    \"    out['mlp/ln'] = n_embd\\n\",\n    \"    out['mlp/ffw'] = n_embd * ffw_size\\n\",\n    \"    out['mlp/proj'] = ffw_size * n_embd\\n\",\n    \"    out['mlp'] = out['mlp/ln'] + out['mlp/ffw'] + out['mlp/proj']\\n\",\n    \"    \\n\",\n    \"    # the transformer and the rest of it\\n\",\n    \"    out['block'] = out['attention'] + out['mlp']\\n\",\n    \"    out['transformer'] = n_layer * out['block']\\n\",\n    \"    out['ln_f'] = n_embd # final layernorm\\n\",\n    \"    out['dense'] = 0 # 0 because of parameter sharing. This layer uses the weights from the embedding layer\\n\",\n    \"\\n\",\n    \"    # total\\n\",\n    \"    out['total'] = out['embedding'] + out['transformer'] + out['ln_f'] + out['dense']\\n\",\n    \"\\n\",\n    \"    return out\\n\",\n    \"\\n\",\n    \"# compare our param count to that reported by PyTorch\\n\",\n    \"p = params()\\n\",\n    \"params_total = p['total']\\n\",\n    \"print(f\\\"we see: {params_total}, expected: {124337664}, match: {params_total == 124337664}\\\")\\n\",\n    \"# create a header\\n\",\n    \"print(f\\\"{'name':20s} {'params':10s} {'ratio (%)':10s}\\\")\\n\",\n    \"for k,v in p.items():\\n\",\n    \"    print(f\\\"{k:20s} {v:10d} {v/params_total*100:10.4f}\\\")\\n\",\n    \"    \"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 4,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"name\": \"stdout\",\n     \"output_type\": \"stream\",\n     \"text\": [\n      \"est checkpoint size: 1.49 GB\\n\",\n      \"measured with wc -c ckpt.pt: 1542470366\\n\",\n      \"fluff ratio: 103.38%\\n\"\n     ]\n    }\n   ],\n   \"source\": [\n    \"# we can now calculate the size of each checkpoint\\n\",\n    \"# params are stored in fp32, and the AdamW optimizer has 2 additional buffers per param for statistics\\n\",\n    \"params_bytes = params_total*4\\n\",\n    \"params_and_buffers_bytes = params_bytes + 2*params_bytes\\n\",\n    \"print(f\\\"est checkpoint size: {params_and_buffers_bytes/1e9:.2f} GB\\\")\\n\",\n    \"measured_bytes = 1542470366 # from wc -c ckpt.pt\\n\",\n    \"print(f\\\"measured with wc -c ckpt.pt: {measured_bytes}\\\")\\n\",\n    \"print(f\\\"fluff ratio: {measured_bytes/params_and_buffers_bytes*100:.2f}%\\\")\"\n   ]\n  },\n  {\n   \"attachments\": {},\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"We can also estimate the ratio of our GPU memory that will be taken up just by the weights and the buffers inside the AdamW optimizer\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 5,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"name\": \"stdout\",\n     \"output_type\": \"stream\",\n     \"text\": [\n      \"memory ratio taken up just for parameters: 3.73%\\n\"\n     ]\n    }\n   ],\n   \"source\": [\n    \"gpu_memory = 40e9 # 40 GB A100 GPU, roughly\\n\",\n    \"print(f\\\"memory ratio taken up just for parameters: {params_and_buffers_bytes / gpu_memory * 100:.2f}%\\\")\"\n   ]\n  },\n  {\n   \"attachments\": {},\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"i.e. not that much of the memory for this tiny model, most of the memory is activations (forward and backward). This of course changes dramatically for larger and larger models.\"\n   ]\n  },\n  {\n   \"attachments\": {},\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"Let's estimate FLOPs for a single forward pass.\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 6,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"name\": \"stdout\",\n     \"output_type\": \"stream\",\n     \"text\": [\n      \"name                 flops          ratio (%) \\n\",\n      \"attention/kqv            3623878656     1.2426\\n\",\n      \"attention/scores         1610612736     0.5522\\n\",\n      \"attention/reduce         1610612736     0.5522\\n\",\n      \"attention/proj           1207959552     0.4142\\n\",\n      \"attention                8053063680     2.7612\\n\",\n      \"mlp/ffw1                 4831838208     1.6567\\n\",\n      \"mlp/ffw2                 4831838208     1.6567\\n\",\n      \"mlp                      9663676416     3.3135\\n\",\n      \"block                   17716740096     6.0747\\n\",\n      \"transformer            212600881152    72.8963\\n\",\n      \"dense                   79047426048    27.1037\\n\",\n      \"forward_total          291648307200   100.0000\\n\",\n      \"backward_total         583296614400   200.0000\\n\",\n      \"total                  874944921600   300.0000\\n\"\n     ]\n    }\n   ],\n   \"source\": [\n    \"def flops():\\n\",\n    \"    # we only count Weight FLOPs, all other layers (LayerNorm, Softmax, etc) are effectively irrelevant\\n\",\n    \"    # we count actual FLOPs, not MACs. Hence 2* all over the place\\n\",\n    \"    # basically for any matrix multiply A (BxC) @ B (CxD) -> (BxD) flops are 2*B*C*D\\n\",\n    \"\\n\",\n    \"    out = OrderedDict()\\n\",\n    \"    head_size = n_embd // n_head\\n\",\n    \"\\n\",\n    \"    # attention blocks\\n\",\n    \"    # 1) the projection to key, query, values\\n\",\n    \"    out['attention/kqv'] = 2 * block_size * (n_embd * 3*n_embd)\\n\",\n    \"    # 2) calculating the attention scores\\n\",\n    \"    out['attention/scores'] = 2 * block_size * block_size * n_embd\\n\",\n    \"    # 3) the reduction of the values (B, nh, T, T) x (B, nh, T, hs) -> (B, nh, T, hs)\\n\",\n    \"    out['attention/reduce'] = 2 * n_head * (block_size * block_size * head_size)\\n\",\n    \"    # 4) the final linear projection\\n\",\n    \"    out['attention/proj'] = 2 * block_size * (n_embd * n_embd)\\n\",\n    \"    out['attention'] = sum(out['attention/'+k] for k in ['kqv', 'scores', 'reduce', 'proj'])\\n\",\n    \"\\n\",\n    \"    # MLP blocks\\n\",\n    \"    ffw_size = 4*n_embd # feed forward size\\n\",\n    \"    out['mlp/ffw1'] = 2 * block_size * (n_embd * ffw_size)\\n\",\n    \"    out['mlp/ffw2'] = 2 * block_size * (ffw_size * n_embd)\\n\",\n    \"    out['mlp'] = out['mlp/ffw1'] + out['mlp/ffw2']\\n\",\n    \"\\n\",\n    \"    # the transformer and the rest of it\\n\",\n    \"    out['block'] = out['attention'] + out['mlp']\\n\",\n    \"    out['transformer'] = n_layer * out['block']\\n\",\n    \"    out['dense'] = 2 * block_size * (n_embd * vocab_size)\\n\",\n    \"\\n\",\n    \"    # forward,backward,total\\n\",\n    \"    out['forward_total'] = out['transformer'] + out['dense']\\n\",\n    \"    out['backward_total'] = 2 * out['forward_total'] # use common estimate of bwd = 2*fwd\\n\",\n    \"    out['total'] = out['forward_total'] + out['backward_total']\\n\",\n    \"\\n\",\n    \"    return out\\n\",\n    \"    \\n\",\n    \"# compare our param count to that reported by PyTorch\\n\",\n    \"f = flops()\\n\",\n    \"flops_total = f['forward_total']\\n\",\n    \"print(f\\\"{'name':20s} {'flops':14s} {'ratio (%)':10s}\\\")\\n\",\n    \"for k,v in f.items():\\n\",\n    \"    print(f\\\"{k:20s} {v:14d} {v/flops_total*100:10.4f}\\\")\\n\",\n    \"    \"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 7,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"name\": \"stdout\",\n     \"output_type\": \"stream\",\n     \"text\": [\n      \"palm_flops: 875062886400, flops: 874944921600, ratio: 1.0001\\n\"\n     ]\n    }\n   ],\n   \"source\": [\n    \"# now here is an estimate copy pasted from the PaLM paper\\n\",\n    \"# this formula is often used to calculate MFU (model flops utilization)\\n\",\n    \"def palm_flops():\\n\",\n    \"    \\\"\\\"\\\"estimate of the model flops following PaLM paper formula\\\"\\\"\\\"\\n\",\n    \"    # non-embedding model parameters. note that we do not subtract the\\n\",\n    \"    # embedding/token params because those are tied and get used in the last layer.\\n\",\n    \"    N = params()['total'] - params()['emebedding/position']\\n\",\n    \"    L, H, Q, T = n_layer, n_head, n_embd//n_head, block_size\\n\",\n    \"    mf_per_token = 6*N + 12*L*H*Q*T\\n\",\n    \"    mf = mf_per_token * block_size\\n\",\n    \"    return mf\\n\",\n    \"\\n\",\n    \"print(f\\\"palm_flops: {palm_flops():d}, flops: {flops()['total']:d}, ratio: {palm_flops()/flops()['total']:.4f}\\\")\"\n   ]\n  },\n  {\n   \"attachments\": {},\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"Ok they are quite similar, giving some confidence that my math in flops() function was ~ok. Now, A100 is cited at 312TFLOPS bfloat16 on tensor cores. So what is our model flops utilization (MFU)? I trained the model above with a batch_size of 20 and grad_accum of 5, which runs in about 755ms on a single A100 GPU. We get:\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 8,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"name\": \"stdout\",\n     \"output_type\": \"stream\",\n     \"text\": [\n      \"fraction of A100 used: 37.14%\\n\"\n     ]\n    }\n   ],\n   \"source\": [\n    \"# here is what we currently roughly measure\\n\",\n    \"batch_size = 20 * 5 # 5 is grad_accum, so total batch size is 100\\n\",\n    \"measured_time = 0.755 # in seconds per iteration\\n\",\n    \"measured_throughput = batch_size / measured_time\\n\",\n    \"flops_achieved = f['total'] * measured_throughput\\n\",\n    \"\\n\",\n    \"# A100 is cited to be 312 TFLOPS of bloat16 running on tensor cores\\n\",\n    \"a100_flops_promised = 312e12\\n\",\n    \"\\n\",\n    \"# the fraction of the A100 that we are using:\\n\",\n    \"print(f\\\"fraction of A100 used: {flops_achieved / a100_flops_promised * 100:.2f}%\\\")\"\n   ]\n  },\n  {\n   \"attachments\": {},\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"For reference, we'd prefer to be somewhere around 50%+, and not just for a single GPU but for an entire DDP run. So we still have some work to do, but at least we're within a factor of ~2X of what is achievable with this GPU.\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 9,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"name\": \"stdout\",\n     \"output_type\": \"stream\",\n     \"text\": [\n      \"time needed to train the model: 3.46 days\\n\"\n     ]\n    }\n   ],\n   \"source\": [\n    \"# Finally let's check out the 6ND approximation as total cost of training in FLOPs\\n\",\n    \"model_size = params()['total'] # this is number of parameters, N\\n\",\n    \"tokens_num = 300e9 # 300B tokens, this is dataset size in tokens, D\\n\",\n    \"a100_flops = 312e12 # 312 TFLOPS\\n\",\n    \"assumed_mfu = 0.3 # assume this model flops utilization (take the current 37% from above and add some DDP overhead)\\n\",\n    \"flops_throughput = a100_flops * 8 * assumed_mfu # assume an 8XA100 node at 30% utilization\\n\",\n    \"flops_needed = 6 * model_size * tokens_num # 6ND\\n\",\n    \"time_needed_s = flops_needed / flops_throughput # in seconds\\n\",\n    \"print(f\\\"time needed to train the model: {time_needed_s/3600/24:.2f} days\\\")\"\n   ]\n  },\n  {\n   \"attachments\": {},\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"This is not a bad estimate at all. I trained this model and it converged in roughly 4 days. Btw as a good reference for where 6ND comes from and some intuition around it I recommend [Dzmitry's post](https://medium.com/@dzmitrybahdanau/the-flops-calculus-of-language-model-training-3b19c1f025e4).\"\n   ]\n  },\n  {\n   \"attachments\": {},\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"Now, FLOPs are just one constraint, the other that we have to keep a close track of is the memory bandwidth. TODO estimate LOAD/STORE costs of our model later.\"\n   ]\n  }\n ],\n \"metadata\": {\n  \"kernelspec\": {\n   \"display_name\": \"pytorch2\",\n   \"language\": \"python\",\n   \"name\": \"python3\"\n  },\n  \"language_info\": {\n   \"codemirror_mode\": {\n    \"name\": \"ipython\",\n    \"version\": 3\n   },\n   \"file_extension\": \".py\",\n   \"mimetype\": \"text/x-python\",\n   \"name\": \"python\",\n   \"nbconvert_exporter\": \"python\",\n   \"pygments_lexer\": \"ipython3\",\n   \"version\": \"3.10.8\"\n  },\n  \"orig_nbformat\": 4,\n  \"vscode\": {\n   \"interpreter\": {\n    \"hash\": \"7f5833218766b48e6e35e4452ee875aac0e2188d05bbe5298f2c62b79f08b222\"\n   }\n  }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 2\n}\n"
  },
  {
    "path": "utils.py",
    "content": "import torch\n\nclass dotdict(dict):\n    \"\"\"dot.notation access to dictionary attributes\"\"\"\n    __getattr__ = dict.get\n    __setattr__ = dict.__setitem__\n    __delattr__ = dict.__delitem__\n\n"
  }
]