Repository: dipanjanS/tensorflow2-crash-course Branch: main Commit: 435dd3b5d951 Files: 5 Total size: 1.3 MB Directory structure: gitextract_zgykkyzr/ ├── .gitignore ├── LICENSE ├── README.md └── notebooks/ ├── TF2_Crash_Course_01_NN_Essentials_Layers,_Ops,_Gradient_Descent.ipynb └── TF2_Crash_Course_02_Deep_Learning_Essentials.ipynb ================================================ FILE CONTENTS ================================================ ================================================ FILE: .gitignore ================================================ # Byte-compiled / optimized / DLL files __pycache__/ *.py[cod] *$py.class # C extensions *.so # Distribution / packaging .Python build/ develop-eggs/ dist/ downloads/ eggs/ .eggs/ lib/ lib64/ parts/ sdist/ var/ wheels/ pip-wheel-metadata/ share/python-wheels/ *.egg-info/ .installed.cfg *.egg MANIFEST # PyInstaller # Usually these files are written by a python script from a template # before PyInstaller builds the exe, so as to inject date/other infos into it. *.manifest *.spec # Installer logs pip-log.txt pip-delete-this-directory.txt # Unit test / coverage reports htmlcov/ .tox/ .nox/ .coverage .coverage.* .cache nosetests.xml coverage.xml *.cover *.py,cover .hypothesis/ .pytest_cache/ # Translations *.mo *.pot # Django stuff: *.log local_settings.py db.sqlite3 db.sqlite3-journal # Flask stuff: instance/ .webassets-cache # Scrapy stuff: .scrapy # Sphinx documentation docs/_build/ # PyBuilder target/ # Jupyter Notebook .ipynb_checkpoints # IPython profile_default/ ipython_config.py # pyenv .python-version # pipenv # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. # However, in case of collaboration, if having platform-specific dependencies or dependencies # having no cross-platform support, pipenv may install dependencies that don't work, or not # install all needed dependencies. #Pipfile.lock # PEP 582; used by e.g. github.com/David-OConnor/pyflow __pypackages__/ # Celery stuff celerybeat-schedule celerybeat.pid # SageMath parsed files *.sage.py # Environments .env .venv env/ venv/ ENV/ env.bak/ venv.bak/ # Spyder project settings .spyderproject .spyproject # Rope project settings .ropeproject # mkdocs documentation /site # mypy .mypy_cache/ .dmypy.json dmypy.json # Pyre type checker .pyre/ ================================================ FILE: LICENSE ================================================ GNU GENERAL PUBLIC LICENSE Version 3, 29 June 2007 Copyright (C) 2007 Free Software Foundation, Inc. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The GNU General Public License is a free, copyleft license for software and other kinds of works. The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users. We, the Free Software Foundation, use the GNU General Public License for most of our software; it applies also to any other work released this way by its authors. You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things. To protect your rights, we need to prevent others from denying you these rights or asking you to surrender the rights. Therefore, you have certain responsibilities if you distribute copies of the software, or if you modify it: responsibilities to respect the freedom of others. For example, if you distribute copies of such a program, whether gratis or for a fee, you must pass on to the recipients the same freedoms that you received. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. Developers that use the GNU GPL protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License giving you legal permission to copy, distribute and/or modify it. For the developers' and authors' protection, the GPL clearly explains that there is no warranty for this free software. For both users' and authors' sake, the GPL requires that modified versions be marked as changed, so that their problems will not be attributed erroneously to authors of previous versions. Some devices are designed to deny users access to install or run modified versions of the software inside them, although the manufacturer can do so. This is fundamentally incompatible with the aim of protecting users' freedom to change the software. The systematic pattern of such abuse occurs in the area of products for individuals to use, which is precisely where it is most unacceptable. Therefore, we have designed this version of the GPL to prohibit the practice for those products. If such problems arise substantially in other domains, we stand ready to extend this provision to those domains in future versions of the GPL, as needed to protect the freedom of users. Finally, every program is threatened constantly by software patents. States should not allow patents to restrict development and use of software on general-purpose computers, but in those that do, we wish to avoid the special danger that patents applied to a free program could make it effectively proprietary. To prevent this, the GPL assures that patents cannot be used to render the program non-free. The precise terms and conditions for copying, distribution and modification follow. TERMS AND CONDITIONS 0. Definitions. "This License" refers to version 3 of the GNU General Public License. "Copyright" also means copyright-like laws that apply to other kinds of works, such as semiconductor masks. "The Program" refers to any copyrightable work licensed under this License. Each licensee is addressed as "you". "Licensees" and "recipients" may be individuals or organizations. To "modify" a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a "modified version" of the earlier work or a work "based on" the earlier work. A "covered work" means either the unmodified Program or a work based on the Program. To "propagate" a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy. Propagation includes copying, distribution (with or without modification), making available to the public, and in some countries other activities as well. To "convey" a work means any kind of propagation that enables other parties to make or receive copies. Mere interaction with a user through a computer network, with no transfer of a copy, is not conveying. An interactive user interface displays "Appropriate Legal Notices" to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion. 1. Source Code. The "source code" for a work means the preferred form of the work for making modifications to it. "Object code" means any non-source form of a work. A "Standard Interface" means an interface that either is an official standard defined by a recognized standards body, or, in the case of interfaces specified for a particular programming language, one that is widely used among developers working in that language. The "System Libraries" of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form. A "Major Component", in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it. The "Corresponding Source" for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the work's System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work. For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those subprograms and other parts of the work. The Corresponding Source need not include anything that users can regenerate automatically from other parts of the Corresponding Source. The Corresponding Source for a work in source code form is that same work. 2. Basic Permissions. All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met. This License explicitly affirms your unlimited permission to run the unmodified Program. The output from running a covered work is covered by this License only if the output, given its content, constitutes a covered work. This License acknowledges your rights of fair use or other equivalent, as provided by copyright law. You may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force. You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you. Conveying under any other circumstances is permitted solely under the conditions stated below. Sublicensing is not allowed; section 10 makes it unnecessary. 3. Protecting Users' Legal Rights From Anti-Circumvention Law. No covered work shall be deemed part of an effective technological measure under any applicable law fulfilling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention of such measures. When you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work's users, your or third parties' legal rights to forbid circumvention of technological measures. 4. Conveying Verbatim Copies. You may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program. You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee. 5. Conveying Modified Source Versions. You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions: a) The work must carry prominent notices stating that you modified it, and giving a relevant date. b) The work must carry prominent notices stating that it is released under this License and any conditions added under section 7. This requirement modifies the requirement in section 4 to "keep intact all notices". c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it. d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so. A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an "aggregate" if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate. 6. Conveying Non-Source Forms. You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways: a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange. b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge. c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b. d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements. e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d. A separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work. A "User Product" is either (1) a "consumer product", which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling. In determining whether a product is a consumer product, doubtful cases shall be resolved in favor of coverage. For a particular product received by a particular user, "normally used" refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product. A product is a consumer product regardless of whether the product has substantial commercial, industrial or non-consumer uses, unless such uses represent the only significant mode of use of the product. "Installation Information" for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made. If you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information. But this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product (for example, the work has been installed in ROM). The requirement to provide Installation Information does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modified or installed by the recipient, or for the User Product in which it has been modified or installed. Access to a network may be denied when the modification itself materially and adversely affects the operation of the network or violates the rules and protocols for communication across the network. Corresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying. 7. Additional Terms. "Additional permissions" are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permissions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid under applicable law. If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions. When you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it. (Additional permissions may be written to require their own removal in certain cases when you modify the work.) You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission. Notwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms: a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or d) Limiting the use for publicity purposes of names of licensors or authors of the material; or e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors. All other non-permissive additional terms are considered "further restrictions" within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying. If you add terms to a covered work in accord with this section, you must place, in the relevant source files, a statement of the additional terms that apply to those files, or a notice indicating where to find the applicable terms. Additional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above requirements apply either way. 8. Termination. You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11). However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation. Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice. Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, you do not qualify to receive new licenses for the same material under section 10. 9. Acceptance Not Required for Having Copies. You are not required to accept this License in order to receive or run a copy of the Program. Ancillary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require acceptance. However, nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so. 10. Automatic Licensing of Downstream Recipients. Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License. You are not responsible for enforcing compliance by third parties with this License. An "entity transaction" is a transaction transferring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations. If propagation of a covered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the party's predecessor in interest had or could give under the previous paragraph, plus a right to possession of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable efforts. You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, offering for sale, or importing the Program or any portion of it. 11. Patents. A "contributor" is a copyright holder who authorizes use under this License of the Program or a work on which the Program is based. The work thus licensed is called the contributor's "contributor version". A contributor's "essential patent claims" are all patent claims owned or controlled by the contributor, whether already acquired or hereafter acquired, that would be infringed by some manner, permitted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modification of the contributor version. For purposes of this definition, "control" includes the right to grant patent sublicenses in a manner consistent with the requirements of this License. Each contributor grants you a non-exclusive, worldwide, royalty-free patent license under the contributor's essential patent claims, to make, use, sell, offer for sale, import and otherwise run, modify and propagate the contents of its contributor version. In the following three paragraphs, a "patent license" is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement). To "grant" such a patent license to a party means to make such an agreement or commitment not to enforce a patent against the party. If you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the benefit of the patent license for this particular work, or (3) arrange, in a manner consistent with the requirements of this License, to extend the patent license to downstream recipients. "Knowingly relying" means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipient's use of the covered work in a country, would infringe one or more identifiable patents in that country that you have reason to believe are valid. If, pursuant to or in connection with a single transaction or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties receiving the covered work authorizing them to use, propagate, modify or convey a specific copy of the covered work, then the patent license you grant is automatically extended to all recipients of the covered work and works based on it. A patent license is "discriminatory" if it does not include within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are specifically granted under this License. You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of distributing software, under which you make payment to the third party based on the extent of your activity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discriminatory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with specific products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007. Nothing in this License shall be construed as excluding or limiting any implied license or other defenses to infringement that may otherwise be available to you under applicable patent law. 12. No Surrender of Others' Freedom. If conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot convey a covered work so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not convey it at all. For example, if you agree to terms that obligate you to collect a royalty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to refrain entirely from conveying the Program. 13. Use with the GNU Affero General Public License. Notwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU Affero General Public License into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the special requirements of the GNU Affero General Public License, section 13, concerning interaction through a network will apply to the combination as such. 14. Revised Versions of this License. The Free Software Foundation may publish revised and/or new versions of the GNU General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies that a certain numbered version of the GNU General Public License "or any later version" applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the GNU General Public License, you may choose any version ever published by the Free Software Foundation. If the Program specifies that a proxy can decide which future versions of the GNU General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program. Later license versions may give you additional or different permissions. However, no additional obligations are imposed on any author or copyright holder as a result of your choosing to follow a later version. 15. Disclaimer of Warranty. THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 16. Limitation of Liability. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. 17. Interpretation of Sections 15 and 16. If the disclaimer of warranty and limitation of liability provided above cannot be given local legal effect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively state the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see . Also add information on how to contact you by electronic and paper mail. If the program does terminal interaction, make it output a short notice like this when it starts in an interactive mode: Copyright (C) This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, your program's commands might be different; for a GUI interface, you would use an "about box". You should also get your employer (if you work as a programmer) or school, if any, to sign a "copyright disclaimer" for the program, if necessary. For more information on this, and how to apply and follow the GNU GPL, see . The GNU General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License. But first, please read . ================================================ FILE: README.md ================================================ # TensorFlow 2 Crash Course A quick crash course in understanding the essentials of TensorFlow 2 and the integrated Keras API. ![](https://i.imgur.com/ZNRlig7.png) # Essentials Include - Understanding Tensors - Understanding Layers in TensorFlow - Tensor Operations - Gradient Descent - Autodiff using Gradient Tape - Custom Training Loops - Sequential, Functional and Subclassing APIs - Wide and Deep Networks - Deep Learning Essentials - Layers, Optimizers, Learning Rate - Dense Neural Nets (ANNs) - Activation Functions - Batch Normalization, Dropout - Dynamic Learning Rate Policies - Convolutional Neural Networks (CNNs)

![](https://i.imgur.com/kZJDvHg.png) __Author:__ [Dipanjan (DJ) Sarkar](https://www.linkedin.com/in/dipanzan/) ================================================ FILE: notebooks/TF2_Crash_Course_01_NN_Essentials_Layers,_Ops,_Gradient_Descent.ipynb ================================================ { "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "name": "TF2 Crash Course 01: NN Essentials - Layers, Ops, Gradient Descent.ipynb", "provenance": [], "collapsed_sections": [], "machine_shape": "hm" }, "kernelspec": { "name": "python3", "display_name": "Python 3" }, "accelerator": "GPU" }, "cells": [ { "cell_type": "markdown", "metadata": { "id": "kqaMA85vFQ9D" }, "source": [ "# Neural Network Layers and Ops\n", "\n", "In this session we will focus on understanding various tensor operations, NN layers, models and loss functions" ] }, { "cell_type": "markdown", "metadata": { "id": "JiN487VzFrue" }, "source": [ "# Import Dependencies" ] }, { "cell_type": "code", "metadata": { "id": "fcd9jenuFuIo" }, "source": [ "import matplotlib.pyplot as plt\n", "import pandas as pd\n", "import numpy as np\n", "import sklearn\n", "import tensorflow as tf" ], "execution_count": 2, "outputs": [] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "RrtzzfS4BmTR", "outputId": "b29085a0-7e7e-4d93-d110-529998c9e21e" }, "source": [ "!nvidia-smi" ], "execution_count": 3, "outputs": [ { "output_type": "stream", "text": [ "Sat Feb 13 01:34:34 2021 \n", "+-----------------------------------------------------------------------------+\n", "| NVIDIA-SMI 460.39 Driver Version: 460.32.03 CUDA Version: 11.2 |\n", "|-------------------------------+----------------------+----------------------+\n", "| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |\n", "| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |\n", "| | | MIG M. |\n", "|===============================+======================+======================|\n", "| 0 Tesla V100-SXM2... Off | 00000000:00:04.0 Off | 0 |\n", "| N/A 35C P0 24W / 300W | 0MiB / 16160MiB | 0% Default |\n", "| | | N/A |\n", "+-------------------------------+----------------------+----------------------+\n", " \n", "+-----------------------------------------------------------------------------+\n", "| Processes: |\n", "| GPU GI CI PID Type Process name GPU Memory |\n", "| ID ID Usage |\n", "|=============================================================================|\n", "| No running processes found |\n", "+-----------------------------------------------------------------------------+\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "XSxoE154F25s", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "14633355-f96f-4b55-8169-a6d33632966d" }, "source": [ "print(tf.__version__)" ], "execution_count": 4, "outputs": [ { "output_type": "stream", "text": [ "2.4.1\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "HUUC7dHKF5Uk", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "09ad2b93-7555-4efc-fa30-2575e278cbba" }, "source": [ "print(sklearn.__version__)" ], "execution_count": 5, "outputs": [ { "output_type": "stream", "text": [ "0.22.2.post1\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "SEz5MykHF-vM" }, "source": [ "# 1. Tensors & Tensor Ops" ] }, { "cell_type": "markdown", "metadata": { "id": "5IMLwmlMGU6T" }, "source": [ "## Constants" ] }, { "cell_type": "code", "metadata": { "id": "00g8pgcdGEr5", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "ee63a552-8662-42a5-aafb-91fc63048e98" }, "source": [ "t = tf.constant([[1., 2., 3.], \n", " [4., 5., 6.]])\n", "t" ], "execution_count": 6, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 6 } ] }, { "cell_type": "code", "metadata": { "id": "siI6vU6cGK_o", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "7d07c4a5-bfa0-4d04-d469-42146178be4b" }, "source": [ "t.dtype" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "tf.float32" ] }, "metadata": { "tags": [] }, "execution_count": 5 } ] }, { "cell_type": "code", "metadata": { "id": "WF2Ne61gG5KC", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "757e8334-6a41-419b-e135-2f5e305fb9f6" }, "source": [ "t.shape" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "TensorShape([2, 3])" ] }, "metadata": { "tags": [] }, "execution_count": 6 } ] }, { "cell_type": "markdown", "metadata": { "id": "xpoefe8vHD76" }, "source": [ "## Variables" ] }, { "cell_type": "code", "metadata": { "id": "lym63HTTHGqw", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "6f961a2f-f91a-460e-bea3-1217e1e26d99" }, "source": [ "t = tf.Variable([[1., 2., 3.], [4., 5., 6.]])\n", "t" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 7 } ] }, { "cell_type": "code", "metadata": { "id": "AErUzrRIHgQd", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "d22a2381-9c0b-4e9e-8a15-70a4133ab9dd" }, "source": [ "t.dtype, t.shape" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "(tf.float32, TensorShape([2, 3]))" ] }, "metadata": { "tags": [] }, "execution_count": 8 } ] }, { "cell_type": "code", "metadata": { "id": "xSs7VWJaHj_M", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "4416300a-8cdf-4cf7-d05a-149fad03a121" }, "source": [ "t.numpy()" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "array([[1., 2., 3.],\n", " [4., 5., 6.]], dtype=float32)" ] }, "metadata": { "tags": [] }, "execution_count": 9 } ] }, { "cell_type": "code", "metadata": { "id": "Xe-0k59nHnK2", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "8d407fa0-475d-48cb-b7b7-200885dd7a88" }, "source": [ "t[0,0].assign(99)\n", "t" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 10 } ] }, { "cell_type": "code", "metadata": { "id": "1WeX9brAHtEc", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "c54d36fd-5b54-48e3-88c6-90e7e303f40f" }, "source": [ "t[1].assign([10, 20, 30])\n", "t" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 11 } ] }, { "cell_type": "code", "metadata": { "id": "5jpSgDH0H3ie", "colab": { "base_uri": "https://localhost:8080/", "height": 166 }, "outputId": "873efa79-455f-4186-b127-74be9ecf1200" }, "source": [ "t[1] = [10, 20, 30]" ], "execution_count": null, "outputs": [ { "output_type": "error", "ename": "TypeError", "evalue": "ignored", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mTypeError\u001b[0m Traceback (most recent call last)", "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mt\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;36m10\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m20\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m30\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[0;31mTypeError\u001b[0m: 'ResourceVariable' object does not support item assignment" ] } ] }, { "cell_type": "code", "metadata": { "id": "8Rw18RXpH8iY", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "714eb14f-5f0c-4171-f73b-7da3c5121ab0" }, "source": [ "t.scatter_nd_update(indices=[[0, 0], [1, 1]],\n", " updates=[33., 44.])" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 13 } ] }, { "cell_type": "markdown", "metadata": { "id": "Jy6hyYXWJWbs" }, "source": [ "## Indexing" ] }, { "cell_type": "code", "metadata": { "id": "qr4UZheNJo4Z", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "75300d48-b6fe-4ba2-e46f-802e18f4c27d" }, "source": [ "t[1:, :]" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 14 } ] }, { "cell_type": "code", "metadata": { "id": "SUL2iyHCJt4w", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "266c87bc-62e6-4a47-f8a2-3fd552bf673c" }, "source": [ "t[1:, ...]" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 15 } ] }, { "cell_type": "code", "metadata": { "id": "U0qlIxzGI6iv", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "d480704d-dd18-4eb5-a979-0a7238d2f8a3" }, "source": [ "t[:, 1:]" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 16 } ] }, { "cell_type": "code", "metadata": { "id": "n80qXRxeJZLg", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "ca5e8317-0689-4224-af63-956c9e007011" }, "source": [ "t[..., 1:]" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 17 } ] }, { "cell_type": "markdown", "metadata": { "id": "s2xWdHmOKM01" }, "source": [ "## Basic Ops" ] }, { "cell_type": "code", "metadata": { "id": "HYnK3pr-KOUK", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "6b7afce2-1137-4f1d-9884-3378bf1f9f38" }, "source": [ "t = t + 10\n", "t" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 18 } ] }, { "cell_type": "code", "metadata": { "id": "lT423HOYKPvC", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "53666247-5385-4a1c-ce93-70bd9e24c921" }, "source": [ "t = t + tf.Variable([[1, 2, 3]], dtype='float32')\n", "t" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 19 } ] }, { "cell_type": "code", "metadata": { "id": "ojyw5MvKKbSE", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "db64a35a-d313-42f5-e893-2bf324157fd2" }, "source": [ "tf.square(t)" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 20 } ] }, { "cell_type": "code", "metadata": { "id": "3RSMi7p6K1Db", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "40a47ec8-b4c4-4e30-8c35-bc92c19492ea" }, "source": [ "np.square(t)" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "array([[1936., 196., 256.],\n", " [ 441., 3136., 1849.]], dtype=float32)" ] }, "metadata": { "tags": [] }, "execution_count": 21 } ] }, { "cell_type": "code", "metadata": { "id": "r2YYpOpRKk2Y", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "0d6c3b4f-c812-4d8e-db68-b7e7e71a75c2" }, "source": [ "t @ tf.transpose(t)" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 22 } ] }, { "cell_type": "markdown", "metadata": { "id": "195zwlS8Ma85" }, "source": [ "## Ragged Tensors" ] }, { "cell_type": "code", "metadata": { "id": "QANNKuoLK2nN", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "ce2c95bc-8291-40fd-b650-e9b6f281c7e4" }, "source": [ "r = tf.ragged.constant([[1, 2], \n", " [3, 4, 5], \n", " [], \n", " [6]])\n", "r" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 23 } ] }, { "cell_type": "code", "metadata": { "id": "V8N-kDxxLt0H", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "157689c9-66c1-4ec8-8427-5cf656c8375b" }, "source": [ "r[1]" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 24 } ] }, { "cell_type": "code", "metadata": { "id": "6fIRzVqoL1fp", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "6da63158-be9a-4a73-bafa-351f4225a1cd" }, "source": [ "r2 = tf.ragged.constant([[11, 22], [], [33]])\n", "tf.concat([r, r2], axis=0)" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 25 } ] }, { "cell_type": "code", "metadata": { "id": "zhqZHJRML7SQ", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "71a2bb0b-8ed3-439d-aef7-97d7310c8e4a" }, "source": [ "r3 = tf.ragged.constant([[11, 22, 33], \n", " [44], \n", " [], \n", " [55, 66]])\n", "tf.concat([r, r3], axis=1)" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 26 } ] }, { "cell_type": "markdown", "metadata": { "id": "OY9wjqanMD0X" }, "source": [ "## Sparse Tensors" ] }, { "cell_type": "code", "metadata": { "id": "QcC7On8zMe1j", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "35905de0-32b2-4dd5-abc9-21f1287aae88" }, "source": [ "t = tf.SparseTensor(indices=[[0, 0], [1, 1], [2, 2]],\n", " values=[1., 2., 3.],\n", " dense_shape=[3, 3])\n", "t" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 27 } ] }, { "cell_type": "code", "metadata": { "id": "qTC-1FzpMqLU", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "11ae26a5-b78a-489c-8d92-a71b8559700e" }, "source": [ "tf.sparse.to_dense(t)" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 28 } ] }, { "cell_type": "code", "metadata": { "id": "711otJT7Mszc", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "610e4861-31ab-4109-d6a7-27f37b73cc35" }, "source": [ "t2 = t * 5\n", "tf.sparse.to_dense(t2)" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 29 } ] }, { "cell_type": "markdown", "metadata": { "id": "2iD9cjbnNdVw" }, "source": [ "# 2. Basic Regression with Simple NN + Custom Loss Function" ] }, { "cell_type": "code", "metadata": { "id": "BpCgoQ5sOHrE", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "936dcc1c-afa0-4fb4-a50e-ce5251ab296c" }, "source": [ "from sklearn.datasets import fetch_california_housing\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.preprocessing import StandardScaler\n", "\n", "housing = fetch_california_housing()\n", "\n", "housing" ], "execution_count": 10, "outputs": [ { "output_type": "stream", "text": [ "Downloading Cal. housing from https://ndownloader.figshare.com/files/5976036 to /root/scikit_learn_data\n" ], "name": "stderr" }, { "output_type": "execute_result", "data": { "text/plain": [ "{'DESCR': '.. _california_housing_dataset:\\n\\nCalifornia Housing dataset\\n--------------------------\\n\\n**Data Set Characteristics:**\\n\\n :Number of Instances: 20640\\n\\n :Number of Attributes: 8 numeric, predictive attributes and the target\\n\\n :Attribute Information:\\n - MedInc median income in block\\n - HouseAge median house age in block\\n - AveRooms average number of rooms\\n - AveBedrms average number of bedrooms\\n - Population block population\\n - AveOccup average house occupancy\\n - Latitude house block latitude\\n - Longitude house block longitude\\n\\n :Missing Attribute Values: None\\n\\nThis dataset was obtained from the StatLib repository.\\nhttp://lib.stat.cmu.edu/datasets/\\n\\nThe target variable is the median house value for California districts.\\n\\nThis dataset was derived from the 1990 U.S. census, using one row per census\\nblock group. A block group is the smallest geographical unit for which the U.S.\\nCensus Bureau publishes sample data (a block group typically has a population\\nof 600 to 3,000 people).\\n\\nIt can be downloaded/loaded using the\\n:func:`sklearn.datasets.fetch_california_housing` function.\\n\\n.. topic:: References\\n\\n - Pace, R. Kelley and Ronald Barry, Sparse Spatial Autoregressions,\\n Statistics and Probability Letters, 33 (1997) 291-297\\n',\n", " 'data': array([[ 8.3252 , 41. , 6.98412698, ..., 2.55555556,\n", " 37.88 , -122.23 ],\n", " [ 8.3014 , 21. , 6.23813708, ..., 2.10984183,\n", " 37.86 , -122.22 ],\n", " [ 7.2574 , 52. , 8.28813559, ..., 2.80225989,\n", " 37.85 , -122.24 ],\n", " ...,\n", " [ 1.7 , 17. , 5.20554273, ..., 2.3256351 ,\n", " 39.43 , -121.22 ],\n", " [ 1.8672 , 18. , 5.32951289, ..., 2.12320917,\n", " 39.43 , -121.32 ],\n", " [ 2.3886 , 16. , 5.25471698, ..., 2.61698113,\n", " 39.37 , -121.24 ]]),\n", " 'feature_names': ['MedInc',\n", " 'HouseAge',\n", " 'AveRooms',\n", " 'AveBedrms',\n", " 'Population',\n", " 'AveOccup',\n", " 'Latitude',\n", " 'Longitude'],\n", " 'target': array([4.526, 3.585, 3.521, ..., 0.923, 0.847, 0.894])}" ] }, "metadata": { "tags": [] }, "execution_count": 10 } ] }, { "cell_type": "code", "metadata": { "id": "MopR_4g2OT8H", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "7f01e74b-d29f-4863-da8c-17e7fba6d430" }, "source": [ "print(housing['DESCR'])" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ ".. _california_housing_dataset:\n", "\n", "California Housing dataset\n", "--------------------------\n", "\n", "**Data Set Characteristics:**\n", "\n", " :Number of Instances: 20640\n", "\n", " :Number of Attributes: 8 numeric, predictive attributes and the target\n", "\n", " :Attribute Information:\n", " - MedInc median income in block\n", " - HouseAge median house age in block\n", " - AveRooms average number of rooms\n", " - AveBedrms average number of bedrooms\n", " - Population block population\n", " - AveOccup average house occupancy\n", " - Latitude house block latitude\n", " - Longitude house block longitude\n", "\n", " :Missing Attribute Values: None\n", "\n", "This dataset was obtained from the StatLib repository.\n", "http://lib.stat.cmu.edu/datasets/\n", "\n", "The target variable is the median house value for California districts.\n", "\n", "This dataset was derived from the 1990 U.S. census, using one row per census\n", "block group. A block group is the smallest geographical unit for which the U.S.\n", "Census Bureau publishes sample data (a block group typically has a population\n", "of 600 to 3,000 people).\n", "\n", "It can be downloaded/loaded using the\n", ":func:`sklearn.datasets.fetch_california_housing` function.\n", "\n", ".. topic:: References\n", "\n", " - Pace, R. Kelley and Ronald Barry, Sparse Spatial Autoregressions,\n", " Statistics and Probability Letters, 33 (1997) 291-297\n", "\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "IY3FOBA3ObXZ", "colab": { "base_uri": "https://localhost:8080/", "height": 195 }, "outputId": "522c1153-da95-4f3f-88a9-f6ff88101900" }, "source": [ "X = pd.DataFrame(housing['data'], columns=housing['feature_names'])\n", "X.head()" ], "execution_count": 12, "outputs": [ { "output_type": "execute_result", "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
MedIncHouseAgeAveRoomsAveBedrmsPopulationAveOccupLatitudeLongitude
08.325241.06.9841271.023810322.02.55555637.88-122.23
18.301421.06.2381370.9718802401.02.10984237.86-122.22
27.257452.08.2881361.073446496.02.80226037.85-122.24
35.643152.05.8173521.073059558.02.54794537.85-122.25
43.846252.06.2818531.081081565.02.18146737.85-122.25
\n", "
" ], "text/plain": [ " MedInc HouseAge AveRooms ... AveOccup Latitude Longitude\n", "0 8.3252 41.0 6.984127 ... 2.555556 37.88 -122.23\n", "1 8.3014 21.0 6.238137 ... 2.109842 37.86 -122.22\n", "2 7.2574 52.0 8.288136 ... 2.802260 37.85 -122.24\n", "3 5.6431 52.0 5.817352 ... 2.547945 37.85 -122.25\n", "4 3.8462 52.0 6.281853 ... 2.181467 37.85 -122.25\n", "\n", "[5 rows x 8 columns]" ] }, "metadata": { "tags": [] }, "execution_count": 12 } ] }, { "cell_type": "code", "metadata": { "id": "TIFsG3kwOmOm", "colab": { "base_uri": "https://localhost:8080/", "height": 195 }, "outputId": "91277ed2-7dd5-43ce-f2c6-bb5b242d464e" }, "source": [ "y = pd.DataFrame({'price': housing['target']})\n", "y.head()" ], "execution_count": 13, "outputs": [ { "output_type": "execute_result", "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
price
04.526
13.585
23.521
33.413
43.422
\n", "
" ], "text/plain": [ " price\n", "0 4.526\n", "1 3.585\n", "2 3.521\n", "3 3.413\n", "4 3.422" ] }, "metadata": { "tags": [] }, "execution_count": 13 } ] }, { "cell_type": "code", "metadata": { "id": "S3T9NU_dOyVa", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "1b3cfd3d-a131-4c88-ad89-fec9f7e934dc" }, "source": [ "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)\n", "X_train.shape, X_test.shape" ], "execution_count": 14, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "((14448, 8), (6192, 8))" ] }, "metadata": { "tags": [] }, "execution_count": 14 } ] }, { "cell_type": "code", "metadata": { "id": "OUkulk-APF2O" }, "source": [ "scaler = StandardScaler()\n", "X_train_scaled = scaler.fit_transform(X_train)\n", "X_test_scaled = scaler.transform(X_test)" ], "execution_count": 15, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "Xht6pIuzPPFe", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "ae8e6a44-119b-4ead-dfd6-b4eaa8bf9c3f" }, "source": [ "from sklearn.linear_model import LinearRegression\n", "from sklearn.metrics import r2_score, mean_squared_error\n", "\n", "lr = LinearRegression()\n", "lr.fit(X_train_scaled, y_train)" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=False)" ] }, "metadata": { "tags": [] }, "execution_count": 36 } ] }, { "cell_type": "code", "metadata": { "id": "EAdcOhJzQjXW", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "040c1008-39a7-48e2-c31d-e35eaf10a48e" }, "source": [ "predictions = lr.predict(X_test_scaled)\n", "print('R2:', r2_score(y_test, predictions))\n", "print('MSE:', mean_squared_error(y_test, predictions))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "R2: 0.5957702326061665\n", "MSE: 0.5305677824766752\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "3kXNdLtIQzXM" }, "source": [ "## 2.1: Creating a custom loss function\n", "\n", "Create a `mse_loss(...)` function with two arguments: \n", "- the true labels `y_true` \n", "- the model predictions `y_pred` \n", "\n", "Make it return the mean squared error using TensorFlow operations. Note that you could write your own custom metrics in this way. \n", "\n", "__Tip:__ Recall that the MSE is the mean of the squares of prediction errors, which are the differences between the predictions and the labels, so you will need to use `tf.reduce_mean()` and `tf.square()` ops." ] }, { "cell_type": "code", "metadata": { "id": "fS65zON8Rh4X" }, "source": [ "def mse_loss(y_true, y_pred):\n", " return tf.reduce_mean(tf.square(y_pred - y_true))" ], "execution_count": 20, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "4hGWe6G6Ropr" }, "source": [ "## 2.2: Create a simple 1-layer NN\n", "\n", "Here we leverage the `Sequential` API of `tf.keras` to create a simple NN with 1 hidden layer" ] }, { "cell_type": "code", "metadata": { "id": "uAcYQPiySAda" }, "source": [ "model = tf.keras.models.Sequential([\n", " tf.keras.layers.Dense(16, activation=\"relu\", \n", " input_shape=(X_train.shape[1],)),\n", " tf.keras.layers.Dense(1),\n", "])" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "6MUJ3YqTSoeU" }, "source": [ "model.compile(loss=mse_loss, \n", " optimizer=tf.keras.optimizers.SGD(lr=1e-3),\n", " metrics=['mean_squared_error'])" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "1A5kxuhoSyNS", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "36046b90-f167-453e-c57c-f28e988a16a6" }, "source": [ "model.summary()" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Model: \"sequential\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "dense (Dense) (None, 16) 144 \n", "_________________________________________________________________\n", "dense_1 (Dense) (None, 1) 17 \n", "=================================================================\n", "Total params: 161\n", "Trainable params: 161\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "FdWNezYlSzSn" }, "source": [ "## 2.3: Train the model and Test its performance\n", "\n", "We will now train our simple NN and test its performance" ] }, { "cell_type": "markdown", "metadata": { "id": "TpQZbDQKGuKF" }, "source": [ "```\n", "3200 houses\n", "32 batch size\n", "\n", "every batch of data passed to the NN will have 32 houses\n", "\n", "1 epoch = 3200 // 32 = 100\n", "\n", "loss is computed after each batch of 32 houses and error\\loss is backpropagated based on gradients in each layer\n", "```" ] }, { "cell_type": "code", "metadata": { "id": "gvuo490CTGuY", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "eeeea379-7e1f-4c67-8a55-f206af57cf84" }, "source": [ "model.fit(X_train_scaled, y_train, \n", " epochs=30,\n", " batch_size=32,\n", " validation_split=0.1)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Epoch 1/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 2.3144 - mean_squared_error: 2.3144 - val_loss: 1.0867 - val_mean_squared_error: 1.0867\n", "Epoch 2/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.8915 - mean_squared_error: 0.8915 - val_loss: 0.7776 - val_mean_squared_error: 0.7776\n", "Epoch 3/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.7593 - mean_squared_error: 0.7593 - val_loss: 0.7196 - val_mean_squared_error: 0.7196\n", "Epoch 4/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.7178 - mean_squared_error: 0.7178 - val_loss: 0.6881 - val_mean_squared_error: 0.6881\n", "Epoch 5/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6878 - mean_squared_error: 0.6878 - val_loss: 0.6625 - val_mean_squared_error: 0.6625\n", "Epoch 6/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6611 - mean_squared_error: 0.6611 - val_loss: 0.6412 - val_mean_squared_error: 0.6412\n", "Epoch 7/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6387 - mean_squared_error: 0.6387 - val_loss: 0.6218 - val_mean_squared_error: 0.6218\n", "Epoch 8/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6205 - mean_squared_error: 0.6205 - val_loss: 0.6076 - val_mean_squared_error: 0.6076\n", "Epoch 9/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6045 - mean_squared_error: 0.6045 - val_loss: 0.5939 - val_mean_squared_error: 0.5939\n", "Epoch 10/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5907 - mean_squared_error: 0.5907 - val_loss: 0.5828 - val_mean_squared_error: 0.5828\n", "Epoch 11/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5782 - mean_squared_error: 0.5782 - val_loss: 0.5720 - val_mean_squared_error: 0.5720\n", "Epoch 12/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5672 - mean_squared_error: 0.5672 - val_loss: 0.5627 - val_mean_squared_error: 0.5627\n", "Epoch 13/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5571 - mean_squared_error: 0.5571 - val_loss: 0.5553 - val_mean_squared_error: 0.5553\n", "Epoch 14/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5479 - mean_squared_error: 0.5479 - val_loss: 0.5476 - val_mean_squared_error: 0.5476\n", "Epoch 15/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5396 - mean_squared_error: 0.5396 - val_loss: 0.5400 - val_mean_squared_error: 0.5400\n", "Epoch 16/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5320 - mean_squared_error: 0.5320 - val_loss: 0.5341 - val_mean_squared_error: 0.5341\n", "Epoch 17/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5250 - mean_squared_error: 0.5250 - val_loss: 0.5284 - val_mean_squared_error: 0.5284\n", "Epoch 18/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5186 - mean_squared_error: 0.5186 - val_loss: 0.5231 - val_mean_squared_error: 0.5231\n", "Epoch 19/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5126 - mean_squared_error: 0.5126 - val_loss: 0.5191 - val_mean_squared_error: 0.5191\n", "Epoch 20/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5074 - mean_squared_error: 0.5074 - val_loss: 0.5128 - val_mean_squared_error: 0.5128\n", "Epoch 21/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5025 - mean_squared_error: 0.5025 - val_loss: 0.5093 - val_mean_squared_error: 0.5093\n", "Epoch 22/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4980 - mean_squared_error: 0.4980 - val_loss: 0.5052 - val_mean_squared_error: 0.5052\n", "Epoch 23/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4936 - mean_squared_error: 0.4936 - val_loss: 0.5017 - val_mean_squared_error: 0.5017\n", "Epoch 24/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4898 - mean_squared_error: 0.4898 - val_loss: 0.4997 - val_mean_squared_error: 0.4997\n", "Epoch 25/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4864 - mean_squared_error: 0.4864 - val_loss: 0.4964 - val_mean_squared_error: 0.4964\n", "Epoch 26/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4831 - mean_squared_error: 0.4831 - val_loss: 0.4940 - val_mean_squared_error: 0.4940\n", "Epoch 27/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4798 - mean_squared_error: 0.4798 - val_loss: 0.4922 - val_mean_squared_error: 0.4922\n", "Epoch 28/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4769 - mean_squared_error: 0.4769 - val_loss: 0.4894 - val_mean_squared_error: 0.4894\n", "Epoch 29/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4745 - mean_squared_error: 0.4745 - val_loss: 0.4873 - val_mean_squared_error: 0.4873\n", "Epoch 30/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4723 - mean_squared_error: 0.4723 - val_loss: 0.4857 - val_mean_squared_error: 0.4857\n" ], "name": "stdout" }, { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 44 } ] }, { "cell_type": "code", "metadata": { "id": "OTPwbW1YTRC9" }, "source": [ "predictions = model.predict(X_test_scaled)" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "46I3RukVTcIp", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "4d5954d6-d0a5-4df7-a781-13d7a59befa0" }, "source": [ "print('R2:', r2_score(y_test, predictions))\n", "print('MSE:', mean_squared_error(y_test, predictions))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "R2: 0.6389823667167998\n", "MSE: 0.47385012331223736\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "rLA5GxjATeen" }, "source": [ "model = tf.keras.models.Sequential([\n", " tf.keras.layers.Dense(16, activation=\"relu\", \n", " input_shape=(X_train.shape[1],)),\n", " tf.keras.layers.Dense(32, activation=\"relu\"),\n", " tf.keras.layers.Dense(32, activation=\"relu\"),\n", " tf.keras.layers.Dense(1),\n", "])" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "N6MFhm3IT6IW" }, "source": [ "model.compile(loss=mse_loss, \n", " optimizer=tf.keras.optimizers.SGD(lr=1e-3),\n", " metrics=['mean_squared_error'])" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "TYrNOeLWT91V", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "5d8e9647-e780-4bf9-a67b-d7d6d6eacbe5" }, "source": [ "model.summary()" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Model: \"sequential_1\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "dense_2 (Dense) (None, 16) 144 \n", "_________________________________________________________________\n", "dense_3 (Dense) (None, 32) 544 \n", "_________________________________________________________________\n", "dense_4 (Dense) (None, 32) 1056 \n", "_________________________________________________________________\n", "dense_5 (Dense) (None, 1) 33 \n", "=================================================================\n", "Total params: 1,777\n", "Trainable params: 1,777\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "tX1Hjj7DUAWx", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "10f20612-d0a2-4ac3-a67e-add7a3b79af4" }, "source": [ "model.fit(X_train_scaled, y_train, \n", " epochs=30,\n", " batch_size=32,\n", " validation_split=0.1) " ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Epoch 1/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.6648 - mean_squared_error: 1.6648 - val_loss: 0.7879 - val_mean_squared_error: 0.7879\n", "Epoch 2/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.7352 - mean_squared_error: 0.7352 - val_loss: 0.6796 - val_mean_squared_error: 0.6796\n", "Epoch 3/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6742 - mean_squared_error: 0.6742 - val_loss: 0.6476 - val_mean_squared_error: 0.6476\n", "Epoch 4/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6394 - mean_squared_error: 0.6394 - val_loss: 0.6229 - val_mean_squared_error: 0.6229\n", "Epoch 5/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6128 - mean_squared_error: 0.6128 - val_loss: 0.6045 - val_mean_squared_error: 0.6045\n", "Epoch 6/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5887 - mean_squared_error: 0.5887 - val_loss: 0.5884 - val_mean_squared_error: 0.5884\n", "Epoch 7/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5676 - mean_squared_error: 0.5676 - val_loss: 0.5723 - val_mean_squared_error: 0.5723\n", "Epoch 8/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5490 - mean_squared_error: 0.5490 - val_loss: 0.5574 - val_mean_squared_error: 0.5574\n", "Epoch 9/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5317 - mean_squared_error: 0.5317 - val_loss: 0.5429 - val_mean_squared_error: 0.5429\n", "Epoch 10/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5158 - mean_squared_error: 0.5158 - val_loss: 0.5289 - val_mean_squared_error: 0.5289\n", "Epoch 11/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5014 - mean_squared_error: 0.5014 - val_loss: 0.5177 - val_mean_squared_error: 0.5177\n", "Epoch 12/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4891 - mean_squared_error: 0.4891 - val_loss: 0.5062 - val_mean_squared_error: 0.5062\n", "Epoch 13/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4780 - mean_squared_error: 0.4780 - val_loss: 0.4965 - val_mean_squared_error: 0.4965\n", "Epoch 14/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4685 - mean_squared_error: 0.4685 - val_loss: 0.4876 - val_mean_squared_error: 0.4876\n", "Epoch 15/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4607 - mean_squared_error: 0.4607 - val_loss: 0.4814 - val_mean_squared_error: 0.4814\n", "Epoch 16/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4541 - mean_squared_error: 0.4541 - val_loss: 0.4776 - val_mean_squared_error: 0.4776\n", "Epoch 17/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4478 - mean_squared_error: 0.4478 - val_loss: 0.4727 - val_mean_squared_error: 0.4727\n", "Epoch 18/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4435 - mean_squared_error: 0.4435 - val_loss: 0.4655 - val_mean_squared_error: 0.4655\n", "Epoch 19/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4393 - mean_squared_error: 0.4393 - val_loss: 0.4623 - val_mean_squared_error: 0.4623\n", "Epoch 20/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4351 - mean_squared_error: 0.4351 - val_loss: 0.4586 - val_mean_squared_error: 0.4586\n", "Epoch 21/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4315 - mean_squared_error: 0.4315 - val_loss: 0.4551 - val_mean_squared_error: 0.4551\n", "Epoch 22/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4282 - mean_squared_error: 0.4282 - val_loss: 0.4525 - val_mean_squared_error: 0.4525\n", "Epoch 23/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4250 - mean_squared_error: 0.4250 - val_loss: 0.4505 - val_mean_squared_error: 0.4505\n", "Epoch 24/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4227 - mean_squared_error: 0.4227 - val_loss: 0.4478 - val_mean_squared_error: 0.4478\n", "Epoch 25/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4200 - mean_squared_error: 0.4200 - val_loss: 0.4451 - val_mean_squared_error: 0.4451\n", "Epoch 26/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4178 - mean_squared_error: 0.4178 - val_loss: 0.4427 - val_mean_squared_error: 0.4427\n", "Epoch 27/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4153 - mean_squared_error: 0.4153 - val_loss: 0.4416 - val_mean_squared_error: 0.4416\n", "Epoch 28/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4134 - mean_squared_error: 0.4134 - val_loss: 0.4386 - val_mean_squared_error: 0.4386\n", "Epoch 29/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4117 - mean_squared_error: 0.4117 - val_loss: 0.4384 - val_mean_squared_error: 0.4384\n", "Epoch 30/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4096 - mean_squared_error: 0.4096 - val_loss: 0.4425 - val_mean_squared_error: 0.4425\n" ], "name": "stdout" }, { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 50 } ] }, { "cell_type": "code", "metadata": { "id": "pdtdmbGSUDDc" }, "source": [ "predictions = model.predict(X_test_scaled)" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "Hp1AA1T5UFBn", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "f591aa8d-4960-4574-a1f2-bca4917e6819" }, "source": [ "print('R2:', r2_score(y_test, predictions))\n", "print('MSE:', mean_squared_error(y_test, predictions))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "R2: 0.682790557546519\n", "MSE: 0.4163501157974675\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "Gy9lY1b5DraC" }, "source": [ "## 2.4: Activation Functions" ] }, { "cell_type": "code", "metadata": { "id": "9E9Z47YFUImv", "colab": { "base_uri": "https://localhost:8080/", "height": 391 }, "outputId": "ab38aae1-78b7-4e1b-f288-f70196ef9571" }, "source": [ "z = np.linspace(-5, 5, 200)\n", "plt.figure(figsize=(15,6))\n", "plt.plot(z, np.sign(z), \"r-\", linewidth=1, label=\"Step\")\n", "plt.plot(z, tf.nn.sigmoid(z), \"g--\", linewidth=2, label=\"Sigmoid\")\n", "plt.plot(z, tf.nn.tanh(z), \"b-\", linewidth=2, label=\"Tanh\")\n", "plt.plot(z, tf.nn.relu(z), \"m-.\", linewidth=2, label=\"ReLU\")\n", "plt.grid(True)\n", "plt.legend(loc=\"upper left\", fontsize=14)\n", "plt.title(\"Activation functions\", fontsize=14)\n", "plt.axis([-5, 5, -1.2, 2]);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3IAAAF2CAYAAAAr2areAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3xUVf7/8deZSW8ECAkkgQQIoSRIVVF0DSgIKrCKrN3FBsi6ay+r/sC2rqusfm2rYkPXXnAVVtm1xQauAoIEgUiHEAhESkjPzPn9ccOQmEKAkEnC+/l43MfM3HvuPZ87DJN8cpqx1iIiIiIiIiIth8vfAYiIiIiIiMjBUSInIiIiIiLSwiiRExERERERaWGUyImIiIiIiLQwSuRERERERERaGCVyIiIiIiIiLYwSOREROaKMMcnGGGuMGdwEdWUaY55ogno6GmP+a4wpNMb4fR0fY8x6Y8xN/o5DRESajhI5ERGpxhgz0BjjMcZ8cwjn1pZIbQI6AUsaJUCnnonGmL21HDoH+HNj1VOPm4B4oD/OvTUJY8xdxpisWg4dC/yjqeIQERH/UyInIiK/diVOUpBujOl9uBez1nqstVuttRWHH9oB6/rFWltwpOsBUoBF1tqfrbVbm6C+ellrt1tri/wdh4iINB0lciIi4mOMCQUuBGYC7wBX1FJmiDHms8puhbsrn8cbY2YBpwB/qOxKaSu7Vfq6VhpjXMaYTcaYP/7qmqmVZQZWvr7BGPNjZR05xpjnjDHRlccygBeB8Cr13FV5rFqLoDGmrTHmJWPMTmNMsTHmE2NMWpXjE40xe40xpxpjsirr+9wY07We92g9MA64tLLuWZX7rTHm3F+XrdrlsbLMJGPM25V1rTXGXPyrc+KNMa8aY/KNMUXGmCXGmGHGmInAdCCtyn1PrKOeLsaY94wxBZXbbGNMYpXjd1Xe7/nGmDWVZf5ljImpUqavMeZTY8yeyvdoqTFmWF3vi4iINC0lciIiUtW5wAZr7TLgnzjJSuC+g8aYfsDnwGpgKDAEeBMIAK4FFuAkWZ0qt01VL26t9QKvAxf9qt6LgBXW2sWVr73AdUAaTmJ5HPB45bH5lceKqtQzo477mQUcj5N4HVd5zrzKhHWfYJzumJcDJwDRwNN1XA+cboyfAG9V1n1tPWVrMw14H+iH8969YIzpAmCMCQe+AJKB3wJ9gXsqz3sT+Duwiv33/eavL26McVVePw4YVrnFA/8yxpgqRZOB84CzgZHAAOAvVY6/BuTivG/9gbuAkoO8VxEROUIC/B2AiIg0K1fgJHDgJBRFOEnQO5X7bgGWWGsnVTlnxb4nxpgyoKhqd8PquQMArwA3G2O6W2vXVO67ECcBBMBa+39Vyq83xtwCvG+M+b21tswYs9spVne3RmNMD2AscIq19svKfZcAG3ESx+cqiwYAf7DWrqosMwMnuTLW2hoTmVhrtxtjSoHiQ+xW+U9r7SuVdf0/nETwNzjvy4VAR+AEa+2OyvL73iMqxwVWHKDeU4FjgO7W2vWV512Ik3yfipOE7rvvidba3ZVlZgKXVblOEjDDWruy8vXqQ7hXERE5QtQiJyIiABhjUoCTcFpiqExiXqV698oBwGeHU4+19kdgGZWtcsaY44HulXXti2W4MeZjY8xmY0wBMBsIwklyGqo3Tsvegip1766su0+VcqX7krhKWyrransw93UQfqwSTwWwHYit3DUA+LFKEncoegNb9iVxlfWsxbmvqve9YV8SV2lLlTgAHgaeq+w6e4cxptdhxCQiIo1MiZyIiOxzJeAGNhpjKowxFcBtwEhjTOdGrusV9nevvAj42lq7AcAYkwT8G6elbwIwCKfbIzgJVmOo2tL260lY9h072J+RFvh182NgLeXKazmvqX4eV73veuOw1t6Fk/j9CzgR+NEYczkiItIsKJETERGMMQHA73HGivWvsvXDaUHa1+XuB2B4PZcqw0kGD+Q1IMUYMwRnnNYrVY4NxknYrrfWLrDWZuOM8TrYelbg/Jw7Yd8OY0wUzriznxoQ48HaTpWlCIwxcRz80gQ/AMdUnXTkVxp63/HGmOQqsXTDeQ8P6r4rZ+V8zFp7JvA8TrIvIiLNgBI5EREBOBOIAZ611mZV3YA3gMsqJ8p4CBhgjJlpjOlnjOlpjLly32QdwHrgOOPMVBlTOfFGDdbazThj8J4G2gBvVzn8M87Pp+uMMV2NMRfgTG5S1XogxBgzorKesFrq+Bln0o9njDEnG2P64iSMe6jsPtrIPsOZsXOwMWYAzkQrBzs5yGtAHs54wJONMd2MMWOrzBa5Hkgyzlp/McaY4Fqu8QlO8v1qZSyDcbqtLqaB3WKNMaHGmCeNMRmV/5bH43S7PRIJsIiIHAIlciIiAs44uM+ttfm1HHsbZ4bDEdbaJcBpQC/gW+B/wPns76Y3A6fV6CecFqouv75YFa/gtPh9aK3duW9n5Ri6a4EbKq9zJc4C3FQpMx8nCXy9sp5b6qjjMuA74IPKxzBglLW2uJ64DtWNwFogE2dymOdwkrIGs9YW4izhsBmYA2QBd7O/S+S7wIfApzj3fUEt17A4E9Rsx5lh9HNgK/Db2iZvqYMHZ4zgLJxZMt/DGWt4w8Hcj4iIHDmm4d/pIiIiIiIi0hyoRU5ERERERKSFOexEzhjT2RjzuTHmJ2PMcmNMjYVRjeMxY8xqY8yPxpiBh1uviIiIiIjI0aoxFgSvAG601i42xkQCi4wxH1trqw6IHg30qNyOB56qfBQREREREZGDdNgtctbaXGvt4srnBTjTHif8qtg44GXr+BaINsYc7JTMIiIiIiIiQiOPkatcs2YAzixmVSUAm6q83kzNZE9EREREREQaoDG6VgJgjInAmRb5OmvtnkO8xiRgEkBoaOigzp07N1Z4Rz2v14vLpbltpHnS51OaK302pdnJBQpwfoNL4sDLw4v4gb47G092dvYOa22H2o41SiJnjAnESeJetdbOrqVIDlA1K0us3FeNtXYmMBNg8ODBduHChY0RngCZmZlkZGT4OwyRWunzKc2VPpvSnOS9k8dPE37CHeHG85iHjMsy/B2SSK303dl4jDEb6jrWGLNWGuB5YIW19uE6in0AXFo5e+UQYLe1Nvdw6xYRERE5WnQ4pwNJ05Po/Wpv6OrvaETE3xqjRW4ocAmwzBizpHLf7UAXAGvt08CHwBnAaqAIuKwR6hURERE5ahiXoetdlRlcpl9DEZFm4LATOWvt14A5QBkL/OFw6xIRERE5mpTnl5N9dTYpj6QQnBDs73BEpBnRKEQRERGRZmr1javZ/vZ2sqdk+zsUEWlmGm3WShERERFpXN1ndMdb4qX7jO7+DkVEmpkWmch5vV527NjBrl278Hg8/g6nRWjTpg0rVqxo0jrdbjfR0dHExMRoCloREZFDEBQTRNobaf4OQ0SaoRaZyG3evBljDMnJyQQGBuJMnCn1KSgoIDIyssnqs9ZSXl7Otm3b2Lx5M126dGmyukVERFqyXV/sYscHO+j2t264AvSHUBGpXYv8digsLCQhIYGgoCAlcc2UMYagoCASEhIoLCz0dzgiIiItQvG6YrLGZ7H54c1sfWGrv8MRkWasRSZygLrqtRD6dxIREWmYir0VZI3LoiK/gnaj2tHpik7+DklEmjH9li0iIiLiZ9ZrWXnpSgqXFRLaM5Ter/fGuNXrSETqpkRORERExM/W37WeHe/twN3GTd8P+hIYHejvkESkmVMiJyIiIuJHeW/lseHeDeCCtDfTCEsN83dIItICKJFrYtu3b2fq1KkkJycTHBxMXFwcp556Kh9//DEAycnJzJgxw89RioiISFMo+KGAlRNXAs6ace1Ob+fniESkpWiRyw+0ZOPHj6eoqIjnn3+elJQU8vLy+OKLL8jPz/d3aCIiItKEyraVkTUuC2+xl44TO5J4XaK/QxKRFkQtck1o165dfPXVVzzwwAOceuqpJCUlceyxx3LTTTdx/vnnk5GRwYYNG7j55psxxlRbWmH+/PmccsophIWFkZCQwNVXX82ePXt8xzMyMpgyZQrXXnstbdu2pW3bttx88814vV5/3KqIiIjUw1vqJeucLEo3lRJ1QhSpT6dqSSUROShK5JpQREQEERERfPDBB5SUlNQ4Pnv2bBITE5k2bRq5ubnk5uYCsGzZMkaOHMnYsWNZunQps2fPZsmSJVx++eXVzn/11Vfxer0sWLCAZ555hpkzZ/J///d/TXJvIiIi0jDWWrKnZrNn/h6CE4NJm52GK1i/konIwWldXSv98ZcsaxtcNCAggFmzZnHVVVcxc+ZMBgwYwNChQ5kwYQLHH3887dq1w+12ExkZSceOHX3nPfTQQ5x33nnceOONvn1PPfUUAwYMIC8vj9jYWAA6derEY489hjGGXr16kZ2dzcMPP8wNN9zQePcrIiIihy3imAjcEW7S308nuGOwv8MRkRaodf35x9qm3w7S+PHj2bJlC3PmzGH06NHMnz+fIUOGcP/999d5zqJFi3jllVd8LXoREREMHToUgDVr1vjKDRkypFq3jBNOOIGcnJxqXTBFRETEv4wxJF6byPHrjidyYKS/wxGRFqp1tci1ECEhIYwYMYIRI0Ywbdo0rrzySu666y5uuummWst7vV6uvPJKrr/++hrHEhISjnS4IiIi0giKVhcBEJbiLC8QFBPkz3BEpIVTItcM9OnTh4qKCkpKSggKCsLj8VQ7PnDgQJYvX05KSkq91/nf//6HtdbXKvftt98SHx9PVFQUBQUFRyx+ERERqV/F7gqyxmRRtq2Mfh/3I3KQWuJE5PC0rq6VzVx+fj7Dhw/nlVde4ccff2TdunW8/fbbPPjgg5x66qlERUWRnJzMV199RU5ODjt27ADg1ltv5bvvvmPKlCn88MMPrF69mrlz5zJ58uRq19+yZQvXXXcdq1at4p133uGhhx6qtRVPREREmpgLQnuEEpwQTGhqqL+jEZFWQC1yTSgiIoIhQ4bw6KOPsnr1akpLS0lISODCCy/kzjvvBOCee+5h8uTJdO/endLSUqy1HHPMMXz55ZfceeednHLKKXg8Hrp168bZZ59d7foXXXQRHo+H448/HmMMV1xxhRI5ERGRZiAgMoD0f6VTnl9OQKR+/RKRw6dvkiYUHBzM/fffX+/EJkOGDGHp0qU19g8ePJh58+bVe/2AgACeeOIJnnjiicOOVURERA7frq93ETk4EneIG+MyBHXQuDgRaRzqWikiIiJyBOz53x6WnraUpcOW4inyHPgEEZGDoEROREREpJGV5pSS9dssbKklYkAE7jC3v0MSkVZGXStbiczMTH+HICIiIoCn2EPWb7Mo21pGdEY0KY/WP+u0iMihUIuciIiISCOx1rLqylUULCwgpGsIfd7ugytQv26JSOPTN4uIiIhII9n04CbyXsvDHeEm/YN0LfotIkeMEjkRERGRRrBjzg7W/nktAL1f6U1EeoSfIxKR1kyJnIiIiMhhKlxeyIoLV4CFrvd1JWZcjL9DEpFWTomciIiIyGEozy9n2dhlePZ66HBeB7rc3sXfIYnIUUCJnIiIiMgh8pZ7Wf675ZSsLSFiYAS9XuiFMcbfYYnIUUCJXDOTkZHBNddc4+8wgIbFkp6ezl133dU0AYmIiDQzBYsK2P31bgLjAkn/V7rWixORJqN15JrY9u3bmT59Oh9++CG5ublER0eTnp7ObbfdxogRI5g9ezaBgYH+DhOgWcUiIiLSHLUZ0ob+mf3BQEjnEH+HIyJHESVyTWz8+PEUFRXx/PPPk5KSQl5eHl988QX5+fkAtGvXzs8R7tecYhEREWlOvGVeXEFOx6Y2J7TxczQicjRS18omtGvXLr766iseeOABTj31VJKSkjj22GO56aabOP/884Ga3Rm3bdvG2LFjCQ0NJSkpiRdffLFGd0ZjDE899RTjxo0jLCyM1NRUPv/8czZv3szpp59OeHg4Q4cOZfHixdXimT17Nn379iU4OJjOnTvzl7/8BWut7/ivY8nLy2PcuHG+WF544YUj9E6JiIg0XyUbSviu13dsf2+7v0MRkaNYq2qRM3fXPbj4mbOeYdKgSQDMXDSTyXMn11nWTt+fzAyaOYjFuYsPWK4hIiIiiIiI4IMPPuCkk04iJOTAXTB+//vfk5uby2effUZoaCg33ngjGzZsqFHuvvvuY8aMGcyYMYPp06dz/vnn079/f6ZOncrjjz/ONddcw8SJE/nxxx8BWLRoERMmTODOO+/koosu4vvvv2fy5MlERUXxxz/+sdZYJk6cyIYNG/jkk08ICwvj+uuvZ/369Qf1HoiIiLR0uS/mUrKuhNxnc4n5bYwmNxERv2hViVxzFxAQwKxZs7jqqquYOXMmAwYMYOjQoUyYMIHjjz++RvlVq1bxn//8hwULFjBkyBAAZs2aRXJyco2yl156KRdccAEAt99+O6+//jqnn34648aNA+C6667jzDPPZMeOHcTExPDwww9zyimncPfddwOQmprKzz//zN/+9rdaE7ns7Gw++ugjvv76a4YOHQrASy+9RLdu3RrlvREREWkpkqcnE9wpmA7ndVASJyJ+06oSuYa2kE0aNMnXOncgiyYtOpyQahg/fjxnnnkmX331FQsWLGDevHn8/e9/5y9/+Qu33357tbIrV67E5XIxePBg377OnTsTHx9f47rHHHOM73lcXBwAffv29e2LjY0FnO6RMTExrFixgjPPPLPaNU466STuvvtu9uzZQ1RUVLVjK1aswOVycdxxx/n2JSUl1RqLiIhIa+St8OIKcGGMIX6yfv6JiH9pjJwfhISEMGLECKZNm8b8+fO54ooruOuuuygrKzvka1adXXLfXwdr2+f1eg94rfr+uqi/PIqIyNEo7508Fg1aRPG6Yn+HIiICKJFrFvr06UNFRQUlJSXV9vfq1Quv18uiRftbBTdv3syWLVsOu87evXvzzTffVNv39ddfk5iYSGRkZI3y+2L57rvvfPs2btzYKLGIiIg0ZwVLClj5+5UU/lhI/tx8f4cjIgIokWtS+fn5DB8+nFdeeYUff/yRdevW8fbbb/Pggw9y6qmn1ujO2LNnT04//XSmTJnCt99+y5IlS7jssssICws77JaxG2+8kS+++IK77rqL7OxsXn31Vf7+979zyy231Fq+Z8+ejBo1ismTJ7NgwQKWLFnCxIkTCQ0NPaw4REREmrOybWVkjc3CW+Ql7vdxJFyT4O+QREQAJXJNKiIigiFDhvDoo49yyimnkJaWxu23386FF17Im2++Wes5s2bNIjExkYyMDMaOHctFF11EbGxsg2a8rM/AgQN5++23effdd30Lkt92223VlhuoLZauXbsyfPhwxowZw4UXXljrxCsiIiKtgbfMS9b4LEo3lRI1JIrUp1M1xEBEmo1WNdlJcxccHMz999/P/fffX2eZzMzMaq87duzInDlzfK937NjBpEmTSElJ8e2ruvYbQExMTI19qampNfadc845nHPOOQ2OJS4ujg8++KDaviuvvLLO80VERFoqay3ZU7PZ880eghKCSJudhjvE7e+wRER8lMg1c5999hkFBQX07duXvLw87rjjDmJiYhg1apS/QxMREWm1ch7PYevzW3GFuEj/VzrBnYL9HZKISDWN0rXSGPOCMSbPGJNVx/EMY8xuY8ySym1aY9R7NCgvL+fOO++kb9++jBkzhrCwML788kvCw8P9HZqIiEir9MvHv7D6+tUA9HyxJ1GDow5whohI02usFrlZwBPAy/WU+cpae1Yj1XfUOP300zn99NP9HYaIiMhRoejnIn467yfwQpfbuxB3fpy/QxIRqVWjtMhZa78EfmmMa4mIiIj4Q8XuCrLGZlGxs4L2Y9vT9d6u/g5JRKROTTlr5QnGmKXGmI+MMWlNWK+IiIjIAa2atIqilUWEpYXR+5XeGJdmqBSR5sv8eibDQ76QMcnAXGttei3HogCvtXavMeYM4FFrbY9ayk0CJgHExcUNeuONN2qtq02bNtVmbZQD83g8uN3+mW1r9erV7N692y91S8uwd+9eIiIi/B2GSA36bB5l1gIPAtOAeD/HcgD6bEpzps9n4xk2bNgia+3g2o41SSJXS9n1wGBr7Y66ygwePNguXLiw1mMrVqygd+/ehxboUaqgoIDIyEi/1K1/LzmQzMxMMjIy/B2GSA36bB59rLUtYq04fTalOdPns/EYY+pM5Jqka6UxpqOp/FY0xhxXWW9+U9QtIiIiUpc9/9tD7ou5vtctIYkTEYFGmrXSGPM6kAHEGGM2A9OBQABr7dPAucDVxpgKoBg43zZWU6CIiIjIIajYXUHW2VmU5ZbhjnQTe26sv0MSEWmwRknkrLUXHOD4EzjLE8gRZIzh7bff5txzz/V3KCIiIs1eQJsAku9OZvvb24kZG+PvcEREDkpTzlp5VDPG1LtNnDjR3yGKiIgcdeKviueY/xyDK0i/EolIy9JYC4LLAeTm7u9/P3fuXK666qpq+0JDQ/0RloiIyFFny3NbaHNCG8LTwgGNixORlkl/fmoiHTt29G3R0dHV9hUWFnLppZfSsWNHwsPDGThwIHPnzq12fnJyMvfddx+TJ08mKiqKxMREHnrooRr1/PLLL0yYMIHw8HC6devGK6+80iT3JyIi0hLk/zuf7EnZLD5xMWXby/wdjojIIWs1iZwx/tkaw969exk9ejQff/wxS5cuZfz48ZxzzjmsXLmyWrlHHnmEvn37snjxYm699VZuueUWFixYUK3MPffcw7hx41i6dCnnnXcel19+ORs3bmycQEVERFqwwhWF/HTBT2Ch882dCeoQ5O+QREQOWatJ5Fqyfv36MWXKFPr27UtKSgp33HEHAwcO5J133qlWbuTIkVxzzTWkpKTwxz/+kZSUFD799NNqZS655BIuvvhiUlJSuPfeewkICODLL79sytsRERFpdsp/KSdrbBaeAg8dJnQg6Y4kf4ckInJYWs0YuZa8mEFhYSF33303c+fOJTc3l/LyckpKSjjmmGOqlfv16/j4ePLy8uosExAQQIcOHWqUEREROZp4K7z8dN5PFK8uJmJABL1e7KVxcSLS4rWaRK4lu+mmm5g3bx4zZsygR48ehIWFcemll1JWVr3vfmBgYLXXxhi8Xu9BlxERETmarLlxDTs/2UlgbCDp/0rHHe72d0giIodNiVwz8PXXX3PppZcyfvx4AEpKSlizZg2pqal+jkxERKRl2/LcFnIey8EEGtJnpxPSJcTfIYmINAqNkWsGUlNTee+991i8eDHLli3j4osvpqSkxN9hiYiItGi7vt7Fz1N/BiD16VTaDG3j54hERBqPErlm4OGHHyY2NpaTTz6Z0aNHM2TIEE4++WR/hyUiItJilWwsYfk5y7HllsTrEul0eSd/hyQi0qjUtdIPzj33XGyV2VmSkpL45JNPqpW56aabqr1ev359jetkZmZWe21rmfFl33kFBQWHFqyIiEgL4yn0sGzsMsq3l9N2RFu6PdTN3yGJiDQ6tciJiIhIq1L+Szm23BKaEkqfN/vgCtCvOyLS+qhFTkRERFqVkM4hDFwwkPId5QS2DTzwCSIiLZD+RCUiIiKtQtHqIt8wg4CoAEK7hfo5IhGRI0eJnIiIiLR4e5fuZWG/hay6fBXecq2fKiKtnxI5ERERafFKNpaABeuxmADj73BERI44jZETERGRFi9mTAwDvxtIaEooxiiRE5HWTy1yIiIi0iJZayleV+x7HZEegTvE7ceIRESajhI5ERERaZFynszhu97fse21bf4ORUSkyalrpYiIiLQ4Oz/dyerrVoMHUE9KETkKqUVOREREWpSi1UUsn7AcPNDlz12IuyDO3yGJiDQ5JXJNbOLEiRhjMMYQEBBAly5duPrqq9m5c2eDr5GcnMyMGTNqPWaM4Z133qmxf8qUKZx11lmHHLeIiEhzULGngqyxWVTsrKD9mPZ0va+rv0MSEfELJXJ+cNppp5Gbm8v69et57rnnmDNnDlOnTvV3WCIiIs2a9Vh+uvAnilYUEdYnjN6v9Ma41K9SRI5OSuT8IDg4mI4dO5KYmMjIkSM577zz+O9//+s7/uKLL9KnTx9CQkJITU3lkUcewevV4qYiInJ0W3fnOn759y8EtAug7wd9CYjSUH8ROXq1qm/ATJN5UOUjBkYweNHgGudn2AzfvoWDFrJ38d5az69a7lCtXbuWefPmERgYCMCzzz7LtGnTePzxxxk0aBBZWVlcddVVBAYGcs011xx2fSIiIi3Rtte2sfGBjeCGtLfTCO0e6u+QRET8qlUlci3FvHnziIiIwOPxUFJSAsDDDz8MwL333suDDz7IueeeC0DXrl257bbb+Mc//qFETkREjkp7vt/DqitWAdDj0R60Hd7WzxGJiPhfq0rkDreFrLbzq7bYNZbf/OY3zJw5k+LiYp599lnWrFnDn/70J7Zv386mTZuYPHkyV199ta98RUUF1tpGj0NERKS5K80tJeu3WXhLvHSa1In4qfH+DklEpFloVYlcSxEWFkZKSgoAjz32GMOGDePee+/1JW9PP/00J5544iFdOzIykt27d9fYv3v3btq0aXPoQYuIiPjB+rvXU7aljDa/aUOPx3tgjCY3EREBTXbSLEyfPp2//e1veDwe4uPjWbNmDSkpKTW2hujZsyeLFi2qts/j8ZCVlUXPnj2PRPgiIiJHTMojKSRen0jaO2m4gvRri4jIPmqRawYyMjLo06cP9913H3fffTd//OMfiY6O5owzzqC8vJzFixeTk5PDn//8Z985W7ZsYcmSJdWuk5iYyA033MBll11GWloaI0aMoKioiMcff5ydO3cyadKkpr41ERGRw+IOdZPycMP+mCkicjTRn7aaiRtvvJHnn3+eESNG8MILL/DPf/6Tfv36cfLJJzNz5ky6dq2+4OkjjzzCgAEDqm1vvPEGF1xwAS+++CIvvvgigwcPZtSoUWzdupV58+bRsWNHP92diIhIw+V/lM/yCcup2Fvh71BERJottcg1sVmzZtW6/8ILL+TCCy8EICkpiQsuuKDOa6xfv77eOi644IIa5xcUFBxUnCIiIv7gLfOSfXU2pRtKiR4WTcLUBNu0q6gAACAASURBVH+HJCLSLCmRExERkWbDFeSi33/6seXZLcRP0QyVIiJ1USInIiIizUpYzzBSZmhcnIhIfTRGTkRERPxuzc1ryHkyR+umiog0kFrkRERExK9yX8hl04xNmEBD25FtCesR5u+QRESavRbbIqe/2LUM+ncSEZH67P5mN9lTsgFIfSpVSZyISAO1yEQuMDCQ4uJif4chDVBcXExgYKC/wxARkWaoZGMJWedkYcstCX9KoNMVnfwdkohIi9EiE7nY2FhycnIoKipSi08zZa2lqKiInJwcYmNj/R2OiIg0M55CD1njsijPK6ftaW3p/vfu/g5JRKRFaZFj5KKiogDYsmUL5eXlfo6mZSgpKSEkJKRJ6wwMDCQuLs737yUiIgLOH/tWXraSvUv2EpoSSp83++AKaJF/WxYR8ZsWmciBk8wpQWi4zMxMBgwY4O8wRERE2PCXDWx/ezvuSDfpH6QT2E5d8EVEDpb+/CUiIiJNZvt721n//9aDgT6v9yG8d7i/QxIRaZEaJZEzxrxgjMkzxmTVcdwYYx4zxqw2xvxojBnYGPWKiIhIy7H3x72suGQFAN0e6Eb7M9v7OSIRkZarsVrkZgGj6jk+GuhRuU0CnmqkekVERKQFKM8vJ2tcFt5CL3EXx9H55s7+DklEpEVrlDFy1tovjTHJ9RQZB7xsnSkmvzXGRBtjOllrcxujfhEREWne3FFu2p/Vnj3f7SH12VSMMf4OSaRBrAWPByoqnEePB7xeZ39tj/Uda+jjoZ67b9sX9+E8Hs65y5fHsmVL48VyMDHXpSWWOZCmmuwkAdhU5fXmyn1K5EREpHl77TX4wx/8HUWL58LpluOxQbg7lfk7nBZvaEUFBDTfOeushXICKSaUEhtMsQ2hhJAaj2U2kDKCKCeQMhvoPBJEuQ2o3B9Ama08TiDlvvK17XfOr7ABeHDjwYUHNxUE4LH7X+/bnHLV93msm4qqr3FjNaXEIejj7wCOCs3qG8AYMwmn6yVxcXFkZmb6N6BWZO/evXo/pdnS51Oaq71797I2M5Og4cNZP3Giv8Npkbz/c2P6eDCR/o6kdSksLCQ8/PAmirEWyspcFBQGUlgUSHGxm+KSAIpK3BQXB1BcEuDbV1y5r6jK89JSN2XlLkrL3JSV7Xt0U1rmoqzcjdfbelpd3S4vLpfF5QKX2+IyFmMsxoDLAMbiMjj7XFQex1dmf1kLlef8+pjZdw3Xr8vuO16z7P569z8H57jzpNqDb/++BnGPrcBjvVjjxVqLNV681mLxEmDchLvDMAYqrIdfKn7B4hzzVp6zr2zHoA6EBjjLXO0o38H20nxwWee4qSyPlyBXIH0jevti+W7PYspsua/ufde31pISlkRySBeMgdyybSwu+BGwsO/e2HePllHthhPsdma/nb/7e7aX5df679gpOI7jopxZ3Is8RXy884taSjnXPbHNscQGdQBgeeFKfi5au7+I2d+cFuYK5fT2w3yv5+74mHJbvj++KvqE96RXWAoAm0tz+a5gUa1xApzVfiRBriBm/7vOIk2WyOUAVTvDJ1buq8ZaOxOYCTB48GCbkZHRJMEdDTIzM9H7Kc2VPp/SXGVmZtKtc2eIiyNxzBh/h9Pi7MzcydIHlhLaNZRBiwcRENms/n7comVmZnJS5femxwP5+ZCX52w7dsDOnbBrl7NVff7r12VHsHE0IABCQiA01Nn2Pa/6GBTkbIGBDXs8UJnAQKdet3v/9uvXB7vf5QJjjkyrnMfroai8iKLyIgrLC53HMufRGENGcoav7HOLn2N3yW5KKkooqSihuKLY93xcz3GM6el8R32x/gtu/+x2p0z5/jL7tk3Xb6J9mDPR0Mh/juTjtR/XGtvZvc5m9nmzAdiwawPJj/ar8z7+cdFHjEpxpsu449N/cP/X99darnNUZ/5z/Ubf6w4PDaSwaEetZS8Ydi93/uZOAN5bsYhz3roSgEBXIAGuAALdzmOAK4B/Th1PTFgMAFd98Bfmb57vK1d1OyXpFO4edjYA2wu3M+Xfn+A2blzGhdvl3v/cuLn+hN+QHpsMwNzsLD5d+zNu1/7j+563DWnL9Sd09cX95HfllHpKa73usQkd6N/RKbtpdwDzN22q9Zpu42ZY164EuYOorxd6U32jfgBcY4x5Azge2K3xcSIi0iKUlzu/HcpBC+0eSsQxEbQd2VZJ3EEqK4MtW2DzZti2bX+Stn2785id3Z/ycud5fv6hj7cJCoK2bSEycv8WEXHgLTwcwsLqT9Kacc/PQ7K7ZDc7S3ZSUFrAntI9FJQVUFBaQEFZAXHhcZyZeiYAO4t38udP/1wtMav6/OmznvYlaNMzp/OXr/5Sa33xkfHk3LC/3ePOz+5kW+G2WssmRiX6Erk9pXuYv2l+nfdR6in1PY+LiCM+Mp4gd1CNrUe7Hr5ykcGR/C7td84xV/VywQHBdI3en8iM6zWO4m3F9O3dt8Y1I4IiqsXy+e8/B3ASs18lXpHBkdWu6ZnmwdWAhPrZsc8esAxAh/AOvPu7dxtU9qzUszgr9awGlf3DcQ3rit+5TWfOa3Neg8rWpVH+ixljXgcygBhjzGZgOhAIYK19GvgQOANYDRQBlzVGvSIiIkdcWZnzm6kctJDOIQz4egCuYI0xqsrjgZwcWLsWNmxwkrXNm519+x7z8g50lWjfM2MgJgY6dIDYWOexbVuIjt7/WNfW2j/axeXF7CrZVW3r0qYLabFpAKzcsZKZi2b6ErKCssokrfL1N5d/Q8eIjgBc/N7FzM2eW2s9o1JG+RK5cm85zyx6ps6Y8ov2d/uLCIogLDCM8MBwwoPCqz2PDY+tdt6VA6+kuLyYkIAQ3xYaGEpIQAgDO+1f2evEzify9WVfVytXdQsLDPOV/efZ/2zQ+9gutB1vnvtmg8oel3AcRfFFZAzIOGDZ9Nj0Bl2zIQnc0aixZq284ADHLaCR4iIi0vKUlzvNFtIg1mvJeyuP2PNiMcbgDnP7OyS/KCyEVatg9WonYVu3ztnWroWNG52PVX3cboiPh4QE6NRpf5K2L1HbsmUJI0f2p0MHaN++9bV+1SZnTw4/bf+JXSW72Fmy05eY7Szeicd6mDlmpq/s4JmDycrLqtb6tM/NJ97MgyMe9F3zkW8fqbPO3SW7fYlcfEQ8Xdp0ITIokqjgKCKDI4kMiiQyOJL+cf1950SHRPPUmU8RHliZmFVJ0MICw+jcZv9oo9tOuo3bTrqtQfd/3/D7GlSufVh7hnYZ2qCy0rIdBf/tRUREDoO6Vh6Udf9vHRvv38iuzF30fLqnv8M5oqx1uj2uXAkrVjiP+7aNG+s/t2NH6NYNkpOhc2dITHSStsREZ4uNdZK5umRm7iK9YY0ZzYa1lt2luykuL6ZTZCcAyj3lPPHdE+QX55NflM+O4h3OY9EO8ovz+b/T/48JaRMAeGv5W9zw3xtqvXagK5BnznrGt6xFudcZpxTkDqJtSFuiQ6J9W2r7VN95PWN6MmPEjGpJWVRwlO95lzZdfGWfGVN3K1tVQe4gpgyeckjvkcjBUCInIiJSn7IyJXINtO31bWy8fyO4IXZC7IFPaEHKy51k7YcfYMkSZ1u61Jk4pDaBgdCjB6SmOglb167Oti95ay1dGj1eD/nF+Wzbu41ubbsRHuTMpPlG1ht8tPojtu3dxrbCbWzbu43tRdup8FYwOH4w31/1PeB0mbvp45vwWm+t19+6d6vveWr7VIZ3He5LzKomaG1D22KxmMr5GT+79DPCAsMICQipd83CxKhEbjzxxsZ6O0SalBI5ERGR+qhFrkH2LNzDqstXAZDySAptT23r54gOXVmZk7B9//3+xC0rq/YZHqOjoXdv6NVr/2OvXk7S1pK7OxaXF5NT4Ey0kdLOmS59S8EWbv3k1hrJ2b4k7MuJX3Jy0skAfJ/zPS8vfbnGdSODIgkN2J/Ful1ubjnxFsICw2gf1p72oe2JCYvxPa86TuzM1DN949AOZN+sjCKtWQv+ihEREWkCGiN3QKW5pWT9NgtviZdOV3Yi4ZoEf4d0ULZvhwULYP58Z/v+eygpqVkuJQX694cBA5zH/v2d8Wv1TQ/e3FhryS/Op21IW9wup+/miz+8yDebviGnIIecPTls3rOZnSVOU2PVKeittbzy4ys1rtkutB1x4XHVWtUmpE0gLTaNuPA44iLiiAuPIzY8luCA4Brn//W0vx6JWxVp9ZTIiYiI1EctcvXylHjIOjuLspwy2pzUhh5P9qi3K1tzsGEDfPIJfP21k7hlZ9cs06sXDBkCAwc6CVu/fhAV1fSxHqrcglzeX/U+G3ZtYMPuDWzes9mXqJV6Sln7p7V0betMGf+fNf/hzeXVZyQMdAUSHxlP+9D9LVux4bG8OO7FGslZoLvm/48hiUMYkjjkyN6kyFFOiZyIiEh9NEauTtZasidnU/C/AoK7BJP2bhquoOY3TfiePZCZCR9/DP/9b83ELTQUjjsOhg6FE090Erj2zbRn3p7SPazbuY5vdnzDsv8tY/2u9WzY7SRro1NGc8+wewDYuHsjV//76lqv0Sa4Db8U/+JL5H7f7/dkJGeQEJlAQlQCiVGJxITF1JjyPdAdyMT+E4/o/YlIwymRExERqY+6VtZp88Ob2fbyNlxhLtLfTycotnm8T16v0z3yv/91krcFC6CiYv/xqCgYPhwyMpzkrV+/5pOrW2vZuncra3auYc0va1izcw3TTplGgMv5le30V07n283fOoWXVz83IXJ/l9bu7bpzef/LSY5OJik6ic5RnUmISiAhMsE3Ick+o3uMPqL3JCJHhhI5ERGR+qhrZa3yP8pnzS1rAOj9cm8i+0f6NZ6KCvjiC3j3XXjvPdi6f7JD3G6npW3ECBg50ml98+dEJNZaSipKCA10Jv1YunUp0zOns2bnGtbuXEtReVG18hP7T6Rb224A9I3ty57SPUR6IhnQdYAvUUtqk+QrAxATFsPz455vupsSkSanRE5ERKQ+6lpZQ+HKQn46/yfwQtL0JDqM7+CXOEpLnbFus2fD++9Dfv7+Y0lJMHq0k7gNG+bMLtnUCssKyc7PZuWOlazKX+VsO1aRnZ/NFQOu4NHRjwLOmmfvr3rfd1670HZ0b9ud7u26071td0ICQnzH9i16nZmZSUZGRpPej4g0L0rkRERE6qMWuRp2frITzx4PMeNjSJ6W3KR1l5XBRx/BW2/B3LnO+Ld9evaE8eOdbcCApplN0lpLXmEey7cvZ3necqYMnuKb/GPkKyOZv2l+redt2bvF97x3TG/ePPdNX/IWHeKHrFNEWhwlciIiIvXRGLkaEq9JJCQ5hOiMaIzryGdL1sLChfDyy/D669Vb3vr1cxK3c86BPn2OfPK2be82Zq+YTVZelpO8bV/OjqIdvuOnp5xOavtUAI6JPYZdJbvo2b6ns8Xsf2wX2s53TnhQOL9L+92RDVxEWh0lciIiIvVRi5xPRUEFAZHOrw4xZ8Uc8fry853k7dlnYcWK/fvT0+Hii50ELiWl8estKi9i2bZl/LjtR5ZvX05q+1SmHjsVcBbFnvrh1GrlI4MiSY9NJ61Dmm9SEoCnznqq8YMTEamkRE5ERKQ+GiMHQO6Luay7cx3p/0on6tgjt6CatfDVVzBzJrzzjjMODqBDB7joIrj0Umddt8Zuefvn0n/y75//zZKtS8jOz8ZifceGdx3uS+R6xfRiYv+JpHdIJy02jbQOaSRGJTb7tfNEpPVRIiciIlIftcg548Bey6NsSxl7l+49IolccTG89ho89hj8+KOzzxgYNQomTYKzzjq8f4YKbwXZ+dks2bqEpVuXsmTbEp468ynfTI9fbfzKtyi227jp06EPx8QdQ3psOoPjB/uuExoYyovjXjz0QEREGokSORERkfpojBzGGPr+uy873ttB7HmxjXrt3Fx44gl45pn9Y9/i4uCqq+CKKyA5+dCvnV+Uz91f3M3CLQv5YesPlFSUVDu+OHexL5G75JhLOCHxBPp17EefDn2qzRQpItIcKZETERGpz1HctdJT4sEEGFwBLlxBrkZN4lavhoceglmznLcYYNAguPZa+N3vIDi4Ydex1rJm5xoWblnI9znf4zIuHhr5EABhgWE8tfApKrzOauDJ0cn0i+tH/4796d+xPyd1Ocl3nZOTTubkpJMb7f5ERI40JXIiIiL1OUq7VlprWTlxJRW7KujzRh8CoxvnPVi6FO6/3xn/5vU63SfPPhtuvNFZtLshQ82+z/me2StmszB3IQu3LGRXyS7fsXah7XhwxIMYYwgNDOUfZ/yD5OhkBsUPqjZTpIhIS6dETkREpD5HadfKjfdvZPub23FHuinLLTvsRC4rC+66C95913kdGAgTJ8LNN0OvXrWfU+YpY8nWJSzYtICTupzEoPhBAHy7+Vse+OYBX7m48DiOTTiWY+OPZXD8YLzWi9u4Abhq0FWHFbeISHOlRE5ERKQ+R2GL3I73d7DuznVgoPdrvQnvHX7I11q50kng3nrLmZEyJAQmT4abboLExOpltxRsYcGmBSzY7GyLtiyi1ONMW/nnk/7sS+RO63Yad5x8hy9xi4+M16yRInLUUSInIiJSn6NsjNzeZXtZcbGzaFu3v3Y75PXicnNh+nR4/nmnC2VQkJPA3XYbxMeDx+thed5K0mLTfOec+vKprNyxstp1esX04oTEExjaeahvX+8Ovblv+H2HFJeISGuhRE5ERKQ+R1GLXNmOMrLGZeHZ6yH2wlg639L5oK9RUOBMYvL3v0NREbjdTgJ3822lbHUt5OW1X/FV5ld8s/EbdpfuZvP1m0mISgBgZLeRdI7qzAmJJzAkcQjHJx6vcW0iInVQIiciIlIXrxc8Hgho/T8uveVefprwEyXrSog8NpKez/U8qO6KXi+89JLT4paX5+w75xy4+tZN3PvTxaS/+l2N6f+T2iSxac8mXyL36OhHG+1+RERau9b/k0lEROQQmYoKpzXuKBh/tfra1ezK3EVQpyDS30vHHepu8LnffQdT/1DBooXOrxXxfTbw1swkhg6FvWVt+WbeN3ish7QOaZzcxZnm/+QuJ9O5zcG3+ImIiEOJnIiISB1cHs9R0a0y56kctjy1BRNsSH8vneCEAy/itrdsL3N/+B/3T49k2X+OAwIgMgdG3Ezp8R9z4ol5gCEiKIKPL/mYY+KOoX1Y+yN+LyIiRwslciIiInUwFRWtfumBnZk7Wf2n1QD0fLYnUcdH1VquqLyIMk8Z0SHRWAtX3vMlbz58HBTHgKsM14mPctwFHzOi9xBOSboCi8XgtGQO6zqsye5HRORooURORESkDr6ula2Up9DDT+f9hK2wdL65Mx0v6eg7Vu4p59vN3/Lpuk/5fP3nfLv5W24/6XYuSpzOlCnw6adnAJA0YDXTHtzKeb+ZSnjQzf66FRGRo44SORERkTq4Wnki5w530+ulXmx7eRvd/toNgJeXvsw7P71D5vpMCsoK9hf2uvnwpTQeeANKSqB9e3j4YbjkkhSMSfHTHYiIHL2UyImIiNShtbfI5Rfl82nnTxn5wkiM2+kG+cnaT5iTPQeA3jG9Oa3bafQyY3jp7mF8963za8MllzjLC3To4LfQRUSOekrkRERE6tDaxsiVecqYv2k+P//lZz6N+pS3ot7CYnlnwjuM7zMegKsGXsWw5GGM6D6C+IhEnnwSbroViouhUyd47jk44ww/34iIiCiRExERqUtr6VpZ4a3g7DfP5vN1n9MjuwePvPQISe4kvrjhC/qk9yE8KNxX9uQkZ3mALVtg5Dnw6afO/osugsceg3Zan1tEpFlQIiciIlKHlti1ck/pHj5Z+wkLNi3gwREPYowhwBXAloItFJYX4jnWw8/FP5PcI5k109cQFhhW4xpz5sBll0F+PsTEwDPPOIt7i4hI86FETkREpA4toWultZbl25fz4c8f8tHqj/h649dUeCsAuHzA5fTu0BuAZ8c8S8eIjsRHxsMfar9WcTHcfDM8+aTzeuRIeOkl6Nix9vIiIuI/SuRERETq0Nxb5LLzsznt5dPYtGeTb5/buDmpy0mckXIG0SHRAHhLvUTeH0nMnTEQWce1suHcc2HZMueW//pXuP56cLma4k5ERORgKZETERGpQ3MZI2etZeWOlXz484fsKtnFvcPvBSA5OpldJbuIC49jVMoozuhxBiO6jaBtaNtq52ZPyWbrrK0U/FDAgK8GYIypdv1333W6UhYUQI8e8PrrMGhQk96iiIgcJCVyIiIidfBni1xhWSGfr//c12Vy/a71AIQGhHLHb+4gJCCEIHcQS6YsITk6GZepvels8/9tZuusrbjCXPR4vEe1JK68HG67zVkPDpwWueefh6ioI313IiJyuJTIiYiI1MFfY+T+tfJfnP/O+ZR6Sn37YsJiGJ0ymtEpo6uV7da2W53X+eU/v7DmpjUA9JrVi8gB+/tVbt0KEybA119DQADMmAF/+hP8qrFORESaKSVyIiIidTjSXSu91svi3MXMWTWHuIg4ph47FYBj4o6hzFPGcQnHMTplNGf0OIPB8YPrbHWrTVF2EcvPWw5eSJqWROyEWN+xhQvht7+FnBxISIC33oITT2z02xMRkSNIiZyIiEgdjkTXyuLyYj5d9ylzVs1hTvYccvfmAtCnQx9fItetbTfybs4jJizmkOoo31XOsrHL8Oz2EHN2DMnTk33HXnsNrrgCSkrgpJPgnXcgLu6wb0tERJqYEjkREZE6NHbXyie/e5KbP76Z4opi377EqETGpI5hTOoYrLW+MWyHmsRZj2XFBSsoXlVMeN9wer3cC+MyeDxwxx3wt7855a66Cp54otmvriAiInVQIiciIlIHl8dzSC1y1lqWblvKnFVzODbhWEaljAIgKTqJ4opiBscPZkzqGMb2HEu/uH41ZpE8HGtuXcMv834hMCaQ9A/SCYgIoLAQLroI3n8f3G549FGYOlXj4UREWjIlciIiInUw5eUNTuRKK0r5fP3nvi6T+9Z2m9Bngi+RG9FtBDk35DiLch8BW1/ayua/b8YEGNLeTSM0OZStW2HMGGdcXHS0s9TA8OFHpHoREWlCSuRERETq0NAxctM+n8bDCx6msLzQt69jREfO6nEW5/Y517cvOCD4iCVxu7/dzapJqwDo8WQPon8TzfLlcMYZsHEjdOsG//439Op1RKoXEZEm1iiJnDFmFPAo4Aaes9Y+8KvjE4GHgJzKXU9Ya59rjLpFRESOFJfHU20QmbWW5duXM2fVHMb1GkefDn0ACA8Mp7C8kH5x/RjbcyxjUscwKH7QQc0yedi8EBAdQIcJHYifFM+nn8I558CePTBkCHzwAXTo0HThiIjIkXXYiZwxxg08CYwANgPfG2M+sNb+9Kuib1prrznc+kRERJqKKS+nLDSIL9d+4usyuW7XOgDKPGVMz5gOwOUDLueCvhfQpU0Xv8Xa5sQ2DF48mMDYQN56Cy6+2Fnw+9xz4eWXITTUb6GJiMgR0BgtcscBq621awGMMW8A44BfJ3IiIiItyrSoL5kbvo49/3zEt69DWAfOSj2LU5JP2b8v3D9NXdZaCr4vIOq4KACCE4J54glnYW9r4dpr4eGHwdWEDYMiItI0GiORSwA2VXm9GTi+lnLjjTG/AbKB6621m2opIyIi4herdqxiTvYcJg2aRFSwkxgV2zL2mFLSOqT5ukwel3Acbpfbz9E6Nj6wkXW3r6P7jO4k3tCZadPgvvucY3/9K9x6q2amFBFprZpqspM5wOvW2lJjzGTgJaDGnFnGmEnAJIC4uDgyMzObKLzWb+/evXo/pdnS51P8wWM9LNu9jPn581mQv4DNxZsB2Juzl4wOGQBcvTWVPxUMglOchbpL15Ty1Zqv/BVyTesBF2QXreEP49zMmROPy2W54YZVDBmylS++8HeAcqToe1OaM30+m0ZjJHI5QOcqrxPZP6kJANba/CovnwMerO1C1tqZwEyAwYMH24yMjEYITwAyMzPR+ynNlT6f0pQqvBVM/NdEPvz5Q3aW7PTtbxfajjN6nMGoY0cxJHEIABuffpou3QdCc/18ZsDeG4uZck8oc+ZAcDC8+aZh3LhegKanbM30vSnNmT6fTaMxErnvgR7GmK44Cdz5wIVVCxhjOllrcytfjgVWNEK9IiIiB/Rz/s98tu4zJg2ahDGGAFcAK3esZGfJTnq27+lbmPuEzicQ4Kr+Y9HVwOUHmlJ5fjmevR5CkkIoLYVLbwvlvfcgPBzmzIFhw/wdoYiINIXDTuSstRXGmGuA/+AsP/CCtXa5MeYeYKG19gPgT8aYsUAF8Asw8XDrFRERqU25p5xvNn3D3Oy5zMmeQ3Z+NgAnJ53sWy7g8dGPExMWQ4/2Peq9lqmoqLb8gL95y70sn7CcwqxCUt/py6V/jWLePGjTBubNc5YZEBGRo0OjjJGz1n4IfPirfdOqPP8z8OfGqEtERKQ2eYV5XDfvOj5a/RG7Snb59keHRDM6ZXS1sid0PqFB12zoguBNZfX1q9n1+S4CYgO54tYg5n0LMTHw3//CgAH+jk5ERJpSU012IiIi0qhW7VjF8u3LOaf3OfD/27vz+Kjqe//jr28m+0JICAQIe0BAFkGDgguCyloVrdqKW3+oRb16675gtVVbta1ttVarUK/eq1QBa1VUqIAaRIEKIir7vgUIBgjZk8nM9/fHSYYACVuWOZN5Px+PecyZs35m5pDkzfd7zhcnsH20/iMKygsCXSYvPuVizul0zhFdJo+Xm7pW7py8k50v7sREG17O6MsHi2Np1w7mzYNTTw12dSIi0tQU5EREJCR4fV6+2PYFH6z7gA/Xfcj6feuJi4xj7wN7iYuKI9oTzdTLp9Irrdcxu0weL7e0yOXPz2f9HesB+FfXnvzjm2Tat4fsbOjRMG9VRERCjIKciIi42pq8NTyW/Rj/3vBvDpQfCMxPiU1hbI+x5JflvWXpWQAAIABJREFUExcVB8AlPS9p0GO74Rq50s2lrLhiBbbSsrBDB55f21YhTkREFORERMQ9rLUs372c/WX7uaCrM9xotCea6SunA9A7rXegy2Rtd5lsaMHuWllZVMmKcSuo3FvJ+pRUHt2RqRAnIiKAgpyIiATZgbIDzN00l9nrZzN7w2x2Fe2iT+s+rPivFQB0S+nGq5e+ytDOQ8lMzWzS2oLZtdL6LWuuX0Px98XsjY/j7v29adveKMSJiAigICciIkEyd+NcnlzwJF9u/5JKf2VgfkZSBkM6DMHr8xLlcULUhIETglKj8fmC1rVyy2NbyHsvj/IoD3eV9COhTRSffaYQJyIiDgU5ERFpdIXlhXyy+RPaJLTh7I5nA1DuK2f+1vl4jIehnYcypvsYxvYYS782/TDGBLliR4TXG5QWuT0z9rD1N1vxG3jU24eilvHMnwunnNLkpYiIiEspyImISIOz1rI6bzWz189m1oZZLNi6AK/fy0/6/CQQ5IZ3Gc6MK2cwInMELWNbBrni2gWja6W/0s+Wx7YA8JLNZFVCKvNmQ//+TVqGiIi4nIKciIg0qD8v+jPPLn6WHQU7AvMiTARndzybszucHZiXEJ3AVX2uCkaJx834fE0e5CIiI5g1YgCrV+/mg+gOzJoJgwc3aQkiIhICFOREROSklFeW8+X2L5mzcQ7j+47ntLanAc54bzsKdtA6vjWju49mbI+xjMwcSWpcapArPnFNOfyA9VtMhOHpp+Hx56OJjOzEv/4JF1zQJIcXEZEQoyAnIiLHxVrLyh9WMnfjXOZsmsP8LfMprSwFID4qPhDkrj/tekZ1H0X/9P5EmIhgllxvTXWNnLWWtRPXsmJLFI980g1jDK+/Dpc07LB4IiLSjCjIiYjIcRn090F8vevrQ+b1T+/PiG4jGJU5KjCvfVJ72ie1b+ryGkVTda0sXlnMrv/NJdFnyKAtj7ycwPjxjX5YEREJYQpyIiISUFheyIJtC/hs82d8svkTPrnhE1LiUgDo3bo3OYU5jOg2gpGZI7mo20W0TWwb5IobV1N1rZy3IZEn/P1JpJI7/5jAxImNfkgREQlxCnIiImGswlfB/C3z+WzLZ3y25TOW5CzBZ32B5Z9u/pQrTr0CgL+N/RuJ0YmuGRqgKUQ08l0rrbUsXGi4+mootyn86ldw772NdjgREWlGFORERMJIeWU5W/K30DOtJwAF5QWMnDoysNxjPJyVcRYXdL2AC7teyDmdzgksS4pJavJ6g60xhx+oPFDJf4Z+y683daG8vBW33gqPPdYohxIRkWZIQU5EpBkrryxnyc4lgVa3L7d/SVp8Gtvu2oYxhrT4NK7pdw3tE9szvOtwzu10Li1iWgS7bNdorCBnfZZvfrwK73eFXM0mEi9J5YUXDGHU2CkiIvWkICci0gxlb8nmV5/9iq9yvqLcV37IspTYFPLL8gPXvv3jx/8IRokhIaKRrpFbc+8mij/dxwEieee0vrwzzeDxNPhhRESkGVOQExEJYXuK97Bg6wIWbFvA6e1O54bTbgDAYFiwbQEAfdv05bxO5zG8y3CGdRlG64TWwSw5pDRGi1zOa7vJ/ct2KjFMad+H1+fGER/foIcQEZEwoCAnIhJCNu3fxBfbvmDB1gV8vu1z1u1dF1g2pvuYQJA7q8NZzLx6Jud0OickB+J2BWsb/GYnBxYXsPrmtUQCryV15/nPU2itXC0iIidBQU5ExKWKKopYkrOErPZZgRuNPPzJw0xfOT2wTnxUPIM7DOa8TudxYdcLA/NjI2O5pKdGk64Xnw8bEYGJaJhBzctzyll80Qpi/JaPIttz1ycZZGY2yK5FRCQMKciJiLiAtZYN+zawaMciFm1fxKIdi/h+z/f4rZ+PrvmIsT3GAjAqcxRllWWc2+lczut0Hqe3O50oT+MPWB2WKirwR0bSEJeu+Up9fHrOCuKKK1hOS4a/051BgxpgxyIiErYU5EREgqC4opiE6ITAdJe/dCGvJO+QdTzGwxntzsBv/YF5EwZOYMLACU1aa9jyerGR9f81aa3l09FridtayE5iSX72VMZe2jCtfCIiEr4U5EREGll+WT7Ldi1jSc4Slu5aytKdS/H5fWy7exsACdEJpMal4jEehnQcwpAOQxjcYTBZ7bOIj9JdMIKmgYLcov/eRtTneyjBQ85tfbn/roa/C6aIiIQfBTkRkQZkrcVUDQY2c+1M7ptzH+v3rT9ivfioePLL8mkZ2xKARTctIiU2JbCtuIDXi7+eQW7dP/ZR9uJmIoDFF/Tm8RcTG6Y2EREJewpyIiInaXfRbpbvXs7y3cv5Nvdblu9ezoQBE3jgnAcAJ6yt37eeaE80A9oOIKtdFoMyBpHVPoteab2IjDj4I1h3lnShiop6tcgVFcHPft+CcaRS0jWZR2anacBvERFpMApyIiLH4PP78EQcvOXFzTNv5sN1H5JbnHvEul/lfBWYHtJhCF9P/Jq+bfoS7VF3upBTj66VPh9ccw0s/j6Sfd37sXBxo4wrLiIiYUxBTkSkirWW3OJcVuxZwco9K1mxZwXf5n7Lij0r2H73dlrFtwJgf9l+cotzaRHTggFtB3Ba+mkMaDuAAW0HcGrrUwP7S4hO4PR2pwfr7Uh9eb1Yz4nfs9Lv9fPyiBz+PT+DlJQIPpxlaNWqEeoTEZGwpiAnImEprySPEm8JnZI7AfD1zq8ZOXUk+0r31br+qh9WcV7n8wB46oKn+OOIP9KlZRdd09acVVTgP4nBwN8fs4k+83fwqDnAef/qS48ejVCbiIiEPQU5EWnWdhftZtUPq1ibt5bVeatZ+YPT0raneA9XnXoVM66aAUBGiwz2le4jOSaZvm360qd1H/q06cNp6afRP70/KXEpgX32TOsZrLcjTekkWuTmzYNfftaGX5JH5qSODBvWOKWJiIgoyIlIyCvxlrBu7zrW5q1l7d61/Neg/yItPg2AX8z+BW+vevuIbRKjEw+57i09IZ0dd++gfVJ7tbKJ4wSvkVu9Gq68Eg74W7DivjN5+kmNFSciIo1HQU5EQoLf+okwzh/GuUW5/Pbz37J271rW5K1he8H2Q9Y9r9N5DO86HICs9lnsKNhBz7Se9E7rTZ/Wfejbpi8dkzsG9gdgjCGjRUbTvSFxP6/3uLtW7lxWymNjCjlwoA0//jE8+XuFOBERaVwKciLiGvll+WzYt4GN+zaycf/Gg8/7N5LVPot3f/ouABEmgheWvBDYLjIiku6p3enZqic9W/WkXVK7wLIHznkgMByAyAmpqDiurpUleyv5/LwV3FZSTMsufp59oy0RynEiItLIFOREpMnkl+WzNX8rW/K3sPXAVrbmb+WeIfcElt8+63be/P7NWrdNiT14jVpafBrPjnqWzJRMeqX1omtK10PGZBNpEMfRtdLvs7x9+ho6lxSzKzKOR2a1Ij6+ieoTEZGwpr98RKRBVPor2V20m5yCHKI90QxsNxCATfs3cfn0y9mSv4WC8oIjthvTYwyRVT+K+rXpR782/chMzSQzpepRNV19d0lwukHeNfiupnljEr68XvzHCHLTRm+h87Y8ivDQY0Y/OvY+8btcioiInAwFORE5pqKKInIKcuiW0o0oj/OH6gtfvcC8TfPIKcwhpyCH3OJc/NYPwNgeY/nomo8ASI5J5rvc7wCIj4qnc3JnurTsQufkznRu2ZmuLbuyfZtzjdtD5z7EQ+c+FIR3KFKLioqjtsh9PGkP7edtxQf4H+lD1uVqihMRkaajICcSpqy1HCg/gM/vCwx0vf3AdiZ/PTkQznYU7CCnMCfQkrb69tX0SusFwJKdS3h/7fuB/RkMbRPbkpGUQfeU7oH5qXGpLP35Ujq37EyruFa13hFyO9uPmCcSdEfpWvndO4XwuzUAbBmVyU2/SW3KykRERBTkRJoTay37y/aTW5RLr7RegdD0wlcvsHz3cnKLc9ldtJvcolz2FO+h3FfOTQNv4pVLXwFgf9l+nlzw5BH7jfHE0KFFB4origPzbjnjFi455RIykjLIaJFBu8R2gda6mowxnNH+jEZ6xyKNqI6ulblrKlh/9Qpa4WdN53QmzuoQhOJERCTcKciJuJTf+jlQdoC8kjz2lu4lrySPzJRMerfuDcDiHYt5ZuEz7C3ZG1hnb8levH4vAAceOkCLmBYAvLfmPT7Z/MkRx0iMTsRwsIWsU3Infn3+r8lIyqBDiw5ktMggIymD1LjUI1rSzu54dmO9dRF3qKVFrrzYz5zBK+hYWc6W+BZct+wUIiI07qCIiDQ9BTmRJrC/dD/bC7azv3Q/+WX55Jfls7/MmbbW8vjwxwPrnvvquazdu5Z9pfsC15xVe3Toozwx/InAPv+1+l9HHKtFTAvaJraloLwgEORuH3Q7V516FemJ6bRNbEt6QjrpienERx16TU/L2JY8NuyxBn73IiHqsGvkrLW8eeY6uh4oYG9ENOfP70Ni6rGHJxAREWkMCnIitbDWUuItobCikMLyQgorCikoLyA2MpbBHQYDUFZZxlMLnqKgvIDC8kIOlB84JKQ9N+o5Lul5CQCTv57MpE8m1XqsxOjEQ4JcdesbOKEsLT6NVnGtSItPIzMlM7De6e1OZ8aVM2gV3yqwTqv4VsRGxh5xjMt7X95gn41I2Disa+Xb43Poumo35UTQ9pV+dM2KCWJxIiIS7hTkJKRZayn3lVPiLaG4oth59jrPafFpgRtz7CrcxbQV0wLLaoazwopCplw8ha4pXQG45YNbeOWbV45oDQMY3GEwi25aBDiDUv/m89/UWVtucW5gumOLjvRt05eWsS1pGduSlNiUQ56ttYGui7OumUVcVBypcalEe6Lr3H96YjpX9bnqxD80ETk+Xm9gQPDsbMu3MwoZARTc2pNRE5KCW5uIiIQ9BTlpMNZavH4vpd5SyirL8Fs/7ZLaBZbN3zqfssqyWh/DugxjQNsBAHy57Ute//Z1ynzOspohrbSylOW3LA+EnkF/H8TXu76utZ6fn/5zplwyBYCdhTu5Z849ta4H8EPJD4Eg54nw4Ld+YiNjaRHTgqToJJJikkiKTqJfm36BbaI90Twx7AkSoxNJikkiOSb5YFCLS6FdYrvAutf2v5Zr+197XJ9jdR0iEmQVFdioKDZvhiuvNOy1vUj4STsefqllsCsTERFpmCBnjBkN/AXwAK9Ya3932PIY4HXgDGAv8FNr7ZaGOHZz5rd+Kv2VgYfX56XSX0lidCIJ0QkA5Jfls6NgB16flwpfReDh9TuvLz7lYiJMBODc8GJHwY5D1qt+DGw7kPH9xgOw7cA2Hpj7QK3rVfgqeHXcq/RP7w/ApHmTmPz15EAgs9hA/f3a9OO725zxw4wxXPT6Rfisr9b3+tcxfw0EufX71jNl2ZQ6P5dyX3mg+2BsZCzRnmgSohKIj4onITohMN0tpVtgm/ZJ7bnzrDsDy6qDWfXzKa1OCaz751F/5i+j/1LrHRgP9+j5jx5zHREJUV4vP+zoyS9HFLJ3bxJjxhgefFMhTkRE3KHeQc4Y4wFeBEYAO4AlxpiZ1tpVNVa7Cdhvre1ujLka+D3w06Ptt8JXwZKcJfitH5/14fP78FlfoKWk5h3zPlj7ARW+isC6fuvH53ees9pn0adNHwDW5q1lzsY5gf3V3Lff+nnw3AcDXdkmL53Mxv0bDzlu9fSg9oO46fSbAMgpyOH+uffj9XsPCV3Vwev5Mc8HQs8fvvwDr3/7+sHlNbbJTMlk4U0LA+8p4akESrwltX42z416jjsH3xl47ze8d0Odn2PJwyXERcUB8KdFf+KLbV/Uut61/a4NBLniimKmr5xe5z4PlB0ITJf7ytlftj/wOioiitjIWGIjY0mJSzlku4u6XYTFBpbHRsYSFxlHbGRs4DMCGNJhCC/96KXAsvio+EBIi4+KJyriYMD6fMLngaB6NO2S2vHc6OeOuR5Q6zVmIhJ+cr5qTeGC3oxjHdt7nc5bbxk8ureJiIi4REO0yJ0JbLDWbgIwxkwDxgE1g9w44LGq6X8CLxhjjLXWUod9e/bx0jUv1bqsVVwr2g062G3twy8+xOv3MnvgbHan7HaKWn8mfbb3oXR8KX1udYLc8s+W880L39T5RjafszkQ5HZ/u5u8A3ks6b6EFZ1WAJC5O5Ohq4ayKWsTnO5sU7CrgNjnY4ml9j/+9329j80tNwPQcmNLzsw5k03pm5jfZz4ALYpbcMV/riCmVYwTd6uM/2w8/go/HuMhwkQc8sjYlsHmj5x9pu9N54HND1CWWMaikYuI9kQT7Ylm2AfDiPZF4yv3QVXuuXn9zVyz+RoiTASeCA8e4wk8t85tzeZvnH2aSsMH+z7AE+HBd7uP6Dhnn563PUTkRNCztGegzrvL7+bm0puJjIjEE+HUSkXVwgLY/OjmwLov4XyfbW9sS1xXJ1zunb2Xgi8LSE1Khc7Oehk5GYx6f1Sd39M2th0xL3V0KsnnJANQ9G0RP/zzBxL6J9DmqjYAVORVkPOXnDr3WZvato9sFUnHuzoG1tnymy3YijpP4yPUtX3nRzoTEeME0l3/s4uyLWUnVGtt2x/xOS8sOKF91rZ9bZ/ziajre6K1s1zfk7u/p3D79/Rybi+6E8+iuDa89x4kJ5/QbkRERBqVOUqWOr4dGHMlMNpae3PV6+uBs6y1d9RYZ0XVOjuqXm+sWifvsH1NBCYCZMRnnDG1ZOoJ1fLaL15jW89tRJgIRrwzgsGfDGbrDVvpPMFJCDs/3En7P7U/oX0uH7+c9WPXE0EEnRZ1YtDfBrHvvH2kPpEKQMmWEuInxB9jL4cqPr+Y/Afz8RgP0TnRpN6cij/DT8TUgy1LdqzFlJ7A2EQdgDdqvB4LlAIfAfFQVFRE4iOJ8O0JlRrYHoC7cLZ/FhhQNe9vwNsnuM/atr8N+EnVvHnAkWNSH11t218AVPd83A7U3XBZu9q2r+tzPl7H+J6Ag5/ziQjx76no7iISExP1Pbn8ewqnf0/zFrbhySdPJYpKnv7jCs44I/8EdyLSuIqKqn5uiriQzs+GM3z48K+ttVm1LXPVzU6stVOAKQADThlgu1zf5YS2f/n6l4nr4vzP9L7u+yg4v4CBFw0keUjV/yynFZGXnHe0XRzhkO0HFJHXNo8+ffrQepjThODd6yXniRP8n+k+CbQeU2P7nTlEpUaRMSwjsM62X2/DX3HkXRPrUtf2nS7sRERMBNnZ2fS6pxdlW0/sf6artwfYfc9uyraWkX5Z+sHP2buPgn4n1oJQ2/YpF6Uc+j3FnNj3VNv2CX0S6v89HbZ9Y39PcPBzPhGh/j2tTFzJsGHD9D25/HsKl39Pe5I78ac/Ods/NPRd7r1Xd4cV98nOzmbYsGHBLkOkVjo/m0ZDtMgNAR6z1o6qej0JwFr7dI11Pq5aZ5ExJhLYDbQ+WtfKrKwsu3Tp0nrVJgfpH5S4mc5PcYudO2HQIOd5Yu/PuWfku/R87tlglyVyBP3cFDfT+dlwjDF1tsgd+y4Rx7YE6GGM6WqMiQauBmYets5M4GdV01cCnx4txImIiDS1khIYN84JcUOHwl8HvYGNclXHFRERkYB6BzlrbSVwB/AxsBqYYa1daYx5whhzadVq/wO0MsZsAO4BHqrvcUVERBqKtXDjjbB0KXTrBu+8A9H+MmykgpyIiLhTg/yGstbOAmYdNu9XNabLAF1kICIirvSb38D06ZCUBDNnQloa4PUqyImIiGs1RNdKERGRkPX22/DrX4Mx8NZb0KdP1QKvF7+CnIiIuJSCnIiIhK1ly+BnVVdwP/MM/OhHNRZWVKhFTkREXEtBTkREwtKuXXDppVBaChMmwD33HLaCulaKiIiLKciJiEjYKS2Fyy6DnBw491x46SWna+UhFORERMTFFORERCSsWAs33QRffQWdOzt3qIyJqWXFigpdIyciIq6lICciImHlqaecm5okJsIHH0CbNnWsqBY5ERFxMQU5EREJG+++C4884nSjfPNN6NfvKCsryImIiIspyImISFhYvhyuu86Z/t3v4JJLjrGB14vf42n0ukRERE6GgpyIiDR7u3c7d6gsKYEbboD77z+OjSoqsFFRjV6biIjIyVCQExGRZq2sDC6/HLZvhyFDYPLkWu5QWRt1rRQRERdTkBMRkWbLWvj5z2HxYujUyblGLjb2ODdW10oREXExBTkREWm2fv97mDoVEhJg5kxITz+BjdW1UkREXExBTkREmqUZM2DSJGd66lQ47bQT3IG6VoqIiIspyImISLOzcKFzUxOAZ56Byy47iZ2oa6WIiLiYgpyIiDQrGzY4d6gsL4dbb4V77z3JHalFTkREXExBTkREmo29e2HsWOd5zBj461+P8w6VtdE1ciIi4mIKciIi0iyUlzvDDKxf71wPN306nHSDmt8Pfj82Qr8mRUTEnfQbSkREQp61cOONsGABtG8PH34ISUn12KHXC1FR9WjOExERaVwKciIiEvJ+9St4801ITISPPoIOHeq5w4oKiI5ukNpEREQag4KciIiEtFdfhd/+FjweZ8iBAQMaYKfVLXIiIiIupSAnIiIha948uOUWZ/qFF5wbnDQIBTkREXE5BTkREQlJ330HV14JlZVw333OUAMNRkFORERcTkFORERCzubNMGoUHDjghLnf/76BD6Br5ERExOUU5EREJKTs2QMjR8Lu3TB8OLzxBjT4KAFqkRMREZdTkBMRkZBRUOBcB7dhAwwcCO+9B7GxjXAgBTkREXE5BTkREQkJ1QN+L1sGmZkweza0aNFIB1PXShERcTkFORERcT2fD667Dj79FNq2hTlzID29EQ+oFjkREXE5BTkREXE1a+GOO+Cf/4TkZPj4Y+jWrZEPqiAnIiIupyAnIiKu9vjj8PLLEBMDM2dC//5NcFAFORERcTkFORERca0XX3SCXEQETJ8OQ4c20YF1jZyIiLicgpyIiLjStGnw3//tTP/97zBuXBMeXC1yIiLicgpyIiLiOu+959zcxFp4+mm48cYmLkBBTkREXE5BTkREXOXf/4af/tS5U+WkSfDgg0EoQl0rRUTE5RTkRETENbKznbHiKirgzjvhySfBmCAUohY5ERFxOQU5ERFxhUWL4OKLoawMJk6EZ58NUogDBTkREXE9BTkREQm6xYth1CgoLobrr4eXXgpiiAMFORERcT0FORERCapFi2DkSCgsdK6Ne/VVZ7iBoNI1ciIi4nLB/lUpIiJhbNEipyWuOsRNnQqRkcGuCrXIiYiI6ynIiYhIULg2xIGCnIiIuJ6CnIiINLnPPz+0O6WrQhw4XSsV5ERExMUU5EREpEl9/DGMHg1FRTB+vAtDHDgtcrpGTkREXExBTkREmsz778Oll0JpKdx0E7zxhgtDHKhrpYiIuF69gpwxJtUYM9cYs77qOaWO9XzGmOVVj5n1OaaIiISmt96CK65wei3+4hcwZQp4PMGuqg4KciIi4nL1bZF7CPjEWtsD+KTqdW1KrbUDqh6X1vOYIiISYl5+Ga69Fnw+mDQJnnvOBUMMHI2GHxAREZer76/RccD/VU3/H3BZPfcnIiLNiLXw+ONw223O9FNPOY+gDvZ9PNQiJyIiLmestSe/sTH51tqWVdMG2F/9+rD1KoHlQCXwO2vte3XsbyIwESA9Pf2MadOmnXRtcqiioiISExODXYZIrXR+Nk8+Hzz/fA9mzswgIsJyzz3r+NGPdgW7rONyyh//SGHPnqwfPlznpriSfm6Km+n8bDjDhw//2lqbVduyY15iboyZB7StZdEva76w1lpjTF2psLO1NscY0w341BjzvbV24+ErWWunAFMAsrKy7LBhw45Vnhyn7Oxs9HmKW+n8bH7Ky+G662DmTIiJgWnTDJdd1hPoGezSjs9rr0HfvuxKTNS5Ka6kn5viZjo/m8Yxg5y19qK6lhljco0x7ay1u4wx7YA9dewjp+p5kzEmGxgIHBHkREQk9O3dC5dfDgsWQHKyE+aGDg12VSdIww+IiIjL1fcauZnAz6qmfwa8f/gKxpgUY0xM1XQacA6wqp7HFRERF9qwAYYMcUJcRgbMnx+CIQ50jZyIiLhefYPc74ARxpj1wEVVrzHGZBljXqlapzew1BjzLfAZzjVyCnIiIs3MwoUweDCsXw+nnQaLFzvPIamiQkFORERcrV7DsFpr9wIX1jJ/KXBz1fRCoF99jiMiIu721lswYYJzbdyYMTB9OiQlBbuqelDXShERcTk3j+IjIiIuVz0u3DXXOCHu1luda+JCOsSBulaKiIjr1atFTkREwldBgRPgPvoIPB549lm4444QGCPueCjIiYiIyynIiYjICVu/HsaNg9WrITUVZsyAC4/oaB/Cqq+Rq6wMdiUiIiK1UtdKERE5Ie+9B1lZTojr0weWLGlmIQ50jZyIiLiegpyIiByXykp44AFnjLiCAvjxj2HRIujWLdiVNQJ1rRQREZdT10oRETmmXbvg6qvh88+d6+H+8Ae4++5mcj1cbTT8gIiIuJyCnIiIHNXHH8PPfga5udCunTO0wHnnBbuqRqaulSIi4nLqWikiIrUqL4d774XRo50QN3w4LFsWBiEO1LVSRERcTy1yIiJyhHXrYPx4J7h5PPDEE/Dgg850WFCQExERl1OQExGRAGth8mS47z4oLoauXeHNN2Hw4GBX1sR0jZyIiLicgpyIiACwfTvcdBPMneu8Hj8eXnoJkpODW1dQ6Bo5ERFxOV0jJyIS5qyF//1f6NvXCXGtWjk3NHnzzTANcaCulSIi4npqkRMRCWObNsFtt8GcOc7rceOcrpXp6cGtK+jUtVJERFxOLXIiImGoshKeecZphZszB1JS4PXX4d13FeKw1vmAFORERMTF1CInIhJm/vMfuPVWWL7ceT1+PDz3HLRpE9y6XKOy0rk9Z4T+r1NERNxLv6VERMJEbi7ceKNzB8rly6FLF5g927kWTiGuBl0fJyLJFpolAAALVElEQVQiIUBBTkSkmfN6nRa3U06B115zMsqDD8KKFc5g33IYXR8nIiIhQF0rRUSaKWth5kx46CFYs8aZN3asE+p69Ahuba6moQdERCQEKMiJiDRDixfD/ffDF184rzMznQB38cXBrSskqGuliIiEAHWtFBFpRlauhCuvhCFDnBCXlgbPPw+rVinEHTd1rRQRkRCgFjkRkWZgzRp44gmYNs3pUhkbC/fcAw88EMaDep8sda0UEZEQoCAnIhLCVq2Cp5927jzp9zsNSRMnwqRJkJER7OpClLpWiohICFCQExEJQf/5jxPg3n/feR0ZCT//OTz8MHTqFNzaQp6CnIiIhAAFORGREOH3w6xZ8Kc/QXa2My8mxhkb7v77oWvXoJbXfOgaORERCQEKciIiLldQ4Iz/9te/wsaNzrwWLeD22+HOOyE9Pbj1NTu6Rk5EREKAgpyIiEt99x1MmQKvvw6Fhc68Ll3gjjvg5pt1E5NGo66VIiISAhTkRERcpLgYZsyAyZOd6+CqnX++0/p26aXg8QSvvrCgrpUiIhICFORERILM74f5852Wt3/+E4qKnPnJyXD99c5NTPr3D26NYUUtciIiEgIU5EREgsBa+P57mD4dpk6FbdsOLhsyxBlC4Cc/gfj44NUYtnSNnIiIhAAFORGRJmKtM+7bjBlOgFu79uCyzp2d1rcbboAePYJXo6AWORERCQkKciIijcjng0WL4L33nDHfNmw4uCwtDa64Aq6+GoYOhYiI4NUpNegaORERCQEKciIiDWzXLpg7F+bMcR4//HBwWatWcNll8NOfwvDhzkDe4jLqWikiIiFAf0KIiNRTSQksWHAwuK1Ycejybt2c8DZuHJx9tsKb66lrpYiIhAD9OSEicoKKi2HJEli4ED791AlxFRUHl8fHw7BhMHIkjBgBvXuDMUErV06UulaKiEgIUJATETmG7dud0LZwIXz5JSxf7lz7Vs0YyMo6GNyGDIGYmODVK/WkFjkREQkBCnIiIjXk5sI33zhhbdky50YlO3Ycuo7HA2ec4XSTPPdcuPBC59o3aSZ0jZyIiIQABTkRCUs+H2zadDC0LV/uTO/efeS6LVs6oa36MWgQJCY2fc3SRNQiJyIiIUBBTkSatdJSZ7y2NWucx+rVzvO6dVBWduT6SUkwYAAMHOg8n3UW9OqloQHCiq6RExGREKAgJyIhzVrYvx82b3Za2DZvPvhYuxa2bnXWqU379gdDW3Vw69pVoS3seb26yFFERFxPQU5EXK2sDHJynMeOHc4jJ8e5AUl1eCsoqHv7yEjo3t1pVevVy7mDZK9e0LMnJCc33fuQEOL1qu+siIi4noKciDS5ykrIy4M9e5ybi2Rnp7NsmTOdm+vM37nTCWx5ecfeX2Ki05JW/ejWzXnu0QMyM9VLTk6QulaKiEgIUJATkZPi90NRERQWQn4+7NvnPPbvPzh9+KN6WX7+4XvrXedxIiOdLpAdOkBGxqHP1cEtLU3jtEkD0s1OREQkBNQryBljrgIew/kr7Exr7dI61hsN/AXwAK9Ya39Xn+OKyPHx+52uiaWlzqOk5NDn2uaVlDhdFQsLnUdd00VFJ1+XMc7t+tu0gfR0MCaXfv3SSU8/OK9tWyestWmja9akiWn4ARERCQH1bZFbAfwYmFzXCsYYD/AiMALYASwxxsy01q6q57FFGpy1TviprHRuT+/zHZyubV5d08e7bkXF8T3Ky49/vZoBrba7MjakxETnLo/JyZCaeuQjJaX2eS1bOmOxVcvOXs2wYemNW6zI8VKLnIiIhIB6BTlr7WoAc/Q+TWcCG6y1m6rWnQaMA44a5LZu9nPLDaVVx6l5zMNrOM5lR9R+vPs0R1kWzLoOX2aOut6+fT1JTS0PSl1+/8GAZOHQ1zWX25rzD1tObcvA7zdH2bZq+RHHPPhsrcFfI7z5/c2vf15srCUu1hIfV/MZZzr+0HnV67RIsiQlOo8Wif7AdFLiwWWJCfbkWsp8wN5DZ0Xt3+9cGCfiBoWFCnIiIuJ6TXGNXAawvcbrHcBZta1ojJkITHRencGUN+Iau7Yw0i7YBYSMCHx4qh6RVOLBj8dUT/uIxEeEqbnchwc/kaayxnZV+zA1llevaw7uN5oKokwF0XiJMeVE4yWaCqKNlygqiK5aVj0vmoqq6Ypa5jnbxJky4igl3pQSSxkRpipdl1Y99jfM51TZMLsBIMtaKnSRm7iFMaw6+2zys7MpKioiOzs72BWJHEHnpriZzs+mccwgZ4yZB7StZdEvrbXvN2Qx1topwBSAzp2z7MMP16yj9mk3LXNzXd9//z39+vULSl0REc7DmJN/rs+2x7uvyEinu5/TG9iDNJ3s7GyGDRsW7DJEAgZUPevcFLfSuSlupvOzaRwzyFlrL6rnMXKAjjVed6iad1StW8Mtt9TzyBKQnLwX/XsSEREREWkemuJecEuAHsaYrsaYaOBqYGYTHFdERERERKRZqleQM8ZcbozZAQwBPjLGfFw1v70xZhaAtbYSuAP4GFgNzLDWrqxf2SIiIiIiIuGrvnetfBd4t5b5O4GxNV7PAmbV51giIiIiIiLi0DC7IiIiIiIiIUZBTkREREREJMQoyImIiIiIiIQYBTkREREREZEQoyAnIiIiIiISYhTkREREREREQoyCnIiIiIiISIhRkBMREREREQkxCnIiIiIiIiIhRkFOREREREQkxCjIiYiIiIiIhBgFORERERERkRCjICciIiIiIhJiFORERERERERCjIKciIiIiIhIiDHW2mDXUCtjzA/A1mDX0YykAXnBLkKkDjo/xa10bopb6dwUN9P52XA6W2tb17bAtUFOGpYxZqm1NivYdYjURuenuJXOTXErnZviZjo/m4a6VoqIiIiIiIQYBTkREREREZEQoyAXPqYEuwCRo9D5KW6lc1PcSuemuJnOzyaga+RERERERERCjFrkREREREREQoyCXBgyxtxrjLHGmLRg1yICYIx5xhizxhjznTHmXWNMy2DXJOHNGDPaGLPWGLPBGPNQsOsRqWaM6WiM+cwYs8oYs9IYc2ewaxKpyRjjMcZ8Y4z5MNi1NHcKcmHGGNMRGAlsC3YtIjXMBfpaa/sD64BJQa5HwpgxxgO8CIwBTgXGG2NODW5VIgGVwL3W2lOBwcDtOj/FZe4EVge7iHCgIBd+ngUeAHRxpLiGtXaOtbay6uVioEMw65GwdyawwVq7yVpbAUwDxgW5JhEArLW7rLXLqqYLcf5gzghuVSIOY0wH4EfAK8GuJRwoyIURY8w4IMda+22waxE5ihuB2cEuQsJaBrC9xusd6A9lcSFjTBdgIPCf4FYiEvAcToOBP9iFhIPIYBcgDcsYMw9oW8uiXwIP43SrFGlyRzs3rbXvV63zS5xuQ/9oytpEREKNMSYReAe4y1pbEOx6RIwxFwN7rLVfG2OGBbuecKAg18xYay+qbb4xph/QFfjWGANO17VlxpgzrbW7m7BECVN1nZvVjDH/D7gYuNBqXBQJrhygY43XHarmibiCMSYKJ8T9w1r7r2DXI1LlHOBSY8xYIBZoYYyZaq29Lsh1NVsaRy5MGWO2AFnW2rxg1yJijBkN/Bk431r7Q7DrkfBmjInEuenOhTgBbglwjbV2ZVALEwGM87+x/wfss9beFex6RGpT1SJ3n7X24mDX0pzpGjkRcYMXgCRgrjFmuTHm5WAXJOGr6sY7dwAf49xIYoZCnLjIOcD1wAVVPy+XV7WAiEiYUYuciIiIiIhIiFGLnIiIiIiISIhRkBMREREREQkxCnIiIiIiIiIhRkFOREREREQkxCjIiYiIiIiIhBgFORERERERkRCjICciIiIiIhJiFORERERERERCzP8HOWJqYeVObJ4AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "8fSaGVWwYRf8", "colab": { "base_uri": "https://localhost:8080/", "height": 409 }, "outputId": "75536e38-30c9-4d33-c40a-5e24a1447c1a" }, "source": [ "z = np.linspace(-5, 5, 200)\n", "plt.figure(figsize=(15,6))\n", "plt.plot(z, np.sign(z), \"r-\", linewidth=1, label=\"Step\")\n", "plt.plot(z, tf.nn.elu(z), \"k.\", linewidth=2, label=\"ELU\")\n", "plt.plot(z, tf.nn.selu(z), \"c.-\", linewidth=2, label=\"SELU\")\n", "plt.plot(z, tf.nn.swish(z), \"y--\", linewidth=2, label=\"Swish\")\n", "plt.plot(z, tf.nn.softplus(z), \"r--\", linewidth=2, label=\"Softplus\")\n", "plt.plot(z, tf.nn.leaky_relu(z, alpha=0.05), \"m--\", linewidth=2, label=\"Leaky Relu\")\n", "\n", "plt.grid(True)\n", "plt.legend(loc=\"upper left\", fontsize=14)\n", "plt.title(\"Activation functions\", fontsize=14)\n", "plt.axis([-5, 5, -1.2, 2])" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "(-5.0, 5.0, -1.2, 2.0)" ] }, "metadata": { "tags": [] }, "execution_count": 55 }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3IAAAF2CAYAAAAr2areAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeVyU1f7A8c8ZYFiHRTFUVBAxFcstu4laUmQ367Zdtcx217RsMVssr3rzZv6sLNObZXmzrlk3zW62aFepMQsql7REFHEnEBAFWWSbOb8/npFAwYUZ9u/79ZqXM/Oc55zzDCPMd84536O01gghhBBCCCGEaDxM9d0BIYQQQgghhBAXRgI5IYQQQgghhGhkJJATQgghhBBCiEZGAjkhhBBCCCGEaGQkkBNCCCGEEEKIRkYCOSGEEEIIIYRoZCSQE0IIUauUUuFKKa2U6lsHbVmVUgvroJ3WSqn/KaUKlFL1vo+PUuqAUmpKffdDCCFE3ZFATgghRCVKqT5KKZtS6ocanFtVIHUYaANsc0kHjXbuV0rlV3Hor8BUV7VzFlOAtkAvjGurE0qpmUqpHVUcuhx4o676IYQQov5JICeEEOJ0YzCCgkuUUt2crUxrbdNaH9FalznftXO2dUxrnVfb7QCRwBat9R6t9ZE6aO+stNZZWuvC+u6HEEKIuiOBnBBCiHJKKW9gJLAYWAmMrqJMP6XUN45phbmO+22VUkuBQcBDjqmU2jGtsnxqpVLKpJQ6rJSadFqdFzvK9HE8nqyU+tXRxu9KqXeUUoGOYzHAu4BvhXZmOo5VGhFUSgUppd5TSh1XSp1USq1XSnWvcPx+pVS+UipWKbXD0d63SqmOZ3mNDgC3APc62l7qeF4rpYadXrbilEdHmXFKqRWOtvYppe4+7Zy2SqkPlFLZSqlCpdQ2pdTVSqn7gRlA9wrXfX817XRQSn2qlMpz3FYppdpVOD7Tcb0jlFJ7HWX+q5QKrlDmUqVUnFLqhOM12q6Uurq610UIIUTdkkBOCCFERcOAg1rr34B/YwQrHqcOKqV6At8CKcAAoB/wH8AdeBRIwAiy2jhuhytWrrW2Ax8Cd53W7l1AktZ6q+OxHXgM6I4RWP4JWOA4Fu84VlihnZeruZ6lwBUYgdefHOesdQSsp3hiTMccBUQDgcCb1dQHxjTG9cDHjrYfPUvZqkwHPgN6Yrx2/1JKdQBQSvkCG4Bw4FbgUuB5x3n/AV4BdvPHdf/n9MqVUiZH/SHA1Y5bW+C/SilVoWg4cAdwG3Ad0Bt4ocLx5UA6xuvWC5gJFF3gtQohhKgl7vXdASGEEA3KaIwADoyAohAjCFrpeO4pYJvWelyFc5JO3VFKlQCFFacbVo4dAFgGPKmU6qS13ut4biRGAAiA1vq1CuUPKKWeAj5TSt2ntS5RSuUaxaqf1qiU6gzcDAzSWn/neO4e4BBG4PiOo6g78JDWerejzMsYwZXSWp+RyERrnaWUKgZO1nBa5b+11sscbf0NIxC8CuN1GQm0BqK11kcd5U+9RjjWBZado91YoAfQSWt9wHHeSIzgOxYjCD113fdrrXMdZRYDD1SoJwx4WWu9y/E4pQbXKoQQopbIiJwQQggAlFKRwECMkRgcQcwHVJ5e2Rv4xpl2tNa/Ar/hGJVTSl0BdHK0daov1yil1imlUpVSecAqwIwR5JyvbhgjewkV2s51tB1VoVzxqSDOIc3RVtCFXNcF+LVCf8qALOAix1O9gV8rBHE10Q1IOxXEOdrZh3FdFa/74KkgziGtQj8A5gHvOKbOPqeU6upEn4QQQriYBHJCCCFOGQO4AYeUUmVKqTLgGeA6pVR7F7e1jD+mV94FfK+1PgiglAoDvsQY6RsOXIYx7RGMAMsVKo60nZ6E5dSxC/0bqYHThx89qihXWsV5dfX3uOJ1n7UfWuuZGIHff4H+wK9KqVEIIYRoECSQE0IIgVLKHbgPY61Yrwq3nhgjSKem3P0CXHOWqkowgsFzWQ5EKqX6YazTWlbhWF+MgO1xrXWC1joZY43XhbaThPF3LvrUE0opf4x1ZzvPo48XKosKWxEopUK48K0JfgF6VEw6cprzve62SqnwCn2JwHgNL+i6HVk5X9da3wgswQj2hRBCNAASyAkhhAC4EQgG3tZa76h4Az4CHnAkyngJ6K2UWqyU6qmU6qKUGnMqWQdwAPiTMjJVBjsSb5xBa52KsQbvTSAAWFHh8B6Mv0+PKaU6KqXuxEhuUtEBwEspNdjRjk8VbezBSPrxllLqSqXUpRgB4wkc00dd7BuMjJ19lVK9MRKtXGhykOVAJsZ6wCuVUhFKqZsrZIs8AIQpY6+/YKWUZxV1rMcIvj9w9KUvxrTVrZzntFillLdS6p9KqRjHz/IKjGm3tREACyGEqAEJ5IQQQoCxDu5brXV2FcdWYGQ4HKy13gZcC3QFfgR+AkbwxzS9lzFGjXZijFB1OL2yCpZhjPh9pbU+fupJxxq6R4HJjnrGYGzATYUy8RhB4IeOdp6qpo0HgJ+B1Y5/fYDrtdYnz9KvmnoC2AdYMZLDvIMRlJ03rXUBxhYOqcDnwA7g7/wxJfIT4CsgDuO676yiDo2RoCYLI8Pot8AR4NaqkrdUw4axRnApRpbMTzHWGk6+kOsRQghRe9T5/04XQgghhBBCCNEQyIicEEIIIYQQQjQyTgdySqn2SqlvlVI7lVKJSqkzNkZVhteVUilKqV+VUn2cbVcIIYQQQgghmitXbAheBjyhtd6qlLIAW5RS67TWFRdEDwE6O25XAIsc/wohhBBCCCGEuEBOj8hprdO11lsd9/Mw0h6HnlbsFuB9bfgRCFRKXWhKZiGEEEIIIYQQuHiNnGPPmt4YWcwqCgUOV3icypnBnhBCCCGEEEKI8+CKqZUAKKX8MNIiP6a1PlHDOsYB4wC8vb0va9++vau61+zZ7XZMJsltIxomeX+Khkrem6I2mUpL0W5u6ArvscPASYwPaOFU/sb95MmTlB0uw4KFUkqxhdrw8vWqYet5QDqgMHYJqWpLQiFqRn53uk5ycvJRrXWrqo65JJBTSnlgBHEfaK1XVVHkd6BiVNbO8VwlWuvFwGKAvn376s2bN7uiewKwWq3ExMTUdzeEqJK8P0VDJe9NUWsOHoQrr4SQEFizBgIDeTg5mX+mpQHgbjLxfs+eRAcElJ/y3p/fI+xwGIUU8rB6mAcfepCpU6fWqPm8vG3s3Hk77ds/Qdu2411ySUKcIr87XUcpdbC6Y67IWqmAJUCS1npeNcVWA/c6slf2A3K11unOti2EEEII0eikp8O118Lhw2AygYcH3x4/zhuOIA6g1G7HmpNT/ljbNOHZ4dixM9s0mzRzmlMflC2WXvTtu402bcY5cyVCiHrkihG5AcA9wG9KqW2O557FGKdHa/0m8BVwA5ACFAIPuKBdIYQQQojGJTsbBg+GlBTo3Ru+/JJUNzfu2LkTDbgrhdYas8lETGBg+WnKTXFlwpUkLEzgxqIbudP/TqKjoy+4+RMnNuHvfzkAbm4+rroqIUQ9cDqQ01p/jzHB+mxlNPCQs20JIYQQQjRaJ07A9ddDYiJ06wZff02Jvz/Dt20jq7SUa4OCmBEWxsbcXGICA4kOCCB+fTwbftxATGwM0dHRDHh8AAMYgNVqveDmMzI+IinpTkJDJ9G58+uuvz4hRJ1yWbITIYQQQghRjcJCuPFG2LwZIiJg/XoSzGYm//ILP+bl0d7Tk+XdutHKbGagYyQufmM8m/68CQ+7B7e8cAufffNZjUbhjOb3kJw8FgAfn24uuywhRP2RdDJCCCGEELXN09MI4Nq1g7g4Enx9idm2jR/z8gCYERZGK7O50imHnjpET3tPIojAVGKq0SgcgM1WRGLicGy2fFq1up22bR909mqEEA1AoxyRs9vtHD16lJycHGw2W313p1EICAggKSmpvrtRY25ubgQGBhIcHCzpbIUQQjQ+bm7w7ruQkQFt2vDhnj2UaA0Y36pnlpZWKp72Vhqtf2xNKaXMMM3ghOeJGic32bt3MgUF2/Hy6kSXLm9j5KkTQjR2jTKQS01NRSlFeHg4Hh4e8gvpPOTl5WGxWOq7GzWitaa0tJSMjAxSU1Pp0KFDfXdJCCGEOLeSEvj73+GppyAgwMhQ2aYNx0tL+SQrCzCSDHieltgk57sc9jy8BwDvad6M9BlJTExMjaZVZmZ+TFraIpQy0737x7i7+7vk0oQQ9a9RBnIFBQV06dJFRmaaCaUUZrOZ0NBQdu/eXd/dEUIIIc6trAxGjoRPPoEtW2DtWgDsWnPvrl2klZRwsbc3d4WEMDgoqHy/uPhV8RTcU4BHmQftnmhH5KxIBjKwRl3Q2s6hQ3MBiIych8XSxzXXJoRoEBplIAdIENcMyc9cCCFEo2CzwX33GUFcQADMng1AQm4u0/fvZ31ODkHu7qzt0YOO3t7lpyUkJLDi9hXcYruFzabNmG81E0lkjbuhlIlevb7hyJGltG070enLEkI0LI02kBNCCCGEaHDsdhg7FpYvBz8/+Ppr6NOHhNxcrt62jWLHurhpHTpUCuIArFYr/9T/JJNMvuALntn4DP0H9neqO+7u/rRr94hTdQghGiYZ4hBCCCGEcAWt4aGHjKQmPj7w1VdwxRUAfJqVVR7EKSi/X36qXRMTE4O7pzv/cfsPpZ6lNU5ukpm5kt27x2OznXTmaoQQDZyMyAkhhBBCuMLy5fDmm+DlBZ9/DldeCcBJm43PsrPLi3mdltwkc0UmaW+kcdmKy4iLi8NqtdY4ucnJk3vZvXs0NtsJAgIG0rr1Pc5flxCiQZIRuTqWlZXFxIkTCQ8Px9PTk5CQEGJjY1m3bh0A4eHhvPzyy/XcSyGEEEJcsBEjYMIE+PRTuOYawMi8/NCePSSfPElbs5m/hYUR17NneXKTvF/y2HXfLnKsORxddZTo6GimTp1aoyDObi8mMfEObLYTBAf/lZCQu116eUKIhkVG5OrY0KFDKSwsZMmSJURGRpKZmcmGDRvIrvBNnRBCCCEakeJiY8NvNzd4441Kh95JT+fdI0fwMpn4qkcPevr5lR+L/zKevLvy8DzpSesHWtNmbBunurF375Pk52/By6sjXboske2ZhGjiZESuDuXk5LBx40bmzJlDbGwsYWFhXH755UyZMoURI0YQExPDwYMHefLJJ1FKVfoFHB8fz6BBg/Dx8SE0NJQJEyZw4sSJ8uMxMTE8+OCDPProowQFBREUFMSTTz6J3W6vj0sVQgghmofZs+Gqq+D48TMOLUlLY0JyMgBvXXxx5SBuQzzbbt6GZ64nO007OXbfMacCr6ysVfz++wKU8iAq6mM8PALPfZIQolGTQK4O+fn54efnx+rVqykqKjrj+KpVq2jXrh3Tp08nPT2d9PR0AH777Teuu+46br75ZrZv386qVavYtm0bo0aNqnT+Bx98gN1uJyEhgbfeeovFixfz2muv1cm1CSGEEM3OvHnw3HOwaRN8/32lQ2uysxmbnIwNY/pT5woZKrXWpD6RSpQ9ikwymcEMNsRvqHE3Tp7cz65dxmeCTp1ext+/b43rEkI0Hk1ramV9TCE4LevU2bi7u7N06VLGjh3L4sWL6d27NwMGDGD48OFcccUVtGjRAjc3NywWC61bty4/76WXXuKOO+7giSeeKH9u0aJF9O7dm8zMTC666CIA2rRpw+uvv45Siq5du5KcnMy8efOYPHmy665XCCGEELBgAZz6u7xkCdx0U/khm9ZM2rOHU58QNGDNySlfF5e9OpuLtlxEEUVMN02nwLOgxhkqATw8WtKy5RDs9mJCQyfVuB4hROPStEbktK772wUaOnQoaWlpfP755wwZMoT4+Hj69evHbMdmoVXZsmULy5YtKx/R8/PzY8CAAQDs3bu3vFy/fv0qTcuIjo7m999/rzQFUwghhBBOWrgQHnHszbZoETzwQKXD0/fvZ69j5o0bYD4tS2XLm1rS4dkO+D3vx33/uI+4uLgaJTc5xd3dn27dltOt23JZFydEM9K0RuQaCS8vLwYPHszgwYOZPn06Y8aMYebMmUyZMqXK8na7nTFjxvD444+fcSw0NLS2uyuEEEKIU/75T5g06Y/7Dz5Y6fBnR48y+9AhTMD8yEjybDZiAgOJDgggISHhj60FXogmgggGMrDGXTlxYhO+vpfg5uaNUgo3Ny8nLkwI0dhIINcAREVFUVZWRlFREWazGZvNVul4nz59SExMJDIy8qz1/PTTT2ity7+N+/HHH2nbti3+/v7k5eXVWv+FEEKIZkFr2LbNuL9wIUycWOnwxxkZ3LtrFwBzIiJ4uF278mPx6+JZM2QNb+m3mOU5y+lROPid7dsn4OXVkV69vsXDo4UTdQkhGqOmNbWygcvOzuaaa65h2bJl/Prrr+zfv58VK1Ywd+5cYmNj8ff3Jzw8nI0bN/L7779z9OhRAJ5++ml+/vlnHnzwQX755RdSUlL44osvGD9+fKX609LSeOyxx9i9ezcrV67kpZdeqnIUTwghhBA1oBS89RasXw8PPVTp0DfHjzMiKYlirXEDBvj7lx/TNk3GwxnE2mKZbJ9MSUkJVqu1xt2w2YqAmdhseXh7R+LuHlTjuoQQjZeMyNUhPz8/+vXrx/z580lJSaG4uJjQ0FBGjhzJtGnTAHj++ecZP348nTp1ori4GK01PXr04LvvvmPatGkMGjQIm81GREQEt912W6X677rrLmw2G1dccQVKKUaPHi2BnBBCCOGslSshNhaCgsBkMu5XoLVmckoKFVfOb8jNpb9jXdy+qfsISg4il1wWmhZiNpudSm6SkvIYkIKXVwRdu/5L1sUJ0UxJIFeHPD09mT179lkTm/Tr14/t27ef8Xzfvn1Zu3btWet3d3dn4cKFLFy40Om+CiGEEAJ4+20YNw4uuwzi48FsPqPIgt9/Z3tBAXBmcpMj7x/h8EuHUe6K4NeCeejEQ8YauRpOq8zIWE56+luAB927r8DdPaCmVyaEaOQkkBNCCCGEqMo77xhBHMBdd1UZxH2fk8MTjgzS/wgPx6RUeXKTH97+geIJxZgwEbkgktAHQxnAgBp3p6BgF7t3O/rDw1gsfWpclxCi8ZNATgghhBDidEuWwNixxv1XXoEqliqkFxczfOdOyrTmiXbteC48vPxY/FfxZIzLoAUt+MLtC4b2HEoozmWazshYht1ewEUXjSQz86ZznyCEaNIkkGsinFk0LYQQQogK/vWvP4K4l1+GyZPPKFJqtzPk1185UlJCL19f5kREVDq+YdsGdqld9NQ9WaAX0NLa0sksldCx4yx8fLoQHHwbmZmbnapLCNH4SdZKIYQQQohTfvgBxowxthqYOxeeeKLKYvckJZWvi9t18iSbTtvmJ+bqGFZ4reAZ0zO4ebo5ldxEayONilKK1q3vwd3dr8Z1CSGaDhmRE0IIIYQ4JTraGI3r1AmefLLKIv/JzOQ/WVnlj0vtdqw5OUQHBHDk/SMEXh1IdHQ0cXFxf2wAXsPRuPz8HezePYZu3d7Dx6dLjeoQQjRNEsgJIYQQQpSVgbu7sb3Am28ae8ZVIbGggNGOTb89lMKudXmWyu9f+p7Sp0tRLRUD9w4kOjraqemUZWX5JCYO4+TJ3aSmvsbFFy+qcV1CiKZHplYKIYQQonl791244go4dsx4XE0Qd6KsjL/u2EGB3c5dF12EtWdPZnXsSFzPnti/2MGJp06gtGJpzlI2JW5yqktaa5KTx3Py5G58fS+hU6dXnKpPCNH0SCAnhBBCiOZr8WIYNQq2boXPPqu2WHxODtFbt5J88iSX+vryVpcu9A8MZGpYGH3LfMh7JA8ffPiGb1hmX+Z0ErL09MVkZi7HZPIlKmoFbm4+TtUnhGh6ZGqlEEIIIZqnhQth0iTj/ksvwQMPVFksITeXmO3bKXUkHflbhw74urkBYC+1k3h7Il7HvNij9vCKegWzp9mp5CZ5eb+wZ8+jAHTpshhf3641rksI0XRJICeEEEKI5ufVV//YVmD+fHjkkWqLLklPLw/iTEBKUVH5sb2T95LzTQ4eIR50f7M705KmOZXcxGYrIDFxOFoX06bNeEJCRtaoHiFE0ydTK+vY/fffj1LqjFu/fv0ACA8P5+WXX67yXKUUK1eurLLOv/zlL7XabyGEEKLJmDv3jyDujTfOGsQdLiriE0eGSgV4OhKbAMTHxxOfEg+ecMmnl9D/1v5MnTrVqQQnJpMPHTo8ib9/NJGRr9W4HiFE0ycjcvXg2muv5d///nel58xmcz31RgghhGhGtIYDB4yEJm+/DaNHV1u02G5nWGIiOTYbf7JYuDk4mGsCA4kOCCAhIYFrr72WkpISQjxCWMlKonFuw28wvrRt23Y8bdqMRSn5vl0IUb1mH8glJCQ4vcfLhfL09KR169Z10pYQQgghKlDKWBt3110wYEC1xRJyc3ksJYWf8/II8/Tkqx49aOnhAUDRwSISViVQUlKCzWYjgwysVqtTnyPy8rbi5uZbvlecBHFCiHNp1oFcQkICsbGxlJSUYDabiYuLq7NgTgghhBB1RGtYsABGjoTgYGOvuHMEcTHbtlHiWBc3PSysPIgryy/jt5t/o8/BPkS5R7GTnZjNziU3KS09xo4dt1FWdoyePb/F379vjesSQjQfzfrrHqvVWv5tWklJidOpgs/X2rVr8fPzq3R7+umn66RtIYQQolnRGp58Eh59FG64AWy2c56yPCOjPIgzARmlpUZVds2ue3dR8GsB3iHevLn6TWbNmuXUF8Fa29m16z6Kiw/h4xOFn1+PGtUjhGh+mvWIXExMDGazuXxEzplv0y7EVVddxeLFiys9F+hYOC2EEEIIF9HaCOAWLAAPD5g6FRzbBlTnWGkpnxw9ClRObpKQkEDKtBTaf9MetwA3Ll19KT5dfOh/XX+nunj48CtkZ3+Bu3sQUVH/wWSSNfNCiPPTrAO56Oho4uLi6nyNnI+PD5GRkRd8nsViITc394znc3JyCAgIcEXXhBBCiKbBboeHHoI33wSzGVauhJtuOvspWnN3UhLpJSV08fbmrpAQrg0Kgp07mREzg2dLnsWGDY8ZHvh0cX6D7pyc79m3byoAXbu+j7d3uNN1CiGaj2Y9tRKMYM7ZVMF1pUuXLmzZsqXSczabje3bt9OlS5d66pUQQgjRwNhsMHasEcR5esJnn50ziAOYdfAga44do4W7O1/37MnfwsOJDgjg5+U/M7nE2K5gsVrMd0XfOd3FkpIsdu4cAdho3/5JgoNlGyEhxIVp1iNy9aW4uJgjR45Ues7NzY1WrVoBkJaWxrZt2yodb9euHZMnT+aBBx6ge/fuDB48mMLCQhYsWMCxY8cYN25cnfVfCCGEaNA+/hj+9S/w9obVq+Haa895yiuHDjHzwAEAPoyKIszLq/zYZYGXUUwxX6uv+dzzcx6LeczpLublbaK09Cj+/gPo2PEFp+sTQjQ/EsjVg/Xr19OmTZtKz4WGhpKamgrAq6++yquvvlrp+IIFC3j44YcBeOWVV5g6dSo+Pj5cdtllbNy4UbYzEEIIIU4ZMQJ++gluvRXOY/37ysxMpuzbB4CHUlhOW0c3cNZAfmj/AwFHAogb7JoM1y1b3kCfPgl4eLTCZPJwuj4hRPMjgVwdW7p0KUuXLq32+AHHt4HVufPOO7nzzjtd2ykhhBCisSsshPx8uOgiY6+41147r9NO2mw8kpJS/tiuNdacHHRiIvFfxDPgpgFER0czYNwABlD9lgXny24vw2QyPn5ZLL2drk8I0Xw1+zVyQgghhGjkcnPh+uuNKZTHjp33aVprJiQnk15SggLcALPJRMvUVF6PeZ0uL3ZhSswUEhISXNLN4uIjbNrUjYyMD11SnxCieXNJIKeU+pdSKlMptaOa4zFKqVyl1DbHbbor2hVCCCFEM5edDbGxsHGjEcRlZ5/3qYvT03kvIwNvk4n3unZlVseOxPXsSdmS7YwtHYsFC96l3i7ZZ9ZuLyMp6U5OnkwhPf1ttLY7XacQonlz1dTKpcBC4P2zlNmotZaUTEIIIYRwjfR0GDwYEhMhIgLWr4eOHc/r1HfS0piYnAzA4osv5m7HWvPCPYUUr4wC4AP1AfFe8cyKmeV0V/fvn0ZOjhWzuTXdui1HKZkUJYRwjksCOa31d0qpcFfUJYQQQghxTgcPGiNxe/dCVBSsWwdt257XqV9mZzMuORkNuCtFJ29vAMpyy9hxyw7IA9NAExHXRxB3jfPJTY4eXc3hw/8HuBEV9R88PSVBmRDCeXWZ7CRaKbUdSAOmaK0T67BtIYQQQjQVWVkwcCCkpkKfPvD11xAcfF6nltntPLJnD9rxWJ9KbvJbIhljMgjaHYRPdx/6fNWHqyxXOd3Vkyf3kpR0LwARES8SGOh8nUIIAXUXyG0FwrTW+UqpG4D/Ap1PL6SUGgeMAwgJCal2TnpAQAB5eXm119smyGazNYnXrKioyCVrFUTDkp+fLz9X0SDJe7OB0prIyy/HEhjIr3//O7YdVS7Rr9JiYJ/jvgnjg9DRuDgmPDifl0tf5gQnSB6eTOGWQld0FJgE5AID2bevL/v2WV1Qr7w3RcMm78+6USeBnNb6RIX7Xyml3lBKBWutj55WbjHG71j69u2rY6rZ+yUpKQmLxVKLPW568vLymsRr5uXlRe/ekq65qbFarVT3/12I+iTvzQZGa2NrAYBBg6C4mCsd0yLPx6qsLD5MTMQNeL1zZ3LLyogJDMSakECiPZHJTMZsMjPMPMxlP/e8vGXs3z+VqKiPcHcPcEmdIO9N0bDJ+7Nu1Ekgp5RqDWRorbVS6k8YX4Kdf1opIYQQQjRvVis8/TR88QW0agUmE1xAEPdRRgb37doFwNxOnZgYGgqAvcwOMTGYzWaSSpIwm80u/QBqsfSiR481LqtPCCFOcUkgp5T6EIgBgpVSqcAMwANAa/0mMAyYoJQqA04CI7TWuprqhBBCCCH+8OWXMGwYFBXBP/8JM2de0Olxx44xMikJjbFXXD/HDJXi34vZFruNzi91Ji4urnwUwdnkJnl5v1BYuJOQkLucqkcIIc7GVVkr7zzH8YUY2xMIIYQQQpy/Dz+Ee++FsjJ48EGYfmFb0WqteXzvXmfw06EAACAASURBVCp+e7whN5crPC3suG0HJ3efJPXVVPrF9XM6gAMoLc0hMXEYRUX7ABMhIWf9iCSEEDUmm5jUsaysLCZOnEh4eDienp6EhIQQGxvLunXrAIiJiUEpdcZtxIgR5XUopVi5cuUZdR84cAClFJs3bz7j2A033MDDDz9cexcmhBBCuNqCBTBypBHEPfkkvPGGMaXyAsxPTeW3ggLAGI0zm0wMCgjgh1t/IG9THrSBqP9EoU6tvXOC1nZ27bqXoqJ9+Pn1ITj4NqfrFEKI6tTl9gMCGDp0KIWFhSxZsoTIyEgyMzPZsGED2dl/LBl84IEHmD17dqXzvC9gHYAQQgjRqGkNM2bALMdG3HPnGoHcBdqYk8OUvXsBeKFjRxQQExhI2bO/Yv+fnZOc5IljT7AkZQnRrZwfjTt0aC7Z2Z/j7h5I9+4rcXPzcrpOIYSoTrMP5BJyc7Hm5BATGEh0gOuySVUlJyeHjRs3sm7dOmJjYwEICwvj8ssvr1TOx8eH1q1ls1AhhBDNlFJQXAxubvD22/DAAxdcRXpxMbfv3IkNeLJ9e54NCwMg+8tsfl1UhkIxm9kklyVjtVqdnlZ5/Pi37N//HADdui3D27ujU/UJIcS5NJlATtXTXhX6AjJb+fn54efnx+rVqxk4cCBeXvJNnRBCCFGlOXPgjjuMDb8v0MacHO5JSuJISQkxgYHM7mgEVbZCG7tG7UJpxfvu75OgE1ySpbK4OI2dO0cAdjp0eJaWLW90qj4hhDgfskauDrm7u7N06VKWLVtGYGAg0dHRTJkyhZ9++qlSucWLF5cHfadub7zxRj31WgghhKgDeXkwahSkpxuPlapREJeQm8s127dzsLgYBUxp3x53x7o6Nx83Lv38UkIfDmXMhjHMmjWLuLg4p0fjyspO4O7uT2BgLB07Pu9UXUIIcb6azIjchYyMnZKQm0vs9u2U2O2YTSbievas9emVQ4cO5cYbb2Tjxo0kJCSwdu1aXnnlFV544QWeffZZAO644w5mzJhR6bxWrVrVar+EEEKIepOZCTfcAFu2QFoarF1b46rmp6ZS5tjhSAG/5udzY8uWJCQk/LG9wIJoOtOZ/v37u6T7vr5dueyyzdjtpSjl5pI6hRDiXJpMIFcT0QEBxPXsWWdr5E7x8vJi8ODBDB48mOnTpzNmzBhmzpzJlClTAAgICCAyMvKC6/X39wcgNzf3jGO5ubkE1NH1CSGEEOdt/374859hzx6IjIRFi2pc1W/5+fz36FHAmHLkaTIRExjIxns2MuvjWXxj+waz2eySUTiAoqLDeHm1B8DdXf7GCiHqVrOfWhkdEMDUsLA6C+KqEhUVRVlZGUVFRU7V06JFC4KDg9myZUul50+cOMG+ffvo0qWLU/ULIYQQLvXbbzBggBHE9e4N338PHWuWJCS3rIy/JiZSrDVDgoKY1bEjcT17EraiANsyG1NKphBkC6KkpASrC9bVFxYms2lTd3bvHofdXup0fUIIcaGa9YhcXcvOzmb48OGMGjWKHj16YLFY2Lx5M3PnziU2NrZ8RK2wsJAjR45UOtdsNtOiRYvyxwcOHGDbtm2VykRERDB58mTmzJlD27ZtiY6OJjs7m1mzZhEcHMzw4cNr/yKFEEKI8/H993DTTZCTAzEx8N//Qg2/VP0hJ4exycmknDxJT19fVl5yCT5ubuR8n8P2iXsA+KfHPzluP+6S5CY2WwGJiUOx2fIoK8tFKfk4JYSoe/Kbpw75+fnRr18/5s+fT0pKCsXFxYSGhjJy5EimTZtWXu7dd9/l3XffrXTugAED+P7778sfP1nFfjqff/45Tz31FH5+fsydO5d9+/YRGBjIwIED+eKLL2QvOiGEEA3Hhg1GEHfbbbB8OdQwk3NCbi5Xb99OqWNd3HNhYfi4uVF0qIjEvyaiSzXtHmvHM7c/Qz9rP2ONnBPTKrXWJCc/SEHBDry9u9Clyzsu2UxcCCEulARydcjT05PZs2efsdl3Recz3UM7/lhVZ9KkSUyaNKnSc3l5eefVRyGEEKJOPPssRETA8OHgXvOPI2+np5cHcSYg5eRJ4r+J5/jdx/HN8iVocBARL0UQ6R7pknVxaWlvkZGxDJPJh0su+QR3d4vTdQohRE00+zVyQgghhKgDWsP8+XDokPFYKbjzTqeCuENFRXySlWVUh5HcpGVqKmuuW4Nvui+/q98peKIAk7trPu6cOLGJlJRHAejS5W18fbu7pF4hhKgJCeSEEEIIUbvKymDiRHjsMbjxRih1PjlIkc3G0MRETthsXGGxlCc3yf7uO1boFexjH9PUNDZs3eCCCzDs3z8NrUto23YiISEjXVavEELUhEytFEIIIUTtKSgwRt4+/9xYB/f3v4OHh9PVPpqSwua8PMK9vPiqRw9anKozJoZZnrMYXzweD08PpxObVNS9+0oOH55LWNi0cxcWQohaJoGcEEIIIWpHVhb85S/w88/QogWsXm1sN+CkZ/ftY3F6Oh7AJ92708LDg/zt+RTsKCD6rmji4uL+2PzbBeviTnF3t9Cx4yyX1SeEEM6QQE4IIYQQrrd3L1x/PaSkQHg4rFkDXbs6Xe3S9HRerLDOrthuJ/7LeE7cfQKvHC+UhyL69miXBXBHj67m6NHVdO68EDe3mmXWFEKI2iBr5IQQQgjhemvXGkFc796QkOCSIC67tJTH9+4tf2zXmmVbtrP15q145XixW+0m5aIUp9s5pbAwmaSkezhyZAmZmR+6rF4hhHAFCeSEEEII4XoTJ8LixcZ+ca1bO12dTWvu2rmTnLIyFOAGmJWiz9+Ocon9ErLI4m/qb2xIcE1yk7KyfHbsuA2b7QStWg2jdev7XVKvEEK4ikytFEIIIYRrvP02xMRA587G9gJjx7qs6r8fOMDXx48T7OHB4osvZldhIVeuKKUsXlNMMTNNM8n3zHdJchOtNbt3j6KwcCc+Pt3o0uVfsum3EKLBkUBOCCGEEM6x2eCpp2DePCOI+/VXI0Oli7x86BCzDh5EAR9FRREbFMTxb46z/ZntAPjO9OVu890uS26SmjqPrKwVuLlZuOSST2XTbyFEgySBXCNmtVq5+uqrycrKIjg4+JzllVKsWLGCYcOG1UHvhBBCNAuFhXD33fDpp8a2As8959IgbkVGBk/u2weAu1L4mEwkJCTw0+qf6BXSiw73diBiRgQDGeiS9nJyvmfv3qcA6Nr1fXx8urikXiGEcDVZI1fHsrKymDhxIuHh4Xh6ehISEkJsbCzr1q274Lr69+9Peno6LVu2rIWeCiGEEOdw5IgxlfLTTyEwEL7+Gu67z2XVF9psPJLyR/ISu9a8/+uvxMbGMmXhFG4/djtH/nLEZe0BWCy9ueii2+nQ4VlatbrVpXULIYQryYhcHRs6dCiFhYUsWbKEyMhIMjMz2bBhA9nZ2Rdcl9lsprULFpALIYQQFywxEW64AQ4dgo4d4csvoVs3l1WvtWZ8cjJHSktRGN88e2lF6+UplJSUYLPZOMYxNny3gf4D+rusXTc3X7p1Ww5ol9UphBC1QUbk6lBOTg4bN25kzpw5xMbGEhYWxuWXX86UKVMYMWIEb775Jl0rpGdev349SinmzJlT/tzdd9/NmDFjAGNqpVKKo0ePApCbm8s999zDRRddhJeXFxEREbz22muV+nDs2DGGDx+Or68vERERLFu2rA6uXAghRJPz449GENevn3HfhUEcwKK0NJZlZOBjMvF+167M6tiRNZ9fxKA3OzJajcbNzQ2z2eyS5CYA6envYrMVAsZSBKXkI5IQomFrUiNyVmv1GaUuvvgt2rYdB0Ba2mKSk8dXWzYm5o9v4TZvvoz8/K3nLHc+/Pz88PPzY/Xq1QwcOBCv09YQxMTEMGHCBI4cOULr1q2xWq0EBwdjtVp55plnANiwYQMvvvhilfVPmzaN3377jS+++IKQkBD2799PVlZWpTLPP/88c+bM4cUXX2TJkiWMGjWKq666ig4dOlzQtQghhGjmRo8Gb2+47TbjXxdanJbGpD17AHinSxfuDAkhY3kGSa/uBze4d969hOeHuyy5SVra2yQnjyM9fTG9e/8gQZwQolGQ31R1yN3dnaVLl7Js2TICAwOJjo5mypQp/PTTTwB07dqV1q1b8+233wLGiNuUKVP4/vvvKSsrIyUlhdTU1Gq/fTx48CB9+vThT3/6E2FhYcTExDB8+PBKZe655x7uvvtuIiMjmTVrFu7u7nz33Xe1et1CCCGaALsdZswwplSeMnKky4O4L7OzGZ+cjB0juUm4lxc/LPmBxPuMdjvP78yASQOYOnWqS4K4Eyd+Zs+ehwFo2/ZBCeKEEI1GkxqRO98RsrZtx5WPzp1L375bnOnSGYYOHcqNN97Ixo0bSUhIYO3atbzyyiu88MILPPvsswwaNAir1cott9zCpk2b+OSTT1i0aBGbNm0iMTGRTp060a5duyrrnjBhAsOGDWPLli0MHjyYm266iUGDBlUq06NHj/L77u7utGrViszMTJdeoxBCiCYmP9/ITPnZZ/DBB7BzJ5jNLm+mzG4vH4kDY53cx99sZ8CYTIIJ5iu3r7i1962EEuqS9kpKMklMHIrWJbRt+xCtW7suUYsQQtQ2+dqpHnh5eTF48GCmT59OfHw8o0ePZubMmZSUlBATE8O3335LfHw8kZGRhISElD9ntVrPuhZgyJAhHDx4kClTpnD06FFuvPFGHnjggUplPDw8Kj1WSmG322vjMoUQQjQFBw/CgAFGEBcUBG+9VStBHMCz+/ezv6gIADfAt1Qx8PF8gglmO9t5Xb/Ohg0bXNKW3V7Gzp13UFycir9/fyIj57mkXiGEqCsSyDUAUVFRlJWVUVRURExMDHv27OGDDz4oD9pOBXIbNmw456Lu4OBg7rnnHpYuXcqSJUt47733KC4urv2LEEII0fQkJMCf/mRs8H3xxUZSk9jYWmnqk6wsXjp8GDfgjc6djeQmbaMI1r5kqAyeNz2PydPksuQm+/dPJSfHitncmu7dV2Ay1U5wKoQQtaVJTa1s6LKzsxk+fDijRo2iR48eWCwWNm/ezNy5c4mNjcXf3x9/f39at27NsmXL+PDDDwEjkBs7dixlZWVn/QM2ffp0+vTpQ/fu3SkrK2PVqlVERETg6elJSUlJHV2lEEKIJuGDD2DUKCgpgWuvhY8/NkbkasGuggLu37ULgJc7dWJC6B9TJ0u39+fntT8zZf8UlyU30dpOSUkmSrkTFbUCT8+2TtcphBB1TQK5OuTn50e/fv2YP38+KSkpFBcXExoaysiRI5k2bVp5uUGDBvHxxx+Xr28LDw8nNDQUd3f3atfHAXh6evLcc8+xf/9+vLy86NevH59//nmtX5cQQogmqqQEJk6E116D06bmu8r6Y8cYmZREvs3GHa1a8Wi7dsSvimfDrg3EXG0EbgNGDGAAA1zWplImunZdSvv2k/Hz6+myeoUQoi4prRvmhpd9+/bVmzdvrvJYUlIS3Vy8X01Tl5eXh8Viqe9uOE1+9k3TudZ/ClFfmt17U2tQFbby+eknuOKKWmsuPieHq7ZtwwYoYF2PHnh9nUTOyBy+Vd/yludb/O+b/7lkFA6gtDQbpcy4uzf+v4fN7r0pGhV5f7qOUmqL1rpvVcdkjZwQQgghjKQmV14JWyvsnVqLQRzAPw4dwua4bwK2pOZwYtIJfPHFT/tRVFKE1Wp1SVt2eyk7dvyVrVujOXlyv0vqFEKI+iSBnBBCCNHcWa3Qty/88ANMmVI3TR4/ztfHjgHGhxEvu+KKx47jne3NXrWXl00v4+Hp4bJv9VNSHic39zvKyo5hMnm6pE4hhKhPskZOCCGEaK60hoUL4fHHwWaD668HR6Kt2vR7cTF37NyJHbgnJIRuPj4M/L98bN9m4dHKg25vdeO5Xc+5LLlJevoS0tL+iVJmundfJclNhBBNggRyQgghRHNUVGQkMnn3XePx00/DCy+Am1utNvtdTg73JCWRWVpKbGAgY48fZ/e0H7CtikR5KLqv6k7gwED6098l7eXmJpCcPAGAiy9eREBAP5fUK4QQ9U2mVgohhBDNjdbwl78YQZy3tzEKN2dOrQdxCbm5xG7fzqHiYhQwJD+fIbHXYV9lB8BtihuBAwNd1l5x8e8kJv4VrUsJDX2YNm1GuaxuIYSobzIiJ4QQQjQ3ShmjcXv3wqpV0Lt3nTT7amoqZY5s2Sbgf3v3UlRaxGM8xjXqGgZYBjCQgS5rLyPjQ0pKjhAYGEOnTvNcVq8QQjQEMiInhBBCNBcpKX/c/+tfYefOOgvifs3P57OjRwHjw4ePTTG0UyfMZjMlbiV84/WNy9OVt2//BF26/IuoqI8xmWpnHzwhhKgvMiInhBBCNHXFxUZCkyVLYMMG6OdYJ+btXSfN55SW8tcdOyjRmhtatGCAvz8DH8uhVYAn679az4aEDS5LbAKgtQ2l3FBK0abNAy6pUwghGhoJ5IQQQoim7PBhGDYMfv4ZzGbYv/+PQK4O2LXm3l272FtURC8/P57Mz+fQE9uwr+/AUYsblz13Gf2nuiaxCcDx49+SkvII3buvxMeni8vqFUKIhkamVjZCn332GZ07d8bd3Z3777+/xvUcOHAApRSbN292XeeEEEI0HHFx0KePEcR16GDsE3fnnXXahQnJyXyenY3FzY3nSkqYdfXf6LC+A3bsqOkKn4t9XNbWyZMHSEwcTkHBDjIyPnBZvUII0RC5JJBTSv1LKZWplNpRzXGllHpdKZWilPpVKdXHFe02RllZWUycOJHw8HA8PT0JCQkhNjaWdevWnXcdo0ePZujQoRw8eJD58+dz//3385e//KUWey2EEKJRsdvhxRfhuuvg6FHj3y1bjE2/69Crhw+zOD0dgGK7nc2rE5lSYmw4/o56h42lG13Wls1WwI4dt1JWlk2LFkMID5/hsrqFEKIhctXUyqXAQuD9ao4PATo7blcAixz/NjtDhw6lsLCQJUuWEBkZSWZmJhs2bCA7O/u8zs/JySE7O5s///nPhIaG1nJvhRBCNEppacZ2AnY7TJsGM2fW+tYCpztw8iTP7d9f/tjvuObK9zrjDaxT6/iv53+ZFDPJJW1prdm1azQFBdvx9u5Mt27LUapur1cIIeqaS0bktNbfAcfOUuQW4H1t+BEIVEq1cUXbjUlOTg4bN25kzpw5xMbGEhYWxuWXX86UKVMYMWIEAMePH+e+++4jKCgIb29vrr32WhITEwGwWq0EBQUBcM0116CUIiYmhvfee48vv/wSpRRKKaxWa/m0yeXLlzNw4EBatWpF165d+d///ldt/6xWK0opjjqyisGZ0y9LS0t55JFHaNu2LZ6enrRv355nnnmmtl4yIYQQNdGuHXzwAXz+OcyaVedBXJHNxtDERE7a7ZgAN+CBpeB7BFSUwjLTQtw3cS5LbnLo0P+RlfUf3NwsXHLJZ3h4uG4vOiGEaKjqKtlJKHC4wuNUx3PpLm1FqeqPvfUWjBtn3F+8GMaPr76sY48bAC67DLZuPXe58+Dn54efnx+rV69m4MCBeHl5nVHm/vvvZ/fu3Xz22WcEBQXx3HPPcf3115OcnEz//v1JTEyke/fufPLJJ/Tv3x8fHx/Gjh3LsWPH+Pe//w1AixYtSEtLA+Cpp55i3rx5dOrUiaVLl3LLLbeQkpJS49G8119/nU8//ZSPPvqI8PBwUlNT2b17d43qEkKIRmH5cnjoofruxbmVlBh/lzw967snPDxxIlsHDybiyBHeePNNtnbqxKDfkwk29ycsdQWDXj0Or/6fS9oqaG9j/2t5YIJus+z4bnJd4pSGbEBZGbhLzjrRMMn7s240qFdYKTUOGAcQEhKC1WqtslxAQAB5eXlnPG85S91FRUWUOs7xKCrizBDqDxXr9rHbqe57zKr6cC6LFi1i0qRJLF68mB49etCvXz9uvfVWLr/8clJSUli9ejVr1qyht2NfnzfeeIPu3buzZMkS7rvvPnx8jEXhXl5e+Pr6AuDu7o6bm1v54+LiYvLz8wEYNWoUQ4YMwWaz8Y9//IM1a9bw2muvMX369PIyBQUF5OXlUVhYCEB+fj6ejg8Cp5fZs2cPnTp1olevXiilCAoK4tJLL63Ra1ETRUVF1b4vROOVn58vP1fRIOXn57PPasV8zTUccCK5VG0yFRfT8Z13aPP119jd3Ng6bx5F9Tj1/k2zmQ88PXHXmmkWC/tvvJEDv/1G0O03c8n0rmS6cMPvU0wl/wOVx65JQ11ed0NVUFBQ/ndfiIZG3p8udPPN1R6qq0Dud6B9hcftHM9VorVeDCwG6Nu3r65uY9CkpCQslirCtrOMkHk5bgA88ohxq0almn/55fzKnae7776bYcOGsXHjRhISEli7di0LFizghRde4JJLLsFkMhEbG4uHh7FxqcVi4dJLL2Xfvn1YLBaKi4sB8PHxKX8NPDw8cHd3r/Sa+Pn5ARATE4PFYiEvLw+LxUJ0dDR79+7FYrGUl/H19cVisZQHiX5+fuV1nV5m3LhxDB48mMsuu4zrrruOG264gSFDhmAy1U0CVC8vr/IgVzQdVqvV5RsBC+EKVquViPbtISSEdjfdVN/dOVNyMtx+O2zfDp6emF57jb7jx599hkotWpKWxgfJyQCYTCbc9/pzYGo+y/Qn/NtzJXFxrptOWVkD/NnUMqvVykD5vSkaKHl/1o262n5gNXCvI3tlPyBXa+3aaZWNiJeXF4MHD2b69OnEx8czevRoZs6cedZzVB38UT4VjOkKAXFpaWmlMn369OHAgQO8+OKL2O127rvvPgYPHozdbq/1/gkhRL0oLQXHl2sNykcfGdP/t2+HyEj48Ud48MF6C+KOlpTwxN695Y8vStUEP1vC1barGWIfQklJictG3u32EpKS7iE/v8pk2UII0Sy4avuBD4EEoItSKlUpNVop9aBS6kFHka+AfUAK8DYw0RXtNhVRUVGUlZURFhaG3W4nISGh/NiJEyf47bffiIqKqvZ8s9mMzWar8tiPP/5Yfl9rzc8//0y3bt2qLNuqVSsA0tP/iLG3bdt2RjmLxcKwYcNYtGgRX375Jd988w0pKSlnv0ghhGisSkoaXiD3yivGfnD5+caI3JYt0KtXvXXHpjUjk5LItdkwAZYC+Mc08C5w42fTz6w2rcZsNrtk5F1rTXLyBDIylrFz53C0rvrvnxBCNHUumVqptT7r7qLaGOJpBCvFa1d2djbDhw9n1KhR9OjRA4vFwubNm5k7dy6xsbH07NmTW265hfHjx7N48WICAwN57rnn8Pf3Z+TIkdXWGx4ezpo1a9i9ezctW7YkICCg/NiiRYu4+OKLiYiI4L333uPgwYNMmDChynoiIyNp3749M2fOZM6cORw4cIB//OMflcrMmzePNm3a0KtXLzw8PFi+fDn+/v60a9fONS+SEEI0NKWlYDbXdy8qu+02eOklmDGjXkfhTpmxfz/rjh+nlYcHb0d2xnb3AVocKMSnmw8DXx/I3zf9nZiYGJdMq0xNfZUjR/6FyeRNt27LZJsBIUSz1aCSnTR1fn5+9OvXj/nz55OSkkJxcTGhoaGMHDmSadOmAfDuu+/y2GOPcfPNN1NUVMSAAQNYu3Yt3t7e1dY7duxYrFYrffv2JT8/n2+//Zbw8HAA5syZw7x589i6dSthYWF8+umn1QZdHh4efPTRR0ycOJGePXvSq1cvZs+eXWmzcYvFwksvvcSePXtQStG7d2/WrFlTvr5OCCGanIYytfK77+DKK42gLSIC9u6FBpBMYO6hQ7xw6BAK+CgqCo/HdmBbbwMLXLL6Enwifeh/rWsySWZnf8XevU8C0LXre1gsl7mkXiGEaIwkkKtDnp6ezJ49m9mzZ1dbJigoiPfee6/a48HBwZXWsIExJfL0/eEOHDgAQJcuXYiPjy9PdlJReHj4GXX179//jOmUFcuMHTuWsWPHVts/IYRocup7amVBATz+OLz9Nrz2Gjz6qPF8AwjiPs7I4Ol9+wBwV4r0JTsIfd+GDRvTiqcxL2se0ZGuSW5SULCTnTvvBOyEh8/koouGu6ReIYRorCSQE0IIIc6mPkfktm0z1sLt2mXsD1fF/qP1pcBm45EK66PtWvNBwD56qd3s0rvYZNuE1Wp1yXRKm62I3367GZvtBK1aDScs7G9O1ymEEI2dBHJCCCHE2dTHGjmtYf58ePppY0Swe3f48EO49NK67Uc1tNaM272bjNJSFEbmNLPJxK1RkTzmOYGSkhKXJTcBcHPzomPH5/n99zfo2nUpStVV0m0hhGi4/p+9+w6PolofOP6drdmUTS8EQkAgSAsgSBeCDRUURbHiBQv2cu29XHv7qViv5WLv2As2FFREBQFJCL0FCAkppGfrzO+PSXYTsoFANpX38zzz7OzMmTNn03bfnHPeI4FcJxVo2KQQQoiD0No9ciUlcO65MH++/vzyy/UslfuYK93antu5k3d37ybMYODl1D6oj+Vx2G2pjOkWw6CfFvjWhgzmmnGJieeSkHC2BHFCCFFDAjkhhBBiX1p7jlxoKBQUQEwM/O9/cOqprXfvJlhcWsr1NevFze3bl6QL1qHOV/GsyILfxzN69OigBXA7d76I3T7Cl9REgjghhPCTv4hCCCHEvrTG0EqnE0pL9X2LBT78UF/ou50FcV8VFnLiqlV4NI3ru3WjywPrUeerVFPNFcuvqLcOanMVFHzChg1XsGLFBFyu/KDVK4QQnYUEckIIIcS+tPTQyqwsGDUK/vUvfW4cQM+e0M7W5/y1pISpWVmU1yz6PXWFBc/zHgAe4RHWetaycOHCoNyrrGwpa9bMAKBHj7uwWBKDUq8QQnQmEsgJIYQQ+9JSQytVVZ/7NmyYnp0yKwvy22/P0+2bN6PW7HfPAddFW1A0hXdM77DYuDhoyU0cjhyysk5BVatJSrqQlJSbm12nEEJ0RjJHTgghhNiXluiRRwEfNgAAIABJREFU27YNZs6ERYv057Nnw5NPQnh4cO8TJB/u3s1vZWUARFTA/XeCqVwj7vQ4LrjuAg775bCgJDfxeMrJzJyCy5VHVNRE0tJeRFGUYLwEIYTodCSQE0IIIfYl2HPk3noLrroKysogIUFPaDJlSvDqD7LsykouXLsWgLOMRkrzNxM+tCdhMTb6vdEPY5iRMWPHNPs+mqaxZs25VFZmYrP1ZcCAjzEYWnnZByGE6EBkaGUnoygK8+bNa+tmNJCRkcFVV13V1s0QQogDF+weuVWr9CDutNP04ZTtOIj7sbiYCStWUKmqHG8w8PmkSfxw6UWc+/0JuJ9wYwwzBu1eiqKQlHQBVmsK6elfYzZHB61uIYTojCSQa0WzZs1iSjt+ww7k3nvvRVEUFEXBYDCQnJzMeeedx/bt29u6aUII0TqCMUduzx7//gMPwEcfwccfQ3x88+ptQb+XlHDCqlUUejwowIgPNmNxmvF6vbjcLhYtXRT0e8bHT2PkyA3YbL2CXrcQQnQ2EsiJ/erbty+7du1ix44dfPDBB2RmZnLmmWe2dbOEEKJ1NKdHbs8emDUL0tP9wZzVCmecAe187tf927bhrdkfmAUZr/Tgee157AZ70BKbABQVfUtp6e++5waDNSj1CiFEZyeBXDuSnZ3N5MmTiYiIICEhgXPOOYe8vDzf+aVLl3L88ccTFxeH3W5n3Lhx+12z59FHHyUuLo4//vgDu93eYNjlDz/8gNlsJn8fmdJMJhNJSUkkJydz1FFHMXv2bP744w/Kaia+A3z55ZcMGzaMkJAQevbsyR133IHL5Wq0zh49evDEE0/UOybDL4UQ7dLBzpH78ksYMADeeENf4DuIa6y1tJ/27OG7msAzcTfcew8Y3ZB6eiq3PnArCxYsCMqi3+Xlf7N69RmsXHk0FRWrml2fEEIcSjpVspOFysJGz6W9lEbyJckA5L6cy/pL1zdaNkPL8O0vG7aMiuUV+y3XXLt27WL8+PFcdNFFPPHEE7jdbu644w6mTp3KkiVLMBgMlJeXc/755zNnzhwUReG5557jpJNOYuPGjcTGxtarT9M0brrpJt5//30WLVpE9+7dOeecc5g7dy5nnHGGr9zcuXOZMmUKiYlNW6MnLy+PTz75BKPRiNGoz4347rvvOO+885gzZw7jx48nJyeHyy67DKfT2SBYE0KIDudAh1YWFcG118I77+jPx4yBuXOhb9+WaV+QbXc4ODs7Gw042aVw7s0OYoqtRB0TRfp76RxlOioo96mu3sqqVZNR1UoSE88nLGxQUOoVQohDRacK5DqyF198kcGDB/Poo4/6jr355pvExMSwbNkyRowYwdFHH13vmmeffZaPP/6Y+fPnM2PGDN9xr9fLhRdeyOLFi1m8eDGpqamUl5cze/ZsRo0axc6dO+natSt79uzhs88+46OPPtpn29asWUN4eDiqqlJdXQ3ANddcQ1hYGAAPPvggN910ExdccAEAvXr14tFHH2XGjBk8/vjjkjpaCNGxHcjQym+/1YdS5ueDzQYPPQRXXw3G4CUFaUmLSkqYkZ1NgdvNkcCwkxaR5B1PrpKL5UYLBlNwBvK43XvIzDwRtzufqKij6dv3VXmvEEKIA9SpArmm9pAlX5Ls653bn+F/D29Gi5ru77//5pdffiE8wBpCmzZtYsSIEezevZu77rqLn3/+mfz8fLxeL9XV1eTk5NQrf+ONN2Iymfjzzz9JSEjwHR8+fDiDBg3ijTfe4Pbbb+fdd98lJiaGE088cZ9t69WrF9988w1Op5PPP/+cjz/+mIceeqhe2//66696QWht0JeXl0eXLl0O9ssihBBt70CGViqKHsQddZTeC9e7d8u2LYiWlJZy7MqVeAAFOPvJ3RzhHU8lldyl3MXFKy5mzAnNX2ZAVZ1kZZ1KVdVawsIGMnDgJ7LMgBBCHIROFch1ZKqqMnny5IBDEWuHPc6cOZP8/HyeeuopevTogdVq5ZhjjmkwF+24447jvffe45tvvmHWrFn1zl188cXMmTOH22+/nblz5zJz5kzfEMnGWCwWetd8GBkwYAAbNmzgyiuv5PXXX/e1/Z577mH69OkNro1vJCObwWBA07R6x9xu9z7bIYQQbWJfPXKaBitWwBFH6M8nTYIffoCjjwZDx5qG/uSOHXhq9g0ahNkSUfHysOFhdll3BS25ybp1sykt/QWLJZlBg77BZIoMSr1CCHGokUCunTjiiCP48MMPSU1NxdzIB4bffvuNZ555hsmTJwOQn5/Prl27GpQ76aSTmDZtGtOnT0dRFGbOnOk7d95553HTTTfx3HPPsXz5ct5///0Dbuudd95J3759ufrqqxk2bBhHHHEEa9eu9QV7TREfH1+v7Q6Hg7Vr1zJ06NADbo8QQrSoxubIbdsGV1wB8+fD4sVQm/zj2GNbt31BsLK8nC8KCwE9C5rFaCD95cGoJ2cxeftkHs94PCjJTQASEs5hz54fSU//hpCQlKDUKYQQhyIJ5FpZWVkZK1eurHcsKiqKK6+8kldeeYWzzjqLW265hfj4eDZv3syHH37I//3f/xEREUFaWhpvv/02I0eOpLKykptvvhlLI8N9pkyZwkcffeQL5k477TTfvaZPn84NN9zA+PHj6dOnzwG/hl69ejF16lTuuusuvvnmG+6++26mTJlCamoqZ555JiaTiaysLP766y8ee+yxgHUcffTRzJ07l1NOOYX4+HgefPBBPB5PwLJCCNGm9u6R83jg2WfhzjuhqgoiIyHAP9U6ij1uN9NWr8alaRxXCqHbt3PS0MMYHRkJM8YylrFBvV9s7ImMHLkZozEkqPUKIcShpmON++gEfv31V4YOHVpvu/HGG0lOTmbx4sUYDAZOOOEEBgwYwJVXXonVasVq1dfUmTt3LhUVFQwbNoyzzz6bCy+8kB49ejR6rylTpvDhhx9y6aWX8u677/qOX3TRRbhcLi666KKDfh033HAD8+fP5/fff2fSpEl8/fXX/Pzzz4wYMYIRI0bwyCOP0L1790avv+222zj66KOZOnUqxx9/POPGjZPeOCFE+6Oq4PWCqeb/nsuXw8iRcP31ehB35pmwZg1Mm9a27TxIqqYxY80atjgc9PfClNMzOetqE/93zIX7Xd7mQBQVfc2ePQt8zyWIE0KI5lP2nqfUXgwfPlxbtmxZwHNr1qyhX79+rdyijq28vJyIiAgAPvjgAy699FJyc3MJDQ1t45YdGPned04LFy4M2vwbIYJp0fffM2HKFH145XvvwYwZenCXkgIvvABTprR1E5tl9tq1vJqXR4TRyLO35pH6ezyFFHKl4Ur+/cC/ue2225p9j7KypaxcmYGmuRk2bCnh4YOD0HIhfzdFeyY/n8GjKMrfmqYFzL4oQysPIVVVVeTl5fHQQw8xe/bsDhfECSFEazN4vf5hlUcfDTExcP75cN99ECDLcEfyZE4Or+blAXDcp15Sf4/HhYt7DfdSbi0PyoewqqqNZGZOQVWrSEqaRVhYerPrFEIIoZOhlYeQxx57jL59+xITE8Ndd93V1s0RQoj2betWer34oj+QS0yEjRvhySc7fBC3pbqaO7ZuBWDICrj8Gf146F2hnPfAeSxYsKDZyU2czjxWrZqE272b6OjjSUt7WdaKE0KIIJIeuUPIvffey7333tvWzRBCiPbN6YQnnoAHHyS5uhrCwvznIjt+qvxqr5fTV6/Goap0zYV779UweRWM1yQw7r7+jGNcs+/h8ZSRmXkiDsdmIiKGM2DAPAyGJi6qLoQQokmkR04IIYSo9e23MHCgnpGyuprCsWM7fO9bXZqmccWGDayoqKArMPrOL4ksU/jTno1xeknQ7pOdfTYVFSux2frUrBUXEbS6hRBC6DpsINdek7SIliPfcyFEi8nJgdNPhxNP1IdP9usHP/3ExiuvhJDOk2Hx1s2beT0vD4uiMO2ff/g0Zw53czcPVdzGol8XBe0+3bvfQmhoP9LTv8diiQ9avUIIIfw6ZCBnNpuprq5u62aIVlZdXd3oYulCCNEsCxbAJ5/owygffxz++QcmTkTxeKCR9To7mv/l5vLY9u0AWB0aA4cPx2Kx8Lvxd9xWd1AzzEVFTeDIIzOx2XoErU4hhBD1dcg5cgkJCezcuZOuXbtis9lk8nQnp2ka1dXV7Ny5k8TExLZujhCiM9A0WLtW73kDmDkTtmyBSy6Bbt18xQweT/3FwDuoApeL6zdtAmDiT3Dpy5D3v3gWLFjgSxPe3OQm27Y9QljYAOLiTgZAUYzNbrcQQojGdchAzm63A5Cbm4vb7W7j1nQMDoeDkA48PMhsNpOYmOj73gshxEFbsQKuuw7++APWrYPUVDAY9CUF9qJ0gkDOo6qcnZ1NmddL2nq4+TGNEKdCcpbC6FtGNzuAA8jNfYktW25DUSyMHLmRkJCUILRcCCHEvnTIQA70YE4+1DfdwoULGTp0aFs3Qwgh2k5enp7EZO5cvUcuLg7Wr9cDuUZ0hqGVd23dyk8lJfQohgeuKSXEGcm3MX9yylFHAX2aXX9BwSesX38FAH36PCNBnBBCtJIOOUdOCCGEaDKHAx55BPr0gf/9D0wmuOEG2LABjjtun5d29B65TwsKeCQnhxAXPHZjFfHOSDLJZE7JvSxa1PzkJnv2LCQ7+1xApUeP/5CcfGnzGy2EEKJJOmyPnBBCCNEkl18Or7+u759yir5GXJ+m9UR15DlyH+Tn86+1a0GDt16NIG4L7FZ28x/lPyhWpdnJTcrK/iIr62Q0zUly8hWkpt4VnIYLIYRoEgnkhBBCdD6Vlf6FvG+8EVau1LNRHnvsAVXTUXvkFhQXc86aNWjAwDUQ91E5BpuBri905YZdNzQ7uYmquli9ejpebwUJCefSp8+zknhMCCFamQRyQgghOo9//oFbb9WHU/70EygKDBgAy5fr+weoI86R0zSN6zZtonblzTX9Yd0LCZwWH0fCGQmMZWyz72EwWOjf/31yc/9L376voigyU0MIIVqbBHJCCCE6vq1b4e674e239UQmdjts2wY9eujnD7K3qCMOrXxm504yKysBUDQNk6KQfm5XEiIjm123qnowGPSPDpGRo4mMbH7GSyGEEAdH/oUmhBCi4yoqguuvh7594a239EQm//43bNrkD+KaoaMNrfy1pIQbN20irAKePGcXhz/2Jdr110N2drPrdrkK+PvvYezePS8ILRVCCNFcEsgJIYTomNxuGDwYnnpK358xQ18X7qmn9KUFgqAjDa38srCQkzIzUT0aT93hYGheF67+tg/ef1axcOHCZtXtdpewatUkKitXkZPzEKrqCU6jhRBCHDQZWimEEKLjKC8Hq1UPrsxmmDkTli2DRx+FIUOCfjuD19sheuR+LSnh1KwsVODyl6HPqhBKKeUBwwNYrJZmZaj0eivJzJxMRcUKbLY+pKfP9w2vFEII0XakR04IIUT7V1WlZ5087DB47TX/8fvug+++a5EgDkBxuztEIHfr5s2owKRv4cwPQTVB3PNxXPXAVSxYsOCgM1SqqpOsrNMoK/sdqzWFwYN/xGJJDG7jhRBCHBT5l5oQQoj2q7oaXnpJX9A7P18/9tNPcGnNwtNGY4veviPMkXsvP5/fy8rolw3XP6kBCpYnUhh7Ra9mZahUVTfZ2WezZ88PmM0JDB78IyEh3YPXcCGEEM0SlB45RVFOUBRlnaIoGxVFuTXA+VmKohQoirKyZrs4GPcVQgjRSVVVwTPPQO/ecN11ehB35JHw7bfw/vut1gyD19uu58hlVVRw8bp1WB1w383VWNwKX8T9hnHE7mbXXV29gT17fsJkimLw4B8IDU0LQouFEEIES7N75BRFMQLPA8cBO4CliqJ8oWna3imyPtA07arm3k8IIcQh4LPP4Npr9f0hQ/QhlFOmHPQyAgdLcbvBZmvVezbVD8XFnJOdTZWqMrS8gKer53ACJ/B88f3ELzQ2a8FvgLCw/gwe/D1gIDw8PTiNFkIIETTBGFo5AtioadpmAEVR3gemAs3PdSyEEOLQUFQEf/wBkyfrz888Ez75RM9EecopYGibKd1KO012srikhBNWrUIFFODCLl242bqcP1x/YLEcfHITTfNSVvaXb304u31k0NoshBAiuIIRyHUFttd5vgMI9Jf/dEVRxgPrges0TdseoIwQQohDya5d8H//B//9L3i9+sLeiYn6enDz2n69sva6IPj927ahAqd9Apt7Q/nUFBYsWMDChQvJyMg4qN44TfOydu2F5Oe/Q//+75CQcFbwGy6EECJoWivZyZfAe5qmORVFuRR4Azh670KKolwCXAKQmJjY7HVvhF9FRYV8PUW7JT+fhx7bjh10++gjusyfj8HtBqBoxAg2/vAD1d26tXHr/LpWVbF5505y2tHP59/A98CYxXDVcxoeE2zpsgVnPIwePRqn03kQv08q8AQwHwghOzuf7OwDrUO0Jvm7Kdoz+flsHcEI5HYCKXWed6s55qNpWlGdp68CjwWqSNO0l4GXAYYPH641Z90bUV/tf2mFaI/k5/MQ4nbDWWfpc+A0TT82bRrcfjuxw4YR27atayDnv/+le1oah7WTn88ch4Ppf/9N6no3d9znwaCZeDvuay7vNu2g58Rpmsr69Zexa9d8DAYbgwZ9Q3R0RnAbLoJO/m6K9kx+PltHMCYdLAX6KIrSU1EUC3A28EXdAoqidKnz9BRgTRDuK4QQoiPwev37ZrOekdJshosugtWr4eOPYdiwtmvfPrSnoZVOVeWM1atxFbp57DYvoS4TC1jAu/lPHfR/vjVNY8OGq9i16xUMhhAGDfpSgjghhOggmt0jp2maR1GUq4DvACMwV9O01Yqi3Acs0zTtC+AaRVFOATxAMTCrufcVQgjRzlVW6ot3P/UUvPUWjBmjH3/mGbDbISmpbdvXBIrH026WHzhr9WqW7ylnzn0K8flGNigbeFJ5Eov14JObbN58G7m5L6IoVgYO/Jzo6GOC22ghhBAtJihz5DRN+wb4Zq9jd9fZvw24LRj3EkII0c5t3QovvgivvgrFxfqxuoFcWsdZj6y9LAh+x+bNfF5UxNUvwIDlGiSaGPDSAO7MvvOgk5sAxMdPIy/vdQ4//DViYo4PcquFEEK0pNZKdiKEEKKz++UXPQPll1/657+NGgU33QRTp7Zt2w5Sexhauby8nEdzcgBYMhoyFsLOF+O5empfxkwd06y67fYRjBq1CaMxLAgtFUII0ZraZmEeIYQQnc/XX8MXX+hLB8yYoa8Lt2SJnszEaGzr1h2Utu6RK3K7OX31arwAXi/Lhno45zUH1oTCg6pPVT2sWTOT/Px3fMckiBNCiI5JeuSEEEIcuLVr4YUXYOhQuOAC/dgVV0BEBMyera8F1wm05Rw5r6ZxXnY2ji0Ojt9czQ/zbkfr3x/PqlUUnXsujB17QPWpqos1a86joGAeRUVfEBNzEmZzdAu1XgghREuTQE4IIUTTVFfri3S/8gr8+qt+7PDDYdYsUBRITYU772zTJgZbWw2tXFJayp1btrAkt4QX74Tu22y4jFH8mvkBFsuBJzdRVSerV0+nqOhLjEY7gwZ9I0GcEEJ0cBLICSGE2LcNG+C55+DNN6GkRD8WFgbnngtXXqkHcZ1UWwytXFJaSsbKlbi9Gv95GFI3g62vjQefeZBFfy864OQmXm8VWVnT2LPnO0ymaAYP/oGIiPa53IMQQoimk0BOCCHEvi1fri8ZADB8OFxyCZx9tj6MspNTvN5WH1r5cUEBLk3jgtfhqN/AE2Fg0BeDCE0LZczxB5bcxOOpICvrFEpKfsZsjmfw4B8JD09vmYYLIYRoVRLICSGE0Kkq/PYbvPEGhIfDnDn68VNPheuug/PP1+fEHUIMbner9shVeb18sGMHGT/Dv94Cr0Ej716F0LTQg6rP6dxGRcUKLJYuDB68gLCwfkFusRBCiLYigZwQQhzqNm3S13l7803YskU/FhoKDz2kD6G0WuHJJ9u2jW2kNYdWaprGZevXE7IBbnlEAxReSviUoc4uwFEHVWdY2ADS07/HZIoiNLRPUNsrhBCibUkgJ4QQh6qlS+H66/VeuFopKXrP2/nn60HcIU7xelstkPtvbi5v5eczzAFObyULld/4quRVrstYcED1VFdvpazsDxITzwbAbj+yJZorhBCijUkgJ4QQhwqHA7Zvhz41PTMREXoQFxoKp58OM2fCxIlgkCVGa7XW8gOv5OZy1YYNANxwZj/iu20iKiuKBccuOKDEJhUVWaxadTwuVz4mUxSxsSe0VJOFEEK0MQnkhBCiM3O74ccf4f334dNPoVcvWLFCP3f44fDZZ3D00YdE4pKD0Rpz5L4uKuKSdevptQm29oYeISGMPm0MY047sMQmpaWLycycgsdTQmTkBCIjmx4ACiGE6HgkkBNCiM7G69XXeXv/fX3dt6Ii/zmjEcrKwG7Xn0+d2jZt7CBaemilR1W5JDOT0z+GK16AFy7z8qZxFaOPOrA5cUVFX7N69XRUtZq4uFPp1+89jMaQFmq1EEKI9kACOSGE6Gy++krPNFmrf3845xw46yz/sErRJC09tPKOLVtIXgqXv6hh0BSKorzEr1wJBxDI5eW9xdq1FwBekpIuIi3tvxgM8vYuhBCdnfylF0KIjqq6Gn74AT75RE9M8vzz+vHjjoOBA+GUU/T13gYO7NSLdrckQwtmrfy4oIB3f9/OC/eBUVV4a+AKfv/2LR6q/T42gcdTxqZNNwJeune/lZ49H0KR77UQQhwSJJATQoiOpKQEvv1WD96++QYqK/Xjdjs89ZTeexQaCpmZbdvOTqKllh94Lz+fK5atYc4dEFEBhqMM9J7kYOHRzx9QchOTyc6gQV9TVraYbt2uDXo7hRBCtF8SyAkhREfxzjswaxZ4PP5jw4fDtGlw2mmtkl3xUNMSgdyPxcWcn7WGB+6H7ttB6RfCmK+HMz5ifJOuV1UnxcXfExd3MgB2+3Ds9uFBbaMQQoj2TwI5IYRob1wufVmAr76CtDS47DL9+ODBoGkwYYIeuJ16KqSmtm1bOzlDkOfIaZrGpZmZJORD2nootWv8eo+LCRFNezt2u4vIyjqV0tLf6N//QxISpgetbUIIIToWCeSEEKI92LYNvvsOvv9en/dWVqYfP/JIfyA3YAAUFEB0dNu18xAT7B65p3bsYLOmQReNS19Qid/lYcLu9cD+e+OqqjaQmTmZ6uoNWCxdCQ1NC1q7hBBCdDwSyAkhRFu75RZ47LH6xwYMgClT9K2WokgQ15o0LajJThaVlHDvqk0QBqbXXqNI0yhbvZr/NSG5SUnJb2RlTcXjKSY8fAiDBn2F1do1KO0SQgjRMUkgJ4QQrcHjgb//hp9/1nverroKTj9dPzd0qJ6s5Jhj4PjjYdIk6NmzbdsrwOtFMxhQDIZmV/VFYSFXL1rN/y6Dkhl2Blx2GYsWLSLjiiv2m9wkP/891q6dhaa5iImZTP/+72EyyQLuQghxqJNATgghWsqKFbBggR68/forlJf7z/Xs6Q/kpk3T91tw4WlxEFwuVJMJYzOr+aWkhDP/zuLpOyC+EOKXehn12CjGjBmz32u93ko2b74ZTXORnHwlvXs/LWvECSGEACSQE0KI4PB69ZT/gwaBseaj/+WXw59/+sv06QMTJ8Kxx+q9b7Uk22T75HajmZr/NnnlypXc+Dgcvg5ykzT+vKWaDHPTevmMxjAGDvyS0tJf6dr1KlkjTgghhI8EckIIcTCqquCvv/Tskr/9Br//rve4/fMPpKfrZc46S5/rNnEiZGRAt25t2mRxgIIQyL2Tn8+gd+HYBVBl07jjHhcnbN13chOXK5/Cws9JTr4EgIiIIUREDGlWO4QQQnQ+EsgJIcSByMvT0/7//Xf99dxAHy65e7f/+XXXtW7bRHC53ajNCORWVVTwyty13P2q/vyhET+S+/J8/rWP5CZlZUtZvXoaTucOTKZoWV5ACCFEoySQE0KIvZWXw7Jleo/bX3/pxz7+WH+Mi4PsbFBVPUnJuHH6NnYsdJUsgp2Ky3XQPXIlbjdnrMri9pc0DBoYLzEyuUc4j2c832hyk7y8N1m37hI0zYndPobIyKOa03ohhBCdnARyQggB+vDIuXP1wC07W194u5bVCm63nozEZIKfftIX6rbb2669ouUd5NBKVdOYkpnJBpeD514I4b2/Euh9T0+OUgIHZqrqZtOmm9i5cw4AycmX0bv3HAwGmTsphBCicRLICSEOHQ4HrF4NK1fq25Qpeqp/gM2b4bXX9H2zGYYMgREj/FvdD/TDh7d+20Xrc7vRjAees/LydetYXFYGmkZmqIPC62Lp00iSEperkOzsMykp+RlFMdOnz3O+uXFCCCHEvkggJ4To3J5/Hv74Qw/c1qzRs0vWUhR/IDdxIjzzjB60DRmi98KJQ5vLhXqAS0J8X1yM9fY8LrZp/O8iBZfXw5urVjH6qMC9cYpiwOHYisWSxIABHxMZuf8lCYQQQgiQQE4I0dG5XLBxo97T9s8/kJWlz2er7Ul5+209kAMwGKBfPz1QGzIEJkzw15OSAldf3frtF+3XAfbIba2uZu4jWVz2KbjM8GOGh60pHv0fCHsFcpqmoigGzOYYBg36CpMpEqtV5lgKIYRoOgnkhBAdg6bpPWigr9f2n//oc9k2bGiYPXLDBjj8cH3/2mvhwgv1wG3AAAgNbd12i47rAObIObxebn5rFZc8qQLwTPIHbPulEsvq1fWyVHq91WzceA2KYiEtTT8eFtY/+G0XQgjR6UkgJ4RoX0pL9UBs7Vo9UKvdJk+Gp57Sy6iqP4ukokCvXnpPW3q6HrB16eKv7+yzW/81iM7B7W7S0MolpaXc/et6rry5GpMX4v6dzB1nzmTswoVkXHGFL0tlZeVasrOnU1mZhaJYSUm5CZutRwu/CCGEEJ2VBHJCiNZXVqYPh9ywQV+CHWovAAAgAElEQVSTrdapp8Lnnwe+JjPTv9+3L7zzDvTvr+/bbC3bXnFocrn2O7RySWkpx/++gqevhahSqM4Ipf/jvTGY0uotM5CX9ybr11+OqlZhs6UxYMCHEsQJIYRoFgnkhBAtKzcXXn9dD9pqt7qLZq9e7d+Pj9eDst699fT+AwbowVr//vrzWiEhcO65rfYSxCGqCUMrn/jnHy54WaPXZoWcbhrLbnJwosngO+/1VrJhw1Xk5b0OQELCeaSlvYjJFNGSLRdCCHEIkEBOCHFwCgv1lP1bt+rbli3+/eOO0zNAAhQXwx131L82JEQfDtmnT/3jc+bAyy/758IJ0ZbcbtR9BHKFLhffOxwYZllI2K3x8gUuJm9aD4z3ldmy5R7y8l7HYLDRp8+zJCVdiCI/30IIIYJAAjkhRH2aBiUlsGNHw+2pp/yLYJ97LvzwQ+A6unXz7/fsCTfcoAdtffrovW3duukZJGvV9tBJIhLRnrhcjfbIeTWN89asocJigV3buLv3T5hfyKyX2ASgR4+7qKpay2GHPUJ4+MDWaLUQQohDhARyQhxKqqshLw927fI/9u0Lxx6rn//1VzjhBKiqCnz9ddfBwJoPo+npeq9cjx7+rWdP/TE11X9NWBg88UTLvSYhWkojQyuXlJbyzLcb6PFZBQmXmXipTx/W5OaScdllDBvWm40br6dnzwcxGm2YTJGkp3/VBo0XQgjR2UkgJ0RH53TqAVVBgb7l5UF+vt4LVjuE69RTYeFCPSPk3i64wB/IxcbqQVxEhL6uWrdu9bfERP91EpyJzi7A0MolpaVMXbCCZ6+BxN2Q0SeSU+8bxKljxlBY+AVLl56K270bRbHQq9cjbdRwIYQQhwIJ5IRobyoq/EFZQUH9IO244/xB16efwqxZegbIQC6+GKKi9P3qaj2IM5shKUlPz1/7OG6c/5q0NL1c7fBJIQ5lAXrknv/7H+66RyNxt8LqfhrZ4/cw3VPGxo3/Ji/vNQAiIyeQnHxZW7RYCCHEIUQCOSFaQu0C1bUfAtetg+XL9cQfe/bU34xGmDfPf+1hh+lBWyAWiz+Qs9n0IM5ohLg4PeNjfLw/SFNV/3WvvQZWK8TE7DuRiMkkQZwQtfaaI1fh8dDjUReDskwUxGncfaeTq3K+ZunS83E6czAYQujZ82G6dbsGRTHso2IhhBCi+SSQE2JvDoee7KOsrP5WXg6RkXDKKXq5qiq49FL/+ZISf6BWXg4ffgjTp+tlP/sMbr018P32TvDRvbue1bE2MKvd4uJgvD8bHhkZUFSk97oZ9vOhMTn5oL4UQhzS6gyt1DSNZ+5ZybHfm3CaNe4c/TXJH3zJxOvW43RCRMRwDj/8TcLC+rVxo4UQQhwqJJATHVtFhR40VVVBZWX9LSUFjjhCL7d9O7z4on68qkq/pm6Q9sknejZFgNmz4e23A99v1Ch/IGcyNV5OUfS21Ro0CM48E6Kj9V6x6Oj6m6b5e8qWLWvaaw8J0TchRMtwu30Lgr82byMjH9F/pyvvNDHLHMuECa8RGfk0YWED6N79dgwGc1u2VgghxCFGAjkRHJoGLpfem+Vw6HOyQM9gWHv+22/95+qWczhg0iQYPlwvu3AhvPKK/1zd4Ky6Wl+vrDboOeooWLkycJtmz9bXJAM9vf3DDzfe/uJi/37tMEW7veHWt6+/nMUCb76pJwaJiNB7xmqDNLu9fi/ZSSfpmxCi43C50Mxmbvn1V160uLn3CJVBQ9Zw/PUTmBZ+GwCaNlrWhRNCCNEmghLIKYpyAjAHMAKvapr2yF7nrcCbwDCgCDhL07Stwbh3p6Vp+jyrupvbrT/WBg6gD+PLydHPuVwNt1NP9QcU8+bpPVMul57psG654cNhxgy93LZtepr5QPW5XHrwMmSIXvbGG+GFF/SAS9Pqv4b0dPjnH31fUfSerNq5Y3uLjPQHclu3wrvvNv61cTr9PVHx8ZCQoKe4DwvThynW7qen+69JSYH77/efs9v1r2FtgFZ3YeqnntK3pjj//KaVE0J0PG43r9uieMzjZkLkL3R/+HnMhiI2bBjH0KG/AEgQJ4QQos00O5BTFMUIPA8cB+wAliqK8oWmadl1il0E7NE0rbeiKGcDjwJn7bNilwv++ktP2OD1+jdV1ZM8jBnjL/vll3r52vN1yx55JAwYoJdbtw6++67++br7t96q97KAPgxv06bAZUeM0DMCgr5I8o03+oOsvQOvZ5+FwYP1so88Am+80TAw83igVy/480//awoN1YOjQJ56Cv79b/9rnzmz8a9jVZX+9QJ4+mlYvDhwufPO8wdylZV6RsTG1M2SqKr+3jezWQ+wbDb9MSmp/nVTpuiPtedrH0NCYNgwf7nx4/Uhi7Xn6gZnYWH+7xHA99833s66EhLgzjubVlYIcchzF7nZOK8PqRVH8PC5dzDK+AeYYIejN8P7PL//CoQQQogWFoweuRHARk3TNgMoivI+MBWoG8hNBe6t2Z8HPKcoiqJpe3fh1JGfDyNHBj6XlqYHZbXOPrvxBYyfeMIfyC1bBtde2/gruf56f5Dw/vvwyy+By1VU+AO5ykr44IPG69yzx79fUABr1wYuFxNT/7nJVH8zm/37tYEZ6MMA09P141ar3v66W90v8fTpemC7dxmLBfrVmaCfkqL33gWqz2rVsyrWeuABvacrJETPnrgv+woO6zrssPr3EEKIFlaxqgLPHg+eUg+OrQ62PbANd0EXulucpG7YQ0XfMF72XMiU8OmEhw9q6+YKIYQQKPuKpZpUgaKcAZygadrFNc/PB0ZqmnZVnTJZNWV21DzfVFOmcK+6LgEuAegdETHs7+RkMBjQarbafWdCAmvrZADsd//9GNxuvYyi6OWNRjAY2D1xIsU1AWHEunUkfvddwDo1g4GcGTNQawK5hAULsBYWotXUh9Ho26/u1o2SmiQaxqoqYv/4A81oRDMaUWsea7fK3r3xhIcDYCkqwlRRUe+8ZjL5rvPWlAPqJ78IgoqKCsLr1i9EOyI/n+KguIBKoKLOY2+gZvlE/qzZ6p6v3bcDL9epa0rN8TrUwf9guPFxlib2Yv66Yxga2o2T5Z9Mop2Qv5uiPZOfz+CZOHHi35qmDQ90rl0FcnUNHz5cW9bU7H1ivxYuXEhGRkZbN0OIgOTn89BT+95TO8esIqsCx1YH3lIvnhK9Z6z2MTQtlJQbUgBw7nKybOgyvKVeVIfaoN6BXw4kbkocAFv/s5Wt924NeH9znJmxBWN9z/854R885Q7M0SGUuEtZmLSD32etosobyUfjbyCx7pBuIdoB+bsp2jP5+QweRVEaDeSCMbRyJ5BS53m3mmOByuxQFMUERKInPWmUpqm43XswGkNRFItMKBdCiHZEdat4Sj0NAq+QniFEDNWTMVWuriTn8Rw8JQ3LeUo9jNo8ipBUPXHR1nu2UvhJ4P/tRU2M8gVyxlAj7nw3AIpJwRRlwhRlwhhp1Pcj/G9rMSfEYIo2YYo0NSwX6S/n9TqIemk+OTkPoiiXMemkl3C88AIoJ3D+n4tJPFaCOCGEEO1PMAK5pUAfRVF6ogdsZwPn7lXmC2AmsAQ4A/hpn/PjAJcrl8WLa+eNKRgMNgwGG0ajDZstjSFDFvjKrl59NuCtKROK0WjzlY+JOR67XR9a6XBsp6JiRb26DIYwjMZwjMZwTKZICRiFEJ2epmmoVSoYwRiiz22t3lRN2V9lvkCrbuClOlUGfjzQd/3SwUupXFUZsO6u13b1BXLuPW7y38hvtB2eUn8W24jhEajVqi/Iqht42Xr65wUb7UZG7xyNKcqEwWbY599s+0g79pH2fX4dCgrmsXnzrTgcmwHYvftnHLNnQ7dusGsX0T/8AHfc0WgdQgghRFtpdiCnaZpHUZSrgO/Qlx+Yq2naakVR7gOWaZr2BfA/4C1FUTYCxejB3n4oGI2RqGo1muZCVatQ1So8HjAa678xFxZ+hqY5A9ZiNEb4ArmSkkWsXdt4uvhx48owmfQPIKtXn0VFxT81QV5YvcfIyKPo0uUCANzuPRQUzGtQrjZAtFq7YDBY9/9yhRCiiTSvhqfMU6+HqzbwMoQaSJieAIDqUck+KztgOc2j0fd/felyYRcAir8rZsOVG/Z5T8WoB00GiwEM1Au4avfDBob5rgntG0rfuX0DljPajRjM/rUWU29Lhdv2/9oVRcGa3Py/qSUlv7Jp002Ul+vZgkND+9O799PMzbbBFL3Hj9hYYupm1BVCCCHakaCsI6dp2jfAN3sdu7vOvgOYfiB1Wq1dOeqoZTXXe/F6q1HValS1ir078wYMmFcT6FXXlPPv2+0j6tTZhdjYk/F6q3x1eb2Vvs1oDPWVdTg2U129jsAUXyDndOawfv0ljb6OoUMXExmpL5WwefMd7N79HkajHZMpoubRjtFox2brTffuN/muKyj4tCYwjPCV0R/D0Vd8EEJ0RJqmoTr0YYmKomBJ1IftuYvdFMwr8A89rDsMscRD2n/TCB+oTxzfcNUGcv+bG7D+sIFhvkBOMSoUfVWE5mo4AMIQYkB1+eeYhfYLJf7M+AY9YrX7dQ35ZQiGkH33hgFY4i10uaBL0784raS4+HtWrZoEgNmcSM+e/yEp6SIMBhOfhy73LbFiMBopqZvRVwghhGhHghLItTRFMWIyhQOBs9/ExU1pUj3R0ccQHX1Mk8oOHPg5Hk9JTZBXgddbiarq+zZbb185o9FOUtJFeL0VvvN1rzGZIn1lXa5cHI4tAe8XEXGkL5DTNC+rV09rtG1paS+TnDwbgKKir9m58wVMpmhMpijfZjbrz+Pipvk+bHk85TVzDiUQFKI5vA4v7nx3gzlftUFX91u6671WwMbrNlL6W2m9crWBVcK5CfR/pz8A7gI36y9d3+g9XXkuqBndaIqumesVaWoQeIX0DPFdoygKAz4agMFmqB+YRZowWA316o+eGE30xOgmvX6jreP9DXG7SzCb9XSW0dHHYLePJiZmEt263cDSpZm88cbjpE+YwFK33htnBCyqyrDy8jZstRBCCNG4DhHItQWrNRmrNXm/5Wy2nhx++KtNqrN376dJTb0Lj6cMr7fM9+j1lmMyRfnKqaqLuLjT65XRH8vxessxGv1Dl6qq1lJc/E2g26EoViZM8C8qvmLFGCors2p696LqbfHx00hK0hcWd7kKKCn5CZMpFrPZvxkMoTKHUHR4qlPvDfOUelAdKuGD/P8g2vHcjkYDtOTLk+l6WVcAir4qInt6dmO3IPmSZF9PW9X6KsqX1Q8GFIuepMMY6g+IzAlmki5KarRHLCzd/3t/2EOHcdhDTUuDH3dKXJPKdVYORw7btj3A7t3vceSRawgJ6YaiGBk69DcUxcCSJUs45phjcLlcKJdcgufMMxlrtzM5NpaMBx8kPDW1rV+CEEIIEZAEcq3IZIqs10PXGKPRxsCB8wKe0zQV8A+Tios7HZstDY+npM62B4+npKbs3tcqNcFjGU5nju9cWFh/335FxT9kZzecxqgoVszmGIYOXYzN1hOAXbvmUl29oUHQZzLFYLEkYjbHNKhHiOZSPSpVa6sazYaYOCPRF6DlvpzL9ie3+8rVTVlvTjAzNt+fgn77o9tx7gg839a5zX/cHGfGmmL1z/eqzYRYE3gpFv8/PA57+DB63N2jXpnaBCN1maPNHP7q4c3+2ghddfVmcnIeJi/vDTTNDRgoKfmJpKR/AaAoeo/kwoULcblceMPCYIo+uuPJ3r0ZYbdDTg6revVqq5cghBBC7JMEch1M7YePWjZbD2y2Hk26dsSI1WiaisdT1iDos9n8H1b0IZmn4/EU4XYX4XYX4/EUoaoOXK5dGI3+HoyCgnkUF88PeL+YmBNJT9d7C93uPWRlnYrFkoDZnLDXYzxhYem+YU+i81I9aoPAyz7Kn7xo9we7KV1SGjBACx8a7suc6C33smxQ4+tMhg8O9wVy3gov1euqfedqU9YbI41YEuqnle96dVdUhxowXb21qz/BRnRGNKNzRjfpNYeny4Koramyci05OQ+Rn/8u4AUMJCScQ48e9xAa2rdB+YyMDCwWC47p09FCQxmpKHoQB+B2o5nkbVIIIUT7JO9QhxhFMWA2R+0zaLLbhwfsEfR6q3C7izCbY33HunSZTWTk2JqAT99qA8CQkB6+ci5XHqWlvzR6z/T074mJOQ6AnJzHyM9/B7M5vkHAFxLSnZiYSQfxykVzaZpWb2ht+YpyXHmugOnq7WPsJM1IAqBsWRlZU7P0oYyVDRdwHp7pX+Oy+Nti8l7PC3h/c6zZt2+ymwjtHxpwjpgx0kj4YH/wlDgjUV9PrKacIbTxJB3db+5+YF8U0e5s2XI7hYWfAkaSkmbRvftthIamNVp+9OjRfPrjj0ytqsIJPDlkiP+ky4UqgZwQQoh2St6hRJMZjaH1MnsCxMefBpy232tDQrozePACXK7duN27cbsLfPsuVwFWazdf2erqDVRWrgpYT1jYIF8gp2kaS5Z0w2SKxGLpgsWS5Hu0Wrtgt49tcm/locCXsn6vwCt0QCihvfXva8lvJeS9lhcwXb2n3MP4qvEoBj0IWnfxOiqWVwS8l+pUfYGcwWzAlevST9RNWV/zWJvSHiD+zHjC0sMCBmimaP+fK8WoMGL1CJrCkmBp0PMmOgdN81JY+DlWazdfhuKUlJsxmxPo3v0W3xDwupYsWcLChQvJyMhg9Gi9V/WP5GScW7dyTFQUYyLrDH+XHjkhhBDtmLxDiVZhNIYRHX10k8r27PkQyclX1Av2avctFn8qc6+3DJcrF5crl6qqNQ3q6dv3NWy2WQDk5r7Ktm33Y7F0wWqtH/RZLF2IjZ3S7hO5eB1eFJOCwaQPr63MrqQyq7JBUg5vqRdTlIk+z/YB9IB3cfxiPEWegPX2fro3odfqgZxjq4O8uYF7xEAfpmiy63827KPtmOPMAZNzhA2qs5ZYv1BG5YzSe8vCjYG/zjVrRseeGEvsibENzwtRh9tdzK5dr7Jz5ws4nduIjp7E4MHfAhAZOYrIyFEBr6ub2MRisbBgwQIGHHkkT2zfDsC0+Pi9bySBnBBCiHZL3qFEu2OxxGOxxO+3nNFoZ8yYAlyuPFyuXXUe9f26CVyczhzftnc2cZMpmnHjin3PV66ciKo6sVq7YrF0xWrthtXaFau1K6Ghh2OxJBzwa9JUDW+FN+D6YJYkCzHH6UlhnHlONl6zsUEZT4mesn7wgsFEH62niM97PY/tj28PeD9rd6svkKsbOBntxgbZEK3d/XO/IkdHkvZKWuAesb1S1qc91/hwtboMFgMhKSH7LyjEfpSXLyc390Xy899BVfV5jyEhvYiNndxg6G8gvsQmXi8ul4uFCxcyNzqaCq8XgBs3bWJoeDija3vl3G5UY8dbakEIIcShQQI50WEpioLFEofFEodvga1GdO9+G0lJM3G58nA66wd9iuKfe6VpGmVlf6GqVfUrcJugMoyUw26hV/ptAOzOWsTWefMxVsdjqIpDqYqEinCosKGWWxj46WAssfqQvswpmRTPLyaQ2JNjfYEcGhR8VBD49ZoVvFVe3/PwoeHEnR4XMOAyx5vrXTtq8yiMYcZ6wxgDsfWyYetl22cZIdrCrl1zWbfuIt/z6OhJdOt2DTExJzRIAtWY2sQmtT1yR0yYwH/y833nXarKwpISfyDncqGZzY3UJoQQQrQtCeTEocFtQSnqirEkEWvpQIwlHsylHqwlHjS3BnWS2UW/+DPVG6vwlHrwlqio5QY0h/6rUn1tFTytlyv5J4eq/5wQ6GaAG1dhBZZYPUBzmTeihNkx2sEUacQUbcUSHYopykLE8AjfleZYM/3f7+9PVV8nQDOE1E/SkXhOIonnJDbp5dcOhxSiI9A0L3v2/IzHU0JCwhkAxMZOwWxOJCHhbJKTLyMsrGlLNew9J27BggW+518lJuLMycEAKIDFYCAjqk4iKBlaKYQQoh2TdyjR7vlS1pd60FTNl5hDdansfH5nwKyJnhIPPe/vSexkfb7Vzud2svmmzQHrN9gMdLtWT7aiKArVq6AqG+r+eigmBWOkkbDIfr5j8eljqTpnI4SXo4WWoIYW4rXl4wnZhRZajC3Fv1C757Zb0a7bgAeoO1PNbI4ntOs1wJ16OQph4iJM1hRCQrpjsSQ1ubdBiI5M0zQqKlaQn/8Ou3e/h8u1C6s1hfj4aSiKAYslgdGjd2AwNP1tK9CcuNGjRzN69Gh2u1zM+eMPAP6blkah201GVJS/Nw5kaKUQQoh2TQI50SpUp0r15urA2RBLPXS9qivWZH2u1tb7t7L7/d2+cnVT1keMiGDYn8P0Jwpsun5To/esu7CzJd6CJdnSaLp6TdV82Rj7vtoXNOqVC5SyPnrQYUS/e1iTXn9q6m1UV2/E6czB4dheM19vO253/WGUZWV/kZ19pu+5ophr5ud1JySkO717P+1b/sHp3InRGIHJZEeIjsrpzCUv7zXy89+mqmqt73hISC8SE89DVZ0Yjfpw3wMJ4iDwnLjaTJWP5ORQqapMiY1ldnJy4ApkaKUQQoh2TAI5sU+aWpOyvk7gFTk+0hfU5L6SS9XaqoABWuzUWHo/0RvQMyz+fcTfjd4n9uRYXyDnLnBTlV1njppBHxpoijL5yoCe1r7bDd0whhkDBmghPf0JNpJmJpE0M6lJrzlydOT+Cx2gLl0uaHBM07y4XPkoij81vr4Y+2k4HHpiFre7AIdjKw7HVkpLIS3tJV/ZNWvOp6TkZ4zGSEJCuhMSkuoL+Oz2UURFTQj66xCiuTRNw+stw2TSf8+qqtayZYveI202x5OQcBaJiTOIiBjR7Eyye8+Jy8jIAGCn08kLO3cCcF+PHo1XIEMrhRBCtGPyDnUIKVtahmePJ2C6+thT/Ak3Cj8vZMPVG/Q5YmXeBvWMKx+HKVz/0cl/J5/SRaUB7+fM8feImWPN2PraGs2GWDdAS7khhS6zu/jOGcONvt6yvdUGih2RohixWuv3BERFjSMqapzvuddbjdO5o6YHb1e9dfwUxYzBYMPrLaWyMpPKykzfuS5dLvUFchUVmWRlnYLVmkpISPeaYK92PxWbrRcGg/Q6iJajqi5KShZRVPQlRUVfYrP19S0XEBU1geTky4mNPZno6GOb/bO4rzlxtb1xD27bhlPTOCM+nqEREY1XJkMrhRBCtGMSyLVj9VLW10lFH3FkBNYkPfAp/KqQoi+K6pepeTTHmBmR7V80edWkVXj2BF5LzJxo9mdOBJzb/UGY0W6sF3hpTg3C9XPJs5OJnRIbMEAzx/k/kIV0D2Hk2pFNet0hqZKqvpbRaCM0tA+hoX0anBs8+Ds0TcPtLqoZsrnN9xgZ6Q8GHY4t9Xr19jZixDpCQ/WlBHbseJbq6k0NevjM5oR2v86eaF9crt0UF39HUdGXFBd/i9frX/dD07yoqhuDwYyiGElLeyEo99zXnLhaH+/ezUu5uQD8Z1+9cSA9ckIIIdo1eYdqQapLRTEpvt6k8hXlODY76q8PVrMf2jeU1NtTAX0tsaX9luIp84DasN6Bnw3EOlUP5CpWVLDrlV0B76+5tHrPI4+KxFvpDdgjZh/jn2cVdUwUI7eM1M9FmPaZsj7xvKZlTRQto+4SDBERRwQsExMziREj1tcL9GofHY4crNYUX9mCgo8pLV0U4D5WkpLOp2/fVwDweCooKJhXJ+DrhsFgbXCdOHQ4nTvRNJWQEP3nqbDwC9avn+07HxY2kNjYU4iNPRm7fUSLJPHZ15w4gCWlpZyVnY0KGIFST+B/bPnIHDkhhBDtmARyjdA0DW+lN2A2RFtvG/YReuBTubqSrfdvDVhOrVYZuXkktp76RP2ch3MaXSMsckKkL5AzhhvxlOgfMAxhhoaBV4z/2xZ7UiyWBEvAdPWmyPrf3kGfD2rSazeFm3xDJ0XHZzBYG+3V21v37rdQWTmlQcDn8eypt95edfV61q2rP+/PYknyDd/s2fMhQkP1Ya9OZx4GgwWTKVp69ToJTdNwOLZRVvY7JSULKSlZSHX1Brp2vYY+feYAEB19NDExJxITcyKxsSdjs/Vo8XY1Nieu1nu7d1N3sHi9NeP2pqqgqmgGyRorhBCifeq0n9ZVj4q3zIvBasAYps9xqFpfRenvpb7esLqBl+pQSf8m3Xf90oFL6yfcqKPr1V19gZyn1EPBB4GDM4zgLfd/bLCPtKO5NX+gVZMx0RRlqjec0BhmZGzhWIyRRgymfX+IiBgWQcSwfczxEOIAxMaeSGzsiQ2OezzlaJrL99xgCCEh4bw6Ad/OmkXW8ygv/5OePR/yld28+Sby89/GaAz3zc+rfQwPHxLwfqL90DS9Z782CN+48Qby8t7A4ymqV85oDKfuEAKb7TDS07+hpTVlTlytv8rKgEbWjNub2w1mM8g/H4QQQrRT7TeQ08C5yxkwG6IhzEDSDD0DoebVyDwls0E5b4UeQKW9kkbyxXpCiZKfS1h/2fpGb6l6VF/gZIwwYgg1BJz7FX5EuO+a0MND6fduv4Dl9k5Zn3JDCik3pDS4794URcEcK8N5RPthMtX/Z0FYWH/693/b91xVPbhcu3y9eLXD63QGjMZwvN4KqqqyqdIX6QP0RZ5rAzmXK5+lSwdjtSZjsSQ3eIyMHO1bekG0DI+ngurqdVRVraWycg0VFcspL1/GkCE/ExY2AABNc+PxFGEyxWK3jyAqKoOoqAzCw4844OUBmqspc+Jq/bRnD3+WlxNqMHBdt25Mjo1tvDcOwOUCi6Xx80IIIUQba7eBnGO7gyXJSwKeC+0X6gvkFKNCyU8lqI69JpMp6EML64yjCR0QSuK/EusHXHUCr7pB1xGLj9jn3LBa5hgziefIPDFxaDMYTISEpOwVwOn69XsDTXsdj6ekznDNHJzObYSG+hdYdzpzcbvzcbvzgRUN6hkyZKEvE+e2bQ9TWPipL9CzWrtisVNaIIsAABqESURBVHTBbE4gJCSF8PDBLfZaOzpVdeF07kBVnYSF6V9/pzOX5ctH4nTuCHhNefkKXyCXknIjKSk3YLV2b/OhsvubE1dL1TRu3qSvOXl7aip3pKbuv/LaHjkhhBCinWq3gZzBZMCcaA68PtheWQ0Hfj4Qg63+XLJAKeujxkURNW4fQ2nqaEoQJ/6/vXuPsuos8zz+fc+pC8UlQCABUgEkIInmMrElaumYBqPpTBt19dK2Z7wkbVozWd06yXQyjhijmXYmydga6YtrOo7ay7Xs1km8jI53RTDa4IXYYCxuIZAQQCAUdYOCqnPqvPNHnSqL4tT1VNXep+r7WatWnX1q194vrJ0Kv3qe932lkQkhUF09n+rq+cyZc23Jc2bPvoaGhkN0dh6hq+vIeZ9nzHhB37mnTzfS3v7LQa7zUtas2Qb0rI7485+voqpqATU1F1FdfTHV1Rf1vZ43b13f3K3u7tPEmCebnTMhC3FMtBgL5PMtZLOzyWR6KklNTd+iufmH52xC39V1DIhccEEDv/d7WwCorr6Yrq7jhFDDzJmrmTnzCurqLmf27H/DnDlrzvm7nzFjWQJ/utKGmxPX69Hjx3ni1CmW1NRw16WXjuziBjlJUsqlNsjVXFLDq7a9akTnXnjjhcOfJCnVevbVq6e2tn7Yc1eu/Gvq6//ivMCXyx2nrm5133m53EnOnn0GeKbkdV70oi/2BbnDh/8X+/f/FyCQzV5AVdU8qqrmUlU1l5qaxVx55aN933fo0N9RKJwlk6kjm51JJlPX97qu7oXU1a0AIJ9v5cyZA4SQJYSq4ucsMRaIsZu6ulV97YgdHXvJ5U4SY54YO+nuPt33UVtb368F9Xn2719PPt9ELnei+NFELtcEFLj22h8zb971ADQ3/4hDhzYM+FP37F/Yv001k6ni5S9/itraekJI975po5kTB/B4Swt//tRTQM92A7NGui+cQU6SlHKpDXKSNJja2iXU1i4Z9rzq6gt5+cv3k8sdp6vreXK55895XVfXf0P5Qt9cvu7uVrq7W+ksbqdYU7P4nOsePPgQXV1HSt5z+fL7WLHirwBobf0pTz5586Dja2g40vfn2LfvLzl58lslz1uw4A19QS7GLo4e/WzJ86qq5tHd3XHO9/WsJrq0uAH8UmpqlpScy5amSttgRjMnDnq2G3jtjh3kYiQAV8ycOfKbOUdOkpRyBjlJU1YIWerqVvRVyIaybNn7Wbbs/RQKebq728jnW4sfLcR47n5j9fXvJZdrolA4Q6HQQXf3mb7XM2Zc1ndeNjubWbOuIcZuoLtYbesGMsXK1+9auGfOvJxc7nlCqCKTqSGTmUU22/Mxe/bv9gisqlrA6tWPUF29kOrqBcXPC6mqupBM5twK0vz5a5k/f+1Y/upSaaRz4np95+RJcr2rbgI/bW3l1UOtVNmfFTlJUsoZ5CSpn0ymikzmQqqrB2/ZXr58/YiuNW/e73PddTtGdO6qVZ8Y0XnZ7AwuueT2EZ1b6Qa2UY50TlyvPR2/q07WDrfdwEAGOUlSyhnkJEmpM1gb5XBz4nrt6ejgqydOAPDeSy7hbYsWDb3dwEC2VkqSUs4gJ0lKncHaKIeaE9ffPU8/TT5G3r1kCX+3evWw55/HipwkKeUqb41tSdKU19tGmc1mR9RG2d/3T57km01NzMlm+e8rhp8fWZJBTpKUclbkJEmpMNqtBUrJFwr85337APjQ8uUsGmt7pEFOkpRyBjlJUuJGu7XAYD6wfz87Ozq4pKaGO0e6+XcpzpGTJKWcrZWSpMSVmhM3Wt9tauIThw4BcCKX41ft7WMfkBU5SVLKWZGTJCWifyvlaLcWKOUjzzzT97o7Rja3tIxupcr+DHKSpJQzyEmSJl2pVsqxzInrta2tjV8UK3BZoGa0+8YNZGulJCnlDHKSpElXqpVy/fr1ow5wAIUY+fOnngLgbRdfzFWzZrF23ryxV+PAipwkKfUMcpKkSTHerZS9Pvvb3/LL9nYuqanhH1avZk7VOPyvzSAnSUo5g5wkacKNdytlr6Zcjg/s3w/Aw6tWjU+IA4OcJCn1DHKSpAk3nq2U/f3Z7t2czOd56ezZvPWii8ZptDhHTpKUegY5SdKEmKhWyl6fOXKErzc1AdB4+jQ/a2srb15cf1bkJEkpZ5CTJI27iWql7JUrFPjQgQO/Oy53u4HzbmCQkySlm0FOkjTuJqqVstfDhw5xLJcjABnGYbuBgbq6DHKSpFQzyEmSyta/jbKhoWFCWil7PX3mDPcXN//+5MqVdBQK5W83MFAu5xw5SVKqGeQkSWUp1UbZ0NAwrq2UvWKM3LF3L2cLBd6xaBF3Ll06Ltc9Ty4Hs2ZNzLUlSRoHZQW5EMKFwP8BXgA8A7w1xthc4rxu4Mni4cEY4xvLua8kKT1KtVE2NDT0fYynjzzzDD9sbmZONsvDK1eO67XP4Rw5SVLKZcr8/g8AG2OMLwQ2Fo9LORNjvLb4YYiTpAq3detWHnzwQbZu3drXRpnNZse9jbK/7zQ18dFnnwWgs1Bg35kzE3IfwO0HJEmpV25r5ZuAtcXXnwc2A/+1zGtKklJsolekHMz64sbfAN3jvUrlQFbkJEkpF2KMY//mEFpijPOKrwPQ3Hs84Lw8sB3IAw/FGP/vINe7HbgdYNGiRS/90pe+NOax6VynTp1i9uzZSQ9DKsnnM/0aGxvZvn071157Ldu3b+dzn/schUKBTCbDbbfdxtvf/vYJvf9PgfuKrzNANfAJ4MoJut/qj3+c9ssv56l163w2lUr+3FSa+XyOn3Xr1j0RY1xT6mvDBrkQwg+BxSW+dC/w+f7BLYTQHGOcX+Ia9THGwyGEy4AfATfEGJ8e6r5r1qyJ27ZtG3JsGrne35RLaeTzmW4DK3AbNmzgrrvuOm9xk4lyMpfjxb/4BcdyOe6qr+fimprxX6VyoFtvhbVr2bxihc+mUsmfm0ozn8/xE0IYNMgN21oZY3ztEBc+FkJYEmP8bQhhCXB8kGscLn7eH0LYDLwEGDLISZLSYeBiJk1NTZPSStnrzn37OJbL8eq5c/nEqlVkQpjQ+wFuPyBJSr1y58h9A7gVeKj4+esDTwghzAc6YoydIYSFwKuAj5V5X0nSBOq/L1ypPeEmYkXKUv7fiRN84dgx6jIZPnv55ZMT4sA5cpKk1Cs3yD0EPBpC+DPgWeCtACGENcAdMcZ3Ay8CHgkhFOiZ2vBQjHFnmfeVJE2QpBYzGeh7TU28fdcuAB5YsYIXzpw5KfcFelatNMhJklKsrCAXY2wCbijx/jbg3cXXW4Cry7mPJGli9a/AldoXbv369ZMW4AC2trby+iefpJue3wCumTNn0u4N2FopSUq9citykqQKV2oxk4GtlJPtbw8fprv4OgA/aW3l3847b1HkiWNrpSQp5QxykjTN9K++NTQ0JL6YyUCHOzv5VlMT0FONq8lkWDuZIQ4McpKk1DPISdI0Umr+W5KLmQxUiJF37d5Ne3c3DXPmcPPChayb6K0GSumdI5fPT+59JUkaIYOcJE1xI5n/lmQFrr+/P3yYHzQ3s7C6mq9edRWLa2uTGUjvHDmDnCQppQxykjSFjXT+W1IVuP7++dgx7n66Z4vR/716dXIhDmytlCSlnkFOkqaYoSpwSc9/G8zm5mbesWsXEcgCi5JeMdLtByRJKWeQk6QpZCQVuDRU3wb64IEDxH7Hm1taJn9eXH9uPyBJSjmDnCRVuEqswPX39RMn2NrWBvRU4xJZpXIgWyslSSlnkJOkClapFbhez509y7t27wbgP9XXs7imhrVJrFI5kEFOkpRyBjlJqjCVXoHrlS8UeNuuXTTn87z+wgvZsGoVIYSkh9XDOXKSpJQzyElSBan0Clx/t+/dy09bW1lQVcU/XnFFekIcOEdOkpR6BjlJSrH+1beGhoaKrsD1t+G55/jHo0cBONXdzb4zZ7goTcHJ1kpJUsoZ5CQpRfoHN+Cc6tvGjRtZu3ZtxVbgeh08e5Z7DxzoO87HmPwqlQPZWilJSjmDnCSlxMC2yVtvvfWc6tvmzZtZv359RVbgenUWCvxxYyMdhQIZIJCSVSr7ixHyeYOcJCnVDHKSlKChFi4Bzqu+ARVXgevv7n37+EV7O8tqa3lk9Wr+9dSpdKxS2V8+D9ksZDJJj0SSpEEZ5CRpEg3VOjlw4ZJbbrmFW265pWKrbwP9twMH+NSRI1QBX77ySq674AJuWrAg6WGdz/lxkqQKYJCTpEkyXOvkYAuXVHqAA/jC0aPc/+yzAIQQyMeY8IiG4Pw4SVIFMMhJ0gQZbsVJOL91spLbJgdzoquL9+3b13dcSOPiJv259YAkqQIY5CRpnIx2xcmp1jpZSq5Q4K07d9KSzxOADClc3GQgWyslSRXAICdJYzRUcBvNipNTMcD1uufpp9nU0sKi6moeWb2anR0d6VvcZCBbKyVJFcAgJ0kjNJrgBlNvxcnR+uD+/fzt4cNkga9edRWvnDuXNyU9qJGwtVKSVAEMcpI0iHKC23RomxzKp48c4cGDBwHIhEBIeDyjYmulJKkCGOQkifMXJhluhUkYWXCbbgEOYP+ZM/xlJS1uMpBBTpJUAQxykqa9gaGtdx6bwW30WnI5Xv/kk5wuFMgAgQpY3GQg58hJkiqAQU7StFCq4tZ7PDC09b5vcBudXKHAWxob2d3RwVWzZvHJlSv5ZXt7+hc3Gcg5cpKkCmCQkzQlDTW/bcOGDdx1113nHJfaz226rTBZji0tLdz59NNsa2/n4upqvnn11SyfMYPXXnhh0kMbPVsrJUkVwCAnaUoYzcIkX/nKV845bmpqGjS0GdyGt7W1lbU7dpCLEYD/sWIFy2fMSHhUZbC1UpJUAQxykipOuQuTvPnNb+YnP/nJeRU4Q9vYPHzoUF+IywDP53LJDqhcVuQkSRXAICcp9Yaqto11YZKrr7562m4NMJ6+29TE155/HugJcbWVtrBJKc6RkyRVAIOcpMQ1NjaydevWkguRwNBtkmNdmMQKXPm2tbXxlsZGuoG3X3wxV86aVXkLm5RiRU6SVAEMcpIm3cCgdvfdd5PP50suRDJcm6QLkyTjsWPHeNeePZwuFHjHokV8/ooryISK2vZ7cM6RkyRVAIOcpHE11DL/g81ny+VyFAqFkguRwMj3bzO4TY5vnDjBn+zaRaSnnfI9S5ZMnRAHtlZKkiqCQU5SWUazzP9g89mqq6v7KnIDFyJx/7Z0OdHVxbv37CEWjwPwL62tXF/p8+L6s7VSklQBDHKShjRUhQ1Gt8z/YPPZrrrqKtra2oZciMTglrzWfJ4/+PWveT6XI9BTjauZCoubDGRrpSSpAhjkpGlstG2Qo52/Ntgy/wPns3V2dvYFQ7BNMo1+1NzMbbt382xnJytnzOCTq1bxm9Onp8biJgNZkZMkVQCDnDSFjCSYldMGOZb5a4NV1wxqlWNzczOv27GDAj2tlB+77DLesHAhb1i4MOmhTQznyEmSKoBBTkqp0YSykVTPhqumjaQNcizz1wxtla2zUOA/7t1LoXicAfacOZPkkCaeFTlJUgUwyEmTYGDoKvXeaKplY6meDVdNG2kbpPPXpo/OQoE3/+Y37C0GtyxTdE7cQM6RkyRVAIOcNAbltDBu3LjxvPdGWy0bS/VsJNW0kbRBWmGbHn7c0sIde/eyu6ODBVVVfHLVKg51dk7NOXED5XJQW5v0KCRJGpJBTlPeWFoUJ7KFcfPmzQBlVcvGWj0brppmSBPA4y0tvGb79r52yodXreKdixcnOqZJlcvB7NlJj0KSpCEZ5JSY0bYbTkbomowWxt4q3Wjnno00lA0VzAxqGs6Z7m7es2dPX4jLAoc7O5Mc0uSztVKSVAEMctNUuQGp3GvA6NsNJyN0TVYLIzDquWeGMk20U/k8b5yOc+IGcrETSVIFKCvIhRD+GLgfeBHwshjjtkHOuwn4G3r+XfCZGOND5dx3vCQdZibzmo2NjWzdunVMC2mUCkjlXmMs7YaTEbomq4Wx99hgprT4flMT79m7l4OdnSypqeHjK1fy7Nmz02NO3EBuPyBJqgQxxjF/0BPgLgc2A2sGOScLPA1cBtQAO4AXD3ft+vr6uGXLlthry5Yt8YEHHuh7bzyO6+rqYjabjXV1dfGRRx4Z1fGWLVvKvsZkXrO2trbvvTvuuCNms9kIxGw2G2+88cZRHT/wwAPxgQceKOsad9xxx6T82Sfi2dH427RpU9JDmNa+feJEDJs2RTZtimHTpvjo0aNJDylZ73pXjJ/5TIzRZ1Pp5bOpNPP5HD/AtjhIXiqrIhdj3AUQQhjqtJcB+2KM+4vnfgl4E7BzqG86fPgwN7zmNWz88pcBuOEtb6Erl6OmupoNH/0od91335iPN375y2zesuXcas4Xvziq483f/CZAWdeYzGvmcjkKhULPnK2ODmqqq+kCaqqrefPrXsdPHn98xMdrr7kGiq/Heo1bbr6ZW26+mc1btrD2la+kYeVKADY+9tjv3rvuOq6urx/78cqVcPw4DStX9l1/PI41/qqbm/27TcizXV3ccuAAsXicAfY1NcHQP9entvZ2WyslSakXeoJemRcJYTNwTyzRWhlCeAtwU4zx3cXjdwIvjzG+t8S5twO3Fw9fmgXunzEDgPvPnqWbnvLeuqoqNuXzYz6+f8YMrq+q4qZTp3qCBfDxujruOXNmxMffLa5oVs41krpm73uP5/NcX1XFK6qq+Fk+P6pjYNTfU+oaEvR0BgzzCyFNgJ1Ll3Lzhz/M4YULCYUCmRipyef57kc+wiv27El6eMkJgZ333UfLS17CqVOnmO0Klkohn02lmc/n+Fm3bt0TMcY1pb42bJALIfwQKLXu9L0xxq8Xz9nMOAS5Ad8X6+rqJmwRjDTPZ5uIa37qU5+ira3tnPektOi/CI4m3tbWVj5/9Cj/fPw47d3dvHruXD60bBlPnDo1PefEDcFnU2nls6k08/kcPyGEsQe5Ed5gM4MHuQbg/hjjHxSP1wPEGB8c6pqXXnppfOyxxyY03Ewn/gelNPP5nDxbW1tZu307XcWf/dfPnct3r7mGumw24ZGlk8+m0spnU2nm8zl+hgpyk9Hf9kvghSGEFcBh4N8DbxvumxYvXjyqpdfdP0uShvfQwYN9IS4AN86fb4iTJKkCZcr55hDCH4UQDgENwLdCCN8rvn9JCOHbADHGPPBe4HvALuDRGGNjecOWJI1GIUbu2bePbzQ1AT0hbkYmw2vmz092YJIkaUzKXbXya8DXSrx/BPjDfsffBr5dzr0kSWOzqbmZ9z31FI0dHVSFwAeWLWNmcaNv58NJklSZXDpQkqawb5w4wR/95jcUiscPr1zJ+y69NNExSZKk8pXVWilJSq9tbW28Y9euvhCXBU51dyc5JEmSNE6syEnSFLO1tZW/OXSIr504QVeMZOiZE1dTbKeUJEmVzyAnSVPIT1taWLdjB/niypRvWLCAe5Yu5V9aW50TJ0nSFGKQk6QpoimX47bdu/tCXAZ4xQUXcP28eVxvJU6SpCnFOXKSNAX8qr2dNU88wVNnzwI9P9xrMxnWGeAkSZqSrMhJUoW7d/9+/ufBg3QD182Zw/ply9jd0WErpSRJU5hBTpIq1Onubv6ksZFvnTwJ9KxK+bHLLmOtm3xLkjTl2VopSRWo8fRpXvbEE30hrtfWtraERiRJkiaTFTlJqiAxRj504AAfe+458jGyvLaWY11d5GJ0ewFJkqYRg5wkVYiWXI637tzJD5qbgZ5Wys9dcQV1mQybW1qcEydJ0jRikJOkCvB4Swvv3LWLg52d57z/87Y21i9fboCTJGmaMchJUoo93tLChw8c4MetrQC8aOZMDpw9S65QsJVSkqRpzCAnSSn1haNHuWX3bmLx+E8XL+bTq1ezrb3dVkpJkqY5g5wkpUy+UOBjzz3HfQcO9IW4DLC6ro7qTIaGuXMNcJIkTXMhxjj8WQkIITwPPJv0OKaQhcCJpAchDcLns9fMmbOYOXMOHR3tANTXryaEQIyRw4f30tFxOuERTjc+m0orn02lmc/n+FkeY7yo1BdSG+Q0vkII22KMa5Ieh1SKz6fSymdTaeWzqTTz+ZwcbgguSZIkSRXGICdJkiRJFcYgN318OukBSEPw+VRa+WwqrXw2lWY+n5PAOXKSJEmSVGGsyEmSJElShTHITUMhhLtDCDGEsDDpsUgAIYS/DiHsDiH8OoTwtRDCvKTHpOkthHBTCGFPCGFfCOEDSY9H6hVCWBpC2BRC2BlCaAwh3Jn0mKT+QgjZEMK/hhC+mfRYpjqD3DQTQlgK3AgcTHosUj8/AK6KMV4D7AXWJzweTWMhhCzwKeDfAS8G/kMI4cXJjkrqkwfujjG+GHgF8Bc+n0qZO4FdSQ9iOjDITT+fBN4PODlSqRFj/H6MMV88/BlwaZLj0bT3MmBfjHF/jLEL+BLwpoTHJAEQY/xtjPFXxdft9PyDuT7ZUUk9QgiXAq8HPpP0WKYDg9w0EkJ4E3A4xrgj6bFIQ7gN+E7Sg9C0Vg881+/4EP5DWSkUQngB8BLg58mOROqzgZ6CQSHpgUwHVUkPQOMrhPBDYHGJL90LfJCetkpp0g31bMYYv14851562ob+aTLHJkmVJoQwG/gKcFeMsS3p8UghhJuB4zHGJ0IIa5Mez3RgkJtiYoyvLfV+COFqYAWwI4QAPa1rvwohvCzGeHQSh6hparBns1cI4U+Bm4EbovuiKFmHgaX9ji8tvielQgihmp4Q908xxq8mPR6p6FXAG0MIfwjMAC4IIXwhxviOhMc1ZbmP3DQVQngGWBNjPJH0WKQQwk3Aw8DvxxifT3o8mt5CCFX0LLpzAz0B7pfA22KMjYkOTAJCz29jPw+cjDHelfR4pFKKFbl7Yow3Jz2Wqcw5cpLS4O+BOcAPQgjbQwj/kPSANH0VF955L/A9ehaSeNQQpxR5FfBO4DXFn5fbixUQSdOMFTlJkiRJqjBW5CRJkiSpwhjkJEmSJKnCGOQkSZIkqcIY5CRJkiSpwhjkJEmSJKnCGOQkSZIkqcIY5CRJkiSpwhjkJEmSJKnC/H98YuO6YPvcIQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "markdown", "metadata": { "id": "Xx9AFyqYal8X" }, "source": [ "# 3. Custom Layers and Ops\n", "\n", "Some layers have no weights, such as `tf.keras.layers.Flatten` or `tf.keras.layers.ReLU`. If you want to create a custom layer without any weights, the simplest option is to create a `tf.keras.layers.Lambda` layer and pass it the function to perform. \n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "AROT3BDig-N7" }, "source": [ "## 3.1: Create a softplus layer\n", "\n", "Hint: You have used the softplus function earlier which is (log(exp(X) + 1), and the `tf.nn.softplus` function can be used for the same" ] }, { "cell_type": "code", "metadata": { "id": "TQ0VS7w9hnYZ", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "174dfdb4-c78f-43ef-8b74-7252558f6ffa" }, "source": [ "softplus_fn = tf.keras.layers.Lambda(lambda x: tf.nn.softplus(x))\n", "softplus_fn" ], "execution_count": 18, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 18 } ] }, { "cell_type": "code", "metadata": { "id": "v71KlglmiSxP", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "78e655bf-c66f-4298-8412-86b326b0c69b" }, "source": [ "softplus_fn([-10., -5., 0., 5., 10.])" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 57 } ] }, { "cell_type": "markdown", "metadata": { "id": "5pJm874_iYlT" }, "source": [ "## 3.2: Create regression model with softplus layer\n", "\n", "Create a regression model but add your softplus layer at the top (i.e., after the existing 1-unit dense layer). This can be useful to ensure that your model never predicts negative values.\n", "\n" ] }, { "cell_type": "code", "metadata": { "id": "Lkl508SSi3Fp", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "13774898-ed06-4f90-9cb3-165c0b9c6419" }, "source": [ "model = tf.keras.models.Sequential([\n", " tf.keras.layers.Dense(16, activation=\"relu\", \n", " input_shape=(X_train.shape[1],)),\n", " tf.keras.layers.Dense(32, activation=\"relu\"),\n", " tf.keras.layers.Dense(32, activation=\"relu\"),\n", " tf.keras.layers.Dense(1),\n", " softplus_fn\n", "])\n", "\n", "model.compile(loss=mse_loss, \n", " optimizer=tf.keras.optimizers.SGD(lr=1e-3),\n", " metrics=['mean_squared_error'])\n", "\n", "model.summary()" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Model: \"sequential_2\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "dense_6 (Dense) (None, 16) 144 \n", "_________________________________________________________________\n", "dense_7 (Dense) (None, 32) 544 \n", "_________________________________________________________________\n", "dense_8 (Dense) (None, 32) 1056 \n", "_________________________________________________________________\n", "dense_9 (Dense) (None, 1) 33 \n", "_________________________________________________________________\n", "lambda (Lambda) (None, 1) 0 \n", "=================================================================\n", "Total params: 1,777\n", "Trainable params: 1,777\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "nMKttJuOj7In" }, "source": [ "## 3.3: Train and Predict" ] }, { "cell_type": "code", "metadata": { "id": "B1HYHd6AjDUC", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "f2c74604-1de3-4bb1-9a0b-f3b02d9d15b8" }, "source": [ "history = model.fit(X_train_scaled, y_train, \n", " epochs=30,\n", " batch_size=32,\n", " validation_split=0.1)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Epoch 1/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 2.0329 - mean_squared_error: 2.0329 - val_loss: 0.9527 - val_mean_squared_error: 0.9527\n", "Epoch 2/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.8912 - mean_squared_error: 0.8912 - val_loss: 0.7253 - val_mean_squared_error: 0.7253\n", "Epoch 3/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.7200 - mean_squared_error: 0.7200 - val_loss: 0.6603 - val_mean_squared_error: 0.6603\n", "Epoch 4/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6587 - mean_squared_error: 0.6587 - val_loss: 0.6239 - val_mean_squared_error: 0.6239\n", "Epoch 5/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6242 - mean_squared_error: 0.6242 - val_loss: 0.5995 - val_mean_squared_error: 0.5995\n", "Epoch 6/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5989 - mean_squared_error: 0.5989 - val_loss: 0.5816 - val_mean_squared_error: 0.5816\n", "Epoch 7/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5775 - mean_squared_error: 0.5775 - val_loss: 0.5674 - val_mean_squared_error: 0.5674\n", "Epoch 8/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5599 - mean_squared_error: 0.5599 - val_loss: 0.5509 - val_mean_squared_error: 0.5509\n", "Epoch 9/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5449 - mean_squared_error: 0.5449 - val_loss: 0.5405 - val_mean_squared_error: 0.5405\n", "Epoch 10/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5326 - mean_squared_error: 0.5326 - val_loss: 0.5296 - val_mean_squared_error: 0.5296\n", "Epoch 11/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5221 - mean_squared_error: 0.5221 - val_loss: 0.5231 - val_mean_squared_error: 0.5231\n", "Epoch 12/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5130 - mean_squared_error: 0.5130 - val_loss: 0.5125 - val_mean_squared_error: 0.5125\n", "Epoch 13/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5055 - mean_squared_error: 0.5055 - val_loss: 0.5084 - val_mean_squared_error: 0.5084\n", "Epoch 14/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4981 - mean_squared_error: 0.4981 - val_loss: 0.5028 - val_mean_squared_error: 0.5028\n", "Epoch 15/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4915 - mean_squared_error: 0.4915 - val_loss: 0.4961 - val_mean_squared_error: 0.4961\n", "Epoch 16/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4852 - mean_squared_error: 0.4852 - val_loss: 0.4893 - val_mean_squared_error: 0.4893\n", "Epoch 17/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4799 - mean_squared_error: 0.4799 - val_loss: 0.4855 - val_mean_squared_error: 0.4855\n", "Epoch 18/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4745 - mean_squared_error: 0.4745 - val_loss: 0.4799 - val_mean_squared_error: 0.4799\n", "Epoch 19/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4702 - mean_squared_error: 0.4702 - val_loss: 0.4787 - val_mean_squared_error: 0.4787\n", "Epoch 20/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4657 - mean_squared_error: 0.4657 - val_loss: 0.4753 - val_mean_squared_error: 0.4753\n", "Epoch 21/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4616 - mean_squared_error: 0.4616 - val_loss: 0.4702 - val_mean_squared_error: 0.4702\n", "Epoch 22/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4577 - mean_squared_error: 0.4577 - val_loss: 0.4652 - val_mean_squared_error: 0.4652\n", "Epoch 23/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4541 - mean_squared_error: 0.4541 - val_loss: 0.4624 - val_mean_squared_error: 0.4624\n", "Epoch 24/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4502 - mean_squared_error: 0.4502 - val_loss: 0.4605 - val_mean_squared_error: 0.4605\n", "Epoch 25/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4468 - mean_squared_error: 0.4468 - val_loss: 0.4560 - val_mean_squared_error: 0.4560\n", "Epoch 26/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4437 - mean_squared_error: 0.4437 - val_loss: 0.4539 - val_mean_squared_error: 0.4539\n", "Epoch 27/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4403 - mean_squared_error: 0.4403 - val_loss: 0.4521 - val_mean_squared_error: 0.4521\n", "Epoch 28/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4376 - mean_squared_error: 0.4376 - val_loss: 0.4520 - val_mean_squared_error: 0.4520\n", "Epoch 29/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4350 - mean_squared_error: 0.4350 - val_loss: 0.4467 - val_mean_squared_error: 0.4467\n", "Epoch 30/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4322 - mean_squared_error: 0.4322 - val_loss: 0.4445 - val_mean_squared_error: 0.4445\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "foUGuxTAjMzc", "colab": { "base_uri": "https://localhost:8080/", "height": 283 }, "outputId": "bb033170-d52a-4511-b43c-71ed6ddb0c6e" }, "source": [ "history_df = pd.DataFrame(history.history)\n", "history_df[['loss', 'val_loss']].plot(kind='line')" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 60 }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAD4CAYAAAATpHZ6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZRcZZ3/8fe3tu6u3jvdWTshCQQCJCFgWPwJyKJsKrjCBFzgKPGHCG7HH/4cZ2AQj47OMKOjA8MgIg5bRlBRUHQkEvipSAhZCYQYSOisna3Te9fy/f1xq5NO0lvSlVSq6vM6p05V3ftU3eemTj736ec+97nm7oiISGEI5boCIiKSPQp1EZEColAXESkgCnURkQKiUBcRKSCRXG24vr7eJ0+enKvNi4jkpZdeemmbuzcMtD5noT558mQWLVqUq82LiOQlM1s32Hp1v4iIFBCFuohIAVGoi4gUkJz1qYtIcUokEjQ1NdHV1ZXrqhzVSktLaWxsJBqNHtTnFOoickQ1NTVRWVnJ5MmTMbNcV+eo5O5s376dpqYmpkyZclCfVfeLiBxRXV1djBo1SoE+CDNj1KhRh/TXjEJdRI44BfrQDvXfKO9C/dXNu/mnp19jZ3tPrqsiInLUybtQf3NbO99fsIaNLZ25roqI5KmKiopcV+GwGTLUzWyimS0ws1fMbKWZfa6fMmZm3zOzNWa2zMxOOzzVhZp4DIBdHYnDtQkRkbw1nJZ6EviSu58EnAXcaGYn7VfmUmBa5jEPuCurteyjNhPqOzvU/SIiI+PufPnLX2bGjBnMnDmTRx99FIBNmzZx7rnnMnv2bGbMmMFzzz1HKpXi2muv3VP2X/7lX3Jc+/4NOaTR3TcBmzKvW81sFTABeKVPsSuABzy4N96fzazGzMZlPptVtfFgzOZOtdRF8t4//HIlr2zcndXvPGl8Fbe+7+RhlX388cdZsmQJS5cuZdu2bZx++umce+65PPTQQ1x88cX87d/+LalUio6ODpYsWcKGDRtYsWIFALt27cpqvbPloPrUzWwycCrwwn6rJgBv9XnflFm2/+fnmdkiM1vU3Nx8cDXN2NP9ohOlIjJCzz//PHPnziUcDjNmzBje+c538uKLL3L66afzox/9iNtuu43ly5dTWVnJ1KlTWbt2LTfddBO/+c1vqKqqynX1+zXsi4/MrAJ4DPi8ux/SodXd7wHuAZgzZ84h3fE6FglRHgurpS5SAIbboj7Szj33XBYuXMiTTz7Jtddeyxe/+EU+/vGPs3TpUp5++mnuvvtu5s+fz3333Zfrqh5gWC11M4sSBPqD7v54P0U2ABP7vG/MLDssauIx9amLyIidc845PProo6RSKZqbm1m4cCFnnHEG69atY8yYMVx//fV86lOfYvHixWzbto10Os2HPvQh7rjjDhYvXpzr6vdryJa6BSPgfwiscvc7Byj2BPBZM3sEOBNoORz96b3qyhXqIjJyH/jAB/jTn/7EKaecgpnx7W9/m7Fjx/LjH/+Y73znO0SjUSoqKnjggQfYsGED1113Hel0GoBvfvObOa59/yw4tzlIAbOzgeeA5UA6s/irwCQAd787E/zfBy4BOoDr3H3QO2DMmTPHD/UmGR/74Qvs7kryixvfcUifF5HcWbVqFSeeeGKuq5EX+vu3MrOX3H3OQJ8ZzuiX54FBr1fNjHq5cZj1HLHaeIz1OzqO1OZERPJG3l1RCsGwRk0TICJyoLwM9Zp4jN1dSZKp9NCFRUSKSF6Geu8FSC2dGtYoItJXfoZ6ee9UAQp1EZG+8jLU907qpX51EZG+8jLUNf+LiEj/8jTUNVOjiBwZg829/uabbzJjxowjWJuh5WWo12Ra6up+ERHZ17An9DqaVJREiIRM3S8i+e7XX4HNy7P7nWNnwqXfGnD1V77yFSZOnMiNNwbXS952221EIhEWLFjAzp07SSQS3HHHHVxxxRUHtdmuri5uuOEGFi1aRCQS4c477+T8889n5cqVXHfddfT09JBOp3nssccYP348V155JU1NTaRSKf7u7/6Oq666akS73SsvQ93MqInH1FIXkYN21VVX8fnPf35PqM+fP5+nn36am2++maqqKrZt28ZZZ53F5ZdfflA3f/7BD36AmbF8+XJeffVVLrroIlavXs3dd9/N5z73Oa655hp6enpIpVI89dRTjB8/nieffBKAlpaWrO1fXoY69F5Vqpa6SF4bpEV9uJx66qls3bqVjRs30tzcTG1tLWPHjuULX/gCCxcuJBQKsWHDBrZs2cLYsWOH/b3PP/88N910EwDTp0/nmGOOYfXq1bz97W/nG9/4Bk1NTXzwgx9k2rRpzJw5ky996UvccsstvPe97+Wcc87J2v7lZZ86BCdLdaJURA7FRz7yEX7605/y6KOPctVVV/Hggw/S3NzMSy+9xJIlSxgzZgxdXV1Z2dbVV1/NE088QVlZGZdddhnPPPMMxx9/PIsXL2bmzJl87Wtf4/bbb8/KtiCPW+o18Shvbm/PdTVEJA9dddVVXH/99Wzbto1nn32W+fPnM3r0aKLRKAsWLGDdunUH/Z3nnHMODz74IBdccAGrV69m/fr1nHDCCaxdu5apU6dy8803s379epYtW8b06dOpq6vjox/9KDU1Ndx7771Z27e8DfXaeIyX3zo67xEoIke3k08+mdbWViZMmMC4ceO45ppreN/73sfMmTOZM2cO06dPP+jv/MxnPsMNN9zAzJkziUQi3H///ZSUlDB//nx+8pOfEI1GGTt2LF/96ld58cUX+fKXv0woFCIajXLXXXdlbd+GnE/9cBnJfOoA3/z1Ku57/g1W33HpQZ3MEJHc0nzqw3co86nnbZ96XTxGIuW096RyXRURkaPGcG5ndx/wXmCrux9w6ZSZVQP/RXAnpAjwT+7+o2xXdH97ript76GiJG97kUQkDyxfvpyPfexj+ywrKSnhhRdeyFGNBjacNLyf4FZ1Dwyw/kbgFXd/n5k1AK+Z2YPufliHpuy9qjTBxLrDuSURyTZ3z6tu05kzZ7JkyZIjus1D7RofsvvF3RcCOwYrAlRm7lNakSmbPKTaHIS90+9qWKNIPiktLWX79u2HHFrFwN3Zvn07paWlB/3ZbPRbfB94AtgIVAJXuXu/tyQys3nAPIBJkyaNaKN7Z2pUqIvkk8bGRpqammhubs51VY5qpaWlNDY2HvTnshHqFwNLgAuAY4Hfmdlz7r57/4Lufg9wDwSjX0ay0b1zquuqUpF8Eo1GmTJlSq6rUbCyMfrlOuBxD6wB3gAOfpDnQaopU0tdRGR/2Qj19cCFAGY2BjgBWJuF7x1UJByisjSilrqISB/DGdL4MHAeUG9mTcCtQBTA3e8Gvg7cb2bLAQNucfdth63GfWj+FxGRfQ0Z6u4+d4j1G4GLslajg1Abj2pOdRGRPvL2ilJAc6qLiOwnr0M9aKkr1EVEeuV1qNfEY+zSjTJERPbI61Cvjcdo7U7Sk+z3WicRkaKT36Fenpn/pVNdMCIikOehrqtKRUT2ldehvmf+l3a11EVEIO9DvXemRrXURUQg30O9vLf7RS11ERHI91DfM/2uWuoiIpDnoV4WDROLhNRSFxHJyOtQNzNdVSoi0kdehzr0ztSo7hcRESiAUK+JR9X9IiKSkfehrpa6iMheeR/qmn5XRGSvIUPdzO4zs61mtmKQMueZ2RIzW2lmz2a3ioOrjUfZ1ZHAfUT3sRYRKQjDaanfD1wy0EozqwH+Hbjc3U8GPpKdqg1PbTxGMu20dieP5GZFRI5KQ4a6uy8EdgxS5GrgcXdfnym/NUt1G5aazAVImlddRCQ7ferHA7Vm9gcze8nMPj5QQTObZ2aLzGxRc3NzFja9d/6XHepXFxHJSqhHgLcB7wEuBv7OzI7vr6C73+Puc9x9TkNDQxY2vXdOdV2AJCISBPJINQHb3b0daDezhcApwOosfPeQ9s6prlAXEclGS/0XwNlmFjGzOHAmsCoL3zsse6bfVZ+6iMjQLXUzexg4D6g3sybgViAK4O53u/sqM/sNsAxIA/e6+4DDH7OtuiyKmVrqIiIwjFB397nDKPMd4DtZqdFBCoeMqtKorioVEaEArigFqCuP6USpiAgFEuo1matKRUSKXUGEejCpl1rqIiIFEepqqYuIBAoi1NVSFxEJFEioR+noSdGdTOW6KiIiOVUQob73qlJ1wYhIcSuIUN9zVam6YESkyBVIqGcm9dJUASJS5Aoi1GvUUhcRAQok1DX9rohIoDBCXSdKRUSAAgn10miY0miIne1qqYtIcSuIUIfeC5DUUheR4lYwoV4Tj2lOdREpekOGupndZ2ZbzWzQG1+Y2elmljSzD2evesNXG4/qRKmIFL3htNTvBy4ZrICZhYF/BH6bhTodktp4TCdKRaToDRnq7r4Q2DFEsZuAx4Ct2ajUoagtV0tdRGTEfepmNgH4AHDXMMrOM7NFZraoubl5pJveR208RktngnTas/q9IiL5JBsnSv8VuMXd00MVdPd73H2Ou89paGjIwqb3qonHSDvs7lIXjIgUryFvPD0Mc4BHzAygHrjMzJLu/vMsfPew7Zn/pSOxZ9oAEZFiM+JQd/cpva/N7H7gV0c60GHfmRqnUH6kNy8iclQYMtTN7GHgPKDezJqAW4EogLvffVhrdxBqMi11jVUXkWI2ZKi7+9zhfpm7Xzui2ozAnpa6pt8VkSJWMFeU6kYZIiIFFOqVpRFCplAXkeJWMKEeChk1mtRLRIpcwYQ6BCdLdaJURIpZQYV6bTymE6UiUtQKLNQ1/4uIFLeCCvUazdQoIkWuoEJdLXURKXYFFeo18RjdyTSdPalcV0VEJCcKKtR1AZKIFLuCCvW68t6ZGhXqIlKcCirUe6fc1clSESlWBRXq6n4RkWJXYKG+90YZIiLFqKBCfU/3S7ta6iJSnAoq1GOREOWxsFrqIlK0hgx1M7vPzLaa2YoB1l9jZsvMbLmZ/dHMTsl+NYcvuKpULXURKU7DaanfD1wyyPo3gHe6+0zg68A9WajXIastj7JDoS4iRWo4t7NbaGaTB1n/xz5v/ww0jrxah65Wc6qLSBHLdp/6J4FfD7TSzOaZ2SIzW9Tc3JzlTQfU/SIixSxroW5m5xOE+i0DlXH3e9x9jrvPaWhoyNam91Ebj7JTo19EpEhlJdTNbBZwL3CFu2/Pxnceqpp4jN1dSZKpdC6rISKSEyMOdTObBDwOfMzdV4+8SiPTewFSS6f61UWk+Ax5otTMHgbOA+rNrAm4FYgCuPvdwN8Do4B/NzOApLvPOVwVHsreqQISjKooyVU1RERyYjijX+YOsf5TwKeyVqMRqsm01HWyVESKUUFdUQr7ttRFRIpNAYe6WuoiUnwKLtRrytX9IiLFq+BCvbIkQiRk6n4RkaJUcKFuZrqqVESKVsGFOvReVaqWuogUnwIN9ZhOlIpIUSrIUK+JR3XzaREpSgUZ6rXxmOZUF5GiVJChXlMeZVdHD+6e66qIiBxRBRnqtfEYiZTT3pPKdVVERI6oAg314AIkzasuIsWmIEO9JjNVgE6WikixKchQ1/wvIlKsCjTUM90vCnURKTIFGerqfhGRYjVkqJvZfWa21cxWDLDezOx7ZrbGzJaZ2WnZr+bBqVFLXUSK1HBa6vcDlwyy/lJgWuYxD7hr5NUamWg4RGVJRC11ESk6Q4a6uy8EdgxS5ArgAQ/8Gagxs3HZquChqimPqqUuIkUnG33qE4C3+rxvyiw7gJnNM7NFZraoubk5C5seWDCpl1rqIlJcjuiJUne/x93nuPuchoaGw7qtWs2pLiJFKBuhvgGY2Od9Y2ZZTtXG1f0iIsUnG6H+BPDxzCiYs4AWd9+Uhe8dkZp4jF26UYaIFJnIUAXM7GHgPKDezJqAW4EogLvfDTwFXAasATqA6w5XZQk2Cs2vwugTBy1WG4/R2p0kkUoTDRfkcHwRkQMMGeruPneI9Q7cmLUaDWXJg/CLG+GGP8GYkwYsVlu+d6z66MrSI1U7EZGcyr8m7LSLIRSBpQ8NWkxXlYpIMcq/UK9oCIJ92XxIJQcspul3RaQY5V+oA8y+Gtq2wF+fGbDI3pka1VIXkeKRn6E+7SIoqwv61wfQO/+LxqqLSDHJz1CPxGDWlfDaU9C5s98iaqmLSDHKz1CHoAsm1QMrHut3dTwWJhYOqaUuIkUlf0N97CwYfTIs6X8UjJlRo6tKRaTI5G+omwWt9Q0vQfNr/RaZ2lDOH/+6nUQqfYQrJyKSG/kb6hD0q1t4wNb6vHOn0rSzk8cXNx3hiomI5EZ+h3rFaJj2blj2KKRTB6w+/4TRzGqs5vsL1qi1LiJFIb9DHYIumNZNsHbBAavMjJsvmMZbOzr52cs5nzhSROSwy/9QP/4SKKsdsAvmwhNHM2NCFT9YsIakWusiUuDyP9QjJTDjw7DqV9C564DVva31dds7+PmSjTmooIjIkZP/oQ6ZMevdsPJn/a5+90ljOGlcFd9/5nW11kWkoBVGqI8/FRpOHHTM+s0XTuPN7R08sVStdREpXMMKdTO7xMxeM7M1ZvaVftZPMrMFZvaymS0zs8uyX9VBKwiz50LTX2Db6/0WueikMUwfW8n3n1lDKu1HtHoiIkfKkKFuZmHgB8ClwEnAXDPb/+4UXwPmu/upwN8A/57tig5p1lVgIVj6cL+rQyHjcxdOY+22dn6p1rqIFKjhtNTPANa4+1p37wEeAa7Yr4wDVZnX1cCRT83KsXDcu2DpI/2OWQe4+OSxnDCmku8987pa6yJSkIYT6hOAt/q8b8os6+s24KOZe5g+BdzU3xeZ2TwzW2Rmi5qbmw+hukOYfTXs3gBvPNvv6lAo6Ftf29zOr5aptS4ihSdbJ0rnAve7eyPBTah/YmYHfLe73+Puc9x9TkNDQ5Y23cfxl0JpNSzpvwsG4NIZYzl+TAX/pr51ESlAwwn1DcDEPu8bM8v6+iQwH8Dd/wSUAvXZqOBBiZZmxqz/Erpa+i3S21pfs7WNp5ZvOsIVFBE5vIYT6i8C08xsipnFCE6EPrFfmfXAhQBmdiJBqB+G/pVhmH0NJDth5c8HLHLZjHFMG13Bvz3zOmm11kWkgAwZ6u6eBD4LPA2sIhjlstLMbjezyzPFvgRcb2ZLgYeBa909N2k54TSoP37AUTAQtNZvunAaq7e08esVm49g5UREDq9h9am7+1Pufry7H+vu38gs+3t3fyLz+hV3f4e7n+Lus939t4ez0oPqnWd9/Z9g+18HLPaemeM4tqGc7/5+tVrrIlIwCuOK0v0NMWYdIJzpW1+9pY3frFRrXUQKQ2GGetV4mHp+Zsz6wHO9vHfWeKY2lPO936tvXUQKQ2GGOgRdMC1vwZvPDVgkHDJuuuA4Xt3cym9fUWtdRPJf4Yb69PdASTX8z62we+ALjd43azxT6sv57u81bl1E8l/hhnq0DK74PjSvhrvPgbV/6LdYJBzi8++axqpNu5l7z5/ZsKvzyNZTRCSLCjfUAU66HOYtgPgoeOD98Ox3+u1jv/yU8fzzR05h5cYWLv3XhbooSUTyVmGHOkDDCXD9MzDzI7DgDnjoI9C+fZ8iZsaH3tbIkzefw5T6cj7z4GL+7+PL6OhJ5qjSIiKHpvBDHaCkAj54D7znTnhjIfzHudC06IBik+vL+e///b+44bxjeeTFt3jfvz3Pyo39TzcgInI0Ko5Qh+CipNM/CZ/8LYRCcN8l8MJ/wH4XvsYiIW65ZDr/9ckzae1K8oEf/JEfPv8GubpAVkTkYBRPqPcafyp8eiEcdyH8+v/Af18LXbsPKPaO4+r5zefP5dzj6/n6r17huvtfpLm1+8jXV0TkIBRfqAOU1cLfPAzvug1WPQH/eT5sWXlAsbryGP/58TncfsXJ/PGv27n0u8/x7OrczFMmIjIcxRnqEHTBnP0F+MQvobsV7jkfnrgJNq/Yp5iZ8fG3T+aXnz2buvIon7jvL3zh0SUsXr9TXTIictSxXAXTnDlzfNGiA09W5kTrFljwDVg2P5i295iz4cxPwwmXQTiyp1hXIsU///Y1HnphPe09KU4cV8XVZ07i/bPHU1kazeEOiEixMLOX3H3OgOsV6n107ICXfwJ/uRda1kP1xODk6mmfgHjdnmJt3Ul+sWQDD/55Pa9s2k08FuaK2RO45sxJzJhQncMdEJFCp1A/FOkUvPZreOHuYO6YSCnMuhLO+DSMnbGnmLuztKmFB/+8jl8u20hXIs0pjdVcc+YxvPeUccRjkUE2IiJy8BTqI7VlZTD0sW/XzKnXwHHvgorRe4q1dCR4/OUmHnphPa9vbaOyJML7T53AJTPGcvrkOmKR4j19ISLZk5VQN7NLgO8CYeBed/9WP2WuBG4DHFjq7lcP9p15E+q99u+aARg3G6a9G457NzTOgVAYd+fFN3fy4Avr+PWKzfQk01SWRDj3+AYuPHE0550wmrryWG73RUTy1ohD3czCwGrg3UATwT1L57r7K33KTCO48fQF7r7TzEa7+9bBvjfvQr1XOg2bl8Ga38Hr/wNNfwFPB8Mkj70gCPjj3gUVDbR3J/l/a7bx+1Vbeea1rTS3dhMyOG1SLRecOJoLp4/h+DEVmFmu90pE8kQ2Qv3twG3ufnHm/f8FcPdv9inzbWC1u9873Irlbajvr2MHrF0QBPya30F7Zhz7+FODcG88AyacRrpsFCs2tvD7VVv5/atbWLEhuOCpsbaMC6eP5tzjGzh9Sh1VGkUjIoPIRqh/GLjE3T+Vef8x4Ex3/2yfMj8naM2/g6CL5jZ3/00/3zUPmAcwadKkt61bt+7g9+holk7D5qV7A77pxaAVD1BzDEx4257H5vITWLC2jd+v2sLza7bRlUgTMpgxoZqzpo7i7VNHMWdyrYZKisg+jlSo/wpIAFcCjcBCYKa77xroewumpT6Y7lbYtBQ2vJR5LA7uxgRgYRh9EjS+jcTYU1nlk/nDjjqeX9fOkvW76EkFIT9zQjVnHTuKs6aO4vTJdVSUaESNSDEbKtSHkxAbgIl93jdmlvXVBLzg7gngDTNbDUwj6H8vXiWVMPns4NGrdQtsXLw36Ff+jOhL9zMLmIVxc+0xpE6azqbYMSzvHscfdo7ioeeb+Y9nY4RDxowJ1cxurGZWYw2nTKxhan05oZD65EUkMJyWeoSga+VCgjB/Ebja3Vf2KXMJwcnTT5hZPfAyMNvdt/f3nVAkLfXhcIcda2HLCtj6KjSvCp63r4F0IiiC0VUxkY3RY1jWM56FreN4KXEM6300lSVRZkyoZtbEamY31jBrYg3jq0t18lWkQI24pe7uSTP7LPA0QX/5fe6+0sxuBxa5+xOZdReZ2StACvjyYIEufZjBqGODx0lX7F2eSgRhv3UV1vwqZVtXcWzzaxzb8ic+EEpCCSQiFTSVHMfSHZN5bv147kxNZq2Po7aijFmNNZw8vorpY6s4YWwlk0fFiYQ1Vl6k0Onio3yT7Iatq4K++k1Lg+GVm1cEF0YByVApTSXHsiw5icUdo2nyejZ4PVvCYxg3ejQnjK3kxEzQTx9bSUNliVr1InlEV5QWg1QStr8Om5btG/bd+84T327lbPR63kzV7wn7XbGxROuOIT5mKqNHj2NyQwVT6suZVBenNBrO0Q6JyEAU6sXKHdq3BVe/7noLdq0PRt7sWk9yx3poWU8k0bbPR1q9jCavp8lH85Y30Fo6nlT1RCKjplA97jjGjxnNMaPiTKgpo1yjcERyIhujXyQfmUFFQ/CY8LZ9VkUgCP2uXXsDf9c6YtveZHzzWibsWsc7214hluyE7QSP1bDTK9jitSyngrZQFYmSOojXES4fRVl1AxV1Y6gZNZaGMeOoqB0LpVU52HGR4qZQL1ZmwdQGZbUwbhYAJZkHEIR+xw7Y9SbsXEfH1jdIbv0rVS1bqOrYTrhrC6WJ16jYuZvwznQwqHU/rZSzIzqW1tLxdFc0Qs0kYqMmUz52KrXjj6OmdpT680WyTKEu/TOD8lHBY8LbiAPx/sql03h3Czu3bWbb1s20bNtM666tJFo2E2vbQEXnRupa32TK7r9Qvmnfe7y2eDlbw6PpjNSQipZDrJxQSQXhsipiZZWUxKuIV1ZTXlFNWUUVVlIJpTVQVhM8x8qDeorIHgp1GZlQCCurpW5iLXUTTxywWCKZYnPzJnZu/CudW9eS2LGOcMtblLY3UdrTQqRzByXtHZR6F+V0UWqJITedsjA9kSqSsWq8tBorqyVSXkusso5wfBTER0F5fXCDk9738VEQLcvmv4DIUUWhLkdENBJm7LhGxo5rBN45YLlkKs2O9h7W7m5nV8suWlp20ra7hfbWXXS1tpDs2Il37sK6dhHpaaEi0UZ1VzvVu9uptvVUs4pqa6fa2gnR/yCAZLiMREkt6bI6LD6KcEUD0ap6QuX1meCv73NAGBV0UYU0Ekjyg0JdjiqRcIjRVaWMriqFxlGDlnV3WruTbG/rYXtbN5vaelje1s22tm52tXXS1bqDVNs2vGM74c4dRLp3UpXeTW2ylbqeVupad1Nnb1HLKuqslUrr7H87GN3RalLRSjxWDtE4oZKgqyhSWkG4tAKLBd1HRONQUhF0D5XWQGn13u6ishoIa4I2ObwU6pK3zIyq0ihVpVGm1JcPWd7d6Uyk2NHew872BNvbu3mjo4eXOxLs7EjQ2tZGsnUb6fZtWMd2It07KOneSTzVQl1yNxVdncTpJk4XcWsNnumm3LqIWw9xuoasQyoSJxWrgtIaLF5DKFZGKBzFLBz8NRCK7PcIZZ6jQbdRNB48x+J7X+/znDmolFQFj5CuIi42CnUpGmZGPBYhHovQWDv8zyVSaXZ1JNjdlWB3Z4LdXUk2dva+T+5Z3trZQ1dHG4mO3dC1C+tuIdy9m3i6NegSop2qZEfQXdTaTrW1EGMbEdLEQmmiliZiTsRSREkTsTRhUoRJEfEU0XQXYR/6XMM+YpXBXwulmZAvrQre977es6zmwGUlVRCr0IEhzyjURYYQDYdoqCyhobJk6ML96EqkMgeDBC2dew8EWzoTtHYlae8OHm3dqeB1T5K27t7lqT2vk2knTIpSeiijh1Lrpoweysg8Wzdx62ZUpJu6cBd14S6qQwt0W5wAAAfUSURBVJ1UJzqoSnZS0dpOua8j7m2UptopSbUR9uSgdXcLQUklFi2HSEnwF0GkBCJl/b+PlAZdTOEohGPBc6jP677Lo/FgJtNYRfDc+zpaplFNI6BQFznMSqNhSqPh4DzBIXJ3upPpPQHfezBo6/No707S1pWkvSfFlu4ka3tSew4YHT17Dxjt3Snae5K4OyUkqKKdKuugkk4qrYMqOqi0Diozz1WJDipDCeLhBHFLUBZKUmZtlLCDUhLE6KGEHqLeQ8R7CHuSsCcJeerQdtbCe7uQYhXBuYpISeaAUAKRWObAkFkWKcm8731EgudQdN8DSSjaZ10EyBw49jmA2IEvo+WZv3Yyj6N8KK1CXSQPmNmeg0N9xaH9xdBX7/mFjp4UHZmQ7+hJ0ZEJ/Y6e4ODQ2ZNkR3eKDZn1nT2ZzySCdb3L9nxXT5JEKhh1ZKSJkiJKMvMIXkcsRYwEZfRQYZ3Uhbupi3RTHe6mJtxFVaiLSuuiItlJebKLeEcXMdoz35Eg6gnCniTiCcKeIJxOEPIEoVQPoSH+8sgKC+8b8n27t8KxfQ9A4VjmILTfAWj0SXsu+ss2hbpIEep7foGK7H53IpXeE/C9od/enaQjERxA9ixP7D0gdPak2JxI8UY/yzsTKboSwXN3Ik1PKj3wfpEmQjo4ePQ5kMQsRWXUqYg65ZE08QjEwkYsEiIaDlESCRELG9GwEQuHiUWMWDhYFrcu4t5OebqdeLqNslQbJak2SpKtlCRbiba1Etm5kUiyHUv1EEr1YOkEpLoxH6CuZ39BoS4i+SEaDlFdFqK67PAM30ym0nQl00HQ96ToTqbo7EnvE/5dmb8c9jkoZP7C6Mos25VM050MDhTdPcHrrkRmWTJNdyJNVzKFexyoO6S6hkgTDyWDA0o4RWU0TTyc5hKbxkez+8+yh0JdRPJKJByiIhw6IvfrdXeSac+EfIqeVBD23ZkDQk9y7+ve5V2J1AHPvQeL3ueKujGHrc7D+lfJ3K7uuwR3PrrX3b81QLkPAT8FTnd3zasrInnNLOiSiR6hg0g2DDkA1czCwA+AS4GTgLlmdlI/5SqBzwEvZLuSIiIyPMO5quAMYI27r3X3HuAR4Ip+yn0d+EcYxmV1IiJyWAwn1CcAb/V535RZtoeZnQZMdPcnB/siM5tnZovMbFFzc/NBV1ZERAY34ut/zSwE3Al8aaiy7n6Pu89x9zkNDQ0j3bSIiOxnOKG+AZjY531jZlmvSmAG8AczexM4C3jCzAa8h56IiBwewwn1F4FpZjbFzGLA3wBP9K509xZ3r3f3ye4+GfgzcLlGv4iIHHlDhrq7J4HPAk8Dq4D57r7SzG43s8sPdwVFRGT4hjXw0t2fAp7ab9nfD1D2vJFXS0REDoW593/Lr8O+YbNmYN0hfrwe2JbF6hwNCm2fCm1/oPD2qdD2Bwpvn/rbn2PcfcCRJjkL9ZEws0XuXlAnYgttnwptf6Dw9qnQ9gcKb58OZX90SxMRkQKiUBcRKSD5Gur35LoCh0Gh7VOh7Q8U3j4V2v5A4e3TQe9PXvapi4hI//K1pS4iIv1QqIuIFJC8C3Uzu8TMXjOzNWb2lVzXJxvM7E0zW25mS8ws76ZXMLP7zGyrma3os6zOzH5nZq9nnmtzWceDNcA+3WZmGzK/0xIzuyyXdTwYZjbRzBaY2StmttLMPpdZnpe/0yD7k8+/UamZ/cXMlmb26R8yy6eY2QuZzHs0M13LwN+TT33qmRt2rAbeTTAF8IvAXHd/JacVG6HMRGhz3D0vL5ows3OBNuABd5+RWfZtYIe7fytz8K1191tyWc+DMcA+3Qa0ufs/5bJuh8LMxgHj3H1x5oY2LwHvB64lD3+nQfbnSvL3NzKg3N3bzCwKPE9w46EvAo+7+yNmdjew1N3vGuh78q2lPtwbdsgR5O4LgR37Lb4C+HHm9Y8J/sPljQH2KW+5+yZ3X5x53Uowj9ME8vR3GmR/8pYH2jJvo5mHAxcQ3CYUhvEb5VuoD3nDjjzlwG/N7CUzm5frymTJGHfflHm9GTh8d9o9sj5rZssy3TN50VWxPzObDJxKcOvJvP+d9tsfyOPfyMzCZrYE2Ar8DvgrsCszsSIMI/PyLdQL1dnufhrBfWBvzPzpXzA86OPLn36+gd0FHAvMBjYB/5zb6hw8M6sAHgM+7+67+67Lx9+pn/3J69/I3VPuPpvgvhVnANMP9jvyLdSHumFHXnL3DZnnrcDPCH7MfLcl0+/Z2/+5Ncf1GTF335L5T5cG/pM8+50y/bSPAQ+6++OZxXn7O/W3P/n+G/Vy913AAuDtQI2Z9c6oO2Tm5VuoD3rDjnxkZuWZEz2YWTlwEbBi8E/lhSeAT2RefwL4RQ7rkhW94ZfxAfLod8qchPshsMrd7+yzKi9/p4H2J89/owYzq8m8LiMYELKKINw/nCk25G+UV6NfADJDlP4VCAP3ufs3clylETGzqQStcwjmt38o3/bJzB4GziOYJnQLcCvwc2A+MIlgiuUr3T1vTjwOsE/nEfxZ78CbwKf79Ecf1czsbOA5YDmQziz+KkE/dN79ToPsz1zy9zeaRXAiNEzQ4J7v7rdnMuIRoA54Gfiou3cP+D35FuoiIjKwfOt+ERGRQSjURUQKiEJdRKSAKNRFRAqIQl1EpIAo1EVECohCXUSkgPx/m7Fvw6VgEOoAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "0mnngrfOjWkp", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "105452fc-8900-403a-8c55-998872332791" }, "source": [ "predictions = model.predict(X_test_scaled)\n", "print('R2:', r2_score(y_test, predictions))\n", "print('MSE:', mean_squared_error(y_test, predictions))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "R2: 0.6650092010460874\n", "MSE: 0.4396888593756206\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "mD88HcyQjohR" }, "source": [ "## 3.4: Creating a custom dense layer\n", "\n", "Now let's create a custom layer with its own weights. \n", "\n", "\n", "Use the following template to create a MyDense layer that computes $\\phi(\\mathbf{X} \\mathbf{W}) + \\mathbf{b}$, \n", "\n", "where $\\phi$ is the (optional) activation function, \n", "\n", "$\\mathbf{X}$ is the input data, \n", "\n", "$\\mathbf{W}$ represents the kernel (i.e., connection weights), \n", "\n", "and $\\mathbf{b}$ represents the biases, \n", "\n", "then train and evaluate a model using this instead of a regular Dense layer.\n", "\n" ] }, { "cell_type": "code", "metadata": { "id": "T-xPJsVAk51k" }, "source": [ "class CustomDense(tf.keras.layers.Layer):\n", "\n", " def __init__(self, units, activation=None, initializer='uniform', **kwargs):\n", " self.units = units\n", " self.activation = tf.keras.layers.Activation(activation)\n", " self.initializer = initializer\n", " super(CustomDense, self).__init__(**kwargs)\n", "\n", " def build(self, input_shape):\n", " self.kernel = self.add_weight(name='kernel', \n", " shape=(input_shape[1], self.units),\n", " initializer=self.initializer,\n", " trainable=True)\n", " self.biases = self.add_weight(name='bias', \n", " shape=(self.units,),\n", " initializer='zeros',\n", " trainable=True)\n", " super(CustomDense, self).build(input_shape)\n", "\n", " def call(self, X):\n", " return self.activation(X @ self.kernel + self.biases)" ], "execution_count": 16, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "R9UDrR0ulSYh" }, "source": [ "## 3.5 Build NN Model with custom dense layer" ] }, { "cell_type": "code", "metadata": { "id": "pUT3X8rbla8S", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "3f743079-0232-4bb6-a32d-dadd82e666c4" }, "source": [ "model = tf.keras.models.Sequential([\n", " CustomDense(16, activation=\"relu\", \n", " input_shape=(X_train.shape[1],)),\n", " CustomDense(32, activation=\"relu\"),\n", " CustomDense(32, activation=\"relu\"),\n", " CustomDense(1),\n", " softplus_fn\n", "])\n", "\n", "model.compile(loss=mse_loss, \n", " optimizer=tf.keras.optimizers.SGD(lr=1e-3),\n", " metrics=['mean_squared_error'])\n", "\n", "model.summary()" ], "execution_count": 21, "outputs": [ { "output_type": "stream", "text": [ "Model: \"sequential_1\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "custom_dense_8 (CustomDense) (None, 16) 144 \n", "_________________________________________________________________\n", "custom_dense_9 (CustomDense) (None, 32) 544 \n", "_________________________________________________________________\n", "custom_dense_10 (CustomDense (None, 32) 1056 \n", "_________________________________________________________________\n", "custom_dense_11 (CustomDense (None, 1) 33 \n", "_________________________________________________________________\n", "lambda (Lambda) (None, 1) 0 \n", "=================================================================\n", "Total params: 1,777\n", "Trainable params: 1,777\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "3xwwvACNlrFE", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "30756fbb-98fb-41d7-d4b7-c3211fdb1167" }, "source": [ "history = model.fit(X_train_scaled, y_train, \n", " epochs=30,\n", " batch_size=32,\n", " validation_split=0.1)" ], "execution_count": 22, "outputs": [ { "output_type": "stream", "text": [ "Epoch 1/30\n", "407/407 [==============================] - 2s 2ms/step - loss: 3.0793 - mean_squared_error: 3.0793 - val_loss: 2.4810 - val_mean_squared_error: 2.4810\n", "Epoch 2/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 2.2862 - mean_squared_error: 2.2862 - val_loss: 1.8639 - val_mean_squared_error: 1.8639\n", "Epoch 3/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.7079 - mean_squared_error: 1.7079 - val_loss: 1.5399 - val_mean_squared_error: 1.5399\n", "Epoch 4/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.4233 - mean_squared_error: 1.4233 - val_loss: 1.4153 - val_mean_squared_error: 1.4153\n", "Epoch 5/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3991 - mean_squared_error: 1.3991 - val_loss: 1.3769 - val_mean_squared_error: 1.3769\n", "Epoch 6/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3471 - mean_squared_error: 1.3471 - val_loss: 1.3660 - val_mean_squared_error: 1.3660\n", "Epoch 7/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3472 - mean_squared_error: 1.3472 - val_loss: 1.3629 - val_mean_squared_error: 1.3629\n", "Epoch 8/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3136 - mean_squared_error: 1.3136 - val_loss: 1.3621 - val_mean_squared_error: 1.3621\n", "Epoch 9/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3288 - mean_squared_error: 1.3288 - val_loss: 1.3619 - val_mean_squared_error: 1.3619\n", "Epoch 10/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3391 - mean_squared_error: 1.3391 - val_loss: 1.3618 - val_mean_squared_error: 1.3618\n", "Epoch 11/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.2816 - mean_squared_error: 1.2816 - val_loss: 1.3617 - val_mean_squared_error: 1.3617\n", "Epoch 12/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3057 - mean_squared_error: 1.3057 - val_loss: 1.3617 - val_mean_squared_error: 1.3617\n", "Epoch 13/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3699 - mean_squared_error: 1.3699 - val_loss: 1.3617 - val_mean_squared_error: 1.3617\n", "Epoch 14/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3347 - mean_squared_error: 1.3347 - val_loss: 1.3616 - val_mean_squared_error: 1.3616\n", "Epoch 15/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3319 - mean_squared_error: 1.3319 - val_loss: 1.3616 - val_mean_squared_error: 1.3616\n", "Epoch 16/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3409 - mean_squared_error: 1.3409 - val_loss: 1.3615 - val_mean_squared_error: 1.3615\n", "Epoch 17/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3386 - mean_squared_error: 1.3386 - val_loss: 1.3615 - val_mean_squared_error: 1.3615\n", "Epoch 18/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3259 - mean_squared_error: 1.3259 - val_loss: 1.3615 - val_mean_squared_error: 1.3615\n", "Epoch 19/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3107 - mean_squared_error: 1.3107 - val_loss: 1.3614 - val_mean_squared_error: 1.3614\n", "Epoch 20/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3078 - mean_squared_error: 1.3078 - val_loss: 1.3614 - val_mean_squared_error: 1.3614\n", "Epoch 21/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3232 - mean_squared_error: 1.3232 - val_loss: 1.3613 - val_mean_squared_error: 1.3613\n", "Epoch 22/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3469 - mean_squared_error: 1.3469 - val_loss: 1.3613 - val_mean_squared_error: 1.3613\n", "Epoch 23/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3519 - mean_squared_error: 1.3519 - val_loss: 1.3613 - val_mean_squared_error: 1.3613\n", "Epoch 24/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3291 - mean_squared_error: 1.3291 - val_loss: 1.3612 - val_mean_squared_error: 1.3612\n", "Epoch 25/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3221 - mean_squared_error: 1.3221 - val_loss: 1.3611 - val_mean_squared_error: 1.3611\n", "Epoch 26/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3192 - mean_squared_error: 1.3192 - val_loss: 1.3611 - val_mean_squared_error: 1.3611\n", "Epoch 27/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3165 - mean_squared_error: 1.3165 - val_loss: 1.3610 - val_mean_squared_error: 1.3610\n", "Epoch 28/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3307 - mean_squared_error: 1.3307 - val_loss: 1.3609 - val_mean_squared_error: 1.3609\n", "Epoch 29/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3577 - mean_squared_error: 1.3577 - val_loss: 1.3609 - val_mean_squared_error: 1.3609\n", "Epoch 30/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3731 - mean_squared_error: 1.3731 - val_loss: 1.3607 - val_mean_squared_error: 1.3607\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "0LOkyDW9l87k", "colab": { "base_uri": "https://localhost:8080/", "height": 282 }, "outputId": "af44aaa4-0c91-44c0-d505-c87d8a29842a" }, "source": [ "history_df = pd.DataFrame(history.history)\n", "history_df[['loss', 'val_loss']].plot(kind='line')" ], "execution_count": 23, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 23 }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAD4CAYAAAATpHZ6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3df3xcdZ3v8dcnySSTNj/bpknatEkrpAVa+RVcsICgV0AUEQS7iAhcgV3lIqxcLy67rPjrsWqv6N7rSrcXEHUrUmmvggjo1WqpKJDWltIWyq+2pKRt0jb9nTY/PvePmaRJOkkmzaSTc+b9fDz6mMmZ75zzORl4zzff8z3nmLsjIiLhkJXuAkREJHUU6iIiIaJQFxEJEYW6iEiIKNRFREIkJ10bnjBhgtfU1KRr8yIigbRixYpmdy/r7/W0hXpNTQ319fXp2ryISCCZ2aaBXtfwi4hIiCjURURCRKEuIhIiaRtTF5HM1NbWRkNDA62trekuZVSLRqNUVVURiUSG9D6FuogcVw0NDRQWFlJTU4OZpbucUcnd2bFjBw0NDUybNm1I79Xwi4gcV62trYwfP16BPgAzY/z48cf014xCXUSOOwX64I71dxS4UH9l6x6+/fQr7D7Qlu5SRERGncCF+qYdB/jBH95g884D6S5FRAKqoKAg3SWMmMCF+qTifAAadx9McyUiIqPPoKFuZlPMbKmZrTOztWZ2e4I2xWb2hJmtjre5cWTKhYriKACNuzUdSkSGx9354he/yKxZs5g9ezaPPvooAI2NjZx//vmcdtppzJo1i2effZaOjg5uuOGG7rbf/e5301x9YslMaWwH7nT3lWZWCKwws9+6+7oebW4F1rn7ZWZWBrxqZgvd/XCqCx4/NpdItinURULgK0+sZd07e1K6zpMnFfHly05Jqu2SJUtYtWoVq1evprm5mbPOOovzzz+fn/70p1x88cX80z/9Ex0dHRw4cIBVq1axZcsWXn75ZQBaWlpSWneqDNpTd/dGd18Zf74XWA9M7tsMKLTY4doCYCexL4OUy8oyyouibNXwi4gM0/Lly7nmmmvIzs6mvLyc973vfbz44oucddZZ/PCHP+Tee+9lzZo1FBYWMn36dN58801uu+02nn76aYqKitJdfkJDOvnIzGqA04Hn+7z0feBx4B2gEJjr7p0J3n8LcAvA1KlTh15tXGVxVD11kRBItkd9vJ1//vksW7aMJ598khtuuIEvfOELfPrTn2b16tU888wzzJ8/n0WLFvHQQw+lu9SjJH2g1MwKgMXAHe7e9++li4FVwCTgNOD7ZnbU15i7L3D3OnevKyvr93LAg6oszmfrHoW6iAzPeeedx6OPPkpHRwdNTU0sW7aM97znPWzatIny8nJuvvlmbrrpJlauXElzczOdnZ18/OMf5+tf/zorV65Md/kJJdVTN7MIsUBf6O5LEjS5Efimuzvwupm9BcwEXkhZpT1UFkd5em0r7q6TGETkmF1xxRX8+c9/5tRTT8XM+Pa3v01FRQU/+tGPmDdvHpFIhIKCAn784x+zZcsWbrzxRjo7Y4MQ//qv/5rm6hMbNNTj4+QPAuvd/b5+mm0GPgA8a2blwAzgzZRV2UdFcZTD7Z3s3H+Y8QV5I7UZEQmpffv2AbGzNufNm8e8efN6vX799ddz/fXXH/W+0do77ymZnvoc4DpgjZmtii+7G5gK4O7zga8BD5vZGsCAu9y9eQTqBWI9dYhNa1Soi4gcMWiou/tyYkE9UJt3gItSVdRgKuInIG3d3cqsycXHa7MiIqNe4M4oBZjU1VPXwVIRkV4CGerjC/LIyTIaWzRXXUSkp0CGenb3CUjqqYuI9BTIUIfYDBidgCQi0lugQ10nIImI9BbYUJ9UHKVx90Fi5zuJiIyMga69vnHjRmbNmnUcqxlcYEO9ojif1rZOWnQHJBGRbkO6oNdo0vMEpNKxuWmuRkSOyVNfgq1rUrvOitnwoW/2+/KXvvQlpkyZwq233grAvffeS05ODkuXLmXXrl20tbXx9a9/ncsvv3xIm21tbeWzn/0s9fX15OTkcN9993HhhReydu1abrzxRg4fPkxnZyeLFy9m0qRJfOITn6ChoYGOjg7uuece5s6dO6zd7hLYUO+6WcbWPQc5edLovASmiIw+c+fO5Y477ugO9UWLFvHMM8/w+c9/nqKiIpqbmzn77LP56Ec/OqRrS/37v/87ZsaaNWt45ZVXuOiii9iwYQPz58/n9ttv59prr+Xw4cN0dHTw61//mkmTJvHkk08CsHv37pTtX2BD/cht7XSwVCSwBuhRj5TTTz+d7du3884779DU1ERpaSkVFRX8wz/8A8uWLSMrK4stW7awbds2Kioqkl7v8uXLue222wCYOXMm1dXVbNiwgXPOOYdvfOMbNDQ0cOWVV3LiiScye/Zs7rzzTu666y4+8pGPcN5556Vs/wI7pl5WmEd2ltHYolAXkaG5+uqreeyxx3j00UeZO3cuCxcupKmpiRUrVrBq1SrKy8tpbU1Ntnzyk5/k8ccfJz8/n0svvZTf//731NbWsnLlSmbPns0///M/89WvfjUl24IA99Szs4yJhXnqqYvIkM2dO5ebb76Z5uZm/vjHP7Jo0SImTpxIJBJh6dKlbNq0acjrPO+881i4cCHvf//72bBhA5s3b2bGjBm8+eabTJ8+nc9//vNs3ryZl156iZkzZzJu3Dg+9alPUVJSwgMPPJCyfQtsqEPXXHVdKkBEhuaUU05h7969TJ48mcrKSq699louu+wyZs+eTV1dHTNnzhzyOj/3uc/x2c9+ltmzZ5OTk8PDDz9MXl4eixYt4ic/+QmRSISKigruvvtuXnzxRb74xS+SlZVFJBLh/vvvT9m+WbrmedfV1Xl9ff2w1vG5hSt4Zetefn/nBakpSkRG3Pr16znppJPSXUYgJPpdmdkKd6/r7z2BHVOH+G3tdrfqBCQRkbhAD79UFkc5cLiDPQfbKR4TSXc5IhJSa9as4brrruu1LC8vj+effz5NFfUv0KFe0X1d9YMKdZEACdr9hWfPns2qVasGb5hCxzoCMejwi5lNMbOlZrbOzNaa2e39tLvAzFbF2/zxmKoZop5nlYpIMESjUXbs2KFh0wG4Ozt27CAajQ75vcn01NuBO919pZkVAivM7Lfuvq6rgZmVAD8ALnH3zWY2cciVHIOet7UTkWCoqqqioaGBpqamdJcyqkWjUaqqqob8vmTuUdoINMaf7zWz9cBkYF2PZp8Elrj75ni77UOu5BhMLMwjy9RTFwmSSCTCtGnT0l1GaA1p9ouZ1QCnA32PDtQCpWb2BzNbYWaf7uf9t5hZvZnVp+JbOpKdRVlhnm5rJyISl3Som1kBsBi4w9339Hk5BzgT+DBwMXCPmdX2XYe7L3D3OnevKysrG0bZR1QU5+tmGSIicUnNfjGzCLFAX+juSxI0aQB2uPt+YL+ZLQNOBTakrNJ+VBZFeb1p30hvRkQkEJKZ/WLAg8B6d7+vn2a/BM41sxwzGwP8DbA+dWX2r6JYN6AWEemSTE99DnAdsMbMuiZq3g1MBXD3+e6+3syeBl4COoEH3P3lkSi4r0klUfYdamdvaxuFUc1VF5HMlszsl+XAoGcJuPs8YF4qihqKih7XVVeoi0imC/S1X0AnIImI9BT4UK8oit/WbremNYqIBD7Uy4uimE5AEhEBQhDquTlZTCjI0wwYERFCEOoQG1d/R6EuIhKOUK8oimpMXUSEkIR6ZXFUY+oiIoQl1Evy2dvazr5D7ekuRUQkrcIR6sVd0xrVWxeRzBa8UH/l1zDvBNi1sXtR11z1Ro2ri0iGC16o546F/U2wa1P3osoelwoQEclkwQv10urYY8uRUC8vzgM0/CIiErxQL6oCy+7VU8/LyWZCQa566iKS8YIX6tk5UDy5V08duq6rrjF1EclswQt1gJLqXj11gIqifPXURSTjBTPUS6uP6qnrBCQRkaCGekkN7NsGbUeGWypLouw+2MaBwzoBSUQyVzL3KJ1iZkvNbJ2ZrTWz2wdoe5aZtZvZVakts4/uGTCbuxfpBCQRkeR66u3Ane5+MnA2cKuZndy3kZllA98CfpPaEhMoiYd6j3H1iqLYXHWFuohkskFD3d0b3X1l/PleYD0wOUHT24DFwPaUVphIgrnqXT11XYJXRDLZkMbUzawGOB14vs/yycAVwP2pKmxABeWQE+19qYBi3dZORCTpUDezAmI98TvcfU+fl78H3OXunYOs4xYzqzez+qampqFXe2RFUDK1V089Gslm3FidgCQimS0nmUZmFiEW6AvdfUmCJnXAz8wMYAJwqZm1u/svejZy9wXAAoC6ujofTuGJ56pHNaYuIhlt0FC3WFI/CKx39/sStXH3aT3aPwz8qm+gp1xpNTS80GuR5qqLSKZLpqc+B7gOWGNmq+LL7gamArj7/BGqbWAl1dC6Gw62QH4JEBtXX7l5V1rKEREZDQYNdXdfDliyK3T3G4ZTUNJ6zoCJh3plcZRdB9pobesgGsk+LmWIiIwmwTyjFBLOVe+6rrrG1UUkUwU31AeYq65xdRHJVMEN9fxSyCvufVZp11z1PZqrLiKZKbihDlDae656V6i/06KeuohkpmCHep+56mNycyjOj2hMXUQyVvBDvWUz+JHzmDRXXUQyWbBDvbQa2g/CviPXEKssjmpMXUQyVrBDveToGTAVxfkafhGRjBXsUC9NNFc9SvO+wxxq70hTUSIi6RPsUC+ZGnts2di9qGuu+rbdh9JQkIhIegU71HPHwtiyhGeVNuq66iKSgYId6hCfAZPoBCSNq4tI5gl+qJdWJzyrVCcgiUgmCn6ol1TD7gboaAegIC+HwmiObmsnIhkp+KFeWg3eAXu2dC+aVJyvE5BEJCMFP9QTzlWPakxdRDJS8EO9n7nq6qmLSCYKfqgXTwHLOqqn3rzvEIfbO9NYmIjI8TdoqJvZFDNbambrzGytmd2eoM21ZvaSma0xs+fM7NSRKTeB7AgUTT6qp+4O2zQEIyIZJpmeejtwp7ufDJwN3GpmJ/dp8xbwPnefDXwNWJDaMgfRZ656923tFOoikmEGDXV3b3T3lfHne4H1wOQ+bZ5z913xH/8CVKW60AH1mauu29qJSKYa0pi6mdUApwPPD9DsM8BT/bz/FjOrN7P6pqamoWx6YCXVsG8rtMXmpnefVaq56iKSYZIOdTMrABYDd7j7nn7aXEgs1O9K9Lq7L3D3OnevKysrO5Z6E+u+CfXbABRGIxTk5eisUhHJOEmFuplFiAX6Qndf0k+bdwMPAJe7+47UlZiEBHPVK4ujuq66iGScZGa/GPAgsN7d7+unzVRgCXCdu29IbYlJ6J6rvrF7UUVxlEYdKBWRDJOTRJs5wHXAGjNbFV92NzAVwN3nA/8CjAd+EPsOoN3d61Jfbj8KKiA776ie+oZtKRy3FxEJgEFD3d2XAzZIm5uAm1JV1JBlZUHJlD5Xa8xn+95DtHV0EskO/jlWIiLJCE/aHTVXPXYC0va9ugOSiGSO8IR6P3PVNa1RRDJJeEK9pBpaW6B1N9DztnY6WCoimSM8od7nao1HTkBSqItI5ghPqPeZq14UzWFMbrZ66iKSUcIT6qU1scd4T93MqCyO0rDrQPpqEhE5zsIT6vmlkFvYawbMiRMLeW37vjQWJSJyfIUn1M2OmgFTW17Axub9tLZ1pLEwEZHjJzyhDkfNVa+tKKTT4Y0m9dZFJDOEK9RLq6FlM7gDMKO8EIDXtinURSQzhCvUS6qh7QDsj13zpWbCWCLZxqvb9qa5MBGR4yNcod5nrnokO4vpEwrYsFWhLiKZIVyhnuC66rUVheqpi0jGCFmoT4099riu+ozyAhp2HWT/ofb01CQichyFK9TzCmDMhN499a6DpZqvLiIZIFyhDkfNVZ9REQt1jauLSCYIX6j3mas+pXQM0UiWxtVFJCMkc4/SKWa21MzWmdlaM7s9QRszs/9lZq+b2UtmdsbIlJuE0mrY3QCdsbNIs7KMEycWskGhLiIZIJmeejtwp7ufDJwN3GpmJ/dp8yHgxPi/W4D7U1rlUJRUQ2c77NnSvai2vJBXNfwiIhlg0FB390Z3Xxl/vhdYD0zu0+xy4Mce8xegxMwqU15tMvrMVQeYUVHA9r2HaDlwOC0liYgcL0MaUzezGuB04Pk+L00G3u7xcwNHBz9mdouZ1ZtZfVNT09AqTVaiuerxGTAbdLkAEQm5pEPdzAqAxcAd7r7nWDbm7gvcvc7d68rKyo5lFYMrngJYn6s1xkJdB0tFJOySCnUzixAL9IXuviRBky3AlB4/V8WXHX85uVA0uVdPvbI4SmFejqY1ikjoJTP7xYAHgfXufl8/zR4HPh2fBXM2sNvdG1NY59D0matuZrpcgIhkhJwk2swBrgPWmNmq+LK7gakA7j4f+DVwKfA6cAC4MfWlDkFJNby5tNei2vJCnn65EXcn9j0lIhI+g4a6uy8HBkxBd3fg1lQVNWyl1bC3EdpaIRIFYndBeuSFNpr2HWJiYTTNBYqIjIzwnVEKR2bA7D4yIafrhhkbtmoGjIiEVzhDPcFc9doKzYARkfALZ6h3z1Xf2L1oQkEe48fmagaMiIRaOEO9sBKyc3v11CF2sHTDdoW6iIRXOEM9Kyt2ElJL31CP3drO4zemFhEJm3CGOhw1Vx1i4+r7D3ewpeVgmooSERlZ4Q31PtdVhx4zYHSwVERCKryhXloNB3dB65HL1JzYdQ0YTWsUkZAKb6gnuFpjcX6EyuIor6mnLiIhFd5QTzBXHWK9dc1VF5GwCm+ol9TEHo8aVy/gte376OjUDBgRCZ/whvqYcZBbkHCu+uH2Tjbt2J+mwkRERk54Q90Mxp8A29f1WjyjQjNgRCS8whvqAFPPgYYXof1Q96ITJhYAurWdiIRTuEO9Zg60t8KWld2LxuTmMHXcGB0sFZFQCneoT31v7HHT8l6La8sLdWEvEQmlcIf62PEw8WTY9FyvxTMqCnireT+H2jvSVJiIyMgId6gDVM+Bzc9DR1v3otryQto7nbeaNQNGRMIlmRtPP2Rm283s5X5eLzazJ8xstZmtNbP03p+0r+r3Qtt+aFzdvai2+xowOlgqIuGSTE/9YeCSAV6/FVjn7qcCFwDfMbPc4ZeWItVzYo8bj4yrTy8bS3aWaVxdREJn0FB392XAzoGaAIVmZkBBvG17aspLgcJyGH8ibPpT96K8nGymTRirGTAiEjqpGFP/PnAS8A6wBrjd3TsTNTSzW8ys3szqm5qaUrDpJNXMgc1/gc4jB0ZnlBfqBCQRCZ1UhPrFwCpgEnAa8H0zK0rU0N0XuHudu9eVlZWlYNNJqj4XDu2BrS91L6otL2TzzgMcPKwZMCISHqkI9RuBJR7zOvAWMDMF602dmvi4eo+pjbXlBbjD69t1sFREwiMVob4Z+ACAmZUDM4A3U7De1CmaBKXTYOORcfXa+DVgNK4uImGSM1gDM3uE2KyWCWbWAHwZiAC4+3zga8DDZrYGMOAud28esYqPVfUcePVJ6OyErCyqx40hNydL4+oiEiqDhrq7XzPI6+8AF6WsopFSMwdW/Wfsqo0Vs8jJzuKEsgJe1bRGEQmR8J9R2qVrvnqPqY215QXqqYtIqGROqJdWQ/GUXich1VYU0ri7lT2tbQO8UUQkODIn1CHWW9/0HHjsVnYz4pcL0I2oRSQsMivUa+bAgWZo3gAcuQbMq1s1rVFEwiGzQr3PdWAml+QzNjdb4+oiEhqZFerjpkNBRffB0qws44TyQs2AEZHQyKxQN4sNwWz8U49x9QJe265QF5FwyKxQh9gQzL6tsDN20mtteSHN+w7TvO/QIG8UERn9Mi/Ua86NPcbH1WdUdN0wQ711EQm+zAv1CbUwtqx7XL1rWqNumCEiYZB5oW4Wu8Vd/IqNZYV5FOdHeFW3thOREMi8UIfYuPrut2HXJsyMGeWFOgFJREIhc0MduodgaisKeHXbXjw+I0ZEJKgyM9Qnngz5pd3XV59RXsje1nYadh1Mc2EiIsOTmaGelQVT3wubYjNg5pwwAYAn1zSmsyoRkWHLzFCH2ElIuzbC7i1MLyvgzOpSfl7/toZgRCTQMjfU+4yrX31mFW807eevb7eksSgRkeEZNNTN7CEz225mLw/Q5gIzW2Vma83sj6ktcYRUzIa8ou5Q//C7K4lGsvh5fUOaCxMROXbJ9NQfBi7p70UzKwF+AHzU3U8Brk5NaSMsKxumnt19sLQwGuHSWZX8avU7HDzckebiRESOzaCh7u7LgJ0DNPkksMTdN8fbb09RbSOveg7seA32bgPgqroq9h5q55m1W9NcmIjIsUnFmHotUGpmfzCzFWb26f4amtktZlZvZvVNTU0p2PQwdV0HJj4Ec/a08VSV5vPzFW+nsSgRkWOXilDPAc4EPgxcDNxjZrWJGrr7Anevc/e6srKyFGx6mCpPhcjYXtdXv+rMKp57YwcNuw6kuTgRkaFLRag3AM+4+353bwaWAaemYL0jLzsCU/+me1wd4ONnVOEOi1dsSWNhIiLHJhWh/kvgXDPLMbMxwN8A61Ow3uOjeg40rYf9OwCYMm4M733XeB5b+TadnZqzLiLBksyUxkeAPwMzzKzBzD5jZn9vZn8P4O7rgaeBl4AXgAfcvd/pj6NO17j65ue6F11dV8XbOw/y/FsDHR8WERl9cgZr4O7XJNFmHjAvJRUdb5NOh5xobAjmpMsAuOSUSv4lby0/X/E257xrfJoLFBFJXuaeUdolJw+qzuq+DgxAfm42Hzm1kqfWbGXfofY0FiciMjQKdYgNwWx9GfY3dy+66swpHGzr4MmX3kljYSIiQ6NQBzjpo2BZ8LuvdC86Y2oJ08vG6rIBIhIoCnWA8pPhnFth5Y/hrWcBMDOuPnMK9Zt28WaTbnUnIsGgUO9ywT9CaQ08cTu0xW6WceUZk8kyeGyFeusiEgwK9S65Y+Aj34Odb8Cy2ESe8qIo76stY8nKLXRozrqIBIBCvad3XQinfhL+9G+xA6fA1XVT2LqnlWdfGwXXqhERGYRCva+LvwHREnj8Nujs4AMnTaRkTISfawhGRAJAod7XmHHwoW/BOyvhhQXk5WTzsdMm89u122g5cDjd1YmIDEihnsisj8MJH4TffQ1aNnPVmVUc7ujk8dWasy4io5tCPREz+Mh9see/+gKzJhVxUmWR5qyLyKinUO9PyVT4wD3w+m/h5cVcfWYVa7bs5pWte9JdmYhIvxTqA3nPLTD5THjqLq6YmU8k29RbF5FRTaE+kKxs+Oj/htYWSp/9Ch+YWc4v/rqFto7OdFcmIpKQQn0w5afAnNth9U/5u6pN7Nh/mN+/Epx7a4tIZlGoJ+P8/wHjT+C0l77ClAL45lOv8PZO3cNUREYfhXoyIlG47N+wXRv5+cw/sGPfIT5+/3Osb9RBUxEZXZK5nd1DZrbdzAa8RZ2ZnWVm7WZ2VerKG0VqzoUzrqdi7QM8ceUYzOAT//FnXtAt70RkFEmmp/4wcMlADcwsG/gW8JsU1DR6ffCrMHYi1b+4gt/VLuHUsTv51IPP88zaremuTEQESCLU3X0ZMFh39DZgMRDuI4j5JXDz7+DM6ylY/xg/OfA5Foz9D76z8Jc88sLmdFcnIjL8MXUzmwxcAdyfRNtbzKzezOqbmgJ61cPiKvjwd+COl7CzP8f7Ol/g6dy7KH7iMzzyyydw1yV6RSR9UnGg9HvAXe4+6ORtd1/g7nXuXldWVpaCTadRYQVc/A3sjjX4uXdyYWQd1/z1U7z23Uvo2PSXdFcnIhkqFaFeB/zMzDYCVwE/MLOPpWC9wTB2PNn/5R7y/vta/lD194zfvZbsH15M5w8/DBt+A/uaQL13ETlOcoa7Anef1vXczB4GfuXuvxjueoMma0wJF9z0LR5aeh1b/t/93Pr2U4z76dWxF3MLYVwNlE6DcdNg3PQjz4smx85cFRFJgUFD3cweAS4AJphZA/BlIALg7vNHtLoA+q8XzmJJ8T9y7mMXcWXpG1xWdZAZuU0Ut27Btq+HV5+CzrYjb8jOhZJqGFsWmw8fGQOR/Ni/nPhjr2XR2JeAZYNlQVZWj+d9l3f9IWaxK08e9djntb4swbK+7Xq1Gei1ftr0s2jkJLufSb633/cnuc5+tz0CdSa9/eFseyjvH8rvc6DXbfivDbjtY/l9WoL19VgWGQN5BcmvdwgGDXV3vybZlbn7DcOqJiSuPKOK0rG5fPmXRfzn6tiZpxML85hzwgTee3YJ55cfpry9EXa+Cbvegp1vwcFd0Lob9m6DtgOxm1+3H4w9dujmHCKhMucO+OBXRmTVlq7ZGnV1dV5fX5+WbR9Pm3cc4E9vNPOn15v58xs72LE/FtDTJozlve8az5wTJnDO9PGUjs3tfyWdHbFw7wr6zg7wzti/7ucdRy/H4+P5CR6hz7K+Eiw7qp0nfJr8+/tpN1KGs/1+/z9Jdj+TfG+/7x9unUluf7ifUbLbP+bfp4/Aa8luezA9/7/qs76+yyreDVPeM4R1H2FmK9y9rt/XFerHT2en8+q2vfzp9VjIP//WTg4c7sAs1pMvyc+leEyEkvwIJWMiFOdHKBmTG3+MUJKfS2E0h0h2FpFsIzvLiGRnkZ1l5GQbOVlZ8cfYa9lmmBlG7C8+G9Kf0CIyGinUR7G2jk5Wv93Cc2/sYMuug7QcPEzLgTZ2H4z9aznQxsG2jpRvNyse8N1hj/UeeuzT3nq9lviLIdH3xUh8hST7xTSkbSc7/D2UVSZbZ3/DysNZZ1KtBtr+0QuH0h9IXHuidsf239JAv4d+f5+DbL/v68lsL5lfXX/r+duzpnDTedMTrncwg4X6sGe/yLGLZGdRVzOOuppx/bZpbetgz8E2WuJBv+dgG20dTntnJx2dTluH09HZGX902jpiy9s7Yz+7g9P1CLjjQKcfWdbze92P+jM14dPeTRJ0DEair5D0oMZQRguSXOuI7E8/K004MJDsqMYQhkoSrXM42+5vDQm309/I0yDvP2oAcKD/dvsvqdeivp9D79eSXuWA6+n7hgkFeYlXnAIK9VEuGskmGslmYlE03aWISADo0rsiIiGiUGOgJrYAAAP6SURBVBcRCRGFuohIiCjURURCRKEuIhIiCnURkRBRqIuIhIhCXUQkRNJ2mQAzawI2HePbJwDNKSxnNAjbPoVtfyB8+xS2/YHw7VOi/al2935vHZe2UB8OM6sf6NoHQRS2fQrb/kD49ils+wPh26dj2R8Nv4iIhIhCXUQkRIIa6gvSXcAICNs+hW1/IHz7FLb9gfDt05D3J5Bj6iIiklhQe+oiIpKAQl1EJEQCF+pmdomZvWpmr5vZl9JdTyqY2UYzW2Nmq8wscPf4M7OHzGy7mb3cY9k4M/utmb0WfyxNZ41D1c8+3WtmW+Kf0yozuzSdNQ6FmU0xs6Vmts7M1prZ7fHlgfycBtifIH9GUTN7wcxWx/fpK/Hl08zs+XjmPWpmA9ylPmBj6maWDWwAPgg0AC8C17j7urQWNkxmthGoc/dAnjRhZucD+4Afu/us+LJvAzvd/ZvxL99Sd78rnXUORT/7dC+wz93/ZzprOxZmVglUuvtKMysEVgAfA24ggJ/TAPvzCYL7GRkw1t33mVkEWA7cDnwBWOLuPzOz+cBqd7+/v/UEraf+HuB1d3/T3Q8DPwMuT3NNGc/dlwE7+yy+HPhR/PmPiP0PFxj97FNguXuju6+MP98LrAcmE9DPaYD9CSyP2Rf/MRL/58D7gcfiywf9jIIW6pOBt3v83EDAP8g4B35jZivM7JZ0F5Mi5e7eGH++FShPZzEp9N/M7KX48Ewghir6MrMa4HTgeULwOfXZHwjwZ2Rm2Wa2CtgO/BZ4A2hx9/Z4k0EzL2ihHlbnuvsZwIeAW+N/+oeGx8b4gjPO17/7gXcBpwGNwHfSW87QmVkBsBi4w9339HwtiJ9Tgv0J9Gfk7h3ufhpQRWxkYuZQ1xG0UN8CTOnxc1V8WaC5+5b443bg/xL7MINuW3zcs2v8c3ua6xk2d98W/5+uE/g/BOxzio/TLgYWuvuS+OLAfk6J9ifon1EXd28BlgLnACVmlhN/adDMC1qovwicGD8anAv8LfB4mmsaFjMbGz/Qg5mNBS4CXh74XYHwOHB9/Pn1wC/TWEtKdIVf3BUE6HOKH4R7EFjv7vf1eCmQn1N/+xPwz6jMzEriz/OJTQhZTyzcr4o3G/QzCtTsF4D4FKXvAdnAQ+7+jTSXNCxmNp1Y7xwgB/hp0PbJzB4BLiB2mdBtwJeBXwCLgKnELrH8CXcPzIHHfvbpAmJ/1juwEfi7HuPRo5qZnQs8C6wBOuOL7yY2Dh24z2mA/bmG4H5G7yZ2IDSbWId7kbt/NZ4RPwPGAX8FPuXuh/pdT9BCXURE+he04RcRERmAQl1EJEQU6iIiIaJQFxEJEYW6iEiIKNRFREJEoS4iEiL/H7yaU/R3CGe/AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "eAqcfYkJmH0c", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "939c80a2-0dd6-4ecb-baad-98d4b6db7df9" }, "source": [ "predictions = model.predict(X_test_scaled)\n", "print('R2:', r2_score(y_test, predictions))\n", "print('MSE:', mean_squared_error(y_test, predictions))" ], "execution_count": 26, "outputs": [ { "output_type": "stream", "text": [ "R2: 0.0009308488769875778\n", "MSE: 1.3113183313285037\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "lYSf_Ipusgbh" }, "source": [ "## 3.6: Custom NN model with custom dense and weight initializers" ] }, { "cell_type": "code", "metadata": { "id": "hIf31L4nmK3t", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "678e61d1-2e01-43f9-c003-47db94aa4085" }, "source": [ "model = tf.keras.models.Sequential([\n", " CustomDense(16, activation=\"relu\", initializer='glorot_uniform',\n", " input_shape=(X_train.shape[1],)),\n", " CustomDense(32, activation=\"relu\", initializer='glorot_uniform'),\n", " CustomDense(32, activation=\"relu\", initializer='glorot_uniform'),\n", " CustomDense(1),\n", " softplus_fn\n", "])\n", "\n", "model.compile(loss=mse_loss, \n", " optimizer=tf.keras.optimizers.SGD(lr=1e-3),\n", " metrics=['mean_squared_error'])\n", "\n", "model.summary()" ], "execution_count": 27, "outputs": [ { "output_type": "stream", "text": [ "Model: \"sequential_2\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "custom_dense_12 (CustomDense (None, 16) 144 \n", "_________________________________________________________________\n", "custom_dense_13 (CustomDense (None, 32) 544 \n", "_________________________________________________________________\n", "custom_dense_14 (CustomDense (None, 32) 1056 \n", "_________________________________________________________________\n", "custom_dense_15 (CustomDense (None, 1) 33 \n", "_________________________________________________________________\n", "lambda (Lambda) (None, 1) 0 \n", "=================================================================\n", "Total params: 1,777\n", "Trainable params: 1,777\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "CKmOMZjBnC1Z", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "38a62a69-c0da-4657-f24a-0158509586ce" }, "source": [ "history = model.fit(X_train_scaled, y_train, \n", " epochs=30,\n", " batch_size=32,\n", " validation_split=0.1)" ], "execution_count": 28, "outputs": [ { "output_type": "stream", "text": [ "Epoch 1/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 2.7581 - mean_squared_error: 2.7581 - val_loss: 1.4834 - val_mean_squared_error: 1.4834\n", "Epoch 2/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3492 - mean_squared_error: 1.3492 - val_loss: 1.0399 - val_mean_squared_error: 1.0399\n", "Epoch 3/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.0163 - mean_squared_error: 1.0163 - val_loss: 0.8873 - val_mean_squared_error: 0.8873\n", "Epoch 4/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.8699 - mean_squared_error: 0.8699 - val_loss: 0.7972 - val_mean_squared_error: 0.7972\n", "Epoch 5/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.7901 - mean_squared_error: 0.7901 - val_loss: 0.7371 - val_mean_squared_error: 0.7371\n", "Epoch 6/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.7297 - mean_squared_error: 0.7297 - val_loss: 0.6934 - val_mean_squared_error: 0.6934\n", "Epoch 7/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6943 - mean_squared_error: 0.6943 - val_loss: 0.6555 - val_mean_squared_error: 0.6555\n", "Epoch 8/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6371 - mean_squared_error: 0.6371 - val_loss: 0.6281 - val_mean_squared_error: 0.6281\n", "Epoch 9/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6193 - mean_squared_error: 0.6193 - val_loss: 0.6051 - val_mean_squared_error: 0.6051\n", "Epoch 10/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6116 - mean_squared_error: 0.6116 - val_loss: 0.5880 - val_mean_squared_error: 0.5880\n", "Epoch 11/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5766 - mean_squared_error: 0.5766 - val_loss: 0.5752 - val_mean_squared_error: 0.5752\n", "Epoch 12/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5761 - mean_squared_error: 0.5761 - val_loss: 0.5633 - val_mean_squared_error: 0.5633\n", "Epoch 13/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5438 - mean_squared_error: 0.5438 - val_loss: 0.5526 - val_mean_squared_error: 0.5526\n", "Epoch 14/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5489 - mean_squared_error: 0.5489 - val_loss: 0.5448 - val_mean_squared_error: 0.5448\n", "Epoch 15/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5439 - mean_squared_error: 0.5439 - val_loss: 0.5369 - val_mean_squared_error: 0.5369\n", "Epoch 16/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5255 - mean_squared_error: 0.5255 - val_loss: 0.5256 - val_mean_squared_error: 0.5256\n", "Epoch 17/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5114 - mean_squared_error: 0.5114 - val_loss: 0.5168 - val_mean_squared_error: 0.5168\n", "Epoch 18/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5090 - mean_squared_error: 0.5090 - val_loss: 0.5089 - val_mean_squared_error: 0.5089\n", "Epoch 19/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5031 - mean_squared_error: 0.5031 - val_loss: 0.5024 - val_mean_squared_error: 0.5024\n", "Epoch 20/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5052 - mean_squared_error: 0.5052 - val_loss: 0.4957 - val_mean_squared_error: 0.4957\n", "Epoch 21/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4574 - mean_squared_error: 0.4574 - val_loss: 0.4896 - val_mean_squared_error: 0.4896\n", "Epoch 22/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4694 - mean_squared_error: 0.4694 - val_loss: 0.4846 - val_mean_squared_error: 0.4846\n", "Epoch 23/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4644 - mean_squared_error: 0.4644 - val_loss: 0.4803 - val_mean_squared_error: 0.4803\n", "Epoch 24/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4492 - mean_squared_error: 0.4492 - val_loss: 0.4757 - val_mean_squared_error: 0.4757\n", "Epoch 25/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4367 - mean_squared_error: 0.4367 - val_loss: 0.4706 - val_mean_squared_error: 0.4706\n", "Epoch 26/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4449 - mean_squared_error: 0.4449 - val_loss: 0.4674 - val_mean_squared_error: 0.4674\n", "Epoch 27/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4387 - mean_squared_error: 0.4387 - val_loss: 0.4644 - val_mean_squared_error: 0.4644\n", "Epoch 28/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4206 - mean_squared_error: 0.4206 - val_loss: 0.4629 - val_mean_squared_error: 0.4629\n", "Epoch 29/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4375 - mean_squared_error: 0.4375 - val_loss: 0.4622 - val_mean_squared_error: 0.4622\n", "Epoch 30/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4321 - mean_squared_error: 0.4321 - val_loss: 0.4607 - val_mean_squared_error: 0.4607\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "cmUc8Gf3nGnx", "colab": { "base_uri": "https://localhost:8080/", "height": 282 }, "outputId": "6fdf3845-f854-40d8-83c0-120138d89612" }, "source": [ "history_df = pd.DataFrame(history.history)\n", "history_df[['loss', 'val_loss']].plot(kind='line')" ], "execution_count": 29, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 29 }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZxU1Znw8d9TS1fvTS/VNHQ3mxtRENAOaAZQkxmjTiKaTKJojOaNMlHjkuR14iSTxBgzM2+cSfJm05cxjMu4MW4x0ajMqEEmojTIIqKIyNIN0hu9d9f6vH/caqimV7oLqrvq+X4+91N1z7n31nMtee7tc0+dI6qKMcaY1OdKdgDGGGOOD0v4xhiTJizhG2NMmrCEb4wxacISvjHGpAlPsgPoT0lJiU6bNi3ZYRhjzLixfv36BlX1D7bNmEz406ZNo7q6OtlhGGPMuCEiu4faxpp0jDEmTVjCN8aYNGEJ3xhj0sSYbMM3xqSfUChETU0N3d3dyQ5lTMvMzKSiogKv13vU+1rCN8aMCTU1NeTl5TFt2jREJNnhjEmqSmNjIzU1NUyfPv2o97cmHWPMmNDd3U1xcbEl+0GICMXFxSP+K8gSvjFmzLBkP7TR/DdKmYSvqvzyv9/nT9vrkx2KMcaMSSmT8EWE5a/t5JV365IdijFmnMrNzU12CMdUyiR8AH+ej/r2QLLDMMaYMSm1En6uj/o2S/jGmNFRVW677TZmzZrF7NmzefzxxwHYv38/ixcvZu7cucyaNYvXXnuNSCTCNddcc2jbn/3sZ0mOfmAp1S3Tn+fjnX2tyQ7DGDNKP/z91oT/Wz51cj4/+Oxpw9r2qaeeYuPGjWzatImGhgY+/vGPs3jxYh555BE+/elP893vfpdIJEJnZycbN26ktraWt99+G4Dm5uaExp1IqXWHn2d3+MaY0VuzZg1Lly7F7XYzceJEzjnnHNatW8fHP/5x/v3f/5077riDLVu2kJeXx4wZM9i5cyc33XQTL7zwAvn5+ckOf0Apd4ffFgjTFYyQleFOdjjGmBEa7p348bZ48WJWr17Nc889xzXXXMM3v/lNvvzlL7Np0yZefPFF7r33XlauXMmKFSuSHWq/hrzDF5FKEXlFRN4Rka0icks/21wpIptFZIuI/FlE5sTV7YqVbxSRYzrmsT/XB0CDPbg1xozCokWLePzxx4lEItTX17N69Wrmz5/P7t27mThxItdddx3XXnstGzZsoKGhgWg0yuc//3nuuusuNmzYkOzwBzScO/ww8C1V3SAiecB6EVmlqu/EbfMhcI6qHhSRC4HlwIK4+vNUtSFxYffPn+ck/Lq2AJVF2cf644wxKerSSy/l9ddfZ86cOYgIP/nJTygrK+OBBx7g7rvvxuv1kpuby4MPPkhtbS1f+cpXiEajAPzTP/1TkqMf2JAJX1X3A/tj79tEZBtQDrwTt82f43ZZC1QkOM5h6Un41o5vjBmJ9vZ2wPldz913383dd9/dq/7qq6/m6quv7rPfWL6rj3dUD21FZBowD3hjkM2+Cvwxbl2Bl0RkvYgsG+TYy0SkWkSq6+tH9mvZQwnfmnSMMaaPYT+0FZFc4EngVlXtt7+UiJyHk/AXxhUvVNVaESkFVonIu6q6+sh9VXU5TlMQVVVVehTncEhxjg+X2B2+Mcb0Z1h3+CLixUn2D6vqUwNsczpwH7BEVRt7ylW1NvZaBzwNzB9t0ANxu4SiHOuaaYwx/RlOLx0BfgtsU9WfDrDNFOAp4CpV3R5XnhN70IuI5ADnA28nIvCBWF98Y4zp33CadP4CuArYIiIbY2XfAaYAqOq9wPeBYuA3saE7w6paBUwEno6VeYBHVPWFhJ7BEWw8HWOM6d9weumsAQYdgFlVrwWu7ad8JzCn7x7Hjj/Xxwd17cfzI40xZlxIqaEV4HCTjuqInvsaY0zKSsmEH4xEae0KJzsUY0wKG2zs/F27djFr1qzjGM3wpGTCB6hvH9mcj8YYk6pSavA0ODyeTl1bgBNL85IcjTFmRP54O3y0JbHHLJsNF/7zgNW33347lZWV3HjjjQDccccdeDweXnnlFQ4ePEgoFOKuu+5iyZIlR/Wx3d3dXH/99VRXV+PxePjpT3/Keeedx9atW/nKV75CMBgkGo3y5JNPMnnyZL74xS9SU1NDJBLhe9/7HpdddtmoTjte6iV8G17BGDMCl112GbfeeuuhhL9y5UpefPFFbr75ZvLz82loaOCss87i4osvPqqJxH/9618jImzZsoV3332X888/n+3bt3Pvvfdyyy23cOWVVxIMBolEIjz//PNMnjyZ5557DoCWlpaEnqMlfGPM2DPInfixMm/ePOrq6ti3bx/19fUUFhZSVlbGN77xDVavXo3L5aK2tpYDBw5QVlY27OOuWbOGm266CYCZM2cydepUtm/fztlnn82Pf/xjampq+NznPsdJJ53E7Nmz+da3vsW3v/1tPvOZz7Bo0aKEnmPKteHnZ3rI8LisL74x5qh94Qtf4IknnuDxxx/nsssu4+GHH6a+vp7169ezceNGJk6cSHd3Yp4PXnHFFTz77LNkZWVx0UUX8fLLL3PyySezYcMGZs+ezT/8wz9w5513JuSzeqTcHb6I2Ny2xpgRueyyy7juuutoaGjgT3/6EytXrqS0tBSv18srr7zC7t27j/qYixYt4uGHH+aTn/wk27dvZ8+ePZxyyins3LmTGTNmcPPNN7Nnzx42b97MzJkzKSoq4ktf+hITJkzgvvvuS+j5pVzCBxtewRgzMqeddhptbW2Ul5czadIkrrzySj772c8ye/ZsqqqqmDlz5lEf84YbbuD6669n9uzZeDwe7r//fnw+HytXruShhx7C6/VSVlbGd77zHdatW8dtt92Gy+XC6/Vyzz33JPT8ZCz+QKmqqkqrq0c+OdZ1D1azt6mTF25dnMCojDHH0rZt2/jYxz6W7DDGhf7+W4nI+tiQNgNKuTZ8cO7wbZpDY4zpLTWbdHJ9NHYECUeieNwpeU0zxowBW7Zs4aqrrupV5vP5eOONweaISp7UTPh5PlShqSNIaX5mssMxxgyTqh5VH/dkmz17Nhs3bhx6wwQaTTN8St7+xk9mbowZHzIzM2lsbLSBDwehqjQ2NpKZObIb2ZS9wwf78ZUx40lFRQU1NTWMdE7rdJGZmUlFRcWI9k3NhJ9rCd+Y8cbr9TJ9+vRkh5HSUrpJx35ta4wxhw1nTttKEXlFRN4Rka0icks/24iI/EJEdojIZhE5I67uahF5P7ZcnegT6E+m101epsfu8I0xJs5wmnTCwLdUdUNsQvL1IrJKVd+J2+ZC4KTYsgC4B1ggIkXAD4AqQGP7PquqBxN6Fv2wX9saY0xvQ97hq+p+Vd0Qe98GbAPKj9hsCfCgOtYCE0RkEvBpYJWqNsWS/CrggoSewQBsPB1jjOntqNrwRWQaMA848lcF5cDeuPWaWNlA5f0de5mIVItIdSKe0vvzfNaGb4wxcYad8EUkF3gSuFVVWxMdiKouV9UqVa3y+/2jPp416RhjTG/DSvgi4sVJ9g+r6lP9bFILVMatV8TKBio/5vx5PtoDYTqDNpm5McbA8HrpCPBbYJuq/nSAzZ4FvhzrrXMW0KKq+4EXgfNFpFBECoHzY2XHXE9f/Ia24PH4OGOMGfOG00vnL4CrgC0i0jNoxHeAKQCqei/wPHARsAPoBL4Sq2sSkR8B62L73amqTYkLf2CH++J3M6U4+3h8pDHGjGlDJnxVXQMMOpqROoNf3DhA3QpgxYiiGwUbXsEYY3pLyV/agiV8Y4w5Usom/OIcHy6xhG+MMT1SNuG7XUJRjvXFN8aYHimb8MH64htjTDxL+MYYkyZSO+HbeDrGGHNIaif82Hg6NmWaMcakQcIPRZSWrlCyQzHGmKRL+YQP1jXTGGMg1RO+zW1rjDGHpHbCt7ltjTHmkPRI+HaHb4wxqZ3w8zM9ZHhclvCNMYYUT/giYn3xjTEmJqUTPtjctsYY0yM9Er7d4RtjjCV8Y4xJF8OZ03aFiNSJyNsD1N8mIhtjy9siEhGRoljdLhHZEqurTnTww+HP9dHUGSQUiSbj440xZswYzh3+/cAFA1Wq6t2qOldV5wJ/D/zpiHlrz4vVV40u1JHx5/lQhaYOm8zcGJPehkz4qroaGO7E40uBR0cVUYJZX3xjjHEkrA1fRLJx/hJ4Mq5YgZdEZL2ILBti/2UiUi0i1fX19YkKyxK+McbEJPKh7WeB/zmiOWehqp4BXAjcKCKLB9pZVZerapWqVvn9/oQFZePpGGOMI5EJ/3KOaM5R1drYax3wNDA/gZ83LDaejjHGOBKS8EWkADgH+F1cWY6I5PW8B84H+u3pcyxlet3kZXrsDt8Yk/Y8Q20gIo8C5wIlIlID/ADwAqjqvbHNLgVeUtWOuF0nAk+LSM/nPKKqLyQu9OGzvvjGGDOMhK+qS4exzf043Tfjy3YCc0YaWCLZeDrGGJMGv7QFG0/HGGMgnRK+3eEbY9Jc2iT89kCYzmA42aEYY0zSpEfCj/XFb2iz4RWMMekrPRL+ob743UmOxBhjkie9Er614xtj0pglfGOMSRNpkfCLc3y4xBK+MSa9pUXCd7uEohzri2+MSW9pkfDB+uIbY4wlfGOMSRPpk/BtPB1jTJpLn4QfG09HVZMdijHGJEVaJfxQRGnpCiU7FGOMSYq0SvhgXTONMekrfRK+zW1rjElz6ZPwbW5bY0yaGzLhi8gKEakTkX7noxWRc0WkRUQ2xpbvx9VdICLvicgOEbk9kYEfLWvSMcaku+Hc4d8PXDDENq+p6tzYcieAiLiBXwMXAqcCS0Xk1NEEOxr5mR4yPC5L+MaYtDVkwlfV1UDTCI49H9ihqjtVNQg8BiwZwXESQkSsL74xJq0lqg3/bBHZJCJ/FJHTYmXlwN64bWpiZf0SkWUiUi0i1fX19SOLoqUGWmoHrLa5bY0x6SwRCX8DMFVV5wC/BJ4ZyUFUdbmqVqlqld/vP/oDhAPwi3nwxr0DbmLDKxhj0tmoE76qtqpqe+z984BXREqAWqAybtOKWNmx4fHBpLmw980BN7GEb4xJZ6NO+CJSJiISez8/dsxGYB1wkohMF5EM4HLg2dF+3qAq58O+t5y7/X74c300dQYJRaLHNAxjjBmLhtMt81HgdeAUEakRka+KyNdE5GuxTf4GeFtENgG/AC5XRxj4OvAisA1Yqapbj81pxFQugEgA9m/qt9qf50MVmjpsMnNjTPrxDLWBqi4dov5XwK8GqHseeH5koY1A5QLnde8bzt3+EeL74k/MzzxuYRljzFiQWr+0zZsIhdOchN8P+/GVMSadpVbCB+cuf++b0M8wyDaejjEmnaVgwp8P7QegeXefKhtPxxiTzlIw4fe04/ftnpnpdZOX6bE7fGNMWkq9hF96KmTkDtqObwnfGJOOUi/hu9xQUTVwwrfxdIwxaSr1Ej44zToHtkKgrU+VjadjjElXqZvwNQo11X2qrEnHGJOuUjPhV1QB0u+DW3+ej/ZAmM5g+PjHZYwxSZSaCT+zwHl42087fk9f/IY2G17BGJNeUjPhg9Mfv2YdRHsPlHa4L353MqIyxpikSeGEvwACrVD/bq9iG17BGJOuUjjhxwZPO6JZxxK+MSZdpW7CL5oB2SV9HtwW5/hwiSV8Y0z6Sd2ELwJTzupzh+92CUU51hffGJN+Ujfhg9Os0/QBtPeeFN364htj0lGKJ/zYQGo1vZt1LOEbY9LRcKY4XCEidSLy9gD1V4rIZhHZIiJ/FpE5cXW7YuUbRaTvz16PtUlzweXt++DWxtMxxqSh4dzh3w9cMEj9h8A5qjob+BGw/Ij681R1rqpWjSzEUfBmwuS5fR7c9oyno/1MkmKMMalqyISvqquBpkHq/6yqB2Ora4GKBMWWGJULoHYDhA//snaGP4dQRNlS25LEwIwx5vhKdBv+V4E/xq0r8JKIrBeRZYPtKCLLRKRaRKrr6+sH2/ToVM6HSAA+2nyo6NOnlpHhdvHMW/sS9znGGDPGJSzhi8h5OAn/23HFC1X1DOBC4EYRWTzQ/qq6XFWrVLXK7/cnKqy4GbAOt+MXZHv55MxSnt20j3AkOsCOxhiTWhKS8EXkdOA+YImqNvaUq2pt7LUOeBqYn4jPOyp5ZTBhap8Ht5fMK6ehPcCaHQ3HPSRjjEmGUSd8EZkCPAVcparb48pzRCSv5z1wPtBvT59jrnKB8+A27iHteTP95Gd6eOat2qSEZIwxx5tnqA1E5FHgXKBERGqAHwBeAFW9F/g+UAz8RkQAwrEeOROBp2NlHuARVX3hGJzD0Crnw5aV0LwHCqcC4PO4+evTJ/PMW7V0BMLk+Ib8T2GMMePakFlOVZcOUX8tcG0/5TuBOX33SIJD7fhvHkr4AJfOK+fRN/fw0jsfcem8sdW5yBhjEi21f2nbo/RUyMjt045fNbWQisIsnrbeOsaYNJAeCd/tgfIz+yR8l0u4ZG45a96vp67VJkQxxqS29Ej44DTrHHgbAu29ii+ZN5mowrOb7C7fGJPa0ifhT1kAGoXa9b2KTyzNY3Z5Ac9stN46xpjUlj4Jv7wKkD7j6oDz8Pbt2lbeP9B2/OMyxpjjJH0SftYEKP1Yn3Z8gM/OmYzbJXaXb4xJaemT8MHpj1/zJkR7D6fgz/Ox8MQSnnlrH9GojaBpjElNaZbwF0B3CzS816fq0nnl1DZ3sW7XgAODGmPMuJZ+CR/6bdY5/7SJZGe4rVnHGJOy0ivhF82A7JJ+H9xmZ3i44LQy/rB5P92hSBKCM8aYYyu9Er5IbCC1vnf44Iyg2dYd5tX36o5zYMYYc+ylV8IH58Ft4w7oaOxT9YkTivHn+XjaRtA0xqSgNEz4sXb8mr7NOh63i4vnTObld+to7gz2qTfGmPEs/RL+5Lng8g7YrHPpvHJCEeW5LfuPc2DGGHNspV/C92bBpDn9PrgFOG1yPieV5trEKMaYlJN+CR+cZp3a9dDZt8+9iHDJvHLW7TrI3qbOJARnjDHHRnom/LlXQDQML9zeb/WSuZMB+J31yTfGpJBhJXwRWSEidSLS75y04viFiOwQkc0ickZc3dUi8n5suTpRgY9K2SxY9C3Y/Di813fWxYrCbOZPL+Lpt2pRtaEWjDGpYbh3+PcDFwxSfyFwUmxZBtwDICJFOHPgLgDmAz8QkcKRBptQi/43lJ4Gf7gVupr7VH9uXjkf1HewpbYlCcEZY0ziDSvhq+pqYLBBZpYAD6pjLTBBRCYBnwZWqWqTqh4EVjH4heP48WTAJb+G9jp48bt9qi+cPYkMt8v65BtjUkai2vDLgb1x6zWxsoHK+xCRZSJSLSLV9fX1CQprCJPnwV/cAhv/A97/r15VBVlePvWxUn6/aR/hSHSAAxhjzPgxZh7aqupyVa1S1Sq/33/8Pvicb0PJKfD7m6G7tVfVJfPKaWgP8tqOhuMXjzHGHCOJSvi1QGXcekWsbKDyscObCZf8Btr2w6rv9ao675RS/Hk+fvzcNtq6Q0kK0BhjEiNRCf9Z4Mux3jpnAS2quh94EThfRApjD2vPj5WNLRVVcPaNsP5+2PnqoeIMj4tfXD6PDxs6+ObKTTY5ijFmXBtut8xHgdeBU0SkRkS+KiJfE5GvxTZ5HtgJ7AD+DbgBQFWbgB8B62LLnbGysee870LxifDsTRBoP1R89gnFfO+vP8aqdw7wi5ffT2KAxhgzOjIW+5lXVVVpdXX18f/gPWthxQXw8Wvhr//lULGqctsTm3lifQ3LrzqT808rO/6xGWPMIERkvapWDbbNmHloOyZMOQsWfA3W/RvsWnOoWES465JZzKko4BuPb+T9A21JDNIYY0bGEv6RPvU9KJwGv/s6BA+PpZPpdXPvVWeSleFh2UPraemyh7jGmPHFEv6RMnLg4l/BwQ/h5R/1qppUkMW9XzqDmoOd3PLYW0TsIa4xZhyxhN+f6Yucdvy19zjt+nGqphVxx8Wn8ep79fzrS+8lKUBjjDl6lvAH8pc/hIJK+N2NEOrqVXXlgqksnT+F37z6AX/YvC9JARpjzNGxhD8QXy5c/Atn/ttnboBw7ykPf3jxaVRNLeS2/9zMtv2tAxzEGGPGDkv4gznhPOdOf+tT8MgXeg29kOFx8ZsvnUF+lodlD1VzsMPmwDXGjG2W8Iey8Fa45B748DW4/yJoO3CoqjQvk/93VRUHWgLc9OhbNsiaMWZMs4Q/HHOvgCtWQuNO+O1fQsPhX9zOrZzAXZfOYs2OBv7x+XdtwhRjzJhlCX+4TvpLuOYPTt/8354Pe9cdqvpiVSXXfGIaK/7nQ25+bCPtgXASAzXGmP5Zwj8a5WfAV1+CzAJ44LO9pkf8/mdO5e8uOIXnNu/j4l+t4b2P7Ne4xpixxRL+0So+Ab66CkpnwmNLYf0DALhcwg3nnsjD155Fa1eYJb9ew5Pra5IcrDHGHGYJfyRy/XD1H2DGec7EKa/+H4i13Z99QjHP37yQORUT+NZ/buLvn9pMdyiS5ICNMcYS/sj5cuGKx2HOUnj1H53J0CNO231pfiYPX7uAG849gUff3MvnfvNndjd2JDlgY0y6s4Q/Gm6v02Vz4TedyVMeuwLaPgLA43bxdxfMZMU1VdQ2d/GZX6zhhbf3JzdeY0xas4Q/WiLwlz+Ai/4Fdr4Cv6yC138NEWc0zU/OnMgfblrIDH8OX/uPDdz1h3cIWX99Y0wSWMJPlPnXwQ1rnTH1X/wO3LvI+bEWUFmUzcqvnc3VZ0/lvjUfcvnytext6hzigMYYk1jDneLwAhF5T0R2iMjt/dT/TEQ2xpbtItIcVxeJq3s2kcGPOcUnwJX/CZc/AqEOeOAz8MT/gtZ9+DxufrhkFr9cOo9397fyqX/9Ez/8/VYa2wPJjtoYkyaGnOJQRNzAduCvgBqcuWmXquo7A2x/EzBPVf9XbL1dVXOPJqikTXGYSKEuWPMzWPNzp63/nL+DBdeDJ4P9LV38fNX7/Of6vWRneFi2eAZfXTidHJ8n2VEbY8apRE1xOB/Yoao7VTUIPAYsGWT7pcCjww8zRXmz4LzvwI1rYdoiWPV9uPcvYOerTCrI4v/8zem89I3FfOKEYn66ajvn3P0qD72+y9r3jTHHzHASfjmwN269JlbWh4hMBaYDL8cVZ4pItYisFZFLBvoQEVkW2666vr5+GGGNE0Uz4IrHnLF4IkF4cAmsvBoaP+DE0jyWf7mKJ6//BDNKcvje77byVz/9E7/ftI+ozaZljEmwRD+0vRx4QlXjf2k0NfZnxhXAz0XkhP52VNXlqlqlqlV+vz/BYY0BJ38abngDzvsubH8BfnkmPHI57HyVM6dM4PG/PYsV11Th87i56dG3WPLr/2HN+w3JjtoYk0KGk/Brgcq49YpYWX8u54jmHFWtjb3uBF4F5h11lKnCm+m05d+yCRbfBjXrnDv+ez6BbHiAT56Qz/O3LOJfvzCHpo4gX/rtG1zxb2v5720HbP5cY8yoDeehrQfnoe2ncBL9OuAKVd16xHYzgReA6Ro7qIgUAp2qGhCREuB1YMlAD3x7pMRD2+EIdcPbT8Ib98BHWyCrEM68Bj5+Hd3ZZfzH2t0sX72TurYAlUVZXHXWVL5YVcmE7IxkR26MGWOG89B2yIQfO9BFwM8BN7BCVX8sIncC1ar6bGybO4BMVb09br9PAP8PiOL8NfFzVf3tUJ+XNgm/hyrs/jOs/Q289zwgcOoSOOt6QpPO5MV3DvDg67t588MmfB4XS+ZO5stnT2NWeUGyIzfGjBEJS/jHW9ol/HgHd8Oby2HDQxBogUlz4YyrYNbn2dbs5sHXd/PMW7V0hSKcObWQL589lQtnTSLDY7+hMyadWcIfzwLtsOlRqP53qNsKbh/M/GuYeyUtkxfyxFv7eej1Xexq7KQk18cV8yv5mzMrmVKcnezIjTFJYAk/FajC/k2w8RHYshK6DkLeJJhzOdHTr2D1wQk89PpuXn6vDlWYVZ7PRbMncdGsSUwryUl29MaY48QSfqoJB5wunW89DDv+CzQCFfNh7hXUVlzI89s7eW7LfjbudUa2OHVSPhfNLuOi2ZOY4T+qHzsbY8YZS/iprO0j2Py4k/wb3gNPpjMhywnnccD/CX5fk83zb3/Ehj1O8p9Zlufc+c+exImllvyNSTWW8NOBKuzbABsfhR2r4OAup7ygEmacS9Okhfyx/RSefq+L6t0HATixNJdFJ5Ww6KQSFkwvtjF8jEkBlvDTUdNO+OAVZ2z+naudnj4ITJ5He8UiXovM5vEDk3h9VxuBcBSPSzhjSiELTyph4UklnF5egMdtPX6MGW8s4ae7SNi5+//gFfjgZeeXvRoBTxbRyWdQm3c6a0Mn8lR9OWs/iqIKeZkezp5R7FwATixhekkOIpLsMzHGDMESvumtuwV2rXEmZtn7Bny0GaLOPLyRopOozZvDmxHnAvDnliJAKM7J4PSKAk6vmMCcSue1JNeX3PMwxvRhCd8MLtgJ+96CvWth75vORaDLaeePZBbyUd7pvMM03myfyKstJXwYLSOMh/IJWYeS/+kVBcwuLyAv05vkkzEmvVnCN0dHFRp3wJ61TvLf+yY0vg/qjNEfdXlpzprKTtcU3uouY13HRLZrBXuZyLSSPGaW5XPyxDxOKXOWKUXZuF3WHGTM8WAJ34xeqNtJ+nXboO4dqHvXeW3efWiTsMvHPk8FuyMlvBcookb97FU/9e6JZPqnM2XSRE6ZmMfJZXnMLMujNM9nzwWMSbDhJHzrj2cG582EstnOEi/Q7vT/r9uGp24bUxq2M6V5DwubtyChuAnam6C5KY/dW/zUaAnPqJ9G90Qi+RX4iqdQUDaDyWVlzCjNZXpJDtkZ9r+kMceK/esyI+PLhfIznSWOqEJHAzTvgeZd0LyHCc17yG74kJObdpPRvhF3NAhtOMsuaNMs9mkxb2gxB71lhHIn45pQSZZ/GgWlU/BPnkpFaRG59nsBY0bF/gWZxBKBXL+zVBy+GBwawT8ahc4GaN4LLXsJNe0meGAXBY17KGzdS3b3WnJbW6AV2HP4sE2ay3YpptXrJ5A1kWjeJDwTyskpqaSwbCr+sgoy8/3OhPHGmH5ZwjfHlzMrNygAAAu6SURBVMsFuaXOUnEmXqD4yG2CHWjzXtrqdtF8YA+dDXuJNNfi6thPSVcd+W3vU9Ta0u+8a+2SQ6e7gKCvkGhmIa6cYjLy/GQXlpI9YSKunGLIjluyCsHlPg4nbkzyWcI3Y09GDlI6k/zSmeTP6n+TaChAw0e7ady/m9a6PXQ1HyDU1oB2NOLuPkhm+0Hy22oobHiXXNrIlkC/x1GEUEY+0axiXDnFeHNLkJxiyC6BvDLIL4eCcsivgBy/c8EyZpyyhG/GJZfXR2nlyZRWntxvvarS2hVmX0sX7zV3caDxIAcbDtBx8ACBtnoi7Y24uxrJ11YKw20UdbVR2NRGkWylxNXOBFrxEu51zKjLSySnDNeECtwTKiB/snMhyJ8EOaVOM1aOHzJynaYtY8aYYSV8EbkA+L84Uxzep6r/fET9NcDdHP4j+1eqel+s7mrgH2Lld6nqAwmI25hBiQgF2V4Ksr18bFI+MBGY2WsbVaWpI8j+lm4+aulmZ2s3f27pctabu+huqcPdvo+CUD2TpJHJ0sikUCOTWpqYvOcDyqSpz0UBIOLORLNLkFw/7ryJkFPiXBBy/M777KLezUrebLtAmONiOJOYu3EmMf8roAZnEvOl8RORxxJ+lap+/Yh9i4BqoApQYD1wpqoeHOwzrR++GUvaA2EOtHbHLQE+aummvrWTzuYD0LofV2cDBdFmSqSFYmmlRFopoQW/q41SVwsTtAUPkX6Pr55MyC5G4i8EWUWQNQEyCyAz9nrkui/fmpjMIYnqhz8f2KGqO2MHfQxYArwz6F6OTwOrVLUptu8q4ALg0WHsa8yYkOvzkOvP5YRBJpFRVVq6QhxoDVDX1k1da4Ctsdf6tgAHWrroamsk0t5IdriZImmjUNoooo3CcBvFgTYmtnVQ4vqIQnmfgmgrmdEOXEQHiUwgM9+5AGRNGN5rZsHhi4UnY5Bjm1Q0nIRfDuyNW68BFvSz3edFZDHOXwPfUNW9A+xb3t+HiMgyYBnAlClThhGWMWOHiDAhO4MJ2RmcUpY34HaqSkcwQkNbgPr2wKHXPW0BNrQ7F4f69iANbQEa2rvwhjspoIN86aRAOsingwLpwO/tpiyjG7+rm6JIJwUdneS1HyQnupfMcCsZoVZc0eDgQXuyYheA/MMXgZ51X57zLCIj1/nNxYDvc5wmKevpNC4k6qHt74FHVTUgIn8LPAB88mgOoKrLgeXgNOkkKC5jxhQRcf5i8HmGnHM4/uLQ2BGgvi1IY0eAxvYgDe0B3mgPUt8eoKkjSGN7gOauEIdbaJVMghTQQaGrg4qsAJN9AfzeACXeAMXuLgpcXeTTSS6dZIfa8XU3kNH4IZ5gKxLsQMJdwz8xT6aT+HsuABnZ4M1xXjNy+nnfs557eL9eS65z0bHfVSTUcBJ+LVAZt17BET2gVbUxbvU+4Cdx+557xL6vHm2QxqSjo7k4AIQjUQ52hmjsCNDUHqQhdiFo6gjS0B7kQEeAdztDNHcGOdjivIYiA99b5ftclGWGmZgZxp8RpCQjRLE3SJEnSIE7QL47QJ50ky1BsgiQpd1kaDcZ0W7c4S4IdUJ7nfMa7HCWUCdEhvjLI57b51wAfLmQkXf4L4yedW8WeHzOBceTGffed0SdzzlW/PqRZW5vyj88H07CXwecJCLTcRL45cAV8RuIyCRV3R9bvRjYFnv/IvCPIlIYWz8f+PtRR22M6cPjduHP8+HPG958BT1/QRzsCHKwM8jBnotBR5CWrjAtXaFDS01XiK3th9e7Qv0/gO6R4XFRkOUlP9NDfpaX/Ale8jI95GV6KciAQm+ICYcuHCHyXAFypZtsAmTRhS/ajTfcgSvU7ozbFIx77W6GlhrnfagLwgEIdx0a1XV0/xEzweV1mqhcnrilv3U3ICAu50Ihrn7WcV577XvketyxMwvgU98f/XkMdHpDbaCqYRH5Ok7ydgMrVHWriNwJVKvqs8DNInIxEAaagGti+zaJyI9wLhoAd/Y8wDXGJFf8XxCVRdlHtW8gHKE1dlFo7Q7R2hWitTu2fqgsfOj9wc4ge5s6aQuEaesO0R3qLzlnxpaCQyXZGW5yfB5yMtxkZ3jI8TmvuYUesjPczuLzkO11k+OFPE+EXHeIXHeEHFeIbHeELFeILIJ4CeMjiJcQnmgQiQRjF4vu2GvsfTTiTAx0aDlyPa4MdYYV12jsfTS23vM+thw6RtyxtJ+yrKJjmvBteGRjzHEXikRp7w7THgjT1u1cBNoDznprd5iuYJiOQISOQJiOYITO2Hpn0FnvCITpCITpDEboCkYIRo7u7l4EMj1ufF5Xr9dMrwtfz7rX7Swe573P01MWW/e68bldZHhc+Dw9r+5e673q3G68HiHD7Tom80bb8MjGmDHJ63ZRmJNBYU5iuoaGIlE6YxeGzmCEzkDc+1h5dzhKIBQhEI7SHffa+73z2tYdpr4t0Gub7lCU7nCERNwju8Rp9spwu8jwuMlwCxkeF6V5maz82tmj/4ABWMI3xox7XreLgiznucGxpKqEIkp32LkIBMNRguEogSNeg5EIgVCUYCRKIBQlEIke2jbU8z7Se59QJEp2xrHt3moJ3xhjhklEyPA4d+P543AeZ/tdtjHGpAlL+MYYkyYs4RtjTJqwhG+MMWnCEr4xxqQJS/jGGJMmLOEbY0yasIRvjDFpYkyOpSMi9cDuEe5eAjQkMJxkS7XzgdQ7p1Q7H0i9c0q184G+5zRVVf2D7TAmE/5oiEj1UAMIjSepdj6QeueUaucDqXdOqXY+MLJzsiYdY4xJE5bwjTEmTaRiwl+e7AASLNXOB1LvnFLtfCD1zinVzgdGcE4p14ZvjDGmf6l4h2+MMaYflvCNMSZNpEzCF5ELROQ9EdkhIrcnO55EEJFdIrJFRDaKyLic5FdEVohInYi8HVdWJCKrROT92GthMmM8GgOczx0iUhv7njaKyEXJjPFoiEiliLwiIu+IyFYRuSVWPp6/o4HOaVx+TyKSKSJvisim2Pn8MFY+XUTeiOW8x0VkyPkiU6INX0TcwHbgr4AaYB2wVFXfSWpgoyQiu4AqVR23PxgRkcVAO/Cgqs6Klf0EaFLVf45dnAtV9dvJjHO4BjifO4B2Vf2XZMY2EiIyCZikqhtEJA9YD1wCXMP4/Y4GOqcvMg6/JxERIEdV20XEC6wBbgG+CTylqo+JyL3AJlW9Z7Bjpcod/nxgh6ruVNUg8BiwJMkxGUBVVwNNRxQvAR6IvX8A5x/juDDA+YxbqrpfVTfE3rcB24Byxvd3NNA5jUvqaI+temOLAp8EnoiVD+s7SpWEXw7sjVuvYRx/wXEUeElE1ovIsmQHk0ATVXV/7P1HwMRkBpMgXxeRzbEmn3HT/BFPRKYB84A3SJHv6IhzgnH6PYmIW0Q2AnXAKuADoFlVw7FNhpXzUiXhp6qFqnoGcCFwY6w5IaWo06Y43tsV7wFOAOYC+4F/TW44R09EcoEngVtVtTW+brx+R/2c07j9nlQ1oqpzgQqcFo2ZIzlOqiT8WqAybr0iVjauqWpt7LUOeBrni04FB2LtrD3trXVJjmdUVPVA7B9kFPg3xtn3FGsXfhJ4WFWfihWP6++ov3Ma798TgKo2A68AZwMTRMQTqxpWzkuVhL8OOCn21DoDuBx4NskxjYqI5MQeOCEiOcD5wNuD7zVuPAtcHXt/NfC7JMYyaj2JMeZSxtH3FHsg+Ftgm6r+NK5q3H5HA53TeP2eRMQvIhNi77NwOqdsw0n8fxPbbFjfUUr00gGIdbH6OeAGVqjqj5Mc0qiIyAycu3oAD/DIeDwnEXkUOBdnKNcDwA+AZ4CVwBScYbC/qKrj4kHoAOdzLk4zgQK7gL+Na/8e00RkIfAasAWIxoq/g9PmPV6/o4HOaSnj8HsSkdNxHsq6cW7SV6rqnbEc8RhQBLwFfElVA4MeK1USvjHGmMGlSpOOMcaYIVjCN8aYNGEJ3xhj0oQlfGOMSROW8I0xJk1YwjfGmDRhCd8YY9LE/wcSHAo4yFqJMwAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "comqT3N0nJbI", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "9568b436-44d9-4ea5-ff1d-11c3bcd00c5a" }, "source": [ "predictions = model.predict(X_test_scaled)\n", "print('R2:', r2_score(y_test, predictions))\n", "print('MSE:', mean_squared_error(y_test, predictions))" ], "execution_count": 30, "outputs": [ { "output_type": "stream", "text": [ "R2: 0.6702474810411744\n", "MSE: 0.43281340678610997\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "AnDcPrmGE02I", "outputId": "12b591ee-cf48-4bb6-8de5-f9ff57c416c3" }, "source": [ "model = tf.keras.models.Sequential([\r\n", " CustomDense(16, activation=\"relu\", initializer='random_normal',\r\n", " input_shape=(X_train.shape[1],)),\r\n", " CustomDense(32, activation=\"relu\", initializer='random_normal'),\r\n", " CustomDense(32, activation=\"relu\", initializer='random_normal'),\r\n", " CustomDense(1),\r\n", " softplus_fn\r\n", "])\r\n", "\r\n", "model.compile(loss=mse_loss, \r\n", " optimizer=tf.keras.optimizers.SGD(lr=1e-3),\r\n", " metrics=['mean_squared_error'])\r\n", "\r\n", "model.summary()" ], "execution_count": 41, "outputs": [ { "output_type": "stream", "text": [ "Model: \"sequential_5\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "custom_dense_24 (CustomDense (None, 16) 144 \n", "_________________________________________________________________\n", "custom_dense_25 (CustomDense (None, 32) 544 \n", "_________________________________________________________________\n", "custom_dense_26 (CustomDense (None, 32) 1056 \n", "_________________________________________________________________\n", "custom_dense_27 (CustomDense (None, 1) 33 \n", "_________________________________________________________________\n", "lambda (Lambda) (None, 1) 0 \n", "=================================================================\n", "Total params: 1,777\n", "Trainable params: 1,777\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "0fMhDyTSGYUQ", "outputId": "aafe9ed1-50ed-4ab0-8c99-8199b6401df1" }, "source": [ "history = model.fit(X_train_scaled, y_train, \r\n", " epochs=60,\r\n", " batch_size=32,\r\n", " validation_split=0.1)" ], "execution_count": 42, "outputs": [ { "output_type": "stream", "text": [ "Epoch 1/60\n", "407/407 [==============================] - 1s 3ms/step - loss: 3.0029 - mean_squared_error: 3.0029 - val_loss: 2.4778 - val_mean_squared_error: 2.4778\n", "Epoch 2/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 2.2250 - mean_squared_error: 2.2250 - val_loss: 1.8568 - val_mean_squared_error: 1.8568\n", "Epoch 3/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.7037 - mean_squared_error: 1.7037 - val_loss: 1.5328 - val_mean_squared_error: 1.5328\n", "Epoch 4/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.4727 - mean_squared_error: 1.4727 - val_loss: 1.4105 - val_mean_squared_error: 1.4105\n", "Epoch 5/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.4053 - mean_squared_error: 1.4053 - val_loss: 1.3736 - val_mean_squared_error: 1.3736\n", "Epoch 6/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3080 - mean_squared_error: 1.3080 - val_loss: 1.3637 - val_mean_squared_error: 1.3637\n", "Epoch 7/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3467 - mean_squared_error: 1.3467 - val_loss: 1.3611 - val_mean_squared_error: 1.3611\n", "Epoch 8/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3413 - mean_squared_error: 1.3413 - val_loss: 1.3601 - val_mean_squared_error: 1.3601\n", "Epoch 9/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3298 - mean_squared_error: 1.3298 - val_loss: 1.3595 - val_mean_squared_error: 1.3595\n", "Epoch 10/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.2972 - mean_squared_error: 1.2972 - val_loss: 1.3590 - val_mean_squared_error: 1.3590\n", "Epoch 11/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3260 - mean_squared_error: 1.3260 - val_loss: 1.3585 - val_mean_squared_error: 1.3585\n", "Epoch 12/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3390 - mean_squared_error: 1.3390 - val_loss: 1.3579 - val_mean_squared_error: 1.3579\n", "Epoch 13/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3311 - mean_squared_error: 1.3311 - val_loss: 1.3571 - val_mean_squared_error: 1.3571\n", "Epoch 14/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3111 - mean_squared_error: 1.3111 - val_loss: 1.3562 - val_mean_squared_error: 1.3562\n", "Epoch 15/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3574 - mean_squared_error: 1.3574 - val_loss: 1.3551 - val_mean_squared_error: 1.3551\n", "Epoch 16/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3579 - mean_squared_error: 1.3579 - val_loss: 1.3537 - val_mean_squared_error: 1.3537\n", "Epoch 17/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3312 - mean_squared_error: 1.3312 - val_loss: 1.3519 - val_mean_squared_error: 1.3519\n", "Epoch 18/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3399 - mean_squared_error: 1.3399 - val_loss: 1.3496 - val_mean_squared_error: 1.3496\n", "Epoch 19/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3387 - mean_squared_error: 1.3387 - val_loss: 1.3465 - val_mean_squared_error: 1.3465\n", "Epoch 20/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3086 - mean_squared_error: 1.3086 - val_loss: 1.3423 - val_mean_squared_error: 1.3423\n", "Epoch 21/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3049 - mean_squared_error: 1.3049 - val_loss: 1.3364 - val_mean_squared_error: 1.3364\n", "Epoch 22/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.3243 - mean_squared_error: 1.3243 - val_loss: 1.3278 - val_mean_squared_error: 1.3278\n", "Epoch 23/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.2903 - mean_squared_error: 1.2903 - val_loss: 1.3145 - val_mean_squared_error: 1.3145\n", "Epoch 24/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.2843 - mean_squared_error: 1.2843 - val_loss: 1.2925 - val_mean_squared_error: 1.2925\n", "Epoch 25/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.2601 - mean_squared_error: 1.2601 - val_loss: 1.2535 - val_mean_squared_error: 1.2535\n", "Epoch 26/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.1917 - mean_squared_error: 1.1917 - val_loss: 1.1790 - val_mean_squared_error: 1.1790\n", "Epoch 27/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.1264 - mean_squared_error: 1.1264 - val_loss: 1.0381 - val_mean_squared_error: 1.0381\n", "Epoch 28/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.9753 - mean_squared_error: 0.9753 - val_loss: 0.8523 - val_mean_squared_error: 0.8523\n", "Epoch 29/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.8000 - mean_squared_error: 0.8000 - val_loss: 0.7429 - val_mean_squared_error: 0.7429\n", "Epoch 30/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.7323 - mean_squared_error: 0.7323 - val_loss: 0.7003 - val_mean_squared_error: 0.7003\n", "Epoch 31/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6973 - mean_squared_error: 0.6973 - val_loss: 0.6766 - val_mean_squared_error: 0.6766\n", "Epoch 32/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6779 - mean_squared_error: 0.6779 - val_loss: 0.6618 - val_mean_squared_error: 0.6618\n", "Epoch 33/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6545 - mean_squared_error: 0.6545 - val_loss: 0.6488 - val_mean_squared_error: 0.6488\n", "Epoch 34/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6547 - mean_squared_error: 0.6547 - val_loss: 0.6342 - val_mean_squared_error: 0.6342\n", "Epoch 35/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6222 - mean_squared_error: 0.6222 - val_loss: 0.6223 - val_mean_squared_error: 0.6223\n", "Epoch 36/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6023 - mean_squared_error: 0.6023 - val_loss: 0.6109 - val_mean_squared_error: 0.6109\n", "Epoch 37/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5993 - mean_squared_error: 0.5993 - val_loss: 0.6000 - val_mean_squared_error: 0.6000\n", "Epoch 38/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5698 - mean_squared_error: 0.5698 - val_loss: 0.5884 - val_mean_squared_error: 0.5884\n", "Epoch 39/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5757 - mean_squared_error: 0.5757 - val_loss: 0.5780 - val_mean_squared_error: 0.5780\n", "Epoch 40/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5659 - mean_squared_error: 0.5659 - val_loss: 0.5679 - val_mean_squared_error: 0.5679\n", "Epoch 41/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5403 - mean_squared_error: 0.5403 - val_loss: 0.5561 - val_mean_squared_error: 0.5561\n", "Epoch 42/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5544 - mean_squared_error: 0.5544 - val_loss: 0.5468 - val_mean_squared_error: 0.5468\n", "Epoch 43/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5465 - mean_squared_error: 0.5465 - val_loss: 0.5376 - val_mean_squared_error: 0.5376\n", "Epoch 44/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5068 - mean_squared_error: 0.5068 - val_loss: 0.5312 - val_mean_squared_error: 0.5312\n", "Epoch 45/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5079 - mean_squared_error: 0.5079 - val_loss: 0.5238 - val_mean_squared_error: 0.5238\n", "Epoch 46/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4915 - mean_squared_error: 0.4915 - val_loss: 0.5170 - val_mean_squared_error: 0.5170\n", "Epoch 47/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4858 - mean_squared_error: 0.4858 - val_loss: 0.5110 - val_mean_squared_error: 0.5110\n", "Epoch 48/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5143 - mean_squared_error: 0.5143 - val_loss: 0.5067 - val_mean_squared_error: 0.5067\n", "Epoch 49/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4860 - mean_squared_error: 0.4860 - val_loss: 0.5030 - val_mean_squared_error: 0.5030\n", "Epoch 50/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4715 - mean_squared_error: 0.4715 - val_loss: 0.5011 - val_mean_squared_error: 0.5011\n", "Epoch 51/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4781 - mean_squared_error: 0.4781 - val_loss: 0.4973 - val_mean_squared_error: 0.4973\n", "Epoch 52/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4970 - mean_squared_error: 0.4970 - val_loss: 0.4955 - val_mean_squared_error: 0.4955\n", "Epoch 53/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4673 - mean_squared_error: 0.4673 - val_loss: 0.4934 - val_mean_squared_error: 0.4934\n", "Epoch 54/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4665 - mean_squared_error: 0.4665 - val_loss: 0.4903 - val_mean_squared_error: 0.4903\n", "Epoch 55/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4580 - mean_squared_error: 0.4580 - val_loss: 0.4904 - val_mean_squared_error: 0.4904\n", "Epoch 56/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4824 - mean_squared_error: 0.4824 - val_loss: 0.4876 - val_mean_squared_error: 0.4876\n", "Epoch 57/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4753 - mean_squared_error: 0.4753 - val_loss: 0.4851 - val_mean_squared_error: 0.4851\n", "Epoch 58/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4641 - mean_squared_error: 0.4641 - val_loss: 0.4838 - val_mean_squared_error: 0.4838\n", "Epoch 59/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4535 - mean_squared_error: 0.4535 - val_loss: 0.4822 - val_mean_squared_error: 0.4822\n", "Epoch 60/60\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4571 - mean_squared_error: 0.4571 - val_loss: 0.4799 - val_mean_squared_error: 0.4799\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 282 }, "id": "_rpxoEzgGg4X", "outputId": "7f98b4f8-fd00-4002-aa72-6986a1523275" }, "source": [ "history_df = pd.DataFrame(history.history)\r\n", "history_df[['loss', 'val_loss']].plot(kind='line')" ], "execution_count": 43, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 43 }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZxcZZ3v8c+vtq5ek+5Op7uTzgohIaTDYsAAJipuyEtFUIkIKIyKAiOojDO4jON4dfTqHb0uDAwXEXBQyYALCogMoBGFkIXsCYHsnfTenU5vtT/3j6oknZClk1Sncqq+79frvOrUOaeqfk9S+dbJc55zjjnnEBER7/PlugAREckOBbqISJ5QoIuI5AkFuohInlCgi4jkiUCuPnjMmDFu8uTJufp4ERFPWrZsWYdzruZQ63IW6JMnT2bp0qW5+ngREU8ys22HW6cuFxGRPKFAFxHJEwp0EZE8kbM+dBEpTPF4nKamJiKRSK5LOaWFw2EaGhoIBoPDfo0CXUROqqamJsrLy5k8eTJmlutyTknOOTo7O2lqamLKlCnDfp26XETkpIpEIlRXVyvMj8DMqK6uPub/xSjQReSkU5gf3fH8GXku0De07OG7T22guz+W61JERE4pngv0rR0D3PncJnbuHsx1KSLiUWVlZbkuYUR4LtDHlIUA6NQeuojIATwX6NVlRQB09kVzXImIeJ1zji984QvMmjWLxsZGHn74YQCam5uZP38+55xzDrNmzeIvf/kLyWSS66+/ft+23//+93Nc/et5bthidWYPvUt76CKe96+/W8u6XXuy+p4zx1XwL+89a1jb/upXv2LFihWsXLmSjo4Ozj//fObPn8/Pf/5z3vWud/HlL3+ZZDLJwMAAK1asYOfOnaxZswaA3bt3Z7XubPDcHnp5UYCg3+joU6CLyIl5/vnnufrqq/H7/dTW1vLmN7+ZJUuWcP755/PTn/6Ur33ta6xevZry8nKmTp3K5s2b+cxnPsMf/vAHKioqcl3+63huD93MqC4tUpeLSB4Y7p70yTZ//nwWLVrE448/zvXXX8/nP/95PvrRj7Jy5Uqeeuop7r77bhYuXMh9992X61IP4Lk9dEh3u+igqIicqHnz5vHwww+TTCZpb29n0aJFXHDBBWzbto3a2lo++clP8olPfILly5fT0dFBKpXiAx/4AN/4xjdYvnx5rst/Hc/toUP6wKj20EXkRF1xxRW88MILnH322ZgZ3/nOd6irq+OBBx7gu9/9LsFgkLKyMh588EF27tzJDTfcQCqVAuBb3/pWjqt/PXPO5eSD58yZ4473Bheff3gFi7d08dc7LslyVSIy0tavX8+ZZ56Z6zI84VB/Vma2zDk351Dbe7jLRXvoIiJDeTTQi4jEUwzEErkuRUTklOHJQK8qzZwtqqGLIiL7eDLQ957+36EDoyIi+3gy0KtL957+rz10EZG9vBno+y7QpT10EZG9vBnomT10nf4vIrKfJwO9OOSnNORXl4uIjLgjXTt969atzJo16yRWc2SeDHRID13sUpeLiMg+njz1H3Q9F5G88OQd0LI6u+9Z1wjv/vZhV99xxx1MmDCBW265BYCvfe1rBAIBnnvuObq7u4nH43zjG9/g8ssvP6aPjUQi3HTTTSxdupRAIMD3vvc93vrWt7J27VpuuOEGYrEYqVSKRx99lHHjxnHVVVfR1NREMpnkn//5n1mwYMEJNRu8HOilIXbuPrY7YouILFiwgM9+9rP7An3hwoU89dRT3HrrrVRUVNDR0cHcuXN53/ved0w3ar7zzjsxM1avXs2GDRt45zvfycaNG7n77ru57bbbuOaaa4jFYiSTSZ544gnGjRvH448/DkBPT09W2ubhQC9iVVN2/hBEJEeOsCc9Us4991za2trYtWsX7e3tVFZWUldXx+c+9zkWLVqEz+dj586dtLa2UldXN+z3ff755/nMZz4DwIwZM5g0aRIbN27kwgsv5Jvf/CZNTU1ceeWVTJs2jcbGRm6//Xb+6Z/+ife85z3MmzcvK23zcB96iK7+GKlUbi4uJiLe9aEPfYhHHnmEhx9+mAULFvDQQw/R3t7OsmXLWLFiBbW1tUQi2ekB+MhHPsJjjz1GcXExl112Gc8++yxnnHEGy5cvp7Gxka985St8/etfz8pneXcPvayIRMqxJxJndEko1+WIiIcsWLCAT37yk3R0dPDnP/+ZhQsXMnbsWILBIM899xzbtm075vecN28eDz30EJdccgkbN25k+/btTJ8+nc2bNzN16lRuvfVWtm/fzqpVq5gxYwZVVVVce+21jB49mnvvvTcr7fJsoO8//T+mQBeRY3LWWWfR29vL+PHjqa+v55prruG9730vjY2NzJkzhxkzZhzze958883cdNNNNDY2EggEuP/++ykqKmLhwoX87Gc/IxgMUldXx5e+9CWWLFnCF77wBXw+H8FgkLvuuisr7fLk9dABnn+1g2t/spiFn7qQC6ZUZbEyERlJuh768GX9euhmNsHMnjOzdWa21sxuO8Q2bzGzHjNbkZm+etwtGKZ9p//rAl0iIsDwulwSwO3OueVmVg4sM7OnnXPrDtruL86592S/xEOrzlxCt0Nj0UVkhK1evZrrrrvugGVFRUUsXrw4RxUd2lED3TnXDDRn5nvNbD0wHjg40E+qylLtoYt4lXPumMZ451pjYyMrVqw4qZ95PN3hxzRs0cwmA+cCh/pZutDMVprZk2Z21mFef6OZLTWzpe3t7cdc7FBBnzG6JKjruYh4TDgcprOz87gCq1A45+js7CQcDh/T64Y9ysXMyoBHgc865/YctHo5MMk512dmlwG/AaYdosh7gHsgfVD0mCrda91j8Jub4FOLqC7VvUVFvKahoYGmpiZOdKcu34XDYRoaGo7pNcMKdDMLkg7zh5xzvzp4/dCAd849YWb/YWZjnHMdx1TNcBSVQawP+lqpLivSJXRFPCYYDDJlypRcl5GXhjPKxYCfAOudc987zDZ1me0wswsy79uZzUL3KcucitvbzJjM2aIiIjK8PfSLgeuA1Wa296jAl4CJAM65u4EPAjeZWQIYBD7sRqqDrHxvoLdSXTqdzr6R+d0QEfGa4YxyeR444uFo59yPgR9nq6gjKq4EfxH0NlNVGqJ7IE4imSLg9+xlaUREssJ7KWgG5bXQ17rv9P+uAXW7iIh4L9AByuuht5nqsvS9RTV0UUTEq4FeVpvpQ997cpECXUTEm4FeXg+9Lfv30DUWXUTEq4FeC9EexhQlAO2hi4iAZwO9HoCKeBcBn2kPXUQErwZ6WS0Avv5WKktD2kMXEcGrgZ7ZQ6e3merSkE7/FxHBs4GeOVu0r5UxZUXqchERwauBXlwJ/lBmLLq6XEREwKuBbpa+SFdvK9WlRbrJhYgIXg10SHe7ZPbQ+2NJIvFkrisSEckpDwd6LfS27LueS6cuoysiBc7DgV4PfS1Ul+69nou6XUSksHk30MtqIdJDdTgF6GxRERHvBnpmLPpYdgPQoT10ESlwHg709NmiVan0HYvUhy4ihc7DgZ7eQw9H2gkHfepDF5GC5/lAt8yBUe2hi0ih826g7ztbND10UQdFRaTQeTfQ950tmr7Rha7nIiKFzruBDpmbRbdQpUvoioh4PdD37qGnA905l+uKRERyxtuBnulyGVNaRCyZojeayHVFIiI54+1AL6+DyG5qinW2qIiI9wMdqPfvAaBLB0ZFpIDlRaDXuC4A2nsV6CJSuLwd6GXpQK/1pa/n0tQ9mMtqRERyytuBnjlbtCTaTnlRgB1dAzkuSEQkd7wd6CVV4Atifa1MqCphuwJdRAqYtwPdbN9Y9IkKdBEpcN4OdEjf6KK3mYnVJezoHiSV0slFIlKYjhroZjbBzJ4zs3VmttbMbjvENmZmPzSz18xslZmdNzLlHkJ5HfSmu1xiiRRtGukiIgVqOHvoCeB259xMYC5wi5nNPGibdwPTMtONwF1ZrfJIyuvSe+hVJQDqdhGRgnXUQHfONTvnlmfme4H1wPiDNrsceNClvQiMNrP6rFd7KJmzRSdVpJuiQBeRQnVMfehmNhk4F1h80KrxwI4hz5t4feiPjMzQxXH+PZgp0EWkcA070M2sDHgU+Kxzbs/xfJiZ3WhmS81saXt7+/G8xetlTi4KDbYxblSxxqKLSMEaVqCbWZB0mD/knPvVITbZCUwY8rwhs+wAzrl7nHNznHNzampqjqfe18uc/k9vMxOqirWHLiIFazijXAz4CbDeOfe9w2z2GPDRzGiXuUCPc645i3Ue3t5A72tlYlWJ9tBFpGAFhrHNxcB1wGozW5FZ9iVgIoBz7m7gCeAy4DVgALgh+6UeRnH6bNG9I13aeqMMxpIUh/wnrQQRkVPBUQPdOfc8YEfZxgG3ZKuoY+LzZU4uamXClPTQxabuAabVluekHBGRXPH+maKgsegiIuRToGf60EGBLiKFKX8CvbeZqtIQpSG/Al1EClJ+BHpZHQx2Y8kYEzTSRUQKVH4E+r6x6LqMrogUrvwK9CE3ukgPvBERKRx5EuiZ64D17GBiVQmReIr2Pl1GV0QKS34EevXpYH5oXbtvpIv60UWk0ORHoAfDUDMDmlcxQUMXRaRA5UegA9TPhpZVNFQWA7C9czDHBYmInFz5E+h1s6GvlXCkg7qKsPbQRaTg5E+g189OP7as0lUXRaQg5U+g1zWmH5tX7hu6KCJSSPIn0MOjoHLyvj30lj0RIvFkrqsSETlp8ifQId2P3ryKidXpA6NN3TowKiKFI78CvX42dG9hclkC0Fh0ESks+RXodWcDMDmxFYAd3Qp0ESkc+RXomZEuo3vWEw762N6pQBeRwpFfgV5WC6U1WMtqXXVRRApOfgW62f4Dowp0ESkw+RXokO52aV/P5NEBdugyuiJSQPIv0OtmQyrB7FAz/bEkXf2xXFckInJS5F+g16dHupye3AzoqosiUjjyL9Arp0ConHGRjYACXUQKR/4Fus8HdbMo370eM3itrS/XFYmInBT5F+gAdbPxt67lvIZyFm1sz3U1IiInRX4Gev1siPdzxaQoK5t6aOuN5LoiEZERl5+BXpc+Y/TNFS0A/GmD9tJFJP/lZ6DXzABfkIbIq9RVhHl2Q1uuKxIRGXH5GeiBEIw9E2tdzSVnjuUvr7YTTeja6CKS3/Iz0CHdj968irdNr6E/luSlLV25rkhEZETlb6DXnQ0DHVxUG6co4OOZ9ep2EZH8lr+BnrmUbnHHWi46rZpnN7Tpui4ikteOGuhmdp+ZtZnZmsOsf4uZ9ZjZisz01eyXeRxqzwLzw+bnuOTMWrZ3DbCpvT/XVYmIjJjh7KHfD1x6lG3+4pw7JzN9/cTLyoKicph9FSx7gLdPSC96dkNrbmsSERlBRw1059wiwJtHFOd/AZIx6lffzYy6cvWji0hey1Yf+oVmttLMnjSzsw63kZndaGZLzWxpe/tJONmn+jQ4+2pY8hPed5qxdFs3PQPxkf9cEZEcyEagLwcmOefOBn4E/OZwGzrn7nHOzXHOzampqcnCRw/D/H8Al+SDA4+QTDkWvaqzRkUkP51woDvn9jjn+jLzTwBBMxtzwpVlS9UUOOcj1Gz8OTNKenXWqIjkrRMOdDOrMzPLzF+Qec/OE33frJr3D5hzfGXUkzz3ShvJlIYvikj+Gc6wxV8ALwDTzazJzD5uZp82s09nNvkgsMbMVgI/BD7sTrUB35WT4LzruGj345QMNPPy9u5cVyQiknWBo23gnLv6KOt/DPw4axWNlHm3Yy//F58J/pbfr5rDnMlVua5IRCSr8vdM0YONasDO+xhX+f/E/7ywhMdW7sp1RSIiWVU4gQ4w7/P4/AHuL7uTexc+xoubT62ufhGRE1FYgV4xDrvyP5ka6OTXwS+y6cGbeW17U66rEhHJisIKdICzrsB36zIGZ3+MD/NHqu+7kJ6//RRSqVxXJiJyQixXA1LmzJnjli5dmpPP3mvTqr+x59HbONc2kqpowFc7E8acATXT04+jJ0GoND35/DmtVUQEwMyWOefmHGrdUUe55LPTZl/EoqLf8/mf/YD3DKykcdcWqjf/GV8y+vqN/UUQKoFgCfgCB03+9GS+9BUezTfkuR24/IDJ9s/7hq4f8h4+f/q5zz/ksw7+7L3zwfRzfzA97w+CP5SZMvOBMASKhkxhCBZDoDi9TfqUAhHxoIIOdID502uJfeQ27vzzJpZv78ZcirlV/VzR0Me5o/qp8McosSjFRPAnBiEegVTi9ZNLpadUcv+8S6W7cvYtT4JzmXV7Hw/efu82yfR8KpGZTw2Zz3xmtpk/He7BkvT/SorKIFSeeSyD4tFQXAnh0fvny2qhbGz6MVic/ZpEZNgKPtAB3j6zlrfPrKVtT4Q/rmvlqbUtfHFNJ4mDzigtDwcYXRIk6PcR8vsI+I2g30fQ58PnA58Zfp9hZvgt/dx8hi8z7/OBmaXn964nvcyfef3Q+b3v5/ftnQe/GT6fETDw+xwhkgR8ECRB0FIESRK0IZNLELIkIRIESRAiRsjFCRAlmIoRJE4wFSWYihBIRfElIli8H2L9EO2DWB/0tUF0E0R2w+Du9I/KoRSNSod75SSompqeKqfsn/fr6yYykgq6D/1IegbirN3VQ9dAjO7+GF39cboHYvQMxoklU8QTKRIpRzyZIp5MkXKQSjlSzpHMzDtcesfaOZyDpHO4zHzKufRrDnjuSKbAOUfSOZKpzOtS6eepzONI/pX5DMJBPyUhP8UhPyXBAMUhP6VFfsqKApQXBagKxqjxD1Dt72es7aGaLkYluymPdxGOthHo2YZ1bYFY7/43DpZA/TnQMCc9jZ8Do8aPXENE8pT60I/DqJIgF51+6lxjbKi9wZ5MORKp9GMy5Ugk0z8yiaQjkRr6g5NeF8vMxxLpH6FYIj1FEkmi8RTRRJJIPMVgPJmeYkkGYgkGYkkGYkk6egfojcTpjSboiyYyPywVmWnyvvpKQ37qR4WZXhPjrKIOTvO3Mi21ifre1YQX3439LZbecMwZ0HgVNH4wfRE1ETkh2kOX45JKOfpiCXb3x+kaiNHVH6WrP05nX5TmngjNPYM090TYtTtCR9/+g8yl/gRvG93Gm0u2cFH8Rep3L0uvaLggfYeps66E0uoctUrk1HekPXQFuoy4SDzJ5vZ+Nrb28kprLxtbetnQ0svO3YOMo4NrSl/iysDfqI9uxoXKsXf8K7zhBvAV3mkSIkejQJdT0o6uAZ5/rYO/vNrO8692MD66iX8JPcRcW0Os/nxCV/wIxp6Z6zJFTikKdDnlJZIpVjbt5qEXt+Fb/TBf8v+MCl+EPefdTNWlX4ZgONclipwSFOjiKU3dA/z8uZc5Y8W3eb9vEc2hyZTf9DRllWNzXZpIzh0p0NVJKaechsoS/vHKi3nTPz7Cf0//PlXRJl678wO0dvce/cUiBUyBLqesMWVFfOjqv2Pzhf/GOYlV/PVHH+eVFoW6yOEo0OWUd+aln6Lj7E9zZeopHrnrq/z1tY5clyRySlKgiyeMufzfiEx9J3fY/fznT3/CI8t0HXuRgynQxRt8fsIL7oOaGdwV+iF3PvIH3XFK5CAKdPGOonL8H/klJeEiHgj/O//2m2XEk7oxicheCnTxlspJ2AfvZaLbRWPnkzz4wrZcVyRyylCgi/dMfStu3LncUvw0P3h6A229kVxXJHJKUKCL95hhc29mXGIHFyRf5ttPbMh1RSKnBAW6eNPM90N5PV+pfo5fvbyTJVu7cl2RSM4p0MWbAiE4/xNM7nmJi8vb+OffrCGhA6RS4BTo4l1z/g4CYb41/q9saOnlv17UAVIpbAp08a6SKjj7w0xo+h3vnhrk35/eeMDNNEQKjQJdvO2NN2GJCP+rYQm9kQQLl+7IdUUiOaNAF28bOwNOextj1j3IhZPKeHRZE7m6JLRIrinQxfvm3gx9Ldxat4ZN7f2sbOrJdUUiOaFAF+87/W0wZjoXtPySooDxqC7cJQVKgS7eZwZzP42/dRWfntrJYyt3EU0kc12VyEl31EA3s/vMrM3M1hxmvZnZD83sNTNbZWbnZb9MkaNovAoCYa4KL6ZnMM4z69tyXZHISTecPfT7gUuPsP7dwLTMdCNw14mXJXKMispg2jsYt+tp6ssD6naRgnTUQHfOLQKOdF715cCDLu1FYLSZ1WerQJFhm/l+rK+FW07v5E8b22nv1Zh0KSzZ6EMfDwwd/NuUWfY6ZnajmS01s6Xt7e1Z+GiRIc64FAJh3uNfTDLl+O2KnbmuSOSkOqkHRZ1z9zjn5jjn5tTU1JzMj5ZCkOl2Gb31D5w7voxHlyvQpbBkI9B3AhOGPG/ILBM5+Wa+H/pa+NTUDtY372HtLo1Jl8KRjUB/DPhoZrTLXKDHOdechfcVOXaZbpe3JP5K0G88ukz7FlI4hjNs8RfAC8B0M2sys4+b2afN7NOZTZ4ANgOvAf8PuHnEqhU5mky3S/jVx3n79Bp+u2Kn7jsqBSNwtA2cc1cfZb0DbslaRSInaub7Yf3v+LvzW3lynfGnV9p5x8zaXFclMuJ0pqjkn0y3y3l9f6KmvIifL9Z10qUwKNAl/2S6Xfzrf8fV5zfwp43tbO3oz3VVIiNOgS75KTPa5fqGFvxmupuRFAQFuuSnTLdL1dbHedesOhYu3cFALJHrqkRGlAJd8lOm24V1j/GxuRPZE0nw2xW7cl2VyIhSoEv+ynS7nO/bwIy6ch7421bdzUjymgJd8tcZl0JRBfa3H/GxiyazoaWXJVu7c12VyIhRoEv+KiqDebfDxj9wxaiNVIQDPPDC1lxXJTJiFOiS3+beBKMnEX7mqyx4wzieWtNCS08k11WJjAgFuuS3QBG84+vQtpZPlf+NpHP8/KXtua5KZEQo0CX/zbwcJl7EmCXf5dLTS/jFS9uJJXR9F8k/CnTJf2bwrm9Cfzt3lD1Je2+UJ9fogqCSfxToUhjGnwezP8zEjfczr6afr/52LRta9uS6KpGsUqBL4XjbVzHz8Z91vyMc9HHtvS+xRdd4kTyiQJfCMWo8XHwbJa8+xqOXQco5rr13MTt3D+a6MpGsUKBLYbn4VqhooOHx6/jd3FfYE4lx7b2Lae+N5roykROmQJfCEiqFj/8RJryR8X/9Ms+P/w+SPbu47ieL2T0Qy3V1IidEgS6FZ9R4uO7XcNn/YVTrSzxTfAdndjzNe3/8PA8t3kYknsx1hSLHxXJ1saI5c+a4pUuX5uSzRfbp3AS//hQ0LWF58A3c3T+flcVz+di807l27iQqwsFcVyhyADNb5pybc8h1CnQpeMkEvPBj3It3YX0t9PhG83DsYn7vfxtvmDOXd8+q5w2TKvH7LNeViijQRYYlmYBNz8DLP8NteBJzCda5SbyQnMm60CzKps3jwsbpzD9jDCWho95fXWREKNBFjlVfO6x6mMT6x7Gdy/Cn0qNgXkuNYzkz6KpspGTKXE4/6w2cN3kM4aA/xwVLoVCgi5yIRBR2vUxy61/Zs2ER4dZlFCd7Aeh3RaxlKi1lM0k2vJExZ85n1rTTqCwN5bhoyVcKdJFsSqWgewuDW16k45UX8O1aztj+VwiSvmfpplQ9G4rOoq9mDmVTL2DqmedyRr364CU7FOgiIy0eIbp9Ga1r/0Rq2wvUdK+gNJXei4+4IBuZSGvpdJJjGxk15Q1MOet86sZU5bho8SIFusjJlkrhOl6h49Ul9Gxeiq91NWP7X6HMpa8dk3TGDqunrfQMkmPPonzSuUyccT4VtZPSV4cUOQwFusipwDmiHVtp2rCYPVtext+2hjH9rzLOte7bpI9S2oqnEK06k5KJs6mbdj5F4xvTt9MTQYEuckrr6e5k27oldG19GVrXMbr3VaamtlFhAwCkMDpD4+mvnElo/GzGTD2bUP1ZUDkZfBpdU2gU6CIe09ozyMaNG+jctBTXvIpRPRs4PbWFib72fdvELURv2RQYeyblE2YRrD0Tamakg96vcfL5SoEu4nHOOZp7Iqzd0kTrppVEdq2jqHsjExLbmeZrYrx17ts2aUEio6YSqG+kaMI5UDcb6mdDcWUOWyDZokAXyUPOOXbuHmTNzh5e3dHM7u1rcW2vMDa6lWnWxEzfNuqta9/2A8XjSNXMoLhuOv6aaVB9OoyZBuX1OhDrIQp0kQLS2RdlXfMeNjT3sn3HdpLNqxjds44ZbOU028UUa6HE9l//PeEvIVE5heDYM/CPyQR95WQor0tPgaLcNUZe54QD3cwuBX4A+IF7nXPfPmj99cB3gZ2ZRT92zt17pPdUoIucPPFkik3tfWxs7WNjcw9tu7YSb3uFst6tTLFmplgzU30tNFg7flIHvDZVXI1V1GHl42BUA4yeAKP2Tg3pPXz12Z80JxToZuYHNgLvAJqAJcDVzrl1Q7a5HpjjnPv74RalQBfJvYFYgs3t/bzW1sem9j62tHYz2PIa9OygynVRRxe11k29fzcT/d3U0UF56sCbazvzQVktVjEeKsZBxXgor4Wyuv2PZbVQPFqjcrLgSIE+nJ/VC4DXnHObM2/2S+ByYN0RXyUip7ySUIBZ40cxa/yoIUvnkkw5du0eZFvnAFs6+3mho59fdA7Q1D1AZ3c3FbFWxlsH46yTeuuioaeLSQO7qW99merU04RTr79Pq8MgPAorrkwfoC2uhPAoCFekH4syj6EyCIQgEE539wTCECyG8Oj92wV0rZxDGU6gjwd2DHneBLzxENt9wMzmk96b/5xzbschthERD/D7jAlVJUyoKuFN08YcsM45x57BBDu60wG/a3eEDT2DPLs7ws7dgzT3DDLQt4dq18VYdlNjPYy1bkZbH9UD/YyNDTKmr5/R1kQ5r1Cc6qc42YffxYdfYKAYisqHhH5m8hdBMJxZnvkhCBSlbz0YKk+foBUqSz/6AnsbNKThQQiWpLcJleyfDxanp1P8fxjZ6vj6HfAL51zUzD4FPABccvBGZnYjcCPAxIkTs/TRInIymRmjSoKMKjl4z36/VMrRPRCjvS9Ke2966uiLsqMvxvK+KJ19MTr6onT3x+iOxhmMJykiRjmDlFiEEHGK9k4WpyqUoCYQoToQoco/SKVvkAobpNgShFMJwvE4RYkEIaAGGKwAAAYlSURBVBclSD+BVJRAKoY/GcWXjOBL9ONLRE688ft+JIrT4e8Pgi/z6A9lpmDmxyW0/wciWJx5bWZ+whth8sUnXs/B5Q1jm53AhCHPG9h/8BMA51znkKf3At851Bs55+4B7oF0H/oxVSoinuHzGdVlRVSXFTGj7ujbR+JJugdidPfH6RmMsyeSeRyMsyeSYM9gnP5ogtZogr5ogt5Igt5InN7+9PzgMO4D6ydJKRHKLcKYUJySIISDPooCfoqCfooCfkr8Kcr9UcosSqlFKSVCicUoJkaJRQkTJeyihIgStCRBlyRgCQIugd/F8bkEvugANtiDLxmDZAxLRrD4IMQHIZ4++5c3fT5ngb4EmGZmU0gH+YeBjwzdwMzqnXPNmafvA9ZntUoRyWvhoJ/6UcXUjyo+rtfHkyn6o/vDfSCWZDCWZDCeYCC2/3l/LMFANP0YGbJdVzz9GIklicZTRBMpookkkXiKSDxJNJE6ehGHYQbhgJ9w0EdxyEdFMMmCwARuOO53PLyjBrpzLmFmfw88RXrY4n3OubVm9nVgqXPuMeBWM3sfkAC6gOtHoFYRkUMK+n2MLgkxumRkDpamUo7BeDI97f2BiCcZiCUOeJ5IOhKpFPGkI5FMkUg5opnX7f1xiCRSVI46dFfVidKJRSIiHnKkYYu+k12MiIiMDAW6iEieUKCLiOQJBbqISJ5QoIuI5AkFuohInlCgi4jkCQW6iEieyNmJRWbWDmw7zpePATqyWE6uqT2nrnxqC+RXe/KpLTD89kxyztUcakXOAv1EmNnSw50p5UVqz6krn9oC+dWefGoLZKc96nIREckTCnQRkTzh1UC/J9cFZJnac+rKp7ZAfrUnn9oCWWiPJ/vQRUTk9by6hy4iIgdRoIuI5AnPBbqZXWpmr5jZa2Z2R67rOVZmdp+ZtZnZmiHLqszsaTN7NfNYmcsah8vMJpjZc2a2zszWmtltmeVebU/YzF4ys5WZ9vxrZvkUM1uc+c49bGYjc1ucEWBmfjN72cx+n3nu5bZsNbPVZrbCzJZmlnn1uzbazB4xsw1mtt7MLsxGWzwV6GbmB+4E3g3MBK42s5m5reqY3Q9cetCyO4BnnHPTgGcyz70gAdzunJsJzAVuyfx9eLU9UeAS59zZwDnApWY2F/jfwPedc6cD3cDHc1jjsbqNA+/x6+W2ALzVOXfOkPHaXv2u/QD4g3NuBnA26b+jE2+Lc84zE3Ah8NSQ518Evpjruo6jHZOBNUOevwLUZ+brgVdyXeNxtuu3wDvyoT1ACbAceCPps/cCmeUHfAdP5QloyATDJcDvAfNqWzL1bgXGHLTMc981YBSwhcyglGy2xVN76MB4YMeQ502ZZV5X65xrzsy3ALW5LOZ4mNlk4FxgMR5uT6aLYgXQBjwNbAJ2O+cSmU289J37v8A/AntvWV+Nd9sC4IA/mtkyM7sxs8yL37UpQDvw00x32L1mVkoW2uK1QM97Lv3z7KmxpGZWBjwKfNY5t2foOq+1xzmXdM6dQ3rv9gJgRo5LOi5m9h6gzTm3LNe1ZNGbnHPnke5yvcXM5g9d6aHvWgA4D7jLOXcu0M9B3SvH2xavBfpOYMKQ5w2ZZV7Xamb1AJnHthzXM2xmFiQd5g85536VWezZ9uzlnNsNPEe6W2K0mQUyq7zynbsYeJ+ZbQV+Sbrb5Qd4sy0AOOd2Zh7bgF+T/sH14netCWhyzi3OPH+EdMCfcFu8FuhLgGmZI/Uh4MPAYzmuKRseAz6Wmf8Y6b7oU56ZGfATYL1z7ntDVnm1PTVmNjozX0z6eMB60sH+wcxmnmiPc+6LzrkG59xk0v9OnnXOXYMH2wJgZqVmVr53HngnsAYPfteccy3ADjObnln0NmAd2WhLrg8QHMcBhcuAjaT7Nr+c63qOo/5fAM1AnPQv9cdJ920+A7wK/A9Qles6h9mWN5H+b+EqYEVmuszD7ZkNvJxpzxrgq5nlU4GXgNeA/waKcl3rMbbrLcDvvdyWTN0rM9Pavf/2PfxdOwdYmvmu/QaozEZbdOq/iEie8FqXi4iIHIYCXUQkTyjQRUTyhAJdRCRPKNBFRPKEAl1EJE8o0EVE8sT/B5foSpgqq4RUAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "6auA4aw8Gz38", "outputId": "ca75872c-293a-4ca8-f0cd-ac9030db4b9d" }, "source": [ "predictions = model.predict(X_test_scaled)\r\n", "print('R2:', r2_score(y_test, predictions))\r\n", "print('MSE:', mean_squared_error(y_test, predictions))" ], "execution_count": 44, "outputs": [ { "output_type": "stream", "text": [ "R2: 0.6493164194181347\n", "MSE: 0.46028626466547323\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "4XW8eIhbnRwX" }, "source": [ "## Different layer weight initializers" ] }, { "cell_type": "code", "metadata": { "id": "W9uePY_JoQz-", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "18b9eaa8-b53f-4159-f1e0-e3500c797a73" }, "source": [ "[name for name in dir(tf.keras.initializers) if not name.startswith(\"_\")]" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "['Constant',\n", " 'GlorotNormal',\n", " 'GlorotUniform',\n", " 'HeNormal',\n", " 'HeUniform',\n", " 'Identity',\n", " 'Initializer',\n", " 'LecunNormal',\n", " 'LecunUniform',\n", " 'Ones',\n", " 'Orthogonal',\n", " 'RandomNormal',\n", " 'RandomUniform',\n", " 'TruncatedNormal',\n", " 'VarianceScaling',\n", " 'Zeros',\n", " 'constant',\n", " 'deserialize',\n", " 'get',\n", " 'glorot_normal',\n", " 'glorot_uniform',\n", " 'he_normal',\n", " 'he_uniform',\n", " 'identity',\n", " 'lecun_normal',\n", " 'lecun_uniform',\n", " 'ones',\n", " 'orthogonal',\n", " 'random_normal',\n", " 'random_uniform',\n", " 'serialize',\n", " 'truncated_normal',\n", " 'variance_scaling',\n", " 'zeros']" ] }, "metadata": { "tags": [] }, "execution_count": 71 } ] }, { "cell_type": "code", "metadata": { "id": "6M2MsyW5oSjY", "colab": { "base_uri": "https://localhost:8080/", "height": 265 }, "outputId": "3f629227-f06d-4922-ed8e-bc92ee6be13d" }, "source": [ "input_x = np.array([[1]])\n", "dense1 = tf.keras.layers.Dense(100, activation=\"relu\", kernel_initializer=\"uniform\", input_shape=(input_x.shape[0],))\n", "y = dense1(input_x)\n", "plt.hist(dense1.weights[0][0]);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAM8ElEQVR4nO3dbYxmd1nH8e+PXSlUCN3SoUJLnWIISduomImIJIoUpFCkBHlRIloezL7wCVGCS2oCPrwo4AMajGQDlSZiQQtEYqOyIA2aYHW2VGkppaWt0FLoIAalRLDh8sWcxukwuzN7nzP3vZfz/SSTuR/O3Of672y+e3Luh01VIUnq52GLHkCSNBsDLklNGXBJasqAS1JTBlySmto/z52dccYZtby8PM9dSlJ7R48e/XJVLW2+fa4BX15eZnV1dZ67lKT2kvzbVrd7CkWSmjLgktSUAZekpgy4JDVlwCWpKQMuSU0ZcElqyoBLUlMGXJKamus7MSV9u+VD1y56hLm764qLF7LfRf5Z78aaPQKXpKYMuCQ1ZcAlqSkDLklNGXBJasqAS1JTBlySmjLgktSUAZekpgy4JDVlwCWpqW0DnuTKJPcluWnDbW9J8ukk/5rkA0lO290xJUmb7eQI/F3ARZtuOwJcUFXfC3wGeP3Ec0mStrFtwKvqY8BXNt32oap6YLj6j8DZuzCbJOk4pjgH/krgryd4HEnSCRgV8CSXAw8A7z7ONgeTrCZZXVtbG7M7SdIGMwc8ycuBFwA/VVV1rO2q6nBVrVTVytLS0qy7kyRtMtP/yJPkIuB1wI9W1denHUmStBM7eRnh1cDHgackuTvJq4C3AY8GjiS5Mcnbd3lOSdIm2x6BV9VLt7j5nbswiyTpBPhOTElqyoBLUlMGXJKaMuCS1JQBl6SmDLgkNWXAJakpAy5JTRlwSWrKgEtSUwZckpqa6dMINR/Lh65d9Ahzd9cVFy96BM3BXvy7vRs8Apekpgy4JDVlwCWpKQMuSU0ZcElqyoBLUlMGXJKaMuCS1JQBl6SmDLgkNWXAJampbQOe5Mok9yW5acNtpyc5kuS24fuB3R1TkrTZTo7A3wVctOm2Q8BHqurJwEeG65KkOdo24FX1MeArm26+BLhquHwV8KKJ55IkbWPWc+BnVtW9w+UvAmdONI8kaYdGP4lZVQXUse5PcjDJapLVtbW1sbuTJA1mDfiXkjweYPh+37E2rKrDVbVSVStLS0sz7k6StNmsAf8gcNlw+TLgL6cZR5K0Uzt5GeHVwMeBpyS5O8mrgCuA5yS5DXj2cF2SNEfb/p+YVfXSY9x14cSzSJJOgO/ElKSmDLgkNWXAJakpAy5JTRlwSWrKgEtSUwZckpoy4JLUlAGXpKYMuCQ1te1b6aV5Wj507UL2e9cVFy9kv9IYHoFLUlMGXJKaMuCS1JQBl6SmDLgkNWXAJakpAy5JTRlwSWrKgEtSUwZckpoy4JLUlAGXpKZGBTzJa5LcnOSmJFcnecRUg0mSjm/mgCc5C/glYKWqLgD2AZdONZgk6fjGnkLZDzwyyX7gVOAL40eSJO3EzAGvqnuA3wE+B9wLfLWqPrR5uyQHk6wmWV1bW5t9UknSQ4w5hXIAuAQ4F3gC8J1JXrZ5u6o6XFUrVbWytLQ0+6SSpIcYcwrl2cCdVbVWVf8DvB/44WnGkiRtZ0zAPwf8UJJTkwS4ELhlmrEkSdsZcw78euAa4Abgk8NjHZ5oLknSNkb9p8ZV9QbgDRPNIkk6Ab4TU5KaMuCS1JQBl6SmDLgkNWXAJakpAy5JTRlwSWrKgEtSUwZckpoy4JLU1Ki30u8Vy4euXfQIkvRtPAKXpKYMuCQ1ZcAlqSkDLklNGXBJasqAS1JTBlySmjLgktSUAZekpgy4JDVlwCWpKQMuSU2NCniS05Jck+TTSW5J8vSpBpMkHd/YTyP8A+BvquolSR4OnDrBTJKkHZg54EkeA/wI8HKAqvom8M1pxpIkbWfMEfi5wBrwJ0m+DzgKvLqq7t+4UZKDwEGAc845Z+ad+Znc2k3+/VJHY86B7wd+APjjqnoqcD9waPNGVXW4qlaqamVpaWnE7iRJG40J+N3A3VV1/XD9GtaDLkmag5kDXlVfBD6f5CnDTRcCn5pkKknStsa+CuUXgXcPr0C5A3jF+JEkSTsxKuBVdSOwMtEskqQT4DsxJakpAy5JTRlwSWrKgEtSUwZckpoy4JLUlAGXpKYMuCQ1ZcAlqSkDLklNGXBJasqAS1JTBlySmjLgktSUAZekpgy4JDVlwCWpKQMuSU0ZcElqyoBLUlMGXJKaMuCS1JQBl6SmRgc8yb4kn0jyV1MMJEnamSmOwF8N3DLB40iSTsCogCc5G7gYeMc040iSdmrsEfhbgdcB3zrWBkkOJllNsrq2tjZyd5KkB80c8CQvAO6rqqPH266qDlfVSlWtLC0tzbo7SdImY47AnwG8MMldwHuAZyX500mmkiRta+aAV9Xrq+rsqloGLgX+rqpeNtlkkqTj8nXgktTU/ikepKquA66b4rEkSTvjEbgkNWXAJakpAy5JTRlwSWrKgEtSUwZckpoy4JLUlAGXpKYMuCQ1ZcAlqSkDLklNGXBJasqAS1JTBlySmjLgktSUAZekpgy4JDVlwCWpKQMuSU0ZcElqyoBLUlMGXJKaMuCS1NTMAU/yxCQfTfKpJDcnefWUg0mSjm//iJ99APjVqrohyaOBo0mOVNWnJppNknQcMx+BV9W9VXXDcPm/gFuAs6YaTJJ0fJOcA0+yDDwVuH6L+w4mWU2yura2NsXuJElMEPAkjwLeB/xyVf3n5vur6nBVrVTVytLS0tjdSZIGowKe5DtYj/e7q+r904wkSdqJMa9CCfBO4Jaq+r3pRpIk7cSYI/BnAD8NPCvJjcPX8yeaS5K0jZlfRlhV/wBkwlkkSSfAd2JKUlMGXJKaMuCS1JQBl6SmDLgkNWXAJakpAy5JTRlwSWrKgEtSUwZckpoy4JLUlAGXpKYMuCQ1ZcAlqSkDLklNGXBJasqAS1JTBlySmjLgktSUAZekpgy4JDVlwCWpKQMuSU2NCniSi5LcmuT2JIemGkqStL2ZA55kH/BHwPOA84CXJjlvqsEkScc35gj8B4Hbq+qOqvom8B7gkmnGkiRtZ/+Inz0L+PyG63cDT9u8UZKDwMHh6teS3Dpin/N0BvDlRQ+xAHtx3XtxzeC65ypvGvXj373VjWMCviNVdRg4vNv7mVqS1apaWfQc87YX170X1wyue9FzTGHMKZR7gCduuH72cJskaQ7GBPyfgScnOTfJw4FLgQ9OM5YkaTszn0KpqgeS/ALwt8A+4MqqunmyyRav3WmfiezFde/FNYPrbi9VtegZJEkz8J2YktSUAZekpvZ0wJOcnuRIktuG7weOsd1lwza3Jblsi/s/mOSm3Z94GmPWneTUJNcm+XSSm5NcMd/pT8x2H/eQ5JQk7x3uvz7J8ob7Xj/cfmuS585z7rFmXXeS5yQ5muSTw/dnzXv2Mcb8vof7z0nytSSvndfMo1TVnv0C3gwcGi4fAt60xTanA3cM3w8Mlw9suP/FwJ8BNy16PfNYN3Aq8GPDNg8H/h543qLXdIx17gM+CzxpmPVfgPM2bfNzwNuHy5cC7x0unzdsfwpw7vA4+xa9pjms+6nAE4bLFwD3LHo981j3hvuvAf4CeO2i17OTrz19BM76W/+vGi5fBbxoi22eCxypqq9U1X8AR4CLAJI8CvgV4LfnMOuUZl53VX29qj4KUOsfoXAD6+8BOBnt5OMeNv5ZXANcmCTD7e+pqm9U1Z3A7cPjdTDzuqvqE1X1heH2m4FHJjllLlOPN+b3TZIXAXeyvu4W9nrAz6yqe4fLXwTO3GKbrT4y4Kzh8m8Bvwt8fdcm3B1j1w1AktOAnwA+shtDTmDbNWzcpqoeAL4KPHaHP3uyGrPujX4SuKGqvrFLc05t5nUPB2O/BvzGHOaczK6/lX7RknwY+K4t7rp845WqqiQ7fk1lku8HvqeqXrP5PNrJYLfWveHx9wNXA39YVXfMNqVOVknOB94E/PiiZ5mTNwK/X1VfGw7IW/h/H/Cqevax7kvypSSPr6p7kzweuG+Lze4Bnrnh+tnAdcDTgZUkd7H+5/i4JNdV1TM5Ceziuh90GLitqt46wbi7ZScf9/DgNncP/yg9Bvj3Hf7syWrMuklyNvAB4Geq6rO7P+5kxqz7acBLkrwZOA34VpL/rqq37f7YIyz6JPwiv4C38NAn8968xTans35e7MDwdSdw+qZtlun1JOaodbN+zv99wMMWvZZt1rmf9Sdfz+X/ntQ6f9M2P89Dn9T68+Hy+Tz0Scw76PMk5ph1nzZs/+JFr2Oe6960zRtp8iTmwgdY8C/8sayfv70N+PCGQK0A79iw3StZfxLrduAVWzxOt4DPvG7Wj2oKuAW4cfj62UWv6ThrfT7wGdZfnXD5cNtvAi8cLj+C9Vcd3A78E/CkDT97+fBzt3KSvtJm6nUDvw7cv+F3eyPwuEWvZx6/7w2P0SbgvpVekpra669CkaS2DLgkNWXAJakpAy5JTRlwSWrKgEtSUwZckpr6Xy2Llu9nFmsqAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "YOvYsYYqocG7", "colab": { "base_uri": "https://localhost:8080/", "height": 265 }, "outputId": "9b664579-a3da-403d-e5b0-8da21b02ae0a" }, "source": [ "input_x = np.array([[1]])\n", "dense1 = tf.keras.layers.Dense(100, activation=\"relu\", kernel_initializer=\"glorot_uniform\", input_shape=(input_x.shape[0],))\n", "y = dense1(input_x)\n", "plt.hist(dense1.weights[0][0]);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAMsElEQVR4nO3db6xk9V3H8fdHVqzUWsAdKQLrRSUktWmKGeufJiUpqGtpSh+QCAl1UZL7SK3ahGwlsYmP0BpbE426oQgqUuPapqSkFdi2QRNAL39KgW0B60qXAnsR/9TSSDf9+uBO3et4987cOefO7I99v5LNzpw5e8+3v6bvnJ6dczZVhSSpPd+26AEkSbMx4JLUKAMuSY0y4JLUKAMuSY3aMc+D7dy5s5aWluZ5SElq3gMPPPBCVQ3Gt8814EtLS6ysrMzzkJLUvCT/stF2L6FIUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1amLAk9yU5EiSRzf47L1JKsnO7RlPknQ805yB3wzsHt+Y5Dzgp4Gne55JkjSFiQGvqnuAFzf46IPAdYAPFJekBZjpTswklwPPVNXnkkzadxlYBti1a9cshwNgae8dM//ZVh264bJFjyDpBLblv8RMchrwG8BvTrN/Ve2rqmFVDQeD/3crvyRpRrN8C+UHgfOBzyU5BJwLPJjkdX0OJkna3JYvoVTV54Hv/db7UcSHVfVCj3NJkiaY5muEtwH3AhcmOZzk2u0fS5I0ycQz8Kq6asLnS71NI0mamndiSlKjDLgkNcqAS1KjDLgkNcqAS1KjDLgkNcqAS1KjDLgkNcqAS1KjDLgkNcqAS1KjDLgkNcqAS1KjDLgkNcqAS1KjDLgkNcqAS1KjDLgkNcqAS1KjDLgkNWqaf5X+piRHkjy6btsHknwhySNJPpbk9O0dU5I0bpoz8JuB3WPb7gLeUFVvBJ4A3tfzXJKkCSYGvKruAV4c23ZnVR0dvb0POHcbZpMkbaKPa+C/CHzyeB8mWU6ykmRldXW1h8NJkqBjwJNcDxwFbj3ePlW1r6qGVTUcDAZdDidJWmfHrH8wyTXAO4BLqqp6m0iSNJWZAp5kN3AdcHFVvdTvSJKkaUzzNcLbgHuBC5McTnIt8AfAa4C7kjyc5I+3eU5J0piJZ+BVddUGmz+8DbNIkrbAOzElqVEGXJIaZcAlqVEGXJIaZcAlqVEGXJIaZcAlqVEGXJIaZcAlqVEzP8xKUtuW9t6x6BFOKoduuKz3n+kZuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1amLAk9yU5EiSR9dtOzPJXUmeHP1+xvaOKUkaN80Z+M3A7rFte4EDVXUBcGD0XpI0RxMDXlX3AC+Obb4cuGX0+hbgXT3PJUmaYNZr4GdV1bOj188BZx1vxyTLSVaSrKyurs54OEnSuM5/iVlVBdQmn++rqmFVDQeDQdfDSZJGZg3480nOBhj9fqS/kSRJ05g14LcDe0av9wAf72ccSdK0pvka4W3AvcCFSQ4nuRa4AfipJE8Cl47eS5LmaOK/iVlVVx3no0t6nkWStAXeiSlJjTLgktQoAy5JjTLgktQoAy5JjTLgktQoAy5JjTLgktQoAy5JjZp4J6YWZ2nvHQs57qEbLlvIcSVtjWfgktQoAy5JjTLgktQoAy5JjTLgktQoAy5JjTLgktQoAy5JjTLgktQoAy5JjeoU8CS/luSxJI8muS3Jq/oaTJK0uZkDnuQc4FeAYVW9ATgFuLKvwSRJm+t6CWUH8J1JdgCnAV/pPpIkaRozP42wqp5J8rvA08DXgTur6s7x/ZIsA8sAu3btmvVwOkks6gmMi+TTHzWrLpdQzgAuB84Hvg94dZKrx/erqn1VNayq4WAwmH1SSdL/0eUSyqXAP1fValV9A/go8JP9jCVJmqRLwJ8GfjzJaUkCXAIc7GcsSdIkMwe8qu4H9gMPAp8f/ax9Pc0lSZqg0z+pVlXvB97f0yySpC3wTkxJapQBl6RGGXBJapQBl6RGGXBJapQBl6RGGXBJapQBl6RGGXBJalSnOzH1ynQyPtJ1kVxvzcozcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEYZcElqlAGXpEYZcElqVKeAJzk9yf4kX0hyMMlP9DWYJGlzXR9m9fvAp6rqiiSnAqf1MJMkaQozBzzJa4G3AtcAVNXLwMv9jCVJmqTLJZTzgVXgT5M8lOTGJK8e3ynJcpKVJCurq6sdDidJWq9LwHcAPwL8UVVdBHwN2Du+U1Xtq6phVQ0Hg0GHw0mS1usS8MPA4aq6f/R+P2tBlyTNwcwBr6rngC8nuXC06RLg8V6mkiRN1PVbKL8M3Dr6BsqXgF/oPpIkaRqdAl5VDwPDnmaRJG2Bd2JKUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1yoBLUqMMuCQ1qnPAk5yS5KEkn+hjIEnSdPo4A38PcLCHnyNJ2oJOAU9yLnAZcGM/40iSptX1DPxDwHXAN3uYRZK0BTMHPMk7gCNV9cCE/ZaTrCRZWV1dnfVwkqQxXc7A3wK8M8kh4CPA25L8xfhOVbWvqoZVNRwMBh0OJ0lab+aAV9X7qurcqloCrgQ+XVVX9zaZJGlTfg9ckhq1o48fUlWfBT7bx8+SJE3HM3BJapQBl6RGGXBJapQBl6RGGXBJapQBl6RGGXBJapQBl6RGGXBJapQBl6RGGXBJapQBl6RGGXBJapQBl6RGGXBJapQBl6RGGXBJapQBl6RGGXBJapQBl6RGGXBJatTMAU9yXpLPJHk8yWNJ3tPnYJKkze3o8GePAu+tqgeTvAZ4IMldVfV4T7NJkjYx8xl4VT1bVQ+OXn8VOAic09dgkqTN9XINPMkScBFw/wafLSdZSbKyurrax+EkSfQQ8CTfBfwN8KtV9Z/jn1fVvqoaVtVwMBh0PZwkaaRTwJN8O2vxvrWqPtrPSJKkaXT5FkqADwMHq+r3+htJkjSNLmfgbwHeDbwtycOjX2/vaS5J0gQzf42wqv4eSI+zSJK2wDsxJalRBlySGmXAJalRBlySGmXAJalRBlySGmXAJalRBlySGmXAJalRBlySGmXAJalRBlySGmXAJalRBlySGmXAJalRBlySGmXAJalRBlySGmXAJalRBlySGmXAJalRnQKeZHeSLyZ5KsnevoaSJE02c8CTnAL8IfCzwOuBq5K8vq/BJEmb63IG/mbgqar6UlW9DHwEuLyfsSRJk+zo8GfPAb687v1h4MfGd0qyDCyP3v5Xki92OOY87AReWPQQJwDX4RjXYo3rsGamdchvdzrm92+0sUvAp1JV+4B9232cviRZqarhoudYNNfhGNdijeuw5kRahy6XUJ4Bzlv3/tzRNknSHHQJ+D8CFyQ5P8mpwJXA7f2MJUmaZOZLKFV1NMkvAX8LnALcVFWP9TbZ4jRzuWebuQ7HuBZrXIc1J8w6pKoWPYMkaQbeiSlJjTLgktSokz7gSc5McleSJ0e/n7HBPm9Kcm+Sx5I8kuTnFjHrdppmHUb7fSrJvyf5xLxn3E6THguR5DuS/NXo8/uTLM1/yu03xTq8NcmDSY4muWIRM87LFGvx60keHzXhQJINv6u9nU76gAN7gQNVdQFwYPR+3EvAz1fVDwO7gQ8lOX2OM87DNOsA8AHg3XObag6mfCzEtcC/VdUPAR8Eut2WcQKach2eBq4B/nK+083XlGvxEDCsqjcC+4Hfme+UBhzWbv+/ZfT6FuBd4ztU1RNV9eTo9VeAI8BgbhPOx8R1AKiqA8BX5zXUnEzzWIj167MfuCRJ5jjjPExch6o6VFWPAN9cxIBzNM1afKaqXhq9vY+1e2HmyoDDWVX17Oj1c8BZm+2c5M3AqcA/bfdgc7aldXiF2eixEOccb5+qOgr8B/A9c5lufqZZh5PFVtfiWuCT2zrRBrb9VvoTQZK7gddt8NH1699UVSU57vcqk5wN/Dmwp6qaOwPpax0kHZPkamAIXDzvY58UAa+qS4/3WZLnk5xdVc+OAn3kOPt9N3AHcH1V3bdNo26rPtbhFWqax0J8a5/DSXYArwX+dT7jzY2PxzhmqrVIcilrJ0AXV9V/z2m2/+UllLXb//eMXu8BPj6+w+hRAR8D/qyq9s9xtnmauA6vYNM8FmL9+lwBfLpeeXfB+XiMYyauRZKLgD8B3llViznhqaqT+hdr1zEPAE8CdwNnjrYPgRtHr68GvgE8vO7XmxY9+7zXYfT+74BV4OusXRf8mUXP3tN//rcDT7D2dxvXj7b9Fmv/4wR4FfDXwFPAPwA/sOiZF7QOPzr67/1rrP0/kMcWPfMC1+Ju4Pl1Tbh93jN6K70kNcpLKJLUKAMuSY0y4JLUKAMuSY0y4JLUKAMuSY0y4JLUqP8BPV+cbV4ZIQQAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "muC-thr4qrXp", "colab": { "base_uri": "https://localhost:8080/", "height": 265 }, "outputId": "f1f4dee3-107d-4448-f3b1-5454ab4ddcb1" }, "source": [ "input_x = np.array([[1]])\n", "dense1 = tf.keras.layers.Dense(100, activation=\"relu\", kernel_initializer=\"lecun_uniform\", input_shape=(input_x.shape[0],))\n", "y = dense1(input_x)\n", "plt.hist(dense1.weights[0][0]);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAANBUlEQVR4nO3df4xl9V3G8fcjK0WQFOiOlALbgYSg2JhAJpWCqQ2gQdpAjTWBWAXFjI2pVmPSbNPEJv5jq8aoqdpsKBYjoU2xtdgWyxZKiElBZymUH0vLD7FdBHYqCRU1pdiPf8zBjMPM3Lv3nLl3vrvvVzKZc889c88zZyfPnvnec76TqkKS1J7vm3UASdJkLHBJapQFLkmNssAlqVEWuCQ1asc0d7Zz586an5+f5i4lqXn79u37VlXNrV0/1QKfn59naWlpmruUpOYl+df11juEIkmNssAlqVEWuCQ1ygKXpEZZ4JLUKAtckhplgUtSoyxwSWqUBS5JjZrqnZiSBDC/+3Mz2e+TH3zrTPa7VTwDl6RGWeCS1CgLXJIaZYFLUqMscElqlAUuSY2ywCWpURa4JDXKApekRlngktQoC1ySGjWywJNcn+RgkgdXrfvDJI8k+WqSTyc5YWtjSpLWGucM/GPApWvW7QXeUFU/BnwdeN/AuSRJI4ws8Kq6C3huzbrbquql7uHdwGlbkE2StIkhxsB/Bbh1gNeRJB2CXvOBJ3k/8BJw4ybbLAKLALt27eqzuyOOcyZL2szEZ+BJrgHeBvxCVdVG21XVnqpaqKqFubm5SXcnSVpjojPwJJcC7wV+sqr+a9hIkqRxjHMZ4U3Al4GzkxxIci3wYeB4YG+S+5J8ZItzSpLWGHkGXlVXrbP6o1uQRZJ0CLwTU5IaZYFLUqMscElqlAUuSY2ywCWpURa4JDXKApekRlngktQoC1ySGmWBS1KjLHBJalSv+cCnaVZzYx+JZnmsnYtcGp9n4JLUKAtckhplgUtSoyxwSWqUBS5JjbLAJalRFrgkNcoCl6RGWeCS1CgLXJIaZYFLUqNGFniS65McTPLgqnUnJdmb5NHu84lbG1OStNY4Z+AfAy5ds243cHtVnQXc3j2WJE3RyAKvqruA59asvgK4oVu+AXj7wLkkSSNMOgZ+clU93S0/A5w8UB5J0ph6zwdeVZWkNno+ySKwCLBr166+u9Nh7kic99050DWpSc/An01yCkD3+eBGG1bVnqpaqKqFubm5CXcnSVpr0gK/Bbi6W74a+MwwcSRJ4xrnMsKbgC8DZyc5kORa4IPATyV5FLikeyxJmqKRY+BVddUGT108cBZJ0iHwTkxJapQFLkmNssAlqVEWuCQ1ygKXpEZZ4JLUKAtckhplgUtSoyxwSWqUBS5Jjeo9naykfmY1ha7T2LbPM3BJapQFLkmNssAlqVEWuCQ1ygKXpEZZ4JLUKAtckhplgUtSoyxwSWqUBS5JjbLAJalRFrgkNapXgSf57SQPJXkwyU1JjhkqmCRpcxMXeJJTgd8EFqrqDcBRwJVDBZMkba7vEMoO4AeS7ACOBf6tfyRJ0jgmng+8qp5K8kfAN4D/Bm6rqtvWbpdkEVgE2LVr16S7k6TeZjX3OmzN/Ot9hlBOBK4AzgBeBxyX5J1rt6uqPVW1UFULc3NzkyeVJP0/fYZQLgH+paqWq+q7wKeAC4aJJUkapU+BfwM4P8mxSQJcDOwfJpYkaZSJC7yq7gFuBu4FHuhea89AuSRJI/T6o8ZV9QHgAwNlkSQdAu/ElKRGWeCS1CgLXJIaZYFLUqMscElqlAUuSY2ywCWpURa4JDXKApekRlngktQoC1ySGmWBS1KjLHBJapQFLkmNssAlqVEWuCQ1ygKXpEZZ4JLUKAtckhplgUtSoyxwSWqUBS5JjbLAJalRvQo8yQlJbk7ySJL9Sd40VDBJ0uZ29Pz6PwX+oarekeRo4NgBMkmSxjBxgSd5NfBm4BqAqnoReHGYWJKkUfoMoZwBLAN/leQrSa5LctzajZIsJllKsrS8vNxjd5Kk1foU+A7gPOAvq+pc4D+B3Ws3qqo9VbVQVQtzc3M9didJWq1PgR8ADlTVPd3jm1kpdEnSFExc4FX1DPDNJGd3qy4GHh4klSRppL5XofwGcGN3BcoTwC/3jyRJGkevAq+q+4CFgbJIkg6Bd2JKUqMscElqlAUuSY2ywCWpURa4JDXKApekRlngktQoC1ySGmWBS1KjLHBJalTfuVAkNWp+9+dmHUE9eQYuSY2ywCWpURa4JDXKApekRlngktQoC1ySGmWBS1KjLHBJapQFLkmNssAlqVEWuCQ1ygKXpEb1LvAkRyX5SpLPDhFIkjSeIc7A3wPsH+B1JEmHoFeBJzkNeCtw3TBxJEnj6nsG/ifAe4HvbbRBksUkS0mWlpeXe+5OkvSyiQs8yduAg1W1b7PtqmpPVS1U1cLc3Nyku5MkrdHnDPxC4PIkTwIfBy5K8jeDpJIkjTRxgVfV+6rqtKqaB64E7qiqdw6WTJK0Ka8Dl6RGDfJHjavqTuDOIV5LkjQez8AlqVEWuCQ1ygKXpEZZ4JLUKAtckhplgUtSoyxwSWqUBS5JjbLAJalRFrgkNcoCl6RGWeCS1CgLXJIaZYFLUqMscElqlAUuSY2ywCWpURa4JDXKApekRlngktQoC1ySGmWBS1KjLHBJatTEBZ7k9CRfSvJwkoeSvGfIYJKkze3o8bUvAb9TVfcmOR7Yl2RvVT08UDZJ0iYmPgOvqqer6t5u+T+A/cCpQwWTJG1ukDHwJPPAucA96zy3mGQpydLy8vIQu5MkMUCBJ/lB4G+B36qqb699vqr2VNVCVS3Mzc313Z0kqdOrwJN8PyvlfWNVfWqYSJKkcfS5CiXAR4H9VfXHw0WSJI2jzxn4hcAvAhclua/7uGygXJKkESa+jLCq/hHIgFkkSYfAOzElqVEWuCQ1ygKXpEZZ4JLUKAtckhplgUtSoyxwSWqUBS5JjbLAJalRFrgkNcoCl6RGWeCS1CgLXJIaZYFLUqMscElqlAUuSY2ywCWpURa4JDXKApekRlngktQoC1ySGmWBS1KjLHBJalSvAk9yaZKvJXksye6hQkmSRpu4wJMcBfw58DPAOcBVSc4ZKpgkaXN9zsDfCDxWVU9U1YvAx4ErhoklSRplR4+vPRX45qrHB4AfX7tRkkVgsXv4QpKv9djnEHYC35pxhkPRWl4w87SYeesNljcf6vXlr19vZZ8CH0tV7QH2bPV+xpVkqaoWZp1jXK3lBTNPi5m33nbP22cI5Sng9FWPT+vWSZKmoE+B/zNwVpIzkhwNXAncMkwsSdIoEw+hVNVLSd4NfAE4Cri+qh4aLNnW2TbDOWNqLS+YeVrMvPW2dd5U1awzSJIm4J2YktQoC1ySGnVYF3iSn0/yUJLvJdnwUqAkTyZ5IMl9SZammXGdLONm3jbTGCQ5KcneJI92n0/cYLv/6Y7xfUlm8ob3qOOW5FVJPtE9f0+S+emnfEWmUZmvSbK86tj+6ixyrspzfZKDSR7c4Pkk+bPu+/lqkvOmnXGdTKMyvyXJ86uO8e9OO+O6quqw/QB+BDgbuBNY2GS7J4Gds847bmZW3jR+HDgTOBq4Hzhnhpn/ANjdLe8GPrTBdi/M+NiOPG7ArwMf6ZavBD7RQOZrgA/PMueaPG8GzgMe3OD5y4BbgQDnA/c0kPktwGdnnXPtx2F9Bl5V+6tq1nd+HpIxM2+3aQyuAG7olm8A3j7DLJsZ57it/l5uBi5OkilmXGu7/VuPVFV3Ac9tsskVwF/XiruBE5KcMp106xsj87Z0WBf4ISjgtiT7ulv/t7v1pjE4dUZZAE6uqqe75WeAkzfY7pgkS0nuTjKLkh/nuP3fNlX1EvA88JqppFvfuP/WP9cNR9yc5PR1nt9OttvP77jelOT+JLcm+dFZh4Ep3Eq/1ZJ8EXjtOk+9v6o+M+bL/ERVPZXkh4C9SR7p/kfeEgNlnqrNMq9+UFWVZKNrU1/fHeczgTuSPFBVjw+d9Qj098BNVfWdJL/Gym8QF8040+HmXlZ+fl9Ichnwd8BZM87UfoFX1SUDvMZT3eeDST7Nyq+tW1bgA2Se+jQGm2VO8mySU6rq6e5X4YMbvMbLx/mJJHcC57Iyvjst4xy3l7c5kGQH8Grg36cTb10jM1fV6nzXsfKexHbW3DQcVfXtVcufT/IXSXZW1Uwn5jrih1CSHJfk+JeXgZ8G1n0nehvZbtMY3AJc3S1fDbzit4gkJyZ5Vbe8E7gQeHhqCVeMc9xWfy/vAO6o7l2sGRmZec348eXA/inmm8QtwC91V6OcDzy/aghuW0ry2pffC0nyRla6c5b/sa+Y9buoW/kB/Cwr42vfAZ4FvtCtfx3w+W75TFbe2b8feIiVYYxtnbl7fBnwdVbOYGed+TXA7cCjwBeBk7r1C8B13fIFwAPdcX4AuHZGWV9x3IDfAy7vlo8BPgk8BvwTcOYsj+2YmX+/+9m9H/gS8MMzznsT8DTw3e5n+VrgXcC7uufDyh+Debz7WdjwCrFtlPndq47x3cAFs85cVd5KL0mtOuKHUCSpVRa4JDXKApekRlngktQoC1ySGmWBS1KjLHBJatT/AvE3IZ5jNprdAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "55XrpZ1CodnE", "colab": { "base_uri": "https://localhost:8080/", "height": 265 }, "outputId": "11d4bbf7-c678-4516-d30e-57092021dcd0" }, "source": [ "input_x = np.array([[1]])\n", "dense1 = tf.keras.layers.Dense(100, activation=\"relu\", kernel_initializer=\"lecun_normal\", input_shape=(input_x.shape[0],))\n", "y = dense1(input_x)\n", "plt.hist(dense1.weights[0][0]);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAQAklEQVR4nO3df6xfdX3H8edrgPsDyQC5Q4TWmo2QVCNobopGZ1AQSyWixm00i8OJuWowkcTE4Exk0X9YjJpMjKyTBlxYZZuiJBShYyZIgsiFFCwUhBEMrZUW6wCniam+90dPk6+X77f32+/59t7y8flIvrnnfD6f7/m8eyKvezz3/EhVIUlq1x8tdwGSpMPLoJekxhn0ktQ4g16SGmfQS1Ljjl7uAoY56aSTatWqVctdhiS9aNx3333PVNXMsL4jMuhXrVrF/Pz8cpchSS8aSX4yqs9TN5LUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxiwZ9khVJvpfk4SQPJfl4135iki1JHut+njDi+5d0Yx5Lcsm0/wGSpIMb54h+H/CJqloNvAG4LMlq4Argjqo6HbijW/89SU4ErgTOBtYAV476hSBJOjwWDfqq2lVV93fLzwPbgVOBi4Dru2HXA+8e8vV3AFuqam9V/QLYAqydRuGSpPEc0p2xSVYBrwPuAU6uql1d18+Ak4d85VTgqYH1HV3bsG3PAXMAK1euPJSypCWz6opblmXeJ69657LMqzaM/cfYJC8FvglcXlXPDfbV/tdU9XpVVVVtqKrZqpqdmRn6uAZJ0gTGCvokx7A/5G+oqm91zU8nOaXrPwXYPeSrO4EVA+undW2SpCUyzlU3Aa4FtlfVFwe6bgYOXEVzCfCdIV+/DTg/yQndH2HP79okSUtknCP6NwHvB96WZGv3WQdcBbw9yWPAed06SWaTfA2gqvYCnwPu7T6f7dokSUtk0T/GVtVdQEZ0nztk/DzwoYH1jcDGSQuUJPXjnbGS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYt+uKRJBuBC4HdVfWaru1G4IxuyPHA/1bVWUO++yTwPPBbYF9VzU6pbknSmBYNeuA64Grg6wcaquqvDywn+QLw7EG+/9aqembSAiVJ/YzzKsE7k6wa1te9OPyvgLdNtyxJ0rT0PUf/F8DTVfXYiP4Cbk9yX5K5nnNJkiYwzqmbg1kPbDpI/5urameSPwW2JHmkqu4cNrD7RTAHsHLlyp5lSZIOmPiIPsnRwHuBG0eNqaqd3c/dwE3AmoOM3VBVs1U1OzMzM2lZkqQF+py6OQ94pKp2DOtMcmyS4w4sA+cD23rMJ0mawKJBn2QTcDdwRpIdSS7tui5mwWmbJK9IsrlbPRm4K8kDwA+BW6rqu9MrXZI0jnGuulk/ov0DQ9p+Cqzrlp8AzuxZnySpJ++MlaTGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMaN8yrBjUl2J9k20PYPSXYm2dp91o347tokjyZ5PMkV0yxckjSecY7orwPWDmn/UlWd1X02L+xMchTwFeACYDWwPsnqPsVKkg7dokFfVXcCeyfY9hrg8ap6oqp+A3wDuGiC7UiSeuhzjv5jSR7sTu2cMKT/VOCpgfUdXdtQSeaSzCeZ37NnT4+yJEmDJg36rwJ/BpwF7AK+0LeQqtpQVbNVNTszM9N3c5KkzkRBX1VPV9Vvq+p3wL+w/zTNQjuBFQPrp3VtkqQlNFHQJzllYPU9wLYhw+4FTk/yqiQvAS4Gbp5kPknS5I5ebECSTcA5wElJdgBXAuckOQso4Engw93YVwBfq6p1VbUvyceA24CjgI1V9dBh+VdIkkZaNOirav2Q5mtHjP0psG5gfTPwgksvJUlLxztjJalxBr0kNc6gl6TGGfSS1DiDXpIat+hVN9KRaNUVtyx3CdKLhkf0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4xYN+iQbk+xOsm2g7fNJHknyYJKbkhw/4rtPJvlRkq1J5qdZuCRpPOMc0V8HrF3QtgV4TVW9Fvgx8KmDfP+tVXVWVc1OVqIkqY9Fg76q7gT2Lmi7var2das/AE47DLVJkqZgGufoPwjcOqKvgNuT3Jdk7mAbSTKXZD7J/J49e6ZQliQJegZ9kk8D+4AbRgx5c1W9HrgAuCzJW0Ztq6o2VNVsVc3OzMz0KUuSNGDioE/yAeBC4G+qqoaNqaqd3c/dwE3AmknnkyRNZqKgT7IW+CTwrqr61YgxxyY57sAycD6wbdhYSdLhM87llZuAu4EzkuxIcilwNXAcsKW7dPKabuwrkmzuvnoycFeSB4AfArdU1XcPy79CkjTSou+Mrar1Q5qvHTH2p8C6bvkJ4Mxe1UmSevPl4JqYL+heOsu5r5+86p3LNremw0cgSFLjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS47wztgHeoSrpYDyil6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0bK+iTbEyyO8m2gbYTk2xJ8lj384QR372kG/NYkkumVbgkaTzjHtFfB6xd0HYFcEdVnQ7c0a3/niQnAlcCZwNrgCtH/UKQJB0eYwV9Vd0J7F3QfBFwfbd8PfDuIV99B7ClqvZW1S+ALbzwF4Yk6TDqc2fsyVW1q1v+GXDykDGnAk8NrO/o2l4gyRwwB7By5coeZUmapuW689p31U7PVP4YW1UFVM9tbKiq2aqanZmZmUZZkiT6Bf3TSU4B6H7uHjJmJ7BiYP20rk2StET6BP3NwIGraC4BvjNkzG3A+UlO6P4Ie37XJklaIuNeXrkJuBs4I8mOJJcCVwFvT/IYcF63TpLZJF8DqKq9wOeAe7vPZ7s2SdISGeuPsVW1fkTXuUPGzgMfGljfCGycqDpJUm/eGStJjTPoJalxBr0kNc6gl6TGGfSS1DhfDj4lvqBb0pHKI3pJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxEwd9kjOSbB34PJfk8gVjzkny7MCYz/QvWZJ0KCZ+qFlVPQqcBZDkKGAncNOQod+vqgsnnUeS1M+0Tt2cC/xPVf1kStuTJE3JtIL+YmDTiL43Jnkgya1JXj1qA0nmkswnmd+zZ8+UypIk9Q76JC8B3gX8x5Du+4FXVtWZwJeBb4/aTlVtqKrZqpqdmZnpW5YkqTONI/oLgPur6umFHVX1XFX9slveDByT5KQpzClJGtM0gn49I07bJHl5knTLa7r5fj6FOSVJY+r1KsEkxwJvBz480PYRgKq6Bngf8NEk+4BfAxdXVfWZU5J0aHoFfVX9H/CyBW3XDCxfDVzdZw5JUj/eGStJjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mN6x30SZ5M8qMkW5PMD+lPkn9K8niSB5O8vu+ckqTx9XqV4IC3VtUzI/ouAE7vPmcDX+1+SpKWwFKcurkI+Hrt9wPg+CSnLMG8kiSmc0RfwO1JCvjnqtqwoP9U4KmB9R1d267BQUnmgDmAlStXTlzMqitumfi7ktSiaRzRv7mqXs/+UzSXJXnLJBupqg1VNVtVszMzM1MoS5IEUwj6qtrZ/dwN3ASsWTBkJ7BiYP20rk2StAR6BX2SY5Mcd2AZOB/YtmDYzcDfdlffvAF4tqp2IUlaEn3P0Z8M3JTkwLb+raq+m+QjAFV1DbAZWAc8DvwK+Luec0qSDkGvoK+qJ4Azh7RfM7BcwGV95pEkTc47YyWpcQa9JDXOoJekxhn0ktS4aT3rRpKasVx32D951TsPy3Y9opekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOB+BIOmItFyPIWiRR/SS1LiJgz7JiiTfS/JwkoeSfHzImHOSPJtka/f5TL9yJUmHqs+pm33AJ6rq/u4F4fcl2VJVDy8Y9/2qurDHPJKkHiY+oq+qXVV1f7f8PLAdOHVahUmSpmMq5+iTrAJeB9wzpPuNSR5IcmuSVx9kG3NJ5pPM79mzZxplSZKYQtAneSnwTeDyqnpuQff9wCur6kzgy8C3R22nqjZU1WxVzc7MzPQtS5LU6RX0SY5hf8jfUFXfWthfVc9V1S+75c3AMUlO6jOnJOnQ9LnqJsC1wPaq+uKIMS/vxpFkTTffzyedU5J06PpcdfMm4P3Aj5Js7dr+HlgJUFXXAO8DPppkH/Br4OKqqh5zSpIO0cRBX1V3AVlkzNXA1ZPOIUnqzztjJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXF9Xw6+NsmjSR5PcsWQ/j9OcmPXf0+SVX3mkyQduj4vBz8K+ApwAbAaWJ9k9YJhlwK/qKo/B74E/OOk80mSJtPniH4N8HhVPVFVvwG+AVy0YMxFwPXd8n8C5yY56HtmJUnTNfHLwYFTgacG1ncAZ48aU1X7kjwLvAx4ZuHGkswBc93qL5M82qO2k4bNoRdwPy3OfTQe99N4Drqf0u+cxytHdfQJ+qmqqg3AhmlsK8l8Vc1OY1stcz8tzn00HvfTeJZrP/U5dbMTWDGwflrXNnRMkqOBPwF+3mNOSdIh6hP09wKnJ3lVkpcAFwM3LxhzM3BJt/w+4L+rqnrMKUk6RBOfuunOuX8MuA04CthYVQ8l+SwwX1U3A9cC/5rkcWAv+38ZLIWpnAL6A+B+Wpz7aDzup/Esy36KB9iS1DbvjJWkxhn0ktS4ZoM+yeeTPJLkwSQ3JTl+uWs60iT5yyQPJfldEi+NW2CxR3wIkmxMsjvJtuWu5UiWZEWS7yV5uPtv7uNLOX+zQQ9sAV5TVa8Ffgx8apnrORJtA94L3LnchRxpxnzEh+A6YO1yF/EisA/4RFWtBt4AXLaU/3tqNuir6vaq2tet/oD91/lrQFVtr6o+dyC3bJxHfPzBq6o72X9FnQ6iqnZV1f3d8vPAdvY/OWBJNBv0C3wQuHW5i9CLyrBHfCzZf5hqV/cU39cB9yzVnEfMIxAmkeS/gJcP6fp0VX2nG/Np9v/fphuWsrYjxTj7SNLSSPJS4JvA5VX13FLN+6IO+qo672D9ST4AXAic+4d6R+5i+0gjjfOID2lsSY5hf8jfUFXfWsq5mz11k2Qt8EngXVX1q+WuRy864zziQxpL93j2a4HtVfXFpZ6/2aAHrgaOA7Yk2ZrkmuUu6EiT5D1JdgBvBG5Jctty13Sk6P6Qf+ARH9uBf6+qh5a3qiNPkk3A3cAZSXYkuXS5azpCvQl4P/C2Lo+2Jlm3VJP7CARJalzLR/SSJAx6SWqeQS9JjTPoJalxBr0kNc6gl6TGGfSS1Lj/B+5bTkJ0OqWEAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "flgRy7dMpGae", "colab": { "base_uri": "https://localhost:8080/", "height": 265 }, "outputId": "fc4c6597-f894-44da-b938-e1ba19b354aa" }, "source": [ "input_x = np.array([[1]])\n", "dense1 = tf.keras.layers.Dense(100, activation=\"relu\", kernel_initializer=\"random_normal\", input_shape=(input_x.shape[0],))\n", "y = dense1(input_x)\n", "plt.hist(dense1.weights[0][0]);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAOc0lEQVR4nO3df4xlZ13H8ffHLgWLDW3dS0HKOFsDTQqBoKOiBASKZaHAktg/2gApP5JJMCIalCw2SmJiUpCoJBqbDSwtEQtYKjQ0AkuhVhMs7JYCLaV0KRW2FnZL5YeI1Iavf8xpMr3Ozr1zz5m5++y+X8nNnPucc+/zfebc+eTMuedHqgpJUnt+at4FSJJmY4BLUqMMcElqlAEuSY0ywCWpUdu2srPt27fX4uLiVnYpSc07cODAfVU1Gm/f0gBfXFxk//79W9mlJDUvyb+v1e4uFElqlAEuSY0ywCWpUQa4JDXKAJekRhngktQoA1ySGmWAS1KjDHBJatSWnokpHasWd183t77vvuyCufWttrkFLkmNMsAlqVEGuCQ1ygCXpEYZ4JLUKANckhplgEtSowxwSWrUxABPsjfJ4SS3jrW/IclXktyW5O2bV6IkaS3TbIFfAexc3ZDkecAu4OlV9RTgHcOXJklaz8QAr6obgfvHml8PXFZVP+6WObwJtUmS1jHrtVCeDDw7yZ8B/wP8QVV9bq0FkywDywALCwszdqcTxTyvSTIv8xqz12Bp36xfYm4DzgCeCfwh8MEkWWvBqtpTVUtVtTQajWbsTpI0btYAPwRcUys+C/wE2D5cWZKkSWYN8A8DzwNI8mTgZOC+oYqSJE02cR94kquA5wLbkxwC3grsBfZ2hxY+AFxSVbWZhUqSHm5igFfVxUeZ9cqBa5EkbYBnYkpSowxwSWqUAS5JjTLAJalRBrgkNcoAl6RGGeCS1CgDXJIaZYBLUqMMcElqlAEuSY0ywCWpUQa4JDXKAJekRhngktSoiQGeZG+Sw93NG8bnvSlJJfF2apK0xabZAr8C2DnemOSJwPnANwauSZI0hYkBXlU3AvevMesvgTcD3kpNkuZg4i3V1pJkF3BPVX0hyaRll4FlgIWFhVm6k3ScWdx93Vz6vfuyC+bS72bZ8JeYSU4B/gj4k2mWr6o9VbVUVUuj0Wij3UmSjmKWo1B+AdgBfCHJ3cBZwM1JHjdkYZKk9W14F0pVfQl47EPPuxBfqqr7BqxLkjTBNIcRXgV8BjgnyaEkr9v8siRJk0zcAq+qiyfMXxysGknS1DwTU5IaZYBLUqMMcElqlAEuSY0ywCWpUQa4JDXKAJekRhngktQoA1ySGmWAS1KjDHBJapQBLkmNMsAlqVEGuCQ1ygCXpEZNc0OHvUkOJ7l1VdufJ/lKki8m+cckp21umZKkcdNsgV8B7Bxr2wc8taqeBnwVeMvAdUmSJpgY4FV1I3D/WNsnqurB7um/sXJjY0nSFhpiH/hrgX8a4H0kSRvQK8CTXAo8CLxvnWWWk+xPsv/IkSN9upMkrTJzgCd5NfAS4BVVVUdbrqr2VNVSVS2NRqNZu5MkjZl4V/q1JNkJvBn4jar672FLkiRNY5rDCK8CPgOck+RQktcBfw2cCuxLckuSyze5TknSmIlb4FV18RrN796EWiRJG+CZmJLUKANckhplgEtSowxwSWqUAS5JjTLAJalRBrgkNWqmMzF1fFvcfd28S5A0BbfAJalRBrgkNcoAl6RGGeCS1CgDXJIaZYBLUqMMcElqlAEuSY2a5o48e5McTnLrqrYzkuxLcmf38/TNLVOSNG6aLfArgJ1jbbuB66vqScD13XNJ0haaGOBVdSNw/1jzLuDKbvpK4OUD1yVJmmDWa6GcWVX3dtPfAs482oJJloFlgIWFhRm7kzQ0r3nTvt5fYlZVAbXO/D1VtVRVS6PRqG93kqTOrAH+7SSPB+h+Hh6uJEnSNGYN8GuBS7rpS4CPDFOOJGla0xxGeBXwGeCcJIeSvA64DPjNJHcCL+ieS5K20MQvMavq4qPMOm/gWiRJG+CZmJLUKANckhplgEtSowxwSWqUAS5JjTLAJalRBrgkNcoAl6RGGeCS1CgDXJIaZYBLUqMMcElqlAEuSY0ywCWpUQa4JDWqV4An+f0ktyW5NclVSR41VGGSpPXNHOBJngD8LrBUVU8FTgIuGqowSdL6+u5C2Qb8dJJtwCnAf/QvSZI0jZkDvKruAd4BfAO4F/heVX1ifLkky0n2J9l/5MiR2SuVJD1Mn10opwO7gB3AzwGPTvLK8eWqak9VLVXV0mg0mr1SSdLD9NmF8gLg61V1pKr+F7gG+PVhypIkTdInwL8BPDPJKUnCyl3qbx+mLEnSJH32gd8EXA3cDHype689A9UlSZpgW58XV9VbgbcOVIskaQM8E1OSGmWAS1KjDHBJapQBLkmNMsAlqVEGuCQ1ygCXpEYZ4JLUKANckhplgEtSowxwSWqUAS5JjTLAJalRBrgkNcoAl6RGGeCS1KheAZ7ktCRXJ/lKktuT/NpQhUmS1tfrjjzAO4GPVdWFSU4GThmgJknSFGYO8CSPAZ4DvBqgqh4AHhimLEnSJH22wHcAR4D3JHk6cAB4Y1X9cPVCSZaBZYCFhYUe3Z14FndfN+8SJB3D+uwD3wb8IvC3VfUM4IfA7vGFqmpPVS1V1dJoNOrRnSRptT4Bfgg4VFU3dc+vZiXQJUlbYOYAr6pvAd9Mck7XdB7w5UGqkiRN1PcolDcA7+uOQLkLeE3/kiRJ0+gV4FV1C7A0UC2SpA3wTExJapQBLkmNMsAlqVEGuCQ1ygCXpEYZ4JLUqL7HgUtSM+Z5faG7L7tg8Pd0C1ySGmWAS1KjDHBJapQBLkmNMsAlqVEGuCQ1ygCXpEYZ4JLUqN4BnuSkJJ9P8tEhCpIkTWeILfA3ArcP8D6SpA3oFeBJzgIuAN41TDmSpGn1vRbKXwFvBk492gJJloFlgIWFhZ7dzcc8r58gSUcz8xZ4kpcAh6vqwHrLVdWeqlqqqqXRaDRrd5KkMX12oTwLeFmSu4H3A89P8neDVCVJmmjmAK+qt1TVWVW1CFwEfKqqXjlYZZKkdXkcuCQ1apAbOlTVDcANQ7yXJGk6boFLUqMMcElqlAEuSY0ywCWpUQa4JDXKAJekRhngktQoA1ySGmWAS1KjDHBJapQBLkmNMsAlqVEGuCQ1ygCXpEYZ4JLUqD73xHxikk8n+XKS25K8ccjCJEnr63NDhweBN1XVzUlOBQ4k2VdVXx6oNknSOvrcE/Peqrq5m/4BcDvwhKEKkyStb5B94EkWgWcAN60xbznJ/iT7jxw5MkR3kiQGCPAkPwN8CPi9qvr++Pyq2lNVS1W1NBqN+nYnSer0CvAkj2AlvN9XVdcMU5IkaRp9jkIJ8G7g9qr6i+FKkiRNo88W+LOAVwHPT3JL93jxQHVJkiaY+TDCqvpXIAPWIknaAM/ElKRGGeCS1CgDXJIaZYBLUqMMcElqlAEuSY0ywCWpUX0uJ7ulFndfN+8SJOmY4ha4JDXKAJekRhngktQoA1ySGmWAS1KjDHBJapQBLkmNMsAlqVF974m5M8kdSQ4m2T1UUZKkyfrcE/Mk4G+AFwHnAhcnOXeowiRJ6+uzBf4rwMGququqHgDeD+wapixJ0iR9roXyBOCbq54fAn51fKEky8By9/S/ktzRo89WbQfum3cRc3CijhtO3LGfqOOGCWPP23q998+v1bjpF7Oqqj3Ans3u51iWZH9VLc27jq12oo4bTtyxn6jjhvmMvc8ulHuAJ656flbXJknaAn0C/HPAk5LsSHIycBFw7TBlSZImmXkXSlU9mOR3gI8DJwF7q+q2wSo7vpyou5BO1HHDiTv2E3XcMIexp6q2uk9J0gA8E1OSGmWAS1KjDPCBJDkjyb4kd3Y/Tz/Kch9L8t0kHx1r35Hkpu6yBB/ovhg+5m1g3Jd0y9yZ5JJV7Td0l2O4pXs8duuqn82kS0gkeWS3Dg9263Rx1by3dO13JHnhVtbd16zjTrKY5Eer1vHlW117X1OM/TlJbk7yYJILx+at+dkfRFX5GOABvB3Y3U3vBt52lOXOA14KfHSs/YPARd305cDr5z2mocYNnAHc1f08vZs+vZt3A7A073FsYLwnAV8DzgZOBr4AnDu2zG8Dl3fTFwEf6KbP7ZZ/JLCje5+T5j2mLRj3InDrvMewyWNfBJ4GvBe4cFX7UT/7QzzcAh/OLuDKbvpK4OVrLVRV1wM/WN2WJMDzgasnvf4YNM24Xwjsq6r7q+o/gX3Azi2qb2jTXEJi9e/kauC8bh3vAt5fVT+uqq8DB7v3a0Gfcbdu4tir6u6q+iLwk7HXbupn3wAfzplVdW83/S3gzA289meB71bVg93zQ6xcqqAF04x7rcsurB7fe7p/rf+4gT/4SWN52DLdOv0eK+t4mtceq/qMG2BHks8n+eckz97sYgfWZ71t6jrf9FPpjydJPgk8bo1Zl65+UlWV5Lg5PnOTx/2KqronyanAh4BXsfJvqI4f9wILVfWdJL8EfDjJU6rq+/MurHUG+AZU1QuONi/Jt5M8vqruTfJ44PAG3vo7wGlJtnVbLsfUZQkGGPc9wHNXPT+LlX3fVNU93c8fJPl7Vv5dPZYDfJpLSDy0zKEk24DHsLKOW778xMzjrpWdwT8GqKoDSb4GPBnYv+lVD6PPejvqZ38I7kIZzrXAQ98wXwJ8ZNoXdh/wTwMPfXu9odfP2TTj/jhwfpLTu6NUzgc+nmRbku0ASR4BvAS4dQtq7mOaS0is/p1cCHyqW8fXAhd1R2vsAJ4EfHaL6u5r5nEnGXX3DyDJ2ayM+64tqnsIfS4bsuZnf7DK5v0N7/HyYGVf3/XAncAngTO69iXgXauW+xfgCPAjVvaHvbBrP5uVP+aDwD8Aj5z3mAYe92u7sR0EXtO1PRo4AHwRuA14Jw0clQG8GPgqK0cmXNq1/Snwsm76Ud06PNit07NXvfbS7nV3AC+a91i2YtzAb3Xr9xbgZuCl8x7LJoz9l7u/5x+y8t/Wbate+/8++0M9PJVekhrlLhRJapQBLkmNMsAlqVEGuCQ1ygCXpEYZ4JLUKANckhr1f5qegbAMqLT8AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "markdown", "metadata": { "id": "JNLaa7Ju3C-j" }, "source": [ "# 4. Gradient Descent & Custom Training Loops" ] }, { "cell_type": "markdown", "metadata": { "id": "1qH1Cl_a3eH_" }, "source": [ "## 4.1: Implement Gradient Descent manually \n", "\n", "Find the value of x that minimizes the following function f(x).\n", "\n" ] }, { "cell_type": "code", "metadata": { "id": "th6T9ySdr6S3" }, "source": [ "def f(x):\n", " return 5. * x ** 2 + 3. * x + 1." ], "execution_count": 45, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "wd_XcD31szPV", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "7f0648a0-1c04-4ae8-eeda-2f28fa09d959" }, "source": [ "f(1)" ], "execution_count": 46, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "9.0" ] }, "metadata": { "tags": [] }, "execution_count": 46 } ] }, { "cell_type": "code", "metadata": { "id": "t3uyAKm8s0Ko" }, "source": [ "def approximate_diff(f, x, eps=1e-5):\n", " return (f(x + eps) - f(x - eps)) / (2. * eps)" ], "execution_count": 47, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "HQqBpTMJs4JX", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "f57a1ac9-0fac-4a1c-edc1-a29a391daeac" }, "source": [ "approximate_diff(f, 1) # true derivative = 13" ], "execution_count": 50, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "13.000000000040755" ] }, "metadata": { "tags": [] }, "execution_count": 50 } ] }, { "cell_type": "markdown", "metadata": { "id": "btpRqeKX3j-L" }, "source": [ "# 4.2: Visualize function space" ] }, { "cell_type": "code", "metadata": { "id": "pEkwnvd3s601", "colab": { "base_uri": "https://localhost:8080/", "height": 287 }, "outputId": "ab6b1350-97e6-454c-9a81-57cdf37d47c9" }, "source": [ "xs = np.linspace(-2, 2, 200)\n", "fs = f(xs)\n", "x0 = 0.25\n", "df_x0 = approximate_diff(f, x0)\n", "tangent_x0 = df_x0 * (xs - x0) + f(x0)\n", "plt.plot([-2, 2], [0, 0], \"k-\", linewidth=1)\n", "plt.plot([0, 0], [-5, 15], \"k-\", linewidth=1)\n", "plt.plot(xs, fs)\n", "plt.plot(xs, tangent_x0, \"r--\")\n", "plt.plot(x0, f(x0), \"ro\")\n", "plt.grid(True)\n", "plt.xlabel(\"x\", fontsize=14)\n", "plt.ylabel(\"f(x)\", fontsize=14, rotation=0)\n", "plt.axis([-2, 2, -5, 15]);" ], "execution_count": 49, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEOCAYAAACuOOGFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3gU9fbH8fdJBxJ6qAkEhIAgvdkVFERAsSv2iti9FtTr9aq32Mv9ISiCotjFdkVEETFI79J7L6HXhPTs+f0xyzXGhIRks7ObPa/n2Se7s7Ozn0ySPZk5M98RVcUYY4wprzC3AxhjjKkcrKAYY4zxCSsoxhhjfMIKijHGGJ+wgmKMMcYnrKAYY4zxCb8WFBEZIyJ7RGR5gWnPiMgOEVnsvfUr5rV9RWSNiKwXkcf9l9oYY0xpiD/PQxGRs4F04ANVPcU77RkgXVVfOc7rwoG1QG9gOzAfGKSqKys8tDHGmFLx6xaKqk4DDpThpd2B9aq6UVVzgM+AgT4NZ4wxplwi3A7gda+I3AgsAB5W1YOFnm8MbCvweDvQo6gFichgYDBAZEy1Ls2aNK6AuL7j8XgICwv8Vpbl9K21a9eSnJzsdowSBcv6dDvnngwlK19JjA1DpPj53M5ZGmvXrt2nqvFlerGq+vUGJAHLCzyuD4TjbC39GxhTxGuuAN4p8PgGYHhJ7xXToIUeyczRQJaSkuJ2hFKxnL7l/OkFvmBZn27m3Lg3XZMen6Av/rCqxHmDYX0CC7SMn++ul0pV3a2q+arqAUbj7N4qbAeQWOBxgnfacXmAT+Zu9UlOY4wpyqhpG4kMD+PmM5LcjuI61wuKiDQs8PBSYHkRs80HWopIMxGJAq4Bxpe07JhwGDNzE9l5+b4Ja4wxBexJy+KrRdu5vHMC9eJi3I7jOn8fNvwpMBtoJSLbReQ24CURWSYiS4GewF+88zYSkYkAqpoH3AtMAlYB41R1RUnvVyNa2H0km29/S62g78gYE8rGztpMbr6HO85q5naUgODXpryqDipi8rvFzJsK9CvweCIw8UTer0qEcFLD6oyctoEruiQQFnacbpkxxpyA9Ow8Ppy9hb5tG9A8PtbtOAHB9V1eFe3Oc5qzce9RJq/a7XYUY0wl8tm8rRzJymPw2c3djhIwKn1B6d+uIYm1qzDy1w3HjhAzxphyycnz8O6MTfRoVptOTWq5HSdgVPqCEhEexh1nNee3rYeYv7nw6S3GGHPivluSys7DWQw55yS3owSUSl9QAK7skkjtalGM/HWD21GMMUFOVXl72gZa1Y/j3FZlO/+vsgqJglIlKpybTkvil9V7WLMrze04xpgglrJmD2t3p3PnOc2R450WH4JCoqAA3HhaU6pEhvP2NNtKMcaU3cipG2lUI4aLOjRyO0rACZmCUqtaFNd0T2T84lR2HMp0O44xJgjN33yAeZsPcNtZzYkMD5mPz1ILqTVy25nNUODd6ZvcjmKMCUIjUtZTu1oUg7onljxzCAqpgpJQqyoXd2jEZ/O3cvBojttxjDFBZPmOw0xds5fbzmxG1ahAGag9sIRUQQEYcs5JZOTk896szW5HMcYEkREp64mLjuCG05q6HSVghVxBadUgjgva1uf9mZs4kpXrdhxjTBBYvyeNH1fs4sbTm1I9JtLtOAEr5AoKwL09W3IkyxmHxxhjSvLm1A3ERIRz6xk2COTxhGRBaZdQg3NbxfPujE1k5OS5HccYE8C2Hcjg28WpDOrehDqx0W7HCWghWVAA7uvVggNHc+wCXMaY43p72gbCBBsEshRCtqB0aVqb05rXYdS0jWTl2gW4jDF/tudIFuMWbOeKLgk0qGEX0CpJyBYUcLZS9qRl88XC7W5HMcYEoNHTN5KX77FBIEsppAvKaSfVoXOTmoycuoHcfI/bcYwxAWR/ejYfz93KxR0a0bRONbfjBIWQLigiwn29WrLjUCbf/LbD7TjGmAAyevomMnPzubdXC7ejBA2/FRQRGSMie0RkeYFpL4vIahFZKiLfiEjNYl672Xvd+cUissCXuc5tFU/bRtV5M2U9ebaVYowBDhzN4YPZm7mofSNa1ItzO07Q8OcWyvtA30LTJgOnqGp7YC3wxHFe31NVO6pqV1+GcrZSWrB5fwbfLU315aKNMUFq9PSNZObmc/95tnVyIvxWUFR1GnCg0LSfVPXYiSBzgAR/5SmoT5sGtG4Qx7AptpViTKg7eDSHD2Ztpn+7hrZ1coICqYdyK/BDMc8p8JOILBSRwb5+47Aw4cHzk9m07yjfLratFGNC2TszNpKRm8/957V0O0rQEVX135uJJAETVPWUQtOfBLoCl2kRgUSksaruEJF6OLvJ7vNu8RT1HoOBwQDx8fFdxo0bV6psqsrTs7LIzleeO7MK4WH+uRJbeno6sbGxfnmv8rCcvtWzZ09SUlLcjlGiYFmfvsqZnqM88msG7ePDubuj7887CYb12bNnz4Vlbi2oqt9uQBKwvNC0m4HZQNVSLuMZ4JHSzJucnKwnYtLyndr0sQk6bv7WE3pdeaSkpPjtvcrDcvqW86cX+IJlffoq58s/rtakxyfo6p1HfLK8woJhfQILtIyf8a7u8hKRvsBQ4GJVzShmnmoiEnfsPtAHWF7UvOXVu019TmlcneF2xJcxIedQRg7vz9pMv1Ma0qqB9U7Kwp+HDX+KsyXSSkS2i8htwHAgDpjsPSR4pHfeRiIy0fvS+sAMEVkCzAO+V9UfKygjD56XzJb9GXxt56UYE1LenbGJ9Ow8652Ug98uO6aqg4qY/G4x86YC/bz3NwIdKjDaH5x3cj3aNa7BG7+s49JOje260caEgINHc3hv5mb6tWtgWyflYJ+WhYgID57fkm0HMvl6kY3xZUwoGDltA0dz8njw/GS3owQ1KyhF6NW6Hh0SavDGL+vJybNeijGV2Z60LMbO2szADo1Irm9bJ+VhBaUIIsKDvZPZfjCTLxZuczuOMaYCvZmygdx8ta0TgIwij40qNSsoxTg3OZ6uTWsxbMo6u16KMZXUjkOZfDJ3K1d1TSCpboiPKPzbb9C5c7kWYQWlGCLCoxe0YveRbD6YvdntOMaYCvDGlHUA3NcrhI/s8njglVegRw9ISyvXoqygHEeP5nU4OzmeN6duIC0r1+04xhgf2rTvKF8s3M61PZrQqGYVt+O45+ab4dFHYcAAWLq0XIuyglKCR/u04lBGLu9M3+R2FGOMD/3n57VEhYdxT88QHVH42ChXN98Mo0bBV19BnTrlWqQVlBK0S6hBv3YNeGf6RvanZ7sdxxjjA2t2pTF+SSo3n5FEfFy023H86+hRuPNOeOop53GvXnDHHSDlH7/QCkopPNQ7mczcfN6ausHtKMYYH3ht8hpioyK48+zmbkfxr99+gy5dYPRoyPf9wUZWUEqhRb04LuucwAdztrDzcKbbcYwx5fDb1oNMWrGb289qTs2qUW7H8Y/Cjfeff4bnn/f521hBKaUHzmuJqjJsynq3oxhjykhVef6H1dSNjeL2s5q5Hcd/1q6Fv/7198Z7r14V8jZWUEopsXZVru3ehHELtrFp31G34xhjymDqmr3M23SAB85rSbVovw1l6J7Fi52vrVvDwoU+abwfjxWUE3Bvr5ZER4Tx8qTVbkcxxpygfI/y4o+rSapTlWu6N3E7TsXKyIAhQ6BTJ5g0yZnWrp1PGu/HYwXlBMTHRTP47OZMXLaLRVsPuh3HGHMCvvltB6t3pfHIBa0q9yjix854HzUKhg6Fnj399taVeK1WjDvOak7d2Gien7jq2BUkjTEBLis3n9d+WkOHhBr0b9fQ7TgV5803/9h4f/FFiPLfgQdWUE5QtegI/tK7JfM3H+TnVXvcjmOMKYUPZ28h9XAWj13YGqng3T6uiour8Mb78VhBKYOruybSPL4aL/ywyi4VbEyAO5yZy/CU9ZyTHM/pJ9V1O47vffstfPCBc//66yu88X48VlDKICI8jMf7tmbD3qOMW2AX4TImkL01dQNHsnJ5rG9rt6P41rHG+yWXOCcqejxO093FLTArKGXUu019uiXV4vWf13I0O8/tOMaYIqQeyuS9mZsY2KERbRpVdzuO7xw74/1Y433KFAhz/+PcrwlEZIyI7BGR5QWm1RaRySKyzvu1VjGvvck7zzoRucl/qYsmIjzR72T2pmXbwJHGBKiXfnQO8X+0Mm2dbNsGp50GR4640ng/Hn+XtPeBvoWmPQ5MUdWWwBTv4z8QkdrA00APoDvwdHGFx586N6lFv3YNeHvaBvakZbkdxxhTwOJth/jv4lRuP6sZjSvD8PSZ3mGfEhPh7bdda7wfj18LiqpOAw4UmjwQGOu9Pxa4pIiXXgBMVtUDqnoQmMyfC5MrHr2gNTl5Hl6fvNbtKMYYL1XlXxNWUjc2mrvOrQTD03/7LTRrBrNnO49vusm1xvvxBMLYA/VVdaf3/i6gfhHzNAYKXtx9u3fan4jIYGAwQHx8PFOnTvVd0mKclxjOZ/O20TpiL02rh5/Qa9PT0/2Ssbwsp+8FQ85gWZ+Fc87blceCLdnc0jaKBbNnuBeskBNdn2FZWbR4800affcdaS1bsnLdOjKzA/gyGqrq1xuQBCwv8PhQoecPFvGaR4C/FXj8FPBISe+VnJys/nAoI0c7PjtJr357lno8nhN6bUpKSsWE8jHL6VvOn17gC5b1WTBnZk6envniFL3g9V81L//E/h4r2gmtz0WLVFu3VhVRHTpUNTu7wnIVBCzQMn6+u39YAOwWkYYA3q9FnS24A0gs8DjBOy0g1KgSyUN9WjFn4wF+Wrnb7TjGhLT3Z21m24FM/ta/DeFhQXwS448/BmTj/XgCoaCMB44dtXUT8G0R80wC+ohILW8zvo93WsAY1C2R5PqxPDdxFdl5vr9wjTGmZPvSsxnxy3rOa12PM1sG4UmMqakwc6Zzf+hQWLYs4Brvx+Pvw4Y/BWYDrURku4jcBrwA9BaRdcD53seISFcReQdAVQ8A/wTme2//8E4LGBHhYTw1oA1b9mfw/szNbscxJiS9PnktGbn5PNHvZLejnLhvv4X27Z2z3XNzITwcatd2O9UJ8WtTXlUHFfPUeUXMuwC4vcDjMcCYCormE2e1jOe81vV445f1XNY5IfSuVW2Mi1bvOsKn87Zyw6lNaVEv1u04pZeRAQ895BwK3LkzfPIJREa6napMAmGXV6Xy1/4nOyOb2mHExviNqvL0tyuoXiWSB89PdjtO6e3f75zx/vbb8OijzmHBrVq5narMrKD42Enxsdx0ehKfz9/KytQjbscxJiTM25XP3E0HeKRPK2pVC/zm9f/Urg3nnec03l96KSga78djBaUC3N+rJTWrRvH0+OV2zRRjKlhGTh6fr8mhbaPqDAqGKzGmpsKll8K6dc5AjsOHO0WlErCCUgFqVI3ksb6tmL/5IN/8FjBHNxtTKY1IWc+BLOXZi9sG/GHCdWbOdBrvP/0EK1a4HcfnrKBUkCu7JNIxsSbPTVzNkaxct+MYUylt3neU0dM2cVqjcLomBfARURkZcNddtPvb36BpU1i0yBl2vpKxglJBwsKEfwxsy/6j2fxn8jq34xhTKf1zwkoiw4WrkwO89/DyyzByJFuvvjroG+/HYwWlArVPqMmg7k0YO3szq3dZg94YX0pZvYcpq/fwwPktqRkTgB9lHo/TLwHnCK5ff2XjkCFB33g/ngD8KVQuj/ZpRVxMBH//doU16I3xkey8fJ79bgXN46tx8+nN3I7zZ6mp0LcvnHOOs7uralU4+2y3U1U4KygVrFa1KIZe0Jp5mw4wfkmq23GMqRRGTt3I5v0ZPHNRW6IiAuxjbPx4p/E+Y4azZVKlElyLpZQC7CdROV3dLZH2CTX49/erSLMGvTHlsmnfUUZMXc+A9g05Ozne7Ti/y8qCu+6CgQN/b7wPHuzqNd79zQqKH4SHCf8YeAp707N53Rr0xpSZqvLUf5cTHR7G3we0cTvOH0VEwPLlv5/x3roSXXa4lKyg+EnHxJpc16MJ78/axLLth92OY0xQGr8klRnr9/Fo31bUqx7jdhyn8T58OOzd6xSUX36pFGe8l5UVFD8a2rc1dWKjeeKbpeTle9yOY0xQOZyZyz8nrKJ9Qg2u69HU7Tiwc6fTeL/vPhjjHbc2SAd19BUrKH5UPSaSZy5qy/IdRxg7e4vbcYwJKi9PWs2Bo9k8d2k798+IHz8e2rVzGu9vv+1cu8RYQfG3fu0a0Kt1PV79aQ07DmW6HceYoPDb1oN8PHcrN52exCmNa7gbZvRop/HepElINt6PxwqKn4kIz17cFlV4+lsbPNKYkuTle3jym+XUj4vh4T4unmHu8e6mvuQSePrpkG28H48VFBck1q7KQ72T+XnVHhbutssFG3M878zYxMqdR3j6ojbERvv1moAOjwdefdUZETgvD+Lj4ZlnINouoFeYFRSX3HJGEm0aVuejVTk2eKQxxdi4N53XJ6/lgrb16XtKA/8HONZ4f+QRqFHDOevdFMv1giIirURkcYHbERF5sNA854rI4QLz/N2tvL4SER7G85e143C28uIPq92OY0zA8XiUx79aRnREGP8ceAri7z5F4cb7N99A9er+zRBkXNh+/CNVXQN0BBCRcGAH8E0Rs05X1QH+zFbROiTWpE9SBB/P3Ur/dg05vUVdtyMZEzA+nreVeZsP8NIV7f1/zklOjnOCYpMmzjXerVdSKq5voRRyHrBBVUPmmNrLWkaRVKcqj329lIycPLfjGBMQdhzK5IWJqzirZV2u7JLgvzdeutTZrRUVBZMmWeP9BEkgHWUkImOARao6vND0c4GvgO1AKvCIqhZ5uTMRGQwMBoiPj+8ybty4Cs1cXunp6ezIqcIL87I4v2kE150cmI2+9PR0YmNj3Y5RomDJ2bNnT1JSUtyOUSI31qeq8vrCbNYczOdfZ1QhvmrJ//eWO6fHQ8KXX9L8nXfYfsUVbBw8uOzLOo5g+P3s2bPnQlXtWqYXq2pA3IAoYB9Qv4jnqgOx3vv9gHWlWWZycrIGupSUFFVVffrb5Zr0+ASdt2m/u4GKcSxnoAuWnM6fXuBzY31+vWibNn1sgo6ZsbHUrylXztRU1d69VUF14EDVvXvLvqwSBMPvJ7BAy/g5Hki7vC7E2TrZXfgJVT2iqune+xOBSBGpVA2HoX1bkVCrCkO/XEpmjh1KbELT3rRsnv1uJZ2b1OTG05Iq/g2nTv19qPmRI53Ge91K9dHiV4FUUAYBnxb1hIg0EO8hHiLSHSf3fj9mq3BVoyJ48bL2bNp3lNcmr3E7jjF+p6o88fUyMnLyeemK9v4ZXiUhAU4+GRYuhDvvtDPeyykgCoqIVAN6A18XmDZERIZ4H14BLBeRJcAw4BrvplmlcnqLulzbownvztjEwi0H3Y5jjF99uXA7P6/azdALWtGiXlzFvdHixc4RXKrQogVMm+YUFVNuAVFQVPWoqtZR1cMFpo1U1ZHe+8NVta2qdlDVU1V1lntpK9YTF7amYY0qPPLFEjvqy4SM7Qcz+Md3K+nRrDa3nlFBl/T1eOC116BHD/j449+v9258JiAKivldXEwkr17Vgc37j/LcxFVuxzGmwnk8yqNfLMWjyitXdiCsInZ17dwJF14IDz/sfF26FBo39v37hDgrKAHo1OZ1uOOs5nw0Zyspa/a4HceYCjV29mZmb9zPUwPakFi7qu/fwOOB88+H6dOt8V7BrKAEqIf7JNO6QRxDv1zKgaM5bscxpkKs35POCz+splfrelzdLdG3C8/IgPx8CAuDESOs8e4HVlACVHREOK9f3ZHDGbn89etlNsy9qXTy8j08/MUSqkSF88Jl7Xw7VtfixdC1K7z4ovP43HOt8e4HVlAC2MkNq/Nwn2R+XLGLrxftcDuOMT41bMo6lmw7xL8uOcV3Y3UVbLwfOgTduvlmuaZUrKAEuNvPak73ZrV5evwKth2wobNN5TB3436Gp6zn8s4JDGjfyDcLLdh479vXabz37u2bZZtSsYIS4MLDhFev7ADAXz5fTF6+x+VExpTPoYwcHvx8MU1qV+XZgW19t+AtW5zBHEeOhP/+1xrvLrCCEgQSa1flucvasWDLQf7z8zq34xhTZqrONU72pWfzxqDO5b8CY0YGfPmlc//UU2HrVmu8u6hUBUVEwkTkbRHZLyLqveBVLRHZLSInlXIZ0SKyVUTKNopliLu4QyOu7prIiKnrmbl+n9txjCmTT+Zt5ccVu3j0gla0S6hRvoUda7xfdRVVtm1zptWsWf6QpsxKu4XSD7gFuAhoCMwC/gpMVNUNpVmAqmYDLwMvliGnAZ6+uA0nxcfy4OeL2ZuW7XYcY07Iut1p/HPCSs5qWZfbz2xe9gUVbrxPmkRmoo8POTZlUtqC0gLYqaqzVHUXzpUebwfePcH3+xg4U0R8uOM0dFSNimD4tZ04kpnLw18sweOxQ4lNcMjKzee+T3+jWlQEr15VjrPhVeHyy63xHqBKLCgi8j7wOtDEu7trM84WiwIzC8z3lIjsEpF6BaZ9KiKLRCQKQFUPeF8zyKffRQhp3aA6f7+oDdPW7mXU9I1uxzGmVJ79biWrd6XxypUdqBdXjkOEReCii+Ctt6zxHoBKs4XyAPAPnKslNgS6AWcBCwuN+PscsA4YAyAiNwIDgWtVteCp3vOAc8ofPXRd270J/do14JVJa1i01UYlNoHtq4Xb+XTeVoaccxI9W9cr+QWFZWTA3XfDBx84j2+9FYYMscZ7ACqxoHhHAE4D8lV1l6ruBZriXIq34Hz5wPU4u7ReAoYDD6vq6kKLTAWSfJA9ZIkIz1/WngY1Yrj340XsT7d+iglMq3cd4cn/LqNHs9o80if5xBewZInTeH/rLdhQqnatcVFZDxuuAmQVnqiqW3C2aB4FpqnqW0W8NtP7elMONapEMvL6Luw7msP9n/1GvvVTTIBJy8rlro8WERcTyRvXdiIi/AQ+bjweeP116N7dabz/9BM8+2zFhTU+UdaCsg+oVcxzZwP5QKKIRBfxfG1gbxnf1xRwSuMa/GvgKcxcv59Xf7KrPJrAcex8k60HMnhjUKcT75vMmAEPPWSN9yBT1oLyG9Cm8EQRuQy4DugF1ACeL+K1pwCLyvi+ppCruiUyqHsib07dwE8rdrkdxxgA3pu5me+X7eTRC1pxavM6pX/h5s3O17PPdq73bo33oFLWgjIJOFlE/vebIiKNgdHAX1V1GnADcJ+InF/otWcBP5bxfU0Rnr6oLe0a1+DhcUvYtO+o23FMiFuw+QDPTVzF+SfX586zS3m+SUYG3HMPtGrlbJEAnHOONd6DTJkKiqouwzla6xoAccadfh9ny+V17zzTgReAsccKj4ichrPl8mXhZYrIZhFZJiKLRWRBEc+LiAwTkfUislREOpcle2UUExnOW9d3JjxcGPLhQrt0sHFN6qFMhny0kMa1qvDqlR1KNyT9scb7m2/Cffc5RcUEpVIVFFV9RVWTCk1+FrhfRMLV0VtVzy94KLGqPqWqjVV1v3fSQ8DLqppZzFv1VNWOqlrU8CwXAi29t8FAUQ3/kJVQqyrDrunE2j1pDP1yqV0/xfhdZk4+gz9cQFauh3du7EqNqpElv2jYsD823l95BaKLar2aYFDmwSFV9UdgBJBQmvm9DfqleLdgymAg8IG3eM0BaopIwzIuq1I6OzmeoRe0ZsLSnbzxy3q345gQoqo89tVSVqQe4T9Xd6Rl/bjSvXDvXmu8VyISKP/Jisgm4CDOGfhvq+qoQs9PAF5Q1Rnex1OAx1R1QaH5BuNswRAfH99l3Lhx/ohfZunp6cTGxvpsearK6GU5zErN456O0XRrUM7RXL18nbOiBEvOnj17kpKS4naMEpV2fX6/MYcv1uZyectILjop6rjz1pk9m/yYGA516vT7JXrL2SsJlp97MOTs2bPnwmL2EpVMVQPiBjT2fq0HLAHOLvT8BODMAo+nAF2Pt8zk5GQNdCkpKT5fZmZOnl4yYoa2+ttEXbb9kE+WWRE5K0Kw5HT+9AJfadbnlFW7NOnxCXrPxwvV4/EUP2NGhurdd6uCav/+vgupwfNzD4acwAIt4+d4wFwPRVV3eL/uAb4BuheaZQdQcEjRBO80U0hMZDhv39CF2lWjuOODBew58qdzUI3xifV70njg08W0aVidl684ThO+YOP94Yfhq6/8G9T4RUAUFBGpJiJxx+4DfYDlhWYbD9zoPdrrVOCwqu70c9SgUS8uhtE3deVQRi6DP1xIVm6+25FMJbM3LZub35tPdGQYo27sSpWo8KJnXLLEabwfPGiN90ouIAoKUB+YISJLcA5H/l5VfxSRISIyxDvPRGAjsB7nfJe73YkaPNo2qsHrV3dg8bZDPPrlUhvu3vhMRk4et42dz/70HMbc3I3GNYsYTSnf+09M+/bw9NPWeA8BvunYlpOqbgQ6FDF9ZIH7Ctzjz1yVQd9TGvJY39a8+ONqGtWI4Yl+J7sdyQS5fI9y/6e/sXzHYUbd0JX2CUVcJXHCBGfolMmToWlT+Otf/R/U+F2gbKGYCjTknObccGpT3p62kTEzNrkdxwQxVeXZ71bw86o9PHNxW85vU/+PM2RmOme8X3QRVK0KOTlFL8hUSgGxhWIqlojwzMVt2X0ki39+v5IGNWLo185O4TEn7p3pm/hg9hbuOKsZN56W9Mcnly6FQYNg5Uqn8f7vf1uvJMTYFkqICA8Thg3qROcmtXjw88XM3bi/5BcZU8B3S1L598RV9GvXgCcuLGLX6RtvWOM9xFlBCSExkeG8c2NXEmpV4Y4PFrB2d5rbkUyQSFmzh798vphuSbV47aqOv18TftcuWLvWuf/aa84RXdZ4D1lWUEJMrWpRjL2lO9GR4dz47jy2HchwO5IJcPM2HeCujxbSqkEc797cjZhI7+HBEyZAu3Zw/fWgCnFxEB/vbljjKisoISixdlU+vK07mbn5DBo9h52Hixur04S6zYfzue39+TSuWYUPbu1O9ZjIPzbeGzeGsWNtmHkDWEEJWa0bVOeDW7tzKCOX60bPZW+aXZfe/NH6Pem8uiCL6lUi+fC2HtSJjYZt234/4/2hh2DuXDjZDkU3DisoIaxDYk3eu6UbO2vCpIMAABjLSURBVA9nccO7czl41A7xNI7tBzO44d25iAgf3d6DRsdOXKxfH5o1g0mT4NVXrfFu/sAKSojrllSb0Td2ZePeo9z03jzSsnLdjmRctu1ABteMmsPR7Dwe6RpNs7w0uOUW5wiuqCind9Knj9sxTQCygmI4s2Vd3ryuMytTj3Dze/M5YkUlZB0rJmlZeXx8+6l0WjHPGTrl889hwZ8upGrMH1hBMQCc36Y+wwZ1Ysm2Q9zwzlwOZ1hRCTVb9zvFJD07j0+ua0+7F56k3V//Co0awcKFdjiwKZEVFPM//do15K3ru7BqZxqDRs9hf7o16kPFlv1HuXrUbI7m5PHJHT1o++qzMGIE26680hrvptSsoJg/6N2mPqNv6sqGvekMGj2HPWl2LZXKbtO+o1z99hyys3P57Oo2tG1UA556Cn76iQ13322Nd1NqVlDMn5yTHM97t3Rj24FMrnnbzlOpzJbvOMwVb80i7tA+pv36Cq3vuhE8Hmc3l+3iMifICoop0ukn1eXD27qzJy2bK0fOZme6x+1Ixsdmrd/HNaPm0Gv9PH4ccw+xc2fBVVfZSYqmzKygmGJ1TarNJ3f0IDMnn3/NzWThloNuRzI+MnHZTu4cNYPnpozk5Q/+RnhCgtN4HzLECoopMyso5rjaJ9Tk67tPJzZSuHb0HH5ascvtSKacPpyzhXs+WUTH+lXov/03O+Pd+IwVFFOipnWq8eSpVWjdsDpDPlrIR3O2uB3JlIHHo7z64yoW//P/6N28JqPuO5/wZcvsjHfjM64XFBFJFJEUEVkpIitE5IEi5jlXRA6LyGLv7e9uZA1l1aOET+/owbmt6vG3/y7npR9X2zXqg0hGTh5PvPkTXe++nlcnvs6brKRKVLgzQrAxPhIIV2zMAx5W1UUiEgcsFJHJqrqy0HzTVXWAC/mMV9WoCEbd0IWnvl3Om1M3sHZ3Oq9f3YG4mEi3o5nj2Hk4k1FDhzH0w+eo4clB33yTiNtvdzuWqYRc30JR1Z2qush7Pw1YBTR2N5UpTkR4GM9d2o5nLmpDypo9XPbmLDbvO+p2LFOMxdsO8cXVD/D0qMeJapJIxG+LkLvussa7qRCiGji7LUQkCZgGnKKqRwpMPxf4CtgOpAKPqOqKYpYxGBgMEB8f32XcuHEVG7qc0tPTiY2NdTtGiYrKuXJ/PiMWZ6EKd3WIpl28+xu8wbI+e/bsSUpKSoW+x+zUPMYsz6bT4a08s3Mqe4fchkZFndAygmV9Wk7f6dmz50JV7VqmF6tqQNyAWGAhcFkRz1UHYr33+wHrSrPM5ORkDXQpKSluRyiV4nJu3X9UL3j9V232+AQdOXW9ejwe/wYrJFjWp/OnVzEys3L0u1uH6ift++iVb83SfWlZZV5WsKxPy+k7wAIt4+e467u8AEQkEmcL5GNV/brw86p6RFXTvfcnApEiUtfPMU0REmtX5au7TqfvKQ14/ofV3DZ2gY0B5m8ffwxJSRAWRl6jxqQ2a82AMS/RpWouH9/Y0bkwljF+4HpBEREB3gVWqeprxczTwDsfItIdJ/d+/6U0x1MtOoIR13bm2YvbMmPdPvoNm87sDfbj8YuPP4bBg2HLFlAlYmcqzXZuZMelV5M8awqRVau4ndCEENcLCnAGcAPQq8Bhwf1EZIiIDPHOcwWwXESWAMOAa7ybZiZAiAg3nZ7EN/ecTrWoCK59Zw6v/bSGvHwbsqVCPfkkZGT8YZIAjRfNsca78TvXu6iqOgPnb+B48wwHhvsnkSmPto1q8N19Z/L0+BUM+2U9Mzfs5+Ur2tM8PrAbkcFKt24t+o9n61Z/RzEmILZQTCVTLTqCV67swH+u7si63Wlc+H/TGTVtA/l2IqTPZGXnMumOx6G4DfUmTfwbyBisoJgKdEmnxkx+6BzOTo7nuYmrueytWazdneZ2rKC3dMFqlnY4gwveeZHUZq3RKoX6JFWrwr//7U44E9KsoJgKVb96DKNu6MIbgzqx7UAG/YdN5/9+XkdWbr7b0YLOoYwcxvz9bRqdcyodNixh/TMv0XjDSmT0aGja1OmZNG0Ko0bBdde5HdeEINd7KKbyExEu6tCI00+qwzPfreT1n9fy1aLt/H1AG847uR5izePj8niUr3/bwfMTV9F6wwEubNCIvK8+o0XH9s4M111nBcQEBNtCMX5TJzaaNwZ14qPbehAVEcbtHyzglvfns852gxVr7sb9PPjkWJY88W+a1qnKky/fQ8N1y6l2rJgYE0BsC8X43Zkt6/LDA2cxdtZm/u/ndVzwn2lc0SWBB89PplFNO28CYP2edF75YSUNP3yHl399H0+t2kR/8SJhNaq7Hc2YYllBMa6IDA/j9rOac1nnBEakrOfD2Vv47+JUbjy1KYPPbk696jFuR3TFpn1HGTZlHTNnLOe1H/7DmRsWkt+vP+HvvwdWTEyAs4JiXFW7WhRPDWjDzacn8frPaxkzcxMfzNnCoG6J3HnOSSGzxbJq5xFGT9vIfxfvoLrmMe2zR4lLPwQjRhBuowObIGEFxQSExNpVee2qjtzfqyVvTd3Ax3O38vHcrfRr15BbzkiiU5Nabkf0OVVl5vr9jJq+kWlr91I9XLn1jObcec5JVG/3CnTsCG3buh3TmFKzgmICSlLdarx4RXvuP78lY2ZsYtz8bYxfkkqHxJpc36MJ/do1pFp0cP/aHjyaQ1y3Sznv1V/ZuO8odWOjeaGFcuVrjxHe6VmIa2NHbZmgFNx/mabSalyzCk8NaMNfeifz1cLtjJ29mUe/XMrT41fQv11DLu+SQLek2oSHBceuoOy8fKau2cv4xalMXrWb2r1uo2bVSF65vB0Dp39F5L1PQM2aUKOG21GNKTMrKCagxUZHcNPpSdx4WlMWbjnIuAXb+H7pTr5YuJ26sdH0PaU+F57SkG5JtYmKCKyj4NOz85ixbi8/r9rDpBW7SMvKo061KAZ1S+T52wewedkcuPlmmDQJBgyAMWMgPt7t2MaUmRUUExREhK5JtemaVJtnLm7LL6v38MOyXXy1cAcfzdlK1ahwTmteh4aSS8NdabSsF0uYn7decvM9LNtxmLkbDzBrwz7mbjxATr6HuJgIep9cn4s7NuKMFnWJDA/jH5dsgalT4ddfYcQIsMa7qQSsoJigUzUqggHtGzGgfSMyc/KZvm4v09btZdrafUw5kMNHq6YRFx1BxyY16ZhYk1YN4mhVP46kutWIDPfNVkxmTj6b9x9l1c4jrEg9wvIdh1m24zAZOc6QMi3qxXLT6U3p1bo+XZNq/f6+mZkwf75z/5pr4MwzISHBJ5mMcZsVFBPUqkSF06dtA/q0bQDAuIm/EF4/mYVbD7Joy0FGpKzn2CDHEWFCw5oxNK5ZhUY1qxAfF02NKpHUqBJJbHQEkeFhhIcJ4SLk5HvIzMknKy+fw5m57EvLYV96NruPZLFlfwa7jmT9L0NMZBgnN6zOlV0S6NG8Dt2b1aZuUVdJXLoUBg2CTZv4344tKyamErGCYiqVelXDOLdLApd3cT6os3Lz2bA3nXW701m7O40dhzLZcTCT2Rv2sz89h5xSXgCsWlQ4deOiiY+N5vQWdWhWpxpN61ajdYM4mtetRsTxtnw8Hhg2DB57DGrVgm++YW/fvr74do0JKFZQTKUWExlO20Y1aNvoz0dPqSpZuR4OZ+aSnp1LvgfyPB7yPUp0RDgxkWFUiQwnLiaSKlHhZQuQlwcXXww//GCNd1PpWUExIUtEqBIV7i0WFTTUS0QEdOoE/fvD3Xdb491UagFznKWI9BWRNSKyXkQeL+L5aBH53Pv8XBFJ8n9KY0ohMxPuuw9mznQe//vfcM89VkxMpRcQBUVEwoERwIVAG2CQiLQpNNttwEFVbQG8Drzo35TGlMLSpdCtGwwfDtOnu53GGL8KiIICdAfWq+pGVc0BPgMGFppnIDDWe/9L4DyxKzOZQOHxwP/9H3TvDvv2OT2Tx/+0oW1MpRYoPZTGwLYCj7cDPYqbR1XzROQwUAfYV3AmERkMDC7wuCLyGvMHV+P8F/QdcOvu3ey78MISX2O/m6ayCZQtFJ9R1VGq2lVVuyYnJ6OqAX1LSUlxPYPlLMdt715Ulc/y8uDLL7nI42FvKV7n/V0N+Jv93EMvZ3kESkHZASQWeJzgnVbkPCISAdQA9vslnTGFZWbCvffCySfDzp0QHg6XX26NdxPSAqWgzAdaikgzEYkCrgHGF5pnPHCT9/4VwC9a3nJqTFkca7yPGAHXX++crGiMCYweijo9kXuBSUA4MEZVV4jIP4AFqjoeeBf4UETWAwdwio4x/qP6+xnvNWs6jXc7492Y/wmIggKgqhOBiYWm/b3A/SzgSn/nMuYPZs6E3r3h3XehXj230xgTUAKmoBgTsCZOhBYtIDkZxo6FmBjrlRhThEDpoRgTeI6d8d6/P/zrX860KlWsmBhTDNtCMaYoS5fCtdfCihXw4IPw/PNuJzIm4FlBMaawqVOdZrs13o05IbbLy5hjjh2F3qMHDB7sbKVYMTGm1KygGAPw/ffO5XjT0pw+ybBhdhSXMSfICooJbcca7wMGOMVk376SX2OMKZIVFBO6Cg41/+CDMG8eNGvmdipjgpY15U3oevjh34eat16JMeVmBcWElt27ISzMua77e+9BVJT1SozxEdvlZULH999Du3YwZIjzOCHBiokxPmQFxVR+BRvvDRrAP/7hdiJjKiXb5WUqt3Xr4LLLYPlyeOABeOEFZywuY4zPWUExlVutWk6fxBrvxlQ42+VlKp/du2HoUMjNhbp1YcECKybG+IEVFFOp1J4zB9q3hzfegEWLnIk2OrAxfmEFxVQOmZlw//20f+IJqF/f2Srp0cPtVMaEFOuhmMrh+uvh66/ZfvnlJHz0kTXejXGBqwVFRF4GLgJygA3ALap6qIj5NgNpQD6Qp6pd/ZnTBChVp08SFQV/+xvccQfrY2JIsGJijCvc3uU1GThFVdsDa4EnjjNvT1XtaMXEAE7jvX9/uP9+53GnTtZ4N8ZlrhYUVf1JVfO8D+cACW7mMUFi4kSn8Z6S4pz5bowJCG5voRR0K/BDMc8p8JOILBSRwX7MZAJJVpazRdK//++N93vucTuVMcZL9NhV6irqDUR+BhoU8dSTqvqtd54nga7AZVpEIBFprKo7RKQezm6y+1R1WjHvNxgYDBAfH99l3LhxPvpOKkZ6ejqxsbFuxyhRIOSMSU2l6x13sOvCC9k4eDCeqKg/zRMIOUujZ8+epKSkuB2jRMGyPi2n7/Ts2XNhmVsLqurqDbgZmA1ULeX8zwCPlGbe5ORkDXQpKSluRygV13J6PKrjxztfVVV37Tru7MGyPp0/vcAXLOvTcvoOsEDL+Hnu6i4vEekLDAUuVtWMYuapJiJxx+4DfYDl/ktpXHOs8X7xxU7fBJxdXcaYgOR2D2U4EAdMFpHFIjISQEQaiYj3E4T6wAwRWQLMA75X1R/diWv8pmDjffhw6NfP7UTGmBK4eh6KqrYoZnoq0M97fyPQwZ+5jMueeQaefdY5guuXX6BtW7cTGWNKwe0tFGP+rEcPZ6j5efOsmBgTRGzoFeM+VWe3VmamM0rwhRc6N2NMULEtFOOugme8z5oFHo/biYwxZWQFxbincOP9m28gzH4ljQlWtsvLuGPrVhg4EE4+2RrvxlQSVlCMf+3aBQ0aQJMmzhbKWWfZUPPGVBK2f8H4h6pzFcVmzZzruwP07m3FxJhKxLZQTMXbvRtuucUpJP37Q5cubicyxlQA20IxFevHH//YeP/uO6hXz+1UxpgKYFsopmJt2+aMv2WNd2MqPdtCMb63fDl8+61z//bbYf58KybGhAArKMZ3jjXeu3aFhx92rvcuAtHRbiczxviBFRTjG7t3w4ABzhnv553nnPUeGel2KmOMH1kPxZTfvn3QoQMcPuw03u++29kyMcaEFCsopuxUncJRty785S/OFor1SowJWbbLy5TN8uVOr2ThQufxY49ZMTEmxFlBMSemYON9+3ZnN5cxxmAFxZyIwo33pUuhVy+3UxljAoQVFFN6Y8bAlCnOFsqECc4Ji8YY4+VqQRGRZ0Rkh4gs9t76FTNfXxFZIyLrReRxf+cMaVlZsGKFc//RR2HJErj3XjuKyxjzJ4GwhfK6qnb03iYWflJEwoERwIVAG2CQiLTxd8iQtHw5dO8OffoQlpUFERHQqpXbqYwxASoQCkpJugPrVXWjquYAnwEDXc5UuR27xnvXrk7f5J138Ngw88aYEgTCeSj3isiNwALgYVU9WOj5xsC2Ao+3Az2KW5iIDAYGex9mi8hyX4atAHWBfW6HKNaePdCvHwR6zt8FTU4RCYqcBMn6xHL6Spl3Q1R4QRGRn4EGRTz1JPAW8E9AvV9fBW4tz/up6ihglPe9F6hq1/Isr6IFQ0awnL5mOX3LcvqOiCwo62srvKCo6vmlmU9ERgMTinhqB5BY4HGCd5oxxpgA4vZRXg0LPLwUKGr31HygpYg0E5Eo4BpgvD/yGWOMKT23eygviUhHnF1em4E7AUSkEfCOqvZT1TwRuReYBIQDY1R1RSmXP6oCMvtaMGQEy+lrltO3LKfvlDmjqKovgxhjjAlRwXDYsDHGmCBgBcUYY4xPVKqCIiIvi8hqEVkqIt+ISM1i5nNtKBcRuVJEVoiIR0SKPXxQRDaLyDLvkDRlPoyvrE4gp6vD4ohIbRGZLCLrvF9rFTNffoEhfvx2UEdJ60dEokXkc+/zc0UkyV/ZCuUoKefNIrK3wDq83YWMY0RkT3HnloljmPd7WCoinf2d0ZujpJznisjhAuvy7y5kTBSRFBFZ6f07f6CIeU58fapqpbkBfYAI7/0XgReLmCcc2AA0B6KAJUAbP2Y8GefEoalA1+PMtxmo6+K6LDGn2+vSm+El4HHv/ceL+pl7n0t3YR2WuH6Au4GR3vvXAJ8HaM6bgeH+zlYow9lAZ2B5Mc/3A34ABDgVmBugOc8FJri8LhsCnb3344C1RfzMT3h9VqotFFX9SVXzvA/n4JyzUpirQ7mo6ipVXeOv9yurUuYMhGFxBgJjvffHApf4+f2PpzTrp2D+L4HzRPw+8mYg/BxLpKrTgAPHmWUg8IE65gA1C52a4BelyOk6Vd2pqou899OAVTijkhR0wuuzUhWUQm7Fqa6FFTWUS+EVGQgU+ElEFnqHkwlEgbAu66vqTu/9XUBxY+rHiMgCEZkjIv4qOqVZP/+bx/vP0GGgjl/SFZHBq7if4+XeXR9fikhiEc+7LRB+H0vrNBFZIiI/iIirlzr17mbtBMwt9NQJr0+3z0M5YccbykVVv/XO8ySQB3zsz2zHlCZjKZypqjtEpB4wWURWe//z8Rkf5axwJQzf8z+qqiJS3HHwTb3rsznwi4gsU9UNvs5aiX0HfKqq2SJyJ85WlV1drWwW4fw+potzyY7/Ai3dCCIiscBXwIOqeqS8ywu6gqIlDOUiIjcDA4Dz1LsjsJAKH8qlpIylXMYO79c9IvINzm4JnxYUH+T0y7A4x8spIrtFpKGq7vRuju8pZhnH1udGEZmK8x9ZRReU0qyfY/NsF5EIoAawv4JzFVZiTlUtmOkdnN5VoAmKYZoKfnCr6kQReVNE6qqqXweNFJFInGLysap+XcQsJ7w+K9UuLxHpCwwFLlbVjGJmC/ihXESkmojEHbuPc7BBII6aHAjrcjxwk/f+TcCftqxEpJaIRHvv1wXOAFb6IVtp1k/B/FcAvxTzj1BFKjFnoX3nF+Pscw8044EbvUcnnQocLrA7NGCISINjfTIR6Y7zOezXfyK87/8usEpVXytmthNfn24eaeDrG7AeZ5/fYu/t2NEzjYCJBebrh3NUwwac3Tv+zHgpzr7IbGA3MKlwRpyjbZZ4byv8nbG0Od1el973rwNMAdYBPwO1vdO74gzfA3A6sMy7PpcBt/kx35/WD/APnH96AGKAL7y/u/OA5v5eh6XM+bz3d3EJkAK0diHjp8BOINf7u3kbMAQY4n1ecC7Gt8H7cy72KEqXc95bYF3OAU53IeOZOH3apQU+L/uVd33a0CvGGGN8olLt8jLGGOMeKyjGGGN8wgqKMcYYn7CCYowxxiesoBhjjPEJKyjGGGN8wgqKMcYYn7CCYowxxiesoBjjByISLyI7ReTpAtPai0iWiFzpZjZjfMXOlDfGT0TkApxRe8/BGepiATBPVW9xNZgxPmIFxRg/EpH/4Ayu+CtwFtBRVdPdTWWMb1hBMcaPvKMeL8G5/sXpqlr4okbGBC3roRjjX0k415hQnFGljak0bAvFGD/xXtBoDs4w8XOBp4EOqrrV1WDG+IgVFGP8REReAK4F2uNcO/4HnOuh9FJVj5vZjPEF2+VljB+IyDnAw8CNqnpInf/kbgbaAI+5mc0YX7EtFGOMMT5hWyjGGGN8wgqKMcYYn7CCYowxxiesoBhjjPEJKyjGGGN8wgqKMcYYn7CCYowxxiesoBhjjPGJ/wcOGt9G8B6TfQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "markdown", "metadata": { "id": "zPVKnTCH4EIH" }, "source": [ "## 4.4: Gradient Tape for fast diff" ] }, { "cell_type": "code", "metadata": { "id": "SobkFYXdvti0", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "5829c15d-e4af-4199-9d0c-baf19d0a6fcb" }, "source": [ "x = tf.Variable(1.0)\n", "\n", "with tf.GradientTape() as tape:\n", " z = f(x)\n", "grads = tape.gradient(z, [x])\n", "grads" ], "execution_count": 71, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "[]" ] }, "metadata": { "tags": [] }, "execution_count": 71 } ] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "uBc9oG_pPE5R", "outputId": "0230cb16-2e85-4732-afb2-80e9a815b453" }, "source": [ "x = tf.Variable(0.)\r\n", "\r\n", "with tf.GradientTape() as tape:\r\n", " z = f(x)\r\n", "grads = tape.gradient(z, [x])\r\n", "grads" ], "execution_count": 73, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "[]" ] }, "metadata": { "tags": [] }, "execution_count": 73 } ] }, { "cell_type": "markdown", "metadata": { "id": "BgEtmUQg4N7O" }, "source": [ "## 4.5: Gradient descent with Gradient Tape" ] }, { "cell_type": "code", "metadata": { "id": "WNTgOqiKxYfs" }, "source": [ "def f(x):\n", " return 5 * x ** 2 + 3 * x + 1." ], "execution_count": 59, "outputs": [] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 287 }, "id": "2OwtUVrwMnED", "outputId": "d08550fd-543d-4707-d120-75d027705032" }, "source": [ "xs = np.linspace(-2, 2, 200)\r\n", "fs = f(xs)\r\n", "x0 = 0.5\r\n", "df_x0 = approximate_diff(f, x0)\r\n", "tangent_x0 = df_x0 * (xs - x0) + f(x0)\r\n", "plt.plot([-2, 2], [0, 0], \"k-\", linewidth=1)\r\n", "plt.plot([0, 0], [-5, 15], \"k-\", linewidth=1)\r\n", "plt.plot(xs, fs)\r\n", "plt.plot(xs, tangent_x0, \"r--\")\r\n", "plt.plot(x0, f(x0), \"ro\")\r\n", "plt.grid(True)\r\n", "plt.xlabel(\"x\", fontsize=14)\r\n", "plt.ylabel(\"f(x)\", fontsize=14, rotation=0)\r\n", "plt.axis([-2, 2, -5, 15]);" ], "execution_count": 68, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEOCAYAAACuOOGFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3wUdf7H8dcnHUjooQcCSkCqCoKiqIAUEcV6ig0VRTw95c5e0fPsd+J5IAKC4okFOyCglCAiCgLSewm91ySk5/P7Y9b7xZiQkOzu7GY/z8djH9mdnZ15MyH55Dvzne9XVBVjjDGmvMLcDmCMMaZisIJijDHGK6ygGGOM8QorKMYYY7zCCooxxhivsIJijDHGK/xaUERkvIjsF5FVBZY9KyK7RGSZ59G3mM/2EZH1IrJJRB7zX2pjjDGlIf68D0VELgTSgPdVtY1n2bNAmqr+8ySfCwc2AD2BncAvwABVXePz0MYYY0rFry0UVZ0HHC7DRzsBm1R1i6pmAx8D/b0azhhjTLlEuB3A4z4RuRVYDDyoqkcKvd8Q2FHg9U6gc1EbEpHBwGCAyJgqHZo2buiDuN6Tn59PWFjgX8qynN61YcMGkpKS3I5RomA5nm7n3H9CycxTEmLDECl+PbdzxuzbR+SxY2TVqkV2rVpFrrNhw4aDqhpfph2oql8fQCKwqsDrukA4TmvpBWB8EZ+5FninwOtbgBEl7Sum3ul6PCNbA1lycrLbEUrFcnqX86MX+ILleLqZc8uBNE18bKq+Mn1tieu6ljM/X/Wvf1UF1UcfdV4XA1isZfz97vqfHqq6T1XzVDUfGItzequwXUBCgdeNPMtOKh/4cOF2r+Q0xpiijJm3hcjwMG47P9HtKMUbMQKGD4e//AVeeomTNqPKwfWCIiL1C7y8ClhVxGq/AM1FpKmIRAE3AJNL2nZMOIz/cStZuXneCWuMMQXsT83k86U7uebsRtSJi3E7TvEGDIAXXoA33vBZMQH/dxv+CPgJaCEiO0VkEPCqiKwUkRVAN+CvnnUbiMg0AFXNBe4DvgXWApNUdXVJ+6sWLew7nsXXv+720b/IGBPKJixIIScvn7u6NnU7StGmTIHsbKhdG554Anx8/cavF+VVdUARi8cVs+5uoG+B19OAaaeyv0oRwmn1q/L2vM1c26ERYWG+q8zGmNCSlpXLf3/aRp/W9WgWH+t2nD96+2245x545RV45BG/7NL1U16+dvdFzdhyIJ2Za/e5HcUYU4F8vGg7xzNzGXxhM7ej/NGECU4x6dcPhg71224rfEG5rG19EmpW4u3vN//WQ8wYY8olOzefcfO30rlpTc5qXMPtOL/3ySdwxx1wySXw6acQFeW3XVf4ghIRHsZdXZvx6/aj/JJS+PYWY4w5dVOW72bPsUyGXHSa21F+LzUV7rsPzj8fvvoKYvzbUaDCFxSA6zokULNKFG9/v9ntKMaYIKeqjJ63mRZ147i4Rdnu//OZuDiYMwemToUqVfy++5AoKJWiwhl4XiJz1u1n/d5Ut+MYY4JY8vr9bNiXxt0XNUN82AX3lMydC6++6jxv2xaqVnUlRkgUFIBbz2tCpchwRs+zVooxpuzenruFBtViuLx9A7ejOBYscC6+T5gA6emuRgmZglKjShQ3dEpg8rLd7Dqa4XYcY0wQ+iXlMItSDjOoazMiwwPg1+eSJXDppVC/Psya5cpproIC4Ij4z6ALmqLAuB+2uh3FGBOERiZvomaVKAZ0Sih5ZV9buRJ69YIaNWD2bKeouCykCkqjGpW5on0DPv5lO0fSs92OY4wJIqt2HWPu+gMMuqAplaMCYKD2ZcucFsmcOdC4sdtpgBArKABDLjqNE9l5vLsgxe0oxpggMjJ5E3HREdxyXhN3g+TmOl9vuQXWroVmgXNjZcgVlBb14ujdui7v/biV45k5bscxxgSBTftTmbF6L7d2aULVmEj3guzYAe3awXffOa9dvmZSWMgVFID7ujXneKYzDo8xxpTkrbmbiYkI547zXRwEcs8e6N4ddu92BnsMQCFZUNo2qsbFLeIZN38rJ7Jz3Y5jjAlgOw6f4OtluxnQqTG1YqPdCXHggDOUyp49MH06nH22OzlKEJIFBeAv3U/ncHq2TcBljDmp0fM2Eya4Nwjk8eNOb64tW5w74M87z50cpRCyBaVDk5qc16wWY+ZtITPHJuAyxvzR/uOZTFq8k2s7NKJeNZcm0KpSBbp0ccbmuvhidzKUUsgWFHBaKftTs/h0yU63oxhjAtDYH7aQm5fvziCQ6emwcyeEh8PIkdC7t/8znKKQLijnnVaLsxtX5+25m8nJy3c7jjEmgBxKy2Liwu1c0b4BTWr5uTdVZib07w8XXeQ8DxIhXVBEhL90b86uoxl8+esut+MYYwLI2B+2kpGTx33dT/fvjrOz4dprnbvfn3nG70PQl4ffCoqIjBeR/SKyqsCy10RknYisEJEvRaR6MZ9N8cw7v0xEFnsz18Ut4mndoCpvJW8i11opxhjgcHo27/+UwuXtGnB6nTj/7Tg3F268Eb75xpnCd+BA/+3bC/zZQnkP6FNo2Uygjaq2AzYAj5/k891U9UxV7ejNUE4r5XRSDp1gyord3ty0MSZIjf1hCxk5edzfw8+tk3/8Az7/HIYPh7vv9u++vcBvBUVV5wGHCy37TlV/uxHkZ6CRv/IU1KtVPVrWi+PN2dZKMSbUHUnP5v0FKVzWtr5/WyfgzP/+3nt+nQfemwLpGsodwPRi3lPgOxFZIiKDvb3jsDBh6CVJbD2YztfLrJViTCh7Z/4WTuTkcX+P5v7ZoSqMHetcfK9ePehOcxUUAENmgog8CeQCE4tZ5QJV3SUidYCZIrLO0+IpaluDgcEA8fHxzJ07t1QZolVpHBfGq9+soPqxjYSH+WcmtrS0tFJndJPl9L5gyBksx9NbOdOylXHzTnBO3XB2r13C7rXlz/a77RfOqUqzMWNo/PHHrF+7lj1XXOHdHfqbqvrtASQCqwotuw34Cahcym08CzxUmnWTkpL0VHy7ao82eXSqTvpl+yl9rjySk5P9tq/ysJze5fzoBb5gOZ7eyvnajHWa+NhUXbfnuFe2V9gfcj77rCqo3nOPan6+T/Z5qoDFWsbf8a6e8hKRPsAjwBWqeqKYdaqISNxvz4FewKqi1i2vnq3q0qZhVUZYjy9jQs7RE9m8tyCFvm3q06KeH66dvPoqPPss3HYbjBgBgTI/fTn4s9vwRzgtkRYislNEBgEjgDic01jLRORtz7oNRGSa56N1gfkishxYBHyjqjN8lJGhPZLYdugEX9h9KcaElHHzt5KWleufayeHDsFrr8H118M770BYIF3OLju/XUNR1QFFLB5XzLq7gb6e51uA9j6M9js9zqhD24bV+M+cjVx1VsPAmDfaGONTR9KzeffHFPq2reef1kmtWrBwISQkOEOrVBD227IQEWHoJc3ZcTiDL5baGF/GhIK3520mPTuXoZck+XQ/dWbOhGHDnJ5dzZpBpIuTdfmAFZQidG9Zh/aNqvGfOZvIzrVrKcZUZPtTM5mwIIX+7RuQVNeHrZPPP+eMl1+GefMgp2LOFmsFpQgiwtCeSew8ksGnS3a4HccY40NvJW8mJ0992zr55hu44QaOn3EGTJkCUVG+25eLrKAU4+KkeDo2qcGbszfafCnGVFC7jmbw4cLt/KljIxJr+2hE4Vmz4JproH17Vrz8MsTG+mY/AcAKSjFEhId7t2Df8Sze/ynF7TjGGB/4z+yNAPyluw97du3fD61awbffkleBiwlYQTmpzs1qcWFSPG/N3UxqZsU852lMqNp6MJ1Pl+zkxs6NaVC9kvd3cMJza92NN8KiRU7PrgrOCkoJHu7VgqMncnjnh61uRzHGeNEbszYQFR7Gvd18MKLwsmVOL64ZnlvmIgJilCufs4JSgraNqtG3bT3e+WELh9Ky3I5jjPGC9XtTmbx8N7edn0h8XLR3N75mDfTs6Vx4P+MM7247wFlBKYW/9UwiIyePUXM3ux3FGOMFr89cT2xUBHdf2My7G964EXr0cO4vmTMHmjTx7vYDnBWUUji9ThxXn92I93/exp5jGW7HMcaUw6/bj/Dt6n3c2bUZ1St7sfvu/v1OMcnNdXp2ne7nybkCgBWUUnqgR3NUlTdnb3I7ijGmjFSVl6avo3ZsFHd2berdjdeuDTfd5BSTVq28u+0gYQWllBJqVubGTo2ZtHgHWw+mux3HGFMGc9cfYNHWwzzQozlVor10oXzfPtiyxRng8aWXoL3fhh4MOFZQTsF93ZsTHRHGa9+uczuKMeYU5eUrr8xYR2KtytzQqbF3NnrokHMBvk8f51RXiLOCcgri46IZfGEzpq3cy9LtR9yOY4w5BV/+uot1e1N5qHcL74wifvQo9OoFGzbAqFEh0zX4ZKygnKK7ujajdmw0L01b+9sMksaYAJeZk8fr362nfaNqXNa2fvk3mJoKffvCypXwxRfOxXhjBeVUVYmO4K89m/NLyhFmrd3vdhxjTCn896dt7D6WyaOXtkS8MTPi0087d79//LFTWAxgBaVMru+YQLP4Krw8fa1NFWxMgDuWkcOI5E1clBRPl9Nqe2ejzz/vjCB89dXe2V4FYQWlDCLCw3isT0s2H0hn0mKbhMuYQDZq7maOZ+bwaJ+W5dtQTo5TSNLTIS4Oevf2TsAKxApKGfVsVZdzEmswfNYG0rOsd4cxgWj30Qze/XEr/ds3oFWDqmXfUF4e3HILPPMMTJ/uvYAVjF8LioiMF5H9IrKqwLKaIjJTRDZ6vtYo5rMDPetsFJGB/ktdNBHh8b5ncCA1ywaONCZAvTrD6eL/cHlaJ/n5MGgQfPIJvPYaXHutl9JVPP5uobwH9Cm07DFgtqo2B2Z7Xv+OiNQEhgGdgU7AsOIKjz+d3bgGfdvWY/S8zexPzXQ7jjGmgGU7jvLVst3c2bUpDcs6PL0q3HsvTJgAzz0HDz3k3ZAVjF8LiqrOAw4XWtwfmOB5PgG4soiP9gZmquphVT0CzOSPhckVD/duSXZuPsNnbnA7ijHGQ1X5x9Q11I6N5p6LyzGm1p498OWX8OijTs8uc1KBcCdOXVXd43m+F6hbxDoNgYKTu+/0LPsDERkMDAaIj49n7ty53ktajB4J4Xy8aActIw7QpGr4KX02LS3NLxnLy3J6XzDkDJbjWTjnor25LN6Wxe2to1j80/yybVQVRIh86y1yatSA77/3es6KJhAKyv+oqopIue4WVNUxwBiAFi1a6MUXX+yNaCd1VuccFr2WzPS9lfno8nNPqZ/73Llz8UfG8rKc3hcMOYPleBbMmZmTx9PDv6dlvTieuqkr4WFluO/khRfgyBHnmok37lspImdFFAi9vPaJSH0Az9ei7hbcBSQUeN3IsywgVKsUyd96teDnLYf5bs0+t+MYE9LeW5DCjsMZPHVZq7IVk9dfh6eecoajt9EwTkkgFJTJwG+9tgYCXxexzrdALxGp4bkY38uzLGAMOCeBpLqxvDhtLVm5eW7HMSYkHUzLYuScTfRoWYcLmpfhJsZRo+DBB52eXOPHOyMIm1Lzd7fhj4CfgBYislNEBgEvAz1FZCNwiec1ItJRRN4BUNXDwPPAL57H3z3LAkZEeBhP92vFtkMneO/HFLfjGBOShs/cwImcPB7vW4apdydMgD//GS6/HCZOtMEey8CvR0xVBxTz1h9GVlPVxcCdBV6PB8b7KJpXdG0eT4+WdfjPnE1cfXYj789VbYwp1rq9x/lo0XZuObcJp9eJPfUNxMbCZZfBpEnOfPDmlFl7zsueuOwMZ2RT60ZsjN+oKsO+Xk3VSpEMvSTp1D584IDz9ZprYMoUiInxfsAQYQXFy06Lj2Vgl0Q++WU7a3YfdzuOMSFh0d48Fm49zEO9WlCjyim0LmbMgKZNna/g1R5docgKig/c37051StHMWzyKpszxRgfO5Gdyyfrs2ndoCoDTmUmxuRkuOoqaN4cOnf2XcAQYgXFB6pVjuTRPi34JeUIX/4aML2bjamQRiZv4nCm8twVrUvfTXjBAufie7NmMHMm1HB9JKcKwQqKj1zXIYEzE6rz4rR1HM/McTuOMRVSysF0xs7bynkNwumYWLN0H9q2DS69FBo0gNmzobaX5kgxVlB8JSxM+Hv/1hxKz+KNmRvdjmNMhfT81DVEhgvXJ53CdZPGjeHJJ51iUq+e78KFICsoPtSuUXUGdGrMhJ9SWLfXLtAb403J6/Yze91+HrikOdVjSvGrbN065yECjzwCCQklf8acEisoPvZwrxbExUTwzNer7QK9MV6SlZvHc1NW0yy+Crd1aVryBzZvhh494PrrnflNjE9YQfGxGlWieKR3SxZtPczk5bvdjmNMhfD23C2kHDrBs5e3JiqihF9j27c7xSQry7kD3oZT8Rk7sn5w/TkJtGtUjRe+WUuqXaA3ply2Hkxn5NxN9GtXnwuT4k++8p49TjE5ehS++w7atPFPyBBlBcUPwsOEv/dvw4G0LIbbBXpjykxVefqrVUSHh/FMv1Ylf2DYMNi717lx8eyzfR8wxFlB8ZMzE6pzU+fGvLdgKyt3HnM7jjFBafLy3czfdJCH+7SgTtVSDJHyxhswdy6ce67PsxkrKH71SJ+W1IqN5vEvV5CbZxcGjTkVxzJyeH7qWto1qsZNnZsUv+Lx43Dffc7XypWhQwf/hQxxVlD8qGpMJM9e3ppVu44z4adtbscxJqi89u06Dqdn8eJVbYu/Iz493RkxePRoWLzYvwGNFRR/69u2Ht1b1uFf361n19EMt+MYExR+3X6EiQu3M7BLIm0aVit6pYwM6N/fGVZl4kTo3t2/IY0VFH8TEZ67ojWqMOxrGzzSmJLk5uXz5JerqBsXw4O9WhS5juTkOLMszp4N774Lf/qTn1MasILiioSalflbzyRmrd3Pkn02XbAxJ/PO/K2s2XOcYZe3Ija66DkBow4fhpUr4e234dZb/ZzQ/MYKiktuPz+RVvWr8sHabBs80phibDmQxvCZG+jdui592hQx7lZ+PqiSVbcurFkDd9/t/5Dmf1wvKCLSQkSWFXgcF5Ghhda5WESOFVjnGbfyektEeBgvXd2WY1nKK9PXuR3HmICTn6889vlKoiPCeL5/G6Tw5Ff5+TB4MNx7L6g6U/gaV7leUFR1vaqeqapnAh2AE8CXRaz6w2/rqerf/ZvSN9onVKdXYgQTF25nwaaDbscxJqBMXLSdRSmHeapfqz/ec6IKDzwA48Y5w8/bTIsBwfWCUkgPYLOqhkyf2qubR5FYqzKPfrGCE9m5bscxJiDsOprBy9PW0rV5ba7r0Oj3b6rCo4/CiBHw4IPw3HPuhDR/IIHUy0hExgNLVXVEoeUXA58DO4HdwEOqurqYbQwGBgPEx8d3mDRpkk8zl1daWhq7sivx8qJMLmkSwU1nRLsdqUhpaWnEBsEphWDJ2a1bN5KTk92OUSI3jqeqMnxJFuuP5PGP8ysRX/n3f/c2ef99mr77Lrv692fjAw+ASNB834MhZ7du3ZaoascyfVhVA+IBRAEHgbpFvFcViPU87wtsLM02k5KSNNAlJyerquqwr1dp4mNTddHWQ+4GKsZvOQNdsOR0fvQCnxvH84ulO7TJo1N1/PwtRa8wdarqkCGqeXn/WxQs3/dgyAks1jL+Hg+kU16X4rRO9hV+Q1WPq2qa5/k0IFJEKtS8nY/0aUGjGpV45LMVZGRbV2ITmg6kZvHclDWc3bg6t56X+Ps3N21yvl52GYwaZcPQB6BA+o4MAD4q6g0RqSeeLh4i0gkn9yE/ZvO5ylERvHJ1O7YeTOf1mevdjmOM36kqj3+xkhPZebx6bbvfD68ydiy0bOncuGgCVkAUFBGpAvQEviiwbIiIDPG8vBZYJSLLgTeBGzxNswqly+m1ubFzY8bN38qSbUfcjmOMX322ZCez1u7jkd4tOL1O3P+/8cEHzv0lPXvCBRe4F9CUKCAKiqqmq2otVT1WYNnbqvq25/kIVW2tqu1V9VxVXeBeWt96/NKW1K9WiYc+XW69vkzI2HnkBH+fsobOTWtyx/kFpvT97DMYOBAuvhi++AKiA7PTinEEREEx/y8uJpJ//ak9KYfSeXHaWrfjGONz+fnKw5+uIF+Vf17XnrDfTnWtXQsDBsB558HkyVCpkrtBTYmsoASgc5vV4q6uzfjg5+0kr9/vdhxjfGrCTyn8tOUQT/drRULNyv//RsuWzsX3b76xu+CDhBWUAPVgryRa1ovjkc9WcDg92+04xvjEpv1pvDx9Hd1b1uH6cxKchfPnw4oVzt3vd94J1YoZrt4EHCsoASo6Ipzh15/JsRM5PPHFShvm3lQ4uXn5PPjpcipFhfPy1W2dsboWLoRLL4U//9m5I94EFSsoAeyM+lV5sFcSM1bv5Yulu9yOY4xXvTl7I8t3HOUfV7Zxxur69Vfo0wfq1oVJk2x8riBkBSXA3dm1GZ2a1mTY5NXsOHzC7TjGeMXCLYcYkbyJa85uRL92DWD1aujVC6pWde41adDA7YimDKygBLjwMOFf17UH4K+fLCM3L9/lRMaUz9ET2Qz9ZBmNa1bmuf6tnYWvvAKRkU4xadLE3YCmzKygBIGEmpV58eq2LN52hDdmbXQ7jjFlpurMcXIwLYv/DDj7/2dgHDPGuRh/+unuBjTlUqqCIiJhIjJaRA6JiHomvKohIvtE5LRSbiNaRLaLSNlGsQxxV7RvwPUdExg5dxM/2twpJkh9uGg7M1bv5eHeLWhLqjP3++HDEBMDzZq5Hc+UU2lbKH2B24HLgfrAAuAJYJqqbi7NBlQ1C3gNeKUMOQ0w7IpWnBYfy9BPlnEgNcvtOMacko37Unl+6hq6Nq/Nnc2rQI8eMGMGbAuZ6Y8qvNIWlNOBPaq6QFX3AhHAncC4U9zfROACEWl9ip8zOANIjrjxLI5n5PDgp8vJz7dulSY4ZObk8ZePfqVKVASv92hIWK+esHMnTJsGZ53ldjzjJSUWFBF5DxgONPac7krBabEo8GOB9Z4Wkb0iUqfAso9EZKmIRAGo6mHPZwZ49V8RQlrWq8ozl7di3oYDjPlhi9txjCmV56asYd3eVN7onUj8Nf1h40aYMsUGe6xgStNCeQD4O85sifWBc4CuwJJCI/6+CGwExgOIyK1Af+BGVS14q/ci4KLyRw9dN3ZqTN+29fjnt+tZut1GJTaB7fMlO/lo0XaGXHQaXetGQXo6fPkldO/udjTjZSUWFM8IwKlAnqruVdUDQBOcqXgLrpcH3IxzSutVYATwoKquK7TJ3UCiF7KHLBHhpavbUa9aDPdNXMqhNLueYgLTur3HefKrlVzQqAoPXXK60yV41SrnbnhT4ZS123AlILPwQlXdhtOieRiYp6qjivhshufzphyqVYrk7Zs7cDA9m/s//pU8u55iAkxqZg73fLCUmhHKuC9fIOLP9zhvRES4G8z4TFkLykGgRjHvXQjkAQkiUtTkBTWBA2XcrymgTcNq/KN/G37cdIh/fWezPJrA8dv9JrsPHGfq/P8QPWeWMwy9qdDKWlB+BVoVXigiVwM3Ad2BasBLRXy2DbC0jPs1hfzpnAQGdErgrbmb+W71XrfjGAPAuz+mMH35Tr75ZTQ1Z82AESPgjjvcjmV8rKwF5VvgDBGp9dsCEWkIjAWeUNV5wC3AX0TkkkKf7QrMKON+TRGGXd6atg2r8eCk5Ww9mO52HBPiFqcc5sVpa3n3l/c4PXka/POfcO+9bscyflCmgqKqK3F6a90AICICvIfTchnuWecH4GVgwm+FR0TOw2m5fFZ4myKSIiIrRWSZiCwu4n0RkTdFZJOIrBCRs8uSvSKKiQxn1M1nEx4uDPnvEps62Lhm99EMhnywhIY1KtHhwcHw0kvw4INuxzJ+UqqCoqr/VNXEQoufA+4XkXB19FTVSwp2JVbVp1W1oaoe8iz6G/CaqmYUs6tuqnqmqhY1PMulQHPPYzBQ1AX/kNWoRmXevOEsNuxP5ZHPVtj8KcbvMrLzGPz+L5yespZ3bu1IbL8+8NhjbscyflTmwSFVdQYwEmhUmvU9F+hX4GnBlEF/4H1P8foZqC4i9cu4rQrpwqR4Hundkqkr9vCfOZvcjmNCiKry6OcruPTjkXw87gGab1zudiTjAgmUv2RFZCtwBOcO/NGqOqbQ+1OBl1V1vuf1bOBRVV1caL3BOC0Y4uPjO0yaNMkf8cssLS2NWC/Ol62qjF2ZzYLdudx7ZjTn1PNOF01v5/SVYMnZrVs3kpOT3Y5RotIez2+2ZFP3vf/y0A8fsLtfPzb87W9+nSArWL7vwZCzW7duS4o5S1QyVQ2IB9DQ87UOsBy4sND7U4ELCryeDXQ82TaTkpI00CUnJ3t9mxnZuXrlyPna4qlpunLnUa9s0xc5fSFYcjo/eoGvNMdz9tq9+o9ud6iC5t9yi2penu+DFRIs3/dgyAks1jL+Hg+Y+VBUdZfn637gS6BToVV2AQkFXjfyLDOFxESGM/qWDtSsHMVd7y9m//E/3INqjFds2p/K6H9N4snk8eRecy0yfjyEBcyvFeNnAfGdF5EqIhL323OgF7Cq0GqTgVs9vb3OBY6p6h4/Rw0adeJiGDuwI0dP5DD4v0vIzMlzO5KpYA6kZnHbu7+wuUlLDr3/IREfTrS74ENcQBQUoC4wX0SW43RH/kZVZ4jIEBEZ4llnGrAF2IRzv8uf3YkaPFo3qMbw69uzbMdRHv5shQ13b7zmRHYu44e+Sr0Nqxh/2znUumUAREW5Hcu4LCD+nFDVLUD7Ipa/XeC5AnZ31Cnq06Y+j/ZpySsz1tGgWgyP9z3D7UgmyOXlK+8+9DoPvvssRy66hPhG97sdyQSIgCgoxreGXNSM3UczGD1vC3WrxnDHBU3djmSClKry4bNvc9fIJznSqh3xk/9wj7IJYVZQQoCI8OwVrdl3PJPnv1lDvWox9G1rt/CYUzft3xO57qWhHG7anHrzkyEuzu1IJoAEyjUU42PhYcKbA87i7MY1GPrJMhZuOVTyh4wpYMry3eSOeYfD9RpRZ8H3UL2625FMgLGCEkJiIsN559aONKpRibveX8yGfaluRzJBInn9fv76yTI+/hpIGKEAABmoSURBVPNz1Fy0gLA68W5HMgHICkqIqVEligm3dyI6Mpxbxy1ix+ETbkcyAW7ltB+o1LsnnarkMvrOLsTUr+t2JBOgrKCEoISalfnvoE5k5OQxYOzP7DlW3FidJmRNnAiJiVzUrRut+l3EGfu3MrJ/ElVjIt1OZgKYFZQQ1bJeVd6/oxNHT+Rw09iFHEi1eemNx8SJMHgwbNuGAOGqxGkONZYvcTuZCXBWUEJY+4TqvHv7Oew5lskt4xZyJD3b7UgmEDz5JJz4/anQsMxMZ7kxJ2EFJcSdk1iTsbd2ZMuBdAa+u4jUzBy3IxmX6fbtRb9R3HJjPKygGC5oXpu3bjqbNbuPc9u7v3DcikrI2rllF/uqFdODq3Fj/4YxQccKigHgklZ1eXPAWSzfcZRb3lnIsRNWVELNzk07ybjgInZUq0t+pUq/f7NyZXjhBXeCmaBhBcX8T9+29Rl1cwfW7kllwNifOZRmF+pDxfatuzne7RKa7N9O/EvPETZ2LDRpgopAkyYwZgzcdJPbMU2As4Jifqdnq7qMHdiRzQfSGDD2Z/an2lwqFV3Ktv0c6daL5rs3sfed/5I44CqneKSk8P2cOZCSYsXElIoVFPMHFyXF8+7t57DjcAY3jLb7VCqyVbuOsa13f9psX8u+UeNofNv1bkcyQcwKiilSl9Nq899BndifmsV1b//EnrR8tyMZL1uw6SA3jPmZTy66noMjx9Bo8EC3I5kgZwXFFKtjYk0+vKszGdl5/GNhBku2HXE7kvGS6b/uYNxTo2hQPYZnXh1C3XsGuR3JVABWUMxJtWtUnS/+3IXYSOHGsT/z3eq9bkcy5fTBj1vIuvkWxn3yDJ+fW5l61WLcjmQqCCsopkRNalXhyXMr0bJ+VYZ8sIQPft7mdiRTBvn5yr9mrCXyniFcueZ7cl54kbjzznE7lqlAXC8oIpIgIskiskZEVovIA0Wsc7GIHBORZZ7HM25kDWVVo4SP7urMxS3q8NRXq3h1xjqboz6InMjO5d6JS6j95MNcv3ImeU89TeQTj7sdy1QwgTBjYy7woKouFZE4YImIzFTVNYXW+0FV+7mQz3hUjopgzC0dePrrVbw1dzMb9qUx/Pr2xNkItAFtz7EM7np/MbE//8jApd+gDz5I+N+fczuWqYBcb6Go6h5VXep5ngqsBRq6m8oUJyI8jBevasuzl7cief1+rn5rASkH092OZYqxbMdR+o/4ka0H0rnrqdth7lzktddAxO1opgIS1cA5bSEiicA8oI2qHi+w/GLgc2AnsBt4SFVXF7ONwcBggPj4+A6TJk3ybehySktLIzY21u0YJSoq55pDeYxclokq3NM+mrbx7jd4g+V4duvWjeTkZJ/u46fduYxflcWQpV/TuntbYjq1PeVtBMvxtJze061btyWq2rFMH1bVgHgAscAS4Ooi3qsKxHqe9wU2lmabSUlJGuiSk5PdjlAqxeXcfihdew//Xps+NlXfnrtJ8/Pz/RuskGA5ns6Pnm9kZOfq41+s0CaPTtXx1w1VBdVBg8q0rWA5npbTe4DFWsbf466f8gIQkUicFshEVf2i8PuqelxV0zzPpwGRIlLbzzFNERJqVubze7rQp009Xpq+jkETFtsYYC5KOZjO1W8t4MOF2xl1YjG3f/oGXHUVjBrldjQTAlwvKCIiwDhgraq+Xsw69TzrISKdcHIf8l9KczJVoiMYeePZPHdFa+ZvPEjfN3/gp8327fG3aSv30O8/89l1NINvamzl0hHPwaWXwkcfQaR1nDC+53pBAc4HbgG6F+gW3FdEhojIEM861wKrRGQ58CZwg6dpZgKEiDCwSyJf3tuFKlER3PjOz7z+3Xpy82zIFl9Lz8rlqa9W8ueJS2leN5Zp919A6+ULoFs3+PxziI52O6IJEa5fRVXV+cBJu5yo6ghghH8SmfJo3aAaU/5yAcMmr+bNOZv4cfMhXru2Hc3iA/tCZLBatPUwD326nB1HTjD4wmY8dElzoqIi4P33ITsbCs9rYowPBUILxVQwVaIj+Od17Xnj+jPZuC+VS//9A2PmbSbPboT0msycPJ6fuobrx/wEwCeDz+OJyJ1EdT4Hdu+GiAhnUixj/MgKivGZK89qyMy/XcSFSfG8OG0dV49awIZ9qW7HCnpLtx+h75s/MG7+Vm7u3ITpD3Sl045V0L8/qEKMjc1l3GEFxfhU3aoxjLmlA/8ZcBY7Dp/gsjd/4N+zNpKZk+d2tKBz9EQ2j3+xkmtGLSAzO48PBnXm+SvbUGXZErjsMkhMhO++g5o13Y5qQpTr11BMxSciXN6+AV1Oq8WzU9YwfNYGPl+6k2f6taLHGXUQu2v7pPLzlS9+3cVL09ZyNCOHQec3ZWjPJGKjI2D5cujTB+rWhVmzoE4dt+OaEGYtFOM3tWKj+c+As/hgUGeiIsK48/3F3P7eL2y002DFWrjlEFe+9SMPfbqcJrUqM+W+C3iqXyunmADUqwfnnw9z5kCDBu6GNSHPWijG7y5oXpvpD3RlwoIU/j1rI73fmMe1HRox9JIkGlS3XkkAm/an8dq36/h29T7qV4vh9T+158ozGxIW5mnN7djhFJO6dWHqVHfDGuNhBcW4IjI8jDu7NuPqsxsxMnkT//1pG18t282t5zZh8IXNqFM1NC8sbz2YzpuzN/L1sl1UigznoV5JDLqgGZWiwgustBUuvNA51TV2rHthjSnECopxVc0qUTzdrxW3dUlk+KwNjP9xK+//vI0B5yRw90WnhUyLZe2e44ydt4Wvlu0iOiKcu7o2Y/CFzagVW+imxJ07oXt3SE+H++5zJ6wxxbCCYgJCQs3KvP6nM7m/e3NGzd3MxIXbmbhwO33b1uf28xM5q3ENtyN6nary46ZDjPlhC/M2HKByVDh3nN+Uuy86jfi4Iu5u37sXevSAw4dh9mxo397/oY05CSsoJqAk1q7CK9e24/5LmjN+/lYm/bKDyct30z6hOjd3bkzftvWpEh3c/22PpGcTd85V9PjX92w5mE7t2Gge7t2Cmzo3pnrlqKI/pApXXw27dsG330LHso0ubowvBfdPpqmwGlavxNP9WvHXnkl8vmQnE35K4eHPVjBs8moua1ufazo04pzEmoSHBUeX46zcPOauP8DkZbuZuXYfNbsPonrlSP55XXv6tatPTGT4yTcgAm+84ZzqOv98/4Q25hRZQTEBLTY6goFdErn1vCYs2XaESYt38M2KPXy6ZCe1Y6Pp06Yul7apzzmJNYmKCKxe8GlZuczfeIBZa/fz7eq9pGbmUqtKFAPOSeClO/uRciCl5I2kpsKUKXDjjdCpk88zG1MeVlBMUBAROibWpGNiTZ69ojVz1u1n+sq9fL5kFx/8vJ3KUeGc16wW9SWH+ntTaV4n9v+72PpJTl4+K3cdY+GWwyzYfJCFWw6TnZdPXEwEPc+oyxVnNuD802sTGR7G36/cVvIGT5yAfv3gxx+dU1xJSb7/RxhTDlZQTNCpHBVBv3YN6NeuARnZefyw8QDzNh5g3oaDzD6czQdr5xEXHcGZjatzZkJ1WtSLo0XdOBJrVyEy3DutmIzsPFIOpbN2z3FW7z7Oql3HWLnrGCeynSFlTq8Ty8AuTejesi4dE2uc+n4zM+HKK+GHH+DDD62YmKBgBcUEtUpR4fRqXY9eresBMGnaHMLrJrFk+xGWbjvCyORN/DbIcUSYUL96DA2rV6JB9UrEx0VTrVIk1SpFEhsdQWR4GOFhQrgI2Xn5ZGTnkZmbx7GMHA6mZnMwLYt9xzPZdugEe49n/i9DTGQYZ9SvynUdGtG5WS06Na1J7cLdfU9FTg786U8wcyaMHw833FCeQ2SM31hBMRVKncphXNyhEdd0aAQ4w7xvPpDGxn1pbNiXyq6jGew6ksFPmw9xKC2b7FJOAFYlKpzacdHEx0bT5fRaNK1VhSa1q9CyXhzNalchwkstH8DpEjx1KowcCbff7r3tGuNjVlBMhRYTGU7rBtVo3aDaH95TVTJz8jmWkUNaVg55+ZCbn09evhIdEU5MZBiVIsOJi4n8/Z3qvtanD6xYAW3a+G+fxniBFRQTskSESlHhnmLh8lAvqvDXvzpzmnTrZsXEBKWA6WcpIn1EZL2IbBKRx4p4P1pEPvG8v1BEEv2f0hgfUIWhQ+Hf/4bkZLfTGFNmAVFQRCQcGAlcCrQCBohIq0KrDQKOqOrpwHDgFf+mNMYHVOGJJ+DNN+GBB+C559xOZEyZBURBAToBm1R1i6pmAx8D/Qut0x+Y4Hn+GdBDbGYmE+xeeAFefhnuvhuGD3fuiDcmSAXKNZSGwI4Cr3cCnYtbR1VzReQYUAs4WHAlERkMDC7w2hd5jSk3EeE9z/PbR49GR492M44x5RYoLRSvUdUxqtpRVTsmJSWhqgH9SE5Odj2D5fT/I9r5v8rAvDwG5uSQHwCZgvl4Wk7vPcojUArKLiChwOtGnmVFriMiEUA14JBf0hnjTe++y0pwRg4OC4OIQDlRYEz5BEpB+QVoLiJNRSQKuAGYXGidycBAz/NrgTla3nJqjL999BEMGsQWgFq13E5jjFcFREFR1VzgPuBbYC0wSVVXi8jfReQKz2rjgFoisgn4G/CHrsXGBLQvv4RbboELL+QqgJjQnObYVFwB09ZW1WnAtELLninwPBO4zt+5jPGK5GS4/npnCPopU8ioWtXtRMZ4XUC0UIyp8Nq2hZtvhmnTIC7O7TTG+IQVFGN8aeVKyMqC2rWdkYOrV3c7kTE+YwXFGF/55Rdnut6//c3tJMb4hRUUY3xh+XLo3dtpmTz+uNtpjPELKyjGeNvatdCzJ1SpAnPmQKNGbicyxi+soBjjTfn5cN11EB7uFJPERLcTGeM3AdNt2JgKISwMPvgAIiOheXO30xjjV9ZCMcYbdu+Gt95ynp95JrRu7W4eY1xgLRRjymv/fujRA3buhMsvh4SEkj9jTAVkBcWY8jh82LkAv20bzJhhxcSENCsoxpTVsWNO1+B162DqVLjwQrcTGeMqu4ZiTFl9/71zJ/xnnzmtFGNCnLVQjCmrK66AzZuhYUO3kxgTEKyFYsypyMqCa66B6dOd11ZMjPkfKyjGlFZODgwYAF984cy2aIz5HSsoxpRGXh4MHOhMkvXvf8Odd7qdyJiAYwXFmJLk58NddznT9778Mtx/v9uJjAlIVlCMKY2oKBg2DB591O0kxgQsV3t5ichrwOVANrAZuF1VjxaxXgqQCuQBuara0Z85TYhShSNHoGZNGDXK7TTGBDy3WygzgTaq2g7YAJxs4ohuqnqmFRPjN8OGwVlnwd69IOI8jDHFcrWgqOp3qprrefkzYBNHmMDw0kvw/PPODYt16ridxpig4HYLpaA7gOnFvKfAdyKyREQG+zGTCUVvvAFPPAE33gijRztD0htjSiSq6tsdiMwC6hXx1pOq+rVnnSeBjsDVWkQgEWmoqrtEpA7OabK/qOq8YvY3GBgMEB8f32HSpEle+pf4RlpaGrGxsW7HKFGo5IyfO5fWzz3Hga5dWTNsGBoe7sV0/69bt24kJyf7ZNveFCrfd38JhpzdunVbUuZLC6rq6gO4DfgJqFzK9Z8FHirNuklJSRrokpOT3Y5QKiGT88gR1cceU83K8kqe4jg/eoEvZL7vfhIMOYHFWsbf56625UWkD/AIcIWqnihmnSoiEvfbc6AXsMp/KU1ImDsXMjKgenXn+klUlNuJjAk6bp8cHgHEATNFZJmIvA0gIg1EZJpnnbrAfBFZDiwCvlHVGe7ENRXS5MnOxfenn3Y7iTFBzdX7UFT19GKW7wb6ep5vAdr7M5cJId99B9dd53QPfuYZt9MYE9TcbqEY457vv4crr4QzznBmW6xa1e1ExgQ1KygmNOXkwB13QGIizJzp3A1vjCkXm2DLhKbISGfa3urVIT7e7TTGVAjWQjGhZdUqePFFZ5yuM86A+vXdTmRMhWEtFBM6NmyASy6B8HAYPBhq13Y7kTEVihUUExq2bIHu3Z25TebOtWJijA9YQTEV344d0KOHc+NicjK0bOl2ImMqJLuGYiq+JUvg+HH49lto187tNMZUWNZCMRVXfr4zUvCVV0K3blCtmtuJjKnQrIViKqYjR6BLF/jqK+e1FRNjfM4Kiql4UlPh0kth6VKIiXE7jTEhw055mQolLDMT+vWDxYvhs8+gTx+3IxkTMqygmIojO5s2Tz/ttEw+/NC5dmKM8Rs75WUqjshI0hMTYdw4uP56t9MYE3KsoJjgl5sLO3eCCJvvvRduu83tRMaEJCsoJrjl58Ptt0OnTk7PLmOMa6ygmOClCkOGwAcfwL33Qo0abicyJqRZQTHBSRWGDoWxY+GJJ+DJJ91OZEzIs4JigtPo0fDmm/DXv8I//uF2GmMMLhcUEXlWRHaJyDLPo28x6/URkfUisklEHvN3ThOAbr4Z/v1v+Ne/QMTtNMYYAqOFMlxVz/Q8phV+U0TCgZHApUArYICItPJ3SBMgPvkE0tIgNhbuv9+KiTEBJBAKSkk6AZtUdYuqZgMfA/1dzmTcMHIk3HADvP6620mMMUUQVXVv5yLPArcBx4HFwIOqeqTQOtcCfVT1Ts/rW4DOqnpfMdscDAz2vGwDrPJJeO+pDRx0O0QpWE7vspzeZTm9p4WqxpXlgz4fekVEZgH1injrSWAU8Dygnq//Au4oz/5UdQwwxrPvxarasTzb87VgyAiW09ssp3dZTu8RkcVl/azPC4qqXlKa9URkLDC1iLd2AQkFXjfyLDPGGBNA3O7lVb/Ay6so+vTUL0BzEWkqIlHADcBkf+QzxhhTem6PNvyqiJyJc8orBbgbQEQaAO+oal9VzRWR+4BvgXBgvKquLuX2x/ggs7cFQ0awnN5mOb3LcnpPmTO6elHeGGNMxREM3YaNMcYEASsoxhhjvKJCFRQReU1E1onIChH5UkSqF7Oea0O5iMh1IrJaRPJFpNjugyKSIiIrPUPSlLkbX1mdQk5Xh8URkZoiMlNENnq+FjnksIjkFRjix2+dOko6PiISLSKfeN5fKCKJ/spWKEdJOW8TkQMFjuGdLmQcLyL7RaTIe8vE8abn37BCRM72d0ZPjpJyXiwixwocy2dcyJggIskissbzc/5AEeuc+vFU1QrzAHoBEZ7nrwCvFLFOOLAZaAZEAcuBVn7MeAbQApgLdDzJeilAbRePZYk53T6WngyvAo95nj9W1Pfc816aC8ewxOMD/Bl42/P8BuCTAM15GzDC39kKZbgQOBtYVcz7fYHpgADnAgsDNOfFwFSXj2V94GzP8zhgQxHf81M+nhWqhaKq36lqruflzzj3rBTm6lAuqrpWVdf7a39lVcqcgTAsTn9gguf5BCCQJpIvzfEpmP8zoIeI3wcoC4TvY4lUdR5w+CSr9AfeV8fPQPVCtyb4RSlyuk5V96jqUs/zVGAt0LDQaqd8PCtUQSnkDpzqWlhDYEeB1zv544EMBAp8JyJLPMPJBKJAOJZ1VXWP5/leoG4x68WIyGIR+VlE/FV0SnN8/reO54+hY0Atv6QrIoNHcd/HazynPj4TkYQi3ndbIPx/LK3zRGS5iEwXkdZuBvGcZj0LWFjorVM+nm7fh3LKTjaUi6p+7VnnSSAXmOjPbL8pTcZSuEBVd4lIHWCmiKzz/OXjNV7K6XMlDN/zP6qqIlJcP/gmnuPZDJgjIitVdbO3s1ZgU4CPVDVLRO7GaVV1dzlTsFqK8/8xTZwpO74CmrsRRERigc+Boap6vLzbC7qCoiUM5SIitwH9gB7qORFYiM+HcikpYym3scvzdb+IfIlzWsKrBcULOf0yLM7JcorIPhGpr6p7PM3x/cVs47fjuUVE5uL8RebrglKa4/PbOjtFJAKoBhzyca7CSsypqgUzvYNz7SrQBMUwTQV/cavqNBF5S0Rqq6pfB40UkUicYjJRVb8oYpVTPp4V6pSXiPQBHgGuUNUTxawW8EO5iEgVEYn77TlOZ4NAHDU5EI7lZGCg5/lA4A8tKxGpISLRnue1gfOBNX7IVprjUzD/tcCcYv4Q8qUScxY6d34Fzjn3QDMZuNXTO+lc4FiB06EBQ0Tq/XadTEQ64fwe9usfEZ79jwPWqmpx80Gc+vF0s6eBtx/AJpxzfss8j996zzQAphVYry9Or4bNOKd3/JnxKpxzkVnAPuDbwhlxetss9zxW+ztjaXO6fSw9+68FzAY2ArOAmp7lHXGG7wHoAqz0HM+VwCA/5vvD8QH+jvNHD0AM8Knn/+4ioJm/j2Epc77k+b+4HEgGWrqQ8SNgD5Dj+b85CBgCDPG8LziT8W32fJ+L7UXpcs77ChzLn4EuLmS8AOc67YoCvy/7lvd42tArxhhjvKJCnfIyxhjjHisoxhhjvMIKijHGGK+wgmKMMcYrrKAYY4zxCisoxhhjvMIKijHGGK+wgmKMMcYrrKAY4wciEi8ie0RkWIFl7UQkU0SuczObMd5id8ob4yci0htn1N6LcIa6WAwsUtXbXQ1mjJdYQTHGj0TkDZzBFb8HugJnqmqau6mM8Q4rKMb4kWfU4+U48190UdXCkxoZE7TsGoox/pWIM8eE4owqbUyFYS0UY/zEM6HRzzjDxC8EhgHtVXW7q8GM8RIrKMb4iYi8DNwItMOZO346znwo3VU1381sxniDnfIyxg9E5CLgQeBWVT2qzl9ytwGtgEfdzGaMt1gLxRhjjFdYC8UYY4xXWEExxhjjFVZQjDHGeIUVFGOMMV5hBcUYY4xXWEExxhjjFVZQjDHGeIUVFGOMMV7xf7fXDepjh3MTAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "dz-BjXzQyc2w", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "49dc0a33-f0b8-4628-a25c-b0598c86546b" }, "source": [ "learning_rate = 0.1\n", "x = tf.Variable(0.)\n", "\n", "for i, epoch in enumerate(range(10)):\n", " with tf.GradientTape() as tape:\n", " z = f(x)\n", " dz_dx = tape.gradient(z, x)\n", " print('Epoch:', i, 'Grad:', dz_dx.numpy())\n", " x.assign_sub(learning_rate * dz_dx)" ], "execution_count": 60, "outputs": [ { "output_type": "stream", "text": [ "Epoch: 0 Grad: 3.0\n", "Epoch: 1 Grad: -4.7683716e-07\n", "Epoch: 2 Grad: 4.7683716e-07\n", "Epoch: 3 Grad: -4.7683716e-07\n", "Epoch: 4 Grad: 4.7683716e-07\n", "Epoch: 5 Grad: -4.7683716e-07\n", "Epoch: 6 Grad: 4.7683716e-07\n", "Epoch: 7 Grad: -4.7683716e-07\n", "Epoch: 8 Grad: 4.7683716e-07\n", "Epoch: 9 Grad: -4.7683716e-07\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "xYJVp-q0y3jz", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "ba282a61-f3a1-47e1-c6a1-58c22866ddfd" }, "source": [ "x.numpy()" ], "execution_count": 61, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "-0.29999995" ] }, "metadata": { "tags": [] }, "execution_count": 61 } ] }, { "cell_type": "code", "metadata": { "id": "LVhhitHUy9EY", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "bd988222-0b35-4997-85a2-2158777465d6" }, "source": [ "f(x.numpy())" ], "execution_count": 62, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "0.5500000000000114" ] }, "metadata": { "tags": [] }, "execution_count": 62 } ] }, { "cell_type": "code", "metadata": { "id": "XtX8DczaR_H8", "colab": { "base_uri": "https://localhost:8080/", "height": 287 }, "outputId": "8007fd14-c997-4ba8-da00-5a8232eea156" }, "source": [ "xs = np.linspace(-2, 2, 200)\r\n", "fs = f(xs)\r\n", "x0 = x.numpy()\r\n", "df_x0 = approximate_diff(f, x0)\r\n", "tangent_x0 = df_x0 * (xs - x0) + f(x0)\r\n", "plt.plot([-2, 2], [0, 0], \"k-\", linewidth=1)\r\n", "plt.plot([0, 0], [-5, 15], \"k-\", linewidth=1)\r\n", "plt.plot(xs, fs)\r\n", "plt.plot(xs, tangent_x0, \"r--\")\r\n", "plt.plot(x0, f(x0), \"ro\")\r\n", "plt.grid(True)\r\n", "plt.xlabel(\"x\", fontsize=14)\r\n", "plt.ylabel(\"f(x)\", fontsize=14, rotation=0)\r\n", "plt.axis([-2, 2, -5, 15]);" ], "execution_count": 69, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEOCAYAAACuOOGFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3hUZfr/8fedTkjooSYQUBKkCwEERYkgIoviujZ0xYKi7qrLrn1d17ZrWXX1ZxcVK7piR8SCmiwiHaTXEEIJndDS2/37Y4b9xpCQkJzMmUnu13XNlTPnPDPzyZnJ3DnteURVMcYYY2oryO0Axhhj6gcrKMYYYxxhBcUYY4wjrKAYY4xxhBUUY4wxjrCCYowxxhE+LSgiMkVE9ojIqjLzHhSRTBFZ5r2NruSxo0RkvYikicg9vkttjDGmOsSX16GIyJlANvCOqvb0znsQyFbVp47zuGBgA3AOsB1YBIxT1TV1HtoYY0y1+HQLRVVnA1k1eOhAIE1V01W1EPgPMNbRcMYYY2olxO0AXreIyHhgMXC7qh4ot7wDsK3M/e3AoIqeSEQmAhMBQiMa9+/csUMdxHVOaWkpQUH+fyjLcjprw4YNJCQkuB2jSoGyPt3OuSdXyS9R4qKCEKm8nds5q2PDhg37VDWmRg9WVZ/egHhgVZn7bYBgPFtL/wSmVPCYi4HXy9y/CnihqteKaHuyHs4rVH+WkpLidoRqsZzO8vzp+b9AWZ9u5kzfm63x98zQJ75eW2XbQFifwGKt4fe766VSVXeraomqlgKv4dm9VV4mEFfmfqx33nGVAu8v2OpITmOMqcjk2emEBgdxzenxbkdxnesFRUTalbn7W2BVBc0WAV1FpLOIhAGXA9Oreu6IYJjy82YKikucCWuMMWXsOZLPJ0u387t+sbSOjnA7jut8fdrwB8A8IFFEtovIBOBfIrJSRFYAycCfvW3bi8hMAFUtBm4BvgXWAtNUdXVVr9c0XNh9uIAvftlRR7+RMaYhe3tuBkUlpdwwtLPbUfyCTw/Kq+q4Cma/UUnbHcDoMvdnAjNP5PUahQgntWvCK7M3cXH/WIKCjnO0zBhjTkB2QTHvztvCqB5t6RIT5XYcv+D6Lq+6duNZXUjfm8OstbvdjmKMqUf+s3Arh/OLmXhmF7ej+I16X1B+06sdcS0a8cp/Nx09Q8wYY2qlsLiUN+ZsZlDnFpzasbnbcfxGvS8oIcFB3DC0C79sPciijPKXtxhjzIn7cvkOdh7K56azTnI7il+p9wUF4JL+cbRoHMYr/93kdhRjTIBTVV6dvYnENtEMS6zZ9X/1VYMoKI3Cgrl6cDw/rtvD+l1H3I5jjAlgKev3sGF3Njee1QU53mXxDVCDKCgA4wd3olFoMK/Otq0UY0zNvZKaTvumEZzfp73bUfxOgykozRuHcfnAOKYv20HmwTy34xhjAtCijCwWZmQxYWgXQoMbzNdntTWoNTLhjM4o8MZPm92OYowJQC+mpNGicRjjBsZV3bgBalAFJbZ5JBf0ac9/Fm3lQE6h23GMMQFkVeYhUtfvZcIZnYkM85eO2v1LgyooADeddRK5hSW8OTfD7SjGmADyYkoa0eEhXDW4k9tR/FaDKyiJbaM5t0cb3vp5M4fzi9yOY4wJAGl7jvDN6l2MH9KJJhGhbsfxWw2uoADcktyVw/mefniMMaYqL6VuIiIkmOtOt04gj6dBFpResU0ZlhjDG3M2k1tY7HYcY4wf25aVyxfLdjBuYEdaRoW7HcevNciCAnDr2SeTlVNoA3AZY47r1dmbCBKsE8hqaLAFpX+nFgzu0pLJs9PJL7IBuIwxx9pzOJ9pi7dzcf9Y2ja1AbSq0mALCni2UvYcKeCjJdvdjmKM8UOv/ZROcUmpdQJZTQ26oAw+qSX9OjbjldRNFJWUuh3HGONH9mcXMHXBVi7o055OLRu7HScgNOiCIiLcenZXMg/m8dkvmW7HMcb4kdd+2kxeUQm3nH2y21EChs8KiohMEZE9IrKqzLwnRWSdiKwQkc9EpFklj83wjju/TEQWO5lrWGIMPdo34aWUNIptK8UYA2TlFPLOvAzO792ek1tHux0nYPhyC+UtYFS5ebOAnqraG9gA3Hucxyeral9VTXIylGcr5WQy9ufy5YodTj61MSZAvfZTOnlFJdw23LZOToTPCoqqzgayys37TlWPXggyH4j1VZ6yRnZvS7e20Tz3g22lGNPQHcgp5J25GfymVzvbOjlB/nQM5Trg60qWKfCdiCwRkYlOv3BQkDBpRAKb9+XwxTLbSjGmIXt9Tjq5RSXcNryr21ECjqiq715MJB6Yoao9y82/D0gCLtIKAolIB1XNFJHWeHaT3erd4qnoNSYCEwFiYmL6T5s2rVrZVJUH5uZTUKI8ekYjgoN8MxJbdnY2UVFRPnmt2rCczkpOTiYlJcXtGFUKlPXpVM7sQuWO/+bSOyaYP/R1/rqTQFifycnJS2p8aEFVfXYD4oFV5eZdA8wDIqv5HA8Cd1SnbUJCgp6Ib1ft1E53z9Bpi7ae0ONqIyUlxWevVRuW01mePz3/Fyjr06mcT36zTuPvmaHrdh525PnKC4T1CSzWGn7Hu7rLS0RGAXcBF6hqbiVtGotI9NFpYCSwqqK2tXVO9zb07NCEF+yML2ManIO5hbw1N4PRPduR2NaOndSEL08b/gDPlkiiiGwXkQnAC0A0MMt7SvAr3rbtRWSm96FtgDkishxYCHylqt/UUUYmDU9gy/5cPrXrUoxpUN6Ys5nsgmI7dlILPht2TFXHVTD7jUra7gBGe6fTgT51GO1Xhp/Sml4dmvL8jxv57akdbNxoYxqAAzmFvPlzBqN7tbWtk1qwb8tyRIRJI7qyLSuPT5daH1/GNASvzN5ETmExk0YkuB0loFlBqcDZ3VrTJ7Ypz/+YRmGxHUsxpj7bcySft+dmMLZPexLa2NZJbVhBqYCIMOmcBLYfyOOjJdvcjmOMqUMvpWyiqERt68QBVlAqMSwhhqROzXnuh402Xoox9VTmwTzeX7CVS5NiiW9lPQrXlhWUSogId56byO7DBbwzL8PtOMaYOvD8DxsBuPVsO7PLCVZQjmNQl5acmRDDS6mbOJJf5HYcY4yDNu/L4aMl27liUEfaN2vkdpx6wQpKFe4cmcjB3CJe/2mz21GMMQ569vsNhAUH8cdk61HYKVZQqtArtimje7Xl9Z/S2Z9d4HYcY4wD1u86wvTlO7jm9HhiosPdjlNvWEGphr+ck0BeUQkvp25yO4oxxgH/nrWeqLAQbjyzi9tR6hUrKNVwcutoLuoXyzvzt7DzUJ7bcYwxtfDL1gN8u3o31w/tQrPIMLfj1CtWUKrpT8O7oqo890Oa21GMMTWkqjz29TpaRYVx/dDObsepd6ygVFNci0iuGNiRaYu3sXlfjttxjDE1kLp+Lws3Z/Gn4V1pHO6zrgwbDCsoJ+CWs7sSHhLEk9+uczuKMeYElZQqT3yzjviWkVw+sKPbceolKygnICY6nIlndmHmyl0s3XrA7TjGmBPw2S+ZrNt1hDvOTbRexOuIrdUTdMPQLrSKCuexmWuPjiBpjPFz+UUl/Pu79fSJbcpverVzO069ZQXlBDUOD+HP53RlUcYBvl+7x+04xphqeHfeFnYcyufu87ohIm7HqbesoNTAZUlxdIlpzONfr7Whgo3xc4fyinghJY2zEmIYclIrt+PUa1ZQaiAkOIh7RnVj094cpi22QbiM8Wcvp27icH4Rd4/q5naUes8KSg2d070NA+Kb88z3G8gpKHY7jjGmAjsO5vHmz5sZ26c93ds3cTtOvefTgiIiU0Rkj4isKjOvhYjMEpGN3p/NK3ns1d42G0Xkat+lrpiIcO/oU9h7pMA6jjTGT/3rG88p/nfa1olP+HoL5S1gVLl59wA/qGpX4Afv/V8RkRbAA8AgYCDwQGWFx5f6dWzO6F5teXX2JvYcyXc7jjGmjGXbDvL5sh1cP7QzHax7ep/waUFR1dlAVrnZY4G3vdNvAxdW8NBzgVmqmqWqB4BZHFuYXHHnud0oLC7lmVkb3I5ijPFSVf4xYw2tosK5eZh1T+8r/tD3QBtV3emd3gW0qaBNB6Ds4O7bvfOOISITgYkAMTExpKamOpe0EsPjgvnPwm10C9lLpybBJ/TY7Oxsn2SsLcvpvEDIGSjrs3zOhbuKWbylgGt7hLF43hz3gpUTKOuzpvyhoPyPqqqI1OpqQVWdDEwGSExM1GHDhjkR7bhOHVTEwidT+HpXJB+cf9oJneeempqKLzLWluV0XiDkDJT1WTZnflEJ9z/zX7q1jeZvVw4lOMh/rjsJlPVZU/5wltduEWkH4P1Z0dWCmUBcmfux3nl+oWmjUP4yMpH56Vl8t2a323GMadDempvBtqw8/vab7n5VTBoCfygo04GjZ21dDXxRQZtvgZEi0tx7MH6kd57fGDcgjoQ2UTw6cy0FxSVuxzGmQdqXXcCLP6YxvFtrzuhqFzH6mq9PG/4AmAckish2EZkAPA6cIyIbgRHe+4hIkoi8DqCqWcAjwCLv7WHvPL8REhzE/WO6s2V/Lm/9nOF2HGMapGdmbSC3qIR7R5/idpQGyafHUFR1XCWLhlfQdjFwfZn7U4ApdRTNEUO7xjC8W2ue/zGNi/rF2ljVxvjQul2H+WDhVq46rRMnt45yO06D5A+7vOqVv/7mFE/PpnYasTE+o6o88MVqmjQKZdKIBLfjNFhWUBx2UkwUVw+J58NFW1mz47DbcYxpEBbuKmHB5izuGJlI88Y2TrxbrKDUgdvO7kqzyDAemL7Kxkwxpo7lFhbz4fpCerRvwjgbidFVVlDqQNPIUO4elciijAN89ovfnN1sTL30YkoaWfnKQxf0sNOEXWYFpY5c0j+OvnHNeHTmOg7nF7kdx5h6KWNfDq/N3szg9sEkxbdwO06DZwWljgQFCQ+P7cH+nAKenbXR7TjG1EuPzFhDaLBwWYIdN/EHVlDqUO/YZowb2JG352WwbpcdoDfGSSnr9vDDuj38aURXmkXYV5k/sHehjt05MpHoiBD+/sVqO0BvjEMKikt46MvVdIlpzDVDOrsdx3hZQaljzRuHcde53Vi4OYvpy3e4HceYeuGV1HQy9ufy4Pk9CAuxrzF/Ye+ED1w2II7esU3551drOWIH6I2plc37cngxNY0xvdtxZkKM23FMGVZQfCA4SHh4bE/2ZhfwjB2gN6bGVJX7P19FeHAQfx/T3e04phwrKD7SN64ZVw7qyFtzN7Ny+yG34xgTkKYv38GctH3cOSqR1k0i3I5jyrGC4kN3jepGy6hw7v1sBcUlpW7HMSagHMor4pEZa+kd25QrB3VyO46pgBUUH2oSEcqD5/dgVeZh3p63xe04xgSUJ79dR1ZOAY/+tpddEe+nrKD42OhebTm7W2ue/m49mQfz3I5jTED4ZesBpi7YytVD4unZoanbcUwlrKD4mIjw0AU9UIUHvrDOI42pSnFJKfd9too20RHcPjLR7TjmOKyguCCuRSR/OSeB79fuYcluGy7YmON5fc5m1uw8zAPndycq3KdjApoTZAXFJdeeHk/3dk14b22hdR5pTCXS92bzzKwNnNujDaN6tnU7jqmC6wVFRBJFZFmZ22ERmVSuzTAROVSmzd/dyuuUkOAgHruoF4cKlCe+Xud2HGP8Tmmpcs8nKwkPCeKRsT0RsQPx/s717UdVXQ/0BRCRYCAT+KyCpj+p6hhfZqtrfeKaMTI+hKkLtvKbXu0YcnIrtyMZ4zemLtzKwows/nVxb7vmJEC4voVSznBgk6o2mHNqL+oaRnzLSO7+dAW5hcVuxzHGL2QezOPxmWsZ2rUVl/SPdTuOqSbxp7OMRGQKsFRVXyg3fxjwCbAd2AHcoaqrK3mOicBEgJiYmP7Tpk2r08y1lZ2dTWZhIx5fmM+ITiFceUq425EqlJ2dTVRUlNsxqhQoOZOTk0lJSXE7RpXcWJ+qyjNLClh/oIR/nN6ImMiq/+8NlPc9EHImJycvUdWkGj1YVf3iBoQB+4A2FSxrAkR5p0cDG6vznAkJCervUlJSVFX1gS9Wafw9M3Th5v3uBqrE0Zz+LlByev70/J8b6/PTpdu0090zdMqc9Go/JlDe90DICSzWGn6P+9Mur/PwbJ3sLr9AVQ+rarZ3eiYQKiL16oDDXaMSiW3eiLs+XkFeoZ1KbBqmvUcKeOjLNfTr2Izxg+PdjmNOkD8VlHHABxUtEJG24j3FQ0QG4sm934fZ6lxkWAhPXNSbzfty+Pes9W7HMcbnVJV7P11JbmEJ/7q4t3WvEoD8oqCISGPgHODTMvNuEpGbvHcvBlaJyHLgOeBy76ZZvTLk5FZcMagjb8zZzJItB9yOY4xPfbxkO9+v3c1d5yZycutot+OYGvCLgqKqOaraUlUPlZn3iqq+4p1+QVV7qGofVT1NVee6l7Zu3XteN9o1bcQdHy23s75Mg7H9QC4Pf7mGQZ1bcN3pNqRvoPKLgmL+T3REKE9f2oeM/Tk8OnOt23GMqXOlpcqdH62gVJWnLulDkO3qClhWUPzQaV1acsPQLrw3fysp6/e4HceYOvX2vAzmpe/n/jHdiWsR6XYcUwtWUPzU7SMT6NY2mrs+XkFWTqHbcYypE2l7snn863Wc3a01lw2IczuOqSUrKH4qPCSYZy7ry6HcIv766Urr5t7UO8Ulpdz+0XIahQXz+EW9rK+uesAKih87pV0Tbh+ZwDerd/Hp0ky34xjjqOd+2MjybQf5x4U9ra+uesIKip+7fmgXBnZuwQPTV7MtK9ftOMY4YkH6fl5ISeN3/WIZ07u923GMQ6yg+LngIOHpS/oA8OcPl1FcUupyImNq52BuIZM+XEbHFpE8NLaH23GMg6ygBIC4FpE8elEvFm85wLPfb3Q7jjE1puoZ42RfdgHPj+tnIzDWM9UqKCISJCKvish+EVHvgFfNRWS3iJxUzecIF5GtIlKzXiwbuAv6tOeypDheTE3j57R9bscxpkbeX7iVb1bv4s5zE+kV29TtOMZh1d1CGQ1cC5wPtAPmAn8FZqrqpuo8gaoWAE8CT9QgpwEeuKA7J8VEMenDZew9UuB2HGNOyMbdR3hkxhqGdm3F9Wd0cTuOqQPVLSgnAztVda6q7sIz0uP1wBsn+HpTgTNExHac1kBkWAgvXHEqh/OKuP2j5ZSW2qnEJjDkF5Vw6we/0DgshKcvtavh66sqC4qIvAU8A3T07u7KwLPFosDPZdrdLyK7RKR1mXkfiMhSEQkDUNUs72PGOfpbNCDd2jbh7+d3Z/aGvUz+Kd3tOMZUy0NfrmHdriM8dUkfWkfbKcL1VXW2UP4EPIxntMR2wABgKLCkXI+/jwIbgSkAIjIeGAtcoaplL/VeCJxV++gN1xUDOzK6V1ue+nY9S7dar8TGv32yZDsfLNzKTWedRHK31lU/wASsKguKtwfgI0CJqu5S1b1AJzxD8ZZtVwL8Hs8urX8BLwC3q+q6ck+5A4h3IHuDJSI8dlFv2jaN4JapS9mfbcdTjH9at+sw932+kkGdW3DHyAS345g6VtPThhsB+eVnquoWPFs0dwKzVfXlCh6b5328qYWmjUJ55ff92ZdTyG3/+YUSO55i/MyR/CJufm8p0RGhPH/FqYQE21UK9V1N3+F9QPNKlp0JlABxIhJewfIWwN4avq4po2eHpvxjbE9+TtvP09/ZKI/Gfxy93mRrVi7PjzvVjps0EDUtKL8A3cvPFJGLgCuBs4GmwGMVPLYnsLSGr2vKuXRAHOMGxvFS6ia+W73L7TjGAPDmzxl8tXInd56byGldWrodx/hITQvKt8ApIvK/T4qIdABeA/6qqrOBq4BbRWREuccOBb6p4euaCjxwfg96dWjK7dOWs3lfjttxTAO3OCOLR2euZcQpbbjxTLvepCGpUUFR1ZV4zta6HEA8/U6/hWfL5Rlvm5+Ax4G3jxYeERmMZ8vl4/LPKSIZIrJSRJaJyOIKlouIPCciaSKyQkT61SR7fRQRGszLv+9HcLBw07tLbOhg45odB/O46b0ldGjeiKcv6WNd0jcw1SooqvqUqsaXm/0QcJuIBKvHOao6ouypxKp6v6p2UNX93ll/AZ5U1bxKXipZVfuqakXds5wHdPXeJgIVHfBvsGKbR/Lc5aeyYc8R7vp4hY2fYnwur7CEie8uJr+olNfHJ9E0MtTtSMbHanzahap+A7wIxFanvfcA/Qq8WzA1MBZ4x1u85gPNRKRdDZ+rXjozIYa7zu3GjBU7ef7HNLfjmAZEVbn7kxWs3nGYZy/rS9c20W5HMi4Qf/lPVkQ2AwfwXIH/qqpOLrd8BvC4qs7x3v8BuFtVF5drNxHPFgwxMTH9p02b5ov4NZadnU1UVJRjz6eqvLaykLk7ivlj33AGtHWmN1enc9aVQMmZnJxMSkqK2zGqVN31+VV6IR9tKOJ3XUM5/6QwHyT7tUB53wMhZ3Jy8pJK9hJVTVX94gZ08P5sDSwHziy3fAZwRpn7PwBJx3vOhIQE9XcpKSmOP2deYbFe+OIcTfzbTF25/aAjz1kXOetCoOT0/On5v+qszx/W7tL4e2boH6cu0dLS0roPVYFAed8DISewWGv4Pe43Vxqpaqb35x7gM2BguSaZQFyZ+7HeeaaciNBgXr2qPy0iw7jhncXsOXzMNajGOCJtzxH+9MEyurdrwpMX20H4hs4vCoqINBaR6KPTwEhgVblm04Hx3rO9TgMOqepOH0cNGK2jI3jt6iQO5hYx8d0l5BeVuB3J1DN7jxRwzZuLCA8NYvL4JBqFBbsdybjMLwoK0AaYIyLL8ZyO/JWqfiMiN4nITd42M4F0IA3P9S5/cCdq4OjRvinPXNaHZdsOcufHK6y7e+OY3MJiJry9iP3ZhUy5ZgAdmllvSsYzronrVDUd6FPB/FfKTCvwR1/mqg9G9WzH3aO68cQ362jfNIJ7R5/idiQT4EpKlds++IVVmYeYfFUSvWObuR3J+Am/KCimbt10Vhd2HMzj1dnptGkSwXVndHY7kglQqspDX67m+7V7eHhsD0Z0b+N2JONHrKA0ACLCgxf0YPfhfB75ag1tm0YwupddwmNO3Os/beadeVu4YWhnxg+OdzuO8TP+cgzF1LHgIOG5cafSr2NzJn24jAXp+6t+kDFlfLl8B/+cuZbRvdpy73m269QcywpKAxIRGszr45OIbd6IG95ZzIbdR9yOZAJEyvo9/PnDZQyIb86/L+1rY8KbCllBaWCaNw7j7WsHEh4azPg3FrItK9ftSMbPLdycxc3vLSGxbTRvXDOAiFA7PdhUzApKAxTXIpJ3Jwwkr6iEca/NZ+ehyvrqNA1dxqESJry1iA7NGvHOdQNpEmEdPprKWUFpoLq1bcI71w3kYG4RV762gL1HbFx682tpe7J5enE+TRqF8u6EQbSMqmgAVmP+jxWUBqxPXDPevHYAOw/lc9UbCziQU+h2JOMnth/I5ao3FiAivHf9INrbhYumGqygNHAD4lvw2vgk0vfmcPWbCzmSX+R2JOOybVm5XD55PjkFxdyRFE7nVo3djmQChBUUwxldW/HSlf1Ys+Mw17y5iMNWVBqso8XkSH4xU68/jY5N7AC8qT4rKAaAEd3b8Ny4U1m+7SBXvb6AQ7lWVBqarfs9xSS7oJip1w+iV2xTtyOZAGMFxfzP6F7tePn3/Vm78wjjXpvP/mw7UN9QbNmfw2WT55FTWMz7NwyiZwcrJubEWUExv3JO9za8dnUSm/ZmM+61+ew5YmOp1Heb9+Vw2avzyS8q4f3rT6NHeysmpmasoJhjnJUQw5vXDmBbVh6Xv2rXqdRnqzIPcfHLcyksKeX9G06je/smbkcyAcwKiqnQkJNa8e6Egew5UsAlr8xjZ3ap25GMw+am7ePyyfOJCA3mo5sGc0o7KyamdqygmEolxbfg/RsGkVdYwj8W5LFkywG3IxmHzFy5k2veXET7ZhF8cvMQToqJcjuSqQesoJjj6h3bjE//MISoUOGK1+bz3epdbkcytfTu/C388f2l9IptyrQbB9O2aYTbkUw9YQXFVKlTy8bcd1ojurVrwk3vLeG9+VvcjmRqoLRUefq79dz/+SqSE1vz3oRBNIsMczuWqUdcLygiEiciKSKyRkRWi8ifKmgzTEQOicgy7+3vbmRtyJqECR/cMIhhia352+er+Nc362yM+gCSW1jMH99fyvM/pnFpUiyvXtWfRmF20aJxlj+M2FgM3K6qS0UkGlgiIrNUdU25dj+p6hgX8hmvyLAQJl/Vn/u/WMVLqZvYsDubZy7rQ7T1QOvXdh7K44Z3FrN6x2HuG30K1w/tjIiNZ2Kc5/oWiqruVNWl3ukjwFqgg7upTGVCgoN49Le9ePD87qSs38NFL80lY1+O27FMJZZtO8jYF35m894cXh+fxA1ndrFiYuqMqPrPbgsRiQdmAz1V9XCZ+cOAT4DtwA7gDlVdXclzTAQmAsTExPSfNm1a3YaupezsbKKi/P8Mm4pyrtlfwovL8lGFm/uE0yvG/Q3eQFmfycnJpKSk1OlrzNtRzJRVBTQNFyb1iyA2+sT/fwyU9Wk5nZOcnLxEVZNq9GBV9YsbEAUsAS6qYFkTIMo7PRrYWJ3nTEhIUH+XkpLidoRqqSzn1v05eu4z/9XO98zQV1LTtLS01LfBygmU9en506sbeYXFeu+nK7TT3TP0kpfn6r4j+TV+rkBZn5bTOcBireH3uOu7vABEJBTPFshUVf20/HJVPayq2d7pmUCoiLTycUxTgbgWkXxy8xBG9WzLY1+vY8Lbi60PMBdl7Mvhopfm8v6Crdx4Vhem3mADYxnfcb2giGeH7hvAWlX9dyVt2nrbISID8eTe77uU5ngah4fw4hX9eOiCHszZuI/Rz/3EvE329vjazJU7GfP8HDIP5vHG1Unce94phAa7/iduGhD3d3rD6cBVwEoRWead91egI4CqvgJcDNwsIsVAHnC5d9PM+AkR4eoh8STFN+fW93/hitfnc2vyydw2vCsh9qVWp3IKinns67W8N38rp3ZsxgtX9KODjbBoXOB6QVHVOcBxTztR1ReAF3yTyNRGj/ZN+fLWM3hg+mqe+zGNnzft58mLezMF6gQAABFMSURBVNPFuvaoEws3Z3HHR8vZdiCXiWd24Y6RiYSFWAE37rBPnnFc4/AQnrqkD89e1peNu49w3v/7icmzN1FiF0I6Jr+ohEdmrOGyyfMA+HDiYP46+hQrJsZVrm+hmPrrwlM7MPiklvzt81U8OnMdX63cxZMX9yahTbTb0QLa0q0HuOOj5aTvzeGq0zpxz3ndaBxuf8rGffbvjKlTbZpEMPmq/jw/7lS2ZeXym+d+4v99v5H8ohK3owWcg7mF3PvpSn738lzyC0t4b8IgHrmwpxUT4zfsk2jqnIhwfp/2DDmpJQ9+uYZnvt/AJ0u38/cx3Rl+Smu7crsKpaXKp79k8tjMtRzMK2LC6Z2ZdE4CUVZIjJ+xT6TxmZZR4Tw/7lQuS4rjwS9Xc/07ixmWGMN9o0+hq+0Gq9CC9P38c+ZaVmw/RL+OzXj3wl42qqLxW1ZQjM+d0bUVX/9pKG/PzeD/fb+Rc5+dzcX9Y5k0IoH2drorAGl7snny23V8u3o37ZpG8O9L+3Bh3w4EBdnWnPFfVlCMK0KDg7h+aBcu6hfLiylpvDtvC58v28H40zox8cwutG7SMAd92rwvh+d+2MgXyzJpFBrMHSMTmHBGF+tq3gQEKyjGVS0ah3H/mO5cMySeZ77fwJSfN/PO/C2MGxDHjWed1GC2WNbuPMxrs9P5fFkm4SHB3DC0CxPP7GLdppiAYgXF+IW4FpH8+9K+3HZ2V15O3cTUBVuZumAro3u149rT4zm1Y3O3IzpOVfk5bT+Tf0pn9oa9RIYFc93pnbnxrJOIibZCYgKPFRTjV+JbNeaJi3tz24iuTJmzmWmLtjF9+Q76xDXj94M6MrpXu4A/TfZATiHRA37L8Kf/S/q+HFpFhXPnuYlcOaijDclrAlpg/2WaeqtDs0bcP6Y7fz4ngU+WbOfteRnc+fEKHpi+mt/0asfv+scyIL4FwQFykLqguITU9XuZvmwHs9bupsXZE2gWGcpTl/RhTO92RITaMRIT+KygGL8WFR7C1UPiGT+4E0u2HGDa4m18tWInHy3ZTquocEb1bMN5PdsxIL6F33U7kl1QzJyNe/l+7R6+Xb2LI/nFtGwcxrgBcTx2/Rgy9ma4HdEYR1lBMQFBREiKb0FSfAsevKAHP67bw9crd/HJkkzem7+VyLBgBndpSTspot2uI3RtHXXip9hOnQr33Qdbt0LHjvDPf8KVV1b74UUlpazMPMSC9CzmbtrHgvQsCktKiY4I4ZxT2nBB3/acfnIrQoODePjCLSe4Bozxf1ZQTMCJDAthTO/2jOndnrzCEn7auJfZG/cye8M+fsgq5L21s4kOD6Fvx2b0jWtGYttoEttEE9+qceXjg0ydChMnQm6u5/6WLZ77UGFRySssIWN/Dmt3Hmb1jsOsyjzEysxD5BZ6upQ5uXUUVw/pxNnd2pAU39zGJTENgl+NKe+0UyMj9ZeBA38989JL4Q9/8HxxjB597IOuucZz27cPLr742OU33wyXXQbbtsFVVx27/Pbb4fzzYf16uPHGY5f/7W8wYgQsWwaTJnHw4EGaNWv2f8sffRSGDIG5c+Gvfz328c8+C337wvffwz/+cezyV1+FxET48kt4+uljl7/7LsTFwYcfwssvH7v844+hVSt46y3Pzet/OWfOhMhIeOklmDbt2Menpnp+PvUUzJjx62WNGsHXX3umH3kEfvjh18tbtoRPPvFM33svzJv36+WxsfDee57pSZM867CshASmXXg5wW0SiLt3EhHpm8gtLP7f4rVtTmLKJbfRoVkj7nz3H7Q6tJeQICEkSGi1+heCCwuP+XVKQsPY1b0vRSXK6m5JvDPiKrbsz+Xx1+8motgzMmWQCJHhwWSePpySv9zOwM4taDXm3GPXTZnPXmrjxgw766xfL/fxZ+8YFXz2fvX5dOmz9z/H+ewdPHiQZkc/Dy599pg82TM9cSJs2PDr5X37wrPPkpqayrDXX4ft23+9fPBgeOwxz/Tvfgf7yw1QN3w43H+/Z/q88yAv79fLx4yBO+7wTA8bxjFO4HtPYmJqPKa8baGYeqV1ZBDD+sdCfAso3EepKnlFJeQVlhDSoQn9OjYn80Aee7MLKD6Uz9F/qFpXUEwAgooK2ZaVS1CQsOtwPiWlypCTW9K5VSSNS8OIDAumUWgwItCzR1vo1c6Xv64xfqVeb6EkJibq+vXr3Y5xXKmpqQyr6D8KP1Mfc6oq+UWlHMoromXPBEK3bzumTUlcHIVpmx2/Ul1ECIS/vfr4vrspEHKKSI23UGzHrmmwRIRGYcG0bRpB6OOPeXanlBUZSfBjj1m3J8ZUk98UFBEZJSLrRSRNRO6pYHm4iHzoXb5AROJ9n9LUW1de6dkH3qkTiHh+Tp58Qmd5GdPQ+cUxFBEJBl4EzgG2A4tEZLqqrinTbAJwQFVPFpHLgSeAy3yf1tRbV15pBcSYWvCXLZSBQJqqpqtqIfAfYGy5NmOBt73THwPDxUZmMsYYv+EXWyhAB6DsEdHtwKDK2qhqsYgcAloC+8o2EpGJwMQy9+sirzG1Zp9NU9/4yxaKY1R1sqomqWpSQkICqurXt5SUFNczWE7f37yfVb+/Bcr6tJzO3WrDXwpKJhBX5n6sd16FbUQkBGgKlLv6xxhjjFv8paAsArqKSGcRCQMuB6aXazMduNo7fTHwo9a2nBpjjHGMXxxDUc8xkVuAb4FgYIqqrhaRh4HFqjodeAN4V0TSgCw8RccYY4yf8IuCAqCqM4GZ5eb9vcx0PnCJr3MZY4ypHn/Z5WWMMSbAWUExxhjjCCsoxhhjHGEFxRhjjCOsoBhjjHGEFRRjjDGOsIJijDHGEVZQjDHGOMIKijHGGEdYQTHGGOMIKyjGGGMcYQXFGGOMI6ygGGOMcYQVFGOMMY6wgmKMMcYRVlCMMcY4wgqKMcYYR1hBMcYY4whXhwAWkSeB84FCYBNwraoerKBdBnAEKAGKVTXJlzmNMcZUze0tlFlAT1XtDWwA7j1O22RV7WvFxBhj/JOrBUVVv1PVYu/d+UCsm3mMMcbUnNtbKGVdB3xdyTIFvhORJSIy0YeZjDHGVJOoat2+gMj3QNsKFt2nql9429wHJAEXaQWBRKSDqmaKSGs8u8luVdXZlbzeRGAiQExMTP9p06Y59JvUjezsbKKiotyOUSXL6azk5GRSUlLcjlGlQFmfltM5ycnJS2p6aKHOC0qVAUSuAW4EhqtqbjXaPwhkq+pTVbVNTEzU9evX1zpjXUpNTWXYsGFux6iS5XSWiOD23151BMr6tJzOEZEaFxRXd3mJyCjgLuCCyoqJiDQWkeij08BIYJXvUhpjjKkOt4+hvABEA7NEZJmIvAIgIu1FZKa3TRtgjogsBxYCX6nqN+7ENcYYUxlXr0NR1ZMrmb8DGO2dTgf6+DKXMcaYE+f2Fooxxph6wgqKMcYYR1hBMcYY4wgrKMYYYxxhBcUYY4wjrKAYY4xxhBUUY4wxjrCCYowxxhFWUIwxxjjCCooxxhhHWEExxhjjCCsoxhhjHGEFxRhjjCOsoBhjjHGEFRRjjDGOsIJijDHGEVZQjDHGOMIKijHGGEdYQTHGGOMIVwuKiDwoIpkissx7G11Ju1Eisl5E0kTkHl/nNMYYU7UQtwMAz6jqU5UtFJFg4EXgHGA7sEhEpqvqGl8FNMYYU7VA2OU1EEhT1XRVLQT+A4x1OZMxxphy/GEL5RYRGQ8sBm5X1QPllncAtpW5vx0YVNmTichEYKL3boGIrHIybB1oBexzO0Q1WE5ntRKRgMhJgKxPLKdTEmv6wDovKCLyPdC2gkX3AS8DjwDq/fk0cF1tXk9VJwOTva+9WFWTavN8dS0QMoLldJrldJbldI6ILK7pY+u8oKjqiOq0E5HXgBkVLMoE4srcj/XOM8YY40fcPsurXZm7vwUq2j21COgqIp1FJAy4HJjui3zGGGOqz+1jKP8Skb54dnllADcCiEh74HVVHa2qxSJyC/AtEAxMUdXV1Xz+yXWQ2WmBkBEsp9Msp7Msp3NqnFFU1ckgxhhjGqhAOG3YGGNMALCCYowxxhH1qqCIyJMisk5EVojIZyLSrJJ2rnXlIiKXiMhqESkVkUpPHxSRDBFZ6e2Spsan8dXUCeR0tVscEWkhIrNEZKP3Z/NK2pWU6eLHZyd1VLV+RCRcRD70Ll8gIvG+ylYuR1U5rxGRvWXW4fUuZJwiInsqu7ZMPJ7z/g4rRKSfrzN6c1SVc5iIHCqzLv/uQsY4EUkRkTXev/M/VdDmxNenqtabGzASCPFOPwE8UUGbYGAT0AUIA5YD3X2Y8RQ8Fw6lAknHaZcBtHJxXVaZ0+116c3wL+Ae7/Q9Fb3n3mXZLqzDKtcP8AfgFe/05cCHfprzGuAFX2crl+FMoB+wqpLlo4GvAQFOAxb4ac5hwAyX12U7oJ93OhrYUMF7fsLrs15toajqd6pa7L07H881K+W52pWLqq5V1fW+er2aqmZOf+gWZyzwtnf6beBCH7/+8VRn/ZTN/zEwXETEhxnBP97HKqnqbCDrOE3GAu+ox3ygWblLE3yiGjldp6o7VXWpd/oIsBZPryRlnfD6rFcFpZzr8FTX8irqyqX8ivQHCnwnIku83cn4I39Yl21Udad3ehfQppJ2ESKyWETmi4ivik511s//2nj/GToEtPRJugoyeFX2Pv7Ou+vjYxGJq2C52/zh81hdg0VkuYh8LSI93Azi3c16KrCg3KITXp9uX4dywo7XlYuqfuFtcx9QDEz1ZbajqpOxGs5Q1UwRaQ3MEpF13v98HONQzjpXRfc9/6OqKiKVnQffybs+uwA/ishKVd3kdNZ67EvgA1UtEJEb8WxVne1ypkC1FM/nMVs8Q3Z8DnR1I4iIRAGfAJNU9XBtny/gCopW0ZWLiFwDjAGGq3dHYDl13pVLVRmr+RyZ3p97ROQzPLslHC0oDuT0Sbc4x8spIrtFpJ2q7vRuju+p5DmOrs90EUnF8x9ZXReU6qyfo222i0gI0BTYX8e5yqsyp6qWzfQ6nmNX/iYgumkq+8WtqjNF5CURaaWqPu00UkRC8RSTqar6aQVNTnh91qtdXiIyCrgLuEBVcytp5vdduYhIYxGJPjqN52QDf+w12R/W5XTgau/01cAxW1Yi0lxEwr3TrYDTAV+Mp1Od9VM2/8XAj5X8I1SXqsxZbt/5BXj2ufub6cB479lJpwGHyuwO9Rsi0vbocTIRGYjne9in/0R4X/8NYK2q/ruSZie+Pt0808DpG5CGZ5/fMu/t6Nkz7YGZZdqNxnNWwyY8u3d8mfG3ePZFFgC7gW/LZ8Rzts1y7221rzNWN6fb69L7+i2BH4CNwPdAC+/8JDzd9wAMAVZ61+dKYIIP8x2zfoCH8fzTAxABfOT97C4Euvh6HVYz52Pez+JyIAXo5kLGD4CdQJH3szkBuAm4ybtc8AzGt8n7Pld6FqXLOW8psy7nA0NcyHgGnuO0K8p8X46u7fq0rleMMcY4ol7t8jLGGOMeKyjGGGMcYQXFGGOMI6ygGGOMcYQVFGOMMY6wgmKMMcYRVlCMMcY4wgqKMcYYR1hBMcYHRCRGRHaKyANl5vUWkXwRucTNbMY4xa6UN8ZHRORcPL32noWnq4vFwEJVvdbVYMY4xAqKMT4kIs/i6Vzxv8BQoK+qZrubyhhnWEExxoe8vR4vxzP+xRBVLT+okTEBy46hGONb8XjGmFA8vUobU2/YFooxPuId0Gg+nm7iFwAPAH1UdaurwYxxiBUUY3xERB4HrgB64xk7/ms846GcraqlbmYzxgm2y8sYHxCRs4DbgfGqelA9/8ldA3QH7nYzmzFOsS0UY4wxjrAtFGOMMY6wgmKMMcYRVlCMMcY4wgqKMcYYR1hBMcYY4wgrKMYYYxxhBcUYY4wjrKAYY4xxxP8Hn1f6uVL37UMAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "CQhi7LNjzDqu", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "66ba45fe-ef40-44ed-93a7-7ebb63fea12f" }, "source": [ "learning_rate = 0.01\n", "x = tf.Variable(0.)\n", "\n", "for i, epoch in enumerate(range(150)):\n", " with tf.GradientTape() as tape:\n", " z = f(x)\n", " dz_dx = tape.gradient(z, x)\n", " print('Epoch:', i, 'Grad:', dz_dx.numpy())\n", " x.assign_sub(learning_rate * dz_dx)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Epoch: 0 Grad: 3.0\n", "Epoch: 1 Grad: 2.7\n", "Epoch: 2 Grad: 2.43\n", "Epoch: 3 Grad: 2.187\n", "Epoch: 4 Grad: 1.9683\n", "Epoch: 5 Grad: 1.7714701\n", "Epoch: 6 Grad: 1.5943232\n", "Epoch: 7 Grad: 1.4348907\n", "Epoch: 8 Grad: 1.2914017\n", "Epoch: 9 Grad: 1.1622615\n", "Epoch: 10 Grad: 1.0460356\n", "Epoch: 11 Grad: 0.94143176\n", "Epoch: 12 Grad: 0.8472886\n", "Epoch: 13 Grad: 0.7625599\n", "Epoch: 14 Grad: 0.6863041\n", "Epoch: 15 Grad: 0.61767364\n", "Epoch: 16 Grad: 0.5559063\n", "Epoch: 17 Grad: 0.5003152\n", "Epoch: 18 Grad: 0.450284\n", "Epoch: 19 Grad: 0.4052558\n", "Epoch: 20 Grad: 0.3647306\n", "Epoch: 21 Grad: 0.32825756\n", "Epoch: 22 Grad: 0.2954316\n", "Epoch: 23 Grad: 0.26588845\n", "Epoch: 24 Grad: 0.2392993\n", "Epoch: 25 Grad: 0.2153697\n", "Epoch: 26 Grad: 0.1938324\n", "Epoch: 27 Grad: 0.17444944\n", "Epoch: 28 Grad: 0.15700436\n", "Epoch: 29 Grad: 0.14130402\n", "Epoch: 30 Grad: 0.12717342\n", "Epoch: 31 Grad: 0.11445618\n", "Epoch: 32 Grad: 0.103010654\n", "Epoch: 33 Grad: 0.09270978\n", "Epoch: 34 Grad: 0.083438635\n", "Epoch: 35 Grad: 0.07509494\n", "Epoch: 36 Grad: 0.06758523\n", "Epoch: 37 Grad: 0.06082678\n", "Epoch: 38 Grad: 0.054744005\n", "Epoch: 39 Grad: 0.049269915\n", "Epoch: 40 Grad: 0.044342756\n", "Epoch: 41 Grad: 0.03990841\n", "Epoch: 42 Grad: 0.03591776\n", "Epoch: 43 Grad: 0.032325745\n", "Epoch: 44 Grad: 0.029093266\n", "Epoch: 45 Grad: 0.026183844\n", "Epoch: 46 Grad: 0.02356553\n", "Epoch: 47 Grad: 0.021209002\n", "Epoch: 48 Grad: 0.01908803\n", "Epoch: 49 Grad: 0.017179012\n", "Epoch: 50 Grad: 0.015461683\n", "Epoch: 51 Grad: 0.013915062\n", "Epoch: 52 Grad: 0.012523651\n", "Epoch: 53 Grad: 0.011271715\n", "Epoch: 54 Grad: 0.010144711\n", "Epoch: 55 Grad: 0.009130001\n", "Epoch: 56 Grad: 0.0082166195\n", "Epoch: 57 Grad: 0.007395029\n", "Epoch: 58 Grad: 0.006655693\n", "Epoch: 59 Grad: 0.00598979\n", "Epoch: 60 Grad: 0.005391121\n", "Epoch: 61 Grad: 0.0048520565\n", "Epoch: 62 Grad: 0.0043668747\n", "Epoch: 63 Grad: 0.0039305687\n", "Epoch: 64 Grad: 0.003537178\n", "Epoch: 65 Grad: 0.0031833649\n", "Epoch: 66 Grad: 0.002865076\n", "Epoch: 67 Grad: 0.0025787354\n", "Epoch: 68 Grad: 0.0023207664\n", "Epoch: 69 Grad: 0.0020887852\n", "Epoch: 70 Grad: 0.0018796921\n", "Epoch: 71 Grad: 0.0016918182\n", "Epoch: 72 Grad: 0.001522541\n", "Epoch: 73 Grad: 0.0013701916\n", "Epoch: 74 Grad: 0.0012333393\n", "Epoch: 75 Grad: 0.0011100769\n", "Epoch: 76 Grad: 0.0009989738\n", "Epoch: 77 Grad: 0.00089907646\n", "Epoch: 78 Grad: 0.00080895424\n", "Epoch: 79 Grad: 0.0007286072\n", "Epoch: 80 Grad: 0.0006556511\n", "Epoch: 81 Grad: 0.0005903244\n", "Epoch: 82 Grad: 0.0005311966\n", "Epoch: 83 Grad: 0.00047779083\n", "Epoch: 84 Grad: 0.00043010712\n", "Epoch: 85 Grad: 0.00038719177\n", "Epoch: 86 Grad: 0.00034880638\n", "Epoch: 87 Grad: 0.00031375885\n", "Epoch: 88 Grad: 0.00028276443\n", "Epoch: 89 Grad: 0.00025439262\n", "Epoch: 90 Grad: 0.00022912025\n", "Epoch: 91 Grad: 0.00020599365\n", "Epoch: 92 Grad: 0.00018548965\n", "Epoch: 93 Grad: 0.000166893\n", "Epoch: 94 Grad: 0.0001502037\n", "Epoch: 95 Grad: 0.00013542175\n", "Epoch: 96 Grad: 0.000121831894\n", "Epoch: 97 Grad: 0.00010967255\n", "Epoch: 98 Grad: 9.846687e-05\n", "Epoch: 99 Grad: 8.869171e-05\n", "Epoch: 100 Grad: 7.9631805e-05\n", "Epoch: 101 Grad: 7.176399e-05\n", "Epoch: 102 Grad: 6.4611435e-05\n", "Epoch: 103 Grad: 5.7935715e-05\n", "Epoch: 104 Grad: 5.2452087e-05\n", "Epoch: 105 Grad: 4.696846e-05\n", "Epoch: 106 Grad: 4.2438507e-05\n", "Epoch: 107 Grad: 3.8146973e-05\n", "Epoch: 108 Grad: 3.385544e-05\n", "Epoch: 109 Grad: 3.0994415e-05\n", "Epoch: 110 Grad: 2.7894974e-05\n", "Epoch: 111 Grad: 2.527237e-05\n", "Epoch: 112 Grad: 2.3126602e-05\n", "Epoch: 113 Grad: 2.0742416e-05\n", "Epoch: 114 Grad: 1.835823e-05\n", "Epoch: 115 Grad: 1.66893e-05\n", "Epoch: 116 Grad: 1.4781952e-05\n", "Epoch: 117 Grad: 1.335144e-05\n", "Epoch: 118 Grad: 1.21593475e-05\n", "Epoch: 119 Grad: 1.0967255e-05\n", "Epoch: 120 Grad: 9.775162e-06\n", "Epoch: 121 Grad: 8.583069e-06\n", "Epoch: 122 Grad: 7.867813e-06\n", "Epoch: 123 Grad: 7.1525574e-06\n", "Epoch: 124 Grad: 6.4373016e-06\n", "Epoch: 125 Grad: 5.722046e-06\n", "Epoch: 126 Grad: 5.2452087e-06\n", "Epoch: 127 Grad: 4.7683716e-06\n", "Epoch: 128 Grad: 4.053116e-06\n", "Epoch: 129 Grad: 3.8146973e-06\n", "Epoch: 130 Grad: 3.33786e-06\n", "Epoch: 131 Grad: 3.0994415e-06\n", "Epoch: 132 Grad: 2.861023e-06\n", "Epoch: 133 Grad: 2.6226044e-06\n", "Epoch: 134 Grad: 2.3841858e-06\n", "Epoch: 135 Grad: 1.66893e-06\n", "Epoch: 136 Grad: 1.4305115e-06\n", "Epoch: 137 Grad: 1.4305115e-06\n", "Epoch: 138 Grad: 1.4305115e-06\n", "Epoch: 139 Grad: 1.4305115e-06\n", "Epoch: 140 Grad: 1.4305115e-06\n", "Epoch: 141 Grad: 1.4305115e-06\n", "Epoch: 142 Grad: 1.4305115e-06\n", "Epoch: 143 Grad: 1.4305115e-06\n", "Epoch: 144 Grad: 1.4305115e-06\n", "Epoch: 145 Grad: 1.4305115e-06\n", "Epoch: 146 Grad: 1.4305115e-06\n", "Epoch: 147 Grad: 1.4305115e-06\n", "Epoch: 148 Grad: 1.4305115e-06\n", "Epoch: 149 Grad: 1.4305115e-06\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "QLT7cNt10BFS", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "0400e92b-6e49-4a22-c1f1-e360a53e0ded" }, "source": [ "x.numpy()" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "-0.29999983" ] }, "metadata": { "tags": [] }, "execution_count": 106 } ] }, { "cell_type": "code", "metadata": { "id": "Lb23k-Bi0W-v", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "4860f908-c6cf-4deb-ae2c-a87860136e1a" }, "source": [ "f(x.numpy())" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "-1.4499999999998607" ] }, "metadata": { "tags": [] }, "execution_count": 107 } ] }, { "cell_type": "code", "metadata": { "id": "z-1TmqE30h_8", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "ecc5bb73-8a91-43b8-d531-8698b313ad3a" }, "source": [ "x = tf.Variable(0.)\n", "optimizer = tf.keras.optimizers.SGD(lr=0.01)\n", "\n", "for iteration in range(150):\n", " with tf.GradientTape() as tape:\n", " z = f(x)\n", " dz_dx = tape.gradient(z, x)\n", " print('Epoch:', i, 'Grad:', dz_dx.numpy())\n", " optimizer.apply_gradients([(dz_dx, x)])" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Epoch: 149 Grad: 3.0\n", "Epoch: 149 Grad: 2.7\n", "Epoch: 149 Grad: 2.43\n", "Epoch: 149 Grad: 2.187\n", "Epoch: 149 Grad: 1.9683\n", "Epoch: 149 Grad: 1.7714701\n", "Epoch: 149 Grad: 1.5943232\n", "Epoch: 149 Grad: 1.4348907\n", "Epoch: 149 Grad: 1.2914017\n", "Epoch: 149 Grad: 1.1622615\n", "Epoch: 149 Grad: 1.0460356\n", "Epoch: 149 Grad: 0.94143176\n", "Epoch: 149 Grad: 0.8472886\n", "Epoch: 149 Grad: 0.7625599\n", "Epoch: 149 Grad: 0.6863041\n", "Epoch: 149 Grad: 0.61767364\n", "Epoch: 149 Grad: 0.5559063\n", "Epoch: 149 Grad: 0.5003152\n", "Epoch: 149 Grad: 0.450284\n", "Epoch: 149 Grad: 0.4052558\n", "Epoch: 149 Grad: 0.3647306\n", "Epoch: 149 Grad: 0.32825756\n", "Epoch: 149 Grad: 0.2954316\n", "Epoch: 149 Grad: 0.26588845\n", "Epoch: 149 Grad: 0.2392993\n", "Epoch: 149 Grad: 0.2153697\n", "Epoch: 149 Grad: 0.1938324\n", "Epoch: 149 Grad: 0.17444944\n", "Epoch: 149 Grad: 0.15700436\n", "Epoch: 149 Grad: 0.14130402\n", "Epoch: 149 Grad: 0.12717342\n", "Epoch: 149 Grad: 0.11445618\n", "Epoch: 149 Grad: 0.103010654\n", "Epoch: 149 Grad: 0.09270978\n", "Epoch: 149 Grad: 0.083438635\n", "Epoch: 149 Grad: 0.07509494\n", "Epoch: 149 Grad: 0.06758523\n", "Epoch: 149 Grad: 0.06082678\n", "Epoch: 149 Grad: 0.054744005\n", "Epoch: 149 Grad: 0.049269915\n", "Epoch: 149 Grad: 0.044342756\n", "Epoch: 149 Grad: 0.03990841\n", "Epoch: 149 Grad: 0.03591776\n", "Epoch: 149 Grad: 0.032325745\n", "Epoch: 149 Grad: 0.029093266\n", "Epoch: 149 Grad: 0.026183844\n", "Epoch: 149 Grad: 0.02356553\n", "Epoch: 149 Grad: 0.021209002\n", "Epoch: 149 Grad: 0.01908803\n", "Epoch: 149 Grad: 0.017179012\n", "Epoch: 149 Grad: 0.015461683\n", "Epoch: 149 Grad: 0.013915062\n", "Epoch: 149 Grad: 0.012523651\n", "Epoch: 149 Grad: 0.011271715\n", "Epoch: 149 Grad: 0.010144711\n", "Epoch: 149 Grad: 0.009130001\n", "Epoch: 149 Grad: 0.0082166195\n", "Epoch: 149 Grad: 0.007395029\n", "Epoch: 149 Grad: 0.006655693\n", "Epoch: 149 Grad: 0.00598979\n", "Epoch: 149 Grad: 0.005391121\n", "Epoch: 149 Grad: 0.0048520565\n", "Epoch: 149 Grad: 0.0043668747\n", "Epoch: 149 Grad: 0.0039305687\n", "Epoch: 149 Grad: 0.003537178\n", "Epoch: 149 Grad: 0.0031833649\n", "Epoch: 149 Grad: 0.002865076\n", "Epoch: 149 Grad: 0.0025787354\n", "Epoch: 149 Grad: 0.0023207664\n", "Epoch: 149 Grad: 0.0020887852\n", "Epoch: 149 Grad: 0.0018796921\n", "Epoch: 149 Grad: 0.0016918182\n", "Epoch: 149 Grad: 0.001522541\n", "Epoch: 149 Grad: 0.0013701916\n", "Epoch: 149 Grad: 0.0012333393\n", "Epoch: 149 Grad: 0.0011100769\n", "Epoch: 149 Grad: 0.0009989738\n", "Epoch: 149 Grad: 0.00089907646\n", "Epoch: 149 Grad: 0.00080895424\n", "Epoch: 149 Grad: 0.0007286072\n", "Epoch: 149 Grad: 0.0006556511\n", "Epoch: 149 Grad: 0.0005903244\n", "Epoch: 149 Grad: 0.0005311966\n", "Epoch: 149 Grad: 0.00047779083\n", "Epoch: 149 Grad: 0.00043010712\n", "Epoch: 149 Grad: 0.00038719177\n", "Epoch: 149 Grad: 0.00034880638\n", "Epoch: 149 Grad: 0.00031375885\n", "Epoch: 149 Grad: 0.00028276443\n", "Epoch: 149 Grad: 0.00025439262\n", "Epoch: 149 Grad: 0.00022912025\n", "Epoch: 149 Grad: 0.00020599365\n", "Epoch: 149 Grad: 0.00018548965\n", "Epoch: 149 Grad: 0.000166893\n", "Epoch: 149 Grad: 0.0001502037\n", "Epoch: 149 Grad: 0.00013542175\n", "Epoch: 149 Grad: 0.000121831894\n", "Epoch: 149 Grad: 0.00010967255\n", "Epoch: 149 Grad: 9.846687e-05\n", "Epoch: 149 Grad: 8.869171e-05\n", "Epoch: 149 Grad: 7.9631805e-05\n", "Epoch: 149 Grad: 7.176399e-05\n", "Epoch: 149 Grad: 6.4611435e-05\n", "Epoch: 149 Grad: 5.7935715e-05\n", "Epoch: 149 Grad: 5.2452087e-05\n", "Epoch: 149 Grad: 4.696846e-05\n", "Epoch: 149 Grad: 4.2438507e-05\n", "Epoch: 149 Grad: 3.8146973e-05\n", "Epoch: 149 Grad: 3.385544e-05\n", "Epoch: 149 Grad: 3.0994415e-05\n", "Epoch: 149 Grad: 2.7894974e-05\n", "Epoch: 149 Grad: 2.527237e-05\n", "Epoch: 149 Grad: 2.3126602e-05\n", "Epoch: 149 Grad: 2.0742416e-05\n", "Epoch: 149 Grad: 1.835823e-05\n", "Epoch: 149 Grad: 1.66893e-05\n", "Epoch: 149 Grad: 1.4781952e-05\n", "Epoch: 149 Grad: 1.335144e-05\n", "Epoch: 149 Grad: 1.21593475e-05\n", "Epoch: 149 Grad: 1.0967255e-05\n", "Epoch: 149 Grad: 9.775162e-06\n", "Epoch: 149 Grad: 8.583069e-06\n", "Epoch: 149 Grad: 7.867813e-06\n", "Epoch: 149 Grad: 7.1525574e-06\n", "Epoch: 149 Grad: 6.4373016e-06\n", "Epoch: 149 Grad: 5.722046e-06\n", "Epoch: 149 Grad: 5.2452087e-06\n", "Epoch: 149 Grad: 4.7683716e-06\n", "Epoch: 149 Grad: 4.053116e-06\n", "Epoch: 149 Grad: 3.8146973e-06\n", "Epoch: 149 Grad: 3.33786e-06\n", "Epoch: 149 Grad: 3.0994415e-06\n", "Epoch: 149 Grad: 2.861023e-06\n", "Epoch: 149 Grad: 2.6226044e-06\n", "Epoch: 149 Grad: 2.3841858e-06\n", "Epoch: 149 Grad: 1.66893e-06\n", "Epoch: 149 Grad: 1.4305115e-06\n", "Epoch: 149 Grad: 1.4305115e-06\n", "Epoch: 149 Grad: 1.4305115e-06\n", "Epoch: 149 Grad: 1.4305115e-06\n", "Epoch: 149 Grad: 1.4305115e-06\n", "Epoch: 149 Grad: 1.4305115e-06\n", "Epoch: 149 Grad: 1.4305115e-06\n", "Epoch: 149 Grad: 1.4305115e-06\n", "Epoch: 149 Grad: 1.4305115e-06\n", "Epoch: 149 Grad: 1.4305115e-06\n", "Epoch: 149 Grad: 1.4305115e-06\n", "Epoch: 149 Grad: 1.4305115e-06\n", "Epoch: 149 Grad: 1.4305115e-06\n", "Epoch: 149 Grad: 1.4305115e-06\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "A6tZXEci1gPo", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "72b8db46-23e0-479c-8207-a4582bc24527" }, "source": [ "x.numpy()" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "-0.29999983" ] }, "metadata": { "tags": [] }, "execution_count": 110 } ] }, { "cell_type": "markdown", "metadata": { "id": "e9YTXSel-rK7" }, "source": [ "## 4.6 Training NN with Custom Loops & SGD" ] }, { "cell_type": "markdown", "metadata": { "id": "nsvBOvBZ-w2k" }, "source": [ "### Define function to extract a batch of data" ] }, { "cell_type": "code", "metadata": { "id": "rwR1V9a81qv1" }, "source": [ "def get_random_batch_data(X, y, batch_size = 32):\n", " idx = np.random.choice(X.index.tolist(), size=batch_size, replace=False)\n", " return X.loc[idx], y.loc[idx]" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "xeK-dohA-1bd" }, "source": [ "### Define training params\n", "\n", "- Epoch: One full pass of the data\n", "- Batch: a batch of data based on batch size\n", "- steps per epoch: total number of batches to be passed in an epoch (total data / batch size)" ] }, { "cell_type": "code", "metadata": { "id": "wA-AXYai6qzw", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "24692e2f-ccda-48dd-a8f9-82b7b8482f33" }, "source": [ "epochs = 30\n", "batch_size = 32\n", "steps_per_epoch = len(X_train) // batch_size\n", "steps_per_epoch" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "451" ] }, "metadata": { "tags": [] }, "execution_count": 112 } ] }, { "cell_type": "markdown", "metadata": { "id": "k-GLyTdP_Br8" }, "source": [ "### Build NN Model" ] }, { "cell_type": "code", "metadata": { "id": "TrP3ETon7xz1" }, "source": [ "optimizer = tf.keras.optimizers.SGD(lr=1e-3)\n", "loss_fn = tf.keras.losses.mean_squared_error\n", "\n", "model = tf.keras.models.Sequential([\n", " tf.keras.layers.Dense(16, activation=\"relu\",\n", " input_shape=(X_train.shape[1],)),\n", " tf.keras.layers.Dense(32, activation=\"relu\"),\n", " tf.keras.layers.Dense(32, activation=\"relu\"),\n", " tf.keras.layers.Dense(1),\n", " softplus_fn\n", "])" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "jlK4gUL99zf4", "colab": { "base_uri": "https://localhost:8080/", "height": 202 }, "outputId": "eb9620c6-d2c7-4bc5-9daf-861360049469" }, "source": [ "X_train_scaled_df = pd.DataFrame(X_train_scaled, columns=X_train.columns, index=X_train.index)\n", "X_train_scaled_df.head()" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
MedIncHouseAgeAveRoomsAveBedrmsPopulationAveOccupLatitudeLongitude
70610.1335060.5093570.181060-0.273850-0.184117-0.010825-0.8056820.780934
14689-0.532218-0.679873-0.422630-0.047868-0.376191-0.089316-1.3394731.245270
173230.170990-0.3627450.073128-0.242600-0.611240-0.044800-0.496645-0.277552
10056-0.402916-1.1555650.175848-0.008560-0.987495-0.0752301.690024-0.706938
15750-0.2992851.857152-0.259598-0.0709930.086015-0.0663570.992350-1.430902
\n", "
" ], "text/plain": [ " MedInc HouseAge AveRooms ... AveOccup Latitude Longitude\n", "7061 0.133506 0.509357 0.181060 ... -0.010825 -0.805682 0.780934\n", "14689 -0.532218 -0.679873 -0.422630 ... -0.089316 -1.339473 1.245270\n", "17323 0.170990 -0.362745 0.073128 ... -0.044800 -0.496645 -0.277552\n", "10056 -0.402916 -1.155565 0.175848 ... -0.075230 1.690024 -0.706938\n", "15750 -0.299285 1.857152 -0.259598 ... -0.066357 0.992350 -1.430902\n", "\n", "[5 rows x 8 columns]" ] }, "metadata": { "tags": [] }, "execution_count": 115 } ] }, { "cell_type": "markdown", "metadata": { "id": "pKKI-4Yq_ENa" }, "source": [ "### Train and Test Model with Custom Training Loop" ] }, { "cell_type": "code", "metadata": { "id": "tI3L-bYz8eCV", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "cfc3242b-13ee-40bf-c305-8480ff3baf04" }, "source": [ "for epoch in range(epochs):\n", "\n", " losses = []\n", "\n", " for step in range(steps_per_epoch):\n", " X_batch, y_batch = get_random_batch_data(X_train_scaled_df, \n", " y_train, \n", " batch_size)\n", " with tf.GradientTape() as tape:\n", " y_pred = model(X_batch.values)\n", "\n", " loss = tf.reduce_mean(loss_fn(y_batch, y_pred))\n", "\n", " losses.append(loss.numpy())\n", "\n", " grads = tape.gradient(loss, model.variables)\n", "\n", " grads_and_vars = zip(grads, model.variables)\n", "\n", " optimizer.apply_gradients(grads_and_vars)\n", " \n", " print(\"Epoch\", epoch, \"avg. mse:\", np.mean(losses))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Epoch 0 avg. mse: 1.6084788\n", "Epoch 1 avg. mse: 1.0368944\n", "Epoch 2 avg. mse: 0.80251783\n", "Epoch 3 avg. mse: 0.7421061\n", "Epoch 4 avg. mse: 0.7103475\n", "Epoch 5 avg. mse: 0.6750144\n", "Epoch 6 avg. mse: 0.664873\n", "Epoch 7 avg. mse: 0.6497538\n", "Epoch 8 avg. mse: 0.62351125\n", "Epoch 9 avg. mse: 0.59369886\n", "Epoch 10 avg. mse: 0.58272564\n", "Epoch 11 avg. mse: 0.555705\n", "Epoch 12 avg. mse: 0.55840635\n", "Epoch 13 avg. mse: 0.5533785\n", "Epoch 14 avg. mse: 0.54142505\n", "Epoch 15 avg. mse: 0.5180923\n", "Epoch 16 avg. mse: 0.5043723\n", "Epoch 17 avg. mse: 0.4995421\n", "Epoch 18 avg. mse: 0.51144665\n", "Epoch 19 avg. mse: 0.49376386\n", "Epoch 20 avg. mse: 0.486438\n", "Epoch 21 avg. mse: 0.4733166\n", "Epoch 22 avg. mse: 0.47285277\n", "Epoch 23 avg. mse: 0.46000323\n", "Epoch 24 avg. mse: 0.47040913\n", "Epoch 25 avg. mse: 0.45553696\n", "Epoch 26 avg. mse: 0.44937178\n", "Epoch 27 avg. mse: 0.4565334\n", "Epoch 28 avg. mse: 0.43180448\n", "Epoch 29 avg. mse: 0.42487937\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "yHJqJhAb9I8t", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "443dab51-caba-473e-b3af-0f2f6ec822d3" }, "source": [ "predictions = model.predict(X_test_scaled)\n", "print('R2:', r2_score(y_test, predictions))\n", "print('MSE:', mean_squared_error(y_test, predictions))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "R2: 0.6641889726463044\n", "MSE: 0.4407654420479039\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "Tl8CIQBKFAEr" }, "source": [ "# 5. The Functional API\n", "\n", "Not all neural network models are simply sequential. Some may have complex topologies. Some may have multiple inputs and/or multiple outputs. For example, a Wide & Deep neural network (see [paper](https://ai.google/research/pubs/pub45413)) connects all or part of the inputs directly to the output layer, as shown on the following diagram:\n", "\n", "![](https://i.imgur.com/B6Y6coM.png)" ] }, { "cell_type": "markdown", "metadata": { "id": "w3W3mQI2SHyf" }, "source": [ "## 5.1: Build, Train & Test a Wide & Deep NN with Functional API\n", "\n", "Use Keras' functional API to implement a Wide & Deep network to tackle the California housing problem.\n", "\n", "**Tips**:\n", "* You need to create a `tf.keras.layers.Input` layer to represent the inputs. Don't forget to specify the input `shape`.\n", "* Create the `Dense` layers, and connect them by using them like functions. For example, `hidden1 = tf.keras.layers.Dense(30, activation=\"relu\")(input)` and `hidden2 = tf.keras.layers.Dense(30, activation=\"relu\")(hidden1)`\n", "* Use the `tf.keras.layers.concatenate()` function to concatenate the input layer and the previous hidden layer's output.\n", "* Create a `tf.keras.models.Model` and specify its `inputs` and `outputs` (e.g., `inputs=[input]`).\n", "* Then use this model just like a `Sequential` model: you need to compile it, display its summary, train it, evaluate it and use it to make predictions." ] }, { "cell_type": "code", "metadata": { "id": "p3_cJjRESDCS" }, "source": [ "input = tf.keras.layers.Input(shape=(X_train.shape[1],))\n", "\n", "x = tf.keras.layers.Dense(16, activation=\"relu\")(input)\n", "x = tf.keras.layers.Dense(32, activation=\"relu\")(x)\n", "x = tf.keras.layers.Dense(32, activation=\"relu\")(x)\n", "\n", "\n", "concat = tf.keras.layers.concatenate([input, x])\n", "\n", "output = tf.keras.layers.Dense(1)(concat)" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "Rr-3DY6ETC84", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "41c1f1e7-5323-46ff-a87f-90515645e16a" }, "source": [ "model = tf.keras.models.Model(inputs=[input], outputs=[output])\n", "\n", "model.compile(loss=\"mean_squared_error\", \n", " optimizer=tf.keras.optimizers.SGD(1e-3))\n", "model.summary()" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Model: \"functional_1\"\n", "__________________________________________________________________________________________________\n", "Layer (type) Output Shape Param # Connected to \n", "==================================================================================================\n", "input_1 (InputLayer) [(None, 8)] 0 \n", "__________________________________________________________________________________________________\n", "dense_23 (Dense) (None, 16) 144 input_1[0][0] \n", "__________________________________________________________________________________________________\n", "dense_24 (Dense) (None, 32) 544 dense_23[0][0] \n", "__________________________________________________________________________________________________\n", "dense_25 (Dense) (None, 32) 1056 dense_24[0][0] \n", "__________________________________________________________________________________________________\n", "concatenate (Concatenate) (None, 40) 0 input_1[0][0] \n", " dense_25[0][0] \n", "__________________________________________________________________________________________________\n", "dense_26 (Dense) (None, 1) 41 concatenate[0][0] \n", "==================================================================================================\n", "Total params: 1,785\n", "Trainable params: 1,785\n", "Non-trainable params: 0\n", "__________________________________________________________________________________________________\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "ZHCJnFrvTKG9", "colab": { "base_uri": "https://localhost:8080/", "height": 644 }, "outputId": "4e66b3ba-4f9a-43b8-d481-464847bdfd14" }, "source": [ "tf.keras.utils.plot_model(model, show_shapes=True)" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdMAAAJzCAYAAABUCvneAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdeVxTV/o/8E8wQAgSFgVEBMoiKBX3DQQd6g4KuCBY9TVY27q0P2J1OhSsI2KxdXSULyrtiFTburCIght2xgUR91ZRi0sBNxAVBQWEKAHO7w+/5GtkkUDCJeF5v175Y27OPee5d1Ifzr1n4THGGAghhBDSYlpcB0AIIYSoO0qmhBBCSCtRMiWEEEJaiZIpIYQQ0kp8rgPQNP7+/lyHQAgh7ZqrqyuWLFnCdRhKRT1TJduzZw8KCgq4DoOQFjt37hzOnTvHdRhqpaCgAHv27OE6DLVw7tw5nD17luswlI5HU2OUi8fjISEhATNmzOA6FEJapO7pSlJSEseRqI/ExEQEBASA/jl9N039fVHPlBBCCGklSqaEEEJIK1EyJYQQQlqJkikhhBDSSpRMCSGEkFaiZEoIUYnDhw/D0NAQBw4c4DqUdmnBggXg8Xiyz+zZs+uVOXr0KEJDQ1FbW4spU6bA2toaAoEAlpaW8PX1xdWrV1vU9q5duzBkyBAYGBjAxsYGc+fOxaNHj2Tf79+/H2vWrEFNTY3ceSkpKXIxd+3atUXtayJKpoQQlaBpIu9mYmKCtLQ03Lp1C3FxcXLfrVixAtHR0QgLC0NtbS1OnTqFXbt2oaSkBJmZmZBIJBg5ciQKCwsVajMhIQGzZs2Cv78/CgoKkJqaioyMDEycOBHV1dUAAB8fHwgEAowePRrPnz+Xnevr64uCggJkZGTAy8ur9TdAg1AyJYSohLe3N0pLSzF58mSuQ4FEIoGbmxvXYdSjp6eHCRMmwNHREbq6urLj3333HeLj45GYmAgDAwMAr1cNcnd3h1AohK2tLSIjI1FaWort27cr1Oa///1vdO/eHV9++SUMDQ3Rv39/LFmyBFlZWTh//rysnFgsRr9+/eDl5SVLsjweD5aWlvDw8EDPnj1bfwM0CCVTQojGi4uLQ1FREddhNEtubi6WL1+OlStXQiAQAAD4fH69x+V2dnYAgLy8PIXqz8/Ph4WFBXg8nuyYlZUVAODevXtyZcPDw5GVlYWoqCiFr6OjoWRKCFG6zMxMWFtbg8fjYdOmTQCAmJgY6OvrQygUIjU1FRMnToRIJEKPHj2we/du2bnR0dEQCAQwMzPDggULYGFhAYFAADc3N7meU3BwMHR0dNCtWzfZsc8++wz6+vrg8Xh4+vQpAGDx4sVYunQp8vLywOPx4ODgAAA4cuQIRCIRIiMj2+KWNFt0dDQYY/Dx8WmynEQiAQCIRCKF6rezs6v3h0Xd+9K6BF3H2NgYo0aNQlRUFD22fwdKpoQQpXN3d8eZM2fkji1atAhffPEFJBIJDAwMkJCQgLy8PNjZ2eGTTz6BVCoF8DpJBgUFobKyEmKxGHfv3sWlS5dQXV2NsWPHIj8/H8DrpPP2sp2bN2/GypUr5Y5FRUVh8uTJsLe3B2MMubm5ACAbXFNbW6uSe9BShw4dgpOTE4RCYZPlLly4AOD1vVZEWFgYHj16hI0bN6K8vBzZ2dmIiorC+PHjMXz48HrlBwwYgAcPHuDKlSsKtdPRUDIlhLQ5Nzc3iEQimJqaIjAwEBUVFbh//75cGT6fj969e0NXVxfOzs6IiYlBeXk5tm3bppQYvL29UVZWhuXLlyulPmWoqKjAnTt3YG9v32iZx48fIz4+HmKxGK6uru/swb5t1KhRCAkJQXBwMEQiEfr06YPy8nJs3bq1wfJ170avXbumUDsdDSVTQgindHR0AEDWM23M4MGDIRQKcfPmzbYIixNFRUVgjDXZK3V1dYVYLIafnx/S0tKgra2tUBvLli3Dli1bcOzYMbx48QK3b9+Gm5sbXF1dZb3+N9XF8vjxY8UupoOhZEoIURu6urp48uQJ12GozMuXLwFAbmTv28zMzHD8+HFs3LgRhoaGCtX/8OFDrFmzBp9++ik++OAD6Ovrw9bWFrGxsSgsLMTatWvrnaOnpycXG2kYJVNCiFqQSqV4/vw5evTowXUoKlOXuN5eLOFNpqamMDIyalH9OTk5qKmpQffu3eWOi0QimJiYIDs7u945VVVVcrGRhvG5DoAQQpojPT0djDG5QTJ8Pv+dj4fViZmZGXg8HkpLSxst05oVper+EHn48KHc8fLycpSUlMimyLypLhZzc/MWt9sRUM+UENIu1dbW4tmzZ6iursbVq1exePFiWFtbIygoSFbGwcEBJSUlSElJgVQqxZMnT+rNlQRerzRUWFiIu3fvory8HFKpFGlpae1uaoxQKISdnR0KCgoa/D43Nxfm5uYICAio911gYCDMzc1x6dKlRuu3tbWFp6cnYmNjkZGRAYlEgvz8fMyfPx8AMG/evHrn1MXi4uLSkkvqMCiZEkKUbtOmTRgyZAgAICQkBL6+voiJicGGDRsAAH379sXt27cRGxuLpUuXAgAmTJiAnJwcWR0vX76Ei4sL9PT04OHhAUdHR5w4cULufeKiRYvg6emJmTNnwsnJCatWrZI9jnxzQM3ChQthZmYGZ2dneHl5oaSkpE3uQ0t4e3sjOztbNo/0TU3N9ayqqkJRURFSU1MbLcPj8ZCUlITAwEDMmzcPxsbGcHZ2xv3795GcnAwPD49651y8eBGWlpbo27dvyy6oo2BEqQCwhIQErsMgpMWmT5/Opk+fzmkM8+fPZyYmJpzGoIiEhASm6D+n8+fPZ5aWlvWO5+TkMD6fz3755ReF6qupqWEeHh4sLi5OofOa8vTpUyYQCNi6devqfScWi1mXLl0UrrM9/L5UgXqmhJB2qalBOJpCIpHg119/RU5Ojmygj4ODAyIiIhAREYEXL140q56amhqkpKSgvLwcgYGBSosvPDwc/fv3R3BwMIDXPePCwkJkZmbKFr8gr1EyJYQQjpSUlMgWuv/oo49kx0NDQ+Hv74/AwMAmByPVSU9PR3JyMtLS0t65clJzrV+/HllZWTh8+LBsLmtqaqpsoftDhw4ppR1NQcmUY5q252NtbS02bNjQqh06zp07h969e0NLSws8Hg/m5ub45ptvlBhl6yUnJ8POzk62r2O3bt0a3I+SKC4sLAzbtm1DaWkpbG1tsWfPHq5DUokffvgBjDHZZ8eOHXLfR0ZGIjg4GN9+++076xo9ejR27twpt05xa6SmpuLVq1dIT0+HsbGx7Lifn59czHXrHxOaGsM5pkGLR+fk5GDu3Lk4ffo0+vXr1+J6hg8fjhs3bmDChAn49ddfcevWrRbPq1OVadOmYdq0aXBwcMDTp0/lNlYmrbN69WqsXr2a6zDahXHjxmHcuHFt3q6vry98fX3bvF11Rj1TjmnKno9XrlzBV199hYULF6J///5Kjox77XU/TEJI+0DJlMi0Zs/Hfv36ITk5GbNmzWpyKTR1pU77YRJC2h4lUw6pw56PytaaPSTV/d6cOnUKzs7OMDQ0hEAggIuLC3799VcAwMcffyx7/2pvb4/Lly8DAObOnQuhUAhDQ0Ps378fwOuRm//4xz9gbW0NPT099O3bFwkJCQCAf/7znxAKhTAwMEBRURGWLl0KS0tL3Lp1q0UxE0KaibtZOZoJCs4zzc/PZwDYxo0bZceWLVvGALBjx46x0tJSVlRUxDw8PJi+vj6rqqqSlZs/fz7T19dn169fZy9fvmTZ2dlsyJAhzMDAgN2/f19WbtasWczc3Fyu3bVr1zIA7MmTJ7Jj06ZNY/b29i25bDnDhg1j/fr1a/C7gwcPMgMDAxYREfHOesaPH88AsGfPnsmOtbd7Y29vzwwNDd95LYwxlpSUxMLDw1lJSQkrLi5mw4cPl5unN23aNNapUyf24MEDufM+/PBDtn//ftn//tvf/sZ0dXXZnj172LNnz1hYWBjT0tJiFy9elLtHYrGYbdy4kU2dOpXduHGjWTEyprnzAFWpJfNMOypN/X1Rz7Qdaw97PiqbsvaQVMd7M336dKxYsQLGxsYwMTGBj48PiouLZbugLFy4EDU1NXLxlZWV4eLFi/Dy8gLwelWgmJgYTJkyBdOmTYORkRG+/vpraGtr17uu7777Dp9//jmSk5PRq1evtrtQQjogGs2rJmjPx8ap672pm7tXtzjBBx98AEdHR/z4448ICwsDj8dDfHw8AgMD0alTJwDArVu3UFlZiT59+sjq0dPTQ7du3ZR6XXv27AGPx1NafR0F3bPmmT59OtchKB0lUw2k6Xs+tgaX9+bQoUNYu3YtsrOzUVZWVi/583g8LFiwAEuWLMGxY8cwZswY/Pzzz9i5c6esTEVFBQDg66+/xtdffy13voWFhdJiHT58OL744gul1afpzp49i6ioKNm7a9K4uvWZNQ0lUw3TEfZ8bKm2vjcZGRn4/fff8cUXX+D+/fuYMmUKpk6dih9//BHdu3fHxo0b8fe//13unKCgIISFhWHr1q2wsrKCSCSCjY2N7HtTU1MAr/9BWrx4scpi79GjB2bMmKGy+jVRVFQU3bNmSEpK4joElaBkqmE6wp6PLdXW9+b333+Hvr4+AODatWuQSqVYtGgR7OzsADT8SNDY2BgBAQGIj4+HgYEBPvnkE7nvraysIBAIkJWVpZKYCSEtQwOQ1Jyq93xUtrbcQ5KreyOVSvH48WOkp6fLkqm1tTUA4OjRo3j58iVycnLkpum8aeHChXj16hUOHjxYbzEPgUCAuXPnYvfu3YiJiUFZWRlqampQUFBQb8NnQkgb4no4saaBAlNjNm7cyLp168YAMKFQyHx8fNjmzZuZUChkAFjPnj1ZXl4e27JlCxOJRAwAs7GxYX/++Sdj7PX0D21tbWZpacn4fD4TiUTMz8+P5eXlybVTXFzMPD09mUAgYLa2tuz//b//x7788ksGgDk4OMimily6dInZ2NgwPT095u7uzh49etTs6z579iwbMWIEs7CwYAAYANatWzfm5ubGTp48KSt3+PBhZmBgwL755ptG6zp37hx7//33mZaWlqyeyMjIdnVvvv/+e2Zvby+71sY+e/fulbUVEhLCTExMmJGREfP392ebNm1iAJi9vb3cdB3GGBswYAALDQ1t8P68evWKhYSEMGtra8bn85mpqSmbNm0ay87OZmvWrGF6enoMALOyslJ4Gy/GNHfqgirR1Jjm09TfF48xDVocth3g8XhISEhok3cnCxYsQFJSEoqLi1XelrpR93vj7e2NTZs2wdbWts3b9vf3B6C577ZUITExEQEBARq11raqaOrvix7zqrmOsOdjS6nTvXnzsfHVq1chEAg4SaSEkJahZEoadPPmTdnydk19lLkRcUcWEhKCnJwc/Pnnn5g7dy5WrVrFdUhExRYsWCD331JDW/gdPXoUoaGhqK2txZQpU2BtbQ2BQABLS0v4+vri6tWrLWp7165dGDJkCAwMDGBjY4O5c+fK7Xy0f/9+rFmzpt4fpCkpKXIxd+3atUXtayJKpmpK1Xs+9urVS27fwsY+8fHxSm1XGdRxP0yhUIhevXphzJgxCA8Ph7OzM9chkTZgYmKCtLQ03Lp1C3FxcXLfrVixAtHR0QgLC0NtbS1OnTqFXbt2oaSkBJmZmZBIJBg5ciQKCwsVajMhIQGzZs2Cv78/CgoKkJqaioyMDEycOBHV1dUAAB8fHwgEAowePRrPnz+Xnevr64uCggJkZGTIVuUi/4ubV7WaCwquzUtIe9MeBohUVlYyV1dXtWmjJQOQ5s+fzywtLRv87ttvv2WOjo5MIpEwxhiTSqVs0qRJcmUuXLjAALDIyEiF2vX09GTdu3dntbW1smN1g+EyMzPlygYHBzNXV1cmlUrr1SMWi+XWlm6u9vD7UgXqmRJC2p222PKuvW6rl5ubi+XLl2PlypUQCAQAXs+HPnDggFy5uvnKeXl5CtWfn58PCwsLuXnOVlZWAFBvWlh4eDiysrIQFRWl8HV0NJRMCSGtxhjD+vXrZRsLGBsbw8/PT2694NZseddW2+q1ZotAZYmOjgZjDD4+Pk2Wk0gkAACRSKRQ/XZ2dvX+iKh7X1qXoOsYGxtj1KhRiIqKopHK70DJlBDSauHh4QgNDcWyZctQVFSEjIwM5Ofnw8PDA48fPwbwOkm8PWVs8+bNWLlypdyxqKgoTJ48Gfb29mCMITc3F8HBwQgKCkJlZSXEYjHu3r2LS5cuobq6GmPHjkV+fn6r2wD+bwR4bW2t8m6Ogg4dOgQnJycIhcImy124cAEA4O7urlD9YWFhePToETZu3Ijy8nJkZ2cjKioK48ePl1sdrM6AAQPw4MEDXLlyRaF2OhpKpoSQVpFIJFi/fj2mTp2K2bNnw9DQEC4uLvjhhx/w9OlTbNmyRWltqXpbPWVtEdhSFRUVuHPnDuzt7Rst8/jxY8THx0MsFsPV1fWdPdi3jRo1CiEhIQgODoZIJEKfPn1QXl6OrVu3Nli+Z8+eAF4viUkaR8mUENIq2dnZePHiBQYPHix3fMiQIdDR0Wl02URlaG/b6rVWUVERGGNN9kpdXV0hFovh5+eHtLQ02VZ+zbVs2TJs2bIFx44dw4sXL3D79m24ubnB1dVV1sN/U10sdU8YSMMomRJCWqVu6kTnzp3rfWdkZITy8nKVtq9JWw6+fPkSwOtraoyZmRmOHz+OjRs3wtDQUKH6Hz58iDVr1uDTTz/FBx98AH19fdja2iI2NhaFhYVYu3ZtvXP09PTkYiMNo2RKCGkVIyMjAGgwaap6yztN23KwLnE1tXqXqamp7J4rKicnBzU1NejevbvccZFIBBMTE2RnZ9c7p6qqSi420jDago0Q0ip9+vRB586d8dtvv8kdP3/+PKqqqjBo0CDZMWVveadpWw6amZmBx+OhtLS00TJvT5FRRN0fHW/vMFReXo6SkhLZFJk31cVibm7e4nY7AuqZEkJaRSAQYOnSpdi7dy927NiBsrIyXLt2DQsXLoSFhQXmz58vK9vaLe9Uva1eW24R2BChUAg7OzsUFBQ0+H1ubi7Mzc0REBBQ77vAwECYm5vj0qVLjdZva2sLT09PxMbGIiMjAxKJBPn5+bL/j+bNm1fvnLpYXFxcWnJJHQYlU0JIq61YsQKrV69GREQEunbtilGjRuG9996T29MVABYtWgRPT0/MnDkTTk5OWLVqlezx4ZsDYBYuXAgzMzM4OzvDy8sLJSUlAF6/t3NxcYGenh48PDzg6OiIEydOyL1jbG0bXPP29kZ2drZsHumbmprrWVVVhaKiIqSmpjZahsfjISkpCYGBgZg3bx6MjY3h7OyM+/fvIzk5GR4eHvXOuXjxIiwtLdG3b9+WXVBHwd3iS5oJtJwgUXPtdbm3+fPnMxMTE67DaJAylxPMyclhfD5f4b1oa2pqmIeHB4uLi1PovKY8ffqUCQQCtm7dunrf0XKC8qhnSghRG+q0rV5zSCQS/Prrr8jJyZEN9HFwcEBERAQiIiLw4sWLZtVTU1ODlJQUlJeXK3Unp/DwcPTv3x/BwcEAXveMCwsLkZmZKVvogrxGyZQQQjhSUlKCCRMmwNHRER999JHseGhoKPz9/REYGNjkYKQ66enpSE5ORlpa2jtXTmqu9evXIysrC4cPH5bNZU1NTYWlpSU8PDxw6NAhpbSjKSiZEkLaPXXcVu9dfvjhB7ntDHfs2CH3fWRkJIKDg/Htt9++s67Ro0dj586dcmsSt0ZqaipevXqF9PR0GBsby477+fnJxVy31jGhqTGEEDWwevVqrF69musw2ty4ceMwbty4Nm/X19cXvr6+bd6uOqOeKSGEENJKlEwJIYSQVqJkSgghhLQSJVNCCCGklWgAkgqcPXuW6xAIabG65eMSExNVUj9jDDweTyV1c6Xuv3lV3TNNUlBQoDEbE7yJx1gT61MRhWnaPxKEEKJs06dPR1JSEtdhKBX1TJWM/jYh77J9+3Z89NFH+OmnnzBnzhyuw2lz27Ztw7x587B27VosXbqU63AIUQpKpoS0oVOnTmHBggX4+uuvO2QiBYC5c+eirKwMX3zxBQwNDfHxxx9zHRIhrUbJlJA2cvv2bUybNg2TJk1CeHg41+FwSiwW4+nTp1iwYAEMDAwa3FKMEHVCyZSQNlBWVgYfHx9YW1vj559/hpYWDaRftWoVJBIJ5syZg86dO8Pb25vrkAhpMRqARIiKSaVSTJw4ETdu3MCFCxdgaWnJdUjtBmMMH330ERITE3Hs2DEMHz6c65AIaRHqmRKiYmKxGOfPn0dmZiYl0rfweDxs3boVz58/x6RJk3D69Gk4OTlxHRYhCqOeKSEqtH79enz55ZfYu3cvLRzeBIlEgtGjR+Px48c4c+YMzM3NuQ6JEIVQMiVERdLS0jB58mSsWbOGpoA0w9OnT+Hm5gZDQ0Okp6dDX1+f65AIaTZKpoSowPXr1+Hm5oapU6fixx9/5DoctZGXlwc3NzcMGTIEKSkp4PPpTRRRD5RMCVGyR48eYdiwYbCxscHRo0eho6PDdUhq5cyZMxgzZgzmzp2LzZs3cx0OIc1C4/MJUaKXL19iypQp4PP52Lt3LyXSFnBzc8OOHTvwww8/4Pvvv+c6HEKahZIpIUrCGMO8efPw559/Ii0tDV27duU6JLU1depUrFixAmKxGCdOnOA6HELeiR7zEqIkK1aswLfffou0tDSMHj2a63DUHmMMM2fOxNGjR3H+/HnY29tzHRIhjaJkSogSJCUlISAgADExMViwYAHX4WgMiUQCDw8PSKVSnDlzhkb4knaLkikhrfTbb79h1KhRWLRoEdauXct1OBrn3r17GDp0KEaMGIHk5GTa5pC0S5RMCWmFe/fuYdiwYRg4cCAOHDiATp06cR2SRjp58iTGjh2L8PBwhIWFcR0OIfVQMiWkhcrLy+Hu7o6amhqcOXMGIpGI65A0WnR0NJYuXYpjx45h5MiRXIdDiBxKpoS0QG1tLfz8/HDx4kWcP38e1tbWXIfUIUybNg1nzpzB5cuX0a1bN67DIUSGpsYQ0gJLlizBf//7X+zbt48SaRuKi4uDnp4e5s6di9raWq7DIUSGkikhCoqLi0N0dDR+/PFH2jKsjRkZGSEhIQHHjx+nwV6kXaHHvIQo4OTJkxg3bhy+/vprLF++nOtwOqx//etf+Oqrr3DixAm4u7tzHQ4hlEwJaa6bN2/Czc0NY8eORXx8PE3R4BBjDL6+vsjKysLVq1dhZGTEdUikg6NkSkgzFBcXw9XVFaampjh27BgEAgHXIXV4xcXF6NOnD8aPH4/t27dzHQ7p4OidKSHvIJVK4e/vD6lUir1791IibSe6dOmC2NhY/PTTT9i3bx/X4ZAOjnqmhLzDp59+ivj4eJw+fRouLi5ch0Pe8te//hX//e9/8ccff8DExITrcEgHRT1TQprw3Xff4ccff8SuXbsokbZT0dHR6NSpE8RiMdehkA6Mkikhjdi3bx+WLVuGDRs2YNKkSVyHQxphaGiIH3/8ETt37kRycjLX4ZAOih7zEtKAy5cvw8PDAzNnzkRsbCzX4ZBm+Oijj5CWloY//vgDXbp04Toc0sFQMiXkLQ8fPsSwYcPQu3dvHDp0CHw+n+uQSDOUlpaid+/e8PX1xffff891OKSDoWRKyBskEgn+8pe/oKysDGfPnqX5i2pmx44d+Otf/4pz585hyJAhXIdDOhBKpoT8L8YYZs6ciaNHj+LcuXNwcHDgOiSiIMYYPD09UVVVhdOnT9PCGqTN0AAkQv5XWFgY9u7di6SkJEqkaorH42HTpk24ePEifv75Z67DIR0I9UwJAfDTTz8hKCgIW7ZswSeffMJ1OKSVPv/8c+zZswc3b96kR/WkTVAyJR1eZmYmxowZg6VLlyIyMpLrcIgSPHv2DE5OTpg9ezbWr1/PdTikA6BkSjqEI0eOIC0tDRs2bICW1v+93bhz5w6GDRsGDw8PJCUlyX1H1FtsbCwWLVqE69evo2fPnlyHQzQcJVPSIcyYMQNJSUmYPHkydu/eDX19fZSVlWHEiBHQ0dFBRkYG9PX1uQ6TKFFNTQ1cXFwwaNAg/PLLL1yHQzQcJVOi8UpKStCtWzdIpVLw+Xz06tULBw8exKJFi5CVlYXz58+jR48eXIdJVCA+Ph6zZ8/GtWvX0Lt3b67DIRqMkinReJs2bcLixYtRU1MDANDW1oaenh6kUikyMzMxcOBAjiMkqlJbW4sBAwagd+/eiI+P5zocosHoBRHReFu2bEFtba3sf0ulUlRUVKC6uho3b97kMDKialpaWli+fDkSExNx5coVrsMhGox6pkSjXbt2DX379m2yzIoVK7BixQqa4K+hGGMYOHAg7OzsaCF8ojLUMyUaLS4uDjo6Ok2WWbVqFebMmYNXr161UVSkLfF4PKxcuRL79u3DpUuXuA6HaCjqmRKNVVVVBTMzM5SWljZapq43OnDgQOzatQuOjo5tFR5pQ4wxDBo0CL169cKuXbu4DodoIOqZEo21f/9+lJWVNfo9n8+HSCTChg0bcOHCBUqkGozH4yE4OBh79uxBYWEh1+EQDUTJlGisrVu3olOnTvWO8/l8aGlpYe7cucjLy4NYLKbFGjqAmTNnwtjYGFu3buU6FKKB6DEv0UiPHj2CpaWl3CheLS0tMMYwYsQIxMTEwMXFhcMICRdCQ0Px008/4d69e9DW1uY6HKJB6M9xopG2b98u19vk8/no2rUrtm/fjoyMDEqkHdTChQtRVFSEffv2cR0K0TDUMyUayd7eHrdv3wafzwePx0NISAhCQ0MhFAq5Do1wbMqUKSgpKcHJkye5DoVoEOqZEo1z+vRp3L59GwAwduxY3LhxA6tWraJESgAAn332GTIyMnD16lWuQyEapF7PNDExEQEBAVzFQwghLZKQkIAZM2a8sxxjDI6Ojpg0aRI2bNjQBpGRjoDf2BcJCQltGQfpIM6ePYuoqCiV/b5qampw7NgxfPDBB+DzG/15q52AgAAsXrwYrq6uXIfSLinSAeDxeJg9ezZiYmKwdu1ajfqdEO402jOlV6lEFej31TI8HugzAGsAACAASURBVK/ZPa+OSNH7c/fuXdjZ2eHgwYPw8vJScXSkI6B3poSQDue9996Dm5sb7XNKlIaSKSGkQ5ozZw5SUlKaXG6SkOaiZEoI6ZACAwPB4/FoJxmiFJRMCSEdkqGhIby8vOhRL1EKSqaEkA5rzpw5OHnyJO7cucN1KETNUTIlaunw4cMwNDTEgQMHuA5FLR09ehShoaGora3FlClTYG1tDYFAAEtLS/j6+rZ4QYNdu3ZhyJAhMDAwgI2NDebOnYtHjx7Jvt+/fz/WrFmDmpoaZV1Kq3h5eaFLly7YvXs316EQNUfJlKglmlrTcitWrEB0dDTCwsJQW1uLU6dOYdeuXSgpKUFmZiYkEglGjhyp8FZlCQkJmDVrFvz9/VFQUIDU1FRkZGRg4sSJqK6uBgD4+PhAIBBg9OjReP78uSouTyHa2toIDAzETz/9xHUoRM1RMiVqydvbG6WlpZg8eTLXoUAikcDNzY3rMJrlu+++Q3x8PBITE2FgYAAAcHV1hbu7O4RCIWxtbREZGYnS0lJs375dobr//e9/o3v37vjyyy9haGiI/v37Y8mSJcjKysL58+dl5cRiMfr16wcvLy9ZkuXSnDlz8Oeff+LixYtch0LUGCVTQlopLi4ORUVFXIfxTrm5uVi+fDlWrlwJgUAA4PVuOm8/KrezswMA5OXlKVR/fn4+LCwswOPxZMesrKwAAPfu3ZMrGx4ejqysLERFRSl8Hco2dOhQ9OrViwYikVahZErUTmZmJqytrcHj8bBp0yYAQExMDPT19SEUCpGamoqJEydCJBKhR48ecu/DoqOjIRAIYGZmhgULFsDCwgICgQBubm5yvafg4GDo6OigW7dusmOfffYZ9PX1wePx8PTpUwDA4sWLsXTpUuTl5YHH48HBwQEAcOTIEYhEIkRGRrbFLWmW6OhoMMbg4+PTZDmJRAIAEIlECtVvZ2dX74+KuveldQm6jrGxMUaNGoWoqKh28ch+1qxZiI+Ph1Qq5ToUoqYomRK14+7ujjNnzsgdW7RoEb744gtIJBIYGBggISEBeXl5sLOzwyeffCL7RzI4OBhBQUGorKyEWCzG3bt3cenSJVRXV2Ps2LHIz88H8DrxvL003ebNm7Fy5Uq5Y1FRUZg8eTLs7e3BGENubi4AyAbYvLk5OdcOHToEJyend+6ec+HCBQCv77MiwsLC8OjRI2zcuBHl5eXIzs5GVFQUxo8fj+HDh9crP2DAADx48ABXrlxRqB1VmDNnDoqLi3HkyBGuQyFqipIp0Thubm4QiUQwNTVFYGAgKioqcP/+fbkyfD4fvXv3hq6uLpydnRETE4Py8nJs27ZNKTF4e3ujrKwMy5cvV0p9rVVRUYE7d+7A3t6+0TKPHz9GfHw8xGIxXF1d39mDfduoUaMQEhKC4OBgiEQi9OnTB+Xl5di6dWuD5Xv27AkAuHbtmkLtqIKNjQ08PDzoUS9pMUqmRKPp6OgAwDsf3w0ePBhCoRA3b95si7DaXFFRERhjTfZKXV1dIRaL4efnh7S0NGhrayvUxrJly7BlyxYcO3YML168wO3bt+Hm5gZXV1dZj/9NdbE8fvxYsYtRkTlz5uDAgQPtYpQxUT+UTAn5X7q6unjy5AnXYajEy5cvAby+xsaYmZnh+PHj2LhxIwwNDRWq/+HDh1izZg0+/fRTfPDBB9DX14etrS1iY2NRWFiItWvX1jtHT09PLjau+fv7Q0tLC0lJSVyHQtQQJVNC8Lrn+vz5c/To0YPrUFSiLnE1tViCqakpjIyMWlR/Tk4Oampq0L17d7njIpEIJiYmyM7OrndOVVWVXGxcE4lEmDx5Mj3qJS1CyZQQAOnp6WCMyQ2U4fP5GjO608zMDDwer8kdUg4cOABLS8sW1V/3R8jDhw/ljpeXl6OkpEQ2ReZNdbGYm5u3qE1VmDNnDjIzM3H79m2uQyFqhpIp6ZBqa2vx7NkzVFdX4+rVq1i8eDGsra0RFBQkK+Pg4ICSkhKkpKRAKpXiyZMn9eZLAoCJiQkKCwtx9+5dlJeXQyqVIi0trV1NjREKhbCzs0NBQUGD3+fm5sLc3BwBAQH1vgsMDIS5uTkuXbrUaP22trbw9PREbGwsMjIyIJFIkJ+fj/nz5wMA5s2bV++culhcXFxackkqMX78eJibm2Pnzp1ch0LUDCVTonY2bdqEIUOGAABCQkLg6+uLmJgYbNiwAQDQt29f3L59G7GxsVi6dCkAYMKECcjJyZHV8fLlS7i4uEBPTw8eHh5wdHTEiRMn5N4pLlq0CJ6enpg5cyacnJywatUq2SPJNwfVLFy4EGZmZnB2doaXlxdKSkra5D4oytvbG9nZ2bJ5pG9qaq5nVVUVioqKkJqa2mgZHo+HpKQkBAYGYt68eTA2NoazszPu37+P5ORkeHh41Dvn4sWLsLS0RN++fVt2QSrA5/MREBCAn3/+uV3MfyVqhL0lISGBNXCYEKVoD7+v+fPnMxMTE05jUBQAlpCQ0Ko6cnJyGJ/PZ7/88otC59XU1DAPDw8WFxfXqvbf9PTpUyYQCNi6deuUUp8y7k+d3377jQFg58+fV0p9pGOgninpkNrLriVtycHBAREREYiIiMCLFy+adU5NTQ1SUlJQXl6OwMBApcUSHh6O/v37Izg4WGl1KsugQYPQs2dPJCQkcB0KUSMqSaYff/wxDAwMwOPxkJWVpYomVC4iIgLOzs4QiUTQ1dWFg4MD/v73v9f7R2jNmjXo1asX9PT0oK+vj169emH58uUoKytTuM3k5GTY2dmBx+PJfXR0dGBmZoa//OUvWLt2LZ49e6asyyQdTGhoKPz9/REYGNjkYKQ66enpSE5ORlpa2jtXTmqu9evXIysrC4cPH1Z4LmtbmTFjBhITE9vVClaknXu7q6qsx3C7d+9mANjly5dbXRcXRo0axTZv3syKi4tZWVkZS0hIYNra2mzChAly5by9vdm6detYUVERKy8vZ4mJiUxbW5uNHTu2xW3b29szQ0NDxhhjtbW17NmzZ+zEiRMsKCiI8Xg8ZmFhwS5evNiq6+MK1495Q0NDmY6ODgPA3nvvPZaUlMRZLIqAEh9jMsbYr7/+ykJCQpRWX3OlpKSw1atXs+rqaqXWq+z7c/XqVQaAZWZmKq1OotkomTbC29u73n/wM2bMYADY/fv3ZcemTJnCJBKJXDl/f38GgBUWFrao7TeT6duSkpKYlpYWMzMzY8+fP29R/VziOpmqK2UnC02jivvj7OzMgoODlVon0Vwqe2f65jZM6ujgwYPo1KmT3LGuXbsCACorK2XH9u7dK9vOqk7dXL3mvpdSxPTp0xEUFISioiL88MMPSq+fEPJa3aPejvh+nShOKcmUMYa1a9fCyckJurq6MDQ0xJdfflmvXE1NDf7xj3/A2toaenp66Nu3r+wlf3O30AKAkydPYujQoRAKhRCJRHBxcZG9o2yqjdZ68OAB9PT0YGtr22S5nJwcGBkZwcbGRnZMmVty1c2FTEtLkx1T93tLSHsTGBiIR48eITMzk+tQiDp4u6vaksdwy5YtYzwej/3rX/9iz549Y5WVlWzz5s31HvP+7W9/Y7q6umzPnj3s2bNnLCwsjGlpacne/y1btowBYMeOHWOlpaWsqKiIeXh4MH19fVZVVcUYY+zFixdMJBKxNWvWMIlEwh49esSmTp3Knjx50qw2WqqiooIZGBg0+tinqqqKFRQUsI0bNzJdXd160w8OHjzIDAwMWERExDvbauoxL2OMlZWVMQDMyspKdkxd7i095m0Z0GPeJqnq/ri4uLCFCxcqvV6ieVqdTCsrK5lQKKw34Obtd6YSiYQJhUIWGBgod66uri5btGgRY+z//sF/8x1kXVLOzc1ljDH2xx9/MADs4MGD9WJpThsttWzZMubo6MjKysoa/N7c3JwBYF26dGH/8z//I0tQLfGuZMoYYzwejxkZGTHG1OveUjJtGUqmTVPV/fnmm2+Yqakpk0qlSq+baBZ+a3u2ubm5qKysxOjRo5ssd+vWLVRWVqJPnz6yY3p6eujWrVuT2169vYWWnZ0dzMzMMHv2bIjFYgQFBeG9995rVRvvsnfvXiQmJuI///kPDAwMGiyTn5+P58+f4/LlywgNDcWWLVtw/PhxmJmZtbjdxlRUVIAxBpFIBEA9721iYqLC53R0Z8+e5TqEDicwMBBff/010tPTMWbMGK7DIe3Z29lV0Z7D4cOHGYB6q6O83TM9ffo0A9DgZ/jw4YyxhntPsbGxDAC7ceOG7Ngff/zBJk2axPh8PuPxeCwgIIBVVlY2qw1F7d69mw0ZMoQ9ePCg2ef8+eefDAATi8UtavNdPdNLly4xAGzcuHGMMfW6t3W/L/rQR9kfVfXcBw4cyD755BOV1E00R6sHINWNZH316lWT5UxNTQEAGzZsAHv9eFn2UfQv7vfffx8HDhxAYWEhQkJCkJCQgHXr1im1DQDYuHEjduzYgePHj9fbWqopDg4O6NSpU4PbTinDkSNHAAATJ04EoJ739u066NP0BwASEhI4j6O9flQpICAAe/bskW0ZR0hDWp1M+/TpAy0tLZw8ebLJclZWVhAIBK1eEamwsBDXr18H8DqJfPvttxg4cCCuX7+utDYYYwgJCcG1a9eQkpKCzp07N1iuuLgYH374Yb3jdXs7NrTtVGs9evQIGzZsQI8ePfDRRx8BUK97S4i6CQwMxPPnz3H8+HGuQyHtWKuTqampKaZNm4Y9e/YgLi4OZWVluHr1KrZs2SJXTiAQYO7cudi9ezdiYmJQVlaGmpoaFBQU1NsDsSmFhYVYsGABbt68iaqqKly+fBn37t3D8OHDldbG9evX8c9//hOxsbHQ1taut7zfunXrAAD6+vr4z3/+g+PHj6OsrAxSqRSXL1/GX//6V+jr62PJkiWyOhXdkosxhhcvXqC2thaMMTx58gQJCQkYMWIEOnXqhJSUFNk7U3W6t4SoG2trawwdOpSmgZGmsbe0ZLRleXk5+/jjj1mXLl1Y586dmbu7O/vHP/7BALAePXqwK1euMMYYe/XqFQsJCWHW1taMz+czU1NTNm3aNJadnc02b97MhEIhA8B69uzJ8vLy2JYtW5hIJGIAmI2NDfvzzz/Z3bt3mZubGzM2NmadOnVi3bt3Z8uWLZOtVtRUG8117dq1Jt/NrF27VlbWx8eH2drass6dOzNdXV1mb2/PAgMD2bVr1+TqPHz4MDMwMGDffPNNo+3u37+f9e3blwmFQqajo8O0tLQYANnI3aFDh7KIiAhWXFxc71x1ubc0mrdloMJ3gppA1ffnX//6FzMyMmIvX75UWRtEvfEYk3/hkJiYiICAAJW/hyAdE/2+WobH4yEhIQEzZszgOpR2SdX3p6CgADY2NkhNTcWkSZNU0gZRb7QFGyGEvEOPHj3g6upKj3pJozpMMr1582a9d58NfZS5ZyMhRHMEBAQgNTUVEomE61BIO9RhkmmvXr2aNcQ+Pj6e61AJUaqjR48iNDQUtbW1mDJlCqytrSEQCGBpaQlfX19cvXq1xXXX1tZiw4YNcHNza7SMVCrF6tWr4eDgAB0dHRgZGaFPnz64e/cuAGD//v1Ys2ZNu19Q3t/fH5WVlbKpaYS8qcMkU0I6ohUrViA6OhphYWGora3FqVOnsGvXLpSUlCAzMxMSiQQjR45EYWGhwnXn5ORg5MiRWLJkidxOSm8LCAjAzz//jJ07d6KyshI3btyAvb29bFclHx8fCAQCjB49Gs+fP2/xtapat27d4OHhQY96SYMomZIORyKRNNmTUpc23uW7775DfHw8EhMTZctgurq6wt3dHUKhELa2toiMjERpaSm2b9+uUN1XrlzBV199hYULF6J///6NlouPj0dKSgqSkpIwbNgw8Pl8WFhYIDU1VW5pSrFYjH79+sHLywvV1dUtut62MGPGDBw6dAgvX77kOhTSzlAyJR1OXFwcioqK1L6NpuTm5mL58uVYuXKlbJUyPp+PAwcOyJWzs7MDAOTl5SlUf79+/ZCcnIxZs2ZBV1e30XLff/89Bg4cCBcXl3fWGR4ejqysLERFRSkUS1uaMmUKKisr8d///pfrUEg7Q8mUtHuMMaxfvx69e/eGrq4ujI2N4efnJ7fAfnBwMHR0dNCtWzfZsc8++wz6+vrg8Xh4+vQpAGDx4sVYunQp8vLywOPx4ODggOjoaAgEApiZmWHBggWwsLCAQCCAm5sbzp8/r5Q2AOXuafsu0dHRYIzBx8enyXJ1g2nqFgBRpqqqKpw7d67JnuubjI2NMWrUKERFRbXbqVPdunXD0KFDkZKSwnUopJ2hZEravfDwcISGhmLZsmUoKipCRkYG8vPz4eHhgcePHwN4nTzenmO4efNmrFy5Uu5YVFQUJk+eDHt7ezDGkJubi+DgYAQFBaGyshJisRh3797FpUuXUF1djbFjxyI/P7/VbQCQDbCpra1V3s1pxKFDh+Dk5AShUNhkuQsXLgAA3N3dlR5DYWEhqqqq8Pvvv8PT01P2R0rv3r2xefPmBhPmgAED8ODBA1y5ckXp8SiLn58fUlNT2/XjaNL2KJmSdk0ikWD9+vWYOnUqZs+eDUNDQ7i4uOCHH37A06dP6y1b2Rp8Pl/W+3V2dkZMTAzKy8uxbds2pdTv7e2NsrIyLF++XCn1NaaiogJ37tyBvb19o2UeP36M+Ph4iMViuLq6vrMH2xJ1A4xMTU0RGRmJ7OxsPH78GH5+fvj888+xa9eueuf07NkTAHDt2jWlx6MsU6dORXFxMU6fPs11KKQdoWRK2rXs7Gy8ePECgwcPljs+ZMgQ6OjoyD2GVbbBgwdDKBS2ai9cLhQVFYEx1mSv1NXVFWKxGH5+fkhLS4O2trbS46h7l/r+++/Dzc0NJiYmMDQ0xMqVK2FoaNjgH0J1Mdc9cWiPevbsCWdnZ3rUS+RQMiXtWt1UiYZ27jEyMkJ5eblK29fV1cWTJ09U2oay1Y00bWpgkJmZGY4fP46NGzfC0NBQJXFYWFgAgOxdch0dHR3Y2Ng0OOhJT08PANr9aFk/Pz/s27ev3b7bJW2Pkilp14yMjACgwaT5/Plz9OjRQ2VtS6VSlbehCnUJqalFEExNTWX3VlU6d+6Mnj17yrb1e1N1dXWDSbxuz9C6a2ivpkyZgnv37rXrd7ukbVEyJe1anz590LlzZ/z2229yx8+fP4+qqioMGjRIdozP50MqlSqt7fT0dDDGMHz4cJW1oQpmZmbg8XgoLS1ttMyBAwdgaWmp8lgCAgJw+fJl3L59W3assrIS9+7da3C6TF3M5ubmKo+tNQYNGgQrKyvs27eP61BIO0HJlLRrAoEAS5cuxd69e7Fjxw6UlZXh2rVrWLhwISwsLDB//nxZWQcHB5SUlCAlJQVSqRRPnjzBvXv36tVpYmKCwsJC3L17F+Xl5bLkWFtbi2fPnqG6uhpXr17F4sWLYW1tjaCgIKW0oeieti0lFAphZ2eHgoKCBr/Pzc2Fubk5AgIC6n0XGBgIc3NzXLp0SSmxLFmyBDY2NggKCsL9+/dRXFyMkJAQSCQSfPXVV/XK18XcnHmpXOLxePD19aX3pkSGkilp91asWIHVq1cjIiICXbt2xahRo/Dee+8hPT0d+vr6snKLFi2Cp6cnZs6cCScnJ6xatUr2uNDV1VU2xWXhwoUwMzODs7MzvLy8UFJSAuD1ezoXFxfo6enBw8MDjo6OOHHihNy7x9a20Va8vb2RnZ3d4KLsTb3nq6qqQlFREVJTU5us/9y5c3B3d0f37t1x/vx5XLlyBRYWFhgxYgQyMjJk5YyNjXHq1Cn06NED/fv3h6WlJS5cuIBDhw41OP/04sWLsLS0RN++fRW4Wm74+Pjg6tWrsjWGSQf39gantHkzUaX2+vuaP38+MzEx4TqMRkHBza9zcnIYn89nv/zyi0Lt1NTUMA8PDxYXF6doiK329OlTJhAI2Lp16xQ+V9H7owyvXr1iBgYGbPPmzW3aLmmfqGdKyP9q77uWKMLBwQERERGIiIiQzfd8l5qaGqSkpKC8vJyTrQjDw8PRv39/BAcHt3nbLaGjo4OxY8fi0KFDXIdC2gFKpoRoqNDQUPj7+yMwMLDJwUh10tPTkZycjLS0tHeunKRs69evR1ZWFg4fPqySOa+q4u3tjWPHjjX7DxaiuSiZkg4vLCwM27ZtQ2lpKWxtbbFnzx6uQ1KayMhIBAcH49tvv31n2dGjR2Pnzp1yaw+3hdTUVLx69Qrp6ekwNjZu07Zba9KkSZBKpTh+/DjXoRCO8bkOgBCurV69GqtXr+Y6DJUZN24cxo0bx3UYjfL19YWvry/XYbSImZkZBg8ejEOHDqlkSUaiPqhnSgghreDt7Y0DBw7QakgdHCVTQghphUmTJuHhw4e4fPky16EQDlEyJYSQVhgwYAAsLS1x8OBBrkMhHKJkSgghrcDj8TBx4kQcOXKE61AIhxodgOTv79+WcZAOom65OPp9KW7Dhg1ISkriOgzSgHHjxmHbtm0oKSmBiYkJ1+EQDtTrmVpZWWH69OlcxEI03OXLl/HkyRP6fbXA9OnT1W73mrY0ffp0WFlZcdb+mDFjAAAnTpzgLAbCLR6jIWikjcyYMQMAkJiYyHEkhCifq6sr+vbti3//+99ch0I4QO9MCSFECcaNG4dff/2V6zAIRyiZEkKIEowfPx737t3DrVu3uA6FcICSKSGEKMGwYcNgbGxMvdMOipIpIYQoQadOneDp6Yn//Oc/XIdCOEDJlBBClGT8+PFIT0/Hq1evuA6FtDFKpoQQoiRjxoxBRUUFzp8/z3UopI1RMiWEECWxs7ODjY0NzTftgCiZEkKIEv3lL3+hZNoBUTIlhBAl8vT0xLlz5yCRSLgOhbQhSqaEEKJEH3zwAV69eoWzZ89yHQppQ5RMCSFEiaysrGBvb0+PejsYSqaEEKJknp6elEw7GEqmhBCiZJ6enrhw4QIqKiq4DoW0EUqmhBCiZJ6enpBKpTh9+jTXoZA2QsmUEEKUzMLCAvb29pRMOxBKpoQQogIjRoygZNqBUDIlhBAVcHNzw7lz51BdXc11KKQNUDIlhBAVcHNzQ0VFBa5du8Z1KKQNUDIlhBAVeP/992FsbEyPejsISqaEEKICWlpaGDZsGM6cOcN1KKQNUDIlhBAVcXNzo55pB0HJlBBCVGTEiBG4f/8+CgoKuA6FqBglU0IIUZGhQ4eiU6dOtFl4B0DJlBBCVKRz585wcnLC77//znUoRMUomRJCiAoNHjwYv/32G9dhEBWjZEoIISo0aNAg/Pbbb2CMcR0KUSFKpoQQokKDBw/Gs2fPcPv2ba5DISpEyZQQQlRowIAB0NbWpke9Go6SKSGEqJCenh6cnZ1pEJKGo2RKCCEqNnjwYFy8eJHrMIgKUTIlhBAVGzhwIC5fvkyDkDQYJVNCCFGxvn37orS0FPn5+VyHQlSEz3UARDNVVlbi1atXcseqqqoAAM+ePZM7rqurC6FQ2GaxEdLWXFxcwOPxcO3aNVhbW3MdDlEB6pkSldi+fTtMTEzkPqmpqUhNTa13fPv27VyHS4hKGRoawtLSkvY21WCUTIlK+Pv7o1OnTu8s16lTJ/j7+7dBRIRwy8XFBX/88QfXYRAVoWRKVMLU1BSjR49uMqF26tQJY8aMgampaRtGRgg3XFxcqGeqwSiZEpWZPXt2k6MXGWOYPXt2G0ZECHdcXFxw8+ZN2dgBolkomRKV8fPzg7a2dqPf8/l8+Pj4tGFEhHDHxcUFVVVVyMnJ4ToUogKUTInKGBgYYPLkyQ0mVD6fD19fX4hEIg4iI6Tt9erVC3w+nx71aihKpkSlZs2aherq6nrHa2pqMGvWLA4iIoQburq6sLGxoZ6phqJkSlTKy8sLnTt3rndcX18fEyZM4CAiQrjj6OhIyVRDUTIlKqWjowN/f3/o6OjIjmlrayMgIAC6urocRkZI2+vZsyclUw1FyZSo3Icffig3glEqleLDDz/kMCJCuNGzZ0/cunWL6zCIClAyJSrn6ekpN5e0a9euGDVqFIcREcINR0dHPHv2DCUlJVyHQpSMkilROS0tLXz44YfQ0dGBtrY2Zs2a1azVkQjRND179gQAetSrgSiZkjYxc+ZMVFVV0SNe0qHZ2NhAIBBQMtVA9XaNKSgowJkzZ7iIhWgwxhi6dOkCALhz5w7u3r3LbUBE47i5uaFHjx5ch9EkLS0t2NnZUTLVQPWS6ZkzZxAQEMBFLKSDCAwM5DoEooESEhIwY8YMrsN4J2tra9y/f5/rMIiSNbqfKe0IT5Tt+vXrOHLkCJYuXUq/LwXxeDy1SRZc4PF4XIfQbNbW1sjNzeU6DKJk9M6UtBlnZ+d2/xiOEFWzsrJCfn4+12EQJaNkSgghbagumdLTGc1CyZQQQtqQlZUVXr58ieLiYq5DIUpEyZQQQtqQlZUVANAgJA1DyZQQQtqQlZUVeDwevTfVMJRMCSGkDQkEAnTp0gUFBQVch0KUiJIpUUuHDx+GoaEhDhw4wHUo7d7Ro0cRGhqK2tpaTJkyBdbW1hAIBLC0tISvry+uXr3a4rpra2uxYcMGuLm5NVpGKpVi9erVcHBwgI6ODoyMjNCnTx/Zwh379+/HmjVrUFNT0+I41I25uTmKioq4DoMoESVTopZoJGTzrFixAtHR0QgLC0NtbS1OnTqFXbt2oaSkBJmZmZBIJBg5ciQKCwsVrjsnJwcjR47EkiVLUFlZ2Wi5gIAA/Pzzz9i5cycqKytx48YN2Nvb48WLFwAAHx8fCAQCjB49Gs+fP2/xtaoTU1NTPHnyhOswiBJRMiVqydvbG6WlpZg8eTLXoUAikTTZM+PKd999h/j4eCQmJsLAwAAA4OrqCnd3dwiFQtja2iIyMhKlpaXYvn27QnVfuXIFX331FRYuXIj+/fs3Wi4+Ph4pKSlISkrCsGHDwOfzYWFhgdTUS3NaUQAAIABJREFUVPTp00dWTiwWo1+/fvDy8kJ1dXWLrledmJqaUs9Uw1AyJaSV4uLi2t0/jLm5uVi+fDlWrlwJgUAAAODz+fUei9vZ2QEA8vLyFKq/X79+SE5OxqxZs5rc5P3777/HwIED4eLi8s46w8PDkZWVhaioKIViUUdmZmbUM9UwlEyJ2snMzIS1tTV4PB42bdoEAIiJiYG+vj6EQiFSU1MxceJEiEQi9OjRA7t375adGx0dDYFAADMzMyxYsAAWFhYQCARwc3PD+fPnZeWCg4Oho6ODbt26yY599tln0NfXB4/Hw9OnTwEAixcvxtKlS5GXlwcejwcHBwcAwJEjRyASiRAZGdkWt6Se6OhoMMbg4+PTZDmJRAIAEIlESo+hqqoK586da7Ln+iZjY2OMGjUKUVFRGv8Ynx7zah5KpkTtuLu719vZaNGiRfjiiy8gkUhgYGCAhIQE5OXlwc7ODp988gmkUimA10kyKCgIlZWVEIvFuHv3Li5duoTq6mqMHTtWNl0hOjq63jq4mzdvxsqVK+WORUVFYfLkybC3twdjTLbmat1gmtraWpXcg3c5dOgQnJycIBQKmyx34cIFAK/vqbIVFhaiqqoKv//+Ozw9PWV/uPTu3RubN29uMGEOGDAADx48wJUrV5QeT3tCyVTzUDIlGsfNzQ0ikQimpqYIDAxERUVFvQnyfD4fvXv3hq6uLpydnRETE4Py8nJs27ZNKTF4e3ujrKwMy5cvV0p9iqioqMCdO3dgb2/faJnHjx8jPj4eYrEYrq6u7+zBtkTdACNTU1NERkYiOzsbjx8/hp+fHz7//HPs2rWr3jl1m2dfu3ZN6fG0J2ZmZigpKekQ74c7CkqmRKPp6OgAgKxn2pjBgwdDKBTi5s2bbRGWShUVFYEx1mSv1NXVFWKxGH5+fkhLS4O2trbS46h7l/r+++/Dzc0NJiYmMDQ0xMqVK2FoaIgtW7bUO6cu5sePHys9nvaka9euqK2txbNnz7gOhShJo1uwEdLR6OrqasSjt5cvXwJAkwODzMzMEBcXh/fff19lcVhYWACA7P1yHR0dHdjY2DQ46ElPTw/A/12DpqobXV1eXg5TU1OOoyHKQD1TQvC65/r8+XON2CKuLiE1tQiCqakpjIyMVBpH586d0bNnT1y/fr3ed9XV1TA0NKx3vKqqCsD/XYOmqkumdY/CifqjZEoIgPT0dDDGMHz4cNkxPp//zsfD7ZGZmRl4PB5KS0sbLXPgwAFYWlqqPJaAgABcvnwZt2/flh2rrKzEvXv3GpwuUxezubm5ymPjUufOnQG87pkSzUDJlHRIde+rqqurcfXqVSxevBjW1tYICgqSlXFwcEBJSQlSUlIglUrx5MkT3Lt3r15dJiYmKCwsxN27d1FeXg6pVIq0tDTOpsYIhULY2dk1uvZrbm4uzM3NERAQUO+7wMBAmJub49KlS0qJZcmSJbCxsUFQUBDu37+P4uJihISEQCKR4KuvvqpXvi7m5sxLVWdvPuYlmoGSKVE7mzZtwpAhQwAAISEh8PX1RUxMDDZs2AAA6Nu3L27fvo3Y2FgsXboUADBhwgTk5OTI6nj58iVcXFygp6cHDw8PODo64sSJE3LvGRctWgRPT0/MnDkTTk5OWLVqlezxo6urq2wazcKFC2FmZgZnZ2d4eXmhpKSkTe5DU7y9vZGdnS2bR/qmpuZwVlVVoaioCKmpqU3Wf+7cObi7u6N79+44f/48rly5AgsLC4wYMQIZGRmycsbGxjh16hR69OiB/v37w9LSEhcuXMChQ4canH968eJFWFpaom/fvgpcrfqpm69MyVSDsLckJCSwBg7/f/buPS6qav0f+GdggIGRqwIilwARCcFLqQFKZnzVklJJUKxOWekhtQOWpzyKGKFgqAc4eKn0kKcfaYCXA6Iipkh5Rc0LiJmAoSDKReQioAzM+v3hd+bryHVgZvYwPO/Xa/5wz5q1nr0ZediXtR5CFEIdvl9BQUHMzMyM0xjkBYAlJyd3u31BQQHj8/ksMTFRrnFaW1uZt7c3S0hIkDfEXquqqmICgYBt3LhR7s/Ke3zUgVAoZN9//z3XYRAFoTNT0i9peoUSJycnREREICIiotsPubS2tiI1NRX19fUIDAxUcoRthYeHY/To0QgODlb52FwwNDSkM1MNopRkumDBAhgaGoLH4+Hy5cvKGELpIiIi4OrqCiMjI+jp6cHJyQlffPFFl7+YHj16BBcXF6xatUruMffu3QtHR0fweDyZl66uLiwsLPDKK69gw4YNNDeNdMuKFSsQEBCAwMDATh9GksjOzsbevXuRkZHR5cpJihYTE4PLly/j0KFDSpnzqo6EQiEaGhq4DoMoiFKS6b///W9s375dGV2rTFZWFj755BMUFxejqqoKUVFRiIuLQ0BAQKefCw0NxR9//NGjMWfPno2bN29i6NChMDY2BmMMYrEYFRUVSElJgYODA5YvX44RI0bgwoULPRqjv1u5ciV27NiB2tpaODg4YM+ePVyHpFSRkZEIDg7GunXrumzr4+ODnTt3yqxHrAppaWl4/PgxsrOzYWpqqtKxucTn82kFJA1CizZ0YMCAAQgKCoK2tjYAYM6cOdi7dy9SUlJQUlICW1vbNp85ffo0rl69qtA4eDweTExM8Morr+CVV16Br68v5s6dC19fX9y4caPduXqkY1FRUYiKiuI6DJWaOnUqpk6dynUYHZo5cyZmzpzJdRgqR8lUsyjtnimPx1NW1ypx4MABaSKVGDRoEAC0Wwi5qakJn3/+udLLR/n7+2P+/PmoqKjAt99+q9SxCCHKo6Oj0yfnMZP2KSSZMsawYcMGDB8+HHp6ejA2Nsbnn3/epl1raytWr14NOzs76OvrY+TIkUhOTgbQ/RJaAPDLL79g/PjxMDAwgJGREdzd3VFXV9flGL11584d6Ovrw8HBoc17oaGhWLJkSYdLgymyJJdkLmRGRoZ0W18/toT0N3RmqlkUkkzDwsKwfPlyBAUFoby8HPfu3Wt3QvY//vEPrF+/HrGxsbh79y7efPNNvP3227hw4UK3S2g1NDRgxowZ8Pf3R3V1NQoKCuDs7CxdhqyzMXqjsbERWVlZWLhwoXTxdIlTp06hqKgIb7/9doefV2RJLsn8vKdXlenLx5aQ/ojOTDXMs3Nl5J0H2NjYyAwMDNiUKVNktv/0008MALt06RJjjLGmpiZmYGDAAgMDZT6rp6fHFi9ezBhjLDQ0lAFgTU1N0jZbtmxhAFhhYSFjjLGrV68yAOzAgQNtYunOGD0VGhrKnJ2dWV1dXZv9Hzt2LCstLWWMMVZZWckAsNDQ0B6PNXToUGZsbNxpGx6Px0xMTBhjfevYqsM8074IfXAepSr1xePj7e3NPvnkE67DIArS6weQCgsL0djYCB8fn07b/fHHH2hsbISbm5t0m76+PgYPHtxp2atnS2g5OjrCwsIC7777LkJCQjB//nzY29v3aoyu7Nu3DykpKThy5Ih0GTCJlStX4q9//atK1jmVaGhoAGMMRkZGAPrmse3qqWjSVmxsLHbv3s11GERB6MxUs/T6Mq9kLc2uyghJ5lOtWrVKZg7lrVu32n2gpyP6+vrIysrCxIkTERkZCUdHRwQGBqKpqUlhYzwtKSkJX3/9NbKzs6WJReLkyZPIy8vDggULetR3T924cQMA4OLiAqDvHltCCNEUvT4zFQgEAIDHjx932k6SbGNjY7F06dJejTlixAikp6ejsrISMTEx+PrrrzFixAjpqi2KGAMANm3ahMzMTGRlZUmrPDwtISEBx44dg5ZW279JIiMjERkZifPnz2Ps2LG9juVphw8fBgC8/vrrAPrmsaUzLPnweDx8+umnmDNnDtehqKW+OHtAJBL1mwUq+oNen5m6ublBS0sLv/zyS6ftbG1tIRAIer0iUllZmbQ+orm5OdatW4cXXngB165dU9gYjDEsX74ceXl5SE1NbTeRAsCOHTvAGJN5SYpLh4aGgjGm8ER67949xMbGwsbGBh9++CGAvnVsCSFPtLS0gM+nqf6aotfJ1NzcHLNnz8aePXuQkJCAuro65ObmYtu2bTLtBAIBPvjgA/z000/YunUr6urq0NraitLSUty9e7fb45WVleHjjz/G9evX0dzcjEuXLuHWrVvw8PBQ2BjXrl3D+vXrsX37dujo6LRZ3m/jxo3d7ktC3pJcjDE8fPgQYrFYmqSTk5MxYcIEaGtrIzU1VXrPtC8dW0LIE3RmqmGefSKpJ09b1tfXswULFrCBAweyAQMGsIkTJ7LVq1czAMzGxoZduXKFMcbY48eP2fLly5mdnR3j8/nM3NyczZ49m+Xn57MtW7YwAwMDBoANGzaMFRUVsW3btjEjIyMGgD333HPsxo0brLi4mHl5eTFTU1Omra3NhgwZwkJDQ1lLS0uXY3RXXl4eA9Dha8OGDR1+tqOneQ8dOsQMDQ3Z2rVrO/zs/v372ciRI5mBgQHT1dVlWlpaDID0yd3x48eziIgIdv/+/Taf7SvHlp7m7Rn0wadVVakvHp/Ro0ezFStWcB0GURAeY7LFDVNSUjB37txOax4S0lP0/eoZHo+H5ORkumfagb54fNzc3DB79mx89dVXXIdCFIBKsBFCCAfonqlm6TfJ9Pr1623ufbb34qKOIyHKdPToUaxYsQJisRh+fn6ws7ODQCCAtbU1Zs6cidzcXLn7jI6OhouLC/T19SEUCuHi4oKwsDDp0pMS3SlluH//fkRHR2t8jdlnUTLVLP0mmbq4uLR58ra9V1JSEtehEqIwX375JeLj47Fy5UqIxWKcOHECu3btQnV1NU6ePImmpia8/PLLKCsrk6vfEydOYOHChbh9+zbKy8uxZs0aREdHw9/fX6Zdd0oZzpgxAwKBAD4+PqipqVHIfvcFDQ0NEAqFXIdBFKTfJFNCJJqamuDl5dXnx+jK119/jaSkJKSkpEhX7vL09MTEiRNhYGAABwcHREZGora2Fv/5z3/k6ltXV1da2GHAgAEICAjArFmz8PPPP8s83S0pZWhmZgZDQ0PMmTMHfn5+OHz4MEpKSqTtQkJCMGrUKEyfPr3fLP5eX1/fZkU10ndRMiX9TkJCAioqKvr8GJ0pLCxEWFgYvvrqK+nCKnw+H+np6TLtHB0dAQBFRUVy9b9v3z5pvxKSJTWfvoQrTynD8PBwXL58WellDNWBWCxGY2Njh3PYSd9DyZSoPcYYYmJi8Pzzz0NPTw+mpqaYNWuWzJrAwcHB0NXVxeDBg6XblixZAqFQCB6Ph6qqKgDA0qVLsWzZMhQVFYHH48HJyQnx8fEQCASwsLDAxx9/DCsrKwgEAnh5eSEnJ0chYwCKLcPXlfj4eDDGMGPGjE7bNTU1AYB0znJvFBQUwMTEBM8991yn7ToqZWhqaopJkyYhLi5O45/2lqyvTWemmoOSKVF74eHhWLFiBUJDQ1FRUYFff/0VJSUl8Pb2Rnl5OYAnyePZaRFbtmxpM+0gLi4Ob775JoYOHQrGGAoLCxEcHIz58+ejsbERISEhKC4uxsWLF9HS0oIpU6ZIL0f2ZgxAsWX4unLw4EEMHz4cBgYGnbY7d+4cAGDixIk9GkckEuHOnTvYvHkzjh49ik2bNrUpUfi0zkoZAsCYMWNw584dXLlypUfx9BX19fUAQMlUg1AyJWqtqakJMTExeOutt/Duu+/C2NgY7u7u+Pbbb1FVVdVmpa3e4PP50rNfV1dXbN26FfX19dixY4dC+vf19UVdXR3CwsIU0l9HGhoa8Oeff2Lo0KEdtikvL0dSUhJCQkLg6enZ5RlsR2xtbWFjY4Pw8HCsX78ec+fO7bR9VFQUrKyssHbt2nbfHzZsGAAgLy+vR/H0FZRMNQ8lU6LW8vPz8fDhwzZrHI8bNw66uroyl2EVbezYsTAwMOhV+T4uVFRUgDHW6Vmpp6cnQkJCMGvWLGRkZPR4WbuSkhJUVFRg165d+OGHHzBmzJgO7xVLShlmZmZ2mEQkMUuuOGgqyX1lSqaagyY5EbUmmSrR3oMaJiYm0r/wlUVPT09avKCvePToEYAnsXfEwsICCQkJGDFiRK/G0tHRgbm5OaZOnQoHBwc4OztLp788LSkpCTExMcjOzsaQIUM67E9fX19mHzSV5HtLDyBpDkqmRK2ZmJgAQLtJs6amBjY2NkobWyQSKX0MZZAkpM4WQTA3N5ceW0VxcnKCtrY28vPzZbZ3Vcrwac3NzQD+bx80VUVFBbS0tGBmZsZ1KERB6DIvUWtubm4YMGAALly4ILM9JycHzc3NePHFF6Xb+Hw+RCKRwsbOzs4GYwweHh5KG0MZLCwswOPxUFtb22Gb9PR06VQWed2/fx9vv/12m+0FBQVobW2Fra0tgO6XMnyaJGZLS8sexdZXVFZWYuDAgW2mDZG+i5IpUWsCgQDLli3Dvn378OOPP6Kurg55eXlYtGgRrKysEBQUJG3r5OSE6upqpKamQiQSobKyErdu3WrTp5mZGcrKylBcXIz6+nppchSLxXjw4AFaWlqQm5uLpUuXws7ODvPnz1fIGPKW4espAwMDODo6orS0tN33CwsLYWlp2e7DQoGBgbC0tMTFixc77F8oFOLIkSPIyspCXV0dRCIRLl26hPfffx9CoRCfffYZgJ6VMpTE7O7u3pNd7zMqKythbm7OdRhEgSiZErX35ZdfIioqChERERg0aBAmTZoEe3t7ZGdnyyzHtnjxYkyePBnz5s3D8OHDsWbNGunlQk9PT+kUl0WLFsHCwgKurq6YPn06qqurATy5T+fu7g59fX14e3vD2dkZx48fl7n32NsxVMXX1xf5+fnSeaRP62wOZ3NzMyoqKpCWltZhG4FAgAkTJmDBggWwtraGoaEhAgICYG9vj7Nnz8LNza3LcTpy/vx5WFtbY+TIkXJ/ti+hZKqBnq3JRvUmiTKp6/crKCiImZmZcR1GhyBnvc6CggLG5/NZYmKiXOO0trYyb29vlpCQIG+IvVZVVcUEAgHbuHGj3J+V9/hwzd/fnwUEBHAdBlEgOjMl5H9pUtUSJycnREREICIiQmZ5v860trYiNTUV9fX1nFRPCg8Px+jRoxEcHKzysVWNzkw1DyVTQjTUihUrEBAQgMDAwE4fRpLIzs7G3r17kZGR0eXKSYoWExODy5cv49ChQz2e89qXVFRUUDLVMJRMSb+3cuVK7NixA7W1tXBwcMCePXu4DklhIiMjERwcjHXr1nXZ1sfHBzt37pRZe1gV0tLS8PjxY2RnZ8PU1FSlY3OloqICFhYWXIdBFIjmmZJ+LyoqClFRUVyHoTRTp07F1KlTuQ6jQzNnzsTMmTO5DkNlGhsbcf/+fekUIqIZ6MyUEEJUSPLENyVTzULJlBBCVIiSqWaiZEoIISpUUlICAwMDDBw4kOtQiAJRMiWEEBUqKSmhs1INRMmUEEJUiJKpZurwaV4ej6fKOEg/Q98v+c2dO7fL4ttE/VEy1UxtkqmXlxeSk5O5iIVouNjYWADAp59+ynEkRBN5eXlxHUK33L59Gy+99BLXYRAFa5NMbWxsMGfOHC5iIRpOshgCfb9IfyUWi3Hz5k0MGzaM61CIgtE9U0IIUZFbt27h8ePHlEw1ECVTQghRkRs3bgAAJVMNRMmUEEJUpKCgAAMHDoSZmRnXoRAFo2RKCCEqUlBQQGelGoqSKSGEqMiNGzfg7OzMdRhECSiZEkKIitCZqeaiZEoIISrw+PFj3Lp1i5KphqJkSgghKvD777+jpaUF7u7uXIdClICSKSGEqEBeXh709PTozFRDUTIlhBAVyMvLg4uLC3R0dLgOhSgBJVNCCFGBvLw8uLm5cR0GURJKpoQQogJXr16l+6UajJIpIYQoWU1NDe7cuUPJVINRMiWEECXLy8sDY4ySqQajZEoIIUqWm5sLExMT2NjYcB0KURJKpoQQomS//fYbXnjhBfB4PK5DIUpCyZQQQpTswoULGDt2LNdhECWiZEoIIUrU2NiI33//nZKphqNkSgghSnTp0iW0tLRQMtVwlEwJIUSJLly4ADMzM9jb23MdClEiSqaEEKJEFy5cwLhx4+jhIw1HyZQQQpSIHj7qHyiZEkKIktTX1+PGjRt44YUXuA6FKBklU0IIUZKcnByIxWK89NJLXIdClIySKSGEKMmpU6dgb28Pa2trrkMhSkbJlBBClOT06dOYMGEC12EQFaBkSgghSiAWi5GTkwMvLy+uQyEqQMmUEEKU4OrVq6itraUz036CkikhhCjBqVOnYGhoCDc3N65DISpAyZQQQpTg9OnT8PDwgLa2NtehEBWgZEoIIUpw6tQpul/aj1AyJYQQBSsrK8Off/5J90v7ET7XARDNlJOTgytXrshsu3nzJgBg27ZtMttHjRpFk9qJRjl+/Dh0dHTozLQfoWRKlKKyshJBQUHQ1taGltaTCyCMMQDAJ598AuDJ1IHW1lakp6dzFichynD8+HG89NJLEAqFXIdCVIQu8xKlmDZtGoyMjNDa2gqRSASRSISWlha0tLRI/93a2gojIyNMnTqV63AJUajjx49j8uTJXIdBVIiSKVEKHR0dzJs3D7q6ur1qQ0hfU1JSgps3b1Iy7WcomRKlmTdvHpqbmzt8XyQS4e2331ZhRIQo37Fjx6CnpwcPDw+uQyEqxGOSG1mEKJhYLMaQIUNQXl7e7vvm5ua4d++e9J4qIZrg/fffx+3bt3H8+HGuQyEqRL/FiNJoaWnhL3/5S7uXcXV1dTF//nxKpETj/PLLL3SJtx+i32REqTq61Nvc3Ix58+ZxEBEhylNUVIRbt25RMu2HKJkSpXrhhRfg5OTUZrujoyPGjBnDQUSEKM/Ro0chFApp3nQ/RMmUKN27774LHR0d6b91dXXx/vvvcxgRIcqRmZmJyZMn0xPq/RAlU6J077zzDkQikfTfzc3NCAwM5DAiQhSvpaUFWVlZNG+6n6JkSpTOyckJo0aNAo/HA4/Hw6hRo+Ds7Mx1WIQoVE5ODmprazFt2jSuQyEcoGRKVOK9996DtrY2tLW18d5773EdDiEKl5mZCXt7e/pDsZ+iZEpUIjAwULoW79y5c7kOhxCFO3LkCJ2V9mNtFro/c+YMYmJiuIiFaDgzMzMAwNKlSzmOhGiizz77DJ6enpyM/eDBA1y4cAFffPEFJ+MT7rU5My0pKcGePXu4iIVoODs7OxgZGdH3qwf27NmD0tJSrsNQW3v27EFJSQln4//8888AQPNL+7EOS7Dt3r1blXGQfqC6uhqpqan46KOP6PslJx6Ph08//RRz5szhOhS1xOPxOB3/yJEjeOmll2BqasppHIQ7dM+UqIyZmRkGDBjAdRiEKBRjDBkZGXjttde4DoVwiJIpIYT0wm+//YaysjK8+eabXIdCOETJlBBCeuHAgQMYMmQIRo0axXUohEOUTAkhpBcOHjyIGTNmcH7flnCLkikhhPTQ3bt38dtvv8HX15frUAjHKJmSPunQoUMwNjZGeno616GovaNHj2LFihUQi8Xw8/ODnZ0dBAIBrK2tMXPmTOTm5srdZ3R0NFxcXKCvrw+hUAgXFxeEhYWhrq5Opl1ERARcXV1hZGQEPT09ODk54YsvvsDDhw+lbfbv34/o6Gi0trb2el9V7eDBgxAIBHj11Ve5DoVwjJIp6ZMYY1yH0Cd8+eWXiI+Px8qVKyEWi3HixAns2rUL1dXVOHnyJJqamvDyyy+jrKxMrn5PnDiBhQsX4vbt2ygvL8eaNWsQHR0Nf39/mXZZWVn45JNPUFxcjKqqKkRFRSEuLg4BAQHSNjNmzIBAIICPjw9qamoUst+qcvDgQfj4+MDAwIDrUAjX2DOSk5NZO5sJUQhN/H41NjYyT09PpY4BgCUnJ8v1mXXr1jFnZ2fW1NTEGGNMJBKxN954Q6bNuXPnGAAWGRkpV99+fn7SfiUCAgIYAFZWVibd5uvry1paWmTazZkzhwFgt2/fltkeHBzMPD09mUgkkisWxnp2fHrr0aNHzNDQkH3zzTcqHZeoJzozJaSXEhISUFFRwXUYMgoLCxEWFoavvvoKAoEAAMDn89tcFnd0dAQAFBUVydX/vn37pP1KWFtbA4DMJdwDBw5AW1tbpt2gQYMAAI2NjTLbw8PDcfnyZcTFxckVC1eOHz+O+vp6vP7661yHQtQAJVPS55w8eRJ2dnbg8XjYvHkzAGDr1q0QCoUwMDBAWloaXn/9dRgZGcHGxgY//fST9LPx8fEQCASwsLDAxx9/DCsrKwgEAnh5eSEnJ0faLjg4GLq6uhg8eLB025IlSyAUCsHj8VBVVQXgyTrDy5YtQ1FREXg8HpycnAAAhw8fhpGRESIjI1VxSNqIj48HYwwzZszotF1TUxMAwMjIqNdjFhQUwMTEBM8991yn7e7cuQN9fX04ODjIbDc1NcWkSZMQFxfXJy7jHzhwAKNHj+5yf0n/QMmU9DkTJ07E6dOnZbYtXrwYn376KZqammBoaIjk5GQUFRXB0dERCxculBYnDw4Oxvz589HY2IiQkBAUFxfj4sWLaGlpwZQpU6Tru8bHx7dZum/Lli346quvZLbFxcXhzTffxNChQ8EYQ2FhIQBIH6YRi8VKOQZdOXjwIIYPH97lvbxz584BeHJMe0IkEuHOnTvYvHkzjh49ik2bNkFXV7fD9o2NjcjKysLChQvbbTdmzBjcuXMHV65c6VE8qsIYQ1paGmbNmsV1KERNUDIlGsfLywtGRkYwNzdHYGAgGhoacPv2bZk2fD4fzz//PPT09ODq6oqtW7eivr4eO3bsUEgMvr6+qKurQ1hYmEL6k0dDQwP+/PNPDB06tMM25eXlSEpKQkhICDw9Pbs8g+2Ira0tbGxsEB4ejvXr13dZXi8qKgpWVlZYu3Ztu+8PGzYMAJCXl9ejeFTl3LlzKC0thZ+fH9ehEDVByZRoNMnZj+TMtCNjx46FgYFnphJyAAAgAElEQVQBrl+/roqwlKqiogKMsU7PSj09PRESEoJZs2YhIyMDOjo6PRqrpKQEFRUV2LVrF3744QeMGTOmw/vH+/btQ0pKCjIzM2FoaNhuG0nM5eXlPYpHVVJTU2Fvb4+RI0dyHQpREx1WjSGkv9HT00NlZSXXYfTao0ePADzZn45YWFggISEBI0aM6NVYOjo6MDc3x9SpU+Hg4ABnZ2fp9JenJSUlISYmBtnZ2RgyZEiH/enr68vsg7pKTU3FW2+9xXUYRI1QMiUET85ca2pqYGNjw3UovSZJSJ0tgmBubg4TExOFjuvk5ARtbW3k5+fLbN+0aRMyMzORlZXVZdWg5uZmAP+3D+roxo0buH79OrZt28Z1KESN0GVeQgBkZ2eDMQYPDw/pNj6f3+XlYXVkYWEBHo+H2traDtukp6dLp7LI6/79+3j77bfbbC8oKEBraytsbW0BPHlIZ/ny5cjLy0Nqamq3yu9JYra0tOxRbKqwd+9eWFhYwMvLi+tQiBqhZEr6JbFYjAcPHqClpQW5ublYunQp7OzsMH/+fGkbJycnaUFzkUiEyspK3Lp1q01fZmZmKCsrQ3FxMerr6yESiZCRkcHZ1BgDAwM4OjqitLS03fcLCwthaWnZ7sNCgYGBsLS0xMWLFzvsXygU4siRI8jKykJdXR1EIhEuXbqE999/H0KhEJ999hkA4Nq1a1i/fj22b98OHR0d8Hg8mdfGjRvb9C2J2d3dvSe7rhKpqamYMWNGm/mzpH+jZEr6nM2bN2PcuHEAgOXLl2PmzJnYunUrYmNjAQAjR47EzZs3sX37dixbtgwA8Nprr6GgoEDax6NHj+Du7g59fX14e3vD2dkZx48fl7nPuHjxYkyePBnz5s3D8OHDsWbNGunlR09PT+k0mkWLFsHCwgKurq6YPn06qqurVXIcOuPr64v8/HzpPNKndTaHs7m5GRUVFUhLS+uwjUAgwIQJE7BgwQJYW1vD0NAQAQEBsLe3x9mzZ+Hm5tblOB05f/48rK2t1fbBnrKyMpw/f56mxJC2nl0SSROXeyPqQx2+X0FBQczMzIzTGOQFOZfLKygoYHw+nyUmJso1TmtrK/P29mYJCQnyhthrVVVVTCAQsI0bN8r9WXmPT09t2bKFDRgwoM1SioTQmSnpl/pihRJ5ODk5ISIiAhERETLL+3WmtbUVqampqK+vR2BgoJIjbCs8PByjR49GcHCwysfuruTkZLz55pttllIkhJIpIRpqxYoVCAgIQGBgYKcPI0lkZ2dj7969yMjIUHkVlJiYGFy+fBmHDh3q8ZxXZbt79y5OnTrV5cIUpH9SSjJdsGABDA0NwePxcPnyZWUMoXTdqcMIAGvXrm3zYAWPx5PeN5LH3r174ejo2KYvXV1dWFhY4JVXXsGGDRvw4MEDRe1mv7Ny5Urs2LEDtbW1cHBwwJ49e7gOSakiIyMRHByMdevWddnWx8cHO3fulFmPWBXS0tLw+PFjZGdnw9TUVKVjyyMlJQVCoRDTpk3jOhSijp697quoe1o//fQTA8AuXbrU6764MGnSJLZlyxZ2//59VldXx5KTk5mOjg577bXXZNqtWbOGAWjzGjFiRI/HHjp0KDM2NmaMMSYWi9mDBw/Y8ePH2fz58xmPx2NWVlbs/Pnzvdo/rqjDPdO+CByUGOtLVHF8PD092XvvvafUMUjfRZd5OzBgwAAEBQXBzMwMhoaGmDNnDvz8/HD48GHpU5wSiYmJYIzJvK5evaqQOHg8HkxMTPDKK69gx44dSElJQXl5OXx9fbt16Y4Q0nslJSU4e/Zsm+IHhEgoLZnyeDxlda0S8tRhVCV/f3/Mnz8fFRUV+PbbbzmLg5D+JCUlBcbGxpgyZQrXoRA1pZBkyhjDhg0bMHz4cOjp6cHY2Biff/55m3atra1YvXo17OzsoK+vj5EjRyI5ORlA9+tRAsAvv/yC8ePHw8DAAEZGRnB3d0ddXV2XY/RWR3UYu0OR9S0lCwtkZGRIt/X1Y0uIOktOTsZbb73VaXk50s89e923J/e0QkNDGY/HY//85z/ZgwcPWGNjI9uyZUube6Z///vfmZ6eHtuzZw978OABW7lyJdPS0pLe/wsNDWUA2LFjx1htbS2rqKhg3t7eTCgUsubmZsYYYw8fPmRGRkYsOjqaNTU1sXv37rG33nqLVVZWdmuMnmpoaGCGhoYsODhYZvuaNWuYjY0NMzExYTo6Osze3p7NnDmTnTt3TqbdgQMHmKGhIYuIiOhyrKfvmbanrq6OAWC2trbSbX3l2NI9054B3TPtlDKPz82bNxmPx2OZmZlK6Z9ohl4n08bGRmZgYMCmTJkis/3ZB5CampqYgYEBCwwMlPmsnp4eW7x4MWPs/37hPz0hWpKUCwsLGWOMXb16lQFgBw4caBNLd8boqdDQUObs7Mzq6upktt++fZtdvHiR1dfXs8ePH7MzZ86wMWPGMH19fXb16tUejdVVMmWMMR6Px0xMTBhjfevYUjLtGUqmnVPm8Vm3bh0bNGgQE4lESumfaIZeX+YtLCxEY2MjfHx8Om33xx9/oLGxUWbKiL6+PgYPHtxpDcln61E6OjrCwsIC7777LsLDw1FcXNzrMbrSWR1GW1tbjBkzBgMGDICuri48PDywY8cONDU1YcuWLT0eszMNDQ1gjMHIyAhA3zy27U0nolfHLwCYO3cu53Go60uZkpOTMXv2bPD5VGSLdKzX3w7JwtTm5uadtmtoaAAArFq1CqtWrZJ5z8rKqtvj6evrIysrC//4xz8QGRmJiIgIzJkzBzt27FDYGE/rbh3Gp7m7u0NbWxs3btzo0ZhdkfTr4uICoG8eW7rXKp+5c+di6dKl8PT05DoUtaSshRRu3LiBy5cv45///KdS+ieao9fJVLKs1uPHjzttJ0m2sbGxWLp0aa/GHDFiBNLT01FZWYmYmBh8/fXXGDFihHQJNEWMAchXh/FpYrEYYrG40+LMvXH48GEAwOuvvw6gbx5bmmIgn7lz58LT05OOWweUlUyTk5NhaWmJSZMmKaV/ojl6fZnXzc0NWlpa+OWXXzptZ2trC4FA0OsVkcrKynDt2jUAT5LIunXr8MILL+DatWsKG4PJUYexvdVQzp8/D8aYUs4i7t27h9jYWNjY2ODDDz8E0LeOLSF9SUpKCvz9/ancGulSr5Opubk5Zs+ejT179iAhIQF1dXXIzc1tU4VeIBDggw8+wE8//YStW7eirq4Ora2tKC0txd27d7s9XllZGT7++GNcv34dzc3NuHTpEm7dugUPDw+FjSFPHcY7d+4gKSkJNTU1EIlEOHPmDBYsWAA7OzssWrRI2k7e+paMMTx8+BBisRiMMVRWViI5ORkTJkyAtrY2UlNTpfdM+9KxJaSvuH79Oq5evUpr8ZLuefaJpJ48bVlfX88WLFjABg4cyAYMGMAmTpzIVq9ezQAwGxsbduXKFcYYY48fP2bLly9ndnZ2jM/nM3NzczZ79myWn5/PtmzZwgwMDBgANmzYMFZUVMS2bdvGjIyMGAD23HPPsRs3brDi4mLm5eXFTE1Nmba2NhsyZAgLDQ1lLS0tXY7RXXl5ee0uESh5bdiwQdp22bJlbOjQoUwoFDI+n89sbGzYwoULWVlZmUyfhw4dYoaGhmzt2rUdjrt//342cuRIZmBgwHR1dZmWlhYDIH1yd/z48SwiIoLdv3+/zWf7yrGlp3l7BvQ0b6eUcXxWr17NrKysWGtrq0L7JZqJx5hsBd+UlBTMnTu3R4V9CekKfb96hsfjITk5me6ZdkAZx8fV1RXTpk2TFp0npDO0Ni8hhDzjypUr+P333+mPF9Jt/SaZXr9+vVvz1bgoikyIMh09ehQrVqyAWCyGn58f7OzsIBAIYG1tjZkzZyI3N1fuPqOjo+Hi4gJ9fX0IhUK4uLggLCxMuvSkRHdKGe7fvx/R0dFqVbA9OTkZtra28PDw4DoU0kf0m2Tq4uLSprJLe6+kpCSuQyVEYb788kvEx8dj5cqVEIvFOHHiBHbt2oXq6mqcPHkSTU1NePnll1FWViZXvydOnMDChQtx+/ZtlJeXY82aNYiOjoa/v79Mu6ysLHzyyScoLi5GVVUVoqKiEBcXh4CAAGmbGTNmQCAQwMfHBzU1NQrZ797avXu3dJEMQrqj3yRTQiSamprg5eXV58foytdff42kpCSkpKRIV+7y9PTExIkTYWBgAAcHB0RGRqK2thb/+c9/5OpbV1cXS5Ysgbm5OQYMGICAgADMmjULP//8s8zT3d0tZRgSEoJRo0Zh+vTpaGlpUcj+99SFCxdQWFhIl3iJXCiZkn4nISEBFRUVfX6MzhQWFiIsLAxfffWVdGEVPp+P9PR0mXaOjo4AgKKiIrn637dvn7RfCWtrawCQuYQrTynD8PBwXL58GXFxcXLFomiJiYkYNmwYxo4dy2kcpG+hZErUHmMMMTExeP7556GnpwdTU1PMmjVLZk3g4OBg6OrqYvDgwdJtS5YsgVAoBI/HQ1VVFQBg6dKlWLZsGYqKisDj8eDk5IT4+HgIBAJYWFjg448/hpWVFQQCAby8vJCTk6OQMQDFluHrSnx8PBhjmDFjRqftmpqaAEA6Z7k3CgoKYGJigueee67Tdh2VMjQ1NcWkSZMQFxfH2dPeLS0tSE5OxnvvvUeXeIl8np0rQ/MAiTL15Pu1evVqpquryxITE1lNTQ3Lzc1lL7zwAhs0aBC7d++etN0777zDLC0tZT67YcMGBkBaRo4xxmbPns2GDh0q0y4oKIgJhUJ27do19ujRI5afn8/GjRvHDA0N2e3btxUyhjxl+J4FOedROjo6MldX1y7b7d27lwFge/bskTsmxhhrbm5mpaWlbNOmTUxPT48lJiZ22r6jUoYSK1asaFO6sTvkPT4dSU9PZzwejxUVFfW6L9K/0JkpUWtNTU2IiYnBW2+9hXfffRfGxsZwd3fHt99+i6qqqjYrbfUGn8+Xnv26urpi69atqK+vx44dOxTSv6+vL+rq6hAWFqaQ/jrS0NCAP//8E0OHDu2wTXl5OZKSkhASEgJPT88uz2A7YmtrCxsbG4SHh2P9+vVdrhYUFRUFKysrrF27tt33hw0bBgDIy8vrUTy9lZiYCG9vb+nlb0K6i5IpUWv5+fl4+PBhm/tX48aNg66ursxlWEUbO3YsDAwMelW+jwsVFRVgjMHAwKDDNp6enggJCcGsWbOQkZEBHR2dHo1VUlKCiooK7Nq1Cz/88APGjBnT4b3izkoZSkhiLi8v71E8vVFXV4f09HT85S9/UfnYpO+jAn1ErUmmSrRXbMDExAT19fVKHV9PTw+VlZVKHUPRHj16BACdVi2ysLBAQkICRowY0auxdHR0YG5ujqlTp8LBwQHOzs7S6S9P624pQ319fZl9UKWUlBQwxtpM7yGkOyiZErVmYmICAO0mzZqaGtjY2ChtbJFIpPQxlEGSkDpbBMHc3Fx6bBXFyckJ2trayM/Pl9kuTynD5uZmAP+3D6qUmJiIGTNmKPy4kP6BLvMStebm5oYBAwbgwoULMttzcnLQ3NyMF198UbqNz+dDJBIpbOzs7GwwxmRWwVH0GMpgYWEBHo+H2traDtukp6dLp7LI6/79+3j77bfbbC8oKEBraytsbW0ByFfKUEISs6WlZY9i66lbt27h5MmTdImX9BglU6LWBAIBli1bhn379uHHH39EXV0d8vLysGjRIlhZWSEoKEja1snJCdXV1UhNTYVIJEJlZSVu3brVpk8zMzOUlZWhuLgY9fX10uQoFovx4MEDtLS0IDc3F0uXLoWdnR3mz5+vkDHkLcPXUwYGBnB0dERpaWm77xcWFsLS0rLdh4UCAwNhaWmJixcvdti/UCjEkSNHkJWVhbq6OohEIly6dAnvv/8+hEIhPvvsMwDylTKUkMTs7u7ek13vscTERAwcOLDd+sSEdAclU6L2vvzyS0RFRSEiIgKDBg3CpEmTYG9vj+zsbAiFQmm7xYsXY/LkyZg3bx6GDx+ONWvWSC8Xenp6SlfcWbRoESwsLODq6orp06ejuroawJP7dO7u7tDX14e3tzecnZ1x/PhxmXuPvR1DVXx9fZGfny+dR/o01skczubmZlRUVCAtLa3DNgKBABMmTMCCBQtgbW0NQ0NDBAQEwN7eHmfPnoWbm1uX43Tk/PnzsLa2xsiRI+X+bG/s3LkT8+bN6/GDWITQPFOiUur6/QoKCmJmZmZch9EhyDmPsqCggPH5/C7nfT6rtbWVeXt7s4SEBHlD7LWqqiomEAjYxo0b5f6svMfnaWfPnmUA2Pnz53v0eUIYo3mmhEipU9WS3nJyckJERAQiIiJklvfrTGtrK1JTU1FfX89J9aTw8HCMHj0awcHBKh03MTERzs7OtHwg6RVKpoRoqBUrViAgIACBgYGdPowkkZ2djb179yIjI6PTOarKEBMTg8uXL+PQoUMqvdQqEomQnJwsc1+ckJ6gZEr6vZUrV2LHjh2ora2Fg4MD9uzZw3VIChMZGYng4GCsW7euy7Y+Pj7YuXOnzNrDqpCWlobHjx8jOzsbpqamKh370KFDuH//PubNm6fScYnmoXmmpN+LiopCVFQU12EozdSpUzF16lSuw+jQzJkzMXPmTE7GTkxMxCuvvAJ7e3tOxieag85MCSH90oMHD3DgwAGaW0oUgpIpIaRfSklJAY/Hw1tvvcV1KEQDUDIlhPRLiYmJ8PPzg7GxMdehEA1AyZQQ0u8UFxfj9OnTdImXKEyHDyClpKSoMg7ST5w5cwYAfb96QnLsSO/98MMPMDc3x5QpU7gOhWgIHmOya36lpKR0WeCXEELUTXJyMubMmdNlO8YYnJ2d8cYbbyA2NlYFkZH+oE0yJaSvKysrw/jx4+Hu7o709HTw+TQDjPyfrKws+Pj44PLlyxg1ahTX4RANQfdMicYZMmQI0tLS8Ouvv+Lzzz/nOhyiZrZu3YqJEydSIiUKRcmUaKQXX3wRP/zwA/71r39h69atXIdD1MTdu3exf/9+LFq0iOtQiIahZEo0lr+/P8LDwxESEoKjR49yHQ5RA9999x2MjY0xe/ZsrkMhGobumRKNxhjDe++9h/379+P06dMYMWIE1yERjrS0tMDe3h5/+ctfurVWMSHyoGRKNN6jR4/g4+ODu3fvIicnB+bm5lyHRDjw448/4oMPPkBBQQGtxUsUjpIp6Reqqqrg4eEBKysrHD16FHp6elyHRFSIMYaRI0dizJgx+H//7/9xHQ7RQHTPlPQLgwYNQnp6OvLy8hAUFMR1OETF/vvf/yI/Px9ffPEF16EQDUVnpqRfyczMxBtvvIG1a9di+fLlXIdDVGT8+PGwtbXF3r17uQ6FaCg6MyX9yrRp07Bx40asWLECycnJXIdDVCAzMxPnz5/HihUruA6FaDA6MyX90pIlS7Bjxw5kZ2dj/PjxXIdDlIQxBk9PT5iamiIjI4PrcIgGo3XWSL8UHx+PW7duYdasWTh37hxsbGy4Dokowe7du3H+/HmcO3eO61CIhqMzU9Jv1dXVYcKECdDV1cWvv/4KoVDIdUhEgUQiEUaMGAEPDw96gpcoHd0zJf2WkZER9u/fj9LSUrz33nsQi8Vch0QU6Ntvv8WtW7cQHh7OdSikH6BkSvo1BwcH7N27FwcPHkRYWBjX4RAFqa+vR2RkJIKDg+Ho6Mh1OKQfoGRK+r2JEydi27ZtiIqKwvbt27kOhyhAREQERCIRPcFLVIYeQCIEwHvvvYfff/8dS5YsgZOTEyZPnsx1SKSH8vPz8a9//Qvx8fEwMzPjOhzST9ADSIT8L8YY5s2bh6NHj+Ls2bNwcnLiOiQiJ8YYfHx8UFdXh5ycHGhra3MdEuknKJkS8pSmpia88sorqKurw5kzZ2BiYsJ1SEQOiYmJmD9/Pk6fPo2XXnqJ63BIP0LJlJBn3L17Fy+99BJcXFxw6NAh8Pl0N6QvqKmpgYuLC/z8/PDNN99wHQ7pZ+gBJEKeYWVlhbS0NJw+fRqLFi3iOhzSTX//+9/BGENUVBTXoZB+iJIpIe0YM2YMEhMT8f3332PTpk1ch0O6cPToUXz//ffYsmULTE1NuQ6H9EN0mZeQTnz99ddYtWoVUlNT8cYbb3AdDmlHbW0t3N3dMXHiROzatYvrcEg/RcmUkC789a9/RVJSEk6dOgV3d3euwyHP+PDDD3HgwAFcvXoVFhYWXIdD+ilKpoR0QSQSYdq0afjzzz9x9uxZWFpach0S+V+HDh2Cr68v9u3bBz8/P67DIf0YJVNCuuH+/fvw9PSEubk5jh07BoFAwHVI/V55eTlGjRqFKVOmIDExketwSD9HyZSQbrp+/Tq8vLwwZcoUJCUlgcfjcR1Sv8UYw4wZM3Dt2jVcunQJRkZGXIdE+jl6mpeQbnJxccF///tfpKamYu3atVyH06/Fxsbi8OHD+PHHHymRErVAZ6aEyCkhIQELFy7Ezp07MW/ePK7D6XcuXrwILy8vfPnll7SQPVEblEwJ6YGlS5fiu+++w/Hjx+Hh4cF1OP1GXV0dxo0bBxsbG/z888/Q0qKLa0Q9UDIlpAfEYjFmzZqF8+fPIycnB3Z2dlyHpPEYY/D398fp06fx22+/YciQIVyHRIgU/VlHSA9oaWlh586dsLCwwPTp01FXV8d1SBpv/fr12L9/P5KSkiiRErVDyZSQHjI0NMT+/ftRVVWFwMBAtLa2ch2Sxjp+/DhWrVqF6OhoTJo0ietwCGmDLvMS0ksXLlzApEmTsHjxYmzYsIHrcDROaWkpXnzxRUyaNAnJyck0JYmoJTozJaSXxo4di//85z/45z//iW+//bbN+z///DM+//xzDiLrO4qKivDw4cM225uamuDn54eBAwciISGBEilRW5RMCVGAgIAAhIWFITg4GMeOHZNu37p1K1577TVs3boVDQ0NHEao3sLCwvDyyy+jqqpKuo0xho8++gg3b95EWloaDA0NOYyQkM7RZV5CFIQxhnfffReHDx/GyZMn8c0330jLt2lpaeH777/H+++/z3GU6ufevXuwtbVFa2srHBwccPz4cdjZ2SEiIgJr1qzB4cOH4ePjw3WYhHSKkikhCtTU1ISXX34Zd+/exb1796QPJWlpacHDwwOnTp3iOEL1Ex4ejsjISLS0tEBHRwfGxsZYuXIl/v73v2Pz5s1UoJ30CZRMCVGg0tJSTJkyBUVFRRCJRDLv8Xg83LhxA05OThxFp36am5sxZMgQ3L9/X7qNz+dDS0tLWg2GkL6A7pkSoiBnz57F6NGj202kAKCjo4MffviBg8jUV0pKCqqrq2W2tbS0oLW1FYcOHUJGRgZHkREiHzozJUQBdu7ciQ8++ABisbjT+aaDBw9GaWkptLW1VRid+nrxxRdx5cqVdo8Zj8cDj8dDQkIC5s+fr/rgCJEDnZkS0ksikQgnT55Ea2trl1M37t27h6NHj6ooMvV27tw5XLx4scM/PhhjEIvF+PDDD6UPchGiriiZEtJLOjo6+Oabb/Dbb79h7Nix0jOq9vD5fCQkJKg4QvX0r3/9Czo6Op220dbWxuDBg6nMGlF7dJmXEAVijGHPnj0IDg5GVVUVWlpa2rTh8/m4e/cuBg0axEGE6uHu3buws7Nr9/gAT/5A0dfXx8qVKxESEgKBQKDiCAmRD52ZEqJAPB4PAQEBKCwsRGhoKHR0dKCrq9umXXJyMgfRqY/vvvuu3e06OjrQ09PDZ599hlu3bmH58uWUSEmfQGemhChRQUEB/va3vyEzMxNaWloQi8Xg8Xhwc3NDbm4u1+Fxor3pMDo6OmhtbcVHH32E8PBwqgpD+hw6MyVEiYYNG4bDhw9j3759sLKyAp/PB2MMeXl5uHLlCtfhceLp6TB8Ph8AMGXKFPz+++/Ytm0bJVLSJ1EyJUQF/Pz8UFBQgFWrVkkv++7YsYPjqLgRGxsLyQUxLy8vnDt3DgcPHoSzszPHkRHSc2p3mZeqQhBCCFFn/v7+2L17t8w2PkexdGrp0qXw9PTkOgxClCo3NxfGxsZ47rnnlDpObGwsAODTTz9V6jjd8dNPP8Hc3ByvvvoqtLTU98LYmTNnEBcX1+8fFCNtSf4/PUstz0yTk5MxZ84crkMhRCMEBAQAQJu/pLkgFovVOolKpKSkYO7cuVCzX49EDXT0/0n9v9WEEI3RFxIpIT1B32xCCCGklyiZEkIIIb1EyZQQQgjpJUqmhBBCSC9RMiWEdMuhQ4dgbGyM9PR0rkNRSx9//LG0YhCPx8O7777bps3Ro0exYsUKiMVi+Pn5wc7ODgKBANbW1pg5c2aPl5jctWsXxo0bB0NDQzz33HP44IMPcO/ePen7+/fvR3R0dKe1duWhjP2Ijo6Gi4sL9PX1IRQK4eLigrCwMNTV1cm0i4iIgKurK4yMjKCnpwcnJyd88cUXePjwobRNR/ubmpoq8zNSZLEJSqaEkG6haSJdMzMzQ0ZGBv744482pfa+/PJLxMfHY+XKlRCLxThx4gR27dqF6upqnDx5Ek1NTXj55ZdRVlYm15jJycl45513EBAQgNLSUqSlpeHXX3/F66+/Lq3KM2PGDAgEAvj4+KCmpqZX+6is/Thx4gQWLlyI27dvo7y8HGvWrEF0dDT8/f1l2mVlZeGTTz5BcXExqqqqEBUVhbi4OOmUlc72d+bMmSgtLcWvv/6K6dOn9+o4tMHUDACWnJzMdRiEaAx/f3/m7+/PdRgK1djYyDw9PZXWf3JyMpP312NQUBCztrZu971169YxZ2dn1tTUxBhjTCQSsTfeeEOmzblz5xgAFhkZKde4kydPZkOGDGFisVi6bfPmzQwAO3nypEzb4OBg5unpyUQikVxjqGI//Pz8pP1KBAQEMACsrKxMus3X15e1tLTItJszZw4DwG7fvi2zvbP9DQkJYQMHDpQrRsY6/v9EZ6aEkD4nISEBFRUVXIfRLZnOfF0AACAASURBVIWFhQgLC8NXX30lLSfH5/PbXC53dHQEABQVFcnVf0lJCaysrGSWYrW1tQUA3Lp1S6ZteHg4Ll++jLi4OLXbj3379rUpt2dtbQ0AMpdwDxw4AG1tbZl2ksu1jY2NMtt7s7/yomRKCOnSyZMnYWdnBx6Ph82bNwMAtm7dCqFQCAMDA6SlpeH111+HkZERbGxs8NNPP0k/Gx8fD4FAAAsLC3z88cewsrKCQCCAl5cXcnJypO2Cg4Ohq6uLwYMHS7ctWbIEQqEQPB4PVVVVAJ4sN7ps2TIUFRWBx+PByckJAHD48GEYGRkhMjJSFYek2+Lj48EYw4wZMzpt19TUBAAwMjKSq39HR8c2f1hI7pdKEpuEqakpJk2ahLi4OLkv2yt7P9pTUFAAExOTLpfcvHPnDvT19eHg4CCzvTf7Ky9KpoSQLk2cOBGnT5+W2bZ48WJ8+umnaGpqgqGhIZKTk1FUVARHR0csXLgQIpEIwJMkOX/+fDQ2NiIkJATFxcW4ePEiWlpaMGXKFJSUlAB48sv62WVEt2zZgq+++kpmW1xcHN58800MHToUjDEUFhYCgPRhE7FYrJRj0FMHDx7E8OHDYWBg0Gm7c+fOAXhyrOWxcuVK3Lt3D5s2bUJ9fT3y8/MRFxeHadOmwcPDo037MWPG4M6dO3KXAFT2fkiIRCLcuXMHmzdvxtGjR7Fp0yZppaX2NDY2IisrCwsXLmy3XU/3V16UTAkhvebl5QUjIyOYm5sjMDAQDQ0NuH37tkwbPp+P559/Hnp6enB1dcXWrVtRX1+vsFJ0vr6+qKurQ1hYmEL6U4SGhgb8+eefGDp0aIdtysvLkZSUhJCQEHh6enZ55vesSZMmYfny5QgODoaRkRHc3NxQX1+Pf//73+22HzZsGAAgLy9PrfZDwtbWFjY2NggPD8f69esxd+7cTttHRUXBysoKa9eubff9nuxvT1AyJYQolOTsQHJm2pGxY8fCwMAA169fV0VYnKioqABjrNOzOU9PT4SEhGDWrFnIyMiAjo6OXGOEhoZi27ZtOHbsGB4+fIibN2/Cy8sLnp6e0rP+p0liKS8vV6v9kCgpKUFFRQV27dqFH374AWPGjOnw/vi+ffuQkpKCzMxMGBoattumJ/vbE5RMCSGc0dPTQ2VlJddhKM2jR48APNnPjlhYWCArKwubNm2CsbGxXP3fvXsX0dHR+Otf/4pXX30VQqEQDg4O2L59O8rKyrBhw4Y2n9HX15eJTR3242k6OjowNzfH1KlTkZSUhPz8fERFRbVpl5SUhK+//hrZ2dmwt7fvsL+e7G9PqGU9U0KI5hOJRKipqYGNjQ3XoSiN5Bd5Z4slmJubw8TEpEf9FxQUoLW1FUOGDJHZbmRkBDMzM+Tn57f5THNzs0xs3aHs/eiIk5MTtLW12+zHpk2bkJmZiaysLAwYMKDTPnqyvz1BZ6aEEE5kZ2eDMSbzkAyfz+/y8nBfYmFhAR6Ph9ra2g7bpKenS6eAyEvyh8jdu3dlttfX16O6ulo6ReZpklgsLS27PY6y9+P+/ft4++2322yX/LEg2Q/GGJYvX468vDykpqZ2mUiBnu1vT1AyJYSohFgsxoMHD9DS0oLc3FwsXboUdnZ2mD9/vrSNk5MTqqurkZqaCpFIhMrKyjZzJYEnKw2VlZWhuLgY9fX1EIlEyMjIULupMQYGBnB0dERpaWm77xcWFsLS0rLdh2wCAwNhaWmJixcvdti/g4MDJk+ejO3bt+PXX39FU1MTSkpKEBQUBAD46KOP2nxGEou7u3u3x1H2fgiFQhw5cgRZWVmoq6uDSCTCpUuX8P7770MoFOKzzz4DAFy7dg3r16/H9u3boaOjI7M0II/Hw8aNG7vcX2WhZEoI6dLmzZsxbtw4AMDy5csxc+ZMbN26FbGxsQCAkSNH4ubNm9i+fTuWLVsGAHjttddQUFAg7ePRo0dwd3eHvr4+vL294ezsjOPHj8vch1u8eDEmT56MefPmYfjw4VizZo308tzTD9QsWrQIFhYWcHV1xfTp01FdXa2S49ATvr6+yM/Pl86/fFpncx+bm5tRUVGBtLS0DtvweDzs3r0bgYGB+Oijj2BqagpXV1fcvn0be/fuhbe3d5vPnD9/HtbW1hg5cmS3x1H2fggEAkyYMAELFiyAtbU1DA0NERAQAHt7e5w9exZubm5djtORZ/dXaeReS0nJQMsJEqJQ6rCcYFBQEDMzM+M0BnkocjnBgoICxufzWWJiolz9tba2Mm9vb5aQkCDX5zpTVVXFBAIB27hxo9zjqNN+dFd7+ytBywkSQvokRVUsUWdNTU3IzMxEQUGB9MEXJycnREREICIiQmZZvM60trYiNTUV9fX1CAwMVFh84eHhGD16NIKDg+UeR532o7ue3V/GGMrKynDy5EnpYh+KQsmUKNQff/yBv/3tbxgxYgQMDQ3B5/NhbGwMZ2dn+Pr64syZM1yHSIjSVFdX47XXXoOzszM+/PBD6fYVK1YgICAAgYGBnT7EI5GdnY29e/ciIyOjyxWHuismJgaXL1/GoUOHpHNA5R1HHfaju9rb37S0NFhbW8Pb2xsHDx5U7IA9OXVWJtBl3j7r3//+N9PR0WEvv/wyO3z4MHvw4AF79OgRKyoqYklJSczLy4t99913XIfZ73B9mXfFihVMV1eXAWD29vZs9+7dnMXSXT25zNsdmZmZbPny5QrvtyupqaksKiqqTbWVnuJqP7pL0fv7tI7+P1EyVTFll47iauwzZ84wbW1t9uqrr3ZY3unw4cNs06ZNShlfETT1Z8N1Mu2LlJVMSd/X0f8nWrRBxbgsHaXMsdeuXYvW1lasW7cOfH77X6tp06Zh2rRpShlfETT1Z0MIUT6NuGeamJiIsWPHQiAQQCgUwt7eHmvWrAHw5IZzTEyMdIFtU1NTzJo1S2Y90O6WkurOeCdOnICrqyuMjY0hEAjg7u6OzMxMAB2XjmptbcXq1athZ2cHfX19jBw5EsnJyXLHpuixge6VtWpubsaxY8cwcOBAjB8/vts/N/rZ9O5nQwhRIyo/R+4C5LzMGxsbywCwdevWsfv377Pq6mr23XffsXfeeYcxxtjq1auZrq4uS0xMZDU1NSw3N5e98MILbNCgQezevXvSfkJDQxkAduzYMVZbW8sqKiqYt7c3EwqFrLm5udvj7d69m4WHh7Pq6mp2//595uHhIfP49ezZs9nQoUNl9uHvf/8709PTY3v27GEPHjxgK1euZFpaWuz8+fNyxaaMsQ8cOMAMDQ1ZREREhz+DGzduMADMw8Ojez+0/0U/m96N3V10mVd+dJmXdEQj75k2NzczExMTNnnyZJntLS0tLC4ujjU2NrIBAwawwMBAmffPnTvHAMgkCMkvxaamJum2LVu2MACssLCwW+O1JyoqigFgFRUVjLG2vzSbmpqYgYGBTIyNjY1MT0+PLV68uNuxKWvs7rhw4QIDwP7nf/6n25+hn41qfjaMUTLtCUqmpCMaec80NzcXNTU1be7DaWtrIyQkBBcuXMDDhw8xduxYmffHjRsHXV1d5OTkdNr/s6WkuhqvPZJHsjuaY/fHH3+gsbFRusIH8GRB5sGDB3damqo7Za6UNfazJOtjNjY2dvsz+fn59LNRwtgdKS0tRUpKityf668kU7jomJFnlZaWtlucoU8n07q6OgDosFJBTU0NALS7GLKJiQnq6+sVOh7wpBr9hg0bkJ+fL11jsjMNDQ0AgFWrVmHVqlUy71lZWckVH1dj29vbQyAQ4MaNG93+DP1sVDc2AJw9e7bLIsukLTpmpD3+/v5ttvXpB5AkZYeqqqrafV/yi7W9X8w9Kf3U1Xi3b9+Gn58fBg8ejJycHNTW1iI6OrrTPs3NzQEAsbGxYE8uu0tf8ixwwOXYenp6mDZtGqqqqnDq1KkO21VXV2PBggUA6GejqrEl/P392/RDr45fkge9uI6DXur3ai+RAn08mdrb28PMzAxHjhxp9303NzcMGDAAF/5/e3caFNWV9gH839JA0wiKBpCgGBbFiLgkJmUjaCxGK0KUcUExcabIMnGbAqPJS4A4CorL6CDFKKZ0CNYkRsClUKMkKWOIwUSjpQjixAjuOrIEZW2koc/7weqedBqQphsa2v+vig+ee+59nr60PHXucs65czrtZ86cQVNTE1588UWTxisqKoJKpcLSpUvh5eUFmUwGiUTS7jGHDBkCmUyGgoICg3LpSbGBx9N22draYsWKFa1OhA0Aly5d0r42w99N9/1uiKjr9epiamtri7i4OJw8eRJRUVG4e/cu1Go1amtrcfnyZchkMqxcuRIHDx7EZ599hpqaGhQVFWHJkiVwc3PTLlNkqngeHh4AgOPHj6OxsRFXr17Vu/f3+6WjrKys8Oabb2Lv3r1IS0tDTU0NWlpacOfOHb01CtvTVbE7uqzV2LFjsWfPHly6dAlBQUE4duwYqquroVKpcP36dezatQtvv/229l4hfzfdE5uIuonoYdCJGZC2bdsm/P39hUwmEzKZTIwbN05s375dCCGEWq0WmzdvFsOGDRPW1tbCyclJzJo1S1y5ckW7//bt24VcLhcAxLBhw0RpaanYuXOncHR0FADE0KFDxS+//NKheDExMWLAgAGif//+Ijw8XGzbtk0AEN7e3uLWrVvi/PnzYujQocLOzk4EBgaK+/fvi0ePHomYmBjh4eEhpFKpcHZ2FnPmzBHFxcUG5Wbq2EIIcezYMeHg4CDWrVvXod/FrVu3xPvvvy/8/f1F3759hZWVlejfv78YN26cePvtt8WpU6e0ffm7Me5301F8mtdwfJqX2tLW/yeJEMLwBeK6kEQiQVZWFubNm2fuVIgsQnh4OABg3759Zs6k98jOzsb8+fPRw/48Ug/Q1v+nXn2Zl4iIqCdgMSUiIjISiykRkQU5fvw4YmNjoVarMWvWLHh4eEAmk8Hd3R1hYWEoLCw0OkZjYyNGjBih9w40AOTn52PixImQy+Vwc3NDTEwMHj16pN1++PBhbNq0yeIWi2cxJSKyEKtXr0Zqairi4uKgVqvx/fff4/PPP0dVVRXy8/OhVCoxadIk3Lt3z6g48fHxuHLlil57cXExpk2bhuDgYFRUVODgwYP45JNPsGTJEm2fmTNnQiaTITg4WDt5iyVgMSWiLqdUKhEQENDrY/RkGzduRGZmJrKzs+Hg4AAAUCgUCAwMhFwuh6enJ5KSklBdXY3du3d3Os4PP/yAS5cutbpt7dq1GDRoEBISEmBvbw+FQoGYmBjs3r1bZxrM6OhojBkzBiEhIWhubu50Lj0JiykRdbnuWK/1aV4TtqSkBKtWrUJCQgJkMhkAQCqV4siRIzr9vLy8AAClpaWdiqNUKvHBBx8gJSVFb1tzczOOHj2KyZMn60xKMn36dAghcOjQIZ3+a9asQUFBQavH6o1YTIlIjxBPXms2KioKNjY2GDRokLZt2bJlsLe3h0Qi0U7t2Np6rampqZDJZHBxccHixYvh5uYGmUyGgIAAnQktjIkBdGw9XkuQmpoKIQRmzpzZbj/N7GSOjo6dihMfH49ly5Zpp7v8rWvXrqGurk47SYmGt7c3AOjdq3VycsLkyZORkpJiEa8gsZgSkZ41a9YgNjYW8fHxKC8vx8mTJ3H79m0EBQWhrKwMwOM/4L9/H3z79u1ISEjQaUtJScGMGTPg7e0NIQRKSkoQFRWFyMhINDQ0IDo6Gjdu3MD58+fR3NyMqVOn4vbt20bHAP63Ko9arTbdyemBjh49Cl9fX8jl8nb7/fTTTwCAwMBAg2OcOnUKpaWleP3111vdfv/+fQDQXmLWkMlksLOz035vfmvcuHG4e/cuLl68aHA+PQ2LKRHpUCqVSE5OxuzZs7Fw4UL069cP/v7++Pjjj1FZWYmdO3eaLJZUKtWOfkeOHIm0tDTU1tYiIyPDJMcPDQ1FTU0NVq1aZZLj9UT19fW4fv26dgTYmrKyMmRmZiI6OhoKheKJI9jfUyqVWL58OdLS0trso3li18rKSm+btbV1q3N2Dxs2DMDj+at7u169BBsRmZ6xa80aY/z48ZDL5Z1as/VpVV5eDiFEu6NShUKB+vp6zJs3D+vWrdPOkd1RcXFxePfdd+Hu7t5mH8292tYeKGpqaoKdnZ1euybn1katvQ2LKRHpMPVas4aytbVFRUVFl8awJI2NjQAen7e2uLi4ID09HX5+fgYfPz8/H0VFRUhOTm63n+a+tmZtYY2GhgY0Nja2ug6vpsBqPkNvxsu8RKTD1GvNGkKlUnV5DEujKUjtTYLg7Oys/b0aKj09Hd988w369OkDiUQCiUSifQApKSkJEokE586dg6enJxwcHHDz5k2d/TX3r0ePHq137KamJp3P0JuxmBKRDkPWmpVKpVCpVCaLnZeXByEEJkyY0GUxLI2LiwskEgmqq6vb7HPkyJF2L9G2JyMjQ2+BbM2Vg/j4eAghMH78eEilUoSEhODkyZM6D3zl5uZCIpG0ep9Wk7Orq2uncutJWEyJSIcha836+PigqqoKOTk5UKlUqKio0BuZAPrrtWqKo1qtxoMHD9Dc3IzCwkIsX74cHh4eiIyMNEmMjq7H25vJ5XJ4eXnhzp07rW4vKSmBq6sr5s+fr7ctIiICrq6uOH/+vElyWbVqFcrKyrB69WrU19fjxx9/xObNmxEZGQlfX1+9/pqc/f39TRLfnFhMiUjP6tWrsX79eiQmJuKZZ57B5MmT8dxzzyEvLw/29vbafkuXLsWUKVOwYMEC+Pr6Yu3atdpLdgqFQvuKy5IlS+Di4oKRI0ciJCQEVVVVAB7fK/P394ednR2CgoIwfPhwfPvttzr3/4yN8TQIDQ1FcXFxq0/MtvcOZ1NTE8rLy/UmVOgsPz8/fPXVV/j6668xcOBAzJkzB2+99RZ27NjRav+zZ8/C3d291UvAvU7XL6VqGHRicXAialtPXRx80aJFYsCAAeZOo1W9bXHwq1evCqlUKj799FOD9mtpaRFBQUEiPT29izJrW2VlpZDJZGLLli3dHtsYbf1/4siUiMzG0lYOMRcfHx8kJiYiMTERdXV1HdqnpaUFOTk5qK2tRURERBdnqG/NmjUYO3YsoqKiuj12V2AxJSKyALGxsQgPD0dERES7DyNp5OXl4cCBA8jNzX3izEmmlpycjIKCAhw7dszgd157KhZTIup2cXFxyMjIQHV1NTw9PbF//35zp2QRkpKSEBUVhQ0bNjyxb3BwMPbs2aMz73F3OHToEB49eoS8vDw4OTl1a+yuxEkbiKjbrV+/HuvXrzd3GhZp2rRpmDZtmrnTaFNYWBjCwsLMnYbJcWRKRERkJBZTIiIiI7GYEhERGYnFlIiIyEgSIXrWEucSiQQTJkzgRNdEJnL69GkA0Jnvltp3584dnD59GnPnzjV3KtTDnD59GhMmTMC+fft02ntcMQ0PDzd3CkQW4/79+7hw4QKmT59u7lSILIZCocCKFSt02npcMSUi08nOzsb8+fPbnZ+ViIzHe6ZERERGYjElIiIyEospERGRkVhMiYiIjMRiSkREZCQWUyIiIiOxmBIRERmJxZSIiMhILKZERERGYjElIiIyEospERGRkVhMiYiIjMRiSkREZCQWUyIiIiOxmBIRERmJxZSIiMhILKZERERGYjElIiIyEospERGRkVhMiYiIjMRiSkREZCQWUyIiIiOxmBIRERmJxZSIiMhILKZERERGYjElIiIyEospERGRkVhMiYiIjMRiSkREZCQWUyIiIiOxmBIRERmJxZSIiMhIUnMnQESmoVKpUFdXp9NWX18PAHjw4IFOu0QiQf/+/bstNyJLJxFCCHMnQUTGKysrg7u7O1paWp7Yd8qUKThx4kQ3ZEX0dOBlXiIL4erqikmTJqFPn/b/W0skEixYsKCbsiJ6OrCYElmQP/3pT0/sY2VlhdmzZ3dDNkRPDxZTIgsyZ84cSKVtPwphZWWFV199FQMHDuzGrIgsH4spkQVxdHTE9OnT2yyoQggsXLiwm7MisnwspkQWZuHChW0+hGRjY4PXXnutmzMisnwspkQW5rXXXoNcLtdrt7a2xqxZs2Bvb2+GrIgsG4spkYWRyWSYPXs2rK2tddpVKhXeeOMNM2VFZNlYTIks0Ouvvw6VSqXT5ujoiKlTp5opIyLLxmJKZIH+8Ic/YMCAAdp/W1tbY8GCBbCxsTFjVkSWi8WUyAJJpVIsWLBAe6lXpVLh9ddfN3NWRJaL0wkSWahTp04hMDAQwOPZke7du/fE2ZGIqHP4P4vIQgUEBMDd3R0A8Oc//5mFlKgLcdWYHiI7O9vcKZAFeumll3D37l0MHDiQ3zEyuSFDhkChUJg7jR6Bl3l7CIlEYu4UiIgMMnfuXOzbt8/cafQIHJn2IFlZWZg3b5650yALs3//foSHh/P7ZaDw8HAAYLFog+b80GO8iUJk4ebOnWvuFIgsHospERGRkVhMiYiIjMRiSkREZCQWUyIiIiOxmBIRERmJxZSIOuzYsWPo168fjhw5Yu5Uerzjx48jNjYWarUas2bNgoeHB2QyGdzd3REWFobCwkKjYzQ2NmLEiBH46KOP9Lbl5+dj4sSJkMvlcHNzQ0xMDB49eqTdfvjwYWzatKnNheTJMCymRNRhnOOlY1avXo3U1FTExcVBrVbj+++/x+eff46qqirk5+dDqVRi0qRJuHfvnlFx4uPjceXKFb324uJiTJs2DcHBwaioqMDBgwfxySefYMmSJdo+M2fOhEwmQ3BwMB4+fGhUHsRiSkQGCA0NRXV1NWbMmGHuVKBUKhEQEGDuNPRs3LgRmZmZyM7OhoODAwBAoVAgMDAQcrkcnp6eSEpKQnV1NXbv3t3pOD/88AMuXbrU6ra1a9di0KBBSEhIgL29PRQKBWJiYrB79278/PPP2n7R0dEYM2YMQkJC0Nzc3OlciMWUiHqp9PR0lJeXmzsNHSUlJVi1ahUSEhIgk8kAPF4O7/eXxb28vAAApaWlnYqjVCrxwQcfICUlRW9bc3Mzjh49ismTJ+tMUzp9+nQIIXDo0CGd/mvWrEFBQUGrx6KOYzElog7Jz8+Hh4cHJBIJtm3bBgBIS0uDvb095HI5Dh06hOnTp8PR0RGDBw/G3r17tfumpqZCJpPBxcUFixcvhpubG2QyGQICAnDmzBltv6ioKNjY2GDQoEHatmXLlsHe3h4SiQSVlZUAgOXLl2PlypUoLS2FRCKBj48PAODLL7+Eo6MjkpKSuuOU6ElNTYUQAjNnzmy3n1KpBAA4Ojp2Kk58fDyWLVsGZ2dnvW3Xrl1DXV0dPDw8dNq9vb0BQO9erZOTEyZPnoyUlBRexjcCiykRdUhgYCB++OEHnbalS5fivffeg1KphIODA7KyslBaWgovLy/85S9/gUqlAvC4SEZGRqKhoQHR0dG4ceMGzp8/j+bmZkydOhW3b98G8LgY/X7+4O3btyMhIUGnLSUlBTNmzIC3tzeEECgpKQEA7cM0arW6S87Bkxw9ehS+vr6Qy+Xt9vvpp58AQLverCFOnTqF0tLSNhd7v3//PgBoLzFryGQy2NnZoaysTG+fcePG4e7du7h48aLB+dBjLKZEZBIBAQFwdHSEs7MzIiIiUF9fj1u3bun0kUqleP7552Fra4uRI0ciLS0NtbW1yMjIMEkOoaGhqKmpwapVq0xyPEPU19fj+vXr2hFga8rKypCZmYno6GgoFIonjmB/T6lUYvny5UhLS2uzj+aJXSsrK71t1tbW2lHxbw0bNgwAUFRUZFA+9D9cNYaITM7GxgYAtCPTtowfPx5yuVznoZjeqry8HEKIdkelCoUC9fX1mDdvHtatWwdra2uDYsTFxeHdd9/VLvreGs292tYeKGpqaoKdnZ1euybn1kat1DEspkRkVra2tqioqDB3GkZrbGwE8PjztMXFxQXp6enw8/Mz+Pj5+fkoKipCcnJyu/0095tramp02hsaGtDY2Ag3Nze9fTQFVvMZyHC8zEtEZqNSqfDw4UMMHjzY3KkYTVOQ2psEwdnZGf379+/U8dPT0/HNN9+gT58+kEgkkEgk2geQkpKSIJFIcO7cOXh6esLBwQE3b97U2V9zX3n06NF6x25qatL5DGQ4FlMiMpu8vDwIITBhwgRtm1QqfeLl4Z7IxcUFEokE1dXVbfY5cuRIu5do25ORkQEhhM6PZkQfHx8PIQTGjx8PqVSKkJAQnDx5UudBrNzcXEgkklbv02pydnV17VRuxGJKRN1IrVbjwYMHaG5uRmFhIZYvXw4PDw9ERkZq+/j4+KCqqgo5OTlQqVSoqKjQG2UBwIABA3Dv3j3cuHEDtbW1UKlUyM3NNdurMXK5HF5eXrhz506r20tKSuDq6or58+frbYuIiICrqyvOnz9vklxWrVqFsrIyrF69GvX19fjxxx+xefNmREZGwtfXV6+/Jmd/f3+TxH8asZgSUYds27YNL730EgAgJiYGYWFhSEtLw9atWwE8vnx47do17Nq1CytXrgQAvPrqq7h69ar2GI2NjfD394ednR2CgoIwfPhwfPvttzr3GZcuXYopU6ZgwYIF8PX1xdq1a7WXHxUKhfY1miVLlsDFxQUjR45ESEgIqqqquuU8tCc0NBTFxcWtPjHb3jucTU1NKC8v15tQobP8/Pzw1Vdf4euvv8bAgQMxZ84cvPXWW9ixY0er/c+ePQt3d/dWLwFTBwnqEQCIrKwsc6dBFqonfL8WLVokBgwYYNYcDDF37lwxd+5cg/a5evWqkEql4tNPPzVov5aWFhEUFCTS09MN2s8UKisrhUwmE1u2bDFov86cH0vGkSkRdRtLX6HEx8cHiYmJSExMRF1dXYf2aWlpQU5ODmpraxEREdHFGepbs2YNxo4di6ioqG6PbUlYTC3EO++8AwcHB0gkEhQUFJg7nU5JTEzEyJEj4ejoCFtbW/j4+OD//u//zB4gjQAADi5JREFUWv2jpFKpsH79evj4+MDGxgb9+/fHqFGjcOPGDYNiHjhwAF5eXtqnIzU/NjY2cHFxwSuvvILNmzfjwYMHJvqUZOliY2MRHh6OiIiIdh9G0sjLy8OBAweQm5v7xJmTTC05ORkFBQU4duyYwe+8ki4WUwvxr3/9C7t27TJ3GkY5ceIE/vrXv+LGjRuorKzE+vXrkZKSgvDwcL2+8+fPx7///W/s2bMHDQ0N+M9//gNvb+8OjwY05syZg2vXrsHb2xv9+vWDEAJqtRrl5eXIzs6Gp6cnYmJi4Ofnh3Pnzpnqoz514uLikJGRgerqanh6emL//v3mTqlLJSUlISoqChs2bHhi3+DgYOzZs0dnPuLucOjQITx69Ah5eXlwcnLq1tgWydzXmekxmOCe1t69ewUAceHCBRNl1b1CQ0NFc3OzTtu8efMEAHHr1i1t2969e4VEIhGFhYUmi+3t7S369evX6rZ9+/aJPn36CBcXF/Hw4UOTxexOpvh+PW14T7B9PD+6ODK1IL9dbqk3+uKLL/TmE33mmWcAPJ69RWPHjh144YUXuu0x/rlz5yIyMhLl5eX4+OOPuyUmEfUuLKa9lBACmzdvhq+vL2xtbdGvXz988MEHev1aWlrwt7/9DR4eHrCzs8Po0aORlZUFoOPLZwHAd999h5dffhlyuRyOjo7w9/fXTlfWXgxj3b17F3Z2dvD09ATw+BWC06dPY+zYsU/c15TLcWneg8zNzdW29fZzS0QmZO6hMT0GAy/DxcfHC4lEIv7xj3+IBw8eiIaGBrF9+3a9y7zvv/++sLW1Ffv37xcPHjwQcXFxok+fPuLs2bPa4wAQ33zzjaiurhbl5eUiKChI2Nvbi6amJiGEEHV1dcLR0VFs2rRJKJVKcf/+fTF79mxRUVHRoRidVV9fLxwcHERUVJS27fr16wKAGDt2rHjllVfEoEGDhK2trRgxYoTYtm2bUKvV2r5ffPGFcHBwEImJiU+M1d5lXiGEqKmpEQDEkCFDtG296dwa+v0iXsZ8Ep4fXSymPYQhf+waGhqEXC4XU6dO1Wn//T1TpVIp5HK5iIiI0NnX1tZWLF26VAjxvz/4SqVS20dTlEtKSoQQQly6dEkAEF988YVeLh2J0Vnx8fFi+PDhoqamRttWVFQkAIipU6eKU6dOiV9//VU8fPhQfPjhhwKA+OyzzzoV60nFVAghJBKJ6N+/vxCi951bFlPDsVi0j+dHF1eN6YVKSkrQ0NCA4ODgdvtduXIFDQ0NGDVqlLbNzs4OgwYNanfJq98vn+Xl5QUXFxcsXLgQ0dHRiIyMxHPPPWdUjCc5ePAgsrOz8fXXX+sscqyZKcfPzw8BAQHa9oSEBOzYsQM7d+7EG2+80em4bamvr4cQAo6OjgB657ndunUr9u3bZ/B+T6vTp08DQKtPk9Pj8/PbOZWfdrxn2gtp5tHUrBjRlvr6egDARx99pPMO5c2bN3Ue6HkSOzs7nDhxAoGBgUhKSoKXlxciIiKgVCpNFuO3MjMzsXHjRuTl5WkLi4Zm+ajKykqddhsbGwwdOhSlpaWdivkkv/zyCwBgxIgRAHrvuSWirsGRaS+kWfz30aNH7fbTFNutW7di+fLlRsX08/PDkSNHUFFRgeTkZGzcuBF+fn7aGVtMEQMA/vnPf+Krr77CiRMn0LdvX73tffv2xbBhw3D58mW9bc3NzejXr5/RObTmyy+/BABMnz4dQO88t++99x7mzZtn9HGeFpoRKUfzreOIXRdHpr3QqFGj0KdPH3z33Xft9hsyZAhkMpnRMyLdu3dPW7ycnZ2xYcMGvPDCC7h8+bLJYgghEBMTg6KiIuTk5LRaSDXmz5+PCxcu4Nq1a9q2hoYG3Lx5s0tel7l//z62bt2KwYMH46233gLQu84tEXU9FtNeyNnZGXPmzMH+/fuRnp6OmpoaFBYWYufOnTr9ZDIZ3nzzTezduxdpaWmoqalBS0sL7ty5g//+978djnfv3j0sXrwYP//8M5qamnDhwgXcvHkTEyZMMFmMy5cv4+9//zt27doFa2trven9tmzZou27YsUKDB06FJGRkbh16xZ+/fVXxMTEQKlU4sMPP9T2M3Q5LiEE6urqoFartWtFZmVlYeLEibCyskJOTo72nmlvOrdE1A3M+/wTacDApy1ra2vFO++8IwYOHCj69u0rAgMDxd/+9jcBQAwePFhcvHhRCCHEo0ePRExMjPDw8BBSqVQ4OzuLOXPmiOLiYrF9+3Yhl8sFADFs2DBRWloqdu7cKRwdHQUAMXToUPHLL7+IGzduiICAAOHk5CSsrKzEs88+K+Lj47WzFbUXo6M0T+m29bN582ad/rdv3xYLFiwQTk5OwtbWVrz88ssiNzdXp8+xY8eEg4ODWLduXZtxDx8+LEaPHi3kcrmwsbERffr0EQC0T+6+/PLLIjExUfz66696+/aWcysEn+btDD6t2j6eH10SIdpZZI+6jUQiQVZWFu9pUZfg98twvGfaPp4fXbzMS0REZCQWU+oyP//8s969z9Z+zLGGI1F3O378OGJjY6FWqzFr1ix4eHhAJpPB3d0dYWFhKCws7PSx1Wo1tm7dqvPutcbhw4exadMmi19L1txYTKnLjBgxAuLxLFvt/mRmZpo7VaIutXr1aqSmpiIuLg5qtRrff/89Pv/8c1RVVSE/Px9KpRKTJk3CvXv3DD721atXMWnSJKxYsaLV949nzpwJmUyG4OBgPHz40BQfh1rBYkpEXU6pVLY6auptMTpj48aNyMzMRHZ2tnY2L4VCgcDAQMjlcnh6eiIpKQnV1dXYvXu3Qce+ePEiPvzwQyxZsqTdxR+io6MxZswYhISEoLm52ZiPQ21gMSWiLpeeno7y8vJeH8NQJSUlWLVqFRISErSTrUilUhw5ckSnn5eXFwAYPIPXmDFjcODAAbzxxhvaqTbbsmbNGhQUFCAlJcWgGNQxLKZEpEcIgeTkZDz//POwtbWFk5MT/vjHP+rMCRwVFQUbGxsMGjRI27Zs2TLY29tDIpFop3xcvnw5Vq5cidLSUkgkEvj4+CA1NRUymQwuLi5YvHgx3NzcIJPJEBAQgDNnzpgkBmDaZfg6IzU1FUIIzJw5s91+SqUSALTvMXcFJycnTJ48GSkpKeBLHKbHYkpEetasWYPY2FjEx8ejvLwcJ0+exO3btxEUFISysjIAjwvF71+12b59OxISEnTaUlJSMGPGDHh7e0MIgZKSEkRFRSEyMhINDQ2Ijo7GjRs3cP78eTQ3N2Pq1Km4ffu20TEAaB+6UavVpjs5Bjh69Ch8fX0hl8vb7ffTTz8BAAIDA7s0n3HjxuHu3bu4ePFil8Z5GrGYEpEOpVKJ5ORkzJ49GwsXLkS/fv3g7++Pjz/+GJWVlXozbRlDKpVqR78jR45EWloaamtrkZGRYZLjh4aGoqamBqtWrTLJ8QxRX1+P69evw9vbu80+ZWVlyMzMRHR0NBQKxRNHsMYaNmwYAKCoqKhL4zyNONE9EekoLi5GXV0dxo8fr9P+0ksvwcbGRucyrKmNHz8ecrncqOX7eory8nIIIdodlSoUCtTX12PevHlYt24drK2tuzQnTS6aqwtkOiymRKRD8/pEa4sN9O/fH7W1tV0a39bWFhUVFV0aozs0NjYCQLsPBrm4uCA9PR1+fn7dkpOdnZ1ObmQ6vMxLRDr69+8PAK0WzYcPH2Lw4MFdFlulUnV5jO6iKVztTZbg7OysPd/doampCcD/ciPT4ciUiHSMGjUKffv2xblz53Taz5w5g6amJrz44ovaNqlUCpVKZbLYeXl5EEJgwoQJXRaju7i4uEAikaC6urrNPr9/RaaraXJxdXXt1rhPA45MiUiHTCbDypUrcfDgQXz22WeoqalBUVERlixZAjc3NyxatEjb18fHB1VVVcjJyYFKpUJFRQVu3rypd8wBAwbg3r17uHHjBmpra7XFUa1W48GDB2hubkZhYSGWL18ODw8PREZGmiSGocvwmZJcLoeXlxfu3LnT6vaSkhK4urpi/vz5etsiIiLg6uqK8+fPmzQnTS5dse7v047FlIj0rF69GuvXr0diYiKeeeYZTJ48Gc899xzy8vJgb2+v7bd06VJMmTIFCxYsgK+vL9auXau9hKhQKLSvuCxZsgQuLi4YOXIkQkJCUFVVBeDxvTt/f3/Y2dkhKCgIw4cPx7fffqtzn9HYGOYUGhqK4uJi7Xukv9Xeu55NTU0oLy/HoUOH2j3+6dOnERgYiGeffRZnzpzBxYsX4ebmhokTJ+LkyZN6/c+ePQt3d3eMHj3a8A9D7evmJd+oDeB6k9SFeuL3a9GiRWLAgAHmTqNNpliv8+rVq0IqlYpPP/3UoP1aWlpEUFCQSE9PNyr+b1VWVgqZTCa2bNlikuNxPVNdHJkSkdlY+komPj4+SExMRGJiIurq6jq0T0tLC3JyclBbW2vSFZXWrFmDsWPHIioqymTHpP9hMSUi6kKxsbEIDw9HREREuw8jaeTl5eHAgQPIzc194sxJHZWcnIyCggIcO3asy99lfVqxmBJRt4uLi0NGRgaqq6vh6emJ/fv3mzulLpWUlISoqChs2LDhiX2Dg4OxZ88enfmIjXHo0CE8evQIeXl5cHJyMskxSR9fjSGibrd+/XqsX7/e3Gl0q2nTpmHatGndHjcsLAxhYWHdHvdpw5EpERGRkVhMiYiIjMRiSkREZCQWUyIiIiOxmBIRERlJIkQ7c1pRt5FIJOZOgYjIIHPnzsW+ffvMnUaPwFdjeoisrCxzp0BEZJAhQ4aYO4UegyNTIiIiI/GeKRERkZFYTImIiIzEYkpERGQkKQA+ikVERGSE/wdWMV9UYoL0twAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 121 } ] }, { "cell_type": "code", "metadata": { "id": "vTdcBX7xTeCS", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "4f2d36e8-cd94-46c1-ac9d-5c1f25b01b04" }, "source": [ "history = model.fit(X_train_scaled, y_train, \n", " epochs=30,\n", " batch_size=32,\n", " validation_split=0.1)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Epoch 1/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 2.4045 - val_loss: 0.7922\n", "Epoch 2/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.7079 - val_loss: 0.6603\n", "Epoch 3/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6424 - val_loss: 0.6243\n", "Epoch 4/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6112 - val_loss: 0.6006\n", "Epoch 5/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5875 - val_loss: 0.5798\n", "Epoch 6/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5678 - val_loss: 0.5647\n", "Epoch 7/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5523 - val_loss: 0.5509\n", "Epoch 8/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5387 - val_loss: 0.5397\n", "Epoch 9/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5271 - val_loss: 0.5295\n", "Epoch 10/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5173 - val_loss: 0.5218\n", "Epoch 11/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5093 - val_loss: 0.5136\n", "Epoch 12/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5012 - val_loss: 0.5079\n", "Epoch 13/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4948 - val_loss: 0.5026\n", "Epoch 14/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4895 - val_loss: 0.4976\n", "Epoch 15/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4839 - val_loss: 0.4923\n", "Epoch 16/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4789 - val_loss: 0.4876\n", "Epoch 17/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4748 - val_loss: 0.4836\n", "Epoch 18/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4702 - val_loss: 0.4811\n", "Epoch 19/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4667 - val_loss: 0.4770\n", "Epoch 20/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4634 - val_loss: 0.4725\n", "Epoch 21/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4598 - val_loss: 0.4700\n", "Epoch 22/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4565 - val_loss: 0.4674\n", "Epoch 23/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4530 - val_loss: 0.4656\n", "Epoch 24/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4501 - val_loss: 0.4616\n", "Epoch 25/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4474 - val_loss: 0.4601\n", "Epoch 26/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4454 - val_loss: 0.4569\n", "Epoch 27/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4420 - val_loss: 0.4539\n", "Epoch 28/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4402 - val_loss: 0.4516\n", "Epoch 29/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4375 - val_loss: 0.4490\n", "Epoch 30/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4354 - val_loss: 0.4469\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "BHvy4VW5TsjB", "colab": { "base_uri": "https://localhost:8080/", "height": 287 }, "outputId": "bf193277-5b14-4cd0-bf79-116028fae154" }, "source": [ "history_df = pd.DataFrame(history.history)\n", "history_df[['loss', 'val_loss']].plot(kind='line')" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 123 }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3xdZZ3v8c9vX5K9c2vTNk3pvQzVVtoBNKAeuTojAi8FbyNUROA12DmMIiqHI+OoIOKMIzM6Z5SB4WBFHW4dAaceEOw5opVRmaadQGkLpZRCEwpN70maZN9+54+90u6mSbOb7DbNXt/367Vfe61nrbX3s7rhu9Z6npVnmbsjIiLlLzLaFRARkWNDgS8iEhIKfBGRkFDgi4iEhAJfRCQkFPgiIiExZOCb2Qwze8rM1pnZWjO7foB1zjWzPWbWEry+VrDsAjN70cw2mtlNpd4BEREpTqyIdTLADe6+2sxqgVVmttzd1/Vb77fu/oHCAjOLAncA7wNagZVmtmyAbUVE5Cgb8gzf3be6++pgugNYD0wr8vPPADa6+yZ3TwEPApcMt7IiIjJ8xZzh72dms4HTgGcGWPxuM3sWeB34H+6+lvyBYUvBOq3AO4f6nkmTJvns2bOPpGoiIqG2atWq7e7ecLh1ig58M6sBHgY+7+57+y1eDcxy904zuwj4GTD3SCprZouBxQAzZ86kubn5SDYXEQk1M3t1qHWKukvHzOLkw/4+d3+k/3J33+vuncH040DczCYBbcCMglWnB2WHcPe73b3J3ZsaGg57kBIRkWEo5i4dA34ArHf37wyyzpRgPczsjOBzdwArgblmNsfMKoDLgGWlqryIiBSvmCad9wBXAGvMrCUo+zIwE8Dd7wI+BlxrZhmgG7jM88NwZszss8CTQBRYErTti4jIMWbH4/DITU1NrjZ8kXBJp9O0trbS09Mz2lU5riUSCaZPn048Hj+o3MxWuXvT4bY9ort0RESOltbWVmpra5k9ezZBC7H04+7s2LGD1tZW5syZc8Tba2gFETku9PT0MHHiRIX9YZgZEydOHPZVkAJfRI4bCvuhjeTfqKwC/3v/7yV+s6F9tKshInJcKqvA/5cVm/jNiwp8ERmempqa0a7CUVVWgV+XiLG3Jz3a1RAROS6VV+An4+ztVuCLyMi4OzfeeCMLFixg4cKFPPTQQwBs3bqVs88+m1NPPZUFCxbw29/+lmw2y1VXXbV/3e9+97ujXPvBldVtmXWJuM7wRcrA13++lnWv9x+ya2TeNrWOmz94clHrPvLII7S0tPDss8+yfft2Tj/9dM4++2zuv/9+3v/+9/PXf/3XZLNZ9u3bR0tLC21tbTz//PMA7N69u6T1LqUyO8OP0dGTGe1qiMgY9/TTT7No0SKi0SiNjY2cc845rFy5ktNPP50f/vCH3HLLLaxZs4ba2lpOPPFENm3axHXXXccTTzxBXV3daFd/UGV1hl+biPPimx2jXQ0RGaFiz8SPtbPPPpsVK1bw2GOPcdVVV/HFL36RT33qUzz77LM8+eST3HXXXSxdupQlS5aMdlUHVF5n+IkYe7t1hi8iI3PWWWfx0EMPkc1maW9vZ8WKFZxxxhm8+uqrNDY28ulPf5prrrmG1atXs337dnK5HB/96Ee57bbbWL169WhXf1BldYZfl4zT0ZPG3fUHHCIybB/+8If5/e9/zymnnIKZ8e1vf5spU6bwox/9iNtvv514PE5NTQ0//vGPaWtr4+qrryaXywHwt3/7t6Nc+8GVV+An4uQculJZairLatdE5Bjo7OwE8n/Nevvtt3P77bcftPzKK6/kyiuvPGS74/msvlBZNenUJvIhr1szRUQOVVaBX5fMDxeqWzNFRA5VXoGfCAJfHbciIoco5hGHM8zsKTNbZ2Zrzez6Ada53MyeM7M1ZvY7MzulYNnmoLzFzI7qU03qkvkmnQ6d4YuIHKKYns0McIO7rzazWmCVmS1393UF67wCnOPuu8zsQuBu4J0Fy89z9+2lq/bA9p/hK/BFRA4xZOC7+1ZgazDdYWbrgWnAuoJ1flewyR+A6SWuZ1EOdNqqSUdEpL8jasM3s9nAacAzh1ntz4FfFMw78EszW2Vmi4+0gkeidn8bvs7wRUT6KzrwzawGeBj4vLsPOKqRmZ1HPvC/VFB8pru/HbgQ+IyZnT3ItovNrNnMmtvbhzemfUUsQjIeVZOOiBx1hxs7f/PmzSxYsOAY1qY4RQW+mcXJh/197v7IIOv8MXAPcIm77+grd/e24H0b8ChwxkDbu/vd7t7k7k0NDQ1HthcFNICaiMjAhmzDt/wYBT8A1rv7dwZZZybwCHCFu28oKK8GIkHbfzVwPnBrSWo+iFoNkSwy9v3iJnhjTWk/c8pCuPBbgy6+6aabmDFjBp/5zGcAuOWWW4jFYjz11FPs2rWLdDrNbbfdxiWXXHJEX9vT08O1115Lc3MzsViM73znO5x33nmsXbuWq6++mlQqRS6X4+GHH2bq1Kl8/OMfp7W1lWw2y1e/+lUuvfTSEe12oWLu0nkPcAWwxsxagrIvAzMB3P0u4GvAROCfgzFsMu7eBDQCjwZlMeB+d3+iZLUfgAZQE5HhuPTSS/n85z+/P/CXLl3Kk08+yec+9znq6urYvn0773rXu7j44ouPaKyuO+64AzNjzZo1vPDCC5x//vls2LCBu+66i+uvv57LL7+cVCpFNpvl8ccfZ+rUqTz22GMA7Nmzp6T7WMxdOk8Dh907d78GuGaA8k3AKYducfTUJePs7Eody68UkVI7zJn40XLaaaexbds2Xn/9ddrb26mvr2fKlCl84QtfYMWKFUQiEdra2njzzTeZMmVK0Z/79NNPc9111wEwb948Zs2axYYNG3j3u9/NN7/5TVpbW/nIRz7C3LlzWbhwITfccANf+tKX+MAHPsBZZ51V0n0sq7+0hfy9+GrDF5Hh+LM/+zN++tOf8tBDD3HppZdy33330d7ezqpVq2hpaaGxsZGenp6SfNcnPvEJli1bRjKZ5KKLLuJXv/oVb3nLW1i9ejULFy7kK1/5CrfeWtoW8LIbUrI2EdNtmSIyLJdeeimf/vSn2b59O7/5zW9YunQpkydPJh6P89RTT/Hqq68e8WeeddZZ3Hfffbz3ve9lw4YNvPbaa7z1rW9l06ZNnHjiiXzuc5/jtdde47nnnmPevHlMmDCBT37yk4wfP5577rmnpPtXdoFfl8x32mpMfBE5UieffDIdHR1MmzaNE044gcsvv5wPfvCDLFy4kKamJubNm3fEn/mXf/mXXHvttSxcuJBYLMa9995LZWUlS5cu5Sc/+QnxeJwpU6bw5S9/mZUrV3LjjTcSiUSIx+PceeedJd0/c/eSfmApNDU1eXPz8IbdufPXL/N3T7zA+lsvIFkRLXHNRORoWb9+PfPnzx/taowJA/1bmdmq4GaZQZVfG74GUBMRGVD5NekUDKA2uS4xyrURkXK2Zs0arrjiioPKKisreeaZw40+M3rKLvD7BlDbo3vxRcacsdb3tnDhQlpaWoZesYRG0gxfhk06GiJZZCxKJBLs2LFjRIFW7tydHTt2kEgMr/Wi7M7w6zRipsiYNH36dFpbWxnu4IlhkUgkmD59eCPQl1/g7++0VZOOyFgSj8eZM2fOaFejrJVfk46eeiUiMqCyC/zKWISKaEQDqImI9FN2gW9m1CVjOsMXEemn7AIfNICaiMhAyjLwNYCaiMihyjLw+wZQExGRA4YMfDObYWZPmdk6M1trZtcPsI6Z2T+Z2UYze87M3l6w7Eozeyl4XVnqHRhIXSKuM3wRkX6KuQ8/A9zg7qvNrBZYZWbL3X1dwToXAnOD1zuBO4F3mtkE4GagCfBg22Xuvquke9FPvtNWbfgiIoWGPMN3963uvjqY7gDWA9P6rXYJ8GPP+wMw3sxOAN4PLHf3nUHILwcuKOkeDCDfaaszfBGRQkfUhm9ms4HTgP5DwU0DthTMtwZlg5UfVbWJGD3pHL2Z7NH+KhGRMaPowDezGuBh4PPuvrfUFTGzxWbWbGbNIx1Lo28ANd2aKSJyQFGBb2Zx8mF/n7s/MsAqbcCMgvnpQdlg5Ydw97vdvcndmxoaGoqp1qA0gJqIyKGKuUvHgB8A6939O4Ostgz4VHC3zruAPe6+FXgSON/M6s2sHjg/KDuqNICaiMihirlL5z3AFcAaM+sb6f/LwEwAd78LeBy4CNgI7AOuDpbtNLNvACuD7W51952lq/7AajWAmojIIYYMfHd/GjjsI2g8/8SCzwyybAmwZFi1G6YDTTo6wxcR6VOmf2mbP47pDF9E5IDyDPxE3106CnwRkT5lGfhVFVGiEVOTjohIgbIMfDPLj5ipM3wRkf3KMvBBA6iJiPRXvoGvAdRERA5SvoGvAdRERA5StoGff+qVzvBFRPqUbeDXJfTUKxGRQuUb+El12oqIFCrfwE/E6UplyWRzo10VEZHjQtkGfm0iP7xCZ6/a8UVEoIwDv+8hKOq4FRHJK9/AT2gANRGRQuUb+EmNiS8iUqh8A19j4ouIHGTIB6CY2RLgA8A2d18wwPIbgcsLPm8+0BA87Woz0AFkgYy7N5Wq4kOpVZOOiMhBijnDvxe4YLCF7n67u5/q7qcCfwX8pt9jDM8Llh+zsIfCTlsFvogIFBH47r4CKPY5tIuAB0ZUoxKprYxhhgZQExEJlKwN38yqyF8JPFxQ7MAvzWyVmS0u1XcVIxIxaipjGkBNRCQwZBv+Efgg8B/9mnPOdPc2M5sMLDezF4IrhkMEB4TFADNnzixJhfJj4usMX0QESnuXzmX0a85x97bgfRvwKHDGYBu7+93u3uTuTQ0NDSWpkJ56JSJyQEkC38zGAecA/15QVm1mtX3TwPnA86X4vmJpADURkQOKuS3zAeBcYJKZtQI3A3EAd78rWO3DwC/dvatg00bgUTPr+5773f2J0lV9aHWJOK/v7j6WXykictwaMvDdfVER69xL/vbNwrJNwCnDrVgp1CVivKAmHRERoIz/0hbUpCMiUqi8Az8Ro6M3Qy7no10VEZFRV96Bn4zjDp0p3ZopIlLegR8MoNahv7YVESnvwN8/gJra8UVEyjvwNYCaiMgB5R34fWPiq0lHRKTMAz+Zb9LRAGoiImUe+LUJNemIiPQp88Dve+qVmnRERMo68OPRCFUVUZ3hi4hQ5oEP+Y5b3YcvIhKGwE9qTHwREQhB4Ncm4gp8ERFCEPh1iZgecygiQhgCP6kzfBERCEPgq9NWRAQoIvDNbImZbTOzAZ9Ha2bnmtkeM2sJXl8rWHaBmb1oZhvN7KZSVrxYtYkYe7vTuGtMfBEJt2LO8O8FLhhind+6+6nB61YAM4sCdwAXAm8DFpnZ20ZS2eGoS8bJ5JzudPZYf7WIyHFlyMB39xXAzmF89hnARnff5O4p4EHgkmF8zojsH0BNHbciEnKlasN/t5k9a2a/MLOTg7JpwJaCdVqDsgGZ2WIzazaz5vb29hJVSwOoiYj0KUXgrwZmufspwPeAnw3nQ9z9bndvcvemhoaGElQrb/8Aagp8EQm5EQe+u+91985g+nEgbmaTgDZgRsGq04OyY6pu/1Ov1KQjIuE24sA3sylmZsH0GcFn7gBWAnPNbI6ZVQCXActG+n1Hav9Tr3SGLyIhFxtqBTN7ADgXmGRmrcDNQBzA3e8CPgZca2YZoBu4zPP3QGbM7LPAk0AUWOLua4/KXhxGncbEFxEBigh8d180xPLvA98fZNnjwOPDq1ppaEx8EZG8sv9L20Q8SkUsoiYdEQm9sg98yDfrqNNWRMIuHIGvMfFFREIS+BpATUQkHIHfN4CaiEiYhSLwNSa+iEhYAl+dtiIiIQn8ZEyDp4lI6IUj8BNxejM5ejQmvoiEWEgCv2+IZDXriEh4hSPwNYCaiEhIAl8DqImIhCTwk2rSEREJReDrqVciIiEJfD3IXEQkLIGf7BsTX2f4IhJeQwa+mS0xs21m9vwgyy83s+fMbI2Z/c7MTilYtjkobzGz5lJW/Egk41FiEdMfX4lIqBVzhn8vcMFhlr8CnOPuC4FvAHf3W36eu5/q7k3Dq+LImVkwgJqadEQkvIp5xOEKM5t9mOW/K5j9AzB95NUqPQ2gJiJhV+o2/D8HflEw78AvzWyVmS0+3IZmttjMms2sub29vcTV6htATYEvIuE15Bl+sczsPPKBf2ZB8Znu3mZmk4HlZvaCu68YaHt3v5ugOaipqclLVa8++QHU1KQjIuFVkjN8M/tj4B7gEnff0Vfu7m3B+zbgUeCMUnzfcNQl1KQjIuE24sA3s5nAI8AV7r6hoLzazGr7poHzgQHv9DkW1GkrImE3ZJOOmT0AnAtMMrNW4GYgDuDudwFfAyYC/2xmAJngjpxG4NGgLAbc7+5PHIV9KIrO8EUk7Iq5S2fREMuvAa4ZoHwTcMqhW4yOumScfaks6WyOeDQUf28mInKQ0CRf35j4neq4FZGQCk3gawA1EQm70AT+/oegqONWREIqPIGf0ABqIhJu4Qn84AxfA6iJSFiFJvBr+87w1aQjIiEVmsDXg8xFJOxCE/g1FTHM9CBzEQmv0AR+JGLUVsbYq/vwRSSkQhP4oDHxRSTcQhX4tYm4Om1FJLRCFfh1iZjO8EUktMIV+Ek99UpEwitcgZ+I66lXIhJaoQr8WjXpiEiIhSrw65JxOnsz5HIlf2SuiMhxr6jAN7MlZrbNzAZ8RKHl/ZOZbTSz58zs7QXLrjSzl4LXlaWq+HDUJWK4Q0evmnVEJHyKPcO/F7jgMMsvBOYGr8XAnQBmNoH8IxHfSf4B5jebWf1wKztSGkBNRMKsqMB39xXAzsOscgnwY8/7AzDezE4A3g8sd/ed7r4LWM7hDxxHVV1CY+KLSHiVqg1/GrClYL41KBus/BBmttjMms2sub29vUTVOpjGxBeRMDtuOm3d/W53b3L3poaGhqPyHQeeeqXAF5HwKVXgtwEzCuanB2WDlY+K/U06uhdfREKoVIG/DPhUcLfOu4A97r4VeBI438zqg87a84OyUVGXzDfpqNNWRMIoVsxKZvYAcC4wycxayd95Ewdw97uAx4GLgI3APuDqYNlOM/sGsDL4qFvd/XCdv0dVTaWeeiUi4VVU4Lv7oiGWO/CZQZYtAZYcedVKLxaNUF0RVaetiITScdNpe6xoADURCavwBb4GUBORkApd4GsANREJq9AFvh5zKCJhFb7AT8R0l46IhFL4Al9n+CISUuEL/KDTNn8nqYhIeIQu8GsTMbI5Z18qO9pVERE5pkIX+PsHUFOzjoiETPgCX2Pii0hIhS/wNYCaiIRU6AK/NqEmHREJp9AF/v6nXqlJR0RCJnyBr05bEQmp8gn8dDc8ei1s+OVhV6tN9LXh6wxfRMKlfALfHd5YA49cAzs3DbpaZSxKZSyiIZJFJHSKCnwzu8DMXjSzjWZ20wDLv2tmLcFrg5ntLliWLVi2rJSVP0hFFVz6E8DgwU9CqmvQVTW8goiE0ZCBb2ZR4A7gQuBtwCIze1vhOu7+BXc/1d1PBb4HPFKwuLtvmbtfXMK6H2rCHPjYD2DbOlh2Xf6sfwAaQE1EwqiYM/wzgI3uvsndU8CDwCWHWX8R8EApKjcsJ/0p/MlX4fmH4fd3DLiKzvBFJIyKCfxpwJaC+dag7BBmNguYA/yqoDhhZs1m9gcz+9BgX2Jmi4P1mtvb24uo1mGc+UWY/0FY/jV4ZcUhi+sScfaq01ZEQqbUnbaXAT9198KRyWa5exPwCeAfzeyPBtrQ3e929yZ3b2poaBhZLczgQ3fCxJPg366C3VsOWlybiNGhTlsRCZliAr8NmFEwPz0oG8hl9GvOcfe24H0T8GvgtCOu5XBU1sJl90E2DUuvgHTP/kVq0hGRMCom8FcCc81sjplVkA/1Q+62MbN5QD3w+4KyejOrDKYnAe8B1pWi4kWZNBc+/C/w+n/BYzfs78StS8TVaSsioTNk4Lt7Bvgs8CSwHljq7mvN7FYzK7zr5jLgQT/4ySLzgWYzexZ4CviWux+7wAeYdxGc/T+h5V+heQkAjXWVpLI5blm2ls5eBb+IhIMdj09+ampq8ubm5tJ9YC4HD1wKLz8FVz9Od+M7+JvH1/Ovz7zKlLoEX7/4ZM4/eUrpvk9E5Bgzs1VBf+mgyucvbQ8nEoGP3A3jpsNDV5DsbecbH1rAw9f+N8Yl4yz+ySoW/7iZrXu6R7umIiJHTTgCHyBZn+/E7d0LS6+ETIq3z6zn59edyU0XzmPFS+386T/8hh/+xytkc8ffVY+IyEiFJ/ABGk+Gi78HW/4Aj30RuncTj0b47+f8Ecu/cA5Nsyfw9Z+v48P//B8837ZntGsrIlJS4Qp8gIUfg/dcD//1E7j9JHhgEaz5KTOqc9x79el8b9FpvL67h4u//zS3/Z91dKlTV0TKRDg6bftzh7bVsPYReP4R6HgdYkl46wWw4KPsmX4uf/d/N3P/M68xbXySPz9zDn86v5GZE6uOXp1EREagmE7bcAZ+oVwu38Tz/MOw9mewbztU1ML8D/DS5PO5cdUEWl7Pj7x50uQa/mT+ZP5kXiNvnzmeWDR8F0gicnxS4B+pbAY2r8iH/7qfQ+8eSNbTNe1Mno/O47FdM/m3tvF0Z6OMr4pz7lsaeO/8Rs6Z28C4qvixr6+ISECBPxKZXnj5V7D2UXj1d7AnPx6Px5LsHL+AFt7Kz3dO59f75tARqeP02fWc+9bJnD67ngXTxlEZi45u/UUkVBT4pbSnDVr/E7b8J2x5BrY+C7l8h+6OxCyac3N5qms2a3Insjkyk3nTJ/KOWfW8fWY975hVT0Nt5SjvgIiUMwX+0ZTaB1tb8uH/2jP59+6dAGQszubobFamZvFsdjZrcnPoHv8WTpk9mXfMyh8A5k6uUR+AiJSMAv9Ycoddr8DrLfnB2ra24FufxXry9/OnifMSM2nJzGKNz+FVm0Fs8klMmzaTt00dx9um1jFvSh3VlbFR3hERGYsU+KOt8CCwtQV/Pf+K9B74o65OkrySa2SzT2GzT6GrehaxySdRP30+J86ayfwTxtFYV4mZjeKOiMjxToF/PHKHXZthx0bY8TK+YyO92zaS2/Eyic42Ihx4dsxer+IVn8Lr1khHchqZulnEJ82hbupcGmecxJzJ4xiX1N1BIlJc4Kv94Fgzyz9sfcIcmPs+DEj0LcukYPdrsPNlut/YwL62F2nYvpHpna2M61lJrCcD24B1kPEIbT6J9ZEpdCSnkq6dSWzCTKomzaB+yiwmT53DpPpxujIQkf0U+MeTWAVMOgkmnUTyLe8nWbgsl4W9r5PavoldbS/R9cZGcjtfYWrHa4zr+T3jup/IHwwK7PYadkYn0lnZSLqqEaubSuWE6dQ2zKS+cTo1k6Zj1Q0Q0S2kImGgwB8rIlEYP4OK8TNoPOmcQ5f3dtKzYwvbt77C3m2v0r2jFd/zOrGuN6jq3cbUfS8yYfteIq8c3ISXJcLeyHg6KyaRSjTgNY1Ex51Aon4qdQ3TqRrfiFVNhKoJkBgPUf0nIzJWFfV/r5ldAPwvIArc4+7f6rf8KuB2Djzr9vvufk+w7ErgK0H5be7+oxLUW/qrrCExdT7Tp84fcLG7s7tjH1vbNrPrjVfp3tlGes9W6HyT+L5tVKW2M25fK5N3Pc/ELXuJ2MB9O93RGnrj48lUjseTE4hUT6SidiKJugbitZOgauLBr+SE/JWLiIy6IQPfzKLAHcD7gFZgpZktG+BRhQ+5+2f7bTsBuBloAhxYFWy7qyS1l6KZGfV11dTXnQzzTx5wnUw2x/bOFC27Oti1rY297a307m0n07mdXNdOIt27iKd2k+jaQ31XB+NtC/Wsp8I6idvgD4/pidaQqqgnk6jHk/VEkuOJV42jonoc8eQ4LFGXf+j8Qa86qKiByhqIV+cfYiMiI1LMGf4ZwEZ33wRgZg8Cl1Dcw8jfDyx3953BtsuBC4AHhlddOZpi0QhTxiWYMi4BsxuAUwdcL5PNsbMrxbaOXjZ29tLe0cvOvZ307t1OuqMd79oB+3YQ791FRWoXyd7d1Kc6mNDZQb29Si0vUGPdxOnGLF1U3bKxanIV1VBRSyRRQ6SyFquszR8QKqqDg0NtMB3MVxQuqylYVqumKQmlYv6rnwZsKZhvBd45wHofNbOzgQ3AF9x9yyDbThvoS8xsMbAYYObMmUVUS0ZLLBphcl2CyXWJoVcGsjlnb3eanftS7OpK8ca+NLu6Uuzcl2JPRxf7OnbT27WbVNceMj17yXXvJZLqpMa6qaabGuuhOtNNdU9PUNZDjW2nNtJKrfVQTQ9J7yZBb9H7kIsm8uFfWYtVVmMVfQePvoND38GjKl8Wryo4YASvePWBdeJVEK3I34Ulcpwq1WnOz4EH3L3XzP4C+BHw3iP5AHe/G7gb8vfhl6hechyIRoz66grqqyugobhtMtkce7rTdPRk2NsTvAfzb/ak2Vswv7cnTVdvhu6eXjK9XXhvF6Q6Id1FtfdQFRwUqqyHGnqoppvqTA81vd1Ud/ZQYz2Mi3RQa9upsW6q6KHKu6n0HiLkit5Pt0j+uQoVVRBLYPEqiCfzB4N4Mv+KVeYPDNF48N7/FZTHKgq2G+BzCsui+lsMKU4xgd8GzCiYn86BzlkA3H1Hwew9wLcLtj2337a/PtJKSvjEohEm1lQysWb4g865O93pLJ29Gbp6s3T1ZujoydDVm6ErlZ9+szfDyz0ZOguW5afzB5FM7z48tQ9SXVR4cDVhvfkDCL1UWf49QS8JS5FMp0h295K0FNWRFNWWosr2UmXtJC1FBRniwStGhpiniXmGmKeG/48ViRVccVQduBqJVwXzhVch8QEOOINNF7xig5RF4vltInH1s4wBxQT+SmCumc0hH+CXAZ8oXMHMTnD3rcHsxcD6YPpJ4G/MrD6YPx/4qxHXWqQIZkZVRYyqihjUjvzzUpkc+1L5A8K+VP4A0vfenc7Sk87SncrSkc6xrWC+Ox28UvnXvnSWfcG23eksXakMvZksUXLEyVBBhkpSJCxFgjRJekmSImm9JEjl5y3/Xm291ETT1PSmqEn1Ur0vRbX1kKSLKnaSoCd/MPIeKnI9RMkS89+HXv4AAAapSURBVOL6TY6cBeEfCw4EsQPTkeihVyexxMBXMbGK/HYWzW8XiRbMxw4ui1bmr5piieC9cD5xYFp/awIUEfjunjGzz5IP7yiwxN3XmtmtQLO7LwM+Z2YXAxlgJ3BVsO1OM/sG+YMGwK19HbgiY01FLEJFrILxVaW/zTSby1+N9B0I9qWy9Gay9KRz9GSy9KZzwXyW3kyOnnSwLJ1lWybHlkx+eW8mt3/dVLZvOnfgs9JZetIZspk0uUwqOMBk81cdlr/yqCS9/yqkwjL7r0r2v1uaCjIkIlmSkSyVkRyJqFNpOSojOSoiOSojWeKWo9JyxMlR4RkqUmkqe3up8G4qfDcV3kM820ss10Ms10ss203Ej9IzpC0aHAAqCg4Slf2mK/LrHHQVVHDFE4kVXOHEBr4SOuyVU/zAFdFg00e5D0hj6YiEVC7npLIHHzx6ggNDqu8ActAB59CDSk/6QFnhwSi/zsEHp1Q2/7mpTH56ILHgwBIlR5Rs8J4jRpaI5d8Ly/oOUMlIhqpImmQkSzKSJmmZ4D1NwrIkLUUikiFh+VelpUmQP3BVkKaCNHHSxD1F1DPBK000lyHiGSKeJpJLE82lsSPo1zlitSfADS8Ma1ONpSMig4pEjEQkSiJ+7Js73PMHm95MwUEgc+Cg0Feezh5c3v+gkcnmSGedTC5HJuv7p7uyzp5sjkzOSWfzn1N4oOoNvqP/wak3M3SYRwqa3mJkiJOlwg70zRx8NVRYniVuGSotRyK4MqqMZKm0LBWWpdJyxEhy+VH8d1fgi8gxZ2ZUxqLH3aNA3Z1Mzsnm8u+Z4KCRKTioZHL56XTGSecOPjD1HVjSWT+4fP8BKThAZXOkck5XtqA859RUHt1/DwW+iEjAzIhHjVG46DkmdB+ViEhIKPBFREJCgS8iEhIKfBGRkFDgi4iEhAJfRCQkFPgiIiGhwBcRCYnjciwdM2sHXh3m5pOA7SWszmgrt/2B8tunctsfKL99Krf9gUP3aZa7H/aJE8dl4I+EmTUPNYDQWFJu+wPlt0/ltj9QfvtUbvsDw9snNemIiISEAl9EJCTKMfDvHu0KlFi57Q+U3z6V2/5A+e1Tue0PDGOfyq4NX0REBlaOZ/giIjKAsgl8M7vAzF40s41mdtNo16cUzGyzma0xsxYzG5PPfDSzJWa2zcyeLyibYGbLzeyl4L3+cJ9xPBlkf24xs7bgd2oxs4tGs45HwsxmmNlTZrbOzNaa2fVB+Vj+jQbbpzH5O5lZwsz+08yeDfbn60H5HDN7Jsi8h8xsyIctl0WTjplFgQ3A+4BW8g9NX+Tu60a1YiNkZpuBJncfs/cPm9nZQCfwY3dfEJR9G9jp7t8KDs717v6l0axnsQbZn1uATnf/+9Gs23CY2QnACe6+2sxqgVXAh4CrGLu/0WD79HHG4O9kZgZUu3unmcWBp4HrgS8Cj7j7g2Z2F/Csu995uM8qlzP8M4CN7r7J3VPAg8Alo1wnAdx9BbCzX/ElwI+C6R+R/59xTBhkf8Ysd9/q7quD6Q5gPTCNsf0bDbZPY5LndQaz8eDlwHuBnwblRf1G5RL404AtBfOtjOEfuIADvzSzVWa2eLQrU0KN7r41mH4DaBzNypTIZ83suaDJZ8w0fxQys9nAacAzlMlv1G+fYIz+TmYWNbMWYBuwHHgZ2O3umWCVojKvXAK/XJ3p7m8HLgQ+EzQnlBXPtymO9XbFO4E/Ak4FtgL/MLrVOXJmVgM8DHze3fcWLhurv9EA+zRmfyd3z7r7qcB08i0a84bzOeUS+G3AjIL56UHZmObubcH7NuBR8j90OXgzaGfta2/dNsr1GRF3fzP4HzIH/G/G2O8UtAs/DNzn7o8ExWP6Nxpon8b67wTg7ruBp4B3A+PNLBYsKirzyiXwVwJzg17rCuAyYNko12lEzKw66HDCzKqB84HnD7/VmLEMuDKYvhL491Gsy4j1BWPgw4yh3ynoEPwBsN7dv1OwaMz+RoPt01j9ncyswczGB9NJ8jenrCcf/B8LVivqNyqLu3QAglus/hGIAkvc/ZujXKURMbMTyZ/VA8SA+8fiPpnZA8C55Ef2exO4GfgZsBSYSX5U1I+7+5joCB1kf84l30zgwGbgLwrav49rZnYm8FtgDZALir9Mvs17rP5Gg+3TIsbg72Rmf0y+UzZK/iR9qbvfGmTEg8AE4L+AT7p772E/q1wCX0REDq9cmnRERGQICnwRkZBQ4IuIhIQCX0QkJBT4IiIhocAXEQkJBb6ISEgo8EVEQuL/AwsRzg7XQzTNAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "FFwvaeBFTzsT", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "28860719-d12c-4ac2-b88d-4c7bba35f4e3" }, "source": [ "predictions = model.predict(X_test_scaled)\n", "print('R2:', r2_score(y_test, predictions))\n", "print('MSE:', mean_squared_error(y_test, predictions))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "R2: 0.6656533177672821\n", "MSE: 0.438843430344939\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "mfCsG2swUGpG" }, "source": [ "## 5.2 Build, Train & Test a Wide & Deep NN with Subclassing API\n", "\n", "After the Sequential API and the Functional API, let's try the Subclassing API:\n", "* Create a subclass of the `tf.keras.models.Model` class.\n", "* Create all the layers you need in the constructor (e.g., `self.hidden1 = tf.keras.layers.Dense(...)`).\n", "* Use the layers to process the `input` in the `call()` method, and return the output.\n", "* Note that you do not need to create a `tf.keras.layers.Input` in this case.\n", "* Also note that `self.output` is used by Keras, so you should use another name for the output layer (e.g., `self.output_layer`).\n", "\n", "**When should you use the Subclassing API?**\n", "* Both the Sequential API and the Functional API are declarative: you first declare the list of layers you need and how they are connected, and only then can you feed your model with actual data. The models that these APIs build are just static graphs of layers. This has many advantages (easy inspection, debugging, saving, loading, sharing, etc.), and they cover the vast majority of use cases\n", "\n", "* If you need to build a very dynamic model (e.g., with loops or conditional branching), or if you want to experiment with new ideas using an imperative programming style, then the Subclassing API is for you. You can pretty much do any computation you want in the `call()` method, possibly with loops and conditions, using Keras layers of even low-level TensorFlow operations.\n", "\n", "* However, this extra flexibility comes at the cost of less transparency. Since the model is defined within the `call()` method, Keras cannot fully inspect it. All it sees is the list of model attributes (which include the layers you define in the constructor), so when you display the model summary you just see a list of unconnected layers. Consequently, you cannot save or load the model without writing extra code. So this API is best used only when you really need the extra flexibility." ] }, { "cell_type": "code", "metadata": { "id": "YePM2zqXT9xu" }, "source": [ "class MyRegressionModel(tf.keras.models.Model):\n", "\n", " def __init__(self):\n", " super(MyRegressionModel, self).__init__()\n", " self.hidden1 = tf.keras.layers.Dense(16, activation=\"relu\")\n", " self.hidden2 = tf.keras.layers.Dense(32, activation=\"relu\")\n", " self.hidden3 = tf.keras.layers.Dense(32, activation=\"relu\")\n", " self.output_layer = tf.keras.layers.Dense(1)\n", "\n", " def call(self, input):\n", " x = self.hidden1(input)\n", " x = self.hidden2(x)\n", " x = self.hidden3(x)\n", " concat = tf.keras.layers.concatenate([input, x])\n", " output = self.output_layer(concat)\n", " return output" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "GHiz9zKzVbki" }, "source": [ "model = MyRegressionModel()\n", "model.compile(loss=\"mean_squared_error\", \n", " optimizer=tf.keras.optimizers.SGD(1e-3))" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "PTpNFnbVVa7l", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "7ca9d786-57cd-4c72-f9bc-4721cf9b390a" }, "source": [ "history = model.fit(X_train_scaled, y_train, \n", " epochs=30,\n", " batch_size=32,\n", " validation_split=0.1)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Epoch 1/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 1.7095 - val_loss: 0.8031\n", "Epoch 2/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.7786 - val_loss: 0.6947\n", "Epoch 3/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6866 - val_loss: 0.6500\n", "Epoch 4/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6444 - val_loss: 0.6170\n", "Epoch 5/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6156 - val_loss: 0.5920\n", "Epoch 6/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5931 - val_loss: 0.5757\n", "Epoch 7/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5772 - val_loss: 0.5623\n", "Epoch 8/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5612 - val_loss: 0.5504\n", "Epoch 9/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5500 - val_loss: 0.5432\n", "Epoch 10/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5403 - val_loss: 0.5349\n", "Epoch 11/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5315 - val_loss: 0.5309\n", "Epoch 12/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5243 - val_loss: 0.5230\n", "Epoch 13/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5184 - val_loss: 0.5163\n", "Epoch 14/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5126 - val_loss: 0.5121\n", "Epoch 15/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5077 - val_loss: 0.5080\n", "Epoch 16/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5033 - val_loss: 0.5045\n", "Epoch 17/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4991 - val_loss: 0.5018\n", "Epoch 18/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4955 - val_loss: 0.4984\n", "Epoch 19/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4922 - val_loss: 0.4959\n", "Epoch 20/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4891 - val_loss: 0.4924\n", "Epoch 21/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4863 - val_loss: 0.4913\n", "Epoch 22/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4835 - val_loss: 0.4889\n", "Epoch 23/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4812 - val_loss: 0.4857\n", "Epoch 24/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4787 - val_loss: 0.4841\n", "Epoch 25/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4765 - val_loss: 0.4821\n", "Epoch 26/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4741 - val_loss: 0.4797\n", "Epoch 27/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4718 - val_loss: 0.4791\n", "Epoch 28/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4695 - val_loss: 0.4775\n", "Epoch 29/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4678 - val_loss: 0.4745\n", "Epoch 30/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4659 - val_loss: 0.4718\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "7RyFsP0bVvCp", "colab": { "base_uri": "https://localhost:8080/", "height": 283 }, "outputId": "e5bda015-0970-4f77-a645-67e1c5895f68" }, "source": [ "history_df = pd.DataFrame(history.history)\n", "history_df[['loss', 'val_loss']].plot(kind='line')" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 128 }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAD4CAYAAAATpHZ6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZhcVZ3/8fe3tq6q3rvT6awkQQNREgENUVSC6IiBYfnhQtiFGcBxARWHkVFnRAYet3GbR4VBBhEHhQygMsKAozIEFCELCSEgISYkdNZO0t3prbq28/vjVjdN6C1JpSv31uf1PPVU1b2nb52bgs89de6555pzDhERCYZQqSsgIiLFo1AXEQkQhbqISIAo1EVEAkShLiISIJFSffCECRPczJkzS/XxIiK+tGLFil3Ouabh1pcs1GfOnMny5ctL9fEiIr5kZptGWq/uFxGRAFGoi4gEiEJdRCRAStanLiLlKZPJ0NLSQiqVKnVVDmvxeJxp06YRjUb36+8U6iIyrlpaWqiurmbmzJmYWamrc1hyzrF7925aWlqYNWvWfv2tul9EZFylUikaGxsV6CMwMxobGw/o14xCXUTGnQJ9dAf6b+S7UH9xeyf/+siLtHWnS10VEZHDju9CfeOubr7/6Hq2dvSWuioi4lNVVVWlrsIh47tQr0t6Z4I7ejIlromIyOHHt6He3qtQF5GD45zj2muvZe7cucybN4977rkHgG3btrFw4UKOO+445s6dy+OPP04ul+PSSy8dKPud73ynxLUfmu+GNNYlYgC0q6Uu4ntf+e+1PL91b1G3+eYpNXz5zGPGVPb+++9n1apVrF69ml27dnHCCSewcOFCfvazn/GBD3yAL37xi+RyOXp6eli1ahVbtmzhueeeA6C9vb2o9S4WH7fUdaJURA7OE088wfnnn084HKa5uZmTTz6ZZcuWccIJJ/DjH/+Y66+/njVr1lBdXc2RRx7Jhg0buOqqq3j44YepqakpdfWH5LuWejwaJh4NqaUuEgBjbVGPt4ULF7J06VIefPBBLr30Uq655houueQSVq9ezSOPPMItt9zCkiVLuP3220td1dfxXUsdvC6Y9h611EXk4Jx00kncc8895HI5WltbWbp0KQsWLGDTpk00NzdzxRVXcPnll7Ny5Up27dpFPp/nQx/6EDfeeCMrV64sdfWH5LuWOnhdMGqpi8jBOuecc3jyySc59thjMTO+8Y1vMGnSJH7yk5/wzW9+k2g0SlVVFXfeeSdbtmzhsssuI5/PA/DVr361xLUfmjnnSvLB8+fPdwd6k4zF//4kDljysROLWykROeReeOEF3vSmN5W6Gr4w1L+Vma1wzs0f7m/82f2SjGqcuojIEPwZ6omYRr+IiAzBn6GejNLWk6FUXUciIocrn4Z6jHQ2TyqTL3VVREQOKz4NdV2AJCIylFFD3cxuN7OdZvbcCGXeY2arzGytmT1W3Cq+Xl2iEOo6WSoi8hpjaanfASwabqWZ1QE/BM5yzh0DfKQ4VRtebVKhLiIylFFD3Tm3FNgzQpELgPudc5sL5XcWqW7D6p/Uq0PdLyJyiI009/rLL7/M3Llzx7E2oytGn/pRQL2Z/Z+ZrTCzS4qwzRH196m3qaUuIvIaxZgmIAK8DXgfkACeNLM/OefW7VvQzK4ErgQ44ogjDvgD65OaflckEP7nOti+prjbnDQPTvvasKuvu+46pk+fzic/+UkArr/+eiKRCI8++ihtbW1kMhluvPFGzj777P362FQqxcc//nGWL19OJBLh29/+Nqeccgpr167lsssuI51Ok8/nue+++5gyZQrnnnsuLS0t5HI5/umf/onFixcf1G73K0aotwC7nXPdQLeZLQWOBV4X6s65W4FbwZsm4EA/MB4NEYuENPpFRPbb4sWL+cxnPjMQ6kuWLOGRRx7h6quvpqamhl27dvGOd7yDs846a79u/vyDH/wAM2PNmjX8+c9/5tRTT2XdunXccsstfPrTn+bCCy8knU6Ty+V46KGHmDJlCg8++CAAHR0dRdu/YoT6r4Dvm1kEiAFvBw7pLUHMjLqEpgoQ8b0RWtSHyvHHH8/OnTvZunUrra2t1NfXM2nSJD772c+ydOlSQqEQW7ZsYceOHUyaNGnM233iiSe46qqrAJgzZw4zZsxg3bp1nHjiidx00020tLTwwQ9+kNmzZzNv3jw+97nP8fnPf54zzjiDk046qWj7N5YhjT8HngSONrMWM/tbM/s7M/s7AOfcC8DDwLPA08Btzrlhhz8Wi2ZqFJED9ZGPfIR7772Xe+65h8WLF3PXXXfR2trKihUrWLVqFc3NzaRSqaJ81gUXXMADDzxAIpHg9NNP5/e//z1HHXUUK1euZN68eXzpS1/ihhtuKMpnwRha6s6588dQ5pvAN4tSozHS/C8icqAWL17MFVdcwa5du3jsscdYsmQJEydOJBqN8uijj7Jp06b93uZJJ53EXXfdxXvf+17WrVvH5s2bOfroo9mwYQNHHnkkV199NZs3b+bZZ59lzpw5NDQ0cNFFF1FXV8dtt91WtH3z5Xzq4I1Vf2VPT6mrISI+dMwxx9DZ2cnUqVOZPHkyF154IWeeeSbz5s1j/vz5zJkzZ7+3+YlPfIKPf/zjzJs3j0gkwh133EFFRQVLlizhpz/9KdFolEmTJvGFL3yBZcuWce211xIKhYhGo9x8881F2zdfzqcOcO1/rebxl3bxpy+8r4i1EpFDTfOpj13ZzKcOUF+p7hcRkX35t/slESWVyZPK5IhHw6WujogE2Jo1a7j44otfs6yiooKnnnqqRDUanm9Dvf+q0o7ejEJdxGecc/s1BrzU5s2bx6pVq8b1Mw+0a9y33S/9879oWKOIv8TjcXbv3q2b3IzAOcfu3buJx+P7/be+b6m396hfXcRPpk2bRktLC62traWuymEtHo8zbdq0/f4734Z6bUKTeon4UTQaZdasWaWuRmD5tvulvlLT74qI7Mu3oa67H4mIvJ5vQz0ZCxMNG+29CnURkX6+DXUzozYRU0tdRGQQ34Y6eCNg1KcuIvIqf4d6Ikpbt1rqIiL9/B3qyZj61EVEBvF5qEfp0MVHIiID/B3qiaha6iIig/g71JNRetI5+rK5UldFROSw4OtQr032X1Wq1rqICPg81HVVqYjIa/k71JMKdRGRwXwd6vXJ/jnVNQJGRAR8Hur90+9qBIyIiGfUUDez281sp5k9N0q5E8wsa2YfLl71RjZwSzt1v4iIAGNrqd8BLBqpgJmFga8DvylCncasqiJCOGS0a/4XERFgDKHunFsK7Bml2FXAfcDOYlRqrMzMm/9FLXUREaAIfepmNhU4B7h5DGWvNLPlZra8WPcnrE1G1f0iIlJQjBOl3wU+75zLj1bQOXerc26+c25+U1NTET7aGwGj7hcREU8xbjw9H7jbzAAmAKebWdY598sibHtUdYko2/emxuOjREQOewcd6s65gduCm9kdwK/HK9DB63758/bO8fo4EZHD2qihbmY/B94DTDCzFuDLQBTAOXfLIa3dGNQlYpr7RUSkYNRQd86dP9aNOecuPajaHIC6ZJSuvizpbJ5YxNfXUomIHDTfp+DABUhqrYuI+D/U+6cK0A2oRUQCEOqvTuqllrqIiO9DXdPvioi8yv+hnii01NWnLiLi/1CvHWipq09dRMT3oV5dESFk6n4REYEAhHooZNQmopr/RUSEAIQ6FCb1UktdRCQYoV6bjOriIxERAhLqdYmoWuoiIgQl1DWnuogIEJBQr01Eae9WS11EJBChXpeM0tmXJZMb9eZLIiKBFoxQL0zqtVcnS0WkzAUi1OsrNVWAiAgEJNT7p9/VCBgRKXeBCPW6wvS7mlNdRMpdMEK90FJv0wgYESlzwQj1/pka1acuImUuEKFeHY9iBh2afldEylwgQj08MFOjWuoiUt4CEeqg+V9ERGAMoW5mt5vZTjN7bpj1F5rZs2a2xsz+aGbHFr+ao6tNxtRSF5GyN5aW+h3AohHWbwROds7NA/4FuLUI9dpvXktdfeoiUt5GDXXn3FJgzwjr/+icayu8/RMwrUh12y91SXW/iIgUu0/9b4H/GW6lmV1pZsvNbHlra2tRP1gtdRGRIoa6mZ2CF+qfH66Mc+5W59x859z8pqamYn004F1VujeVJZd3Rd2uiIifFCXUzewtwG3A2c653cXY5v7qvwBJMzWKSDk76FA3syOA+4GLnXPrDr5KB0ZXlYqIQGS0Amb2c+A9wAQzawG+DEQBnHO3AP8MNAI/NDOArHNu/qGq8HDqEt6kXm09aWZROd4fLyJyWBg11J1z54+y/nLg8qLV6ADVFlrqHRoBIyJlLFBXlAK6AbWIlLXghHphTnWNVReRchaYUNfdj0REAhTq4ZBRE4/QodEvIlLGAhPq4HXBtOmqUhEpYwELdc3/IiLlLVChrhtliEi5C1So1yVjuqWdiJS1QIV6fVItdREpb4EK9bpElI7eDHnN1CgiZSpQoV6bjOEcdKaypa6KiEhJBCrU+6cK0LBGESlXwQp1Tb8rImUumKGulrqIlKlAhXptYU51TRUgIuUqUKFen9SkXiJS3gIV6pqpUUTKXaBCPRIOUV0R0egXESlbgQp18G5rpz51ESlXgQt1b6ZGtdRFpDwFL9QTMY1TF5GyFbxQT0bp0IlSESlTgQx1tdRFpFyNGupmdruZ7TSz54ZZb2b2b2a23syeNbO3Fr+aY1eXiNHek9ZMjSJSlsbSUr8DWDTC+tOA2YXHlcDNB1+tA1eXjJJ30NmnmRpFpPyMGurOuaXAnhGKnA3c6Tx/AurMbHKxKri/+i9AUr+6iJSjYvSpTwVeGfS+pbDsdczsSjNbbmbLW1tbi/DRr1eX9OZ/ae/VsEYRKT/jeqLUOXerc26+c25+U1PTIfkMzf8iIuWsGKG+BZg+6P20wrKS0JzqIlLOihHqDwCXFEbBvAPocM5tK8J2D0j/9Lu6qlREylFktAJm9nPgPcAEM2sBvgxEAZxztwAPAacD64Ee4LJDVdmx0EyNIlLORg1159z5o6x3wCeLVqODFIuEqIyFFeoiUpYCd0UpeCNgNPpFRMpRIEO9NqH5X0SkPAUy1OsrNf+LiJSnQIZ6XSKmux+JSFkKZKjXavpdESlTgQz1uoTX/eINzBERKR/BDPVklFze0aWZGkWkzAQz1AeuKlUXjIiUl2CGemH+lw6NgBGRMhPQUFdLXUTKU0BD3Wupa1ijiJSbYIZ6QtPvikh5CmSo1wzc0k4tdREpL4EM9Xg0TCKqmRpFpPwEMtTB61dX94uIlJsAh3pMLXURKTvBDfVEVLe0E5GyE9xQV/eLiJShYIe6ul9EpMwENtRrEzE6etOaqVFEykpgQ70uGSWTc/Skc6WuiojIuAlsqNcndVWpiJSfwIZ6bWH63bZujYARkfIxplA3s0Vm9qKZrTez64ZYf4SZPWpmz5jZs2Z2evGrun80/a6IlKNRQ93MwsAPgNOANwPnm9mb9yn2JWCJc+544Dzgh8Wu6IDu3fDYNyA/cl95f6hrBIyIlJOxtNQXAOudcxucc2ngbuDsfco4oKbwuhbYWrwq7mPDo/DoTbDyJyMWG7j7Ua+6X0SkfIwl1KcCrwx631JYNtj1wEVm1gI8BFw11IbM7EozW25my1tbWw+gusDcD8HMk+C3X4Gu4behlrqIlKNinSg9H7jDOTcNOB34qZm9btvOuVudc/Odc/ObmpoO7JPM4K+/Belu+O2Xhy0Wj4aJR0PqUxeRsjKWUN8CTB/0flph2WB/CywBcM49CcSBCcWo4JCajoZ3XgWr7oJNfxy2WF0iptEvIlJWxhLqy4DZZjbLzGJ4J0If2KfMZuB9AGb2JrxQP8D+lTFaeC3UHgEPfg5yQ7fGj5pUzSNrt7OlvfeQVkVE5HAxaqg757LAp4BHgBfwRrmsNbMbzOysQrHPAVeY2Wrg58Cl7lBfnx9Lwmlfh53Pw1O3DFnkX84+hlze8dm7V5HLa7oAEQk+K9XcKPPnz3fLly8/+A397DzYuBQ+tQxq9z1/C/etaOFz/7Waz73/KK563+yD/zwRkRIysxXOufnDrff/FaWnfQ1cDh75xyFXf/CtUznr2Cl893cvsXJz2zhXTkRkfPk/1OtnwsK/h+d/BS/99nWrzYwbz5nL5No4n777GTpTGg0jIsHl/1AHeOfV0PhGeOjvIZN63eqaeJTvnXccW9p6+edfrS1BBUVExkcwQj1SAaf/K7RthD98b8gib5vRwNXvm80vntnCL5/Zd0SmiEgwBCPUAd5wine16ePfgj0bhizyqVPeyPwZ9Xzpl8+xeXfPOFdQROTQC06oA5x6E4Rj8NC1MMSonkg4xHfPOw4z+Mw9z5DN5UtQSRGRQydYoV4zGd77RVj/W3jhv4csMq0+yU3nzGPl5nb+7ffrx7mCIiKHVrBCHeCEK6B5Hjx8HfR1DVnkrGOn8OG3TeP7v3+JpzfuGecKiogcOsEL9XAEzvg27N0Cj3192GLXn3UM0xuSfPaeVZr0S0QCI3ihDjB9Abz1EvjTD2HH80MWqaqI8L3zjmfH3hRf+MUaSnVlrYhIMQUz1AH+6itQUQP/+cEhL0oCOG56HdecehQPPruNe1e0jHMFRUSKL7ihnmyAS34J8Tq460PwwFWQ2vu6Yh9b+AZOPLKRLz+wlvtXtqjFLiK+FtxQB5h8LHzsMXjXZ+CZ/4Sb3wkbHntNkXDI+N55xzFnUjXXLFnNJbc/zSt7NIZdRPwp2KEO3tWm7/8K/M1vvNd3ngUP/r1356SCiTVx7v27d3LD2cewclMbp35nKbc9vkHj2EXEd4If6v2mnwAfexze8QlYdhvc/C7Y9OTA6lDIuOTEmfzvNSfzzjc0cuODL/DBm//I81tf32UjInK4Kp9QB+/GGou+Cpc+CC4PPz4NHvkiZF69M9KUugS3fXQ+37/geLa293Lm95/g6w//mVQmV8KKi4iMTXmFer+Z74KP/xHmXwZPfh/+fSG0vHrDDjPjjLdM4bfXnMyH3jqVm//vLyz67lL++JddJay0iMjoyjPUASqq4IzvwMW/8PrXb/sruO9yaF03UKQuGeMbHz6Wuy5/Ow644EdP8Q/3rtbNrEXksOX/29kVQ6rDm93x6dsg0wPzPgwL/wGajhoo0pvO8b3fvcSPHt9ALBziQ2+byqXvnMUbJ1aVsOIiUm5Gu52dQn2w7l3wx3+Dp38E2RTM/TAsvPY14b5uRyc/WrqBX63aSjqX5z1HN/E375rFSbMnYGYlrLyIlAOF+oEYKtxP/geY8OqNq3d19fGzpzZz55Ob2NXVx+yJVVz2rlmcc/xUErFwCSsvIkGmUD8YYwj3vmyOB5/dxn88sZG1W/dSl4xywYIjuPjEGUyuTZSw8iISREUJdTNbBHwPCAO3Oee+NkSZc4HrAQesds5dMNI2fRHq/bpavXBfdpsX7kefDsdfBG98vzcrJOCcY9nLbdz+xEZ+8/x2QmYsmjuJD751Ku9+YxOxSPmekxaR4jnoUDezMLAOeD/QAiwDznfOPT+ozGxgCfBe51ybmU10zu0cabu+CvV+Xa3eEMhn/hN6dkHlRDh2MRx3EUycM1DslT093PnkyyxZ3kJHb4baRJQPHNPMmcdO4cQjG4mEFfAicmCKEeonAtc75z5QeP+PAM65rw4q8w1gnXPutrFWzJeh3i+XgZd+A6t+BusehnwWprwVjr/Qu09qoh6AdDbPE+tb+fXqbfzm+R109WVprIyxaO4kznjLFBbMaiAc0slVERm7YoT6h4FFzrnLC+8vBt7unPvUoDK/xGvNvwuvi+Z659zDI23X16E+WFcrrFkCz9wFO9dCuALm/LUX8EeeAiHvpGkqk+P/Xmzl189u5Xcv7KQ3k2NidQWnz5vMmcdO5vjp9YQU8CIyivEK9V8DGeBcYBqwFJjnnGvfZ1tXAlcCHHHEEW/btGnTAe3UYck52LYaVt0Fa/4LetugahIceTLMeBfMfDc0HAlm9KSz/O6Fnfz62a08+mIr6WyehsoY82fUs2BWA2+f1cibJlerm0ZEXme8ul9uAZ5yzv248P53wHXOuWXDbTcwLfWhZPvgxf+Btb+ATX+A7lZvefXkVwN+5ruh8Y109mX57Qs7+MP63Ty9cQ+bC9P+VsbCvG1mA2+f1cAJMxt4y7Ra4lENlRQpd8UI9Qhe18r7gC14J0ovcM6tHVRmEd7J04+a2QTgGeA459zu4bYb6FAfzDnY9RK8/LgX8C//Abq2e+uqmmHGO72An/52aJrD9q4cT7+8h2Ub9/D0xj28uKMTgFgkxHHT61gws4EFsxp424x6KisiJdwxESmFYg1pPB34Ll5/+e3OuZvM7AZguXPuAfMupfwWsAjIATc55+4eaZtlE+r7cg72bPBC/uU/eEG/d4u3LhKHSfNgyvEDj7bETJa/spenN3ot+ee27iWXd4RDxtwpNSyY1cCCWY2cMLOeumSstPsmIoecLj463DkHbS/DlhWw9RnvsW01pLu89dFKmPyWgZDvmTCXFZ0NPPVyB09v3MOqV9pJF27mMWdSdSHkvcfE6njp9ktEDgmFuh/lc7B7/ashv/UZ2PYsZAvzvkfiMPFN0HwMmQlvZr3N4Im9zSzdkmfFpjZ60t7c75Nr48xurubo5ipmN1dzVHM1sydWqdtGxMcU6kGRy8KuF2HrKtixFnY85z16Bp22qJ5MfuKbaU3OZm1uGqu6G1nZnmT57iip7KvFptUnvIBvruKoiV7Yz5yQpDoeHf/9EpH9Mlqoq8nmF+EINB/jPfo5B107CwG/FnasJbRjLc0bl9Kcz/De/mLRMLm6ZjormtlljbySq2f99hrWrq9iWa6eba6RHdTTUJXgyAmVzJyQZNaEKmYVnmc0JjXyRsQn1FIPolzGG3HTvsk7Cbt3a+Ex6HWm57V/YhHaIxPZahP5S6aR9ekGXnFNtLgmttBEpGYyM5qqmNFYyRENSWY0JJnekOSIxiQ1auGLjBu11MtROArNb/YeQ3HOuzFIf8B3vEK4fTON7ZtpbN/EvPbnwO14zZ9k01F2bGtiR0sNbbkEnSRY6Sp5jASZaDWxZD2J6nqqahuoa5hA44RJNEyZRXNjg6YiFhlHCvVyZAaJOu8xXPBneqH9Fa+1376JSPtmprZtYmrPbnK97eR6WqBvL+FMF2GXhW68x/bXbmaPq2KjNdEWbaYnPol01TSsdhrRCTOonjiLCc1TaapJUBOP6CYjIkWgUJehRRPeHZ8G3fWpX7jwALxWf6YHUnuhby/ZnnbadrfStnsH6T2bof0Vot1bmdm7jfrO1SQ7e2Hbq9tKuSjbXQPrqKc90kh3rIm++ESylc1YzWQidVNINEylvq6BCdUxGisraKiMaSI0kWEo1OXgmEGs0nswmQjQNAOahirrHKTaSe/ZTMe2DfTs3Ehmzybo3M7Enh3M7NtETd8KKlIpaMe7frmg0yXY6epooZLnSdIXriIdrSVfUQMVtYSSdcSq6qmoaiBZ20BlTSNVtROobZhATWVSvwKkbCjUZfyYQaKe2NR6mqYeO3QZ56CvEzq34zq30rt7Cz27W8i0byXRuZ14qoNJfXuJpjcTy3aRTHcS6cyN+LG9LkaXVdITqiQVriIdqSYXq/EOCPE6QvEawolqYslaKiprSVTVkayuJ1lTTzheDRXVEE169Rc5zCnU5fBiBvEaiNdgTUeRPBKSI5Uf6P7poLdzD3vbdtHZvou+rj2ku9rJ9bST723H+vYSTncQyXRRkW4n0dtCpeummm5iNvJBASBHiD6LkwnFyYQT5MJx8pEkLprEogksVkm4opJwvJJovIposoZYsoZQRTVUVEGsCipqBr2uglj1wJ2zRIpF/0WJvw3q/knUTCExFZrH+KfOOVKZPNs7u+ja20b33jZ6utrp6+4g091BpreDfO9eXF8n1rcXMj2EMr2Esr1E+nqJ5VMkrI0k24mTJml9ROmjgtSYDhQAWYuSC8e9g0Q4josmIJKAaIJQLEkoliBcUUmkIkkoXu0dGPoPChXV3oFh4HXhOVIBoSiEIhDS9M3lRqEuZcvMSMTCJBprobEWmLlff5/J5elKZdmbytDRm2Fbr/d6b2+Gnt4e+rr3ku31Hi7VSb6vE/o6CWW6CKe7iWS7CWV7iWfSJOgjbmkSpImTJm7tJNhBHG9Zwvqoope4Zfarjo4QLhQmH4qChb3hrqEIhCJYOIZFK7BYEoskIBovHFAGP8e9k+bR5KADR1XhQFq9zy+PKm/7UlIKdZEDFA2HqK+MUV954LNjOufozeTo7svR3Zelqy9Ld1+WnnSOnYXX3rIcPZksvakUud4u8invFwTpLkJ9XYSyXYQz3USzXYTzGcLkiJIjbIVn8kTJEiZPhCwR8kQtO3DQSIbaSViGhGWIm3dgqSBNzKWJuj5CjO0ixXwohgvHcKEohGMQjmKRGBap8J7DsYHlhCu8XxX9j3CFdxCJxLzncOE5UlE4sLz6K8Z7HfcONtHCcyTubddChUd5ngNRqIuUkJmRjEVIxiI0VVcUZZvZXJ7eTI7edI7eTI6etPdIDbzOksrkaC8sH1yuN+0dUPr/vn+Zy/QSznQRynYTzfZQSYpK66WKFElLUUWKSnoLXVDZgUeFZV/zPh7qI27dVFiWCst5Bw6yxMgQJUPMpYm4DBGyo+/oWAwEfMj7pdL/OhQqHFz6DzAx76ASju6zLOYdQGKV3i+R/pFer3kUlkeThefCL5to0jsgjfPBRaEuEjCRcIjqcOiQTdDmnKMvm6cvkyeV9Q4WqUy+8Jwjlc3Tm84NvN9bWN+beXVZb9or15fJ0Zf1/rb/OZ3Nk85kyWdSkOvDZVJEXN+rXVOkSQz6NdH/Ok6aMDlCOMLkCZkjYo5oiIFHzBxRc8TMUUGOinyOCpclls0StRyxgQNQiojLEnVpIi5NNNdDJOs9jPzY/60shPUHfP/BIZqAY8+HBVccku9HoS4i+8XMiEfDxKNhahmfPvRMLv+6g0fvoIND/0GkLzu4TJ7u/vfZ3MCBo2/wdvYp31dYlskN193kqCBDkhSV1uc9F36tVJIaOOAkSXndWtZHZS5NVY2N/1oAAAR3SURBVDpNpaWptD4SlqavuoOTFhyafyuFuogc9qLhENFwiPG670su7wYOHulcnkzWkc7lSGed9z6X935RZPOkh3jdl80VnvN0Fp77Bi3/q6PHOkZr/ynURUT2EQ4ZlRURX95QRoNYRUQCRKEuIhIgCnURkQBRqIuIBIhCXUQkQBTqIiIBolAXEQkQhbqISICYc2Obfa3oH2zWCmw6wD+fAOwqYnUOB0Hbp6DtDwRvn4K2PxC8fRpqf2Y454a8YySUMNQPhpktd87NL3U9iilo+xS0/YHg7VPQ9geCt08Hsj/qfhERCRCFuohIgPg11G8tdQUOgaDtU9D2B4K3T0HbHwjePu33/viyT11ERIbm15a6iIgMQaEuIhIgvgt1M1tkZi+a2Xozu67U9SkGM3vZzNaY2SozW17q+uwvM7vdzHaa2XODljWY2f+a2UuF5/pS1nF/DbNP15vZlsL3tMrMTi9lHfeHmU03s0fN7HkzW2tmny4s9+X3NML++Pk7ipvZ02a2urBPXyksn2VmTxUy7x4zi424HT/1qZtZGFgHvB9oAZYB5zvnni9pxQ6Smb0MzHfO+fKiCTNbCHQBdzrn5haWfQPY45z7WuHgW++c+3wp67k/htmn64Eu59y/lrJuB8LMJgOTnXMrzawaWAH8P+BSfPg9jbA/5+Lf78iASudcl5lFgSeATwPXAPc75+42s1uA1c65m4fbjt9a6guA9c65Dc65NHA3cHaJ61T2nHNLgT37LD4b+Enh9U/w/ofzjWH2ybecc9uccysLrzuBF4Cp+PR7GmF/fMt5ugpvo4WHA94L3FtYPup35LdQnwq8Muh9Cz7/Igsc8BszW2FmV5a6MkXS7JzbVni9HTh0d9odX58ys2cL3TO+6KrYl5nNBI4HniIA39M++wM+/o7MLGxmq4CdwP8CfwHanXPZQpFRM89voR5U73bOvRU4Dfhk4ad/YDivj88//XzDuxl4A3AcsA34Vmmrs//MrAq4D/iMc27v4HV+/J6G2B9ff0fOuZxz7jhgGl7PxJz93YbfQn0LMH3Q+2mFZb7mnNtSeN4J/ALvy/S7HYV+z/7+z50lrs9Bc87tKPxPlwd+hM++p0I/7X3AXc65+wuLffs9DbU/fv+O+jnn2oFHgROBOjOLFFaNmnl+C/VlwOzC2eAYcB7wQInrdFDMrLJwogczqwROBZ4b+a984QHgo4XXHwV+VcK6FEV/+BWcg4++p8JJuP8AXnDOfXvQKl9+T8Ptj8+/oyYzqyu8TuANCHkBL9w/XCg26nfkq9EvAIUhSt8FwsDtzrmbSlylg2JmR+K1zgEiwM/8tk9m9nPgPXjThO4Avgz8ElgCHIE3xfK5zjnfnHgcZp/eg/ez3gEvAx8b1B99WDOzdwOPA2uAfGHxF/D6oX33PY2wP+fj3+/oLXgnQsN4De4lzrkbChlxN9AAPANc5JzrG3Y7fgt1EREZnt+6X0REZAQKdRGRAFGoi4gEiEJdRCRAFOoiIgGiUBcRCRCFuohIgPx/ZFMkwc8jabMAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "vKM7F26fV2qk", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "16c43222-1460-46c9-baf8-cb2d75aa9411" }, "source": [ "predictions = model.predict(X_test_scaled)\n", "print('R2:', r2_score(y_test, predictions))\n", "print('MSE:', mean_squared_error(y_test, predictions))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "R2: 0.6474512594797042\n", "MSE: 0.4627343618921509\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "obzFfuwmZZ9j", "colab": { "base_uri": "https://localhost:8080/", "height": 202 }, "outputId": "94b7eea7-bdb2-45bd-d6f3-53385417dc84" }, "source": [ "X_train.head()" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
MedIncHouseAgeAveRoomsAveBedrmsPopulationAveOccupLatitudeLongitude
70614.131235.05.8823530.9754901218.02.98529433.93-118.02
146892.863120.04.4012101.076613999.02.01411332.79-117.09
173234.202624.05.6175440.989474731.02.56491234.59-120.14
100563.109414.05.8695651.094203302.02.18840639.26-121.00
157503.306852.04.8012051.0662651526.02.29819337.77-122.45
\n", "
" ], "text/plain": [ " MedInc HouseAge AveRooms ... AveOccup Latitude Longitude\n", "7061 4.1312 35.0 5.882353 ... 2.985294 33.93 -118.02\n", "14689 2.8631 20.0 4.401210 ... 2.014113 32.79 -117.09\n", "17323 4.2026 24.0 5.617544 ... 2.564912 34.59 -120.14\n", "10056 3.1094 14.0 5.869565 ... 2.188406 39.26 -121.00\n", "15750 3.3068 52.0 4.801205 ... 2.298193 37.77 -122.45\n", "\n", "[5 rows x 8 columns]" ] }, "metadata": { "tags": [] }, "execution_count": 130 } ] }, { "cell_type": "markdown", "metadata": { "id": "zGI1mva4WjFa" }, "source": [ "## 7.3: Build, Train & Test Wide & Deep Model with feature subsets\n", "Now suppose you want to send only features 0 to 3 directly to the output, and only features 4 to 7 through the hidden layers, as shown on the following diagram. Use the functional API to build, train and evaluate this model.\n", "\n", "**Tips**:\n", "* You need to create two `tf.keras.layers.Input` (`input_A` and `input_B`)\n", "* Build the model using the functional API, as above, but when you build the `tf.keras.models.Model`, remember to set `inputs=[input_A, input_B]`\n", "* When calling `fit()`, `evaluate()` and `predict()`, instead of passing `X_train_scaled`, pass `(X_train_scaled_A, X_train_scaled_B)` (two NumPy arrays containing only the appropriate features copied from `X_train_scaled`).\n", "\n", "![](https://i.imgur.com/dFkDs1W.png)" ] }, { "cell_type": "code", "metadata": { "id": "5b25ZeICW0dy", "colab": { "base_uri": "https://localhost:8080/", "height": 202 }, "outputId": "36ce3a68-e4ef-45c4-8afb-82e7146de1cf" }, "source": [ "X_train.head()" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
MedIncHouseAgeAveRoomsAveBedrmsPopulationAveOccupLatitudeLongitude
70614.131235.05.8823530.9754901218.02.98529433.93-118.02
146892.863120.04.4012101.076613999.02.01411332.79-117.09
173234.202624.05.6175440.989474731.02.56491234.59-120.14
100563.109414.05.8695651.094203302.02.18840639.26-121.00
157503.306852.04.8012051.0662651526.02.29819337.77-122.45
\n", "
" ], "text/plain": [ " MedInc HouseAge AveRooms ... AveOccup Latitude Longitude\n", "7061 4.1312 35.0 5.882353 ... 2.985294 33.93 -118.02\n", "14689 2.8631 20.0 4.401210 ... 2.014113 32.79 -117.09\n", "17323 4.2026 24.0 5.617544 ... 2.564912 34.59 -120.14\n", "10056 3.1094 14.0 5.869565 ... 2.188406 39.26 -121.00\n", "15750 3.3068 52.0 4.801205 ... 2.298193 37.77 -122.45\n", "\n", "[5 rows x 8 columns]" ] }, "metadata": { "tags": [] }, "execution_count": 131 } ] }, { "cell_type": "code", "metadata": { "id": "4J2m47qfWO_e" }, "source": [ "input_A = tf.keras.layers.Input(shape=[4])\n", "\n", "input_B = tf.keras.layers.Input(shape=[4])\n", "x = tf.keras.layers.Dense(16, activation=\"relu\")(input_B)\n", "x = tf.keras.layers.Dense(32, activation=\"relu\")(x)\n", "x = tf.keras.layers.Dense(32, activation=\"relu\")(x)\n", "\n", "\n", "concat = tf.keras.layers.concatenate([input_A, x])\n", "\n", "output = tf.keras.layers.Dense(1)(concat)" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "qmM0HbDdXcYI", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "628ca293-0929-40a8-cea8-ce5c846c7094" }, "source": [ "model = tf.keras.models.Model(inputs=[input_A, input_B], outputs=[output])\n", "model.compile(loss=\"mean_squared_error\", \n", " optimizer=tf.keras.optimizers.SGD(1e-3))\n", "model.summary()" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Model: \"functional_3\"\n", "__________________________________________________________________________________________________\n", "Layer (type) Output Shape Param # Connected to \n", "==================================================================================================\n", "input_3 (InputLayer) [(None, 4)] 0 \n", "__________________________________________________________________________________________________\n", "dense_31 (Dense) (None, 16) 80 input_3[0][0] \n", "__________________________________________________________________________________________________\n", "dense_32 (Dense) (None, 32) 544 dense_31[0][0] \n", "__________________________________________________________________________________________________\n", "input_2 (InputLayer) [(None, 4)] 0 \n", "__________________________________________________________________________________________________\n", "dense_33 (Dense) (None, 32) 1056 dense_32[0][0] \n", "__________________________________________________________________________________________________\n", "concatenate_1 (Concatenate) (None, 36) 0 input_2[0][0] \n", " dense_33[0][0] \n", "__________________________________________________________________________________________________\n", "dense_34 (Dense) (None, 1) 37 concatenate_1[0][0] \n", "==================================================================================================\n", "Total params: 1,717\n", "Trainable params: 1,717\n", "Non-trainable params: 0\n", "__________________________________________________________________________________________________\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "GIKa4N5XXnLQ", "colab": { "base_uri": "https://localhost:8080/", "height": 644 }, "outputId": "96a73b0d-6bef-4f28-f8e3-3f349ffdf7f1" }, "source": [ "tf.keras.utils.plot_model(model, show_shapes=True)" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAqgAAAJzCAYAAADDQeGwAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdeVSUR7o/8G8rQtPIZmQLiMPihuKSMTNCXK+jV+UKGheIOjMkN04QM4AyGQRCVBQSkznIoDI5Zhhy5kZlkVzQKGauGqLGuMUFQ6IBFDcMICggtLJ0/f7wR48tgiy9vOD3c07/Yb31Vj1dpz3nod63qmRCCAEiIiIiIonoY+gAiIiIiIgexwSViIiIiCSFCSoRERERSQoTVCIiIiKSFKMnC7799lskJCQYIhYioh5j9erV8PLyMnQYRES9UqsZ1Bs3bmD37t2GiIVIa3bv3o2bN28aOowe5cSJEzhx4oShw+gRdu/ejRs3bhg6DCKiXqvVDGqLzMxMfcZBpFUymQyrVq3C4sWLDR1Kj7Fo0SIA/L/fETKZzNAhEBH1anwHlYiIiIgkhQkqEREREUkKE1QiIiIikhQmqEREREQkKUxQiYiIiEhSmKAStWP//v2wtLTE3r17DR2KJAUFBUEmk6k/y5Yta1Xn4MGDiIyMhEqlwvz58+Hs7Ay5XA5HR0f4+fkhPz+/23E8ePAAw4cPx7vvvqsu27NnDzZt2oTm5maNutnZ2RoxDxw4sNv9ExGRdjFBJWqHEMLQIUjegAEDkJubi8uXLyMlJUXj2tq1a5GUlISoqCioVCocPXoUO3fuRFVVFY4dOwalUonJkyejtLS0WzFER0fj8uXLGmW+vr6Qy+WYPn067t27py738/PDzZs3ceTIEcyZM6db/RIRkW4wQSVqh4+PD6qrqzF37lxDhwKlUglvb29Dh9GKqakpZs2ahaFDh8LExERd/sEHHyAtLQ0ZGRkwNzcHAHh5eWHixIlQKBRwcXFBXFwcqqur8emnn3a5/+PHj+P7779/6rXQ0FCMGTMGc+bMQVNTE4BHe5g6Ojpi0qRJGDJkSJf7JSIi3WGCStRDpKSkoLy83NBhdEhRURFiYmKwfv16yOVyAICRkVGrVyVcXV0BAMXFxV3qR6lU4p133kFiYmKbddatW4fz58+3W4eIiKSFCSpRG44dOwZnZ2fIZDJs3boVAJCcnAwzMzMoFArk5ORg9uzZsLCwgJOTE3bt2qW+NykpCXK5HLa2tggKCoKDgwPkcjm8vb1x8uRJdb2QkBAYGxvD3t5eXbZy5UqYmZlBJpPhzp07AICwsDCEh4ejuLgYMpkM7u7uAIADBw7AwsICcXFx+hiSDktKSoIQAr6+vu3WUyqVAAALC4su9RMdHY2VK1fCxsamzTrW1taYMmUKEhMT+coGEVEPwQSVqA0TJ07E8ePHNcqCg4OxatUqKJVKmJubIz09HcXFxXB1dcXy5cvR2NgI4FHiGRgYiPr6eoSGhqKkpARnz55FU1MTZsyYoT7HPSkpqdVxrNu2bcP69es1yhITEzF37ly4ublBCIGioiIAUC8AUqlUOhmDrtq3bx+GDRsGhULRbr1Tp04BeDTWnfXNN9+guLgYS5YseWbdcePG4datW7hw4UKn+yEiIv1jgkrURd7e3rCwsICNjQ0CAgJQV1eH69eva9QxMjLCiBEjYGJiAg8PDyQnJ6O2thapqalaicHHxwc1NTWIiYnRSnvaUFdXh6tXr8LNza3NOmVlZUhLS0NoaCi8vLyeOdP6JKVSibCwMCQnJ3eofsu7phcvXuxUP0REZBhGhg6AqDcwNjYGAPUMalvGjx8PhUKBS5cu6SMsgygvL4cQot3ZUy8vL9TV1WHx4sXYuHEj+vXr16k+oqKi8Ic//AGOjo4dqt8SS1lZWaf6ISIiw2CCSqRnJiYmqKioMHQYOvPgwQMA0FjR/yRbW1ukpKRg5MiRnW7/2LFjuHjxIhISEjp8j6mpqUZsREQkbXzET6RHjY2NuHfvHpycnAwdis60JINPbpD/OBsbG1hZWXWp/ZSUFBw6dAh9+vRRb7bfskgqLi4OMpkMZ86c0binoaFBIzYiIpI2JqhEepSXlwchBCZMmKAuMzIyeuarAT2Jra0tZDIZqqur26yzd+/eDj+ef1JqaiqEEBqflhnp6OhoCCEwfvx4jXtaYrGzs+tSn0REpF9MUIl0SKVS4e7du2hqakJ+fj7CwsLg7OyMwMBAdR13d3dUVVUhOzsbjY2NqKiowLVr11q1NWDAAJSWlqKkpAS1tbVobGxEbm6u5LaZUigUcHV1xc2bN596vaioCHZ2dvD39291LSAgAHZ2djh79qxWY2qJxdPTU6vtEhGRbjBBJWrD1q1b8fLLLwMAIiIi4Ofnh+TkZGzevBkAMHr0aFy5cgWffPIJwsPDAQCzZs1CYWGhuo0HDx7A09MTpqammDRpEoYOHYqvvvpK4/3M4OBgTJs2Da+99hqGDRuGDRs2qB9Fe3l5qbekWrFiBWxtbeHh4YE5c+agqqpKL+PQFT4+PigoKFDvc/q49vYibWhoQHl5OXJycrQaz+nTp+Ho6IjRo0drtV0iItINLpIiasPbb7+Nt99+u1V5cHCwxr9b9kB9GnNz8zZnElsMGDAAhw8fblX+4Ycfavx73LhxKCkp0SibPXs2ampq2m3fEP74xz8iOTkZWVlZWLZsmca1IUOGtLmafvfu3Zg6dSoGDx7cqf4GDhzYZuJbWVmJQ4cOYePGjZDJZJ1ql4iIDIMzqEQ61N5Cod5CqVTiyy+/RGFhoXoxkru7O2JjYxEbG4v79+93qJ3m5mZkZ2ejtrYWAQEBWotv3bp1GDt2LEJCQgA8msEtLS3FsWPH1AceEBGRtDBBJaJuqaqqwqxZszB06FC88cYb6vLIyEgsWrQIAQEB7S6YapGXl4esrCzk5uY+8wSqjkpISMD58+exf/9+9V6rOTk5cHR0xKRJk7Bv3z6t9ENERNqllQR1//79sLS0xN69e7XRnMFs2rQJw4cPh6mpKczMzDB8+HDExMR06RHqiRMnMGLECPVWOHZ2dti4caMOou66rKwsuLq6qrfqsbe3b/U4lromKioKqampqK6uhouLC3bv3m3okHTi448/1lhN/9lnn2lcj4uLQ0hICN5///1ntjV9+nTs2LED9vb2WoktJycHDx8+RF5eHqytrdXl8+bN04j5zp07WumPiIi0RyvvoLa36KEnOXr0KJYvX47f/e53MDU1RW5uLpYuXYqTJ0/iX//6V6famjBhAn788UfMmjULX375JS5fvtzlfR91ZcGCBViwYAHc3d1x584d/Pzzz4YOqdeIj49HfHy8ocOQhJkzZ2LmzJl679fPzw9+fn5675eIiLpPKzOoPj4+qK6uxty5c7XRXLcolUp4e3t36V5jY2OsXLkSNjY26N+/PxYtWoR58+bh//7v/3D79m0tR6p/3RkbIiIiIn3pdav4U1JSUF5e3qV7P//881ZlLZuJd3Shh5R1Z2yIiIiI9KXbM6jHjh2Ds7MzZDIZtm7dCgBITk6GmZkZFAoFcnJyMHv2bFhYWMDJyQm7du1S35uUlAS5XA5bW1sEBQXBwcEBcrkc3t7eOHnypLpeSEgIjI2NNd5NW7lyJczMzCCTydTvkIWFhSE8PBzFxcWQyWRwd3fv7tdDYWEhrKysNLa9OXDgQJc3R+/pY3P06FF4eHjA0tIScrkcnp6e+PLLLwEAb775pvp9Vjc3N5w7dw4A8Prrr0OhUMDS0hJ79uwB8GjF9nvvvQdnZ2eYmppi9OjRSE9PB/BoeyWFQgFzc3OUl5cjPDwcjo6OuHz5cpdiJiIioh5GPCE9PV08pbhdN27cEADEli1b1GXR0dECgDh06JCorq4W5eXlYtKkScLMzEw0NDSo67311lvCzMxM/PDDD+LBgweioKBAvPzyy8Lc3Fxcv35dXW/p0qXCzs5Oo9+PPvpIABAVFRXqsgULFgg3N7dOxf+khoYGcfPmTbFlyxZhYmIi/ud//kfj+hdffCHMzc1FbGzsM9v6z//8TwFA3L17V10mtbFxc3MTlpaWzx4YIURmZqZYt26dqKqqEpWVlWLChAnihRde0Oijb9++4tatWxr3LVmyROzZs0f97z/96U/CxMRE7N69W9y9e1dERUWJPn36iNOnT2uMUWhoqNiyZYt49dVXxY8//tihGIUQAoBIT0/vcH0SYuHChWLhwoWGDqNH4O+LiEi3dL7NlLe3NywsLGBjY4OAgADU1dXh+vXrGnWMjIwwYsQImJiYwMPDA8nJyaitrUVqaqquw3uqQYMGwcnJCevWrcOHH37Y6khGHx8f1NTUICYmplv99MSxWbhwIdauXQtra2sMGDAAvr6+qKysVJ+FvmLFCjQ3N2vEV1NTg9OnT2POnDkAHp2ulJycjPnz52PBggWwsrLCu+++i379+rX6Xh988AHefvttZGVlYfjw4fr7okRERGQwen0H1djYGADQ2NjYbr3x48dDoVDg0qVL+girlRs3buDevXs4d+4cIiMjsX37dhw+fBi2trY667OnjM2TWvaWbNmQ/j/+4z8wdOhQ/OMf/0BUVBRkMhnS0tIQEBCAvn37AgAuX76M+vp6jBo1St2Oqakp7O3ttfq9/P39n3reO7WPpy0REZGhSXaRlImJiXpWTt/69esHGxsbzJw5Ey4uLhg6dCji4+ORmJhokHieZMix2bdvHz766CMUFBSgpqamVUItk8kQFBSE1atX49ChQ/jNb36Df/7zn9ixY4e6Tl1dHQDg3Xffxbvvvqtxv4ODg9ZiDQsLg5eXl9ba6+02b94MAFi1apWBI5E+/uFDRKRbkkxQGxsbce/ePTg5ORk6FLi7u6Nv374oKCgwdCgA9D82R44cwXfffYdVq1bh+vXrmD9/Pl599VX84x//wIsvvogtW7bgz3/+s8Y9gYGBiIqKwt///ncMGjQIFhYWGovMbGxsADxKiMLCwnQWu5eXFxYvXqyz9nubzMxMAOCYdQATVCIi3ZLkUad5eXkQQmDChAnqMiMjo2c+/u6OyspKLFmypFV5YWEhmpubMWjQIJ313Rn6HpvvvvsOZmZmAICLFy+isbERwcHBcHV1hVwuf+rjYGtra/j7+yM7Oxt/+ctfsHz5co3rgwYNglwux/nz53USMxEREfVskkhQVSoV7t69i6amJuTn5yMsLAzOzs4IDAxU13F3d0dVVRWys7PR2NiIiooKXLt2rVVbAwYMQGlpKUpKSlBbW9vhxM3MzAz/+te/cPjwYfWj63PnzuH3v/89zMzMsHr1anXd3NzcLm8z1VmGGpvGxkaUlZUhLy9PnaA6OzsDAA4ePIgHDx6gsLBQY8urx61YsQIPHz7EF1980eoAB7lcjtdffx27du1CcnIyampq0NzcjJs3b/aKAxGIiIiom55c1t/Zbaa2bNki7O3tBQChUCiEr6+v2LZtm1AoFAKAGDJkiCguLhbbt28XFhYWAoAYPHiw+Omnn4QQj7ZS6tevn3B0dBRGRkbCwsJCzJs3TxQXF2v0U1lZKaZNmybkcrlwcXERf/zjH8U777wjAAh3d3f1tktnz54VgwcPFqampmLixIni559/7vB38fX1FS4uLqJ///7CxMREuLm5iYCAAHHx4kWNevv37xfm5uZi48aNbbZ14sQJMXLkSNGnTx8BQNjb24u4uDhJjc3f/vY34ebmJgC0+/n888/VfUVERIgBAwYIKysrsWjRIrF161YBQLi5uWlsfSWEEOPGjRORkZFPHZ+HDx+KiIgI4ezsLIyMjISNjY1YsGCBKCgoEJs2bRKmpqYCgBg0aFCrbb46AtwGqNO4zVTH8fdFRKRbMiGEeDxhzcjIgL+/P54o1pmgoCBkZmaisrJSL/31JD19bHx8fLB161a4uLjovW+ZTIb09HS+T9kJixYtAvDvd1Gpbfx9ERHpliQe8bdsUUSt9aSxefyVgfz8fMjlcoMkp0RERNSzSSJB1ZVLly6pj95s7xMQEGDoUHuFiIgIFBYW4qeffsLrr7+ODRs2GDok0rGgoCCN/0vLli1rVefgwYOIjIyESqXC/Pnz4ezsDLlcDkdHR/j5+SE/P7/bcTx48ADDhw/X2LZsz5492LRpU6s/8rKzszViHjhwYLf7JyIi7TJoghoVFYXU1FRUV1fDxcUFu3fv1mr7w4cPhxDimZ+0tDSt9qsNuh4bXVAoFBg+fDh+85vfYN26dfDw8DB0SKQHAwYMQG5uLi5fvoyUlBSNa2vXrkVSUhKioqKgUqlw9OhR7Ny5E1VVVTh27BiUSiUmT56M0tLSbsUQHR2Ny5cva5T5+vpCLpdj+vTpuHfvnrrcz88PN2/exJEjR9SnmxERkbQYNEGNj4/Hw4cPIYTA1atXsXDhQkOGIyk9cWw2btyI5uZmXL9+vdXK/eeNUqmEt7d3j++jI0xNTTFr1iwMHToUJiYm6vIPPvgAaWlpyMjIgLm5OYBHe9NOnDgRCoUCLi4uiIuLQ3V1NT799NMu93/8+HF8//33T70WGhqKMWPGYM6cOWhqagLw6P1RR0dHTJo0CUOGDOlyv0REpDu9+hE/kaGkpKSgvLy8x/fRVUVFRYiJicH69eshl8sBPNqvd+/evRr1XF1dAQDFxcVd6kepVOKdd95p95S3devW4fz585I5CY6IiJ6NCSoRACEEEhISMGLECJiYmMDa2hrz5s3DpUuX1HVCQkJgbGwMe3t7ddnKlSthZmYGmUyGO3fuAHh0xGp4eDiKi4shk8ng7u6OpKQkyOVy2NraIigoCA4ODpDL5fD29tbYS7Y7fQDAgQMH9LZHb3uSkpIghICvr2+79ZRKJQDAwsKiS/1ER0dj5cqV6tPJnsba2hpTpkxBYmKi3nYnISKi7mGCSoRHs2yRkZGIjo5GeXk5jhw5ghs3bmDSpEkoKysD8CjpenJboW3btmH9+vUaZYmJiZg7dy7c3NwghEBRURFCQkIQGBiI+vp6hIaGoqSkBGfPnkVTUxNmzJiBGzdudLsP4N+7PqhUKu0NThfs27cPw4YNg0KhaLfeqVOnAAATJ07sdB/ffPMNiouLn3oC3JPGjRuHW7du4cKFC53uh4iI9I8JKj33lEolEhIS8Oqrr2LZsmWwtLSEp6cnPv74Y9y5cwfbt2/XWl9GRkbqWVoPDw8kJyejtrYWqampWmnfx8cHNTU1iImJ0Up7XVFXV4erV6/Czc2tzTplZWVIS0tDaGgovLy8njnT+iSlUomwsDAkJyd3qH7Lu6YXL17sVD9ERGQYRoYOgMjQCgoKcP/+fYwfP16j/OWXX4axsXGbx7lqw/jx46FQKDReJejpysvLIYRod/bUy8sLdXV1WLx4MTZu3Ih+/fp1qo+oqCj84Q9/gKOjY4fqt8TSMhtORETSxgSVnnstWxD179+/1TUrKyvU1tbqtH8TExNUVFTotA99evDgAQBorOh/kq2tLVJSUjBy5MhOt3/s2DFcvHgRCQkJHb7H1NRUIzYiIpI2PuKn556VlRUAPDURvXfvHpycnHTWd2Njo8770LeWZLC9U9BsbGzU495ZKSkpOHToEPr06aPebL9lkVRcXBxkMhnOnDmjcU9DQ4NGbEREJG1MUOm5N2rUKPTv379VUnPy5Ek0NDTgl7/8pbrMyMhI40jX7srLy4MQAhMmTNBZH/pma2sLmUyG6urqNuvs3bu3w4/nn5SamtrqsI2WGejo6GgIIVq9rtESi52dXZf6JCIi/WKCSs89uVyO8PBwfP755/jss89QU1ODixcvYsWKFXBwcMBbb72lruvu7o6qqipkZ2ejsbERFRUVuHbtWqs2BwwYgNLSUpSUlKC2tladcKpUKty9exdNTU3Iz89HWFgYnJ2dERgYqJU+cnNzDb7NlEKhgKurK27evPnU60VFRbCzs4O/v3+rawEBAbCzs8PZs2e1GlNLLJ6enlptl4iIdIMJKhEeHckZHx+P2NhYDBw4EFOmTMEvfvEL5OXlwczMTF0vODgY06ZNw2uvvYZhw4Zhw4YN6sfGXl5e6u2iVqxYAVtbW3h4eGDOnDmoqqoC8OgdSE9PT5iammLSpEkYOnQovvrqK433NbvbhxT4+PigoKBAvc/p49rbi7ShoQHl5eXIycnRajynT5+Go6MjRo8erdV2iYhIN7hIigiPjr/805/+hD/96U/t1hswYAAOHz7cqvzDDz/U+Pe4ceNQUlLSqp65uXmbM4va6GP27Nmoqalpt319+OMf/4jk5GRkZWVh2bJlGteGDBnS5mr63bt3Y+rUqRg8eHCn+hs4cGCbiW9lZSUOHTqEjRs3QiaTdapdIiIyDM6gEulRewuHeiqlUokvv/wShYWF6sVI7u7uiI2NRWxsLO7fv9+hdpqbm5GdnY3a2loEBARoLb5169Zh7NixCAkJAfBoBre0tBTHjh1TH3BARETSwgSViLqlqqoKs2bNwtChQ/HGG2+oyyMjI7Fo0SIEBAS0u2CqRV5eHrKyspCbm/vME6g6KiEhAefPn8f+/fvVe63m5OTA0dERkyZNwr59+7TSDxERaRcTVCI9iIqKQmpqKqqrq+Hi4oLdu3cbOiSt+PjjjzVW03/22Wca1+Pi4hASEoL333//mW1Nnz4dO3bsgL29vVZiy8nJwcOHD5GXlwdra2t1+bx58zRivnPnjlb6IyIi7eE7qER6EB8fj/j4eEOHYRAzZ87EzJkz9d6vn58f/Pz89N4vERF1H2dQiYiIiEhSmKASERERkaQwQSUiIiIiSWGCSkRERESS0uYiqYyMDH3GQaR13377raFD6FFaDhDg/30iIjI0mXji+JWMjIynnpFNRET/lp6ejsWLFxs6DCKiXqlVgkokNS1JAGf2iIiIng98B5WIiIiIJIUJKhERERFJChNUIiIiIpIUJqhEREREJClMUImIiIhIUpigEhEREZGkMEElIiIiIklhgkpEREREksIElYiIiIgkhQkqEREREUkKE1QiIiIikhQmqEREREQkKUxQiYiIiEhSmKASERERkaQwQSUiIiIiSWGCSkRERESSwgSViIiIiCSFCSoRERERSQoTVCIiIiKSFCaoRERERCQpTFCJiIiISFKYoBIRERGRpDBBJSIiIiJJYYJKRERERJLCBJWIiIiIJIUJKhERERFJChNUIiIiIpIUJqhEREREJClMUImIiIhIUpigEhEREZGkMEElIiIiIklhgkpEREREksIElYiIiIgkRSaEEIYOgqjFjh07kJKSApVKpS67evUqAMDFxUVd1qdPH/z3f/83li5dqvcYiYiISLeYoJKk5OfnY8yYMR2qe+HCBYwePVrHEREREZG+MUElyRk+fDguX77cbh13d3cUFhbqKSIiIiLSJ76DSpLz29/+Fv369Wvzer9+/fD666/rMSIiIiLSJ86gkuRcuXIF7u7uaO+nWVhYCHd3dz1GRURERPrCGVSSHFdXV7z00kuQyWStrslkMowfP57JKRERUS/GBJUk6Xe/+x369u3bqrxv37743e9+Z4CIiIiISF/4iJ8kqby8HA4ODhrbTQGPtpcqLS2FnZ2dgSIjIiIiXeMMKkmSra0tpkyZojGL2rdvX0ydOpXJKRERUS/HBJUk67e//W2rhVK//e1vDRQNERER6Qsf8ZNk1dTUwMbGBg0NDQAebS9VXl4OKysrA0dGREREusQZVJIsCwsLzJo1C0ZGRjAyMsKcOXOYnBIRET0HmKCSpC1btgzNzc1obm7G0qVLDR0OERER6QEf8ZOkPXjwAAMHDoQQAnfu3IGpqamhQyIiIiId01qCmpGRAX9/f200RUSkN+np6Vi8eLGhwyAioscYabvB9PR0bTdJz7nz58/j0qVL+N///V/+vjrJ398fYWFh8PLyMnQoksQ/qomIpEnrM6h8Y4C0rampCZmZmViyZAl/X50kk8k4Q9gOjg8RkTRxkRRJnpGR0VOPPSUiIqLeiQkqEREREUkKE1QiIiIikhQmqEREREQkKUxQiYiIiEhSmKDSc2X//v2wtLTE3r17DR1Kj3Tw4EFERkZCpVJh/vz5cHZ2hlwuh6OjI/z8/JCfn9/tPh48eIDhw4fj3XffVZft2bMHmzZtQnNzc7fbJyIi6WOCSs8VblPVdWvXrkVSUhKioqKgUqlw9OhR7Ny5E1VVVTh27BiUSiUmT56M0tLSbvUTHR2Ny5cva5T5+vpCLpdj+vTpuHfvXrfaJyIi6WOCSs8VHx8fVFdXY+7cuYYOBUqlEt7e3oYOo0M++OADpKWlISMjA+bm5gAALy8vTJw4EQqFAi4uLoiLi0N1dTU+/fTTLvdz/PhxfP/990+9FhoaijFjxmDOnDloamrqch9ERCR9TFCJDCQlJQXl5eWGDuOZioqKEBMTg/Xr10MulwN4tDftk69JuLq6AgCKi4u71I9SqcQ777yDxMTENuusW7cO58+fb7cOERH1fExQ6blx7NgxODs7QyaTYevWrQCA5ORkmJmZQaFQICcnB7Nnz4aFhQWcnJywa9cu9b1JSUmQy+WwtbVFUFAQHBwcIJfL4e3tjZMnT6rrhYSEwNjYGPb29uqylStXwszMDDKZDHfu3AEAhIWFITw8HMXFxZDJZHB3dwcAHDhwABYWFoiLi9PHkHRIUlIShBDw9fVtt55SqQQAWFhYdKmf6OhorFy5EjY2Nm3Wsba2xpQpU5CYmMjXNYiIejEmqPTcmDhxIo4fP65RFhwcjFWrVkGpVMLc3Bzp6ekoLi6Gq6srli9fjsbGRgCPEs/AwEDU19cjNDQUJSUlOHv2LJqamjBjxgzcuHEDwKNk7sljM7dt24b169drlCUmJmLu3Llwc3ODEAJFRUUAoF4EpFKpdDIGXbFv3z4MGzYMCoWi3XqnTp0C8GicO+ubb75BcXExlixZ8sy648aNw61bt3DhwoVO90NERD0DE1Si/8/b2xsWFhawsbFBQEAA6urqcP36dY06RkZGGDFiBExMTODh4YHk5GTU1tYiNTVVKzH4+PigpqYGMTExWmmvu+rq6nD16lW4ubm1WaesrAxpaWkIDQ2Fl5fXM2dan6RUKhEWFj5/VXMAACAASURBVIbk5OQO1R8yZAgA4OLFi53qh4iIeg4jQwdAJEXGxsYAoJ5Bbcv48eOhUChw6dIlfYSld+Xl5RBCtDt76uXlhbq6OixevBgbN25Ev379OtVHVFQU/vCHP8DR0bFD9VtiKSsr61Q/RETUczBBJeomExMTVFRUGDoMnXjw4AGAR9+xLba2tkhJScHIkSM73f6xY8dw8eJFJCQkdPgeU1NTjdiIiKj34SN+om5obGzEvXv34OTkZOhQdKIlGWxvg3wbGxtYWVl1qf2UlBQcOnQIffr0gUwmg0wmUy+SiouLg0wmw5kzZzTuaWho0IiNiIh6HyaoRN2Ql5cHIQQmTJigLjMyMnrmqwE9ha2tLWQyGaqrq9uss3fv3g4/nn9SamoqhBAan5bZ6OjoaAghMH78eI17WmKxs7PrUp9ERCR9TFCJOkGlUuHu3btoampCfn4+wsLC4OzsjMDAQHUdd3d3VFVVITs7G42NjaioqMC1a9datTVgwACUlpaipKQEtbW1aGxsRG5urqS2mVIoFHB1dcXNmzefer2oqAh2dnbw9/dvdS0gIAB2dnY4e/asVmNqicXT01Or7RIRkXQwQaXnxtatW/Hyyy8DACIiIuDn54fk5GRs3rwZADB69GhcuXIFn3zyCcLDwwEAs2bNQmFhobqNBw8ewNPTE6amppg0aRKGDh2Kr776SuMdzeDgYEybNg2vvfYahg0bhg0bNqgfR3t5eam3pFqxYgVsbW3h4eGBOXPmoKqqSi/j0Fk+Pj4oKChQ73P6uPb2Im1oaEB5eTlycnK0Gs/p06fh6OiI0aNHa7VdIiKSDpnQ0m7XGRkZ8Pf35+bZpBNS+H0FBQUhMzMTlZWVBouhs2QyGdLT01vtzdoZRUVFGDFiBFJTU7Fs2bIO36dSqTB16lQEBgbijTfe6HL/j6usrISTkxM2btyo/iOiO7QxPkREpH2cQSXqhPYWC/VW7u7uiI2NRWxsLO7fv9+he5qbm5GdnY3a2loEBARoLZZ169Zh7NixCAkJ0VqbREQkPZJKUN98802Ym5tDJpPh/Pnzhg6nSzZt2oThw4fD1NQUZmZmGD58OGJiYlBTU/PU+iqVCps3b4a3t3eX+8zKyoKrq6t6FXTLx9jYGLa2tpg6dSo++ugj3L17t8t90PMtMjISixYtQkBAQLsLplrk5eUhKysLubm5zzyBqqMSEhJw/vx57N+/v9N7rRIRUc8iqQT173//Oz755BNDh9EtR48exfLly3H9+nWUlZVhw4YN2LRpExYuXNiqbmFhISZPnozVq1ejvr6+y30uWLAAV65cgZubGywtLSGEgEqlQnl5OTIyMuDi4oKIiAiMHDmy1ZY91DFRUVFITU1FdXU1XFxcsHv3bkOHpHdxcXEICQnB+++//8y606dPx44dO2Bvb6+VvnNycvDw4UPk5eXB2tpaK20SEZF0caN+LTM2NsbKlSshl8sBAIsWLUJmZiYyMzNx+/ZtODg4AAAuXLiA2NhYrFixAnV1dVp/t1Imk8HKygpTp07F1KlT4ePjA39/f/j4+OCnn36CpaWlVvvr7eLj4xEfH2/oMAxu5syZmDlzpt779fPzg5+fn977JSIiw5DUDCrwKLHqyT7//HN1ctqiZY/Ix9/fGzNmDLKysrB06dJ2T+nRloULFyIwMBDl5eX4+OOPdd4fERERUVcZNEEVQuCjjz7CsGHDYGJiAktLS7zzzjut6jU3N+O9996Ds7MzTE1NMXr0aKSnpwMAkpOTYWZmBoVCgZycHMyePRsWFhZwcnLCrl27NNr5+uuv8atf/QoKhQIWFhbw9PRUvxvaXh/dVVhYCCsrKwwePLjT9x44cEBr+2K27NWZm5urLuvpY0tERES9j0ET1JiYGEREROCtt95CWVkZfv75Z6xZs6ZVvTVr1uDDDz/E5s2bcfv2bcydOxdLlizBmTNnEBwcjFWrVkGpVMLc3Bzp6ekoLi6Gq6srli9frj7Rp66uDr6+vli4cCGqqqpQWFiIoUOHqo9NbK+PrmhsbMStW7ewdetWHDx4EFu2bIGxsXGn22lZNa5SqboUx+PGjh0LALhy5Yq6rCeOLREREfVyQkvS09NFZ5qrr68XCoVCzJgxQ6N8165dAoA4d+6cEEIIpVIpFAqFCAgI0LjXxMREBAcHCyGEiI6OFgCEUqlU19m2bZsAIIqKioQQQnz//fcCgPjiiy9axdKRPjrLzs5OABAvvPCC+Otf/yoaGhrarPvrX/9ajBkzpkv9PM7NzU1YWlq2W0cmkwkrKyshRM8a287+vugRACI9Pd3QYUgWx4eISJoMtkiqqKgI9fX1mD59erv1Ll++jPr6eowaNUpdZmpqCnt7e1y6dKnN+1pmK1tm+VxdXWFra4tly5YhNDQUgYGB+MUvftGtPtpz48YN3Lt3D+fOnUNkZCS2b9+Ow4cPw9bWtkvtaUPLYiwLCwsAPXNsMzIyOn3P8+7bb781dAhERESdo61Mt7MzXPv37xcAREpKikb5kzOo33zzjQDw1M+ECROEEE+f5fvkk08EAPHjjz+qy77//nvxX//1X8LIyEjIZDLh7+8v6uvrO9RHd/z0008CgAgNDX3qdX3NoJ49e1YAEDNnzhRC9Kyxbfl98cOPtj+cQSUikh6DvYPastL94cOH7dazsbEBAGzevBlCCI1PZ2eGRo4cib1796K0tBQRERFIT0/HX/7yF6328TTu7u7o27cvCgoKut1Wdxw4cAAAMHv2bAA9c2yfbIOf9j8AkJ6ebvA4pPohIiJpMliCOmrUKPTp0wdff/11u/UGDRoEuVze7ZOlSktL8cMPPwB4lJi9//77eOmll/DDDz9orY/KykosWbKkVXlhYSGam5sxaNCgbrXfHT///DM2b94MJycn9bnoPWlsiYiI6PlhsATVxsYGCxYswO7du5GSkoKamhrk5+dj+/btGvXkcjlef/117Nq1C8nJyaipqUFzczNu3ryJ27dvd7i/0tJSBAUF4dKlS2hoaMC5c+dw7do1TJgwQWt9mJmZ4V//+hcOHz6MmpoaNDY24ty5c/j9738PMzMzrF69usNttcjNze3UNlNCCNy/fx8qlQpCCFRUVCA9PR2vvPIK+vbti+zsbPU7qD1pbImIiOg5IrSkK6usa2trxZtvvileeOEF0b9/fzFx4kTx3nvvCQDCyclJXLhwQQghxMOHD0VERIRwdnYWRkZGwsbGRixYsEAUFBSIbdu2CYVCIQCIIUOGiOLiYrF9+3ZhYWEhAIjBgweLn376SZSUlAhvb29hbW0t+vbtK1588UURHR0tmpqantlHZ/j6+goXFxfRv39/YWJiItzc3ERAQIC4ePGiRr1vv/1WvPLKK8LBwUH9Lpy9vb3w9vYWX3/9tbre/v37hbm5udi4cWObfe7Zs0eMHj1aKBQKYWxsLPr06SMAqFfs/+pXvxKxsbGisrKy1b09ZWy5ir9rwHcs28XxISKSJpkQ2nkRKyMjA/7+/nyvi3SCv6+ukclkSE9Px+LFiw0diiRxfIiIpElyR50SERER0fONCeozXLp0CTKZ7JmfgIAAQ4dKRERE1CswQX2G4cOHd2i7mrS0NEOHSqRVBw8eRGRkJFQqFebPnw9nZ2fI5XI4OjrCz88P+fn5XW5bpVJh8+bN8Pb2brNOY2Mj4uPj4e7uDmNjY1hZWWHUqFEoKSkBAOzZswebNm1SHwdMRES9BxNUImpl7dq1SEpKQlRUFFQqFY4ePYqdO3eiqqoKx44dg1KpxOTJk1FaWtrptgsLCzF58mSsXr0a9fX1bdbz9/fHP//5T+zYsQP19fX48ccf4ebmhvv37wMAfH19IZfLMX36dNy7d6/L35WIiKSHCSpRBymVynZn/HpKH8/ywQcfIC0tDRkZGTA3NwcAeHl5YeLEiVAoFHBxcUFcXByqq6vx6aefdqrtCxcuYM2aNVixYgXGjh3bZr20tDRkZ2cjMzMTv/71r2FkZAQHBwfk5ORoHJsbGhqKMWPGYM6cOWhqaurS9yUiIulhgkrUQSkpKSgvL+/xfbSnqKgIMTExWL9+vfq0NyMjI+zdu1ejnqurKwCguLi4U+2PGTMGWVlZWLp0KUxMTNqs97e//Q0vvfQSPD09n9nmunXrcP78eSQmJnYqFiIiki4mqNRrCSGQkJCAESNGwMTEBNbW1pg3bx4uXbqkrhMSEgJjY2PY29ury1auXAkzMzPIZDLcuXMHABAWFobw8HAUFxdDJpPB3d0dSUlJkMvlsLW1RVBQEBwcHCCXy+Ht7Y2TJ09qpQ/g0RG1nTmsoTuSkpIghICvr2+79ZRKJQCoD33QpoaGBpw4caLdGdbHWVtbY8qUKUhMTOQ2ZEREvQQTVOq11q1bh8jISERHR6O8vBxHjhzBjRs3MGnSJJSVlQF4lJA9uQfmtm3bsH79eo2yxMREzJ07F25ubhBCoKioCCEhIQgMDER9fT1CQ0NRUlKCs2fPoqmpCTNmzMCNGze63QcA9SIglUqlvcFpw759+zBs2DAoFIp26506dQoAMHHiRK3HUFpaioaGBnz33XeYNm2aOvEfMWIEtm3b9tQkdNy4cbh16xYuXLig9XiIiEj/mKBSr6RUKpGQkIBXX30Vy5Ytg6WlJTw9PfHxxx/jzp07rY7U7Q4jIyP1LK2HhweSk5NRW1uL1NRUrbTv4+ODmpoaxMTEaKW9ttTV1eHq1atwc3Nrs05ZWRnS0tIQGhoKLy+vZ860dkXLIigbGxvExcWhoKAAZWVlmDdvHt5++23s3Lmz1T1DhgwBAFy8eFHr8RARkf4xQaVeqaCgAPfv38f48eM1yl9++WUYGxtrPILXtvHjx0OhUGi8StATlJeXQwjR7uypl5cXQkNDMW/ePOTm5qJfv35aj6Pl3dSRI0fC29sbAwYMgKWlJdavXw9LS8un/nHREnPLzDgREfVsRoYOgEgXWrYd6t+/f6trVlZWqK2t1Wn/JiYmqKio0Gkf2vbgwQMAaHfxkq2tLVJSUjBy5EidxeHg4AAA6ndzWxgbG2Pw4MFPXZhlamoK4N/fgYiIejbOoFKvZGVlBQBPTUTv3bsHJycnnfXd2Nio8z50oSXJa2/jexsbG/XY6kr//v0xZMgQ/PDDD62uNTU1wdLSslV5Q0MDgH9/ByIi6tmYoFKvNGrUKPTv3x9nzpzRKD958iQaGhrwy1/+Ul1mZGSExsZGrfWdl5cHIQQmTJigsz50wdbWFjKZDNXV1W3W2bt3LxwdHXUei7+/P86dO4crV66oy+rr63Ht2rWnbj3VErOdnZ3OYyMiIt1jgkq9klwuR3h4OD7//HN89tlnqKmpwcWLF7FixQo4ODjgrbfeUtd1d3dHVVUVsrOz0djYiIqKCly7dq1VmwMGDEBpaSlKSkpQW1urTjhVKhXu3r2LpqYm5OfnIywsDM7OzggMDNRKH7m5uXrZZkqhUMDV1RU3b9586vWioiLY2dnB39+/1bWAgADY2dnh7NmzWoll9erVGDx4MAIDA3H9+nVUVlYiIiICSqUSa9asaVW/JeaO7JtKRETSxwSVeq21a9ciPj4esbGxGDhwIKZMmYJf/OIXyMvLg5mZmbpecHAwpk2bhtdeew3Dhg3Dhg0b1I+Kvby81NtFrVixAra2tvDw8MCcOXNQVVUF4NF7j56enjA1NcWkSZMwdOhQfPXVVxrvcna3D33x8fFBQUGBep/Tx7W3x2hDQwPKy8uRk5PTbvsnTpzAxIkT8eKLL+LkyZO4cOECHBwc8Morr+DIkSPqetbW1jh69CicnJwwduxYODo64tSpU9i3b99T90c9ffo0HB0dMXr06E58WyIikiqZ0NLO1hkZGfD39+dG2aQTUv19BQUFITMzE5WVlYYO5alkMhnS09Nb7cPalqKiIowYMQKpqalYtmxZh/tRqVSYOnUqAgMD8cYbb3Q13C6prKyEk5MTNm7ciPDw8E7d29nxISIi/eAMKlE3tbeoqKdxd3dHbGwsYmNj1fuRPktzczOys7NRW1uLgIAAHUfY2rp16zB27FiEhITovW8iItINJqhEpCEyMhKLFi1CQEBAuwumWuTl5SErKwu5ubnPPIFK2xISEnD+/Hns379fJ3uyEhGRYTBBJeqiqKgopKamorq6Gi4uLti9e7ehQ9KauLg4hISE4P33339m3enTp2PHjh2wt7fXQ2T/lpOTg4cPHyIvLw/W1tZ67ZuIiHSLG/UTdVF8fDzi4+MNHYbOzJw5EzNnzjR0GG3y8/ODn5+focMgIiId4AwqEREREUkKE1QiIiIikhQmqEREREQkKUxQiYiIiEhStL5IatGiRdpukkh9lCV/X523efNmZGZmGjoMIiKiDtPaSVLffvstEhIStNEUkYZz584BAMaNG2fgSKg3Wr16Nby8vAwdBhERPUZrCSqRrrQcQ5mRkWHgSIiIiEgf+A4qEREREUkKE1QiIiIikhQmqEREREQkKUxQiYiIiEhSmKASERERkaQwQSUiIiIiSWGCSkRERESSwgSViIiIiCSFCSoRERERSQoTVCIiIiKSFCaoRERERCQpTFCJiIiISFKYoBIRERGRpDBBJSIiIiJJYYJKRERERJLCBJWIiIiIJIUJKhERERFJChNUIiIiIpIUJqhEREREJClMUImIiIhIUpigEhEREZGkMEElIiIiIklhgkpEREREksIElYiIiIgkhQkqEREREUkKE1QiIiIikhQmqEREREQkKUxQiYiIiEhSmKASERERkaQwQSUiIiIiSWGCSkRERESSwgSViIiIiCTFyNABED2uvr4eDx8+1ChraGgAANy9e1ej3MTEBAqFQm+xERERkX7IhBDC0EEQtUhOTsbKlSs7VHfbtm0IDg7WcURERESkb0xQSVIqKirg4OCA5ubmduv17dsXt2/fho2NjZ4iIyIiIn3hO6gkKTY2Npg+fTr69u3bZp2+ffviN7/5DZNTIiKiXooJKknOsmXL0N7EvhACy5Yt02NEREREpE98xE+SU1tbCxsbm1aLpVoYGxujoqICFhYWeo6MiIiI9IEzqCQ55ubmmDt3Lvr169fqmpGREfz8/JicEhER9WJMUEmSli5diqamplblzc3NWLp0qQEiIiIiIn3hI36SpIaGBgwcOBC1tbUa5f3798edO3dgYmJioMiIiIhI1ziDSpJkbGyMRYsWwdjYWF3Wr18/+Pv7MzklIiLq5ZigkmQtWbJEfYoUADQ2NmLJkiUGjIiIiIj0gY/4SbJUKhXs7e1RUVEBABg4cCB+/vnndvdIJSIiop6PM6gkWX369MGSJUtgbGyMfv36YenSpUxOiYiIngNMUEnSXnvtNTQ0NPDxPhER0XPESFsN3bx5E8ePH9dWc0QAHp0a9cILLwAArl69ipKSEsMGRL2Ot7c3nJycDB0GERE9RmvvoGZkZMDf318bTRER6U16ejoWL15s6DCIiOgxWptBbcE1V6RtP/zwAw4cOIDw8HD+vjpJJpMxAWuHTCYzdAhERPQUfAeVJM/Dw4OPYImIiJ4jTFCJiIiISFKYoBIRERGRpDBBJSIiIiJJYYJKRERERJLCBJWIiIiIJIUJKj1X9u/fD0tLS+zdu9fQoUjewYMHERkZCZVKhfnz58PZ2RlyuRyOjo7w8/NDfn5+l9tWqVTYvHkzvL2926zT2NiI+Ph4uLu7w9jYGFZWVhg1apT6sIY9e/Zg06ZNaG5u7nIcREQkTUxQ6bnCfVQ7Zu3atUhKSkJUVBRUKhWOHj2KnTt3oqqqCseOHYNSqcTkyZNRWlra6bYLCwsxefJkrF69GvX19W3W8/f3xz//+U/s2LED9fX1+PHHH+Hm5ob79+8DAHx9fSGXyzF9+nTcu3evy9+ViIikhwkqPVd8fHxQXV2NuXPnGjoUKJXKdmcQDeWDDz5AWloaMjIyYG5uDgDw8vLCxIkToVAo4OLigri4OFRXV+PTTz/tVNsXLlzAmjVrsGLFCowdO7bNemlpacjOzkZmZiZ+/etfw8jICA4ODsjJycGoUaPU9UJDQzFmzBjMmTMHTU1NXfq+REQkPUxQiQwkJSUF5eXlhg5DQ1FREWJiYrB+/XrI5XIAgJGRUatXIlxdXQEAxcXFnWp/zJgxyMrKwtKlS2FiYtJmvb/97W946aWX4Onp+cw2161bh/PnzyMxMbFTsRARkXQxQaXnxrFjx+Ds7AyZTIatW7cCAJKTk2FmZgaFQoGcnBzMnj0bFhYWcHJywq5du9T3JiUlQS6Xw9bWFkFBQXBwcIBcLoe3tzdOnjyprhcSEgJjY2PY29ury1auXAkzMzPIZDLcuXMHABAWFobw8HAUFxdDJpPB3d0dAHDgwAFYWFggLi5OH0PSSlJSEoQQ8PX1bbeeUqkEAFhYWGg9hoaGBpw4caLdGdbHWVtbY8qUKUhMTOQrHEREvQQTVHpuTJw4EcePH9coCw4OxqpVq6BUKmFubo709HQUFxfD1dUVy5cvR2NjI4BHiWdgYCDq6+sRGhqKkpISnD17Fk1NTZgxYwZu3LgB4FGC9+S599u2bcP69es1yhITEzF37ly4ublBCIGioiIAUC/4UalUOhmDZ9m3bx+GDRsGhULRbr1Tp04BeDSm2lZaWoqGhgZ89913mDZtmvqPgREjRmDbtm1PTULHjRuHW7du4cKFC1qPh4iI9I8JKtH/5+3tDQsLC9jY2CAgIAB1dXW4fv26Rh0jIyOMGDECJiYm8PDwQHJyMmpra5GamqqVGHx8fFBTU4OYmBittNcZdXV1uHr1Ktzc3NqsU1ZWhrS0NISGhsLLy+uZM61d0bIIysbGBnFxcSgoKEBZWRnmzZuHt99+Gzt37mx1z5AhQwAAFy9e1Ho8RESkf0xQiZ7C2NgYANQzqG0ZP348FAoFLl26pI+wdKq8vBxCiHZnT728vBAaGop58+YhNzcX/fr103ocLe+mjhw5Et7e3hgwYAAsLS2xfv16WFpaYvv27a3uaYm5rKxM6/EQEZH+GRk6AKKezsTEBBUVFYYOo9sePHgAAO0uXrK1tUVKSgpGjhypszgcHBwAQP2+bgtjY2MMHjz4qQuzTE1NAfz7OxARUc/GGVSibmhsbMS9e/fg5ORk6FC6rSXJa2/jexsbG1hZWek0jv79+2PIkCH44YcfWl1ramqCpaVlq/KGhgYA//4ORETUszFBJeqGvLw8CCEwYcIEdZmRkdEzXw2QIltbW8hkMlRXV7dZZ+/evXB0dNR5LP7+/jh37hyuXLmiLquvr8e1a9eeuvVUS8x2dnY6j42IiHSPCSpRJ6hUKty9exdNTU3Iz89HWFgYnJ2dERgYqK7j7u6OqqoqZGdno7GxERUVFbh27VqrtgYMGIDS0lKUlJSgtrYWjY2NyM3NNdg2UwqFAq6urrh58+ZTrxcVFcHOzg7+/v6trgUEBMDOzg5nz57VSiyrV6/G4MGDERgYiOvXr6OyshIRERFQKpVYs2ZNq/otMXdk31QiIpI+Jqj03Ni6dStefvllAEBERAT8/PyQnJyMzZs3AwBGjx6NK1eu4JNPPkF4eDgAYNasWSgsLFS38eDBA3h6esLU1BSTJk3C0KFD8dVXX2m8txkcHIxp06bhtddew7Bhw7Bhwwb1o2cvLy/1llQrVqyAra0tPDw8MGfOHFRVVellHNrj4+ODgoIC9T6nj2tvj9GGhgaUl5cjJyen3fZPnDiBiRMn4sUXX8TJkydx4cIFODg44JVXXsGRI0fU9aytrXH06FE4OTlh7NixcHR0xKlTp7Bv376n7o96+vRpODo6YvTo0Z34tkREJFUyoaWdrTMyMuDv78+NskknpPD7CgoKQmZmJiorKw0WQ2fJZDKkp6e32pu1LUVFRRgxYgRSU1OxbNmyDvejUqkwdepUBAYG4o033uhquF1SWVkJJycnbNy4Uf2HRUd1dnyIiEg/OINK1AntLSDqDdzd3REbG4vY2Fj1fqTP0tzcjOzsbNTW1iIgIEDHEba2bt06jB07FiEhIXrvm4iIdENSCeqbb74Jc3NzyGQynD9/3tDhdMmmTZswfPhwmJqawszMDMOHD0dMTAxqamo06sXGxsLDwwMWFhYwMTGBu7s7/vznP3c4KXhcVlYWXF1dIZPJND7GxsawtbXF1KlT8dFHH+Hu3bva+prUi0VGRmLRokUICAhod8FUi7y8PGRlZSE3N/eZJ1BpW0JCAs6fP4/9+/frZE9WIiIyDEklqH//+9/xySefGDqMbjl69CiWL1+O69evo6ysDBs2bMCmTZuwcOFCjXqHDx/G22+/jZKSEty5cwfx8fFITEzEokWLOt3nggULcOXKFbi5ucHS0hJCCKhUKpSXlyMjIwMuLi6IiIjAyJEjcebMGW191edKVFQUUlNTUV1dDRcXF+zevdvQIelUXFwcQkJC8P777z+z7vTp07Fjxw7Y29vrIbJ/y8nJwcOHD5GXlwdra2u99k1ERLrFjfq1zNjYGCtXroRcLgcALFq0CJmZmcjMzMTt27fVm5D3798fb731Fvr27QsAWLx4MbKyspCRkYEbN25g0KBB3YpDJpPBysoKU6dOxdSpU+Hj4wN/f3/4+Pjgp59+eupektS2+Ph4xMfHGzoMvZo5cyZmzpxp6DDa5OfnBz8/P0OHQUREOiCpGVTgUWLVk33++efq5LRFy76Rjz++/+KLL9TJaYuBAwcCeLTfo7YtXLgQgYGBKC8vx8cff6z19omIiIi0xaAJqhACH330EYYNGwYTExNYWlrinXfeaVWvubkZ7733HpydnWFqaorRo0cjPT0dAJCcnAwzMzMoFArk5ORg9uzZsLCwgJOTE3bt2qXRztdff41f/epXUCgUsLCwgKenp/rd0Pb66K7CwkJYWVlh8ODB7da7desWTE1N4eLioi47cOCA1vbFbNmrMzc3V13W08eWiIiIk2bW9AAAIABJREFUeh+DJqgxMTGIiIjAW2+9hbKyMvz8889P3YR7zZo1+PDDD7F582bcvn0bc+fOxZIlS3DmzBkEBwdj1apVUCqVMDc3R3p6OoqLi+Hq6orly5erT/Spq6uDr68vFi5ciKqqKhQWFmLo0KHqIxLb66MrGhsbcevWLWzduhUHDx7Eli1bYGxs3Gb9+vp6HD58GMuXL9eo17JqXKVSdSmOx7XsH/n46Tw9cWyJiIiolxNakp6eLjrTXH19vVAoFGLGjBka5bt27RIAxLlz54QQQiiVSqFQKERAQIDGvSYmJiI4OFgIIUR0dLQAIJRKpbrOtm3bBABRVFQkhBDi+++/FwDEF1980SqWjvTRWXZ2dgKAeOGFF8Rf//pX0dDQ0G796OhoMXToUFFTU9Ol/oQQws3NTVhaWrZbRyaTCSsrKyFEzxrbzv6+6BEAIj093dBhSBbHh4hImgy2SKqoqAj19fWYPn16u/Uu/z/27j0uyjL/H/9r5DQcFRQ8gBgHERE0SwtItNbyU7oqpii1for2o2LSVzS3UMlEDBRzkTywastSu2WI5OIRNSVKM80ShXBVoPAABiIKKCgDXL8//DHbCMNxZu4BXs/HY/7onmvu633fzbx9c9/XfV2XLqGqqgqenp7KbaampujXrx8uXryo9nMNVyEbrvI5OzvDzs4Os2fPRmhoKIKCgvDYY491qI/mXLt2DXfu3EFmZiaWLVuG7du3Iz09HXZ2do3a7t69G8nJyThy5AgsLS3b1V9r3Lt3D0IIWFlZAeic57Y9sxx0dxs2bMCuXbukDoOIiKjVJLvF37B2tq2tbbPt7t27BwB47733VOb4vHLlSpseJjI1NUV6ejrGjBmDqKgoODs7IzAwENXV1Rrr4/eMjIxga2uLCRMmICkpCTk5OU0+BZ6UlIS1a9ciIyNDWdRpy+XLlwEA7u7uADrvuSUiIqKuTbIrqA1Puj948KDZdg0F7IYNG7Bo0aIO9Tls2DDs27cPN2/eRGxsLNauXYthw4YpV7/RRB9NcXV1hYGBAXJyclS2b9q0CYcPH0Z6ejosLCw03u+jDh06BAB46aWXAHTOc8srgW0jk8mwePFiLuWpRmefNYSIqKuS7Aqqp6cnevTogW+++abZdgMHDoRcLu/wylJFRUW4cOECgIeF2Zo1a/DEE0/gwoULGuvj1q1bePXVVxttz83NRV1dnXJuUyEEwsLCkJ2djdTUVJ0Up7/99hs2bNgABwcH5VrpnencEhERUfchWYFqa2uL6dOnIyUlBQkJCaioqEBWVha2b9+u0k4ul+ONN97AF198gfj4eFRUVKCurg7Xr1/HjRs3Wt1fUVER5s+fj4sXL6KmpgaZmZm4cuUKvL29NdaHubk5jhw5gvT0dFRUVEChUCAzMxOvv/46zM3N8fbbbwMALly4gHXr1uHjjz+GkZFRoyVK169fr9xnWlpam6aZEkLg7t27qK+vhxACN2/exM6dO/HMM8/AwMAAqampyjGonencEhERUTeiqaet2vOUdWVlpZgzZ47o3bu3sLCwEGPGjBHvv/++ACAcHBzE+fPnhRBCPHjwQISFhQlHR0dhaGgobG1txfTp00VOTo7YsmWLMDMzEwDE4MGDRX5+vti+fbuwsrISAMSgQYPE5cuXRUFBgfD19RXW1tbCwMBADBgwQISHh4va2toW+2iLKVOmCCcnJ2FhYSFMTEyEi4uLCAwMFNnZ2co22dnZAoDa14cffqhse/DgQWFpaSk++OADtX3u3btXDB8+XJiZmQljY2PRo0cPAUD5xP5TTz0lIiMjxa1btxp9trOcWz7F3z7gU+rN4vkhItJPMiGE0EShm5ycjFmzZkFDuyNSwe9X+8hkMuzcuZNjUNXg+SEi0k96t9QpEREREXVvLFBbcPHixUZjRJt6NTytTtRVHD16FMuWLUN9fT2mTZsGR0dHyOVy2NvbY+rUqcjKymrzPmNiYuDu7g5TU1OYm5vD3d0dK1asUC6L2yAyMhIeHh6wsrKCiYkJXF1d8e677+Lu3bvKNnv37kVMTIxytTUiIuo6WKC2wN3dHUKIFl9JSUlSh0qkMStXrsTGjRuxfPly1NfX4/jx49ixYwfKyspw4sQJVFdXY+zYsSgqKmrTfo8fP465c+fi6tWrKC4uxurVqxETE4MZM2aotEtPT8dbb72FgoIClJaWIjo6GnFxcSoLNUyZMgVyuRzjx4/HnTt3NHLcRESkH1igErVSdXU1fH19O30fLVm7di2SkpKQnJysXNnMx8cHY8aMgZmZGZycnBAVFYXy8nJ88sknbdq3sbExQkJCYGtrCwsLCwQEBMDf3x9fffWVyqwOFhYWCA4Oho2NDSwtLTFz5kxMmzYNhw4dwrVr15TtQkNDMWLECEycOBG1tbUaOX4iIpIeC1SiVkpISEBJSUmn76M5eXl5WLFiBVatWqVcTMPQ0BD79u1Taefs7AwAyM/Pb9P+d+/erdxvA3t7ewBQuX2/f/9+GBgYqLTr06cPADRagSwiIgLnzp1DXFxcm2IhIiL9xQKVuiwhBGJjYzF06FCYmJjA2toa/v7+uHjxorLNwoULYWxsjH79+im3hYSEwNzcHDKZDKWlpQCARYsWYcmSJcjPz4dMJoOrqys2btwIuVwOOzs7zJ8/H/3794dcLoevry9Onz6tkT6AhyuAtWUu3I7YuHEjhBCYMmVKs+2qq6sBQDmnbkfk5uaiV69eGDRoULPtCgsLYWpqCicnJ5Xt1tbWGDduHOLi4jjLAxFRF8EClbqsiIgILFu2DOHh4SgpKcG3336La9euwc/PD8XFxQAeFmSPTjG0ZcsWrFq1SmVbXFwcJk+eDBcXFwghkJeXh4ULFyIoKAhVVVUIDQ1FQUEBzp49i9raWrzwwgvKW9Ed6QOA8iGg+vp6zZ0cNQ4cOIAhQ4bAzMys2XY//PADAGDMmDHt6kehUKCwsBCbN2/G0aNHsWnTJhgbG6ttX1VVhfT0dMydO7fJdiNHjkRhYSHOnz/frniIiEi/sEClLqm6uhqxsbF4+eWXMXv2bPTs2RNeXl7YunUrSktLG61Y1hGGhobKq7QeHh6Ij49HZWUlEhMTNbL/SZMmoaKiAitWrNDI/tS5d+8efv31V7i4uKhtU1xcjKSkJISGhsLHx6fFK63qDBw4EA4ODoiIiMC6deswa9asZttHR0ejf//++OCDD5p8f/DgwQCA7OzsdsVDRET6hQUqdUk5OTm4e/cuRo0apbJ99OjRMDY2VrkFr2mjRo2CmZmZylCCzqCkpARCiGavnvr4+CA0NBT+/v5IS0uDkZFRu/q6du0aSkpKsGPHDnz66acYOXKk2rG3u3fvRnJyMg4fPqx8aOtRDTE3XBknIqLOzVDqAIi0oWHaIQsLi0bv9erVC5WVlVrt38TEBDdv3tRqH5p2//59AA9jV8fOzg4JCQkYNmxYh/oyMjKCra0tJkyYACcnJ7i5uSmnkvq9pKQkxMbGIiMjAwMGDFC7P1NTU5VjICKizo0FKnVJvXr1AoAmC9E7d+7AwcFBa30rFAqt96ENDUVecxPf29raKs+tpri6usLAwAA5OTkq2zdt2oTDhw8jPT29yT80fq+mpgbAf4+BiIg6N97ipy7J09MTFhYW+PHHH1W2nz59GjU1NXjyySeV2wwNDaFQKDTWd0ZGBoQQ8Pb21lof2mBnZweZTIby8nK1bfbt26ecFqqtbt26hVdffbXR9tzcXNTV1WHgwIEAHs6+EBYWhuzsbKSmprZYnAJQxty3b992xUZERPqFBSp1SXK5HEuWLMHu3bvx2WefoaKiAtnZ2XjzzTfRv39/BAcHK9u6urqirKwMqampUCgUuHnzJq5cudJonzY2NigqKkJBQQEqKyuVBWd9fT1u376N2tpaZGVlYdGiRXB0dERQUJBG+khLS9PJNFNmZmZwdnbG9evXm3w/Ly8Pffv2bfKBpsDAQPTt2xdnz55Vu39zc3McOXIE6enpqKiogEKhQGZmJl5//XWYm5vj7bffBgBcuHAB69atw8cffwwjI6NGywqvX7++0b4bYvby8mrPoRMRkZ5hgUpd1sqVKxEdHY3IyEj06dMH48aNw2OPPYaMjAyYm5sr2y1YsADPPfccXnnlFQwZMgSrV69W3ir28fFRThf15ptvws7ODh4eHpg4cSLKysoAPBz36OXlBVNTU/j5+cHNzQ1ff/21yljOjvahK5MmTUJOTo5yntPfa26O0ZqaGpSUlGDPnj1q28jlcjzzzDOYM2cO7O3tYWlpiYCAADz22GM4deoUPD09W+xHnTNnzsDe3h7Dhw9v82eJiEj/yISGZrZOTk7GrFmzOFE2aYW+fr/mz5+PXbt24datW1KH0iSZTIadO3c2modVnby8PAwdOhSJiYmYPXt2q/upr6/Hs88+i6CgIPz5z39ub7jtcuvWLTg4OOCDDz7AkiVL2vTZtp4fIiLSDV5BJeqg5h4q6mxcXV0RGRmJyMhIlaVHm1NXV4fU1FRUVlYiMDBQyxE2FhERgccffxwLFy7Ued9ERKQdLFCJSMWyZcsQEBCAwMDAZh+YapCRkYEvv/wSaWlpLa5ApWmxsbE4d+4cDh482O45WYmISP+wQCVqp+XLlyMxMRHl5eVwcnJCSkqK1CFpTFRUFBYuXIg1a9a02Hb8+PH4/PPP0a9fPx1E9l979uzBgwcPkJGRAWtra532TURE2sV5UInaKTo6GtHR0VKHoTUTJkzAhAkTpA5DralTp2Lq1KlSh0FERFrAK6hEREREpFdYoBIRERGRXmGBSkRERER6hQUqEREREekVFqhEREREpFc0/hS/TCbT9C6JlPj9artZs2Zh1qxZUodBRETUahpb6vT69es4efKkJnZFpGLDhg0AgMWLF0scCXVFvr6+cHBwkDoMIiL6HY0VqETa0rBOenJyssSREBERkS5wDCoRERER6RUWqERERESkV1igEhEREZFeYYFKRERERHqFBSoRERER6RUWqERERESkV1igEhEREZFeYYFKRERERHqFBSoRERER6RUWqERERESkV1igEhEREZFeYYFKRERERHqFBSoRERER6RUWqERERESkV1igEhEREZFeYYFKRERERHqFBSoRERER6RUWqERERESkV1igEhEREZFeYYFKRERERHqFBSoRERER6RUWqERERESkV1igEhEREZFeYYFKRERERHqFBSoRERER6RUWqERERESkV1igEhEREZFeYYFKRERERHqFBSoRERER6RUWqERERESkV1igEhEREZFeYYFKRERERHrFUOoAiH7v9OnTOH/+vMq2X375BQCwfft2le0jRozA008/rbPYiIiISDdkQgghdRBEDfbv34/JkyfDwMAAPXo8vMDf8BWVyWQAgPr6etTV1WHfvn344x//KFmsREREpB0sUEmvKBQK9OnTBxUVFc22s7Kyws2bN2FsbKyjyIiIiEhXOAaV9IqRkRFeeeWVZgvP1rQhIiKizosFKumdV155BTU1NWrfVygUePXVV3UYEREREekSb/GT3qmvr8eAAQNQXFzc5Pu2trb47bfflGNUiYiIqGvhv/Ckd3r06IH//d//bfIWvrGxMYKCglicEhERdWH8V570krrb/DU1NXjllVckiIiIiIh0hbf4SW8NHjwYeXl5KtucnZ2Rn58vUURERESkC7yCSnpr9uzZMDIyUv63sbExXn/9dQkjIiIiIl3gFVTSW3l5eRg8eLDKtkuXLsHNzU2iiIiIiEgXeAWV9JarqytGjBgBmUwGmUyGESNGsDglIiLqBligkl577bXXYGBgAAMDA7z22mtSh0NEREQ6wFv8pNeKioowcOBACCFw7do12NvbSx0SERERaZnGCtTvv/8esbGxmtgVkYqMjAwAwLPPPitpHNQ1vf322/Dx8dHKvpkXiYha1lQe1tgt/mvXriElJUVTuyNScnR0hJWVFb9f7ZCSkoLr169LHYbeSklJwbVr17S2f+ZF6gqYR9ru1KlTOHXqlNRhdArq8rChpjvatWuXpndJ3VxZWRlSU1Pxf//3f/x+tZFMJsPixYsxc+ZMqUPRSzKZTCf98HtLnRnzSNsFBAQA4G+/NdTlYT4kRXrPxsYGFhYWUodBREREOsIClYiIiIj0CgtUIiIiItIrLFCJiIiISK+wQCUiIiIivcIClbqVgwcPomfPnti3b5/Uoei9o0ePYtmyZaivr8e0adPg6OgIuVwOe3t7TJ06FVlZWW3eZ0xMDNzd3WFqagpzc3O4u7tjxYoVqKioUGkXGRkJDw8PWFlZwcTEBK6urnj33Xdx9+5dZZu9e/ciJiYGdXV1HT5WItI+5t/mzZ8/X7m0t0wmw+zZsxu10UZeftT9+/fh7u6O9957T7lNXb5NTU1ViblPnz4d7r8BC1TqVrhwWuusXLkSGzduxPLly1FfX4/jx49jx44dKCsrw4kTJ1BdXY2xY8eiqKioTfs9fvw45s6di6tXr6K4uBirV69GTEwMZsyYodIuPT0db731FgoKClBaWoro6GjExcUpp24BgClTpkAul2P8+PG4c+eORo6biLSH+bdlNjY2SEtLw6VLl5CQkKDynrby8qPCw8Nx6dIllW3q8u3UqVNx/fp1fPvtt5g4cWKH+n0UC1TqViZNmoTy8nJMnjxZ6lBQXV0NX19fqcNoZO3atUhKSkJycjIsLS0BAD4+PhgzZgzMzMzg5OSEqKgolJeX45NPPmnTvo2NjRESEgJbW1tYWFggICAA/v7++Oqrr3Djxg1lOwsLCwQHB8PGxgaWlpaYOXMmpk2bhkOHDqlM6BwaGooRI0Zg4sSJqK2t1cjxE5F2MP+2zNTUFC+++CLc3NxgYmKi3K7NvPx7J0+exM8//9zke03lW5lMBnt7e/j5+WHw4MHt7rcpLFCJJJKQkICSkhKpw1CRl5eHFStWYNWqVZDL5QAAQ0PDRrfknJ2dAQD5+flt2v/u3buV+21gb28PACq37/fv3w8DAwOVdg23jqqqqlS2R0RE4Ny5c4iLi2tTLETUfelj/lVH23m5QXV1Nd55551mc6ku8y0LVOo2Tpw4AUdHR8hkMmzevBkAEB8fD3Nzc5iZmWHPnj146aWXYGVlBQcHB3zxxRfKz27cuBFyuRx2dnaYP38++vfvD7lcDl9fX5w+fVrZbuHChTA2Nka/fv2U20JCQmBubg6ZTIbS0lIAwKJFi7BkyRLk5+dDJpPB1dUVAHDo0CFYWVkhKipKF6ekkY0bN0IIgSlTpjTbrrq6GgBgZWXV4T5zc3PRq1cvDBo0qNl2hYWFMDU1hZOTk8p2a2trjBs3DnFxcbyFSKSnmH/bT1d5OTw8XHmHSx1d5lsWqNRtjBkzBidPnlTZtmDBAixevBjV1dWwtLTEzp07kZ+fD2dnZ8ydOxcKhQLAw8QXFBSEqqoqhIaGoqCgAGfPnkVtbS1eeOEF5W3njRs3NloOcMuWLVi1apXKtri4OEyePBkuLi4QQiAvLw8AlAPQ6+vrtXIOWnLgwAEMGTIEZmZmzbb74YcfADw8p+2hUChQWFiIzZs34+jRo9i0aROMjY3Vtq+qqkJ6ejrmzp3bZLuRI0eisLAQ58+fb1c8RKRdzL/tp4u8/N133yE/Px+vvvpqi211lW9ZoBL9/3x9fWFlZQVbW1sEBgbi3r17uHr1qkobQ0NDDB06FCYmJvDw8EB8fDwqKyuRmJiokRgmTZqEiooKrFixQiP7a4t79+7h119/hYuLi9o2xcXFSEpKQmhoKHx8fFr8i16dgQMHwsHBAREREVi3bh1mzZrVbPvo6Gj0798fH3zwQZPvN4x9ys7Oblc8RCSt7p5/1dFFXq6ursaiRYsQHx/fqva6yrcsUIma0HCVruEveHVGjRoFMzMzXLx4URdhaVVJSQmEEM3+le7j44PQ0FD4+/sjLS0NRkZG7err2rVrKCkpwY4dO/Dpp59i5MiRaseD7d69G8nJyTh8+LDy4YBHNcRcXFzcrniISH90x/yrji7y8vLlyzFv3jzl8wAt0VW+NdTq3om6ARMTE9y8eVPqMDrs/v37AKDy5Oij7OzskJCQgGHDhnWoLyMjI9ja2mLChAlwcnKCm5ubciqp30tKSkJsbCwyMjIwYMAAtfszNTVVOQYi6h66Sv5VR9t5+cSJE8jOzkZsbGyrP6OrfMsrqEQdoFAocOfOHTg4OEgdSoc1JJ3mJr63tbVFr169NNqvq6srDAwMkJOTo7J906ZN+Oyzz5Cent5scQoANTU1AP57DETU9XWl/KuOtvNyQkICjh07hh49eign2294SCoqKgoymQw//vijymd0lW9ZoBJ1QEZGBoQQ8Pb2Vm4zNDRs8daUPrKzs4NMJkN5ebnaNvv27Wv1baBH3bp1q8kB+Lm5uairq8PAgQMBPJzMOywsDNnZ2UhNTYWFhUWL+26IuW/fvu2KjYg6n66Uf9XRdl5OTEyEEELl1XBFOjw8HEIIjBo1SuUzusq3LFCJ2qC+vh63b99GbW0tsrKysGjRIjg6OiIoKEjZxtXVFWVlZUhNTYVCocDNmzdx5cqVRvuysbFBUVERCgoKUFlZCYVCgbS0NMmmOTEzM4OzszOuX7/e5Pt5eXno27dvkw80BQYGom/fvjh79qza/Zubm+PIkSNIT09HRUUFFAoFMjMz8frrr8Pc3Bxvv/02AODChQtYt24dPv74YxgZGaksoyeTybB+/fpG+26I2cvLqz2HTkSdQFfOv+poOy+3h67yLQtU6jY2b96M0aNHAwDCwsIwdepUxMfHY8OGDQCA4cOH45dffsHHH3+MJUuWAABefPFF5ObmKvdx//59eHl5wdTUFH5+fnBzc8PXX3+tMj5owYIFeO655/DKK69gyJAhWL16tfJWiI+Pj3JKlDfffBN2dnbw8PDAxIkTUVZWppPz0JxJkyYhJydHOZ/e7zU3511NTQ1KSkqwZ88etW3kcjmeeeYZzJkzB/b29rC0tERAQAAee+wxnDp1Cp6eni32o86ZM2dgb2+P4cOHt/mzRKR9zL/tp8283B46y7dCQ3bu3Ck0uDsiFfrw/QoODhY2NjaSxtBWAMTOnTtb3T43N1cYGhqKf/3rX23qp66uTvj5+YmEhIS2hthhpaWlQi6Xi/Xr17f5s209P22lD99boo7S9u+kNTpb/p0xY4aYMWNGmz4THBws7O3tG23Xp7zcXL4NDQ0VvXv3bvM+1X2/eAWVqA2aG6jeFbi6uiIyMhKRkZEqS482p66uDqmpqaisrERgYKCWI2wsIiICjz/+OBYuXKjzvolId7p6/gUezkl6+PBh5ObmKh9G0qe8/Gi+FUKgqKgIJ06cUC54oCksUIlIxbJlyxAQEIDAwMBmB+Y3yMjIwJdffom0tLQWVzrRtNjYWJw7dw4HDx5s95ysRET6oqysDC+++CLc3Nzw5z//WbldH/JyU/l2z549sLe3h5+fHw4cOKCRfhroVYE6Z84cWFpaQiaT4dy5c1KH0y4xMTFwd3eHqakpzM3N4e7ujhUrVqCioqJd7Vrjyy+/hLOzc6OHSYyNjWFnZ4dnn30WH374IW7fvq2pw+x2li9fjsTERJSXl8PJyQkpKSlSh6RVUVFRWLhwIdasWdNi2/Hjx+Pzzz9XWf9aF/bs2YMHDx4gIyMD1tbWOu1bmw4ePIiePXti3759UofSIZGRkfDw8ICVlRVMTEzg6uqKd999t9VXgH7v1KlTGDp0qHIqnL59+6pdVUwqj+bhfv36Yfbs2VKH1SV0l/y7detWlafpP/vsM5X3pczL6vKtv7+/SsylpaUa6Q+A/o1B/eKLLwQAkZmZqYGodG/SpEli/fr1oqSkRFRWVork5GRhZGQkXnjhhXa1awsXFxfRs2dPIYQQ9fX14vbt2+Lrr78WQUFBQiaTif79+4szZ8506PikwrF87QM9GDumz7R9ftrzvd2/f7+wsrISe/fu1VJUujFu3DixZcsWcevWLVFRUSF27twpjIyMxIsvvtjuff7P//yPACBu376twUg16/d5uKtgHmm79oxB7a7Ufb/06gpqV2BsbIyQkBDY2trCwsICAQEB8Pf3x1dffYUbN260uV17yWQy9OrVC88++ywSExORnJyM4uJiTJo0qVW3B4hIGg2/0cmTJ0sdCqqrq+Hr69uuz1pYWCA4OBg2NjawtLTEzJkzMW3aNBw6dEj5JHVn1pFzQ0Qt07sCVSaTSR1Ch+zevRtyuVxlW8MEur+/tdXadpoyY8YMBAUFoaSkBFu3btX4/omo60lISEBJSUm7Prt//34YGBiobOvTpw8AoKqqqsOxSa0j54aIWiZpgSqEwIcffoghQ4bAxMQEPXv2xDvvvNOoXV1dHd5//304OjrC1NQUw4cPx86dOwEA8fHxMDc3h5mZGfbs2YOXXnoJVlZWcHBwwBdffKGyn2+++QZPPfUUzMzMYGVlBS8vL+WYz+b66Kjc3Fz06tULgwYNanO7Q4cOaWzi4IbJjNPS0pTbOvu5JepKTpw4AUdHR8hkMmzevBlA63+HGzduhFwuh52dHebPn4/+/ftDLpfD19cXp0+fVrZbuHAhjI2NVcamhYSEwNzcHDKZTDmGbNGiRViyZAny8/Mhk8ng6ura4eMrLCyEqakpnJyclNs6kuM6+7k5fvw4PDw80LNnT8jlcnh5eeHw4cMAHj6T0TCe1cXFBZmZmQCAN954A2ZmZujZsyf27t0LoPkcu27dOpiZmcHS0hIlJSVYsmQJ7O3tcenSpXbFTKQzmhpD0J6xVuHh4UImk4m//vWv4vbt26Kqqkps2bKl0RjUv/zlL8LExESkpKSI27dvi+XLl4sePXoox1OGh4cLAOLYsWOivLxclJSUCD8/P2Fubi5qamqEEELcvXtXWFlZiZiYGFFdXS1+++038fLLL4ubN2+XzuHlAAAgAElEQVS2qo+2qqmpEdevXxebNm0SJiYmaucva6nd/v37haWlpYiMjGyxz5bGPlVUVAgAYuDAgcptneXccgxq+4Bjx5ql7fPTnu/ttWvXBACxadMm5bbW/A6FeDiPorm5ubhw4YK4f/++yMnJEaNHjxaWlpbi6tWrynZ/+tOfRN++fVX6/fDDDwUA5e9WCCGmT58uXFxc2nrYTbp3756wtLQUCxcuVNnelhzX1BhUfTs3bRmDumvXLhERESHKysrErVu3hLe3t8o8ktOnTxcGBgaisLBQ5XOvvvqqyhjl1ubx0NBQsWnTJvHyyy+L//znP62KUQjmkfbgGNTWU/f9kqxAraqqEmZmZo0eCnr0Ianq6mphZmYmAgMDVT5rYmIiFixYIIT474+vurpa2aah0M3LyxNCCPHzzz8LAGL//v2NYmlNH23Vt29fAUD07t1bfPTRRyqJsj3tWqM1iVEmk4levXoJITrXuWWB2j78h6V5na1Abe53KMTDIuzRHHDmzBkBQKxatUq5TYoCNTw8XLi5uYmKiop276O5AlVfzk1HHpKKjo4WAERJSYkQQoijR48KAOKDDz5QtikvLxeDBw8WtbW1Qoj25/G2YB5pOxaorafu+yXZLf68vDxUVVVh/Pjxzba7dOkSqqqqlMsgAoCpqSn69euHixcvqv2csbExAEChUAAAnJ2dYWdnh9mzZyMiIgIFBQUd7qM5165dQ0lJCXbs2IFPP/0UI0eObHK8UmvbacK9e/cghICVlRWAznluH51Ki6/mXwAwa9YsyePQ11dn9ujvUJ1Ro0bBzMys3blME3bv3o3k5GQcPnwYlpaWWu+vM52b32uYW7JhQvo//OEPcHNzwz/+8Q/lkpZJSUkIDAxUju/Vxr9fTWEeadsrJSUFKSkpksfRGV7qGGrs29tG169fBwDY2to22+7evXsAgPfeew/vvfeeynv9+/dvdX+mpqZIT0/H0qVLERUVhcjISMycOROJiYka6+P3jIyMYGtriwkTJsDJyQlubm6Ijo5GXFxcu9ppwuXLlwEA7u7uADrnueXY1baZNWsWFi1aBB8fH6lD0UuzZs2SOgSdMDExwc2bNyXpOykpCbGxscjIyMCAAQMkiaE5Up6bAwcO4MMPP0ROTg4qKioaFdQymQzz58/H22+/jWPHjuH555/HP//5T3z++efKNtr496spzCNts2HDBgDA4sWLJY5E/6nLw5IVqA1PsD948KDZdg0F7IYNG7Bo0aIO9Tls2DDs27cPN2/eRGxsLNauXYthw4YplwHTRB9NcXV1hYGBAXJycjTSrr0OHToEAHjppZcAdM5zO3PmzA7vozuZNWsWfHx8eN7U6A4FqkKhwJ07d+Dg4KDzvjdt2oTDhw8jPT0dFhYWOu+/Jbo+N99++y1++uknLF68GFevXsW0adPw8ssv4x//+AcGDBiATZs24d1331X5TFBQEJYvX46///3vGDhwIKysrFQepNVkHm8O80jb7Nq1CwD/zWoNdXlYslv8np6e6NGjB7755ptm2w0cOBByubzDK0sVFRXhwoULAB7+oNesWYMnnngCFy5c0Fgft27dwquvvtpoe25uLurq6jBw4MA2tdOk3377DRs2bICDg4Ny+bTOdG6JqH0yMjIghIC3t7dym6GhYYu3vztCCIGwsDBkZ2cjNTVVL4tTQPfn5qeffoK5uTkAIDs7GwqFAgsWLICzszPkcnmTtzutra0xa9YspKamYv369Zg7d67K+8yx1FVJVqDa2tpi+vTpSElJQUJCAioqKpCVlYXt27ertJPL5XjjjTfwxRdfID4+HhUVFairq8P169fbNKF9UVER5s+fj4sXL6KmpgaZmZm4cuUKvL29NdaHubk5jhw5gvT0dOXtmszMTLz++uswNzfH22+/3aZ2wMMpodoyBYsQAnfv3kV9fT2EELh58yZ27tyJZ555BgYGBkhNTVWOQe1M55aIWqe+vh63b99GbW0tsrKysGjRIjg6OiqnmQMe3q0pKytDamoqFAoFbt68iStXrjTal42NDYqKilBQUIDKyspWF24XLlzAunXr8PHHH8PIyKjRmLP169cr27Y1x3WEVOdGoVCguLgYGRkZygLV0dERAHD06FHcv38fubm5KlNe/d6bb76JBw8eYP/+/Y0WcGCOpS5LU09htedp1crKSjFnzhzRu3dvYWFhIcaMGSPef/99AUA4ODiI8+fPCyGEePDggQgLCxOOjo7C0NBQ2NraiunTp4ucnByxZcsWYWZmJgCIwYMHi/z8fLF9+3ZhZWUlAIhBgwaJy5cvi4KCAuHr6yusra2FgYGBGDBggAgPD1c+CdlcH20xZcoU4eTkJCwsLISJiYlwcXERgYGBIjs7u13tDh48KCwtLVWe4nzU3r17xfDhw4WZmZkwNjYWPXr0EACUT+w/9dRTIjIyUty6davRZzvLueVT/O0DPn3bLG2fn7Z+bzdt2iT69esnAAgzMzMxZcqUVv8OhXj4pLqRkZGwt7cXhoaGwsrKSvj7+4v8/HyVfm7duiWee+45IZfLhZOTk/h//+//iXfeeUcAEK6ursppl86ePSsGDRokTE1NxZgxY8Rvv/3WquPIzs4WANS+PvzwQ2Xb1uS4U6dOiWHDhilzW79+/URUVJRenZu//e1vwsXFpdnjBiB2796t7CssLEzY2NiIXr16iYCAALF582YBQLi4uKhMfSWEECNHjhTLli1r8vw0l2NjYmKEqampcopBdVMeNod5pO34FH/rqft+SVqgErUWv1/tw39YmqdvBWpHBQcHCxsbG53115l09nMzceJE8csvv0jSN/NI27FAbT113y+9W+qUiIjar2GKImqsM52b3w8ZyMrKglwux+9X4CLq6ligtuDixYutmser4Wl1oq7i6NGjWLZsGerr6zFt2jQ4OjpCLpfD3t4eU6dORVZWVpv3GRMTA3d3d5iamsLc3Bzu7u5YsWKFclncBpGRkfDw8ICVlRVMTEzg6uqKd999F3fv3lW22bt3L2JiYjpV0dGZMRfqVlhYGHJzc3H58mW88cYbWL16tdQhkZbNnz9f5bc0e/bsRm20kZcfdf/+fbi7u6tMW6Yu36ampqrE3KdPnw7334AFagvc3d0hHg6FaPaVlJQkdahEGrNy5Ups3LgRy5cvR319PY4fP44dO3agrKwMJ06cQHV1NcaOHYuioqI27ff48eOYO3curl69iuLiYqxevRoxMTGYMWOGSrv09HS89dZbKCgoQGlpqXJu4ICAAGWbKVOmQC6XY/z48bhz545GjrszW758ORITE1FeXg4nJyekpKRodP+dORdq+9xog5mZGdzd3fH8888jIiICHh4eUodEOmBjY4O0tDRcunQJCQkJKu9pKy8/Kjw8HJcuXVLZpi7fTp06FdevX8e3336LiRMndqjfRjQ1hoBjBEmb9OH7VVVVJXx8fDpVH2jH2LE1a9YINzc35bKICoVC/PGPf1Rp88MPPwgAIioqqk37njZtWqPlFgMCAgQAUVRUpNw2adIk5UN2DWbOnCkANHp4ZOHChcLHx0coFIo2xSJE1xuDSqQN2v6dtKQz5t72jEENDg4W9vb2Tb6nzbz8e999952YMGGCACDCw8Mbvd9cvg0NDRW9e/duc5/qvl+8gkrUSgkJCVpbhlaXfTQnLy8PK1aswKpVq5SLaRgaGmLfvn0q7ZydnQEA+fn5bdr/7t27lfttYG9vDwAqt+/379+vXMqxQcOto6qqKpXtEREROHfunFZWXyMi6XWH3NscbeflBtXV1XjnnXeazaW6zLcsUKnLEkIgNjYWQ4cOhYmJCaytreHv76+yPvXChQthbGyMfv36KbeFhITA3NwcMpkMpaWlAB4u87dkyRLk5+dDJpPB1dUVGzduhFwuh52dHebPn4/+/ftDLpfD19dXZT7DjvQBPFwBTFfzRG7cuBFCCEyZMqXZdtXV1QCgnFO3I3Jzc9GrVy+V1XGaUlhYCFNT00YPilhbW2PcuHGIi4tTrldORNJh7tUsXeXl8PBwhISENLsEvS7zLQtU6rIiIiKwbNkyhIeHo6SkBN9++y2uXbsGPz8/FBcXA3j4w390KbotW7Zg1apVKtvi4uIwefJkuLi4QAiBvLw8LFy4EEFBQaiqqkJoaCgKCgpw9uxZ1NbW4oUXXsC1a9c63Afw3yeP6+vrNXdy1Dhw4ACGDBkCMzOzZtv98MMPAIAxY8a0qx+FQoHCwkJs3rwZR48exaZNm2BsbKy2fVVVFdLT0zF37twm240cORKFhYU4f/58u+IhIs1h7tUsXeTl7777Dvn5+U2ucvkoXeVbFqjUJVVXVyM2NhYvv/wyZs+ejZ49e8LLywtbt25FaWlpoxXLOsLQ0FB5pcDDwwPx8fGorKxEYmKiRvY/adIkVFRUYMWKFRrZnzr37t3Dr7/+ChcXF7VtiouLkZSUhNDQUPj4+LT4F706AwcOhIODAyIiIrBu3Tq1azE3iI6ORv/+/fHBBx80+f7gwYMBPFw+koikw9yrWbrIy9XV1Vi0aBHi4+Nb1V5X+ZYFKnVJOTk5uHv3LkaNGqWyffTo0TA2Nla7pKAmjBo1CmZmZiq3szqDkpISCCGa/Svdx8cHoaGh8Pf3R1paGoyMjNrV17Vr11BSUoIdO3bg008/xciRI9WO/9q9ezeSk5Nx+PBhWFpaNtmmIeaGqzNEJA3mXs3SRV5evnw55s2bp3weoCW6yreGWt07kUQapsGwsLBo9F6vXr1QWVmp1f5NTExw8+ZNrfahaffv3wfwMHZ17OzskJCQgGHDhnWoLyMjI9ja2mLChAlwcnKCm5ubciqp30tKSkJsbCwyMjIwYMAAtfszNTVVOQYikgZzr2ZpOy+fOHEC2dnZiI2NbfVndJVveQWVuqRevXoBQJPJ8M6dO3BwcNBa3wqFQut9aEND0mlu4ntbW1vludUUV1dXGBgYICcnR2X7pk2b8NlnnyE9Pb3Z4hQAampqAPz3GIhIGsy9mqXtvJyQkIBjx46hR48eysn2Gx6SioqKgkwmw48//qjyGV3lWxao1CV5enrCwsKi0Q/r9OnTqKmpwZNPPqncZmhoqLKsYEdlZGRACAFvb2+t9aENdnZ2kMlkKC8vV9tm3759rb4N9Khbt241OQA/NzcXdXV1GDhwIICHTwCHhYUhOzsbqampTV6JeVRDzH379m1XbESkGcy9mqXtvJyYmNhosY2GK9Dh4eEQQjQarqGrfMsClbokuVyOJUuWYPfu3fjss89QUVGB7OxsvPnmm+jfvz+Cg4OVbV1dXVFWVobU1FQoFArcvHkTV65cabRPGxsbFBUVoaCgAJWVlcqkV19fj9u3b6O2thZZWVlYtGgRHB0dERQUpJE+0tLSdDLViZmZGZydnXH9+vUm38/Ly0Pfvn2bfKApMDAQffv2xdmzZ9Xu39zcHEeOHEF6ejoqKiqgUCiQmZmJ119/Hebm5nj77bcBABcuXMC6devw8ccfw8jIqNFSmuvXr2+074aYvby82nPoRKQhzL2ape283B66yrcsUKnLWrlyJaKjoxEZGYk+ffpg3LhxeOyxx5CRkQFzc3NluwULFuC5557DK6+8giFDhmD16tXKWxc+Pj7KKUvefPNN2NnZwcPDAxMnTkRZWRmAh+NwvLy8YGpqCj8/P7i5ueHrr79WGTPU0T50ZdKkScjJyVHOp/d7zc15V1NTg5KSEuzZs0dtG7lcjmeeeQZz5syBvb09LC0tERAQgMceewynTp2Cp6dni/2oc+bMGdjb22P48OFt/iwRaRZzr2ZpMy+3h87ybZvXpFKDS/qRNunr9ys4OFjY2NhIHYZaaOMShbm5ucLQ0FD861//alM/dXV1ws/PTyQkJLQ1xA4rLS0VcrlcrF+/vs2fbev5aSt9/d4StYW2fyftoe+5V5NLnepTXm4u33KpUyI909zg9c7G1dUVkZGRiIyMVFl6tDl1dXVITU1FZWUlAgMDtRxhYxEREXj88cexcOFCnfdNRNLpSrm3QXV1NQ4fPozc3Fzlw0j6lJcfzbdCCBQVFeHEiRPKBQ40hQUqEalYtmwZAgICEBgY2OzA/AYZGRn48ssvkZaW1uJKJ5oWGxuLc+fO4eDBg+2ek5WISF+UlZXhxRdfhJubG/785z8rt+tDXm4q3+7Zswf29vbw8/PDgQMHNNJPAxaoRO20fPlyJCYmory8HE5OTkhJSZE6JI2JiorCwoULsWbNmhbbjh8/Hp9//rnKete6sGfPHjx48AAZGRmwtrbWad9EJJ2umnu3bt2q8jT9Z599pvK+lHlZXb719/dXibm0tFQj/QGcqJ+o3aKjoxEdHS11GFozYcIETJgwQeow1Jo6dSqmTp0qdRhEpGNdPfc2R6q8LEW+5RVUIiIiItIrLFCJiIiISK+wQCUiIiIivcIClYiIiIj0isYfkkpOTtb0Lonw/fffA+ic3y8hBGQymWT9N5w7kk5n/N6SelL/pqXAPNI2DcuB8rffAW2e8l+NhhVT+OKLL74600sXK0nxxRdffPGl/tVUHpYJ0Y6Fr4moVWpqahAVFYU1a9Zg9OjRSEhIgLu7u9RhEVEbCCHwySefYMmSJbC2tsb27dsxfvx4qcMi6tI4BpVIi4yNjbFq1SqcPXsWCoUCI0eOREREhHIJOyLSbwUFBXjxxRcxZ84cBAQE4Pz58yxOiXSABSqRDnh6euLkyZNYu3Yt1q9fj9GjR+Onn36SOiwiUkMIge3bt8PLywuFhYX47rvvsG3bNlhYWEgdGlG3wAKVSEcMDQ0RGhqK8+fPo3fv3vD29sbSpUvx4MEDqUMjot/Jzc3Fc889h7feegshISH46aef4O3tLXVYRN0KC1QiHXNxccGxY8ewZcsWxMfH48knn8Tp06elDouo26utrUVMTAy8vLxQXl6OU6dOYe3atTAxMZE6NKJuhwUqkQRkMhnmzZuHrKwsDBgwAL6+vggODsbdu3elDo2oWzp37hyefvpprFq1CqtWrcKPP/6IJ554QuqwiLotFqhEEnrsscdw5MgRJCUlISUlBSNGjMCxY8ekDouo26iursbSpUsxatQomJubIzMzE2FhYTAwMJA6NKJujQUqkR4ICAhATk4OHn/8cbzwwgsIDg5GRUWF1GERdWknTpzAyJEjsXXrVvz1r39FRkYGhgwZInVYRAQWqER6o1+/fvjyyy+xc+dO/Pvf/4a7uzv27NkjdVhEXU55eTlCQ0Mxbtw4uLq6Ijs7G6GhoejRg/8kEukL/hqJ9ExAQAAuXbqEyZMnw9/fHzNnzkRpaanUYRF1CQcPHoSXlxeSkpKQmJiI/fv3Y+DAgVKHRUSPYIFKpIesra2xbds2HDhwAN9//z08PT2RkpIidVhEndbt27cRHByMSZMmwdvbGzk5OXjttdekDouI1GCBSqTHJk6ciJ9//hlTp07FzJkzMXnyZBQWFkodFlGnsmvXLgwZMgT79+/Hv//9byQnJ6NPnz5Sh0VEzWCBSqTnevbsiW3btiEjIwOXLl2Cp6cntm/fLnVYRHrvxo0bmDZtGmbNmoVp06bh4sWL8Pf3lzosImoFFqhEncTYsWNx7tw5BAcH480338TEiRNx9epVqcMi0jsNy5S6u7sjOzsbx44dw7Zt22BpaSl1aETUSixQiToRMzMzrF27FsePH8evv/4KLy8vfPTRR6ivr5c6NCK9kJ+fj+effx4hISEICgrC+fPn8dxzz0kdFhG1EQtUok7I19cXmZmZWLx4Md555x2MGzcOly9fljosIsnU1tbio48+wogRI1BaWorvv/8eH330EczNzaUOjYjagQUqUScll8sRERGBM2fOoKqqCo8//jhiYmJQV1cndWhEOpWVlQVfX18sXboUf/nLX3DmzBmMGjVK6rCIqANYoBJ1ciNGjMCpU6ewcuVKrFy5EmPGjMGFCxekDotI6xQKBWJiYjB69GgYGRnh7NmziIiIgLGxsdShEVEHsUAl6gKMjIwQFhaGH3/8EfX19Xj88cexdOlS1NTUSB0akVacPHkSI0aMQGRkJCIjI3H8+HEMHTpU6rCISENYoBJ1IZ6envj++++xefNmbN68GaNHj8aPP/4odVhEGlNVVYWlS5di7NixGDRoEC5cuICwsDAuU0rUxfAXTdTF9OjRA/PmzUNWVhZ69+4NHx8fLF26FA8ePJA6NKIOOXz4MDw8PLBt2zbEx8cjLS0NgwYNkjosItICFqhEXZSzszOOHTuGLVu2ID4+Hp6envjmm2+kDouoze7cuYPg4GC89NJL8PLyQk5ODubNmyd1WESkRSxQibowmUyGefPm4eLFi/Dw8MBzzz2H4OBg3L17V+rQiFpl37598PT0xN69e7Fr1y7s27cPAwYMkDosItIyFqhE3cCAAQOwZ88e7Ny5E19++SWGDx+Oo0ePSh0WkVq//fYbZsyYgalTp+IPf/gDcnJyMH36dKnDIiIdYYFK1I0EBATg559/xhNPPIEJEybgtddeQ1lZmdRhEanYtWsXPD09cfbsWRw5cgT//Oc/YWNjI3VYRKRDLFCJupl+/fohJSUFO3fuxOHDh+Hp6YnU1FSpwyLCr7/+igkTJiAwMBDTp09HVlYWnn/+eanDIiIJsEAl6qYCAgJw8eJFTJ48GdOmTcPMmTNRWloqdVjUDdXX12P79u0YPnw4bty4gZMnT2Lbtm2wsLCQOjQikggLVKJuzNraGtu2bcOBAwdw6tQpDBs2DP/85z+lDou6kZycHPj6+uKtt95CSEgIfvzxRzz99NNSh0VEEmOBSkSYOHEisrOz4e/vj6CgIEyePBmFhYVSh0VdWMMypU8++SRqampw+vRprF27FiYmJlKHRkR6gAUqEQEAevbsiW3btuGbb77BpUuX4Onpie3bt0MIIXVo1MWcO3cO3t7eWLVqFVatWoUzZ85g5MiRUodFRHqEBSoRqfDz88P58+cRHByMBQsWYOLEibh69arUYVEXUF1djaVLl2LUqFGwsLDAuXPnEBYWBgMDA6lDIyI9wwKViBoxNTXF2rVrcfz4cVy5cgVDhw5FTEwM6uvrpQ6NOqnjx4/j8ccfx9atWxEfH4+MjAy4ublJHRYR6SkWqESklo+PD86ePYt33nkH77//PsaOHYtLly5JHRZ1IuXl5QgNDcWzzz4LNzc3/Pzzz5g3bx5kMpnUoRGRHmOBSkTNksvliIiIwJkzZ3D//n2MHDkSMTExqKurkzo00nMHDhyAl5cXdu7cicTEROzbtw8ODg5Sh0VEnQALVCJqleHDh+PUqVNYuXIlVq5cidGjRyMzM1PqsEgPlZSU4LXXXsMf//hHeHt74+eff8Zrr70mdVhE1ImwQCWiVjM0NERYWBh++uknGBsb4+mnn8bSpUtRU1PT4mc5G0D3sGvXLgwbNgzHjh1DamoqkpOT0adPH6nDIqJOhgUqEbXZsGHDcPLkSWzevBmbN2/GqFGjcObMGbXtMzMzsWDBAh1GSJq2cuVKHD16VO37RUVF8Pf3x6xZs/Dyyy/j4sWLmDp1qg4jJKKuhAUqEbVLjx49MG/ePGRlZcHW1ha+vr5YunQp7t+/r9KupqYGf/rTn7B161YkJiZKFC11RFpaGlavXo05c+agqqpK5T0hBLZv3w53d3fk5OQgPT0d27Ztg6WlpUTRElFXwAKViDrE2dkZR48exZYtW/C3v/0Nnp6eyMjIUL6/Zs0aXL58GQAwf/58nD9/XqJIqT0KCgoQGBgImUyGwsJCvP/++8r38vPzMX78eISEhGDBggXIzs7Gs88+K12wRNRlyAQHhhGRhhQVFWHBggXYu3cv5s6di3nz5sHb2xu1tbUAHo5hHTBgALKystCzZ0+Jo6WWPHjwAN7e3sjJyYFCoQAAyGQyfPvtt/jpp5+wfPlyDB48GH//+98xatQoiaMloq6EBSoRadynn36KxYsXo66uDtXV1criBgCMjIzwwgsvYP/+/ZwLU8/NnTsXn3zyifIPDODhHxmWlpa4f/8+Vq5ciSVLlsDQ0FDCKImoK+ItfiLSuNdffx0hISG4e/euSnEKAAqFAocOHcJf//pXiaKj1vj888/x97//XaU4BYDa2lpUVFRg7ty5CAsLY3FKRFrBK6hEpHGXL1+Gl5dXs9NP9ejRA19//TXGjh2rw8ioNbKysvDUU0/hwYMHatsYGhoiMzMTnp6eOoyMiLoLXkElIo2qr69HUFBQi/OeymQyTJs2DUVFRTqKjFrjzp07mDx5cosrhclkMrz22mtcUYyItIIFKhFp1N/+9jd8//33jW7tP6qurg6VlZUIDAxkkaMnhBCYPXs2bty40ejW/qMUCgUyMzOxadMmHUVHRN0JC1Qi0ihTU1P4+/ujV69eAB7eClY3TlGhUODkyZNYsWKFLkMkNWJiYpCWlqb2jwuZTAYjIyMAgImJCfz8/FosZImI2oNjUIlIa3755RccPXoUx48fx5EjR1BSUqIsVn9f2MhkMnz55ZeYNm2aVKF2e19//TWef/551NfXK7fJZDIYGBigtrYWcrkc3t7eGDduHMaMGQM/Pz+YmJhIGDERdWUsUIlIJ4QQ+M9//oNvvvkGGRkZOHbsGG7dugVDQ0PU1tbC0tIS58+fh5OTk9ShdjuFhYUYPnw4ysrKYGBggLq6OlhZWWHs2LH4wx/+AD8/P4wcORIGBgZSh0pE3QQL1C6G80oSEVF3MmPGDOzatUvqMEjDOIFdF7Ro0SL4+PhIHQZRm924cQPGxsbo3bu31KFozYYNGwAAixcvljiSh27fvo379++jf//+Uoei1vfff4+4uDjs3LlT6lBIzzT8nqjrYYHaBfn4+GDmzJlSh0FETWi40sPfaNvExcXxnFEjvHLadfEpfiIiIiLSKyxQiYiIiEivsEAlIiIiIr3CApWIiIiI9AoLVJmveS4AACAASURBVCIiIiLSKyxQiYg6oYMHD6Jnz57Yt2+f1KHopfnz50Mmkylfs2fPbtTm6NGjWLZsGerr6zFt2jQ4OjpCLpfD3t4eU6dORVZWVofjuH//Ptzd3fHee+8pt+3duxcxMTGoq6vr8P4B7RxHTEwM3N3dYWpqCnNzc7i7u2PFihWoqKhQaRcZGQkPDw9YWVnBxMQErq6uePfdd3H37l1lG3XHm5qaqvL/qE+fPu07AdQlsUAlIuqEuMZKy2xsbJCWloZLly4hISFB5b2VK1di48aNWL58Oerr63H8+HHs2LEDZWVlOHHiBKqrqzF27FgUFRV1KIbw8HBcunRJZduUKVMgl8sxfvx43Llzp0P719ZxHD9+HHPnzsXVq1dRXFyM1atXIyYmBjNmzFBpl56ejrfeegsFBQUoLS1FdHQ04uLiEBAQ0OLxTp06FdevX8e3336LiRMndug8UNfDApWIqBOaNGkSysvLMXnyZKlDQXV1NXx9faUOoxFTU1O8+OKLcHNzg4mJiXL72rVrkZSUhOTkZFhaWgJ4OH/0mDFjYGZmBicnJ0RFRaG8vByffPJJu/s/efIkfv755ybfCw0NxYgRIzBx4kTU1ta2a//aPA5jY2OEhITA1tYWFhYWCAgIgL+/P7766ivcuHFD2c7CwgLBwcGwsbGBpaUlZs6ciWnTpuHQoUO4du1as8crk8lgb28PPz8/DB48uF3ngLouFqhERNQhCQkJKCkpkTqMVsnLy8OKFSuwatUqyOVyAIChoWGjoRLOzs4AgPz8/Hb1U11djXfeeQdxcXFq20RERODcuXPNtlFH28exe/du5X4b2NvbA4DK7fv9+/fDwMBApV3DrfqqqiqV7R05Xup+WKASEXUyJ06cgKOjI2QyGTZv3gwAiI+Ph7m5OczMzLBnzx689NJLsLKygoODA7744gvlZzdu3Ai5XA47OzvMnz8f/fv3h1wuh6+vL06fPq1st3DhQhgbG6Nfv37KbSEhITA3N4dMJkNpaSmAh0srL1myBPn5+ZDJZHB1dQUAHDp0CFZWVoiKitLFKWm1jRs3QgiBKVOmNNuuuroaAGBlZdWufsLDw5VXINWxtrbGuHHjEBcX1+YhG7o6jt/Lzc1Fr169MGjQoGbbFRYWwtTUFE5OTirbO3K81P2wQCUi6mTGjBmDkydPqmxbsGABFi9ejOrqalhaWmLnzp3Iz8+Hs7Mz5s6dC4VCAeBh4RkUFISqqiqEhoaioKAAZ8+eRW1tLV544QXlbdmNGzc2Wlp0y5YtWLVqlcq2uLg4TJ48GS4uLhBCIC8vDwCUD8TU19dr5Ry014EDBzBkyBCYmZk12+6HH34A8PBct9V3332H/Px8vPrqqy22HTlyJAoLC3H+/Pk29aGL4wAAhUKBwsJCbN68GUePHsWmTZtgbGystn1VVRXS09Mxd+7cJtu193ip+2GBSkTUxfj6+sLKygq2trYIDAzEvXv3cPXqVZU2hoaGGDp0KExMTODh4YH4+HhUVlYiMTFRIzFMmjQJFRUVWLFihUb2pwn37t3Dr7/+ChcXF7VtiouLkZSUhNDQUPj4+LR4hfJR1dXVWLRoEeLj41vVvmHsZXZ2dqv70MVxNBg4cCAcHBwQERGBdevWYdasWc22j46ORv/+/fHBBx80+X57jpe6JxaoRERdWMNVrIYrqOqMGjUKZmZmuHjxoi7CkkRJSQmEEM1edfTx8UFoaCj8/f2RlpYGIyOjNvWxfPlyzJs3TzlesyUNsRQXF7e6D10cR4Nr166hpKQEO3bswKeffoqRI0eqHW+8e/duJCcn4/Dhw8qHth7VnuOl7slQ6gCIiEg/mJiY4ObNm1KHoTX3798HAJUn+h9lZ2eHhIQEDBs2rM37P3HiBLKzsxEbG9vqz5iamqrE9v+1d6dRUV3ZHsD/JVAUVcyGKSAq4CwOiSaCOC1bEqUVjRDR+F7otD6nfmCkexk1RkTBmPRCngOdp49FuuMAoi6IrRjbGBJNNw4tisFWAYMTCgoyCKUU1HkfXFSsZiwoqAL+v7XqQ26de/a+tyBubt27T2t09HG8zMzMDA4ODvD390f//v0xcOBATSuplyUlJSE2NhYZGRl49dVXm5yvLcdLPROvoBIREVQqFcrKyuDm5mboVDpMfXHUXIN8BwcH2Nratmn+hIQEfPvtt+jVq5em+Xz9Q1LR0dGQSCS4ePGi1j41NTVaubVGRx9HU7y8vGBiYoKcnByt7Tt27MDevXtx+vTpZotToG3HSz0TC1QiIkJGRgaEEBg3bpxmm6mpaYu3BnQljo6OkEgkKC8vb3LM0aNHW/31/L9LTEyEEELrVX9Fet26dRBCYMyYMVr71Ofi5OTU6jgdfRwlJSWNPuCVm5uLuro69OnTB8CLxSJWr16Nq1evIjU1FZaWli3O3ZbjpZ6JBSoRUQ+kVqvx5MkT1NbWIjs7GytXroS7uztCQ0M1Y7y8vFBaWorU1FSoVCo8evQIt2/fbjCXvb09CgsLUVBQgMrKSqhUKqSnpxtdmym5XA4PDw/cu3ev0ffz8vLg5OTU6INAISEhcHJywqVLl/SaU30u3t7erY7T0cehUChw8uRJnD59GhUVFVCpVMjKysL7778PhUKBVatWAQCuXbuGzz77DHv27IGZmZnWsqUSiQR//OMfWzxeoqawQCUi6mJ27tyJsWPHAgBWr16NwMBAxMfHY9u2bQCAESNG4NatW9izZw8iIiIAAG+//TZyc3M1czx79gze3t6wsLDAhAkTMHDgQHz33Xda9zUuX74cU6ZMwfz58zFo0CBs2rRJ89Wsj4+PpiXVsmXL4OjoiKFDh2LGjBkoLS3tlPPQFgEBAcjJydH0B31Zc705a2pqUFxcjLS0NL3mc+HCBbi6umLEiBE6xenI45DJZBg/fjwWLVoEV1dXWFlZITg4GP369UNmZiaGDx/eYpym/PvxEjVJULcCQCQnJxs6DSJqQlBQkAgKCjJoDkuWLBH29vYGzUEXycnJQtd/rpYsWSJcXV0bbM/NzRWmpqbiq6++0mm+uro6MWHCBJGQkKDTfs15/PixkMlk4o9//KPOcYzpOFqrseOtFx4eLnr37q3znMbw+0Qdg1dQiYh6oOYesOkulEolvvnmG+Tm5moezvHy8kJUVBSioqK0luxsTl1dHVJTU1FZWYmQkBC95RcZGYlRo0YhLCxM5zjGdByt9e/HK4RAYWEhzp49q1nggageC1QiI6JWq7Ft2zb4+vrqZb4bN27gv//7vzFs2DBYWVnB1NQUNjY2GDhwIAICAvCPf/xDL3GIjFFpaSnefvttDBw4EB988IFm+5o1axAcHIyQkJBmHzSql5GRgcOHDyM9Pb3FlZtaKzY2FpcvX8bx48c1PUp1jWMMx9FajR1vWloaXF1dMWHCBBw7dqxT8yHjxwKVyEjk5uZi4sSJWLVqFaqrq9s9X0JCAry9vZGdnY3Y2FjcvXsXVVVVyMrKwqZNm1BWVsbVXHqgtWvXIjExEeXl5ejfvz8OHTpk6JQ6xBdffKH1NP3evXu13o+OjkZYWBi2bNnS4lxTp07Fvn374OzsrJfc0tLS8Pz5c2RkZMDOzq5dcQx5HK3V1PHOnj1b6zN6/Phxp+ZFxo2N+qlLUSqVmDp1aoN1yLt67CtXriAqKgrLli1DVVVVmx4+eFlmZiaWLFmCSZMm4ZtvvoGp6S+/6h4eHvDw8ICtra3WQzPGprt+1oYWExODmJgYQ6dhFPz9/eHv79/pcQMDAxEYGKi3+Qx1HK2l7+OlnoEFKnUpCQkJTS6z15Vjjxw5EocPHwbwoul1e1dZ2bx5M+rq6rBlyxat4vRlb731Ft566612xelI3fWzJiKilvErfsJXX32FMWPGQCaTQaFQoF+/fti0aROAFzexx8bGYsiQITA3N4ednR1mz56ttV53fHw8FAoF5HI50tLSMH36dFhbW8PNzQ0HDhzQKd6ZM2cwdOhQ2NjYQCaTwdvbG9988w0AYOXKlYiIiEB+fj4kEgm8vLwAvLjx/5NPPoG7uzssLCwwYsQIJCcn65ybvmN3hBMnTrTYW7KmpgbffvstevfujTfeeKPVc/OzNq7PmoioRzNU+wDqGNCxzdS2bdsEALFlyxZRUlIiSktLxf/+7/+K9957TwghxCeffCKkUqn46quvRFlZmcjOzhavvfaaeOWVV8TDhw8186xbt04AEN9++60oLy8XxcXFYsKECUKhUIiamppWx0tJSRGRkZGitLRUlJSUiHHjxmm1Hpk7d67w9PTUOobf//73wtzcXBw6dEg8efJErF27VvTq1UtcuHBBp9w6InZbvPnmm2LkyJGNvvfXv/5VWFlZiaioqCb3v3nzpgAgxo0bp1Ncftad81mzLY7u2tJminoG/j51X/yN72Z0KVBramqEra2tmDJlitb22tpaERcXJ6qrq4WlpaUICQnRev/8+fMCgFaRVF8YKJVKzbZdu3YJACIvL69V8RoTExMjAIji4mIhRMPCQalUCrlcrpVjdXW1MDc3F8uXL291bh0Vuy2aK1Bb4+LFiwKA+NWvftXqffhZd95nzX9QdccClZrC36fui/eg9mDZ2dkoKytrcB+iiYkJwsPDcfHiRTx9+rTB2tFjx46FVCrFuXPnmp1fKpUCgGYt75biNaa+HUlTPRtv3LiB6upqzcomAGBhYQFnZ2etr6Zbyq0zY3e0+vWwdekEkJOTw8+6A2I35d69ezh48KDO+/VU9e3QeM7o3927dw9ubm6GToM6AAvUHqyiogIAYGtr2+j7ZWVlAH4peF5ma2uLyspKvcYDgGPHjuHzzz9HTk6OZg3o5lRVVQEAPv74Y3z88cda77m4uOiUnyFj61O/fv0gk8lw8+bNVu/Dz7pzP+vMzMxG10mn5vGcUWOCgoIMnQJ1AD4k1YO9+uqrANBk77n64qKx4qSsrEznv1pbinfnzh3MmTMHzs7OOHfuHMrLy7F169Zm53RwcAAAbNu2TaufnhBCpyb0hoytb+bm5njrrbfw+PFj/Pjjj02OKy0txaJFiwDws+7szzooKKjBPHw1/ap/GM3QefBlfC8Wp90XC9QerF+/frC3t8fJkycbfX/48OGwtLTExYsXtbafO3cONTU1eP311/Ua7+rVq1CpVFi+fDk8PDwgk8kgkUianbNPnz6QyWS4fPmyTrkYU+yOEBkZCXNzc6xatQpKpbLRMT/99JOmBRU/6677WRMRdUcsUHswc3NzrF27Fj/88APCwsJw//59qNVqVFZW4tq1a5DJZIiIiMCRI0ewd+9eVFRU4OrVq1i2bBlcXFywZMkSvcZzd3cHAJw6dQrPnj1Dbm5ug3sf7e3tUVhYiIKCAlRWVsLExAS/+c1vcODAAcTHx6OiogJ1dXW4d+8eHjx40OrcDBlbF+np6S22mQKAUaNGYd++ffjpp58wYcIEHD9+HOXl5VCpVPj555+xZ88e/Pa3v9Xce8nP2vg+ayKiHk1QtwId20wJIcTOnTuFt7e3kMlkQiaTidGjR4tdu3YJIYRQq9Xi888/FwMGDBBmZmbCzs5OzJkzR9y4cUOz/65du4RcLhcAxIABA0R+fr7YvXu3sLa2FgBE3759xc2bN1sVb/Xq1cLe3l7Y2tqK4OBgsXPnTgFAeHp6ijt37ohLly6Jvn37CgsLC+Hn5ycePnwonj9/LlavXi3c3d2FqampcHBwEHPnzhU5OTk65abv2Lr4xz/+IcaPHy9cXFwEAAFAODs7C19fX/H9999rxh0/flxYWVmJzZs3t2reO3fuiN///vfC29tbWFpaChMTE2FraytGjx4tfvvb34off/xRM5afded81nzqWHd8ip+awt+n7ksihGjfmopkVCQSCZKTk/Huu+8aOhUiakRwcDAAICUlxcCZdB0HDx7EvHnzwH+u6N/x96n74lf8RERERGRUWKASdYDr169DIpG0+AoJCTF0qkREREaHBSpRBxg8eHCrWqQkJSUZOlUi6kCnTp3CmjVroFarMWfOHLi7u0Mmk8HV1RWBgYHIzs7Wec6tW7di8ODBsLCwgEKhwODBg7F+/XpN/+GXqVQqxMTEwMvLC1KpFLa2thg+fDgKCgoAAF9//TW2bt3a5CIVRIbCApWIiKgDbNiwAdu3b8fatWuhVqtx5swZ7N+/H6WlpTh79iyUSiUmTpyIwsJCneY9c+YMFi9ejDt37qCoqAibNm3C1q1bG+0JOm/ePPzlL3/Bvn37UF1djX/961/w9PTE06dPAQCzZs2CTCbD1KlTNQt2EBkDFqhERD2MUqmEr69vl49hzD799FMkJSXh4MGDsLKyAgD4+PjAz88Pcrkc/fv3R3R0NMrLy/Hll1/qNLdUKsWKFSvg4OAAS0tLBAcHY/bs2fjb3/6m1fYsKSkJqampSElJwZtvvglTU1O4uLggLS1Na9ne8PBwjBw5EjNmzEBtba1ejp+ovVigEhH1MAkJCSguLu7yMYxVXl4e1q9fj40bN0ImkwEATE1NcfToUa1xHh4eAID8/Hyd5j9y5Ihm3nqurq4AoLkyCgB/+tOf8Nprr8Hb27vFOSMjI3H58mXExcXplAtRR2GBSkRk5IQQiI2NxZAhQ2Bubg47OzvMnj0b169f14wJCwuDVCqFs7OzZtuKFSugUCggkUg0y86uXLkSERERyM/Ph0QigZeXF7Zv3w6ZTAZHR0csXboULi4ukMlk8PX11VrEoD0xAODEiROtWmiiq9u+fTuEEJg1a1az4+pXebO2tm53zNzcXNja2qJv374AgJqaGmRmZmLUqFGt2t/Ozg6TJk1CXFwc23mRUWCBSkRk5CIjI7FmzRqsW7cOxcXF+OGHH3D37l1MmDABRUVFAF4URf/e/3jXrl3YuHGj1ra4uDjMnDkTnp6eEEIgLy8PYWFhCA0NRXV1NcLDw1FQUIBLly6htrYW06ZNw927d9sdA4DmQRy1Wq2/k2OEjh07hkGDBkEulzc77vz58wAAPz+/NsVRqVS4f/8+du7ciVOnTmHHjh2QSqUAgMLCQtTU1OCf//wnpkyZovmjY8iQIdi1a1ejRejo0aNx//59XLlypU35EOkTC1QiIiOmVCoRGxuLd955BwsXLoSNjQ28vb3xxRdf4PHjx9i9e7feYpmammqu0g4dOhTx8fGorKxEYmKiXuYPCAhARUUF1q9fr5f5jFFVVRV+/vlneHp6NjmmqKgISUlJCA8Ph4+PT4tXWpvSp08fuLm5ITIyEp999hnmzZunea/+q34HBwdER0cjJycHRUVFmD17Nn73u99h//79DeYbMGAAAODq1attyodIn1igEhEZsZycHDx9+hRjxozR2j527FhIpVKtr+D1bcyYMZDL5Vq3ElDziouLIYRo9uqpj48PwsPDMXv2bKSnp8PMzKxNse7evYvi4mLs378ff/7znzF69GjNfb/m5uYAgGHDhsHX1xf29vawsbHBxo0bYWNj0+gfNvU511+VJzIkFqhEREasvvWPpaVlg/dsbW1RWVnZofHNzc3x6NGjDo3RnTx79gzALwViYxwdHXH69Gns2LEDNjY2bY5lZmYGBwcH+Pv7IykpCTk5OYiJiQEAuLi4AIDmvuB6UqkUffv2bfTBLAsLC61jIDIkFqhEREbM1tYWABotRMvKyuDm5tZhsVUqVYfH6G7qi7zmGt87ODhoPld98fLygomJCXJycgC8+INmwIABuHbtWoOxtbW1jRbGNTU1AH45BiJDYoFKRGTEhg8fDktLS1y8eFFr+7lz51BTU4PXX39ds83U1BQqlUpvsTMyMiCEwLhx4zosRnfj6OgIiUSC8vLyJsccPXpU0xZKVyUlJViwYEGD7bm5uairq0OfPn002+bNm4esrCzcunVLs626uhq3b99utPVUfc5OTk5tyo1In1igEhEZMZlMhoiICBw5cgR79+5FRUUFrl69imXLlsHFxQVLlizRjPXy8kJpaSlSU1OhUqnw6NEj3L59u8Gc9vb2KCwsREFBASorKzUFp1qtxpMnT1BbW4vs7GysXLkS7u7uCA0N1UuM9PT0bt9mSi6Xw8PDA/fu3Wv0/by8PDg5OWk90FQvJCQETk5OuHTpUpPzKxQKnDx5EqdPn0ZFRQVUKhWysrLw/vvvQ6FQYNWqVZqxq1atQt++fREaGoo7d+6gpKQEq1evhlKpxEcffdRg7vqcW9M3laijsUAlIjJyGzZsQExMDKKiovDKK69g0qRJ6NevHzIyMqBQKDTjli9fjilTpmD+/PkYNGgQNm3apPm61sfHR9MuatmyZXB0dMTQoUMxY8YMlJaWAnhx76G3tzcsLCwwYcIEDBw4EN99953W/ZTtjdETBAQEICcnR9Pn9GXN9RitqalBcXEx0tLSmhwjk8kwfvx4LFq0CK6urrCyskJwcDD69euHzMxMrRWi7OzscObMGbi5uWHUqFFwdXXF+fPncezYsUb7o164cAGurq4YMWKEjkdMpH8SwY683YpEIkFycnKDXoVEZByCg4MBACkpKQbORNvSpUuRkpKCkpISQ6fSwMGDBzFv3rwu00A+Ly8PQ4YMQWJiIhYuXNjq/dRqNSZPnozQ0FB88MEHHZhhQyUlJXBzc8PmzZsRERHRqbHbw1h/n6j9eAWViIgANP9gD7Wel5cXoqKiEBUVpbX0aHPq6uqQmpqKyspKhISEdHCGDUVGRmLUqFEICwvr9NhEjWGBSkREpGdr1qxBcHAwQkJCmn1gql5GRgYOHz6M9PT0Fleg0rfY2FhcvnwZx48fb3NPViJ9Y4FKRNTDrV27FomJiSgvL0f//v1x6NAhQ6fULURHRyMsLAxbtmxpcezUqVOxb98+ODs7d0Jmv0hLS8Pz58+RkZEBOzu7To1N1BxTQydARESGFRMTo2nwTvrl7+8Pf39/Q6fRpMDAQAQGBho6DaIGeAWViIiIiIwKC1QiIiIiMiosUImIiIjIqLBAJSIiIiKjwkb93YxEIsG4cePg5uZm6FSIqBGZmZkAoLW+PTXv3r17yMzMRFBQkKFTISOTmZmJcePGsVF/N8QCtZupX1WDiDrPw4cPkZWVhenTpxs6FaIex8fHB6tWrTJ0GqRnLFCJiNqpqy3FSURk7HgPKhEREREZFRaoRERERGRUWKASERERkVFhgUpERERERoUFKhEREREZFRaoRERERGRUWKASERERkVFhgUpERERERoUFKhEREREZFRaoRERERGRUWKASERERkVFhgUpERERERoUFKhEREREZFRaoRERERGRUWKASERERkVFhgUpERERERoUFKhEREREZFRaoRERERGRUWKASERERkVFhgUpERERERoUFKhEREREZFRaoRERERGRUWKASERERkVFhgUpERERERoUFKhEREREZFRaoRERERGRUWKASERERkVFhgUpERERERoUFKhEREREZFRaoRERERGRUWKASERERkVFhgUpERERERsXU0AkQEXUlKpUKT58+1dpWVVUFAHjy5InWdolEAltb207LjYiou5AIIYShkyAi6iqKiorg6uqKurq6FsdOmTIFp0+f7oSsiIi6F37FT0SkAycnJ0ycOBG9ejX/v0+JRIL58+d3UlZERN0LC1QiIh39x3/8R4tjTExM8M4773RCNkRE3Q8LVCIiHc2dOxempk3fwm9iYoK3334bvXv37sSsiIi6DxaoREQ6sra2xvTp05ssUoUQWLhwYSdnRUTUfbBAJSJqg4ULFzb5oJRUKsWvf/3rTs6IiKj7YIFKRNQGv/71ryGXyxtsNzMzw5w5c6BQKAyQFRFR98AClYioDWQyGd555x2YmZlpbVepVHjvvfcMlBURUffAApWIqI0WLFgAlUqltc3a2hrTpk0zUEZERN0DC1Qiojb61a9+BXt7e81/m5mZYf78+ZBKpQbMioio62OBSkTURqamppg/f77ma36VSoUFCxYYOCsioq6PS50SEbXDjz/+CD8/PwAvVpkqLCxscZUpIiJqHv8vSkTUDr6+vnB1dQUA/Od//ieLUyIiPWh6KRSibujgwYOGToG6obFjx+L+/fvo3bs3f8ZI7/r06QMfHx9Dp0HUqfgVP/UoEonE0CkQEekkKCgIKSkphk6DqFPxCir1OMnJyXj33XcNnQZ1M4cOHUJwcDB/vnQUHBwMACzAmlB/foh6Gt4sRUSkB0FBQYZOgYio22CBSkRERERGhQUqERERERkVFqhEREREZFRYoBIRERGRUWGBSkRERERGhQUqEZGROX78OGxsbHD06FFDp2L0Tp06hTVr1kCtVmPOnDlwd3eHTCaDq6srAgMDkZ2drfOcW7duxeDBg2FhYQGFQoHBgwdj/fr1qKioaDBWpVIhJiYGXl5ekEqlsLW1xfDhw1FQUAAA+Prrr7F161bU1dW191CJehQWqERERobrp7TOhg0bsH37dqxduxZqtRpnzpzB/v37UVpairNnz0KpVGLixIkoLCzUad4zZ85g8eLFuHPnDoqKirBp0yZs3bq10VZi8+bNw1/+8hfs27cP1dXV+Ne//gVPT088ffoUADBr1izIZDJMnToVZWVlejluop6ABSoRkZEJCAhAeXk5Zs6caehUoFQq4evra+g0Gvj000+RlJSEgwcPwsrKCgDg4+MDPz8/yOVy9O/fH9HR0SgvL8eXX36p09xSqRQrVqyAg4MDLC0tERwcjNmzZ+Nvf/sbHjx4oBmXlJSE1NRUpKSk4M0334SpqSlcXFyQlpaG4cOHa8aFh4dj5MiRmDFjBmpra/Vy/ETdHQtUIiJqUkJCAoqLiw2dhpa8vDysX78eGzduhEwmAwCYmpo2uCXCw8MDAJCfn6/T/EeOHNHMW8/V1RUANFdGAeBPf/oTXnvtNXh7e7c4Z2RkJC5fvoy4uDidciHqqVigEhEZkbNnz8Ld3R0SiQQ7d+4EAMTHTknlBgAAFKhJREFUx0OhUEAulyMtLQ3Tp0+HtbU13NzccODAAc2+27dvh0wmg6OjI5YuXQoXFxfIZDL4+vri3LlzmnFhYWGQSqVwdnbWbFuxYgUUCgUkEgkeP34MAFi5ciUiIiKQn58PiUQCLy8vAMCJEydgbW2N6OjozjglDWzfvh1CCMyaNavZcUqlEgBgbW3d7pi5ubmwtbVF3759AQA1NTXIzMzEqFGjWrW/nZ0dJk2ahLi4ON7CQdQKLFCJiIyIn58f/v73v2ttW758OT788EMolUpYWVkhOTkZ+fn58PDwwOLFi6FSqQC8KDxDQ0NRXV2N8PBwFBQU4NKlS6itrcW0adNw9+5dAC8KvHfffVcrxq5du7Bx40atbXFxcZg5cyY8PT0hhEBeXh4AaB74UavVHXIOWnLs2DEMGjQIcrm82XHnz58H8OKctoVKpcL9+/exc+dOnDp1Cjt27IBUKgUAFBYWoqamBv/85z8xZcoUzR8DQ4YMwa5duxotQkePHo379+/jypUrbcqHqCdhgUpE1IX4+vrC2toaDg4OCAkJQVVVFe7cuaM1xtTUFEOGDIG5uTmGDh2K+Ph4VFZWIjExUS85BAQEoKKiAuvXr9fLfLqoqqrCzz//DE9PzybHFBUVISkpCeHh4fDx8WnxSmtT+vTpAzc3N0RGRuKzzz7DvHnzNO/Vf9Xv4OCA6Oho5OTkoKioCLNnz8bvfvc77N+/v8F8AwYMAABcvXq1TfkQ9SQsUImIuqj6q3n1V1CbMmbMGMjlcly/fr0z0upQxcXFEEI0e/XUx8cH4eHhmD17NtLT02FmZtamWHfv3kVxcTH279+PP//5zxg9erTmflxzc3MAwLBhw+Dr6wt7e3vY2Nhg48aNsLGxwe7duxvMV59zUVFRm/Ih6klYoBIR9QDm5uZ49OiRodNot2fPngH4pUBsjKOjI06fPo0dO3bAxsamzbHMzMzg4OAAf39/JCUlIScnBzExMQAAFxcXANDcr1tPKpWib9++jT6YZWFhoXUMRNQ0FqhERN2cSqVCWVkZ3NzcDJ1Ku9UXec01vndwcICtra1e43p5ecHExAQ5OTkAAEtLSwwYMADXrl1rMLa2trbRwrimpgbAL8dARE1jgUpE1M1lZGRACIFx48ZptpmamrZ4a4AxcnR0hEQiQXl5eZNjjh49qmkLpauSkhIsWLCgwfbc3FzU1dWhT58+mm3z5s1DVlYWbt26pdlWXV2N27dvN9p6qj5nJyenNuVG1JOwQCUi6mbUajWePHmC2tpaZGdnY+XKlXB3d0doaKhmjJeXF0pLS5GamgqVSoVHjx7h9u3bDeayt7dHYWEhCgoKUFlZCZVKhfT0dIO1mZLL5fDw8MC9e/cafT8vLw9OTk5aDzTVCwkJgZOTEy5dutTk/AqFAidPnsTp06dRUVEBlUqFrKwsvP/++1AoFFi1apVm7KpVq9C3b1+Ehobizp07KCkpwerVq6FUKvHRRx81mLs+59b0TSXq6VigEhEZkZ07d2Ls2LEAgNWrVyMwMBDx8fHYtm0bAGDEiBG4desW9uzZg4iICADA22+/jdzcXM0cz549g7e3NywsLDBhwgQMHDgQ3333ndZ9m8uXL8eUKVMwf/58DBo0CJs2bdJ89ezj46NpSbVs2TI4Ojpi6NChmDFjBkpLSzvlPDQnICAAOTk5mj6nL2uux2hNTQ2Ki4uRlpbW5BiZTIbx48dj0aJFcHV1hZWVFYKDg9GvXz9kZmZqrRBlZ2eHM2fOwM3NDaNGjYKrqyvOnz+PY8eONdof9cKFC3B1dcWIESN0PGKinkci2DGYehCJRILk5OQGPSCJ9MEYfr6WLl2KlJQUlJSUGCwHXQQHBwMAUlJSWr1PXl4ehgwZgsTERCxcuLDV+6nVakyePBmhoaH44IMPdM61PUpKSuDm5obNmzdr/rBojbacH6LugFdQiYi6meYeIOoOvLy8EBUVhaioKK2lR5tTV1eH1NRUVFZWIiQkpIMzbCgyMhKjRo1CWFhYp8cm6opYoBLpYNGiRbCysoJEIsHly5cNnU6bbN26FYMHD4aFhQUUCgUGDx6M9evXo6Kiotn9nj17hsGDB+Pjjz/WOebhw4fh4eEBiUSi9ZJKpXB0dMTkyZPx+eef48mTJ209LOph1qxZg+DgYISEhDT7wFS9jIwMHD58GOnp6S2uQKVvsbGxuHz5Mo4fP97mnqxEPQ0LVCId/N///R/27Nlj6DTa5cyZM1i8eDHu3LmDoqIibNq0CVu3bkVQUFCz+61btw43btxoU8y5c+fi1q1b8PT0hI2NDYQQUKvVKC4uxsGDB9G/f3+sXr0aw4YNw8WLF9sUg4C1a9ciMTER5eXl6N+/Pw4dOmTolDpUdHQ0wsLCsGXLlhbHTp06Ffv27YOzs3MnZPaLtLQ0PH/+HBkZGbCzs+vU2ERdmamhEyCiziWVSrFixQrIZDIAL+5xS0lJQUpKCh48eKBpQP6yv//97/jpp5/0modEIoGtrS0mT56MyZMnIyAgAPPmzUNAQABu3rzZrgbrPVVMTIymkXxP4e/vD39/f0On0aTAwEAEBgYaOg2iLodXUIl0JJFIDJ1Cuxw5ckRTnNar7xnZ2P18SqUSf/jDHxAXF9eheQUFBSE0NBTFxcX44osvOjQWEREZNxaoRM0QQuDzzz/HoEGDYG5uDhsbG/zhD39oMK6urg6ffPIJ3N3dYWFhgREjRiA5ORkAEB8fD4VCAblcjrS0NEyfPh3W1tZwc3PDgQMHtOb5/vvv8cYbb0Aul8Pa2hre3t6ae0Obi9Feubm5sLW1Rd++fRu8t27dOqxYsQIODg6N7nvixAm99cSs79OZnp6u2dbVzy0REemOBSpRM9avX4/Vq1djyZIlKCoqwsOHDxttwP3RRx/hs88+w7Zt2/DgwQPMnDkTCxYswMWLF7F8+XJ8+OGHUCqVsLKyQnJyMvLz8+Hh4YHFixdrVvOpqqrCrFmzEBQUhNLSUuTm5mLgwIGa5RGbi9EWKpUK9+/fx86dO3Hq1Cns2LEDUqlUa8yPP/6I/Pz8RlfWqVf/xLharW5THi+r7x358so8XfHcEhFROwmiHgSASE5ObtXY6upqIZfLxbRp07S2HzhwQAAQWVlZQgghlEqlkMvlIiQkRGtfc3NzsXz5ciGEEOvWrRMAhFKp1IzZtWuXACDy8vKEEEL89NNPAoD461//2iCX1sTQlZOTkwAgevfuLf7nf/5H1NTUNDj+MWPGiHv37gkhhHj06JEAINatW9emeEII4enpKWxsbJodI5FIhK2trRCi651bXX6+6IWgoCARFBRk6DSMFs8P9VR8SIqoCXl5eaiursbUqVObHXfjxg1UV1drrTBjYWEBZ2dnXL9+vcn96q9W1l/l8/DwgKOjIxYuXIjw8HCEhoaiX79+7YrRnLt376KsrAxZWVlYs2YNdu/ejdOnT8PR0RHAiyfC/+u//qvNa5q3RVVVFYQQsLa2BtA1z+22bdvYVF0HmZmZAH5pSE/aMjMzMW7cOEOnQdTp+BU/URPq181u6t7LelVVVQCAjz/+WKvH5+3bt1FdXd3qeBYWFjh9+jT8/PwQHR0NDw8PhISEQKlU6i3Gy8zMzODg4AB/f38kJSUhJydH8wT42bNncfXqVSxatKhNc7fVzZs3AQCDBw8G0HXPLRERtQ+voBI1of5J9+fPnzc7rr6A3bZtG1auXNmumMOGDcPRo0fx6NEjxMbG4tNPP8WwYcM0K9/oI0ZjvLy8YGJigpycHABAQkICvv32W/Tq1fBv2OjoaERHR+PChQsYM2aMXvM4ceIEAGD69OkAuua5/fDDD7mUrg64lGfzeGWZeipeQSVqwvDhw9GrVy98//33zY7r06cPZDJZu1eWKiwsxLVr1wC8KMy2bNmC1157DdeuXdNbjJKSkkYfeMrNzUVdXR369OkDAEhMTIQQQuv16NEjAC+e6hdC6L04ffjwIbZt2wY3NzfNOuld6dwSEZH+sEAlaoKDgwPmzp2LQ4cOISEhARUVFcjOzsbu3bu1xslkMvzmN7/BgQMHEB8fj4qKCtTV1eHevXt48OBBq+MVFhZi6dKluH79OmpqapCVlYXbt29j3LhxeouhUChw8uRJnD59GhUVFVCpVMjKysL7778PhUKBVatWtXqueunp6Tq1mRJC4OnTp1Cr1ZrCNzk5GePHj4eJiQlSU1M196B2pXNLRER6ZLjns4g6H3R8yrqyslIsWrRI9O7dW1haWgo/Pz/xySefCADCzc1NXLlyRQghxPPnz8Xq1auFu7u7MDU1FQ4ODmLu3LkiJydH7Nq1S8jlcgFADBgwQOTn54vdu3cLa2trAUD07dtX3Lx5UxQUFAhfX19hZ2cnTExMxKuvvirWrVsnamtrW4yhi1mzZon+/fsLS0tLYW5uLjw9PUVISIi4evVqs/s19RT/8ePHhZWVldi8eXOT+3799ddixIgRQi6XC6lUKnr16iUAaJ7Yf+ONN0RUVJQoKSlpsG9XOre6/nwRn1JvCc8P9VQSIYQwWHVM1MkkEgmSk5N5jyB1CP586Y73oDaP54d6Kn7FT0RERERGhQUqURd3/fp1rfZITb3qn1Yn6s5OnTqFNWvWQK1WY86cOXB3d4dMJoOrqysCAwORnZ3d5rnVajW2bdsGX1/fBu99/fXX2Lp1q2ZlNSJqHxaoRF3c4MGDGzxx39grKSnJ0KkSdagNGzZg+/btWLt2LdRqNc6cOYP9+/ejtLQUZ8+ehVKpxMSJE1FYWKjz3Lm5uZg4cSJWrVrVaH/cWbNmQSaTYerUqSgrK9PH4RD1aCxQiYi6CaVS2ejVva4Woy0+/fRTJCUl4eDBg7CysgIA+Pj4wM/PD3K5HP3790d0dDTKy8vx5Zdf6jT3lStX8NFHH2HZsmUYNWpUk+PCw8MxcuRIzJgxA7W1te05HKIejwUqEVE3kZCQgOLi4i4fQ1d5eXlYv349Nm7cqFlgw9TUFEePHtUa5+HhAQDIz8/Xaf6RI0fi8OHDeO+992Bubt7s2MjISFy+fBlxcXE6xSAibSxQiYgMRAiB2NhYDBkyBObm5rCzs8Ps2bNx/fp1zZiwsDBIpVI4Oztrtq1YsQIKhQISiQSPHz8GAKxcuRIRERHIz8+HRCKBl5cXtm/fDplMBkdHRyxduhQuLi6QyWTw9fXFuXPn9BIDeLECmC69cPVt+/btEEJg1qxZzY5TKpUAoOmz2xHs7OwwadIkxMXFgU1yiNqOBSoRkYFERkZizZo1WLduHYqLi/HDDz/g7t27mDBhAoqKigC8KL7+vW3Vrl27sHHjRq1tcXFxmDlzJjw9PSGEQF5eHsLCwhAaGorq6mqEh4ejoKAAly5dQm1tLaZNm4a7d++2OwYAzYNBarVafydHB8eOHcOgQYMgl8ubHXf+/HkAgJ+fX4fmM3r0aNy/fx9Xrlzp0DhE3RkLVCIiA1AqlYiNjcU777yDhQsXwsbGBt7e3vjiiy/w+PHjBiuWtYepqanmKu3QoUMRHx+PyspKJCYm6mX+gIAAVFRUYP369XqZTxdVVVX4+eef4enp2eSYoqIiJCUlITw8HD4+Pi1eaW2vAQMGAACuXr3aoXGIujNTQydARNQT5eTk4OnTpxgzZozW9rFjx0IqlWp9Ba9vY8aMgVwu17qVoKsqLi6GEKLZq6c+Pj6oqqrCu+++i82bN8PMzKxDc6rPpf4qOBHpjgUqEZEB1LcisrS0bPCera0tKisrOzS+ubk5Hj161KExOsOzZ88AoNmHlxwdHZGQkIBhw4Z1Sk4WFhZauRGR7vgVPxGRAdja2gJAo4VoWVkZ3NzcOiy2SqXq8Bidpb4YbK5BvoODg+Z8d4aamhoAv+RGRLrjFVQiIgMYPnw4LC0tcfHiRa3t586dQ01NDV5//XXNNlNTU6hUKr3FzsjIgBAC48aN67AYncXR0RESiQTl5eVNjvn3dlMdrT4XJyenTo1L1J3wCioRkQHIZDJERETgyJEj2Lt3LyoqKnD16lUsW7YMLi4uWLJkiWasl5cXSktLkZqaCpVKhUePHuH27dsN5rS3t0dhYSEKCgpQWVmpKTjVajWePHmC2tpaZGdnY+XKlXB3d0doaKheYqSnpxuszZRcLoeHhwfu3bvX6Pt5eXlwcnLCvHnzGrwXEhICJycnXLp0Sa851efi7e2t13mJehIWqEREBrJhwwbExMQgKioKr7zyCiZNmoR+/fohIyMDCoVCM2758uWYMmUK5s+fj0GDBmHTpk2ar499fHw07aKWLVsGR0dHDB06FDNmzEBpaSmAF/dCent7w8LCAhMmTMDAgQPx3Xffad232d4YhhQQEICcnBxNn9OXNdeLtKamBsXFxUhLS2t2/szMTPj5+eHVV1/FuXPncOXKFbi4uGD8+PH44YcfGoy/cOECXF1dMWLECN0PhogAABLBTsLUg0gkEiQnJzfo+UikD8b487V06VKkpKSgpKTE0Kk0Kjg4GACQkpLS5jny8vIwZMgQJCYmYuHCha3eT61WY/LkyQgNDcUHH3zQ5vgvKykpgZubGzZv3oyIiIh2z6eP80PUFfEKKhFRN9fcA0TdgZeXF6KiohAVFYWnT5+2ap+6ujqkpqaisrISISEhesslMjISo0aNQlhYmN7mJOqJWKASEVGXt2bNGgQHByMkJKTZB6bqZWRk4PDhw0hPT29xBarWio2NxeXLl3H8+PEO77VK1N2xQCUi6qbWrl2LxMRElJeXo3///jh06JChU+pQ0dHRCAsLw5YtW1ocO3XqVOzbtw/Ozs56iZ2Wlobnz58jIyMDdnZ2epmTqCdjmykiom4qJiYGMTExhk6jU/n7+8Pf37/T4wYGBiIwMLDT4xJ1V7yCSkRERERGhQUqERERERkVFqhEREREZFRYoBIRERGRUWGBSkRERERGhStJUY8ikUgMnQIRkU6CgoK4khT1OGwzRT1KcnKyoVMgItJJnz59DJ0CUafjFVQiIiIiMiq8B5WIiIiIjAoLVCIiIiIyKixQiYiIiMiomALgo4FEREREZDT+H2wL5Q9tpmmOAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 134 } ] }, { "cell_type": "code", "metadata": { "id": "qY8bJyIWX6rP" }, "source": [ "X_train_scaled_A = X_train_scaled[:, :4]\n", "X_train_scaled_B = X_train_scaled[:, 4:]\n", "\n", "X_test_scaled_A = X_test_scaled[:, :4]\n", "X_test_scaled_B = X_test_scaled[:, 4:]" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "BdnWjz2lat-V", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "3509b1dc-ffb2-48ba-8556-3add7607c675" }, "source": [ "X_train_scaled_A.shape, X_train_scaled_B.shape" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "((14448, 4), (14448, 4))" ] }, "metadata": { "tags": [] }, "execution_count": 137 } ] }, { "cell_type": "code", "metadata": { "id": "NuiFPAVWYGv8", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "15172969-3f5c-48fa-c742-705a2f3ace74" }, "source": [ "history = model.fit([X_train_scaled_A, \n", " X_train_scaled_B], y_train, \n", " epochs=30,\n", " ba\n", " validation_split=0.1)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Epoch 1/30\n", "407/407 [==============================] - 1s 3ms/step - loss: 2.0971 - val_loss: 0.8004\n", "Epoch 2/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.7077 - val_loss: 0.6807\n", "Epoch 3/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.6300 - val_loss: 0.6332\n", "Epoch 4/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5937 - val_loss: 0.6050\n", "Epoch 5/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5704 - val_loss: 0.5854\n", "Epoch 6/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5521 - val_loss: 0.5691\n", "Epoch 7/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5373 - val_loss: 0.5565\n", "Epoch 8/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5250 - val_loss: 0.5458\n", "Epoch 9/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5148 - val_loss: 0.5365\n", "Epoch 10/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.5058 - val_loss: 0.5301\n", "Epoch 11/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4983 - val_loss: 0.5222\n", "Epoch 12/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4914 - val_loss: 0.5174\n", "Epoch 13/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4859 - val_loss: 0.5115\n", "Epoch 14/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4807 - val_loss: 0.5073\n", "Epoch 15/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4760 - val_loss: 0.5031\n", "Epoch 16/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4714 - val_loss: 0.4992\n", "Epoch 17/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4675 - val_loss: 0.4956\n", "Epoch 18/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4637 - val_loss: 0.4922\n", "Epoch 19/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4603 - val_loss: 0.4893\n", "Epoch 20/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4571 - val_loss: 0.4850\n", "Epoch 21/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4545 - val_loss: 0.4825\n", "Epoch 22/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4519 - val_loss: 0.4799\n", "Epoch 23/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4494 - val_loss: 0.4783\n", "Epoch 24/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4474 - val_loss: 0.4751\n", "Epoch 25/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4452 - val_loss: 0.4735\n", "Epoch 26/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4433 - val_loss: 0.4716\n", "Epoch 27/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4413 - val_loss: 0.4699\n", "Epoch 28/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4396 - val_loss: 0.4677\n", "Epoch 29/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4377 - val_loss: 0.4655\n", "Epoch 30/30\n", "407/407 [==============================] - 1s 2ms/step - loss: 0.4361 - val_loss: 0.4640\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "CoSFroUvYLPW", "colab": { "base_uri": "https://localhost:8080/", "height": 283 }, "outputId": "87918155-db6c-4af7-fea9-3eee7ff5b83c" }, "source": [ "history_df = pd.DataFrame(history.history)\n", "history_df[['loss', 'val_loss']].plot(kind='line')" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 139 }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZRcdZ338fe3lu6q3rLvCQQUDJgI+ISojxCQMyJuMOooICowo8yggtswMo4OiDrjkedRZ9QHDjIRVEAyiDOZAUXOyBhQcbKYEAIYQiChk5B0urP0ku7avs8f93Z39d7prqTSdT+vc+rc/dbvdiWf363fvfW75u6IiEjli5W7ACIicmwo8EVEIkKBLyISEQp8EZGIUOCLiEREotwFGMz06dN94cKF5S6GiMiEsW7dun3uPmO4dY7LwF+4cCFr164tdzFERCYMM9s+0jpq0hERiQgFvohIRCjwRUQi4rhswxeR6MlmszQ2NtLZ2VnuohzXUqkU8+fPJ5lMHvG2CnwROS40NjZSX1/PwoULMbNyF+e45O40NzfT2NjISSeddMTbq0lHRI4LnZ2dTJs2TWE/DDNj2rRpY/4WpMAXkeOGwn5k4/kbVUzguzv//F/P8+stTeUuiojIcaliAt/MuGP1Nv77j3vLXRQRmaDq6urKXYSjqmICH6AhleDQ4Vy5iyEiclyqrMBPJznUmS13MURkgnN3brjhBhYvXsySJUu4//77Adi9ezfLly/nzDPPZPHixTz++OPk83muuuqqnnW/9a1vlbn0Q6uo2zIb0kkOHVbgi0x0X/6PzTyz61BJ93n63AZuevdrR7Xugw8+yIYNG9i4cSP79u3j7LPPZvny5dx777287W1v4+/+7u/I5/N0dHSwYcMGdu7cydNPPw3AgQMHSlruUqqsM/xUkoMKfBEZpyeeeILLL7+ceDzOrFmzOO+881izZg1nn302P/jBD7j55pvZtGkT9fX1nHzyyWzbto3rrruOX/ziFzQ0NJS7+EOqsDP8BK271YYvMtGN9kz8WFu+fDmrV6/moYce4qqrruKzn/0sH/nIR9i4cSOPPPIIt99+OytXrmTFihXlLuqgKu4MX006IjJe5557Lvfffz/5fJ6mpiZWr17NsmXL2L59O7NmzeJjH/sYH/3oR1m/fj379u2jUCjwvve9j69+9ausX7++3MUfUkWd4U9KJ2ntypEvOPGYfsAhImPznve8h9/97necccYZmBnf+MY3mD17NnfffTe33noryWSSuro6fvjDH7Jz506uvvpqCoUCAP/4j/9Y5tIPraICvyEddCbU2pllck1VmUsjIhNNW1sbEPyu59Zbb+XWW2/ts/zKK6/kyiuvHLDd8XxWX2zEJh0zW2Bmj5nZM2a22cw+Ncg6Zmb/bGZbzewpM3t90bIrzez58DXwL1VCDamg/tK9+CIiA43mDD8HfM7d15tZPbDOzB5192eK1nk7cEr4egNwG/AGM5sK3AQsBTzcdpW77y/pUYS6z/B1L76IyEAjnuG7+253Xx+OtwLPAvP6rXYJ8EMPPAlMNrM5wNuAR929JQz5R4GLSnoERSZ1B74u3IqIDHBEd+mY2ULgLOD3/RbNA14umm4M5w01f7B9X2Nma81sbVPT2DpAa0gFga978UVEBhp14JtZHfBT4NPuXtqfwAHufoe7L3X3pTNmzBjTPhrSYRu+mnRERAYYVeCbWZIg7O9x9wcHWWUnsKBoen44b6j5R0VPG74u2oqIDDCau3QM+BfgWXf/5hCrrQI+Et6t80bgoLvvBh4BLjSzKWY2BbgwnHdU1FUliJnO8EVEBjOaM/w3Ax8GLjCzDeHrHWb2V2b2V+E6DwPbgK3A94GPA7h7C/AVYE34uiWcd1TEYka9+tMRkWNguL7zX3rpJRYvXnwMSzM6I96W6e5PAMP+bNXdHfjEEMtWAMesY4mGdEJ36YiIDKKifmkLYX86nWrDF5nQfn4jvLKptPucvQTe/vUhF994440sWLCAT3wiOHe9+eabSSQSPPbYY+zfv59sNstXv/pVLrnkkiN6287OTq699lrWrl1LIpHgm9/8Jm95y1vYvHkzV199NZlMhkKhwE9/+lPmzp3LBz7wARobG8nn83zpS1/i0ksvHddhF6u4wJ+kPvFFZAwuvfRSPv3pT/cE/sqVK3nkkUe4/vrraWhoYN++fbzxjW/k4osvPqIHiX/ve9/DzNi0aRPPPfccF154IVu2bOH222/nU5/6FFdccQWZTIZ8Ps/DDz/M3LlzeeihhwA4ePBgSY+x4gK/IZXkhaa2chdDRMZjmDPxo+Wss85i79697Nq1i6amJqZMmcLs2bP5zGc+w+rVq4nFYuzcuZM9e/Ywe/bsUe/3iSee4LrrrgNg0aJFnHjiiWzZsoU3velNfO1rX6OxsZH3vve9nHLKKSxZsoTPfe5zfP7zn+dd73oX5557bkmPsaK6R4awDV936YjIGLz//e/ngQce4P777+fSSy/lnnvuoampiXXr1rFhwwZmzZpFZ2dnSd7rgx/8IKtWrSKdTvOOd7yDX/3qV5x66qmsX7+eJUuW8MUvfpFbbrmlJO/VrSLP8HUfvoiMxaWXXsrHPvYx9u3bx69//WtWrlzJzJkzSSaTPPbYY2zfvv2I93nuuedyzz33cMEFF7BlyxZ27NjBa17zGrZt28bJJ5/M9ddfz44dO3jqqadYtGgRU6dO5UMf+hCTJ0/mzjvvLOnxVVzgT0onOZzNk8kVqEpU3BcYETmKXvva19La2sq8efOYM2cOV1xxBe9+97tZsmQJS5cuZdGiRUe8z49//ONce+21LFmyhEQiwV133UV1dTUrV67kRz/6EclkktmzZ/OFL3yBNWvWcMMNNxCLxUgmk9x2220lPT4L7qg8vixdutTXrl07pm3v/u1L3LRqM2u/+CdMr6succlE5Gh59tlnOe2008pdjAlhsL+Vma1z96XDbVdxp8A9/enoTh0RkT4qrkmnu8dM3YsvIkfbpk2b+PCHP9xnXnV1Nb//ff8OhY8PFRf46hNfZOJy9yO6x73clixZwoYNG47pe46nGb4Cm3TUJ77IRJRKpWhubh5XoFU6d6e5uZlUKjWm7SvuDL+3SUeBLzKRzJ8/n8bGRsb6AKSoSKVSzJ8/f0zbVl7gp/Ugc5GJKJlMctJJJ5W7GBWt4pp00sk4ybjpDF9EpJ+KC3wzC39tq8AXESlWcYEPwYVbXbQVEemrMgM/ldB9+CIi/VRm4KtPfBGRASo38HXRVkSkj8oMfF20FREZYMT78M1sBfAuYK+7D3gMu5ndAFxRtL/TgBnu3mJmLwGtQB7IjdSTW6kEDzLPTbifaYuIHE2jOcO/C7hoqIXufqu7n+nuZwJ/C/za3VuKVnlLuPyYhD0E/elk8gW6coVj9ZYiIse9EQPf3VcDLSOtF7ocuG9cJSqBnu4V1KwjItKjZG34ZlZD8E3gp0WzHfilma0zs2tK9V4jUQdqIiIDlbIvnXcDv+nXnHOOu+80s5nAo2b2XPiNYYCwQrgG4IQTThhXQRpSYX86ulNHRKRHKe/SuYx+zTnuvjMc7gV+BiwbamN3v8Pdl7r70hkzZoyrIL194uvHVyIi3UoS+GY2CTgP+PeiebVmVt89DlwIPF2K9xtJd5OOzvBFRHqN5rbM+4Dzgelm1gjcBCQB3P32cLX3AL909/aiTWcBPwtvi0wA97r7L0pX9KF1X7RVG76ISK8RA9/dLx/FOncR3L5ZPG8bcMZYCzYeepC5iMhAFflL2+pEnFQypg7URESKVGTgg7pXEBHpr3IDX33ii4j0UbmBn0roLh0RkSIVG/iT0kndhy8iUqRiA1994ouI9FW5gZ9SG76ISLHKDfx0gkOHs7h7uYsiInJcqNjAn5ROUnBoz+TLXRQRkeNCxQa++sQXEemrcgNffeKLiPRRuYGvM3wRkT4qNvB7+sRXfzoiIkAFB756zBQR6atyA1994ouI9FGxgV+v59qKiPRRsYGfiMeoq06oPx0RkVDFBj6ox0wRkWKVHfjqE19EpEdlB76eeiUi0qOyAz+d1H34IiKhEQPfzFaY2V4ze3qI5eeb2UEz2xC+/r5o2UVm9kcz22pmN5ay4KPR3WOmiIiM7gz/LuCiEdZ53N3PDF+3AJhZHPge8HbgdOByMzt9PIU9Ug0pPQRFRKTbiIHv7quBljHsexmw1d23uXsG+AlwyRj2M2YN6SStnTnyBfWJLyJSqjb8N5nZRjP7uZm9Npw3D3i5aJ3GcN6gzOwaM1trZmubmppKUqju/nTa1I4vIlKSwF8PnOjuZwDfAf5tLDtx9zvcfam7L50xY0YJihXchw/6ta2ICJQg8N39kLu3heMPA0kzmw7sBBYUrTo/nHfMqE98EZFe4w58M5ttZhaOLwv32QysAU4xs5PMrAq4DFg13vc7EuoTX0SkV2KkFczsPuB8YLqZNQI3AUkAd78d+DPgWjPLAYeByzx4cnjOzD4JPALEgRXuvvmoHMUQevvEV+CLiIwY+O5++QjLvwt8d4hlDwMPj61o49fbJ74u2oqIVPwvbUFn+CIiUOGBX1eVwEwXbUVEoMIDPxYzdaAmIhKq6MCHsD8d/fBKRCQCga8zfBERICKBrzZ8EZEIBP6ktHrMFBGBCAR+0Ce+2vBFRCo/8NUnvogIEIXATyfpyOTJ5gvlLoqISFlVfOD39KejC7ciEnEVH/g9/enoXnwRibjKD3x1kSwiAkQh8PUQFBERIAKBrz7xRUQCFR/4vU06asMXkWir/MBP60HmIiIQgcBPJ+MkYqY2fBGJvIoPfDML+tNR4ItIxFV84ENwp47uwxeRqItG4KcSOsMXkcgbMfDNbIWZ7TWzp4dYfoWZPWVmm8zst2Z2RtGyl8L5G8xsbSkLfiQa0uoTX0RkNGf4dwEXDbP8ReA8d18CfAW4o9/yt7j7me6+dGxFHL8G9YkvIkJipBXcfbWZLRxm+W+LJp8E5o+/WKUVPOZQbfgiEm2lbsP/C+DnRdMO/NLM1pnZNcNtaGbXmNlaM1vb1NRU0kIFDzLXGb6IRNuIZ/ijZWZvIQj8c4pmn+PuO81sJvComT3n7qsH297d7yBsDlq6dKmXqlwQnOFncgU6s3lSyXgpdy0iMmGU5AzfzF4H3Alc4u7N3fPdfWc43Av8DFhWivc7UuoTX0SkBIFvZicADwIfdvctRfNrzay+exy4EBj0Tp+jrUEdqImIjNykY2b3AecD082sEbgJSAK4++3A3wPTgP9nZgC58I6cWcDPwnkJ4F53/8VROIYRNaSCwzyoC7ciEmGjuUvn8hGWfxT46CDztwFnDNzi2GtQk46ISDR+aas+8UVEIhL4esyhiEhEAr8+pQeZi4hEIvBTyTjViZjO8EUk0iIR+BC046sDNRGJssgEvjpQE5Goi07gpxLqQE1EIi06ga8zfBGJuMgEvtrwRSTqIhP4QZ/4CnwRia7oBH46waHOHO4l7XlZRGTCiE7gp5LkC05HJl/uooiIlEVkAr+7Px2144tIVEUm8NUnvohEXXQCv6cDNd2LLyLRFJ3AT4cdqKlJR0QiKjKBrzZ8EYm6yAR+T5OO2vBFJKIiE/g9feKrDV9EIioygZ+Ix6itiusMX0QiKzKBD+pPR0SibVSBb2YrzGyvmT09xHIzs382s61m9pSZvb5o2ZVm9nz4urJUBR+LhrT60xGR6BrtGf5dwEXDLH87cEr4uga4DcDMpgI3AW8AlgE3mdmUsRZ2vBpS6iJZRKJrVIHv7quBlmFWuQT4oQeeBCab2RzgbcCj7t7i7vuBRxm+4jiqGtJ6CIqIRFep2vDnAS8XTTeG84aaP4CZXWNma81sbVNTU4mK1VeD2vBFJMKOm4u27n6Huy9196UzZsw4Ku+hJh0RibJSBf5OYEHR9Pxw3lDzy6IhnaStK0ehoD7xRSR6ShX4q4CPhHfrvBE46O67gUeAC81sSnix9sJwXlk0pBK4Q2uX2vFFJHoSo1nJzO4Dzgemm1kjwZ03SQB3vx14GHgHsBXoAK4Ol7WY2VeANeGubnH34S7+HlXd/ekcOpztGRcRiYpRBb67Xz7Ccgc+McSyFcCKIy9a6TUUdaC2YIR1RUQqzXFz0fZYUAdqIhJl0Qr8tDpQE5HoilTgF7fhi4hETaQCX8+1FZEoi1Tg11UlMNMZvohEU6QCPxYz6qsTHOpUG76IRE+kAh9gUo360xGRaIpc4Dek1Ce+iERTNANfF21FJIKiF/jqE19EIipygT8prTN8EYmmyAV+Q0oXbUUkmqIX+OkkHZk82Xyh3EURETmmohf4qaA/nVbdiy8iERO5wJ9Uo/50RCSaIhf43V0kqx1fRKKmsgL/lU3QMfwDtdSBmohEVeUEfkcLrLgIVl0HPvRDynsegqJ78UUkYion8Gumwvk3wnP/CWvuHHK1STrDF5GIqpzAB3jjJ+CUC+GRL8DupwZdpfupV2rDF5GoGVXgm9lFZvZHM9tqZjcOsvxbZrYhfG0xswNFy/JFy1aVsvADxGLwp7dBzTR44GroahuwSjoZJxEz3aUjIpEzYuCbWRz4HvB24HTgcjM7vXgdd/+Mu5/p7mcC3wEeLFp8uHuZu19cwrIPrnY6vPf70LINHv7rAYvNjAZ1ryAiETSaM/xlwFZ33+buGeAnwCXDrH85cF8pCjdmJ50Ly/8GNt4HGwYWZVI6qYu2IhI5own8ecDLRdON4bwBzOxE4CTgV0WzU2a21syeNLM/HXNJj9TyG+DEN8NDn4N9z/dZ1JBKqA1fRCKn1BdtLwMecPd80bwT3X0p8EHg22b2qsE2NLNrwophbVNT0/hLEk8ETTuJavjXqyHb2bNITToiEkWjCfydwIKi6fnhvMFcRr/mHHffGQ63Af8NnDXYhu5+h7svdfelM2bMGEWxRmHSvOAi7p5N8OiXembrqVciEkWjCfw1wClmdpKZVRGE+oC7bcxsETAF+F3RvClmVh2OTwfeDDxTioKP2msuCm7X/J874Nn/ALrP8NWGLyLRMmLgu3sO+CTwCPAssNLdN5vZLWZWfNfNZcBP3Pv8zPU0YK2ZbQQeA77u7sc28AH+5CaYcyb8+yfgwA4a0mrDF5HoSYxmJXd/GHi437y/7zd98yDb/RZYMo7ylUaiGt7/A7h9OTzwF8w85btkcgVWbdzFxWfMLXfpRESOicr6pe1wpp4M7/42NP4PH+q8l2UnTeX6+/7AHatfwIfpe0dEpFJEJ/ABlvwZnPVhqn/3T/z4/Hbe+bo5/MPDz3Hzqs3kCwp9Eals0Qp8gLd/A6afStW/fZTvnLKRa85ZwN2/2861P17H4Ux+5O1FRCao6AV+VQ1cdi9MP5XYQ5/mCy/+OSuW7ebRZ1/hg3c+SUt7ptwlFBE5KqIX+ADTXw1//kgQ/Bbjgqc+xx/mfIOaXb/nfbf9lu3N7eUuoYhIyUUz8AHMYNE74drfwsXfYXK2iXsSX+bLbV/mb753HxtePjDyPkREJpDoBn63eAJe/xG4fj38yZd5c/UL3Ff4a178/od5fM0fyl06EZGSUeB3S6bhnE8T/9QGOs/+OO+M/Y5l//lWNt91HRwcqicJEZGJQ4HfX81Uat75D+Q/uY419Rew6MUfwbdO59B3zqPw+Leh+YVyl1BEZEzsePzR0dKlS33t2rXlLga5fIEfPvQY7X94gOX5Jzkjtg2A/PTTiL/2Yjjt3TBrcXA9QESkjMxsXdgz8dDrKPBH1pnN8/Ond/PzJ9Yw75X/4h2Jtfwv+yMxCjBlYRD8p10M85YGj1kUETnGFPhHweZdB/nxkzt4fMMzvDm/hvfX/IGzchuIew5qpsOCN8CCZcFr7lnBtQERkaNMgX8UHerM8rP1O/nxk9t5Ze9e3pl6isumbOHU7LPUtG0PVoolYPYSmB9WAPPPhsknqAlIREpOgX8MuDu/f7GFHz+5nV8+s4dMrsDMWCvvnbmLC+q2syj3HPXNT2HZjmCDullB8M89E2a/LrgG0DBXlYCIjIsC/xjrzOZZt30/T2zdx2+27mPTzoO4Q0MVvHfuAd7asIPF/kca9m3A9r/Yu2F6Ksxe3FsBzF4M018DiaryHYyITCgK/DI70JHhyW3N/GZrM7/Zuo9t+4IuG6bXVfHm+UnOm9TE65I7OCGzjap9m2Hvs5ALn70bS8KMRTDrdJj26qB756knwdRXQXpyGY9KRI5HCvzjzK4Dh/nN1n389oVm/rBjPy81d/QsWzithjPn1XHO1IOcVdXICdkXSDY9E1QCh/r98Cs9Faa9KqwETg4qgaknw5QToWaamodEIkiBf5w72JHlqZ0HeKrxIBtfDoavHArO8GMGp86qZ/G8SZw+Pcni2v28OraHKV2NWMsL0LINWl6Eg41A0WeYrIFJ84OLw5MWwOQFMOmEYDj5BKibrVtHRSqQAn8C2nuok42NB3mqMagANu86yL623i6b66sTvHpWHafOrOeUWXUsmpbkNakWpne9jB18GQ68DAd3hMOXoaO57xvEksFF4oa5UD+naDgH6ueGwznBYyFFZMJQ4FeIlvYMW/a08vzeNp7f0xqM72mjuajv/vpUgpNn1HHi1BoWTqvhxGm1LJxew4n1MC23BzvYCAd2BJXAwUY4tBtadwXD3OGBb1ozLQj+2umQnhI0I6WnQM3UwadTk4OO6ESkLBT4Fa65rYste9p4fm9QAbzU3M5Lze3s3H+Y4ic21lbFeyuAabWcMLWGuZPTzJ2UYu6kFLXeDq274dCucFhUGXQ0w+H9cLglGHphiNJYEP61M6FuRjicCbUzeofd46nJUFWraw0iJVSywDezi4B/AuLAne7+9X7LrwJuBbqvLn7X3e8Ml10JfDGc/1V3v3uk91Pgj08mV6BxfwfbmzvY3tzOS+Fwe3MHL+/vIJvv+5k3pBJBBTA5zdzJKeZMSjNvcpo5k4LxmQ3VpJJxKBSg61Bv+Hfs760MOpqhbS+0N/UO25sg0zZ4IS0GVfVQPdSrIRimJwcVxGBD/YpZpMdoAn/E7+BmFge+B7wVaATWmNkqd3+m36r3u/sn+207FbgJWEpwZXFduO3+IzgOOUJViRgnz6jj5Bl1A5bl8gVeOdTJ7oOd7DpwmF0HguHug4fZeaCT9Tv2c6AjO2C7SekksxtSzGyoZlZDilkNtcxqmBaMz00xs76aaXVVVCfifTfMdED7XmhrCod7g0qjqzV8tfVOdx4Mmpu6l2Vahz/QeHVvBVBdD9V1UNX9qh1mujZ8FY0na9UkJRVvNP/ClwFb3X0bgJn9BLgE6B/4g3kb8Ki7t4TbPgpcBNw3tuLKeCXiMeZPqWH+lJoh1+nI5Hoqgj2HOtnb2sUrBzvZc6iTPa1dbN27j72tXeQLA78d1lcnmFZXxbS6aqbVBsPpdVVMq53OtLp5TJtSxdTaKqbWVDG5poqqxDB3DBXyQSXQeSD8JnEgHO8/3B9UHJn2oGLJtAbjXW2Q7zqCP05qYGWQrCmqFIYZT9YEz0vuHk+mw2XpYDoWH/n9RY6y0QT+PODloulG4A2DrPc+M1sObAE+4+4vD7HtvMHexMyuAa4BOOGEE0ZRLDlaaqoSvHpmHa+eOfAbQrd8wWlpzwSVQFgpNLd1sa8tQ3N7hua2LrY3d7B+x35a2jMMUjcAQQUxpbaKKTVJpoQVwZTaoFKYXJOkIZWkIV1PQ2oK9Q1JGmYlaEglgyam0chng/DPtPVWAtn23vHu+cXrFI9nO6CjJZjOdgTfWLLtw1zLGEK8uqhCSPdWBH2GReOJNCRTQSWUqA6mE9XBdJ/5qaIKqi6Yp2sjMoRSfYf9D+A+d+8ys78E7gYuOJIduPsdwB0QtOGXqFxylMRjxoz6ambUV7N43qRh180XnAMdQUWwr62L/e1ZWjoyHGjP0NKRYX97hpaOLM1tGbbubWN/e4b2TH7YfVYlYmFlEFQA9aneYd/x7mGa+lQ9DekkdVMS1FUnhv92MRz34BfR3eGf6Qgqg2wHZA+HFcXhYFn2cL/l4TrZw73jHc0D53X/4vpIWXzopqxkTVhJVAcVUM94VW8F0jNe1W+dodZPBrf6xpP6FjMBjCbwdwILiqbn03txFgB3L77Z+07gG0Xbnt9v2/8+0kLKxBaPWdDEU1fNqbPqR7VNZzbPwcNZWjuzHDyc41BnlkOHsxzqzIXDLIeK5rd25th14DCtncG8zuzIZ+BViRh11YneV2rgeG1VgtrqOPWpBLXVwat3mzS11fXU1sWpisewUp5ZFwpBc1SuE3JdYSXQFdxCmwvnZzuD6UxH+K2kqCmr/3THy8G3lHwm2D6fCfaRz4xcllGzoIfY7kogFu87nkj1fpPpM170bSZZE1Q2PZVIMri2EksUzUv0vk/POlW9093j/ed37yMWj+y3oNEE/hrgFDM7iSDALwM+WLyCmc1x993h5MXAs+H4I8A/mNmUcPpC4G/HXWqpeKlknFQyzqyG1Ji2z+YLtHbmaO3M9lQCrWFl0d6Voz2Tp7UzR1tXlvau3vG9rZ28uC9Pa2eWtq7cqCoOCCq1mqo4NVVxaqsSpIuGwfxEz/J0zzBBbdF4TVWcdLJ3/XQyQXWygepUiSuTYu694Z/LhJVM+OoznumtfPqvX8hBPhcMC9mgGa2Q7zves01n+K2nLbiLq/vbTPc3nMF+E3I0xJJFlVOib0XS/S0mngy+2cSTRfPCV6J7vLq3kun+xjPYvD7vFy+qfBJFFWMi2Gbaq47aYY8Y+O6eM7NPEoR3HFjh7pvN7BZgrbuvAq43s4uBHNACXBVu22JmXyGoNABu6b6AK3I0JeOx4OJw7fh6HM3lC7Rn8rR15WjvytHWlaOts2g8nN+RydORyXM4k6c9k+NwOH2gI8OuA/lwebBeV+7I2v/NIB1WgMEwRroqTioRVB7V4TCViAXrhOPVPesH26SScaoTMaoTcaoSMaoTsZ5hdbKKqniK6uoY1bWx0n9jGa3uCqiQCyuLomEhG1Ys2aJKJhusn88E0/lMb0WTz4TDbO8+e/bXfx9FFVP3dHflls/A4Y6w0sr0zst1Fb1PBnz4ZshRqZ0JNzw//v0MQT+8EjnG8gXncDasALrCisC2cBUAAAXASURBVCLbt9LoyOTpzOY5nA2GveOFYJjJ05nrXq9A1yDrjFdPZdCvkuitKIJ5VfFYnwqk+1UdLxpPxEnGYyTiRlU81mc8ETeS4bxkOF683+LpZNzKUxGNRs83me6KoKu3kuiusAr5fpVYrm9FFE8Gj0wdg5Lchy8ipRWPWc91AEZ3SeOIuTtduUJYWRR6KoNMrkAmX6ArW6ArF0x35QrhMPj20fvqvzyoWLq378jkOHC40Ged7ldXPhgeDd3B310ZFFcUiXiMqp7xgRXJYOOJcLwqHE/EiraPBcNEPEYy1r1uuDyc7q20urerJhlLkUxMIlHdu248Vv7KSoEvUoHMrOc6SLm4O9m891QcuYKTyRXI5nvHcwUnmy+QzRXIFjwY5oNKqbtyynYP895TsWTD5blCgUzOw30G47lCIdyn05bL9Yxnw/m5vBdtH75//ti0dASVRG9FEo91VzpGIhZjel0V//pX//vovf9R27OIRJqZUZWwsd/+egy5e5/wz+V7K4Nc3vtVJgOXZ/NBhZXLF/rsI5sPKppc0Tbd2+UL3mdfuYJTW3V0K2gFvohEnllvk0wlq+yjExGRHgp8EZGIUOCLiESEAl9EJCIU+CIiEaHAFxGJCAW+iEhEKPBFRCLiuOw8zcyagO1j3Hw6sK+ExSm3SjseqLxjqrTjgco7pko7Hhh4TCe6+4zhNjguA388zGztSD3GTSSVdjxQecdUaccDlXdMlXY8MLZjUpOOiEhEKPBFRCKiEgP/jnIXoMQq7Xig8o6p0o4HKu+YKu14YAzHVHFt+CIiMrhKPMMXEZFBKPBFRCKiYgLfzC4ysz+a2VYzu7Hc5SkFM3vJzDaZ2QYzm5BPdTezFWa218yeLpo31cweNbPnw+GUcpbxSAxxPDeb2c7wc9pgZu8oZxmPhJktMLPHzOwZM9tsZp8K50/kz2ioY5qQn5OZpczsf8xsY3g8Xw7nn2Rmvw8z734zqxpxX5XQhm9mcWAL8FagEVgDXO7uz5S1YONkZi8BS919wv5gxMyWA23AD919cTjvG0CLu389rJynuPvny1nO0RrieG4G2tz9/5SzbGNhZnOAOe6+3szqgXXAnwJXMXE/o6GO6QNMwM/Jgief17p7m5klgSeATwGfBR5095+Y2e3ARne/bbh9VcoZ/jJgq7tvc/cM8BPgkjKXSQB3Xw209Jt9CXB3OH43wX/GCWGI45mw3H23u68Px1uBZ4F5TOzPaKhjmpA80BZOJsOXAxcAD4TzR/UZVUrgzwNeLppuZAJ/wEUc+KWZrTOza8pdmBKa5e67w/FXgFnlLEyJfNLMngqbfCZM80cxM1sInAX8ngr5jPodE0zQz8nM4ma2AdgLPAq8ABxw91y4yqgyr1ICv1Kd4+6vB94OfCJsTqgoHrQpTvR2xduAVwFnAruB/1ve4hw5M6sDfgp82t0PFS+bqJ/RIMc0YT8nd8+7+5nAfIIWjUVj2U+lBP5OYEHR9Pxw3oTm7jvD4V7gZwQfdCXYE7azdre37i1zecbF3feE/yELwPeZYJ9T2C78U+Aed38wnD2hP6PBjmmif04A7n4AeAx4EzDZzBLholFlXqUE/hrglPCqdRVwGbCqzGUaFzOrDS84YWa1wIXA08NvNWGsAq4Mx68E/r2MZRm37mAMvYcJ9DmFFwT/BXjW3b9ZtGjCfkZDHdNE/ZzMbIaZTQ7H0wQ3pzxLEPx/Fq42qs+oIu7SAQhvsfo2EAdWuPvXylykcTGzkwnO6gESwL0T8ZjM7D7gfIKuXPcANwH/BqwETiDoBvsD7j4hLoQOcTznEzQTOPAS8JdF7d/HNTM7B3gc2AQUwtlfIGjznqif0VDHdDkT8HMys9cRXJSNE5ykr3T3W8KM+AkwFfgD8CF37xp2X5US+CIiMrxKadIREZERKPBFRCJCgS8iEhEKfBGRiFDgi4hEhAJfRCQiFPgiIhHx/wEiHAiXlQlwrAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "I-DU63qYYQ91", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "c683b48f-9d32-45ee-a97c-611723f93dd3" }, "source": [ "predictions = model.predict([X_test_scaled_A, \n", " X_test_scaled_B])\n", "print('R2:', r2_score(y_test, predictions))\n", "print('MSE:', mean_squared_error(y_test, predictions))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "R2: 0.6663551247588422\n", "MSE: 0.4379222805205722\n" ], "name": "stdout" } ] } ] } ================================================ FILE: notebooks/TF2_Crash_Course_02_Deep_Learning_Essentials.ipynb ================================================ { "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "name": "TF2 Crash Course 02: Deep Learning Essentials.ipynb", "provenance": [], "collapsed_sections": [] }, "kernelspec": { "name": "python3", "display_name": "Python 3" }, "accelerator": "GPU" }, "cells": [ { "cell_type": "markdown", "metadata": { "id": "ib9kB4AYaTV8" }, "source": [ "# Deep Neural Nets Essentials\n", "\n", "In this session we will deepen our focus on understanding various aspects of Deep Neural Nets" ] }, { "cell_type": "code", "metadata": { "id": "0w3-LfYdjPX6", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "2c5848c8-6283-466e-f586-4bfc0261ff1f" }, "source": [ "!nvidia-smi" ], "execution_count": 1, "outputs": [ { "output_type": "stream", "text": [ "Sat Feb 13 03:04:02 2021 \n", "+-----------------------------------------------------------------------------+\n", "| NVIDIA-SMI 460.39 Driver Version: 460.32.03 CUDA Version: 11.2 |\n", "|-------------------------------+----------------------+----------------------+\n", "| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |\n", "| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |\n", "| | | MIG M. |\n", "|===============================+======================+======================|\n", "| 0 Tesla P100-PCIE... Off | 00000000:00:04.0 Off | 0 |\n", "| N/A 31C P0 26W / 250W | 0MiB / 16280MiB | 0% Default |\n", "| | | N/A |\n", "+-------------------------------+----------------------+----------------------+\n", " \n", "+-----------------------------------------------------------------------------+\n", "| Processes: |\n", "| GPU GI CI PID Type Process name GPU Memory |\n", "| ID ID Usage |\n", "|=============================================================================|\n", "| No running processes found |\n", "+-----------------------------------------------------------------------------+\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "IAGT9QbdaeYG" }, "source": [ "# Import Dependencies" ] }, { "cell_type": "code", "metadata": { "id": "DYc1V3lEaSWF" }, "source": [ "import matplotlib.pyplot as plt\n", "import pandas as pd\n", "import numpy as np\n", "import sklearn\n", "import tensorflow as tf" ], "execution_count": 2, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "bGsGZnrUbfFY", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "c97de57a-2654-4fd5-de35-e16ec46856da" }, "source": [ "print(tf.__version__)" ], "execution_count": 3, "outputs": [ { "output_type": "stream", "text": [ "2.4.1\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "chxffg2hbhB_", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "152a5748-0741-4885-fbfb-34c5d5a62a96" }, "source": [ "print(sklearn.__version__)" ], "execution_count": 4, "outputs": [ { "output_type": "stream", "text": [ "0.22.2.post1\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "yid423I2biJ9" }, "source": [ "# 1. Building a DNN based Image Classifier\n", "\n", "We will keep things simple here with regard to the key objective. We will build a simple apparel classifier by training models on the very famous [Fashion MNIST](https://github.com/zalandoresearch/fashion-mnist) dataset based on Zalando’s article images — consisting of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28x28 grayscale image, associated with a label from 10 classes. The idea is to classify these images into an apparel category amongst 10 categories on which we will be training our models on.\n", "\n", "Here's an example how the data looks (each class takes three-rows):\n", "\n", "\n", " \n", " \n", "
\n", " \"Fashion\n", "
\n", " Fashion-MNIST samples (by Zalando, MIT License).
 \n", "
\n", "\n", "Fashion MNIST is intended as a drop-in replacement for the classic [MNIST](http://yann.lecun.com/exdb/mnist/) dataset—often used as the \"Hello, World\" of machine learning programs for computer vision. You can access the Fashion MNIST directly from TensorFlow.\n", "\n", "__Note:__ Although these are really images, they are loaded as NumPy arrays and not binary image objects." ] }, { "cell_type": "markdown", "metadata": { "id": "tcunPKn0c4e1" }, "source": [ "## 1.1: Understanding the data" ] }, { "cell_type": "code", "metadata": { "id": "F_hrm3uvcAop", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "9287e153-2fd3-4e00-cd49-607e9fc1a648" }, "source": [ "fashion_mnist = tf.keras.datasets.fashion_mnist\n", "(X_train, y_train), (X_test, y_test) = fashion_mnist.load_data()\n", "\n", "X_train.shape, X_test.shape" ], "execution_count": 5, "outputs": [ { "output_type": "stream", "text": [ "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz\n", "32768/29515 [=================================] - 0s 0us/step\n", "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz\n", "26427392/26421880 [==============================] - 0s 0us/step\n", "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz\n", "8192/5148 [===============================================] - 0s 0us/step\n", "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz\n", "4423680/4422102 [==============================] - 0s 0us/step\n" ], "name": "stdout" }, { "output_type": "execute_result", "data": { "text/plain": [ "((60000, 28, 28), (10000, 28, 28))" ] }, "metadata": { "tags": [] }, "execution_count": 5 } ] }, { "cell_type": "markdown", "metadata": { "id": "5KZ0rZzocYdV" }, "source": [ "We scale the pixel intensities down to the 0-1 range and convert them to floats, by dividing by 255 (NNs like normalized inputs)." ] }, { "cell_type": "code", "metadata": { "id": "Kwm4P6wzcNS6" }, "source": [ "X_train = X_train / 255.\n", "X_test = X_test / 255." ], "execution_count": 6, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "Eww0SuIrcbxU", "colab": { "base_uri": "https://localhost:8080/", "height": 265 }, "outputId": "25c18e6a-ae7d-4b85-ac2b-ef7b3a9c4d8f" }, "source": [ "plt.imshow(X_train[0], cmap=\"binary\");" ], "execution_count": 7, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAASLElEQVR4nO3dXYyV5bUH8P8CBmUAkYFx5GOEiiRiwEPJhiA1jYdGAiQGuTHlouEkRnqBCU0aPYSTWC/NiW3TixMSUCw9qRYSULkgHjhAghOlstE5iGiB4iDDx3xIhEFABNa5mJdmxHnXGt93f5X1/yWTmdlr3r2fvWf+7M1e7/M8oqogotvfoGoPgIgqg2EnCoJhJwqCYScKgmEnCmJIJW9s7NixOnny5EreJFEobW1t6O7ulv5qucIuIgsB/AHAYACvqOpL1s9PnjwZxWIxz00SkaFQKKTWMr+MF5HBAP4LwCIADwFYJiIPZb0+IiqvPP9nnwPgmKoeV9WrAP4CYElphkVEpZYn7BMAnOzzfXty2XeIyAoRKYpIsaurK8fNEVEeZX83XlXXqWpBVQuNjY3lvjkiSpEn7KcANPf5fmJyGRHVoDxh3w9gqoj8SESGAvg5gG2lGRYRlVrm1puqXhORZwH8D3pbbxtU9ZOSjYyISipXn11VtwPYXqKxEFEZ8XRZoiAYdqIgGHaiIBh2oiAYdqIgGHaiIBh2oiAYdqIgGHaiIBh2oiAYdqIgGHaiIBh2oiAqupQ0VZ63cadIv6sOD1hPT49Zb2lpSa0tWrQo12179+369euptSFDqvunn2dD1ay/Mz6zEwXBsBMFwbATBcGwEwXBsBMFwbATBcGwEwXBPvtt7saNG2Z98ODBZv3YsWNm/ZVXXjHrw4YNS60NHz7cPPbOO+8063PmzDHreXrpXh/ce1y94/OMzTp/wMJndqIgGHaiIBh2oiAYdqIgGHaiIBh2oiAYdqIg2Ge/zXk9Wa/Pvnv3brO+c+dOs97c3Jxa++abb8xjL126ZNZ37Nhh1p955pnUWlNTk3msN2fce9w8Fy9eTK0NGmQ/B9fX12e6zVxhF5E2AD0ArgO4pqqFPNdHROVTimf2f1XV7hJcDxGVEf/PThRE3rArgB0ickBEVvT3AyKyQkSKIlLs6urKeXNElFXesD+qqrMALAKwUkR+eusPqOo6VS2oaqGxsTHnzRFRVrnCrqqnks+dAN4EYE9DIqKqyRx2ERkuIiNvfg1gAYBDpRoYEZVWnnfjmwC8mfQjhwB4XVXfKcmoqGSGDh2a6/j9+/eb9ba2NrNuzfv25oQvWLDArH/00Udm/fnnn0+tFQp2l3jGjBlmfdq0aWb9gw8+MOvW4zpv3jzz2EceeSS1Zq6Vb16rQVWPA/iXrMcTUWWx9UYUBMNOFATDThQEw04UBMNOFASnuN4GrGWLvama3hTVYrFo1u+66y6z/vXXX6fWjhw5Yh7r1WfPnm3WH3jggdSaNcUUAN577z2zvnXrVrPuLRVtLYO9fv1681irnWpNC+YzO1EQDDtREAw7URAMO1EQDDtREAw7URAMO1EQ4m0tW0qFQkG9vm1E5fwdeH32uXPnmnVvCqvHum/ecsx33HFHrtu2tnz2HpdZs2aZ9alTp5p177698076bPDjx4+bx54+fTq1VigUUCwW+71zfGYnCoJhJwqCYScKgmEnCoJhJwqCYScKgmEnCoLz2WuA1/Mtp9GjR5v1M2fOmPVhw4aZdWtb5m+//dY81ptzbvXRAeDy5cupNe8xb2lpMevefHfv3ImOjo7U2sKFC81js+IzO1EQDDtREAw7URAMO1EQDDtREAw7URAMO1EQ7LMHZ60zDthbAAP+tstWH/7ee+81jx0zZoxZ9+baDxqU/lzm9cG9+2318L3bBuz57u3t7eaxWbnP7CKyQUQ6ReRQn8saRGSniBxNPttnZhBR1Q3kZfwfAdx6Ss9qALtUdSqAXcn3RFTD3LCr6l4A5265eAmAjcnXGwE8WeJxEVGJZX2DrklVb540fRZAU9oPisgKESmKSLGrqyvjzRFRXrnfjdfedzpS3+1Q1XWqWlDVQmNjY96bI6KMsoa9Q0TGAUDyubN0QyKicsga9m0AlidfLwfwdmmGQ0Tl4vbZReQNAI8BGCsi7QB+A+AlAJtF5GkAJwA8Vc5B3u68nq/Xy7Z6tt6ccGsNcsBfu93aKxwArl69mvm6hw8fbtbPnz9v1q0+vXd+gTVuABgxYoRZv3DhglmfMWNGas3a0x4ArL0XrPvlhl1Vl6WUfuYdS0S1g6fLEgXBsBMFwbATBcGwEwXBsBMFwSmuNcBb1tibbmm13jZt2mQe6y0V7Z316E31tMbmtZi++OILs15XV2fWrWWshwyx//S9Za69+93d3W3WV65cmVprbW01j7127VpqzWrj8pmdKAiGnSgIhp0oCIadKAiGnSgIhp0oCIadKAj22WuA1TcF/GmklunTp5t1b5qp12/Ocw5AZ6e95om3JXNDQ4NZtx5X73555wB4W103Nzeb9ddffz219txzz5nHzp07N7VmTQvmMztREAw7URAMO1EQDDtREAw7URAMO1EQDDtREP9UfXZrrm7erYW95ZytudPe9rweb251HosWLTLr3pLI1pbLgL/kssWbK++df3DlyhWznuf8BO934v3Ovb/HgwcPptZGjRplHpsVn9mJgmDYiYJg2ImCYNiJgmDYiYJg2ImCYNiJgqipPnueudHl7FWX2969e836li1bzHpLS0tqrb6+3jzW2tYYsNdeB/w1763fizc27+/BG5vVh/fG7W0X7fHOP7Cuf+vWreaxTzzxRKYxuc/sIrJBRDpF5FCfy14UkVMi0pp8LM5060RUMQN5Gf9HAAv7ufz3qjoz+dhe2mERUam5YVfVvQDOVWAsRFRGed6ge1ZEDiYv81MX5BKRFSJSFJFiV1dXjpsjojyyhn0tgCkAZgI4A+C3aT+oqutUtaCqBW/iAxGVT6awq2qHql5X1RsA1gOYU9phEVGpZQq7iIzr8+1SAIfSfpaIaoPbnBaRNwA8BmCsiLQD+A2Ax0RkJgAF0Abgl6UYjNVHz+vcOfs9xtOnT5v1I0eOZD7W65ta1w34a7tbc/W9fvGXX35p1sePH2/WvbXdrfXZOzo6zGO9+33p0iWzPm/evNRaT0+Peey7775r1r357N6cdGt9hH379pnHZuWGXVWX9XPxq2UYCxGVEU+XJQqCYScKgmEnCoJhJwqCYScKoqbmhb7//vtm/YUXXkiteafifvXVV2bda6VY7a27777bPNZrKY4cOdKsey0oaxlsbyloqz0FAJs2bTLrs2fPNusXLlxIrXltu7a2NrPusZZrvnjxonnsxIkTzbrX0vTagtaW0Hnvdxo+sxMFwbATBcGwEwXBsBMFwbATBcGwEwXBsBMFUfE+u7U88KpVq8xjramkebfYzbN0sLeksdfr9uqe8+fPp9ZOnDhhHrt69Wqz7o1t7dq1Zn3cuHGpNa/PPn/+fLM+ZcoUs3706NHUmje115qCCvjbSXtbhFt/r/fcc495bFZ8ZicKgmEnCoJhJwqCYScKgmEnCoJhJwqCYScKoqJ99u7ubmzcuDG17vWE77///tSaNT8Y8JcO9vquFq/navXBAX/u9IQJE8z65cuXU2tNTU3mscuXLzfrb731lln3tg/+/PPPU2ve7+zAgQNmfc+ePWbdOqfDWyPAO3fC25LZY/XZves+efJkpmP5zE4UBMNOFATDThQEw04UBMNOFATDThQEw04UREX77HV1deZcXa/fbPXKvb7pfffdl/m6AXvrYWttdABoaGgw65MmTTLr3tiseeHenHFvTfulS5ea9RkzZph1aw1079wG73fqrddvzUn37vfQoUPNutcL99ZPsNb6t2qAvcW3dX6A+8wuIs0iskdEDovIJyKyKrm8QUR2isjR5PNo77qIqHoG8jL+GoBfq+pDAOYCWCkiDwFYDWCXqk4FsCv5nohqlBt2VT2jqh8mX/cA+BTABABLANw893UjgCfLNUgiyu8HvUEnIpMB/BjAXwE0qeqZpHQWQL8nYYvIChEpikjRO0eciMpnwGEXkREAtgD4lap+5x0p7X1Hod93FVR1naoWVLUwatSoXIMlouwGFHYRqUNv0P+sqluTiztEZFxSHwegszxDJKJScFtvIiIAXgXwqar+rk9pG4DlAF5KPr/tXVddXZ3ZXvPaFc3Nzak1b7qkt6Wz18ZpbGzMVAP8KbDedErv+CtXrqTWvK2JrWmgADBmzBizfvjwYbM+YsSI1JrXDh092m7wWPcbsH8v3tLj3lLS3vHWtGMAOHv2bGrNewXc2tqaWrO2ih5In/0nAH4B4GMRuXkra9Ab8s0i8jSAEwCeGsB1EVGVuGFX1RYAklL+WWmHQ0TlwtNliYJg2ImCYNiJgmDYiYJg2ImCqOgU1/r6esycOTO17k2nfO2111Jr48ePN4/1tvf1poJa/WpvuqPXc7WmzwJ+n90au3ds72kU6err6826tSUzYJ874U0z9cbunRuRZ0q0d91e3Zsia/XxreW3AXt5cOt6+cxOFATDThQEw04UBMNOFATDThQEw04UBMNOFIR4y9aWUqFQ0GKxmPn47du3p9Zefvll89jOTnttDW9OutVX9ebh37hxw6x789m9OedWP9r7/Xp9dq/X7Z1jYNW96877t2kdby1pPhDeuRHe34Q1n/3hhx82j928eXNqrVAooFgs9vtL5TM7URAMO1EQDDtREAw7URAMO1EQDDtREAw7URAVnc8O2D1nrze5ePHiTDUA2L17t1lfs2aNWbe2Hva2tfL6xV4f3evpWmuYe7ft9Zu9Pry3zbY1195aUx7wH5c8vPnm3jx+79yJxx9/3KxPmzYttTZv3jzz2Kz4zE4UBMNOFATDThQEw04UBMNOFATDThQEw04UxED2Z28G8CcATQAUwDpV/YOIvAjgGQA3Nz5fo6rpE84TXi+9XObPn2/W9+3bl/m6P/vsM7Pu7Q3v7UPe3t5u1idNmpRa8/rJ3nr6dPsYyEk11wD8WlU/FJGRAA6IyM6k9ntVtVeNIKKaMJD92c8AOJN83SMinwKYUO6BEVFp/aDX1CIyGcCPAfw1uehZETkoIhtEpN/XoiKyQkSKIlL0Xs4SUfkMOOwiMgLAFgC/UtULANYCmAJgJnqf+X/b33Gquk5VC6pa8NZ5I6LyGVDYRaQOvUH/s6puBQBV7VDV66p6A8B6AHPKN0wiyssNu/ROe3oVwKeq+rs+l/fdvnMpgEOlHx4RlcpA3o3/CYBfAPhYRFqTy9YAWCYiM9HbjmsD8MuyjPCfwIMPPpir7pk+fXqu44mAgb0b3wKgv0nNbk+diGoHz6AjCoJhJwqCYScKgmEnCoJhJwqCYScKgmEnCoJhJwqCYScKgmEnCoJhJwqCYScKgmEnCoJhJwpCvC19S3pjIl0ATvS5aCyA7ooN4Iep1bHV6rgAji2rUo5tkqr2u/5bRcP+vRsXKapqoWoDMNTq2Gp1XADHllWlxsaX8URBMOxEQVQ77OuqfPuWWh1brY4L4NiyqsjYqvp/diKqnGo/sxNRhTDsREFUJewislBE/iYix0RkdTXGkEZE2kTkYxFpFZFilceyQUQ6ReRQn8saRGSniBxNPtv7PVd2bC+KyKnksWsVkcVVGluziOwRkcMi8omIrEour+pjZ4yrIo9bxf/PLiKDARwB8DiAdgD7ASxT1cMVHUgKEWkDUFDVqp+AISI/BXARwJ9UdXpy2X8COKeqLyX/UI5W1X+vkbG9COBitbfxTnYrGtd3m3EATwL4N1TxsTPG9RQq8LhV45l9DoBjqnpcVa8C+AuAJVUYR81T1b0Azt1y8RIAG5OvN6L3j6XiUsZWE1T1jKp+mHzdA+DmNuNVfeyMcVVENcI+AcDJPt+3o7b2e1cAO0TkgIisqPZg+tGkqmeSr88CaKrmYPrhbuNdSbdsM14zj12W7c/z4ht03/eoqs4CsAjAyuTlak3S3v+D1VLvdEDbeFdKP9uM/0M1H7us25/nVY2wnwLQ3Of7icllNUFVTyWfOwG8idrbirrj5g66yefOKo/nH2ppG+/+thlHDTx21dz+vBph3w9gqoj8SESGAvg5gG1VGMf3iMjw5I0TiMhwAAtQe1tRbwOwPPl6OYC3qziW76iVbbzTthlHlR+7qm9/rqoV/wCwGL3vyP8dwH9UYwwp47ofwP8lH59Ue2wA3kDvy7pv0fvextMAxgDYBeAogP8F0FBDY/tvAB8DOIjeYI2r0tgeRe9L9IMAWpOPxdV+7IxxVeRx4+myREHwDTqiIBh2oiAYdqIgGHaiIBh2oiAYdqIgGHaiIP4fcKosV18KmAoAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "markdown", "metadata": { "id": "nF4qF5P0cmya" }, "source": [ "The labels are the class IDs (represented as uint8), from 0 to 9:" ] }, { "cell_type": "code", "metadata": { "id": "rPdWjqbhcked", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "158c5f5b-659d-4955-d047-f4d1f0102c34" }, "source": [ "y_train" ], "execution_count": 8, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "array([9, 0, 0, ..., 3, 0, 5], dtype=uint8)" ] }, "metadata": { "tags": [] }, "execution_count": 8 } ] }, { "cell_type": "markdown", "metadata": { "id": "JCx593p5csMY" }, "source": [ "Here are the corresponding class names:" ] }, { "cell_type": "code", "metadata": { "id": "_pSvtZAIco4v" }, "source": [ "class_names = [\"T-shirt/top\", \"Trouser\", \"Pullover\", \"Dress\", \"Coat\",\n", " \"Sandal\", \"Shirt\", \"Sneaker\", \"Bag\", \"Ankle boot\"]" ], "execution_count": 9, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "c7qx4Si1cu3X" }, "source": [ "So the first image in the training set is a Ankle Boot:" ] }, { "cell_type": "code", "metadata": { "id": "BtaXMu2NctiY", "colab": { "base_uri": "https://localhost:8080/", "height": 35 }, "outputId": "24cc8953-dc93-4a4d-cbe0-642b7d85a6c7" }, "source": [ "class_names[y_train[0]]" ], "execution_count": 10, "outputs": [ { "output_type": "execute_result", "data": { "application/vnd.google.colaboratory.intrinsic+json": { "type": "string" }, "text/plain": [ "'Ankle boot'" ] }, "metadata": { "tags": [] }, "execution_count": 10 } ] }, { "cell_type": "code", "metadata": { "id": "qtuU21-dcwSf", "colab": { "base_uri": "https://localhost:8080/", "height": 308 }, "outputId": "4d4b79a7-bf0c-4139-a78e-c6931358e9ee" }, "source": [ "n_rows = 4\n", "n_cols = 10\n", "plt.figure(figsize=(n_cols * 1.2, n_rows * 1.2))\n", "for row in range(n_rows):\n", " for col in range(n_cols):\n", " index = n_cols * row + col\n", " plt.subplot(n_rows, n_cols, index + 1)\n", " plt.imshow(X_train[index], cmap=\"binary\", interpolation=\"nearest\")\n", " plt.axis('off')\n", " plt.title(class_names[y_train[index]], fontsize=12)\n", "plt.subplots_adjust(wspace=0.2, hspace=0.5);" ], "execution_count": 11, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAqsAAAEjCAYAAAAVEEjZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydeZhcRdW43zN7JttMFkI2EkJYYkKMKGERBAIoRBBUQJEA+fihIC4gfCjygSKCoMiiIsIH8rHJIgKyyBIhBMISkB0CARISsi+QZTLJ7FO/P+qe6urbt2dfuqHe55lnuvvevn3r3FpOnTrnlBhjCAQCgUAgEAgEcpGC3r6BQCAQCAQCgUAgG0FZDQQCgUAgEAjkLEFZDQQCgUAgEAjkLEFZDQQCgUAgEAjkLEFZDQQCgUAgEAjkLEFZDQQCgUAgEAjkLN2urIqIEZHx7T3WyjVnisgznb+73kVElojIQVmO7Ssi7/b0PfUmQR6Blgj1IzvxPrGjfWvA0lJdy0dC/eh62ipDERkbnVvUE/fVW3S3PNqsrIrIHBHZICKl7fmBfEJE9heR5W04r9r7axaRGu/9cV1xL8aYucaYnVu5j8QOVUSOFZHbe6qRBHl0HVEZakRks4hsFJHnRORUEcnbVZBQP9qHVweqRWSNiNwkIv168556ExHZJ2oHm0RkvYg8KyK79/Z99RahfrRMqC/pfFLk0aYBUETGAvsCBvhaN95PXmCM6ad/wFLgcO+zv3X377dhMP0q8HB334cS5NHlHG6M6Q+MAS4Ffgb8NelEESnsyRvrCKF+dIjDI3ntBnwBOK+X76dFukvBF5EBwEPAn4BBwEjgV0Bdd/xeV9LNk55QP5J/J2/rS3fwSZJHW601JwDzgJuAE/0D0azuzyLyr8ga9IKI7JB0kUjDXyYi+yccKxWR34vI0mi2eK2I9GnhnkREro5mCwtE5EDvwAgReSCaRSwUke/GfucqEVkZ/V0VfdYXeAQY4Vl9RrRRPi3d5BAReSiykq0XkbkxK9kUEXkjKsddIlIWfS/NyhvNpn8mIm8AW0TkDmA74MHoXn8anVcAHAw8CjwdfX1jdM5eIlIgIueJyIcislZEbhGRgdF31bL0vUg2q0TkvzsrgyCPjmGM2WSMeQD4FnCiiEyK2ttfRORhEdkCHBDV93tEZJ2ILBaRH3tymioiL4lIVdSurog+LxOR20Tk4+hZ/EdEhvVU2bIR6kc6xpgV2H5pksSsvmJXu05u7RoiMjC673VROc6LylUayXmSd+5QsVa7baL3h4nIa5Ky8k/2zo3LuDsUkp0AjDF3GGOajDE1xphZxpg3JFraFjtubIjq/qGxcv81ei4rROQiiSZ3IrKDiMyO6v9HIvI3EanIIr8J0bWPzRGZOEL9yKCl+tLiM4/u97+T+pfo+NlRXVopIif5PyoiXxWRV8X2s8tE5IIeKGtb+OTIwxjT6h+wEDgN+DzQAAzzjt0EfAxMBYqAvwF3escNMB44BFgGTI0fi15fCTyA1f77Aw8Cl2S5n5lAI/AToBg7mG8CBkXHnwauAcqAKcA6YFp07EKs4r0NMBR4Dvh1dGx/YHlbZOLdyxLgoBaOXwJcG91nMdZCLd53XwRGROV+Bzg16V6ic18DRgN9sv02sCfwfPR6bCTjIu/4SdHzHAf0A+4Fbo2dfwfQF9g1kl3W8gV5dO1fNvlhLZLfx7a3TcAXsZPNcuBl4BdASVSOD4CvRN97Hjg+et0P2DN6fQq2jZUDhdi2PaC7yhXqR8dkFN3/fODWhHubA5wcvZ4JPOMd8/vWW4D7sf3qWOA94P9Fx24ELva+9wPg0ej154C1wB5RHTkxurfSbDLuhvoyADu+3AwcClR6x2Zix6PvRvf3fWClV1/uA66Lns02Ud05JTo2HjtJKcWOA08DV8WfAdZyuRQ4LIdkEupHx+pLW555tv7lEGANMCmqT7fHZLg/tv0XAJOjc4/M1q/01N8nSR5tKew+2A5hSPR+AfAT7/hNwA3e++nAglij+DnwITApdm1VZAXYAuzgHdsLWJzlnmbidUrRZy8Cx0cNowno7x27BLgper0ImO4d+wqwxBNwVyurF2I7gvFZvjvDe/874Nqke4nOPam13wZ+DZyfrVIATwCnee93jp5vkXf+LrF7+muQR8fk0d6/bPLDTrD+B9vebvE+3wNYGjv358D/Ra+fxi77DImdcxJ2oja5u8oS6kenZFQNbMT2m9cAExLubQ6tKCNYJaIe+Ix37BRgTvT6IGCRd+xZ4ITo9V+IJvLe8XeB/bLJuJvqzISo3i/HGikeAIZFZV7onVcelXvb6HgdnpIEHAs8meU3jgRejT2DX0W/ub/3ea/LJNSPjtWXNj7zbP3LjcCl3rGd8JSzhGtfBVwZvR4bfzZBHu2XR1vcAE4EZhljPore307MFQBY7b3eirVA+JwB/N0Y81aW3xhKZCGKlhM2YpflhrZwXytMVPKID7EzgBHAemPM5tixkdHrEdH7+Pc6jYhsJ14wSfTxZVjLzCwR+UBEzol9rTXZ+Sxrw21Mp2X/u6TyF2E796Tf6bB8gjy6lJHA+ui1fz9jsK4rG722cy6p+/9/2I5kgdil/sOiz28FHgPujJZxficixd1fjBShfrTIkcaYCmPMGGPMaUBNB68zBGuhjpdB+8MngXIR2UNsbMIUrEUSbN06K1a3RpNevrbIuFMYY94xxsw0xozCWnJGYAc/8OqDMWZr9LJfdO/FwCrv3q/DWlgRkWEicmfkHlAF3IaVlc+pwHPGmDneZzkhE0L9yEq2+tLGZ56tfxlBZjt3RPJ5MnKl2IStO/Fr9wqfFHm0qKyK9Rk9BthPRFaLyGrs0vtnReSz7fido4EjReT0LMc/wja2iVEDrDDGDDTWgTwbI0VEvPfbYa2tK4FBItI/dmxF9HoltpHFvwdW2+8wxpilJj2YBGPMZmPMWcaYcdjgtDPF869t70+09F5EtgWGA69kOR+Sy9+INdMro2PHV9IBgjy6BrGRmyMBTT3jl2MZdgWiwvvrb4yZDmCMed8Ycyx2kP4t8A8R6WuMaTDG/MoY8xlgb+AwrG96jxHqR7vYEv0v9z7btg3f+whrCY6XYQWAMaYJ+DvW6ngs8JA30V+GXQL261a5MeYO71qd6jPbizFmAdZKNKmVU5dhLatDvHsfYIyZGB3/DfbedzXGDABmYFf4fE4FthORK2PXzSmZRIT6kUCsvrTlmWdjFZnt3Od2rMVytDFmINaVqa3X7jHyWR6tWVaPxC6pfwY7o5qCNSnPpX0D20rgQOB0Efl+/KAxphm4HrhSUo7bI0XkKy1ccxvgxyJSLCJHR/f1sDFmGXZ58xKxQSSTsdal26Lv3QGcJ9ZRfAjW10+PrQEGSxRA0RWIdUAfHynWm7DybO6iy6/B+tIph2L9ibSDWBf9ln/OHcBPRGR7selOfgPcZYxp9M45X0TKRWQi8F/AXV10v0Ee7UBEBkSW0DuB24wxbyac9iKwWWwgQx8RKRQbiLV7dI0ZIjI0amMbo+80i8gBIrKr2ICTKuyA1VXPocOE+pGMMWYdVoGYET3jk4DEQNbY91TZuFhE+ovIGOBMUn0e2IHlW8Bx0WvleuDUyEoiItJXbOCEbwjoVkRkFxE5S0RGRe9HY5WmeS19zxizCpgFXB61owKxASX7Raf0xy6lbxKRkcDZCZfZjPXN+5KIXBp91usySeLTWj/itFJf2vLMs/F3YKaIfEZEyoFfxo73x67o1orIVOA7nS1LV/BJkkdryuqJWN+3pcaY1foHXA0cJ+2I7jPGLMUqrOdIcoTiz7DLf/PEmqQfx/qHZeMFYEfszPBi4ChjzMfRsWOxfhErsUsWvzTGPB4duwh4CXgDeBNrVbkouscF2MHpA7HLGl2xnLdjVJZqbLDLNcaYJ7vgumB9cc+L7vW/iaXgiZbFLgaejc7ZE+trcivWl3ExUAv8KHbdp7DP4gng98aYWV10vxDk0RYeFJHNWMvF/wBXYJWgDKLB5jDsRHIxtj3cAOiE6xBgvthl9j8A3zbG1GCtLv/AKqrvYMt4a3cVqB2E+pGd72IHlI+BidhJeVv4Edby9gHWOn87tlwAGGNeiI6PwEaW6+cvRb95NbABW8aZnSxDe9mM9ct+QWz2i3nAW8BZbfjuCdigw7ex9/8PrCUdrD/qbtgJ0b+wgXMZGGM2YoNQDhWRX+eITLLxaawfcVqqL2165kkYYx7Bup7MxpZzduyU04ALo377F1hlLhf4xMhDoyYDeU40cVgNjDPGVHXwGmOxA3JxzHKUdwR5BFoi1I9AIBDIH/J2V5xABoOwUc0dGng/gQR5BFoi1I9AIBDIE4JlNeAIlqJ0gjwCLRHqRyAQCPQMQVkNBAKBQCAQCOQswQ0gEAgEAoFAIJCzBGU1EAgEAoFAIJCztDn1VAvkux9BVyfubVUexhhEsv/sO++8A8APf/hDAI455hg+97nPAVBSUgJAUVER8+fPB+C+++yGIuPG2fSQP/3pT6moqOjo/fe4PLKxdu1aAG666SYATjjBpvbddtuWc12/9tprACxYsACAb37zmxQXd3hzppyQx+LFi3nqqacAuP/++wEYNGgQAMcffzy77bYbkCrzPffcw+OP22xtffv2BWDGjBkAfO973+vovUOOyKMzrFxpc/aPGNE1G9d1xUU8sspDXbay9R3aXmbPtllkrr/+egAqKiqYMGECAKWlpQBs2LCB559/HoA999wTgN/85jcA9OnTJ/G3W+qzPPK+fnQxvTK+uB9v4Zlpf7LDDjswatSojOOLFy8G4KWXXgLg6KOPbt+dJhPqRzrdsXHAJ1ImXeGz+okUTCfIkEdLg8yrr74KwF133cU999wDQGFhIQDV1XYHypqaGtavX5/xXWWnnXYCoKDAGsoXLFjgFLqvfMXuq3DWWWex6667tuX+c6Izqa6u5s477wTgqqvszoqqqA8dOtS9ViW0urqauro6AJYts7vAHXnkkQDstddeneloe0Uejzxi0xleeaXdPKdPnz7U19cDUFZWBkBVlQ1knz9/PmvW2A2Vxo4dC9jJzPDhNqXkwIE25arKZ/ny5Rx00EEA/PGPf2zv/edE/Zg2bRobNmwAYMgQu4ufKmcqA5+VK1dywAEHALY9AWy3nd105bHHHnMKfQfolf7jo4/s7td/+MMfAHj88cepra0FUpMTrS8LFixg82Z/92nbbkaOtDtqaj1RuQwaNIj99rO583/0I5tStrKysq33nxP1I4focXk0Nze7sUBZvnw5N95oU6defvnlQKr/aA29lva1v/3tbzn99MzNKJubm9POz0KoH+kEZTWToKxmoccbT1VVlbMSvv766/ZLxtCvn91dVi0bRUXW8F1YWEhjow023rRpEwDl5eVOqU1SgnXg0gGovr6effbZB4Dbbrst43yPnOlM7r77biAlj4svvhiwiocqZ6qAVVRU0L+/3ThFFbHvfMdumlFdXe0U1w7Q4/JYtGgRF1xwAQDbbLMNYJ9jfDDQ+qHKOaTqQmFhIQMGDABSg4yeP3jwYJYvXw7gLPA6gLWBnKgf+++/P4sWLQJSdUDrer9+/TjqqKOAVF1vampySr6WWeuVtsEO0uPK6qJFizjssMOA1CpDWVmZe87aL6gVddCgQW7i6x9TZXbdunUAro+pq6ujoaEBsP0MwCmnnMI3vvGNttx/TtSPHKLH5JGkLOqK3Pvvv+/aiT5T/V9bW+smI9o2Vq1a5dqTthMdU6qrq92qzoEH2p2Qb789tbFVK0prztcPY0xGGfwxNq4zZbNeP/ec3ZNh7733BuDdd98FrHHJ+07eKKttLXc2ZsyYwZlnngngVgK1TmpfpZdO+n7wWQ0EAoFAIBAI5Cw5aVlN8o/avHkzzzzzDACHHnpoxvlNTU1AynqU7bpKN85sWpXHgQceyNKlSwFr5dL70TKo9cNHZ3pqPdFzIXPGk3YznlVm1apVADz66KMAzo8tRs7MfNUqNmzYMABnTf3Tn/7kloB9y+rnP/95AP7rv+zOpEuWLAGs28AhhxzS0dvocXmcdtppzgqo9XTLli3OsqH1Q5d7i4qK3FK/niMiTjaKb6lXa8lbb70FWL9Xtda1Qk7Uj2984xu8/PLLQMpCpK4ya9eudfX+S1/6EgBvvPGGq0dqNVR3AfXx7CA9Lo9jjjnGuQGoRayxsdHVFe0j1CpUWlrqLBf6v76+3q3SqDz8fkT7G7W+1tfXO19pXQHKQk7UjxyiV9xE9tprLyDlbzps2DD3LPU8taQXFBSwZcuWtGv16dPH9RdaP7RP8r+r9fCII47gn//8Z6v3RR7UD9+ymjQWt8ScOXMAePPNN3n//fcB2/fodQFmzZrlWxJzxrKapHclPcOWfKK1rhQXF/Pmm28CuFWu9957jyOOOALA1RWtk+rWp5dNur9gWQ0EAoFAIBAI5CxdkQ2gy2lubnYzmoULFwJwww03OGuQWpR0pjd16tQMi6o/O9KZgH9OS1bM7kItQUuXLnVBITpDhZTP3YoVK9LeNzc3u3vX+/b9gXR2ohaV/v37u+hOv8z6nRtuuAFol59ir6A+qDp7HzNmDGDvW2Wk/nZjx451MtXzVbb5tvHFzJkzXWDV0KFDAWsZ0SCZeGaDkpISJwdlwIABzuIYp6SkhI0bNwK4etJGq2rOsMMOOzBv3jwg00fTR62nc+fOddH/2q62bt3aA3fadejKyOrVq50/sloyioqKXHnUSub3cSoj7QNqa2vd+fEAmsLCQmc91T52y5YtPPDAA0DKFzyQG8StW/fdd59rG6NHjwbsGKJ1Rc/3/2t90r6yubnZvfbrjJ6vdcUPUtSgUF35bK9PY0+TLfBZRBL1gltuuQVIZc+YO3cuYINUtW9R//eddtrJ+WVqgPCUKVO6ughdiohk9Uv1V3JVr2psbHQ6WXzl9+mnn+brX/86kLKa7rLLLvz5z39Ou357svQEy2ogEAgEAoFAIGfJSctqU1OTm9moP9m///1vN0tUXzy1DMyaNYvvfve7QMq/MWl2pBGxBQUFWa1O3cmTTz4J2PvXWarOWpubm51l6He/+x2QSikzevRolxtSP2tubnazErWsavleeeUVl5JILXMNDQ3utzRFVq5bVuPP7+OPP3av1Yqq0dBbt2511tZ4loRcn+HHmTp1qvM5Uz/BPfbYw1mKtd5rRG5JSYl7zmoJ27p1q7OkqD+r5uGElHXx0ksv7daydBcTJkxws3l9vrriUlJS4vzElLKyMmc1ULmoNSlfUD/t1atXuzru94X6mfYH2t79Vaa4v6L/mb8CpZZ6bWf19fUub2+wrOYG2VYHv/GNb7jnpqsxFRUVbryIW1h9f+ekCP74ZwUFBa6uaF2rqKhg+vTpQGoFQPvmxsbGFmNJchnNed7Y2Oj8UdUPWH3kTzzxRJfqTa2pL730kjtPLYu6Sjx+/PieufkOkG2s9OuYvvatolpHNDPN9OnT3cqo1tPLL7/cpctrLWd0EjlZg3xn2//85z+ADZbRDlf/f/nLXwZsrtKf/vSnAHzhC18AYNddd3UBRC+++GLatfbee2+nDOhA3hP84x//AOzD1jJoI966dau7F1W8Z82aBVj3gZNOOgmA6667DoCJEyc6hVcrg6Y5+slPfsI111wDpDqm2tpaN5hr4vj33nsPSOVpzTXiFVobSVNTk1vGbsv3/IE5X/jxj38MpJaQxowZ4xRSfY464fKVLi3r0KFD3eu4crZp0ya3VJdvCpsyatSoDNcYHTiHDx/uUvZo+UaNGuXanNKTbb8rUAW8sbGR1atXA6T1iTpR0SXJHXbYAbCuEFpXfFcqHWxU4dWAiAcffNCdp+2surrauRcEcoO4kqrBKxUVFc6NQ4NMKyoqMtw9FH+JtyX8NE7621r/ysvLXf1Tpe7b3/524n3mCtkUpa1bt7q0U6pwDxw40I3B6qKliteZZ57pDAF6zV122YVXXnkFsIY2SBkScllZbSntmAY4q5L+8ccfO9dGPaZjzqBBg5zsNJBTdbOOEtwAAoFAIBAIBAI5S05ZVn2LmM5G1JQ+YMAAN7NXi6D+33333d1sRZfCn3vuOe69914gZb2cOnUqYHe6UevttGnTurdQHup8PXr0aDeb9dML6QxE0d2n+vXr55Yjfv/73wPw9a9/nQcffBBIzWbUmvTKK6+kWWzBzpR0tqTuFLrdYq5aVvVZqox0ZtrU1OTKonJMSruj/9UCnS/4y2bPPvssAP/zP//jjquVTC0kNTU1zhKm8qipqXFuJXGLYnNzM4cffng3lqD7GT58uCu/Pnt/eWrixIlAyqrc3NycsZNXXC65jlqq9t13X/72t78BqdRj5557Lrvsskvi97Zu3ercPvS/nwpNLfW6vH/JJZew++67AzgLbnl5OR988EGXlynQdWh/DmSkrfMtZUnuUW0JQvW/Fw++amhocPVJVzC1vuaqG1Y8WFnvs7q62vWd2r7mzJnjVjU19aOOz5Ba1VTWrl3r3LTUPU13EPviF7/IpEmTur5AXUBcJrrxyhlnnOFWWXR5f/78+W4V5+233wbsZi1grc7xhP+trXC2FvQeLKuBQCAQCAQCgZylVy2rLc3mzj//fCDlrA0pK2E8Vc0zzzzjLLA6O9ptt93Ycccd086/+uqrAfjggw9ckFFPoL5g6nNYWFjoZhG+JUxnYsr8+fMBW06Vg1rYjDEZliV/Zq2BWBqYVVhY6GSjVrinn34asA7iuUg89VRSWpWkz9QqqZaztvpk5Qp+MII+x3HjxrF48WIgZWHWGW5BQYH7TMvcr18/FyQTl4emm8lnhg4d6vzx1KKoMjDGOIuqUlxcnGENiiWiznnUL7+goIADDjgASK2mVFVVOTloOdVfd/DgwW4bTe0zfOuYruioFWn8+PHOcqu+j4MHD05MDZbLtJSGJ25NaykIqLm5ucX97v30Yf41exrt1+vr6xNTOcYDrOI+30CaL2pcfr7FS/tmbXN1dXXOQq9br+Z6AG/SVqpg5ahl1wDvGTNmcO2117b52h9//DFVVVUAbrMa7W/q6upcsLBuCpQrxP2Z1e/9pptuatO9qo5TW1vrrMff+ta3AOtLH7fc+iujrQXh9aqy2lKj1p1ZVEnr06ePMytrY9Nl4rKyMre8pdd85plnnJO0Vjx1Au7EbkYd4re//S2QWoLr27dvxjK9v7e3Kt5aodevX+/KrGUoLi52HYUGlqiZ/q677nKRw36ghL7Wa6lzdK7iO+9DqmL7HanfgcbrU74Nri1hjEnLZgGppb7+/fu7OqB1wlfE4ssq8SWrfESd94GMKH9/ed9XRnSA1XqkfUy+oMuOTzzxhJtsaxDmiSee6IIqVfnU6OPq6uqMYMOGhgZXR7Q+zZgxA7D1SbNEaBuqrKx0blXar8Yn17lGtvElaaeepIFS5XnRRRe5SX8S7ckV2R2oe5lOTgcOHOiW5PUZ19bWprlRQbpimpRBJR6o6hsGVF762YYNG1xdyZfI/2z1o3///m7nO/0PqfE7vrsgZAb1rlq1yvUvOmnUoNZVq1bx4YcfArmnrGZj8ODBGcafpHqvk+h77rnHlf+pp54C4Gc/+1nGWOS/b02BD24AgUAgEAgEAoGcJWenQGpx9K1pahlUq4pq4EuWLEnLKajf05mQHlMtfvny5T1RBMfee+8NpKyiCxcudNYPLeeOO+7o7nOPPfZIu18/OEpnNw0NDRnL3iqrAQMGuKApDUrzrZHqFH3kkUd2fWG7kHgAjL+E4NeLOGo90pm+yj2fiKcQGTlypEtdpMe0fAUFBRl5e2tra117UUuA7uylu1ZBSlb5Yg3x8fcqjxMPIiksLMywBuVb2q5zzjkHsM9K27Cm53vggQe48MIL085Xy0dpaWmG5ayoqCgjtZn2FRUVFa4P0r72gAMOcEGsuW5RjRO3eiXV9dtvv53XXnsNgLvvvhtI1a+hQ4dy7LHHAnDHHXdkfFdXNTQ/9nnnndeVt98q+hz9vjC+CmOMyXCr8nd4jC/PZlu90vP9NIJg65p+t6fH1+4gSR6Kvm4pJde6deucC01c3tXV1XnX3/qrEb5FNT5+nHDCCYBtQ1puXeHxA4EVDcz6wQ9+4FKB3XbbbYn3ECyrgUAgEAgEAoGcJScCrOIzlerqaucjpNajkpISN4PVz9She9OmTc7KqpbK+vp6N7NRR+ddd90VsBYE9QvtbKLatnDaaael/d+wYQPvv/8+AH/5y18AmxpDLRZ6nxoUUV9f32Kanbgcy8rKnOV28uTJQMrpPV/YsGFDRloqndllk4XOguPO/1u3bnWWx5ascbnM2LFjM5Lfq1/ymDFj3MxW/X4qKyvdZ+q3FrfE5zvZfM6S/O1EJOuOV/mC7rX9xBNPOH9z9YP72te+5hKTawCd1peGhga3yuQH02g9iKdC27x5s/Op0wToH374oUv2rkFd+j8X8S1B8Xry/vvvO+upBqXOmjWLcePGAamVBw1gXLJkCQ8//HDW37rzzjsBeOGFF7qwBG1Hk89rv+DXf237ffr0cZbzuK+hiGQEG/mrV3F/Z78O+QHCGlyj467KQ630+USSb2U8PaCS5AO9ZcsWbr75ZgAOO+wwIJUarl+/fhkWxlwnW18bDzzUslZWVroYGl3BeuKJJ1zaTO3LlA0bNrSqowTLaiAQCAQCgUAgZ8mJbADx6MS77rrLZQHQ2VpNTY07rjPEpUuXAnamqJHRai3wrQnqq/eDH/wAgNdee61Xt+CsrKx0GxSolXj27NlOHloWLWdjY2PGDMYYk5GKR79XXFzsLInqL5tvlJaWOtkkzerin/n7nytarwYOHJi3FlWlvLw8Y7bv+1PFfVYrKytddLD6rylqgcl3sqW+8/3zfH9ulY3+V0tkvqAbg5SXlztf0j333BOwm0doirykFYi4ldGXXdw/b9ttt3VWoClTpgCw/fbbO6vIzjvv3NVFaxdxf+76+vqMNGR+/6AWnnPPPRew44ta1TU13NSpU53vrq7OaSqwFStWuFSKytq1a7nrrrsAu90mpLaxfvnll126op4gnsavsLAwMVJbj+sx7TP8VIpJ9UPxxyfdYEP7lsbGxpoWUrMAACAASURBVIzte3Wr6CQ/396iI3vSx4n76/qfKYMHD3YrD7qKe8oppwA20X6+jMtJ8vLrWzY5jh49ms2bNwOp7Vn9jWiGDRsGpOriAQcc4NpiNnpVWdUBJd7RTJo0ySkq2oE0NTW5CqGDjCoggwYNSgwW0NQJ2smqmfnss892nXxP4qfY0TLrw+7fv3+G0t5SaoyW8AcpdSWA7Dt25CIi0uH8qHGlPx+JT06KiorcxE3rjp96SZ+zHqupqXEdgiqt+bbs3RotKavxJX9fgdUOUvO05gu6m0xTUxPLli0DUgFQ5eXlrlzxwI6ioqLEtq/HVTnT769du9a5BqgysmLFCqf06a5WumzeU8SVMiUpX66f3kv7fXWzmjhxopvEqLtUVVWVW5rVdqJKxrbbbuvyzl522WWAXVZXdy3tZ1T5U/eBnkKft9LU1OSepe8ClK0/bW0nt7hLwcaNG1270olvRUWFu46fLivX6MoxLynASoP0PvvZz7qgvIceegiAxx57DLAyU50k12lJXi3lHn799dedC6IaHu+8807nkvmLX/wCSPUvBx98cKv3EtwAAoFAIBAIBAI5S5daVv20UZCeGkNner42ni3Q49BDD3WzRX9XDkUtTGopqa2tzZhdFxUVZeySoKl/dAmjp0lK/aA7RAwYMCCrpTkpYCQJ/Z4vK7+sbUm5kSv4VoCk9CFtOeaXN750mOvE77eqqipjowcNpoJUm1Ar2aZNmzLqkV5T3Wcgv4Ot4hY2v7+Jn+Nb6rX+55tlVctSVlbmnpta8bZu3ZrRFvwAxXh9MsZknK/9RnNzM0OGDEn77fXr17v+SYNfe9qy6qchi/PHP/4RSAWsrlmzxlmvdCcdlZmfyq6l/e61Tak1CFJuVffdd5/77KKLLgLgz3/+M2ADHjX9jqb76k5+85vfAKlxxXeL0yXYwYMHt7hjZEtoPdL+pLCwMGODnn79+rm+R63y//znP4GWl4vzkaQ97HXjH5X3qaeeyq233gqkUmxOnz4dsP1Ovu2eB5mru42NjRmrwHpOaWmp65uS6t3FF18MpPqeo48+utXfz4+ROxAIBAKBQCDwqaTLzCq+T2lbrTW6N736Fj3zzDOAnZnpbERncCKSkWpFZzh1dXXOPyYpLY1aDPSze++9N83Zt6fxt7dTK1lpaakrQ3wPZ39m6vttxa2JfqqmbAmd84Xa2tqsQSG+pTTJDyspbVF8O9JcJ24BHjp0KBMnTgRSqYn8rXrVWqQz9jFjxriyqmVIHdhXrFjRzXff/bz33ntpqXog3bqe5N8YT92lgZf5gm859gPpwPooxy2lvjUrqS3FA2L85PLq76x1qLCw0B3XwIme5JVXXuHf//43AO+++y6Q8olcuXKluyf13R41apTzR9Xy6XtIjSF+onyVn/a7+r5Pnz5ODpqOafjw4S4AVpOZ60YsW7du5frrrwdSFrfu5IMPPgBSwbp1dXWubYwZM8bdU0ctq4ofoKXyVln5qeG0Tx47dqw79kkivjJzwQUXuHqkW1nfc8897LjjjkBKRroikQtWVV9/8OtFPAi1JQoKCjKeraYCPeCAA5yPro/WS60jWj/jKzlJdJmymqQUqUl85cqVvPfee+41WIVRP9NGpsLr27evW+LUnVrKysrcQ9eBWb+3detWtzyjjWju3LlO4LoUrp3zvHnzuqDEHcd/wH7laClit60Dsl4rHlAR/91cx8920FJEYmvXUFoLIsh15s6d61xGtIHrANq/f39X7zUIpry83HWK8X3N16xZ44IUtXNtbm7OGxcJsJHxmg9Ty6nKO6Qvgcc/035DA4Wee+65vInOBatgaVvQACu/7Iq/rB3f5ai5uTnDLcif+KmMkvaD72jgY0e4+uqrATteaHaXeMBPQ0ODM0ToserqalefVTFVRdaXnyq8xhin1Kpc9Pdqa2vdIKtjSWFhoZso6Lii5/eUMq+TTv1dHfC3bt2a4XbnZ8OI96eFhYWJGULiOz/6S7yq+OszKCsrc+XWeuK7G/UGScv17f2ulrm+vt7VI83KcfbZZwN2kqIBj5dffjmQPlZp0JVOKvbaa692309HSDJy+TpGZw1Z/njxjW98A0jldf+///s/d8x3QdJ+SCd67cnVnD+jUyAQCAQCgUDgU0eXWVaff/55l45AU+WolaegoMBp1zq7LSwsdA648Vl8nz59nKVDc9ntvvvubjlTLUp+gIQGT2kqhFGjRrnZtlodVJvPxcCKlStXOtkk7ZDRHkticXGxe92b+WQ7Q2vWmyTrc5IDuF4rn+TgWzl1xv7222+7gBYNtNLVh/Hjx7u6rbP3ysrKtMAQn379+rl0PmeccQaQP4FnyhNPPJGx2uDP4JOs8fHlcQ1++ctf/pIXltWklRG17vn1O25BKyoqyii7f624xdQY46x12if5aeB6MiXR8ccfD9j+/9lnnwXgrbfeAnC7bG3evNm1CV1988usqwjq9uGvPKlFsaGhIaOP1SDfvn37pgUXgZWfjkN6DbW8lZaW8tWvfrUrit8ic+fOTXuv91ZfX+8sq3qP69evz0iXmJQisS2rb6WlpW5s1bpWXV2dsXNgb69m+ZbD+LNtrZzx1Yby8nJnyb7iiisAmDZtGmBdQ3RHtCTi+eS1nnQ3bQ3M1vzAN954o7MWa3Ch4o9J/m6Q5513HpDS+e69996M6/tjS9xVSVcLofX0nPk1QgUCgUAgEAgEPlV02rKqs4XTTz/d+cbpTF21aD/YSWfoffr0ydgfV/1gPvzwQ8455xx3HljrhwaI6MxNZzY77LAD77//PpCyNhUXF2f4ael9qZ9eb5E0c/BngTpT9y0kSf6b8ZmIfq+0tDTDupjtd3MVY0zWtBhJO/P4r+MzemOMq1u6T3Eu489E1Un9M5/5jJvRahnUsjRy5Eg3O1aZjRo1yq02aLCMto3KykpnJdB2o8EA+cK8efNce/aDZJQky7zWC5Wjrug899xz3XqvPUFtbW2Gb2Hc4gzJ/vJ6XOWxceNGZ1nVevHaa6+lJZjvKfS3Jk2alLHHvI4lixcvZuHChUBq1WzlypVp/qiQbnnXAF5d3Rs8eLCzIqtfqr4vLy/PsIbV19dnyEF9Rvv27dsjfW18lyo/9kPvTVc3m5qaXHuJpzHz/Zf1mkltyU+9qGOwnr9hwwZ3Xi6mw2vP8/B9Pf1x+YILLgBScTTav+rqbzZUzmrZ7+4AKz8wW39bn4laQm+44Qbn764sXryY+++/H0gFMSr+apU++2XLljmL8sMPP5x2fk1NjdPd/PqmKyAq33322cd9J1hWA4FAIBAIBAJ5S6enQDfffDNgrTzqU6f+cxod6CcvVyvIpk2bXDSvpv7Q2fywYcM48cQTgVRi4cMPP5zFixenXf/ll18G4Mknn8yI9PXTd7jCRrOL+vp65wuYK9uelZaWZkTn+jOSuKW0uLg4wxdG3/szW51Z5xsNDQ2JqTX0fVtmyioHEcnJrf/ags7eJ0+enJG83fcjjPvk+nXGnwmDtczGrbP5ZlldsmSJ89dMmpHH24RPPNp79erVTpbaf+QiagWsrq7OWD2oqalxVi7tP5IygfgrNPE641tO9fqaJu2ll15ysunJbABq3dyyZYvbtjHeHwwaNIj9998fICP9H2TWhebmZnee77uqFik9pvEP69aty0jV5K/caUyEPp+ioiKXsUO3ZO0O9ttvv7T3Wr6CgoIMK2pxcXFG1h1/TNGyqDWsoaEhw2/TH3vi2Wb883MFv1/QcVAzCa1atcrVmThJfcYvf/lLJ1Ptk/2NIRS/TcV9wXsqVV7c4u7zyiuvAFYO8ZXIbbbZxvl3P/jggwBpKT7jcjn22GM55JBDgHTfUyBj1VzRDCy62t6eWIFOK6u6pD5q1CjXoLVRaEe3efNm18g16GPQoEGuQev3dFAtKytzFf/rX/86YBu9LvGo8qu/U1FRkdFRl5SUJC6n639Nm5UrympSQ29pEPYV0qT0FPF0KvHr5DqNjY0ZO6K19f7jy1B+wFm+oBMzdX2pra11AR/aIWqd8Z+x74LjK7OQcuxfvXq1myCqY3y+oMtI69atc32PltNf1vQHbkh3pdHzv/zlLwPw97//3U18czHQKp5P1hiT4c7S0NCQUe/1fF+RSFrejSu3IuLqmObKbGhoyMhD2pP07ds3zZ3Mp6amxt2TlqW6ujpjlyWlqakpcTe/eD+jyufIkSMzAlaTlDl937dvX7dU3J3861//Snuvk42SkhLXrtUFqKSkxJUhnke2sLAwUZGN70TpK7fxICo/z3quKK3+ePH2228D6RP2+I5bSai71HPPPecmMfHAtqTfTJoo9lQqL81fv3TpUo466iggpVvphA9S7i466e/Tp49rY6effjpAYj76I444AoD58+c7t4G2ou54STJvbYwObgCBQCAQCAQCgZyl05ZVXcovKChwVkpdptfZXUVFhUuFoP8bGxszdk7xl190VqeO8G+//bazLKnFVmcEtbW17rpqXSguLnav/eU+sDMKTdR74IEHdlYEXULS0lqSJTFp9hE35/sJf5OShecDvgtHfLba1pQofkCA1sl8QS0AWubGxkYnE20nWr99y5FaHouKijKW77bffnvABlXpMZ3prl+/nkGDBnVfgbqIV1991b2Ot2+/fqiMVGaFhYVp9QFSQQRNTU0u0XcuWlbj6aYaGhqcZVxpampKtOpAcoBmQUFB1h2vioqK3GqXvwtPfIUqV0gK1tWx4ZPMo48+mvZe20Npaal7fn/5y18AOO6441w/oeOoPs+SkpK0zQMgORBP21Rtba3rN9QV4cMPP3QuG3HWrFnjLLxdTUtBOf6xjrbr7373u4DdMe+hhx5q9fykFQyVnwbBdjeavvCUU07h/PPPB1LPXC3F/fr1c/2gjjUrVqzI6BN++tOfAnDyySfzs5/9DLBulwAHHXSQ08/ailp2ddXCp7WV02BZDQQCgUAgEAjkLJ22rE6ZMgWwvqW6xZb666jTbVlZmXNWV0uH72cUTybsb4envg3Dhw/P2P5Nv1dRUZHhL1tRUeFmeurLozOJxYsXd9tMry1km0G0ZDGMWzN8S2xSaqfe2BqxK6mvr88IGGurRSfuj1ZcXMyiRYuA9m3v1pvE066Vl5c7K7m2Gz9RubYNf8tDbQs6m9Z9m59++mnnC6u/s2HDhrywrKp1Y8iQIVkDiqqrqzN8DKurq52fp9YnXWkpLCzkzTff7KESdBx/BSXuE+lvp6n1w/c5TLK2xtPz+O1LLWcTJ05018gW8BjoHXRlUq1U2j/4z1pjPn784x+7jUC0j9Dt0IcPH57h3560EqFjeGFhoUsjpr6NTz31VGK6J4AHHnjAWSi7mpascUnbc0+fPh2wfaKmx/zOd76T8d0LL7wQSFmvzzjjjA4Hy2k71FWv7mbmzJkA/O///q/z1dXf1rqx7bbbuuepwWdDhgzJSPd22WWXuf+6eq2rGL/61a/cb8bToWVDfyvJCt/ad7ssIdq5557rFNff//73QCpIZOjQoWk568AWThtIfLnS70j1s/r6erfcl5RXUV/r9Tdv3uwaowpBB6fJkyczY8aMrip6u0mKZi8pKUkLlPHxdwDzMwXEH66vtCbt951PAVb+fvbxZUp/B5qkHYqSdvDRHIj5ggYR6uRu6NChbucerSfqIF9fX+8UMH9SqJM/jV7VXXUqKircdbUTy5cdvnTSsXnzZtee4y5Dq1evdtGshx12GGA7WB3MdUlM2bp1K/Pnz+/+m+8kfvvW4FSltLTUDSaqvPhKQzw63J8YxwPPamtrXT3y3Q3iRoJA76L1QZXPbMvwAJdeeimXXnpp4rHa2lp3DX/pXK+vk+LWclTHA7i0/3nwwQe7TVmdM2eOu0et9zrp1mCh0tLStOBtgIULF3L55ZcDdjkbUsHis2bN4g9/+AOAyxiQTXbZSBqXejrTyNixY5k3bx6Qcp3Ufn/NmjXuvrQ/rKury9AR1J3Gv3fNz+or7y3pFn5ufZ0Ex42FtbW17tlkI7gBBAKBQCAQCARylk5bVn1rlprY9f/s2bMBa3XVtFOqWRtjnEXE389Zj+ksRzX2UaNGOc1bZwJJS9w6CywvL3f3dvDBBwMwYcIEIDeDKCBz+dpf1m9p/3MlaQenfHUDKCsrc/Uink82m+VYl6viOQCrq6vdzDJf0OBEfZaDBw9O240GUu429fX1bgas1oQklxJtN5WVlU6mev6qVavYeeedu6UsXYlaSufMmZMWLATpwYRx62lRUVFG/kHtb8rKyro1H2ZniQdHQWaAQl1dXUaOUbXOFxUVJS71K1pX1HLm5zTVPreuri5tlSvQ+/z1r38FUvuxaxBp0qpbS5SVlbVq1WqNsWPHpgVUQyog64tf/GKnrt0SqlcsWbLE5QhVK7G2g8rKSlf/NQh8xowZTJ48GYDHH38cSO1k9+abb7qdldT6WlJS0uFczLps/pWvfKWdpescP//5z7njjjuAVBCVtv/+/fu79q7l8VPW+btgga1TKld1J9HPoeUlfL/P0ToRt6y2JWg6WFYDgUAgEAgEAjlLpy2rLWnU06ZNA3B+E5BK37Bu3TpnDVq+fDmA88MqKSnJ2BHhk0SSf8eIESPcPu1+Ynf9H9/gwPcpSkphpOSrz+rUqVPdxg1qUfRn/74/KiSXTa1DBQUFeWE19FErifpg+875OjvVVYTGxkZn1VC/xS1btrjP9L/6e/pW+bjfW66jvm/f+973XBnUV9X30Yz3S0OGDHH1SOWmG5RUVVW5QJFcRNuwH1AXt5AeddRRrjxaB+LJ6v3P/HRW8f3DBw4c6ILxlOLi4sTVjEDvoRZM3YVOVwyrqqoSg4bi+Kt18biA+GtIH4/i/cchhxzCDTfcAKT85tVHXlMedQcaTJSEriwsX77cxa+ormGMcXJTi6q2n+nTpzv5+ZsGddTnVC2rV1xxBYBLJ9Xd7Lrrru45aaDYL37xCwD+85//uPK2lX333ReAAw44oF3f8/tilXU8QLQtukmwrAYCgUAgEAgEcpYuywbQVnbZZZe0/wCTJk3q6dvIOTZu3OhmpGoh1Zmhvz1gkvU0nqZq1KhRLmJcrWnQ9vQSuUB5eTknnHACkEpCrHsrb9myJW2faiW+Fa1uFTlt2rQWt9TLRdTKron81ZoKqeeoPpplZWXOqqL+RI2NjW7Di7i/88aNG508xo0bB7R/ttzbvPHGG87nTPEtH+q/pqxevTpjP3i1Jj/22GMZ0fW5hLZl/zmqlVj5+c9/3q33ICJp9SeQO8QjvTdv3uwsiMqWLVsytqv1LaXtJd7XTpkyJSMjyQ9/+MN2X7cr0RWX9iau72p0HOpNeRxyyCFp/wG3cqlbTb/xxhsuzaFaotXiOXLkSK699tq0axpj2lR3/H5ZNxmIr3TqqlFLSBfkzMv3pHtdvTbeqjySUledffbZzoFbl3d8xTSeZkJEMpbCteIUFxe7AWXq1KlAKjClDfS4PBK/lCAjZf369S5tkQbsiYhLqaH/k9wGOuAK0SvyUGXcTzmkz1cnIKpgLVu2zCm1PUBO1A8f3atbd6GaPXs2V155JYDLJ3v22Wc7BfZb3/oWkAoE7SQ9Jo8zzzwTsMqrLrFqu/b78e5w9zn33HNdKkKdRB566KFJp+Zc/ehlul0e+uxvueUWwKZu0nqv7hyNjY2uL+kK4i4m9957LyeffDKQUk5uvvlmAL785S/7Xw31I53u8M37RMok901sgUAgEAgEAoFPLV1hWQ0EAoFAIBAIBLqFYFkNBAKBQCAQCOQsQVkNBAKBQCAQCOQsQVkNBAKBQCAQCOQsQVkNBAKBQCAQCOQsQVkNBAKBQCAQCOQsQVkNBAKBQCAQCOQsQVkNBAKBQCAQCOQsPaqsisgSETkoy7F9ReTdnryf3kBEjIiMb++xVq45U0Se6fzdfbJoqzxFZGx0bo9vP9yTBHmkaKkvynVCHxJoC2G8bTut1X8ReURETuzJewqk0yZlVUSqvb9mEanx3h/XFTdijJlrjNm5pXOyNT4ROVZEbu/JQVZE5ojIBhEpbf3s/ERE9heR5a2f2e7r7iMiz4nIJhFZLyLPisjuXf07+cKnXR6f1vKHPqRd1+n2MShXCONt99HRvsYYc6gx5uYWrpu3kz0R+Y6IvBTVr1WRYr5PJ685R0RO7qp7hDYqq8aYfvoHLAUO9z77W1feUBJtaAxfBR7u7vtQRGQssC92D96v9dTvfhIQkQHAQ8CfgEHASOBXQF1v3ldv8WmXRz6XvzODdOhD2kdbx6BcUJw6ew9hvO0euquvyYU611FE5EzgKuA3wDBgO+Aa4IjevK9EjDHt+gOWAAe1cHwItkJsBNYDc4EC77v/DbwBbALuAsqiY/sDy2O/87Po3DrgDqAZqAGqgZ9G5xUAa6LfXYrt/Kujv72i4+cBHwJrgVuAgdF3x0bnfw9YCawC/rsNMvgF8CxwBfBQ7NhNwJ+BfwGbgReAHbzjBhgfvd4HWAbsn3CsFPh9VKY1wLVAnyz3MzO6n6sjuS4ADvSOjwAeiJ7HQuC73rFSbGVdGf1dFX3WN5J1syfPEe2tLwn3+gVgY5ZjOwCzgY+Bj4C/ARWxOpFYf6LjZ0fPcCVwUkyeXwVeBaoimV/gfU/rQVFnyxfk0aXlnwk8E7WDDcBi4FDv+EDgr1EZVwAXAYXtkN1B0esJ0bWPjd4fBryG7cOeAybHvuf3Sx2SEaEP6Uyd8Z/d/sDy6JmsBm7Ndj9+nYpdz5fZdODtSO4r8MaDnqgXLZU1y/FP/HibI33NHODkWFu5Etu/3APUAk2RHBJ/I9f+sP1nNXB0luMttaPKqN6ti+T1EDAqOnZxJIva6PpXd8n9dkPjuQTbKRZHf/sC4n33RWzHNwh4Bzi1hcbzGjCaqINN+m1gT+D5WGMo8o6fhO1cxwH9gHuBW2Pn34HtWHeNhJ+1fNH3FgKnAZ8HGoBh3rGbogo8FSjCDpJ3escNMB44BDvITI0fi15fiR0cBgH9gQeBS1poaI3ATyKZfwvbOQ2Kjj+NnS2VAVOiMk6Ljl0IzAO2AYZiO+FfJz2TLmogAyL53AwcClR6x8YDB2MbydDovq+K1Yls9ecQbCc6KXqWt8fkuX/0fAuAydG5R2arNz3192mXRyvln4ltX98FCoHvYztN7U/uA66LyrdNJItT2iG7g4DdsIPuYdHnn8MOsntEv3lidG6p9720fqmD5Q59SMdlt4R0ZbUR+G30rPu0cj8zaVlZXQXsG72uBHbryXrRUlmzHP/Ej7c50tfMIV1ZbQR+hG2ffZLqVa7/YfuPRrL08620o8HAN4FybN9yN/BP77tOXl12vx0oYGuN50LgfqLGn/DdGd773wHXRq+TGs9Jrf028Gvg/Oh1UuN5AjjNe79zVCmLvPN3id3TX1so3z7R94dE7xcAP/GO3wTc4L2fDizw3hvg59iZ56TYtXUQEmAL6daUvYDFWe5ppt+wos9eBI7Hdj5NQH/v2CXATdHrRcB079hXgCVJz6QLG8mESE7Lo8byAN5g7Z13JPBqG+vPjcCl3rGd8AahhGtfBVyZrd705N+nXR7Zyh/V64XeeeXRfW0bHa/DUwyAY4Ens/xGkux+Ff3m/t7nfyHqkL3P3gX28753UkfK6V0v9CGdk98S0pXVetJXFFq6n5m0rKwuBU4BBsTO6fZ60VpZsxz/RI+33SDPdvc10fs5pCurSxPaT74pq8cBq1s4nrUdJZw7BdjgvXfy6qq/TmUDEJHtfGfw6OPLsDOrWSLygYicE/vaau/1VuzsKxvL2nAb02nZf2YEtlNXPsQ2nGFZfufD6DvZOBGYZYz5KHp/e/SZT2tlPAP4uzHmrSy/MRTbWF4WkY0ishF4NPo8GytMVEsitBwjgPXGmM2xYyOj10nyaan8ncYY844xZqYxZhTW8jcCuEpEhonInSKyQkSqgNuwy00+2WQ7gszn6BCRPUTkSRFZJyKbgFMTrt0rfNrlka380eHV3nlbo5f9gDFYS9Iqr41ch7UC0EbZnQo8Z4yZ4302BjhLrxlddzTpbaIt/VJLhD6ka1lnjKn13nfmfr6JHVM+FJGnRGSv6POeqBct8ikdb7uUDvY1SXTrs+4hPgaGtOBzm7UdiUi5iFwnIh9G/evTQIWIFHbXzXZKWTXGLDXpzuAYYzYbY84yxozDBg6cKSIHdvQnWnovItsCw4FXspwP1lowxnu/HXZGtcb7bHTs+MqkmxGRPsAxwH4islpEVmOXzT4rIp9tuShpHA0cKSKnZzn+EdZXaKIxpiL6G6gyzsJIEZGEcqwEBolI/9ixFdHrJPlo+ZPk2aUYYxZgZ7qTsE7eBtjVGDMAmIG1ELWFVWQ+R5/bsbPo0caYgdils7Zeu8f4tMsjVv6WWIa1rA7x2sgAY8zE6HhbZHcqsJ2IXBm77sXeNSuMMeXGmDv82+xY6UIf0k3Ef6Ol+9mCVeIBN4akLmTMf4wxR2AnPf8E/h4d6tZ60RY+beNtd9OOvibx6628zweex/ahR2Y53lI7OgtrNd8j6l+/FH2u/UeXy6PL86yKyGEiMj7q9DZhl4+au+jya7C+MMqhwKOeNWBd9Fv+OXcAPxGR7UWkH3YQu8sY0+idc340U5gI/BfWET2JI7Hl+QzW7D0Fu6wwFzihHeVYCRwInC4i348fNMY0A9cDV4qIWopGishXWrjmNsCPRaRYRI6O7uthY8wyrK/JJSJSJiKTgf+HtTSBlc95IjJURIZgAz/02BpgsIgMbEfZWkREdhGRs0RkVPR+NHb5dh7W96Ua2CQiI7EBQm3l78BMEfmMiJQDv4wd74+1DtWKyFTgO50tS1fwaZdHK+XPijFmFTALuFxEBohIgYjsICL7Rae0RXabsX5bXxKRS6PPrgdOjSzPIiJ9ReSrMUWtM4Q+pPtp6X5eByaKyBQRKQMu0C+JKhPiJQAAIABJREFUSImIHCciA40xDdjgQx27urtedIhP+HjbpXS0r2kja4BRIlLSBdfqEYwxm7Bt488icmT0TIpF5FAR+R0tt6P+2MnwRhEZROb4Eq87naY7NgXYEXgcO1A8D1xjjHmyi659CVZ4G0Xkv4ml0IhM9xcDz0bn7In13bsVa6ZejI1Q+1Hsuk9hl1KeAH5vjJmV5fdPBP4vmuGu1j9sBO1x0o4UFsaYpdjB5hxJzkf2s+ie5ok1sz+Onclk4wWs7D/CyuAoY8zH0bFjsf5CK7FBKb80xjweHbsIeAkbBfomdtZ8UXSPC7AV9oNInl2xXLMZG6TwgohswXYUb2Fnar/CBrxswkZC39vWixpjHsEu58zGym127JTTgAtFZDO20f2d3ODTLo+Wyt8aJwAl2OjtDcA/sJYfaKPsjDEbsYFYh4rIr40xL2GDLK6OrrkQ64/WVYQ+pPtp6X7ew/p5Pg68j40A9zkeWBLJ61SsXx89UC86yid5vO1qOtPXtMZsYD6wWkQ+au3kXMEYczlwJjaDwzrsCsIPsasKWdsRdmzpg+0r5mFdjHz+ABwlNo/0H7viXjXSLe+IOvXVwDhjTFUHrzEW26CKYzO/QCAQCAQChPE20Pv06HarXcwgbFRihxpOIBAIBAKBNhHG20CvkreW1a4gzPQCgUAgEOh+wngb6AyfamU1EAgEAoFAIJDb5LMbQCAQCAQCgUDgE05QVgOBQCAQCAQCOUub06S0QLv8CLZs2QLA+eefz3PPPQfACSfY9IKnnXZau3747rvvBuCGG27g0EMPBeCMM85o1zXo+kTo3epX8e677wLw6KOPMmjQIADKysoA2HvvvQEYOXJk8pcj1PVDJLHoeSWPHiDII50gj3SCPNLJGXn85z//AeCWW24BYPDgwQD079+foiI79H30kc0yJCJst53dN+O1114DYO3atQCsW7eOJ5/scDaonJFHnI8/tlnJBg4c6OTR7puJxhJjDAUFbbJ9dbs8mpttmlm9n+bm5ox7q6+vZ+nSpQDMnz8fgD322AOAbbdN2ycigw8/tJs6vf322wAccsghiWNp/D6y0B0bsbSrjiTdZ3W13SBNZTN//nwmT54MQGlpKQCrVq1i2DC7MdlnP5u+n4kxJpt+0RaSFZMu8Flt0wVOPfVUAJ566inACkgLqgIZOtTuBDh69Gh23HFHwDYkgPXr1zvltr6+HoCqKhuYOHz4cKcEjxo1CoDrr7+ecePalJM2ZzuTJKVy2rRpALz44os0Nlof9bq6urTvnXzyybz++usAbN1qd4370pe+xOWXXw5Anz59AGhqagKgsDBth7SclUcvEeSRTpBHOkEe6eSMPC677DIAHn7YpgbVfnTx4sVuMFZltbKy0o01FRUVAAwZYnfnXbhwIYsXL+7obeSEPIwxPPbYYwD8/e82pbIq4GvWrKG21u5Wq+P0q6++6pSYd955B4BddtkFsMYhVVzivwFZjSBKt8vDV6AhXQk75ZRTADtmqtK1Zo3dXEv1CRGhoaEBgM997nMA1NTUOIVeldT+/e1+EOPGjWPjxo0AfO1rXwPgm9/8pvvNVpTWXldWfdQYtnmz3VlZn/3LL7/MvvvuC9i2ArbtqKFMJ3pTpkzp6E/79J6yOnv2bH77298CqdltVVWVe4jaUNatWwdY66vObr7whS8Adpas52lnosru2rVrnQC10gwYMID77ruvLfefE51JEkmV/DOf+QxgK5Mq7SUldtMMLXtjY6OTVXFxMQANDQ386Ec2N/Mf/2hz9NbU1AAp5TUiZ+XRSwR5pBPkkU6QRzo5I48LLrgAwFnQ1q9f7/5r36rU1dW5z+LK6ty5c52hZOzYse29jV6Rh1r/jjnmGMCOqZs2bQJS44muzDU0NLjzR4+2O6H6yrmOwarA1NfXu/Hle9/7HgDnnHNO6gZ7eeUuadz8+c9/DsCiRYsAGDFihBs/1Vij8lm1ahVf//rXAfj+9+3mcHvttZfTN/r27Quk6kdjY6Mrs9axPffck5/85CdAVqOQkjPK6qJFi1iyZAkAY8bYXVbvvdfupbJ48WKOO+44INUGFi1a5PQ51T1UedU600ESZRJ8VgOBQCAQCAQCOUtX+Ky2yr///W+njeuSdXFxsTO16wxFzezGGDcbUReBPn360K9fPyBlfl+xYgUA5eXlbmajbgBVVVU884zdSW+fffbpvsJ1I/4MUWeBaiXo27evcwNQFwiVT2VlpZsZq9XVGONmekobfYwCgUAg73jvvfeA1IqdLv1v2bLF9ZnbbLMNYK1jOh6pBVHHlMbGRp5++mmgQ5bVXmHmzJkAbNiwAbBjgloEtd9Xy2dlZSXbb789kHKLOPDAAxkwYACQWh7X8cX3R1QXiwceeMBZnzvhq9glxC2rH3zwAW+99RaQshzX1dW5+9TzNNajrq7OjbMaF1NeXu70FJWL6igFBQXuGiNG2N2E33zzzQyLaisW1l5n48aNznqsLhKqT916661upXr69OkAHHTQQUyYMAFIrXKrZbampia+YttpgrYSCAQCgUAgEMhZesSyunLlSjcb8S2rOtPQz9QK2K9fP2dJVAoLC90MT4OGysvL3fk6s9FrikjeWlZ1Ru/7Vc2ePRtIWVH79++f4XelMtuyZYuTqVoLJk+e7L67evVqIBX1mBQtGQgEAvmMWgnVUqr936ZNm5y/po4Xxhh3XC1u2oc2NjY6C2U+cP3117ugIbUGNjU1ZfTxOs40NDS4MVWtYf4YErcIFhYWOt9EDYquqqrinnvuAdKDi3qDeGaDJ554wpVdy1lWVuZWJhWtJ8OHD3fW+AcffBCwgUNqmddYD71mcXGxG4t9mc6dOxeA/fffP+1YrqD3/MEHHwB25UEzYagFWq3NCxcudPqZ6hkrV6501nS1RKtP8KhRozj22GPd664gaCiBQCAQCAQCgZylWy2rqrlXVVW5tCD6X6MJIWX90xlcdXW1m/WoNt/Q0OCup8f0fUNDg5vpKSLifJbyDZ3Za9khlTNQraEVFRUuzYSer5ZmnRVCyr/miCOOYNasWQB8/vOfT7tWrs34AoFA96A+eNdeey0TJ04ErH8i2D7ik4RGdw8fPhxIWcLefvttZyn1x414P6j9qTHGpSvKB6655hpXFh1TIZUZJl5OEcn4rKioyH1XxyG1WJaWlmaMwYWFhdx6661A71tW47z99tuufGo9LykpyViZVMtxfX2989lUP13/M9VXtD6VlZW5uqZ6jYg4P1m1rHY0l213oRZVtYqWl5czfvx4AN544w0Apk6dClhdQf1R1WI8depUXnzxRSBlidXUmoWFhTz77LMA7LTTTkAqDVhH6VbpaZBPc3OzM52r8lRZWZnh0K4Ps76+3lUuffjGGNfYtKH4KTK04mgHA6kArHwjKfVHPCl1RUUFBx98MJCqdHr+unXrXL4zNevX19e7TkTTUii56vDdHpYsWcLy5cuB/HP7CAR6innz5gG2z9UJ8J/+9CcATj/9dACuuuqqxO/qQH/RRRcBqfyU1113neubc4W6ujrnNqZKuSpdIuJS7egYUV1d7cYmHUN0CX3YsGGsWrWq526+C1BFTJds6+rq0pRvSA+wjQcbiYgbj/WYr6Cqkq9uA83NzU6ZWblyJZAKNuptFi1a5MqiOkdNTY0rg9YLLbsxxpXVD/rWcVI/0/91dXXuu6rnGGPSjEa5iLYBDTLcuHGje9Zf/vKXgZS+9uCDD7rPdBIzbdo0JwOVl6bu6tu3r5O1tp0dd9zRKf8dIbgBBAKBQCAQCARylm61rKpGXVpamjZrAWvdUw1dU1HpserqajdT952YVVPX2ZzOdEpKStxSj87+q6urXcJaneGoM3iuo3Lxlw3UeqoO4vPmzXNBAio3dbHYf//9nZVRnZx/85vfuGu1caeRvECXNc8//3wOOeQQIJXUe9KkSe261m233QbYZQtd/ggE8pWkVDm6NDdw4EC3oqXLlH/4wx8AOP74452rkLJx40Z3nm7TqX3RiSeeyH777dc9hegg69evd+OK9vs6DtTX17t71z6wpqaGL37xi0BqzFG5lZWV5YWr1EknnQTYsmj5li1bBlgLmbp96XK2Wl39lcmkVbZ46iVIWSM1WPejjz5y8tZdKnXs6S1UX+jXr5+zsut9r1ixwi1dq4U17mYIpAV6a8BZkox0BVgti6NHj3Zjdi5SU1PjyqZ6Rt++fZ1lWNuKlmvMmDFOLjo2jhw50qUWVX1NZdjc3JzhKrJ8+XK3C1pHCJbVQCAQCAQCgUDO0q2WVZ2BDx8+3Dkga3Ll4447zvm0qAVWZy59+vRJCy4Cq/3rZ3F/km222cb5YukMccKECW42tWDBAiB/LKvxmdvcuXNZu3YtkLIWfvzxx2kJnyE1G9p2221ZuHAhgEvam8/EkzyvWLGCH//4x+412P2Z1SlctwDUtBo+mn7kxhtvdKltdDap/jS54msVpyWLuG6hu9tuuwHpbULrve7nrelIWuOSSy4BrM+f7nkdyB/8eqIWH40j2GWXXVx/q35pGlzxhS98gaOOOgpI7fl9xRVXuMTxaqHT/lVXsHKJDRs2ZPgiqqWtpKTEWQk1cGrEiBEu0EQT/+sK3oABA3LOJzcJ3U571qxZruzatzU0NLhVRx03dZzx64m+9oOutOy6qUBZWZnzf9UgopqaGvddHeN727KqesXWrVtdHdD+f/369ey8885Aevoyfa/n69iTFISmcunTpw+vvPIKkJJRQ0OD8wnNRT766CNXNrUs19bWutVa7Ru0/mzcuJEbbrgBSG3Tq1Z1SA+EB1vHVP56bM2aNZ2yrHarsqrK0+bNm12AkH728ssv86UvfQlIRZ7p8q0fqacVor6+3impaprWxjdmzBjXeF544QV3jub3ev311wHYd999u6WcXU1cGbntttvSdlMBO0BoZxoPPPN3jjj66KMBOPPMM7niiivSrp8v7gDxTmL9+vUuE4IOLL5ypnVMl/UPOOAAHnroIQC3C0dNTY2rDyeeeCLQfreBnibJPQTg8ccf59vf/jaQUkzvu+8+F1ynbeOaa64BrGK/++67A6nMEBMmTHABEk888QSQ2l+8pqYm75VVYwyPP/44YMsPsMMOO7hjud4GOoIfQHP77bcDqT62ubnZ1SM1Kmg92XnnnXnkkUeAVB87YcIEt3yshgddal6+fHnOtZ3a2lqnOCg6kFZXV7vgKX3uFRUVbtDWdqBKeFFRUUbe71xEo62XL1/ugml1kjJu3DinjKuS5kezx3OO+gFZekzlWVVV5RQWHWMHDRrkdkjUvqW3UQXSf3ZaZxsbGzOCrlQeBQUFif2BfhY3JhUWFrprqYK67bbbuvqj9SmXdj+rra11epS6A1VUVDj5qCy0zZeXl3P//fcDKbehsWPHur4gvuRfU1PjlFUN9vaV244Q3AACgUAgEAgEAjlLt1pWTz75ZAAOPvhgN+PQ5cobb7zRLc/HU0jU19dnLN0YY9xxnfHpjODFF190gTZqPVyxYgXXXnstkJod5DrZ9g6eNWuWm6XpsvfWrVvTlrV81FICNlhCr6l5FHWGlEvWJH+5xf8PmfLYdddd3cxeHbwrKyudlVDLr8tio0aN4rOf/SwAZ511FmCtqBqUp/jW61xZ9vNdIHT2/s477wCp4LLly5e7PbrVJaS4uNgt4er3/FzHGnih6YuGDh3qzjvmmGOA1DJaLuYrbs0aqsENF154IWCtABr4cfjhhwN2tQHa3g6uvvpqwFoK8i092sUXXwyk6kBVVVVaWh5ITxOowScqm/79+7slQW2P2v/MmzfPBTfmCiKSsTe5tqWBAwdm5E2trKx0bkA77rgjkMo/WVtb64KH8gXdTUr5zne+41acdIlXx92CgoKMsaegoCBjNU/rS58+fVwA76OPPtqdxegUmloNUjqArhQMGDAgbckaSNuFKj4e+WNQPF9tQ0ODu772OzvvvLM7ritcuWRZ3bRpk6vvqkf169fP6RJaR5SamhoOOuggIJVT1U//pX2H/33V0/Sz2traTq3mBstqIBAIBAKBQCBn6ZEtFcaMGcO9996b9tmkSZPcTgjq95KUHsRPhaCzPw0IUAtaU1OTs7Rpwup8JD7bUF/eJUuWuOAGf1asMxydzWngjO+rphsAPPvssxx33HHdePftx7caxvetbo3LLrsMSO2+c//997uZoloUhw0bBliLWFtS66j8e8uqqvXfGONe+zN6tWJceeWVAPzwhz8ErD9a3Pq5Zs0aVx71Q1Kfs+bm5rRk3mBn1erfrM9Cra8bNmxwlpSu2ue5Pfj9QpLlXS0kamV/4IEHMpK4v/nmm87vVgMTtf/J5sv+8ssvA3Daaae5awAceeSROW1ZjVsvFi9e7PzFNDiqtrbWWZR8iyqk716k/n7av+pxSNWT559/vvsK00GMMRl9ir6vq6tzgTbK+PHjXWyDWlbVMrRp06a83zjFf6ba9vW59+3bN2PsLSwsTBt7IT0AKe7j6p+X1EZ7A92n3t/hUn14d9ppJ1cf4mVpbm7O2AjBl098x81+/fq51/pfRJw8NL4iF9CxoKqqylk8Neiyb9++zqc9/gzr6+vd6kKSTqZy8jdH0ABm/WzLli1OZ1Of8fYQLKuBQCAQCAQCgZylWy2rOhtpbm7OiO7fddddnSVMtXc/KW989q6f++frzFctQD5+AuN8mRXHLQGzZs0CrMzi+xI3NjY664BaWzXl0rp165yc1e/q/PPPd9edOXMmADfddFM3lCKdJB+VpO3+1PKj+0s/8sgjzJ49O+t199hjDyDlX/nII49k+B6pBeHuu+/OsKw2NTW5SEaVo/rkrVy50vl+aiRjT+DLJT4r33nnnfnVr34FWH9vSPlfbb/99syYMSPrddVf/LHHHgOsD5Va49XausMOOzifNvX10ll4cXFxt1lWfWtyNotMNguN1u1zzz0XSD337bbbzvmHqUWwf//+PPDAA0AqIl4zQ7zwwgvOJ1zbzYIFC9z1NVm8RvVqup5cQ+uv9hUqtwsvvNBlidBVKb9PjlvVmpqaXD+j1pSGhoa0rSohleptzpw53VKezuBvF6rjhPYH69aty/Bn3WmnndyGCWqFUyv0qlWr0saTfETTjPn4fqrx+IeGhoYM30ylsLAwI9MCZI5fvY1u+1pbW+vavPZppaWlGfqEkvSsk8rmb66g44XKsbm52Y0rubRVr++Dq+OH1o2k2B7fYqrt3m87WkZ/VQZsnIOOGSqTvn37unG+I5bVblVWfcHEH7a/R6zvgAvpSxa+IqvX8Jcv/O/7ZEs/kas0NTU5pVqVTw1GmzJlilNadFmupKQkw+lfK8CiRYsyXCZuuukm5xKgg4umczrssMO6pUw+/tJKnDPOOIMXX3wRSN3vhg0b3PKrplxK4rrrrgPgjjvucOVSpUIdx2+++WY3oTn44IMBG5SkjTQeQFBcXOyWArtTWY0r8lq/X3zxRbdUrUE906ZN41//+hf/n70vD5OiOtd/v559ZRhEFkFAQRZBXFGMxn3fouZe0USjJtH4y6JZjEuIWU1csprEeG/iTeIeY4xRk7hFRQmokKgILoAsAiMwzDAzzNbTPX1+f1S9X5+u7hmGYaa7Gs77PDzTVFVX1zl1znfO934bkGwfN6gMtMrUFiC5Obvgggv0Lzdcv/rVrwAAzz77rAoVCnmONQriwUBvJkNuplpbW9WkxA1kY2MjVqxYASDp8M8gujfeeEPHP8fO0KFDNUCAoNxYv369jg+2uaSkRDd4XOBOP/10AN5mnoI+06KdCxhj0jZgTzzxBABv7jOHKsd8YWFh2vizA664YeMC1t7enpa+ieNi5cqVqgidcsopA9uwnUDQlMs2xeNxnRPEtGnT9DNNleyf4cOH59VakgmNjY06PoImW7vaEGErM8EKVjZ5EmbQ5E+5BiBNgQGSciaYb9X+3NXVlVLRDEiu0yKSMU0a1x/K0zCAfVFeXq6f+S6HDRumYz84dxKJhLaR46irqystbSbR1tam8pNBnXa6rP4gXKqQg4ODg4ODg4ODg4WsBFgB6QnNY7FYCpMFJCnlsrIy3YHbNWd5PeloMiP77bdf2u/lW6Jv21WBQWJke2pqapQ9shnWnlgd2zme2nNZWVlayhKmO2ppacFFF100oO0h+uJsv//+++P+++8HkAxumDhxopppr7/+egDJwCkbHB8zZ85UFo3tpPn2oIMOwowZMwAk04fMmjUrLT2HnSR9zz333JFm9hl2EEKwT379618D8JjT/fffH0AyAfOzzz6rn+fPnw8gyfTZ498u+NBb8Qey9tT6u7u7lQkg+0b2qbKyctCreiUSCU1lZ7OngBfgQnaT1oR4PK7tIivGyjnDhg3T4Dq+x5aWljQXBrKjM2fOVFcJzqnW1lb9TQY3kiF47bXXlOkdTGbVdpGw/wLJeZ1pfrHy2Pe+9z0AXrUqMh+UwyUlJcqUBs28thuWza5RRgXHU2lpqQYnhYVZNcZo+4JpdSKRSJpV6rDDDkurDW9bumwmLh+xZcsWtVpx/eTY7e7uTguOAtIZac6H6urqFMtoWMF2GmP0sz1vMwWJAanVqjhf2Af2Mc7Brq6ujGmaOP7C5ELCd9jd3a3vmnI2Go2mrRm2HOAY4fUVFRUplhfel9+jyZ9uZcOGDXPMqoODg4ODg4ODw66JrDGrQWzYsEFZseBuu62tLS19UCQS6TGJbyQSSQsAyZQGK4wIMqBAMvCJzNa6detUKyZDuHLlSvVrJFtIZjpT6qWOjg5lE44//ngAvfuCDhSoXba3tyszFWQUP/vZz+LBBx8EkGQSb7rpJhxxxBEAkoFBn/3sZwF4/cHSqgwU6uzsxAEHHAAgWe6PzEE0GlV/3cWLF+s9yKaRtea7WL9+vbKWA43eghA4dg866CDV+sk0T58+Xcf4wQcfrM8JpDrGZwomDPb3008/jd/85jcAoMncly9frj7PZAn4vgoKCgaNWWVJ3Ouvvx6XX345AKQFAw0bNkyfjYxOfX29XkdNn/7FkyZNUiaF6b2MMWkBiWQBOI8AYPPmzQA8BiZo8WH5RvoMDzZ2JAXQ448/jq9//esAktYX+vDawRHsv3g8rscoT+3fC/op2uxiMK1VWVmZvoOwoLu7OyVpO5B8z8aYFB9VACk+rJlSGuWTlS6TVTGRSCg7yMBSu/RupiC7YNEIvu+urq4UprGn38wVghazzs5OZVQpI2pqapTxCwYkZgoIt/1eg76/0WhULTlcc7Zs2ZLGTvIeuSw4w37o6urSeWxbDYIFPzgX7Lgg9k1paala4WyWGfBkNuWLfX2mYPi+Imub1eBAXrhwob40NpCNKikpSYtsLSgoSDNfcLB0dHToIsMFP5FIhDYLgD2x7c0LAyK4CaEw6ejo0EFBU+3MmTO131jDnYOvpqZG72sH2rAm+t133z0IrcoM5rp7+umnU7JDAMlJX1lZqQoLTf92NY0rrrgCQKrDPM9NmTIFgCcwuPFiVSbmnQWSApo5NZcsWaI5WtnfHE+TJ0/OSWQrg8zszRMFamlpqW64abpn39pgGz788ENtM6ue0exdV1enFW7oWjF06FCdV5yX7OP29vaUcTSQOO200wB445TtzxRtT0WLMmD16tX6TFyceK6joyMt/19zc7Oe57u1FyJujNkHlZWVaTKLsmjTpk2ag5XvJBtg8MNzzz2nVXEYJLl06VJ1B6GyZkfv811mUuIzBeXZih6QWv+c9+L37PkYFsRiMX3fVHQYlR2NRlVBJaqqqtKCRWwXiLC1b0dhb06CLnYdHR1pZm/bTY/rKPsjGo2GLvLfhi0/Aa9NwU2VvfnkMbtNmeZJkCSzg50pW7nxW758uW6MeR33KPa6lG2w3V1dXZqbm+0ZOXKkyl4qtTaRGHznxcXFuh9hsCXJoCFDhugGnmuYMUY3y/1BeEecg4ODg4ODg4PDbo+sMavBXfnKlStVY6MGbKeNCZr8bZaUu31qMbFYTE1fNJGGxSSRCT0920033QQgadanCXjdunXaZpog58+fr0wK+/aFF14A4PUZWUtbg+zJuXkwTTjU0KZMmaLPy0AaslmbNm1SEzDbvHr1alx99dUAvIpBQPLddnV1KcvF9EXjxo3TCkNkpMkKdXV1aT/w+pqaGq1gRHMpNcy6urpBC7BicNSjjz6KUaNGpTwvx3NbW5u+K77H1tZWvPPOOwCSWj/ZoaeeeiqNHWtra0tj18mWTZkyJa1K1dtvv619FDRXbd68GZ/+9KcBYMD7hb8xZ84czJkzp8/fi8ViKRWJ7HsZY9LGekFBgZ7n350Z82QUBgsvvvgivvvd7wJIviMyM6NHj1bXBL7To48+OqXaDJBsn82A2GwQ55+dGxLwxlww4KampiatSg/7v7W1FbNnzx7I5g8ImDc4yBJ3dXVpKi8blFW8zq70lA8BRUQmeW6nDSJLbLu58Ho7GCgoU+zUkkG3jzCtt3Tv4vMmEgkNAKIlJBqNpjGlNpsaTJ0JpLtY2fON69H06dMBeHOW/cx7kX3NJfjOu7u7U0z2gNcO9lNwvLe1tWkbKXvsKlhcw1gNa9q0aZg1axaAZOXFGTNmaB9zD0DLaF/gmFUHBwcHBwcHB4fQIivMqp3Qn9pafX19mmOzrdnYLCuQmsQ66ENTVFSUVn83zD41QHpg1aJFi9QPjYEj9L2sqanR9DlkBLZt26YBH9SCWK/8lVdeSWFQAK+PGTATxGBqxWQnNmzYoNo4/YdYXWjr1q3aZrJIBx54oKYwIqNK5nTo0KHqw8jAn7q6OtWkg0mbo9Go9jd/p7u7W/uDKTbsOtCDlaqGmveGDRvUl5S+iPTxGTVqlLaFfbVlyxZtT7BqyPe//31l46kZ28EhwdRHb731lrbdDnqzWXggmRJqw4YNuOSSS3ay5ZlBdq+lpUXbalfWAVKLhNjsBsfer9ZGAAAgAElEQVRtMCWbnbqNMMak+aFlqrDHe8Xj8TQmxfZVI8McTIG0s6B/11VXXaVzlz6X/FtYWKjvjdc0NjampdGya3hn8t+n7OH32J+VlZU6FujnmUgk0tLU2AzkRz/60Z1o9cDDjnEI+qTHYrGM6Q7JMJGZ47vNlGYu3zBu3Dhl1zlP2KZYLJa2BtsBVsGApXg83mPapzAguPa1t7drG5habcmSJSlpMYHUwDpebxcMCBZTsH1/+Zu0DD788MMqp/kcYWBWOScKCgq0jdw3RCKRlHRmNgoLC9MKBQBJ2cE5w/YDSXnFeVVcXKz36I/varh3dA4ODg4ODg4ODrs1ssKs2owG2ZNhw4apDxbZFWp+RUVFaTt7Owl40J8kHo9j5cqVKddHIpGMydBzAZvhALxnCzK/1113XZrGy/9v2LBBfVV5bPLkycp8kY1hVoDp06erTwgZleLi4kEtm9kT+G4jkUhKXXIgyZpXVVWp9kVt9O2339brOE6otWXSfGtqatT/hQwufULfffdd/S1+b8iQIcrsUkNmZPy77747aD5qwdKnNuxoXWrh7I9EIqFzx67fDXjjhZot+2P06NEp5SWBJCPW1NSkx2y/NY4Vvieyy2PGjNG+GixUV1fr7+7OYDq5rq6ulDKFNuy5xHFi1/oOlk+NRCJq4eA77uzs1LHC90x2aN26dWptGDlyJABvLnGOknHkvZqamkJTdjYTgoyWMSbjeGYmGfqG2/Mnl+mGdhS2JZNroJ3Rw/ZbJIJWB7u8aLBIQiQSycishmW9DWYuMcboMa4DsVhMx0AmZpXj2faLD5Yb5nyIRCJqESRLWVNTo/KZawnldy7BtaOyslLHd5BhtkHZM2TIEP0u2dnq6mq1DvK7zDi0YcMGtWJyDevq6tI4i/7Ii6xvVvlSW1pa0sx4dmAAX7Rt2uX17CwKzcLCwrQFvKioKK22cTZhjNFnsSunBHH77bcD8Ez3xxxzDABgwYIFAJLPPWTIkBQTFuCZ57iJI37729/qvehSYAdRcPBkE2z7kCFD1FWDx5hipLm5WSd0ML0VkGwzN6G2SZwKTiKRSAsQIqqrq7XvadZct26dLtJBd5QDDzwwJ24kfN8VFRU6mQcr0MshnLAD5Dg+g/moI5GIbibtDQIXH45nu/Z70HQZiUR0DnHB4WJ67LHHavUr5jg2xqRtcrgI7Uw6msEEZQiVNG5ai4qKMirunGtU9KkANjU15TTd0M7ArncfTNFkbziD6bqCn4HUsTnQ7i8DCb5vPm9paamOdXuzyjnEscB9RWNjo44FrjnFxcXaZrpt0U2tpqZG5wLXkI0bN2pOdM5L3j+XoLyIx+O6N+Da29bWltHFktcHlZZEIqHHgkHN9fX1up9joFV7e7sqzf0ZP84NwMHBwcHBwcHBIbTIegUraq0tLS0pATZAqnmTO3abWeV5arzc9ZeWlup1TIS+xx575LSKlYikJZKmlvvBBx/gF7/4BQDgpz/9KQBg9uzZqukdeeSRAJIVc7Zt25bG/tnM3+OPPw4AOOusswAAf//73/WcbQ4KBlhl02xz3nnnqbbG9FFs7+bNm7USFRnFzs5O1Yz53BwTEyZMUG3YNknS7M9zmdhRvoPNmzfr+LETn9u/5+CQbXzzm98E4DETzz//PIDkPOG4TCQSyobYaYWCqXj4t7CwMIVlBTx2iMe+8pWvAACuueaatOe59957AXhuAEH3HTvBeBgRZIns2vaZkvzbgSBAsm+7uroGrSjGYMNO32QX2AFSmVU7kAjw1gQe43X8vp0E30ZY3AA4HsmKNjc3p6RZA7xnDBZHsCuzMe0Z9ytHHHFEGtvK67dt26b3p9vMyJEj1S2N610Y5oldmY7rHgOhmNDfBt95d3d3WmGRlpYWZY3Zr0RlZaUyz5MnTwYAvPTSS3o/7uF2BI5ZdXBwcHBwcHBwCC2yri5SK7GT8pINpWbb3d2dUqcX8LQ7sgnB1CKJREK1HLIQ1BZyiUceeQQAcNlllwFAGlsMJNmSZcuW4ZBDDgHgpdUAgH333ReAV0oxGBCzceNGLU1KRpUIBqcBniYZrO+ebZ9eap/UtPg3m2BbycI6OIQRd9xxhzKXP/vZzwAA99xzDwDPx5TWKFoiqqqqlPkIFgSJRqPql8pzc+fOxY033rjd56As+uCDD3TukKGkD/ymTZvS0gXlGnYifzJB9BkMykGC6d/YFrJGQHrQTphhWxR7Y0BtFjXox2qnriJ4fXFxsY6BMIIsn832sV2vvvoqAG9/wJRmHLO83k6Vx31EZWWlnuc57leWLl2q1rxnn30WgDfnuNexyzSHCZQFRGlpqfZTsCBEQUFBGiNfVFSkc4rjgfuZ6upqlT/0Z7X3JeyTHUHWA6xY4cCOruQgYCSZndeNG9mhQ4emDSp2VElJiW687MoyuXQD+PDDD3HttdcCSL7wTBHmfIHRaBQLFy4E4JkcAKhpvLKyUoOpKHzOPfdcrewUhL1g2BGcHDRELvvHwcGhZyQSCZWRlCP8C0BdBOgqtHTpUs0GQmWei1FJSQm+8IUvAACuv/76Xn8TSHWfueWWWwB4GQPsSGIguQhRyQ4TYrGYyj5uPrmuBOUgQTMviRK7QlEugnQHAlw/MxFAduWmYLaDeDyu61awchmQHsQaJnCtZE7ypqYm3RfQTN/Z2akbJu4jMlXA41gvKCjQcRTMmtLW1qaKDa+prKzUgGL2Y67dI2xEo1HNfsPAyrfffluDwoIKTTwe18+2+yU34MGgzqKiIp1vttKTKRNFX+HcABwcHBwcHBwcHEKLrNs2qJWUlpamMKNActfd1dWlDCKp9AkTJqQ5KNtVbeyck0SmvGHZwuOPP67PTm2Oberu7k5xfAdS88KychXz/h166KHKoDDf6qOPPqq/FUzLYecws/uMFZIcHBzCje2lTjv++ONT/g7Wb37qU58asPtnE7FYLI0tpHy1q9P1JfVXdXV1iktA2JHJDcAYkzE4F0hdJ23Gi2uHnQ6S1+dybd0egkx6SUmJpldjH5SVlSlryndr9xutE6wcaZ/n+LGrIrJPec/Kykrd69hVOHMNMszr1q3DgQceCCCZn33NmjWYOXMmgOQ4sIM22R660TQ0NOgx9glZ2vLycmW47bzPDFzrj6XCMasODg4ODg4ODg6hRda3+vR7sn1A6CtE7aSkpCTNP6S2tladeMkcZkqVYWvAuUjsTlxyySV4+OGHASQrothpM4LBTXaVGZ57//33AXg+ONT0XnjhhbTfCmpsNqPAc3aAGhHUnhwcHBx2FQQT/5P9sQNLbDnMgBm78heQyrrmG8isZvIx5Vpss6kMniktLdW+CbY9EokouxaWdFU2uD8gizh+/Hj13SWz19raquOBx/i+KyoqdK0kO2sn9Od9eSwSiWj7ma6pqKhI12GOQ5ulzRWmT58OwHsWBkOxyME555yjeyy+V84FY0xK6jLA84dmkLtdSQ/w5hjZbO7lzjvvvJQqpTsKx6w6ODg4ODg4ODiEFlmn1JYvXw7A8wnh7pppWPi3q6tLd/v0gVi5cqVGnrGU6OzZswF4WhI1gbCkTikrK8M///lPANAUGb///e8BAH/729/UB7UviYI7Ozs10f+xxx673esnTZqkn8nI7rPPPth///1TrsvXCFcHBweH3lBfX68+c0w/ZBeYIWxmNVgkhGxZLBZLS3oeZtgsJ5/bGKNMV11dHYDUIhN2GU7AYwvtAiq8DvBYuaVLlwJIri+ZytfmClznyJwuWbIEN998M4AkU9jQ0KDjggwpk/c//vjjmsaMbOvy5cu1jeyjk08+GYDXL+wH3rO5uVmT7DP7xEc+8pEBb+uOghkM+BdIZhQB0lNK2anP2BdkRwsKCvS8nY4T8MadXQQJ8LIz7EyZ3qxsVm1z/KGHHgrAqydN8z87iHn7CgoKdELx7yGHHKJChA7BnJTl5eW6gWUwU/B3cwkGSs2dOzflL5DcvK9atUo366zsxVRe9uazL7j22mtx2GGHAUi6WNTW1qqZi3DmfwcHh10RM2bM0PzT3FxQrh533HF6nb1GcO2wUx4B3roUVPTDDFuun3rqqQCAp59+WoNzuWnlBqO1tVU3ICRPIpGIug5wU8v+qays1LXJ3qSGxRWApu7rrrsOADB//nycffbZAPpGZrGK3M6gubkZV199NQDgqKOOAhC+9TZYmcyuBBrMwVpUVKTjgcqeHTDFvRs3r9XV1XoPe2OcKT1eXxGO3ZyDg4ODg4ODg4NDBohLDO/g4ODg4ODg4BBWOGbVwcHBwcHBwcEhtHCbVQcHBwcHBwcHh9DCbVYdHBwcHBwcHBxCC7dZdXBwcHBwcHBwCC3cZtXBwcHBwcHBwSG0cJtVBwcHBwcHBweH0MJtVh0cHBwcHBwcHEILt1l1cHBwcHBwcHAILdxm1cHBwcEh7yEia0TkxB7OHS0i72X7mcICEblUROZb/zciMjGXz+TgsCMY1M2qLzw6RGSbiDSJyAIR+ZyI7NKbZBFptf4l/D7g/z+R6+cLM0TkIhFZ7PfVhyLyDxE5aifv+aKIfGagnjHbsOZRq4hsFZG/icjYXD9XLrC7j49dUaZmQ14aY142xkzeznNk3OyKyIUi8oCIjPc3eTkt8h6QB5tE5PciUpnLZ8on7O4yJIh86Y9sCLizjDFVAMYBuAXAdQDuznShiBRk4XkGHcaYSv4D8AG8PuCx+3ldroVeWJ6BEJGvAPgZgB8AGAFgbwB3Ajgnl88VEpzlj6dRADYB+EWOnyfrcONDsUvJ1L7Ky8FCH2TgGQD+PtjPsYOgPDgYwKEA5ub4eXpFWNYZJ0NSkVf9YYwZtH8A1gA4MXBsFoAEgOkAfg/g1/AEQRuAEwGMBvBnAPUAVgP4UuC7iwG0wFuwf+IfLwVwH4AGAE0AFgEYMZht608fADgWwHp4i8tGAPcCKIE3WOr8fz8DUOJffymA+YH7GQAT/c+nA3gbwDYAGwB8zbruTABv+P2xAMABgWe6DsASAFEAhSHopyEAWgH8Vw/ne+unoQCe9MfMVv/zGP/czQC6AXT69/9lrtu6M2PIeu/L/c9nAHjdnxPrAHw78N1LAKz158Y3M83JfPjnxkfmseAf22Vk6vbGJ4A9/PfXBKARwMsAItZ3v+bLtWYAfwRQ6p87FsD6wO/YMvBBvw87/HHwdf+6iN8ve8DbSBv/fCuA2f75uf4c2wzgHgBD/O+O96+/wh+TH8KS0QPVRwBu9/vEwJLlAF4E8Bn/86Ww1hKkriND/Oeu99sx129Xid/P063vDff7aE///3mzzsDJkLzuj8HujJRJZR3/AMBV8ARrM4CP+JOjHMC/AdwEoBjAPgBWATjF/95CABf7nysBHOF/vhLAE/73CwAcAqA614Mh2AfwBGYcwK3+QCgD8F0ArwDY0xcECwB8z7/+UvS+Wf0QwNHW4DnY/3wQPMF5uN8fn/Kfo8R6pjcAjAVQlus+8p/pVL9vMgq07fTTMADn+++/CsCfADxmffdF+EI7H/8FxlA5gD8AuMcaUzP8+XMAvIX1Y/65ab6wOMqfTz8CEMs0J8P+z42P9LEQOL5LyNSe2med/yGAuwAU+f+OBiDWd1+DtzmvBfAOgM/5545F+mY1RQZm+m0ARwBY6H8ej/QN4eUAVvr9WgngUQD3Bq5/EECFP0/rd3b+IVUejAWwDB7x0d/N6j0A/urPjfEAlgP4tH/u/wDcbH3v8wCe8j/n1ToDJ0Pyuj9y5edUB0+YAMBfjTH/MsYk4E3m4caY7xpjuowxqwD8BsAc/9oYgIkisocxptUY84p1fBi8yddtjPm3MaYli+3ZESQAfMsYEzXGdAD4BIDvGmM2G2PqAXwHwMV9vFcMwDQRqTbGbDXG/Mc/fgWA/zHGvOr3xx/gabZHWN+9wxizzn+GMGAYgC3GmHgP53vsJ2NMgzHmz8aYdmPMNnia3TFZeers4TERaYK3ETkJHpsCY8yLxpi3jDEJY8wSeAsj2/5xAE8YY+YbY7rgbVhMDp59IODGR+/YXWRqDJ4rzDhjTMx4vqj2mL7DGFNnjGmEt9k+sJd79UUGbs8F4BPw2OhVxphWADcAmBMwe3/HGNNmjHkLwO8AXNjL/foKyoP5AObBM+PuMHw3kTkAbjDGbDPGrAHwYyTXoAeQHCsAcJF/DMi/dcbJkFTkVX/karO6FzwTDuCZLolxAEb7gQNN/mS8EZ4vBQB8GsB+AN4VkUUicqZ//F4ATwN4SETqROQ2ESka/Gb0C/XGmE7r/6PhmV6Itf6xvuB8eCbhtSIyT0Rm+8fHAfhqoB/HBu67DuFCA4A9evFt6rGfRKRcRP5HRNaKSAuAlwDU5IO/3g7gY8aYGnjm2S8AmCciI0XkcBF5QUTqRaQZwOfgmSwBr3/0PRtj2uH1cz7CjY/escvJVBHZ2w6+8g/fDo/JfEZEVonI9YGvbbQ+t8NjO3tCX2Tg6eh9s5pp3BUi2b/B39kR+d4bPmaMqTHGjDPG/D94pvn+YA94DHWwDXv5n18AUO7LmfHwNv9/8c/l2zrjZEgq8qo/sr5ZFZHD4E0EptGwteJ1AFb7k5D/qowxpwOAMWaFMeZCeLT0rQAeEZEKX8P+jjFmGoAj4fnRXJK1Ru0YgsxWHbxJT+ztHwM8n7NynhCRkSk3MmaRMeYceP3xGICH/VPr4Jlu7H4sN8Y82Mtz5BoL4WnlH+vhfG/99FUAkwEcboypBvBR/7j4f8PW1n7DZzAehecTdBQ8luNxAGONMUPgmUjZ7g8BjOF3RaQMnjadj3DjowfsqjLVGPOBSQ2+gs/+fdUYsw+AswF8RURO6O9P9PZ/X96OAvCfHq4HMo+7ODx3HGJs4HwdBh5t/t9y69jITBcGsAUeWx1swwbAkzfw1pUL/X9P+kwakH/rjJMhqcir/sjaZlVEqn2t/SEA9/kmkSBeA7BNRK4TkTIRKRCR6b4whoh8UkSG++atJv87CRE5TkRm+Lv6FniTL5GFZg0EHgQwV0SGi8ge8Ey19/nn3gSwv4gcKCKlAL7NL4lIsYh8QkSGGGNi8NrNNv8GwOd8bVhEpEJEzhCRqqy1agdhjGmG1/ZficjHfM2tSEROE5Hb0Hs/VcFjFppEpBbAtwK33wTPpyzv4b/Pc+D5KL8Dr+2NxphOEZkFz0xHPALgLBE5UkSK4Y0fCd4zH+DGRzp2R5kqImeKyEQREXguMd0YuOcKjoPT4PlncuGt93/LvuZBAF8WkQnipY/6AYA/Bkyr3/TH6/4ALoMX+DWg8M20GwB80n/HlwPYtw/f42b0ZhGpEpFxAL6C5NwBPIX4Anhm4Qes43m1zjgZkoq86w8zuA68a/wGbYMnWBbCc9Au8M//HsD3A98Z7XfSRnhRZq8g6Ux+HzyH7lZ4TuUMJLkQwHvwtMtNAO5ACCLcrT5IyQYQOF/qP++H/r874Eew+ue/AU/7XQfgk/Ad4+EFSzzl91ELvGjdo6zvneofa/Lv+ycAVcFnCts/eAJxsf8uNwL4Gzxmp8d+8sfMi/64WA4vOESDDeBF7S73++qOXLexn2OIUcrbACwF8An/3MfhmWe2wYvI/CW8jQu/eym84BtmA9gAPygvH//t7uMDu7hM3Z5sAvBl/5o2eJlVvtnTd+EpZ/f5n49FeoBVMJjqHH+uNMHLKvAIgI8HrvkuvE1rEzzfzAi8RXydf/w+AEP9a8cjNRvARvhZBgajj+Btrlf7z/ZjeL6sfQmwGuo/d73fjpvgZ1iwrl8Jz82kOHA879YZ7OYyJF/7g1GUDg4Ouzh85qcJwCRjzOpcP4+DQ1jh+/FtBLCP6WdgmXg+nqsBFJmeg1gcHBz6gLyteuLg4LB9iMhZvnmnAl7qqrfgMR4ODg49oxYeaxuGDAgODrs93GbVwWHXxjlIJnWeBGCOceYUB4deYbx0Pb/O9XM4ODh4cG4ADg4ODg4ODg4OoYVjVh0cHBwcHBwcHEKLnpLB7gi2S80aY+BlGknFCy+8AABYtWoVAODTn/70Dv3wnXfeCQA44IADcNRRR+3Qdy0MdCqfflHVHR0dKCsrG5AHiMfjKCzs96vNaX9kYvqDY2fDhg148sknAQBbt24FAMRiMRx33HEAkDYW7PHH+2cajz0gFOMjRAhdfzzwgJdN55///CcAYMuWLYjFYgCAlhbP5XCPPfbARz7yEQDAtddeu7M/aSN0/UGw7dXV1RnP//vf/wYAHHLIIQP1k0BI+mPBggWYN28eAOB3v/sdAOALX/gCAODQQw9FTU0NAKC11as1sHbtWtx///0AgMWLFwMAPv/5zwMA5syZgwkTJvT3+Qe9P/oq05555hkAwF133QUAWL9+PQBgwoQJaGtrS7lHd3c3VqxYAQCYMmUKAODrX/86AODoo4/emecPxfgYaCQSXva0SMTj/3ra82TAYKQSHPA+mT9/Piorvfoa3Fts27YNBx10EACgtLR0IH8uY584ZtXBwcHBwcHBwSG0GAif1R1iVsmEnX/++fq5qMir4jd7tlcttLu7WzUUaiyNjY16v40bvYp6mzdvBuBpM9zZv/baazv6/KHR9Lq6ugAk27fXXl7FO/sddXR4VfU6Ozv1c0ODV0GzttYrDT5unF10YocRGmY1qJmSTf3f//1fbevw4cMBeOPkww8/BACceOKJAIDLL798h+7fA0IzPkKCnPRHkLkAgKFDhwIAmpubAQBDhgwBAIwcOVKZooqKCgBAU1OTXsd7dXYmqx73g3EnQjM+yBhznnR3d+tfytZly5YBAFauXInycq/YEeXMl7/8ZQDAxRdfjJ1A1vujsbERb7zxBgAom9rQ0KDtWrfOq/j561/3Hi9FufGZz3wGAPDee+8B8MbO2LFeEaoTTvCKZR1wwAF9ff6cjI81a9YASLLJCxYsQHt7OwAoq8x5UF9fn/Eee+zhVW3mnOB1tbW12n6y1uPHj+/r8+ekP4Lzu7OzE0899RSAJOP87rvvAvD6Z8uWLQCSLGJNTY323xFHHAEA+PjHPw4AmD59uv4O51xBQZ+rjoaaWZ06dSoAbw6NGjUKQFIGt7e3676LLDyRSCS0rwdKpmZls2rj0ksvBeANkEmTJgFIvlguHmPHjsW2bV5Ft9NOOw0AsHDhQjWTv/XWWynfKygo0M669dZbU36nDwjNYnPZZZcBgE4iChVjDEpKSgBAzZv2YGBfUfjU1e1UNb9Q9EcikdBJ8Ze/eKWo77nnHgDeOOHE4WQZNixZRXT1ai+FKBfvmTNnZtzs9BGh6I8QIRT9sXLlSjVf77fffgC8TQXgbb74nrl4tLS06BziQs759sMf/lDv249xEor+uOmmm/D9738fAHRjRdkejUZVVrJ9IqJmPcpdKv/PPPMMjjnmmP4+/6D1RzzupSrlBuLRRx8F4LmPcbPEDWptba22mXKUCkxdXV2am9S4ceNUAaZ5nPK1tbVVFWGemzhxIn7yk58ASC7oPWDQx0ewX958802cffbZKceGDRum/VFcXAwgqeQNHTpUP7P/3nvvPZ0vHE/cwLW3t6uLCdejP/3pT+p+tR3FL+vzJRaLKSFGIujoo4/WcT9r1iwASUVORNI29E1NTbr/4L5l5cqVALz3T1ckfaiQugFkejckyTguAOC2224DAPziF7/Qc5Sl7MtYLKb3IWmUiSzkGCksLNypPnFuAA4ODg4ODg4ODqFF1phVaigMcgCSmh135dR0urq61Hw3YsQIAMDy5cvTtGFqfolEQrUDmsZpwukDQsGMAMBhhx0GIKkpE9FoVLVaBgSMGzdOAyfYdmp+CxYs6O8jACFxA7A1MAbekT3/8MMPcdJJJwFIjpmVK1cqK0BmeeLEiQA88x+1xlwzZ8xx2g/TSFrf2P+n1st2iUiv1+eD2TvTM5I1v/HGGzF//nwASfaU86a4uFjnBE13Y8eOVfnB6yg/5syZg+uvvz7lt21mfzsIhfw47LDD8P777wNItcgAqe+bfZBIJLTfgkzbtGnTNPi1H8haf9CK1tDQgDPOOANAUhZGo9G067mm2O+VYysejyvzqj/s91lJSQn23HNPAMl15a9//ata/S655JLenj/r4+Okk07Cm2++CQAYPXo0AK8N7BuuG5xLra2tqKqqAgC1WK1Zs0YZNN6Dbns2y8YxM3bsWLz66qt9ef6cyo877rgDAPCPf/xDg4PoDvPb3/5Wr2NAHZn0lStX4uCDDwYAHH/88QCSTP3rr7+ORx55BECSmd4B5MQNoCf5Rte5119/HUBSXtTX1+sY4d+6ujpl3clYc7/2ve99TwOeUx6sb+uOY1YdHBwcHBwcHBzyC1ljVuk/yPQyxcXFqv0yUCJTMBV3/3Y6JmpC1O46OjrUH4c79m984xs4//zz+/JooWBGgKRvEH2nyAQMGTJE2VJeU1ZWlpYu4l//+heAZD/2E6HpD+L2228HAGWOpk6dquOCfVBaWqrMARmApUuX6vfJ3gf9u/qAQesP+l6TwSBrfuihhw7wTw4ocjI+qPE/++yzAJLBHwCwadOmlGsTiYQyREyLV11drSw85xXHRENDgwaMPPbYYyn3AbbLwodivowcOTKNOcvEYthBD5SfZJj5/Q8++GBnZMig98dLL70EAOqX+alPfUoZMPrH2cFzdjoh+//2Mbu9lA0208r70vo1cuRILFq0CEDSd7YHZG18cB4cdNBBynBRtpSUlKjsY1tpiVu8eDEmT54MICkf169fr+OCPp1cg2yfcK67DQ0NePHFFwEk16gekJP5cssttwBIrg3GGG0D/Z05rh599FHdk1Amx+Nx/PjHPwaQDDRjP8ZiMb3uyCOPBOBZa/qInPus0t/2vvvuU0sU2dO1a9cC8OYTA7nprzxq1CiMHDkSQLKfKFtjsRhmzpwJIBk/lIlp7QGOWXVwcHBwcHBwcMgvDERRgO0ikUhg4cKFAFL9VKm5Edz1V97c+7kAACAASURBVFRUqA+EfQ/6zvA6+qcmEgnVpHnsV7/6VV+Z1dCAzAa1VTLPxcXFqinzXDweT2MXqS2tXbt2Z9NXhQrvvPMOgCTjPnHiRI3cZMLqaDSqWiHHArXDlpaWlBRXwA5Faw4o+IwPP/wwHn/8cQDJNDh8ny+99BL23ntvAMkI95aWFo1CpWbPNtnXcU5FIhFlzni9HeEanHsiouOP7ArHX319vTKbnIPZxAsvvKD+qeyDRCKhrBG1efpeiohmyKCP/LZt25R9I3PG8TFixAgdY9/73vcAAN/85jf7kzkiZ9i0aZPKiEw+yjxmZ1AJ+qySPTTG4IMPPgAAHYdhApkwZnOIx+PK7Oy///4AvOcOWp7Y9kgkov3BdxyJRNJiBXj91q1blWEim9rW1rYzhQIGBb/85S8BeP6DbDvTudXX1ysLxvlCWbjXXnupnz/TMC1ZskSzbHAs8F62/OAYa2trw9133w0A+Pa3vz1ILewf1q5dq5ku9tlnHwBef5AhJKPKGIeRI0fq++Y4Ofzww9USRgsf08ENHz5c+433OvHEE1OsP2GBveZdeeWVAIC3334bgGfhpnynLOD8GDZsWFo2gIqKCr2OY4vjrqCgQFPGfeUrXwEAnHrqqSmZV3YUWdmsRiIRXWzojL5o0aK09BDsjMLCQjWFk7YfNWpUmnmGHdnZ2anXkWp++OGHB7dRA4zOzk6dUFwgKCxjsZgKGJpuSkpKVEATTN9UV1eXV5tVY0xGEx2F35gxYwAkFyLbzMtNSTwe16CQadOmpdzrsccew1e/+lUASAm0ysVm9YknngAAvPHGG5pq6OWXXwaQTFlWWlqKAw88EEAyCKKoqEgVPgpBmv22bNmirjHcwL777rs6HniMwrasrEznHhedl156Sc08/G0qAm1tbRrclovN6r333qtznxtqICl42QbOl5KSkjTBWlBQoJsbCtHly5cD8BQImv24ac0XcLEA0jep7A9jjPYfzxljUvKwAqlBSXQpCuNm9emnnwaQDLg899xzVTZwHXjnnXf03QfdAOxjtqsHZaud7ozYd999AQBf/OIXAXimbppPOY64ucsVKP+GDRumc4Ljuq6uDq+88gqA5HPy3dbW1mq/cW4ASXc7BvdyTS4oKND+owzatGlTf3KcZwWLFy/W9YLraG1trbaV7aNM3GeffXSPwT4YMWKEjgESafbc4PVUpObNmxdqsuyLX/yiKl5z584F4CklJFDoysG1tKWlRccI50lpaanKDK7D7K9IJKJ9zT5ZtmyZpqE899xzd/iZ84c6cHBwcHBwcHBw2O2QFWbVBhO7n3/++RpsRSaHJvxIJKIaL9khEVEmkWZesgUtLS244YYbACQp53xDY2OjtofaMLUVAGlm29LSUtX8+T1S8XS1yBdkSr30/PPPa+1yanJsZyQS0SAStrW2thZnnXWWfgaSaUfWr1+Pq6++GgDw85//XO+xE+mb+g0ykwUFBVqDnIwEzWxDhgxRcxKTs2/YsEHnzqmnngogmdy+oKAAF1xwAYBkYvf29nZ1m6DrAVnDI488UucVWaGtW7dq/7Jv6T7w8ssvKyuZC8RisbRUOfvuu2/K/LARj8fVpMvviYhWqOE5skIVFRXKwtnMUj5gw4YN+tk2cwOpTGvQxC0i2mayI3aQUdANK0ygtYGWgF/+8pdqMWGhiCOPPDKtH4hMwZXxeFz7gXKBLFFra6umJqJ5eJ999tH167zzzgOQe2aV87ysrCzNLWjKlCnaHpr12d799ttP5RJTXomI3oPBV/xeZWWl9hFlzF577aVJ8sOGpqamlBRlQbB9dDG68sorlb2nnDzssMPUxG1XzgQ8VpV9y35hEYmwgc+3fPlyZU/pzrJ06VK11NjB64BnkWR7uT5UVlZixowZAJLttWUI+5r7tZkzZ+Jb3/oWAMesOjg4ODg4ODg47GLIGrMaTAPz5z//WZkhpgOhT0g0GlUNgDv8RCKBYJotOgZv27ZNNZt8RWNjozKIth8akOwDIHOib36PPiL0wcknBGspL1iwIM2vkizS9OnTNTk3HdtbW1uVKaJPJxmYUaNGKQtNJqC2tlbH5A7Ucd5p8Lk3bNigLB7bQMf9NWvWYMmSJQCSPtgbN27UAAAySiyXaftOkWEaO3aszg/2G7VkIJm8mT60I0aM0OcgQ0Kfpm3btqV8N9vYsmVLmo/c6NGjdVxQg7d9NG25AXj9wv5mHzHAatmyZcqusJ0rVqxQpiXM4DgBkgwimUObVWYf2YFW7C8ycsceeywA4I9//KOOxTCC85sBLnaNciZlf+qpp7RdBOWlza7Z44T34Hn+3XfffXHRRRcBAB566CEAnhWLxzIVIMgFaAkB0oNuW1pa1PJGyxN9fhctWqRphnhNR0eHzgVauOizWFhYmBYXMGzYsJSUk2GCnYKM8qCoqChlXQWS7GAkEtFALMqY1tZWterwffP7BQUFOra4dofVMkG/5fr6erUEMCBu8+bNKjvsvgC8NcD2/+c5vnP2q+3XynuRka+oqNC5m6nE6/aQlc2qndPMfsF8cJr3bed/u4Y1kFpxJZgbL983qkGQNufgAJIThAK0vLw8LZCC5+wglHxApsj8lpaWlAAHILk5GzFihE4ALkgiopt0ThiOuZaWFu2TN954A4BXhSQXm1UuFJs3b9aFgRsDPk+mc4899pjmX+WmiwvM888/rxtzmmUWLVqkOf+Y/5Cb///85z/aZs6pNWvW6GLHRcquhBRUoLIJ29zP97hx40aVHxR49uJDBc52B+B1/Mt2dnZ26vjj3NuwYUNebFa54QCSi7ItNwBvc8F+s+dZMKsKa7v/8Y9/zGguDQuefPJJAEmFq7W1Vd/9n//8ZwCekkfljmOB772kpCQtT6jdL+wPrkvz589XJZPj46CDDlKZzGo/rGiVK9DtacyYMfpsnLe2GxnHAvtvy5Yt+M9//gMgKWOrq6uV/ODGl2bjlpYWlUtU+Nrb2/X3w4ZoNKpzg+92ypQpKleClc0ikYgqxRw7Q4YMSclRDKT2LQkBrkfBamhhAQlCm3wgqVFUVKTvmjKEm+6amhrtJ2aOGDFihI4zrr3ci23btk37ievIBx98oJtgKtk7klPcuQE4ODg4ODg4ODiEFllhVjNVUAHSg4aoqUSjUd3hB90B7OsyOcrnImhmINDV1aWaHdtFLbezs1O1E5orX3vtNXV0DuYMzCZTOBBIJBL6zGT32tvbNeUFtVY6gi9atEjbTE1txowZyjjSKZ7s0LJly5SpZPqm448/PidjhBr3hAkTcPTRRwNIpqyitjt16lR9t9Twr7nmGjz//PMAklosAzw+8pGP6L3YV6effroGSzCw6sILLwTgBWhRwyY7+8orr6SZ8WjqmzJlirIwucB7772n6bdskyvnepBFEhGdC7aVgeMhaJGwXWqIt956S83iYYYdEEamOZgjsaCgICPbGpSVmcylYQRTqrGdDz30EO6//34ASabv6aefTsmzDCQZ+u7ubp1rHB/RaFTHBxlYpjuqrq5WNpeuMn/961/1PAMjc40999wTgNcWvlO2r6KiQlk/jhPKzkmTJilzTMvPHnvsod/luOD3J0+erAwz+ywWi6WNsbCgpaVF36mdP9hOYwUk5e/QoUPTGPeioiLtB/afbe0MWjcpv8MGrifxeFzdyWhBmjx5srKs7Auumy0tLdpeBjG++eabui+hrGHw2ejRo9XKR8tecXGx7vn6I18cs+rg4ODg4ODg4BBaZD11lc1mUROj1kOtxNbQqN3Zia13RV/V1tbWNGdjW1sLaiTt7e2q1QSTXwcdx8MOm9ViUuLhw4cri8z3Ts1u2rRpqsHRr6yxsVHHA1l5MiltbW2q6dqO75mY+cEGNdthw4ap/yyZH77H5uZmfU6yoyeccIKeJwvyox/9CIA3D+69914ASWb1sssuU2aQycLtICKm4mF6mokTJypbQp8katfTpk3rMU3UYIJMb2trq6bWsYMbyHTwufmOOzs701Kh2T6YZOQoW+yKRvzLoJKwg+MJAL72ta8BAH7wgx8AyFzv3o4FYFspd+wiI8GCI2EAxyPHIpnESCSiaeuWLl0KALjzzjuVgaXsJBNaVVWlViuyr+Xl5Tq2yDgxoMb2xfz85z8PwGOLTjrpJAC590/k83KMV1VV6dpgM8dsB60qZL7q6+tTkt8DnlzgPGFfsf9jsZiOD1o8RCSnfu29Ydu2bSo/OObb29uVEQ8WHLELxthziOsUmVXbH54MJK1ewXiLsICxDR0dHfqsfPb169frnKHsZWpDEVFmnezr8OHDdU0iO83v1dTUqBWTY3HLli26p+lPAJpjVh0cHBwcHBwcHEKLrFFLvfmSkvGhJlteXq5ai12KNRjBHBZfoYFAZ2dnStJ7IJVZpTZnl7sMarL8fjBlS9hhjwn6rBpj1NeF/cESsk1NTaqZkRl56623VIPj/ajlDRkyRP2TyAhEo9E0n+lsgP4+jz32mEYrk/GZN28eAM/v55prrgGQZM5uvfVWfd7bb78dQFIj/vnPf65aMlmyhQsXKtv0pS99CUDSd2jjxo3qq8q598QTT6Sl0iLT8Oabb+KII44YqC7oM+hflqk0ru0jx3liM+XBEps2Q2KX9OX3gj6r/O2ww7ainHjiiQCAW265BUCScS4tLU2JdAZSSxxTjjDaG0BOU5X1BDI1tApwLiUSCZUDjC7+1re+hYMOOghAcs7bLDvZWZ6LxWI6RqZOnarHgFTLD5mnuXPnqnxhadpPfvKTAJI+89kCxyqfp7y8XOUj2eeOjg71QSdbxucuKSnRqHc7qpsyMxhJv3z5cp17lEn2msN1PNfrM8d/S0tLSjEWwJvzwexCdhGJILNqp38LWuRKS0uV3aZVj9apsIFjOh6PazuYPaepqUnHULBcb0dHh/YJ50xRUVGaNZf7lMbGRp1jZOaNMcru94dZzVrqqt6CWdgoO09i0DRVVVWVkrYJSA6urVu3aqfmc4BVEBQIdn+wnRQufb1XmGEHhNGMd/DBB+uEoXBlH4wZM0bfLzddnBBActHlxOjs7Eyr7rNixQrdlGUTXAD+8Y9/aD1zBj5R4DU0NGgbWH+8paVFq+dw48i8xBdffDEeffRRAEnhevDBB6vJh+ZNCiUR0d/igt7Q0KDnmYLnd7/7HQCv/4KbuWyAG/WioqK0uvf2ghIUmN3d3WnPKyJpKeE4lzZu3KgLcdA0HnbYm1XOl2CdbiB9g24fY7/YC3NYcofaoMy79NJLASTHNc2SQNI0+53vfEcVMcLeZNj5Zvl/9oNdmSgIVlscPny4msBPOOEEAMngkmxvVoM5cQsKClT2UY5Eo1Htp2DwTzQa1b6kWxCQlFVjxowBkJwTLS0tKSkogVRlkBu1XG9WOQ/s9ZCfm5ubdb6w7cH8qUCqqT+YGtKWMdygczPY3t7er1yigw2+m0QikdHtku/MdpcAvDYHScKOjo60fL62ayddStg38Xhcz/cnGM+5ATg4ODg4ODg4OIQWWU9dZVeyuvvuuwEkNRmacTs6OtLMVoWFhaq1kDEglXzDDTfgrrvuSrk+39Dd3Z3GZtjpR4Jml3g8nqbd2qm/8g1k9ciQbtq0Sc1+weoqq1atUq2Wmt+IESPUhYCMIln57u5urQvP8bFo0aKcMKsMjjr44IP1Wch4MP1ULBZTE90BBxwAwGNDmIKKFavuu+8+vSdN/nYic44ZskFkpoYPH65z6G9/+xsArz74l7/8ZQBJJtuugJSLhN98n5kS1Ntsa3Ae2AFTNoKVndgHNnPM75ENCDtsdofvl8EdDHro6XvBAFd+v6SkJCdM+vbw8ssvA0BaIJTtzkNmcMaMGTj44IMBJN83mcKSkhJts53gnePMLiYCJFPmAUk2fuzYsXo/VjuiJYBuBNlC0B2spqZGrTD2NXxeMmO0qthmfbt6F03n7F+6YdXV1encpLwpLCzUvsxFMGYmBC0GQHJ8dHR0aH8QfN+dnZ3aLq5HNrPI8cd+icViacy0fSxMzCrXiXnz5qnJn+uOXZGK7eb6alcFtC0yZN1tRhlItfzye3aKLwYC7wjyc2fn4ODg4ODg4OCwWyCnqauY1DwYOAUktSJqJbYDPP9SM8qXNDO9obu7W9sVdPiORqOq0ZNdtNnWIAsS1hQivYH+PdRuTz75ZPUBo4ZKLa6mpkZZDNax7+zs1OAAMgC2HyKDC8jG5Kp2MxMwd3R06Lukbx3TT02bNk3Zme9///sAvDrofOa///3vAJLM47p165QJoA/R/fffj3POOQdAsq0MxKiurtaE4GeffTYAr4//8pe/AAAOP/xwAKnBYKwjnU2QbWc/AUiZB5QRwfFvF5nIVJCE84zjatasWcoocBzavnthRqbyqUGGMFPMgF0IgfKCPpoFBQU5CT7cHljXnHKBJRvt9F2UH8ccc4wGQ/E924UigkGoxhh990ErViYW6F//+peWuqVfKGVRtotJ8L2RMbWti3acB9dXWploaZgwYYLOK5ZdnTp1qsoU9jOxZs0aZV0pg+xE+rlO5UVwXNup7OhnbJcrty1OQKrVxp5LZBvZv9yb2GsxLTJFRUU6xhj8FwbY1ic+Ky1pU6dO1XFgM+ZAarl7zqfKykp9/7wvrTlNTU0pPs6Ax7qy7/qTGi+nm1UKmaA5z47OpQApKSnJ6BQPZHbWTSQSeecSwHYEA0daW1ux1157AYBGkNsVr4LCIcx1vXsCa3rbZhe269VXXwXgBSUBXnsZuX7DDTcA8OqZU+hQeeFkPPHEE9U0xYWHG9psgwL06KOP1s0W86AuXrwYgJfxgZtOmhjpPgAkx8fxxx8PwBMGFBpcYGfMmIFZs2YBSG7a7TrOnGtcaFesWKGbVd7r3HPPBeBtaHksm7Cfm23mgjx8+HDtE5rzbWUvU4RvMMsG7z9p0qQUEyCQP3PINrcFN+9B9yAgVf6yjcHqOyKSs/nRG+655x4AyU1h0E0ISG6sfvWrX6nCx3fLdlZVVWUcM/zMBZr9lqka0TXXXJNW0YduB9kGlTq2r7S0VNtApau2tlYDsdgvPNfW1qbzmxsru494f8rTyspKJQZs0zDHYlg2q3Z1rWAkf0dHR4rLGZCcP++//76uxdx/vPHGGxrQyrWEfSUi2vfso+HDh4cyowYzaZSXl2s72O7GxkbdYAaDzeLxeIp8ALzNJ8eI7U4DeBt5zhvK3fb2dh0jHFM7gvzazTk4ODg4ODg4OOxWyGmeVZoiqe1wh19RUZES3AF4u3PeIxhQ0dLSkhK4lY+w2SP2h13Zi2YX5s1ra2tLyzPJ/+ejGwBNJtR2ly5dqvlHX3/9dQDJsVBeXq7MI80WRUVFqskG8+etXLlS02iwagtNeNkGx3x1dbWOVQYW8tmqq6uVRaLWX1tbq1opg6/43g8//HA107MPvvjFLyrDHExTVV9fr1Vsnn/+eQBeuioyQ2RcOKfGjh2bk4AbvmM7bQxNu8OGDVNTEtlWW2YE5UEkEklhWYHkPBk1ahQWLVqUdg+eD2vNcyB1rgeDwmzZGaxqZ6eLIztC83Bra2vK+bCB1iX+tcGgybvuukutBsFUQ6WlpWkuE/ZnjhO+d1aas8EqYWGCbeLlZ86N+vp6dUEiK0ork53Si/Nm9erVahan/CDbFo/H0+ZSQUGBfg5LgC/HfFFRkbK9fG47rRtN15QnJ5xwglq52DfTpk3TfgjmQ6+qqtK5R7Zy3LhxoekHG7ZLHV0+bBN+pnfNv8H9SUlJia7bXMP4va6urjRmvru7W9fr/vRNfu7sHBwcHBwcHBwcdgvklFkNOobbPqvB79nfDQYLtLe3q2bDnXu+IRqNaluDDHJzc7P6x5BZjcViaX1KLSjfKlgBqcmHAY9lJ9NjF4sgOEbsCkU9FYTYsmWLsiz0X8tV7XP68axfv14Dn1h1h+z5+++/r5/Hjx8PwAtqoD/qcccdByDZV1OmTNEAIbIEmzZt0vNMUs50NrW1tSk1wAGPrWWg1+mnnw4g6Xi/ZcsWnHHGGQPVBX2GzYDys12lLOhjaLOpQSaRx+3rKX+GDRuWllC/tLRUE2iH0X+ToN/yfvvtl8ZW2PKEDKzdV+wPjgHKlptvvjknad22h0yBdDzOdtH/+xvf+IZaKvie7RRG7Cu7EEIQZFbXrFmj6X2OOuqogWnMACJYUam5uVn7igVH7rrrLrVU0PrCdbeiokLn1ZtvvgnA819k+ykrOD4KCgqwbNkyAJktmWFJe2Zb6yjv+b6Li4tTmFf7nJ06005iz/6irOC4Ki4uTvORLywsDOU6bFfw4meuQ1OnTtU+4NrB9SqRSCjragfzBqtmsm96SlfK+zlm1cHBwcHBwcHBYZdC1rMB2GCEO9kM7tLttCq2b1GmkouAx7yQWcpXZtVmP4KRyK2trcqwEdXV1erjG4xszcQqhR1ksY488kgAnibLaN8gK5QpXY8dzcu/djQ0tUL6cba1talmHEwOPZiw05EtXLgQQNJ/ls/d1NSkkfh87wsWLFCmiH+pnf7mN79RBoC1mVtbW3HqqacCSDK3t956KwBg2bJl+OxnPwsgGUn9wx/+UP1pOR9ZCMCOls8mMqVNoR9zJh8qjhPb39LW6oOFRsgwDR06VCOkmXGirKxMfa3CzKxeeeWV+nnBggUAkJZJxWah7fRdQR93svJf+tKXsvDkO44g+5nJr5Zy5Nhjj1UWh3KS15eVlfWY9g9Ijg/6iDc2NmYsihGW0t7B57CtdB/96EcBAD/60Y+UQQumuFq+fLkyYyxCsmTJEqxatQpA0nec1qnKysq0kt72mh2WKHjK95qaGpVpZIlLSkrSUthRxpSUlKRlD4hGo3qe97VLjQaZwurq6tBkRbBhPxNZUL77goKCFB/V4Pe4TrLfOjs7VWZQhvCviOhv2es351R/xkjW3QCI7u7utHrcdmqE4LHgd4FU8zAXWqaXyEfYaSKA5CQqLCzUFEZEbW2t0vE04eQrmpubtS0cJ++//74GAvSGoFIDJCcTF6SWlhYdayeeeCIA4JlnntHfzOZmlcKyrKxMN0jB6ienn346jjnmGADJ4LLZs2frGKBw4ffGjx+vgVhsUzQa1XQ0S5cuBQDsv//+ADyzN69nta99991X+40bU87BysrKrNc7B5BSZYhtpYK7YsUKPR8M9rArWPF7IqLjgdfbwVRUCihY7TyEYYa9qHIDQYXddidiu7jAxmIx7Qe+Wyp0/A6QeUMYFmTaLDIH6yGHHKJKfFBJAVJdAoIILtgFBQWaWm/OnDm9/n4uQKXKDgRkoCrXhpaWlrRNp+0KEazcNnbsWO0/blqpVM+cOVOr4jF3M5CUo1yLcw3Oh9ra2rRUW8XFxdr+YCW37u7ujIRY0E2RSlBRUZEqSURNTU3ahj4MsN3muFnlWtDQ0JBGCNmbUG4+OS6qqqrSriNmz56tefTtSl47s1l1bgAODg4ODg4ODg6hRdaY1aD2adfjtnf79v9t2MeCzsyJRCI0pof+oqioSDX5YLBHYWFhGvtXU1OjLBo1PpsVyifU19crY8ZE7/F4XE2+DLSi+QpID7bLZOalJrhx40a9B9OxRKNRvP322wCSbGc2wPY99NBDGkTF4Dma8B944AFN4E2T/+rVq9UUefLJJwNIsq4NDQ3KpBBbt27V5OlkzhgUUVFRoceYlmfJkiVqNmW/UctesWKFVg/KZuCNreWTpSAr9MYbb6iW3luQjB18Egyo4PftWuY249YXZj/XsNtMxiMYoJmpSILdzkyBrblmC3cUtB6wutvEiRPVAkG5ynba1QLtQhFBucn5UFZWFqr67kFQ7nNONzc3K/PJdbGrq0sZd45rMmr77befMoM0644aNUrlACvacQ5u2bJF09xRJq9cuTIlIC0MsNlRznVaD0pKSpQZJfPHvioqKtL3zWNlZWVpqav4vdGjR6sM4RgqKSkJtTueHQDGNbGmpkYtbUErlIhoe5jybOTIkbrucI5xHd+0aZOuZ9y7xONx3dv0ZtnoCfm1q3FwcHBwcHBwcNitkLMAq8LCwrQyq7YPapBdtQMqgkxKJBJRf5x8hV2rmBoP21dVVZXmOzZq1ChlC6kB8/vBVCZhR0NDg5ZPpV/mpEmTtPwntbFgoQj7mO13Rd9P+i9PmjRJ2Utq2EVFRTlJscLnOPnkk1WzJyvEdh5++OFpzu8tLS2q7TJIjP5odhou3n///fdXbTfoQ7Zp0yZlP9h/e++9tzIHZBXoxzl+/HhNa5VN2EELdvAD/x8c53bZ0GDJUTsAhOBYMMaklRwtKSnJWXqz/iKYwsuWl8Ggw0zMapiLH2SCLQfoZ3311VcD8OrdB8vJ2kxbsI/s9Sjo0xyNRrVoBP02mWA/DAi2paGhAWeeeSaA1HdKH3a2iwU2urq6dK6TWYvH48qaUU7ZlgiyZbQOrVq1So+FhYW2y3zy3TPILBqNqmzlvLFlAI/ZqRSDfpn2es1ztG60tbWFch22rY+UryyuUVVVpTIv6PPc0dGRkvYK8PqJ/cN1h2z9+vXrNbCXftCvvvpqWsGeHUHOerOkpCTNUbk301Omc3aDg/Wbw5LrbUcwYcIEAEmzDk0QweAqwKPbadYNRkXn28a9vr5eJwUHe0dHhwrHYPAVkNmMEFyIL730UgDAmWeeiZNOOgmAtykjclH/nYvq8OHD9T0/99xzAJIVpmbNmqUmlJdffhmAZ26jcGVABTMG/Pvf/9ZABwqj0aNHqwChIsCNXiwW0/6m+Wry5Mlq0mNE/AknnADAE9jc3DLfZDZAJcyuQMPFJlNQXKZgO9tdJHieC/nWrVt17LCPiouLVS7lC+zNN4CU7Bgc67arVfAY256P+NnPfgYA6j5jVwTM5GZGWWkru0HYcpXjjW4wYdqsZsqSwgwWdCMaPny4ugnYGUkArw84/rlp7ejo0H7j5tMmQ5i15Z13RZm1JQAAIABJREFU3gEAvPTSSzpfwpJf1K7wyGeijJswYYJutLhecvzbm0zOKTuDBPuR99xnn310w8c8tbbyEybY7i/cWFNpKS8vT2kvkNyst7e3a7vtecExwXOcfySMgKRCk0gkdCzZrld9hXMDcHBwcHBwcHBwCC1yylNTkyFN3FtgkG2qsx3l+T0yUUQYtZresHr1ajXD0smd2gm1WBt77723tp8aLbWVsDi49xVDhgxRxpNtaWpq0vZkCqZiNRUGBtgBZxwLP/3pTwF41WyYT5Qa8HPPPZeT1ETTpk0DkGom+q//+i8AyfH89ttva3AZ/86cORNPPvkkgKRDPFno6upqZXyo4cZiMe1LOr3zXps2bdK5RIZ13bp12qdMqcWArtWrV+OCCy4YsD7oK6i1V1VVKXPMPqqoqFBTlR0kQwQDaOx0eATb29LSom3OlC4rXxC0qNiMYjAHsZ2eh32azynw6PLDNpSUlPSY57K7uzuNdQWS8oVjgPOzqqpKZbFt4g7LGsM2cP0YOnSoznWivr5ejwVTJEYiEWXQ7ByzPBbMObpq1SqMGTMGQHKtiUajKpfIpOUafD/V1dVqpWRKwJdeeklTcgXzrR5yyCEpgVWAJzOXLFmScn/bXYoylkGvdt70MMEO2uZ7JZve1NSkMpRt4xpp52DlufLycmWX6WLCMVBRUaFWxJEjRwIALrjggpQgvR2FY1YdHBwcHBwcHBxCi5wyq8FEusHUKzYyacB2HV5+N9P1+YDJkyerzyo1EWp+TENi44ADDtBUJPTDocZ8yimnDPrzDiQ6OjpUC6M/ZnNzs7KtZNXIbnR3d6uGxsplDQ0N+l0yANQAbX9Mpo4688wzs1oMgAhWodoRXHLJJQP9OHmBqqoqZT/IoD3zzDOq9dNCE0xRBGRmv4LBVKtWrVLGJVPwQL6AFoVgYEckEknzZ29vb9djlJV2lbwwy89MyfiDc7+oqCijrz+vCfovi0iazzvfv804USbbv5/rAgpk9SjPOjo60op4LFu2TMdHcE709K6D17Fv6+vrsd9++wFIMom25eKII47od1sGEnZlR7KcTPs3d+5cZaLZb2RRR44cmWLBAbw+5fX0XaUlb926dfjJT34CAHj++ecBeP0RxhSSTNNoB8nxXW7evFkT+RNkTjs7O5VtJgNdWlqaFtRMWbxlyxa1XB133HEAgCuuuEItgP0p3hS+3nRwcHBwcHBwcHDwkbOiAEDSv44RvkRra2tK2Uj+DTIG1IQyJd8Niz9RX3HmmWdqupG+4JhjjsGLL74IINwlEfuCkpISTcdEf90xY8bggQceSLmOkZZ1dXWq5bKE6FlnnaXjwE7fBHh+qjx33nnn6T0OOeSQQWuTw86DDMbatWuV7aL14ZRTTtFMCcHiAIlEIi11lc1y8DOZ2SOPPFKju+nn29jYmBdFAex2cdzTb4zJu+2iKZw3VVVVaSnvOF+C9w0bgmm44vE4/vSnPwGApst55513dO0IMrF2ajObVeT6Emx7UVGRsk+LFy8GANx0000D3Kr+g4wpWbBYLIbZs2enXENf+YEALXpA0gf0xhtv1LFFy2CuYZeQ5Zjhsz322GM7fL9rr722x3Pcw3BM2r8ZJixfvhyAF79AtpnHfvCDH6gPMi0wdkYFtseeJ7ROMRsT4wjKy8tx+eWXAwAuvvhiAKmpCPmbO4KcblZpvqDTL2n72tpaNeHa6WuCVVhIPXd2dqalrso3dHV19ZqfjosoB0phYWHaJtV2nM+nNDTHHHOMpp7h5Lj99tvTrmOQFP/aYNqnnsC+4pirra3VdFYO4QSVt+eee04FJU1LV111Fa666qoB/82zzz4bgDeXzj///AG//0Ajk1y94oorAEBNeuecc47Oj4ceeghAah5NKgDs27AjuJksLCzEvHnzACSDCNvb21OC64DM7iGZ+s92LwO8zbFNmmzvebINVpNiKruSkpK0YLl4PK7PmcmNYkfQ3d2tcppK3kUXXaRm5Uxua7nA5s2bAXhrJwNrqdgONDiXmDKsubk5lHmLb7vtNgDAE088oQoM5QUA3HnnnQP+m9ysFhUVaZ9QqdwRhFd9dnBwcHBwcHBw2O0hYXakd3BwcHBwcHBw2L3hmFUHBwcHBwcHB4fQwm1WHRwcHBwcHBwcQgu3WXVwcHBwcHBwcAgt3GbVwcHBwcHBwcEhtHCbVQcHBwcHBwcHh9DCbVYdHBwcHBwcHBxCC7dZdXBwcHBwcHBwCC3yerMqIkZEJvbhuvH+tVmr2OUwcBCRS0Vkfi/n/yEin8rmM+USu2t/iMgaETmxh3NHi8h72X4mh10Xu9L6EpQZfW2bw+6JMK4xg7JZFZGjRGSBiDSLSKOI/EtEDhuM38o3+Atuh4hsE5Emv58+JyJ5rTgMBPo7bowxpxlj/tDLfXudeGHFrtIfItJq/Uv445///8RA/IYx5mVjzOTtPEfGza6IXCgiD+TDpiMTLJnSKiJbReRvIjI21881WNjd15fA+94kIr8XkcpcP1cukQ0Zsysin9aYAd8giUg1gCcB/AJALYC9AHwHQHpR5d0XZxljqgCMA3ALgOsA3J3pQhEpyOaD5QqDNW7ybeNB7Er9YYyp5D8AH8Ab/zx2/2D/fh/afAaAvw/2cwwyzvL7dxSATfDGzS4Ht74o+L4PBnAogLk5fp5eMdhyp68yJgzrQRieAci/NWYw2Lz9AMAY86AxptsY02GMecYYs0RE9hWR50WkQUS2iMj9IlLDL/oa49dEZIm/0/+jiJRa568VkQ9FpE5ELrd/VETOEJHXRaRFRNaJyLcHoW0DCmNMszHmcQAXAPiUiEz3teRfi8jfRaQNwHEiMlpE/iwi9SKyWkS+xHuIyCwRWey3e5OI/MQ/Xioi9/l93SQii0RkRI6a2hf0OG54gYj8yGeOVovIadbxF0XkM/7nS33t8Kci0gDgjwDuAjDb17Kbstyu/mK37A8R2UNEnvTHbKOIvCypVocDM8kHETlWRNZb91kjIteJyBIAbSLyIIC9ATzht/vr/nURACcBeArAS/7Xm/xrZotIRETmishaEdksIveIyBD/u2Rir/Bl0oci8rXB76WeYYzpBPAIgGn+M/YqF0XkEr9tDSLyTenF1SIkcOuLBWPMBgD/ADBdAlYBWw70BhEZ4o/ren8szPXHfYk/D6db1w4Xj7Xc0///mSLyhiSthAdY1wbnYNY3aZQL/nNsBPA7v10/899znf+5xL8+jREUy2VCRE4XkbfFs4xusOd72PsiA/JqjRmMzepyAN0i8gcROU1EhlrnBMAPAYwGMBXAWADfDnz/vwGcCmACgAMAXAoAInIqgK/BW1gmAQgK1DYAlwCogceUXCUiHxuwVg0ijDGvAVgP4Gj/0EUAbgZQBWABgCcAvAlP8zkBwDUicop/7c8B/NwYUw1gXwAP+8c/BWAIvD4eBuBzADoGvTH9R2/jBgAOB/AegD0A3AbgbhGRHu51OIBVAEYA+CS8ti/0teyaHr4TNuyu/fFVeHNhOLznvRGAsc5nlA894EJ4sqDGGHMhUhmX2/xrZgFYZYzZAuCj/rEa/5qF/v0vBXAcgH0AVAL4ZeB3joMnk04GcF0uN3siUg5P+X3FP9SjXBSRaQDuBPAJeIzsEHgyJsxw64sF8dw9TgewdSdu8wt4734fAMfAa+dlxpgogEfhzSPivwHMM8ZsFpGDAPwfgCvhrTH/A+Bxbvx82HMwvhPPuDMYCY85HAfgCgDfAHAEgAMBzIQnA/rKTN8N4ErfMjodwPMAkEd9YSOv1pgB36waY1oAHAVvgfkNgHoReVxERhhjVhpjnjXGRI0x9QB+Am9y2LjDGFNnjGmEt0k70D/+3wB+Z4xZaoxpQ0AIGWNeNMa8ZYxJ+JrBgxnuHWbUwZtQAPBXY8y/jDEJADMADDfGfNcY02WMWQWvX+f418YATBSRPYwxrcaYV6zjwwBM9LWmf/vvJpTobdz4l6w1xvzGGNMN4A/wFteemOI6Y8wvjDFxY0yYN+g9Yjfujxi8towzxsR8X1R7s9qTfMiEO4wx67bT5u25AHwCwE+MMauMMa0AbgAwJ8CMfMcY02aMeQvA75C6uGcLj/kMRjO8DdftwHbl4scBPGGMmW+M6QJwE1IVg9DBrS8Kvu/5AOYB+EF/biKem9kcADcYY7YZY9YA+DGAi/1LHkByrQE8IuUB//MVAP7HGPOqv8b8AZ4J+Qjr+r7MwcFGAsC3/HHRAW9Of9cYs9kfJ99Bsr3bQwzANBGpNsZsNcb8xz+eL32hyLc1ZlCCeowx7xhjLjXGjIGnfYwG8DMRGSEiD/n0eQuA++Dt2m1stD63w2My4N9jnXVurf0lETlcRF7wTRnN8Hb2wXuHGXsBaPQ/2+0cB2C0b1po8gXUjUgOmk/Do/PfFc/Uf6Z//F4ATwN4yDd13CYiRYPfjP6jp3Hjn95oXdfuf+wpqGBdD8fzCrt6f4jI3mIFRviHbwewEsAzIrJKRK4PfK0n+ZAJfWn36eh9szoaqbJmLYBCpArtoFwa3YffHWh8zGcwSgF8AcA8ERm5HbmYIlP9cdSQ7QffUbj1BYD/vo0x44wx/w/9t5rtAaAI6WOcDPsLAMr99o+Ht7n/i39uHICvBtamsUgd/2GQPfXGc48hMs3pvs7Z8+HJjLUiMk9EZvvH86UvUpBPa8ygR6AbY94F8Ht4HfEDeLv4GcYzW38SnummL/gQ3ssn9g6cfwDA4wDGGmOGwPOZ6Ou9cwrxou/2gqclA6nsxjoAq33BxH9VxpjTAcAYs8J4Js49AdwK4BERqfBZqe8YY6YBOBLAmfDMO3mBwLjZ4a9v5/95h12xP4wxH5jUwAj47M5XjTH7ADgbwFdE5IT+/kRv/xeRkfDYgv/0cD3gWTzGWf/fG0AcXhATEZRLdf152IGAz+o8CqAbHmvSm1z8EMAYfldEyuBZY/IGbn1RtPl/y61jI/vwvS3w2MLgGN8AeOMJnmvZhf6/J40x2/zr1gG4ObA2lRtjHrTuFQZZE3yGTHOac7YNVh/6MiJ5I2MWGWPOgbfePoak212+9EWPCPsaMxjZAKaIyFdFZIz//7HwBvkr8HwwWwE0i8heAK7dgVs/DOBSEZnm+2V9K3C+CkCjMaZTRGbBM1eEGiJS7TOhDwG4zzcjBvEagG3iOWeXiUiBeIFYh/n3+KSIDDeeywAdmRMicpyIzPDNPC3wBFIiC83qF7YzbnYWmwCMEZHiAbhXVrC79od4QQoTfd+oZnibroEat5vg+eURpwF4ynIzqPd/y77m/7d33tFxVHff/45W2lXFko0s2RhMBxewQ4mxQ0kwJUBoB3zgQIKB8CZPCBDycp5DCeQ8b2JCAg8lJKQQH3LokAChBhwcHAixMdhYIm7EFWwjN1RWWmmlbe8f19/f3LkzclHbWXM//0janV3N3Llzy/fXngbwQ8dxDnJUeqCfAXg25/U5u8NxnHLHcSYAuAoqwCAvOIrzAdQAWIGdj4vPATjXcZxpO/rC/yBcCzAfdn4JZoc5exOAb+6YI66GimHY1ee4GL3TcZwqx3HGAvi/UKo0eQrKD/pyuC4AgDId/9cO1dVxHKfCUYFoVQN0WYPF0wBud1Sw2L5Q7i+83kYAExzHmeyo4Lv/4Yccx4k6jnO54zjDcrlcCmpe5dhUcG1RaHPMYCir7VDOtgsdFc3+HoClUIET/w8q1UYbgNegnLd3i1wu9zqUPP0WlJnwLeOQawH8xHGcdqjO9yeEl1d2nOcGKGfv+6AmOR87BpNvQJlf1kHthGdDOcQDKlhg2Q4z6i8BXLrDZ6QeajKKQ01ab0O5BoSVnfWb/vIWgGUANjuOs30Avm8o+KK2x2EA5kItOhYA+E0ul5s3QN99F9Qk1eqoKF6Pv+oOU9edAP6145gToIImHofKFLAOQBLA9cb3vg01Jv0dwP/mcrm/DdD57gmv7BgD4lDXMDOXyy3DTsbFHe9fD7VZboJq860IdxooO7/0zv+BWqB/DmACVHDu7nA9lKK4Fsq69xRUvwcA5HK5hTveHw2VeYCvL9rxP38NFeC1GjsPeAwLswAsAvARgH9DWVZmAUAul/sPgJ9AjUGr4Fo7ybcArN/hZvJfUAv4Qm2LgppjnFwu1Mq0xWKxDDiOCpDaDODgXB8DD3f48K0DUJILR3Rvv9ihHLcCOCyXy63L9/lYLBYL+cJXTbJYLF9IhgO4o68L1b0Fx3HO3eHCUAHgf6GUpvX5PSuLxWLxYherFovlC8eOtDW/zfd5hIDzoYJLPoNywbg0Z81tFoslZFg3AIvFYrFYLBZLaLHKqsVisVgsFosltAxEfdpCl2YHOlXLHrUHlW1Hq2J27bXXAgCuv14FHY8bNy7ws0uWLAEA/OEPfwAA/OY3v9nDUw1kyNsjl8tJOxQV9b5/mj9fBbd++umniMeVq2F9vUqDt3XrVmzcqErDH3ywyjx0xRU7TyubzWZ3+T+R5/7h+aDRV5JJlef68MMPx5gxKl1mJpMBACxfvlz6z89+9jPf9zi9Vs3bJaFpj+XLlwMA5s6dC8B9XnZ1bd/+9rcBAD//+c8BALW1tX09BSBE7WGSTquYr+LiYqxatQqA+0y8/fbbiEYHJXNZaNujr3zwwQcy1i5btgwA0NTUhH33VTUB/vu/VYasgw46KOjjg9YeplWU/X5Xz/fatWsBAP/85z8BADNnzgw8btGiRQCA5mZVq+aMM87wHcMx1HEcz//Xz8cgNP2jvV2li73tttsAwDOGck7gM7R582aUlKiaOqWlpfIaADz44IOoqupzhqrBSBW3R23Ce5jL5RCJRDzv8RqnTp0qbcLjq6urcffdqmr16aef7vlcJpOR+7+L+TWIwDaxyqrFYrFYLBaLJbQMhLJqGSC4Y6msVBXNbr75Zvn7kENUfudUKgUAWLNmjexcjjjiiKE+1QFBVwZ4LUFqZ1eXqiT4ne98B4BSmrnjGzFCFdxpbGwURZUKLP8+8cQTfSpENpvty44vr5hKBZUBx3FEZWX7nX/++dI2u/qesJPL5fDJJ6o6Iu97dXW1KKJ//vOfAQDvvqtSIqbTaXz+uaoaWl7uFvTh+9dddx0AYPjw4QCAlStXelRIQD1ThdZOhH2A15JMJkVFe+QRlT7zoYcewg9/+EPP8YX2PAwWtFA1NTUBAKqqqqQfff3rXwcAfP7557j33nsBAMcdd5y8NpSYSmaQoplIqMJWS5YsweLFiwG49/vNN98EAPz1r3/FggULAAAbNqiqmaeeeir22WcfABDVcM2aNQCAfffdF1/+8pcBAGPH6oWgCovZs2cDAH7961/3eoypJgIQhZVz8SmnnIIrr7xykM5y8DDHvCBoNVi/fj1isRgAt3+tX79expClS5d6PqcrtLvzf3YHu1gNEewEHBwOPfRQAGqy4UBIM29xcTFGjx7teU1nF6aYUBB0bkGvXXjhhQCUuRsAWltbsf/++8vvAHDxxRfLwDJnzhwArnvE/vvv7xtUI5GIbwEbZuLxOFpaWgC4i/cjjzwSAHDZZZeJSZsDQldXl0ymPT09ANyJqLq6GsOGDfMcH2YaGxtlsuB9HD58uJix33nnHQAQU/ctt9wir3HDMn36dKxevRoAUFdX5/n+I488Uhb+fM4WL14si5BCZ+7cuaiurgbguhRdccUVMtHYRaoXTrwUDcrKynDYYYcBAM4880w57qOPPgIAPP64qrXC/sdjB5vexvhNmzZJ///Pf/4jr/N6uAjlAqukpARnn302AHeRUVxcLAt0ulxt2aIqDC9evBjvv/8+AHeuOv7442UhH+Y5R0ff7AOu6JNIJGQhyjbOZDLSHnQDWLdOpSLmuFwI6H3GHPtfeOEF3HPPPQCA995TRaw4Vo4aNUrGRn7H8OHD0dbWBsBtk9NOOw0AcN5554m4NFBzjB2lLBaLxWKxWCyhJfyyyl5OkFM81cKamhoAavdK9YO7lO7ubtnpbNq0aUjPeTB55ZVXAAALFixAQ0MDAMjujW2wYsUKaYeRI0cCUOrAs8+qkuxUNhgUcfXVV4vJ+Bvf+AYA4Jvf/GaoFQCqGTR7V1RUyDWb5v2TTjoJb72lqkNOmDABgFLj6Q5BNYi730QiIX2HysioUaMG7Vr6Cs2w5eXlcp5UU3t6etDZ2QnAfU6ovL/wws6rbHZ3q2qiNE/p8P9ks1msX78eAHDggQf24yqGHtOsP2/ePFx1lbeac1lZme/6rDuAgm4A7Ccvv/wynn/+eQDAHXfcAUAp9XweaQF7/XVViXSolFVz/OJ4N3fuXDk33lvHccQCR2WM84zjOKKo8mdXV5e8T/MvFdm6ujr5Lh6zcOFCsfjQEhbm8RWAWFoI55nu7m5po4qKCgCqzTgmU3VlG3z88cdDcr4DgX5PzD7d1NTk6zckkUigrKwMgNtH9KAy9o1//etfAJS16/777wcAPPDAAwC8Vom+8MUelSwWi8VisVgsocYqq3kmyO+IfiL0MaqqqpJdK9UgXW3lzlcn7Ltak9/97ncAgEcffRSAUtCo9tGXiLu3RCIh/pqNjY0A1C6XqVVM/9Rt27ZJcAG/f/HixbLzCyP0g2IQUCwW8ymBn332GQBg8uTJoqLqju1UzqhAUiVIpVKyS6Yy0tbWJn6s+YbKxcqVKwEon9Jt27YBcFXl8vJy6f/0PaNPmZl+hd9pPi/8mUqlpI0YqBaLxcRHj21l+rqGFdNHbN26dZg4caLntYsuuggffvghgMJTjgeDoHRP7B8zZszAjBkzALjP5WmnnYajjz4aAHDWWWcBcJ+loYappRgwVV1d7QksJGZgEK0U0WhUrp1qYTablfmH8Pnp7u6WPkZ1beTIkeLrO23aNABuWsGwQn9etpWePpFjrR4PQsuUOY5wHC4kuru7JdCU97K+vl7iG9gWemoythP7TVtbm8zJnFtINBpFR0cHAODSSy8FoObqAw44oM/nbJVVi8VisVgsFktoscpqnglSVrmDYdqRWCwmuznujktKSsSniq/pFJL/WS6Xk7RCupq63377AXCVA+7e6uvrZTfIXdt9990nCgfbj2p0TU2NZE5ghPeqVatEiTATGuebtrY2nzqYTCblNe72eZ3l5eWiAvKaq6urJWk5j+euWU/bxZ1xPB4PjbLKa6GK2tLSIudJX9vRo0fvlvWA115SUiLPGn28qVqXlpZKe/CY5uZmUVLoyxZ2ZdVMEcN0VUHJ6qdPny5ZM8jOxoxCTPW2M8xxV+9LzJpB3+d33nlHVEOOubNmzRILDtWifEWFsziG/uyz35uKKeAqhBwPenp6pM/o8ww/w2vWv1PPSgMo6w3nIUbJh11Z5ThDBVnvC7wWWlr0/sFrpsK4ffv2oTnhAeT++++Xa2PcR09PjydFF+Cqx/p6g23BDCOAt7AAv4sWKSqst99+Ox577LE+n7NdrOaZoAmAHYRmJX2i0M0THGw4kesUkhvAggULZLHKlFSHHHKImFe4KOMD0dzcLA8Cf15++eVi9mYb8e9JkyZJmhmaPOfPny+L4bAtVtPptDz0nIDKysp895QTRltbm/zOBX06nfblYdT7mpnXNkxpvHjfmFu4qalJBryjjjoKgEoXxKCyk08+GYA76KZSKVmQckI64ogjZHHBzQ/N4J9//rmYeRms1dbWJmmv+B1hx5xoGEDB9tGZOHEiXn75Zc9r+hjDCbmQNr17QlBeZ/7OACtulq655hrZCOswNd6XvvQlAPCZzYcK5j/lWKgvPvlaeXm5r39wsZXJZHyL9+LiYhlTOB/pG2gex/Gpq6tLxmm670ydOnXArnEw4KKdbhFsn0gkIq5FfC2bzcpmntfO+x3khhd29LyoXEeUlJR4zP6Au1HJ5XIyr7If6OsSc3zQq1KSFStW9Ouc964RyGKxWCwWi8WyV2GV1RDCHS9NEfoOxXR8BrxyPCkkZXX+/PlSl5m716VLl4qyZu5gZ86ciVtvvRWAqssMqAAtJqrmrpjO/3V1dZg+fToASPLr5cuXexJmh4lIJCJ9gNdSWlrqq0xE5UNXPNhGPT09vp0t/y4pKRG10AwaCANUT5m6qri4WFR2qkgnn3yyKIZUTH/yk58AUGrA9ddfDwD4/ve/D0AFyTz55JMAIIr67bffDkA9S6z5TjeDsrIyaXu6U4Qd9hmycOFCAJCCESa0yDD1DgMZ9bElTP1iIAlyv+K1TpkyBQBEuZ8zZw5uuukmAG6w4hVXXCFVnDg+5UtZpQWOZt2enh4ZG/SgTPNZ1wPxgsYSPdhKp6enR9Q4flcmkxEVlymewg6vlX2AY2cymZS5l1ae+vp6UWA5R/F5K0RlVbcWUT0FvEUQ9L8Bf+Aq+wDg9i3iOI4vSI3uVH1l7xyJLBaLxWKxWCx7BVZZDSHcsVDZGTZsmOyCuYvLZDKy0yskFTWIVatWSRoY+js1NzeLYsAdLIOk/vjHP0oi7ocffhiA8smi0vHBBx8AAG688UYAqvb15ZdfDsBV3yZNmiR+kGEjnU7LNVPJKSoqEh86+omRWCwmfYGqhq7EUv3Qgy0YvKcnwQ8LVC543h9//LFcH/tEaWmp9AEyc+ZMAOp+M3iKAXh1dXWipJrBiu3t7RIcwzKV06ZNE6WRfqyFAv3RqGzoqofeJ9jOTOTN69X9N/dWZXVnY+YFF1zg+anDMfmXv/ylBLCx0Ei+SlxT8WKf7+rqEgsEFbRp06aJwsVnXg/IYx/ha+l02qc86lYeKo60eEybNk2ep7COqya0SFLx49za0dEh/Z/PSDKZ9LURLRNBMSNhh1YrwKug814Hpf8zUyfqRQE4T3Gd4jiOzFNmisG+YherIYSOyHreSEYc8sbX1dXJ4Mia1IXKhg0bJG8qOeaYY6Tjb926FQDwq1/9CoAyy9JtgEycOFEWYJdddhkASLTz/vvvL1U0GAzR2dkZqgWaTk9Pjy9vXSrgqWxZAAAcAUlEQVSV8plo9IAYDi66m4geCQ+45p7S0lJZ1NLErS94g7JL5AMOhvvttx/eeOMNAJD73tLSIps6PidczH/ve9+TgZWBUx0dHXLNbCMOsKtXr5b2vuiii+R/FtoilTzxxBMAgK985SvyWtDic9y4cQCARx55BICq9GYeo3+er++tC1nTxUqfsOke8tvf/haA2hRyA2x+fqgWq2Y/5qIpEol4AgsBtdHn8aYbkf5dQS4Cem5vQLUP+w7bBXDHF55P2MYTEwZkMvhWD+Q1XWocx5HrYzsw9zOrBhYSW7ZskXusm/p31nfNDBCZTMbXX3RXM7YXNwEMbO0re9doY7FYLBaLxWLZq7DKap4J2o1zZ88dXCaTEXMLdzejR4+WnWC+KqcMFLlcTnbh3KFdffXVkueQgVavvvoqAKUG0jyzefNmeY95MJme6qSTTgKgqlbRLEYFN51O429/+9ugXldf0fMYUtUoKiryuX3o6YWoAlHFKCoq8qQYAeDJe8ddr55flGabsCkho0aNEhP+Sy+9BEAFWplVdGh+rKmpkbYi+jPCa2ZASmdnp+QiZbqqQobPybx58+S1IBX0mGOOAQDcfPPNu/xO/fN7m6JKTBPoM888A0CZ/NlXWPf+lltuEbN7vpRmjhE8t9raWgCqr5977rkA3FSAK1asECXRNOfqY0VQCjtdLQO8ad1OPfVU+f7DDz9c/j/gWsQY0Bg2OBe8+OKLANz7l0qlfFaV/fbbDxs3bvS8RqWQ81MhEY/HxZrEe19UVOSz2u0MPcAq6HhTrTfH5D1l7xx1LBaLxWKxWCx7BXlTVhsaGiSYhrs07gKj0aj4nwWRL0f2wUC/FgZ5UAGjP+GWLVskeIj+QM3NzeKjRAdxfVcU5HcVNrjTSiaTUoWF/rfz5s0Tteu+++4DAMyePRsA8PTTT8sul31n7Nix8h3c8TKV1fjx43HCCScAcNto5cqV8lnuEE0/pXzhOI5PaQa8qql5vB4YAQSnoOFOury83Pf9juOEqjBAb1A9X7lypYwRZtGI9vZ2T7sBKq2QqYDR3y6dTnuCBYDgWvH5gsULqKC1t7dLsAf7xLhx43DllVcCgKThop/70qVLRXXjmFFVVSXjhln8IJFIiPJOtT2ZTMr3MRiSFoy9jTvuuAOA68v40EMPiQqtk+95yEwfpVtJGIzKZ769vV1e0601gPJh57xipsfTf+eYEo/H5ZkbP348AFUtjQGP/H5+Z1iZOHEiAH+6pnQ67au+VVNTI3MO24HPyGGHHTYk5zuQpFIpT8EDQPXjoOT+/Gmq747jyGtB846pyOvWO65x9gSrrFosFovFYrFYQku/ldWgFTXRd5yMwuZr11xzDebPnw/AVQa5++/p6ZEVOBXFCy64QCIbzVJ5e7qzDYtiAnjP5d///jcAV0HhjmT79u0SBc30D83NzbKT5Q6bigqVj7CjK4VUaahubtiwAX/6058AAHfeeScA4KmnngIALFq0SHa106ZNAwC8/PLLUpOaUd+s2X3wwQdLOisqAZMnT5b+x+8Kk7LK54nnpJeYpU8Yd66RSESeBV4LfXoB+N4LSnsWjUYLQlnVsxeY6qmO7k8FqD5mqgZUU8NaHIIwTRD9sysqKkRZ1ROSmyVp6bN64IEHyjWyPyUSCVHd2A5vvvkmAOWfxzGZ43Amk8Gnn34KwE3lVcjKqq4YAap/cNygYk//aB09eX6+5xFz7tWTr/M99oWnn35afEo5HvAZiUQikkmFhQ1yuZwvmpvPT0tLi8xH/N+dnZ2+JPB6svkwwjUJ0VVCrkXIiBEjZEw1E+BPmjRpEM9yYKEfMQBfekQ9dZVJUVGRL85BJyizAI/XU6CxAMnkyZP3+Nz7vVjd2YRBnnvuOWkEmrQA4NprrwXgXuB7770HQAVPsGOwcWfPni0P3iWXXAKg75UjcrlcaMzkeucwJxTS3t7uG2BaW1vl3LmIYRDR0Ucfnffr2hOKiorElEI3gJdeeknMNH/5y18AuAPpscceKxMn22PKlCkygHKxTreA1157TVwDePyZZ56JxsZGAP13/B4oeI9zuZwnLRWg0qSYfTYo5YieC9AMGOF36ptBfSAJSzuY6CZ5LqzKy8t9gSK89tLSUrnPPCabzUr78fmiSVyvpR5GuHji4F9fXy/ny81rfX29BLKwXzC9UHFxsYyV/FxJSYksKuhawU1yIpGQduOCNhaLidtE2ALwdsXumuv5fNAdLQg+Lw0NDWhoaAAAcb8Y6kAr032JP+PxuJixed8rKip8+ZnZdyKRiG8B1tXV5RsP9A202Sf17wvLpn9X8Plgf2ZbVVdX+9IaRqNRz8ZeP76QFqucEwH/JsdxHLlG85nR3QB0d5Pejs9kMp7UaITze18Wq9YNwGKxWCwWi8USWvotJyxevBiA2mWY6R642p4xY4aYskg6nfYEdwDA1KlTAagdD5MvU3G75ZZbpDLL73//ewDA+eefD8CtDb676GbWsNDd3S0O3Nwxc3fX1dUlSgd3xy0tLT4zDZMUFwq8lng8LtfKXe4pp5ziUzioFixZskTMNOxD27dvFxMx03zRzFNSUiJtyvQtnZ2dYuqkes9UNPlCN6OZu9i6ujpRvoipvgJecwwVAN1dgMfrlbEAb5BWmOE1lZaWepRiwBtoElTwwbRO6KmvCkUNArwVzHgfW1tbxRrFwMRNmzYBUAqyacqNRqNyPFO4McBz/Pjx2LBhAwC3zfTa76ZCF3Z6U390stmsWG4YPMQUYNXV1eKCwTFjyZIlvmTwQ62s0nSvuyYAytLGPkBqamrk/lJFpZUilUpJ/9erDvE6TLO+Pnfzu/Sk+aRQFHimntItUWxbksvlfNYXs5BNIUAzPOD2F84r8Xjc4z4G7PyZyeVyvbqOZTIZX9Av4M7NfcEqqxaLxWKxWCyW0NJvZfXZZ58FoJz5Tz/9dADwpc247rrrJE0V/QTT6bSsuLlqZ2qU9vZ2+Z3pVJYvXy7+DnSE5yp9+vTpsqvjbm7btm2enSDg+namUinxf2UAV75pamryBMCY0H+Iu6FEIuELkgl7qhCTTz75BIC3L/D6KisrZefPHT6V0xEjRohSRMX+ww8/lP5HhZQBVgsXLhSVlm2l13+msprvnbK+i+U109e2oqJCrtns15FIRPzFeH3Dhg2T3800VbW1tR7FTH8v7Oiqlalk6RaJIKWUx1MN0n3++OxwzMp38IwOlQ+qe7pyzPvX09Mj/Z6Bhjzmq1/9qvxOxSiRSEifoY83/8/WrVtFhdP9fPm7WQq4UDAVViBYBeVzxvLOpaWlvv5QVFTkU5iHuigA76Xpyx6LxXypgSKRiNxf3W+ZPzmW8LVkMunzaSSZTMY3XhQXF/vGm0IpVkMLHFOVRSIR31xaXl7usUIBCExnFnZoMdELz/Bejhw5Uu4Z+49egtgsR6yXYGabsB+Vlpb6ynw7jtMv62+/Z6i7774bAPDggw/ilVdeAQBfHfuKigqJdOcCZNOmTRJNaprsstmsTCT8mcvl5GFjw7BhGxsbfQtffcLnzaB5p6ysTMzBYWHLli2+zqO3C9uBHcA0UwDuwiaZTPoc5sMIA8oSiQTeffddAG7HrqiokE0J24ML0/b2dgmwYlDZRRddJBMsJ1P2OT1yl32ooaHBl28zLESjUZkoOCGOGTMm0KxCOLjoE4w5AXESqampkYU/27iurq4gzHbsC9FoVJ4XLi521wzLdtFdCmgyN6OAwwDPiYvRsrIyX7aDjRs34uSTTwagcl4Cbnucc8450o/4MxaLyb1nX9Dbj+Mjx5loNCoTTV9yJA4mvdU2Z//g+6bbGeBmUMjlcnjuuecAuIt9bnb1vLZ6ppEzzjjDcx5DvcHhZqO6utrzek1Njc/NraWlRY7TRRtA3XfOjbw+/fkyo/wdx/E9a7W1tTK383/z/MIOz1sfV80xNhqNyjPH+8wAzUKCwkw0GpW5kOLiiSeeiCeeeAKAm01Gzwair60A7zjLe845+NBDD5XAzUWLFsn/7M9ca90ALBaLxWKxWCyhZcBsfzfccANuuOEGAO6uhCb/5uZmMedTDU2n0z7HcN1Mw509dz3Dhg0TZYG7XO6UM5mMvKYHEFAp4k4yzMpRU1OT53oAb2AMFQ69eoauOgOu+trU1ORzsA8jvMcjR46UXRiDzDo7O2XnSgWAu7z169eLsnrOOecAUIF+fI2Bd9zZJ5NJNDU1AVB15gGVumrOnDkAwtMveJ2ZTMajMAOq/5uKIf/W8+PprgRBZhtA9RO6RegKfZhM3zr6eXH8KCkpkdf53OhpqnYHjguVlZUSOBNGeF3sz6Wlpb7KMVVVVaKa0AJB5eTDDz+U4ESaNyORiPQPqmmktLTU52JUUVEhbRS2YLTegj92FkQ7a9YsAK5l8P7775c83ldddRUA4MYbb5TjmXKRc9xll122W+cxmJh9Vs+Vap5LJpOR+2aa9XO5nO9+BwV56m4zHIs5PtXW1vpyQw+1W0RfMd3oUqmULy1mNBqVZ43PVZC7Xtih1busrExU0JNOOgmAStnWW25cPa1VkDsN0d0Ajj/+eAAQq2l9fX2/XEMKozdZLBaLxWKxWL6QDEpUBZWOY489djC+fq9k27Ztvp2vXqHLVNra2to8yXn14zdv3lwQyiqvqbS0VFRWVpr67LPPZAfHlGj8GY/HZec7ZcoUAF4lgFBp3bZtm7QRE6cfd9xxvvrP+UYPlKCKpvtZc+fP/qH7ZPM9vYKVGUik1/2mwkwVbmfVS8IE/SZjsZjPJ31XqbfMVF+09pSVlYnvom7tMS0/+YJ9gPc2lUr5rA0VFRVyvmZlmi1btkgSd76mK0fm9cXjcZ8fZCKRkP7T12Isg0lQRSoG4LJfM7jkqaeewurVqwGo4F8AeP/99/Hkk08CcPsALTutra2ituqKqhknMdT9xIzroIVIDw7S20VXTQGvfzv7k56uylRWzUBNndraWmnvoJiLMMP2ojKdTqdxwgkneI6ZMGECXnzxRQCudYdjRiHBeaWqqkqUVabu0sfP3tK76e/pqavMIL/m5mZRbGm9cBynX0HgVlm1WCwWi8VisYSWwshX8wWgtbXVt2PVd+7cPdOHNx6P+3x99bKrLLAQZrgT33fffSUimerpqFGjRN1hYQju7JctWyYq02uvvQZApR+hPw5VZSpSHR0dkvaHu0nAVaWosOUbKhglJSWyy9dLiPK1IB9b9gUer/sk8nj+nUqlpK9QHdB9ksIM71lZWZnHZxfwKjpBmQF6e75qa2tFcWBWgAMOOCA0Pnd8Dhihq/uK8RpSqZQnWhtwFaOtW7f60rLptb5NS47e/2jJ0X1W86Uo7SxBuelP19DQIGkJL7zwQgDAL37xCwDATTfdhJ/+9KcA3DatqamR7DS8dr0c6Xe/+13f/8x3ujfeG7Msqt4+9EdOJpO+NEO6EhzUtr0po3q8CamqqhJfTo5BYYkF2BVU3HVlcdq0aZ5jJk2a5EvPxPiKQoLPrj62cX6dO3eur08H9Yveftf/7ujo8JWh1S2GfcEuVkPCtm3bxCzNBZWeyooDAQeQZDLp60j8SfN32NEDNlasWAHAXayOGDFCHhxOqpy0KysrZaBmm7311luSL5KDJAPyNm7cKBPRBRdcAEC5G7D9wuIoz8lDD6gLcmbncewL3d3dchxNtEF1m/kznU5L+7Ff7SogJSxw8h02bJjP/G8uXvXXzNf199rb26XPcMNzwAEHhGbxzvRsfF5isZhsxHhNqVRKqtPwOC4a1q1bJ9WW9Ipv+uZIf6+rq8sTqAqotuLzlO/nxTRjz5kzBw0NDQDcjWcmk8EjjzwCwO3jXLROnTpVroVtFY/HPTmNATev7XnnnSdjKt2UKisr5f9zIzBmzJiBv9idwPFQz7ULwFNJUg9o4f3l4lYPqgrKqWq6lbA/dXd3S8AqXSX0IGdSKPl46Qqlu9UxnROZMGGCbyHHvO+FRNAGhPnJH3vsMRExTJcqPWDX/Kn/zvknkUj4viMSifSrSmI4pAOLxWKxWCwWiyUAq6zmGe40dDVDT9YMeE12QdWqzOpW/akSMZS88847AJTyQ1Vg3LhxAJQJn7s87mhpwti0aZMoS0x5dcwxx0gyb7Yp1dSOjg5RUKgi/eMf/xB1gA7m+UZPVs97yp2qvrMNSm3G4/kdsVjMt4vWK8xQEeHx3d3dUhu9ENCrOOmvAeq5MVWQbDbrSasCeAORqLSFsQoczZRMtTZ27Fhfgvv6+npRAmlt0E1+VAb1vsPvYB+jqlxcXOxLT1VTUyPPDoO1hhrTfM3+39jYKAGD9957LwDgrrvu8qXWmTFjBgBgyZIlnkpNgOoTVAI5tvDn448/jocffhiA+7zU1NRIe912220AgEsvvXTgLnY30CtQAe691c3vumpouv6wfbq7uwOVVdOyxfaoqKgITEGkp7YyzyPMsA+QXC7n6/96UCH7YaEUPdgVfJ47Ojp86TOJbmXSlVXzGWMf2759u7hU6Z/rj2uVVVYtFovFYrFYLKHFKqt5hupeKpWS3YuZxN1xHJ8Tve6naNZ67uzs9ASihJXx48cDUH5VVEW5Y9d3trwWlp3M5XJSto3X19raKr59hx56KABXgdGDC55//nkASomlkkIFNt/oATJUKfSSxKZvsq4w6cneAdV+5nHsQ7ovGXfGnZ2dvnRFYcJMVh2kAJnPjY4edGUGguilFNnXdJ/ffEPVkMp3TU2NnC/vbWdnp6R/ow8lg4cOPPBAKRhAhaikpMTXpvw/mUxG2oGqSzQalbGK7w01TC6+ZMkSAG4/Lioqkns1e/ZsAMpaQ2uKOVZMmjRJ1GqON6lUSgJm+BqvN51Oi1rNcam7u1v6oBmMM9TwudYtLWTt2rUAlH+rWURGf07MuQRw+4oZiFdWVuZTFfX/qfveFwJUjDmG6v2J6ONN0DhaKAT59TPQWU8Jyff1uBjdasf3zBLxQenT9HLf/VmP2MVqnvnkk08AqE7EG28uTLPZrC9XJgBf9K9uCubCbeLEiYN49v2DHffiiy8WlwCaK9etW+cLKuNEsXLlSpk8nnnmGQAqGIITKwNN3njjDQDeKk3EcRwJiOiP0/dAoptXOEHoeSOJOZAmk0kZQPR61aZJh4NrU1OTVLCi+bunpyfQjBgWGPxCampqxJzJZ4OTTjQalT6jm/55HF/T77seCQ+Eq5oX+z1r1Y8YMcLnMgS4wWd8j32opaXFs6gF1ELCrOHOTCOpVErahsd3dHTI95pm06GCz/Hrr78OwO3PW7ZskfvG1yorK6V/mNW+4vG4LGD1xaoZLc8xpqioSN7Tc4/y+cpXvmZz4cHrY2YVwA3WLS8v920ygrJKBH0/j+Pn29vbxeWERCIReb5IoeRZZe5tbk4qKyslQp7U19f7FmJ8HgsJCjkUhwD3eW5vb/cEzAHuHFJcXOxzP9MX9XxPD7LT3dr4OY7RfcG6AVgsFovFYrFYQotVVvMMd/Z6kEyQisrX9OArc+eqm/4p7YdZWaUpDnDPnbnZGhsbJRUVlQu9DjVzKL7//vsAlLJ61llnAXDzrPK7li1bJi4EVGeSyaTsCvXcq/lEz4eq51fla1Q6TCd1fderu46wz5iVnuLxuK/iVUlJSWCt57DA/sFdejwe77VGfTab9V2z/hrbSG8fPdARUM9ZWBRmmvd1s70ZhBmPx0UhpbLGvq7XOuc1ZzIZ6TO8Zv3aTZcJwB2rZs6cOaDXt7vQvcG8j0VFRaKictwLSp2jV2miS0WQKdesJKibgPVnkJah3vrhYNNbAKV+PqzUNXr06F6VVb1alW665/FBFheqkCQWi4kyb7oNhB3OLzzf8vJyn0tUJBKRvsLAurFjxw7hWQ4MRx11FADg1VdfFRcz3UxvulTp75kpAoPSHer3nMqzniaRrkl9wSqrFovFYrFYLJbQYpXVPLNs2TIA3gpC5s4+FouJ+keFZMyYMbKDpfpGFaSoqEgUxzPOOGMoLqNPcFe2Zs0aUT1+8IMfAACuvPJK8edlKi7uyjo6OrBy5UoAro/mAw88gL///e8AXEVJ9zNbunSp5/vnzZsnO+SwqM9UvSorK8UnjKra8OHDpT+YO1w9rZXua6nX+QbcdjnooINEOaAvaCQSyXtFnp3Ba9bTmZnqKdFVVP3Z4HFBn9N9swClVOYrRZPJEUcc4fl72bJlYj0gZWVlPiWQSlcmk/EEjwDefmKmgqqoqPD5myUSCfks08sNNVRqqBZyPIhEIj6fbb2QCvsMr7Orq8uX+isSiXisGIDfFxRwlSa9kES+MIPs9NdNK0lHR4cvxRXJZrM+X2398+b/6erq8qnW3d3dMh/pATiFwMEHH+z5u7cUfiy2wHRg+XoO+oNe5IPPAJVP/Rk3U5ml02n5LH/qVhf9ewF179muVK43bNjgi8fZE6yyarFYLBaLxWIJLeGVUr4gMH3TihUrpHwbU6hQ/dq+fbu8xiT4a9euFZ9P7oyYAaCqqsq3WwwjTOAdxN13341LLrkEgOsrx51tcXEx5s6dC8CN5Hz00UdlBzd//nwA7s6vo6MD5557LgC35CJ/hgndB40+ePRDzGazcj16eipAlYwMUlv1RNaAG8WqKwdM3NzZ2RmqCHgTRm/rUPExFVM9eltH93HUP6+/RhKJRGiUVXLzzTcDAO68807JeKGXXeXvelldQClhZqQ24F6zqYrEYjHpd/yOuro6zJo1a+Avag9g/6Q16p577gGgykTSEqGXEOXvZlaMIMUvm836VB/9OH4H1bRMJoOvfe1rA3BVfYepuWhl4vM+ceJEaauHHnoIQHC5zN3FVFv1AhsklUr5/OALJWm+mVUkKHsM4C8YElQYIeww205ra6s8D8wQcOutt+Kuu+4C4Fr0eO9bW1vl+eD9LSoqkrZiVhn+ze8EgI8++kiON9Pl7QnOAEj1haH1985Az9B9ao+uri6sWLECgFvfevHixQCU+YuDA3MhHnXUUXjyyScBuAFFHBzOPvtsOa4PhKI9dJgjkhP0pk2bpNPz59atW3H44YcDcANM2C6myXQPyUt78H7rpmoz9RIXX+3t7TKQ0AyqO8ubFZt0syHdS6qrq3d3sZqX9uAgS3p6egIXYIA7iQPeVEOmawDbcZ999hGTOV8rLy+X13bBkLdHNpvFvHnzALiBLnw2APeZ0NuHzwSvKajCGftQW1ubbIA5aX3rW9/a3fPPS//g9dPdZ82aNbLB5z3lJq2srExcCOgiUFFRIf1CP45/m+9VV1fL5nkXDFp7MFUZ3SJ4PnV1df1KEdQXtm/fLmIJ+9GECRM857WD0M0vnHd/9KMfAVBCRlB/f+GFFwCozRGgFncAMGXKlP78+8FQCHptEwYgPvzwwzImXHfddfI+BTBeKzeGmzdvloA7iiYlJSUyx3CxSvHtxz/+sXwn3fOWL18uVd4Y8NwLgW1i3QAsFovFYrFYLKFlIJRVi8VisVgsFotlULDKqsVisVgsFosltNjFqsVisVgsFosltNjFqsVisVgsFosltNjFqsVisVgsFosltNjFqsVisVgsFosltNjFqsVisVgsFosltPx/tkFWYfVuW7UAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "_jItLcbfc0ij" }, "source": [ "np.random.seed(42)\n", "tf.random.set_seed(42)" ], "execution_count": 12, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "bwls5ex8kGKj", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "dfbb10e8-432c-4554-d6bb-dd8b752a04ec" }, "source": [ "28*28" ], "execution_count": 13, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "784" ] }, "metadata": { "tags": [] }, "execution_count": 13 } ] }, { "cell_type": "markdown", "metadata": { "id": "WclsZdzGduHj" }, "source": [ "## 1.2: Building a 2-layer DNN" ] }, { "cell_type": "markdown", "metadata": { "id": "upYAnpXFfF2u" }, "source": [ "Build a `Sequential` model (`tf.keras.models.Sequential`) and add four layers to it:\n", " * a `Flatten` layer (`tf.keras.layers.Flatten`) to convert each 28x28 image to a single row of 784 pixel values. Since it is the first layer in your model, you should specify the `input_shape` argument, leaving out the batch size: `[28, 28]`.\n", " * a `Dense` layer (`tf.keras.layers.Dense`) with 300 neurons (aka units), and the `\"relu\"` activation function.\n", " * Another `Dense` layer with 100 neurons, also with the `\"relu\"` activation function.\n", " * A final `Dense` layer with 10 neurons (one per class), and with the `\"softmax\"` activation function to ensure that the sum of all the estimated class probabilities for each image is equal to 1." ] }, { "cell_type": "code", "metadata": { "id": "pgTNq_98d527" }, "source": [ "model = tf.keras.models.Sequential([\n", " tf.keras.layers.Flatten(input_shape=[28, 28]),\n", " tf.keras.layers.Dense(300, activation='relu'),\n", " tf.keras.layers.Dense(100, activation='relu'),\n", " tf.keras.layers.Dense(10, activation=\"softmax\")\n", " \n", "])" ], "execution_count": 14, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "tJQ9GaMcfVgg" }, "source": [ "After a model is created, you must call its `compile()` method to specify the `loss` function and the `optimizer` to use. In this case, you want to use the `\"sparse_categorical_crossentropy\"` loss, and the `keras.optimizers.SGD(lr=1e-3)` optimizer (stochastic gradient descent with learning rate of 1e-3). In this case you should specify `metrics=[\"accuracy\"]`." ] }, { "cell_type": "code", "metadata": { "id": "gamiwmWteKUM", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "ca11b215-46ff-4798-83d0-e791f89a7966" }, "source": [ "model.compile(loss=\"sparse_categorical_crossentropy\", \n", " optimizer=tf.keras.optimizers.SGD(1e-3),\n", " metrics=[\"accuracy\"])\n", "model.summary()" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Model: \"sequential\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "flatten (Flatten) (None, 784) 0 \n", "_________________________________________________________________\n", "dense (Dense) (None, 300) 235500 \n", "_________________________________________________________________\n", "dense_1 (Dense) (None, 100) 30100 \n", "_________________________________________________________________\n", "dense_2 (Dense) (None, 10) 1010 \n", "=================================================================\n", "Total params: 266,610\n", "Trainable params: 266,610\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "EtiTBHrEimDP" }, "source": [ "## 1.3: Train & Test" ] }, { "cell_type": "markdown", "metadata": { "id": "GePIEjOwfYWe" }, "source": [ "Now your model is ready to be trained. Call its `fit()` method, passing it the input features (`X_train`) and the target classes (`y_train`). Set `epochs=10` (or else it will just run for a single epoch). You can also (optionally) pass 10% validation data by setting `validation_split=0.1`. If you do, Keras will compute the loss and the additional metrics (the accuracy in this case) on the validation set at the end of each epoch. If the performance on the training set is much better than on the validation set, your model is probably overfitting the training set (or there is a bug, such as a mismatch between the training set and the validation set).\n", "**Note**: the `fit()` method will return a `History` object containing training stats. Make sure to preserve it (`history = model.fit(...)`)." ] }, { "cell_type": "code", "metadata": { "id": "kC-SwgN5eRWe", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "a243354a-cb08-47a3-dec5-2f7c9d9bc2b0" }, "source": [ "history = model.fit(X_train, y_train, \n", " epochs=20,\n", " batch_size=32,\n", " validation_split=0.1)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Epoch 1/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 1.5028 - accuracy: 0.5739 - val_loss: 1.0122 - val_accuracy: 0.7038\n", "Epoch 2/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.8731 - accuracy: 0.7313 - val_loss: 0.7603 - val_accuracy: 0.7532\n", "Epoch 3/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.7203 - accuracy: 0.7667 - val_loss: 0.6677 - val_accuracy: 0.7747\n", "Epoch 4/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.6482 - accuracy: 0.7883 - val_loss: 0.6158 - val_accuracy: 0.7895\n", "Epoch 5/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.6018 - accuracy: 0.8038 - val_loss: 0.5773 - val_accuracy: 0.8050\n", "Epoch 6/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.5688 - accuracy: 0.8135 - val_loss: 0.5563 - val_accuracy: 0.8138\n", "Epoch 7/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.5445 - accuracy: 0.8203 - val_loss: 0.5312 - val_accuracy: 0.8183\n", "Epoch 8/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.5257 - accuracy: 0.8254 - val_loss: 0.5145 - val_accuracy: 0.8230\n", "Epoch 9/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.5104 - accuracy: 0.8302 - val_loss: 0.5052 - val_accuracy: 0.8277\n", "Epoch 10/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.4978 - accuracy: 0.8338 - val_loss: 0.4915 - val_accuracy: 0.8297\n", "Epoch 11/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.4873 - accuracy: 0.8367 - val_loss: 0.4822 - val_accuracy: 0.8348\n", "Epoch 12/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.4779 - accuracy: 0.8387 - val_loss: 0.4762 - val_accuracy: 0.8363\n", "Epoch 13/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.4699 - accuracy: 0.8406 - val_loss: 0.4685 - val_accuracy: 0.8377\n", "Epoch 14/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4627 - accuracy: 0.8429 - val_loss: 0.4635 - val_accuracy: 0.8412\n", "Epoch 15/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4568 - accuracy: 0.8447 - val_loss: 0.4612 - val_accuracy: 0.8377\n", "Epoch 16/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4509 - accuracy: 0.8465 - val_loss: 0.4504 - val_accuracy: 0.8405\n", "Epoch 17/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4456 - accuracy: 0.8477 - val_loss: 0.4453 - val_accuracy: 0.8453\n", "Epoch 18/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.4410 - accuracy: 0.8499 - val_loss: 0.4478 - val_accuracy: 0.8437\n", "Epoch 19/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4363 - accuracy: 0.8506 - val_loss: 0.4390 - val_accuracy: 0.8475\n", "Epoch 20/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.4324 - accuracy: 0.8521 - val_loss: 0.4375 - val_accuracy: 0.8492\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "NBl9eujxfx9D" }, "source": [ "Try running `pd.DataFrame(history.history).plot()` to plot the learning curves." ] }, { "cell_type": "code", "metadata": { "id": "dDT-K5yneh5O", "colab": { "base_uri": "https://localhost:8080/", "height": 324 }, "outputId": "7729af24-fd91-419b-cd61-b9fb293bd3c3" }, "source": [ "pd.DataFrame(history.history).plot(figsize=(8, 5))\n", "plt.grid(True)\n", "plt.gca().set_ylim(0.3, 1);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeMAAAEzCAYAAAACSWsXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3xUVf7/8deZnsxkUmaSUBIIpFACoQQQLBSVFRUR66rYdXV/iq7uV3fV7xbX71p21V3rqqu76tpwRVAQBEWJKCrSCaGHmtASICF92vn9MSEECGQCKWTyeT4e85i5d865c44ib++5556rtNYIIYQQou0Y2roBQgghREcnYSyEEEK0MQljIYQQoo1JGAshhBBtTMJYCCGEaGMSxkIIIUQbazSMlVL/VkrtVUqtPs73Sin1glJqk1JqlVJqcPM3UwghhAhfoZwZvwWMO8H3FwLpta87gFdOvVlCCCFEx9FoGGutFwD7T1DkUuA/OuhHIEYp1bm5GiiEEEKEu+a4ZtwV2FFvu6B2nxBCCCFCYGrNH1NK3UFwKJuIiIjs5OTkZjt2pb+SYn8xCaYEbAZbk+tX+TR7KjWd7QasxmZr1ikLBAIYDOE1zy4c+wTh2S/pU/sRjv0Ktz5t2LChWGsd3+CXWutGX0AKsPo4370GXFtvez3QubFjZmdn6+Y0+6vZuv9b/fUrK145qfordxzQ3X/7mf4yb3eztutUzZ8/v62b0OzCsU9ah2e/pE/tRzj2K9z6BCzRx8nE5vhfjhnAjbWzqocDpVrrXc1w3CaJMESQGpPKqqJVJ1Xf5bACUFxe05zNEkIIIRrV6DC1UuoDYDTgVkoVAH8EzABa61eB2cBFwCagErilpRrbmKz4LL7a/hVaa5RSTarrslsA2FfhaYmmCSGEEMfVaBhrra9t5HsN3N1sLToFWe4spm2cxvay7XR3dm9SXZvZSJTVJGfGQgghWl34XBkneGYMnMJQtYXicjkzFkII0brCKox7RvfEbrazsmjlSdV3O6zskzNjIYQQrSyswthoMNLP3e8Uz4wljIUQQrSusApjCF433nBgA1W+qibXdTms7JNhaiGEEK0s7MJ4QPwA/NrPmn1rmlzX7bCyv9KDP6BboGVCCCFEw8IujPvH9wdObhKX22FBa9gvtzcJIYRoRWEXxnG2OJKjkk8qjF324MIf+yrkurEQQojWE3ZhDNDf3Z9VxSd3ZgzIdWMhhBCtKizDOCs+i72Ve9ldsbtJ9WRJTCGEEG0hLMN4QPwAoOnXjQ+dGcvCH0IIIVpTWIZxr9heWAyWJodxdIQZk0HJmbEQQohWFZZhbDaa6evq2+TrxkopXA6LrMIlhBCiVYVlGEPwuvGafWvw+r1NqueWhT+EEEK0srAO4xp/DRsObGhSPZfDKsPUQgghWlXYhvGhSVxNfWiE2y5PbhJCCNG6wjaMEyMTSYhIaPJ1Y3eUlX0VNQQf0yyEEEK0vLANY6UUWfFZTZ5R7bJbqPYGqPD4W6hlQgghxJHCNowheN14R9kO9lfvD7nOoYU/ZEa1EEKI1hL2YQywunh1yHUOL/whYSyEEKJ1hHUY93X1xaiMTZrE5a5bElMmcQkhhGgdYR3GEaYIMmIzmnTd2CUPixBCCNHKwjqMIThUnVuciz8Q2oSsQ49RlGFqIYQQraVDhHGFt4ItpVtCKm8xGXDaTDKBSwghRKsJ/zB2BydxNeV+Y7fDSnGFDFMLIYRoHWEfxt2d3XFanE26bux2WCkukzNjIYQQrSPsw/jQ4h9NmVHtcljYJ2fGQgghWknYhzEErxvnl+RT7ikPqbxbHhYhhBCiFXWIMB7gHoBGs3pfaIt/uBwWSiq9eP2BFm6ZEEII0UHCuF98P4CQrxsfWhLzgAxVCyGEaAUdIoydFic9o3uSW5QbUvn42oU/imSoWgghRCsIKYyVUuOUUuuVUpuUUg818H13pdRXSqlVSqkcpVRS8zf11GTFZ7GqeFVIj0Y8/LAIOTMWQgjR8hoNY6WUEXgZuBDoC1yrlOp7VLFngP9orbOAx4Anm7uhp6q/uz/7q/dTUF7QaFmXXR4WIYQQovWEcmY8DNiktd6stfYAU4BLjyrTF/i69vP8Br5vcwPiBwChXTd2R8mZsRBCiNYTShh3BXbU2y6o3VffSuDy2s+XAVFKKdepN6/5pMakEmGKCCmMo6wmLEYDxRVyZiyEEKLlmZrpOA8ALymlbgYWAIXAMU9mUErdAdwBkJiYSE5OTjP9PJSXlzd6vCRjEt9t/o6cqsZ/12HWrN64nZyIPc3TwJMUSr/am3DsE4Rnv6RP7Uc49isc+3Q8oYRxIZBcbzupdl8drfVOas+MlVIO4AqtdcnRB9Ja/xP4J8CQIUP06NGjT67VDcjJyaGx461YuoK3895m+NnDsZlsJyzbNfc7zA4Lo0cPa7Y2noxQ+tXehGOfIDz7JX1qP8KxX+HYp+MJZZh6MZCulOqhlLIA1wAz6hdQSrmVUoeO9TDw7+ZtZvPIis/Cp32s27+u0bIuh0WuGQshhGgVjYax1toHTAbmAmuB/2qt85RSjymlJtQWGw2sV0ptABKBx1uovackKz74BKdQ1ql22WVJTCGEEK0jpGvGWuvZwOyj9v2h3uepwNTmbVrzc0e46eroGuKM6uCZsdYapVQrtE4IIURH1SFW4Kovy50V0rON3XYrHn+Ag9W+VmiVEEKIjqzjhXF8FrsrdrO3cu8Jy7mjggt/7JOhaiGEEC2sQ4Yx0Og61S577cIf8rAIIYQQLazDhXHvuN6YDWZWFp94Eper9mERxWVyZiyEEKJldbgwthgt9Inr0+gkrvjah0UUy5mxEEKIFtbhwhiCQ9V5xXn4AsefnBVntxBpMbK6oLQVWyaEEKIj6rBhXO2vZuOBjcctYzIaGJfZidmrd1HtPWZlTyGEEKLZdNgwhsaf4DRxUFfKqn3MX3fimddCCCHEqeiQYdzF3gWXzdXo/cZnprqIj7IyfXnhCcsJIYQQp6JDhrFSiqz4rEbPjE1GA5cO6ML89XspqZSJXEIIIVpGhwxjCA5Vbz24lZLqYx4udYSJg7ri9Wtm5e5qpZYJIYToaDpsGA+IHwBAbvGJF//I7OIkPcHB9GUyVC2EEKJlhPSgiHCU6crEoAysKl7FOUnnHLecUoqJg7ry9Nz1bN9XSTdXZCu2UgghRLMJBMBfA75q8HlqP3sa3uerBqWgzyWt0rQOG8aR5kjSY9IbXRYT4NKBXXh67no+XVHIPeelt0LrhBCindA6GFzeKvBWHvVedey2rwoCPgj4gy9d+x7wgQ7U++wno7AASj8KqSx+L/hqasO09uX31PtcEyzbUBcCoP2KQO279it0QKFNDmxPSxi3uKz4LOZsnUNABzCo44/YJ8VGckaPOKavKGTyuWnySEUhRNsKBGrP5Kprw6aKyIodsDs3GEB+b733Q5/r7Q8cZ/8R5b2HQ9bXQKjWD95TosBgAoMRlDH4XvvZ5fNDRSQoI4GAwu8xEvAY8HsM+GtUsJm1Oat9ikDAUBukFrTfWpff2qfRfk3AF0D7AmhvAO3zE/D60V5/8J9nAwxRDnqdYu9C1eHD+KMNH7H14FZ6Rvc8YdnLBnXloWm55BaWkpUU00otFEKctrSuDcLqeu/VR257q4/z/dHfVTVQp973R+/zH3t3xzCAxU3rQsAP/mojvmoDPo8Fv8eMr8aEv8aEr9pAwG/AYDGhzCaUxYQymzFYolBWN8pqQVltKIsVgy0CZYtARUSibJEYIuyoCDsqMgpDhAMV6UTZo1CR0ShrJIHKKvxlFfgPluE/eBB/SSn+0lL8B4PvgdJS/CWllO7cSYTPh7+0FF1dDXgb7ojRiCEiAmU1o6wWDBYrymJBWa212xaUxVq3rSy1ZepvW621ZeptR0Q09U/FSevYYew+vPhHY2F8Yf/O/OHTPKYtK5QwFuJ0dmjY1FNZe/ZWCZ6KurM4d9FiWLHr8HfeqnrfB991TQW6sgLtrcKgalB+T8NheyqUAUwRYLKCufbdZDv8sjjAHn/s/gbLW8nbkE/ffgMIVPnwlVXjP1iF72AVvtJy/Acr8JWU4S85iO9AKb4Dpfj3HyBQ0fBZrcFux+hyYYiIQHs86PIaAh4PuqYG7SlH17TMA3SUzYYxOrru5Xe7saelYoyOObw/JvhucDqD+2OiMdjt7X7EskOHcUp0ClHmKFYVrWJi2sQTlo2OMHNenwRmrtzJ/17cB7Oxw05EF6LpAv7a4c7qI9/rhkCrD797K49Trv6+yqPqHD1sqoOXEz0G/F4DAY8KDm16DXT1KIq9waHOgEcFv/ca8de+ArUjtxwauVRgjDBhsJkwRsRgtFsxRNow2m0YHBEYHXYMDjvGqCiMzigMzkNhEYMx1oUhMurIEDVaat/Ndf94tNbomhoC5eUEKirqXv6KCgLltdsHKup9V06gYm/w+4oKAhWV6F27WFfxEXgbOHtUCmNsLCaXC6PLRURyD4yuOEwuNyZXHEaXC1PtyxgXh6GRM0KtNdrrDYZzTQ3a4yFQ+35oXzC8PWjPUds+L8Yo57GhGu3EYLMd8TtbcnIYNHr0Kf3Ray86dBgblIH+8f0bXfzjkImDuvL56t18t6mYMb0SWrh1QrQSvw88ZVBTDp5yqCkLvjzlJO5eCks2HyckGw5Q7a2Cmiq0p4ZAdXXwL2OvNzhJJlBvcsyh7SP2H9pX7zMWNKbgS5vQ2lj7MgRfASM6EEXAY8dfHcBf5SVQ5Q1eCzwBZbVgiIrC6HRijHNijHZiiXJicEYFwyLaGfzHc7CMQNnB4JBq2UECpQfxlpURKCzFf3BHo2eJymIJBo7TiTEqChUZga6swl9RTqCisi5g8Ye2Br6KjMRgj8Rod2Cw2zHY7Zg7dcITG0NSZmZtsB4K2dr32FiU0Rjan4dQ2qAUymIBiwWioprtuB1Zhw5jCF43/ueqf1LprSTSfOLblsb0SiAm0swnywsljEXb81ZDdQlUlUB1aW2IHqwXqOW1Idtw0B7ar2uq8NUY8FUb8VUdfvdXG3F6FQWHgrEuRGuDMGAgUD9E/aD9wYkyaAAjYK99nSKTKXgN0Gw+8mUxo8wWlNWMMd6OyRmNMSrqiEA1RDkxOoOha3A6WZSby9njxmGwWk+9XUCgpoZAWVkwtA+W4i8LXgcNHDx4ZJDX7gtUVWGw2zElJGBwHA7U4CsS4zH77IfLRUQcN1Tzc3JI6CBnkeFIwtidRUAHyNuXx9BOQ09Y1mIycHH/zny8rIDyGh8Oa4f/xydO1RGB2sh71YEj953gmqXW4PcY8Hkj8Pvs+Lw2fB4LvmpTMHArrfgrjPjK7fiP88xuQ5Qdn9mMLSoKZTs02cVaO/mldlKMOTjZ5ciXua5M3T7zsd8fDtXafYc+1w9ZixllMqEMzXdZKLB9e7MFMYDBasVgtWJyu5vtmKLj6fBp0t/dH4CVRSsbDWMIzqp+b9F2vsjbzeWDk1q6eaI90RqqD0JFEVQU177vrfe5CMqLoHLf4WD1VaN1vfscfcEzzUDtvY4BZUcbHQQMkWgVSUDFoFUiAW1BazOBgAkdMBLwKfzl1fgOlOHbX4rvwAHw1R/29AE+lNWKKT4ek9uNJdVNhNuNye3G5I7HFH/osxuj243BYiEnJ4fRcrYlRIvr8GEcY4shxZkS8nXj7O6xJMVGMH15oYRxR+D3BsOzogjKjwrWI17FnF26B+8cH/5qA74aY+27Ifjus+H32vDVGAl4FAFfBNpnI+Dxo33+2mHd4wkA5bWvIymrFWWzYbDZghNyErtgzRxQF6qHAtbodmOKjw+LWadChKMOH8YQvG68sHAhWutG/6JSSnHZoK68PH8Tew5Wk+i0nbC8OA0F/LWhuhfK9wRD9tCr3j5dthd/aUkwTKsN+GsOvRuD92L6IvB5TMH9lYpA1XGGKU0mTLGxwYk1XWIxR0djiIjEYLMG79O0Be/TPLRtiLAdu782cJXNhrJag9cOLZZmHb4VQrQdCWOC141n5M9gV8Uuuji6NFp+4qCuvPj1Jmau3Mnt55z4/mTRSgKB4NDvMQG7py5kddkeAvuL8B04gL9a1Z211oWsx4LPa8XvMeGr0vgrbaA7HftbtbeJGONiMSW4sLrisMfGsaPsIGnZ2Rjj4jDFxWGMc2FyxWFwOuVsVAhxQhLGBM+MIbj4RyhhnBrvYEBSNNOXF0oYtxa/Dw4WQukOKNkOJdvRB7YT2LsV/94C/EV78VUFaoPVWC9gTcGVhWoM+Co1BExA/DGHN0RFBe+x7OzCEhdLRJwrGLa1gWqse4/DGBPT4IzWdTk5xMr1VSHESZAwBtJj07EZbawsWsm4HuNCqjNxUFf+NHMNG/aUkZEo99mdMp8HXbKdQOF6fDs24N+5Fd+eAvzFe/Dv3x9cRahGBYeMPcHbbvye4Dq0QbFHHE5F2IJnp/FuzC4XttpgDS504MIYG3c4ZGNjgvdMCiFEG5EwBkwGE5nuTFYVhzaJC2B8Vhf+PGst05cX8ttxvVuwdeHBX16OZ80KalYsJHHJD+x+79ngcHFpOf7yGnxVfvw1BtAND+cabNEYnQ6MsTGY3fHY3J2Dk5Li4jDGxgXPYl2u2uHhxlcQEkKI04mEca0sdxbvrn0Xj9+Dxdj4WVJ8lJVz0t18uryQB3/WC4NBrglqrfHv30/NhvV4Vv1AzdpVeLZspWbnfnxlRz66rNQSwBRhwBhlw9zZTURcLEZ3AqaELhi7pGDskorJHR8cFo6Nbdb7QoUQ4nQjYVwrKz4Lb56XdfvX1V1Dbsxlg7ryqykr+Gnrfob3dLVwC08fOhDAt2sXNZs2UZO3DM+6VdRs3oKncB/+qsOhq0wBrE4/9q52LN1SsKb3wtJvKMsqDYwYdxUY5Y+fEEJAiGGslBoHPE9wfbs3tNZPHfV9N+BtIKa2zENa69nN3NYWVX8SV6hh/LO+nbBbjHyyvDAsw1h7vXi2b6cmPx/P+rXUrF2BZ/MWagqLj1j312j1Y3H6iEqxYU3ujCWjF9bMIZj6jkC5049YEB+gJidHglgIIepp9G9EpZQReBkYCxQAi5VSM7TWa+oV+x3wX631K0qpvsBsIKUF2ttiEiIT6GTvFPLiHwARFiMX9OvErNxdPDohE5u5+RZibyue7dsp/+YbKubNomJ5Htpz+EzXFOnD6vQRm2bAktwJa1o6ln7ZmNKGQkIfsEW3YcuFEKL9CuX0ZBiwSWu9GUApNQW4FKgfxhpw1n6OBnY2ZyNbS5Y7q0mTuCA4VD1tWSHz1+3lwv6dW6hlLSdQWUnFTz9R8dVcyhd8g3fPAQDMdh8x3WuISHFjSUvH2ncQhm4DITETopNA7psVQohmE0oYdwV21NsuAM44qsyjwBdKqXsIPqLl/GZpXSvLis/ii21fUFxVjDsitEXfz0x1Ex9lZdrywnYRxlprPPn5lH+TQ8W82VTmrkf7AihjAHuih7gzrTjOGoFl+AToORoiYhs7pBBCiFOktD7horgopa4Exmmtb6/dvgE4Q2s9uV6ZX9ce61ml1AjgX0A/rXXgqGPdAdwBkJiYmD1lypRm60h5eTkOh+OUjrG5ejN/3/N3fhH/C7IiQ7tuDPDBuhrmbfPx/JhIHJbmPWNsjn6pqios69YRuXIJljVr4GDwaT8Wpxd7Zy+kdaI8cwgHEoZSYe/e4me9zdGn01E49kv61H6EY7/CrU9jxoxZqrUe0tB3oZwZFwLJ9baTavfVdxswDkBr/YNSyga4gb31C2mt/wn8E2DIkCG6OZ8G0xxPlxnuG86LH7xIIDHA6OzQjxWfUcrcF76jxNmT8cO7n1IbjnYy/dKBADXr1lGe8zUVX82hcu1mCGgM5gD2xBrsgxw4zhmFeeglkHI2WFv3D3u4PgkoHPslfWo/wrFf4din4wkljBcD6UqpHgRD+BrguqPKbAfOA95SSvUBbEBRcza0NdhMNnrH9ia3OLdJ9fp2dpKR6OCT5YVc38xhHCrfgQNULFxIxZezKP9hEf6DVQBYYz24+gZwDO5LxKjxqN4/A1dqm7RRCCFEwxoNY621Tyk1GZhL8Lalf2ut85RSjwFLtNYzgP8BXldK3U9wMtfNurHx79NUVnwW0zdNxx/wYzSENjtaKcXEQV3565z1bN9XSTdXZAu3EgIeD1XLllPx/UIq5n9B9aZtoMFoCWDvVI39DDeOUWMwDR4P3UaASRbNEEKI01VIN3vW3jM8+6h9f6j3eQ1wVvM2rW1kxWfx/rr32VSyiV5xvUKud+nAYBh/sqKQe89Lb/Z2aa2p2bCRiu+/p+L776lcvBhdXQ0KIlw1uLPAMWwgtlETUOljIbprs7dBCCFEy5CVF45St/hH8aomhXHXmAjO6BHHJ8sLuefctGZ5ZJ53715sP/5I4ezZVPzwA/6iYgAsbisx3Q5gT6wicvAgjGfeAn0vBbOsxyyEEO2RhPFRkhxJxFpjWVW0iqsyrmpS3csHd+W3H+eyqqCUAckxTf7tQGUllYsX15391mzcRDRQERONPS0We9992J27MMfHwsAbYNCNEJ/R5N8RQghxepEwPopSiqz4rCatxHXIuH6d+f2neUxfXhhSGGu/n+q8vGD4LvyeyhUrwOtFWSxEZg8mekQG3qqfSDSuRhGAnmMg+3HodTGY5JF/QggRLiSMG5AVn8U3Bd9w0HMQp8XZeIVa0RFmzu+TwMyVO/nfi/tgNhqOKePdu5fy+TnBAP7xRwKlpQBY+/Qh7sYbsPfvSWRgOYY1/4XyPdRExKGG/w8Muh5iU5qri0IIIU4jEsYNOHTdeHXRas7semaT6k4c2JXZubv5blMxY3olAKB9PsoXLKDko6mUL1gAfj+mTp2IOu887GeeiX3oYEx7F8Kyt+HHP4MyQPoFkH0TPxZaGHXuec3eRyGEEKcPCeMG9HP1Q6FYWbyyyWE8ulcCMZFmpi8r5KyIako+nkbptGn4ioowxrtx3Xor0ZdOwJKaiipaD8v+A29Nhqr9ENMNxvwOBk0CZxcA9K6cFuihEEKI04mEcQMcFgepMakndd3Y5Pdytz8f5ysvkf+njWAw4Bg5kpirr8Jxzjko7YW86fDvu2DHIjCYoffFkH0T9BgNhmOHtoUQQoQ3CePjGBA/gHnb56G1Duk2per1GyiZOpXSGTM4u7SUXZFx7L3qZs6cfDPmxMRgoU3zYOb9ULodXOkw9v9gwLXgiG/h3gghhDidSRgfR1Z8Fh9v/JjtZdvp7mx4iUt/eQUHZ8+iZOrHVK9ahTKbiRo7lugrr+DObyvp7o5iVGIiVOyDuY/AqingzoAbPgk+EUkeQyiEEAIJ4+PKcgcncS3fu/yIMNZaU7ViBSVTp3Lw8znoykqs6WkkPvIwzksuwRQbfOTgxPL1vDR/I6U/vU90zu+hugRG/gZGPiBLUwohhDiChPFx9IzpSZIjiTdXv8nFPS9GlZZT+umnlEydimdTPioykuiLLyLmyiuxZWUdM5R9eRoM+PYZomcvh67ZMGEGJGa2UW+EEEKcziSMj8OgDPx2yIP88817WDTjatyL89FeL7YBWXT6v8dwXngRRof92IqBACz5FynzHqWT0ctrEb/gztv+AiE+dEIIIUTHI2F8HDWbt5A8+Vl+vzlAecR6rFdeQedrbsDW6wTLTxathxn3BGdJp57LZ53/hyfnlTJ6byW9OkW1XuOFEEK0K3IfTQOqVq5k23XX4S8pwfLYb5l8r5XXz/UfP4h9Hsj5C7x6NhRvgMteg+unMXr4UIwGxScrClu3A0IIIdoVCeOjlOXksO2mmzFERZHywfukXn0zkwbcwszNM1m2Z9mxFQqWwD9HQc4T0GcC3L0YBlwDSuF2WBmZ7ubT5YUEAu3y8c5CCCFagYRxPSUff0zB3ZOxpqaS8sH7WLoHZ1Hf3v92Otk78cSiJ/AH/MHCNeXw+UPwxvlQXQrXfghX/uuYe4YnDurKztJqftq6v7W7I4QQop2QMCZ4u1LxK6+w639/h334cLq9/TYmt7vu+0hzJA8OeZD1B9bz0YaPgot3/GMELHoFht4Od/0IvcY1eOyf9e2E3WJk+jIZqhZCCNGwDh/G2u9n92OPUfT8CzgnXELyK/9ocJb02O5jOSMhmxcWPcX+968Csw1unQsXPwO24z/ZKcJiZFy/zszO3UW119+SXRFCCNFOdegwDlRXU3jffZR8MAXX7bfR5amnUJYGnhOsNWr1xzy85juqtI8X+o6EO7+FbsND+p3LBnWlrMbH1+v2NnMPhBBChIMOG8b+0lK233Y7ZfO+IvGRh0l44AFUQw9pKC2AD66Bj28j1dmNST3GM61iC7klG0P+rRGpLhKirExfLkPVQgghjtUhw9i7axfbrr+e6lWr6Pq3Z4m78caGCy59G14+A7YsgAuehNu+5JcjfocrwsUTi54goAMh/Z7RoLh0YBdy1u/lQIWnGXsihBAiHHS4MK7ZuJGt11yLd9dukl9/HeeFFzZccP8WmHkvdBkEd/0AI+4CgxGHxcGvs3/N6n2r+WTTJyH/7sRBXfH6NbNydzVTT4QQQoSLDhXGlUuWsHXS9RAI0P3dd7APP+P4hVdPDb5PfAViU474anzP8QxOGMxzS5+jtKY0pN/u29lJr8QoGaoWQghxjA4Txge/+ILtt96GyeUiZcoH2Hr3Pn5hrSF3KnQbATHJx3ytlOLhMx6m1FPKyyteDun3lVJMHNSVpdsOsH1f5cl2QwghRBjqEGF84IMPKPzVfdj69KH7++9h7tr1xBX25EHROuh/5XGL9I7rzdUZV/Ph+g9Zv399SO24dGAXAFkeUwghxBHCOoy11ux97jl2/+kxHKNH0+2tN+ueN3xCuR+BMkLfiScsNnnQZKIt0Tyx6Am0bny5yy4xEYzo6eK9RdvYLxO5hBBC1ArbMNY+H7t+9zv2vfoaMVddSdKLL2CIiGi8YiAAq6dB6rlgd5+waLQ1ml8N/hXL9i5j1pZZIbXrfy/uw4FKL/d9uELWqxZCCAGEaRgHKispuHsypR9Pw3333XR67DGUKcSnRRb8BKXbTzhEXd9l6ZfRz9WPZ5c8S7mnvNHy/bpG88dL+rJgQxEvz98UWpuEEBtRsIUAACAASURBVEKEtbALY9+BA2y7+RbKv/2WTo8+Svw9k1FKhX6A3KlgskHvi0MqblAGHjnjEfZV7ePVla+GVOe6Yd2YOLALf5u3gYWbikNvmxBCiLAUVmHsKShg27XXUbN+PUkvvkDsNT9v2gH8PsibDhnjwBoVcrX+8f25PP1y3lv7Hvkl+Y2WV0rx+GX9SYt38Kspy9lzsLpp7RRCCBFWQgpjpdQ4pdR6pdQmpdRDDXz/d6XUitrXBqVUSfM39cRMO3aw9dpr8R04QLc3/03Ueec1/SBbcqCyOOQh6vruHXwvEeYInvzpyZAmc9mtJl65fjCVHj/3vL8cnz+01byEEEKEn0bDWCllBF4GLgT6AtcqpfrWL6O1vl9rPVBrPRB4EZjWEo09nooffyT22b+hTGZS3n+PyMGDT+5AuVPBGg1pY5tcNc4Wxz2D7mHRrkV8ue3LkOqkJUTx5OX9+Wnrfp7+IrTbo4QQQoSfUM6MhwGbtNabtdYeYApw6QnKXwt80ByNC5WyWPB36kTKlA+wpqae3EG8VbD2M+hzSfDxiCfhqoyr6BXbi6eXPE2lN7SFPS4d2JVJZ3TjtW828+WaPSf1u0IIIdo31diQqlLqSmCc1vr22u0bgDO01pMbKNsd+BFI0lof8/BepdQdwB0AiYmJ2VOmTDn1HtQqLyvDERX6dd6juYu+p1/eX1iZ9ScOxA086ePkV+fz3J7nuMB5AeNjx4dUx+PXPL6omqLKAH86M4L4yMP/j1ReXo7D4Tjp9pyOwrFPEJ79kj61H+HYr3Dr05gxY5ZqrYc0+KXW+oQv4ErgjXrbNwAvHafsb4EXGzum1prs7GzdnObPn39qB5gySeu/pmnt951yWx5a8JAe9J9BelvptpDrbCuu0P3+OEePf+FbXeU53IZT7tdpKBz7pHV49kv61H6EY7/CrU/AEn2cTAxlmLoQqL9Ac1LtvoZcQysPUTeL6lLY8AX0uxwMxlM+3K+zf43ZYOYvi/8Scp1urkj+dvVAcgtL+fOsNafcBiGEEO1HKGG8GEhXSvVQSlkIBu6MowsppXoDscAPzdvEVrD2M/DXQL+mz6JuSHxkPHcNvIsFBQv4Zsc3Idcb2zeRO0f25N0ft/OprF8thBAdRqNhrLX2AZOBucBa4L9a6zyl1GNKqQn1il4DTKk9FW9fVk+FmO6Q1PBQ/sm4rs919IzuyVM/PUWNvybkeg9c0IuhKbE8PC2XTXvLmq09QgghTl8h3WestZ6ttc7QWqdqrR+v3fcHrfWMemUe1Vofcw/yaa98L2zOCd5b3JSVuhphNph5aNhDFJQX8ObqN0OvZzTw4rWDiTAb+X/vLqPG1/7+30YIIUTThNUKXCcl7xPQAeh/VbMfekSXEYztPpY3ct+gsDz0YedO0Taev2YQm4rKeWtNTUiLiAghhGi/JIxzP4KETEjo0yKHf3DIgygUzyx+pkn1zk53c//5Gfyw088HP+1okbYJIYQ4PXTsMD6wNfiUpv5XtNhPdHZ05hdZv2De9nl8X/h9k+pOHpNGP7eRR2fmsbqwtIVaKIQQoq117DBe/XHwvV/LhTHATZk3kRyVzJM/PYnX7w25nsGguDPListu4a73llFaFXpdIYQQ7UfHDuPcjyFpGMSmtOjPWI1WHhr2EFsPbuXdte82qW6URfHSdYPZWVLFgx+tlOvHQggRhjpuGO9ZA3vzWmTiVkNGJo1kdNJoXl35KnsqmrYGdXb3WB66sDdfrNnDG99uaaEWCiGEaCsdN4xXTwVlgMyJrfaTvxn6G3wBH08segJ/4Jilu0/otrN7MC6zE0/NWceSrftbqIVCCCHaQscMY62Dj0vsORocCa32s8nOZO4dfC9f7/ia3yz4TZOuHyul+OtVWSTFRjD5/eXsKw99IREhhBCnt44ZxgVLoGRbsy1/2RQ3Zd7EA0Me4IttX3Dv/Hup8lWFXNdpM/OPSYPZX+nhvg9X4A/I9WMhhAgHHTOMV08FoxX6hPaIw+Z2U+ZN/HHEH1lYuJD/N+//Ue4pD7luZpdoHpuQybcbi3nx640t2EohhBCtpeOFsd8Hq6dBxs/AFt1mzbgy40r+MvIvrNy7ktu/uJ2S6pKQ6/58aDKXD+7K819tZMGGohZspRBCiNbQ8cJ467dQsbfVZlGfyIU9LuS5Mc+x8cBGbpl7C0WVoQWrUoo/T+xHeoKD+z5cwa7S0Ie6hRBCnH46XhjnTgVLFKT/rK1bAsCo5FG8cv4r7CzfyY2f30hBWUFI9SItJv4xKZsar5/J7y/H6w+0cEuFEEK0lI4Vxt5qWDsT+lwC5oi2bk2dYZ2H8frPXueg5yA3zbmJzaWbQ6qXluDgySuyWLrtAH+ds66FWymEEKKldKww3vQl1JS26FrUJysrPot/X/Bv/AE/N39+M2v3rQ2p3oQBXbhxRHde/3YLz8/bKCt0CSFEO9Sxwjh3KkS6ocfotm5Jg3rF9eLtC9/GZrJx29zbWL53eUj1fndxXy4f3JW/z9vAfR+uoNrbtAVFhBBCtK2OE8bVB2HDHMi8DIymtm7NcXV3duftcW/jinBx55d3sq6q8eFni8nAs1cN4MELevHpip1c9/qPFJXJoiBCCNFedJwwXj8bfNXQv/UX+miqzo7OvDnuTbpFdeO1va/x1bavGq2jlOLuMWn8Y9Jg1uw6yMSXF7J+d1krtFYIIcSp6jhhnPsRRHcLPqWpHXBHuPnXBf8iyZLE/3zzP8zMnxlSvYv6d+a/d47A6w9wxSvfM3/d3hZuqRBCiFPVMcK4ohjy50O/y8HQfrocbY1mcuJkhiQO4ZHvHmHKuikh1ctKiuHTyWfRLS6S295ezJsLt8jELiGEOI21n2Q6FXnTQftPi4U+mspqsPLy+S8zOnk0jy96nDdy3wipXufoCD765QjO65PIn2au4fefrpZ7kYUQ4jTVMcJ49ccQ3xsSM9u6JSfFarTyt9F/46IeF/H8sud5bulzIZ3p2q0mXrs+mztH9eTdH7dz61uLKa0K/UlRQgghWkf4h3HJDtj+Q3DillJt3ZqTZjaYefKcJ7kq4yr+tfpfPL7ocQK68TNdg0Hx8IV9+OsVWfyQv4/L/7GQbfsqWqHFQgghQhX+Ybz64+B7v9NvoY+mMigDvx/+e27JvIUP13/I7777Hb6AL6S6Vw9N5p3bzmBfhYeJLy/kpy37W7i1QgghQhX+YZw7FboOgbiebd2SZqGU4v7s+7ln0D3M3DyTB755AI/fE1LdEakupt91FrGRFia98SNTl4a2DrYQQoiWFd5hXLQe9uS2i3uLm0IpxR1Zd/DQsIf4avtXTP5qMpXeypDq9nDbmX7XWQxNieOBj1by1znrCARkprUQQrSl8A7j3KmgDMFVt8LQpD6T+L+z/o9Fuxdx55d3UlxVHFK96Egzb986jGuHdeMfOfnc9d4yKj2hDXcLIYRofuEbxloHF/pIOQeiOrV1a1rMxLSJPDPqGdbuX8vln17O19u/Dqme2Wjgicv68fvxfZm7Zjc/f+1H9hysbuHWCiGEaEj4hvHOZXBgS7u8t7ipxnYfy4fjPyTRnsiv5v+KP37/Ryq8jc+YVkpx29k9eOPGIWwuKufSlxayurC0FVoshBCivvAN49ypYLQEn13cAaTGpPL+Re9zW7/bmL5xOlfOuJIVe1eEVPe8PolM/X9nYjQornr1B+bm7W7h1gohhKgvpDBWSo1TSq1XSm1SSj10nDJXK6XWKKXylFLvN28zmyjgh9XTIG0sRMS0aVNak9lo5r7s+3hz3JsEdICb5tzES8tfwhtofKGPPp2dTL/7THp1iuKX7y7llZx8WUJTCCFaSaNhrJQyAi8DFwJ9gWuVUn2PKpMOPAycpbXOBO5rgbaGbut3UL477GZRhyo7MZuPJ3zM+J7jeW3Va9w4+0a2lG5ptF5ClI0pdwxnfFYX/jJnHb+ZugqPT5bQFEKIlhbKmfEwYJPWerPW2gNMAS49qswvgJe11gcAtNZt+6ig1VPB4oCMcW3ajLbksDh4/OzHeXbUs+wo38HVM6/mw3UfNnq2azMbeeGagfzqvHQ+WlrAJS9+x6LN+1qp1UII0TGpxv5yVkpdCYzTWt9eu30DcIbWenK9Mp8AG4CzACPwqNZ6TgPHugO4AyAxMTF7ypTQnkIUivLychwOByrg5czvb2Kfayjr+tzfbMdvK4f6dSpKfaW8u+9d1lWvo29EXya5JuE0Ohutt2Kvj3fWeNhXrRnR2cjPe1mIsZ36NIPm6NPpKBz7JX1qP8KxX+HWpzFjxizVWg9p8Eut9QlfwJXAG/W2bwBeOqrMZ8B0wAz0AHYAMSc6bnZ2tm5O8+fPD35YO0vrPzq1Xj+3WY/fVur6dYr8Ab9+d827OvudbH3OB+foedvmhVSvssann527Tqf/72yd+Yc5+vUF+drj859SW5qrT6ebcOyX9Kn9CMd+hVufgCX6OJkYymlOIZBcbzupdl99BcAMrbVXa72F4Flyekj/q9Dccj+CiDhIHdMmP3+6MigDk/pM4sPxH9LJ3on75t8X0i1QERYjv/5ZL764byRDU2L586y1XPzCt/yQL0PXQgjRXEIJ48VAulKqh1LKAlwDzDiqzCfAaACllBvIADY3YztDU1MO6z+HzIlgNLf6z7cHqTGpvHfRe9ze//Ym3QKV4rbz75uH8vqNQ6j0+Ln29R+594PlslCIEEI0g0bDWGvtAyYDc4G1wH+11nlKqceUUhNqi80F9iml1gDzgQe11q1/6rR+NviqOsRCH6fCbDTzq8G/4q1xb6HR3DTnJl5c/mKjt0AppRjbN5F5vx7FveelMydvN+c+k8PrCzbj9cusayGEOFkhzcbRWs/WWmdorVO11o/X7vuD1npG7Wettf611rqv1rq/1rr5ZmY1Re5UcHaF5OFt8vPtzeDEwUy9ZCqX9LyEf676JzfMviGkW6BsZiO/HpvBl/eP5IyeLh6fvZaLnv+W7/NDWxtbCCHEkcJmBS6T9yDkfxV8brEhbLrV4hwWB38++8/8bfTfKCgvCPkWKIDuruDQ9Rs3DqHK6+e61xdxzwfL2V0qQ9dCCNEUYZNa8UXfQ8DXYRf6OFVju49l2oRpZCdm8+dFf+bur+4O+SlQ59cOXf/qvHTm5u3mvGdz+OeCfBm6FkKIEIVNGCfu+RbcGdApq62b0m4lRCbwyvmv8PCwh/lp909c/unlvLf2PUprGn94hM1s5P6xGcy7fxQjUl08MXsdFz7/Ld9vkqFrIYRoTHiEcWkh0aV50O9KUKqtW9OuKaW4rs91/Hf8f+nm7MZTPz3FmP+O4cFvHuT7wu/xB/wnrN/NFckbNw3lXzcNocbn57o3FjH5/WUydC2EECdgausGNIs1n6LQMkTdjHrG9OTdi95l3f51fLLpEz7b/Blzts6hk70TE1InMDFtIslRycetf16fRM5Kc/PqN/m8kpPP1+v2cu956dx6Vo9W7IUQQrQP4XFmPOwXLB/4BLhS27olYad3XG8eGvYQX1/1Nc+MeobUmFTeyH2Di6ZdxK1zb2Vm/kyqfFUN1rWZjdx3fgbzfj2KM1PdPPX5Oi58fgGrinzyRCghhKgnPMLYaKY0JrOtWxHWLEYLF6RcwKvnv8rcK+Zy76B72V2xm0e+e4Rz/3suf/rhT6wqWtVgyCbHRfLGTUP4981D8Po1f1taw4XPf8tHS3ZQ4zvxsLcQQnQE4RHGolV1snfiF1m/YNZls3jzgjc5t9u5zNo8i0mzJ3HZp5fxdt7bDc7EPrd3Il/cP5Jb+1nQGh6cuoqznprPi19tZH+Fpw16IoQQpwcJY3HSlFIM6TSEx89+nK+v+ppHRzyKw+LgmSXPMPajsdz79b3M3z7/iJW9bGYjI5PMzLnvHN65bRj9ujp59ssNjHjyKx6etoqNe8rasEdCCNE2wmMCl2hzDouDKzKu4IqMK9hcuplPNn3CjE0zmL9jPi6bq27SV8+YnkAwyM9Jj+ec9Hg27S3jX99tZdqyAj74aQejMuK5/ZwenJ3mRsnseCFEByBhLJpdz+ie/Dr719wz6B4WFi5k+sbpvLPmHd7Me5MB8QPIDGQywj8Cq9EKQFpCFE9e3p8HfpbB+4u28/YP27jhXz/RKzGK287uwYSBXbCZjW3cKyGEaDkyTC1ajNlgZnTyaJ4/93m+vOpLHhjyAOWect7f9z4XTL2A11e9fsSCIi6HlXvOS2fhQ2N45qoBGAyK33y8irP/8jXPzdtAcXlNG/ZGCCFajoSxaBXuCDc3Zd7E9EunMzlhMr3jevPC8hcYO3Usf/npL+ws31lX1moycmV2ErPvPZv3bz+DAUkxPDdvI2c+9TW/nbqK9bvlurIQIrzIMLVoVUopekX04s7Rd7J+/3reznubKeum8MG6D/hZys+4JfMW+rj61JU9M83NmWlu8ovKeXPhFqYuLeDDJTs4J93NbWf3YFRGvFxXFkK0e3JmLNpMr7hePHHOE3x+xedc3+d6FhQs4OrPruYXX/yC7wu/P+Ke5dR4B3+e2J8fHjqPBy/oxfrdZdz85mLG/n0BH/y0nWqv3K8shGi/JIxFm+tk78QDQx/gyyu/5P7s+9lcspk7593JVTOvYmb+zCNujYq1W7h7TBrf/fZc/v7zAdjMBh6elsuIJ7/i95+sZtHmfQQCsrqXEKJ9kWFqcdqIskRxa79buaHPDczaMou3897mke8e4YXlL3B9n+u5MuNK7GY7ABaTgcsGJTFxYFcWbdnPOz9u46OlO3jnx20kOq1c3L8L4wd0ZlByjAxjCyFOexLG4rRjNpqZmDaRS1Mv5dvCb3kr7y2eWfIMr618jat6XcWkPpNIiEwAgteVh/d0Mbyni4oaH1+t28tnK3fy7qJt/HvhFrrGRDA+qzPjs7rQr6tTglkIcVqSMBanLaUUI5NGMjJpJHnFebyZ9yZv5b3Ff9b8h/E9x3Nz5s2kxhx+OIjdamLCgC5MGNCFg9Vevszbw2erdvKv77bw2oLNpLgiGZ8VPGPulRglwSyEOG1IGIt2IdOdyTOjnmFH2Q7eWfMO0zdO55NNnzAyaSQ3Z97MkMQhR4Sr02bmiuwkrshOoqTSw9y83cxcuYt/5GzipfmbSEtw1J0xpyU42rBnQgghYSzameSoZB454xHuGnAXU9YHb4m6de6tZLoyuajHRYxOHk03Z7cj6sREWvj50G78fGg3istr+Hz1bj5buZPnv9rIc/M20qezk/FZnbkkqwvdXJFt1DMhREcmYSzapRhbDL8c8EtuzryZGfkzmLJ+Ck8veZqnlzxNz+iejEoexZjkMWS5szAaDi+l6XZYuWF4d24Y3p09B6uZtWoXn63aydNz1/P03PUMSIpmfFYXLs7qTJeYiDbsoRCiI5EwFu2azWTj6l5Xc3WvqykoK+Cbgm+Yv2M+7+S9w5ur3yTWGss5SecwOnk0Z3Y5s242NkCi08atZ/fg1rN7UHCgktm5u5i5chePz17L47PXkt09lgsyExmZES/XmIUQLUrCWISNpKgkJvWZxKQ+kyjzlLGwcCE5BTnk7MhhRv4MzAYzwzoNY3TyaEYnj6aTvdPhurGR3DEylTtGprK1uIJZubuYuXInT8xexxOz15HotHJOejwjM+I5J81NrN3Shj0VQoQbCWMRlqIsUYzrMY5xPcbhC/hYvnc5OTuCwfz4osd5fNHj9I7rzaik4HB2H1cfDCq4Bk6K287dY9K4e0wau0qr+HZDMd9sLOLLNXuYurQApSArKYZR6W5GZsQzMDkGk1HWzxFCnDwJYxH2TAYTQzsNZWinoTww5AG2HNzCNzu+IWdHDq/nvs5rq14jPiKeUcmjGJ00mjM6n4HNZAOgc3QEVw9N5uqhyfgDmpUFJSzYUMSCDUW8NH8TL3y9iSibibNS3XTCS9qBSpJiZRKYEKJpJIxFh6KUomd0T3pG9+SWfrdwoPoA3xZ+S86OHGZvns3UDVOxGW0M7zKcMcljyE7MJsmRhNFgxGhQDO4Wy+Busdx3fgallV4W5hfXhfOcUg9v5c0nNd7OyIzgkPbwHi4iLPIsZiHEiUkYiw4t1hbLhNQJTEidgMfvYcnuJXXXmXN25ABgNVrpEd2D1JhU0mLSSItJIzUmla6OrlzUvzMX9e+M1pr3Z82nKjqFBRuLeX/Rdt5cuBWLycCwlDhGZrhlIpgQ4rgkjIWoZTFaOLPrmZzZ9UweHvYwG0s2smbfGvJL8tlYspGle5Yya/OsuvIRpgh6RPeoC+hKYyWXDerNbWcPpcYX4Kct+1mwoYhvNhQdMRHs7LR4hqbEMiQllp5uBwaDhLMQHV1IYayUGgc8DxiBN7TWTx31/c3A00Bh7a6XtNZvNGM7hWhVSikyYjPIiM04Yn+5p5z80nw2HdjEppJN5Jfk8+POH5mRPwOAVz9+lUhTZN1ZdGqPVB4dlIbDkMnaHYpvN+5j/vq9fLysAICYSDODu8WS3T2WId1jGZAcg80sw9pCdDSNhrFSygi8DIwFCoDFSqkZWus1RxX9UGs9uQXaKMRpw2FxMCB+AAPiBxyxv7SmlKnzpxLVI4r8knzyS/L5puAbpm+aXlcmyhxFamwql52fQbfI/vgre7C+ULFk236+XrcXALNRkdklmiHdg2fO2d3jiI+ytmofhRCtL5Qz42HAJq31ZgCl1BTgUuDoMBaiw4q2RpNqS2V0r9FH7D9QfaDuDHpTSfBsetaWWVR4/wtAijOFUWcO5ZexgzBUp7Jxl4Gl2/bznx+38cZ3WwDoFhfJkO6xZKfEMqR7HOkJMrQtRLgJJYy7AjvqbRcAZzRQ7gql1EhgA3C/1npHA2WE6FBibbF1t1Ud4gv4WL9/PYt3L2bxnsXM3jKbjzZ8BATDeeiAodxwfjb2QAabdxtZsm0/CzYWMW158CqQ02ZicO2wdnb3OAYkRxNpkekfQrRnSmt94gJKXQmM01rfXrt9A3BG/SFppZQLKNda1yil7gR+rrU+t4Fj3QHcAZCYmJg9ZcqUZutIeXk5Dkf4PX0nHPsVjn2Ck++XX/sp9BSysXojG2s2kl+dT7WuBiDBlEC6LZ00axoxgVR2lznYeCDAxhI/O8uD/+0aFXSLMpASbaC700A3p4EkhwGL8dTPnsPx31U49gnCs1/h1qcxY8Ys1VoPaei7UMJ4BPCo1vqC2u2HAbTWTx6nvBHYr7WOPtFxhwwZopcsWRJC80OTk5PD6NGjm+14p4tw7Fc49gmar1/+gJ91B9axZPcSFu9ezNI9Syn3lgPBM+chnYYwNHEoGdED2F5kYsnWAyzbfoC8nQcpq/YBYDQo0uIdZHZx0reLk8wu0fTt4iQ6wtwmfTqdhGOfIDz7FW59UkodN4xDGdtaDKQrpXoQnC19DXDdUT/QWWu9q3ZzArD2FNorRIdmNBjJdGWS6crkpsybjgnnOVvmMHXDVOBwON9w/hB6xfbC4Etg/e4K8nYeJG9nKd9tKq4b3gZIjosgs3M0mV2cZHYNhnRClFXufRaijTUaxlprn1JqMjCX4K1N/9Za5ymlHgOWaK1nAPcqpSYAPmA/cHMLtlmIDqUp4WwymEhxppAWk8YZg1OZdG4aseY0Sg46WbergjW1IT0nb3fd8d0OC327BAO6b2cnmV2cpLjsMklMiFYU0qwPrfVsYPZR+/5Q7/PDwMPN2zQhREMaCueNJRvZeGBj3czt3OJc5mydU1fHbDAHVxFLTuW6/ml0iUxBezqxd38ka3cFz6RfX7AZXyB42cpuMdKnsxOnrmGHbRsZCQ4yEqPkaVVCtBCZgilEO2c0GOkd15vecb2P2F/prWRL6ZYjbq1auXcln2/5vK6M1WilZ3RPBg5OZaKzJzbdheqKeAqLbKzZVc4PO318vX11Xfn4KCsZiQ7SE6Lo1Skq+DkxCqetadeihRBHkjAWIkxFmiPJdGeS6c48Yn+Ft4LNJZsPh3TpJhbvXsxnmz+rKxNhiqBHcg+GuW307NIX7XVSWeVgf2kku/ZV8+Hi/VR5D0/+7BxtIz0xKngG3SmKjMQo0hMc2K3yV4wQoZD/UoToYOxmO/3j+9M/vv8R+8s8ZXWrh9UP6rwtedT4aw4XdIA13Uhnqxu7yYUxEIO3xsmWcjs/rbbhWepEe6PRviiSYoPD2+mJDnolBkM6Nd4hT7IS4iinVRh7vV4KCgqorq5uct3o6GjWrg2/Sdxt2S+bzUZSUhJmswxBdgRRligGJgxkYMLAun05OTmMGjWK0ppS9lTuYU/lHnZX7GZ3xe667T0VOynyLKcqogpzBBz606IwUEU0K3xOfsyPwr/OScAbjfbF4LZ2pnt0N9LdCaS47PSMt5PispMcF4nZaGibfwBCtKHTKowLCgqIiooiJSWlybdalJWVERUV1UItaztt1S+tNfv27aOgoIAePXq0+u+L04dSihhbDDG2GHrF9WqwjNaaMm8Zeyr2HBXU9QN8C5W+CgAqCK6nu2Z/JP7dLgKeOAIeF/jcJNi6khKdTLqrCz3j7fRwO0hxR9IlOkJmeIuwdVqFcXV19UkFsWh+SilcLhdFRUVt3RTRDiilcFqcOC1O0mPTj1uu3FPOropd7CjbwY6yHWw7uI0tJdvZdnA7xdW5aAKUAiuBFfusBHYFQzrgcWH0u0mMSCIlOple7mR6xjvo4XbQw23H7bDI3xuiXTutwhiQ/6BOI/LvQjQ3h8Xx/9u79/Coqnvh49+VuWSSDJkkBBMCCIEj9wiIN+gRUA6iFEWtmFK0IVR88IJW2yJF2vJWtCpqa/vwgNRXBQsHKZQjVayVAxSpoEZeBASMilwSIOQ6yeQ2t/X+MZMxJBMyQMhc/H2eZ57sy9p71m/2zPyy1+y9FpeZLwuasF0eFydqT3Cs+hjHao5xvOY4X1Uc5Uj1UcoaDuHRbsqBcg0FJUa8RWlop++s2qS7kZ5wCVnWblxqy+Q/eNk14gAAGxZJREFUumbSLz2NXmkJ9ExNlGEpRcSLuGQcblarFYfDEe5qCPGdYzKY6J3cm97JvVut83g9lNSVBJL0UfsxCiuOcKz6GCX1Bbh1YyBR76sCqkAXmtGeLnjdVszY6GJKpWtCOqYGxQfeUvqlZTLgkiwGpmeRZE7o9HiFaE6SsRAi4hniDGRZs8iyZnFt92vPWKe1pryhnNK6UsrqyyivL+eo/RTH7Kc5WXOa8oZy7M4yHN6vsbtrwQhfHOeMseiU14JZ2UgyppIan8Ylien0TM6gZ3JX0pKSSDQmYjFaSDAmkGBMwGKwkGDy/zUmYDFaiFNy4Zk4f5KM26C1Zu7cubz77rsopViwYAG5ubmcPHmS3NxcqqurcbvdLF26lNGjR/OTn/yEgoIClFLMnDmTRx99NNwhCPGdoJQiPSGd9IT0dsu6PC7e+t+/k9qvN4VlJzhSVcIJx2lK68qoaqygsrGK0rov+apmN6rs3O7qsBgsgYRtMVoCiTqQwP3rrGYr/VP7M7jrYHp36Y0hTprQRQQn4//z9885cKI65PIejweD4exv6sFZyfzmliFnLdPkb3/7G3v27OGzzz6jrKyMq666ijFjxrB69WomTpzIE088gcfjoa6ujj179lBcXMz+/b6eiqqqqkKutxCi85gMJtLNaYzrN5Lx/UYGLeNodHO8oo7DZVUcqSjjRE01JdU1lDiqqahzUF5Xi1s3QpwTFecC5STO4MQQ78UU78Ft9uI0ufAY3NQbXKDK0Thx6UYaPQ1UO6txeV2Ar3OVQWmDGNR1EIO7DmZw2mD62PpgjIvYr2ZxkcgRb8OOHTuYNm0aBoOBjIwMxo4dyyeffMJVV13FzJkzcblc3HbbbQwfPpy+ffty+PBh5syZw/e//31uvPHGcFdfCHGerPFGBnVPZlD3ZODSVuu11tjrXZy0N3CquoFT9maP6gZOVTRwsroBe72r1bbJFiMZNhO25EpMiSfwGI9TUnOE/WXrcXpXAb4z7AFpAxiU5k/QXQfTN6Uvpji53z+WRWwyDvUMtkln3Y87ZswYtm/fzjvvvMOMGTN47LHH+PGPf8xnn33Ge++9x7Jly1i7di2vvvrqRa+LEKLzKaVISTSTkmj2J+zg6pxuSqobOWmvp6S6wZe8/Y+SajPHTtkoc/TFq8cCXuLMpcRZinEnnmB//Un2nd6AV60BwKBMdE/I5jLbQHK6DeHqHsMY3LU/JoMk6FgRsck43K677jpefvll8vLyqKioYPv27SxevJijR4/Ss2dPZs2aRWNjI7t372bSpEmYzWZ+8IMfMGDAAO6+++5wV18IEWaJZiPZ6Uay05PaLOP2eCl1NJ55Zl3dQIm9gRP2Ok7WHqfMdRinqYgjlmKO1/yTrSc3wl5AGzB5s0iJyyYz4T8wOgzsiYNLbRn0TbuEzOQkulrN0qNZlJBk3Ibbb7+dnTt3MmzYMJRSPPfcc2RmZrJixQoWL16MyWTCarWycuVKiouLyc/Px+v1AvC73/0uzLUXQkQDoyGO7rYEutvavrVKa011vZuT1fWcrKrnUPkRDpUf5GhtIacbv6Zcf0ppw3Ywwv/7pmkbhfYkod1WjDoZS1wKVlMqqeaupCd2JTMpnV62DPqkZJKddgmZyQkdOqiHy+uizlVHvbueenc9de466l2+abfXTc8uPemd3BuzQYbkbCLJuIWme4yVUixevJjFixefsT4vL4+8vLxW2+3evbtT6ieE+G5RSmFLNGFLNDEwM5nryQCuCazXWnOq9hR//2ATtt6ZHLef5kRNCaX15VQ0lFPtrKTO+zVluppSt4vCaqAaONm0fRzak4TydMFMMomGVLqYU0mL70qKJQmz2Y3R4CLO6CROufCqRly6wZdkWyZc97cJtz0GZaBnl55kJ2eTnZJNdnI2fVP6km3LJtncdvN/rJJkLIQQUUwpRXdrd/on9GPc4HFtltNa43A5KK0r42hVCUeqTnHcXsKp2jLK6n23djncldR6D2H3VFNU74H6Ztt7jeA1o7UZ5TVjUPGY4ixYDAlYjMkkmRLIjE+iS3ISKZYkUhOSSEvsQrekLiTHJ5FgTCDRlEgccRyrOcY39m84bD/MN/Zv+PeJfweuMAdIT0gn25ZNvCOe4oPFZNuy6WvrS0ZiRsz2DCjJWAghvgOUUnQxd6GLuQt9U84++IvWmmpnNfb6OuqdBmrr46iq91DucFJe66Si1kmZo5Fyh5OKGifljkaO1Tpxur1B95dkdpJmha5JXtKtZtKSetLV2o/BSWbGXBpPSpIBT1w5NZ5iyhqLOFrzDd/Yv2F/3X52fLwjsJ9EYyLZtuxAcm7626tLr6i/mE2SsRBCiDMopbDF27DF20LeRmtNrdNDuaOR8lqnL3E3n65tpKLWSVFlPXuL7FTUOnF7dZA9ZWBL6ElX63hs7jqGdo/HlFCGNpXQqE5R7Szm38Uf8/bhtwNbGJQBq9mKOc6M2eB/+KdNcaYzlpkMplblmpbFG+J90/7liaZEJvSe0AGvaPskGQshhLhgSims8Uas8UZ6d237CvImTRemldX6z7BrGylzfJu4y2udfF1Uz/EyI+W1aVTWWdG6X7MnbCQuvhRDfClJSeW44p1g9KKNXjwGD06Dh7g4j79jlgbAhRc3Xtx4tAuX14nL48LpdeL0ONG0/scg2ZwsyVgIIUTsan5hWr9uwcts27aNcePGAuDxairrvj3jLqt1UuE/8y5zOKmqc1JZ56SqxoW93kVlnZMGV/BmcwCzIY6URBNpiSZsCSZsCXEkJ0KXBEWSBZLiITmh824Lk2QshBAi4hniFOnWeNKt8UBoHTw1uDyBxFxV5/I/nFTVN5uu860vqnRSVeybbvT/9p2SaOKeKy9iUM1IMhZCCBGTLCYDFpOBjGTLOW3X4PJQVefC0dj+LVodRZJxmLjdboxGefmFECLSWEwGMm2dO5qW9JMWxG233cbIkSMZMmQIy5cvB+Af//gHV1xxBcOGDWP8+PGAr4OQ/Px8cnJyuPzyy1m/fj0AVqs1sK9169YxY8YMAGbMmMHs2bO55pprmDt3Lh9//DGjRo1ixIgRjB49mi+++ALwjUD185//nKFDhzJq1Cj+9Kc/sWXLFm677bbAft9//31uv/32zng5hBBCXGSRe2r27jw4tS/k4gkeNxjaCSczB25+pt19vfrqq6SlpVFfX89VV13FlClTmDVrFtu3byc7O5uKigoAnnzySWw2G/v2+epZWVnZ7r6Lior48MMPMRgMVFdX88EHH2A0Gtm8eTPz589n/fr1LF++nCNHjrBnzx7q6+txuVykpqbywAMPUFpaSrdu3XjttdeYOXNm+y+MEEKIiBe5yTiM/vjHP7JhwwYAjh8/zvLlyxkzZgzZ2b4b5dPS0gDYvHkza9asCWyXmpra7r6nTp0aGHfZbreTl5fHl19+iVIKl8sV2O/s2bMDzdhNz3fPPffwl7/8hfz8fHbu3MnKlSs7KGIhhBDhFLnJOIQz2ObqO2gIxW3btrF582Z27txJYmIi48aNY/jw4Rw6dCjkfTTvrq2hoeGMdUlJ395/96tf/Yrrr7+eDRs2cOTIEcaNG3fW/ebn53PLLbdgsViYOnWq/OYshBAxQn4zbsFut5OamkpiYiKHDh1i165dNDQ0sH37dr75xjckSlMz9YQJE1iyZElg26Zm6oyMDA4ePIjX6w2cYbf1XD169ADg9ddfDyyfMGECL7/8Mm63+4zny8rKIisri0WLFpGfn99xQQshhAgrScYt3HTTTbjdbgYNGsS8efO49tpr6datG8uXL+eOO+5g2LBh5ObmArBgwQIqKysZOnQow4YNY+vWrQA888wzTJ48mdGjR9O9e/c2n2vu3Ln88pe/ZMSIEYHEC3Dvvfdy6aWXcvnllzN69GhWr14dWDd9+nR69erFoEGDLtIrIIQQorOF1M6plLoJeAkwAK9orYO2ISulfgCsA67SWhd0WC07UXx8PO+++27QdTfffPMZ81arlRUrVrQqd+edd3LnnXe2Wt787Bdg1KhRFBYWBuYXLVoEgNFo5MUXX+TFF1+kpkXz+44dO5g1a1bI8QghhIh87SZjpZQBWAJMAIqAT5RSG7XWB1qU6wI8Anx0MSoqYOTIkSQlJfHCCy+EuypCCCE6UChnxlcDX2mtDwMopdYAU4ADLco9CTwL/KJDaygCPv3003BXQQghxEUQym/GPYDjzeaL/MsClFJXAL201u90YN2EEEKI74QLvjdGKRUHvAjMCKHsfcB94LvieNu2bWest9ls1NTUnFc9PB7PeW8bycIdV0NDQ6vjdKEcDkeH7zMSxGJcElP0iMW4YjGmtoSSjIuBXs3me/qXNekCDAW2+e+vzQQ2KqVubXkRl9Z6ObAc4Morr9Qt76s9ePDged8r3PJCp1gR7rgsFgsjRozo0H36hkUb16H7jASxGJfEFD1iMa5YjKktoTRTfwJcppTKVkqZgR8CG5tWaq3tWut0rXUfrXUfYBfQKhELIYQQIrh2k7HW2g08BLwHHATWaq0/V0r9Vil168WuoBBCCBHrQvrNWGu9CdjUYtmv2yg77sKrFR2sVisOhyPouiNHjjB58mT279/fybUSQggRbaQHLiGEECLMInakgWc/fpZDFaEPzuDxeAKjIbVlYNpAHr/68TbXz5s3j169evHggw8CsHDhQoxGI1u3bqWyshKXy8WiRYuYMmVKyPUC3xXJ999/PwUFBYHeta6//no+//xz8vPzcTqdeL1e1q9fT1ZWFnfddRdFRUWBcY2bxkMWQggRmyI2GYdDbm4uP/3pTwPJeO3atbz33ns8/PDDJCcnU1ZWxrXXXsutt956xshM7VmyZAlKKfbt28ehQ4e48cYbKSwsZNmyZTzyyCNMnz4dp9OJx+Nh06ZNZGVl8c47vlu2i4qKLkqsQgghIkfEJuOzncEG0xG3AI0YMYLTp09z4sQJSktLSU1NJTMzk0cffZTt27cTFxdHcXExJSUlZGZmhrzfHTt2MGfOHAAGDhxI7969KSwsZNSoUTz11FMUFRVxxx13cNlll5GTk8PPfvYzHn/8cSZPnszw4cMvKCYhhBCRT34zbmHq1KmsW7eON998k9zcXFatWkVpaSmffvope/bsISMjo9UYxefrRz/6ERs3biQhIYFJkyaxZcsW+vfvz+7du8nJyWHBggU888y5jesshBAi+kTsmXG45ObmMmvWLMrKyvjXv/7F2rVrueSSSzCZTGzdupWjR4+e8z6vu+46Vq1axQ033EBhYSHHjh1jwIABHD58mL59+/Lwww9z7Ngx9u7dy8CBA0lLS+Puu+8mJSWFZcuWXYQohRBCRBJJxi0MGTKEmpoaevToQffu3Zk+fTq33HILOTk5XHnllQwcOPCc9/nAAw9w//33k5OTg9Fo5PXXXyc+Pp61a9fyxhtvYDKZyMzMZP78+XzyySf84he/IC4uDpPJxPPPP38RohRCCBFJJBkHsW/fvsB0eno6O3fuDFqurXuMAfr06RO4x9hisfDaa6+1KjNv3jzmzZt3xrKJEycyceLEwHws9rcthBDiTPKbsRBCCBFmcmZ8gfbt28c999xzxrL4+Hg++uijMNVICCFEtJFkfIFycnLYs2dPuKshhBAiikkztRBCCBFmkoyFEEKIMJNkLIQQQoSZJGMhhBAizCQZXwCr1RruKgghhIgBkoxjgNvtDncVhBBCXICIvbXp1NNP03gw9PGM3R4PFe2MZxw/aCCZ8+e3ub4jxzN2OBxMmTIl6HYrV67k+eefRynF5ZdfzhtvvEFJSQmzZ8/m8OHDACxdupSsrCwmTZrEgQMHAHj++edxOBwsXLiQcePGMXz4cHbs2MG0adPo378/ixYtwul00rVrV1atWkVGRgYOh4M5c+ZQUFCAUorf/OY32O129u7dyx/+8AcA/vznP3PgwAF+//vft/9CCyGE6HARm4zDoSPHM7ZYLGzYsKHVdgcOHGDRokV8+OGHpKenU1FRAcDDDz/M2LFj2bBhAx6PB4fDQWVl5Vmfw+l0UlBQAEBlZSW7du1CKcUrr7zCc889xwsvvMCTTz6JzWYLdPFZWVmJyWTiqaeeYvHixZhMJl577TVefvnlC335hBBCnKeITcZnO4MNJtLGM9ZaM3/+/FbbbdmyhalTp5Keng5AWloaAFu2bGHlypUAGAwGbDZbu8k4Nzc3MF1UVERubi4nT57E6XSSnZ0NwObNm1mzZk2gXGpqKgA33HADb7/9NoMGDcLlcpGTk3OOr5YQQoiOErHJOFyaxjM+depUq/GMTSYTffr0CWk84/Pdrjmj0YjX6w3Mt9w+KSkpMD1nzhwee+wxbr31VrZt28bChQvPuu97772Xp59+moEDB5Kfn39O9RJCCNGx5AKuFnJzc1mzZg3r1q1j6tSp2O328xrPuK3tbrjhBv76179SXl4OEGimHj9+PEuXLgXA4/Fgt9vJyMigtLSU8vJyGhsbefvtt8/6fD169ABgxYoVgeUTJkxgyZIlgfmms+1rrrmG48ePs3r1aqZNmxbqyyOEEOIikGTcQrDxjAsKCsjJyWHlypUhj2fc1nZDhgzhiSeeYOzYsQwbNozHHnsMgJdeeomtW7eSk5PDyJEjOXDgACaTiccff5yrr76aCRMmnPW5Fy5cyNSpUxk5cmSgCRxgwYIFVFZWMnToUIYNG8bWrVsD6+666y6+973vBZquhRBChIc0UwfREeMZn227vLw88vLyzliWkZHBW2+91ars/fffz9y5c1st37Zt2xnzU6ZMCXqVt9VqPeNMubkdO3bw6KOPthWCEEKITiJnxt9BVVVV9O/fn4SEBMaPHx/u6gghxHeenBlfoGgczzglJYXCwsJwV0MIIYSfJOMLJOMZCyGEuFAR10yttQ53FYSfHAshhOgcEZWMLRYL5eXlkgQigNaa8vJyLBZLuKsihBAxL6KaqXv27ElRURGlpaXnvG1DQ0NMJo5wxmWxWOjZs2dYnlsIIb5LQkrGSqmbgJcAA/CK1vqZFutnAw8CHsAB3Ke1PnCulTGZTIFuHM/Vtm3bGDFixHltG8liNS4hhBDfareZWillAJYANwODgWlKqcEtiq3WWudorYcDzwEvdnhNhRBCiBgVym/GVwNfaa0Pa62dwBrgjN4ltNbVzWaTAPnRVwghhAhRKM3UPYDjzeaLgGtaFlJKPQg8BpiBGzqkdkIIIcR3QIddwKW1XgIsUUr9CFgA5LUso5S6D7jPP+tQSn3RUc8PpANlHbi/SBGLccViTBCbcUlM0SMW44q1mHq3tSKUZFwM9Go239O/rC1rgKXBVmitlwPLQ3jOc6aUKtBaX3kx9h1OsRhXLMYEsRmXxBQ9YjGuWIypLaH8ZvwJcJlSKlspZQZ+CGxsXkApdVmz2e8DX3ZcFYUQQojY1u6ZsdbarZR6CHgP361Nr2qtP1dK/RYo0FpvBB5SSv0X4AIqCdJELYQQQojgQvrNWGu9CdjUYtmvm00/0sH1Oh8Xpfk7AsRiXLEYE8RmXBJT9IjFuGIxpqCUdD0phBBChFdE9U0thBBCfBdFXTJWSt2klPpCKfWVUmpekPXxSqk3/es/Ukr16fxanhulVC+l1Fal1AGl1OdKqVbN/kqpcUopu1Jqj//x62D7iiRKqSNKqX3++hYEWa+UUn/0H6u9SqkrwlHPUCmlBjR7/fcopaqVUj9tUSYqjpNS6lWl1Gml1P5my9KUUu8rpb70/01tY9s8f5kvlVIRc31IGzEtVkod8r+/NiilUtrY9qzv1XBqI66FSqniZu+zSW1se9bvy3BpI6Y3m8VzRCkVdGzaSD5WF0RrHTUPfBeQfQ30xde5yGfA4BZlHgCW+ad/CLwZ7nqHEFd34Ar/dBegMEhc44C3w13Xc4zrCJB+lvWTgHcBBVwLfBTuOp9DbAbgFNA7Go8TMAa4AtjfbNlzwDz/9Dzg2SDbpQGH/X9T/dOp4Y7nLDHdCBj9088Gi8m/7qzv1QiMayHw83a2a/f7MpJiarH+BeDX0XasLuQRbWfG7XbN6Z9f4Z9eB4xXSqlOrOM501qf1Frv9k/XAAfx9XwW66YAK7XPLiBFKdU93JUK0Xjga6310XBX5HxorbcDFS0WN//srABuC7LpROB9rXWF1roSeB+46aJV9BwEi0lr/U+ttds/uwtfPwlRpY1jFYpQvi/D4mwx+b+v7wL+u1MrFWbRloyDdc3ZMmkFyvg/hHaga6fUrgP4m9VHAB8FWT1KKfWZUupdpdSQTq3Y+dHAP5VSn/p7X2splOMZqX5I218W0XacmmRorU/6p08BGUHKRPMxm4mvJSaY9t6rkeghf/P7q238pBCtx+o6oERr3VZ/FdF4rNoVbck4pimlrMB64Kf6zME3AHbjaxIdBvwJ+J/Ort95+E+t9RX4Rvx6UCk1JtwV6gj+zm9uBf4aZHU0HqdWtK89MGZutVBKPQG4gVVtFIm29+pSoB8wHDiJr1k3Vkzj7GfF0XasQhJtyTiUrjkDZZRSRsAGlHdK7S6AUsqELxGv0lr/reV6rXW11trhn94EmJRS6Z1czXOitS72/z0NbMDXbNbcuXa1GiluBnZrrUtarojG49RMSdPPBP6/p4OUibpjppSaAUwGpvv/yWglhPdqRNFal2itPVprL/Bngtc3Go+VEbgDeLOtMtF2rEIVbcm43a45/fNNV3jeCWxp6wMYKfy/kfxf4KDWOuhY0EqpzKbfvpVSV+M7dhH7T4ZSKkkp1aVpGt+FNPtbFNsI/Nh/VfW1gL1ZM2kka/M/92g7Ti00/+zkAW8FKfMecKNSKtXfNHqjf1lEUkrdBMwFbtVa17VRJpT3akRpcW3F7QSvbyjfl5Hmv4BDWuuiYCuj8ViFLNxXkJ3rA98VuIX4rhJ8wr/st/g+bAAWfM2HXwEfA33DXecQYvpPfE2Ce4E9/sckYDYw21/mIeBzfFdE7gJGh7ve7cTU11/Xz/z1bjpWzWNSwBL/sdwHXBnueocQVxK+5GprtizqjhO+fyZO4uvCtgj4Cb5rK/4XX9/ym4E0f9krgVeabTvT//n6CsgPdyztxPQVvt9Nmz5XTXdaZAGbzvZejZRHG3G94f/M7MWXYLu3jMs/3+r7MhIewWLyL3+96bPUrGzUHKsLeUgPXEIIIUSYRVsztRBCCBFzJBkLIYQQYSbJWAghhAgzScZCCCFEmEkyFkIIIcJMkrEQQggRZpKMhRBCiDCTZCyEEEKE2f8HKBnxIct1t2oAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "markdown", "metadata": { "id": "erIeE8WHgnkw" }, "source": [ "Call the model's `predict()` method to estimate the probability of each class for each instance" ] }, { "cell_type": "code", "metadata": { "id": "609l6PpVfkav", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "d4066e65-05a8-42d8-a959-7099a69a1dea" }, "source": [ "y_pred = model.predict(X_test)\n", "y_pred" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "array([[2.9123144e-05, 5.6710069e-06, 3.9882616e-05, ..., 2.1425611e-01,\n", " 5.3555961e-03, 6.3845092e-01],\n", " [4.6123072e-04, 1.9474322e-05, 7.7543676e-01, ..., 1.2149670e-10,\n", " 4.4851447e-05, 4.5031276e-09],\n", " [2.1345770e-05, 9.9986351e-01, 4.3233595e-06, ..., 8.8093415e-11,\n", " 8.9394973e-07, 5.7581708e-09],\n", " ...,\n", " [7.6113254e-02, 1.6507634e-04, 2.3978287e-03, ..., 3.3976123e-04,\n", " 7.2901016e-01, 1.4659639e-04],\n", " [1.2318377e-04, 9.9756974e-01, 2.4066707e-05, ..., 2.7917453e-08,\n", " 9.3446106e-06, 3.1744657e-06],\n", " [5.4539222e-04, 1.8302802e-04, 9.8623044e-04, ..., 1.5533124e-01,\n", " 2.5186859e-02, 2.4260342e-02]], dtype=float32)" ] }, "metadata": { "tags": [] }, "execution_count": 21 } ] }, { "cell_type": "code", "metadata": { "id": "3x1qvSBUliNa", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "ac541d08-359d-468f-8d2f-00520a07bece" }, "source": [ "y_pred.shape" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "(10000, 10)" ] }, "metadata": { "tags": [] }, "execution_count": 22 } ] }, { "cell_type": "markdown", "metadata": { "id": "ndL-o4oGgtfR" }, "source": [ "Often, you may only be interested in the most likely class. Use `np.argmax()` to get the class ID of the most likely class for each instance. **Tip**: you want to set `axis=1`." ] }, { "cell_type": "code", "metadata": { "id": "fJLoDvgFgozI", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "4982f1f3-9a5e-4438-ea8d-dabf077fe3a3" }, "source": [ "y_pred = y_pred.argmax(axis=1)\n", "y_pred" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "array([9, 2, 1, ..., 8, 1, 5])" ] }, "metadata": { "tags": [] }, "execution_count": 23 } ] }, { "cell_type": "code", "metadata": { "id": "EzW_v9JJgwwi", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "8ae88235-3d64-4d00-fab3-60651fb01e22" }, "source": [ "from sklearn.metrics import classification_report\n", "\n", "print(classification_report(y_test, y_pred,\n", " target_names=class_names))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " T-shirt/top 0.80 0.81 0.80 1000\n", " Trouser 0.98 0.95 0.97 1000\n", " Pullover 0.79 0.66 0.72 1000\n", " Dress 0.83 0.87 0.85 1000\n", " Coat 0.73 0.77 0.75 1000\n", " Sandal 0.94 0.91 0.92 1000\n", " Shirt 0.58 0.61 0.59 1000\n", " Sneaker 0.90 0.91 0.90 1000\n", " Bag 0.93 0.94 0.93 1000\n", " Ankle boot 0.92 0.94 0.93 1000\n", "\n", " accuracy 0.84 10000\n", " macro avg 0.84 0.84 0.84 10000\n", "weighted avg 0.84 0.84 0.84 10000\n", "\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "4bGeQep6jgjD" }, "source": [ "# 2. Other Activation Functions" ] }, { "cell_type": "markdown", "metadata": { "id": "clcLw2BjirNr" }, "source": [ "## 2.1: Vanishing/Exploding Gradients Problem\n", "\n", "When training a deep neural network with differentiation i.e gradient computations via backpropagation, we find the partial derivatives by traversing the network from the the output layer to the initial layer. \n", "\n", "Using the chain rule, layers that are deeper into the network go through a large number of matrix multiplications in order to compute derivatives.\n", "\n", "n a network of n hidden layers, n derivatives will be multiplied together. If the derivatives are large then the gradient will increase exponentially as we propagate down the model until they eventually explode, and this is what we call the problem of exploding gradient. \n", "\n", "Alternatively, if the derivatives are small then the gradient will decrease exponentially as we propagate through the model until it eventually vanishes, and this is the vanishing gradient problem.\n", "\n", "[Source](https://towardsdatascience.com/the-vanishing-exploding-gradient-problem-in-deep-neural-networks-191358470c11)\n", "\n", "\n", "Activation functions whose gradients contract as they saturate\n", "are typically referred to as saturating nonlinearities; in that\n", "sense, rectified linear units (ReLU) (Nair and Hinton 2010)\n", "are non-saturating and thus help reduce vanishing gradients\n", "in deep feed-forward networks\n", "\n", "[Source](https://arxiv.org/pdf/1902.06704.pdf)" ] }, { "cell_type": "code", "metadata": { "id": "Z5ZBwAOvg6Pm" }, "source": [ "def logit(z):\n", " return 1 / (1 + np.exp(-z))" ], "execution_count": 17, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "QC2SbPR8jLke", "colab": { "base_uri": "https://localhost:8080/", "height": 282 }, "outputId": "a195ab5b-2c38-41ff-9e91-366662aa865e" }, "source": [ "z = np.linspace(-5, 5, 200)\n", "\n", "plt.plot([-5, 5], [0, 0], 'k-')\n", "plt.plot([-5, 5], [1, 1], 'k--')\n", "plt.plot([0, 0], [-0.2, 1.2], 'k-')\n", "plt.plot([-5, 5], [-3/4, 7/4], 'g--')\n", "plt.plot(z, logit(z), \"b-\", linewidth=2)\n", "props = dict(facecolor='black', shrink=0.1)\n", "plt.annotate('Saturating', xytext=(3.5, 0.7), xy=(5, 1), arrowprops=props, fontsize=14, ha=\"center\")\n", "plt.annotate('Saturating', xytext=(-3.5, 0.3), xy=(-5, 0), arrowprops=props, fontsize=14, ha=\"center\")\n", "plt.annotate('Linear', xytext=(2, 0.2), xy=(0, 0.5), arrowprops=props, fontsize=14, ha=\"center\")\n", "plt.grid(True)\n", "plt.title(\"Sigmoid activation function\", fontsize=14)\n", "plt.axis([-5, 5, -0.2, 1.2]);" ], "execution_count": 18, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEJCAYAAACXCJy4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3wURf/A8c+kk0YJAQwtSA8llNDRhKY06b0LShMsFH0UpCoq8iCIoPCToiK91wCPEmmGKsVQIhAkEFqAQHq7+f2xl5BLIyGX3CWZ9+u1r9zuzs18b3P53mRvdlZIKVEURVEKPgtTB6AoiqLkDZXwFUVRCgmV8BVFUQoJlfAVRVEKCZXwFUVRCgmV8BVFUQoJlfALOCGEnxDiO1PHAVmLRQjxtxBiRh6FlLLdVUKIXXnQjo8QQgohSuZBWyOFEDeFEDpTHNNUsQwTQkSYMgYFhBqHn38JIVyBmUBH4CUgDPgb+FJKeUBfpgQQL6UMN1mgelmJRQjxN7BJSjkjl2LwAQ4CrlLK0BTbi6L9PYQZsa0bwHdSynkpttkAJYB7Mhf/+IQQxYH7wARgExAupcyThCuEkEBvKeWmFNuKAE5Syvt5EYOSPitTB6DkyGbAHhgBXAVKAd6AS1IBKeUj04SWljnFkpqU8kketRMH3M2Dpiqi/X3vklLeyYP2MiWljAaiTR1HoSelVEs+XIBigATaPqecH1ovM2m9NLAD7Y/vX+BNtP8KZqQoI4ExwHYgCggEWgHlgH1AJHAWaJCqrR7ABSAWCAamoP8vMoNYSunbSIpleOpY0nk9lfXPuauP4wzQOVUZG2COvs5Y4DrwLuCuf20pl1X656xCS44AI4F7gGWqetcAO7ISh/61GrSl3+6jXy+ZjeN2A5gKLAWeAreAyZkco2HpvE53YAbwdzplI1Ksz9D/DvoB14BwYFvKePXlhqaI+R7wU4pYU7Z7I7129NtGoXVU4vQ/3061X+p/Fxv1x/g6MMjUf3v5eVHn8POvCP3SRQhhl43n/YTW+2sNdAUG6ddTmwqsAzyBU/rHy4ElQH0gBC1JAiCEaIj2h7kFqAP8B/gYGJdJLKuAKkBboBswBC0xZcYR2Au008e2GdgihKiR6jUOQTudURPtP6AwtGTaU1+mFtppsPfSaWMjUFTfRtLrc0Q7XquzGEcPtMQ8S9/OS+m9mGwctw/QEmwD4CtgrhCiWXp1AuuB9vrHjfVtB2dQNj3uQF+gO/Aa2u/78xQxj0L78FkJ1EU7pfi3fncj/c+39e0mrRsQQnQHvgMWALWBhcASIcQbqYpOQ/tg9dS/rhVCiArZeC1KSqb+xFHLiy9oyesREAP8CcwDmqQq44e+Vw1UR+s1NU2xvzyQSNoe/hcp1mvrt01Isc2HFD1V4Ffg91RtzwBuZRBLNf3zW6TYXzF1LFk8Dv7AVP3jqvp622dQ1iDuFNtXoe/h69e3AL+kWB8EPAHsshKHfv0GMCmz9rN43G4Aa1OV+SdlW+nE4qVvxz1VvVnp4ccARVNsmwJcTbF+C+17oozalkCv57RzFFiRzu/gSCbvQyu0/zhVL/8FF9XDz8eklJsBN+ANtN5mc8BfCPFJBk+pAejQeuxJdQSj9dZTO5/i8T39zwvpbCul/1kT7Y84pSNAWSGEczr119THciJFLP9mEEsyIYSDEGKuEOKiEOKxfuSHF5DU66uvr/dgZvVkwWqgmxDCXr8+ENgspYzJYhxZldXjdj5VmRCeHXtj+1cafqeR3JYQohRQFvgth21k9Lo9Um1Lft1SygTgAbn3ugs8lfDzOSlljJTygJRylpSyOdpplxn60SA5EZ+ymUy2ZeU9lNlolOyOVJkH9AY+RfuCuh7ah0ZOX29qu4EEoKs+ybXl2emcvIoj5bGJT2dfdv9+dYBItc06nXLGaOtFpX4/mDKWAkcduILnItq/vumd17+M9jtvmLRBCFEO7b+EnLoEtEi1rSXaqYn0hmEmxdI4RSwVshBLS+BnKeVmKeV5tNMLlVPsP6uvt1UGz4/T/7TMrBEpZSzaufWBaOez76KdkspqHEltZdoO2T9uOfEAKC2ESJn062WnAqkNq7wNtMmkWDwv/rovZiceJXtUws+nhBAuQojfhRCDhBB1hRCVhBC9gQ+B36SUT1M/R0p5BW2UzQ9CiKZCiHpoX7xFkf2edmr/BbyFEDOEENWEEAOBicDc9ArrY/EFlgohmuljWcXzh+4FAt2FEA2EEHXQet3JH25SykBgA/CjEKKn/ri8IoQYrC/yL9pr7SSEcNV/GZuR1cDrwGi0c+i6rMahdwN4RQhRNpMLrbJ13HLID+0agE+EEJWFECOAXi9Qz+fA+0KID/Qx1xNCTEyx/wbQRghRRn89QHq+BgYLId4RQlQVQoxH+3DNjdet6KmEn39FoH1J+B7wBxCANhRxDVqPNCPD0HqjfmjDM39Fu0AnJifBSCnPoJ3i6In+4i/9ktmVtcOAIOB3YKc+9hvPaWqCPt7DaN9b+OsfpzREX9e3aP9JrEIbdYOU8jYwHS1p3XtOfIfRerMeGJ7OyWoc09C+FL+G1rtO4wWP2wuRUl5CG247Eu3ceDu090x26/keeAdtJM7faB/ctVIUmYj2H1Yw8FcGdWwDxqONPrqI9j4eK6Xcmd14lKxTV9oWcvqeZwjQX/8lsKIoBZS60raQEUK0BpzQRtyUQuvphqL10hRFKcCMckpHCLFCCHFfPw9KevsHCiHOCyEuCCGOCSE8jdGu8kKsgc/QEv5OtPP3r0opI00alaIouc4op3SEEK+inVP+WUpZO539zYFLUsrHQogOaBfWNMlxw4qiKEqWGeWUjpTykBDCPZP9x1Ks+qPNyaIoiqLkIVOcwx+BNqohDSHESLQRBBQpUqRh+fLl8zKudOl0Oiws1GAmUMciSXBwMFJKKlRQU7pA3rwvQmNDeRT3iJK2JSlhUyJX28oJc/gbCQwMDJVSuqa701hzNKBNuPT3c8q0QrvgwuV59TVs2FCag4MHD5o6BLOhjoXG29tbenp6mjoMs5Hb7wvff3wlM5Bv73hb6nS6XG0rp8zhbwQ4JTPIq3nWwxdC1AV+BDpIKR/mVbuKouRvbV9uy5KOS3irwVsYXiSsZFee/O+hv2R+CzBYaldCKoqiZOrSg0vcfnobSwtLxjQag7VletP+KNlhlB6+EGIt2rSvJYUQt9CuZLQGkFL+gHbFoQvafNcACVJKL2O0rShKwXM34i7tf21PaYfSHH/ruOrZG4mxRun0f87+t4C3jNGWoigFW2RcJG+sfYPQqFC29Nmikr0RqSttFUUxG4m6RAZuGciZO2fY1ncbDd0aPv9JSpaphK8oitmYd2we269s59v23/JG9dR3O1RySiV8RVHMxiivURSzK8Yor1GmDqVAUlfRKIpicidvnyQ6Plol+1ymEr6iKCb1152/aPVTK973fd/UoRR4KuErimIyt57eovPazpQoUoIZPjNMHU6Bp87hK4piEk9jn9JpTSfCY8M5OvwoLzm9ZOqQCjyV8BVFMYnRu0YTcD+APQP3UKd0HVOHUyiohK8oiklM955Ol+pdeK3ya6YOpdBQ5/AVRclT/rf8kVJSvWR1+tXuZ+pwChWV8BVFyTObL26m2fJmLDu9zNShFEoq4SuKkif8b/kzaOsgmpVrxhDPIaYOp1BSCV9RlFwX9DiILmu74ObkxvZ+2yliXcTUIRVKKuEripKrEnWJdF3XlQRdAnsG7MHVIf277ym5T43SURQlV1laWDK33VyKWBWhesnqpg6nUFMJX1GUXCGl5PSd03i5edG+SntTh6OgTukoipJLZh+aTeP/a4z/LX9Th6LoqYSvKIrRrT6/mul+0xnsOZgmZZuYOhxFTyV8RVGM6tC/hxixYwQ+7j783xv/p25RaEZUwlcUxWjuRtyl27puVCpWiS19tmBjaWPqkJQU1Je2iqIYTWmH0sxqNYuOVTtSvEhxU4ejpKISvqIoORaTEMO/Yf9SvWR1xjUeZ+pwlAwY5ZSOEGKFEOK+EOLvDPYLIcS3QoirQojzQogGxmhXURTT00kdw7YNo+nypoRGhZo6HCUTxjqHvwrIbKBtB6CqfhkJfG+kdhVFMbHlQctZH7CeT1p+Qkn7kqYOR8mEUU7pSCkPCSHcMynSFfhZSikBfyFEMSHES1LKOxk94cqVK/j4+Bhs69OnD2PHjiUqKoqOHTumec6wYcMYNmwYoaGh9OrVK83+MWPG0LdvX4KDgxk8eHCa/RMnTuSNN97gypUrjBql3Ug5LCyMYsWKATB16lTatm3L2bNnef/9tPffnDNnDs2bN+fYsWN88sknafYvWLCAevXq8b///Y/PPvsszf6lS5dSvXp1du7cyX//+980+3/55RfKly/P+vXr+f77tJ+ZmzZtomTJkqxatYpVq1al2b9nzx7s7e1ZsmQJGzZsSLPfz88PgHnz5rFr1y6DfUWKFOGjjz4CYPbs2fz2228G+11cXNi8eTMAH3/8MX/++afB/nLlyrF69WoA3n//fc6ePWuwv1q1aixbps2gOHLkSAIDAw3216tXjwULFgAwaNAgbt26ZbC/WbNmfPHFFwD07NmThw8fGuxv06YNn376KQAdOnQgOjraYH/nzp2ZNGkSQJr3HRi+986ePUtCQoJBudx476Vkru+9Oy/dIbBGIENrDWVS80m59t7bu3cvYP7vvWnTpmFhYdiPNuZ770XyXkp5dQ6/LBCcYv2WfptBwhdCjET7DwBra2vCwsIMKgkMDMTPz4+YmJg0+wAuX76Mn58fT548SXd/QEAAfn5+3L9/P939Fy5cwMnJiZs3bybvT0xMTH587tw5rKysuHr1arrPP3PmDHFxcfz999/p7j916hRhYWGcO3cu3f3Hjx/nzp07XLhwId39f/75J9euXSMgICDd/UePHqVo0aJcvnw53f2HDh3Czs6OwMDAdPcn/dFdu3Ytzf7o6GgiIiLw8/MjKCgozX6dTpf8/JTHL4m1tXXy/lu3bqXZHxISkrw/JCQkzf5bt24l7793716a/Tdv3kze/+DBA54+fWqwPygoKHn/o0ePiI2NNdh/7dq15P3pHZuU772EhASklAblcuO9l5I5vvcii0dytdpVHO860sWzC3/88UeuvfeS9pv7ey8hIYGoqCiD/S/63pPSAp3OnjNn7rF69XHCw+O5fbs8Utqi0xVBp7NFp7Nl3TpnTp26SlhYHJcuDQD+ICNC63TnnL6Hv0tKWTudfbuAL6WUR/TrvwEfSSlPZVSfl5eXPHUqw915xs/PL91P3cJIHQuNj48PYWFhaXqKhU1cYhyfHfqMJolN6NS2k6nDMQtJfyNSQkQEPHwIjx5pS8rHT59qS3i4tqT3ODLyRaMQp6WUXuntyase/m2gfIr1cvptiqLkM3cj7mJtYY2LvQuzWs1K7p0WdLGxcP8+3L1ruNy79+xxcHAjYmK0pJ6QkPM2nZzA0RHs7bWlSJFnP1M+Trlt+vSM68urhL8DGCeEWAc0AZ5kdv5eURTzFBkXSec1ndFJHadGnsJCFIxrN6WEBw/g33/h5k3DJWnbgwdZqcnh2SMHKFECXFy0nymXokW1ZO7sbPgz5WMHB7DIxuE9cOAAlStXzv2EL4RYC/gAJYUQt4DpgDWAlPIHYA/QEbgKRAFvGqNdRVHyTqIukQFbBvDX3b/Y3m97vkv2SUn9n38gMPDZz8BAuHoVUn2XmoalJZQuDWXKGC5J20qXhuvXT9ChQ2OKFwdb27x5XQBLlizhnXfeSf7yOCPGGqXT/zn7JfCOMdpSFMU0Ju6fyI4rO/i2/bd0rtbZ1OFkKjwc/v4bLlwwXB49yvg5xYtDxYpQocKzJeV6mTLP73FLGUWZMsZ9LZm3J5k9ezZfffUVoH1BnBl1pa2iKM+14q8VLDy+kPeavMf4JuNNHY6BiAg4fRpOnIDjx7XHN26kX7ZoUahWDapWffYzadGPvs43pJS8++67rFixInlkkEr4iqLkWKeqnfik5SfMajXLpHFICVeuwKFDWnI/cQIuXgSdzrCcjQ14eECdOoaLmxsUhMk7ExISGDx4MDt27DAYBpr6GoHUVMJXFCVD1x9fp7xzeUo7lubzNp/neftSaufXDx4EPz/t5927hmWsrKB+fWjSBBo3Bi8vqF5d214QxcTE0K1bNw4fPpxmzP/9+/czfW4BPSSKouRU8JNgWq5oSYcqHVjedXmetfvkCezbB7t3w2+/we1UA7hLlQIfH2jeXEvy9eqBnV2ehWdS4eHhtG3blgsXLqS5YhfSv2gwJZXwFUVJ42nsUzqt6URkfCQfNPsg19sLDISdO7Ukf/iw4Rh2FxctwbdqpS01axaM0zLZ9eDBA7y9vbl+/Xqaq3WTWFtbk5iYmGFeVwlfURQD8Ynx9N7Ym0uhl9gzYA+1S6W5eN4o/vkHNmzQlvPnn223tIRXX4XOneH116F27eyNRy+IgoODadGiBXfv3iU+Pj7DcjY2NsTExFhntF8lfEVRDEw+MJn91/bz4xs/0q5yO6PWfesWrF4N69dDypkpihbVEnxSki+u7p2S7MaNGzRq1IjHjx+TmJiYladkeJsxlfAVRTEw1HMobk5ujGgwwij1xcbC9u2wciXs3/9sRI2zM3TrBn36QNu2eXuhUn7y9OlTXFxciIqKIi4ujoRM5mzQ9/5VD19RlMxdf3ydl4u/TP2X6lP/pfo5ru/yZViyBH799dkFTzY20LUrDBqk9eRVkn++unXrcvnyZf7++2+WLl3K999/n2FPX/9FboY9/EJ+ZkxRFAD/W/7UWlKLxScW56genQ727IH27bUvVxct0pJ9vXrw7bcQEqKds+/SRSX77KpduzaTJ0/G2jrDDnySDI+s6uErSiF3/fF1uqztQlmnsvSt3feF6oiMhOXLtQR/9aq2rUgRGDwYRo/WxskrObdy5UpST2lfrFgxSpYsSUhICDExMeh0OtXDVxQlrcfRj+m0phOJMpE9A/dk+xaFT57A6tUVcHeH997Tkn2FCjB3rvYF7dKlKtkbi5SSH374wWBIpq2tLe+++y7//PMPx44dS7pbWlRGdaiEryiFlJSSnht6cv3xdbb13UY1l2pZfm5oKHz6qTa52PLlLxMaql0EtWkTXLsGkydr0wArxnP06FEiIiLSbB8xQvty3dPTkyVLloDh3QUNqFM6ilJICSEY7TWatxq8xSsVX8nSc8LDYd48+O9/n92RqV69x8ybV5zWrQvnBVF55fvvvycy1W2wPD09qVChQpbrUAlfUQqhm09uUqFoBfrU6pOl8nFxsGwZzJr17EYg7dvD1KkQH39O3foyl0VGRrJ161aD8/eOjo6MGzcuW/WoUzqKUsisPr+aqouqcvjfw88tK6V2kZSHB4wfryX75s216Q/27oUWLfIgYIXNmzdjaWlpsC0xMZGePXtmqx7Vw1eUQuSPG38wfPtwWlZoSZNyTTIte/48vPMOHDmirdeoAV98oY2jV6du8ta3335rcP5eCEGPHj2wt7fPVj2qh68ohcSV0Ct0X9+dyiUqs7nPZmws0x+99/QpfPABNGigJftSpbTTORcuaFfGqmSft27cuEFAQIDBNgcHB8aMGZPtulQPX1EKgbCYMDqu6YiVhRW7B+ymeJG0k9VICWvXwsSJ2pzzFhYwbhzMnp3/7gZVkKxYsQJdqju8ODs707x582zXpRK+ohQCzrbODK47mPZV2vNy8ZfT7A8OhpEjwddXW2/WDBYvVmPoTU2n07F06VLi4uKSt9nZ2TFmzBjEC/yrpRK+ohRgOqnjbsRd3JzcmOEzI81+KWHFCpgwQTuVU6IEfP01DBumpiQ2B+nd1Qpg2LBhL1Sf+pUqSgH2yW+fUO+HeoSEh6TZFxwMHTvCW29pyb5rVwgIgOHDVbI3F0uWLEkz9r5BgwaUK1fuheozyq9VCNFeCHFFCHFVCPGfdPZXEEIcFEL8JYQ4L4ToaIx2FUXJ2P+d/j++OvoVPWv25CXHlwz2/fqrdmMRX1+tV//rr7B1K5QpY6JglTQiIiLYsWOHwdh7Jycnxo8f/8J15jjhCyEsgcVAB8AD6C+E8EhVbCqwQUpZH+gHLMlpu4qiZGzf1X2M2T2G9lXas6jjouTzvRER2umaQYMMe/UDBqjRN+Zmz549ab6sTUxMpFu3bi9cpzF6+I2Bq1LK61LKOGAd0DVVGQk46x8XBdL+f6koilFcenCJ3ht7U6tULdb3Wo+VhfZV3V9/QcOG8NNP2kyWP/6oevXmrEOHDnz22WdUrFgRBwcHLC0t6d27N3Y5uGO7SD3VZrYrEKIX0F5K+ZZ+fTDQREo5LkWZl4D9QHHAAWgrpTydTl0jgZEApUuXbrhu3bocxWYMERERODo6mjoMs6COheb9998nMTGRRYsWmTqUdEUnRrP42mKGVhyKq60rUsKWLWVZurQy8fEWVKoUwbRpF3F3z3BSxWxR74tncuNYSCkJDAxk//79dO3a9blz57Rq1eq0lNIrw8pysgC9gB9TrA8GvktVZgIwUf+4GXARsMis3oYNG0pzcPDgQVOHYDbUsdB4e3tLT09PU4eRRkRshHwa89RgW3i4lD17SqmNx5Fy9Ggpo6KM2656XzxjDscCOCUzyKvGGJZ5GyifYr2cfltKI4D2+g+YP4UQdkBJ4L4R2leUQi9Rl0j/zf0JCQ/B/y1/rCysuHpVuzI2IEC7f+zy5dCrl6kjVUzJGOfwTwJVhRCVhBA2aF/K7khV5ibQBkAIUROwAx4YoW1FUYAJ+yawM3Anb9Z7EysLK3x9oVEjLdnXqAEnTqhkrxgh4UspE4BxwD7gEtponAAhxCwhRBd9sYnA20KIc8BaYJj+Xw9FUXLo2+Pf8u2Jb/mg6QeMbfQOX3yhja8PC9PuHXv8OFSvbuooFXNglCttpZR7gD2ptk1L8fgioCZSVRQj2x24mw/2fUD3Gt357NWvGThQmw8HYOZMbb56dRGVkkRNraAo+VjtUrUZVHcQnzX9nvavW3L4MDg6ahdSdeny/OcrxuPj40Px4sXN+mYw6rNfUfKhh1EP0UkdFYtVZFqdn2jzqj2HD0PZstqUxvkl2T948ICxY8fi7u6Ora0tpUuXpk2bNhw4cCBLz/fz80MIQWhoaC5H+syqVavSHXq5ZcsW3n777TyL40WoHr6i5DNPYp7g85MPTco2YYTrj3Tpot1U3NMTdu/Wkn5+0bNnT6Kioli+fDlVqlTh/v37/PHHHzx8+DDPY4mLi8PGJv17BGRFiRIlsn1DkrymeviKko/EJ8bTe2NvLodepvztd2ndWkv27duT3MPPL8LCwjh8+DBffvklbdq0oWLFijRq1IhJkybRr18/AFavXk2jRo1wcnKiVKlS9O7dm9u3tVHfN27coFWrVgC4uroihEieRdLHxyfN/V6HDRtG586dk9d9fHwYM2YMkyZNwtXVlRb6+zXOnz+funXr4uDgQNmyZXnrrbcICwsDtP8o3nzzTSIjIxFCIIRgxowZyfUtXLgwuX53d3c+++wzRo0ahbOzM+XKlePrr782iCkwMBBvb2/s7OyoXr06e/bswdHRkVWrVhnnIKeiEr6i5BNSSt7Z8w4Hrh9gUMLvzBpXl5gYGDUKdu4EJydTR5g9jo6OODo6smPHDmJiYtItExcXx8yZMzl37hy7du0iNDSU/v37A1C+fHk2b94MQEBAAHfu3DFIuFmxevVqpJQcPnyYn3/+GQALCwsWLFhAQEAAa9as4cSJE8kTljVv3pwFCxZgb2/PnTt3uHPnDpMmTcqw/m+++YY6depw5swZPvroIz788EP+/PNPQJvrvnv37lhZWeHv78+qVauYOXMmsbGx2XoN2ZLRFVmmXtSVtuZHHQuNqa60nXtkrmQGss2o3clXzs6cKaVOl+ehGMjJ+2LTpk2yePHi0tbWVjZt2lROnDhR+vv7Z1j+0qVLEpDBwcHJbQPywYMHBuW8vb3lO++8Y7Bt6NChslOnTgZl6tSp89wY9+7dK21sbGRiYqKUUsqVK1dKBweHNOW8vb1lt27dktcrVqwo+/XrZ1CmSpUqcvbs2VJKKX19faWlpaW8detW8v6jR49KQK5cufK5cWWETK60VT18RcknmpRtSoOLe/ltqTa7+MKFMG1a/p7lsmfPnoSEhLBz5046dOjAsWPHaNq0KXPmzAHgzJkzdO3alYoVK+Lk5ISXlzZFzM2bN43SfsOGDdNs+/3332nXrh3lypXDycmJHj16EBcXx927d7Ndf926dQ3W3dzcuH9fm2Dg8uXLuLm5UTbFebhGjRphkYvjaFXCVxQz9yj6EYmJsParVzizoT2WlvDzz/Duu6aOzDjs7Oxo164d06ZN49ixY4wYMYIZM2bw5MkTXn/9dezt7fnll184efIkvvp7MKa85V96LCwsDOaRB4iPj09TzsHBwWD933//pVOnTtSsWZONGzdy+vRpVqxYkaU202NtbW2wLoRIM+VxXlKjdBTFjF17dI2my1pS6Y8/OOlbDVtb2LAh/wy7fBEeHh4kJCRw9uxZQkNDmTNnDpUqVQK0oY8pJY2qSUxMNNju6urKnTt3DLadO3cOd3f3TNs+deoUcXFxfPPNN1haWgKwa9euNG2mbu9F1KhRg5CQEEJCQnBzc0tuPzc/EFQPX1HM1KPoR3T8pQtP133HSd9qODjA3r0FJ9k/fPiQ1q1bs3r1as6fP09QUBAbN25k7ty5tGnTBg8PD2xtbfnuu++4fv06u3fv5tNPPzWoo2LFiggh2L17Nw8ePCAiIgKA1q1bs3fvXnbs2MGVK1eYMGECwcHBz42patWq6HQ6FixYQFBQEGvXrmXBggUGZdzd3YmJieHAgQOEhoame8/ZrGjXrh3Vq1dn6NChnDt3Dn9/fyZMmICVldUL3aA8K1TCVxQzFJsQS7c1vbj64wzizvXEyQn27QP9KMQCwdHRkaZNm7Jw4UK8vb2pVasWn3zyCQMGDGD9+vW4urry008/sW3bNjw8PJg5cybz5883qKNs2bLMnDmTKVOmULp06eShmMOHD09eWrRogZOTE927d39uTHXr1mXhwoXMnz8fDw8PfvzxRz1Xv8sAACAASURBVObNm2dQpnnz5owePZr+/fvj6urK3LlzX+j1W1hYsHXrVmJjY2ncuDFDhw5lypQpCCFydJOTTGX0ba6pFzVKx/yoY6HJ7VE6Op1ODtgwVOKxXoKUTk5SHjuWa83lmHpfPJPTY3H27FkJyFOnTr1wHeTyfPiKohhRQoLg8g/T4OLLODvD/v3QpImpo1Jyw9atW3FwcKBq1arcuHGDCRMm4OnpSYMGDXKlPXVKR1HMyMOIJ/TvD2d+f5miReHAAZXsC7Lw8HDGjRuHh4cHAwcOpGbNmuzbty/XzuGrHr6imInfrvnRodd94s/2oWhR+N//wCv9O5MqBcSQIUMYMmRInrWneviKYgYuPbhMxwE3iD/bB0dHia+vSvaK8amErygmdi/iPk17/UnciWHY2unYtUvQtKmpo1IKIpXwFcWEouOjadB3D08PvYmVtY5tWy3w9jZ1VEpBpRK+opjQ/K9tCNkzDAtLHRs3WNC+vakjUgoylfAVxUQWfR/D1CmWCAG//GxBt26mjkgp6FTCVxQTGP31ft59R5tYa/FiGDDAxAEphYJK+IqSx+b+eoKln7wK0pJp0xMZM8bUESmFhVESvhCivRDiihDiqhDiPxmU6SOEuCiECBBCrDFGu4qS36z/XyAfjagBCXa8NSqOGdMtTR2SUojk+MIrIYQlsBhoB9wCTgohdkgpL6YoUxX4GGghpXwshCiV03YVJb85evYeA7oXh1hnOneP4ofF9vn65iVK/mOMHn5j4KqU8rqUMg5YB3RNVeZtYLGU8jGAlPK+EdpVlHzjzh0Y1KMkughXGr8Szqa19liqzr2Sx4yR8MsCKSeavqXfllI1oJoQ4qgQwl8IoQafKYXGo8eJtG+v40aQJQ0bwv92O2Fra+qolMIor+bSsQKqAj5AOeCQEKKOlDIsZSEhxEhgJEDp0qXx8/PLo/AyFhERYRZxmAN1LDRhYWEkJiZm6VjExwsGjC9B6JU6lCsXydSpZzl9Ou2t9vIz9b54xtyPhTES/m2gfIr1cvptKd0Cjksp44EgIUQg2gfAyZSFpJTLgGUAXl5e0sfHxwjh5Yyfnx/mEIc5UMdCU6xYMcLCwp57LKSExh0vEXqlJg4lnnL4sDPu7i3yJsg8pN4Xz5j7sTDGKZ2TQFUhRCUhhA3QD9iRqsw2tN49QoiSaKd4rhuhbUUxW33HXuGUb00sbaM5uM+R59xOVVFyXY4TvpQyARgH7AMuARuklAFCiFlCiKS7b+4DHgohLgIHgclSyoc5bVtRzNXUeTfY+EN1EIls2CBo5KUueVFMzyjn8KWUe4A9qbZNS/FYAhP0i6IUaL6+8OV/KgLw9YJIenRxNnFEiqJR3Q5FMSL/k7H07i1JTBR8/DFMelcle8V8qISvKEZyLSge79ciiIgQDBgAn31m6ogUxZBK+IpiBI8fSxr53CcuzIXqDe+yYgVYqL8uxcyot6Si5FBcHHi1vcnjm2UpWeE+fx4ooy6sUsySSviKkgNSQtteN7h+piJ2xR5z8g9Xihc3dVSKkj6V8BUlB6ZPh8M73bG0jeY3X3vc3dVsaIr5yqupFRSlwPlhWTyzZ1tjYQHbN9vRvIlK9op5Uz18RXkBYdGNGTNGS/BLlkCnTirZK+ZPJXxFyabwSHf+DZoHOisGjglm1ChTR6QoWaMSvqJkw+3bkrOXPod4Z5q+fpOfvyv//CcpiplQCV9Rsig8HLx87qKLLIu1ywkObqugxtor+Yp6uypKFiQkQL9+cPfqS1g6X6dqmcnY2Zk6KkXJHjVKR1GeQ0oYNz6RPXsscXGBl1+eSlzcE1OHpSjZpnr4ivIcH826x9IfLLGx1bF9O9jbhxjsj4qKwtPTk+7du7Nw4UL+/PNPoqOjTRStomRM9fAVJRM/rg7j6xmlAfjvkoe0aOGapkyRIkWIj49n27Zt+Pr6YmNjQ1RUFOXLl6dZs2Z4e3vTqFEjateujbW1dV6/BEVJphK+omTg4OEYRg7XTtS/83Ew44anPyJHCMGcOXMYPHgwERERxMTEABAUFERQUBDbtm3D0tKSmJgYqlSpwiuvvELLli1p0aIFL7/8cp69HkVRCV9R0vHPVR3tO8Ui44vyWp8gFn1eKdPyXbp0oVixYkRERKTZFxUVlfz40qVLXLp0iVWrVuHq6sqtW7eMHruiZESdw1eUVB4+1K6cjQsvSo2mQez+tRLiORfSWlhYMGvWLBwdHbPUhqWlJb/88osRolWUrFMJX1FSiI2Fbt0l/wQKPD0lx/e5Y5XF/4MHDhyIbRbmRba3t2f69Om0atUqh9EqSvaohK8oejodvN4rhCOHBaVfSmDXLoGzc9bnyLGxseGTTz7B3t4+03KWlpYMHTo0p+EqSraphK8oem+9d58/drlhYRvJlu1xlCuX/TpGjRqFxXMuv42OjqZWrVr4+/u/YKSK8mJUwlcUYPbXYaz8rhRYJPDTmmiaN8q8l54RBwcH3nvvPewyuQw3ISGBR48e0bp1a5YsWYKU8kXDVpRsMUrCF0K0F0JcEUJcFUL8J5NyPYUQUgjhZYx2FcUYVq+LZtpHzgDMnB/CoB4lc1TfBx98gEj1LW96HwDR0dFMnjyZgQMHJg/lVJTclOOEL4SwBBYDHQAPoL8QwiOdck7Ae8DxnLapKMZy6BC8NcwOpAVDP7jCtPcq5LhOFxcX3nzzTWxsbADtS9quXbtStGhRLC0tDcpGRUWxbds26tevz82bN3PctqJkxhg9/MbAVSnldSllHLAO6JpOudnAV4Dqyihm4cIFSZcukthYwZgxkpX/rW60uj/++GMsLCywtramUaNGrFmzhgsXLlC9enWKFCliUDY6Opp//vmHOnXq8NtvvxktBkVJzRgJvywQnGL9ln5bMiFEA6C8lHK3EdpTlBwLDoZX2kTw5Imga7cEFi0Szx1rnx3lypWje/fuuLi4sHXrViwsLChfvjynT5+mZ8+eaUbyJCYm8vTpU9544w0+//xzdV5fyRUip28sIUQvoL2U8i39+mCgiZRynH7dAvgdGCalvCGE8AMmSSlPpVPXSGAkQOnSpRuuW7cuR7EZQ0RERJYvpinoCsqxCA+3YsTY6jy45Ypz5fOs++4RRbIx1fH7779PYmIiixYtyrRcVFQUMTExlChRIs2+nTt3snjxYmJjY9Pss7Ozo27dukyfPv25QzzNQUF5XxiDORyLVq1anZZSpv89qZQyRwvQDNiXYv1j4OMU60WBUOCGfokBQgCvzOpt2LChNAcHDx40dQhmoyAci6goKes1eSpByiIvXZO37kVmuw5vb2/p6emZ41iOHz8uXVxcpLW1tQQMFltbW1mhQgV5+fLlHLeT2wrC+8JYzOFYAKdkBnnVGKd0TgJVhRCVhBA2QD9gR4oPlCdSypJSSncppTvgD3SR6fTwFSU3xcdD525RnD3uhGXROxz53YmypUzXg27cuDEXL16kfv36aXrysbGxBAcH07BhQ7Zu3WqiCJWCJscJX0qZAIwD9gGXgA1SygAhxCwhRJec1q8oxpCYCEOGwO/77bF0eMzWXVE0qJF2quO8VqpUKY4ePcqIESPSJH0pJZGRkQwcOJDJkyeTmJhooiiVgsIo4/CllHuklNWklJWllJ/rt02TUu5Ip6yP6t0reUlKGDtWsm4dODnB0d+deaNlZVOHlczKyopvv/2W5cuXp3vOPjo6miVLluDj48PDhw9NEKFSUKgrbfMBHx8fxo0bZ+ow8q2PP5YsWyawsolnxw5Jk8aWz3+SCfTr14/jx4/j5uaWZhK2qKgojh8/Tq1atfjrr79MFKGS3xXYhP/gwQPGjh2Lu7s7tra2lC5dmjZt2nDgwIEsPd/Pzw8hBE+e5N29S1etWpXuN/xbtmzhiy++yLM4CpIvv4SvvhJgEU+vGevx8THi2MtcULt2bS5evEjLli3T9Pbj4+O5d+8eLVu25KeffjJRhEp+VmATfs+ePTlx4gTLly8nMDCQXbt20aFDB5P8SxwXF5ej55coUQInJycjRVN4/PADfPwxgI4W7y5jzX8GmjqkLClatCj79+9n0qRJaS7SAq23P3bsWN5+++0cv7eUQiaj4TumXnIyLPPx48cSkAcOHMiwzC+//CK9vLyko6OjdHV1lb169ZK3bt2SUkoZFBSUZpjc0KFDpZTakLx33nnHoK6hQ4fKTp06Ja97e3vL0aNHy4kTJ8qSJUtKLy8vKaWU//3vf2WdOnWkvb29dHNzkyNGjJCPHz+WUmrDuVK3OX369HTbrFixopw9e7YcOXKkdHJykmXLlpVz5841iOnKlSvy1Vdflba2trJatWpy9+7d0sHBQa5cufKFjmlSjPnFr79KKYROgpRVhsyTMfExRqvbWMMys2L37t3SyclJWlhYpHl/2NvbS09PT3n79u08iSUj+el9kdvM4ViQy8MyzY6joyOOjo7s2LEjw0mp4uLimDlzJufOnWPXrl2EhobSv39/AMqXL8/mzZsBWLlyJXfu3GHhwoXZimH16tVIKTl8+DA///wzoN0VacGCBQQEBLBmzRpOnDjB+PHjAWjevDkLFizA3t6eO3fucOfOHSZNmpRh/d988w116tThzJkzfPTRR3z44Yf8+eefAOh0Orp3746VlRX+/v6sWrWKmTNnpnuRT0G0YYM2IkdKQZlu33D8hzextXr+jUnMUceOHfnrr7+oVKlSmgnYoqKiCAgIoHbt2hw5csREESr5SkafBKZecnrh1aZNm2Tx4sWlra2tbNq0qZw4caL09/fPsPylS5ckIIODg6WUz3rc27ZtMyiX1R5+nTp1nhvj3r17pY2NjUxMTJRSSrly5Urp4OCQplx6Pfx+/foZlKlSpYqcPXu2lFJKX19faWlpmfwfi5RSHj16VAIFvoe/caOUlpZaz37qVCnjE+ON3kZe9vCTREZGyh49ekh7e/s0PX1AFilSRH7zzTdSp9PlaVxS5o/3RV4xh2NBYevhg3YOPyQkhJ07d9KhQweOHTtG06ZNmTNnDgBnzpyha9euVKxYEScnJ7y8tCuRjTVjYcOGDdNs+/3332nXrh3lypXDycmJHj16EBcXx927d7Ndf926dQ3W3dzcuH//PgCXL1/Gzc2NsmWfTWnUqFGj596YI7/bsgX695ckJgq6vf03s2aBlUUW709o5uzt7dm0aROfffZZuuf1o6OjmTJlCn369DG4abqipFSgM4CdnR3t2rVj2rRpHDt2jBEjRjBjxgyePHnC66+/jr29Pb/88gsnT57E19cXeP4XrBYWFklTRiSLj49PU87BwcFg/d9//6VTp07UrFmTjRs3cvr0aVasWJGlNtNjbW1tsC6EQKfTZbuegmLbNujbV5KQIKDlF/QdH2DUydDMgRCCDz74gH379lGsWDGsUt1sNyoqil27duHp6UlYWJiJolTMWYFO+Kl5eHiQkJDA2bNnCQ0NZc6cObz66qvUqFEjuXecJGku89RXN7q6unLnzh2DbefOnXtu26dOnSIuLo5vvvmGZs2aUa1aNUJCQtK0aYyrKWvUqEFISIhB/adOnSqwHwg7dkCfPmjJvsVXfP459KvT19Rh5ZpXXnmFgIAAPDw80vT2Y2JiCA8PTzPvvqJAAU34Dx8+pHXr1qxevZrz588TFBTExo0bmTt3Lm3atMHDwwNbW1u+++47rl+/zu7du/n0008N6qhYsSJCCPz9/Xnw4AEREREAtG7dmr1797Jjxw6uXLnChAkTCA4OTi8MA1WrVkWn07FgwQKCgoJYu3YtCxYsMCjj7u5OTEwMBw4cIDQ09IX/NW/Xrh3Vq1dn6NChnDt3Dn9/fyZMmICVlVWaOzHldxs2QM+e2jw5NP+a4ZP/4eNXMrzpWoHh5ubGyZMn6d+/v8F4fXt7e3x9fdUwXiVdBTLhOzo60rRpUxYuXIi3tze1atXik08+YcCAAaxfvx5XV1d++ukntm3bhoeHBzNnzmT+/PkGdZQtW5aZM2eyfPlySpcunXyl6/Dhw5OXFi1a4OTkRPfu3Z8bU926dVm4cCHz58/Hw8ODH3/8kXnz5hmUad68OaNHj6Z///64uroyd+7cF3r9FhYWbN26ldjYWBo3bszQoUOZMmUKQohM77Wa36xcCf37Q0ICNOt7iDaj9vND5+8L3IdaRmxsbFi+fDmLFi2iSJEi2NnZ8f3331OvXj1Th6aYq4y+zTX1oqZHNq6zZ89KQJ46deqF6zCnY7FokZTaLDlSzp4tpU4nZVxCXJ60bYpROs9z+vRpuWjRIpO0bU7vC1Mzh2NBJqN0CsYQBiWNrVu34uDgQNWqVblx4wYTJkzA09OTBg0amDq0HPvyy6QraKFin4W0H94CIbywtrTO/IkFWIMGDQrE71bJXQXylI4C4eHhjBs3Dg8PDwYOHEjNmjXZt29fvj7dISVMmaIleyEk7oM/54HnJ6YOS1HyDdXDL6CGDBnCkCFDTB2G0cTHw6hR2nl7S0tJ/TELOF3yU7b22IqXW/p3c1MUxZBK+IrZi4iA3r3B1xfs7eG1j1awTU7gm9e/oWuNrqYOT1HyDXVKRzFr9+6Bj4+W7EuWhAP/SyCuyhbGNRrHe03eM3V4BYq7u3uakWNKwaJ6+IrZCgyE9u0hKAgqV4a9eyVVq1qxvcl2BCJffx9hKsOGDSM0NJRdu3al2Xfy5Mk0V4grBUu+7uEfOnSIuXPnEhgYaOpQFCM7eBCaNdOSvZcXLNv2N28fbcWd8DtYWVhhaaGuJDU2V1fXdG+xmNfUHP+5J18n/P/85z9MnTqVevXqUa5cOSZMmMDjx49NHZaSQ99/D6+9Bo8eQefOsHr7bQbvf51rj68hkc+vQHkhqU/pCCFYtmwZvXv3xsHBgZdffpnVq1cbPOf27dvMmjWL4sWLU7x4cTp16sQ///yTvP/atWt07dqVMmXK4ODgQIMGDdL8d+Hu7s6MGTMYPnw4xYoVY+DA/HGjmvwo3yb8p0+fcvr0aeLj44mOjub27dssWrSI69evmzo05QXFx8OYMTB2rHb17Icfwi/rw+m7oxPhseHsHrAbNyc3U4dZqMyaNYuuXbty7tw5+vbty/Dhw5NnlI2KiqJVq1bY2Njwxx9/8Oeff/LSSy/Rtm3b5GlBIiIi6NChAwcOHODcuXP07NmTHj16cPnyZYN25s+fT40aNTh16lTyjLaK8eXbhL9v3740N3p2cHCgfv36JopIyYnQUGjXTrstoa0t/PILfP5FAgO39ePv+3+zsfdG6pau+/yKFKMaPHgwgwYNokqVKsyePRsrKysOHToEwLp165BS8tFHH1G3bl1q1KjB0qVLiYiISO7Fe3p6Mnr0aOrUqUOVKlWYMmUKDRo0YNOmTQbteHt78+GHH1KlShWqVq2a56+zsMi3X9quWbOG8PDw5HUhBF26dCnwc74XRCdParNd3rgBL72kTXXcuDHci3jI9cfXWdJpCa9Xed3UYRZKKe+7YGVlhaura/LMsqdPnyYoKIiOHTsazM4ZFRXFtWvXAIiMjGTmzJns2rWLO3fuEB8fT0xMTJr7OSTdj0LJXUZJ+EKI9sBCwBL4UUr5Zar9E4C3gATgATBcSvnvi7YXHx/P/v37DbY5OTnRr1+/F61SMQEpYfFimDBBO53TqBFs3QpJ920p7Viav0b9hZ1VwZnwLb/J7L4LOp2OevXq8cEHH9CkSRODciVKlABg0qRJ+Pr6Mm/ePKpWrYq9vT1DhgxJ88WsGh2UN3LcHRZCWAKLgQ6AB9BfCOGRqthfgJeUsi6wCXixaSD1jhw5kubmD3FxcbRu3Ton1Sp56MkT6NsXxo/Xkv348XD4sJbst17ayqAtg4hJiFHJ3ow1aNCAq1evUrRoUapUqWKwJCX8I0eOMGTIEHr27EndunUpV65ccu9fyXvG6OE3Bq5KKa8DCCHWAV2Bi0kFpJQHU5T3BwblpMENGzYkz0+fxNvbu0BN/VuQnT2rXTl79So4OcHy5do6wInbJxi4ZSB1S9dNc2cxxTiePn3K2bNnDbYVK1Ys2/UMHDiQefPmMWXKFJycnKhQoQLBwcFs376d0aNHU7VqVapVq8bWrVvp2rUr1tbWzJw5k5iYGGO9FCWbjJHwywIp7wByC2iSQVmAEcDe9HYIIUYCIwFKly6Nn59fmjJSStatW2dw96YiRYpQv379dMvnVERERK7Umx/l9FgkJsKmTeVZvrwS8fEWVK4cwYwZAbi6RuPnB3dj7jL2zFiKWRXjowofcfzocaPFbkxhYWEkJibmy/fF3bt3OXz4cJrBDa+++ioxMTFcu3bN4HUFBARQsmTJ5PXUZb744guWLFlCt27diIyMxMXFhXr16nHx4kVu375N7969+frrr2nRogWOjo706tULDw8P7t69m1xHeu3mV2afLzKaNzmrC9AL7bx90vpg4LsMyg5C6+HbPq/ejObDP3/+vHRwcJBA8mJjYyNDQ0NzPpF0OsxhfmtzkZNjcf26lK+88mwO+1GjpIyKerb/cfRj6bHYQxb7spi8eP9izoPNReY4H74pqb+RZ8zhWJDL8+HfBsqnWC+n32ZACNEWmAJ4SyljX7SxLVu2pLlpeK1atXBxcXnRKpVcJKV2yuaDD7RJ0MqU0dY7djQsd/XRVUKjQtnadys1XWuaJlhFKeCMkfBPAlWFEJXQEn0/YEDKAkKI+sBSoL2U8n7aKrJuzZo1Bt/w29nZqSvzzFRwsHYh1e7d2nrv3tpVtOl9Nnu5eXH93es42KjRGoqSW3I8SkdKmQCMA/YBl4ANUsoAIcQsIUQXfbGvAUdgoxDirBBix4u0dfv2bf79N+1ozm7dur1Y8EquSEiABQugZk0t2RcrBr/+CuvXp032cw7P4csjXyKlVMleUXKZUcbhSyn3AHtSbZuW4nFbY7SzY8cOgws8AEqVKkXlypWNUb1iBKdOwciR8Ndf2nrPnrBw4bOx9SmtubCGKb9PYVDdHA3aUhQli/LVZamrV69OnqMDtCv/+vbta8KIlCSPH8O770KTJlqyr1ABdu6ETZvST/aH/z3Mm9vf5NWKr/LjGz+qqY4VJQ/km4QfHh7OqVOnDLbZ2dnRs2dPE0WkAMTFaT34ypVh0SIQAiZPhosXtZku0xP4MJBu67vhXsydrX23Ymtlm35BRVGMKt/MpePr64utra3BF7aWlpY0atTIhFEVXlLC9u1acr96VdvWujXMnw+enpk/98TtE9hY2rBnwB5KFCmR+8EqigLko4S/du1ag8nSADVZmokcOQJTp8Iff2jr1avD119rPfqsnJkZVHcQXat3xcnWKXcDVRTFQL7IlvHx8ezbt89gm7Ozs5osLY9duOBM27bwyitasndx0U7jXLgAb7yRebLXSR1v7XiLXYHatLkq2StK3ssXCf/IkSNpRufExsaqydLyyLFj2h2o3n23Ab/9Bs7OMH26dipn3DhINaFiuj79/VOW/7WcgPsBuR+woijpMstTOkKIZnXq1Ele37hxI5GRkQZlXn31VTVZWi5KTNTO0c+fD0ePatscHBKYONGK99+H4sWzXteKv1Yw58gc3m7wNh+2+DB3AlYU5bnMMuEDmy5cuECNGjUYMGAAGzduNJgszdHRkUGD1Njt3BARAStXahdOJd0tslgxrSffqJE/Xbq0zFZ9/7v+P0btGsVrlV9jccfFavilopiQuSb8a4DblStX+Pzzz9OczomLi6NTp06miayAunAB/u//tFsLhoVp215+Gd5/H958Exwdwc8vIdv17v1nLzVK1mBDrw1YW2bh3I+iKLnGXBP+SeAVIM2dcQASEhJ4++23GTBgAK+//jpOTuoLwBcRGQkbNsCyZeDv/2x78+YwcSJ07QqpPmuzbd5r83ga+5SidkVzVpGiKDlmrl/anstsuKVOp2Pr1q0MHz4cFxcXfv311zwMLX9LSIADB7Reu5sbDB+uJXtnZxg7VrtK9uhR6NHjxZN9VHwUfTb24dKDSwghVLJXFDNhrj38i88vot0guUyZMmq0znPodHD8OKxdq01gdj/FfKXNmmlz3/TuDca4rWiiLpGBWway/fJ2BtUdpKY6VhQzYq4J/1LKL2nTI4TAxcWFY8eO8dJLL+VRWPlHbCz8/rs20mbHDrhz59m+qlVh4EDo3x+qVTNuu5MPTGbb5W0seH0BXap3ef4TFEXJM2aZ8KWUkdbW1iQkZPwlYbFixTh69CgVK1bMw8jM282b2ukaX19tSXnb33LloE8fGDAAGjTI2hWx2bX4xGK+8f+G8Y3H817T94zfgKIoOWKWCR+0idFS36g8ibOzM4cPH6Zq1ap5HJV5efQIDh/WkvyBAxAYaLjf01P74rVrV6hfP3eSfBKd1LHp0ibeqPYG37z+Te41pCjKCzPbhG9vb59uwndwcODgwYPUqlXLBFGZjpTauPijR7W5bI4e1WakTMnZGVq1gnbtoFMncHfPu/gshAW+A31J0CVgaZHDoT2KouQKs074jo6OBknf3t6e/fv306BBAxNGljfu3oUzZ7RRM6dPa9Mb3LtnWMbWFho1gjZttKkPGjcGqzz+jd56eovJByazpOMSihcpji1qqmNFMVdmm/Dt7OwMZsK0t7dn+/btNG/e3IRRGV9sLPzzD1y6BOfPa0n+zBkt4adWsiS0aKEtLVtq5+JtTZhfn8Y+pdOaTgQ9DuL2K7cpXiQb8y0oipLnzDrhJ93dqkiRIqxfv562bY1yp8Q8J6WWwIOCniX3pOX6dW3emtScnbXz7klL06ba6BpzmZkgQZdA3019CbgfwJ6Be6hdqrapQ1IU5TnMNuFbWlpSvHhxnjx5wsqVK+mc0e2TzIBOp41tDwmBGze0xJ60XL+ubYuJSf+5FhZQpYp2w+9atbRee4MGUKmSts8cSSkZv2c8vld9WdZ5Ga9Vfs3UISmKkgVmm/AB+vXrR8OGDU1y31opITwc0DtnpQAACWBJREFUQkLsOHECQkO1pH7nDty+rSX3pJ9372pXsGbGxUVL4i+/rCX3pKVaNchvk34+jH7Inqt7+KjFR7zd8G1Th6MoShaZdcL/9ttvc/T8xEQtaT95Ak+faj9TPk76GRamJfTUizaNT9MsteXiok1VUKHCs8ReqdKzxdk5Ry/FrJS0L8mZkWfUOXtFyWeMkvCFEO2BhYAl8KOU8stU+22Bn4GGwEOgr5TyRmZ1hoXBunUQFZX9JTJSS+YZDOPPMgcHcHSMoVw5O0qW1L40dXODsmW1n0mPy5TJf730F3Hx6UW2+27n69e+xsXexdThKIqSTTlO+EIIS2Ax0A64BZwUQuyQUqYcJT4CeCylrCKE6Ad8BWR6nubaNe3S/5xycoKiRZ8tzs5p14sVIzmhJy0uLlCkCPj5+ePj45PzQPK5oMdBTP17KsUdizP11akq4StKPmSMHn5j4KqU8jqAEGId0BXDCdC6AjP0jzcB3wkhhJRSZlSppWUEJUr8jqVlDBYWsVhYxGBpmf5PC4vYNNusrKKwtIxCCMMmoqO1Jb1hj+kJCwujWLFiWTwUBVO8VTxnG5wl1jqWKoer0HNLT1OHZFJnz54lISFBdQT01N/IM+Z+LIyR8MsCwSnWbwFNMiojpUwQQjwBXIDQlIWEECOBkQDW1ta4uU3IchBSaufs0xvimBOJiYmEJd0RpBDSCR1BLYKILhKN+yF34h7HEUfaexQUJgkJCUgpC/X7IqXC/jeSkrkfC7P60lZKuQxYBuDl5SVPnTpl4ojAz8+vUPfkToWcwnuVNz93/plyrcoV6mORxMfHh7CwMM6ePWvqUMxCYf8bSckcjkVmtxE1xkjv20D5FOvl9NvSLSOEsAKKon15q5g5Lzcvrr17jUF11T2EFSW/M0bCPwlUFUJUEkLYAP2AHanK7ACG6h/3An7P7Py9YnprLqxh6amlAJRxLGPiaBRFMYYcJ3wpZQIwDtgHXAI2SCkDhBD/3979x1Z112Ecfz/F4ggIiPuZlTFCliXgj6EETcjEwEQEtIRsNKLCZhhZIkkXIQYl8Q8TFhdX7BJMnKtLMNaNMSE4fo2KEohC1VVnmDoohKkrCrpuEHQ20I9/3FPLoD+A2/Z7d8/zSpqe0/ttzpPvH09Pz7nne78pqesTMH4AvE9SK/AVYE2xx7XBc+DVAzyw7QGePvw0FzoH+KaImSUzINfwI2InsPOSn33jou23gPsG4lg2uI786wgLNy1k4tiJbKnZ4qWOzcpIia7WYimcPneaeY3zqFAFO5bsYNyIcakjmdkAKql36Vhau1p30Xa2jb1L9zJp3KTUccxsgLnw7f+WfmgpsybOomp0VeooZjYIfEnHeOTAI+x/dT+Ay96sjLnwc66hpYG1P1/LpsObUkcxs0Hmws+xpmNNPLT9IeZMmkP93PrUccxskLnwc+rwqcPcu/leJt8wmc33baZyWGXqSGY2yFz4OdXQ0sDIypHsWLKD0e8uo09nMbNeufBzav2n1nNo+SHGjxnf/2AzKwsu/By50HmB1XtWc+KNE1SogtvG3JY6kpkNIRd+jqzes5q6g3XsObYndRQzS8CFnxMbfr2B+uZ6aj9ay4qPrEgdx8wScOHnwPYj26ndXUv1ndXUzalLHcfMEnHhl7mI4NFfPsrUm6fSuKjRq1+a5ZjX0ilzktj1+V2c6zjHyOEjU8cxs4R8hl+mzvz3DKteWMW5jnOMGj6Km0bdlDqSmSXmwi9D5zvPU/NcDY83P07LyZbUccysRPiSTpmJCFbuXMnu1t08+ZknuXvC3akjmVmJ8Bl+mXnsV4/xxItPsGbGGpZ/eHnqOGZWQlz4ZeTNt96k7mAdi6csZt3sdanjmFmJ8SWdMjLmujE0L2/mxpE3UiH/LTezt3MrlIHj7cdZt38dndHJhLETGFE5InUkMytBRRW+pHGSmiQdzb6/t4cxd0k6KOllSX+QVFPMMe3t2v/Tzvwfz6fuYB1tZ9tSxzGzElbsGf4aYG9E3AHszfYv9W9gaURMAeYC9ZLGFnlcAzoudLDo2UUce/0YW2u2+vNozaxPxRZ+NbAx294ILLx0QEQciYij2XYbcAq4ocjj5l5E8ODzD7LvxD6eqn6KmbfPTB3JzEqcIuLaf1l6IyLGZtsC2rv2exk/ncIfhikR0dnD6yuArqUc7wReueZwA+d64J+pQ5QIz0U3z0U3z0W3UpiLCRHR40l1v4Uv6WfAzT28tBbYeHHBS2qPiMuu42ev3QLsA5ZFxKErDJ6cpN9GxLTUOUqB56Kb56Kb56Jbqc9Fv2/LjIh7entN0j8k3RIRJ7NCP9XLuNHADmDtO6nszczKSbHX8H8KLMu2lwHbLh0gaTiwFfhhRDxX5PHMzOwaFVv43wI+KekocE+2j6RpkhqyMYuBjwP3S/p99nVXkccdSt9PHaCEeC66eS66eS66lfRcFHXT1szM3jn8pK2ZWU648M3McsKFfxUkrZIUkq5PnSUVSd+W9OdsmYyteXtqWtJcSa9IapXU05PluSBpvKRfSPpjtmxKbepMqUkaJul3kranztIbF/4VkjQemAP8JXWWxJqA90fEB4EjwNcS5xkykoYB3wU+DUwGPidpctpUyZwHVkXEZOBjwJdzPBddaoE/pQ7RFxf+lfsO8FUg13e5I2JPRJzPdg8BeVrAZzrQGhHHI6IDeIbC8iK5ExEnI6Il2z5LoehuTZsqHUlVwHygob+xKbnwr4CkauC1iHgpdZYS8yVgV+oQQ+hW4K8X7f+NHJdcF0m3A1OB5rRJkqqncEJ42ZIxpcQfgJLpZwmJr1O4nJMLfc1FRGzLxqyl8G9941Bms9IiaRTwE+DhiDiTOk8KkhYApyLiRUmfSJ2nLy78TG9LSEj6ADAReKmwPhxVQIuk6RHx9yGMOGT6Wk4DQNL9wAJgduTrQY7XgPEX7VdlP8slSZUUyr4xIrakzpPQDOCzkuYB1wGjJf0oIr6QONdl/ODVVZJ0ApgWEalXxEtC0lxgPTAzIk6nzjOUJL2Lwo3q2RSK/jfAkoh4OWmwBLLVcTcCr0fEw6nzlIrsDH91RCxInaUnvoZvV2sD8B6gKVsm43upAw2V7Gb1SuAFCjcpn81j2WdmAF8EZl20ZMq81KGsbz7DNzPLCZ/hm5nlhAvfzCwnXPhmZjnhwjczywkXvplZTrjwzcxywoVvZpYT/wPxyu/uNIUhPwAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "markdown", "metadata": { "id": "CSGJegGLjqLB" }, "source": [ "## 2.2: Nonsaturating Activation Functions" ] }, { "cell_type": "markdown", "metadata": { "id": "FtymrUn-XMPl" }, "source": [ "### ReLu" ] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 282 }, "id": "LBdvL_pOXXe_", "outputId": "6ba24c09-368e-4d56-9e1f-32f0975b9094" }, "source": [ "plt.plot(z, tf.nn.relu(z), \"b-\", linewidth=2)\r\n", "plt.plot([-5, 5], [0, 0], 'k-')\r\n", "plt.plot([0, 0], [-0.5, 4.2], 'k-')\r\n", "plt.grid(True)\r\n", "props = dict(facecolor='black', shrink=0.1)\r\n", "plt.title(\"ReLU activation function\", fontsize=14)\r\n", "plt.axis([-5, 5, -0.5, 4.2]);" ], "execution_count": 21, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAEJCAYAAAC9uG0XAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAeU0lEQVR4nO3deZhU5Z328e+PphVQsBkJHYK8YFxBjWI6TAxRO2oMr0uMcTQo4IKIGgmGAZUgBDVBFJWIiLhhQFYnGjMazTg40kYmBkVADSBGcQFfjKJU6EZZuvt5/3gOdtn0UjRV9Zyquj/XVRenqk7Vuft09c2ps5pzDhERia9WoQOIiEjTVNQiIjGnohYRiTkVtYhIzKmoRURiTkUtIhJzKmqJHTO7wcz+loXp9DAzZ2ZlWZjWWWb2dzOrNrOZmZ5eM1nKo5+7U8gckjoVdcyZ2czoj8pFf+Tvm9l0M+u4m+/zrpmNauQ5Z2b/1si0/9jS7ClkaqwobwdOTPO0Kszs7noPrwO6ACvSOa1GzAAeA7oDV2dhekCjv/e/4H/uT7KVQ/ZM69ABJCXPAoPwv69ewENACXB+yFCZ4pyrAqqyMJ0a4MNMT8fMSoD9gWeccx9kenrNcc5tJws/t6SPlqhzwzbn3IfOufXOuf8GHgFOTR7BzC4xs1VmttXM3jSzEWaW0d+vme1vZvPNbL2ZfW5mK83sknrjmJmNjL72b4vGnRg9/U7078vRknVF9JovVn2Y2almtt3M9q/3vjeb2Wup5IhWNZwIXJX07aRHQ0v0ZnaCmS2J5uM/zOw3ZrZX0vMVZnZPNP2NZvaRmd3e2Lw2s3JgU3T3uWh65WZ2sZlV1R83eZXEznHM7GQz+5uZbTGzRWZ2YL3XnRZl/tzMPjGzJ82sTTQ/uwO37fy5G5pO9NiPzez16He0zsyuNzNLev5dMxtrZveZ2eZoXl/T0M8s6aeizjFm9nWgH7Aj6bHLgJuBXwI9gZHAdcBPMxynDbAMOAM4ApgC3GdmJyeNczMwDpgYjXMufpUDQJ/o3374r+I/bmAa/wNsjF4H+PIHLgDmpJjjauBF4LfRdLokZfiCmXUF/gQsB3oDl+K/tUysN+oAoBr4DjAM+Dnwkwayg1/NcEQ0fE407b80Mm5D9gZ+AQwGjsN/k7o3KXM/4AlgIfBN4HvA8/i/7R8D64GbqPu5d2Fm3wR+B/weOAoYHU1zWL1RRwCvA8cCtwKTzOy43fhZpKWcc7rF+AbMxJdCFfA54KLbiKRx3gcG1Xvdz4FVSfffBUY1Mg0H/Fsj0/7jbuZdADwYDe8LbAWuaGTcHtG0y+o9fgPwt6T7k4EXku5/F6gBDkglR3S/Ari7qekDE4C/A62SxrkY2Aa0S3qfF+u9z8LkaTWQpVM0nfJ671tVb7zyaLxOSeM44LCkcQZEeSy6/7/AgiamvcvvvYHpzAWea+B3sL7e+8yvN87fgbGh/0YK4aYl6tzwZ+AY/BLoVOBp4C4AM/sK0A2/BFm18wbcAhyUyVBmVhR9RX4t+spdhV+K+z/RKL3wS4T/s4eTmgP0NbPu0f0BwPPOufUp5khVT+CvzrnapMcWA3sBByc99lq91/0/oPNuTitV25xza+pNay9g58bk3uz5/O2JL/xki4GuZtYh6bFs/tySREWdGz5zzr3lnHvdOTccaIdfnQB1v8Mr8GW+83YkdV+5m1MJ7NfA4yXAP5t43Sj8apbbgJOj6f4BXyRp45xbBrwBXGBmxfjVIHOSRslGjuTTTO5o4Lnd/VuqBazeY8UNjFfdSI5s/e2m++eWFtBMzk03AteZ2decc//AL9kcFJX5l24pvt8a/PrNL5hZEXB09Fxjvgs86Zyb7ZxbAbwNHJr0/Gr81/STG3oxsD36tyiFjHPwS9L9gH2AR3cjx85pNTed1cC3620Y/G702rdTyLg7Pgba1VtiPaYF77OcxucvpP5z96332Hfxqz4qW5BJ0kxFnYOccxXAKmBs9NB44NpoT4/DzOxIM7vQzH5R76VfM7Nj6t064dcBDzazq8zsUDM7Brgf+Jfo38a8CZxsZt81s8OBu4Ev9kiI/sinABPN75VykJn1MbMro1E+wq93/4GZlZpZQ0v1O83Fr0r5Fb6UN6eaI/Iu0Cfa06NTI3tp3AN8DbjHzHqa2en4VUh3O+c+ayJbSywBtuDnzcFmdg4t2/g7ATjXzH5tZr3M7Ijoc9Auev5d4Hgz62qNH+ByB3Ci+b1tDjWzAfhvKJNakEcyIfRKct2avtHIBj38Xg/bgO7R/fPxez5sxe8OthjonzT+u9RtiEy+DUt6/VJgM34f26eAbzSTrSN+T4FKfOlOwpddRdI4rfB7EazFL92tAyYkPT8EvzG0ZufrqLcxMWncP0eZf9iCHIfi9/z4LHqPHjSwMRM4AV+i24B/AL8B9k56voJdN0o2+DtKen6XjYnR42fh/5P5HHgGGMiuGxOb3OAYPfZD4JUo80b8XiBtoue+DbwafS5cE+/xY/weHTt/R9cTbbBM+vzU3yi5y7zQLTO3nVuORUQkprTqQ0Qk5lTUIiIxp6IWEYk5FbWISMxl5Ox5nTp1cj169MjEW6dsy5Yt7LPPPkEzxIXmhbdmzRpqamro1atX6CixoM9FnYbmxZtvQmUldOgAhxyS+QyvvPLKRufcVxp6LiNF3aNHD5YuXZqJt05ZRUUF5eXlQTPEheaFV15eTiKRCP7ZjAt9LurUnxcTJ8KYMdC5M7z2GpSWZj6Dmb3X2HNa9SEikmTJEhgXnaBh1qzslHRzVNQiIpF//hPOPx9qauDf/x369QudyFNRi4gAzsFPfwrvvAO9e8PNN4dOVEdFLSICzJ4N8+ZBu3Ywfz7svXfoRHVSLuronL/LLYMXOxURCeGDD9py1VV+eOpUOOywsHnq250l6qvxp0MUEckb27fDr37Vk6oq+MlP4JJLmn9NtqVU1GZ2AHA68GBm44iIZNfYsbBmTQe6d4d77wWrfzmHGEh1ifpO4Fr8VSlERPLCwoVw223QqpVj3jwoKQmdqGHNHvBiZmcAHznnXjGz8ibGGwoMBSgtLaWioiJdGVukqqoqeIa40LzwEokENTU1mheRQv9cJBLFXHppGbA355//Jtu3byCusyOVIxP7Aj80s9OANkAHM5vjnBuYPJJz7n6iq4GUlZW50Ec86airOpoXXklJCYlEQvMiUsifC+fgjDPg00/hhBPgkks2xHpeNLvqwzn3C+fcAc65HkB//GXlBzbzMhGR2LrrLnj6aejYEebMgaJUrtoZkPajFpGCsmIFXHutH54xA7p1C5snFbt1UibnL6pakZEkIiIZtmWLP0R8+3a4/HI4++zQiVKjJWoRKRgjRsAbb0CvXjB5cug0qVNRi0hBePRReOABf2j4ggX+UPFcoaIWkbz3/vtw2WV++Pbb4aijwubZXSpqEclr1dUwYAAkEnDmmXxxTo9coqIWkbw2YQIsXgxdusBDD8XzEPHmqKhFJG+98ALcdJMv5zlzoFOn0IlaRkUtInlp0ya/yqO2Fq67Dk46KXSillNRi0jecc5vPFy3Dvr08UvVuUxFLSJ558EH4bHHoH17f7WW4uLQifaMilpE8srq1XD11X54+nT4+tfD5kkHFbWI5I2tW/0h4p9/DoMG+XXU+UBFLSJ5Y/RoePVVOPhgmDYtdJr0UVGLSF546imYMgVat/ZXE2/fPnSi9FFRi0jO27ABLr7YD0+YAN/6VtA4aaeiFpGcVlsLF14IGzfCKafAqFGhE6WfilpEctodd8Czz/qjDh9+GFrlYavl4Y8kIoXi5ZdhzBg/PHOmP59HPlJRi0hOqqz0u+JVV8Pw4XD66aETZY6KWkRy0rBh8PbbcPTRcOutodNklopaRHLO3Ll+fXTbtv4Q8TZtQifKLBW1iOSUtWvhyiv98JQp0LNn2DzZoKIWkZyxY4dfL11ZCeecA0OGhE6UHSpqEckZ48fDSy9Bt27+QrW5eLWWllBRi0hOeO45uOUWv5/03LnQsWPoRNmjohaR2Nu4EQYO9BcEGDcOjj8+dKLsUlGLSKw5B4MH+/N59O0LY8eGTpR9KmoRibV77oEnn4T99vOrPFq3Dp0o+1TUIhJbr78OI0f64QcegO7dw+YJRUUtIrH02WfQvz9s2+Z3wzv33NCJwlFRi0gsjRwJq1bB4YfDnXeGThOWilpEYufxx+Hee2Gvvfwh4vvsEzpRWCpqEYmVdevg0kv98KRJcMwxYfPEgYpaRGKjpsZfPXzTJjjtNH/6UlFRi0iMTJwIzz8PpaXw298WziHizVFRi0gs/OUvcMMNfnj2bOjcOWicWFFRi0hwiQRccIFf9XHNNfD974dOFC8qahEJyjm44gp47z0oK4Nf/zp0ovhRUYtIUDNnwiOP+F3w5s3zu+TJlzVb1GbWxsxeMrNXzWylmd2YjWAikv/WrIGf/cwP33MPHHJI2DxxlcrpTbYBJznnqsysGFhsZn9yzv01w9lEJI9t2+av1rJli18/PWhQ6ETx1WxRO+ccUBXdLY5uLpOhRCT/jRkDy5fDgQfC9OnaFa8pKZ0w0MyKgFeAg4FpzrklDYwzFBgKUFpaSkVFRRpj7r6qqqrgGeJC88JLJBLU1NRoXkRCfi5eeulfmDz5G7Rq5Rg1ajnLlm0OkmOn2P+NOOdSvgElwCLgyKbG++Y3v+lCW7RoUegIsaF54Z144onu6KOPDh0jNkJ9Lj780LnOnZ0D526+OUiEXcThbwRY6hrp1N3a68M5l4iKul/a/8cQkbxXWwsXXQQffQTf+x5ce23oRLkhlb0+vmJmJdFwW+D7wBuZDiYi+efOO+GZZ2D//f3Rh0VFoRPlhlTWUXcBZkXrqVsB/+Gc+2NmY4lIvlm2DEaP9sMzZkDXrmHz5JJU9vp4DeidhSwikqeqqvyueDt2wFVXwVlnhU6UW3Rkoohk3PDh8OabcOSRcNttodPkHhW1iGTUI4/4U5a2aQMLFkDbtqET5R4VtYhkzDvvwNChfnjyZDjiiLB5cpWKWkQyoroaBgyAzZvhRz/yZ8iTllFRi0hG3HgjvPii37vjwQd1iPieUFGLSNo9/zxMmODLec4cv9+0tJyKWkTS6pNPYOBAf0GA66+H8vLQiXKfilpE0sY5GDIE1q+H446D8eNDJ8oPKmoRSZv77oM//AE6dPBXa2md0vk5pTkqahFJi5UrYcQIP3zffdCjR9A4eUVFLSJ77PPP/SHiW7fCJZdA//6hE+UXFbWI7LFrroHXX4dDD4W77gqdJv+oqEVkjzzxBEybBsXFMH8+7Ltv6ET5R0UtIi32wQcweLAfnjgRjj02bJ58paIWkRapqfFXDv/kE/jBD+o2JEr6qahFpEUmTYJFi6BzZ5g1C1qpTTJGs1ZEdtuSJTBunB+eNQtKS8PmyXcqahHZLZs3+13xamr86o5+utR1xqmoRSRlzsGVV/rzTPfu7TcgSuapqEUkZbNn+0PD27Xzu+LtvXfoRIVBRS0iKXnrLX9hWoCpU+Gww8LmKSQqahFp1vbtfr10VRWcd54/TFyyR0UtIs0aNw6WLoXu3f0Jl3S1luxSUYtIkxYu9PtMFxX59dMlJaETFR4VtYg06uOP4cIL/fD48fCd74TNU6hU1CLSIOfg4ovhww/hhBNgzJjQiQqXilpEGnTXXfD009Cxo79AbVFR6ESFS0UtIrtYsQKuvdYPz5gB3bqFzVPoVNQi8iVbtvhd8bZvh8svh7PPDp1IVNQi8iUjRsAbb0CvXjB5cug0AipqEUny6KPwwAP+0PAFC/yh4hKeilpEAHj/fbjsMj98++1w1FFh80gdFbWIUF0NAwZAIgFnnll3Tg+JBxW1iDBhAixeDF26wEMP6RDxuFFRixS4F16Am27y5TxnDnTqFDqR1KeiFilgmzb5VR61tXDddXDSSaETSUNU1CIFyjkYOhTWrYM+ffxStcRTs0VtZt3MbJGZrTKzlWZ2dTaCiUhmPf10Fx59FNq392fFKy4OnUga0zqFcaqBkc65ZWbWHnjFzBY651ZlOJuIZMjq1TB16sEATJ8OBx0UOJA0qdklaufcBufcsmi4ElgNdM10MBHJjK1b/SHi27YVMWiQX0ct8ZbKEvUXzKwH0BtY0sBzQ4GhAKWlpVRUVOx5uj1QVVUVPENcaF54iUSCmpqagp8Xd999MK++egBdumyhf/9lVFTUhI4UXNz/RlIuajPbF3gM+LlzbnP9551z9wP3A5SVlbny8vJ0ZWyRiooKQmeIC80Lr6SkhEQiUdDz4qmn4LHHoHVr+OUv3+C0044PHSkW4v43ktJeH2ZWjC/puc6532c2kohkwoYN/kIA4A9wOfzwyqB5JHWp7PVhwAxgtXNO59ISyUG1tf6SWhs3wimnwKhRoRPJ7khlibovMAg4ycxWRLfTMpxLRNLojjvg2Wf9UYcPPwytdARFTml2HbVzbjGgI/9FctTLL9dd73DmTH8+D8kt+n9VJI9VVvpd8aqrYfhwOP300ImkJVTUInls2DB4+204+mi49dbQaaSlVNQieWruXL8+um1bmD8f2rQJnUhaSkUtkofWroUrr/TDU6ZAz55h88ieUVGL5JkdO/x66cpKOOccGDIkdCLZUypqkTwzfjy89BJ06+YvVKurteQ+FbVIHnnuObjlFr+f9Ny50LFj6ESSDipqkTyxcSMMHOgvCDBuHByv03jkDRW1SB5wDgYP9ufz6NsXxo4NnUjSSUUtkgfuuQeefBL228+v8mi9WycwlrhTUYvkuNdfh5Ej/fADD0D37mHzSPqpqEVy2GefQf/+sG2b3w3v3HNDJ5JMUFGL5LCRI2HVKjj8cLjzztBpJFNU1CI56vHH4d57Ya+9/CHi++wTOpFkiopaJAetWweXXuqHJ02CY44Jm0cyS0UtkmNqamDQINi0CU47zZ++VPKbilokx0ycCM8/D6Wl8Nvf6hDxQqCiFskhL74IN9zghx9+GDp3DhpHskRFLZIjEgl/VryaGrjmGjj11NCJJFtU1CI5wDm44gp47z0oK4Nf/zp0IskmFbVIDpg5Ex55xO+CN2+e3yVPCoeKWiTm1qyBn/3MD0+bBoccEjaPZJ+KWiTGtm3z66W3bIELLoALLwydSEJQUYvE2JgxsHw5HHggTJ+uXfEKlYpaJKb+679g8mQoKvLrpTt0CJ1IQlFRi8TQP/4BF13kh2+6Cb797bB5JCwVtUjM1Nb6kv7oI/je9+C660InktBU1CIxc+ed8MwzsP/+MHu2X/UhhU1FLRIjy5bB6NF+eMYM6No1bB6JBxW1SExUVfld8XbsgKuugrPOCp1I4kJFLRITw4fDm2/CkUfCbbeFTiNxoqIWiYFHHvGnLG3TBhYsgLZtQyeSOFFRiwT27rswdKgfnjwZjjgiaByJIRW1SEDV1f7Q8M2b4Uc/8mfIE6lPRS0S0I03+osBdO0KDz6oQ8SlYSpqkUCefx4mTPDlPGeO329apCEqapEAPv0UBg70FwQYMwbKy0MnkjhrtqjN7CEz+8jM/paNQCL5zjkYMgTWr4fjjoPx40MnkrhLZYl6JtAvwzlECsZ998Hjj/uz4c2bB8XFoRNJ3DVb1M65PwOfZiGLSN5buRJGjPDD990HPXoEjSM5onW63sjMhgJDAUpLS6moqEjXW7dIVVVV8AxxoXnhJRIJampqgs2Lbdta8dOfHsvWrfvSr98GvvrVNYT8tehzUSfu8yJtRe2cux+4H6CsrMyVB946UlFRQegMcaF54ZWUlJBIJILNi2HDYO1af83D3/2uC/vu2yVIjp30uagT93mhvT5EsuCJJ/yFaYuL/SHi++4bOpHkEhW1SIZ98AEMHuyHJ06EY48Nm0dyTyq7580HXgQOM7P1ZnZp5mOJ5IeaGhg0CD75BE49tW5DosjuaHYdtXPu/GwEEclHkybBokXQuTPMmgWt9B1WWkAfG5EMWbIExo3zw7NmwVe/GjaP5C4VtUgGbN7sr9ZSU+NXd/TTIWOyB1TUImnmHFx5JbzzDvTu7TcgiuwJFbVIms2e7Q8Nb9cO5s+HvfcOnUhynYpaJI3eestfmBZg6lQ47LCweSQ/qKhF0mT7dr9euqoKzjsPLrkkdCLJFypqkTQZNw6WLoXu3f0Jl3S1FkkXFbVIGixc6PeZLiry66dLSkInknyiohbZQx9/DBde6IfHj4fvfCdsHsk/KmqRPeCcXxf94Ydwwgn+sloi6aaiFtkDU6fCU09Bx47+ArVFRaETST5SUYu00IoVcM01fnjGDOjWLWweyV8qapEW2LLF74q3fTtcfjmcfXboRJLPVNQiLTBiBLzxBvTqBZMnh04j+U5FLbKbHn0UHnjAHxo+f74/VFwkk1TUIrvh/ffhssv88O23wze+ETaPFAYVtUiKqqthwABIJODMM+vO6SGSaSpqkRRNmACLF0OXLvDQQzpEXLJHRS2SghdegJtu8uU8ezZ06hQ6kRQSFbVIMzZt8qs8amvhuuvg5JNDJ5JCo6IWaYJzMHQorFsHffr4pWqRbFNRizRhxgy/O1779v6seMXFoRNJIVJRizRi9WoYPtwPT58OBx0UNo8ULhW1SAO2bvWHiH/+OQwa5NdRi4SiohZpwOjR8Oqrfil62rTQaaTQqahF6nnqKZgyBVq39oeIt28fOpEUOhW1SJING+Dii/3whAnwrW8FjSMCqKhFvlBb6y+ptXEjnHIKjBoVOpGIp6IWidxxBzz7rD/q8OGHoZX+OiQm9FEUAV5+ue56hzNn+vN5iMSFiloKXmWl3xWvutrvN3366aETiXyZiloK3rBh8PbbcPTRcOutodOI7EpFLQVt3jy/PrptW78rXps2oROJ7EpFLQVr7Vq44go/PGUK9OwZNo9IY1TUUpB27PDrpSsr4ZxzYMiQ0IlEGqeiloI0fjy89BJ06+YvVKurtUicqail4Dz3HNxyi99Peu5c6NgxdCKRpqmopaBUVxsDB/oLAowbB8cfHzqRSPNSKmoz62dma8zsLTMbnelQIpngHKxb144NG6BvXxg7NnQikdS0bm4EMysCpgHfB9YDL5vZE865VY29ZtMm+N3v0heyJVau/Aoffxw2Q1xoXkBNDSxfDpWVxey3n1/l0brZT79IPKTyUe0DvOWcWwtgZguAs4BGi3rt2jWcd155WgKKpM8KzODAA8u56KLQWcJLJBKUlJSEjhELcZ8XqRR1V2Bd0v31wL/WH8nMhgJD/XA7OnTYkZaALeWcw7QpH9C82Omzz2opLq7FuQSJROg04dXU1JDQjADiPy/S9uXPOXc/cD9AWVmZW7r0f9P11i1SUVFBeXl50AxxoXnhlZeXk0gkWLFiRegosaDPRZ04zIumFqZS2Zj4AdAt6f4B0WMiIpIFqRT1y8AhZnagme0F9AeeyGwsERHZqdlVH865ajMbBjwDFAEPOedWZjyZiIgAKa6jds49DTyd4SwiItIAHZkoIhJzKmoRkZhTUYuIxJyKWkQk5lTUIiIxp6IWEYk5FbWISMypqEVEYk5FLSIScypqEZGYU1GLiMScilpEJOZU1CIiMaeiFhGJORW1iEjMqahFRGJORS0iEnPmnEv/m5p9DLyX9jfePZ2AjYEzxIXmRR3NizqaF3XiMC+6O+e+0tATGSnqODCzpc65stA54kDzoo7mRR3Nizpxnxda9SEiEnMqahGRmMvnor4/dIAY0byoo3lRR/OiTqznRd6uoxYRyRf5vEQtIpIXVNQiIjFXEEVtZiPNzJlZp9BZQjGz28zsDTN7zcweN7OS0Jmyycz6mdkaM3vLzEaHzhOKmXUzs0VmtsrMVprZ1aEzhWZmRWa23Mz+GDpLY/K+qM2sG3Aq8H7oLIEtBI50zn0DeBP4ReA8WWNmRcA04P8CvYDzzaxX2FTBVAMjnXO9gG8DVxXwvNjpamB16BBNyfuiBn4DXAsU9FZT59x/O+eqo7t/BQ4ImSfL+gBvOefWOue2AwuAswJnCsI5t8E5tywarsQXVNewqcIxswOA04EHQ2dpSl4XtZmdBXzgnHs1dJaYGQz8KXSILOoKrEu6v54CLqedzKwH0BtYEjZJUHfiF+RqQwdpSuvQAfaUmT0LfLWBp64HxuBXexSEpuaFc+4/o3Gux3/9nZvNbBIvZrYv8Bjwc+fc5tB5QjCzM4CPnHOvmFl56DxNyfmids6d0tDjZnYUcCDwqpmB/6q/zMz6OOc+zGLErGlsXuxkZhcDZwAnu8Lagf4DoFvS/QOixwqSmRXjS3quc+73ofME1Bf4oZmdBrQBOpjZHOfcwMC5dlEwB7yY2btAmXMu9BmygjCzfsBk4ETn3Meh82STmbXGb0A9GV/QLwMXOOdWBg0WgPmlllnAp865n4fOExfREvUo59wZobM0JK/XUcuX3A20Bxaa2Qozuzd0oGyJNqIOA57Bbzz7j0Is6UhfYBBwUvQ5WBEtUUqMFcwStYhIrtIStYhIzKmoRURiTkUtIhJzKmoRkZhTUYuIxJyKWkQk5lTUIiIx9/8BAmLhq3Pqe68AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "markdown", "metadata": { "id": "rDowraJ8jtw6" }, "source": [ "### Leaky ReLu\r\n", "\r\n", "Proposed in 2013 as an activation function that introduces a small negative slope to the ReLU function to prevent dead neurons" ] }, { "cell_type": "code", "metadata": { "id": "UfleETAejOID" }, "source": [ "def leaky_relu(z, alpha=0.01):\n", " return np.maximum(alpha*z, z)" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "iRMguyFdjwXO", "colab": { "base_uri": "https://localhost:8080/", "height": 282 }, "outputId": "3e28af82-67e0-4d60-cb1c-fac7ac96d133" }, "source": [ "plt.plot(z, leaky_relu(z, 0.05), \"b-\", linewidth=2)\n", "plt.plot([-5, 5], [0, 0], 'k-')\n", "plt.plot([0, 0], [-0.5, 4.2], 'k-')\n", "plt.grid(True)\n", "props = dict(facecolor='black', shrink=0.1)\n", "plt.annotate('Leak', xytext=(-3.5, 0.5), xy=(-5, -0.2), arrowprops=props, fontsize=14, ha=\"center\")\n", "plt.title(\"Leaky ReLU activation function\", fontsize=14)\n", "plt.axis([-5, 5, -0.5, 4.2]);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAEJCAYAAAC9uG0XAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3wU1f3/8deHhEuAINgIIlJovQJaQJFqVYw35AuotdYLKooU0VarUrFaFbVW6gW1WPCKUEBuKurv+63Sb5ViqPilyKVYi4pVRBRRVIwk3EKS8/vjLLCEXDYhmzO7+34+Hvtgdmcy897J7oeTM2dmzDmHiIhEV6PQAUREpHoq1CIiEadCLSIScSrUIiIRp0ItIhJxKtQiIhGnQp0izMyZ2U9D50hlZjbEzIobaFsN8vsys+PN7F9mVmJmBcneXg1ZOsfed6+QOdKRCnU9MLPJZvZS6By1YWZ3xr5UzszKzewzM5tuZh1ruZ4CMxtfxbzVZjayim3/u67ZE8xVWaF8Bvh+PW+nqt99e+DP9bmtKjwMvAUcBPykAbYHVPl7/wT/vpc3VI5MoUKd2Vbiv1gHAhcARwLPBk2URM65Lc659Q20rc+dc9saYFMHA/Occ5845zY0wPaq5Jwri73v0pA50pEKdQMws65m9rKZFZnZejObaWb7x80/xsxeMbOvzGyjmS0ws+NqWOdNseWPj/3MTyvMP93MtptZu2pWUxr7Yn3mnHsdmAAca2at4tZzppktNbOtZvaRmY02syZ13BUJMbMsM5sY294WM/uPmf3azBpVWO4yM3vbzLaZ2RdmNiX2+urYIs/FWtarY6/v7Pows0Nj846ssM7hsf3auKYcZnYncBkwIO6vk/zYvN1a9GZ2pJnNja1nQ6wlvk/c/Mlm9pKZXWdma83sGzP7k5k1r2IfdTYzB+wDTIptb4iZ5cem8youu6NLIm6ZU81skZltNrMlZnZUhW0ca2bzzGyTmX0bmz7AzCYDJwFXx73vzpV1fZhZn9g2tsZ+R3+I//zEWuaPmtnvY/t9vZk9UPF3nem0M5LMzNoDfwf+DfQGTgNaAv8d92HMBZ4GTowtsxyYY2bfqWR9ZmYPAL8ETnLOvQHMBIZWWHQo8JJz7osEc+6P/9O5LPbAzM4ApgPjgW6xdf4U+H1Cb77uGgFrgfOBLsCtwC3A5XF5rwSeAP4E/ADoj9/HAMfE/r0C/xfDjuc7OefeBxYDF1eYdTHwrHNuewI5HsD/BTI3tp32wP9V3JaZtQD+ChTjf7/nAD8CJlVY9ETgCPxn5ILYctdVXF/Mjm6GzcD1selnqli2KvcANwNHAV8D083MYpm7A68BHwDHA8fG1p8dy7QQv+93vO9PKnnfHYC/AP8EegI/AwbFthvvYqAUv0+uib2fC2r5XtKbc06PvXwAk/FFsbJ5dwF/q/BaG8ABvav4GQPWAZfEvebwH94/Ae8DneLm9cJ/0DvErX8LMLCazHfiC3Ix/svuYo+H45b5OzCqws/9OPYzFnteAIyvYhurgZFVbPvftdzH9wJz455/CtxbzfIO+GmF14YAxXHPrwU+jnsv3wXKgR/VIkelv/v47eP/w/gWyI2bnx9b5uC49XwCZMUtMyF+W1XkKQaGVLLevLjXOsde61VhmTPiljk+9tqBsefTgYXVbHeP33sl2xkN/AdoVOF3sA1oHreehRXW8yrwVF2/j+n4UIs6+Y4G+phZ8Y4Hu1ofBwGYWVsze8LM3jezb4EioC2+cMR7AP8lO8E59/GOF51zS4C38X+GA1wEbMC3ZqrzIdAD3+K8FViGbzHGZ7+1QvYZQAtg/4orq09mdlXsz/EvY9sdQWx/mFlboAPwt73czCzgAHxLFnxr7yPn3M5WcXU5aqEL8C/nXFHca/+H/0+ha9xr7zjnyuKef4b/HCTLvypsi7jt9QTm7eX6uwD/cM6Vx722AGiC71uvLMeOLMl83ylHhTr5GgEv4wti/OMQYMdogSn4YjkC/+dfD3yLsWJf8Kv4Atm/ku08hW+tgO+imFLhS1+ZEufcB865Fc653+O/MI9UyP7bCrl/EMv+ZQ3rBtiI70OtqDW+hVkpM7sAGItvZZ4R2+6j7Lk/9orzBxZfZVf3x8X4lmRD5oi/fOX2SubV9ju6oyha3GuNq1g2fns7cjRUTajv953WskMHyADL8H2cHzvf71mZE4BrnXMvA5g/ANi+kuXmAC8QO0jmnJsSN286MMbMrsH3OV5Yh6x3AyvNbJxzbmks++HOuQ/qsC7wo0qOruT1o2LzqnICsMg5t3P4l5kdtGPaObfezNYCp+ILbWW2A1kJZJwGjDezJ/GjXuIPylabI6Ykge28Cww1s9y4VvWP8MXo3QQy1saO/0Dbx033qMN6/gmcUs38RN/3+WbWKK5VfULsZz+sQ6aMpf+16k8rM+tR4dEZ30LdB3jGzH5oZt83s9PM7Ekzy4397PvAJeZHhxyD/5O8pLKNOOdeAs4DHjezS+NeLwSeAx4E/u6c+09t34Bz7kPgv4HfxV66C7jIzO4ysyPM7HAz+6mZ3V/hR/Mqee8HAH8AzjCzUbH31s3MRgPHxeZV5X3gKDP7LzM7xMxG4UcZxBsNXG9mI8yP4OhhZjfEzV8NnGpm+5tZm2q29f/wLc6JwGLnDzLWJsdq4AgzO8zM8sysstbrdPxxgKnmR3/0wR8IfWEv/hOsygf4rrU7Y/ulL3BbHdYzBugZ+5x2j72/YWa2o9tnNdA7NtIjr4pRGo/iu5YeNbMuZjYA38c/3jm3uQ6ZMlfoTvJ0eOD/NHaVPGbH5h8CzAa+wR/kWwmMA5rE5ncHFsXmfQgMxo9guDNuG7sdHAPOjC1/adxrfWLLXZpA5jup5IAevqXniB1QA/oCr+MLzUZgCXBN3PIFVbz3Byr8/Ab8yIICoE8N2ZrgC+c3QGFs+nZgdYXlfga8g/9P7XNgUoX98x98y3p17LUhxB1MjFt2aizztbXNAewHvII/ruCA/Cp+X0fi+9S3xNY3GdinwmfopQrbr/R3VGGZ3Q4mxv0Ol8e2tRAYQOUHE6s84Bh77QT8AeUtsfc/F2gfm3dobN07DkR3rmIdffCf7W3AF/j/oJtW+PxUPCi5x77I9MeOo92SBmJ9qk8ABzi1WETShvqo04D5kyL2x4/YmKAiLZJe1EedHn6N707ZwK7+ZRFJE+r6EBGJOLWoRUQiLil91Hl5ea5z587JWHXCNm3aRIsWLYJmiArtC2/lypWUlZXRtWvXmhfOAPpc7FLZvnj/fSgqglat4JBDkp9h6dKlXznn9qtsXlIKdefOnVmyZEkyVp2wgoIC8vPzg2aICu0LLz8/n8LCwuCfzajQ52KXivvinnvgllugbVv417+gXXXXoKwnZvZxVfPU9SEiEmfRIhg1yk9PmdIwRbomKtQiIjHffguDBkFZGfzqV9CvX+hEngq1iAjgHPziF/DRR9CzJ/w+2VddrwUVahER4OmnYcYMaN4cZs6Epk1DJ9ol4UJt/rZE/7QUu4mriEhN1q7N4eqr/fS4cXDYYWHzVFSbFvV11P8lGUVEgiopgd/9rgvFxXDBBXD55TX/TENLqFCb2YH4K3A9ldw4IiIN67bbYOXKVnTqBI8/DmY1/0xDS7RFPRZ/PYnymhYUEUkVr74KY8ZAo0aOGTOgdevQiSpX4wkvZjYQWO+cW2pm+dUsNxwYDtCuXTsKCgrqK2OdFBcXB88QFdoXXmFhIWVlZdoXMZn+uSgsbMzPftYLaMqgQe9TUrKOqO6ORM5MPB44y8z6A83wdzKZ5py7JH4h59yTwJMAvXr1cqHPeNJZV7toX3itW7emsLBQ+yImkz8XzsHAgbBhA/TpA5dfvi7S+6LGrg/n3G+ccwc65zrj78M3r2KRFhFJJX/8I8yZA23awLRpkJXI3TUD0jhqEckoy5fDr3/tpydOhI4dw+ZJRK0uyuScK8Df40xEJOVs2uRPES8pgSuvhHPOCZ0oMWpRi0jGGDEC3nsPunaFhx4KnSZxKtQikhFmz4YJE/yp4bNm+VPFU4UKtYikvTVr4Ior/PQDD8CRR4bNU1sq1CKS1kpL4eKLobAQzjyTndf0SCUq1CKS1kaPhgULoH17mDQpmqeI10SFWkTS1uuvw113+eI8bRrk5YVOVDcq1CKSlr75xnd5lJfDTTfBKaeETlR3KtQiknacg+HD4ZNPoHdv36pOZSrUIpJ2Jk70w/Fyc/3dWho3Dp1o76hQi0haefdduPZaP/3YY/D974fNUx9UqEUkbWzd6k8R37IFBg/2fdTpQIVaRNLGzTfDW2/BwQfDI4+ETlN/VKhFJC28/DI8/DBkZ/u7iefmhk5Uf1SoRSTlrVsHQ4b46dGj4ZhjgsapdyrUIpLSysvh0kvhq6/gtNNg5MjQieqfCrWIpLQHH4S5c/1Zh1OnQqM0rGpp+JZEJFMsXgy33OKnJ0/21/NIRyrUIpKSior8ULzSUj9uesCA0ImSR4VaRFLSNdfAhx9C9+5w332h0ySXCrWIpJzp031/dE6OP0W8WbPQiZJLhVpEUsqqVfDzn/vphx+GLl3C5mkIKtQikjK2b/f90kVFcO65MGxY6EQNQ4VaRFLGHXfAm29Cx47+RrWpeLeWulChFpGUMG8e3HuvHyc9fTq0aRM6UcNRoRaRyPvqK7jkEn9DgFGj4MQTQydqWCrUIhJpzsHQof56HscfD7fdFjpRw1OhFpFIe/RR+POfYZ99fJdHdnboRA1PhVpEIuvtt+GGG/z0hAnQqVPYPKGoUItIJG3eDBdeCNu2+WF4550XOlE4KtQiEkk33ADvvAOHHw5jx4ZOE5YKtYhEzosvwuOPQ5Mm/hTxFi1CJwpLhVpEIuWTT+BnP/PT998PPXqEzRMFKtQiEhllZf7u4d98A/37+8uXigq1iETIPffA/PnQrh386U+Zc4p4TVSoRSQSFi6EO+/0008/DW3bBo0TKSrUIhJcYaG/Kl5ZGdx4I5x+euhE0aJCLSJBOQdXXQUffwy9esHdd4dOFD0q1CIS1OTJ8MwzfgjejBl+SJ7srsZCbWbNzOxNM3vLzFaY2W8bIpiIpL+VK+GXv/TTjz4KhxwSNk9UJXJ5k23AKc65YjNrDCwws7845/6R5Gwiksa2bfP90ps2wUUX+WF5UrkaC7VzzgHFsaeNYw+XzFAikv5uuQX++U/43vfgscc0FK86CV0w0MyygKXAwcAjzrlFlSwzHBgO0K5dOwoKCuoxZu0VFxcHzxAV2hdeYWEhZWVl2hcxIT8Xb765Lw899AMaNXKMHPlPli3bGCTHDpH/jjjnEn4ArYHXgCOqW+7oo492ob322muhI0SG9oV30kknue7du4eOERmhPheff+5c27bOgXO//32QCHuIwncEWOKqqKm1GvXhnCuMFep+9f4/hoikvfJyuOwyWL8eTj4Zfv3r0IlSQyKjPvYzs9ax6RzgdOC9ZAcTkfQzdiz89a/wne/4sw+zskInSg2J9FG3B6bE+qkbAc86515KbiwRSTfLlsHNN/vpiROhQ4eweVJJIqM+/gX0bIAsIpKmiov9ULzt2+Hqq+Hss0MnSi06M1FEku7aa+H99+GII2DMmNBpUo8KtYgk1TPP+EuWNmsGs2ZBTk7oRKlHhVpEkmb1ahg+3E8/9BB06xY0TspSoRaRpCgt9aeGb9wIP/6xv0Ke1I0KtYgkxW9/628G0KEDPPWUThHfGyrUIlLv5s+H0aN9cZ42zY+blrpToRaRerVhA1xyib8hwK23Qn5+6ESpT4VaROqNczBsGHz6KRx3HNxxR+hE6UGFWkTqzRNPwIsvQqtW/m4t2Qldn1NqokItIvVixQoYMcJPP/EEdO4cNE5aUaEWkb22das/RXzrVrj8crjwwtCJ0osKtYjstRtvhLffhkMPhT/+MXSa9KNCLSJ75X/+B8aPh8aNYeZMaNkydKL0o0ItInW2di0MHeqn77kHjjoqbJ50pUItInVSVgaXXgpffw1nnLHrQKLUPxVqEamTMWNg3jxo2xamTIFGqiZJo10rIrW2aBHcdpufnjIF2rULmyfdqVCLSK1s3OiH4pWV+e6OfrrVddKpUItIwpyDn/8cPvoIevb0BxAl+VSoRSRhTz/tTw1v3twPxWvaNHSizKBCLSIJ+eADf2NagHHj4LDDwubJJCrUIlKjkhLfL11cDOef708Tl4ajQi0iNRo1CpYsgU6d/AWXdLeWhqVCLSLVevVVuP9+yMry/dOtW4dOlHlUqEWkSl9+6c8+BH8TgB/9KGyeTKVCLSKVcs73RX/+OfTpA7fcEjpR5lKhFpFKjRsHL78Mbdr4G9RmZYVOlLlUqEVkD8uX+2tMA0ycCB07hs2T6VSoRWQ3mzb5oXglJXDllXDOOaETiQq1iOxmxAh47z3o2hUeeih0GgEVahGJM3s2TJjgTw2fNcufKi7hqVCLCABr1sAVV/jpBx6AI48Mm0d2UaEWEUpL4eKLobAQzjxz1zU9JBpUqEWE0aNhwQJo3x4mTdIp4lGjQi2S4RYsgLvu8sV52jTIywudSCpSoRbJYN98AxddBOXlcNNNcMopoRNJZVSoRTKUczB8OHzyCfTu7VvVEk01Fmoz62hmr5nZO2a2wsyua4hgIpJcc+a0Z/ZsyM31V8Vr3Dh0IqlKdgLLlAI3OOeWmVkusNTMXnXOvZPkbCKSJO++C+PHHwzAY4/BQQcFDiTVqrFF7Zxb55xbFpsuAt4FOiQ7mIgkx9at/hTxrVuzGDzYD8uTaEukRb2TmXUGegKLKpk3HBgO0K5dOwoKCvY+3V4oLi4OniEqtC+8wsJCysrKMn5fjB9/MG+9dSDt22/iwguXUVBQFjpScFH/jiRcqM2sJfA8cL1zbmPF+c65J4EnAXr16uXy8/PrK2OdFBQUEDpDVGhfeK1bt6awsDCj98WcOfD885CdDbff/h79+58YOlIkRP07ktCoDzNrjC/S051zLyQ3kogkw7p1MGSInx49Gg4/vChoHklcIqM+DJgIvOuc07W0RFJQebm/pdaXX8Jpp8HIkaETSW0k0qI+HhgMnGJmy2OP/knOJSL16MEHYe5cf9bh1KnQSGdQpJQa+6idcwsAnfkvkqIWL951v8PJk/31PCS16P9VkTRWVOSH4pWWwrXXwoABoRNJXahQi6Sxa66BDz+E7t3hvvtCp5G6UqEWSVMzZvj+6JwcmDkTmjULnUjqSoVaJA2tWgVXXeWnH34YunQJm0f2jgq1SJrZvt33SxcVwbnnwrBhoRPJ3lKhFkkzd9wBb74JHTv6G9Xqbi2pT4VaJI3Mmwf33uvHSU+fDm3ahE4k9UGFWiRNfPUVDB7sbwgwahScqMt4pA0VapE04BwMHQqffQbHHw+33RY6kdQnFWqRNPDoo/DnP8M++/guj+xaXcBYok6FWiTFvf023HCDn54wATp1CptH6p8KtUgK27zZD8Xbts0PwzvvvNCJJBlUqEVS2A03wIoVcPjhMHZs6DSSLCrUIinqxRfh8cehSRN/iniLFqETSbKoUIukoE8/3XXG4f33Q48eYfNIcqlQi6SYsjK45BLYsAH69/eXL5X0pkItkmLuuQfmz4d27eBPf9Ip4plAhVokhSxcCHfe6aenToW2bYPGkQaiQi2SIr79Fi66yHd93Hgj9O0bOpE0FBVqkRTgHFx5JaxeDb16wd13h04kDUmFWiQFTJ4Mzzzjh+DNmOGH5EnmUKEWibj334df/tJPP/IIHHJI2DzS8FSoRSJs2zZ/ivimTb5/+tJLQyeSEFSoRSLs1lth2TL43vfgscc0FC9TqVCLRNT//i88+CBkZfl+6VatQieSUFSoRSLoiy/gssv89F13wbHHhs0jYalQi0RMeTkMGQLr18PJJ8NNN4VOJKGpUItEzNixvtvjO9+Bp5/2XR+S2VSoRSJk2TK4+WY/PXEidOgQNo9Egwq1SEQUF/uheNu3w9VXw9lnh04kUaFCLRIR113nT2454ggYMyZ0GokSFWqRCHjmGZg0CZo1g1mzICcndCKJEhVqkcBWr4bhw/30Qw9Bt25B40gEqVCLBFRa6k8N37gRfvxjuOqq0IkkilSoRQK66y5/M4AOHeCpp3SKuFROhVokkPnz/XWlzWDaND9uWqQyKtQiAWzY4G9Q6xzccgvk54dOJFFWY6E2s0lmtt7M/t0QgUTSnXMwbBh8+ikcdxzccUfoRBJ1ibSoJwP9kpxDJGM8+SS8+KK/Gt6MGdC4cehEEnU1Fmrn3N+BDQ2QRSTtrVgB11/vp594Ajp3DhpHUkR2fa3IzIYDwwHatWtHQUFBfa26ToqLi4NniArtC6+wsJCysrJg+6KkpBE///lRbN3akn791rH//isJ+WvR52KXqO+LeivUzrkngScBevXq5fIDHx0pKCggdIao0L7wWrduTWFhYbB98ctfwqpV/p6Hzz3XnpYt2wfJsYM+F7tEfV9o1IdIA/jzn2H8eN8fPWsWtGwZOpGkEhVqkSRbuxYuv9xP33MPHHVU2DySehIZnjcTWAgcZmafmtnPkh9LJD2Ulfk7h3/9NfTtCyNGhE4kqajGPmrn3KCGCCKSjsaMgXnzoG1bmDIFGulvWKkDfWxEkmTRIhg1yk9PmQL77x82j6QuFWqRJNi40d+tpbTUd3f00yljshdUqEWS4Be/gI8+gp49/QFEkb2hQi1Sz55+GqZPh+bNYeZMaNo0dCJJdSrUIvXogw98axpg3Dg47LCweSQ9qFCL1JOSEt8vXVwM55+/a+y0yN5SoRapJ6NGwZIl0KmTv+CS7tYi9UWFei+ZGbNnzw4dQwJ79VW4/37IyvKXLm3dOnQiSSdpX6iHDBnCwIEDQ8eQNPbll/7sQ/A3AfjRj8LmkfST9oVaJJmc833Rn38Offr422qJ1LeMLtTvvPMOAwYMIDc3l7Zt2zJo0CA+//zznfMXL15M3759ycvLo1WrVpxwwgksXLiw2nXed9995OXl8Y9//CPZ8SUCxo2Dl1+GNm38DWqzskInknSUsYV63bp19OnThyOOOII333yTuXPnUlxczNlnn015eTkARUVFDB48mNdff50333yTHj160L9/f77++us91uecY+TIkYwbN4758+dz7LHHNvRbkgb21ltw441+euJE6NgxbB5JX/V244BU89hjj9G9e3fuu+++na9NnTqVfffdlyVLltC7d29OOeWU3X5m3LhxPP/88/zlL3/hkksu2fl6WVkZQ4cO5Y033uCNN96gU6dODfY+JIxNm+DCC/2QvCuvhHPOCZ1I0lnGFuqlS5fy97//nZaVXMH9ww8/pHfv3qxfv55Ro0bx2muv8cUXX1BWVsaWLVtYs2bNbsuPHDmS7OxsFi1aRNu2bRvqLUhAI0bAe+9B167w0EOh00i6y9hCXV5ezoABA3jggQf2mNeuXTsALrvsMr744gv+8Ic/0LlzZ5o2bcqpp55KSUnJbsuffvrpzJw5kzlz5jBkyJCGiC8BzZ4NEyb4U8NnzvSnioskU8YW6qOOOopnn32WTp060bhx40qXWbBgAX/84x8ZMGAAAF988QXr1q3bY7n+/fvzk5/8hPPOOw8z47LLLktqdglnzRq44go//cAD8IMfhM0jmSEjDiZu3LiR5cuX7/YYMGAA3377LRdccAGLFi1i1apVzJ07l+HDh1NUVATAoYceyrRp03jnnXdYvHgxF154IU2aNKl0GwMHDuS5557jqquuYurUqQ359qSBlJbCxRdDYSGceSZcfXXoRJIpMqJF/frrr9OzZ8/dXjv33HN54403+M1vfkO/fv3YunUr3/3ud+nbty9NY5c7mzRpEsOHD+foo4/mgAMO4M477+TLL7+scjsDBw7k2Wef5fzzzwfg0h1nQUhaGD0aFiyA9u1h0iSdIi4NJ+0L9eTJk5k8eXKV86s7/bt79+4sWrRot9cGDx6823Pn3G7PzzzzTLZs2VL7oBJpCxbAXXf54vz005CXFzqRZJKM6PoQ2RvffOO7PMrL4aab4NRTQyeSTKNCLVIN52D4cH8QsXdv36oWaWgq1CLVmDjRD8fLzfVXxatigJBIUqlQi1Thvffguuv89GOPwUEHhc0jmStlC/X69es566yzWLJkSegokoa2bvWniG/eDIMH+z5qkVBSslCvXLmS7t27M2fOHE4//XQ+/vjj0JEkzdx8s7/o0kEHwSOPhE4jmS7lCvXrr7/OMcccs/PaGxs3buSkk06isLAwdDRJE3PmwMMPQ3a2P0U8Nzd0Isl0KVWoZ8yYwRlnnEFRUdHO8cvl5eWsXbuWYcOGBU4n6WDdOthxuZbRo+GYY4LGEQFSpFA75/jd737HsGHD9jiZxMzIyclhxIgRgdJJuigvh8su87fWOu00GDkydCIRL/JnJpaWljJ06FCef/75PYp0dnY2++23HwUFBRx66KGBEkq6ePBBf5PavDyYOhUapUQzRjJBpAt1UVERAwYMYOnSpWzevHm3ec2aNeOQQw7hb3/7G/vtt1+ghJIulizZdb/DyZP99TxEoiKyhfqzzz4jPz+fNWvWsG3btt3mNW/enD59+vDCCy+Qk5MTKKGki6IiGDTIXx3v2mshdlVbkciI5B93b7/9Nt27d2fVqlWVFukhQ4bw0ksvqUhLvbjmGvjgA+jeHeLuzCYSGZEr1K+88grHHXccX331FWVlZbvNy8nJ4e677+aRRx4hS7d7lnowY4bvj87J8UPxmjULnUhkT5Hq+pgwYQLXXXddpZcJbd68OTNmzODss88OkEzS0apVcNVVfvrhh6FLl7B5RKrS4C3qp556ikGDBlFeXr7zNeccN910E9dff/0eRbpRo0a0bt2agoICFWmpN9u3w0UX+f7pc88FDcOXKGvQQl1eXs7tt9/OCy+8wK9+9SsASkpKOO+88xg/fvweIzuaNGnCgQceyLJlyzhGZx5IPbrjDli0CDp29Deq1d1aJMoatOtj3rx5FBUVUVJSwoQJE9h///154YUX+Pe//71HSzonJ4du3brxyiuv0KZNm4aMKWlu3jy4914/Tnr6dNDHS6KuQQv1mDFjKC4uBmDz5s3ccccdgG9Vx2vevDn9+u6+NX8AAAc2SURBVPVjxowZO+9fKFIfSkuNwYP9DQFuvx1OPDF0IpGaJdT1YWb9zGylmX1gZjfXZUOffvop8+fP3+21kpKSSov0Nddcw+zZs1WkpV45B5980pzPPoPjj4fbbgudSCQxNbaozSwLeAQ4HfgUWGxm/+Oce6c2G3r00UdrXCYnJ4exY8dyxRVX1GbVIpXats3f73DDBli/HpYvh40bG7PPPr7LIztSY55EqmYV76K9xwJmxwF3OufOiD3/DYBz7p6qfiY3N9cdffTRO5+Xl5ezcOFCSktLq91Wly5daNu2beLpq1FYWEjr1q3rZV2pLtX3RWnprsf27ZX/W9lrcQOLYpYD0KNHD/bZp8HfRuSk+ueiPkVhX8yfP3+pc65XZfMSaVN0AD6Je/4p8MOKC5nZcGA4QOPGjXe7PnRhYeFuw/EqY2Z8/PHHZGdn06geroZTVlama1THRGFfOAdlZY0oLTXKyvwjfnr357svtzeysx1ZWeVkZTlKShyNG5fhXCH6aETjcxEVUd8X9fbHn3PuSeBJgF69ern4W2Qdc8wxNd6FxTmHc44uXbowa9YsbC/HSxUUFJCfn79X60gX9bUvnPPjjjds8I8d3QqJTG/aVPft5ubCvvv6R5s2iU+3aLH7sLv8/HwKCwtZvnz5Xu+LdKDvyC5R2BfV1bxECvVaoGPc8wNjryXk3XffZcWKFQktu23bNp599lkuvvhizjrrrEQ3IbVUUuILaG0K7Y7pCmf1Jyw7u/aFdt99oXVr3flbJJFCvRg4xMy+hy/QFwIXJbqBsWPHsn379krnmRm5ubls2bKFgw8+mIEDB3LGGWfQp0+fRFefsZyD4uLEiuuqVd0pL9/1PDZCsk5atqxdod3xvGVLnVQiUlc1FmrnXKmZXQP8FcgCJjnnEmoib9q0iWnTpu12EDE3N5dt27bRoUMHBgwYQL9+/TjxxBNp1apVXd9DStu+vfrWbVVF+Jtv/AGzxOx+RkdWVt1bt02a1PsuEJEaJNRH7ZybA8yp7cqfeeYZtm7dStOmTcnLy6Nv374MGDCAk046iby8vFqHjSrnfB9sbQrtjumiorpvt0WLxArtmjXLOeWUHjtfz81V61YklSR1JOkPf/hDpk6dysknn8wBBxyQzE3Vi9LSPVu3ifbfJt663V2jRnVr3bZpk3jrtqCgkJ4965ZPRMJLaqHu1q0b3bp1S+Ym9rCjdbt+fVPeeqt2B8o2bqz7dps3r1vfbW6u7s0nItWL7LlZpaVQWFj7YWAbNvh+Xziu1tts1MgXz9oU2h3/6mx3EUmWpBZq52Dz5roNA/v227pvNycHWrTYRvv2TWtVdFu1UutWRKInKYV6xQp/F+cNG/yY3bowq3vrtlkzKChYGHwAu4hIfUhKod66FT7/3E83a1a7Qrtjep991LoVEYEkFequXeHVV33B1Y3CRUT2TlIKdU4OpMBoPBGRlKDOBRGRiFOhFhGJOBVqEZGIU6EWEYk4FWoRkYhToRYRiTgVahGRiFOhFhGJOBVqEZGIM+dc/a/U7Eug+tuOJ18e8FXgDFGhfbGL9sUu2he7RGFfdHLO7VfZjKQU6igwsyXOuV6hc0SB9sUu2he7aF/sEvV9oa4PEZGIU6EWEYm4dC7UT4YOECHaF7toX+yifbFLpPdF2vZRi4iki3RuUYuIpAUVahGRiMuIQm1mN5iZM7O80FlCMbMxZvaemf3LzF40s9ahMzUkM+tnZivN7AMzuzl0nlDMrKOZvWZm75jZCjO7LnSm0Mwsy8z+aWYvhc5SlbQv1GbWEegLrAmdJbBXgSOccz8A3gd+EzhPgzGzLOAR4L+ArsAgM+saNlUwpcANzrmuwLHA1Rm8L3a4Dng3dIjqpH2hBv4A/BrI6KOmzrlXnHOlsaf/AA4MmaeB9QY+cM6tcs6VALOAswNnCsI5t845tyw2XYQvUB3CpgrHzA4EBgBPhc5SnbQu1GZ2NrDWOfdW6CwRMxT4S+gQDagD8Enc80/J4OK0g5l1BnoCi8ImCWosviFXHjpIdZJyF/KGZGZzgf0rmXUrcAu+2yMjVLcvnHP/HVvmVvyfv9MbMptEi5m1BJ4HrnfObQydJwQzGwisd84tNbP80Hmqk/KF2jl3WmWvm9mRwPeAt8wM/J/6y8yst3Pu8waM2GCq2hc7mNkQYCBwqsusAfRrgY5xzw+MvZaRzKwxvkhPd869EDpPQMcDZ5lZf6AZ0MrMpjnnLgmcaw8Zc8KLma0GejnnQl8hKwgz6wc8BJzknPsydJ6GZGbZ+AOop+IL9GLgIufciqDBAjDfapkCbHDOXR86T1TEWtQjnXMDQ2epTFr3UctuxgO5wKtmttzMHg8dqKHEDqJeA/wVf/Ds2Uws0jHHA4OBU2Kfg+WxFqVEWMa0qEVEUpVa1CIiEadCLSIScSrUIiIRp0ItIhJxKtQiIhGnQi0iEnEq1CIiEff/AYJFjhVN5s79AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "wN06D61njxtO", "colab": { "base_uri": "https://localhost:8080/", "height": 282 }, "outputId": "90104eaf-5d1c-4da5-d0ea-67ea164aaea9" }, "source": [ "plt.plot(z, tf.nn.leaky_relu(z, 0.05), \"b-\", linewidth=2)\n", "plt.plot([-5, 5], [0, 0], 'k-')\n", "plt.plot([0, 0], [-0.5, 4.2], 'k-')\n", "plt.grid(True)\n", "props = dict(facecolor='black', shrink=0.1)\n", "plt.annotate('Leak', xytext=(-3.5, 0.5), xy=(-5, -0.2), arrowprops=props, fontsize=14, ha=\"center\")\n", "plt.title(\"Leaky ReLU activation function\", fontsize=14)\n", "plt.axis([-5, 5, -0.5, 4.2]);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAEJCAYAAAC9uG0XAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3wU1f3/8deHhEuAINgIIlJovQJaQJFqVYw35AuotdYLKooU0VarUrFaFbVW6gW1WPCKUEBuKurv+63Sb5ViqPilyKVYi4pVRBRRVIwk3EKS8/vjLLCEXDYhmzO7+34+Hvtgdmcy897J7oeTM2dmzDmHiIhEV6PQAUREpHoq1CIiEadCLSIScSrUIiIRp0ItIhJxKtQiIhGnQp0izMyZ2U9D50hlZjbEzIobaFsN8vsys+PN7F9mVmJmBcneXg1ZOsfed6+QOdKRCnU9MLPJZvZS6By1YWZ3xr5UzszKzewzM5tuZh1ruZ4CMxtfxbzVZjayim3/u67ZE8xVWaF8Bvh+PW+nqt99e+DP9bmtKjwMvAUcBPykAbYHVPl7/wT/vpc3VI5MoUKd2Vbiv1gHAhcARwLPBk2URM65Lc659Q20rc+dc9saYFMHA/Occ5845zY0wPaq5Jwri73v0pA50pEKdQMws65m9rKZFZnZejObaWb7x80/xsxeMbOvzGyjmS0ws+NqWOdNseWPj/3MTyvMP93MtptZu2pWUxr7Yn3mnHsdmAAca2at4tZzppktNbOtZvaRmY02syZ13BUJMbMsM5sY294WM/uPmf3azBpVWO4yM3vbzLaZ2RdmNiX2+urYIs/FWtarY6/v7Pows0Nj846ssM7hsf3auKYcZnYncBkwIO6vk/zYvN1a9GZ2pJnNja1nQ6wlvk/c/Mlm9pKZXWdma83sGzP7k5k1r2IfdTYzB+wDTIptb4iZ5cem8youu6NLIm6ZU81skZltNrMlZnZUhW0ca2bzzGyTmX0bmz7AzCYDJwFXx73vzpV1fZhZn9g2tsZ+R3+I//zEWuaPmtnvY/t9vZk9UPF3nem0M5LMzNoDfwf+DfQGTgNaAv8d92HMBZ4GTowtsxyYY2bfqWR9ZmYPAL8ETnLOvQHMBIZWWHQo8JJz7osEc+6P/9O5LPbAzM4ApgPjgW6xdf4U+H1Cb77uGgFrgfOBLsCtwC3A5XF5rwSeAP4E/ADoj9/HAMfE/r0C/xfDjuc7OefeBxYDF1eYdTHwrHNuewI5HsD/BTI3tp32wP9V3JaZtQD+ChTjf7/nAD8CJlVY9ETgCPxn5ILYctdVXF/Mjm6GzcD1selnqli2KvcANwNHAV8D083MYpm7A68BHwDHA8fG1p8dy7QQv+93vO9PKnnfHYC/AP8EegI/AwbFthvvYqAUv0+uib2fC2r5XtKbc06PvXwAk/FFsbJ5dwF/q/BaG8ABvav4GQPWAZfEvebwH94/Ae8DneLm9cJ/0DvErX8LMLCazHfiC3Ix/svuYo+H45b5OzCqws/9OPYzFnteAIyvYhurgZFVbPvftdzH9wJz455/CtxbzfIO+GmF14YAxXHPrwU+jnsv3wXKgR/VIkelv/v47eP/w/gWyI2bnx9b5uC49XwCZMUtMyF+W1XkKQaGVLLevLjXOsde61VhmTPiljk+9tqBsefTgYXVbHeP33sl2xkN/AdoVOF3sA1oHreehRXW8yrwVF2/j+n4UIs6+Y4G+phZ8Y4Hu1ofBwGYWVsze8LM3jezb4EioC2+cMR7AP8lO8E59/GOF51zS4C38X+GA1wEbMC3ZqrzIdAD3+K8FViGbzHGZ7+1QvYZQAtg/4orq09mdlXsz/EvY9sdQWx/mFlboAPwt73czCzgAHxLFnxr7yPn3M5WcXU5aqEL8C/nXFHca/+H/0+ha9xr7zjnyuKef4b/HCTLvypsi7jt9QTm7eX6uwD/cM6Vx722AGiC71uvLMeOLMl83ylHhTr5GgEv4wti/OMQYMdogSn4YjkC/+dfD3yLsWJf8Kv4Atm/ku08hW+tgO+imFLhS1+ZEufcB865Fc653+O/MI9UyP7bCrl/EMv+ZQ3rBtiI70OtqDW+hVkpM7sAGItvZZ4R2+6j7Lk/9orzBxZfZVf3x8X4lmRD5oi/fOX2SubV9ju6oyha3GuNq1g2fns7cjRUTajv953WskMHyADL8H2cHzvf71mZE4BrnXMvA5g/ANi+kuXmAC8QO0jmnJsSN286MMbMrsH3OV5Yh6x3AyvNbJxzbmks++HOuQ/qsC7wo0qOruT1o2LzqnICsMg5t3P4l5kdtGPaObfezNYCp+ILbWW2A1kJZJwGjDezJ/GjXuIPylabI6Ykge28Cww1s9y4VvWP8MXo3QQy1saO/0Dbx033qMN6/gmcUs38RN/3+WbWKK5VfULsZz+sQ6aMpf+16k8rM+tR4dEZ30LdB3jGzH5oZt83s9PM7Ekzy4397PvAJeZHhxyD/5O8pLKNOOdeAs4DHjezS+NeLwSeAx4E/u6c+09t34Bz7kPgv4HfxV66C7jIzO4ysyPM7HAz+6mZ3V/hR/Mqee8HAH8AzjCzUbH31s3MRgPHxeZV5X3gKDP7LzM7xMxG4UcZxBsNXG9mI8yP4OhhZjfEzV8NnGpm+5tZm2q29f/wLc6JwGLnDzLWJsdq4AgzO8zM8sysstbrdPxxgKnmR3/0wR8IfWEv/hOsygf4rrU7Y/ulL3BbHdYzBugZ+5x2j72/YWa2o9tnNdA7NtIjr4pRGo/iu5YeNbMuZjYA38c/3jm3uQ6ZMlfoTvJ0eOD/NHaVPGbH5h8CzAa+wR/kWwmMA5rE5ncHFsXmfQgMxo9guDNuG7sdHAPOjC1/adxrfWLLXZpA5jup5IAevqXniB1QA/oCr+MLzUZgCXBN3PIFVbz3Byr8/Ab8yIICoE8N2ZrgC+c3QGFs+nZgdYXlfga8g/9P7XNgUoX98x98y3p17LUhxB1MjFt2aizztbXNAewHvII/ruCA/Cp+X0fi+9S3xNY3GdinwmfopQrbr/R3VGGZ3Q4mxv0Ol8e2tRAYQOUHE6s84Bh77QT8AeUtsfc/F2gfm3dobN07DkR3rmIdffCf7W3AF/j/oJtW+PxUPCi5x77I9MeOo92SBmJ9qk8ABzi1WETShvqo04D5kyL2x4/YmKAiLZJe1EedHn6N707ZwK7+ZRFJE+r6EBGJOLWoRUQiLil91Hl5ea5z587JWHXCNm3aRIsWLYJmiArtC2/lypWUlZXRtWvXmhfOAPpc7FLZvnj/fSgqglat4JBDkp9h6dKlXznn9qtsXlIKdefOnVmyZEkyVp2wgoIC8vPzg2aICu0LLz8/n8LCwuCfzajQ52KXivvinnvgllugbVv417+gXXXXoKwnZvZxVfPU9SEiEmfRIhg1yk9PmdIwRbomKtQiIjHffguDBkFZGfzqV9CvX+hEngq1iAjgHPziF/DRR9CzJ/w+2VddrwUVahER4OmnYcYMaN4cZs6Epk1DJ9ol4UJt/rZE/7QUu4mriEhN1q7N4eqr/fS4cXDYYWHzVFSbFvV11P8lGUVEgiopgd/9rgvFxXDBBXD55TX/TENLqFCb2YH4K3A9ldw4IiIN67bbYOXKVnTqBI8/DmY1/0xDS7RFPRZ/PYnymhYUEUkVr74KY8ZAo0aOGTOgdevQiSpX4wkvZjYQWO+cW2pm+dUsNxwYDtCuXTsKCgrqK2OdFBcXB88QFdoXXmFhIWVlZdoXMZn+uSgsbMzPftYLaMqgQe9TUrKOqO6ORM5MPB44y8z6A83wdzKZ5py7JH4h59yTwJMAvXr1cqHPeNJZV7toX3itW7emsLBQ+yImkz8XzsHAgbBhA/TpA5dfvi7S+6LGrg/n3G+ccwc65zrj78M3r2KRFhFJJX/8I8yZA23awLRpkJXI3TUD0jhqEckoy5fDr3/tpydOhI4dw+ZJRK0uyuScK8Df40xEJOVs2uRPES8pgSuvhHPOCZ0oMWpRi0jGGDEC3nsPunaFhx4KnSZxKtQikhFmz4YJE/yp4bNm+VPFU4UKtYikvTVr4Ior/PQDD8CRR4bNU1sq1CKS1kpL4eKLobAQzjyTndf0SCUq1CKS1kaPhgULoH17mDQpmqeI10SFWkTS1uuvw113+eI8bRrk5YVOVDcq1CKSlr75xnd5lJfDTTfBKaeETlR3KtQiknacg+HD4ZNPoHdv36pOZSrUIpJ2Jk70w/Fyc/3dWho3Dp1o76hQi0haefdduPZaP/3YY/D974fNUx9UqEUkbWzd6k8R37IFBg/2fdTpQIVaRNLGzTfDW2/BwQfDI4+ETlN/VKhFJC28/DI8/DBkZ/u7iefmhk5Uf1SoRSTlrVsHQ4b46dGj4ZhjgsapdyrUIpLSysvh0kvhq6/gtNNg5MjQieqfCrWIpLQHH4S5c/1Zh1OnQqM0rGpp+JZEJFMsXgy33OKnJ0/21/NIRyrUIpKSior8ULzSUj9uesCA0ImSR4VaRFLSNdfAhx9C9+5w332h0ySXCrWIpJzp031/dE6OP0W8WbPQiZJLhVpEUsqqVfDzn/vphx+GLl3C5mkIKtQikjK2b/f90kVFcO65MGxY6EQNQ4VaRFLGHXfAm29Cx47+RrWpeLeWulChFpGUMG8e3HuvHyc9fTq0aRM6UcNRoRaRyPvqK7jkEn9DgFGj4MQTQydqWCrUIhJpzsHQof56HscfD7fdFjpRw1OhFpFIe/RR+POfYZ99fJdHdnboRA1PhVpEIuvtt+GGG/z0hAnQqVPYPKGoUItIJG3eDBdeCNu2+WF4550XOlE4KtQiEkk33ADvvAOHHw5jx4ZOE5YKtYhEzosvwuOPQ5Mm/hTxFi1CJwpLhVpEIuWTT+BnP/PT998PPXqEzRMFKtQiEhllZf7u4d98A/37+8uXigq1iETIPffA/PnQrh386U+Zc4p4TVSoRSQSFi6EO+/0008/DW3bBo0TKSrUIhJcYaG/Kl5ZGdx4I5x+euhE0aJCLSJBOQdXXQUffwy9esHdd4dOFD0q1CIS1OTJ8MwzfgjejBl+SJ7srsZCbWbNzOxNM3vLzFaY2W8bIpiIpL+VK+GXv/TTjz4KhxwSNk9UJXJ5k23AKc65YjNrDCwws7845/6R5Gwiksa2bfP90ps2wUUX+WF5UrkaC7VzzgHFsaeNYw+XzFAikv5uuQX++U/43vfgscc0FK86CV0w0MyygKXAwcAjzrlFlSwzHBgO0K5dOwoKCuoxZu0VFxcHzxAV2hdeYWEhZWVl2hcxIT8Xb765Lw899AMaNXKMHPlPli3bGCTHDpH/jjjnEn4ArYHXgCOqW+7oo492ob322muhI0SG9oV30kknue7du4eOERmhPheff+5c27bOgXO//32QCHuIwncEWOKqqKm1GvXhnCuMFep+9f4/hoikvfJyuOwyWL8eTj4Zfv3r0IlSQyKjPvYzs9ax6RzgdOC9ZAcTkfQzdiz89a/wne/4sw+zskInSg2J9FG3B6bE+qkbAc86515KbiwRSTfLlsHNN/vpiROhQ4eweVJJIqM+/gX0bIAsIpKmiov9ULzt2+Hqq+Hss0MnSi06M1FEku7aa+H99+GII2DMmNBpUo8KtYgk1TPP+EuWNmsGs2ZBTk7oRKlHhVpEkmb1ahg+3E8/9BB06xY0TspSoRaRpCgt9aeGb9wIP/6xv0Ke1I0KtYgkxW9/628G0KEDPPWUThHfGyrUIlLv5s+H0aN9cZ42zY+blrpToRaRerVhA1xyib8hwK23Qn5+6ESpT4VaROqNczBsGHz6KRx3HNxxR+hE6UGFWkTqzRNPwIsvQqtW/m4t2Qldn1NqokItIvVixQoYMcJPP/EEdO4cNE5aUaEWkb22das/RXzrVrj8crjwwtCJ0osKtYjstRtvhLffhkMPhT/+MXSa9KNCLSJ75X/+B8aPh8aNYeZMaNkydKL0o0ItInW2di0MHeqn77kHjjoqbJ50pUItInVSVgaXXgpffw1nnLHrQKLUPxVqEamTMWNg3jxo2xamTIFGqiZJo10rIrW2aBHcdpufnjIF2rULmyfdqVCLSK1s3OiH4pWV+e6OfrrVddKpUItIwpyDn/8cPvoIevb0BxAl+VSoRSRhTz/tTw1v3twPxWvaNHSizKBCLSIJ+eADf2NagHHj4LDDwubJJCrUIlKjkhLfL11cDOef708Tl4ajQi0iNRo1CpYsgU6d/AWXdLeWhqVCLSLVevVVuP9+yMry/dOtW4dOlHlUqEWkSl9+6c8+BH8TgB/9KGyeTKVCLSKVcs73RX/+OfTpA7fcEjpR5lKhFpFKjRsHL78Mbdr4G9RmZYVOlLlUqEVkD8uX+2tMA0ycCB07hs2T6VSoRWQ3mzb5oXglJXDllXDOOaETiQq1iOxmxAh47z3o2hUeeih0GgEVahGJM3s2TJjgTw2fNcufKi7hqVCLCABr1sAVV/jpBx6AI48Mm0d2UaEWEUpL4eKLobAQzjxz1zU9JBpUqEWE0aNhwQJo3x4mTdIp4lGjQi2S4RYsgLvu8sV52jTIywudSCpSoRbJYN98AxddBOXlcNNNcMopoRNJZVSoRTKUczB8OHzyCfTu7VvVEk01Fmoz62hmr5nZO2a2wsyua4hgIpJcc+a0Z/ZsyM31V8Vr3Dh0IqlKdgLLlAI3OOeWmVkusNTMXnXOvZPkbCKSJO++C+PHHwzAY4/BQQcFDiTVqrFF7Zxb55xbFpsuAt4FOiQ7mIgkx9at/hTxrVuzGDzYD8uTaEukRb2TmXUGegKLKpk3HBgO0K5dOwoKCvY+3V4oLi4OniEqtC+8wsJCysrKMn5fjB9/MG+9dSDt22/iwguXUVBQFjpScFH/jiRcqM2sJfA8cL1zbmPF+c65J4EnAXr16uXy8/PrK2OdFBQUEDpDVGhfeK1bt6awsDCj98WcOfD885CdDbff/h79+58YOlIkRP07ktCoDzNrjC/S051zLyQ3kogkw7p1MGSInx49Gg4/vChoHklcIqM+DJgIvOuc07W0RFJQebm/pdaXX8Jpp8HIkaETSW0k0qI+HhgMnGJmy2OP/knOJSL16MEHYe5cf9bh1KnQSGdQpJQa+6idcwsAnfkvkqIWL951v8PJk/31PCS16P9VkTRWVOSH4pWWwrXXwoABoRNJXahQi6Sxa66BDz+E7t3hvvtCp5G6UqEWSVMzZvj+6JwcmDkTmjULnUjqSoVaJA2tWgVXXeWnH34YunQJm0f2jgq1SJrZvt33SxcVwbnnwrBhoRPJ3lKhFkkzd9wBb74JHTv6G9Xqbi2pT4VaJI3Mmwf33uvHSU+fDm3ahE4k9UGFWiRNfPUVDB7sbwgwahScqMt4pA0VapE04BwMHQqffQbHHw+33RY6kdQnFWqRNPDoo/DnP8M++/guj+xaXcBYok6FWiTFvf023HCDn54wATp1CptH6p8KtUgK27zZD8Xbts0PwzvvvNCJJBlUqEVS2A03wIoVcPjhMHZs6DSSLCrUIinqxRfh8cehSRN/iniLFqETSbKoUIukoE8/3XXG4f33Q48eYfNIcqlQi6SYsjK45BLYsAH69/eXL5X0pkItkmLuuQfmz4d27eBPf9Ip4plAhVokhSxcCHfe6aenToW2bYPGkQaiQi2SIr79Fi66yHd93Hgj9O0bOpE0FBVqkRTgHFx5JaxeDb16wd13h04kDUmFWiQFTJ4Mzzzjh+DNmOGH5EnmUKEWibj334df/tJPP/IIHHJI2DzS8FSoRSJs2zZ/ivimTb5/+tJLQyeSEFSoRSLs1lth2TL43vfgscc0FC9TqVCLRNT//i88+CBkZfl+6VatQieSUFSoRSLoiy/gssv89F13wbHHhs0jYalQi0RMeTkMGQLr18PJJ8NNN4VOJKGpUItEzNixvtvjO9+Bp5/2XR+S2VSoRSJk2TK4+WY/PXEidOgQNo9Egwq1SEQUF/uheNu3w9VXw9lnh04kUaFCLRIR113nT2454ggYMyZ0GokSFWqRCHjmGZg0CZo1g1mzICcndCKJEhVqkcBWr4bhw/30Qw9Bt25B40gEqVCLBFRa6k8N37gRfvxjuOqq0IkkilSoRQK66y5/M4AOHeCpp3SKuFROhVokkPnz/XWlzWDaND9uWqQyKtQiAWzY4G9Q6xzccgvk54dOJFFWY6E2s0lmtt7M/t0QgUTSnXMwbBh8+ikcdxzccUfoRBJ1ibSoJwP9kpxDJGM8+SS8+KK/Gt6MGdC4cehEEnU1Fmrn3N+BDQ2QRSTtrVgB11/vp594Ajp3DhpHUkR2fa3IzIYDwwHatWtHQUFBfa26ToqLi4NniArtC6+wsJCysrJg+6KkpBE///lRbN3akn791rH//isJ+WvR52KXqO+LeivUzrkngScBevXq5fIDHx0pKCggdIao0L7wWrduTWFhYbB98ctfwqpV/p6Hzz3XnpYt2wfJsYM+F7tEfV9o1IdIA/jzn2H8eN8fPWsWtGwZOpGkEhVqkSRbuxYuv9xP33MPHHVU2DySehIZnjcTWAgcZmafmtnPkh9LJD2Ulfk7h3/9NfTtCyNGhE4kqajGPmrn3KCGCCKSjsaMgXnzoG1bmDIFGulvWKkDfWxEkmTRIhg1yk9PmQL77x82j6QuFWqRJNi40d+tpbTUd3f00yljshdUqEWS4Be/gI8+gp49/QFEkb2hQi1Sz55+GqZPh+bNYeZMaNo0dCJJdSrUIvXogw98axpg3Dg47LCweSQ9qFCL1JOSEt8vXVwM55+/a+y0yN5SoRapJ6NGwZIl0KmTv+CS7tYi9UWFei+ZGbNnzw4dQwJ79VW4/37IyvKXLm3dOnQiSSdpX6iHDBnCwIEDQ8eQNPbll/7sQ/A3AfjRj8LmkfST9oVaJJmc833Rn38Offr422qJ1LeMLtTvvPMOAwYMIDc3l7Zt2zJo0CA+//zznfMXL15M3759ycvLo1WrVpxwwgksXLiw2nXed9995OXl8Y9//CPZ8SUCxo2Dl1+GNm38DWqzskInknSUsYV63bp19OnThyOOOII333yTuXPnUlxczNlnn015eTkARUVFDB48mNdff50333yTHj160L9/f77++us91uecY+TIkYwbN4758+dz7LHHNvRbkgb21ltw441+euJE6NgxbB5JX/V244BU89hjj9G9e3fuu+++na9NnTqVfffdlyVLltC7d29OOeWU3X5m3LhxPP/88/zlL3/hkksu2fl6WVkZQ4cO5Y033uCNN96gU6dODfY+JIxNm+DCC/2QvCuvhHPOCZ1I0lnGFuqlS5fy97//nZaVXMH9ww8/pHfv3qxfv55Ro0bx2muv8cUXX1BWVsaWLVtYs2bNbsuPHDmS7OxsFi1aRNu2bRvqLUhAI0bAe+9B167w0EOh00i6y9hCXV5ezoABA3jggQf2mNeuXTsALrvsMr744gv+8Ic/0LlzZ5o2bcqpp55KSUnJbsuffvrpzJw5kzlz5jBkyJCGiC8BzZ4NEyb4U8NnzvSnioskU8YW6qOOOopnn32WTp060bhx40qXWbBgAX/84x8ZMGAAAF988QXr1q3bY7n+/fvzk5/8hPPOOw8z47LLLktqdglnzRq44go//cAD8IMfhM0jmSEjDiZu3LiR5cuX7/YYMGAA3377LRdccAGLFi1i1apVzJ07l+HDh1NUVATAoYceyrRp03jnnXdYvHgxF154IU2aNKl0GwMHDuS5557jqquuYurUqQ359qSBlJbCxRdDYSGceSZcfXXoRJIpMqJF/frrr9OzZ8/dXjv33HN54403+M1vfkO/fv3YunUr3/3ud+nbty9NY5c7mzRpEsOHD+foo4/mgAMO4M477+TLL7+scjsDBw7k2Wef5fzzzwfg0h1nQUhaGD0aFiyA9u1h0iSdIi4NJ+0L9eTJk5k8eXKV86s7/bt79+4sWrRot9cGDx6823Pn3G7PzzzzTLZs2VL7oBJpCxbAXXf54vz005CXFzqRZJKM6PoQ2RvffOO7PMrL4aab4NRTQyeSTKNCLVIN52D4cH8QsXdv36oWaWgq1CLVmDjRD8fLzfVXxatigJBIUqlQi1Thvffguuv89GOPwUEHhc0jmStlC/X69es566yzWLJkSegokoa2bvWniG/eDIMH+z5qkVBSslCvXLmS7t27M2fOHE4//XQ+/vjj0JEkzdx8s7/o0kEHwSOPhE4jmS7lCvXrr7/OMcccs/PaGxs3buSkk06isLAwdDRJE3PmwMMPQ3a2P0U8Nzd0Isl0KVWoZ8yYwRlnnEFRUdHO8cvl5eWsXbuWYcOGBU4n6WDdOthxuZbRo+GYY4LGEQFSpFA75/jd737HsGHD9jiZxMzIyclhxIgRgdJJuigvh8su87fWOu00GDkydCIRL/JnJpaWljJ06FCef/75PYp0dnY2++23HwUFBRx66KGBEkq6ePBBf5PavDyYOhUapUQzRjJBpAt1UVERAwYMYOnSpWzevHm3ec2aNeOQQw7hb3/7G/vtt1+ghJIulizZdb/DyZP99TxEoiKyhfqzzz4jPz+fNWvWsG3btt3mNW/enD59+vDCCy+Qk5MTKKGki6IiGDTIXx3v2mshdlVbkciI5B93b7/9Nt27d2fVqlWVFukhQ4bw0ksvqUhLvbjmGvjgA+jeHeLuzCYSGZEr1K+88grHHXccX331FWVlZbvNy8nJ4e677+aRRx4hS7d7lnowY4bvj87J8UPxmjULnUhkT5Hq+pgwYQLXXXddpZcJbd68OTNmzODss88OkEzS0apVcNVVfvrhh6FLl7B5RKrS4C3qp556ikGDBlFeXr7zNeccN910E9dff/0eRbpRo0a0bt2agoICFWmpN9u3w0UX+f7pc88FDcOXKGvQQl1eXs7tt9/OCy+8wK9+9SsASkpKOO+88xg/fvweIzuaNGnCgQceyLJlyzhGZx5IPbrjDli0CDp29Deq1d1aJMoatOtj3rx5FBUVUVJSwoQJE9h///154YUX+Pe//71HSzonJ4du3brxyiuv0KZNm4aMKWlu3jy4914/Tnr6dNDHS6KuQQv1mDFjKC4uBmDz5s3ccccdgG9Vx2vevDn9+u6+NX8AAAc2SURBVPVjxowZO+9fKFIfSkuNwYP9DQFuvx1OPDF0IpGaJdT1YWb9zGylmX1gZjfXZUOffvop8+fP3+21kpKSSov0Nddcw+zZs1WkpV45B5980pzPPoPjj4fbbgudSCQxNbaozSwLeAQ4HfgUWGxm/+Oce6c2G3r00UdrXCYnJ4exY8dyxRVX1GbVIpXats3f73DDBli/HpYvh40bG7PPPr7LIztSY55EqmYV76K9xwJmxwF3OufOiD3/DYBz7p6qfiY3N9cdffTRO5+Xl5ezcOFCSktLq91Wly5daNu2beLpq1FYWEjr1q3rZV2pLtX3RWnprsf27ZX/W9lrcQOLYpYD0KNHD/bZp8HfRuSk+ueiPkVhX8yfP3+pc65XZfMSaVN0AD6Je/4p8MOKC5nZcGA4QOPGjXe7PnRhYeFuw/EqY2Z8/PHHZGdn06geroZTVlama1THRGFfOAdlZY0oLTXKyvwjfnr357svtzeysx1ZWeVkZTlKShyNG5fhXCH6aETjcxEVUd8X9fbHn3PuSeBJgF69ern4W2Qdc8wxNd6FxTmHc44uXbowa9YsbC/HSxUUFJCfn79X60gX9bUvnPPjjjds8I8d3QqJTG/aVPft5ubCvvv6R5s2iU+3aLH7sLv8/HwKCwtZvnz5Xu+LdKDvyC5R2BfV1bxECvVaoGPc8wNjryXk3XffZcWKFQktu23bNp599lkuvvhizjrrrEQ3IbVUUuILaG0K7Y7pCmf1Jyw7u/aFdt99oXVr3flbJJFCvRg4xMy+hy/QFwIXJbqBsWPHsn379krnmRm5ubls2bKFgw8+mIEDB3LGGWfQp0+fRFefsZyD4uLEiuuqVd0pL9/1PDZCsk5atqxdod3xvGVLnVQiUlc1FmrnXKmZXQP8FcgCJjnnEmoib9q0iWnTpu12EDE3N5dt27bRoUMHBgwYQL9+/TjxxBNp1apVXd9DStu+vfrWbVVF+Jtv/AGzxOx+RkdWVt1bt02a1PsuEJEaJNRH7ZybA8yp7cqfeeYZtm7dStOmTcnLy6Nv374MGDCAk046iby8vFqHjSrnfB9sbQrtjumiorpvt0WLxArtmjXLOeWUHjtfz81V61YklSR1JOkPf/hDpk6dysknn8wBBxyQzE3Vi9LSPVu3ifbfJt663V2jRnVr3bZpk3jrtqCgkJ4965ZPRMJLaqHu1q0b3bp1S+Ym9rCjdbt+fVPeeqt2B8o2bqz7dps3r1vfbW6u7s0nItWL7LlZpaVQWFj7YWAbNvh+Xziu1tts1MgXz9oU2h3/6mx3EUmWpBZq52Dz5roNA/v227pvNycHWrTYRvv2TWtVdFu1UutWRKInKYV6xQp/F+cNG/yY3bowq3vrtlkzKChYGHwAu4hIfUhKod66FT7/3E83a1a7Qrtjep991LoVEYEkFequXeHVV33B1Y3CRUT2TlIKdU4OpMBoPBGRlKDOBRGRiFOhFhGJOBVqEZGIU6EWEYk4FWoRkYhToRYRiTgVahGRiFOhFhGJOBVqEZGIM+dc/a/U7Eug+tuOJ18e8FXgDFGhfbGL9sUu2he7RGFfdHLO7VfZjKQU6igwsyXOuV6hc0SB9sUu2he7aF/sEvV9oa4PEZGIU6EWEYm4dC7UT4YOECHaF7toX+yifbFLpPdF2vZRi4iki3RuUYuIpAUVahGRiMuIQm1mN5iZM7O80FlCMbMxZvaemf3LzF40s9ahMzUkM+tnZivN7AMzuzl0nlDMrKOZvWZm75jZCjO7LnSm0Mwsy8z+aWYvhc5SlbQv1GbWEegLrAmdJbBXgSOccz8A3gd+EzhPgzGzLOAR4L+ArsAgM+saNlUwpcANzrmuwLHA1Rm8L3a4Dng3dIjqpH2hBv4A/BrI6KOmzrlXnHOlsaf/AA4MmaeB9QY+cM6tcs6VALOAswNnCsI5t845tyw2XYQvUB3CpgrHzA4EBgBPhc5SnbQu1GZ2NrDWOfdW6CwRMxT4S+gQDagD8Enc80/J4OK0g5l1BnoCi8ImCWosviFXHjpIdZJyF/KGZGZzgf0rmXUrcAu+2yMjVLcvnHP/HVvmVvyfv9MbMptEi5m1BJ4HrnfObQydJwQzGwisd84tNbP80Hmqk/KF2jl3WmWvm9mRwPeAt8wM/J/6y8yst3Pu8waM2GCq2hc7mNkQYCBwqsusAfRrgY5xzw+MvZaRzKwxvkhPd869EDpPQMcDZ5lZf6AZ0MrMpjnnLgmcaw8Zc8KLma0GejnnQl8hKwgz6wc8BJzknPsydJ6GZGbZ+AOop+IL9GLgIufciqDBAjDfapkCbHDOXR86T1TEWtQjnXMDQ2epTFr3UctuxgO5wKtmttzMHg8dqKHEDqJeA/wVf/Ds2Uws0jHHA4OBU2Kfg+WxFqVEWMa0qEVEUpVa1CIiEadCLSIScSrUIiIRp0ItIhJxKtQiIhGnQi0iEnEq1CIiEff/AYJFjhVN5s79AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "Tpglilsyj3QU", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "3edcdb09-777c-43e7-a058-b4a2170dad36" }, "source": [ "[m for m in dir(tf.keras.activations) if not m.startswith(\"_\")]" ], "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "['deserialize',\n", " 'elu',\n", " 'exponential',\n", " 'get',\n", " 'hard_sigmoid',\n", " 'linear',\n", " 'relu',\n", " 'selu',\n", " 'serialize',\n", " 'sigmoid',\n", " 'softmax',\n", " 'softplus',\n", " 'softsign',\n", " 'swish',\n", " 'tanh']" ] }, "metadata": { "tags": [] }, "execution_count": 30 } ] }, { "cell_type": "markdown", "metadata": { "id": "kaQpt2m5kDwS" }, "source": [ "### Training with LeakyReLu" ] }, { "cell_type": "code", "metadata": { "id": "2LF2djQQj7D5" }, "source": [ "leaky_relu = tf.keras.layers.LeakyReLU(alpha=0.1)\n", "model = tf.keras.models.Sequential([\n", " tf.keras.layers.Flatten(input_shape=[28, 28]),\n", " tf.keras.layers.Dense(300, activation=leaky_relu),\n", " tf.keras.layers.Dense(100, activation=leaky_relu),\n", " tf.keras.layers.Dense(10, activation=\"softmax\")\n", "])" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "BfDAByxpkM8c", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "2fd0898d-f1a4-49d9-f207-a2c0cb0d0d2f" }, "source": [ "model.compile(loss=\"sparse_categorical_crossentropy\", \n", " optimizer=tf.keras.optimizers.SGD(1e-3),\n", " metrics=[\"accuracy\"])\n", "model.summary()" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Model: \"sequential_1\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "flatten_1 (Flatten) (None, 784) 0 \n", "_________________________________________________________________\n", "dense_3 (Dense) (None, 300) 235500 \n", "_________________________________________________________________\n", "dense_4 (Dense) (None, 100) 30100 \n", "_________________________________________________________________\n", "dense_5 (Dense) (None, 10) 1010 \n", "=================================================================\n", "Total params: 266,610\n", "Trainable params: 266,610\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "jt3L74j-kdTz", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "b56d12ba-f654-46b4-9fb2-8e7c8b1db12e" }, "source": [ "history = model.fit(X_train, y_train, \n", " epochs=20,\n", " batch_size=32,\n", " validation_split=0.1)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Epoch 1/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 1.3688 - accuracy: 0.6149 - val_loss: 0.9200 - val_accuracy: 0.7220\n", "Epoch 2/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.8234 - accuracy: 0.7391 - val_loss: 0.7263 - val_accuracy: 0.7640\n", "Epoch 3/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.6984 - accuracy: 0.7737 - val_loss: 0.6450 - val_accuracy: 0.7857\n", "Epoch 4/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.6349 - accuracy: 0.7924 - val_loss: 0.5999 - val_accuracy: 0.7935\n", "Epoch 5/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.5936 - accuracy: 0.8047 - val_loss: 0.5649 - val_accuracy: 0.8077\n", "Epoch 6/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.5640 - accuracy: 0.8132 - val_loss: 0.5470 - val_accuracy: 0.8123\n", "Epoch 7/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.5419 - accuracy: 0.8184 - val_loss: 0.5231 - val_accuracy: 0.8212\n", "Epoch 8/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.5247 - accuracy: 0.8239 - val_loss: 0.5081 - val_accuracy: 0.8230\n", "Epoch 9/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.5105 - accuracy: 0.8271 - val_loss: 0.4987 - val_accuracy: 0.8253\n", "Epoch 10/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.4987 - accuracy: 0.8309 - val_loss: 0.4870 - val_accuracy: 0.8302\n", "Epoch 11/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.4888 - accuracy: 0.8336 - val_loss: 0.4775 - val_accuracy: 0.8342\n", "Epoch 12/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.4800 - accuracy: 0.8354 - val_loss: 0.4719 - val_accuracy: 0.8318\n", "Epoch 13/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4724 - accuracy: 0.8381 - val_loss: 0.4648 - val_accuracy: 0.8353\n", "Epoch 14/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4656 - accuracy: 0.8406 - val_loss: 0.4594 - val_accuracy: 0.8385\n", "Epoch 15/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.4599 - accuracy: 0.8418 - val_loss: 0.4556 - val_accuracy: 0.8388\n", "Epoch 16/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.4542 - accuracy: 0.8436 - val_loss: 0.4470 - val_accuracy: 0.8385\n", "Epoch 17/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.4492 - accuracy: 0.8458 - val_loss: 0.4424 - val_accuracy: 0.8425\n", "Epoch 18/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.4447 - accuracy: 0.8473 - val_loss: 0.4447 - val_accuracy: 0.8430\n", "Epoch 19/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4403 - accuracy: 0.8488 - val_loss: 0.4362 - val_accuracy: 0.8455\n", "Epoch 20/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.4365 - accuracy: 0.8496 - val_loss: 0.4353 - val_accuracy: 0.8447\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "I0tr75QwkgeO", "colab": { "base_uri": "https://localhost:8080/", "height": 324 }, "outputId": "2d04348d-6f00-44f0-fc07-b5fa7931a376" }, "source": [ "pd.DataFrame(history.history).plot(figsize=(8, 5))\n", "plt.grid(True)\n", "plt.gca().set_ylim(0.3, 1);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeMAAAEzCAYAAAACSWsXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXxU1f3/8deZfZKZ7CuBLISwb7JvYlChWBdcq2i17vZXbbW2ftt+a61atO1Xv+2v/Ym2WnH5flFU3NC6gRg1bIIIyE4CJCQEspFlss12fn/MEBIEkkAWMvk8H4/7mJl7zz1zjgjvnJN7z1Vaa4QQQgjRcww93QAhhBCir5MwFkIIIXqYhLEQQgjRwySMhRBCiB4mYSyEEEL0MAljIYQQooe1GcZKqUVKqVKl1NaTHFdKqb8rpfKUUluUUuM6v5lCCCFE6GrPyPhFYO4pjl8EZAW3O4FnzrxZQgghRN/RZhhrrb8AKk9RZB7wsg5YC0QppZI7q4FCCCFEqOuM3xmnAAdafC4K7hNCCCFEO5i688uUUncSmMrGbrePHzBgQKfV7ff7MRg6/rPFgVo/dpMizq46rS2d6XT7dTYLxT5BaPZL+tR7hGK/Qq1Pu3fvLtdax5/oWGeEcTHQMlX7B/d9h9b6WeBZgAkTJugNGzZ0wtcH5OTkkJ2d3eHzLv1/ucQ6LLx4y6ROa0tnOt1+nc1CsU8Qmv2SPvUeodivUOuTUqrgZMc640eOZcBNwauqpwDVWuuSTqi3WyRGWDlc09TTzRBCCNGHtTkyVkq9CmQDcUqpIuD3gBlAa/0P4APg+0AeUA/c0lWN7QoJETa+Kazq6WYIIYTow9oMY631/DaOa+DuTmtRN0t02qioc+P2+rGYQud3E0IIIXqPPp8+CRFWAMpcMlUthBCiZ/T5ME4MhnFpTWMPt0QIIURf1efDOMFpA5CLuIQQQvSYPh/GiRGBMC6tlZGxEEKIntHnwzg23ILRoDgs09RCCCF6SJ8PY4NBkeCUe42FEEL0nD4fxkAwjGVkLIQQomdIGBNY+KOsVkbGQggheoaEMUeXxJSRsRBCiJ4hYUxgFa4j9R6avL6ebooQQog+SMKYFrc3yUVcQggheoCEMceWxJR7jYUQQvQECWNkFS4hhBA9S8IYWZ9aCCFEz5IwBqLDLJiNisNye5MQQogeIGHM0VW4bHJ7kxBCiB4hYRyUEGGVq6mFEEL0CAnjoEQZGQshhOghEsZBCbIKlxBCiB4iYRyUGGGjptFLo0dW4RJCCNG9JIyDEpxHb2+S3xsLIYToXhLGQUeXxDwsq3AJIYToZhLGQc1hLL83FkII0c0kjIOOrsIlS2IKIYTobhLGQZF2MxaTQZbEFEII0e1CIoxzi3P526G/0eg9/SBVSpHgtFIqS2IKIYToZiERxlpr8pry2Hh44xnVkxghC38IIYTofiERxhOTJmJWZr4s/vKM6kmUhT+EEEL0gJAIY5vJxiDrIHKLc8+ongSnTe4zFkII0e1CIowBhtuHs79mP0W1RaddR2KEjdomL3VN3k5smRBCCHFq7QpjpdRcpdQupVSeUurXJzieppT6VCm1RSmVo5Tq3/lNPbXh9uEArCpeddp1NK/CJRdxCSGE6EZthrFSyggsBC4ChgPzlVLDjyv2JPCy1no08Cjwx85uaFviTfGkOFLOaKr66MIfcnuTEEKI7tSekfEkIE9rvVdr7QaWAPOOKzMcWBl8/9kJjnc5pRQzUmaw7tA63D73adXRvPCHjIyFEEJ0o/aEcQpwoMXnouC+ljYDVwbfXwE4lVKxZ968jjk35VwavA1sLD29W5wSZGQshBCiB5g6qZ5fAk8ppW4GvgCKge88i1ApdSdwJ0BiYiI5OTmd9PXgcrkw7zFjwsSra1+lMbrjgaq1xmKADdv2MMhX2GltOxMul6tT/zudDUKxTxCa/ZI+9R6h2K9Q7NPJtCeMi4EBLT73D+5rprU+SHBkrJRyAFdprauOr0hr/SzwLMCECRN0dnb26bX6BHJycsjOzmbpJ0spqC/gdOtO3vAZlsgosrPP6bS2nYmj/QolodgnCM1+SZ96j1DsVyj26WTaM029HshSSmUopSzAdcCylgWUUnFKqaN1/QZY1LnNbL8ZKTPIr86nxFVyWucnOGXhDyGEEN2rzTDWWnuBe4CPgR3A61rrbUqpR5VSlwWLZQO7lFK7gUTgsS5qb5vOTTkXgNyDp3dVdUKEjTK5gEsIIUQ3atfvjLXWHwAfHLfvoRbvlwJLO7dppycjMoPk8GRyi3K5ZvA1HT4/0Wkjp6a0C1omhBBCnFjIrMB1VMtbnDw+T4fPT4ywUuf24ZJVuIQQQnSTkAtjCPzeuM5Tx6ayTR0+9+jCH/J7YyGEEN0lJMN4cvJkTAbTaT3FKeHowh8SxkIIIbpJSIZxuDmccQnjTmtpzARnYGQsF3EJIYToLiEZxhCYqt5zZA+H6w536LxEGRkLIYToZiEbxtNTpgOw6mDHnuLksJoIsxg5LM81FkII0U1CNoyzorJICEvo8FS1UorECJuMjIUQQnSbkA1jpRTnppzLmoNr8Pg7dotTgtNKqYyMhRBCdJOQDWMITFW7PC62lG3p0HmJETYO18rIWAghRPcI6TCekjwFkzJ1eKr66MhYa91FLRNCCCGOCekwdlqcjEkYw6rijl3ElRhho8Hjo1ZW4RJCCNENQjqMIXCL047KHZQ3lLf7nKMLf5TKRVxCCCG6QZ8IY6BDU9XHlsSUi7iEEEJ0vXY9tak3GxI9hHh7PLnFuVw+6PJ2nSPrUwshRC+hNfi9rTffcZ/9vjY+H93naf1ZGWDU1d3SjZAPY6UU01Oms7JwJV6/F5Oh7S4nOI+uwiUjYyGEaKY1+DzgawJvcPM1gdcN3kbwuVvsO3o8eMzrPrbf5w7uD+5rfu9uUYebsZVlsMfeZjnooottbZESxp1pesp03sl7h63lWxmbMLbN8uFWEw6riVK5vUkIcbbTGryNmDy1UF0Mngbw1J/kNfje23jyY61eG48L204coBgtzZtWVvzahN9nwe834/eb8PtMNNT4MNgM+L3h+H3h+L0qMHj1gN+j8bs12uPH7/YBCowGlNEIRiMquLV6bzKhjCYwGVFH35vNx46ZzCiTGUxmlMmEwR5GdOf1+JT6RBhPTZ6KQRn4svjLdoUxBC7ikoU/hBDfofWxKU2fOzAl6nMHPwe3Ux07OqLzNgRePQ3HPnsag6PI4Naez8GAnAHQgRtHtNGGVnZ8Pjs+nw2f14zPY8HnNuBzm/E1mfF7ItHKQODyIgMaA6BAG9AoQAVedfC9VsHKFYE7QxVo0H4dPK7Bp/E3NuKvr8ff0IC/vh48HsAHNAS3Y6qobd1woxFDWFhgs9sxhDlREXaUMqB9Pvw+L3h96CYf+Hxonwd9dJ/PF3jv86N9PvB6g/t8wTa0ZoiMJPq2u9v/H/UM9IkwjrRGMiY+cIvTT8/5abvOSXTKkphC9EpeN7hd4K47tnmOvU8+uBHWbAN3fetyLcrgdqEbXGh3E9rrAW8gSLXfGwjU4Kxo81IEOrihWu9rLqNafVbBzEIFCipFYJRotoLJhjJZwGyD4KsyWcEcB+FWMNlRFhuYrMEyNrDYyN9zgNSkNHxNGl+9D1+9J7DVNeJzNeJz1eOrqcNfU4u3ugZ/dTW6OYAag9sxKiwMQ3gYymAEgwGlFBiNYFAoZQiMOA0q8HtVY/C44eh7Q+Acw7HXo+8xGjDbW4Zp4HsMdnvgO48eCwvjmx07mHjujGPHwsNRZnPgu7qA9vuDAe5De32BH7q6SZ8IY4Dp/abz1KanqGioINYe22b5xAgrXxce6YaWCdGHBadYaXKBuzb4GghDmmqDr65gaLZ4f6LywWDVPi/ap/B5FH6Pwu8x4PMYgu8VcR4DZe7AZ5/PhN9rxu814vMa8LtVYCDb5Ed7j6aqObh1Jy/g6vBZ+06wT1ksGKOiMEZGYoyKwjIwE3tUZKt9hshITFFRze+NUVEYLJYz7sWZ8jY2YB00qNu+r/mHB3N3/3n3oTCe0X8GT216itUHV3Np5qVtlg88LCKwCldX/RQmxNnE39CA78gRfJXl+MoO4assJWLHVmor9mIwKQxmUGaFwaQDm9GPMvhR/qbgtGnTcVOpTSd+9TS0DlfdevShNYEwdQeD1G0IvPdZAtOpfit+rxmfxxQ8Dr6mcPyNdvxN0fgaPOBv+4IeQ1gYBqcTg8OBMcqB0enE7HRgdDgwOJwYnA4MVisYjKCC/1ATHP0d/ayCI8Pmz8F9BnVspMgJPqPB7w9M32oN2g9an+CzPzjq9gff60DfgmV0i895xcUMmzSxVcgao6JQNpv8G9YL9JkwHhYzjBhbDLnFue0K44QIG26vn+oGD1FhPf8TohAn5W2CxurmkaR2VeGtKMV/pAxfRQW+6ip8VdX4amrx1bjw1dYHpi7rmgLTmA0+fI36+EwEwA4UtfH1yugPhrMOhPXRzWJAmY0YrCYMFhMGqxllNaNMEfi90c0Xx/obffgaffjr3fjqm/DVN4L3VNODflSYEaPTgTHCiSEmErPDgSEiAoMjHKPDicHpxOh0YHAENqMzuM/hYPWmTcycOzdw0U4I+TYnh4js7J5uhjhNfSaMDcrA9H7T+bL4S3x+H0bDqf8iHr29qbS2ScJYdD2/H5qqoeEINFQFXhsDr7q2Em/5IXzl5Xgrj+A9Uo2v2oW3pgGfy423wY+v0YC3yYDfbcDvPcVaPgaN0aYw2QwYwkyYY2zYHFaMjjCMznCMkQ6MEcFRVXQM+QdKGJiWid+j0d6jV7D68Lv9aLcXv9uLv8mDv8mNbmgKXJjT0IC/oR5vfUPgfUXgVde7jv2O0mzGGBGBMSICQ0QUxpgILBFODBERGJ0RGCKcgXZEODE4IzBGRgQCNSICo8OBOoMpVL1nT8gFsej9+kwYQ2A1rvf2vse2im2Mjh99yrItF/4YnOjsjuaJ3szvw+RxwZECaKqBxprjXquDo9cadEM1uqYSb2VlIFir6vDWNuBtNOBrMuBtNDaHq6/RiM994nBVJgNGZySmiHCMCU6skcHQCgapMToWY2wCxvhEjHHJGGPiUGFhHZqyLMvJYUQnjrZ08OpVZbHI1KkQLfSpMJ7WbxoGZSC3OLcdYSwLf/RJTS6oPQSuQ+iqYvzVZfirK9GuKvy1VfjratGuGvz1dc23ZuiGRvxNbgZ7FYd8Cu1VgfshfQq/19DisyGweRS61TNIrMENDOE2TJFOjInRWGNjMcUnYExIxpSQiDEmBlNcHKaYGIxxcYErS3tZoCmTCWXqU//sCNEufepvRZQtipFxI1lVvIqfjP3JKcsmRtgwKNh9uPaU5UTP0h4P7sJC3AWF+Ovr0U2N+Jua0I1NaHdg2lQ3NaHravG7qtD1NfjrgretNDYEj7vRHg9+tw/t02hfIEjR7Q06W2BTCoPNgsFmQdlsgds2osMwhAUvCgpzBPaFh2OKi8UYE3vca8xZcQWrEKL79akwBpjRbwbPbH6GI41HiLadfG0Vm9nIBcMSeWPDAe6fPRibWX7H1JP89fU07duHOz+fpvy9uPcGXwsL2rjYB5QxeOWv8eh7jTJqlElhtFpQkTYMNgcqLBxld2JwRKDCI1GOaAwRMRgcUSiHM3j/oz1wz6M98Hp0U2FhfLFqFdmzZnXTfxEhRCjpe2GcMoOnNz/N6oOruXjgxacse9uMDJZvP8zb3xQzf1JqN7Wwb/MeOYJ7zy6atm8KvO7bh7uwBE95zbFCCiyRYHE04cxyY43wYonwYjAHg9ZqR0UlYIhKgsgklDMZnIngTAZH8NWZCNaIFqsvdIJeNmUshDh79LkwHhE3gmhrNKuKV7UZxpMzYhjRL4JFufu4buKAXvf7ubOO3wdVBURXfINn5W6a8nbj3n+ApqJS3CVVNJU34Ws4dn+oMvqxOH3YIz1EJXuxxNuxJsdiGZCMikwCRwI4jr4mgjMpsFnlgjshRO/S58LYoAxM7TeVVQdX4dd+DOrkt4Eopbh1ega/eGMzX+wp57zB8d3Y0t5Fa42/pgZvRSW+w4V49+/AV7QH7+ED+EoP4a2swldbh69RYaszktfi9huDFayxFhzDYrCmJGBN648lMxNzxmBURFIgaMPjwdj9q+IIIUR36HNhDIGp6g/2fcCOih2MiBtxyrKXjEnmTx/tZFHuvj4Xxv7GRrxlgYUjvJWV+CorA2FbWYG3ogJf6UG8ZaX4jlThrak/6apHBiuYHFaMkclYUuOoM1lImTod67DRWIeOxBgXJ7MOQog+rV1hrJSaC/wNMAL/0lr/6bjjqcBLQFSwzK+11h90cls7zfSU6SgUucW5bYax1WTkxilp/GX5bvJKaxmUELpToH63m4ZNm6hfu466deto2LLlxE8yMYPR6sNk9WG2+bDF+DENsGCKi8OY0A9jvwxMA4ZgzBiNKX0Uyh7e6vy8nBxGyUpBQgjRrM0wVkoZgYXAbAIr461XSi3TWm9vUexB4HWt9TNKqeHAB0B6F7S3U8TYYhgeO5zc4lzuGnNXm+VvmJzKU5/lsWjVfh6/YlQ3tLB7aK+Xxm3bqFu7jvp1a6n/eiO6qQmUwtYvjNjhXiy2I5isfow2Pyabxpg0AEPiYIjLCm6DITYLwuPkAiYhhDhN7RkZTwLytNZ7AZRSS4B5QMsw1kBE8H0kcLAzG9kVZqTM4Llvn6O6qZpIa+Qpy8Y6rFx5TgpvbSzigTlDiA7vnfeCar+fpl27AuG7di31G9bjr6sHwBpvJjqjlrD4OsLi3RjjUmDAJEgcGQjcuMEQkxF4bJsQQohO1Z4wTgEOtPhcBEw+rszDwCdKqZ8C4cCFndK6LjQjZQb/3PJP1pSsYW763DbL3zI9gyXrD/DKV4XcPav7Hul1JrTWuPfto27tWurXrKV+3Vp8NYFFTCyREJFcR3hCE2FJfkwZo2HAxYEA7j8JIlN6uPVCCNF3KK1P/agxpdTVwFyt9e3BzzcCk7XW97Qoc3+wrv9WSk0FngdGaq39x9V1J3AnQGJi4vglS5Z0WkdcLhcOh6Pd5f3az2+KfsMo+yh+GPfDdp3zxPoGil2aJ8+zYzJ0z5RsR/tlKK/AsmsXth1bsezaiaptAMAU5iM8sYnwhCZM/W3UJw+hJmIo1ZFDcTky8Ru7b7Tf0T71FqHYL+lT7xGK/Qq1Ps2aNetrrfWEEx1rz8i4GBjQ4nP/4L6WbgPmAmit1yilbEAcUNqykNb6WeBZgAkTJujsTryIJycnh47WN/PzmWw4vIGZ58085S1OR+nkUm55YT2u6MFcfk73jBxP1i+/243nwAHcBYW4C/bTtGUD9Ru+xlNWDYDR5guMeod5CR+RgXnkFNSAyYGRb1Qq4UrRU9eGn86fVW8Qiv2SPvUeodivUOzTybQnjNcDWUqpDAIhfB1w/XFlCoELgBeVUsMILNZb1pkN7QozUmbw0f6P2FW5i2Gxw9osf15WPAPjw1m0ah/zxvbr8ttx/E1NGA+WULtyJe79BbgLC3AXFOApKMRTUhJ4wHiQ0eojLN5NzBQT4eeMxHLOuajUydDvHLCEn+JbhBBC9LQ2w1hr7VVK3QN8TOC2pUVa621KqUeBDVrrZcAvgOeUUj8ncDHXzbqt+e+zwPSU6QCsOriqXWFsMAQWAXnwna1sKDjCxPSYM26Dv6kJT2Fh88MO3AXHQtdbcog4rZsf7m6MjMSckoi9v5XIFBMWYymWCD+WkVMwjr8K0mdAzEC5qlkIIXqZdt1nHLxn+IPj9j3U4v12YHrnNq3rxdnjGBYzjC+LvuT2Ube365wrx6XwxMe7WJS7r8Nh7G9spHb5curXbwiGbwHeQ4daj3CjojCnpRI2YQKW1DTy6+sYM2UkFtfXGPd/AKU5gAoE78j7YNhlgduKhBBC9Fp9cgWulmakzGDR1kXUuGuIsES0WT7MYuL6yan88/N8DlTWMyAmrM1zGnftpuqNN6hetgx/TQ3GyEgs6emETZyAJS0NS2oalvQ0LKmpGCODt1kd2Q9b38K27hXsK/cF9qVOhYuegOHzAg86EEIIERIkjIP3G68rWcfstNntOuemqWk898VeXly9n99dMvyEZfx1dVR/8AFVbyylccsWlNmMc/Zson5wDWGTJqEMJ7hgrLoIVr8MW9+CgxsB0M7B8L3HYfjlcruREEKEqD4fxqPjR+M0O8ktzm13GCdH2vn+qGReW3+A+y7MwmkLPMBAa03j1q1Uvf4GNf/+N/76eiyZmST8+ldEzpuHKfoEz0+uPQTb3oFtb8GBdcEvGAMXPgIjrmDj5n1kT83upN4KIYQ4G/X5MDYZTEzpN4Xc4ly01u2+QvrWGRks23yQNzYU8aPRsVQve4+qpUtp2rkTZbMRcdFFRF1zDfZzxn63TlcZ7HgXtr4NBasADQkj4PwHYcSVEJvZovC+TuurEEKIs1OfD2OAc1POZXnBcvZU7WFw9OB2nTOmfyRXmsrgjw+zp2gzuqkJ6/BhJP3+ISIuuQSj8wQPlKg9DJ/8Fra+CdofWGLyvF/ByCshfkgn90oIIURvIWEMTOs3DYDc4tw2w9hbWUn1O+9StXQpd+zdS73JSs35cxh154+wjzzJE6D8ftj4Eqz4PXgaYOrdMPo6SBwhtyEJIYSQMAZIDE9kcPRgcotzuXXkrd85rv1+6tasoWrpUmpXfAoeD/axY0lYsICrd4SRkBDFaycL4tKd8N69cGAtpJ8Ll/w18LQjIYQQIkjCOGhGygxe3vYyLrcLhyWwFqrncCnVb79F1dI38RQVYYyMJHr+dURdfTW2wYER9PVf7OWxD3awtbiakSktnv7kaYQvn4Tc/wtWB8x7GsZeLyNhIYQQ39H2gsx9xIyUGXi1l3WHAlc0V735Fvnf+x5l//dvmFNS6Pfkkwz64nOS/vM/m4MY4NpJAwi3GFmU2+JCq72fwzNT4YsnYORVcM8GOOcGCWIhhBAnJCPjoLEJYwk3h7NmXw7D/rmS6jffImzyZJIfeRhLevpJz4uwmblmwgAWryvgN9nxxK/+A2x+FaIz4MZ3IHNW93VCCCFEryRhHGQ2mJljHMWU379L9SEPsT++i/if/hRlNLZ57i3T0qhd9zKO534M/jo49xcw8wEw27uh5UIIIXo7CeOgmo8/4Qd/Wk+99mD6y8MkfP/a9p1YkU/av+/jv81fsMk7hGF3LsLab2TXNlYIIURI6fO/M9ZuN4cef5zie+/FMjCD/7jVyOq0xrZP9Lrh8yfg6alwcDP5kxdwRePveLuo7fWthRBCiJb6dBh7SkoouPEmjrz8P0T/8Idkvfoa0WlZfFn85alPLFgD/5gBny2Aod+He75i4Nx7GJYcxaJV++gFT48UQghxFumzYez6Mpd9V1xJ0549pPz1LyQ9+FuUxcKMlBlsPLyRek/9d09qOALLfgYvzA0s3nH9G3DNi+BMQinFrTMy2H3YRW5eebf3RwghRO/V58JY+3yU/f3vHLjzTkzx8aQvXUrERRc1H5+eMh2P38NXh75qcZKGb5fCU5Pgm/+FqffA3Wth8JxWdV86Jpk4h5Xnc2U9aSGEEO3Xp8LYW1FB4e23U/70M0TOm0f6669hHZjRqsy4hHHYTXZyi3MDO47sh8VXw5u3BR5heOdn8L3HwBL+nfqtJiM3TU0jZ1cZeaWubuiREEKIUNBnwrj+66/Zd8WVNGz8huQFfyD5j49jsH/31iOL0cLk5MmBpziteQYWToHCtTD3z3D7p4HHG57C9ZNTsZgMvLBKRsdCCCHaJ+TDWGtNxfOLKLjpRyibjfQlrxJ19dWnfFTiuSnnUuwqZv/KByF9Bty9Dqb8GAxt33Mc57ByxdgU3txYRFW9uzO7IoQQIkSFdBj7amoo+ulPKX3iCZznn0/Gm0uxDRvW5nnTU6YDkOuIhKufh8j+HfreW2ak0+jx88pXhafVbiGEEH1LyIZxw7Zt7Lvqalw5n5Pw61+R8ve/nfgZwyeQ4jeQ4fGQm5AGtsi2TzjO0KQIZgyK4+XVBXh8/g6fL4QQom8JuTDWWnPktdcpmH892u0m7eWXib355lNOS3/H+ueYUd/IBm81Dd6G02rHbTMyOFTTyAfflpzW+UIIIfqOkApjf309Jb/+NYd+/3vCJk4k4+23CBt3TscqcdfDhkXMiBuN2+9h/aH1p9WW8wbHMzA+nEW5sgiIEEKIUwuZMDYeOsT+a6+letl7xN1zDwOe/SemmJiOV7T5VWg4wvgpvyDKGsU/N/8Tr9/b4WoMBsUt0zPYXFTNxsIjHW+HEEKIPiMkwrh25WfE/PFPeMsrGPCv54i/5+52PW3pO/x+WPsMJI/FmjGT/5z8n2wp38KL2148rXZdNS6FSLtZFgERQghxSiERxubkJDwZ6WS8/RaO6dNPv6K8FVCxJ7DCllLMTZ/L7LTZPL3pafYc2dPh6sIsJuZPSuWjrYc4UHmC5TWFEEIIQiSMbcOGUXXffZiTks6sojVPgbMfjLgcAKUUD055EKfFyW9zf4vH7+lwlT+aloZSipfX7D+ztgkhhAhZIRHGneLQVtj3OUy6A4zm5t0xthgemvIQOyp38NyW5zpcbXKkne+PSmbJVwdwNXX8d89CCCFCn4TxUWufAXMYjL/5O4cuSLuAiwdezHNbnmN7xfYOV33bjAxqm7ws3XCgExoqhBAi1EgYA9Qehm9fh7HXQ9iJr8D+zaTfEG2L5re5v8Xt69gyl2MHRDE+LZoXVu/H55fbnIQQQrTWrjBWSs1VSu1SSuUppX59guN/VUptCm67lVJVnd/ULrThefC5YfL/OWmRSGskD097mLyqPJ7e9HSHv+LW6RkUVNSzcmfpmbRUCCFECGozjJVSRmAhcBEwHJivlBresozW+uda67Fa67HA/wPe6orGdglPA6x/HgZfBHGDTll0Zv+ZXDHoCl7Y9gKbyzZ36Gu+NyKRlCg7z+fuPZPWCiGECEHtGRlPAvK01nu11m5gCTDvFOXnA692RuO6xZbXob4cpv6kXcUfmPgACWEJPJj7II3exnZ/jclo4EfT0li7t5JtB6tPt7VCCCFCkGprqUal1NXAXK317cHPNyuFY9cAACAASURBVAKTtdb3nKBsGrAW6K+19p3g+J3AnQCJiYnjlyxZcuY9CHK5XDgcjo6dpDUT1/8MrYxsmPBXaOf61TsbdrKwdCGznLO4MubKdn9dnUdzf0495yQY+fEYW7vOOa1+neVCsU8Qmv2SPvUeodivUOvTrFmzvtZaTzjRMVMnf9d1wNITBTGA1vpZ4FmACRMm6Ozs7E774pycHDpcX96n8HkhXP4PssfOavdp2WRTvrac13e9zo+m/4jxiePbfe42306ezsnn0ikZ3DA5rc3yp9Wvs1wo9glCs1/Sp94jFPsVin06mfZMUxcDA1p87h/cdyLX0ZumqNcsBEcijLyqw6feP/5+UhwpPJj7IPWe9q+udf/swWQPieehd7eRs0su5hJCCNG+MF4PZCmlMpRSFgKBu+z4QkqpoUA0sKZzm9hFSndC/qcw8Q4wWTp8epg5jD9M/wPFrmL+8vVf2n2eyWjgqevHMSTRyd2LN7L9YE2Hv1sIIURoaTOMtdZe4B7gY2AH8LrWeptS6lGl1GUtil4HLNG95XmBa58Gkw0m3HraVUxImsANw27gtV2vsbZkbbvPc1hNLLp5IhF2M7e+uJ6S6tN7ZrIQQojQ0K77jLXWH2itB2utM7XWjwX3PaS1XtaizMNa6+/cg3xWqiuHzUtgzHUQHntGVd077l7SI9J5aNVDuNyudp+XFGlj0c0TcTV5ueWF9dQ2dnzdayGEEKGhb67AtWER+JpgSvtuZzoVm8nGghkLOFx/mCc2PNGhc4clR/D0DePYU+ri7le+wePzn3F7hBBC9D59L4y9TfDVczDoQogf0ilVjokfw80jbuatPW/xRdEXHTp35uB4Flw+ki92l/HQu1vpLbP8QgghOk/fC+Nvl0JdKUy9u1OrvXvs3QyKGsQjqx+huqlji3rMn5TKT7IzefWrA/zjc1mhSwgh+pq+FcZaBy7cShgOA9t/X3F7WIwWFsxYQEVjBX/66k8dPv+Xc4Zw6Zh+/Pmjnby/5WCntk0IIcTZrW+F8b4v4PBWmPJ/2r3aVkeMiB3BHaPv4P297/Np4acdOtdgUDxx9Wgmpkdz/+ub2bC/stPbJ4QQ4uzUt8J4zUIIi4NRP+iyr7hz1J0MjRnKo2sepbKxY4FqMxt59sYJpETZuePlDewrr+uiVgohhDib9J0wLt8Dez6GibeDuX3rQp8Os9HMYzMeo8Zdw4K1Czp8QVZ0uIUXbp4IwC0vfEWtWy7oEkKIUNd3wnjtM2C0wMTbuvyrBkcP5idjfsLyguV8tP+jDp+fHhfOv340gYPVjfxtYyONnhMu9S2EECJE9I0wrq+ETa8EpqcdCd3ylbeMvIVRcaN4bN1jlDeUd/j88Wkx/PUHY8mr8vPLNzbj98sIWQghQlXfCOOvXwBvQ7ufWdwZTAYTC6YvoMHTwCOrHzmt+4cvHp3MD4aYeX9LCU98sqsLWimEEOJsEPph7HUHFvkYmA2JI7r1qwdGDeRn435GTlEOy/K/82yNdrko3cz1k1N5JiefV9YVdnILhRBCnA1CP4y3vQ21JTClcxf5aK8fDvsh5yScw5+/+jOH6g51+HylFI9eNoLzBsfzu3e3ymMXhRAiBIV2GGsNaxdC3ODA8pc9wGgwsmD6Arzay+9X//60pqtNRgMLbxjHYHnsohBChKTQDuOC1VCyObDIh6Hnupoakcp94+5j9cHVLN2z9LTqcFhNvHDzRJy2wGMXD1U3dnIrhRBC9JTQDuM1C8EeDaOv6+mWcN3Q65iUNIkn1z9Jsav4tOpIirTxwi3Bxy6+uB5Xk7eTWymEEKInhG4YV+TDrg9gwm1gCevp1mBQBh6d/ihKKX6Z88sOr8511LDkCBbeMI7dh2u5e/FGvPLYRSGE6PVCN4zX/RMMJph0R0+3pFmKI4XHpj/Gnqo9zH9/PrsqT+92pfMGx/OHeSP5fHcZv3t3mzx2UQghernQDOOGKvjmf2HkVeBM6unWtHJB2gW8OPdFvH4vN354I8sLlp9WPddPTuX/ZGfy6leF/PMLeeyiEEL0ZqEZxhtfAk9dty7y0REj40ay5JIlZEVncX/O/Ty96Wn8uuPTzQ/MGcIlo5P504c7+feWki5oqRBCiO4QemHs88C6ZyH9XEge09OtOan4sHgWfW8Rl2VexjObn+EXOb+g3lPfoToMBsWT14xhQlo0P399E29+XdRFrRVCCNGVQi+Mt78LNUUw5ewcFbdkNVpZMH0BD0x4gJUHVnLjhzd2+Eprm9nIczdN4JwBUfzijc088MZmGtzyYAkhhOhNQiuMtYa1T0PMQBg8t6db0y5KKW4acRNPX/A0JXUlzH9/PusPre9QHdHhFhbfPpmfnT+IpRuLmLcwlz2Ha7uoxUIIITpbaIXxga+g+OvAqLgHF/k4HdNTpvPK918h0hrJnZ/cyeu7Xu/Q+SajgfvnDOHlWydR4XJz2VOrZNpaCCF6id6VWG1Z8xTYImHM/J5uyWlJj0znlYtfYUq/Kfxh7R9YsHYBPt2xKedzs+L54N5zGd0/UqathRCilwiZMLY1HIad78P4m8Hq6OnmnDanxclT5z/FLSNv4bVdr7Hw8EKONB7pUB2JETaZthZCiF4kZMI4pfh9UAaYdFdPN+WMGQ1G7h9/P4/PeJx9TfuY/++OLxAi09ZCCNF7hEYYN9aQXLIchl8OkSk93ZpOc2nmpdybdC8en4cbP7yRTws+7XAdMm0thBBnv9AI402LMfkaztpFPs5EujWdVy95lUFRg7gv5z7+sfkfHV7+UqathRDi7BYaYXzOjWwb/gCkjO/plnSJhLAEXpj7ApcOvJSFmxbyi887vkCITFsLIcTZKzTC2OqgLGFGT7eiS1mNVh6b8Ri/nPBLPi38lJs+vImDroMdruf4aetfvrGZerc8ilEIIXpSu8JYKTVXKbVLKZWnlPr1Scr8QCm1XSm1TSn1Suc2U0BggZAfjfgRCy9YyEHXQeb/ez5fH/66w/W0nLZ+c2MR855aJdPWQgjRg9oMY6WUEVgIXAQMB+YrpYYfVyYL+A0wXWs9ArivC9oqgmakzGDxxYuJsERw+ye3s3T30g7X0XLaurIuMG29VKathRCiR7RnZDwJyNNa79Vau4ElwLzjytwBLNRaHwHQWpd2bjPF8TIiM1h88WImJ0/mkTWP8Njax2jyNXW4nqPT1mMGRPJLmbYWQogeodq6MlcpdTUwV2t9e/DzjcBkrfU9Lcq8A+wGpgNG4GGt9UcnqOtO4E6AxMTE8UuWLOmsfuByuXA4eu9iHyfTVr/82s+yqmV8WvMpCaYE5sfOZ5BtUIe/x6817+Z5WJbvIdmhuHusjRRH11xS0Ff/rHoj6VPvEYr9CrU+zZo162ut9YQTHtRan3IDrgb+1eLzjcBTx5V5H3gbMAMZwAEg6lT1jh8/Xnemzz77rFPrO1u0t1+5Rbn6e0u/p0e+OFI/vPphXd1UfVrf98XuUj3u0U/00Ac/1G9sOHBadbSlr/9Z9SbSp94jFPsVan0CNuiTZGJ7hj7FwIAWn/sH97VUBCzTWnu01vsIjJKz2vWjgugU01Om89Zlb3HT8Jt4a89bXP7O5awoWNHheo6ftr7j5Q3klcrFXUII0ZXaE8brgSylVIZSygJcByw7rsw7QDaAUioOGAzs7cR2inYIM4fxwMQHeOX7rxBrj+XnOT/n3pX3crjucIfqCVxtPYX/mDuENfkVzPnrF/z6zS0cqm7sopYLIUTf1mYYa629wD3Ax8AO4HWt9Tal1KNKqcuCxT4GKpRS24HPgAe01hVd1WhxaiPiRvDKxa/w8/E/Z9XBVVz+7uW8tvM1/Nrf7jqMBsVPsgfx+QPZ3Dwtgzc3FnHeE5/x5492Ut3g6cLWCyFE39OuK3S01h9orQdrrTO11o8F9z2ktV4WfK+11vdrrYdrrUdprTvvyixxWswGM7eOvJW3L3ubEXEjWLBuATd/dDP5VfkdqifWYeWhS4ez8hfZfH9UMv/4PJ+Z//UZz32xl0aPrHEthBCdITRW4BInNSBiAM/Nfo4/TP8D+VX5XP3e1Tyz6RncPnfH6okJ46/XjuX9n85g7IAoHvtgB+c/mcMbGw7g83dsrWwhhBCtSRj3AUopLh90OcsuX8bstNk8vflprnnvGr4p/abDdY3oF8lLt07ilTsmE++08sDSLVz0ty/4dMfhDj/AQgghRICEcR8Sa4/lv2b+FwsvWEiDt4GbPryJBWsX4HK7OlzXtMw43rl7Ok/fMA6PT3PbSxu49p9r+brgSBe0XAghQpuEcR80s/9M3pn3Dj8c9kNe3/U6896dx8rClR2uRynF90cl88nPZ7Lg8pHsLa/jqmdWc9f/bCCvtOMBL4QQfZWEcR8VZg7jV5N+xeLvLybSGsm9n93L/Tn3U1Zf1uG6zEYDP5ySxucPZPOL2YNZlVfBnL9+zm/ektuhhBCiPSSM+7hR8aN47ZLXuHfcvXx+4HPmvTOPN3a/0aHboI4Kt5r46QVZzbdDLf1abocSQoj2kDAWmA1mbh91O2/Ne4uhsUN5dM2j3Prxreyr3nda9cntUEII0TESxqJZWkQaz895nkemPcLuI7u5atlVPLz6YXKLc/H4Oj6yPdntUJ8f8NDgllAWQoijTD3dAHF2UUpxZdaVzOw/k79v/Dsf7f+IN/e8idPiJLt/NhemXci0ftOwmWztrvPo7VCr88r500c7eWFbNUsfX8GV56Rw/eQ0hiQ5u7BHQghx9pMwFicUZ4/j0emP8tspv2XtwbUsL1jOZwc+472972E32ZnZfyYXpl3IzJSZhJnD2lXntEFxvHv3dJ59eyXb3TG8+tUBXlpTwLjUKK6fnMbFo5KxW4xd3DMhhDj7SBiLU7IarZw34DzOG3AeHr+H9YfWs6JgBZ8WfsrH+z/GarQyrd80ZqfN5rwB5xFhiThlfUophsQYuSv7HH5/qZu3NhbxyleF/PKNzTz63jauHNef+ZNSZbQshOhTJIxFu5kNZqb1m8a0ftP47eTfsrF0IysKVrCicAWfHfgMk8HE5OTJzE6dzazUWcTYYk5ZX0y4hdvPHchtMzJYt6+SV9YV8sq6Ql5cvZ/xadHMn5TKJaOTsZlltCyECG0SxuK0GA1GJiZNZGLSRH416Vd8W/4tKwpWsLxgOQ+veZhH1z7KhMQJXJh2IRekXkBCWMJJ61JKMWVgLFMGxlJZ5+bNr4t49bjR8vWTUxmcKKNlIURokjAWZ8ygDIyJH8OY+DHcP/5+dlbuZHnBclYUruDxdY/zx3V/ZEz8GC5Mu5DZabNPWVdMuIU7Zg7k9nMzWLu3kle/OjZanhAcLV8so2UhRIiRMBadSinFsNhhDIsdxs/G/Yz8qvxAMBes4MkNT/LkhidJt6RTt7eOOWlzMBvNJ61namYsUzNjqXA18ebGIl796gC/eGMzjwRHyzdMTiVLRstCiBAgYSy6VGZUJplRmfx4zI8prClkecFyXtnyCr/+8tf894b/5toh13LNkGtO+fvlWIeVO2dmcse5A1mzt4JXvzrA4nUFvLh6PxPTA6Pl74+S0bIQoveSMBbdJjUildtG3UZGeQamLBOLdyzmqU1P8eyWZ7l44MXcMOwGhsQMOen5SimmZcYxLTOOCtfw5tHy/a9v5uFl25g9PIk5IxKZmRUvt0gJIXoVCWPR7QzKwMz+M5nZfyZ7q/ayeMdi3tv7Hm/nvc3EpIncMOwGsvtnYzScPFCPHy0v3VDE8u2HeHNjETazgZlZ8cwZkcQFQxOIDrd0Y++EEKLjJIxFjxoYNZDfTf0dPxv3M97c8yav7nyV+z67jxRHCtcPvZ4rsq7AaTn574VbjpY9Pj9f7avkk22H+GT7YT7ZfhijQTEpPYY5IxKZPTyR/tHtW6BECCG6k4SxOCtEWiO5deSt3DT8JlYWrmTxjsU8seEJFm5ayLxB87hh2A2kRaSdsg6z0cD0QXFMHxTHw5eN4Nviaj7ZdphPth/ikfe288h72xmZEsGc4HT2kEQnSqlu6qEQQpychLE4q5gMJuakz2FO+hy2VWxj8fbFvLH7DV7d+Soz+8/khmE3MDV5apshqpRidP8oRveP4pffG8K+8rrmEfNfV+zmL8t3kxYbxpzhicwZkcS41GiMBglmIUTPkDAWZ60RsSN4/NzHuX/C/by+63Ve2/Uady2/i8zITK4fdj2XZl6K3WRvV10ZceHcdV4md52XSWltIyu2l/LJ9kO8tLqA577cR5zDwoXDEpkzIpFpmXFyZbYQoltJGIuzXpw9jp+M/Qm3j7qdj/Z/xP9u/1/+sPYP/P2bv3NV1lXMHzqfpPCkdteX4LRx/eRUrp+cSm2jh893l/HxtsO8v6WEJesPEG4xkj0kgTkjEjlvcDxRYXIBmBCia0kYi17DYrRwWeZlXDrwUjaWbmTxjsW8uO1FXtr2Euenns/stNlMSZ5CtC263XU6bWYuGd2PS0b3o8nrY+3eSj7edojl2w/z729LUAqGJ0cwLTOWaZlxTMyIwWGVvzZCiM4l/6qIXkcpxfjE8YxPHM9B10GW7FzCW3lvsbxgOYrACmBTk6cyrd80xiaMxWJs38jWajJy3uB4zhscz4J5I9lUVMWqPeWszq/gpTWB6WyjQTGmf2TwCu5YxqVFy5S2EOKMSRiLXq2fox/3T7ife8fdy/aK7aw+uJrVB1fz0raXeH7r89hNdiYkTmh+2lRGZEa7rqA2GBTjUqMZlxrNTy/IotHjY2PBEVbnV7A6v5xnPs/nqc/ysJgMjEuNYlpmHPYaH9N9fsxGQzf0XAgRSiSMRUgwGoyMih/FqPhR3DXmLlxuF+sPrWdNyRrWHFzDn9f/GYDEsMTmYJ6cPLndU9o2s5Fpg+KYNigOGIKrycv6fZWszg+MnP+6Yjdaw1+/+YSJ6THN09rD+0XIVdpCiDZJGIuQ5LA4mJU6i1mpswAodhWz5uAaVh9czYrCFbyd93bzlPbRcB4bP/akD674Tv1WE7OGJjBraODRkEfq3Dz/3hfU2JNYnV/BHz/cCUCEzcSUgbGBcB4UR1aCQ+5tFkJ8h4Sx6BNSHClcPfhqrh58NT6/r9WU9otbX+Rf3/4Lu8nOxKSJTOs3jan9ppIR0b4pbYDocAsTkkxkZ48EoLSmkTV7K1iTX8Hq/Ao+2X4YgDiHhckZsZyTGsXYAVGM6Bcp62gLIdoXxkqpucDfACPwL631n447fjPwBFAc3PWU1vpfndhOITrNyaa0Vx9czZqSNXxR9AUASeFJjIkfw6CoQc3bAOeAU66ZfVRChI15Y1OYNzYFgAOV9c3h/NW+Sv79bUmwLYqhSU7GDgiE8zmpUQyMc2CQqW0h+pQ2w1gpZQQWArOBImC9UmqZ1nr7cUVf01rf0wVtFKJLnWpKe3vFdj7Z/wkaDYDVaCUjMoNBUYPIjMokKyqLzKhM+jn6nfI7BsSEMSAmjB9MGABAaW0jmw9Us+nAETYdqGLZpoMsXlcIgNNqYvSAyGBARzN2QBTxTmsX/hcQQvS09oyMJwF5Wuu9AEqpJcA84PgwFiIktJzSBmjwNrC3ei95R/LIqwpsGw5v4P297zefYzfZiTfE8+mqT5tH0ZlRmSSGJZ5wqjvBaWP2cBuzhycC4Pdr8stcbDpQ1bz94/O9+PyBHwJSouzNo+exqVGMlOltIUJKe8I4BTjQ4nMRMPkE5a5SSs0EdgM/11ofOEEZIXodu8nOiNgRjIgd0Wp/rbuW/Kp88qvyyavK46t9X5FbnMs7ee80l3GanWRGZTIo+thUd1Z0FjG2mFZ1GQyKrEQnWYlOrgmOnhvcPrYerGZTYRWbiqrYVFj1nentMUentwdEMTDeIVduC9FLKa31qQsodTUwV2t9e/DzjcDkllPSSqlYwKW1blJK3QVcq7U+/wR13QncCZCYmDh+yZIlndYRl8uFw+HotPrOFqHYr1DsExzrV52vjhJPCQc9Bylxl3DIc4iDnoPU++uby8aaYkmzpJFmTSPdkk5/S38shrYXJ6lq8rOv2k9+lZ991T72Vvtp8AaOmQ3Q32Ggv9PAgBabw3L6AR2Kf1ah2CcIzX6FWp9mzZr1tdZ6womOtSeMpwIPa62/F/z8GwCt9R9PUt4IVGqtI09V74QJE/SGDRva0fz2ycnJITs7u9PqO1uEYr9CsU9w6n5pralorGDPkT3sqtzFlvItfFv+LYfqDgFgUiayorMYHT+aUXGjGBU3ivTIdAzq1AuI+P2aveUuNh2oZmdJDTsP1bKjpIaKOndzmcQIK0OTIhia7GR4cgRDkyIYGB/ersVJQvHPKhT7BKHZr1Drk1LqpGHcnmnq9UCWUiqDwNXS1wHXH/cFyVrrkuDHy4AdZ9BeIUKOUoo4exxx9jim9pvavL+svoxvy78NbGXf8v7e93lt12tAYIp7RNyI5nAeFT+KOHtcq3oNBsWgBCeDEpyt9pfWNrKzpJadh2rYWVLLjkO1rM4vx+ML/PBtNgbOG5bkZGiyszms4x1WuQ9aiB7QZhhrrb1KqXuAjwnc2rRIa71NKfUosEFrvQz4mVLqMsALVAI3d2GbhQgZ8WHxnJ96PuenBn6r49d+9lXvY0tZYOS8tXwri7Yuwqd9APQL7xe4LSsY0MNih53wMZIJThsJThszB8c37/P4/Owtq2PnoRp2lARG0Kvyy3nrm+LmMrHhFoYmOxmWFMHQ5Ahqq31MbPISLg/HEKJLtetvmNb6A+CD4/Y91OL9b4DfdG7ThOh7DMpAZlQmmVGZXJF1BRC4mntHxY5WI+iP938MgFEZyYrOYlTcKIZED2m+UCzS+t3fEpmNBoYkORmS5GTe2GP7K+vczSPonYcCU93/s7aAJq8fgEfWfExKlJ3MBAeD4h0MSji2xYTL4yWF6Azy464QZzm7yc64xHGMSxzXvK+8oZyt5VubR9Af7fuIN3a/0Xw8ISyBrKiswBXc0YPIispiYNTAE46iY8ItwadQHZsC9/k1+8rreHvlWmzxaeSVucgrdfHVvgoaPf5W5w6KdwSCusXWL9Im091CdICEsRC9UJw9juwB2WQPyAYCF4gdrj/MniN7mu+F3nNkD6/ufBW3P3Axl0IxwDmgVUBnRWeRGpGK2dB6TW6jQTEowcHEJBPZ2VnN+/1+TXFVA3llLvJLAwGdV+riw60lVNV7msuFW4zNI+mWQZ0WE4ZJnmolxHdIGAsRApRSJIUnkRSexLn9z23e7/P7OFB7IBDOVXvIOxJ4/bzo8+bfQ5sMpuZVxY4G9KCoQSdcVcxgUM2ric0aktC8X2tNRZ27OZzzSl3kl7lYs7ei1e+kzUZFWmw46bFhpMWGk3b0NSaMlGi7PH5S9FlnVRh7PB6KiopobGzs8LmRkZHs2BF6F3H3ZL9sNhv9+/fHbG7fk4zE2cdoMJIemU56ZDoXpl3YvL/J18T+6v3NAZ1XlceWsi18uO/D5jJ2k51oFc2SFUtIDEsk3h5PQlhC4DU8gQR7AjG2GIwGY+BqcYeVOIeVKQNjW7XB1eQ9NooOjqgLK+tZlVdBg8fXoq2KlCh7MKDDSI8NJzUmjPS4wKvNLCuOidB1VoVxUVERTqeT9PT0Dv++qba2FqfT2XbBXqan+qW1pqKigqKiIjIyMrr9+0XXshqtDIkZwpCYIa32u9wu8qvzm0fQW/Zvoaqxij2VeyhvLMev/a3KG5SBOFsc8WGBoG4O66Pvw+JJsCcwun8kYwZEtTpXa01ZbRMFlfXsL6+jsLKe/RX1FFbU8d7mEqobPK3KJ0ZYm0fRRwM6PTac1NgwIu3yA6Po3c6qMG5sbDytIBadTylFbGwsZWVlPd0U0Y0cFgdj4scwJn4MADn1xxZd8Pl9VDRWUFZfRml9KWUNZRyuPxz43FBKsauYb0q/oaqp6jv1WgyWVoHdz9GP/o7+pDhSSIlOYcyAfliMra/Mrqp3U1BRz/6KOgorgkFdWUfO7jLKvi5qVTY6zMyAmDD6R9tJibLTPzrwvn90YPrbIbdmibPcWfd/qATx2UP+LERLRoOxOUxHMOKk5Zp8TZQ3lFNW3zqsj4b4zsqdrCxcicd/bOSrUMSHxR8LaGdK4NWRwuSsAVwyemCrR1fWNXkprKynoKKegoo6CirrKTrSwM5DtazYUYrb23oEHx1mDgRzlB1d18R+875AYMcEwttpk5G16FlnXRj3NIfDgcvl6ulmCNFrWY3W5iA9Gb/2U1ofGE0Xu4opri2myFVEsauY9YfX8/7e95sfWwmBi8z6hfdrFdT9Hf1JSUxh0qAUoq0Dm3949Ps15XVNFB9poKh5C4R1XpmLwnIvH+9v/dC5qDBzcETdelSdHGkjKdJGTJhFnjEtupSEsRCi2xmUofnq7/GJ479z3OPzUFJX0hzQxbXFzcG9snAllY2VrcrbTXYSwxKJs8cRb48nLiyw9Gi8I56x8XHMtscTH5ZKhCWCnJwcRk2c1iqki4Pv95bV8cXu8lYXlkHgKvAEp42ECCuJzkBAt3yfGGElMcKGw2qSGSVxWiSMT0JrzX/8x3/w4YcfopTiwQcf5Nprr6WkpIRrr72WmpoavF4vzzzzDNOmTeO2225jw4YNKKW49dZb+fnPf97TXRCi1zIbzaRGpJIakXrC4/We+mOjalcxRbVFlDWUUVZfxraKbZQVldHgbfjOeRaDBYdy0L+hfyC07XHER8VzTr94ZtvjiLcnEGePQ/vCKalyU1LdyOGawHaoppHSmibyy1ysyi+ntvH/t3fv0VFV58PHvzuZSSYXkkwuJgRQgpWLEAICIrQCwg9QioBWTCnaEF9xgQoqbZEibflVtCpoa7tYAvUFwcKLFMpbVCw/eSGlLEBBGi4CooUAAcGEDJOEXCYzs98/ZjLkMiEDCZkLz2etWXMu+8zsZ85Mnpx9ztnb3uj1oyPCSY27kpyvPCJJc0+ntIuUK8NFIwGb4K0oHAAAGbFJREFUjP/7wy85cq7U5/IOh4Pw8Kt/we9Mj+M3DzZ9rquuv/3tb+Tn53PgwAGKi4sZMGAAQ4YMYc2aNYwePZqXXnoJh8NBRUUF+fn5nD17lsOHDwNw6VLjC1iEEK0n2hjNHWbXPdFNuVxzmaKKIooqizznsIsrizlccJgwQxgF1gL2nt9Lqa3x35lwFU6iKZGkqCTiIuKIN8UTFxdHx8g44iPiiY+MxxQei70mCpstksqqSMoqjJSUKS6UVfNdaRX/Pn2J86VVjc5fg+scdt1EXTdxp7mXJcVGyvjUN5GATcb+tnPnTiZNmkR4eDipqakMHTqUvXv3MmDAAJ544glqamqYMGECffr0oUuXLpw4cYIZM2bwwx/+kFGjRvm7+kLc9GKMMcTEx9A5vnO95Xnl9Yflq3vBWXFlsecIu7iyGEuVBavNyolLJ7DarFirrfUuPGvIoAzERcYRnxbPrbfGkRkRT5ShHeE6GpzROOxR1NiicdiSqCiPprjM1S94UVk1zgaj2YaHKVJiIxsk6yvTafEmUtuZiIuSpvFQELDJ2Ncj2FptdT/ukCFD2LFjBx9//DFTpkxh1qxZ/PSnP+XAgQNs2bKFJUuWsG7dOpYvX37D6yKEaDlfLjirpbWm0l5Jqa0Ua7W10XO9aZuVosoiSm3/wVptpbym8YWhaWlpZHXN4La4zqSYOhGr0jE6U6msiuW70mpP8/ipixV8XlBSr8vRWiZjGKlxJiIcVXxQ+IWn85WUdpEkx0a4nyOleTzABWwy9rd7772XpUuXkpOTQ0lJCTt27GDhwoWcOnWKjh07MnXqVKqrq9m/fz9jxowhIiKCH/3oR3Tr1o3HHnvM39UXQtwASimijdFEG6NJi0m7pm3tTjtltjKKK4s5VXqKk9aTnseHJzZxueayp2yMMYaMuAwy0jIY0i2DjPgMMuJv5xZTByyXnZwvvXIu25Wwq/nmzHm+/s7VBam3pA3QLtJAcrtIUmIjSW4X4XqOjayz7EoSjzRI4m5Lkoyb8NBDD7F7926ysrJQSvHGG2+QlpbGypUrWbhwIUajkdjYWFatWsXZs2fJzc3F6XSdG/rd737n59oLIQKNIcyA2WTGbDI3Otettaaosqhegj5pPcneC3v58MSHnnLhKpyO7Tq6EnW8K1n375ZBRnwX9u+6xH33DQXAZndy8XI1xWU2isqr3M/VFJVVU1ReTXFZNV+dL2NnWTGlXi5EA2hnMpASG0liTATmmAiS6jzXWxYdQVJsBNERkk5aQj69BmrvMVZKsXDhQhYuXFhvfU5ODjk5OY22279/f5vUTwgRepRSng5VBrYfWG9dRU0FBaUF9RN16Ul2ndvlGZELwKiMJK1PItGUiNlkJjEy0ZP8kxKS6Jlmdi9PwWwyE2OMQSlFVY2Di5dtFJe5knVx+ZXn4nIbJZdtnCmpIP/MJSyXbdgbntx2MxnDSIyOIDHWnaBjIkiMiSQxxog5OoJ20U5iTJq4qHAyzKkkRkfKvdt1SDIWQogAFm2M5s6kO7kz6c56yx1OB+cun/Mk6H8f/zcxKTFYqixYqiycvHQSS7XF6y1e4LrNy2wye5K32WTGHGkmKSqJ9AQzPU1mTOEmKh2VVNmrqLRXUmmvxFpVgaXyMtaqy5RWV1Buq+ByTQWVNVVUOaooclRyzlmNvcKGs6Iaim2osPrN5tppxGlLxuBIJVqlEW/oQEpkJ9rH3Motse08SbywyE5SoZXE2AgSoyOIigjdpnNJxkIIEYTCw8Lp1K4Tndp1YkjHIdxWdBvDfjCsUblKeyWWKgslVSWUVJV4knVJdQkllSVYql3zp0pPUVJV0mTyritMhWEKNxFliMJkMBEVEUVydBQmQztM4SlXlhuiMIZFgjMCh8OAw2Gkwubg28tnuVB5hhLbWS7rA1xGc07DgXJwWuJw2lJcj+oU/nDsEE5bMromgSijkUQvzeSJMUYSoiNIiDaSEOV6jo8ykhBtDJqOWCQZCyFECIsyRBEVG+V1fGpvquxVnmRdba92bV8nuboSrLHVEpzNYeN06WkKSgsoKC3gxKUT/OfSSU6VHuay/coV6AYVQUxYGibSqLanUliVzNeXEim1JnC5uum+xcPDlCsxRxmJj3Y9J0RHEB91JWHXJvG42nn3OkMbjq8tyVgIIYSHyWCifWx72se2b5P3iwiP4Hvm7/E98/fqLdda8+G2D0nvmc7J0pMUWAs8587Plu/DGeWEKAhLgi5RKcQY41A6DO1+OHUYTqfC6VQ4nGHYHYoih+KcQ1FzCWqKocau0DoMCAMdDu5prcNAhxETEc2/Z7bNBbmSjIUQQgQcpRRx4XH0T+tP/7T+9dbZHDbOlJ3hpPWkJ0FX1FRg13YcTgcO7cDhdNSbtzvtjeYdTgc1Tjt2Z+16B073eid2wlRsm8UryVgIIURQiQiP4PaE27k94fYb+j5O3bgr0xul7RrEhRBCiCASptouRUoy9hO73fuN9kIIIW4+koy9mDBhAv369aNnz54sW7YMgH/84x/cddddZGVlMWLECMDVQUhubi6ZmZn07t2bDRs2ABAbe+U8w/r165kyZQoAU6ZMYdq0aQwcOJDZs2fz+eefM2jQIPr27cvgwYP56quvANcIVD//+c/p1asXgwYN4k9/+hPbtm1jwoQJntf99NNPeeihh9ri4xBCCHGDBe4540/mwPlDPhePctghvJlw0jLhgdeafa3ly5eTmJhIZWUlAwYMYPz48UydOpUdO3aQkZFBSYlrYPOXX36Z+Ph4Dh1y1dNisTT72oWFhezatYvw8HBKS0v517/+hcFgYOvWrcydO5cNGzawbNkyCgoKyM/Pp7KykpqaGsxmM08//TRFRUWkpKSwYsUKnnjiieY/GCGEEAEvcJOxH/3xj39k48aNAJw5c4Zly5YxZMgQMjIyAEhMTARg69atrF271rOd2Wxu9rUnTpzoGXfZarWSk5PD119/jVKKmpoaz+tOmzYNg8FQ7/0ef/xx/vKXv5Cbm8vu3btZtWpVK0UshBDCnwI3GftwBFtXZSsNoZiXl8fWrVvZvXs30dHRDBs2jD59+nDs2DGfX6PuzfBVVVX11sXExHimf/WrX3HfffexceNGCgoK6o2x6k1ubi4PPvggJpOJiRMnepK1EEKI4CbnjBuwWq2YzWaio6M5duwYe/bsoaqqih07dnDy5EkATzP1yJEjWbx4sWfb2mbq1NRUjh49itPp9BxhN/VeHTq4xlB97733PMtHjhzJ0qVLPRd51b5feno66enpLFiwgNzc3NYLWgghhF9JMm7g/vvvx26306NHD+bMmcM999xDSkoKy5Yt4+GHHyYrK4vs7GwA5s2bh8VioVevXmRlZbF9+3YAXnvtNcaOHcvgwYNp377pXmxmz57NL3/5S/r27Vvv6uonn3ySW2+9ld69ezN48GDWrFnjWTd58mQ6depEjx49btAnIIQQoq351M6plLofeBsIB97VWnttQ1ZK/QhYDwzQWu9rtVq2ocjISD755BOv6x544IF687GxsaxcubJRuUceeYRHHnmk0fK6R78AgwYN4vjx4575BQsWAGAwGHjrrbd46623KGvQ/L5z506mTp3qczxCCCECX7PJWCkVDiwGRgKFwF6l1Cat9ZEG5doBzwGf3YiKCujXrx8xMTG8+eab/q6KEEKIVuTLkfHdwDda6xMASqm1wHjgSINyLwOvA79o1RoKjy+++MLfVRBCCHED+HLOuANwps58oXuZh1LqLqCT1vrjVqybEEIIcVNo8b0xSqkw4C1gig9lnwKeAtcVx3l5efXWx8fHU1ZWdl31cDgc171tIPN3XFVVVY32U0uVl5e3+msGglCMS2IKHqEYVyjG1BRfkvFZoFOd+Y7uZbXaAb2APPf9tWnAJqXUuIYXcWmtlwHLAPr3768b3ld79OjR675XuOGFTqHC33GZTCb69u3bqq+Zl5fX7D3VwSgU45KYgkcoxhWKMTXFl2bqvcAdSqkMpVQE8GNgU+1KrbVVa52ste6ste4M7AEaJWIhhBBCeNdsMtZa24FngS3AUWCd1vpLpdRvlVLjbnQFhRBCiFDn0zljrfVmYHODZb9uouywllcrOMTGxlJeXu51XUFBAWPHjuXw4cNtXCshhBDBRnrgEkIIIfwsYEcaeP3z1zlW4vvgDA6HwzMaUlO6J3bnxbtfbHL9nDlz6NSpE8888wwA8+fPx2AwsH37diwWCzU1NSxYsIDx48f7XC9wXZE8ffp09u3b5+ld67777uPLL78kNzcXm82G0+lkw4YNpKen8+ijj1JYWOgZ17h2PGQhhBChKWCTsT9kZ2fz/PPPe5LxunXr2LJlCzNnziQuLo7i4mLuuecexo0bV29kpuYsXrwYpRSHDh3i2LFjjBo1iuPHj7NkyRKee+45Jk+ejM1mw+FwsHnzZtLT0/n4Y9ct24WFhTckViGEEIEjYJPx1Y5gvWmNW4D69u3Ld999x7lz5ygqKsJsNpOWlsYLL7zAjh07CAsL4+zZs1y4cIG0tDSfX3fnzp3MmDEDgO7du3Pbbbdx/PhxBg0axCuvvEJhYSEPP/wwd9xxB5mZmfzsZz/jxRdfZOzYsfTp06dFMQkhhAh8cs64gYkTJ7J+/Xo++OADsrOzWb16NUVFRXzxxRfk5+eTmpraaIzi6/WTn/yETZs2ERUVxZgxY9i2bRtdu3Zl//79ZGZmMm/ePF577drGdRZCCBF8AvbI2F+ys7OZOnUqxcXF/POf/2TdunXccsstGI1Gtm/fzqlTp675Ne+9915Wr17N8OHDOX78OKdPn6Zbt26cOHGCLl26MHPmTE6fPs3Bgwfp3r07iYmJPPbYYyQkJLBkyZIbEKUQQohAIsm4gZ49e1JWVkaHDh1o3749kydP5sEHHyQzM5P+/fvTvXv3a37Np59+munTp5OZmYnBYOC9994jMjKSdevW8f7772M0GklLS2Pu3Lns3buXX/ziF4SFhWE0Glm0aNENiFIIIUQgkWTsxaFDhzzTycnJ7N6922u5pu4xBujcubPnHmOTycSKFSsalZkzZw5z5sypt2z06NGMHj3aMx+K/W0LIYSoT84ZCyGEEH4mR8YtdOjQIR5//PF6yyIjI/nss8/8VCMhhBDBRpJxC2VmZpKfn+/vagghhAhi0kwthBBC+JkkYyGEEMLPJBkLIYQQfibJWAghhPAzScYtEBsb6+8qCCGECAGSjEOA3W73dxWEEEK0QMDe2nT+1VepPur7eMZ2h4OSZsYzjuzRnbS5c5tc35rjGZeXlzN+/Hiv261atYpFixahlKJ37968//77XLhwgWnTpnHixAkA3nnnHdLT0xkzZgxHjhwBYNGiRZSXlzN//nyGDRtGnz592LlzJ5MmTaJr164sWLAAm81GUlISq1evJjU1lfLycmbMmMG+fftQSvGb3/wGq9XKwYMH+cMf/gDAn//8Z44cOcLvf//75j9oIYQQrS5gk7E/tOZ4xiaTiY0bNzba7siRIyxYsIBdu3aRnJxMSUkJADNnzmTo0KFs3LgRh8NBeXk5Fovlqu9hs9nYt28fABaLhT179qCU4t133+WNN97gzTff5OWXXyY+Pt7TxafFYsFoNPLKK6+wcOFCjEYjK1asYOnSpS39+IQQQlyngE3GVzuC9SbQxjPWWjN37txG223bto2JEyeSnJwMQGJiIgDbtm1j1apVAISHhxMfH99sMs7OzvZMFxYWkp2dzbfffovNZiMjIwOArVu3snbtWk85s9kMwPDhw/noo4/o0aMHNTU1ZGZmXuOnJYQQorUEbDL2l9rxjM+fP99oPGOj0Ujnzp19Gs/4erery2Aw4HQ6PfMNt4+JifFMz5gxg1mzZjFu3Djy8vKYP3/+VV/7ySef5NVXX6V79+7k5uZeU72EEEK0LrmAq4Hs7GzWrl3L+vXrmThxIlar9brGM25qu+HDh/PXv/6VixcvAniaqUeMGME777wDgMPhwGq1kpqaSlFRERcvXqS6upqPPvroqu/XoUMHAFauXOlZPnLkSBYvXuyZrz3aHjhwIGfOnGHNmjVMmjTJ149HCCHEDSDJuAFv4xnv27ePzMxMVq1a5fN4xk1t17NnT1566SWGDh1KVlYWs2bNAuDtt99m+/btZGZm0q9fP44cOYLRaOTFF1/k7rvvZuTIkVd97/nz5zNx4kT69evnaQIHmDdvHhaLhV69epGVlcX27ds96x599FG+//3ve5quhRBC+Ic0U3vRGuMZX227nJwccnJy6i1LTU3l73//e6Oy06dPZ/bs2Y2W5+Xl1ZsfP36816u8Y2Nj6x0p17Vz505eeOGFpkIQQgjRRuTI+CZ06dIlunbtSlRUFCNGjPB3dYQQ4qYnR8YtFIzjGSckJHD8+HF/V0MIIYSbJOMWkvGMhRBCtFTANVNrrf1dBeEm+0IIIdpGQCVjk8nExYsXJQkEAK01Fy9exGQy+bsqQggR8gKqmbpjx44UFhZSVFR0zdtWVVWFZOLwZ1wmk4mOHTv65b2FEOJm4lMyVkrdD7wNhAPvaq1fa7B+GvAM4ADKgae01keutTJGo9HTjeO1ysvLo2/fvte1bSAL1biEEEJc0WwztVIqHFgMPADcCUxSSt3ZoNgarXWm1roP8AbwVqvXVAghhAhRvpwzvhv4Rmt9QmttA9YC9XqX0FqX1pmNAeSkrxBCCOEjX5qpOwBn6swXAgMbFlJKPQPMAiKA4a1SOyGEEOIm0GoXcGmtFwOLlVI/AeYBOQ3LKKWeAp5yz5Yrpb5qrfcHkoHiVny9QBGKcYViTBCacUlMwSMU4wq1mG5raoUvyfgs0KnOfEf3sqasBd7xtkJrvQxY5sN7XjOl1D6tdf8b8dr+FIpxhWJMEJpxSUzBIxTjCsWYmuLLOeO9wB1KqQylVATwY2BT3QJKqTvqzP4Q+Lr1qiiEEEKEtmaPjLXWdqXUs8AWXLc2Lddaf6mU+i2wT2u9CXhWKfVfQA1gwUsTtRBCCCG88+mcsdZ6M7C5wbJf15l+rpXrdT1uSPN3AAjFuEIxJgjNuCSm4BGKcYViTF4p6XpSCCGE8K+A6ptaCCGEuBkFXTJWSt2vlPpKKfWNUmqOl/WRSqkP3Os/U0p1bvtaXhulVCel1Hal1BGl1JdKqUbN/kqpYUopq1Iq3/34tbfXCiRKqQKl1CF3ffd5Wa+UUn9076uDSqm7/FFPXymlutX5/POVUqVKqecblAmK/aSUWq6U+k4pdbjOskSl1KdKqa/dz+Ymts1xl/laKRUw14c0EdNCpdQx9/dro1IqoYltr/pd9acm4pqvlDpb53s2poltr/r30l+aiOmDOvEUKKW8jk0byPuqRbTWQfPAdQHZf4AuuDoXOQDc2aDM08AS9/SPgQ/8XW8f4moP3OWebgcc9xLXMOAjf9f1GuMqAJKvsn4M8AmggHuAz/xd52uILRw4D9wWjPsJGALcBRyus+wNYI57eg7wupftEoET7meze9rs73iuEtMowOCeft1bTO51V/2uBmBc84GfN7Nds38vAymmBuvfBH4dbPuqJY9gOzJutmtO9/xK9/R6YIRSSrVhHa+Z1vpbrfV+93QZcBRXz2ehbjywSrvsARKUUu39XSkfjQD+o7U+5e+KXA+t9Q6gpMHiur+dlcAEL5uOBj7VWpdorS3Ap8D9N6yi18BbTFrr/9Fa292ze3D1kxBUmthXvvDl76VfXC0m99/rR4H/06aV8rNgS8beuuZsmLQ8Zdw/QiuQ1Ca1awXuZvW+wGdeVg9SSh1QSn2ilOrZphW7Phr4H6XUF+7e1xryZX8Gqh/T9B+LYNtPtVK11t+6p88DqV7KBPM+ewJXS4w3zX1XA9Gz7ub35U2cUgjWfXUvcEFr3VR/FcG4r5oVbMk4pCmlYoENwPO6/uAbAPtxNYlmAX8C/m9b1+86/EBrfReuEb+eUUoN8XeFWoO785txwF+9rA7G/dSIdrUHhsytFkqplwA7sLqJIsH2XX0HuB3oA3yLq1k3VEzi6kfFwbavfBJsydiXrjk9ZZRSBiAeuNgmtWsBpZQRVyJerbX+W8P1WutSrXW5e3ozYFRKJbdxNa+J1vqs+/k7YCOuZrO6rrWr1UDxALBfa32h4Ypg3E91XKg9TeB+/s5LmaDbZ0qpKcBYYLL7n4xGfPiuBhSt9QWttUNr7QT+jPf6BuO+MgAPAx80VSbY9pWvgi0ZN9s1p3u+9grPR4BtTf0AA4X7HMn/Bo5qrb2OBa2USqs9962UuhvXvgvYfzKUUjFKqXa107gupDncoNgm4Kfuq6rvAax1mkkDWZP/uQfbfmqg7m8nB/i7lzJbgFFKKbO7aXSUe1lAUkrdD8wGxmmtK5oo48t3NaA0uLbiIbzX15e/l4Hmv4BjWutCbyuDcV/5zN9XkF3rA9cVuMdxXSX4knvZb3H92ABMuJoPvwE+B7r4u84+xPQDXE2CB4F892MMMA2Y5i7zLPAlrisi9wCD/V3vZmLq4q7rAXe9a/dV3ZgUsNi9Lw8B/f1dbx/iisGVXOPrLAu6/YTrn4lvcXVhWwj8L1zXVvw/XH3LbwUS3WX7A+/W2fYJ9+/rGyDX37E0E9M3uM6b1v6uau+0SAc2X+27GiiPJuJ63/2bOYgrwbZvGJd7vtHfy0B4eIvJvfy92t9SnbJBs69a8pAeuIQQQgg/C7ZmaiGEECLkSDIWQggh/EySsRBCCOFnkoyFEEIIP5NkLIQQQviZJGMhhBDCzyQZCyGEEH4myVgIIYTws/8PwYRgSoBxB3sAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "30GbhYf6kqcR", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "bcacf17e-3eb2-4fe3-ab57-b90452a44fdb" }, "source": [ "y_pred = model.predict(X_test)\n", "y_pred = y_pred.argmax(axis=1)\n", "print(classification_report(y_test, y_pred,\n", " target_names=class_names))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " T-shirt/top 0.79 0.82 0.80 1000\n", " Trouser 0.97 0.95 0.96 1000\n", " Pullover 0.78 0.64 0.71 1000\n", " Dress 0.83 0.85 0.84 1000\n", " Coat 0.71 0.76 0.74 1000\n", " Sandal 0.94 0.92 0.93 1000\n", " Shirt 0.57 0.60 0.59 1000\n", " Sneaker 0.90 0.91 0.91 1000\n", " Bag 0.94 0.94 0.94 1000\n", " Ankle boot 0.92 0.94 0.93 1000\n", "\n", " accuracy 0.83 10000\n", " macro avg 0.83 0.83 0.83 10000\n", "weighted avg 0.83 0.83 0.83 10000\n", "\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "Bs-6h3DZlvTB" }, "source": [ "### ELU\r\n", "\r\n", "Very Similar to leaky ReLU, ELU has a small slope for negative values. Instead of a straight line, it uses a log curve" ] }, { "cell_type": "code", "metadata": { "id": "sIl-VXANkx-T" }, "source": [ "def elu(z, alpha=1):\n", " return np.where(z < 0, alpha * (np.exp(z) - 1), z)" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "NBOM_DG9lxAu", "colab": { "base_uri": "https://localhost:8080/", "height": 284 }, "outputId": "5a75c5b7-3be5-4795-f4f6-883de605f07b" }, "source": [ "plt.plot(z, elu(z), \"b-\", linewidth=2)\n", "plt.plot([-5, 5], [0, 0], 'k-')\n", "plt.plot([-5, 5], [-1, -1], 'k--')\n", "plt.plot([0, 0], [-2.2, 3.2], 'k-')\n", "plt.grid(True)\n", "plt.title(r\"ELU activation function ($\\alpha=1$)\", fontsize=14)\n", "plt.axis([-5, 5, -2.2, 3.2]);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAELCAYAAADECQ0AAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd5xU1d3H8c+PpRcBQREFxR6xhCjhsbuxa7DFFgsGjWJsAQMaRX2eGAnG2DCiKGpCROzYe2OCQYKCQnBpUgxVijjAwlJ29zx/nFl22R22zs6Zu/N9v173xeycmXt/c/bulztnztxrzjlERCS6GoUuQERE6kZBLiIScQpyEZGIU5CLiEScglxEJOIU5CIiEacgFxGJOAW5iEjEKcilTsxslJm91YC208jMHjez783MmVlufW+zklrS8poT22pvZsvNbO90bK+mzOwlMxsYuo5MZfpmZ/qY2SjgV0maJjnnDk+0d3TO9d7O82PA186568vd3xcY7pxrndKCq7fttvj9KB6l7VSy/d7AK0AuMB9Y7ZzbXJ/bTGw3RrnXna7XnNjWvfh97/L63laSbR8LDAIOA3YFLnfOjSr3mIOBfwJ7OufWpLvGTNc4dAFZ6COgT7n76j0o6ku6/qjS+Me7D7DMOfdZmra3Xel6zWbWErgSOCMd20uiNfA18HRiqcA5N93M5gOXAo+ksbZI0NBK+m1yzn1Xblld3xs1s1PN7FMz+8HMVpvZ+2Z2QJl2M7OBZvaNmW0ys8VmdneibRRwHHBdYrjBmVm3kjYze8vM+iXemueU2+6zZvZGdeqoznbKrKeZmQ1LbHOjmf3bzI4u0x4zs0fNbKiZrTKzFWZ2n5ltd59PbP9BYPfEtr8ts67h5R9bUk91tlWb/q3pa67t6wZOBxwwIUmfHGZmH5tZgZnNNbNjzewCM6vw2Npyzr3jnBvsnHsZKK7koW8AF6Vquw2Jgjx7tAKGAb3wwwZrgDfNrGmifShwB3A3cCBwPrAo0dYfmAj8HeicWEraSrwEtAVOKrnDzFoDZwHPVLOO6mynxF+AC4ErgJ8A04H3zKxzmcdcAhQCRwLXAwMSz9me/sAfgcWJbf+0kseWV9W26tq/UL3XXJ1ayjsGmOLKjbOa2U+BT4FxwCHAv4E7gdsSr4Vyjx9sZvlVLMdUUkdVPgd6mVmLOqyjYXLOaUnTAozC/4Hll1vuKdP+ViXPj+HHwsvf3xfIr2EtrYAi4Gj8W9uNwG9qse2tNePHlkeXabsUH9TNq1NHDbbTCj8cdVmZ9hxgHjCkzHomllvHh8CTVfTLIODbql57uXoq3VZt+7emr7m2rxt4DfhHkvvHAy+U+fn0xO9q3HbWsyN+aKqypUUV/Z8P9N1O2yH4dw5712Rfz4ZFY+TpNx7oV+6+dHyYtTdwF/A/wE74d2ONgN3xAdEM+LiOm3kG+IeZtXTObcAfGY51zm2sZh3VtTfQhDJDAc65IjObCHQv87j/lHveUmDnGmynJirbVnfq3r/Vfc1V1ZJMC2B52TvMbBf8kfrPyty9Gf+7qnA0nqhnNVCfw4QFiX91RF6Ogjz9Njjn5tbyuWvxwxfltcMf+VbmLfyQwdXAEvw7gxlA08qeVENvJ9Z7lpl9DJwInJLmOsoOD2xJ0lab4cRiwMrd16Tcz6naVm2Un3pW01pWAe3L3Vfy+cnkMvftD8x2zv0r2UrMbDAwuPJSOc0592kVj9meHRP/rqzl8xssBXm0zAZONzNzifeaCYcm2pIysw7Aj4BrnXPjEvcdSunvfyawCTgB+GY7q9mMfyu/Xc65TWb2Ev5IvCPwHf6tfnXrqNZ28MMJm4GjErdJfMh6BPBsFc+tjZX4ceuyfgx8W83np6J/6/M1f4UfniurHf4/gKLEttrgx8a/q2Q9jwEvVrGtJbUrEYCDgCXOueVVPjLLKMjTr1nibWtZRc65kqOMHcysR7n2uHPuW2AE/sOrh83sCfy46+n4T/LPrGSbP+CPuq4ys0XAbsC9+KNhnHPrzOwh4G4z24Qf/ukAHOacG5FYx7f4D5q64ccxVzvnks0weAY/hLAn8Fy5x1RaR3W345xbb2YjgHvMbBWwALgR6AQ8Wkk/1NYnwDAzOxP/H+bVQFeqGeS17d9y66jP1/x+Yr0dnHPfJ+6bin8XcquZjcH/npYB+5jZvs65Cv8h1XZoJfGh+D6JHxvhZw31wP/uF5Z56DGJWqW80IP02bTgP7xySZbFVbS/XGYdP8XvzMvxwymTgLOrse3j8XN1Nyb+PYUyHyzh/4BuwX8JZjN+1sSfyjx/P/zMig2JmrqVqfmtMo8zfCg54JBa1FHd7TTDz35Zjj/a/TeJD0wT7TEq+fCwkn5K9mFnE/zc5VWJ5U4qfthZ6bZq0781fc11fN0TgevK3TcY/25kIzAGP/wyAViZ4r+LXJLv96PKPKY5fn8/PPTfcSYu+maniGBmpwIPAd2dc0Wh6ynPzK4DznLOnRy6lkykeeQignPuPfy7ji6ha9mOLcANoYvIVDoiFxGJOB2Ri4hEnIJcRCTigkw/7Nixo+vWrVuITW+1fv16WrVqFbSGTKG+8GbPnk1RURHdu5f/omR2ytT9orAQZs2CTZugfXvYa6/632am9MWUKVNWOed2Kn9/kCDv1q0bkydPrvqB9SgWi5Gbmxu0hkyhvvByc3OJx+PB981MkYn7xebNcMopPsQPPRQ+/RRatqz/7WZKX5jZf5Pdr6EVEYkE5+CGGyAWg86d4fXX0xPiUaAgF5FIePhhGDkSmjeH116DLpk6UTIABbmIZLz334cbb/S3//Y36NUrbD2Zps5BbmbNzexzM5tmZnlmdmcqChMRAf/B5oUXQnEx3H47XKRrBFWQig87NwHHO+fyzawJ8C8ze9c59+8UrFtEstjq1XDGGbBmDfziF3CnDhOTqnOQO//V0PzEj00Si74uKiJ1smULnH8+zJ0LPXrA009DIw0GJ5WS6YeJ8yJPwZ+K8hHn3KQkj+lH4so4nTp1IhaLpWLTtZafnx+8hkyhvvDi8ThFRUXqi4TQ+8WDD+7LJ5/sRvv2m7n11il88cWmYLWE7osqpfh0lO3wF2o9qLLHHXbYYS60cePGhS4hY6gvvOOOO879+Mc/Dl1Gxgi5Xwwf7hw416yZcxMnBitjq0z5GwEmuySZmtI3Ks65eCLIT03lekUke3z4IfTv728/9RQcfnjYeqIgFbNWdjKzdonbLYCTgFl1Xa+IZJ85c+CCC6CoCG69FS65JHRF0ZCKMfLO+Cun5+D/Y3jROfdWCtYrIlnkhx/8DJV4HM4+G4YMCV1RdKRi1sp/gJ+koBYRyVKFhf5IfM4cOOQQGD1aM1RqQl0lIsHdeCN89BHsvDO88Qa0bh26omhRkItIUI89BsOHQ9Om8OqrsMceoSuKHgW5iATzySdw/fX+9hNPwJFHhq0nqhTkIhLEN9/Aeef5GSo33wyXXRa6ouhSkItI2sXjfoZKyUyVoUNDVxRtCnIRSavCQn82w9mz4eCDYcwYyMkJXVW0KchFJK0GDYIPPoCOHf0MlTZtQlcUfQpyEUmbJ56Ahx6CJk38DJXA12BvMBTkIpIWsRhce62//fjjcPTRQctpUBTkIlLv5s2Dc8/14+MDB8Lll4euqGFRkItIvVqzxs9MWb0afv5zuOee0BU1PApyEak3RUX+GpszZ8KBB8Kzz2qGSn1QkItIvbnpJnj3XejQwc9Q2WGH0BU1TApyEakXTz0FDz4IjRvDK6/AXnuFrqjhUpCLSMqNHw/XXONvjxgBxx4btp6GTkEuIim1YIGfobJlCwwYAFdeGbqihk9BLiIps3atn6GyahWceirce2/oirKDglxEUqKoCC6+GPLy4IAD4Pnn/fi41D8FuYikxC23wNtvw447wptvQtu2oSvKHgpyEamzUaPgvvv8EfjYsbD33qEryi4KchGpkwkT4Oqr/e1HHoHc3KDlZCUFuYjU2rffwjnnwObNcMMN0K9f6Iqyk4JcRGpl3To480xYuRJOPhkeeCB0RdlLQS4iNVZcDJdeCtOnw/77wwsvaIZKSApyEamxwYP9uVPat/czVNq1C11RdlOQi0iNPP20PxVtTg68/DLsu2/oikRBLiLVNnEiXHWVv/3ww3D88WHrEU9BLiLVsnAhnH22n6Fy3XWlJ8WS8BTkIlKl/Hw/Q2XFCjjhBH96WskcCnIRqVRxMfTpA9Om+fHwl16CJk1CVyVlKchFpFJ33AGvveZnprz5pp+pIpmlzkFuZl3NbJyZzTCzPDPrn4rCRCS8MWNg6FA/Q+XFF/2ccck8qTgiLwQGOue6A4cD15lZ9xSsV0QCmjGjDb/+tb89bBicdFLYemT76hzkzrllzrkvE7fXATOB3eq6XhEJZ9EiuP32g9m0CX7zGz9LRTJXSsfIzawb8BNgUirXKyLps349nHUW/PBDU372M/jrX8EsdFVSmZSdHcHMWgNjgQHOubVJ2vsB/QA6depELBZL1aZrJT8/P3gNmUJ94cXjcYqKirK6L4qL4c47D+Srr3aic+f19O//FRMmFIYuK7hM/xtJSZCbWRN8iI9xzr2S7DHOuZHASICePXu63MAnLY7FYoSuIVOoL7x27doRj8ezui/+939h/HjYYQe4++48zjrr6NAlZYRM/xupc5CbmQFPATOdczqRpUhEPf883HUXNGrkz2bYvPmG0CVJNaVijPwooA9wvJlNTSynp2C9IpImn38Ol1/ubz/wAJx6ath6pGbqfETunPsXoI9CRCJqyRJ/DpWNG/0JsX7729AVSU3pm50iWWzDBj9DZdkyOO44GD5cM1SiSEEukqWc88MpU6bAXnv5c4s3bRq6KqkNBblIlvrjH/3X7tu08Vf76dgxdEVSWwpykSz00kvwhz/4GSrPPw8HHhi6IqkLBblIlpkyBX71K3/73nvhdM0xizwFuUgWWbrUXyCioACuuAJuvDF0RZIKCnKRLFFQ4KcZLl0KxxwDI0ZohkpDoSAXyQLO+SPwL76Abt1g7FjNUGlIFOQiWeBPf/IfarZu7a/ys9NOoSuSVFKQizRwY8f6y7WZwXPPwUEHha5IUk1BLtKAffUVXHaZv33PPdC7d9h6pH4oyEUaqGXL/AyVDRv8dMNBg0JXJPVFQS7SAG3cCOecA4sXw1FHweOPa4ZKQ6YgF2lgnIMrr4RJk2CPPeCVV6BZs9BVSX1SkIs0MH/+M4wZA61a+XOo7Lxz6IqkvinIRRqQ116DwYP9MMqYMXDIIaErknRQkIs0ENOmwaWX+ttDh/rzjEt2UJCLNADLl8MZZ8D69dCnD/z+96ErknRSkItEXMkMlUWL4PDDYeRIzVDJNgpykQhzDvr1g4kToWtXP0bevHnoqiTdFOQiEXbvvTB6NLRs6WeodOoUuiIJQUEuElFvvAG33OJvP/MM9OgRth4JR0EuEkHTp8Mll/ihlSFD/Bi5ZC8FuUjErFjhZ6jk58PFF/t545LdFOQiEbJpE/ziF/Df/0KvXvDkk5qhIgpykchwDn7zG5gwAbp08TNUWrQIXZVkAgW5SETcfz+MGuXD+/XXoXPn0BVJplCQi0TA22/DzTf726NHw6GHhq1HMouCXCTD5eXBRRf5oZU//hHOPTd0RZJpFOQiGWzVKj9DZd06uPBCuP320BVJJlKQi2SozZv90feCBdCzJ/z975qhIsmlJMjN7G9mtsLMvk7F+kSynXNw7bUwfjzsuqv/cFMzVGR7UnVEPgo4NUXrEsl6w4bBU0+VzlDZddfQFUkmS0mQO+fGA6tTsS6RbPfuu6VXvB81yg+riFRGY+QiGWTGDPjlL6G4GP7v/+CCC0JXJFHQOF0bMrN+QD+ATp06EYvF0rXppPLz84PXkCnUF148HqeoqChYX6xZ05hrrz2MtWtbcNxxKzj22BmE/LVovyiV6X2RtiB3zo0ERgL07NnT5ebmpmvTScViMULXkCnUF167du2Ix+NB+mLzZjjlFFi61H/Z5513dqZly53TXkdZ2i9KZXpfaGhFJDDn4IYbIBbzX7t//XV/oQiR6krV9MPngInA/ma22Mx+nYr1imSDhx/219ls3tyfCKtLl9AVSdSkZGjFOXdRKtYjkm3efx9uvNHf/tvf/KlpRWpKQysigcya5b92X1zsv3p/kQ6HpJYU5CIBrF7tz6GyZo2/UMSdd4auSKJMQS6SZlu2wPnnw9y5/oLJTz8NjfSXKHWg3Uckzfr3h08+gU6d4I03oFWr0BVJ1CnIRdLokUdgxAho1szPUOnaNXRF0hAoyEXS5MMP/dE4+BNiHX542Hqk4VCQi6TBnDn+vClFRXDrrXDJJaErkoZEQS5Sz374wc9Qicfh7LNhyJDQFUlDoyAXqUdbtvgj8Tlz4JBD/IWTNUNFUk27lEg9+t3v4KOPYOed/QyV1q1DVyQNkYJcpJ489hgMHw5Nm8Krr8Iee4SuSBoqBblIPfjkE7j+en/7iSfgyCPD1iMNm4JcJMW++QbOO8/PULn5ZrjsstAVSUOnIBdJoXjcz1ApmakydGjoiiQbKMhFUqSw0J/NcPZsOPhgGDMGcnJCVyXZQEEukiIDB8IHH8BOO/kZKm3ahK5IsoWCXCQFRo6Ev/4VmjSBV16Bbt1CVyTZREEuUkfjxsF11/nbI0fC0UeHrUeyj4JcpA7mzvUzVAoLYdAg6Ns3dEWSjRTkIrW0Zg2ceaa/2k/v3vDnP4euSLKVglykFgoL4Ze/hJkz4cADNUNFwlKQi9TCTTfBe+9Bx47w5puwww6hK5JspiAXqaEnn4Rhw0pnqOy5Z+iKJNspyEVq4J//hGuu8bcfewyOOSZsPSKgIBeptvnz4dxz/fj4734HV1wRuiIRT0EuUg1r1/pzp3z/PZx2GvzlL6ErEimlIBepQlERXHQRzJgBBxwAzz2nGSqSWRTkIlW4+WZ45x3YcUc/Q6Vt29AViWxLQS5SiaeeggcegMaNYexY2Hvv0BWJVKQgF9mO8eNLZ6g8+ijk5gYtR2S7FOQiSSxY4GeobNkC/fvDVVeFrkhk+xTkIuWUzFBZtQpOOQXuuy90RSKVS0mQm9mpZjbbzOaa2S2pWKdICM7BxRdDXh786Efwwgt+fFwkk9V5FzWzHOAR4CRgMfCFmb3hnJtR13WLpNuyZS34z380Q0WiJRXHGr2Auc65+QBm9jxwFrDdIJ89eza5gT85isfjtGvXLmgNmUJ94X3++VQKCgBy6doVrrwydEVhab8olel9kYog3w1YVObnxcD/lH+QmfUD+gE0adKEeDyegk3XXlFRUfAaMoX6Atavb5wIcejSZQOwmSzvEu0XZWR6X6Rt9M85NxIYCdCzZ083efLkdG06qVgsFvxdQabI9r7Iyyu5PFsuHTtuYtGiiaFLygjZvl+UlSl9YWZJ70/Fh51LgK5lfu6SuE8k4y1eDKeeCvE4dOgAu+5aELokkRpLxRH5F8C+ZrYnPsB/CVycgvWK1KsffvAhvngxHHUUNGrkpx6KRE2dj8idc4XA9cD7wEzgRedcXl3XK1Kf8vP9XPG8PH8irDfe8EEuEkUpGSN3zr0DvJOKdYnUt/Xr4ec/hwkToEsXf8m2HXcMXZVI7ekYRLLK+vX+ivfjx8Nuu8G4cbD77qGrEqkbBblkjZLhlFgMOnf2Ib7PPqGrEqk7fflYssKqVX445fPPYZddfIjvu2/oqkRSQ0fk0uAtXOjniX/+Oeyxh7+A8v77h65KJHUU5NKg5eX5qYWzZ8PBB8Nnn8F++4WuSiS1FOTSYL31FhxxhJ8nfvTR/gPOXXcNXZVI6inIpcFxzl/l/swzYd06uPBC+OADyOBzHonUiYJcGpT8fOjTB37/ex/oQ4b4q963aBG6MpH6o1kr0mBMnw4XXACzZkGrVjB6NJxzTuiqROqfjsgl8pyDJ5+EXr18iHfv7meoKMQlWyjIJdK++84H9lVXwcaNcMUV8MUXPsxFsoWCXCLrhRfgoIPg9ddhhx3g6afhqaegZcvQlYmkl8bIJXIWLoT+/eG11/zPJ53kh1Z0zhTJVjoil8jYssVPKzzgAB/irVvDY4/B++8rxCW76YhcMp5z8M47cNNNMHOmv+/88+GBB/xpaEWynYJcMtqXX8KgQf4kVwB77w3Dh/sr+4iIp6EVyUjTp/uj7sMO8yHevr0/As/LU4iLlKcjcskoU6fCn/4EL7/sf27WDK6/Hm67zYe5iFSkIJfgiorg7bfhwQf9RR/AB/jVV/uv2utEVyKVU5BLMOvWwahR8NBDMG+ev69NG7jySj8urgAXqR4FuaSVc/50sn//ux8+Wb/e39+tG/z2t/DrX/sv94hI9SnIJS0WLIBnnvFH4PPnl95/zDH+yz1nnw05OcHKE4k0BbnUmzlz/FH32LF+GmGJLl3gssugb19dN1MkFRTkkjKFhTBpErz3nv/m5ddfl7a1bu2vYN+3L5xwgo6+RVJJQS51snAhfPihD+8PP4Q1a0rb2rb1V+k591w4+WRd3EGkvijIpdqc8xcx/vRT/4Hl+PE+yMvad1//hZ3TTvNH3k2bhqlVJJsoyCUp5/xFiydPLl2mTIHvv9/2ce3awbHH+vA+5RTYa68w9YpkMwW5sHFjI7780n/9fcYMmDbNh/aKFRUf26mTD+6S5aCDoJFO9CASlII8S2zZAosW+al/8+fD3Ln+TIJ5efDtt8fgXMXntG8PPXtuu3TtCmbpr19Etk9B3kDk58OSJbB0qV8WLiwN7fnzfYgXFSV/bk6OY//9je7d4cAD/XLYYbDnngptkShQkGeo4mKIx/2Y9KpV2y4rV8KyZT6wS8J73brK12fmj6b32ssve+7pL9Bw4IGwZMmnnHjicel5YSKScnUKcjM7H/gDcADQyzk3ORVFRV1RERQUwIYNPmDXrvXT8qr6d82a0uD+/nsf5tXVvLk/N8luu/l/u3QpDe299oI99vAnokpm+fIk4yoiEhl1PSL/GvgF8HgKaqmR4mIfmCVLYWHFn7dsgc2bky+TJ+/IDz9U/piSpaCgNJg3bKj69ubNqXmNbdtCx44Vlw4doHPnbYO7XTsNg4hkqzoFuXNuJoDVMEG++mo2rVvn4hxbP2Rr2fICWre+li1bNrBq1elb20qWnJy+mPWlsHAVxcXnJVnrNcCFwCKgT5L2gcAZwGzg6iTttwMnAlOBAUnahwJHAp8Bg5O0DwN6AB8BQ2jUyM/maNzYf4vxgAMeZ5dd9mfdujf55pv7yckpbWvcGAYNGs0++3Tliy9e4JVXRtCkybbBPGrUy3Ts2JFRo0YxatSoClt/5513aNmyJY8++igvvvhihfZY4vyw9913H2+99dY2bQUFBUyaNAmAu+66i48//nib9g4dOjB27FgAbr31ViZOnLhNe5cuXXjmmWcAGDBgAFOnTt2mfb/99mPkyJEA9OvXjzlz5mzT3qNHD4YNGwbApZdeyuLFi7dpP+KII7j77rsBOPfcc/m+3BzIE044gTvuuAOA0047jYKCgm3ae/fuzaBBgwDIzc2lvAsuuIBrr72W4uJi5s6dW+Exffv2pW/fvqxatYrzzqu4711zzTVceOGFLFq0iD59Ku57AwcO5IwzzmD27NlcfXXFfe/222/nxBNPZOrUqQwYUHHfGzp0KEceeSSfffYZgwdX3PeGDRtGjx49+OijjxgyZEiF9scff5z999+fN998k/vvv79C++jRo+natSsvvPACI0aM2Hp/PB6nXbt2vPxy/e17LVq04N133wWye9/bsGEDp59+eoX2qva9EmkbIzezfkA//1PrrWe9K1FQUHGOclnbG2bwYedo0qSIpk23AFvYuNEBDjMfpmaO9u0LaN9+DYWFa1m6tBBwiTbfvs8+39O581Ly85fz9debMHOJNmjUyHHMMQvp1q09K1fOZ9y49TRq5Lauu1EjxxVXTOVHP8onL28azz8fr1DnDTdMYvfdl/HZZ9OJxyu2t2kzEefmsXZtHhs2VGyfMGECbdu2ZdasWUmfP378eJo3b86cOXOStpf8Mc2bN69Ce05Oztb2BQsWVGgvLi7e2r5w4cIK7U2aNNnavnjx4grtS5cu3dq+dOnSCu2LFy/e2r58+fIK7QsXLtzavnLlStauXbtN+4IFC7a2r169mk2bNm3TPm/evK3tyfpmzpw5xGIx4vE4zrkKj5k1axaxWIw1a9YkfX5eXh6xWIwVK1YkbZ8+fTpt2rRJ2ncA06ZNo3HjxsydOzdp+5dffsnmzZv5+uuvk7ZPnjyZeDzOtGnTkrZPmjSJZcuWMX168n1v4sSJzJs3j7y8vG3ai4qKiMfj9brvFRQURGLfy8/Pr9d9b+PGjUnbq9r3SphLNu+s7APMPgJ2SdJ0m3Pu9cRjYsCg6o6Rd+/e0z377GRycqh0KTliTbbUde5yLBZL+j9kNlJfeLm5ucTj8QpHddlK+0WpTOkLM5vinOtZ/v4qj8idcyemupiWLaFHj1SvVUQkO+k7eSIiEVenIDezc8xsMXAE8LaZvZ+askREpLrqOmvlVeDVFNUiIiK1oKEVEZGIU5CLiEScglxEJOIU5CIiEacgFxGJOAW5iEjEKchFRCJOQS4iEnEKchGRiFOQi4hEnIJcRCTiFOQiIhGnIBcRiTgFuYhIxCnIRUQiTkEuIhJxCnIRkYhTkIuIRJyCXEQk4hTkIiIRpyAXEYk4BbmISMQpyEVEIk5BLiIScQpyEZGIU5CLiEScglxEJOIU5CIiEacgFxGJOAW5iEjEKchFRCKuTkFuZvea2Swz+4+ZvWpm7VJVmIiIVE9dj8g/BA5yzh0CzAFurXtJIiJSE3UKcufcB865wsSP/wa61L0kERGpiVSOkV8BvJvC9YmISDU0ruoBZvYRsEuSptucc68nHnMbUAiMqWQ9/YB+AJ06dSIWi9Wm3pTJz88PXkOmUF948XicoqIi9UWC9otSmd4X5pyr2wrM+gJXAyc45zZU5zk9e/Z0kydPrtN26yoWi5Gbmxu0hkyhvvByc3OJx+NMnTo1dCkZQftFqUzpCzOb4pzrWf7+Ko/Iq1jpqcDNwHHVDXEREUmtuo6RDwfaAB+a2VQzeywFNYmISA3U6QY7FMAAAAKVSURBVIjcObdPqgoREZHa0Tc7RUQiTkEuIhJxCnIRkYir8/TDWm3UbCXw37RveFsdgVWBa8gU6otS6otS6otSmdIXezjndip/Z5AgzwRmNjnZfMxspL4opb4opb4olel9oaEVEZGIU5CLiERcNgf5yNAFZBD1RSn1RSn1RamM7ousHSMXEWkosvmIXESkQVCQA2Y20MycmXUMXUsoumyfPwmcmc02s7lmdkvoekIxs65mNs7MZphZnpn1D11TaGaWY2ZfmdlboWtJJuuD3My6AicDC0PXElhWX7bPzHKAR4DTgO7ARWbWPWxVwRQCA51z3YHDgeuyuC9K9Admhi5ie7I+yIEH8afizeoPC3TZPnoBc51z851zm4HngbMC1xSEc26Zc+7LxO11+ADbLWxV4ZhZF+DnwJOha9merA5yMzsLWOKcmxa6lgyTjZft2w1YVObnxWRxeJUws27AT4BJYSsJahj+YK84dCHbU6fT2EZBZZeqAwbjh1WyQqou2yfZwcxaA2OBAc65taHrCcHMegMrnHNTzCw3dD3b0+CD3Dl3YrL7zexgYE9gmpmBH0r40sx6Oee+S2OJabO9viiRuGxfb/xl+7JtqGkJ0LXMz10S92UlM2uCD/ExzrlXQtcT0FHAmWZ2OtAc2MHMnnHOXRq4rm1oHnmCmX0L9HTOZcKJcdIucdm+B/CX7VsZup50M7PG+A95T8AH+BfAxc65vKCFBWD+yOYfwGrn3IDQ9WSKxBH5IOdc79C1lJfVY+Syjay+bF/ig97rgffxH+69mI0hnnAU0Ac4PrEvTE0ckUqG0hG5iEjE6YhcRCTiFOQiIhGnIBcRiTgFuYhIxCnIRUQiTkEuIhJxCnIRkYhTkIuIRNz/AzT4EGRg4ufNAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "markdown", "metadata": { "id": "He9x901xl2Q8" }, "source": [ "### SELU\n", "\n", "This activation function was proposed in this [excellent paper](https://arxiv.org/pdf/1706.02515.pdf) by Günter Klambauer, Thomas Unterthiner and Andreas Mayr, published in June 2017. \n", "\n", "During training, a neural network composed exclusively of a stack of dense layers using the SELU activation function and LeCun initialization will self-normalize: \n", "\n", " - the output of each layer will tend to preserve the same mean and variance during training, which solves the vanishing/exploding gradients problem\n", " - As a result, this activation function outperforms the other activation functions very significantly for such neural nets, so you should really try it out. \n", " \n", "Unfortunately, the self-normalizing property of the SELU activation function is easily broken: you cannot use ℓ1 or ℓ2 regularization, regular dropout, max-norm, skip connections or other non-sequential topologies (so recurrent neural networks won't self-normalize). However, in practice it works quite well with sequential CNNs. \n", "\n", "If you break self-normalization, SELU will not necessarily outperform other activation functions.\n", "\n", "Source: https://www.google.co.in/books/edition/Hands_On_Machine_Learning_with_Scikit_Le/bRpYDgAAQBAJ" ] }, { "cell_type": "code", "metadata": { "id": "lMyoRjN4lzTA" }, "source": [ "from scipy.special import erfc\n", "\n", "# alpha and scale to self normalize with mean 0 and standard deviation 1\n", "# (see equation 14 in the paper):\n", "alpha_0_1 = -np.sqrt(2 / np.pi) / (erfc(1/np.sqrt(2)) * np.exp(1/2) - 1)\n", "scale_0_1 = (1 - erfc(1 / np.sqrt(2)) * np.sqrt(np.e)) * np.sqrt(2 * np.pi) * (2 * erfc(np.sqrt(2))*np.e**2 + np.pi*erfc(1/np.sqrt(2))**2*np.e - 2*(2+np.pi)*erfc(1/np.sqrt(2))*np.sqrt(np.e)+np.pi+2)**(-1/2)" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "I2Q8f3_9mSHr" }, "source": [ "def selu(z, scale=scale_0_1, alpha=alpha_0_1):\n", " return scale * elu(z, alpha)" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "Ip8eeDl8mpQr", "colab": { "base_uri": "https://localhost:8080/", "height": 282 }, "outputId": "caedd658-23fa-4424-b86d-324f3945a669" }, "source": [ "plt.plot(z, selu(z), \"b-\", linewidth=2)\n", "plt.plot([-5, 5], [0, 0], 'k-')\n", "plt.plot([-5, 5], [-1.758, -1.758], 'k--')\n", "plt.plot([0, 0], [-2.2, 3.2], 'k-')\n", "plt.grid(True)\n", "plt.title(\"SELU activation function\", fontsize=14)\n", "plt.axis([-5, 5, -2.2, 3.2]);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEJCAYAAACJwawLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de5xV8/7H8deni0iIbi5FHYQOQrkfmqMcclcRqiM6in6OItdKp+OSa8RxOfVzKhK6uXb0Q2mO4yAVETFJOhWVop2mezPf3x/fPWbas6eamT37uy/v5+OxHq3Za81an1ntec/a37XW92vOOUREJH1VC12AiIhUjoJcRCTNKchFRNKcglxEJM0pyEVE0pyCXEQkzSnIJe2Y2Wgzm5yE/eSYmTOz+knYV08zW2xmhWY2uKr3t4NauptZfsgapHwU5GnOzBqY2VNmtsjMNpnZCjObZmZnllgnNxpIsdNLJdZxZtYpzvabRpe1jrMs18yeqMKfrawg7QN0TfC+FpnZzTEvfwDsB/yUyH3F2ffewJPAQ8ABwMNVub+Yfcf7fx8H/CZZNUjl1QhdgFTaJKA20ANYADQE2gD1YtYbBfSPeW1DlVdXBZxza5K0n83A8iTs6iD87+Jk59yyJOxvu5xzG0jT90a20hl5GjOzusBpwO3OuWnOuf8652Y65x52zr0Us/p659zymKlKA9HMDjaz18xsuZmtM7NPzOy8mHV2MbMhZvbf6CeKhWZ2g5k1BaZHV1sZPXMcHf2eX5tWok0SK8ysesx2XzCz13emDjPLxYfpQ0WfVqKvl/pEYGYdzGxutNYlZjbAzKzE8kVmNtDMhpvZL2a21Mxu2c4x6g58Gv1yYXR/Tc1ssJl9EbtuySaPonXM7DIz+9bM1prZq7GfYMzsyhI1rzCzZ4tqja4yIbrfRfH2E32tl5ktMLPN0X+viVnuov8XE6LHeKGZJfRTk5RNQZ7e8qPTBWa2a+hi4qgDTAHOBFriPz28bGaHl1jnWeCPwE3AEfhPFhFgCdAxus5v8U0cfeLsYwKwV3QfAJhZHeBC4PmdrKMDsBS4K7qf/eL9MGbWKrq/l4GjgNuBO4DrY1a9EZgLHAc8ADxoZifH2ya+GePs6PwJ0X0vKWPdeJoCnYGLgT8AxwL3lqi5FzAc/4nsaOAcoOgPxPHRf6+J7rfo622Y2cXAE8Aw4EjgMeApMzs/ZtVBwGv4YzwOGGlmB5bjZ5GKcs5pSuMJH3Y/AxuBD/HtqyfGrJMLbKY4+Ium3iXWcUCnONtvGl3WOs6yXOCJctb7ETAwOn9odNtnl7FuTnR5/ZjXR+ObIYq+fhkYU+LrrsAaYNedqSP69SLg5u3tHxgLvBuzzmBgacx2XoxZ55uS+4pTS+vofprGbPeLmPW6A/kx62wE9irx2gBgQYmvlwL3b2ffpf7f4+znP8DIOP8H78ds574SX9cA1gNdQ/+OZMOkM/I055ybBOwPnI8/6zwF+MjMYtvDxwHHxExjq7I2M9vdzB40s3lmtjr6cb01UHSWdixQSHETSkU9D1xkZrWjX3cBJjnnNu5kHTvrCHyolfQ+cICZ7Vnitc9j1vkBf+2iKvzXbdtE9uu+zKwh/uLptEruo6yfu0XMa7/+3M65rcBKqu7nlhJ0sTMDRAPrneh0l5k9Aww2s4edv2AHsMY5t6ACm/8l+u9ecZbVxZ/5luVhfLPBzfiz0vXAc8AuFahje/4JbAUuNLNpQDvgrCTXUbIb0S1xlpX3pKkQsJjXasZZLxH7qqjYrlND1pLVdJAz0zz8H+lKt5s7534GVgGtSr4ePQM9BMjbzrf/DnjOOTfJOfc5/mP+wSWWz8G/B39fxvcX/RGqXsbyoho34duuu+Dbi5fjm312to6ifW13P8BXwKkxr/0O37SydgffW14rgUYlL6TiP0XtNOfcj8D3QNvtrLaFiv/c88pTj1QdnZGnMTOrhw+wkfiPtWvxTQa3AtOcc7+UWL22me0bs4nN0aAu0tTMYsNiIfAIcLuZ/YBvh68H3IkPmwnbKXE+cLGZvYYPjL9Q4o+Lc26+mY0HnjGzPsAnQGN8W/EY4L/4s7pzzewNYINzrqwHVZ7HNyE0w7dRF+5sHVGLgNPM7Hlgk3NuVZx9DAVmmn9g5wX8xcF+lL6tMxFygX2A/ubv988BSt3nvxPuBR41sxX4Ty61gbbOuaHR5YuAtmb2L/zPvTrONh7C39kyG3gb/+mmC/4isaSC0I30mio+AbWAIcBMYDW+yeAbfPDuU2K9XHwgxk6xF6viTefhz9j+jP9jkY8/o32JEhfnyqjvIGAqsC76PTcDk4HRMT/Dg/gzx03At8D1JZbfCSzDNzWMjr42mhIXO6OvGT6UHHB0Beo4CfgMf/HQRV/LIeZiKz685uLP4JfgLy5aieWLKH3RNJftXBQmzsXO6Ou98H/M1kWPdx9KX+zc7gXR6Gs98GfPRffFjyyx7Pzoe2YLsGg727gW/5zClui/18Qsj3fRtNSx0FQ1k0UPuIiIpCm1kYuIpDkFuYhImlOQi4ikOQW5iEiaC3L7Yf369V3Tpk1D7PpX69atY/fddw9aQ6rQsfDy8vIoKCigRYvYBxazUyq8LzZtgq++goIC2H9/2C9uLzhVLxWOBcDs2bNXOecaxL4eJMibNm3KrFmzQuz6V7m5ueTk5AStIVXoWHg5OTlEIpHg781UEfp9kZ8PJ5/sQ/yii2DSJKgWqA0h9LEoYmb/jfe6mlZEJOU4B3/6E3zxBRx2GDz7bLgQTwc6NCKSch55BMaNgzp14JVXYM89d/w92UxBLiIp5d134dZb/fxzz8ERR4StJx1UOsjNbFcz+9jMPjOzL83sr4koTESyz+LF0LkzFBbCHXfAxReHrig9JOJi5ybgDOdcvpnVBN43synOuY8SsG0RyRIbN0LHjrBqFZx1Ftx9d+iK0kelg9z5zlqKeqSrGZ3UgYuI7DTnoHdvmDULmjWDF16A6jvqXFd+lZDbD6MD387G90/9pHNuRpx1egI9ARo1akRubm4idl1h+fn5wWtIFToWXiQSoaCgQMciKpnvi9df359Ro5pTq1YB/ft/yuefl9VbcRgp/zuSyK4U8SPGTAeO3N56rVq1cqFNnz49dAkpQ8fCa9OmjWvZsmXoMlJGst4XH3zgXM2azoFzY8YkZZflliq/I8AsFydTE3rXinMuEg3ys3e0rojI8uXQqRNs2QI33ABdu4auKD0l4q6VBmZWNzq/G3Am8HVltysimW3LFrjkEvjhBzjtNHj44dAVpa9EtJHvBzwbbSevBox3zk1OwHZFJIP16wfvv+/7UBk/HmrGG1padkoi7lr5HDg2AbWISJYYMwb+9jcf3pMmwb6xo8lKuejJThFJqk8/hZ49/fwTT8BJJ4WtJxMoyEUkaX76CTp08A//9OgB11wTuqLMoCAXkaQoKIDLL4dFi+D44/3ZuFnoqjKDglxEkmLgQHjnHWjQwLeL77pr6Ioyh4JcRKrcpElw//3+sfvx46FJk9AVZRYFuYhUqXnzoHt3P//QQ5ACA+1kHAW5iFSZNWt8V7T5+XDZZdC3b+iKMpOCXESqRGEhXHklzJ8PRx0Fzzyji5tVRUEuIlViyBB47TWoW9cP15YCg9BnLAW5iCTclCkwaJA/A3/hBTj44NAVZbaE9EcuIlLk22/hiiv8YBF33QXt24euKPPpjFxEEmbdOn9xMxKBCy6AAQNCV5QdFOQikhDO+Ufu586F5s3hueegmhImKXSYRSQhhg2DF1+EOnX8xc299gpdUfZQkItIpeXmwi23+PnRo6FFi5DVZB8FuYhUypIlcOmlvlOs226Djh1DV5R9FOQiUmEbN/rgXrkSzjwT7r03dEXZSUEuIhX25z/DzJlw0EG+fbx69dAVZScFuYhUyIgR/rH7XXf1Fzfr1QtdUfZSkItIuX30EVx/vZ8fMQKO1ai9QSnIRaRcVqyATp1gyxYf5t26ha5IFOQistO2bPF3qHz/PfzudzB0aOiKBBTkIlIOt9wC770H++0HEybALruErkhAQS4iO2nsWHjsMahZ0w/dtu++oSuSIgpyEdmhOXN8PyoAjz8OJ58cth7ZloJcRLbr55+hQwfYsAGuugp69QpdkcRSkItImQoKfN/i330HrVvDU09puLZUpCAXkTINGgRvvQX16/t28V13DV2RxKMgF5G4/v3v+gwZ4vsUHzcODjwwdEVSFgW5iJTy9ddw//2HA/DAA3DGGYELku1SkIvINn75BS66CNavr0HnztCvX+iKZEcU5CLyq8JCuPJKyMuDZs3y+cc/dHEzHVQ6yM2siZlNN7N5ZvalmfVJRGEiknz33w+vvuqHabv77i/ZfffQFcnOqJGAbWwF+jnnPjGzPYDZZvaOc25eArYtIkny1lswcKCfHzsWdt99Q9iCZKdV+ozcObfMOfdJdH4t8BVwQGW3KyLJs3AhXH45OAeDB8O554auSMojEWfkvzKzpsCxwIw4y3oCPQEaNWpEbm5uInddbvn5+cFrSBU6Fl4kEqGgoCDrjsXGjdW4/vrjWL26DqecsorTTvuC3Fy9L0pK+WPhnEvIBNQBZgMddrRuq1atXGjTp08PXULK0LHw2rRp41q2bBm6jKQqLHSuSxfnwLlDD3UuEilepvdFsVQ5FsAsFydTE3LXipnVBCYBY51zLydimyJS9R5/vKg93A/XttdeoSuSikjEXSsG/AP4yjn3SOVLEpFk+Ne/iu8RHzUKfvvbsPVIxSXijPxUoBtwhpnNiU7nJGC7IlJFli71I/0UFPjBIi65JHRFUhmVvtjpnHsf0CMDImli0yY/5uaPP0LbtjBkSOiKpLL0ZKdIlrnhBpgxw3eC9dJLUCOh965JCApykSzyzDMwYgTUqgUvv+y7p5X0pyAXyRIffwz/8z9+fvhwaNUqbD2SOApykSzw44/QsSNs3gy9e/uOsSRzKMhFMtzWrf4OlaVL4ZRT4NFHQ1ckiaYgF8lwt97q7xnfd1+YMAF22SV0RZJoCnKRDPbii/4MvEYNmDgR9t8/dEVSFRTkIhnq88+hRw8//9hjcOqpYeuRqqMgF8lAq1fDxRfDhg3+wuZ114WuSKqSglwkwxQUQJcuvo/x446Dp5/WcG2ZTkEukmEGD4YpU6BePf/Qz267ha5IqpqCXCSDvPYa3HMPVKvmH78/6KDQFUkyKMhFMkReHnTr5ufvuw/atQtbjySPglwkA6xd6y9url3ru6S95ZbQFUkyKchF0pxzcNVV8NVX0KIFjBypi5vZRkEukuYeeAAmTYI99/TDtdWpE7oiSTYFuUgae/ttGDDAzz//PDRvHrYeCUNBLpKmvvsOLr8cCgth0CA4//zQFUkoCnKRNLR+PXToAD//DOecA3/5S+iKJCQFuUiacQ6uvRbmzIGDD/ZNKtX0m5zV9N8vkmaefBLGjIHateHVV2HvvUNXJKEpyEXSyL//DTfe6OdHjoQjjwxbj6QGBblImvj+e/+wz9at0K8fdO4cuiJJFQpykTSwaRN06gQrVsDvfw/33x+6IkklCnKRNNC3L3z0ETRpAuPG+RF/RIooyEVS3MiR8Pe/Q61avlvaBg1CVySpRkEuksJmzYLevf38009D69Zh65HUpCAXSVErV/qHfjZt8veNX3VV6IokVSnIRVLQ1q1w2WWwZAmcdJIfPFmkLApykRR0++3w7rvQqJHv2XCXXUJXJKlMQS6SYsaNg6FD/Z0pEybA/vuHrkhSXUKC3MxGmtmPZvZFIrYnkq3mzoWrr/bzjzwCp50Wth5JD4k6Ix8NnJ2gbYlkpUjED9e2fr0fe/P660NXJOkiIUHunHsP+DkR2xLJRoWF0KULfPstHHMMDB+u4dpk5yXt+TAz6wn0BGjUqBG5ubnJ2nVc+fn5wWtIFToWXiQSoaCgIMixGDWqKW++2ZQ999zCbbfNZsaMjUmvIZbeF8VS/VgkLcidcyOAEQCtW7d2OTk5ydp1XLm5uYSuIVXoWHh169YlEokk/Vi88QY895zvU3zixJqceeZJSd1/WfS+KJbqx0J3rYgENH8+dO3q5++9F848M2w9kp4U5CKB5Of7i5u//AIdO8Jtt4WuSNJVom4/fBH4EDjMzJaaWY9EbFckUznnH7mfNw+OOAJGjdLFTam4hLSRO+cuT8R2RLLFww/DxImw557wyiuwxx6hK5J0pqYVkSSbOtU/gg/+Iudhh4WtR9KfglwkiRYt8p1hFRbCwIFw4YWhK5JMoCAXSZING/xFzZ9+gvbtYfDg0BVJplCQiySBc3DddfDJJ/Cb38DYsVC9euiqJFMoyEWS4Omn4dlnoXZtf3Fz771DVySZREEuUsX+8x/o08fPP/MMHH102Hok8yjIRarQsmXQqZMf8efGG+Fy3agrVUBBLlJFNm/2Ib58OeTkwIMPhq5IMpWCXKSK3HgjfPABNG7sR/2pkbQu6iTbKMhFqsDo0fDUU36szUmToGHD0BVJJlOQiyTY7Nlw7bV+/skn4YQTwtYjmU9BLpJAq1ZBhw6waRP07Al/+lPoiiQbKMhFEmTrVv/4/eLFcOKJ8PjjoSuSbKEgF0mQAQNg2jTfHj5xItSqFboiyRYKcpEEmDDB315Yvbqfb9w4dEWSTRTkIpX0xRd+kAiAoUPh9NPD1iPZR0EuUgmRiL+4uW4ddOkCN9wQuiLJRgpykQoqLIRu3eCbb6BlSxgxQsO1SRgKcpEKuucemDzZ92T48su+Z0OREBTkIhXwz3/6gSHM4MUXfR/jIqGo9weRcvrmG98e7hzcey+cdVboiiTb6YxcpBzy8/3FzTVr4OKL4Y47QlckoiAX2WnOQY8e/nbDww/3HWPp4qakAgW5yE565BEYPx722MMP17bnnqErEvEU5CI74d134dZb/fyzz/ozcpFUoSAX2YHFi6FzZ3/feP/+vm1cJJUoyEW2Y+NG6NjRd0971llw112hKxIpTUEuUgbnoHdvmDULmjWDF17wnWKJpBoFuUgZhg+HUaNgt938xc199gldkUh8CnKROD78sLgDrP/9X9+XikiqUpCLxFi+3LeLb9kCffr4pzhFUllCgtzMzjazPDNbYGa3J2KbIiE4B5dcAsuW+X7FH3oodEUiO1bpvlbMrDrwJHAmsBSYaWavO+fmVXbbIsm2bNlufP45HHCAf/inZs3QFYnsWCI6zToBWOCcWwhgZi8BFwJlBnleXh45OTkJ2HXFRSIR6tatG7SGVKFj4X3yyRzWrgXIoWFDf+94NtP7oliqH4tEBPkBwJISXy8FToxdycx6Aj0BatasSSQSScCuK66goCB4DalCx8Jbv94Bxj77bKawcD3Zfkj0viiW6sciad3YOudGACMAWrdu7WbNmpWsXceVm5sb/FNBqtCxgI8/hhNPzMHM8fnn/+KAA0JXFJ7eF8VS5VhYGb20JeJi5/dAkxJfN46+JpIWnIPbo5foGzTYrBCXtJOIIJ8JHGpmzcxsF+Ay4PUEbFckKd5+G6ZPhxo1oGHDjaHLESm3SjetOOe2mtn1wFtAdWCkc+7LSlcmkgSFhcVn4wceCNWru7AFiVRAQtrInXNvAm8mYlsiyfTSSzBnDjRu7G85/OWX0BWJlJ+e7JSstXkz3Hmnnx88GKrpt0HSlN66krVGjICFC/0gEVdeGboakYpTkEtWWrOmuG/x++7zFzpF0pWCXLLSfffBypVw6qlw4YWhqxGpHAW5ZJ3vvoNHH/Xzjz4KZTxjIZI2FOSSdW6/3V/o7NoVjj8+dDUilacgl6zywQe+V8Ndd4UhQ0JXI5IYCnLJGoWFcOONfv7mm6FJk+2vL5IuFOSSNZ591neOte++cNttoasRSRwFuWSFVavgllv8/IMPQp06YesRSSQFuWSFm2+Gn36Ctm39RU6RTKIgl4yXm+ubVWrVgqef1u2GknkU5JLRNm2CXr38/IABcOihYesRqQoKcsloQ4bA/Pm+P5Vbbw1djUjVUJBLxvr4Y7j3Xt+UMny4b1oRyUQKcslI69fDH/8IBQX+3vHTTw9dkUjVUZBLRrr9dsjLgxYt/Fm5SCZTkEvGmToV/vY33zXtmDH+cXyRTKYgl4yyciV07+7n//IXOO64oOWIJIWCXDJGQYF/2Of77+GUU4oHVRbJdApyyRj33gtvvw3168O4cRr1R7KHglwywtSpfgBlMxg7Fho3Dl2RSPIoyCXtLV0KV1wBzsGdd8If/hC6IpHkUpBLWlu3Di64wF/kbNcOBg0KXZFI8inIJW0VFvqLm59+CoccAi+9BNWrh65KJPkU5JK2+veHV1+FunVh8mSoVy90RSJhKMglLY0cCQ884M/AJ06Eww4LXZFIOApySTuvvgrXXOPnn3zSDxYhks0U5JJWpk2Dzp19+/igQcV9jYtkMwW5pI0ZM+DCC2HzZvjzn/194yKiIJc0MXs2tG/vbzfs1g2GDdOQbSJFKhXkZnaJmX1pZoVm1jpRRYmU9OGHcMYZsHo1XHQR/OMfUE2nICK/quyvwxdAB+C9BNQiUsp77/knNX/5BS65BMaPh5o1Q1clkloq1a2Qc+4rANNnXKkCU6ZAx46wYYN/8GfUKHWEJRJP0n4tzKwn0BOgUaNG5ObmJmvXceXn5wevIVWk4rGYPHk/Hn20OYWFRvv2y+jePY/336/afUYiEQoKClLuWISSiu+LUFL+WDjntjsBU/FNKLHThSXWyQVa72hbRVOrVq1caNOnTw9dQspIpWNRWOjcwIHO+S6wnBswwL+WDG3atHEtW7ZMzs7SQCq9L0JLlWMBzHJxMnWHZ+TOuXZV9DdEZBvr1/sHfV54wT+x+dRT0LNn6KpEUp9aHCUlfPcddOgAc+bA7rv7gSHOPTd0VSLpobK3H15sZkuBk4F/mtlbiSlLssk770Dr1j7EDzkEPvpIIS5SHpUKcufcK865xs65Ws65Rs65sxJVmGS+LVt8D4ZnnQU//wznnAMzZ8KRR4auTCS9qGlFgliwALp0gY8/9g/3DBrkJz3oI1J+CnJJKufgmWfgppsgPx+aNPFjbJ52WujKRNKXglyS5ttv/V0p06f7ry+5BIYPh733DluXSLrTB1mpcps3w0MPwVFH+RBv0MAPyzZunEJcJBF0Ri5V6v/+D/r0gfnz/dddu8Kjj0L9+mHrEskkOiOXKjFvnh/dvn17H+LNm/tQHzNGIS6SaApySajFi+Hqq30zyhtvQJ06vlll7lx/m6GIJJ6aViQhFi6EoUP9HSmbN/teCnv1gjvvhP32C12dSGZTkEulfPaZH81+3Dg/jibA5ZfDXXf5pzRFpOopyKXcCgv9IMjDhsGbb/rXatTwQ7Ddeiu0aBG2PpFsoyCXnfbjj35whxEjfFMKQO3a/t7wm26CAw8MW59ItlKQy3Zt2QLvvusD/OWX/dfgQ/uaa+Daa3UXikhoCnIppbAQ/vMfePFFmDABVq3yr1er5m8p7NXL34FSvXrYOkXEU5ALAJs2+YGOJ0/2Z95LlxYvO/xwuOIK6N7d940iIqlFQZ7Fli2Dt96CUaN+yyef+E6sihx0EFx2mb8D5eijQeNri6QuBXkWWbUKcnN9fyfvvgtff120pAHgA/u88+D88+HEExXeIulCQZ6htm6FL76AGTOKp3nztl1n993h9NOhefP53HRTc911IpKmFOQZYPNmf3Y9d65/QGfGDJg1yw9mXFKtWnDqqXDGGfD738Pxx0PNmpCb+wMHHtg8TPEiUmkK8jSyaZMfpHj+fH+2PXeun/Ly/Bl4rN/8Bk46yTeTnHgiHHOMD3MRySwK8hTinB+7cskS3/nUwoXwzTfF0+LFxY/Bl2TmH4c/6ijfzn388XDCCb7fbxHJfAryJNm0yT8ZuWKFn5Yv97f4LV5cHNxLlpRuDimpWjVo1gwOPdQ/Bn/UUX5q0cK3d4tIdlKQl5NzsG4drF7tz55Xr44/v2pVcWj/+CNEIju3/T328E9NNmkCTZv60C6amjVT04iIlJaRQb51K2zc6KcNG7b9t2h+5sx6rFjhz4DXrvX3UJf8N95r+fmwZk3xY+rlUb06NGwIjRoV/9ukiZ+KgvvAA2GvvRJ/PEQkswUJ8mXLYNAgH4iJmDZv3jas4134K+2oCte/225+rMl99vH/xpvfZx8f1kXTPvv4phERkUQLEuQ//JDH3XfnxLx6KdAbWA+cE+e7ukenVUCnOMuvAzoDS4BuVKvGNlPDhv1o2PB8Cgvz+O67XhQUbKFWrZpUq+bPlk8/fSBHHtmONWvm8PrrfalenW2m224bQps2p/Dllx/w17/232bPa9bAX/86jGOOOYapU6dyzz33lKpu+PDhHHbYYbzxxhsMHTq01PIxY8bQpEkTxo0bx9NPP11q+cSJE6lfvz6jR49m9OjRpZa/+eab1K5dm6eeeorx48eXWp6bmwvAww8/zOTJk7dZtmHDBmbMmAHA3XffzbRp07ZZXq9ePSZNmgTAHXfcwYcffrjN8saNG/P8888D0LdvX+bMmbPN8ubNmzNixAgAevbsyfyiATyjjjnmGIYNGwZA165dWVqyfwDg5JNP5r777gOgY8eO/PTTT9ssb9u2LXfeeScA7du3Z8OGDdssP++887j55psByMnJIdall15K7969KSwsZMGCBaXW6d69O927d2fVqlV06lT6vXfdddfRuXNnlixZQrdu3Uot79evH+effz55eXn06tWr1PKBAwfSrl075syZQ9++fUstHzJkCKeccgoffPAB/fv3L7V82LCqee9FIhHq1q1bpe+93XbbjSlTpgDZ/d5bv34955xTOvd29N4rEiTId9nFjxpTrZq/48IMWrWCtm39XRmPPeZfK7n87LP9U4fr1sHAgaWXX321f6R85Uro0aP0Pvv1808s5uX5Tp8ikXXUrVv31+U9ekC7djBnDnz8cenvP+AA3ySyYEEVHhgRkQow51zSd9q6dWs3a9aspO+3pNzc3Lh/IbORjoWXk5NDJBIpdVaXrfS+KJYqx8LMZjvnWse+rlZbEZE0pyAXEUlzCnIRkTSnIBcRSXMKchGRNFepIDezh8zsazP73MxeMbO6O/4uERFJpMqekb8DHOmcOxqYD9xR+ZWHj/cAAANJSURBVJJERKQ8KhXkzrm3nXNFD8R/BDSufEkiIlIeiXyy82pgXFkLzawn0BOgUaNGvz62G0p+fn7wGlKFjoUXiUQoKCjQsYjS+6JYqh+LHQa5mU0F9o2zaIBz7rXoOgOArcDYsrbjnBsBjAD/ZGfop6RS5UmtVKBj4dWtW5dIJKJjEaX3RbFUPxY7DHLnXLvtLTez7sB5QFsX4nl/EZEsV6mmFTM7G7gVaOOc287YNiIiUlUqe9fKE8AewDtmNsfM/p6AmkREpBwqdUbunDskUYWIiEjF6MlOEZE0pyAXEUlzQQaWMLOVwH+TvuNt1cePGyc6FiXpWBTTsSiWKsfiIOdcg9gXgwR5KjCzWfFG2shGOhbFdCyK6VgUS/VjoaYVEZE0pyAXEUlz2RzkI0IXkEJ0LIrpWBTTsSiW0scia9vIRUQyRTafkYuIZAQFuYhImlOQA2bWz8ycmdUPXUsoGrbPdwJnZnlmtsDMbg9dTyhm1sTMppvZPDP70sz6hK4pNDOrbmafmtnk0LXEk/VBbmZNgD8Ai0PXElhWD9tnZtWBJ4H2QAvgcjNrEbaqYLYC/ZxzLYCTgP/J4mNRpA/wVegiypL1QQ48iu+KN6uv+mrYPk4AFjjnFjrnNgMvARcGrikI59wy59wn0fm1+AA7IGxV4ZhZY+Bc4JnQtZQlq4PczC4EvnfOfRa6lhRzNTAldBFJdgCwpMTXS8ni8CpiZk2BY4EZYSsJahj+ZK8wdCFlSeSYnSlpe0PVAf3xzSpZIVHD9kl2MLM6wCSgr3Pul9D1hGBm5wE/Oudmm1lO6HrKkvFBXtZQdWZ2FNAM+MzMwDclfGJmJzjnliexxKTRsH3b9T3QpMTXjaOvZSUzq4kP8bHOuZdD1xPQqcAFZnYOsCuwp5k975zrGriubeiBoCgzWwS0ds6lQg9nSRcdtu8R/LB9K0PXk2xmVgN/kbctPsBnAlc4574MWlgA5s9sngV+ds71DV1Pqoiekd/snDsvdC2xsrqNXLaR1cP2RS/0Xg+8hb+4Nz4bQzzqVKAbcEb0vTAnekYqKUpn5CIiaU5n5CIiaU5BLiKS5hTkIiJpTkEuIpLmFOQiImlOQS4ikuYU5CIiae7/ASyl5YB1GwCIAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "markdown", "metadata": { "id": "jUcP98XFnDee" }, "source": [ "By default, the SELU hyperparameters (`scale` and `alpha`) are tuned in such a way that the mean output of each neuron remains close to 0, and the standard deviation remains close to 1 (assuming the inputs are standardized with mean 0 and standard deviation 1 too). Using this activation function, even a 1,000 layer deep neural network preserves roughly mean 0 and standard deviation 1 across all layers, avoiding the exploding/vanishing gradients problem:" ] }, { "cell_type": "code", "metadata": { "id": "fUuEeE-kmrUT", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "7f51305a-d532-4741-ec75-1eaafaf6cda1" }, "source": [ "np.random.seed(42)\n", "Z = np.random.normal(size=(500, 100)) # standardized inputs\n", "for layer in range(1000):\n", " W = np.random.normal(size=(100, 100), scale=np.sqrt(1 / 100)) # LeCun initialization\n", " Z = selu(np.dot(Z, W))\n", " means = np.mean(Z, axis=0).mean()\n", " stds = np.std(Z, axis=0).mean()\n", " if layer % 100 == 0:\n", " print(\"Layer {}: mean {:.2f}, std deviation {:.2f}\".format(layer, means, stds))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Layer 0: mean -0.00, std deviation 1.00\n", "Layer 100: mean 0.02, std deviation 0.96\n", "Layer 200: mean 0.01, std deviation 0.90\n", "Layer 300: mean -0.02, std deviation 0.92\n", "Layer 400: mean 0.05, std deviation 0.89\n", "Layer 500: mean 0.01, std deviation 0.93\n", "Layer 600: mean 0.02, std deviation 0.92\n", "Layer 700: mean -0.02, std deviation 0.90\n", "Layer 800: mean 0.05, std deviation 0.83\n", "Layer 900: mean 0.02, std deviation 1.00\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "F7ZZYv4onQY7" }, "source": [ "### Training with SELU" ] }, { "cell_type": "code", "metadata": { "id": "iDB_4UrGnGri" }, "source": [ "model = tf.keras.models.Sequential([\n", " tf.keras.layers.Flatten(input_shape=[28, 28]),\n", " tf.keras.layers.Dense(300, \n", " activation='selu', \n", " kernel_initializer='lecun_normal'),\n", " tf.keras.layers.Dense(100, \n", " activation='selu', \n", " kernel_initializer='lecun_normal'),\n", " tf.keras.layers.Dense(10, activation=\"softmax\")\n", "])" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "BrKrU2dqndHu" }, "source": [ "model.compile(loss=\"sparse_categorical_crossentropy\", \n", " optimizer=tf.keras.optimizers.SGD(1e-3),\n", " metrics=[\"accuracy\"])" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "pXBgq78Ano16" }, "source": [ "Now let's train it. Do not forget to scale the inputs to mean 0 and standard deviation 1:" ] }, { "cell_type": "code", "metadata": { "id": "WG2_L_MQnllo" }, "source": [ "pixel_means = X_train.mean(axis=0, keepdims=True)\n", "pixel_stds = X_train.std(axis=0, keepdims=True)\n", "X_train_scaled = (X_train - pixel_means) / pixel_stds\n", "X_test_scaled = (X_test - pixel_means) / pixel_stds" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "xunchzdqn11r", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "42019c8d-f202-4de4-c019-bacea07bd5a0" }, "source": [ "history = model.fit(X_train_scaled, y_train, epochs=20,\n", " validation_split=0.1)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Epoch 1/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.6931 - accuracy: 0.7608 - val_loss: 0.5174 - val_accuracy: 0.8122\n", "Epoch 2/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.5005 - accuracy: 0.8251 - val_loss: 0.4639 - val_accuracy: 0.8295\n", "Epoch 3/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.4590 - accuracy: 0.8402 - val_loss: 0.4405 - val_accuracy: 0.8380\n", "Epoch 4/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.4355 - accuracy: 0.8478 - val_loss: 0.4244 - val_accuracy: 0.8455\n", "Epoch 5/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4190 - accuracy: 0.8536 - val_loss: 0.4144 - val_accuracy: 0.8515\n", "Epoch 6/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4063 - accuracy: 0.8584 - val_loss: 0.4076 - val_accuracy: 0.8530\n", "Epoch 7/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.3958 - accuracy: 0.8625 - val_loss: 0.3982 - val_accuracy: 0.8563\n", "Epoch 8/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.3872 - accuracy: 0.8653 - val_loss: 0.3931 - val_accuracy: 0.8563\n", "Epoch 9/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.3793 - accuracy: 0.8682 - val_loss: 0.3906 - val_accuracy: 0.8598\n", "Epoch 10/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.3728 - accuracy: 0.8699 - val_loss: 0.3867 - val_accuracy: 0.8605\n", "Epoch 11/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.3666 - accuracy: 0.8721 - val_loss: 0.3818 - val_accuracy: 0.8628\n", "Epoch 12/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.3607 - accuracy: 0.8745 - val_loss: 0.3787 - val_accuracy: 0.8628\n", "Epoch 13/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.3555 - accuracy: 0.8761 - val_loss: 0.3770 - val_accuracy: 0.8653\n", "Epoch 14/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.3505 - accuracy: 0.8778 - val_loss: 0.3756 - val_accuracy: 0.8652\n", "Epoch 15/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.3461 - accuracy: 0.8789 - val_loss: 0.3701 - val_accuracy: 0.8648\n", "Epoch 16/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.3415 - accuracy: 0.8804 - val_loss: 0.3663 - val_accuracy: 0.8663\n", "Epoch 17/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.3374 - accuracy: 0.8818 - val_loss: 0.3666 - val_accuracy: 0.8667\n", "Epoch 18/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.3335 - accuracy: 0.8829 - val_loss: 0.3649 - val_accuracy: 0.8682\n", "Epoch 19/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.3295 - accuracy: 0.8851 - val_loss: 0.3631 - val_accuracy: 0.8667\n", "Epoch 20/20\n", "1688/1688 [==============================] - 3s 2ms/step - loss: 0.3259 - accuracy: 0.8860 - val_loss: 0.3613 - val_accuracy: 0.8685\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "-a17BL6Wn5uL", "colab": { "base_uri": "https://localhost:8080/", "height": 324 }, "outputId": "f2abfae4-e207-4d0e-85e3-de599a674334" }, "source": [ "pd.DataFrame(history.history).plot(figsize=(8, 5))\n", "plt.grid(True)\n", "plt.gca().set_ylim(0.3, 1);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeMAAAEzCAYAAAACSWsXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXxU5d3//9c1+0z2PUACBFmFgAqKGxBECu1txbqhtlZo1dpWbavVorXVn9W2t91tua3W24VWRYrl1p8LtiAp4g4URHZkDQSyL5Nk9uv7x0yGSUjIBCaZZPJ59jGPs13nzHWFPnzPdc65zlFaa4QQQggRP4Z4V0AIIYQY6CSMhRBCiDiTMBZCCCHiTMJYCCGEiDMJYyGEECLOJIyFEEKIOOsyjJVSzyilKpRSn3WyXSmlHldK7VFKfaqUOif21RRCCCESVzQ94+eAuSfZ/kVgVOhzK/DE6VdLCCGEGDi6DGOt9Vqg5iRF5gFLdNCHQLpSalCsKiiEEEIkulhcMx4CHIpYLgutE0IIIUQUTL35ZUqpWwmeysZut08uLCyM2bEDgQAGQ+Ldj5aI7UrENkFitkva1H8kYrsSrU27du2q0lrndLQtFmF8GIhM1YLQuhNorZ8CngKYMmWKXr9+fQy+Pqi0tJSSkpKYHa+vSMR2JWKbIDHbJW3qPxKxXYnWJqXUgc62xeInx2vA10N3VZ8P1Guty2NwXCGEEGJA6LJnrJR6CSgBspVSZcCDgBlAa/1n4E3gS8AeoBlY2FOVFUIIIRJRl2Gstb6+i+0a+G7MaiSEEEIMMIlzZVwIIYTopySMhRBCiDiTMBZCCCHiTMJYCCGEiDMJYyGEECLOJIyFEEKIOJMwFkIIIeJMwlgIIYSIMwljIYQQIs4kjIUQQog4kzAWQggh4kzCWAghhIgzCWMhhBAiziSMhRBCiDiTMBZCCCHiTMJYCCGEiDMJYyGEECLOJIyFEEKIOJMwFkIIIeJMwlgIIYSIMwljIYQQIs4kjIUQQog4kzAWQggh4kzCWAghhIgzCWMhhBAiziSMhRBCiDiTMBZCCCHiTMJYCCGEiDMJYyGEECLOJIyFEEKIOIsqjJVSc5VSO5VSe5RSizrYPkwptVop9alSqlQpVRD7qgohhBCJqcswVkoZgcXAF4EzgeuVUme2K/ZrYInWeiLwMPCLWFdUCCGESFTR9IzPA/ZorfdqrT3AUmBeuzJnAu+E5td0sF0IIYQQnYgmjIcAhyKWy0LrIm0GrgzNfwVIUUplnX71hBBCiMSntNYnL6DU1cBcrfXNoeUbgala69sjygwG/gQUAWuBq4AJWuu6dse6FbgVIC8vb/LSpUtj1hCn00lycnLMjtdXJGK7ErFNkJjtkjb1H4nYrkRr08yZMzdorad0tM0Uxf6HgcKI5YLQujCt9RFCPWOlVDJwVfsgDpV7CngKYMqUKbqkpCSa+keltLSUWB6vr0jEdiVimyAx2yVt6j8SsV2J2KbORHOa+hNglFKqSCllAa4DXossoJTKVkq1Hus+4JnYVlMIIYRIXF2GsdbaB9wOvA1sB5ZprbcqpR5WSl0eKlYC7FRK7QLygEd7qL5CCCFEwonmNDVa6zeBN9ut+2nE/HJgeWyrJoQQQnQi4AdvM3hbglO/FwK+iI+/7bLf1257B2XaLxvNcOEdvdKcqMJYCCGEaCMQgIA3FIKhaZfznuC8t+V4iEYGauvUE5yfVHEY9tjalQ2V83t6vo22NAljIYQY0LQOBZkn+PG5we8Gn6ft1O8Bn4fsyg2wpSq0T+v21jLe0P6Rx+povu0x8XtCvUrP8Z5n67z2x7a9JhuY7WB2hKZ2DAEvWDIhKSe87vh2B1hC8yY7mKxgMILBFPHparl1nbmTMsbYtvFkze+1bxJCiL4i4AefKxhE3pbj8+2nrUEW8EX07nzBaXhdN7e1BmxHgdh+ysmHnkaaALD1JAUMJjBawWQBoyU4bzQHQ8wYWmeygjk9NG85vt5gCpaNet4MRlMn8+a2gdoash0E338G0N3UEsZCiN6jdSjoXG17eD5Xu16fu20w+VwR8+7jZfye48fqLFDD3xdcnu5tgdIY9+qU4XjQtIZSa2/LaDpxm9EK1pTjQWiyRgRlxDQcipHbIvcxh9et/8+nTDn/ojbrwvNGCxjkvUB9mYSxEKKtiBtjbC1H4djW4DU8jzPiel5Tu2kzeJo62N5+fTPd6e11ShmCpzUjg8xsC86bbMGPI6vtsjk4PXT4KMPOGBPaZo8oEzE124+HXmvPLjJQ2wduHwg65x4n5IyJdzXEKZIwFqI/0Pp4L9Dr6rj319FNLt6WUBi2u0Gmdd7T/gaa5jY3xpwP8FEXdTOYwJwUun7Xeh0vCWypkJIPlqTQ+qTjpyVNtrY9vRN6hra2PcL2wWs89f907SstZdiMklPeX4ieIGEsRCwE/Ji8TqgvA3cjuJ3gbgjOe5xt13ma2p0+PfFUaofTU2W0hEKwgxtfHFknXr+zJIWv423//CDjJk5uF7ZJbUPXZInd31H0KVpr8PnQHg8Bjwft8aK9HrTnxE8gPO9tu83bdjs+H5hMKIsFg8WCslhQ5tA0/DGjLBYs27fTnJzcbltreXN4f0wm0BrtdhNwuYLf63IRcLvRoU/A5UZ73KH1HrTbdXy9203A7UKH1gfcbrTLjTKbGfKbX/fK31rCWAxcPncwGMOB6QxOI+fbhGlkoLZb523iYoD3uvhOk61d7zDi9KglGRzZJ542NdtOPI3afl+TtePANdlPqxd5rKmUcWeWnPL+p0sHAuiWFgLNzQRap83NBJpbCDQ3EWhuDm1vAR0ApUJ7quC8UhBapULL9j17qDlU1royuD20X2uZ4/uDMhhC88GpMqjOl0Pzx/dpvxw8nd0+qLoMtNayXm+nZbKcTvbY7aEhRwE0GgI6eFalg2W0DoZt5Hz7Zb8/uBwLSgWD1GRChwKeLt6NkAEciPLYXR2ry0NYrSirFUNoqmxWjOnpp3XM7pAwFv2P3weuOmiugZaa0LQ2FJyNoYCNDNOmjsM24I3u+wzm4M021hSwpoI1OTjUIqMoYn0Kew4dY+SZZ7VZhzUlGLKt80Zzz/5tuqD9fgItLrSrJRRuLeiWUNC1LruC00BLC0l79lC5bVtsK+Hzh76vORyuujkiaCNCV7e0xPa7gVTgWMyPGmMGwwk9RcMJvUcLBocjvL2+qorM/EHBHweo4HVspcCgUOr4PCq0bIj4MdFa3qBCP0hC20zGk/ZcDRYLymw+sefaSe9VhX8sRdfr3vjRR5w1fnywpxr548N74g8YZTShbK1hakNZLRhsNpTFisEWClirDYPVgopcb7MF2xDn6/4SxiK+vC1tQ7W5OjRfGxG07ba56k9+TIPpeABakoPhaUmG5Lx265LAknJ8e+s0PB/aZrJG1ZSy0lJGTi5psy7g8RBoaMB/tI5Aw0H8jY0EmprRfl+wt+L3gz+ADkRMff62yx1MCfjR/kDwOKH1usUVDjnd3ELA1W65pQXtdnfrnycZqOrWHtFRdjsGhwND69ThwOCwY8zOCq13tFnfOh/cL6ntersdZXeEQoVQrw5AH+8ttfb6tGbdunVcdNFFbddHzAd7iKGK6sDxnqVuv3z8mG2WW3uWrb1Q3bocCB1XR4Tb8bBq8zF2f3zrntJSzulHw4CUUmA2o8xmDElJHZbx1tSQdOGFvVyz+JAwFrHl80BTJTRVQFMVOCtCy5Xh+SnH9sNGTzBgfSfp+ViSwZ4JjozgNGNYaDnz+LR13p5xvNcaZXhGQ/v9BOrq8Dc04G9oJNDYgL++AX9jA4GGRvwNDcF1DY2k79/P/j8/GQzchgb8DQ3dDr8uGQxgNAZ/xYemymgEozHYC7DbgwFnt2PMysRsDwWevXVbawDaUTZ7eL410AwO+/HjOBysfe89ZsyYEds2tJ62jROdnIwpIyNu3y9ERySMxclpHTyt21QJztaQbZ0PLUfOd9ZrNdkhOQeScnHZskkuGNUmTLUtnQAOAn4rfp+JgNeAv9kdDLpGZ3B6NDRtrCTQ8Dl+pzMYek4nAafztK8ZddR27enikXtGI8aUFAypqRgAQ8EQTHl5GFNTMKSkYkxNxZCagjElFWNaKoaUFAyOJJTJGDwVaTSCwYgyhsLV2MX6iNN8vaK1LkKIHiVhPJB5XdBYHvw0HAlNy6HxCLr+CIHqIwTqqoN3IAYUAb8KnnHzK3RAoY3JaEsa2pSCNuYSMBWhDQ60sqENVjRWNGa0NqHdGt0YvL5TVXYYp7WRQGN5sIfZGF2YKpstGHwpKRhTUjCmpmIpGIIhOQVDUlIwuGJM2ezBYE0NBmsweNPCYWtIcoQDsrS0lIn96DShEKLvkDBORFoHb2gKB2zbqa4/gr+yHF9NPd5mI75mI96W0NRlxuey4G0C7dVAV3cTukKfyhM3GY0d3oBi8PtQgwZjLijAFhGuhpSUYMglt5umpGAMDW8QQohEJGHcHwUC4DwKNfugZi/U7oPaA9BwBF1/GF9lBb5GXzhgfc2G4LzbhrfFhM+p0X4rkHv8mEYDppwczPmDsObnk5yXhykvD0NKcsRYQHMHd0t2/yYU6UEKIURbEsZ9ld8LdQfJqNkIH+8OBm/tPnT1XnxHDuCt9+NxGvE6TXicJrwuB95mA74mPwTa3pyizGZMeXmYC/Kx5+djzs/DlJuHKT8Pc34+prx8TNlZcm1QCCHiRMI4njxNULu/bQ+3Zh+6ai/eI+V4GhWFTiPHnCY8TRa8LTY8DaC9EWFrMGDOz8c8vJCk/HxMrWGbF5rm52PMyOj9G3+EEEJETcK4t7idcOA9+HwNlG/Cf3Qf3mNVwV6t04THacTTbMPbZMHb6AedHd5VWa1YCgsxjx5KUmEB5sKhWIYWYi4sxDJkiFxLFUKIfk7CuKcEAuiyDXg+eg3PpnV4Pt+Du17hcVrxNFnwNxuIvGZrTE/DPHQY9nMKSR1aiCUUuOvLypg2b570bIUQIoFJGMeAv64O9759eLZuxLP5Pdx7duA5Uo2nQYFuDdFkjBmpWM8YRUrRCMwRgWsuLMSYktLhsQNOpwSxEEIkOAnjKGmfD29ZWTB09+7Ds38f7j178Ozdg7/eGS6nDBpzKlgLckkZNQ7LWRdjHTcJS1ERxtTUOLZACCFEXyVh3AGtNa6t23CuWYN7107ce/fhOXgQvMdfLGB0GLEkuUjJ8mA5Q2EZORrr2TMwn3c5anBxxNtjhBBCiJOTMA7RPh/N6zfQuGoVjatX4ysvB4MBy9ACLNl2UgbnYuEAFmst1hQfxqHj4Ywr4IxLYOgFwdfWCSGEEKdgQIdxoKWFpvfeo3HVapxr1uCvr0dZrSRdfDEpt32D5MpnMVV9GCyclAtnzAyG74iZkJIX38oLIYRIGAMujP11dTSWluJcvRrnu+vQLheG1FRSZpaQfOmlJF90EQaLEZZcAbVbYNaDMGo25I4PvjFHCCGEiLEBEcbeo0dpXLWaxtWraP74E/D7MeXlkX7llaTMvhTHlCkoc+il71rDim/Bwffh6mdgwlXxrbwQQoiEl7Bh7P78cxr/Fbz+69qyBQDLiBFkffObpMy+FNv48R2/U/Xfj8GnL8MlD0gQCyGE6BWJE8aBAC2bNwdvwFq1Gs++fQDYJk4k5667SLl0FtYRI05+jE//DqU/h0k3wLQf9kKlhRBCiAQJ44aVb5P94IPsr68Hk4mk884l48avkXLJJZjz86M7yIEP4NXvwLCL4ct/kKFJQgghek1ChLEpNwfviCIKr7uO5JISjGlp3TtA9eew9AZIHwrz/womedazEEKI3hPV7cFKqblKqZ1KqT1KqUUdbB+qlFqjlPqPUupTpdSXYl/VzjnOOYf6b32LtHnzuh/ELbXw4rXB+RuWgSMz9hUUQgghTqLLMFZKGYHFwBeBM4HrlVJntiv2ALBMa302cB3wP7GuaI/weeDlG6HuIFz3ImSdEe8aCSGEGICi6RmfB+zRWu/VWnuApcC8dmU00Prg5TTgSOyq2EO0hte/D/vfhXmLYdgF8a6REEKIASqaa8ZDgEMRy2XA1HZlHgL+qZS6A0gCLo1J7XrSu7+BTS/AjEUw8dp410YIIcQAprTWJy+g1NXAXK31zaHlG4GpWuvbI8rcFTrWb5RSFwD/C0zQWgfaHetW4FaAvLy8yUuXLo1ZQ5xOJ8nJyVGVzal4l/Hbfs2x3BlsH/eDPn3ndHfa1V8kYpsgMdslbeo/ErFdidammTNnbtBaT+lwo9b6pB/gAuDtiOX7gPvaldkKFEYs7wVyT3bcyZMn61has2ZNdAUPfqT1wzla/+8crb2umNahJ0Tdrn4kEdukdWK2S9rUfyRiuxKtTcB63UkmRnPN+BNglFKqSCllIXiD1mvtyhwEZgEopcYBNqCyO78YekXNPnjpekgdDPNfAJM13jUSQgghug5jrbUPuB14G9hO8K7prUqph5VSl4eK3Q3copTaDLwELAj9Cug7WuqCQ5gCPvjq3yEpK941EkIIIYAoH/qhtX4TeLPdup9GzG8DLopt1WLI74VlXw/2jG9cAdmj4l0jIYQQIiwhnsB1UlrDG3fBvn/DFU9A0bR410gIIYRoI/Ff0PveH2DjkuCLH866Id61EUIIIU6Q2GG87VVY9SCMvxJm/jjetRFCCCE6lLhhXLYB/nErFJwXPD3d0buLhRBCiD4gMROq7iC8dB0k58H1L4HZFu8aCSGEEJ1KvBu4XPXwwrXgc8OC1yEpO941EkIIIU4qscLY74O/L4Dq3fC1VyBnTLxrJIQQQnQpccJYa3jrHvj8Hbj8jzCiJN41EkIIIaKSMGFcUPYafP4MXPR9OOfr8a6OEEIIEbXEuIFrxxuc8fmzMO5ymPVgvGsjhBBCdEtihHH6MKqyz4evPClDmIQQQvQ7iZFc+RPYOmERWBzxrokQQgjRbYkRxkIIIUQ/JmEshBBCxJmEsRBCCBFnEsZCCCFEnEkYCyGEEHEmYSyEEELEmYSxEEIIEWcSxkIIIUScJcyzqYUQYqDyer0kJyezffv2eFclptLS0vplm2w2GwUFBZjN5qj3kTAWQoh+rqysjLy8PAoKClBKxbs6MdPY2EhKSkq8q9EtWmuqq6spKyujqKgo6v3kNLUQQvRzLpeLtLS0hAri/kopRVZWFi6Xq1v7SRgLIUQCkCDuO07l30LCWAghhIgzCWMhhBCnLTk5Od5V6NckjIUQQog4kzAWQggRM1pr7rnnHiZMmEBxcTEvv/wyAOXl5UyfPp2zzjqLCRMm8O677+L3+1mwYEG47O9+97s41z5+ZGiTEEIkkP/v/9/KtiMNMT3mmYNTefDL46Mq+49//INNmzaxefNmqqqqOPfcc5k+fTovvvgic+bM4cc//jF+v5/m5mY2bdrE4cOH+eyzzwCoq6uLab37E+kZCyGEiJl169Zx/fXXYzQaycvLY8aMGXzyySece+65PPvsszz00ENs2bKFlJQURowYwd69e7njjjtYuXIlqamp8a5+3ETVM1ZKzQX+ABiBp7XWv2y3/XfAzNCiA8jVWqfHsqJCCCG6Fm0PtrdNnz6dtWvX8sYbb7BgwQLuuusuvv71r7N582befvtt/vznP7Ns2TKeeeaZeFc1LrrsGSuljMBi4IvAmcD1SqkzI8torX+gtT5La30W8EfgHz1RWSGEEH3btGnTePnll/H7/VRWVrJ27VrOO+88Dhw4QF5eHrfccgs333wzGzdupKqqikAgwFVXXcUjjzzCxo0b4139uImmZ3wesEdrvRdAKbUUmAds66T89cCDsameEEKI/uQrX/kKH3zwAZMmTUIpxWOPPUZ+fj7PP/88v/rVrzCbzSQnJ7NkyRIOHz7MwoULCQQCAPziF7+Ic+3jR2mtT15AqauBuVrrm0PLNwJTtda3d1B2GPAhUKC19new/VbgVoC8vLzJS5cuPf0WhDidzoQc55aI7UrENkFitkva1D+kpaVRVFSE0WiMd1Viyu/399s27dmzh/r6+jbrZs6cuUFrPaWj8rG+m/o6YHlHQQygtX4KeApgypQpuqSkJGZfXFpaSiyP11ckYrsSsU2QmO2SNvUP27dvx2g09ruXKnSlP74oopXNZuPss8+Ounw0d1MfBgojlgtC6zpyHfBS1N8eIxsO1PCXT914/YHe/mohhBDitEUTxp8Ao5RSRUopC8HAfa19IaXUWCAD+CC2VexaRYOb94742Higtre/WgghhDhtXYax1toH3A68DWwHlmmttyqlHlZKXR5R9Dpgqe7qInQPuGhUNkYFpbsqe/urhRBCiNMW1TVjrfWbwJvt1v203fJDsatW96TazIxMN1C6s5IfzR0br2oIIYQQpyRhnsBVnGNke3kDxxq690JnIYQQIt4SJown5QQ7+f/eKaeqhRBC9C8JE8YFyYr8VBtrdlbEuypCCCF6iM/ni3cVekTChLFSihmjc1i3u0qGOAkhRBxcccUVTJ48mfHjx/PUU08BsHLlSs455xwmTZrErFmzgOCDVxYuXEhxcTETJ07klVdeAWjzMJbly5dz2223AbBgwQJuu+02pk6dyr333svHH3/MBRdcwNlnn82FF17Izp07geBDQn74wx8yYcIEJk6cyB//+EfeeecdrrjiivBx//Wvf/GVr3ylV/4e3ZFQr1AsGZPDy+sPsfFALVNHZMW7OkII0fveWgRHt8T2mPnF8MVfdlnsmWeeITMzk5aWFs4991zmzZvHLbfcwtq1aykqKqKmpgaAn/3sZ6SlpbFlS7CetbVdD0stKyvj/fffx2g00tDQwLvvvovJZGLVqlXcf//9vPLKKzz11FPs37+fTZs2YTKZqKmpISMjg+985ztUVlaSk5PDs88+yze+8Y3T+3v0gIQK44tGZWMyKEp3VUoYCyFEL3v88cdZsWIFAIcOHeKpp55i+vTpFBUVAZCZmQnAqlWriHwcckZGRpfHvuaaa8KPxqyvr+emm25i9+7dKKXwer3h4952222YTKY233fjjTfyt7/9jYULF/LBBx+wZMmSGLU4dhIqjFNtZs4ZliFDnIQQA1cUPdieUFpayqpVq/jggw9wOByUlJRw1llnsWPHjqiPoZQKz7tcbUfGJCUlhed/8pOfMHPmTFasWMH+/fu7fLzpwoUL+fKXv4zNZuOaa64Jh3VfkjDXjFuVjMmRIU5CCNHL6uvrycjIwOFwsGPHDj788ENcLhdr165l3759AOHT1LNnz2bx4sXhfVtPU+fl5bF9+3YCgUC4h93Zdw0ZMgSA5557Lrx+9uzZPPnkk+GbvFq/b/DgwQwePJhHHnmEhQsXxq7RMZR4YTw6F5AhTkII0Zvmzp2Lz+dj3LhxLFq0iPPPP5+cnByeeuoprrzySiZNmsT8+fMBeOCBB6itrWXChAlMmjSJNWvWAPDLX/6Syy67jAsvvJBBgwZ1+l333nsv9913H2effXabu6tvvvlmhg4dysSJE5k0aRIvvvhieNtXv/pVCgsLGTduXA/9BU5P3+urn6Zxg1LIS7VSuquCa88t7HoHIYQQp81qtfLWW291uO2LX/xim+Xk5GSef/75E8pdffXVXH311eHlxsZGoG3vF+CCCy5g165d4eVHHnkEAJPJxG9/+1t++9vfnnDsdevWccstt0TXmDhIuDBWSlEyOpc3PyvH6w9gNiZc518IIUQ3TJ48maSkJH7zm9/EuyqdSsikKhmTQ6NL3uIkhBACNmzYwNq1a7FarfGuSqcSMowjhzgJIYQQfV1ChnHkECchhBCir0vIMAYZ4iSEEKL/SNwwliFOQggh+omEDePIIU5CCCFEX5awYdz6Fqd3d1fhk7c4CSFEnxL5hqb29u/fz4QJE3qxNvGXsGEMMHNMbnCI08G6eFdFCCGE6FTCPfQjUusQpzU7KzivKDPe1RFCiB733x//Nztqon85QzTGZo7lR+f96KRlFi1aRGFhId/97ncBeOihhzCZTKxZs4ba2lq8Xi+PPPII8+bN69Z3u1wuvv3tb7N+/frwE7ZmzpzJ1q1bWbhwIR6Ph0AgwCuvvMLgwYO59tprKSsrw+/385Of/CT8CM6+LqF7xjLESQghesf8+fNZtmxZeHnZsmXcdNNNrFixgo0bN7JmzRruvvtutNbdOu7ixYtRSrFlyxZeeuklbrrpJlwuF3/+85/53ve+x6ZNm1i/fj0FBQWsXLmSwYMHs3nzZj777DPmzp0b62b2mITuGUNwiNNjK3dyrMFFXqot3tURQoge1VUPtqecffbZVFRUcOTIESorK8nIyCA/P58f/OAHrF27FoPBwOHDhzl27Bj5+flRH3fdunXccccdAIwdO5Zhw4axa9cuLrjgAh599FHKysq48sorGTVqFMXFxdx999386Ec/4rLLLmPatGk91dyYS+ieMcgQJyGE6C3XXHMNy5cv5+WXX2b+/Pm88MILVFZWsmHDBjZt2kReXt4J7yk+VTfccAOvvfYadrudL33pS7zzzjuMHj2ajRs3UlxczAMPPMDDDz8ck+/qDQkfxjLESQghesf8+fNZunQpy5cv55prrqG+vp7c3FzMZjNr1qzhwIED3T7mtGnTeOGFFwDYtWsXBw8eZMyYMezdu5cRI0Zw5513Mm/ePD799FOOHDmCw+Hga1/7Gvfccw8bN26MdRN7TMKfpm4d4vTWZ0fx+QOY5C1OQgjRI8aPH09jYyNDhgxh0KBBfPWrX+XLX/4yxcXFTJkyhbFjx3b7mN/5znf49re/TXFxMSaTieeeew6r1cqyZcv461//itlsJj8/n/vvv59PPvmEe+65B4PBgNls5oknnuiBVvaMhA9jgJIxuSxbX8bGg3VyV7UQQvSgLVu2hOezs7P54IMPOizndDo7Pcbw4cP57LPPaGxsxGaz8eyzz55QZtGiRSxatKjNujlz5jBnzpxTrHl8DYhu4sWjsjEaFKU75VS1EEKIvmdA9IxTbWYmh4Y43Tu3+6dJhBBCxN6WLVu48cYb26yzWq189NFHcapR/AyIMAYZ4iSEEH1NcXExmzZtinc1+oQBcZoaZIiTEEKIviuqMFZKzVVK7VRK7VFKLeqkzLVKqW1Kqa1KqVGdARgAACAASURBVBdjW83TJ0OchBBC9FVdnqZWShmBxcBsoAz4RCn1mtZ6W0SZUcB9wEVa61qlVG5PVfhUyRAnIYQQfVU0iXQesEdrvVdr7QGWAu2f9H0LsFhrXQugte6T3c8SeYuTEEKIPiiaG7iGAIcilsuAqe3KjAZQSr0HGIGHtNYr2x9IKXUrcCtAXl4epaWlp1Dljjmdzi6Pp70ag4Il/1pP82hLzL67J0XTrv4mEdsEidkuaVP/kJaWht/vp7GxMd5VidqgQYMoLy8/aZn+1qZILperW/8/i9Xd1CZgFFACFABrlVLFWus2XVCt9VPAUwBTpkzRJSUlMfp6KC0tJZrjPb/3A/a2+Cgp6R8PEI+2Xf1JIrYJErNd0qb+Yfv27RiNRlJSUuJdlW7pqr6NjY093iafz4fJFPuBRTabjbPPPjvq8tHU4DBQGLFcEFoXqQz4SGvtBfYppXYRDOdPoq5JL5EhTkKIRHb05z/HvT227zO2jhtL/v33n7RMLN9n7HQ6mTdvHtXV1fj9/jb7LVmyhF//+tcopZg4cSJ//etfOXbsGLfddht79+4F4IknnmDw4MFcdtllfPbZZwD8+te/xul08tBDD1FSUsJZZ53FunXruP766xk9ejSPPPIIHo+HrKwsXnjhBfLy8nA6ndxxxx2sX78epRQPPvgg9fX1fPrpp/z+978H4C9/+Qvbtm3jd7/73Sn/fSG6MP4EGKWUKiIYwtcBN7Qr83/A9cCzSqlsgqet955WzXpIyehcHlu5k3/vrOTacwu73kEIIUSX5s+fz/e///1wGC9btoy3336bO++8k9TUVKqqqjj//PO5/PLLUUqd9Fg2m40VK1aglMLtdof327ZtG4888gjvv/8+2dnZ1NTUAHDnnXcyY8YMVqxYgd/vx+l0Ultbe9Lv8Hg8rF+/HoDa2lo+/PBDlFI8/fTTPPbYY/zmN7/hZz/7GWlpaeFHfNbW1mI2m3n00Uf51a9+hdls5tlnn+XJJ5883T9f12GstfYppW4H3iZ4PfgZrfVWpdTDwHqt9WuhbV9QSm0D/MA9Wuvq065dD4gc4iRhLIRINF31YHtKLN9nrLXm/vvvp7S0FJPJFN7vnXfe4ZprriE7OxuAzMzguwbeeecdlixZAoDRaCQtLa3LMJ4/f354vqysjPnz51NeXo7H46GoqAiAVatWsXTp0nC5jIwMAC655BJef/11xo0bh9frpbi4uJt/rRNFdaJca/0m8Ga7dT+NmNfAXaFPnyZDnIQQome0vs/46NGjJ7zP2Gw2M3z48KjeZ9y639q1a8nMzIx6v0gmk4lAIBBebr9/UlJSeP6OO+7grrvu4vLLL6e0tJSHHnropMe++eab+fnPf87YsWNZuHBht+rVmQGZRDLESQghYi9W7zPubL9LLrmEv//971RXB0+8tp6mnjVrVvh1iX6/n/r6evLy8qioqKC6uhq3283rr79+0u8bMmQIAM8//3x4/ezZs1m8eHF4ubW3PXXqVA4dOsSLL77I9ddfH+2f56QGZBhfNFLe4iSEELHW0fuM169fT3FxMUuWLIn6fcat+51//vlt9hs/fjw//vGPmTFjBpMmTeKuu4InY//whz+wZs0aiouLmTx5Mtu2bcNsNvPTn/6U8847j9mzZ5/0ux966CGuueYaJk+eHD4FDvDAAw9QW1vLhAkTmDRpEmvWrAlvu/baa7nooovCp65P14B5UUSkNLuZyUPlLU5CCBFrsXifcet+HQ1tuummm7jpppvarMvLy+PVV1894Th33nknd9555wnr24//nTdvXod3eScnJ7fpKUdat24dP/jBDzptQ3cNyJ4xQMnYHLaVN1DR0L3rEEIIIQauuro6Ro8ejd1uZ9asWTE77oDsGcPxIU6luyq5dorcVS2EEL2tP77POD09nV27dsX8uAM2jFuHOP17p4SxEKL/Cw5q6V8S9X3Gp/JvMWBPU7cOcVq7uxKfP9D1DkII0UfZbDbq6+v7ZSAnGq011dXV2Gzde8LjgO0ZQ3CI07L1ZWw8WMd5RZnxro4QQpySgoICNm/efNKbovojl8vV7VDrC2w2GwUFBd3aZ0CHceQQJwljIUR/ZTabcTqdTJkyJd5VianS0tJuvWyhPxuwp6mh7RAnIYQQIl4GdBgDzBgjQ5yEEELE14AP45ljcgEo3SW9YyGEEPEx4MM4coiTEEIIEQ8DPoxliJMQQoh4G/BhDPIWJyGEEPElYYy8xUkIIUR8SRgjQ5yEEELEl4RxiAxxEkIIES8SxiElY3IAGeIkhBCi90kYh5w5KJXcFBniJIQQovdJGIcopSgZk8O7MsRJCCFEL5MwjlAyJpcGl4//HJIhTkIIIXqPhHEEGeIkhBAiHiSMI7QOcVqzQ64bCyGE6D0Sxu3IECchhBC9LSHCuMnbxLrGdWitT/tYMsRJCCFEb0uIMF6xewUv17zMY588dtqBLEOchBBC9LaECOOvjvsqM1Jm8Lftf+PRjx4loE99aFLrW5xkiJMQQojekhBhrJTiqoyrWDh+IS/vfJmHP3gYf8B/ysebOVaGOAkhhOg9UYWxUmquUmqnUmqPUmpRB9sXKKUqlVKbQp+bY1/VLuvIDyb/gG9N/Bav7H6Fn7z3E3wB3ykdS4Y4CSGE6E2mrgoopYzAYmA2UAZ8opR6TWu9rV3Rl7XWt/dAHaOmlOL2s2/HbDDzp01/whvw8vNpP8dsMHfrOJFvcbpnztgeqq0QQggRFE3P+Dxgj9Z6r9baAywF5vVstU7PtyZ9i7sn383K/Su559/34PV7u32MGWNy2HpEhjgJIYToedGE8RDgUMRyWWhde1cppT5VSi1XShXGpHanYcGEBSw6bxGrD67m+6Xfx+13d2v/1iFOf/vwQEyGTAkhhBCdUV0FjVLqamCu1vrm0PKNwNTIU9JKqSzAqbV2K6W+BczXWl/SwbFuBW4FyMvLm7x06dKYNcTpdJKcnHzC+vca32NpzVLG2sZyS84tWAyWqI6ntebx/7j5T4WfqflGFkywYjepmNU3Wp21qz9LxDZBYrZL2tR/JGK7Eq1NM2fO3KC1ntLhRq31ST/ABcDbEcv3AfedpLwRqO/quJMnT9axtGbNmk63rdi9Qhc/V6wXrlyomzxNUR/T7w/oP72zWxctel3P/NUave1IfQxq2j0na1d/lYht0jox2yVt6j8SsV2J1iZgve4kE6M5Tf0JMEopVaSUsgDXAa9FFlBKDYpYvBzY3q2fCz3sipFX8Itpv2DjsY3ctuo2nB5nVPsZDIrvzhzJCzefT6PbxxWL3+PlTw7KaWshhBAx1WUYa619wO3A2wRDdpnWeqtS6mGl1OWhYncqpbYqpTYDdwILeqrCp+q/RvwXj01/jC2VW7j1X7dS766Pet8LzsjizTunMWV4Bj96ZQt3L9tMs+fUhk0JIYQQ7UU1zlhr/abWerTW+gyt9aOhdT/VWr8Wmr9Paz1eaz1Jaz1Ta72jJyt9qr4w/Av8tuS37KjZwS3/vIVaV23U++akWFnyjal8/9JRrNh0mMv/9B67jjX2YG2FEEIMFAnxBK7umDl0Jo9f8jh76/fyzX9+k+qW6qj3NRoU3790NH/75lTqmj3M+9N7LN9Q1oO1FUIIMRAMuDAGuHjIxfxp1p841HCIhW8vpKK5e0/aumhkNm/eOY2JBWn88O+buXf5Zlo8p/74TSGEEAPbgAxjgPMHnc8Tlz7BsaZjLFy5kKNNR7u1f26qjRdunsrtM0fy9w1lXLH4PfZURHdjmBBCCBFpwIYxwJT8KTw5+0lqXDUsWLmAssbunXI2GQ38cM4Ynlt4HpVON5f/aR2vbjrcQ7UVQgiRqAZ0GAOclXsWT3/haRo9jSx8eyEHGw52+xgzRufwxp0XM35wKt9buon7/rEFl1dOWwshhIjOgA9jgPHZ43lmzjO4fW4WrFzA3vq93T7GoDQ7L95yPrfNOIOXPj7Ilf/zPvurmnqgtkIIIRKNhHHImMwxPDPnGQI6wMKVC9ldu7vbxzAbDSz64lieWTCFI/UtXPbHdbzxaXkP1FYIIUQikTCOMDJjJM/OfRaTMvGNt7/B9upTe5DYJWPzeOPOaYzKS+a7L27kp69+htsnp62FEEJ0TMK4naK0Ip6b+xx2k51v/vObvLH3DbyB7r+CcUi6nZdvvYCbLy5iyQcHuPqJDzhY3dwDNRZCCNHfSRh3oDC1kOfmPkeeI49F7y5izvI5PLH5Capaqrp1HIvJwAOXnclTN07mQHUT//XHd1n5WfeGUAkhhEh8EsadGJw8mFcuf4XFsxYzOnM0/7Ppf5i9fDb3vXsfWyq3dOtYXxifzxt3TmNEdhK3/W0Di175lM8rZUyyEEKIIFO8K9CXGZSB6QXTmV4wnX31+1i6Yymvfv4qr+99neLsYq4fez1zhs/BYuz6HcmFmQ6W3XYBj63cyZIP9rP0k0NMH53DgguHUTI6F4Oh99+VLIQQom+QnnGUitKKuG/qfay6ehX3nXcfjZ5G7l93P19Y/gUWb1oc1SM1rSYjP7nsTN5bdAk/uHQ0O8ob+MZz65n5m1Kefncv9S3dvzYthBCi/5Mw7qZkSzI3jLuBV694lT9f+mfGZ4/nyc1PMmf5HO79971sqtjU5fuOc1NsfO/SUaz70SU8fv3ZZCdbeeSN7Vzwi9X8eMUWdsvboIQQYkCR09SnyKAMXDTkIi4achEHGw6ydOdSVuxewVv73+LMrDO5YewNzC2ai9Vo7fQYFpOByycN5vJJg/nscD3Pvb+fv28o44WPDnLhGVksuHA4pi6CXQghRP8nPeMYGJo6lHvPvZfV16zmgakP4PK5eOC9B5j999k8vvHxqF5CMWFIGr++ZhIfLLqEe+aMYV9VE7f+dQP3rm3hyX9/Tl2zpxdaIoQQIh4kjGPIYXYwf+x8/m/e//GXL/yFSbmTeHrL08x9ZS53l97NhmMbujyFnZVs5bszR/LuvTN54qvnkGVT/OKtHZz/i9UseuVTtpc39FJrhBBC9BY5Td0DlFKcP+h8zh90PmWNZby882Ve2f0K/zzwT8ZkjOHq0Vczo2AGg5IHdXoMk9HAF4sHYa/eSd6Yc1jywX5W/OcwSz85xHlFmSy4cDhfODMPk1F+TwkhRH8n/yXvYQUpBdw95W5WX7OaBy94kAABHv3oUb7wyhe48rUr+f2G37Px2EZ8AV+nxxg3KJVfXDmRD++bxf1fGsuRuha+88JGpj22hsVr9lDtdPdii4QQQsSa9Ix7id1k5+rRV3PVqKvYV7+PtWVrWXt4Lc9vfZ7//ex/SbWkctGQi5heMJ2LB19Mui39hGOkOyzcOv0MvnnxCN7ZUcHz7+/nV2/v5A+rd3PZxEH8V/EgLhqZjc1sjEMLhRBCnCoJ416mlGJE+ghGpI9gwYQFNHoaef/I+6wtW8u6w+t4a99bGJSBidkTmV4wHbvHjtYapY4/FMRoUMw+M4/ZZ+axp6KR598/wIr/HOYfGw9jMxu4eGQ2l47L45KxueSm2uLYWiGEENGQMI6zFEsKc4bPYc7wOQR0gK1VW1l7eC1ry9by+H8eB+C55c8xrWAa04dMZ+qgqTjMjvD+I3NT+NkVE/jJZWfy0b5qVm07xqrtFazaHnwIyaSCNGaNy+PScXmMG5TSJtSFEEL0DRLGfYhBGSjOKaY4p5jvnvVdKpsrefqdp6lIquDNvW+yfNdyLAYL5+afGwzngukUphQCwTHL00blMG1UDg9drtl5rJHV2yv417Zj/G7VLn77r10MTrMxa1wes8blcsEZWVhNcjpbCCH6AgnjPizHkcMFyRdQUlKC1+9lQ8UG1pat5d2yd/nlx7/klx//kqK0IqYPmc7FBRdzZtaZpFpSUUoxNj+VsfmpfHfmSCob3azZUcGq7cdYvqGMv354AIfFyPRROcwal8slY3PJSu784SRCCCF6loRxP2E2msPDpe49914ONhwM3gRWtpYXd7zI89ueB2Bw0mBGZ45mdMZoxmSMYXTGaApTCrn23ODH5fXzwd7g6ezV2ytYufUoSsHZhenMGhe8Dj0qN1lOZwshRC+SMO6nhqYO5Wtnfo2vnfk1mr3NbDi2gZ21O9lVs4tdtbt4t+xd/NoPBO/kHpk+ktEZoZDOHMOP/msUj1wxga1HGli9vYLVO47xq7d38qu3d1KYaWfW2OB15inDM+TubCGE6GESxgnAYXYwrWAa0wqmhde5/W721O0Jh/Ou2l2sOriKV3a/Ei4zOGlwMKCzRvPdy0aTbR7J9kMW1uyo4qWPD/Lc+/uxGA0UF6QxZVgGk0MfOaUthBCxJWGcoKxGK+OzxjM+a3x4ndaaiuaKYA+6dtfxXvThdr3onJFcO3IUBu9gaupT2X+0hWffr+LJtcEeclF2EpOHZTBlWAZThmcwIjtZ3scshBCnQcJ4AFFKkZeUR15SHtMLpofXd9SLXlO2mnp3fbCAHayjIMecgU1l4/Ok88/yJF7dm0rAm06yMZezBxVx7rDBTBmWwaTCdDm1LYQQ3SBhLE7aiy5zlnHEeYQjziOUN5WHp07DZnRa8E1SAWADsH6fjcCuDPBlkGnJoyi9gOL84Vw4bBTjc4eTbk2XG8OEEKIDUYWxUmou8AfACDyttf5lJ+WuApYD52qt18eslqLXRfaiJ+dNPmF7QAeocdUEg7opGNb76srYVXWQw84jNPo+Z1Ozm0174a97g/sYsJBhySNdp7Jh/UZGZ4xiVMYoitKKTvreZyGESHRdhrFSyggsBmYDZcAnSqnXtNbb2pVLAb4HfNQTFRV9i0EZyLZnk23PZmLOxBO2a62pbK5l3b7dfHRoD9sqDnCw4TBHqabCUsWez5agVPA6tcJAnn0IY7NGMTZrNCPTRzIyfSRDU4diNph7u2lCCNHroukZnwfs0VrvBVBKLQXmAdvalfsZ8N/APTGtoeiXlFLkJmVy5YSpXDlhKhAM6APVzbz0zw/wp+Xxn/Ld7K7bg4sjHLIe5Ujdp5QeWgMq+M5nozIxLGU4Y7JGhQN6VPoohqQMwaDkhWNCiMQRTRgPAQ5FLJcBUyMLKKXOAQq11m8opSSMRYeUUgzPTuKCwSZKSoqBYrTWlNe72Hqkga1H6tlyuJqtlbup8hzEYD3KroZj7Kv5CG16K3wcq9HGGekjwuE8MmMkZ6SdQbYjW3rSQoh+SWmtT15AqauBuVrrm0PLNwJTtda3h5YNwDvAAq31fqVUKfDDjq4ZK6VuBW4FyMvLm7x06dKYNcTpdJKcnByz4/UVidiuaNrk9GgONgY40BDgYIOf/c4WKrxHMViPYbAew2Q7hsl2lICxsc1+yYZkUowppBpTw58UYwqphtQ26x0GR8x71wP136q/ScQ2QWK2K9HaNHPmzA1a6ykdbYumZ3wYKIxYLgita5UCTABKQ3fK5gOvKaUubx/IWuungKcApkyZoktKSqJtQ5dKS0uJ5fH6ikRs16m2qcXjZ+exRrYeqQ/1pBvYcewoPmM5BmsFBlMj3qQWXPZmPGYnR/0HafbX4g14TjiWSZnItGeSZcsKX/vOtmeTZQ8ut65PtabiMDmwGq1d3gku/1b9QyK2CRKzXYnYps5EE8afAKOUUkUEQ/g64IbWjVrreiC7dflkPWMhTofdYuSswnTOKkwPr/P5A+yramJbeQOfVzj5vLKJzyud7K1qwuMLABoMbjJSXAzO8pKV6iElqQWLrQmMjbT466hqqWJnzU6qXdXhh5+0Z1AGHCYHDpMDu9kenJrsOMzBdQ6zg9qaWjZt2NRmXevUbrKH900yJZFmTcNmkndNCyGCugxjrbVPKXU78DbBoU3PaK23KqUeBtZrrV/r6UoK0RmT0cCovBRG5aW0We8PaA7XtvB5pfP4p6KJrbudVDcd7ylbjAaKspMYk5vEiDwHgzIDpKe4sdubafLV0uBpoMXXQrO3OTj1NbeZr3XVcth3mGZvM42uRt7b+h4+7Yuq7lajlTRLGqnWVNKsacfnLWnBZWsHy5ZUks3yIg8hEk1U44y11m8Cb7Zb99NOypacfrWEOD1Gg2JoloOhWQ5mjs1ts622ycPeqmA4twb19vJGVn52lEDELRSD0hwUZecwPDuJ4VkOxmYlUZSdRGGmo8MnjLWeUvP6vTT7msMh3hrgresaPY00eBpocDdQ76mn3h38lDnLqK+uD/8A6LRtykiqJfWEsM6wZZBlyyLLnkWWLSt8Gj7LloXZKDe2CdGXyRO4xICTkWRhclImk4dltlnv9vk5WN0cCugm9lQ42VfVxFtbyqlt9obLKQWD0+wMz3YwPCsp+MlOotIZwO3zYzWZSTMGA/JUuf3uYFi769sEdoOnoc203l1PtauavfV7qXHVdBriqZZUMm2Z4aA+IbBbl22ZOMyOU663EOLUSBgLEWI1GTs85Q1Q1+xhf3Uz+6ua2F/dxP6qJvZVN/PGlnLqIoL6x++tZHCanaLsJIZlOSjKbg1rB4WZDqym6J7ZbTVayXHkkOPI6VYbmr3NVLuqqW6pDk9rXDVtlnfV7qK6vJpGT2OHx7Cb7OGA9jl9vLX2LZLNySRbkkmxpJBkTgouh9a1n8rwMiG6T8JYiCikOyyc5bC0uXmsVV2zJ9iDXrcBe87QYFhXN/P6p+XUtxwPaoOCwel2hmU5GJJuZ3DoMyT0yU+znfYLNhzm4A1jhSmFXZb1+D3BoG4N71BgR4b3Yf9htlRtwelx0uhtxBfo+nq41WhtG9Ch+SRzEimWFBwmBzaTDavRisVowWa0YTFaOly2Gq1YTdbwNqvRisVgkWvmIuFIGAtxmtIdFs4eaqF+sImSktFtttU2eULh3MS+qmYOVDdxoLqZ0p2VVDS6TzhWdrKVIem2cEi3CewMOxkOc8yCyGK0kJ+UT35Sfqdl2g8tcfvdOD1OnN7Qx9PJtN266oZqnF4nTZ4mnF4nmpM/36ArkeHc+jEbzFiMFswGM2aDGZPRhMUQWjaaw+srairYsH5DeDlyW+u8xWDBbDSTZE4ix55DriOXVEuq/AgQPUbCWIgelJFkISPJwtlDM07Y5vb5OVbv5nBdC0fqWtpMdx1rZM3OClzeQJt9bGbD8aBOCwZ0MLBtDE6LTe/6ZKxGK1a7lSx71ikfQ2uNL+DD7Xfj9rvx+D24/C48fk94XeS2cBmfC08gooyv7f7egBdvwIvP78Mb8NLkbQqva/14/B5aPC18vPNjvH5v1He+A1gMFnIcOWTbs8l15Lad2nPJdmSTY8+Rt5OJUyJhLEScWE3G8B3fHdFaU9vsbRPUwY+LsroWdhytoLKD3nVWkoVB6TbyU4MhPSjt+HRQmo38NBtmY/ye7a2UCvZAjWaS6f2nK0X29gM6gC/gw+P3tA1uvxdPwEOjp5HKlkoqmyvbTD+v+5wPyz/s8Lq72WAm254dvOZvD30cx6fZ9mzSremkWlKxm+wS3AKQMBaiz1JKkZlkITPJwoQhHd+Z7fb5OVrv4nBtC0fqXRytD07L61ooq23m433VNLh87Y4LOclWBqWFAjrUqx4UEdy5KTaMhsQPCYMyYDFasBgtp7S/y+eisqWSqpYqKporTpgeaDjA+mPrqXfXd7i/2WAOjzHvaFx5m/URyzLWPPFIGAvRj1lNRoZlJTEsK6nTMk1uH+X1wR51eX0L5fUuyutcHKlvYU+lk3d3V9LkafvkMaNBkZdixaE8/P3wRvLTbAxKs5GXagv3rnNTbFhMA/vtWTaTjcKUwi5vmHP73VS1VFHZHAzuzoaslTvL2eHZQb27vltjzVMtqdTV1LF89XI0moAOBK/La8LzGo3Wx6cBHbwEEl4mQHCX4/sbMLS5Nh95o11H67rabjV0fByjoecurfQXEsZCJLgkq4mRuSmMzD1xyBYET4c3uIKBXV7nCoZ1KLy3HzjC9qMNrNlZQXO7wFYKspKs4XBuneanti7byU+1YbfIf2itRitDkocwJHlI1Pt4/J4248k7C/B6dz01rhoafA34m/0opWj9n0EZQBGeVwR70wZlwGgwBktFlFeq7XJAB/AEPDT5mqhz13V4Ld/td5/2DXkmZQreSW9qe2e9u8nNs2892+au+s5+FHT4Ce3Teod+eBrxPX3ldawSxkIMcEop0uxm0uxmxuanttlWWlpLSUlJOLCPNQTD+mioh3203sXRBheHapr5eF9Nm6FcrdLs5jaBnZdqIyfFSk6yldzQfHayJeox2AOFxWgJv8AkGvF6qUJHN+S1znsD3g7Xtw/zzsqUt5RjMpho9jVT56474Ua/1vnTYTaYsRltbcK+9ZNmTeNPs/4Uo7/UyUkYCyG6FBnYozt4KEqrFo+fow3BnvXRelebwD5a7+Kzww1UOTv+j2ea3RwR0sFpTkrbT26KjXS7GcMAuJ7dX/TkDXnR/MDQWofvsm+96779Xflunzsc5OFpJ+Va511+V/hUfm+QMBZCxIzdYqQoO/gM7854/QGqnR4qG91UOl1UNLhD86Fpo5tNh+qoaHDT4j3xLVomgyI7Mqgj5rOTg73s7NB8qs0kNzolOKVUuCfbn0kYCyF6ldloCF5bTrMBJ39+t9PtCwd08OOi0ukOBrjTzbEGF58drqfK6W7zko9WFpOB7CRLRFBbaarxsM+8L7zcGuipdgluET8SxkKIPivZaiLZajppTxuCr8ysbfZQ5XRT1RiaOo/3uKucHsrrXWw5XE+V08vre7edcAyL0UBWsuV47zrZSnaKlUxH8MEtWaEHuGQ6LGQmW0iyGCW8RcxIGAsh+j1j6NR1drIVOn+6JwDvrFnDpHMvpMrZNrSrQqfOq5xuKhrdbCtvoKbJg9ff8Z3CFqOBjCQzmUlWMpPMZDgs4XHhmUkWMhwRAR5aHuhDwUTnJIyFEAOKQSmykq1kJVsZQ+c3o0Hw5qBGt4/aJg81EZ/aZg/VTZ7Qei81TW6O1AXDu6M7ylul1h5EZwAACW9JREFUWE3BXnZyMKizkqxkts4nW8hMskbMyx3mA4mEsRBCdEIpRarNTKrNfNIHq0Ty+gPUNXuDge0MBndN04mfw3UuPi2rp6bJg6+jC94Ew7s1mLOSg0EdOd+6LTvZireTY4j+QcJYCCFiyGw0hO/uJq/r8lprGlp8VDe5qW4KBnhNk4dqZ2i5yUNNk5tDNc1sOlRHTZMHfyfBm1S6knSHhYzQafPgxxx8YYnDQrrj+On01nmHXPvuEySMhRAijpRSpDnMpDnMjMjpunwgoGlweSOCO3i9e+PWnaTnFlDXHOqNN3v5f+3df4wUZx3H8ffn7vbKHsVbkKRCIa00xARjrIRUNLUhtqEUTavGGKqJaE0aYklsjDEkbQjpf63RPzSkpiqxmmqJP6pNQ9NiNTE2AUU8flko1xYrhB9puR/AHd4efP1jnj2Gvdm7OVhudqbfVzLZ+fHM8Xx5Zva788zss2+fHqLv3Mi48cnjOjvaooRdS94zS1FCD+sqXZ1UyiUqXdHUXe6ku1zy+99N5snYOedypK1NUYLs6uSWWPJecP4tVqxYkrjP6IWL9A9XQ6KucvrcyNh8X7gH3jcUbT904gz9Q1X6h6sNr8ABujrbqZRLdCck60pXaWzd2HJXiUq5kxmlNr8ST+DJ2DnnCq6jve3S0+YpXbwYPbw2OFwNyXlkLEkPDF2a7x+qMjA8Qu+ps2MJv9ET6BBdiVfK0VV3d1dp7Aq8u3YlXi6FDxsljp65yMnB81S6SoV/mM2TsXPOuXHa2i4NgbpwTvr9zIzh6oUoWYckPhBL3P1jiTy6Gn/rnXP8a6if/qEqIxfGDz/56KuvAFAutTO761KijifzSjma7y5HD9t1l0tjy3n5PrgnY+ecc00jia7ODro6O5hfKafer5bE+2IJ+9VdPcy/eTEDw7Xu9OgqvG+oysETg6m60zvaxPvCh4raazR1xOaTtpe4/rrpG5XNk7FzzrnMxZP4jSGJV492sGL5TRPuV/su+MBQlYHhKoPD0WvDaWiEt989F5U9PzphIp81o4N9m+5uapyNeDJ2zjmXW/Hvgi+c4r5mxtn/jV6WrOPJPKHX/JrxZOycc+49SRKzZpSYNaPEgtnZ1sW/KOacc85lzJOxc845lzFPxs4551zGPBk755xzGUuVjCWtknRIUq+kDQnb10naJ6lH0t8kJY/J5pxzzrlxJk3GktqBzcA9wBLg/oRk+ysz+4iZ3Qo8Afyg6TV1zjnnCirNlfFtQK+ZvWlmI8CzwH3xAmY2GFucCfgPazrnnHMppfme8Y3Af2PLR4GP1xeS9BDwbaAT+HRTauecc869BzRt0A8z2wxslvRl4FFgbX0ZSQ8CD4bFs5IONevfB+YC7zTx77WKIsZVxJigmHF5TPlRxLiKFlPDsT3TJONjcNkoYwvCukaeBZ5M2mBmTwFPpfg3p0zSLjNbdi3+dpaKGFcRY4JixuUx5UcR4ypiTI2kuWf8D2CxpA9K6gTWAM/HC0haHFv8DHC4eVV0zjnnim3SK2MzG5W0HngJaAe2mNkBSY8Bu8zseWC9pLuAKtBHQhe1c84555KlumdsZtuAbXXrNsbmv9Xkel2Ja9L93QKKGFcRY4JixuUx5UcR4ypiTIlk5t9Ccs4557Lkw2E655xzGctdMk4xNOd1kraG7Tsl3Tz9tZwaSQsl/UXSvyUdkDSu21/SCkkDYcjRHkkbk/5WK5F0JDZM6q6E7ZL0w9BWeyUtzaKeaUn6UOz/v0fSoKSH68rkop0kbZF0StL+2Lo5krZLOhxeE3/hVdLaUOawpJZ5PqRBTN+TdDAcX89JqjTYd8JjNUsN4tok6VjsOFvdYN8J3y+z0iCmrbF4jkjqabBvy7bVVTGz3ExED5C9ASwiGlxkD7Ckrsw3gR+H+TXA1qzrnSKuecDSMD8LeD0hrhXAC1nXdYpxHQHmTrB9NfAiIGA5sDPrOk8htnbgBHBTHtsJuANYCuyPrXsC2BDmNwCPJ+w3B3gzvM4O87OzjmeCmFYCHWH+8aSYwrYJj9UWjGsT8J1J9pv0/bKVYqrb/n1gY97a6mqmvF0ZTzo0Z1h+Osz/FrhTkqaxjlNmZsfNbHeYPwO8RjTyWdHdB/zCIjuAiqR5WVcqpTuBN8zsP1lX5EqY2V+B03Wr4+fO08DnEna9G9huZqfNrA/YDqy6ZhWdgqSYzOxlMxsNizuIxknIlQZtlUaa98tMTBRTeL/+EvDraa1UxvKWjJOG5qxPWmNlwkk4ALx/WmrXBKFb/WPAzoTNn5C0R9KLkj48rRW7Mga8LOmfYfS1emnas1WtofGbRd7aqeYGMzse5k8ANySUyXObPUDUE5NksmO1Fa0P3e9bGtxSyGtbfQo4aWaNxqvIY1tNKm/JuNAkXQ/8DnjYLv/xDYDdRF2iHwV+BPxhuut3BW43s6VEv/j1kKQ7sq5QM4TBb+4FfpOwOY/tNI5F/YGF+aqFpEeAUeCZBkXydqw+CdwC3AocJ+rWLYr7mfiqOG9tlUreknGaoTnHykjqALqBd6eldldBUokoET9jZr+v325mg2Z2NsxvA0qS5k5zNafEzI6F11PAc0TdZnFTHWq1VdwD7Dazk/Ub8thOMSdrtwnC66mEMrlrM0lfAz4LfCV8yBgnxbHaUszspJldMLOLwE9Irm8e26oD+AKwtVGZvLVVWnlLxpMOzRmWa094fhH4c6MTsFWEeyQ/A14zs8Tfgpb0gdq9b0m3EbVdy37IkDRT0qzaPNGDNPvrij0PfDU8Vb0cGIh1k7ayhp/c89ZOdeLnzlrgjwllXgJWSpodukZXhnUtSdIq4LvAvWY21KBMmmO1pdQ9W/F5kuub5v2y1dwFHDSzo0kb89hWqWX9BNlUJ6IncF8nekrwkbDuMaKTDWAGUfdhL/B3YFHWdU4R0+1EXYJ7gZ4wrQbWAetCmfXAAaInIncAn8y63pPEtCjUdU+od62t4jEJ2Bzach+wLOt6p4hrJlFy7Y6ty107EX2YOE40hO1R4BtEz1a8QjS2/J+AOaHsMuCnsX0fCOdXL/D1rGOZJKZeovumtfOq9k2L+cC2iY7VVpkaxPXLcM7sJUqw8+rjCsvj3i9bYUqKKaz/ee1cipXNTVtdzeQjcDnnnHMZy1s3tXPOOVc4noydc865jHkyds455zLmydg555zLmCdj55xzLmOejJ1zzrmMeTJ2zjnnMubJ2DnnnMvY/wFjR0Y9EShJwAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "rh8YMrfpoM3u", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "d17ee3a9-9223-43a0-caac-6dc32ccc0e6f" }, "source": [ "y_pred = model.predict(X_test_scaled)\n", "y_pred = y_pred.argmax(axis=1)\n", "print(classification_report(y_test, y_pred,\n", " target_names=class_names))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " T-shirt/top 0.81 0.82 0.82 1000\n", " Trouser 0.98 0.96 0.97 1000\n", " Pullover 0.78 0.76 0.77 1000\n", " Dress 0.83 0.89 0.86 1000\n", " Coat 0.78 0.77 0.78 1000\n", " Sandal 0.95 0.92 0.94 1000\n", " Shirt 0.67 0.64 0.65 1000\n", " Sneaker 0.91 0.94 0.93 1000\n", " Bag 0.95 0.96 0.95 1000\n", " Ankle boot 0.95 0.95 0.95 1000\n", "\n", " accuracy 0.86 10000\n", " macro avg 0.86 0.86 0.86 10000\n", "weighted avg 0.86 0.86 0.86 10000\n", "\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "dkEUyaYOomnP" }, "source": [ "# 3. Batch Normalization\n", "\n", "Batch normalization is a technique for training deep neural networks that standardizes the inputs for each mini-batch before going into a layer. This has the effect of stabilizing the learning process and preventing overfitting.\n", "\n", "- fixed distributions of inputs would remove the ill effects of the internal covariate shift (change in the distributions of neurons when training) [https://arxiv.org/abs/1502.03167]\n", "\n", "- Batch normalization acts to standardize only the mean and variance of each unit in order to stabilize learning [https://amzn.to/2NJW3gE]" ] }, { "cell_type": "markdown", "metadata": { "id": "j0VwKMWmpR7I" }, "source": [ "## 3.1 Build Model with Batchnorm" ] }, { "cell_type": "code", "metadata": { "id": "-GqOrG5ooRtn" }, "source": [ "model = tf.keras.models.Sequential([\n", " \n", " tf.keras.layers.Flatten(input_shape=[28, 28]),\n", "\n", " tf.keras.layers.BatchNormalization(),\n", "\n", " tf.keras.layers.Dense(300, \n", " activation='relu'),\n", " tf.keras.layers.BatchNormalization(),\n", "\n", " tf.keras.layers.Dense(100, \n", " activation='relu'),\n", " tf.keras.layers.BatchNormalization(),\n", " \n", " tf.keras.layers.Dense(10, activation=\"softmax\")\n", "])" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "uePW2uVWpn_R", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "44fae16a-b2ea-41d2-bc48-f547c1389f93" }, "source": [ "model.compile(loss=\"sparse_categorical_crossentropy\", \n", " optimizer=tf.keras.optimizers.SGD(1e-3),\n", " metrics=[\"accuracy\"])\n", "model.summary()" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Model: \"sequential_3\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "flatten_3 (Flatten) (None, 784) 0 \n", "_________________________________________________________________\n", "batch_normalization (BatchNo (None, 784) 3136 \n", "_________________________________________________________________\n", "dense_9 (Dense) (None, 300) 235500 \n", "_________________________________________________________________\n", "batch_normalization_1 (Batch (None, 300) 1200 \n", "_________________________________________________________________\n", "dense_10 (Dense) (None, 100) 30100 \n", "_________________________________________________________________\n", "batch_normalization_2 (Batch (None, 100) 400 \n", "_________________________________________________________________\n", "dense_11 (Dense) (None, 10) 1010 \n", "=================================================================\n", "Total params: 271,346\n", "Trainable params: 268,978\n", "Non-trainable params: 2,368\n", "_________________________________________________________________\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "97Fxj8KBq__C" }, "source": [ "## Train & Test with Batchnorm" ] }, { "cell_type": "code", "metadata": { "id": "7lHt_06Vpr4z", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "59390bea-d45c-48ae-901a-dcfcfd6a0942" }, "source": [ "history = model.fit(X_train, y_train, epochs=20,\n", " validation_split=0.1)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Epoch 1/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.8514 - accuracy: 0.7147 - val_loss: 0.5683 - val_accuracy: 0.8027\n", "Epoch 2/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.5785 - accuracy: 0.7996 - val_loss: 0.4920 - val_accuracy: 0.8270\n", "Epoch 3/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.5180 - accuracy: 0.8195 - val_loss: 0.4556 - val_accuracy: 0.8390\n", "Epoch 4/20\n", "1688/1688 [==============================] - 5s 3ms/step - loss: 0.4842 - accuracy: 0.8311 - val_loss: 0.4310 - val_accuracy: 0.8473\n", "Epoch 5/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.4597 - accuracy: 0.8383 - val_loss: 0.4159 - val_accuracy: 0.8520\n", "Epoch 6/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.4434 - accuracy: 0.8438 - val_loss: 0.4031 - val_accuracy: 0.8567\n", "Epoch 7/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.4271 - accuracy: 0.8499 - val_loss: 0.3920 - val_accuracy: 0.8608\n", "Epoch 8/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.4141 - accuracy: 0.8534 - val_loss: 0.3843 - val_accuracy: 0.8635\n", "Epoch 9/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.4016 - accuracy: 0.8576 - val_loss: 0.3791 - val_accuracy: 0.8650\n", "Epoch 10/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.3923 - accuracy: 0.8619 - val_loss: 0.3712 - val_accuracy: 0.8682\n", "Epoch 11/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3843 - accuracy: 0.8633 - val_loss: 0.3663 - val_accuracy: 0.8692\n", "Epoch 12/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3768 - accuracy: 0.8661 - val_loss: 0.3610 - val_accuracy: 0.8717\n", "Epoch 13/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3670 - accuracy: 0.8696 - val_loss: 0.3582 - val_accuracy: 0.8732\n", "Epoch 14/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3627 - accuracy: 0.8702 - val_loss: 0.3545 - val_accuracy: 0.8740\n", "Epoch 15/20\n", "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3550 - accuracy: 0.8736 - val_loss: 0.3496 - val_accuracy: 0.8773\n", "Epoch 16/20\n", "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3489 - accuracy: 0.8750 - val_loss: 0.3455 - val_accuracy: 0.8775\n", "Epoch 17/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.3469 - accuracy: 0.8761 - val_loss: 0.3441 - val_accuracy: 0.8773\n", "Epoch 18/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3426 - accuracy: 0.8776 - val_loss: 0.3424 - val_accuracy: 0.8792\n", "Epoch 19/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3353 - accuracy: 0.8804 - val_loss: 0.3414 - val_accuracy: 0.8785\n", "Epoch 20/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3297 - accuracy: 0.8827 - val_loss: 0.3360 - val_accuracy: 0.8807\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "xYhUc3eCpwIr", "colab": { "base_uri": "https://localhost:8080/", "height": 324 }, "outputId": "dd9eb7ea-a892-4c05-d139-38bfe50a4711" }, "source": [ "pd.DataFrame(history.history).plot(figsize=(8, 5))\n", "plt.grid(True)\n", "plt.gca().set_ylim(0.3, 1);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeMAAAEzCAYAAAACSWsXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeVxVdf7H8df3LnBZL5uAgCLmgiK4a2oZarbMlLaZOVbmTDXNTDnVTGV70zLV1LROU+M0lbZZ2TItlr8pJTO1XFJRUTRXcEHZBOEC997v749zQUQU1AsXLp/n43EfZ73nfL9gvfme8z3fo7TWCCGEEMJ3TL4ugBBCCNHRSRgLIYQQPiZhLIQQQviYhLEQQgjhYxLGQgghhI9JGAshhBA+1mQYK6VeU0oVKKXWH2e7Ukq9oJTaqpRap5Qa5P1iCiGEEP6rOS3jN4ALTrD9QqCn53Mj8PLpF0sIIYToOJoMY631YqDoBLtMBOZow3IgQinV2VsFFEIIIfydN+4ZJwK76y3nedYJIYQQohksrXkypdSNGJeyCQoKGtylSxevHdvtdmMy+V9/NH+slz/WCfyzXlKn9sMf6+VvdcrNzT2ote7U2DZvhHE+UD9VkzzrjqG1ngXMAhgyZIheuXKlF05vyMrKIjMz02vHayv8sV7+WCfwz3pJndoPf6yXv9VJKbXzeNu88SfHp8C1nl7VZwKlWuu9XjiuEEII0SE02TJWSr0LZAIxSqk84EHACqC1fgWYD/wC2ApUANNbqrBCCCGEP2oyjLXWU5rYroE/eK1EQgghRAfjP3fGhRBCiHZKwlgIIYTwMQljIYQQwsckjIUQQggfkzAWQgghfEzCWAghhPAxCWMhhBDCxySMhRBCCB+TMBZCCCF8TMJYCCGE8DEJYyGEEMLHJIyFEEIIH5MwFkIIIXxMwlgIIYTwMQljIYQQwsckjIUQQggfkzAWQgghfEzCWAghhPAxCWMhhBDCxySMhRBCCB+TMBZCCCF8TMJYCCGE8DEJYyGEEMLHJIyFEEIIH5MwFkIIIXxMwlgIIYTwMQljIYQQwsckjIUQQggfkzAWQgghfEzCWAghhPCxZoWxUuoCpdRmpdRWpdTMRrYnK6W+UUqtU0plKaWSvF9UIYQQwj81GcZKKTPwEnAh0BeYopTq22C3p4E5WusM4GHgcW8XVAghhPBXzWkZDwO2aq23aa2rgbnAxAb79AUWeuYXNbJdCCGEEMfRnDBOBHbXW87zrKtvLXCZZ/5SIEwpFX36xRNCCCH8n9Jan3gHpa4ALtBaX+9ZvgYYrrW+ud4+CcA/gBRgMXA50E9rXdLgWDcCNwLExcUNnjt3rtcqUl5eTmhoqNeO11b4Y738sU7gn/WSOrUf/lgvf6vTmDFjVmmthzS2zdKM7+cDXeotJ3nW1dFa78HTMlZKhQKXNwxiz36zgFkAQ4YM0ZmZmc0pf7NkZWXhzeO1Ff5YL3+sE/hnvaRO7Yc/1ssf63Q8zblMvQLoqZRKUUoFAFcBn9bfQSkVo5SqPdbdwGveLaYQQgjhv5oMY621E7gZWADkAO9rrTcopR5WSk3w7JYJbFZK5QJxwGMtVF4hhBDC7zTnMjVa6/nA/AbrHqg3Pw+Y592iCSGEEB1Ds8JYCCGE8GtuNzgroboCajwfVw10zmiV00sYCyGEaB+0hppKqC6HqjLPtNyYVh82ArR+mFYfNvavW65AVx9GV1bgrjiMrqzE7XDgrnSgq6txOxXapXA7FW6XAmsQka9ta5WqSRgLIUQr0E4n7vJy3IcPU/dIqdZHPp5lrTXUPXHazO2AJS8Px6ZNoBSgQBl7KKWMdapuRdPba/dxu3A7HOiqanSVA3dVFdrzcTtq5x24q6rRDge6usoINkcFbkclurLS2O6oPPKdmmoU2iiC0qC0sYwbpdyg3ShcgIuuNVXsflKhtBO0E6VrjP1rq6BqjwPaDW5PkNYGqnabcbvMuF0K7VS4nRrtbPibCfB8jmUKthF5kr/nUyVhLIQQTdBaox0OXIfKcJeX4Tp0CHd5uTEtK8dV1si0bt8y3GVluCsqWrSM0cD2Fj1D05RZo0wak1kb8+b682D1bAfP3xBaGVM3aEyABa1MaMxozLicCixW0IFolGd/deRvELfxu8GtUVYrKsiGKSgIFRSMKTgYc1Cwsc4WhCnIhrIFYbLZjl0XZEPZjO+agoKOzNtsrfazkzAWQviMdrnQ1dVHfdzV1ejqGmO5pvo42+vtU/txucDlRDtdaLcLnC7j+C5n3TwuF/Z9+9g9b169dU60y33UfrXz7ioH7kNluMrKwHlMk+poFjPm0BBMIcGYgwMwBQUQEGnGFB+FOSAKk9WN2eLEZKoB7QS3yzN1orQL3E7Pp3Z9jTHvdnpaivUoGl9uOIZTg2WjEa0arFegzGAyo5XZM28BZQKzBVOAFRVoxWQNQNmsqIAATIGBqMBAVKDNCDdbkPGxBIIlEMwBDaaBYAnwTAMhIBQCQz3TMGNqObZ12pGeM5YwFkKckK6uxl1Rgbuy0phW1E4999yOWle732HcFRXo2vWVR7brysq6QMXlaroAzWU2o8ymuqkymcFsMubNnnmTiYDqKmqKC1AmdWS7yYQyKeP7VoUKVCiTBWUNwxxox2TVmK0uTBYnZnM1JpMDs6rERDkmXYZZHzJafw1Dso4CWzjY7BAQVi+gwozAOuZj9exjbXy7pcG+5gDWbsih/4DBR75jshjzJiuYLZ6pZ//620xmTlBw0UokjIVow7TW6MpKXGXluMvLjEujnnlXWRnu8sPGJdDD5eiaGnSN02jZOWvA6WklOp2eZc987fa6fRtsdzo933US63Cw6WQC02LBFGxcIjQFBdXNm6OjsHZJMpZtQaiAAONjtRqXMpUbpZwoVYPS1Z5PFUpXotwOTO4KlKsC5Sz3fMpQzkPGx6RRplbIk0C7EaZ1n4QGy8f5BEUYAWxq2dfHF+8Ngu7ntOg5RMuRMBbiNGmnE11djTp8mJqCgqYvsTqqPGFaXhekdfPl5bjKjXlXeTnu8vJmtR5NwcGogACwWlAWK8psRlksYDEbyxaL0Tq0WlBWq3Fp0WIxls0WY7vFDJZ637da2L1vP936pBrHrxeupkArJqvCZHFjMjkxmV2YTE6UrjzSu7WqHKrLjGlVGVTvO9IDtrIEHKVwqBR0E/ULCAVbhBFsdjsEdasXdp71tnCj1adMxsdk9syb6y0bl2PXrMtmwMBBR29XJiMsG37HEmicIzDMWBaihUgYC7+mXS7jEunhw7gPVxhTzyVW9+ETTWsvwzo8wWoErLvm2HuVuN0AxAJbT7aAFgvm0FBMoaGYwsIwh4ZiTUjAFBqCOTTMWBfm2R7qmQ8LwxQSemS9xY2qKoEaB7iqjGcjnVXgqj7ycXrWH7W9sX3LwVk7X4UlJJ9o60YoL4eisiNB66puXv3MAUfuCdZOg6Mh6owjrcZjgrW2RRkJgeHGJVYvKtmtoNtZXj2mEKdLwli0eW6HA1dJCa7i4rqps6TEM++ZlpYYl2xr71t6QldXVjb7PMpmO9LyCwnxtAZtmOzhmAICUNaAI5dX6z6eDi0BAWzduYteaX0b7Hdke926wEBMoaGYw8JQNpvxaEmtmko4fBAqCj2fIs/0Z2O6vxC2119faHT0OR0mS4P7kbX3KgOx1jjB0hmCY+p1uAk1LrsGhjWyrulOOUKIY0kYi1ajtUZXVGA6WEhl9nojREuKjwRqiRG2zuJiXCWldcGrHY7jHtMUGoo5MhKz3Y4pNBRrRERdkNZN68/Xn4bU2xYUZFy2PQ3ZWVlE1u/56aw6OjQr86C8EArqrWsYujXHe/xFGS3F4GgIiYGoFEgabCwHRxvbrEGeQA2s18EnsEFnoMB6HYA8605w+XV1B+rNKoQvSRiL0+KuqsJVVISzqMiYFhbiKizCVVyEs7AIZ1EhrqJiXIWFOIuK0A4HnYAdDQ+kFObwcMwREZgjI7HGxWFLTTWWIyIwRxpTS2TkkXV2u3GftDU4q6GyYYgWHRW26flbIfehI9uqy49/vEA7BEcZQRoaD7FpR5Yb+wRFyD1LIfyY34RxSZXb10XwC9rpNC4D14VrEa6iQs/UE7qeYHUVFRkdjBqhrFbM0dFYoqIwR0cT2D0Fc1Q0lugotuwvIG3EmZ5QjTSCNjzc6GDUkpxVRqeh2s5DjlJwlBifY9Z55msDt7rs+McNCIPgKKyuAIjqBjG9PCHqCdegBiEbFCmXb4UQR/GLMH5z+U7uX1TJyJEOYsNab8SU9kC73bhKS4+0WuumxUartUHIukpLGz+Q2Yw5MtITrlEEpadjjorCEh1lTKOi6sLWHB2NKSTk6Huh9WRnZRF2upc+ndVQvg9K8+FQPpTtg8riY8O0LmRLwHn8y90AWGxHdyQKjjGCtS5MG049QesJVrmkK4Q4VX4Rxr3jwgBYn1/K2NSOE8buigqqd+ygavt2anbvNlqvta3WwkLj3mtx8XEfjTFHRBit18hIAnv2xHLmcMyRRthaoqIxR0ViiY7GHBVlXBJu4eck6zir4NCeeh9P4NbN74HyAo4ZXkiZj32+Myz+2J66QZGN9961dpx/O0KItsUvwjgtIRwFZOcdYmxqnK+L411uN9V5eVRv30719u1Ubd9O9fYdVO/YgXPfvqN2NYWG1gWpNbkrQQMG1AvW2lZsNJaoSMyRkafdYemUVFcQVLEXti8+Olxr50vzoeLgsd8LtIM9EcITID4dwj3z4YnGpzZ0ZSQhIUQ75BdhHBJooXOIIju/xNdFOWWu0lJP2O6oC97qHduJ3b6Dn+uNiWsKCyMgJYWQ4cMISEkhoFs3Y9q1K6agIN9VoKocyvdD2V7jknHZPuMycplnXfl+Y13VIYYD/Fjvu0GRR8I1YeCRgK0L287GozJCCOGn/CKMAbrZzazLO879zjZCa01Nfj5Vmzcf3crdvt24nFzLYiGgSxcCUlIo6taNM0aPJjAlhYCUFMxRUce9F9sCBTZGTKoL2f2egN3XIHD3Nd5z2BxotFjD4iG2D3QfA2Fx5OSX0mfYuCNBGxDSOvURQog2yn/CONzE0j1V7D/kIC68bdz7c5UfxrF+PZVr19Z9XIWFddvNnWIITO5G2LnnGq3blG5GSzcpCWW1AvBzw2dXva2mEkp2QfEOKN5pTEt2HllurBexJQjC4iCsM8T1gx7jjyyHxh0JYFtEo5eN92dl0UfG0BVCiDp+E8YpdqNzUXZeKXF9Wz+MtdtN9bZtRuiuWUvlunVUbdlSN1RiQPfuhJ59NkED+mPr29do5Ya1wqVXt9to1R4VsjuOBG/50fedsQRBZDJEdoPkUcZ92tD4IwEbFm8MUSj3ZoUQwmv8Joy7hpkwKcjOL+Xcvi3fictZXFzX2nWsXUflunV1z9ya7HaCMjIIGz+eoP79CcpIx2y3t2BhquHApnpBu+NI8JbsajCOsDIuD0d2gx7nHgneCM80NFaCVgghWpnfhHGgRdEjNpTsfO/fN9Y1NTg251K5dk1dANfs3GVsNJsJ7N2L8IsvIiijP0H9+xPQLbnlHgNyu6HoZ8hfdeSzL/vowLVFGCEblwapvzwStJHdwJ5kDI0ohBCizfCbMAZIT4xg8ZYDaK1Pu5NTdV4+JR98QMWKFTg2bEBXVQFg6dSJoAH9iZw0iaD+/bGlpWEKDvZG8RsVUFUIOZ8bobtnNeT/BFWePzisIUbv4+E3QcIAiO5hBG9QRIuVRwghhPf5WRiH8+HqPPYfqiLefmr3jSt++omiN2ZT9r//gVIEZWQQOWUKQQOMVq8lPr7lejM7SmHPmnqt3tWMLNsDyzDerBOXBumXQ8IgSBwMnXrLeMVCCOEH/CuMk4wW4bq8EuLt8c3+nnY6Kfu//6Nw9mwca9dhCg8n+tfTiZw6FWvnzi1TWGcV7F8P+auPhO/B3CPbo7pDt1FsqbTT85zJxkAXVh8+RyyEEKLF+FUY9+0cjkkZw2Kel9Z0GLsOHaLkg3kUvf0Wzj17sSZ3Je7++4i45BJMIS3w7OuBXFj3HmxbdPR93pBYo6WbfiUkDjIuPQdHAZCflUXPLsO8XxYhhBBthl+FcVCAmV5xYaxrohNX9a5dFM15k5KPPkJXVBA8bBjx991HaGam9ztelR+A9fOMEN7zEygTdB0BZ/7uyOVme5L0YBZCiA7Mr8IYID3RzqLNBcd04tJaU7lyJYVvzKZ84UKwWLD/4hdETbsWW9++3i1EdQVsnm8E8NZvQLugc384/3Hod7kxQIYQQgjh4X9hnGTng1V57C11kBARhK6u5tBXX1H0xmwcGzdijogg+rc3EvmrX2GNjfXeid0u2PEdrHsfNn5qjFwVngSjZkDGVRCb6r1zCSGE8Cv+F8aJxuAa63N2EbA2i+J33sFZUEBA9+7E/+Uv2Cdc7N0XKuzfYLSA130AZXuM0anSJhoBnDwKWuu1g0IIIdqtZoWxUuoC4HnADLyqtX6iwfauwGwgwrPPTK31fC+XtVnOcBRyy9oPSfhiFQdqqgkZOZLOjz5CyFlnee9+8KG9xn3gte/B/mzjsaMe58L5j0HvC6XXsxBCiJPSZBgrpczAS8B4IA9YoZT6VGu9sd5u9wHva61fVkr1BeYD3VqgvI3SWhOQk8Oud9/l8LeLGW+2kN13JJc+9idsvXp55yRV5ZDzmdEK3v4taLfR+erCp6DfZRAS453zCCGE6HCa0zIeBmzVWm8DUErNBSYC9cNYA+GeeTuwx5uFbErxm28S+fwLOKKjibn5Zv4RmsFnux1c1bPn6R3Y5YTtWUYLeNPnUFNhjHB19p8h40qIOc3jCyGEEDQvjBOB3fWW88B4P3w9DwH/p5S6BQgBzvVK6Zop/MILyc3LY9if/oQpMJAey3dStGk9+SWVJEWe4lCVFUXw6rnGONA2O2RMNj5dz5THkIQQQniV0lqfeAelrgAu0Fpf71m+Bhiutb653j63e471d6XUCOA/QD+ttbvBsW4EbgSIi4sbPHfuXK9VpLy8nNDQUAC2lbp4eJmDPwwIZGj8qfVRS815nrj9WeT0uY0DnUagTVavlfVk1K+Xv/DHOoF/1kvq1H74Y738rU5jxoxZpbUe0uhGrfUJP8AIYEG95buBuxvsswHoUm95GxB7ouMOHjxYe9OiRYvq5iurnfqMu7/QT36Zc2oH2/I/rR8M1/rrv3incKehfr38hT/WSWv/rJfUqf3wx3r5W52Alfo4mdic7sUrgJ5KqRSlVABwFfBpg312AeMAlFJ9ABtw4GT+YvAmm9VM7/iwU3udYlU5fHYbRPeE0Xd6v3BCCCFEA02GsdbaCdwMLAByMHpNb1BKPayUmuDZ7U/ADUqptcC7wHWevwJ8JiPJTnZ+KSddjIWPQOlumPgPsJ7am5+EEEKIk9GsG6raeGZ4foN1D9Sb3wiM8m7RTk+/RDvv/ribvOJKukQ1sxPXrh/gh3/BsBuMjlpCCCFEK/Db4aEyEmtfp9jMS9U1Dvj0ZghPhHEPNL2/EEII4SV+G8a94kOxmlXz7xt/97TxPuGLn4PAsJYtnBBCCFGP34ZxoMVManw42fklTe+8bz0sedYYT7rn+JYvnBBCCFGP34YxGPeNs/Oa6MTlchqXp20RcMHjrVc4IYQQwsOvwzgjyc4hh5NdRRXH32n5P2HPT/CLpyA4qvUKJ4QQQnj4dRjXvk7xuPeNC3+GRY9B719C2qWtWDIhhBDiCL8O415xYQSYTWQ31qNaa/jsj2AOgF8+LeNNCyGE8JlTG7i5nQiwmOjTOazxx5tWz4Yd38HFz0N4QusXTgghhPDw65YxGJ241u8pxe2u14nr0B74v/uh29kwaJrvCieEEELQAcI4I8lOmcPJztpOXFrD57eDq8ZoFcvlaSGEED7m92Gc7hmJq64T14aPIPdLGHsvRJ/hw5IJIYQQBr8P455xoQRYTGTnlUBFEcy/ExIGwvDf+bpoQgghBODnHbgArGYTfTuHG524vnoeHCUw4b9g9vuqCyGEaCf8vmUMxvPGkXu+hXVz4azbIb6fr4skhBBC1OkQzcOB8RaGM4vqyJ4EjP6zr4sjhBBCHKVDtIwzd/+TzhSxPP0vYAn0dXGEEEKIo/h/GO9cRuSGObypL+DbihRfl0YIIYQ4hn9fpq5xwKe3oOxdWGC9HmdjI3EJIYQQPubfLePFf4PCLXDxc/TsEs+GPaW43Cd4naIQQgjhA/4bxnvXwZLnYMBU6DGO9KQIDle72H6w3NclE0IIIY7in5epXU749GYIjobzHgWOfp1ij9gwX5ZOCCG8qqamhtDQUHJycnxdFK+y2+3tsk42m42kpCSsVmuzv+OfYbzsH7B3LVw5B4KjADijUwhBVjPr8kq5dGCSjwsohBDek5eXR1xcHElJSSg/Gm+/rKyMsLD21XjSWlNYWEheXh4pKc3vNOx/l6kPboWsxyH1Iug7sW61xWwiLSG88XcbCyFEO+ZwOLDb7X4VxO2VUoro6GgcDsdJfc+/wtjths9mgDkQfvn3Yzb3S7SzYc8h6cQlhPA7EsRtx6n8LvwrjFe9Dju/h/Mfg7D4YzZnJNmprHHx8wHpxCWEEKLt8JswDnQchP89CCnnwMCrG92nrhOXXKoWQgivCg0N9XUR2jX/CGOt6ZX7MmgXXPw8HOcSQfdOoQQHmI+821gIIYRoA/wjjNd/SHTRShh7H0Qdv/ea2aTol2CXMBZCiBaiteaOO+6gX79+pKen89577wGwd+9eRo8ezYABA+jXrx/fffcdLpeL6667rm7fZ5991sel9x3/eLSp8wDyE35B4vCbmty1X6Kdd37cidPlxmL2j79FhBCi1l8+28DGPYe8esy+CeE8eHFas/b96KOPWLNmDWvXruXgwYMMHTqU0aNH884773D++edz77334nK5qKioYM2aNeTn57N+/XoASkpKvFru9sQ/0iimB1t6/RZM5iZ3zUiy46hxs1U6cQkhhNctWbKEKVOmYDabiYuL45xzzmHFihUMHTqU119/nYceeojs7GzCwsLo3r0727Zt45ZbbuGrr74iPDzc18X3mWa1jJVSFwDPA2bgVa31Ew22PwuM8SwGA7Fa6whvFtRb+tXrxJUa33F/8UII/9TcFmxrGz16NIsXL+aLL77guuuu4/bbb+faa69l7dq1LFiwgFdeeYX333+f1157zddF9YkmW8ZKKTPwEnAh0BeYopTqW38frfVtWusBWusBwIvARy1RWG/oHhNCiHTiEkKIFnH22Wfz3nvv4XK5OHDgAIsXL2bYsGHs3LmTuLg4brjhBq6//npWr17NwYMHcbvdXH755Tz66KOsXr3a18X3mea0jIcBW7XW2wCUUnOBicDG4+w/BXjQO8XzPpNJkZYonbiEEKIlXHrppSxbtoz+/fujlOJvf/sb8fHxzJ49m6eeegqr1UpoaChz5swhPz+f6dOn43a7AXj88cd9XHrfUVqfeDQqpdQVwAVa6+s9y9cAw7XWNzeybzKwHEjSWrsa2X4jcCNAXFzc4Llz555+DTzKy8ub/Zzbu5uqWLjLycvnBmMxte1Ra06mXu2FP9YJ/LNeUqf2wW63k5KSgtncdL+Z9sTlcrXbOm3dupXS0qMbfWPGjFmltR7S2P7e7k19FTCvsSAG0FrPAmYBDBkyRGdmZnrtxFlZWTT3eKUR+SzYsYaE1MH0TWjb941Ppl7thT/WCfyzXlKn9iEnJwez2dzuXqrQlPb4oohaNpuNgQMHNnv/5vSmzge61FtO8qxrzFXAu80+u4/UjsS1Xi5VCyGEaAOaE8YrgJ5KqRSlVABG4H7acCelVCoQCSzzbhG9r1t0CGGBFtbld9xn2oQQQrQdTYax1toJ3AwsAHKA97XWG5RSDyulJtTb9Spgrm7qJnQbYHTiCic737sPxgshhBCnoln3jLXW84H5DdY90GD5Ie8Vq+VlJEXwxtIdVDvdBFj8Y+wTIYQQ7VOHTaH0RDvVTje5+8t8XRQhhBAdXIcOY5BOXEIIIXyvw4ZxcnQwYTYL6ySMhRCi3XA6nb4uQovosGGslCI90S4tYyGE8JJLLrmEwYMHk5aWxqxZswD46quvGDRoEP3792fcuHGAMfDK9OnTSU9PJyMjgw8//BDgqMFY5s2bx003GW/iu+6667jpppsYPnw4d955Jz/++CMjRoxg4MCBjBw5ks2bNwPGICF//vOf6devHxkZGbz44ossXLiQSy65pO64//vf/7j00ktb5edxMvzjFYqnKD3JzmtLtlPldBFoaZ+jvAghxFG+nAn7sr17zPh0uPCJJnd77bXXiIqKorKykqFDhzJx4kRuuOEGFi9eTEpKCkVFRQA88sgj2O12srONchYXFzd57Ly8PJYuXYrZbObQoUN89913WCwWvv76a+655x4+/PBDZs2axY4dO1izZg0Wi4WioiIiIyP5/e9/z4EDB+jUqROvv/46v/71r0/v59ECOnYYJ9qpcWly95WTnmT3dXGEEKJde+GFF/j4448B2L17N7NmzWL06NGkpKQAEBUVBcDXX39N/eGQIyMjmzz2pEmT6obGLC0tZdq0aWzZsgWlFDU1NXXHvemmm7BYLEed75prruGtt95i+vTpLFu2jDlz5nipxt7TocM4I9F4y2N2fqmEsRDCPzSjBdsSsrKy+Prrr1m2bBnBwcFkZmYyYMAANm3a1OxjKHXkXQEOh+OobSEhIXXz999/P2PGjOHjjz9mx44dTQ5vOn36dC6++GJsNhuTJk2qC+u2pMPeMwboEhWEPchKtozEJYQQp6W0tJTIyEiCg4PZtGkTy5cvx+FwsHjxYrZv3w5Qd5l6/PjxvPTSS3Xfrb1MHRcXR05ODm63u66FfbxzJSYmAvDGG2/UrR8/fjz/+te/6jp51Z4vISGBhIQEHn30UaZPn+69SntRhw7j2k5c8jpFIYQ4PRdccAFOp5M+ffowc+ZMzjzzTDp16sSsWbO47LLL6N+/P5MnTwbgvvvuo7i4mH79+tG/f38WLVoEwBNPPMFFF13EyJEj6dy583HPdeedd3L33XczcODAo3pXX3/99XTt2pWMjAz69+/PO++8U7dt6tSpdOnShTRVXLwAACAASURBVD59+rTQT+D0tL22eitLT7Lz6nfbpBOXEEKchsDAQL788stGt1144YVHLYeGhjJ79uxj9rviiiu44oor6pbLyoxBmeq3fgFGjBhBbm5u3fKjjz4KgMVi4ZlnnuGZZ5455thLlizhhhtuaF5lfKBDt4zhSCeuzftkJC4hhPBHgwcPZt26dVx99dW+LspxScvYMxLXurxSMpIifFwaIYQQ3rZq1SpfF6FJHb5lnBQZRGSwlew8uW8shBDCNzp8GCul6CeduIQQQvhQhw9jgIwkO7n7y3DUuHxdFCGEEB2QhDHGfWOnW7NJOnEJIYTwAQljIN3TcSs7Twb/EEII0fokjIEEu43okADWSScuIYRoFfXf0NTQjh076NevXyuWxvckjJFOXEIIIXyrwz9nXCsjyc4/sw7iqHFhs8pIXEKI9unJH59kU1HzX87QHKlRqdw17K4T7jNz5ky6dOnCH/7wBwAeeughLBYLixYtori4mJqaGh599FEmTpx4Uud2OBz87ne/Y+XKlXUjbI0ZM4YNGzYwffp0qqurcbvdfPjhhyQkJHDllVeSl5eHy+Xi/vvvrxuCs62TMPbol2jH5dZs3HuIQV2bfp2XEEKIIyZPnsytt95aF8bvv/8+CxYsYMaMGYSHh3Pw4EHOPPNMJkyYcNTbmZry0ksvoZQiOzubTZs2cd5555Gbm8srr7zCH//4R6ZOnUp1dTUul4v58+eTkJDAF198ARgvlGgvJIw9MjyvUMzOK5UwFkK0W021YFvKwIEDKSgoYM+ePRw4cIDIyEji4+O57bbbWLx4MSaTifz8fPbv3098fHyzj7tkyRJuueUWAFJTU0lOTiY3N5cRI0bw2GOPkZeXx2WXXUbPnj1JT0/nT3/6E3fddRcXXXQRZ599dktV1+vknrFHfLiNmFDpxCWEEKdq0qRJzJs3j/fee4/Jkyfz9ttvc+DAAVatWsWaNWuIi4s75j3Fp+pXv/oVn376KUFBQfziF79g4cKF9OrVi9WrV5Oens59993Hww8/7JVztQZpGXvUvk5xvXTiEkKIUzJ58mRuuOEGDh48yLfffsv7779PbGwsVquVRYsWsXPnzpM+5tlnn83bb7/N2LFjyc3NZdeuXfTu3Ztt27bRvXt3ZsyYwa5du1i3bh2pqalERUVx9dVXExERwauvvtoCtWwZEsb1pCdF8G3uFiqqnQQHyI9GCCFORlpaGmVlZSQmJtK5c2emTp3KxRdfTHp6OkOGDCE1NfWkj/n73/+e3/3ud6Snp2OxWHjjjTcIDAzk/fff580338RqtRIfH88999zDihUruOOOOzCZTFitVl5++eUWqGXLkMSpJz3RjltDzt5DDE6O8nVxhBCi3cnOzq6bj4mJYdmyZY3uV15eftxjdOvWjfXr11NWVobNZuP1118/Zp+ZM2cyc+bMo9adf/75nH/++adYct+Se8b11HbikvvGQgghWpO0jOuJC7fRKSxQXqcohBCtIDs7m2uuueaodYGBgfzwww8+KpHvSBg3kCEjcQkhRKtIT09nzZo1vi5GmyCXqRtIT7Kz9UA5h6ucvi6KEEKIDqJZYayUukAptVkptVUpNfM4+1yplNqolNqglHrHu8VsPemJdrSGjXsP+booQgghOogmw1gpZQZeAi4E+gJTlFJ9G+zTE7gbGKW1TgNubYGytor0ROnEJYQQonU1p2U8DNiqtd6mta4G5gINR/q+AXhJa10MoLUu8G4xW09suI248EAZ/EMIIUSraU4HrkRgd73lPGB4g316ASilvgfMwENa668aHkgpdSNwI0BcXBxZWVmnUOTGlZeXe+14CTYny3P3kJVV4pXjnQ5v1qut8Mc6gX/WS+rUPtjtdlwuF2VlZb4uSrN17tyZvXv3nnCf9lan+hwOx0n9O/NWb2oL0BPIBJKAxUqpdK31UWmmtZ4FzAIYMmSIzszM9NLpISsrC28db61zC899k8uQEWcRGujbDuferFdb4Y91Av+sl9SpfcjJycFsNhMWFubropyUpspbVlbW4nVyOp1YLN7//7zNZmPgwIHN3r85JcgHutRbTvKsqy8P+EFrXQNsV0rlYoTzimaXpA3JSDI6cW3IL2V492hfF0cIIZpt31//SlWOd99nHNgnlfh77jnhPt58n3F5eTkTJ06ksLAQl8t11PfmzJnD008/jVKKjIwM3nzzTfbv389NN93Etm3bAHj55ZdJSEjgoosuYv369QA8/fTTlJeX89BDD5GZmcmAAQNYsmQJU6ZMoVevXjz66KNUV1cTHR3N22+/TVxcHOXl5dxyyy2sXLkSpRQPPvggpaWlrFu3jueeew6Af//732zcuJFnn332lH++0LwwXgH0VEqlYITwVcCvGuzzCTAFeF0pFYNx2XrbaZXMh/p5OnFlSxgLIUSzePN9xjabjY8//hilFFVVVXXf27hxI48++ihLly4lJiaGoqIiAGbMmME555zDxx9/jMvlory8nOLi4hOeo7q6mpUrVwJQXFzM8uXLUUrx6quv8re//Y2///3vPPLII9jt9rohPouLi7FarTz22GM89dRTWK1WXn/9df71r3+d7o+v6TDWWjuVUjcDCzDuB7+mtd6glHoYWKm1/tSz7Tyl1EbABdyhtS487dL5SKewQDrbbTL4hxCi3WmqBdtSvPk+Y60199xzD1lZWVgslrrvLVy4kEmTJhETEwNAVJTxDoGFCxcyZ84cAMxmM3a7vckwnjx5ct18Xl4ekydPZu/evVRXV5OSkgLA119/zdy5c+v2i4w03nU/duxYPv/8c/r06UNNTQ3p6ekn+dM6VrMulGut5wPzG6x7oN68Bm73fPxCeqJdhsUUQoiTUPs+43379h3zPmOr1Uq3bt2a9T7j2u8tXryYqKioZn+vPovFgtvtrltu+P2QkJC6+VtuuYXbb7+dCRMmkJWVxUMPPXTCY19//fX89a9/JTU1lenTp59UuY5HRuA6jvREO9sOHuaQo8bXRRFCiHZh8uTJzJ07l3nz5jFp0iRKS0tP6X3Gx/ve2LFj+eCDDygsNC681l6mHjduXN3rEl0uF6WlpcTFxVFQUEBhYSFVVVV8/vnnJzxfYmIiALNnz65bP378eF566aW65drW9vDhw9m9ezfvvPMOU6ZMae6P54QkjI8j3fMGpw35MhKXEEI0R2PvM165ciXp6enMmTOn2e8zrv3emWeeedT30tLSuPfeeznnnHPo378/t99uXIx9/vnnWbRoEenp6QwePJiNGzditVp54IEHGDZsGOPHjz/huR966CEmTZrE4MGD6y6BA9x3330UFxfTr18/+vfvz6JFi+q2XXnllYwaNaru0vXpkhdFHEftSFwfrc7jzO5RTXY4EEII4Z33Gdd+r7FHm6ZNm8a0adOOWhcXF8d///vfY44zY8YMZsyYccz6hs//Tpw4sdFe3qGhoUe1lOtbsmQJt91223HrcLKkZXwc0aGB/Pac7nywKo8XF271dXGEEEK0ASUlJfTq1YugoCDGjRvnteNKy/gE7jo/lQNlVTzzv1yiQwOYOjzZ10USQgi/0R7fZxwREUFubq7Xj+sXYay1ZnfV7qZ3PEkmk+LJyzMoqajh/k/WEx0SwAX9Onv9PEIIcbqMh1raF399n/Gp/C784jL12zlv8/S+p1lT4P1fqtVs4qVfDWJAlwhmzF3D8m3t9vFpIYSfstlslJaWtstA9jdaawoLC7HZbCf1Pb9oGU/sMZFXf3qVmd/N5IOLPyAswLtjmQYFmHntuqFMemUZN8xeydzfnklagt2r5xBCiFOVlJTE2rVrT9gpqj1yOBwnHWptgc1mIykp6aS+4xdhHBYQxrSYaTy//3keWf4IT579pNd7P0cEBzDnN8O4/J9LmfbaCj763Ui6Rgd79RxCCHEqrFYr5eXlDBkyxNdF8aqsrKyTetlCe+YXl6kBUgJT+P2A3/Pl9i/5bNtnLXKOzvYg5vxmGE63m2te+4EDZVUtch4hhBAdi9+EMcBv+v2GIXFDeGz5Y+w6tKtFztEjNozXrhtKwaEqrnv9R8pkhC4hhBCnya/C2Gwy8/jZj2MxWbhr8V3UuFomKAd1jeSfVw9i874yfvvmKqqcrhY5jxBCiI7Br8IYID4knr+M/AvrC9fzjzX/aLHzjOkdy1OTMlj6cyG3vbcGl1t6MQohhDg1fhfGAOcmn8sVva7g9fWvs3zv8hY7z6UDk7jvl32Yn72Phz7dII8VCCGEOCV+GcYAdwy5g272btzz3T0UO078XsvTcf3Z3fnt6O68uXwnL3wjw2YKIYQ4eX4bxsHWYP42+m+UVJXwwNIHWrTVOvPCVC4flMSzX+fy1vLmvSJMCCGEqOW3YQyQGpXKbYNvI2t3Fu9tfq/FzqOU4onL0xmbGsv9/13Pl9l7W+xcQggh/I9fhzHA1X2u5qzEs3h65dNsKd7SYuepHTZzUNdI/jh3DUt/Pthi5xJCCOFf/D6MlVI8MuoRQq2h3Ln4ThxOR4udKyjAzH+mDSE5Opgb56xifX5pi51LCCGE//D7MAaICYrh0bMeZWvJVp5Z9UyLnqt22Mxwm4XrXl/BzsLDLXo+IYQQ7V+HCGOAsxLP4pq+1/DupnfJ2p3VoueqP2zmta/9KMNmCiGEOKEOE8YAtw66lT5Rfbj/+/spqCho0XP1iA3jdRk2UwghRDN0qDAOMAfw5OgnqXJVce+Se3Frd4ueb2DXSF72DJt545xVOGpk2EwhhBDH6lBhDJBiT+GuoXexfO9y3tjwRoufL9MzbOaybTJsphBCiMZ1uDAGuKznZYxPHs+Lq19kw8ENLX6+2mEzv1y/jwf+u16GzRRCCHGUDhnGSikeHPEgMcEx3Ln4Tg7XtHyP5+vP7s5vz+nO2z/s4q/zc6h2tuwlciGEEO1HhwxjAHugnSfOfoK88jwe/+HxVjnnzAtS+dXwrvz7u+388oXvWL6tsFXOK4QQom3rsGEMMDhuMDek38B/f/4vX27/ssXPp5Tir5em859pQ6iscXHVrOXc/v4aDpbLo09CCNGRdegwBrip/03079Sfh5c9TH55fqucc1yfOP532zn8PvMMPlu7h3F//5Z3ftiFWzp3CSFEh9SsMFZKXaCU2qyU2qqUmtnI9uuUUgeUUms8n+u9X9SWYTFZeHL0kwDctfgunG5nq5w3KMDMnRek8uUfz6ZP5zDu+Tiby19ZyoY9MoSmEEJ0NE2GsVLKDLwEXAj0BaYopfo2sut7WusBns+rXi5ni0oMTeSBEQ+w9sBa/rXuX6167h6xYbx7w5k8c2V/dhVWcPGLS3j4s42UV7XOHwVCCCF8rzkt42HAVq31Nq11NTAXmNiyxWp9F6ZcyIQzJjBr3SxW7V/VqudWSnHZoCQW/imTKcO68vrS7Yz7exbzs/fKY1BCCNEBNCeME4Hd9ZbzPOsaulwptU4pNU8p1cUrpWtl9wy/h6TQJGZ+N5PSqta/XGwPtvLYpel89LuRxIQG8vu3V/PMqip52YQQQvg51VTLSyl1BXCB1vp6z/I1wHCt9c319okGyrXWVUqp3wKTtdZjGznWjcCNAHFxcYPnzp3rtYqUl5cTGhp62sfZWbWTZ/Y9Q0ZwBr+O+TVKKS+U7uS53JqFu5x8uKUKl1Zc1N3KL7pbsZp8Ux5v8tbvqq3xx3pJndoPf6yXv9VpzJgxq7TWQxrbZmnG9/OB+i3dJM+6Olrr+g/Mvgr8rbEDaa1nAbMAhgwZojMzM5tx+ubJysrCW8dzrXfx7KpnKU4q5rKel3nlmKdiHDDkq4V8XRTBx+v2srYkgEcu6ceoHjE+K5M3ePN31Zb4Y72kTu2HP9bLH+t0PM25TL0C6KmUSlFKBQBXAZ/W30Ep1bne4gQgx3tFbH3XpV3H8M7DeeLHJ8gp9G1VIm0mXvrVIOb8ehgurZn66g/8ce5PFJQ5fFouIYQQ3tNkGGutncDNwAKMkH1fa71BKfWwUmqCZ7cZSqkNSqm1wAzgupYqcGswKRN/PeuvhFhDmDp/KrPWzaLG7dtXII7u1YkFt47mj+N68mX2Psb9/VvmLNshL54QQgg/0KznjLXW87XWvbTWZ2itH/Ose0Br/aln/m6tdZrWur/WeozWelNLFro1xAbH8uGEDxnbdSwv/vQiU7+YyuaizT4tk81q5rbxvfjq1rPpnxTBA//dwKX//J51eSU+LZcQQojT0+FH4DqRKFsUT5/zNM9kPsP+iv1c9cVVvLzmZWpcvm0ld+8Uypu/GcYLUwayt9TBxJe+5/5P1rOvVC5dCyFEeyRh3Azjk8fzycRPOC/5PP659p9M+WKKz+8lK6WY0D+Bb/50DtNGdOPtH3Zy1pMLuf29NTKKlxBCtDMSxs0UaYvkydFP8sKYFyh0FDLliym8+NOLVLuqfVqucJuVhyak8e0dY7hmRDILNuzjly8sYcqs5XyTs1/GuxZCiHZAwvgkjek6hk8mfsIvu/+SWetmMfnzyWw4uMHXxaJLVDAPXpzG0rvHcc8vUtlReJjfzF7Juc9+y9s/7KSy2uXrIgohhDgOCeNTYA+089hZj/HSuJc4VH2IqfOn8tyq56hy+f5ViPYgKzeOPoPFd47h+asGEBJg4d6P1zPyiW/4+/9tlkeihBCiDZIwPg2jk0bz8cSPmXDGBP6z/j9c+dmVrD2w1tfFAsBqNjFxQCKf3jyK9387giHdovjHoq2c9cQi7vhgLZv2HfJ1EYUQQnhIGJ+m8IBwHh71MK+c+woVzgqu/fJanl7xNA5n22iBKqUYlhLFv68dwsI/ZXLVsC58vm4vFzz3Hdf85we+zT0gL6MQQggfkzD2klGJo/h4wsdc3vNyZm+czaTPJvFTwU++LtZRUmJCeHhiP5bdPZY7zu/N5n1lTHvtR857djHvrdiFo0buKwshhC9IGHtRaEAoD4x4gH+f929q3DVM+3IaT/74JBU1Fb4u2lEiggP4w5geLLlrLM9c2R+L2cRdH2Zz1pMLef7rLRSW+/7etxBCdCQSxi3gzM5n8tGEj5jcezJv5bzFFZ9dwYp9K3xdrGMEWExcNiiJ+TPO4p3rh5ORFMGzX+cy4omF3P3ROrYWlPm6iEII0SE0561N4hQEW4O598x7Oa/beTy49EF+veDXXNX7Km4bfBvB1mBfF+8oSilG9ohhZI8YthaU8Z8lO/hodR7v/ribQV0jGNcnjrGpsaTGh/nslZJCCOHPpGXcwobGD2XexfO4us/VvLf5PS779DIW5y1us52mesSG8fhl6SydadxXdro1Ty3YzIXPf8eoJxZy3yfZLNy0X+4vCyGEF0nLuBUEW4O5a9hdnNftPO7//n7+8M0f6BXZi2v7XsuFKRcSYA7wdRGPER0ayB/G9OAPY3pQcMjBos0FfJNTwEer83lr+S5sVhOjzohhbJ9YxqbG0tke5OsiCyFEuyVh3IoGxg7kowkf8cW2L5izcQ73fX8fz69+nimpU7iy95XYA+2+LmKjYsNtTB7alclDu1LldPHDtiK+ydnPN5sK+GZTAQB9O4czzhPM/ZMiMJnkcrYQQjSXhHErCzAHcGnPS7mkxyUs27OMORvn8MJPL/Dv7H8z4YwJXNP3GpLDk31dzOMKtJgZ3asTo3t14qEJmi0F5SzcVMDCnAJeWrSVFxduJSY0gMzesYxLjeWsnjGE2ay+LrYQQrRpEsY+opRiZOJIRiaOZEvxFt7c+CYfbfmI9ze/T2aXTKalTWNQ7CBfF/OElFL0igujV1wYN51zBsWHq1m85QDf5BTwfxv2MW9VHlazMejI2NQ4xqXG0i0mxNfFFkKINkfCuA3oGdmTh0c9zIxBM5i7aS7vbX6PRbsXkRadxhA1hFHuUVhNbb91GRkSwMQBiUwckIjT5WbVzmIWei5lP/L5Rh75fCPdO4XQK6SKgC4HGdotCqtZ+hAKIYSEcRsSExTDzQNv5jfpv+Gznz/jzY1vMvvQbBZ8tICr+1zNZT0vIywgzNfFbBaL2cTw7tEM7x7N3b/ow87Cw0Yw5xTwv58P89W/fyDMZmF0r06MS40ls3csUSFtryObEEK0BgnjNijIEsSVva/kil5X8M8v/8lq82qeXvk0L699mct6XsbUPlNJDE30dTFPSnJ0CNNHpTB9VApffr0IFd+HhZv2s3DTAb5YtxelYGAX45nmMb1j6dNZnmkWQnQcEsZtmEmZ6Bfcj5szb2Zj4UbmbJzDuznv8nbO24xPHs+0vtNI75Tu62KetCCLIrNfPBf0i8ft1qzfU8o3OQUs2lzAUws289SCzXS22xiTanQCG3lGDEEBZl8XWwghWoyEcTvRN7ovT5z9BLcOupV3Nr3DvM3zWLBjAQNjBzKt7zQyu2RiNrW/wDKZFBlJEWQkRXDb+F51zzQv3FTAJz/l884Puwi0mBh5RjRjPSOBJUbIM81CCP8iYdzOxIfEc/vg2/ltxm/5ZOsnvLnxTW7NupXE0ETGdBnDyISRDI4b3OaG3Gyuxp5pXrjJCOdFm9dzP5AaH8bY1FjG9YllQJdIzPJMsxCinZMwbqdCrCFM7TOVq3pfxcLdC/kw90M+yP2At3LewmqyMihuEKMSRjEyYSS9Inu1y/uv9Z9pfvDivvx84DALN+3nm5wC/rV4G//M+pnIYCuZvWMZkxrLmd2jiA2z+brYQghx0iSM2zmzycz45PGMTx6Pw+lgdcFqluYv5fs93/PMqmd4ZtUzxATFMKLzCEYmjmRE5xFEB0X7utgnTSlFj9hQesSGcuPoMyitrGFx7gEWbioga3MBH/+UD0C36GCGdItiWLcohnSLJCUmpF3+ISKE6FgkjP2IzWJjZMJIRiaM5M/8mYKKApbuWcrSPUtZkr+Ez7Z9BkCfqD51+w2MHYjV3PafYW7IHmTl4v4JXNw/AZdbsy6vhBU7ilixo5hvcvYzb1UeADGhAQxJNoJ5WEoUfTuHY5Fnm4UQbYyEsR+LDY7lkh6XcEmPS3BrNzmFOSzdY7SaZ2+YzX/W/4cgSxBD44cyMmEkoxJGkRye3O5akmaTYmDXSAZ2jeTG0eB2a7YdLGfFjmJWbC9ixc4ivtqwD4DgADODukYa4dwtigFdIwgOkP8MhBC+Jf8X6iBMykRaTBppMWnckHEDh2sO8+PeH/l+z/cs27OMxXmLAUgISWBkohHMwzoPIzwg3MclP3kmk6JHbBg9YsOYMqwrAPtKHZ6Ws9F6fv6bLWgNFpMiLdHO0ORIhnSLYmi3SKJDA31cAyFERyNh3EGFWEMY03UMY7qOAWB32W6W7VnG9/nf8+X2L5mXOw+zMpMalcrA2IF1n07BnXxc8lMTb7fVXdYGOOSoYdXOYlbuKGLF9mLmLN/Jq0u2A9C9U4jnnrMRzl2jgtvd1QIhRPsiYSwA6BLWhS69u3Bl7yupcdeQfSCbpXuWsrpgNfNy5/FWzlsAJIYmMih2EANiBzAodhDdI7pjUu3vHmy4zcqY3rGM6R0LQJXTRXZeKSt2GAE9P3svc1fsBiAmNJDByREMTo5kcHIk/RLtBFra3zPdQoi2S8JYHKP20ahBccZbo2rcNWwq3MTqgtWsKVjD0j1L6zqDhQWEMaDTAAbFDWJApwH0i+mHzdL+Hi8KtJgZ4mkNwxm43ZrcgjJW7Syu+yzYsB+AALOJ9CQ7g5MjGdTVCOhOYXJpWwhx6poVxkqpC4DnATPwqtb6iePsdzkwDxiqtV7ptVIKn7KarKR3Sie9UzrT0qahtWZ32W5+Kvip7vNd/ncAWEwW+kb3ZWCngQyMMy5tR9mifFyDk2cyKVLjw0mND2fqcOP90gfKqli960g4v/H9DmYt3gZAcnQwiYHV5AftZHByJD1jw2QwEiFEszUZxkopM/ASMB7IA1YopT7VWm9ssF8Y8Efgh5YoqGg7lFJ0De9K1/CuTOwxEYASRwlrDqzhp4KfWFOwhnc2vcPsjbMBSA5PPuq+s9bal8U/ZZ3CAjk/LZ7z0+IB49L2+vzSunBetmU/Sz9eD0BYoIWByZEM9rScB3SNIDRQLkQJIRrXnP87DAO2aq23ASil5gITgY0N9nsEeBK4w6slFO1ChC2CzC6ZZHbJBKDaVc3Gwo2sLljNTwU/kbU7i0+2fgJAsCmY/v/Xn77RfUmLSaNvdF8SQhLaXSepQIuZwclRDE42Wv6LFi2ie8Ywo2PYzmJW7yzmuW9y0RpMClLjw41g7hJB7/gwesSGYrPKvWchRPPCOBHYXW85Dxhefwel1CCgi9b6C6WUhLEgwBzAgNgBDIgdAIDWmu2HtrOmYA1frfuKkqoSZm+YjVM7AYgIjCAt2gjm2ml8SHy7CmilFMnRISRHh3DZoCTA6LW9ZldJXTh/tDqPN5fvBIyA7hoVTM+4MHrHhdEzLpRecWF07xQiHcSE6GBUU5cMlVJXABdora/3LF8DDNda3+xZNgELgeu01juUUlnAnxu7Z6yUuhG4ESAuLm7w3LlzvVaR8vJyQkNDvXa8tsIf61Vbpxpdw57qPeyq3sXu6t3sqtrF3pq9uHEDEGoKpUtAF7oGdjWmAV2JMEe02YBuzu/KrTV7D2vyy93kl7mNabmb/RUat+c/RZOCuGBFYqjJ+IQZ07hghaWV70P7878/f+OP9fK3Oo0ZM2aV1npIY9ua0zLOB7rUW07yrKsVBvQDsjz/k4wHPlVKTWgYyFrrWcAsgCFDhujMzMzm1qFJWVlZePN4bYU/1utEdXI4HeQW57KxcCMbCjewsXAjX5d8jUu7AIiyRR3Vek6LTiM2OLZNBPTp/K6qnC62HThM7v4ytuwvJ3d/Gbn7y1i1rYLav5etZkVKTAi94sI8H6MlnRwd0mKdxTrav7/2zB/r5Y91Op7mhPEKoKdSKgUjhK8CflW7UWtdCsTULp+oZSxEU2wWGxmdMsjolFG3zuF0sLl4MxsObqgL6aV7luLWRgs6j67HywAAE6VJREFU2hZN3+i+dLN3IzksmS7hXega1pXOIZ3bzTueAy1m+nQOp0/no0c8c9S42FpQG87lbNn//+2da2wc13XHf2dn9s3dJXdJkbRMUlIsO0kdyHEeTRPXNpIgddIiaYOiSFqgaVMgMBoDDdqiCJDCCPItKdoPLYwUaWskLfJwH0lrBIqdtKnaOIVsK7Zly5Gsl0nqRYncJXeX+37cfpjhaJdcPmRJ3F3q/IDB3Ln3zvIe3pn9771z7pk8R88v8f2XL3l1AraPN40MsG84yt7hKHuGo+wdjrAnFSUZDfTEDxVFUTZmUzE2xtRF5BHgaZylTY8bY14VkS8BR4wxT97sRiq3NiE7xIGRAxwYOeDlFWtFTi6e9EbPxzPHeX7uecqNslfH9tncPnA7E7EJpuJTTMQmHC/w2CTjA+P4fb3/goyQ3+Lu3Qnu3p1oyy9U6p5In3L3xy5meerVORrNq4+e4iHbE+g9KUesV44T4d63X1FuFba01sIYcxA4uCrv0XXqPnj9zVKUjYn4I20OYgBN02S+OM9sfpZz+XPM5ma99JHLRyjVS15dSyxuG7jNE+fJmLNUayI2we0Dt/f8m6yiQZsDE4McmBhsy6/Wm5xfLDKdLnB2vsB0usD0QpEj04s8efQirS4iyWiAPamIM5JORdk7clWwo7oMS1G2Fb3jlB2DT3yMRkcZjY7yrrF3tZUZY0iX055Az+Zcwc7PcvTKUZZry22fMx4dZzI2yVh0jOHwMKlwiuHwsJMOOemov/felRywfewbGWDfyADvf3N7WbnW4FymyOsLBV5fcIT69YUC/3c6zXdfuNBWdyQWZG8qSrBW4VjzFJOpKFPJCFOpCIORwDZapCi3BirGyi2BiHhiuhLmcwVjDIuVxTaBns0525mlM2TKGW8JVishK9Qm0pVMheMvHW/LWxHyoNX9cJkhv8X+0Rj7R2NryorVOjPpItMLBc4uFJh2xfrlhQY/+eHJtrrxkM1UKspkKuIJ9GQyylQqwlg8hE8jjynKNaNirNzyiAjJUJJkKNk27b1C0zTJVrIslBa8LV1Kky6nveOZ3AxzxTmeOfpMx78R88c8kR6JjDAZm2QqPsVkfJKp2BSDocGO520XkYDd0YHs0KFDvPu99zGbKTKTLjKbLjrpTJFXL2R5+tgc9ZZn1AHbx8RQ2BFrV6hXxHoiGdb104qyDirGirIJPvExFBpiKDTE/qH969Y7dOgQ77v/fSyWF9tEe6G00CbcL8+/zNPTT3ve4ADxQLxNnCfjk57TWSKYWPdvbgeRgO3F6V5NvdHkUrbMTLrITKbAbLropos8ezZNodrw6orAeDzEHnd51lvGY7x5LM6dozHCARVp5dZGxVhRbiB+n59dkV3siuzasF61UeX88nlmc7PM5GacfX6GFy6/wMGzBzFcHW0OBgfXiPTK8UCguwERbMvHRDLCRDLCfVdXOALuc/pC1RlRZwreyPrsQoF/PnKOoivUIrAnFeXNYzHuGnME+i3jMSaGIjrlrdwyqBgrShcIWAH2JfaxL7FvTVmlUeFc7uqz65m8I9bPzT3nvbpyhWQo6XmC74rsIhVKkQqn2vaJYKIrjmYiwvBA0H0f9FBbWbNpOLdY5PilPK/N5Tkxl+PEXJ6nXp3zPL4jAattBO0IdUwdyJQdiYqxovQYQSvIHUN3cMfQHWvKSvWSt2xrJjfDbN7ZH750mHQp7UUqa8UWm2QoSSqcIhlOet7gnmC3iPdgcBCf+G66jT7f1TjeD9095uUXq3VOXV7mxFzOE+qnjs3x7eeuhscfT4TcUXTcE+p9I1H81s1vt6LcLFSMFaWPCNth7hy6kzuH7lxTtuJotuJctt7+9OJp0uU09eZaD3FLLIZCQ54417N1XvzZi47jWXjE8xAfiYwQ9UdvuH2RwNr108YYruQrnJjLc+KSM4I+MZfnmdNnqTWcYbTtc0bhQ9EAyaifoUjA2aIBkhG/m+/kpUtNStWGPqdWegoVY0XZIbQ6mt3B2lF1K8YYctXcWrEupcmUM17e+fJ5Xvz5i9SatTWfEbbDnkinwilGwiOMREZIhVKMREa8/GQoeV2jbRFhNB5iNB7igTtHvPxao8nZ+QIn5nKcvJxnPl8hU6ixWKzy84s5MsUq2VKNTu/C+ZP/eYqQ30dyRbBdoV7ZD7mCPhoPsScVYSQW7Lk15crOQsVYUW5BRIREMEEimOj43HqFQ4cO8cADD5Cr5pgvzjNfmve8wlvTpxZPcfjiYfK1/JrPsMTyRtojkRGSoSQD/gHCdpioP+ptEX+EiB3pmGf71n5V+S0fd7lOX+vRaBqypRqZQpXFYpVMocrhF15hZGIvi4UqmUKNpWKVTLHKuUyRTKFKrrx2xiDst5hKOfG+p4YjTCWj7ElFmBqOMq5rq5UbgIqxoigb0ircnZ5jt1Kql7wlXfOleeaL823rs+eL85zInKBYK1KsF9uWd21E0Ao64mxHiPgjnlBH7auiHQ/EiQfiJIIJJx2MkwgkiAfjTKbi3LHL8TwPzp/gwQfXt6PWaLJUdAR8Lldmxg0pOpMucOpKnh+fuEK1cbXdAdvHZDLCHnc99Z7hCFMpR6x3D4ax9Vm2sgVUjBVFuWGE7TATsQkmYhOb1jXGUG6UKdQKFGtFZ18vth2v5HnHdaesWCuSLWe5WL/olbWGNF2vbbFADKtm8fWnvr5GvNeIeCTOPUOD3L9/uG2KutE0jkgvFJhOOyI9nXaWbv30dJpS7aoTne0Tbh8KM+mK85QbVnR8MMRtiTCDEb9OfyuAirGiKF1CRAjbYcJ2GMLX/3mNZoPl2jLZSpZcNUeukiNbzbbtc9UcZy6cQRDOL5/38lpfIrIa2+d6o4eueqOveKAnh5M8MJHi46EUqfAeEoEEi4UG0+miK9COYM+mi7w4s0i+0j4FHvL7GE+EGYuHGB8MMZ4IMZ4IM54IMZZQwb6VUDFWFGVHYPksbzp9Izq9sL7WqDmC7Yp4rpojW8myWF4kXW53ajuzdIZ0Kd3RqU0QhkJDbeI9eUeKt78tRTKYJOhLUC6HKZVD5AoBMnnhUq7CXLbM4TNpLucrba/AhI0Fe2U/GOntt4wpm6NirCjKLY/f8nvLtraCMYZ8Lb/G+9xLu8fHFo6RLqUp1osdP8cWm3gwzuD4IPv3DPKOQIKQL4bPRDGNCLVamFIpSL4UYCnv5/XXbeazNo1m+3PokN9HQAyJ535MyLYI+S1Cfh8hv0XQdtJhf3u+U3Y1HW4r8xG0LQaCNkPRAPGQraPzm4yKsaIoyjUiIt4z572JvZvWL9VLnlBnyhmWKktkK1mWKktt6fPL57z0mpF3xNkiYxCxo0TtOEFfDMsV7sJynXB4gGbTptT0UWj4aDQs6lUfjaZFve6jVhdqdYtqQ6BpYYwNxgJjY4y1Ku2UmaYfW/wMRsJr13C7x1eXhAXc5WJ+BoIq4NeCirGiKMpNJmyH2T2wm90Du7dU3xhDqV7yxHo98V6qLJEtZ1mqzFAL52jYPmrNGtVGlZrUOn7DW7yxR/RVfFzGzxVjY2p+TMamsWBjmrYj6k2/K+Q2NG0EP0ErSNgOEfYHiQZCRP0h4sEwqWiY4YEIu2IRUtEwAcvG9jmbJRa2z8bv8zNdmeZ4+jiWz80Tv5deqev3+b36/Sz+KsaKoig9hog4a6z9EW4buG1L56x+Fm6Mod6sU2vWrgr0RulGbU1epVGh1nD2rVu1UfXShWqZYs3ZyvUK5cYy1UaFarNKvVmlSI1lasw3gYq75a7hn/H9rVULWSGGw8PsiuzyosStBKVZCUIzEh7pWqz2zVAxVhRF2YGICH7Lj9/qvnOXMYZq0xHwYrXMhaUC5xaXObe0zIWlZS5lC1zKFpjLFyhUqiANkCZIk1hISA3YpAZskgMWQxGLRMRHPGIRDQpN06DerLNUWfIC0ZxcPMlPL/6UQq2wpi1+n98T65UQr22C7Yr49UaOu1ZUjBVFUZSbiogQtIIErSDxQJyxAXjH7Z3rZos1zi0Wmc0UOfT8K1iD45zLFJm9WORnS6U2b3O/JeweDDORdAKsjMVDHBgMMjrhhE+NR5o0fTkylQUvgtx8aZ6FohNBbjo7zfNzz5Orrh2qW2IxHh3n4McPbstIWsVYURRF6RkSET+JSIK7dyeIpF/jwQff5pXVG00uZcvMZortW7rIibk8C8uVNbHILZ8wMhBkND7Arvgwo/F7GI+FuGcoxK6pIKPxEENRoS5ZL1LcleIVFkoL1Jq1bZvSVjFWFEVR+gLb8jGRjDCRjPC+DuW1RpOF5QqXcxUu58pcyZW99OV8hXOZIkemMywW164RD1g+dsWDjMYjjMbvYlfsAGOJEMYYHRkriqIoylbxWz43EMrG/uLlWoP5vCvSnliXueKmX5vL85OTCwRsHw8/8KZtabuKsaIoinJLEfJb3gh7I8otccZvNvo6EUVRFEXpQMhvbdvfUjFWFEVRlC6jYqwoiqIoXUbFWFEURVG6jIqxoiiKonQZFWNFURRF6TJbEmMReUhEXhOR0yLy+Q7lD4vIKyLykog8IyJvvfFNVRRFUZSdyaZiLCIW8BjwYeCtwCc7iO23jDFvM8bcA3wF+Ksb3lJFURRF2aFsZWT8buC0MeasMaYKfAf4WGsFY0xrlO0osCo6qKIoiqIo67GVCFy7gXMtx+eBX1xdSUQ+C/wxEADef0NapyiKoii3ADcsHKYx5jHgMRH5beDPgU+triMinwE+4x4ui8hrN+rvA8PAwg38vF5hJ9q1E22CnWmX2tQ/7ES7dppNU+sVbEWMLwATLce3u3nr8R3gq50KjDFfA762hb95zYjIEWPMO2/GZ3eTnWjXTrQJdqZdalP/sBPt2ok2rcdWnhk/D+wXkb0iEgA+ATzZWkFE9rcc/ipw6sY1UVEURVF2NpuOjI0xdRF5BHgasIDHjTGvisiXgCPGmCeBR0Tkg0ANWKTDFLWiKIqiKJ3Z0jNjY8xB4OCqvEdb0n90g9v1Rrgp0989wE60ayfaBDvTLrWpf9iJdu1EmzoixugqJEVRFEXpJhoOU1EURVG6TN+J8RZCcwZF5Am3/FkR2bP9rbw2RGRCRP5bRH4uIq+KyJppfxF5UESybsjRl0Tk0U6f1UuIyHRLmNQjHcpFRP7a7auXReTebrRzq4jIXS3//5dEJCcin1tVpy/6SUQeF5ErInKsJS8pIj8SkVPufmidcz/l1jklIj3jH7KOTX8hIifc6+t7IjK4zrkbXqvdZB27vigiF1qus4+sc+6G35fdYh2bnmixZ1pEXlrn3J7tq+vCGNM3G44D2RlgH05wkaPAW1fV+UPgb930J4Anut3uLdg1DtzrpmPAyQ52PQh8v9ttvUa7poHhDco/AvwAEOA9wLPdbvM12GYBc8BUP/YTcD9wL3CsJe8rwOfd9OeBL3c4LwmcdfdDbnqo2/ZsYNOHANtNf7mTTW7ZhtdqD9r1ReBPNzlv0+/LXrJpVflfAo/2W19dz9ZvI+NNQ3O6x99w0/8KfEBEZBvbeM0YYy4ZY15w03ngOE7ks53Ox4B/NA6HgUERGe92o7bIB4AzxpiZbjfkjWCM+V8gsyq79d75BvDrHU79FeBHxpiMMWYR+BHw0E1r6DXQySZjzA+NMXX38DBOnIS+Yp2+2gpb+b7sChvZ5H5f/xbw7W1tVJfpNzHuFJpztWh5ddybMAuktqV1NwB3Wv3twLMdin9JRI6KyA9E5Be2tWFvDAP8UER+5kZfW81W+rNX+QTrf1n0Wz+tMGqMueSm54DRDnX6uc8+jTMT04nNrtVe5BF3+v3xdR4p9Gtf/TJw2RizXryKfuyrTek3Md7RiMgA8G/A50z7yzcAXsCZEj0A/A3w79vdvjfAfcaYe3He+PVZEbm/2w26EbjBbz4K/EuH4n7spzUYZz5wxyy1EJEvAHXgm+tU6bdr9avAm4B7gEs407o7hU+y8ai43/pqS/SbGG8lNKdXR0RsIAGkt6V114GI+HGE+JvGmO+uLjfG5Iwxy276IOAXkeFtbuY1YYy54O6vAN/DmTZr5VpDrfYKHwZeMMZcXl3Qj/3UwuWVxwTu/kqHOn3XZyLye8CvAb/j/shYwxau1Z7CGHPZGNMwxjSBv6Nze/uxr2zg48AT69Xpt77aKv0mxpuG5nSPVzw8fxP48Xo3YK/gPiP5B+C4Mabju6BFZGzl2beIvBun73r2R4aIREUktpLGcaQ5tqrak8Dvul7V7wGyLdOkvcy6v9z7rZ9W0XrvfAr4jw51ngY+JCJD7tToh9y8nkREHgL+DPioMaa4Tp2tXKs9xSrfit+gc3u38n3Za3wQOGGMOd+psB/7ast024PsWjccD9yTOF6CX3DzvoRzswGEcKYPTwPPAfu63eYt2HQfzpTgy8BL7vYR4GHgYbfOI8CrOB6Rh4H3drvdm9i0z23rUbfdK33VapMAj7l9+Qrwzm63ewt2RXHENdGS13f9hPNj4hJOCNvzwB/g+Fb8F05s+f8Ekm7ddwJ/33Lup9376zTw+922ZRObTuM8N125r1ZWWtwGHNzoWu2VbR27/sm9Z17GEdjx1Xa5x2u+L3th62STm//1lXuppW7f9NX1bBqBS1EURVG6TL9NUyuKoijKjkPFWFEURVG6jIqxoiiKonQZFWNFURRF6TIqxoqiKIrSZVSMFUVRFKXLqBgriqIoSpdRMVYURVGULvP/Zh+w/r4LeRYAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "-Hr3xulQqKW6", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "ff447a38-ea07-4c45-816f-bcab990818e0" }, "source": [ "y_pred = model.predict(X_test)\n", "y_pred = y_pred.argmax(axis=1)\n", "print(classification_report(y_test, y_pred,\n", " target_names=class_names))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " T-shirt/top 0.82 0.83 0.83 1000\n", " Trouser 0.98 0.96 0.97 1000\n", " Pullover 0.79 0.77 0.78 1000\n", " Dress 0.86 0.89 0.87 1000\n", " Coat 0.78 0.80 0.79 1000\n", " Sandal 0.94 0.94 0.94 1000\n", " Shirt 0.69 0.65 0.67 1000\n", " Sneaker 0.92 0.94 0.93 1000\n", " Bag 0.95 0.96 0.96 1000\n", " Ankle boot 0.94 0.94 0.94 1000\n", "\n", " accuracy 0.87 10000\n", " macro avg 0.87 0.87 0.87 10000\n", "weighted avg 0.87 0.87 0.87 10000\n", "\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "CmYKyJzsrEMT" }, "source": [ "## Changing the position of Batchnorm" ] }, { "cell_type": "markdown", "metadata": { "id": "0YED2_GiqKrS" }, "source": [ "Sometimes applying BN before the activation function works better (there's a debate on this topic). Moreover, the layer before a `BatchNormalization` layer does not need to have bias terms, since the `BatchNormalization` layer already includes it, hence it would be a waste of parameters, so you can set `use_bias=False` when creating those layers\n", "\n", "Source: https://www.google.co.in/books/edition/Hands_On_Machine_Learning_with_Scikit_Le/bRpYDgAAQBAJ" ] }, { "cell_type": "code", "metadata": { "id": "72dxtaVBqLOg" }, "source": [ "model = tf.keras.models.Sequential([\n", " tf.keras.layers.Flatten(input_shape=[28, 28]),\n", " tf.keras.layers.BatchNormalization(),\n", "\n", " tf.keras.layers.Dense(300, use_bias=False),\n", " tf.keras.layers.BatchNormalization(),\n", " tf.keras.layers.Activation(\"relu\"),\n", "\n", " tf.keras.layers.Dense(100, use_bias=False),\n", " tf.keras.layers.BatchNormalization(),\n", " tf.keras.layers.Activation(\"relu\"),\n", "\n", " tf.keras.layers.Dense(10, activation=\"softmax\")\n", "])" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "OAz4wwcuqyBk", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "9c7523e0-fc76-4061-8c25-0c4e1866f537" }, "source": [ "model.compile(loss=\"sparse_categorical_crossentropy\", \n", " optimizer=tf.keras.optimizers.SGD(1e-3),\n", " metrics=[\"accuracy\"])\n", "model.summary()" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Model: \"sequential_4\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "flatten_4 (Flatten) (None, 784) 0 \n", "_________________________________________________________________\n", "batch_normalization_3 (Batch (None, 784) 3136 \n", "_________________________________________________________________\n", "dense_12 (Dense) (None, 300) 235200 \n", "_________________________________________________________________\n", "batch_normalization_4 (Batch (None, 300) 1200 \n", "_________________________________________________________________\n", "activation (Activation) (None, 300) 0 \n", "_________________________________________________________________\n", "dense_13 (Dense) (None, 100) 30000 \n", "_________________________________________________________________\n", "batch_normalization_5 (Batch (None, 100) 400 \n", "_________________________________________________________________\n", "activation_1 (Activation) (None, 100) 0 \n", "_________________________________________________________________\n", "dense_14 (Dense) (None, 10) 1010 \n", "=================================================================\n", "Total params: 270,946\n", "Trainable params: 268,578\n", "Non-trainable params: 2,368\n", "_________________________________________________________________\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "VYdCb1OSq12r", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "af2d2e2a-5b09-4278-f509-9fda2f50f569" }, "source": [ "history = model.fit(X_train, y_train, epochs=20,\n", " validation_split=0.1, batch_size=32)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Epoch 1/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 1.0356 - accuracy: 0.6697 - val_loss: 0.6713 - val_accuracy: 0.7822\n", "Epoch 2/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.6719 - accuracy: 0.7804 - val_loss: 0.5560 - val_accuracy: 0.8118\n", "Epoch 3/20\n", "1688/1688 [==============================] - 5s 3ms/step - loss: 0.5891 - accuracy: 0.8045 - val_loss: 0.5056 - val_accuracy: 0.8243\n", "Epoch 4/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.5418 - accuracy: 0.8173 - val_loss: 0.4735 - val_accuracy: 0.8337\n", "Epoch 5/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.5118 - accuracy: 0.8240 - val_loss: 0.4513 - val_accuracy: 0.8415\n", "Epoch 6/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.4900 - accuracy: 0.8311 - val_loss: 0.4353 - val_accuracy: 0.8468\n", "Epoch 7/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.4720 - accuracy: 0.8366 - val_loss: 0.4223 - val_accuracy: 0.8502\n", "Epoch 8/20\n", "1688/1688 [==============================] - 5s 3ms/step - loss: 0.4577 - accuracy: 0.8421 - val_loss: 0.4119 - val_accuracy: 0.8518\n", "Epoch 9/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.4452 - accuracy: 0.8451 - val_loss: 0.4045 - val_accuracy: 0.8552\n", "Epoch 10/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.4330 - accuracy: 0.8493 - val_loss: 0.3960 - val_accuracy: 0.8577\n", "Epoch 11/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.4258 - accuracy: 0.8519 - val_loss: 0.3901 - val_accuracy: 0.8590\n", "Epoch 12/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.4153 - accuracy: 0.8555 - val_loss: 0.3841 - val_accuracy: 0.8613\n", "Epoch 13/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.4061 - accuracy: 0.8577 - val_loss: 0.3799 - val_accuracy: 0.8610\n", "Epoch 14/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.4014 - accuracy: 0.8591 - val_loss: 0.3759 - val_accuracy: 0.8645\n", "Epoch 15/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3947 - accuracy: 0.8623 - val_loss: 0.3698 - val_accuracy: 0.8642\n", "Epoch 16/20\n", "1688/1688 [==============================] - 5s 3ms/step - loss: 0.3868 - accuracy: 0.8636 - val_loss: 0.3665 - val_accuracy: 0.8637\n", "Epoch 17/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3819 - accuracy: 0.8650 - val_loss: 0.3647 - val_accuracy: 0.8653\n", "Epoch 18/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3792 - accuracy: 0.8674 - val_loss: 0.3607 - val_accuracy: 0.8672\n", "Epoch 19/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3710 - accuracy: 0.8698 - val_loss: 0.3586 - val_accuracy: 0.8687\n", "Epoch 20/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3676 - accuracy: 0.8706 - val_loss: 0.3544 - val_accuracy: 0.8712\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "-ajYkTQsq4gX", "colab": { "base_uri": "https://localhost:8080/", "height": 324 }, "outputId": "f024e4ed-c778-4d7b-bc3a-749e29d446c9" }, "source": [ "pd.DataFrame(history.history).plot(figsize=(8, 5))\n", "plt.grid(True)\n", "plt.gca().set_ylim(0.3, 1);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeMAAAEzCAYAAAACSWsXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3hUVf7H8feZlsnMpCcEQgoJRVoIvUoTUbBhQ8S1oeDaUNddXd1Vl7Xsuvb1JxZUUNxVRBRlLbiygogiSA8djEBCDRBC+iQz5/fHnYTQE5hkksn39TzzzG1z7zn4yIdz7rnnKq01QgghhAgcU6ALIIQQQjR1EsZCCCFEgEkYCyGEEAEmYSyEEEIEmISxEEIIEWASxkIIIUSAnTaMlVJTlVL7lFJrT7JfKaVeVkptVUqtUUp1938xhRBCiOBVk5bxO8CIU+wfCbT1fW4DXjv7YgkhhBBNx2nDWGu9EDh4ikNGAdO14ScgUinVwl8FFEIIIYKdP+4ZtwSyq63n+LYJIYQQogYs9XkxpdRtGF3ZhIaG9khKSvLbub1eL+XaxO4iL80cCodF+e3cgeT1ejGZgmucXTDWCYKzXlKnxiMY6xVsddq8efN+rXXcifb5I4x3AtVTNdG37Tha6ynAFICePXvqZcuW+eHyhgULFtClV3+6P/ENj1zcgfED0/x27kBasGABQ4YMCXQx/CoY6wTBWS+pU+MRjPUKtjoppbafbJ8//skxB7jRN6q6L5Cvtd7th/PWWrTTRqTDyi+5RYG4vBBCCHFGTtsyVkp9AAwBYpVSOcBfACuA1vp14EvgImArUAyMq6vC1kRarJNf9xcGsghCCCFErZw2jLXWY0+zXwN3+a1EZyk11sX3W3IDXQwhhBCixoLnzrhPWpyTfQVlFJSWB7ooQgghRI0EXRi3jnMCsG1/cYBLIoQQQtRM0IVxWpwLgCy5byyEEKKRCLowTo52oBQyoloIIUSjEXRhbLeaSYwKJStXWsZCCCEah6ALY4C0WBe/7peWsRBCiMYhKMM4NdbJr/uLMJ66EkIIIRq2oAzj1nFOit0e9hwuDXRRhBBCiNMKyjCuHFH9qwziEkII0QgEaRgbzxr/IveNhRBCNAJBGcbxYXZCrWYZUS2EEKJRCMowNpkUqbFOsqSbWgghRCMQlGEMRle1PN4khBCiMQjeMI51kpNXTFmFJ9BFEUIIIU4peMM4zoVXw/YD8sIIIYQQDVsQh7ExolruGwshhGjogjaMU2N9YSxvbxJCCNHABW0Yh9mtxIWFSMtYCCFEgxe0YQzGIC4ZUS2EEKKhC+4wjnPJxB9CCCEavOAO41gnecXl5BW5A10UIYQQ4qSCO4zjZBCXEEKIhi/Iw9h4e5MM4hJCCNGQBXUYJ0WFYjEpsmQQlxBCiAYsqMPYYjaRHOOQQVxCCCEatKAOY4C0WJc83iSEEKJBC/owbh3nZNuBYjxeHeiiCCGEECcU9GGcGuvEXeFlZ15JoIsihBBCnFDQh3HliOpf5PEmIYQQDVQTCGPjWeNf5fEmIYQQDVSNwlgpNUIptUkptVUp9dAJ9qcopf6nlFqjlFqglEr0f1HPTIzTRrjdIhN/CCGEaLBOG8ZKKTMwGRgJdATGKqU6HnPYc8B0rXUX4HHg7/4u6JlSSpEa55KJP4QQQjRYNWkZ9wa2aq2ztNZuYAYw6phjOgLf+pbnn2B/QLWWtzcJIYRowGoSxi2B7GrrOb5t1a0GrvQtXwGEKaVizr54/pEW52R3finF7opAF0UIIYQ4jtL61M/fKqWuBkZorcf71m8A+mit7652TALwCpAKLASuAjprrQ8dc67bgNsA4uPje8yYMcNvFSksLMTlcp1w39I9Fby6qoy/9reTEm722zXrw6nq1VgFY50gOOsldWo8grFewVanoUOHLtda9zzRPksNfr8TSKq2nujbVkVrvQtfy1gp5QKuOjaIfcdNAaYA9OzZUw8ZMqQm5a+RBQsWcLLzxe8+zKurvic6pQNDMhL8ds36cKp6NVbBWCcIznpJnRqPYKxXMNbpZGrSTf0z0FYplaqUsgHXAnOqH6CUilVKVZ7rYWCqf4t5dlJjnSglb28SQgjRMJ02jLXWFcDdwNfABmCm1nqdUupxpdRlvsOGAJuUUpuBeOCpOirvGbFbzSREhMrjTUIIIRqkmnRTo7X+EvjymG2PVVueBczyb9H8Ky3OKS1jIYQQDVLQz8BVKc33eNPpBqwJIYQQ9a3phHGci8KyCnILygJdFCGEEOIoTSaMU2ONOap/ka5qIYQQDUyTCeOqF0bITFxCCCEamCYTxgkRoditJrJyZUS1EEKIhqVGo6mDgcmkaBXjJEtaxkII0bh5KqC8CNzFUO77eNzg9fg+FaA96PIyvEUleEtK8JYU4y0uQZeU+tZL8ZaU4S2t/C7DW+pGl5bhLS3HW+ZGmc0kfvBVvVSpyYQxGF3V63cdDnQxhBAi+HnK0SWH0UWH8BYcQhfnowsP4y06jC4pQJcUokuK0KVF6NISdGkx3rISdGkp2l2KLiujTX4+udNM6PJytLscb0UFurwCXeFFexXao9Be0B6F16PwVhgfXWEyvr2q5uVVGpPlyEdZvFjs9Td9ctMK41gXX6/bi7vCi83SZHrohRC1oLVGl5TgKShAl5WByQQolEmBUideVwpVbRllQimOrJtMKHzr1Y+vh7pQUYG3rMwIvuICdEkx3pIiXxgWo8uK8ZaWGMulpeiyEuP40hJ0WSnekmJ0WanRonSXoUvL0G43Xnc52m2Eo7e8Al3uxVuh0b6P16ugNmF4EgWAMiuUxYyy2FBWM8piQVktKJsVZbVistkw20Owhtox2e2YHHZMoaEoux2Tw4HJEer7dmByOI1vpxOTw4lyujA5XSh7KMpsAWUGkwVMZmO5njStMI5z4vFqdhwspk2z4Jl8XAhxNO124zl8GM/hArwFh33Lh/EWFOBYtZp9y5bhOVyAp+Aw3vzDeAoK8B42vj0FBVBeXveFVApMvlCuWja2G0EP+Hbhy7SqSYeVNrZz5LtleQVbtTKC0IMRil5An20gapRZV2WTsoDJYkJZTSirGZPNgtnlQNmMUFQhNkwhISi7HWUPxWS3o0KdKLsv/EJdKIcTZXehHGGYHGEoR4RxvM121Of7xYsZfP759fIPl0BrUmFc+XhTVm6hhLEQp6G1hvJyPIcO+e6xlRj33EqPLHtLitFVy759xSVHji8pRleul5YYy263L2RM1VqWJ1mvbEWafIlU1dJUqOrrSuEtLqoKVl1aetJ6hQEHrVZMERGYw8Iwh4djjozElpSEKTwMc3gE5jAXJpfDaBi5i333JkvR7mIoLwF3CZSXoN0lUFEC5aVVH11eCuVlUGF8V000VPmlAa2omn9Ig9bqmP0AJjRmUL5vTMY2ZTqyjNFK15igogK7y4my+VqNVisqxBeQlZ+QEEwhdlSIDRXiC0vft/FxYAp1okIdKLsD5Qg3PiEusDrAXM+RYbU2iSCGJhbGaXFGAMvjTaIx0Fqjy8vxFhWhi4t9AVjs+1QuFxlhWFKCdruNT3n5kY+7ctldbbn6fvcpt8VrzeZalFmFhBgtIYcDU2ho1cccFYU1NMHoOgwJqUoc7fUaweP1gtZoXbnuAU8F2lMOFeXGgBxPBdpjfBv7y9HeCvB4wOPB6jJhjnNisodjDlGYQkyYQ8Bs9WKygdnmwWT1UlR8gMgIhzHgp2IPeLLBU+Zbd0OhGw7XsGVstUJYOISEgz0cQqKrLVd+h1VbjgCbEywhYLb5vq3Gsrn6stXoJq2FpvSGo2DUpMI4ItRKrMsmc1SLOqO1RhcXG92evi5Pb0GB0V1aWPldeCRUS0qMQC0uOWabsYzHU6vrK5vNaBFVflf/VNtmcjiOP8ZmRVlt1Zat/LprF206dkKF2jGF+u69+QJWhVZfdmAKsaLKC6E0H0oPQcmhat/VtpXtM1qX5ZUtTt9y1bdvZGxtKZMRaBbb0eFWFXhGAHo9oeCKr7bPduRT/ViLDSyhxwTrMctWe+3LKcQJNKkwBmMQl7y9SVSqbH3qsrKqj7eyhVlWhreo6OhgPVyAp/DE33F5eWwsLT19gFqtmB0Oo/VY2YJ0OLA0a4bJEeprVTqODDgJDcXkNJaV71hTqKNqW2UgnlWXntZQUQbuInAXVn3vXrqI6HahviDdbnzv84VsaX61wM2HssNU9bWeiMkKoZFGiNkcRrenzQHOOLCG+j6OI982x/HbrKFgdZ7g+FAjRGtgjbQgRQPU5MI4NdbJ/zbuDXQxRC143e6qwTVHtTarh2NBoTHi0+3GW+Y+Eq5utxGup1ivLZPLZdxbdIVhCg/D2iweU5s25OcfJqlDe8xhYZh89yJNrjDM4WFHvsPCMIWE+OkPxmsEYXEuHD4IZQW+EK0M1MJj1ouOXi87Zr8+/h8RXQAyq22whBqBao8AeySEJ0Czjr5tvu2Vy8duszp8o5GEEMdqcmGcFufkw2Vu8kvKiQit2b+kxdnzut14cnOpyM3FtmYN+dVHuhYUHvk+fBhP4dHfpw1Mk8kIyMrRmCEhxohOWwjKZsMcEeFbt6FsIdX2H72ubL5RoL7fKZsVk9NpDPDxBazJ6USZT3wvb8uCBTQ70xZXVbAehOIDxqek2nLxwSP7KreX5GEMlz0VBTaXcZ+y8hMSZrRGo1r5trmqHVP9WBcr1m+le/+hR8LV4qd/SAghjtIEw9gYxJWVW0i35KgAl6Zx01rjLSykIjeXitz9vm/fZ3+15dz9ePPzq34XBeyqdh4VGorZ5cIUHo7Z5cIcEYEtsSWmsHBjVGtYOKYwF+bK7/BwTC5XVcvT5HQEfsSl14O5ogjyc4wWalmB0W1btVztU3oIivOODt1TBavZBo4YCI0GRzQ062CsV22LMbaHhB0VpNhcRvftWfzZHN5pgbhzzvj3QoiaaXJhfOTxpiIJ41PwlpTg3pFNxb69VOzLPSZojwTviR4hUTYblrg4LHFxhKSm4ezdB0uzOCyxsVji4li1bRu9hwypCl9lbSA9FF4PFO2Hwj1QsAcKdvsGHBWcPmDLixgIsOg017C5jNCsDND4TkeC1RF9dOhWbrO5pHtXiCDX5MI4OdqB2aTk8SYqA3cH7u3bcW/fTvmOHbi3bce9YwcVe4+/r24KC6sK2dAuXaqWLc3ijizHxmIKDz9lS7VCa2wpKXVZtaNVhmzBbiNkq8J2z9HrhftOeN8UZTryeEpImC9MoyEq5ajtW7P30qZTtyPHVD8+JMwI1Vo+riKEaBqaXBjbLCaSox1NZkS1t7gYd3a2L2R9obvdCOCKffuOOtYcHY0tJQVn377YWqVgTUrGmtDiSMiGhgaoFifh9UBRri9k9xrfhXuPXy/ce+IuYEcshDU3PvGdwNX8yHpYC+PxF0d0jQce5SxYQJvuQ/xfTyFE0GtyYQyQFusMqmeNvWVluH/9tapV696+7Ujg5uYedaw5JsYI3P79sbVKwZacjDXF+DaHhQWoBsc4ZchWa9EW7TtxyDrjjgRr885HgjWsxZGwdTYzniMVQogGoEmGcWqsk0Vb9+P1akymxnMvTrvdlG3bhnvrVkq3bMG9dStlW7bi3rHDGI3rY46NNQL33HOxpaRgS0nGlpKCNTkZsyuA04BqDcUHcBVshU2lZxmy6UeCtXqLVkJWCNEINckwTotzUVbhZVd+CYlRjkAX5zi6ogL3jmzKtmzBOe8bcj77jLItW3Bv2w4VFcZBZjO25GRC2rUj/KKLCGnbBlurVliTkjG7nIErfFkB5G2HQ9tP/F1eRE+A5ZU/UOCMPRKq1UM2rMWRoHU1q/GkDkII0dg00TA+MqI6kGGsvV7Kc3Io87Vwy7ZsoWzrVtxZWVXP1jqVojQpiZA2bQg7bxghbdsawZua6r/JI2qjogwOZcOhbScO25KDRx9vc0FkivFMa+pgiEphbXY+nfsNl5AVQgifphnG1d7eNKhdXL1cU5eXU7pxEyUrV1K6fr0RvFlZ6JKSqmMsCS0IadMG54D+hLRpS0jbtizZtZMhF1xQL2UEjInyD+cYgZufDYd2HB22Bbs5aspDsw0ikoyRxS26Gt+RKb7vVsYAqGMGP+0vXQCJPeqvTkII0cA1yTCOCwvBFWKp08ebKvLyKFm9mpKVqyhZsYKSzMyqZ3LNcbHY27Yl6prR2Nq0wd62LbY2bU58P3d/7vHbzoa76Oigzc+utp59fNiiILylEa5pQ44J2xSjK9lkOvG1hBBC1EiTDGOlFGlxTrL8FMba68W9bRslK1ZQvHIlJStX4c7KMnZaLNg7dCDymtE4unUjtGtXrC1a+OW6xxdEGzM5HRey1daP7UY2WSGipdG6bT3U+I5MOvIdnigDooQQoo41yTAGo6v65215Z/Rbb0kJJWsyKVm50visWoXHN92jOSKC0G7diBg1itBuXQlNT6+b53O1hv1bIPsn2PET7FpphK77mOenrc4j4dqyhy9kk4+ErSteJqIQQogAa7JhnBrr4tNVuygt92C3njqMynfvpmTlSopXrjLu+W7cWDWq2da6Na7h5+Po1p3Qbt2wpbaqm3mSK8pg1yrYsRiylxgBXNnKDY2GpN5GN/JRLdtkCI2SqRSFEKKBa7JhXDmi+tf9RXRoEX7c/rKsXzkwZQpFP/1ExZ49gPFCg9AuXYgZf6vR5ZyRgTkysm4KWHwQspeS9suHkPU07FwBnjJjX0wbOOciSO5rfGLaSOAKIUQj1uTDOCv36DAu37eP/ZNf5dCsWZhCQnANGUJot26EduuG/Zx2dfNSA60h71fYseRIyzd3IwCJygItu0HvCZDcD5L6gKt+RoALIYSoHzUKY6XUCOCfgBl4S2v99DH7k4F3gUjfMQ9prb/0c1n9qvLtTb/65qj2FBRw4O23OfjudHRFBVFjxxJ7x+1YYmL8f3FPOexZY3Q17/jJCN9C34sZQiIguQ+kj4bkfiz6pZBBwy70fxmEEEI0GKcNY6WUGZgMDAdygJ+VUnO01uurHfYIMFNr/ZpSqiPwJdCqDsrrNw6bhRYRdrbtPsTBd99l/2uv4zl0iPCLLiLuvnuxJSf794JaG8G75DXY8g2UFxvbI5ONe71JfYyWb1z7ox4V8m5b4N9yCCGEaHBq0jLuDWzVWmcBKKVmAKOA6mGsgcq+3giOfnd8g6S9Xi7LXc2g2bPZe3g/zv79iLv/94R27uTfC1W4Yf2n8NOrxohneyR0/Q2k9Dfu94Yn+Pd6QgghGp2ahHFLILvaeg7Q55hjJgH/VUpNBJzA+X4pXR3QWlO06Af2Pf88l2/cSFZUIhlvvYXr3AH+vVDRAVg+DX5+y5hII6YtXPwCZFwLtgDOHS2EEKLBUVrrUx+g1NXACK31eN/6DUAfrfXd1Y6533eu55VS/YC3gc5aH/3qHaXUbcBtAPHx8T1mzJjht4oUFhbiOs0biSzbtuGa/SkhmzZRERvD4v6X8HfViX+e5yI8xD+jkR1FO0jM+Q/xexdg9ro5GNWVnMTLOBjdzXhJfS3VpF6NTTDWCYKzXlKnxiMY6xVsdRo6dOhyrXXPE+7UWp/yA/QDvq62/jDw8DHHrAOSqq1nAc1Odd4ePXpof5o/f/5J95Vt26az77tPrz+nvd7Ut58+MP097S0r0/M37tUpf/xcL8k6cHYX93i03vyN1tMv1/ov4Vo/0UzrzyZqvXf92Z1Xn7pejVUw1knr4KyX1KnxCMZ6BVudgGX6JJlYk27qn4G2SqlUYCdwLXDdMcfsAIYB7yilOgB2wM+TKtdexf797H/1VfJmfoSy2Yi9806ibxlXNQd06zjjOyu3kN6p0bW/gLsY1syAn16D/ZuN1/2d9wj0uAWcdTAKWwghRFA6bRhrrSuUUncDX2M8tjRVa71OKfU4RsrPAX4PvKmU+h3GYK6bff8KCAhPYREHp07lwDvvoN1uoq4ZTewdd2CJO/r53ITIUGwWU+1fGHF4FyydAsvfMeaCbtEVrnwTOl4u8zgLIYSotRo9Z6yNZ4a/PGbbY9WW1wN+HgFVe9rtJu/Dmex/7TU8Bw8SNnIEze69F1urVic83mxStIpx8EtuDcN453JY/KoxOlp7of0l0PdOY1S0zIAlhBDiDAXFDFza6yXk55/55am/UZ6djaNPH5r94feEpqef9rdpsS427ys4+QGeCtj4H6MrOnsJhIRDn9uNGbGiWvmvEkIIIZqsoAjjg9OnE/n2VEznnEPSm1NwnntujV/WkBbnZN6GvZR7vFjNx4x2/vltWPSi8frBqFYw4h/Q7TcQEub/SgghhGiygiKMI6+4gi1799LngQdQtXzRfWqskwqvJievpGqKTMCYLeuL+42ZsUb+A9qNkFcNCiGEqBNBEcbmiAhK+/SpdRADpFUbUX1UGC94GpxxcMOnYHP4q6hCCCHEcWqfXkGmdbW3N1XJXgpZ86H/PRLEQggh6lyTD+NIh40oh5Ws6o83LXgaHLHQ69bAFUwIIUST0eTDGIyu6qxc41WKZP8Mv/wPBtwjc0gLIYSoFxLGQFqs80jL+LunwREDvcYHtlBCCCGaDAljjJZxbkEZRVlLYOs86D9RWsVCCCHqjYQxVI2i9sz/O4RGQ68JAS6REEKIpkTCGGNEdRf1C+HZ841WcUjwvLJLCCFEwydhDCTHOLjP8gkllnBjmkshhBCiHkkYAyF7V3OeeSXfRIyWqS6FEELUOwljgO+eodAUxrueCwNdEiGEEE2QhPGuVbD5K35uPpb1BzReb8BewyyEEKKJkjD+7h9gj2Rfh5soKfewt6A00CUSQgjRxDTtMN69GjZ9Cf3uIqlFc+CYOaqFEEKIetC0w/i7Z8AeAX1+e9Tbm4QQQoj61HTDePca2Pg59L0T7BHEh4fgsJn5RVrGQggh6lnTDePv/gEhEdDndgCUUqTGOvl1v4SxEEKI+mUJdAECYs9ao1U8+CEIjazanBbnYlV2XgALJoQQtVdeXo7L5WLDhg2BLopfRURENMo62e12EhMTsVqtNf5N0wzj7/4BIeHQ9/ajNqfFOvl8zS5Kyz3YreYAFU4IIWonJyeH+Ph4EhMTUUoFujh+U1BQQFhY45qISWvNgQMHyMnJITU1tca/a3rd1HvXwYY5Rvd0aNRRu9LinGgNOw4WB6hwQghRe6WlpURERARVEDdWSiliYmIoLa3dY7JNL4y/+wfYwqDvHcftSouVEdVCiMZJgrjhOJP/Fk0rjPeuh/WfGd3TjujjdqfGGa9SlBHVQggh6lPTCuOFz/haxXeecLcrxEJ8eIiMqBZCiFpyueTVs2ej6YTxvg2w7lPoc9sJW8WVUmOd0k0thBCiXjWdMP7uGbA5od/dpzwsLc5FlrSMhRDijGiteeCBB+jcuTPp6el8+OGHAOzevZtBgwbRtWtXOnfuzPfff4/H4+Hmm2+uOvbFF18McOkDp2k82rRvI6ybDef+7pStYjAebzpUXM7BIjfRTls9FVAIIfzjr/9Zx/pdh/16zo4J4fzl0k41OvaTTz5h1apVrF69mv3799OrVy8GDRrE+++/z4UXXsif//xnPB4PxcXFrFq1ip07d7J27VoADh065NdyNyZNo2W88FmwOk7bKgZo7Zuj+tf90lUthBC1tWjRIsaOHYvZbCY+Pp7Bgwfz888/06tXL6ZNm8akSZPIzMwkLCyMtLQ0srKymDhxInPnziU8PDzQxQ+YGrWMlVIjgH8CZuAtrfXTx+x/ERjqW3UAzbTWkTQEuZth7ccw4F5wxpz28NTYIyOqe6ScuhUthBANTU1bsPVt0KBBLFy4kC+++IKbb76Z+++/nxtvvJHVq1fz9ddf8/rrrzNz5kymTp0a6KIGxGlbxkopMzAZGAl0BMYqpTpWP0Zr/TutdVetdVfg/4BP6qKwZ2ThM2ANhf4Ta3R4YlQoVrOSVykKIcQZGDhwIB9++CEej4fc3FwWLlxI79692b59O/Hx8UyYMIHx48ezYsUK9u/fj9fr5aqrruLJJ59kxYoVgS5+wNSkZdwb2Kq1zgJQSs0ARgHrT3L8WOAv/ineWdq/xWgV97sbnLE1+onFbCIlxind1EIIcQauuOIKFi9eTEZGBkopnnnmGZo3b867777Ls88+i9VqxeVyMX36dHbu3Mm4cePwer0A/P3vfw9w6QNHaa1PfYBSVwMjtNbjfes3AH201sfdgFVKpQA/AYlaa88J9t8G3AYQHx/fY8aMGWdfA5/CwsLjnnNrv+FF4nIX81PfKZTbat5r/s8Vpewt9vK3cx1+K9+ZOlG9GrtgrBMEZ72kTo1DREQEqampmM3BNae+x+NptHXaunUr+fn5R20bOnTocq11zxMd7+/R1NcCs04UxABa6ynAFICePXvqIUOG+O3CCxYs4Kjz7d8K3y2Efncx4ILLa3WuxSUbmLZoGwMHDcZsCuwUc8fVKwgEY50gOOsldWocNmzYgNlsbnQvVTidxviiiEp2u51u3brV+PiajKbeCSRVW0/0bTuRa4EPanz1urTwWTCHQP97av3T1rEu3B4vOXnywgghhBB1ryZh/DPQVimVqpSyYQTunGMPUkq1B6KAxf4t4hk48AtkzoRet4KrWa1/nuabo1om/xBCCFEfThvGWusK4G7ga2ADMFNrvU4p9bhS6rJqh14LzNCnuwldHxY+B2bbGbWK4cjjTTKiWgghRH2o0T1jrfWXwJfHbHvsmPVJ/ivWWTiYBWs+NN5XHBZ/RqeIdtqICLXKHNVCCCHqRfDNwLXweTBbjUk+zpBSirQ4p7y9SQghRL0IrjA+mAWrP4Ae4864VVzJeHuThLEQQoi6F1xh/P3zYLKcVau4Uus4F3sOl1JUVuGHggkhhPCHiorg/Ds5aMLYXrIXVs+AnuMgvMVZny/NN4hLuqqFEKJmLr/8cnr06EGnTp2YMmUKAHPnzqV79+5kZGQwbNgwwJh4Zdy4caSnp9OlSxc+/vhjgKMmY5k1axa33347ADfffDO33347ffr04cEHH2Tp0qX069ePbt260b9/fzZt2gQYk4T84Q9/oHPnznTp0oX/+7//49tvv+Xyy4/MNfHNN99wxRVX1MufR20ExSsUtdZ4sv8FyvHNxRYAACAASURBVAwD7vPLOdN8b2/K2l9E55YRfjmnEELUua8egj2Z/j1n83QY+fRpD5s6dSrR0dGUlJTQq1cvRo0axYQJE1i4cCGpqakcPHgQgCeeeIKIiAgyM41y5uXlnfbcOTk5/Pjjj5jNZg4fPsz333+PxWJh3rx5/OlPf+Ljjz9mypQpbNu2jVWrVmGxWDh48CBRUVHceeed5ObmEhcXx7Rp07jlllvO7s+jDgRFGL+/4hWesf3KnIyrSfFDqxggJcaB3WriP6t3cWmXFigV2Jm4hBCioXv55ZeZPXs2ANnZ2UyZMoVBgwaRmpoKQHS08Sa8efPmUX065KioqNOee/To0VVTY+bn53PTTTexZcsWlFKUl5dXnff222/HYrEcdb0bbriBf/3rX4wbN47Fixczffp0P9XYf4IijC+sMPGihqnhDv7qp3ParWbuH96Ov325kTmrdzGqa0s/nVkIIepQDVqwdWHBggXMmzePxYsX43A4GDJkCF27dmXjxo01Pkf1Rk9paelR+5xOZ9Xyo48+ytChQ5k9ezbbtm077fSm48aN49JLL8VutzN69OiqsG5IguKecWyfu+jn6sec7P+xp2iP385767lpdEuO5C9z1pFbUOa38wohRLDJz88nKioKh8PBxo0b+emnnygtLWXhwoX8+uuvAFXd1MOHD2fy5MlVv63spo6Pj2fDhg14vd6qFvbJrtWypdFAeuedd6q2Dx8+nDfeeKNqkFfl9RISEkhISODJJ59k3Lhx/qu0HwVFGAMMjRoJGt5d967fzmk2KZ69ugvFbg+PfJpJQ5hcTAghGqIRI0ZQUVFBhw4deOihh+jbty9xcXFMmTKFK6+8koyMDMaMGQPAI488Ql5eHp07dyYjI4P58+cD8PTTT3PJJZfQv39/WrQ4+S3HBx98kIcffphu3bodNbp6/PjxJCcn06VLFzIyMnj//fer9v3mN78hKSmJDh061NGfwNlpeG31MxRtiebitIuZtXkWE7pMINoe7ZfztmkWxv3D2/H0Vxv5fM1uLs1I8Mt5hRAimISEhPDVV1+dcN/IkSOPWne5XLz77vENp6uvvpqrr766ar2goAA4uvUL0K9fPzZv3ly1/uSTTwJgsVh44YUXeOGFF44796JFi5gwYULNKhMAQdMyBrgl/RbKPGX8a/2//HreCQPT6JoUyWOfrWV/oXRXCyFEY9KjRw/WrFnD9ddfH+iinFRQhXFaRBrnp5zPjI0zKHAX+O28ZpPiudFdKHJ7ePTTtdJdLYQQjcjy5ctZuHAhISEhgS7KSQVVGAOMTx9PQXkBH2760K/nbdMsjN+d346v1u7hi8zdfj23EEKIpi3owrhjTEcGtBzAe+vfo6SixK/nnjAwlYzECB77bJ10VwshhPCboAtjgAnpEzhYepDZW04+NP5MWMwmnh2dQWFpBX/5bJ1fzy2EEKLpCsow7hHfg+7NujNt3TTKPeV+PXe7+DDuPb8tX2Tu5os10l0thBDi7AVlGANM6DKBPUV7+Dzrc7+f+7eD0uiSGMGjn63lgHRXCyGEOEtBG8YDEgbQIboDU9dOxeP1+PXcFrOJZ682uqsfmyPd1UIIUVvV39B0rG3bttG5c+d6LE3gBW0YK6UYnz6ebYe3MW/HPL+f/5zmvu7qNbv5UkZXCyGEOAtBMwPXiQxLHkar8Fa8lfkWF6Rc4Pc3L/12UBpz1+7h0U/X0jcthminza/nF0KI2vrH0n+w8WDNX85QE+2j2/PH3n885TEPPfQQSUlJ3HXXXQBMmjQJi8XC/PnzycvLo7y8nCeffJJRo0bV6tqlpaXccccdLFu2rGqGraFDh7Ju3TrGjRuH2+3G6/Xy8ccfk5CQwDXXXENOTg4ej4dHH320agrOhi5oW8YAZpOZW9NvZePBjSzaucjv5zdGV3fhcGk5j3221u/nF0KIxmLMmDHMnDmzan3mzJncdNNNzJ49mxUrVjB//nx+//vf13rSpMmTJ6OUIjMzkw8++ICbbrqJ0tJSXn/9de69915WrVrFsmXLSExMZO7cuSQkJLB69WrWrl3LiBEj/F3NOhPULWOAi9Mu5tVVr/JW5lsMTBzo9/O3bx7OvcPa8tx/N3Nx+m5GpvvnfcpCCHEmTteCrSvdunVj37597Nq1i9zcXKKiomjevDm/+93vWLhwISaTiZ07d7J3716aN29e4/MuWrSIiRMnAtC+fXtSUlLYvHkz/fr146mnniInJ4crr7yStm3bkp6ezu9//3v++Mc/cskllzBwoP//zq8rQd0yBrCarNzc6WZW7FvB8r3L6+Qavx3cms4tw3n0s7UcLHLXyTWEEKKhGz16NLNmzeLDDz9kzJgx/Pvf/yY3N5fly5ezatUq4uPjj3tP8Zm67rrrmDNnDqGhoVx00UV8++23tGvXjhUrVpCens4jjzzC448/7pdr1YegD2OAK9teSbQ9mjcz36yT81t9o6vzS8qZJKOrhRBN1JgxY5gxYwazZs1i9OjR5Ofn06xZM6xWK/Pnz2f79u21PufAgQP597//DcDmzZvZsWMH55xzDllZWaSlpXHPPfcwatQo1qxZw65du3A4HFx//fU88MADrFixwt9VrDNNIoztFjs3dLyBH3b+wLoDdROWHVqEM/G8tsxZvYu5a/fUyTWEEKIh69SpEwUFBbRs2ZIWLVrwm9/8hmXLlpGens706dNp3759rc9555134vV6SU9PZ8yYMbzzzjuEhIQwc+ZMOnfuTNeuXVm7di033ngjmZmZ9O7dm65du/LXv/6VRx55pA5qWTeC/p5xpTHnjGFq5lTeznybF4Yc/65Lf7hjSGu+XreHRz5dS5/UaKJkdLUQoonJzMysWo6NjWXx4sUnPK6wsPCk52jVqhVr166loKAAu93OtGnTjjvmoYce4qGHHjpq24UXXsiFF154hiUPrCbRMgYIs4Vxbftrmbd9HlmHsurkGlaziedGZ3Co2M2k/0h3tRBCiJppMmEMcH3H67Fb7Ly99u06u0Zld/Vnq3bx9TrprhZCiJPJzMyka9euR3369OkT6GIFRJPppgaItkdzVdur+GDjB9zZ9U5aulrWyXXuHGp0V/95ttFdHemQ7mohhDhWeno6q1atCnQxGoQm1TIGuKnTTSileGftO3V2DatvMpBDxW7++p/1dXYdIYQQwaFGYayUGqGU2qSU2qqUeugkx1yjlFqvlFqnlHrfv8X0n+bO5oxqPYpPtnzC/pL9dXadTgkR3DW0DbNX7uSb9Xvr7DpCCCEav9OGsVLKDEwGRgIdgbFKqY7HHNMWeBgYoLXuBNxXB2X1m3Gdx1GhK3hv/Xt1ep27hrahQ4tw/jQ7k0PFMhmIEEKIE6tJy7g3sFVrnaW1dgMzgGNn+p4ATNZa5wForff5t5j+lRKewoUpF/Lhpg/JL8uvs+vYLCaeG92FvCI3j0t3tRBCiJNQp5u0Wyl1NTBCaz3et34D0EdrfXe1Yz4FNgMDADMwSWs99wTnug24DSA+Pr7HjBkz/FUPCgsLT/l+zGPtdO/k6d1Pc3HExYyIrNvJxGdvcfPZL+Xc2z2Ebs1qN2autvVqDIKxThCc9ZI6NQ4RERGkpqZiNpsDXZQaa9GiBbt3n/r1sx6Pp1HVqbqtW7eSn390Y2/o0KHLtdY9T3S8v0ZTW4C2wBAgEViolErXWh+qfpDWegowBaBnz556yJAhfro8LFiwgNqeb/H/FrModxGPDXgMh9Xht7Icq/+5Xja9sogPtri55ZIBRDisNf7tmdSroQvGOkFw1kvq1Dhs2LABs9lMWFhYoItSK6crb0FBQZ3XqaKiAovF/w8W2e12unXrVuPja1KCnUBStfVE37bqcoAlWuty4Fel1GaMcP65xiUJgPHp47nhqxuYtXkWN3a6sc6uY3RXZzBq8g/89fN1vHBN1zq7lhCiadvzt79RtsG/7zMO6dCe5n/60ymP8ef7jAsLCxk1ahQHDhzA4/Ec9bvp06fz3HPPoZSiS5cuvPfee+zdu5fbb7+drCxjQqfXXnuNhIQELrnkEtauNV5v+9xzz1FYWMikSZMYMmQIXbt2ZdGiRYwdO5Z27drx5JNP4na7iYmJ4d///jfx8fEUFhYyceJEli1bhlKKv/zlL+Tn57NmzRpeeuklAN58803Wr1/Piy++eMZ/vlCzMP4ZaKuUSsUI4WuB64455lNgLDBNKRULtAPqZporP+rarCu9mvfi3XXvcm37a7GZ6+554M4tI7hzSGv+79utXJzegmEd4uvsWkIIUd/GjBnDfffdVxXGM2fO5Ouvv+aee+4hPDyc/fv307dvXy677DKUUqc8l91uZ/bs2SilKCsrq/rd+vXrefLJJ/nxxx+JjY3l4MGDANxzzz0MHjyY2bNn4/F4KCwsJC8v75TXcLvdLFu2DIC8vDx++uknlFK89dZbPPPMMzz//PM88cQTREREVE3xmZeXh9Vq5amnnuLZZ5/FarUybdo03njjjbP94zt9GGutK5RSdwNfY9wPnqq1XqeUehxYprWe49t3gVJqPeABHtBaHzjr0tWD8enj+e03v2XOL3O4ut3VdXqtiee15Zv1e7nvw1W8cX0P+reJrdPrCSGantO1YOuKP99nrLXmT3/6EwsWLMBisVT97ttvv2X06NHExhp/d0ZHRwPw7bffMn36dADMZjMRERGnDeMxY8ZULefk5DBmzBh2796N2+0mNTUVgHnz5lF9bFNUVBQA5513Hp9//jkdOnSgvLyc9PT0Wv5pHa9Gzxlrrb/UWrfTWrfWWj/l2/aYL4jRhvu11h211ulaa/+NzKpj/Vr0o1NMJ6aunUqFt6JOr2WzmHjrpp40D7dz49SlfLQsu06vJ4QQ9clf7zOu/N3ChQvP+D3IFosFr9dbtX7s751OZ9XyxIkTufvuu8nMzOSNN9447bXGjx/PO++8w7Rp0xg3blytynUyTW4GrmMppZiQPoHsgmz+u+2/dX69xCgHs+7oT9+0GB6YtYbn/7uJ041oF0KIxsBf7zM+2e/OO+88PvroIw4cMDpeK7uphw0bxmuvvQYYI7Dz8/OJj49n3759HDhwgLKyMj7//PNTXq9lS2N65Hfffbdq+/Dhw5k8eXLVemVru0+fPmRnZ/P+++8zduzYmv7xnFKTD2OAoclDaR3Rmjcz38Srvaf/wVmKCLUybVwvxvRM4v++3cq9M1ZRWu6p8+sKIURd8tf7jCt/17dv36N+16lTJ/785z8zePBgMjIyuP/++wH45z//yfz580lPT6dHjx6sX78eq9XKY489Ru/evRk+fPgprz1p0iRGjx5Njx49qrrAAR555BHy8vLo3LkzGRkZzJ8/v2rfNddcw4ABA6q6rs+a1jognx49emh/mj9//ln9fs7WObrzO531/B1nd57a8Hq9evL8LTrlj5/rq179QR8oLDvumLOtV0MUjHXSOjjrJXVqHNavX68PHz4c6GL4XUOu08UXX6znzZt30v3r168/bhvGOKsTZqK0jH1Gpo6kpaslb655s966jZVS3DmkDa9c1401O/O54tUfyMo9+Qu3hRBCBNahQ4do164doaGhDBs2zG/nlTD2sZgs3NL5FtbsX8PSPUvr9dqXdEnggwl9KSit4MrXfmRJVqMYiC6EEGelMb7PODIyks2bN/PRRx/59bwSxtWMajOK2NBY3sx8s96v3SMlik/vHECM08b1by9h9sqcei+DEKLxqq8ePX+qfJ9x9c+SJUsCXayzdib/LSSMqwkxh3BTx5tYsnsJa3LX1Pv1k2McfHLHAHqmRPO7D1fz0rzNjfJ/MCFE/bLb7eTn58vfFw2A1poDBw5gt9tr9Tv/T8jZyI0+ZzRvZr7JW5lv8fJ5L9f79SMcVt69pTcPf5LJS/O20C/BTP+BHkIsjXOydCFE3UtMTGT16tUUFgbXmJPS0tJah1pDYLfbSUxMrNVvJIyP4bQ6+U2H3/Da6tfYkreFtlFt670Mla9ebBXj4PlvNnPD20uZckMPIh11N12nEKLxslqtFBYW0rPnCV8I1GgtWLCgVi9baMykm/oErmt/HaGWUN5e+3bAyqCUYuKwtvy2Swirdhziyld/ZPuBooCVRwghRN2RMD6BSHsk17S7hq9+/Yrsw4GdsrJfgoV/T+hDXrGbK179kWXbDga0PEIIIfxPwvgkbux0IzaTjXvm38Peor0BLUuvVtF8cucAIkKtXPfWEuas3hXQ8gghhPAvCeOTaOZoxivDXmFX4S5u/OpGtuVvC2h5UmOdfHJHf7omRnLPByuZPH+rjJwUQoggIWF8Cn1a9GHqiKmUekq58asbWXdgXUDLE+W08d743lzeNYFnv97Eg7PW4K6o+7m0hRBC1C0J49PoFNOJd0e8S6gllFvm3sKS3YF9ID3EYubFMV25d1hbPlqew83TlpJfUh7QMgkhhDg7EsY10CqiFe9d9B4JrgTumHdHvbxq8VSUUvxueDueH53Bz9sOctVrP5J9sDigZRJCCHHmJIxrqJmjGe+MeIdOMZ34w3d/YOammYEuElf1SOS9W/uQW1DG5ZN/YOaybDxeuY8shBCNjYRxLUSERDDlgikMTBzIEz89wRur3wj4IKq+aTF8cmd/EqMdPDhrDRf983vmb9oX8HIJIYSoOQnjWgq1hPLS0Je4NO1SXln1Ck8vfRqvDuwgqtZxLj69sz+Tr+tOaYWHcdN+5ro3l7Am51BAyyWEEKJmZDrMM2A1WXny3CeJskcxff108sryeGrAU1jN1oCVSSnFxV1aMLxjPB8s3cE//7eFy175gUszEnjggnNIjnEErGxCCCFOTcL4DJmUiT/0/APR9mheWvESh8sO88KQF3BYAxt6NouJm/q34sruLXnjuyzeWpTF3LW7uaFvKyae14Yop8xvLYQQDY10U58FpRS3pt/KX/v/lcW7FzPhvxM4VNowuobD7Fb+cOE5LPjDUK7qnsg7P/7KoGfm8+qCrZSWewJdPCGEENVIGPvBlW2v5IUhL7Dx4EZumnsTe4r2BLpIVZpH2Hn6qi58fd8g+qRF88zcTQx9boGMvBZCiAZEwthPhiUP4/Xhr7OveB83fHUDWflZgS7SUdrGh/HWTb2YcVtfmoWFyMhrIYRoQCSM/ahX815MvXAqbo+bm766iczczEAX6Th902L49K4BMvJaCCEaEAljP+sQ04H3Rr6H0+rk1v/eyo+7fgx0kY5TOfL6m98NZtKlHdm0t4DLXvmBiR+sZMcBmclLCCHqm4RxHUgOT+a9ke+RFJbEXf+7i7nb5ga6SCdks5i4eUAq3z0whLuHtuGb9XsY9sICHv/Peg4WuQNdPCGEaDIkjOtInCOOaSOm0SW2Cw9+9yAzNs4IdJFO6kQjrwf7Rl4XuysCXTwhhAh6EsZ1KNwWzhvD32Bw4mCeWvIUr656tUEPlqoceT232sjrPk/9j0c+zWTdrvxAF08IIYJWjcJYKTVCKbVJKbVVKfXQCfbfrJTKVUqt8n3G+7+ojZPdYufFoS8yqvUoXlv9Gk8teQqPt2E/59vON/L64zv6cX7HeGYuy+Hilxcx6pVFfLB0B4Vl0loWQgh/Ou0MXEopMzAZGA7kAD8rpeZordcfc+iHWuu766CMjZ7FZOGJAU8QbY9m2rpp7C3ay5/7/pnmzuaBLtop9UiJpkdKNH+5tCOzV+5kxtJsHv4kkyc/X89lXRMY2zuZ9JYRKKUCXVQhhGjUajIdZm9gq9Y6C0ApNQMYBRwbxuIUlFLc3/N+mjma8dKKl7h09qXcmn4rN3e6GbvFHujinVKkw8a4Aanc3L8VK3YcYsbSHcxeuZMPlmbTsUU4Y/skM6prAuH2wM3NLYQQjVlNuqlbAtnV1nN82451lVJqjVJqllIqyS+lC0LXd7yeOZfPYWDiQCavmszln13OvO3zGvS95EpKKXqkRPHs6AyW/vl8nhjVCYBHP11Ln6f+xwMfrWb59rxGURchhGhI1On+4lRKXQ2M0FqP963fAPSp3iWtlIoBCrXWZUqp3wJjtNbnneBctwG3AcTHx/eYMcN/I4wLCwtxuVx+O1992Fy6mVkHZ7G7fDft7O24OupqWthaHHVMQ6+X1ppth70syK5gye4KSj2Q6FIMTrLSP8GC03p8F3ZDr9OZCsZ6SZ0aj2CsV7DVaejQocu11j1PtK8mYdwPmKS1vtC3/jCA1vrvJzneDBzUWkec6rw9e/bUy5Ytq0Hxa2bBggUMGTLEb+erLxXeCmZumsnkVZMpKi/i2vbXckfGHUSEGH98jalehWUV/Gf1LmYs3cHqnHxCLCYuTm/Btb2T6dUqqurecmOqU20EY72kTo1HMNYr2OqklDppGNfknvHPQFulVCqwE7gWuO6YC7TQWu/2rV4GbDiL8jYpFpOF6zpcx8jUkbyy8hXe3/A+X2Z9ycTuE7myzZWBLl6tuEIsjO2dzNjeyazblc+Mpdl8unInn6zcSes4J2N7J3Nl98RAF1MIIRqc094z1lpXAHcDX2OE7Eyt9Tql1ONKqct8h92jlFqnlFoN3APcXFcFDlZR9ige7fcoMy+dSWpEKo8vfpyxX4wlq7RhvXCipjolRPDE5Z1Z8udhPHt1FyJCrTz5xQb6/u1/TF5VyhdrdsuEIkII4VOTljFa6y+BL4/Z9li15YeBh/1btKapfXR73hnxDnO3zeX5Zc/z4sEX2bRwE/f3uJ94Z3ygi1drDpuF0T2TGN0ziU17Cvhg6Q4+WbaNu95fgd1qYki7ZoxMb8557ZsRJqOxhRBNVI3CWNQvpRQjU0cyOHEwj33xGPO2z2N+9nwmpE/gxk43EmIOCXQRz8g5zcOYdFknBoXtw5HSha8ydzN33R7mrtuDzWxiYNtYRqa3YHiHeCIcEsxCiKZDwrgBc1gdXBJ5CfcNu4/nlj3Hyytf5pMtn/BgrwcZkjSk0U62YVKKvmkx9E2L4S+XdmJldh5fZu5h7to9/G/jPiwmRf82sVzUuTnDO8YT42qc//gQQoiakjBuBBLDEnlp6Ess3rWYfyz9B/fMv4f+Cf35Y68/khaZFujinRWTSVXN9PXIxR1Yk5PPl2t3M3ftHh76JJM/zc6kb1oMIzs358JOzWkW3rAnSBFCiDMhYdyI9Evox0eXfcSHGz/k1VWvctWcqxjbYSx3ZNxBmC0s0MU7a0opMpIiyUiK5KER7Vm/+zBz1+7hi8zdPPrZOh6bs45eKdGM6NycEZ2bkxAZGugiCyGEX0gYNzJWk5XrO17PRWkX8fKKl/nX+n/xRdYX3NPtHi5rfRlWc3Dca1VK0Skhgk4JEdw/vB1b9hXyVeYevlq7m8c/X8/jn6+na1IkF6U3Z2TnFiRFOwJdZCGEOGMSxo1UtD2aSf0ncc051/D3JX9n0uJJvLzyZS5vczlXt72apPDgmZFUKUW7+DDaxYdx7/ltycot5Ku1RjD/7cuN/O3LjXRuGc6gtnH0bx1Lz1ZR2K3mQBdbCCFqTMK4kesY05HpI6ezaOciPtr8Ee+ue5epa6fSr0U/Rp8zmiFJQ7CagqO1XCktzsVdQ9tw19A27DhQzNx1u/nvur28sTCLVxf8gs1iokdyFAPaxNC/TSxdWkZgMcuru4UQDZeEcRBQSjEwcSADEweyt2gvn2z9hE+2fML9C+4nNjSWK9pcwVXtrqKl60Tv92jckmMc3DaoNbcNak1hWQVLfz3Aj1sP8MMvB3juv5vhv5sJC7HQJy2a/q1j6d8mhnPiwxrtSHQhRHCSMA4y8c547si4g9vSb6tqLb+99m3eynyL/i37M7rdaAYnDsZiCr7/9K4QC+e1j+e89sbkKAcKy1icdYAfth7gx1/2M2/DPgBiXTb6tY5lQOsYBrSJlfvNQoiAC76/kQUAZpOZwUmDGZw0mD1Fe/h4y8d8suUT7pt/H81Cm3FF2yu4qu1VtHC1OP3JGqkYVwiXdEngki4JAOTkFfPjLwf4cet+fvjlAP9ZvQuApOhQBrSOpX+bWPqlxRAXJs81CyHql4RxE9Dc2Zy7ut7Fb7v8lu9zvmfm5plMWTOFNzPf5NyW5zK63WgGthyI2RTcg54Soxxc09PBNT2T0FqzdV8hP/iC+YvM3cz42Xht9znxYfRvE8OA1rH0So0mIjS47rkLIRoeCeMmxGKyMDR5KEOTh7KrcBcfb/mY2VtmM/HbicQ74rmq7VVc0fYKmjubB7qodU4pRdv4MNrGh3HzgFQqPF7W7TrMD7/s58etB3h/yQ6m/bANpaBdszC6p0TRw/dpFeOQe85CCL+SMG6iElwJTOw2kdszbmdh9kI+2vwRr61+jdfXvM6gxEGMbjeaAQkDgr61XMliNlVNOHLnkDaUlntYsSOP5dvyWL4jjy/W7OKDpTsAiHba6J58JJy7JEbIo1RCiLMiYdzEWU1WhqUMY1jKMHIKcqpaywuyF9DC2YIRqSMY1HIQGc0ygu4RqVOxW83G6OvWsQB4vZpfcgtZtj2P5dvzWLE9j3kb9gJgMSk6tYwg3lRGUfRueraKIl6m7RRC1IKEsaiSGJbIvd3v5c6MO5mfPZ+Pt3zMe+veY9raaYRZw+jfsj8DWw7k3JbnEhMaE+ji1iuT6Ui39tjeyQAcLHKzYrvRcl6+PY/52yv47/srAGgZGVrVcu6REkX75mHyrLMQ4qQkjMVxrGYrF7S6gAtaXUChu5Cfdv/EwpyFfL/ze77e9jUKRefYzgxsOZBBiYPoENMBk2p6QRPttHF+x3jO72g8SjXv2/nEte3GMl/LecmvB5jjG7EdajXTNSmyKpy7JUcS6bAFsvhCiAZEwlicksvm4vyU8zk/5Xy82svGgxurgvm11a/x6upXibHHcG7LcxmUOIh+Cf2C4qUVZ8JiOvKii1vPTUVrza780qpu7eXb83jtu1/weDUAabFOuiVH0T0lku7JUbSLD8NskoFhQjRFEsaixkzKDxxCyQAAGStJREFURMeYjnSM6cjtGbdzsPQgP+z8ge9zvufb7G/57JfPsCgL3eK7VbWa0yLSmuzIY6UULSNDaRkZymUZxrPOxe4KVmfns2JHHit35DF/0z4+XpEDGJOWZCRF0D3ZaDl3S4oiyimtZyGaAgljccai7dFc2vpSLm19KRXeCtbkrqlqNb+w/AVeWP4CLV0tq1rNvZv3xm5p2gObHDYL/VrH0K+1cc9da832A8W+cD7Eih15vLpAWs9CNDUSxsIvLCYL3eO70z2+O/f1uI89RXv4fuf3LMxZyJxf5vDhpg8JMYfQu3lvmhU3o/nB5rSNbNtkHp06GaUUrWKdtIp1cmX3RODUrWenzUxGkhHM3VOk9SxEsJAwFnWiubM5o9uNZnS70ZR5yli+Z3lVOH9f8D0f/+djXFYXGc0y6N6sO92bdadzbOcm33KGE7eedxw0Ws8rthut5+r3nlNjnXRLiuT/27v32LjO887j32fu9+FwqCElkeLFkiVZqdfyLbZbp8y2TdwkiLe7xSJt0KbtLoJga6DForsNkCIIslhg3cUusBsYLdJusN0iqdM67tpYq3Aujpzd1JEvsixZ1tUSJVESSQ0vM5wLZzjDd/84Z4Yz5FAcSZSGM3o+wODc3kO+r84Mf3rPnPOeexIh+mN+dnQHGOgOEA967tqvCJRqNxrG6rbzOr08sf0Jntj+BH/86B/z4g9fxDPi4d3Jdzk8dZhvvPsNwLrneV98H/t79/NQ4iEeSDxA1Bttce1bT0QYjAcZjAf5tf3Lveej46lqQP/fs0lefPdy3X4Bj5OBmBXMA91+BmKBalAPdPsJePTjr9RmoZ9Gdcd1u7oZHRnlMyOfASBVSPHulBXMhycP89cfWPc2A+zs2mn1nHut3nMnP9jiRgQ8Lh4bifPYyPL93rliiUszeS7N5Lg0m+PiTI5LM3n7ARlJcsVy3c/oCXnorwZ0fVhvjfr0vmil7iANY9VyUW+U0YFRRgdGAVgoLXAsecwK6MnDvHL+Ff729N8CsDW4lf2J/TzU+xD7E/u5p+ueu/Ie50YCHhe7+8Ls7lt9a5kxhpls0QroWTuw7dA+cmmOV45drZ72BnA6hG1dPmKOIm8VTrJvW5R92yLs6NZxuZW6HTSM1abjc/l4pO8RHul7BIDyUpnTs6erPec3J97kwPkDAEQ8EfYn9rM/sZ/74vext3svXb6uVlZ/UxIR4iEv8ZCX/Ttiq7aXyktcTS1UA/rSTN76nvrDq/z56+eqQR32uti7LcJH7HDetz3CPVtCuLUXrdQt0TBWm57T4WRvfC9743v5/N7PY4xhfH7cCmc7oF8ff71avjfQy97uvezu3s3e7r3sie9hW3Cb9uiuw+V02N8lB+rWHzyY4rGff5LTk/Mcv5Lm+JUUx6+k+c6bF1hYXALA43Kwpy/Mvm0R7rNDem9fBL/n7r5SXqkboWGs2o6IMBAZYCAywNM7nwZgdmGWkzMnOTVzihMzJzg1c4qfXP4JS8YKjLAnzJ7uPeyO7WZvfC+7Y7sZ6Rq5qx5+cbN8bif393dxf//yGYfykuF8MsPxK2nev2wF9IFjE/zNm9YzoR0CI1tC7KvpRd+3LaJDgCq1Bg1j1RFivhiPb3ucx7c9Xl2XL+U5M3uGkzMnq0H9wukXWCgvANbV2zu7dlbDuTINuANr/RplczqEnYkwOxNhnn5gO2B9L315Lm/3oNN8cCXFm+dneOnIlep+27v83Lctwr29Ie7tDbMrEWZkS1AfQanuehrGqmP5XX7u33I/92+5v7qutFTiYvpitfd8YuYEr118jRfPvAiAIOyI7GBP9x72dO9hJDpCf7if/lC/hvQ6RIT+WID+WIBP7uurrp/OFPjgarraiz5xNc1rJ6eq30M7BAbjQXYl7IDuDWlIq7uOhrG6q7gcLka6RhjpGuHTI58GrB7dZG6y7hT3+8n3eXXs1bp94744/eF+BsIDy9OQNe3x97SiOW0hHvLy5K4tPLlrS3VdsbTE+WSW05PznJmc5/RkhjNT8/xoRUgPxYPssnvRO+2wHtkSxOvSkFadpakwFpGngP8GOIG/NMb8pzXK/QvgBeARY8zbG1ZLpW4jEaEv2EdfsI9fHPjF6vp0Mc2l9CUuzV9iPDNuTefHeWfyHV459wqG5VuBfE4fXY4uvvej71k96ZrQ3h7ajtfpbUXTNi2Py9HwNqxCqWyHdIazdkifnprnhyeWQ9rpEAbjgZqedJhdiRBboz4iPjcOHbtbtaF1w1hEnMBzwK8A48BbIvKyMeaDFeXCwB8Ah25HRZW60yKeCPt69rGvZ9+qbYvlRa5kr1QD+tL8JY6cP8Ll7GUOTRwiX8rXlU8EEtWe9I7IDgYjgwxHhxmMDGpQ1/C6nOzpi7CnL1K3vjakrZ70PGemMnUhDdZjLGNBD/Ggh3jIQzzopTvooSfkIR6y5i/OlhlMZomHPIS9Lr3KXm0KzfSMHwXOGmPOAYjI88DTwAcryv0H4Fng321oDZXahNxON4ORQQYjg9V1B7MHGR0dtQbYWJhZ1aMenx/njStv8NKHL1X3EYRtoW0MRYYYjg4zFBliKDrEUGSIRCChQWG7Xkifu5bl7FSGqfkCM9kC05ki09ki05kCR2fnmM4UmS+U6vb7j4cOAuBxOugOeui2w7vHDmwryD0kIj76u/xsj+nwoer2aubdtR24VLM8Dny0toCIPAgMGGNeERENY3VXExHi/jhxf5wHEg+s2p5bzHEhfYGx9BhjqTHOp88zlhrj8NThuh51wBWoBvNQdIjhyDBD0SEGI4P4Xf472aRNy+tysndrhL1bI9ctVyiVmckWmc4U+fE/vsX2e/Ywky2SzFihPZMtkswWGZvOMp0prho6FKA76GF7l5/+mH95GgvYUz8Rn94mp26eGGOuX0Dk14GnjDH/2l7+LeCjxphn7GUH8BrwO8aYMRE5CPxRo++MReSLwBcBent7H3r++ec3rCGZTIZQKLRhP2+z6MR2dWKb4NbbZYxhrjzH1OIUk6VJa7poTWfLs3XfUcecMXrdvSTcCRKuBL3uXrpd3USdUbyOjTvt3YnHqpk2FcqGdMEwVzBM5w3J/BLJBUMyb5jOL5HMG+wxT6oCLoj7HfT4xX45iPuW54NubuuZjrv1WLWTj3/84+8YYx5utK2ZnvFlYKBmud9eVxEGPgIctN9ofcDLIvLZlYFsjPkm8E2Ahx9+2IyOjjbbhnUdPGidIuw0ndiuTmwT3N52LZQWuJC+UO1FV3rVb6feJlfK1ZUNe8L0BnrpDfZaU/uVCCSq6yKeSFPB0InHaiPaZIxhOltkfDbP5VnrYRyX5yrzef7xao5ssVi3T9DjZHvMT38swGA8wHBPkKG49doe8+O8xQvP9Fi1t2bC+C1gl4gMY4Xw54DfrGw0xqSA6n0d1+sZK6Vujs/lY3f3bnZ3765bb4zhWv4aF9IXmMhOMJmbZDI7yVRuqnq71nR+uq5XDdbV35VgTgQS1fBOBBL0BfroDfbS7eu+k01sKyJCT8hLT8jLAwOrx0I3xpDKLzJuh/PlOTuwZ/Ncms3zs3PTdafC3U5hoDtQDefhngCD8SDDPUG2dd16UKvNb90wNsaUROQZ4FWsW5u+ZYw5LiJfB942xrx8uyuplGpMREgEEiQCiTXLLC4tkswlmcxNMpGbYCprBfVkzgrtw5OHmcpNUTL1Fzm5xEXYEWbwwGB9L7tmvifQo0OKNiAidAU8dAU8fGT76mdyG2O4Nl/gfDLL2HSWsekcY8ks55NZ3vhwmvzi6qAejgcZ6rFfcSu4Nag7R1OXBxpjDgAHVqz76hplR2+9WkqpjeJ2uNka2nrdZ0EvmSVmFmaqPevK9Oj5o4hTODlzktcvvV4dSrRCEHr8PatOg1emfYE+EsGE3r61goiQiPhIRHx8tOaZ1GAF9VQlqJPLQT02neWnHyarD+gA62rwgW4/wz1BypkC7y6epifsZYt9K5fVe/cQ0lu4Nj29Vl8phUMc9Ph76PH3sC++fF/1wfnl7+yMMaSL6brArpwOn8xOcnH+Im9NvMX84vyqn9/l7aqG9Bb/FmK+GF3eLrq8XXXzXb4uwu7wXR0cIkJvxEdvxMdjK4J6aakmqKftVzLLhekc49MlfnzpTMOf6XU5qsFcOb3eE7bm4/b6Lfb6qF8HTmkFDWOlVFNEhKg3StQb5d7YvWuWyy3mqqfB60Lbnj+ePE6qkFp1WrzCKU6i3mjjsLYDO+aNEfVGq9vCnjAO6fxnKjscQl/UR1/Ux+P31Af1wYMH+fknP8Zstsi1TIFkpkhyvkAyU2A6a81fyxS4klrg2OUU09li3YApFS6H2AOleIkF3QQ9LoJeFwGPk5DXRcDjIuh1VtdVtge9TgIel1XGa63XU+jN0zBWSm2ogDvAcHSY4ejwmmWMMWQWM8wtzDFXmGO2MEuqkGJ2YZa5wlzd60L6Au8V3mOuMEdpqXGAO8RB1BOt/mehy9tVN19Z7vJ2MV4cZyI7QdQb7bj7td1OR/X093qWlgxz+UWSmYL9qgnvTJFkpsBszro3O1sskSuUyRZLdafJ1+NzOwh6lsM56HUR9rnY0W1dTT7cE2SkJ7QhV5O3Ow1jpdQdJyKEPWHCnjADdXdOrs0YQ3YxWw3p2uCeXZglXUxXlydzk5yaPUWqkFo1NCnAsy88C4DX6bVC3Lcc2hFPpDof98frvgP3udYPuXbhsHvA3UEP9/aG19/BViovkVsskyuUyRRK5IolsoUy2ULJCu2iPV8okyuW7DLL25OZAu+MzdaNiuZxOqq3ew1vCTLSE2S4J0SqYDDG3BVfW2gYK6XagogQ8oQIeUL0h/ub3q9QLpAqpJgrzJEqpPjp4Z/Sv7O/ulyZpgopPpz7kLnCHOlCuuFp9Mp3333BvmpIV+ft5U7rba/kcjqIOB23NOKYMYZkpsj5ZJbzyQznklnOX8tyLpnlx6emWCwvnz7/kze+b4ezFdDLYR0k6O2cCOucliilVANep7fu9q9sIMvovaPX3afSC0/m7VvCau7hnshNMJGd4L1r1qnzlaLeaH1gV+Zrbgnzu/x3RW9vLSLClrCXLWEvjw7X389eXjJcns1zLpnhB2+8hzO2lfPJLG+NzfLSe1eoHTSyN+KthnQi7CXidxPxuYj43UT9biI+NxG/tRzyuDb1hWkaxkoptUJtL3woOrRmuXwpX3dx2srQPnbtGLOF2VX7OcVp/Xx3iLAnTMht/a6w2zp1X5mv1KEyX13nDnVsoDsdwo54gB3xAFx1Mzr6keq2hcUyY9PLvejz9uvV4xPMZIvX+akgAmGvyw5sO6z9Ljuwa4LbXo7ar5WP+bxdNIyVUuom+V3+VU/vWmmhtFC9BWwiO8G1/DXmi/PMF+fJLGbIFDPMF+e5krlizS/OkylmVo2atpJLXHWBXsgW+Pb3v43X6V1+ubz1yyvWe5wefE5fdbqyTMAVIOgObprQ97kbP70LrB51ZqFEemGRVH6R9MIi6fwi6XxpeX6hRDq/vH0smatuyzZ4OEjY5+LY1z55J5qmYayUUreTz+VjR2QHOyI7mt5nySyRW8yRWcxUQ3u+aIX0ynWV+YncBPlSnlQhxUJ5gWK5yELJnpYXWFxavKn6u8RFxBtZdVV67fzKK9e7vF24nXd2ZDanQ4gG3EQD7iYvCay3WF5i3g5rK6BLFMurA/p20TBWSqlNxiGO6inqvmBfU/us91CFJbNEoVxYFdKVaaFcoFAqUFhaLpNbzFWvUE8VUqSKKcYz4xxPHmeuMEdxae1TwwFXYFVQV+b9Lj9+lx+v04vP5cPv9ON1efE5ffhdfnwuq5eeLqfJLmbxOr24HLc3rtw1z7ZuBQ1jpZS6CzjEUQ3BqHf1eNk3yhhDvpSvu6VsrjBHasG+Qr2Yql6tPleY42r2avWq9fVOwdf6yne+AoDL4aoLbZ/Ltzx1+Qh7wtYgMSsGhKkMHLPZ7yvXMFZKKXXDRISAO0DAHWi69w5WiBeXrJ73QmmBhbI1zZfyFMoFa76cp1AqcPTEUXaM7KgrV53WlJtdmOVi+qJ1W1oxvebv9jl9q4K6MqpbNbi9sbrlgDuwEf9c69IwVkopdceISPUisfV66NHxKKP7Rm/o55eWStUe+GzBHhhmoX6Ut8q2q9mr1fvKG/XWA64Ahz5/6IZ+/83SMFZKKdUxXA4XcX+cuD++fmFbealMupi2wnth+ZT7YvnmLnq7GRrGSiml7mpOh5OYL0bMF4Nb/zr9pnT+Y06UUkqpTU7DWCmllGoxDWOllFKqxTSMlVJKqRbTMFZKKaVaTMNYKaWUajENY6WUUqrFNIyVUkqpFtMwVkoppVpMw1gppZRqMQ1jpZRSqsU0jJVSSqkW0zBWSimlWkzDWCmllGoxDWOllFKqxZoKYxF5SkROichZEflyg+1fEpFjInJERP6fiNy38VVVSimlOtO6YSwiTuA54FeB+4DfaBC23zHG/Jwx5gHgT4H/uuE1VUoppTpUMz3jR4Gzxphzxpgi8DzwdG0BY0y6ZjEImI2rolJKKdXZXE2U2Q5cqlkeBz66spCI/D7wbwEP8E83pHZKKaXUXaCZMG6KMeY54DkR+U3gT4AvrCwjIl8EvmgvZkTk1Eb9fqAHSG7gz9ssOrFdndgm6Mx2aZvaRye2q9PaNLjWhmbC+DIwULPcb69by/PAnzXaYIz5JvDNJn7nDRORt40xD9+On91KndiuTmwTdGa7tE3toxPb1YltWksz3xm/BewSkWER8QCfA16uLSAiu2oWPw2c2bgqKqWUUp1t3Z6xMaYkIs8ArwJO4FvGmOMi8nXgbWPMy8AzIvLLwCIwS4NT1EoppZRqrKnvjI0xB4ADK9Z9tWb+Dza4Xjfjtpz+3gQ6sV2d2CbozHZpm9pHJ7arE9vUkBijdyEppZRSraTDYSqllFIt1nZh3MTQnF4R+a69/ZCIDN35Wt4YERkQkR+LyAciclxEVp32F5FREUnZQ44eEZGvNvpZm4mIjNUMk/p2g+0iIv/dPlZHReTBVtSzWSKyu+bf/4iIpEXkD1eUaYvjJCLfEpEpEXm/Zl23iPxARM7Y09ga+37BLnNGRDbN9SFrtOk/i8hJ+/319yLStca+132vttIa7fqaiFyueZ99ao19r/v3slXWaNN3a9ozJiJH1th30x6rW2KMaZsX1gVkHwIjWIOLvAfct6LMvwH+3J7/HPDdVte7iXZtBR6058PA6QbtGgX+T6vreoPtGgN6rrP9U8A/AAI8BhxqdZ1voG1OYAIYbMfjBHwMeBB4v2bdnwJftue/DDzbYL9u4Jw9jdnzsVa35zpt+gTgsuefbdQme9t136ubsF1fA/5onf3W/Xu5mdq0Yvt/Ab7absfqVl7t1jNed2hOe/mv7PkXgF8SEbmDdbxhxpirxpjD9vw8cAJr5LNO9zTwv4zlZ0CXiGxtdaWa9EvAh8aYC62uyM0wxvwEmFmxuvaz81fAP2uw6yeBHxhjZowxs8APgKduW0VvQKM2GWO+b4wp2Ys/wxonoa2scaya0czfy5a4Xpvsv9f/EvibO1qpFmu3MG40NOfK0KqWsT+EKSB+R2q3AezT6vuBQw02Py4i74nIP4jIvjtasZtjgO+LyDv26GsrNXM8N6vPsfYfi3Y7ThW9xpir9vwE0NugTDsfs9/DOhPTyHrv1c3oGfv0+7fW+EqhXY/Vk8CkMWat8Sra8Vitq93CuKOJSAj4HvCHpv7hGwCHsU6J/hPgG8D/vtP1uwm/YIx5EOuJX78vIh9rdYU2gj34zWeBv2uwuR2P0yrGOh/YMbdaiMhXgBLw7TWKtNt79c+Ae4AHgKtYp3U7xW9w/V5xux2rprRbGDczNGe1jIi4gCgwfUdqdwtExI0VxN82xry4crsxJm2MydjzBwC3iPTc4WreEGPMZXs6Bfw91mmzWjc61Opm8avAYWPM5MoN7XicakxWviawp1MNyrTdMROR3wE+A3ze/k/GKk28VzcVY8ykMaZsjFkC/oLG9W3HY+UC/jnw3bXKtNuxala7hfG6Q3Pay5UrPH8deG2tD+BmYX9H8j+AE8aYhs+CFpG+ynffIvIo1rHbtP/JEJGgiIQr81gX0ry/otjLwG/bV1U/BqRqTpNuZmv+z73djtMKtZ+dLwAvNSjzKvAJEYnZp0Y/Ya/blETkKeDfA581xuTWKNPMe3VTWXFtxa/RuL7N/L3cbH4ZOGmMGW+0sR2PVdNafQXZjb6wrsA9jXWV4FfsdV/H+rAB+LBOH54F3gRGWl3nJtr0C1inBI8CR+zXp4AvAV+yyzwDHMe6IvJnwBOtrvc6bRqx6/qeXe/KsaptkwDP2cfyGPBwq+vdRLuCWOEarVnXdscJ6z8TV7GGsB0H/hXWtRU/whpb/odAt132YeAva/b9PfvzdRb43Va3ZZ02ncX63rTyuarcabENOHC99+pmea3Rrr+2PzNHsQJ268p22cur/l5uhlejNtnr/2fls1RTtm2O1a28dAQupZRSqsXa7TS1Ukop1XE0jJVSSqkW0zBWSimlWkzDWCmllGoxDWOllFKqxTSMlVJKqRbTMFZKKaVaTMNYKaWUarH/D+nCoOuQDBjfAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "gp2X_yOHq7Ht", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "43006a3a-8d4d-41d0-d82f-13f9adecca41" }, "source": [ "y_pred = model.predict(X_test)\n", "y_pred = y_pred.argmax(axis=1)\n", "print(classification_report(y_test, y_pred,\n", " target_names=class_names))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " T-shirt/top 0.82 0.83 0.83 1000\n", " Trouser 0.99 0.96 0.97 1000\n", " Pullover 0.78 0.77 0.78 1000\n", " Dress 0.84 0.90 0.87 1000\n", " Coat 0.78 0.79 0.79 1000\n", " Sandal 0.94 0.94 0.94 1000\n", " Shirt 0.69 0.64 0.66 1000\n", " Sneaker 0.92 0.94 0.93 1000\n", " Bag 0.95 0.96 0.96 1000\n", " Ankle boot 0.94 0.94 0.94 1000\n", "\n", " accuracy 0.87 10000\n", " macro avg 0.87 0.87 0.87 10000\n", "weighted avg 0.87 0.87 0.87 10000\n", "\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "4v8L9Ku9rsi1" }, "source": [ "# 4. Optimizers\n", "\n", "Optmizers help in reducing the loss in your network via gradient descent following optimized strategies to get a better minima." ] }, { "cell_type": "markdown", "metadata": { "id": "US10MYNSsrme" }, "source": [ "## Gradient Descent\n", "\n", "Gradient descent is a first-order optimization algorithm which is dependent on the first order derivative of a loss function. It calculates that which way the weights should be altered so that the function can reach a minima.\n", "\n", "GD: θ=θ−α⋅∇J(θ)\n", "\n", "Need to compute gradient over the whole dataset which can be inefficient\n", "\n", "\n", "## Stochastic Gradient Descent\n", "\n", "It’s a variant of Gradient Descent. It tries to update the model’s parameters more frequently. In this, the model parameters are altered after computation of loss on each training example. So, if the dataset contains 1000 rows SGD will update the model parameters 1000 times in one cycle of dataset instead of one time as in Gradient Descent.\n", "\n", "θ=θ−α⋅∇J(θ;x(i);y(i)) , where {x(i) ,y(i)} are the training examples.\n", "\n", "Can lead to high variance\n", "\n", "## Mini-Batch Gradient Descent\n", "\n", "It is an improvement on both SGD and standard gradient descent. It updates the model parameters after every batch of data. So, the dataset is divided into various batches and after every batch, the parameters are updated.\n", "\n", "θ=θ−α⋅∇J(θ; B(i)), where {B(i)} are the batches of training examples.\n", "\n", "## Momentum\n", "\n", "Momentum was invented for reducing high variance in SGD and softens the convergence. It accelerates the convergence towards the relevant direction and reduces the fluctuation to the irrelevant direction. One more hyperparameter is used in this method known as momentum symbolized by ‘γ’.\n", "\n", "V(t)=γV(t−1)+α.∇J(θ)\n", "\n", "V is the exponentially weighted average of past gradients\n", "\n", "Now, the weights are updated by θ=θ−V(t).\n", "The momentum term γ is usually set to 0.9 or a similar value.\n", "\n", "## Adaptive Moment Estimation\n", "\n", "Adam (Adaptive Moment Estimation) works with momentums of first and second order. The intuition behind the Adam is that we don’t want to roll so fast just because we can jump over the minimum, we want to decrease the velocity a little bit for a careful search.\n", "\n", "M(t) and V(t) are values of the first moment which is the Mean and the second moment which is the uncentered variance of the gradients respectively.\n", "\n", "![](https://i.imgur.com/8qZUgZJ.png)\n", "\n", "Final paramter updates occur as:\n", "\n", "![](https://i.imgur.com/RmA9ghf.png)\n", "\n", "η is a learning rate which is modified for given parameter θ(i) at a given time based on previous gradients calculated for given parameter θ(i).\n", "\n", "Usual values for β1 is 0.9 , 0.999 for β2, and (10 x exp(-8)) for ‘ϵ’.\n", "\n", "[Source](https://towardsdatascience.com/optimizers-for-training-neural-network-59450d71caf6)\n" ] }, { "cell_type": "markdown", "metadata": { "id": "8MTPK92vvehs" }, "source": [ "## 4.1: Train DNN with Adam" ] }, { "cell_type": "code", "metadata": { "id": "nV3MiOhCq9LQ" }, "source": [ "model = tf.keras.models.Sequential([\n", " tf.keras.layers.Flatten(input_shape=[28, 28]),\n", " tf.keras.layers.BatchNormalization(),\n", " tf.keras.layers.Dense(300, use_bias=False),\n", " tf.keras.layers.BatchNormalization(),\n", " tf.keras.layers.Activation(\"relu\"),\n", " tf.keras.layers.Dense(100, use_bias=False),\n", " tf.keras.layers.BatchNormalization(),\n", " tf.keras.layers.Activation(\"relu\"),\n", " \n", " tf.keras.layers.Dense(10, activation=\"softmax\")\n", "])" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "D9Jaq3otvVaP", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "8e8af326-b2d5-45b8-ff8e-2745b8e7e75a" }, "source": [ "model.compile(loss=\"sparse_categorical_crossentropy\", \n", " optimizer=tf.keras.optimizers.Adam(lr=1e-3, \n", " beta_1=0.9, \n", " beta_2=0.999),\n", " metrics=[\"accuracy\"])\n", "model.summary()" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Model: \"sequential_5\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "flatten_5 (Flatten) (None, 784) 0 \n", "_________________________________________________________________\n", "batch_normalization_6 (Batch (None, 784) 3136 \n", "_________________________________________________________________\n", "dense_15 (Dense) (None, 300) 235200 \n", "_________________________________________________________________\n", "batch_normalization_7 (Batch (None, 300) 1200 \n", "_________________________________________________________________\n", "activation_2 (Activation) (None, 300) 0 \n", "_________________________________________________________________\n", "dense_16 (Dense) (None, 100) 30000 \n", "_________________________________________________________________\n", "batch_normalization_8 (Batch (None, 100) 400 \n", "_________________________________________________________________\n", "activation_3 (Activation) (None, 100) 0 \n", "_________________________________________________________________\n", "dense_17 (Dense) (None, 10) 1010 \n", "=================================================================\n", "Total params: 270,946\n", "Trainable params: 268,578\n", "Non-trainable params: 2,368\n", "_________________________________________________________________\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "tx2P6N3YvbH_", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "14e3c1ae-dd6b-424e-ce87-12f4bb655ca5" }, "source": [ "history = model.fit(X_train, y_train, epochs=20,\n", " validation_split=0.1)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Epoch 1/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.4620 - accuracy: 0.8336 - val_loss: 0.3520 - val_accuracy: 0.8715\n", "Epoch 2/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.3569 - accuracy: 0.8679 - val_loss: 0.3220 - val_accuracy: 0.8820\n", "Epoch 3/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.3192 - accuracy: 0.8832 - val_loss: 0.3124 - val_accuracy: 0.8848\n", "Epoch 4/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.2930 - accuracy: 0.8904 - val_loss: 0.3089 - val_accuracy: 0.8880\n", "Epoch 5/20\n", "1688/1688 [==============================] - 7s 4ms/step - loss: 0.2726 - accuracy: 0.8974 - val_loss: 0.3152 - val_accuracy: 0.8878\n", "Epoch 6/20\n", "1688/1688 [==============================] - 7s 4ms/step - loss: 0.2540 - accuracy: 0.9041 - val_loss: 0.3133 - val_accuracy: 0.8850\n", "Epoch 7/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2393 - accuracy: 0.9109 - val_loss: 0.2908 - val_accuracy: 0.8937\n", "Epoch 8/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2279 - accuracy: 0.9147 - val_loss: 0.2947 - val_accuracy: 0.8938\n", "Epoch 9/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.2170 - accuracy: 0.9167 - val_loss: 0.3080 - val_accuracy: 0.8897\n", "Epoch 10/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.2038 - accuracy: 0.9240 - val_loss: 0.3020 - val_accuracy: 0.8960\n", "Epoch 11/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.1943 - accuracy: 0.9264 - val_loss: 0.3150 - val_accuracy: 0.8913\n", "Epoch 12/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.1865 - accuracy: 0.9298 - val_loss: 0.3088 - val_accuracy: 0.8972\n", "Epoch 13/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.1748 - accuracy: 0.9337 - val_loss: 0.3247 - val_accuracy: 0.8963\n", "Epoch 14/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.1695 - accuracy: 0.9369 - val_loss: 0.3198 - val_accuracy: 0.8958\n", "Epoch 15/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.1628 - accuracy: 0.9386 - val_loss: 0.3320 - val_accuracy: 0.8923\n", "Epoch 16/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.1563 - accuracy: 0.9412 - val_loss: 0.3145 - val_accuracy: 0.8958\n", "Epoch 17/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.1493 - accuracy: 0.9436 - val_loss: 0.3347 - val_accuracy: 0.8960\n", "Epoch 18/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.1443 - accuracy: 0.9449 - val_loss: 0.3332 - val_accuracy: 0.8928\n", "Epoch 19/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.1390 - accuracy: 0.9471 - val_loss: 0.3501 - val_accuracy: 0.8957\n", "Epoch 20/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.1356 - accuracy: 0.9492 - val_loss: 0.3648 - val_accuracy: 0.8903\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "G4GuQd24vinz", "colab": { "base_uri": "https://localhost:8080/", "height": 324 }, "outputId": "c242579f-c5fe-4eee-fd62-6d2ba6bb6f11" }, "source": [ "pd.DataFrame(history.history).plot(figsize=(8, 5))\n", "plt.grid(True)\n", "plt.gca().set_ylim(0, 1);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeMAAAEzCAYAAAACSWsXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3xc1Z338c+ZPqORRqMuWbIlV7nI3aaYYkMAUw0kYEgglAAJoW1INvAQNrAJqSS7STYE8BIgsBAvIWtCCAuLg4VxaLbBBTdZ7nJVl0Zl6nn+uKPRSJYs2ZY90vj35jWv285cnSPh+c45tymtNUIIIYRIHFOiKyCEEEKc6iSMhRBCiASTMBZCCCESTMJYCCGESDAJYyGEECLBJIyFEEKIBOszjJVSzyqlDimlPu9lu1JK/UYpVamUWqeUmj7w1RRCCCGSV396xs8D84+w/WJgTPR1B/Dk8VdLCCGEOHX0GcZa6+VA3RGKLABe0IaPgHSlVP5AVVAIIYRIdgNxzHgYsCduuSq6TgghhBD9YDmZP0wpdQfGUDZOp3NGUVHRgO07EolgMiXf+WjJ2K5kbBMkZ7ukTUNHMrYr2dpUUVFRo7XO7mnbQITxXiA+VQuj6w6jtV4ELAKYOXOmXrVq1QD8eEN5eTlz584dsP0NFsnYrmRsEyRnu6RNQ0cytivZ2qSU2tXbtoH4yvE68NXoWdWnA41a6/0DsF8hhBDilNBnz1gp9UdgLpCllKoCHgGsAFrrp4A3gUuASqAVuOVEVVYIIYRIRn2Gsdb6+j62a+CuAauREEIIcYpJniPjQgghxBAlYSyEEEIkmISxEEIIkWASxkIIIUSCSRgLIYQQCSZhLIQQQiSYhLEQQgiRYBLGQgghRIKd1AdFCCGEEEekNYT8EPZjDTRAY5WxHF1HKAChdggH4tZ1bI9bFw4evi4U6GUaLd99my0Fvr3ppDRbwlgIIZJdR8CF2g+fhqPh1uO2oPGKhKKvcHQa7LYcipaLW450Xw5BuGPaPUDbO0MwHIhVew7AB8fRbrMNzHaw9Da1G4HrzOi6zmwzpvbU4/3N95uEsRBCnGhadwZWR08t1GZMg21xIRh9Bdu7Lncp1/P7ptcdgk32zpDrEqr+gW2PyQomC5itYDIb8yZLdH38sgXMlq7LFhuYvUbYWeydARgfgtH1FTt2MbZ0ElgcXYPU4jg8XC2OaPjaOvel1MC2+wSSMBZCJLdwEIKtRngFWkjx7YQ9KyHYYqwLtkKgtXM+9mrr2jsMB6KBGujsLXbMh4NGTzAciPb+4ubDAWP5eJks0VCKvqxx8xY7IYsb0guiYeboZWqPW+6tTNzUZI0L02gAn8TnC+8LlDN2xtyT9vMSScJYCDF4hPzg90GgOTqNvvzx0+Zuyz7wN3cGaPdw7RaEswD6epS6xQk2l9HrMlujL1u0p2frXGd1RZejYdUxb7ZFl6Ov7tssduNnWOxgdcYtO3pYFw1G85E/rtcl2bN/TzUSxkKIo6O10dsLtMQFX0t02hq3vqftbUcO2P72IM02sLnB7gZbqjG1p4I71wgyqxOsKdGpywjW6PznFTuYNHVWZzlbXDmrK9ojlAtNxMklYSzEUBeJQFu9EXzxxwrDvZywE/J3O67Y+0k902oOwCbr4WGrw0dXx46epjXFmHYEqTvHCNFYsLr7Xra5jWOEx6imvhzGzD3m9wtxIkgYCzEYaQ3tjeA7BL6DxqulOjp/KG56yFh/tOHYQZnihkLtXY8Zmu1ETDZIL4zrXUbD1OqK9ijjp71st7qkpylEHySMhTjRIuGu1z4GfJ1BGh+u3cM27hKPGJPV6E26cyCtAPKnGEOzKdlG+HU5CcfW80k5Znu/j0OuleOQQpwUEsbi1BIJR49ZtsSdINTSdeqPm48N+QY57OYAsZsKGPOnNTfAp5a4GwdEr5vUkT4qpYww7QjZrLHGNCXHCFp33NTpHVKXawgh+kfCWAwt4RC01hi9yJZqaKnpnMafFBQfsP7m6HKLcY1mf1mcxuUj5o7rH22d1zF2nBHr8MS2Nal6nAXDjbNnLR1n4ca/N7rOlhIN2mjIujL77KEKIZKbfAKIxNIa/E1xoVodPQ7aU+BWQ1tdz/sxWTpP9LG5jcCzpXQO39pSOrfZ3V2XO7bbU+PWpxg3LzgKm8rLyZUhXSHEMZAwFgMv2Na1x9qtJ1u2pwK2fL+zTE/HRgEc6UaYpmRDTimknB1dzupcn5JjLDs8MnwrhBiyJIxF38LBbqHaw7S123BxT8x2cOdgC9sgcxTkTuoWrHEvV+ZxXb4ihBBDiYTxqcrv6zYsHB0a7pj3VXdub2/oeR8mC7g6wjQTvMWdPVdXfMhmRoeL3aAUq+UM3WOmIxHCjY2E6+oI19cTqqsjXN9AuLERk8OOyZ2KOS0VkzsVU6obc2oqptRUzG43ymo9cfUKhwk3NRl1aTjyK9LSYoximE0ok9mYKhOYzSiTybgM6gjblNlkHEKIbYuWNZlAKZRJGZdsKQUmhVLRZZPJOFdu9x6q13/eud1kApSxb5My1kffr6xWTC4XJpcTk9OJyeVCOV2HrztBv1utNToQINLaim5rI9LaSqStjUhLK5G2uHWtbbgqKqjdsbPjjXFT3WWdjm3rXo7DymIyY3I6UU4HJocTk8uJcjiMdju7zTudKKvV+H0PUNsJhQg3N6Pb24n4/UZ72/1of7sxbe9cVjY7Zk8aZo8HU1oa5vR0zKmpKMvQiLmhUUvRN62N0PRVR8O047hr3LWoseCtNm7e0JOOoWF3DuROiA4DdwwNxwdsllH2JA0Na62N8DlwgOCBgwQP7Cd04CChg8ZyqLoawmE0R/igiZt2KddLGQBls2FyuzG5UzClpGBOcWNKSem6zm2ss23fTmuaJ7rO2GZKSTniB3UkEDCCta6OUH094bp6wvVx8x2hWx+db2w0bvJxDJTTadQ1tSOo04ypOxrYqW5MqWnRaSomtxvr5s00tbcTrq/vEqihLgHbSKSpKe4DvRuz2fhg7HhlZUbv4hUBHUGHI8bfLhBARyLoSHQ5EjHaGgkbZXrYpiNhYz/hsPGnjESiD2WIGB/mPcynaE1Nb3U9Vh2h7ewMaJPTiUpxYXK6Ote5nCirjYi/3QjY1riAjQarbm3tso5w/64hTwUODWyrjp7ZjMnhQHX8Hnqat9uIBALotnYi/nZ0u9+Yxi3r9nYi7e3kRiJUHGeVTCkpRkB7PJjTjLDuDO3ounRjakrzdM673dEvaieHhPFgpjWWoA+qK9DNB9AN+9B1B4jU7yPSeAjdUEOkqZaIrwHtazQu2wFQcR80ygT2NHB6wJGGcowB5yzISAOH11jvTDeOuTrSjbN9o/mqoj0D5XBicthRDgfKbsdkc2AyO4wyA9HMSIRwXZ0Rqgf2G9OOkD1wgOCBA4QOHkQHuh1btliw5GRjzc3DPmZM5zdgpTqncW0hvsZKdS0X35i4Lxg6ECDiayHi8xGuqSW4cxfh1hYivhZ0W9czs73Arv/47WHtUw5HNMCj4Wy2GOFWV0ektZcvRSaTEVwZXizeDOyjRmGeNROz11g2Z2Rg9qZjyYjOp6URaW8n4vMRaW4m3Owj4msm3NRsLPuaiXRf19REcO9eY1tTM9p/+JN9MoC98dVyuboEq21YYdeg9aZ3XU5PNz7UBtHx/PLycs4991wjnHsIa2MZ44tCINA1JDtCs4eeaWeAtsaCNlxTS7Ctqss+dDBo/DvqCO+UaG/b6cSalxe3zmkEeUc5V0fP3InJldKtt+7kHx99xFlnnRX3/68xVYoe/l8//N9B7C8Ut02HQkYwtrURaWs3eqKHzbcZZVrbiLS3GaHadvh8qKbGKOv3o+x2lMOOye5AOexY09O7LJvsDpTTwa59+xk5fnz08yf6OWR3dF12OFA2OzrgJ9xofDEMNzYSboxOmxqJxOab8G/fZpRrbDr8MyWeyYQlJ4cx5csG6P+8I5MwPk4dw0i6rc0YRmlv7xxGaWuPDqe0o/3+zv9pWxrRjbVEmuuJ+BrRviYirb7oP+52Iu0BdCBIJBAmJwRbwopIWIE+0geap4+a+qKvfQPXeJPJCBq78Q/C5HB0Xbbbo9+EO7Yb/5DcO3aw969vEDx4gND+AwQPHYJgt3sSW61Yc3Kw5OXhnDwZS14u1tw8Y5qXhyUvD0tmJsp8dGc8DyQdChkfwD4fkZYWVr3/PlPGjCXS0kKkxVgXjm7rCPRISws6HMZWUmyEqteLOcNrhGxGBmavEbJmj+eov5WbnE7weo+9PYGAUd+OMG9uYs269cyYOzcWtCZbchzHV/FfxszmAflS2V9a6xPy5UQ7nZhTB/b5u8piAYcDc3r6gO63vzaWl5N5Ag9pRdrbo8HdLcSbGqOjUAM8gnIEEsZROhwmuP8Awao9BPbsIbinimBVFeEW3+HDJx3HLvx+ozdxTENeGmXRmMwak0WjzBqTzYLJbsXstmNyejA5U2gOR0gvKEK5PZjSMjGlZaLSMoxvxk5H7JiNsttRSvV8PKj7caCOXgDdjh91L6s1OhiM+1LRw5eMdr/xDbhjqCmuTLCpscuQU8fvzqUUbfn5WHNzcU6bRlp+HpbcPKx5ucY0Pw9zRsZJHSI6FspiMYa40tIACO7di/vssxJcq2OnbDYsGRmQkRFbF/T7cYwbm8BaJZ/BNEpwqjNFOxHW3NxEV+XUCuOwz0dwT0fYdoZuoGoPwb37IBTqLGyxYM3PN3ooDjtmdyoqKzva++sYTonvDVpRoWZMbQdRLVWYfLtQjTswaZ8RtBYzKncUalgZpvxxKO8wVGr0pg8pOeDK6PG61vLycsYl2clO5cuWMXfevERXQwghBo2kCmMdDhM6cIDAnqpoD7eqS/iGG7qeFWz2eLAWFeGYMIG0i+ZjLSrEVlSEtbAIa15u72fhhYNQvQX2r+18Va03nmwDxiU8wydB/lXGvYPzp0DOBOOuTEKuBxZCiG6SIoyb3nqbzB//mM319V2PPZrNWAsKsBUV4bjoImxFhUbQRkO3Y3jxiILtcGhj1+A9uMG45zAYT6nJnwzTb+wM3qyxxolQQgghRD8kRRibvV5ChYXkLrgCa2ERtuFFWIuKsOblHds1ZsF2+PzPsPo52PcZRKLD13aPEbyzb4f8qUbwZo466tsmCiGEEPGSIoxTTptN4x23M+14j602H4RVv4dVzxrX4maPhzPv7ezxeotliFUIIcSAS4owPm77PoOPnjJ6w5EQjL0ITr8TSs6V8BVCCHHCnbphHA7Blr/BR0/C7g+NWzXOvBVO+7ox9CyEEEKcJKdeGLfVw6cvwieLoHEPpI+Ai34M024w7kIlhBBCnGSnThjXbIWPn4I1Lxv3ZS4+Gy7+GYydLydgCSGESKjkDmOtYdvfjePBle+A2QZl1xpD0fmTE107IYQQAkjWMA60wNrF8PHTULPFuMvVvO/BjFvAnZ3o2gkhhBBdJFcYN+yBlf8Jq/9gPE4wfypctQgmXiUPqhdCCDFoJUcY71/HhA0/h/c+AjSMvxxO/yYUnSaXJgkhhBj0kiOM67bhrV8DZ9xl3B0rfXiiaySEEEL0W3KEcenlfHiGk3POn5/omgghhBBHbXA/MLa/zBYiZkeiayGEEEIck+QIYyGEEGIIkzAWQgghEkzCWAghhEiwfoWxUmq+UmqLUqpSKfVgD9uHK6WWKaU+U0qtU0pdMvBVFUIIIZJTn2GslDIDTwAXAxOA65VSE7oVexh4RWs9DbgO+N1AV1QIIYRIVv3pGc8GKrXW27XWAWAxsKBbGQ2kRec9wL6Bq6IQQgiR3JTW+sgFlPoSMF9rfVt0+UbgNK313XFl8oH/A7xACvAFrfXqHvZ1B3AHQG5u7ozFixcPVDvw+Xy43e4B299gkYztSsY2QXK2S9o0dCRju5KtTfPmzVuttZ7Z07aBuunH9cDzWutfKqXOAF5USk3SWkfiC2mtFwGLAGbOnKnnzp07QD8eysvLGcj9DRbJ2K5kbBMkZ7ukTUNHMrYrGdvUm/4MU+8FiuKWC6Pr4n0NeAVAa/0h4ACyBqKCQgghRLLrTxivBMYopUqUUjaME7Re71ZmN3A+gFJqPEYYVw9kRYUQQohk1WcYa61DwN3A28AmjLOmNyilfqCUuiJa7NvA7UqptcAfgZt1XwejhRBCCAH085ix1vpN4M1u674fN78RmDOwVRNCCCFODXIHLiGEECLBJIyFEEKIBJMwFkIIIRJMwlgIIYRIMAljIYQQIsEkjIUQQogEkzAWQgghEkzCWAghhEgwCWMhhBAiwSSMhRBCiASTMBZCCCESTMJYCCGESDAJYyGEECLBJIyFEEKIBJMwFkIIIRJMwlgIIYRIMAljIYQQIsEkjIUQQogEkzAWQgghEkzCWAghhEgwCWMhhBAiwSSMhRBCiASTMBZCCCESTMJYCCGESDAJYyGEECLBJIyFEEKIBJMwFkIIIRJMwlgIIYRIMAljIYQQIsEkjIUQQogEkzAWQgghEkzCWAghhEgwCWMhhBAiwSSMhRBCiASTMBZCCCESTMJYCCGESDAJYyGEECLBJIyFEEKIBJMwFkIIIRJMwlgIIYRIMAljIYQQIsEkjIUQQogEkzAWQgghEkzCWAghhEgwCWMhhBAiwfoVxkqp+UqpLUqpSqXUg72UuVYptVEptUEp9fLAVlMIIYRIXpa+CiilzMATwAVAFbBSKfW61npjXJkxwP8D5mit65VSOSeqwkIIIUSy6U/PeDZQqbXerrUOAIuBBd3K3A48obWuB9BaHxrYagohhBDJqz9hPAzYE7dcFV0XbywwVin1D6XUR0qp+QNVQSGEECLZKa31kQso9SVgvtb6tujyjcBpWuu748q8AQSBa4FCYDlQprVu6LavO4A7AHJzc2csXrx4wBri8/lwu90Dtr/BIhnblYxtguRsl7Rp6EjGdiVbm+bNm7daaz2zp219HjMG9gJFccuF0XXxqoCPtdZBYIdSqgIYA6yML6S1XgQsApg5c6aeO3duvxrQH+Xl5Qzk/gaLZGxXMrYJkrNd0qahIxnblYxt6k1/hqlXAmOUUiVKKRtwHfB6tzKvAXMBlFJZGMPW2wewnkIIIUTS6jOMtdYh4G7gbWAT8IrWeoNS6gdKqSuixd4GapVSG4FlwD9rrWtPVKWFEEKIZNKfYWq01m8Cb3Zb9/24eQ3cH30JIYQQ4ijIHbiEEEKIBJMwFkIIIRJMwlgIIYRIMAljIYQQIsEkjIUQQogEkzAWQgghEkzCWAghhEgwCWMhhBAiwSSMhRBCiASTMBZCCCESTMJYCCGESDAJYyGEECLBJIyFEEKIBJMwFkIIIRJMwlgIIYRIMAljIYQQIsEkjIUQQogEkzAWQgghEkzCWAghhEgwCWMhhBAiwSSMhRBCiASTMBZCCCESTMJYCCGESDAJYyGEECLBJIyFEEKIBJMwFkIIIRJMwlgIIYRIMAljIYQQIsEkjIUQQogEkzAWQgghEkzCWAghhEiwpAjjPXWtvLTJTziiE10VIYQQ4qglRRiv3FnHO7tC/PrvWxNdFSGEEOKoJUUYXz29kDkFFv7j3a38o7Im0dURQgghjkpShDHAVyfYGJXt5r7FazjU3J7o6gghhBD9ljRhbLcofveV6fj8Qf5p8Ro5fiyEEGLISJowBhibm8oPFkzig221/PbdykRXRwghhOiXpApjgGtmFHL1tGH86u8VfLBNjh8LIYQY/JIujJVS/PDKSYzMSuG+xWuobvYnukpCCCHEESVdGAOk2C088ZXpNLUF+dZ/y/FjIYQQg1tShjFAaV4aP1gwkRWVNfxumRw/FkIIMXglbRgDXDuziCunFvDvSyv4aHttoqsjhBBC9Cipw1gpxY+uKqM4K4V7//gZNT45fiyEEGLwSeowhujx4y9PpzF6/Dgix4+FEEIMMkkfxgDj89N49IqJvL+1hiff25bo6gghhBBd9CuMlVLzlVJblFKVSqkHj1Dui0oprZSaOXBVHBjXzSriiikF/PL/tvDJjrpEV0cIIYSI6TOMlVJm4AngYmACcL1SakIP5VKB+4CPB7qSA0EpxY+vLmNEZgr3/PFTauX4sRBCiEGiPz3j2UCl1nq71joALAYW9FDuh8DPgEH7lAa33cJvvzyN+tYg97+yVo4fCyGEGBT6E8bDgD1xy1XRdTFKqelAkdb6bwNYtxNiYoGH7182gfcqqnlquRw/FkIIkXhK6yP3DpVSXwLma61viy7fCJymtb47umwC3gVu1lrvVEqVA9/RWq/qYV93AHcA5Obmzli8ePGANcTn8+F2u/tVVmvNk2v9rDoY5sHZDsZ6zQNWj4F2NO0aKpKxTZCc7ZI2DR3J2K5ka9O8efNWa617PqdKa33EF3AG8Hbc8v8D/l/csgeoAXZGX+3APmDmkfY7Y8YMPZCWLVt2VOWb2gL63J+/q0/70VJd6/MPaF0G0tG2ayhIxjZpnZztkjYNHcnYrmRrE7BK95KJ/RmmXgmMUUqVKKVswHXA63Fh3qi1ztJaF2uti4GPgCt0Dz3jwSTVYeW3X55OXUuAb78i1x8LIYRInD7DWGsdAu4G3gY2Aa9orTcopX6glLriRFfwRJo0zMO/XDaeZVuqWfT+9kRXRwghxCnK0p9CWus3gTe7rft+L2XnHn+1Tp4bTh/BR9vrePztLcwq9jJjREaiqySEEOIUc0rcgetIlFL85ItlDEt3cs/Ln1HfEkh0lYQQQpxiTvkwBkhzWHniy9Op8QX4zp/WdpyYJoQQQpwUEsZRZYUevnfpeP6++RDPvL8j0dURQghxCpEwjvPVM0Ywf2IeP3trM5/urk90dYQQQiRIMBxkT/OevgsOkH6dwHWqUErxsy9N5rL/eJ97Xv6Mv917FukuW6KrJYQQ4gRqCjSxpW4LW+q2sLluM1vqt1DZUInb6mb5wuUopU54HSSMu/E4jePHX3zyA77zp3X851dnnJQ/hBBCiBNLa82+ln1G4HYEb90W9rXsi5XJcmYxLmMccwrmUJpRSkRHMKsTf5dGCeMeTC5M56FLxvOvf93IHS+u5r7zxzBpmCfR1RJCCNFPgXCAbQ3b2Fy3mYr6iljwNgebATApE8VpxUzJmcK13mspzShlXMY4spxZCalvUoTxxtqNvFjzIikHUpiZO3NAerI3n1mMrz3Eove3887Gg8wdl83d80Yzs1iuQxZCiMGk0d/YZYh5c91mtjdsJ6RDADgtTsZ6x3LJyEsYlzGOUm8po72jcVqcCa55p6QI411Nu1jXuo5b376VotQirhx9JVeMuoK8lLxj3qdSinvOH8NNc4p58cNd/H7FDr701IecVpLB3eeN5qzRWTJ8LYQQJ1kwHKSivoJ1NetYX72e9TXr2dm0M7Y9x5nDuIxxnFN4Tix4i1KLMJsG7wOBIEnC+OKSizHvMBMoDvDa1tf4j8/+g99+9lvOHHYmV42+inlF87CZj+1ErDSHlbvmjeaWOcX88ZM9LFq+jRt//wlTitK5a+4ovjA+F5NJQlkIIQaa1pq9vr2sr17P2uq1rK9Zz6baTQQixs2ZspxZlGWVsWD0AiZkTmCcdxyZzswE1/rYJEUYA9hMNi4ceSGXjbyMquYq/rLtL7xW+Rrfee87eOweLi25lKvGXEVpRukx7d9ls/C1s0q44fTh/Hn1Xp56bxt3vLiacbmpfHPeKC6bXIBZQlkIIY5Zc6CZz2s+Z33NetZXr2f1vtU07zaO8drNdiZkTuC60uuYnD2ZyVmTyUvJS5oRyqQJ43iFqYXcNfUuvjH5G3x84GNe2/oar1a8ysubX2Z8xniuHH0ll468FI/96E/KslvMfPm04Vw7s5C/rtvH75Zt477Fa/j3dyq4c+4orppWiM0il28LIcSRhCIhKhsqWVe9jnXV61hfs54djTvQGHdALPGUMN45ngsnXUhZdhljvGOwmqwJrvWJk5Rh3MFsMnNmwZmcWXAmjf5G3tzxJku2LuEnn/yEX6z6BecPP58rR1/J6fmnH/XxBIvZxFXTClkwZRj/t/EAv11WyQN/Xs+vl27ljnNGct3s4Tisg/sYhRDi5Gr0N1LXXkdRahEW0+D7+G0NtrL64Go+OfAJH+//mH0t+7AoCxaT8bKarLH5+PXxL6vJ2us2i8lCMBxkY+1GNtVtoi3UBoDX7qUsu4yLSy5mctZkJmZNxGP3UF5eztzSuYn9pZwkg+//hhPEY/dwfen1XF96PZvrNvNa5Wu8sf0N3tr5FrmuXBaMXsCVo6+kKLXoqPZrMinmT8rnool5vFdRzRPLKnn0rxv57bJKvnbWSG44fTipjuT9NieE6CocCbO/ZT87GncYr6Ydsfm69jrAOLt3UtYkpmZPZUr2FCZnT8br8J70uvrDftYeWsvHBz7mk/2f8HnN54R0CKvJypTsKcwvnk9YhwlFQoe9gjoYm28LtXXdrqNlIkHCkXBsORQJoVCM9Y7li2O+SFlWGWXZZRS6C5NmuPlYnTJhHK80o5QHZz/I/TPup3xPOUsql/DM+mdYtG4Rs/JmcdXoq/jCiC8c1WnvSinmjsth7rgcPtlRx2+XVfKztzbzZHklN88p4ZYzi/GmyN28hBhowXCQHU07MCszqbZUUm2pOMyOE/7h3hpsZWfTzs7QjQbv7qbd+MP+WDmP3cNIz0jmFs2lJK0Ej93DprpNrK1ey7OfP0tYhwEYkTaCKdlTYq/R6aMH/AzgYCTIhpoNfHLgEz7Z/wmfHfqMQCSASZmYlDmJmyfdzOy82UzNmTqoLvs5FZySYdzBZrZxYfGFXFh8IQdaDvDXbX9lSeUSHlrxED/6+EfML57P1JypFKQUUOAuIDclt1/HLGaXZPBCyWzWVTXw23cr+c3ft/LM+9u54fQR3HZWCTlpjpPQOiGSTzASZFvDNjbWbmRDzQY21G6gor6CYCTYpZzVZCXVlkqaLS0W0B3L9fX1VK6v7LKt+3zH1Rdaaw61HuoxdA+0HIj9PJMyUegupMRTwpyCOZR4SihOK6bEU9Jjj/cqrgKgLdTGhpoNrKlew9rqtazYu4LXt70OQIo1hUlZk7oE9NGe5xLREbbUbYkNO68+uJrWUCsA4+KPOrAAACAASURBVLzjWFi6kNPyTmN67nRSbalHtW8xsE7pMI6Xl5LH7ZNv57ay21h9cDVLKpfw5o43+fPWP8fKmJSJbGc2Be4C8lPyu0wLUgrId+d3+TY5uTCdRV+dyZYDzfyuvJJn3t/O8x/s5MqpBVw+pYAzRmZiMcvJXkL0JBQJsb1xeyx4N9ZuZHPd5thlLW6rmwmZE7hh/A2UZpRiUiaaAk00BZpoDjQf9trn20dToInG9kaWfrr0iD/bbraTakulNdgaCy8Al8VFiaeEmbkzKfGUGK+0EoanDT+myyedFicz82YyM28mYIR/VXNVLJzXVq/lmfXPENERwDipKT6cR6WPwqQ6P0O01uxo3BEbdl55cCWN/kYAitOKuXzU5czOm82svFkJGRYXvZMw7kYpFfvH8cgZj7C/ZT/7fPsOm66tXsv/7fy/2B1eOnjtXvLd+bFw7ph+44ICbjtnGi99WM1f1+7nlVVVeF1W5k/K45Ky/IQEs9aatlAbde11NPgbYtP69nrq2+tp8DcQ0RGsJitWs9WYxs3bTLYu621mW2eZ+PeYo2Wj8/6Iv+/KiYQIhAPsatrFtsZtbG/YTmV9Jbs27CLdno7X4TWmdi9eh5cUa8qADQWHI2F2Nu00grd2AxtqNrC5bjPt4XbACMHxmeO5rvQ6JmZOZGLWRIpSi7oEUX8tW7aMM84+g+ZAcyy4uwd4R6jbzfYuoZvjyjmhw99KKYrSiihKK+LyUZcDxnD45zWfx8K5fE85r1W+BhhfSCZnT2Zi5kQ+q/6Mf/3Tv1LTVgNAfko+84rmMTtvNrPzZpObknvC6i2On4TxEdjMNkakjWBE2oget4cjYarbqtnn28e+ln3s9+2PTbc1bmPF3hWxD5MOLouLEVMyCYdttLSb+OtBxZK9VmwmB8O9XsZkZzA6K5MUqwunxcku3y5Cu0K4LC6cVqcxtThxWpy4rC4cZkfsuFIoEqLR32iEqb8zULuEbXtDbFt9e32sl9GdRVlId6RjUibjRIxwkGAkSCASiH1LP1YmTPzxrT9y1rCzOHvY2Yz1jj3lT9442QLhADsad7C9cTuVDZVsb9jOtsZt7G7aHTuGqaL/vbPqnR73YTFZ8Nq9pDuMgO4S2HHBHdvuSMdpcRLREXY17eoSvPFn1jotTsZnjOdLY7/EhMwJTMyaSHFa8TEFb0+UUjgsDhwWB9mu7AHZ54nksrqYnT+b2fmzAeNL9O7m3aw51Nl7/v3nv8dtcnPW8LM4Lf80ZuXNkpOihhgJ4+NgNpnJS8kjLyWP6Uw/bLvWmnp/fSykO3rWdW11tIXaaAu10RJspbqlmcb2g+xs38zOqgBL93U9/vVi+YtHrIfD7MBqsuIL+mLX6HWXak01PhQdXnJduZRmlMZ6OB0fnl6HN7bObXX3+g85HAkTjETDORyIzQcjwVhox893L/P++vfZE9zDrz/9Nb/+9NfkOHOYM2wOZxeezen5pyfdsavWYCs7GndQ2VDJtoZt1LTV4LF78Ng9eO1ePA5PLMzS7emkO9Kxm+0D8rP9YT87GnewrWFb7LW9cTu7m3fHvlSZlInhqcMZ6RnJF4Z/gVHpoxiVPoritGI+fP9DZs6ZGfsS1zFy0tN0a8NWGtobaPA39Pr/YceJVR3BazfbKc0o5crRVxo93syJlHhKBv2tCxNJKRXrJCwYvQAw/s4fLP+AeefMS3DtxLGSMD6BlFJkODLIcGQwMWtin+Xbg2Heq6jmjbV7eXfLHlpD7aTY/ZxTmsHsUW5G5VgJRPy0hlppC7XRGmyNhXogEsBj88QCt+PDPcORQbo9Hat54C6vMpvMmE1mHBzbiWjpVenMnTuX6tZqVuxdwYq9K1i6aylLKpdgURam5EwZkr3mQCTAptpNsdDd1rCNrQ1b2efbFwsnm8lGpjOTpkATLcGWXvfltDg7wzkupHta9tq9uKwu9vn2Gb3cuN5ula8qFrpmZWZ42nBGp4/mwuILGZ0+mpGekRR7insNf6VU7MSmIvp32V84EqY50NxreAcjQcZ5xzEhcwKj0kcNyutthxq72T5k/p2Insm/gkHEYTVz0cQ8LpqYR3twKu9VVPPc0jW89zm8+WkrXpeViybmcenkfC4ePfRP/sp2ZXPVmKu4asxVhCIh1lWvY8XeFby/9/1B3WvuGOLd1rCtM3ijQ7x6jxG6FpOF4rRiyrLKuHL0lYxJH8Oo9FEUphbGwicYDtLgb+jyqm+vp9Hf2HV9ewN7fXup99fTHGjus34WZWFE2gjGZYzjkpGXGD1dzyhGpI045nu0Hw2zyWx8UXCkn/CfJUSykDAepDqC2V7t4PQ5Z/NeRTV/W7efv67dx+KVe7oEczKclW0xWZieO53pudO5d/q9Ce81a63xh/3s9e1la8PWWE+3sqGyy3FVszIzIm0EY71jmaAmcP7U8xmdPprhacP7vAzOaraS7co+quOWHecFNPobYz3PhvYGfEEfeSl5xs9OHT6gIyFCiBNPwngI6NpjDvcYzBdOyOPssVmcMTKTTPfAHG9MpO695o5rMFfsXXFYr/msYcZJK2ZlpjXUSkuwhdZQq3FZSrBzuSXYErtUJX5dW7Cta5lQK23Bti5nypuUiaLUIkanj+aCERcwOn107LhqR2+zvLycucVzT+jvxWKykOnMHLJPphFC9EzCeIjpLZj/tn4//71qDwCleanMGZ3FmaMymV2SMeRvx2kxWZiRO4MZuTO4b/p9Pfaaj4bD7MBldeGyuEixpuCyuvA4PORb8o3luPW5rlzGeMdQnFaMwyI3axFCnBgSxkNYfDCHwhHW7W3kw221/KOyhhc/2sXvV+zAbFJMLvRw5qhMzhyVxYwR3iH/AIuees2fHfoMq8naNWSjU6fVSYolJbZNztQVQgw2EsZJwmI2MX24l+nDvdw1bzTtwTCf7q6PhfNT723niWXbsFlMzBjuNcJ5dBaTCz1Yh/Dx5vhesxBCDFUSxknKYTVz5qgszhyVxbcvHIfPH2Lljjr+UVnDB9tq+eU7FfzynQpSbGZOG5nJmaMyOWNUJuPz0jCZ5BIJIYQ4mSSMTxFuu4V5pTnMK80BoK4lwMfba/nHNiOc3918CACvy8oZozI5Y5RxzHlk1sDd8lAIIUTPJIxPURkpNi4uy+fisnwADjS280E0mD+orOHN9cYTabJT7cwq9jKrOINZxRmMz0/DLD1nIYQYUBLGAoA8j4Orpxdy9fRC4963da38o7KWlTvr+GRHXSyc3XYL00d4mTXCy6ySDKYWpQ/5E8KEECLRJIzFYZRSjMhMYURmCl8+bTgA+xraWLmzjpU761i1s55/W1qB1mA1K8qGeWI955nFXtJdJ/4uT0IIkUwkjEW/FKQ7WTB1GAumDgOgsTXI6t11fLKjnpU763j2Hzt4evl2AMbmumPhPKskg2HpziPtWgghTnkSxuKYeFxWzivN5bxS4xmp7cEwa/c0sGpXPZ/sqOP1Nft46ePdABR4HMwqyWBmcQazizOI6J6f6COEEKeqQRXGwWCQqqoq2tvb+y7cjcfjYdOmTSegVomVyHY5HA4KCwuxWvu+g5fDalwiddrITO6aB+GIZvOBJlbuqGPlzno+2FbLX9bsA8BpgbItHzKhII0J+WlMKEhjTK4bu0WOPQshTk2DKoyrqqpITU2luLj4qC+naW5uJjU18U/0GWiJapfWmtraWqqqqigpKTnq95tNiokFHiYWeLh5TknspLCVO+t586MNNGrNK6v20BowHrhgMSlG57hj4TwhP43x+Wl4U+T4sxAi+Q2qMG5vbz+mIBYDTylFZmYm1dXVA7a/jpPCspormTv3TCIRza66Vjbua2Lj/kY27mviH9tq+J/P9sbeV+BxMD4uoCcUpFHkdcmNSYQQSWVQhTEgQTyInOi/hcmkKMlKoSQrhUsn58fW1/j8bNrfFA3pJjbtb2LZlkNEooea3XYL4/NT43rRHsbkuuUSKyHEkDXowjjR3G43Pp8v0dU4pWW57Zw9Jpuzx3Q+57c9GKbiYHMsoDfua+LV1VW0fGgMc5sUFHpdFGelMDIrheLMjnk3BemOIf+8ZyFEcpMwFkOCw2pmcmE6kwvTY+siEeM49Mb9TWze38T2mhZ21rawemcdLdFj0WBcC12U4aIkM4XiaE+8JMuYz09zyJC3ECLhJIx7obXmu9/9Lv/7v/+LUoqHH36YhQsXsn//fhYuXEhTUxOhUIgnn3ySM888k6997WusWrUKpRS33nor3/rWtxLdhKRnMimKo6F6SVnnMLfWmmqfn501reysaTFCOhrU/9hWQ3swEitrt5gozkyhOMvoSZdkdoZ1dqpdDpsIIU6KQRvG//rXDWzc19Tv8uFwGLP5yMcMJxSk8cjlE/u1v//5n/9hzZo1rF27lpqaGmbNmsU555zDyy+/zEUXXcT3vvc9wuEwra2trFmzhr179/L5558D0NDQ0O96i4GnlCIn1UFOqoPZJRldtkUimoPN7eyobmFHrRHSO2paqTzkY9nmagLhzqBOsZmjJ5254qbGvPSohRADadCGcaKtWLGC66+/HrPZTG5uLueeey4rV65k1qxZ3HrrrQSDQa688kqmTp3KyJEj2b59O/fccw+XXnopF154YaKrL3phMinyPU7yPU7OHJ3VZVs4otnX0MaOaC96e3ULu2pb2HKwmaWbDhIMd96sxGY2UZTh7AzpDBdN1SGGV/so9LqwWeQYtRCi/wZtGPe3B9vhZF2Pe84557B8+XL+9re/cfPNN3P//ffz1a9+lbVr1/L222/z1FNP8corr/Dss8+e8LqIgWU2GceWizJcnEN2l23hiGZ/Yxu7alujrxZ21bays7aFj7bXxq6X/rfV72FSxu1DizNTGJ7pojjTxfAMYyh8eIYLl23Q/rMTQiSIfCr04uyzz+bpp5/mpptuoq6ujuXLl/P444+za9cuCgsLuf322/H7/Xz66adccskl2Gw2vvjFLzJu3DhuuOGGRFdfDDCzSVHodVHodTFndNdtHceo/7L0H3iHj2N3bQs7a1vZVdfKm+v309Aa7FI+N83OmJxUxuS6GZubythcN2NyU0lz9H2nMyFEcpIw7sVVV13Fhx9+yJQpU1BK8fOf/5y8vDz+8Ic/8Pjjj2O1WnG73bzwwgvs3buXW265hUjEON74k5/8JMG1FydTxzHqMV4zc2cUHra9sTXIrrqWWI96e00LWw/6WPzJHtqCnWd956U5DgvoMTluUiWkhUh6/QpjpdR84NeAGXhGa/3TbtvvB24DQkA1cKvWetcA1/Wk6LjGWCnF448/zuOPP95l+0033cRNN9102Ps+/fTTk1I/MfR4XFYmu7pelgXGyWRV9W1UHGxm6yEfWw82U3GomZc+3tXljO98j4MxuamMzTGCekw0qN12+S4tRLLo81+zUsoMPAFcAFQBK5VSr2utN8YV+wyYqbVuVUrdCfwcWHgiKixEsjCZFMMzXQzPdPGFCbmx9eGIpqq+lYqDPiOoDzZTcdDHx9tr8Yc6Q3pYutMI5hwjnIdnuBiW7iTP48AqNzkRYkjpz1fr2UCl1no7gFJqMbAAiIWx1npZXPmPADloKsQxMps67+N9QbeQ3lPXGutJV0RD+oNttQTiQtqkjCHvYV4nw9Kd0amrczndidMmtw4VYjBRuo9nyyqlvgTM11rfFl2+EThNa313L+V/CxzQWj/Ww7Y7gDsAcnNzZyxevLjLdo/Hw+jRo7u/rV/6c53xUJTodlVWVtLY2Dig+/T5fLjd7gHd52CQqHZFtKa6VVPTpqlpj1DTpqlt09S2GfP1fh27r3eHVBtkOUxkOhWZTtV13mnCZTEO1STj3yoZ2wTJ2a5ka9O8efNWa61n9rRtQA86KaVuAGYC5/a0XWu9CFgEMHPmTD137twu2zdt2nTMlyfJIxRPDIfDwbRp0wZ0n+Xl5XT/2yeDwdquUDjCwWY/e+vb2NvQGp22URWdrq9qwx8KdXmP225hWLoTe9hCabGXnFQHuWl2slMd5KTZyU1zkO22D8nrqQfr3+l4JWO7krFNvelPGO8FiuKWC6PrulBKfQH4HnCu1to/MNUTQhwvi9kUG56GjMO2a62pbQnEQjo+rLfubaF8SzU1Pv9hvWsAr8tq3O0szR43NcI6J7VznTxRS4gj608YrwTGKKVKMEL4OuDL8QWUUtOApzGGsw8NeC2FECeMUoost50st50pRV3P+O7omYQjmlqfn0PNfg41t3Owyc+hJmP+ULOfQ03tVB7yUd3sJ9RDaqc5LOREAzrP4zDuAZ5tPFWrOEtuhCJEn/8CtNYhpdTdwNsYlzY9q7XeoJT6AbBKa/068DjgBv4UvbH+bq31FSew3kKIk8hsUkaYpjkAT6/lIhFNXWvgsKA2pn4ONrfzQWUt//Np18G1fI+DkqwURmanUJLlZmT0YR2FXqc8/lKcEvr1dVRr/SbwZrd134+b/8IA1yvphUIhLBbpDYjkYjJ19rInkNZrudZAiB01Lcar2phur2nh9TX7aGrvPH5tNSuGZ7iMgM42AnpkltGrznbLU7VE8pA06MGVV17Jnj17aG9v57777uOOO+7grbfe4qGHHiIcDpOVlcXf//53fD4f99xzT+zRiY888ghf/OIXcbvdsZuHvPrqq7zxxhs8//zz3HzzzTgcDj777DPmzJnDddddx3333Ud7eztOp5PnnnuOcePGEQ6HeeCBB3jrrbcA+PrXv87EiRP5zW9+w2uvvQbAO++8w+9+9zuWLFmSsN+TEMfKZbMwscDDxIKuvWytNXUtgVg476hpYXu1jx01LSzfWt3lEi633RJ73GVJVgpZqXbSnVY80Ve6y5imOqyY5QlbYpAbvGH8vw/CgfX9Lu4Mh8DcR3PyyuDinx65DPDss8+SkZFBW1sbs2bNYsGCBdx+++0sX76ckpIS6urqAPjhD3+Ix+Nh/XqjnvX19X3uu6qqig8++ACz2UxTUxPvv/8+FouFpUuX8tBDD/HnP/+ZRYsWsXPnTtasWUNbWxvBYBCv18s3v/lNqquryc7O5rnnnuPWW2/t+xcjxBCilCLTbSfTbWdmcdeTzeKfqtUR0NtrWvh0dz1/XbeP3q7SVApS7RY8LivpThuR9jb+tPdTPNGwjg/w2DqXDY/TSorNLL1vcVIM3jBOoN/85jexHueePXtYtGgR55xzDiUlJQBkZBgfEkuXLiX+Wmmv19vnvq+55prYdcONjY3cdNNNbN26FaUUwWAwtt9vfOMbsWHsjp9344038l//9V/ccsstfPjhh7zwwgsD1GIhBr8uT9Ua2/WpWv5QmMbWII1txqshOt8QXW5qC9LQGqCxLchuH2w60BRdF+zxhLMOVrMiM8VORoqNTLeNLHfcfNz6zBQ7mW4bLglvcYwGbxj3owcbr22ArsctLy9n6dKlfPjhh7hcLubOncvUqVPZvHlzv/cR/4+xvb29y7aUlJTY/L/8y78wb948lixZws6dO/u8nu6WW27h8ssvx+FwcM0118gxZyGi7BYzOWnm6AlmRxZ/7arWmtZA2AjtWJgHYoFe3xqk1uenriVATXT4vK4lEHtkZncOqykWzJkpNjJS7GS5bdHQjl9vBLjcCU10kE/zbhobG/F6vbhcLjZv3sxHH31Ee3s7y5cvZ8eOHbFh6oyMDC644AKeeOIJfvWrXwHGMLXX6yU3N5dNmzYxbtw4lixZ0uuXhMbGRoYNGwbA888/H1t/wQUX8PTTTzNv3jyA2M8rKCigoKCAxx57jKVLl57YX4QQpwClFCl2CynRm5z0V2sgRK0vQF1LgNoWP7W+ALUtAWp9/ug0QLXPz5YDzdS0BLoc647nspmjwRwX2NH5+F53x7xcApa85C/bzfz583nqqacYP34848aN4/TTTyc7O5tFixZx9dVXE4lEyMnJ4Z133uHhhx/mrrvuYtKkSZjNZh555BGuvvpqfvrTn3LZZZeRnZ3NzJkzYydzdffd736Xm266iccee4xLL700tv62226joqKCyZMnYzab+frXv87ddxt3H/3KV75CdXU148ePPym/DyHE4Vw2C64MC0UZrj7Laq3x+UNG79oXiPW0a1uMMO+YP9TsZ/OBZmqPEN7xPe9YYEdDvLoqiN5yiKwUO1mpRogPxTuknar6vDf1iTJz5ky9atWqLus2bdp0zCGT6NtGnijd23X33Xczbdo0vva1r52Un388f5PeJOst7pKxXdKmk09rTUsgTJ2vs9fdGd7+WIjH98zjH7kZz+O0xo51Z7uNIfPM6KVnWW4bWan2WHgPxl73YP9bHS2l1Mm5N7U4sWbMmEFKSgq//OUvE10VIcQJopTCbbfgtlsYntl3zxuMYfM3li5n9KRp1DT7qfEFqPH5qfH5Y0Pmmw40UdPs73IddzyXzRwL6Y7Azkyxkea04HFaSXNYSYuedZ7mMKZuh0UuGxsgEsZDyOrVqxNdBSHEIOSyWchxmZg+vO8rOgKhCLUtfmqajcCujgZ2R3jX+Pzsrm3ls9311LUEerwnebxUu4U0Z0dQWw4L7fgw97g6g9zjtOKwmuTs8ygJYyGEOIXYLCbyPU7yPX2fsNYxZN5xeVjHtKk9FDffsT5EU1uQ3XWtsbItvZx1Hl8XT/yNWjpCPDqt3huk7tOqLmU6rgW3W5LrTHQJYyGEED2KHzI/mrPNO4TCEZraQ11Cu/urKe668ANN7Ww52Exja5BmvzGc/vLmtT3u22E1RQM8eoMWuxmXzYLTZsZlMxtTq6VzPrY+us7asa7zPdYE3gddwlgIIcQJYTGbYmd9H61QOMJbf3+PidNndw3w6M1but/gpdrnpzXQSlsgTGsgTFsgTCDc84ltvbGaVTSkjcDOdNv40zfOPOq6HwsJYyGEEIOOxWzCbVOUZKX0XbgXoXCE1mA4FtCtgVDcfJi2YCgW3LF1AWNdazCM5SSenCZhLIQQIilZzCbSzCbSHNZEV6VPckX4cXC73b1u27lzJ5MmTTqJtRFCCDFUSRgLIYQQCTZoh6l/9snP2FzX/4czhMPh2NOQelOaUcoDsx/odfuDDz5IUVERd911FwCPPvooFouFZcuWUV9fTzAY5LHHHmPBggX9rhcYD4u48847WbVqFRaLhX/7t39j3rx5bNiwgVtuuYVAIEAkEuHPf/4zBQUFXHvttVRVVREOh/nOd77DzTfffFQ/TwghxNAyaMM4ERYuXMg//dM/xcL4lVde4e233+bee+8lLS2NmpoaTj/9dK644oqjulD9iSeeQCnF+vXr2bx5MxdeeCEVFRU89dRT3HfffXzlK18hEAgQDod58803KSgo4G9/+xtgPP9YCCFEchu0YXykHmxPBuLe1NOmTePQoUPs27eP6upqvF4veXl5fOtb32L58uWYTCb27t3LwYMHycvL6/d+V6xYwT333ANAaWkpI0aMoKKigjPOOIMf/ehHVFVVcfXVVzNmzBjKysr49re/zQMPPMBll13G1KlTj6tNQgghBj85ZtzNNddcw6uvvsp///d/s3DhQl566SWqq6tZvXo1a9asITc397BnFB+rL3/5y7z++us4nU4uueQS3n33XcaOHcunn35KWVkZDz/8MD/96dE911kIIcTQM2h7xomycOFCbr/9dmpqanjvvfd45ZVXyMnJwWq1smzZMnbt2nXU+zz77LN56aWXOO+886ioqGD37t2MGzeO7du3M3LkSO699152797NunXrKC0tJSMjgxtuuIH09HSeeuqpE9BKIYQQg4mEcTcTJ06kubmZYcOGkZ+fz1e+8hUuv/xyysrKmDlzJqWlpUe9z29+85vceeedlJWVYbFYeP7557Hb7bzyyiu8+OKLWK1W8vLyeOihh1i5ciX//M//jMlkwmq18otf/OIEtFIIIcRgImHcg/Xr18fms7Ky+PDDD3ss5/P5et1HcXExn3/+OQAOh4PnnnvusDIPPvggDz74YJd1F110ERdddFFsubm5+ajqLoQQYuiRY8ZCCCFEgknP+DitX7+eG2+8scs6u93Oxx9/nKAaCSGEGGokjI9TWVkZa9asSXQ1hBBCDGEyTC2EEEIkmISxEEIIkWASxkIIIUSCSRgLIYQQCSZhfByO9DxjIYQQor8kjJNAKBRKdBWEEEIch0F7adOBH/8Y/6b+P884FA5T18fzjO3jS8l76KFetw/k84x9Ph8LFizo8X0vvPACv/jFL1BKMXnyZF588UUOHjzIN77xDbZv3w7Ak08+SUFBAZdccgkbN24E4Be/+AU+n49HH32UuXPnMnXqVFasWMH111/P2LFjeeyxxwgEAmRmZvLSSy+Rm5uLz+fjnnvuYdWqVSileOSRR2hsbGTdunX86le/AuA///M/2bhxI//+7//e9y9aCCHEgBu0YZwIA/k8Y4fDwZIlSw5738aNG3nsscf44IMPyMrKoq6uDoB7772Xc889lyVLlhAOh/H5fNTX1x/xZwQCAVatWgVAfX09H330EUopnnnmGX7+85/zy1/+kh/+8Id4PJ7YLT7r6+uxWq386Ec/4vHHH8dqtfLcc8/x9NNPH++vTwghxDEatGF8pB5sTwbb84y11jz00EOHve/dd9/lmmuuISsrC4CMjAwA3n33XV544QUAzGYzHo+nzzBeuHBhbL6qqoqFCxeyf/9+AoEAJSUlACxdupTFixfHynm9XgDOO+883njjDcaPH08wGKSsrOwof1tCCCEGyqAN40TpeJ7xgQMHDnuesdVqpbi4uF/PMz7W98WzWCxEIpHYcvf3p6SkxObvuece7r//fq644grKy8t59NFHj7jv2267jR//+MeUlpZyyy23HFW9hBBCDCw5gaubhQsXsnjxYl599VWuueYaGhsbj+l5xr2977zzzuNPf/oTtbW1ALFh6vPPP58nn3wSgHA4TGNjI7m5uVRXV1NbW4vf7+eNJ3PAIAAAB91JREFUN9444s8bNmwYAH/4wx9i6y+44AKeeOKJ2HJHb/u0005jz549vPzyy1x//fX9/fUIIYQ4ASSMu+npecarVq2irKyMF154od/PM+7tfRMnTuR73/se5557LlOmTOH+++8H4Ne//jXLli2jrKyMGTNmsHHjRqxWKw888ACzZ8/mggsuOOLPfvTRR7nmmmuYMWNGbAgc4OGHH6a+vp5JkyYxZcoUli1bFtt27bXXMmfOnNjQtRBCiMSQYeoeDMTzjI/0vptuuombbrqpy7rc3Fz+8pe/HFb2zjvv5Lvf/e5h68vLy7ssL1iwoMezvN1ud5eecrwVK1bwrW99q7cmCCGEOEmkZ3wKamhoYOzYsTidTs4///xEV0cIIU550jM+TkPxecbp6elUVFQkuhpCCCGiJIyPkzzPWAghxPEadMPUWutEV0FEyd9CCCFOjkEVxg6Hg9raWgmBQUBrTW1tLQ6HI9FVEUKIpDeohqkLCwupqqqiurr6qN/b3t6elMGRyHY5HA4KCwsT8rOFEOJU0q8wVkrNB34NmIFntNY/7bbdDrwAzABqgYVa651HWxmr1Rq7jePRKi8vZ9q0acf03sEsWdslhBCiU5/D1EopM/AE/P/27jbEiiqO4/j3R2qBia4JZiWWFYG9qBYRCxPBMJXQigolyFIQKSFfRAiCiO8sKiiisJRKpJYerCVW1B6gV1omPubDrmKkrCuhaBJR1r8Xcy5Md2fuztV156H/B4Y7d8651/P3PzPn7pm55zILmADMlzShrtoi4KyZ3Qa8Bqzp74Y655xzVZXlmvEkoMvMjpnZn8BHQP3sEnOB2swSnwDT1dfPGjnnnHMOyNYZ3wj8Ent+ImxLrGNmF4FzwHX90UDnnHOu6gb0Bi5Ji4HF4ekFSYf78e1HAb/24/sVRRXjqmJMUM24PKbyqGJcVYtpXFpBls74JDA29vymsC2pzglJg4DhRDdy/YeZrQXWZvg3myZpp5lNvBLvnacqxlXFmKCacXlM5VHFuKoYU5osw9Q/ALdLukXSEGAe0F5Xpx2o/fLBY8A35l8Wds455zLp8y9jM7soaSmwheirTevN7ICk1cBOM2sH1gEbJHUBZ4g6bOecc85lkOmasZl1AB1121bG1v8AHu/fpjXtigx/F0AV46piTFDNuDym8qhiXFWMKZF8NNk555zLV6HmpnbOOef+j0rXGUuaKemwpC5JyxPKr5bUFsp3SLp54FvZHEljJX0r6SdJByQ9n1BnmqRzknaHZWXSexWJpOOS9oX27kwol6TXQ672SmrNo51ZSboj9v+/W9J5Scvq6pQiT5LWSzotaX9s20hJ2yR1hseWlNcuCHU6JS1IqpOHlJhelnQo7F+bJI1IeW3DfTVPKXGtknQytp/NTnltw/NlXlJiaovFc1xS4m/TFjlXl8XMSrMQ3UB2FBgPDAH2ABPq6jwLvB3W5wFtebc7Q1xjgNawPgw4khDXNODLvNvaZFzHgVENymcDmwEBk4Edebe5idiuAk4B48qYJ2Aq0Arsj217CVge1pcDaxJeNxI4Fh5bwnpL3vE0iGkGMCisr0mKKZQ13FcLGNcq4IU+Xtfn+bJIMdWVvwKsLFuuLmcp21/GlZya08y6zWxXWP8NOEjvWc6qaC7wgUW2AyMkjcm7URlNB46a2c95N+RSmNl3RN98iIsfO+8DDye89EFgm5mdMbOzwDZg5hVraBOSYjKzrRbNCgiwnWiehFJJyVUWWc6XuWgUUzhfPwF8OKCNylnZOuPKT80ZhtXvAXYkFN8raY+kzZLuHNCGXRoDtkr6Mcy+Vi9LPotqHukni7LlqWa0mXWH9VPA6IQ6Zc7ZQqKRmCR97atFtDQMv69PuaRQ1lzdD/SYWWdKeRlz1aeydcaVJula4FNgmZmdryveRTQkehfwBvD5QLfvEkwxs1aiX/x6TtLUvBvUH8LkN3OAjxOKy5inXiwaD6zMVy0krQAuAhtTqpRtX30LuBW4G+gmGtativk0/qu4bLnKpGydcTNTc6IGU3MWjaTBRB3xRjP7rL7czM6b2YWw3gEMljRqgJvZFDM7GR5PA5uIhs3isuSziGYBu8ysp76gjHmK6aldJgiPpxPqlC5nkp4GHgKeDB8yesmwrxaKmfWY2d9m9g/wDsntLWOuBgGPAm1pdcqWq6zK1hlXcmrOcI1kHXDQzF5NqXN97dq3pElEuSvshwxJQyUNq60T3Uizv65aO/BUuKt6MnAuNkxaZKmf3MuWpzrxY2cB8EVCnS3ADEktYWh0RthWSJJmAi8Cc8zs95Q6WfbVQqm7t+IRktub5XxZNA8Ah8zsRFJhGXOVWd53kDW7EN2Be4ToLsEVYdtqooMN4Bqi4cMu4HtgfN5tzhDTFKIhwb3A7rDMBpYAS0KdpcABojsitwP35d3uPmIaH9q6J7S7lqt4TALeDLncB0zMu90Z4hpK1LkOj20rXZ6IPkx0A38RXUtcRHRvxddAJ/AVMDLUnQi8G3vtwnB8dQHP5B1LHzF1EV03rR1XtW9a3AB0NNpXi7KkxLUhHDN7iTrYMfVxhee9zpdFWJJiCtvfqx1LsbqlydXlLD4Dl3POOZezsg1TO+ecc5XjnbFzzjmXM++MnXPOuZx5Z+ycc87lzDtj55xzLmfeGTvnnHM5887YOeecy5l3xs4551zO/gW8aF+jRJC45wAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "kAILOHZ9voS0", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "61e8a983-22b4-4987-9b7e-25b6e275c11e" }, "source": [ "y_pred = model.predict(X_test)\n", "y_pred = y_pred.argmax(axis=1)\n", "print(classification_report(y_test, y_pred,\n", " target_names=class_names))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " T-shirt/top 0.80 0.87 0.83 1000\n", " Trouser 0.99 0.98 0.99 1000\n", " Pullover 0.82 0.80 0.81 1000\n", " Dress 0.93 0.86 0.89 1000\n", " Coat 0.82 0.81 0.82 1000\n", " Sandal 0.95 0.98 0.97 1000\n", " Shirt 0.69 0.72 0.71 1000\n", " Sneaker 0.95 0.96 0.96 1000\n", " Bag 0.98 0.96 0.97 1000\n", " Ankle boot 0.97 0.94 0.96 1000\n", "\n", " accuracy 0.89 10000\n", " macro avg 0.89 0.89 0.89 10000\n", "weighted avg 0.89 0.89 0.89 10000\n", "\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "pX0ogutpwCR0" }, "source": [ "# 5. Dynamic Learning Rate Schedules\n", "\n", "Why keep the learning rate constant when changing it over the training process can yield better results often?" ] }, { "cell_type": "markdown", "metadata": { "id": "F0aqbL2-wR4v" }, "source": [ "## 5.1 Training with exponential decay schedule\n", "\n", "```\n", "lr = lr0 * 0.1**(epoch / s)\n", "```" ] }, { "cell_type": "code", "metadata": { "id": "yG9h7cD7wiMF" }, "source": [ "def exponential_decay(lr0, s):\n", " def exponential_decay_fn(epoch):\n", " return lr0 * 0.1**(epoch / s)\n", " return exponential_decay_fn\n", "\n", "exponential_decay_fn = exponential_decay(lr0=0.01, s=20)" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "ZMqKn-Tpwv21" }, "source": [ "model = tf.keras.models.Sequential([\n", " tf.keras.layers.Flatten(input_shape=[28, 28]),\n", " tf.keras.layers.BatchNormalization(),\n", " tf.keras.layers.Dense(300, use_bias=False),\n", " tf.keras.layers.BatchNormalization(),\n", " tf.keras.layers.Activation(\"relu\"),\n", " tf.keras.layers.Dense(100, use_bias=False),\n", " tf.keras.layers.Activation(\"relu\"),\n", " tf.keras.layers.BatchNormalization(),\n", " tf.keras.layers.Dense(10, activation=\"softmax\")\n", "])" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "KdwCH16Lwyw2", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "96797f63-cc8e-4e2c-97b7-8cb58d02689e" }, "source": [ "model.compile(loss=\"sparse_categorical_crossentropy\", \n", " optimizer=tf.keras.optimizers.Adam(lr=1e-3, \n", " beta_1=0.9, \n", " beta_2=0.999),\n", " metrics=[\"accuracy\"])\n", "model.summary()" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Model: \"sequential_6\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "flatten_6 (Flatten) (None, 784) 0 \n", "_________________________________________________________________\n", "batch_normalization_9 (Batch (None, 784) 3136 \n", "_________________________________________________________________\n", "dense_18 (Dense) (None, 300) 235200 \n", "_________________________________________________________________\n", "batch_normalization_10 (Batc (None, 300) 1200 \n", "_________________________________________________________________\n", "activation_4 (Activation) (None, 300) 0 \n", "_________________________________________________________________\n", "dense_19 (Dense) (None, 100) 30000 \n", "_________________________________________________________________\n", "activation_5 (Activation) (None, 100) 0 \n", "_________________________________________________________________\n", "batch_normalization_11 (Batc (None, 100) 400 \n", "_________________________________________________________________\n", "dense_20 (Dense) (None, 10) 1010 \n", "=================================================================\n", "Total params: 270,946\n", "Trainable params: 268,578\n", "Non-trainable params: 2,368\n", "_________________________________________________________________\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "HqQGymlHw3pO", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "090e3e0d-7422-4ea0-8353-62532e729591" }, "source": [ "lr_scheduler = tf.keras.callbacks.LearningRateScheduler(exponential_decay_fn,\n", " verbose=1)\n", "history = model.fit(X_train, y_train, epochs=20,\n", " validation_split=0.1,\n", " callbacks=[lr_scheduler])" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "\n", "Epoch 00001: LearningRateScheduler reducing learning rate to 0.01.\n", "Epoch 1/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.5103 - accuracy: 0.8161 - val_loss: 0.4049 - val_accuracy: 0.8513\n", "\n", "Epoch 00002: LearningRateScheduler reducing learning rate to 0.008912509381337455.\n", "Epoch 2/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3948 - accuracy: 0.8548 - val_loss: 0.3580 - val_accuracy: 0.8688\n", "\n", "Epoch 00003: LearningRateScheduler reducing learning rate to 0.007943282347242816.\n", "Epoch 3/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.3520 - accuracy: 0.8719 - val_loss: 0.3339 - val_accuracy: 0.8767\n", "\n", "Epoch 00004: LearningRateScheduler reducing learning rate to 0.0070794578438413795.\n", "Epoch 4/20\n", "1688/1688 [==============================] - 7s 4ms/step - loss: 0.3230 - accuracy: 0.8806 - val_loss: 0.3439 - val_accuracy: 0.8790\n", "\n", "Epoch 00005: LearningRateScheduler reducing learning rate to 0.006309573444801933.\n", "Epoch 5/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.2992 - accuracy: 0.8879 - val_loss: 0.3137 - val_accuracy: 0.8878\n", "\n", "Epoch 00006: LearningRateScheduler reducing learning rate to 0.005623413251903491.\n", "Epoch 6/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.2757 - accuracy: 0.8974 - val_loss: 0.3068 - val_accuracy: 0.8888\n", "\n", "Epoch 00007: LearningRateScheduler reducing learning rate to 0.005011872336272724.\n", "Epoch 7/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.2573 - accuracy: 0.9048 - val_loss: 0.3078 - val_accuracy: 0.8877\n", "\n", "Epoch 00008: LearningRateScheduler reducing learning rate to 0.004466835921509631.\n", "Epoch 8/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2389 - accuracy: 0.9094 - val_loss: 0.2979 - val_accuracy: 0.8953\n", "\n", "Epoch 00009: LearningRateScheduler reducing learning rate to 0.0039810717055349725.\n", "Epoch 9/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.2242 - accuracy: 0.9161 - val_loss: 0.3252 - val_accuracy: 0.8852\n", "\n", "Epoch 00010: LearningRateScheduler reducing learning rate to 0.003548133892335755.\n", "Epoch 10/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.2073 - accuracy: 0.9219 - val_loss: 0.3009 - val_accuracy: 0.8943\n", "\n", "Epoch 00011: LearningRateScheduler reducing learning rate to 0.0031622776601683794.\n", "Epoch 11/20\n", "1688/1688 [==============================] - 7s 4ms/step - loss: 0.1954 - accuracy: 0.9267 - val_loss: 0.3053 - val_accuracy: 0.8960\n", "\n", "Epoch 00012: LearningRateScheduler reducing learning rate to 0.002818382931264454.\n", "Epoch 12/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.1864 - accuracy: 0.9286 - val_loss: 0.3069 - val_accuracy: 0.8945\n", "\n", "Epoch 00013: LearningRateScheduler reducing learning rate to 0.0025118864315095803.\n", "Epoch 13/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.1727 - accuracy: 0.9347 - val_loss: 0.3105 - val_accuracy: 0.8978\n", "\n", "Epoch 00014: LearningRateScheduler reducing learning rate to 0.0022387211385683395.\n", "Epoch 14/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.1637 - accuracy: 0.9380 - val_loss: 0.3151 - val_accuracy: 0.8983\n", "\n", "Epoch 00015: LearningRateScheduler reducing learning rate to 0.0019952623149688802.\n", "Epoch 15/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.1528 - accuracy: 0.9425 - val_loss: 0.3287 - val_accuracy: 0.8995\n", "\n", "Epoch 00016: LearningRateScheduler reducing learning rate to 0.001778279410038923.\n", "Epoch 16/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.1465 - accuracy: 0.9446 - val_loss: 0.3267 - val_accuracy: 0.9005\n", "\n", "Epoch 00017: LearningRateScheduler reducing learning rate to 0.0015848931924611134.\n", "Epoch 17/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.1364 - accuracy: 0.9488 - val_loss: 0.3399 - val_accuracy: 0.8990\n", "\n", "Epoch 00018: LearningRateScheduler reducing learning rate to 0.0014125375446227546.\n", "Epoch 18/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.1320 - accuracy: 0.9502 - val_loss: 0.3368 - val_accuracy: 0.8987\n", "\n", "Epoch 00019: LearningRateScheduler reducing learning rate to 0.0012589254117941673.\n", "Epoch 19/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.1250 - accuracy: 0.9525 - val_loss: 0.3507 - val_accuracy: 0.8955\n", "\n", "Epoch 00020: LearningRateScheduler reducing learning rate to 0.0011220184543019637.\n", "Epoch 20/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.1208 - accuracy: 0.9555 - val_loss: 0.3604 - val_accuracy: 0.8997\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "dzv_n9QUxDUm", "colab": { "base_uri": "https://localhost:8080/", "height": 296 }, "outputId": "0d442c82-811e-4c66-fe4b-70caf5149fba" }, "source": [ "from matplotlib.ticker import MaxNLocator\n", "\n", "plt.figure().gca().xaxis.set_major_locator(MaxNLocator(integer=True))\n", "plt.plot(history.epoch, history.history[\"lr\"], \"o-\")\n", "n_epochs=20\n", "plt.axis([0, n_epochs - 1, 0, 0.011])\n", "plt.xlabel(\"Epoch\")\n", "plt.ylabel(\"Learning Rate\")\n", "plt.title(\"Exponential Scheduling\", fontsize=14)\n", "plt.grid(True);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEXCAYAAAC6baP3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU1f3/8dcnCxAIEPYlgOwgoIgguBe3Qt1Qi4paq61K61etta3+tLWtWv260K+7VlFr3ZWqVVQURQwulVUQWUQQUAirssgSIMDn98e9wTHMhBnI5A7k/Xw85pG5556585lJMp+555x7jrk7IiIiycqKOgAREdm7KHGIiEhKlDhERCQlShwiIpISJQ4REUmJEoeIiKREiUNkD5jZhWa2PsXHFJnZ/emKKXyOhWb2hzQcd7CZpTSGv/x7tDvvmWQWJQ7ZLWb2LzPzOLfxUceWLuHrG1yu+AWgfRqe62Izm2pm681srZlNN7ObK/t5IpKW90yqTk7UAchebQxwfrmyLVEEEhV3LwFKKvOYZvZL4F7gKuBdIBfoARxWmc8TlXS8Z1K1dMYhe2Kzuy8rd1sFYGY/MrNSM+tfVtnMfmVm35lZ+3C7yMweMrN7zGx1eBtmZlkxj2lgZk+E+0rMbIyZdY/Zf2H4rfw4M5thZhvM7D0zaxcbqJmdYmZTzGyTmS0ws1vMrEbM/oVmdr2ZPRzGuNjMro7dH979d3jmsTD2+WPqdTCzV81sWRjLJ2Z2corv66nAy+7+sLvPc/fZ7v5vd/9dudd0oplNCN+Xb83sNTOrFVOlVqLXEz6+vpkNN7MVZrbOzMaZWZ9ydX5uZl+Z2UYzex1oVm7/DWY2o1xZhU1Rcd6zG8Lf3RAz+zKM5RUzaxxTJ8fM7or5O7nLzP5hZkW7fjulsilxSFq4+zhgGPBU+OHfFbgTuMLd58dUPY/g7/Aw4FfAUOC3Mfv/BfQDBgF9gY3AW2aWF1OnJnAd8MvwOAXAQ2U7zWwA8AxwP9A9rDcY+N9yYV8FfAYcDNwO3GFmZd/yDwl/XgK0iNkuLx94EzgB6Am8BLwcvv5kLQP6liXYeMxsIDASeAfoDRwDjOOH/9MJX4+ZGfAGUAicDPQC3gfGmlmLsE4/gvd/OHAQ8BpwUwqvIxVtgbOB04Efh/HcErP/D8CFwMXAoQSv89w0xSK74u666ZbyjeADZSuwvtzt9pg6ucAk4GXgE+CFcscoAr4ALKbsemBxeL8T4MDRMfvrA2uBi8PtC8M6XWLqnAdsLjsuwQfin8s992lhvGV1FgLPlaszF7g+ZtuBweXqXAis38V7Nb7ccYqA+yuo3wL4OHy+ucDTwM+B3Jg6HwHPV3CMCl8PcGz4+vPK1ZkGXBPefxZ4p9z+R4OPjR3bNwAzKnpPkti+AdgE1I8p+xMwL2Z7KXBtzLYBc4CiqP8XquNNZxyyJ94n+CYaextWttPdSwm+FZ4MNCU4oyhvvIefBKGPgUIzqwfsD2wPy8qOuZbgW3S3mMdsdvc5MdtLgBpAg3C7N/CnsElrfdhM8ixQB2ge87jp5WJbEsadNDOrY2Z3mNmssEllPdAHaJPsMdx9qbsfBhwA3E3wIfkwMNHMaofVehH0f1SkotfTG6gNrCz3vvQAOoR19ifmvQ+V364sX4W/251iNbP6BL+niWU7w7+ZiUgk1Dkue2Kju8/bRZ2yZoUCoAmwppKeOzbZbE2wLyvm543Av+McZ2XM/dI4x0n1y9XfgYEETStzCZrWniRIZClx9xnADOABMzsS+AA4i+BsLxkVvZ4sYDlwVJzHfZdCmNsJElus3BQeX6Yy3nupIvrFSNqEHdT3A5cRtMU/bWblv6z0C9vbyxwKLHH374DZfN//UXbMegTfxGelEMonQFcPOprL38onnYqUAtm7qHMk8KS7v+Tu04HFfP8Nfk+Uvd788OdU4Lg9ON4nBB3d2+O8JyvCOrMJfh+xym+vBJqV+x0etAdx7SQ8E1lGTL9S+HyJ+pkkzXTGIXuippk1L1e2zd1Xmlk28BQwzt0fNrMXCZqY/gr8OaZ+S+BuM3uQICFcDdwM4O5zzexV4GEzG0pwtnILwTfiZ1OI8ybgdTP7ChhBcIbSA+jr7tekcJyFwHFmNo6geWx1nDpfAKeHcZcSvN5aceolZGb/IGiqGUuQeFoQ9P1sBN4Oq90CvGZm8wjeCyPoVH7Y3Tcm8TRjCPpJXjWza4DPCZqDBgJj3P0DgiHB/zWz64AXgf4EndexioCGwB/N7PmwTvlrXSrDPcA1ZvYFQRL9FcH7sjQNzyW7oDMO2RPHE/zjxt6mhvv+CHQELgJw92+BC4Brw2aXMs8QfIufADwCPAbcFbP/FwRt2SPDn7WBgR5cC5AUdx8NnEQw8mhieLsW+Dr5lwrA78NjLOL711ne74AVBM1KbxJ0jH+Q4vO8QzCSbARBIvpPWH6Cu38B4O6jCD7EfxLGMi6MbXsyTxD2EZxIkJweIehoHgF0IUhauPt4gt/fpQT9JWcQdGTHHmd2uH9oWOcEdh6tVhn+TvBF5HGC9xSC92VTGp5LdqFsRIlIlQvH4M9w98ujjkX2PmY2FfjQ3a+IOpbqRk1VIpLxzGw/YADBmVUuwfU0B4Y/pYopcYjI3mA7wbUswwia2GcBP3H3yZFGVU2pqUpERFKiznEREUlJtWiqKigo8I4dO0YdBgAbNmygTp06UYcBKJaKZFI8iiW+TIoFMiueyohlypQp37h7k7g7o57zpCpunTt39kzx3nvvRR3CDoolsUyKR7HEl0mxuGdWPJURCzDZNVeViIhUBiUOERFJiRKHiIikRIlDRERSosQhIiIpUeIQEZGUKHGIiEhKlDhERCQlShwiIpISJQ4REUmJEoeIiKREiUNERFKixCEiIilR4hARkZQocYiISErSmjjMbKCZzTGzeWZ2bZz9Nc3shXD/BDNrG5Y3MrP3zGy9md1f7jG9zeyz8DH3mpntKo6F323niNvG8srU4sp6aSIi1VbaEoeZZQMPAD8BugHnmFm3ctUuAla7e0fgLuD2sHwT8GfgD3EO/Q/gEqBTeBuYTDzFa0q47uXPlDxERPZQOs84+gLz3H2+u28BngcGlaszCHgivP8icJyZmbtvcPcPCRLIDmbWAqjn7uPDFaqeBE5LNqCS0m0MGz1nN1+OiIhAetccLwQWxWwvBvolquPuW81sLdAI+KaCYy4ud8zCeBXNbCgwFKBG8+/XGy9eU0JRUVGyr6HSrV+/PtLnj6VYEsukeBRLfJkUC2RWPOmOJZ2JI1LuPhwYDlCzRScvKy8syKN///5RhUVRUVGkzx9LsSSWSfEolvgyKRbIrHjSHUs6m6qKgdYx263Csrh1zCwHqA98u4tjttrFMRPKyTKuHtAl2eoiIhJHOhPHJKCTmbUzsxrAEGBkuTojgQvC+4OBsWHfRVzuvhT4zswODUdT/Rx4NZlg8nKz2brdqZWbnerrEBGRGGlLHO6+FbgcGA3MBka4+0wzu8nMTg2rPQY0MrN5wO+AHUN2zWwhcCdwoZktjhmR9T/Ao8A84EvgzV3F0rZeFlP/cgI9WxfwuxHTmL30u8p5kSIi1VBa+zjcfRQwqlzZX2LubwLOTPDYtgnKJwM9Uo2lVm42w8/vzan3f8glT05m5OVH0rBOjVQPIyJS7VWrK8eb1avF8PP7sGLdZi59egql27ZHHZKIyF6nWiUOgJ6tC7jjpwcyYcEqbnxtZtThiIjsdfbZ4bgVOa1XIbOXfcfD4+bTpXk9zj90v6hDEhHZa1S7M44y1wzoyjFdmnDjyJl8/GVFI4BFRCRWtU0c2VnGPef0Yr9GtfmfZ6awaNXGqEMSEdkrVNvEAVCvVi6PXnAI27Y7lzw5mQ2bt0YdkohIxqvWiQOgXeM6PHDewXyxfB2/GzGN7dsTXn8oIiIocQBwVKcm/OmkboyeuZy7350bdTgiIhmtWo6qiueXR7Tl86Xfce+7c+nSrC4nHdgi6pBERDKSzjhCZsbNp/fg4DYF/P7f05hRvDbqkEREMpISR4yaOdk8dH5vGtSuwdAnJ/PN+s1RhyQiknGUOMppWrcWj/y8D6s2buHSp6ewZaumJRERiaXEEUePwvoMG9yTSQtX85dXZ1DBTO8iItWOOscTOKVnSz5f9h0PvPclb85YxnclpbQsyOPqAV04rVfc1WpFRKoFJY4KdGycT5bB2pJSIFiv/LqXPwNQ8hCRaktNVRX4+ztfUP56wJLSbQwbPSeagEREMoASRwWWrClJqVxEpDpQ4qhAy4K8uOXN6tWq4khERDKHEkcFrh7Qhbzc7J3Ks7Pgu02lEUQkIhI9JY4KnNarkFvPOIDCgjwMKCzIY+jR7Vj+3WYueWIym0q3RR2iiEiV06iqXTitV+FOI6i6t6zPlc9P44rnpvKP8w4mJ1v5V0SqD33i7YZBBxVywyndeGfWcq57+TNdICgi1YrOOHbThUe0Y/XGUu55dy4N69TguhP3jzokEZEqocSxB357fCdWb9zCw+/Pp0GdGvz6Rx2iDklEJO2UOPaAmXHDKd1ZvbGU2978nIa1a3DWIa2jDktEJK2UOPZQVpbxf2f2ZG1JKde+PJ36tXMZ0L151GGJiKSNOscrQY2cLB762cH0bF3AFc9N5eMvv406JBGRtFHiqCS1a+Tw+IWHsF/D2lzy5GStICgi+ywljkpUULsGT13Uj/p5uVzwz4nMX7k+6pBERCqdEkcla16/Fk9d1BeA8x+byLK1myKOSESkcilxpEH7Jvn86xd9WVtSyvmPTWDNxi1RhyQiUmnSmjjMbKCZzTGzeWZ2bZz9Nc3shXD/BDNrG7PvurB8jpkNiCm/ysxmmtkMM3vOzDJyqtoDWtVn+M9789W3G/nFvyaxccvWqEMSEakUaUscZpYNPAD8BOgGnGNm3cpVuwhY7e4dgbuA28PHdgOGAN2BgcCDZpZtZoXAb4A+7t4DyA7rZaTDOzTm3nN68emiNZz+4H85/NZ3ufCtDRxx21hemVocdXgiIrslnWccfYF57j7f3bcAzwODytUZBDwR3n8ROM7MLCx/3t03u/sCYF54PAiuPckzsxygNrAkja9hjw3s0Zwz+7RizrJ1LAn7O8qWoFXyEJG9UTovACwEFsVsLwb6Jarj7lvNbC3QKCwfX+6xhe7+sZn9HfgaKAHedve34z25mQ0FhgI0adKEoqKiPX5Bu2vMZxt3Kisp3cbfXv2UgrVzI4gosH79+kjfl1iZFAtkVjyKJb5MigUyK550x7JXXTluZg0IzkbaAWuAf5vZz9z96fJ13X04MBygS5cu3r9//6oM9QdWvfVG/PJNTpRxFRUVRfr8sTIpFsiseBRLfJkUC2RWPOmOJZ1NVcVA7MRNrcKyuHXCpqf6wLcVPPZ4YIG7r3T3UuBl4PC0RF+JEi1B26IgI/v1RUQqlM7EMQnoZGbtzKwGQSf2yHJ1RgIXhPcHA2M9WNxiJDAkHHXVDugETCRoojrUzGqHfSHHAbPT+BoqRaIlaBvVqcGWrdsjiEhEZPelLXG4+1bgcmA0wYf7CHefaWY3mdmpYbXHgEZmNg/4HXBt+NiZwAhgFvAWcJm7b3P3CQSd6J8An4XxD0/Xa6gssUvQQrAE7Rm9WvJZ8Xf8+ukpWoJWRPYqae3jcPdRwKhyZX+Jub8JODPBY28BbolT/lfgr5UbafqVLUEb2/bYp20j/vTKZ1zy5GSGn9+HvBo7n5WIiGQaXTkeoXP7tWHY4J58NO8bLnx8Ihs26yJBEcl8ShwRG9y7FXedfRCTv1rN+Y9N4LtNpVGHJCJSISWODDDooEIeOLcXnxWv5WePam4rEclsShwZYmCPFjz0s958vnQd5zwygW/Xb446JBGRuJQ4Mshx+zfj0Qv6sOCb9QwZPp4V6zQlu4hkHiWODHN05yY8fmFfiteUMOTh8SxdWxJ1SCIiP6DEkYEO69CIpy7qy8p1mznr4Y9ZtGrnua5ERKKixJGheu/XkKcv7sfajaWc/fDHLPxmQ9QhiYgAShwZrWfrAp4beiibtm7nrIc/Zt4KrWEuItHbq2bHrY66t6zP80MP5dxHJnDaAx9Su0YOK9dtpmVBHlcP6MJpvQqjDlFEqhmdcewFOjery9Cj27F+8zZWrNuMo8WgRCQ6u0wcZtbZzN41sxnh9oFmdn36Q5NYT/z3q53KSkq3MWz0nAiiEZHqLJkzjkeA64BSAHefTgav872vWrIm/rDcROUiIumSTOKo7e4Ty5VpNr4qlmgxqPxaOQRLmIiIVI1kEsc3ZtYBcAAzGwwsTWtUspN4i0Flm7Fu01b+8O/pWhBKRKpMMqOqLiNYLKmrmRUDC4Dz0hqV7KRs9NSw0XNYsqaElgV5/OHHnfl6VQl3jfmCJWtKeOj83tTPy404UhHZ1yWTONzdjzezOkCWu68Ll3OVKla2GFR5bRrlcc2L0/npP/7L4xceQuuGtSOITkSqi2Saql4CcPcN7r4uLHsxfSFJqk7v1YqnLurHynWbOf3Bj5i2aE3UIYnIPixh4jCzrmb2U6C+mZ0Rc7sQqFVlEUpSDm3fiJcuPZy8GtkMGf4xb81YFnVIIrKPquiMowtwMlAAnBJzOxi4JP2hSao6Ns3nP/9zBPu3qMelz0zh0Q/ma8SViFS6hH0c7v4q8KqZHebuH1dhTLIHGufX5LlLDuWqF6Zx8xuz+erbjfz1lG7kZGuSABGpHMl0jk81s8uA7sQ0Ubn7L9MWleyRWrnZPHDuwdz+1uc8/P58iteUcN85vahTU1OTicieS+Zr6FNAc2AAMA5oBayr8BESuaws47oT9+fm03ow7ouVnPnQxyxbqxUFRWTPJZM4Orr7n4EN7v4EcBLQL71hSWX52aH78egFffjq2w2c/uBHzFryXdQhicheLpnEURr+XGNmPYD6QNP0hSSV7ZguTfn3rw/HHc586L/876hZHHHbWC58awNH3DZWM+yKSEqSSRzDzawBcD0wEpgF3J7WqKTSdWtZj/9cdjj183IZ/v4CisPJETU9u4ikapeJw90fdffV7v6+u7d396bAm1UQm1SyFvXziDc4V9Ozi0gqKkwcZnaYmQ02s6bh9oFm9izwUZVEJ5UuUQe5pmcXkWRVdOX4MOCfwE+BN8zsZuBtYALQqWrCk8qWaHr2Rvk1qjgSEdlbVXTGcRLQy93PAX4M/BY41N3vcXeN69xLxZue3YBv129h+Ptf6kpzEdmlihLHprIE4e6rgbnuvjCVg5vZQDObY2bzzOzaOPtrmtkL4f4JZtY2Zt91YfkcMxsQU15gZi+a2edmNtvMDkslpurutF6F3HrGARSGZx6FBXncekYPBnRvzv+O+pxfPz2F7zaV7uIoIlKdVXQpcXszGxmz3S52291PrejAZpYNPACcACwGJpnZSHefFVPtImC1u3c0syEEo7XONrNuBMvTdgdaAmPMrLO7bwPuAd5y98FmVgPQHOIpKpuevaioiP79+wNw9iFteOzDBdz65uecet+HPHheb7q1rBdtoCKSkSpKHIPKbf9fisfuC8xz9/kAZvZ8eMzYxDEIuCG8/yJwv5lZWP68u28GFpjZPKCvmc0CjgYuBHD3LcCWFOOSOMyMi49qT8/WBVz+7Cec/uBH3HxaD87s0zrq0EQkw1i62rTDJWYHuvvF4fb5QD93vzymzoywzuJw+0uCq9JvAMa7+9Nh+WMEQ4DnEaxGOAvoCUwBrnT3DXGefygwFKBJkya9R4wYkZbXmar169eTn58fdRhA4ljWbnYe+nQTs1dt5+hWOfxs/xrUyLZIYolKJsWjWOLLpFggs+KpjFiOOeaYKe7eJ+5Od0/LDRgMPBqzfT5wf7k6M4BWMdtfAo2B+4GfxZQ/Fh6vD7CVIAFB0Gz1t13F0rlzZ88U7733XtQh7FBRLKVbt/ntb872/f7f637iPe/7V99siCyWKGRSPIolvkyKxT2z4qmMWIDJnuAzNZ1zbRcDse0crcKyuHXMLIdgOpNvK3jsYmCxu08Iy18kWB9EKllOdhbXDOzKYxf0YdGqjZx83weMmbU86rBEJAOkM3FMAjqZWbuwE3sIwZQlsUYCF4T3BwNjw0w3EhgSjrpqR3DdyER3XwYsMrMu4WOO44d9JlLJjtu/GW/85ijaNKrNxU9O5o63Pmfrtu1RhyUiEdrlAg1m9hrsNFPFWmAy8LAnuKbD3bea2eXAaCAb+Ke7zzSzmwhOgUYSNEE9FXZ+ryJILoT1RhAkha3AZR6MqAK4AngmTEbzgV+k9IolZa0b1ubFXx/Oja/N5MGiL5n69RruPacXTerWjDo0EYlAMiv7zAeaAM+F22cTrMfRGXiEoO8iLncfBYwqV/aXmPubgDMTPPYW4JY45dMI+jqkCtXKzebWMw6k934N+dN/PuOkez9gSN/WvDSlmCVrSmhZkMfVA7pwWq/CqEMVkTRLJnEc7u6HxGy/ZmaT3P0QM5uZrsAkMw3u3YruLetx/qPjuffdeTvKy2bZBZQ8RPZxyfRx5JtZm7KN8H7ZOC9dQ1EN7d+iHrk52TuVa5ZdkeohmTOO3wMfhtdYGNAO+B8zqwM8kc7gJHNpll2R6muXicPdR5lZJ6BrWDQnpkP87rRFJhmtZUHejsWgYtXKzWbNxi0U1NZsuyL7qmSH4/YmmDeqJ3CWmf08fSHJ3iDeLLs5Wcam0m0MuPt9iuasiCgyEUm3XSYOM3sK+DtwJHBIeNOopmoudpZdI5hl9+9n9uS1K46kfl4uFz4+iT/+5zM2bN4adagiUsmS6ePoA3QLL8wT2aFslt3yRl5+JHe+8wWPfDCfD+d+w51n9aRP24YRRCgi6ZBMU9UMoHm6A5F9R63cbP544v48f8mhOM6ZD3/MrW/OZvPWbbt+sIhkvGQSR2NglpmNNrORZbd0ByZ7v37tG/HmlUcz5JA2PDxuPqfe9xEzl6yNOiwR2UPJNFXdkO4gZN+VXzOHW884gB93a8Y1L03ntAc+4rfHd+ZXR7cnJzudU6WJSLokMxx3XFUEIvu2Y7o25e3fHs31r85g2Og5jJm9nDvPOoh2jetEHZqIpChh4jCzD939SDNbxw8nOTTA3V3rikpKGtSpwQPnHsyA7kv48ysz+Mk973PiAS0YP/9blqzZROH4sZrvSmQvkDBxuPuR4c+6VReOVAen9mxJ37YNueDxibz8yfdLtGi+K5G9Q1KNzGaWbWYtzaxN2S3dgcm+rXn9WqzfVLpTuea7Esl8yazHcQXwV2A5ULaCjwMHpjEuqQaWrNF8VyJ7o2TOOK4Eurh7d3c/ILwpacgea1mQF7fcgetf+Yy1G3c+IxGR6CWTOBYRrPgnUqnizXdVKzeLozo15tkJX3PcnUW8/MliNGmBSGZJdgXAIjN7A9hcVujud6YtKqkWyjrAh42eQ/GaEgpjVhGcUbyW61+Zwe9GfMoLkxZx82k96NRM4zREMkEyiePr8FYjvIlUmrL5roqKiujfv/+O8h6F9Xn50sN5ftIibn/rc35yzwdcfFR7fnNcR2rXSObPVkTSpcL/QDPLBjq7+3lVFI/IDllZxrn92jCgezNuffNzHhr3Ja99uoQbTu3OCd2aRR2eSLVVYR+Hu28D9jMznWlIZBrl1+TvZ/ZkxK8Oo07NbC55cjIXPzGJRas2Rh2aSLWUbB/HR+HEhhvKCtXHIVWtb7uGvPGbo3j8owXcPWYuJ9w1jiuO7cQlR7WnRo7mvRKpKskkji/DWxag3kmJVG52FkOP7sDJB7bkxtdmMmz0HF7+ZDEDejTn1alLWLKmhJYxnewiUvmSmeTwxqoIRCQVLQvyePj8Poz9fDl/+PenPPjelzv2aeoSkfRK5srxJsA1BGuO1yord/dj0xiXSFKO7dqMWjnZwA8vFiybukSJQ6TyJdMw/AzwOdAOuBFYCExKY0wiKVm6Nv7UJcVrSijdtj3uPhHZfckkjkbu/hhQ6u7j3P2XgM42JGMkmroEYMDd7zN65jJdfS5SiZJJHGVtAEvN7CQz6wU0TGNMIimJN3VJXm4WFx3RFgN+9dQUBj/0MVO+WhVNgCL7mGRGVd1sZvWB3wP3AfWAq9IalUgKYqcuKT+q6rpt2xkxeTF3jfmCn/7jYwZ0b8Y1A7vSoUl+xFGL7L2SGVX1enh3LXBMesMR2T1lU5eUl5Odxbn92nBar5Y8+sECHh73JWNmv8+QQ1pz5fGdaFq3VpyjiUhFdtlUZWadzexdM5sRbh9oZtenPzSRylO7Rg6/Oa4T4645hvP6teGFSYvoP6yIu975gg2bt0YdnsheJZk+jkeA6wj7Otx9OjAkmYOb2UAzm2Nm88zs2jj7a5rZC+H+CWbWNmbfdWH5HDMbUO5x2WY21cxeL39MkYo0zq/JTYN68M7vfkT/Lk245925/GhYEU+N/4rSbdt5ZWoxR9w2lgvf2sARt43llanFuz6oSDWTTB9HbXefaGaxZbv8ihZOkPgAcAKwGJhkZiPdfVZMtYuA1e7e0cyGALcDZ5tZN4Lk1B1oCYwxs87h3FkQLC41m6C/RSRl7RrX4cHzevPJ16u5bdTn/PmVGdw75gvWlpSyZVswAksXEorEl8wZxzdm1oFgYTbMbDCwNInH9QXmuft8d98CPA8MKldnEPBEeP9F4DgLMtQg4Hl33+zuC4B54fEws1bAScCjScQgUqGD2zTghV8dyqM/78Pqjd8njTJaA11kZ8mccVwGDAe6mlkxsABIZpr1QoLVA8ssBvolquPuW81sLdAoLB9f7rFlX/nuJriSvcJ5s8xsKDAUoEmTJhQVFSURcvqtX79escQRdSw5wNbt8a/1KF5TEmlsUb83sRRLYpkUT7pjSWZU1XzgeDOrA2S5+zoz+y3BB3iVMrOTgRXuPsXM+ldU192HEyQ8unTp4rGLBEWp/IJFUVIsP1Q4fizFa0p2Ks/JMkoadWFA9+ZkZVmcR6ZXJrw3ZRRLYpkUT7pjSXouanff4O7rws3fJfGQYqB1zHarsCxuHTPLAeoD31bw2COAU81sIUHT17Fm9nSyr0GkIvEuJMzNNgpq53LpM58w4O73eXVaMVs1jYlUc7u7iEEyX7smAZ3MrF24ENQQYCuVpqIAABQ3SURBVGS5OiOBC8L7g4GxHswNMRIYEo66agd0Aia6+3Xu3srd24bHG+vuP9vN1yDyA6f1KuTWMw6gMJzCpLAgj2GDezLhj8dzz5CDMIMrn5/G8XeOY8TkRZoHS6qt3V28eZcT/4R9FpcDo4Fs4J/uPtPMbgImu/tI4DHgKTObB6wiHOYb1hsBzCIYwXVZzIgqkbRJtAb6oIMKOeXAlrw9azn3jZ3LNS9O554xc/l1/w6c2bsVtcqdqYjsyxImDjNbR/wEYUDiWeViuPsoYFS5sr/E3N8EnJngsbcAt1Rw7CKgKJk4RCpDVpYxsEdzBnRvRtGcldw7di5/fmUG94+dy9CjO3Bu3zbk1VACkX1fwsTh7lrtTyQOM+OYrk3p36UJH3/5LfeOncvfXp/Fg+/N46Kj2tGwdi73jf1SqxHKPmt3m6pEqj0z4/COjTm8Y2MmLVzF/WPnccdbP7zmQxcRyr5odzvHRSTGIW0b8sQv+9Ikv+ZO+3QRoexrlDhEKtE36zfHLS9eU0LRnBVsT3CRocjeRIlDpBIlWo0wy+DCxydx/F3jeOrjhZqRV/ZqShwilSj+aoTZ3PHTA7n77IOoWzOHP786k0NvfZdb3pjFolUbI4pUZPepc1ykElW0GiHAoINa8snXa3j8owX886OFPPbhAo7fvxm/OKIdh7ZvSLlZqEUykhKHSCVLtBohBCOxeu/XgN77NWDp2hKeHv8Vz074mrdnLadr87r88oh2nHpQS2rlZvPK1GKGjZ5D8ZoSCseP1bBeyRhKHCIRaVE/j6sHdOWKYzvx6rRiHv9oIde8NJ3b3vqc3m0K+GDuN2zaGkxromG9kknUxyESsVq52Zx9SBvevPIonr2kH733a8A7s1fsSBplNKxXMoUSh0iGMDMO79CYR37eJ+EsokviTPsuUtWUOEQyUKJhvQ5c/MQk3p65TLPzSmSUOEQyULxhvTVzsjhu/6ZMX7yWoU9N4bBbx3LrqNnMW7E+oiilulLnuEgGih3WW7ymhMKYYb1bt22naM5KRkxexGMfLuDh9+fTe78GnN2nNScd2II6NfVvLemlvzCRDJVobZCc7CyO79aM47s1Y+W6zbz8yWJemLyIa16azg2vzeTkA1tw9iGtObhNA16dtiThNSUiu0uJQ2Qv1qRuTX71ow4MPbo9n3y9mhcmLeL16UsZMXkxTevWYNWGUraG82NpSK9UFvVxiOwDggsLG3LH4J5M/NPx3P7TA1i98fukUUZDeqUyKHGI7GPya+Zw9iFt2Lot/ky8xWtKGDNrOVu2alSW7B41VYnso1oW5FEc57oPM7j4ycnUq5XDwB7NOaVnSw5r34icbH2PlOQocYjso64e0IXrXv6MktJtO8rycrP526DuNMqvyWufLmHUZ8sYMXkxjfNrcOIBLTilZ0t6t2lAVpYmW5TElDhE9lG7mqn3mK5N2VS6jaI5K3jt06WMmLyIJz/+ihb1a3HygUESOaCwPmamCRflB5Q4RPZhFc3UC8E8WQN7tGBgjxas37yVd2cv57VPl/Cv/y7kkQ8WsF+j2nRums/7c79hsyZclJASh4gAQaf6oIMKGXRQIWs3ljJ65jJem76Ed2av2Klu2egsJY7qSb1hIrKT+rVzOeuQ1jx1Ub+EEy4Wrynhg7krNTqrGtIZh4hUKNHoLIDzH5tI3Zo5/KhLE07o1oz+XZpSPy+3iiOUqqbEISIVSjQ666ZB3WlQuwZjZi9nzOwVvD59KTlZRt92DTmhWzOO378ZrRvWjjBySRclDhGpUEUTLgIc360Z27c70xavYcys5bwzazk3vjaLG1+bRdfmdXckkQMK6zPyU82dtS9Q4hCRXUo04WKZrCzj4DYNOLhNA64Z2JWF32xgzOwgiTzw3jzuGzuPerWy2bBlO9s0d9ZeT53jIlLp2jauw8VHteeFXx3GlOtP4M6zerJlm+9IGmVKSrfxv6Nm4x5/ehTJTEocIpJWDerU4IyDW7G5NP7oqxXrNnPILe9y1QvTePmTxaxYt6mKI5RUpbWpyswGAvcA2cCj7n5buf01gSeB3sC3wNnuvjDcdx1wEbAN+I27jzaz1mH9ZgSraA5393vS+RpEpHIkGp1VUDuXwzs0YtwXK/nP1GIAujavy9Gdm3BUp8Yc0rYhtWJWQ9RV7NFLW+Iws2zgAeAEYDEwycxGuvusmGoXAavdvaOZDQFuB842s27AEKA70BIYY2adga3A7939EzOrC0wxs3fKHVNEMlCi0Vk3nNKd03oVsn27M2vpd3ww9xs+mLuSf320kOHvz6dmThZ92zXk6E5NKN2+nfvenUtJqa5ij1I6zzj6AvPcfT6AmT0PDAJiP+QHATeE918E7jczC8ufd/fNwAIzmwf0dfePgaUA7r7OzGYDheWOKSIZaFdzZ2VlGT0K69OjsD6X9u/Axi1bmbBgFR98ESSSW0bNjntcXcVe9SxdnVJmNhgY6O4Xh9vnA/3c/fKYOjPCOovD7S+BfgTJZLy7Px2WPwa86e4vxjy2LfA+0MPdv4vz/EOBoQBNmjTpPWLEiDS8ytStX7+e/Pz8qMMAFEtFMikexRJYtWk7vyuKfyEiwC961KBrg2ya1jaC759Va1/7PR1zzDFT3L1PvH175XBcM8sHXgJ+Gy9pALj7cGA4QJcuXTzeEMIoJBrOGAXFklgmxaNYvvd/08bG7SfJMnh8xhYAmtatSd92DenXvhH92jWkY5P8KpkmPur3Jla6Y0ln4igGWsdstwrL4tVZbGY5QH2CTvKEjzWzXIKk8Yy7v5ye0EUkEyXqJ/nf03twQKv6TFiwignzVzFhwbe8Pn0pAA1q59K3XUP6tgsSyf4t6pGd9f1U8boYMXXpTByTgE5m1o7gQ38IcG65OiOBC4CPgcHAWHd3MxsJPGtmdxJ0jncCJob9H48Bs939zjTGLiIZaFdXsXdsWpfz+u2Hu7NoVQnjF3zLxAWrmLhgFaNnLgegbs0cWjXIY+6K9TvWZFcne2rSljjcfauZXQ6MJhiO+093n2lmNwGT3X0kQRJ4Kuz8XkWQXAjrjSDo9N4KXObu28zsSOB84DMzmxY+1R/dfVS6XoeIZJZdXcUOYGa0aVSbNo1qc1afoPFi6doSJi5YxYQFqxgxadGOpFGmpHQbN742kz5tG1BYkBdJP8neIq19HOEH+qhyZX+Jub8JODPBY28BbilX9iEknOVZRCShFvXzdqw38tyEr+PWWb2xlCNvf4/G+TXp1aaAg1oX0KtNAQe2KiC/ZvyPy+p4Xcle2TkuIrInEl2M2KRuTX5zbEemfr2GaYvW8M6soHkry6Bzs7o7EslBrRvQsWk+r3265Ad9LtWlyUuJQ0SqnUSd7H86cX9O61XI+YcFZas3bGHa4jVM+3oNUxetYdRnS3l+0iIgWDGxdNv2HUvqlqkO15UocYhItbOrixHLNKhTg2O6NOWYLk0B2L7dWfDthjCRrObp8fGbvIrXlPDG9KV0b1mPNg1rV8lw4KqkxCEi1VJZJ3sqsrKMDk3y6dAkn5/2bsV7n69MuDriZc9+AgSjuLq1rEf3lvXpURj87NCkDjnZ388xu7cNDVbiEBHZTRWtjti1eT1mLFnLzCVrmVH8Hc9O/IpN4RxbNXOy6NqiHj1a1mPLtu2MnLZkR5PX3tBPosQhIrKbdnVdyQGt6u+ou3XbduZ/s2FHIpm5ZC0jP13Cuk1bdzpuSek2bnp9Fge3aUCrBnkZ19SlxCEisgeSua4EICc7i87N6tK5WV1O7xWUuTvtrxtFvBkDV23YwtHD3iMvN5vOzfLp0jx4bJfmdenSrC5N6tbc6VqTqhoarMQhIhIRM0s8NDi/Jr//cWfmLF/HnGXrGPv5CkZMXrxjf0Ht3CCRhMlkxbpNDH9//o7msHQ2eSlxiIhEKOHQ4JP23+kD/5v1m/li+Tq+WLaOOcvX88XydbwytZh1m3du7oKgyetvr8/ikHYNaVGvVqU1eSlxiIhEKNmhwQCN82vSOL8mh3dovKPM3Vm6dhOH3zY27vG/3bCFI24bS63cLNo3zqd9kzq0b5JPhyZ16NAkn3aN61An5qr4suauGs079k4UsxKHiEjEdmdocJmy5q7CBE1ejfNrcNUJnZm/cgNfrlzP9MVrGfXZUmKn6mperxYdmtYBh4kLV1G6reJ1mpQ4RET2AYmavK4/qdtOSWlT6Ta+XrWRL1esZ/43QUKZv3IDny5aE7ejvjwlDhGRfcCuhgbHqpWbvWOEV6x2176R1HMpcYiI7COSHRqcSKIRXuVl7bKGiIhUC1cP6EJebvYu6+mMQ0REgB82dy2toJ7OOEREZIfTehXy0bXHsmXZvCmJ6ihxiIhISpQ4REQkJUocIiKSEiUOERFJiRKHiIikRIlDRERSosQhIiIpUeIQEZGUKHGIiEhKlDhERCQlShwiIpISJQ4REUmJEoeIiKREiUNERFKS1sRhZgPNbI6ZzTOza+Psr2lmL4T7J5hZ25h914Xlc8xsQLLHFBGR9Epb4jCzbOAB4CdAN+AcM+tWrtpFwGp37wjcBdwePrYbMAToDgwEHjSz7CSPKSIiaZTOM46+wDx3n+/uW4DngUHl6gwCngjvvwgcZ2YWlj/v7pvdfQEwLzxeMscUEZE0SufSsYXAopjtxUC/RHXcfauZrQUaheXjyz22MLy/q2MCYGZDgaHh5mYzm7EbryEdGgPfRB1ESLEklknxKJb4MikWyKx4KiOW/RLt2GfXHHf34cBwADOb7O59Ig4JUCyJZFIskFnxKJb4MikWyKx40h1LOpuqioHWMdutwrK4dcwsB6gPfFvBY5M5poiIpFE6E8ckoJOZtTOzGgSd3SPL1RkJXBDeHwyMdXcPy4eEo67aAZ2AiUkeU0RE0ihtTVVhn8XlwGggG/inu880s5uAye4+EngMeMrM5gGrCBIBYb0RwCxgK3CZu28DiHfMJMIZXskvb08olvgyKRbIrHgUS3yZFAtkVjxpjcWCL/giIiLJ0ZXjIiKSEiUOERFJyT6dODJlehIza21m75nZLDObaWZXRhVLTEzZZjbVzF7PgFgKzOxFM/vczGab2WERxnJV+DuaYWbPmVmtKn7+f5rZitjrjsysoZm9Y2Zzw58NIoxlWPh7mm5m/zGzgqhiidn3ezNzM2scZSxmdkX43sw0szuqIpZE8ZjZQWY23symmdlkM+tbmc+5zyaODJueZCvwe3fvBhwKXJYBU6VcCcyOOIYy9wBvuXtXoCcRxWVmhcBvgD7u3oNgAMaQKg7jXwTT7MS6FnjX3TsB74bbUcXyDtDD3Q8EvgCuizAWzKw18GPg6yqKI24sZnYMwSwWPd29O/D3KOMB7gBudPeDgL+E25Vmn00cZND0JO6+1N0/Ce+vI/hgLKz4UeljZq2Ak4BHo4ohJpb6wNEEI+xw9y3uvibCkHKAvPC6otrAkqp8cnd/n2CEYazYqXmeAE6LKhZ3f9vdt4ab4wmupYokltBdwDVAlY3ySRDLpcBt7r45rLMi4ngcqBfer08l/x3vy4kj3pQnkX1YlwlnAO4FTIgwjLsJ/tm2RxhDmXbASuDxsOnsUTOrE0Ug7l5M8E3xa2ApsNbd344ilnKaufvS8P4yoFmUwcT4JfBmVE9uZoOAYnf/NKoYYnQGjgpn+R5nZodEHM9vgWFmtojgb7pSzwz35cSRccwsH3gJ+K27fxdRDCcDK9x9ShTPH0cOcDDwD3fvBWyg6ppifiDsOxhEkMxaAnXM7GdRxJJIeIFs5GPozexPBE2wz0T0/LWBPxI0w2SCHKAhQVP01cCIcMLWqFwKXOXurYGrCM/oK8u+nDgyanoSM8slSBrPuPvLUcUBHAGcamYLCZrvjjWzpyOMZzGw2N3LzsBeJEgkUTgeWODuK929FHgZODyiWGItN7MWAOHPKmsGicfMLgROBs7z6C4E60CQ4D8N/5ZbAZ+YWfOI4lkMvOyBiQRn81XSWZ/ABQR/vwD/Jmi6rzT7cuLImOlJwm8ejwGz3f3OKGIo4+7XuXsrd29L8J6MdffIvlW7+zJgkZl1CYuOI5gxIApfA4eaWe3wd3YcmTGAIHZqnguAV6MKxMwGEjRznuruG6OKw90/c/em7t42/FteDBwc/j1F4RXgGAAz6wzUINqZcpcAPwrvHwvMrdSju/s+ewNOJBj58SXwpwjjOJKgeWE6MC28nZgB709/4PUMiOMgYHL4/rwCNIgwlhuBz4EZwFNAzSp+/ucI+ldKCT4MLyJYauDd8J9/DNAwwljmEfQdlv0dPxRVLOX2LwQaR/i+1ACeDv9uPgGOjfhv5khgCvApQX9q78p8Tk05IiIiKdmXm6pERCQNlDhERCQlShwiIpISJQ4REUmJEoeIiKREiUOkEpjZtnAm0rJbpV39bmZt480KKxKVtC0dK1LNlHgwE6nIPk9nHCJpZGYLzewOM/vMzCaaWcewvK2ZjQ3XtXjXzNqE5c3CdS4+DW9lU55km9kj4VoPb5tZXmQvSqo9JQ6RypFXrqnq7Jh9a939AOB+gpmJAe4DnvBgXYtngHvD8nuBce7ek2DOrplheSfgAQ/WelgD/DTNr0ckIV05LlIJzGy9u+fHKV9IMP3E/HCiy2Xu3sjMvgFauHtpWL7U3Rub2UqglYfrOoTHaAu848FCTpjZ/wNy3f3m9L8ykZ3pjEMk/TzB/VRsjrm/DfVPSoSUOETS7+yYnx+H9//L98vSngd8EN5/l2AthbJ14etXVZAiydK3FpHKkWdm02K233L3siG5DcxsOsFZwzlh2RUEqx5eTbAC4i/C8iuB4WZ2EcGZxaUEM5+KZAz1cYikUdjH0cfdo1ybQaRSqalKRERSojMOERFJic44REQkJUocIiKSEiUOERFJiRKHiIikRIlDRERS8v8BmARNtZw3ynIAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "7R8ZX2pVxr_E", "colab": { "base_uri": "https://localhost:8080/", "height": 324 }, "outputId": "cfbce53b-1366-4e88-b80b-a84a64ca531b" }, "source": [ "pd.DataFrame(history.history).plot(figsize=(8, 5))\n", "plt.grid(True)\n", "plt.gca().set_ylim(0, 1);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeMAAAEzCAYAAAACSWsXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXzU1b3/8deZLTOTfd+BAGEPu6AoEEQrVkWxIrV1o1Wrrfa29rZaa73e6u2iVrtcN/S6/myRqljrRkUSQUVlEQTZ92xAyL5nlvP74zuZTBZIgIRJJp/n4zGP7zrfOSeEeeec7/l+v0prjRBCCCGCxxTsAgghhBADnYSxEEIIEWQSxkIIIUSQSRgLIYQQQSZhLIQQQgSZhLEQQggRZF2GsVLqOaXUUaXU1uNsV0qpvyil9iilvlJKTe75YgohhBChqzst4xeAeSfYfjGQ7XvdAjx5+sUSQgghBo4uw1hrvRooP8EulwMvacNnQIxSKrWnCiiEEEKEup44Z5wOFAQsF/rWCSGEEKIbLGfyw5RSt2B0ZeNwOKZkZmb22LG9Xi8mU+iNRwvFeoVinSA06yV16j9CsV6hVqddu3Yd01ondratJ8K4CAhM1Qzfug601kuAJQBTp07V69ev74GPN+Tn55Obm9tjx+srQrFeoVgnCM16SZ36j1CsV6jVSSl18HjbeuJPjreA632jqs8GqrTWJT1wXCGEEGJA6LJlrJT6O5ALJCilCoH/AqwAWuungHeBbwJ7gHpgcW8VVgghhAhFXYax1vqaLrZr4Ec9ViIhhBBigAmdM+NCCCFEPyVhLIQQQgSZhLEQQggRZBLGQgghRJBJGAshhBBBJmEshBBCBNkZvR2mEEIIcUZ43OCqB1dDwLShk3Un2Ga2whVPnJHiShgLIYQ4dVqD1wNeN3hdxtQTOO/ybQ/Y5m4ETxO4m33zvqm7yXh5jOnQfbuh4f12+za17hewb2ug+sLU6zr5upisYHWC1WG8whN6/ud1HBLGQggRqrSG5jpoqjFezTWt8021vmk1NNe2W19tzLsaAkLW4wvWwHn3qYVeN2UoCxx1gtkGFjtYbGAOA0vAKyzSmLYEqD9MnSdYFzgN2M9s7bW6dEXCWAghzgSvxxeItW1bdR5XQAuvud3Ut73DuuYO0/FHCmG3pfUzWsIV3XXZTFYj1AJfEUlGAJqtxnazBUwWY95k8S23zFvBZPbt51vX8gpcbpm3hLULVbsvcMNat5ltrF69OqQeFHEiEsZCCHEiWhvdo43VvhZjtW++puO8f9m3LnBbc+3pl8Vk8YWYzR9YLfNmjxvsGRCd7gvUKLBFdAzZztZZwk6/bOK0SBgLIfo/r9c36Mb3am4ZiFNnTJvr/INyMg9tgby1xrYO+7U7RrOvy9br7roM1nCwRxkhGBZpzEelGcv26ICADG/X5RrQ9drSOjTb2s63TE3m4378lyH2uMGBRsJYCNG3aA2NVVBfBvXlvmn7V7v1DRV0qzsWGAawD7A4wOZsPW/YMu+MB1umb124L0x9ARsWHTAfELq2SKPbVohTJL89QoiT1zKC1tPse7l806aA+ebO510NnYRswHJD+fFboiarEZbOeHDGQfLY1nlbhBGeLQNy/PPONqG7+vMNzJrzDTDJbRZE3yFhLMRA4HEZ5y8bK33nNat8r4D5TtZPqy6DLy2dh2s3W6LHpcxGiLaEa8JwcE4PCNuA0G2Zt0WAUqf1sV6zXYJY9DkSxkL0Fx63Eab15Ubrsf30ROHqqu/i4MrobrVH+7pioyEmkxodizMt0xgF23Ie0z/f2bpuzFvsRsCGRUsoCuEjYSxEMDTXdx6o9cdZ1xK2x2OygD2mNVDt0RCV2jp4qP22sIB5e7TR4uwkGLfn55Msg4KE6HUSxkL0BFcDYY2lULTROO9ZVxrwOuZ7+ebrjxmXyhyPLcJoOTrijGlsVttlRxw4Y9suh0WedvetECJ4JIyFaM9/16Jqo1XqD9RSI0gDl+tKoa4Mmms4B+Czdscyhxk3TwhPgPBESBoD4fHtgrXd1GILQqWFEMEkYSxCi9bGtaEtN17wT6vaLXc2rWq9SYP2dH58ZTZCNTzBeMUOMZad8ewsLGfk5PPabu+BAUdCiNAnYSz6Plcj1B2F2qNQe8T3Otpx2ljpC1LviY+nTAHXjvquG43OAPuY1vUt51Wdcb6w9YWrPea4g45K8vMZOSq35+svhAh5EsYiOLwe49zqicK1ZdvxBi45EyAi2egGzswCR0xAyLYL28D1tnBprYYwrbXvOmgvaG0s++bxelGNjXhqajrdrlv2863TXg0eN9rjQbvd4PGgPd6TW+f2oD1u43huT+s6rwc8XmPa6bLXdxwPeFuO127Z6/EfK6aigsLX30BZjftGK4vvZbWAxYKyWE+wztx22WrsZ9xvWqGUMv7PKAW0zHP89SZTu/UB+5rMmOxhKIcDk92OyeFA2e0o8/HvLtYrvyfNzXjq6vDW1eGtrfVPPf75OtCa+O9/74yUR8JY9DytjVZqZQFUFQRMD0FVAeccOwAfVXXegrX5blAfkWycXx06p3W5JXgjko1Wai88YUVrjaeyEndpKZ5jx1B2O9b0DCyJCcYXTB+lm5vxNjejXS60ywVuN9rtNpbdbnSzC+3uZJsrYB9XsxEkAduc+/dRtnev8cVstrT9om758ra2fvG3WWc2d76PUngbG/HW16MbGvD6Xv75+pZ19ej6BmPfBt++9QH719cb842N6IYGX0h10SsCJAG7ev+f5OSZTGA2o8xm43fNN083l03VVTQ3NBj/pr4/CrTbBS7fv7fvhbsbt/YMAmWzGcHsC2nldBDb1Myhl15GOR2Y7A5MDjvK7sDk6DivXa42QeoP17qO67x1dejm5i7LZAoPlzAWfZjXa3QbVxZA1aFOQrfAeFRbIIsDYjIhOpOy+ATSsid1ErJJRqu1F2i3G3dZOe7SUtylR33Tltex1vljx8DV8ZFwymbDmp6ONSMDa0Y6towMrBmZ/nlzdHSvlBtAe714yspwHT6Mq7gEV0kx7pIS37zx8pSV9cpnRwJHe+XI3aCU8cUc8FJOByaHE2t0tPFl7DCWTQ67EfxKGachTEYrzPgDSoHJhPKt27tvP8OyhwfsawpozfladP7WoMnX0jSjzBYwm1BmX0uyJTjNZt8fKp2v8wdmyx8nJpOxzRwQqCaT8XmnIT8/n/HduAxNa936B1lAQPuXXb4Qd7uN1jftehk0xjrd+vKv19q/zehlaLsOrdEeL7q5yfjDqrEB3dBo/HHV2IC33bwuLjYCtKzM2Lflj7PGxk7/n/p/dRwOTOHhmMPDMYWHY4qIwJqW5pv3rY+IwOT0TVvW++d9U6fztP5NToaEsWhLa3R1Ka4DO3Af2o23vBhqjqJrjkJtKbr2GNSWGa0Q//8vBRYn2h4LYbHosLMhLBptM+7Zq62RYA5Du71w1EPxoUNY6sNR5kawFKLMh40vMlPLF5a59QvPZA74QmtZZzJaWYHrAHdZWbuQbQ1aT3l5p60mc0wMlsRELImJhGVlYUlK9C9bEhLwNjTQXFiIq7AIV2EhzYUFNGzejLe6us1xTJGRxMVEU/j661jTM7BmZvgCOwNrejomu/24P3Jvfb0vVA93GrTukhKjRRtAOZ1YU1OxpqZiHz0aS0oyJofT+LnYWroarcaXv9Xa2jq1HmebLbAL05jHamX1Rx8xa8aM1paWyw1uV5sv8g7rXG6jW7bNl3vAF7zWRivH6QvYgHkjcJ3GNCzstAOqM1vy84kfwNdOK6XAajV+B/q4vfn5TDzOv5V2uXy9JkavirJafQFr/D/ob/pficXJaxlhXHsUXVWMu2AProKDuIsLcR05gutYJe6yGlxVTbhrPbgbfK2J44o6zvo636uw882+7jSn1pRpDZ7jjFg+XWYzlvh4LImJWJOScIwb2xqwiW3DVtlO7TIiT3W1L5xbg/rw5s007d1H7eo16KamtkVKTMCWboSzOSoS1+EjRtAWF+OpandO3GTCkpyMNSUFx7hxWL9xIZbUVKypaVjTjAA2RUX1SlB1YLViCu+d3gohToeyWjFbrZgjI4NdlB4hYdyfeb3Gda9VhVBTgq4+jLv4IO7iAlyHj+AuLcdVVoOrshF3HbjqzbgbTaDbfokrK1ijbFjjYgkbFoM1OdH48k8fhCkhExWVAjZHu241s2/Z193mXw7Ybmk9txXYDZfve9Rby8AZ7fH4usQ8/gEq/oEu3V2nNZa4OCyJiZhjY3t9MIg5KgrzmDHYx4zxr9vh+ytee724jx0zQrqosDW0Cwpp2LgRT10d1pQUI2wnTjBCNjXVH7SWpKR++Ze9EOLUyf/4M8Q/MOhoKd7qKrxNzejmZnRzE7qpyRh8E7DO29iIrqsibt9OSt54AV1XjW6oxdtQh26sRzc1GYN2PBrtUXjdCneDuZOgNWGNjcOSGE14coI/ZK2Zw7AMycaalo4pMvLMtLLaUUq1hvcptlD7ImUyYU1KwpqUBJMnBbs4Qoh+QML4NLWG7FHcR0uNaWnbqav0KJ7SYx3O+3VxZJRZE2aCGrPGZNYos9E1o2w2TGGRmKIcmB3hKEcEpsgYLKkZWDOzsKSlY01JwZKSgjkmJihBK4QQovskjE/AU11tnNc7erRDyLpalks7H31rio7GkpiANTGJ8JyRWMKGYFHlWFxFmJuKUKrZGMhp1iiTNgbXRCejYtIwxadDTDoqOpOth8oZd+5FEJVhPEKuD19eI4QQ4tRIGPtoj4emPXto+HITDV9+ScOmTTQfPNhhP1N0NFbfiNuwIVlYkpKMwUBJScZI3AgLlsYDmEo3Q9F6KPrAuN0iGHd1Sp8CyfOMOz5FpUNUmjHvTOg0aI815EOadHUKIUQoG7Bh7KmspGHzZuo3baJh0yYav9qCt64OAHNcHI6JE4m+8kpsgwe3Bm1iIqawsNaDuJvg8BYoXAeF+bBzPVQcMLYpMySPhZyFkDEV0qdC/HBp2QohhOhgQISx0erdS4MveBs2baJ5/35jo9lM2MgRRF8+H8fEiTgmTsSamdnxPKvWRtDu2uAL3/Vw+Cvw+O7iEpVutHqnft8I39SJYDtzF4wLIYTov0IyjD1VVTRs3twavpu/am31xsYard4rrjDCd9zYzq+j1BoOfQYHPja6mwvXG5cRAVidRtfx2bcZLd6MqUZ3sxBCCHEKQiKMXcXFONZ8TPG/PzBavfv2GRtMJsJGjiRq/mU4W1q9gwadeHSx1rBnJXz0B6MFDJAwEkZc1NrdnDQGzCHxoxNCCNEHhESi1K1dS9Qrr1AbE2O0euf7upxzxnX/7kFaw64VRggXb4ToQXDpYzD2SuNpQEIIIUQvCYkwjpw7ly1uD+ddvfDkr6nVGna+a4RwyWaIGQzz/wrjvw2W0LkRhRBCiL4rJMLYHBODJznp5ILY64Ud/4KPHoYjWyBuKFzxpDH6uRcezSeEEEIcT0iE8UnxemDbP2H1w3B0G8Rnw4IlMO5bch5YCCFEUAyc9PF64Ovl8NFDcGynMSjrW/8HYxeAqXcfKiCEEEKcSOiHsccNW18zWsJle4yR0AtfgNGXyw04hBBC9AmhG8YeF3y1DNY8AuX7IDkHrn4ZRl0qISyEEKJPCb0wdjfDV0thzR+NO2alToBv/w1GfhPk6UVCCCH6oJAJY+V1wfrnYM1jUHUI0ibDvD8YN+uQEBZCCNGHhUYY71rB9M9/CE3HIOMs42Ydw+dKCAshhOgXQiOMHbE0hSViv/oZGDpHQlgIIUS/0q2RTEqpeUqpnUqpPUqpuzvZPkgplaeU+lIp9ZVS6ps9X9QTyJzGl5N/D8POlyAWQgjR73QZxkopM/A4cDEwBrhGKTWm3W73Asu01pOAbwNP9HRBhRBCiFDVnZbxNGCP1nqf1roZWApc3m4fDUT55qOB4p4rohBCCBHalNb6xDsodRUwT2t9k2/5OmC61vr2gH1SgX8DsUA4cIHWekMnx7oFuAUgOTl5ytKlS3uqHtTW1hIREdFjx+srQrFeoVgnCM16SZ36j1CsV6jVac6cORu01lM729ZTA7iuAV7QWv9RKXUO8LJSapzW2hu4k9Z6CbAEYOrUqTo3N7eHPh7y8/PpyeP1FaFYr1CsE4RmvaRO/Uco1isU63Q83emmLgIyA5YzfOsCfR9YBqC1XgvYgYSeKKAQQggR6roTxuuAbKVUllLKhjFA6612+xwC5gIopUZjhHFpTxZUCCGECFVdhrHW2g3cDqwAtmOMmv5aKfUbpdR8324/A25WSm0G/g7cqLs6GS2EEEIIoJvnjLXW7wLvtlt3X8D8NuDcni2aEEIIMTDI44uEEEKIIJMwFkIIIYJMwlgIIYQIMgljIYQQIsgkjIUQQoggkzAWQgghgkzCWAghhAgyCWMhhBAiyCSMhRBCiCCTMBZCCCGCTMJYCCGECDIJYyGEECLIJIyFEEKIIJMwFkIIIYJMwlgIIYQIMgljIYQQIsgkjIUQQoggkzAWQgghgkzCWAghhAgyCWMhhBAiyCSMhRBCiCCTMBZCCCGCTMJYCCGECDIJYyGEECLIJIyFEEKIIJMwFkIIIYJMwlgIIYQIMgljIYQQIsgkjIUQQoggkzAWQgghgkzCWAghhAgyCWMhhBAiyCSMhRBCiCCTMBZCCCGCTMJYCCGECDIJYyGEECLILMEugBD9hcvlorCwkMbGxk63R0dHs3379jNcqt7Vl+tkt9vJyMjAarUGuyhCnDYJYyG6qbCwkMjISIYMGYJSqsP2mpoaIiMjg1Cy3tNX66S1pqysjMLCQrKysoJdHCFOm3RTC9FNjY2NxMfHdxrE4sxSShEfH3/cXgoh+hsJYyFOggRx3yH/FiKUSBgL0Y9EREQEuwhCiF4gYSyEEEIEmYSxEP2Q1pqf//znjBs3jpycHF599VUASkpKmDVrFhMnTmTcuHGsWbMGj8fDjTfe6N/3scceC3LphRDtyWhqIU7Bf//ra7YVV7dZ5/F4MJvNp3zMMWlR/NdlY7u17xtvvMGmTZvYvHkzx44d46yzzmLWrFn87W9/46KLLuJXv/oVHo+H+vp6Nm3aRFFREVu3bgWgsrLylMsohOgd0jIWoh/6+OOPueaaazCbzSQnJzN79mzWrVvHWWedxfPPP8/999/Pli1biIyMZOjQoezbt4877riD999/n6ioqGAXXwjRTrdaxkqpecCfATPwrNb6953sczVwP6CBzVrr7/RgOYXoUzprwfaFa3JnzZrF6tWreeedd7jxxhu58847uf7669m8eTMrVqzgqaeeYtmyZTz33HNBLacQoq0uW8ZKKTPwOHAxMAa4Rik1pt0+2cAvgXO11mOBn/RCWYUQPjNnzuTVV1/F4/FQWlrK6tWrmTZtGgcPHiQ5OZmbb76Zm266iY0bN3Ls2DG8Xi/f+ta3ePDBB9m4cWOwiy+EaKc7LeNpwB6t9T4ApdRS4HJgW8A+NwOPa60rALTWR3u6oEKIVgsWLGDt2rVMmDABpRQPPfQQKSkpvPjiizz88MNYrVYiIiJ46aWXKCoqYvHixXi9XgB+97vfBbn0Qoj2uhPG6UBBwHIhML3dPiMAlFKfYHRl36+1fr9HSiiE8KutrQWMG148/PDDPPzww22233DDDdxwww0d3ietYSH6tp4aTW0BsoFcIANYrZTK0Vq3GbaplLoFuAUgOTmZ/Pz8Hvp440uqJ4/XV4RivfprnaKjo6mpqTnudo/Hc8Lt/VFfr1NjY+NJ/y7119+/roRivUKxTsfTnTAuAjIDljN86wIVAp9rrV3AfqXULoxwXhe4k9Z6CbAEYOrUqTo3N/cUi91Rfn4+PXm8viIU69Vf67R9+/YTDtDqCwO4elpfr5PdbmfSpEkn9Z7++vvXlVCsVyjW6Xi6c2nTOiBbKZWllLIB3wbearfPmxitYpRSCRjd1vt6sJxCCCFEyOoyjLXWbuB2YAWwHVimtf5aKfUbpdR8324rgDKl1DYgD/i51rqstwothBBChJJunTPWWr8LvNtu3X0B8xq40/cSQgghxEmQO3AJIYQQQSZhLIQQQgSZhLEQog232x3sIggx4EgYC9GPXHHFFUyZMoWxY8eyZMkSAN5//30mT57MhAkTmDt3LmBcn7l48WJycnIYP348r7/+OgARERH+Y7322mvceOONANx4443ceuutTJ8+nV/84hd88cUXnHPOOZx33nnMmDGDnTt3AsZ1x//5n//JuHHjGD9+PH/9619ZtWoVV1xxhf+4H3zwAQsWLDgTPw4hQoY8QlGIU/He3XB4S5tVDo8bzKfxXyolBy7u8AyWNp577jni4uJoaGjgrLPO4vLLL+fmm29m9erVZGVlUV5eDsADDzxAdHQ0W7YYZayoqOjy4wsLC/n0008xm81UV1ezZs0aGhoa+Pzzz7nnnnt4/fXXWbJkCQcOHGDTpk1YLBbKy8uJjY3lhz/8IaWlpSQmJvL888/zve9979R/DkIMQBLGQvQjf/nLX1i+fDkABQUFLFmyhFmzZpGVlQVAXFwcACtXrmTp0qX+98XGxnZ57IULF/qfx1xVVcUNN9zAzp07MZvNuFwu/3FvvfVWLBZLm8+77rrr+H//7/+xePFi1q5dy0svvdRDNRZiYJAwFuJUdNKCbejlu1Xl5+ezcuVK1q5di9PpJDc3l4kTJ7Jjx45uH0Mp5Z9vbGxssy08PNw//+tf/5o5c+bw0ksvUVZW1uVdkBYvXsxll12G3W5n4cKF/rAWQnSPnDMWop+oqqoiNjYWp9PJjh07+Oyzz2hsbGT16tXs378fwN9NfeGFF/L444/739vSTZ2cnMz27dvxer3+FvbxPis9PR2AF154wb/+wgsv5Omnn/YP8mr5vLS0NNLS0njwwQdZvHhxz1VaiAFCwliIfmLevHm43W5Gjx7N3Xffzdlnn01iYiJLlizhyiuvZMKECSxatAiAe++9l4qKCsaNG8eECRPIy8sD4Pe//z2XXnopM2bMIDU19bif9Ytf/IJf/vKXnHfeeW1GV990000MGjSI8ePHM2HCBP72t7/5t333u98lMzOT0aNH99JPQIjQJX1JQvQTYWFhvPfee51uu/jii9ssR0RE8OKLL3bY76qrruKqq67qsD6w9QtwzjnnsGvXLv+DIh588EEALBYLjz76KI8++miHY3z88cfcfPPN3a2OECKAhLEQ4rRNmTKF8PBw/vjHPwa7KEL0SxLGQojTtmHDhmAXQYh+Tc4ZCyGEEEEmYSyEEEIEmYSxEEIIEWQSxkIIIUSQSRgLIYQQQSZhLESICnxCU3sHDhxg3LhxZ7A0QogTCYkw3nCwgsc2NNLo8gS7KEIIIcRJC4nrjOua3Gwu9fDoB7u455tyKz7R+/7wxR/YUd72AQ0ej8f/1KNTMSpuFHdNu+u42++++24yMzP50Y9+BMD999+PxWIhLy+PiooKXC4XDz74IJdffvlJfW5jYyO33XYb69ev999ha86cOXz99ddcf/31eDwevF4vr7/+OmlpaVx99dUUFhbi8Xj49a9/7b8FpxDi1IVEGM8akcicTAvPrNnHBaOTmZYVF+wiCdHjFi1axE9+8hN/GC9btowVK1bw4x//mKioKI4dO8bZZ5/N/Pnz2zydqSuPP/44Sim2bNnCjh07+MY3vsGuXbt46qmnuO2227jppptobm7G4/Hw7rvvkpaWxjvvvAMYD5QQQpy+kAhjgEUjbeyts/Gf/9jMe/8xk/CwkKma6IM6a8HW9PIjFCdNmsTRo0cpLi6mtLSU2NhYUlJS+OlPf8rq1asxmUwUFRVx5MgRUlJSun3cjz/+mDvuuAOAUaNGMXjwYHbt2sU555zDAw88QFlZGVdeeSXZ2dnk5OTws5/9jLvuuotLL72UmTNn9lZ1hRhQQuKcMYDdonhk4QQKKur53Xvbg10cIXrFwoULee2113j11VdZtGgRr7zyCqWlpWzYsIFNmzaRnJzc4TnFp+o73/kOS5cuxeFw8M1vfpNVq1YxYsQINm7cSE5ODvfeey+/+c1veuSzhBjoQqr5OC0rjpvOy+KZNfv5xpgUZo1IDHaRhOhRixYt4uabb+bYsWN89NFHLFu2jKSkJKxWK3l5eRw8ePCkjzlz5kxeeeUVzj//fHbt2sWhQ4cYOXIk+/btIysriwkTJnDo0CG++uorRo0aRVxcHNdeey0xMTE8++yzvVBLIQaekApjgJ99YyR5O0v5xWtfseKns4h2WINdJCF6zNixY6mpqSE9PZ3U1FS++93vctlll5GTk8PUqVMZNWrUSR/zhz/8Ibfddhs5OTlYLBZeeOEFwsLCWLZsGS+++CJhYWGkpKRwzz33sG7dOn7+859jMpmwWq08+eSTvVBLIQaekAtju9XMo1dPYMETn/Lf//qaR6+eGOwiCdGjtmzZ4p9PSEhg7dq1ne5XW1t73GMMGTKErVu3AmC323n++ec77HP33Xfzox/9qM158IsuuoiLLrroVIsuhDiOkDlnHGh8Rgw/mjOcNzYWseLrw8EujhBCCHFCIdcybnH7nOF8uP0Iv1q+hamDY4mPCAt2kYQ447Zs2cJ1113XZl1YWBiff/55kEokhOhMyIaxzWLij1dPYP5fP+HeN7fyxHcnn9S1l0KEgpycHDZt2hTsYgghuhCS3dQtRqVE8dMLR/De1sO8tbk42MURQgghOhXSYQxwy6yhTB4Uw6/f3MqR6p65/lIIIYToSSEfxmaT4o9XT6TZ4+Wu179Cax3sIgkhhBBthHwYA2QlhPPLi0eTv7OUV9cVBLs4QgghRBsDIowBrjt7MDOGxfPA29soKK8PdnGE6HUnep6xEKJvGTBhbDIpHl44AaUU//mPzXi90l0txJngdruDXQQh+ryQvbSpM+kxDu67bAy/eO0rXvj0AN87LyvYRRL91OHf/pam7W2fZ+z2eCg/jecZh40eRco99xx3e08+z7i2tpbLL7+80/e99NJLPPLIIyilGD16NEuXLuXIkSPceuut7Nu3D4Ann3yStLQ0Lr30Uv+dvB555BFqa2u5//77yc3NZeLEiXz88cdcc801jBgxggcffJDm5mbi4+N55ZVXSJUvmRoAACAASURBVE5Opra2ljvuuIP169ejlOK//uu/qKqq4quvvuJPf/oTAM888wzbtm3jscceO+WfrRB93YAKY4CFUzJYsfUwf3h/B7NHJjIsUbryRP/Qk88zttvtLF++vMP7tm3bxoMPPsinn35KQkKC/8ETP/7xj5k9ezbLly/H4/FQW1tLRUXFCT+jubmZ9evXA1BRUcFnn32GUopnn32Whx56iD/+8Y888MADREdH+2/xWVFRgdVq5X/+5394+OGHsVqtPP/88zz99NOn++MTok8bcGGslOJ3V+bwjT+t5mfLNvParedgMQ+Y3nrRQzprwfan5xlrrbnnnns6vG/VqlUsXLiQhIQEAOLi4gBYtWoVL730EgBms5no6Oguw3jRokX++cLCQhYtWkRJSQnNzc1kZRm9UitXrmTp0qX+/WJjYwE4//zzefvttxk9ejQul4ucnJyT/GkJ0b8MyBRKirLz4BXj2FRQydOr9wW7OEJ0W089z7gnnoNssVjwer3+5fbvDw8P98/fcccd3H777WzZsoWnn366y8+66aabeOGFF3j++edZvHjxSZVLiP5oQIYxwKXj07h0fCp/WrmLbcXVwS6OEN2yaNEili5dymuvvcbChQupqqo6pecZH+99559/Pv/4xz8oKysDoLy8HIC5c+f6H5fo8XioqqoiOTmZo0ePUlZWRlNTE2+//fYJPy89PR2AF1980b/+wgsv5PHHH/cvt7S2p0+fTkFBAX/729+45ppruvvjEaLfGrBhDPDA5eOIdti4c9kmmt3ert8gRJB19jzj9evXk5OTw0svvdTt5xkf731jx47lV7/6FbNnz2bChAnc4+uO//Of/0xeXh45OTlMmTKFbdu2YbVaue+++5g2bRoXXnjhCT/7/vvvZ+HChUyZMsXfBQ5w7733UlFRwbhx45gwYQJ5eXn+bVdffTXnnnuuv+taiJCmtQ7Ka8qUKbon5eXlndL7Pvj6sB5819v64fd39Gh5esqp1qsv66912rZt2wm3V1dXn6GSnDnBrNMll1yiV65cecJ9uvo36Ux//f3rSijWK9TqBKzXx8nEAd0yBrhgTDILp2TwRP4evjx04gEpQojeV1lZyYgRI3A4HMydOzfYxRHijBhwo6k7c99lY/h0bxk/W7aZd348E4ft1K8VFaIv6Y/PM46JiWHXrl3BLoYQZ5SEMRBpt/LQVeP57rOf8/CKndx32ZhgF0mIHiHPMxaif+hWN7VSap5SaqdSao9S6u4T7PctpZRWSk3tuSKeGecOT+CGcwbz3Cf7Wbu3LNjFEUIIMYB0GcZKKTPwOHAxMAa4RinVoemolIoE/gM44/1fWmuqPad/edJdF49iSLyTn7+2mdomuZ+uEEKIM6M7LeNpwB6t9T6tdTOwFOjs5rcPAH8ATu7OAT3g7X1v85ui3/Di1y/i8rpO+ThOm4U/Xj2B4soG/uedbT1YQiGEEOL4uhPG6UDgQ4ALfev8lFKTgUyt9Ts9WLZum5g4keH24Tyy/hGueusqPi859cb5lMFx3DJrGH//ooC8nUd7sJRCnD55LKIQoUkZlz6dYAelrgLmaa1v8i1fB0zXWt/uWzYBq4AbtdYHlFL5wH9qrdd3cqxbgFsAkpOTpwTek/Z01dbWst+0n9crXqfMXcYk5yQWxC4g1nLyNwxweTX3f9pAnQsePNdBhO3EN93vTbW1tSH3Bdxf6xQdHc3w4cOPu93j8WA+jac2dUdqaiolJSVt1rndbiyW3hmLeSbqdDr27NlDVVXVSb2nv/7+dSUU6xVqdZozZ84GrXXnY6qOdwFyyws4B1gRsPxL4JcBy9HAMeCA79UIFANTT3Tc3rrpR4OrQT+x6Qk95eUp+qz/d5Z+5qtndLO7+aSPt6WwUg+/5x095YEP9MtrD+hmt6dHy9tdoXbRu9b9t0594aYf4eHhWmvjZ3jeeefpyy67TGdnZ/fa5/X1G5nITT9ahWK9Qq1OnOCmH935c3odkK2UygKKgG8D3wkI8yrAf3+7E7WMzwS7xc5tE27jsqGX8dC6h/jzxj/zzz3/5JfTfsmM9BndPs649Gj+cesMfvvOdu59cyvPfbKfu+aN4htjkrt8PJ0IfWuW7eJYQW2bdafbikzIjGDm1SO6vf/GjRvZunWr/wlIQoj+q8tzxlprN3A7sALYDizTWn+tlPqNUmp+bxfwVGVEZvCX8//CE3OfwKu9/GDlD/hp3k8pri3u9jEmZsbw6g/O5pnrp2JSih+8vIGrnlrLhoPlvVhyIbpn2rRpEsRChIhunWjSWr8LvNtu3X3H2Tf39IvVc2ZmzGR66nRe/PpFlny1hI+LPuamnJu4cdyNhJnDuny/UooLxyQzZ2Qi/9hQyKMf7OJbT65l3tgUfjFvJEMTQ+d8hui+zlqwvf084/YCH1EohOjfBsS9qW1mGzePv5m3rniLmRkz+d9N/8uCfy5gdeHqbh/DYjZxzbRBfPTzXO68cARrdpdy4WOruffNLZTWNPVi6YUQQoS6ARHGLVIjUnk091GevvBpzMrMjz78EXd8eAcFNQVdv9nHabPw47nZfPSLOXx3+iCWflFA7sN5/HnlburkRiFCCCFOwYAK4xYz0mbwxvw3uHPKnXx++HOuePMKHt/0OI3u7t+vJCEijN9cPo5//3QWs0Yk8tjKXeQ+ks8rnx/E7ZFnI4veUVtrDBrLzc3l7bffDnJphBA9ZUCGMYDVbGXxuMX864p/MXfQXJ7a/BRX/PMKVh1a1XLJVrcMTYzgyWun8PptMxgS7+RXy7dy0Z9Ws+Lrwyd1HCGEEAPXgA3jFsnhyTw0+yH+7xv/h91s5z/y/oMffvhDDlYfPKnjTBkcy7IfnMOS66aggR+8vIGFT61lw0F5RrIQQogTk0co+kxLncY/5v+Dv2//O09sfoIF/1zAjWNv5Pox1xNjj+nWMZRSfGNsCuePSmLZ+kIeW7mLbz35qYy8FkKIPkhrTUVTBcW1xa2vutapy+PiXwv+dUbKImEcwGqycv3Y67k462Ie3fAoz2x5hue2PsekpEnkZuZyfub5ZEZldnkci9nEd6YP4vKJaTy7Zj9LVu/lg+1H+M60Qfx4bjaJkV1fUiX6Jq213PSlj5DTQKIrWmvKGssoqi2ipLaEotqiNoFbUldCg7uhzXsirZGkRaSRHpFORkTGGfs/L2HciURnIr+b+TtuGHsDHxz8gLyCPB5Z/wiPrH+EYdHDmDNoDrmZueQk5GBSx+/pDw+z8B8XZPOd6YP4y4e7+dsXh3hjYyE3zBjCwqmZZCXIdaL9id1up6ysjPj4eAnkINNaU1ZWht1uD3ZRRJDVu+rZU7mHQzWH/IFbUlfib+k2e5vb7B8dFk1aeBpZ0Vmcm34u6RHppIanGtOIVKJsUUGph4TxCYyKG8WouFHcMekOCmsKyS/IJ78gn+e3Ps+zW54l3h5PbmYuczLnMD11OnZL518MiZFhPHDFOG48dwiPrNjJUx/t5Yn8vUweFMOVkzO4bHwa0U7rGa6dOFkZGRkUFhZSWlra6fbGxsaQC4e+XCe73U5GRkawiyHOELfXzaHqQ+yq3MWu8l3srtzN7ordFNUWtdkvzh5HWngaI2JHkJuZS1pEGmnhacY0Io1wa99sBEkYd1NGZAbXjrmWa8dcS1VTFR8XfUxeQR7vH3if13e/jsPi4JzUc8jNzGV25mzi7HEdjjHMN/L6SHUjb35ZxOsbC7n3za385l/buGBMEldOymD2yESs5u6Nq6tz1VFYU0hxbTE1rhpmpM0gwZHQ9RvFKbFarSe8/WR+fj6TJk06gyXqfaFYJ9G3aa0pbShld8VuVlat5N9r/s3uyt3sq9znb+WalZnBUYMZlzCOBcMXkB2bzZCoIaSEp+C0OoNcg1MjYXwKosOiuWToJVwy9BJcHhfrDq8jryCP/MJ8VhWsQqGYmDSROZlGd3ZWdNsv8OQoOz+YPYxbZg3l6+JqXt9YyFubinl3y2Hiw23Mn5jGtyZnkJVooaS5hNWFqymqLaKopojiumJjvraIqqa2j46zKAuzMmaxIHsB56Wfh8Uk/7xCiJNX3ljO3sq9VDRWYLfYcVgcOCwO7GY7dovdv85utmM2nfrDUepcdeyp3MOuil3srjBaursrd7f5bktqSiI7NpuzR59Ndmw2I2JHkBWd1a3bGfcn8m19mqxmKzPSZzAjfQb36HvYUb7DCOaCfB7d8CiPbniUIVFDmDNoDnMy5zA+Ybz/l7fZ20x4RDkXTK5kxPByPi/YzebD+1lWVMKyoxWYLL6nAvkeX2sz2YyBBZHpjIsfR3pkOmkRaWREZGBWZt7d/y5v7X2LVQWrSHAkMH/YfK4YfkWHPwaEEAKgorGCPZV72Fu513hVGdPyxu4/DMdmsnUI6MBlh9nhX7Zb7CgU+6r2dehidlqcDI8dzgWDLvCH7tGvj3LJ3Et6o+p9joRxD1JKMTp+NKPjR/PDiT+kpLaE/MJ88g7l8fK2l3l+6/PE2ePIjMykuLaY0oa25x4tJgtpEWlMS0zD1TSGwlI7haUOcMUyNWM4V08azcXj0nDYOv9LdHT8aH48+cesKVzD8j3LefHrF/2jwRcMX8BFQy7qt104QohTV9VU5Q/dwGlg6IZbwxkWM4zczFyGRQ9jeMxw4h3xNHmaaHQ30uBuoMHTQKO70f9qWW5wN3RY1+hupKqpynhfwHaP9nToYh4RO4K0iLQOA2Lzd+Sf4Z9U8EgY96LUiFSuGXUN14y6hprmGj4p+oS8gjyONRzzj+JreaVFpJHoSOzQ5bP0nVUUW9N5fWMRdy77il+/+TXfzEnlyskZTM+Kw2RqO6rXarJy/qDzOX/Q+RxrOMZbe99i+e7l3Pfpffzui98xb8g8FmQvYGLiRBkRLMQZ5Pa6KagpoKi2CJMyYTPZsJqtxtRkbZ03W41lkxWb2YZZmbv9f7WqqYp9Vfs6BO+xhmP+fZwWJ8NjhjMrYxbDY4YzLMYI3mTnmXlWu1we2DkJ4zMk0hbJvKx5zMuad1LvSwk38e3ckfzkghF8caCcNzYW8u6Ww/xjQyHpMQ6unJzOgknpnd5QJMGRwPfGfY/FYxezuXQzy/cs5/3977N8z3KGRA1hQfYCLht6GYnOxJ6qpuhCQU0BhTWFTEqadNzR96J/01pzpP4Iuyp2sadyD3sq9vhDsf1lNt2hUNjMNn9At4R1yzqbyYbZZGZ/+X6ql1b73+ewOBgWPYxz085tE7op4SlBDUMJ4s5JGPcTJpPi7KHxnD00nv+eP45/bzvM6xuLeDxvD39dtYdJg2K4aGwKF4xOYlhiRJtfeKWMAWUTkyZy11l3seLACt7c8yaPbXiMv2z8CzPTZ3JF9hXMypiF1SSXWPW0o/VHWXFgBe/tf48tx7YAxhfljLQZzMmcw+yM2d2+y5voW1rOue6u2N1mWuuq9e+T5EwiOyabaaOmkR2bTWZkJhqNy+vC5XHR7G32z7u8Lpo9vuXA+RNsa/Y24/a4GWUfxbmjWoM3NTz1hPdBEH2LhHE/5LCZuXxiOpdPTOdIdSP/3FTEPzcV8/v3dvD793YwJN7J3NHJXDA6malDYttcKuW0OlmQvYAF2Qs4UHWAN/e8yVt73yK/MJ84exyXDb2MBdkLGBYzLIg17P8qGyv54NAHvLf/PdYfXo9GMzpuNHdOuZNhMcNYXbiavII8Pjz0ISZlYnLSZOZkzmHOoDlkRnZ9lzdxZtW76v3dvv4Wb+WeNt2/UbYosmOzuWToJWTHZDM8djjDY4YTHRZ9RsqYn59P7rjcM/JZoudJGPdzyVF2bpk1jFtmDaO4soEPdxzlw+1HeHntQf7v4/1E2S3MGZXE3NHJzB6RSLSjteU7JHoIP5nyE26fdDufFH3C8j3LeWX7K7y47UXGJ45nwXDj3HK8I57osGj5K7sLTd4m3t73Nu/tf49Piz7Frd0MiRrCrRNu5eKsi9uMap+VMYtfTf8V28q2sapgFXkFeTy8/mEeXv8w2bHZzMmcw/mZ5zMmfox06/UwrTUN7gZqm2upcdVQ21xLbXMt1a5q/3zL+uK64g6jfu1mO8NijO7f7Nhsf/AmOhLl30qcMgnjEJIW4+C6swdz3dmDqWtys2b3MVZuP0LejqP8c1MxFpNiWlYcc0cnc+HoZAbFGyOrLSYLszNnMztzNmUNZby9722W717Of6/9b/+xzcpMrD2WeHs8cfY44h3xxrwjruM6exxW88Do7m72NLOmaA3v7X+PVYWrcBW4SHYmc+2Ya7k462JGx40+7he0UoqxCWMZmzCWOybdQUFNAXmH8sgryOPZLc+y5KslJDmT/MF8VspZA+bn2hmX12WMzHU1UO+u94/SrXcZ87WuWmqaa/zTmmZf0Aasr22upaqpCu8rJ37muEmZCLeGk+RIIichhwXDFzA8djgjYoxRv6dzba0QnZEwDlHhYRbmjUth3rgUPF7NpoIKVm4/ysptR3jg7W088PY2spMiuGBMMheMTmJiZixmkyLeEc8NY2/g+jHXs718O4eqD1HWWEZZQxnljeX+6aGaQ5Q1lNHoaez086NsUcQ7fCEdGNaOeA7VHaJ2Xy1e7cXj9eDRHmNeezosd2cfj/agUCQ5k/wj09Mj0jsdnd4T3F43Xxz+gvf2v8eHBz+kxlVDbFgs08Onc9N5NzEpadIp9SJkRmZy/djruX7s9VQ2VrK6aDWrDq3irb1v8erOV4mwRnBe+nnMyZzDzIyZRNoie7xux6O1xqM9uL1u3F43Lq+rw7zL68Kt3bg8vm3t5lv2afY0+4M0MExbArb9css6l9fV7fJGWCOIsEUQYY0g0hZJojORLGsWkbZIykvKGTN8DFG2KP9+kbZIIq2R/nmnxSmtXHFGSRgPAGaTYsrgOKYMjuOueaM4WFbHyu1Gd/Yzq/fxZP5e4sNtzBmVxAWjk5mZnUB4mIUx8WMYEz/mhMeud9VT1lBmBHYnoV3WWMauil2UN5ZT3dw60pM1p1APZcakTK1Tk9k/3/IotEAWk4XU8FR/OLfcnzYjMoO08DQSnYndDk2v9rK5dDPv7nuXfx/8N+WN5YRbw5k7aC4XZ13M9NTpfLL6E6YkTzn5inUixh7D/GHzmT9sPo3uRj4v+Zy8gjz/LVgtysJZKWf5byaTEp7S4Rhur5s6V52/RRg4rXPVUdNc03bqMqaB+9Y116Ff1ri9bjQ9/5Qki7LgsBp3d3JanP47PcXaY0mzpLWus7Zud1qdHfZ3WBxE2owwDbeEn/CPsPz8fHJzcnu8LkKcDgnjAWhwfDjfPy+L75+XRVWDi492lbJy2xH+/fVhXttQiM1iYsaweN8gsCRSox3HPZbT6sRpdXbr0ZIuj4vyxnLyPs3j7GlntwnTwGln603K1GVLpcnT5H9Si/9RabXFFNUV8VHBR5Q1lrXZ32qytg1r37RlPsGRwK6KXby7/13e3/8+JXUl2Ew2ZmfO5ptZ32Rmxswzcks+u8XuP43wa++v2XJsi3Ge+VAev/38t/z2898yInYEYeYwf7DWumo7PBquM2Zl9rcgW1qJSc4khtqGEmGNoLSklKxBWVhMFqwmKxaTxf9qudQmcLn9dv+ysrS5NKclTAdyt7sQgSSMB7hoh5X5E9KYPyENl8fLugPlfLj9KCu3H+HXb27l12/CmNQoLhhtDALLSY/ucKOR7rKarSSHJ5NiTWFI9JCerQgQZg4jKzrruLf/bHA3tHm0WktgF9UWkVeQ1+EWgBZlwa3dmJWZc9LO4Y5JdzAncw4Rto7XdJ8pZpPZf5nanVPuZF/VPvIO5fF5yeeYTCbSItLaBGunU998uDUcu9l+wj9y8vPzyZ2Se+YqKMQAJWEs/KxmEzOGJTBjWAL3XjKavaW1/u7s/83bw19W7SExMozzRyYxd3QS52Un4LT1n18hh8XB0OihDI0e2un2BndDmweQF9UVkRGRwYWDLyTWHnuGS9s9Q6OHMjRnKN/P+X6wiyKEOA3955tUnFFKKYYnRTI8KZJbZw+jvK6Z/J1H+XDHUd7dUsKr6wuwWUyc6+vOnttFd3Z/4LA4GBozlKExnYe1EEL0Fglj0S1x4TaunJzBlZMzaHYb3dkrtx/hw+1Hydu5lXt7sDtbCCEGGgljcdJsFhPnDk/g3OEJ3HfpGPYcrfXfbCSwO3vuqCTOH9X/urOFEOJMk29IcVqUUmQnR5Kd3LE7+52vSli6roCwgNHZc0cnBbvIQgjR50gYix7Vne7sjAjFnMqtTB0Sy1lD4kiL6d/nmoUQ4nRJGIte0747u2V09r++2MUbGwt5+bODAKTHOJg6JJapQ+I4a0gsI5Ii5XyzEGJAkTAWZ0Tg6OxRuoDzZs5ix+Ea1h0oZ/3BCtbuLeOfm4oBiLRbmDLYaDVPHRzLhMwY7Fa5F7AQInRJGIugsJhNjEuPZlx6NIvPzUJrTWFFA+sOlLPuQAXrD5STv3MnAFazIic92gjnIXFMGRxLXLgtyDUQQoieI2Es+gSlFJlxTjLjnFw5OQOAirpmNhysYN3BctYfqOD5Tw7w9Op9AAxPimDq4Nau7UFxcmN/IUT/JWEs+qzYcJvxVKkxyQA0ujxsKaoyurYPVPDuFmO0NkBiZBhnDYll6uA4zhoSx+jUSCxmef6yEKJ/kDAW/YbdauasIUbYAni9mt1Ha31d2y0BfRgAp83M5EGx/hHbEzNjCA+TX3chRN8k306i3zKZFCNTIhmZEsm1Zw8GoLiygfUHjXPO6w5U8OcPd6O18RjJsWlRrQPDhsSSFGkPcg2EEMIgYSxCSlqMg/kxDuZPSAOgutHFl4cqfeFczt+/OMTznxwAYHC809etbZx7HpYYLuedhRBBIWEsQlqU3crsEYnMHpEIQLPby9fFVaw/UMG6A+Xk7TzK6xsLAeOGJUbL2QjncWnR2Cxy3lkI0fskjMWAYrOYmDQolkmDYrl51lC01uw7Vufv1l5/oJwPth0BIMxiYmxaFKNToxiTFsWY1ChGpUThsMk1z0KIniVhLAY0pRTDEiMYlhjBorMGAVBa08SGg0Y4bymq4q1Nxbzy+SEATAqGJIQzJiCgx6RGkRgZFsxqCCH6OQljIdpJjAxj3rhU5o1LBfDfkGRbSTXbiqvZXlLNpoJK3v6qxP+ehAgbKWEe1jZs9wd0VkK4XF4lhOgWCWMhuhB4Q5KLxqb411c1uNheYoTztuJqvthdzPMfH6DZ4wWMbu6RKZFtWtGjUqOIkEushBDtyLeCEKco2mHl7KHxnD00HoD8/ArOnTmLvaW1bCs2AnpbSTXvf33Yf3MSgCHxTiZmxhivQbGMSY2SgWJCDHASxkL0IKvZxKgUY6DXlZONdVprDlc3+gN6S1EVn+4t403fgzFsZhNj06P8AT0pM5bMOIdcZiXEACJhLEQvU0qRGu0gNdrB3NHGrT211pRUNbKpoNJ4Hapscw10fLiNCZkxTMqMYeKgGMZnxBDtsAaxFkKI3iRhLEQQKKVIi3GQFuPgmznGQDGXx8vOwzWtAV1QyaodR/3vGZYYzsTMWCYNMlrQo1Lk/ttChIpuhbFSah7wZ8AMPKu1/n277XcCNwFuoBT4ntb6YA+XVYiQZg14rGTL7T2rG118VVDFpoIKNhVUkh9wkxK71UROerSvezuWEckRDIp3EmaR66CF6G+6DGOllBl4HLgQKATWKaXe0lpvC9jtS2Cq1rpeKXUb8BCwqDcKLMRAEmW3cl52AudlJwCtl1l96eva3lRQwYtrD/LMmv2AcR10ZpyToQnhDE2MICshnKGJ4QxLjCApMkzOQwvRR3WnZTwN2KO13geglFoKXA74w1hrnRew/2fAtT1ZSCGEIfAyq5b7bze7vew6UsPe0lr2ltaxr7SWfaV1fLavnAaXx//eiDCLP5yHJkQwNDHcv+y0yRkrIYJJaa1PvINSVwHztNY3+ZavA6ZrrW8/zv7/CxzWWj/YybZbgFsAkpOTpyxduvQ0i9+qtraWiIiIHjteXxGK9QrFOkHfq5dXayoaNYfrNCV1Xo7Ueymp0xyu81LWoAn8nx9nV6SEK1LCTaQ6Tf75ME89UZF9p049oa/9O/WUUKxXqNVpzpw5G7TWUzvb1qN/DiulrgWmArM72661XgIsAZg6darOzc3tsc/Oz8+nJ4/XV4RivUKxTtC/6tXo8nCgrI59AS3pvcfqWHe0lprGZv9+ZqVIifaSGm0nLcZBaoydtGhH63K0nbhwW7/q/u5P/04nIxTrFYp1Op7uhHERkBmwnOFb14ZS6gLgV8BsrXVTzxRPCNEb7Faz/3roQFpryuqa/SH98aYd2KLjKK5qYHNhJe9vbfTfYaxFmMXkD+bUaAdpMcY0NcZOum99pF0uyxLiRLoTxuuAbKVUFkYIfxv4TuAOSqlJwNMY3dlHOx5CCNEfKKVIiAgjISKMaVlxpNTvIzd3on+712uEdUlVA8WVjZRUNVBS1UhRZQMllQ18uvcYR6ob8bY7+xUZZiHVF9IZsQ5GJEcyMiWSUSmRxDhtZ7iWQvQ9XYax1tqtlLodWIFxadNzWuuvlVK/AdZrrd8CHgYigH/4uqsOaa3n92K5hRBBYDIpEiPDSIwMY3xG5/u4PV6O1jRRUtVAUWUjJZVGYBf7pl8eqqC60e3fPykyjJEpkYz0B3QUw5Mi5FGVYkDp1jljrfW7wLvt1t0XMH9BD5dLCNFPWcwm/w1NpgzuuF1rzZHqJnYeqWHn4Wp2Hq5l55FqXv7sIE1uowtcKRgSH86I5AhGpkQxKiWSEcmRDIl3yo1OREiS6xmEEGeUUoqUaDsp0XZmj0j0r/d4NQfL6th5uIYdh2vYdaSGnYdr+GDbEX+3t81iIjspwt+KbnmlRNn71SAyIdqTMBZC9Almk2JoYgRDEyO42HeLUDBGfu85WusP6B2Ha/h0bxlvfNk6jjTKbiEzzmm0yKPtG7XmswAADrxJREFUpPpa5i3zyZFh0qIWfZqEsRCiT7Nbzf7bhAaqrG9mZ0sL+kgNRRUNHCqr57N9ZdQEnJMG485kyVF2UqPtWJob+bR+u//yrDTfyO/4fnaJlggtEsZCiH4pxmlj+tB4pvueJx2optHVZtBYcWXr6O+9x7xs+vQAze6Ol2i1Xp5lXKKVEetgUFw4QxKcJEfaMZkkrEXvkDAWQoScSLuVSLuVEcmRHbbl5+cze/ZsyuuaKa5spLjKuCyrOCC8O7tEy2YxMSjOyZB4pz+gjeVw0mMdWKUbXJwGCWMhxICjlCI+Ioz4iDByMqI73cft8VJS1cjBsnoOlNVxqLyeA8eM6cd7jtHoam1Zm02K9BgHg+Odxisu3DcfzqA4p1ymJbokYSyEEJ2wmE3+h3K0PDWrhdaa0pomDpTVc7CsjoNl9Rwsr+dQWR3/2lxCVYOrzf7JUWH+gM6McxIfYSM+3EZceBhx4VbiwsOIcVilG3wAkzAWQoiTpJQiKcpOUpSdaVlxHbZX1jf7A/rgsTpjWlbHR7tKOVrT+d2CTco4Dx4X7ns5bcT5QjvWaSM+wljfMh/rtGG3Sos7VEgYCyFED4tx2ohx2pjw/9u7+xg57ruO4+/v3t7tPWKf7aRJWqcQN0KCPwATlQQVY6kQ0gg1BbU0VSUCLUQVRKJ/IBSpUhT1LwKiiIcKVNqItqpK1EKpRVOlAWogVZMmTZ3YebId1/TO+OnOF5/3HvZub7/8Mb/ZnZ2dvdvzw83u5vOSVjvze5rfb3+7+52dmZvbvb0lb3l1jTcWV5ldqHBhYSXzMbuwwuvnyzx7coW5xZWW24vGxoYG2DEeBe7i6jLfWXg5+jU/OcruHSO8dbsOkfcKBWMRkS00PDjADdsGuGHbcEflazXn4tIqswtRYJ4tRwG7sVxhdmGF43M1jny3cRez2K7xErt3jNQD9O7JUd4Wlm/argvPuoWCsYhIFysUjMmxISbH1v+HGgcPHmTfvl9mplxham6RqQtLTIfnqblFfjA1xzcOn2Yt8TO7YNT/ecfuHaPR8+RoOFc+wvUTwwzoPPaWUDAWEekThULjXHbWfcHjK8Sn5haZnlti+sIiU3NLTF1Y5H+OnefsfOv57LGhASaGBxkfLjIxXGS8VOTHhgcZLxVb0xLrE8PFqF6pyOjQgG6osgEFYxGRN4nkFeJZllfXOPXGEtMhQJ+7VKG8XKVcWeXScpVypcql5Sr/98ZSfXlxZW3D7RaMEKAHmRgusnN8qP6vOqNHYn1iiJ1jJYaKb67D5wrGIiICROez91w3zp7rxjuus1bzEJhX6wG6vFxlPqyXl6v1QD6/vMr8UpXZhQrP/2iO2fJK22C+bWSQ0UKV3a99l+uSAXuixM6xIXZNlEJ6qS8uUlMwFhGRyzZQMLaNDLJtZPCy6i+uVJm5tML5coWZcoXZ8gozYfnlE9Pg8MqZeWYuVZr+D3bS2NAA20YGGQuHzsdL0WOs1FgeH47WJ0L6WGmAidIgY6WBep2RwfwOpysYi4hIbkaHity8s8jNO1sPnR88OMP+/XfU1yvVtXqwni03AvjMpRXml1dZqES/wMuVKmcuLrNQqXKpUmWhUm3752FJBaMpYO8aL/Hl+2+/msNtS8FYRER6Qqk4EP6Jx8im6rk7S6tr9cPmC5U1LlVWWaisUa6sUq6shfRGMC8vV7f0SnIFYxER6WtmxuhQkdGhIte3/u+QrvDmulxNRESkCykYi4iI5EzBWEREJGcKxiIiIjlTMBYREcmZgrGIiEjOFIxFRERypmAsIiKSMwVjERGRnCkYi4iI5EzBWEREJGcKxiIiIjlTMBYREcmZgrGIiEjOFIxFRERypmAsIiKSMwVjERGRnCkYi4iI5EzBWEREJGcKxiIiIjlTMBYREcmZgrGIiEjOFIxFRERypmAsIiKSMwVjERGRnCkYi4iI5EzBWEREJGfFTgqZ2V3AXwEDwGfd/U9T+SXgC8DPA7PAB9395NXtantTr15g6qkaTxw/QqFgWMGynweMQgEsI68wYJiF53oeTeXMmtOb2wAza0m3AtGypdYT7WE0lbeCUbAoXURE+t+GwdjMBoBPA78KTAPPmtkBd385UeyjwJy7v8PM7gUeAT54LTqcZWWxSmUeZlbL1GqOrznuTi3xXKs5XoufwWu+Vd27MgavfOXb9SBvlgzu1IN6HOSJdxisOc+sTVqizaguGW00p5Fu32jdTqJtS+5wGJz/ofP95ZOhbKMelu5nKj3VflP5eD3uI81txGNoux73AZr7Q5QHjXIQ16cxBoPVJWdxfiXVv+btxGlGm3wReVPq5JfxO4Hj7n4CwMz+CbgHSAbje4CHw/JXgb81M3P3LYl4e/Zez9R8gf37b++4jrvjDt4SqBPPyaBei+rEeU3ra07NE+lxG4kdgqb0Wth2WMdJpDfyazXn5A9PcvPumxN5JLbVaCdZp769RFtNy8m21mqpdEKfnFoN8Oa0zHYTdZvXHWqNtKRzh09c3TdBlzj69aeuuI1kgK7vECSCf4jj0U4OGYEfWsqkdyzico02QnJy3YyFco2z3/leY0fFEvXjbSb6V+8PNO+40Nq/pp2aROOWbKeet0H55A5O/YVsfT0w4/TpGgdPvdr0OjS12bTNRrvNbSfqJTeYrlsfTKpuYr/LmjeWykuuhL60KTd73HlhbaqlT63boqly5vba9K9RP1Usq73065eQ3vFM14nrzU87Jw6db1PWUn2gRXo8WWUtnZlYLQwYN9062drwNdBJMH4rMJVYnwZ+oV0Zd6+a2UVgJzBzNTp5LdR/HRWMgbw7s46lgz/ijv178u7GFXOPdjrcnYMH/4t9+/bVgzbJIJ4I+LQJ9qTKtaQ3rcfLoXzcl7gurX2Il2vx0ZOmvLj90A6Nvh09epRb33Frfbxea4w9q4941HhTe/Emax6ym/vkEPWdxvjqfWoqH9dPbDc9lkR5T/QlWWalVmZ8eylRN16g8dqRSg9t19a80R8a24z716iXeC1Tr8u65RNjbq7beH2S9eMurqzAifMzLfVa6tBcr74/mXo9m7aXKpcssxXO/ODY1m1si0w9dTi3bZdGi/zep/ZtybY6Omd8tZjZ/cD9YbVsZq9dxeZ30cXB/wr047j6cUzQn+PSmHpHP44r9zH9/l9e1ebe3i6jk2B8CtidWH9bSMsqM21mRWAb0YVcTdz9M8BnOtjmppnZc+5+27VoO0/9OK5+HBP057g0pt7Rj+PqxzG108mfNj0L3GpmP2FmQ8C9wIFUmQPAfWH5/cB/btX5YhERkV634S/jcA74AeAJoj9tetTdXzKzTwLPufsB4HPAF83sOHCBKGCLiIhIBzo6Z+zujwOPp9IeSiwvAx+4ul3btGty+LsL9OO4+nFM0J/j0ph6Rz+Oqx/HlMl0NFlERCRfuh2miIhIznouGJvZXWb2mpkdN7MHM/JLZvZYyH/GzH5863u5OWa228y+bWYvm9lLZvZHGWX2m9lFMzsUHg9ltdVNzOykmR0O/X0uI9/M7K/DXL1oZnvz6GenzOwnE6//ITObN7OPp8r0xDyZ2aNmds7MjiTSdpjZk2Z2LDxn3u3AzO4LZY6Z2X1ZZfLQZkx/bmavhvfX18xse5u6675X89RmXA+b2anE++zuNnXX/b7MS5sxPZYYz0kzO9SmbtfO1RWJ/ri/Nx5EF5C9DtwCDAEvAD+VKvMHwN+H5XuBx/LudwfjuhHYG5YngKMZ49oP/Fvefd3kuE4Cu9bJvxv4JtHNbm4Hnsm7z5sY2wBwBnh7L84TsA/YCxxJpP0Z8GBYfhB4JKPeDuBEeJ4My5N5j2edMd0JFMPyI1ljCnnrvle7cFwPA3+8Qb0Nvy+7aUyp/L8AHuq1ubqSR6/9Mq7fmtPdV4D41pxJ9wCfD8tfBd5tXX7TX3c/7e7Ph+VLwCtEdzXrd/cAX/DI08B2M7sx70516N3A6+7+v3l35HK4+38T/eVDUvKz83ngfRlVfw140t0vuPsc8CRw1zXr6CZkjcndv+Xu1bD6NNF9EnpKm7nqRCffl7lYb0zh+/q3gC9vaady1mvBOOvWnOmg1XRrTiC+NWdPCIfVfw54JiP7DjN7wcy+aWY/vaUduzwOfMvMvh/uvpbWyXx2q3tp/2XRa/MUe4u7nw7LZ4C3ZJTp5Tn7CNGRmCwbvVe70QPh8PujbU4p9Opc/RJw1t3b3duzF+dqQ70WjPuamY0D/wx83N3nU9nPEx0S/Rngb4B/3er+XYZ3ufte4D3AH5rZ1tzk9RoLN795L/CVjOxenKcWHh0P7Js/tTCzTwBV4EttivTae/XvgD3AzwKniQ7r9osPsf6v4l6bq470WjDezK05sXVuzdltzGyQKBB/yd3/JZ3v7vPuXg7LjwODZrZri7u5Ke5+KjyfA75GdNgsqZP57EbvAZ5397PpjF6cp4Sz8WmC8Hwuo0zPzZmZ/Q7w68CHw05Giw7eq13F3c+6+5q714B/ILu/vThXReA3gcfalem1uepUrwXjvrw1ZzhH8jngFXf/VJsyN8Tnvs3snURz17U7GWY2ZmYT8TLRhTRHUsUOAL8drqq+HbiYOEzazdruuffaPKUkPzv3AV/PKPMEcKeZTYZDo3eGtK5kZncBfwK8190X25Tp5L3aVVLXVvwG2f3t5Puy2/wK8Kq7T2dl9uJcdSzvK8g2+yC6Avco0VWCnwhpnyT6sAEMEx0+PA58D7gl7z53MKZ3ER0SfBE4FB53Ax8DPhbKPAC8RHRF5NPAL+bd7w3GdEvo6wuh3/FcJcdkwKfDXB4Gbsu73x2Ma4wouG5LpPXcPBHtTJwGVonOJX6U6NqK/wCOAf8O7AhlbwM+m6j7kfD5Og78bt5j2WBMx4nOm8afq/gvLW4CHl/vvdotjzbj+mL4zLxIFGBvTI8rrLd8X3bDI2tMIf0f489SomzPzNWVPHQHLhERkZz12mFqERGRvqNgLCIikjMFYxERkZwpGIuIiORMwVhERCRnCsYiIiI5UzAWERHJmYKxiIhIzv4fDCR+C3mK2gwAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "ExCUj6fCyY0e", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "83683300-2e1e-49ce-f8f1-6db3214376b4" }, "source": [ "y_pred = model.predict(X_test)\n", "y_pred = y_pred.argmax(axis=1)\n", "print(classification_report(y_test, y_pred,\n", " target_names=class_names))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " T-shirt/top 0.84 0.84 0.84 1000\n", " Trouser 0.99 0.98 0.99 1000\n", " Pullover 0.83 0.80 0.82 1000\n", " Dress 0.91 0.89 0.90 1000\n", " Coat 0.82 0.83 0.83 1000\n", " Sandal 0.97 0.97 0.97 1000\n", " Shirt 0.70 0.73 0.71 1000\n", " Sneaker 0.93 0.97 0.95 1000\n", " Bag 0.98 0.96 0.97 1000\n", " Ankle boot 0.97 0.94 0.95 1000\n", "\n", " accuracy 0.89 10000\n", " macro avg 0.89 0.89 0.89 10000\n", "weighted avg 0.89 0.89 0.89 10000\n", "\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "FQPnaQLIyiG6" }, "source": [ "## 5.2 Training with Piecewise constant scheduling\n" ] }, { "cell_type": "code", "metadata": { "id": "h3BZROoZyc_X" }, "source": [ "def piecewise_constant_fn(epoch):\n", " if epoch < 5:\n", " return 0.01\n", " elif epoch < 15:\n", " return 0.005\n", " else:\n", " return 0.001\n", "\n", "def piecewise_constant(boundaries, values):\n", " boundaries = np.array([0] + boundaries)\n", " values = np.array(values)\n", " def piecewise_constant_fn(epoch):\n", " return values[np.argmax(boundaries > epoch) - 1]\n", " return piecewise_constant_fn" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "Na_BlZbFyrvz" }, "source": [ "piecewise_constant_fn = piecewise_constant([5, 15], [0.01, 0.005, 0.001])" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "UFhfcjJ2yxcV", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "cb2bb1f7-0b1d-4659-b06d-f0adcccbfa4d" }, "source": [ "model = tf.keras.models.Sequential([\n", " tf.keras.layers.Flatten(input_shape=[28, 28]),\n", " tf.keras.layers.BatchNormalization(),\n", " tf.keras.layers.Dense(300, use_bias=False),\n", " tf.keras.layers.BatchNormalization(),\n", " tf.keras.layers.Activation(\"relu\"),\n", " tf.keras.layers.Dense(100, use_bias=False),\n", " tf.keras.layers.Activation(\"relu\"),\n", " tf.keras.layers.BatchNormalization(),\n", " tf.keras.layers.Dense(10, activation=\"softmax\")\n", "])\n", "\n", "model.compile(loss=\"sparse_categorical_crossentropy\", \n", " optimizer=tf.keras.optimizers.Adam(lr=1e-3, \n", " beta_1=0.9, \n", " beta_2=0.999),\n", " metrics=[\"accuracy\"])\n", "\n", "lr_scheduler = tf.keras.callbacks.LearningRateScheduler(piecewise_constant_fn,\n", " verbose=1)\n", "history = model.fit(X_train, y_train, epochs=20,\n", " validation_split=0.1,\n", " callbacks=[lr_scheduler])" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "\n", "Epoch 00001: LearningRateScheduler reducing learning rate to 0.01.\n", "Epoch 1/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.5128 - accuracy: 0.8155 - val_loss: 0.4115 - val_accuracy: 0.8550\n", "\n", "Epoch 00002: LearningRateScheduler reducing learning rate to 0.01.\n", "Epoch 2/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.4027 - accuracy: 0.8530 - val_loss: 0.3609 - val_accuracy: 0.8683\n", "\n", "Epoch 00003: LearningRateScheduler reducing learning rate to 0.01.\n", "Epoch 3/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3653 - accuracy: 0.8674 - val_loss: 0.3619 - val_accuracy: 0.8725\n", "\n", "Epoch 00004: LearningRateScheduler reducing learning rate to 0.01.\n", "Epoch 4/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.3414 - accuracy: 0.8743 - val_loss: 0.3577 - val_accuracy: 0.8723\n", "\n", "Epoch 00005: LearningRateScheduler reducing learning rate to 0.01.\n", "Epoch 5/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.3231 - accuracy: 0.8791 - val_loss: 0.3229 - val_accuracy: 0.8832\n", "\n", "Epoch 00006: LearningRateScheduler reducing learning rate to 0.005.\n", "Epoch 6/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2722 - accuracy: 0.9002 - val_loss: 0.3064 - val_accuracy: 0.8862\n", "\n", "Epoch 00007: LearningRateScheduler reducing learning rate to 0.005.\n", "Epoch 7/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2566 - accuracy: 0.9047 - val_loss: 0.3072 - val_accuracy: 0.8897\n", "\n", "Epoch 00008: LearningRateScheduler reducing learning rate to 0.005.\n", "Epoch 8/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.2429 - accuracy: 0.9097 - val_loss: 0.2995 - val_accuracy: 0.8950\n", "\n", "Epoch 00009: LearningRateScheduler reducing learning rate to 0.005.\n", "Epoch 9/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.2350 - accuracy: 0.9121 - val_loss: 0.3176 - val_accuracy: 0.8825\n", "\n", "Epoch 00010: LearningRateScheduler reducing learning rate to 0.005.\n", "Epoch 10/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2240 - accuracy: 0.9157 - val_loss: 0.3094 - val_accuracy: 0.8888\n", "\n", "Epoch 00011: LearningRateScheduler reducing learning rate to 0.005.\n", "Epoch 11/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2182 - accuracy: 0.9177 - val_loss: 0.2981 - val_accuracy: 0.8930\n", "\n", "Epoch 00012: LearningRateScheduler reducing learning rate to 0.005.\n", "Epoch 12/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2118 - accuracy: 0.9211 - val_loss: 0.3182 - val_accuracy: 0.8867\n", "\n", "Epoch 00013: LearningRateScheduler reducing learning rate to 0.005.\n", "Epoch 13/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.2031 - accuracy: 0.9238 - val_loss: 0.3260 - val_accuracy: 0.8933\n", "\n", "Epoch 00014: LearningRateScheduler reducing learning rate to 0.005.\n", "Epoch 14/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.1955 - accuracy: 0.9266 - val_loss: 0.3264 - val_accuracy: 0.8895\n", "\n", "Epoch 00015: LearningRateScheduler reducing learning rate to 0.005.\n", "Epoch 15/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.1909 - accuracy: 0.9281 - val_loss: 0.3344 - val_accuracy: 0.8952\n", "\n", "Epoch 00016: LearningRateScheduler reducing learning rate to 0.001.\n", "Epoch 16/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.1516 - accuracy: 0.9436 - val_loss: 0.3132 - val_accuracy: 0.8982\n", "\n", "Epoch 00017: LearningRateScheduler reducing learning rate to 0.001.\n", "Epoch 17/20\n", "1688/1688 [==============================] - 7s 4ms/step - loss: 0.1388 - accuracy: 0.9475 - val_loss: 0.3282 - val_accuracy: 0.8947\n", "\n", "Epoch 00018: LearningRateScheduler reducing learning rate to 0.001.\n", "Epoch 18/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.1366 - accuracy: 0.9484 - val_loss: 0.3292 - val_accuracy: 0.8948\n", "\n", "Epoch 00019: LearningRateScheduler reducing learning rate to 0.001.\n", "Epoch 19/20\n", "1688/1688 [==============================] - 6s 4ms/step - loss: 0.1302 - accuracy: 0.9515 - val_loss: 0.3388 - val_accuracy: 0.8960\n", "\n", "Epoch 00020: LearningRateScheduler reducing learning rate to 0.001.\n", "Epoch 20/20\n", "1688/1688 [==============================] - 6s 3ms/step - loss: 0.1257 - accuracy: 0.9526 - val_loss: 0.3395 - val_accuracy: 0.8998\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "zq30k3GBy-ey", "colab": { "base_uri": "https://localhost:8080/", "height": 296 }, "outputId": "01e97dc9-944b-4897-b1fc-2073c5ec2353" }, "source": [ "plt.figure().gca().xaxis.set_major_locator(MaxNLocator(integer=True))\n", "plt.plot(history.epoch, [piecewise_constant_fn(epoch) for epoch in history.epoch], \"o-\")\n", "n_epochs=20\n", "plt.axis([0, n_epochs - 1, 0, 0.011])\n", "plt.xlabel(\"Epoch\")\n", "plt.ylabel(\"Learning Rate\")\n", "plt.title(\"Piecewise Constant Scheduling\", fontsize=14)\n", "plt.grid(True);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEXCAYAAAC6baP3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de5xcdX3/8dd7N7fNZbMBAhs2hESS7JKAmIAgom0QNfFS4WfxZ6xaUSy/WrSISktaReUnP6+ttRWtUawUUaCUYiopUIHV2kq4hXsSWcMlWYIETEI2bC6bfH5/nLNhMszsziRzdiY77+fjMY+d8z3nfOczl53PnO/3e85XEYGZmVmpGqodgJmZHVycOMzMrCxOHGZmVhYnDjMzK4sTh5mZlcWJw8zMyuLEUWcknSOpp9pxDERSSDq72nFYaST9QNJPM6j3sPSzsKCMfaan+5xUaNkqw4ljmEn/iSO97ZK0VtLXJI1LN7kWeEU1YyzBFODfs3wASRMk/V9Jj0rqlfRbSZ2S3iNpSP4vsvxSK6duSb8v6TZJz0l6UdJvJF0tqbnScVXBOpLP0/3VDmQ4GVHtACwTPwPeD4wEXg98DxgHfCQieoHeKsY2qIh4Jsv6JbUAvwQmAZ8G7gJ2Aq8DPgP8CngiyxhqhaQ5wM3APwIfB7YBxwD/CxhdxdAqIiJ2A5l+nupSRPg2jG7AD4Cf5pV9F9iQ3j8H6Mlb/wfAvcB24HHgMmBUzvpRwP8DngR2AGuBP89ZPwe4CdgKPAv8GGhN13UAkbM8Nq3j5pz9Pwx05SwHcHbO8iU5j/0M8M856wT8BfAbkoT4EPC+QV6jb5F8QU4tsG4MMCa9Pwm4EtiU1v0zYG7OtucAPcAZwMNpnXcAM3K2OQr4CfA74EVgNbA453nm3jrT8lcDtwLPAS+QJLlT8+IM4DzgX9LHXZv7vIvVXeD5fhxYX8LnqgNYBmxJn/OvgONzP3PABUB3+nr9EzC2nPcpfd79n8OVwNvS2Bek6xeky4fl7DM9LTupxOX+Os4AVqTvyT3A/LxYPgQ8la7/d+DPgKj2/3et3NxUVR96SY4+XkbSQuBq4JvAXJJ/mLNJEkW/K4E/Bj4BHAucC2xO958C/ILki/Nk4I3AeOAnkhoiYjXJl/2CtK7XknwZniap/4h3AdBZJL4/BD5F8o87C3g7yRFCvy+k8ZxPksC+CHxH0tuK1NcALAaujoj1+esjYntEbE8XfwCcApyZPrcXgZslNeXsMhpYQvK6nQq0kPx67/ctkmR5Osnr+3HS1y6tE2ARSXPKO9PlCcBVJEeLJ5M0syyXdGheuJeQJKUTSJogvy9p2iB153sGmCzp9CLrkXQkSfIK4E3AfOByoDFns9cDx5G8/+8mOWK5IGf9gO+TpPEkPz7WAicBFwNfKxZTBXwxfYz5wPPA1ZKUxnIqyVH65cCrSBLm5zOM5eBT7czlW2Vv5B1xkHyBPAdcmy6fQ84RB8mX/mfy6jiL5FelSL6sA1hU5PEuBW7LK5uU7nNyunwN8J30/heAb5M0BZ2alq3j5b+Wz07vfwJYA4ws8NjjSJLi6/PK/w5YXiTew9P6Lxzkdex/3r+XUzaR5Bf3h3NeywDac7Z5L8mRkdLlB4HPFnmM6eT8Gh4gFgEbCrxGX8xZHkGS2N5XZt2NJEcHAfyW5Nf1J4DJOdtcRnLEN6pIHT9I38PGnLLvAj8r9X0iOXraDIzPWf8+sjviWJhTx2lp2dR0+cfkHBGnZUvxEcfem484hqdFknokbSdpUvgF8LEi254I/HW6fU864upHJP/srcA8YA9JE0yx/X8vb/916bpj0r+dvHTEsSCtqxNYIGkmMJUiRxwkTTFjgMclXSHpXZL6297npOtuznv8j+Q8dj4VKc93LMnz/lV/QURsIWlimZOz3Y6IWJOz/DRJ096kdPkbwKcl/UrSFySdONgDSzpc0nck/VrSFpImwMOBaXmbPpgTWx+wMd2uZBGxOyI+SPIefIqkeeYiYLWkuelm84BfRsTOAap6NJL+hH5P58RSyvt0LPBgROSO+PsV2Xkw5/7T6d/+eDvY96gWkmYtS7lzfHj6BckvuF3A0xGxa4BtG0gOw/+lwLqNJTxWA0kTw6cKrPtt+rcT+HaaJE5Kl8cCf5Q+xm+iQLMRQESsk9RO0ib9RuBvgM9KOoWXRgX+AckXXq5iz3kjyS/bYwd7YgPIvaR0X5F1DQARcYWkW4C3ksT/P5K+GBGfG6D+K4EjgAtJjsx2ALeRJKRc+c8x2M+RkhHRTdI8dpWkTwO/Jkkg55RYxUCx7M/7VMie9G9u8i/YBFuC3Mfd5z2zwfmFGp5ejIiuiHhykKQBcB/QkW6ff+sjaV9vIGmjL7b/XODJAvtvBYiX+jn+miRJPEuSPE4jaTPvHCjASPodboqIC0k6UOem+z5K8qV6dIHHfrJIXXtIms7eK2lq/npJYySNAValz/vUnHXNwPHp45YsItZHxNKI+N8k/RLnpav6f8E35u3yOuAf0uf8CMkRx5RyHnOAukuJdxNJ09j4tGgl8DpJ+YmrVKW8T6uA43OGjQO8Jq+e/h8yua/Fq/YzpoGsJvmc5Tq50Ib1yonDLgX+SNKlko6T1CHpbElfAYiIXwPXAd+T9IeSZkh6vaT3p/tfTtL2f62kUyS9QtIbJS2VNCHncX5O0mZ9R1rvEyRfBO9kgMSRnrD4YUnHS5oBfJDk1+JjaWL6GvA1SR+SNFPSqyT9qaTzitVJksCeAlZI+qCkuem+7ycZ1dMaEY+RdDx/J32+xwM/JOnY/1GJry2SviFpUfq6vIqks7o/8TxL0va/UNIRkiam5b8G3idpjqRXkyS6gZqJCilWd358/0fStyW9WdIx6WvxZZIE+W/pZt8iSSLXSXp1+lq9J30+gyrxffoRydHb99MY3kTyPuXqImkG/Zyk2ZLeTDKcutL+HnizpIskzZJ0Lklnv/WrdieLb5W9UWA4bt76c3j5cNw3A/9F0rn6AsnwxI/mrB8NfIVkqOUOkiGVuetnAdfz0rDVNcA/sO+Q3j/l5cNsf0BOp2ROeW7n+Fkkbd2bSYad3g28PWdbkfTf9P+q3Qj8J/CmQV6niSSdvqtJhn/2HwUtBhrSbUoajptX7wJyOnDT1+Gx9DE2kiSBtpztP0ySxHbz0nDcE0ja1HvT1/r9JKPWPlfoNcopewL41EB1F3gd5qXPsX+Y7PPAncD787abCywnGTSxFfgf4Lhinzngc8DD5bxPJCPY7kvXP0DStLW3czzd5rUkR8G96eeif8huuZ3jRTvY07IPkSSpXpIBA58Eeqv9/10rt/6RH2ZmVoSkrwNvjIjjqx1LLXDnuJlZHkkXkRwR9ZAMavhT4K+qGlQN8RGHmVkeSdeSNGtNJLmawneAb4S/MAEnDjMzK5NHVZmZWVnqoo+jpaUlZs6cWe0wANi2bRvjxo0bfMMh4FiKq6V4HEthtRQL1FY8lYjl3nvvfS4iJhdcWe1hXUNxmz17dtSKO+64o9oh7OVYiquleBxLYbUUS0RtxVOJWIB7osh3qpuqzMysLE4cZmZWFicOMzMrixOHmZmVxYnDzMzK4sRhZmZlceIwM7OyOHGYmVlZnDjMzKwsThxmZlYWJw4zMyuLE4eZmZXFicPMzMrixGFmZmVx4jAzs7JkmjgkLZK0RlKXpIsLrB8t6dp0/QpJ09PyQyXdIalH0jfz9jlR0kPpPn8vSYPF8cQLezjtS7dz48rusp/DjSu7Oe1LtzPj4pv2u47ces65eduwiMXM6ldmiUNSI3A58BZgDvAeSXPyNjsX2BQRM4GvA19Oy7cDnwE+VaDqbwN/AsxKb4tKiad7cy9LbniorC/JG1d2s+SGh+je3EvsZx359QyHWMysvmU5dezJQFdErAWQdA1wJvBozjZnAp9L718PfFOSImIb8EtJ+8z3KmkK0BwRd6bL/wycBfxHKQH17trNX/3bQ/yy67mSnsDyhzbQu2v3AdVRqXqyjuWrt6zhrHltJddjZvUry8TRBqzLWV4PnFJsm4jok7QFOBQo9k3YltaTW2fBbztJ5wHnAYxqfSn/vLhzN3c8Utqv6xd3RpHy0uuoVD1Zx9K9uZfOzs6S66m0np6eqj5+vlqKx7EUVkuxQG3Fk3UsWSaOqoqIpcBSgNFTZu39tmxraeK/L35DSXWc9qXb9zbp5CqnjkrVMxSxLFiwoOR6Kq2zs7Oqj5+vluJxLIXVUixQW/FkHUuWnePdwFE5y1PTsoLbSBoBTASeH6TOqYPUWVTTyEYuWthe6uZctLCdppGNB1RHpeqppVjMrL5lmTjuBmZJmiFpFLAYWJa3zTLgA+n9s4HbI6JwWwoQERuAFyS9Jh1N9cfAT0oJpq2liS++8/iy2vHPmtfGF995PG0tTWg/68ivp1ZiaW0eA8DEppH7VY+Z1a/MmqrSPouPArcAjcD3I+IRSZcC90TEMuAK4CpJXcDvSJILAJKeAJqBUZLOAt4cEY8Cfwb8AGgi6RQftGN8enNDWc05uc6a11aRL9X+eg7kELKSsZz5qiM57pLlvPX4KU4aZlaWTPs4ImI5sDyv7JKc+9uBdxXZd3qR8nuA4yoXZX2SxNTxDax+5oVqh2JmBxmfOV7HjprQwJpntrJnT9HWQTOzl3HiqGNHTWjgxZ27Wb/p5aOszMyKceKoY1MnJG//KjdXmVkZnDjqWNv4BiRY88zWaodiZgcRJ446NmaEmHbIWHeQm1lZnDjqXEfrBFZv8BGHmZXOiaPOdbQ288Tz2+jduXvwjc3McOKoex2tE9gT8NizPuows9I4cdS5jinNAG6uMrOSOXHUuWmHjKVpZCOrPbLKzErkxFHnGhvE7CPGe2SVmZXMicPoaG1m9TNbGeDCxGZmezlxGB1TJvC7bTvZ2LOj2qGY2UHAicNob50AuIPczErjxGF0tKYjq9zPYWYlcOIwDhk3iiOaR3tklZmVxInDAGhvbXZTlZmVxInDADi2dQJdz/bQt3tPtUMxsxrnxGFAMrJq5+49PP7ctmqHYmY1zonDAGg/IukgX+V+DjMbhBOHAXDM4eMY0SDWeGSVmQ3CicMAGD2ikVdMHucOcjMblBOH7dV/6REzs4E4cdheHVMm0L25lxe276p2KGZWw5w4bK+O9NIja3zUYWYDcOKwvfZeemSDO8jNrDgnDttrysQxNI8Z4X4OMxuQE4ftJckd5GY2KCcO20fHlAms8aROZjYAJw7bR0drMz07+li/qbfaoZhZjXLisH3sndTJzVVmVkSmiUPSIklrJHVJurjA+tGSrk3Xr5A0PWfdkrR8jaSFOeUXSnpE0sOSfixpTJbPod607x2S65FVZlZYZolDUiNwOfAWYA7wHklz8jY7F9gUETOBrwNfTvedAywG5gKLgG9JapTUBvw5cFJEHAc0pttZhYwfPYJph4z1xQ7NrKgsjzhOBroiYm1E7ASuAc7M2+ZM4Mr0/vXAGZKUll8TETsi4nGgK60PYATQJGkEMBZ4OsPnUJfaWyf4XA4zK2pEhnW3AetyltcDpxTbJiL6JG0BDk3L78zbty0ifiXpa8BTQC9wa0TcWujBJZ0HnAcwefJkOjs7D/gJVUJPT0/Nx9K0YydrN+7i1tvuYFSjqhpLtdRSPI6lsFqKBWornqxjyTJxVJykSSRHIzOAzcC/SHpfRPwwf9uIWAosBWhvb48FCxYMZahFdXZ2UuuxbDtkA8t+cx9HdsznuLaJVY2lWmopHsdSWC3FArUVT9axZNlU1Q0clbM8NS0ruE3a9DQReH6Afd8IPB4RGyNiF3AD8NpMoq9jHVOSDvJVbq4yswKyTBx3A7MkzZA0iqQTe1neNsuAD6T3zwZuj+TMs2XA4nTU1QxgFnAXSRPVaySNTftCzgBWZfgc6tL0Q8cxekSDL3ZoZgVl1lSV9ll8FLiFZPTT9yPiEUmXAvdExDLgCuAqSV3A70hHSKXbXQc8CvQB50fEbmCFpOuB+9LylaTNUVY5jQ1i9hETfC6HmRWUaR9HRCwHlueVXZJzfzvwriL7XgZcVqD8s8BnKxup5etoncAda56tdhhmVoN85rgV1DGlmed6drJx645qh2JmNcaJwwrypE5mVowThxXUsfeaVR5ZZWb7cuKwgg4dP5rJE0a7g9zMXsaJw4rqaJ3gIw4zexknDiuqo3UCj/22h77de6odipnVECcOK6qjtZkdfXt44vkXqx2KmdUQJw4rqt0d5GZWgBOHFTXz8PE0NshDcs1sH04cVtSYkY3MOGwcqzY4cZjZS5w4bEAeWWVm+QZNHJJmS7pN0sPp8islfTr70KwWHDulmfWbetm6fVe1QzGzGlHKEcd3gSXALoCIeBDP81032o9IOsh//Vs3V5lZopTEMTYi7sor68siGKs9L03q5MRhZolSEsdzko4BAkDS2cCGTKOymtHW0sSE0SM8ssrM9iplPo7zSSZL6pDUDTwOvDfTqKxmSKLdHeRmlqOUI46IiDcCk4GOiHhdifvZMNExJZkNMJnV18zqXSkJ4F8BImJbRPS3V1yfXUhWazpam9m6vY+nt2yvdihmVgOKNlVJ6gDmAhMlvTNnVTMwJuvArHbsnZtjwwu0tTRVORozq7aB+jjagbcDLcAf5JRvBf4ky6Cstszee82qrZxx7BFVjsbMqq1o4oiInwA/kXRqRPxqCGOyGtM8ZiRTJzV5UiczA0obVbVS0vkkzVZ7m6gi4kOZRWU1p6N1Aqs3eGSVmZXWOX4V0AosBH4OTCVprrI60tHazNrntrGjb3e1QzGzKislccyMiM8A2yLiSuBtwCnZhmW1pmPKBHbvCbqe7al2KGZWZaUkjv6r222WdBwwETg8u5CsFr00ssoHm2b1rpQ+jqWSJgGfBpYB44HPZBqV1Zzph45j1IgG1vhih2Z1b9DEERHfS+/+AngFgKRpWQZltWdEYwOzDh/PKneQm9W9AZuqJJ0q6WxJh6fLr5T0I+C/hyQ6qykdrc0ekmtmxROHpK8C3wf+ELhJ0heAW4EVwKyhCc9qybFTJrBx6w6e79lR7VDMrIoGaqp6GzAvIranfRzrgOMi4okhicxqTnvaQb7mma28duboKkdjZtUyUFPV9ojYDhARm4DHyk0akhZJWiOpS9LFBdaPlnRtun6FpOk565ak5WskLcwpb5F0vaTVklZJOrWcmGz/dbQ2A7DKzVVmdW2gI45XSFqWszwjdzki3jFQxZIagcuBNwHrgbslLYuIR3M2OxfYFBEzJS0Gvgy8W9Ickulp5wJHAj+TNDsidgPfAG6OiLMljQLGlvxs7YBMnjCaw8aPYo3n5jCrawMljjPzlv+mzLpPBroiYi2ApGvSOnMTx5nA59L71wPflKS0/JqI2AE8LqkLOFnSo8DvAecARMROYGeZcdkBSCZ18hGHWT1TVpPzpFPMLoqID6fL7wdOiYiP5mzzcLrN+nT5NyRnpX8OuDMifpiWXwH8B9BFMhvho8AJwL3ABRGxrcDjnwecBzB58uQTr7vuukyeZ7l6enoYP358tcMA9i+WH6/awR3r+vjHN42lQapqLFmqpXgcS2G1FAvUVjyViOX000+/NyJOKrSulBMAa8kIYD7wsYhYIekbwMUUOCExIpaSJBna29tjwYIFQxlnUZ2dnRzMsWwcv45bnnyQ6ce9mldMrtw/SS29LlBb8TiWwmopFqiteLKOJcspYLuBo3KWp6ZlBbeRNILkcibPD7DvemB9RKxIy68nSSQ2RPo7yN1cZVa/skwcdwOzJM1IO7EXk1yyJNcy4APp/bOB2yNpO1sGLE5HXc0gOW/kroh4BlgnqT3d5wz27TOxjM06YjwNcuIwq2eDNlVJ+ncgvyNkC3AP8J3+Ibv5IqJP0keBW4BG4PsR8YikS4F7ImIZcAVwVdr5/TuS5EK63XUkSaEPOD8dUQXwMeDqNBmtBT5Y1jO2AzJmZCMzDhvnuTnM6lgpfRxrgcnAj9Pld5PMxzEb+C7w/mI7RsRyYHle2SU597cD7yqy72XAZQXK7wcKdtjY0Ohobeah7i3VDsPMqqSUxPHaiHh1zvK/S7o7Il4t6ZGsArPa1dE6gZse2sC2HX2MG32wja8wswNVSh/H+Nyr4ab3+4fT+ByKOrT30iO+xLpZXSrl5+IngV+m51gImAH8maRxwJVZBme16dgp6ciqDVuZP21SlaMxs6FWynwcyyXNAjrSojU5HeJ/l1lkVrPaWpoYP3qELz1iVqdKbaA+EZiebn+CJCLinzOLympaQ4OYfcR4X+zQrE6VMhz3KuAY4H6gf0hsAE4cdaxjSjM/feBpIgJV8NIjZlb7SjniOAmYE1ld1MoOSse2TuBHK/p45oXtTJnYVO1wzGwIlTKq6mGgNetA7ODS3vpSB7mZ1ZdSjjgOAx6VdBewd87QwebjsOGtf0ju6me2cnrH4VWOxsyGUimJ43NZB2EHn4lNI2lraWK1R1aZ1Z1ShuP+fCgCsYNPe+sEN1WZ1aGiiUPSLyPidZK2su9FDgVERDRnHp3VtAYlZ4/PuPgmjmxp4qKF7Zw1r62sOm5c2c1Xb1lD9+Ze2u68fb/qyK3n6c29+x1LpeJxLNnGU6lYbP8VTRwR8br074ShC8cOFjeu7Obnv94IJL8qujf3suSGhwBK/ie+cWU3S254iN5dySjv/amj1upxLNnWU6lY7MCUNHWspEbgCHISTUQ8lWFcFdXe3h5r1qypdhjA8Jkl7LQv3U735t6XlY9sFHOOnFhSHY8+vYVdu1/++Sunjlqrx7FkW0+xOtpamvjvi99QcixZGC7/2/0k7f/UsZI+BnwW+C2wJy0O4JUHFJUd1J4ukDQAdu0OWppGllRHoS+AcuuotXocS7b1FKuj2OfRslHKqKoLgPaIeD7rYOzgcWRLU8EjjraWJq780Mkl1VHsqKWcOmqtHseSbT3F6jiyxSehDqVSTgBcRzLjn9leFy1sp2lk4z5lTSMbuWhhe5E9sqmj1upxLNnWU6lY7MCUOgNgp6Sb2PcEwL/NLCqref0dkQcyQia3ju7NvbTt5yibSsRSqXgcS7bx5McyokF88Z3Hu2N8iA3aOS7ps4XKI+LzmUSUAXeOF+ZYiquleBxLYRd+71Zu/M0uHvjsm2keU3pfS1Zq6bWpaud4OppqdkS894AiMDOrsGNaGonYxQPrNvP6WZOrHU5dGbCPIyJ2A0dLGjVE8ZiZleSYlgYkuO/JzdUOpe6U2sfx35KWAdv6C93HYWbV1DRCzD58Avc9tanaodSdUhLHb9JbA+CzyM2sZsw/uoWbHtzAnj1BQ4MnFBsqpVzk8KDpBDez+jJv2iR+fNc61j7Xw8zD/bt2qJRy5vhk4C+AucCY/vKIqO75/WZW9+ZPmwQk/RxOHEOnlBMArwZWAzOAzwNPAHdnGJOZWUlecdg4JjaN5N4n3c8xlEpJHIdGxBXAroj4eUR8CPDRhplVXUODmDetxR3kQ6yUxLEr/btB0tskzQMOyTAmM7OSnThtEo8928OW3l2Db2wVUUri+IKkicAngU8B3wMuzDQqM7MSzT866ee4f53P5xgqpYyq+ml6dwtwerbhmJmV54SjWmgQ3PfkJn5/ts8gHwqDHnFImi3pNkkPp8uvlPTp7EMzMxvc+NEjmH2ETwQcSqU0VX0XWELa1xERDwKLS6lc0iJJayR1Sbq4wPrRkq5N16+QND1n3ZK0fI2khXn7NUpaKemn+XWaWf2Zf/Qk7l+3mT17Bp/R1A5cKYljbETclVfWN9hO6QUSLwfeAswB3iNpTt5m5wKbImIm8HXgy+m+c0iS01xgEfCttL5+FwCrSojdzOrA/GmT2Lq9j66NPdUOpS6Ukjiek3QMyXSxSDob2FDCficDXRGxNiJ2AtcAZ+ZtcyZwZXr/euAMSUrLr4mIHRHxONCV1oekqcDbSDrpzcyYP60FSPo5LHulXKvqfGAp0CGpG3gcKOUy620kswf2Ww+cUmybiOiTtAU4NC2/M2/f/pla/o7kTPYBTxOVdB5wHsDkyZPp7OwsIeTs9fT0OJYCaikWqK14HEthubFEBONHwk0rVtH64tqqx1NtWcdSyqiqtcAbJY0DGiJiq6SPk3yBDylJbweejYh7JS0YaNuIWEqS8Ghvb4/hNMFKpTiW4mopHsdSWH4sJz95N0/97kUWLPj9moinmrKOpZSmKgAiYltEbE0XP1HCLt3AUTnLU9OygttIGgFMBJ4fYN/TgHdIeoKk6esNkn5Y6nMws+Fr/rQWup7tYcuLPhEwayUnjjylXL/4bmCWpBnpRFCLgWV52ywDPpDePxu4PZK5bJcBi9NRVzOAWcBdEbEkIqZGxPS0vtsj4n37+RzMbBjpv+DhynXu58ja/iaOQce8RUQf8FHgFpIRUNdFxCOSLpX0jnSzK4BDJXWRHMVcnO77CHAd8ChwM3B+OhuhmVlBe08EfMpnkGetaB+HpK0UThACmkqpPCKWA8vzyi7Jub8deFeRfS8DLhug7k6gs5Q4zGz4Gzd6BO2tzaz0iYCZK5o4IsIXtzezg8r8aS0su/9pzwiYsf1tqjIzqznzp01i644+HnvWJwJmyYnDzIaN/ivlemKnbDlxmNmwMf3QsRwybpQveJgxJw4zGzYkMd8zAmbOicPMhpV50yaxduM2Nr+4s9qhDFtOHGY2rOw9EdDnc2TGicPMhpUTjppIY4PcXJUhJw4zG1bGjhpBR6tnBMySE4eZDTvzp03i/qc2s9szAmbCicPMhp35R7ewbedufv3brYNvbGVz4jCzYae/g9zNVdlw4jCzYWfaIWM5dNwo7nvSI6uy4MRhZsOOJOZNm+Qr5WbEicPMhqX5R7ew9rltbNrmEwErzYnDzIYlzwiYHScOMxuWXjk1PRHQ/RwV58RhZsPS2FEjOHaKTwTMghOHmQ1b86dN4oF1m+nbvafaoQwrThxmNmzNnzaJbTt3s8YnAlaUE4eZDVsvnQjofo5KcuIws2HrqEOaOGz8aFZ6KtmKcuIws2HLMwJmw4nDzIa1+UdP4onnX+T5nh3VDmXYcOIws2HNMwJWnhOHmQ1rr5w6kRGeEbCinDjMbFgbM7KROfl8oCEAAAvNSURBVEc2O3FUkBOHmQ17yYmAW3wiYIU4cZjZsDdvWgu9u3az+hmfCFgJThxmNuy91EHu5qpKyDRxSFokaY2kLkkXF1g/WtK16foVkqbnrFuSlq+RtDAtO0rSHZIelfSIpAuyjN/Mhoepk5qYPGG0zyCvkMwSh6RG4HLgLcAc4D2S5uRtdi6wKSJmAl8HvpzuOwdYDMwFFgHfSuvrAz4ZEXOA1wDnF6jTzGwfPhGwsrI84jgZ6IqItRGxE7gGODNvmzOBK9P71wNnSFJafk1E7IiIx4Eu4OSI2BAR9wFExFZgFdCW4XMws2Fi/rRJPPn8izznEwEP2IgM624D1uUsrwdOKbZNRPRJ2gIcmpbfmbfvPgkibdaaB6wo9OCSzgPOA5g8eTKdnZ379ywqrKenx7EUUEuxQG3F41gKKzcWbdoNwFXL/4t5h1f+q+9gfm3KlWXiyIyk8cC/Ah+PiBcKbRMRS4GlAO3t7bFgwYKhC3AAnZ2dOJaXq6VYoLbicSyFlRvLa3bt5it338LO5qksWNBR9XiylHUsWTZVdQNH5SxPTcsKbiNpBDAReH6gfSWNJEkaV0fEDZlEbmbDzpiRjcw9spn7fKXcA5Zl4rgbmCVphqRRJJ3dy/K2WQZ8IL1/NnB7RERavjgddTUDmAXclfZ/XAGsioi/zTB2MxuG5k2bxAPrN7PLJwIekMwSR0T0AR8FbiHpxL4uIh6RdKmkd6SbXQEcKqkL+ARwcbrvI8B1wKPAzcD5EbEbOA14P/AGSfent7dm9RzMbHiZf/Qktu/aw+oNPhHwQGTaxxERy4HleWWX5NzfDryryL6XAZfllf0SUOUjNbN6cOLR/TMCbuL4qROrHM3By2eOm1ndOHLiGI5oHu3zOQ6QE4eZ1Y3kRMBJThwHyInDzOrK/GmTWPe7XjZu9YmA+8uJw8zqyvyjWwB81HEAnDjMrK7MPXIiIxs9I+CBcOIws7qSnAg4kZVP+kq5+8uJw8zqzvxpk3iw2ycC7i8nDjOrO/OPbmH7rj2s2lDwUnc2CCcOM6s7/TMC+rpV+8eJw8zqzpEtTbQ2j/GMgPvJicPM6tL8oz0j4P46KOfjMDM7UKMaG1i/qZcZF9/EkS1NXLSwnbPmlT+h6I0ru/nqLWvo3txL252371c9/XU8vbm3ZmIZ1TrzxGLbOHGYWd25cWU3//HwMwAE0L25lyU3PARQ1hftjSu7WXLDQ/TuSmYX3J96KlFHVrEU48RhZnXnq7esYUffvkNxe3ft5i//9UFuWJk/31xxK9Y+f8D1VKKOLGMpxInDzOrO05t7C5bv6NvDC727Sq6n2JdsOfVUoo6sY8nnxGFmdefIlia6CySPtpYmbjz/tJLrOe1Ltx9wPZWoI+tY8nlUlZnVnYsWttM0snGfsqaRjVy0sH3I66n1WArxEYeZ1Z3+zuIDHcmUW0/35l7a9qOeWo1lwwDbOXGYWV06a17bfg15LVZPZ2cnCxYsGDaxaEnXvcW2cVOVmZmVxYnDzMzK4sRhZmZlceIwM7OyOHGYmVlZnDjMzKwsThxmZlYWJw4zMyuLE4eZmZXFicPMzMrixGFmZmVx4jAzs7JkmjgkLZK0RlKXpIsLrB8t6dp0/QpJ03PWLUnL10haWGqdZmaWrcwSh6RG4HLgLcAc4D2S5uRtdi6wKSJmAl8HvpzuOwdYDMwFFgHfktRYYp1mZpahLI84Tga6ImJtROwErgHOzNvmTODK9P71wBmSlJZfExE7IuJxoCutr5Q6zcwsQ1nOx9EGrMtZXg+cUmybiOiTtAU4NC2/M2/f/ovVD1YnAJLOA85LF3dIeng/nkMWDgOeq3YQKcdSXC3F41gKq6VYoLbiqUQsRxdbMWwncoqIpcBSAEn3RMRJVQ4JcCzF1FIsUFvxOJbCaikWqK14so4ly6aqbuConOWpaVnBbSSNACYCzw+wbyl1mplZhrJMHHcDsyTNkDSKpLN7Wd42y4APpPfPBm6PiEjLF6ejrmYAs4C7SqzTzMwylFlTVdpn8VHgFqAR+H5EPCLpUuCeiFgGXAFcJakL+B1JIiDd7jrgUaAPOD8idgMUqrOEcJZW+OkdCMdSWC3FArUVj2MprJZigdqKJ9NYlPzANzMzK43PHDczs7I4cZiZWVmGdeKolcuTSDpK0h2SHpX0iKQLqhVLTkyNklZK+mkNxNIi6XpJqyWtknRqFWO5MH2PHpb0Y0ljhvjxvy/p2dzzjiQdIuk/JT2W/p1UxVi+mr5PD0r6N0kt1YolZ90nJYWkw6oZi6SPpa/NI5K+MhSxFItH0qsk3Snpfkn3SDq5ko85bBNHjV2epA/4ZETMAV4DnF8Dl0q5AFhV5Rj6fQO4OSI6gBOoUlyS2oA/B06KiONIBmAsHuIwfkBymZ1cFwO3RcQs4LZ0uVqx/CdwXES8Evg1sKSKsSDpKODNwFNDFEfBWCSdTnIVixMiYi7wtWrGA3wF+HxEvAq4JF2umGGbOKihy5NExIaIuC+9v5Xki7Ft4L2yI2kq8Dbge9WKISeWicDvkYywIyJ2RsTmKoY0AmhKzysaCzw9lA8eEb8gGWGYK/fSPFcCZ1Urloi4NSL60sU7Sc6lqkosqa8DfwEM2SifIrF8BPhSROxIt3m2yvEE0Jzen0iFP8fDOXEUuuRJ1b6s+6VXAJ4HrKhiGH9H8s+2p4ox9JsBbAT+KW06+56kcdUIJCK6SX4pPgVsALZExK3ViCXPERGxIb3/DHBENYPJ8SHgP6r14JLOBLoj4oFqxZBjNvD69CrfP5f06irH83Hgq5LWkXymK3pkOJwTR82RNB74V+DjEfFClWJ4O/BsRNxbjccvYAQwH/h2RMwDtjF0TTH7SPsOziRJZkcC4yS9rxqxFJOeIFv1MfSS/pqkCfbqKj3+WOCvSJphasEI4BCSpuiLgOvSC7ZWy0eACyPiKOBC0iP6ShnOiaOmLk8iaSRJ0rg6Im6oVhzAacA7JD1B0nz3Bkk/rGI864H1EdF/BHY9SSKphjcCj0fExojYBdwAvLZKseT6raQpAOnfIWsGKUTSOcDbgfdG9U4EO4YkwT+QfpanAvdJaq1SPOuBGyJxF8nR/JB01hfxAZLPL8C/kDTdV8xwThw1c3mS9JfHFcCqiPjbasTQLyKWRMTUiJhO8prcHhFV+1UdEc8A6yS1p0VnkFwxoBqeAl4jaWz6np1BbQwgyL00zweAn1QrEEmLSJo53xERL1Yrjoh4KCIOj4jp6Wd5PTA//TxVw43A6QCSZgOjqO6Vcp8Gfj+9/wbgsYrWHhHD9ga8lWTkx2+Av65iHK8jaV54ELg/vb21Bl6fBcBPayCOVwH3pK/PjcCkKsbyeWA18DBwFTB6iB//xyT9K7tIvgzPJZlq4Lb0n/9nwCFVjKWLpO+w/3P8j9WKJW/9E8BhVXxdRgE/TD839wFvqPJn5nXAvcADJP2pJ1byMX3JETMzK8twbqoyM7MMOHGYmVlZnDjMzKwsThxmZlYWJw4zMyuLE4dZBUjanV6JtP9WsbPfJU0vdFVYs2rJbOpYszrTG8mVSM2GPR9xmGVI0hOSviLpIUl3SZqZlk+XdHs6r8Vtkqal5Uek81w8kN76L3nSKOm76VwPt0pqqtqTsrrnxGFWGU15TVXvzlm3JSKOB75JcmVigH8AroxkXourgb9Py/8e+HlEnEByza5H0vJZwOWRzPWwGfjDjJ+PWVE+c9ysAiT1RMT4AuVPkFx+Ym16octnIuJQSc8BUyJiV1q+ISIOk7QRmBrpvA5pHdOB/4xkIick/SUwMiK+kP0zM3s5H3GYZS+K3C/Hjpz7u3H/pFWRE4dZ9t6d8/dX6f3/4aVpad8L/Fd6/zaSuRT654WfOFRBmpXKv1rMKqNJ0v05yzdHRP+Q3EmSHiQ5anhPWvYxklkPLyKZAfGDafkFwFJJ55IcWXyE5MqnZjXDfRxmGUr7OE6KiGrOzWBWUW6qMjOzsviIw8zMyuIjDjMzK4sTh5mZlcWJw8zMyuLEYWZmZXHiMDOzsvx/TJMX3205T0IAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "_p5BEeSSzqfz", "colab": { "base_uri": "https://localhost:8080/", "height": 324 }, "outputId": "4cf3e05d-c445-48db-931f-bfcc87d39246" }, "source": [ "pd.DataFrame(history.history).plot(figsize=(8, 5))\n", "plt.grid(True)\n", "plt.gca().set_ylim(0, 1);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeMAAAEzCAYAAAACSWsXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3xc1Z3//9eZ3jTq3b135LhhqkxZygIGg2MIfb9AEkJJyJIQUiAJsAklYclCgORHMWExBAJhqcFgxSYYY4N7wZa7LFu9S6Np5/fHnRlV27Ite6TR5/l4zOPee+6Zq3Mke95zy7lXaa0RQgghRPyY4t0AIYQQYqCTMBZCCCHiTMJYCCGEiDMJYyGEECLOJIyFEEKIOJMwFkIIIeLssGGslHpOKVWulNpwkPVKKfWEUqpYKbVOKfWN3m+mEEIIkbh6smf8AnD+IdZfAIyOvG4B/njszRJCCCEGjsOGsdZ6KVB9iCpzgYXa8DmQopTK7a0GCiGEEImuN84Z5wN72y2XRMqEEEII0QOWE/nDlFK3YBzKxul0Ths8eHCvbTscDmMyJd71aInYr0TsEyRmv6RP/Uci9ivR+rR169ZKrXVmd+t6I4z3Ae1TdVCkrAut9bPAswDTp0/Xq1at6oUfbygqKqKwsLDXttdXJGK/ErFPkJj9kj71H4nYr0Trk1Jq98HW9cZXjreB6yJXVZ8M1Gmt9/fCdoUQQogB4bB7xkqpV4BCIEMpVQLcB1gBtNZPA+8BFwLFQDNw4/FqrBBCCJGIDhvGWuurDrNeA9/rtRYJIYQQA0zinBkXQggh+ikJYyGEECLOJIyFEEKIOJMwFkIIIeJMwlgIIYSIMwljIYQQIs4kjIUQQog4O6H3phZCCCEOKhSEYAsEWiDQjKtpD+z7MrLsg0BzbF3HaQuEAxAOQjhkvHQostxdWbtph7JO77N54DvLTkjXJYyFEGKgCQWgtQH8jeBvgtZG8De0m4+8Ar5IOIXaTcOdlkMQDndcjtXprm6oLUCjYRoN4JC/QzNnAqw8TF+UCSxOsNhAmcFkAZM58rJ0LYstR8sO8T6753j9BbqQMBZCiP4q4IPyTXBgPUN2fwGLiw4drtH5TqF3aKpdiEWnpk7L7cs7r+u8bAGHF5JywOqMvFydpk6wONm4bScTT5redZ3VBVaHMTXbQKnj9Rs+YSSMhRCiPwi0QNlGKF0N+9fA/rVQvtk4nAqMANhtMQ6t2pPA5o7Me8CT1TZvc4Mtqd18N/VtkWWrM65BV1FXBGML4/bzTyQJYyGE6Gv8TXBgvRG4pZHgrdhiHOYFcKZBXgGccq4xzZnCP9cUc+accxJiL3EgkjAWQoh48tVHgndNW/hWbgW0sd6dZQTuuAshtwByT4LkQV1CV5t2SxD3YxLGQghxLLRud3VuwLg4KhyMTAPGFcLt17XWw/51beFbVdy2raRcI3AnzTNCN7fAOLcqIZvwJIyFEP2b1sYFSYEWCEaHv/jaDZGJlEXW5Zesh09Xt7uiN3o1r6/jlb3Rq3s7hGo0aNsHb+Do2p082AjcKVdGgvckSMru3d+N6DckjIUQvS8UhEAT+JuN85/R+eg02NoWgMF2r2iIBlsjwdhNvfbl0XrRQ7o9MBogujNqccSu3I1dnRst82QZV+qarWCyRqaRITHR+eg6kwXMlo71YvWtHddZXZA9EdwZx+EXL/orCWMhEpnWxp5csLVtLy/UGpn6I+WR+W7Kc0vXwmcbOgVqJGD9TW3zgeZI8DYa80c0dCZCmSPBaG8LR4ujLRxdae2WHZGxpfa298SC1NW23ursMFQGq4NPv/iK0848x6hrkpsQir5BwliIeAqH2oLN39R2E4ZAc9t8+/Iu89Hl5naB2+l1DMYCbI0smO1gcxnDX6wuY97qNvYgra7I0Bh323z7OrZombtdOHYKVvOJ+TgKWouN9gjRh0gYC9Ge1ka4+eqMlz9yrjHUGjk06jvItJWRu4qh8e125ZF1nd8baGkL0mBLz9umzEYQRkMvOi7Uk2OEi8VhHAY129sOr1rskTJbpDxaZjtE3bayz1au5pQzz4ncXEE+LoQ4XuR/l0gsWhth56ttC9QOr1poOdi6yCs6lvNImKzkKgtUuyN7fLbI1N52mNWZaoSc1dkxTHs6H4c7Dfntu427JQkhjisJY9F3hMORW/Y1GMM/fPWR+Tpj6qs3yjvMt6sXDdPDXd1qdYEjOfJKAU82ZIxpV5YMzhSwe40gjAZq54CNTs12MJn4tKiIwsLCE/KrEkIkFgljcWKEQ8Zt/LYvYfymf0LJ/3QK1kgAH/aqWGWEpMNr3MLP7jXOWaaPNIK1faC2D9boOrvXCFUhEoDWGh0IoFtaoLUVHQ6j5KK0Q9KhEOGWFsJNTYSbmo1pc7tpc6S8uQmAzO9974S0S8JYHD/VO2D7EtixBHYuNfZaAa8jByy5RjC6MyMhmdQ1ZB1eY2qPlDm8xgVAx/HDRmtNuKmJUG0d4fo6TN5krHm58gEnjprxb6qZUG0tobpawvX1hFt8aF8L4RYf4VYfusVH2NeC9vmMda3GNOxriazzGet8PnRLC+F282jjC2w2sAVQLhcmp9N4RefdLpTT1bYcK4vWcWFytdVXkTJli35x1dHOxH5e56mOLdP1PZ2mOhRCh0IQCqGDIQgFjbKg8VQnHQyhQ0Hs69ZR19hkrA+G0OFO7wlGtxMk7GvtGqxdgrbZ+J31kDk9XcJY9EPN1Ubo7lhihHDtbqPcOwjGXwwj5sCIQlas3HBcD+dqrdE+H6H6+lioRudD9fWE6usI19URqqsnVGeUGct1hBoaINTxnLFyOLANH459xAhsI0dgHzES24jh2IYNw2TrP3vZOhiM7A00EWpsJNzYFFluNKaNjYSamozyRqPMW1VJ2YovMKemYklLxZyWhjklMp+aisnrRR3n89g6HCZcX0+wuoZQdRXBqmpjWl1NqKramNbWosxmTG63ESjdTd1GGFmLi/FlZ3dYpxyOw/ZDa41uaYmEal27aV03ZZFp5EWgZzcGUVarEZAOB8rpwGSPTB1OzBkZKIejbZ3DiXLYY9PtX29leG4O4eYWI3haWgi3NKObm42yquq28majvK9LAUp7Wvkgf39rSsqh/10c5N+JyeVCmc3HsXcdSRiLoxdshb1ftIVv6WpAG3uyw06H2bfByDmQPuqQFx4Zh418hJubjG/9kQ8L40MlUhZdjs63/5CJfONtC9p6tP8QQ3qUwuz1YkpOxpycjNnrxTYo31j2GsvmlGRMSUmEamrx79hB644dtKxeTf2777Ztx2zGNmgQtpEjsY8YjsMfoCU1FduIEZiTknrv9xz9PUX3sGqqCdXUGCFUXUOopoZQbQ2hhgbj8Fpj+3BtjJVpn69HP0fZ7Zg8HkxuN7bGRmrWrD34ey0WzKkpWFIiQZ2WiiU1FXNqWscATzXC25KSAlar0Y/qKoJVVYSqq9sFa5XRp1joVhOsqYFgsNsfb05OxpyejjklhXAoSLC8jFBTE7qpmVBzc7chmAbsfPSxjoUmU+wDuP0HM+EwobraWODqQ4SqcjqN9qSkYE5Oxj5qVIdlc0oK5hTj35dyujA5HSi7A5MzErAOxzF9+K8vKiLzCL7kRr+0xgK6qRnd0tzh/5/2+9v+7yoFsdl2Ze2ndC4/SH2zGWWxGP01W1AWc2Te3KFs5VdfMfPk2SizqWO96HtNZqPMZAKr9bh/MTyeJIzFQcXOR0UPnbU0Ey7diN6+nPDuVYT3bUS3+gmHzeikYYSTL0a7hxC2p6N3tRLevI+w709oX2uHw23pVZUU//qB2IdAT0MiSjkcnQ61GR+i9pEjMSd7MXm9mJNTjFBN9mJOTsbkTY59EJo8nqM+7BxuacG/cyet23fQumM7/h078e/YTuOyZSQHAuxauBAAS2ZmJKSje9MjsI0YiSUrM/aBoUMhY8+ppiYSSJFgrYnMV0dCt6Zt/qBfMqxWzJEANXk8mDxuzJkZxt57tMztNsrb13O7MbmNqdnjNvYSrdbYZosiF6WFm5sjXwAiwd+5vTU1hKpraN28heaaGmNv8GCs1oPuKZrcbszp6VjS0rDm5+OcMhlzWjqW9DTMqWnGND3dCPbU1A5t7Y72+41wbm6OTVd/tpxJo0cd/pBmUxMKhW3YsA6BauocsMlGyJrs9kO2pa9RShl74U5nvJtyUKEDB7CPGB7vZpwQEsYDnA4ECOzfj3/PXvx7dhPYsxf/3r0E9uzGv7fkMEHpaTdfA6wyXmazcd4pcpjN5HSgHMahN3NqCkGLGdeQoShX+/NVkXNW7c5XtT+HZXK52vYmTuCho85MTieOCRNwTJjQoVwHg3z6xt8oyMygdbsR0q07tlP39tuEGxvb3p+UhCU9PXYoM3YerfPP8Xgie5QpWLOzcYwbZ+x1Rg4Tx/ZA09Iwp6UZIXoc9wqie43W/Pwe1dfBoNHH6ragjoa2bmkx+pCeZvQnGrZpab0eaMpmw2KzQWoq0dj219bilaveRR8jYTwAhBvr8W//msDOrfh37TTCdt9+/KUVBCrrINwWCMqisKVYsHoV7tEBzNSjLNoIx9zxqMFTMA35Bip9cCRkI3upkcNsJofjsHsrO4qKmJpgH4bKYiGUnUVSYSFJZ50VK9daEyyvwL9zhxHS23cQrK42Du1GDuV2CNZUI4D707no7iiLBUt6Opb0dPrX/qIQ8SFhfAJFL6DpcKhRqa7nXdrt4ajGRmMPKqq1DpqqoLkSmqugqRKaK9ENFQT2V+AvryFQXo+/qgV/bYBAnSbY0vGQrMkaxpYUxOkJ4R0XxOYJYku1Y81wY0nzoqJjbB1eyJ5knPfNniz38T0KSims2VlYs7Nwn3xyvJsjhOijJIwPQ4dCxvmjxsbYhTGh6FWo0Qtlmhq7XqEarXsUF9C0l0XbrYGPhCXJijUjFfcQL7bcDKx52dgG5WEbMgxzZk7bmNvosCJT/A79CiHEQCdhjHEoMVRVReu2bbRuKzamxcW0bt9OuL6+R9tQNlvkwpnIxTBut3ERz/DhmBxWTKoFU7gec6AK5a+J3OSiEe1vosONLqKzJgvYkvBpCw5vBlg9YPfEbo+oo7dItHrA5kJZrFjz87AOHoxt8OA+fVGGEEKIjgZcGAdravAXF+Pbtg1/cXEsfNsfCjYnJ2MbPQrvhRdgychsuwo1dqVq+6tQXZjdbmNwfDgMNTuhbAMc2GBMy/4FtXvaGpCcBhmjwT3YuHOUO9N4xeazwJNp7LUqFbuaVQghROJK2DAONTQYe7fRvdzINFRRGatjcruxjx5N0rnnYB89GvuoUdhHjzYG1x/uytTWBijbBMXroWyjEb7lm4x7KwMoE6SPhkEzYNoNxjnXnEmQlHvCb/YvhBCib0uIMA7s24fjs+WUrfgiFrzBAwdi65XTiX3kSDynnW4E7hgjeC05OYcPXa2NO0lFA7dsvTGt2dlWx5FsXOhUcLURuNmTIGu88XQeIYQQ4jASIoybPv+c5IULqbHZsI0ciWvGjHZ7uqOw5ucf2U0etIbixfD5U1CyKvIAAwAFaSMgd0rH4E0eJHu7QgghjlpChLHnrLOo/OX9nHb55SjLMXQpHILNb8Oyx+DAeuOeylO+CdkTjcPM2ROMi6aEEEKIXpQQYWxJTSWUnX30QRz0w/rX4NPfQ1Wxca537lMweb48bk8IIcRxlxBhfNT8zfDVQvjsD1BfAjlTYP6LxhOGZNytEEKIE2RghnFLLaz8M3z+R+NOVkNOgYv/G0adLed+hRBCnHADK4wbK4yLslb+2bgoa9S5cPpdMPSUeLdMCCHEADYwwrh2r3Eo+qsXjWfwTrwUTvsB5J4U75YJIYQQCR7Gldvg08dh3SJj+aQr4dTvG3fAEkIIIfqIxAzj/Wth2e9g09/B4oAZN8Hs2yBlcLxbJoQQQnSRWGG8+zNjjHDxYuPezqffBbO+a9zrWQghhOijEiOM96ygYPVPoGgTuDLg7F8Ye8OO5Hi3TAghhDisxAhjXy0OXzlc8DBMvRZsrni3SAghhOixHt2wWSl1vlLqa6VUsVLqnm7WD1FKLVFKrVZKrVNKXdj7TT2E0f/GillPw6xvSxALIYTodw4bxkopM/AkcAEwAbhKKTWhU7WfAa9pracCVwJP9XZDD9NItMl6Qn+kEEII0Vt6smc8EyjWWu/QWvuBRcDcTnU04I3MJwOlvddEIYQQIrEprfWhKyh1BXC+1vqmyPK1wCyt9W3t6uQC/wBSATdwjtb6y262dQtwC0B2dva0RYsW9VY/aGxsxOPx9Nr2+opE7Fci9gkSs1/Sp/4jEfuVaH2aM2fOl1rr6d2t660LuK4CXtBaP6aUmg28pJSapLUOt6+ktX4WeBZg+vTpurCwsJd+PBQVFdGb2+srErFfidgnSMx+SZ/6j0TsVyL26WB6cph6H9D+bhmDImXt/T/gNQCt9XLAAWT0RgOFEEKIRNeTMF4JjFZKDVdK2TAu0Hq7U509wNkASqnxGGFc0ZsNFUIIIRLVYcNYax0EbgM+BDZjXDW9USn1K6XUJZFqPwRuVkqtBV4BbtCHOxkthBBCCKCH54y11u8B73Uq+0W7+U3Aqb3bNCGEEGJg6NFNP4QQQghx/EgYCyGEEHEmYSyEEELEmYSxEEIIEWcSxkIIIUScSRgLIYQQcSZhLIQQQsSZhLEQQggRZxLGQgghRJxJGAshhBBxJmEshBBCxJmEsRBCCBFnEsZCCCFEnEkYCyGEEHEmYSyEEELEmYSxEEIIEWcSxkIIIUScSRgLIYQQcSZhLIQQQsSZhLEQQggRZxLGQgghRJxJGAshhBBxJmEshBBCxJmEsRBCCBFnEsZCCCFEnEkYCyGEEHEmYSyEEELEmYSxEEIIEWcSxkIIIUScSRgLIYQQcSZhLIQQQsSZhLEQQggRZxLGQgghRJxJGAshhBBxJmEshBBCxJmEsRBCCBFnEsZCCCFEnEkYCyGEEHEmYSyEEELEmYSxEEIIEWcSxkIIIUScSRgLIYQQcSZhLIQQQsSZhLEQQggRZxLGQgghRJz1KIyVUucrpb5WShUrpe45SJ1vKqU2KaU2KqX+t3ebKYQQQiQuy+EqKKXMwJPAuUAJsFIp9bbWelO7OqOBnwCnaq1rlFJZx6vBQgghRKLpyZ7xTKBYa71Da+0HFgFzO9W5GXhSa10DoLUu791mCiGEEImrJ2GcD+xtt1wSKWtvDDBGKfUvpdTnSqnze6uBQgghRKJTWutDV1DqCuB8rfVNkeVrgVla69va1XkHCADfBAYBS4HJWuvaTtu6BbgFIDs7e9qiRYt6rSONjY14PJ5e215fkYj9SsQ+QWL2S/rUfyRivxKtT3PmzPlSaz29u3WHPWcM7AMGt1seFClrrwRYobUOADuVUluB0cDK9pW01s8CzwJMnz5dFxYW9qgDPVFUVERvbq+vSMR+JWKfIDH7JX3qPxKxX4nYp4PpyWHqlcBopdRwpZQNuBJ4u1Odt4BCAKVUBsZh6x292E4hhBAiYR02jLXWQeA24ENgM/Ca1nqjUupXSqlLItU+BKqUUpuAJcDdWuuq49VoIYQQIpH05DA1Wuv3gPc6lf2i3bwG7oq8hBBCCHEE5A5cQgghRJxJGAshhBBxJmEshBBCxJmEsRBCCBFnEsZCCCFEnEkYCyGEEHEmYSyEEELEmYSxEEIIEWcSxkIIIUScSRgLIYQQcSZhLIQQQsSZhLEQQggRZxLGQgghRJxJGAshhBBxJmEshBBCxJmEsRBCCBFnEsZCCCFEnCVEGDe1BllaEoh3M4QQQoijkhBh/OrKvTy3wc/f1+yLd1OEEEKII5YQYXzd7KGMSjHxszc3sLe6Od7NEUIIIY5IQoSxxWzi21PsANz12hqCoXCcWySEEEL0XEKEMUCmy8SvL53Eyl01PFW0Pd7NEUIIIXosYcIY4NKp+VxakMd/f7yNr/bUxLs5QgghRI8kVBgD/OrSSeQmO/j+ojU0+OQKayGEEH1fwoWx12Hl8QUFlNQ0c9/bG+PdHCGEEOKwEi6MAaYPS+O2s0bzt6/28fba0ng3RwghhDikhAxjgDvOGsU3hqTw0zfXU1Ijw52EEEL0XQkbxhaziccXTEVruOvVtYTCOt5NEkIIIbqVsGEMMCTdxa/mTuSLXdU8taQ43s0RQgghupXQYQxw2dR8Ljkpj8c/3sZqGe4khBCiD0r4MFZK8etLJ5HjdXDnojU0tgbj3SQhhBCig4QPY4Bkp5XHr4wMd/q7DHcSQgjRtwyIMAaYMSyN2+aM4o2vSvg/Ge4khBCiDxkwYQxwx9mjmTokhXvfXM++2pZ4N0cIIYQABlgYG8OdCgiHNT9YtEaGOwkhhOgTBlQYAwxNd/OruZP4Ylc1T/9Tnu4khBAi/gZcGAPM+0Y+F03J5fcfbWXN3tp4N0cIIcQANyDDWCnFg5dNJtvr4M5Fq2W4kxBCiLgakGEMxnCn3y8oYG91M7+UpzsJIYSIowEbxgAzh6dxa+Eo/vplCe+u2x/v5gghhBigBnQYA9x5zmgKBqfwk7+tk+FOQggh4mLAh7HVbOK/rywgFNbc9aoMdxJCCHHiDfgwBmO40/2XTGTFThnuJIQQ4sSTMI64Ytog/j0y3GmtDHcSQghxAiVMGId1+Jjer5TioUsnk5Vk585Fq2mS4U5CCCFOkB6FsVLqfKXU10qpYqXUPYeod7lSSiulpvdeEw9v5YGV/Nf+/2JN+Zpj2k6yy8rvFhSwu7qZX/6fDHcSQghxYhw2jJVSZuBJ4AJgAnCVUmpCN/WSgDuBFb3dyMPRWtMabuW696/joRUP0RRoOuptnTwinVsLR/LaqhLeWy/DnYQQQhx/PdkzngkUa613aK39wCJgbjf1fg38FvD1Yvt6ZGbuTO7Nu5erxl3Foi2LuOzvl7GsZNlRb+/754zhpEHJ3PPGOorLG3uxpUIIIURXPQnjfGBvu+WSSFmMUuobwGCt9bu92LYj4jA5+Mmsn7DwgoU4LU5u/fhW7ll2DzW+miPeljHcaSpaw3mPL+X7i1bz9YGG49BqIYQQApTWhx5Xq5S6Ajhfa31TZPlaYJbW+rbIsgn4BLhBa71LKVUE/KfWelU327oFuAUgOzt72qJFi3qtI42NjXg8HgACOsA/6v7BR3Uf4TQ5uTztcqa5pqGUOqJt1vjCfLgrwJK9QVpDMDXLzEUjrIxMMfdauw+nfb8SRSL2CRKzX9Kn/iMR+5VofZozZ86XWuvur6nSWh/yBcwGPmy3/BPgJ+2Wk4FKYFfk5QNKgemH2u60adN0b1qyZEmXsq3VW/W33vmWnvTCJP3dj76rSxtKj2rb1Y2t+nf/+FpPuf9DPfTH7+irnl2uP91WocPh8DG2+vC661d/l4h90jox+yV96j8SsV+J1idglT5IJvbkMPVKYLRSarhSygZcCbzdLszrtNYZWuthWuthwOfAJbqbPeMTbXTqaBZesJAfzfgRq8pWcenfL+WVLa8c8TCoVLeNH5w7hn/dcxY/vXA8xeWNXP3nFVz65L/4cOMBwnLXLiGEEMfgsGGstQ4CtwEfApuB17TWG5VSv1JKXXK8G3iszCYz1064ljfnvklBVgEPrXiI69+/nh21O454Wx67hZvPGMHSH83hocsmU9Mc4Nsvfcl5jy/lzdUlBEPHNtZZCCHEwNSjccZa6/e01mO01iO11g9Gyn6htX67m7qFfWGvuLN8Tz5Pn/M0D572IDvrd3LF/13B02ufJhAKHPG2HFYz35o1hE9+eCb/fWUBJqX4watrKXy0iJc+340vEDoOPRBCCJGoEuYOXD2hlOKSkZfw1ty3OHvI2Ty55kkWvLuA9RXrj2p7FrOJuQX5vH/n6fz5uulkeOz8/K0NnPbbJTz9z+00+I486IUQQgw8AyqMozKcGTxy5iP84aw/UNdaxzXvX8PDKx+mOdB8VNszmRTnTMjmzVtP4X9vnsX43CR+8/4WTv3NJ/zuH19T3eTv5R4IIYRIJAMyjKMKBxfy1ty3mD9mPi9teol5b89jeenyo96eUopTRmbw0v+bxd+/dyqzR6bzxCfFnPqbT/jV/21if508L1kIIURXAzqMAZJsSfzs5J/xwvkvYDVZueWjW/jZpz+jrrXumLZ70uAUnrl2Oh/94AwumJzDi8t3ccbDS/jx6+vYWtYQHRYmhBBCYIl3A/qKadnTeP2S13lm7TM8t+E5lu1bxk9m/YRT804lrMOEdMiYhkMdlztPw6EuZd88PcSZUxy8u6GUv2/bzOubFblJKcwenkfh6MGcPCwPr8OD1WSN969BCCFEHEgYt2M327njG3dw3rDz+MVnv+Duf97d6z/DmgdWoA74oBY+WIkxkhuwKjsem5skmwe31Y3b6sZX7+ODZR/gtrhx29x4rG3rPFYPLqsLj9VDmiONQUmDer29Qgghjj8J426MTRvLyxe+zPs736faV41ZmTEpE2Zlxmwyx5ajZSaTqUOdbqemtuVgOEhzoJnKljrWlJSxrrSMbRWVNIWaaTG3or0aS5LGrILUBmupq6ijKdBEU6CJ1lDrQds9NWsq1064lrMGn4XZdOJu2SmEEOLYSBgfhMVk4eKRFx/3n3PJKGMaCmu+2lPD4k1lLN5cxtqvjcdADk4ycdmMEZw9PpvJ+cmEdNAI5mATjf7GWEjvqNvBK1te4a6iu8j35HP1+Ku5bNRleGyJc19XIYRIVBLGfYTZpJgxLI0Zw9L4yYXj2VHRyMeby3l9+df8z5JinvikmKwkO2ePz+bcCVmcMjKHfE/b3u/pg07nmvHXULS3iIWbFvLwyod5cs2TzBs9j6vHX02+J/8QP10IIUQ8SRj3USMyPYzI9DA6vIeTZpzCkq/L+XhzOW+v2ccrX+zBaTVz2ugMzh2fzZxxWWQm2TGbzJw99GzOHno2Gys3snDTQl7Z/Aovb36Zs4eczXUTrqMgqyDeXRNCCNGJhHE/kOq2Me8bg5j3jUG0BkOs2FHN4s1lLN5UxkebylAKThqUwskj0pkxLJVpQ1OZmDGR357xW34w7Qe8suUV/rr1r3y0+yOmZEzh2gnXcs7Qc7CY5M8vhOj/AuEA/pCfkA6htSH0bI8AACAASURBVCaswx1eGh0b3aK17lAvpENodId10fco1AnbgZFP437GbjFzxphMzhiTyS8vmcim/fV8vLmcoq/L+f8+3cHT/zTGL4/O8jB9WBrTh6Zy+bBvc8vkW3h7x9v8ZdNfuHvp3eS4c7h63NXMGzMPr80b514JIcTB+YI+9jftp7SxlNKmUmMafTWVUtFcgab3792QZEvis6s+6/XtdkfCuB9TSjExL5mJecnccfZofIEQa/fWsmp3Dat2VfPOulJe+WIPAFlJdqYPG8O8IU9gHr6Ff5a9zmNfPsZTa5/islGXcc34axjsHRznHgnR//mCPipaKqhorqC8pZyK5o7zLcEWTso8idl5s5mePR2X1RXvJsddc6C5a9A2lbJl/xbuf/V+qnxVHeqblZkcdw55njxOzj2ZPE8eHqsHhYqNdOn8UijMJnOsjlmZUSpSn3b1lIqtO5H3fpAwTiAOq5lZI9KZNSIdgHBYs7W8gVW7jHBeuauG99YfAMBlW8C4IRegnct49evXeGXLK8wZPIdrJ1zLtOxpKKXi2RUh+pxAOEBVSxXlzeUdwrWsucwI3JYKypvLqffXd3mvzWQj05VJlisLu9nOa1+/xl82/wWLyUJBZgGz82YzO3c2E9In9JlhiU2BJsqbywnrMMFwMHbIN6iDHW6AFF1uXy96qLfL+8JhWkOtHGg+wP7G/exr3Mf+pv3UttZ2+NlWk5U8Tx4Ok4Ppg6eT684lz5NnvNx5ZLoyE+40W2L1RnRgMinG5XgZl+PlmpOHArC/riUWzqt217B5+/lo8ynYUpdTFF7BJ3s/Ic85iusmXMc3x1+E1Sx3BRN9S1OgKRZ+lS2VVDRXUOmrjD0Otf35v+ihS601YYyy0qpSlny25JB1NBo0NAYaYyFb46vpcijUrMxkODPIcmUxJGkI07Knke3KNoLXmRULYK/N2+ELri/o46vyr/i89HOW71/OH1b/gT+s/gNem5dZubM4OfdkZufNZnDSiTlaVdlSyeaqzWyp3sLm6s18Xf01exr2HLef57Q4YwE7OWNyLGhz3bnke/JJd6ZjUiaKioooPKXwuLWjL5EwHmByk51cfJKTi0/KA6DBF2D1nlpW7Z7GF7v2s672E/YmL+M3X/6C337xGBNTTmX+pNkUZE1hWPIwTGrA385cHAdaa+r99Ua4Rg7xVrZUUt5cHiuLBm9zsOvT1awmKzazDYVCKdVx2qnM7/ezvWS7UXaQOtGpy+Ii25XNpIxJHcI105lJpiuTNEfaUf2fcFgcnJJ3CqfknQJAVUsVK/av4PP9Rjh/tPsjAAZ5Bhl7zXmzmZkzk2R78jH9nsM6TElDCZurOwZvZUtlrE6+J5/xaeO5ZOQl5CflY1GWDjctir6iNzs62E2Rossd3mcyYzVZu3w5ERLGA16Swxq7IAzGEAidzsbSWl7f9AlLSv/G+trFbFj+HgAeq4eJ6ROZmDGRyRmTmZQxiWxXtvyn6iGtdeywXIYzg2xX9oA6X9gSbGFN+Ro2Vm1sC9l2e7jd3V3OaXGS6cwkw5nBuLRxnJ5/OpmuzFhZNBSP5MO9qKiIwsLCXu7dsUl3pnPhiAu5cMSFaK3ZVb+L5aXLWb5/Oe/tfI+/bv0rJmViQtqEWDgXZBYc8shVIBRge9322B7vluotfF3zNU0B44ZCZmVmRMoITsk7hXFp4xiXNo6xaWPlgs44kTAWHVjNJgoGp1Ew+ArgCt5bX8LP3yuiQe9gzIg66v0lLNy0kGA4CBjPhp6UMSkWzhPTJx7zt/dEEQgH2FK1hdXlq1lTsYY15WuoaKnoUMdj9ZDlyur2le3KJsuVRbojvc+cRzwSgVCAdZXr+GL/F3xx4AvWVqwlEDYOJSdZk8hwGWF6UuZJZLmyOoRr9NCv2+qOcy9OPKUUw5OHMzx5ON8a/y0C4QAbKjcY4Vy6nOc2PMef1v8Jp8XJtOxpzM6dzazcWWz3bWff5n2x4C2uLY79P3VanIxJHcNFIy5ifNp4xqWPY1TKKOxme5x7K6IkjMUhXTh5EKeOXMCD723itVUljMhw8+S8sXiSKthQuYENlRtYX7meor1FsfcM9Q5lUsYkJqVPYlLGJMaljcNhccSvEydIc6iZpSVLWVO+htXlq9lQuQFfyAcYh/5m5s5kauZUBnsHxy4EKm8up6LFuAjoiwNfUNlcSVAHO2zXrMykO9Nj4dzdK8eVE/e97GA4yOaqzaw4sIIv9n/Bmoo1tARbUCjGp4/nmvHXMCNnBlOzpsptWo+A1WRlatZUpmZN5daCW2nwN7DywErjkHbpch7Z90hb5TJItacyLm0c10641gjetHEMSRrSL7/QDSQqXs/VnT59ul61alWHskAgQElJCT6f74i35/P5cDgS7wM/nv1yOBwMGjQIq9U4FPbptkru+ds6SmpauG72UH50/jg8duP7XL2/nk1Vm4xwrljPhqoNlDeXA2BRFkanjo7tQVdtr6KgoKBtwH27gfbdLXcewN9dXYA0R9pRH7o8UlprShpKWF2x2tjzLV9DcW0xYITnuLRxTM2aSkFWAVOzppLlyurRdsM6TLWvmrLmMsqbjLAuay7rEtwN/oYu701zpDEkaQhDvEMYkjSEod6hsfmjDb9DHdIN6zBba7bG9ny/LPuSxkAjAKNSRjErdxYzc2YyLXtanzpa0hcPUx+L/Y37WVW2it1f72b+mfPJcmUlzKmjRPtbKaW+1FpP73ZdXwrjnTt3kpSURHp6+hH/Y2poaCApKak3m9gnxKtfWmuqqqpoaGhg+PDhsfJmf5BHP9zK85/tJNfr4MF5k5kztvugKWsqY0PVBjZWbmR95Xo2Vm6kIdA1RI4Xm8lGhjODDFcGWc7IYdD25xsjh0N7chFOIBRgU/Um1pSvie35Rsc+JlmTmJI1hZTGFObNmsekjEnHfS+1OdAcu8q3rLmMsqYy9jbsZU/DHnbX7459EYpKc6QZ4RwJ6cHewQxNMsL6UIeC238Yaq3ZWbeTFQdWsPLASlYeWBkbkjLMO4wZOTOYmTuTGdkzSHemH7e+H6tE+4CPSsR+JVqfDhXGfeowtc/nY9iwYQnzra4/U0qRnp5ORUXHc5wum4VfXDyBf5+Syz1vrOPG51dy2dR8fn7RBNLctg51s93ZZLuzOXvI2YCxJ7Wnfg8ffPYBUwumdnwMZbvB+d0tRwfid7dsUiY0mhpfTYcLg2JX5rZUxEKkuz1KszKT7khvC+3IucwMZwaljaWsLl/NxqqNsQuM8j35zM6bHdvzHZUyKjYMY2buzOP0F+nIZXUx1DqUod6h3a5vDjSzt2Evexv2srt+N3sa9rCnfg/LS5fz9+1/71A3w5kR26Me6h3K4KTBseCuDFTyxtY3YgEcveo2151L4eBCZubMZEbODHLcOce9z0Iksj4VxoAEcR9yqL/FtKGpvHPHaTy5ZDtPLSlm6dYK7r9kIhdNyT3o+0zKxLDkYYxzjmNW7qxeb6/X5j1oOEX5gj4qWyoPOoSmtKmUdZXrqPZVA8Yh9nFp45g/Zn7svF2mK7PX297bXFYXY9PGMjZtbJd10aCO7kXvqd/DnoY9/Gvfv3ir+K2uGys1Antmzkxm5c5iRs4MBnkGyf9VIXpRnwvjePN4PDQ2Nsa7Gf2C3WLmrnPHcMGkHH78xjpuf2U1f19TyoOXTSLb2zfP3zssDgYlDWJQ0qBD1ovebclr88b9wqjedrigju5F72nYQ+muUq454xqGe4dL+ApxHEkYi2M2PtfL3757Cs//axePffQ15/zun/z0wvEsmDG4336AW03WAXno1WV1xcacAhRVFTEieUScWyVE4pPbKR2E1pq7776bSZMmMXnyZF599VUA9u/fzxlnnEFBQQGTJk1i2bJlhEIhbrjhhljd3//+93Fu/YlnMZu4+YwRfHDnGUzM83LP39bzrT+tYHdVU7ybJoQQfV6f3TP+5f9tZFNp1xuuH0woFMJsPvQ4ugl5Xu67eGKPtve3v/2NNWvWsHbtWiorK5kxYwZnnHEG//u//8t5553HT3/6U0KhEM3NzaxZs4Z9+/axYcMGAGpraw+z9cQ1LMPNKzefzKKVe3no3c2c9/hSfnjuWP7jtOGYTf1zL1kIIY432TM+iE8//ZSrrroKs9lMdnY2Z555JitXrmTGjBk8//zz3H///axfv56kpCRGjBjBjh07uP322/nggw/wegf27eSUUlw1cwgf3XUmp43K4MH3NjPvqX+x5UDPv1wJIcRA0mf3jHu6Bxt1osbjnnHGGSxdupR3332XG264gbvuuovrrruOtWvX8uGHH/L000/z2muv8dxzzx33tvR1OckO/nTddN5Zt5/7397IRU98yq1zRjHZHJ+x7UII0Vf12TCOt9NPP51nnnmG66+/nurqapYuXcojjzzC7t27GTRoEDfffDOtra189dVXXHjhhdhsNi6//HLGjh3LNddcE+/m9xlKKS4+KY9TR2Xw63c28cTH20i2K2aVrGJyfjKTBiUzOT+ZDI/cI1cIMXBJGB/EZZddxvLlyznppJNQSvHwww+Tk5PDiy++yCOPPILVasXj8bBw4UL27dvHjTfeSDgcBuC//uu/4tz6vifNbeP3CwqYW5DHH9//iuLyRv6xqSy2PjfZwcQ8I5gnD/IyKT+ZrKS+OTxKCCF6m4RxJ9ExxkopHnnkER555JEO66+//nquv/76Lu/76quvTkj7+rvCsVmw30FhYSENvgAbS+vZsK+O9ZHXx1vKiN6hNdtrZ1JeMpPyoyGd3GfHLwshxLGQMBZxk+SwcvKIdE4e0XYf48bWIJtK61m/ry4W0p98XR4L6MwkO5PyvMYh7khA53gd/XY8sxBCgISx6GM8dgszh6cxc3harKypNcjm/fWxvecN++r459YKwpGAzvDYmJiXzLjcJMZmJzE2J4mRmR4cVnlknBCif5AwFn2e225h+rA0pg9rC+hmf5DN+xtie88b9tXx2fZKAiEjoc0mxbB0F2Nzkhib7WVsjoexOV6GpLlkvLMQos+RMBb9kstmYdrQVKYNTY2VBUJhdlU28XVZA18fMF4bS+t5f8OB2GFuh9XE6KwkxmQnMS4niTE5xjQryS6HuoUQcSNhLBKG1WxidHYSo7OTuGhKW3mzP8i2ssZYSG8ta2DZtgre+KokVifFZWVMdtth7rE5RmAnO61x6IkQYqCRMBYJz2WzcNLgFE4anNKhvLrJz9boXnRk+tbqfTS0BmN1Ul1Wsr0OMpPsZHsdZHuNaVaSMZ/ldZDpsWOzyM3shBBHT8JYDFhpbluXq7m11pTW+dgaCeh9NS2U1fsoa2iluLyS8oZWQuGudxBLd9twmYKM3PkFWZHgzvI6yI7N28nw2LGaJbSFEF1JGMdJMBjEYpFff1+jlCI/xUl+ipM547K6rA+HNVVNfsrqfVQ0tBpBXd9KWYOPTTv2UdXoZ1NpPZWNrXTObKUg3W0nN9nBkHQXw9JdDE13MyzdzbB0F5ly3lqIAUvSoBuXXnope/fuxefzceedd3LLLbfwwQcfcO+99xIKhcjIyODjjz+msbGR22+/nVWrVqGU4r777uPyyy/H4/HEbh7y+uuv88477/DCCy9www034HA4WL16NaeeeipXXnkld955Jz6fD6fTyfPPP8/YsWMJhUL8+Mc/5oMPPgDg29/+NhMnTuSJJ57grbfeAuCjjz7iqaee4s0334zb72kgMpkUmUl2MpO63r6zqKiKwsLTAAiFNVWNrUZQ1/soazBCu7zex77aFjbsq+ODDQc67GU7rWaGprsYmu5iWLo7EtQuhma4yfU6MMlV4EIkrL4bxu/fAwfW97i6MxQE82G6kzMZLvjNYbf13HPPkZaWRktLCzNmzGDu3LncfPPNLF26lOHDh1NdXQ3Ar3/9a5KTk1m/3mhnTU3NYbddUlLCZ599htlspr6+nmXLlmGxWFi8eDH33nsvb7zxBs8++yy7du1izZo1tLS0EAgESE1N5dZbb6WiooLMzEyef/55/uM//uPwvxgRF2aTIityqHoyyd3WCYTC7KtpYXd1M7urmthVaUyLyxtZsqUCfygcq2szmxic5mwL6QxXLKzzUpxy+FuIfq7vhnEcPfHEE7E9zr179/Lss89yxhlnMHz4cADS0ozxrosXL2bRokWx96WmpnbdWCfz58+PPXe5rq6O66+/nm3btqGUIhAIxLb7ne98J3YYO/rzrr32Wv7yl79w4403snz5chYuXNhLPRbxYDWbGJbhZliGG8jssC4U1hyo97G7soldVUZI765qZldVE59tr6IlEIrVNZuMQ+sZHhupLhspLhspLiupLispLqMsNu+2kuqyyQ1RhOhj+m4Y92APtr2WXnqEYlFREYsXL2b58uW4XC4KCwspKChgy5YtPd5G+/N+Pp+vwzq32x2b//nPf86cOXN488032bVrF4WFhYfc7o033sjFF1+Mw+Fg/vz5cs45gUUDNj/FySmjOq7TWlPR0NohpHdXN1Pd1MqBeh9bDjRQ0+yn2R/qfuMY462jwZ3qskbm26bR8p21IYZXNZHmtuGxW+ScthDHiXyad1JXV0dqaioul4stW7bw+eef4/P5WLp0KTt37owdpk5LS+Pcc8/lySef5PHHHweMw9SpqalkZ2ezefNmxo4dy5tvvnnQLwl1dXXk5+cD8MILL8TKzz33XJ555hnmzJkDEPt5eXl55OXl8cADD7B48eLj+4sQfZZSbYfA2982tDNfIERdS4CaZj81TQFqm/3UNBvLdS0BapqM5dpmP1sO1FPbHKC2JdDlavEHPi8CwGYxkeaykea2ke4xpmluG+luG2lue4fydLcNr8Mq57mF6CEJ407OP/98nn76acaPH8/YsWM5+eSTyczM5Nlnn2XevHmEw2GysrL46KOP+NnPfsb3vvc9Jk2ahNls5r777mPevHn85je/4aKLLiIzM5Pp06fHLubq7Ec/+hHXX389DzzwAP/+7/8eK7/pppvYunUrU6ZMwWw28+1vf5vbbrsNgKuvvpqKigrGjx9/Qn4fov9yWM04rOYjetJVOKxpaA3Ggvufn68if8Q4qptaqWryU93op7rJT1WTn91VzVQ3+WlsNy67PbNJkeqKhrWNNI+NNJeNDI9xRXlOsiM2TXLIzVXEwNajMFZKnQ/8N2AG/qy1/k2n9XcBNwFBoAL4D6317l5u6wlht9t5//33u113wQUXdFj2eDy8+OKLXepdccUVXHHFFV3K2+/9AsyePZutW7fGlh944AEALBYLv/vd7/jd735HQ6fD759++ik333xzj/sjxJEwmRTJTivJTitD06F2u4XCaYMO+Z7WYMgI6EhQR8O6uqmV6qZAZOpn8/56qpv81DYHumzDY7d0Cmhnh+VcrxOvUw6Ti8R12DBWSpmBJ4FzgRJgpVLqba31pnbVVgPTtdbNSqnvAg8DC45HgweyadOm4Xa7eeyxx+LdFCFi7BYzuclOcpOdParvD4Ypq/dxoN7H/jofB+paKK31caDOx/56H1vLKihvaI3dTzzKZTO3hbXX2WXvOtvrIM1lk0Pjol/qyZ7xTKBYa70DQCm1CJgLxMJYa72kXf3PgWt6s5HC8OWXX8a7CUIcM5vFxOA0F4PTXAetEwiFqWhoZX9dSySwfe2mLSzfXklZN3dDM5sUGR4bWUkOsiLjwbOS7GRGblua5bVT2RKmNRjCbpErykXfoXTnr5+dKyh1BXC+1vqmyPK1wCyt9W0Hqf8/wAGt9QPdrLsFuAUgOzt7WvthQQDJycmMGjWq89t6JBQKxYYMJZJ496u4uJi6urpe3WZjYyMej6dXt9kXJGK/+nKfQmFNvV9T7TNeda2a2lZjGp2vbdU0+DXdfcq5rZBsVyTbFCl2RbLdFJmq2DTJpnBawNIP9rb78t/qaCVan+bMmfOl1np6d+t69QIupdQ1wHTgzO7Wa62fBZ4FmD59uu48lGfz5s1HPTyp87nVRBHvfjkcDqZOndqr2ywqKjrsMK7+KBH7lQh9CobCVDf5KW9opbzBx7JV60jLHUZ5QysVkbK9Da18WdGKPxjudhtOq5kkhyXyspLksOB1WvFGl+2WWFl0fZLDgjc2bz3uz9FOhL9VZ4nYp4PpSRjvAwa3Wx4UKetAKXUO8FPgTK11a+80Twghjo3FbIoNBYNkTAesFBaO7lJPa029L0hFgy8W1NVNfhp8QRp8ARp8Qepj0yD7aluM+ZYArQcJ8fbcNjNJDivZXjvXzR7G3II8LHLnNBHRkzBeCYxWSg3HCOErgW+1r6CUmgo8g3E4u7zXWymEEMeZUm1Xko/KOrKjUf5gOBbY0fCu9wWob7ccna4rqeOHf13LHz7Zxu1njZZQFkAPwlhrHVRK3QZ8iDG06Tmt9Ual1K+AVVrrt4FHAA/w18jQgz1a60uOY7uFEKLPsFlMpHvspHu6PkCkM601/9hUxn8v3iahLGJ6dM5Ya/0e8F6nsl+0mz+nl9vVL7R/OlNnu3bt4qKLLmLDhg0nuFVCiL5MKcV5E3P4twnZEsoiRv7iQggRB9FQfveO03jm2mm4bBZ++Ne1nPv7pfztqxKCocOfhxaJo8/eDvO3X/yWLdU9fzhDT4YAjUsbx49n/vig6++55x4GDx7M9773PQDuv/9+LBYLS5YsoaamhkAgwAMPPMDcuXN73C4wHhbx3e9+l1WrVsXurjVnzhw2btzIjTfeiN/vJxwO88Ybb5CXl8c3v/lNSkpKCIVC/Od//ic33HDDEf08IUT/0d2e8l2vreUPnxRz+1mjuOQk2VMeCPpsGMfDggUL+P73vx8L49dee40PP/yQO+64A6/XS2VlJSeffDKXXHLJEd2W78knn0Qpxfr169myZQv/9m//xtatW3n66ae58847ufrqq/H7/YRCId577z3y8vJ49913AeP5x0KIxCehPLD12TA+1B5sd3pjPO7UqVMpLy+ntLSUiooKUlNTycnJ4Qc/+AFLly7FZDKxb98+ysrKyMnJ6fF2P/30U26//XYAxo0bx9ChQ9m6dSuzZ8/mwQcfpKSkhHnz5jF69GgmT57MD3/4Q3784x9z0UUXUVBQcEx9EkL0LxLKA5P8RTuZP38+r7/+Oq+++ioLFizg5ZdfpqKigi+//JI1a9aQnZ3d5RnFR+tb3/oWb7/9Nk6nkwsvvJBPPvmEMWPG8NVXXzF58mR+9rOf8ZvfHNlznYUQiaHzOWWn1cxdr8k55UQlYdzJggULWLRoEa+//jrz58+nrq6OrKwsrFYrS5YsYffuI38Y1emnn87LL78MwNatW9mzZw9jx45lx44djBgxgjvuuIO5c+eybt06SktLcblcXHPNNdx9992sXbu2t7sohOhHJJQHhj57mDpeJk6cSENDA/n5+eTm5nL11Vdz8cUXM3nyZKZPn864ceOOeJu33nor3/3ud5k8eTIWi4UXXngBu93Oa6+9xksvvYTVaiUnJ4d7772XlStXcvfdd2MymbBarTz66KPHoZdCiP7mcIevXcFDP2dA9G0Sxt1Yv359bD4jI4Ply5d3W+9gY4wBhg0bFhtj7HA4eP7557vUueeee7jnnns6lJ133nmcd955seWGhoYjarsQIrEdLJQB8ld9wqgsD6OyPIzO8jA628OozCSSXdY4t1ocjoSxEEL0Q+1Dedm2St5atpqwO5Vt5Y2s2FmFL9B2+Dozyc6ozEg4Z7W9Mj32IxoZIo4fCeNjtH79eq699toOZXa7nRUrVsSpRUKIgUQpxRljMgmX2igsNJ6wFg5r9tW2sK28geLyRraVNVJc0cibX+2joTUYe69xH25jL7p9SOclOzH1g8dGJhIJ42M0efJk1qxZE+9mCCFEjMmkGJzmYnCai7PGZcfKtdaUN7Qa4VzewLbyRorLG/loUxmLVu6N1XPZzIzM9JCX4sBuMWOzmLBbTJFp23JbmTG1mc1dyjrXt1vNJNktEvadSBgLIcQAoZQi2+sg2+vgtNEZHdZVN/kpjoRzdI96V2Uz/lCY1kAoMg3TGgof9LnPPWU2KVJdNtLcVtLcNtLddtLcNlLdNtLdtkiZjb0NYcrrfaS6bVgTfGy1hLEQQgjS3DZmDk9j5vC0w9bVWuOPhHJrsPM01GW5fR1fIERNs5/qJj9VjX5qmv1sPlBPdZOf2uZAl5/18399DIDXYSHdYyfVZSXNbTdC22MjzWWEd7LTSpLDQpIjOrXgsVv6zQ1SJIyFEEIcEaUUdosZu8XMsd33sKNgKExtSyAW1MtWriZ36CiqmwJUN7VS1WSEeElNM+tKaqlu8hMMH3pIl9NqNoI5GtR2S7uwbgvuaJB77G3zXoeFLK+jF3t4cBLGQggh+gSL2USGx06Gxw7Z0LrXQuHsYQetr7WmoTVIdaOfel+ABl+Qhtg0SGNru+XWSJkvQFm9L7a+sd0FbZ0l2S2s/+V5B13fmySMj8GhnmcshBDi+FJK4XVY8TqOfhx1KKxp8gdjQd4YCfKG1uAJvbuZhHECCAaDWCzypxRCiCNlNrUPdGfc2tFnP8EPPPQQrZt7/jzjYChE9WGeZ2wfP46ce+896PrefJ5xY2Mjc+fO7fZ9Cxcu5NFHH0UpxZQpU3jppZcoKyvjO9/5Djt27ADgj3/8I3l5eVx44YVs2rQJgEcffZTGxkbuv/9+CgsLKSgo4NNPP+Wqq65izJgxPPDAA/j9ftLT03n55ZfJzs6msbGR22+/nVWrVqGU4r777qOuro5169bx+OOPA/CnP/2JTZs28fvf//7wv2ghhBC9rs+GcTz05vOMHQ4Hb775Zpf3bdq0iQceeIDPPvuMjIwMqqurAbjjjjs488wzefPNNwmFQjQ2NlJTU3PIn+H3+1m1ahUANTU1fP755yil+POf/8zDDz/MY489xq9//WuSk5Njt/isqanBarXy4IMP8sgjj2C1Wnn+MTV6MgAAC2tJREFU+ed55plnjvXXJ4QQ4ij12TA+1B5sd/ra84y11tx7771d3vfJJ58wf/58MjKMMX5pacYwgk8++YSFCxcCYDabSU5OPmwYL1iwIDZfUlLCggUL2L9/P36/n+HDhwOwePFiFi1aFKuXmpoKwFlnncU777zD+PHjCQQCTJ48+Qh/W0IIIXpLnw3jeIk+z/jAgQNdnmdstVoZNmxYj55nfLTva89isRAOt11A0Pn9brc7Nn/77bdz1113cckll1BUVMT9999/yG3fdNNNPPTQQ4wbN44bb7zxiNolhBCid/WP0dAnUG89z/hg7zvrrLP461//SlVVFUDsMPXZZ5/NH//4RwBCoRB1dXVkZ2dTUVFBVVUVra2tvPPOO4f8efn5+QC8+OKLsfJzzz2XJ598MrYc3due9f+3d/+xcdd1HMef77XXXrciGw4nWoKAzmRjqaxLQWHQZA4HMasaN7aYWAUloCQCMYNkhCz8JRoxU4kGgcgI6BTFLWaEH9pGF2VSlgFjwDbmTEu2Mbpfjq4/rn37x/fbeb3dtde1ve/3e309kku/9/l8vnefdz93977v5/u97/eKK+jo6OCpp55i9erVxf57RERkEigZ58h3PeP29nYWLFjAhg0bir6ecaH15s+fz9q1a7n22mupr6/nrrvuAmD9+vW0trayYMECGhoa2LVrF6lUirvvvpvGxkaWLl064nOvW7eOFStW0NDQcHoKHODee+/l6NGjXHbZZdTX19Pa2nq6buXKlVx11VWnp65FRCQamqbOYyKuZzzSei0tLbS0tAwrmzNnDps2bTqj7W233caaNWvOKG9raxt2v7m5Oe9R3rW1tcO2lLNt3bqVO++8s1AIIiJSItoynoKOHTvG3LlzqampYcmSJVF3R0RkytOW8Tgl8XrGM2fOZPfu3VF3Q0REQkrG46TrGYuIyHhpmlpERCRiSsYiIiIRUzIWERGJmJJxjtra2qi7ICIiU4yScREymcIXnxYRERkvJeMC2traWLx4McuXL2fevHlRd0dERMpYbH/a9Pff7eb9jsJnuMo1MDBAxSjXM559YS2LV84t+jG3b9/Ozp07T18BSUREZDJoy3gEjY2NSsQiIjLpYrtlPJYtWJiY6xnnyr5EoYiIyGTRlrGIiEjElIxFREQiFttp6qgMXRaxqamJpqamaDsjIiJTgraMRUREIqZkLCIiEjElYxERkYjFLhm7e9RdkJDGQkSkNGKVjNPpNF1dXUoCMeDudHV1kU6no+6KiEjZi9XR1HV1dXR2dnL48OExr9vT01OWiSPKuNLpNHV1dZE8t4jIVFJUMjazZcB6oAJ4xN1/kFNfDWwAGoAu4EZ33z/WzqRSqbM6/WTnW0do37KD888/f8zrxt3hw0cjjOsk71TtoXp6JdU1lVRPT1E9vZKqmsqgbHpYVlNJKl2BmUXUTxGRZBs1GZtZBfAQsBToBF42s83uviur2c3AUXf/pJmtAh4AbpyMDufTeypD73E4kuku1VOWTO8HEcblTqZvkN7ufvp6BkZsagZVWUm7qqaS9PTKYWVDifxEp7Nvx9hnP+KuHONSTBPPphmpqmmk0pVUpStIVQ/9rcCm6QvtVFXMlnEjsNfd9wGY2W+BZiA7GTcD68Llp4Gfm5l5iXb+Xnr5R+g4Po2mpitK8XQl1dbWFou4BgedvlMZersz4d/+4EtQd1bZB2HZqQy9H2Q4crCbvrBdpm9w2ON1bH09okgmVznGpZhKp7K6gqrqClLpCqrSlaSqK4JEnQ5mn4K6/yfvqqHydAXd7zsH9x2POoQJFXVMZsaciz9UkucqJhl/HOjIut8J5GaH023cPWNmx4EPA+9PRCcletOmGekZKdIzUme1/kD/IL2ngqT9z39sY1HDognuYfTaX2kvu7gU08Rzd/p7BujrHaC/J0Nfz0B4P0N/T1jWG5b1ZDh5rJe+nm76e4L6TP9gwcf+94uvlDCS0ogypurplXzrwWtK8lwlPYDLzG4BbgnvnjSztyfw4WdTnsm/HOMqx5igPONSTMlRjnFFHtO3fzKhD3dRoYpikvG7wIVZ9+vCsnxtOs2sEjiX4ECuYdz9YeDhIp5zzMys3d3L6ys85RlXOcYE5RmXYkqOcoyrHGMqpJjfGb8MfMrMLjazKmAVsDmnzWagJVz+KvDXUu0vFhERSbpRt4zDfcC3A88R/LTpMXd/w8zuB9rdfTPwKPCEme0FjhAkbBERESlCUfuM3X0LsCWn7L6s5R5gxcR2bcwmZfo7BsoxrnKMCcozLsWUHOUYVznGlJdpNllERCRasTo3tYiIyFSUuGRsZsvM7G0z22tm9+SprzazjWH9NjP7ROl7OTZmdqGZtZrZLjN7w8y+l6dNk5kdN7Md4e2+fI8VJ2a238xeD/vbnqfezOyn4Vi9ZmYLo+hnsczs01n//x1mdsLM7shpk4hxMrPHzOw9M9uZVXaemb1gZnvCv7MKrNsSttljZi352kShQEw/MrO3wtfXM2Y2s8C6I75Wo1QgrnVm9m7W6+yGAuuO+HkZlQIxbcyKZ7+Z7SiwbmzHalzcPTE3ggPI3gEuAaqAV4F5OW2+A/wyXF4FbIy630XEdQGwMFw+B9idJ64m4M9R93WMce0HZo9QfwPwLGDAlcC2qPs8htgqgIPARUkcJ+AaYCGwM6vsh8A94fI9wAN51jsP2Bf+nRUuz4o6nhFiug6oDJcfyBdTWDfiazWGca0Dvj/KeqN+XsYpppz6HwP3JW2sxnNL2pbx6VNzunsfMHRqzmzNwOPh8tPAEov5FQzc/YC7bw+X/wu8SXBWs3LXDGzwwEvATDO7IOpOFWkJ8I67/yfqjpwNd/8bwS8fsmW/dx4HvpRn1S8AL7j7EXc/CrwALJu0jo5Bvpjc/Xl3z4R3XyI4T0KiFBirYhTzeRmJkWIKP69XAr8paacilrRknO/UnLlJa9ipOYGhU3MmQjitfjmwLU/1Z83sVTN71szml7RjZ8eB583slfDsa7mKGc+4WkXhD4ukjdOQOe5+IFw+CMzJ0ybJY3YTwUxMPqO9VuPo9nD6/bECuxSSOlaLgUPuvqdAfRLHalRJS8ZlzcxqgT8Ad7j7iZzq7QRTovXAz4A/lbp/Z+Fqd18IXA9818xKc5LXSRae/GY58Ps81UkcpzN4MB9YNj+1MLO1QAZ4skCTpL1WfwFcCnwGOEAwrVsuVjPyVnHSxqooSUvGYzk1JzbCqTnjxsxSBIn4SXf/Y269u59w95Ph8hYgZWazS9zNMXH3d8O/7wHPEEybZStmPOPoemC7ux/KrUjiOGU5NLSbIPz7Xp42iRszM/sG8EXga+GXjDMU8VqNFXc/5O4D7j4I/Ir8/U3iWFUCXwE2FmqTtLEqVtKScVmemjPcR/Io8Ka7P1igzUeH9n2bWSPB2MX2S4aZzTCzc4aWCQ6k2ZnTbDPw9fCo6iuB41nTpHFW8Jt70sYpR/Z7pwXYlKfNc8B1ZjYrnBq9LiyLJTNbBqwBlrt73guDF/lajZWcYyu+TP7+FvN5GTefB95y9858lUkcq6JFfQTZWG8ER+DuJjhKcG1Ydj/Bmw0gTTB9uBf4F3BJ1H0uIqarCaYEXwN2hLcbgFuBW8M2twNvEBwR+RLwuaj7PUpMl4R9fTXs99BYZcdkwEPhWL4OLIq630XENYMguZ6bVZa4cSL4MnEA6CfYl3gzwbEVfwH2AC8C54VtFwGPZK17U/j+2gt8M+pYRolpL8F+06H31dAvLT4GbBnptRqXW4G4ngjfM68RJNgLcuMK75/xeRmHW76YwvJfD72XstomZqzGc9MZuERERCKWtGlqERGRsqNkLCIiEjElYxERkYgpGYuIiERMyVhERCRiSsYiIiIRUzIWERGJmJKxiIhIxP4HfiCO7jHk9YEAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "o6wTDcroztmh", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "fdababc5-40a5-4777-ce6b-8a9934847bf7" }, "source": [ "y_pred = model.predict(X_test)\n", "y_pred = y_pred.argmax(axis=1)\n", "print(classification_report(y_test, y_pred,\n", " target_names=class_names))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " T-shirt/top 0.84 0.83 0.84 1000\n", " Trouser 0.99 0.98 0.98 1000\n", " Pullover 0.84 0.81 0.83 1000\n", " Dress 0.91 0.90 0.91 1000\n", " Coat 0.83 0.85 0.84 1000\n", " Sandal 0.98 0.97 0.97 1000\n", " Shirt 0.72 0.74 0.73 1000\n", " Sneaker 0.94 0.97 0.95 1000\n", " Bag 0.98 0.97 0.97 1000\n", " Ankle boot 0.97 0.94 0.96 1000\n", "\n", " accuracy 0.90 10000\n", " macro avg 0.90 0.90 0.90 10000\n", "weighted avg 0.90 0.90 0.90 10000\n", "\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "vKeKY-j8zwiQ" }, "source": [ "# 6. Other Regularization Strategies\n", "\n", "Other strategies to prevent overfitting" ] }, { "cell_type": "markdown", "metadata": { "id": "9ERvEyyq0D-A" }, "source": [ "## 6.1 $\\ell_1$ and $\\ell_2$ regularization" ] }, { "cell_type": "code", "metadata": { "id": "TrvETrA5z4h1", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "ee89dd85-c51a-42ea-a3fc-77e3683817e0" }, "source": [ "from functools import partial\n", "\n", "RegularizedDense = partial(tf.keras.layers.Dense,\n", " activation=\"elu\",\n", " kernel_initializer=\"glorot_uniform\",\n", " kernel_regularizer=tf.keras.regularizers.l2(0.001))\n", "\n", "model = tf.keras.models.Sequential([\n", " tf.keras.layers.Flatten(input_shape=[28, 28]),\n", " RegularizedDense(300),\n", " RegularizedDense(100),\n", " RegularizedDense(10, activation=\"softmax\")\n", "])\n", "model.compile(loss=\"sparse_categorical_crossentropy\", \n", " optimizer=\"adam\", metrics=[\"accuracy\"])\n", "\n", "lr_scheduler = tf.keras.callbacks.LearningRateScheduler(exponential_decay_fn,\n", " verbose=1)\n", "history = model.fit(X_train, y_train, epochs=20,\n", " validation_split=0.1,\n", " callbacks=[lr_scheduler])" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "\n", "Epoch 00001: LearningRateScheduler reducing learning rate to 0.01.\n", "Epoch 1/20\n", "1688/1688 [==============================] - 4s 3ms/step - loss: 1.0361 - accuracy: 0.7427 - val_loss: 0.8779 - val_accuracy: 0.7858\n", "\n", "Epoch 00002: LearningRateScheduler reducing learning rate to 0.008912509381337455.\n", "Epoch 2/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.8286 - accuracy: 0.7866 - val_loss: 0.7501 - val_accuracy: 0.8138\n", "\n", "Epoch 00003: LearningRateScheduler reducing learning rate to 0.007943282347242816.\n", "Epoch 3/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.7788 - accuracy: 0.7978 - val_loss: 0.8271 - val_accuracy: 0.7565\n", "\n", "Epoch 00004: LearningRateScheduler reducing learning rate to 0.0070794578438413795.\n", "Epoch 4/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.7440 - accuracy: 0.8018 - val_loss: 0.7839 - val_accuracy: 0.7862\n", "\n", "Epoch 00005: LearningRateScheduler reducing learning rate to 0.006309573444801933.\n", "Epoch 5/20\n", "1688/1688 [==============================] - 4s 3ms/step - loss: 0.7251 - accuracy: 0.8068 - val_loss: 0.6761 - val_accuracy: 0.8152\n", "\n", "Epoch 00006: LearningRateScheduler reducing learning rate to 0.005623413251903491.\n", "Epoch 6/20\n", "1688/1688 [==============================] - 4s 3ms/step - loss: 0.6838 - accuracy: 0.8143 - val_loss: 0.7351 - val_accuracy: 0.7958\n", "\n", "Epoch 00007: LearningRateScheduler reducing learning rate to 0.005011872336272724.\n", "Epoch 7/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.6569 - accuracy: 0.8187 - val_loss: 0.6765 - val_accuracy: 0.8128\n", "\n", "Epoch 00008: LearningRateScheduler reducing learning rate to 0.004466835921509631.\n", "Epoch 8/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.6420 - accuracy: 0.8214 - val_loss: 0.7063 - val_accuracy: 0.8020\n", "\n", "Epoch 00009: LearningRateScheduler reducing learning rate to 0.0039810717055349725.\n", "Epoch 9/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.6258 - accuracy: 0.8241 - val_loss: 0.6044 - val_accuracy: 0.8317\n", "\n", "Epoch 00010: LearningRateScheduler reducing learning rate to 0.003548133892335755.\n", "Epoch 10/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.5996 - accuracy: 0.8290 - val_loss: 0.6113 - val_accuracy: 0.8303\n", "\n", "Epoch 00011: LearningRateScheduler reducing learning rate to 0.0031622776601683794.\n", "Epoch 11/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.5827 - accuracy: 0.8324 - val_loss: 0.5997 - val_accuracy: 0.8242\n", "\n", "Epoch 00012: LearningRateScheduler reducing learning rate to 0.002818382931264454.\n", "Epoch 12/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.5679 - accuracy: 0.8362 - val_loss: 0.5715 - val_accuracy: 0.8340\n", "\n", "Epoch 00013: LearningRateScheduler reducing learning rate to 0.0025118864315095803.\n", "Epoch 13/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.5542 - accuracy: 0.8392 - val_loss: 0.5609 - val_accuracy: 0.8393\n", "\n", "Epoch 00014: LearningRateScheduler reducing learning rate to 0.0022387211385683395.\n", "Epoch 14/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.5452 - accuracy: 0.8411 - val_loss: 0.5617 - val_accuracy: 0.8333\n", "\n", "Epoch 00015: LearningRateScheduler reducing learning rate to 0.0019952623149688802.\n", "Epoch 15/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.5326 - accuracy: 0.8444 - val_loss: 0.5229 - val_accuracy: 0.8492\n", "\n", "Epoch 00016: LearningRateScheduler reducing learning rate to 0.001778279410038923.\n", "Epoch 16/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.5210 - accuracy: 0.8483 - val_loss: 0.5356 - val_accuracy: 0.8407\n", "\n", "Epoch 00017: LearningRateScheduler reducing learning rate to 0.0015848931924611134.\n", "Epoch 17/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.5121 - accuracy: 0.8507 - val_loss: 0.5194 - val_accuracy: 0.8518\n", "\n", "Epoch 00018: LearningRateScheduler reducing learning rate to 0.0014125375446227546.\n", "Epoch 18/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.5001 - accuracy: 0.8561 - val_loss: 0.4913 - val_accuracy: 0.8547\n", "\n", "Epoch 00019: LearningRateScheduler reducing learning rate to 0.0012589254117941673.\n", "Epoch 19/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4955 - accuracy: 0.8549 - val_loss: 0.5040 - val_accuracy: 0.8545\n", "\n", "Epoch 00020: LearningRateScheduler reducing learning rate to 0.0011220184543019637.\n", "Epoch 20/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4881 - accuracy: 0.8579 - val_loss: 0.4988 - val_accuracy: 0.8543\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "1qRL_89j0TyW", "colab": { "base_uri": "https://localhost:8080/", "height": 324 }, "outputId": "5da162d5-f652-458d-cda5-e15c8c2d6fff" }, "source": [ "pd.DataFrame(history.history).plot(figsize=(8, 5))\n", "plt.grid(True)\n", "plt.gca().set_ylim(0, 1);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeMAAAEzCAYAAAACSWsXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3wUdf7H8dds3002mx7SC51A6EVEmqBIUVDRU85yeuKd4v2woWe/U2yod7bz7O1EsQAKYsGCioVOEhKKkBBCet20TbbN74+NIRECAQKB5PN8POYxs1O/3yXknZn5zncUVVURQgghRMfRdHQBhBBCiK5OwlgIIYToYBLGQgghRAeTMBZCCCE6mISxEEII0cEkjIUQQogOdsQwVhTlNUVRihVF2dbKckVRlGcURdmtKEqaoihD2r+YQgghROfVljPjN4Aph1l+HtCzcZgLvHD8xRJCCCG6jiOGsaqq3wPlh1nlAuAt1ecXIFBRlMj2KqAQQgjR2bXHPeNoILfZ5/2N84QQQgjRBrqTeTBFUebiu5SN2WweGhsb22779nq9aDS+vy0q6lXsTpVYqwat0m6H6BDN69VZdMY6Qeesl9Tp9NEZ69XZ6rRr165SVVXDDrlQVdUjDkACsK2VZS8ClzX7vBOIPNI+hw4dqranb7/9tmn616IqNf6OlerL3+9p12N0hOb16iw6Y51UtXPWS+p0+uiM9epsdQI2qq1kYnv8yfEJcGVjq+pRgF1V1YJ22O8x6xFuZVBsIB9s3P/bHwhCCCHEKastjza9C/wM9FYUZb+iKNcqivIXRVH+0rjKKiAL2A28DNxwwkp7FGYPi2FnUTXpefaOLooQQghxWEe8Z6yq6mVHWK4CN7ZbidrJjIFR/HNFJh9s3E9KTGBHF0cIIYRoVee5M/47ASY9U/p34+OtedS7PB1dHCGEEKJVnTaMAWYPjaWq3s3qzKKOLooQQgjRqk4dxqO7hxAdaOaDTfs7uihCCCFEqzp1GGs0ChcNieaHX0sosDs6ujhCCCHEIXXqMAa4eGgsqgpLN+d1dFGEEEKIQ+r0YRwXYmFkYjAfbMyVZ46FEEKckjp9GAPMHhbL3rI6NuZUdHRRhBBCiIN0iTCeOqAbfgYtH2zMPfLKQgghxEnWJcLYYtAxLSWST9MKqHO6O7o4QgghRAtdIozBd6m61ulhVXphRxdFCCGEaKHLhPGw+CASQ/3kUrUQQohTTpcJY0VRuHhoDOuyy9lXVtfRxRFCCCGadJkwBrhwSDQaBT7cJGfHQgghTh1dKowjbWbG9Azjo815eL3yzLEQQohTQ5cKY4DZQ2PIq3Tw056yji6KEEIIAXTBMJ7cL4IAk44P5FK1EEKIU0SXC2OTXssFg6L5fFshdoero4sjhBBCdL0wBpg9LIYGt5eVafkdXRQhhBCia4bxgGgbvSOsfLBR3nMshBCi43XJMFYUhdnDYtiaW8nu4uqOLo4QQogurkuGMcDMwdHoNIqcHQshhOhwXTaMQ/2NTOgTztItebg93o4ujhBCiC6sy4Yx+J45Lqlu4LtdJR1dFCGEEF1Ylw7jCX3CCfU3yKVqIYQQHapLh7Feq2HmoGi+3lFEea2zo4sjhBCii+rSYQy+9xy7PCrLt+R1dFGEEEJ0UV0+jHt3s5ISY+ODTXKpWgghRMfoFGG8q2IXi8sW4/a6j2n72UNj2F5QxbY8ezuXTAghhDiyThHGOVU5/FzzM+/uePeYtj9/YDQGnYYP5exYCCFEB+gUYTwpbhL9TP14bstzFNYWHvX2Nouec/pFsHxrHg1uzwkooRBCCNG6ThHGiqIwO3g2HtXD4xseP6Z9zB4WS2Wdi6+3F7dz6YQQQojD6xRhDBCqD2VuylxW56zmh/0/HPX2Y3qEEmkz8cFGec+xEEKIk6vThDHA1clXkxCQwMPrHqbeXX9U22o1ChcOiea7XSUUVR3dtkIIIToBrxecdVBbBpW5ULbnpB1ad9KOdBIYtAbuHXUv1355LS+nv8xNg286qu0vHhrL89/uYenmPP46vvsJKqUQQnRS7gZwVKDWlqPai/FWlaJWleKtKkOtrkCtqcTrqPGtq2hB4xuUxjGKBjS6pvkxhYXU7f28aZ6i0TQtQ9GBVgtofGPVC+56Xxnc9eBpAHcDits3xvP7Zc3GjdspnoaW9TFZMT6y96R8dZ0qjAFGRI5getJ0Xtv2GtOSppFkS2rztomhfgyLD+KDTbn8ZVwSiqKcwJIKIcQpwusFTwOqsw53Xi7Ovdk49+XiLilCra3CW1uDWleDt96B6nDgbWhArW/A63ShOl14XV5UlxevB1SPguppv9+dOWxut321ztQ4tKTxM9H7kZNweDphGAPcOuxWvtv/HQt/Wcgr57xyVKE6e1gMd3yUzuZ9lQyNDzqBpRRCdAWqy0XD7t04tm2jPiMD175ctCEh6LtFoAuPQNctAn23br7p0BAUrfbAxl4POGugoQacteCs9o0P8Tkpaxc4Pmt5dniIsep04LI34Cp34qxw47SrOKsUnDVaXDU6VG/L35eKRkXRqmi0jWOdgmLQotHr0Jr0KDYrGpMJxWRCY/ZDMVvQ+FlR/ALQ+AWg+AeiCQhG8Q9CYzajGIygKKCqv31DB6Ybx6pXBbykb93CgOR+vrNej9s39iU+eN3g9aJ6G6c1etDqQWs8MP3b7/7f9qs2PxYtljXNaF4W3cmLyE4ZxqHmUOYPmc+DvzzIp9mfMj1pepu3nZYSxQOfZPLhplwJYyFE23g94KpDdVTTsHM79RkZOLbvpH5nFg3Z+1Fdvg6JNGYDhggbzp2pVFXWgUdtuR8FdH6gt3jRmdzoTU50Fg86s8c3z+xBb/agaA8uQoyih2IL6IyoGiOuegPOah3Oag0uOzgrPTgrXbgqnKgeFd+vfx2KXoshzIohyYZ/ZAiGbiEYosIxxESii+iG4h8C5iAwBYI5EHTGE/51/qbB7cF//PiTdryO1CnDGODiXhezfPdyFm1YxFnRZ2Ez2tq0nb9Rx3kDurEitYD7pidjNhzip14IccpQ3W68leVojBoUDY33B53gcfnuCf427W4Aj4uQ0k2QUdG43Nk037de88EFLge46nxnn6463+fGabW+loaSeuqLXdSXKjjK9TRU6psu0Wr0XkxBLoK6uzAFOTEHu9D7e3wna4oGVW/Fo/rhajDjbjDirtfjqtXgrlVxVbtpqHJSU1SP2nBwz4LaoAD0YWHoIsLRdYtC1y2SnIwMIjxenPv24dy/31fWRorJhCEuDuPgOKzx8ejj4jDEJ2CIj0MXHu67Fys6VKcNY42i4d5R9/KHT//As1ue5Z5R97R529lDY1m6OY/PMwqYNTjmBJZSiE5OVX0h1lDju9zqrPUNbkdj0PnCTnU68Fbb8djteO12PNW1eKpr8NTU4alx4K1rwFPrxONw4nG48dZ78dSreBrA6zpwWVWj86I1Ng6GltO6xukko5f6Hw4s0xzq722twTfojKD3Q9WacNYYcZRqqC/xUF+oUl+ooLqMgBGNUYcpPoyg4VGYesZh6pWIITYWxeQPegsY/EBvBr0fGCygM6EoSuO56eG+PhVvTQ3uwkJcRcW4iwpxFRbiLirGVVSIq7AIR/p2PJWVWAwGXImJGHv2xDrp7MbAjccQH+8LXGkDc0rrtGEM0DekL5f3uZx3tr/DBd0vYEDYgDZtNzIxmNhgMx9s3C9hLLoOr+fAGWDjOMC+HXa7f3efsvm4WcD+PnCdNXjr6nDVanDVanHWanHV6nDXafA4fYPXpcHToOBxaUA9TFhoQGvWojXr0Jot6EKNGCxGtP5mtP4WNP4WVBe4a52+0K6px1NTj7PGgaekDm9d648rKmYTWpsNXVAQ2sBAtMEhvnFAEJ4qO/VbM6jfvh3VUeZb32LB1LcvQROTMfXvjyk5GUNCwgk5u1QUBa3VitZqxdizZ6vreZ1Ovv/xR8ZPmNDuZRAnR6cOY4AbB93Il3u/5MFfHmTxtMXoNEeuskajcPGQWP711S5yy+uIDbachJKKzsxTXU3Vqs+wL1uGIy0NtFqU3wadDnQ6FJ3O13in1Wktik7vm9ZofA1rFC8KXhSNF43ZgNZPj86ibQwupensT6d3o3jqGoO2Dly1jeO6A+H5+8c6gCEAWw5RIUUDBitejR+uBgsuhwFXrQ5XjYKzyoLLrsdVYcZT03Kfik6LLjQIrc0XMAabDU1gIFpbINqgELSBIWgCbWhtjUNAAFqbDcVsPq4zO9XpxGO3466oYNOaNQyIT8BTWYmnoqLF2F1ZgTMvH09lJd6qKhSzGVOfPgRefDHm/sm+4E1MbNnI6hSgMRgONFYSp6VOH8b+Bn8WjFjAbd/dxpKdS5jTd06btrtoaDRPf72L69/exOMXp9A/um33nI/kzYw3ybZnMzdlLlH+Ue2yT3FqUr1e6n75hcply6levRq1vh5Dj+6E/OlPoHhRG+qgwYHaUIfqrAenA7WhHtXVAK4GVFctqsuJ6nCBy4XqdqN63KhuD6rX62v1qvoamKpepelMszUaQ+PZpUWHzk+P1t+I1mpDa/VDG+CPLjAAbaANbXAQ2uBQtIHBpG37lT4xPXCV1eAqteMqrsBZWIKroBBXXj6esjKgrnEARa9HHxWFPjYa06ho9NEtB11YaIfcn1QMBnRhYejCwnDl5xPQhkZBqsvl+6PnFAte0Tl1+jAGOCf+HM6MOpNntzzL5PjJhFvCj7hNTJCF//5xKHcv38b5z63l2jGJ3Dy5FxbDsX1lqqry3NbneCntJRQUVuxZwZy+c/hzyp8JMAQc0z5FO/K4fI9+uOob72ce/bh3bjaUvo2z2I59cxGVaRW4qzxojGDroSGwhxdTYBpK/TpfAyHw9YFnbhya0+jBZDvEENBsOtA3Ngb45hv8UTVGPLVu3HUNeKob8Nir8FRW4KmowF1Rgaf8wHR9cQWenWWojlbeVqbRYPB6yWo2qylso6MxTZx4yoTtiaDo9R1dBNGFdIkwVhSFu0bexayPZ7FowyIWjVvUpu3OSe7GyKQQHvt8By//kM2q9EIWzurP+N5HDvPmVFXl35v/zWvbXuOinhcxN2Uuz299njcy3mDp7qVcn3I9l/a+FIPWcCzV6xo8LmiohoYq37i+cdxQDQ123/1Kl6NZQDYOLsfvxq0sU4/jbV06M15M6LN05Ow1UJfve1zFL9FMxNnB+PcLQ2PxA52psRGPpZWgbTboTMd02VGBIzYK+j2vw9F0qdZdUYGnohJPeTmeygr27s+j11lndcqwFeJU0iXCGCAuII7rUq7j+a3PM6vHLEZHjz7iNp6aGgL8/Hh41gBmDorm70vTuPr1DVwwKIp7p/cj1P/Iz9upqsoTG5/grcy3uLT3pdw18i40ioaFYxZyZb8reXLjkzy+4XHe2f4O84fM59yEc0+JVo+u4mKqV69G4+eH3/Dh6KOjj2+H7gaoKYaaYoLLNkJ6aSvBWnVgfvNlbseRjwG+B/71JtCZG8emAyFo8Ae/MF8L2ablzcY6o2+939Y/aL2W26haE46t26hctoyqL75AravDEB9H2M0XYrvgfPTduh3fd3aSaMxmNGYz+sjIg5ZlrFmDrYs85ylER+oyYQxwTf9r+DTrUx5a9xBLz1+KSXdw92fuigqqv/gC+4qVODZtwjJsGFFPLGJEYjdW/d9ZvLBmD//5dg9rdpZw97S+zB4a02p4qqrKYxse453t73B5n8u5c8SdLdbtHdybl855iZ/yfuLJTU9y+/e381bmW9w67FaGRgw9Yd9Da7z19VR/9TX2jz+m9scffV3kNdJHRWEZPhzLiOFYhg9HHxuLAlBXDjVFjUNxs+nfzXNUNO0rBSC92YEVDRitvsutxgDftCUUgpMOnm/yjb2qAcfeUup/3Y9jRxYN2TnoQsN8j3OExWGIi0Uf6xtrbe1zv/83rrw8Kj9+F/uy5bhyc9H4+WGbNpWs+ARGX3vNKfHHlBDi9NKmMFYUZQrwNKAFXlFV9dHfLY8D3gQCG9e5U1XVVe1c1uNm0Bq4e9TdXPfldby67VVuHHQj4LtMV/Ptt9hXrKRm7VpwuTD06E7QlVdQ+eFHZF8wk8iHF2I9+2zmT+rF9JRI/r40nQUfprFscx4PXziAxFC/Fsfyql4eXvcwS3Yu4cp+V3LbsNta/SU9Ono0IyNHsiJrBc9ueZarP7+aCbETuHnozSf8O1FVFcemTVQuX07151/gralB1y2ckDkXYhs7GLW6hLrNqdRl7KHmq1XYP/4Y8PUSZAmtxxJWjyW8AYPVc+Cqqs4M1gjwj4DQnpBwlm/aPxz8I9i0M4ehZ0xoDFqr7xnMwwSY6vXizMrCkZqGIzUVR1oaDb/+Ch7fpWV9XBym3r1wl1dQ88P3eEpKW2yvtdl8IR0biz4+DkNjSOvj4tCFhbUpPL0OB9VffUXl0qXU/bIOVBXLqFGEzbsR6+TJaCwWdq5ZI0EshDgmRwxjRVG0wPPAZGA/sEFRlE9UVc1stto9wPuqqr6gKEo/YBWQcALKe9xGRY5iauJUXkt9hSnFkZi/2UD1l1/iratDFx5O8BVXYJsxHWOfPiiKQtBll5F3663sv3EeQXPmEL7gdnqEW1ky9wze25DLI59t59x/f8//nd2T685KwqDT4FW9/PPnf/LRrx9xTf9rmD9k/hF/SWs1Wmb2mMm5Cefyv8z/8eq2V5n18SzO8DuD/o7+hJpD21ZBjwvq7Y1DJTgqD0z/Nt9RiTO/CPvGXOxplbjsHjQ6FWtcA7bhNVjC81FcW+Fr3y5NQHAfDeqQMJzOYGpLzNTle6nNrqIqx/f8pjbIhmXwACwjz8DvjDEYevZstc7VBWsgrFerVXCXl/tCNzWV+rQ0HGnpeGt8b3rRWK2YU1KwTpyAeeBATCkp6IJadlvqravDmbsf574cXPtycebuw7VvH460NKo+/7zFGb9iNmOIifGFdVyzM+r4OPSRkTjS07EvXUbVZ5/hralBHx1N6I03Yps5E0PMcV66F0KIRm05Mx4B7FZVNQtAUZT3gAuA5mGsAr81CbYB+e1ZyPaiqir1GZn89TsjMz9uwFlzN25/f6znTcE2YwaW4cMPeozBmJhIwnvvUfLkk5S/+RZ1GzcS/dSTGLt35/KRcUzqG84/VmSy6IudfLI1n4UX9mN57r/5eM/HzE2Zy7xB8458tuRx+zp8b6jB7KzhuqBBXDj47/x37wo+KP6RaR+ewzUByVxhjsfiqm/sOL7xnupvncg3VPuC1lXb+mGcClX7/bDn+OMo8jXC8Uu0EDYpEuuQBDS20JatdE023z1W/wjwC0XRaPH1NwTBjd+nM3svdRs2NA3V36wFFqENCsIybFjTpW1jr16HbPjjdTppyMzEkZbWdObr2t/Yulerxdi7FwHTp2EeOAjzwJQ2da6gsVgw9e6FqffBga+6XLjy831dBu7b1xjWuThz9lK7di1qQ7PnYhs7s1fMZgLOOQfbhRdiGT5MGjAJIdqdoqrq4VdQlIuBKaqq/rnx8xXASFVV5zVbJxL4EggC/IBJqqpuOsS+5gJzASIiIoa+99577VUPampq8Pf3P+QybUkJpvUbMK1fj66oCFWnI793N97rlU+/kVcwJHBUm45hSE/H9uZbKE4nVZdeQv3o0U2XV7cUu3kr04Ej+AP0ti1Mtp7HBYHnYKovwezIx+woaBoMznJ0bgdaT+PgdbZ6zL06Hf8ODuRrPwthbg9/rapnar0WVWfGozXh0ZrxaM24dWbcOn/cOr+WY8WMN7sE3abt6LftQHG5cHfrhmPUKOpHjsAb1I4vw1BVtKWl6Hf9iuHXXRh2/Yq2vBwAr8WCs0cPXL16UmcwYC0oRJ+djW7/fhS3r+9dT1AQroQEXImJuBITcMXHg+EktjD3etHYq9CWFKMtKUVbWoInNIyGoUNQTQe3L/i9w/0Mnq6kTqePzlivzlanCRMmbFJVddihlrVXGN/SuK8nFUU5A3gV6K+qqveQOwWGDRumbty48ehr04o1a9YwvlmrT3d5OVWrPqNqxQocqakAWIYPJ2DGdALOPRes/sxZNYeiuiI+mfkJVoO1TcdxFRWTf8cd1P3yCwFTz6PbvXej9ZTjLvuVO9Jf4svaPcwoM/HXqiqilRI0zR+ZMfj7GiUFRDXeK/UHoz8YrI3jxs/GgKbpX7ZkMGrsJDZX7ubJrc+SVppGj8Ae3DL0FsZEj2n1rLt+507sy5ZjX7kST2kp2sBAAqZNwzbzAkz9+5+0e5uuvDxqm86cN+Latw/wXR42JydjGpiCeeBAzAMHoo+IOCllOlF+/zPYGUidTh+dsV6drU6KorQaxm25TJ0HxDb7HNM4r7lrgSkAqqr+rCiKCQgFio++uMfOW1dH9dffYF+5gtq1P4LHg7FXL8JuvQXbtGnoo1r2eHXvGfdy+aeX8+yWZ7lr5F2t79hVDxV7oTwLfXkWcRcGUmYMoeSzz3B8t4KI0RXc39fGaj8Lt9gdXGqwsV7fi2WOUdiiejNjwhiCY/v4LvkeZQjWm8vAL5QhfqH8L2okX+Z8ydObn+aGr29gZORIbh16K31D+gLgLi3FvnIl9uUf07BjB+j1+I8bS+DMmfiPHYtyMs8yG+mjowmMjiZw5kwAXEVFrPvqK8689FJfN5BCCCHaFMYbgJ6KoiTiC+E/AJf/bp19wNnAG4qi9MXX5qekPQt6OI70bQS89jq7brkVta4OXWQkIdf8iYDpMw553/A3ySHJXNr7Ut7b8R4XdL+A5NDkAwvzt8J3j0NhGtj3c+BN1KCYgwgdloSlzwDyPsgi52sdFpeG26+7gSuH/AUUhTM9XjJ/yOLhr37lifcc3HGeg8tHgOY4TkgVReHchHOZGDuRJTuX8N+0/zJn+SXMrR7C2ds0uH/eAB4PpgEDiLjnHgKmTT2ocVNH00dE4I6OliAWQohmjvgbUVVVt6Io84Av8D229JqqqhmKovwT2Kiq6ifArcDLiqLcjC+1rlaPdP27Hbny9mPclo5t2nQCZkzHMqztjWxuGnwTq3NW889f/sniqYvRVhfCNw9C6ntgCYYek32XlpuGRN98QOtp4D89bmLAq2uZs8aLn3MDrkcvQh8ejl6r4YbxPZjaP5K7l6dzz/JtLN+SxyMXDqBnRNsuibdGp9FxkTqYsTsnUrHiEww16ym2Ktj+cD49L/szxh49jmv/QgghTq42nZ40PjO86nfz7ms2nQmc2b5Fazvr2WdT8thjJE+efPTbGqwsGL6ABd8v4P0V13BZ+he+nvfP/D846xZfi+JDqHfXM//b+fxY/jNnPHo/3dIUih5+hOyZs4h69BH8x44FICHUj/9dO5KPNufx0KeZTH3mB64Zk8g1ZyYSEXDkRkHNucvKsK9YgX3pMhp27UIxGAidNAn3eWO5ueJ50G7lg7gIjtwvmBBCiFNJp7hWqOj1cKyduns9TCkrYqnTwzPlG5nUaxJhkxdCUHyrmzjcDv72zd9YV7COf4z+Bxf2vBD6gGXIEPJuuZXcudcTfPXVhN9yM4rBgKIoXDw0hgm9w1j46XZe+j6L19ZmMyMlimvGJB7+jVAeD9Vff03l0mXUfPcduN2YUlLodv99BEyd2tS71KPF8Vz9+dU8+PODPDb2Mel8QgghTiOdIoyP2e6v4ct7UYozuCd2KBfqKljULYrHDxPEda465n0zj42FG3nwzAe5oMcFTcuMPXqQ8P4Sih9/nPI33qBuwwain3wCQ0ICACH+Rp66dBB/O7snb/y0l/c35rJ0Sx6jkoL585gkJvYJR9N4U7l+5y7sS5cStvQj9lfXoA0NJfjKKwmcNfOQLxkfFD6IGwbdwLNbnuWMqDOY1XNW+35XQgghTpiuGcbF2+HLe2D3VxCUALPfJL7fBVyb+gIvpL7ArB6zOCPqjIM2q3XVcsNXN7C1ZCuPnPUI05KmHbSOxmSi23334Td6NPl330P2hRfR7YH7sZ1/ftM6CaF+PHB+MjdP7sV76/fxxk97+fNbG0n2h5vUPfTc8h3OzEzQ63H2T6bH3Ln4n3XWERs9Xdv/Wn4p+IVH1j/CoPBBJNoSj/urEkIIceJ1ra6EaophxXx4YTTs3wDnPAQ3rofkmaAoXDvgWuKscSxct5AGT0OLTaud1Vy/+npSS1J5bOxjhwzi5qyTJpG0fBnGfn3JX3AH+XfciaemZe9YNrOeuWfG89koLe8XruDRxXcS/84L7My3kz7rWmwrv8B+/fVYJ0xoU+tjrUbLI2Mewag1suD7BTg9rXcmIoQQ4tTRNcLY5YDvn4BnBsOWt2HEXPjbVhh9k+81eY2MWiN3j7ybnKocXtv2WtP8KmcV16++nozSDBaNW8SUhCltOqw+MpL4N94gdN487CtWsPeii3BsywCgISuL4iefZPeEiRT89a8E7kwjbM5lOJ5/nRXzHuNO+jL2xS28mFpP+n57m6sa4RfBQ2c+xI7yHTy16ak2byeEEKLjdO7L1F4vpH8AX/8TqvZDn+kw6R8Q2vqjP6OjRzMlYQqvpL3CtMRp2Iw2rvvyOn6t/JUnxz/JxLiJR1UERacjbN6N+I0cQd7tC9h72WWYevWiPiMDtFr8x47FduEsrOPGoRgMdANeAHLL63j9x70s/iWbGc+tZURiMNeOSWRS3wi0R3hYeVzsOOb0ncM7299hVOQoxseOP6oyCyGEOLk6bxjn/ARf3AX5WyByIFz4IiSMadOmtw+/nbV5a3ng5weodlazp3IPT094mrExY4+5OJbhw0lctpSihxbi3LuX8AULsM2Yji4s7JDrxwZbuG9GP4aZisg3xfP6j3u5/u1NxIdY+NPoBGYPi8XP2Po/3y1Db2FT0Sbu/fFePpzxIRF+p3dXk0II0Zl1vsvUZXvgvTnw+nm+e8SzXoTr1rQ5iAHCLeHMGzyPDYUbyLZn8+zEZ48riH+jCwoi+sknSPzoQ0Ku+VOrQdycRa/w57OS+O728Tx/+RBC/Aw8sCKTUY98zSOrtpNf6TjkdgatgcfHPk6Dp4G71t6Fx+s55HpCCCE6Xqc5M9a5quGzO2HDy6A1wsR7YNSNYLAc0/7+0PsPlNSVMCZ6DMO6HbJf75NKp+DnEIsAACAASURBVNUwLSWSaSmRbN5Xwatrs3n5hyxeWZvN1AGR/OnMBAbFBDY9GgWQaEvk7yP+zn0/3cer215lbsrcDqyBEEKI1nSOMM78hJHr/gIeBwy+AibcDdbjuyyr1WiZP3R+OxWwfQ2JC2LI5UHsr6jjzZ/28t76XFak5hNo0TMsPpiRicGMSAwmOSqAmT1m8nPBz/xn638Y0W0Eg8IHdXTxhRBC/E7nCOOgeKoCehNy6fMQ0a+jS3PSxARZuHtaP/5vUi8+31bI+uwy1meX89X2IgAsBi1D44MYGHc5Qcat3P7dAj664EMCDAEdXHIhhBDNdY4wjhxIesr9jO9CQdycv1HHxUNjuHhoDADFVfWs31vOhuxy1mWX8/w3pSjGWVgS/su5b9/EzKg7GZkUwtD4IKymY+xG9BioqsriHYv5ruw7BjcMxmY8TDegQgjRhXSOMBYthAeYmJ4SxfQU3/ub7XUuNuaU80ZGJVtrF/Na2nu88N0INAr0iwpgeILv0vbwhGBC/E/MaybsDXbu+fEe1uSuAeCSFZewaNwiUsJSTsjxhBDidCJh3AXYLHrO7hvBhD53cP3qLLZoV/HQeTPILQpgQ3Y5i9ft4/Uf9wLQPcyPEYkhjEgMYkRiCNGB5uM+flpJGrd/dzvFjmLuHHEnrn0u3q1+l6s+u4r5Q+dzZb8r5cUWQoguTcK4C9EoGh4e8zAXr7iYxdkLeXfau5gm98Lp9pKeZ2d9djnrs8tYmZbPu+v3ATAyMZgFU3ozND74qI+nqipvZ77Nvzb9iwi/CN4+7236h/ZnTdEa3h//Pvf9eB9PbHyCDYUbeOjMhwg0BbZ3lYUQ4rTQ+Z4zFocVZglj4ZiF7K7czRMbnwDAoNMwND6Iv47vzut/GsHW+87h07+N4c7z+rCnpJaLXviZa97YQGZ+VZuPY2+w83/f/h+LNi5iXOw43p/xPv1D+zcttxlt/HvCv7lzxJ38lP8Ts1fOZmvx1navrxBCnA4kjLugMdFjuKrfVSzZuYSvc74+aLlWo5AcZeMv47rz/YLxLJjSm417y5n6zA/c9O4WsktrD7HXA9JL0rlkxSX8kPcDdwy/g3+N/9chW3ArisKcvnN4e+rb6BQdV39+Na+mv4pX9bZbXYUQ4nQgYdxF/d+Q/6NfSD/u++k+CmoKWl3PYtBxw/ge/HDHROZN6MFXmUVMeuo7/r407aDev367LH3l51cC8NaUt/hjvz8e8X5wckgy7894n7Pjzubfm//NjV/fSHl9+fFXUgghThMSxl2UXqvn8bGP4/a6ufOHO3F73Ydd32bWc9u5vfl+wQSuGBXPR5vyGP/EGh5cmUlZTQP2Bjvzv53P4xse56zos3h/xvsMCBvQ5vJYDVaeGPcE94y8h/UF65n9yWw2Fm483moKIcRpQcK4C4sPiOeeUfewuXgzL6W91KZtwqxGHjg/mW9uG8cFA6N4/cdsxj79FlM+uJDv9n/PguELeHrC08f0DLGiKFza51LemfYOZr2Za7+8lpfSXjrpl61LHaW8mfEml6y4hGc2P3NSjy2E6JokjLu4Gd1nMCNpBi+mvXhUZ6IxQRYevziFv80qQhvzPFX1TtT8G6grGU296/jCs09wH5ZMX8K5Cefy7JZn+cvqv1DqKD2ufR6Jy+vi631fc9M3NzH5g8k8sfEJKhsqeTn9ZT7L/uyEHlsIISSMBXePupsY/xju/OFOKusr27RNlbOKW9bcwqvb/8W42LN4ddJiBoUP5JHPdjBu0be8/UsOTvexh7Kf3o/HznqM+8+4n83Fm5m9YjbrC9Yf8/5as6tiF49veJxJH0xi/rfz2Va6jSuSr+DjCz7m0ws/ZXD4YO7/6X6yKrPa/dhCCPEbCWOBn96Px8c9Tll9Gff9dB+qqh52/YzSDC5ZcQlrctdw27DbeGbCM4xKiOXNa0awZO4o4kMs3Lt8G5Oe+o5lW/bj8R5+f61RFIWLe13MO1PfwV/vz3Wrr+OFrS8c9+sg7Q123t3xLpeuvJSLPrmId3e8y9CIoTx/9vOsvng1twy9haTAJPQaPYvGLsKsM3Pzmpupc9Ud13GFEKI1EsYC8LVonj9kPt/mfsuSnUsOuY6qqryz/R3++Nkf8ageXp/yOlclX9WitfTIpBDev/4MXv/TcPyNOm5eksrUp3/gy4zCI4Z8a3oH92bJ9CVMTZzKf1L/w/Wrrz/qy9Yer4cf837ktu9uY8L7E3h43cN4VS93jriTb2Z/w1Pjn2JszFh0mpb94ET4RbBo7CL2Vu3lgZ8eOOY6CCHE4UgPXKLJFf2u4OeCn1m0YRGDwwfTO7h307JqZzX3/3Q/q3NWMy5mHAvHLGy1kZaiKEzoHc64nmGs2lbAU1/uYu7bmxgYG8iCc3sfcpsjsegtPDzmYUZ0G8HD6x7mok8u4tGzHuWMqDMOu11OVQ4f7/6YT/Z8QlFdETajjdm9ZjOzx0z6hvRt07FHRI7gpsE38fTmpxkYPpA5feccUx2EEKI1EsaiiUbRsPDMhVz0yUUs+H4B701/D7POTEZZBretuY2C2gJuHXrrQWfDre5PozA9JYopyd1YujmPf3+1izmvrKNnoIY8cw7n9Y8k2M/Q5vIpisKsnrMYEDqA2767jetXX891Kdfx14F/bXFGW+uq5cu9X7J893I2F29Go2g4M+pMFgxfwPjY8Ri0bT/mb67pfw2pxak8sfEJ+of2Z2DYwKPehxBCtEbCWLQQYg7h4bMe5vrV1/PY+sfoHdybRRsWEWIO4Y0pbzAofNBR71On1XDJ8FjOHxTF4nX7eOnb7dy9bBv3fZzBmB6hTE+J5JzkbtjMbXudY4+gHiyetphH1j/CS2kvsaloE4+d9Rj7qvexfPdyVuesxuF2kBCQwPwh85nRfQbhlvCjLndzGkXDQ2Me4tKVl3Lrmlt5f8b7BJuOvr9uIYQ4FAljcZDRUaO5pv81vLbtNQDGxoxl4ZkLj/tFDia9lmvGJJLo2kt47yGsTCtgRWo+t3+Yxt3LtjGudxgzBkYxqW84FsPhfzQtegsPnvkgI7qN4MFfHuScj87Bq3rx0/sxNXEqM3vMZGDYwHZ9G5TNaONf4//FH1f9kTu+v4P/TvovWo223fYvhOi6JIzFIc0bPI8yRxk9g3pyRb8r0Cjt19ZPUXx9XydH2Vhwbm+25layIrWAT9PzWZ1ZhFmvZWLfcGakRDG+dxgmfeuBN6P7DJJDk1m8fTEDwwYyKX4SZt3xv/axNX1D+nL3qLu5/6f7eSH1BeYNnnfCjiWE6DokjMUh6TV6Hhrz0Ak/jqIoDI4LYnBcEPdM68uGveWsSMtnVXohn6YV4G/UcU5yBDMGRjGmRyh67cF/FCTZkrhn1D0nvKy/ubDnhWwt3sqLaS+SEpbC2JixJ+3YQojOScJYnDI0GoWRSSGMTArhgRnJ/LTH927lz7YVsnRzHoEWPef178aMlChGJoWg1bTfJeijddfIu9hevp2///B33p/xPtH+0R1WFiHE6U+eMxanJJ1Ww9heYTx+8UA23jOJV64cxrheYXy8NZ/LX1nHqEe+5oFPMti4txzvMXYqcjxMOhNPjXsKVVW5Zc0tNHgaTnoZhBCdh5wZi1OeUadlUr8IJvWLwOH08O3OYlak5rN4/T7e+GkvUTYTUwdEMiwhiH6RNmKDze3acKs1sQGxLByzkL99+zceW/8YY5HL1UKIYyNhLE4rZoOWqQMimTogkup6F19tL2JlagFv/ryXV9ZmA2A16ugbFUByVAD9IgNIjrLRI9wfg679LwRNiJvQ1PLcGGJkPOPb/RhCiM5PwlictqwmPbMGxzBrcAz1Lg87C6vJyK8is8BORn4V763PxeHy9WNt0GroGeHfGM4B9Iuy0TfSitXUtmebD+emwTeRXprOkqIlzKqYRa+gXse9TyFE1yJhLDoFk17LwNhABsYeeBba41XJLq0ls6CKjHw7mflVfLOjmA827W9aJz7E0uIMul9UAOFW41Fd5tZpdDw+9nEu+OgCbllzC+9Oexerwdqu9RNCdG4SxqLT0moUeoT70yPcn/MHRgG+l10UVzc0hXNGfhXb8qpYlV7YtF2ov4G+jeHcPzqAgTGBxAQd/j50qDmUa0Kv4dniZ7nvx/t4avxTJ+W+tRCic5AwFl2KoihEBJiICDAxsU9E0/yqehfb86saz6KryMyv4tW1Wbg8vpbawX4GBkTbGBhjY0BMIANjbIQHmFrsu7upOzcPvZknNj7BW5lvcVXyVSe1bkKI05eEsRBAgEnf9IzzbxrcHnYV1pC6v5K0/ZWk7bfz3Lcl/PYkVbcAEykxNgbGBjIg2kaNU+XKfleSWpLKvzb9i+SQZIZ1G9ZBNRJCnE4kjIVohVGnZUCMjQExNiAeAIfTQ0a+ndT99qaA/jKzqGmbRalr6Bd9MVZdBvO/uZXFU5cQF9itg2oghDhdSBgLcRTMBi3DEoIZlnDgjU12h4tteXaWfb+ZWkMAafvsFDguwZLwPFPenUt0/XwGxoQ0XeLuG2nFqJMXTAghDpAwFuI42cx6zuwRimu/gfHjhwJQUt3Aa6km/rfnUTB/zpqdk/mwsRX3by/CmDYgkgm9wzEbJJiF6OokjIU4AcKsRu4YMwenNov3d73Pv684lz62iaTvr2Tt7lI+3+Z7EYYEsxACJIyFOKHuGHEHGWUZ3PPjPSyZvoQp/eOY0j+Sf5zfn3XZZaxKL5BgFkJIGAtxIhm0Bp4a/xSXrLyEm9fczP+m/g+zzoxWozC6eyiju4fyj/P7sz67nE/T81sGc59wpqUcOZhdXhdVDVUEGgPRaiTAhTgdtSmMFUWZAjwNaIFXVFV99BDrXAI8AKhAqqqql7djOYU4bUX5R/HoWY9yw1c3sPCXhTx45oMtOgTRahTO6B7CGd1DmoL547QsVu/YxWe7f8FkrKNPtEJihIrNv4EqVwVljjLK6ssory/H3mAHfO+gjrPGERcQR0JAAvEB8cQHxJNgSyDEFCKdkAhxCjtiGCuKogWeByYD+4ENiqJ8oqpqZrN1egJ/B85UVbVCUZTwE1VgIU5HY6LHcP3A6/lv6n+JtcYSFxBHeX05ZY6yluPGgHW4HRAFfo3b/+qFXwtA9ZixaAOJ8AshKbg7I7qNIMQcQoAhgKK6InLsOeyr3sfavLW4vK6m4/vp/ZrCuSmkGwNbuu4UouO15cx4BLBbVdUsAEVR3gMuADKbrXMd8LyqqhUAqqoWt3dBhTjd/SXlL6SVpPHc1uea5mkVLUGmIEJMIQSbgokLiCPYFEyI2fc5xBRCiDmEQEMQewoVvswo5bNthaTXNLC78VL24AGRTOgehsVw4L+zx+uhsK6QHHsOe6v2klOVQ05VDmklaXye/TkqB94BHWwKbnEm/dsQFxB3Ur8fIbqytoRxNJDb7PN+YOTv1ukFoCjKj/guZT+gqurn7VJCIToJrUbLcxOfI700nQBDACHmEGxGGxqlba92jLLCWT278cD5yazPLmdVegGfbSvk0/QCTHoNA2MCGRQXyODYIAbHBRIdEE20fzSjo0e32I/T42R/9f4WIb23ai8/5P3Ast3LmtZTUIg1xJKbmct5iecRag5t1+9DCHGAoqrq4VdQlIuBKaqq/rnx8xXASFVV5zVbZyXgAi4BYoDvgQGqqlb+bl9zgbkAERERQ9977712q0hNTQ3+/v7ttr9TRWesV2esE3RMvbyqys5yL5uL3eyu9LKvyktjd9oEmxSSbBqSAjV0t2lJsGkwag9/39jhdVDiKqHEXUKRq4i0mjTyPHkoKPQx9WG433BSLCkYNcaTULsTQ37+Th+drU4TJkzYpKrqIfvIbcuZcR4Q2+xzTOO85vYD61RVdQHZiqLsAnoCG5qvpKrqS8BLAMOGDVPHjx/fpgq0xZo1a2jP/Z0qOmO9OmOdoOPqNbHZdL3LQ2ZBFVv3VbI1t5ItuRVs3OkAXGg1Cr0jrAyKC2RQbCBD4gJJCvVHo2k9oNesWUPcoDhWZq1kVfYq3ip7C7PdzMS4iUxLnMYZUWeg05xeD2XIz9/pozPWqTVt+V+0AeipKEoivhD+A/D7ltLLgcuA1xVFCcV32TqrPQsqhDgyk17LkLgghsQFNc0rrWkgNdcXzltzK1mRms/idfsAsJp0vsvbsY1DXCCh/i3PepMCk/jbkL8xb/A8thZv5dOsT/l87+d8mvUpwaZgzks8j+lJ00kOSZYW20IcoyOGsaqqbkVR5gFf4Lsf/JqqqhmKovwT2Kiq6ieNy85RFCUT8AC3q6padiILLoRom1B/I2f3jeDsvr5XRnq9KlmlNWzZdyCgX/huD57G11HFBpsZFBvEoNhAvBUehjW48Tfq0CgahkQMYUjEEO4YcQdr89ayMmslH+z8gHe2v0NCQALTkqYxLWkasdbYwxVJCPE7bbq+pKrqKmDV7+bd12xaBW5pHIQQpzCNRqFHuJUe4VZmD/OFpsPpYVu+nS37KtiaW8mmveWsSM0HYOG6L0gIsZAcZaNfVAD9IgPoFxXAhNgJTIybSJWziq9yvmJl1kqe3/o8z299noFhA5meNJ1zE84lyBR0uOIIIZAeuIQQ+N5GNTwhmOHN3kZVVFXPu5+vRRcaT0Z+Fdvy7XyaXtC0PNTfQN/GYO4XOZK7Bp+DyVzFF3s/Y2XWShauW8hj6x9jTPQYpnWfxviY8Zh0po6onhCnPAljIcQhRQSYGBSuY/z4nk3zqutd7CisJiPPTmZBFZkFVby+di9OjxcAk15Dn2596Bs5nLMSyij0/MTG0q9Zs38Nfno/JsVN4pyEc0iyJdHNr9tp1/hLiBNF/icIIdrMatIfdAbt8njZXVxDZr4vnDPzq/g0rYCqejcwCI2SQnRkPqagVFZlfcnHez4GQKfoiPSPJMY/hlhrLDHWGGKsjdP+MfgbOs8jLUIciYSxEOK46LUa+kYG0DcygIsa56mqSl6loymgM/IjyczrSbn9HLTmfWj05Zj9KrG77dTUFbO1OAOHp6rFfoOMQb6A9m8W0o3jcEt4mztLEeJ0IGEshGh3iqIQE2QhJsjCOcndmubb61xsL6xiZ2E1O4uq2VlYza6caqob3KCpR6MvIySwhpDAasxKJbX1ZWyuTeXLnC/xqJ6m/eg1eqL9o1uEtV6jR0XFq3qbBo/qQVWbzcNLdkU2qZtTW6x30IAXVVVJtCVyca+L8dP7HaqaQrQbCWMhxEljs+gZlRTCqKSQpnmqqpJvr2dnYRU7C2vYWVjFjsJqtu2uwdXYnZhW4yU23ElUSB22gCp0xnJcSilljgJSi1OpdlW3uQwKCtoMLRo0aDVaFBQ0iuagAeCjXz/i5fSXuarfVVzW5zK5dC5OGAljIUSHUhSF6EAz0YFmJvaJaJrv8njZW1rbdAa9o7CaXYXV/JxZh6omAGDWa+kR4cfwcC1RgQaibH5E2sxE2ixEBVow63VoFS2KojSNj6ZXp/SSdF5Me5FntjzD6xmvc0W/K5jTdw4BhoAT8E2IrkzCWAhxStJrNfSMsNIzwsr0lAPz65xufi2qORDQRdX89Gs1JdUVB+0jxM9AVKCZSJuJqEAzUYEmKgrcWHPKiQo0E241oT1Md6ADwgbw3NnPkVGWwYupL/Kfrf/h7Yy3mdNvDn/s+0dsRtuJqLrogiSMhRCnFYtBx8DYQAbGBraYX+/yUGivJ9/uoKCynvxKB/n2egrsDvaW1fLznjLfvWnghdSfAdBqFCKsRl9gN4Z1lO1AeMcGW7CZ9SSHJPPMxGfYUb6Dl9Je4r+p/+XtzLe5vM/lXNnvSgJNgQeVU4ijIWEshOgUTHotCaF+JIS23tiqqt7FJ6t/ILpX/2aB7QvvtP2VfLGtvumZ6d+EWY30CPOnR7g/3cP8uDD6LmYnXcNHWW/wSvorvLP9Hf7Q5w9clXwVwabgVo4sxOFJGAshuowAk54Yq4bxvcMPudzrVSmrdVJgd5Bf6SCnrI7dxTXsLqlh+dY8quvdTetajZOJ6TYSt3U1r217nf9lLmZq/IXMG/JnIvzDTlaVRCchYSyEEI00GoUwq5Ewq5GUmJaXnlVVpaSmgd3FNewprmkMaRt79l1Kbf0ZuEK+ZZnnXZbt+QC/hjH097uAfhExjWfUvsFs0LZbWV0eFzlVOWTZs8ipyqG0ppTI8ki6B3aXns1OQ/IvJoQQbaAoCuFWE+FWE6O7h7ZYVl3vYk/JLNbl7mRV7v/I0nzHes8P/Jg5gobvx6G6fQ29ogPN9Aj3JzHUj6QwPxJDfUOUzdzqe6XrXHVkV2WTVZlFlj2raZxbndvi2WuAxSsWY9aZ6RPch+SQZJJDk0kOSSY+IF46STnFSRgLIcRxspr0je+EHsn1jCS3OpdX0l/h492fYA7ewLCQc0nSzaCwwszu4ho27i2n1nkgSA06DXGhKmHBdvz8y0FfSK2aT0lDLkV1B17OoVN0xAbE0j2wO5PjJ5MUmESSLYmEgARWrFmBJclCZlkm20q38eGuD/nf9v/5yqe30i+kH/1C+9E/pD/JoclE+UXJ+6dPIRLGQgjRzmKtsfxj9D+YmzKXV9NfZdnuZWzkc2b2mMlT5/yB8vpy0ot+Jb1kF9n2LIoc+yhS7RSpQDWoXh1eZxjehnD0noGEm2NJDEiib1giPcICSWxsqBZg0jcdM0Ifwfju45nRfQYAbq+bPZV7msI5oyyDtzPfxu313fcOMgY1hXP/0P4khyQTZpF73R1FwlgIIU6QaP9o7jvjPq4bcB2vbnuVpb8u5cNdHzYtt+qtJAYmMjhqAt1t3UkKTCLemojqDiSnrJ7skhqyS2vJKq1l5/5avk7fi6oe2H+ov5HEUAuJoX64Kp2U+OcS3fiYVqTNRO/g3vQO7s2snrMAcHqc/Frxa1M4byvbxsv5L+NVfS3Iwy3hJIckN4XzgLAB0sHJSSJhLIQQJ1ikfyT3jLqH6wZcx0/5PxHpH0mSLYkwc1irl4oTQqyM69XyTLXe5SG3vI6s0lqyS2vZ2xjU3+4soaTaxbLdaS3WD/YzHOjwxGZqDOlAugdOZkzy+URYjbjUBnaU7yCj1BfOGaUZfJv7LQA6jY6x0WOZljSNcbHjMGqNJ+YLEhLGQghxskT4RTSdpR4Lk17b1CvZ763+5lt6DRxBfuPz0wX2xk5PKh3sK6vjl6yyFo9mAWgUCLeaiAw0ERXYjyjbEGaFmQlK8uAghz21G/hm/xd8k/sNVr2VSfGTmJ40nWHdhkmDsHYmYSyEEJ2AXqMQH+JHfEjrnZ7UNLgpqHSQV+mgoDGo8+2+8M7Mr+KrzCIa3M07PRmAzTKI6PD9qNrNrNzzGct2LyPEFM7UxPO4oMcMegf3PvGV6wIkjIUQoovwN+paPbMG37PU5bVOChoDel/jJfGsEhtZOQmU10xG57+dItsW3nK8zdvb38SsRtPDbxxnRU5mUGQi3cP8CLMapaX2UZIwFkIIAfiepQ7xNxLib6R/9MEvwaiud5FdOoGskloyi/LZULKGfc61pNctJn3PYtxpibirBmNqGExiSChJoX4khfmTFOZHUqjv+erj6fhEVVXK68tbPG+dZc8i256NikqAIcA3GAMOOW0z2g6ab9AajucrazcSxkIIIdrEatKTEhNISkwgM4kGhgO3k2Pfx5LtH/NlzmcU+S1FwyeUe1MoKBjI8tTuoB6ImkibCZtZT4BJj9Wkaxz0+DefNmrwaCrYXJzOhp93UVy/j7y6HPZVZWN32pv2ZdFZSLQlMrzbcHQaHVUNVVQ5q8ivyWeHcwdVDVXUuesOWyeT1tRqgAeZgpibMvcEfZstSRgLIYQ4LvG2OBaMuonbR84jsyyTlVkr+Sz7M+yazURFWBkePp4k81jUukT2VTiocriprndRUFXLjrJCarx5NCgFYChCYyhBYyxG0bh8O98FXrdf43PXvdF5umEmigBtNDZDKMYGA1VVOkL8DP/f3t1HR1Xf+x5/f0liAgR5NpEHC1QpChEQRKFHCFIUWQpqxUjVAh5wqRVbbVWKtOVWbFWs9uGwVOoRweIFxHLLUdQjFyJyBRU5CAiIFFFCUQPEQNQQkvzuH/PAzGQmmUCSPTN8XmtlZT/8fr/9/c6e5Dt7z8zenJ2dyRntfZcz7ZidRcdWmZzewiiv+jpYqA9XHI49XXGY/V/v56OSjzhccZjMtEwVYxERSS5m5rsEZ4fe/Hzgz3l3/7u8vPtlVn62klWV/0Vuy1yGdh2KKz9Ecelu9h/5LHgRkgwgp0UuXbO7kdvi+3TMPIuDnx6lX8+hVFe24Eh5ZcjPMY6UV1J2tJKvvqlg76FvOPh1BaXfHosaV7uWp3FGq0CRzqZjq/Z0bJVJTqtM+rTN5IzTfcX79ObpYe91V1VXRR2vMagYi4hIg0tvls6QzkMY0nkIM47NoHBvIa988grL/7mc3Ja5dG/dneFdhwcv6dm9dXdaZoR/ErywvJD8/r3i3mb5sSoOfl3Bl4fLKT5ylOKyo77fgZ+yo+wu/prisqNUVFbX6H9aWjM6tsqkQ6tMOmZncmbrLB68us9JPxbxUDEWEZFG1SKjBaN7jGZ0j9GNup2sjDQ6t2lO5zbNa23nnONweWVYkQ5Mf3nEV8iLSr6hqKT295sbkoqxiIicUsyM1s0zaN08g7PPyPY6HAB0CRURERGPqRiLiIh4TMVYRETEYyrGIiIiHlMxFhER8ZiKsYiIiMdUjEVERDymYiwiIuIxFWMRERGPqRiLiIh4TMVYRETEYyrGIiIiHlMxFhER8ZiKsYiIiMdUjEVERDymYiwiIuIxFWMRERGPqRiLiIh4LK5ibGajzOwjM9tlZtNqafdDM3NmNrDhQhQREUltdRZjM0sD5gBXAOcB483svCjtWgE/Bd5p6CBFRERSWTxHxoOAXc653c65CmARMDZKuweBR4DyBoxPREQk5cVTjDsDe0Pmi/zLgszsjUJJ1gAAGjxJREFUAqCrc+6VBoxNRETklGDOudobmF0HjHLOTfbP3wxc5Jy70z/fDFgFTHTO7TGzQuAXzrkNUca6FbgVICcnZ8CiRYsaLJGysjKys7MbbLxEkYp5pWJOkJp5KafkkYp5pVpOw4cPf985F/UzVelx9N8HdA2Z7+JfFtAK6AMUmhlALrDczMZEFmTn3FxgLsDAgQNdfn5+vDnUqbCwkIYcL1GkYl6pmBOkZl7KKXmkYl6pmFMs8Zymfg84x8y6m9lpwA3A8sBK51ypc66Dc66bc64bsB6oUYhFREQkujqLsXOuErgTeB3YDixxzn1oZr81szGNHaCIiEiqi+c0Nc65FcCKiGW/jtE2/+TDEhEROXXoClwiIiIeUzEWERHxmIqxiIiIx1SMRUREPKZiLCIi4jEVYxEREY+pGIuIiHhMxVhERMRjKsYiIiIeUzEWERHxmIqxiIiIx1SMRUREPKZiLCIi4jEVYxEREY+pGIuIiHhMxVhERMRjKsYiIiIeUzEWERHxmIqxiIiIx9K9DiDUsWPHKCoqory8vN59W7duzfbt2xshKm95mVdWVhZdunQhIyPDk+2LiJwqEqoYFxUV0apVK7p164aZ1avvkSNHaNWqVSNF5h2v8nLOcfDgQYqKiujevXuTb19E5FSSUKepy8vLad++fb0LsTQ8M6N9+/YndJZCRETqJ6GKMaBCnEC0L0REmkbCFWOvZWdnex2CiIicYlSMRUREPKZiHINzjnvvvZc+ffqQl5fH4sWLAdi/fz9Dhw6lX79+9OnTh7feeouqqiomTpwYbPvEE094HL2IiCSThPo0daj/9V8fsu1fh+NuX1VVRVpaWq1tzut0Or+5qndc4/39739n06ZNfPDBBxw4cIALL7yQoUOH8sILL3D55ZfzwAMPUFVVxTfffMOmTZvYt28fW7duBeCrr76KO24REREdGcewdu1axo8fT1paGjk5OQwbNoz33nuPCy+8kHnz5jFz5ky2bNlCq1at6NGjB7t372bq1Km89tprnH766V6HLyIiSSRhj4zjPYINaKrv4w4dOpQ1a9bwyiuvMHHiRO655x5+/OMf88EHH/D666/z1FNPsWTJEp599tlGj0VERFKDjoxjuOSSS1i8eDFVVVUUFxezZs0aBg0axKeffkpOTg5Tpkxh8uTJbNy4kQMHDlBdXc0Pf/hDZs2axcaNG70OX0REkkjCHhl77ZprrmHdunX07dsXM+PRRx8lNzeX+fPnM3v2bDIyMsjOzmbBggXs27ePSZMmUV1dDcDvf/97j6MXEZFkomIcoaysDPBd8GL27NnMnj07bP2ECROYMGFCjX46GhYRkROl09QiIiIeUzEWERHxmIqxiIiIx1SMRUREPKZiLCIi4jEVYxEREY+pGIuIiHhMxdgjlZWVXocgIiIJQsU4iquvvpoBAwbQu3dv5s6dC8Brr73GBRdcQN++fRkxYgTgu0DIpEmTyMvL4/zzz+ell14CIDs7OzjW0qVLmThxIgATJ07ktttu46KLLuK+++7j3XffZfDgwfTv358hQ4bw0UcfAb47UP3iF7+gT58+DB48mL/85S+sWrWKq6++OjjuG2+8wTXXXNMUD4eIiDSyxL0C16vT4PMtcTdvXlUJaXWkk5sHVzxc51jPPvss7dq149tvv+XCCy9k7NixTJkyhTVr1tC9e3cOHToEwIMPPkjr1q3ZssUXZ0lJSZ1jFxUV8fbbb5OWlsbhw4d56623SE9PZ+XKlUyfPp2XXnqJuXPnsmfPHjZt2sS3337LsWPHaNu2LXfccQfFxcV07NiRefPmccstt9T9wIiISMJL3GLsoT//+c8sW7YMgL179zJ37lyGDh1K9+7dAWjXrh0AK1euZNGiRcF+bdu2rXPscePGBe+7XFpayoQJE/j4448xM44dOxYc97bbbiM9PT1sezfffDN/+9vfmDRpEuvWrWPBggUNlLGIiHgpcYtxHEewob5toFsoFhYWsnLlStatW0eLFi3Iz8+nX79+7NixI+4xzCw4XV5eHrauZcuWwelf/epXDB8+nGXLlrFnzx7y8/NrHXfSpElcddVVZGVlMW7cuGCxFhGR5Kb3jCOUlpbStm1bWrRowY4dO1i/fj3l5eWsWbOGTz75BCB4mnrkyJHMmTMn2DdwmjonJ4ft27dTXV0dPMKOta3OnTsD8NxzzwWXjxw5kqeffjr4Ia/A9jp16kSnTp2YNWsWkyZNarikRUTEUyrGEUaNGkVlZSXnnnsu06ZN4+KLL6Zjx47MnTuXa6+9lr59+1JQUADAjBkzKCkpoU+fPvTt25fVq1cD8PDDD3PllVcyZMgQzjzzzJjbuu+++/jlL39J//79wz5dPXnyZM466yzOP/98hgwZwgsvvBBcd+ONN9K1a1fOPffcRnoERESkqcV1ntPMRgF/AtKAZ5xzD0esvweYDFQCxcAtzrlPGzjWJpGZmcmrr74add0VV1wRNp+dnc38+fNrtLvuuuu47rrraiwPPfoFGDx4MDt37gzOz5o1C4D09HQef/xxHn/8cY5EnH5fu3YtU6ZMiTsfERFJfHUeGZtZGjAHuAI4DxhvZudFNPsfYKBz7nxgKfBoQwcqMGDAADZv3sxNN93kdSgiItKA4jkyHgTscs7tBjCzRcBYYFuggXNudUj79YCqRSN4//33vQ5BREQagTnnam9gdh0wyjk32T9/M3CRc+7OGO3/A/jcOTcryrpbgVsBcnJyBoR+LQigdevWnH322SeSB1VVVcGvDKUSr/PatWsXpaWlDTpmWVlZ2IVRUkUq5qWckkcq5pVqOQ0fPvx959zAaOsa9LsxZnYTMBAYFm29c24uMBdg4MCBLvKrPNu3bz/hrydFvreaKrzOKysri/79+zfomIWFhXV+jSsZpWJeyil5pGJeqZhTLPEU431A15D5Lv5lYczsB8ADwDDn3NGGCU9ERCT1xfPVpveAc8ysu5mdBtwALA9tYGb9gaeBMc65Lxs+TBERkdRVZzF2zlUCdwKvA9uBJc65D83st2Y2xt9sNpANvGhmm8xseYzhREREJEJc7xk751YAKyKW/Tpk+gcNHFdSyM7OpqysLOq6PXv2cOWVV7J169YmjkpERJKNrsAlIiLisYS908Aj7z7CjkPx35whnq8A9WrXi/sH3R9z/bRp0+jatSs/+clPAJg5cybp6emsXr2akpISjh07xqxZsxg7dmzccYHvZhG33347GzZsCF5da/jw4Xz44YdMmjSJiooKqqureemll+jUqRPXX389RUVFwfsaB+6HLCIiqSlhi7EXCgoK+NnPfhYsxkuWLOH111/nrrvu4vTTT+fAgQNcfPHFjBkzJuzOTHWZM2cOZsaWLVvYsWMHl112GTt37uSpp57ipz/9KTfeeCMVFRVUVVWxYsUKOnXqxCuvvAL47n8sIiKpLWGLcW1HsNE0xPdx+/fvz5dffsm//vUviouLadu2Lbm5udx9992sWbOGZs2asW/fPr744gtyc3PjHnft2rVMnToVgF69evGd73yHnTt3MnjwYB566CGKioq49tprOeecc8jLy+PnP/85999/P1deeSX9+vU7qZxERCTx6T3jCOPGjWPp0qUsXryYgoICFi5cSHFxMe+//z6bNm0iJyenxj2KT9SPfvQjli9fTvPmzRk9ejSrVq2iZ8+ebNy4kby8PGbMmMHDD9fvvs4iIpJ8EvbI2CsFBQVMmTKFAwcO8Oabb7JkyRLOOOMMMjIyWL16NZ9+Wv+bUV1yySUsXLiQSy+9lJ07d/LZZ5/xve99j927d9OjRw/uuusuPvvsMzZv3kyvXr1o164dN910E23atOGpp55qhCxFRCSRqBhH6N27N0eOHKFz586ceeaZ3HjjjVx11VXk5eUxcOBAevXqVe8x77jjDm6//Xby8vJIT0/nueeeIzMzkyVLlvD888+TkZFBbm4u06dP57333uPee++lWbNmZGRk8NhjjzVCliIikkhUjKPYsmVLcLpDhw6sW7cuartY3zEG6NatW/A7xllZWcybN69Gm2nTpjFt2rSwZZdffjmXX355cP7IkSP1il1ERJKP3jMWERHxmI6MT9KWLVu4+eabw5ZlZmbyzjvveBSRiIgkGxXjk5SXl8emTZu8DkNERJKYTlOLiIh4TMVYRETEYyrGIiIiHlMxFhER8ZiK8UnIzs72OgQREUkBKsYpoLKy0usQRETkJCTsV5s+/93vOLo9/vsZV1ZVcaiO+xlnntuL3OnTY65vyPsZl5WVMXbs2Kj9FixYwGOPPYaZcf755/P888/zxRdfcNttt7F7924AnnzySTp16sTo0aPZtm0bAI899hhlZWXMnDmT/Px8+vXrx9q1axk/fjw9e/Zk1qxZVFRU0L59exYuXEhOTg5lZWVMnTqVDRs2YGb85je/obS0lM2bN/PHP/4RgL/+9a9s27aNJ554ou4HWkREGlzCFmMvNOT9jLOysli2bFmNftu2bWPWrFm8/fbbdOjQgUOHDgFw1113MWzYMJYtW0ZVVRVlZWWUlJTUuo2Kigo2bNgAQElJCevXr8fMeOaZZ3j00Uf5wx/+wIMPPkjr1q2Dl/gsKSkhIyODhx56iNmzZ5ORkcG8efN4+umnT/bhExGRE5Swxbi2I9hoEu1+xs45pk+fXqPfqlWrGDduHB06dACgXbt2AKxatYoFCxYAkJaWRuvWressxgUFBcHpoqIiCgoK2L9/PxUVFXTv3h2AlStXsmjRomC7tm3bAnDppZfy8ssvc+6553Ls2DHy8vLq+WiJiEhDSdhi7JXA/Yw///zzGvczzsjIoFu3bnHdz/hE+4VKT0+nuro6OB/Zv2XLlsHpqVOncs899zBmzBgKCwuZOXNmrWNPnjyZ3/3ud/Tq1YtJkybVKy4REWlY+gBXhIKCAhYtWsTSpUsZN24cpaWlJ3Q/41j9Lr30Ul588UUOHjwIEDxNPWLECJ588kkAqqqqKC0tJScnh+LiYg4ePMjRo0d5+eWXa91e586dAZg/f35w+ciRI5kzZ05wPnC0fdFFF7F3715eeOEFxo8fH+/DIyIijUDFOEK0+xlv2LCBvLw8FixYEPf9jGP16927Nw888ADDhg2jb9++3HPPPQD86U9/YvXq1eTl5TFgwAC2bdtGRkYG999/P4MGDWLkyJG1bnvmzJmMGzeOAQMGBE+BA8yYMYOSkhL69OlD3759Wb16dXDd9ddfz/e///3gqWsREfGGTlNH0RD3M66t34QJE5gwYULYspycHP7xj3/UaHv77bdz33331VheWFgYNj927Nion/LOzs4OO1IOtXbtWu6+++5YKYiISBPRkfEp6KuvvqJnz540b96cESNGeB2OiMgpT0fGJykZ72fcpk0bdu7c6XUYIiLip2J8knQ/YxEROVk6TS0iIuIxFWMRERGPqRiLiIh4TMU4gm6LKCIiTU3FOA66RaGIiDQmFeMYCgsLueSSSxgzZgznnXee1+GIiEgKS9ivNr21ZCcH9sa+wlWkqqoq0uq4n3GHrtlccn3PuMfcuHEjW7duDd4BSUREpDHoyLgWgwYNUiEWEZFGl7BHxvU5goWGuZ9xpNBbFIqIiDQWHRmLiIh4TMVYRETEYwl7mtorgdsi5ufnk5+f720wIiJyStCRsYiIiMdUjEVERDymYiwiIuKxhCvGzjmvQxA/7QsRkaaRUMU4KyuLgwcPqggkAOccBw8eJCsry+tQRERSXkJ9mrpLly4UFRVRXFxc777l5eUpWTi8zCsrK4suXbp4sm0RkVNJXMXYzEYBfwLSgGeccw9HrM8EFgADgINAgXNuT32DycjIOKHLT+7dcYgNr2zijNwzaNbMsGYW/Xea0awZWJR1zdIMM//v4DrC2pmFLw8fA8ysxnJrhm/aIuZDxsMIa2/NjGbmW/7mm2/Sv3//ej8mIiKSPOosxmaWBswBRgJFwHtmttw5ty2k2b8DJc65s83sBuARoKAxAo6m4ptKjh6GA8fKqK52uCqHc47qkN/V1Q5XHfgNrjpJToUbbH9xdbDIm4UWd4JFPVDkCbxgsPB1ZjGWhYzp60uUMcKXETm+UXM7IWNb6AsOg+JPHO+X7/G3Pd4Pi4wzYnnE+GHtA/OBGAkfI5BDzPlADBAeD751cLwdBPpzPAeDY986vjlcERFf+HYCy4wY60XklBTPkfEgYJdzbjeAmS0CxgKhxXgsMNM/vRT4DzMz10Rv/n73gjPYe7gZ+fkXx93HOYdz4GoU6pDfoUW92tcnsC5svspR7UKWB8YIeUEQtrzav23/PI6Q5cfXV1c79nyyh7O6nhWyjpBtHR8ntE9weyFjhU2HjlVVHbEcf0yO6mrAhS+LOm5I3/B5B9XHl4X6csvuhn0SJIid/1h70mOEFujgC4KQ4u+v474XOUQp/FCjTeQLi0C742P4F4fOm/F1WTVf/L93j79QsZD+gW2GxBeMB8JfuFAzvrAXNSGDW+g4wXV1tA99gRN8IGs+Hpixf381hft2hD0OYWOGbfP4uOFjh/QL3WBk32AyEX1DXndZ+MYi1oXO+GOJ0e7gLscHVXtrxFRzW4R1jrq9GPEd7x/RLNp4kY9fiMgXnpF9Av0OFzl2byqO0dYiYqCGyHyitbXIlSGzzdKMTue0rTlwI4inGHcG9obMFwEXxWrjnKs0s1KgPXCgIYJsDMGjo2ZG7Tde9Na3hZ8xOP+7Xodx0pzzvehwzlFY+CZDhw4NFm1Ci3hIwSdGsSeiXY3lYfOBaX/7QCyBvtSMITBdHTh7ErYuMP7xT5sHYtu5cyfnnH1OMF9XfTz3aDHifIOHjRfYZLXzrw6PyYEvdo7nF4wprH2gf8h2I3MJae9CYgltU1FdRnabzJC+gQmOP3ZELPePXV3ljsfD8W0G4jveL+SxjHhcam0fknN43+OPT2j/QIgVFbC7+ECNfjX6EN4v+Hoy4vEM215Eu9A2TeHz//m46TbWRPau3eLZtjNbpDP58aFNsq0m/QCXmd0K3OqfLTOzjxpw+A4kcPE/CamYVyrmBKmZl3JKHqmYl+c5TXmiQYf7TqwV8RTjfUDXkPku/mXR2hSZWTrQGt8HucI45+YCc+PYZr2Z2Qbn3MDGGNtLqZhXKuYEqZmXckoeqZhXKuYUSzzfM34POMfMupvZacANwPKINsuBCf7p64BVTfV+sYiISLKr88jY/x7wncDr+L7a9Kxz7kMz+y2wwTm3HPhP4Hkz2wUcwlewRUREJA5xvWfsnFsBrIhY9uuQ6XJgXMOGVm+Ncvo7AaRiXqmYE6RmXsopeaRiXqmYU1Sms8kiIiLeSqhrU4uIiJyKkq4Ym9koM/vIzHaZ2bQo6zPNbLF//Ttm1q3po6wfM+tqZqvNbJuZfWhmP43SJt/MSs1sk//n19HGSiRmtsfMtvjj3RBlvZnZn/37arOZXeBFnPEys++FPP6bzOywmf0sok1S7Ccze9bMvjSzrSHL2pnZG2b2sf931KsdmNkEf5uPzWxCtDZeiJHTbDPb4X9+LTOzNjH61vpc9VKMvGaa2b6Q59noGH1r/X/plRg5LQ7JZ4+ZbYrRN2H31Unxfbk/OX7wfYDsn0AP4DTgA+C8iDZ3AE/5p28AFnsddxx5nQlc4J9uBeyMklc+8LLXsdYzrz1Ah1rWjwZexXexm4uBd7yOuR65pQGfA99Jxv0EDAUuALaGLHsUmOafngY8EqVfO2C3/3db/3Rbr/OpJafLgHT/9CPRcvKvq/W5moB5zQR+UUe/Ov9fJlJOEev/APw62fbVyfwk25Fx8NKczrkKIHBpzlBjgfn+6aXACEvwi/465/Y75zb6p48A2/Fd1SzVjQUWOJ/1QBszO9ProOI0Avinc+5TrwM5Ec65Nfi++RAq9G9nPnB1lK6XA2845w4550qAN4BRjRZoPUTLyTn33865Sv/senzXSUgqMfZVPOL5f+mJ2nLy/7++HvjfTRqUx5KtGEe7NGdk0Qq7NCcQuDRnUvCfVu8PvBNl9WAz+8DMXjWz3k0a2IlxwH+b2fv+q69Fimd/JqobiP3PItn2U0COc26/f/pzICdKm2TeZ7fgOxMTTV3P1UR0p//0+7Mx3lJI1n11CfCFcy7WtT2TcV/VKdmKcUozs2zgJeBnzrnDEas34jsl2hf4C/B/mjq+E/BvzrkLgCuAn5hZ01zktZH5L34zBngxyupk3E81ON/5wJT5qoWZPQBUAgtjNEm25+qTwHeBfsB+fKd1U8V4aj8qTrZ9FZdkK8b1uTQnVsulORONmWXgK8QLnXN/j1zvnDvsnCvzT68AMsysQxOHWS/OuX3+318Cy/CdNgsVz/5MRFcAG51zX0SuSMb9FOKLwNsE/t9fRmmTdPvMzCYCVwI3+l9k1BDHczWhOOe+cM5VOeeqgb8SPd5k3FfpwLXA4lhtkm1fxSvZinFKXprT/x7JfwLbnXOPx2iTG3jv28wG4dt3Cfsiw8xamlmrwDS+D9JsjWi2HPix/1PVFwOlIadJE1nMV+7Jtp8ihP7tTAD+EaXN68BlZtbWf2r0Mv+yhGRmo4D7gDHOuW9itInnuZpQIj5bcQ3R443n/2Wi+QGwwzlXFG1lMu6ruHn9CbL6/uD7BO5OfJ8SfMC/7Lf4/tgAsvCdPtwFvAv08DrmOHL6N3ynBDcDm/w/o4HbgNv8be4EPsT3icj1wBCv464jpx7+WD/wxx3YV6E5GTDHvy+3AAO9jjuOvFriK66tQ5Yl3X7C92JiP3AM33uJ/47vsxX/F/gYWAm087cdCDwT0vcW/9/XLmCS17nUkdMufO+bBv6uAt+06ASsqO25mig/MfJ63v83sxlfgT0zMi//fI3/l4nwEy0n//LnAn9LIW2TZl+dzI+uwCUiIuKxZDtNLSIiknJUjEVERDymYiwiIuIxFWMRERGPqRiLiIh4TMVYRETEYyrGIiIiHlMxFhER8dj/B5efi5bszbJeAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "gbsHOcXH1dFA", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "13552783-90bc-4620-d5d0-a69487f728c5" }, "source": [ "y_pred = model.predict(X_test)\n", "y_pred = y_pred.argmax(axis=1)\n", "print(classification_report(y_test, y_pred,\n", " target_names=class_names))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " T-shirt/top 0.76 0.85 0.81 1000\n", " Trouser 0.98 0.94 0.96 1000\n", " Pullover 0.78 0.71 0.74 1000\n", " Dress 0.78 0.92 0.84 1000\n", " Coat 0.76 0.75 0.75 1000\n", " Sandal 0.97 0.88 0.92 1000\n", " Shirt 0.62 0.55 0.58 1000\n", " Sneaker 0.86 0.97 0.91 1000\n", " Bag 0.96 0.93 0.94 1000\n", " Ankle boot 0.96 0.91 0.94 1000\n", "\n", " accuracy 0.84 10000\n", " macro avg 0.84 0.84 0.84 10000\n", "weighted avg 0.84 0.84 0.84 10000\n", "\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "nfmfMil61irw" }, "source": [ "## 6.2: Dropout" ] }, { "cell_type": "code", "metadata": { "id": "JpyqsFqG1e02", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "a3aea1db-4c25-4fb0-b455-f37ebc3ab139" }, "source": [ "model = tf.keras.models.Sequential([\n", " tf.keras.layers.Flatten(input_shape=[28, 28]),\n", " tf.keras.layers.Dropout(rate=0.2),\n", " tf.keras.layers.Dense(300, activation=\"elu\", kernel_initializer=\"glorot_uniform\"),\n", " tf.keras.layers.Dropout(rate=0.2),\n", " tf.keras.layers.Dense(100, activation=\"elu\", kernel_initializer=\"glorot_uniform\"),\n", " tf.keras.layers.Dropout(rate=0.2),\n", " tf.keras.layers.Dense(10, activation=\"softmax\")\n", "])\n", "model.compile(loss=\"sparse_categorical_crossentropy\", \n", " optimizer=\"adam\", metrics=[\"accuracy\"])\n", "\n", "lr_scheduler = tf.keras.callbacks.LearningRateScheduler(exponential_decay_fn,\n", " verbose=1)\n", "history = model.fit(X_train, y_train, epochs=20,\n", " validation_split=0.1,\n", " callbacks=[lr_scheduler])" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "\n", "Epoch 00001: LearningRateScheduler reducing learning rate to 0.01.\n", "Epoch 1/20\n", "1688/1688 [==============================] - 4s 3ms/step - loss: 0.8610 - accuracy: 0.7043 - val_loss: 0.6167 - val_accuracy: 0.7830\n", "\n", "Epoch 00002: LearningRateScheduler reducing learning rate to 0.008912509381337455.\n", "Epoch 2/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.7543 - accuracy: 0.7453 - val_loss: 0.5290 - val_accuracy: 0.8085\n", "\n", "Epoch 00003: LearningRateScheduler reducing learning rate to 0.007943282347242816.\n", "Epoch 3/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.6671 - accuracy: 0.7685 - val_loss: 0.5249 - val_accuracy: 0.8060\n", "\n", "Epoch 00004: LearningRateScheduler reducing learning rate to 0.0070794578438413795.\n", "Epoch 4/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.6351 - accuracy: 0.7816 - val_loss: 0.4758 - val_accuracy: 0.8340\n", "\n", "Epoch 00005: LearningRateScheduler reducing learning rate to 0.006309573444801933.\n", "Epoch 5/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.5971 - accuracy: 0.7935 - val_loss: 0.4258 - val_accuracy: 0.8552\n", "\n", "Epoch 00006: LearningRateScheduler reducing learning rate to 0.005623413251903491.\n", "Epoch 6/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.5628 - accuracy: 0.8049 - val_loss: 0.5085 - val_accuracy: 0.8177\n", "\n", "Epoch 00007: LearningRateScheduler reducing learning rate to 0.005011872336272724.\n", "Epoch 7/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.5321 - accuracy: 0.8150 - val_loss: 0.4548 - val_accuracy: 0.8468\n", "\n", "Epoch 00008: LearningRateScheduler reducing learning rate to 0.004466835921509631.\n", "Epoch 8/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.5128 - accuracy: 0.8208 - val_loss: 0.3931 - val_accuracy: 0.8578\n", "\n", "Epoch 00009: LearningRateScheduler reducing learning rate to 0.0039810717055349725.\n", "Epoch 9/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4923 - accuracy: 0.8262 - val_loss: 0.4239 - val_accuracy: 0.8398\n", "\n", "Epoch 00010: LearningRateScheduler reducing learning rate to 0.003548133892335755.\n", "Epoch 10/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4718 - accuracy: 0.8317 - val_loss: 0.3923 - val_accuracy: 0.8692\n", "\n", "Epoch 00011: LearningRateScheduler reducing learning rate to 0.0031622776601683794.\n", "Epoch 11/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4550 - accuracy: 0.8378 - val_loss: 0.3766 - val_accuracy: 0.8652\n", "\n", "Epoch 00012: LearningRateScheduler reducing learning rate to 0.002818382931264454.\n", "Epoch 12/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4436 - accuracy: 0.8418 - val_loss: 0.4123 - val_accuracy: 0.8618\n", "\n", "Epoch 00013: LearningRateScheduler reducing learning rate to 0.0025118864315095803.\n", "Epoch 13/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4349 - accuracy: 0.8446 - val_loss: 0.3621 - val_accuracy: 0.8687\n", "\n", "Epoch 00014: LearningRateScheduler reducing learning rate to 0.0022387211385683395.\n", "Epoch 14/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4218 - accuracy: 0.8485 - val_loss: 0.3560 - val_accuracy: 0.8658\n", "\n", "Epoch 00015: LearningRateScheduler reducing learning rate to 0.0019952623149688802.\n", "Epoch 15/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4063 - accuracy: 0.8526 - val_loss: 0.3742 - val_accuracy: 0.8687\n", "\n", "Epoch 00016: LearningRateScheduler reducing learning rate to 0.001778279410038923.\n", "Epoch 16/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4027 - accuracy: 0.8537 - val_loss: 0.3643 - val_accuracy: 0.8682\n", "\n", "Epoch 00017: LearningRateScheduler reducing learning rate to 0.0015848931924611134.\n", "Epoch 17/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.3947 - accuracy: 0.8575 - val_loss: 0.3521 - val_accuracy: 0.8762\n", "\n", "Epoch 00018: LearningRateScheduler reducing learning rate to 0.0014125375446227546.\n", "Epoch 18/20\n", "1688/1688 [==============================] - 4s 3ms/step - loss: 0.3838 - accuracy: 0.8605 - val_loss: 0.3520 - val_accuracy: 0.8790\n", "\n", "Epoch 00019: LearningRateScheduler reducing learning rate to 0.0012589254117941673.\n", "Epoch 19/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.3759 - accuracy: 0.8638 - val_loss: 0.3364 - val_accuracy: 0.8787\n", "\n", "Epoch 00020: LearningRateScheduler reducing learning rate to 0.0011220184543019637.\n", "Epoch 20/20\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.3750 - accuracy: 0.8616 - val_loss: 0.3411 - val_accuracy: 0.8800\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "0HbareD91w8J", "colab": { "base_uri": "https://localhost:8080/", "height": 324 }, "outputId": "b82c74f1-91e6-4194-bd3b-0c88dee1b2b5" }, "source": [ "pd.DataFrame(history.history).plot(figsize=(8, 5))\n", "plt.grid(True)\n", "plt.gca().set_ylim(0, 1);" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeMAAAEzCAYAAAACSWsXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3hUVf7H8fedPpPeGwlJaEJACJ0oXcAKYlvXvgoqFlzbT11FxL6ra1srKsquq66oICgoShElgPSqQGihp/c+c35/3ElIqAGSTDL5vp5nnltn5lxKPjnnnnOuppRCCCGEEJ5j8HQBhBBCiNZOwlgIIYTwMAljIYQQwsMkjIUQQggPkzAWQgghPEzCWAghhPCwU4axpmnTNE3L0DRt0wmOa5qmvaFpWpqmaRs0TevZ8MUUQgghvFd9asYfAxee5PhFQAf363bgnbMvlhBCCNF6nDKMlVJLgJyTnDIG+LfSLQcCNU2LaqgCCiGEEN6uIe4ZxwB7a23vc+8TQgghRD2YmvLLNE27Hb0pG7vd3is2NrbBPtvlcmEweF9/NG+8Lm+8JvDO65Jrajm88bq87Zq2bduWpZQKO96xhgjj/UDtVG3j3ncMpdRUYCpA79691apVqxrg63WLFy9myJAhDfZ5zYU3Xpc3XhN453XJNbUc3nhd3nZNmqbtOdGxhviVYzZwk7tXdX8gXyl1sAE+VwghhGgVTlkz1jTtM2AIEKpp2j5gMmAGUEq9C8wFLgbSgBLgL41VWCGEEMIbnTKMlVJ/PsVxBdzdYCUSQgghWhnvuTMuhBBCtFASxkIIIYSHSRgLIYQQHiZhLIQQQniYhLEQQgjhYRLGQgghhIdJGAshhBAeJmEshBBCeJiEsRBCCOFhEsZCCCGEh0kYCyGEEB4mYSyEEEJ4mISxEEII4WESxkIIIYSHSRgLIYQQHiZhLIQQQniYhLEQQgjhYRLGQgghhIdJGAshhBAeJmEshBBCeJiEsRBCCOFhEsZCCCGEh0kYCyGEEB4mYSyEEEJ4mISxEEII4WESxkIIIYSHmTxdACGEEMITlMsFVVWoqiqU04mqqtK3nU5UlROUC0tsbJOURcJYCCHEMVylpTjz81FlZbjKy1GlpbjKylHlZbhKy/Tl8bbLynCVlR15X53tMlR5hf4FmgYaaGj6usFwZJ+mARrBJSXsfPU19/6jzq95j75PuVwoZxVUOd3hWnvdWSd0q9dR6qR/BoaAADqtWN74f9hIGAshRKuiKiqoysykMiODqoxMqjIyjrwyM2r2uwoKTvuzNZsNg9WKZrfrS5sNzWbFYLNjCA3BbLWhWa3ugqgjLxRKKXDV3efMyMQcGlr3XKVQVK8DLpe+bjSimUxoJiMYTfq60QgmI5rJfIJ1E5rR/Z7jrBts1ob8oz8pCWMhWjilFJV79qCUwuDjg9HHB83hcNcuWgdXWRkYDGhmc7O9bqUUqrISVVEBaGiGI7U7rXZNr3r7dDmdVB4+EqrVAVtZE7aZVB0+jDM399j3ms2YwkIxh4VjTUjEp19/TOHhGIMCMdjtaFYrBpsNzWrDYLOi2ezu5ZHw1SyWBv+zT1u8mOQhQxr0M5srCWMhWqjKwxkUzJlN3qxZVKTtqHtQ0zA4HBh8fI59+frUhPZxj9d6aYWFuEpK9B+2TRhyyunEmZtLVXYOzpxsfZmdRVV2DlU52Tizc6jKzsaZnU1VTg6qtLTmumsCwmbTA+So7YDCQg7Mm6cHi10PGM1mxWC1odlt+tKmhw+apjevlrqbWOvbLFvdrFt7u6xMr8XVV62m22PCGmrW0TQ0ILyggLSjm10NBkyhoZjCwzFHR2Pv0V1fDw/HVOtlDAxEM0h/Xk/ymjAurDh5278Q3sBVXk7RggXkzZxF8dKl4HJh79mTiCcnYfTzw1VcXPNyFhW510tq9lXu31/nHL2WdmLhwFb3uma3Y6h+OexodgcGm02vOTnsGOyOI8dsNn3boZ+vv9eBwW4DgxFnXi5VWVk4c3LcQZuth2t18ObmHv9+ntGIKTgYY0gIpuBgLPFtMQWHYAwKAqVwlZWiysprgrN6uzo4qwoLMeXkUHLoUJ37mTidp/cXYTSeuDnWYccYFFSzfUzQWyz6ZyiX3oGopqnVdaSp1uUClH68pun2eMdVTTPt3rw82vftowdsmDtoQ4LRTM38x7xSUFkKlSVQUexeL4aKEoKzV8HmXKgo0Y9XlrjX3efVXkcDkwWMFjBa3eu1l1b3Mcux++ocq7XPZIPghCb5Y2jmf0v188XKvUxeUkLXnsUkhPp4ujjCSzgLCvSaVfUPTw9RSlG2fj15M2dRMHcursJCTFFRhNxxO4FjxmCJjz/zz66sPBLeNSHtDu+iIrZu2EC7NjF6La+kFFdpKa7SElRpmXu9lMrMDFTNsVJUaSmqsrLeZTD4+mIMCcYUEoolPh57z16YQoIxBodgCg3BGByMKURfGgMCzroGt3jxYoYc1fSpKiuP7WxUVg5K6bXno++Fms1nVYbG8MfixQQ1ZJNudUhWlbmDsLTWq+T4+096bmmtYyV1A/YEzgXYeJwDJjuY7WDxAbNDX0eBsxKqysFZ4V6WQ1WFvlSn0SpRzRoAj6Wf/vvOgFeE8fkdQjFocO9na/hqQgpWk9HTRWrxqnJzKV27Fmti4ln9sG/OlFI4c3Ko2JNORfoeKtPTqUjfS0V6OpV79uDMz0ezWrF164ojORl7ck/syT0wBQU1SfkqDx0i/5vZ5M+cScXu3Wh2O/4jRxBw+eU4+vVrkGZFzWzGGBiIMTCQ48VLaUgwoWfwA15VVuIqK8NVUooqLakJaldpKTidGIOCa4LWYG26TjInopnNGM1m8PX1dFHOmOZyQmmuXrssL9KXFYV1148+dtLtIvRq+2kymN0BadND0uzQa5hmBzhC3ft9wOJwn+dwr/u4A9ZRc3z1xt/p1W9g3XPNDr15/nQ5q/RQdlYcCeiaZe0ArzyyrwlvzXhFGEcH2hnXzcrrawp4cd4fTL4sydNFanFcFRWUrllD8dJUilNTKduyRf/N2GDA/8JRhNx+O7ZzzvF0MU+bcrmoysykYo87bPekU5GuvyrT03EVFx852WDAHB2NJS4W20UXYomNpSozi5K1a8j+eDq8/wEAloQE7D2TcfTsiT05GUtCQoPdT3WVllL400/kz5xF8bJloBSO3r0JGT8ev1GjMPq2jJaf6nAz+vl5uiiNy1l1JPDKC/UAKy+stV69vzoU3UHnqgKXU18q55H1Y/bV/5zBrkpYUs9yW3z1WmX10uoHvhF1t2tqnY4joXq8gDXbj7xMdjA2XKwU7nFBRJeG+TCjyV225vl/yCvCGCA53MQtKVF8tHQ3Ke1CGdElwtNFataUUpRv21YTviWrVukdTEwm7N27E3rP3Th696H411/J/fRTCubOw3fwYELuuANHz2RPF/8YlQcPUrFrlx60e6rDdg8V6XtR5eVHTjSZsLRpg7ltHI5evbDExWFpG4c5Lg5LTMwJm6RdZWWUbdpEyZq1lK5ZQ9FPC8j/6msAjIGB2JOTcQQEUOLri61rV73zTz0ppShds4a8mTMpnPc9ruJizDExhN51FwGXj2mySQdaBZcTY1Ux5O+vR+2wqG6oVhRBeUHdY1Vl9fteoxWsvkfCzmByv4z6UjPq4Va9bTCBZjj2nJrjR+8zsmvfYRI6dXOHafV3HRWuFt8zr1mKRuU1YQzw2MXnsHJ3Dg9/uZ65EwcSHWj3dJGalcrDhylOXUZxairFy5bhzMoCwJKYSOBVV+GTkoKjb986tS+ffn0JGT+O3E8/Jefj6ey57jocffoQcucd+KSkeHQYSVVuLgXffkf+rFmUbd5cs1+zWrHExWKOjcPnvPOPhG3btpgjI8+oQ4vBZsPRuzeO3r0BPUArdu2idM0aStaupXTNWvx27WLPrFlgNmPr0hlHck+9Bp2cjCks7JjPrNy/n7xvviF/1jdUpqejORz4jxpFwNjLcfTu3Tp7tzora917rH0PsgyqSmsta9+jLDvS+edUAVtVykCAX+tRFnOtULP6gtUf/NvU2ufnDrnq435g8Tv+cVPj9zvYs3gxCQOGNPr3iMbhVWFsNRl587qeXPrGL9z3+Vo+G98fk7EV/kBzcxUXU7xypR6+qak1w1+MwcH4DBiAT0oKPuelYI6MPOnnGP39Cb3zToJvuom8GTPInvYRe28bh61rV0LuuB2/4cObLDhURQVFS5aQN2sWRT8vgcpKrJ07E/5//4ctKQlL2zhM4eGNXh5N07AmJmJ1/yID8PO339LT4UPp2jWUrFnr/gXmYwDMsbE4eiZjT05GM1vInz2bkhUrAHD070/oXRPwHzECg0/zbEI7hlJH7ise3Sx73H21m3KLjwRrZWndkHVVnVl5DCZ3zc+vVs3QBxwhx9QU0/Yeon2X7kdqjtXn1n6v2Udqj6JJeVUYAySE+vDc2G789X/reH3Bdh4c2cnTRWoyyumkbNMmPXyXplKyfj1UVqJZrTh69SJw7Fh8UlKwdup0RmFlcDgIvvlmAv/8Z/K/+Ybs9z9g/70TsbRvR+jtt+N/8cWNMoxCKUXZ5i3kz5pFwXff4czNxRgaSvD11xMw9nJsnZrH37Hy9cVvyBD8hg3VtysqKNuyRW/aXruWoqWp5H8zGwBzXBxh900kYPRozDExnimwswrK8qAku9Yrp8561307YM8/jwrZotPo3KPVqj36HQk+e6D7nqO91r1Hm7uXrO3I/cc6+xwnfs9p3Kfct3gx7XsNOdM/NSEahdeFMcDlyTEsTcvizUVpDEgMIaV9qKeL1GhcpaUULlxI4fc/ULxiRc0UdtYunQm5+SZ8UlKw9+rVoD1WDRYLQVdfTeDYsRT88APZ703lwP89QuYb/yJk3G0EjB3bIN9XmZFBwZw55M+aRfn2NDSzGd/hwwm4fAy+55/f7MdPahYL9h49sPfoAfxFnylr3z6c+QXYkro0bBN/dU/aOmGaDaU5R4VsreNleSf+PLMD7MHYnEZwRIEjGALjjm2atbibb2s351r8jgSv2dGkPVKFaKma90+zszBlTBJr0nO573/rmHffQEJ9PT98oqGoqiqKly2n4Ns5FP74E66SEkzh4fiNuEBveh4wAFNwcKOXQzOZCLjkEvwvuoiixT+T9d67HHpqCplvvUXILX8h6No/nXazq6usjMIFC8if9c2RSS26dyfyqcn4X3QRxoCARrqaxqdpmt4Z62T9sZyVUJqnh2hpbv1fZfkn/kyTTW+udQTry8BYfWkPrru/emkP1oeSAKuOMyZXCNHwvDaMHRYTb17XkzFvLeXBL9bz0S19MBha7m/oSinKNm0if84cCubOw5mVhcHPD7+LLyLgstE4+niuw49mMOA3bCi+Q4dQsuI3sqe+R8ZLL5E1dSrBN95I8A3XYwwMPOH7lVKUrl2nN0PPm3dkUovx4wkYMwZrYtPMgNNglNI7FNWEZZ47MPPqbtd5ufdVFJ74czUD2ALBHqS/HKEQ0sG9HegOVHeo1gRtSE2wCiGaL68NY4DOUf5MurQLk2Zt4v1fdnLH4HaeLtJpM2ZkkPnmWxTMmUPFnj16U+2QIfhfdim+gwc3iwkTqmmahk//fvj070fp+vVkTX2frDffJGfaNAKvvZbgW27GHB5ec37l/v3kz55N/qxv9GtrhEktzkp1LfXoEC3LO27Q9sneDysr9G3XSWagMpiOBKo9CPyjISKp7r7jvaz+0qlICC/l1WEMcEO/OFLTsnjph630SQimZ1zTzJ50NqqysymYO4/8b+cQun4DWZqGo08fgsfdhv/IkS2iqdbevTuxb71J2bZtZL//ATkff0zuJ58QcOUV2Ewm9nz08ZHexH37EnLHHfiNHNk4k1oopXc8Ks3R75mW5kBJbq3t3KOOufeVn+IRclZ/vUbqrq0W+8ThE9dRD86aGmzgsdsWX7mPKoSow+vDWNM0XrzyXDbs+4WJn63lu4kDCbA3v3llXcXFFC5cSP7sORSnpoLTifWccyi8YizJEyeecvhRc2Xr2JGYl/5B2L33kP3Bh+R/+RUBlZVUxsUROvFeAkaPwdLmDHoTK6V3QsrdA3m7IX8fFGfVDdrqDk2nqqnaAvRmXXuQ3qwb0uFIU29NzbRW87AtUH/PUT14tyxeTLjcXxVCnAGvD2OAALuZf12XzDXvLuOxrzfw1nU9m8UzT1VlJcWpqeTP+ZbCBQtQpaWYoqMIufVW/C+7FFvHjqQvXtxig7g2S1wcUU9PIfSeu1nx/fecd+ONp/47KCuAvD3uwK21zEvX1yuL655vtNa6XxoMoR3rhmrtY9VLW2CDTt8nhBBnol4/hTRNuxB4HTACHyilXjzqeBwwHQh0n/OoUmpuA5f1rPSMC+KhUZ14cd4ffPpbOtf3a3vCc51FRRQt/pmin39GlZWhWa1oVov+tBaLtda27TjHLO6HcOvbBqvFfY4VzaIfK9+6lfw531Iwbx7OnBwMAQEEXHYZAZddir1XL8/fK21E5vBwquLi9CCuLNODNW8P5O6uFbjufaVHPQTd4gdBbSEoARKH6ENtAtvq+wJi9eE0zeCXLCGEOF2nDGNN04zAW8AIYB+wUtO02UqpLbVOewL4Qin1jqZpXYC5QHwjlPes3D4wkdQd2Tw9Zwu92gZxTqR/zTFnfj6FCxdROH8+xUuXoioqap6Z6qooR5VXoMrLz+wh4cehWSz4DhtGwGWX4jNwIAYPP6av0ZTlQ85OyN5Rs0zevQ5W5UHRobrnGq16wAa1hZhe+rImcOP12q2ErRDCC9WnZtwXSFNK7QTQNO1zYAxQO4wVUJ1sAcCBhixkQzEYNF65pjsXvf4Ld/93DbOu60zVkp/1AF6xAqqqMEVFEfTna/EbORJ7jx5oxuM/jlFVVenhXKGHtCorw1VegaooPxLa7m1XWVlNmKuKcoyhofgNH+49T7QpL4KcHe7A3QHZO49sl2TVPde/DS5DELS7AALj6waub4T0FhZCtEqaUief0k7TtKuAC5VS49zbNwL9lFL31DonCpgPBKE/n+oCpdTq43zW7cDtABEREb0+//zzhroOioqK8K3Hs0gNuXnkLl1N2bI1dM3ZhUEpqsLCKE9Opiw5mar4ts2q9lXf62psBmc59tKD2EsP4Cg5ULNuLz2ItaJuc3K5JZhSezQljihK7dGU2quXkbiM1mZzTQ3NG69Lrqnl8Mbr8rZrGjp06GqlVO/jHWuonit/Bj5WSv1T07QBwH80TeuqlKrTlquUmgpMBejdu7dqyJl9Fp9kpqCKffso/GE+hfPnU7p+PWFAYWQsn3ccTsqtV3Ph6IHNokPX8Zzsuhqcs0q/V5u1HbK3Q3aaXrvN3gGFRzV2+IRDSDtI6AkhiRDcTt8OTsRq8cGK3oHgeJr0mpqQN16XXFPL4Y3X5Y3XdCL1CeP91J3Ar417X223ARcCKKWWaZpmA0KBjIYo5Jko37mLwvl6AJdt0VvUrV06E/bX+/TxrG3j+cfU5Xy9uoDO55UQH9pCnpbTEIqz9bCtDt2sNH2Zs6vuECB7MIS0h8TB7rB1h25wItj8T/z5QgghTkt9wngl0EHTtAT0EL4WuO6oc9KB4cDHmqZ1BmxAZkMW9JSUomzrVr0G/ON8yrenAfrkE+EPP4zfyBHHPKT99T8nc/Hrv3DPZ2v4akIKVtPx7w+3SFXlerjWhG7akfCt3UvZYNbDNbQjdLoYQjvo42xD3WNthRBCNLpThrFSqkrTtHuAH9CHLU1TSm3WNO1pYJVSajbwIPC+pmn3o3fmukWd6mZ0AypcuJCQyU+xKzMTDAYcvXoR8fjj+I244KRjdGMC7bx01bnc/p/VvDjvDyZfltRURW44JTmQ+Yf+qq7hZm3Xm5tr3yXwjdQDtsvlem03tIO+DGwr42yFEMLD6vVT2D1meO5R+56stb4FOK9hi1Z/Bl9fnKGhtLn7bvwuGI4ptP6PTByZFMktKfF8tHQ3Ke1CGdElohFLehaqQzfjd8jcCpm/Q8YfUFzrToDJrgdsdA/odvWRwA1pL83KQgjRjHlFlcinb1/y7ptIjzO80f/YxeewcncOD3+5nrkTBxIdaG/YAp6Oo0K3+7ZlsPJQ3dC1+EJYJ+gwUl+Gd9aX/m1kaJAQQrRAXhHGZ8tqMvLmdT259I1f+Ovn6/h0fD9MxkYOtfrUdC2+GK1RR4XuORDQplkNvxJCCHF2JIzdEkJ9eHZsV+7/33reWLCdB0Z2atgvKDgI23+AbfNh38p61HT10F3z88+tpmu/EEK0VhLGtYxNbsPStGz+tSiN/okhpLSv/73nY7hccGANbPsBtn0Phzbo+wNiof0FeuBKTVcIIQQSxseYMjqJNem53Pe/dcy7byChvtb6v7ksH3Ys0gM47UcozgTNALH9YPhk6DgKwrtI8AohhKhDwvgoPlYTb13XkzFvLeXBL9bz0S19MBhOEJ5K6eN3q2u/6cvAVaU/lq/9BdDxQmg/XMbrCiGEOCkJ4+PoHOXPpEu7MGnWJt5YuJ2/XtDxyMGqctizVL/3u/0H/UlEoNd4B9yjB3CbPjJ2VwghRL1JYpzADf3iWJuey2s/bSfeWsTlPpv12u/OxVBRpD/uL2EQ9L9Lb34OjPN0kYUQQrRQEsYnoJXl81LEj9zj/wWJC7bpO/1j9Mk0Ol6oB7HF4dlCCiGE8AoSxkerKIYV78HS1zGW5dE2pg//yb2JGQVJPDn2anonhHi6hEIIIbyMTNdUraocVkyF13vAgil6D+g7fsE4/icuvutlCgPP4bZ/r2b74UJPl1QIIYSXkTB2VsHaT+BfvWDew/rTi26dD9d/AVHnAhDia+Xft/bFYjJw07TfOJhf6uFCCyGE8CatN4xdLtg8E97uD9/cDT6hcONMuOVbiOt3zOmxwQ4+/ksfCsuquHnab+SXVB7nQ4UQQojT1/rCWCl9WNLUQTDjFjCY4E+fwPhF0G7YSSfkSIoOYOqNvdidVcK4f6+krNLZdOUWQgjhtVpXGO9eCtMuhE+vhvJCGDsVJiyFzpfVe1aslPahvPKn7qzak8vEz9ZS5XSd+k1CCCHESbSO3tT718DCZ2DHQvCLgktegeQbwWQ5o4+79NxosgrLeWrOFiZ9s5nnx3ZFkykuhRBCnCHvDuOM32HRc/D7HLAHw8hnoc84MJ/984pvOS+Bw4XlvLN4BxH+1rqzdAkhhBCnwTvDOGcXLH4RNvxPfzzhkMf0mbJs/g36Nf83qhOZheW89tN2wvysXN+vbYN+vhBCiNbBu8K44AAseQnW/FvvmJVyL5x/f6M9qEHTNF64ohvZReVMmrWJUF8ro5IiG+W7hBBCeC/v6MBVnE3ijo/gjWQ9iHveDBPXwchnGv2JSWajgbeu78m5bQK597O1rNyd06jfJ4QQwvt4Rxhv/prYvd9A0li4ZxVc+gr4RzXZ1zssJqbd0oc2QXZu+3glWw/JLF1CCCHqzzvCuOfNrOzzBox9F4ITPFKEYB8L0//SF5vZyM3TfmN/nszSJYQQon68I4xNFkp8PP8Iw9hgB9Nv7UtxuT5LV15JhaeLJIQQogXwjjBuRjpH+TP1pt6kZ5dw2/RVlFbILF1CCCFOTsK4EQxoF8Jr1/ZgTXou98osXUIIIU5BwriRXNwtiimjk/jp98M8MWsTSilPF0kIIUQz5V3jjJuZmwbEk1FQzpuL0gj3s/LAyE6eLpIQQohmSMK4kT04siMZhWW8sTCNMH8bN/aXWbqEEELUJWHcyDRN4/mx3cguquDJbzYR5mvhwq5NNwZaCCFE8yf3jJuAyWjgzet60iM2kImfr2NpWpaniySEEKIZkTBuInaLkWk39yE2yM71H6zgmveW8c26/ZRXydAnIYRo7aSZugkF+Vj4esJ5fLYynU9XpHPf5+sI9rFwde82XN+3LXEhDk8XUQghhAdIGDexAIeZOwe34/aBifyalsV/V+zhg1928d7POxnUMYzr+8Ux/JxwTEZptBBCiNZCwthDDAaNQR3DGNQxjEP5Zfxv5V4++y2dO/6zmgh/K9f2iePavrGeLqYQQogmIGHcDEQG2Ljvgg7cPbQdC//I4L8r0nlj4Xb+tXA73cOMaNGZDGwfisGgebqoQgghGoGEcTNiMhoYmRTJyKRI0rNL+GxlOp+k7uDmab8RG2znur5tubp3G0J9rZ4uqhBCiAYkYdxMxYU4eOTCc+hpOUhZSCf+u2IPf//+D175cSsXdo3ihn5x9E0IRtOktiyEEC2dhHEzZzZojOgezWXdo0nLKOS/K9L5avU+5qw/QPtwX67vF8cVyW0IcJg9XVQhhBBnSLrstiDtw/2YfFkSK/52AS9ddS6+VhNT5myh3ws/8fCM9aRlFHq6iEIIIc6A1IxbILvFyNW9Y7m6dyyb9ufz6W/pzFyzny/X7OOirpHcPbQ9SdEBni6mEEKIepKacQvXNSaA58d2Y+mjw7h7SHt+2ZbFJW/8yrjpK1mbnuvp4gkhhKgHCWMvEexj4aFRnfj10WE8OKIjq/bkMvbtVG78cAUrdmZ7unhCCCFOQsLYywTYzdw7vANLHxnG3y4+h98PFvKnqcu55t1lLNmWiVLK00UUQghxFK8I4z0Fe/hv1n+pcFZ4uijNho/VxO2D2vHrI0OZMjqJvbkl3DTtNy5/ayk/bjksoSyEEM2IV4TxluwtLC9ezhO/PoFLuTxdnGbFZjZyc0o8Pz88lBev6EZuSSXj/72Ki17/hW83HMDpklAWQghPq1cYa5p2oaZpWzVNS9M07dETnHONpmlbNE3brGnapw1bzJO7KOEixgSOYd7ueby86uWm/OoWw2IycG3fOBY+OJhXrulOpdPFPZ+uZcSrP/PV6n1UOeWXGCGE8JRTDm3SNM0IvAWMAPYBKzVNm62U2lLrnA7AY8B5SqlcTdPCG6vAJzLcfzh+UX78Z8t/iHBEcHPSzU1dhBbBZDRwRc82jOkRw/ebDvGvhdt5cMZ6XluwjQmD23NlrxisJqOniymEEK1KfWrGfYE0pdROpVQF8Dkw5qhzxgNvKaVyAZRSGQ1bzFPTNI2H+zzMqPhRvLzqZb7d+W1TF6FFMRo0Ljk3ivoSRNYAACAASURBVHn3DeSDm3oT7GPlbzM3Mvgfi/lo6S5KK5yeLqIQQrQa9QnjGGBvre197n21dQQ6apq2VNO05ZqmXdhQBTwdBs3A8+c/T9/IvkxaOonUA6meKEaLomkaF3SJYNZdKfzntr7EhTiYMmcLA/+xkHd/3kFhWaWniyiEEF5PO1WvWk3TrgIuVEqNc2/fCPRTSt1T65xvgUrgGqANsAToppTKO+qzbgduB4iIiOj1+eefN9iFFBUV4evrC0Cpq5TXDr1GdlU290XcR6y15T4XuPZ1NZWtOU7m7KhkU7YTuwkGxpi4oK2ZcEfD9PfzxDU1BW+8LrmmlsMbr8vbrmno0KGrlVK9j3tQKXXSFzAA+KHW9mPAY0ed8y7wl1rbC4A+J/vcXr16qYa0aNGiOtuHiw+rkTNGqsGfD1bpBekN+l1N6ejrakrr0nPVxM/WqHaPfafiH/1WjZu+UqWmZSmXy3VWn+vJa2pM3nhdck0thzdel7ddE7BKnSAT61PVWQl00DQtQdM0C3AtMPuoc2YBQwA0TQtFb7beeRq/MDS4cEc474x4hypVxZ0/3klOWY4ni9MidY8N5PVrk2um2ly9J5c/v7+ci9/4lS9W7aWsUu4rCyFEQzhlGCulqoB7gB+A34EvlFKbNU17WtO00e7TfgCyNU3bAiwCHlZKeXwOxsSARN4c9iYZJRnc/dPdlFSWeLpILVKEv42HRnUi9dFh/P3Kbrhciv/7cgPnvbiQV+ZvJaOgzNNFFEKIFq1eNwGVUnOVUh2VUu2UUs+59z2plJrtXldKqQeUUl2UUt2UUg13M/gs9QjvwUuDX2JLzhYe/PlBKl3SIelM2cxG/tQnju//OpBPx/UjOS6Qfy1K47y/L+T+/61j4758TxdRCCFaJK+YgetUhsQO4cn+T/Lr/l+ZkjpFpoI8S5qmkdI+lA9u7sOiB4dwfb+2zN98iMve/JWr3kll7saDMomIEEKchlbzPOMrO15JRmkGb697m3BHOBN7TvR0kbxCfKgPT41O4oGRHZmxah8fp+7irv+uISbQzk0D2nJtnzgCHGZPF1MIIZq1VhPGAHeeeycZJRm8v/F9whxh/PmcP3u6SF7D32bmtvMTuCUlngW/H2ba0l28MO8PXvtpO1f2iuGWlATah3vPEAUhhGhIrSqMNU3j8X6Pk12azQsrXiDUHsqItiM8XSyvYjRojEyKZGRSJFsOFPDR0l18sWofnyxPZ3DHMG49P0FuEwghxFFaxT3j2kwGE/8Y9A+6h3Xn0SWPsurQKk8XyWt1ifbnpau7k/roMB4Y0ZEtBwu4edpvPPZrKVOX7CCrqNzTRRRCiGah1YUxgM1k483hb9LGrw0TF05ke+52TxfJq4X6Wpk4vANLHxnGq3/qjp9Z4/m5f9D/+QVM+GQ1i7dmyKMchRCtWqsMY4AAawDvXvAudpOdO3+6k0PFhzxdJK9nMRkYm9yGx/vb+emBQdySEs+KXTnc8tFKBv1jEa/9tI39eaWeLqYQQjS5VhvGAFG+Ubwz4h1KK0u588c7yS+XcbJNpX24H09c2oVljw3jret6khjmw+sLtnP+3xdy87TfmLfxIBVVMjxKCNE6tOowBugY1JHXh71OemE6ExdOpKxKZpNqSlaTkUvOjeI/t/VjycNDuXdYB7YdLmTCf9cw4IUFPD/3d9IyijxdTCGEaFStPowB+kT24YWBL7A2Yy2PLHkEp0vmXPaE2GAHD4zoyK+PDOOjv/ShT3ww037dxQWv/MzV76by5ep98pxlIYRXkjB2GxU/ikf6PsLCvQt5fsXzMvzGg4wGjaGdwnn3xl4se2w4j110DtlFFTw0Yz19n/uJx2duZNN+uaUghPAerWqc8alc3/l6Mksy+XDTh4Q7wrmj+x2eLlKrF+Zn5Y7B7bh9UCK/7crhfyv38uXqffx3RTpJ0f5c2yeW0T1iCLDLLF9CiJZLwvgo9/W8j8zSTN5c9ybhjnDGdhjr6SIJ9Alb+iWG0C8xhMmjk5i9bj+f/baXSd9s5tnvfueirpEM6RROv8RgogLsni6uEEKcFgnjo2iaxlMpT5Fdms2UZVOYv2c+vmZffMw++Jh98DX74jA76qxXH6+9bjFaPH0pXivAbubGAfHcOCCeTfvz+XxlOrPXHWDWugMAxAU76JsQTL+EYPonhtAmyI6maR4utRBCnJiE8XGYDWZeGfIKzyx/hp35O9lXuI/iymKKKosorarfOFizwVwT4NWv6rDuHNyZG7rcgNVobeQr8X5dYwJ4NqYbU0Z35feDBfy2K4cVu7JZ8Pthvly9D4CoABv9EoLplxhC34RgEkN9JJyFEM2KhPEJOMwOXhj4wjH7nS4nJVUlFFcWU1JZQlFl0THrx31VFZNfls/+wv38sPsHZqbNZFL/SfSL6ueBq/M+RoNG15gAusYEcOv5CbhcirTMIlbszGbFrhyW7siuqTmH+lrd4RxMv4QQOoT7YjBIOAshPEfC+DQZDUb8LH74WfzO+DNSD6Ty7PJnGTd/HKPbjebB3g8SbAtuwFIKg0GjY4QfHSP8uHFAPEopdmUVu2vOOazYmc13Gw8CEOQw0yderzn3Swimc5Q/RglnIUQTkjD2gJToFL4e/TVTN0zlo80fsWTfEh7s/SBj2o2R5tNGomkaiWG+JIb5cm3fOJRS7MstrQnm33bnMH/LYQD8rCZ6xwfRLzGEAYkhdI0JkHAWQjQqCWMPsZlsTOw5kYsTLubp5U8zaekkvkn7hkkDJpEYkOjp4jWaKlcVJa4STxcDTdOIDXYQG+zgql5tADiYX1qn5rxoayYA/jYT/RNDOK99KOe1D6FdmK/80iSEaFASxh7WPqg9H1/4MTO3z+Sfq//JVbOvYly3cdzW7Tav6+BVUlnChJ8msClzE4V/FHJNp2uaVahFBdgZ0yOGMT1iAMgsLGfZzmxS07JYuiOrpuYc4W8lpV0oKe30gI4OlKFUQoizI2HcDBg0A1d2vJLBsYN5edXLvLP+Hebtmsek/pM8XbQGU+Gs4K+L/sq6zHXEmmN5dsWzLNy7kCkpU4j0ifR08Y4rzM/K6O7RjO4eDcDenBKWpmWxdEc2S7ZlMnPtfgASQn1IaRdCYHkVPUoqCHTIsDYhxOmRMG5GQu2hvDjwRUa3G82zy5/ltvm30denL93LuhNkC/J08c5YpauSh39+mGUHl/F0ytME7gskMzqTl1e9zBXfXMFj/R7j0sRLm1Ut+Xhigx1c2zeOa/vG4XIpth4uZGlaFqk7spm1dj/FFU7eXv8jSdH+nNculJT2ofSJD8Jhkf9mQoiTk7mpm6HqDl7ju41nVfEqRs8azTdp37TI+bJdysWkpZNYuHchj/Z9lLEdxqJpGtd0uoavLvuK9kHt+duvf+OBxQ+QU5bj6eLWm8Gg0TnKn3EDE5l2Sx/WTR7J4/1s3H9BR3wsJj5aupubp/1G9ynzuea9Zbz+03ZW7c6h0imPhRRCHEt+ZW+mqjt4hWWFMc85jyeWPsE3O75hUv9JJAQkeLp49aKU4tnlz/Ldzu+YmDyR6ztfX+d4rH8sH436iOlbpvPm2jdZ880aJg+YzLC4YR4q8ZkzGw10CDIyZEgHJg7vQGmFk5W7c1i6I4vUtGxeW7CNV38CH4uRvgnBnNc+lAHtQugc6S9jnIUQEsbNXZQlio8Hf8zX27/mldWvcOXsKxnfbTy3dbutWU+5qZTin6v+yYxtMxjXbRzjzx1/3POMBiO3dr2V82PO5/FfH+e+Rfcxut1oHu376FmN5fY0u8XIoI5hDOoYBkBeSQXLd2azNC2bpTuyWPTd7wAE+1gYkBjCAHdnsPgQR7NvrhdCNDwJ4xbAoBm4quNVDIkdwksrX+Lt9W8zd9dcnhzwJH0i+3i6eMf17vp3mb5lOtedcx0Tkyee8vyOQR359OJPeXfDu3y48UN+O/Qbz5z3DP2j+jdBaRtfoMPChV2juLBrFKAPo1q2Qw/n1B1ZNROQRAfYGFCrp3ZkgM2TxRZCNBEJ4xYk1B7K3wf9nTHtxvDM8me49YdbGdNuDA/2frBZdfCavnk6b69/mzHtxvBI30fqXdMzG83cm3wvQ9oM4W+//o3x88fz53P+zP297sdu8q7hQ1EBdq7o2YYrerZBKcXubL2n9rId2Sz84zBfrdHn1U4M03tqn9culP6JIQT5NN/WECHEmZMwboFSYlKYOWYm7214j483fczP+37mod4PMbrdaI83cX6x9QteXvUyI9uOZErKFAza6fcR7BbWjS8u+4I31rzBJ79/wrIDy3j2/GfpHta9EUrseZqmkRDqQ0KoDzf0b4vLpfj9UIG75pzFzDX7+WR5OpoGXaL8a+43940Pxscq/4WF8AbyP7mFspls3NfzPn0Gr2VP88TSJ5izcw6T+08m1j/WI2Was2MOzy5/lkFtBvHiwBcxGoxn/Fl2k51H+j7C0NihPLH0CW6adxO3db2NCd0nYDaaG7DUzY/BoJEUHUBSdADjBiZS6XSxYV+efr85LYuPl+5m6pKdmAwaPWIDSWmvN2t3bxOI3XLmf+ZCCM+RMG7hOgR1YPpF0/ly25e8uvpVxs4ey4TuE7gp6SbMhqYLrQV7FjBp6ST6RPbhn4P/2WCB2TeqL1+P/pp/rPwH7298nyX7lvDc+c/RKbhTg3x+S2A2GujVNphebYNremqv2pND6g59drA3F27njQXbMRo0zon0o0dsID1iA0mOCyQxVJ5IJURLIGHsBQyagWs6XcPgNoN58bcXeW3Na8zbNY8pKVNICk1q9O9P3Z/Kw0seJik0iX8N+xc2U8N2OvK1+PL0eU8zLG4YT6U+xbXfXcvdPe7mlqRbMBla3z9hu8XIwA5hDOyg99TOL61k5a4c1u7NZd3ePGavO8B/V6QD4Gcz0b1NYE1A94gLJNTXu6ZZFcIbtL6fZF4swieCV4e+yoL0BTy//Hmum3sd13e+nnt63IPD7GiU71x9eDX3LbqPdoHteHv42432PQBDYocwc8xMnln+DK+veZ3Fexfz3PnP0da/baN9Z0sQYDdzQZcILugSAYDLpdiRWcTavXms25vHuvQ83vl5B06XPmlMmyA73WMDSXYHdNeYAGxmad4WwpMkjL3Q8Ljh9I3sy+trXuc/W/7Dgj0LeKL/EwxsM7BBv2dz1mbuXnA3Ub5RvHvBuwRYAxr0848nyBbEPwf/k7m75vLciue4es7V3N/rfv7U6U9n1FnMGxkMGh0i/OgQ4cc1vfX+A6UVTjYdyGdd+pGA/m6DPpzKZNA4J6q6eTuIHrGBJIb6SPO2EE1IwthL+Vn8eKL/E1ySeAlPpT7FXQvu4qKEi3ikzyOE2EPO+vO3527njp/uINAayNQRUxvkM+tL0zQuSbyE3hG9mbxsMs+veJ6F6Qt5/vznCXOENVk5WhK7xUif+GD6xAfX7MsoKNOD2f2atfYAnyw/0rzdIzYQn8oKDjrSiQ/Re3tH+Fs93mNfCG8kYezlksOTmXHZDD7c+CHvb3yf1AOpPNT7Ica0G3PGP1T3FOxh/PzxWA1W3h/5vseeuhThE8E7w99hxrYZvLzqZa759hpeGfIKyeHJHilPSxPub2NkUiQjk/S/P6e7eXtdeh7r9um152WHK/l+98aa99jNRtqGOEgI9SE+1IeEEH0ZH+ogzFeCWogzJWHcCliMFib0mMCo+FFMWTaFSUsn8e3Ob3my/5PE+ced1mcdLDrIuPnjcCkX00ZNI9bPM8OoqlU/dKJHeA/uX3Q/t35/Kw/1eYjrzrlOguE0GQ0aHSP86BjhxzV99L/XhYsW0bFHP3ZnlbArq4hdWSXszi5m66FCftxymCrXkYeX+FpNxIc6amrR8e6gTgj1Ichhlr8PIU5CwrgVSQxM5KMLP6oZBnXF7CtOaxhUVmkW4+aPo7iimA9HfUhiYGITlLp+OgZ15LNLP+PxXx7nxd9eZEPmBiYPmNyoHcpaA4Om0SbIQZsgB+d3CK1zrMrpYn9eKbuyitmdVczu7BJ2ZRWzYV8+czcepFZO428z1dSm24b4EB1gIyrQTlSAjcgAG/427x47LsSpSBi3MtXDoIbEDuGFFS/UDIN6KuUpuoZ2PeH78sryGD9/PJmlmUwdMZXOIZ2bsNT142/x5/Vhr/PBxg94c+2bbM/bzmtDXjvt2r+oH5PRQNsQPVw5ath3RZWLvbkl7M4q1sM6u5jdWSWs2p3L7PUHOPppoL5WE5EBNj2c/esGdXSA3R3YJqldC68lYdxKhTvC6wyDun7u9Vx3znXcm3zvMbXJoooi7vzpTtIL0nn7grfpEd7DQ6U+NYNm4PZzbycpJIlHfnmEa7+9lhcGvsDg2MGeLlqrYjEZaBfmS7sw32OOVVS5yCgs41B+GQfyyziUX8rB/DIO5pVxsKCMbYczySgsPyawHRYjUQE2otzhHB1gIzJAD+2oQBvxIT4yREu0WBLGrVztYVCf/P4JC9MX1hkGVVpVyt0L7mZrzlZeG/oa/aL6ebjE9XNezHl8fsnnPLD4Ae5ZeA93nHsHE7pPOKspOkXDsJgMNU3fJ1LpdJFRWM6h/FIO5OnBfTC/jIPu4P51exYZhWV1msI1DeKCHXQI96NjhC8dInzpEO5H+3BfCWnR7EkYizrDoKakTtGHQcVfxAO9H2By6mTWZa7j74P+3uJql2382vDvi/7Ns8uf5b0N77EpexN/H/j3JhkPLc6O2WggJtBOTKCdXieY06XS6SKzsJyD+WXszytlR0YRaRlFbDtcyOKtGTWdywzukG7vDumqnCrCDuTTLkxCWjQfEsaiRnJ4Ml9c9gUfbvqQ9ze8zw97fsClXDyd8jQXxl/o6eKdEZvJxjPnPcO5Yefywm8v8Kdv/8SrQ1712D3vCmcFRZVFBNuCT32yOCmz0UB0oJ3oQDu92tZ9hGhFlYvd2cVsO1zI9sNFbM8oZNvhopqQnrrh15qQ7hChh3THCL0WLSEtPEHCWNRhMVqY0H0Co9qO4rU1rzGozSDGdhjr6WKdlerhT52CO/HA4ge4cd6NPDngSUa3G91kZdiRt4Ovtn/FnB1zyCvPo39U/5qOdE35QI/WwmIy1AzTqq2iysWMeYsJaHsO2w4Xsf1wIdszilj4R0bNdKEGDdqG+NAh3JdzIv1IigmgW0wAUQE26UAmGo2EsTiuxMBE3hj2hqeL0aC6h3Xni0u/4OElD/P4r4+zIXMDj/R5pNEeyVhSWcIPu3/g6+1fsy5zHSaDiWGxw0gISGD2jtk8sPgBQu2hjG0/lis7XkmMb0yjlEMcYTEZiPEzMOTc6Dr7K6pc7Mpy16Qz9JDedriQn34/XHNfOtjHQlK0P13d4dw1OoDYYLsEtGgQEsaiVQmxhzB1xFReX/M6H2/+mN9zfueVwa8Q4RPRIJ+vlGJL9ha+2v4Vc3fNpbiymISABB7q/RCXJl5aM23ohO4TWHpgKTO2zeDDTR/ywcYPSIlJ4eoOVzM4dnCrfBqVJ1lMBjpF+tEpsm5NurTCye+HCti8P5+N+/PZtL+A95fsrLkf7Wcz0TU6gK4xekh3jQkgIUTm9Ranr17/4zVNuxB4HTACHyilXjzBeVcCXwJ9lFKrGqyUQjQgk8HEg70fpGtoVyYtncQ1317Dy4Nfpk9knzP+zPzyfJYULuHNOW+yNXcrNqONkfEjubLDlSSHJx9TezIajAxqM4hBbQZxqPgQX2//mq+2f8VfF/+VMHsYYzuM5aoOVxHlG3W2l9ssKKVYdnAZ0zdPJy03jTt73MmVHa5s9g/3sFuM9IwLomfckXvS5VVOth0qYtMBPaA3789n+rI9VFS5APCxGOkS7U9StLsGHRNAuzAfTMbmfa3Cs04ZxpqmGYG3gBHAPmClpmmzlVJbjjrPD7gPWNEYBRWioY2KH0X7wPb8ddFfGT9/PPf3up+butxU72ZHpRSrD6/mq+1f8eOeHyl3ltM5uDNP9HuCixMvxs/id+oPASJ9Irmrx13cfu7t/LLvF2Zsm8H7G97n/Q3vc37M+Vzd8WoGthnYImvLla5Kvt/1PdM3T2dr7lZC7aFE+0bz9LKn+W7nd0weMJmEgARPF/O0WE1GurUJoFubAP7s3lfpdJGWUVQTzpsOFPC/lXv5OHW3+z0GOkf50zXGn06R/vjbTPhYTDisRnwsJnysRhwWEw6LvrSYJLhbm/r87+4LpCmldgJomvY5MAbYctR5zwB/Bx5u0BIK0YjaBbbjs0s+44mlT/DyqpfZmLWRp1OePuk0mlmlWczeMZuZ22eyu2A3vmZfLm9/OW3z23LjqBvPuCwmg4mhcUMZGjeUA0UH+Gr7V8zcPpOJiyYS4Yjgig5XcEWHKzz2YI7TUVRRxFfbv+I/W/7D4ZLDJAYk8nTK01ySeAlmg5lZabN4adVLXDX7Ku7ofgd/SfpLo927bwpmox62naP8wf3YSqdLsSurqKZ5e9P+fGatPUBReXo9Pk/DYTHhYzFitxjxsepBrQd4rf21Av3wwSp8ducQ4Wcj3N8qPcJbmPqEcQywt9b2PqDOzA+apvUEYpVS32maJmEsWhRfiy+vDnmVaZum8cbaN0jLTePVoa/WqbE5XU5SD6Ty9favWbx3MVWqip7hPRnXbRwj40diN9lZvHhxg5Up2jeae5Pv5c7ud7Jk7xJmbJvBu+vf5b0N7zEoZhBXd7qa86LPa3aTmBwqPsSnv3/KjG0zKKosok9kH54c8CTnx5xfp0l6bIexDGwzkBd/e5F/rf0X83bNY0rKFM4NO9eDpW9YRoNG+3A/2of7Mdb9IDGXS5FRWE5ReRUlFVUUlzspqaiipMJZZ7u4wklphZPicv1YcUUVJeVODheWUZJ1ZLu4oqrOxCdvr19Wsx5gNxPhbyXC30a4n40Ifyvhfu5t/+ptm9TCmwlNHT3n3NEnaNpVwIVKqXHu7RuBfkqpe9zbBmAhcItSaremaYuBh453z1jTtNuB2wEiIiJ6ff755w12IUVFRfj6Hjv1XkvnjdfVnK/pj9I/mJ41nSpVxQ2hNxBriWV50XKWFy0n15mLr8GXvr59GeA7gEhz3RpqY19XVmUWqUWpLC9aTqGrkCBjECm+KQzwHUCAqXEmMqnvNe2v2M+CggWsLl6NQpHsSGa4/3DirKeeF3xjyUa+yPmCfGc+g/0Gc2ngpVgN1oYo/nE1539/p0spRaULypxwOLeYcqOdvHIXeWWK3HJFXrkir8y9LFc4j/Pj3s8MgTYDgVZNf9k0gtzrwTaNCB8DdpNnOqR5098VwNChQ1crpXof71h9wngA8JRSapR7+zEApdQL7u0AYAdQ5H5LJJADjD5ZJ67evXurVasaro/X4sWLGTJkSIN9XnPhjdfV3K/pYNFB7l98P5uzN6Oh/xAaED2AKztcydDYoSdsTm2q66p0VrJo7yJmbJvB8oPLMWp6Z7D+Uf1JCk2iU1AnbCZbg3zXya5JKcXyg8v5ePPHpB5IxW6yc0WHK7ih8w208WtzWt9TVFHEa2te44utXxDpE8mk/pNqpmRtaM3939+ZOtV1uVyK3JIKDheUc7iwjIyCMn3dvcwoLONwQRmZheV1atsAIT4W2oY43A8G0R+TWb3dmI/H9La/K03TThjG9WmmXgl00DQtAdgPXAtcV31QKZUP1Dxb7WQ1YyFagijfKKZfNJ1pG6fhwsXl7S9vVmOAzUYzI+NHMjJ+JOkF6Xy5/Uu+3fEti/YuAsCoGUkMTCQpJIkuIV3oEtKlQQP66E5ZIbYQJiZP5JpO15zxVKO+Ft+aKVmfSn2KuxbcxcUJF/NI30dktrIGYjBohPhaCfG10gX/E57ndCmyi8o5XFDOvtwS9uSUsCe7hD3Zxfy2K4dZ6/bXeYiHn81UK5z1gI4P8SE+xEGYn1XGYdfTKcNYKVWlado9wA/oQ5umKaU2a5r2NLBKKTW7sQspRFOzGq1M6DHB08U4pTj/OB7o9QD397yfwyWH2Zy9mc1Zm9mSs4Ul+5YwK20WoAd0u8B2dAnpUhPSHYM6nlZAH90pKyEggSkpU7g08VIsRkuDXE9yeDIzLpvBhxs/ZOrGqaQeSOX/+vwflyZeKj/Um4jRoBHuvq/crc2xv1yVVTrZl1vKnmz9GdbVy03785m36VDNTGYAdrOxJqTj3Y/bjAt2EBlgJdzfhp9VHotZrV5jJZRSc4G5R+178gTnDjn7YgkhToemaUT6RBLpE8nwuOGA3ox8qPgQW7K3sDlbD+if9/58TEDXqUEHd8JqrHu/9uhOWb0jetc0IzfGOGGL0cKEHhMYGT+SyamT+duvf2POjjk8OeDJ027+Fg3PZjbSPtyX9uHH3sutdLo4kFfK7uwS0muF9Y7MYhZtzawZi13NbjYS7m8lws9GmHsZ7m+t6Vx2oMhFfmllq3iWdcsbuCiEqBdN04jyjSLKN4rhbU8Q0NlbWLx3MTPTZgJg0kw1NejOIZ35Kesn1ny1BhcuRrQdwS1Jt9A1tGuTlL9dYDv+fdG/+WLrF7y6+lWumH0Fd/e4m+s7X98ix1y3BmajwX1f2QcIq3PM5VIcKihjT3YJGYVlZNTcp9aXvx8o4OfCTIrKq+q872+/zsdmNtTqEa4Hdu3tCH8rdosRo0HDqGkYDBoGrXpdr+0bNK3O8eZG/kUL0YqcKKAPFh9kS/aWmteivYuYmTYTi2bhmk7XcGOXGz1SKzVoBq4951qGxA7hueXP8fKql2uGQXUK7tTk5TlT5c5yNmdtZm3GWnbl7yLcEU60bzTRvtHE+MYQ5RPVYE39zZXBoNU8ZetkisuryCjUO5YtXrGW0DaJNdsZBeX8fqiAJdvKKTwqtE9XdTBrGnVC+khwQ6Ddwg/3Dzqr76kvCWMhWjlN02qC4YK2FwBHatDrflvHRf0u8nAJ9VnK3hj2BvP3zOf5Fc/zp2//xC1Jt3Bn9zsbrGNaQ8opy2FdxjrWc2sY8wAAHj9JREFUZaxjTcYatmRvodJVCUCoPZTcslycyllzvoZGmCOMGN8Y/e/CJ7pmvTqsW/KkKKfDx2oiwWoiIdSHsnQTQwYmHve8kooqMty9wTMKyymtdOJyKZxK6UuXPpSrep/TpVBK4XRx5Jw656pa5+rvs5qbbgy2hLEQ4hjVNeithq2eLkoNTdMYFT+K/lH9+eeqf/Lhpg/5cc+PTB4wmb5RfT1WLqUUuwt2sy5jHWsz1rI2Yy27C3YDYDaYSQpJ4oYuN5AclkyP8B4E2YKoclWRWZLJ/qL9HCg+oC+L9OW6jHV8X/z9MWEd7gg/Eta+tcLaJ6ZFzMrW0BwWE/GhJuJDfTxdlAYhYSyEaFECrAE8fZ4+teaUZVO4bf5tXNHhCobFDiPQFkiQNYgA6/+3d+fhUZV3/8ffdxYIkhASlgABBauWLUIEEVAgSHG7ENSKKVUL8YFeaIuttipF2/JUbK1Y7fJwqdRHBIs/QJSf/lwrDyBScUEadhp5WCTIEkIISSVkmfv3xyzMTGaSAZKcmfHzuq5cc865l/O95yT5zjlzlnTSWqU1ywlm1XXVbC/d7ku8hUcKKTtV5ostt1MuN110E5dlXUbfDn3rnRAH7lufer8uCKXWVcuRr4/4krQ3UX/176/455F/8s6ed+ol68ykTPqs7EOv9F7un3bu18yUzLg/+SkeKBmLSEy6ousVvDb+NZ7Z9AwLty3ktS9eCyhPNImkt04no3UG7VPa0761++dE2Qn2bdtHRkqGb5m3Tmpyar3EdbzqOIUlhb7Eu/XoVqpd1QCcn3Y+I7uPJLdzLrmdc+mZ3rNJPgAkJST59oBDqXXVcvjrw74kfaDyAJ9+8SlHTx5lw6ENVNVV+eq2a9XudIL2S9LZadkkJ3wzDn3HAiVjEYlZKUkp3DfoPu7seyeH/n2Isqoyjp867nv1/pRVlbHvxD42ndrEsZPHeH/D+yH7SzJJ7gTuSdTHqo6xu3y3uywhib6ZfZnUexK5nXMZ0HkAHdt0DNlPc0tKSCI7NZvs1Gwux/3oz37H+5GXl4fLujj070PsKd/DnvI97D2xlz3le/jHgX/4Lmvz9nF+2vn0bNczIFn3TO9Ju1bhbwoizUPJWERiXsc2HSNOjKtXr2bwlYM5XnWcslNlAQnb+1p+qpxjVcfITs1m3IXjyO2cS/+O/aPyZLFgCSbBt1d9ZfaVAWUV1RXsLd/LnhN7fMl6T/ke1havpdaePju5Y5uO9ErvRac2nZr8EHdyQjIXtb/IfflcZh9SW8XPvafPhZKxiHyjGGNIa5VGWqs0etDD6XBaVFqrNHI65ZDTKSdgeY2rhgMVB9zJ2ZOod5fvZsvRLU0ew8nakwF76Be0u4C+mX19N57p06FPxM8CjydKxiIi33DJCcn0TO9Jz/SejGZ0s6+v9GQpO47t8F3XXlhSyDt73/GV90jrQd8OfUkpTyHlYAp9Mvuc9X3PY4WSsYiItKgObTpwVfZVXJV9lW/Zsapj7Ch1J+gdx3aw9ehWDlQe4PW/vw5A99TuAXvP/Tr0O6sEfaruFBXVFVRUV1BZXUlFjee1uoLKmsBXg2HOVXOabNwNUTIWERHHZaZkcmX2lQHfc7/1P2+R0SfDtwe9rXQbf9/3d195dmq2L0GnJKa4k6wnuVbWVHKi+oRv2puAvTdfCcdgSE1OJa1VGh3Pa7kT9JSMRUQkKrVNbMvwbsMZ3m24b1n5qfKAQ9zbS7fz/r7TZ8e3SWrjPicgOY3UVqlkpGRwftr5pLZKJbVVKu1atSM1uf50WrL7PILzks9rluvTG6NkLCIiMSO9dTpDuw5laNehvmUV1RW4rIu2yW1j9iEisRm1iIiIRzycfd3y++IiIiISQMlYRETEYUrGIiIiDlMyFhERcVhUncBVU1NDcXExVVVVjVcOkp6ezo4dO5ohKmc5Oa6UlBS6d+9OcrKe7CIi0pyiKhkXFxeTlpZGz549z/jm5BUVFaSlxf4ZdcGcGpe1ltLSUoqLi+nVq1eLr19E5Jskqg5TV1VV0aFDBz0IOwoYY+jQocNZHaUQEZEzE1XJGFAijiLaFiIiLSPqkrHTUlP1bE0REWlZSsYiIiIOUzIOw1rLAw88QP/+/cnJyWHp0qUAHDx4kJEjRzJw4ED69+/Phx9+SF1dHVOmTPHVffrppx2OXkREYklUnU3t7z//3za2f3Ui4vp1dXUkJiY2WKdvt3b8+sZ+EfX32muvUVhYyKZNmzh69CiXX345I0eO5OWXX+baa6/l4Ycfpq6ujq+//prCwkIOHDjA1q1bATh+/HjEcYuIiGjPOIx169YxadIkEhMTycrKYtSoUXz22WdcfvnlLFiwgNmzZ7NlyxbS0tK48MIL2b17NzNmzODdd9+lXbt2TocvIiIxJGr3jCPdg/VqqetxR44cydq1a3nrrbeYMmUK999/Pz/4wQ/YtGkT7733Hs8++yzLli3jhRdeaPZYREQkPmjPOIwRI0awdOlS6urqKCkpYe3atQwZMoR9+/aRlZXFtGnTmDp1Khs3buTo0aO4XC6++93vMmfOHDZu3Oh0+CIiEkOids/YaTfffDPr169nwIABGGN44okn6NKlCwsXLmTu3LkkJyeTmprKokWLOHDgAAUFBbhcLgB+97vfORy9iIjEEiXjIJWVlYD7hhdz585l7ty5AeWTJ09m8uTJ9dppb1hERM6WDlOLiIg4TMlYRETEYUrGIiIiDlMyFhERcZiSsYiIiMOUjEVERBymZCwiIuIwJWOH1NbWOh2CiIhECSXjEG666SYGDRpEv379mD9/PgDvvvsul112GQMGDGDMmDGA+wYhBQUF5OTkcOmll/Lqq68CkJqa6utr+fLlTJkyBYApU6Ywffp0rrjiCh588EE+/fRThg0bRm5uLsOHD+df//oX4H4C1c9//nP69+/PsGHD+Mtf/sKqVau46aabfP2+//773HzzzS3xdoiISDOL3jtwvTMTDm2JuHqbulpIbGQ4XXLg+scb7euFF14gMzOTkydPcvnllzNhwgSmTZvG2rVr6dWrF8eOHQPg0UcfJT09nS1b3HGWlZU12ndxcTEfffQRiYmJnDhxgg8//JCkpCRWrlzJrFmzePXVV5k/fz579+6lsLCQkydPUlNTQ0ZGBvfccw8lJSV06tSJBQsWcNdddzX+xoiISNSL3mTsoD//+c+sWLECgP379zN//nxGjhxJr169AMjMzARg5cqVLFmyxNcuIyOj0b4nTpzoe+5yeXk5kydP5osvvsAYQ01Nja/f6dOnk5SUFLC+O++8k7/97W8UFBSwfv16Fi1a1EQjFhERJ0VvMo5gD9bfySZ6hOKaNWtYuXIl69ev57zzziMvL4+BAweyc+fOiPswxvimq6qqAsratm3rm/7lL3/J6NGjWbFiBXv37iUvL6/BfgsKCrjxxhtJSUlh4sSJvmQtIiKxTd8ZBykvLycjI4PzzjuPnTt38vHHH1NVVcXatWvZs2cPgO8w9dixY5k3b56vrfcwdVZWFjt27MDlcvn2sMOtKzs7G4AXX3zRt3zs2LE899xzvpO8vOvr1q0b3bp1Y86cORQUFDTdoEVExFFKxkGuu+46amtr6dOnDzNnzmTo0KF06tSJ+fPnc8sttzBgwADy8/MBeOSRRygrK6N///4MGDCA1atXA/D4448zbtw4hg8fTteuXcOu68EHH+QXv/gFubm5AWdXT506lfPPP59LL72U4cOH8/LLL/vKbr/9dnr06EGfPn2a6R0QEZGWFtFxTmPMdcCfgETgeWvt40Hl9wNTgVqgBLjLWruviWNtEa1bt+add94JWXb99dcHzKemprJw4cJ69W699VZuvfXWesv9934Bhg0bRlFRkW9+zpw5ACQlJfHUU0/x1FNPURF0+H3dunVMmzYt4vGIiEj0a3TP2BiTCMwDrgf6ApOMMX2Dqv0TGGytvRRYDjzR1IEKDBo0iM2bN3PHHXc4HYqIiDShSPaMhwC7rLW7AYwxS4AJwHZvBWvtar/6HwPKFs3g888/dzoEERFpBsZa23AFY24FrrPWTvXM3wlcYa39cZj6/wUcstbOCVH2Q+CHAFlZWYP8LwsCSE9P56KLLjqbcVBXV+e7ZCieOD2uXbt2UV5e3qR9VlZWBtwYJV7E47g0ptgRj+OKtzGNHj36c2vt4FBlTXptjDHmDmAwMCpUubV2PjAfYPDgwTb4Up4dO3ac9eVJwd+txgunx5WSkkJubm6T9rlmzZpGL+OKRfE4Lo0pdsTjuOJxTOFEkowPAD385rt7lgUwxnwHeBgYZa091TThiYiIxL9ILm36DLjYGNPLGNMK+B7whn8FY0wu8Bww3lp7pOnDFBERiV+NJmNrbS3wY+A9YAewzFq7zRjzG2PMeE+1uUAq8IoxptAY80aY7kRERCRIRN8ZW2vfBt4OWvYrv+nvNHFcMSE1NZXKysqQZXv37mXcuHFs3bq1haMSEZFYoztwiYiIOCxqnzTw+09/z85jkT+cIZJLgHpn9uahIQ+FLZ85cyY9evTgRz/6EQCzZ88mKSmJ1atXU1ZWRk1NDXPmzGHChAkRxwXuh0XcfffdbNiwwXd3rdGjR7Nt2zYKCgqorq7G5XLx6quv0q1bN2677TaKi4t9zzX2Pg9ZRETiU9QmYyfk5+fz05/+1JeMly1bxnvvvce9995Lu3btOHr0KEOHDmX8+PEBT2ZqzLx58zDGsGXLFnbu3Mk111xDUVERzz77LD/5yU+4/fbbqa6upq6ujrfffptu3brx1ltvAe7nH4uISHyL2mTc0B5sKE1xPW5ubi5Hjhzhq6++oqSkhIyMDLp06cJ9993H2rVrSUhI4MCBAxw+fJguXbpE3O+6deuYMWMGAL179+aCCy6gqKiIYcOG8dhjj1FcXMwtt9zCxRdfTE5ODj/72c946KGHGDduHAMHDjynMYmISPTTd8ZBJk6cyPLly1m6dCn5+fksXryYkpISPv/8cwoLC8nKyqr3jOKz9f3vf5833niDNm3acMMNN7Bq1SouueQSNm7cSE5ODo888giPP35mz3UWEZHYE7V7xk7Jz89n2rRpHD16lA8++IBly5bRuXNnkpOTWb16Nfv2nfnDqEaMGMHixYu5+uqrKSoq4ssvv+Tb3/42u3fv5sILL+Tee+/lyy+/ZPPmzfTu3ZvMzEzuuOMO2rdvz7PPPtsMoxQRkWiiZBykX79+VFRUkJ2dTdeuXbn99tu58cYbycnJYfDgwfTu3fuM+7znnnu4++67ycnJISkpiRdffJHWrVuzbNkyXnrpJZKTk+nSpQuzZs3is88+44EHHiAhIYHk5GSefPLJZhiliIhEEyXjELZs2eKb7tixI+vXrw9ZL9w1xgA9e/b0XWOckpLCggUL6tWZOXMmM2fODFh27bXXcu211/rmKyoqzih2ERGJPfrOWERExGHaMz5HW7Zs4c477wxY1rp1az755BOHIhIRkVijZHyOcnJyKCwsdDoMERGJYTpMLSIi4jAlYxEREYcpGYuIiDhMyVhERMRhSsbnIDU11ekQREQkDigZx4Ha2lqnQxARkXMQtZc2Hfrtbzm1I/LnGdfW1XGskecZt+7Tmy6zZoUtb8rnGVdWVjJhwoSQ7RYtWsSTTz6JMYZLL72Ul156icOHDzN9+nR2794NwDPPPEO3bt244YYb2L59OwBPPvkklZWVzJ49m7y8PAYOHMi6deuYNGkSl1xyCXPmzKG6upoOHTqwePFisrKyqKysZMaMGWzYsAFjDL/+9a8pLy9n8+bN/PGPfwTgr3/9K9u3b+fpp59u/I0WEZEmF7XJ2AlN+TzjlJQUVqxYUa/d9u3bmTNnDh999BEdO3bk2LFjANx7772MGjWKFStWUFdXR2VlJWVlZQ2uo7q6mg0bNgBQVlbGxx9/jDGG559/nieeeII//OEPPProo6Snp/tu8VlWVkZycjKPPfYYc+fOJTk5mQULFvDcc8+d69snIiJnKWqTcUN7sKFE2/OMrbXMmjWrXrtVq1YxceJEOnbsCEBmZiYAq1atYtGiRQAkJiaSnp7eaDLOz8/3TRcXF5Ofn8/Bgweprq6mV69eAKxcuZIlS5b46mVkZABw9dVX8+abb9KnTx9qamrIyck5w3dLRESaStQmY6d4n2d86NChes8zTk5OpmfPnhE9z/hs2/lLSkrC5XL55oPbt23b1jc9Y8YM7r//fsaPH8+aNWuYPXt2g31PnTqV3/72t/Tu3ZuCgoIziktERJqWTuAKkp+fz5IlS1i+fDkTJ06kvLz8rJ5nHK7d1VdfzSuvvEJpaSmA7zD1mDFjeOaZZwCoq6ujvLycrKwsSkpKKC0t5dSpU7z55psNri87OxuAhQsX+paPHTuWefPm+ea9e9tXXHEF+/fv5+WXX2bSpEmRvj0iItIMlIyDhHqe8YYNG8jJyWHRokURP884XLt+/frx8MMPM2rUKAYMGMD9998PwJ/+9CdWr15NTk4OgwYNYvv27SQnJ/PQQw8xZMgQxo4d2+C6Z8+ezcSJExk0aJDvEDjAI488QllZGf3792fAgAGsXr3aV3bbbbdx5ZVX+g5di4iIM3SYOoSmeJ5xQ+0mT57M5MmTA5ZlZWXx+uuv16t799138+CDD9ZbvmbNmoD5CRMmhDzLOzU1NWBP2d+6deu47777wg1BRERaiPaMv4GOHz/OJZdcQps2bRgzZozT4YiIfONpz/gcxeLzjNu3b09RUZHTYYiIiIeS8TnS84xFRORc6TC1iIiIw5SMRUREHKZkLCIi4jAl4yB6LKKIiLQ0JeMI6BGFIiLSnJSMw1izZg0jRoxg/Pjx9O3b1+lwREQkjkXtpU0fLivi6P7wd7gKVldXR2IjzzPu2COVEbddEnGfGzduZOvWrb4nIImIiDQH7Rk3YMiQIUrEIiLS7KJ2z/hM9mChaZ5nHMz/EYUiIiLNRXvGIiIiDlMyFhERcVjUHqZ2ivexiHl5eeTl5TkbjIiIfCNoz1hERMRhSsYiIiIOUzIWERFxWNQlY2ut0yGIh7aFiEjLiKpknJKSQmlpqZJAFLDWUlpaSkpKitOhiIjEvag6m7p79+4UFxdTUlJyxm2rqqriMnE4Oa6UlBS6d+/uyLpFRL5JIkrGxpjrgD8BicDz1trHg8pbA4uAQUApkG+t3XumwSQnJ5/V7Sf37zzGhrcK6dylMwkJBpNgQr8mGhISwIQoS0g0GON59ZURUM+YwOWBfYAxpt5yk4B72gTN+/WHIaC+STAkGPfyDz74gNzc3DN+T0REJHY0moyNMYnAPGAsUAx8Zox5w1q73a/afwBl1tqLjDHfA34P5DdHwKFUf13LqRNwtKYSl8ti6yzWWlx+ry6Xxbq8r2BdMXIo3MCOV1b7krwx/skdX1L3Jnm8HxhMYJkxYZb59eluS4g+ApcR3L+h/nr8+jb+HzgMlOyxfF6111P3dDtMcJxBy4P6D6jvnffGSGAf3jGEnffGAIHx4C6D0/XA257TYzBQc9Ly9YnqoPgC1+NdZghTLiLfSJHsGQ8BdllrdwMYY5YAEwD/ZDwBmO2ZXg78lzHG2Bb68vdbl3Vm/4kE8vKGRtzGWou1YOslar9X/6TucrfxlgXM11lc1m+5tw+/DwQBy12edXvmsfgtP13ucln27tnL+T3O9yvDb12n+/Fv41ufX18B0/591bmCluOJyeJyATZwWch+/doGzltwnV7m78iW3U37SxAlil5fd859+Cdo3wcCv+TvyePuDzmESPxQr07wBwtvvdN9eBb7zxvDvytdHP7Hp6c/qBi/9t51+sXniwcCP7hQP76ADzV+nRv/fnxljdT3/4DjeyPrvx8Yw8GDLtYc2BnwPgT0GbDO0/0G9u3Xzn+FwW19gwlq6/e5ywSuLKjMf8YTS5h6pbssm+r214up/roIaBxyfWHiO90+qFqo/oLfPz/BHzyD23jbnSi27C4sCVPXBMVAPcHjCVXXBBf6zSYkGrpdnFG/42YQSTLOBvb7zRcDV4SrY62tNcaUAx2Ao00RZHPw7R0lGBp+8KKzTq75kmF533I6jHNmrftDh7WWNWs+YOTIkb6kjX8S90v4hEn2BNWrtzxg3jvtqe+NxduW+jF4p13eoycBZd7+T59t7o2tqKiIiy+62Dde6zo99lAxYt2dB/TnXaXLeooDY7Lgjp3T4/PFFFDf295vvcFj8atv/WLxr1PtqiS1fWu/tt4JTr93BC339O2qs6fj4fQ6vfGdbuf3Xga9Lw3W9xtzYNvT749/e2+I1dWwu+RovXb12hDYzvd5Muj9DFhfUD3/Oi3h0D+/aLmVtZD967Y4tu7W5yUx9amRLbKuFj2ByxjzQ+CHntlKY8y/mrD7jkRx8j8H8TiueBwTxOe4NKbYEY/jcnxM055u0u4uCFcQSTI+APTwm+/uWRaqTrExJglIx30iVwBr7XxgfgTrPGPGmA3W2sHN0beT4nFc8TgmiM9xaUyxIx7HFY9jCieS64w/Ay42xvQyxrQCvge8EVTnDWCyZ/pWYFVLfV8sIiIS6xrdM/Z8B/xj4D3clza9YK3dZoz5DbDBWvsG8N/AS8aYXcAx3AlbREREIhDRd8bW2reBt4OW/cpvugqY2LShnbFmOfwdBeJxXPE4JojPcWlMsSMexxWPYwrJ6GiyiIiIs6Lq3tQiIiLfRDGXjI0x1xlj/mWM2WWMmRmivLUxZqmn/BNjTM+Wj/LMGGN6GGNWG2O2G2O2GWN+EqJOnjGm3BhT6Pn5Vai+ookxZq8xZosn3g0hyo0x5s+ebbXZGHOZE3FGyhjzbb/3v9AYc8IY89OgOjGxnYwxLxhjjhhjtvotyzTGvG+M+cLzGvJuB8aYyZ46XxhjJoeq44QwY5prjNnp+f1aYYxpH6Ztg7+rTgozrtnGmAN+v2c3hGnb4P9Lp4QZ01K/8ew1xhSGaRu12+qcuC/uj40f3CeQ/S9wIdAK2AT0DapzD/CsZ/p7wFKn445gXF2ByzzTaUBRiHHlAW86HesZjmsv0LGB8huAd3Df7GYo8InTMZ/B2BKBQ8AFsbidgJHAZcBWv2VPADM90zOB34dolwns9rxmeKYznB5PA2O6BkjyTP8+1Jg8ZQ3+rkbhuGYDP2+kXaP/L6NpTEHlfwB+FWvb6lx+Ym3P2HdrTmttNeC9Nae/CcBCz/RyYIyJ8pv+WmsPWms3eqYrgB2472oW7yYAi6zbx0B7Y0xXp4OK0Bjgf621+5wO5GxYa9fivvLBn//fzkLgphBNrwXet9Yes9aWAe8D1zVboGcg1JistX+31tZ6Zj/GfZ+EmBJmW0Uikv+XjmhoTJ7/17cB/6dFg3JYrCXjULfmDE5aAbfmBLy35owJnsPqucAnIYqHGWM2GWPeMcb0a9HAzo4F/m6M+dxz97VgkWzPaPU9wv+ziLXt5JVlrT3omT4EZIWoE8vb7C7cR2JCaex3NRr92HP4/YUwXynE6rYaARy21oa7t2csbqtGxVoyjmvGmFTgVeCn1toTQcUbcR8SHQD8Bfi/LR3fWbjKWnsZcD3wI2NMy9zktZl5bn4zHnglRHEsbqd6rPt4YNxcamGMeRioBRaHqRJrv6vPAN8CBgIHcR/WjReTaHivONa2VURiLRmfya05MQ3cmjPaGGOScSfixdba14LLrbUnrLWVnum3gWRjTMcWDvOMWGsPeF6PACtwHzbzF8n2jEbXAxuttYeDC2JxO/k57P2awPN6JESdmNtmxpgpwDjgds+HjHoi+F2NKtbaw9baOmutC/groeONxW2VBNwCLA1XJ9a2VaRiLRnH5a05Pd+R/Deww1r7VJg6XbzffRtjhuDedlH7IcMY09YYk+adxn0izdagam8AP/CcVT0UKPc7TBrNwn5yj7XtFMT/b2cy8HqIOu8B1xhjMjyHRq/xLItKxpjrgAeB8dbar8PUieR3NaoEnVtxM6HjjeT/ZbT5DrDTWlscqjAWt1XEnD6D7Ex/cJ+BW4T7LMGHPct+g/uPDSAF9+HDXcCnwIVOxxzBmK7CfUhwM1Do+bkBmA5M99T5MbAN9xmRHwPDnY67kTFd6Il1kydu77byH5MB5nm25RZgsNNxRzCutriTa7rfspjbTrg/TBwEanB/l/gfuM+t+B/gC2AlkOmpOxh43q/tXZ6/r11AgdNjaWRMu3B/b+r9u/JeadENeLuh39Vo+Qkzrpc8fzObcSfYrsHj8szX+38ZDT+hxuRZ/qL3b8mvbsxsq3P50R24REREHBZrh6lFRETijpKxiIiIw5SMRUREHKZkLCIi4jAlYxEREYcpGYuIiDhMyVhERMRhSsYiIiIO+/+fkjEf5QnkyAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "gKcvEXd31_8A", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "e89798bc-3b3f-4d6c-91d7-c5b32cbd922d" }, "source": [ "y_pred = model.predict(X_test)\n", "y_pred = y_pred.argmax(axis=1)\n", "print(classification_report(y_test, y_pred,\n", " target_names=class_names))" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " T-shirt/top 0.82 0.84 0.83 1000\n", " Trouser 0.99 0.96 0.98 1000\n", " Pullover 0.82 0.74 0.78 1000\n", " Dress 0.86 0.91 0.88 1000\n", " Coat 0.79 0.79 0.79 1000\n", " Sandal 0.99 0.94 0.97 1000\n", " Shirt 0.65 0.69 0.67 1000\n", " Sneaker 0.93 0.97 0.95 1000\n", " Bag 0.98 0.97 0.98 1000\n", " Ankle boot 0.95 0.96 0.96 1000\n", "\n", " accuracy 0.88 10000\n", " macro avg 0.88 0.88 0.88 10000\n", "weighted avg 0.88 0.88 0.88 10000\n", "\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "ItQpQ2RtbYeF" }, "source": [ "# Convolutional Neural Networks - Building CNN Classifiers\r\n", "\r\n", "\r\n", "The most popular deep learning models leveraged for computer vision problems are convolutional neural networks (CNNs)!" ] }, { "cell_type": "markdown", "metadata": { "id": "6iyNfby1bZYC" }, "source": [ "# Brief on CNNs\r\n", "\r\n", "CNNs typically consist of multiple convolution and pooling layers which help the deep learning model in automatically extracting relevant features from visual data like images. Due to this multi-layered architecture, CNNs learn a robust hierarchy of features, which are spatial, rotation, and translation invariant.\r\n", "\r\n", "![](https://miro.medium.com/max/1456/1*NKL76WYQwH5LuqyaQTjBNw.png)\r\n", "\r\n", "The key operations in a CNN model are depicted in the figure above. Any image can be represented as a tensor of pixel values. The convolution layers help in extracting features from this image (forms feature maps). Shallower layers (closer to the input data) in the network learn very generic features like edges, corners and so on. Deeper layers in the network (closer to the output layer) learn very specific features pertaining to the input image. The following graphic helps summarize the key aspects of any CNN model.\r\n", "\r\n", "![](https://miro.medium.com/max/1366/1*nCQeDMjKoTGst1RiCDo9Fw.png)\r\n", "\r\n", "We will be building a CNN from scratch in this tutorial. " ] }, { "cell_type": "code", "metadata": { "id": "ESNsiHVZvhcY", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "33f8936a-5e46-401a-b25e-eb6acfcdff57" }, "source": [ "X_train.shape" ], "execution_count": 22, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "(60000, 28, 28)" ] }, "metadata": { "tags": [] }, "execution_count": 22 } ] }, { "cell_type": "markdown", "metadata": { "id": "gzH0Vk6hSvJt" }, "source": [ "## Reshaping Image Data for Modeling\n", "\n", "We do need to reshape our data before we train our model. Here we will work on grayscale, 1-channel images (image pixel tensors)" ] }, { "cell_type": "code", "metadata": { "id": "RypFy7KkS5-0", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "4c275940-155a-4959-ba53-b0a8b0c8421f" }, "source": [ "# reshape for feeding into the model\n", "train_images = X_train.reshape(X_train.shape[0], 28, 28, 1)\n", "test_images = X_test.reshape(X_test.shape[0], 28, 28, 1)\n", "\n", "print('\\nTrain_images.shape: {}, of {}'.format(train_images.shape, train_images.dtype))\n", "print('Test_images.shape: {}, of {}'.format(test_images.shape, test_images.dtype))" ], "execution_count": 25, "outputs": [ { "output_type": "stream", "text": [ "\n", "Train_images.shape: (60000, 28, 28, 1), of float64\n", "Test_images.shape: (10000, 28, 28, 1), of float64\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "8h7Lc6PTTeLn" }, "source": [ "## Build CNN Model Architecture\n", "\n", "We will now build our basic 2-layer CNN model architecture." ] }, { "cell_type": "code", "metadata": { "id": "VynDIiOaUXL3", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "6f17d6c5-5064-4e6a-ce40-33e3cbc87dc3" }, "source": [ "# define input shape\n", "INPUT_SHAPE = (28, 28, 1)\n", "\n", "# define sequential model\n", "model = tf.keras.models.Sequential()\n", "# define conv-pool layers - set 1\n", "model.add(tf.keras.layers.Conv2D(filters=16, kernel_size=(3, 3), strides=(1, 1), \n", " activation='relu', padding='valid', input_shape=INPUT_SHAPE))\n", "model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2)))\n", "# define conv-pool layers - set 2\n", "model.add(tf.keras.layers.Conv2D(filters=32, kernel_size=(3, 3), strides=(1, 1), \n", " activation='relu', padding='valid'))\n", "model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2)))\n", "\n", "# add flatten layer\n", "model.add(tf.keras.layers.Flatten())\n", "\n", "# add dense layers with some dropout\n", "model.add(tf.keras.layers.Dense(256, activation='relu'))\n", "model.add(tf.keras.layers.Dropout(rate=0.3))\n", "model.add(tf.keras.layers.Dense(256, activation='relu'))\n", "model.add(tf.keras.layers.Dropout(rate=0.3))\n", "\n", "# add output layer\n", "model.add(tf.keras.layers.Dense(10, activation='softmax'))\n", "\n", "# compile model\n", "model.compile(optimizer='adam', \n", " loss='sparse_categorical_crossentropy', \n", " metrics=['accuracy'])\n", "\n", "# view model layers\n", "model.summary()" ], "execution_count": 27, "outputs": [ { "output_type": "stream", "text": [ "Model: \"sequential_1\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "conv2d (Conv2D) (None, 26, 26, 16) 160 \n", "_________________________________________________________________\n", "max_pooling2d (MaxPooling2D) (None, 13, 13, 16) 0 \n", "_________________________________________________________________\n", "conv2d_1 (Conv2D) (None, 11, 11, 32) 4640 \n", "_________________________________________________________________\n", "max_pooling2d_1 (MaxPooling2 (None, 5, 5, 32) 0 \n", "_________________________________________________________________\n", "flatten_1 (Flatten) (None, 800) 0 \n", "_________________________________________________________________\n", "dense_3 (Dense) (None, 256) 205056 \n", "_________________________________________________________________\n", "dropout (Dropout) (None, 256) 0 \n", "_________________________________________________________________\n", "dense_4 (Dense) (None, 256) 65792 \n", "_________________________________________________________________\n", "dropout_1 (Dropout) (None, 256) 0 \n", "_________________________________________________________________\n", "dense_5 (Dense) (None, 10) 2570 \n", "=================================================================\n", "Total params: 278,218\n", "Trainable params: 278,218\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "A-308F78YFoL" }, "source": [ "## Model Training\n", "\n", "Let’s train our model for 100 epochs and look at the performance. We do apply an early-stopping to stop the model training immediately once we don't see an improvement in validation-loss over the last 2 epochs using the `EarlyStopping` callback." ] }, { "cell_type": "code", "metadata": { "id": "5wwvN3kJYhgD", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "fece6751-73ab-4cd0-d67a-a45dc09808ae" }, "source": [ "EPOCHS = 100\n", "es_callback = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=2, \n", " restore_best_weights=True,\n", " verbose=1)\n", "\n", "history = model.fit(train_images, y_train,\n", " batch_size=32,\n", " callbacks=[es_callback], \n", " validation_split=0.1, epochs=EPOCHS,\n", " verbose=1)" ], "execution_count": 29, "outputs": [ { "output_type": "stream", "text": [ "Epoch 1/100\n", "1688/1688 [==============================] - 8s 3ms/step - loss: 0.7408 - accuracy: 0.7275 - val_loss: 0.3754 - val_accuracy: 0.8615\n", "Epoch 2/100\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.3688 - accuracy: 0.8666 - val_loss: 0.2996 - val_accuracy: 0.8898\n", "Epoch 3/100\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.3073 - accuracy: 0.8859 - val_loss: 0.2806 - val_accuracy: 0.8928\n", "Epoch 4/100\n", "1688/1688 [==============================] - 4s 3ms/step - loss: 0.2740 - accuracy: 0.8984 - val_loss: 0.2670 - val_accuracy: 0.9023\n", "Epoch 5/100\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.2548 - accuracy: 0.9072 - val_loss: 0.2596 - val_accuracy: 0.9080\n", "Epoch 6/100\n", "1688/1688 [==============================] - 4s 3ms/step - loss: 0.2331 - accuracy: 0.9131 - val_loss: 0.2744 - val_accuracy: 0.9027\n", "Epoch 7/100\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.2158 - accuracy: 0.9185 - val_loss: 0.2550 - val_accuracy: 0.9060\n", "Epoch 8/100\n", "1688/1688 [==============================] - 4s 3ms/step - loss: 0.2022 - accuracy: 0.9254 - val_loss: 0.2592 - val_accuracy: 0.9117\n", "Epoch 9/100\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.1912 - accuracy: 0.9282 - val_loss: 0.2625 - val_accuracy: 0.9088\n", "Restoring model weights from the end of the best epoch.\n", "Epoch 00009: early stopping\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "owt0MTEmpuLp" }, "source": [ "## Plot Learning Curves" ] }, { "cell_type": "code", "metadata": { "id": "GZ-aI4KOo8DY", "colab": { "base_uri": "https://localhost:8080/", "height": 267 }, "outputId": "b037bcb7-0253-4974-de50-9660973515bb" }, "source": [ "import pandas as pd\n", "\n", "fig, ax = plt.subplots(1, 2, figsize=(10, 4))\n", "\n", "history_df = pd.DataFrame(history.history)\n", "history_df[['loss', 'val_loss']].plot(kind='line', ax=ax[0])\n", "history_df[['accuracy', 'val_accuracy']].plot(kind='line', ax=ax[1]);" ], "execution_count": 30, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlkAAAD6CAYAAACS71IuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeVxU1fvA8c9hERRkHVwAFVQUFVxxFyotMy1Ny8yyb9pe2r7v/sq+7ct3sdLKTFvMNMvU8puagWviviEgbuDGAIKI7Of3xx2VCGXMWVie9+vFy5k7957zDI7jc8997jlKa40QQgghhLAtF2cHIIQQQghRF0mSJYQQQghhB5JkCSGEEELYgSRZQgghhBB2IEmWEEIIIYQdSJIlhBBCCGEHViVZSqkhSqk9SqlUpdQzVbw+XimVqZTaYvm5q8JrZRW2L7Rl8EIIIYQQNZWqbp4spZQrkAxcBaQDG4CxWutdFfYZD8RorSdVcXy+1trb2oBMJpMOCwuzdnchRB2wceNGs9Y6yNlx2IJ8hwlRv1zo+8vNiuN7Aala6zQApdQcYASw64JH/U1hYWEkJibao2khRA2llDrg7BhsRb7DhKhfLvT9Zc3lwhDgUIXn6ZZtld2glNqmlJqnlGpRYbunUipRKbVOKXW9dSELIYQQQtRutip8/wkI01p3Bn4FvqjwWiutdQxwC/CBUqpN5YOVUvdYErHEzMxMG4UkhBBCCOE81iRZGUDFkalQy7aztNZZWusiy9NPgR4VXsuw/JkGrAS6Ve5Aaz1dax2jtY4JCqoTZRlCCCGEqOesqcnaAEQopcIxkqubMUalzlJKNddaH7E8HQ7stmz3Bwq01kVKKRPQH3jLVsELUVOUlJSQnp5OYWGhs0Op0Tw9PQkNDcXd3d3ZoTiUfD5qlvr6ORSOV22SpbUuVUpNApYCrsAMrfVOpdQrQKLWeiHwkFJqOFAKZAPjLYd3AKYppcoxRs3eqHhXohB1RXp6Oo0bNyYsLAyllLPDqZG01mRlZZGenk54eLizw3Eo+XzUHPX5cygcz5qRLLTWS4Allba9VOHxs8CzVRy3Boi+xBiFqPEKCwvlP9BqKKUIDAykPtZdyuej5qjPn0PheDLjuxA2Iv+BVq8+/47q83uvaeTvQjiKVSNZNdXcxEO4KMWNPUKdHYoQTuft7U1+fr6zwxBCiFrpVFEpOw/nsT0jlyaNPbiuS/Alt1mrk6yFWw6TebJIkiwhhKgBSktLcXOr1f+tiHoiv6iUnRm5bM/IZYflzzTzKc4sgjM0uplNkqxafbkwNsLEnmMnOZYnd+wIcYbWmieffJKoqCiio6P59ttvAThy5AhxcXF07dqVqKgoEhISKCsrY/z48Wf3ff/9950cvbCX66+/nh49etCpUyemT58OwC+//EL37t3p0qULgwYNAiA/P58JEyYQHR1N586dmT9/PmCMlJ4xb948xo8fD8D48eO577776N27N0899RR//PEHffv2pVu3bvTr1489e/YAUFZWxhNPPEFUVBSdO3fmP//5DytWrOD668/NUf3rr78ycuRIR/w6RD1ysrCEdWlZfJqQxsNzNjPw3ZVET17KmOnrmLJ4N+vSsgk3efPIoHbMGB/DH88P4sNbe1TfsBVq9SlHbEQQr/+cREKKWUazhLD4/vvv2bJlC1u3bsVsNtOzZ0/i4uL4+uuvufrqq3n++ecpKyujoKCALVu2kJGRwY4dOwA4ceKEk6MX9jJjxgwCAgI4ffo0PXv2ZMSIEdx9993Ex8cTHh5OdnY2AK+++iq+vr5s374dgJycnGrbTk9PZ82aNbi6upKXl0dCQgJubm4sW7aM5557jvnz5zN9+nT279/Pli1bcHNzIzs7G39/fx544AEyMzMJCgri888/54477rDr70HUbScLS9iRkXd2dGpHRi77ss6NUDX39SQqxJfru4YQHeJLVIgvQY097BZPrU6yIps1xuTtQUJKpiRZosb4v592sutwnk3b7Bjsw8vXdbJq31WrVjF27FhcXV1p2rQpl112GRs2bKBnz57ccccdlJSUcP3119O1a1dat25NWloaDz74IMOGDWPw4ME2jVv8lbM+H//+979ZsGABAIcOHWL69OnExcWdncYgICAAgGXLljFnzpyzx/n7+1fb/+jRo3F1dQUgNzeX22+/nZSUFJRSlJSUnG33vvvuO3s58Ux/t912G19++SUTJkxg7dq1zJo162LeuqjH8gpL2Fkhodqekcs+86mzr59JqEZ2CyEq1JeoYPsmVFWp1UmWi4siNsJEfHIm5eUaFxe5Y0SI84mLiyM+Pp7Fixczfvx4HnvsMf7xj3+wdetWli5dyscff8zcuXOZMWOGs0MVNrZy5UqWLVvG2rVradSoEZdffjldu3YlKSnJ6jYq3pFXeVJVLy+vs49ffPFFrrjiChYsWMD+/fu5/PLLL9juhAkTuO666/D09GT06NFS0yWqlFdYwo6z9VNGYlUxoQq2JFSjLAlVdIgvJm/HJlRVqfWf5tgIEws2Z7DrSB5RIb7ODkcIq0ec7CU2NpZp06Zx++23k52dTXx8PG+//TYHDhwgNDSUu+++m6KiIjZt2sTQoUNp0KABN9xwA+3bt2fcuHFOjb0+cMbnIzc3F39/fxo1akRSUhLr1q2jsLCQ+Ph49u3bd/ZyYUBAAFdddRVTp07lgw8+AIzLhf7+/jRt2pTdu3fTvn17FixYQOPGjc/bV0hICAAzZ848u/2qq65i2rRpXHHFFWcvFwYEBBAcHExwcDBTpkxh2bJldv9diNqhoLiUVSlmlu8+zvp9WezPKjj7WohfQ6JCfLihewhRlkt+NSGhqkqtT7IGtDUBkJBiliRLCGDkyJGsXbuWLl26oJTirbfeolmzZnzxxRe8/fbbuLu74+3tzaxZs8jIyGDChAmUl5cD8Prrrzs5emEPQ4YM4eOPP6ZDhw60b9+ePn36EBQUxPTp0xk1ahTl5eU0adKEX3/9lRdeeIGJEycSFRWFq6srL7/8MqNGjeKNN97g2muvJSgoiJiYmPNOF/LUU09x++23M2XKFIYNG3Z2+1133UVycjKdO3fG3d2du+++m0mTJgFw6623kpmZSYcOHRzy+xA109HcQpYnHWP57uOsTjVTVFpOYw83+rYJ5MYeoUSFGCNUgTU0oaqK0meqwWqImJgYnZiYeFHHDPkgngCvBnx9dx87RSXEhe3evVv+g7BSVb8rpdRGrXWMPfpTSg0B/oWxLNinWus3Kr3eCpgBBGEsCzZOa52ulOoKfAT4AGXAa1rrb6vrr6rvMPl8XNikSZPo1q0bd955p8P6lL8T59Nas/NwHst2G4nV9oxcAFoENOTKDk25skNTeoYF0MCtZk+EcKHvr1o/kgUQ1y6Imav3U1BcSqMGdeItCSFsQCnlCkwFrgLSgQ1KqYWV1lB9B5iltf5CKTUQeB24DSgA/qG1TlFKBQMblVJLtdZyC6YN9ejRAy8vL959911nhyIcoLCkjLVpWSy3JFZHcgtRCrq39OepIe25skNTIpp415lZ+etERhIbYWJ6fBrr92VzRfsmzg5HCFFz9AJStdZpAEqpOcAIoGKS1RF4zPL4N+AHAK118pkdtNaHlVLHMUa7JMmyoY0bNzo7BGFn5vwiViQdZ/nuYySkmCkoLqNRA1diI0w8dlU7rohsUmNrqi5VnUiyeoYF4OHmQkKyWZIsIURFIcChCs/Tgd6V9tkKjMK4pDgSaKyUCtRaZ53ZQSnVC2gA7LVvuELUflprUo7n8+uuYyzffYzNh06gtTGlwqjuIQzq0JS+rQPxdHd1dqh2VyeSLE93V3qFB5CQIquqCyEu2hPAf5VS44F4IAOjBgsApVRzYDZwu9a6vKoGlFL3APcAtGzZ0t7xClHjFJeWs2F/tpFYJR3jUPZpAKJDfHlkUDsGdWhCp2CfOnMZ0Fp1IskCiIsI4rUluzmSe5rmvg2dHY4QombIAFpUeB5q2XaW1vowxkgWSilv4IYzdVdKKR9gMfC81nrd+TrRWk8HpoNR+G7LNyBETXWioJiVezL5dfcx4vdkcrKoFA83F/q3NXH/ZW0Z1KEJTX08nR2mU9WZJCu2nQmWGFM53BTTovoDhBD1wQYgQikVjpFc3QzcUnEHpZQJyLaMUj2LcachSqkGwAKMovh5Do1aiBpqn/kUy3YdY9nuYyQeyKGsXGPy9mBodHMGdWjCgAiT3IBWQZ35TbRv2pigxh6SZAkhztJalyqlJgFLMaZwmKG13qmUegVI1FovBC4HXldKaYzLhRMth98ExAGBlkuJAOO11lsc+R6EcKZTRaWs3ZvF78mZxKdkcsAyKWhks8bcf1kbBnVoQpdQP1lx5TzqTJKllLHEzm9Jx2WJHSGq4e3tfd7JJPfv38+11157dtHo2k5rvQRYUmnbSxUezwP+MlKltf4S+NLuAdZAF/p8iLqtvFyz60ge8SmZxCdnsvFADiVlmoburvRtE8gd/cMZGNmEFgGNnB1qrVBnkiww6rK+35TBzsN5RIfK7O9CCFGblZaWylqGDmDOLyIhJZP4ZDMJKZmY84sB6NDchzsGhHNZRBA9wvzxcKv7dwPaWp369Pa3LLETn5IpSZaoV5555hlatGjBxInGla7Jkyfj5ubGb7/9Rk5ODiUlJUyZMoURI0ZcVLuFhYXcf//9JCYm4ubmxnvvvccVV1zBzp07mTBhAsXFxZSXlzN//nyCg4O56aabSE9Pp6ysjBdffJExY8bY4+2Ki2TLz0d+fj4jRoyo8rhZs2bxzjvvoJSic+fOzJ49m2PHjnHfffeRlpYGwEcffURwcPCfRkvfeecd8vPzmTx58tnFq1etWsXYsWNp164dU6ZMobi4mMDAQL766iuaNm1Kfn4+Dz74IImJiSilePnll8nNzWXbtm1n11385JNP2LVrF++//749fq21VnFpOZsO5hBvuQS4IyMPgACvBgxoayKuXRBxESaa1POidVuwKsmyYlmK8cDbnLtr579a608tr90OvGDZPkVr/YUN4q5SUGMPOjb3ISElk4lXtLVXN0Jc2M/PwNHttm2zWTRc88Z5Xx4zZgyPPPLI2f9E586dy9KlS3nooYfw8fHBbDbTp08fhg8fflG3UE+dOhWlFNu3bycpKYnBgweTnJzMxx9/zMMPP8ytt95KcXExZWVlLFmyhODgYBYvXgwYCwWLKtTyz4enpycLFiz4y3G7du1iypQprFmzBpPJRHZ2NgAPPfQQl112GQsWLKCsrIz8/HxycnIu2EdxcTFnlibKyclh3bp1KKX49NNPeeutt3j33Xd59dVX8fX1Zfv27Wf3c3d357XXXju7Rufnn3/OtGnTrP411mUHswr4PSWT3/dksnavmVPFZbi6KHq09OeJwe2IaxdEVLCvlNrYWLVJlpXLUgB8q7WeVOnYAOBlIAbQGMtSLNRaX/hf2CWIbWdixqp9nCoqxcujTg3UCXFe3bp14/jx4xw+fJjMzEz8/f1p1qwZjz76KPHx8bi4uJCRkcGxY8do1qyZ1e2uWrWKBx98EIDIyEhatWpFcnIyffv25bXXXiM9PZ1Ro0YRERFBdHQ0jz/+OE8//TTXXnstsbGx9nq74iLZ8vOhtea55577y3ErVqxg9OjRmEzGFYWAgAAAVqxYwaxZswBwdXXF19e32iSr4ghoeno6Y8aM4ciRIxQXFxMeHg7AsmXLmDNnztn9/P39ARg4cCCLFi2iQ4cOlJSUEB0dfZG/rbrhTMH6mdqq/ZaC9VD/hozoFkJcRBD92gbi4+nu5EjrNmuyEGuWpTifq4FftdbZlmN/BYYA3/y9cKsXFxHEtN/TWL8vi4GRTe3VjRDnd4ERBXsaPXo08+bN4+jRo4wZM4avvvqKzMxMNm7ciLu7O2FhYRQWFtqkr1tuuYXevXuzePFihg4dyrRp0xg4cCCbNm1iyZIlvPDCCwwaNIiXXnqp+sbqm1r++bDF58rNzY3y8nPzulY+3svL6+zjBx98kMcee4zhw4ezcuVKJk+efMG277rrLv75z38SGRnJhAkTLiqu2qy8XLP7aJ5xF2AVBevj+4UR1y6IcJNXvZsQ1JmsSbKsWZYC4AalVByQDDyqtT50nmND/masVunRyh9Pdxfik82SZIl6ZcyYMdx9992YzWZ+//135s6dS5MmTXB3d+e3337jwIEDF91mbGwsX331FQMHDiQ5OZmDBw/Svn170tLSaN26NQ899BAHDx5k27ZtREZGEhAQwLhx4/Dz8+PTTz+1w7sUf5etPh+5ublVHjdw4EBGjhzJY489RmBgINnZ2QQEBDBo0CA++ugjHnnkkbOXC5s2bcrx48fJysrC29ubRYsWMWTIkPP2FxJi/LfxxRfnqk2uuuoqpk6derb+KicnB39/f3r37s2hQ4fYtGkT27Ztu5RfWY1nzi9iVYrZUltlxpxfBBjTK9zRP5y4dkHESMG6U9nqetpPwDda6yKl1L3AF8BAaw+25ZIUnu6u9A4PlCV2RL3TqVMnTp48SUhICM2bN+fWW2/luuuuIzo6mpiYGCIjIy+6zQceeID777+f6Oho3NzcmDlzJh4eHsydO5fZs2fj7u5Os2bNeO6559iwYQNPPvkkLi4uuLu789FHH9nhXYq/y1afj/Md16lTJ55//nkuu+wyXF1d6datGzNnzuRf//oX99xzD5999hmurq589NFH9O3bl5deeolevXoREhJywb4nT57M6NGj8ff3Z+DAgezbtw+AF154gYkTJxIVFYWrqysvv/wyo0aNAuCmm25iy5YtZy8h1iWnikpZuvMo32/KYPVeM1qDfyN3YiOCpGC9BlJaX3gFCKVUX2Cy1vpqy/NnAbTWr59nf1eM2ZN9lVJjgcu11vdaXpsGrNRan/dyYUxMjD5T8Ph3fZqQxpTFu1n9zEBC/GSJHWF/u3fvpkOHDs4Oo1ao6nellNqotY5xUkg2VdV3mHw+HOvaa6/l0UcfZdCgQefdpzb9nZSXa9alZTFvUzq/7DhKQXEZLQIaMrKrsdhyVIgvrnW1YN2cCvnHwBQBXkFQAy91Xuj7y5qRLGuWpWiutT5ieToc2G15vBT4p1LqzOnEYIxlK+wqrl0QLN7NqpRMxvSUxVqFEKI+OHHiBL169aJLly4XTLBqi9Tj+Xy/KZ0fNmdwOLeQxh5uDO8SzA09Qolp5V83a6vKyyFjI+xZDEmLwZx87jVPXwiMAFM7I+kytTN+AsLBtWYW8FebZFm5LMVDSqnhQCmQDYy3HJutlHoVI1EDeOVMEbw9RTTxpqmPB/EpZkmyhDiP7du3c9ttt/1pm4eHB+vXr3dSRKImqY2fDz8/P5KTk6vfsQbLOVXMT9sOM39TBlsPncBFGQMHzwztwOCOTfF0r4P1VaVFsC8BkhbBniXGyJWLG7TqDz3vhsDWkLXXSLjMyZD2G2z9+tzxLm7gH/bX5MsUAQ2de8nYqposK5aleJbzjFBprWdgWXDVUYwldoJYtvsYZeW67g6jCnEJoqOj2bJFluETVZPPh+MUl5azcs9x5m9KZ0XScUrKNJHNGvP80A6M6BpcN2usCnMh5VcjsUpZBsUnwd0LIq6EyGsh4qo/J0htr6x0fB5kpYD5zE+y8WfqMigrPrefV5Bl9KtS8uXXElzsn7DW2YmkYiNMzNuYzo6MXLq08HN2OKIe0FrXzeF7G6quBrQuk89HzVETPodaa7al5/L9pnQWbj1MTkEJJm8Pbu8bxqjuoXQM9nF2iLaXm2GMVCUthv2roLzESIKiRhmJVXgcuFuZUHr6QEgP46eislI4caBC4pUMWalGMleQdW4/Vw8IbPPX5CswAjy8bfaW62ySNcCyxE5CSqYkWcLuPD09ycrKIjAwUP4jPQ+tNVlZWXh61sGz8mrI56PmcPbn8EjuaRZszuD7TRmkHs+ngZsLV3Vsyo3dQ4mNMOHm6uKUuOxCa8hMMhKcpCVweJOxPaAN9H0A2g+D0Bjbjii5uhnJU2AbaF9pWpBTWZbRr+RzI19Ht8Pun0Cfm7cNnxDoNBKufu2Sw6mzSVagtwdRIT7Ep5iZNDDC2eGIOi40NJT09HQyM2XqkAvx9PQkNDTU2WE4nHw+ahabfg61huO7jJoiUwSEDQA3jz/tUlBcyi87/jztQkwrf14fFc3Q6Ob4NqyZRdt/S3kZHPrjXOF6trFmJSExMOhliBxmjBo542TDK9D4adnnz9tLiyB735+TL68gm3RZZ5MsgNiIID6JTyO/qBRvWWJH2JG7u/vZ5T6EqEw+H3VMeRkcXGckEXsWQ87+c6+5e0GbKyhvO5iNDWKYk1TKzzuOnJ124aGBEYzqHkKrQK/zNl/rlJyGtN8thes/Q4EZXNyh9WXQdxK0Hwo+zZ0d5fm5eUCTSOPH1k3bvMUaJDbCxEcr97JubxZXdpTZ34UQQvxNxQXGXW1JSyD5Z6O+x7UBtL4cBjwKra+A47vJ3bYIlfI/fJIW0RPw0uEMbRpHs54j6NA9Dhe3OvLfbkE2pPzPSKxSl0NJAXj4GAXrkcOg7VVG3VQ9V0f+tqvWo5U/Dd1dSUjJlCRLCCHExSnIhuRfjBGr1OVQeho8fKHd1ZZEYhB4NCavsIQftxxm/sbGbDl0DS5qCLeEneIW/yQiT66hY/psWPIFrAw0ko92g6HNIGhYi+qFSwqN+qqD64zE6sAa0GXQuDl0GWv8PsJiwa2BsyOtUep0kuXh5kqf1gEkpJidHYoQQojaIGe/MVqVtBgOrjEKon1CoPttxmWvsAFnJ77cdTiP2eu28+OWDAqKy84/7UJBNuxdAclLIWUpbJsDytWoDYoYbCRtQZE1Zzbzk8fg2HY4ugOO7TD+NCcbSRUYsQ54xEismncDlzpUrG9jdTrJAqMu67c9uziUXUCLgEbODkcIIURNojUc3WYkVUmLjaQCoEkniH3ckkh0PZsAFZaU8fO2dL5cd5CNB3LwcHNheJdgxvVpRedQ36rvHm0UANE3Gj/lZZC+wZJw/Q+WvWz8+LWEiKuNhCtsALg7YEm40mIjeTq2w7jL7tgOOLYTTlW4QcMnFJpFQeRQaBoFwd2MGdaFVep8khXXzpjKYVWqmbG9ZPZ3IYSo98pKjMtdZ+Zsyj0EygVa9IHBrxkJRUDrPx1yMKuAr/44wHeJ6WSfKibc5MULwzpwY49Q/BpdxCUyF8sIVss+cOXLkJtuJFvJ/4MtX8GGT8CtoVE0fmaUy9cGd0Keyvrr6FRmkjFXFRjzRjWJNBK9ZlFGQtW0k5Egir+tzidZbYK8ae7rSUJKpiRZQghRXxXlw97llsL1X6DwBLh5QpuBcPkz0G4IeJn+dEhZuWblnuPMXneA35MzcVGKKzs04bY+YfRrE4iLLVYT8Q2FmDuMn5JCY5LOlKXGSFfyL7AYY1St3WAjAQrtacwFdT5lpcbkm5VHp04eObePdzMjkWo7yEimmkUZk3BeqF3xt9T536ixxI6JX3YclSV2hBCiPsnPNO4ETFoMe3+DsiJjqZb2Q43LgG2ugAZ/nUrBnF/EtxsO8fX6g2ScOE2Txh48ODCCsb1a0NzXjpfx3D2NZWUiroRr3jIu5Z25rLjmP7DqffD0M5aYaXe1MRp24qBldGr7udGp0kKjPRd3CGoP4ZedG51qFv2XZFLYT51PssCoy5qbmM629BN0a+ncxSKFEELYUclp2DQLdnwPh9YD2qh36nmnkVy17FvliI3Wmg37c/hy3QF+3nGEkjJNvzaBPD+sA1d1bIq7o2diV8pIkILaQ/+HjLX+9q4wLium/A92zPvz/o1MRiLV865zo1Om9nK3n5PViySrf1sTSkFCilmSLCHqGaXUEOBfgCvwqdb6jUqvt8JYxD4IyAbGaa3TLa/dDrxg2XWK1voLhwUuLk5ZiZFcxb9tXBprGm1cBowcZiQd57lzL7+olAWbM/hy7QH2HDtJY083xvVpxa29W9G2ie3WsLtknr7GUi+dRkJ5ORzebCxT4x9uJFTeTWvO3YnirHqRZAV4NSA6xJeElEweGiRL7AhRXyilXIGpwFVAOrBBKbVQa72rwm7vALO01l8opQYCrwO3KaUCgJeBGEADGy3H5jj2XYgLKi+D7d/ByteN6Rda9IEbPjXu0LuApKN5fLnuAAs2ZXCquIxOwT68eUM013UJplGDGv5fo4sLhPYwfkSNVsM/SbYTG2Hi49/TOFlYQmPPOrROlBDiQnoBqVrrNACl1BxgBFAxyeoIPGZ5/Bvwg+Xx1cCvWutsy7G/AkOAbxwQt6iO1sakmCumGHVIzaLhlu+MGcfPM6JTVFrGLzuO8uW6A2zYn0MDNxeu6xzMuD4t6drCTxbvFjZXj5KsIKb+tpe1e7MY3KmZs8MRQjhGCHCowvN0oHelfbYCozAuKY4EGiulAs9zbIj9QhVW0dqoTVrxqnHJLDACRs+EDiPOOynmoewCvvnjIN9uOETWqWJaBTbi+aHG9Av+XlKzJOyn3iRZ3Vv606iBKwkpZkmyhBAVPQH8Vyk1HogHMoCyi2lAKXUPcA9Ay5YyVYzdHFwHy1+FA6vAtwWMmAqdb66ykL2sXBOfnMmX6w6wYs9xFDCoQ1Nu69OKAW1Ntpl+QYhq1Jskq4GbC31bB5KQkln9zkKIuiIDaFHheahl21la68MYI1kopbyBG7TWJ5RSGcDllY5dWVUnWuvpwHSAmJgYbaPYxRlHthmXBVOWglcTuOZt6HE7uHn8ZdfSsnJmrT3A52v2cSj7NCZvDyZd0ZaxvVoS7OeAWdSFqKDeJFlg1GUtTzrOwawCWgbKEjtC1AMbgAilVDhGcnUzcEvFHZRSJiBba10OPItxpyHAUuCfSqkztyQPtrwuHMWcAr+9BjsXGPNDXTkZet1T5dxWAHsz83l87la2HDpBr/AAnh4SyeCOzWjgJmvrCeeoX0lWuyAAElIzuTWwlZOjEULYm9a6VCk1CSNhcgVmaK13KqVeARK11gsxRqteV0ppjMuFEy3HZiulXsVI1ABeOVMEL+zsxEFY+SZs/dpYYibuSeg7CRr6Vbl7ebnmi7X7efOXJDzdXfnP2G5c1yXYsTELUQWrkqzq5pmpsOG1gCEAACAASURBVN8NwDygp9Y6USkVBuwG9lh2Wae1vu9Sg/67Wpu8CPFrSEKymVt7S5IlRH2gtV4CLKm07aUKj+dhfG9VdewMzo1sCXs7eQwS3oXEGcZagr3vhwGPgnfQeQ9Jzyngye+2sTYti4GRTXhjVDRNfDwdGLQQ51dtkmXlPDMopRoDDwPrKzWxV2vd1UbxXpIzS+ws3n6E0rJy3Bw9g68QQoi/KsiGNf+G9dOgtAi6jYPLnrrgwshaa+YmHuLVRbvRWvPmDdHcFNNCpmEQNYo1I1nWzDMD8CrwJvCkTSO0sdiIIOZsOMTW9Fx6tJLZ34UQwmmK8mH9R7D6P1CUB9E3wuXPQmCbCx52PK+QZ7/fzvKk4/RpHcDbN3ahRYDU2Yqax5okq9p5ZpRS3YEWWuvFSqnKSVa4UmozkAe8oLVOuJSAL1X/toGWJXYyJckSQghnKCk0LgkmvAsFZmNNwSueN5aHqcaibYd54YcdnC4u4+XrOnJ73zCZjkHUWJdc+K6UcgHeA8ZX8fIRoKXWOksp1QP4QSnVSWudV6kNh80x49eoAZ1D/UhIMfPIle3s2pcQQogKykpgy1fw+1uQlwHhl8HAF6FFz2oPzTlVzIs/7mDRtiN0aeHHezd1oU1QDVpbUIgqWJNkVTfPTGMgClhpuRbeDFiolBqutU4EigC01huVUnuBdkBixQ4cPcdMXISJD1fuJfd0Cb4NZYkdIYSwq/Jy2Pm9MR1DdhqExMD1H0Lry606fEXSMZ6ev50TBcU8eXV77o1rLTW1olawJsm64DwzWutcwHTmuVJqJfCE5e7CIIz5Z8qUUq2BCCDNhvH/LbERQfxnRSpr92YxJEpmfxdCCJs7fQIy98DxnfDHp8afTTrB2DnQbsh51xes6GRhCVMW7ebbxENENmvMFxN60THYxwHBC2Eb1SZZVs4zcz5xwCtKqRKgHLivJswz062lH14NXElIyZQkSwghLkVBtrFAc2aSkVRlJsHxJMg/em6fgNZww2fQadR51xesbM1eM09+t40juad54PI2PHxlBB5urnZ6E0LYh1U1WdXNM1Np++UVHs8H5l9CfHbh7upC3zYmElLMzg5FCCFqh1NmSwK1+1wylbkHTh0/t4+7FwS1hzYDjT+DIqFJJPi2tDq5Kiwp481fkvh89X7CTV58d18/uUlJ1Fr1asb3iuLamVi2+xgHsk7RKrDqJRqEEKJe0Rryj1cYlaqQUBVknduvQWMjiWo32EikgiKN5z6hVidTVdl8MIfHv9tKWuYpxvcL4+khkTRsIKNXovaqt0lWbIQxg3B8ipnbJMkSQtQnWsPJI3++xJe5xxilKjxxbj8PX2MkKnIYBHU4NzrlE2xVTZW1ikvL+dfyZD5auZfmvg35+q7e9Gtrqv5AIWq4eptkhQU2ItS/IQnJmdzWR5bYEULUAym/GtMnZO6Botxz2xv6G0lUp5HnLvEFRYJ3U5smU1XZfSSPx+ZuZfeRPEb3COXF6zri4yl3fYu6od4mWcYSO0Es2nqYkrJy3OV2YCFEXXbyGMy/00ioOo8+d4kvKBK8guyeTFVWWlbOtPg0PliWjG/DBnz6jxiu7NjUoTEIYW/1NskCY76sb/44yNZDJ4gJC3B2OEIIYT8/PwUlp+Gu5WCKcGooaZn5PP7dVjYfPMGw6Oa8en0UAV4NnBqTEPZQr5Osfm1MuCijLkuSLCFEnZW0GHb9AANfcGqCVV6umbV2P2/8koSHmyv/HtuN4V2CnRaPEPZWr6+R+TZyp0sLPxJSMp0dihBC2EdhLix+3JgItP8jTgsjPaeAcZ+tZ/JPu+jTOpD/PRonCZao8+r1SBYYdxn+d0UKuQUl+DaSYkshRB2zbDLkH4MxX4Gr47/jtNZ8tzGdV37ahdaaN0ZFM6ZnC5SDa8CEcIZ6PZIFRl1WuTZmFxZCiDrlwBpInAG974fQHg7v/lRRKfd/uYmn5m2jU7APvzwSx829WkqCJeqNej+S1aWFH4093IhPMXNNdHNnhyOEELZRUggLHwS/ljDweYd3n55TwF1fJJJ87CTPDY3krgGtcXGR5ErUL/U+yTKW2AkkPjkTrbWcYQkh6ob4tyErFcZ9Dw0cO+Fy4v5s7p29keKycmZO6EVcuyCH9i9ETVHvLxcCxLYLIuPEafZnFTg7FCGEuHRHd8DqD6DLWGg7yKFdz9uYzi2frMenoTs/TOwvCZao1+r9SBYYdVkACSmZhJtkiR0hRC1WXgYLJ4GnH1z9T4d1W1aueeuXJKbFp9G/bSBTb+mOXyOZ+0rUbzKSBbQK9KJlQCPik6X4XQhRy63/GA5vhmvehEaOmf8vv6iUe2YlMi0+jdv6tGLmhF6SYAmBjGSdFRth4ofNGbLEjhCi9srZDyumQMTVEHWDQ7o8lG0UuKdm5vPqiE7c1jfMIf0KURtINmERGxHEqeIyNh3IcXYoQggbUkoNUUrtUUqlKqWeqeL1lkqp35RSm5VS25RSQy3b3ZVSXyiltiuldiulnnV89BdBa/jpEVAucO17DlmLcH1aFiOmruZoXiGz7uglCZYQlUiSZdG3TSCuLoqEFLlkKERdoZRyBaYC1wAdgbFKqY6VdnsBmKu17gbcDHxo2T4a8NBaRwM9gHuVUmGOiPtv2ToH0n6DKyeDb6jdu/t2w0HGfbYev0ZGgXv/tia79ylEbSNJloVvQ3e6yhI7QtQ1vYBUrXWa1roYmAOMqLSPBnwsj32BwxW2eyml3ICGQDGQZ/+Q/4b8TFj6LLToDTF32rWrsnLNq4t28fT87fRpHciCB/rLDUNCnIckWRXERpjYlpFLzqliZ4cihLCNEOBQhefplm0VTQbGKaXSgSXAg5bt84BTwBHgIPCO1jrbrtH+Xb88DcWn4Lp/g4v9vtbzCku4Y+YGPlu1j/H9wvh8fE98G8pyZEKcjyRZFcRGBKE1rJYldoSoT8YCM7XWocBQYLZSygVjFKwMCAbCgceVUq2rakApdY9SKlEplZiZ6eDR8D2/wI75EPsENIm0WzcHsk4x6sM1rE4188+R0Uwe3gk3uUlIiAuy6l9IdYWjFfa7QSmllVIxFbY9azluj1LqalsEbS9dQn1p7OlGgkzlIERdkQG0qPA81LKtojuBuQBa67WAJ2ACbgF+0VqXaK2PA6uBGKqgtZ6utY7RWscEBTlw8s3CPFj8GAR1gAGP2q2btXuNAndzfhGz7+zNLb1b2q0vIeqSapMsKwtHUUo1Bh4G1lfY1hGjkLQTMAT40NJejeTm6kL/NiYSUowldoQQtd4GIEIpFa6UaoDxfbSw0j4HgUEASqkOGElWpmX7QMt2L6APkOSguK2z/BXIOwzD/wNu9pmX6uv1B7nts/WYvD34cWJ/+rYJtEs/QtRF1oxkWVM4CvAq8CZQWGHbCGCO1rpIa70PSLW0V2PFtjNxOLeQvZmnnB2KEOISaa1LgUnAUmA3xl2EO5VSryilhlt2exy4Wym1FfgGGK+Ns6ypgLdSaidGsva51nqb49/FeRxcBxs+hd73QoueNm++tKycyQt38tyC7QyIMPH9A/1oFSgF7kJcDGsmI62qcLR3xR2UUt2BFlrrxUqpJysdu67SsZWLTlFK3QPcA9CypXOHoeMijKH+hJRM2jbxdmosQohLp7VeglHQXnHbSxUe7wL6V3FcPsY0DjVPaREsfNCYqmHgizZvPreghEnfbCIhxcxdA8J5dmgHXF3sP++WEHXNJVctWgpE38M4G/xbnFbPUIUWAY0IC2wk82UJIWqu+HfAnAzXvg8etj0ZTMvMZ+SHq1mXlsWbN0TzwrUdJcES4m+yZiSrusLRxkAUsFIZMww3AxZahuKtKTqtcWIjgpi3MZ2i0jI83GpsCZkQoj46thNWvQfRN0HEVTZtelWKmQe+2oibqwtf3dWHXuGOWftQiLrKmpGsCxaOaq1ztdYmrXWY1joM4/LgcK11omW/m5VSHkqpcCAC+MPm78LGYiNMnC4pY9OBE84ORQghzikvg4UPgacvDHnDpk3PXruf2z//g+a+DflxYn9JsISwgWpHsrTWpUqpM4WjrsCMM4WjQKLWuvKdOhWP3amUmgvsAkqBiVrrMhvFbjfnltjJlDtphBA1xx/TISMRRn0CXrb5biopK+f/ftrJl+sOcmWHJnxwcze8Pay5yCGEqI5V/5KqKxyttP3ySs9fA177m/E5RWNPd7q39CMhxcxTQ5wdjRBCADkHYPmr0PYqiLZNPf6JgmImfr2J1alZ3HtZa566OlLqr4SwIZmu9zxiI4LYcTiXrPwiZ4cihKjvtIZFlslGr30P1KUnQqnH87l+6mo27MvhndFdePYauYNQCFuTJOs8YiNMliV2spwdihCivts2F/Yuh0Evgd+lT3MTn5zJyA9Xk19Uyjf39ObGHqE2CFIIUZkkWefROdQPH083EpIdvA6ZEEJUdMoMvzwDoT2h192X1JTWms9X72P8538Q4teQHyb2p0crKXAXwl6kuvE8XF0UAyJMJKSY0VqjbDA8L4QQF+2XZ6DopLF0jsulTSnz+er9vLJoF4M7NuX9MV3xkgJ3IexKRrIuIDYiiKN5haQez3d2KEKI+ij5f7D9O4h9DJp0uOTmftySQZcWfnw8rockWEI4gCRZFzCgrQmAeJn9XQjhaEUnjWJ3U3uI/dsLapyVW1DCtoxcrmgfhIsUuAvhEJJkXUCLgEa0NnmRkCJ1WUIIB1v+KuRlGJcJ3Twuubm1aWa0PnfyKISwv9qdZOUfB3OKXbuIjTCxLi2LotIaP4eqEKKuOPSHMfFoz7ugZW+bNLk6NQuvBq50aeFnk/aEENWrvUmW1jBrBHw3AUqL7dZNbEQQhSXlbNyfY7c+hBDirNJiWPgg+ATDlS/brNnVqWZ6tw7E3bX2fu0LUdvU3n9tSsHAF+HYdlj1vt266dMmEDcXJXVZQgjHWPUeZCbBsPfAo7FNmjx84jRp5lP0k2XChHCo2ptkAUQONZaXiH/bWJneDrw93Ojeyl/qsoQQ9nc8CeLfgagboL3t1vRanWqcJA6IkHosIRypdidZANe8BQ394IcHoKzULl3ERZjYeTgPsyyxI4Swl/Iy4zKhhzcMedOmTa9ONWPybkD7prYZGRNCWKf2J1mNAmDYu3BkC6z5l126iI0IAs6dDQohhM1t+AzS/4CrXwfvIJs1q7Vm9d4s+rUxyaTKQjhY7U+yADqOgI7Xw8o3jOF2G4sK8cWvkTvxyZJkCSHs4MQhWP5/0GYgdLnZpk2nHM8n82SRTN0ghBPUjSQLYOg7RpHojxONYXcbcnVR9G9rIiElE621TdsWQtRzWsPix0CXw7XvGzf12NAqy007/aUeSwiHqztJlneQUZ+VkQhrp9q8+bgIE8dPFpF8TJbYEULY0I75kPI/GPgC+IfZvPk1e82EBTYixK+hzdsWQlxY3UmywLgjJ/JaWDHF5pOUDrDUZcldhkIImzmVBT8/BcHdofd9Nm++pKycdWnZ9JdLhUI4Rd1KspQy5pZxb2jzy4Yhfg1pE+Ql82UJUcsopYYopfYopVKVUs9U8XpLpdRvSqnNSqltSqmhFV7rrJRaq5TaqZTarpTytGlwS5+Dwlxj6RwXV5s2DbAt/QT5RaVSjyWEk9StJAugcVO45k04tB7WT7Np07ERQaxPy6KwRJbYEaI2UEq5AlOBa4COwFilVMdKu70AzNVadwNuBj60HOsGfAncp7XuBFwOlNgsuNRlsG0O9H8EmkXZrNmKVqdmoRT0lUlIhXAKq5IsK84E77Oc5W1RSq068yWmlApTSp22bN+ilPrY1m+gSp3HQMTVsPwVyNprs2bj2pkoKi0nUZbYEaK26AWkaq3TtNbFwBxgRKV9NOBjeewLHLY8Hgxs01pvBdBaZ2mtbXeG5eIGra+AuCdt1mRlq1LNRAX74teogd36EEKcX7VJlpVngl9rraO11l2Bt4D3Kry2V2vd1fJj+6KDqoOG6z4A1wbG5H7l5TZptnd4IA1cXfhwZSoFxfaZ+FQIYVMhwKEKz9Mt2yqaDIxTSqUDS4AHLdvbAVoptVQptUkp9ZRNI2t9OfzjB3C37RXIMwqKS9l8MId+bWUUSwhnsWYkq9ozQa11XoWnXhhnhs7lEwxD/gkHVkPiZzZp0svDjSkjo1iXlsUtn6wn55T9FqYWQjjMWGCm1joUGArMVkq5AG7AAOBWy58jlVKDqmpAKXWPUipRKZWYmVkzbo75Y182JWVa6rGEcCJrkixrzgRRSk1USu3FGMl6qMJL4ZaC0t+VUrGXFO3F6nortBkEv74MOftt0uRNMS34aFwPdh3J48aP15Bx4rRN2hVC2EUG0KLC81DLtoruBOYCaK3XAp6ACeO7Ll5rbdZaF2CMcnWvqhOt9XStdYzWOiYoyHaztV+K1almGri50DMswNmhCFFv2azwXWs9VWvdBngao5AU4AjQ0lJQ+hjwtVLKp/KxdjsLVAqG/xuUCyx8yJj0zwau7tSM2Xf04vjJIm78aA0px07apF0hhM1tACKUUuFKqQYYhe0LK+1zEBgEoJTqgJFkZQJLgWilVCNLEfxlwC6HRX6JVqdm0aOlP57utr9rUQhhHWuSLGvOBCuaA1wPoLUu0lpnWR5vBPZi1Dn8iV3PAn1DYfCrsO932DjTZs32bh3I3Hv7UlquufHjtWw8IMXwQtQ0WutSYBJGwrQb4y7CnUqpV5RSwy27PQ7crZTaCnwDjNeGHIz60g3AFmCT1nqx49/FxcvKL2LXkTwGyCzvQjiVNUlWtWeCSqmICk+HASmW7UGWwnmUUq2BCCDNFoFflB7jIfwy+N+LxhphNtKhuQ/f398P/0bu3PrpOlYkHbNZ20II29BaL9Fat9Nat9Fav2bZ9pLWeqHl8S6tdX+tdRfLDTr/q3Dsl1rrTlrrKK21bQvf7WjN3iwAmYRUCCerNsmy8kxwkmWyvi0YlwVvt2yPA7ZZts/DmG8m2+bvojpKGZP96XL46WGbXTYEaBHQiHn396NtE2/unrWR+RvTbda2EEL8HWv2mmns6UZ0iK+zQxGiXnOzZiet9RKMos+K216q8Pjh8xw3H5h/KQHajH8ruOr/YMkTsPlL6H6bzZo2eXvwzd19uO/LjTz+3VayThVxT1wbm7UvhBAXY1Wqmb6tA3F1se1i00KIi1P3Zny/kJg7odUAWPo85B2ufv+L0NjTnRnjezKsc3P+uSSJ15fsRttwxEwIIaxxMKuAQ9mnpR5LiBqgfiVZLi7G3YZlxfDTIza9bAjg4ebKv2/uxj/6tmJafBpPfLeNkjLbTIQqhBDWWL3XWF+1XxtJsoRwtvqVZAEEtoFBL0HKUtj2rc2bd3VR/N/wTjx6ZTvmb0rn3tkbOV0sax0KIRxjVaqZZj6etAnycnYoQtR79S/JAuh9L7ToDT8/DSeP2rx5pRQPXxnBlOuj+G3PccZ9tp4TBTI7vBDCvsrLNWtSzfRrG4hSUo8lhLPVzyTLxRVGTIXSQlj8uM0vG54xrk8rPrylO9vTcxn98VqO5Mrs8EII+9l9NI+cghJZSkeIGqJ+JlkApgi44nlIWgQ77HcD5DXRzZl5R0+O5BZyw4drSD2eb7e+hBD12+pUox5L5scSomaov0kWQN+JEBIDS56EfPst6tqvjYk59/ShuEwz+uM1bD4os8MLIWxvVWoWbZt409TH09mhCCGo70nWmcuGxfnG/Fl2FBXiy/z7+9LY051bPlnP78n2S+qEEPVPUWkZG/Zly6VCIWqQ+p1kATSJhMufgV0/wK4f7dpVq0Av5t3fl3CTF3fO3MCPWy60BKQQQlhv88ETnC4pk0uFQtQgkmQB9HsYmnc1iuBPZdm1qyaNPZlzbx9iwvx5eM4WZqzaZ9f+hBD1w5pUMy4KercOcHYoQggLSbIAXN3g+g/h9An42f5rwPp4ujNzQi+GdGrGK4t28eYvSTI7vBDikqxKNdOlhR8+nu7ODkUIYSFJ1hlNO0Hck7BjHiQttnt3nu6uTL21O2N7teSjlXt5ev42SmV2eCHE33CysISt6blSjyVEDSNJVkWxj0HTaFj0KBRk2707VxfFP0dG8dDAtsxNTOe+LzdRWCKzwwshLs76tGzKyrUspSNEDSNJVkWu7nD9VCjIgqXPOaRLpRSPDW7P/w3vxPKkY/zjsz/IPV3ikL6FEHXDqlQznu4udG/l5+xQhBAVSJJVWfMuMOBR2PoNJC91WLe39wvjP2O7sflQDmOmreVYXqHD+hZC1G6rU830Cg/Ew83V2aEIISqQJKsqcU9CUAf46REozHVYt9d2Dubz8b04lF3ADR+tIS1TZocXQlzY8bxCUo7n079NoLNDEUJUIklWVdw8jLsN84/B0ucd2vWACBPf3NOH08Vl3PjxWraln3Bo/0KI2mX1XllKR4iaSpKs8wnpDv0fgs2zIXW5Q7vuHOrHd/f1paG7K2Onr2NVitmh/Qshao9VKVn4N3KnY3MfZ4cihKhEkqwLuewZMLWHnx6GopMO7bp1kDffP9CPFgGNmDDzD2av3S9zaQkh/kRrzZq9Zvq1MeHiopwdjhCiEkmyLsTd01jbMC8Dfn3J4d039fHk23v70reNiRd/3MldXyRizi9yeBxCiJopzXyKI7mFcqlQiBrKqiRLKTVEKbVHKZWqlHqmitfvU0ptV0ptUUqtUkp1rPDas5bj9iilrrZl8A7Roif0eQASZ0Da7w7v3rehOzPH9+Tl6zqSkGpmyAfx/LbnuMPjEELUPKtTz9RjSdG7EDVRtUmWUsoVmApcA3QExlZMoiy+1lpHa627Am8B71mO7QjcDHQChgAfWtqrXQa+AAFtYOEkKHL8HX8uLooJ/cNZOKk/gV4eTPh8A5MX7pSJS4WwghUniS2VUr8ppTYrpbYppYZW8Xq+UuoJx0VtndWpZkL9G9IyoJGzQxFCVMGakaxeQKrWOk1rXQzMAUZU3EFrnVfhqRdwpnhoBDBHa12ktd4HpFraq13cGxqXDU8cguX/57QwIpv58OOk/kzoH8bMNfsZ/t9V7D6SV/2BQtRTVp4kvgDM1Vp3wzgp/LDS6+8BP9s71otVVq5ZuzeLAW1NKCX1WELURNYkWSHAoQrP0y3b/kQpNVEptRdjJOuhizm2VmjVF3rfC39Mh02zodQ5tVGe7q68fF0nZk7oSfapEkb8dzWfrdpHebkUxQtRhWpPEjFOCs/cmucLHD7zglLqemAfsNMBsV6UHRm55BWW0k/qsYSosWxW+K61nqq1bgM8jXFmaDWl1D1KqUSlVGJmZqatQrK9QS8ZaxsunATvtIPFT8DhzeCEu/4ub9+EpY/EEtfOxKuLdnH753/ILPFC/JU1J3qTgXFKqXRgCfAggFLKG+P7zHnD1xewylKP1U8mIRWixrImycoAWlR4HmrZdj5zgOsv5lit9XStdYzWOiYoKMiKkJykgRfc+zuM+x7aDoJNs2D65fBRf1g7FU45dj6rQG8PPvlHDK+NjGLD/myGfBDP0p1HHRqDEHXAWGCm1joUGArMVkq5YCRf72utqy3EdMaJ4upUMx2a+2Dy9nBIf0KIi2dNkrUBiFBKhSulGmDULCysuINSKqLC02FAiuXxQuBmpZSHUiociAD+uPSwncjF1UiwbpwBT+yBYe8aM8QvfQ7ebQ9zboWkJVDmmEWelVLc2rsVix6MJdivIffO3siz32+noLjUIf0LUcNZc6J3JzAXQGu9FvAETEBv4C2l1H7gEeA5pdSkqjpx9IliYUkZiQdyZCkdIWo4t+p20FqXWr5YlgKuwAyt9U6l1CtAotZ6ITBJKXUlUALkALdbjt2plJoL7AJKgYla67pzS1xDf+h5l/FzfDds/hK2fQtJi8CrCXS+CbqNgyYd7B5K2ybeLHigP+/+uofp8WmsT8viXzd3IzrU1+59C1GDnT1JxEiubgZuqbTPQWAQMFMp1QEjycrUWsee2UEpNRnI11r/1yFRVyNxfw7FpeX0j5B6LCFqMlXTZhGPiYnRiYmJzg7j7ysrgZRfYctXkPwLlJdCcHcj2Yq6ARr62T2ENXvNPPbtVsz5RTw+uD33xLXGVWaDFjWYUmqj1jrGTm0PBT7g3EniaxVPEi13G34CeGMUwT+ltf5fpTYmYyRZ71TXnyO+w974OYlPE9LY+vJgvDyqPVcWQtjRhb6/JMmyp/xM2D4XNn8Fx3eCmydEXgtdb4HWlxuXHu3kREExzy3YzpLtR+kdHsD7Y7oS7NfQbv0JcSnsmWQ5miO+w4b/dxWebq7Mva+vXfsRQlTvQt9fsqyOPXkHQd+JcP9quGcldLsNUpfBl6Pgg86wYgpkp9mla79GDZh6S3fevrEz2zNyGfJBPIu2Ha7+QCFEjXaioJjtGbmylI4QtYAkWY6gFAR3g2HvwON74MbPoUkkJLwL/+4Gnw81RrtsPJu8UorRMS1Y8lAsrYO8mfT1Zh6fu5WThY4pyhdC2N7avVloLUvpCFEbSJLlaO6eEDUKxs2HR3cac2+dPAo/PmDMvfXDRDiwxqZzb4WZvPjuvr48NLAtCzanM+zfq9h4IMdm7QshHGf1XjNeDVzp0sL+9Z1CiEsjSZYz+QRD7OPw4Ea4Y6mRfO36AT6/Bv7THeLfhtx0m3Tl7urCY4Pb8+29fSnXmpumreVfy1IoLSu3SftCCMdYnZpFn9aBuLvK17cQNZ38K60JlIKWfWDEf+GJZLj+Y/AJMWq23o+C2SPhj08ga+8lj3D1DAtgycOxDO8SzPvLkhkzfR0Hswps9EaEEPaUceI0+8ynZCkdIWoJufe3pmngBV3HGj/Z/9/encdXWd55H//8crIRQraTjWxkY0mCQiDsKLKoaFtpOy5o1elTrdOO2qm209G24zg+nXmc2unUZ8Zq1cfRsa0LjlqtOiiKVQICYXEhKGSRkICQhUXAQJbf88d1gEAFdvs/hQAAGVlJREFUA5yTk3PO7/16nRdnue+b3w3JlW+u67qvuxHefcKtvfXyD93nKSOgZA6UzIaic91aXacoKT6Gf7tiPOeNzuCnz33Axf/3be5aUMHXKnPtRrPGDGLVvlvpzLSQZUxIsJA1mKUVwewfw3m3u6sQ69+A+qXw/jOw5j9BoiB3ogtdxbMhrwo8Mf0+/ILxuUwckcqtT73LrU+/y9KPWvnZV8eSPKT/xzDGDJzqujbSE+MYlZUY7FKMMf1gISsUiIC3xD0mf9steNpcAw1LXfB66x74079A7DDXu1Uy2wWvtGK370nkpSbwxA1Tuf/NOv5tyWbWfNzBL68Yz9TiCLhy6fDQq/XemRCgqlTXtTOj1Gs9zsaECAtZocgTAyOmucfsH8Nnu6DxbV9P1xvw0Utuu5QC18NVMseFr4S0zz9clHDTnJHMHJnB959cx5UPvcNfnVvCd2YVk5IQO4AnNgAO7YeGN2HTYtj8qgus4xa6NcwyxwS7OmNOaNOOfbTtO2jrYxkTQixkhYMhqVB+iXvAsUOLG56DtY+5ocWcSt98rjmQN+nPhhbH56fw0vfO4a4Xa3ngT/U8Ut3IRWOzWTipgKnFaaH72/OuLb5QtdiF0Z6DrtevdA5oL6x8AFb8h/s3OXz7o7hhwa7amGMs883HspBlTOiw2+qEu55uaFlztJerpcYFi9hEKDznaOjylhwzbLZh2x6eWr2V59a18GlnN4XeBC6flM+lE/LITIoP4gn1Q083NK9y947c9Cq0bnTve0th5IUw6kIomAbRvl66fa3u4oJ1j0PrhxCTABVfc71bBVNtOHEA2G11vth1j66moW0/S394nt+PbYw5fXbvQnPUZ7vh47eP9nTtanTvJ+cfnctVNOvI0GJnVw+vfLCdJ1dtZWVjB54oYc6YTBZOymfWqAyiB8taPQc6oO51F6zqlkDnboiKhhEzXKgaeSGkl578GKpurtu6/4IPnoVD+8A70vVujbsShmUNzLlEIAtZJ9fV08v4f3yVr03I5WdfPcuvxzbGnBkLWebEOhpc2GpYCg1vwcE9gMDwcZBZ5ptwXwreUhp7s3jq3Q6eWdNM276DZCXFcdnEfC6vyqfAmzCwdavCzo1uCHDTYti60vXQJaT7QtUFLjTGJ5/e8Q/ug9o/uN6tphUgHnfcymvcsT020u5PFrJOrubjDi59YAX3f2MCF5013K/HNsacmZO1X/aTItKlFbvHpOvcMNu2tS50bVkGDX9y63T5FAG3DcvhR3kltETlUL0rlSVvDeWbbw4nv7iMSycXc0FFFnHRnsDU2tUJHy/zDQMuhj1N7v3ss+GcH7oQlDMBovzQuxaXCJXfcI+2zS5srX8CPnoZErNcz1blNV/cO2aMH1TXtSMC00oi4KpfY8KI9WSZkzu03/V2tdf5HvW+Rx181nFks26i2NqbQXNUDjGZIykaPY6swgrXC5aUe/rBZ+82dxXgpsXuqsCuA27OVPF5R3usknL8caZfrKfL1bL2cfen9kDBdDecWPFVt5BsJFN1Xy+du6FzDyTn9bsn0XqyTu7yB1bQ2d3DCzfN9OtxjTFnznqyzOmLHQrZZ7nH8Q50uMDVUY+nbTOJW2op3LEJ744/kLDzaXjbbabR8Uha8TFDj6T5ng9NP3ZieW+v603btNj1WH3ynns/uQDGfwNGzYfCme5G2wPNEwNjvuQen37ievnWPu5u7v3K37l7T0641i0QG6qT5Xu6oHOvLyjtdnP4OvccDU59Xx//Wece6O0+eqyFT8CYi4N3LmFi/8Fu1jbt4vpzioNdijHmFFnIMqcvIc098ichQIbv7fZPO3l+5XrWrK0hdk8Do3QHUz7bRfG2WuI++h+kt+voMeKSjy60Kh6ofx32t7olJ/Knwrw7XbDKGDO4gsuwbJh5C8z4vpuzte638P4it1xGRhlMuAbOvsKFyIGm6ibt729zjwNtvkDUj6B0aN/Jjx0VA0NSID7F9VIlpLk7Exx+3feznMqBOd8wt+rjDrp71W6lY0wIspBl/M47LJ6r5k3lyrlTWNu0iydXbeXn723ns64eyrMS+NZYD/OH7yNxX9PRYcimlW4osPg8F6pK555w8dRBRQRGTHeP+XfDhmdd79biH8Nr/wCjL3K9WyVzIOo056r1DU0H2l0IPRyeDgep/a2+177Pew6e+HhxSS4EHQ5DaUXHvu4blIakHPtZzJDBFXYjQPXmNmKjo6gqPPX7lBpjgsvmZJkB8WlnFy++u52nVjfxbvMeYqOjmF+RzcJJ+Uwt9hIVFWY/uHdudGHrvSddMErKhfFXuSHP1EI3d2l/qy809Q1JfXqf9re60HSgDbo7P//viUlwV1QO9T3+7HkGDPW6BWvjU1zAGoRXRtqcrBO76N63SU2I4fffnuq3Yxpj/OeMl3AQkfnAvYAHeFhV7z7u81uB64FuoBX4lqpu8X3WA7zv27RJVS852d9lISv81W7by9M1W3l2bTN7O7sZ4U3g8qp8Lp2YR9ZgX+j0VHUfgk2vuMBV/7pbZiI6/sShKXrI0WA0NMMXlPo+9312OEyFyWR7C1mfr23fQap+toS/vXA0N862K1mNGYzOaOK7iHiA+4DzgWZgtYi8oKq1fTZbB1Sp6gER+S7wc+AK32efqer4MzoDE1bKc5K485IKbrtoDIs3fMITq5q4Z/FH/PK1TcwalcFlE/OYW5ZFbPQgWej0TETHQvkC99jT4laW/6yjT2g6rgcqTEKT8Y/l9e0ANh/LmBDVn3GDyUCdqjYAiMiTwALgSMhS1aV9tn8HuNqfRZrwFB/jYcH4XBaMz6WxbT+Larby7NoWvvvhWlITYlgwPpfLqvKoyDnNBUUHm+RcOOfWYFdhQsjyujaS4qMZmxsm3wPGRJj+hKxcYGuf183AlJNsfx3wSp/X8SJSgxtKvFtVnz/lKk3YK0ofyo/mj+EHF4zm7c2tLFrTzO9XNvHo8o8pH57EZVV5fHV8LqlDY4NdqjEDQlV5e3Mb00q8eMJtzqIxEcKv4zEicjVQBdzT5+0RvrHKq4BfiUjJ5+x3g4jUiEhNa2urP0syIcYTJZw3OpP7rprAqp/M5a4FFXiihH98sZYp//w6f/27NSz9cCfdPb3BLtWECBGZLyIfiUidiNz2OZ8XiMhSEVknIu+JyMW+988XkTUi8r7vzzkDWXdTxwFadn/GDBsqNCZk9acnqwXI7/M6z/feMURkHvATYJaqHrl+XFVbfH82iMibQCVQ33dfVX0QeBDcpNFTOwUTrlISYrl2WiHXTitk4/a9LKpp5vn1Lbz8/idkDovj6xPyuKwqj5KMxGCXagapfs4p/SnwtKreLyLlwMtAIdAGfEVVt4nIWGAxrmd/QFTXuflYFrKMCV396claDYwUkSIRiQUWAi/03UBEKoHfAJeo6s4+76eKSJzveTowgz5zuYzpr7LhSdzxlXLeuX0uD1w9kbPzknno7Qbm/uuf+Pqvq3lyVROfdnZ98YFMpDkyp1RVDwGH55T2pUCS73kysA1AVdep6jbf+xuAIYfbs4FQXdfG8OR4itPtYghjQtUX9mSpareI3IT7Lc4DPKKqG0TkLqBGVV/ADQ8mAovELVR4eKmGMuA3ItKLC3R3H/cbpDGnJDY6ivljs5k/Npudn3by/LoWFtU0c9uz73Pnixu4eOxwLq3KY2pRGK69ZU5Hf+aU3gm8KiI3A0OBeZ9znL8A1vbtpQ+k3l5leX0bc8ZkIbb4qzEhq1+rEqrqy7gu9L7v3dHn+ec1SqjqcuBzbnpnzJnLHBbPDeeW8O1zilm/dTeL1jTz4vptPLuuhfy0IfzFhDwunZhHXmpCsEs1g9uVwKOq+q8iMg14XETGqmovgIhUAP8CXHCiA4jIDcANAAUFBWdcUO32vew60MXMkd4zPpYxJngG39LPxpwiEaGyIJXKglTu+HI5izd8wqKaZu59fTP3vr6Z6SVeLpuYz4UV2QyJPc1b25hQ1Z85pdcB8wFUdYWIxAPpwE4RyQOeA65V1XpOwN/zSqvr2gCYUWLzsYwJZRayTFjpu/ZW864DPLu2hUVrtvL9p9YzLC6aL4/L4bKqPCrzU2wYJjIcmVOKC1cLcVc699UEzAUeFZEyIB5oFZEU4CXgNlWtHsCaqa5vZ2RmIpnhdgcEYyKMhSwTtvJSE/je3JHcNLuUlY0dLFqzlefXtfDEqiZKMxO5dGIeX6vMDb9b+Zgj+jmn9AfAQyJyC24S/DdVVX37lQJ3iMjh6REX9L24JxAOdvewqrGdhZPOfNjRGBNcdoNoE1H2Hezmpfe2saimmZotuwAYl5/C+WWZnF+ezaisROvhCgK7d+FRK+rbufKhd3j42irmlWf5sTJjTCCc0b0LjQkniXHRXDGpgCsmFdDQuo9XPviEV2t38ItXN/GLVzeRnzaE88uymVeeyeTCNKI9YXD/RBNSlte34YkSphSnBbsUY8wZspBlIlZxRiI3zi7lxtml7Nzbyesf7uS12h38duUWHqluJHlIDLNHZzCvPItZozIYFh8T7JJNBFhW18a4vGT7ejMmDFjIMgbITIrnyskFXDm5gAOHunlrUxtLNu7gjQ938vz6bcR4hKnFXi4oz2JuWRY5KUOCXbIJQ3s7u3h3625unF0a7FKMMX5gIcuY4yTERh9Z8LSnV1nbtIsltTt4rXYHf/+HDfz9HzZQkZPE+eVZzCvLoiInyeZxGb9Y2dBBr9qtdIwJFxayjDkJT5QwqTCNSYVp3H5xGfWt+3itdgdLandw7+ub+dWSzeQkxzOvPIvzy7OYUuQlNtrmcZnTU13XxpAYD5UFKcEuxRjjBxayjDkFJRmJlMxK5DuzSmjbd5A3PtzJktodPF2zlf9asYVhcdGcOzqDC8qzOG9UJskJNq/G9N+yujYmFaURF22L5hoTDixkGXOa0hPjuLwqn8ur8uns6qG6rs31cm3cyUvvbSc6SphclMa8MtfLlZ9mt/cxJ7Zjbyd1O/dxeVVesEsxxviJhSxj/CA+xsPcMjcpvrdXebd5ty9w7eCuP9Zy1x9rGZM9jHllWcwancHZecnWW2GOceRWOjYfy5iwYSHLGD+Lijp6L8UfzR/Dlvb9vOabOP/rN+v4j6V1xEVHUVmQwpQiL1OK06jMT7X7Kka4ZXVtpA2NpSw7KdilGGP8xEKWMQE2wjuU688p5vpzitl94BArGztY1djBysZ2/v2Nzdz7OsR4hHF5KUwuSmNyURpVhWkkxtm3Z6RQVZbXtTOtxEtUlF2paky4sFbcmAGUkhDLhRXZXFiRDbh1kdZ8vIuVvtD14FsN/PrNejxRQkVOElOK0phc5GVyYZpNog9j9a37+WRvJzNtqNCYsGIhy5ggSoqPYfaYTGaPyQTgwKFu1m7ZzarGdt5p7OCxFVt46O1GRGB01jCmFnuP9HalJ8YFuXrjL8vrffOxSixkGRNOLGQZM4gkxEYzc2Q6M0e6H7adXT28u3W3b3ixg6dWb+XR5R8DUJIxlCnFXqYUpTGlyEt2cnwQKzdnYtnmNvLThlDgtStQjQknFrKMGcTiYzwuSBV7uRno6unl/ZY9LnQ1tPPi+m38fmUTAAVpCb7hxTSmFnvJSx1iK9GHgO6eXlY0tPOls4YHuxRjjJ9ZyDImhMR4ophQkMqEglS+M6uEnl5l4/a9bk5XQzuvbdzBojXNAAxPjj8yp2taiZdCb4KFrkHog217+bSz25ZuMCYMWcgyJoR5ooSxucmMzU3muplF9PYqm3fuOzKnq7q+nefXbwMgJzmeaSXpTC/xMqM03YYXB4nD62NNL/EGuRJjjL/1K2SJyHzgXsADPKyqdx/3+a3A9UA30Ap8S1W3+D77S+Cnvk1/pqqP+al2Y8xxoqKE0dnDGJ09jGumFaKqNLbtZ3l9O8vr23jjwx3891rX01WcPpTppV6ml6QzrdhL6tDYIFcfmZZtbqNseBJeu5DBmLDzhSFLRDzAfcD5QDOwWkReUNXaPputA6pU9YCIfBf4OXCFiKQB/wBUAQqs8e27y98nYoz5cyJCcUYixRmJXD11BL29ysZP9rKivp3l9e08t7aF377ThAiUZScxwxe6JhXZOl0D4bNDPazZsou/nD4i2KUYYwKgP63oZKBOVRsARORJYAFwJGSp6tI+278DXO17fiHwmqp2+PZ9DZgPPHHmpRtjTlVUlFCRk0xFTjLXn1NMV08v7zXvYXldG8vr23lsuVsyIjpKGJefwvQSF7oqC1KIj7EV6f2tZksHh3p6bT6WMWGqPyErF9ja53UzMOUk218HvHKSfXNPpUBjTODEeKKYOCKViSNSuXnuSDq7XM9KtS903be0jn9/w90GaFJhGtN887nG5iQR7YkKdvkhb1ldGzEedyNxY0z48et4gIhcjRsanHWK+90A3ABQUFDgz5KMMacgPsbDjNL0Iz0rezu7WNXQQXV9Gyvq27ln8Ufcs/gjhsVFM6XY63q6Sr2MzhpmVy6ehuV17VQWpJIQa0OzxoSj/nxntwD5fV7n+d47hojMA34CzFLVg332Pe+4fd88fl9VfRB4EKCqqkr7UZMxZgAkxccwrzyLeeVZALTtO+ibz+V6upZs3AFAemIsU4tdL9f0Ei8FabZcxBfZfeAQH2zbwy3zRgW7FGNMgPQnZK0GRopIES40LQSu6ruBiFQCvwHmq+rOPh8tBv5ZRFJ9ry8Abj/jqo0xQZGeGMdXxuXwlXE5ADTvOuCuXPQNL/7xve0A5KYM4f98/SzOHZURzHKBfl0dXQA8BqT4trlNVV/2fXY7bgpED/A9VV3sr7pW1LejCjNKbekGY8LVF4YsVe0WkZtwgckDPKKqG0TkLqBGVV8A7gESgUW+316bVPUSVe0Qkf+NC2oAdx2eBG+MCX15qQlcXpXA5VX5qCr1rftdL1dd+6BYh6ufV0f/FHhaVe8XkXLgZaDQ93whUAHkAEtEZJSq9vijtoS4aOaMyeTsvBR/HM4YMwj1ayKA77e6l497744+z+edZN9HgEdOt0BjTGgQEUozEynNTOTaaYXBLuewL7w6Gre8TJLveTKwzfd8AfCkb/pDo4jU+Y63wh+FzRqVwaxB0NNnjAkcuzzIGBPO+nOF853A1SLSjPtl8uZT2NcYY07IQpYxJtJdCTyqqnnAxcDjInJKbaOI3CAiNSJS09raGpAijTGhx0KWMSac9efq6OuApwFUdQUQD6T3c198+z2oqlWqWpWRYUOAxhjHQpYxJpwduTpaRGJxE9lfOG6bJmAugIiU4UJWq2+7hSIS57u6eiSwasAqN8aEPFsBzxgTtvp5dfQPgIdE5BbcJPhvqqoCG0Tkadwk+W7gRn9dWWiMiQwWsowxYa0fV0fXAjNOsO8/Af8U0AKNMWHLhguNMcYYYwLAQpYxxhhjTACIm3oweIhIK7DlFHZJB9oCVM5gEgnnGQnnCHaen2eEqobFZXmn2IbZ10J4iYTzjIRzBD+1X4MuZJ0qEalR1apg1xFokXCekXCOYOdpjoqUfyM7z/ARCecI/jtPGy40xhhjjAkAC1nGGGOMMQEQDiHrwWAXMEAi4Twj4RzBztMcFSn/Rnae4SMSzhH8dJ4hPyfLGGOMMWYwCoeeLGOMMcaYQSdkQ5aIzBeRj0SkTkRuC3Y9gSAi+SKyVERqRWSDiPxNsGsKJBHxiMg6EfljsGsJBBFJEZFnRORDEdkoItOCXVMgiMgtvq/XD0TkCRGJD3ZNg5G1YeEl3NsviIw2zN/tV0iGLBHxAPcBFwHlwJUiUh7cqgKiG/iBqpYDU4Ebw/Q8D/sbYGOwiwige4H/UdUxwDjC8FxFJBf4HlClqmNx9wtcGNyqBh9rw8JSuLdfEOZtWCDar5AMWcBkoE5VG1T1EPAksCDINfmdqm5X1bW+55/ivqBzg1tVYIhIHvAl4OFg1xIIIpIMnAv8PwBVPaSqu4NbVcBEA0NEJBpIALYFuZ7ByNqwMBLu7RdEVBvm1/YrVENWLrC1z+tmwvAbty8RKQQqgZXBrSRgfgX8COgNdiEBUgS0Av/pG1J4WESGBrsof1PVFuAXQBOwHdijqq8Gt6pBydqw8BLu7RdEQBsWiPYrVENWRBGRROC/ge+r6t5g1+NvIvJlYKeqrgl2LQEUDUwA7lfVSmA/EHbzcEQkFdcjUwTkAENF5OrgVmWCLZzbsAhpvyAC2rBAtF+hGrJagPw+r/N874UdEYnBNU6/U9Vng11PgMwALhGRj3HDJnNE5LfBLcnvmoFmVT38W/wzuAYr3MwDGlW1VVW7gGeB6UGuaTCyNix8REL7BZHRhvm9/QrVkLUaGCkiRSISi5uY9kKQa/I7ERHc+PdGVf1lsOsJFFW9XVXzVLUQ93/5hqqGVe+Hqn4CbBWR0b635gK1QSwpUJqAqSKS4Pv6nUuYTY71E2vDwkQktF8QMW2Y39uvaL+UNcBUtVtEbgIW42b/P6KqG4JcViDMAK4B3heR9b73fqyqLwexJnP6bgZ+5/uh2gD8ryDX43equlJEngHW4q4sW0fkrBDdb9aGWRsWosK6DQtE+2UrvhtjjDHGBECoDhcaY4wxxgxqFrKMMcYYYwLAQpYxxhhjTABYyDLGGGOMCQALWcYYY4wxAWAhyxhjjDEmACxkGWOMMcYEgIUsY4wxxpgA+P9kA1Yz9yynjAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "markdown", "metadata": { "id": "kkCjHY9maEif" }, "source": [ "## Evaluate Model Performance on Test Data" ] }, { "cell_type": "code", "metadata": { "id": "gRx2oIbVZGO2", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "9fdc1a50-c181-4688-81a8-ab71fb313946" }, "source": [ "y_pred = model.predict(test_images)\n", "y_pred = y_pred.argmax(axis=1)\n", "print(classification_report(y_test, y_pred,\n", " target_names=class_names))" ], "execution_count": 34, "outputs": [ { "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " T-shirt/top 0.81 0.89 0.85 1000\n", " Trouser 0.99 0.98 0.98 1000\n", " Pullover 0.82 0.89 0.85 1000\n", " Dress 0.90 0.92 0.91 1000\n", " Coat 0.86 0.84 0.85 1000\n", " Sandal 0.98 0.98 0.98 1000\n", " Shirt 0.76 0.64 0.69 1000\n", " Sneaker 0.95 0.97 0.96 1000\n", " Bag 0.98 0.97 0.98 1000\n", " Ankle boot 0.97 0.96 0.97 1000\n", "\n", " accuracy 0.90 10000\n", " macro avg 0.90 0.90 0.90 10000\n", "weighted avg 0.90 0.90 0.90 10000\n", "\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "QgT4HFXTesh2" }, "source": [ "# Training a CNN with 1-Cycle Learning Rate" ] }, { "cell_type": "code", "metadata": { "id": "3Ziq4Cl5c5FL" }, "source": [ "def lr_function(epoch):\r\n", " start_lr = 1e-4; min_lr = 5e-4; max_lr = 5e-2\r\n", " rampup_epochs = 4; sustain_epochs = 0; exp_decay = .8\r\n", " \r\n", " def lr(epoch, start_lr, min_lr, max_lr, rampup_epochs, \r\n", " sustain_epochs, exp_decay):\r\n", " if epoch < rampup_epochs:\r\n", " lr = ((max_lr - start_lr) / rampup_epochs \r\n", " * epoch + start_lr)\r\n", " elif epoch < rampup_epochs + sustain_epochs:\r\n", " lr = max_lr\r\n", " else:\r\n", " lr = ((max_lr - min_lr) * \r\n", " exp_decay**(epoch - rampup_epochs -\r\n", " sustain_epochs) + min_lr)\r\n", " return lr\r\n", "\r\n", " return lr(epoch, start_lr, min_lr, max_lr, \r\n", " rampup_epochs, sustain_epochs, exp_decay)\r\n", " " ], "execution_count": 50, "outputs": [] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 298 }, "id": "JpaBeFNhc5wa", "outputId": "9cbc6ad9-ff5a-417d-ce4e-be21f42d1914" }, "source": [ "import matplotlib.pyplot as plt\r\n", "\r\n", "rng = [i for i in range(50)]\r\n", "y = [lr_function(x) for x in rng]\r\n", "plt.plot(rng, [lr_function(x) for x in rng])\r\n", "print('start lr:', y[0], '\\nend lr:', y[-1])" ], "execution_count": 51, "outputs": [ { "output_type": "stream", "text": [ "start lr: 0.0001 \n", "end lr: 0.0005021560290768111\n" ], "name": "stdout" }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3Rc5X3u8e9vZnS/WrJkW7Js+YqRYyDGYIekhEChkFCcpiSBQwO0dJG00MtJe05Jz0lKaHtyaM8q6TohTUggIWlTICQ0TkJKLpDAIcRY5hKwjcE2xpJ8k62RLVvXmXnPH7PHDMPIGkkzmtGe57OW1+zbjH97MTzz+t3vfrc55xAREf8K5LsAERHJLQW9iIjPKehFRHxOQS8i4nMKehERnwvlu4BUc+fOde3t7fkuQ0RkVtm6desR51xTun0FF/Tt7e10dnbmuwwRkVnFzN4Yb5+6bkREfE5BLyLicwp6ERGfU9CLiPicgl5ExOcyCnozu9zMdprZLjO7Lc3+MjN70Nu/2czave3tZjZkZi94f76U3fJFRGQiEw6vNLMgcDdwKdANbDGzTc657UmH3QSEnXPLzewa4E7go96+3c65c7Jct4iIZCiTFv35wC7n3B7n3CjwALAx5ZiNwP3e8sPAJWZm2SuzMJwcifDtzi40tbOIzCaZBH0r0JW03u1tS3uMcy4CHAMavX1LzOx5M/uFmf1Gur/AzG42s04z6+zt7Z3UCcykTS/u5789/Gt2HBjIdykiIhnL9cXYA8Ai59w7gU8C3zKz2tSDnHP3OOfWOefWNTWlvYO3IOzrGwTgtcMKehGZPTIJ+h6gLWl9obct7TFmFgLqgKPOuRHn3FEA59xWYDewcrpF50t3eAiA3b0n81yJiEjmMgn6LcAKM1tiZqXANcCmlGM2ATd4y1cDjzvnnJk1eRdzMbOlwApgT3ZKn3nd4XiLfvfhE3muREQkcxOOunHORczsVuAxIAjc55zbZmZ3AJ3OuU3AvcA3zWwX0Ef8xwDgQuAOMxsDYsAnnHN9uTiRmdDVl2jRK+hFZPbIaPZK59yjwKMp2z6TtDwMfDjN+74DfGeaNRaE4bEoR06MUBI09hw5STTmCAZ8N7BIRHxId8ZmKNE/f157A6OR2KluHBGRQqegz1Ai2N+7Mj4qaJf66UVkllDQZyjRor/ojGZA/fQiMnso6DPUHR6iJGisaK5mbnWZWvQiMmso6DPUFR6ktb6CQMBY3lyloBeRWUNBn6Hu8BAL51QCsKypmt29JzXnjYjMCgr6DPWEB1k4pwKA5c3VHBsa48iJ0TxXJSIyMQV9BoZGoxw5MXoq6Jc1VQO6ICsis4OCPgM9/fGhlW0N8a6b5c3xoFc/vYjMBgr6DCSmPki06BfUlVNZGlSLXkRmBQV9BhI3SyUuxpoZy5qq1aIXkVlBQZ+B7vAQpcEATdVlp7Ytb65mj6YrFpFZQEGfge7wEK1z4mPoE5Y1VdHTP8TJkUgeKxMRmZiCPgPdSUMrExIXZNWqF5FCp6DPQPLNUgkaYikis4WCfgInRyIcPTn6thb94sYqggHTBVkRKXgK+gn09L91aGVCaSjA4oZKtehFpOAp6CeQOrQy2bJmDbEUkcKnoJ9AYh76tpQWPcT76fcePUkkGpvpskREMqagn0B3eIiyUICmmrK37VveXM1Y1LGvT48VFJHCpaCfQHd4kNY5FZi9/UHgy5qqANitIZYiUsAU9BPo6nv70MqEZZrcTERmAQX9BNLdLJVQW15Cc40eKygihU1BfxonRiKEB8fGDXqI99NriKWIFDIF/Wn0nBpxk77rBrygP3xCjxUUkYKloD+NN8fQj9+iX9ZUzcBIhN6BkZkqS0RkUhT0p5EYQz/exVjQ06ZEpPAp6E+jq2+QslCAudWl4x6jyc1EpNAp6E8jPmtl+jH0CfNqy6guC6lFLyIFS0F/Gt39g6ceCD6e+GMFq9ilFr2IFCgF/WkkWvQTWdZcze7DujtWRApTRkFvZpeb2U4z22Vmt6XZX2ZmD3r7N5tZe8r+RWZ2wsz+Mjtl597A8Bj9g2OnvRCbsKypmoPHhxkYHpuBykREJmfCoDezIHA3cAXQAVxrZh0ph90EhJ1zy4G7gDtT9v8T8KPplztz3hxxM3GLXo8VFJFClkmL/nxgl3Nuj3NuFHgA2JhyzEbgfm/5YeAS865gmtkHgdeBbdkpeWZkMrQyIRH0Ow8N5LQmEZGpyCToW4GupPVub1vaY5xzEeAY0Ghm1cBfAZ893V9gZjebWaeZdfb29mZae04lbpZKNw99qvbGKipLg2zffzzXZYmITFquL8beDtzlnDvtkBTn3D3OuXXOuXVNTU05Likz3eEhKkqCNFSNP4Y+IRgwVrfU8lLPsRmoTERkckIZHNMDtCWtL/S2pTum28xCQB1wFFgPXG1m/wDUAzEzG3bOfWHaledYYtbK042hT7a6pY4Ht3QRjTmCgczeIyIyEzJp0W8BVpjZEjMrBa4BNqUcswm4wVu+Gnjcxf2Gc67dOdcOfB74X7Mh5CHzoZUJa1rrGBqLskfj6UWkwEwY9F6f+63AY8AO4CHn3DYzu8PMrvIOu5d4n/wu4JPA24ZgzjZdfYMZXYhNWLOwDkDdNyJScDLpusE59yjwaMq2zyQtDwMfnuAzbp9CfXlxbGiM48ORSbXol86torwkwEs9x/jQ2oU5rE5EZHJ0Z2wap+ahn2D6g2ShYICOBbVs69HIGxEpLAr6NDKZhz6dNa11bNt/jFhMDyERkcKhoE9jMjdLJVvdWsfJ0Sh7jugOWREpHAr6NLrDQ1SWBplTWTKp961pjV+Q3bZfF2RFpHAo6NPomuQY+oQVzdWUhQK81K2gF5HCoaBPozs8dNoHgo8nFAywaoHukBWRwqKgTyNxV+xUrGmtZfv+47ogKyIFQ0Gf4tjQGAPDkUlfiE1Y01rHwEiEN/oGs1yZiMjUKOhTTHVoZcLqFt0hKyKFRUGfYqpDKxNWzquhNBhgm4JeRAqEgj5FV9/0WvSloQCrFtSoRS8iBUNBn6I7PER1WYj6SY6hT7a6pY6Xe47hnC7Iikj+KehTJKYnnuwY+mRrWus4Phyhq28oi5WJiEyNgj7FdIZWJiTukFX3jYgUAgV9EuccPeGhKV+ITVg5v5qSoPGypkIQkQKgoE9ybGiMgZHJzUOfTlkoyMp5NbysFr2IFAAFfZLpDq1Mtqa1jpd0QVZECoCCPsl0b5ZKtrq1jv7BMXr6dUFWRPJLQZ8k0aKfyoRmqRIXZNV9IyL5pqBP0h0eoqYsRG1FRo/SPa1V82sIBkwjb0Qk7xT0SbrDg7ROcwx9QnlJkBXN1bysZ8iKSJ4p6JN09U1/aGWyNa26Q1ZE8k9B73HO0R0epK1h+hdiE9YsrOPoyVEOHBvO2meKiEyWgt7TPzjGydFoVlv0iSmLdUFWRPJJQe95cwx99lr0HQtqCZimQhCR/FLQe7I5hj6hojRIR0stW/b2Ze0zRUQmS0HvyeZdscnWL2nkuX39DI9Fs/q5IiKZUtB7usKD1JSHqKuY+jz06WxY2shoJMaLXf1Z/VwRkUwp6D3d4aGs3BGb6vz2Bsxg8+vqvhGR/FDQe7IxD306dZUlrJpfy6/2HM36Z4uIZCKjoDezy81sp5ntMrPb0uwvM7MHvf2bzazd236+mb3g/XnRzH4nu+VnR3wMfXZvlkq2YWkDz+0LMxqJ5eTzRUROZ8KgN7MgcDdwBdABXGtmHSmH3QSEnXPLgbuAO73tLwPrnHPnAJcDXzaz6U8kk2XhwTEGR6M5adFD/ILs8FiMX3ern15EZl4mLfrzgV3OuT3OuVHgAWBjyjEbgfu95YeBS8zMnHODzrmIt70cKMi5AHIxtDLZ+UsaANR9IyJ5kUnQtwJdSevd3ra0x3jBfgxoBDCz9Wa2DXgJ+ERS8J9iZjebWaeZdfb29k7+LKYp8RDvXHXdNFSVsmp+jS7Iikhe5PxirHNus3NuNXAe8CkzK09zzD3OuXXOuXVNTU25LultTrXoszjPTar1Sxro3BtmLKp+ehGZWZkEfQ/QlrS+0NuW9hivD74OeEs/hXNuB3ACeMdUi82V7vAQdRUl1JZndwx9sg1LGxkai/Lrbk2HICIzK5Og3wKsMLMlZlYKXANsSjlmE3CDt3w18LhzznnvCQGY2WJgFbA3K5VnUa6GViZL9NNvfl399CIysyYMeq9P/VbgMWAH8JBzbpuZ3WFmV3mH3Qs0mtku4JNAYgjme4AXzewF4BHgj51zR7J9EtMVH1qZ26BvrC5jRXM1v9qjfnoRmVkZDXV0zj0KPJqy7TNJy8PAh9O875vAN6dZY04lxtBfuDL31wY2LG3ku891E4nGCAV1r5qIzIyiT5ujJ0cZGovSluMWPcD6pQ2cHI3y8n49XlBEZk7RB32uZq1MR+PpRSQfFPQzMLQyobmmnGVNVWxW0IvIDFLQey361vrcBz3A+qWNdO4NE9F4ehGZIQr68CD1lSXU5HAMfbINSxsZGImw/YD66UVkZhR90Hf15X5oZbINifH0GmYpIjOk6IO+OzyYkweOjKe5tpwlc6t0QVZEZkxRB/2b89DPXIse4vPTP7u3j2isICfzFBGfKeqgP3JilJFIbEaGViZbv6SRgeEIO9RPLyIzoKiDPtfz0I9n/VKNpxeRmVPkQT9zN0slW1BXweLGSs17IyIzoqiDvitPLXqA9yyfyy93H2F4LDrjf7eIFJeiDvru8BANVaVUlc38Y2wv7ZjH4GiUX+4uuMk8RcRnij7o89GaB3jXskaqy0L8ZPuhvPz9IlI8ijzoB2ds6oNUZaEg713ZxE93HCamYZYikkNFG/TOOXrCQ7Q1zOyF2GSXrZ5H78AIL3T3560GEfG/og363hMj3hj6/LToAS46o5lQwPjxNnXfiEjuFG3Qd/UlhlbmL+jrKkpYv7SBn2w/mLcaRMT/ijboEzdLzeQ8N+lceuY8dveeZE/vibzWISL+VcRB781Dn8cWPcClq+cDaPSNiORMUQd9Y1UplaUzP4Y+WWt9BatbahX0IpIzRRz0g3ntn092acc8tu4L0zswku9SRMSHijboe8JDMz7HzXgu7ZiHc/D4K2rVi0j2FWXQx2LePPQz8EDwTHQsqKW1vkLdNyKSE0UZ9L0nRhiNzvw89OMxMy7tmMdTrx1hcDSS73JExGeKMujzNQ/96VzWMY+RSIynXtMkZyKSXUUa9PGhlW0FFPTnLWmgtjyku2RFJOuKOuhb6wuj6wagJBjg4lXNPP7KISLRWL7LEREfKdKgH2RudSkVpcF8l/IWl3bMJzw4xtY3wvkuRUR8pCiDvquvcIZWJnvvGU2UBgMafSMiWVWUQV9IN0slqy4LccHyRn68/RDOaY56EcmOjILezC43s51mtsvMbkuzv8zMHvT2bzazdm/7pWa21cxe8l4vzm75kxeLOXr6C7NFD/D+NQvY1zfIc/vUfSMi2TFh0JtZELgbuALoAK41s46Uw24Cws655cBdwJ3e9iPAbzvn1gA3AN/MVuFTdXhghLGoK8gWPcAH1iygqjTIg1u68l2KiPhEJi3684Fdzrk9zrlR4AFgY8oxG4H7veWHgUvMzJxzzzvn9nvbtwEVZlaWjcKnqhDH0CerKgtx5Vkt/ODXBzgxopunRGT6Mgn6ViC5edntbUt7jHMuAhwDGlOO+V3gOefc22buMrObzazTzDp7e3szrX1Kuk4FfWF23QB85Lw2Bkej/PDX+yc+WERkAjNyMdbMVhPvzvl4uv3OuXucc+ucc+uamppyWkt3ATxZaiJrF9WzvLla3TcikhWZBH0P0Ja0vtDblvYYMwsBdcBRb30h8AhwvXNu93QLnq7u8BBNNWWUlxTWGPpkZsZH17Xx3L5+dh0eyHc5IjLLZRL0W4AVZrbEzEqBa4BNKcdsIn6xFeBq4HHnnDOzeuCHwG3OuaezVfR0dPcX5tDKVL+ztpVQwNSqF5FpmzDovT73W4HHgB3AQ865bWZ2h5ld5R12L9BoZruATwKJIZi3AsuBz5jZC96f5qyfxSR0F9A89Kczt7qM3zxzHt99rofRiKZEEJGpy+g5es65R4FHU7Z9Jml5GPhwmvf9HfB306wxa6Ixx/7+Id6/ZkG+S8nIR85byH9uO8jjrxzi8nfMjppFpPAU1Z2xh44PMxZ1tM2CFj3AhSuamFdbpu4bEZmWogr6xKyVs6GPHiAUDHD1uQv5xau9HDw2nO9yRGSWKrKgL+ybpdL5yLo2Yg4e3qpWvYhMTZEFfbxF31I/e4J+cWMVG5Y28FBnN7GYJjoTkckrsqAfpLnAx9Cn89Hz2tjXN8iv9hzNdykiMgsVWdAPzapum4Qr3rGAmvIQD3aq+0ZEJq+ogr4rPEhbw+wYcZOsvCTIxnNa+NHLB+kfHM13OSIyyxRN0EeiMQ70D8/KFj3AdesXMxqJ8c1n3sh3KSIyyxRN0B8aGCESc7Pirth0zlxQyyWrmrn36dc5qemLRWQSiibou/tm39DKVLdcvJz+wTG+tXlfvksRkVmkeIL+1M1Ss7NFD7B20RzevbyRe57aw/BYNN/liMgsUTRB3xUexAxa6svzXcq03Pq+FfQOjPCQRuCISIaKJui7w0PMqymnLDS7xtCn2rC0gXWL5/DlX+zRrJYikpEiCvrZMQ/9RMyMWy5eTk//EP/xfOrzX0RE3q6Ign523iyVzkUrm3hHay1f/PkuIlG16kXk9Ioi6CPRGAeODc/qC7HJzIxb37eCvUcH+eFLB/JdjogUuKII+oPHh4nGnG9a9ACXdcxj5bxq7n5ilyY7E5HTKoqg7+qb/UMrUwUCxi3vW86rh07w4+2H8l2OiBSwogj6xDz0bQ3+adEDXHlWC+2Nldz9xC6cU6teRNIrkqAfwgwW1Pkr6IMB448vWs5LPcd4YufhfJcjIgWqaIJ+fm05pSH/ne4H39nKkrlV/O0PdjAS0d2yIvJ2/ku+NPwyhj6d0lCAz161mtePnOQrT+7JdzkiUoCKJOiHfHUhNtWFK5t4/5r5fOGJXXR5k7eJiCT4PujHojEOHPPPzVLj+fSVHQTM+Oz3t+e7FBEpML4P+oPHhok5aPNxix7iF5r/9JIV/HTHIX62Q8MtReRNvg/6rvDsn4c+U3/w7iUsb67m9u9v0zTGInKK74PeD/PQZ6o0FOCOjavp6hviiz/fne9yRKRAFEXQBwzm183ueegzdcGyuVx1dgtf+sVu9h45me9yRKQAFEHQD/p2DP14/scHzqQ0GOD272/THbMiUgRB3zfEwgb/d9skm1dbzp//5gp+vrOXx7YdzHc5IpJn/g96H98sdTo3XtBOx4JaPvXdl9jfP5TvckQkjzIKejO73Mx2mtkuM7stzf4yM3vQ27/ZzNq97Y1m9oSZnTCzL2S39ImNRmIcPO6feegnIxQM8H//yzsZjcT4k39/njE9oESkaE0Y9GYWBO4GrgA6gGvNrCPlsJuAsHNuOXAXcKe3fRj4NPCXWat4EhJj6IuxRQ+wrKmaz/3uWWx9I8z/eWxnvssRkTzJpEV/PrDLObfHOTcKPABsTDlmI3C/t/wwcImZmXPupHPu/xEP/BnXXURj6Mdz1dktXLd+EV9+cg8/1bz1IkUpk6BvBbqS1ru9bWmPcc5FgGNAY6ZFmNnNZtZpZp29vb2Zvm1CiZul/H5X7EQ+fWUHq1tq+Ytvv3jqx09EikdBXIx1zt3jnFvnnFvX1NSUtc/tDg8RDBgLimQM/XjKS4J88bq1xGKOW771PKMR9deLFJNMgr4HaEtaX+htS3uMmYWAOuBoNgqcjsQ89KFgQfye5dXixir+4eqzeLGrn8/9aEe+yxGRGZRJAm4BVpjZEjMrBa4BNqUcswm4wVu+GnjcFcCdOsU6tHI8V6xZwI0XtPO1p/fyny8fyHc5IjJDJgx6r8/9VuAxYAfwkHNum5ndYWZXeYfdCzSa2S7gk8CpIZhmthf4J+BGM+tOM2InZ/w+D/1U/PX7z+Tstnr+64MvsvWNcL7LEZEZEMrkIOfco8CjKds+k7Q8DHx4nPe2T6O+KXtzDL1a9MlKQwG+cv25fORLz/D7X3uWf795A6tb6vJdlojkkG87r/f3D+GKeAz96TTXlPOvf7ieqrIQ19/7LLt7T+S7JBHJId8GfWJ64rYim+cmUwvnVPJvf7geM/i9r27WsEsRH/Nx0OtmqYksbarmG3+wnpMjEa776mYOH8/LfW0ikmM+Dvr4GPr5tcU9hn4iHS21fP0Pzqd3YISP3fss/YOj+S5JRLLMx0E/yII6jaHPxNpFc/jK9et4/ehJrr/vWY6eGMl3SSKSRb5NwfjQSnXbZOrdy+fyL9etZefBAT74xad57dBAvksSkSzxbdB3hQeLfo6bybrkzHk8+PF3MTQa40P/8kueei178w6JSP74MuhHIlEOHR/RzVJTcE5bPd+79d201ldw49e28K+/eiPfJYnINPky6Pf3x0ePqOtmalrrK3j4jy7gvSub+J//8TJ3fH870VjeZ7QQkSnyZdBraOX0VZeF+Mr16/j9d7dz39Ovc/M3Ojk2NJbvskRkCnwa9PGbpYrtoeDZFgwYf/Pbq/nbD76Dn7/ay2/d9SS/eFX99iKzjS+DvqtvkJDG0GfNxzYs5pE/voCa8hA33Pcsn/ruS5wYieS7LBHJkC+Dvjs8REt9BcGA5bsU3zhrYT3f/5P38PELl/LAln1c/vkneWZ33h85ICIZ8GnQax76XCgvCfKp95/Jtz/+LkIB49qv/IrbN21jcFSte5FC5tOg181SubSuvYFH/+w3uPGCdr7+y71c9I8/51ub9xGJ6hGFIoXId0E/PBbl8IDG0OdaZWmI269azXf+6F0saqjkrx95ics+/yT/+fJBCuDhYiKSxHdBv7/fG3GjFv2MOHdxA9/+xLu452PnYsAn/nUrV3/pGTr39uW7NBHx+C7ouxJDK9WinzFmxmWr5/PYn1/I5z60hq6+Qa7+0jNcf9+z/HznYWK62UokrzJ6lOBskrhZqq1BLfqZFgoGuPb8RWw8p4WvPb2Xr/9yLzd+bQvLmqq48YJ2PrR2IVVlvvvKiRQ837Xou8NDlASN5hqNoc+XytIQt7xvOU//1cV8/qPnUF0W4tPf28aGz/2Mv//hdvYd1dOsRGaS75pXGkNfOEpDAT74zlY2ntPCc/v6+drTr3Pf03v5ylOvs3ZRPVed3cIHzmqhqaYs36WK+JoPg15j6AuNmXHu4jmcu3gOB44N8cjzPWx6YT+3f387d/xgOxcsm8tVZ7fwW6vnU1dZku9yRXzHCm0o3Lp161xnZ+fU3/93P+WSVc3cefVZWaxKcuG1QwNsenE/33thP/u8aSvWLp7De1c2ceGKJla31BLQv8xEMmJmW51z69Lt81WLfngsypETI2rRzxIr5tXwF5edwScvXcmvu4/xo5cP8uSrvfzjYzv5x8d20lhVyntWzOXCFU2c195AW0MFZgp+kcnyVdAnZq1s06yVs4qZcXZbPWe31XPbFavoHRjhqdd6efLVXp567Qjfe2E/AHOry3jnonrWLprD2kX1nLWwnorSYJ6rFyl8Pgt6zUPvB001ZXxo7UI+tHYhsZjjlYMDbN0X5vk3wjzf1c9Pth8CIBQwljdXc8b8GlbNr2XV/BpWLahhfm25Wv4iSXwW9LpZym8CAaOjpZaOllo+tmExAH0nR3l+X5jn9/Wz/cBxOveGT7X6AWrLQ6yYV8PixkqWNFaxeG4V7Y2VtM+torZcF3ul+Pgu6ONj6DVcz88aqkq55Mx5XHLmvFPbjg2N8eqhAV45cJxXDg6wu/cEz+w+ynef63nLe+srS1hQV0FLXTkL6svjy95rU00Zc6vLqC0P6V8E4iu+Cvqu8CCt9RUaqVGE6ipKOK+9gfPaG96yfWg0yr6+QfYePcneIyfZ1zfIgWPD9PQP0flGOO3jEUtDAZqqy5hbU0ZTdSn1laXMqSyhvrKU+soS5lSWUl9RQm1FCTXlIWrK468lQd/dfyg+4aug7w4P6UKsvEVFaZAz5tdwxvyatPtPjkQ4cGyIA8eGOXJihCMDo/SeGOHIwAi9J0bo6R9m2/7jhAdHGR47/TTM5SWBeOiXhagsC1JZGqKqNEhlmfdaGqK8JEh5SYCKkiDlJUEqSoKUlQQoC3mvwQBlJQFKg/H1kmCAkqBRGgpQGkysx7fpXx2SqYyC3swuB/4ZCAJfdc7975T9ZcA3gHOBo8BHnXN7vX2fAm4CosCfOucey1r1KXrCg3R0zJv4QBFPVVmI5c01LG9O/0OQbHgsSv/gGOHBUcKDoxwfinBiJMLA8BgDw/HX+HqEodEoJ0cjHD05yr6+QQZHo5wciTAciTEayc68/cGAEQoYJcEAoaARCsR/ABLbg6f+BAgFjEDACFr8fQF7c3/AjIC33cwIevvMOLUvYPF9AePUdjPe3EZiX/zH59Qx3rJ5y3jHmvHmvqR1kt4TP9ySljn12fHXt/7QJb/nretvPyb5uFSn+/0c78d1vLdM9rd4ydwqLjqjeXJvysCEQW9mQeBu4FKgG9hiZpucc9uTDrsJCDvnlpvZNcCdwEfNrAO4BlgNtAA/NbOVzrlotk9kaDTKkROjuhArOVNeEmR+XZD5ddObRykac4xEogyNRhkaizI8FmXE+wFIfh2JRBmLxhiLOEajsfhyNL5/LOqIxGJEou7U8ljUEYnGiDpHNOaIxBzRqCPqEtshFovvizrHaCR+bMzbHvPel3h1gHMQc/FtsVh82TlwxN/nHLjEfm/ZAThwvPV4520jZT3xngK7dzMvrjxrQX6CHjgf2OWc2wNgZg8AG4HkoN8I3O4tPwx8weI/fRuBB5xzI8DrZrbL+7xnslP+m3r6NbRSZodgwKgsDVFZ6que06xJ3K1/6oeBpB+QU+veK+4t6+PtTz1mvN+U5JkCUo8Z94donO1u3L9lfKEcXefJ5JvWCnQlrXcD68c7xjkXMbNjQKO3/Vcp721N/QvM7GbgZoBFixZlWnvqZ/CBNQtYkcE/wUWkcCV3/yRtzUstflEQwwScc/c459Y559Y1NTVN6TOWNVVz93Vr6SKgrYsAAAQ1SURBVGipzXJ1IiKzWyZB3wO0Ja0v9LalPcbMQkAd8YuymbxXRERyKJOg3wKsMLMlZlZK/OLqppRjNgE3eMtXA4+7eGfXJuAaMyszsyXACuDZ7JQuIiKZmLCP3utzvxV4jPjwyvucc9vM7A6g0zm3CbgX+KZ3sbWP+I8B3nEPEb9wGwFuycWIGxERGZ/v5qMXESlGp5uPviAuxoqISO4o6EVEfE5BLyLicwp6ERGfK7iLsWbWC7wxjY+YCxzJUjmzic67uOi8i0sm573YOZf2jtOCC/rpMrPO8a48+5nOu7jovIvLdM9bXTciIj6noBcR8Tk/Bv09+S4gT3TexUXnXVymdd6+66MXEZG38mOLXkREkijoRUR8zjdBb2aXm9lOM9tlZrflu55cMbP7zOywmb2ctK3BzH5iZq95r3PyWWMumFmbmT1hZtvNbJuZ/Zm33dfnbmblZvasmb3onfdnve1LzGyz931/0JtC3HfMLGhmz5vZD7z1YjnvvWb2kpm9YGad3rYpf9d9EfRJDzC/AugArvUeTO5HXwcuT9l2G/Az59wK4Gfeut9EgL9wznUAG4BbvP/Gfj/3EeBi59zZwDnA5Wa2AbgTuMs5txwIAzflscZc+jNgR9J6sZw3wPucc+ckjZ+f8nfdF0FP0gPMnXOjQOIB5r7jnHuS+Jz/yTYC93vL9wMfnNGiZoBz7oBz7jlveYD4//yt+PzcXdwJb7XE++OAi4GHve2+O28AM1sIfAD4qrduFMF5n8aUv+t+Cfp0DzB/20PIfWyec+6At3wQmJfPYnLNzNqBdwKbKYJz97ovXgAOAz8BdgP9zrmId4hfv++fB/47EPPWGymO84b4j/mPzWyrmd3sbZvyd33CJ0zJ7OKcc2bm2zGzZlYNfAf4c+fc8XgjL86v5+49le0cM6sHHgFW5bmknDOzK4HDzrmtZnZRvuvJg/c453rMrBn4iZm9krxzst91v7Toi/0h5IfMbAGA93o4z/XkhJmVEA/5f3POfdfbXBTnDuCc6weeAN4F1JtZoqHmx+/7u4GrzGwv8a7Yi4F/xv/nDYBzrsd7PUz8x/18pvFd90vQZ/IAcz9Lfjj7DcD38lhLTnj9s/cCO5xz/5S0y9fnbmZNXkseM6sALiV+feIJ4GrvMN+dt3PuU865hc65duL/Pz/unLsOn583gJlVmVlNYhm4DHiZaXzXfXNnrJm9n3ifXuIB5n+f55Jywsz+HbiI+LSlh4C/Af4DeAhYRHyK548451Iv2M5qZvYe4CngJd7ss/1r4v30vj13MzuL+IW3IPGG2UPOuTvMbCnxlm4D8Dzwe865kfxVmjte181fOueuLIbz9s7xEW81BHzLOff3ZtbIFL/rvgl6ERFJzy9dNyIiMg4FvYiIzynoRUR8TkEvIuJzCnoREZ9T0IuI+JyCXkTE5/4/1UzT/PV6lI4AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "id": "DEvv02Eqc9js" }, "source": [ "# define input shape\r\n", "INPUT_SHAPE = (28, 28, 1)\r\n", "\r\n", "# define sequential model\r\n", "model = tf.keras.models.Sequential()\r\n", "# define conv-pool layers - set 1\r\n", "model.add(tf.keras.layers.Conv2D(filters=16, kernel_size=(3, 3), strides=(1, 1), \r\n", " activation='relu', padding='valid', input_shape=INPUT_SHAPE))\r\n", "model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2)))\r\n", "# define conv-pool layers - set 2\r\n", "model.add(tf.keras.layers.Conv2D(filters=32, kernel_size=(3, 3), strides=(1, 1), \r\n", " activation='relu', padding='valid'))\r\n", "model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2)))\r\n", "\r\n", "# add flatten layer\r\n", "model.add(tf.keras.layers.Flatten())\r\n", "\r\n", "# add dense layers with some dropout\r\n", "model.add(tf.keras.layers.Dense(256, activation='relu'))\r\n", "model.add(tf.keras.layers.Dropout(rate=0.3))\r\n", "model.add(tf.keras.layers.Dense(256, activation='relu'))\r\n", "model.add(tf.keras.layers.Dropout(rate=0.3))\r\n", "\r\n", "# add output layer\r\n", "model.add(tf.keras.layers.Dense(10, activation='softmax'))" ], "execution_count": 52, "outputs": [] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "AgAL4vvCdhTI", "outputId": "917f0f13-47ed-44c8-ee73-72673570a854" }, "source": [ "epochs = 100\r\n", "\r\n", "callbacks = [\r\n", " tf.keras.callbacks.EarlyStopping(monitor='val_loss', \r\n", " patience=2, \r\n", " restore_best_weights=True),\r\n", " tf.keras.callbacks.LearningRateScheduler(lambda epoch: \\\r\n", " lr_function(epoch), \r\n", " verbose=True)\r\n", " \r\n", "]\r\n", "\r\n", "model.compile(\r\n", " optimizer=tf.keras.optimizers.Adam(1e-4),\r\n", " loss=\"sparse_categorical_crossentropy\",\r\n", " metrics=[\"accuracy\"],\r\n", ")\r\n", "\r\n", "model.summary()" ], "execution_count": 53, "outputs": [ { "output_type": "stream", "text": [ "Model: \"sequential_5\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "conv2d_8 (Conv2D) (None, 26, 26, 16) 160 \n", "_________________________________________________________________\n", "max_pooling2d_8 (MaxPooling2 (None, 13, 13, 16) 0 \n", "_________________________________________________________________\n", "conv2d_9 (Conv2D) (None, 11, 11, 32) 4640 \n", "_________________________________________________________________\n", "max_pooling2d_9 (MaxPooling2 (None, 5, 5, 32) 0 \n", "_________________________________________________________________\n", "flatten_5 (Flatten) (None, 800) 0 \n", "_________________________________________________________________\n", "dense_15 (Dense) (None, 256) 205056 \n", "_________________________________________________________________\n", "dropout_8 (Dropout) (None, 256) 0 \n", "_________________________________________________________________\n", "dense_16 (Dense) (None, 256) 65792 \n", "_________________________________________________________________\n", "dropout_9 (Dropout) (None, 256) 0 \n", "_________________________________________________________________\n", "dense_17 (Dense) (None, 10) 2570 \n", "=================================================================\n", "Total params: 278,218\n", "Trainable params: 278,218\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "fdkGajqbdswn", "outputId": "7caafa7a-f982-4074-e170-f771fc774251" }, "source": [ "history = model.fit(train_images, y_train,\r\n", " batch_size=32,\r\n", " callbacks=[es_callback], \r\n", " validation_split=0.1, epochs=EPOCHS,\r\n", " verbose=1)" ], "execution_count": 54, "outputs": [ { "output_type": "stream", "text": [ "Epoch 1/100\n", "1688/1688 [==============================] - 5s 3ms/step - loss: 1.2095 - accuracy: 0.5795 - val_loss: 0.5206 - val_accuracy: 0.8035\n", "Epoch 2/100\n", "1688/1688 [==============================] - 4s 3ms/step - loss: 0.5511 - accuracy: 0.7921 - val_loss: 0.4393 - val_accuracy: 0.8412\n", "Epoch 3/100\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.4591 - accuracy: 0.8322 - val_loss: 0.3909 - val_accuracy: 0.8545\n", "Epoch 4/100\n", "1688/1688 [==============================] - 4s 3ms/step - loss: 0.4152 - accuracy: 0.8503 - val_loss: 0.3654 - val_accuracy: 0.8675\n", "Epoch 5/100\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.3809 - accuracy: 0.8600 - val_loss: 0.3464 - val_accuracy: 0.8717\n", "Epoch 6/100\n", "1688/1688 [==============================] - 4s 3ms/step - loss: 0.3653 - accuracy: 0.8685 - val_loss: 0.3389 - val_accuracy: 0.8750\n", "Epoch 7/100\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.3398 - accuracy: 0.8773 - val_loss: 0.3212 - val_accuracy: 0.8798\n", "Epoch 8/100\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.3268 - accuracy: 0.8810 - val_loss: 0.3067 - val_accuracy: 0.8875\n", "Epoch 9/100\n", "1688/1688 [==============================] - 4s 3ms/step - loss: 0.3118 - accuracy: 0.8861 - val_loss: 0.3009 - val_accuracy: 0.8875\n", "Epoch 10/100\n", "1688/1688 [==============================] - 4s 3ms/step - loss: 0.3036 - accuracy: 0.8888 - val_loss: 0.2973 - val_accuracy: 0.8898\n", "Epoch 11/100\n", "1688/1688 [==============================] - 4s 3ms/step - loss: 0.2963 - accuracy: 0.8920 - val_loss: 0.2881 - val_accuracy: 0.8902\n", "Epoch 12/100\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.2909 - accuracy: 0.8933 - val_loss: 0.2876 - val_accuracy: 0.8937\n", "Epoch 13/100\n", "1688/1688 [==============================] - 4s 3ms/step - loss: 0.2825 - accuracy: 0.8968 - val_loss: 0.2784 - val_accuracy: 0.8973\n", "Epoch 14/100\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.2709 - accuracy: 0.9003 - val_loss: 0.2767 - val_accuracy: 0.8977\n", "Epoch 15/100\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.2649 - accuracy: 0.9033 - val_loss: 0.2698 - val_accuracy: 0.8992\n", "Epoch 16/100\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.2545 - accuracy: 0.9067 - val_loss: 0.2656 - val_accuracy: 0.8993\n", "Epoch 17/100\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.2547 - accuracy: 0.9076 - val_loss: 0.2625 - val_accuracy: 0.9032\n", "Epoch 18/100\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.2520 - accuracy: 0.9085 - val_loss: 0.2554 - val_accuracy: 0.9048\n", "Epoch 19/100\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.2346 - accuracy: 0.9131 - val_loss: 0.2572 - val_accuracy: 0.9040\n", "Epoch 20/100\n", "1688/1688 [==============================] - 4s 3ms/step - loss: 0.2380 - accuracy: 0.9114 - val_loss: 0.2546 - val_accuracy: 0.9035\n", "Epoch 21/100\n", "1688/1688 [==============================] - 4s 3ms/step - loss: 0.2303 - accuracy: 0.9152 - val_loss: 0.2489 - val_accuracy: 0.9068\n", "Epoch 22/100\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.2259 - accuracy: 0.9150 - val_loss: 0.2557 - val_accuracy: 0.9057\n", "Epoch 23/100\n", "1688/1688 [==============================] - 4s 2ms/step - loss: 0.2188 - accuracy: 0.9186 - val_loss: 0.2533 - val_accuracy: 0.9057\n", "Restoring model weights from the end of the best epoch.\n", "Epoch 00023: early stopping\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 265 }, "id": "d7D1d4kCd0uP", "outputId": "b42bc368-577c-48cb-f5d3-13ce6c092557" }, "source": [ "fig, ax = plt.subplots(1, 2, figsize=(10, 4))\r\n", "\r\n", "history_df = pd.DataFrame(history.history)\r\n", "history_df[['loss', 'val_loss']].plot(kind='line', ax=ax[0])\r\n", "history_df[['accuracy', 'val_accuracy']].plot(kind='line', ax=ax[1]);" ], "execution_count": 55, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAD4CAYAAADIBWPsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXyU1dXA8d+dJZlsk30lgQQIIexIEBVBFkUEFYUqWDfUQl2wVq1LrVVfq29rbW1r60Yt7lZ5oSiyiKIoioiEnUDYt+wrCQmZLDP3/eMZMKaRDJBtJuf7+cxnZp6595kzAYaTe+9zrtJaI4QQQgghzoypowMQQgghhPBmkkwJIYQQQpwFSaaEEEIIIc6CJFNCCCGEEGdBkikhhBBCiLNg6ag3joqK0snJyR319kKIDrBhw4YSrXV0R8dxtuT7S4iu51TfXx2WTCUnJ5OZmdlRby+E6ABKqUMdHUNrkO8vIbqeU31/yTSfEEIIIcRZkGRKCCGEEOIsSDIlhBBCCHEWOmzNlBDepr6+npycHBwOR0eH0unZbDYSExOxWq0dHYoQQrQ5SaaE8FBOTg4hISEkJyejlOrocDotrTWlpaXk5OSQkpLS0eEIIUSbk2k+ITzkcDiIjIyURKoFSikiIyNlBE8I0WVIMiXEaZBEyjPycxJCdCWdfpqv+Fgtb397iEkD40mLC+nocIQQQgjhhbTWlFTVcai0mkOlxzlUWs20YYn0iAw663N3+mTKUe/kb5/toVtYgCRTossLDg6mqqqqo8MQQohOyeXSFFQ6TiZLB933J55X1zlPtjUpGJgY1jWSqRi7PwAFlbL+QgghhOjKauqc5B6tIa/RLefkYwf5FTXUO/XJ9lazIikikOTIIM5NiSA5MpAeUUEkRwbRLSwAP0vrrHbq9MmUv8VMRJCfJFNCNKK15sEHH2T58uUopXj00UeZPn06+fn5TJ8+ncrKShoaGnjppZe44IILuO2228jMzEQpxa233sq9997b0R9BCCF+lNaaQ6XH2Xi4nE2Hj7I15yhHymsoq677QTuTgji7jYSwAIYkhTFpYDyJ4QEkRwbRIzKQhLAAzKa2X8PZ6ZMpgFi7jSJJpkQn8j8fZbEjr7JVz9kvwc7jV/T3qO1//vMfNm/ezJYtWygpKWH48OGMHj2ad999l0svvZTf/OY3OJ1Ojh8/zubNm8nNzWX79u0AHD16tFXjFkKIs3W8roEtRyrcyZORQJW6E6cgPzODk8KYOCCObmEBdAsLICEsgIQwG3F2GxZzx19L5yXJlL+MTAnRyNdff811112H2WwmNjaWiy66iPXr1zN8+HBuvfVW6uvrueqqqxgyZAg9e/Zk//793H333UyePJkJEyZ0dPhCiC6mrsFF+fE6SqvqKKuuo7S6lrLqOvYXV7PxcDnZBcdwuozpuZ7RQYztG8M53cMZ2j2MPrEh7TK6dDa8IpmKs9vIauVRACHOhqcjSO1t9OjRrF69mqVLlzJz5kzuu+8+brrpJrZs2cKKFSt4+eWXmT9/PvPmzevoUIUQXkJrTXbBMQ6UVFPb4KSuwUVdg4vaBhd1The19cb998edlB+vp6zanThV1VLpaGj23EF+ZoZ0D+POMb04p3s4Q5LCCA/ya+dPePa8IpmKsdsoqaql3unC2gmG84ToaKNGjeKVV17h5ptvpqysjNWrV/Pss89y6NAhEhMTmTVrFrW1tWzcuJFJkybh5+fHtGnTSEtL44Ybbujo8IUQndyRsuOs2VvCmn2lrN1XQklV3SnbW80KP7MJP4sJf4uZsEArEUF+9E+wExnkR0SQPxHBfkQF+RER5EdksHEsLMCKqZOPOnnCK5KpOLsNraGkqpb40ICODkeIDnf11Vezdu1aBg8ejFKKP/7xj8TFxfHGG2/w7LPPYrVaCQ4O5s033yQ3N5dbbrkFl8sFwO9///sOjl4I0dmUVtXyzb5SvtlXwpq9pRwuOw5AdIg/F/aOYmTvKAZ0C8VmNeNnMTVKnIzHvpAQnQ2vSKZiT5RHqHBIMiW6tBM1ppRSPPvsszz77LM/eP3mm2/m5ptv/q9+GzdubJf4hBDewenSbDhUzsqdhXy1p4Sd+cZSmhB/C+f1iuTWkcmM7B1F75hg2dHAA16STNkAKKys7eBIhBBCCO/kqHfyzb4SVmwvZOXOQkqr6/AzmxjWI5wHLk3jgl6RDOwW2imujgPg6BHY9xns/QwOfwt+gRAcByGx7nv3LTjWfR8HgRHQAcmfVyRTcaEnkim5ok8IIYTwVKWjni92FbMiq4AvsouornMS7G9hbN8YLu0fy0V9ogmxWU99EpcLqgrAUQkB4RAQBhb/1g+23gGH1hjJ096VULLLOG7vBr3GgasBqgqhcAfsWwW1zVyYZrKCf4jnCdXUf0Lv8WcdukfJlFJqIvA3wAy8qrX+Q5PXuwNvAGHuNg9rrZeddXRuEYF+WM1KyiMIIYQQP8Lp0pRW1VJ0rJatORWsyCrgm30l1Ds1UcH+XDmkGxP6x3JBr0j8LeYfdnY5oSIHyvY3uh0w7ssPQkPND9tbg9yJlTu5OvnY/dwvGKyBxmiSNbCZx0HGfWWukTjtXQkH1xjvY/aHHhfAOTcZiU503+aTo7pqI7k6VmDcTjyuO40tt0LiTvvn3JwWkymllBl4AbgEyAHWK6UWa613NGr2KDBfa/2SUqofsAxIbpUIAZNJERNik5EpIYQQXVKD08Weoipyy2soPOagqLKWopP3tRRWOiipqsX1/U4qdI8IZOYFyVzaP46h3cN/WKupugR2LYPdK6B4l5Ewueq/f93sDxEpENHTSGjCk41EyXEUasqh5sS9+1ay232sDJynvvKvWZGpMOxm6H0x9BhpJF4t8Qsy4ovoefrv18o8GZk6F9irtd4PoJR6D5gCNE6mNGB3Pw4F8lozSDD26JNkSgjxYzwYQe8BzAOigTLgBq11jvu1mzF+KQR4Smv9RrsFLkQzSqtq2Xj46MmK4FtzKjjeaJNepSAyyI+YEBsxdn/6xduJsfsTY7cRE+JPSlQQqU0Xj1fmwc4lsHOxMZ2mXRDaHRKGQN/J3ycmET0hJB5MZ7B2SmuorzFGjeqr3Y+PG4/rjkO9+3bimC0Meo01kjUv5kky1Q040uh5DjCiSZsngE+UUncDQcDFzZ1IKTUbmA3QvXv30wo0zm5jT9FpDN0JIboMD0fQ/wS8qbV+Qyk1Dvg9cKNSKgJ4HMjA+MVwg7tveft+CtFVNThdZBccY9Ph8pMJ1KFSozSBxaTol2Dn2owkhnYPIzkyiBi7P1HB/p7VXSw7YCRPOz+CnPXGsag0GHU/pF8BcYNad8G2Usaokl8gxu8tXUNrLUC/Dnhda/1npdT5wFtKqQFaa1fjRlrrucBcgIyMDN3MeX5UrN3G13tKWilcIYSP8WQEvR9wn/vxKuAD9+NLgU+11mXuvp8CE4F/t0Pcooupa3Cxp+gYWXmVZOVWGPd5ldTUG6NOUcH+nNM9jOvO7c453cMZ2C2UAD8zOCqMNU31+VDhhPIG0E5jUbbL5b5vdKx0H+xYDIXbjDeOHwzjfmskUNFpHfgT8E2eJFO5QFKj54nuY43dhvHlg9Z6rVLKBkQBRa0RJBjJ1LHaBqprGwjy94qLEIXocMHBwSdrUzV18OBBLr/88pMbIHs5T0bQtwBTMaYCrwZClFKRP9K3W9M3OJuRddE1Ha9rYGf+MXbkVbA9t5Ks/Ap2F1RR5zTGGQL9zKTH27k2I5FzE/wYFnqMWFch6mgmHD0M6w4Z90cPGcnUaVGQNAImPG0kUOE9Wv8DipM8yUrWA6lKqRSMJGoG8NMmbQ4D44HXlVLpgA0obs1ATxTuLKx00DM6uDVPLYToGn4F/EMpNRNYjfF95jxlj0bOZmRddA0Vx+v5dm8huVtXEXhkFabjpVipJ4J6JpmdXO8PoREuQixOgsxO/GhA1dVCdgVsajKrbA2EsO7GLWmE+3GScZWcMoHJ4r6ZjfuTx9zPAyMhKKpjfhBdUIvJlNa6QSk1B1iBsbBzntY6Syn1JJCptV4M3A/8Uyl1L8aag5la61b9solzF+4skGRKdAbLH4aCba17zriBcNkfTtnk4YcfJikpibvuuguAJ554AovFwqpVqygvL6e+vp6nnnqKKVOmnNZbOxwO7rjjDjIzM7FYLDz33HOMHTuWrKwsbrnlFurq6nC5XCxcuJCEhASuvfZacnJycDqd/Pa3v2X69Oln/LFbSYsj6FrrPIyRKZRSwcA0rfVRpVQuMKZJ3y/aMljhG6prG1h/sIzM3Ueo37WSPhVfMc60iXBVRQMWagIiMFv9sfrbsPjZUBYbmAPA4mdcLWdx3/yCv0+cwnoYo0iBkR1SfFKcGY/my9w1o5Y1OfZYo8c7gJGtG9oPxboLdxZJFXTRhU2fPp1f/vKXJ5Op+fPns2LFCn7xi19gt9spKSnhvPPO48orrzytLSBeeOEFlFJs27aN7OxsJkyYwO7du3n55Ze55557uP7666mrq8PpdLJs2TISEhJYunQpABUVpzv90CZaHEFXSkUBZe61nL/GuLIPjF8U/1cpFe5+PsH9uhA/4Kh3sunwUdbuK2HH7j3EFXzOOLWBu01Z+Kt6HP52qrpfTP3Qq7D2uZgQ/5CODlm0E69ZfBTbaGRKiA7XwghSWxk6dChFRUXk5eVRXFxMeHg4cXFx3HvvvaxevRqTyURubi6FhYXExXlejO7rr7/m7rvvBqBv37706NGD3bt3c/755/P000+Tk5PD1KlTSU1NZeDAgdx///089NBDXH755YwaNaqtPq7HPBxBHwP8XimlMab57nL3LVNK/Q4jIQN48sRidNG1nUievt1fyrf7Sjies5VRegMTzBu4z7QPLOAITsKcfhv0uxxb9/Oxmb3mv1XRirzmTz3Y30Kwv0VqTYku75prrmHBggUUFBQwffp03nnnHYqLi9mwYQNWq5Xk5GQcjtb5d/LTn/6UESNGsHTpUiZNmsQrr7zCuHHj2LhxI8uWLePRRx9l/PjxPPbYYy2frI15MIK+AFjwI33n8f1IleiiHPVONh4qN5KnA2VkHy4kQ29jvHkT/7BuJdpiLAV2xg+F9EchbTK2mHSZjhPek0yBFO4UAoypvlmzZlFSUsKXX37J/PnziYmJwWq1smrVKg4dOnTa5xw1ahTvvPMO48aNY/fu3Rw+fJi0tDT2799Pz549+cUvfsHhw4fZunUrffv2JSIightuuIGwsDBeffXVNviUQrS98uo6Nh0pZ+Oho6w7UMqWIxVEOwsZb97MA4HbGOK3DauuRVuDUL3GQuoESJ2A2R7f0aGLTsarkqk4u41CWTMlurj+/ftz7NgxunXrRnx8PNdffz1XXHEFAwcOJCMjg759+572Oe+8807uuOMOBg4ciMVi4fXXX8ff35/58+fz1ltvYbVaiYuL45FHHmH9+vU88MADmEwmrFYrL730Uht8SiFagdbQUAvOWlz1tRwoLCfrcDG7ckvYm19KcXkl/qqeIFXHDPsBXrRvJLpmv9E3OBn63AKpE1DJF7bNxr7CZ6hWvujOYxkZGTozM/O0+tz7/ma+O1DGmofHtVFUQvy4nTt3kp6e3tFheI3mfl5KqQ1a64wOCqnVnMn3l2hjx8sgf8vJmyt/C6psPwoP/48zWaD7+dDnUki9FKJSZfpO/MCpvr+8amQq1m6j6JgDrfVpXakkhBDCh1QVuZOmzd8nUEcPn3y5zBrHhroksp0DqcUPe3AwcZGhdIsMo3tMOFFhIagTZQlOlCiITgNbaAd+KOHNvCqZirP7U+/UlFXXERksQ65CeGLbtm3ceOONPzjm7+/PunXrOigiIRqpq4ZV/wub3zG2RWmJdkHdse+fRvSiNHQQ622T+aAwinU1ibgI57KB8Vw2MI5hPcIJsVnb8AMI4WXJVOPyCJJMiY7gjaOiAwcOZPPmze36nh21fEB4mX2r4KN7jO1S+l8NwZ6V89BhSew192ZhXgSLdlRQmFdLgNXMJf1ieXZwAqP7ROFvMbdx8EJ8z7uSqUaFO/sndHAwosux2WyUlpYSGRnpdQlVe9JaU1pais1m6+hQRGd1vAw+edQYjYroBTOXQvKFp+yitSYrr5Ll2/NZ8nU+h0qP42cu4aK0aK4cnMD49BgC/bzqvzThQ7zqb54U7hQdKTExkZycHIqLW3XbSZ9ks9lITEzs6DBEZ6M17PgAlj0Ix0vhwnvhoofAGvAjzTVbcipYvi2f5dsLOFx2HJOCC3pFcdeY3lw6II7QAJnCEx3Pq5KpmJDvNzsWor1ZrVZSUlI6OgwhvFNlHiz9FexaCvGD4YYFxn0TLpdm4+Fylm0r4OPt+eRVOLCYFCN7R3HnmF5c0i9WlnmITserkimr2URUsJ8kU0II4S1cLtj4Onz6ODjr4JIn4by7oNG2K06X5rsDZSzfns/H2wsoOlaLn8XE6NQo7p+QxsXpsYQGygiU6Ly8KpkCY6pPCncKIYQXKNljLDA/tAaSR8EVf4PIXoAxArXhcDlLtuSxdFsBJVW12KwmxqbFMHFAHOP6xshVeMJreF0yFWe3kV8hI1NCCNEpORtgzyeQOQ/2rgR/O1z5dxh6IxrYeuQoH23JY+m2fPIrHPhbTIxPj2HywATG9o2WReTCK3nd39oYu43NR452dBhCCCEaq8yDjW/BxjegMtcoc3DRg+iMW9l5LJAlK3axZGs+h8uOYzUrLuoTzcOX9WV8eizB/l73X5EQP+B1f4Pj7DZKq+uoa3DhZzF1dDhCCNF1uVywf5UxCrVrOWgn9BoHlz1DVY+LefO7XBbOzWZfcTVmk+KCXpHMGdebS/vFyRoo4VO8LpmKtRtXcRQdc5AYHtjB0QghRBdUXQKb3oYNr0H5QQiMhAvmwLCZOEJ68Pa3h3hxwdeUVddxbkoEt4xM4bIBcXIVnvBZ3pdMuQt3FlbWSjIlhBDtQWso2gn7PjPWQR1cA6566HEhjPstpF9BvbIyP/MIf//sCwoqHVzYO4pfXZrGkKSwjo5eiDbnfclUyIlkShahCyFEm6kph/1fGMnT3s/hWJ5xPLovnHc7DL0RotNwujQfbcnjLyt3c6j0OOd0D+O56YO5oFdUh4YvRHvyumQqLlSSKSGEaHUuJ+Rtgr3u0afcTGNTYf9Q6DUGeo2H3uMh1Khsr7Xmk6wCnvtkN7sKj5Eeb2fezAzGpsXIdkuiy/G6ZCo80Iqf2SRbygghxNnSGnLWw7b/g6xFUF0MKEgYCqN+ZSRP3TJ+UGBTa82avaU8+8kuthw5Ss+oIP5+3VAmD4zHZJIkSnRNXpdMKaWIsftTJIU7hRDizBTuMBKo7Qvg6GEw+0OfSyH9SuNqvKDIZrvlV9Tw2IdZfLqjkIRQG3+cNoip53TDYpYrq0XX5nXJFBjlEQqkcKcQQniu/CBsXwjbFkDRDlBm6DkGxvwa+k4GW+iPdnW6NG+tPcizK3bh1JqHL+vLLSOT8beY2yl4ITo3j5IppdRE4G+AGXhVa/2HJq//BRjrfhoIxGit2+wSjli7jZ35lW11eiGE8A0NtbDhDWMUKuc741jSCJj0J+h3FQRHt3iKnfmVPPyfbWw5cpTRfaJ5+qoBJEXIldRCNNZiMqWUMgMvAJcAOcB6pdRirfWOE2201vc2an83MLQNYj0p1m7ji11FbfkWQgjh3ZwN8H+3wK6lEDsAxj8OA6ZBeA+Pujvqnfztsz38c/V+QgOs/G3GEK4cnCCLy4VohicjU+cCe7XW+wGUUu8BU4AdP9L+OuDx1gmvebF2f6rrnBxz1MtGmEII0ZTWsOQeI5Ga+IxRyuA0fL2nhN98sI1Dpce5Zlgij0xKJzzIr42CFcL7eZJMdQOONHqeA4xorqFSqgeQAnz+I6/PBmYDdO/e/bQCbSyuUeFOSaaEEKKJlU8YFcpHP3haiVRZdR1PLd3BfzbmkhwZyLs/G8EFvaVelBAtae0F6DOABVprZ3Mvaq3nAnMBMjIy9Jm+Saz9+1pTvWOCz/Q0Qgjhe775O6z5K2TcCmMf8bjbR1vyeHxxFpU19cwZ25s543pjs8oCcyE84UkylQskNXqe6D7WnBnAXWcbVEsaJ1NCCCHcNr8LnzxqLC6f9CfwYH2T1prnPt3N3z/fy5CkMJ6ZNoi0uJB2CFYI3+FJMrUeSFVKpWAkUTOAnzZtpJTqC4QDa1s1wmac2OxYCncKIYTbruXw4Ryj3MHUuWBqeVSptsHJgwu28uHmPKZnJPHU1QOwSs0oIU5bi8mU1rpBKTUHWIFRGmGe1jpLKfUkkKm1XuxuOgN4T2t9xtN3ngr0sxBis1AotaaEEAIOrYX/mwnxg2H622Dxb7FLeXUdP39rA98dLOOBS9O4c0wvuVJPiDPk0ZoprfUyYFmTY481ef5E64XVsji7jUKpgi6E6OoKtsO70yE0Ca7/P/BveYruYEk1t76+npzyGp6/bihXDk5oh0CF8F1eWQEdjHVTMs0nhOjSyg7A21PBLwhuXARBLV95t+FQGbPe3IBLa96ZNYLhyRHtEKgQvs1rJ8dj7TaKJJkSQnRVVUXw1tXgrDMSqbCkFrss2ZrHdf9ch91mYdGdIyWREqKVeO3IVFyoP0XHanG5tOxULoToWhwVxohUVSHctBhi+p6yudaal7/czzMfZ5PRI5y5N2UQIUU4hWg1XptMxdptNLg0pdV1RIe0vNhSCCF8QkMtvHc9FO2E696HpOGnbF7vdPHYh9v593dHuGJwAs/+ZJDUjxKilXl1MgVGrSlJpoQQXYLLBYt+Dge/gqmvQurFp2xeXdvA7W9v4Ks9Jdw1thf3X5ImI/lCtAGvXjMFUrhTCGFQSk1USu1SSu1VSj3czOvdlVKrlFKblFJblVKT3MeTlVI1SqnN7tvL7R+9B7SGFY9A1iKY8BQMuqbFLk9+tIM1e0t4ZtpAHri0ryRSQrQRrx2ZinMnU3JFnxBCKWUGXgAuwdg/dL1SarHWuvGG7I8C87XWLyml+mGUe0l2v7ZPaz2kPWM+bd88D+tegvPuhPPntNj88+xC3s88wh1jejF9+JnvhSqEaJnXjkxFBfthUkjhTiEEwLnAXq31fq11HfAeMKVJGw3Y3Y9Dgbx2jO/sbHkfPn0M+k+FCU+3uE1MeXUdDy3cRlpsCL+8OLWdghSi6/LaZMpiNhEV7C+FO4UQAN2AI42e57iPNfYEcINSKgdjVOruRq+luKf/vlRKjWruDZRSs5VSmUqpzOLi4lYMvQX7PocP74TkUXD1y2Bq+Wv78cVZlFfX8edrB+NvkcXmQrQ1r02mAOJCpXCnEMJj1wGva60TgUnAW0opE5APdNdaDwXuA95VStmbdtZaz9VaZ2itM6Kjo9sn4vwt8P6NEJUGM97xaJuYZdvyWbwlj1+MT2VAt9B2CFII4dXJVEyITRagCyHA2IS9cdXKRPexxm4D5gNordcCNiBKa12rtS51H98A7AP6tHnELSk/CG//BALC4YYFYGs5MSo+VsujH2xnUGIod4zp1fYxCiEAL0+m4kL9JZkSQgCsB1KVUilKKT+MjdcXN2lzGBgPoJRKx0imipVS0e4F7CilegKpwP52i7w51aXw1lSjuvkNC8He8t55Wmt+s2gbVbUN/PmawVjNXv31LoRX8dqr+QBiQ2yUH6+ntsEp6wKE6MK01g1KqTnACsAMzNNaZymlngQytdaLgfuBfyql7sVYjD5Ta62VUqOBJ5VS9YALuF1rXdZBHwXqquHda6EyF276EKLTPOq2aFMun+wo5DeT0kmNbXmzYyFE6/HuZCrUKI9QVFlLUkRgB0cjhOhIWutlGAvLGx97rNHjHcDIZvotBBa2eYCecDbAglshbyNc+xZ0P8+jbvkVNTy+OIvhyeHcemFKGwcphGjKq8eBY6XWlBDCV2gNS++F3R/DpD9B+uUedtM8uGArDU7Nn64ZjFkKcwrR7rw6mYqTKuhCCF+xazlsfBNG/QqG3+Zxt3e/O8xXe0p4ZHI6PSKD2jBAIcSP8YlkqkAKdwohvN2ODyAgAsb82uMuh0uP8/TSnYxKjeKGEVLlXIiO4tXJlD3Agr/FRNExKdwphPBiznpjei/tMjB7tpTV5dL86v+2YFaKZ6YNQrVQFV0I0Xa8OplSShmFO2VkSgjhzQ5+BY4K6OvZOimAeWsO8N3BMh6/sj8JYQFtGJwQoiVenUyBUR5B1kwJIbzaziVgDYReYz1qvrfoGH9csYuL02OZdk7TXXOEEO3N+5OpUEmmhBBezOWC7KXQezxYPRthevSD7QT6mfnfqQNkek+ITsD7k6kQY7NjrXVHhyKEEKcvdwNUFUDfKzxqvr+4im/3l3H7Rb2ICbG1cXBCCE94fTIVF2qjpt5JpaOho0MRQojTl/0RmCzQZ4JHzf+zMReTgqlDZXpPiM7Co2RKKTVRKbVLKbVXKfXwj7S5Vim1QymVpZR6t3XD/HGxUmtKCOGttDbWSyWPMjY0boHLpVm0KZdRqdHE2GVUSojOosVkyr0B6AvAZUA/4DqlVL8mbVKBXwMjtdb9gV+2QazNkmRKCOG1indB2T6Pq51/u7+U3KM1TBuW2MaBCSFOhycjU+cCe7XW+7XWdcB7wJQmbWYBL2itywG01kWtG+aPk8KdQgivlf2RcZ822aPmCzbmEGKzMKFfbBsGJYQ4XZ4kU92AI42e57iPNdYH6KOUWqOU+lYpNbG5EymlZiulMpVSmcXFxWcWcRMxdn8AKdwphPA+O5dA4nCwx7fYtLq2gY+3F3D5oHhsVnM7BCeE8FRrLUC3AKnAGOA64J9KqbCmjbTWc7XWGVrrjOjo6FZ5Y5vVTFigVUamhBDe5egRyN/scaHO5dsLOF7nZOo5MsUnRGfjSTKVCyQ1ep7oPtZYDrBYa12vtT4A7MZIrtqFFO4UQnid7KXGfbpnJREWbsihR2QgGT1aXqguhGhfniRT64FUpVSKUsoPmAEsbtLmA4xRKZRSURjTfvtbMc5TksKdQgivk70EotMhsleLTXPKj7N2fylThyZKkU4hOqEWkymtdQMwB1gB7ATma62zlFJPKqWudDdbAZQqpXYAq4AHtNalbRV0U7Eh/rqHFUUAACAASURBVBRIMiWE8BbVpXBojcdX8X2wyZgMmCpbxwjRKXm0PbnWehmwrMmxxxo91sB97lu7iwu1UXysFqdLYzbJb21CiE5u93LQLo/WS2mtWbgxlxEpESRFBLZDcEKI0+X1FdDBqDXl0lBSJVf0CSG8wM4lENod4ge32HTj4aMcKKmW2lJCdGI+k0yBFO4UQniB2irY9zn0nQwerH9auDGHAKuZSQNbLp8ghOgYPpFMSeFOIYTX2LsSnLUerZdy1DtZsiWPiQPiCPb3aFWGEKID+EQyFesu3FkohTuFEJ1d9hIIjITu57fYdOXOQiodDUyT2lJCdGo+kUxFBvtjNikKZWRKCNGZNdTB7k8g7TIwtVzFfOGGHOJDbZzfK7IdghNCnCmfSKbMJkV0sL+smRJCdG4HV0NtBfRtuVBn0TEHq/eUcNXQbnKVshCdnE8kU2AU7pRaU0KITm3nEvALhp5jWmy6eHMeTpeWKT4hvIDPJFNxdhmZEkJ0Yi4X7FoGvS8Gq63F5gs25DA4KYzeMcHtEJwQ4mz4TDIVa7dRWCkL0IUQnVTOeqgq9Ggvvqy8CrILjvETqXguhFfwqWSqoqYeR72zo0MRQoj/lv0RmKyQekmLTRduyMXPbOKKwQntEJgQ4mz5VDIFUrhTCNEJaW2sl+p5EdhCT9m03uniw825jE+PISzQr50CFEKcDZ9JpqRwpxCi0yraAeUHPNqL78tdxZRW18nCcyG8iM8kU1K4UwjRae1cAihjC5kWLNyYQ2SQHxelRbd9XEKIVuE7yVSoe5pPRqaEEJ1N9keQNAKCY07Z7OjxOj7bWcSUId2wmn3m61kIn+cz/1pD/C0E+pml1pQQonMpPwgF2zzai++jrfnUOV1Mlav4hPAqPpNMKaXc5REkmRJCdCLZS417D9ZLLdyQQ9+4EPon2Ns4KCFEa/KZZAqMdVOSTAkhOpWdSyB2AESknLLZvuIqNh85yrRzElFKto8Rwpv4VDKVGhPC9txKyqvrOjoUIUQ7U0pNVErtUkrtVUo93Mzr3ZVSq5RSm5RSW5VSkxq99mt3v11KqUtbNbARs+GiB1tstjXnKABj+8rCcyG8TedPpirzYflDULa/xaY3nt+Dmnon76w71A6BCSE6C6WUGXgBuAzoB1ynlOrXpNmjwHyt9VBgBvCiu28/9/P+wETgRff5Wkf/q6HflBabFVQYVyLHhwa02lsLIdpH50+m0PDdP2HDGy227BMbwkV9onn9m0PUNkgldCG6kHOBvVrr/VrrOuA9oGkGo4ETi5FCgTz34ynAe1rrWq31AWCv+3ztqqCiBrvNQpC/pb3fWghxljp/MmVPgLTLYNPb0NDy9N3s0T0pqarlw015LbYVQviMbsCRRs9z3McaewK4QSmVAywD7j6NviilZiulMpVSmcXFxa0V90n5FQ7iQlveAFkI0fl0/mQKIONWOF5i1GppwQW9IukXb2fuV/txuXQ7BCeE8BLXAa9rrROBScBbSimPvwO11nO11hla64zo6NZf11RY6SBOpviE8ErekUz1HAvhyZD5WotNlVLMGp3C3qIqvtzd+r89CiE6pVwgqdHzRPexxm4D5gNordcCNiDKw75tLr/CQbxdRqaE8EYeJVMeXCUzUylVrJTa7L79rHWjNMGwmXDwKyje3WLzywclEGe38c+vWl60LoTwCeuBVKVUilLKD2NB+eImbQ4D4wGUUukYyVSxu90MpZS/UioFSAW+a7fIMTY3Lq6qPbmTgxDCu7SYTHl4lQzA+1rrIe7bq60cJwy5AUxW2NDy6JTVbOLWC5P5Zl8p23MrWj0UIUTnorVuAOYAK4CdGFftZSmlnlRKXeludj8wSym1Bfg3MFMbsjBGrHYAHwN3aa3b9QqW4mO1aA3xkkwJ4ZU8GZny5CqZthccDf2uhM3vQn1Ni81nnNudYH+LjE4J0UVorZdprftorXtprZ92H3tMa73Y/XiH1nqk1nqw+5e+Txr1fdrdL01rvby9Y8937ykqC9CF8E6eJFMeXekCTHMXwluglEpq5vWzvxpm2C3gOApZH7TY1G6zMmN4Eku25pN3tOXkSwghOkrBiWRK1kwJ4ZVaawH6R0Cy1noQ8CnQbFGos74aJvlCiEyFzHkeNb/lQmP7htfWHDj99xJCiHaSX2H8wifTfEJ4J0+SqRavdNFal2qta91PXwWGtU54TShllEnI+c7Yhb0F3cICmDwwnn9/d4RKR32bhCSEEGersNKBzWoiNMDa0aEIIc6AJ8lUi1fJKKXiGz29EmMBaNsYPAMsNo/KJADMGtWTqtoG3vvucJuFJIQQZyO/wkF8aIBscCyEl2oxmfLwKplfKKWy3FfJ/AKY2VYBExhh7HW1dT7UVrXYfGBiKOf3jOS1NQepd7raLCwhhDhTBRUOYu3+HR2GEOIMebRmyoOrZH6tte7vvkpmrNY6uy2DJuNWqDsG2xd41HzW6BTyKxws3ZrfpmEJIcSZKKh0yAbHQngx76iA3lTicIgdAOv/BbrlLWPG9Imhd0wwc1fvR3vQXggh2ovLpd1bycjicyG8lXcmU0pBxi1QsBXyNrbY3GRSzBqVwo78Sr7ZV9oOAQohhGdKq+uod2opiyCEF/POZApg4LVgDfJ4IfqUId2ICvaXIp5CiE6lsFIKdgrh7bw3mbLZYeBPYPtCqDnacnOrmZvP78EXu4rZVXCsHQIUQoiWnah+LjWmhPBe3ptMgTHVV3/cuLLPAzec1wOb1cSrMjolhOgkCtwFO2WaTwjv5d3JVMJQSDjHqIjuwcLy8CA/rs1I4sPNeRS5h9aFEKIj5Vc4sJgUkcFSGkEIb+XdyRQYo1PFO+HIOo+a3zoyhXqXizfWHmzTsIQQwhMFlQ5i7TbMJinYKYS38v5kasA08Ld7vF9fclQQl/aL4+1vD1Nd29DGwQkhxKkVVEhZBCG8nfcnU35BxhYzWR9AtWdlD2Zf1JOKmnqe/3xPGwcnhBCnVlDhkPVSQng570+mAIbdAs5a2PKuR83P6R7OjOFJzF29n/UHy9o4OCGEaJ7WmgIp2CmE1/ONZCq2HySdZ9Sc8rDC+aOX9yMxPID75m+mSqb7hBAdoNLRwPE6p5RFEMLL+UYyBcZ+fWX74MBqj5oH+1t47toh5JTX8PTSHW0cnBBC/LcCd42pWJnmE8Kr+U4y1W8KBITDd3M97jI8OYKfj+7Fv787wmc7C9swOCGE+G8FlVKwUwhf4DvJlNUG586G7CWwe4XH3e69JJW+cSE8tHAbpVW1bRigEEL80MmCnZJMCeHVfCeZAhh1P8T0h8V3w3HPFpb7W8z8ZfoQKmvq+c2i7WgP11wJIcTZOrGVTEyIJFNCeDPfSqYs/nD1y3C8FJY94HG39Hg7903ow8dZBSzalNuGAQohxPcKKhxEBfvjZ/Gtr2Ihuhrf+xccPwguehi2L4CsRR53mzWqJ8OTw3n8wyxyj9a0YYBCCGEoqHTIeikhfIDvJVMAF95r7Nm35D445tnCcrNJ8edrhuDSml/N34LLJdN9Qoi2VVDhkCv5hPABvplMmS3GdF9dNSz5pce1p7pHBvLby/uxdn8pr39zsG1jFEJ0efkVMjIlhC/wzWQKIDoNLn4cdi2DLf/2uNv04UlcnB7DMx9ns7foWBsGKIToymrqnFTU1MuVfEL4AN9NpgBG3AE9RsLyh+DoEY+6KKX4/dRBBPlbuPf9LdQ7XW0cpBCiK5IaU0L4Dt9OpkwmmPICuJyweA64PEuMokP8+d+rB7Att4K/fyabIQshWl/+iRpTsmZKCK/nUTKllJqolNqllNqrlHr4FO2mKaW0Uiqj9UI8SxEpcOnTsP8LyPyXx90mDohn6jndeOGLfWw6XN528QkhuqRC98iUTPMJ4f1aTKaUUmbgBeAyoB9wnVKqXzPtQoB7gHWtHeRZGzYTeo2HTx+D0n0ed3viyv7E2W3c/vYGsvIq2i4+IUSXc6JgpyRTQng/T0amzgX2aq33a63rgPeAKc20+x3wDOBoxfhah1Iw5R9gtsIHdxrTfh6w26z8a2YGJqW49uW1fLGrqI0DFUJ0FQUVDuw2C4F+lo4ORQhxljxJproBjVdv57iPnaSUOgdI0lovbcXYWpc9AS57Fo58C2v/4XG3vnF2Ft05kh6RQdz2RibvrjvchkEKIboKoyxCQEeHIYRoBWe9AF0pZQKeA+73oO1spVSmUiqzuLj4bN/69A26FtKvgM+fgqKdHneLC7Ux//bzGZUaxSOLtvGH5dlS1FMIcVYKKx0yxSeEj/AkmcoFkho9T3QfOyEEGAB8oZQ6CJwHLG5uEbrWeq7WOkNrnREdHX3mUZ8ppWDyX8DfDot+Dg11HncN9rfw6k0ZXD+iOy9/uY+739uEo96z6UIhhGgqv8IhV/IJ4SM8SabWA6lKqRSllB8wA1h84kWtdYXWOkprnay1Tga+Ba7UWme2ScRnKzgarvgr5G+Bt67yeLsZAIvZxFNXDeDXl/Vl6dZ8bnh1HWXVnidkQggBUO90UVJVKyNTQviIFpMprXUDMAdYAewE5muts5RSTyqlrmzrANtE+hVw9VzI3QivjIJD33jcVSnFzy/qxQs/PYetuRVMfXENB0qq2zBYIYQnWirhopT6i1Jqs/u2Wyl1tNFrzkavLW7at7UVHatFaynYKYSv8GjNlNZ6mda6j9a6l9b6afexx7TW//Wlo7Ue02lHpRobPB1mfQ7+IfD65fDN3z3eww9g8qB4/j1rBJWOBqa+uIbMg2VtGKwQ4lQ8KeGitb5Xaz1Eaz0E+Dvwn0Yv15x4TWvd5r8kFpwo2CnJlBA+wbcroLckth/MWgV9J8Mnj8L8G8FR6XH3YT0i+M8dFxAW6MdPX13Hkq15bRisEOIUPC3hcsJ1gOebdrYyqTElhG/p2skUgM0O174JE56G7GUwdwwUZnncPTkqiP/ccQGDE0OZ8+4mHlqwldKq2raLVwjRnBZLuJyglOoBpACfNzpsc19p/K1S6qof6ddqVyMXuJOpeLuURhDCF0gyBcZVfhfMgZlLoK4a/jketrzvcffwID/eum0Es0f3ZOHGHMb+6QveXHuQBtkkWYjOaAawQGvd+HLcHlrrDOCnwF+VUr2admrNq5ELKhwEWM3YA6RgpxC+QJKpxnpcAD9fDd2GwaLZsOQ+aPBslMlmNfPIpHQ+/uUoBiaG8tiHWVzxjzWsl7VUQrSHlkq4NDaDJlN8Wutc9/1+4AtgaOuH+L18d40ppVRbvo0Qop1IMtVUSCzc9CGMvMfYGHneRDjqedXz3jEhvH3bCF66/hwqjtdxzctruff9zRRVdr5ddoTwIacs4XKCUqovEA6sbXQsXCnl734cBYwEdrRlsAVSY0oInyLJVHPMFrjkSZj+DpTuhRfPh6+eg3rPEiKlFJcNjGfl/Rdx97jeLN2az9g/fcE/V++nXqb+hGh1p1HCZQbwntY/uHQ3HchUSm0BVgF/0Fq3eTIlZRGE8B0yYX8q6ZdD3ABY8Rv47H9g4xvGQvW+k411Vi0I9LNw/4Q0pp2TyJNLdvD0sp28n3mEJ67oz4WpUe3wAYToOrTWy4BlTY491uT5E830+wYY2KbBNeJyaQorHcRKMiWEz5CRqZaEJ8OMd+DGD8ASAO9fD29OgULPf3FNjgpi3szh/OvmDOoaXNzwr3XMfjOTvUVVbRe3EKJTKqmupcGlZWRKCB8iyZSneo2F27+GSX8ytqJ5eSQsvR+Oe77AfHx6LJ/cO5pfTejDmr0lTPjLlzy0YCv57gJ+QgjfV1hhXNQia6aE8B2STJ0OswXOnQW/2ATDfwaZr8HzQ2HdK+Cs9+gUNquZOeNSWf3gWG6+IJlFm3IZ8+wX/H7ZTo4el33+hPB1+VL9XAifI8nUmQiMgEnPGiNV8YNh+YPw8oWwdyW4PFtgHhnsz+NX9Oez+y9i8qB45n61n9F/XMWLX+ylps7Z8gmEEF6poFKqnwvhaySZOhux/YwyCjPeNepRvT0N/pgMb/8EvnwW9n8JtadeF5UUEchz1w5h+T2jGJ4cwR8/3sWYP63i3XWH5co/IXxQQYUDi0kRFeTf0aEIIVqJXM13tpQyru7rfTFkLYJD38CR72Dvp+7XzcYVgUkj3LdzITTpv64G7Btn518zh7P+YBl/WJ7NI4u28epX+7lvQh8uGxCP2STF/YTwBQUVDmLtNkzyb1oInyHJVGux+MPgGcYNoKYccjLhyDrjtukd+G6u8VpIPKRNgtEPgD3+B6cZnhzBgtvP57OdRfxxRTZz3t1EnH0n12Qkcs2wJLpHBrbzBxNCtKb8CodM8QnhYySZaisB4ZB6iXEDcDZAUZYxanVoDWx8Eza/C+fdYVRbDwg72VUpxcX9YhnbN4ZPdxTw/vojvLBqL3//fC8X9Ipk+vAkLu0fh81q7qAPJ4Q4UwWVDvol2Ds6DCFEK5Jkqr2YLcZi9fjBxhWBZQdg1dPw9XOQOQ9G3Q/nzgbr97+xmk2KiQPimTggnvyKGhZuyOH9zCPc895m7DYLVw3txrUZSQzoFtqBH0wI4SmtNQUVDsb3jenoUIQQrUgWoHeUiBSY9ur3Gyt/+lv4+zDY9Da4/vtqvvjQAOaMS+XLX43l3Z+NYGzfGN5bf4TL//41k5//ijfXHqSixrPyDEKIjlFZ00BNvVOm+YTwMZJMdbT4wXDjf+DmjyA4Bj68C14aCdnL4AfbhxlMJsUFvaP424yhrH/kYp6c0h+t4bEPszj36ZXcP38LGw6VoZvpK4ToWPmVUmNKCF8k03ydRcpomPU57PgQPv8dvHcdJJ0HFz8BPc5vtktooJWbzk/mpvOT2Z5bwb+/O8yHm/NYuDGHPrHBXHdud6YOTSQ00NquH0UI0byCCqPGlGwlI4RvkZGpzkQp6H8V3PktXP5XKD8Ir02EeRNh50fNTv+dMKBbKE9fPZB1j4znmWkDCbCa+Z+PdnDu/67kvvmbyTwoo1VCdLQTyVSsbCUjhE+RkanOyGyFjFtg0HTY8Bqsexnev8HYdHnE7TD0BvAPabZrkL+F6cO7M314d7bnVvDe+sN8sCmP/2zMldEqITpYfoUDpSAmRJIpIXyJjEx1Zn6BcP5dcPcmuPZNCI6Djx+G5/rBit/A0cOn7D6gWyhPXdVotMrPwv98tIPhT6/kpnnf8cY3BzlSdrydPowQorDSQVSwP34W+eoVwpfIyJQ3MFug3xTjlrMBvn0Bvn0Jvn0R0q80Eq6kc3+0e+PRqqy8ChZtzOXz7CIeX5zF44uzSI0JZlzfGMb2jWFYj3CsZvmiF6It5Fc4iJMpPtFEfX09OTk5OByOjg5FADabjcTERKxWz2dwPEqmlFITgb8BZuBVrfUfmrx+O3AX4ASqgNla6x0eRyE8lzgMfjIPLnnSqKie+Trs+AC6ZcCgayG2P8T0MzZjbkb/hFD6J4Ty6OX9OFBSzefZRXyeXci8NQd4ZfV+7DYLo/tEMz49hov6xBAR5Ne+n08IH1ZQ4ZBdDMR/ycnJISQkhOTkZJSSbYY6ktaa0tJScnJySElJ8bhfi8mUUsoMvABcAuQA65VSi5skS+9qrV92t78SeA6YeDofQJym0EQjoRr9oFFJfd1LsPzB718PSTA2Yo7p932CFZ1mbHvjlhIVxG0XpnDbhSkcc9Tz9Z4SPs8uYtWuIpZszcekYFBiGKNTo7gwNZqh3cNk1EqIs5BfUcOIns3/oiO6LofDIYlUJ6GUIjIykuLi4tPq58nI1LnAXq31fvcbvQdMAU4mU1rrykbtgwC5bKy9+AfDiNlGVfVj+VC4w9i25sT9gdXgrDPaKjNE9oaYdCMZs3cDewLYEwixJ3BZvzguGxiPy6XZllvBZ9lFfLWnmH+s2svzn+8l2N/CeT0jGJUazYWpUfSMCpJ//EJ46HhdA5WOBqkxJZol36Wdx5n8WXiSTHUDjjR6ngOMaObN7wLuA/yAcT8S4GxgNkD37t1PN1ZxKkqdTIxIvfj74856KN3XKMHaAQVbYffH0NBkfl6ZICgGkz2Bwe7bfSMGUDn1Ir4p8Wf1nhK+3lPCyp1FACSE2k4mViN7R8mUoBCncKIsgqyZEsL3tNoCdK31C8ALSqmfAo8CNzfTZi4wFyAjI0NGr9qD2QoxfY3bgGnfH9caasqhMs+4HXPfV+ZCZb6RgB1YDbWV2IGJ0X2Z2Gs8TB3P4ZARrD5Qzdd7Sli+PZ/3M4+gFAxODGNsWgxj+0YzICEUk0l+0xLihJPJlIxMCeFzPEmmcoGkRs8T3cd+zHvAS2cTlGgHShmL1AMjIG5A8220huJs2PsZ7F0J61+Fb1+gu8XGDckXckPv8TRMGMdWRwyr95Twxa5i/vrZbv6ycjdRwX6M7hPN2LQYRqdGS10r0eUVVJ6ofh7QwZEI0XEaGhqwWHyvkIAnn2g9kKqUSsFIomYAP23cQCmVqrXe4346GdiD8H5KGeurYtLhgjlQdxwOrTESq72fwYpfYwHOCU3inOQL+WVyMDVJipyKOg6V13JgRy37t8IeTMSGBZMSE0pKTChxKf1QyaOM9V5CdBH5Ms0nPPA/H2WxI6+y5YanoV+Cncev6N9iu6uuuoojR47gcDi45557mD17Nh9//DGPPPIITqeTqKgoPvvsM6qqqrj77rvJzMxEKcXjjz/OtGnTCA4OpqqqCoAFCxawZMkSXn/9dWbOnInNZmPTpk2MHDmSGTNmcM899+BwOAgICOC1114jLS0Np9PJQw89xMcff4zJZGLWrFn079+f559/ng8++ACATz/9lBdffJFFixa16s/obLWYTGmtG5RSc4AVGKUR5mmts5RSTwKZWuvFwByl1MVAPVBOM1N8wgf4BULqJcYNoPwQ7PvMSKz2rQJnLQEuJ6kuJ6muBrRqQFncW+BUuW/7gW+hAQuFYUNw9hpP7NDJ+HcbZCRvQvioggoHoQFWAvzMHR2KEM2aN28eERER1NTUMHz4cKZMmcKsWbNYvXo1KSkplJWVAfC73/2O0NBQtm3bBkB5eXmL587JyeGbb77BbDZTWVnJV199hcViYeXKlTzyyCMsXLiQuXPncvDgQTZv3ozFYqGsrIzw8HDuvPNOiouLiY6O5rXXXuPWW29t05/DmfBorE1rvQxY1uTYY40e39PKcQlvEN4DMm41bs1QYEwVahe4GiiqrGLtrgIKstcRmreawWUbSD/6DGx4hjJTODnh56F7jScxYxKRMd3a9aMI0dYKKh2ywbFokScjSG3l+eefPznic+TIEebOncvo0aNP1luKiDDKeqxcuZL33nvvZL/w8PAWz33NNddgNhu/SFRUVHDzzTezZ88elFLU19efPO/tt99+chrwxPvdeOONvP3229xyyy2sXbuWN998s5U+cevxvYlL0bkoZZRkMJmJifBnyvmRcH5/4FbKqutYnZ1NZdYKwnK/on/J14SXLse17gF2mXuSH5ZBWHQ8CTHRREVGYbKFGHsS+gWDv92YJvQPAWugjGqJTq+gwiEbHItO64svvmDlypWsXbuWwMBAxowZw5AhQ8jOzvb4HI1LCjSt5h4UFHTy8W9/+1vGjh3LokWLOHjwIGPGjDnleW+55RauuOIKbDYb11xzTadcc9X5IhJdRkSQH6OHDYJhg4AHqK2rY+fWr6nK+oTwvK+4sGwBljIn7GrhRMoE4SnQbRh0O8e4jxsIVlnoKzqP/AoH/RPsHR2GEM2qqKggPDycwMBAsrOz+fbbb3E4HKxevZoDBw6cnOaLiIjgkksu4YUXXuCvf/0rYEzzhYeHExsby86dO0lLS2PRokWEhIT86Ht162bMPrz++usnj19yySW88sorjB079uQ0X0REBAkJCSQkJPDUU0+xcuXKNv9ZnAlJpkSn4e/nR3rGOMgwypRpl4sDBaVs259D9qE89ucWcPRoGUHUEGqqpU+4JjUUkoOdxNcfIuDg16ht842TmSxG5fduwyDBnWBFp4FJ1quI9lfX4KK0ulbKIohOa+LEibz88sukp6eTlpbGeeedR3R0NHPnzmXq1Km4XC5iYmL49NNPefTRR7nrrrsYMGAAZrOZxx9/nKlTp/KHP/yByy+/nOjoaDIyMk4uRm/qwQcf5Oabb+app55i8uTJJ4//7Gc/Y/fu3QwaNAir1cqsWbOYM2cOANdffz3FxcWkp6e3y8/jdCmtO6bcU0ZGhs7MzOyQ9xbeq6y6jo2Hysk8VE7mwTK25lZQ1+ACwGJSDAmrYVTQEYaa9tGrfhcxx3ZgrT9mdLYGGSNWESkQ1gPCuhvrvsJ6GMVOJdFqc0qpDVrrjI6O42yd7vdXTvlxLnxmFc9MG8j04VKwWPzQzp07O22S0FnMmTOHoUOHctttt7XL+zX3Z3Kq7y8ZmRJeJSLIj4v7xXJxv1gAahucbM+tZF9RFQdLqzlUepxPSmP4Z1E6VbUTUbhIUQUMMe3jfPNhBhYeIr7gM+z1xajGux6ZLMYWO42TLHs3CI6B4FgIioGgKEm4xBk5UbBT1kwJcfqGDRtGUFAQf/7zn/+/vXsPjqrKEzj+/aXTeXbeb0LCGwMYIU4K1J0dFAbFLYTVKgyOY1nWKusuAwts1RarlqRGtKYU1/EPi/KFiMUORemwa7HjOiqo47q6EkAe4SGiQELnnXTSJCEhOfvHvemEkJDENPSD36eqq2/fPuf2OX2rT36559xzAl2UQWkwpUJadKSDn41L4WfjLr2bxBhDw/kOfqxv5bQdZP1P/XneqTvPieoWujsvkCP1jJNaZiZ4uDGuiQmRdWR6qkmoOoKjre7yD5MIiEu3gqueIMuVaQVhObMg56ZLFpJWqkfPHFM6YadSI1dWVhboIgxJgykVlkSENFc0aa7oywKtrm7D6frzHK9q4WhVC8fczbxf3cJpd6svTXp0F7dldlKc0Ulh4gUmx7WScLEBvNXgrbGea4/DEHynQAAADy5JREFU+ZrehaQjnFY34thiezB8MaRN0jsNFdXNupSMUuFMgyl13XFECBMzXEzMcHF3YY5v//kLFzle3WIFWe5mvq3w8MG3Hjq7rO7A3OQbmJWfTFFeMkX5ycwYk0RMZIS1pmFlGVTuhcp9sH8b/N9r1kFjku07DO0AyxkLHV644IULzb3bHV640GI9OrzWAtVxqRCX1u/Rd1+6NZGqP7V7oPqINZ1FbErvw6FNxWi4Pe3EOh0kxuj3qFQ40l+2Urb46Ehuzk/h5vzeK1ntnV0cOdfMgbNN7D/TyP4zTfzXQTcATocwPSeRmXnJTMuZRcENf83UXyQQ7xRrTcPKMqjYaz3/ZaM1eelAJAKi7Dm0ol3WPFoRkVBzFFrrrQWpB8sbGWsNqO9Z9idzuvWcPB4iIq5c4QteqDoE5/bBuf3Wo/7kwGmjEyE2+dIAKzbFGkt2x78O8c2qKo81YafoVUqlwpIGU0pdQYyz75gsaxbgmuZ29p9t8gVY75VVcL7DWjZHBMalxnFDdgIF2bcwbeICCm5LJN/VTUT1Yei+aAdNPZOPJlhXq670R7a7y7pi1Fp/+eN8nRUAVXwDh9/rzeOMs6aC6AmuMqZZn+X+tjdwqjveG6Ql5sKYIpi5DLJnWoFYWxO0NljBXP+HpxLaGqygT4OpIVU1t2sXn1JhTIMppUYoMzGGu2Zkc9eMbAC6uw2VTW0cdTdzrKqFY1XNHHO38OfyanpmHomLcjA1K4GpWYlMyUxgcpaLKZkuxsTHEjHU1YoIh929lwpMGTzdhRZrHFdNuXVVq6bcWpT6wLZL08VnWHNvTV9iBVBjiiAh66d9GQGaWmUgIrIQeBlrDdE3jDG/6/f+S8Ad9ss4INMYk2y/9zDwlP3eBmPM2/4sW5WnnTkTUv15SKVUENFgSqlRiogQ8lLjyEuN4047wAJo6+jiRLUVXB11W8+7j9WwY2+FL01clIPJmS4mZ7jsACuBKZku8lLjcESMsEsoOsEa/D623zQorQ1WcNXuse44TMz136D4IOm2EhEH8AqwAKgAvhGR940x5T1pjDFr+qRfCRTZ26nAeqAYMECZnXfo1VuHobvbUK1XplQYcblcg07Ieb3SYEqpqyQ2ysHMvGRm5iVfsr/hfAcna7ycrPHyXU0LJ2u8fPl9PX/cX+lLExUZwcT0eCZluJiU6WJSRjyTM11MTHcRGzXCua7iUmH8X/mjSsFsNnDSGHMKQES2A0uA8kHSP4AVQAHcBXxkjGmw834ELAT+4I+C1Z2/wMVuo4scq+H5YJ01ltGfsgvh7t8NnS7EXLx4MWjW6QuOUih1HUmNj2L2hFRm9+v2aW7v5PsaL9/Zgdb3NV6OnPPwwWE33XZvmgjkJscyKcPF5EwXkzJcFOQkUJCdQFzUdf1zzgXO9nldAcwZKKGIjMMaALf7CnlzB8i3HFgOkJ8//FnMdcJOFezWrVtHXl4eK1asAKC0tJTIyEj27NlDY2MjnZ2dbNiwgSVLlgx5LK/Xy5IlSwbMt3XrVjZu3IiIcNNNN/HOO+9QXV3N448/zqlTpwDYtGkTY8aMYdGiRRw+fBiAjRs34vV6KS0t9S3A/MUXX/DAAw8wdepUNmzYQEdHB2lpaWzbto2srCy8Xi8rV65k7969iAjr16/H4/Fw8OBB35qCr7/+OuXl5bz00kuj/g6v69ZXqWCSGOOkKD+FovxL58Vq7+zidH2rFWDVen3PX/9QT3unNYBcBCakxzM9J5HpYxJ9z5kJ+gd8AMuAd40xXSPJZIx5DXgNrOVkhptPJ+xUIxKAK0glJSWsXr3aF0zt2LGDDz/8kFWrVpGYmEhdXR233HILixcvHvKO1JiYGHbu3HlZvvLycjZs2MCXX35Jeno6DQ0NAKxatYq5c+eyc+dOurq68Hq9NDZeuYe9o6ODnuWcGhsb+eqrrxAR3njjDZ5//nlefPFFnnnmGZKSkjh06JAvndPp5Nlnn+WFF17A6XTy1ltv8eqrr4726wM0mFIq6MU4HdyQncAN2ZeuwN4z8P1YVQvl55opd3v4tqKJXfbUDQDpriim2YFVQXYC+fbYrgxXdLjdpl8J5PV5PdbeN5BlwIp+eW/vl/dTfxWs58qUjplSwaqoqIiamhrOnTtHbW0tKSkpZGdns2bNGj7//HMiIiKorKykurqa7OzsKx7LGMMTTzxxWb7du3ezdOlS0tPTAUhNta7M7969m61btwLgcDhISkoaMpgqKSnxbVdUVFBSUoLb7aajo4MJE6y7rj/++GO2b9/uS5eSYv2TOm/ePHbt2sW0adPo7OyksLBwhN/WwDSYUipE9R34vmB67914nrZOjrmbKXc320FWM5u/+ME3+ShAjDPCCqxS4nzHyLcfeamxodhl+A0wRUQmYAVHy4Bf9U8kIgVACvC/fXZ/CDwnIj2XBO8E/DbfQ1VzO06HkBYf5a9DKuV3S5cu5d1336WqqoqSkhK2bdtGbW0tZWVlOJ1Oxo8fT3t7+5DH+an5+oqMjKS7u3duvf754+PjfdsrV65k7dq1LF68mE8//ZTS0tIrHvvRRx/lueeeo6CggEceeWRE5bpimf12JKVUUEiKdTJnYhpzJqb59nVc7OZMw3nONrRxtrGVM/WtnGmwHl+dqvfNk9UjOc5JTlIsY5JiyE6KYUxyLDk920mxZCfFEOMMnkWfjTEXReQ3WIGRA9hsjDkiIr8F9hpj3reTLgO2G9M7p4MxpkFEnsEKyAB+2zMY3R+qPO1kJsQQMdK7M5W6hkpKSnjssceoq6vjs88+Y8eOHWRmZuJ0OtmzZw+nT58e1nE8Hs+A+ebNm8e9997L2rVrSUtLo6GhgdTUVObPn8+mTZtYvXq1r5svKyuLmpoa6uvrcblc7Nq1i4ULFw76ebm51hDHt9/undFkwYIFvPLKK77xUY2NjaSkpDBnzhzOnj3Lvn37OHjw4Gi+sktoMKXUdSAqMoLJmQlMzky47D1jDI2tnZxpaOWsHWC5PW24m9o552ln35lGGls7L8uXGh9FfmocO//xtqDoMjTG/An4U799T/d7XTpI3s3A5qtRLrenTe/kU0FvxowZtLS0kJubS05ODg8++CD33HMPhYWFFBcXU1BQMKzjDJZvxowZPPnkk8ydOxeHw0FRURFbtmzh5ZdfZvny5bz55ps4HA42bdrErbfeytNPP83s2bPJzc294meXlpaydOlSUlJSmDdvHj/88AMATz31FCtWrODGG2/E4XCwfv167rvvPgDuv/9+Dhw44Ov68wcxAZp0r7i42PQMIFNKBbe2ji7cnjaqPFaA5W5qw93cTsfFbjYunTns44hImTGmeOiUwW0k7dfz/32M+OhIVtwx+SqXSoWqo0ePMm3atEAX47qxaNEi1qxZw/z58wdNM9A5uVL7pVemlFJDio1y+BaHViPzLwuH9x+9UurqampqYvbs2cycOfOKgdRPocGUUkoppUbk0KFDPPTQQ5fsi46O5uuvvw5QiYaWnJzMiRMnrsqxNZhSSimlAswYExRjD4ersLCQAwcOBLoYV8VPGf4UMZxEIrJQRI6LyEkRWTfA+2tFpFxEDorIJ/YMw0oppZQaQkxMDPX19T/pj7jyL2MM9fX1xMSM7KaRIa9MDWcBUWA/UGyMaRWRfwCeB0ouP5pSSiml+ho7diwVFRXU1tYGuigKK7gdO3bsiPIMp5tvyAVEjTF7+qT/Cvj1iEqhlFJKXaecTqdv5m4VmobTzTesRUD7+Dvgg4HeEJHlIrJXRPZqBK6UUkqpcDCsMVPDJSK/BoqBFwZ63xjzmjGm2BhTnJGR4c+PVkoppZQKiOF08w1rAVER+SXwJDDXGHPBP8VTSimllApuQ86ALiKRwAlgPlYQ9Q3wK2PMkT5pioB3gYXGmO+G9cEitcDwFvuxpAN1I0gf7LQ+wS/c6hQM9RlnjAn5y9LafoVdfSD86hRu9YHA12nQ9mtYy8mIyN8Av6d3AdFn+y4gKiIfA4WA285yxhiz2D9l95VhbzgsQ9FD6xP8wq1O4VafUBJu33241QfCr07hVh8I7joNa9LOoRYQNcb80s/lUkoppZQKCX4dgK6UUkopdb0JpWDqtUAXwM+0PsEv3OoUbvUJJeH23YdbfSD86hRu9YEgrtOwxkwppZRSSqmBhdKVKaWUUkqpoKPBlFJKKaXUKAR9MCUiC0XkuIicFJF1gS6PP4jIjyJySEQOiMjeQJdnpERks4jUiMjhPvtSReQjEfnOfk4JZBlHapA6lYpIpX2eDthThIQEEckTkT0iUi4iR0Tkn+z9IX2eQo22X8Ep3Nowbb8CL6iDKRFxAK8AdwPTgQdEZHpgS+U3dxhjZgXrnBlD2AIs7LdvHfCJMWYK8In9OpRs4fI6Abxkn6dZ9hQhoeIi8M/GmOnALcAK+7cT6ucpZGj7FdS2EF5t2Ba0/QqooA6mgNnASWPMKWNMB7AdWBLgMl33jDGfAw39di8B3ra33wb+9poWapQGqVPIMsa4jTH77O0W4CjWAuUhfZ5CjLZfQSrc2jBtvwIv2IOpXOBsn9cV9r5QZ4A/i0iZiCwPdGH8JMsY0zMDfhWQFcjC+NFvROSgfRk9aC4pj4SIjAeKgK8J3/MUjLT9Ci3h+NvQ9usaCfZgKlz93BhzM9bl/xUi8otAF8ifjDXfRjjMubEJmATMwloq6cXAFmfkRMQFvAesNsY0930vjM6TurbCuv2CsPltaPt1DQV7MFUJ5PV5PdbeF9KMMZX2cw2wE6s7INRVi0gOgP1cE+DyjJoxptoY02WM6QZeJ8TOk4g4sRqibcaYP9q7w+48BTFtv0JLWP02tP26toI9mPoGmCIiE0QkClgGvB/gMo2KiMSLSELPNnAncPjKuULC+8DD9vbDwH8GsCx+0fOjtd1LCJ0nERHgTeCoMebf+rwVducpiGn7FVrC6reh7de1FfQzoNu3c/4ecACbjTHPBrhIoyIiE7H+mwNroel/D7U6icgfgNuBdKAaWA/8B7ADyAdOA/cbY0JmQOQgdbod6xK5AX4E/r5Pf31QE5GfA38BDgHd9u4nsMYdhOx5CjXafgWncGvDtP0KvKAPppRSSimlglmwd/MppZRSSgU1DaaUUkoppUZBgymllFJKqVHQYEoppZRSahQ0mFJKKaWUGgUNppRSSimlRkGDKaWUUkqpUfh//8sTfs3C4P0AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "Zdo1041zeAID", "outputId": "9ee6bf32-272c-4a7a-fbef-346a1ef5b4c5" }, "source": [ "y_pred = model.predict(test_images)\r\n", "y_pred = y_pred.argmax(axis=1)\r\n", "print(classification_report(y_test, y_pred,\r\n", " target_names=class_names))" ], "execution_count": 56, "outputs": [ { "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " T-shirt/top 0.84 0.86 0.85 1000\n", " Trouser 0.99 0.98 0.98 1000\n", " Pullover 0.87 0.86 0.86 1000\n", " Dress 0.90 0.91 0.91 1000\n", " Coat 0.84 0.88 0.86 1000\n", " Sandal 0.98 0.98 0.98 1000\n", " Shirt 0.75 0.70 0.73 1000\n", " Sneaker 0.95 0.96 0.96 1000\n", " Bag 0.97 0.98 0.98 1000\n", " Ankle boot 0.96 0.96 0.96 1000\n", "\n", " accuracy 0.91 10000\n", " macro avg 0.91 0.91 0.91 10000\n", "weighted avg 0.91 0.91 0.91 10000\n", "\n" ], "name": "stdout" } ] } ] }