Showing preview only (832K chars total). Download the full file or copy to clipboard to get everything.
Repository: rasbt/pycon2024
Branch: main
Commit: 3418e3f49ec5
Files: 47
Total size: 802.7 KB
Directory structure:
gitextract_mjxmq1vx/
├── .gitignore
├── 00-1_python-setup-guide/
│ └── README.md
├── 00-2_python-libraries-for-workshop/
│ ├── README.md
│ ├── jupyter_environment_check.ipynb
│ ├── python_environment_check.py
│ └── requirements.txt
├── 01_intro-to-deeplearning/
│ └── README.md
├── 02_pytorch-api/
│ ├── README.md
│ ├── exercise/
│ │ ├── exercise-logreg.ipynb
│ │ ├── exercise.py
│ │ └── toydata-truncated.txt
│ └── solution/
│ ├── solution-logreg.ipynb
│ ├── solution.py
│ └── toydata-truncated.txt
├── 03_training-dnns/
│ ├── README.md
│ ├── exercise/
│ │ ├── example.ipynb
│ │ ├── local_utilities.py
│ │ └── simple-cnn.py
│ └── solution/
│ ├── local_utilities.py
│ └── solution-torchvision-cnn.py
├── 04_accelerating-pytorch/
│ ├── README.md
│ ├── exercise/
│ │ ├── 00_pytorch-vit-random-init.py
│ │ ├── 01_pytorch-vit.py
│ │ └── local_utilities.py
│ └── solution/
│ ├── 02_fabric-vit.py
│ ├── 03_fabric-vit-mixed-precision.py
│ ├── 04_fabric-vit-mixed-fsdp.py
│ └── local_utilities.py
├── 05_finetuning-llms/
│ ├── README.md
│ ├── exercise/
│ │ ├── 01_train-gpt.ipynb
│ │ ├── 02_use-trained-gpt.ipynb
│ │ ├── gpt_download.py
│ │ ├── requirements-extra.txt
│ │ ├── spam_classifier_utils.py
│ │ ├── test.csv
│ │ ├── train.csv
│ │ └── validation.csv
│ └── solution/
│ ├── 01_train-gpt-solution.ipynb
│ ├── 02_use-trained-gpt-solution.ipynb
│ ├── gpt_download.py
│ ├── requirements-extra.txt
│ ├── spam_classifier_utils.py
│ ├── test.csv
│ ├── train.csv
│ └── validation.csv
├── LICENSE
└── README.md
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
.DS_Store
# Temporary files
old
# Solution files
# solution
# **/solution
# Large slide files
*.key
*.pdf
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
.pybuilder/
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock
# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
# in version control.
# https://pdm.fming.dev/#use-with-ide
.pdm.toml
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# pytype static type analyzer
.pytype/
# Cython debug symbols
cython_debug/
# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
================================================
FILE: 00-1_python-setup-guide/README.md
================================================
# Python Setup Tips
> [!TIP]
> A reproducible cloud environment will be shared with participants on the day of the workshop, so no setup steps are required. However, this document provides suggestions for those who wish to install the dependencies locally on their own machines.
---
There are several different ways you can install Python and set up your computing environment. Here, I am illustrating my personal preference.
(I am using computers running macOS, but this workflow is similar for Linux machines and may work for other operating systems as well.)
## 1. Download and install Miniforge
Download miniforge from the GitHub repository [here](https://github.com/conda-forge/miniforge).
<img src="figures/download.png" alt="download" style="zoom:33%;" />
Depending on your operating system, this should download either an `.sh` (macOS, Linux) or `.exe` file (Windows).
For the `.sh` file, open your command line terminal and execute the following command
```bash
sh ~/Desktop/Miniforge3-MacOSX-arm64.sh
```
where `Desktop/` is the folder where the Miniforge installer was downloaded to. On your computer, you may have to replace it with `Downloads/`.
<img src="figures/miniforge-install.png" alt="miniforge-install" style="zoom:33%;" />
Next, step through the download instructions, confirming with "Enter".
## 2. Create a new virtual environment
After the installation was successfully completed, I recommend creating a new virtual environment called `dl-fundamentals`, which you can do by executing
```bash
conda create -n dl-workshop python=3.9
```
<img src="figures/new-env.png" alt="new-env" style="zoom:33%;" />
Next, activate your new virtual environment (you have to do it every time you open a new terminal window or tab):
```bash
conda activate dl-workshop
```
<img src="figures/activate-env.png" alt="activate-env" style="zoom:33%;" />
## Optional: styling your terminal
If you want to style your terminal similar to mine so that you can see which virtual environment is active, check out the [Oh My Zsh](https://github.com/ohmyzsh/ohmyzsh) project.
# 3. Install new Python libraries
To install new Python libraries, you can now use the `conda` package installer. For example, you can install [JupyterLab](https://jupyter.org/install) and [watermark](https://github.com/rasbt/watermark) as follows:
```bash
conda install jupyterlab watermark
```
<img src="figures/conda-install.png" alt="conda-install" style="zoom:33%;" />
You can also still use `pip` to install libraries. By default, `pip` should be linked to your new `dl-workshop` conda environment:
<img src="figures/check-pip.png" alt="check-pip" style="zoom:33%;" />
---
Any questions? Please feel free to reach out in the [Discussion Forum](https://github.com/rasbt/pycon2024/discussions).
================================================
FILE: 00-2_python-libraries-for-workshop/README.md
================================================
# Libraries Used In This Workshop
> [!TIP]
> A reproducible cloud environment will be shared with participants on the day of the workshop, so no setup steps are required. However, this document provides suggestions for those who wish to install the dependencies locally on their own machines.
---
We will be using the following libraries in this workshop, and I highly recommend installing them before attending the event:
- numpy >= 1.26.4 (The fundamental package for scientific computing with Python)
- scipy >= 1.13.0 (Additional functions for NumPy)
- pandas >= 2.2.2 (A data frame library)
- matplotlib >= 3.8.4 (A plotting library)
- jupyterlab >= 4.0 (An application for running Jupyter notebooks)
- ipywidgets >= 8.0.6 (Fixes progress bar issues in Jupyter Lab)
- scikit-learn >= 1.4.2 (A general machine learning library)
- watermark >= 2.4.3 (An IPython/Jupyter extension for printing package information)
- torch >= 2.3.0 (The PyTorch deep learning library)
- torchvision >= 0.18.0 (PyTorch utilities for computer vision)
- torchmetrics >= 1.3.2 (Metrics for PyTorch)
- transformers >= 4.40.1 (Language transformers and LLMs for PyTorch)
- lightning >= 2.2.3 (A library for advanced PyTorch features: multi-GPU, mixed-precision etc.)
To install these requirements most conveniently, you can use the `requirements.txt` file:
```
pip install -r requirements.txt
```
Then, after completing the installation, please check if all the packages are installed and are up to date using
```
python_environment_check.py
```
It's also recommended to check the versions in JupyterLab by running the `jupyter_environment_check.ipynb` in this directory. Ideally, it should look like as follows:
If you see the following issues, it's likely that your JupyterLab instance is connected to wrong conda environment:
In this case, you may want to use `watermark` to check if you opened the JupyterLab instance in the right conda environment using the `--conda` flag:
================================================
FILE: 00-2_python-libraries-for-workshop/jupyter_environment_check.ipynb
================================================
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"id": "18d54544-92d0-412c-8e28-f9083b2bab6f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[OK] Your Python version is 3.9.19\n"
]
}
],
"source": [
"from python_environment_check import check_packages"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "60e03297-4337-4181-b8eb-f483f406954a",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[OK] numpy 1.26.4\n",
"[OK] scipy 1.13.0\n",
"[OK] pandas 2.2.2\n",
"[OK] matplotlib 3.8.4\n",
"[OK] sklearn 1.4.2\n",
"[OK] watermark 2.4.3\n",
"[OK] torch 2.3.0\n",
"[OK] torchvision 0.18.0\n",
"[OK] torchmetrics 1.3.2\n",
"[OK] transformers 4.40.1\n",
"[OK] lightning 2.2.3\n"
]
}
],
"source": [
"d = {\n",
" 'numpy': '1.26.4',\n",
" 'scipy': '1.13.0',\n",
" 'pandas': '2.2.2',\n",
" 'matplotlib': '3.8.4',\n",
" 'sklearn': '1.4.2',\n",
" 'watermark': '2.4.3',\n",
" 'torch': '2.3.0',\n",
" 'torchvision': '0.18.0',\n",
" 'torchmetrics': '1.3.2',\n",
" 'transformers': '4.40.1',\n",
" 'lightning': '2.2.3'\n",
"\n",
"}\n",
"check_packages(d)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "9d696044-9272-4b96-8305-34602807bb94",
"metadata": {},
"outputs": [],
"source": [
"%load_ext watermark"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "ce321731-a15a-4579-b33b-035730371eb3",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"numpy : 1.26.4\n",
"scipy : 1.13.0\n",
"pandas : 2.2.2\n",
"matplotlib : 3.8.4\n",
"sklearn : 1.4.2\n",
"watermark : 2.4.3\n",
"torch : 2.3.0\n",
"torchvision : 0.18.0\n",
"torchmetrics: 1.3.2\n",
"transformers: 4.40.1\n",
"lightning : 2.2.3\n",
"\n",
"conda environment: dl-workshop\n",
"\n"
]
}
],
"source": [
"%watermark --conda -p numpy,scipy,pandas,matplotlib,sklearn,watermark,torch,torchvision,torchmetrics,transformers,lightning"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.19"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
================================================
FILE: 00-2_python-libraries-for-workshop/python_environment_check.py
================================================
import platform
import sys
from packaging.version import parse as version_parse
if version_parse(platform.python_version()) < version_parse('3.8'):
print('[FAIL] We recommend Python 3.8 or newer but'
' found version %s' % (sys.version))
else:
print('[OK] Your Python version is %s' % (platform.python_version()))
def get_packages(pkgs):
versions = []
for p in pkgs:
try:
imported = __import__(p)
try:
versions.append(imported.__version__)
except AttributeError:
try:
versions.append(imported.version)
except AttributeError:
try:
versions.append(imported.version_info)
except AttributeError:
versions.append('0.0')
except ImportError:
print(f'[FAIL]: {p} is not installed and/or cannot be imported.')
versions.append('N/A')
return versions
def check_packages(d):
versions = get_packages(d.keys())
for (pkg_name, suggested_ver), actual_ver in zip(d.items(), versions):
if actual_ver == 'N/A':
continue
actual_ver, suggested_ver = version_parse(actual_ver), version_parse(suggested_ver)
if actual_ver < suggested_ver:
print(f'[FAIL] {pkg_name} {actual_ver}, please upgrade to >= {suggested_ver}')
else:
print(f'[OK] {pkg_name} {actual_ver}')
if __name__ == '__main__':
d = {
'numpy': '1.26.4',
'scipy': '1.13.0',
'pandas': '2.2.2',
'matplotlib': '3.8.4',
'sklearn': '1.4.2',
'watermark': '2.4.3',
'torch': '2.3.0',
'torchvision': '0.18.0',
'torchmetrics': '1.3.2',
'transformers': '4.40.1',
'lightning': '2.2.3'
}
check_packages(d)
================================================
FILE: 00-2_python-libraries-for-workshop/requirements.txt
================================================
numpy >= 1.26.4
scipy >= 1.13.0
pandas >= 2.2.2
matplotlib >= 3.8.4
scikit-learn >= 1.4.2
jupyterlab >= 4.0
ipywidgets >= 8.0.6
watermark >= 2.4.3
torch >= 2.3.0
torchvision >= 0.18.0
torchmetrics >= 1.3.2
transformers >= 4.40.1
lightning >= 2.2.3
================================================
FILE: 01_intro-to-deeplearning/README.md
================================================
# PyCon2024 Workshop
## 1) Introduction to Deep Learning & Setup
No code or exercises in this section.
[🔗 Slides](https://sebastianraschka.com/pdf/pycon2024/01_intro-to-deeplearning_compressed.pdf)
================================================
FILE: 02_pytorch-api/README.md
================================================
# PyCon2024 Workshop
## 2) Understanding the PyTorch API
[🔗 Slides](https://sebastianraschka.com/pdf/pycon2024/02_pytorch-api_compressed.pdf)
## Exercise: Implementing a logistic regression classifier in PyTorch
1. Open the [exercise/exercise-logreg.ipynb](exercise/exercise-logreg.ipynb) notebook and execute the code up to section "6) The training loop"
2. Fill in the missing code in the highlighted area
3. Then, execute the code and the remainder of the notebook
> [!TIP]
> If you feel stuck, check out [slides](https://sebastianraschka.com/pdf/pycon2024/01_intro-to-deeplearning_compressed.pdf) 24 to 29 for help pointers
================================================
FILE: 02_pytorch-api/exercise/exercise-logreg.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"id": "d71bce70-9dc3-448b-9f9a-8896e83b6d09",
"metadata": {},
"source": [
"# Logistic Regression Classifier"
]
},
{
"cell_type": "markdown",
"id": "e5b48fc7-4f46-4d5a-8558-cd06892aaa27",
"metadata": {},
"source": [
"## 1) Installing Libraries"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bc4fa295-5c62-4888-bcf8-d07d6a7afc47",
"metadata": {},
"outputs": [],
"source": [
"%load_ext watermark\n",
"%watermark -v -p numpy,pandas,matplotlib,torch -conda"
]
},
{
"cell_type": "markdown",
"id": "b9549676-2fa5-41a7-bbb9-ce03f5797c34",
"metadata": {},
"source": [
"## 2) Loading the Dataset"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f609024c-3eae-4ad5-8cb8-b95b403b7606",
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"\n",
"df = pd.read_csv(\"toydata-truncated.txt\", sep=\"\\t\")\n",
"df"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "319546d0-e9ed-4542-873e-395edc05ef2f",
"metadata": {},
"outputs": [],
"source": [
"X_train = df[[\"x1\", \"x2\"]].values\n",
"y_train = df[\"label\"].values"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "71792068-9926-41bb-81c0-2a46f6e956fc",
"metadata": {},
"outputs": [],
"source": [
"X_train"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f2571853-0be0-48b2-9985-8a6021d01276",
"metadata": {},
"outputs": [],
"source": [
"X_train.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3a5e5ffb-1bca-4f1b-b4cf-a78be1b07753",
"metadata": {},
"outputs": [],
"source": [
"y_train"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "68bfbbf9-4fed-4111-8391-15f2b338d8b4",
"metadata": {},
"outputs": [],
"source": [
"y_train.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b6800df4-98f6-401e-bb6c-9964c3b6e3cb",
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"\n",
"np.bincount(y_train)"
]
},
{
"cell_type": "markdown",
"id": "fc4663a6-e8a7-472e-b9b0-c64f546a85e9",
"metadata": {},
"source": [
"## 3) Visualizing the dataset"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "36a879c3-0c84-4476-a79a-f41d897c696a",
"metadata": {},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import matplotlib.pyplot as plt"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bd31bb2e-5699-43d4-8874-38e9307ce853",
"metadata": {},
"outputs": [],
"source": [
"plt.plot(\n",
" X_train[y_train == 0, 0],\n",
" X_train[y_train == 0, 1],\n",
" marker=\"D\",\n",
" markersize=10,\n",
" linestyle=\"\",\n",
" label=\"Class 0\",\n",
")\n",
"\n",
"plt.plot(\n",
" X_train[y_train == 1, 0],\n",
" X_train[y_train == 1, 1],\n",
" marker=\"^\",\n",
" markersize=13,\n",
" linestyle=\"\",\n",
" label=\"Class 1\",\n",
")\n",
"\n",
"plt.legend(loc=2)\n",
"\n",
"plt.xlim([-5, 5])\n",
"plt.ylim([-5, 5])\n",
"\n",
"plt.xlabel(\"Feature $x_1$\", fontsize=12)\n",
"plt.ylabel(\"Feature $x_2$\", fontsize=12)\n",
"\n",
"plt.grid()\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "56e353d0-f75d-4267-8b41-68433780d590",
"metadata": {},
"outputs": [],
"source": [
"X_train = (X_train - X_train.mean(axis=0)) / X_train.std(axis=0)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "295f3b29-2b8e-4280-8f25-92b4429002d2",
"metadata": {},
"outputs": [],
"source": [
"plt.plot(\n",
" X_train[y_train == 0, 0],\n",
" X_train[y_train == 0, 1],\n",
" marker=\"D\",\n",
" markersize=10,\n",
" linestyle=\"\",\n",
" label=\"Class 0\",\n",
")\n",
"\n",
"plt.plot(\n",
" X_train[y_train == 1, 0],\n",
" X_train[y_train == 1, 1],\n",
" marker=\"^\",\n",
" markersize=13,\n",
" linestyle=\"\",\n",
" label=\"Class 1\",\n",
")\n",
"\n",
"plt.legend(loc=2)\n",
"\n",
"plt.xlim([-5, 5])\n",
"plt.ylim([-5, 5])\n",
"\n",
"plt.xlabel(\"Feature $x_1$\", fontsize=12)\n",
"plt.ylabel(\"Feature $x_2$\", fontsize=12)\n",
"\n",
"plt.grid()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "db50db02-3696-4f86-b149-74baabeec6c4",
"metadata": {},
"source": [
"## 4) Implementing the model"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3da86d9a-7cd5-467c-bf65-3388fe272bd5",
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"import torch.nn.functional as F\n",
"\n",
"class LogisticRegression(torch.nn.Module):\n",
"\n",
" def __init__(self, num_features, num_classes):\n",
" super().__init__()\n",
" self.linear = torch.nn.Linear(num_features, num_classes)\n",
"\n",
" def forward(self, x):\n",
" logits = self.linear(x)\n",
" return logits"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e88b360e-c33c-4724-b46f-58323a9a7628",
"metadata": {},
"outputs": [],
"source": [
"torch.manual_seed(1)\n",
"\n",
"model = LogisticRegression(num_features=2, num_classes=2)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f2923ae0-cf24-4252-bd19-cc326a39bc72",
"metadata": {},
"outputs": [],
"source": [
"x = torch.tensor([[1.1, 2.1],\n",
" [1.1, 2.1],\n",
" [9.1, 4.1]])\n",
"\n",
"with torch.no_grad():\n",
" logits = model(x)\n",
" probas = F.softmax(logits, dim=1)\n",
"\n",
"print(probas)"
]
},
{
"cell_type": "markdown",
"id": "12068721-daf9-4ea1-9240-5bcc19c75ce9",
"metadata": {},
"source": [
"## 5) Defining a DataLoader"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f6376d80-e7cf-43b4-96eb-bfd99709b63a",
"metadata": {},
"outputs": [],
"source": [
"from torch.utils.data import Dataset, DataLoader\n",
"\n",
"\n",
"class MyDataset(Dataset):\n",
" def __init__(self, X, y):\n",
"\n",
" self.features = torch.tensor(X, dtype=torch.float32)\n",
" self.labels = torch.tensor(y, dtype=torch.int64)\n",
"\n",
" def __getitem__(self, index):\n",
" x = self.features[index]\n",
" y = self.labels[index]\n",
" return x, y\n",
"\n",
" def __len__(self):\n",
" return self.labels.shape[0]\n",
"\n",
"\n",
"train_ds = MyDataset(X_train, y_train)\n",
"\n",
"train_loader = DataLoader(\n",
" dataset=train_ds,\n",
" batch_size=10,\n",
" shuffle=True,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b64f8926-930c-4945-950a-31083608ea7e",
"metadata": {},
"outputs": [],
"source": [
"X_train.shape"
]
},
{
"cell_type": "markdown",
"id": "46bc16a0-ec59-4c54-a209-0a5e22406287",
"metadata": {},
"source": [
"## 6) The training loop"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3dcaa2b1-4019-4128-9ff5-6a966c3abdf2",
"metadata": {},
"outputs": [],
"source": [
"import torch.nn.functional as F\n",
"\n",
"\n",
"torch.manual_seed(1)\n",
"model = LogisticRegression(num_features=2, num_classes=2)\n",
"optimizer = torch.optim.SGD(model.parameters(), lr=0.5)\n",
"\n",
"num_epochs = 20\n",
"\n",
"for epoch in range(num_epochs):\n",
"\n",
" model = model.train()\n",
" for batch_idx, (features, class_labels) in enumerate(train_loader):\n",
"\n",
"\n",
" ###############################\n",
" ### Complete the training loop\n",
" ###\n",
" ### Your code below\n",
" ###############################\n",
"\n",
" outputs = # ?????\n",
"\n",
" loss = F.cross_entropy(outputs, class_labels)\n",
"\n",
" # ?????\n",
" # ?????\n",
" # ?????\n",
"\n",
" ################################\n",
" ## No changes necessary below\n",
" ################################\n",
"\n",
" ### LOGGING\n",
" print(f'Epoch: {epoch+1:03d}/{num_epochs:03d}'\n",
" f' | Batch {batch_idx:03d}/{len(train_loader):03d}'\n",
" f' | Loss: {loss:.2f}')\n"
]
},
{
"cell_type": "markdown",
"id": "bb0d5821-7c8d-46b5-9e7d-02e72cac2acc",
"metadata": {},
"source": [
"## 7) Evaluating the results"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d910ddbb-798f-47ab-8aab-e2dba4aa4005",
"metadata": {},
"outputs": [],
"source": [
"def compute_accuracy(model, dataloader):\n",
"\n",
" model = model.eval()\n",
"\n",
" correct = 0.0\n",
" total_examples = 0\n",
"\n",
" for idx, (features, class_labels) in enumerate(dataloader):\n",
"\n",
" with torch.no_grad():\n",
" logits = model(features)\n",
"\n",
" pred = torch.argmax(logits, dim=1)\n",
"\n",
" compare = class_labels == pred\n",
" correct += torch.sum(compare)\n",
" total_examples += len(compare)\n",
"\n",
" return correct / total_examples"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "27538c8d-61bc-47b0-8289-b6aab4aa16ed",
"metadata": {},
"outputs": [],
"source": [
"train_acc = compute_accuracy(model, train_loader)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5a4ecf35-4745-43a8-8ea8-14f71cba5b59",
"metadata": {},
"outputs": [],
"source": [
"print(f\"Accuracy: {train_acc*100}%\")"
]
},
{
"cell_type": "markdown",
"id": "fbcd412a-02c0-4d2c-835e-4b01368f53f8",
"metadata": {},
"source": [
"## 8) Optional: visualizing the decision boundary"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a76bb67c-358c-4e91-a5c7-5456333827aa",
"metadata": {},
"outputs": [],
"source": [
"plt.plot(\n",
" X_train[y_train == 0, 0],\n",
" X_train[y_train == 0, 1],\n",
" marker=\"D\",\n",
" markersize=10,\n",
" linestyle=\"\",\n",
" label=\"Class 0\",\n",
")\n",
"\n",
"plt.plot(\n",
" X_train[y_train == 1, 0],\n",
" X_train[y_train == 1, 1],\n",
" marker=\"^\",\n",
" markersize=13,\n",
" linestyle=\"\",\n",
" label=\"Class 1\",\n",
")\n",
"\n",
"plt.legend(loc=2)\n",
"\n",
"plt.xlim([-5, 5])\n",
"plt.ylim([-5, 5])\n",
"\n",
"plt.xlabel(\"Feature $x_1$\", fontsize=12)\n",
"plt.ylabel(\"Feature $x_2$\", fontsize=12)\n",
"\n",
"plt.grid()\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5d9f1813-d232-4a7d-aebc-cfd5303fae48",
"metadata": {},
"outputs": [],
"source": [
"def plot_boundary(model):\n",
"\n",
" w1 = model.linear.weight[0][0].detach()\n",
" w2 = model.linear.weight[0][1].detach()\n",
" b = model.linear.bias[0].detach()\n",
"\n",
" x1_min = -20\n",
" x2_min = (-(w1 * x1_min) - b) / w2\n",
"\n",
" x1_max = 20\n",
" x2_max = (-(w1 * x1_max) - b) / w2\n",
"\n",
" return x1_min, x1_max, x2_min, x2_max"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2d71b5df-dd8d-41d5-b6fd-640d6f40d5a2",
"metadata": {},
"outputs": [],
"source": [
"x1_min, x1_max, x2_min, x2_max = plot_boundary(model)\n",
"\n",
"\n",
"plt.plot(\n",
" X_train[y_train == 0, 0],\n",
" X_train[y_train == 0, 1],\n",
" marker=\"D\",\n",
" markersize=10,\n",
" linestyle=\"\",\n",
" label=\"Class 0\",\n",
")\n",
"\n",
"plt.plot(\n",
" X_train[y_train == 1, 0],\n",
" X_train[y_train == 1, 1],\n",
" marker=\"^\",\n",
" markersize=13,\n",
" linestyle=\"\",\n",
" label=\"Class 1\",\n",
")\n",
"\n",
"plt.plot([x1_min, x1_max], [x2_min, x2_max], color=\"k\")\n",
"\n",
"plt.legend(loc=2)\n",
"\n",
"plt.xlim([-5, 5])\n",
"plt.ylim([-5, 5])\n",
"\n",
"plt.xlabel(\"Feature $x_1$\", fontsize=12)\n",
"plt.ylabel(\"Feature $x_2$\", fontsize=12)\n",
"\n",
"plt.grid()\n",
"plt.show()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
================================================
FILE: 02_pytorch-api/exercise/exercise.py
================================================
# %% [markdown]
# # Logistic Regression Classifier
# %% [markdown]
# ## 1) Installing Libraries
# %%
%load_ext watermark
%watermark -v -p numpy,pandas,matplotlib,torch -conda
# %% [markdown]
# ## 2) Loading the Dataset
# %%
import pandas as pd
df = pd.read_csv("toydata-truncated.txt", sep="\t")
df
# %%
X_train = df[["x1", "x2"]].values
y_train = df["label"].values
# %%
X_train
# %%
X_train.shape
# %%
y_train
# %%
y_train.shape
# %%
import numpy as np
np.bincount(y_train)
# %% [markdown]
# ## 3) Visualizing the dataset
# %%
%matplotlib inline
import matplotlib.pyplot as plt
# %%
plt.plot(
X_train[y_train == 0, 0],
X_train[y_train == 0, 1],
marker="D",
markersize=10,
linestyle="",
label="Class 0",
)
plt.plot(
X_train[y_train == 1, 0],
X_train[y_train == 1, 1],
marker="^",
markersize=13,
linestyle="",
label="Class 1",
)
plt.legend(loc=2)
plt.xlim([-5, 5])
plt.ylim([-5, 5])
plt.xlabel("Feature $x_1$", fontsize=12)
plt.ylabel("Feature $x_2$", fontsize=12)
plt.grid()
plt.show()
# %%
X_train = (X_train - X_train.mean(axis=0)) / X_train.std(axis=0)
# %%
plt.plot(
X_train[y_train == 0, 0],
X_train[y_train == 0, 1],
marker="D",
markersize=10,
linestyle="",
label="Class 0",
)
plt.plot(
X_train[y_train == 1, 0],
X_train[y_train == 1, 1],
marker="^",
markersize=13,
linestyle="",
label="Class 1",
)
plt.legend(loc=2)
plt.xlim([-5, 5])
plt.ylim([-5, 5])
plt.xlabel("Feature $x_1$", fontsize=12)
plt.ylabel("Feature $x_2$", fontsize=12)
plt.grid()
plt.show()
# %% [markdown]
# ## 4) Implementing the model
# %%
import torch
import torch.nn.functional as F
class LogisticRegression(torch.nn.Module):
def __init__(self, num_features, num_classes):
super().__init__()
self.linear = torch.nn.Linear(num_features, num_classes)
def forward(self, x):
logits = self.linear(x)
return logits
# %%
torch.manual_seed(1)
model = LogisticRegression(num_features=2, num_classes=2)
# %%
x = torch.tensor([[1.1, 2.1],
[1.1, 2.1],
[9.1, 4.1]])
with torch.no_grad():
logits = model(x)
probas = F.softmax(logits, dim=1)
print(probas)
# %% [markdown]
# ## 5) Defining a DataLoader
# %%
from torch.utils.data import Dataset, DataLoader
class MyDataset(Dataset):
def __init__(self, X, y):
self.features = torch.tensor(X, dtype=torch.float32)
self.labels = torch.tensor(y, dtype=torch.int64)
def __getitem__(self, index):
x = self.features[index]
y = self.labels[index]
return x, y
def __len__(self):
return self.labels.shape[0]
train_ds = MyDataset(X_train, y_train)
train_loader = DataLoader(
dataset=train_ds,
batch_size=10,
shuffle=True,
)
# %%
X_train.shape
# %% [markdown]
# ## 6) The training loop
# %%
import torch.nn.functional as F
torch.manual_seed(1)
model = LogisticRegression(num_features=2, num_classes=2)
optimizer = torch.optim.SGD(model.parameters(), lr=0.5)
num_epochs = 20
for epoch in range(num_epochs):
model = model.train()
for batch_idx, (features, class_labels) in enumerate(train_loader):
###############################
### Complete the training loop
###
### Your code below
###############################
logits = # ?????
loss = F.cross_entropy(logits, class_labels)
# ?????
# ?????
# ?????
################################
## No changes necessary below
################################
### LOGGING
print(f'Epoch: {epoch+1:03d}/{num_epochs:03d}'
f' | Batch {batch_idx:03d}/{len(train_loader):03d}'
f' | Loss: {loss:.2f}')
# %% [markdown]
# ## 7) Evaluating the results
# %%
def compute_accuracy(model, dataloader):
model = model.eval()
correct = 0.0
total_examples = 0
for idx, (features, class_labels) in enumerate(dataloader):
with torch.no_grad():
logits = model(features)
pred = torch.argmax(logits, dim=1)
compare = class_labels == pred
correct += torch.sum(compare)
total_examples += len(compare)
return correct / total_examples
# %%
train_acc = compute_accuracy(model, train_loader)
# %%
print(f"Accuracy: {train_acc*100}%")
# %% [markdown]
# ## 8) Optional: visualizing the decision boundary
# %%
plt.plot(
X_train[y_train == 0, 0],
X_train[y_train == 0, 1],
marker="D",
markersize=10,
linestyle="",
label="Class 0",
)
plt.plot(
X_train[y_train == 1, 0],
X_train[y_train == 1, 1],
marker="^",
markersize=13,
linestyle="",
label="Class 1",
)
plt.legend(loc=2)
plt.xlim([-5, 5])
plt.ylim([-5, 5])
plt.xlabel("Feature $x_1$", fontsize=12)
plt.ylabel("Feature $x_2$", fontsize=12)
plt.grid()
plt.show()
# %%
def plot_boundary(model):
w1 = model.linear.weight[0][0].detach()
w2 = model.linear.weight[0][1].detach()
b = model.linear.bias[0].detach()
x1_min = -20
x2_min = (-(w1 * x1_min) - b) / w2
x1_max = 20
x2_max = (-(w1 * x1_max) - b) / w2
return x1_min, x1_max, x2_min, x2_max
# %%
x1_min, x1_max, x2_min, x2_max = plot_boundary(model)
plt.plot(
X_train[y_train == 0, 0],
X_train[y_train == 0, 1],
marker="D",
markersize=10,
linestyle="",
label="Class 0",
)
plt.plot(
X_train[y_train == 1, 0],
X_train[y_train == 1, 1],
marker="^",
markersize=13,
linestyle="",
label="Class 1",
)
plt.plot([x1_min, x1_max], [x2_min, x2_max], color="k")
plt.legend(loc=2)
plt.xlim([-5, 5])
plt.ylim([-5, 5])
plt.xlabel("Feature $x_1$", fontsize=12)
plt.ylabel("Feature $x_2$", fontsize=12)
plt.grid()
plt.show()
================================================
FILE: 02_pytorch-api/exercise/toydata-truncated.txt
================================================
x1 x2 label
0.77 -1.14 0
-0.33 1.44 0
0.91 -3.07 0
-0.37 -1.91 0
-0.63 -1.53 0
0.39 -1.99 0
-0.49 -2.74 0
-0.68 -1.52 0
-0.10 -3.43 0
-0.05 -1.95 0
3.88 0.65 1
0.73 2.97 1
0.83 3.94 1
1.59 1.25 1
1.14 3.91 1
1.73 2.80 1
1.31 1.85 1
1.56 3.85 1
1.23 2.54 1
1.33 2.03 1
================================================
FILE: 02_pytorch-api/solution/solution-logreg.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"id": "d71bce70-9dc3-448b-9f9a-8896e83b6d09",
"metadata": {},
"source": [
"# Logistic Regression Classifier"
]
},
{
"cell_type": "markdown",
"id": "e5b48fc7-4f46-4d5a-8558-cd06892aaa27",
"metadata": {},
"source": [
"## 1) Installing Libraries"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "be1f5a9a-b3ee-424b-ab02-4371f49bd786",
"metadata": {},
"outputs": [],
"source": [
"# !conda install numpy pandas matplotlib --yes"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "1ea7b3b8-9092-4b37-8b7f-57362be611ad",
"metadata": {},
"outputs": [],
"source": [
"# !pip install torch torchvision torchaudio"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "79dd2077-ba5c-4ab5-95fc-6ee4d8a9f811",
"metadata": {},
"outputs": [],
"source": [
"# !conda install watermark"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "bc4fa295-5c62-4888-bcf8-d07d6a7afc47",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Python implementation: CPython\n",
"Python version : 3.9.16\n",
"IPython version : 8.12.0\n",
"\n",
"numpy : 1.25.0\n",
"pandas : 2.0.3\n",
"matplotlib: 3.7.1\n",
"torch : 2.0.1\n",
"\n"
]
}
],
"source": [
"%load_ext watermark\n",
"%watermark -v -p numpy,pandas,matplotlib,torch -conda"
]
},
{
"cell_type": "markdown",
"id": "b9549676-2fa5-41a7-bbb9-ce03f5797c34",
"metadata": {},
"source": [
"## 2) Loading the Dataset"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "f609024c-3eae-4ad5-8cb8-b95b403b7606",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>x1</th>\n",
" <th>x2</th>\n",
" <th>label</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>0.77</td>\n",
" <td>-1.14</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>-0.33</td>\n",
" <td>1.44</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>0.91</td>\n",
" <td>-3.07</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>-0.37</td>\n",
" <td>-1.91</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>-0.63</td>\n",
" <td>-1.53</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>0.39</td>\n",
" <td>-1.99</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>6</th>\n",
" <td>-0.49</td>\n",
" <td>-2.74</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>-0.68</td>\n",
" <td>-1.52</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>-0.10</td>\n",
" <td>-3.43</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>9</th>\n",
" <td>-0.05</td>\n",
" <td>-1.95</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>10</th>\n",
" <td>3.88</td>\n",
" <td>0.65</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>11</th>\n",
" <td>0.73</td>\n",
" <td>2.97</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>12</th>\n",
" <td>0.83</td>\n",
" <td>3.94</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>13</th>\n",
" <td>1.59</td>\n",
" <td>1.25</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>14</th>\n",
" <td>1.14</td>\n",
" <td>3.91</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>1.73</td>\n",
" <td>2.80</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>16</th>\n",
" <td>1.31</td>\n",
" <td>1.85</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>17</th>\n",
" <td>1.56</td>\n",
" <td>3.85</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>18</th>\n",
" <td>1.23</td>\n",
" <td>2.54</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>19</th>\n",
" <td>1.33</td>\n",
" <td>2.03</td>\n",
" <td>1</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" x1 x2 label\n",
"0 0.77 -1.14 0\n",
"1 -0.33 1.44 0\n",
"2 0.91 -3.07 0\n",
"3 -0.37 -1.91 0\n",
"4 -0.63 -1.53 0\n",
"5 0.39 -1.99 0\n",
"6 -0.49 -2.74 0\n",
"7 -0.68 -1.52 0\n",
"8 -0.10 -3.43 0\n",
"9 -0.05 -1.95 0\n",
"10 3.88 0.65 1\n",
"11 0.73 2.97 1\n",
"12 0.83 3.94 1\n",
"13 1.59 1.25 1\n",
"14 1.14 3.91 1\n",
"15 1.73 2.80 1\n",
"16 1.31 1.85 1\n",
"17 1.56 3.85 1\n",
"18 1.23 2.54 1\n",
"19 1.33 2.03 1"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import pandas as pd\n",
"\n",
"df = pd.read_csv(\"toydata-truncated.txt\", sep=\"\\t\")\n",
"df"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "319546d0-e9ed-4542-873e-395edc05ef2f",
"metadata": {},
"outputs": [],
"source": [
"X_train = df[[\"x1\", \"x2\"]].values\n",
"y_train = df[\"label\"].values"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "71792068-9926-41bb-81c0-2a46f6e956fc",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 0.77, -1.14],\n",
" [-0.33, 1.44],\n",
" [ 0.91, -3.07],\n",
" [-0.37, -1.91],\n",
" [-0.63, -1.53],\n",
" [ 0.39, -1.99],\n",
" [-0.49, -2.74],\n",
" [-0.68, -1.52],\n",
" [-0.1 , -3.43],\n",
" [-0.05, -1.95],\n",
" [ 3.88, 0.65],\n",
" [ 0.73, 2.97],\n",
" [ 0.83, 3.94],\n",
" [ 1.59, 1.25],\n",
" [ 1.14, 3.91],\n",
" [ 1.73, 2.8 ],\n",
" [ 1.31, 1.85],\n",
" [ 1.56, 3.85],\n",
" [ 1.23, 2.54],\n",
" [ 1.33, 2.03]])"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"X_train"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "f2571853-0be0-48b2-9985-8a6021d01276",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(20, 2)"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"X_train.shape"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "3a5e5ffb-1bca-4f1b-b4cf-a78be1b07753",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"y_train"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "68bfbbf9-4fed-4111-8391-15f2b338d8b4",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(20,)"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"y_train.shape"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "b6800df4-98f6-401e-bb6c-9964c3b6e3cb",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([10, 10])"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import numpy as np\n",
"\n",
"np.bincount(y_train)"
]
},
{
"cell_type": "markdown",
"id": "fc4663a6-e8a7-472e-b9b0-c64f546a85e9",
"metadata": {},
"source": [
"## 3) Visualizing the dataset"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "36a879c3-0c84-4476-a79a-f41d897c696a",
"metadata": {},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import matplotlib.pyplot as plt"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "bd31bb2e-5699-43d4-8874-38e9307ce853",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAG2CAYAAACZEEfAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA9RklEQVR4nO3deXxU9b3/8fckhEBCAkLCJgECBjeqKAhaLAVZBXGtFcUKFrEqiBAKgoCEFi9wtWIDFAQ0Wlsu3IcIthaRCAg/BSqXpSDIKpthkWCTIEsyzJzfH3Gm2TPLmczMmdfz8cgD5pwzZz7hCydvvuf7PV+bYRiGAAAALCoq2AUAAAAEEmEHAABYGmEHAABYGmEHAABYGmEHAABYGmEHAABYGmEHAABYWq1gFxAKnE6nTp48qYSEBNlstmCXAwAAPGAYhs6fP6/mzZsrKqry/hvCjqSTJ08qJSUl2GUAAAAfnDhxQi1atKh0P2FHUkJCgqTiP6zExMQgV+Mbu92uNWvWqE+fPoqJiQl2ORGNtggttEfooC1Ch1XaoqCgQCkpKe6f45Uh7EjuW1eJiYlhHXbi4uKUmJgY1n9xrYC2CC20R+igLUKH1dqiuiEoDFAGAACWRtgBAACWRtgBAACWxpgdDzmdThUVFQW7jErZ7XbVqlVLly9flsPhCHY5AVG7du0qpxYCAFARwo4HioqKdOTIETmdzmCXUinDMNS0aVOdOHHCss8KioqKUmpqqmrXrh3sUgAAYYSwUw3DMHTq1ClFR0crJSUlZHsWnE6nfvjhB9WrVy9ka/SH68GPp06dUsuWLS0b6AAA5iPsVOPKlSu6ePGimjdvrri4OL/Olbn2oGZnH9CY3u00qmeaSRUWc91mq1OnjiXDjiQlJyfr5MmTunLliiWmSgIAaoY1fyqayDX+xd9bJ5lrD+r17AMyJL2efUCZaw+aUF1kcbWBVcckAQACg7DjIX9um7iCTkkEHu9x6woA4AvCToBVFHRcCDwAAAQeYSeAqgo6LgQeAAACi7ATIJ4EHZdgBh6bzaaVK1cG5bMBAKgJhJ0A8CbouAQi8Jw+fVrPP/+82rRpo9jYWKWkpGjgwIFau3atqZ/jK8Mw9PLLL6tZs2aqW7euevXqpYMH6eUCAJiLsGMyX4KOi5mB5+jRo+rYsaPWrVunV199Vbt379bq1avVo0cPjRgxwpTP8Nd///d/KzMzUwsWLNA///lPxcfHq2/fvrp8+XKwSwMAWAhhx0T+BB0XswLPc889J5vNpi+//FIPPfSQ2rVrpxtvvFHp6enasmVLpe978cUX1a5dO8XFxalNmzaaMmWK7Ha7e/+//vUv9ejRQwkJCUpMTFTHjh31f//3f5KkY8eOaeDAgbrqqqsUHx+vG2+8UatWrarwcwzD0BtvvKHJkyfrvvvu00033aQ///nPOnnyJLfVAACm4qGCJjEj6Li4zuPrgwe///57rV69Wq+88ori4+PL7W/QoEGl701ISNA777yj5s2ba/fu3Ro+fLgSEhI0fvx4SdLgwYN1yy23aP78+YqOjtbOnTvdD/gbMWKEioqKtHHjRsXHx2vv3r2qV69ehZ9z5MgRnT59Wr169XJvq1+/vrp06aLNmzdr0KBBPn3vAACURdgxyWyTgk7J8/kadg4dOiTDMHTdddd5/d7Jkye7f9+6dWv99re/1dKlS91h5/jx4xo3bpz73Glp/6nx+PHjeuihh/STn/xEktSmTZtKP+f06dOSpCZNmpTa3qRJE/c+AADMwG0sk4zp3S5kzmcYhs/vXbZsmbp27aqmTZuqXr16mjx5so4fP+7en56erqeeekq9evXSzJkzdfjwYfe+UaNGafr06erataumTp2qXbt2+VwHAABmIeyYZFTPNKWbFHjS/Vw7Ky0tTTabTfv27fPqfZs3b9bgwYPVv39/ffTRR9qxY4cmTZqkoqIi9zEZGRnas2ePBgwYoHXr1umGG27QihUrJElPPfWUvvnmG/3qV7/S7t271alTJ82ZM6fCz2ratKkk6cyZM6W2nzlzxr0PAAAzEHZMZEbg8TfoSFLDhg3Vt29fzZs3TxcuXCi3Py8vr8L3bdq0Sa1atdKkSZPUqVMnpaWl6dixY+WOa9euncaMGaM1a9bowQcfVFZWlntfSkqKnnnmGX3wwQcaO3asFi1aVOFnpaamqmnTpqWmwRcUFOif//yn7rjjDi+/YwAAKkfYMZk/gceMoOMyb948ORwOde7cWcuXL9fBgwf19ddfKzMzs9IwkZaWpuPHj2vp0qU6fPiwMjMz3b02knTp0iWNHDlSn332mY4dO6YvvvhCW7du1fXXXy9JGj16tD755BMdOXJE27dv1/r16937yrLZbBo9erSmT5+uv/3tb9q9e7eeeOIJNW/eXPfff78pfwYAAEgMUA4IV2DxZnaWmUFHKh4cvH37dr3yyisaO3asTp06peTkZHXs2FHz58+v8D333nuvxowZo5EjR6qwsFADBgzQlClTlJGRIUmKjo7WuXPn9MQTT+jMmTNKSkrSgw8+qGnTpkkqXo18xIgR+vbbb5WYmKh+/fpp9uzZldY4fvx4XbhwQU8//bTy8vJ05513avXq1apTp45pfw4AANgMf0azWkRBQYHq16+v/Px8JSYmltp3+fJlHTlyRKmpqV7/EPZ0OroZQcfpdKqgoECJiYmKirJmh50/bVGT7Ha7Vq1apf79+7un5SN4aI/QQVuEDqu0RVU/v0uy5k/FEOHJLS2ze3QAAEBphJ0AqyrwEHQAAAg8wk4NqCjwEHQAAKgZDFCuIa5gMzv7gMYQdAAAqDGEnRo0qmcaIQdAeYfXSx+/KN09S2rbI9jVAJbDbayadHi9NLdz8a8AIEmGIa2dJuXuL/7V2wmyXFeAahF2aoq/FzQA1nR4rXRyR/HvT+4ofu0priuARywXdmbOnOl+Om9I8eeCBsCaDENaN12yRRe/tkUXv/Y0tHBdATxiqbCzdetWvfnmm7rpppuCXUpp/l7QAshms2nlypXBLgOITK6wYjiKXxsOz0NLIK8r3BqDxVgm7Pzwww8aPHiwFi1apKuuuirY5ZTmzwXND6dPn9bzzz+vNm3aKDY2VikpKRo4cGCpxTeD6YMPPlCfPn3UqFEj2Ww27dy5M9glATWnbFhx8TS0BOq6wq0xWJBlZmONGDFCAwYMUK9evTR9+vQqjy0sLFRhYaH7dUFBgaTix2fb7fZSx9rtdhmGIafTKafT6X1hhiHbjxc0m+uiJMn48YJmpPaQbDbvz1vuYwz3r06nU0ePHtXPfvYzNWjQQLNmzdJPfvIT2e12rVmzRiNGjNDevXvd7/X5e/PT+fPn1bVrV/3iF7/Qb37zm2rrcDqdMgxDdrtd0dHRlR4XbK6/Q2X/LiE4QrU9bIfXqZbrFlRJP4aWK/vXyGh7V8VvNgxFr50uWwXXFWPtdDladvP5ulKqrurq8FKotkUkskpbeFq/JcLO0qVLtX37dm3dutWj42fMmOFevLKkNWvWKC4urtS2WrVqqWnTpvrhhx9UVFTkdW21jm5QvQouaLYfL2gXdn+kK61/7vV5K3P+/HlJ0m9+8xtJxd9TfHy8e/+wYcP0i1/8wh3wpOLVzF2vp06dqn/84x86efKkGjdurIcffljjx493r52ye/duvfTSS9q5c6dsNpvatGmj2bNn65ZbbtHx48c1fvx4bdmyRXa7XS1bttS0adPUp0+fCmu97777JEnHjx+XJF24cKFUXWUVFRXp0qVL2rhxo65cueLrH1GNyc7ODnYJKCGk2sMw1G1/huorSlEqH/CditL5D1/UxmszKgwtyQW79NNTFV9XbKd2aMuyWTqb6MPt/DJ1VVeHr0KqLSJcuLfFxYsXPTou7MPOiRMn9MILLyg7O9vjxSEnTpyo9PR09+uCggKlpKSoT58+FS4EeuLECdWrV8/7xScNQ7YvZ8so878v925btOK/nC3jJ/f4fSExDEPnz59XQkKC/v3vf2vt2rWaPn26mjVrVu7Yst9j3bp13duSkpL0zjvvqHnz5tq9e7d+85vfKCkpSePGjZMkPfvss+rQoYPefPNNRUdHa+fOnWrQoIESExM1ceJEORwObdiwQfHx8dq7d68SExOrXJxNkurVqydJio+Pr/LYy5cvq27duurWrVvILwSanZ2t3r17h/UCe1YRiu1hO7xOtXYeqXR/lJy66tIRDbiubvleFcNQdFbV15XbL66V45EXvb6ulK2ryjp8EIptEams0hZV/Qe5pLAPO9u2bdN3332nW2+91b3N4XBo48aNmjt3rgoLC8vd8oiNjVVsbGy5c8XExJRrdIfDIZvNpqioKO9XEz/06X9mSlTA1btjO7JeuqaXd+cuw3X7x2az6ZtvvpFhGLr++us9qrnk9zZlyhT39jZt2ujgwYNaunSpXnzxRUnFvTDjxo3TDTfcIEm69tpr3cefOHFCDz30kG6++WZJ0jXXXONR7a7Pru7POCoqSjabrcJ2CkXhUmekCJn2MAxp44zisTkVhBU3W7RqbZwhXdundGg59KlUQa+O+20/9u5EHd/o3XWlsroqq8MPIdMWCPu28LT2sB+g3LNnT+3evVs7d+50f3Xq1EmDBw/Wzp07gze2o7LBh2UFYGaW4ce5li1bpq5du6pp06aqV6+eJk+e7L7NJEnp6el66qmn1KtXL82cOVOHDx927xs1apSmT5+url27aurUqdq1a5df3wdgSWUHFlemogHHgbyuVFZXDU2oAAIp7MNOQkKC2rdvX+orPj5ejRo1Uvv27YNXmD8XND+lpaXJZrNp3759Xr1v8+bNGjx4sPr376+PPvpIO3bs0KRJk0qNVcrIyNCePXs0YMAArVu3TjfccINWrFghSXrqqaf0zTff6Fe/+pV2796tTp06ac6cOaZ9X0DY8zSsuJQNLYG6rlRXVwg9LgPwRdiHnZDk7wXNTw0bNlTfvn01b948Xbhwodz+vLy8Ct+3adMmtWrVSpMmTVKnTp2UlpamY8eOlTuuXbt2GjNmjNasWaMHH3xQWVlZ7n0pKSl65pln9MEHH2js2LFatGiRKd8TYAmehhWXkqElkNeV6uqidwdhzpJh57PPPtMbb7wRvAL8uaCZZN68eXI4HOrcubOWL1+ugwcP6uuvv1ZmZqbuuOOOCt+Tlpam48ePa+nSpTp8+LAyMzPdvTZS8aytkSNH6rPPPtOxY8f0xRdfaOvWrbr++uslSaNHj9Ynn3yiI0eOaPv27Vq/fr17X0W+//577dy50z0Nfv/+/dq5c6dOnz5t2p8DEDK8DSsurtByKEDXlSDecgdqiiXDTlD5e0Ez6ULSpk0bbd++XT169NDYsWPVvn179e7dW2vXrtX8+fMrfM+9996rMWPGaOTIkerQoYM2bdpUasBydHS0zp07pyeeeELt2rXTL3/5S919993uafwOh0MjRozQ9ddfr379+qldu3b605/+VGmNf/vb33TLLbdowIABkqRBgwbplltu0YIFC0z5MwBCirf/CXJxhZbV4wNzXQniLXegptgMf0azWkRBQYHq16+v/Pz8CqeeHzlyRKmpqZ5Ndz70qfSXh3wv5vHlPs3McjqdKigoUGJiovezxsKE120RJHa7XatWrVL//v3DepaDVYREexiGtKiHdGqX92FHUvH/S/148Gdl1xVv67JFS81ukoav92lmVki0BSRZpy2q+vldkjV/KgaLr706LnQTA9bka6+Omx9Bp6rrSgjccgdqAmHHTP5e0LiQANbj+k9QsC63lV1XQuSWO1ATCDtmMe2CFsWFBLASR5GUnyO/emf8VsF1xd8xRPynDGEk7J+gHDJMu6A5pYKc4vPVKv+UZwBhplas9PR66UKu9+91FElLfild+refRZS5rpTs1fGlJ9rVu9O2p6lrZgGBQtjxULXjuP25oJUVn0zQqQBj6RG26rco/vLFM5+bf11x9er4qmTvjp9L3QA1gbBTDddyE0VFRapbt27VB/tzQUO1XE9yDtoSIEAwmH1dKXXL3Z+e6Ch6dxA2CDvVqFWrluLi4nT27FnFxMSE7LRup9OpoqIiXb58OWRr9IfT6dTZs2cVFxenWrX4awv4jFvuiED81KiGzWZTs2bNdOTIkQqXTggVhmHo0qVLqlu3rmwW/V9WVFSUWrZsadnvD6gR3HJHBCLseKB27dpKS0srtSBmqLHb7dq4caO6desW1g+Iqkrt2rUt2WsF1DhuuSPCEHY8FBUVFdJP7Y2OjtaVK1dUp04dy4YdAAB8wX+TAQCApRF2AACApRF2AACApRF2AACApRF2AACApRF2AACApRF2AACApRF2AACApRF2AACApRF2AACApRF2AKAmHV4vze1c/CuAGkHYAYCaYhjS2mlS7v7iXw0j2BUBEYGwAwA15fBa6eSO4t+f3FH8GkDAEXYAoCYYhrRuumSLLn5tiy5+Te8OEHCEHQCoCa5eHcNR/Npw0LsD1BDCDgAEWtleHRd6d4AaQdgBgEAr26vjEq69O8woQ5gh7ABAIFXWq+MSbr07zChDGCLsAEAgVdar4xJuvTvMKEMYIuwAQKBU16vjEi69O8woQ5gi7ABAoFTXq+MSLr07zChDmCLsAEAgeNqr4xLqvSTMKEMYI+wAQCB42qvjEuq9JFabUYaIQtgBALN526vjEqq9JFabUYaIQ9gBALN526vjEqq9JFabUYaIQ9gBADP52qvjEmq9JFabUYaIRNgBADP52qvjYkYviZlPOLbajDJEJMIOAJjF1Qvi96U1yvdeEjOfcGy1GWWIWIQdADCLo0jKz5Hk9PNETqkgp/h83jLzCcdWm1GGiFUr2AUAgGXUipWeXi9dyPX/XPHJxefzRsmeGMPxn56Wtj0lm82/c3nKn88EAoSwAwBmqt+i+CsYSvbqSKV7Wq7p5d+5POXPZwIBwm0sALACM59wbLUZZYh4hB0AsAIzn3AcCjPKABMRdgAg3Jn5hONQmFEGmIywAwDhzswnHIfCjDLAZAxQBoBw5umsKU9nSQV7RhkQAIQdAAhnns6a8maWVDBnlAEBwG0sAAhXPOEY8AhhBwDCFU84BjxC2AGAcOTrs3Do3UEEIuwAQDjy9Vk49O4gAhF2ACDc8IRjwCuEHQAINzzhGPAKYQcAwglPOAa8RtgBgHDCE44Br/FQQQAIJ74+4fjb/5M2ZUo/HSW16FS8jSccI0IQdgAg3Hj7hGPDkD4aLeUdk3b+RbptWNVLRgAWw20sALC6kktKMDAZEYiwAwBWVnaaOtPOEYEIOwBgZWWnqTPtHBGIsAMAVlXZwwfp3UGECfuwM2PGDN12221KSEhQ48aNdf/992v//v3BLgsAgq+yhw/Su4MIE/ZhZ8OGDRoxYoS2bNmi7Oxs2e129enTRxcuXAh2aQAQPNUtKUHvDiJI2E89X716danX77zzjho3bqxt27apW7duQaoKAIKs5AysipTs3bmmV83VBQRB2IedsvLz8yVJDRs2rPSYwsJCFRYWul8XFBRIkux2u+x2e2ALDBBX3eFav5XQFqElItvDMBS9drpstmjZqlg/y7BFy1g7XY6W3WrkuTsR2RYhyipt4Wn9NsOwTh+m0+nUvffeq7y8PH3++eeVHpeRkaFp06aV275kyRLFxcUFskQACLjkgl366eHXPD5+U9vf6mziTQGsCAiMixcv6rHHHlN+fr4SExMrPc5SYefZZ5/Vxx9/rM8//1wtWlT+dNGKenZSUlKUm5tb5R9WKLPb7crOzlbv3r0VExMT7HIiGm0RWiKuPQxD0Vl9ZDu9q8peHffhtmgZTW+S48k1Ae/dibi2CGFWaYuCggIlJSVVG3Yscxtr5MiR+uijj7Rx48Yqg44kxcbGKja2/HowMTExYd3okjW+B6ugLUJLxLTHoU+lU1WM1SnDZjhkO7VDUcc31tjYnYhpizAQ7m3hae1hPxvLMAyNHDlSK1as0Lp165SamhrskgAgOKqbgVUZZmbB4sI+7IwYMUJ/+ctftGTJEiUkJOj06dM6ffq0Ll26FOzSAKBmVfZcnerw3B1YXNiHnfnz5ys/P1/du3dXs2bN3F/Lli0LdmkAUHN87dVxoXcHFhb2Y3YsNL4aAHxX3XN1qsNzd2BhYd+zAwARz9Wr4/clPYreHVgSYQcAwp2jSMrPkeT080ROqSCn+HyAhYT9bSwAiHi1YqWn10v7Vkkfj/P+/Xe/KqV0Lv59fHLx+QALIewAgBUkXi39a0nxQGNvZmPZoovf13l4jSwZAQQDt7EAhKzMtQeVOuEfylx7MNilhD6mnQOVIuwACEmZaw/q9ewDMiS9nn2AwFMVpp0DVSLsAAg5rqBTEoGnCr726rjQuwOLI+wACCkVBR0XAk8FmHYOVIuwAyBkVBV0XAg8ZTDtHKgWs7EAhARPgo6L67hRPdMCWVJ4cE07v5Dr/7mYdg6LIuwACDpvgo4LgaeE+i2KvwBUiNtYAILKl6Djwi0tAJ4g7AAIGn+CjguBJ4AOr5fmdi7+FQhjhB0AQWFG0HEh8ASAYUhrp0m5+4t/ZZYWwhhhB0BQzDYp6ATqfBHP9eweiWfwIOwRdgAExZje7UL6fBGt7BOZecIywhxhB0BQjOqZpnSTAkp673bMyjJT2Scy84RlhDnCDoCgMSPwEHRMVtk6W/TuIIwRdgAElT+Bh6ATAJWts0XvDsIYYQdA0PkSeAg6AVDd6un07iBMEXYAhARvAg9BJ0CqWz2d3h2EKcIOgJDhSeAh6ARIdb06LvTuIAwRdgCElKoCD0EngKrr1XGhdwdhiLADIORUFHgIOgHkaa+OC707CDOEHQAhyRV4bCLoBJynvTou9O4gzNQKdgEAUJlRPdMIOYFWslfH07Aj/ad3p21PyWYLXH2ACUzt2bl48aJ27Nih8+fPl9v3xRdfmPlRAAAzeNur40LvDsKIaWFny5YtatWqle655x41adJE06dPL7X/7rvvNuujAABm8HasTlmM3UGYMC3spKena+7cucrJydG//vUvffTRR3riiSdk/PiPwOAfAwCEFl97dVzo3UGYMC3s7N27V4888ogkKS0tTZ999pm+//57PfDAAyoqKjLrYwAAZnD16vj9YyCK3h2EPNPCTv369ZWTk+N+XadOHa1cuVJ169ZV37595XQ6zfooAIC/HEVSfo4kf6/NTqkgp/h8QIgybTZWr169lJWVpcmTJ//n5LVqacmSJXr66ae1YcMGsz4KAOCvWrHS0+ulC7n+nys+ufh8QIgyLezMnz9fV65cKbfdZrNp0aJFmjJlilkfBQAwQ/0WxV+Axfl8G2vs2LGlXteuXVtxcXGVHt+yZUtfPwoAAMBnPoedOXPm6IEHHtClS5cqPebYsWO+nh4AAPjr8HppbufiXyOYz2Fn1apV2rBhg372s5/p9OnTpfYdO3ZMTz/9tK699lq/CwQAAD4wDGntNCl3f/GvETxjzuew06tXL23atEl5eXm67bbbtHPnzlIh57333tOwYcPMrBUAAHjK9RwlKeKfh+TXAOXrrrtOX375pQYMGKA777xTV65cUXR0tJ599lmNHz9ezZo1M6tOAADgqbJrnkX4WmZ+hZ0TJ05o1qxZ2rlzpwoLC2Wz2TR79mw9++yzZtUHAAC8VbJXRyr9tOtregWvriDx+TbWU089pbS0NL311lsaPny4jh49qmHDhun555/XzJkzzawRAAB4qrI1zyJ4LTOfe3b++te/avjw4Zo4caKaN28uSVq4cKHS0tI0ceJE7d+/XwsXLlRMTIxpxQIAgGqU7dVxKdm70+rnNV9XEPkcdg4fPuwOOSWNGzdOaWlpevzxx/XNN9/w5GQAAGpK2bE6Zbl6d4Z2q/nagsjn21gVBR2X+++/Xxs2bNChQ4d8PT0AAPBWdSvZ/9i7Y/smsp67Y9pCoGV17NhRX375ZaBODwAASqpsrE5ZtmhFbZgRUWN3AhZ2JOnqq68O5OkBAIBLdb06LoZDUad2KPn87pqpKwQENOwAAIAa4GmvjutwW7SuP7k8Ynp3CDsAAIQ7T3t1fmQzHLrq0pGIGbtD2AEAIJx52avj4lRUxIzdIewAABDOvOzVcYmSU1GnImPNLL/DTmFhoTZv3qwPP/xQubm5ZtQEAAA84WOvjvvtEfJUZb/CTmZmppo1a6Y777xTDz74oHbt2iVJys3NVVJSkt5++21TigQAABXwsVfHxVbyqcoW5nPYycrK0ujRo9WvXz+99dZbMkqkwqSkJN11111aunSpKUUCAIAyXL06ft+kibJ8747Pf0J/+MMfdN9992nJkiUaOHBguf0dO3bUnj17/CoOAABUwlEk5edIcvp5IqdUkFN8PovyeW2sQ4cOadSoUZXub9iwoc6dO+fr6QEAQFVqxUpPr5cueD9e1n7lir744gt17dpVMbVqSfHJxeezKJ/DToMGDaockLx37141bdrU19MDAIDq1G9R/OUtu135cTlSs5ulmBjz6woxPt/G6t+/vxYuXKi8vLxy+/bs2aNFixbp3nvv9ac2AAAAv/kcdqZPny6Hw6H27dtr8uTJstlsevfdd/X444+rU6dOaty4sV5++WUzawUAAPCaz2GnefPm2rZtm/r166dly5bJMAy99957+vvf/65HH31UW7ZsUVJSkpm1AgAAeM2nMTsXL17Uz372Mw0fPlyLFy/W4sWLdfbsWTmdTiUnJysqigczAwCA0OBT2ImLi9ORI0dks9nc25KTk00rCgAAwCw+d8H069dPn3zyiZm1AAAAmM7nsDNlyhQdOHBAv/rVr/T5558rJydH33//fbkvAACAYPL5OTs33nijpOLn6SxZsqTS4xwO39brAAAAMIPPYefll18uNWYn2ObNm6dXX31Vp0+f1s0336w5c+aoc+fOwS4LAAAEmc9hJyMjw8Qy/LNs2TKlp6drwYIF6tKli9544w317dtX+/fvV+PGjYNdHgAACCJLzBF//fXXNXz4cD355JO64YYbtGDBAsXFxentt98OdmkAACDIfO7ZCRVFRUXatm2bJk6c6N4WFRWlXr16afPmzRW+p7CwUIWFhe7XBQUFkiS73S673R7YggPEVXe41m8ltEVooT1CB20ROqzSFp7W73PYiYqK8mjMTqAHKOfm5srhcKhJkyaltjdp0kT79u2r8D0zZszQtGnTym1fs2aN4uLiAlJnTcnOzg52CfgRbRFaaI/QQVuEjnBvi4sXL3p0nKkDlB0Oh44ePaqVK1fq2muv1T333OPr6QNq4sSJSk9Pd78uKChQSkqK+vTpo8TExCBW5ju73a7s7Gz17t1bMRGwgm0ooy1CC+0ROmiL0GGVtnDdmalOQAYonzp1SrfffrvatWvn6+k9lpSUpOjoaJ05c6bU9jNnzqhp06YVvic2NlaxsbHltsfExIR1o0vW+B6sgrYILbRH6KAtQke4t4WntQdkgHKzZs30zDPP6Pe//30gTl9K7dq11bFjR61du9a9zel0au3atbrjjjsC/vkAACC0BWyAcnx8vI4cORKo05eSnp6uIUOGqFOnTurcubPeeOMNXbhwQU8++WSNfD4AAAhdAQk7X331lTIzM2vkNpYkPfLIIzp79qxefvllnT59Wh06dNDq1avLDVoGAACRx+ewk5qaWuFsrLy8POXn5ysuLk4rV670pzavjBw5UiNHjqyxzwMAAOHB57Dz85//vFzYsdlsuuqqq9S2bVsNGjRIDRs29LtAAAAAf/gcdt555x0TywAAAAgMn2djHT9+XJcuXap0/6VLl3T8+HFfTw8AAGAKn8NOamqqVqxYUen+v/3tb0pNTfX19AAAAKbwOewYhlHlfrvdrqgoS6wzCgAAwphXY3YKCgqUl5fnfn3u3LkKb1Xl5eVp6dKlatasmd8FAgAA+MOrsDN79mz97ne/k1Q882r06NEaPXp0hccahqHp06f7XSAAAIA/vAo7ffr0Ub169WQYhsaPH69HH31Ut956a6ljbDab4uPj1bFjR3Xq1MnUYgEAALzlVdi544473OtNXbhwQQ899JDat28fkMIAAADM4PNzdqZOnWpmHQAAAAHh19pYly9f1vLly7V9+3bl5+fL6XSW2m+z2fTWW2/5VSAAAIA/fA47x44dU48ePXT06FE1aNBA+fn5atiwofLy8uRwOJSUlKR69eqZWSsAAIDXfH4Qzrhx45Sfn68tW7bowIEDMgxDy5Yt0w8//KBZs2apbt26+uSTT8ysFQAAwGs+h51169bpueeeU+fOnd0PDzQMQ7GxsRo3bpx69uxZ6bR0AACAmuJz2Ll48aJat24tSUpMTJTNZlN+fr57/x133KHPP//c7wIBAAD84XPYadmypb799ltJUq1atXT11Vdry5Yt7v179+5VnTp1/K8QAADADz4PUL7rrrv04YcfuqegDx06VDNmzNC///1vOZ1Ovffee3riiSdMKxQAAMAXPoedCRMmaOvWrSosLFRsbKxeeuklnTx5Uu+//76io6P12GOP6fXXXzezVgAAAK/5HHZatmypli1bul/XqVNHixcv1uLFi00pDAAAwAx+PVRQkgoLC7V9+3Z999136tq1q5KSksyoCwAAwBQ+D1CWpMzMTDVr1kx33nmnHnzwQe3atUuSlJubq6SkJL399tumFAkAAOArn8NOVlaWRo8erX79+umtt96SYRjufUlJSbrrrru0dOlSU4oEAADwlc9h5w9/+IPuu+8+LVmyRAMHDiy3v2PHjtqzZ49fxQEAAPjL57Bz6NAh3X333ZXub9iwoc6dO+fr6QEAAEzhc9hp0KCBcnNzK92/d+9eNW3a1NfTAwAAmMLnsNO/f38tXLhQeXl55fbt2bNHixYt0r333utPbQBgWZlrDyp1wj+UufZgsEsBLM/nsDN9+nQ5HA61b99ekydPls1m07vvvqvHH39cnTp1UuPGjfXyyy+bWSsAWELm2oN6PfuADEmvZx8g8AAB5nPYad68ubZt26Z+/fpp2bJlMgxD7733nv7+97/r0Ucf1ZYtW3jmDgCU4Qo6JRF4gMDy66GCjRs3dj81+ezZs3I6nUpOTlZUlF+P7wEAS6oo6Li4to/qmVaTJQERwatU8tJLL7kfHFhWcnKymjRpQtABgApUFXRc6OEBAsOrZDJz5kx99dVX7tfnzp1TdHS01q1bZ3phAGAVngQdFwIPYD6/u2FKPjkZAFCaN0HHhcADmIt7TgAQIL4EHRcCD2Aewg4ABIA/QceFwAOYw+vZWEePHtX27dslSfn5+ZKkgwcPqkGDBhUef+utt/peHQCEITOCjguztAD/eR12pkyZoilTppTa9txzz5U7zjAM2Ww2ORwO36sDgDA026SgU/J8hB3Ad16FnaysrEDVAQCWMaZ3O9N6dlznA+A7r8LOkCFDAlUHAFiGqxfGjMCT3rsdvTqAnxigDAABMKpnmtL97JEh6ADm8Gu5CACoTObag5qdfUB3tG2kzYfPaUwE/uD2p4eHoAOYh7ADwHQlZyNtOnxOUuTOKvIl8BB0AHNxGwuAqapb7DISnxvjzS0tgg5gPsIOANOw2GXlPAk8BB0gMAg7AEzBYpfVqyrwEHSAwCHsAPAbi116rqLAQ9ABAosBygD84u9il1LkDlqenX0gImepATWNsAPAZ2YtdilFZuCJtO8ZCBZuYwHwidmLXUbiLS0ANYOwA8BrZgYdFwIPgEAh7ADwmtmregf6vAAiG2EHgNcCtQo3q3sDCATCDgCvmbHIZVlMvwYQKIQdAD4xM/AQdAAEEmEHgM/MCDwEHQCBRtgB4Bd/Ak9lQSdz7UGlTvgHs7MAmIKwA8BvvgSeqoLO69kHZMj36eiEJQAlEXYAmMKbwFNd0CnJ28BTNizNXX/Y4/d6cm5CFBB+CDsATONJ4PEm6Lh4GngqOscf1x3WJ9/aqn2vp+f2p8cJQHAQdgCYqqrA40vQcakuYFR1jlUnov3q4TGjxwlA8BB2AJjOFXhskn7atpFs8i/ouFQWMDw5xx/XHfZ5/I+/PU4AgotVzwEEhCerevuyxlbZVdK9DUsl31sdT3ucvDkngJpH2AEQFP4sJlryff6GpcoEMkQBqFmEHQA1zoxV0/15f3XhxIweJwChI6zH7Bw9elTDhg1Tamqq6tatq7Zt22rq1KkqKioKdmkAKmFG0DGDP+N/vD0ngOAK656dffv2yel06s0339Q111yjr776SsOHD9eFCxf02muvBbs8ABWYHQJBx8Wf8T+enhNA8IV1z06/fv2UlZWlPn36qE2bNrr33nv129/+Vh988EGwSwNQiTEmr5buL1f4MrPHiR4eILSEdc9ORfLz89WwYcMqjyksLFRhYaH7dUFBgSTJbrfLbrcHtL5AcdUdrvVbCW1RtWe7tZbD4dAf15n3ZGN/jLqrrex2u+k9TrOzD+jZbq1NPWe4499G6LBKW3hav80wDCPAtdSYQ4cOqWPHjnrttdc0fPjwSo/LyMjQtGnTym1fsmSJ4uLiAlkigB998q1Nq05EB7WG/ikO9W1hBKSekucGEBgXL17UY489pvz8fCUmJlZ6XEiGnQkTJmjWrFlVHvP111/ruuuuc7/OycnRz3/+c3Xv3l2LFy+u8r0V9eykpKQoNze3yj+sUGa325Wdna3evXsrJiYm2OVENNrCc3PXH/aph+eFu9pKkl+9Qy/c1VYje7Q1pR5Pzg3+bYQSq7RFQUGBkpKSqg07IXkba+zYsRo6dGiVx7Rp08b9+5MnT6pHjx766U9/qoULF1Z7/tjYWMXGxpbbHhMTE9aNLlnje7AK2qJ6Y/pcp+joaK/GypR8ErO3763oHP7W4+m58R/82wgd4d4WntYekmEnOTlZycnJHh2bk5OjHj16qGPHjsrKylJUVFiPuQYijisYeBIwygYJb95b2Tn8qcfbcwMIjpAMO57KyclR9+7d1apVK7322ms6e/ase1/Tpk2DWBkAb3gSMCoLEv6EJX/q8fXcAGpeWIed7OxsHTp0SIcOHVKLFi1K7QvBoUgAqlBVwDCjN+aFu9p6FUYCEaIABEdY3/MZOnSoDMOo8AtA+HGtll6SN70xZd/r0j/F4dOA4arO6W19AIInrHt2AFiPKzjMzj6gMV4GiYp6Y164q63aXNrvdz2+9DgBCA2EHQAhZ1TPNJ9DRNmw9Gy31lq1yvewU/KcJQMPQQcIH4QdAJZTMiyZ9YRYf3qcAAQXYQcAPORPjxOA4AnrAcoAAADVIewAAABLI+wAAABLI+wAAABLI+wAAABLI+wAAABLI+wACJrMtQeVOuEfylx7MNilALAwnrMDICgy1x50P5HY9SvPsAEQCPTsAKhxJYOOy+vZB+jhARAQhB0ANaqioONC4AEQCIQdADWmqqDjQuABYDbCDoAa4UnQcSHwADATYQdAwHkTdFwIPADMQtgBEFC+BB0XAg8AMxB2AASMP0HHhcADwF+EHQABYUbQcSHwAPAHYQdAQMw2KegE6nwAIgdhB0BAjOndLqTPByByEHYABMSonmlKNymgpPduF1FLSbBmGGAu1sYCEDCugOLP2J1IDDqsGQaYi54dAAHlTw9PJAcdFwZnA/4j7AAIOF8CD0HnPwg8gH8IOwBqhDeBh6BTHoEH8B1hB0CN8STwEHQqR+ABfEPYAVCjqgo8BJ3qEXgA7xF2ANS4igIPQcdzBB7AO0w9BxAUrmAzO/uAxhB0vMa0dMBzhB0AQTOqZ1rE/bA2e80wicADVIfbWABQg1gzDKh5hB0AqEGsGQbUPMIOANQg1gwDah5hBwBqmBmBh6ADeI6wA8Aywmm1cNYMA2oOs7EAWEI4rhbuy6rwBB3Ae4QdAGGvstXCJenZbq2DUJHnvAk8BB3AN4QdAGGtutXCHQ6H2tRwTd7yJPAQdADfMWYHQNjy5AF9f1x3WJ98a6uhinzHmmFA4BB2AIQlb55EvOpEtOauPxzgivzHmmFAYBB2AIQdX5Zc+OO6w2E1S8smgg5gFsbsAAgr/q4WLoXHLK1QrxEIJ/TsAAgbZq0WHg49PADMQ9gBEBbMXi2cwANEDsIOgLDAauEAfEXYARAWWC0cgK8IOwDCAquFA/AVYQdA2GC1cAC+IOwACCusFg7AW4QdAGHHl8Dzwl1tCTpAhCLsAAhL3gSe/ikOjezRNsAVAQhVPEEZQNjyZLXwF+5qqzaX9tdUSQBCED07AMJadauF06MDgLADIOyxWjiAqnAbC4AluILN7OwDGkPQAVACYQeAZbBaOICKcBsLAABYGmEHAABYGmEHAABYGmEHAABYGmEHAABYmmXCTmFhoTp06CCbzaadO3cGuxwAABAiLBN2xo8fr+bNmwe7DAAAEGIsEXY+/vhjrVmzRq+99lqwSwEAACEm7B8qeObMGQ0fPlwrV65UXFycR+8pLCxUYWGh+3VBQYEkyW63y263B6TOQHPVHa71WwltEVpoj9BBW4QOq7SFp/XbDMMwAlxLwBiGof79+6tr166aPHmyjh49qtTUVO3YsUMdOnSo9H0ZGRmaNm1aue1LlizxODABAIDgunjxoh577DHl5+crMTGx0uNCMuxMmDBBs2bNqvKYr7/+WmvWrNH//u//asOGDYqOjvY47FTUs5OSkqLc3Nwq/7BCmd1uV3Z2tnr37q2YmJhglxPRaIvQQnuEDtoidFilLQoKCpSUlFRt2AnJ21hjx47V0KFDqzymTZs2WrdunTZv3qzY2NhS+zp16qTBgwfr3XffrfC9sbGx5d4jSTExMWHd6JI1vgeroC1CC+0ROmiL0BHubeFp7SEZdpKTk5WcnFztcZmZmZo+fbr79cmTJ9W3b18tW7ZMXbp0CWSJAAAgTIRk2PFUy5YtS72uV6+eJKlt27Zq0aJFMEoCAAAhxhJTzwEAACoT1j07ZbVu3VohON4aAAAEET07AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0gg7AADA0moFu4BQYBiGJKmgoCDIlfjObrfr4sWLKigoUExMTLDLiWi0RWihPUIHbRE6rNIWrp/brp/jlSHsSDp//rwkKSUlJciVAAAAb50/f17169evdL/NqC4ORQCn06mTJ08qISFBNpst2OX4pKCgQCkpKTpx4oQSExODXU5Eoy1CC+0ROmiL0GGVtjAMQ+fPn1fz5s0VFVX5yBx6diRFRUWpRYsWwS7DFImJiWH9F9dKaIvQQnuEDtoidFihLarq0XFhgDIAALA0wg4AALA0wo5FxMbGaurUqYqNjQ12KRGPtggttEfooC1CR6S1BQOUAQCApdGzAwAALI2wAwAALI2wAwAALI2wAwAALI2wY3GFhYXq0KGDbDabdu7cGexyIs7Ro0c1bNgwpaamqm7dumrbtq2mTp2qoqKiYJcWEebNm6fWrVurTp066tKli7788stglxRxZsyYodtuu00JCQlq3Lix7r//fu3fvz/YZUHSzJkzZbPZNHr06GCXEnCEHYsbP368mjdvHuwyIta+ffvkdDr15ptvas+ePZo9e7YWLFigl156KdilWd6yZcuUnp6uqVOnavv27br55pvVt29ffffdd8EuLaJs2LBBI0aM0JYtW5SdnS273a4+ffrowoULwS4tom3dulVvvvmmbrrppmCXUiOYem5hH3/8sdLT07V8+XLdeOON2rFjhzp06BDssiLeq6++qvnz5+ubb74JdimW1qVLF912222aO3eupOI18FJSUvT8889rwoQJQa4ucp09e1aNGzfWhg0b1K1bt2CXE5F++OEH3XrrrfrTn/6k6dOnq0OHDnrjjTeCXVZA0bNjUWfOnNHw4cP13nvvKS4uLtjloIT8/Hw1bNgw2GVYWlFRkbZt26ZevXq5t0VFRalXr17avHlzECtDfn6+JPFvIIhGjBihAQMGlPr3YXUsBGpBhmFo6NCheuaZZ9SpUycdPXo02CXhR4cOHdKcOXP02muvBbsUS8vNzZXD4VCTJk1KbW/SpIn27dsXpKrgdDo1evRode3aVe3btw92ORFp6dKl2r59u7Zu3RrsUmoUPTthZMKECbLZbFV+7du3T3PmzNH58+c1ceLEYJdsWZ62RUk5OTnq16+fHn74YQ0fPjxIlQPBM2LECH311VdaunRpsEuJSCdOnNALL7ygv/71r6pTp06wy6lRjNkJI2fPntW5c+eqPKZNmzb65S9/qb///e+y2Wzu7Q6HQ9HR0Ro8eLDefffdQJdqeZ62Re3atSVJJ0+eVPfu3XX77bfrnXfeUVQU/88IpKKiIsXFxen999/X/fff794+ZMgQ5eXl6cMPPwxecRFq5MiR+vDDD7Vx40alpqYGu5yItHLlSj3wwAOKjo52b3M4HLLZbIqKilJhYWGpfVZC2LGg48ePq6CgwP365MmT6tu3r95//3116dJFLVq0CGJ1kScnJ0c9evRQx44d9Ze//MWyF5NQ06VLF3Xu3Flz5syRVHwLpWXLlho5ciQDlGuQYRh6/vnntWLFCn322WdKS0sLdkkR6/z58zp27FipbU8++aSuu+46vfjii5a+tciYHQtq2bJlqdf16tWTJLVt25agU8NycnLUvXt3tWrVSq+99prOnj3r3te0adMgVmZ96enpGjJkiDp16qTOnTvrjTfe0IULF/Tkk08Gu7SIMmLECC1ZskQffvihEhISdPr0aUlS/fr1Vbdu3SBXF1kSEhLKBZr4+Hg1atTI0kFHIuwAAZWdna1Dhw7p0KFD5YImnaqB9cgjj+js2bN6+eWXdfr0aXXo0EGrV68uN2gZgTV//nxJUvfu3Uttz8rK0tChQ2u+IEQkbmMBAABLY5QkAACwNMIOAACwNMIOAACwNMIOAACwNMIOAACwNMIOAACwNMIOAACwNMIOAACwNMIOAACwNMIOAACwNMIOAJ+98847stlsFX4FamXxTZs2KSMjQ3l5eQE5PwDrYSFQAH773e9+p9TU1FLbArWK8qZNmzRt2jQNHTpUDRo0CMhnALAWwg4Av919993q1KlTsMvw24ULFxQfHx/sMgCYjNtYAAIuJydHv/71r9WkSRPFxsbqxhtv1Ntvv13qmGPHjum5557Ttddeq7p166pRo0Z6+OGHdfToUfcxGRkZGjdunCQpNTXVfcvMdczQoUPVunXrcp+fkZEhm81W4ba9e/fqscce01VXXaU777zT43qr+l7r1KmjX//616W2f/rpp4qJidGYMWM8Og8A89CzA8Bv+fn5ys3NLbUtKSlJknTmzBndfvvtstlsGjlypJKTk/Xxxx9r2LBhKigo0OjRoyVJW7du1aZNmzRo0CC1aNFCR48e1fz589W9e3ft3btXcXFxevDBB3XgwAH9z//8j2bPnu3+jOTkZJ9rf/jhh5WWlqb/+q//kmEYHtdbmauvvlpPPfWUFi5cqKlTp6pVq1bat2+fHn74Yd199936wx/+4HOtAHxkAICPsrKyDEkVfrkMGzbMaNasmZGbm1vqvYMGDTLq169vXLx40TAMw/1rSZs3bzYkGX/+85/d21599VVDknHkyJFyxw8ZMsRo1apVue1Tp041yl7uXNseffTRUts9rbcq3377rREbG2s8++yzRm5urtG2bVujQ4cOxg8//FDtewGYj9tYAPw2b948ZWdnl/qSJMMwtHz5cg0cOFCGYSg3N9f91bdvX+Xn52v79u2SpLp167rPZ7fbde7cOV1zzTVq0KCB+5hAeOaZZ9y/96beqlx99dUaPny43n77bQ0YMECXLl3SRx99xHggIEi4jQXAb507d65wgPLZs2eVl5enhQsXauHChRW+97vvvpMkXbp0STNmzFBWVpZycnJkGIb7mPz8/MAULpWaReZNvdX57W9/q7lz52rXrl36f//v/+nqq68utX/+/PlatGiRdu/erUmTJikjI8Pn7wFA1Qg7AALG6XRKkh5//HENGTKkwmNuuukmSdLzzz+vrKwsjR49WnfccYfq168vm82mQYMGuc9TnbKDkF0cDkel7ynZo+RNvdV55ZVXJElXrlxRw4YNy+1v1qyZMjIytGTJEo/OB8B3hB0AAZOcnKyEhAQ5HA716tWrymPff/99DRkypNQA3suXL5d7eGBlgUaSrrrqqgofNnjs2DHT663Kq6++qsWLF2vu3LkaN26cXnnlFS1evLjUMffff78kadWqVT5/DgDPMGYHQMBER0froYce0vLly/XVV1+V23/27NlSx5a8dSVJc+bMKdcr4xr3UlGoadu2rfLz87Vr1y73tlOnTmnFihWm11uZlStXasKECfr973+vESNG6Omnn9af//xnHTlyxKMaAJiPnh0AATVz5kytX79eXbp00fDhw3XDDTfo+++/1/bt2/Xpp5/q+++/lyTdc889eu+991S/fn3dcMMN2rx5sz799FM1atSo1Pk6duwoSZo0aZIGDRqkmJgYDRw4UPHx8Ro0aJBefPFFPfDAAxo1apQuXryo+fPnq127dh4Pcva03ops27ZNgwcP1uDBgzVp0iRJ0vjx47VgwYIKe3cA1AzCDoCAatKkib788kv97ne/0wcffKA//elPatSokW688UbNmjXLfdwf//hHRUdH669//asuX76srl276tNPP1Xfvn1Lne+2227T73//ey1YsECrV6+W0+nUkSNHFB8fr0aNGmnFihVKT0/X+PHjlZqaqhkzZujgwYMehx1P6y3r22+/1cCBA3XLLbdo0aJF7u3NmzfXr3/9ay1evFiTJk0qt6wGgMCzGWX7jQEANeaZZ55R06ZNmY0FBBBjdgAgCK5cuaLLly/L4XCU+j0A89GzAwBBkJGRoWnTppXalpWVpaFDhwanIMDCCDsAAMDSuI0FAAAsjbADAAAsjbADAAAsjbADAAAsjbADAAAsjbADAAAsjbADAAAsjbADAAAsjbADAAAsjbADAAAs7f8DZv+wmQk1sIoAAAAASUVORK5CYII=",
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.plot(\n",
" X_train[y_train == 0, 0],\n",
" X_train[y_train == 0, 1],\n",
" marker=\"D\",\n",
" markersize=10,\n",
" linestyle=\"\",\n",
" label=\"Class 0\",\n",
")\n",
"\n",
"plt.plot(\n",
" X_train[y_train == 1, 0],\n",
" X_train[y_train == 1, 1],\n",
" marker=\"^\",\n",
" markersize=13,\n",
" linestyle=\"\",\n",
" label=\"Class 1\",\n",
")\n",
"\n",
"plt.legend(loc=2)\n",
"\n",
"plt.xlim([-5, 5])\n",
"plt.ylim([-5, 5])\n",
"\n",
"plt.xlabel(\"Feature $x_1$\", fontsize=12)\n",
"plt.ylabel(\"Feature $x_2$\", fontsize=12)\n",
"\n",
"plt.grid()\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "56e353d0-f75d-4267-8b41-68433780d590",
"metadata": {},
"outputs": [],
"source": [
"X_train = (X_train - X_train.mean(axis=0)) / X_train.std(axis=0)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "295f3b29-2b8e-4280-8f25-92b4429002d2",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAG2CAYAAACZEEfAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA7zklEQVR4nO3deXxU1f3/8fckhEBCAkJYJWjA4EYVBUG/WAsS9uJGbVGsoohFWQpBEEQhVPyiX1EsS0GggLXyhe9DBFtFJAJCLVD5gVQE2VcJIAFJWJMhub8/4kyzZzJzJzNz5vV8PPIIc++dM5+Zg8zbc8+9x2FZliUAAABDRQS6AAAAAH8i7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMBphBwAAGK1aoAsIBvn5+crIyFBcXJwcDkegywEAAB6wLEvnzp1TkyZNFBFR9vgNYUdSRkaGEhMTA10GAADwwtGjR9W0adMy9xN2JMXFxUkq+LDi4+MDXI13nE6nVq1apa5duyoqKirQ5YQ1+iK40B/Bg74IHqb0RXZ2thITE93f42Uh7EjuU1fx8fEhHXZiYmIUHx8f0n9xTUBfBBf6I3jQF8HDtL6oaAoKE5QBAIDRCDsAAMBohB0AAGA05ux4KD8/X7m5uYEuo0xOp1PVqlXT5cuXlZeXF+hy/KJ69erlXloIAEBpCDseyM3N1cGDB5Wfnx/oUspkWZYaNWqko0ePGnuvoIiICCUlJal69eqBLgUAEEIIOxWwLEvHjx9XZGSkEhMTg3ZkIT8/X+fPn1etWrWCtkZfuG78ePz4cTVr1szYQAcAsB9hpwJXrlzRxYsX1aRJE8XExPjU1rTVezU1fY9GdGmpYZ2TbaqwgOs0W40aNYwMO5JUv359ZWRk6MqVK0ZcKgkAqBpmfivayDX/xddTJ9NW79Vb6XtkSXorfY+mrd5rQ3XhxdUHps5JAgD4B2HHQ76cNnEFncIIPJXHqSsAgDcIO35WWtBxIfAAAOB/hB0/Ki/ouBB4AADwL8KOn3gSdFwCGXgcDoeWL18ekNcGAKAqEHb8oDJBx8UfgefEiRMaOnSomjdvrujoaCUmJqp3795avXq1ra/jLcuyNH78eDVu3Fg1a9ZUSkqK9u5llAsAYC/Cjs28CToudgaeQ4cOqU2bNlqzZo3eeOMNbd++XStXrlSnTp00ePBgW17DV//zP/+jadOmafbs2frXv/6l2NhYdevWTZcvXw50aQAAgxB2bORL0HGxK/A899xzcjgc+uqrr9SnTx+1bNlSN998s1JTU7Vp06Yyn/fCCy+oZcuWiomJUfPmzfXyyy/L6XS69//73/9Wp06dFBcXp/j4eLVp00b/7//9P0nS4cOH1bt3b1111VWKjY3VzTffrBUrVpT6OpZl6e2339ZLL72k+++/X7fccov+8pe/KCMjg9NqAABbcVNBm9gRdFxc7Xh748EzZ85o5cqVevXVVxUbG1tif506dcp8blxcnBYuXKgmTZpo+/btGjhwoOLi4jR69GhJUr9+/XTbbbdp1qxZioyM1LZt29w3+Bs8eLByc3O1fv16xcbGaufOnapVq1apr3Pw4EGdOHFCKSkp7m21a9dW+/bttXHjRvXt29er9w4AQHGEHZtMtSnoFG7P27Czb98+WZalG264odLPfemll9x/vvbaa/X8889r8eLF7rBz5MgRjRo1yt12cvJ/ajxy5Ij69Omjn/3sZ5Kk5s2bl/k6J06ckCQ1bNiwyPaGDRu69wEAYAdOY9lkRJeWQdOeZVleP3fJkiXq0KGDGjVqpFq1aumll17SkSNH3PtTU1P19NNPKyUlRa+99pr279/v3jds2DBNmjRJHTp00IQJE/TNN994XQcAAHYh7NhkWOdkpdoUeFJ9XDsrOTlZDodDu3btqtTzNm7cqH79+qlnz576+OOP9fXXX2vcuHHKzc11H5OWlqYdO3aoV69eWrNmjW666SYtW7ZMkvT000/rwIED+u1vf6vt27erbdu2mj59eqmv1ahRI0nSyZMni2w/efKkex8AAHYg7NjIjsDja9CRpLp166pbt26aOXOmLly4UGL/2bNnS33ehg0bdM0112jcuHFq27atkpOTdfjw4RLHtWzZUiNGjNCqVav00EMPacGCBe59iYmJGjRokD788EONHDlSc+fOLfW1kpKS1KhRoyKXwWdnZ+tf//qX7rrrrkq+YwAAykbYsZkvgceOoOMyc+ZM5eXlqV27dlq6dKn27t2r7777TtOmTSszTCQnJ+vIkSNavHix9u/fr2nTprlHbSTp0qVLGjJkiL744gsdPnxY//znP7V582bdeOONkqThw4frs88+08GDB7V161atXbvWva84h8Oh4cOHa9KkSfrb3/6m7du36/HHH1eTJk30wAMP2PIZAAAgMUHZL1yBpTJXZ9kZdKSCycFbt27Vq6++qpEjR+r48eOqX7++2rRpo1mzZpX6nPvuu08jRozQkCFDlJOTo169eunll19WWlqaJCkyMlKnT5/W448/rpMnTyohIUEPPfSQJk6cKKlgNfLBgwfr+++/V3x8vLp3766pU6eWWePo0aN14cIFPfPMMzp79qzuvvturVy5UjVq1LDtcwAAwGH5MpvVENnZ2apdu7aysrIUHx9fZN/ly5d18OBBJSUlVfpL2NPL0e0IOvn5+crOzlZ8fLwiIswcsPOlL6qS0+nUihUr1LNnT/dl+Qgc+iN40BfBw5S+KO/7uzAzvxWDhCentOwe0QEAAEURdvysvMBD0AEAwP8IO1WgtMBD0AEAoGowQbmKuILN1PQ9GkHQAQCgyhB2qtCwzsmEHAAAqhinsarS/rXSjHYFvwEAQJUg7FQVy5JWT5Qydxf85op/AACqhHFh57XXXnPfnTeo7F8tZXxd8OeMrwseAwAAvzMq7GzevFnvvPOObrnllkCXUpRlSWsmSY7IgseOyILHQTC643A4tHz58kCXAQCA3xgTds6fP69+/fpp7ty5uuqqqwJdTlGuUR0rr+CxlVclozsnTpzQ0KFD1bx5c0VHRysxMVG9e/cusvhmIH344Yfq2rWr6tWrJ4fDoW3btgW6JACAgYy5Gmvw4MHq1auXUlJSNGnSpHKPzcnJUU5Ojvtxdna2pILbZzudziLHOp1OWZal/Px85efnV74wy5Ljp1EdhyvsSLJ+Gt2xkjpJDkfl2y3xMpb7d35+vg4dOqSf//znqlOnjl5//XX97Gc/k9Pp1KpVqzR48GDt3LnT/Vyv35uPzp07pw4dOuhXv/qVfve731VYR35+vizLktPpVGRkZBVWWjmuv0PF/y4hMOiP4EFfBA9T+sLT+o0IO4sXL9bWrVu1efNmj46fPHmye/HKwlatWqWYmJgi26pVq6ZGjRrp/Pnzys3NrXRt1Q6tUy3XXJ1CHD+N7lzY/rGuXPuLSrdblnPnzkmSfve730kqeE+xsbHu/QMGDNCvfvUrd8CTClYzdz2eMGGCPvnkE2VkZKhBgwZ6+OGHNXr0aPfaKdu3b9eLL76obdu2yeFwqHnz5po6dapuu+02HTlyRKNHj9amTZvkdDrVrFkzTZw4UV27di211vvvv1+SdOTIEUnShQsXitRVXG5uri5duqT169frypUr3n5EVSY9PT3QJaAQ+iN40BfBI9T74uLFix4dF/Jh5+jRo/r973+v9PR0jxeHHDt2rFJTU92Ps7OzlZiYqK5du5a6EOjRo0dVq1atyi8+aVlyfDVVVrFRHfduR6Riv5oq62e/9Hl0x7IsnTt3TnFxcfrxxx+1evVqTZo0SY0bNy5xbPH3WLNmTfe2hIQELVy4UE2aNNH27dv1u9/9TgkJCRo1apQk6dlnn1Xr1q31zjvvKDIyUtu2bVOdOnUUHx+vsWPHKi8vT+vWrVNsbKx27typ+Pj4chdnk6RatWpJkmJjY8s99vLly6pZs6buueeeoF8IND09XV26dAnpBfZMQX8ED/oieJjSF+X9D3JhIR92tmzZoh9++EG33367e1teXp7Wr1+vGTNmKCcnp8Qpj+joaEVHR5doKyoqqkSn5+XlyeFwKCIiovKrie/7/D9XYJXCNbrjOLhWui6lcm0X4zr943A4dODAAVmWpRtvvNGjmgu/t5dfftm9vXnz5tq7d68WL16sF154QVLBKMyoUaN00003SZKuv/569/FHjx5Vnz59dOutt0qSrrvuOo9qd712RZ9xRESEHA5Hqf0UjEKlznBBfwQP+iJ4hHpfeFp7yIedzp07a/v27UW2Pfnkk7rhhhv0wgsvBG5uR+ErsEoZ1XFzXZnVorMtc3cKXtr7q7yWLFmiadOmaf/+/Tp//ryuXLlSZLQlNTVVTz/9tN577z2lpKTo4YcfVosWLSRJw4YN07PPPqtVq1YpJSVFffr0Cb4r4wAAYSfkr8aKi4tTq1ativzExsaqXr16atWqVeAKK34FVln8cGVWcnKyHA6Hdu3aVannbdy4Uf369VPPnj318ccf6+uvv9a4ceOKzFVKS0vTjh071KtXL61Zs0Y33XSTli1bJkl6+umndeDAAf32t7/V9u3b1bZtW02fPt229wUAgDdCPuwEpeL31amIzffdqVu3rrp166aZM2fqwoULJfafPXu21Odt2LBB11xzjcaNG6e2bdsqOTlZhw8fLnFcy5YtNWLECK1atUoPPfSQFixY4N6XmJioQYMG6cMPP9TIkSM1d+5cW94TAADeCvnTWKX54osvAltA4bsle6Lw6I6Pc3dcZs6cqQ4dOqhdu3b6wx/+oFtuuUVXrlxRenq6Zs2ape+++67Ec5KTk3XkyBEtXrxYd9xxhz755BP3qI1UcNXWqFGj9Ktf/UpJSUn6/vvvtXnzZvXp00eSNHz4cPXo0UMtW7bUjz/+qLVr1+rGG28ss8YzZ87oyJEjysjIkCTt3r1bktSoUSM1atTIls8BAABGduxW2VEdF5tHd5o3b66tW7eqU6dOGjlypFq1aqUuXbpo9erVmjVrVqnPue+++zRixAgNGTJErVu31oYNG4pMWI6MjNTp06f1+OOPq2XLlvr1r3+tHj16uC/jz8vL0+DBg3XjjTeqe/fuatmypf70pz+VWePf/vY33XbbberVq5ckqW/fvrrttts0e/ZsWz4DAAAkyWH5MpvVENnZ2apdu7aysrJKvfT84MGDSkpK8uxy532fS3/t430xjy31anQnPz9f2dnZio+Pr/xVYyGi0n0RIE6nUytWrFDPnj1D+ioHU9AfwYO+CB6m9EV539+FmfmtGCjejuq4BNGaWQAAmIKwYydPr8AqSxWtmQUAQDgh7NjFNarj80cawegOAAA2IuzYJS9XyjomydcFNfOl7GMF7QEAAJ8Zeem5P1Q4j7tatPTMWulCpu8vFlu/oD0UwVx6AIA3CDsVcC03kZubq5o1a5Z/cO2mBT/wC9ednAO2BAgAICQRdipQrVo1xcTE6NSpU4qKigray7rz8/OVm5ury5cvB22NvsjPz9epU6cUExOjatX4awsA8BzfGhVwOBxq3LixDh48WOrSCcHCsixdunRJNWvWlMOmBUWDTUREhJo1a2bs+wMA+AdhxwPVq1dXcnJykQUxg43T6dT69et1zz33hPQNospTvXp1I0etAAD+RdjxUERERFDftTcyMlJXrlxRjRo1jA07AAB4g/9NBgAARiPsAAAAoxF2AACA0Qg7AADAaIQdAABgNMIOAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQcAABiNsAMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMBphBwAAGI2wAwAAjEbYAQAARiPsAAAAoxF2AACA0Qg7AADAaIQdAABgNMIOAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQcAABiNsAMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMBphBwAAGI2wAwAAjEbYAQAARiPsAAAAoxF2AACA0UI+7EyePFl33HGH4uLi1KBBAz3wwAPavXt3oMsCAABBIuTDzrp16zR48GBt2rRJ6enpcjqd6tq1qy5cuBDo0gAAQBCoFugCfLVy5coijxcuXKgGDRpoy5YtuueeewJUFQAACBYhH3aKy8rKkiTVrVu3zGNycnKUk5PjfpydnS1Jcjqdcjqd/i3QT1x1h2r9JqEvggv9ETzoi+BhSl94Wr/DsizLz7VUmfz8fN133306e/asvvzyyzKPS0tL08SJE0tsX7RokWJiYvxZIgAAsMnFixf16KOPKisrS/Hx8WUeZ1TYefbZZ/Xpp5/qyy+/VNOmTcs8rrSRncTERGVmZpb7YQUzp9Op9PR0denSRVFRUYEuJ6zRF8GF/gge9EXwMKUvsrOzlZCQUGHYMeY01pAhQ/Txxx9r/fr15QYdSYqOjlZ0dHSJ7VFRUSHd6ZIZ78EU9EVwoT+CB30RPEK9LzytPeTDjmVZGjp0qJYtW6YvvvhCSUlJgS4JAAAEkZAPO4MHD9aiRYv00UcfKS4uTidOnJAk1a5dWzVr1gxwdQAAINBC/j47s2bNUlZWljp27KjGjRu7f5YsWRLo0gAAQBAI+ZEdg+ZXAwAAPwj5kR0AAIDyEHYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQdAeNm/VprRruA3gLBA2AEQPixLWj1Rytxd8Lsyt64gJAEhi7ADIHzsXy1lfF3w54yvCx57wpeQVGYthCegqhB2AIQHy5LWTJIckQWPHZEFjz0JLt6GpPJqsTs8ASgTYQdAWHAcWFsQVKy8gg1WnmfBxZeQVBa7wxOAchF2AJjPshSxbvJ/AouLJ8HFFUwqG5LKqcX28ASgXIQdAMarf267Io4XCiwuFQWX4sHExZeAYnd4AlAhwg4As1mWbsxYKqt4YHEpL7gUDybuNr0MKP4ITwAqRNgBYDTHgbW66tJBOYoHFpeygktZwcTdsBcBxe7wBMAjhB0A5vpprk5+Rf/UlRZcygom7rYrGVD8EZ4AeISwA8Bc+1cr4vjXilB++ccVDy4VBRMXby5ftys8AfAYYQeAmX4KLGXO1SmucHCpKJi4X8PLy9c9qQGAbQg7AMz0U2Apc65Oca7gsu9zz4KJizeXr1dUA6M7gK0IOwDM4+lISnGOSOnTFzwLJu7X8vLy9fJqYHQHsBVhB4B5PB1JKc7Kk87slxyV/KfRm8vXy6uB0R3AVoQdAGbxdlSnSBsVTGgucbyXl6+XhdEdwFaEHQBm8XZUx1feXL5eFkZ3AFsRdgCYwzWSEoh/2ry9fL0sjgjpfx+V9q2xr0YgTBF2AJgjL1fKOiZVdF8dv4mo/OXrZbHypbwcacXznM4CfFQt0AUAgG2qRUvPrJV2rZA+HVVyf483pMR2Jbfn5UqLfi1d+tHHAvKl7GPSlZxCI0w+Bq8z+wsuh0/u4mNtQPgi7AAwS/zV0r8XFZw+KjSqYjki5fj3IqndQMnhKPm8QV9KFzJ9f/3Y+gXt2znCtOYV6bqU0usGUCFbw87Fixe1e/duXXfddYqLiyuy75///Kc6dOhg58sBQEmu00fFOArPqbkupeTzajct+KnUa60tuC9Pj9elFp2K7ntmrXfh6ehXJUeljv+77LoBVMi2sLNp0yb17t1b1atX148//qgXX3xRL730knt/jx49lJ2dbdfLAUBJhScFlzZXxnXFVIvOvo+SWJa0eqKUubvgd/OORdv0JjxZlvTx8JL121k3EIZsm6CcmpqqGTNm6NixY/r3v/+tjz/+WI8//risnybWWUywA+BvVbnYZuERJLvbLF4/l6IDPrEt7OzcuVO/+c1vJEnJycn64osvdObMGT344IPKzc2162UAoHRVudhm8dfyR5vFcaNBwGu2hZ3atWvr2LFj7sc1atTQ8uXLVbNmTXXr1k35+YG6FBRAWKjKxTaLv5Y/2iyO0R3Aa7aFnZSUFC1YsKDItmrVqmnRokW67rrrdOnSJbteCgCKqsrFNst6LX+0WRyjO4BXbAs7s2bNUmpqaontDodDc+fO1aFDh+x6KQAoqioX2/THvJqqHJUCwpDXYWfkyJFFHlevXl0xMTFlHt+sWTNvXwoAylaVi236Y15NVY5KAWHK67Azffp0Pfjgg+Wenjp8+LC3zQOAZ6pysU1P59V8+ZZ9bZb1GozuAB7zOuysWLFC69at089//nOdOHGiyL7Dhw/rmWee0fXXX+9zgQBQJp8X26zEKEllXmv9G5InF2VU5agUEMa8DjspKSnasGGDzp49qzvuuEPbtm0rEnLee+89DRgwwM5aAaAonxfbrMQoSWVey3lJ+vJNe9ssjNEdoFJ8mqB8ww036KuvvlLTpk1199136/rrr9f777+vZ599VgcOHNDMmTPtqhMAinKNivh8nUVExaMk3ozA/OPN8kd3qnJUCghzPv0rcfToUY0fP17btm3TxYsX5XQ6NWXKFE2dOlWNGze2q0YAKCkv16bFNn9aqTyvnJufejMCU9HoTlWOSgFhzuu1sZ5++mn99a9/lcPh0MCBA/X8889r0qRJGjp0qLKysjRmzBg76wSAoqpFe7TYpvPKFfdCxFHVyvgnL7Z+QXulqWi9rfL8403p7pFSRLH/rywyKuVLWItgzSzAA16Hnffff18DBw7U2LFj1aRJE0nSnDlzlJycrLFjx2r37t2aM2eOoqKibCsWAIrwZLFNp1NZMcekxrdK3vx7VMYq6h5xje7cU2wVc3+MSpUV1gB4H3b279/vDjmFjRo1SsnJyXrsscd04MABrVu3zqcCAQTGtNV7NTV9j0Z0aalhnZMDXU5g+DKq41La6I6Ho1IeKW9UCoAkH8JOaUHH5YEHHtC6det03333eds8gACatnqv3krfI0nu32EZeHwZ1XEpa3THk1EpALawbbmI4tq0aaOvvvrKX80D8JPCQcflrfQ9mrZ6b4AqChDbrvZSxVdmAfArv4UdSbr66qv92TwAm5UWdFzCLvDYNq9GBaM7ez/zvR0AXvH6NBYAs5QXdFzC6pRWefNqrHxpfncpL8ezthwR0rrXpZbdi141tX+t9OkLUo/XpRad7KkbQAmEHQAeBR2XsAo8Zc2rWf+G50FHKghHrnviXJfy0zZLWj1Rytxd8Lt5Ry4fB/zEr6exAAS/ygQdl7A7pVVYfn7BHJzKKn7H48KTn7k5IOBXhB0gjHkTdFzCNvB8+WbBHJzKKnzH4+JLRbD0A+BXPp/GysnJ0datW/XDDz+oQ4cOSkhIsKMuAH7mS9BxCatTWpL3ozouhUNN4UvaCwch12kuALbxaWRn2rRpaty4se6++2499NBD+uabbyRJmZmZSkhI0Pz5820pEoC97Ag6LmE1wuPtqI6LK9R8+kLJBUAZ3QH8xuuws2DBAg0fPlzdu3fXn//8Z1mF/gNNSEjQvffeq8WLF9tSJAB7TbUp6PirvaDk66iOm0M6s7/kHZlZ2BPwG6/Dzptvvqn7779fixYtUu/evUvsb9OmjXbs2OFTcQD8Y0SXlkHdXlDa+5lvozpu5YzcMLoD+IXXYWffvn3q0aNHmfvr1q2r06dPe9s8AD8a1jlZqTYFlNRwWDvLsgrukyM/XxrO6A7gF16HnTp16igzs+xF7Hbu3KlGjRp52zwAP7Mj8IRF0JEK3U25CkZcGN0JTvvXSjPaFfxGyPH6aqyePXtqzpw5eu6550rs27Fjh+bOnaunnnrKp+IA+JcrqHgzWTlsgo5kzyrlR7+SPh1V8XFcmRV8uAFkyPM67EyaNEnt27dXq1at1Lt3bzkcDr377ruaP3++li5dqsaNG2v8+PF21grAD7wJPGEVdFx8WaXcsqSPhxeM2hSfmFwa1+hOi858qQaD0m4ASRANKV6fxmrSpIm2bNmi7t27a8mSJbIsS++9957+/ve/65FHHtGmTZu45w4QIipzSissg46vXF+WngQdibk7wYQbQBrBq7Bz8eJFtWnTRh9++KHmzZunM2fO6OTJkzp+/Lh+/PFHzZ8/Xw0aNLC7VgB+5EngIeh4ofiXpaf4Ug0OxYMqQTQkeRV2YmJidPDgQTkKDa/Wr19fDRs2VEQEK1AAoaq8wEPQ8VJlR3Vc+FINvLKCKkE05HidTLp3767PPvvMzloABIHSAg9Bx0vejuq48KUaWGUFVYJoyPE67Lz88svas2ePfvvb3+rLL7/UsWPHdObMmRI/AEKPK/A4RNDxibejOi58qQZORUGVIBpSvL4a6+abb5ZUcD+dRYsWlXlcXp6X/5EDCKhhnZMJOb5wfVkqQlK+Dw1FcGVWIBS+Aqs03CIgpHgddsaPH19kzk6gzZw5U2+88YZOnDihW2+9VdOnT1e7du0CXRaAcOW+EaEvQUcFz88+VtBetWg7KkNFCo/qlDcqxy0CQobXYSctLc3GMnyzZMkSpaamavbs2Wrfvr3efvttdevWTbt37+aqMACBYceNCF1i6xN0qlJFozoujO6EDK/DTjB56623NHDgQD355JOSpNmzZ+uTTz7R/PnzNWbMmABXByBs+XIjQgSGp6M6LozuhISQDzu5ubnasmWLxo4d694WERGhlJQUbdy4sdTn5OTkKCcnx/04OztbkuR0OuV0Ov1bsJ+46g7V+k1CXwQX+iN4hEJfOPavUTVPRnVcfhrdubJ7lawW9/qvMJuFQl94wtP6vQ47ERERHs3Z8fcE5czMTOXl5alhw4ZFtjds2FC7du0q9TmTJ0/WxIkTS2xftWqVYmJi/FJnVUlPTw90CfgJfRFc6I/gEbR9YVm6Z3eaaitCEZWYa5WvCJ376AWtvz4t5EZ3grYvPHTx4kWPjrN1gnJeXp4OHTqk5cuX6/rrr9cvf/lLb5v3q7Fjxyo1NdX9ODs7W4mJieratavi4+MDWJn3nE6n0tPT1aVLF0VFRQW6nLBGXwQX+iN4BHtfOPavUbVtByv9vAjl66pLB9XrhpohM7oT7H3hKdeZmYr4ZYLy8ePHdeedd6plS8/W2vFFQkKCIiMjdfLkySLbT548qUaNGpX6nOjoaEVHl5zsFxUVFdKdLpnxHkxBXwQX+iN4BGVfWJa0frLnc3WKc0Sq2vrJ0vVdQ2p0Jyj7ohI8rd0vazs0btxYgwYN0iuvvOKP5ouoXr262rRpo9Wr/3PTrfz8fK1evVp33XWX318fAGAAbgBpNL8tZBUbG6uDBys/HOiN1NRUzZ07V++++66+++47Pfvss7pw4YL76iwAAMpU5AaQvojgrspByi9XY3377beaNm1alZzGkqTf/OY3OnXqlMaPH68TJ06odevWWrlyZYlJywAAlMANII3nddhJSkoq9Wqss2fPKisrSzExMVq+fLkvtVXKkCFDNGTIkCp7PQCAIbgBpPG8Dju/+MUvSoQdh8Ohq666Si1atFDfvn1Vt25dnwsEAMDvuAGk0bwOOwsXLrSxDAAAAP/wejbWkSNHdOnSpTL3X7p0SUeOHPG2eQAAAFt4HXaSkpK0bNmyMvf/7W9/U1JSkrfNAwAA2MLrsGNVcGmd0+lURITfrmwHAADwSKXm7GRnZ+vs2bPux6dPny71VNXZs2e1ePFiNW7c2OcCAQAAfFGpoZepU6cqKSnJfdn58OHD3Y8L/9x2221asWKFBg0a5K+6AcBjn33vUMuXV2na6r2BLgVAAFRqZKdr166qVauWLMvS6NGj9cgjj+j2228vcozD4VBsbKzatGmjtm3b2losAFTWjLX7teJopCTprfQ9kqRhnZMDWRKAKlapsHPXXXe515u6cOGC+vTpo1atWvmlMADw1bTVe/XHNfuLbCPwAOHH6/vsTJgwwc46AMBW01bvdQeb4gg8QHjxaW2sy5cva+nSpdq6dauysrKUn190XRGHw6E///nPPhUIAJVVXtBxIfAA4cPrsHP48GF16tRJhw4dUp06dZSVlaW6devq7NmzysvLU0JCgmrVqmVnrQA8NG31Xk1N36MRXVqG3Ze5J0HHhcADhAevb4QzatQoZWVladOmTdqzZ48sy9KSJUt0/vx5vf7666pZs6Y+++wzO2sF4AHXl72lgi/zR+duUtKYT8LiSqTKBB2Xt9L3hMVnA4Qzr8POmjVr9Nxzz6ldu3bumwdalqXo6GiNGjVKnTt31vDhw+2qE4AHSvuy37D/tDv4mPyl7k3QcTH9swHCnddh5+LFi7r22mslSfHx8XI4HMrKynLvv+uuu/Tll1/6XCAAz3g6T8XEL3Vfgo6LqZ8NAB/CTrNmzfT9999LkqpVq6arr75amzZtcu/fuXOnatSo4XuFACpU2XkqJn2p2xF0XEz7bAAU8Drs3Hvvvfroo4/cj/v376+pU6dq4MCBGjBggGbOnKnevXvbUiSAsoX7PJWpNgUdf7UHIPC8vhprzJgx2rx5s3JychQdHa0XX3xRGRkZ+uCDDxQZGalHH31Ub731lp21AijG13kqUskrkULtSq4RXVraNrLjag+AWXw6jdWnTx9FR0dLkmrUqKF58+bpxx9/VGZmphYuXKj4+HjbCgVQlD/mqRS/kquyoz/TVu+19covT9ob1jlZqTYFlNQQCXgAKsenmwpKUk5OjrZu3aoffvhBHTp0UEJCgh11ASiH3fNUSvtz4ceeBIDCNdlx/5rKtOfa7stnQtABzOX1yI4kTZs2TY0bN9bdd9+thx56SN98840kKTMzUwkJCZo/f74tRQL4DzuDjstb6XvKXVqhopGa0mryZV6QN+35MsJD0AHM5nXYWbBggYYPH67u3bvrz3/+syzLcu9LSEjQvffeq8WLF9tSJID/CMQE2vKCRkVrUHlzKszb9rwJPAQdwHxeh50333xT999/vxYtWlTqVVdt2rTRjh07fCoOQEmBmkBbWtCw+94+drRXmcBD0AHCg9dhZ9++ferRo0eZ++vWravTp0972zyAMtg5IbeyCgcNu+/tY2d7nnxGBB0gfHg9QblOnTrKzMwsc//OnTvVqFEjb5sHUA47JuR66630Pdp04LQ27K/c/8yUN8nY23sFldVe4e2ltUvQAcKL1yM7PXv21Jw5c3T27NkS+3bs2KG5c+fqvvvu86U2AOUI5AhPZYOOi7enwirTXmHDOifr9/e2KLKNoAOEH6/DzqRJk5SXl6dWrVrppZdeksPh0LvvvqvHHntMbdu2VYMGDTR+/Hg7awVQTCADj7e8PRXmSXulGdKphXom5skhgg4Qrrw+jdWkSRNt2bJFL774opYsWSLLsvTee+8pLi5OjzzyiF577TXuuQNUgUCe0vKWt6fCymtPKvuUVremlv74TFdFRUXZ8noAQotP99lp0KCB5s2bpzNnzujkyZM6fvy4fvzxR82fP18NGjSwq0YAFQjFER67go4La1oBKEulws6LL77ovnFgcfXr11fDhg0VEeFTfgLgpVALPP/Vop6t7bGmFYCyVCqZvPbaa/r222/dj0+fPq3IyEitWbPG9sIAVJ6nl1wHOhSldmmpRQPvZE0rAFXC52GYwndOBhB4rsDjUMnREztCga8jMoVrsGM0iqADoCKccwIMNKxzsg6+1ss9elL4SiRfroDydUSmtGDCmlYA/M3nVc8BBLdhnZPdgcDXoFN4REaq3BVg5QUTu9sDgMIqHXYOHTqkrVu3SpKysrIkSXv37lWdOnVKPf7222/3vjoAtrEr6LhUJqB4Ekzsbg8AXCoddl5++WW9/PLLRbY999xzJY6zLEsOh0N5eXneVwfAFnYHHRdPAkplgond7Xli2uq9mpq+RyMIUICxKhV2FixY4K86APiJv4KOi91rUFXlmlaFP5uKbkwIIHRVKuw88cQT/qoDgB/YsRyDJ0oLKL4EE7vbK01pnw2BBzATE5QBQ9m17pTk2Ze/6xi7TgnZ3V5h5X02BB7APIQdwEB2juhUNvDYGRLsbk/y7LMh8ABm4T47gIHsXieqovamrd6rpDGflLv6eDCoTAisaDV1AKGDsAMYyO51osprzxUgLAV3QJixdn+lR7uC+f0A8BynsQADeXOTvrKUNzE4VCb5fva9QyuO7vfqucH4fgBUDiM7gKH8ve5URZN8g2VEZMba/VpxNNKnNoLp/QCoPMIOYDB/rTvl6STfQAeEaav36o9rvBvRKS4Y3g8A7xB2AMN5E3h8DTougQ4IVT1RG0BwIuwAYaAygceuoOMSyMBTlRO1AQQvwg4QJjwJPHYHHZdABZ5hnZP1+3tb2NIWi48CoYuwA4SR8gKPv4KOS6ACz5BOLdQz0bcFiQk6QGgj7ABhprTA4++g4xKowNOtqeX1CA9BBwh9hB0gDLkCj0MVf5mbMsl3SKcWtk7UBhA6uKkgEKY8XXdqRJeWtq6cHshJvpW52SJBBzAHIzsAymXHzQldgiFA+DpRG0DoIewAqJC/78Zc1bydqA0gNBF2AHjEX3djDpTKTtQGELqYswPAY94sMBrMAcJV19T0PRoRxHUC8A1hB0ClmDbJ19OJ2gBCF6exAFQak3wBhBLCDgCvMMkXQKgg7ADwGpN8AYQC5uwA8AmTfAEEO8IOAJ8xyRdAMOM0FgAAMFpIh51Dhw5pwIABSkpKUs2aNdWiRQtNmDBBubm5gS4NAAAEiZA+jbVr1y7l5+frnXfe0XXXXadvv/1WAwcO1IULFzRlypRAlwcAAIJASIed7t27q3v37u7HzZs31+7duzVr1izCDgAAkBTiYac0WVlZqlu3brnH5OTkKCcnx/04OztbkuR0OuV0Ov1an7+46g7V+k1CXwQX+iN40BfBw5S+8LR+h2VZlp9rqTL79u1TmzZtNGXKFA0cOLDM49LS0jRx4sQS2xctWqSYmBh/lggAAGxy8eJFPfroo8rKylJ8fHyZxwVl2BkzZoxef/31co/57rvvdMMNN7gfHzt2TL/4xS/UsWNHzZs3r9znljayk5iYqMzMzHI/rGDmdDqVnp6uLl26KCoqKtDlhDX6IrjQH8GDvggepvRFdna2EhISKgw7QXkaa+TIkerfv3+5xzRv3tz954yMDHXq1En/9V//pTlz5lTYfnR0tKKjo0tsj4qKCulOl8x4D6agL4IL/RE86IvgEep94WntQRl26tevr/r163t07LFjx9SpUye1adNGCxYsUERESF9NDwAAbBaUYcdTx44dU8eOHXXNNddoypQpOnXqlHtfo0aNAlgZAAAIFiEddtLT07Vv3z7t27dPTZs2LbIvCKciAQCAAAjpcz79+/eXZVml/gAAAEghHnYAAAAqQtgBAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQcAABiNsAMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMBphBwAAGI2wAwAAjEbYAQAARiPsAAAAoxF2AACA0Qg7AADAaIQdAABgNMIOAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQcAABiNsAMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMBphBwAAGI2wAwAAjEbYAQAARiPsAAAAoxF2AACA0Qg7AADAaIQdAABgNMIOAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQcAABiNsAMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMJoxYScnJ0etW7eWw+HQtm3bAl0OAAAIEsaEndGjR6tJkyaBLgMAAAQZI8LOp59+qlWrVmnKlCmBLgUAAASZaoEuwFcnT57UwIEDtXz5csXExHj0nJycHOXk5LgfZ2dnS5KcTqecTqdf6vQ3V92hWr9J6IvgQn8ED/oieJjSF57W77Asy/JzLX5jWZZ69uypDh066KWXXtKhQ4eUlJSkr7/+Wq1bty7zeWlpaZo4cWKJ7YsWLfI4MAEAgMC6ePGiHn30UWVlZSk+Pr7M44Iy7IwZM0avv/56ucd89913WrVqlf7v//5P69atU2RkpMdhp7SRncTERGVmZpb7YQUzp9Op9PR0denSRVFRUYEuJ6zRF8GF/gge9EXwMKUvsrOzlZCQUGHYCcrTWCNHjlT//v3LPaZ58+Zas2aNNm7cqOjo6CL72rZtq379+undd98t9bnR0dElniNJUVFRId3pkhnvwRT0RXChP4IHfRE8Qr0vPK09KMNO/fr1Vb9+/QqPmzZtmiZNmuR+nJGRoW7dumnJkiVq3769P0sEAAAhIijDjqeaNWtW5HGtWrUkSS1atFDTpk0DURIAAAgyRlx6DgAAUJaQHtkp7tprr1UQzrcGAAABxMgOAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQcAABiNsAMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMBphBwAAGI2wAwAAjEbYAQAARiPsAAAAoxF2AACA0Qg7AADAaIQdAABgNMIOAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjVQt0AcHAsixJUnZ2doAr8Z7T6dTFixeVnZ2tqKioQJcT1uiL4EJ/BA/6IniY0heu723X93hZCDuSzp07J0lKTEwMcCUAAKCyzp07p9q1a5e532FVFIfCQH5+vjIyMhQXFyeHwxHocrySnZ2txMREHT16VPHx8YEuJ6zRF8GF/gge9EXwMKUvLMvSuXPn1KRJE0VElD0zh5EdSREREWratGmgy7BFfHx8SP/FNQl9EVzoj+BBXwQPE/qivBEdFyYoAwAAoxF2AACA0Qg7hoiOjtaECRMUHR0d6FLCHn0RXOiP4EFfBI9w6wsmKAMAAKMxsgMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIO4bLyclR69at5XA4tG3btkCXE3YOHTqkAQMGKCkpSTVr1lSLFi00YcIE5ebmBrq0sDBz5kxde+21qlGjhtq3b6+vvvoq0CWFncmTJ+uOO+5QXFycGjRooAceeEC7d+8OdFmQ9Nprr8nhcGj48OGBLsXvCDuGGz16tJo0aRLoMsLWrl27lJ+fr3feeUc7duzQ1KlTNXv2bL344ouBLs14S5YsUWpqqiZMmKCtW7fq1ltvVbdu3fTDDz8EurSwsm7dOg0ePFibNm1Senq6nE6nunbtqgsXLgS6tLC2efNmvfPOO7rlllsCXUqV4NJzg3366adKTU3V0qVLdfPNN+vrr79W69atA11W2HvjjTc0a9YsHThwINClGK19+/a64447NGPGDEkFa+AlJiZq6NChGjNmTICrC1+nTp1SgwYNtG7dOt1zzz2BLicsnT9/Xrfffrv+9Kc/adKkSWrdurXefvvtQJflV4zsGOrkyZMaOHCg3nvvPcXExAS6HBSSlZWlunXrBroMo+Xm5mrLli1KSUlxb4uIiFBKSoo2btwYwMqQlZUlSfw3EECDBw9Wr169ivz3YToWAjWQZVnq37+/Bg0apLZt2+rQoUOBLgk/2bdvn6ZPn64pU6YEuhSjZWZmKi8vTw0bNiyyvWHDhtq1a1eAqkJ+fr6GDx+uDh06qFWrVoEuJywtXrxYW7du1ebNmwNdSpViZCeEjBkzRg6Ho9yfXbt2afr06Tp37pzGjh0b6JKN5WlfFHbs2DF1795dDz/8sAYOHBigyoHAGTx4sL799lstXrw40KWEpaNHj+r3v/+93n//fdWoUSPQ5VQp5uyEkFOnTun06dPlHtO8eXP9+te/1t///nc5HA739ry8PEVGRqpfv3569913/V2q8Tzti+rVq0uSMjIy1LFjR915551auHChIiL4/wx/ys3NVUxMjD744AM98MAD7u1PPPGEzp49q48++ihwxYWpIUOG6KOPPtL69euVlJQU6HLC0vLly/Xggw8qMjLSvS0vL08Oh0MRERHKyckpss8khB0DHTlyRNnZ2e7HGRkZ6tatmz744AO1b99eTZs2DWB14efYsWPq1KmT2rRpo7/+9a/G/mMSbNq3b6927dpp+vTpkgpOoTRr1kxDhgxhgnIVsixLQ4cO1bJly/TFF18oOTk50CWFrXPnzunw4cNFtj355JO64YYb9MILLxh9apE5OwZq1qxZkce1atWSJLVo0YKgU8WOHTumjh076pprrtGUKVN06tQp975GjRoFsDLzpaam6oknnlDbtm3Vrl07vf3227pw4YKefPLJQJcWVgYPHqxFixbpo48+UlxcnE6cOCFJql27tmrWrBng6sJLXFxciUATGxurevXqGR10JMIO4Ffp6enat2+f9u3bVyJoMqjqX7/5zW906tQpjR8/XidOnFDr1q21cuXKEpOW4V+zZs2SJHXs2LHI9gULFqh///5VXxDCEqexAACA0ZglCQAAjEbYAQAARiPsAAAAoxF2AACA0Qg7AADAaIQdAABgNMIOAAAwGmEHAAAYjbADAACMRtgBAABGI+wA8NrChQvlcDhK/fHXyuIbNmxQWlqazp4965f2AZiHhUAB+OwPf/iDkpKSimzz1yrKGzZs0MSJE9W/f3/VqVPHL68BwCyEHQA+69Gjh9q2bRvoMnx24cIFxcbGBroMADbjNBYAvzt27JieeuopNWzYUNHR0br55ps1f/78IsccPnxYzz33nK6//nrVrFlT9erV08MPP6xDhw65j0lLS9OoUaMkSUlJSe5TZq5j+vfvr2uvvbbE66elpcnhcJS6befOnXr00Ud11VVX6e677/a43vLea40aNfTUU08V2f75558rKipKI0aM8KgdAPZhZAeAz7KyspSZmVlkW0JCgiTp5MmTuvPOO+VwODRkyBDVr19fn376qQYMGKDs7GwNHz5ckrR582Zt2LBBffv2VdOmTXXo0CHNmjVLHTt21M6dOxUTE6OHHnpIe/bs0f/+7/9q6tSp7teoX7++17U//PDDSk5O1n//93/LsiyP6y3L1Vdfraefflpz5szRhAkTdM0112jXrl16+OGH1aNHD7355pte1wrASxYAeGnBggWWpFJ/XAYMGGA1btzYyszMLPLcvn37WrVr17YuXrxoWZbl/l3Yxo0bLUnWX/7yF/e2N954w5JkHTx4sMTxTzzxhHXNNdeU2D5hwgSr+D93rm2PPPJIke2e1lue77//3oqOjraeffZZKzMz02rRooXVunVr6/z58xU+F4D9OI0FwGczZ85Uenp6kR9JsixLS5cuVe/evWVZljIzM90/3bp1U1ZWlrZu3SpJqlmzprs9p9Op06dP67rrrlOdOnXcx/jDoEGD3H+uTL3lufrqqzVw4EDNnz9fvXr10qVLl/Txxx8zHwgIEE5jAfBZu3btSp2gfOrUKZ09e1Zz5szRnDlzSn3uDz/8IEm6dOmSJk+erAULFujYsWOyLMt9TFZWln8Kl4pcRVaZeivy/PPPa8aMGfrmm2/0j3/8Q1dffXWR/bNmzdLcuXO1fft2jRs3TmlpaV6/BwDlI+wA8Jv8/HxJ0mOPPaYnnnii1GNuueUWSdLQoUO1YMECDR8+XHfddZdq164th8Ohvn37utupSPFJyC55eXllPqfwiFJl6q3Iq6++Kkm6cuWK6tatW2J/48aNlZaWpkWLFnnUHgDvEXYA+E39+vUVFxenvLw8paSklHvsBx98oCeeeKLIBN7Lly+XuHlgWYFGkq666qpSbzZ4+PBh2+stzxtvvKF58+ZpxowZGjVqlF599VXNmzevyDEPPPCAJGnFihVevw4AzzBnB4DfREZGqk+fPlq6dKm+/fbbEvtPnTpV5NjCp64kafr06SVGZVzzXkoLNS1atFBWVpa++eYb97bjx49r2bJlttdbluXLl2vMmDF65ZVXNHjwYD3zzDP6y1/+ooMHD3pUAwD7MbIDwK9ee+01rV27Vu3bt9fAgQN100036cyZM9q6das+//xznTlzRpL0y1/+Uu+9955q166tm266SRs3btTnn3+uevXqFWmvTZs2kqRx48apb9++ioqKUu/evRUbG6u+ffvqhRde0IMPPqhhw4bp4sWLmjVrllq2bOnxJGdP6y3Nli1b1K9fP/Xr10/jxo2TJI0ePVqzZ88udXQHQNUg7ADwq4YNG+qrr77SH/7wB3344Yf605/+pHr16unmm2/W66+/7j7uj3/8oyIjI/X+++/r8uXL6tChgz7//HN169atSHt33HGHXnnlFc2ePVsrV65Ufn6+Dh48qNjYWNWrV0/Lli1TamqqRo8eraSkJE2ePFl79+71OOx4Wm9x33//vXr37q3bbrtNc+fOdW9v0qSJnnrqKc2bN0/jxo0rsawGAP9zWMXHjQEAVWbQoEFq1KgRV2MBfsScHQAIgCtXrujy5cvKy8sr8mcA9mNkBwACIC0tTRMnTiyybcGCBerfv39gCgIMRtgBAABG4zQWAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQcAABjt/wOLkSR8+oGcqgAAAABJRU5ErkJggg==",
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.plot(\n",
" X_train[y_train == 0, 0],\n",
" X_train[y_train == 0, 1],\n",
" marker=\"D\",\n",
" markersize=10,\n",
" linestyle=\"\",\n",
" label=\"Class 0\",\n",
")\n",
"\n",
"plt.plot(\n",
" X_train[y_train == 1, 0],\n",
" X_train[y_train == 1, 1],\n",
" marker=\"^\",\n",
" markersize=13,\n",
" linestyle=\"\",\n",
" label=\"Class 1\",\n",
")\n",
"\n",
"plt.legend(loc=2)\n",
"\n",
"plt.xlim([-5, 5])\n",
"plt.ylim([-5, 5])\n",
"\n",
"plt.xlabel(\"Feature $x_1$\", fontsize=12)\n",
"plt.ylabel(\"Feature $x_2$\", fontsize=12)\n",
"\n",
"plt.grid()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "db50db02-3696-4f86-b149-74baabeec6c4",
"metadata": {},
"source": [
"## 4) Implementing the model"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "3da86d9a-7cd5-467c-bf65-3388fe272bd5",
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"import torch.nn.functional as F\n",
"\n",
"class LogisticRegression(torch.nn.Module):\n",
"\n",
" def __init__(self, num_features, num_classes):\n",
" super().__init__()\n",
" self.linear = torch.nn.Linear(num_features, num_classes)\n",
"\n",
" def forward(self, x):\n",
" logits = self.linear(x)\n",
" return logits"
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "e88b360e-c33c-4724-b46f-58323a9a7628",
"metadata": {},
"outputs": [],
"source": [
"torch.manual_seed(1)\n",
"\n",
"model = LogisticRegression(num_features=2, num_classes=2)"
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "f2923ae0-cf24-4252-bd19-cc326a39bc72",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([[0.1312, 0.8688],\n",
" [0.1312, 0.8688],\n",
" [0.6970, 0.3030]])\n"
]
}
],
"source": [
"x = torch.tensor([[1.1, 2.1],\n",
" [1.1, 2.1],\n",
" [9.1, 4.1]])\n",
"\n",
"with torch.no_grad():\n",
" logits = model(x)\n",
" probas = F.softmax(logits, dim=1)\n",
"\n",
"print(probas)"
]
},
{
"cell_type": "markdown",
"id": "12068721-daf9-4ea1-9240-5bcc19c75ce9",
"metadata": {},
"source": [
"## 5) Defining a DataLoader"
]
},
{
"cell_type": "code",
"execution_count": 27,
"id": "f6376d80-e7cf-43b4-96eb-bfd99709b63a",
"metadata": {},
"outputs": [],
"source": [
"from torch.utils.data import Dataset, DataLoader\n",
"\n",
"\n",
"class MyDataset(Dataset):\n",
" def __init__(self, X, y):\n",
"\n",
" self.features = torch.tensor(X, dtype=torch.float32)\n",
" self.labels = torch.tensor(y, dtype=torch.int64)\n",
"\n",
" def __getitem__(self, index):\n",
" x = self.features[index]\n",
" y = self.labels[index]\n",
" return x, y\n",
"\n",
" def __len__(self):\n",
" return self.labels.shape[0]\n",
"\n",
"\n",
"train_ds = MyDataset(X_train, y_train)\n",
"\n",
"train_loader = DataLoader(\n",
" dataset=train_ds,\n",
" batch_size=10,\n",
" shuffle=True,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 28,
"id": "b64f8926-930c-4945-950a-31083608ea7e",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(20, 2)"
]
},
"execution_count": 28,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"X_train.shape"
]
},
{
"cell_type": "markdown",
"id": "46bc16a0-ec59-4c54-a209-0a5e22406287",
"metadata": {},
"source": [
"## 6) The training loop"
]
},
{
"cell_type": "code",
"execution_count": 85,
"id": "3dcaa2b1-4019-4128-9ff5-6a966c3abdf2",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch: 001/020 | Batch 000/002 | Loss: 0.71\n",
"Epoch: 001/020 | Batch 001/002 | Loss: 0.54\n",
"Epoch: 002/020 | Batch 000/002 | Loss: 0.24\n",
"Epoch: 002/020 | Batch 001/002 | Loss: 0.33\n",
"Epoch: 003/020 | Batch 000/002 | Loss: 0.26\n",
"Epoch: 003/020 | Batch 001/002 | Loss: 0.12\n",
"Epoch: 004/020 | Batch 000/002 | Loss: 0.18\n",
"Epoch: 004/020 | Batch 001/002 | Loss: 0.12\n",
"Epoch: 005/020 | Batch 000/002 | Loss: 0.19\n",
"Epoch: 005/020 | Batch 001/002 | Loss: 0.06\n",
"Epoch: 006/020 | Batch 000/002 | Loss: 0.07\n",
"Epoch: 006/020 | Batch 001/002 | Loss: 0.15\n",
"Epoch: 007/020 | Batch 000/002 | Loss: 0.08\n",
"Epoch: 007/020 | Batch 001/002 | Loss: 0.11\n",
"Epoch: 008/020 | Batch 000/002 | Loss: 0.11\n",
"Epoch: 008/020 | Batch 001/002 | Loss: 0.06\n",
"Epoch: 009/020 | Batch 000/002 | Loss: 0.11\n",
"Epoch: 009/020 | Batch 001/002 | Loss: 0.05\n",
"Epoch: 010/020 | Batch 000/002 | Loss: 0.06\n",
"Epoch: 010/020 | Batch 001/002 | Loss: 0.09\n",
"Epoch: 011/020 | Batch 000/002 | Loss: 0.06\n",
"Epoch: 011/020 | Batch 001/002 | Loss: 0.08\n",
"Epoch: 012/020 | Batch 000/002 | Loss: 0.07\n",
"Epoch: 012/020 | Batch 001/002 | Loss: 0.06\n",
"Epoch: 013/020 | Batch 000/002 | Loss: 0.05\n",
"Epoch: 013/020 | Batch 001/002 | Loss: 0.07\n",
"Epoch: 014/020 | Batch 000/002 | Loss: 0.03\n",
"Epoch: 014/020 | Batch 001/002 | Loss: 0.09\n",
"Epoch: 015/020 | Batch 000/002 | Loss: 0.02\n",
"Epoch: 015/020 | Batch 001/002 | Loss: 0.09\n",
"Epoch: 016/020 | Batch 000/002 | Loss: 0.02\n",
"Epoch: 016/020 | Batch 001/002 | Loss: 0.09\n",
"Epoch: 017/020 | Batch 000/002 | Loss: 0.04\n",
"Epoch: 017/020 | Batch 001/002 | Loss: 0.06\n",
"Epoch: 018/020 | Batch 000/002 | Loss: 0.04\n",
"Epoch: 018/020 | Batch 001/002 | Loss: 0.05\n",
"Epoch: 019/020 | Batch 000/002 | Loss: 0.06\n",
"Epoch: 019/020 | Batch 001/002 | Loss: 0.03\n",
"Epoch: 020/020 | Batch 000/002 | Loss: 0.05\n",
"Epoch: 020/020 | Batch 001/002 | Loss: 0.04\n"
]
}
],
"source": [
"import torch.nn.functional as F\n",
"\n",
"\n",
"torch.manual_seed(1)\n",
"model = LogisticRegression(num_features=2, num_classes=2)\n",
"optimizer = torch.optim.SGD(model.parameters(), lr=0.5)\n",
"\n",
"num_epochs = 20\n",
"\n",
"for epoch in range(num_epochs):\n",
"\n",
" model = model.train()\n",
" for batch_idx, (features, class_labels) in enumerate(train_loader):\n",
"\n",
" logits = model(features)\n",
"\n",
" loss = F.cross_entropy(logits, class_labels)\n",
"\n",
" optimizer.zero_grad()\n",
" loss.backward()\n",
" optimizer.step()\n",
"\n",
" ### LOGGING\n",
" print(f'Epoch: {epoch+1:03d}/{num_epochs:03d}'\n",
" f' | Batch {batch_idx:03d}/{len(train_loader):03d}'\n",
" f' | Loss: {loss:.2f}')\n"
]
},
{
"cell_type": "markdown",
"id": "bb0d5821-7c8d-46b5-9e7d-02e72cac2acc",
"metadata": {},
"source": [
"## 7) Evaluating the results"
]
},
{
"cell_type": "code",
"execution_count": 86,
"id": "d910ddbb-798f-47ab-8aab-e2dba4aa4005",
"metadata": {},
"outputs": [],
"source": [
"def compute_accuracy(model, dataloader):\n",
"\n",
" model = model.eval()\n",
"\n",
" correct = 0.0\n",
" total_examples = 0\n",
"\n",
" for idx, (features, class_labels) in enumerate(dataloader):\n",
"\n",
" with torch.no_grad():\n",
" logits = model(features)\n",
"\n",
" pred = torch.argmax(logits, dim=1)\n",
"\n",
" compare = class_labels == pred\n",
" correct += torch.sum(compare)\n",
" total_examples += len(compare)\n",
"\n",
" return correct / total_examples"
]
},
{
"cell_type": "code",
"execution_count": 87,
"id": "27538c8d-61bc-47b0-8289-b6aab4aa16ed",
"metadata": {},
"outputs": [],
"source": [
"train_acc = compute_accuracy(model, train_loader)"
]
},
{
"cell_type": "code",
"execution_count": 88,
"id": "5a4ecf35-4745-43a8-8ea8-14f71cba5b59",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Accuracy: 100.0%\n"
]
}
],
"source": [
"print(f\"Accuracy: {train_acc*100}%\")"
]
},
{
"cell_type": "markdown",
"id": "fbcd412a-02c0-4d2c-835e-4b01368f53f8",
"metadata": {},
"source": [
"## 8) Optional: visualizing the decision boundary"
]
},
{
"cell_type": "code",
"execution_count": 89,
"id": "a76bb67c-358c-4e91-a5c7-5456333827aa",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAG2CAYAAACZEEfAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA7zklEQVR4nO3deXxU1f3/8fckhEBCAkJYJWjA4EYVBUG/WAsS9uJGbVGsoohFWQpBEEQhVPyiX1EsS0GggLXyhe9DBFtFJAJCLVD5gVQE2VcJIAFJWJMhub8/4kyzZzJzJzNz5vV8PPIIc++dM5+Zg8zbc8+9x2FZliUAAABDRQS6AAAAAH8i7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMBphBwAAGK1aoAsIBvn5+crIyFBcXJwcDkegywEAAB6wLEvnzp1TkyZNFBFR9vgNYUdSRkaGEhMTA10GAADwwtGjR9W0adMy9xN2JMXFxUkq+LDi4+MDXI13nE6nVq1apa5duyoqKirQ5YQ1+iK40B/Bg74IHqb0RXZ2thITE93f42Uh7EjuU1fx8fEhHXZiYmIUHx8f0n9xTUBfBBf6I3jQF8HDtL6oaAoKE5QBAIDRCDsAAMBohB0AAGA05ux4KD8/X7m5uYEuo0xOp1PVqlXT5cuXlZeXF+hy/KJ69erlXloIAEBpCDseyM3N1cGDB5Wfnx/oUspkWZYaNWqko0ePGnuvoIiICCUlJal69eqBLgUAEEIIOxWwLEvHjx9XZGSkEhMTg3ZkIT8/X+fPn1etWrWCtkZfuG78ePz4cTVr1szYQAcAsB9hpwJXrlzRxYsX1aRJE8XExPjU1rTVezU1fY9GdGmpYZ2TbaqwgOs0W40aNYwMO5JUv359ZWRk6MqVK0ZcKgkAqBpmfivayDX/xddTJ9NW79Vb6XtkSXorfY+mrd5rQ3XhxdUHps5JAgD4B2HHQ76cNnEFncIIPJXHqSsAgDcIO35WWtBxIfAAAOB/hB0/Ki/ouBB4AADwL8KOn3gSdFwCGXgcDoeWL18ekNcGAKAqEHb8oDJBx8UfgefEiRMaOnSomjdvrujoaCUmJqp3795avXq1ra/jLcuyNH78eDVu3Fg1a9ZUSkqK9u5llAsAYC/Cjs28CToudgaeQ4cOqU2bNlqzZo3eeOMNbd++XStXrlSnTp00ePBgW17DV//zP/+jadOmafbs2frXv/6l2NhYdevWTZcvXw50aQAAgxB2bORL0HGxK/A899xzcjgc+uqrr9SnTx+1bNlSN998s1JTU7Vp06Yyn/fCCy+oZcuWiomJUfPmzfXyyy/L6XS69//73/9Wp06dFBcXp/j4eLVp00b/7//9P0nS4cOH1bt3b1111VWKjY3VzTffrBUrVpT6OpZl6e2339ZLL72k+++/X7fccov+8pe/KCMjg9NqAABbcVNBm9gRdFxc7Xh748EzZ85o5cqVevXVVxUbG1tif506dcp8blxcnBYuXKgmTZpo+/btGjhwoOLi4jR69GhJUr9+/XTbbbdp1qxZioyM1LZt29w3+Bs8eLByc3O1fv16xcbGaufOnapVq1apr3Pw4EGdOHFCKSkp7m21a9dW+/bttXHjRvXt29er9w4AQHGEHZtMtSnoFG7P27Czb98+WZalG264odLPfemll9x/vvbaa/X8889r8eLF7rBz5MgRjRo1yt12cvJ/ajxy5Ij69Omjn/3sZ5Kk5s2bl/k6J06ckCQ1bNiwyPaGDRu69wEAYAdOY9lkRJeWQdOeZVleP3fJkiXq0KGDGjVqpFq1aumll17SkSNH3PtTU1P19NNPKyUlRa+99pr279/v3jds2DBNmjRJHTp00IQJE/TNN994XQcAAHYh7NhkWOdkpdoUeFJ9XDsrOTlZDodDu3btqtTzNm7cqH79+qlnz576+OOP9fXXX2vcuHHKzc11H5OWlqYdO3aoV69eWrNmjW666SYtW7ZMkvT000/rwIED+u1vf6vt27erbdu2mj59eqmv1ahRI0nSyZMni2w/efKkex8AAHYg7NjIjsDja9CRpLp166pbt26aOXOmLly4UGL/2bNnS33ehg0bdM0112jcuHFq27atkpOTdfjw4RLHtWzZUiNGjNCqVav00EMPacGCBe59iYmJGjRokD788EONHDlSc+fOLfW1kpKS1KhRoyKXwWdnZ+tf//qX7rrrrkq+YwAAykbYsZkvgceOoOMyc+ZM5eXlqV27dlq6dKn27t2r7777TtOmTSszTCQnJ+vIkSNavHix9u/fr2nTprlHbSTp0qVLGjJkiL744gsdPnxY//znP7V582bdeOONkqThw4frs88+08GDB7V161atXbvWva84h8Oh4cOHa9KkSfrb3/6m7du36/HHH1eTJk30wAMP2PIZAAAgMUHZL1yBpTJXZ9kZdKSCycFbt27Vq6++qpEjR+r48eOqX7++2rRpo1mzZpX6nPvuu08jRozQkCFDlJOTo169eunll19WWlqaJCkyMlKnT5/W448/rpMnTyohIUEPPfSQJk6cKKlgNfLBgwfr+++/V3x8vLp3766pU6eWWePo0aN14cIFPfPMMzp79qzuvvturVy5UjVq1LDtcwAAwGH5MpvVENnZ2apdu7aysrIUHx9fZN/ly5d18OBBJSUlVfpL2NPL0e0IOvn5+crOzlZ8fLwiIswcsPOlL6qS0+nUihUr1LNnT/dl+Qgc+iN40BfBw5S+KO/7uzAzvxWDhCentOwe0QEAAEURdvysvMBD0AEAwP8IO1WgtMBD0AEAoGowQbmKuILN1PQ9GkHQAQCgyhB2qtCwzsmEHAAAqhinsarS/rXSjHYFvwEAQJUg7FQVy5JWT5Qydxf85op/AACqhHFh57XXXnPfnTeo7F8tZXxd8OeMrwseAwAAvzMq7GzevFnvvPOObrnllkCXUpRlSWsmSY7IgseOyILHQTC643A4tHz58kCXAQCA3xgTds6fP69+/fpp7ty5uuqqqwJdTlGuUR0rr+CxlVclozsnTpzQ0KFD1bx5c0VHRysxMVG9e/cusvhmIH344Yfq2rWr6tWrJ4fDoW3btgW6JACAgYy5Gmvw4MHq1auXUlJSNGnSpHKPzcnJUU5Ojvtxdna2pILbZzudziLHOp1OWZal/Px85efnV74wy5Ljp1EdhyvsSLJ+Gt2xkjpJDkfl2y3xMpb7d35+vg4dOqSf//znqlOnjl5//XX97Gc/k9Pp1KpVqzR48GDt3LnT/Vyv35uPzp07pw4dOuhXv/qVfve731VYR35+vizLktPpVGRkZBVWWjmuv0PF/y4hMOiP4EFfBA9T+sLT+o0IO4sXL9bWrVu1efNmj46fPHmye/HKwlatWqWYmJgi26pVq6ZGjRrp/Pnzys3NrXRt1Q6tUy3XXJ1CHD+N7lzY/rGuXPuLSrdblnPnzkmSfve730kqeE+xsbHu/QMGDNCvfvUrd8CTClYzdz2eMGGCPvnkE2VkZKhBgwZ6+OGHNXr0aPfaKdu3b9eLL76obdu2yeFwqHnz5po6dapuu+02HTlyRKNHj9amTZvkdDrVrFkzTZw4UV27di211vvvv1+SdOTIEUnShQsXitRVXG5uri5duqT169frypUr3n5EVSY9PT3QJaAQ+iN40BfBI9T74uLFix4dF/Jh5+jRo/r973+v9PR0jxeHHDt2rFJTU92Ps7OzlZiYqK5du5a6EOjRo0dVq1atyi8+aVlyfDVVVrFRHfduR6Riv5oq62e/9Hl0x7IsnTt3TnFxcfrxxx+1evVqTZo0SY0bNy5xbPH3WLNmTfe2hIQELVy4UE2aNNH27dv1u9/9TgkJCRo1apQk6dlnn1Xr1q31zjvvKDIyUtu2bVOdOnUUHx+vsWPHKi8vT+vWrVNsbKx27typ+Pj4chdnk6RatWpJkmJjY8s99vLly6pZs6buueeeoF8IND09XV26dAnpBfZMQX8ED/oieJjSF+X9D3JhIR92tmzZoh9++EG33367e1teXp7Wr1+vGTNmKCcnp8Qpj+joaEVHR5doKyoqqkSn5+XlyeFwKCIiovKrie/7/D9XYJXCNbrjOLhWui6lcm0X4zr943A4dODAAVmWpRtvvNGjmgu/t5dfftm9vXnz5tq7d68WL16sF154QVLBKMyoUaN00003SZKuv/569/FHjx5Vnz59dOutt0qSrrvuOo9qd712RZ9xRESEHA5Hqf0UjEKlznBBfwQP+iJ4hHpfeFp7yIedzp07a/v27UW2Pfnkk7rhhhv0wgsvBG5uR+ErsEoZ1XFzXZnVorMtc3cKXtr7q7yWLFmiadOmaf/+/Tp//ryuXLlSZLQlNTVVTz/9tN577z2lpKTo4YcfVosWLSRJw4YN07PPPqtVq1YpJSVFffr0Cb4r4wAAYSfkr8aKi4tTq1ativzExsaqXr16atWqVeAKK34FVln8cGVWcnKyHA6Hdu3aVannbdy4Uf369VPPnj318ccf6+uvv9a4ceOKzFVKS0vTjh071KtXL61Zs0Y33XSTli1bJkl6+umndeDAAf32t7/V9u3b1bZtW02fPt229wUAgDdCPuwEpeL31amIzffdqVu3rrp166aZM2fqwoULJfafPXu21Odt2LBB11xzjcaNG6e2bdsqOTlZhw8fLnFcy5YtNWLECK1atUoPPfSQFixY4N6XmJioQYMG6cMPP9TIkSM1d+5cW94TAADeCvnTWKX54osvAltA4bsle6Lw6I6Pc3dcZs6cqQ4dOqhdu3b6wx/+oFtuuUVXrlxRenq6Zs2ape+++67Ec5KTk3XkyBEtXrxYd9xxhz755BP3qI1UcNXWqFGj9Ktf/UpJSUn6/vvvtXnzZvXp00eSNHz4cPXo0UMtW7bUjz/+qLVr1+rGG28ss8YzZ87oyJEjysjIkCTt3r1bktSoUSM1atTIls8BAABGduxW2VEdF5tHd5o3b66tW7eqU6dOGjlypFq1aqUuXbpo9erVmjVrVqnPue+++zRixAgNGTJErVu31oYNG4pMWI6MjNTp06f1+OOPq2XLlvr1r3+tHj16uC/jz8vL0+DBg3XjjTeqe/fuatmypf70pz+VWePf/vY33XbbberVq5ckqW/fvrrttts0e/ZsWz4DAAAkyWH5MpvVENnZ2apdu7aysrJKvfT84MGDSkpK8uxy532fS3/t430xjy31anQnPz9f2dnZio+Pr/xVYyGi0n0RIE6nUytWrFDPnj1D+ioHU9AfwYO+CB6m9EV539+FmfmtGCjejuq4BNGaWQAAmIKwYydPr8AqSxWtmQUAQDgh7NjFNarj80cawegOAAA2IuzYJS9XyjomydcFNfOl7GMF7QEAAJ8Zeem5P1Q4j7tatPTMWulCpu8vFlu/oD0UwVx6AIA3CDsVcC03kZubq5o1a5Z/cO2mBT/wC9ednAO2BAgAICQRdipQrVo1xcTE6NSpU4qKigray7rz8/OVm5ury5cvB22NvsjPz9epU6cUExOjatX4awsA8BzfGhVwOBxq3LixDh48WOrSCcHCsixdunRJNWvWlMOmBUWDTUREhJo1a2bs+wMA+AdhxwPVq1dXcnJykQUxg43T6dT69et1zz33hPQNospTvXp1I0etAAD+RdjxUERERFDftTcyMlJXrlxRjRo1jA07AAB4g/9NBgAARiPsAAAAoxF2AACA0Qg7AADAaIQdAABgNMIOAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQcAABiNsAMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMBphBwAAGI2wAwAAjEbYAQAARiPsAAAAoxF2AACA0Qg7AADAaIQdAABgNMIOAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQcAABiNsAMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMBphBwAAGI2wAwAAjEbYAQAARiPsAAAAoxF2AACA0UI+7EyePFl33HGH4uLi1KBBAz3wwAPavXt3oMsCAABBIuTDzrp16zR48GBt2rRJ6enpcjqd6tq1qy5cuBDo0gAAQBCoFugCfLVy5coijxcuXKgGDRpoy5YtuueeewJUFQAACBYhH3aKy8rKkiTVrVu3zGNycnKUk5PjfpydnS1Jcjqdcjqd/i3QT1x1h2r9JqEvggv9ETzoi+BhSl94Wr/DsizLz7VUmfz8fN133306e/asvvzyyzKPS0tL08SJE0tsX7RokWJiYvxZIgAAsMnFixf16KOPKisrS/Hx8WUeZ1TYefbZZ/Xpp5/qyy+/VNOmTcs8rrSRncTERGVmZpb7YQUzp9Op9PR0denSRVFRUYEuJ6zRF8GF/gge9EXwMKUvsrOzlZCQUGHYMeY01pAhQ/Txxx9r/fr15QYdSYqOjlZ0dHSJ7VFRUSHd6ZIZ78EU9EVwoT+CB30RPEK9LzytPeTDjmVZGjp0qJYtW6YvvvhCSUlJgS4JAAAEkZAPO4MHD9aiRYv00UcfKS4uTidOnJAk1a5dWzVr1gxwdQAAINBC/j47s2bNUlZWljp27KjGjRu7f5YsWRLo0gAAQBAI+ZEdg+ZXAwAAPwj5kR0AAIDyEHYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQdAeNm/VprRruA3gLBA2AEQPixLWj1Rytxd8Lsyt64gJAEhi7ADIHzsXy1lfF3w54yvCx57wpeQVGYthCegqhB2AIQHy5LWTJIckQWPHZEFjz0JLt6GpPJqsTs8ASgTYQdAWHAcWFsQVKy8gg1WnmfBxZeQVBa7wxOAchF2AJjPshSxbvJ/AouLJ8HFFUwqG5LKqcX28ASgXIQdAMarf267Io4XCiwuFQWX4sHExZeAYnd4AlAhwg4As1mWbsxYKqt4YHEpL7gUDybuNr0MKP4ITwAqRNgBYDTHgbW66tJBOYoHFpeygktZwcTdsBcBxe7wBMAjhB0A5vpprk5+Rf/UlRZcygom7rYrGVD8EZ4AeISwA8Bc+1cr4vjXilB++ccVDy4VBRMXby5ftys8AfAYYQeAmX4KLGXO1SmucHCpKJi4X8PLy9c9qQGAbQg7AMz0U2Apc65Oca7gsu9zz4KJizeXr1dUA6M7gK0IOwDM4+lISnGOSOnTFzwLJu7X8vLy9fJqYHQHsBVhB4B5PB1JKc7Kk87slxyV/KfRm8vXy6uB0R3AVoQdAGbxdlSnSBsVTGgucbyXl6+XhdEdwFaEHQBm8XZUx1feXL5eFkZ3AFsRdgCYwzWSEoh/2ry9fL0sjgjpfx+V9q2xr0YgTBF2AJgjL1fKOiZVdF8dv4mo/OXrZbHypbwcacXznM4CfFQt0AUAgG2qRUvPrJV2rZA+HVVyf483pMR2Jbfn5UqLfi1d+tHHAvKl7GPSlZxCI0w+Bq8z+wsuh0/u4mNtQPgi7AAwS/zV0r8XFZw+KjSqYjki5fj3IqndQMnhKPm8QV9KFzJ9f/3Y+gXt2znCtOYV6bqU0usGUCFbw87Fixe1e/duXXfddYqLiyuy75///Kc6dOhg58sBQEmu00fFOArPqbkupeTzajct+KnUa60tuC9Pj9elFp2K7ntmrXfh6ehXJUeljv+77LoBVMi2sLNp0yb17t1b1atX148//qgXX3xRL730knt/jx49lJ2dbdfLAUBJhScFlzZXxnXFVIvOvo+SWJa0eqKUubvgd/OORdv0JjxZlvTx8JL121k3EIZsm6CcmpqqGTNm6NixY/r3v/+tjz/+WI8//risnybWWUywA+BvVbnYZuERJLvbLF4/l6IDPrEt7OzcuVO/+c1vJEnJycn64osvdObMGT344IPKzc2162UAoHRVudhm8dfyR5vFcaNBwGu2hZ3atWvr2LFj7sc1atTQ8uXLVbNmTXXr1k35+YG6FBRAWKjKxTaLv5Y/2iyO0R3Aa7aFnZSUFC1YsKDItmrVqmnRokW67rrrdOnSJbteCgCKqsrFNst6LX+0WRyjO4BXbAs7s2bNUmpqaontDodDc+fO1aFDh+x6KQAoqioX2/THvJqqHJUCwpDXYWfkyJFFHlevXl0xMTFlHt+sWTNvXwoAylaVi236Y15NVY5KAWHK67Azffp0Pfjgg+Wenjp8+LC3zQOAZ6pysU1P59V8+ZZ9bZb1GozuAB7zOuysWLFC69at089//nOdOHGiyL7Dhw/rmWee0fXXX+9zgQBQJp8X26zEKEllXmv9G5InF2VU5agUEMa8DjspKSnasGGDzp49qzvuuEPbtm0rEnLee+89DRgwwM5aAaAonxfbrMQoSWVey3lJ+vJNe9ssjNEdoFJ8mqB8ww036KuvvlLTpk1199136/rrr9f777+vZ599VgcOHNDMmTPtqhMAinKNivh8nUVExaMk3ozA/OPN8kd3qnJUCghzPv0rcfToUY0fP17btm3TxYsX5XQ6NWXKFE2dOlWNGze2q0YAKCkv16bFNn9aqTyvnJufejMCU9HoTlWOSgFhzuu1sZ5++mn99a9/lcPh0MCBA/X8889r0qRJGjp0qLKysjRmzBg76wSAoqpFe7TYpvPKFfdCxFHVyvgnL7Z+QXulqWi9rfL8403p7pFSRLH/rywyKuVLWItgzSzAA16Hnffff18DBw7U2LFj1aRJE0nSnDlzlJycrLFjx2r37t2aM2eOoqKibCsWAIrwZLFNp1NZMcekxrdK3vx7VMYq6h5xje7cU2wVc3+MSpUV1gB4H3b279/vDjmFjRo1SsnJyXrsscd04MABrVu3zqcCAQTGtNV7NTV9j0Z0aalhnZMDXU5g+DKq41La6I6Ho1IeKW9UCoAkH8JOaUHH5YEHHtC6det03333eds8gACatnqv3krfI0nu32EZeHwZ1XEpa3THk1EpALawbbmI4tq0aaOvvvrKX80D8JPCQcflrfQ9mrZ6b4AqChDbrvZSxVdmAfArv4UdSbr66qv92TwAm5UWdFzCLvDYNq9GBaM7ez/zvR0AXvH6NBYAs5QXdFzC6pRWefNqrHxpfncpL8ezthwR0rrXpZbdi141tX+t9OkLUo/XpRad7KkbQAmEHQAeBR2XsAo8Zc2rWf+G50FHKghHrnviXJfy0zZLWj1Rytxd8Lt5Ry4fB/zEr6exAAS/ygQdl7A7pVVYfn7BHJzKKn7H48KTn7k5IOBXhB0gjHkTdFzCNvB8+WbBHJzKKnzH4+JLRbD0A+BXPp/GysnJ0datW/XDDz+oQ4cOSkhIsKMuAH7mS9BxCatTWpL3ozouhUNN4UvaCwch12kuALbxaWRn2rRpaty4se6++2499NBD+uabbyRJmZmZSkhI0Pz5820pEoC97Ag6LmE1wuPtqI6LK9R8+kLJBUAZ3QH8xuuws2DBAg0fPlzdu3fXn//8Z1mF/gNNSEjQvffeq8WLF9tSJAB7TbUp6PirvaDk66iOm0M6s7/kHZlZ2BPwG6/Dzptvvqn7779fixYtUu/evUvsb9OmjXbs2OFTcQD8Y0SXlkHdXlDa+5lvozpu5YzcMLoD+IXXYWffvn3q0aNHmfvr1q2r06dPe9s8AD8a1jlZqTYFlNRwWDvLsgrukyM/XxrO6A7gF16HnTp16igzs+xF7Hbu3KlGjRp52zwAP7Mj8IRF0JEK3U25CkZcGN0JTvvXSjPaFfxGyPH6aqyePXtqzpw5eu6550rs27Fjh+bOnaunnnrKp+IA+JcrqHgzWTlsgo5kzyrlR7+SPh1V8XFcmRV8uAFkyPM67EyaNEnt27dXq1at1Lt3bzkcDr377ruaP3++li5dqsaNG2v8+PF21grAD7wJPGEVdFx8WaXcsqSPhxeM2hSfmFwa1+hOi858qQaD0m4ASRANKV6fxmrSpIm2bNmi7t27a8mSJbIsS++9957+/ve/65FHHtGmTZu45w4QIipzSissg46vXF+WngQdibk7wYQbQBrBq7Bz8eJFtWnTRh9++KHmzZunM2fO6OTJkzp+/Lh+/PFHzZ8/Xw0aNLC7VgB+5EngIeh4ofiXpaf4Ug0OxYMqQTQkeRV2YmJidPDgQTkKDa/Wr19fDRs2VEQEK1AAoaq8wEPQ8VJlR3Vc+FINvLKCKkE05HidTLp3767PPvvMzloABIHSAg9Bx0vejuq48KUaWGUFVYJoyPE67Lz88svas2ePfvvb3+rLL7/UsWPHdObMmRI/AEKPK/A4RNDxibejOi58qQZORUGVIBpSvL4a6+abb5ZUcD+dRYsWlXlcXp6X/5EDCKhhnZMJOb5wfVkqQlK+Dw1FcGVWIBS+Aqs03CIgpHgddsaPH19kzk6gzZw5U2+88YZOnDihW2+9VdOnT1e7du0CXRaAcOW+EaEvQUcFz88+VtBetWg7KkNFCo/qlDcqxy0CQobXYSctLc3GMnyzZMkSpaamavbs2Wrfvr3efvttdevWTbt37+aqMACBYceNCF1i6xN0qlJFozoujO6EDK/DTjB56623NHDgQD355JOSpNmzZ+uTTz7R/PnzNWbMmABXByBs+XIjQgSGp6M6LozuhISQDzu5ubnasmWLxo4d694WERGhlJQUbdy4sdTn5OTkKCcnx/04OztbkuR0OuV0Ov1bsJ+46g7V+k1CXwQX+iN4hEJfOPavUTVPRnVcfhrdubJ7lawW9/qvMJuFQl94wtP6vQ47ERERHs3Z8fcE5czMTOXl5alhw4ZFtjds2FC7du0q9TmTJ0/WxIkTS2xftWqVYmJi/FJnVUlPTw90CfgJfRFc6I/gEbR9YVm6Z3eaaitCEZWYa5WvCJ376AWtvz4t5EZ3grYvPHTx4kWPjrN1gnJeXp4OHTqk5cuX6/rrr9cvf/lLb5v3q7Fjxyo1NdX9ODs7W4mJieratavi4+MDWJn3nE6n0tPT1aVLF0VFRQW6nLBGXwQX+iN4BHtfOPavUbVtByv9vAjl66pLB9XrhpohM7oT7H3hKdeZmYr4ZYLy8ePHdeedd6plS8/W2vFFQkKCIiMjdfLkySLbT548qUaNGpX6nOjoaEVHl5zsFxUVFdKdLpnxHkxBXwQX+iN4BGVfWJa0frLnc3WKc0Sq2vrJ0vVdQ2p0Jyj7ohI8rd0vazs0btxYgwYN0iuvvOKP5ouoXr262rRpo9Wr/3PTrfz8fK1evVp33XWX318fAGAAbgBpNL8tZBUbG6uDBys/HOiN1NRUzZ07V++++66+++47Pfvss7pw4YL76iwAAMpU5AaQvojgrspByi9XY3377beaNm1alZzGkqTf/OY3OnXqlMaPH68TJ06odevWWrlyZYlJywAAlMANII3nddhJSkoq9Wqss2fPKisrSzExMVq+fLkvtVXKkCFDNGTIkCp7PQCAIbgBpPG8Dju/+MUvSoQdh8Ohq666Si1atFDfvn1Vt25dnwsEAMDvuAGk0bwOOwsXLrSxDAAAAP/wejbWkSNHdOnSpTL3X7p0SUeOHPG2eQAAAFt4HXaSkpK0bNmyMvf/7W9/U1JSkrfNAwAA2MLrsGNVcGmd0+lURITfrmwHAADwSKXm7GRnZ+vs2bPux6dPny71VNXZs2e1ePFiNW7c2OcCAQAAfFGpoZepU6cqKSnJfdn58OHD3Y8L/9x2221asWKFBg0a5K+6AcBjn33vUMuXV2na6r2BLgVAAFRqZKdr166qVauWLMvS6NGj9cgjj+j2228vcozD4VBsbKzatGmjtm3b2losAFTWjLX7teJopCTprfQ9kqRhnZMDWRKAKlapsHPXXXe515u6cOGC+vTpo1atWvmlMADw1bTVe/XHNfuLbCPwAOHH6/vsTJgwwc46AMBW01bvdQeb4gg8QHjxaW2sy5cva+nSpdq6dauysrKUn190XRGHw6E///nPPhUIAJVVXtBxIfAA4cPrsHP48GF16tRJhw4dUp06dZSVlaW6devq7NmzysvLU0JCgmrVqmVnrQA8NG31Xk1N36MRXVqG3Ze5J0HHhcADhAevb4QzatQoZWVladOmTdqzZ48sy9KSJUt0/vx5vf7666pZs6Y+++wzO2sF4AHXl72lgi/zR+duUtKYT8LiSqTKBB2Xt9L3hMVnA4Qzr8POmjVr9Nxzz6ldu3bumwdalqXo6GiNGjVKnTt31vDhw+2qE4AHSvuy37D/tDv4mPyl7k3QcTH9swHCnddh5+LFi7r22mslSfHx8XI4HMrKynLvv+uuu/Tll1/6XCAAz3g6T8XEL3Vfgo6LqZ8NAB/CTrNmzfT9999LkqpVq6arr75amzZtcu/fuXOnatSo4XuFACpU2XkqJn2p2xF0XEz7bAAU8Drs3Hvvvfroo4/cj/v376+pU6dq4MCBGjBggGbOnKnevXvbUiSAsoX7PJWpNgUdf7UHIPC8vhprzJgx2rx5s3JychQdHa0XX3xRGRkZ+uCDDxQZGalHH31Ub731lp21AijG13kqUskrkULtSq4RXVraNrLjag+AWXw6jdWnTx9FR0dLkmrUqKF58+bpxx9/VGZmphYuXKj4+HjbCgVQlD/mqRS/kquyoz/TVu+19covT9ob1jlZqTYFlNQQCXgAKsenmwpKUk5OjrZu3aoffvhBHTp0UEJCgh11ASiH3fNUSvtz4ceeBIDCNdlx/5rKtOfa7stnQtABzOX1yI4kTZs2TY0bN9bdd9+thx56SN98840kKTMzUwkJCZo/f74tRQL4DzuDjstb6XvKXVqhopGa0mryZV6QN+35MsJD0AHM5nXYWbBggYYPH67u3bvrz3/+syzLcu9LSEjQvffeq8WLF9tSJID/CMQE2vKCRkVrUHlzKszb9rwJPAQdwHxeh50333xT999/vxYtWlTqVVdt2rTRjh07fCoOQEmBmkBbWtCw+94+drRXmcBD0AHCg9dhZ9++ferRo0eZ++vWravTp0972zyAMtg5IbeyCgcNu+/tY2d7nnxGBB0gfHg9QblOnTrKzMwsc//OnTvVqFEjb5sHUA47JuR66630Pdp04LQ27K/c/8yUN8nY23sFldVe4e2ltUvQAcKL1yM7PXv21Jw5c3T27NkS+3bs2KG5c+fqvvvu86U2AOUI5AhPZYOOi7enwirTXmHDOifr9/e2KLKNoAOEH6/DzqRJk5SXl6dWrVrppZdeksPh0LvvvqvHHntMbdu2VYMGDTR+/Hg7awVQTCADj7e8PRXmSXulGdKphXom5skhgg4Qrrw+jdWkSRNt2bJFL774opYsWSLLsvTee+8pLi5OjzzyiF577TXuuQNUgUCe0vKWt6fCymtPKvuUVremlv74TFdFRUXZ8noAQotP99lp0KCB5s2bpzNnzujkyZM6fvy4fvzxR82fP18NGjSwq0YAFQjFER67go4La1oBKEulws6LL77ovnFgcfXr11fDhg0VEeFTfgLgpVALPP/Vop6t7bGmFYCyVCqZvPbaa/r222/dj0+fPq3IyEitWbPG9sIAVJ6nl1wHOhSldmmpRQPvZE0rAFXC52GYwndOBhB4rsDjUMnREztCga8jMoVrsGM0iqADoCKccwIMNKxzsg6+1ss9elL4SiRfroDydUSmtGDCmlYA/M3nVc8BBLdhnZPdgcDXoFN4REaq3BVg5QUTu9sDgMIqHXYOHTqkrVu3SpKysrIkSXv37lWdOnVKPf7222/3vjoAtrEr6LhUJqB4Ekzsbg8AXCoddl5++WW9/PLLRbY999xzJY6zLEsOh0N5eXneVwfAFnYHHRdPAkplgond7Xli2uq9mpq+RyMIUICxKhV2FixY4K86APiJv4KOi91rUFXlmlaFP5uKbkwIIHRVKuw88cQT/qoDgB/YsRyDJ0oLKL4EE7vbK01pnw2BBzATE5QBQ9m17pTk2Ze/6xi7TgnZ3V5h5X02BB7APIQdwEB2juhUNvDYGRLsbk/y7LMh8ABm4T47gIHsXieqovamrd6rpDGflLv6eDCoTAisaDV1AKGDsAMYyO51osprzxUgLAV3QJixdn+lR7uC+f0A8BynsQADeXOTvrKUNzE4VCb5fva9QyuO7vfqucH4fgBUDiM7gKH8ve5URZN8g2VEZMba/VpxNNKnNoLp/QCoPMIOYDB/rTvl6STfQAeEaav36o9rvBvRKS4Y3g8A7xB2AMN5E3h8DTougQ4IVT1RG0BwIuwAYaAygceuoOMSyMBTlRO1AQQvwg4QJjwJPHYHHZdABZ5hnZP1+3tb2NIWi48CoYuwA4SR8gKPv4KOS6ACz5BOLdQz0bcFiQk6QGgj7ABhprTA4++g4xKowNOtqeX1CA9BBwh9hB0gDLkCj0MVf5mbMsl3SKcWtk7UBhA6uKkgEKY8XXdqRJeWtq6cHshJvpW52SJBBzAHIzsAymXHzQldgiFA+DpRG0DoIewAqJC/78Zc1bydqA0gNBF2AHjEX3djDpTKTtQGELqYswPAY94sMBrMAcJV19T0PRoRxHUC8A1hB0ClmDbJ19OJ2gBCF6exAFQak3wBhBLCDgCvMMkXQKgg7ADwGpN8AYQC5uwA8AmTfAEEO8IOAJ8xyRdAMOM0FgAAMFpIh51Dhw5pwIABSkpKUs2aNdWiRQtNmDBBubm5gS4NAAAEiZA+jbVr1y7l5+frnXfe0XXXXadvv/1WAwcO1IULFzRlypRAlwcAAIJASIed7t27q3v37u7HzZs31+7duzVr1izCDgAAkBTiYac0WVlZqlu3brnH5OTkKCcnx/04OztbkuR0OuV0Ov1an7+46g7V+k1CXwQX+iN40BfBw5S+8LR+h2VZlp9rqTL79u1TmzZtNGXKFA0cOLDM49LS0jRx4sQS2xctWqSYmBh/lggAAGxy8eJFPfroo8rKylJ8fHyZxwVl2BkzZoxef/31co/57rvvdMMNN7gfHzt2TL/4xS/UsWNHzZs3r9znljayk5iYqMzMzHI/rGDmdDqVnp6uLl26KCoqKtDlhDX6IrjQH8GDvggepvRFdna2EhISKgw7QXkaa+TIkerfv3+5xzRv3tz954yMDHXq1En/9V//pTlz5lTYfnR0tKKjo0tsj4qKCulOl8x4D6agL4IL/RE86IvgEep94WntQRl26tevr/r163t07LFjx9SpUye1adNGCxYsUERESF9NDwAAbBaUYcdTx44dU8eOHXXNNddoypQpOnXqlHtfo0aNAlgZAAAIFiEddtLT07Vv3z7t27dPTZs2LbIvCKciAQCAAAjpcz79+/eXZVml/gAAAEghHnYAAAAqQtgBAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQcAABiNsAMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMBphBwAAGI2wAwAAjEbYAQAARiPsAAAAoxF2AACA0Qg7AADAaIQdAABgNMIOAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQcAABiNsAMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMBphBwAAGI2wAwAAjEbYAQAARiPsAAAAoxF2AACA0Qg7AADAaIQdAABgNMIOAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQcAABiNsAMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMJoxYScnJ0etW7eWw+HQtm3bAl0OAAAIEsaEndGjR6tJkyaBLgMAAAQZI8LOp59+qlWrVmnKlCmBLgUAAASZaoEuwFcnT57UwIEDtXz5csXExHj0nJycHOXk5LgfZ2dnS5KcTqecTqdf6vQ3V92hWr9J6IvgQn8ED/oieJjSF57W77Asy/JzLX5jWZZ69uypDh066KWXXtKhQ4eUlJSkr7/+Wq1bty7zeWlpaZo4cWKJ7YsWLfI4MAEAgMC6ePGiHn30UWVlZSk+Pr7M44Iy7IwZM0avv/56ucd89913WrVqlf7v//5P69atU2RkpMdhp7SRncTERGVmZpb7YQUzp9Op9PR0denSRVFRUYEuJ6zRF8GF/gge9EXwMKUvsrOzlZCQUGHYCcrTWCNHjlT//v3LPaZ58+Zas2aNNm7cqOjo6CL72rZtq379+undd98t9bnR0dElniNJUVFRId3pkhnvwRT0RXChP4IHfRE8Qr0vPK09KMNO/fr1Vb9+/QqPmzZtmiZNmuR+nJGRoW7dumnJkiVq3769P0sEAAAhIijDjqeaNWtW5HGtWrUkSS1atFDTpk0DURIAAAgyRlx6DgAAUJaQHtkp7tprr1UQzrcGAAABxMgOAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQcAABiNsAMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIOwAAwGiEHQAAYDTCDgAAMBphBwAAGI2wAwAAjEbYAQAARiPsAAAAoxF2AACA0Qg7AADAaIQdAABgNMIOAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjVQt0AcHAsixJUnZ2doAr8Z7T6dTFixeVnZ2tqKioQJcT1uiL4EJ/BA/6IniY0heu723X93hZCDuSzp07J0lKTEwMcCUAAKCyzp07p9q1a5e532FVFIfCQH5+vjIyMhQXFyeHwxHocrySnZ2txMREHT16VPHx8YEuJ6zRF8GF/gge9EXwMKUvLMvSuXPn1KRJE0VElD0zh5EdSREREWratGmgy7BFfHx8SP/FNQl9EVzoj+BBXwQPE/qivBEdFyYoAwAAoxF2AACA0Qg7hoiOjtaECRMUHR0d6FLCHn0RXOiP4EFfBI9w6wsmKAMAAKMxsgMAAIxG2AEAAEYj7AAAAKMRdgAAgNEIO4bLyclR69at5XA4tG3btkCXE3YOHTqkAQMGKCkpSTVr1lSLFi00YcIE5ebmBrq0sDBz5kxde+21qlGjhtq3b6+vvvoq0CWFncmTJ+uOO+5QXFycGjRooAceeEC7d+8OdFmQ9Nprr8nhcGj48OGBLsXvCDuGGz16tJo0aRLoMsLWrl27lJ+fr3feeUc7duzQ1KlTNXv2bL344ouBLs14S5YsUWpqqiZMmKCtW7fq1ltvVbdu3fTDDz8EurSwsm7dOg0ePFibNm1Senq6nE6nunbtqgsXLgS6tLC2efNmvfPOO7rlllsCXUqV4NJzg3366adKTU3V0qVLdfPNN+vrr79W69atA11W2HvjjTc0a9YsHThwINClGK19+/a64447NGPGDEkFa+AlJiZq6NChGjNmTICrC1+nTp1SgwYNtG7dOt1zzz2BLicsnT9/Xrfffrv+9Kc/adKkSWrdurXefvvtQJflV4zsGOrkyZMaOHCg3nvvPcXExAS6HBSSlZWlunXrBroMo+Xm5mrLli1KSUlxb4uIiFBKSoo2btwYwMqQlZUlSfw3EECDBw9Wr169ivz3YToWAjWQZVnq37+/Bg0apLZt2+rQoUOBLgk/2bdvn6ZPn64pU6YEuhSjZWZmKi8vTw0bNiyyvWHDhtq1a1eAqkJ+fr6GDx+uDh06qFWrVoEuJywtXrxYW7du1ebNmwNdSpViZCeEjBkzRg6Ho9yfXbt2afr06Tp37pzGjh0b6JKN5WlfFHbs2DF1795dDz/8sAYOHBigyoHAGTx4sL799lstXrw40KWEpaNHj+r3v/+93n//fdWoUSPQ5VQp5uyEkFOnTun06dPlHtO8eXP9+te/1t///nc5HA739ry8PEVGRqpfv3569913/V2q8Tzti+rVq0uSMjIy1LFjR915551auHChIiL4/wx/ys3NVUxMjD744AM98MAD7u1PPPGEzp49q48++ihwxYWpIUOG6KOPPtL69euVlJQU6HLC0vLly/Xggw8qMjLSvS0vL08Oh0MRERHKyckpss8khB0DHTlyRNnZ2e7HGRkZ6tatmz744AO1b99eTZs2DWB14efYsWPq1KmT2rRpo7/+9a/G/mMSbNq3b6927dpp+vTpkgpOoTRr1kxDhgxhgnIVsixLQ4cO1bJly/TFF18oOTk50CWFrXPnzunw4cNFtj355JO64YYb9MILLxh9apE5OwZq1qxZkce1atWSJLVo0YKgU8WOHTumjh076pprrtGUKVN06tQp975GjRoFsDLzpaam6oknnlDbtm3Vrl07vf3227pw4YKefPLJQJcWVgYPHqxFixbpo48+UlxcnE6cOCFJql27tmrWrBng6sJLXFxciUATGxurevXqGR10JMIO4Ffp6enat2+f9u3bVyJoMqjqX7/5zW906tQpjR8/XidOnFDr1q21cuXKEpOW4V+zZs2SJHXs2LHI9gULFqh///5VXxDCEqexAACA0ZglCQAAjEbYAQAARiPsAAAAoxF2AACA0Qg7AADAaIQdAABgNMIOAAAwGmEHAAAYjbADAACMRtgBAABGI+wA8NrChQvlcDhK/fHXyuIbNmxQWlqazp4965f2AZiHhUAB+OwPf/iDkpKSimzz1yrKGzZs0MSJE9W/f3/VqVPHL68BwCyEHQA+69Gjh9q2bRvoMnx24cIFxcbGBroMADbjNBYAvzt27JieeuopNWzYUNHR0br55ps1f/78IsccPnxYzz33nK6//nrVrFlT9erV08MPP6xDhw65j0lLS9OoUaMkSUlJSe5TZq5j+vfvr2uvvbbE66elpcnhcJS6befOnXr00Ud11VVX6e677/a43vLea40aNfTUU08V2f75558rKipKI0aM8KgdAPZhZAeAz7KyspSZmVlkW0JCgiTp5MmTuvPOO+VwODRkyBDVr19fn376qQYMGKDs7GwNHz5ckrR582Zt2LBBffv2VdOmTXXo0CHNmjVLHTt21M6dOxUTE6OHHnpIe/bs0f/+7/9q6tSp7teoX7++17U//PDDSk5O1n//93/LsiyP6y3L1Vdfraefflpz5szRhAkTdM0112jXrl16+OGH1aNHD7355pte1wrASxYAeGnBggWWpFJ/XAYMGGA1btzYyszMLPLcvn37WrVr17YuXrxoWZbl/l3Yxo0bLUnWX/7yF/e2N954w5JkHTx4sMTxTzzxhHXNNdeU2D5hwgSr+D93rm2PPPJIke2e1lue77//3oqOjraeffZZKzMz02rRooXVunVr6/z58xU+F4D9OI0FwGczZ85Uenp6kR9JsixLS5cuVe/evWVZljIzM90/3bp1U1ZWlrZu3SpJqlmzprs9p9Op06dP67rrrlOdOnXcx/jDoEGD3H+uTL3lufrqqzVw4EDNnz9fvXr10qVLl/Txxx8zHwgIEE5jAfBZu3btSp2gfOrUKZ09e1Zz5szRnDlzSn3uDz/8IEm6dOmSJk+erAULFujYsWOyLMt9TFZWln8Kl4pcRVaZeivy/PPPa8aMGfrmm2/0j3/8Q1dffXWR/bNmzdLcuXO1fft2jRs3TmlpaV6/BwDlI+wA8Jv8/HxJ0mOPPaYnnnii1GNuueUWSdLQoUO1YMECDR8+XHfddZdq164th8Ohvn37utupSPFJyC55eXllPqfwiFJl6q3Iq6++Kkm6cuWK6tatW2J/48aNlZaWpkWLFnnUHgDvEXYA+E39+vUVFxenvLw8paSklHvsBx98oCeeeKLIBN7Lly+XuHlgWYFGkq666qpSbzZ4+PBh2+stzxtvvKF58+ZpxowZGjVqlF599VXNmzevyDEPPPCAJGnFihVevw4AzzBnB4DfREZGqk+fPlq6dKm+/fbbEvtPnTpV5NjCp64kafr06SVGZVzzXkoLNS1atFBWVpa++eYb97bjx49r2bJlttdbluXLl2vMmDF65ZVXNHjwYD3zzDP6y1/+ooMHD3pUAwD7MbIDwK9ee+01rV27Vu3bt9fAgQN100036cyZM9q6das+//xznTlzRpL0y1/+Uu+9955q166tm266SRs3btTnn3+uevXqFWmvTZs2kqRx48apb9++ioqKUu/evRUbG6u+ffvqhRde0IMPPqhhw4bp4sWLmjVrllq2bOnxJGdP6y3Nli1b1K9fP/Xr10/jxo2TJI0ePVqzZ88udXQHQNUg7ADwq4YNG+qrr77SH/7wB3344Yf605/+pHr16unmm2/W66+/7j7uj3/8oyIjI/X+++/r8uXL6tChgz7//HN169atSHt33HGHXnnlFc2ePVsrV65Ufn6+Dh48qNjYWNWrV0/Lli1TamqqRo8eraSkJE2ePFl79+71OOx4Wm9x33//vXr37q3bbrtNc+fOdW9v0qSJnnrqKc2bN0/jxo0rsawGAP9zWMXHjQEAVWbQoEFq1KgRV2MBfsScHQAIgCtXrujy5cvKy8sr8mcA9mNkBwACIC0tTRMnTiyybcGCBerfv39gCgIMRtgBAABG4zQWAAAwGmEHAAAYjbADAACMRtgBAABGI+wAAACjEXYAAIDRCDsAAMBohB0AAGA0wg4AADAaYQcAABjt/wOLkSR8+oGcqgAAAABJRU5ErkJggg==",
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.plot(\n",
" X_train[y_train == 0, 0],\n",
" X_train[y_train == 0, 1],\n",
" marker=\"D\",\n",
" markersize=10,\n",
" linestyle=\"\",\n",
" label=\"Class 0\",\n",
")\n",
"\n",
"plt.plot(\n",
" X_train[y_train == 1, 0],\n",
" X_train[y_train == 1, 1],\n",
" marker=\"^\",\n",
" markersize=13,\n",
" linestyle=\"\",\n",
" label=\"Class 1\",\n",
")\n",
"\n",
"plt.legend(loc=2)\n",
"\n",
"plt.xlim([-5, 5])\n",
"plt.ylim([-5, 5])\n",
"\n",
"plt.xlabel(\"Feature $x_1$\", fontsize=12)\n",
"plt.ylabel(\"Feature $x_2$\", fontsize=12)\n",
"\n",
"plt.grid()\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 90,
"id": "5d9f1813-d232-4a7d-aebc-cfd5303fae48",
"metadata": {},
"outputs": [],
"source": [
"def plot_boundary(model):\n",
"\n",
" w1 = model.linear.weight[0][0].detach()\n",
" w2 = model.linear.weight[0][1].detach()\n",
" b = model.linear.bias[0].detach()\n",
"\n",
" x1_min = -20\n",
" x2_min = (-(w1 * x1_min) - b) / w2\n",
"\n",
" x1_max = 20\n",
" x2_max = (-(w1 * x1_max) - b) / w2\n",
"\n",
" return x1_min, x1_max, x2_min, x2_max"
]
},
{
"cell_type": "code",
"execution_count": 91,
"id": "2d71b5df-dd8d-41d5-b6fd-640d6f40d5a2",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAG2CAYAAACZEEfAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABgRElEQVR4nO3dd1gUV8MF8DMLSBNs2EVdql0U1BiNFXvvNUrsihXsiqJii70XVCzRaGLX2Ai22I0VRZGy9hIrqCgs7H5/+MIHCgjLLrM7nN/z7GO2zR6YCMc7d+YKarVaDSIiIiKJkokdgIiIiEiXWHaIiIhI0lh2iIiISNJYdoiIiEjSWHaIiIhI0lh2iIiISNJYdoiIiEjSjMUOoA9UKhWePn0KKysrCIIgdhwiIiLKALVajffv36NYsWKQydIev2HZAfD06VPY2tqKHYOIiIg08OjRI5QoUSLN51l2AFhZWQH48s2ytrYWOY1mlEoljh07hsaNG8PExETsODka94V+4f7QH9wX+kMq+yI6Ohq2trZJv8fTwrIDJB26sra2NuiyY2FhAWtra4P+H1cKuC/0C/eH/uC+0B9S2xffm4LCCcpEREQkaSw7REREJGksO0RERCRpnLOTQSqVCnFxcWLHSJNSqYSxsTE+f/6MhIQEsePoRK5cudI9tZCIiCg1LDsZEBcXB4VCAZVKJXaUNKnVahQpUgSPHj2S7LWCZDIZ5HI5cuXKJXYUIiIyICw736FWq/Hs2TMYGRnB1tZWb0cWVCoVPnz4gNy5c+ttxqxIvPDjs2fPULJkSckWOiIi0j6Wne+Ij49HTEwMihUrBgsLiyxta2lQGBYF3sOoRk4Y3tBRSwm/SDzMZmZmJsmyAwAFCxbE06dPER8fL4lTJYmIKHtI87eiFiXOf8nqoZOlQWFYGHgPagALA+9haVCYFtLlLIn7QKpzkoiISDdYdjIoK4dNEotOciw8mcdDV0REpAmWHR1LregkYuEhIiLSPZYdHUqv6CRi4SEiItItlh0dyUjRSSRm4REEAXv37hXls4mIiLIDy44OZKboJNJF4Xn+/DmGDRsGOzs7mJqawtbWFq1atUJQUJBWP0dTarUaU6ZMQdGiRWFubg53d3eEhXGUi4iItItlR8s0KTqJtFl47t+/D1dXVxw/fhzz5s1DcHAwjhw5gvr168PT01Mrn5FVv/76K5YuXYrVq1fj4sWLsLS0RJMmTfD582exoxERkYSw7GhRVopOIm0VniFDhkAQBFy6dAkdOnSAk5MTypcvDy8vL1y4cCHN940bNw5OTk6wsLCAnZ0dfHx8oFQqk56/ceMG6tevDysrK1hbW8PV1RX//vsvAODBgwdo1aoV8uXLB0tLS5QvXx6HDh1K9XPUajUWL16MyZMno02bNqhUqRI2b96Mp0+f8rAaERFpFS8qqCXaKDqJErej6YUH37x5gyNHjmDmzJmwtLT85vm8efOm+V4rKyts3LgRxYoVQ3BwMPr37w8rKyuMHTsWANCjRw9UqVIFq1atgpGREa5fv550gT9PT0/ExcXh9OnTsLS0REhICHLnzp3q5ygUCjx//hzu7u5Jj+XJkwc1atTA+fPn0bVrV42+diIioq+x7GjJIi0VneTb07TshIeHQ61Wo0yZMpl+7+TJk5P+u3Tp0hg9ejS2b9+eVHYePnyIMWPGJG3b0fH/Mz58+BAdOnRAxYoVAQB2dnZpfs7z588BAIULF07xeOHChZOeIyIi0gYextKSUY2c9GZ7arVa4/fu2LEDtWrVQpEiRZA7d25MnjwZDx8+THrey8sL/fr1g7u7O+bMmYOIiIik54YPHw4/Pz/UqlULU6dOxc2bNzXOQUREpC0sO1oyvKEjvLRUeLyyuHaWo6MjBEHA3bt3M/W+8+fPo0ePHmjevDkOHjyIa9euYdKkSYiLi0t6ja+vL27fvo0WLVrg+PHjKFeuHPbs2QMA6NevHyIjI/Hzzz8jODgYbm5uWLZsWaqfVaRIEQDAixcvUjz+4sWLpOeIiIi0gWVHi7RReLJadAAgf/78aNKkCVasWIGPHz9+8/y7d+9Sfd+5c+dQqlQpTJo0CW5ubnB0dMSDBw++eZ2TkxNGjRqFY8eOoX379ggICEh6ztbWFoMGDcLu3bvh7e0Nf3//VD9LLpejSJEiKU6Dj46OxsWLF1GzZs1MfsVERERpY9nRsqwUHm0UnUQrVqxAQkICqlevjl27diEsLAx37tzB0qVL0ywTjo6OePjwIbZv346IiAgsXbo0adQGAD59+oShQ4fi5MmTePDgAc6ePYvLly+jbNmyAICRI0fi6NGjUCgUuHr1Kk6cOJH03NcEQcDIkSPh5+eH/fv3Izg4GL169UKxYsXQtm1brXwPiIiIAE5Q1onEwpKZs7O0WXSAL5ODr169ipkzZ8Lb2xvPnj1DwYIF4erqilWrVqX6ntatW2PUqFEYOnQoYmNj0aJFC/j4+MDX1xcAYGRkhNevX6NXr1548eIFbGxs0L59e0ybNg3Al9XIPT098fjxY1hbW6Np06ZYtGhRmhnHjh2Ljx8/YsCAAXj37h1q166NI0eOwMzMTGvfByIiIkGdldmsEhEdHY08efIgKioK1tbWKZ77/PkzFAoF5HJ5pn8JZ/R0dG0UHZVKhejoaFhbW0Mmk+aAXVb2RXZSKpU4dOgQmjdvnnRaPomH+0N/cF/oD6nsi/R+fycnzd+KeiIjh7S0PaJDREREKbHs6Fh6hYdFh4iISPdYdrJBaoWHRYeIiCh7cIJyNkksNosC72EUiw4REVG2YdnJRsMbOrLkEBERZTMexspOESeA5dW//ElERETZgmUnu6jVQNA04FXolz95xj8REVG2kFzZmTNnTtLVefVKRBDw9NqX/3567ct9IiIi0jlJlZ3Lly9jzZo1qFSpkthRUlKrgeN+gGD05b5g9OW+HozuCIKAvXv3ih2DiIhIZyRTdj58+IAePXrA398f+fLlEztOSomjOuqEL/fVCdkyuvP8+XMMGzYMdnZ2MDU1ha2tLVq1apVi8U0x7d69G40bN0aBAgUgCAKuX78udiQiIpIgyZyN5enpiRYtWsDd3R1+fn7pvjY2NhaxsbFJ96OjowF8uXy2UqlM8VqlUgm1Wg2VSgWVSpX5YGo1hP+N6giJZQeA+n+jO2p5fUAQMr/dbz5GnfSnSqXC/fv38dNPPyFv3ryYO3cuKlasCKVSiWPHjsHT0xMhISFJ79X4a8ui9+/fo1atWujYsSMGDhz43RwqlQpqtRpKpRJGRkbZmDRzEv8f+vr/JRIH94f+4L7QH1LZFxnNL4mys337dly9ehWXL1/O0Otnz56dtHhlcseOHYOFhUWKx4yNjVGkSBF8+PABcXFxmc5mfP8UcifO1UlG+N/ozsfgg4gvXTfT203L+/fvAQADBw4E8OVrsrS0THq+b9++6NixY1LBA76sZp54f+rUqfjrr7/w9OlTFCpUCJ06dcLYsWOT1k4JDg7GxIkTcf36dQiCADs7OyxatAhVqlTBw4cPMXbsWFy4cAFKpRIlS5bEtGnT0Lhx41SztmnTBgDw8OFDAMDHjx9T5PpaXFwcPn36hNOnTyM+Pl7Tb1G2CQwMFDsCJcP9oT+4L/SHoe+LmJiYDL3O4MvOo0ePMGLECAQGBmZ4ccgJEybAy8sr6X50dDRsbW3RuHHjVBcCffToEXLnzp35xSfVagiXFkH91ahO0tOCESwvLYK6Ysssj+6o1Wq8f/8eVlZWePv2LYKCguDn54eiRYt+89qvv0Zzc/Okx2xsbLBx40YUK1YMwcHBGDhwIGxsbDBmzBgAwODBg+Hi4oI1a9bAyMgI169fR968eWFtbY0JEyYgISEBp06dgqWlJUJCQmBtbZ3u4mwAkDt3bgCApaVluq/9/PkzzM3NUadOHb1fCDQwMBCNGjUy6AX2pIL7Q39wX+gPqeyL9P6BnJzBl50rV67gv//+Q9WqVZMeS0hIwOnTp7F8+XLExsZ+c8jD1NQUpqam32zLxMTkm52ekJAAQRAgk8kyv5p4+N//fwZWKhJHdwTFCcDBPXPb/kri4R9BEBAZGQm1Wo2yZctmKHPyr83HxyfpcTs7O4SFhWH79u0YN24cgC+jMGPGjEG5cuUAAM7Ozkmvf/ToETp06IDKlSsDABwcHDKUPfGzv/c9lslkEAQh1f2kjwwlZ07B/aE/uC/0h6Hvi4xmN/iy07BhQwQHB6d47JdffkGZMmUwbtw48eZ2JD8DK5VRnSSJZ2bZN9TK3J0vH635WV47duzA0qVLERERgQ8fPiA+Pj7FaIuXlxf69euHLVu2wN3dHZ06dYK9vT0AYPjw4Rg8eDCOHTsGd3d3dOjQQf/OjCMiohzH4M/GsrKyQoUKFVLcLC0tUaBAAVSoUEG8YF+fgZUWHZyZ5ejoCEEQcPfu3Uy97/z58+jRoweaN2+OgwcP4tq1a5g0aVKKuUq+vr64ffs2WrRogePHj6NcuXLYs2cPAKBfv36IjIzEzz//jODgYLi5uWHZsmVa+7qIiIg0YfBlRy99fV2d79HydXfy58+PJk2aYMWKFfj48eM3z7979y7V9507dw6lSpXCpEmT4ObmBkdHRzx48OCb1zk5OWHUqFE4duwY2rdvj4CAgKTnbG1tMWjQIOzevRve3t7w9/fXytdERESkKYM/jJWakydPihsg+dWSMyL56E4W5+4kWrFiBWrVqoXq1atj+vTpqFSpEuLj4xEYGIhVq1bhzp0737zH0dERDx8+xPbt21GtWjX89ddfSaM2wJeztsaMGYOOHTtCLpfj8ePHuHz5Mjp06AAAGDlyJJo1awYnJye8ffsWJ06cQNmyZdPM+ObNGzx8+BBPnz4FAISGhgIAihQpgiJFimjl+0BERMSRHW3L7KhOIi2P7tjZ2eHq1auoX78+vL29UaFCBTRq1AhBQUFYtWpVqu9p3bo1Ro0ahaFDh8LFxQXnzp1LMWHZyMgIr1+/Rq9eveDk5ITOnTujWbNmSafxJyQkwNPTE2XLlkXTpk3h5OSElStXpplx//79qFKlClq0aAEA6Nq1K6pUqYLVq1dr5XtAREQEAII6K7NZJSI6Ohp58uRBVFRUqqeeKxQKyOXyjJ3uHP438FsHzcP03KXR6I5KpUJ0dDSsra0zf9aYgcj0vhCJUqnEoUOH0Lx5c4M+y0EquD/0B/eF/pDKvkjv93dy0vytKBZNR3US6dGaWURERFLBsqNNGT0DKy3ZtGYWERFRTsKyoy2JozpZ/pbKOLpDRESkRSw72pIQB0Q9AZDVBTVVQPSTL9sjIiKiLJPkqee68N153MamwIATwMdXWf8wy4JftkcpcC49ERFpgmXnOxKXm4iLi4O5uXn6L85T4suNdCLxSs6iLQFCREQGiWXnO4yNjWFhYYGXL1/CxMREb0/rVqlUiIuLw+fPn/U2Y1aoVCq8fPkSFhYWMDbm/7ZERJRx/K3xHYIgoGjRolAoFKkunaAv1Go1Pn36BHNzcwhaWlBU38hkMpQsWVKyXx8REekGy04G5MqVC46OjikWxNQ3SqUSp0+fRp06dQz6AlHpyZUrlyRHrYiISLdYdjJIJpPp9VV7jYyMEB8fDzMzM8mWHSIiIk3wn8lEREQkaSw7REREJGksO8l0794dN2/eFDsGERERaRHLTjJ//fUXKleujC5duuDOnTtixyEiIiItYNlJpn379gCAP/74AxUqVECvXr0QEREhcioiIiLKCpadZAICAnDjxg20bdsWKpUKW7ZsgbOzM/r376/X19ghIiKitLHsfKVSpUrYs2cPLl++jGbNmiEhIQHr1q2Do6MjPD098fTpU7EjEhERUSaw7KTBzc0Nhw4dwtmzZ9GgQQMolUqsXLkS9vb28PLywn///Sd2RCIiIsoAlp3v+PHHHxEUFIQTJ06gdu3a+Pz5MxYtWgQ7OztMmDABb968ETsiERERpYNlJ4Pq1auH06dP48iRI6hWrRo+fvyIOXPmQC6Xw9fXF1FRUWJHJCIiolSw7GSCIAho0qQJLl68iP3796Ny5cqIjo7GtGnTIJfLMXv2bHz48EHsmERERJQMy44GBEFAq1atcPXqVfz5558oW7Ys3r59i4kTJ8LOzg4LFizAp0+fxI5JREREYNnJEplMho4dOyI4OBi//fYbHBwc8PLlS4wePRr29vZYvnw5YmNjxY5JRESUo7HsaIGRkRF69OiBO3fuYP369ShVqhSePXuGYcOGwdHREf7+/lAqlWLHJCIiypFYdrTI2NgYffr0wb1797Bq1SoUL14cjx49woABA1CmTBls3rwZCQkJYsckIiLKUVh2dCBXrlwYNGgQwsPDsXjxYhQuXBiRkZHo3bs3ypcvj+3bt0OlUokdk4iIKEdg2dEhMzMzjBgxAhEREZg7dy7y58+P0NBQdOvWDS4uLtizZw/UarXYMYmIiCSNZScbWFpaYuzYsVAoFJg+fTry5MmD4OBgtG/fHtWqVcOhQ4dYeoiIiHSEZScbWVtbw8fHBwqFApMmTULu3Llx5coVtGjRArVq1UJQUBBLDxERkZax7IggX7588PPzg0KhwJgxY2Bubo7z58/D3d0d9evXxz///CN2RCIiIslg2RGRjY0Nfv31V0RGRmL48OHIlSsXTp06hTp16qBJkya4dOmS2BGJiIgMHsuOHihSpAiWLFmC8PBwDBw4EMbGxjh27Bhq1KiB1q1b4/r162JHJCIiMlgsO3rE1tYWq1evRmhoKDw8PCCTyXDgwAFUqVIFnTp1QkhIiNgRiYiIDA7Ljh6ys7NDQEAAQkJC0K1bNwiCgJ07d6JChQro2bMnwsLCxI5IRERkMFh29JizszO2bduGmzdvokOHDlCr1di6dSvKli2LPn364P79+2JHJCIi0nssOwagQoUK2LlzJ65evYqWLVsiISEBAQEBcHJywuDBg/H48WOxIxIREektlh0DUqVKFRw4cAAXLlxAo0aNoFQqsXr1ajg4OMDb2xtv374VOyIREZHeYdkxQDVq1MCxY8eSTlOPjY3FsmXLMHDgQEyYMAGvXr0SOyIREZHeYNkxYHXq1MHJkycRGBiIGjVqIC4uDgsWLIBcLoePjw/evXsndkQiIiLRsewYOEEQ4O7ujtOnT2Py5MlwcXHBhw8f4OfnB7lcDj8/P7x//17smERERKJh2ZEIQRDg5uaGixcvYvfu3ahQoQLevXsHHx8fyOVyzJs3DzExMWLHJCIiynYsOxIjCALatWuHGzdu4Pfff4eTkxNev36NsWPHws7ODkuWLMHnz5/FjklERJRtWHYkSiaToWvXrrh9+zY2btwIuVyOFy9eYOTIkXBwcMDq1asRFxcndkwiIiKdY9mROGNjY/Tu3RuhoaFYs2YNSpQogSdPnmDw4MFwdnZGQEAA4uPjxY5JRESkMyw7OYSJiQkGDBiAsLAwLF26FEWKFMH9+/fRp08flCtXDtu2bUNCQoLYMYmIiLSOZSeHMTMzw7BhwxAREYH58+fDxsYGYWFh6NGjBypVqoRdu3ZBpVKJHZOIiEhrWHZyKAsLC3h7eyMyMhIzZ85E3rx5ERISgo4dO8LV1RUHDhyAWq0WOyYREVGWsezkcFZWVpg4cSIUCgWmTJkCKysrXL9+Ha1bt8YPP/yAY8eOsfQQEZFBY9khAEDevHkxbdo0KBQKjBs3DhYWFrh06RKaNGmCunXr4tSpU2JHJCIi0gjLDqVQoEABzJkzB5GRkRg1ahRMTU3xzz//oF69enB3d8f58+fFjkhERJQpLDuUqsKFC2PhwoWIiIjAkCFDYGJigqCgIPz4449o0aIFrly5InZEIiKiDGHZoXQVL14cK1asQFhYGPr27QsjIyMcOnQIbm5uaN++PYKDg8WOSERElC6WHcqQUqVKYd26dbh79y569uwJQRCwZ88eVK5cGd26dUNoaKjYEYmIiFLFskOZ4uDggC1btuDWrVvo1KkT1Go1tm/fjnLlysHDwwORkZFiRyQiIkqBZYc0Uq5cOfzxxx+4fv062rRpA5VKhU2bNsHZ2RkDBgzAw4cPxY5IREQEgGWHsqhy5crYu3cvLl26hKZNmyI+Ph7+/v5wdHTEsGHD8OzZM7EjEhFRDseyQ1pRrVo1HD58GGfOnEH9+vURFxeH5cuXw87ODqNHj8bLly/FjkhERDkUyw5pVa1atXD8+PGk09Q/f/6MBQsWQC6XY9KkSXjz5o3YEYmIKIdh2SGdaNCgAc6cOYPDhw/Dzc0NHz9+xKxZsyCXyzFt2jRER0eLHZGIiHIIlh3SGUEQ0LRpU1y6dAl79+5FpUqVEB0dDV9fX8jlcsyZMwcfP34UOyYREUkcyw7pnCAIaNOmDa5du4YdO3agTJkyePPmDSZMmAC5XI5Fixbh06dPYsckIiKJMviyM3v2bFSrVg1WVlYoVKgQ2rZtywvc6SmZTIbOnTvj1q1b2LJlC+zt7fHy5Ut4eXnBwcEBK1asQGxsrNgxiYhIYgy+7Jw6dQqenp64cOECAgMDoVQq0bhxYx4e0WNGRkbo2bMn7ty5g3Xr1qFkyZJ4+vQphg4dCicnJ6xfvx5KpVLsmEREJBEGX3aOHDkCDw8PlC9fHpUrV8bGjRvx8OFDLlRpAExMTNC3b1/cu3cPK1asQLFixfDw4UP069cPZcuWxZYtW5CQkCB2TCIiMnDGYgfQtqioKABA/vz503xNbGxsisMliWcGKZVKgx1RSMxtiPllMhn69++Pnj17Yu3atZg3bx4iIiLQq1cvzJo1Cz4+PujQoQNkMsPo5oa8L6SI+0N/cF/oD6nsi4zmF9RqtVrHWbKNSqVC69at8e7dO5w5cybN1/n6+mLatGnfPL5t2zZYWFjoMiJlwOfPn3Ho0CHs2bMH79+/BwCULl0a3bp1Q/Xq1SEIgsgJiYhIH8TExKB79+6IioqCtbV1mq+TVNkZPHhw0lV8S5QokebrUhvZsbW1xatXr9L9ZukzpVKJwMBANGrUCCYmJmLH0Yro6GgsXboUixcvThp9c3V1xdSpU9GkSRO9LT1S3BeGjPtDf3Bf6A+p7Ivo6GjY2Nh8t+xI5jDW0KFDcfDgQZw+fTrdogMApqamMDU1/eZxExMTg97pgDS+hkQFChTAtGnTMGLECCxYsABLlizBlStX0Lp1a/z444/w8/ND/fr1xY6ZJintCyng/tAf3Bf6w9D3RUazG8YkiHSo1WoMHToUe/bswfHjxyGXy8WORFqWP39+zJw5EwqFAt7e3jAzM8O5c+fQoEEDNGjQAGfPnhU7IhER6TGDLzuenp747bffsG3bNlhZWeH58+d4/vw5L1InQQULFsT8+fMRGRmJoUOHIleuXDhx4gRq166NZs2a4fLly2JHJCIiPWTwZWfVqlWIiopCvXr1ULRo0aTbjh07xI5GOlK0aFEsW7YMYWFhGDBgAIyNjXHkyBFUr14dbdq0wY0bN8SOSEREesTgy45arU715uHhIXY00rGSJUtizZo1CA0NRe/evSGTybB//364uLigc+fOCAkJETsiERHpAYMvO0R2dnbYuHEjbt++ja5du0IQBPz555+oUKECfv75Z4SHh4sdkYiIRMSyQ5JRpkwZ/P7777hx4wbatWsHtVqN3377DWXKlEG/fv3w4MEDsSMSEZEIWHZIcipWrIjdu3fjypUraNGiBRISErB+/Xo4OjpiyJAhePLkidgRiYgoG7HskGRVrVoVBw8exLlz5+Du7g6lUolVq1bB3t4eo0aNwosXL8SOSERE2YBlhySvZs2aCAwMxMmTJ/HTTz8hNjYWixcvhp2dHcaPH4/Xr1+LHZGIiHSIZYdyjLp16+LUqVM4evQoqlevjpiYGMydOxdyuRxTp07Fu3fvxI5I2SHiBLC8+pc/iShHYNmhHEUQBDRu3BgXLlzAgQMH4OLigvfv32P69OmQy+WYNWsWPnz4IHZM0hW1GgiaBrwK/fJnZpYGZEkiMlgsO5QjCYKAli1b4sqVK9i5cyfKly+Pd+/eYdKkSZDL5Zg/fz5iYmLEjknaFhEEPL325b+fXvtyPyOyUpLSzMLyRJRdWHYoR5PJZOjQoQNu3LiBrVu3wtHREa9evcKYMWNgb2+PZcuWITY2VuyYpA1qNXDcDxCMvtwXjL7cz0hx0bQkpZdF2+WJiNLEskMEwMjICN27d0dISAgCAgJQunRpPH/+HMOHD4eDgwPWrl0LpVIpdkzKAiHyxJeiok748oA6IWPFJSslKS3aLk9ElC6WHaJkjI2N4eHhgdDQUKxevRrFixfH48ePMXDgQDg7O2PTpk2Ij48XOyZllloN2anZ/19YEmWkuCQWk8yWpHSyaL08EVG6WHaIUpErVy4MHDgQ4eHhWLJkCQoXLgyFQgEPDw+UL18ev//+O1QqldgxKYMKvg+G7FmywpLoe8Xl62KSKCsFRdvliYi+i2WHKB1mZmYYPnw4IiMj8euvv6JAgQK4d+8eunfvjsqVK2P37t1Q81/k+k2tRtmnu6D+urAkSq+4fF1MkrapYUHRRXkiou9i2SHKAAsLC4wZMwYKhQJ+fn7Imzcvbt26hQ4dOsDV1RV//fUXS4+eEiJPIN8nBYSvC0uitIpLWsUkacMaFBRtlyciyhCWHaJMsLKywqRJk6BQKODj44PcuXPj2rVraNmyZdKVmll69Mj/5uqovvejLrXiklYxSdp2JguKLsoTEWUIyw6RBvLmzYvp06dDoVBg7NixMDc3x8WLF9G4cWO4u7vj9u3bYkckAIgIguzZNcjwnflVXxeX7xWTRJqcvq6t8kREGcayQ5QFNjY2mDt3LiIjIzFixAiYmprin3/+waRJk9C8eXNcvHhR7Ig51/8KS5pzdb6WvLh8r5gkfYaGp69nJAMRaQ3LDpEWFClSBIsXL0Z4eDgGDBgAY2Nj/P333/jhhx/QqlUrXLt2TeyIOc//Ckuac3W+llhcwv/OWDFJpMnp69/LwNEdIq1i2SHSohIlSmD58uVYsWIFPDw8YGRkhIMHD6Jq1aro0KEDbt26JXbEnCGjIylfE4yAw+MyVkySPkvD09fTy8DRHSKtYtkh0oHChQtj7dq1uHPnDnr06AFBELB7925UqlQJ3bt3x71798SOKG0ZHUn5mjoBeBMBCJn80ajJ6evpZeDoDpFWsewQ6ZCjoyN+++03BAcHo2PHjlCr1fj9999RtmxZ/PLLL1AoFGJHlB5NR3VSbCOTF4zU9PT1tHB0h0irWHaIskH58uXx559/4tq1a2jVqhVUKhU2btwIJycnDBo0CI8ePRI7onRoOqqTVZqcvp4Wju4QaRXLDlE2cnFxwf79+5NOU4+Pj8eaNWvg4OCA4cOH49mzZ2JHNGyJIyli/GjT9PT1tAgy4PfuQPhx7WUkyqFYdohEUL16dRw9ehT//PMP6tati7i4OCxbtgz29vYYM2YMXr58KXZEw5QQB0Q9Ab53XR2dkWX+9PW0qFVAQixwaDQPZxFlEcsOkYhq166NEydO4O+//0bNmjXx6dMnzJ8/H3Z2dpg8eTLevn0rdkTDYmwKDDgBNJuX+vPN5gEDTn176xsImOfTQgAVEP0EiI/V3gjTm4gvp8MTkcaMxQ5AlNMJgoCGDRuiQYMGOHz4MHx8fHD16lXMnDkTy5cvh7e3N0aMGAFra2uxoxoG6+LAjW1fDh8lG1VRC0YQbmwDqvcHBOHb9w06A3x8lfXPtyz4ZfvaHGE6PgNwcE89NxF9l1bLTkxMDEJDQ+Hg4AArK6sUz509exa1atXS5scRSYogCGjevDmaNWuGffv2wcfHB7du3cKUKVOwePFijBs3Dp6enrC0tBQ7qn5LPHz0FSH5nBoH92/fl6fEl1umPuvEl+vyNJsL2NdP+dyAE5qVp0eXgMNjUj727EbauYnou7R2GOvChQsoVaoUWrZsicKFC8PPzy/F882aNdPWRxFJmiAIaNu2LW7cuIHt27fD2dkZb968wbhx42BnZ4fFixfj8+fPYsfUT9m52KZaDQRNA16Ffvnz623mKQEUc8ncrWjl/x+V0lVuohxIa2XHy8sLy5cvx5MnT3Djxg0cPHgQvXr1SloBmitBE2WOTCZDly5dcOvWLWzatAl2dnb477//MGrUKDg4OGDVqlWIi4sTO6Z+yc7FNpOPIGl7m1/n56noRFmitbITEhKCLl26APhyIbWTJ0/izZs3aNeuHX8gE2WBsbExevXqhbt378Lf3x+2trZ48uQJhgwZAicnJ2zYsAHx8fFixxRfdi62+fVn6WKbX+PoDpHGtFZ28uTJgydPniTdNzMzw969e2Fubo4mTZpApRLrVFAiaTAxMUG/fv0QFhaGZcuWoWjRonjw4AH69u2LsmXLYuvWrUhIyOYL6emT7Fxs8+vP0sU2v8bRHSKNaa3suLu7IyAgIMVjxsbG2LZtGxwcHPDp0ydtfRRRjmZqaoqhQ4ciIiICCxYsgI2NDcLDw9GzZ09UqlQJO3fuzHn/uMjOxTbT+ixdbPNrHN0h0ojWys6qVavg5eX1zeOCIMDf3x/379/X1kcREQBzc3N4eXlBoVBg1qxZyJcvH0JCQtCpUydUrVoV+/fvzzlz5bJzsU1dzKvJzlEpohxI47Lj7e2d4n6uXLlgYWGR5utLliyp6UcRUTpy586NCRMmQKFQYOrUqbC2tsaNGzfQpk0b1KhRA0ePHpV26cnOxTZ1Ma8mO0eliHIojcvOsmXL0K5du3QPTz148EDTzRNRJuXJkwe+vr5QKBSYMGECLCwscPnyZTRt2hQ//fQTTp48KXZE3cjOxTYzOq/mzELtbTOtz+DoDlGGaVx2Dh06hFOnTuGnn37C8+fPUzz34MEDDBgwAM7OzlkOSESZkz9/fsyaNQsKhQJeXl4wMzPD2bNnUb9+fTRs2BDnzp0TO6L2ZHmxzUyMkmTms07PAzIybyo7R6WIcjCNy467uzvOnTuHd+/eoVq1arh+/XqKkrNlyxb07dtXm1mJKBMKFSqEBQsWICIiAp6enjAxMcHx48dRq1YtNG/eHFeuXBE7YtZlebHNTIySZOazlJ+AMwu0u83kOLpDlClZmqBcpkwZXLp0CSVKlEDt2rXh7OyMrVu3YvDgwYiMjMSKFSu0lZOINFSsWDEsX74cYWFh6NevH4yMjHD48GG4ubmhXbt2uHnzptgRNZM4KpLl8yxk3x8l0WQE5p8F6Y/uZOeoFFEOl6WfEo8ePcKUKVNw/fp1xMTEQKlUYv78+Vi0aBGKFi2qrYxEpAWlSpWCv78/QkND0atXL8hkMuzduxeVK1dGly5dcOfOHbEjZk5CnJYW2/zfSuUJ6Vz8VJMRmO+N7mTnqBRRDqfxQqD9+vXDb7/9BkEQ0L9/f4wePRp+fn4YNmwYoqKiMH78eG3mJCItsbe3x6ZNmzBhwgT4+vpix44d+OOPP7Bz50706NEDU6dOhb29vdgxv8/YNEOLbSrj45MWIjYxTuNHnmXBL9tLTfIRmMwWk38WALW9AdlX/65MMSqVlbL2v1Ep+4ZcEZ0oHRqXna1bt6J///6YMGECihUrBgBYu3YtHB0dMWHCBISGhmLt2rUwMTHRWlgi0p4yZcpg+/btmDhxIqZOnYq9e/diy5Yt2LZtG3755Rf4+Pjo/yUjMrJSuVKJKIsnXxbZ1OTnURqrqGdI4uhOna9WMdfFqFRaZY2INC87ERERSSUnuTFjxsDR0RE9e/ZEZGQkTp06laWARKRblSpVwp49e/Dvv/9iypQpOHz4MNatW4eATZtgUbExxowdD58utcWOKY6sjOokSm10J4OjUhmS3qgUEQHIQtlJregkatu2LU6dOoXWrVtrunkiymZubm44dOgQzp07B4+h3gi7dgHvr/6FqT0DcXxnD+xYMQeFChUSO2b2ysqoTqK0RncyMipFRFqhteUivubq6opLly7pavNEpCP/fiqIuMaTUbjbLJiWKAd1fBxO7gyAbSk5JkyYgDdv3ogdMXto7WwvfP/MLCLSKZ2VHQAoXry4LjdPRFq2NCgMCwPvAQDMSlZC4e5zUajTNOQq6oi4zzGYM2cO5HI5fH19ERUVJXJaHdPavBp8Gd0JO5r17RCRRjQ+jEVE0pK86CQSBAHmdq4wk1fFp/BLeHfmN0T/p8C0adOwdOlSjBkzBsOGDUPu3LlFSq1D6c2rUauADU2BhNiMbUuQAafmAk5NU541FXECODwOaDYXsK+vndxE9A2djuwQkWFIregkJwgCLBxroKjHEti0GY/CJe3x9u1bTJw4EXZ2dli4cGG66+QZrDwlgGIu394igjJedIAv5ejra+Ko1UDQNOBV6Jc/eXFAIp1h2SHK4b5XdJITBBksy9SGaZeF+HnCfDg4OODly5fw9vaGvb09li9fjtjYTJQAQ6RSfZmDk1lfX/E4+eRnXhyQSKdYdohysMwUneQEmRFOq8pgyIr9WL9+PUqVKoVnz55h2LBhcHR0hL+/P5RKpQ4S64EzC77Mwcms5Fc8/nqpCC79QKRTWS47sbGxOH/+PPbt24dXr7RwzQgiyhaaFp3klhyPxIdSP+HevXtYtWoVihcvjkePHmHAgAEoU6YMNm/ejIQEDa9Po480HdVJlFhqwv9OuVQEl34g0qkslZ2lS5eiaNGiqF27Ntq3b5+0oOCrV69gY2ODDRs2aCUkEWmXNopOooWB97D6nwcYNGgQwsPDsXjxYhQqVAiRkZHo3bs3ypcvjx07dkAlhVOvNR3VSZRYag6P+3YBUI7uEOmMxmUnICAAI0eORNOmTbF+/Xqok/0FtbGxQYMGDbB9+3athCQi7VqkpaLz9fbMzMwwYsQIREZGYu7cucifPz9CQ0PRtWtXuLi4YO/evSl+VhiUrI7qJBGANxHfXpGZoztEOqNx2VmwYAHatGmDbdu2oVWrVt887+rqitu3b2cpHBHpxqhGTjrdnqWlJcaOHQuFQoHp06cjT548CA4ORrt27VCtWjUcOnTI8EpP2NGsjeokSefr5ugOkU5oXHbCw8PRrFmzNJ/Pnz8/Xr9+renmiUiHhjd0hJeWCo9XIycMb+iY6nPW1tbw8fGBQqHApEmTkDt3bly5cgUtWrRArVq1EBQUZBilR63+cp0c6HhlcY7uEOmExmUnb9686U5IDgkJQZEiRTTdPBHpmDYKT3pFJ7l8+fLBz88PCoUCY8aMgbm5Oc6fPw93d3fUr18f//zzT5Zy6FzS1ZSzoZhxdEc/RZwAllf/8icZHI2voNy8eXOsXbsWQ4YM+ea527dvw9/fH3369MlSOCLSrcSioslk5YwWneRsbGzw66+/wsvLC7Nnz8bq1atx6tQp1KlTB40bN8aMGTNQvXr1TGfROW2sUv7oEnB4zPdfl3x0x8Fd888j7fn6ApB29VJeCZv0nsYjO35+fkhISECFChUwefJkCIKATZs2oWfPnnBzc0OhQoUwZcoUbWYlIh3QZIRHk6KTXJEiRbBkyRKEh4dj4MCBMDY2xrFjx1CjRg20bt0a169f13jbOpPW1ZQzcitaGbix7dszsNLC0R39wgtAGjyNy06xYsVw5coVNG3aFDt27IBarcaWLVtw4MABdOvWDRcuXICNjY02sxKRjmSm8GS16CRna2uL1atXIzQ0FB4eHpDJZDhw4ACqVKmCTp06ISQkRCufI7rEX5Zfn4GVFs7d0R+8AKQkaFR2YmJi4Orqit27d2PdunV48+YNXrx4gWfPnuHt27fYsGEDChUqpO2sRKRDGSk82iw6ydnZ2SEgIAAhISHo1q0bBEHAzp07UaFCBfTs2RNhYWFa/8xs8/Uvy4ziL1X98HVRZRE1SBqVHQsLCygUCgjJjlkWLFgQhQsXhkzGFSiIDFV6hUdXRSc5Z2dnbNu2DTdv3kSHDh2gVquxdetWlC1bFn369MH9+/d1+vk6kdlRnUT8pSq+tIoqi6jB0biZNG3aFEePHtVmFiLSA6kVnuwoOslVqFABO3fuxNWrV9GyZUskJCQgICAATk5OGDx4MB4/fpxtWbJE01GdRPylKq60iiqLqMHRuOz4+Pjg3r17+Pnnn3HmzBk8efIEb968+eZGRIYnsfAIyP6ik1yVKlVw4MABnD9/Ho0aNYJSqcTq1avh4OCAkSNH4vnz56LkyjBNR3US8ZeqeL5XVFlEDYrGZad8+fIICQnB1q1bUbduXZQsWRIFCxb85kZEhml4Q0co5rQQregk98MPP+DYsWNJp6nHxsZiyZIlsLOzw7hx4/RzEeLEX5ZZXm9Zxl+qYvheUWURNSgaX2dnypQpKebsiG3FihWYN28enj9/jsqVK2PZsmX6eb0OItJYnTp1cPLkSQQFBcHHxwcXLlzAr7/+ipUrV2LkyJHw9vZG3rx5xY75RdKFCLO6AKoKiH7yZXvGptpIRt+TfFQnvVG5xNEd+4a87o6e07js+Pr6ajFG1uzYsQNeXl5YvXo1atSogcWLF6NJkyYIDQ3lWWFEEiMIAtzd3dGwYUMcOnQIPj4+uHbtGvz8/LB8+XJ4e3tjxIgRsLKyEjeoNi5EmMiyIItOdkp+XZ308AKQBkMSp04tXLgQ/fv3xy+//IJy5cph9erVsLCwwIYNG8SORkQ6IggCWrRogStXrmD37t0oX7483r17Bx8fH8jlcsybNw8xMTHihszKhQiT3/IUz/boOVZmJ5Vz7o5B0HhkR1/ExcXhypUrmDBhQtJjMpkM7u7uOH/+fKrviY2NRWxsbNL96OhoAIBSqYRSqdRtYB1JzG2o+aWE+yL7tWzZEs2aNcOff/6JGTNmICwsDGPHjsWCBQswevRolCpVivtDDxjC3w0h4jiMMzKqk+h/ozvxocegtm+gu2BaZgj7IiMyml9Qa7jksEwmy9CcnYQEDc9CyKCnT5+iePHiOHfuHGrWrJn0+NixY3Hq1ClcvHjxm/f4+vpi2rRp3zy+bds2WFhY6DQvEelWQkICTp06hR07duDFixcAgAIFCqBz585o0KABTExMRE5IekutRp1QX+T59ACyTMy1UkGGKPNSOO3sy7k72SwmJgbdu3dHVFQUrK2t03ydVicoJyQk4P79+9i7dy+cnZ3RsmVLTTevUxMmTICXl1fS/ejoaNja2qJx48bpfrP0mVKpRGBgIBo1asQf5iLjvhBfq1atMHv2bGzatAkzZ87EkydPsGrVKhw+fBiTJk1Cjx49YGxs8APbBkff/24IEcdhfF2R6ffJoEK+Twq0KGNuMKM7+r4vMirxyMz36GSC8rNnz/DDDz/AySlziwtqwsbGBkZGRkn/gkv04sULFClSJNX3mJqawtT028l+JiYmBr3TAWl8DVLBfSEuExMTDB48GD179oSXlxcOHjyI+/fvo3///vj111/h6+uLLl26wMhIwwv+kcb08u+GWg2cnv39M7DSIhjB+PRswLmxQY3u6OW+yISMZtfJBOWiRYti0KBBmDFjhi42n0KuXLng6uqKoKD/v9aBSqVCUFBQisNaRJQzmZmZoWXLlrh79y7mz58PGxsbhIWFoUePHqhUqRJ27doFlSqrp4eTweMFICVNZ2djWVpaQqHI/HCgJry8vODv749Nmzbhzp07GDx4MD5+/IhffvklWz6fiPSfhYUFvL29ERkZiZkzZyJv3rwICQlBx44d4erqioMHD0LDKYxk6HgBSMnTSdm5desWli5dmi2HsQCgS5cumD9/PqZMmQIXFxdcv34dR44cQeHChbPl84nIcFhZWWHixIlQKBSYMmUKrKyscP36dbRq1Qo1a9bEsWPHWHpyGl1cAJL0isZzduRyeapnY7179w5RUVGwsLDA3r17s5ItU4YOHYqhQ4dm2+cRkWHLmzcvpk2bhuHDh2PevHlYtmwZLl68iCZNmuCnn37CjBkzULduXbFjUnbgBSAlT+OyU7du3W/KjiAIyJcvH+zt7dG1a1fkz58/ywGJiHSpQIECmDNnDkaNGoW5c+di5cqV+Oeff1CvXj00bNgQM2bM4Py/nCBPiS83kiSNy87GjRu1GIOISFyFCxfGwoUL4e3tjVmzZsHf3x9BQUEICgpC8+bNMWPGDFStWlXsmESkAY3n7Dx8+BCfPn1K8/lPnz7h4cOHmm6eiEgUxYsXx4oVK3Dv3j307dsXRkZGOHToEFxdXdG+fXsEBweLHZGIMknjsiOXy7Fnz540n9+/fz/kcrmmmyciElXp0qWxbt063L17Fz179oQgCNizZw8qV66Mbt26ITQ0VOyIRJRBGped752toFQqIZNJYp1RIsrBHBwcsGXLFty6dQudOnWCWq3G9u3bUa5cOXh4eCAyMlLsiET0HZlqI9HR0Xj48GHS4anXr18n3U9+u3nzJrZv346iRYvqJDQRUXYrV64c/vjjD1y/fh1t2rSBSqXCpk2b4OzsjAEDBvCwPZEey1TZWbRoEeRyedJp5yNHjky6n/xWpUoVHDp0CIMGDdJVbiKiDDv6WICTzzEsDQrL8rYqV66MvXv34tKlS2jatCni4+Ph7+8PR0dHDBs2DM+ePdNCYiLSpkydjdW4cWPkzp0barUaY8eORbdu3b45O0EQBFhaWsLV1RVubm5aDUtElFnLT0Tg0KMv618tDLwHABje0DHL261WrRoOHz6Ms2fPwsfHBydOnMDy5cuxbt06eHp6Yty4cShYsGCWP4eIsi5TZadmzZpJ15v4+PEjOnTogAoVKugkGBFRVi0NCsOS4xEpHtNm4QGAWrVq4fjx4zh+/Dh8fHxw7tw5LFiwAKtXr8aIESPg7e3Na44RiUzjGcRTp05l0SEivbU0KCyp2HxtYeA9rRzSSq5BgwY4c+YMDh8+DDc3N3z8+BGzZs2CXC7HtGnTEB0drdXPI6KM0/iiggDw+fNn7Nq1C1evXkVUVNQ3KwcLgoD169dnKSARUWalV3QSaXuEB/jyM69p06Zo0qQJ9u/fjylTpuDmzZvw9fXF0qVLMWbMGAwbNgyWlpZa+0wi+j6Ny86DBw9Qv3593L9/H3nz5kVUVBTy58+Pd+/eISEhATY2NsidO7c2sxJRBi0NCsOiwHsY1chJq7/MDUFGik4iXRQe4EvpadOmDVq1aoWdO3di6tSpuHv3LiZMmIBFixZh/PjxGDRoEMzNzbX6uUSUOo0PY40ZMwZRUVG4cOEC7t27B7VajR07duDDhw+YO3cuzM3NcfToUW1mJaIMSPxlr8aXX+bd/S9APv4vrR+20UeZKTqJdHFIK5FMJkPnzp1x69YtbN68Gfb29vjvv//g5eUFBwcHrFy5ErGxsTr5bCL6fxqXnePHj2PIkCGoXr160sUD1Wo1TE1NMWbMGDRs2BAjR47UVk4iyoDUftmfi3idVHykXHg0KTqJdP29MTIyws8//4w7d+5g3bp1KFmyJJ4+fQpPT084OTlh/fr1UCqVOvt8opxO47ITExOD0qVLAwCsra0hCAKioqKSnq9ZsybOnDmT5YBElDEZnacixcKTlaKTKDu+NyYmJujbty/u3buHFStWoFixYnj48CH69euHsmXLYsuWLUhISNBpBqKcSOOyU7JkSTx+/BgAYGxsjOLFi+PChQtJz4eEhMDMzCzrCYnouzI7T0VKhUcbRSdRdn1vTE1NMWTIEISHh2PhwoUoVKgQIiIi0KtXL1SsWBF//PHHNyd8EJHmNC47DRo0wL59+5Lue3h4YNGiRejfvz/69u2LFStWoFWrVloJSURp07d5KtltkZaKjq62lx5zc3OMGjUKkZGRmDNnDvLly4c7d+6gS5cuqFKlCvbt2/fddQiJ6Ps0Ljvjx4/HpEmTkibXTZw4Eb1798bOnTuxb98+dO/eHQsXLtRaUCL6li7mqSwNCjOoCc2jGjnp9fYywtLSEuPGjYNCoYCvry+sra1x8+ZNtG3bFtWrV8eRI0dYeoiyIEuHsTp06ABTU1MAgJmZGdatW4e3b9/i1atX2LhxI6ytrbUWlIhS0sU8la/P5Mps4dF2UcrI9oY3dISXlgqKl8in6ufJkwdTp06FQqHAxIkTYWlpiX///RfNmjVD7dq1ceLECdGyERkyjctOotjYWJw/fx779u3Dq1evtJGJiL5DF/NUUttmZgpPVotSVranjcIjdtFJLn/+/Jg5cyYUCgW8vb1hZmaGc+fOoUGDBmjQoAHOnj0rdkQig5KlsrN06VIULVoUtWvXRvv27XHz5k0AwKtXr2BjY4MNGzZoJSQR/T9tFp1ECwPvZWlphawWJW1sLyuFR5+KTnIFCxbE/PnzERkZiaFDhyJXrlw4ceIEateujWbNmuHy5ctiRyQyCBqXnYCAAIwcORJNmzbF+vXrUxxPtrGxQYMGDbB9+3athCSi/5edE2gTpVc0tL0GVVa2p0nh0deik1zRokWxbNkyhIWFYcCAATA2NsaRI0dQvXp1tGnTBjdu3BA7IpFe07jsLFiwAG3atMG2bdtSPevK1dUVt2/fzlI4IvqWGBNogdSLhrav7aON7WWm8BhC0UmuZMmSWLNmDUJDQ9G7d2/IZDLs378fLi4u6Ny5M+7cuSN2RCK9pHHZCQ8PR7NmzdJ8Pn/+/Hj9+rWmmyeiNGhzQm5mJS8a2r62jza3l5HvkaEVneTs7OywceNG3L59G127doUgCPjzzz9RoUIF/PzzzwgPDxc7IpFe0bjs5M2bN90JySEhIShSpIimmyeidIhdeLr7X9DqtX10ca2g9L5Hhlx0kitTpgx+//133LhxA+3atYNKpcJvv/2GMmXKoF+/fnjw4IHYEYn0gsZlp3nz5li7di3evXv3zXO3b9+Gv78/WrdunZVsRJQOMQvPuQjNRm01PRSWme0lN7yhI0Y0sE/xmFSKTnIVK1bE7t27ceXKFbRo0QIJCQlYv349HB0dMWTIEDx58kTsiESi0rjs+Pn5ISEhARUqVMDkyZMhCAI2bdqEnj17ws3NDYUKFcKUKVO0mZWIviJm4dGUpofCMrK91Aytb4/mtgkQIM2ik1zVqlVx8OBBnDt3Du7u7lAqlVi1ahXs7e0xatQovHjxQuyIRKLQuOwUK1YMV65cQdOmTbFjxw6o1Wps2bIFBw4cQLdu3XDhwgXY2NhoMysRpcJQC48mh8LS2156hadJCTXuzWgs6aKTXM2aNREYGIiTJ0/ip59+QmxsLBYvXgw7OzuMHz+e8ykpx8nSdXYKFSqEdevW4c2bN3jx4gWePXuGt2/fYsOGDShUqJC2MhLRdxhi4dH0UFhaxDglX9/VrVsXp06dwtGjR1G9enXExMRg7ty5kMvlmDp1aqrTEIikKFNlZ+LEiUkXDvxawYIFUbhwYchkWb4oMxFpwNAKz4/2BbS6PbFOydd3giCgcePGuHDhAg4cOAAXFxe8f/8e06dPh1wux6xZs/DhwwexYxLpVKaayZw5c3Dr1q2k+69fv4aRkRGOHz+u9WBElHkZPeVa7FLk1cgJ2/r/IJk1rQyBIAho2bIlrly5gp07d6J8+fJ49+4dJk2aBLlcjvnz5yMmJkbsmEQ6keVhGK7ES6RfEguPgG9HT7RRCrI6IpM8g9TWtDIEMpkMHTp0wI0bN7B161Y4Ojri1atXGDNmDOzt7bFs2TLExsaKHZNIq3jMiUiChjd0hGJOi6TRk+RnImXlDKisjsikVkykuKaVITAyMkL37t0REhKCDRs2oHTp0nj+/DmGDx8OR0dHrF27FkqlUuyYRFrBskMkcYnFRxtFJysjMukVE6muaWUIjI2N8csvvyA0NBSrV69G8eLF8ejRIwwcOBDOzs7YtGkT4uPjxY5JlCWZLjv379/H1atXcfXq1aTJymFhYUmPfX0jIv2graKTSNtrUEl5TStDkCtXLgwcOBDh4eFYsmQJChcuDIVCAQ8PD5QvXx6///47VCqV2DGJNJLpsuPj44Nq1aqhWrVqcHd3BwAMGTIk6bHEm5ubG6pVq6b1wESUedouOom0vQaVGGtaLQ0Kg3z8X5lenV2qzMzMMHz4cERGRuLXX39FgQIFcO/ePXTv3h2VK1fG7t27OVeTDI5xZl4cEBCgqxxEpCO6KjqJEp9P7TM0KSba3l56kn9vEv/kiNEXFhYWGDNmDAYNGoQlS5Zg/vz5uHXrFjp06ICqVati+vTpaN68OQRBEDsq0Xdlquz07t1bVzmISAe0sRxDRqRWULJSTLS9vdSk9r1h4fmWlZUVJk+eDE9PTyxcuBCLFy/G1atX0bJlS/zwww+YMWMGGjZsyNJDeo0TlIkkKjvWnUou+Snv2igm2t5ecul9bzLzNeck+fLlw4wZM6BQKDB27FiYm5vjwoULaNSoEerVq4fTp0+LHZEoTSw7RBKkzRGdzBaexDO/tEHb2wMy9r1h4UmbjY0N5s6di8jISIwYMQKmpqY4ffo06tati8aNG+PixYtiRyT6BssOkQRpe52o723PUCb5ZqYEsvCkr0iRIli8eDHCw8MxaNAgmJiYIDAwED/88ANatWqFa9euiR2RKAnLDpEEaXudqPS2l1gg1NDvgrD8RESmR7v0+evRFyVKlMCqVasQGhqKX375BUZGRjh48CCqVq2Kjh074vbt22JHJGLZIZIibS4Kmt58mbQm+epbQTj6WMCS4xEavVcfvx59JJfLsWHDBoSEhKB79+4QBAG7du1CxYoV0b17d9y7x1XpSTwsO0QSpet1pwxlku/yExE49MgoS9vQp69H3zk5OWHr1q0IDg5Gx44doVar8fvvv6Ns2bLo168fXrx4IXZEyoFYdogkTFfrThnKJN+lQWEaj+h8TR++HkNSvnx5/Pnnn7h27RpatWoFlUqFzZs3Y8iQIfD09MSjR4/Ejkg5CMsOkcRpe90pQ5rkm90TtelbLi4u2L9/Py5evIhGjRohISEB/v7+cHBwwPDhw/Hs2TOxI1IOwLJDlANoa90pTU5pF7PwZOdEbUpf9erV8ddff2HWrFmoU6cO4uLisGzZMtjb22Ps2LF49eqV2BFJwlh2iHKIrK47lZVr94hVeIY3dMSIBvZa2RYXH9WOcuXKITAwEH///Tdq1qyJT58+Yd68eZDL5Zg8eTLevn0rdkSSIJYdohwkvcKjq6KTSKzCM7S+PZrbJmRpGyw62iUIAho2bIizZ8/ir7/+QtWqVfHhwwfMnDkTcrkcM2bMQHR0tNgxSUJYdohymNQKj66LTiKxCk+TEmqNR3hYdHRHEAQ0b94c//77L/bs2YMKFSogKioKU6ZMgVwux6+//oqPHz+KHZMkgGWHKAfKzLpTUpnkO7S+vVYnapP2CIKAtm3b4saNG9i+fTucnZ3x5s0bjBs3DnZ2dli8eDE+f/4sdkwyYCw7RDlURtedktIkX21N1CbdkMlk6NKlC27duoVNmzbBzs4O//33H0aNGgUHBwesWrUKcXFxYsckA8SyQ0Tpyq6rMWeXrE7UJt0zNjZGr169cPfuXaxduxa2trZ48uQJhgwZAmdnZ2zYsAHx8fFixyQDwrJDRN+l66sxZzdNJ2pT9jIxMUH//v0RFhaGZcuWoWjRorh//z769u2LcuXKYevWrUhIyNrkc8oZWHaIKEN0dTVmsWR2ojaJx9TUFEOHDkVERAQWLFgAGxsbhIWFoWfPnqhUqRJ27twJlUoldkzSYyw7RJRh2r4as9gyM1GbxGdubg4vLy8oFArMmjUL+fLlQ0hICDp16oSqVati//79UKvVYsckPcSyQ0SZIrVJvhmdqE36I3fu3JgwYQIUCgWmTp0KKysr3LhxA23atEGNGjVw9OhRlh5KgWWHiDKNk3xJH+TJkwe+vr5QKBQYP348LCwscPnyZTRt2hR16tTByZMnxY5IeoJlh4g0wkm+pC8KFCiA2bNnQ6FQwMvLC2ZmZjhz5gzq16+Phg0b4ty5c2JHJJGx7BCRxjjJl/RJoUKFsGDBAkRERMDT0xMmJiY4fvw4atWqhebNm+PKlStiRySRsOwQUZZwki/pm2LFimH58uUICwtDv379YGRkhMOHD8PNzQ3t2rXDzZs3xY5I2Yxlh4iyjJN8SR+VKlUK/v7+uHv3Ln7++WfIZDLs3bsXlStXRteuXXH37l2xI1I2YdkhIiJJc3BwwObNm3Hr1i107twZALBjxw6UL18evXv3RkREhMgJSdcMuuwkXklTLpfD3Nwc9vb2mDp1KtdOISKib5QtWxY7duzAjRs30LZtW6hUKmzevBnOzs7o378/Hj58KHZE0hGDLjt3796FSqXCmjVrcPv2bSxatAirV6/GxIkTxY5GRER6qlKlStizZw8uX76MZs2aISEhAevWrYOjoyOGDh2Kp0+fih2RtMygy07Tpk0REBCAxo0bw87ODq1bt8bo0aOxe/dusaMREZGec3Nzw6FDh3D27Fk0aNAAcXFxWLFiBezt7eHt7Y3//vtP7IikJcZiB9C2qKgo5M+fP93XxMbGIjY2Nul+dHQ0AECpVEKpVOo0n64k5jbU/FLCfaFfuD/0h77ui2rVquHIkSM4efIkfH19ce7cOSxcuBBr1qyBp6cnvLy8vvt7xdDo677IrIzmF9QSuqZ2eHg4XF1dMX/+fPTv3z/N1/n6+mLatGnfPL5t2zZYWFjoMiIREekxtVqN69evY9u2bQgLCwMAWFhYoFWrVmjdujUsLS1FTkjJxcTEoHv37oiKioK1tXWar9PLsjN+/HjMnTs33dfcuXMHZcqUSbr/5MkT1K1bF/Xq1cO6devSfW9qIzu2trZ49epVut8sfaZUKhEYGIhGjRrBxMRE7Dg5GveFfuH+0B+GtC/UajUOHjyIadOmJV2XJ1++fPDy8oKnpydy584tcsKsMaR9kZ7o6GjY2Nh8t+zo5WEsb29veHh4pPsaOzu7pP9++vQp6tevjx9//BFr16797vZNTU1hamr6zeMmJiYGvdMBaXwNUsF9oV+4P/SHoeyL9u3bo23btti9ezemTJmCO3fuwMfHB0uXLsX48eMxePBgmJubix0zSwxlX6Qlo9n1coJywYIFUaZMmXRvuXLlAvBlRKdevXpwdXVFQEAAZDK9/JKIiMgAyWQydOzYEcHBwfjtt9/g4OCAly9fwtvbG/b29li+fHmKIwWknwy6GSQWnZIlS2L+/Pl4+fIlnj9/jufPn4sdjYiIJMTIyAg9evTAnTt3sH79epQqVQrPnj3DsGHD4OjoCH9/f4Of7CtlBl12AgMDER4ejqCgIJQoUQJFixZNuhEREWmbsbEx+vTpg3v37mHlypUoVqwYHj16hAEDBqBMmTLYvHkzEhISxI5JXzHosuPh4QG1Wp3qjYiISFdy5cqFwYMHIzw8HIsWLUKhQoUQGRmJ3r17o0KFCtixYwdUKpXYMel/DLrsEBERicnc3BwjR45EZGQk5s6di/z58+Pu3bvo2rUrXFxcsHfvXv4DXA+w7BAREWWRpaUlxo4dC4VCgenTpyNPnjwIDg5Gu3btUK1aNRw6dIilR0QsO0RERFpibW0NHx8fKBQKTJo0Cblz58aVK1fQokUL1KpVC0FBQSw9ImDZISIi0rJ8+fLBz88PkZGRGD16NMzNzXH+/Hm4u7ujQYMGOHPmjNgRcxSWHSIiIh0pWLAg5s2bh4iICAwbNgy5cuXCyZMn8dNPP6Fp06a4dOmS2BFzBJYdIiIiHStatCiWLl2K8PBwDBw4EMbGxjh69Chq1KiB1q1b4/r162JHlDSWHSIiomxia2uL1atXIzQ0FB4eHpDJZDhw4ACqVKmCTp06ISQkROyIksSyQ0RElM3s7OwQEBCAkJAQdOvWDYIgYOfOnahQoQJ69uyZtOI6aQfLDhERkUicnZ2xbds23Lx5E+3bt4darcbWrVtRtmxZ9O3bF/fv3xc7oiSw7BAREYmsQoUK2LVrF65evYqWLVsiISEBGzZsgJOTEwYPHozHjx+LHdGgsewQERHpiSpVquDAgQM4f/48GjVqBKVSidWrV8PBwQEjR47kQtcaYtkhIiLSMz/88AOOHTuGU6dOoU6dOoiNjcWSJUtgZ2eHcePG4dWrV2JHNCgsO0RERHqqTp06OHnyJAIDA1GjRg18+vQJv/76K+RyOXx8fPDu3TuxIxoElh0iIiI9JggC3N3dcf78eRw8eBBVqlTBhw8f4OfnB7lcDj8/P7x//17smHqNZYeIiMgACIKAFi1a4MqVK9i1axfKly+Pd+/ewcfHB3K5HPPmzUNMTIzYMfUSyw4REZEBEQQB7du3x40bN7Bt2zY4OTnh9evXGDt2LOzs7LB06VJ8/vxZ7Jh6hWWHiIjIABkZGaFbt264ffs2Nm7cCLlcjhcvXmDEiBFwdHTEmjVrEBcXJ3ZMvcCyQ0REZMCMjY3Ru3dvhIaGYs2aNShRogQeP36MQYMGwdnZGQEBAYiPjxc7pqhYdoiIiCTAxMQEAwYMQFhYGJYuXYoiRYrg/v376NOnD8qVK4dt27YhISFB7JiiYNkhIiKSEDMzMwwbNgwRERGYN28ebGxsEBYWhh49eqBy5crYtWsXVCqV2DGzFcsOERGRBFlYWGD06NGIjIzEzJkzkTdvXty+fRsdO3ZEjRo1cPnyZajVarFjZguWHSIiIgmzsrLCxIkToVAoMGXKFFhZWeHGjRuYOXMmfvrpJxw7dkzypYdlh4iIKAfImzcvpk2bBoVCgdGjR8PU1BSXLl1CkyZNULduXZw6dUrsiDrDskNERJSDFChQALNmzcKaNWswfPhwmJqa4p9//kG9evXQqFEjXLhwQeyIWseyQ0RElAPlzZsX8+fPR0REBAYPHgwTExP8/fffqFmzJlq2bImrV6+KHVFrWHaIiIhysOLFi2PlypW4d+8e+vbtCyMjI/z1119wdXVF+/btERwcLHbELGPZISIiIpQuXRrr1q3D3bt30bNnTwiCgD179qBy5cro1q0bQkNDxY6oMZYdIiIiSuLg4IAtW7bg1q1b6NSpE9RqNbZv345y5crBw8MDkZGRYkfMNJYdIiIi+ka5cuXwxx9/4Nq1a2jdujVUKhU2bdoEZ2dnDBw4EI8ePRI7Yoax7BAREVGaXFxcsG/fPly6dAlNmzZFfHw81q5dCwcHBwwbNgzPnj0TO+J3sewQERHRd1WrVg2HDx/GmTNnUL9+fcTFxWH58uWws7PD6NGj8fLlS7Ejpollh4iIiDKsVq1aOH78OIKCgvDjjz/i8+fPWLBgAeRyOSZNmoQ3b96IHfEbLDtERESUaQ0aNMCZM2dw+PBhuLq64uPHj5g1axbkcjmmTZuG6OhosSMmYdkhIiIijQiCgKZNm+Ly5cvYu3cvKlasiOjoaPj6+kIul2Pu3Ln4+PGj2DFZdoiIiChrBEFAmzZtcP36dezYsQNlypTBmzdvMH78eNjZ2WHRokX49OmTaPlYdoiIiEgrZDIZOnfujFu3bmHz5s2wt7fHf//9By8vLzg4OGDlypWIjY3N/lzZ/olEREQkaUZGRvj5559x584drFu3DiVLlsTTp0/h6ekJJycnrF+/HkqlMtvysOwQERGRTpiYmKBv3764d+8eVqxYgaJFi+Lhw4fo168fypYtiy1btiAhIUHnOVh2iIiISKdMTU0xZMgQREREYOHChShYsCAiIiLQq1cvVKxYEX/88QdUKpXOPp9lh4iIiLKFubk5Ro0ahcjISMyePRv58uXDnTt30KVLF1SpUgX79u2DWq3W+uey7BAREVG2yp07N8aPHw+FQgFfX19YW1vj5s2baNu2LapXr44jR45otfSw7BAREZEo8uTJg6lTp0KhUGDixImwtLTEv//+i2bNmqF27do4ceKEVj6HZYeIiIhElT9/fsycORMKhQLe3t4wMzPDuXPn0KBBAzRo0ABnz57N0vZZdoiIiEgvFCxYEPPnz0dERASGDh2KXLly4cSJE6hduzaaNWuGf//9V6PtsuwQERGRXilWrBiWLVuGsLAw9O/fH8bGxjhy5AiqVauGtm3b4ubNm5naHssOERER6aWSJUti7dq1CA0NRe/evSGTybBv3z5UrlwZnTt3RmhoaIa2w7JDREREes3Ozg4bN27E7du30bVrVwiCgD///BM1atTI0PtZdoiIiMgglClTBr///jtu3LiBdu3aZfj0dJYdIiIiMigVK1bE7t27cfLkyQy9nmWHiIiIDFKVKlUy9DqWHSIiIpI0lh0iIiKSNJYdIiIikjSWHSIiIpI0lh0iIiKSNJYdIiIikjSWHSIiIpI0lh0iIiKSNJYdIiIikjSWHSIiIpI0lh0iIiKSNJYdIiIikjSWHSIiIpI0yZSd2NhYuLi4QBAEXL9+Xew4REREpCckU3bGjh2LYsWKiR2DiIiI9Iwkys7hw4dx7NgxzJ8/X+woREREpGeMxQ6QVS9evED//v2xd+9eWFhYZOg9sbGxiI2NTbofHR0NAFAqlVAqlTrJqWuJuQ01v5RwX+gX7g/9wX2hP6SyLzKaX1Cr1WodZ9EZtVqN5s2bo1atWpg8eTLu378PuVyOa9euwcXFJc33+fr6Ytq0ad88vm3btgwXJiIiIhJXTEwMunfvjqioKFhbW6f5Or0sO+PHj8fcuXPTfc2dO3dw7Ngx/PHHHzh16hSMjIwyXHZSG9mxtbXFq1ev0v1m6TOlUonAwEA0atQIJiYmYsfJ0bgv9Av3h/7gvtAfUtkX0dHRsLGx+W7Z0cvDWN7e3vDw8Ej3NXZ2djh+/DjOnz8PU1PTFM+5ubmhR48e2LRpU6rvNTU1/eY9AGBiYmLQOx2QxtcgFdwX+oX7Q39wX+gPQ98XGc2ul2WnYMGCKFiw4Hdft3TpUvj5+SXdf/r0KZo0aYIdO3agRo0auoxIREREBkIvy05GlSxZMsX93LlzAwDs7e1RokQJMSIRERGRnpHEqedEREREaTHokZ2vlS5dGno435qIiIhExJEdIiIikjSWHSIiIpI0lh0iIiKSNJYdIiIikjSWHSIiIpI0lh0iIiKSNJYdIiIikjSWHSIiIpI0lh0iIiKSNJYdIiIikjSWHSIiIpI0lh0iIiKSNJYdIiIikjSWHSIiIpI0lh0iIiKSNJYdIiIikjSWHSIiIpI0lh0iIiKSNJYdIiIikjSWHSIiIpI0lh0iIiKSNJYdIiIikjSWHSIiIpI0lh0iIiKSNJYdIiIikjSWHSIiIpI0lh0iIiKSNJYdIiIikjSWHSIiIpI0Y7ED6AO1Wg0AiI6OFjmJ5pRKJWJiYhAdHQ0TExOx4+Ro3Bf6hftDf3Bf6A+p7IvE39uJv8fTwrID4P379wAAW1tbkZMQERFRZr1//x558uRJ83lB/b06lAOoVCo8ffoUVlZWEARB7DgaiY6Ohq2tLR49egRra2ux4+Ro3Bf6hftDf3Bf6A+p7Au1Wo3379+jWLFikMnSnpnDkR0AMpkMJUqUEDuGVlhbWxv0/7hSwn2hX7g/9Af3hf6Qwr5Ib0QnEScoExERkaSx7BAREZGksexIhKmpKaZOnQpTU1Oxo+R43Bf6hftDf3Bf6I+cti84QZmIiIgkjSM7REREJGksO0RERCRpLDtEREQkaSw7REREJGksOxIXGxsLFxcXCIKA69evix0nx7l//z769u0LuVwOc3Nz2NvbY+rUqYiLixM7Wo6wYsUKlC5dGmZmZqhRowYuXbokdqQcZ/bs2ahWrRqsrKxQqFAhtG3bFqGhoWLHIgBz5syBIAgYOXKk2FF0jmVH4saOHYtixYqJHSPHunv3LlQqFdasWYPbt29j0aJFWL16NSZOnCh2NMnbsWMHvLy8MHXqVFy9ehWVK1dGkyZN8N9//4kdLUc5deoUPD09ceHCBQQGBkKpVKJx48b4+PGj2NFytMuXL2PNmjWoVKmS2FGyBU89l7DDhw/Dy8sLu3btQvny5XHt2jW4uLiIHSvHmzdvHlatWoXIyEixo0hajRo1UK1aNSxfvhzAlzXwbG1tMWzYMIwfP17kdDnXy5cvUahQIZw6dQp16tQRO06O9OHDB1StWhUrV66En58fXFxcsHjxYrFj6RRHdiTqxYsX6N+/P7Zs2QILCwux41AyUVFRyJ8/v9gxJC0uLg5XrlyBu7t70mMymQzu7u44f/68iMkoKioKAPh3QESenp5o0aJFir8fUseFQCVIrVbDw8MDgwYNgpubG+7fvy92JPqf8PBwLFu2DPPnzxc7iqS9evUKCQkJKFy4cIrHCxcujLt374qUilQqFUaOHIlatWqhQoUKYsfJkbZv346rV6/i8uXLYkfJVhzZMSDjx4+HIAjp3u7evYtly5bh/fv3mDBhgtiRJSuj+yK5J0+eoGnTpujUqRP69+8vUnIi8Xh6euLWrVvYvn272FFypEePHmHEiBHYunUrzMzMxI6TrThnx4C8fPkSr1+/Tvc1dnZ26Ny5Mw4cOABBEJIeT0hIgJGREXr06IFNmzbpOqrkZXRf5MqVCwDw9OlT1KtXDz/88AM2btwImYz/ztCluLg4WFhYYOfOnWjbtm3S471798a7d++wb98+8cLlUEOHDsW+fftw+vRpyOVysePkSHv37kW7du1gZGSU9FhCQgIEQYBMJkNsbGyK56SEZUeCHj58iOjo6KT7T58+RZMmTbBz507UqFEDJUqUEDFdzvPkyRPUr18frq6u+O233yT7w0Tf1KhRA9WrV8eyZcsAfDmEUrJkSQwdOpQTlLORWq3GsGHDsGfPHpw8eRKOjo5iR8qx3r9/jwcPHqR47JdffkGZMmUwbtw4SR9a5JwdCSpZsmSK+7lz5wYA2Nvbs+hksydPnqBevXooVaoU5s+fj5cvXyY9V6RIERGTSZ+Xlxd69+4NNzc3VK9eHYsXL8bHjx/xyy+/iB0tR/H09MS2bduwb98+WFlZ4fnz5wCAPHnywNzcXOR0OYuVldU3hcbS0hIFChSQdNEBWHaIdCowMBDh4eEIDw//pmhyUFW3unTpgpcvX2LKlCl4/vw5XFxccOTIkW8mLZNurVq1CgBQr169FI8HBATAw8Mj+wNRjsTDWERERCRpnCVJREREksayQ0RERJLGskNERESSxrJDREREksayQ0RERJLGskNERESSxrJDREREksayQ0RERJLGskNERESSxrJDREREksayQ0Qa27hxIwRBSPWmq5XFz507B19fX7x7904n2yci6eFCoESUZdOnT4dcLk/xmK5WUT537hymTZsGDw8P5M2bVyefQUTSwrJDRFnWrFkzuLm5iR0jyz5+/AhLS0uxYxCRlvEwFhHp3JMnT9CnTx8ULlwYpqamKF++PDZs2JDiNQ8ePMCQIUPg7OwMc3NzFChQAJ06dcL9+/eTXuPr64sxY8YAAORyedIhs8TXeHh4oHTp0t98vq+vLwRBSPWxkJAQdO/eHfny5UPt2rUznDe9r9XMzAx9+vRJ8fjff/8NExMTjBo1KkPbISLt4cgOEWVZVFQUXr16leIxGxsbAMCLFy/www8/QBAEDB06FAULFsThw4fRt29fREdHY+TIkQCAy5cv49y5c+jatStKlCiB+/fvY9WqVahXrx5CQkJgYWGB9u3b4969e/j999+xaNGipM8oWLCgxtk7deoER0dHzJo1C2q1OsN501K8eHH069cPa9euxdSpU1GqVCncvXsXnTp1QrNmzbBgwQKNsxKRhtRERBoKCAhQA0j1lqhv377qokWLql+9epXivV27dlXnyZNHHRMTo1ar1Ul/Jnf+/Hk1APXmzZuTHps3b54agFqhUHzz+t69e6tLlSr1zeNTp05Vf/3jLvGxbt26pXg8o3nT8/jxY7Wpqal68ODB6levXqnt7e3VLi4u6g8fPnz3vUSkfTyMRURZtmLFCgQGBqa4AYBarcauXbvQqlUrqNVqvHr1KunWpEkTREVF4erVqwAAc3PzpO0plUq8fv0aDg4OyJs3b9JrdGHQoEFJ/52ZvOkpXrw4+vfvjw0bNqBFixb49OkTDh48yPlARCLhYSwiyrLq1aunOkH55cuXePfuHdauXYu1a9em+t7//vsPAPDp0yfMnj0bAQEBePLkCdRqddJroqKidBMcSHEWWWbyfs/o0aOxfPly3Lx5E//88w+KFy+e4vlVq1bB398fwcHBmDRpEnx9fTX+GogofSw7RKQzKpUKANCzZ0/07t071ddUqlQJADBs2DAEBARg5MiRqFmzJvLkyQNBENC1a9ek7XzP15OQEyUkJKT5nuQjSpnJ+z0zZ84EAMTHxyN//vzfPF+0aFH4+vpi27ZtGdoeEWmOZYeIdKZgwYKwsrJCQkIC3N3d033tzp070bt37xQTeD9//vzNxQPTKjQAkC9fvlQvNvjgwQOt503PvHnzsG7dOixfvhxjxozBzJkzsW7duhSvadu2LQDg0KFDGn8OEWUM5+wQkc4YGRmhQ4cO2LVrF27duvXN8y9fvkzx2uSHrgBg2bJl34zKJM57Sa3U2NvbIyoqCjdv3kx67NmzZ9izZ4/W86Zl7969GD9+PGbMmAFPT08MGDAAmzdvhkKhyFAGItI+juwQkU7NmTMHJ06cQI0aNdC/f3+UK1cOb968wdWrV/H333/jzZs3AICWLVtiy5YtyJMnD8qVK4fz58/j77//RoECBVJsz9XVFQAwadIkdO3aFSYmJmjVqhUsLS3RtWtXjBs3Du3atcPw4cMRExODVatWwcnJKcOTnDOaNzVXrlxBjx490KNHD0yaNAkAMHbsWKxevTrV0R0iyh4sO0SkU4ULF8alS5cwffp07N69GytXrkSBAgVQvnx5zJ07N+l1S5YsgZGREbZu3YrPnz+jVq1a+Pvvv9GkSZMU26tWrRpmzJiB1atX48iRI1CpVFAoFLC0tESBAgWwZ88eeHl5YezYsZDL5Zg9ezbCwsIyXHYymvdrjx8/RqtWrVClShX4+/snPV6sWDH06dMH69atw6RJk75ZVoOIdE9Qfz1uTERE2WbQoEEoUqQIz8Yi0iHO2SEiEkF8fDw+f/6MhISEFP9NRNrHkR0iIhH4+vpi2rRpKR4LCAiAh4eHOIGIJIxlh4iIiCSNh7GIiIhI0lh2iIiISNJYdoiIiEjSWHaIiIhI0lh2iIiISNJYdoiIiEjSWHaIiIhI0lh2iIiISNJYdoiIiEjSWHaIiIhI0v4Pvm4jVRFJPQcAAAAASUVORK5CYII=",
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"x1_min, x1_max, x2_min, x2_max = plot_boundary(model)\n",
"\n",
"\n",
"plt.plot(\n",
" X_train[y_train == 0, 0],\n",
" X_train[y_train == 0, 1],\n",
" marker=\"D\",\n",
" markersize=10,\n",
" linestyle=\"\",\n",
" label=\"Class 0\",\n",
")\n",
"\n",
"plt.plot(\n",
" X_train[y_train == 1, 0],\n",
" X_train[y_train == 1, 1],\n",
" marker=\"^\",\n",
" markersize=13,\n",
" linestyle=\"\",\n",
" label=\"Class 1\",\n",
")\n",
"\n",
"plt.plot([x1_min, x1_max], [x2_min, x2_max], color=\"k\")\n",
"\n",
"plt.legend(loc=2)\n",
"\n",
"plt.xlim([-5, 5])\n",
"plt.ylim([-5, 5])\n",
"\n",
"plt.xlabel(\"Feature $x_1$\", fontsize=12)\n",
"plt.ylabel(\"Feature $x_2$\", fontsize=12)\n",
"\n",
"plt.grid()\n",
"plt.show()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.16"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
================================================
FILE: 02_pytorch-api/solution/solution.py
================================================
# %% [markdown]
# # Logistic Regression Classifier
# %% [markdown]
# ## 1) Installing Libraries
# %%
# !conda install numpy pandas matplotlib --yes
# %%
# !pip install torch torchvision torchaudio
# %%
# !conda install watermark
# %%
%load_ext watermark
%watermark -v -p numpy,pandas,matplotlib,torch -conda
# %% [markdown]
# ## 2) Loading the Dataset
# %%
import pandas as pd
df = pd.read_csv("toydata-truncated.txt", sep="\t")
df
# %%
X_train = df[["x1", "x2"]].values
y_train = df["label"].values
# %%
X_train
# %%
X_train.shape
# %%
y_train
# %%
y_train.shape
# %%
import numpy as np
np.bincount(y_train)
# %% [markdown]
# ## 3) Visualizing the dataset
# %%
%matplotlib inline
import matplotlib.pyplot as plt
# %%
plt.plot(
X_train[y_train == 0, 0],
X_train[y_train == 0, 1],
marker="D",
markersize=10,
linestyle="",
label="Class 0",
)
plt.plot(
X_train[y_train == 1, 0],
X_train[y_train == 1, 1],
marker="^",
markersize=13,
linestyle="",
label="Class 1",
)
plt.legend(loc=2)
plt.xlim([-5, 5])
plt.ylim([-5, 5])
plt.xlabel("Feature $x_1$", fontsize=12)
plt.ylabel("Feature $x_2$", fontsize=12)
plt.grid()
plt.show()
# %%
X_train = (X_train - X_train.mean(axis=0)) / X_train.std(axis=0)
# %%
plt.plot(
X_train[y_train == 0, 0],
X_train[y_train == 0, 1],
marker="D",
markersize=10,
linestyle="",
label="Class 0",
)
plt.plot(
X_train[y_train == 1, 0],
X_train[y_train == 1, 1],
marker="^",
markersize=13,
linestyle="",
label="Class 1",
)
plt.legend(loc=2)
plt.xlim([-5, 5])
plt.ylim([-5, 5])
plt.xlabel("Feature $x_1$", fontsize=12)
plt.ylabel("Feature $x_2$", fontsize=12)
plt.grid()
plt.show()
# %% [markdown]
# ## 4) Implementing the model
# %%
import torch
import torch.nn.functional as F
class LogisticRegression(torch.nn.Module):
def __init__(self, num_features, num_classes):
super().__init__()
self.linear = torch.nn.Linear(num_features, num_classes)
def forward(self, x):
logits = self.linear(x)
return logits
# %%
torch.manual_seed(1)
model = LogisticRegression(num_features=2, num_classes=2)
# %%
x = torch.tensor([[1.1, 2.1],
[1.1, 2.1],
[9.1, 4.1]])
with torch.no_grad():
logits = model(x)
probas = F.softmax(logits, dim=1)
print(probas)
# %% [markdown]
# ## 5) Defining a DataLoader
# %%
from torch.utils.data import Dataset, DataLoader
class MyDataset(Dataset):
def __init__(self, X, y):
self.features = torch.tensor(X, dtype=torch.float32)
self.labels = torch.tensor(y, dtype=torch.int64)
def __getitem__(self, index):
x = self.features[index]
y = self.labels[index]
return x, y
def __len__(self):
return self.labels.shape[0]
train_ds = MyDataset(X_train, y_train)
train_loader = DataLoader(
dataset=train_ds,
batch_size=10,
shuffle=True,
)
# %%
X_train.shape
# %% [markdown]
# ## 6) The training loop
# %%
import torch.nn.functional as F
torch.manual_seed(1)
model = LogisticRegression(num_features=2, num_classes=2)
optimizer = torch.optim.SGD(model.parameters(), lr=0.5)
num_epochs = 20
for epoch in range(num_epochs):
model = model.train()
for batch_idx, (features, class_labels) in enumerate(train_loader):
logits = model(features)
loss = F.cross_entropy(logits, class_labels)
optimizer.zero_grad()
loss.backward()
optimizer.step()
### LOGGING
print(f'Epoch: {epoch+1:03d}/{num_epochs:03d}'
f' | Batch {batch_idx:03d}/{len(train_loader):03d}'
f' | Loss: {loss:.2f}')
# %% [markdown]
# ## 7) Evaluating the results
# %%
def compute_accuracy(model, dataloader):
model = model.eval()
correct = 0.0
total_examples = 0
for idx, (features, class_labels) in enumerate(dataloader):
with torch.no_grad():
logits = model(features)
pred = torch.argmax(logits, dim=1)
compare = class_labels == pred
correct += torch.sum(compare)
total_examples += len(compare)
return correct / total_examples
# %%
train_acc = compute_accuracy(model, train_loader)
# %%
print(f"Accuracy: {train_acc*100}%")
# %% [markdown]
# ## 8) Optional: visualizing the decision boundary
# %%
plt.plot(
X_train[y_train == 0, 0],
X_train[y_train == 0, 1],
marker="D",
markersize=10,
linestyle="",
label="Class 0",
)
plt.plot(
X_train[y_train == 1, 0],
X_train[y_train == 1, 1],
marker="^",
markersize=13,
linestyle="",
label="Class 1",
)
plt.legend(loc=2)
plt.xlim([-5, 5])
plt.ylim([-5, 5])
plt.xlabel("Feature $x_1$", fontsize=12)
plt.ylabel("Feature $x_2$", fontsize=12)
plt.grid()
plt.show()
# %%
def plot_boundary(model):
w1 = model.linear.weight[0][0].detach()
w2 = model.linear.weight[0][1].detach()
b = model.linear.bias[0].detach()
x1_min = -20
x2_min = (-(w1 * x1_min) - b) / w2
x1_max = 20
x2_max = (-(w1 * x1_max) - b) / w2
return x1_min, x1_max, x2_min, x2_max
# %%
x1_min, x1_max, x2_min, x2_max = plot_boundary(model)
plt.plot(
X_train[y_train == 0, 0],
X_train[y_train == 0, 1],
marker="D",
markersize=10,
linestyle="",
label="Class 0",
)
plt.plot(
X_train[y_train == 1, 0],
X_train[y_train == 1, 1],
marker="^",
markersize=13,
linestyle="",
label="Class 1",
)
plt.plot([x1_min, x1_max], [x2_min, x2_max], color="k")
plt.legend(loc=2)
plt.xlim([-5, 5])
plt.ylim([-5, 5])
plt.xlabel("Feature $x_1$", fontsize=12)
plt.ylabel("Feature $x_2$", fontsize=12)
plt.grid()
plt.show()
================================================
FILE: 02_pytorch-api/solution/toydata-truncated.txt
================================================
x1 x2 label
0.77 -1.14 0
-0.33 1.44 0
0.91 -3.07 0
-0.37 -1.91 0
-0.63 -1.53 0
0.39 -1.99 0
-0.49 -2.74 0
-0.68 -1.52 0
-0.10 -3.43 0
-0.05 -1.95 0
3.88 0.65 1
0.73 2.97 1
0.83 3.94 1
1.59 1.25 1
1.14 3.91 1
1.73 2.80 1
1.31 1.85 1
1.56 3.85 1
1.23 2.54 1
1.33 2.03 1
================================================
FILE: 03_training-dnns/README.md
================================================
# PyCon2024 Workshop
## 3) Training Deep Neural Networks
[🔗 Slides](https://sebastianraschka.com/pdf/pycon2024/03_training-dnns_compressed.pdf)
## Exercise: Finetune a CNN
1. Open the [exercise/simple-cnn.py](exercise/simple-cnn.py) file and run `simple-cnn.py` on the CPU
2. Switch to a GPU machine (A10G or L4)
3. Run `simple-cnn.py` on the GPU
4. Look at `example.ipynb`
5. Replace the `model` in `simple-cnn.py` by the `model` in `example.ipynb`
6. run the modified code on the GPU
================================================
FILE: 03_training-dnns/exercise/example.ipynb
================================================
{
"cells": [
{
"cell_type": "code",
"execution_count": 2,
"id": "6b7b44c2-57a9-4922-86c4-2b0695123aa9",
"metadata": {},
"outputs": [],
"source": [
"from torchvision.models import mobilenet_v3_large\n",
"\n",
"# from torchvision.models import MobileNet_V3_Large_Weights"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "45ec45b2-4590-4c95-aeb4-26f2e5def5a1",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"MobileNetV3(\n",
" (features): Sequential(\n",
" (0): Conv2dNormActivation(\n",
" (0): Conv2d(3, 16, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
" (1): BatchNorm2d(16, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" (1): InvertedResidual(\n",
" (block): Sequential(\n",
" (0): Conv2dNormActivation(\n",
" (0): Conv2d(16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=16, bias=False)\n",
" (1): BatchNorm2d(16, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): ReLU(inplace=True)\n",
" )\n",
" (1): Conv2dNormActivation(\n",
" (0): Conv2d(16, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(16, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" )\n",
" )\n",
" )\n",
" (2): InvertedResidual(\n",
" (block): Sequential(\n",
" (0): Conv2dNormActivation(\n",
" (0): Conv2d(16, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): ReLU(inplace=True)\n",
" )\n",
" (1): Conv2dNormActivation(\n",
" (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=64, bias=False)\n",
" (1): BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): ReLU(inplace=True)\n",
" )\n",
" (2): Conv2dNormActivation(\n",
" (0): Conv2d(64, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(24, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" )\n",
" )\n",
" )\n",
" (3): InvertedResidual(\n",
" (block): Sequential(\n",
" (0): Conv2dNormActivation(\n",
" (0): Conv2d(24, 72, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(72, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): ReLU(inplace=True)\n",
" )\n",
" (1): Conv2dNormActivation(\n",
" (0): Conv2d(72, 72, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=72, bias=False)\n",
" (1): BatchNorm2d(72, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): ReLU(inplace=True)\n",
" )\n",
" (2): Conv2dNormActivation(\n",
" (0): Conv2d(72, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(24, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" )\n",
" )\n",
" )\n",
" (4): InvertedResidual(\n",
" (block): Sequential(\n",
" (0): Conv2dNormActivation(\n",
" (0): Conv2d(24, 72, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(72, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): ReLU(inplace=True)\n",
" )\n",
" (1): Conv2dNormActivation(\n",
" (0): Conv2d(72, 72, kernel_size=(5, 5), stride=(2, 2), padding=(2, 2), groups=72, bias=False)\n",
" (1): BatchNorm2d(72, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): ReLU(inplace=True)\n",
" )\n",
" (2): SqueezeExcitation(\n",
" (avgpool): AdaptiveAvgPool2d(output_size=1)\n",
" (fc1): Conv2d(72, 24, kernel_size=(1, 1), stride=(1, 1))\n",
" (fc2): Conv2d(24, 72, kernel_size=(1, 1), stride=(1, 1))\n",
" (activation): ReLU()\n",
" (scale_activation): Hardsigmoid()\n",
" )\n",
" (3): Conv2dNormActivation(\n",
" (0): Conv2d(72, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(40, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" )\n",
" )\n",
" )\n",
" (5): InvertedResidual(\n",
" (block): Sequential(\n",
" (0): Conv2dNormActivation(\n",
" (0): Conv2d(40, 120, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(120, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): ReLU(inplace=True)\n",
" )\n",
" (1): Conv2dNormActivation(\n",
" (0): Conv2d(120, 120, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=120, bias=False)\n",
" (1): BatchNorm2d(120, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): ReLU(inplace=True)\n",
" )\n",
" (2): SqueezeExcitation(\n",
" (avgpool): AdaptiveAvgPool2d(output_size=1)\n",
" (fc1): Conv2d(120, 32, kernel_size=(1, 1), stride=(1, 1))\n",
" (fc2): Conv2d(32, 120, kernel_size=(1, 1), stride=(1, 1))\n",
" (activation): ReLU()\n",
" (scale_activation): Hardsigmoid()\n",
" )\n",
" (3): Conv2dNormActivation(\n",
" (0): Conv2d(120, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(40, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" )\n",
" )\n",
" )\n",
" (6): InvertedResidual(\n",
" (block): Sequential(\n",
" (0): Conv2dNormActivation(\n",
" (0): Conv2d(40, 120, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(120, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): ReLU(inplace=True)\n",
" )\n",
" (1): Conv2dNormActivation(\n",
" (0): Conv2d(120, 120, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=120, bias=False)\n",
" (1): BatchNorm2d(120, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): ReLU(inplace=True)\n",
" )\n",
" (2): SqueezeExcitation(\n",
" (avgpool): AdaptiveAvgPool2d(output_size=1)\n",
" (fc1): Conv2d(120, 32, kernel_size=(1, 1), stride=(1, 1))\n",
" (fc2): Conv2d(32, 120, kernel_size=(1, 1), stride=(1, 1))\n",
" (activation): ReLU()\n",
" (scale_activation): Hardsigmoid()\n",
" )\n",
" (3): Conv2dNormActivation(\n",
" (0): Conv2d(120, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(40, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" )\n",
" )\n",
" )\n",
" (7): InvertedResidual(\n",
" (block): Sequential(\n",
" (0): Conv2dNormActivation(\n",
" (0): Conv2d(40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(240, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" (1): Conv2dNormActivation(\n",
" (0): Conv2d(240, 240, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=240, bias=False)\n",
" (1): BatchNorm2d(240, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" (2): Conv2dNormActivation(\n",
" (0): Conv2d(240, 80, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(80, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" )\n",
" )\n",
" )\n",
" (8): InvertedResidual(\n",
" (block): Sequential(\n",
" (0): Conv2dNormActivation(\n",
" (0): Conv2d(80, 200, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(200, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" (1): Conv2dNormActivation(\n",
" (0): Conv2d(200, 200, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=200, bias=False)\n",
" (1): BatchNorm2d(200, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" (2): Conv2dNormActivation(\n",
" (0): Conv2d(200, 80, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(80, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" )\n",
" )\n",
" )\n",
" (9): InvertedResidual(\n",
" (block): Sequential(\n",
" (0): Conv2dNormActivation(\n",
" (0): Conv2d(80, 184, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(184, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" (1): Conv2dNormActivation(\n",
" (0): Conv2d(184, 184, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=184, bias=False)\n",
" (1): BatchNorm2d(184, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" (2): Conv2dNormActivation(\n",
" (0): Conv2d(184, 80, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(80, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" )\n",
" )\n",
" )\n",
" (10): InvertedResidual(\n",
" (block): Sequential(\n",
" (0): Conv2dNormActivation(\n",
" (0): Conv2d(80, 184, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(184, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" (1): Conv2dNormActivation(\n",
" (0): Conv2d(184, 184, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=184, bias=False)\n",
" (1): BatchNorm2d(184, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" (2): Conv2dNormActivation(\n",
" (0): Conv2d(184, 80, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(80, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" )\n",
" )\n",
" )\n",
" (11): InvertedResidual(\n",
" (block): Sequential(\n",
" (0): Conv2dNormActivation(\n",
" (0): Conv2d(80, 480, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(480, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" (1): Conv2dNormActivation(\n",
" (0): Conv2d(480, 480, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=480, bias=False)\n",
" (1): BatchNorm2d(480, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" (2): SqueezeExcitation(\n",
" (avgpool): AdaptiveAvgPool2d(output_size=1)\n",
" (fc1): Conv2d(480, 120, kernel_size=(1, 1), stride=(1, 1))\n",
" (fc2): Conv2d(120, 480, kernel_size=(1, 1), stride=(1, 1))\n",
" (activation): ReLU()\n",
" (scale_activation): Hardsigmoid()\n",
" )\n",
" (3): Conv2dNormActivation(\n",
" (0): Conv2d(480, 112, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(112, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" )\n",
" )\n",
" )\n",
" (12): InvertedResidual(\n",
" (block): Sequential(\n",
" (0): Conv2dNormActivation(\n",
" (0): Conv2d(112, 672, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(672, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" (1): Conv2dNormActivation(\n",
" (0): Conv2d(672, 672, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=672, bias=False)\n",
" (1): BatchNorm2d(672, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" (2): SqueezeExcitation(\n",
" (avgpool): AdaptiveAvgPool2d(output_size=1)\n",
" (fc1): Conv2d(672, 168, kernel_size=(1, 1), stride=(1, 1))\n",
" (fc2): Conv2d(168, 672, kernel_size=(1, 1), stride=(1, 1))\n",
" (activation): ReLU()\n",
" (scale_activation): Hardsigmoid()\n",
" )\n",
" (3): Conv2dNormActivation(\n",
" (0): Conv2d(672, 112, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(112, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" )\n",
" )\n",
" )\n",
" (13): InvertedResidual(\n",
" (block): Sequential(\n",
" (0): Conv2dNormActivation(\n",
" (0): Conv2d(112, 672, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(672, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" (1): Conv2dNormActivation(\n",
" (0): Conv2d(672, 672, kernel_size=(5, 5), stride=(2, 2), padding=(2, 2), groups=672, bias=False)\n",
" (1): BatchNorm2d(672, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" (2): SqueezeExcitation(\n",
" (avgpool): AdaptiveAvgPool2d(output_size=1)\n",
" (fc1): Conv2d(672, 168, kernel_size=(1, 1), stride=(1, 1))\n",
" (fc2): Conv2d(168, 672, kernel_size=(1, 1), stride=(1, 1))\n",
" (activation): ReLU()\n",
" (scale_activation): Hardsigmoid()\n",
" )\n",
" (3): Conv2dNormActivation(\n",
" (0): Conv2d(672, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(160, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" )\n",
" )\n",
" )\n",
" (14): InvertedResidual(\n",
" (block): Sequential(\n",
" (0): Conv2dNormActivation(\n",
" (0): Conv2d(160, 960, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(960, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" (1): Conv2dNormActivation(\n",
" (0): Conv2d(960, 960, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=960, bias=False)\n",
" (1): BatchNorm2d(960, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" (2): SqueezeExcitation(\n",
" (avgpool): AdaptiveAvgPool2d(output_size=1)\n",
" (fc1): Conv2d(960, 240, kernel_size=(1, 1), stride=(1, 1))\n",
" (fc2): Conv2d(240, 960, kernel_size=(1, 1), stride=(1, 1))\n",
" (activation): ReLU()\n",
" (scale_activation): Hardsigmoid()\n",
" )\n",
" (3): Conv2dNormActivation(\n",
" (0): Conv2d(960, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(160, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" )\n",
" )\n",
" )\n",
" (15): InvertedResidual(\n",
" (block): Sequential(\n",
" (0): Conv2dNormActivation(\n",
" (0): Conv2d(160, 960, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(960, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" (1): Conv2dNormActivation(\n",
" (0): Conv2d(960, 960, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=960, bias=False)\n",
" (1): BatchNorm2d(960, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" (2): SqueezeExcitation(\n",
" (avgpool): AdaptiveAvgPool2d(output_size=1)\n",
" (fc1): Conv2d(960, 240, kernel_size=(1, 1), stride=(1, 1))\n",
" (fc2): Conv2d(240, 960, kernel_size=(1, 1), stride=(1, 1))\n",
" (activation): ReLU()\n",
" (scale_activation): Hardsigmoid()\n",
" )\n",
" (3): Conv2dNormActivation(\n",
" (0): Conv2d(960, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(160, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" )\n",
" )\n",
" )\n",
" (16): Conv2dNormActivation(\n",
" (0): Conv2d(160, 960, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
" (1): BatchNorm2d(960, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)\n",
" (2): Hardswish()\n",
" )\n",
" )\n",
" (avgpool): AdaptiveAvgPool2d(output_size=1)\n",
" (classifier): Sequential(\n",
" (0): Linear(in_features=960, out_features=1280, bias=True)\n",
" (1): Hardswish()\n",
" (2): Dropout(p=0.2, inplace=True)\n",
" (3): Linear(in_features=1280, out_features=1000, bias=True)\n",
" )\n",
")\n"
]
}
],
"source": [
"model = mobilenet_v3_large()\n",
"\n",
"print(model)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "b15f916e-cc94-4273-8451-679ae22f427a",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Linear(in_features=1280, out_features=1000, bias=True)"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.classifier[3] = torch.nn.Linear(1280, 10)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
================================================
FILE: 03_training-dnns/exercise/local_utilities.py
================================================
# Imports for ViT finetuning
import torch
from torch.utils.data import sampler
from torchvision import datasets
from torch.utils.data import DataLoader
from torch.utils.data import SubsetRandomSampler
from torchvision import transforms
# Import for LLM finetuning
import os
import sys
import tarfile
import time
import numpy as np
import pandas as pd
from packaging import version
from torch.utils.data import Dataset
from tqdm import tqdm
import urllib
############################
##### VIT finetuning dataset
############################
def get_dataloaders_cifar10(batch_size, num_workers=0,
validation_fraction=None,
train_transforms=None,
test_transforms=None):
if train_transforms is None:
train_transforms = transforms.ToTensor()
if test_transforms is None:
test_transforms = transforms.ToTensor()
train_dataset = datasets.CIFAR10(root='data',
train=True,
transform=train_transforms,
download=True)
valid_dataset = datasets.CIFAR10(root='data',
train=True,
transform=test_transforms)
test_dataset = datasets.CIFAR10(root='data',
train=False,
transform=test_transforms)
if validation_fraction is not None:
num = int(validation_fraction * 50000)
train_indices = torch.arange(0, 50000 - num)
valid_indices = torch.arange(50000 - num, 50000)
train_sampler = SubsetRandomSampler(train_indices)
valid_sampler = SubsetRandomSampler(valid_indices)
valid_loader = DataLoader(dataset=valid_dataset,
batch_size=batch_size,
num_workers=num_workers,
sampler=valid_sampler)
train_loader = DataLoader(dataset=train_dataset,
batch_size=batch_size,
num_workers=num_workers,
drop_last=True,
sampler=train_sampler)
else:
train_loader = DataLoader(dataset=train_dataset,
batch_size=batch_size,
num_workers=num_workers,
drop_last=True,
shuffle=True)
test_loader = DataLoader(dataset=test_dataset,
batch_size=batch_size,
num_workers=num_workers,
shuffle=False)
if validation_fraction is None:
return train_loader, test_loader
else:
return train_loader, valid_loader, test_loader
############################
##### LLM finetuning dataset
############################
import os
import sys
import tarfile
import time
import numpy as np
import pandas as pd
from packaging import version
from torch.utils.data import Dataset
from tqdm import tqdm
import urllib
def reporthook(count, block_size, total_size):
global start_time
if count == 0:
start_time = time.time()
return
duration = time.time() - start_time
progress_size = int(count * block_size)
speed = progress_size / (1024.0**2 * duration)
percent = count * block_size * 100.0 / total_size
sys.stdout.write(
f"\r{int(percent)}% | {progress_size / (1024.**2):.2f} MB "
f"| {speed:.2f} MB/s | {duration:.2f} sec elapsed"
)
sys.stdout.flush()
def download_dataset():
source = "http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz"
target = "aclImdb_v1.tar.gz"
if os.path.exists(target):
os.remove(target)
if not os.path.isdir("aclImdb") and not os.path.isfile("aclImdb_v1.tar.gz"):
urllib.request.urlretrieve(source, target, reporthook)
if not os.path.isdir("aclImdb"):
with tarfile.open(target, "r:gz") as tar:
tar.extractall()
def load_dataset_into_to_dataframe():
basepath = "aclImdb"
labels = {"pos": 1, "neg": 0}
df = pd.DataFrame()
with tqdm(total=50000) as pbar:
for s in ("test", "train"):
for l in ("pos", "neg"):
path = os.path.join(basepath, s, l)
for file in sorted(os.listdir(path)):
with open(os.path.join(path, file), "r", encoding="utf-8") as infile:
txt = infile.read()
if version.parse(pd.__version__) >= version.parse("1.3.2"):
x = pd.DataFrame(
[[txt, labels[l]]], columns=["review", "sentiment"]
)
df = pd.concat([df, x], ignore_index=False)
else:
df = df.append([[txt, labels[l]]], ignore_index=True)
pbar.update()
df.columns = ["text", "label"]
np.random.seed(0)
df = df.reindex(np.random.permutation(df.index))
print("Class distribution:")
np.bincount(df["label"].values)
return df
def partition_dataset(df):
df_shuffled = df.sample(frac=1, random_state=1).reset_index()
df_train = df_shuffled.iloc[:35_000]
df_val = df_shuffled.iloc[35_000:40_000]
df_test = df_shuffled.iloc[40_000:]
df_train.to_csv("train.csv", index=False, encoding="utf-8")
df_val.to_csv("val.csv", index=False, encoding="utf-8")
df_test.to_csv("test.csv", index=False, encoding="utf-8")
class IMDBDataset(Dataset):
def __init__(self, dataset_dict, partition_key="train"):
self.partition = dataset_dict[partition_key]
def __getitem__(self, index):
return self.partition[index]
def __len__(self):
return self.partition.num_rows
================================================
FILE: 03_training-dnns/exercise/simple-cnn.py
================================================
import time
import lightning as L
import torch
import torch.nn.functional as F
import torchmetrics
from torchvision import transforms
from watermark import watermark
from local_utilities import get_dataloaders_cifar10
class PyTorchCNN(torch.nn.Module):
def __init__(self, num_classes):
super().__init__()
self.cnn_layers = torch.nn.Sequential(
torch.nn.Conv2d(3, 6, kernel_size=5),
torch.nn.BatchNorm2d(6),
torch.nn.ReLU(),
torch.nn.MaxPool2d(kernel_size=2),
torch.nn.Conv2d(6, 16, kernel_size=3),
torch.nn.BatchNorm2d(16),
torch.nn.ReLU(),
torch.nn.MaxPool2d(kernel_size=2),
torch.nn.Conv2d(16, 32, kernel_size=3),
torch.nn.BatchNorm2d(32),
torch.nn.ReLU(),
torch.nn.MaxPool2d(kernel_size=2),
)
self.fc_layers = torch.nn.Sequential(
# hidden layer
torch.nn.Linear(128, 64),
torch.nn.BatchNorm1d(64),
torch.nn.ReLU(),
# output layer
torch.nn.Linear(64, num_classes)
)
def forward(self, x):
x = self.cnn_layers(x)
# print(x.shape)
x = torch.flatten(x, start_dim=1)
logits = self.fc_layers(x)
return logits
def train(num_epochs, model, optimizer, scheduler, train_loader, val_loader, device):
for epoch in range(num_epochs):
train_acc = torchmetrics.Accuracy(task="multiclass", num_classes=10).to(device)
model.train()
for batch_idx, (features, targets) in enumerate(train_loader):
model.train()
features = features.to(device)
targets = targets.to(device)
### FORWARD AND BACK PROP
logits = model(features)
loss = F.cross_entropy(logits, targets)
optimizer.zero_grad()
loss.backward()
### UPDATE MODEL PARAMETERS
optimizer.step()
scheduler.step()
### LOGGING
if not batch_idx % 300:
print(f"Epoch: {epoch+1:04d}/{num_epochs:04d} | Batch {batch_idx:04d}/{len(train_loader):04d} | Loss: {loss:.4f}")
model.eval()
with torch.no_grad():
predicted_labels = torch.argmax(logits, 1)
train_acc.update(predicted_labels, targets)
### MORE LOGGING
model.eval()
with torch.no_grad():
val_acc = torchmetrics.Accuracy(task="multiclass", num_classes=10).to(device)
for (features, targets) in val_loader:
features = features.to(device)
targets = targets.to(device)
outputs = model(features)
predicted_labels = torch.argmax(outputs, 1)
val_acc.update(predicted_labels, targets)
print(f"Epoch: {epoch+1:04d}/{num_epochs:04d} | Train acc.: {train_acc.compute()*100:.2f}% | Val acc.: {val_acc.compute()*100:.2f}%")
train_acc.reset(), val_acc.reset()
if __name__ == "__main__":
print(watermark(packages="torch,lightning", python=True))
print("Torch CUDA available?", torch.cuda.is_available())
device = "cuda" if torch.cuda.is_available() else "cpu"
L.seed_everything(123)
##########################
### 1 Loading the Dataset
##########################
train_transforms = transforms.Compose([transforms.Resize((32, 32)),
transforms.ToTensor()])
test_transforms = transforms.Compose([transforms.Resize((32, 32)),
transforms.ToTensor()])
############################################################
train_loader, val_loader, test_loader = get_dataloaders_cifar10(
batch_size=64,
num_workers=3,
train_transforms=train_transforms,
test_transforms=test_transforms,
validation_fraction=0.1)
#########################################
### 2 Initializing the Model
#########################################
############################################################
#### YOUR CODE BELOW: Replace model
############################################################
# use a model from https://pytorch.org/vision/stable/models.html
# see example.ipynb for more info
model = PyTorchCNN(num_classes=10)
model.to(device)
############################################################
NUM_EPOCHS = 50
optimizer = torch.optim.Adam(model.parameters(), lr=5e-5)
num_steps = NUM_EPOCHS * len(train_loader)
scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=num_steps)
#########################################
### 3 Finetuning
#########################################
start = time.time()
train(
num_epochs=NUM_EPOCHS,
model=model,
optimizer=optimizer,
train_loader=train_loader,
val_loader=val_loader,
scheduler=scheduler,
device=device
)
end = time.time()
elapsed = end-start
print(f"Time elapsed {elapsed/60:.2f} min")
print(f"Memory used: {torch.cuda.max_memory_reserved() / 1e9:.02f} GB")
#########################################
### 4 Evaluation
#########################################
with torch.no_grad():
model.eval()
test_acc = torchmetrics.Accuracy(task="multiclass", num_classes=10).to(device)
for (features, targets) in test_loader:
features = features.to(device)
targets = targets.to(device)
outputs = model(features)
predicted_labels = torch.argmax(outputs, 1)
test_acc.update(predicted_labels, targets)
print(f"Test accuracy {test_acc.compute()*100:.2f}%")
================================================
FILE: 03_training-dnns/solution/local_utilities.py
================================================
# Imports for ViT finetuning
import torch
from torch.utils.data import sampler
from torchvision import datasets
from torch.utils.data import DataLoader
from torch.utils.data import SubsetRandomSampler
from torchvision import transforms
# Import for LLM finetuning
import os
import sys
import tarfile
import time
import numpy as np
import pandas as pd
from packaging import version
from torch.utils.data import Dataset
from tqdm import tqdm
import urllib
############################
##### VIT finetuning dataset
############################
def get_dataloaders_cifar10(batch_size, num_workers=0,
validation_fraction=None,
train_transforms=None,
test_transforms=None):
if train_transforms is None:
train_transforms = transforms.ToTensor()
if test_transforms is None:
test_transforms = transforms.ToTensor()
train_dataset = datasets.CIFAR10(root='data',
train=True,
transform=train_transforms,
download=True)
valid_dataset = datasets.CIFAR10(root='data',
train=True,
transform=test_transforms)
test_dataset = datasets.CIFAR10(root='data',
train=False,
transform=test_transforms)
if validation_fraction is not None:
num = int(validation_fraction * 50000)
train_indices = torch.arange(0, 50000 - num)
valid_indices = torch.arange(50000 - num, 50000)
train_sampler = SubsetRandomSampler(train_indices)
valid_sampler = SubsetRandomSampler(valid_indices)
valid_loader = DataLoader(dataset=valid_dataset,
batch_size=batch_size,
num_workers=num_workers,
sampler=valid_sampler)
train_loader = DataLoader(dataset=train_dataset,
batch_size=batch_size,
num_workers=num_workers,
drop_last=True,
sampler=train_sampler)
else:
train_loader = DataLoader(dataset=train_dataset,
batch_size=batch_size,
num_workers=num_workers,
drop_last=True,
shuffle=True)
test_loader = DataLoader(dataset=test_dataset,
batch_size=batch_size,
num_workers=num_workers,
shuffle=False)
if validation_fraction is None:
return train_loader, test_loader
else:
return train_loader, valid_loader, test_loader
############################
##### LLM finetuning dataset
############################
import os
import sys
import tarfile
import time
import numpy as np
import pandas as pd
from packaging import version
from torch.utils.data import Dataset
from tqdm import tqdm
import urllib
def reporthook(count, block_size, total_size):
global start_time
if count == 0:
start_time = time.time()
return
duration = time.time() - start_time
progress_size = int(count * block_size)
speed = progress_size / (1024.0**2 * duration)
percent = count * block_size * 100.0 / total_size
sys.stdout.write(
f"\r{int(percent)}% | {progress_size / (1024.**2):.2f} MB "
f"| {speed:.2f} MB/s | {duration:.2f} sec elapsed"
)
sys.stdout.flush()
def download_dataset():
source = "http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz"
target = "aclImdb_v1.tar.gz"
if os.path.exists(target):
os.remove(target)
if not os.path.isdir("aclImdb") and not os.path.isfile("aclImdb_v1.tar.gz"):
urllib.request.urlretrieve(source, target, reporthook)
if not os.path.isdir("aclImdb"):
with tarfile.open(target, "r:gz") as tar:
tar.extractall()
def load_dataset_into_to_dataframe():
basepath = "aclImdb"
labels = {"pos": 1, "neg": 0}
df = pd.DataFrame()
with tqdm(total=50000) as pbar:
for s in ("test", "train"):
for l in ("pos", "neg"):
path = os.path.join(basepath, s, l)
for file in sorted(os.listdir(path)):
with open(os.path.join(path, file), "r", encoding="utf-8") as infile:
txt = infile.read()
if version.parse(pd.__version__) >= version.parse("1.3.2"):
x = pd.DataFrame(
[[txt, labels[l]]], columns=["review", "sentiment"]
)
df = pd.concat([df, x], ignore_index=False)
else:
df = df.append([[txt, labels[l]]], ignore_index=True)
pbar.update()
df.columns = ["text", "label"]
np.random.seed(0)
df = df.reindex(np.random.permutation(df.index))
print("Class distribution:")
np.bincount(df["label"].values)
return df
def part
gitextract_mjxmq1vx/ ├── .gitignore ├── 00-1_python-setup-guide/ │ └── README.md ├── 00-2_python-libraries-for-workshop/ │ ├── README.md │ ├── jupyter_environment_check.ipynb │ ├── python_environment_check.py │ └── requirements.txt ├── 01_intro-to-deeplearning/ │ └── README.md ├── 02_pytorch-api/ │ ├── README.md │ ├── exercise/ │ │ ├── exercise-logreg.ipynb │ │ ├── exercise.py │ │ └── toydata-truncated.txt │ └── solution/ │ ├── solution-logreg.ipynb │ ├── solution.py │ └── toydata-truncated.txt ├── 03_training-dnns/ │ ├── README.md │ ├── exercise/ │ │ ├── example.ipynb │ │ ├── local_utilities.py │ │ └── simple-cnn.py │ └── solution/ │ ├── local_utilities.py │ └── solution-torchvision-cnn.py ├── 04_accelerating-pytorch/ │ ├── README.md │ ├── exercise/ │ │ ├── 00_pytorch-vit-random-init.py │ │ ├── 01_pytorch-vit.py │ │ └── local_utilities.py │ └── solution/ │ ├── 02_fabric-vit.py │ ├── 03_fabric-vit-mixed-precision.py │ ├── 04_fabric-vit-mixed-fsdp.py │ └── local_utilities.py ├── 05_finetuning-llms/ │ ├── README.md │ ├── exercise/ │ │ ├── 01_train-gpt.ipynb │ │ ├── 02_use-trained-gpt.ipynb │ │ ├── gpt_download.py │ │ ├── requirements-extra.txt │ │ ├── spam_classifier_utils.py │ │ ├── test.csv │ │ ├── train.csv │ │ └── validation.csv │ └── solution/ │ ├── 01_train-gpt-solution.ipynb │ ├── 02_use-trained-gpt-solution.ipynb │ ├── gpt_download.py │ ├── requirements-extra.txt │ ├── spam_classifier_utils.py │ ├── test.csv │ ├── train.csv │ └── validation.csv ├── LICENSE └── README.md
SYMBOL INDEX (135 symbols across 18 files)
FILE: 00-2_python-libraries-for-workshop/python_environment_check.py
function get_packages (line 12) | def get_packages(pkgs):
function check_packages (line 33) | def check_packages(d):
FILE: 02_pytorch-api/exercise/exercise.py
class LogisticRegression (line 118) | class LogisticRegression(torch.nn.Module):
method __init__ (line 120) | def __init__(self, num_features, num_classes):
method forward (line 124) | def forward(self, x):
class MyDataset (line 151) | class MyDataset(Dataset):
method __init__ (line 152) | def __init__(self, X, y):
method __getitem__ (line 157) | def __getitem__(self, index):
method __len__ (line 162) | def __len__(self):
function compute_accuracy (line 224) | def compute_accuracy(model, dataloader):
function plot_boundary (line 284) | def plot_boundary(model):
FILE: 02_pytorch-api/solution/solution.py
class LogisticRegression (line 127) | class LogisticRegression(torch.nn.Module):
method __init__ (line 129) | def __init__(self, num_features, num_classes):
method forward (line 133) | def forward(self, x):
class MyDataset (line 160) | class MyDataset(Dataset):
method __init__ (line 161) | def __init__(self, X, y):
method __getitem__ (line 166) | def __getitem__(self, index):
method __len__ (line 171) | def __len__(self):
function compute_accuracy (line 222) | def compute_accuracy(model, dataloader):
function plot_boundary (line 282) | def plot_boundary(model):
FILE: 03_training-dnns/exercise/local_utilities.py
function get_dataloaders_cifar10 (line 28) | def get_dataloaders_cifar10(batch_size, num_workers=0,
function reporthook (line 105) | def reporthook(count, block_size, total_size):
function download_dataset (line 122) | def download_dataset():
function load_dataset_into_to_dataframe (line 138) | def load_dataset_into_to_dataframe():
function partition_dataset (line 173) | def partition_dataset(df):
class IMDBDataset (line 185) | class IMDBDataset(Dataset):
method __init__ (line 186) | def __init__(self, dataset_dict, partition_key="train"):
method __getitem__ (line 189) | def __getitem__(self, index):
method __len__ (line 192) | def __len__(self):
FILE: 03_training-dnns/exercise/simple-cnn.py
class PyTorchCNN (line 13) | class PyTorchCNN(torch.nn.Module):
method __init__ (line 14) | def __init__(self, num_classes):
method forward (line 45) | def forward(self, x):
function train (line 53) | def train(num_epochs, model, optimizer, scheduler, train_loader, val_loa...
FILE: 03_training-dnns/solution/local_utilities.py
function get_dataloaders_cifar10 (line 28) | def get_dataloaders_cifar10(batch_size, num_workers=0,
function reporthook (line 105) | def reporthook(count, block_size, total_size):
function download_dataset (line 122) | def download_dataset():
function load_dataset_into_to_dataframe (line 138) | def load_dataset_into_to_dataframe():
function partition_dataset (line 173) | def partition_dataset(df):
class IMDBDataset (line 185) | class IMDBDataset(Dataset):
method __init__ (line 186) | def __init__(self, dataset_dict, partition_key="train"):
method __getitem__ (line 189) | def __getitem__(self, index):
method __len__ (line 192) | def __len__(self):
FILE: 03_training-dnns/solution/solution-torchvision-cnn.py
class PyTorchCNN (line 13) | class PyTorchCNN(torch.nn.Module):
method __init__ (line 14) | def __init__(self, num_classes):
method forward (line 45) | def forward(self, x):
function train (line 53) | def train(num_epochs, model, optimizer, scheduler, train_loader, val_loa...
FILE: 04_accelerating-pytorch/exercise/00_pytorch-vit-random-init.py
function train (line 15) | def train(num_epochs, model, optimizer, train_loader, val_loader, device):
FILE: 04_accelerating-pytorch/exercise/01_pytorch-vit.py
function train (line 15) | def train(num_epochs, model, optimizer, train_loader, val_loader, device):
FILE: 04_accelerating-pytorch/exercise/local_utilities.py
function get_dataloaders_cifar10 (line 9) | def get_dataloaders_cifar10(batch_size, num_workers=0,
FILE: 04_accelerating-pytorch/solution/02_fabric-vit.py
function train (line 16) | def train(num_epochs, model, optimizer, train_loader, val_loader, fabric):
FILE: 04_accelerating-pytorch/solution/03_fabric-vit-mixed-precision.py
function train (line 16) | def train(num_epochs, model, optimizer, train_loader, val_loader, fabric):
FILE: 04_accelerating-pytorch/solution/04_fabric-vit-mixed-fsdp.py
function train (line 16) | def train(num_epochs, model, optimizer, train_loader, val_loader, fabric):
FILE: 04_accelerating-pytorch/solution/local_utilities.py
function get_dataloaders_cifar10 (line 9) | def get_dataloaders_cifar10(batch_size, num_workers=0,
FILE: 05_finetuning-llms/exercise/gpt_download.py
function download_and_load_gpt2 (line 15) | def download_and_load_gpt2(model_size, models_dir):
function download_file (line 45) | def download_file(url, destination):
function load_gpt2_params_from_tf_ckpt (line 73) | def load_gpt2_params_from_tf_ckpt(ckpt_path, settings):
FILE: 05_finetuning-llms/exercise/spam_classifier_utils.py
class GPTDatasetV1 (line 23) | class GPTDatasetV1(Dataset):
method __init__ (line 24) | def __init__(self, txt, tokenizer, max_length, stride):
method __len__ (line 39) | def __len__(self):
method __getitem__ (line 42) | def __getitem__(self, idx):
function create_dataloader_v1 (line 46) | def create_dataloader_v1(txt, batch_size=4, max_length=256,
class MultiHeadAttention (line 64) | class MultiHeadAttention(nn.Module):
method __init__ (line 65) | def __init__(self, d_in, d_out, context_length, dropout, num_heads, qk...
method forward (line 80) | def forward(self, x):
class LayerNorm (line 123) | class LayerNorm(nn.Module):
method __init__ (line 124) | def __init__(self, emb_dim):
method forward (line 130) | def forward(self, x):
class GELU (line 137) | class GELU(nn.Module):
method __init__ (line 138) | def __init__(self):
method forward (line 141) | def forward(self, x):
class FeedForward (line 148) | class FeedForward(nn.Module):
method __init__ (line 149) | def __init__(self, cfg):
method forward (line 157) | def forward(self, x):
class TransformerBlock (line 161) | class TransformerBlock(nn.Module):
method __init__ (line 162) | def __init__(self, cfg):
method forward (line 176) | def forward(self, x):
class GPTModel (line 194) | class GPTModel(nn.Module):
method __init__ (line 195) | def __init__(self, cfg):
method forward (line 207) | def forward(self, in_idx):
function generate_text_simple (line 219) | def generate_text_simple(model, idx, max_new_tokens, context_size):
function assign (line 248) | def assign(left, right):
function load_weights_into_gpt (line 254) | def load_weights_into_gpt(gpt, params):
function generate (line 315) | def generate(model, idx, max_new_tokens, context_size, temperature, top_...
class SpamDataset (line 355) | class SpamDataset(Dataset):
method __init__ (line 356) | def __init__(self, csv_file, tokenizer, max_length=None, pad_token_id=...
method __getitem__ (line 380) | def __getitem__(self, index):
method __len__ (line 385) | def __len__(self):
method _longest_encoded_length (line 388) | def _longest_encoded_length(self):
function calc_loss_batch (line 397) | def calc_loss_batch(input_batch, target_batch, model, device):
function calc_loss_loader (line 405) | def calc_loss_loader(data_loader, model, device, num_batches=None):
function calc_accuracy_loader (line 425) | def calc_accuracy_loader(data_loader, model, device, num_batches=None):
function evaluate_model (line 447) | def evaluate_model(model, train_loader, val_loader, device, eval_iter):
function plot_values (line 456) | def plot_values(epochs_seen, examples_seen, train_values, val_values, la...
function classify_review (line 476) | def classify_review(text, model, tokenizer, device, pad_length=None, pad...
FILE: 05_finetuning-llms/solution/gpt_download.py
function download_and_load_gpt2 (line 15) | def download_and_load_gpt2(model_size, models_dir):
function download_file (line 45) | def download_file(url, destination):
function load_gpt2_params_from_tf_ckpt (line 73) | def load_gpt2_params_from_tf_ckpt(ckpt_path, settings):
FILE: 05_finetuning-llms/solution/spam_classifier_utils.py
class GPTDatasetV1 (line 23) | class GPTDatasetV1(Dataset):
method __init__ (line 24) | def __init__(self, txt, tokenizer, max_length, stride):
method __len__ (line 39) | def __len__(self):
method __getitem__ (line 42) | def __getitem__(self, idx):
function create_dataloader_v1 (line 46) | def create_dataloader_v1(txt, batch_size=4, max_length=256,
class MultiHeadAttention (line 64) | class MultiHeadAttention(nn.Module):
method __init__ (line 65) | def __init__(self, d_in, d_out, context_length, dropout, num_heads, qk...
method forward (line 80) | def forward(self, x):
class LayerNorm (line 123) | class LayerNorm(nn.Module):
method __init__ (line 124) | def __init__(self, emb_dim):
method forward (line 130) | def forward(self, x):
class GELU (line 137) | class GELU(nn.Module):
method __init__ (line 138) | def __init__(self):
method forward (line 141) | def forward(self, x):
class FeedForward (line 148) | class FeedForward(nn.Module):
method __init__ (line 149) | def __init__(self, cfg):
method forward (line 157) | def forward(self, x):
class TransformerBlock (line 161) | class TransformerBlock(nn.Module):
method __init__ (line 162) | def __init__(self, cfg):
method forward (line 176) | def forward(self, x):
class GPTModel (line 194) | class GPTModel(nn.Module):
method __init__ (line 195) | def __init__(self, cfg):
method forward (line 207) | def forward(self, in_idx):
function generate_text_simple (line 219) | def generate_text_simple(model, idx, max_new_tokens, context_size):
function assign (line 248) | def assign(left, right):
function load_weights_into_gpt (line 254) | def load_weights_into_gpt(gpt, params):
function generate (line 315) | def generate(model, idx, max_new_tokens, context_size, temperature, top_...
class SpamDataset (line 355) | class SpamDataset(Dataset):
method __init__ (line 356) | def __init__(self, csv_file, tokenizer, max_length=None, pad_token_id=...
method __getitem__ (line 380) | def __getitem__(self, index):
method __len__ (line 385) | def __len__(self):
method _longest_encoded_length (line 388) | def _longest_encoded_length(self):
function calc_loss_batch (line 397) | def calc_loss_batch(input_batch, target_batch, model, device):
function calc_loss_loader (line 405) | def calc_loss_loader(data_loader, model, device, num_batches=None):
function calc_accuracy_loader (line 425) | def calc_accuracy_loader(data_loader, model, device, num_batches=None):
function evaluate_model (line 447) | def evaluate_model(model, train_loader, val_loader, device, eval_iter):
function plot_values (line 456) | def plot_values(epochs_seen, examples_seen, train_values, val_values, la...
function classify_review (line 476) | def classify_review(text, model, tokenizer, device, pad_length=None, pad...
Condensed preview — 47 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (853K chars).
[
{
"path": ".gitignore",
"chars": 3189,
"preview": ".DS_Store\n\n# Temporary files\nold\n\n# Solution files\n# solution\n# **/solution\n\n\n# Large slide files\n*.key\n*.pdf\n\n# Byte-co"
},
{
"path": "00-1_python-setup-guide/README.md",
"chars": 2827,
"preview": "# Python Setup Tips\n\n \n\n> [!TIP]\n> A reproducible cloud environment will be shared with participants on the day of "
},
{
"path": "00-2_python-libraries-for-workshop/README.md",
"chars": 2197,
"preview": "# Libraries Used In This Workshop\n\n \n\n> [!TIP]\n> A reproducible cloud environment will be shared with participants "
},
{
"path": "00-2_python-libraries-for-workshop/jupyter_environment_check.ipynb",
"chars": 2839,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 1,\n \"id\": \"18d54544-92d0-412c-8e28-f9083b2bab6f\",\n \""
},
{
"path": "00-2_python-libraries-for-workshop/python_environment_check.py",
"chars": 1877,
"preview": "import platform\nimport sys\nfrom packaging.version import parse as version_parse\n\nif version_parse(platform.python_versio"
},
{
"path": "00-2_python-libraries-for-workshop/requirements.txt",
"chars": 247,
"preview": "numpy >= 1.26.4\nscipy >= 1.13.0\npandas >= 2.2.2\nmatplotlib >= 3.8.4\nscikit-learn >= 1.4.2\njupyterlab >= 4.0\nipywidgets >"
},
{
"path": "01_intro-to-deeplearning/README.md",
"chars": 200,
"preview": "# PyCon2024 Workshop\n\n## 1) Introduction to Deep Learning & Setup\n\nNo code or exercises in this section.\n\n[🔗 Slides](htt"
},
{
"path": "02_pytorch-api/README.md",
"chars": 644,
"preview": "# PyCon2024 Workshop\n\n## 2) Understanding the PyTorch API\n\n[🔗 Slides](https://sebastianraschka.com/pdf/pycon2024/02_pyto"
},
{
"path": "02_pytorch-api/exercise/exercise-logreg.ipynb",
"chars": 13458,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"id\": \"d71bce70-9dc3-448b-9f9a-8896e83b6d09\",\n \"metadata\": {},\n \"so"
},
{
"path": "02_pytorch-api/exercise/exercise.py",
"chars": 5819,
"preview": "# %% [markdown]\n# # Logistic Regression Classifier\n\n# %% [markdown]\n# ## 1) Installing Libraries\n\n# %%\n%load_ext waterma"
},
{
"path": "02_pytorch-api/exercise/toydata-truncated.txt",
"chars": 268,
"preview": "x1\tx2\tlabel\n0.77\t-1.14\t0\n-0.33\t1.44\t0\n0.91\t-3.07\t0\n-0.37\t-1.91\t0\n-0.63\t-1.53\t0\n0.39\t-1.99\t0\n-0.49\t-2.74\t0\n-0.68\t-1.52\t0\n"
},
{
"path": "02_pytorch-api/solution/solution-logreg.ipynb",
"chars": 119582,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"id\": \"d71bce70-9dc3-448b-9f9a-8896e83b6d09\",\n \"metadata\": {},\n \"so"
},
{
"path": "02_pytorch-api/solution/solution.py",
"chars": 5712,
"preview": "# %% [markdown]\n# # Logistic Regression Classifier\n\n# %% [markdown]\n# ## 1) Installing Libraries\n\n# %%\n# !conda install "
},
{
"path": "02_pytorch-api/solution/toydata-truncated.txt",
"chars": 268,
"preview": "x1\tx2\tlabel\n0.77\t-1.14\t0\n-0.33\t1.44\t0\n0.91\t-3.07\t0\n-0.37\t-1.91\t0\n-0.63\t-1.53\t0\n0.39\t-1.99\t0\n-0.49\t-2.74\t0\n-0.68\t-1.52\t0\n"
},
{
"path": "03_training-dnns/README.md",
"chars": 507,
"preview": "# PyCon2024 Workshop\n\n## 3) Training Deep Neural Networks\n\n[🔗 Slides](https://sebastianraschka.com/pdf/pycon2024/03_trai"
},
{
"path": "03_training-dnns/exercise/example.ipynb",
"chars": 20488,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 2,\n \"id\": \"6b7b44c2-57a9-4922-86c4-2b0695123aa9\",\n \""
},
{
"path": "03_training-dnns/exercise/local_utilities.py",
"chars": 5959,
"preview": "# Imports for ViT finetuning\n\nimport torch\nfrom torch.utils.data import sampler\nfrom torchvision import datasets\nfrom to"
},
{
"path": "03_training-dnns/exercise/simple-cnn.py",
"chars": 5840,
"preview": "import time\n\nimport lightning as L\nimport torch\nimport torch.nn.functional as F\nimport torchmetrics\nfrom torchvision imp"
},
{
"path": "03_training-dnns/solution/local_utilities.py",
"chars": 5959,
"preview": "# Imports for ViT finetuning\n\nimport torch\nfrom torch.utils.data import sampler\nfrom torchvision import datasets\nfrom to"
},
{
"path": "03_training-dnns/solution/solution-torchvision-cnn.py",
"chars": 5981,
"preview": "import time\n\nimport lightning as L\nimport torch\nimport torch.nn.functional as F\nimport torchmetrics\nfrom torchvision imp"
},
{
"path": "04_accelerating-pytorch/README.md",
"chars": 651,
"preview": "# PyCon2024 Workshop\n\n## 4) Accelerating PyTorch Model Training\n\n[🔗 Slides](https://sebastianraschka.com/pdf/pycon2024/0"
},
{
"path": "04_accelerating-pytorch/exercise/00_pytorch-vit-random-init.py",
"chars": 4443,
"preview": "import time\n\nimport lightning as L\nimport torch\nimport torch.nn.functional as F\nimport torchmetrics\nfrom torchvision imp"
},
{
"path": "04_accelerating-pytorch/exercise/01_pytorch-vit.py",
"chars": 4468,
"preview": "import time\n\nimport lightning as L\nimport torch\nimport torch.nn.functional as F\nimport torchmetrics\nfrom torchvision imp"
},
{
"path": "04_accelerating-pytorch/exercise/local_utilities.py",
"chars": 2583,
"preview": "import torch\nfrom torch.utils.data import sampler\nfrom torchvision import datasets\nfrom torch.utils.data import DataLoad"
},
{
"path": "04_accelerating-pytorch/solution/02_fabric-vit.py",
"chars": 4426,
"preview": "import time\n\nimport lightning as L\nfrom lightning import Fabric\nimport torch\nimport torch.nn.functional as F\nimport torc"
},
{
"path": "04_accelerating-pytorch/solution/03_fabric-vit-mixed-precision.py",
"chars": 4499,
"preview": "import time\n\nimport lightning as L\nfrom lightning import Fabric\nimport torch\nimport torch.nn.functional as F\nimport torc"
},
{
"path": "04_accelerating-pytorch/solution/04_fabric-vit-mixed-fsdp.py",
"chars": 4544,
"preview": "import time\n\nimport lightning as L\nfrom lightning import Fabric\nimport torch\nimport torch.nn.functional as F\nimport torc"
},
{
"path": "04_accelerating-pytorch/solution/local_utilities.py",
"chars": 2583,
"preview": "import torch\nfrom torch.utils.data import sampler\nfrom torchvision import datasets\nfrom torch.utils.data import DataLoad"
},
{
"path": "05_finetuning-llms/README.md",
"chars": 889,
"preview": "# PyCon2024 Workshop\n\n## 5) Finetuning Large Language Models\n\n[🔗 Slides](https://sebastianraschka.com/pdf/pycon2024/05_f"
},
{
"path": "05_finetuning-llms/exercise/01_train-gpt.ipynb",
"chars": 103793,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"id\": \"c024bfa4-1a7a-4751-b5a1-827225a3478b\",\n \"metadata\": {\n \"id\""
},
{
"path": "05_finetuning-llms/exercise/02_use-trained-gpt.ipynb",
"chars": 4058,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Use the finetuned LLM in a New Se"
},
{
"path": "05_finetuning-llms/exercise/gpt_download.py",
"chars": 3848,
"preview": "# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).\n# Source for \"Build a Large Language Model"
},
{
"path": "05_finetuning-llms/exercise/requirements-extra.txt",
"chars": 70,
"preview": "tiktoken >= 0.6 # tokenizer\ntensorflow >= 2.16.1 # weight loading"
},
{
"path": "05_finetuning-llms/exercise/spam_classifier_utils.py",
"chars": 18433,
"preview": "# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).\n# Source for \"Build a Large Language Model"
},
{
"path": "05_finetuning-llms/exercise/test.csv",
"chars": 31713,
"preview": "Label,Text\n1,85233 FREE>Ringtone!Reply REAL\n1,Ur cash-balance is currently 500 pounds - to maximize ur cash-in now send "
},
{
"path": "05_finetuning-llms/exercise/train.csv",
"chars": 112506,
"preview": "Label,Text\n0,Dude how do you like the buff wind.\n0,Tessy..pls do me a favor. Pls convey my birthday wishes to Nimya..pls"
},
{
"path": "05_finetuning-llms/exercise/validation.csv",
"chars": 17716,
"preview": "Label,Text\n1,\"Mila, age23, blonde, new in UK. I look sex with UK guys. if u like fun with me. Text MTALK to 69866.18 . 3"
},
{
"path": "05_finetuning-llms/solution/01_train-gpt-solution.ipynb",
"chars": 103939,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"id\": \"c024bfa4-1a7a-4751-b5a1-827225a3478b\",\n \"metadata\": {\n \"id\""
},
{
"path": "05_finetuning-llms/solution/02_use-trained-gpt-solution.ipynb",
"chars": 4160,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Use the finetuned LLM in a New Se"
},
{
"path": "05_finetuning-llms/solution/gpt_download.py",
"chars": 3848,
"preview": "# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).\n# Source for \"Build a Large Language Model"
},
{
"path": "05_finetuning-llms/solution/requirements-extra.txt",
"chars": 70,
"preview": "tiktoken >= 0.6 # tokenizer\ntensorflow >= 2.16.1 # weight loading"
},
{
"path": "05_finetuning-llms/solution/spam_classifier_utils.py",
"chars": 18433,
"preview": "# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).\n# Source for \"Build a Large Language Model"
},
{
"path": "05_finetuning-llms/solution/test.csv",
"chars": 31713,
"preview": "Label,Text\n1,85233 FREE>Ringtone!Reply REAL\n1,Ur cash-balance is currently 500 pounds - to maximize ur cash-in now send "
},
{
"path": "05_finetuning-llms/solution/train.csv",
"chars": 112506,
"preview": "Label,Text\n0,Dude how do you like the buff wind.\n0,Tessy..pls do me a favor. Pls convey my birthday wishes to Nimya..pls"
},
{
"path": "05_finetuning-llms/solution/validation.csv",
"chars": 17716,
"preview": "Label,Text\n1,\"Mila, age23, blonde, new in UK. I look sex with UK guys. if u like fun with me. Text MTALK to 69866.18 . 3"
},
{
"path": "LICENSE",
"chars": 1074,
"preview": "MIT License\n\nCopyright (c) 2024 Sebastian Raschka\n\nPermission is hereby granted, free of charge, to any person obtaining"
},
{
"path": "README.md",
"chars": 3421,
"preview": "# PyCon US 2024: The Fundamentals of Modern Deep Learning with PyTorch\nTutorial materials for **The Fundamentals of Mode"
}
]
About this extraction
This page contains the full source code of the rasbt/pycon2024 GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 47 files (802.7 KB), approximately 325.8k tokens, and a symbol index with 135 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.