Showing preview only (6,794K chars total). Download the full file or copy to clipboard to get everything.
Repository: jonasz/progressive_infogan
Branch: master
Commit: 8c22ad438dc6
Files: 51
Total size: 6.5 MB
Directory structure:
gitextract_uedhrux4/
├── .gitignore
├── README.md
└── src/
├── jonasz/
│ ├── __init__.py
│ ├── cifar10/
│ │ ├── __init__.py
│ │ └── cifar10_dataset.py
│ ├── constants.py
│ ├── experiments/
│ │ ├── 2018_08_28/
│ │ │ ├── __init__.py
│ │ │ ├── exp_append_channels.py
│ │ │ ├── exp_append_simple.py
│ │ │ ├── exp_condition.py
│ │ │ ├── exp_condition_4vars.py
│ │ │ ├── exp_condition_8vars.py
│ │ │ ├── exp_consistency_30.py
│ │ │ ├── exp_consistency_300.py
│ │ │ ├── exp_gradual_loss.py
│ │ │ ├── exp_noinfo.py
│ │ │ ├── exp_unmask.py
│ │ │ └── exp_vanilla.py
│ │ └── __init__.py
│ ├── gan/
│ │ ├── __init__.py
│ │ └── evaluation.py
│ ├── lib/
│ │ ├── __init__.py
│ │ ├── datasets.py
│ │ ├── presentation_model_features.py
│ │ ├── progressive_infogan_ipynb_utils.py
│ │ ├── tensor_util.py
│ │ └── util.py
│ ├── notebooks/
│ │ └── generator.ipynb
│ ├── nvidia_celeb/
│ │ ├── __init__.py
│ │ ├── celeba_align_dataset.py
│ │ └── celeba_hq_dataset.py
│ ├── progressive_infogan/
│ │ ├── __init__.py
│ │ ├── create_animation.py
│ │ ├── export_utils.py
│ │ ├── gcloud_training/
│ │ │ ├── __init__.py
│ │ │ ├── command.sh
│ │ │ ├── config.yaml
│ │ │ ├── config_4gpus.yaml
│ │ │ ├── config_8gpus.yaml
│ │ │ ├── determine_config_yaml.py
│ │ │ ├── task.py
│ │ │ └── test.py
│ │ ├── info_utils.py
│ │ ├── network_utils.py
│ │ ├── networks.py
│ │ ├── progressive_infogan_lib.py
│ │ ├── progressive_infogan_losses.py
│ │ ├── run_evaluation.py
│ │ └── train.py
│ └── tools/
│ └── tensorboard_gcloud.py
└── setup.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
*.pyc
*.swp
*.swo
src/dist/*
src/*.egg-info/
.ipynb_checkpoints
frozen_inception_v1_2015_12_05.tar.gz
src/jonasz/notebooks/progressive_infogan_models_master_WIP
================================================
FILE: README.md
================================================
# Progressive InfoGAN
Progressive InfoGAN combines two techniques:
- progressive training (https://arxiv.org/abs/1710.10196), and
- InfoGAN (https://arxiv.org/abs/1606.03657).
## What it does
- A Generator is trained on any unlabeled high-res dataset of images. For our experiments we use [CelebA-HQ](https://github.com/tkarras/progressive_growing_of_gans#preparing-datasets-for-training).
- After training, the Generator produces novel images, similar to those in the dataset.
- When producing an image, we can independently control its semantic feautres. In case of CelebA-HQ, semantic features include: the direction of the look, hair color, nose shape, and much more. See [Summary of selected isolated features](#summary-of-selected-isolated-features).
- Most importantly, these semantic features are discovered during training in a completely unsupervised fashion - no human input is required.
## Under the hood
The InfoGAN technique was adapted to the progressive architecture of the model by splitting the structured code to parts, and feeding each part to the network by conditioning activations in the corresponding block of the Generator. You can find a detailed description of the architecture, along with quality assessment, in my [Master's Thesis text](https://raw.githubusercontent.com/jonasz/progressive_infogan/master/data/progressive_infogan_draft.pdf) (expected defense: Q4 2018).
## Running
```bash
# Training:
# Fill in src/constants.py, and run
$ cd src
$ python jonasz/experiments/2018_08_28/exp_consistency_300.py
# Working with the trained Generator:
$ cd src
$ python -m jupyter notebook --port 5088
```
Now you can experiment with the generator in a notebook: [src/jonasz/notebooks/generator.ipynb](src/jonasz/notebooks/generator.ipynb)
## Summary of selected isolated features
See also the animated, extended version of the summary at: [https://youtu.be/U2okTa0JGZg](https://youtu.be/U2okTa0JGZg). Some of the subtler changes are better visible when animated. For each of the features listed below, there is a link to the exact corresponding timestamp in the animation.
[Smile: upper lip](#smile-upper-lip-animation02m43s)
[Age](#age-animation04m22s)
[Look direction](#look-direction-animation03m16s)
[Hair color](#hair-color-animation00m06s)
[Left / right rotation](#left--right-rotation-animation00m15s)
[Face oval: size](#face-oval-size-animation00m28s)
[Hairstyle: background size](#hairstyle-background-size-animation01m08s)
[Lower jaw size](#lower-jaw-size-animation01m51s)
[Forward backward inclination](#forward--backward-inclination-animation01m41s)
[Mouth: open / closed](#mouth-open--closed-animation01m31s)
[Hair: wavy / straight](#hair-wavy--straight-animation02m11s)
[Eyebrows: up / down](#eyebrows-up--down-animation02m56s)
[Nose length](#nose-length-animation04m02s)
[Nose: upturned tip](#nose-upturned-tip-animation03m52s)
[Eyebrows shape](#eyebrows-shape-animation03m06s)
[Vertical face stretch](#vertical-face-stretch-animation03m32s)
[Color of the irises](#color-of-the-irises-animation05m07s)
[Shape of the nostrils](#shape-of-the-nostrils-animation05m18s)
[Hair texture](#hair-texture-animation06m10s)
[Lower eyelid](#lower-eyelid-animation05m54s)
[Wrinkles](#wrinkles-animation05m27s)
A comprehensive list of all isolated features is at: [https://youtu.be/mOckeVkM1jU](https://youtu.be/mOckeVkM1jU).
### Smile: upper lip ([animation@02m43s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=02m43s)):
### Age ([animation@04m22s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=04m22s)):
### Look direction ([animation@03m16s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=03m16s)):
### Hair color ([animation@00m06s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=00m06s)):
### Left / right rotation ([animation@00m15s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=00m15s)):
### Face oval: size ([animation@00m28s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=00m28s)):
### Hairstyle: background size ([animation@01m08s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=01m08s)):
### Lower jaw size ([animation@01m51s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=01m51s)):
### Forward / backward inclination ([animation@01m41s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=01m41s)):
### Mouth: open / closed ([animation@01m31s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=01m31s)):
### Hair: wavy / straight ([animation@02m11s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=02m11s)):
### Eyebrows: up / down ([animation@02m56s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=02m56s)):
### Nose length ([animation@04m02s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=04m02s)):
### Nose: upturned tip ([animation@03m52s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=03m52s)):
### Eyebrows shape ([animation@03m06s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=03m06s)):
### Vertical face stretch ([animation@03m32s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=03m32s)):
### Color of the irises ([animation@05m07s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=05m07s)):
### Shape of the nostrils ([animation@05m18s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=05m18s)):
### Hair texture ([animation@06m10s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=06m10s)):
### Lower eyelid ([animation@05m54s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=05m54s)):
### Wrinkles ([animation@05m27s](https://www.youtube.com/watch?v=U2okTa0JGZg&t=05m27s)):
================================================
FILE: src/jonasz/__init__.py
================================================
================================================
FILE: src/jonasz/cifar10/__init__.py
================================================
================================================
FILE: src/jonasz/cifar10/cifar10_dataset.py
================================================
"""Utility class for loading cifar 10 data.
The "CIFAR-10 python version" data format described at
https://www.cs.toronto.edu/~kriz/cifar.html is assumed.
"""
import os.path
import pickle
import numpy as np
import tensorflow as tf
from google.cloud import storage as gcs
from jonasz import constants
from jonasz.lib import util
class DatasetParams(util.Params):
def get_allowed_params_with_defaults(self):
return {
'values_range': (0., 1.),
'gcs_bucket': None, # 'seer-of-visions-ml2',
# Older code uses data_dir. Newer code uses *tfrecord stuff.
'data_dir': None, # 'cifar-10-batches-py',
'train_data_tfrecord': None,
'test_data_tfrecord': None,
'train_max_crop_shift': 0,
'train_size_for_crop': None,
'train_can_flip': False,
'train_hue_max_delta': 0.0,
'train_shuffle': False,
'train_random_contrast': (1.-1e-5, 1.),
'train_brightness_max_delta': 0.,
'test_shuffle': True,
'data_format': 'NCHW',
'include_test_data_for_training': False,
}
def validate(self):
if self.train_data_tfrecord:
assert self.data_dir is None
if self.data_dir:
assert self.train_data_tfrecord is None
assert self.test_data_tfrecord is None
def _load_from_gcs(gcs_bucket, data_dir, filename):
if gcs_bucket is None:
with open(os.path.join(data_dir, filename)) as f:
data = f.read()
return data
client = gcs.Client()
bucket = client.get_bucket(gcs_bucket)
blob = bucket.get_blob(data_dir + '/' + filename)
return blob.download_as_string()
def _load_dict(gcs_bucket, data_dir, filename):
print 'loading', filename
blob = _load_from_gcs(gcs_bucket, data_dir, filename)
d_raw = pickle.loads(blob)
d = {}
d['labels'] = np.array(d_raw[b'labels'], dtype=np.int32)
images = np.array(d_raw[b'data'])
new_images = []
for img in images:
new_img = img.astype(np.float32) / 256.
new_img = new_img.reshape(3, 32, 32)
new_images.append(new_img)
d['data'] = np.array(new_images)
return d
def _load_train_data(gcs_bucket, data_dir):
train_data_dict = None
for i in range(5):
cur_dict = _load_dict(gcs_bucket, data_dir, 'data_batch_%d' % (i+1))
if train_data_dict is None:
train_data_dict = cur_dict
else:
for key in train_data_dict.keys():
train_data_dict[key] = np.concatenate((
train_data_dict[key], cur_dict[key]))
return train_data_dict
def _write_data_to_tfrecord(data_dict, path):
data, labels = data_dict['data'], data_dict['labels']
with tf.python_io.TFRecordWriter(path) as writer:
for img, label in zip(data, labels):
example = tf.train.Example()
example.features.feature['images'].float_list.value.extend(img.reshape(-1))
example.features.feature['labels'].int64_list.value.append(label)
writer.write(example.SerializeToString())
def write_train_data_to_tfrecord(target_path, data_dir):
train_data = _load_train_data(None, data_dir)
_write_data_to_tfrecord(train_data, target_path)
def write_test_data_to_tfrecord(target_path, data_dir):
test_data = _load_test_data(None, data_dir)
_write_data_to_tfrecord(test_data, target_path)
def _load_test_data(gcs_bucket, data_dir):
return _load_dict(gcs_bucket, data_dir, 'test_batch')
def _random_crop(img, params):
size = params.train_size_for_crop
if size is None:
return img
img = tf.image.resize_images(img, [size, size])
max_offset = size - 32
offset_height = tf.random_uniform([], minval=0, maxval=max_offset,
dtype=tf.int32)
offset_width = tf.random_uniform([], minval=0, maxval=max_offset,
dtype=tf.int32)
img = tf.image.crop_to_bounding_box(img, offset_height, offset_width,
32, 32)
return img
def _random_shift(img, params):
max_shift = params.train_max_crop_shift
if not max_shift:
return img
size = 32 + max_shift * 2
img = tf.image.pad_to_bounding_box(img, max_shift, max_shift, size, size)
offset_height = tf.random_uniform([], minval=0, maxval=max_shift*2+1,
dtype=tf.int32)
offset_width = tf.random_uniform([], minval=0, maxval=max_shift*2+1,
dtype=tf.int32)
img = tf.image.crop_to_bounding_box(img, offset_height, offset_width,
32, 32)
return img
def _image_noise(params):
def f(img, label):
img = tf.transpose(img, [1, 2, 0]) # CHW to HWC
if params.train_can_flip:
img = tf.image.random_flip_left_right(img)
img = tf.image.random_brightness(
img, max_delta=params.train_brightness_max_delta)
img = tf.clip_by_value(img, 0., 1.) # Why doesn't random_brightness do that?
img = tf.image.random_hue(img, max_delta=params.train_hue_max_delta)
img = tf.image.random_contrast(img, *params.train_random_contrast)
img = _random_crop(img, params)
img = _random_shift(img, params)
if params.data_format == 'NCHW':
img = tf.transpose(img, [2, 0, 1]) # HWC TO CHW
return img, label
return f
def _shift_pixel_values(img, params):
left, right = params.values_range
img = img * (right - left) + left
return img
def _example_to_img_and_label(serialized_example):
features = {
'images': tf.FixedLenFeature([3, 32, 32], tf.float32),
'labels': tf.FixedLenFeature([], tf.int64)
}
d = tf.parse_single_example(serialized_example, features)
return d['images'], d['labels']
def get_train_input_fn(params, batch_size=128):
# NOTE: When working with Dataset.from_tensor_slices, TensorFlow produces
# ridiculously large log files. It seems that the input itself, represented as
# a tf.constant, is serialized into the graph file, and dumped multiple times
# into the events file as well. This hurts performance considerably.
# For this reason the dataset is serialized into tfrecords, and we're working
# with TFRecordDataset.
if params.data_dir:
train_path = os.path.join(params.data_dir, 'train_data.tfrecord')
test_path = os.path.join(params.data_dir, 'test_data.tfrecord')
else:
train_path = params.train_data_tfrecord
test_path = params.test_data_tfrecord
if params.gcs_bucket:
train_path = os.path.join('gs://', params.gcs_bucket, train_path)
test_path = os.path.join('gs://', params.gcs_bucket, test_path)
paths = [train_path]
if params.include_test_data_for_training:
paths += [test_path]
def train_input_fn():
d = tf.data.TFRecordDataset(paths)
d = d.map(_example_to_img_and_label)
d = d.repeat()
if params.train_shuffle:
d = d.shuffle(batch_size * 100)
d = d.map(
_image_noise(params),
num_parallel_calls=8)
d = d.map(
lambda img, label: (_shift_pixel_values(img, params), label),
num_parallel_calls=8)
# TODO: this can likely be optimized better.
d = d.batch(batch_size)
d = d.prefetch(buffer_size=8) # TODO: this means 8 batches, right?
iterator = d.make_one_shot_iterator()
imgs, labels = iterator.get_next()
return {'images': imgs}, labels
return train_input_fn
def get_test_input_fn(params, batch_size=128):
assert params.data_format == 'NCHW'
if params.test_data_tfrecord:
path = params.test_data_tfrecord
else:
path = os.path.join(params.data_dir, 'test_data.tfrecord')
if params.gcs_bucket:
path = os.path.join('gs://', params.gcs_bucket, path)
def test_input_fn():
d = tf.data.TFRecordDataset(path)
d = d.map(_example_to_img_and_label)
d = d.repeat()
if params.test_shuffle:
d = d.shuffle(batch_size * 10)
d = d.map(
lambda img, label: (_shift_pixel_values(img, params), label),
num_parallel_calls=8)
d = d.batch(batch_size)
iterator = d.make_one_shot_iterator()
imgs, labels = iterator.get_next()
return {'images': imgs}, labels
return test_input_fn
def _get_class_name(label):
return {
0: 'plane',
1: 'car',
2: 'bird',
3: 'cat',
4: 'deer',
5: 'dog',
6: 'frog',
7: 'horse',
8: 'boat',
9: 'truck',
}[label]
def _test_params1(shuffle=True):
return DatasetParams(
gcs_bucket=None,
data_dir=constants.CIFAR10_DATA_DIR,
train_can_flip=True,
train_hue_max_delta=0.01,
train_shuffle=shuffle,
train_random_contrast=(.85, 1.),
train_brightness_max_delta=.05,
train_size_for_crop=40,
# train_max_crop_shift=3,
)
def _test_params2():
return DatasetParams(
gcs_bucket=None,
data_dir=constants.CIFAR10_DATA_DIR)
def test1():
print "Should show an identical frog seven times."
params = _test_params2()
test_imgs = []
for i in range(7):
with tf.Session(graph=tf.Graph()) as sess:
features, labels = get_train_input_fn(params=params, batch_size=1)()
imgs = features['images']
cur_imgs, cur_labels = sess.run([imgs, labels])
print ' '.join(map(_get_class_name, cur_labels))
cur_imgs = list(cur_imgs)
test_imgs.extend(cur_imgs)
util.show_imgs(test_imgs, data_type='CHW')
def test2():
print "Should show a frog seven times, each time reasonably modified."
params = _test_params1(shuffle=False)
test_imgs = []
for i in range(7):
with tf.Session(graph=tf.Graph()) as sess:
features, labels = get_train_input_fn(params=params, batch_size=1)()
imgs = features['images']
cur_imgs, cur_labels = sess.run([imgs, labels])
print ' '.join(map(_get_class_name, cur_labels))
cur_imgs = list(cur_imgs)
test_imgs.extend(cur_imgs)
util.show_imgs(test_imgs, data_type='CHW')
def test3():
print "Should always show first 7 images of the dataset, unchanged."
params = _test_params2()
with tf.Session(graph=tf.Graph()) as sess:
features, labels = get_train_input_fn(params=params, batch_size=7)()
imgs = features['images']
cur_imgs, cur_labels = sess.run([imgs, labels])
print ' '.join(map(_get_class_name, cur_labels))
cur_imgs = list(cur_imgs)
util.show_imgs(cur_imgs, data_type='CHW')
def test4():
print "Should show 7 randomly chosen, reasonably modified images."
params = _test_params1()
with tf.Session(graph=tf.Graph()) as sess:
features, labels = get_train_input_fn(params=params, batch_size=7)()
imgs = features['images']
cur_imgs, cur_labels = sess.run([imgs, labels])
print ' '.join(map(_get_class_name, cur_labels))
cur_imgs = list(cur_imgs)
util.show_imgs(cur_imgs, data_type='CHW')
def test5():
print "Should show 7 randomly chosen images from the test set."
params = _test_params2()
with tf.Session(graph=tf.Graph()) as sess:
features, labels = get_test_input_fn(params=params, batch_size=7)()
imgs = features['images']
cur_imgs, cur_labels = sess.run([imgs, labels])
print ' '.join(map(_get_class_name, cur_labels))
cur_imgs = list(cur_imgs)
util.show_imgs(cur_imgs, data_type='CHW')
================================================
FILE: src/jonasz/constants.py
================================================
# Where to write training data (checkpoints, logs, ...).
TRAINING_OUTPUT_BASE_DIR = None
# Datasets locations.
CIFAR10_DATA_DIR = None
NVIDIA_CELEBA_ALIGN_DATASET_PATH = None
NVIDIA_CELEBA_HQ_DATASET_PATH = None
# GCloud training, datasets.
NVIDIA_CELEBA_ALIGN_DATASET_PATH_GCLOUD = None
NVIDIA_CELEBA_HQ_DATASET_PATH_GCLOUD = None
GCLOUD_BUCKET = None
MISC_DATA = None
================================================
FILE: src/jonasz/experiments/2018_08_28/__init__.py
================================================
================================================
FILE: src/jonasz/experiments/2018_08_28/exp_append_channels.py
================================================
DESCRIPTION = __file__ + """
For each block from 2 to 6, the structured code is projected to a higher number
of channels and then appended to the block's input.
"""
import os
import re
import tensorflow as tf
from jonasz.progressive_infogan import train
from jonasz.progressive_infogan import networks
from jonasz.lib import util
from jonasz import constants
from jonasz.gan import evaluation
from jonasz.nvidia_celeb import celeba_hq_dataset
def training_params(is_gcloud=False, output_dir=None):
if not output_dir:
output_dir=util.construct_experiment_output_dir(__file__)
num_gpus = 1
stop_after = 7
dynamic_batch_size = {
2: 128,
3: 128,
4: 64,
5: 32,
6: 16,
7: 6,
8: 3,
}
imgs_per_phase = 384000
dynamic_steps_per_phase = {
phase: max(imgs_per_phase / batch_size, 6000)
for phase, batch_size in dynamic_batch_size.items()
}
dynamic_steps_per_phase[7] *= 2
return train.TrainingParams(
description=DESCRIPTION,
is_gcloud = is_gcloud,
num_gpus = num_gpus,
dataset_params = celeba_hq_dataset.get_dataset_params(
is_gcloud=is_gcloud,
crop_at_center=True),
checkpoint_every_n_steps = None,
checkpoint_every_n_secs = 2*60*60,
dynamic_steps_per_phase = dynamic_steps_per_phase,
dynamic_batch_size = dynamic_batch_size,
stop_after = stop_after,
eval_every_n_secs = 48*60*60,
write_summaries_every_n_steps = 700,
infogan_summary_reps = 0,
output_dir = output_dir,
allow_initial_partial_restore = True,
noise_size = 64,
noise_stddev = 1.,
summary_grid_size = 3,
# allow_simultaneous_steps = False,
infogan_cont_weight = 10.,
infogan_cont_depth_to_num_vars = {
2: 16,
3: 16,
4: 16,
5: 16,
6: 16,
7: 0,
8: 0,
},
generator_params=networks.GeneratorParams(
# debug_mode = True,
channels_at_4x4 = 2048,
# channels_max = 448, # Works
# channels_max = 496, # Fails at phase 8
channels_max = 480,
optimizer = ('adam_b0_b99', 0.0005),
ema_decay_for_visualization = .999,
weight_norm = 'equalized',
norm = 'batch_norm_in_place',
norm_per_gpu = True,
double_conv = True,
conditioning = False,
infogan_input_method = 'append_channels',
append_channels_div = 1,
),
discriminator_params=networks.DiscriminatorParams(
# debug_mode = True,
channels_at_2x2 = 4096,
channels_max = 512,
conditioning = False,
optimizer = ('adam_b0_b99', 0.0005),
weight_norm = 'equalized',
norm = None,
norm_per_gpu = True,
double_conv = True,
second_conv_channels_x2 = True,
# elastic_block_input = True,
# infogan_max_pool = True,
),
use_gpu_tower_scope = True,
)
def main(*args):
train.run_training(training_params())
if __name__ == '__main__':
tf.app.run()
================================================
FILE: src/jonasz/experiments/2018_08_28/exp_append_simple.py
================================================
DESCRIPTION = __file__ + """
For each block from 2 to 6, the structured code is appended to the block's
input.
"""
import os
import re
import tensorflow as tf
from jonasz.progressive_infogan import train
from jonasz.progressive_infogan import networks
from jonasz.lib import util
from jonasz import constants
from jonasz.gan import evaluation
from jonasz.nvidia_celeb import celeba_hq_dataset
def training_params(is_gcloud=False, output_dir=None):
if not output_dir:
output_dir=util.construct_experiment_output_dir(__file__)
num_gpus = 1
stop_after = 7
dynamic_batch_size = {
2: 128,
3: 128,
4: 64,
5: 32,
6: 16,
7: 6,
8: 3,
}
imgs_per_phase = 384000
dynamic_steps_per_phase = {
phase: max(imgs_per_phase / batch_size, 6000)
for phase, batch_size in dynamic_batch_size.items()
}
dynamic_steps_per_phase[7] *= 2
return train.TrainingParams(
description=DESCRIPTION,
is_gcloud = is_gcloud,
num_gpus = num_gpus,
dataset_params = celeba_hq_dataset.get_dataset_params(
is_gcloud=is_gcloud,
crop_at_center=True),
checkpoint_every_n_steps = None,
checkpoint_every_n_secs = 2*60*60,
dynamic_steps_per_phase = dynamic_steps_per_phase,
dynamic_batch_size = dynamic_batch_size,
stop_after = stop_after,
eval_every_n_secs = 48*60*60,
write_summaries_every_n_steps = 700,
infogan_summary_reps = 0,
output_dir = output_dir,
allow_initial_partial_restore = True,
noise_size = 64,
noise_stddev = 1.,
summary_grid_size = 3,
# allow_simultaneous_steps = False,
infogan_cont_weight = 10.,
infogan_cont_depth_to_num_vars = {
2: 16,
3: 16,
4: 16,
5: 16,
6: 16,
7: 0,
8: 0,
},
generator_params=networks.GeneratorParams(
# debug_mode = True,
channels_at_4x4 = 2048,
# channels_max = 448, # Works
# channels_max = 496, # Fails at phase 8
channels_max = 480,
optimizer = ('adam_b0_b99', 0.0005),
ema_decay_for_visualization = .999,
weight_norm = 'equalized',
norm = 'batch_norm_in_place',
norm_per_gpu = True,
double_conv = True,
conditioning = False,
infogan_input_method = 'append',
),
discriminator_params=networks.DiscriminatorParams(
# debug_mode = True,
channels_at_2x2 = 4096,
channels_max = 512,
conditioning = False,
optimizer = ('adam_b0_b99', 0.0005),
weight_norm = 'equalized',
norm = None,
norm_per_gpu = True,
double_conv = True,
second_conv_channels_x2 = True,
# elastic_block_input = True,
# infogan_max_pool = True,
),
use_gpu_tower_scope = True,
)
def main(*args):
train.run_training(training_params())
if __name__ == '__main__':
tf.app.run()
================================================
FILE: src/jonasz/experiments/2018_08_28/exp_condition.py
================================================
DESCRIPTION = __file__ + """
Each block from 2 to 6 has 16 structured variables. The activations within the
block are conditioned with these variables.
"""
import os
import re
import tensorflow as tf
from jonasz.progressive_infogan import train
from jonasz.progressive_infogan import networks
from jonasz.lib import util
from jonasz import constants
from jonasz.gan import evaluation
from jonasz.nvidia_celeb import celeba_hq_dataset
def training_params(is_gcloud=False, output_dir=None):
if not output_dir:
output_dir=util.construct_experiment_output_dir(__file__)
num_gpus = 1
stop_after = 7
dynamic_batch_size = {
2: 128,
3: 128,
4: 64,
5: 32,
6: 16,
7: 6,
8: 3,
}
imgs_per_phase = 384000
dynamic_steps_per_phase = {
phase: max(imgs_per_phase / batch_size, 6000)
for phase, batch_size in dynamic_batch_size.items()
}
return train.TrainingParams(
description=DESCRIPTION,
is_gcloud = is_gcloud,
num_gpus = num_gpus,
dataset_params = celeba_hq_dataset.get_dataset_params(
is_gcloud=is_gcloud,
crop_at_center=True),
checkpoint_every_n_steps = None,
checkpoint_every_n_secs = 2*60*60,
dynamic_steps_per_phase = dynamic_steps_per_phase,
dynamic_batch_size = dynamic_batch_size,
stop_after = stop_after,
eval_every_n_secs = 48*60*60,
write_summaries_every_n_steps = 700,
infogan_summary_reps = 0,
output_dir = output_dir,
allow_initial_partial_restore = True,
noise_size = 64,
noise_stddev = 1.,
summary_grid_size = 3,
# allow_simultaneous_steps = False,
infogan_cont_weight = 10.,
infogan_cont_depth_to_num_vars = {
2: 16,
3: 16,
4: 16,
5: 16,
6: 16,
7: 0,
8: 0,
},
generator_params=networks.GeneratorParams(
# debug_mode = True,
channels_at_4x4 = 2048,
# channels_max = 448, # Works
# channels_max = 496, # Fails at phase 8
channels_max = 480,
optimizer = ('adam_b0_b99', 0.0005),
ema_decay_for_visualization = .999,
weight_norm = 'equalized',
norm = 'batch_norm_in_place',
norm_per_gpu = True,
double_conv = True,
conditioning = False,
infogan_input_method = 'custom03',
),
discriminator_params=networks.DiscriminatorParams(
# debug_mode = True,
channels_at_2x2 = 4096,
channels_max = 512,
conditioning = False,
optimizer = ('adam_b0_b99', 0.0005),
weight_norm = 'equalized',
norm = None,
norm_per_gpu = True,
double_conv = True,
second_conv_channels_x2 = True,
# elastic_block_input = True,
# infogan_max_pool = True,
),
use_gpu_tower_scope = True,
)
def main(*args):
train.run_training(training_params())
if __name__ == '__main__':
tf.app.run()
================================================
FILE: src/jonasz/experiments/2018_08_28/exp_condition_4vars.py
================================================
DESCRIPTION = __file__ + """
Each block from 2 to 6 has 4 structured variables. The activations within the
block are conditioned with these variables.
"""
import os
import re
import tensorflow as tf
from jonasz.progressive_infogan import train
from jonasz.progressive_infogan import networks
from jonasz.lib import util
from jonasz import constants
from jonasz.gan import evaluation
from jonasz.nvidia_celeb import celeba_hq_dataset
def training_params(is_gcloud=False, output_dir=None):
if not output_dir:
output_dir=util.construct_experiment_output_dir(__file__)
num_gpus = 1
stop_after = 7
dynamic_batch_size = {
2: 128,
3: 128,
4: 64,
5: 32,
6: 16,
7: 6,
8: 3,
}
imgs_per_phase = 384000
dynamic_steps_per_phase = {
phase: max(imgs_per_phase / batch_size, 6000)
for phase, batch_size in dynamic_batch_size.items()
}
dynamic_steps_per_phase[7] *= 2
return train.TrainingParams(
description=DESCRIPTION,
is_gcloud = is_gcloud,
num_gpus = num_gpus,
dataset_params = celeba_hq_dataset.get_dataset_params(
is_gcloud=is_gcloud,
crop_at_center=True),
checkpoint_every_n_steps = None,
checkpoint_every_n_secs = 2*60*60,
dynamic_steps_per_phase = dynamic_steps_per_phase,
dynamic_batch_size = dynamic_batch_size,
stop_after = stop_after,
eval_every_n_secs = 48*60*60,
write_summaries_every_n_steps = 700,
infogan_summary_reps = 0,
output_dir = output_dir,
allow_initial_partial_restore = True,
noise_size = 64,
noise_stddev = 1.,
summary_grid_size = 3,
# allow_simultaneous_steps = False,
infogan_cont_weight = 10.,
infogan_cont_depth_to_num_vars = {
2: 4,
3: 4,
4: 4,
5: 4,
6: 4,
7: 0,
8: 0,
},
generator_params=networks.GeneratorParams(
# debug_mode = True,
channels_at_4x4 = 2048,
# channels_max = 448, # Works
# channels_max = 496, # Fails at phase 8
channels_max = 480,
optimizer = ('adam_b0_b99', 0.0005),
ema_decay_for_visualization = .999,
weight_norm = 'equalized',
norm = 'batch_norm_in_place',
norm_per_gpu = True,
double_conv = True,
conditioning = False,
infogan_input_method = 'custom03',
),
discriminator_params=networks.DiscriminatorParams(
# debug_mode = True,
channels_at_2x2 = 4096,
channels_max = 512,
conditioning = False,
optimizer = ('adam_b0_b99', 0.0005),
weight_norm = 'equalized',
norm = None,
norm_per_gpu = True,
double_conv = True,
second_conv_channels_x2 = True,
# elastic_block_input = True,
# infogan_max_pool = True,
),
use_gpu_tower_scope = True,
)
def main(*args):
train.run_training(training_params())
if __name__ == '__main__':
tf.app.run()
================================================
FILE: src/jonasz/experiments/2018_08_28/exp_condition_8vars.py
================================================
DESCRIPTION = __file__ + """
Each block from 2 to 6 has 8 structured variables. The activations within the
block are conditioned with these variables.
"""
import os
import re
import tensorflow as tf
from jonasz.progressive_infogan import train
from jonasz.progressive_infogan import networks
from jonasz.lib import util
from jonasz import constants
from jonasz.gan import evaluation
from jonasz.nvidia_celeb import celeba_hq_dataset
def training_params(is_gcloud=False, output_dir=None):
if not output_dir:
output_dir=util.construct_experiment_output_dir(__file__)
num_gpus = 1
stop_after = 7
dynamic_batch_size = {
2: 128,
3: 128,
4: 64,
5: 32,
6: 16,
7: 6,
8: 3,
}
imgs_per_phase = 384000
dynamic_steps_per_phase = {
phase: max(imgs_per_phase / batch_size, 6000)
for phase, batch_size in dynamic_batch_size.items()
}
dynamic_steps_per_phase[7] *= 2
return train.TrainingParams(
description=DESCRIPTION,
is_gcloud = is_gcloud,
num_gpus = num_gpus,
dataset_params = celeba_hq_dataset.get_dataset_params(
is_gcloud=is_gcloud,
crop_at_center=True),
checkpoint_every_n_steps = None,
checkpoint_every_n_secs = 2*60*60,
dynamic_steps_per_phase = dynamic_steps_per_phase,
dynamic_batch_size = dynamic_batch_size,
stop_after = stop_after,
eval_every_n_secs = 48*60*60,
write_summaries_every_n_steps = 700,
infogan_summary_reps = 0,
output_dir = output_dir,
allow_initial_partial_restore = True,
noise_size = 64,
noise_stddev = 1.,
summary_grid_size = 3,
# allow_simultaneous_steps = False,
infogan_cont_weight = 10.,
infogan_cont_depth_to_num_vars = {
2: 8,
3: 8,
4: 8,
5: 8,
6: 8,
7: 0,
8: 0,
},
generator_params=networks.GeneratorParams(
# debug_mode = True,
channels_at_4x4 = 2048,
# channels_max = 448, # Works
# channels_max = 496, # Fails at phase 8
channels_max = 480,
optimizer = ('adam_b0_b99', 0.0005),
ema_decay_for_visualization = .999,
weight_norm = 'equalized',
norm = 'batch_norm_in_place',
norm_per_gpu = True,
double_conv = True,
conditioning = False,
infogan_input_method = 'custom03',
),
discriminator_params=networks.DiscriminatorParams(
# debug_mode = True,
channels_at_2x2 = 4096,
channels_max = 512,
conditioning = False,
optimizer = ('adam_b0_b99', 0.0005),
weight_norm = 'equalized',
norm = None,
norm_per_gpu = True,
double_conv = True,
second_conv_channels_x2 = True,
# elastic_block_input = True,
# infogan_max_pool = True,
),
use_gpu_tower_scope = True,
)
def main(*args):
train.run_training(training_params())
if __name__ == '__main__':
tf.app.run()
================================================
FILE: src/jonasz/experiments/2018_08_28/exp_consistency_30.py
================================================
DESCRIPTION = __file__ + """
Same as master_condition, but consistency loss of 30. added.
"""
import os
import re
import tensorflow as tf
from jonasz.progressive_infogan import train
from jonasz.progressive_infogan import networks
from jonasz.lib import util
from jonasz import constants
from jonasz.gan import evaluation
from jonasz.nvidia_celeb import celeba_hq_dataset
def training_params(is_gcloud=False, output_dir=None):
if not output_dir:
output_dir=util.construct_experiment_output_dir(__file__)
num_gpus = 1
stop_after = 7
dynamic_batch_size = {
2: 128,
3: 128,
4: 64,
5: 32,
6: 16,
7: 6,
8: 3,
9: 2,
}
imgs_per_phase = 384000
dynamic_steps_per_phase = {
phase: max(imgs_per_phase / batch_size, 6000)
for phase, batch_size in dynamic_batch_size.items()
}
dynamic_steps_per_phase[7] *= 2
return train.TrainingParams(
description=DESCRIPTION,
is_gcloud = is_gcloud,
num_gpus = num_gpus,
dataset_params = celeba_hq_dataset.get_dataset_params(
is_gcloud=is_gcloud,
crop_at_center=True),
checkpoint_every_n_steps = None,
checkpoint_every_n_secs = 30*60,
dynamic_steps_per_phase = dynamic_steps_per_phase,
dynamic_batch_size = dynamic_batch_size,
stop_after = stop_after,
eval_every_n_secs = 48*60*60,
write_summaries_every_n_steps = 700,
infogan_summary_reps = 0,
output_dir = output_dir,
allow_initial_partial_restore = True,
noise_size = 64,
noise_stddev = 1.,
summary_grid_size = 3,
# allow_simultaneous_steps = False,
infogan_cont_weight = 10.,
infogan_cont_depth_to_num_vars = {
2: 16,
3: 16,
4: 16,
5: 16,
6: 16,
7: 0,
8: 0,
9: 0,
},
generator_params=networks.GeneratorParams(
# debug_mode = True,
channels_at_4x4 = 2048,
# channels_max = 448, # Works
# channels_max = 496, # Fails at phase 8
channels_max = 480,
optimizer = ('adam_b0_b99', 0.0005),
ema_decay_for_visualization = .999,
consistency_loss = 30.,
consistency_temporal = False,
weight_norm = 'equalized',
norm = 'batch_norm_in_place',
norm_per_gpu = True,
double_conv = True,
conditioning = False,
infogan_input_method = 'custom03',
),
discriminator_params=networks.DiscriminatorParams(
# debug_mode = True,
channels_at_2x2 = 4096,
channels_max = 512,
conditioning = False,
optimizer = ('adam_b0_b99', 0.0005),
weight_norm = 'equalized',
norm = None,
norm_per_gpu = True,
double_conv = True,
second_conv_channels_x2 = True,
# elastic_block_input = True,
# infogan_max_pool = True,
),
use_gpu_tower_scope = True,
)
def main(*args):
train.run_training(training_params())
if __name__ == '__main__':
tf.app.run()
================================================
FILE: src/jonasz/experiments/2018_08_28/exp_consistency_300.py
================================================
DESCRIPTION = __file__ + """
Same as master_condition, but consistency loss of 300. added.
"""
import os
import re
import tensorflow as tf
from jonasz.progressive_infogan import train
from jonasz.progressive_infogan import networks
from jonasz.lib import util
from jonasz import constants
from jonasz.gan import evaluation
from jonasz.nvidia_celeb import celeba_hq_dataset
def training_params(is_gcloud=False, output_dir=None):
if not output_dir:
output_dir=util.construct_experiment_output_dir(__file__)
num_gpus = 1
stop_after = 7
dynamic_batch_size = {
2: 128,
3: 128,
4: 64,
5: 32,
6: 16,
7: 6,
8: 3,
9: 2,
}
imgs_per_phase = 384000
dynamic_steps_per_phase = {
phase: max(imgs_per_phase / batch_size, 6000)
for phase, batch_size in dynamic_batch_size.items()
}
dynamic_steps_per_phase[7] *= 2
return train.TrainingParams(
description=DESCRIPTION,
is_gcloud = is_gcloud,
num_gpus = num_gpus,
dataset_params = celeba_hq_dataset.get_dataset_params(
is_gcloud=is_gcloud,
crop_at_center=True),
checkpoint_every_n_steps = None,
checkpoint_every_n_secs = 30*60,
dynamic_steps_per_phase = dynamic_steps_per_phase,
dynamic_batch_size = dynamic_batch_size,
stop_after = stop_after,
eval_every_n_secs = 48*60*60,
write_summaries_every_n_steps = 700,
infogan_summary_reps = 0,
output_dir = output_dir,
allow_initial_partial_restore = True,
noise_size = 64,
noise_stddev = 1.,
summary_grid_size = 3,
# allow_simultaneous_steps = False,
infogan_cont_weight = 10.,
infogan_cont_depth_to_num_vars = {
2: 16,
3: 16,
4: 16,
5: 16,
6: 16,
7: 0,
8: 0,
9: 0,
},
generator_params=networks.GeneratorParams(
# debug_mode = True,
channels_at_4x4 = 2048,
# channels_max = 448, # Works
# channels_max = 496, # Fails at phase 8
channels_max = 480,
optimizer = ('adam_b0_b99', 0.0005),
ema_decay_for_visualization = .999,
consistency_loss = 300.,
consistency_temporal = False,
weight_norm = 'equalized',
norm = 'batch_norm_in_place',
norm_per_gpu = True,
double_conv = True,
conditioning = False,
infogan_input_method = 'custom03',
),
discriminator_params=networks.DiscriminatorParams(
# debug_mode = True,
channels_at_2x2 = 4096,
channels_max = 512,
conditioning = False,
optimizer = ('adam_b0_b99', 0.0005),
weight_norm = 'equalized',
norm = None,
norm_per_gpu = True,
double_conv = True,
second_conv_channels_x2 = True,
# elastic_block_input = True,
# infogan_max_pool = True,
),
use_gpu_tower_scope = True,
)
def main(*args):
train.run_training(training_params())
if __name__ == '__main__':
tf.app.run()
================================================
FILE: src/jonasz/experiments/2018_08_28/exp_gradual_loss.py
================================================
DESCRIPTION = __file__ + """
The structured input is fed to block 2 by appending it to input noise. The
Mutual Information Penalty is gradually activated, for each block of 16
structured variables.
"""
import os
import re
import tensorflow as tf
from jonasz.progressive_infogan import train
from jonasz.progressive_infogan import networks
from jonasz.lib import util
from jonasz import constants
from jonasz.gan import evaluation
from jonasz.nvidia_celeb import celeba_hq_dataset
def training_params(is_gcloud=False, output_dir=None):
if not output_dir:
output_dir=util.construct_experiment_output_dir(__file__)
num_gpus = 1
stop_after = 7
dynamic_batch_size = {
2: 128,
3: 128,
4: 64,
5: 32,
6: 16,
7: 6,
8: 3,
}
imgs_per_phase = 384000
dynamic_steps_per_phase = {
phase: max(imgs_per_phase / batch_size, 6000)
for phase, batch_size in dynamic_batch_size.items()
}
dynamic_steps_per_phase[7] *= 2
return train.TrainingParams(
description=DESCRIPTION,
is_gcloud = is_gcloud,
num_gpus = num_gpus,
dataset_params = celeba_hq_dataset.get_dataset_params(
is_gcloud=is_gcloud,
crop_at_center=True),
checkpoint_every_n_steps = None,
checkpoint_every_n_secs = 2*60*60,
dynamic_steps_per_phase = dynamic_steps_per_phase,
dynamic_batch_size = dynamic_batch_size,
stop_after = stop_after,
eval_every_n_secs = 48*60*60,
write_summaries_every_n_steps = 700,
infogan_summary_reps = 0,
output_dir = output_dir,
allow_initial_partial_restore = True,
noise_size = 64,
noise_stddev = 1.,
summary_grid_size = 3,
# allow_simultaneous_steps = False,
infogan_cont_weight = 10.,
infogan_cont_loss_phase_to_active_coords = {
2: range(0, 16),
3: range(16, 32),
4: range(32, 48),
5: range(48, 64),
6: range(64, 80),
7: [],
8: [],
},
infogan_cont_depth_to_num_vars = {
2: 80,
3: 0,
4: 0,
5: 0,
6: 0,
7: 0,
8: 0,
},
generator_params=networks.GeneratorParams(
# debug_mode = True,
channels_at_4x4 = 2048,
# channels_max = 448, # Works
# channels_max = 496, # Fails at phase 8
channels_max = 480,
optimizer = ('adam_b0_b99', 0.0005),
ema_decay_for_visualization = .999,
weight_norm = 'equalized',
norm = 'batch_norm_in_place',
norm_per_gpu = True,
double_conv = True,
conditioning = False,
infogan_input_method = 'append',
),
discriminator_params=networks.DiscriminatorParams(
# debug_mode = True,
channels_at_2x2 = 4096,
channels_max = 512,
conditioning = False,
optimizer = ('adam_b0_b99', 0.0005),
weight_norm = 'equalized',
norm = None,
norm_per_gpu = True,
double_conv = True,
second_conv_channels_x2 = True,
# elastic_block_input = True,
# infogan_max_pool = True,
),
use_gpu_tower_scope = True,
)
def main(*args):
train.run_training(training_params())
if __name__ == '__main__':
tf.app.run()
================================================
FILE: src/jonasz/experiments/2018_08_28/exp_noinfo.py
================================================
DESCRIPTION = __file__ + """
No InfoGAN penalty at all.
"""
import os
import re
import tensorflow as tf
from jonasz.progressive_infogan import train
from jonasz.progressive_infogan import networks
from jonasz.lib import util
from jonasz import constants
from jonasz.gan import evaluation
from jonasz.nvidia_celeb import celeba_hq_dataset
def training_params(is_gcloud=False, output_dir=None):
if not output_dir:
output_dir=util.construct_experiment_output_dir(__file__)
num_gpus = 1
stop_after = 7
dynamic_batch_size = {
2: 128,
3: 128,
4: 64,
5: 32,
6: 16,
7: 6,
8: 3,
}
imgs_per_phase = 384000
dynamic_steps_per_phase = {
phase: max(imgs_per_phase / batch_size, 6000)
for phase, batch_size in dynamic_batch_size.items()
}
dynamic_steps_per_phase[7] *= 2
return train.TrainingParams(
description=DESCRIPTION,
is_gcloud = is_gcloud,
num_gpus = num_gpus,
dataset_params = celeba_hq_dataset.get_dataset_params(
is_gcloud=is_gcloud,
crop_at_center=True),
checkpoint_every_n_steps = None,
checkpoint_every_n_secs = 2*60*60,
dynamic_steps_per_phase = dynamic_steps_per_phase,
dynamic_batch_size = dynamic_batch_size,
stop_after = stop_after,
eval_every_n_secs = 48*60*60,
write_summaries_every_n_steps = 700,
infogan_summary_reps = 0,
output_dir = output_dir,
allow_initial_partial_restore = True,
noise_size = 64,
noise_stddev = 1.,
summary_grid_size = 3,
# allow_simultaneous_steps = False,
generator_params=networks.GeneratorParams(
# debug_mode = True,
channels_at_4x4 = 2048,
# channels_max = 448, # Works
# channels_max = 496, # Fails at phase 8
channels_max = 480,
optimizer = ('adam_b0_b99', 0.0005),
ema_decay_for_visualization = .999,
weight_norm = 'equalized',
norm = 'batch_norm_in_place',
norm_per_gpu = True,
double_conv = True,
conditioning = False,
),
discriminator_params=networks.DiscriminatorParams(
# debug_mode = True,
channels_at_2x2 = 4096,
channels_max = 512,
conditioning = False,
optimizer = ('adam_b0_b99', 0.0005),
weight_norm = 'equalized',
norm = None,
norm_per_gpu = True,
double_conv = True,
second_conv_channels_x2 = True,
# elastic_block_input = True,
# infogan_max_pool = True,
),
use_gpu_tower_scope = True,
)
def main(*args):
train.run_training(training_params())
if __name__ == '__main__':
tf.app.run()
================================================
FILE: src/jonasz/experiments/2018_08_28/exp_unmask.py
================================================
DESCRIPTION = __file__ + """
The structured input is fed to block 2 by appending it to input noise. The
Mutual Information Penalty is gradually activated, for each block of 16
structured variables. In addition, each block of strucured variables is
masked with zeros up until the phase in which the MIP activates.
"""
import os
import re
import tensorflow as tf
from jonasz.progressive_infogan import train
from jonasz.progressive_infogan import networks
from jonasz.lib import util
from jonasz import constants
from jonasz.gan import evaluation
from jonasz.nvidia_celeb import celeba_hq_dataset
def training_params(is_gcloud=False, output_dir=None):
if not output_dir:
output_dir=util.construct_experiment_output_dir(__file__)
num_gpus = 1
stop_after = 7
dynamic_batch_size = {
2: 128,
3: 128,
4: 64,
5: 32,
6: 16,
7: 6,
8: 3,
}
imgs_per_phase = 384000
dynamic_steps_per_phase = {
phase: max(imgs_per_phase / batch_size, 6000)
for phase, batch_size in dynamic_batch_size.items()
}
dynamic_steps_per_phase[7] *= 2
return train.TrainingParams(
description=DESCRIPTION,
is_gcloud = is_gcloud,
num_gpus = num_gpus,
dataset_params = celeba_hq_dataset.get_dataset_params(
is_gcloud=is_gcloud,
crop_at_center=True),
checkpoint_every_n_steps = None,
checkpoint_every_n_secs = 2*60*60,
dynamic_steps_per_phase = dynamic_steps_per_phase,
dynamic_batch_size = dynamic_batch_size,
stop_after = stop_after,
eval_every_n_secs = 48*60*60,
write_summaries_every_n_steps = 700,
infogan_summary_reps = 0,
output_dir = output_dir,
allow_initial_partial_restore = True,
noise_size = 64,
noise_stddev = 1.,
summary_grid_size = 3,
# allow_simultaneous_steps = False,
infogan_cont_weight = 10.,
infogan_cont_unmask_depth_to_vars = {
2: range(0, 16),
3: range(16, 32),
4: range(32, 48),
5: range(48, 64),
6: range(64, 80),
7: [],
8: [],
},
infogan_cont_loss_phase_to_active_coords = {
2: range(0, 16),
3: range(16, 32),
4: range(32, 48),
5: range(48, 64),
6: range(64, 80),
7: [],
8: [],
},
infogan_cont_depth_to_num_vars = {
2: 80,
3: 0,
4: 0,
5: 0,
6: 0,
7: 0,
8: 0,
},
generator_params=networks.GeneratorParams(
# debug_mode = True,
channels_at_4x4 = 2048,
# channels_max = 448, # Works
# channels_max = 496, # Fails at phase 8
channels_max = 480,
optimizer = ('adam_b0_b99', 0.0005),
ema_decay_for_visualization = .999,
weight_norm = 'equalized',
norm = 'batch_norm_in_place',
norm_per_gpu = True,
double_conv = True,
conditioning = False,
infogan_input_method = 'append',
),
discriminator_params=networks.DiscriminatorParams(
# debug_mode = True,
channels_at_2x2 = 4096,
channels_max = 512,
conditioning = False,
optimizer = ('adam_b0_b99', 0.0005),
weight_norm = 'equalized',
norm = None,
norm_per_gpu = True,
double_conv = True,
second_conv_channels_x2 = True,
# elastic_block_input = True,
# infogan_max_pool = True,
),
use_gpu_tower_scope = True,
)
def main(*args):
train.run_training(training_params())
if __name__ == '__main__':
tf.app.run()
================================================
FILE: src/jonasz/experiments/2018_08_28/exp_vanilla.py
================================================
DESCRIPTION = __file__ + """
The structured input is fed to block 2 by appending it to input noise.
"""
import os
import re
import tensorflow as tf
from jonasz.progressive_infogan import train
from jonasz.progressive_infogan import networks
from jonasz.lib import util
from jonasz import constants
from jonasz.gan import evaluation
from jonasz.nvidia_celeb import celeba_hq_dataset
def training_params(is_gcloud=False, output_dir=None):
if not output_dir:
output_dir=util.construct_experiment_output_dir(__file__)
num_gpus = 1
stop_after = 7
dynamic_batch_size = {
2: 128,
3: 128,
4: 64,
5: 32,
6: 16,
7: 6,
8: 3,
}
imgs_per_phase = 384000
dynamic_steps_per_phase = {
phase: max(imgs_per_phase / batch_size, 6000)
for phase, batch_size in dynamic_batch_size.items()
}
dynamic_steps_per_phase[7] *= 2
return train.TrainingParams(
description=DESCRIPTION,
is_gcloud = is_gcloud,
num_gpus = num_gpus,
dataset_params = celeba_hq_dataset.get_dataset_params(
is_gcloud=is_gcloud,
crop_at_center=True),
checkpoint_every_n_steps = None,
checkpoint_every_n_secs = 2*60*60,
dynamic_steps_per_phase = dynamic_steps_per_phase,
dynamic_batch_size = dynamic_batch_size,
stop_after = stop_after,
eval_every_n_secs = 48*60*60,
write_summaries_every_n_steps = 700,
infogan_summary_reps = 0,
output_dir = output_dir,
allow_initial_partial_restore = True,
noise_size = 64,
noise_stddev = 1.,
summary_grid_size = 3,
# allow_simultaneous_steps = False,
infogan_cont_weight = 10.,
infogan_cont_depth_to_num_vars = {
2: 80,
3: 0,
4: 0,
5: 0,
6: 0,
7: 0,
8: 0,
},
generator_params=networks.GeneratorParams(
# debug_mode = True,
channels_at_4x4 = 2048,
# channels_max = 448, # Works
# channels_max = 496, # Fails at phase 8
channels_max = 480,
optimizer = ('adam_b0_b99', 0.0005),
ema_decay_for_visualization = .999,
weight_norm = 'equalized',
norm = 'batch_norm_in_place',
norm_per_gpu = True,
double_conv = True,
conditioning = False,
infogan_input_method = 'append',
),
discriminator_params=networks.DiscriminatorParams(
# debug_mode = True,
channels_at_2x2 = 4096,
channels_max = 512,
conditioning = False,
optimizer = ('adam_b0_b99', 0.0005),
weight_norm = 'equalized',
norm = None,
norm_per_gpu = True,
double_conv = True,
second_conv_channels_x2 = True,
# elastic_block_input = True,
# infogan_max_pool = True,
),
use_gpu_tower_scope = True,
)
def main(*args):
train.run_training(training_params())
if __name__ == '__main__':
tf.app.run()
================================================
FILE: src/jonasz/experiments/__init__.py
================================================
================================================
FILE: src/jonasz/gan/__init__.py
================================================
================================================
FILE: src/jonasz/gan/evaluation.py
================================================
import tensorflow as tf
import time
import numpy as np
import math
import os
from tensorflow.contrib import gan as tfgan
from tensorflow.python.training import basic_session_run_hooks
from jonasz.progressive_infogan import progressive_infogan_losses
from tensorflow.contrib.gan.python.eval.python import summaries as tfgan_summaries
from tensorflow.python.training import saver
from jonasz.lib import util
from tensorflow.python.training import training_util
class EvalParams(util.Params):
def get_allowed_params_with_defaults(self):
return dict(
eval_dir=None, # By default creates a training_dir/eval subdir.
use_ema_params=True,
inception_num_images = 16000,
frechet_num_images = 5000,
msssim_num_pairs = 2000,
isolation_num_images = None, # NOTE: only works with progressive infogan.
infogan_metrics_num_iters = 3000,
)
def _inception_logits(gan):
# Let's make sure we're not evaluating too many images at a time, so we don't
# run out of memory. Note that during training we might be working with 128
# low res images, but in preprocess_image they are upscaled.
inp = gan.generated_data[:16]
generated_images_preprocessed = tfgan.eval.preprocess_image((inp+1.)*128.)
return tfgan.eval.run_inception(generated_images_preprocessed)
def _inception_final_pool(gan):
# Let's make sure we're not evaluating too many images at a time, so we don't
# run out of memory. Note that during training we might be working with 128
# low res images, but in preprocess_image they are upscaled.
max_imgs = 16
generated_images_preprocessed = tfgan.eval.preprocess_image(
(gan.generated_data[:max_imgs]+1.)*128.)
real_images_preprocessed = tfgan.eval.preprocess_image(
(gan.real_data[:max_imgs]+1.)*128.)
def inception(x):
return tfgan.eval.run_inception(x, output_tensor='pool_3:0')
return (inception(generated_images_preprocessed),
inception(real_images_preprocessed))
def _write_summaries_internal(eval_dir, sess, cur_global_step, summaries=None,
feed_dict=None):
writer = tf.summary.FileWriter(eval_dir)
summaries = summaries or tf.get_collection(tf.GraphKeys.SUMMARIES)
summaries = filter(sess.graph.is_fetchable, summaries)
for x in sess.run(summaries, feed_dict=feed_dict):
writer.add_summary(x, global_step=cur_global_step)
writer.flush()
writer.close()
def _write_summaries(eval_dir, sess, cur_global_step, summaries=None,
feed_dict=None, ema_feed_dict=None):
_write_summaries_internal(eval_dir, sess, cur_global_step,
summaries=summaries, feed_dict=feed_dict)
if ema_feed_dict:
ema_eval_dir = os.path.join(eval_dir, 'ema')
tf.gfile.MakeDirs(ema_eval_dir)
_write_summaries_internal(ema_eval_dir, sess, cur_global_step,
summaries=summaries, feed_dict=ema_feed_dict)
def _accumulate_n(tensor, sess, n):
res = []
i = 0
have = 0
while have < n:
cur = sess.run(tensor)
have += len(cur)
res.append(cur)
if i % 10 == 0:
tf.logging.info('Accumulating tensor values: %d / %d', have, n)
i += 1
return np.concatenate(res)[:n]
# Note: We're evaluating logits_t on sess so that we can use more datapoints
# for the calculation of the inception distance, while still fitting in the
# memory. Perhaps there's a better way of doing this?
# TODO: For cifar, both _calc* functions work on modified (augumented) data.
# That probably shouldn't be the case.
def _maybe_calc_inception_score(gan, sess, params):
if not params.inception_num_images:
return
tf.logging.info('Calculating inception score.')
logits_t = _inception_logits(gan)
logits_t = logits_t[:16]
logits = _accumulate_n(logits_t, sess, params.inception_num_images)
inception_score_t = tfgan.eval.classifier_score_from_logits(
tf.constant(logits))
tf.summary.scalar('inception_score', inception_score_t,
family='quality')
def _maybe_calc_frechet_inception_distance(gan, sess, eval_params):
if not eval_params.frechet_num_images:
return
tf.logging.info('Calculating Frechet inception distance.')
inception_gen_t, inception_real_t = _inception_final_pool(gan)
inception_gen = _accumulate_n(inception_gen_t, sess,
eval_params.frechet_num_images)
inception_real = _accumulate_n(inception_real_t, sess,
eval_params.frechet_num_images)
frechet_dist_t = tfgan.eval.frechet_classifier_distance_from_activations(
tf.constant(inception_real), tf.constant(inception_gen))
tf.summary.scalar('frechet_distance', frechet_dist_t,
family='quality')
def calc_mean(vals):
mean = np.mean(vals)
se = np.std(vals, ddof=1) / len(vals)
return mean, se
def _maybe_calc_infogan_isolation(gan, sess, training_params, version='msssim'):
if not training_params.infogan_cont_num_vars:
return
if not training_params.eval_params.isolation_num_images:
return
assert version in ['msssim', 'mse']
tf.logging.info('Calculating infogan %s isolation' % version)
assert set(gan.generator_inputs.keys()) == {
'noise',
'structured_categorical_input',
'structured_continuous_input',
'one_hot_labels',
}
assert gan.generator_inputs['structured_categorical_input'] is None
noise = gan.generator_inputs['noise']
code = gan.generator_inputs['structured_continuous_input']
# Graph creation is quite costly, so doing it outside of msssim func to make
# it run fast.
_dynamic_scale=2.
_shape=[None, training_params.image_side, training_params.image_side, 3]
_imgs1_placeholder = tf.placeholder(tf.float32, shape=_shape)
_imgs2_placeholder = tf.placeholder(tf.float32, shape=_shape)
_imgs1_t = tf.image.resize_images(
_imgs1_placeholder, size=[256, 256],
method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)
_imgs2_t = tf.image.resize_images(
_imgs2_placeholder, size=[256, 256],
method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)
_sim = tf.image.ssim_multiscale(_imgs1_t, _imgs2_t, _dynamic_scale)
def msssim(a, b):
a = a.copy()
b = b.copy()
def prepare(imgs):
# Assuming dynamic range [-1., 1.]
# assert -1. <= np.min(imgs) <= -0.2, np.min(imgs)
# assert 0.2 <= np.percentile(imgs, 95) <= 1., np.percentile(imgs, 95)
imgs += 1. # Now in [0., 2.]
assert _dynamic_scale == 2.
return imgs
a = prepare(a)
b = prepare(b)
sim = sess.run(_sim, feed_dict={_imgs1_placeholder: a,
_imgs2_placeholder: b})
assert sim.shape == (training_params.batch_size_per_gpu,)
return list(sim)
def mse(a, b):
res = -np.mean((a-b)**2., axis=(1,2,3))
assert res.shape == (training_params.batch_size_per_gpu,)
return list(res)
isolation_per_coord = {}
for coord in range(training_params.infogan_cont_num_vars):
ceil_div = lambda a, b: (a+b-1)/b
reps = ceil_div(training_params.eval_params.isolation_num_images,
training_params.batch_size_per_gpu)
all_isos = []
for _ in range(reps):
fixed_noise, fixed_code = sess.run([noise, code])
def _code_at_coord_val(coord, val):
cur_code = fixed_code.copy()
cur_code[:,coord] = np.array([val] * cur_code.shape[0])
return cur_code
def _imgs_at_coord_val(coord, val):
cur_code = _code_at_coord_val(coord, val)
imgs = sess.run(gan.generated_data, feed_dict={noise: fixed_noise,
code: cur_code})
return imgs
mid_imgs = _imgs_at_coord_val(coord, 0.)
for val in (-3, -2, -1, 1, 2, 3):
cur_imgs = _imgs_at_coord_val(coord, training_params.noise_stddev*val)
if version == 'msssim':
all_isos.extend(list(msssim(cur_imgs, mid_imgs)))
else:
assert version == 'mse'
all_isos.extend(list(mse(cur_imgs, mid_imgs)))
isolation, se = calc_mean(all_isos)
isolation_per_coord[coord] = isolation
tf.logging.info('coord %d, %s isolation %f (se %f)',
coord, version, isolation, se)
tf.summary.scalar('infogan_%s_isolation_coord_%d' % (version, coord),
tf.constant(isolation),
family='infogan_%s_isolation_per_coord' % version)
for block in training_params.block_ids:
coords = block_to_corresponding_cont_coords(training_params, block)
block_isolation = np.mean([isolation_per_coord[coord] for coord in coords])
tf.summary.scalar('infogan_%s_isolation_block_%d' % (version, block),
tf.constant(block_isolation),
family='infogan_%s_isolation' % version)
tf.summary.scalar('infogan_%s_isolation' % version,
tf.constant(np.mean(isolation_per_coord.values())),
family='infogan_%s_isolation' % version)
def block_to_corresponding_cont_coords(tp, block_id):
architecture = tp.infogan_cont_depth_to_vars
unmask = tp.infogan_cont_unmask_depth_to_vars
activate = tp.infogan_cont_loss_phase_to_active_coords
if unmask is None and activate is None:
return architecture[block_id]
# When both are present, they need to agree.
if unmask is not None and activate is not None:
assert unmask[block_id] == activate[block_id]
# When more complex (unmask / activate) settings are present, all vars
# need to be fed to layer 2; or archtecture needs to align with the complex
# settings.
assert (sorted(architecture[2]) == range(tp.infogan_cont_num_vars) or
architecture == (unmask or activate))
return (unmask or activate)[block_id]
def _maybe_calc_infogan_metrics(gan_model_dict, sess, training_params):
tp = training_params
if not tp.eval_params.infogan_metrics_num_iters:
return
if not tp.infogan_cont_num_vars:
return
# tensors
mi_penalty_per_coord_t = {}
abs_error_per_coord_t = {}
# coord -> array of actual values
mi_penalty_per_coord = {}
abs_error_per_coord = {}
for coord in range(tp.infogan_cont_num_vars):
mip, abs_err = (progressive_infogan_losses.
unweighted_mutual_information_penalty_per_coord(
gan_model_dict, tp, coord))
mi_penalty_per_coord_t[coord] = mip
abs_error_per_coord_t[coord] = abs_err
mi_penalty_per_coord[coord] = []
abs_error_per_coord[coord] = []
for i in range(tp.eval_params.infogan_metrics_num_iters):
if i % 10 == 0:
tf.logging.info('_maybe_calc_infogan_metrics, iter %d / %d',
i, tp.eval_params.infogan_metrics_num_iters)
cur_mi_penalty_per_coord, cur_abs_error_per_coord = sess.run([
mi_penalty_per_coord_t, abs_error_per_coord_t])
for coord in range(tp.infogan_cont_num_vars):
mi_penalty_per_coord[coord].append(cur_mi_penalty_per_coord[coord])
abs_error_per_coord[coord].append(cur_abs_error_per_coord[coord])
mean_mi_penalty_per_coord = {}
mean_abs_error_per_coord = {}
for coord in range(tp.infogan_cont_num_vars):
mean, se = calc_mean(mi_penalty_per_coord[coord])
mean_mi_penalty_per_coord[coord] = mean
tf.logging.info('Mutual information penalty, coord %d, val %f, se %f',
coord, mean, se)
tf.summary.scalar('mutual_information_penalty_coord_%d' % coord,
mean, family='mutual_information_penalty_per_coord')
mean, se = calc_mean(abs_error_per_coord[coord])
mean_abs_error_per_coord[coord] = mean
tf.logging.info('InfoGAN absolute prediction error, coord %d, val %f, se %f',
coord, mean, se)
tf.summary.scalar('infogan_absolute_prediction_error_%d' % coord,
mean, family='infogan_absolute_prediction_error_per_coord')
tf.summary.scalar('mutual_information_penalty_overall_mean',
np.mean(mean_mi_penalty_per_coord.values()))
tf.summary.scalar('infogan_absolute_prediction_error_overall_mean',
np.mean(mean_abs_error_per_coord.values()))
for block in tp.block_ids:
coords = block_to_corresponding_cont_coords(tp, block)
tf.summary.scalar('mutual_information_penalty_block_%d_mean' % block,
np.mean([mean_mi_penalty_per_coord[coord]
for coord in coords]),
family='mutual_information_penalty_per_block')
tf.summary.scalar('infogan_absolute_prediction_error_block_%d_mean' % block,
np.mean([mean_abs_error_per_coord[coord]
for coord in coords]),
family='infogan_absolute_prediction_error_per_block')
class _RestoreGANSession(object):
def __init__(self, model_dir = None, vars_to_restore=None,
use_ema_params=None, gan=None):
assert use_ema_params is not None
self.model_dir = model_dir
if use_ema_params:
assert vars_to_restore is None
assert gan is not None
tmp_params_ema = tf.train.ExponentialMovingAverage(decay=0.9)
self.vars_to_restore = tmp_params_ema.variables_to_restore(
moving_avg_variables=gan.generator_variables)
else:
self.vars_to_restore = vars_to_restore
tf.logging.info('_RestoreGANSession variables_to_restore:\n')
for key, val in sorted((self.vars_to_restore or {}).items()):
tf.logging.info('%s: %s', key, val.op.name)
def __enter__(self):
config=tf.ConfigProto(allow_soft_placement=True)
# config.gpu_options.per_process_gpu_memory_fraction = 0.3
self.sess = tf.Session(config=config)
return_val = self.sess.__enter__()
self.sess.run(tf.global_variables_initializer())
self.sess.run(tf.local_variables_initializer())
self.coord = tf.train.Coordinator()
self.threads = tf.train.start_queue_runners(coord=self.coord)
if self.model_dir:
tf.logging.info('Model dir is %s', self.model_dir)
saver_for_restore = saver.Saver(var_list=self.vars_to_restore)
checkpoint_path = saver.latest_checkpoint(self.model_dir)
tf.logging.info('Restoring checkpoints from %s', checkpoint_path)
saver_for_restore.restore(self.sess, checkpoint_path)
return return_val
def __exit__(self, *args):
self.coord.request_stop()
self.coord.join(self.threads)
self.sess.__exit__(*args)
def _get_side_for_dynamic_mse(i, steps, img_side):
phases = int(math.log(img_side, 2))
assert img_side == 2**phases
return 2**(phases * i / steps + 1)
class maybe_gpu_tower_scope(object):
def __init__(self, use_gpu_tower_scope):
if use_gpu_tower_scope:
# Not particularly pretty: make sure global step is not created under
# 'gpu_tower' scope later on.
tf.train.get_or_create_global_step()
self.variable_scope = tf.variable_scope('gpu_tower')
else:
self.variable_scope = None
def __enter__(self):
if self.variable_scope:
return self.variable_scope.__enter__()
def __exit__(self, *args):
if self.variable_scope:
self.variable_scope.__exit__(*args)
def _draw_images(training_params, gan_fn, how_many=32, type_='real',
rescale_list=()):
eval_params = training_params.eval_params
assert type_ in ['real', 'generated']
with tf.Graph().as_default():
with tf.device('/gpu:0'):
with maybe_gpu_tower_scope(training_params.use_gpu_tower_scope):
gan = gan_fn(is_training=False)
imgs = []
with _RestoreGANSession(training_params.output_dir,
use_ema_params=eval_params.use_ema_params,
gan=gan) as sess:
i = 0
while len(imgs) < how_many:
if i % 10 == 0:
tf.logging.info('Draw %s images %d / %d',
type_, len(imgs), how_many)
i += 1
if type_ == 'real':
imgs_t = gan.real_data
else:
assert type_ == 'generated'
imgs_t = gan.generated_data
for rescale in rescale_list:
imgs_t = tf.image.resize_images(
imgs_t, size=[rescale, rescale],
method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)
cur_imgs = sess.run(imgs_t)
imgs.extend(list(cur_imgs))
return np.stack(imgs[:how_many]).astype(np.float32)
def _calc_msssim_diversity_from_images(images):
# Assuming dynamic range [-1., 1.]
# assert -1. <= np.percentile(images, 5) <= -0.2, np.percentile(images, 5)
# assert 0.2 <= np.percentile(images, 95) <= 1., np.percentile(images, 95)
images += 1. # Now in [0., 2.]
dynamic_range = 2.
sims = []
with tf.Session(graph=tf.Graph()) as sess:
x = tf.placeholder(dtype=tf.float32, shape=[1,256,256,3])
y = tf.placeholder(dtype=tf.float32, shape=[1,256,256,3])
sim = tf.image.ssim_multiscale(x, y, dynamic_range)
for a, b in zip(images[::2], images[1::2]):
cur_sim = sess.run(sim, feed_dict={x:a.reshape([1,256,256,3]),
y:b.reshape([1,256,256,3])})
sims.append(cur_sim)
mean, se = calc_mean(sims)
return 1.-mean, se
def _maybe_calc_msssim_diversity(gan_fn, training_params):
params = training_params.eval_params
if not params.msssim_num_pairs:
tf.logging.info('Skipping msssim diversity')
return
tf.logging.info('Calculating msssim diversity')
# First we scale down to current image side, so real data is the same
# resolution. Then we scale to 256, so that msssim works well for small images
# as well.
rescale_list=[training_params.image_side, 256]
real_imgs = _draw_images(training_params, gan_fn,
how_many=2*params.msssim_num_pairs, type_='real',
rescale_list=rescale_list)
msssim_diversity_real, se_real = _calc_msssim_diversity_from_images(real_imgs)
del real_imgs
tf.logging.info('msssim_diversity_real %s (se %f)',
msssim_diversity_real, se_real)
gen_imgs = _draw_images(training_params, gan_fn,
how_many=2*params.msssim_num_pairs,
type_='generated',
rescale_list=rescale_list)
msssim_diversity_gen, se_gen = _calc_msssim_diversity_from_images(gen_imgs)
tf.logging.info('msssim_diversity_gen %s (se %f)',
msssim_diversity_gen, se_gen)
del gen_imgs
# All this just to write summaries.
with tf.Graph().as_default():
tf.summary.scalar('msssim_diversity_real',
tf.constant(msssim_diversity_real),
family='quality')
tf.summary.scalar('msssim_diversity_gen',
tf.constant(msssim_diversity_gen),
family='quality')
training_global_step = training_util.get_or_create_global_step()
# Recovering the global step.
with _RestoreGANSession(training_params.output_dir,
use_ema_params=False) as sess:
cur_global_step = sess.run(training_global_step)
eval_dir = (params.eval_dir
or os.path.join(training_params.output_dir, 'eval'))
_write_summaries(eval_dir, sess, cur_global_step)
def evaluate(gan_fn, training_params):
eval_params = training_params.eval_params
tf.logging.info('Running evaluation, training params %s.',
str(training_params))
if training_params.eval_params is None:
return
eval_dir = (training_params.eval_params.eval_dir
or os.path.join(training_params.output_dir, 'eval'))
with util.TFFileLogger(eval_dir, is_gcloud=training_params.is_gcloud):
tf.logging.set_verbosity(tf.logging.DEBUG)
tf.logging.info('Starting evaluation')
tf.logging.info('Training params: %s', training_params)
_maybe_calc_msssim_diversity(gan_fn, training_params)
with tf.Graph().as_default():
with maybe_gpu_tower_scope(training_params.use_gpu_tower_scope):
gan_model_dict = gan_fn(is_training=False, return_dict=True)
gan = gan_model_dict['gan_model']
# TODO: why generator scope? Probably due to the way I create optimizers
# for training. Fix this.
# with tf.variable_scope('generator'):
global_step = training_util.get_or_create_global_step()
with _RestoreGANSession(training_params.output_dir,
use_ema_params=eval_params.use_ema_params,
gan=gan) as sess:
cur_global_step = sess.run(global_step)
tf.logging.info('Current global_step: %d', cur_global_step)
_maybe_calc_infogan_metrics(gan_model_dict, sess, training_params)
_maybe_calc_infogan_isolation(gan, sess, training_params, 'msssim')
_maybe_calc_infogan_isolation(gan, sess, training_params, 'mse')
_maybe_calc_inception_score(gan, sess, training_params.eval_params)
_maybe_calc_frechet_inception_distance(gan, sess,
training_params.eval_params)
_write_summaries(eval_dir, sess, cur_global_step)
return cur_global_step
================================================
FILE: src/jonasz/lib/__init__.py
================================================
================================================
FILE: src/jonasz/lib/datasets.py
================================================
from jonasz.lib import util
import tensorflow as tf
import numpy as np
from jonasz.nvidia_celeb import celeba_align_dataset
from jonasz.nvidia_celeb import celeba_hq_dataset
from jonasz.cifar10 import cifar10_dataset
class DummyDatasetParams(util.Params):
def get_allowed_params_with_defaults(self):
return dict(
values_range = (-1., 1.),
img_side = 128,
img_channels = 3,
)
def get_dummy_dataset_input_fn(params, batch_size):
def train_input_fn():
low, high = params.values_range
shape = [params.img_channels, params.img_side, params.img_side]
def gen():
return (np.random.uniform(high=high, low=low, size=shape)
for i in xrange(1000))
d = tf.data.Dataset.from_generator(gen, tf.float32)
d = d.repeat()
d = d.batch(batch_size)
d = d.prefetch(buffer_size=8)
iterator = d.make_one_shot_iterator()
imgs = iterator.get_next()
return {'images': imgs}, tf.constant(1, shape=[batch_size, 1])
return train_input_fn
def get_input_fn(dataset_params, batch_size):
if type(dataset_params) is cifar10_dataset.DatasetParams:
return cifar10_dataset.get_train_input_fn(params=dataset_params,
batch_size=batch_size)
elif (type(dataset_params) is
celeba_align_dataset.CelebADatasetParams):
return celeba_align_dataset.get_train_input_fn(params=dataset_params,
batch_size=batch_size)
elif (type(dataset_params) is
celeba_hq_dataset.CelebAHQDatasetParams):
return celeba_hq_dataset.get_train_input_fn(params=dataset_params,
batch_size=batch_size)
elif (type(dataset_params) is DummyDatasetParams):
return get_dummy_dataset_input_fn(params=dataset_params,
batch_size=batch_size)
else:
raise RuntimeError, 'Unknown dataset_params: %s ' % type(dataset_params)
================================================
FILE: src/jonasz/lib/presentation_model_features.py
================================================
"""
Full summary of featuers isolated in a run of exp_consistency_300.
"""
_descriptions = {
79: ('Size of the irises', ', face redness'),
78: ('Eyelids', ', face redness'),
77: ('Nostrils & eyelids', ''),
76: ('Shape of the nostrils, wrinkles', ''),
75: ('Wrinkles', ''),
74: ('Eyelids (subtle)', ', wrinkles'),
73: ('Wrinkles', ', skin redness'),
72: ('Lower eyelid', ''),
71: ('Shape of the eyes', ''),
70: ('Face redness', ''),
69: ('Wrinkles', ''),
68: ('Hair texture', ''),
67: ('Shape of the nostrils', ''),
66: ('Hair texture', ''),
65: ('Size of the irises', ''),
64: ('Color of the irises', ''),
63: ('Skin color, face geometry', ''),
62: ('Look direction', ''),
61: ('Nose tip: left / right, eyebrows', ''),
60: ('Vertical face stretch', ''),
59: ('Nose geometry (subtle)', ''),
58: ('Upper lip + face geometry', ''),
57: ('Look direction', ''),
56: ('Nose: vertical position', ''),
55: ('Eyebrows shape', ''),
54: ('Nose position', ', look direction'),
53: ('Age', ''),
52: ('Nose: upturned tip', ''),
51: ('Cheeckbones and nose, subtle', ''),
50: ('Nose tip shape', ', hair texture'),
49: ('Nose length', ''),
48: ('Eyebrows up / down', ''),
47: ('Face oval: chin', ''),
46: ('Subtle head inclination', ''),
45: ('Hair: curly / straight', ''),
44: ('Mouth open / closed', ', man / woman'),
43: ('Forward / backward inclination', ', hair texture'),
42: ('Face oval, smile, chin size', ''),
41: ('Man / woman', ', slight nod'),
40: ('Smile, chin size', ''),
39: ('Lower jaw width', ''),
38: ('Mouth position: up / down', ''),
37: ('Face oval: width', ''),
36: ('Lower jaw length', ''),
35: ('Head size', ''),
34: ('Jaw size', ''),
33: ('Smile: upper lip', ''),
32: ('Lower jaw size', ''),
31: ('Face oval: width', ''),
30: ('Hairstyle', ''),
29: ('Hairstyle: left side', ''),
28: ('Hairstyle', ''),
27: ('Hairstyle', ''),
26: ('Hairstyle: right side', ''),
25: ('Neck width', ''),
24: ('Hairstyle: background size', ''),
23: ('Hairstyle', ''),
22: ('Hairstyle', ''),
21: ('Skin: pale / red', ''),
20: ('Hairstyle', ''),
19: ('Hairstyle: forhead right size, overall size', ''),
18: ('Hairstyle', ''),
17: ('Hairstyle', ''),
16: ('Face oval: size', ''),
15: ('Rotation + hair color', ''),
14: ('Rotation + hairstyle', ''),
13: ('Skin paleness + hair color', ''),
12: ('Rotation + hairstyle', ''),
11: ('Rotation + hair color', ''),
10: ('Blond / black hair', ', slight rotation'),
9: ('Hairstyle: color, bangs', ''),
8: ('Hair color', ''),
7: ('Hairstyle: color, bangs', ''),
6: ('Lighting, hair color', ''),
5: ('Hairstyle', ''),
4: ('Hair length & color', ''),
3: ('Left / right rotation', ''),
2: ('Hairstyle, light', ''),
1: ('Light bright / dark', ''),
0: ('Head rotation + hairstyle', ''),
}
descriptions = {
key: desc1 + '.' for key, (desc1, desc2) in _descriptions.items()
}
descriptions_full = {
key: desc1 + desc2 + '.' for key, (desc1, desc2) in _descriptions.items()
}
================================================
FILE: src/jonasz/lib/progressive_infogan_ipynb_utils.py
================================================
import tensorflow as tf
import numpy as np
import os
import scipy
import random
from jonasz.lib import util
from jonasz.cifar10 import cifar10_dataset
from jonasz import constants
from matplotlib import pyplot as plt
from jonasz.progressive_infogan import progressive_infogan_lib
import ipywidgets
import collections
import copy
import matplotlib.pyplot as plt
import numpy as np
from IPython import display
import time
# To be filled in by the user.
# See src/notebooks/jonasz/notebooks/generator.ipynb
MODELS = { }
TF_MODEL_SERVER_SUBPROCESS = None
HOST_PORT = ('localhost', 9141)
def restart_server():
global TF_MODEL_SERVER_SUBPROCESS, HOST_PORT
if TF_MODEL_SERVER_SUBPROCESS is not None:
print 'killing previous subprocess, pid', TF_MODEL_SERVER_SUBPROCESS.pid
TF_MODEL_SERVER_SUBPROCESS.kill()
# HOST_PORT = ('localhost', random.randint(8000, 10000))
models_to_versions = {
model: options.get('versions', []) for model, options in MODELS.items()
}
print 'HOST_PORT', HOST_PORT
for key, val in models_to_versions.items(): print key, val
TF_MODEL_SERVER_SUBPROCESS = util.start_tensorflow_model_server(
port=HOST_PORT[1],
models=models_to_versions,
)
print 'tensorflow model server running, pid', TF_MODEL_SERVER_SUBPROCESS.pid
def get_images(model, gen_inputs):
batch_size = MODELS[model]['batch_size']
imgs = progressive_infogan_lib.gen_inputs_to_images(gen_inputs,
model_id=model,
batch_size=batch_size,
host_port=HOST_PORT,
noise_stddev=1.)
return imgs
def manual_animation(model, gen_inp, coords=range(64,80)):
_input = copy.deepcopy(gen_inp)
fig = plt.figure(figsize=(9, 9))
ax = fig.add_subplot(111)
imshow_obj = [None]
# last_update = time.time()
def f(**kwargs):
coords = sorted(kwargs.items())
c = [val for key, val in coords]
print c
inp = copy.deepcopy(_input)
for key, val in coords:
coord = int(key[1:]) # key is 'c29'
inp.cont_structured_input[coord] = val
img = memoized_get_img(model, inp)
img = (img + 1.) / 2
# img = np.clip(img, -1., 1.)
# global imshow_obj, last_update
# elapsed = time.time() - last_update
if imshow_obj[0] is None:
imshow_obj[0] = ax.imshow(img) # ,minv=-1., maxv=1., interpolation='none')
plt.show()
# elif elapsed > 0.1:
else:
# last_update = time.time()
imshow_obj[0].set_data(img)
plt.show()
# for i in range(num_coords): print _noise[i]
_kwargs = collections.OrderedDict([
('c%02d' % i,
ipywidgets.FloatSlider(value=_input.cont_structured_input[i],
min=-2.6, max=2.6, step=0.2)) #, continuous_update=False))
for i in coords
])
interactive_plot = ipywidgets.interactive(f, **_kwargs)
output = interactive_plot.children[-1]
# output.layout.height = '250px'
return interactive_plot
def interpolate(model, inp, coord, steps=3):
inps = []
for val in np.linspace(-3., 3., steps):
cur_inp = inp.copy()
cur_inp.cont_structured_input[coord] = val
inps.append(cur_inp)
imgs = get_images(model, inps)
return imgs
def blur(a, stddev):
a = a.copy()
for channel in range(3):
a[:,:,channel] = scipy.ndimage.filters.gaussian_filter(
a[:,:,channel], stddev)
return a
def diff2(a, b):
a = a.copy()
b = b.copy()
res = b-a
res = np.abs(res)
res = blur(res, 1.)
return res
def avg_change(model, coord,
percentile_thresholds=[99.5, 99.,98., 95.,90., 80.],
num_samples=30,
inputs_for_interpolation=[], interpolation=None):
gen_inputs = progressive_infogan_lib.random_request(
num_images=num_samples, noise_stddev=1.)
left, mid, right = [], [], []
for gi in gen_inputs:
gi_left, gi_right = gi.copy(), gi.copy()
gi_left.cont_structured_input[coord] = -2.5
gi_right.cont_structured_input[coord] = 2.5
mid.append(gi)
left.append(gi_left)
right.append(gi_right)
imgs_left = get_images(model, left)
imgs_mid = get_images(model, mid)
imgs_right = get_images(model, right)
diffs = [diff2(a, b) for (a, b) in zip(imgs_left, imgs_right)]
bigleft = sum(imgs_left) / len(imgs_left)
bigright = sum(imgs_right) / len(imgs_right)
bigmid = sum(imgs_mid) / len(imgs_mid)
bigdiff_rgb = sum(diffs) / len(diffs)
bigdiff_norm = np.linalg.norm(bigdiff_rgb, axis=2, keepdims=True)
bigchange_imgs = []
for percentile_threshold in percentile_thresholds:
thresh = np.percentile(bigdiff_norm, percentile_threshold)
bigchange = np.where(bigdiff_norm > thresh, bigright, np.ones_like(bigright)-.2)
bigchange_imgs.append(bigchange)
util.show_imgs(bigchange_imgs, cols=len(bigchange_imgs), minv=-1., maxv=1.,
interpolation=interpolation)
util.show_imgs([bigleft,
bigright,
bigmid],
cols=4, minv=-1., maxv=1, interpolation=interpolation)
util.show_imgs(imgs_left[:5], minv=-1., maxv=1., cols=5,
interpolation=interpolation)
util.show_imgs(imgs_right[:5], minv=-1., maxv=1., cols=5,
interpolation=interpolation)
for inp in inputs_for_interpolation:
util.show_imgs(interpolate(model, inp, coord, steps=5),
minv=-1., maxv=1., cols=5,
interpolation=interpolation)
================================================
FILE: src/jonasz/lib/tensor_util.py
================================================
import tensorflow as tf
def nchw_to_nhwc(images):
return tf.transpose(images, [0, 2, 3, 1])
def nhwc_to_nchw(images):
return tf.transpose(images, [0, 3, 1, 2])
def nchw_to_nhwc_single(images):
return tf.transpose(images, [1, 2, 0])
def nhwc_to_nchw_single(images):
return tf.transpose(images, [2, 0, 1])
================================================
FILE: src/jonasz/lib/util.py
================================================
"""Miscellaneous utilities."""
import tempfile
import traceback
import logging
import subprocess
import signal
import collections
import datetime
import numpy as np
import os
import re
import random
import time
import copy
import grpc
from grpc.beta import implementations as grpc_beta_implementations
from grpc._cython import cygrpc
import tensorflow as tf
from jonasz import constants
def CHW_to_HWC(img):
"""Converts a np arrray from shape [c, h, w] to [h, w, c]."""
c, h, w = img.shape
new_img = img.reshape([c, h*w]).T
new_img = new_img.reshape([h, w, c])
return new_img
def imshow(img, minv=None, maxv=None, turn_off_axis=True):
from matplotlib import pyplot as plt # Import here so we don't have to list
# this as a requirement in setup.py.
assert minv is not None # Otherwise we distort the images.
assert maxv is not None
minv = minv or min(img.flatten())
maxv = maxv or max(img.flatten())
img = (img - minv) / (maxv - minv)
plt.tight_layout(pad=0., h_pad=0., w_pad=0.)
fig = plt.imshow(img, vmin=0., vmax=1., interpolation='none')
if turn_off_axis:
# This is useful for the purpose of creating animations, but somehow causes
# problems when working in a notebook. (Some imgs are not displayed.)
fig.axes.get_xaxis().set_visible(False)
fig.axes.get_yaxis().set_visible(False)
plt.axis('off')
return fig
def show_imgs(imgs, data_type='HWC', cols=5, minv=None, maxv=None,
size=15):
from matplotlib import pyplot as plt # Import here so we don't have to list
# this as a requirement in setup.py.
assert data_type in ('HWC', 'CHW')
if not isinstance(imgs, (list, tuple)):
imgs = [imgs]
n = len(imgs)
rows = (n + cols-1) / cols
siz = float(size) / cols
plt.figure(figsize=(siz*cols, siz*rows))
for i, img in enumerate(imgs):
size = 1
if data_type == 'CHW':
img = CHW_to_HWC(img)
plt.subplot(rows, cols, i+1).axis('off')
imshow(img, minv=minv, maxv=maxv, turn_off_axis=False)
plt.show()
def random_crop(img, max_shift=7, can_flip=False, data_type='CHW'):
assert data_type == 'CHW'
c, h, w = img.shape
shift_h = random.randint(-max_shift, max_shift)
shift_w = random.randint(-max_shift, max_shift)
cropped = np.zeros_like(img)
i_a_0 = max(0, shift_h)
i_x_0 = max(0, -shift_h)
i_a_1 = min(h, h+shift_h)
i_x_1 = min(h, h-shift_h)
j_a_0 = max(0, shift_w)
j_x_0 = max(0, -shift_w)
j_a_1 = min(w, w+shift_w)
j_x_1 = min(w, w-shift_w)
for channel in range(c):
cropped[channel, i_a_0:i_a_1, j_a_0:j_a_1] = (
img[channel, i_x_0:i_x_1, j_x_0:j_x_1])
if can_flip and random.randint(0, 1)==1:
cropped = np.flip(cropped, 2)
return cropped
class Params(object):
def __init__(self, **kwargs):
params = self.get_allowed_params_with_defaults()
assert_set_covered(kwargs, params)
params.update(kwargs)
self.params = params
self.validate()
def validate(self):
pass
def overwrite(self, **kwargs):
self.params.update(kwargs)
def __getattr__(self, name):
return self.params[name]
def default_param(self, key, val):
self.params[key] = self.params.get(key, val)
def __getstate__(self):
return self.params
def __setstate__(self, vals):
self.params = vals.copy()
def __repr__(self):
res = self.__class__.__name__ + '(\n'
for key, val in sorted(self.params.items()):
cur = '%s=%r,' % (key, val)
cur = (' ' + t for t in cur.split('\n'))
cur = list(filter(None, cur))
cur = '\n'.join(cur) + '\n'
res += cur
res += ')'
return res
def get_allowed_params_with_defaults(self):
return {'x': 2, 'y': 3}
raise NotImplementedError
def __deepcopy__(self, memo):
return self.__class__(**copy.deepcopy(self.params))
TIME_START = None
def reset_timer():
global TIME_START
TIME_START = time.time()
def time_elapsed(log=True):
elapsed = int(time.time() - TIME_START)
hours = elapsed / 60 / 60
minutes = (elapsed / 60) % 60
seconds = elapsed % 60
if log:
print ('time elapsed: %d hours %d minutes %d seconds' % (
hours, minutes, seconds))
return elapsed
def assert_set_covered(covered, covered_by):
unexpected = set(covered) - set(covered_by)
assert not unexpected, unexpected
def get_time_str(microseconds=False):
fmt = '%y_%m_%d___%H_%M_%S'
if microseconds: fmt += '___%f'
return datetime.datetime.now().strftime(fmt)
def get_new_dir(base_path, suffix):
return os.path.join(base_path, get_time_str() + '___' + suffix)
def serialized_example(float_features, int_features):
example = tf.train.Example()
for key, val in float_features.iteritems():
example.features.feature[key].float_list.value.extend(
np.array(val).reshape(-1))
for key, val in int_features.iteritems():
example.features.feature[key].int64_list.value.extend(
np.array(val).reshape(-1))
return example.SerializeToString()
def _chunks(iterable, chunk_size):
first_unused = 0
while first_unused < len(iterable):
yield iterable[first_unused:first_unused+chunk_size]
first_unused += chunk_size
class _DummyCtxMgr():
def __init__(self, *args):
self.args = args
def __enter__(self):
return self.args
def __exit__(self, exc_type, exc_value, traceback):
pass
def tensorflow_model_server_predict(host_port=None,
model_id=None,
signature_name=None,
serialized_examples=None,
serialized_examples_tensor_name=None,
version=None,
batch_size=256,
retries=10):
"""Queries a tensorflow_model_server, potentially spawning it first.
Args:
host_port: tensorflow_model_server address. If None, will spawn a new
tms process, and kill it afterwards.
model_id: If host_port is given - the name of the model to query. Otherwise
a path to the model saved_model dir.
signature_name: model's signature to query against.
serialized_exapmles: list of serialized tf.train.Example.
serialized_examples_tensor_name: when querying the model, supply the
serialized tf exmaples into a tensor with this name.
version: saved_model version or None. None means: query the newest
available version.
batch_size: if number of serialized examples exceeds batch_size, the model
will be queried multiple times, each time with at most batch_size
examples.
retries: it takes some time for the tms to spawn and load the model. We
do a couple retries, each time waiting for 2 secs.
"""
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2
if host_port is None:
versions = [] if version is None else [version]
models = {model_id: versions}
mgr = TFModelServer(models=models)
else:
mgr = _DummyCtxMgr(None, host_port)
trial_num = 0
with mgr as (popen, (host, port)):
while trial_num < retries:
if trial_num >= 2:
print '(Re)trying (%d) to query tensorflow_model_server.' % trial_num
if popen is not None and popen.poll() is not None:
raise RuntimeError(
'tensorflow_model_server exited with code %d' % popen.returncode)
trial_num += 1
try:
channel = grpc_beta_implementations.Channel(grpc.insecure_channel(
target='%s:%s' % (host, port),
options=[(cygrpc.ChannelArgKey.max_send_message_length, -1),
(cygrpc.ChannelArgKey.max_receive_message_length, -1)]))
stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)
results = collections.defaultdict(list)
for i, chunk in enumerate(_chunks(serialized_examples, batch_size)):
if i > 9 and i % 10 == 0:
print 'Prediction for batch %d / %d' % (
i, (len(serialized_examples) + batch_size - 1) / batch_size)
request = predict_pb2.PredictRequest()
request.model_spec.name = model_id
if version is not None:
request.model_spec.version.value = version
request.model_spec.signature_name = signature_name
request.inputs[serialized_examples_tensor_name].CopyFrom(
tf.contrib.util.make_tensor_proto(chunk))
result_future = stub.Predict.future(request, 15.0)
while result_future.done() is False:
time.sleep(0.01)
result = result_future.result()
for key, tensor in result.outputs.iteritems():
results[str(key)] = np.concatenate([
np.array(results[str(key)]),
np.array(tensor.float_val),
np.array(tensor.double_val),
np.array(tensor.int_val),
np.array(tensor.string_val),
np.array(tensor.int64_val),
np.array(tensor.string_val),
])
return results
except Exception, e:
if trial_num > 3:
print 'Failed to query tensorflow_model_server:', e
time.sleep(2.)
raise Exception('Failed to query tensorflow_model_server. Double check ' +
'the model you passed in exists.')
def _create_model_config_file(models):
"""
Args:
models: dict from model path to list of versions (potentially empty)
"""
config_file = tempfile.NamedTemporaryFile(delete=False)
config_file.write('model_config_list: {\n')
for path, versions in models.items():
config_file.write("""
config: {
name: '%s',
base_path: '%s',
model_platform: 'tensorflow'
""" % (path, path))
if versions:
config_file.write("""
model_version_policy: {
specific: {
""")
for version in versions:
config_file.write("versions: %d\n" % version)
config_file.write("}}\n")
config_file.write("},")
config_file.write('}\n')
config_file.close()
print config_file.name
return config_file.name
def start_tensorflow_model_server(port=8100, models=None, interruptible=False):
"""
Args:
models: dict from model path to list of versions (potentially empty)
"""
old_cuda_visible_devices = os.environ.get('CUDA_VISIBLE_DEVICES') or None
if old_cuda_visible_devices:
del os.environ['CUDA_VISIBLE_DEVICES']
print 'starting tensorflow_model_server'
model_config_file = _create_model_config_file(models)
if not interruptible:
preexec_fn = lambda: signal.signal(signal.SIGINT, signal.SIG_IGN)
else:
preexec_fn = None
p = subprocess.Popen([
'tensorflow_model_server',
'--port=' + str(port),
'--model_config_file=' + model_config_file,
'--per_process_gpu_memory_fraction=0.7'],
# Make sure interrupts in notebooks don't kill the server
preexec_fn=preexec_fn,
)
if old_cuda_visible_devices:
os.environ['CUDA_VISIBLE_DEVICES'] = old_cuda_visible_devices
return p
class TFModelServer(object):
def __init__(self, port=8100, models=None, initial_sleep=None,
interruptible=False):
"""
Args:
models: dict from model path to list of versions (potentially empty)
"""
self.port = port
self.models = models
self.subprocess = None
self.initial_sleep = initial_sleep
self.interruptible = interruptible
def __enter__(self):
self.subprocess = start_tensorflow_model_server(
port=self.port, models=self.models, interruptible=self.interruptible)
if self.initial_sleep:
print 'Waiting for tensorflow_model_server to start'
time.sleep(self.initial_sleep)
return self.subprocess, ('localhost', self.port)
def __exit__(self, exc_type, exc_value, traceback):
self.subprocess.kill()
class _TFLoggingFilter(logging.Filter):
forbidden_list = [
'Summary name .* is illegal; using .* instead.',
]
def filter(self, record):
for forbidden in self.forbidden_list:
if re.match(forbidden, record.msg):
return False
return True
class TFFileLogger(object):
""" Context manager for saving tensorflow logs into file.
Usage:
with TFFileLogger(training_directory):
run_training(...)
"""
def __init__(self, dir_path, is_gcloud):
self.dir_path = dir_path
self.logging_handler = None
self.is_gcloud = is_gcloud
def __enter__(self):
if not tf.gfile.Exists(self.dir_path):
tf.gfile.MakeDirs(self.dir_path)
self.tensorflow_logger = logging.getLogger('tensorflow')
self.tensorflow_logger.addFilter(_TFLoggingFilter())
if not self.is_gcloud:
self.logging_handler = logging.FileHandler(
os.path.join(self.dir_path, 'tensorflow.log'))
self.logging_handler.setLevel(logging.DEBUG)
self.tensorflow_logger.addHandler(self.logging_handler)
formatter = logging.Formatter('@@@ [%(asctime)s %(name)s.%(levelname)s] %(message)s')
for handler in self.tensorflow_logger.handlers:
handler.setFormatter(formatter)
def __exit__(self, *args):
tf.logging.error('Exception reported by TFFileLogger:')
tf.logging.error(traceback.format_exc())
if self.logging_handler:
self.tensorflow_logger.removeHandler(self.logging_handler)
def tf_logging_decorator(f):
def new_f(training_params, *args, **kwargs):
with TFFileLogger(training_params.output_dir,
is_gcloud=training_params.is_gcloud):
tf.logging.set_verbosity(tf.logging.DEBUG)
tf.logging.info('logging_decorator calling %s', f.__name__)
tf.logging.info('Training params:\n' + str(training_params))
f(training_params, *args, **kwargs)
return new_f
def get_optimizer(params, scope, global_step=None):
"""Creates and returns a tf.train.Optimizer.
Args:
params.optimzier: (str, float)
params.learning_rate_decay_rate: float or None
params.learning_rate_decay_steps: int or None
scope: str
"""
if global_step is None:
global_step = tf.train.get_or_create_global_step()
with tf.variable_scope(scope):
optimizer, initial_learning_rate = params.optimizer
if params.learning_rate_decay_steps is None:
assert params.learning_rate_decay_rate is None
lr = initial_learning_rate
else:
assert params.learning_rate_decay_rate is not None
lr = tf.train.exponential_decay(
learning_rate=initial_learning_rate,
global_step=global_step,
decay_steps=params.learning_rate_decay_steps,
decay_rate=params.learning_rate_decay_rate,
staircase=True)
tf.summary.scalar('learning_rate', lr)
if optimizer == 'adam':
return tf.train.AdamOptimizer(learning_rate=lr)
elif optimizer=='adam_b0_b99':
return tf.train.AdamOptimizer(learning_rate=lr, beta1=0., beta2=0.99,
epsilon=1e-8)
elif optimizer=='gdo':
return tf.train.GradientDescentOptimizer(learning_rate=lr)
elif optimizer=='rmsprop':
return tf.train.RMSPropOptimizer(lr, decay=.9, momentum=0.1)
else:
assert False, 'Unknown opimizer: ' + optimizer
def construct_experiment_output_dir(fname):
"""
Translates experiment's filename to output directory path, like so:
From "jonasz/experiments/2018_06_17/post_training_infogan_01.py"
To ("/home/jonasz/warcaby/jonasz/2018_06_17/"
"post_training_infogan_01___18_06_18___17_37_11")
"""
subdir = ''
for i in [-2, -3]: # The date dir can have a subdir with experiments.
date = fname.split('/')[i]
if re.match('^\d\d\d\d_\d\d_\d\d$', date):
break
else:
subdir = date
assert re.match('^\d\d\d\d_\d\d_\d\d$', date), date
experiment = fname.rpartition('/')[-1][:-3]
assert re.match('^[a-z0-9_]*$', experiment), experiment
return os.path.join(constants.TRAINING_OUTPUT_BASE_DIR, date, subdir,
experiment + '___' + get_time_str()
)
================================================
FILE: src/jonasz/notebooks/generator.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Setup"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
"# Path to progressive_infogan src tree\n",
"SRC_PATH = ''\n",
"\n",
"# Path to saved_model_ema subdir inside of the training dir\n",
"MODEL = ''\n",
"BATCH_SIZE = 6 # Batch size the model uses."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"import sys\n",
"if SRC_PATH not in sys.path:\n",
" sys.path.append(SRC_PATH)"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"import numpy as np\n",
"import os\n",
"import random\n",
"import time\n",
"from matplotlib import pyplot as plt\n",
"from jonasz.lib import util\n",
"from jonasz.cifar10 import cifar10_dataset\n",
"from jonasz import constants\n",
"from jonasz.progressive_infogan import progressive_infogan_lib\n",
"from jonasz.lib import progressive_infogan_ipynb_utils\n",
"pass"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"reload(progressive_infogan_ipynb_utils)\n",
"progressive_infogan_ipynb_utils.MODELS[MODEL] = {'batch_size': BATCH_SIZE}\n",
"progressive_infogan_ipynb_utils.restart_server()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# A couple samples drawn from generator"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0 20\n",
"1 10\n",
"2 1442\n",
"3 7819\n",
"4 9475\n",
"5 8807\n",
"6 8576\n",
"7 836\n",
"8 14555\n",
"9 7477\n",
"10 9628\n",
"11 26847\n",
"12 16114\n",
"13 18187\n",
"14 3902\n",
"15 16634\n",
"16 25568\n",
"17 29146\n",
"18 3155\n",
"19 21061\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAABEIAAAVGCAYAAACaJB7bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzsvdmPbFmW5vXb0xltcvPhzhEZkUNXVVY3UqNWQwMv/dISfx1C4hWJB8RTq8UfgAA1QkJNIaqqKXKMuDfu5Ndnt+FMe+JhHzP3G5XQWVBkppT2KW64u9mxY8f22Xvttb71rWUixsgBBxxwwAEHHHDAAQcccMABBxxwwB8D5O/7Ag444IADDjjggAMOOOCAAw444IADflc4ECEHHHDAAQcccMABBxxwwAEHHHDAHw0ORMgBBxxwwAEHHHDAAQcccMABBxzwR4MDEXLAAQcccMABBxxwwAEHHHDAAQf80eBAhBxwwAEHHHDAAQcccMABBxxwwAF/NDgQIQcccMABBxxwwAEHHHDAAQcccMAfDQ5EyAEHHHDAAQcccMABBxxwwAEHHPBHgwMRcsABBxxwwAEHHHDAAQcccMABB/zR4ECEHHDAAQcccMABBxxwwAEHHHDAAX800L/vCwCQUsYIkP7/ADH+jIAQ4y/jE98/9rfF7jzjOaVWSCnxzkFI58zKglcvXlLXU96/+YbNZo3JMhbTiq9enmGi4vWHD+iq4OnTF/z0z39CESSTMuPdxw9cX1zz0z//EX3bo6QAEenbgbKsmM+mrNYNm65lPq2x/YAdBjId6LqWaANfvHzKu4+fuL9bc3JyRFHXOOups4qNtWy7lucvnuL6Ht90ZGXNdui437Q8OT0mLw3WOaSNSASbfsB5hyozpmWF96BCwCnFh4tLTiYlEsGb797y5MkTVn7g45u3lIDMNOttSwD61nJ/u2K+POLD+Qc2bc/saMG8mnE8n9MOlqLKWB7V/ODFK3RVUlcVhZIYo+mHjiwG1rc39JsNHz5e8Iv3F2wGy2AjTdtxfHLMpmvYNg2DdwgXOF5OGHoHMVCWBUWm8Z3n5OwMz0Df9Dw9PaHtB+7v18yOpkTh0Shms4rz6zvs0NGttqzWDSqTrLcDX35xysnzU7bn19ze3PH8xVOWiykqBKrKUBUZ0zrn8uMV1nucB+c9i2lFPS3Ji5xu21FPa7Z3G7qhR9clf/kXP+PLr56ynB/x9RdP2G5bLi6uOXn+hMurO0yRc35xQ9v2LJYz7m7X/Pzn3+EjeCHorefbj9dUkwItJJ+uVlgfEEqQZ4q2dzgf8SEtAyk/Xw5CpMdCACnSVFdSEMf5/f2VI4QgPjpBjOkc8bO/xcPxuyU5vldvg+B3jH/xn/z70fuI9R6kIiLHJS2REiAihUSMY7O7QDEeI3afTuw+T0RKiRj/kkKCEAgiWqtkK2JgPqkZBsvl9RVSGaaTCZtuwLlAVhR4H4mA954QPBKICEQU9D4AghACmTZ472m7BhCsNit8jDjbk2mNFNB3PSbLEELQbLZIpdCZQSvF0Fu0UfT9wPHRAqMNLniCd1RlCULSbFZIpfEhoJQihIDRGq01fT/ggmWz2VBVFcMwkBmDF4HoI4MPOCHAOZz1xBCJzuJ8gHG+hNGM7oYyhJBGT+zug0RIAVGgRBpZISVCSIxRxJDGQ4g0j7wP6XgE3luEkCDAKE2Z5RRFjvcOoQz1ZErftMwmE6Q2DNbS2Z7Veo0EnHfkeYazFucdZZ4TfMAYg1IKEEghCD4SBXjnkFqR54bbu1sQgvVqjbWWwafxS58pXWwMHoFAK4UApErzJYSA0JK+7Qkh4H1Aa433AWM0KjN4O5AZzTAMFMYghKTrOo6P52ipmFQFIGi7gbzIublfYW3AO4dQCiEgeE/wIQ29EEghCd6TFxnGZKzuVpSTkrZpycsSpCA4T9c05HnGZDalWW8ZnGM6nSAjSKlAQAhpwVRVTdcPOO+JUkGMyYYIgVLJwETSeOzsQ4yBGNJ6QoCI8cF2yLQabBx488vvfuc241/+1/9NlGJgcA6iR0pFBOLjaxRyb9zS6v98kj/yQMZfHnyRBxsa0+9CECMoEQnej+OU5rMbOu6vPtJ/+sB/+9/9ay5u78iXC7598w4RBdNpxe39mkpphhBASHQmOZnN+eL5Kd++veBoNkNUBZcfL3hyekLney4+XWGkZrqYcX5xQdP01JOSWVmSa810UtG5gPSBp6cnDKHnzZsPLI7mZJnEt5bMFHQ47jYd9aTiZDFje7NhupjR2par2zUvnpyRzaa4wTIxOceLGRC5vrlCaMHFx2uKomA5m3DfDVzf3uIHizaa6+srzp69IEhBs94wen9orZkvj7Ae7i+u+OpHX3D+8QLrBnReEoLlZLmgWd9jB8uTp0vOP1yyWm/IypKP7z7S+4CSaS77GJlWGQyRu7ZDG42PgldPTggCrq7vKCcTTG5oblcsZlM8ns56Zos5P/rxD1ldrpgsp1x8PKdttqzXDUEI7DCgyozV3R2Z0oQQ6PsBoRW2c0wmE4qyZDatKcuS0Hc4G3E48syQS43JC0RZcH93x9nZMW/ffcCu17S9w+SKrh04Ws4RuaHdbIjBsV23iBCJQqCNYbaY0DY9dV3SNC2bpkNmmtXNCmMMXghECBRljowQnKWoCmTwfPHFKT/9yZf8k3/vp7RDx8d3l0xPj/C9H8fP453FDj1CRKJ3eGvHuS2JCKTUac2MKyKOjkPcrZJxXYlxn5VERAyjnfDEEAjeE4PfOxYhBEIE5wNRaJQyXFvFf/lf/avfuc34T//FP4+DgxDT+o2j5/Cw1h9s32MIeHCkePCV2L12/5q4990e/DCxD1PEbuCAGAKRnQ1O54kxQowEn8ZRCIEPARcCzrvkz8T0HlLI5OcoSfCOONok7xxCKkyW4eyAdxalFd5ZnLN4awkhACC1TvFKpnECwmj77dChvEIE8GHAu+R7SmHwfiCbVgythaHHBpBK0XUddnAgJCF4fIzs4sDHPmqMMX2OR6Gbkul9tdJIrfDBEqMgxDiO23ic0ggZ0eN5TGZAwHQ+ZbttCIMlCpBSoE3yq4xWaTwRSNJ81FqT5Rk+BIauByHIqhw/Ht8P9pH/CEIm/8IUOX3T4b1PvofWhBjJM83qbk1RFgzWMvR29JvSdWeZASmJzsG4N4cQ0UYTgqeoKiaTCdv1BtsPtG2PGde4MYrJ8ojgfJqtIVLmJVmWMz86pSxyhBAUeU4InuA9KsvIigIpk9/G+Dql5MP8laMfJ3Z/y9FHEwg5+nePfAEh5H7X3L1+51+D2Pt6Uso0t0NI7y0ejovjvA+7uc7u0mLyH/fvJR6tn0B0jkEZ/ov/7D//rWzGHwQRku5+eGQcxofHn3FnCx5FZ3E/nr8lKbI7bm+EIlIKxDgRxKNDhHPcfPqIeRI5ffGK/ptfkEnBxGh0AKkFr169oMyy5ED0KcB/e3HD7d2ar754jrID3WZLUWSUeUZP5OxkgXeR+80KImw3LVWVs1nfYztHqQVkkTffvaNtegZnwQfqTNNJwTAMuGFAR0+72XJ2Oico2HYW1/cspiW2a/A9nJwtCcExnS5YGs2v/+pvMFrR2HsW8wWmqtg2a744K8mtpP7iB9y1HV3bUivBn/zgFdZ6ri5vmE5qwrZjeZxD8Lx5/5HoeyIR0fVkpcX1a/LguHhzw+atpghw+uwVp6cnxGEgNxrhNZMqJ7RbhrYnMzm3d/dMTk5Yf7qkazpurkIyUECWZzxZzuj7gUVVIYRAGkVucrJKsb67IhOR6aRmaFZMypzqxZLtpgHbs5wdoYEvnx3x7a8+8sMff80v37wltgOL5wv61YarzZbZfMpsMePTpyuGtuPkaEYYOkJpcE3OZDHn5uoOYxSFUcmI3a7IixydZbTtwGQxI242uNby/OkxN+d3hMaRS8nxyZK8Lrm5uGW2mGGHgVwIrBZcX11T1zXTo5q7mzWZMlgbqIzi7q6hzBVKgg/JKPedS861TIvex4fp/cBcwLhv70mA6OPfWgrsD4+PnJfxsQee4NGmzsPmHOPeGP4+4KNIm7ASo5mVD07DOBgBEA+XjYDxsbAnQB7xIfgQUaOJCTEk40rE945cSfqupSXQ2h7bt8Q4QAj4EBBK44cB6wM+QgzpqnwMZFrjY8TbITkvAdbtBu89zlmc99i+x3s/ni9tTAIQIYBSSKXwwZOJDCEkWiu88+RG0zdbRFEipExzZLtlOl/g85K2bRBSEbxDibThECNKK5Q0dG2HEQpTlBA90ZPubRQYpYhSIFXARohSoIRCSAUxEELck2sAYTevRmZDyvHexIAU6dqEFKhx4xtflciI0SGUyETkkZwKAVTjBt0PA0oqFIJgLUWes9luiMDgHP3QpzHQGk1AS4U0EpPlECNSKwZnEcMAgJISJSVSSQQB1/cEK8mNxodIXdcMdkD2Q9rwYXQI455MC84l8sCnCeVjRLpEkLgIUicyR2uFHe23VBLrA5lWKNIkFDGyvltTFjkG0JlJ1J6P1GVBGzsaG0gnTU6M9x5rbQrYY0CJSLdtkbUgK7K0VxjN0HVorcnLAgG0mw3TukZqRUbE9z1CKax16f4Yg7OWvu+QShN9IqGFkKAEYrQFYmRZY9wRIqPTJMdjYM/ECiHHtSeI8Xcez6Tr0gaiRwhPFJo4OmqRx0TIGITsiDv4zDEX8cGWpAd27toD6cz4GiEkIUYEAaU1CIELDqkNvt1ye33B67fv+fW7DyyPT3j97hwRINOS1WqDdBGTSwiCsszIleHpcsHNzRYhPA7B5vKK6aTi9v4eNwwoKZjNKtbrNba3lGXBsp4wrSr6ocf1A0e5wZcZv3rzhjLLMLkmj5ZjU3MnFBfXdxwvap4tS4QF4SPLkwXffbjAmIxXX/8pAkdhPS+/fM7tdkO7bWit5fz8isl8wrMXp8Te0m63eNtT5YIoNCGrMEdztps1k7pkOq2RUWCdIwjB9uaGZ0+fUj9b8Prb15ycnXDxaYP3jkmZs728QEWPbzo+NVvmywVGQdt0SKkQPqBkIie983StpSpywgYwglxLbq9vmS2PybMMYROhvJxVeOfpraWqK0qlME3Py2nFL998hyBQZYohF2w2W2xn6ZsNWE/MwTlLqQ0xQllXWDcQukgvI6FrqOoakwt8G+ju7xHTKfWkRLT3vDiqaVd3fHW0xPz4R/zsF7+kuV+xmFaEYaDfrBidAE7nE/p+oB0cMQSazTZtatbjXCAvNHbwnDw5ZX23wiiJHQLDtiXPM5CKoeupMs31h0tWk4yLZc3s+JizZ6e8efMdk9kcY/SevZA62XuQSKETKZ4mPo9TIHFHGD4K3tkvK7G38bs1h4AgIlFC2B8ICIUIASUUPkSkEOj/t4nP/4+IUiFURIY4rnMxErz/rhemQ6VIn1bIZPfiznayIzN226V44Fv3hgeiiPu4JcZxzxztTBwfS/cjICSjTzFgncM7j5BqJEtGv01CdOO+xfgYgkwrnLMMQ4dAIL3Yvy6RNWEMPNO1e+cIMCZGJAqHtZJMQRA5eWVS4kUWBOFQucDUc4JdY1zEdT2ZqpBZnwgaZxlcigGDcwTESFCnqScF476R7oISiRwwWiILhXWB6AU+hnRc9CQXIyQiYfSfovdIrXCDpaprrGyT3xUjRml0ngJqmSuctSAEdnAE5/HCYoocr1PMOGw7yiqHCFVZEEIgz3O6rkdpRbNt0FJgMo2wKaDvuw6jNW1jKYoMgKqq0HrAOYtSikhk6AeMNliXEi7WeXbUPMCw3dISMFmOyQt8XBGcQ+c53lm6+3uKutpPJDsSXEVVo03yHQORrCgRShOlwMeRqowghUhzjx3JkB4LI8EjhEg2aSQCH/7xsJfuCb0HomQ/gdOJE6EqRPonBWG3FkYyJI6ZXkkiP3bEoYiJdE0+Jvs9WipFDKCLkmEI/Lb4wyBCxpX/WTC3e/iBt3gwEjE9mbK98W8d/8Cg8BsIkMT+7axQiCnjK6VES0nXD4QQ0jGuo8gUP3j5jGG9ZjmrUTplOV+cHHHy5JTm9h5N5MPqhqurG54ez5ECUJqiLphWNSqCkIau63n95i0nx0uKsuT2/p7bq3vyMbgXvue+6WibnsVixotyiUbw8e059fIIaTJO50eUOnDz3Sd+dn7J4ukSOziOzxb4tiHTaaO6/vCBEOBK31AYw7NnS+LguLpdc3/nkWuB7y2TeoIqDfH+ln/6Zz9ifXvFatORTad0Q0+uBCjNmw8f+dW7C4rS8PzpAnrLMAxEIVit77F9TpZlaG0oJxV3dzcUdU3szxAhMqlKglE45wlITFGiFxMWy2UKzIhoBcRAXSejclSVAFSTitOjI4yWtE1HWZVsthuM0UgCtm3wSidmsbfYpqUqDRd3t/QevvzqJT/9Rz/kzev3nM7nFE8L7q6uQWqyMuP8/JrJYsbyaMHN5SXBe5azCSFa4kRD0zKf1JjMYL2j7XuyssQNlq5ZU0xKjFHMJiXZ0QxTFQh5RVFo3n24RGlFXpQEu0EERxYjy7pkOilZNQ13TcNXXzzhZ5sWqQ0BQZUbOpcyynlpqIVg2w1EB87HkSVO0zzEByf8MZf4OPHwmfMeP/vx+Zp7tJY+O35v2x4YWOCzQPh3CR+TAQxCJmM6GkYRRwJkDGQ+v/ZdIBMh7ozto+cgEUuPHyeiYsA7j1aS1eqOwYe0bpSm6QdCjGRSIZRER9BSIknBfl7kOO+4ublNG7tMG3LXtcQYsEOyN9663WWgpNz7Qm3bkpUFVV3hBksMAR/duDGlzVWaFOARYiIvVAq6Z9MZRVlyd3fDMDgyYyDCYBsioE3GYn4EMeDsgHWJ3BBCkBmNMooYHU0PIniiD3vyOMaIjMk5TMGeRBIJxNEBEWPGUCUHOUaikGiRgvYQQ1JP7G3yqLZg3B9VUjlolV4/uAGjc4q6RgWQUtO5gc1mhdKGEBKZXRQl/dChjaGsJggJfdsQYyK3nIvE4NOYj5nNlIVIe0mwfhwbjTaGiEhEUoh475CjIxbig/oljht0CONzgIgj0f4o8yeMwts0jkGk44JMTkJZFQTv8d6zaTqqmAKAYAfmVcW0LNlutjRdR/ABGQNKSZQwOOtI2UqFGu89UuJdYDKbMvQ9TdvinGcymxK9ww49Rir6SBoDJUcVWYCYMmBCSKRIY+p9IjsiMWUSQ0wk2ag08TEiRRz36h2xNRJGAmL0CJIK6PHq+l0ijiTWLusadraAne0U+wBHfM8ofuZSfHZSPrOfe1c9pnNIIDcZTddjPWSZRgyOze0NbuX41//LX/Dq5Ssu2wbbDJRFho8B6VPWsB+VRLOqYjGt2W47VkOHlDnrzQolNJ40DwfvMVnBarul2TZMpzVH8xnOBYJIjv5yMef2/pbbm1ukFHgBz4+OiUrwV2/fk2UFp8dL7m/uklKi6wn3G+ZHC5ZPT3nx4x/jV1tOp1NMrvjZz35Blmc0Xcskz/mTP/0R11d33N9subq/RQoY2o4oBaYqqUXgz19+yf39LX03UM7qFCC0HVVZ8unjljfvP9CHHokm9gGtFNb23N/2KCWpcs1gHYUuaLYNmVKIoqTINwzeIYXAO48PkUJJtoMlAn3v8SpijWKhJcPgcMNAby2L5ZzlssYNkawssDYitaaVgcIY+ui4vLxis96S1yVZHiFEdF2w3mxRQjKbT7DOIVAMK0uxqJISRCiYz/C9pcgNiydH3J1fcP5mxfxkSes7qrqks5Z4u+HVk6e0RUHjPG5ocW4gOE/btDjrWR7P0J2laXu61lLWBdf3a5SUbLYt9bSgyDPM0YLb2/ukeA4B5xw6N4QQGZyn0JLeBa4uriEITl685PR4gUMBAUJEhDjavDEAQRAeExohEndqiV2wvScRxX6t7whFCPvsbwyBsFOFjEH2LrBK5wgp4eMdtS7+XmzA3xXe+0RgwEicjn5PfAgSIe4/507tkfZnmcgKwPu4tyly5zs9shs7lY14bGiIj455yIg/mJy4H1cpE+nqRwJ7sC4lH2IiLWIMo0o1vVJpTZbnyVdRmmHosUNPnuVoJXBuPD4mHzSOqhNIe2cSc8qkdNSR6AKx72gCGCOwXQtSEegpS0N73xHlllwFhqbHukBvA1oLrB/2iboYkkJTKdJeLSTItI8Qk1L3IRmRiCbfd6O/J0AJ4ni9jIksOY6r0jI5yyEQnEMKKMuSGHwidkZCRMrkyyipkpJSR4LzOOfxTYvJDF5A1/TY3hJCpKoLlEzjURQZMUJuDG3TYXKDVpquG5LNdQ7nAnpSIoWiyDK8dfgIdVWkGFSppJIa50Yidh3Wh5TEUopmvSWvI9PJgvz0hLbZpr1ZS9ww0HpPUdcUdc18vqBdr+m7LVlmKIoqJSu8S8SPVHu1FuM9SJvZqNp9RAAmEk/ujxOI0dcZZ2eIDwnSndoyPpCI7NQdMvkX+/h8H5fEkeh4OFZAUpKlVYcg7lVBaY2mZ3ZkSRgTTb8t/kCIkDiSyA8s1N8iPr5PkuyNxN964lFEGD+zNjvGCjEGcPvnkpGyzjGZTVFIzs5OmM1mLBZTtpefKE6OWMynSKGYzCuE8xiVMT8+xgtHJiVfv3pCWRQQJEoIFos5fdtTLY+4fPOeb3/1LS+/fI7MNE3T0Gw2lHVGJSU3V3d0fmBZlzz/6pims7TbAbKMTmnc7ZbTJxn93R2d1GTPn6DfXdCtGxaLnNt3Hzh+8YTgAs72lJlJQZZ3RCR3V2uKWclkMkHEiO0suqjZND1N2zOdBdptR3U0p6hnbD9dUWYZ8tlTrq5XTGc1f/KTZ3z41Qek1Nzj2HaWTKdAwoaQSkrKihgi3gWstdzc3fFkvqBtOlzXYzKDUgbrt5yff0QLwd22AQF5plOQAkyrmrtugMExLXKIkk3vkXnOJkBW1ZQE/Kbh+GSZmMS+p5pO8SentE3Hk0mJCz1Xbz/hljN+8OolzjnOP14xOZoRphVhs+XsZIELAbveMp1Mub/fMJmUxN7jrKV6fooUkW4YsCEQSPNHaYXQktX1ihACz798SQgRGzYcnR2xurpFKM+79+d8/fUP0GVJCBGZ5UzqCm8tx8dz3p5fMgyOvMwpqilZN3C72lL6SNMNSCHJck3nPINPAXOIqTThceIlPpr+8jcsi/A9YuTxctmd4zcszb1jM+7Nny+v3/Sa3wHcGLB/zyTsjebDQ/GR/RCfPS6+b2B2zlyKYhAxydq1VigBtmuJMZIpiVIaqQ1ZFGMgaQhRIDONJMmWQ4SmbdlsN7gY0SoF7NYOxJF88M6N90vsNw/vfZJFSjkqSTxFmUpcovcpuzSStd6GfRkPQuBjIlJijLTtFh8i1jrUWJ6xLxUSSZlhvUNLOaprxjIPwMuU3Qw+jEQGIBI5AQ+mUyCQIo7ZGvGIjJPpM8UduZMyU9b5xOILEEKipEAqNcoa06abnJGkghFS4mNA7IL8rifmBc12hR8GhDKkd0vnsUOPHQakyWi2G5z3iehQAudcWjchZYmC88ToEIApikTcKIkby4FUpsmKHDnsHNqU5Yre4X1yFGN4cBS9TxmItPH7kUxIn0eQshXJt0gb9eAcbryPWuskR945et1AnhkiqWxGSklVlSijaZsGQbIBUgqk0aMfEccM3UjqxECz3SZnCEHbbGlWa6RMGaaqztlum0Rm+PTZId1zqVRynoNAyrTWgndIpQhKEn0qlxJao7ROf3uX7qkxad8JD8HCTkIbhUKMDt7vGgGJYicRf0iGMJJOMKZkxOcW5HMmFWLc56/ZSdMhmRcpZBqXnVMYY1rHRMqqSPvdx9dcX99y4zfMiilrD+cfL5mUeSrDEoIizxBKMM+zVL5rcgYKrvorlMwo6oxSGNqhZ9t2OJeyldvNmtCn8oe8yJFSE2gRumKaaW7vV9xs1pxMJpw+PaGzgYuPN/TBUpU50zxHRYmsCtb3a54+fUq5mCOi5mhREK5uWDw54sOHK3zb0PQd/WbN6ckRajbj0+UNR5mh82m+tG1LPp9DCEyVYbZccHOzop7MiIXH9y3LxZSuyLm9uuP47IQ73zGc97x4cUrXd3TtFtf2FFWJ0grnkgKvbTukniDKgr5tqSYV3eBQWVISxBgYbCBIud8QXQiE1jEMliDSnC/LgjwqZuWMe3tHEaCuS4boyWLg5MUZ3/zsl+AcRkviYMm1wlQ5XdehtWFSlTgfsD5S5JrZUZ3sU1FQFAV2s6WqK7bbhg/ffuDlV1/QZRmX37xhPp9wc9tzcrJEREdzs8aZnMykvUaPpQODC3Rdx8f3aSwGH4hS0A2WrDD0nSMrMlb3LUpqfJQsT45ZrZJ/EiOJ6ZeCrndoIXn/6ZYnT0/xStGu12RFRnu/Rmd5suYx2acYJEImkjslIOBBKbUj/tjbuh0FInbERmJNdnTBXs0wbjIpKIspuSnGktb0JqkMT4bfj/LUhV2ZG+zI/QeX4oH8SVz/AzERx8f83talzMXO2/gsf7vzY0J49NjD87v9are57q5nd1lKKwIRO1j6vsf6mFSQMBIknqGzKXgVoLRB65QUSWVefSLJshykZNs2QBzjhwBj4oMQ8aT9TghwMalXRRSpvEkolNFILSAqdKbRUWDyHJuLMcEMcVCAROORItL3IzE9fvAUVEeE0miVfJxEEsUxLk/qmoCgswFERKrkhxgpcUGOQbTY+zlyPHeUpPKWwaV9bvS5hNFpzD1jcJ1UfFEIpA8gx0RHjPjRn5JKjkQgWJsSMTqX+BQgYHIDErz1GM1DaYgU6XwuIIyk6VoQad/dtons1cYk9YpMSmc72Ic1FwMBlT5fTKRR0zSp1Cym0iedpQRJ17T0XUcYLHld0zUbhEhkT1kUJIWkSkkepRBiJA/2scJYJjO2E03TMM3BfeGL2JU0PVo48ZF/PpIguz1xRw4+kKTigfTY/Q2fqa7CZ6+I+7KaROyJPfm4O58Qkhh/e5vxB0KEjHmU3eB9RpV+79AY9+TS/pV71cdnBz46/U5t8uiGjc5KBLyLCBWQSvKP/6N/hGkU7755zZOTYzbX1/QeaBtW1rE8WVJnBc63hKFjvbpnGHrm05Jm2xACLI+WECJVXnLx8ZKu6yBYTs+WLOYTfv3tG6QbtdlPAAAgAElEQVRJ9Xi31y1iMQEjiFtH9WLB3/ziPSenx0zmM/ptx5PlnK4feP/mPaYs2XYtk03Nj756RrtpWd1tEFrz6//jW1ACUxgWsyndYLE+YLs1d7cbsrqgrnOevniKrCTNtqUsS7Ztg7/fYoWi+e6cSZmTzyrevv3E5n7NfDHj6vyWsjCITPPx4gYtBZPxtVmh0SJS5qne3TmL9Z5m29Deb2hNRtf1VFlB0J5t3/Pxw0fWdyvatsW1DVKMgbqU5CYpL3zf8vLFc4iBy6sbvPQID9PZhGwx52hWUL04w29aiixj8uVLbu7XiN7y9OwE53ruL+/ICsOHj5dcXK6ZH8/RRcnHd9/RrtZM6oosz4khYrKMo6Mps5MF333zji+eHVNXJefn1yxOFwQXyIVMQZJU5FkG1nL2/IzV7Yp3335HOZlipCRGQZZnSJNze9Nws15TFSU3d9ccn51hpMZHjxsCx7MZr9+fE4OnyjOqzPDNdyHVAsfIYAMupKzWYFOwqKQguFH9wAP5sVs6v4n0+My/H5/Yc4l87vI/fqF4dPCeOoifHfI7x27tMjpr46OPyI303Oi/7I5k96fkwWSIkQDxY0AkhUSKiBzLWiAydC3bzZa6Lokh4pwj+khApmw+oIJAaUPfW1brLU27pdluiHFXbuBp2i3BOZy1IxmT5rwPSRUVSBmcQhtMlhFCSMqjttlvuvt7HSArsrTR5TlZXqCNIjrP4ByDG1JQJtWeKddSUmQZdhj2ckmZZWOmPpEogSRVz7QhjhnBlFnZbVzJKfbe7QPKdD9G4kOkbPuosUSQnIiwI6ZFKknROhEgYcwI7ux/IgsCPsR9sKKVou9SOZ5d3aFNkpUGl+qg/UgQQiIc+tjT98mpGGwqhdkRTUomkmb3ihACoetRWiNUes57h932aG3G2llB3zYpkDYZXgWsc8lZFEkpsasLhh0ZEmFHTIyLRwpBVKPjHOKYoIrEMYOtxh4g3qe62CKD4B1GJ2IiFxJpMnrvcH2PEMleOutHEkwRYqoVN8bgnKPbbimqihhKhr5DhuSQtaKjLEvsMND7AWU0eZaN5/KECCpLvUdCTEqQONYQh3ENKJWyoWLcl3ckXsqOuf1i3REGiRz5fVmNyL7vx/ibeGQzH0vo9nYjPhy/u+79vX2UFY7j+kiqr1E+PL6ybVuqIil+nOtZXV9x+enX/NW//SXmaM4333xDMfaYUTGVdAUhiD6SFzllUTCras5vb/HtgCig23h0Jbi5WxG8SwRYk2TeeZERQkRFgdaS/nZgVgru/JaLm2ueL5YU05q//vkb6jzDq6TUmU9m9AQurz/x6tkr/sP/+J/y/vVHgncsZiWXn25RJuPy+op135NHePn8CY23fHP+Cd68YzZZ8L/dfocQgeWTU05nzwnrgR++esab7z6CC9RnR4T1lidVSVjUfPfuHBkDp2cn3F3dkjtPKA2v371nUlVY5ynnNTLAJFP0g8OG5BoHAl3foAP4wUKMtJseIQR5ZuidS/ddCKx15EWO7RzKaMoiw7WeUmuKUnN3v6bddMxfzLi6v+Hm+pwvf/CC7fU9ZVkyeIe7d1RVgfOBvulQUpJlBqOT0sfHQLfdsG0Gnr04I8hIc7uiXsz5cHXPtMqZPjvh8n7N2XTB2XLBarVlfjLnu29e8/LrH5IdFdy/vSAvSvI8ZzJbMAwdx8HTFjnXN7es1huiVBip2Kxb8rrAlBmi95wsJrihI5/N6LuGSVXTtz1RQDf2lopANwSu7xpev70gn0w5Oz2h22ywNknsY2qesFdQ7f49BEP7JTH6BHGfNBExPmR7eaj4Tz0cUkaf8fedbRAiIlEpCBYgFEl1QySo35im+f8diVAeS/3kLsmys/E7WzCSo3FXApP6FzggBo9SZiSTRmXNbkRGOxJ2dmQ/mKNqcqcwDKlni9o1gxtta/QBk+cMQ896dZ8IfOfQOkMKSd+3yU+JPgXvWY7SOiUlSP3uCB43pNLPIfR4NzB0HZD6UEkp0z0as/zBur2SM0hJiIGIIkqZCFQhEEjKPKcuC4KTlJWirnJc1+K6HiY6KS8Hh/WO4ALeJWVRDAI97qVlptBS0AWBltANqUxkR5gwlusEKdCKNHeUxAhwdlSNADJGhJIE8eBfEAPORzZ3a0xh0CYpyoVSQERJiRsseVmgipyhH3DOfaZI0WMZC2NyJfiAUKC1IQaPNgaBxkZwLu3hznqUkBhjiMQxqTEmIVTaM7SUo6JH4AFdZLjgyY1BScl6tSVGmE4neOdY3d+lPds5QoTJbApaIBnnkQ/c3l4zIWCAoWvZcEtdPQepE9EYY/IFRSIe92Uuew9v1zHl8QrgQYnDQ2ywI3x24obxv4c1RSJilHxInEkEftyfd4TJjkrdK0Lko88UH0hGKSVxtCNCyv01q39n/doD/kCIkAfGKPJAUCQP8qF3SNwNzuMP+OCfPI5/9s4JMLJHINQjr0c+GGydGU6/fs7quwtuv71mfrqgOp5iREDNK0RsWZRznr56MgYwcHpyzOXqhqxQaJERY6Qocsq6ppxO+fT+gmK+5PZ+xdLXLM+WXF1ecXN1zeAHqkwnyaEduLu+ZVLVLL885fzDFa++fM7lp1vqoqBa1KzXG/ptQ1HnXK/WVEpSScnHT5/AekRWErQAlZoBdW3HtR2wfWJ0o8kQmabtUmPW1XrL85dnEAPbJtXOC2WQIYBwBF1wc3mNG3qkFlxeXJNVBR8+XDCrDC9OZ7Rby9baNObDgJlN6YaO3OTkeY4WgiIz2KEj2MAPf/pj3r55S3d1j/FJPjcpazbbFKzkWhNMumObbiA3hrOTOcP9mkbAQEC6wIuXT9EiUgnPpC6RznP0/Ix8Pufy3SdKZRBnRzT3G4aupZpOGYaBU52RTyecX3xCK8lPvnjJ1nt+8YtfIbctWinKqqDYGorM8PXL57w/v0A+z8gyzdtvP/H1l8/Iy4z+dsC7wBAteZVze7emrovUnMw76sk0ybBl5NPHa6aTnK5PgVZe1LTbDXVVkhcl6+2W5WKGQ7BtekwmmEyXfP1lx7/91Ru0U1jvmU0qWjsgG4sfWQ5tRGrOOfIAuwap/7fYrZVHa+b/iSwRO3Jq9/z3lt7jJff7wD4g+Sya2T/7KAADRhckGVS1z0glOyEI4yDGcXMXMSJU6vHQtR1t2yKUBqlBkur8x41hsCnw1CajaTo2TUs/dATv0SbHh6SOikDf9+zkfmnTScqHgMNaNzY3jalB6e5zjA2klFJJTeJC2rBjIASJynOstWiToXRF1BHXtmRZkV5nNMGnaxiGnizPMXmGdQ5iyiQJKXf88OjcCJwwj0ivXVHI2DAuRh7l9/bzbpfBkAhicGPGIklJk9xxrDeOSfq6u4/7OmY5Otnjvdo3hAOiSJt6nhekjJUdncWdfFLsHVGiwIdUahJhbJCaSA/rLVobVJY+XxgVHVEInPcopdDGkKscSI0Ri8ywmM9wLil6hIOoFDLLyGIiJJ1LjQRD2Ckqxqm4/wxxP12FSP1tHs/lEJMEF3bNjQe0UgQpiDYkx0iC0IpCK1xIJVuZMeRZzqZpEqk/vo9UKpUihYAdSaGyqtLfw0DfdcwX81HNksYKJTFSYkeHV8YAUqV9VyZSRMuk1NmpYJTWBJITEkXAWYsqChiVS2J3jyFNgt++dPfvFd57pAj7EpnHCZhdE9hdlveRE8LO8D0EhIyBGw9B0N5IjrLeka1MjfYSGbe+u8Z4z/vrt/zVX/6fHC2f8T/99V/h29QUNxLJMkMQiaw7Xc6ZlhVba7naNAQpmS6mrO43TI6mrLqOZrtJzXBFhBDJsoxMaWZ1hY0B6wRPz065367ABf7sqy/Z+MCn82u+ePWcxjq0B13kXFycU2clf/KTP8dLw//8P/6vvHz5Ba139Ldb0AYbLFpEzuoaVRb8+s07unbD8WKCPltyfnXPyYsz/DBA43nyxRmNafj5rz5Qz5Ii8kmV83675tvzT2M5XKRtVnz4kBoGXt/cM6kKtMmwIQUFYRPQUtL2w6iGSHX7yyzH2iHV3XepCXE/uBT4BNBKo3ODHeyoRpOUdYXUiUQQWZ6k+FLzw6+/4pevX/Pm4zkvTk+o8hm5VIQnS9799d/gffKVsjynu19TlCX18YyLtx9Yr5tEDofk/J/MSz59+MTy7IRBCYbrG0xZcH93i2sbqknNm8tzJvWEBlCt48Wf/ohf/OXPOD05ZrGYcnmzohs6yixnGDpCYjp5+vw5558+kecZAoVzjvVqS9b1yecykdbD0djscuhT3xM7DAzjXNcyqQicEPzy9UeOnz7jvrOYrKCoPZDWtpSpsXQUifQfi8vYNcsel8Le194F9rseZrsgZrfkUz+hNFdDGMsRfHhYVolm3wdYQoGKpGN+D/hMuTFKMARiTKqOSsjRtstdz6Ex4ZpsvSI1A931+BhtiHwIHne25EFZwkgah3H7T00pI+n9g7PperSi6Tr6Zou1NjW9HHsRtV0zEq/pWKk1uyy99w6EwnVtsntKEvyA7fvk842Kj13j6/Rmj/aqsXRXKkWUyYfSSgKaENW+xDIgiDiUzjGZIKrIfT8gderb1YoWLRRaKLJgadpEOFkERanRZUZuFFkv8Hagj6mEI3iLEgqpYOhHoscFopAjeZbKUrxLacJd6avcfbZx3I0x2NDTbDry3JAXqZF6DHEsaW4ZuvRlCCGMZZ2CPTGm9ViONJINMQSadcfRicH61ILB2kBW5CmhNaphnU3lKNaljrJ5lo2qzERYBS3Jy9TI1DYdtkl9RYqqxOR5Sh47z2AHfIjoLCPLNV4b2rZJa3BUqwqgqCu6rqNvW1SeJ/WQ7bm9uWJ59hQXSORHjCmJEpLKKJEhcb9/i900EOz9GRHH/jR7//DRetnP8EeB+Y7U2B+zd5IekS/sVZnfd+8fp1EEETHer90U3Z1XCoH7O+Rb/kCIENiTIY+jtJ1UGvaO7vcHm91APZzis+hMfG8U4yiHMpkhxiR7N8ZwUp1w9oMJT5+csblbsfr0kX/z5jX/8B//E/75P/tnfHj9a6ZVjpYFeMmHq0/YfsvpbIorFEoK5tMlUmZokySTl+/eUWcGZSQyBqSMrNYrVBT0fY8YpWEhBjZ9y6zVVNOCdrthUhv6ZgtS4cf6++ubG+q8IIqQvlXlukOIgIh3zJ+ckBvJ/d0KaQzXY0diYzRGwtBtU0lKiJig+dnP33A0nzAp62Qk3Zo8yxHKcHu7oqpL5pMa5zy3/oq+6VlOssQgZwbfWzIXyYuCZkibLULg/MBcClAK4TzeerZNT9t3vHr1BdeXF2zbNau/uaeqc/hkMUrQtm36BpCxA3mMgfXGcrPZYLThyekJhYkIPyCV4epuhdVJGfHuwyUnJ0c03YCwA1/MK8xxzfp2i0ZhS0mmDaXSTPURbTtgMsM0qyj/4T9gc7+hXW8IznP+8YIsM5ydLvnRP3iF7T3VZMJ8UtEFR78NlLMKbz1D3yOI9N1At91ycrzAGJ0aTmnF8ckx1XTC6n7F3e2WprliPp9htOb9u7ccn55SVQWbrmM5LfkP/vzP+B/+zV/SbFp+/JOXRK34+TdvGW6SomcyzalLTdMm5lepxJRbmwKfnYIyrZfHayAtCQn76hAB+y/Pjo/Ijv3vKZbcB6OfEY48nOPvYGv+XvE40Nw1V4LRTuwCTbELYGCfgUoHjYZ27C1CkuhKpVFKpXIYQvrmlqalaxomk2mSZY+khw3JdkzrCTFE7u4SK98Pltl8jvMVbbNN6zwqvA9s1nfJCVWaKHfNXSWD80Qhsd6TKU1eVQx9jx4dJyEFQ9dT1zVEQ2Ash4kp0JpMJ0il6duOKCR1lTqEO2v3Y2IySTmp931GnE01siozBL/L+oELdpSIOobmLjVPJTk/hNTYdJeLTcTF2AR1nAlxdJwRqZeWkImASNl3ORJA6Zz7zNKORRHpW1zYZQyUwui0Rdnx2womk9QsMAS/L0mJDxMhda1PbFfqGC/S+9ph2B8nhMDjxq7nqSQpy7KUcQtJWmqMpsgz9PhNKlKmDPNuVx4Gy2Ad3jn6vsP7wGCHMSvn9w4x+5+C4FM3/zAqe4T8vNGY3HkZpGDFRVhtWuqqQGhJO1hEtPsM9GIyxTpP33cIIVPm13vatsX7FJBIgF3PmeCRSqXmu1lyzpptg5ACrVJjtu16y2RSk+UZw2Dx1iFkQIwqDzF+e0w67ejg20Sk7eqM49hAWI2lQd66vVzt90mcpiAsfOZLPHAdO+nvAzny+JtwHrRmKfh5aED4+PwRRtkxgkSUihRADe0Kv7njZ999w9Wv33B68pL//n//C6yNTOqCYF2qPR/3wJcvntFvO9bWI4VExYHt3YrJbMHRbE7btdxc35LlOUPbkRUZRZZRlyXSGJzvWRQzrBbcrG95OjvGKcFt2zEvJrCc4awnC5HOOY5Pv+TZs2fcXFwRpaAqDD/+8deUR8ds7q5wTUOVVwiT8f71r7i3nrJKjU6DC1zfbcnbwERpzmanrDa33G9vuP50BSGyPJpSTaa8+fWvuTi/wNqBup6RZxpdFLTbFqkk1eQo3RcJTxZHFNOM89eO4ODdh3Om0wlVnjEMOda1DNZxv+1wXfqGm0QAarZNhxcx9QvwgSrPMFrTtR1VXSO8R2vJYnGUCGkpefP2LcO2YVlPOV4uGVzD8ekJq8Eyreq03gfLdDZn8IHV/YrJtKDrHPPFBNv1LOcV202PzjO++vKITdvz5MmC++v71AelqpBSMrjItJ5hjOJoVlBOpkgv+eL/ou49n2RL0vO+X7pjyre9buyaAUCAICiJEkMh8c/WdwYDZFAExV0CCyx2dvx17csel04fMk913yGkACUGBqyImXu7u7pvdZ2Tme/7vI/5/BM2h5YQIh89P8MTef3ta6rZlKZv0VrTbdc8v7jg0PX0Q4+pC1aloe8G2q5nsI5JXWJdwPUOUxQUWjAMHM8cl72ADn2gNJKvvv6e2aTk7GxBWYm8hnMNIGUyOZajx042uxb+yH4YwZHRfjgkb8dsupjWx8j6e+oNEkNAxJCvQWqipXhktyZwWyL+4YPpgLwtpGX9SO1PLil5eh0yKDz+kpFRAqSkfhzGiiSbS+t77GcyO1EmGVkCKMZpfGrqI1lSKSRu6Ak+7a3eu+zHlZjsRVmmfzoGdpv1MaVK5jSYJG0M+Bg4bLeYokjAcJ6g275L+3PCRbIJZdoTZSqoUnOMOA5popIICgoBPjoCEhEDwXkGIkpsSTiMZ6LnDM0BE2CwPU3fpaFyjMgocINDS40UkcE5Qu/xoUdMSp6vThC2o50vGAbH1cNDNhLtKaWkj491xNAlU+9gJFoKSmMgy11jjAgl8C6l07mhx5SaokiyYdtbnHRUVZGlqGUCHNqW2XRCJ2AYLKXRiXmTDcaVSkk9UqWaZfdwQGRTcW00buiZzqZ0nUUARVWy2+4z6BIoTJL9ejsw9DbJXcvIcjmnLAq6piMA+80OU/WcXZ5je0uzPxAQ9N2AVILl2RnFZIId0h4QXDKG7e/XyRQViTBJbquMoWlbuL2nntQoJViuznnK1kpnfmLIyHHAxGOPwFhnj6DesW2Pj0b5P+rphVKMm8M4VxA58e7vaN7zR2kRhuAZ/YPSj5THc/koq85r1Gcm19/38Y8ICPkv26ofewB8oCwSj8hrqnsl0T92gpGI0iZ5umQ98wiCTJfzvAlHTk5PqOs527v3/OyLz+ls4P3t97z6+CV231C6ls3DLcqU3N+tef7RCx6u3kFw/OLVJ6y3W7pmx2I+p6ynXF/fMEXmzSeyOpmzXq8xGva7PUIlJP7+7p7z1YreOpqu4bOPP+FuveZktaQyKSJV1SWhb7l6d0NtFKvllP3+gNY1nR148eyU6WzB999+z/27K06enfPF5x+D1tzfbXn99WuqeUVdG85fXtDsGhaXl0gXePPDa6IT7LcNQVjmiwW6MiwWK0KIvHvzjhgkxbTg/OU5cnfg/k7x22++xg0t89MVfpJ0iLNJiR0s86pMiH5ugqwdKMuS/aGl6ywPNw+4vkFpKHTJvuuOUWmjGdOoQey7DhciL89O2Hcdu809jdaEsMbHwLyc8Obte4xS/JMvfsZuu0Y6y+X5Ke+++R6lFc8uL2iaHYSAdJa+sJRViq4LEdAa27bM64q2Tc3Ecj6js5b7zRahDVJE7u9eM19MOJlN0IXCDineajatcc5TucChsXz3/Vs++fQlpVYJjS8GjJZM6xotNQ8PW/aHhqo+YTKdsnm4p6wNi9WKLoIx8Gd//Af85V/9lv1hw6RSLKc1231D2we0fNTxa/lIPdMaBvsEMhWPq+m4PsQToEPwmLo0fi3+6Ht+vIfE//J5P+XjCH7kONdxQnPUJMOxST6+1vxmRHK8okvRqkrpY5SmHKc9QmKHPhlqTSYU2WROG83gPMokY+B+6HEBvAsUZcFmu2W32+JiZOj7ZCQqJW1zSNpaYMipJUKIpEMNAVUUKKUY+o5aTxOo6Bza6DRRO053BME6VFWilcE7S7PfU02mlHVNDIEhJ4lonRhZUin6piHmhhcpMKWmbx0hPrIXQvAURqfJp0gGi0NwR7qrUiJPe3IzSJZyEPEh3XQu8FjYKpMYGnDUr4bgk0mtGBvOx6JwvDnFk5vVugxYSIEWOpkgesdRavHkv0fpi8oRgsOR9fPjR8xTkyQh0akAiGldESKuH1JsqSkwWoFLyTRKaYTSKBRVoQhKJykc0PcD/dBjbZo+u+DzekmvQUmFNjGZNIZHkF8c/4uPt2l2Pw4h0LQ9oSowKpmLiVzUqZi+22TQyzmLzpK9vh/y75neZ5mL6zHiLwhBCIIUVBTwJHaQ9z5H7pZPpoKPYFPaSxK4JXO3EkL2DtBglCJA0hzLgqT7l4zgQvq+n0bvDzzebx/cEh/eH/GDz4hHyQscr9DRJDEnw+RSLRdmSbtsCoMPnnZ9w2G/ZdftePfN10yqJf/mP/8a7z2VKXAu6cm7zmIKw/PLE+rKoCW0uz02GHprKWvDal7zsDuwXa8pjcJaT1WZo4fMbD5N967VdDLQ7vd8dvmSPlja/Y5PX37M25srRFQQIrPFimfTmmZ9w70omK9OGLqW3kZ27Y7T3lEazXxxzt3mnt31e+rlkjI4BJJiWrF6doqUcPbiHCUE33z5FSezJSe//IKIwwjJ/XrP++9+z3y1ZC4iZxen7O7WxKiQSvPFH/ySZnNHYQSb9wO7Zkdze4uZVFjnCUEwm08JIdDahM4Krbm5XfPRxy9488NrrHOUdY2OAjnY7OuUkr2E0GnoVRi8tayWK67e/AChJIrk23UyXzApK6x3NH1Ls9uzXexAGIQxVEUyhFdaUZQFs2lFWRb8/JcfYaLi+zdvMXXJFy9e8NX799xtHrg4f87y/IxClZSV4eZuy/r2jp99/hnVYkpoLfu95/7qjvXmHh8s0/mMupxw3fU0mz26qLi/fWAyK4kSisUMYTsulidc3V9TFhqpDMYYdts93nmcj8kXoCxo2o5hkEynU/aHJt3PCQdJwH6M3N1vub3bcLKcp4m3UGgVc3pajg5HJhA8JGQgktmD2fiao5lhSOC4gB+7ASUgJEXLk7/n6CYiZZKl5j1qTJpIDdNPs2ekHkQyclqesl+UyrGvWQIYQxqmyHE3iD6faaOhOMdCTMmR4RGxNpnhKp0iU0eGSGIgpOGBc8O4eUH0eDccpTPeJTmiUILDZkeIoE0JJPmGUCoxErzL0qP075uqxA198rIZ9/njuTMOJMSx13JDQGqdf+/EgiSQ2Og+XVciKC3AO7zXhBgYXI84eITQrJ6dc3h3TYw9UhqCEPjOEklpdN4NCARKJZahC5LNruVsMWVZGPbrLZOqQEiDJzXP0ifpC+7RwHykMutCE1W6Lt66IysjZqWBt0kmp3RuqmPEDg4hE7ggRPKncMPAfDbl/mGd+xuJVAJDCoAY7Rl0YbAxyZsHISgy09MOA4xm4kSm0wnW2pRkN1iKSmQGTQKaCpPMVe2QmMKphkrrar/doooyDZ1CYmDaxiXzchTVZEpRpoF6yEai7X6PNTq9ZpOAMa0VznV0rUOZAtNsqcoq1wmZ+Zvr6VRHi9wHJLn1UXUR4uM6z+BFCCnqR+ba6Pgz8usZ8Yx8cz1+/cd/R+S188i4eno6j95HIiciKqWPDN8fn+v/b49/REDI+MIz2DHugHEEP0Y2SAY/sjnf0+YM4JGjKpCFSpq0OG5SElMWeGsRUjGbTvKUzFLoir/6i1+xXJ1xcf4x0bacXlzSe0/TeabzBbvtjte//4ar2zv+1b/8F3SHNV3bEqNAIenbnoeHNSenF5ysZly9fc9kWtE2LXYYMFVJN/Rs9weUEDRdi3WO5WTOQ9MxrUuazY6WSGE0D22L7y0vFgvWmzX3zZ6iNBQKVFAcHlre329xzrIoJ7x7/cC337xhdXHJ85fP+Wf/4o/ZPmx5/+6Oh4eGup7y7W9+z2RWs5pO8UrTDg5dlWxv7wlWcHt9z3K5YD4peLjf0BwUqlKsZhNenk2Zf7Ti1//+1zTbHVEo+hhQMVLX9TG9hHyVXHAcmi0vX32ODZbz+Zw771m3dwRFipWSKbryw6Y9gVZGKW7u1+jSICO4fC8YIRGF5k//8DMWiynNw5qqqgh6wrrvefXROaUuWG8OnJ6dc7+9o7c9FSVGKQZjcY3FD47L01Nu1zs+/fgV682O9++TdKbrHO/eXLFczpjOJjgvGLRkWhaQF50LMaXHhICPnqIo2K53XF4UWD/QtRJTTJL0pyqoZxVN03Fzc8cvf/k597fXNIceaVqmsxlRSsxU8fLzF7x9c8W0mHC2mrJpOqzfUpcF26anKhVN5xER6lKCy+7ZjyqyPIkkFyvHTx3BkA8YI0+X3pPVeARYx+vy5If8lJNdGAGb4256vHdkLjCe0nKPutC8p0RilgPrc8UAACAASURBVKpk34uYKMBCpqZeAs4mrWxZltlnIjmuW2dBKIwWSGnY7DaUZc1ydcJ6s8ZHaLshGY+KFFvb7rdJn14Wj1Oy/PpCdvOPmZngfUDE9HsMw4ApkheG1BrnPPVkkqLb2g4znycWwzjtER5hkoEXEpztk5lZ1mz6fqDHpkm9TM7xfd8zDGkyUxQFzo/vScLhlRCpcJUKgcpu4pkNQiQKlZNTUkGtRchThNTMh6znTJOsBDyMyT0x7xcfSJzG65jGUkfPDa2TPjaEUdowAl1PJgFZ0jM4O8Ll413Co5t/eu9T8Zrv6hCQCKra5IM3YJROqVTZeExIgRIJZNFaJ4+CrE8uqwJBzNrazAiQFh11vr72cb1IhcryJ2tTdPIR0Hlydo3FA0IwOIfoBRQ66Yx9opUXjAVLfFKUOAqtCMFg2zbJosJIOxXH9aAESDH6keS5i0pTNWstUilMWaREGoDoj2k4aTwqMxgpIWbTWOdTJLFO/3ZQIfnAII5TouAfr/M/9GPc3p4CY0cq7tNn5XvlEfx43BiPbvbjDzxuQY/myyNteOg7gh/Y7+5p9ltuvv6a6DV//ru/pTs01GWN7QaElClFoNCcLhas5gvW2z2Tec1qsWLbNjxsG1arMzb7A4e2QUpJUZcI7YnOc3l2QTt0bHY7ptMVr16ccPXme+blgo0dMNGzmp6w7lq6oWM5P2U2neBsz/XNDfPVGdOyotlvCEDXb/nko4+ZT1eE5sDVfkM7tLx6+Yrb2yv2hx4nJNO6BBKj6ve//i0OwS//8I9od7fsrt9jXWS32aB0QVHW2O2eajHn4eqerm2p6pLD5oG7a0uUoJD46HlxecGbqzt2t7tsnphNBRHJyFMoNJYgJc1uj9QJzB66Aes9UqRoz6I0qfkhUeG7piUIiXADk8mU/f2axcmK5tCymq2wztPagaIoaHuHt5FDsyH6yHZ7x3pz4MWrV2xu75M/kIvs+p7ZbEqwnndv7vj2m7eYsmQ2W/H1l19zsT6gJjXD9RXz+Sk/++UvuXrzjmrTUa9mbA9rXr56xXxZISrND9+8pWnvmU1rFsuak+kE9/KCH757w9B2WLvn9GTOYPdMq5pt0+L7Dl1o6lnN0A0M/YDSinpZUXhDFJKut0hiXqcCodLw0FrP7tBS1GWSWcUUpy2jSxLRtFunc8PH1FDnCHVixPMk+YWR5RGPSXYfUNdFZpAEjxg/rzLwkQ4lpNFpGZIaHBkl/r+G5/7f9PFkZ4gQgktmo6rADn1uENPvJDK7I/rMjjl6aOS+JQMgOeguAVFDBuuVymkpiSWgRBqcBzh6xSmlEFLTbg/4EAjegVKYqmIYevYPD3RNx2Q2Sw2vNse0IJ/Bp2BTpLp3NrFK8r7tc/JY6mgfBxwhJCPS9E6kszKl3IpsmA2DjyiZgRCRzht8wOJwHnw/JJC8Umw7h+sbSqWQ04qu6/HSUxiFjwKhFLqoEMFiQ6QS4LoOMZ0zLSqmJwolI62NKOHobESGxFYMkgQ6uZyohqBrO6SAsi6R2RDVWXcEkZRWyZQ0g0WJUZbMSZUSublWhOBRIjKfT9lv9sjsIaKNyazTMVnHJ/aPlKkh9+n9HUg2AlonX5GoJLjEAo4xEnuHNiYNYXygH9yxvhUiDUCD9wxDGoZJAWVZ0Pc9uijwMeJ6y3ReEdyAMMno3fY9Ulb0bYcdbErw8h7vLIVSKKAycwiRvm8pyzKnTDlkSAMTeXwdj2fnyBIam+9jEiuPNff49/F5kiSxO37uOFB4LAnGaj7mBiRmUDWtvyyhi5LHfz7VI3KM4T0OmOKxTvz7PP4RASFPH/mNeDqaftIpj0ZDgnjcZIEn+vC00bim52heJlOec0I5U271/iG58Q9dT2kMp8tT5qUmdHfMTy5Q0TOdTCi1pqoMt+8PVMbwP/7Zn/Ld6285WS1SAWg9MaZNsDYV/dDStw3WDrhB0rZtov7IFPtZFiU2Wtqs6bzdbfh4taRrWqaTGoJnv97ihCTg0bZB11NO9ZyWwORkRqUNb9/f4wdLrQy3Tce0KlmcLHEBdndbrl63mLLgZHVCs99BGDg7nTOfL9HFBFUovv7hLa9//y0fP7vEhoGh3bNRgSAEyEhdT3DB8eb7NxRVRdPtuLi8oO8dV/d3xD651QfncYND5MYghkABnJ2d8+W3X3N2csY6Dvz5//nnfPLp5xy2DUHJNC3MU1uRTW8ESSOPEBRFgRGSIQRKLfn0fEXT9RRKs3vYcvX9a549e87y4oK//tVvOVnO2NR7JnXBx5++4upmQ9sPFKJIC1pH2l3D6WJBFAW9D3z8ssZZiwhT3JCuQ5qsO9abLX0MnMynPFytMc9OWC5qvEumTT4EpvM507rm5uoaF2MuxBR971A6Jb1UVcWSGcTI7e0dm4cNqqzQOtIfDnhnuXz2jN2259mzc969uWJ1NmcIjq6zeGtpmh7vPIvlhKbd4h0MInxgjDoWEGMRctye4pOPnyypJ0vug8//CBf5sPjPf8gfPecf8uFzgZZewKjXf/T6GJFmRnT7aNKY0z2EoFCpoOltnwwpZQIRXHC4oWdsSK1zeCGyUVY6lFLqhqbSJQLNYNN6NrpgvjAMtkNqjV2vMaZAKUMI7vG1CYFKfNlEd25bdFlmmmGSnAXvcXagmkxSg541pZPZjK5J98woV1BakYc3dO0hr6tUWCljKMoSN9g8mRC4oUMUJVolMLLvOobBpkLEGGqpcNbifUtAE0QCkgOpsU0TvRFsSu9/MpmVGJ205AGBjCJNDIJLoA2P9yMjo0+IzEoZEbu0j2uVDOJi8MmYLESGoU+xsvmAePQXyRPKLLfJwwuUFEd/kGPRGtPdK0S6LxKjIWlujVLJyNgFXHQIA33WTFeFQRYiNxIKoTVBuGzWFamrCQiJth5jbbrXgscOybjV58JIKElZjP4r6TWN2mWeFhD5/JNKYb1D2PT7aSnRSiKkRUd1/DljBKWSikldJtNql/wSxoJjdO6X+T2SIpm0Jbd5hVb6mGqUEo0i0XpkZi6l4WzSDkO6h6VSDGGMxIxEladCw4DDpGYqVy5SJtO6n+LxY37psexKk5Unz3vyrCNiPDZz4smeI47NoJQSH8WRNRZCoBs6bLfj/uGO7e0DXVT8zetv2d2nvTtGiy4N/TCAU3zxxaeEoeewb9MgICreb9eEGJkvFxQyst719G2XvWFS03F+fkaQMQF6smBxNuX7L3/LRy8+oRlaaqNZzs/43bdf8uLlZ8wXK+q6xIeUMrFcnYLyHNbXENI1+uzVxzg7cPP2ew59T7PfMdMFX335e6ztGSNFWzfh4HYI4Tk5P8MYzfe/+w210jifaojPf/45+76BtkHKCdb2lHVBLGqCt6wuV7SDZ7CJgXcyS4bUL1+dY5sk9+htMoD23qc6aT5D1jW+a4hSUVUVLgT6fuDZ+Snvrm4JwaZJIY56mlJdiqLA24F9c6CsCkqz4uXlKT+8veKkLjkcdkRrKbL/wq5tsAhsjJjplEkAh08JQVozXUzZXHeoCJ33zFZLTswJTesYhp5Xn33Ezz55zu2m5b5rsV3LDvjn/9v/zM3b9zR9hywU11dXTKcr7t6uEULw/OUL7m9vaNdrXvz853z1V3+NMiXaB4wx3N5vKRXMlwvmVcXgLf0wMJtUtJnh1/UWH0SKmA+Boe2YLWcoIdhsdqnxiqCMxnvPr/7jX/Hy5ARdw+p8iutFAkVjOJ5LQqSY7HEQIfK5GDMQ65wnBvehN9BYk0iBiHnuneU0o1QHqUDpZOxNjpEVMhlAqxRl/lM9noKkKjNThpyKpVU2p9YpuSv4cIxRReQzkARKjJPvQMiy1VSdGVPlAVVawzHXkCHXNyEGZAR8ZHDJY0zEJLFVRuOio28aRFTM5kukEvTdgegDQz/khMEcV08ybkemqF2lkweVKook4+iHZPCqE68l+EBwo3RYJY+vEJOROQmocUAlQEqdJKg+UkqBkQlkUFFSlilSdri/pq4qpqdnNOstqpBM6mWWVKa6WWtNGBqq6QSCZPvQYnVBGzRVWVPOFcL2OCmwhw4ZJFql9yF4nxLkZJLNI9PQZug6tNGUpUFribMu+YuEkL1w0r4efcCTpF1KlyAith+IWrI/NMwWM0Sc0TZtor2TgKLElkhrxGhFP6SfO/r52cFlgMUxmU+JAcqqyhYJMQ3JyACS7Y6yX60UsixyvL1g6AaqKiWCRhJ7yPt0fVIdE2gPhwQgKklZ14iYzFojMUVwO5eCHMqSpm3RVcVytkAJyTBYqlKilMR5j5Yj6DBCFOl/IcQ8YItIqROIOQ4RAAhH5ujorTIOYUexjBiLNAKjf87IOkmziPSsEMMHA7M4Tm6PPUDE+bSe0vIMqDzk+fs+/pEBIY+F8Ac0j+Onn6Kzj4Xw2N3F0SQhv2kjVV5KyfzlCQwWJUr63RaCRxuNVprSGJaVwpjIbFZRK4MPjvPTE6azGc2+YU+aLry4PKcuNTGmibApy2SyF8HUBSenp9y+e089nVJVRSquqwo7DBx2ey6eP+NwOCSsywYaBk5nS4bBg490Q2D7cItEoqSm7xzWtRTaMpea5eUZ13cN7eENy3rGbHbKrj8wW9ZcvvyI1cmc1fkZt9sDt6/fUYRAkIGXzz+nLgRGlbx/857rN2+oVwvmU83ssxcMrWO721PUFZvtgd46Tk5W9PsNMQT26z26DqxOzxnUjkN3z2IyIU4lwdtE862KNNWUIkXGhcjNm/c8u3hFFyNv333FyWxFGQ3Bpqi5GJPJkTGaKGAYMiiSvVvm8xnODVyUFZN6iguWybzm2cUz3r57x+mLF+yd5813P/Bn//uf0veW/nZDlJKrm2tk7Pnk00/Z3d3h+wPKT3l58YJiVrPZbBEhOVHP5jWRyLSfokyalCM0gwvsHzbUheKj8wse7veUVUlZFGiV9PLOORSC6XzO4XCg63rKSZk8AeyAkIq+75jVM5rdno9eveCrL7/jn/4P/4TN7S2m0LRNx/7QMDUlMTo+eXnO3f09GsG0KnhxseJvv3mHd5Gu6Tk9q7i+6XA5OcaojILGJ0tHPOptnX9MkhmX1/h4+vEInDxdkeLpX54896cCQdI//kgzTB+PTU2mc5J9RBgbHQFRkgmdGKVSuoJ3lIVJ9NRhQGudixSFKrI/hFAZkQ9MphOkTPIYmemlQmm8jZRVfaRBDtZh25bBDhTGIATs90Nu8hMDxYUUh6tN8RhdKwRt01DPppRlmbWtpKIqRoauZzKZUFSJwqikTAw3EsMj/e65yc5TmuASU6SoqxxLK+hyk15NKmyeBsUQEmgjswZcKYySeDFOtyBGl/a+EXAabxYp8wQl6TNTwyzzARgfY+ry/4P3x/snMVhGOjDHiQohEIgYnaJ8rXc5hjcxOJA5nSb/pBiz6ZdIB/lYiMvRnTwzSkJu2CFNhKRMWfTWkgpbEdCFQonkzi9NSvARoyzNaAQC5x0h+qO5nzKGWkmMcwxD2hsSWCUwOhmQju+1lBJRFiiVPj/E4YM1OTILYi6MpUzgGyIVBj5ms1ojMNnpXuRr6F1Kz5jUNevtDi11MjnV6ggapemlRMpc1OYpuihMigXMtNSyKOhDl5psYzLDSI6XPIFSIaIzUOKtJcaQ/XQsZPlRYcwjgyf+/Sc1/60fP2aDjKDb3/0ccUR7j/dZHCdTuSjLDR0xILQGn4o2HxwyOvrDFtcO2CD493/1K/o+FchGK8zoC1SVPL+8ZNh1nH3yjDfffMvpcsV2u8umtZHzxQIfIk1/QKvULCAEX3z+Ke+ubmjxLM+fUcrAt19+x0cf/4zdbs3pfI6Lgof9mqmpGHKUos2+Q8KU9LbjcLdlvjhhMqkolGLzcM+knnB9d4NzgZPlKYdhRzWvkb1mspizvLjAkCb6wTvW19cMznF+ecb5yZJ6Muftm9es728wpkJUJYUQdN5ze3tPURiGEFi7A8LbxJAKAqkNptAEZ9l3zbGRYhcpi4LoA13b0RwOEDxN2/PRRy+wQ8fQ9fRtw3xa48sCgeBhkxqkth0ojKJ3AtsPVMYQcQzOcbJasLo84Xq7ZlKsmMynTOcVRT2FvqesKnb7LW3XM1sukMpg+4a2HSi1opxMmLqAHyT32zXTesI/+bM/5avf/IYf3u04W02JixnTxZJQV/zq3/4HYgBdGPquoy4rbq7eowvDRBXcv73F1BW9svz63/2HJNGjZ3F2zu3VFauTEzabLfcPW86Wc8pCEUTB0NuUFNVajDG0h4bpYsbQNGijaPYNs+WCSAI5jNY46zBa0LQd72+vmEwmnD97gZWeohDYvkuNam5AtJJ5PUjwESE0MTpAZls/mSUWmeUWxyQIEnieCxEp0iBMKIPUBVIbhFRHVqcQcvQA/glrjbGLGGsqmQGP5NU09hwiT68RJDaNAGt7tNJZJpTkGyly3R+ZjKlGyYxUIRP4HMbGEHRZoEQk2MQ+8E2SzwkpsdamKNi+QwjB6uIM7x1Nu8+2ACkFxfcdShukUhRViRuGZNIugRiophUCQTVJ0tp2t8e7xGD1OYmGGFE5qc3a4eh3pYxOvg9GoUOKc5ZKYbSirgyFlhTGUxYVdmiQ9QpZGLTzzKoKvbykbRtss8MHQV0aJpMKPV8hygK3vYOqRg0WD8RSUSiFlhNi9KiE/CNkQBGSma8SaBnQRh59JUJIdVlKNRMZJCgYuuF4bU2hCE7mZKNIcA5TaFxMNU3b9tjBcn55mhilQ6orvHPpZ2SwUGmFdInpOfSWqi6IIqJN8phz3YCq6pzwkxiiMeaEHpGlmD6kQVj22go+D9xDTIxjoRip3EpKZFUdlQ4JWBQpUnw6RWuFVDOGrqc9HOjanhiSPNz5AA8PGKVZzJf4oafxA5PpPMcDpyyXKBSjaY3MtagaB9hilHMlufqjO8ijBPXYM4y1Icle9WniWgIP4xFAeSqfDiFm42ZPYv6mGiSMJuc5zUepDEg9kfP+fR7/OICQJ5PAD7ow+HA8nf88Isk8SYV4wv8X5MkgILVienpCf7ulrCu6dgO56CsKw7SeMK3KpIf0lnldsVou2DxsODtdMoTIYDtWesL84oS7u1vKokhSh6qiaxqkKQkIXO+5X99xOLQgIg+bNUPTMAwDk8WKoppgTMEwrFFG0QwNf/LJL3l/fceirpicnfKf/t2/59mLl4QYOTQ9P/v0Ex629wzOcvrqgq9/eM0vPvs53XzBffuAKBwfn3wMCCSavot8+ZdfIQrN57/4GQhBcziwu1vTPrRo0+GjQ5WpeYta54UNVdPw67/4jwQfePbsnHWzx3jHtFCcnMzZtwPv37ylmk84Oz2n71u26wOdi0l3RjIisi7ijGdSSKa1pl3f8+V2y+nlkrOPPibEmAxHD3tmkxnIlkPXEW3SRJrCUBqFEoJgLdO6Jh3AgmenZ0gF1w+3fPHHv+Tm/Q21cJycnvKv/49/je0sL1++QGnB5cmcxXJJf33D+bMFL1/9Afd3DXfX72m3W5SC2bTECUnXDhijWS2n7LaRbjGhO3QUBg4K1ncbnj+74MX5KW2TjKWMmqTYLqXQSnFSnVAWBdv9nmgDRkn6wTGpk6N9KBxVWaKkZDopeXh/w2Q1RfiIspaH62vOn11ghOb88oz7uw1VZVjMa7ZNw8XZnMPbe/aNZTozlIXE2sQIcTGiZFayCo6ymLGILwvBYB+ZVrkHOD6eLr2n28d4SDzpAz54/EQsd0bpw9PtAVJjJkLMEo7UZIcY0dqk9yVAWWiCd/RDB8EjlcZondkB7tHEUiaAzg6W6LPuUymmkwrtIoftHuuSH0SMnv3+wGA9TY6f81kS0XdtKloimKLMB4RASX2MsPXeJzlIBkrawyE9dwR4c/PsrMM6m+QnMQAJ+T8CIUJSFBqpVTJXC8m7JDn5a2KRDoh6NsV2A1LrFNWmVNKdZqPL8QZRWqFFjpaT8hhZNu6vo/1lyM161/XHRjt6z5AnXx8wNWKOfR7HBCIfaDEeD1Ylk4Y1meil7zcqx4oG8hSEY1M9gh8ya6yVGo20nlCUQ5r8hMDxPfPBE3xONvAhUWK1QitJURZURYkPkf2hYVKXGAEM/WNEnJTZtM7nwj3rsguTQJtQ4K1jsD1t1+UJYiqujVJQlhl0JU1TY+Tp+S2EwOVDPcSIcB6lFfiIFJ4hP2fU5KdrE3HRJbmAENihP1Kux6mgHMEYKZFJXZeK72PhGCirlM5TVxXDMODtkF3a03sfUCDjcTITicf4XGM0MqTpr9YmM3gyJ+O/okD5b/kYAcUPP/lkD/sAJHkEQ8ZC7fFVP8b1wSO7Zpy2WmextsN1W7r9lrZv+cvf/Gf6g2O/PyTjyZhiEfuuZzWZM1vMaDZbfvjt16yeXYKC/WGPU4rnJyeIKLjZPqQklCI1mtPplIMdmMwqdtsDGsvVu1s+/uhztIg8O7+gcz1X19d8+vJT1mGDCpayNNjeUk9mKC3ZNg2vPvuc0B6oSsN0NqVvDmitOb+8ZHfY0A47YlREZ3n17AWDt9y/ecdsOsMPDucdi+USHyzdZs9VE+ndW7pdy3x1xs39NZOqxlqHUIpt29Le3HDx7ILNfkvfdRhd4D3EsM9m0gnYjAp6m1mag2WZG/lRC+67gXafjKkLJfFPjGdDiMzrGh8ipyczdrtDLpaTN4MRUCq4220YmhbXDDy0e7qh5/bdPbv7HfPVkpvbBw5NQ9u0zBYLhryfV3XF7f0DYr1FSE29KPjij/6I+6sDv/3NX/KHf/LPebh6h1SS02dn7Hc9tYcv/uBniBi5ubrFGsF0voJK0jct623LsihZt1v6fqCsq7R/T+Yctg+U0wX3d3dU8wkMAw+bHbNJlUGiiAiRotB5UuqxXcvFy+fcvrtGasX9zR2LkwVDPyQQVydvpH5w/MWv/pZ/9b/+T2wf1ggZmM0mCEFizOa99nE6KxAoggBhxmQwfWzixzUX8p9PfYaIMYHn2qBMAkGk1Mc1d2SKADK9yv/f6///y2NkLqbp0rGbSxKu8AjAj6lZwScppA8eLeQxztwdGXP+GDE/yj5jjPlcfZT/j7UKOQnGOc+u7xj6LkWwFsUTSWuNMpEQLLuHe1yIVOUkSZhKQyFTYei9pW9avHOQTU0fpZ/J+NM7iypLXEjRvEppoD/WA0OfUgu1MSgSkOeNhuioDAjSGWt0SqsrakNVgS48ytR4JG7YU04nYCrwB2K/x/khM7Ql/cEhYks1nVKqgqA7KCWx1DApsP0ABMIgMHWJDo5gU3yurg0i5JhbUt2UeaAIQWLV5obdFJqy1MknhyTlkhmsQCTvlaLQFHXJ0PapdoiR/WZPWVXYwSUz4Q/a1lyrFIa+TYbu4dibKpQG21u8SCmSWmQz4WzArbWmy+zOEFSW0HqKOiXY6cIwro8QksQmhoCuiiSXKQy+LBJwojTtbsf8ZJUDQgpC8PRNy9D3EEO6fjYNAKezGUWWuAx9x2QyRYjsEQSPNQ+k9z/GzAR5KptJwR5pWDKOvTgOHEaGWMjTWjmereGRNQpPfMRiltiJxyFmAj+y71pmv0UxMjR5xAT+uwNCxs5tBDqeskFENkw6dna5CcqAyIdtUPpqJNF2lFRMT84Y9msEEjd43GDTza0kM2OotUKoFO30YnFKDJHOWp69uGQ6m6KGnvliQtPumYqEuvVElosFTkQG7zFas5zPubm9Ybvb4HqH9UNCCpXEh0BdVRQlbDfbRJPuIh9fvgBT8uLVS653O5q2Y3p+yWRWE33g9OyUQ79PzViEv/nmNX/46acQHV54Vucfcf5syelyge9S/OHV22tmJ0vOzy+4fn9Fs9+mBqrr6A8d5WyKRBOFYGj2bB4avv/uHRdnC85Xc/7l//JP+fbrdwQvuH/3lqFruXz5PE2afaTUCrvuuNEd1aSkKBWDS3r+KAWFNlRaZSpcRJoCvGc5neI6S9sHbu7fUUjD6XTJD9u3kH0RIpKqNmgpcNYyBMFkPiGKiHKOCs/722t8cBRVxetvvqWuKnaD5/1vfsfLzz9hPq+5/vYtl2fnDDHysN2wmM5Y/7Bhu2mZGMG8qmn3O1RVMziPazqiTYeDJjEKFrM5SEV3aJhVFdtDww/fveZP/uQPmceEru/3DZUW+AiL1TKZmUXYti1KGopC4QeHHRzeWtabDcvFgq7t+OiTl3z31Xe8mtYYo5ktlxwOBzbbPcvZnLqoOLs85/Z2zWxaEoOn0BpTaPrW0rSOyVTRHGJOiICqkPQuorPzuvXjn8kNvDaCzqYDIfXcAj8mWMCRvib4EevjCED+Hc3ETzSqGWNv00sY1fnpRJK5sR71goKI9zabkQmESiZehBwPJxL9FuHxGQGXQlFVdWInZAqsi6TraRyORFklwtBb+q6htw4fApPpFDt0xGAzcDGm/Iw61Hzoav1oVOo93gqkliSxK/lwFkcKaypKIkPbpXz75ARNUSZjS2N0mszHHLlXVkkKoRXBDsnfJL8vUSTNd2JRaGxuNMqyJgaHG0LyTBknWjI1HiGkhnl0+Y5j6TZqiTPwJIKjH4Y09RPkxnFEp56Y2D4pCMfrKkiT3zHqL0aZvVAiKEnf+yNCJ8jmesdmlDyh4AiEueCO9woZwCEm2qpUKt8HaRphR1f+zMDoO0tZGEpjaA4NulMURXHUyGpjjq7piSAWE0iVr5VSEqVLUtKAoGub1KBFksZaJMCnLAqUdPlrMQN4I8D0WExYl7xNhE5TWecD2qdrnN4jiXUO55OnR1kWdF2fmDo58FjKPJmMEW0SJTdFAcojjTUALkd+a60xZUm33VIXRY7FTfdRyEXH2PqkhAPS9NK54wYSM5NC/JRskP+HzxwZY6TdYpxEAR8UZ4wAYZ5SJQZTyAwmjXXJ8d8OHWHYs1/f0TvLV998Tdu07Pse7302lJXsuo7nlxdoAbevr1icryhngVVd8jd/8zvqxZwiSkTy/wAAIABJREFURryH+/2Gptkzm83pupZJWaJ0wW6zJuaEg/XNPb/47DP22z2iLrnbHei7novFBXeHDdOqIsREeZ6fnxN9i+0sn//8M3YP98fUjsPmgUopbm+v6e1Aa132LZBcPH/F7rBBlCXWB95fv+PzL36BDoHNzTXWe9b3Gy4vLjn9+DO2N2+oJwXTsOTh/g5TJdPx6bLi9bcDfdMjlWboBvRUo42madP+ElySEo5myWkfcwTvkKZIe5JNTf9+t0caRZnvVyUVskqF+Waz57DbH82ng1KUhTwajq+3W5bLBVIJilohOgGF4PmrU1aX5wz7A6vzz3nz9j037294+dEFr6On2+9RhaaYlMxmJUMX6Hcdf/mrv2E6mXCxPOP17/6aZxfP6PYdV/sty8mEUtXsd3tMVVNPZ8yWC7QwvLu54+7+PVJqzl99xPqw4+zyBT989x1SQLfvWZ7MUfbA4uSE+7s7ZvMJIbNiJiLLrkPESMHQO6qioG9bNvdrTF3R7fa8+vgjXv/whul8koBZO1AoRW89h32DjIHN9TWq0iyWC/AeXWhsn6RGUuWpOQohI8I/enDJDKKOTUuMCSsNI5gr0llCSM2S0AahDVHpZDWcm8CxEQohx6L+RAbLGZZ4/CD/opE0tUdwnOR773P/kn5nZGJ8JcP09N7IMQY3puZuPMhCZoEcWQGAUKBFukfbrqXPzIyYB4RJUiqopjW73Z6H99dAMsMMIdJ3HSFGZssVfdPS7HaJfcBjsl4E/GDRVRoEuW5AaJ0Gm1KwOFuyu9/Qdx1924MQVHVFXZeImBgh+8HiA0hlkH6UUanUsAbwLqKLCokl9B1GKIQVYFuIkdoUqBgZnEf7jhgkUmkmAozUYATTScV8ktJZotsTRYHH0xGZSoHTmiEE3GDBpPtSBpmkPDEeweooyMl0YAeLlBKtH0EtrVO94p1PjInBURsNZUHfdBgt6boBU5boQuNcoCwMw2DTWa0SM8GUBSEUhNDjXaCsDELkQYSS2YNEYcoqpU9mY+Dxvk9HUjIMDiESrAOSHKrvk4fXaJIbQqDbHqhmdR52xJTIEwIhSvabDdPFHGUUoXFUswluGBJLicRof7i/ZzafcXF2Rl2WNF2HVYLJbJ6ZGOlOVUonLxgSCzBKmavvJ9KYDKZIMQ4M0loQcWRv5FqAJ3kymdgwgjxPpTiPeEY6ixMB5THJjSMoG3+0YP/+j38cQMi4i47ghnj8wrFGHgvb+PQDeKrtHZ+YpoOKojZ06zsKU3L+yc+5+v63aSMJIErNcjHDygh+4OXZc4SGoCTPT08JAtabTYpq9I5mf6BQgrI0+ABaau5v79AyFet363s26zVaSNoYGHYN+/2ek/NzVucX+CwB6foeJSSVkrTtQHd1x6bZUOkJsTRMpABlOL1Y8u71G9pDy3Qyoy5qSm14/fo7quWSs3rKTArWf/Oafa158fHHvL+9o5wkDe63X/4WQaS3DqUl03nNR5+84u5hy/rmDncYkPOamQy8XC7527/+La474+LZMz7/2Sf01nP+7IS//L/+M99/f8Xp+YoXry7RXiSZRWOxg2fbpwWttUQLlanZka4dKOclZVXTH/Z8f/WGzz59iW9bTmZz3uzXNIcW75N3QKEMxbwCERms4/Tsgros2Oy2LEyNLwPfPdwlWqx1uOGOqiopMvhipOTLv/6SSVXwh1/8HFWXfP37bzhZLOn7gaIqWNqaXgqifmAxmXP77orZaoYIPXUhaYdU1J7MajabBlnXWKNpDy0iVgzdwDfffc+nn3wKISYvFOcppKQ5NISY9KGVSiZGRIXSms5a5tMJbkjGSboo8M5y9uKch5t7Vs/OODtZEVyiQK4f1ixWJ1ycnYAT7JrkZaNMgSoMtBZvI21MiL5P3SnWx6MxovMRl3uOGMHGRCaTT/ar44HIh4AHPIIi43PHz/H4lGMz8NM+xBMmwWgGCTHnvqb4vjxRCgFEagRijCDV0aAqMS4E1iYjuNlkCkox2B4hJYN1KF1SKJ1M55QAG+lsjyeCUBSFprcDSkaarqFpuzzhF5jCoLWh7drUKIeAHQaEkEQBpjAE54kuHk1CI2DKkqHr8yQ/TzNIBmJmNkPESF1WNG3L0Lao6TQf8EnO44aBEEyeqo2xeIneevxZNj3f2YFoIkImoNQNQ2rgpU732DixymyZ+OT9lyoBPSKmiLkIyX8EjjKLUZYRxuvFeNinwzTGJ54eEXxM7ASVr23Ik7NISsSRIQEKIlOpZfZ4GWlPySnfZ8DPJSaEGoGZ9Lzok5nsGKfucuEagkd4SVBpwtcKqIoSrSS9TUZqWhsMac0rqZJBYC4aVEwGXoPr8CFgtMkHf4W1lmGwWJ8Ko9FkWuZJ1WjsKuE4ScmoTmo0vMdnaEdnijQxSVeklBhjCEcasEbq5Jh/lDzJbCqcJ5QjDX1M7iIDASF4ZAbltEpNqh8GZFHkfQailIndktknQ5+8dYIPKFNC8ETvElCe025+oiTM4/3+Yyw37XWRMS/8CIkca49HCW4qULMpLgJpZDYETvtM0zXYoaHf72kGx9u3b7m6WfPufkvXtSil6YOnKBVzXRPtQL1aUk9ruvWO88szNoc9p88vub265Q9+9gt+uH5Hs12jdIkNluVsigww+JZ2t2U2W2KD5/nlJYdDy/R8xe7uDucjZ/NT3t2+ZT5Z4pQk2o7L82e0uy26rnj52SvWN9cMneX0dJrikpXmanNN0/cQBVVRsTw/p7UN+/0arSuci/zin/4xzXbN9vY6J2LFlJryR7/k9fsr1r/5NV3nmc4WVIXh8vw5D3fX3F6lPbXUFaDwnYUAD/e7JCvIrCMfkswoOEtvHXVVIkjbtRss3qf1rYJCaIXtB7SUuL6nFoKmHSgrTT2viCQQcWjaJBcMoJVAC8W0mvC733+DjHCxOuf923esbwNDc2DzkIZoURXc393i+oF23+JdAlLbpsX2jkY7epcScubTCRhBKBxGzXh9c8XJ6oTV2QngscJj6hkX5+d8/dW3zPSUzg5MVOT5n/4pr7/+lvd315xcPGN/OLBYzNlttyAit9d3zJcLLpeG2+vA/d2Gk/MFdW1oty0mBFbTBTd3Nwht0rniPX3bUtUVuppgTPKf2m8PPH91TugS2J6A58BXX33HP/uzP0Jpgc+x40obdCSlm8gUM458AoBnwDAiUJl9F8bJNhGlUkOXcMQ8rFSKmD1BiMlEW8TkQvUhq+TRIPsf+vF0zJpo/yInzMXjZDscmzf1mHQj0zlO8I+JXSOrLI4gkXhMOhPq0d8sks89hXOObbOltz1CJp8RYzR93yafL615uLvlsNsTAolFURQ07T4BEtbTNw0u+OO+PoIgUqkjM5IQcnoMxMGmn/v+lvurW1RmqkqT5OBCxKOEx7kkdVQx0otIqbLhrVJIbZJvjxdJYjsEdFEgZWJ3+yGdl+ljmEiJEpouA8rz0wV+aLG+YrKYMlvWuHaPdQY5mcHGUXnJUGp6H+jdhK7ZJ1aLEHgCeAghe8fl3zv53PDoeyMFWiRPFGJMTHTywCdEbDdQzSYYrRm6dJ4fdgfmy0UCYJVCSocb0rUOzuOlT0OpIuCsw9kUcFGUJdZ5rO2wfU89m6F8SuYbmbhKKfxoiqolobeZWZKYJiltUOX16rOEBWzbUZ6d0hwaiqrCbjagEhjX7vYUdYm3/hg0gkqDrkIrvItc/fAD0zrZOBiTWJzD4ChLnRnIySBXiCRxTWdgep/kWN8BicmVhjIh+74IkSaUQoicfji+/+J4vx/TA8kDBnls+IE8FCJJgcYyL+bvHWkRx0Qs+O9PGiMycgTj/BbGDuyxYRsND2FkiYxd3VMGiSoLdGF4/sk/4/aH3/DzLz5m8eqCv/q3/4nCGMr5jOVkgiTSdi2nZysuT5esNwcW1YI/+Nln1Kbg+v6GGCztvmE2ragKg/+/qXuvJsuuLL/vt82x16WpLIcCCkB3j+EMhxyaoBRSyMW8MPTx9KgPohe9SAwpGFSQ0kzMdDcaDTTQKJc+rztuOz2sfW8WRgxpHihieBBdyL5VqMx77tl7r/VffxM8ddPgfWJyHW7sZLG7gPdQWoNWlvfvL5kt5rRLS11VKGvphyHr3h2mUKRiwdmnz7l6c82nLz5h3pTcXN2w/OQp795e8f73P3DxyTM+f3JCGRX7EHlzs+PF0yc8uViix8SsTXDRcrWHX7//gbN6BtOezfqBpiwom4axdxRlyzB5vvnVr5nPW37289e4KeCqgt9/8w3DNPHzP/ySEBwf3nyPSho3OorC8uXrT3nY7jh9+Snv7j7w7rff8Q//5I+IauSHd1c0M4nMrasaGxPnT87YdQPERL1a0UdBHb/49BWqKNmvR5SaxJjHeWlGUTTzBbthxCjFcl4xOce+H5gtlny4u2W33YohUnZlNqbAe5lsxwTeWObzJVVb8Vd/8zVFSrTzkpuHB6qd5Wy1YBP3GB948eKCdzc/oF0k7Dp0VTJbzSF49g8bFqdLPv+jz7h+f81mtxdkfiwYSsvUD8xmM7a7HTpKDJYtC6ZhoHvYMj9ZsljNqTrF9e0DbVOjNbiyoj1ZYJVmnCQvfF63hNGxv72n0IrT0zPe/PCGT1694IcfPjBbLlg+O4NNz5/+8c/56us3nCxmDN3I6AOH7PoYZfMeXTqyIXK/g4uQTb8lFSJvDiHBRySrY9rP/3NtHkCTx43n78MlDY38moe0eXIosbUH6v9hunQoOD42OHTeY00tqDvpOFFv2hZTWpEDhEd649R32OVSDtGUcHgeHtbU9YKyrJlwjLuO2+vtozGZzRnxifz9zBE1F5AgEkMUo9HCHA8FpbVQQGNEG0v0gaothc6sNSEldus1VVkwWCP70uTww5CBtiBshKpEa6FYhqMBWzzG+emM1IdpQpFY39+wWiyoikIkFUoRs/nd49QciaMLk0iOEEOxopQ4PNEfa/qhl2Y8ZPlAplaGKODBYdp+8OwQg7PsexIDqJT1swIMhFxcG5MNlrWWZzvrtg/ngMqASCTkr7OWNU8cD9KQpA8RiGIcBxwNZw+/huBJQRr+KY24fPYYW0DqqOuGuq4ojCYVhqIosYdzCdEAV7qUhkJLEV0UCKjpHNM0ffTehZ6sgheztANYBJklcqCG5OlqyAGCCnTUjM5jjQAhbdsCSmRa1jKGQAqBsiwyCBKOn/1xOpjS0XBcH8An7/CAspayrPFuIkwj1hakDDbFlPApUbcNODHSjRm0UUWBd04mWhns0db8/74//HuvQ/lwZIKkI/Dx6F5/8BaTP/ljs9T0mHgUPS4lTNLCMggeqxJdtyaOHck7vv7Vr7m+3/D9m/ckIvO6YXCOQhXYouR8scI0NTc3N5zN57Szmvv7NbtdhzKG58+f8u2HH5jGifMnT9nsNlS6oKwbIp5x2/H82YVQo4NiuVjxm2++4mQaqWpDbQxTGLFVjSfx5GSF1orRjaxePmNzu+HDt9+zWM5ZrWq0D9xuttze3tG7gaqes5jP2Hd77u5uKZuacfK8OJtzeXnFX/6rf8XJ6UoAgfWe2emK7777huV8ySefv2Ty8OKsIVrF27eXLFcz5qsZKcHN9S22KmhLQ8JSz58xjmKMWrUlhIifxMC6Gxy1Fs2+THblQGrnNX4yaCS+PITENE44H2iqmlldEfBsH3ZSS/pE1dZoAskF7LICBe2s5eRsxeAcZl7hE/TbNQaNCiJDMDbSthV9P7LfSfNjjDBGY4igI24c8QkG7+h9YHu/5enZCW4/8WH4QFnXVEZx9se/4Pbyjp0xLM7mlEnhesfi/Bn7/cjoHfPTE/rNLWM/0SwWjONAUzbMz1fcfbij73tmixlVabm8vmdbaHSE5+dzhmHLk2fPuFuvcf0gcb9liRtHQkxs1neUdYW1mst3N5ydzEkIUzIl+N0Plzx9fsEXP3tFl+tAdZjwmoP/k0R1orLR82GSrWT9qxjRUXyghBEiQyaMRaVI0ImkNDHlAiMmdD6blBXYPvEY3Z3iT7RnHJd+dgo5DG1RWVYZJFJdG7kH+f1LVGjMPijCDjxEmR6nT4e/Co5paFqbI4AyupFdtyUlRVnUoDRuHIiIsfd+uxcGHiK5LZc1fnJ0+x3TOGCrimpeMw4jU9/lc1kdv9fkvEj8lzNhWk2OwoqfnK0tq/KMru/ksywsddtgtJFEomyUGXyAIEOnlEb6oKhNQvmJmKJEX0+aMDkxYTUWRaJSInnVZYVFYcua2iS8TyzaCp8iT/TINgZKG7H7rQQzVDM+Xc0pa3BtiUuJu15i6ze7Pc5UhMkQYhTgM4hxu5vEODzlWllrTToMBaPUzaSIHwNVVaBLwzC44yPQ7fYURYGtKnAiSZqCZ76Y0w9TvjeTpMuUFmWkXinKQlg3IZKiZ3kyx20cysj52m932LLIKTbS0JuiQDlPDFHA2kIkwtMUZBChIDh3lNyEEIRt7D3DZg9K0c5mpJgY+z1aa/b7Du8c58+e4EfHdrvFu2y2X1UslzP80HP5/gOrJ09YFJaUve5CTLTtXJKMYhBvMK1JSerqmNLR5Ndam6VfMYMkCa3FZDnmfl7nNSSdfh6wHc9ZAQ0PSW2PwzapudHqMOeSek5BQkAZPvobP3ak+7tcfy+AkANKCR+hOB81XIfC5TEVJt+QDJCYwhBG/6iVLxo+/4v/AvM/r6GxfPOXf0NTG0pT8bM/+Dl311f02x1nbQPes+kGnj09548+f8WiXfL26gPdfsf5k5XowrXiYbNFZ2M9RRRnd+eJCsqyxhj4/Q/fo6uKoq5pqgpbCm1+2vcMY49BELxqccI/+tOf8+H6lj/8g89J08T9+g6dDL/97bc0VcWT5+c8P18Rk+Jqs8UE+Gd/8jO2d2uu3zxwcXHOv/3r70HisymiZtusuXr/np//7HNso7Ex8vT5mZTku47Z+Tm7/cTXX/+AT4q6banLku1mh2kqdt1EVJpKAzqye1ij6xZrLHG74ZmtmH/xGb/81Tc8f3HBF1++YDd4mCZenJ2C1nTDRIjQjY7ZvudktWK0htuHNa+evGCN58OHGyyO2ckS3Q1SwIRIXVZUhRWtYohMbuL69gatYLFcUhaa7Xon7vFESltQGcPgJsLkJUO6nyjLgrNnZ+yHPTNrefXpC2yASiWev3jC9dU9n336Ap8U6/stdaXxuy1FXfD8k6e8fXPN9f2Wtm2ZOjEK3O96iqLg4X7LMI40dYn2Dj9M7LZ75nMpUnYPO6qmYnmywDvPbt+jrebm+o7TlHh6cSa0x7omJSiKiqgmdus9s+WS9brn2TPHfD7j8s1bytWCpl6ymNc0s4rX7QvuH7bSfHxk1HVobJWS58FqqAvRHzoPg0u4bKyagfAfMcjC4z6Uzf6OS/NHy/FvT1N/qutYUHCEQqQpyw2tslaKNS9gA1E9Njk5KaVEYU0BKYi3DZG6nVFUpRh4DYNIF4oSXVqM1gz7PV2/Z5xGmvmSp+dPiBGubq+4v78X49qqZD5rRQKXpUdxckfa8IG5cNjzDv9fa3VkABTWCoDiHTH4I1OhairpA1Ik5YZ5t9uz0JoqZ8C7YaSoSomGi0lMCUeZ7A3Z3yJGkaMpq46GrBGyN1FPO59ji1IYI9MEKNwoufcqT/hSikyD+OVM40hTSIxsiiIxCj48UiPlgxL3cyXMjEOBEHMqy0Fwm5SWCNyUMJUBcygu5XkXgIvH4hRJb3rMoc8TNSMFhA8iOTFafwR0CDtGpETCNAkhUmbjUbTG+4DVJrMAFDazPQ5+JoeJxzhOEiNuNXUdsc7n2FgBH0mJpm1l769KnPeZxSGSq+A9kwuQv3XM9yzEg/8JjxHXZFkKAgCrGBkz60VYJFJwFPkel2VB5uWIxEofIndFz05mQsTsn2K1MGNCZm+EJLpqjfys2JwqE6MUkoiMLKXE6Fw2WpV0HJ8S2mZ5UI5VPJ7fP9mV9eIfMz7kZQApqOQDfwTX8nUo3Q7yp+gcSSGsBJUYuh1jv2dyIzdX77i9ueOvvv6WFBMugFci9fji2TlV1fBh/YBbb/n8i0+5ef+B+WzGdhholy1+jEw+MPYDZVmxHXtm8yXPVnPe39ww9D2rxQKC+AGoquY33/2Os/Nz3DigguF+6inKhqfnJ7z85CXruw3WwFgmtnd3NHXD/GTFvG25vb7l6vaGh/UtPkTmiyXnyxlTgqqpqds5bpQ41Tff/17WbFlz+7Dj7HSFXVYM+w1/9ge/4OFuy/vfXaFLw7vNjrKwtLOau3VHCF6GGhrSYLn3D5iqoqprtDX0/Z7ddodSwtLqFTRVwWbdUZRGTIsVFBqm3gnFPE/aDdA7kR6lFBi9yB9P5i2jc4yDY1YV4BWLtsSkSHSB++2OqCObzZ6SRNU0JAW7bkAbTZtlsC43/DFCUxYMXWZrlpZ+cKjC0DY1sZuoKs3QOXTV8OLlc95//xbcBOWC7377lnq+4O3NGkh89uo1067nzZvvWS5XfP7FS3771fe8fP2KeH8PMbGYzUhKsbl6wBjF5BPaBjb7iUVTs+96ylnN76+2vHy2ZLd54Hx1yna9ZtcPRCeMAFUURBLWGspCJEn7bpQGRgnYsd4P7Pc94+Co21bMmX3MHlrkuFsyaK8lNSNGEiaz/VLeyqUJOgxSDmPcRHYSOBb9MlFWhxoG2fBELZ99YILnp7oOZ5OwOSSBJYQspTbHTftoECv/0SGp63G0Cx8zTEROLgz1cCDUZIWAlqSxoQcFVVmhjRWT9nFgc3uHtoaqqY/nSsiDnX6/OxpPFip7sBgnPguZ2ROnCW1FWuEmRxhHogvCCiotTTtHF5bzkxM5N1Vi1/d4H1BomroiOjE31krhMsgVsqtt13t0AmsyMyNqJjeK3A2FMoam1JRFSfATYJhXFozFFgFdWealRlnDE6sxLEhoht7h9mvMzBIizBYNNsHzuqXrFavGsH4gs/gj/SjRvj4mvNH0A4yjNPbeB4oqp4iqRymGMcK4rKqSWCYJXxinHMPt5YhWCu88YbunOFtRV6UY0NYl+32fz7hAkWNtrTVkgjLDOEmct1LHGGOVB1vGiD/JLEtXnPcyIMm11mEgEqOwkA5siBAiqlBgNLudxJMrJcSMsjpE6Eq9NQwThdUsFjPGYcI7zziMOBdYnCxE2nj/gPOepmpo2hn7/Q7vA4vFEqsV4pHF4/OtDsakj4xLYiKkePw5YwaPbGGPxvuPFMuU5XR5qJMyQzVmqVn+jEAfBxHC/IgiDUo53ShxZMr8yKD573D9vQBCDszfj0HSQ1zn0WApfUy9ya/LjkoYvSBsVhGdx20e+Df/4/9AZSV5oKpKLp6fsbQVw80NQ9+h8Fx8+Rm72wdev3jJ51++wiR4v37g+v6Wk3mNVTClQDd4cfauK5TWDOMekyIueIJXrOYNu3Fg13Usyppm1tJPnrpt6ZxDaUVTNdiy5PLmlj//45/jfOB8tRLzqvWW65sHgpp4cXrC/GxOYw2u7/j+hw8YAk+eXvDd794yDpJC8qtvNgzdnuVyzrPVHBU119uR+fkp603HvK3wcYRgQRu6BGXVsL/aUMxqvHfcXl+iVGS2OOW33/2exXyG7yX/u1aG5WrB3f2apm14WN+TdIkyBS9fv+Ly3SUvn5yxKkbudonttoeiBCXN5fmq4eWzp7h+ZNxuqedzNmPPw4OAOLeX71AusjxZ0nUDpRe/lWEQWpa1nhg1y1krTcrk2O49upJHdm4KnEp4NMvZnOAd3ThgnESJfv+7D/zzP/uSskj0b24wdYN9dc7vbztev3hKt96hU6KoNWH07PuOZbXCj56nZ0s+XN5DMeG9NK6fvnrKdjPQzzr+r3/9b/mn//mf065O8G4kDXIATv0kfjL7jocHJyj6rGG/3VOXlvvLG5ZzSR1RZYkFzGTRVhbu5n7DP/onf8q333zL5198zhhGdtuOamHZjyNlXbGsGtafv+S737+n70ZciKxmBZvOYazoH4tCEUJiDFIwFgX4BNOhlkiHNfTR13ndZWzlqPn/f7t+SkDkkBr1SFtXj41jRpgTGqxFZXZG5LGpI0bKqpEo3HEiqkdZ0SGpwRaFZM5PDlNbOUBjYJq8sByiAmX48HDJfuiJCMOgrkrRbWZQ5sBqO+z9YqrmCUHegzSnMp44MHC8c9iyQBelyKJ0oN93zBcz0YeGKIAOQlPs9nu8y7FoSjF2nm67QWlDdUi2QWLUgJyUIlKQybmsd5YmexhHIpImorWhyM2vUuAGiXtLRaQsLNpK4xumiaHvOMgPJKI8ZGBNHz8zk8EoObiyTEarLNuS6cLU98L6Ojj0x49ZC5EYQ9b0iuzob4NzB7q20QeQJ2GNzpIbYSpIM6szsC1Gq1EHFvNWDtgowHs3iau8RoufSALnXPbxO0z5stEMieilOZapiUIjXgSSuiDMkoOviLaG0cnPV2mTUxwMzitxyI+i39X5cD8oaj2POQYhNw/jKMwSlcQrxIWEivJcWW1IVcmUokzhtLwXpyRN48DITAEoVKZNa4IPHHTkxARWHZsl4DHGOO8fcZoyTdjgvcijdAwUZYUpSpHGeHF9/yku9dE0V5oajg/OkXV6GNh+xAyJ6QCcyP2XAi3l1B1DjB4XBqZ+j9KGmw9v+eu//A1fXX1g3HXUTSlR6z7w8uIp3pQkn2iMpmxK1DQxXyzY7Pf4yWHqBj1ruLu9YTZrSTHgfWB11nKz3jK5iZPVQrTuSZPKiof1HU/PT4kxYouK9eCYz1vOnjyh8SP9dkvUnuu7DVVZ0rQNq/mcrht4++Yt6wdhGRTNjGQii9mc3ejZdD2VscyX0rSUwRMKzXa7Y9U2WA13H65Yna1Ibctv3nygqRp0odnud7JeXWJzP1HXFcMw0MznxOApraXvIY4TTkNVVVglE9BpmtjvHXX2IGgyQ2/K4F3TtDSzGluKdDVOjqqphdlmNFe3G2bzBjebzjBZAAAgAElEQVQ6TpatGJrbgCWBhoftngtrsU0j0YtT4OWzZ+z6TsBTLww6pRTT5EAJY6soC1CRh/sdASisJWpF3RaM2xEVNHa5ZP2wZbVYcnt7L2DWrGW/2WG85/knF9zfPeC7Dl1W/PI3f8NsNuNstUJbxbs3N5w/e8b1u0tMJSw7ZaSGVMZyf7/BKEffeZqqAGMom5ph9DRtybsPDywXNe7mmpPTU7CWoRtIKVIk8RSrapFcFmWJcw5rLeM0UVqL8xE3jPTdwHLZghF/PfH2iCgxwMiT23wdGJfpMLjMSSOG4yRYYrPzf3EEDSIqicG5+sjw8EDfV+kAiPx04GlK6UcR4IfkK5PTRw7zZ3JDmD5O08jns0S5piOQktBHJsLBF4s83faTJ0Uvg50Q0Y0Ytk/TQL/fY6ylKASUDlHOfm2sSFu9R9cVRVFgskxs6HqAo+QsRvlcUpQkIGH6RKKDxWJJ287YZE+dEAPdwXMqBjHTBlkzCrx3whDJ5uLHXi6KB5pBUVeVpA95h8NgrWJWF5QZjMOLv11hLCWaplpQYFnNXmB9pLRbdF3hg8PHyBTEL2N5ukArTbfZcrGYsdt3VCT2+46uGzBYxgA+RkISk/x+cowu0HUC/hSVOe75B1VCQuEmx2zeooxmzGkxs1kDWuR0Xot0eHe/5vTsBKIwT5q2Zr/vRL4RsrF5IXvE6DzDfhTmcQY+nJMBV3AerSUdq6hKbFUy7nuiTVibpTBa4Zx4s+lcuwgrRMlzZOXvG/b7YzJTUYhptEixHU30jL3IdrTR4AP9dot3gWG7pWobrDVApDCWaRopYpT362uULWTgkoGJY72ndD779fFZll5CHf8RA/uPTE/TERKU81Q9+oPkbYDjrwkBDI0M4oRhpo84gDrcg/zfPp7Xf7fr7wUQcujIHs0PH1//EY2VQzGcHguYKF8rBTFv1jEE4pTwSVNoTXSBRTGjx3N5/Z7lfMk/+PN/zLQf+Bd//g+wRrO933J99YHRey5OF7SNuDLvdlKcKCQDepo83a5jNWtQWlNXNcM0cnl5Td3OWC4XaGu4vbmj7/YMw4Dve6r5iuvLKy7OnzDtA6bV7LcTP7z5JWM/UpYKTIGpDYUuRFf87h1WwXyx4O2bS8bJ05yeYhBpybPzBbd3HQ87jykNy9mcL1+9IEWHGwM+wbffXpKMULLVw8jq/JRi3lLtd7z8/DOG/Ybz8+esnq3YfLgitCVhHCQCdBhpioLtrqOPgaYKnJyeUmjN/OkZv/zLX/H5l69R2tC5ERsDrz95wfvre07PL7i6vuXy3Tv+4Z//Y1arJde3N5TGMK8q+rpBacP7dx9IEdrliuChMAXeO+JhGpsR96IpaFYt67s9p7M5XoMeA6fLOXd3D6z7nrLQ6DBSz2a8fvWUv/n1W86enPDP/+kfE+aG/+1/+jf8yZefs185ZosZyhqufn1NfTInhJL3b66YnOflZy/AFnT7gdPljPW+4/6hY7WcodQZv/3qW375V7/iT/7hHzJra2aVYRwm0cGrQmJZs6dCW5VU1nJ/sybGwJvv3/HisxfM2kZ8RqaSzcMaH6HRoHKqxG63pbAVu+01p2eOSsO8UEzTwJevnnNzfU+/H4XWnxKFVRm0kQbXZFq9kyAIrBZD4JSEMaJTrkVANq10aBF+zBT520NT9bd+76crTz764eFxI81eDcRDvFuZDZ6ya3uS5tUoRZhGgktCN02CnqeUGIaBwlqKskYbR3AeP/TgPCrBxZMnxBDZ7Dsu33xHPznq2QxbiB57Pw4MnaDwxso8PoRIUVaS0HEwmTxM+D96TwcqcER08MZalBL6dt/3DP2AtpIIUBSS3y7SHhjHgRBDjluTDPi2Kun7Xpzkm5a2KhnGKUtP5DBXKJlqW4nl9CHkwkdMC7WW9Bpr8r0FysIyb1qMNdLAmILo3aMUyYh3RcyTA5XIaS8hTzoeddZKK3mfWku8Z0xUVSkFVFEQYxAHfGUoywLvJiIxp0ZkpiA5OCxPHEXSJBKNwlisEUM0rbQwZUIQD4Js7lkZy3zeSjNQlszamhQDVSd7pwuRoCQJJ2SWkcpRvkGsWwgBxkkMGdu6FvaL0dRFoClFr++dozCG2haEaKiMZh8jY4goiix7ihgTGNUkAEVMElMIkMGHGKWgFrNYWQNTjq6VpKCCwhZUpTBemqJA58+TFDGmQJcl4zjkYkL2Kz/JPmayzpvs1SD0dDGDU14dVuARVohJwC2JT7RYlQGpXDilfO+VOSQk/DSX0goVf8zuOPjVCDkm5SnMAdx6NHc7RPYdDB3TYeyGwvUDo4/c393RT/D11Qf2t2tms4a7bqCuKprCsjpd4MZAVUZubgc+/+QZ3TigtWK/3XN2foYqCq4+XDKbNQzO09Yly1Y8WLp+m/d3S9uWDD7Q396CMnRdxxQjT87OePlqxcPVPU8Wc+63ng8frlnOSj59foqbICnN99/9wOQc692WaRoxZYVH8fqz1+y7jt3omC8WnFeW67dXqJnBqcB+N7LIIIPJEeR3D1swMAyO3XaPrUqIkbYqhCpeGqqyQRcF0zDRNA0KmLWGGIX1tnnY0TQzJu84qS3rbS+U9giqimiT5QgxwdBjbUkKgaqygGI+bwjbiNIiaVNRQM9+mChqQxoSTWm5vtthUwKjGYaBuiiwxnJzt+Zqs6euNE1VEmLA+YA2hqq2hDFwd7eh7CfG4FAYzi+ecPnuA05rTFMw+olVWfHZizM2my1NYzGzhtv37wGYL5fsdx1jN7AfR1Lw+P2OiyfPuLu7Zbdbs1iu2A0jJycL3r2/ZvlkRbWYweRZzuekFHm4X6OJ7LqRqi6OtfM4OOrSEkIiWUU3djRFhapL1usdwXnq5ZKh72lakVO6PK09wBTaKIbJ8fCwZjGrWJ6dEIwkkkUvzUfS0ujLd30cQKCNgL5JgxE/Pq0gmrymDjKKeEjy+KiKOK7FRxmKUo/+ST/FFeMhxSs3ZHlybY0lpgMILpGfMkg6+H8kUpJYVZ2jPQ9sEe8lOUkSzkBpMVEPwWfWe8Rm/7JEZhFMI2Uzw+olQUW2D2v8tMVNLgM0k5zVxoifUy3mszGIuTCIEWgi5dSkRx+X3WaPMdKEl3VNQKOt5fb+gWEYcrMbxQvQGKauJ0yOorQi+UjCZj34bqSUqGuLcwGjQEVLbWbM0JR1xawtWNbiO9EUJaXReA9N1ZCCxw0TKvQU2x/QOkfLpp5ZWzG4QF0aYgwsC4stDEs7RxnN3CqWpaYPKz5cXrPZ9sRuxGqDC3LG6aqiLKU+niZHijIkOTqgHZ5BlUje0zYVhsQ4eenTmgpttQxpstfbbrujbiqIiaI0zGiIIUl9oTXGKMYo3jdHxlBKlHWZpWQCCEbnSUjtVhQFI70MIzJYAwKmF1XF5Dwp6sykEFDelpa+G/DeU9U1y9WS4LwEY4yDsEzGSVIEo7DbDhHJykT6cSTlWsxYQz2bsGVJU9TChg2OoPKAR0t6DekQvnAwtM9lbVJE9RHbxmaoIf37e4ajQeqPhhNwkAvJ/qQIUeQ58jxL42K0zt5qGR/I9eZ/ekDIoSA+3KIjNeTHN+ao0csIUMpGqVormeDlgkbyvQOFEkfg5XLG+8srdAq8+vQ1f/KzLygmR/fkKZMDnRRlaziZt5RtSTtvePfdG6pa4iRt1pF754SaicLHiHey4d2v1/gQqeqWKSjqyrA8WbJbbymNwbQzrm+vOTs54eWLp+xiz7f/7jtOn8zp1muapkZFmLcl0Qe+//4HHu4fiDEyW8yFijmbU84VlQuo2uCi4uFhQtcVcXRYF6h14v7dNc3JEl83TNuO9mzJfDmDcaKoT4lG0e97mvmKzfWaMIzchVvG/Za6aRgxaKXZdQPbbUfQBo+SKZgfufzwnvPTM5KyvP75p/z1L3/LH37xGfO2oa5rhnFgMW+4vb7E7Xecnqxww8gm3JH2E4t2xhQi+2HCakEsUQWNKXlyscL1HaP37MqKrtujc5RlTAk3JuYnyxxfG1jMGu5uN1x1PdE5Sl3SzFom73jz7pZoNN+/v2L81yOttZyt5uyD49vfveHlxQmhH/ni5695c3VH6QKvvvyED9f3vP3hii+//IQP17eyqZsCguPmfsO8Krh4csr29o7vvv6WL37xBWVdUbetJCKkRNCWRKSwCpKmalvapfhJmBi5v7lj9tmn9NNE285R2nB9fYNO4IeJV58KVbqdtyjvuHr3jk8//5z7/Z7NemBWGf7gy1d457i+2zEMnra27P3jLnOkWiIeITFBYcRE9eBtZpRIYqxWuHzg572FeFh3R7T8Efj4KZkgP7oOPxjkH+7jgyxT51ICLVRpYgASWpmsqZzw3kkjnKNag5dJTPCCrEtTb1AxspwvuH7YsdkOKKPwBjCGwiaG/f7oKSKTk8wASRGftZzaaIahF8DsI4NWlM5NbQZ8s3tmUsIMicELBTmn3hwLKudl/aQDwi7TniJGyqKQw3QYicYwb2s0CucnApGh77HWEkJ+v9l7J+amWCuRFnnnCU4Q/QObxGiNMYfJgEhylvM5Q7eX38+pPL5pjsa0MSUxN/TCQEB9vJ9n8MJ7lDbUlcVqzaxpGA+UyqzBTvnz0gD64AMjRcfRQwqOhqwmsxecC/kAF/25UwI4KO9pqlKKlWkSOVIM7PYdhdEoK4y6wmbGA4myKo8mrDEI0pii9M/GiPHg1svErClLTF1iCSJZMBqSYQoC9GotDLZaG4bkUDkdSuf3WxQl+/0+A1I600c1SksBqjJzx+SJbIiRyR/YKIpoTH72BLAax4HgHW6KFGUpz0AM4DO7IyVSkGQCYwsBqLzPRn7yOaTCHjX/1lhQGp3EJyTlYlwbQyCgUsyynswESj8xEPLR/w4gx/GVj74+ND+HPTRlAEql3MBp+RxiCmjfY6Lj/vI9g4v8L//6/yD2PW1T0ftIlR3+/+CLn6Mqzdv331EXLXUta3ez3tEu5tiqkHQf78RvJ8u/6qKhGwbGcI+KcDYXuYIqC6a+Z7E84aHrqJTl2fkSY8AGT7Go+Oqb39FUmtOzGY0tsFi6IOytqtA87KRwtrakbmuKquHd1S2r1YLXP3+Be3PLFAoGtefy62uMVbRty66fuNuuUSFQ2IJy0VIbORtddFy9veb07BRlDck5FlUtsfGzOX6hwCfqomAYO/YPA4vlkpNnDVcfPnCyaNEh0tqS3TDltJ2I1ZYUPXVlGQbH6WrBfoRu21HavIcqCC7QNhW2sCTEh6csGnRZsJi3tLOab3/3jt3DlvpkSZ+9A7RJJDcwTmKsOvYTGMVmvaWtG5rzU8r1Bt+PzGYt1azm9uoOUxncNJGGxOsvPmW92fD23T2mtMwqw/79pUxgSUzbDe9v7zHzOcaWnJ+dEM9O6R5uCc4zephZOK8r1psdZaF5uFnz9JNnDHHHtN+iU+LkZMXNzS1aQ7cfmS9awOKdZwoJgqcK0LnALu2ZnSxpl3O6XYfre0pr8MOEz+kLh9SLoGRI4ENkP04krRm6gVhYVFAUdU2MYwYLH5lpgLCjUkBhjolq+VARFQh5j4ny+jH3LQVZb1m3eBiLHgakh/PiJ7uyBCDGx58nkX1PdE5pQ96/InCgGKTjvqd+xCLL422MLuTsPxjBJgGObFESvSf4KZ+XwjQMbsQ7x26zYeg7/OSoZm02RRdWT9028jmEcNxvTR6cAAKMeI8/eLakyDRMVFXBmBKb9TYPEnxmvIsPSgiRsrIMIRBGSQM7SFo1kELAYLDZMDaOk5h72oJZZXn55IQxwLyyFEZTZUP40pQYFLZQFEahDTS1pjSaoi4pbAEa5rMZKDFi9zl5a7FoGCfHmCLKRyprKRdznhSGp8s5N/cbvv7+LaMLbPfCirGVJSrFoipwKfFwvwVEPkNmpCqklvLTKBHwWrGct3T9KN6J1hJJwmZJkp7mhkmCEcaJsi5IVoPz8rkrTVGIBDYEOTtSEDlxWZZ4HwnBUtc10zBQVhZrjbBtEWYFxPz8mOw7ovBehlUywNJUhcUv5vTbPW4Y8LWwvaqmOmAW9N2QGUE6A21iWqyVAh+YhhFbBPYpEiZP/fmn9MoQCoexikorvFcobbFW1qpRwqhVeZqQoYtHcOO4TxycP44Li8e0v0TKqXayFD7+k4/rX5HZWXn4c2BuPnJ5Hhufj+vB/6/r7wUQctgvBS16ZIEc3uShaUgKQXqMPr7Jg9ndgSBt7SE6SVaod5Hz8zNIgSfzUz59+Yq7fsf333/Hv/yL/5ZnqwW/+uo39OuJlxdLvvqbrynnDbUVQEAlceYfnCMZOD075e5+yzhONFXJ1e2auix5cnrCvp8Yp4lh7CnrEhc8J8s5yRjW3cC/+Of/mG/efODm7S2Vdmxu7zlZLRhGmeT1+55N3uS0sZydnaKNZrcfUNpycbIS09TLS0ptaBcLdvuRpxcn1IXltJ1h6oquGzkpA/GshZj42fMXFOdLps7jpiDUyrLm9fMLdustm/v7I6pWtS2b0XF6eoauKnbbDW0sGLpItx+whWW9WXP+5IyzpuZPX7/EKCOLCRj2HbYoWc5qqAsKbWlsydnZBX9z/yvWlw9cPD1j1rYM48RsPscoy7xqaJuWoSzRbqS7ucEqzZAgJdHyKx2ZQuDFcs5m27HuR+77DdElnj19wn635XbTo7XibtyznM8xGq7XPZ+9OqefPE23ZzVrWT9sePr0FO89T2YNoSmhLHj67ALXO9CK05MFdw9bbGFYlS399R13XU9SinY5Z7vZc3N1S9UUrFZLcfNXmjhN+MnTj4miMOiy5PTilK4fKMqSFBP3DxvKpqEwivlizjgO+NFzc//A0/MTVssVSifOzlZc36zZrB9Yzkq2DzvipJiXJWerOW5y4ssSPjJK/WgT0OpgyslRM1cYWWw5/S2DIkpMXaPMPW1OII1J/EIOmEP2lzpeP1l9ov79Xx/p60mozFJzRA4SksP+GolHqq8uS3RZYpSm3+8yndAzTSM6Jpp2zjCM9PuB5eoMYy3fv/mBbuyp6or9rmNykxQNmRYrBq1K0mZMQTNr6LqDfKU6fhjSe2avhnwwKCMTdHP0p5AG11grhplGURhDcE7AAyBlLbU5TCCmxOQdwTmKosZrwzANjMNI27YUxuCdk4hbfWiG81QwyfSnLCuaoqYb+2NaSUqgrPiGOAUmgR96/CheOrawaCXO56UxIk8JAR9Ez6lLg/aByfsjCAIcGTuFLZjPWrwPjCESvBiAksBNoyQ+pIMfjqawhUQUIpHQIrnJRrVGHyUl1mrxI0ExZuCirCqshnF0pGixRlJAdD5zppRIKlIhbJi2bTMA7lC6Bq3o+4GULN4H+umQTCM04JTAMbELQUCW/Ewc/o3SVHVNU5UCnMWAd1JsF1kOMykBu2KMRznQgSl5SCiQl9IRKBG5k9xe0RhbmfQqxayumSbNkE1ardaoFIlW4oIPf5f3IWtuxTuAzKKIUb7PgfatjT6yIrTWTJNMJt0w5DQgULZAV/ZIrT7qXv8jXx9PiNLjix//ieO/Dq8e3Px/JIs5UMqNIvgJhaMbdnz/zbd82Gzp7m5xSaOtIihFUZX82S9+xtvbG26/u+fFxZO8NhMPu56zizPeX95wcXbK6Dx3t/ecXTxh33U8W53RNDX3t9fEFJi3jXiCacPmYU1R1mz2Wz5/9VJ06kBRlLx52FIS+MUvXrFeb+m6npCgwxCnxP1+x83NHd4H6qZFW8vgISjPYrXE7wd+91VHP625vLylrUpOzmbcXu+YfEfQko5QFSLb3PU9VV1zd7/h/OKcz794zs2He3RRkozi5n7LyXKBHyfassL7wPVuJ5rzqhB2RNcRupHeeTBW/JaamkVmcGlbMI4DWiEgZd5/mtKijWIaJxazlu1uxzBOTDFSWktd5/h5gJh4WO95/vyC+aqlNgYw/O7mHtcNYDUXZ6dM+4HnFxVXdxsBdgvN5vaBZjYjTCPrhy2LpKAQae3ZyZKuH/n9myvQCV0qZnXFSV0SFzNUVXHzRnxT/vTP/oj1dsvdD1fc369xIXH+5JzT04Lumw2bd9eop0/BWhbLBVXfc/XDewoV0U1LPatxo2M5n7HbdWBhs9lT1QVlUzENE6RIP0WsTmhrmPqRZj7DKMXN9S3NYiH7tLZUtbAUQvaK0oV8HtW+x2dZRB1qRlvw5OyMh9sbAcGJHA3Lj02QemRKKZEdJiX7sSYSzUHDjwD4MULMAGmOPT/Yoj7OQdPR8P0/9qXVgWl42D+ymaOKOR6UIzByOGtiFId6lRkf8l4k/j0lMdqOyD6tbU5k0ZK2EkMgeTHKLOqSvt8x9hNKK+7ePZC8wzsRR2prmJ8sGLoBrb1IYLX4Qo1DFClWIdP94APTOBJzjXKoB62VmFtdCHiNEXDfBU9wwt50o8g33ehRJDknD71ZOHguiVmvVhGjDfN5S1sUvHz5lGa2oNGaNg9tjszEGNBqygOsiHXCRtIp0VY1Ciisom4amqZkmiaatkApYa5orSmNorHCkJyynLgsCtTJjJOTOU1peX+35mG7lxh5pfEpMflsGL9o6YYRnCeGhDKHuiyzEEJAoRjHiVlb0/WjgOFJ1kxdlbjsr1UUVhJauoHlaoGbhP0ZtT6asRtjKKwheLkH0zSitBUpkzVYoxj6EVuXWWYCKCSVDogpMI0OW1qsFqaJzc9ZSIn5bIbrhDkc4yF5S9EsF8yWC3YxyJqexEzWlqWc4VqYsgc51uQ8w7Bm/NWeV19+yWq1wAWwPklsbhqAirKwebgRj3XNAcQwR7l3hjw/Gq7Gjye2CJNIeA0HyOQwl3hMzdNGhjv64DeW940DdewgbU3xYynr3+36ewGEfNyoKR51usfIrCNMJF8f9OwHLwAAtGZ5dkLZWNyuY3myorvf8sf/6E9YFZbXLz4hzlq++uXX+PUd/81//V9Ru8jN3Q31vGC83NHfr1mdzMBULBctQ98fYyKruoYE0zBiCk2aIr0feXjYcHr6lHpuYfQENdGUBdvtFmKirFv++jff8E/+s3/G1X7g8vKG159ccHd5xTSOuGmSZqWqMFrRdyOztsFWDQFFConVasGibVhvB97dP2BTJNia/RQ5PVux3g7ML04ZY6AYe6xJrLsdi/qUoC1/+dXvOXvxlNm8pqoa6sWccd9xeztgbQI9EX1HtAX3m471bst6vaWqLE1V0e87eh9YrhYMbkRHx/X7S1Y/+4KLV8/44ffXNL2mRDOfzyQWNB+spi4wpeG7d98xL0te/sEXXF7eEL0kQpycroSW4GVz+u677zFG0ywX7F2gsQ1Wiwu0LrQYEzqHw7O7W1MazdmTOZv1himE3PQK0yQ6L47RLvC7b9/wx1+8ZDVviLuB+6GnaWqq2YyqKOg2O0xStIs5Tz97yZvfveXi6Qm1VgzTiEMza2qC27He9ygSdVOx2WyZ+4a6sOytkRQNpTg7O8VNjn23z4tSc3K+5OrdDacnS5x3zEzLlGPryqrCjwE/9HS7gvOLc777/e/RZcXqbMFuu+fsyRkpBvrB0dYtbVNxcjJH7zr23UhhFO6jdXXwYNAfrSt4XGfKyNdaHzw1hB2iMzPAZ1pr0j8GUkiPAMrfHXP9D3uJS70+poDwUdGUMvytMhvgoEU8yDRizM9Jyuh8DKiUKXdKkZLEgFpbooFpmvDjKLTfFLlZXzNOHX7y2fjToKNspTppTHa29nliUtUlU55qKoTJIXpJuaExSvEg5p3CEnDTKEVpPlx0bvJBIgxt3UBu2INz+SAuADkIRjewmM2p24oQE3cP92htWMxbjDIUxrKPHSTwboSU/Si0PBTBe6w2lE1DP/RHx3udEbGQ11qIYrLGsck1GCPsF6sVMcgHo5USNsRhKvgxeykffUZBVRg52E1i1tSo0gg9VytiAGUsTV0J5X2a8G5iciETf6TINvrgqSGMhZSlJAkYhuH4mU3B45SiLCzKCIDkxhGlNFVdyutRnNvHfiQWJSFEVssV/eTwzlFWLbYo6ccRU3q8n450Te8CwzBSVgUx+gwuFdm0VaD7aRwxtcgsq7pGacO+H1DaYItE6hO+sAIkBWk8jvVEOlToch8PqS+HxRCzHpzkSWJago95ApSnt4U2IvE5fBIH0AqRkKm81g7rRqckvgLeZw8ZOEyBQMzKAExhj6yoFCO+7yTe12j038mB6D/89XFKzGHfOjx78lhnUE66nsemhpSTneTZmtyINkLPjjFw8+YHVIT7oeerX/2aZIzI2oKBECV6tKjZZvbnfNZyc3fHyXIpk73J0bYNUWl678Ws2nvqIhemoydGh9aWWV1T1Q3bsedkcUYXHJ+tzmAYKKuWy+v3FEVD3UhS2tvv3xGIrJqa0DsmU3IzbPnw9j1FVaOKkm03sFquaGeVPA8K+jhwe3WFj4FnT04gKW7vtsxmMza7vdCrK8PkHbPZDG2zp4xWbB7WxKnh5OyUfrfPa6ZlnAZUimymkXnbkvB4H7G2kL0CRd1UJKPZ73tqoySStiwpmwqjJHbTB/G2McB81uBKuc9lWVJXBZqGFCM3m14YWdbw9GTOZr1nOW+IheW7796ymDc0J0uubx+YrRZcXt9R65KLkzl3Nw9crj22NIyDY3rYYcpSmtAke/o4iPyxqWs2m55E4vxsTr8dqGYlanIENWO/3bNSFa9+8XO+//prfv1/fsUnX7xmqlq0h4vXJ/zuV9/y5OIFZy9eMPY71ncP1LOS7e29MEpPV3TbLW67Jd57qqYlkJifLsVHTDv6bsJoQztv2OUmOOTBRt8NjOPEsxfP2Q8D/b7j7HSJDyL5qpqG0HUZyE7cP2xJCcYpYArDfLXi8n7DZrNG25KUHs9QUp6UKPFEkjj5w8gkHQgVHJsiLQzLlB4bTn04wPNrh6+JIUsrf5pK47hPHDa4/F4UWjxP0uN+EWIiokmS2Xr0TSHDQMeochPHdk4AACAASURBVC1eVXIuiUQhJQFKvOuFEagU6/U93X7HftuhSxk2lm2LGkaRL5QFrh9QSkB9ZWS/cN5TtzW77U5AexDGUjqY2Oa4+iDtqDGauq1y6p7CdYMwHgHvQgbe5TkyRuc6RWMPElwFdVnw+csnlNZy8eSE11++lrsXAnVRYpUMnVI2jTdKvr/Jk/3ohEkIwqq0SVG3lXi1WUPoB2HtkqR+KqPUHSFQGCtMJlM+1no+UljL86dPWC4W3D6sWW/3jNOEizA4z+QF5KhLyzBN+JDoxhEFhKOPVcqeQZ5pgKIwxARtU7NdbwXAsQfWpaasCoZhYBqEXaqMzmsE6ro8GobXdcEwOnQ0lKWlqAomNyFpxHLWKyXvYxocTVsdBxNucoTJS5SuFvaWImG8R5UJU4jcJSYBXmIMdJsNVV1TVmI27UaH94HZvMrhFB7vfK6ZZLiRUmK3H/jmq9/yxZefYpKhPD3DWgtRkZJBK6k5D2anOqlcgz2yVw+myIe6/LAPPDJGDr1ENg/+uFE5nNlKHb3dYv492TfIfBAFSF0V8wH/nxwQgnok2Ekbkx71Pn/rnhyLmY/eY1EXvHr9Jbv1HePdmrqZsb9b8xf//b9kuLxisx/4X//3f4UtZ7x4+oJ/8F/+dyQC/+63f8WLkws+++QFTYKbhzvOL56yPD/n7vqGyUeMtoyd0NmXyyW3N3c0/zd179VjV5bl+f22O/aaMPRJpumq6p42Y1oWepE+m76FvoBe9SZAgoQRZjSQBjPomeqe7i5flZnMpIkIRlx37nHb6GHte8kU5kENCF09ByAYDAaDcc89e++1/utvbMHd7p4+RP7sj/8Zf/OL/8C7Dx5XlDy6uqQoFMtFy9XTR/ztv/tL/uDLV1yuLvk3/+JfsFyvaVzBrnAMQ0/hSmptstFQpC6c0NucmOa0TUsIiTffveHYzSwurzgOByoSj9ZLrDU8XiwwpaEqFCZJTCcm4VTATx0vXj7l6bNnvP7uHTpoLh5d8aA8r3/5a8zkCdEzj6I93u46xnHCpUSdDPd390wxoI14EDgjTtRVoXnz+ntsUdAUljT2jDHSFJax7wl+pi6lidp82FCXJbEqGbqeED1VYaiKCj+OlFWNTwGC5/nTxwzjwBw9l4sGoxWF0XTH43lanUKgVHDVVoBiHCaMUlwtFvhxxFUlk/d82O/RMfH0Yo3Vmn/zV79muWz4L//sR1zWCx7u93z729d8/tVLlm1DRSQNIxd1g/7sCdvtDp1gWZfcbQ70w8Dl1RIfAoMXZs04zgzLCWOFuXOxXlItGmKYJDrXKJQyxJCw2qJR3N9veGQu2X34gHEFZVNzeXHFNMykELi/35AQ+uu7d51oPg103ZGXzx7x3bsPjNPAj1495y//9tdcLFu6bmTKjYlSCqfgOH8EQU4U71PgRUzCUC0LmZSHKCjtab2dgI/Mzj5/7hNQV9hZmt/LdV7+pxeWqbbpk41W51SLeIqoNUJlVFE2/pTENKqwFmJgmgbauiV4TzfNBC0eD13XUZUVUSn++hd/TVktuFhdsWXLMI34yWdTzkzNDvJzWCMSgsPDhgQUZSX095STQZS8L1VV5UhYoT6eZAlaCSATY5Doxcqd2Sp+nrHGYZxmIOGyF4okfyiKeRbKelOjVPYyGga8T2ChLAsWSKxvDIFFWQCw3W5xzmU/FUl+cc7iY5BEHi1ADCnQpyiynejFx8PPYqw3DmLGOU2grcRbxogP8kskNR8bT61kClSVBdZIZOvlYgVEolJcLFesqgpF4ug9x/7IMM2i0zUWl8SDxQeZXJ2cw0/siHQa6ZHOaS6np0iRjYWTp5tnMdyNQvdXRsx1lS2orOZqVdK2NUpFmrqEpqQwloSirUsKV9INPdM8M4yDTFWOHcMcKIsClCYog1ZGnOO1SLS6YULrRFvVNOslpbNMc2SOARUlwQVEwkQEa8wZYIi5oAa5nzHkpl4pfEyUSIMZfCBahUIaC4l8FXaNc6JXN14zB4vJuuOTcbIAZEIbTkqa3XmccGWRE6ZE/hViAC2GaForbFFgnQE0h/0+NzPp71Sg/P99SRKO1Bgmx4XGeGoQMjCSOAPIRmvR7hMJAWZk+lW3Lf0wMAdwqwv+t//5f+UvfvrXuKpgPE7UTUl3HHn24gkhwtfvvmFRVMQE39/c0hhFs1hzf3fLYZxYrS/YPmyxTcXl1SXd7sBnn7/k3dvvuJ9mnlxes+0OjEqz2e/48vEzNvsNy8US60r64Hn37e/47LOXQMIZGHc7LtsFQRv2w4Gbu1v6t2KmGpJhHie0LVkvV9RVwTDOvH//mpQUl48f09YFj66uOIaJ6dDzB0+uue8HjNUs6pJu8hTO4keRBLZtRWkMdVVxHAY8nqJ09MOReYyU1ohBtEo83N2wvrygbBtsTGzut5kREFFzYGGFb6D8TLftGaylbSpZA9kD6+72nrKQiaQ1Wkwhlwvi7Nl3YuqstcaGxOvvbgAYAiwXLY8fXbLb7Pnum+9ZXV3y9Pqa1aJk+7Blc+jow0RTlXl9ztRNRcrnQZgmLIqgNcdhED8urXDacXuzYRwG3LHBOMWPHjv6QXFze8uLsuDFyxf4YeAn//SP4Ze/YtjuSUPg5ecviBi6/YF+v6dsG5R1qNLR9x1l8pRWYSnwRcEwTPgQmMcREG+spinpjwN9P2ALSwhyvk3TJIxa4OHmBqMM69WS43HEh8By0TKOAyRp5kMUb4fddk/0nuXFmt5D3SzpjwPWOWFLY0F9MqxM0uRrbGaEACkSVRB4JKXzHnCigMcsJ5Ydx5CSgqTOZ+RpLc7+92WwfOpBlBQ8ZDA/+jxAEq8oSYTzPxgsiUGKgA4CJp8t02XgkMc2yXtQYmxdr9YMw8Dtu++IQTyWTCFsuqZpca5kub4kBM/m9kYMfMsSEEblPAcO2z22cFw+fcrduzek4OXnVjHf21wEosQrTisOmwPWWdrsQaO1xpUOdCL6gNGKMAU04AqHNQZjFMYqFs7w+HrFj19cUVqDsQ61v5f0piByV62BGAVIMYaqKFEOXJYkT7OidIUwIdEiWSbgKvHVqkorlgckYjDCLtKQtEE7S0rmzJoFGCdhZNrCsqoKLhc1sw+MXtiZU/DMPjCExMNuz2bfcXN7T2MhoOnz4CpnvVLYQuoX7wnJM6fAclFLbLoV7xQ/jjhXUFcV2+2e1XqBKyvGfmC1WhKV5rDdYaxhuz1QlA5rHdM00+0PYCyPnj4lhplj16OUxjjxWlsaRUicjY69nwn5eVMgDE7rUSlkk+iRkFNnXFFiCkn68ZMnpsT66lL6WpUT/IIW+XBm2qUkZ361aIkx8utffc1u3/FZmLi8eESjwEyKEYV1DlcUmU3is+eKDFjgo0fO2edTn4aAp3rg1ESchjxZ33EqE2KOx03wKYhy8kg7+Yae/o+zd9F/ctKYTzQ/Aiup88RMZe+ClG/CCaI9F9FWs6pbbr79LWXhuFqvefn0CZ+9eMn73/6C4ThwPM48evwZl8sFVeF49+bXvP/uNX/0xRes2or9/oHbuzuaRUNhC97f3DINo+jrkmC8J6aDsxZtxX25sQU//+aX7I89bbvm+nKVJ2cR7Qyvf/krlNa8/OIVf/vv/h3NouWz58/oDnuG/iiTy/wMxBgxCPOkqUpcUaKKkt3hwHdff8fq4pLnLy84zoE/ffUjBi+GeJfLBcdppHEwdx2bw5GrizVzjHz96695/PQR3/7yt7z97fc8f/EZb353y//1r/4VzaLi2eMrQpqYtgeGY8/oAxFxlJ69Z3f01FVBHEZcZvn5aSYYnSfHUJDoh46LdsUc4GGzoSlLybEOUhgbZ+lCz9Xigv1xS2UUKka223uasqYbJCrqYrnAZRM6qzWznzFRyRS1LpnmiWjAm4SLimgNPiYaY2grh3UO0xRM2ciy1popJbq+53J9Qd207Pcd//YvfsZPfvyK2mguHz3i++9ueHS1wD6+ZPaRxWLJcrWksIb9ds+c4PHliq9fH9l8eBAZClA1FeM0EbeR9WrBYlmw3x+pypJDOHI0R9p2IRQxDSnV2LJgOBzojz3uYikTamvZhT3L1ZJuK8Z1XdfTLmpAZBIhCLWxKBuuL1e8vXlgGjquL1egDXcPO/opZLQ0URQ5pjRvlFldkN3YkcIfGKfIaYis87Q+ngYynwAg/9Hr09Hq7+s6nQScEi3EIIzcoH1MqDiPckgoQo74LAqRb8zTIElQ08D97R26rPE+MHUjIQQO+z3TNFOWNU0l8YoqFwtlVdL3Y0bhBZ0/be7JaNq6ORfrKLJniD4N0c7ghlJadO0xieO60VR1ldkYmrHvJa7bWpJK+OhJKk/3Y8IYmbJNkzS7PgQYRhLQNOKATkxM4WMSiR0MEYWfJRnBOWGvpFy8DdPIse9lsqH1D5punwukKb+OcfYo64SFEMSITKim6vxAqcxiOUVeggzOClfQti0QwXtcYahtQ20tOgS0sxzjSJo9TVnhrBjJzT7Q9QMRxTBPTDk67xOKYY5y++gfY4zGZUqnsw7nLOM44YHCiYmpj5E0K2LSVEmM3I59xzwesYWY6taFYzIWrQyRRN+P9MNASh+jBFEaZxXjNOKjmBiOesRoQ1kZWY9KaP1h3uPKkqqsSGnGJPFYD7FCzx7UlFke8QxkqphOPQif8hw+movF7Divc0KM0LDPprwxMUSZxjlbYAiZ3qpIRr6X0T4DN8KkCV4M+ILXwmRSmskPAsArMVuLUROYiUCRIxmnURiWkrTw93+diiudWV/n/Y2TGeNHto0+Lc486dUZbPXzQFU3HPujFIzTgddfv+bt+7dUTgCiwlm8TzRNSdPUfP36DT959oLvNjcCOinN06fPuNvcEZSiqSqGaSAQKIOsuydPnvDh9o7xOKCt4Wa/5dHFNduu4/HlNbqsSHvD5cUl290Dh/7IT778kv1uT9U4bm8eeHn9CKzjt9/+hoTjMGeDxHmiWi1xhaNeNsz9QN8f2Tw88PjpEyoDcfS4xQXvbu+4WiyhKLibjiSNRPBOE21dMvuZ6qLBKMXYD4zeUzqH0Yr793csVg2rRctu19F341nSUtc1/e7APAw4V3J5ecHN7QfquiD4lFltsp8WLvsVTRNaC3uyaQuSr2krR9fLetJETOF49eQxv3nzlqvLljjLnrhaXNKNE5sP9zx7+Rw/jxhd8PLzZ+y3HW/ff8fjxRXTMKHUnsu2JWiDSpHlsmG3O7JcynnslZHoy8mLGaFWTHMgzB1lURCTxs8DbbXg9XfvWC1blqsV9/cPzPPEsyeP+Ff/y//O8bDn8x/9hF/9+rcQE8vFiqQVwzxiQ8nbN29JiDzgOExSFxlF9OK1ZKwVRk1hqJcLDtsDrnT4cSaFRLWo8JMYc/ogyUN20RCmkdIU7MeRxarmOPSQOEsJAfw4gS2ZfaQ/DhhbC5CbhKVsrMIohTm1NClwOmOV0h+bGJVTYmPKkhmVz4FTG5SAmJOocrpMSrIvnoY2CGn493F9OvwBzuwWdZrMooTdkY2UTyb5p4GuMHEFsM6v8Dy0UUrqsQicvMRS8IxDL423SozziMp1QFM3FEXFYTjQ7/YoFHXTUDU1Y258XSWy2+Nux7v974Rl1dQYJ6B+f+zz/pZZmloSSUzS1E1J2xSEMKGMwTkLfqafvADaKlIUjqKQ2sSkiE2aZVHw2fUFq6alMJpF0yCG3JIgqVLCWX0255d0EZfvpUgyypSNWMeZhEhD60pqH5UiWokk42RAisoDgXRK5ZFgjClIUl1hZeBw7AcUisI5SlewVFIz9NPENHsmFMu64snVBderlq+/e8fx2GMLy3H0KCsm60opYfuSa4kYUCkIezoElDXCDFHIkAGJ6nWVeHhN00S9aKnamhgidVNlZqrECHdB0vq6w466boWhmgdSKgT6UWo2MceVwVtRirm6HyeRBo2KxYVhPh6knpkFhFpdt3KeGUtRJA77jmN3oFmvICRK5aDSDP2R4AM6v46T8b1x8v9sHrYyiFYWYy/F90hrkYhlCY5IviQO+5QEKCFLQVaDEuaG+rR3yEBGOoF0H6ec5y/6NDGW3KeAyIF0/nt1WqAxopQ5m8j/f7n+QQAhKldyH4u5XJT8ANFRn9Z5KBS2dCzalmZR4/3Ay2fPuWgblA3cbW/Z7nt0jDRVQVsaluuGqT+y3ww8f/5cNn8r29aZRmwUXddTWtG6T/1AURY0Vcm793coDceHIz5EZt9TFjXL5QUqKcbkaTAc9wfG/sjDhy0vvnjFYb/neDjy8kdf4JqadzffSxNlHHOYcMZQZn2WcyW2dARgd3fPw8OOwlUcdge2mx1/8if/hCFCf+jQvuH7wzsqrel0oC0Ur758wW6zJQVBLLcPW0GKzci7N9BcVPyX/9U/5pd/+QsO91tc5Uhamvoqwe39hnn2GXGTlJzaWWYfpFnSmf6UIlYZpnnCas3oR1RIaOMgBfw0YoCqLHn3/ntePn9OGAcWdcN9f2SOgcZZhv6YjRAN3XBktaqxZiGT9pBQITJNA11/ZM6eCNpHSm3RBTk/XbSMwzQzo2gWNYu6YHfw4vg8R97d3rFc1Fxfrum2e15/f8s//eMvCcORsjB8uLnHacvjR5ccthsePXlMqkourObufoNB8/jRJe9u7lHZuKgoS4nWQ3H/sKNwjovVgn120HdWzKPqWhZuUTiJMY0iOwkxUVhLGAcmpXn85AmHwwFTlHTHI1olSlcyK4NOAeUj4zRSuIIn1xf88tu3RB9QBhaLmsNxEilHTEw+/WA/sUZ9iiNKo6ZUjnD9j6xJshd8+sGyO29N+tMv/D1cKjczp+LpXJxAbtjJiLhoX9XJUPOEHKMxhcW5AqInBZGCdPu9AAh9xzjNEm+W2QwxJlprRWPaD+x2e0KcWaxWFGXFxCA589F/7LpSwpUWYyyT96LhRZNSyEi5SFKsEUqyyjKZqDTzPBHHEW0sVVvRD70YYVmhO1pjMdZijUErMVYNQQpkq2EOEt0douJwOFBWFcZZ4jzTdb2wI1TWQStD3x8zi0J8TeZZYoNVPuxiTiQ5V7m5sFVZ6qGzyaezYnYaQ2IIgaLUGKvxXqYwYjQqaVAhSwEKZ1BEMT02hlIbSqOoK4eh4GG/Y46Bq9Wa7njM6TiOiLAbZh/QJEm2ASZ/ctNXZwAEkGEk4ncCAhZNXnw4VsslKkVSUvgYcEZRGoNKMM8TxjqJoT4OQg9NirLUqJwEpJ1lsagw2nEcM9hROLr+yDSNhNkzJairSvaA2Yu/ktZoYxnnkRgGCi36Xx8l+n3ygaSFRRJyQ06MnKxlTtOVU3FwjrxFzOVqIzTmFIQ2nWIUICieklMUwSe0EpbKyfk1ZANfZ0SuZLNJXD8MIu9BCpyQ15xMSiMmJnACvMyJM4VVGyua698Ti0yfuhhOnsOf3K9cSKnMIhPQOKKVAG5GiwGk4HqJOQRSHEnjkf/wN3/Nzfu7LFfz2LJgt+9YmJb3dzvW7YpN3zPFwKJdsdSwHzxFIVGLsw8c+4F2vQaVsEqjSsPh+y1N03K/3dE0C5LStGVFVdfc3N/w8uXnPDzcsN12fP7qBbo0uEXBOMz86KvPeff2ltubdzhd8+HhgxS5piAy0zaOcZ45fPc9yjoKq3n52XO2Hz5wTHDx+IpuHvns0RU39xsOh70Ya1c1KSkOewHzSQK0ybqXZ3nIz8eqbdg+HPC1xznL1eM1U/67cfaowrBeiB+Qnwau1q0U3SbJ+d0N9OPIPAW8jyzammmc8NMMIaFJaFvw+PEKfGQIniIa1uuW+q5gPowEZ3h6/YjN5oHSFaBg++GeV5+94ObdDYeHBwKWp6sL+qlnWZUcpkg3TbTaYNcryiKxrDSbg2fZtlSlUNOfPr9idax4d7vhsqppq5KQhMq+PQ7su452fcUQE8PmHqc1tqh4/f07vvrxj9lHz/bhA+tVxVd/8o/YfXfDmCLGWD7c3/Lk+WM2DxuOXY9WGh8iU0q0i/LsjWeMZRxmjt09ykjDt2gr7u/39PsebcUzYhokgerYDbhC8+LZJaMfOGw7dOGoyloYks4xdZMYcIbI5uGeqirpD3uULUSOMc2kpIkqooycDSf2R15RGQyR9BilIGp9BmD5JEZXoc6ZMyRIKkMjCaTYhBj12XD87/tKn/wuu6zih0mW4l0owHLM4E3MR3+eVAfxkECL5+BJhkz24lIoqW2DpKOBMBv64chw7AnBUzcVrioZR8/QDYCiXbSyj4dADFHOsu5IzOmDKYbzeSwJbvFTr/VcM2lOptvTMDE4LWbNxrJ/2DH0k4AgIVIWRgIOlDS5+MDFsuXx5YpHF2sq62hKR2kNLqcH2fw6dY5GVkJZxmRjd50ZmGn2zFOWX2aJgx+GfC+TnHfpxHCWtU8MqJTOPlanYbJYJ2himsUPL59j2khSndVQWI1WFhPEQ6cpS9rCcbVa8tc//w0P+w5HEnasgjkEDIFoDZO4kZ9cTIWl4j22KFHaMPlAs2jQKaFydHIMgfF4xLkCT5R6z2hCCLic0pJG6PYdYfRoV5KSSJO0NoyjZ11Xmf0uD6T3kYv1grksOHY9xMTD7Yb5NPAwkqqj4iw1BmDrmmN3ZP9wYDiO1IuGwhma5SXVYsFxvyfMXvxKCkee6shTbzSHfcft29e45LFP5NlyWhOIOWnOCuhx8tWKCaVPCg9ZJ+gMfCp1Zn+cV9kPEMcspfu0x8h19Wn4Yz7xC40kef/z2vR/B/T0HwQQ8vHV5430DIqcite8caqPRV/R1LQXK55erqkqy7Nly+piyc2H9/SHI029ZrVeYzQYHdl9uIE0AZoX14+omxIzB7a7LeuFOBInpVE6ZdrkA0PfYRA9WFnL4TP0PYfNA6NWXC0foV2BWZcMh47uYYNpG+7fv0cpYTIslzKxp7A8efaU71+/Zjx2WGWZ00DhSi6aijkGUvD03YFpGhnGwDBN1O2COM80puLJo0d8891r0IYvP3tOP/U4FWHuGbqBQwp8/5tvKNeXNFVxlgKQC9K793fMt4lVe4sOiTAp+sOBoDyX6wXdrsO6kpg0sx/x52l6pC1L+piY/ASlHH7WCY3N5OlxXZZUzonZVZLGM4TIRVkxDxNOyVTnFPE4zhPWOgrriD6wLEv2uz22KFhfX0KC0jlub27Q/YHCahbVgrBIzNNMP3r6eWbZSDpFP8wcxon+0DH0E+u2zvRvmWANRzF9/OrzF/zqd99yv93zJz/+gg83H6irhvuHgxhJOsNh3zH6GVSirB3DMLNeNsSQ2B4O7A49KkG7aBn7nmk4stsWNHXJetEyjx6N0HVDjFR1SyRRtTXhwwfGUVzaMSYnEnm6/Y5+HMUErdvT97L5GGsodcHN/k6e06KgdJaXzx7x07/6FSOSCmLULic/ZHlGfvd0gnlOHxkimf0R8jQ5xPSDA/+E3J/YaB+LgI8r9fzx75ER8unPc/pzyhvlybSUjECLYZs0+SF4tBHKYFGWhCHkVAzLNM9oazgcjnTHHmOkEToBAOM8o6NMFeYo3hTTOGGLkrJ0ef+yAlI4x/HYMwwDSmlcXaOTUH61tgiI4zFKDDoVMuU4MTyUKwlG0x0OIlFSoie2xoG1H2OCQ8BUFVoZ5jAT4ySFp1IUxhDDTJxgmvYSw5ogzjPzKId4SoLcl0UFkIuqSAqJ4LO0xIg5W5pjnpKL2WyMSQ5cY2Cc8uTLgFZYbfDjIPGMShFJxHjynVBobSlcQVlIRO547DBKDJsdMMSZNEQKY8EISyzGQOEKYoLu2Av4gjTrQWlc5YQN0+VkkiR+HfpUgJ2mSPlnIU/wBu8Zgbau0UoKtIhm8OLG74qacR6x1rC6WJ7qfommm2e5T7MnkSiLMif0iHlqURRMo2OeJuZZtLnWGpKCYRqpyoqyrGjLOrPBhDarlDQbpXMoHfGFY/Yaq2CefXbtV1m+ET7R2WcjMY3s/dNMYS1RSVKU9zMmx1/OGZxTCGiStM/Amj7r8lNeS8ZokUeldL6Ps5eiUBuDtgLkxZiI/YApCiKBlAzKOZETkIjp95Mac9YlcwI/Tn8jzBiTTeeU+jiFCjFkTxfNsT9SNgsOfScsxcGziZZf/exn2SxcdO3DOKOU5kd//BP+/b/9KX/05Ste391xsazRJJ48+Yyfvf6aJ9dPmKeZ7TBQLxpMDDRtQ13XvHv3Bu893ThydbHE2AqsY7laMPqRr5695PbuLfMYuHp8RYyJ3WaPLiyX6xU3dxte37yjKBpuj7cQFdYV+Bjp+56UEtOxp2pqbBhxqiYMM8vlEm013/7uG5btmg/zA4U11O2CaZqYJ5F8Xl8sROOeErt9J8+IszS1w/tImWUlq4sWhWY6HOmHnuvLNdM00bRlTjZKFKWAwFM/Eg57eu/ZE7DWZd+CiNWKqZ9ISeFsgVZJgOhw4ECgXa0kaerxiqIq+OrLl0yHgV3fMU0DV+sVKXgxgC4MD4c9VVmwWC/Y7Tqijufn3cbAT5494/X2Hj0cSRhuHrpsrj5xPBx5cn3B5uaBw+i5rGvmFBjnSSK054ifA7ZwDIcD02ApXcH15ZqHXcd4HNjc33B3c8eLL78keM3f/vSvefToCYaSulLUdcX7729Elma1VOpR9uOhlzQKYx2Jk8+CzvJEz8F7Lh5dsLl9EKmmDhS1JFL4eWaaEr+7ueXi8pLd4UitFa50DEd/9qHSCvpxFunv6DHWU1gHPuQE3CB+LSqd95LcWX9yJksM6Wm9aW1IWuQwSStUPLE4MxPlBPLmJulMv0gKk/2v/v6vj1XGKcZBK01UHyn5meQhn//Uv/D02vTJI8vg/ZTB5pTT2k5moQmVgsSXq497zykqfeiOLC+uSCrJ0KAqsEZMq5U2KCUG//M0inG4kUhprYWdQpaUn3rNc22kcnJdSkzTzHAzCOA5BpzRrFYNzigsCY2WxtcY2rqkdZZnjy54ermiPZieegAAIABJREFUrRyrSpKhLGJ4azMzQ2HOLA65hAlxMrRUSmF19ljTH2Wzp2ZamPMfmYw6e69EL+eIUkGYWfKH89CGKIORk8F3yhIrpaAwjsJKPWeNQRvLqqm5voS2KPnu/R3//j/8XNK3mpY5zMxaamacZpqFORFOigUlcbGLuqIshSk39iN9P2Q/qcg0zrSuwFnLMQSm4EWyZrKJupFhkVstMdqwXD3l5u27POSS56QoLCkklJKUPBlYJxmez5559lS11HE+yNBjPA5oZ3DWYcuS9dU1m7tb+q4nxsBUFOAqFus1Shv6/Y6UGcyn5/vk5RMT3H/YiHS7rKmbmojCaJHL5XkI3s/ZJiedf8789nyymk5Dm1NST/q45vPKk0M67w+5Tj3L37ViPvmI5ufqzBQJgfh3MGX/BwKEcAY5fjC944eUmNPnrXMslmuMn0jDEe8LFssV37z5jqaqaeo1jy4u0QXM4wBJ4woD08QXX36Onz1VJZnvxjiKosIoQ/Seu5tbrp89Y+oHgg9cXF1inGUYB4ZhEAOaomLpSsqi4M3DPc+evyAmT/KB/nDIU3TF4vKCar3iVz/7JU8/f0U/zxzuP0ijkBJaaZaLGg2M/XBG8drFgn23o8qRUSlq6rZlO/RcXCxpqgXaaRZYYvSkpFhfXXHz/obl1SPe3264n2a++OolZe3YPAy0dcHIyLg7MqFxVjPPI8mAUzXdcWaYZZcchwFjFIXOTv8h4JzCLRs22wmTdNacCsuhrWvausbPAaMSjXPZk8Fgi5LopQm12jIrTwqBWmmC0nmT8GAURVvzYr3Cz4HkIw8fPvDoyRWrpqYw1+w2mzylTChnWFQlc0x82O4gRcpCo5LBFC3EHXH2DPNM6RxWa0Jm+HzY7Hh0seR337zDzDP/2X/9n7O7u6esG+4e9vzBly8ZDgdi9IzzyHLVslqtuL29o11WaKupmobNZk9lDGEYIcFus8VaSRZ6+uiabi/MF5UM+/2edrWicJblesWxO9KPgRBHVstWFnUINHWL1onVYkGYJ2yMHMeJSSuGY48rS5F0VCWr2fDy+WN+9c1bxqRZ1yUfDkMGO1ROUxI2yKnGOLHFNKfGQNbdkDW4kmoBYgORwcePS/BMaTttaL8vHCTJhnH2afvBD5JdoyMysVZKGhmZXSdCnHGmwBonU+sgWLIAXzK1mDObxFpNRBFnOfCOx4GiVszTnE0lxUx1sVoxDj0JhbEFaCWFTJTvHWPCzB5tBMCI/iNKLzpOh9UmO3JHXFlQO4c20oTM40BZikmoNQbvZymClcb7QJKBE2UjbutGW45dx34nBofJiuFXignrHKURed80zzLZy5IJMV0tSHGkblvGrsuTifmcFhIzHVVrkYScGntIZ9ZW9Db7Soh+dQ5iRGu0FIh1VVEUDpvrsWmKxFmKHmct9w+3mLKCquUw71EaSlugVWCMUQopoyAFLlcLkS8pBMgqHSvd5rheid/VxmGUPoMxwWevDGsyHVPLdE5lJpQ2kJ8Zp8AVebpiHbvuiFZQlQUqphxBm+i6o4BRcxRpT+HwOZllsVwyZpnRPE1Ms3jzWOcgDdjsrWCqimHqmaeRsqikUcxswSLH1ialwFlJeYghU4SjTF85TRylaFFa40MghJDpwZbjcWCe5ky9lXShU3GZgGgEFFCniUxGRUM20HNWDGj9ybU95qIs5T3fWfbbHQCmLM5yHm2sGK1Ox7+PLeI/eik4F1u5tpJ4z4zq6lyYhyASIe9lz4hRJC8BkQQkH3j89CX/w//434MpWbcl231HU7fM+yNfff4Zr7/+mnVrGcaOue+5ePaMgsDNYU9dSIzk4XCgsI4wBarrJWVdcv/hjsPmwGLV0vUjdb2inwdWiyX9cOTxas3oJ+rlGleMtG1FQjMfZl6s12yniZv7O6yyjPNEDJJ8N88zc5hxWhPDLP1miFRtydV1ixoi0Wne3bznomlBJ1bNgs3mAW0MTV3x+KIiJejnmdom+j7x5asnzOPIh/s9YZT46/HQYYxh2EXKwlHXFZUquHl7J0CGSaRxpGlbXOnY73Y5ztkRlEIFy74bCCRs3jOeffYYbQ0Pmw1OO0q3kmbQaqbRUzuH3x8ZihI1z9SNJamCQmuKwnIYPBerGp8i/+TlKzbvt7zbdug4oiZFWzhefvaMd+9v+ebdGx5dPWIaR54+veDqquH7728x1vHoomW73/Ds6QqnHd++/UBTlCQjZ8FyVbOYLUYbbNvihxkCjCQun19Q1/Du+/c8e/WKh9t7sIrSFoz7PdMwMyuRRj757Bl3t3ccdwcB95Ezew4hN00TRgsILaaJ0gCklOh2e5pFzdBP+DkwDTNFXeWkBc9h13FRVSwzI2eaBqpK6raUEnOU+PndYeApEYL4QGlrJbUBGaDozCrT+fxQeXh2bnCyXt8amwcnsk+lpEEblDn5eUntm0f3WbthpPmxDnvOkvn7vc6lRTpFacvvzlrmaRDwLAPJ58TgzBbQJ/5IBvCTl+YeffKwUsyzJPy4nOylkJj5kBmLWmtU3qf7w47Fcs1xO2GiJepSfDesYY4eP45yL/OcPQUBnFKMhNnnM48zGKy1RvxJJT7w5FtmlBiIr5tSzPM12JRwxlIUlmVdcrlsaQvHk6s1q6ahtIamsFgFyYezr4M9gR76Y/92AgZSkkhsbbTIMD8hBIQsbz4nv5003vk9OQ1TYkyyf5+G50qAo9yESaRxEpZIijHH4WpMfv1Jqcwk1YQYsdZydbFi0baEGPnFL36D7w4sL1dMk2YMIgcmJXwUQ1YfIjEkpuOA0ZamXVC4QqJyyeajRmdJayTFQN3W9P2Qa2sBbExu8mOMTENHs15TVBXz0GNcQdcNcq8SZ4nqPIlcNvhAUdd4f8Tn1KEQAlpLnaFDJLpIVc8UtWV9fY2/vRPpWUz02y0sGozVFE3NPIyM/UBZV1Jnh5NFhdRK0ziy327YNA0hBNarKzk354ArCpT+JBI6RZTJ6XOkMyimENCKFCWFTZ/AEvm6T2Vp+TMksg9fDDKCz4Ms8kCIU9+jFebvQD39hwOEAKfiBMjPccZgzwbUQgV89ad/QpNGFoXh+GFLVVm+e/MbqqKmwPGTH33Fbux58/rXNHWLsZqydDx7ek1hxMEZlYgqsV4vKCqL1oK6zfPMzfdvsUaiGY3W7A8dfp7ph0HM85RM/Vxb43aWqe/op566rNBJdFnlquXJi6fcvHtPd+i4fPyI9998Tbfdsl5foA1U1uGsY7e953js0Ubx4slTTN1iNxv82BOUpagbxnkmoXiyvhSjxGRwhTgYl43jze0Hrh9fkFLiJ18+4vGza37xs9cwFiwfXzJ2R+qmoDAKqxTbvqe2lhQNbx/uePnVC179o684bB6Yx0GYFFq0/DYnDJS2wDrHsqlpm0I8E6pSYiW9GPc8efIIQsRPM/WyZZoCUz/IhhYjCU9bVzy5aIlT5P7QYWxBN84ctnumsmC5XtI4i0qX1K4kzhOzH6iqQpDrec60a2lir1Yth75nszuQIlSFZd1W2MLR7XvmEBiDuL1PIXC33fPq6WNMJR//3//nv+bP//zPGI491miZoFYVtXPc3Lxnsznw4lVLu2yI+46ryxXD7FnUJZvdgeNBigFjDSEEdt2e1aIWo7Wup17UdNuJvhdzorquIEV2mw0Xlyu8DxRNyzSONKuWh/c3NFVF301088yUTQ0X6xWbu4fcrGpsXfD46oL73YHbuy2fffmCn/78a0YfmTMt7MySSGIIqM3HiahREptrrMIERdeHjHCrH7BEPsUZ4ieH1aekrb/vS6YcHzdMoaICfDRM0tpm9pEnIRKZOIrBkrWWmAIKm/PdI9M4EKLEk0mUq8mT71PvJKaTtnBYWzBshAExzon9QabEKUqjPY0TYz9incSgBR9IITHHKYOgp1jXIh/q0qQbZzHGMM4BH4SBEIJnfXlF3w903T5ProVyi5biahqDAIoZhC3LkqQEsIGjNKjWSoIHEySRvpBkcuCcMCmmacpTKEmDqZoFMYV82M5oZySGNinatpXITmNwzolcBKlhpYn02MKiVI2eJ0DAITKlcb/dMvQCBM7TRF2VlFUl7uneg/Jsp3ucdZRFSeOKzKKQKYsy4ptSOoOzmotlw3EQnwKMNPJaiQmsJMpopnk6e5QoxVkio41oWZWVIjbGRIgTSik67zkOA2VREWKPMULhHYYBUsQWJa5wtKsF8+QFDBqkgTZWZ8NGUHkatN/tOXY9MzmS2FlGY7BGU1pHaQuGsZd0ktP6VIqqcIzDgLUF1lj87PHT/MmU5ePaENPBhFby8TRNWYLkKAqHpPmJvM/3Q35Ps0N+ShL7ejLEM6fp52laLD4fBgGeYgKfk5eIgeAVy8sr+u5ACgI/GmNQmaCoze+v7Ph/A7fZrpDTHRSqeMyeKh6QNCY/Tyhb0Pc9beVYB/jn//Jfcv/uHYumoesHqrbmfnPgYr1GNSU3v/mGP/jRV7x/95brqxX9eOSzV5/z01/+nM+fPGezeaAoC5wtKIqCsTvSHbaEyVOvGvbdwOcvXvDhsKeulwzTwEVdg4+82bzn6dU1h67j1ecv+e1vfsvl9SWxsLz59W+43xzEQ4KAdgV13ZC6PYogjdYQKNuGLz5/ynzY8ebtrXhe3fVMymGcwoeZVV1xuWq5WNWgDZt9x7puKayhXSyxxvDt6zfYquUP/+hLhv2ezeHIqqmkMI5BJCEpsh8G/ugnr9jt94zjSNEWmPw1nz29os6JBuMcqKwTydAwMYwjx3HguNkz9gP1sgEikx9RpsYPE2GYSK4g+Jluu6O0ljHMLArHNHmWVUOhDdOxo27FyPbpF8+obh74jsg//sMvuT8cOfqB5arhD63EaQ9VzdAfeP/2jlfPn0HwvH1/z/Wi4cPbe/wccHVD3x2IIYkXRzewH+b8dG1RZYktDMVwpP/2SFKGH//hj/DTxDALCNOPI8+ePubV5y/41a9+x3g8UoUKrTTtsuXYDcLGUkJDT85giCgXKV3JMM2SKhWTxDb7076mSCHSLBtUjKyeP+fu+zc0VcFmt2exWnD7/p45BGbnQEmq1BSEifbtN294+uIxy8UC/IyrSmHceTEgnxFPhJPkQUKkPkpkpEnJrAhA5RQYlCGpnKBySvmyNsd96/PXgMIoi/a/Jz3dx1cBCDsuhBmfjcIlIjQSk6RknCJPUwZWxYA+EJNIPkxmg2Tiah5+CMgs0lHx1jLG5ibbCiiQoN9LAtM8B1AST+9joN/uGI5HTkx6dfKDI0v+QyBmSaQATirHtUf8GZyRhBJTGJqqpCwLmqbAjxNOa1rnqLIsyncdsbBUVUGptSTChIDVDoIw243SZ7NKP3tOxrE2e5NF789ggg4mD1lOLD1JLNNKndP8jHYYY8QgNMsYY4xZZSVgiMo3XmcwiSTrUYZTVuQ0uVBUYu6BycxUYwzRB9l7XUFTGv67P/9TLtqKf/1v/pJ5L54aehqlvjGGQz/llEVDQhHmyDRNJN2xXK8I8UFMyrOHTFKaw6HDuoKqbsV8e5qYpxEQEDPEiHUF0zBx//YtyllsWTINwxlUU0res6gkKCJpYbqGEGjXK1ACfvWHjphT4Cpnsc5yPOwwrqBpV1RVxTCM+f1J0PWcGnBbOoKXeyUDE3kugxc50jhOdLsdD6UjhJmqajC2FPlRTtYy2Xzw9CyS3zeRKgm4pZQw1s/eIOqU1CbPwWkBnrqRM1MqJpQVgP/EIDuxhVQ2XOY/PUZIOmGYZ90dfELfyghfs1rwB//Nf8v0zd8yzp4pRq6vnvD+9jXLdk1Z1lwv1tz3e16//ZqFqyVTOk1cXlxTNxVT9NRti0rprJ9UCjwioSjLgmn2GCXxjZuHLZP3WR+tGPqeoq4xRcF+6Hj69Cmb7T1llnjMg2ffj1y+eIqPidu37/jsyy/YvnvL5t1bFHIoWOeo64aHzQNTP0BMQu9cXfD1L3/BzcOBR48eQxCTJY3m+vqS1bKC4CmsYe4n9t2e3ipePL0kTBP73chRa372i29Y1DXKFfz8L/4G7Uq++vIVONgdjizLFtPWaOX5x5/9mK9/8y2//Iuf8vTHX1CvVxxu7wjjhCksPnjauiD6mReXF5KG0B9Zrdas6gXHfmZR1kwpkSI4ZKNIoz/He+nkiTMsywKsZtgdeTjuQDnGcaayjtoUDMeB3TQT1wuaqmA89vSHnsbWeOWZ5pGyLDgee5ySqfBmuwVtWLWtbDx+xsSI8hOrZc04TsToiZPHa0Fmd4eO61WDsxXdceLmzQc+//wF97cfOHQDq4slM4mnL57z5v177m7vuFit0UvNOIwiuTKa68s17bLhVz/7LeVlIQfaOHL/sOH66prQJR4e7vnqix/z7fffyrRlEtO4OSXmcSathEI4DYm56ylMyTzMKGUJfsSqRELTFI6pLtk+7IlAVdeUtWHZVuz3HXVpefXiEe/f37MbZnwSlNVlpNWHE41VAJCQ0WA/J7DQlJpujOeD/ySROV35bP8HcZ32zY/gad70ziaQHlcI4OSDz2h8kqa/KGQSkGRqMkdPRHT/wrY4fvy+WhN9ICXxwFDK0HcdVbvKTvciJelzpLI2VvLqQ8IUFdZq8UVQJ+8NjTXmBxMg6xwqid+N9x7ShHEOm4sh0Dx8uKNZLGmaFu+ncyxbyK9L3NRFLxuC0N+tEWPnkCIGMY9KITLPXswxC0fhLPMs0a59L9IxTSKFwDgMLK4XTPOIK0r6UeH9JFKNE03Tz3lSQKY4Q/CzNNY+MAyBtmnwQfa9qmhRSXHcH1ApYa3D2ILoA2UhtNL+2GV9qyMkiZVdFhW7w55tt2NO5qMGWml2DxuMtRSFgEoxSNFZaMMUJkk/UTCFEZUUJheIpxjZ03RMooEj2mosimlKWVsNpECKAy5HaPoYsjFvYpiO8u+tgBnWWGbvmXajvNeFpTRWmCPjxHKxoqgqDpsHAkYMDmNgOB4xTYNWIglEKfw8YUxEJQEaCmdlGpQSZeGYx/FcBJFORm4SQQjCNjoxH7z3mYFkKIpCJFVW0zQV0+wz+JnORnrkf5c40WPhxJoQRpBi0dRyHkxyT31KhNmjrBRwyXuscyK1mmesNr83mnv69AN1qjnkEyrLYgBSykw5bZE0hoAylqQV1iSmw8i2Kfk//vn/xFV7wf32gHUFwziyvlyBMfzNX/2CP/0v/hnb79/gjMKoguerFb9785amrpm0xidPURZURYkpCz7c3aCVYZw91kqheAyBZrHAKs10HKkvrun6HRfNgtsP9/yzP/8zvv3mtRiKhsjvfvcNvZ9pKsdmf8TVDXEObMYNfhpzIzXzR3/6h1xXhl/84nf4CG1T8+HugdViwfWq5f7DPRaDSTB6z3Z3BGdoCkdZRCrtKGzB3f6ey8WCl8+e8uvf/paiLPijVy/YHHY4rahcyxQ8tTW81Es+fPiAVonrixWFNZQKKmuZfMBYRVXJgCWGRIwzbVNQLxxl51gvKjCWv/3511ijCEqR5gNPnj9BacXh0HHZNFSVo1lUzL2T9BoVKGqL1p4P9zOrtmZ394D2gRQCL59d8+uvv6V0JatFyd3hiDOavUocNwe0dTx/9IRvf/sd1hquL1Z008S6qjkYz/1BGE6Lpmaz7whzRKPpppnL9ZJyseD+5pZUOC4//4yi73nzu29QRUWIElu5Xl+w3fV8//avMEbSAve7rewT2y11U3LRrrm/eZD3cPJ4rQhxpihdTn4q6cfpzNCLOTVmDoF5mCRhbPuBoizo+5mQIv0csUXBMIy4WvyjnDOkLAPoxxk/zozDkaoqSX4maiSqNEYBjf2EwQlwoT8CFqepvQxhTmkPGqUtKjdDSZmzMbHKpriyZ50CWyWmE/P7mbh8ZMRxlgfIYCmbQ6aUKYQxnyUnhovFz9OpQsAZGczEGM5ssxSDSISwhBjpj0cJaHAu9zwCGPgwY5RlioHQ9bTLBQqYxkF8Y8bxzP412TPkVCApVJZ/ClCgjaKqLcHLMxQ8Z48FjcKcIkyzpALvcVZLXTGOqBAorKU2mmVZsKpK6kI8SFTwGCXxpc7kNMJ5zqabwgZVSSS3+rTZRglnSMGfn53TWQaa6OfMSjYkL3IXlRP2VL5HhJMpvspMyCSJhidgSAHZN+Mk3TRFAUqGwigBpwqj/x/q3mTHsixLz/t2d7rb2zUzNzfvIiOjsjKrikVSYoECCA0EPZeGGuhV9AKCIBGQoJFIQcVSksWsyMrI6Lyx/ran3Y0G61zzEKBBERAqU2cQCI9wa+695+y99lr///2kJAdqk0kj56/+7E8Jw8Cv/+PfMxyPlNMKpYPEdQ8exqaLsm6s1UZVhBPOS/Cj2rjzFEWOtYiKRysBpmpF17XEJOkroetwmSMvS5rjgTyz5GWOJlEfjzKsMAIntc6JtdXIuvmsmwhJoPplSfDSuBo6QRJkmSO1LUym5GXBEEQlHEOUxKihl/tkVGP5EVSvjEEpYXAM/cBhtxdF49ZgreNw2FNNFLnO0VpqTqUs6JP6SBRdCunUnVRkSnxjI+9MJFXPIOIx5jmOE1elPz+DJ1XOs1XmFEcdRvWpksHXP/T642iEpM/F1nPz4ycLDgmsNVy+ecf267+me9owm68oqpLb2295c/EKkwzTRcXmsOf7734kK0uuv3pD+3CL0SWr8zVFIR7ZEGRqt5jPyTPHfvtI33UU5QRnRZbn+x7njDRSlMZmjhg2aJOxXK7Ydx1lWeJjR5Vn5Nay2+8ZvExcVmcX3NzcMJlWLCrLzccbdEpkVUHmNEWWcTgeaOsDWVaSzMDZbMHHj+/5cPPEn/7iK/q+Q5sclKaqSlT0pH6g9z3bzVE60ii2mx33n+4pqgmTSUFzPKKNYXPc04fIv/xX/4xZlfP1198zmUw4X15RH2vU0NOj+Ob+R7768g1/+k++4n/+H/4XlvOKn3/5BcoP3D5sZUEJ4j1bTCseHzcQNDSe2gxcXb/EakscevJcoHw2y8jLCrs7UFaOylq6xtP6Dno4tg0MYYwdhaMW0Nr6bEnXDxy3W/YhsFqveH11yaHtBATqe1KM5FmGTnBsO2bVBQrF5tgQYiAMEXygaxuavqeaVDg3E7r8saEb4PFphw+By8WUYlrw6e6Bssy5WK95fHjAmblM1En87ItX3H+8Z/+4pVpUkGUkBoKNlGXJSinat9d8//0HptMCfGSz2eOqnOliys3NHdYE1osFj08bnJFMeVJivz8yW06JJKbTKfe3D6zXc1Lv0XXHcrHg6fGJQ3Ok7gQGizHc328oqo75csn5esF2f+Bus+GLVxdMnOPvvvvIoQ/EKFL3sjSkRngPUYn9pR/tMMYowiBd8tzqf1hU3alf+QdtjHz+4er5yHYquCCEYYxuk4XZBzmg50YmWHlWcNhtOBxqYW9YJzyNEJ7jP5XRhLYbgZaGfFKx32zYbZ/wXhoIxkjBNvQD1kGeFQxD/ywXNqPFpT3WOGOlAZFnwpHoe6IXGroyluhFfWGsTKGtdRgFRZHTtx1d3xL88CwDNMbRdZ3YP5SkvxiXkZKm73uBMjs5aFuVyUFeTN5EH+hSYuhbYpCfeZrI5UX+bOGRg6IUNzpJqk2KAmgzxmK0krhZpXBW7u2TwsUmKQL92FVLwyDqLAznF2tRiARPkTsmRU5RFBLdGyMmt9hkKPKCwXu2+y3D6K1Fq+dNscgdYejp+55iUjEtC9kgAT9Ymq6nCx6nNQaFydxz48OM08yEom1bhkH2BussJrdoJ5OihIA+YxKgnR59vc5asV2lOE50hJthnCXPC2GCxEBRligFZ7MlOiWGITKtKu6eNjw+brh6cU439NDAfDql7weKLCOCFMhARMk0LCVyJ1v3kBfCXRk5FioGYamMHKYwwglNVKRRpWOMMD6MgjAMKCNQ19PvnsaJbhxlxKLkEbCf0SPkNsoUs8hylLVjEwSxfKHp6xrRXatnC1AabT3pP2FS8//pNXqL5V/Ts0Rc8RnaZowhxgGSIrMOP3T44PEklEnMpitma8d/+9/9NySfU4dOkj66Du8T1sPQHrm+XtPe3tO3NVfrtTTIJguOj/d8cfmav//he7S2VEVgsp7y8f0HrMsxmeNw/0jTJa5fv+Lh8YEXL65lqFEF6uZA13bMZ3OulwsOx46Hpw2XFxcce89ms6f3PfWxFeZZivRDLwBTBc5ZfnZ1Tdg+cn/IeffmHfv9E+3QUZU5IQy8//4j5+szPAN5pcndnH3dcF7muDynG1qmi5yHuw1VVnL+csrNjz/y7voFYQgcj3uu5nPCMACRUlmmk4Jt3XB9fo4ferpxWFFMMqrKUg4aH0EHUWedrSp6P0Ejk+p6GaiKDIXh7K/+jKfNnvvNgZgC282OTMPF+oyyyCjzEQ6qEhfrBZv9no8/fuLy6pKz9RxrDF/+/Jrvv/nA5fmK7aHlcr0kM4liMuXPJhO++/CBxaD4xV9cQezZbGuuV1+iU2R3aLgyJdFobvYN6/MZdTvQx8RiUTEpCw7HlvunPbNpwaenLS43TKqKzftblqslb794Q921tEPH4C03Nx9Zn79gNluy3W3p40A5KaiPDauLFSTFfl8zXa84PG3pGpnUD14UNEaJfroocpqmQY2MJonQHVMgQsKWBYNvYWy2WmvJi4LgZYAzmVWyv1hH6GS//P6HT1y9OB+feU/nI8vVhH60KZ+YbzrqkxxwPJLCeIpFJTUy+AzGapSKxGjk0Ip63nf02AiRrxQgbBoZEX+I69kOcDpQjwMSSbmJ46E7jQfF8SCOWDVOfA41WjB8DJx4S8BoMR1TVEjPTRKbFRinsYOoOXzvMYXGKElWOSWxqRECekrL01qGNX5c56L6zP84MR9IAvEXpYjszae8H6MVVisyo7EkHGMtoBS0Lc45Fssl82nFxXzKxfkZVemwSpG82AWTfNzPryVFhXUaOyplrBYf60p6AAAgAElEQVQbZ/gJk4mTsmMEop4+aa3EVptl7nPscorEgZEdpJ+VISAqiHgahCn5O1mWyecy7mXeyx6e4onlwmjdMDhthfUhWyZBGYw1/Bf/2V/QDz3ffP+BOPSiTokSOy/1TxAFttH4sQlx3NcUkwnOybNVZLk0nDJH0zTUTY1zkoSJlp8dRsVFGKRp4b3H9QOt98xXZ6AtfdtgNNSdZ7qa0B4bxjcB7dxoB5YUOGMN1WImCTZ1TX9oKC8KfNfTtDXT2QQQLMMwDIR+eE6HOznRlFFjIIMhsw43yyAm6t2e4zj8s1lBVVXkRUGMlqgkLSjEMAZhjO/p2ATRjEOWNIJQR7CytC3GlWNsYohq+7PkVSFWY6W01EQpSjrkKZXGGFl3RoXIP/T642iEjJKY54NNjM9dWAVUZ3O++M//Cbd/+zWvr1+w+uItZkg0/ZFpfsHrq7d8rO/49d/9HfMq4xc/+4Kz5Ypvf/871ssFX371Fm00u7oV6rFWuMwyX804HvbcP95RlSVFWdA2wgHJs5y67ignFbFuubm5ZwiJ6XJJ4yNF4XAGktIs5nOe6i0mzxn6I3/+z37Fx/c/8nj/wC9+8XNuP90Q6hozeqvNqErohg6XF5jMsj67JCpHd+z5p3/5K/q6JQyBPojsKzeGYjnjsH2ia2qOdS3ATaSju1rMUGFge3tkspzSh8T5ZMJkOefbv/0duTNcvbrCOE29faRyJWriyPqA0RPq45b6fc1//a/+BYPTfPz9e4o84/JyiU5gQiDPcow1nK8WlHlOVc1QLmMynQpPoVyjomdS5JTTinp/YJbJVCYMAUtg6jJC0lQvryg0PD7uhbtSFOMEdMAZWE1F3WF9ZOh71hP5fOqm5nBouThb8PjpjvPrJcdjTXuoKRdzdscjbRrIF1P8tGDfdhzrHp08U2uYLqY0g6ftPbebA4ddzRevz7nbPuCMInMZs/mMDx/vePPqisf7O/rZhLPLNU3b8HD7wNn5GdpaMmMFFlY4Xr55weNmg1UC2pyvZnz69MDV1RUXZyu2uwPTqVh4opLUkaos2OyPPD1usVnOYjGnKB1Pm53Eoi2ntHXN5euX6Ns77u8eaP3A4D1VWYiX8FhjnWK9XsD9lna/44t3LxlU4m+/+YA0wxP7vWdWOSaVIfjErhkwRrGeOeoQaepANySmleZwTKcG+3iQ+vxoKvX5kf3pn/8w17NglRG9NFLnE6d88dNCorQmDhHrpOgwSg6yEQGrZtpCVGz3NRFNOckZQqCuO7R1FEUhCUFKgXP0h70UF9aM/mc5mPvBk2diIfODcINAQHHzxVLo4W0rkXZAMAa0BT/G55aVeGedkwI2RhKREKFta5FVFoVMEUJgGPrRQytJMiFGwtgYUUrUCWmUMgqBXJJS8jyT2N52INMyNap3W0l8sXYs3CLe9wyDJ4zR4Zm1eD/gnGVaFvR9DzEShhaNHMKxRvzszrAsR4l8Jh7Rtmkps4y8chRW83SsZSLoAzHPSERQCWdkLVjNZqwncza7jRRprsB4/8ynCEEaz8aKhel4rHFWJouZlXx7mxmKXtgloMbYN2GAJCUMjSF4vBf/dvSetmkJ3pPnmcgxR9BcCO3o35bLWDMWW58nWUorun6g061A4Jxj+/SEto7Hx80YxSp7QFUVOGfY7Q44Y6nrni4kqtyRJXB5gfXSrGj7HkOiGTq0SjhjmE5K9ru9FONqbCBZg0oakMI7JIm0U0qhnR1rzwDoMX0iyj6SRun3WJhqpVBBLD4nBKIfGQTGyMS5rsWCoZDpTJHnhNGP7QcPI4smBI8xYs0i/YHicz/LyJ6VICcmiCRw6OcCOY6NU5M7joeO2XSC0Y6hOfDf/4//GusDTgV8TLRNR0ganVtMrri/q3l5XbDbPTD0sKkbfvnuHb/+5rdcX77k43ZLTFDkjqzIqQ97ZO1S3N09cH52hisK6rbjfH3BYjrj08MtZ1VFZUUFeXN3xz/9y7/g3/27v2G9XNH1A+8/fKRtG/wQ8CHhMok8zK1mPs2YVBUuwtP2kT//8g1aW77+8SPdscFYQ1QGZwxv37xkZRMmm3C/bSit5u3lisf9gXUG5XJG+7TnL35+ye8/PeHqjn/+Z19y++mBsnRoW7I+X3B3+4CNhnIx4W7b8GJaUa0qUmNYv3jJzfZINgSW6xm7wxEbFK5w7PYNt9/fYHUiqyZ0/cB0VuK3WxIaU5YYBefzimKSMzOGfJLxm999oNluWS/nzGclCU2ea9qDYr6aYX03cosize0Dby8meDpyLWDzPkTUQVLDXl/M2T0eoNmT5Rnn84L7pwPHuuPl9ZrWR451x3IGP97tsE5TKc2ssAyhI7Utb6+WfLjZcn15xuP9Bt22XJ2fEVTkw3fvQcHjseby4ozL6RSbJOXr4/YoTYxhIJ8UHPcNXdPJQasR9Um1KKl3DVrB5rHmfD3DFYYMQ5FZtps9Nrc4mwnnZmR19J1/ntDnznE41sQYWZ8tedxsOO5qtEYsfUqBsfz44ZY/3+9ZXZ6TWcPTZkdtHZPpfEyQcuMBUqa3Wn9mkj1LycYaQmvhgCQNKsqE+dQIsVrgqkbkWEQUOiZUfD4X/aNfEmsvp8J0qoG0JZ0O0+Nz+1lNJk0hZS1KjdbcEEkoIhFrHNpoulbSYLTWI+zaPzdkYwwU0wnb/W5kvkAcZDCQUqJruvFgrwjej8kpZpyIi1IxhPgcUx6Df4ZxKhK+H2jrHoC8tPI5DRJZm+eOSZGRKUWZWXSMXC7mXK7mnF+sqAqHQTGdlliVhD3hB3Ir6sUQTkwrKLKMzMlARCtRo56UKWq0cKUgSXE/TXizWfa8Lw2D7GcxJpIPY2M+Pb9fYmmRfV2Rnq3LOiXCZ0yFqAPGPS13jue0olHpoxCHQELqQoyVIIMkNeN/9a/+iouzr/n0+MSHmydJ+fTStLXOkmNoOlFneB9Q2hC6HuvE0tO0LX6IZNZRTqa0TU3XHLFK2DguF3WGMZrd055iNsW5DJtldH3P/vEBk2fkVUm93Qkzpm6wmdg1MYZhf2T94gX77Q6loG9brNXPg7qTkt5ozfZ+Q1pGZssZ5aSiORw57HYChnd2ZDGKdTeqUb0VBoa+w9gMW2ZikQmBT+8/UJQVXd+zXJ4xnS+ZTWYYM957WLQ51eFjk3NUD6co7DhROMu9f+KChvB5ICsNq9ESxVjXJ+HsCBRV1CXaWNAR4qlB9g+7/jgaIfC8yqSUxk6fdGHLRcXybMnD335N0Td8sVqRQuSh3nDcPfHi6jUPw5Effvt71us5E1sRUuK7jz+yXKx4+7Nr+iFy++GON29fMily2mNDnmcEHzgeDuLhdhkhJiazmRxouh6lPE+bPQ/3D5J+slrRh8D+/o75ek1VCDisDh2H3YFZWaCSIpsU3P/mgXdfvmW739PXDWHwVHkhkzst8vhh8FRlxWxakmcZn54eOV8vKfKCtg8k65lXBVVZsJhPpMjVOdCjbMmZm5E7y8QZQPO4rzEahq5nMqmwec7jw5blaob3A99/8z1lWTFdzjl6z/3HW5azGXmWsd/XTKY5u/0BrRRvX12SvOfx4Ymu73n9+iU6SXdQG0teVEyWM4wtmc5m6MxiImgv8YHtsUNpx/R8jn7copPQwqeLGcPxyPvvfsBVFZPlCmWlU1sfJd+6KEuMtThrmBQOohILSYI8y5i/KLm7f2K5XvB0v2Poe2wmvnmjFKEKbHcHGhXJcseqKvl090Dfe4xWzEuBWX5xfcbv39/zcLdltZ6yPRz58eMHvvryS1LveXzcEJRh8D314Lm8XFEYRzv0FHmJnkw4HI40h4YUIhdnSx7unySWtSgI3YAfJC5yQDx91llcdHKPjdOArhs47g7M5jMmk4oY9xwOB5bnK9quYz6fs1yveNzupUjwka73UrxbT4hQWsOkzGh6TzN0VNOSZZVxbAe0VvRD4tB4Do0nzzRV6bia5tzsWvoRZps7+XtGK4Zx0dHq2epJev4Ho7TtcyviD3ud8slPoEPPs/NfCYWcKD5ZrRCoY4rEriMMHcMQUCaS20yKL5fRDp627SiKiiwvCEBMPf3QE7zHWEscvZAucxAZfy50bY/NncRKjgt+37VkmSMzBpU5mv1xpIVL4no+meBjII5+zb5tOXQ9k/lMmisuI+bCz3ieJI0qgNP0/pSMosaNQg6jPS7PR8jW568DRmVBhlYwLXK6zBGGQZpfKT6rU5w14mU9KRFiED5E33+2/fQDVmucUZI208vhPflAO3T0vdhxCmc5m8953G45HCJWizpOERmGHt8cMFpTzab0gyczljx3OJuQJDzJqDfjdC6oJM91BG3N6NkWy1fXdQwhUhQZVSGNw6Ef0MZyEj0pLUqAzGWgND7GsekrjIhu8KL6GAFvymiUV8+Tp5M002XZeB8IJf00nagmBZnWpBDpBo9XgT4EUvLiyR5TBsqyABSHY0M41EQq8sxRuYxj25BpPXrnIUcK7hTl6avKkmPTAElApsghRFtLGKeQISbwUiTj5KDhjB591xEfRU6sxwQaSM/gRD9IIf1ceI+HHu8DHimGnZPi01pLGO/JUwNJjeqVE4vjP6VA+f/y+n+A13/S342j4jT4gEYmoyFIxF/TdqKOQjEtMj7c/8CP/+Hf0nUJnxLHYcAnBYWFLtDpgZev19zebFmv50DLcjbj/tjgtGExm/DwcIdSijzL6ZqOtuvAGpp+YH12JvC9wdPVtfBH6poXiwWFc9w9PRJi4Jc/+4Lvv/uGwmqatqHuGjmIjD5pbWEYOorcUU0cvgvS1DKG0uQ87Y98vHtCx8TF+ZKu7ynynFxrhq5n6xWhG1hNC5xNDF3Ny1lFDJ5COWaXM+r6yK+uFgw+kuqGZelQRuNT4odvP/LVV6+YXUz57tff81/+xTseU8D2kcmX19x9d8Pr5YxBSSMp9QPb3uO3whiYzErarme73ZFnjq5pOVtNcc7Stj3rqxVPh4ab+x3FxRwivL2YkylN0w9oovCJQqLShmo6IUUvTZBNQ6osXa8whaPIZdC03dfE3Ml730TOZiX3jzsM0MbItMiY5o6b9w9gNcv1AhK8qHI+PmxRSlLisqrk4mLG067lfDmhLBx6NuXQdjgVGOoWnwZmZ0sWPnFWTfi+viMPOdcvXvKXl0u+/vffiAqrc9JoizK9F6CiF/l9Ic1lPUQenw5MqhxzVuKiYbmc8rQ5oHUkqwqJvI0y/U0pYZ2lKOVgFEOg9j1oRT5GrUrqlHA7iJ6P72+4fvsaoyNGJYwS61IiorQd7Znw08bHc8zsOMmXQ/AYKzFyeJ7VDEqOpieigOzmYmk82WX+EJes7ycF2XiAG6X8xtjnlAo9DnJPUcIxqdEqI8wpBWhEru+HQST8Sfbt6AdcnmPzjNh29MNAe3dL17TYLEcbL+/j+B57L80sRhsiJxU949T95FVVGqMVvhcFozF6HKqMrKfxPdVJVMFVbpnPSs7mU0prcEoxzR3X52uc1by8WEA/0HU9KgzP+50xCu0UTovV1ozpLsYYOawbUcSEELGZGvcTffptn3/3NKpUPANDSnT9gLWGrhfLjTUG4+y4Tqvnz+bElPGDFz3BqP6Q5sv4uf3U551AWVGHSlMlEIOkgxknDX41fv8Tk0drw598+Y6k4OFhJzG2o2NBJ40yGmcNKSiSUwyDRFG3TUdW5kxnM/abHaV1NH2H1YohaPKyxGrDoW4l6cdoYh+Zzmeicu36Z8jqEAJFVaKdNKhO94NSI3g9y0YGi5yfXZ49D21OnDoJesjpm556f2AYPMuLNUV1gR/Pvd6PAHaVyIpMBkgj7DwlYWkqxRiXq0EF+r4jrwpC39EcdmTOMimrERIrNdGpMSc36mktGFOkxtCDUzJQjKeB5mhT/cnE9dQUYVR/m1Hp9KzeOnmhT/v9P+D642mEjNNbdaLXj2+cSprU9xjgL/7lv2DwntvbR+rDgdXyjK5v2WxvWMwqLAZXGu7vHphUJZdX5zw9bOnajsmkZF5WKGuI1BR5TgqJtmnpBwENWudQaIbgGZKi7geaQ4MxFmulgEsxUFQVJkXq45G26ei6Hdbk7B8fUcawf9xgFRiV6OtaHrBcZOdD35Fiyb6pMdpSljlNN6BVQ+UEjPrp0wN931FWJUXuMEbRtK0kBKhI1xwwxlKckhCUxuaWc1XSd47F5TldF+get7y7XLPtWprNgflFSUPgw6c7pkXO2xdnIndGsVqcM3S9pDqQSFEOANdXl9Rtg+p6ZssFRjuUzilmM0xe8OLi4lneNlvN8b0noSijp69bnLKcnZ9LkgQRpzXZRPP2Z18yDJG6aXFOFuyLizX1vma73+GDJ7OWIi8kDioGjseWOGiCcyyWC2LbM7m6YNt0HHZ7tNaU8zmkwGo65e9/fI/qPes38pCrEDg0R94/7XEuo5xN+PnbF3z4cM/h6cjyesnj44bbyQ1vvvw5P373LauzKcY4iIH9rmV+vmL/4ROrScXT7iCxW8sl0XuWVSlcDe/pu4H5YiZgKjT7Q83qbE01m5E5y24bOHghebvM4ceNbTIpqZuG0PQiqctz+r4nq0o6L9GhmbXcP22JfaAbDiwXU1xm8T4QgudQN6ymU87OlujNjrYfKF2i7QMRODae88KhK0e/rYUtYaRh0PSf4/PiT6Rlp4ZI+ok6RCtkGvIHuNRzyfRcfn3+f0a8ukKWFjjj4AdgXLxPX59GG4NR+LYnWXnt9bHGDx4zdvN9DHRtN052xDogoFPxlhLTyG0Q2V9MSeSEWotUNkpTN/aedAKKWkPX9yP41rHbbDDGoK0Zv59GOYnrdtaQUhBprBX6foialOTzzPOcECNt246Rc5qiKmVq0NSEocdYh8slpvoUuWqtkdSZXjZc6zJJ/zDy3lkbJTEqz8A5FGIdSDFgDFgj0lWNdP6zzJIZSWeyCpIe+ShJAMZKaZyR1x2DlwaIVmiiHLCjxxAFAtd3VFlBaQy73RO7p0esMQTk76d0muKLpW7oRWrvihxGxY91IpOMJwUJYDNpePdB4t2MSaSxKNPaYLV41q2Ve2gaI8OowjoV/P5UuI2qxRQTfTdgM0eWZRRW7JRt31Pva2Ke47Jx8iZ3CM5lBB1Gj33kOKa45FXJMDY6Hzdb7Pm5sGmAzBi6MMi9YDKMdRDkXtR9L8Wc1eM+FTlB06SgOzUhhOVhxulbCHIPaf353n2WUxspTgc/jNO2z0W3xOMl0BJdbLQW6FmI41RXwKxN7AiDwPOSkoOVQEj/8a+Tfe70wT3bY5SW4AQlaTFWO1FvxkiWF6KCCZGP9/f8r//b/0TfQ5kZ2raDCIPv8duBl+9e87uvv+VXf/olZ+eJvhn46hdfclHN+Ovf/pb1YsW3P/yINZbldMKh7Ugojk2Ly8X69sXra26fNnRtx/nlJa/O1vx4f8vZZEnfdRIB6xx5UXLz8ZZ3L1/QRs+HD49i5ex6jLUM/YDLpWCmj7x+seTjzROXqzXL64rf/PYbFvOKPM/kPi9EIZYVOatJxe3HGzIDmdHMJhaXEvOykmYC0Pael+dzplXJ/mkvjJJiTnNoWVSO+S9e8f73NxQh8qu/fMeHmy0zpwla0X+65cW65OP9Ft17vFK8ujpne2iIQ6BpRKUwcQX5esYQItYaNvdbVpdL1i/PabctFxcFb756zYdvf6C5feT89QVDUMytoioz6u2OsppzqEq6pmO6mNEdGvIzg4qJph2YGsN8VqGih5hLXLVxWCWWgfVqKutECnQ+Uh89715f8tB2GB9ZTguy3JEXOU0MhBRIHnJj+dn1miEk6l3H1dUaP3TkyXD25Rf86//j3/H0/oG3v3rHhx8feLNcMJ2dieK16ZnMF9S7LcfDkWT0qLrzaHuKgBa5ed95rNUUzlE3Peo+ivoly1m/KNk9bekO9fPaZ5Xwi5yzhBCoqoqmbelGzknwfizn5BnGWkI78PF+S1/XTNZLcr2ja/bEMJB0RjQyYRe11+lh+6zWFEk8CItB5CBKyQT4dMh5VpjyOTBBKUYw5meazz/6NQ4PROEyFj8jH+k0oRZekqgo0wiZTzFIxC5jsz3Gz/89BEBqtaHrICXyohDQ5EmxFoIwvrSmDyemV0aK0PfteGAUa46zFm1FfeJ7ef4ZlSUnxYhSmiy3Y6yrH2s+SENEOy0D1TJnPS1ZVAWTIiczhnmZMSlz6sOB4+FI6RwpRYamJVfQ1S2ZHRtaKAgCsvQ+EEx4bsxoY+j7Mbnm9LsHaY6ckm8AUZyOe1SeZbjcYa3DZcI+SfEnqUQxjQwK9RwlLxGvCRVE3anVqeEjUzs9MhrHdxkQa5Pve6zR6NHuRIwkLw0InTlQMCsnvFpf8q39nqBgOSnZ98MYVyznqNxphkM98tgC1llZk5XcH2hFXuT0+47ptKTvPG5SUOSRzW4HCXz0xK6jnE7ZP21kuKKlEdQda7LM4buevvcUVUmvxJY9mTqO+wN5mdMcj8+sGmGTGOrdnr4VuP10MSOGyPFQc/fhE4vzC6rZnFYf6dqGtm1QSlNOKqkzTu+V1iN7RdRHnZf15+HmVlTMSIRufdxRlhXOZigf6IcWp06wV3i2wIyXOjFelKh8SH78iRFGbs2J1yPrRfpsm5HfTL6P3ByiFvpPUJH9kTRCxg6w1mAUcQjP/zX0PfXG8+bVNb7uuL+/p+s6zhYrlvMFf/f3/x7vYTmd4mPPdtPz4uKM6/mKu/t7QtOwXK959eolxlp2W1nQh+mUqqwYmk7ixaKnqBTTiaVpau5v7gkRhmGQHGujadoxvgiYTEv2hwMptnR1B7mmr1v0ZMrj/YblesXxcJRkh+diVLqXBoXvWpzJeHh4YrU+ow+Brg3Uw4H6uGdSTZhVBXLzJVT0LKcVm80jQzuQVxbfDhz7mq4PTOcVV1drqrklS2Cco7p+QVVl7L7d8+JsQR08TzcPfPFiidOatm6JxpJNK+gjVhl0kA6dzcTTHQdPaSzL2Rw3qciLivV6Tdt6smomEZqDTMjrXct2t2e5nJPljukqoywm1Icd1jliP5BX4vefTkqqvGTz8EQYBlJRYK1icRE5r1u2d3ccjzWZcxhjyWZTJvMpbSOpG77zzC/OGPqeVZ6xmhXsNzvqpqWYTalmBb/KFfXuwPuHJ+aZYTqd8/6T58urgo+bPduHJ64u19irNZunDX/3+1t+8fqCD7c3/MkX73j94opt/cTdfc8vfn5Ne+jxfks5meJTZD6dyOYDHI5HytmEV1eXfLx9YL8/4FPixfUVVVGyedxw2B+YzGY8HY+4zFEUOW07JlMcW/bbA+WLFdVkQu8Tu+0B5xxRJ3SExWzCsW5FhWAt22ODs5a268mynGlZ0A0D28cN5VXO9fkS37Xii1QJ68Ty8uK84G7bcv/dwBAimdEjffsnUxglsbtDOG32Y+Pj1Ax5fnL/MNdPkx4Sz+e8Z/l7QqHVaAMIHt+3MpXR5nNWvRo5CiExnUxpDke6thtlpdJ06LqOvh+eD9xqlHnG8WeemiEnMJ1K6XniffrttFa4Maq38YNsiOMGnVKk61p8CPS9rIHGCEV96Dty5/BaCX/H5NT1gbyqmExLmvo4pjWJneUEC20bUShV0wnGaJrDkTAMYnsZJacn+2GWO6xW9E1N9BrrHM4oqszRtGIPMVpjc7HeaOIIezXj9FzSAGL06Ci4NUZ/sNYGbcAlSRnR2jApC5Q2+N7RN42wl2JABc+pitbjezwrcpyCpmvGqQJYJ82QFAW4ppSiLHK6fmC/29HWibzI0Aaih7zMSEFYe9FHkk7UbUdSsqb3fY+2TuL7tBpTstxz9KErHLYXP33vBWyYjHlOZZC0FfEm98NA3zlaa8nznNwJP+VY18RDoqhKAPKR6xGjx1mHzWSKjtYcjlKAOGdpB89uv2c1EctmkWUYrchG5k30/rlAz5xjGG86Y4x4pFHEnyhkkj7dlHL4F4XXiCRU8kylFMUerLQolEbmiBkjensve7O1FmsTfghjws7o8VUy1TNK9vI8z2WC6UWlpT8/GP/411hInYx0ISaBt2oB7GbOMfh+XEMUvfc4l6MT5ET+429/zfe/+5aI42G/xwdpaPnYMK0KfvjuPT/78ppPHz5xNp8xW005bnbEITDJHLnVxKEH62iHwKKa8t2NwFP37cDV1SW/++Z7jNVcvLjEGRlyTB1AYnvY8WK9ZgiRm7t7zpYLMIqP728Iw4BP0PUem0S9c7mc0XUDu+OR7z9uuJhMuLxY8fvvvmU+LcizjMxozmaVfDbGiqzcD/zqzQvuHx+pbODw1PL65Yp8YiBqUtdxtphSOsfh0DBfL1nOCn783XsuF1NSZnm62/Dlu0vMcsrXf/1bvvr5GzyBoR/YNx31w45SK2ZvLplVOd2x5fL1OTEm+QwidF7UNhqBQL96vabrAkPdiNomRZ7unvjy9SXqS83+YUcxrWiahm7wrF+9ZOgaznxOjeLpYcfq1SWLScnDp9vnWN37hy0vr1csJyVPnx6ZzQuUMjRNx3F/xOYZKkSs1azPp9w+7CidI6pE6ALN4MmLDNMPdF1gtSjZ7luiNTjrmJxNub3ZkGvNwcLf/J//nrPplDf/4uf823/zN7x9cUVeWEJoOYw1j1UwWSyonKbfHTgcG0iRMABGoa0hHj3WacKQiN6TF45jO+Af97x4cY5BURYZIbO0/cDQ9dg8Ixsn1X0noNWTcvLURNXjUNKHgPUS1Tv0PR/e31DNlzR1R9CevIJpJRwaO8Z6yp36WdVx4muk51bGeHRJ/KThcdJ/KBitkaempfrJV/5BrnEdfW7UqJOtUH4/M6oSwth4flbDpPFwN742aYLIsEtFmaB3bSOJO8biu1bUVEOPTpqqnBDLxH
gitextract_uedhrux4/
├── .gitignore
├── README.md
└── src/
├── jonasz/
│ ├── __init__.py
│ ├── cifar10/
│ │ ├── __init__.py
│ │ └── cifar10_dataset.py
│ ├── constants.py
│ ├── experiments/
│ │ ├── 2018_08_28/
│ │ │ ├── __init__.py
│ │ │ ├── exp_append_channels.py
│ │ │ ├── exp_append_simple.py
│ │ │ ├── exp_condition.py
│ │ │ ├── exp_condition_4vars.py
│ │ │ ├── exp_condition_8vars.py
│ │ │ ├── exp_consistency_30.py
│ │ │ ├── exp_consistency_300.py
│ │ │ ├── exp_gradual_loss.py
│ │ │ ├── exp_noinfo.py
│ │ │ ├── exp_unmask.py
│ │ │ └── exp_vanilla.py
│ │ └── __init__.py
│ ├── gan/
│ │ ├── __init__.py
│ │ └── evaluation.py
│ ├── lib/
│ │ ├── __init__.py
│ │ ├── datasets.py
│ │ ├── presentation_model_features.py
│ │ ├── progressive_infogan_ipynb_utils.py
│ │ ├── tensor_util.py
│ │ └── util.py
│ ├── notebooks/
│ │ └── generator.ipynb
│ ├── nvidia_celeb/
│ │ ├── __init__.py
│ │ ├── celeba_align_dataset.py
│ │ └── celeba_hq_dataset.py
│ ├── progressive_infogan/
│ │ ├── __init__.py
│ │ ├── create_animation.py
│ │ ├── export_utils.py
│ │ ├── gcloud_training/
│ │ │ ├── __init__.py
│ │ │ ├── command.sh
│ │ │ ├── config.yaml
│ │ │ ├── config_4gpus.yaml
│ │ │ ├── config_8gpus.yaml
│ │ │ ├── determine_config_yaml.py
│ │ │ ├── task.py
│ │ │ └── test.py
│ │ ├── info_utils.py
│ │ ├── network_utils.py
│ │ ├── networks.py
│ │ ├── progressive_infogan_lib.py
│ │ ├── progressive_infogan_losses.py
│ │ ├── run_evaluation.py
│ │ └── train.py
│ └── tools/
│ └── tensorboard_gcloud.py
└── setup.py
SYMBOL INDEX (282 symbols across 31 files)
FILE: src/jonasz/cifar10/cifar10_dataset.py
class DatasetParams (line 16) | class DatasetParams(util.Params):
method get_allowed_params_with_defaults (line 17) | def get_allowed_params_with_defaults(self):
method validate (line 39) | def validate(self):
function _load_from_gcs (line 48) | def _load_from_gcs(gcs_bucket, data_dir, filename):
function _load_dict (line 59) | def _load_dict(gcs_bucket, data_dir, filename):
function _load_train_data (line 76) | def _load_train_data(gcs_bucket, data_dir):
function _write_data_to_tfrecord (line 89) | def _write_data_to_tfrecord(data_dict, path):
function write_train_data_to_tfrecord (line 99) | def write_train_data_to_tfrecord(target_path, data_dir):
function write_test_data_to_tfrecord (line 104) | def write_test_data_to_tfrecord(target_path, data_dir):
function _load_test_data (line 109) | def _load_test_data(gcs_bucket, data_dir):
function _random_crop (line 113) | def _random_crop(img, params):
function _random_shift (line 129) | def _random_shift(img, params):
function _image_noise (line 144) | def _image_noise(params):
function _shift_pixel_values (line 162) | def _shift_pixel_values(img, params):
function _example_to_img_and_label (line 168) | def _example_to_img_and_label(serialized_example):
function get_train_input_fn (line 178) | def get_train_input_fn(params, batch_size=128):
function get_test_input_fn (line 219) | def get_test_input_fn(params, batch_size=128):
function _get_class_name (line 244) | def _get_class_name(label):
function _test_params1 (line 259) | def _test_params1(shuffle=True):
function _test_params2 (line 273) | def _test_params2():
function test1 (line 279) | def test1():
function test2 (line 294) | def test2():
function test3 (line 309) | def test3():
function test4 (line 321) | def test4():
function test5 (line 333) | def test5():
FILE: src/jonasz/experiments/2018_08_28/exp_append_channels.py
function training_params (line 16) | def training_params(is_gcloud=False, output_dir=None):
function main (line 107) | def main(*args):
FILE: src/jonasz/experiments/2018_08_28/exp_append_simple.py
function training_params (line 16) | def training_params(is_gcloud=False, output_dir=None):
function main (line 106) | def main(*args):
FILE: src/jonasz/experiments/2018_08_28/exp_condition.py
function training_params (line 16) | def training_params(is_gcloud=False, output_dir=None):
function main (line 105) | def main(*args):
FILE: src/jonasz/experiments/2018_08_28/exp_condition_4vars.py
function training_params (line 16) | def training_params(is_gcloud=False, output_dir=None):
function main (line 106) | def main(*args):
FILE: src/jonasz/experiments/2018_08_28/exp_condition_8vars.py
function training_params (line 16) | def training_params(is_gcloud=False, output_dir=None):
function main (line 106) | def main(*args):
FILE: src/jonasz/experiments/2018_08_28/exp_consistency_30.py
function training_params (line 15) | def training_params(is_gcloud=False, output_dir=None):
function main (line 109) | def main(*args):
FILE: src/jonasz/experiments/2018_08_28/exp_consistency_300.py
function training_params (line 15) | def training_params(is_gcloud=False, output_dir=None):
function main (line 109) | def main(*args):
FILE: src/jonasz/experiments/2018_08_28/exp_gradual_loss.py
function training_params (line 17) | def training_params(is_gcloud=False, output_dir=None):
function main (line 116) | def main(*args):
FILE: src/jonasz/experiments/2018_08_28/exp_noinfo.py
function training_params (line 15) | def training_params(is_gcloud=False, output_dir=None):
function main (line 93) | def main(*args):
FILE: src/jonasz/experiments/2018_08_28/exp_unmask.py
function training_params (line 18) | def training_params(is_gcloud=False, output_dir=None):
function main (line 126) | def main(*args):
FILE: src/jonasz/experiments/2018_08_28/exp_vanilla.py
function training_params (line 15) | def training_params(is_gcloud=False, output_dir=None):
function main (line 105) | def main(*args):
FILE: src/jonasz/gan/evaluation.py
class EvalParams (line 14) | class EvalParams(util.Params):
method get_allowed_params_with_defaults (line 15) | def get_allowed_params_with_defaults(self):
function _inception_logits (line 27) | def _inception_logits(gan):
function _inception_final_pool (line 36) | def _inception_final_pool(gan):
function _write_summaries_internal (line 51) | def _write_summaries_internal(eval_dir, sess, cur_global_step, summaries...
function _write_summaries (line 61) | def _write_summaries(eval_dir, sess, cur_global_step, summaries=None,
function _accumulate_n (line 72) | def _accumulate_n(tensor, sess, n):
function _maybe_calc_inception_score (line 92) | def _maybe_calc_inception_score(gan, sess, params):
function _maybe_calc_frechet_inception_distance (line 107) | def _maybe_calc_frechet_inception_distance(gan, sess, eval_params):
function calc_mean (line 125) | def calc_mean(vals):
function _maybe_calc_infogan_isolation (line 131) | def _maybe_calc_infogan_isolation(gan, sess, training_params, version='m...
function block_to_corresponding_cont_coords (line 232) | def block_to_corresponding_cont_coords(tp, block_id):
function _maybe_calc_infogan_metrics (line 253) | def _maybe_calc_infogan_metrics(gan_model_dict, sess, training_params):
class _RestoreGANSession (line 322) | class _RestoreGANSession(object):
method __init__ (line 323) | def __init__(self, model_dir = None, vars_to_restore=None,
method __enter__ (line 339) | def __enter__(self):
method __exit__ (line 358) | def __exit__(self, *args):
function _get_side_for_dynamic_mse (line 364) | def _get_side_for_dynamic_mse(i, steps, img_side):
class maybe_gpu_tower_scope (line 370) | class maybe_gpu_tower_scope(object):
method __init__ (line 371) | def __init__(self, use_gpu_tower_scope):
method __enter__ (line 380) | def __enter__(self):
method __exit__ (line 384) | def __exit__(self, *args):
function _draw_images (line 389) | def _draw_images(training_params, gan_fn, how_many=32, type_='real',
function _calc_msssim_diversity_from_images (line 428) | def _calc_msssim_diversity_from_images(images):
function _maybe_calc_msssim_diversity (line 449) | def _maybe_calc_msssim_diversity(gan_fn, training_params):
function evaluate (line 496) | def evaluate(gan_fn, training_params):
FILE: src/jonasz/lib/datasets.py
class DummyDatasetParams (line 8) | class DummyDatasetParams(util.Params):
method get_allowed_params_with_defaults (line 9) | def get_allowed_params_with_defaults(self):
function get_dummy_dataset_input_fn (line 17) | def get_dummy_dataset_input_fn(params, batch_size):
function get_input_fn (line 34) | def get_input_fn(dataset_params, batch_size):
FILE: src/jonasz/lib/progressive_infogan_ipynb_utils.py
function restart_server (line 26) | def restart_server():
function get_images (line 45) | def get_images(model, gen_inputs):
function manual_animation (line 56) | def manual_animation(model, gen_inp, coords=range(64,80)):
function interpolate (line 102) | def interpolate(model, inp, coord, steps=3):
function blur (line 112) | def blur(a, stddev):
function diff2 (line 120) | def diff2(a, b):
function avg_change (line 128) | def avg_change(model, coord,
FILE: src/jonasz/lib/tensor_util.py
function nchw_to_nhwc (line 4) | def nchw_to_nhwc(images):
function nhwc_to_nchw (line 8) | def nhwc_to_nchw(images):
function nchw_to_nhwc_single (line 12) | def nchw_to_nhwc_single(images):
function nhwc_to_nchw_single (line 16) | def nhwc_to_nchw_single(images):
FILE: src/jonasz/lib/util.py
function CHW_to_HWC (line 23) | def CHW_to_HWC(img):
function imshow (line 30) | def imshow(img, minv=None, maxv=None, turn_off_axis=True):
function show_imgs (line 50) | def show_imgs(imgs, data_type='HWC', cols=5, minv=None, maxv=None,
function random_crop (line 71) | def random_crop(img, max_shift=7, can_flip=False, data_type='CHW'):
class Params (line 98) | class Params(object):
method __init__ (line 99) | def __init__(self, **kwargs):
method validate (line 107) | def validate(self):
method overwrite (line 110) | def overwrite(self, **kwargs):
method __getattr__ (line 113) | def __getattr__(self, name):
method default_param (line 116) | def default_param(self, key, val):
method __getstate__ (line 119) | def __getstate__(self):
method __setstate__ (line 122) | def __setstate__(self, vals):
method __repr__ (line 125) | def __repr__(self):
method get_allowed_params_with_defaults (line 136) | def get_allowed_params_with_defaults(self):
method __deepcopy__ (line 140) | def __deepcopy__(self, memo):
function reset_timer (line 145) | def reset_timer():
function time_elapsed (line 149) | def time_elapsed(log=True):
function assert_set_covered (line 160) | def assert_set_covered(covered, covered_by):
function get_time_str (line 165) | def get_time_str(microseconds=False):
function get_new_dir (line 171) | def get_new_dir(base_path, suffix):
function serialized_example (line 175) | def serialized_example(float_features, int_features):
function _chunks (line 186) | def _chunks(iterable, chunk_size):
class _DummyCtxMgr (line 193) | class _DummyCtxMgr():
method __init__ (line 194) | def __init__(self, *args):
method __enter__ (line 196) | def __enter__(self):
method __exit__ (line 198) | def __exit__(self, exc_type, exc_value, traceback):
function tensorflow_model_server_predict (line 202) | def tensorflow_model_server_predict(host_port=None,
function _create_model_config_file (line 294) | def _create_model_config_file(models):
function start_tensorflow_model_server (line 326) | def start_tensorflow_model_server(port=8100, models=None, interruptible=...
class TFModelServer (line 353) | class TFModelServer(object):
method __init__ (line 354) | def __init__(self, port=8100, models=None, initial_sleep=None,
method __enter__ (line 366) | def __enter__(self):
method __exit__ (line 374) | def __exit__(self, exc_type, exc_value, traceback):
class _TFLoggingFilter (line 378) | class _TFLoggingFilter(logging.Filter):
method filter (line 382) | def filter(self, record):
class TFFileLogger (line 388) | class TFFileLogger(object):
method __init__ (line 396) | def __init__(self, dir_path, is_gcloud):
method __enter__ (line 401) | def __enter__(self):
method __exit__ (line 417) | def __exit__(self, *args):
function tf_logging_decorator (line 424) | def tf_logging_decorator(f):
function get_optimizer (line 435) | def get_optimizer(params, scope, global_step=None):
function construct_experiment_output_dir (line 475) | def construct_experiment_output_dir(fname):
FILE: src/jonasz/nvidia_celeb/celeba_align_dataset.py
class CelebADatasetParams (line 10) | class CelebADatasetParams(util.Params):
method get_allowed_params_with_defaults (line 11) | def get_allowed_params_with_defaults(self):
function get_dataset (line 23) | def get_dataset(is_gcloud=False, **kwargs):
function open_img (line 33) | def open_img(path):
function img_generator (line 40) | def img_generator(imgs_path, gcs_bucket=None):
function process (line 49) | def process(img, params):
function get_train_input_fn (line 74) | def get_train_input_fn(params, batch_size=128):
FILE: src/jonasz/nvidia_celeb/celeba_hq_dataset.py
class CelebAHQDatasetParams (line 12) | class CelebAHQDatasetParams(util.Params):
method get_allowed_params_with_defaults (line 13) | def get_allowed_params_with_defaults(self):
method validate (line 27) | def validate(self):
function get_dataset_params (line 31) | def get_dataset_params(
function process (line 47) | def process(img, params, input_img_size):
function get_train_input_fn (line 75) | def get_train_input_fn(params, batch_size=128):
FILE: src/jonasz/progressive_infogan/create_animation.py
function map_channel (line 64) | def map_channel(ch, ref):
function adjust_img_to_ref (line 73) | def adjust_img_to_ref(img, ref):
function _animate (line 80) | def _animate(imgs, interval=20, size=8, export_path=None, resolution=800):
function _write_on_img (line 115) | def _write_on_img((img, text)):
function _animate_single_image (line 126) | def _animate_single_image(
function _make_grid_img (line 210) | def _make_grid_img(imgs, grid_size):
function create_animation (line 218) | def create_animation(
function _make_batches (line 292) | def _make_batches(data, batch_size):
function main (line 306) | def main(*args):
FILE: src/jonasz/progressive_infogan/export_utils.py
function _define_flags (line 12) | def _define_flags():
function _restore_average_from_multiple_checkpoints (line 31) | def _restore_average_from_multiple_checkpoints(saver_for_restore,
function export_newest_savedmodel (line 51) | def export_newest_savedmodel(training_params,
function _maybe_restore_from_path (line 167) | def _maybe_restore_from_path(sess, path, vars_to_restore=None,
function _handle_legacy_vars_to_restore (line 183) | def _handle_legacy_vars_to_restore(vars_to_restore, path):
function maybe_restore (line 205) | def maybe_restore(sess, training_params=None, vars_to_restore=None,
function checkpoint (line 239) | def checkpoint(sess, cur_global_step, training_params):
function should_checkpoint (line 254) | def should_checkpoint(training_params, steps_passed, secs_passed):
function write_summaries (line 265) | def write_summaries(model_dir, sess, cur_global_step, summaries=None,
function _extract_step (line 284) | def _extract_step(path):
function _newest_checkpoints (line 290) | def _newest_checkpoints(training_params, num_checkpoints=5,
function main (line 301) | def main(*args):
FILE: src/jonasz/progressive_infogan/gcloud_training/determine_config_yaml.py
function main (line 6) | def main(*args):
FILE: src/jonasz/progressive_infogan/gcloud_training/test.py
function main (line 4) | def main(*args):
FILE: src/jonasz/progressive_infogan/info_utils.py
class InfoGanSummary (line 8) | class InfoGanSummary(object):
method __init__ (line 9) | def __init__(self, training_params, generator_inputs, sample_t, reps=3):
method _feed_dict_from_gen_inputs (line 36) | def _feed_dict_from_gen_inputs(self, cur_generator_inputs):
method _construct_cont_infogan_images (line 44) | def _construct_cont_infogan_images(self, sess, coord, cur_generator_in...
method _construct_cat_infogan_images (line 60) | def _construct_cat_infogan_images(self, sess, coord, cur_generator_inp...
method _fetch_generator_inputs (line 80) | def _fetch_generator_inputs(self, sess, generator_inputs):
method construct_feed_dict (line 96) | def construct_feed_dict(self, sess):
method _infogan_images_summary (line 112) | def _infogan_images_summary(self, num_coords, prefix, grid_side,
FILE: src/jonasz/progressive_infogan/network_utils.py
function upscale2d (line 10) | def upscale2d(x, factor=2, data_format='NCHW'):
function downscale2d (line 26) | def downscale2d(x, factor=2, data_format='NCHW'):
function downgrade2d (line 39) | def downgrade2d(x, factor=2, data_format='NCHW'):
function concat_features_stddev (line 45) | def concat_features_stddev(layer):
function pixel_norm (line 61) | def pixel_norm(x, axis, epsilon=1e-8, scope='pixel_norm'):
function append_one_hot_to_tensor (line 67) | def append_one_hot_to_tensor(tensor, one_hot):
function batch_norm (line 94) | def batch_norm(net, axis, scope='batch_norm'):
function batch_norm_in_place (line 109) | def batch_norm_in_place(net, axis, scope='batch_norm_in_place',
function layer_norm (line 129) | def layer_norm(net, axis, scope='layer_norm'):
function dense (line 152) | def dense(inp, units=None, scope='linear', weight_norm=None):
function _prod (line 180) | def _prod(s):
function get_weights (line 185) | def get_weights(shape, weight_norm):
function conv (line 204) | def conv(net, kernel_size=3, strides=1, filters=None, scope='conv',
function conv_trans (line 225) | def conv_trans(*args, **kwargs):
function _norm (line 230) | def _norm(net, axis, version, scope='norm', is_training=None):
function norm (line 247) | def norm(net, axis, version, scope='norm', is_training=None, gpu_id=None,
function _get_shape (line 259) | def _get_shape(tensor):
function condition_tensor (line 268) | def condition_tensor(tensor, conditioning, act=None):
FILE: src/jonasz/progressive_infogan/networks.py
class GeneratorParams (line 13) | class GeneratorParams(util.Params):
method get_allowed_params_with_defaults (line 14) | def get_allowed_params_with_defaults(self):
method validate (line 55) | def validate(self):
class DiscriminatorParams (line 60) | class DiscriminatorParams(util.Params):
method get_allowed_params_with_defaults (line 61) | def get_allowed_params_with_defaults(self):
method validate (line 91) | def validate(self):
function phase_progress_to_alpha (line 95) | def phase_progress_to_alpha(block, phase, progress):
function generator_fn (line 111) | def generator_fn(generator_inputs,
function discriminator_fn (line 414) | def discriminator_fn(image,
function _img_summary (line 683) | def _img_summary(name, img):
function _flatten (line 692) | def _flatten(net):
function _get_progressive_mask (line 698) | def _get_progressive_mask(training_params, phase_t, phase_progress_t):
function get_gpu_batch_size (line 723) | def get_gpu_batch_size(tensor, training_params):
function _split_structured_input (line 729) | def _split_structured_input(tp, inp, num_vars, var_name_prefix):
function _split_structured_continuous_input (line 740) | def _split_structured_continuous_input(tp, inp, phase, phase_progress):
function _split_structured_categorical_input (line 749) | def _split_structured_categorical_input(tp, inp):
function _block_id_to_structured_vars_internal (line 767) | def _block_id_to_structured_vars_internal(phase_to_coords, coord_to_tens...
function _block_id_to_structured_vars (line 785) | def _block_id_to_structured_vars(generator_inputs, tp, phase, phase_prog...
FILE: src/jonasz/progressive_infogan/progressive_infogan_lib.py
class GenInput (line 11) | class GenInput(object):
method __init__ (line 13) | def __init__(self, noise, cont_structured_input, cat_structured_input):
method random (line 19) | def random(cls, noise_stddev):
method copy (line 26) | def copy(self):
method as_dict_key (line 33) | def as_dict_key(self):
method _create_noise (line 39) | def _create_noise(cls, noise_size, noise_stddev):
method _create_cat_structured_input (line 43) | def _create_cat_structured_input(cls):
method as_serialized_example (line 46) | def as_serialized_example(self):
function random_request (line 58) | def random_request(num_images=None, noise_stddev=None, seed=None):
function _query_tf_model_server (line 66) | def _query_tf_model_server(gen_inputs,
function gen_inputs_to_images (line 86) | def gen_inputs_to_images(gen_inputs,
FILE: src/jonasz/progressive_infogan/progressive_infogan_losses.py
function _flatten (line 14) | def _flatten(net):
function _vanilla_consistency_loss (line 20) | def _vanilla_consistency_loss(cur_rgb, prev_rgb, block_id, tp):
function _msssim256_consistency_loss (line 34) | def _msssim256_consistency_loss(cur_rgb, prev_rgb, block_id, tp):
function _add_consistency_loss (line 56) | def _add_consistency_loss(gan_loss, gan_model_dict, training_params):
function _add_categorical_mutual_information_penalty (line 106) | def _add_categorical_mutual_information_penalty(gan_loss,
function unweighted_mutual_information_penalty_per_coord (line 145) | def unweighted_mutual_information_penalty_per_coord(gan_model_dict,
function _add_continuous_mutual_information_penalty (line 169) | def _add_continuous_mutual_information_penalty(gan_loss,
function _add_drift_loss (line 238) | def _add_drift_loss(gan_loss, gan_model_dict, training_params):
function construct_gan_loss (line 251) | def construct_gan_loss(training_params, gan_model_dict):
FILE: src/jonasz/progressive_infogan/run_evaluation.py
function _find_phase (line 35) | def _find_phase(training_params, checkpoint_num):
function _run_evaluation_with_logging (line 47) | def _run_evaluation_with_logging(training_params, step, original_output_...
function get_output_dir (line 63) | def get_output_dir(output_dir_base, job_id, step):
function run_evaluation (line 68) | def run_evaluation(training_module_name, job_id, step, output_dir_base,
FILE: src/jonasz/progressive_infogan/train.py
class TrainingParams (line 32) | class TrainingParams(util.Params):
method get_allowed_params_with_defaults (line 33) | def get_allowed_params_with_defaults(self):
method validate (line 104) | def validate(self):
method max_steps (line 139) | def max_steps(self):
method batch_size (line 148) | def batch_size(self):
method image_side (line 154) | def image_side(self):
method phase (line 160) | def phase(self):
method batch_size_per_gpu (line 165) | def batch_size_per_gpu(self):
method infogan_cont_depth_to_vars (line 170) | def infogan_cont_depth_to_vars(self):
method infogan_cont_num_vars (line 185) | def infogan_cont_num_vars(self):
method target_side_log (line 194) | def target_side_log(self):
method block_ids (line 200) | def block_ids(self):
function _specialize_training_params_for_phase (line 204) | def _specialize_training_params_for_phase(tp, phase):
function _gan_inputs (line 217) | def _gan_inputs(training_params):
function _slice_gan_inputs (line 258) | def _slice_gan_inputs(generator_inputs, images, num_gpus=1):
function _get_phase (line 271) | def _get_phase(training_params, global_step):
function _gan_fn (line 283) | def _gan_fn(training_params, is_training=True, return_dict=False, gpu_id...
function _gan_fn_from_inputs (line 300) | def _gan_fn_from_inputs(training_params,
function _should_do_eval (line 382) | def _should_do_eval(params, time_passed, steps_passed):
function generator_params_ema (line 392) | def generator_params_ema(training_params, gan_model, gen_train_step):
class GradsAccumulator (line 405) | class GradsAccumulator(object):
method __init__ (line 406) | def __init__(self):
method add (line 410) | def add(self, grads_and_vars):
method _summarize_grad_health (line 415) | def _summarize_grad_health(self, t, name):
method _check_gradient_health (line 424) | def _check_gradient_health(self, grad, name):
method avg_grads_and_vars (line 436) | def avg_grads_and_vars(self):
function _get_update_ops (line 459) | def _get_update_ops(gen_scope, dis_scope, check_for_unused_ops=True):
function _assign_to_device (line 478) | def _assign_to_device(device, ps_device=None):
function _get_cur_global_step (line 488) | def _get_cur_global_step(training_params):
function get_vars_device (line 499) | def get_vars_device(training_params):
function _train_gan_multi_gpu (line 510) | def _train_gan_multi_gpu(training_params):
function _download_checkpoint (line 718) | def _download_checkpoint(src_path, dst_path, num_checkpoint=None):
function _single_gpu_training_params (line 752) | def _single_gpu_training_params(training_params):
function _create_animation (line 761) | def _create_animation(training_params, saved_model_subdir, seeds):
function _evaluation (line 793) | def _evaluation(training_params):
function _run_single_phase_training (line 801) | def _run_single_phase_training(training_params, phase):
function _train_and_evaluate_gan (line 826) | def _train_and_evaluate_gan(training_params):
function _create_animations (line 848) | def _create_animations(training_params):
function run_training (line 858) | def run_training(training_params):
function _make_grid (line 864) | def _make_grid(imgs, grid_side):
function _flatten (line 870) | def _flatten(net):
function _default_dataset_params (line 876) | def _default_dataset_params():
FILE: src/jonasz/tools/tensorboard_gcloud.py
function syscmd (line 7) | def syscmd(cmd):
function parse_lines (line 11) | def parse_lines(content):
function promptuser (line 17) | def promptuser(lines):
function get_job_ids (line 40) | def get_job_ids():
function download_last_events (line 59) | def download_last_events(base_dir, job_id):
function main (line 79) | def main():
Condensed preview — 51 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (6,796K chars).
[
{
"path": ".gitignore",
"chars": 161,
"preview": "*.pyc\n*.swp\n*.swo\nsrc/dist/*\nsrc/*.egg-info/\n.ipynb_checkpoints\nfrozen_inception_v1_2015_12_05.tar.gz\nsrc/jonasz/noteboo"
},
{
"path": "README.md",
"chars": 8101,
"preview": "# Progressive InfoGAN\n\nProgressive InfoGAN combines two techniques:\n- progressive training (https://arxiv.org/abs/1710.1"
},
{
"path": "src/jonasz/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "src/jonasz/cifar10/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "src/jonasz/cifar10/cifar10_dataset.py",
"chars": 11007,
"preview": "\"\"\"Utility class for loading cifar 10 data.\n\nThe \"CIFAR-10 python version\" data format described at\nhttps://www.cs.toron"
},
{
"path": "src/jonasz/constants.py",
"chars": 487,
"preview": "# Where to write training data (checkpoints, logs, ...).\nTRAINING_OUTPUT_BASE_DIR = None\n\n# Datasets loca"
},
{
"path": "src/jonasz/experiments/2018_08_28/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "src/jonasz/experiments/2018_08_28/exp_append_channels.py",
"chars": 3473,
"preview": "DESCRIPTION = __file__ + \"\"\"\nFor each block from 2 to 6, the structured code is projected to a higher number\nof channels"
},
{
"path": "src/jonasz/experiments/2018_08_28/exp_append_simple.py",
"chars": 3383,
"preview": "DESCRIPTION = __file__ + \"\"\"\nFor each block from 2 to 6, the structured code is appended to the block's\ninput.\n\"\"\"\nimpor"
},
{
"path": "src/jonasz/experiments/2018_08_28/exp_condition.py",
"chars": 3392,
"preview": "DESCRIPTION = __file__ + \"\"\"\nEach block from 2 to 6 has 16 structured variables. The activations within the\nblock are co"
},
{
"path": "src/jonasz/experiments/2018_08_28/exp_condition_4vars.py",
"chars": 3420,
"preview": "DESCRIPTION = __file__ + \"\"\"\nEach block from 2 to 6 has 4 structured variables. The activations within the\nblock are con"
},
{
"path": "src/jonasz/experiments/2018_08_28/exp_condition_8vars.py",
"chars": 3420,
"preview": "DESCRIPTION = __file__ + \"\"\"\nEach block from 2 to 6 has 8 structured variables. The activations within the\nblock are con"
},
{
"path": "src/jonasz/experiments/2018_08_28/exp_consistency_30.py",
"chars": 3470,
"preview": "DESCRIPTION = __file__ + \"\"\"\nSame as master_condition, but consistency loss of 30. added.\n\"\"\"\nimport os\nimport re\nimport"
},
{
"path": "src/jonasz/experiments/2018_08_28/exp_consistency_300.py",
"chars": 3472,
"preview": "DESCRIPTION = __file__ + \"\"\"\nSame as master_condition, but consistency loss of 300. added.\n\"\"\"\nimport os\nimport re\nimpor"
},
{
"path": "src/jonasz/experiments/2018_08_28/exp_gradual_loss.py",
"chars": 3667,
"preview": "DESCRIPTION = __file__ + \"\"\"\nThe structured input is fed to block 2 by appending it to input noise. The\nMutual Informati"
},
{
"path": "src/jonasz/experiments/2018_08_28/exp_noinfo.py",
"chars": 3119,
"preview": "DESCRIPTION = __file__ + \"\"\"\nNo InfoGAN penalty at all.\n\"\"\"\nimport os\nimport re\nimport tensorflow as tf\nfrom jonasz.prog"
},
{
"path": "src/jonasz/experiments/2018_08_28/exp_unmask.py",
"chars": 3976,
"preview": "DESCRIPTION = __file__ + \"\"\"\nThe structured input is fed to block 2 by appending it to input noise. The\nMutual Informati"
},
{
"path": "src/jonasz/experiments/2018_08_28/exp_vanilla.py",
"chars": 3372,
"preview": "DESCRIPTION = __file__ + \"\"\"\nThe structured input is fed to block 2 by appending it to input noise.\n\"\"\"\nimport os\nimport"
},
{
"path": "src/jonasz/experiments/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "src/jonasz/gan/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "src/jonasz/gan/evaluation.py",
"chars": 21076,
"preview": "import tensorflow as tf\nimport time\nimport numpy as np\nimport math\nimport os\nfrom tensorflow.contrib import gan as tfgan"
},
{
"path": "src/jonasz/lib/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "src/jonasz/lib/datasets.py",
"chars": 1971,
"preview": "from jonasz.lib import util\nimport tensorflow as tf\nimport numpy as np\nfrom jonasz.nvidia_celeb import celeba_align_data"
},
{
"path": "src/jonasz/lib/presentation_model_features.py",
"chars": 3061,
"preview": "\"\"\"\nFull summary of featuers isolated in a run of exp_consistency_300.\n\"\"\"\n\n_descriptions = {\n 79: ('Size of the irises"
},
{
"path": "src/jonasz/lib/progressive_infogan_ipynb_utils.py",
"chars": 5533,
"preview": "import tensorflow as tf\nimport numpy as np\nimport os\nimport scipy\nimport random\nfrom jonasz.lib import util\nfrom jonasz."
},
{
"path": "src/jonasz/lib/tensor_util.py",
"chars": 320,
"preview": "import tensorflow as tf\n\n\ndef nchw_to_nhwc(images):\n return tf.transpose(images, [0, 2, 3, 1])\n\n\ndef nhwc_to_nchw(image"
},
{
"path": "src/jonasz/lib/util.py",
"chars": 16183,
"preview": "\"\"\"Miscellaneous utilities.\"\"\"\n\nimport tempfile\nimport traceback\nimport logging\nimport subprocess\nimport signal\nimport c"
},
{
"path": "src/jonasz/notebooks/generator.ipynb",
"chars": 6536878,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Setup\"\n ]\n },\n {\n \"cell_typ"
},
{
"path": "src/jonasz/nvidia_celeb/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "src/jonasz/nvidia_celeb/celeba_align_dataset.py",
"chars": 2849,
"preview": "import tensorflow as tf\nimport os\nfrom jonasz import constants\nfrom jonasz.lib import util\nimport lmdb\nimport cv2\nimport"
},
{
"path": "src/jonasz/nvidia_celeb/celeba_hq_dataset.py",
"chars": 3597,
"preview": "import tensorflow as tf\nimport os\nimport math\nfrom jonasz import constants\nfrom jonasz.lib import util\nfrom jonasz.lib i"
},
{
"path": "src/jonasz/progressive_infogan/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "src/jonasz/progressive_infogan/create_animation.py",
"chars": 12752,
"preview": "# python jonasz/progressive_infogan/create_animation.py \\\n# --saved_model_path gs://seer-of-visions-ml2/job___2018_07_"
},
{
"path": "src/jonasz/progressive_infogan/export_utils.py",
"chars": 12934,
"preview": "import os\nimport re\nimport time\nimport tensorflow as tf\nimport numpy as np\nfrom tensorflow.python.training import saver\n"
},
{
"path": "src/jonasz/progressive_infogan/gcloud_training/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "src/jonasz/progressive_infogan/gcloud_training/command.sh",
"chars": 1329,
"preview": "cur_path=`pwd`\ncur_dir=`basename $cur_path`\nif [ \"$cur_dir\" != \"master_thesis\" ]; then\n echo \"FAIL: This command should"
},
{
"path": "src/jonasz/progressive_infogan/gcloud_training/config.yaml",
"chars": 38,
"preview": "trainingInput:\n scaleTier: BASIC_GPU\n"
},
{
"path": "src/jonasz/progressive_infogan/gcloud_training/config_4gpus.yaml",
"chars": 112,
"preview": "trainingInput:\n scaleTier: CUSTOM\n masterType: complex_model_m_gpu\n workerCount: 0\n parameterServerCount: 0\n"
},
{
"path": "src/jonasz/progressive_infogan/gcloud_training/config_8gpus.yaml",
"chars": 112,
"preview": "trainingInput:\n scaleTier: CUSTOM\n masterType: complex_model_l_gpu\n workerCount: 0\n parameterServerCount: 0\n"
},
{
"path": "src/jonasz/progressive_infogan/gcloud_training/determine_config_yaml.py",
"chars": 795,
"preview": "#! /usr/bin/python\n\nimport tensorflow as tf\nimport importlib\n\ndef main(*args):\n module = args[0][1]\n output_dir = args"
},
{
"path": "src/jonasz/progressive_infogan/gcloud_training/task.py",
"chars": 890,
"preview": "import argparse\nimport tensorflow as tf\nfrom jonasz.progressive_infogan import train\nimport importlib\n\n\nif __name__ == '"
},
{
"path": "src/jonasz/progressive_infogan/gcloud_training/test.py",
"chars": 429,
"preview": "import tensorflow as tf\nimport importlib\n\ndef main(*args):\n print args\n module = args[0][1]\n output_dir = args[0][2]\n"
},
{
"path": "src/jonasz/progressive_infogan/info_utils.py",
"chars": 5231,
"preview": "import tensorflow as tf\nimport numpy as np\nfrom tensorflow.contrib import gan as tfgan\nimport copy\n\n\n# Not very pretty b"
},
{
"path": "src/jonasz/progressive_infogan/network_utils.py",
"chars": 9015,
"preview": "import tensorflow as tf\nfrom jonasz.lib import tensor_util\nimport numpy as np\nfrom tensorflow import keras\nimport math\nf"
},
{
"path": "src/jonasz/progressive_infogan/networks.py",
"chars": 33670,
"preview": "# TODO: try out weight normalization as an alternative to batch norm.\n\nimport tensorflow as tf\nfrom tensorflow import ke"
},
{
"path": "src/jonasz/progressive_infogan/progressive_infogan_lib.py",
"chars": 4875,
"preview": "import matplotlib\nimport tensorflow as tf\nimport numpy as np\nimport random\nfrom jonasz.lib import util\nimport matplotlib"
},
{
"path": "src/jonasz/progressive_infogan/progressive_infogan_losses.py",
"chars": 11352,
"preview": "import os\nimport math\nimport time\nimport functools\nimport tensorflow as tf\nimport numpy as np\nfrom jonasz.progressive_in"
},
{
"path": "src/jonasz/progressive_infogan/run_evaluation.py",
"chars": 3879,
"preview": "import os\nimport datetime\nfrom jonasz.progressive_infogan import create_animation\nimport importlib\nimport collections\nim"
},
{
"path": "src/jonasz/progressive_infogan/train.py",
"chars": 33872,
"preview": "import os\nimport collections\nimport copy\nimport multiprocessing\nimport random\nimport subprocess\nimport math\nimport cPick"
},
{
"path": "src/jonasz/tools/tensorboard_gcloud.py",
"chars": 2853,
"preview": "import tempfile, os, subprocess, shutil, sys, getopt, random\nimport time\nimport tensorflow as tf\n\nLATEST_EVENTS = 100\n\nd"
},
{
"path": "src/setup.py",
"chars": 402,
"preview": "from setuptools import find_packages\nfrom setuptools import setup\nimport sys, os, os.path\n\nREQUIRED_PACKAGES = [\n 'goog"
}
]
About this extraction
This page contains the full source code of the jonasz/progressive_infogan GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 51 files (6.5 MB), approximately 1.7M tokens, and a symbol index with 282 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.