[
{
"path": ".gitignore",
"content": "*~\n*.pyc\n.DS_Store\nDRIVE_datasets_training_testing"
},
{
"path": ".idea/libraries/R_User_Library.xml",
"content": "\n \n \n \n \n"
},
{
"path": ".idea/misc.xml",
"content": "\n\n \n"
},
{
"path": ".idea/modules.xml",
"content": "\n\n \n \n \n \n \n"
},
{
"path": ".idea/retinaunet-master.iml",
"content": "\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n"
},
{
"path": ".idea/workspace.xml",
"content": "\n\n \n \n \n \n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n plot\n run_GPU\n visualize\n path_data\n border_mode='same'\n compile\n average_mode\n self.DenseBlock\n self.num_seg_class\n model\n nn.module\n device\n as F\n batch_size\n check_path\n cudnn\n lr_schedule\n patches_imgs_train\n resu\n lr\n resume\n total\n best_iou\n RESUME\n \n \n padding='same\n padding='same'\n DenseBlock\n num_seg_class\n nn.Module\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n \n \n
\n \n \n \n \n \n \n \n \n 1531703474486\n \n \n 1531703474486\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n"
},
{
"path": "README.md",
"content": "* Implementation for paper [LadderNet: Multi-path networks based on U-Net for medical image segmentation\n](https://arxiv.org/abs/1810.07810)\n* This implementation is based on [orobix implementation](https://github.com/orobix/retina-unet). Main difference is the structure of the model.\n\n# Requirement\n* Python3.6\n* PyTorch 0.4\n* configparser\n\n# How to run\n* run ```python prepare_datasets_DRIVE.py``` to generate hdf5 file of training data\n* run ```cd src```\n* run ```python retinaNN_training.py``` to train\n* run ```python retinaNN_predict.py``` to test\n\n# Parameter defination\n* parameters (path, patch size, et al.) are defined in \"configuration.txt\"\n* training parameters are defined in src/retinaNN_training.py line 49 t 84 with notes \"=====Define parameters here =========\" \n\n# Pretrained weights\n* pretrained weights are stored in \"src/checkpoint\"\n* results are stored in \"test/\"\n\n# Results\nThe results reported in the `./test` folder are referred to the trained model which reported the minimum validation loss. The `./test` folder includes:\n- Model:\n - `test_model.png` schematic representation of the neural network\n - `test_architecture.json` description of the model in json format\n - `test_best_weights.h5` weights of the model which reported the minimum validation loss, as HDF5 file\n - `test_last_weights.h5` weights of the model at last epoch (150th), as HDF5 file\n - `test_configuration.txt` configuration of the parameters of the experiment\n- Experiment results:\n - `performances.txt` summary of the test results, including the confusion matrix\n - `Precision_recall.png` the precision-recall plot and the corresponding Area Under the Curve (AUC)\n - `ROC.png` the Receiver Operating Characteristic (ROC) curve and the corresponding AUC\n - `all_*.png` the 20 images of the pre-processed originals, ground truth and predictions relative to the DRIVE testing dataset\n - `sample_input_*.png` sample of 40 patches of the pre-processed original training images and the corresponding ground truth\n - `test_Original_GroundTruth_Prediction*.png` from top to bottom, the original pre-processed image, the ground truth and the prediction. In the predicted image, each pixel shows the vessel predicted probability, no threshold is applied.\n\nThe following table compares this method to other recent techniques, which have published their performance in terms of Area Under the ROC curve (AUC ROC) on the DRIVE dataset.\n\n| Method | AUC ROC on DRIVE |\n| ----------------------- |:----------------:|\n| Soares et al [1] | .9614 |\n| Azzopardi et al. [2] | .9614 |\n| Osareh et al [3] | .9650 |\n| Roychowdhury et al. [4] | .9670 |\n| Fraz et al. [5] | .9747 |\n| Qiaoliang et al. [6] | .9738 |\n| Melinscak et al. [7] | .9749 |\n| Liskowski et al.^ [8] | .9790 |\n| orobix | .9790 |\n| **this method** | **.9794** |\n\n\n"
},
{
"path": "__init__.py",
"content": ""
},
{
"path": "_config.yml",
"content": "theme: jekyll-theme-cayman"
},
{
"path": "configuration.txt",
"content": "[data paths]\npath_local = ../DRIVE_datasets_training_testing/\ntrain_imgs_original = DRIVE_dataset_imgs_train.hdf5\ntrain_groundTruth = DRIVE_dataset_groundTruth_train.hdf5\ntrain_border_masks = DRIVE_dataset_borderMasks_train.hdf5\ntest_imgs_original = DRIVE_dataset_imgs_test.hdf5\ntest_groundTruth = DRIVE_dataset_groundTruth_test.hdf5\ntest_border_masks = DRIVE_dataset_borderMasks_test.hdf5\n\n\n\n[experiment name]\nname = test\n\n\n[data attributes]\n#Dimensions of the patches extracted from the full images\npatch_height = 48\npatch_width = 48\n\n\n[training settings]\n#number of total patches:\nN_subimgs = 190000\n#if patches are extracted only inside the field of view:\ninside_FOV = False\n#Number of training epochs\nN_epochs = 150\nbatch_size = 1024\n#if running with nohup\nnohup = True\n\n\n[testing settings]\n#Choose the model to test: best==epoch with min loss, last==last epoch\nbest_last = best\n#number of full images for the test (max 20)\nfull_images_to_test = 20\n#How many original-groundTruth-prediction images are visualized in each image\nN_group_visual = 1\n#Compute average in the prediction, improve results but require more patches to be predicted\naverage_mode = True\n#Only if average_mode==True. Stride for patch extraction, lower value require more patches to be predicted\nstride_height = 5\nstride_width = 5\n#if running with nohup\nnohup = False\n"
},
{
"path": "figures/readme",
"content": "test\n"
},
{
"path": "googlefffa84c6c8a268c9.html",
"content": "google-site-verification: googlefffa84c6c8a268c9.html"
},
{
"path": "lib/__init__.py",
"content": ""
},
{
"path": "lib/densenet.py",
"content": "# -*- coding: utf-8 -*-\n'''\ncode from Keras.contrib. This is just a local copy in case that repo changes.\n\nDenseNet and DenseNet-FCN models for Keras.\n\nDenseNet is a network architecture where each layer is directly connected\nto every other layer in a feed-forward fashion (within each dense block).\nFor each layer, the feature maps of all preceding layers are treated as\nseparate inputs whereas its own feature maps are passed on as inputs to\nall subsequent layers. This connectivity pattern yields state-of-the-art\naccuracies on CIFAR10/100 (with or without data augmentation) and SVHN.\nOn the large scale ILSVRC 2012 (ImageNet) dataset, DenseNet achieves a\nsimilar accuracy as ResNet, but using less than half the amount of\nparameters and roughly half the number of FLOPs.\n\nDenseNets support any input image size of 32x32 or greater, and are thus\nsuited for CIFAR-10 or CIFAR-100 datasets. There are two types of DenseNets,\none suited for smaller images (DenseNet) and one suited for ImageNet,\ncalled DenseNetImageNet. They are differentiated by the strided convolution\nand pooling operations prior to the initial dense block.\n\nThe following table describes the size and accuracy of DenseNetImageNet models\non the ImageNet dataset (single crop), for which weights are provided:\n------------------------------------------------------------------------------------\n Model type | ImageNet Acc (Top 1) | ImageNet Acc (Top 5) | Params (M) |\n------------------------------------------------------------------------------------\n| DenseNet-121 | 25.02 % | 7.71 % | 8.0 |\n| DenseNet-169 | 23.80 % | 6.85 % | 14.3 |\n| DenseNet-201 | 22.58 % | 6.34 % | 20.2 |\n| DenseNet-161 | 22.20 % | - % | 28.9 |\n------------------------------------------------------------------------------------\n\nDenseNets can be extended to image segmentation tasks as described in the\npaper \"The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for\nSemantic Segmentation\". Here, the dense blocks are arranged and concatenated\nwith long skip connections for state of the art performance on the CamVid dataset.\n\n# Reference\n- [Densely Connected Convolutional Networks](https://arxiv.org/pdf/1608.06993.pdf)\n- [The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation]\n (https://arxiv.org/pdf/1611.09326.pdf)\n\nThis implementation is based on the following reference code:\n - https://github.com/gpleiss/efficient_densenet_pytorch\n - https://github.com/liuzhuang13/DenseNet\n\n'''\nfrom __future__ import print_function\nfrom __future__ import absolute_import\nfrom __future__ import division\n\nimport warnings\n\nfrom keras.models import Model\nfrom keras.layers import Dense\nfrom keras.layers import Dropout\nfrom keras.layers import Activation\nfrom keras.layers import Reshape\nfrom keras.layers import Conv2D\nfrom keras.layers import Conv2DTranspose\nfrom keras.layers import UpSampling2D\nfrom keras.layers import MaxPooling2D\nfrom keras.layers import AveragePooling2D\nfrom keras.layers import GlobalMaxPooling2D\nfrom keras.layers import GlobalAveragePooling2D\nfrom keras.layers import Input\nfrom keras.layers import concatenate\nfrom keras.layers import BatchNormalization\nfrom keras.regularizers import l2\nfrom keras.utils.layer_utils import convert_all_kernels_in_model\nfrom keras.utils.data_utils import get_file\nfrom keras.engine.topology import get_source_inputs\nfrom keras.applications.imagenet_utils import _obtain_input_shape\nfrom keras.applications.imagenet_utils import decode_predictions\nfrom keras.applications.imagenet_utils import preprocess_input as _preprocess_input\nimport keras.backend as K\nfrom keras.callbacks import ModelCheckpoint, CSVLogger, EarlyStopping, ReduceLROnPlateau\n\nfrom subpixel_upscaling import SubPixelUpscaling\n\nDENSENET_121_WEIGHTS_PATH = r'https://github.com/titu1994/DenseNet/releases/download/v3.0/DenseNet-BC-121-32.h5'\nDENSENET_161_WEIGHTS_PATH = r'https://github.com/titu1994/DenseNet/releases/download/v3.0/DenseNet-BC-161-48.h5'\nDENSENET_169_WEIGHTS_PATH = r'https://github.com/titu1994/DenseNet/releases/download/v3.0/DenseNet-BC-169-32.h5'\nDENSENET_121_WEIGHTS_PATH_NO_TOP = r'https://github.com/titu1994/DenseNet/releases/download/v3.0/DenseNet-BC-121-32-no-top.h5'\nDENSENET_161_WEIGHTS_PATH_NO_TOP = r'https://github.com/titu1994/DenseNet/releases/download/v3.0/DenseNet-BC-161-48-no-top.h5'\nDENSENET_169_WEIGHTS_PATH_NO_TOP = r'https://github.com/titu1994/DenseNet/releases/download/v3.0/DenseNet-BC-169-32-no-top.h5'\n\n\ndef preprocess_input(x, data_format=None):\n \"\"\"Preprocesses a tensor encoding a batch of images.\n\n # Arguments\n x: input Numpy tensor, 4D.\n data_format: data format of the image tensor.\n\n # Returns\n Preprocessed tensor.\n \"\"\"\n x = _preprocess_input(x, data_format=data_format)\n x *= 0.017 # scale values\n return x\n\n\ndef DenseNet(input_shape=None,\n depth=40,\n nb_dense_block=3,\n growth_rate=12,\n nb_filter=-1,\n nb_layers_per_block=-1,\n bottleneck=False,\n reduction=0.0,\n dropout_rate=0.0,\n weight_decay=1e-4,\n subsample_initial_block=False,\n include_top=True,\n weights=None,\n input_tensor=None,\n pooling=None,\n classes=10,\n activation='softmax',\n transition_pooling='avg'):\n '''Instantiate the DenseNet architecture.\n\n The model and the weights are compatible with both\n TensorFlow and Theano. The dimension ordering\n convention used by the model is the one\n specified in your Keras config file.\n\n # Arguments\n input_shape: optional shape tuple, only to be specified\n if `include_top` is False (otherwise the input shape\n has to be `(224, 224, 3)` (with `channels_last` dim ordering)\n or `(3, 224, 224)` (with `channels_first` dim ordering).\n It should have exactly 3 inputs channels,\n and width and height should be no smaller than 8.\n E.g. `(224, 224, 3)` would be one valid value.\n depth: number or layers in the DenseNet\n nb_dense_block: number of dense blocks to add to end\n growth_rate: number of filters to add per dense block\n nb_filter: initial number of filters. -1 indicates initial\n number of filters will default to 2 * growth_rate\n nb_layers_per_block: number of layers in each dense block.\n Can be a -1, positive integer or a list.\n If -1, calculates nb_layer_per_block from the network depth.\n If positive integer, a set number of layers per dense block.\n If list, nb_layer is used as provided. Note that list size must\n be nb_dense_block\n bottleneck: flag to add bottleneck blocks in between dense blocks\n reduction: reduction factor of transition blocks.\n Note : reduction value is inverted to compute compression.\n dropout_rate: dropout rate\n weight_decay: weight decay rate\n subsample_initial_block: Changes model type to suit different datasets.\n Should be set to True for ImageNet, and False for CIFAR datasets.\n When set to True, the initial convolution will be strided and\n adds a MaxPooling2D before the initial dense block.\n include_top: whether to include the fully-connected\n layer at the top of the network.\n weights: one of `None` (random initialization) or\n 'imagenet' (pre-training on ImageNet)..\n input_tensor: optional Keras tensor (i.e. output of `layers.Input()`)\n to use as image input for the model.\n pooling: Optional pooling mode for feature extraction\n when `include_top` is `False`.\n - `None` means that the output of the model\n will be the 4D tensor output of the\n last convolutional layer.\n - `avg` means that global average pooling\n will be applied to the output of the\n last convolutional layer, and thus\n the output of the model will be a\n 2D tensor.\n - `max` means that global max pooling will\n be applied.\n classes: optional number of classes to classify images\n into, only to be specified if `include_top` is True, and\n if no `weights` argument is specified.\n activation: Type of activation at the top layer. Can be one of\n 'softmax' or 'sigmoid'. Note that if sigmoid is used,\n classes must be 1.\n transition_pooling: `avg` for avg pooling (default), `max` for max pooling,\n None for no pooling during scale transition blocks. Please note that this\n default differs from the DenseNetFCN paper in accordance with the DenseNet\n paper.\n\n # Returns\n A Keras model instance.\n\n # Raises\n ValueError: in case of invalid argument for `weights`,\n or invalid input shape.\n '''\n\n if weights not in {'imagenet', None}:\n raise ValueError('The `weights` argument should be either '\n '`None` (random initialization) or `imagenet` '\n '(pre-training on ImageNet).')\n\n if weights == 'imagenet' and include_top and classes != 1000:\n raise ValueError('If using `weights` as ImageNet with `include_top` '\n 'as true, `classes` should be 1000')\n\n if activation not in ['softmax', 'sigmoid']:\n raise ValueError('activation must be one of \"softmax\" or \"sigmoid\"')\n\n if activation == 'sigmoid' and classes != 1:\n raise ValueError('sigmoid activation can only be used when classes = 1')\n\n # Determine proper input shape\n input_shape = _obtain_input_shape(input_shape,\n default_size=32,\n min_size=8,\n data_format=K.image_data_format(),\n require_flatten=include_top)\n\n if input_tensor is None:\n img_input = Input(shape=input_shape)\n else:\n if not K.is_keras_tensor(input_tensor):\n img_input = Input(tensor=input_tensor, shape=input_shape)\n else:\n img_input = input_tensor\n\n x = __create_dense_net(classes, img_input, include_top, depth, nb_dense_block,\n growth_rate, nb_filter, nb_layers_per_block, bottleneck,\n reduction, dropout_rate, weight_decay, subsample_initial_block,\n pooling, activation, transition_pooling)\n\n # Ensure that the model takes into account\n # any potential predecessors of `input_tensor`.\n if input_tensor is not None:\n inputs = get_source_inputs(input_tensor)\n else:\n inputs = img_input\n # Create model.\n model = Model(inputs, x, name='densenet')\n\n # load weights\n if weights == 'imagenet':\n weights_loaded = False\n\n if (depth == 121) and (nb_dense_block == 4) and (growth_rate == 32) and (nb_filter == 64) and \\\n (bottleneck is True) and (reduction == 0.5) and (subsample_initial_block):\n if include_top:\n weights_path = get_file('DenseNet-BC-121-32.h5',\n DENSENET_121_WEIGHTS_PATH,\n cache_subdir='models',\n md5_hash='a439dd41aa672aef6daba4ee1fd54abd')\n else:\n weights_path = get_file('DenseNet-BC-121-32-no-top.h5',\n DENSENET_121_WEIGHTS_PATH_NO_TOP,\n cache_subdir='models',\n md5_hash='55e62a6358af8a0af0eedf399b5aea99')\n model.load_weights(weights_path, by_name=True)\n weights_loaded = True\n\n if (depth == 161) and (nb_dense_block == 4) and (growth_rate == 48) and (nb_filter == 96) and \\\n (bottleneck is True) and (reduction == 0.5) and (subsample_initial_block):\n if include_top:\n weights_path = get_file('DenseNet-BC-161-48.h5',\n DENSENET_161_WEIGHTS_PATH,\n cache_subdir='models',\n md5_hash='6c326cf4fbdb57d31eff04333a23fcca')\n else:\n weights_path = get_file('DenseNet-BC-161-48-no-top.h5',\n DENSENET_161_WEIGHTS_PATH_NO_TOP,\n cache_subdir='models',\n md5_hash='1a9476b79f6b7673acaa2769e6427b92')\n model.load_weights(weights_path, by_name=True)\n weights_loaded = True\n\n if (depth == 169) and (nb_dense_block == 4) and (growth_rate == 32) and (nb_filter == 64) and \\\n (bottleneck is True) and (reduction == 0.5) and (subsample_initial_block):\n if include_top:\n weights_path = get_file('DenseNet-BC-169-32.h5',\n DENSENET_169_WEIGHTS_PATH,\n cache_subdir='models',\n md5_hash='914869c361303d2e39dec640b4e606a6')\n else:\n weights_path = get_file('DenseNet-BC-169-32-no-top.h5',\n DENSENET_169_WEIGHTS_PATH_NO_TOP,\n cache_subdir='models',\n md5_hash='89c19e8276cfd10585d5fadc1df6859e')\n model.load_weights(weights_path, by_name=True)\n weights_loaded = True\n\n if weights_loaded:\n if K.backend() == 'theano':\n convert_all_kernels_in_model(model)\n\n if K.image_data_format() == 'channels_first' and K.backend() == 'tensorflow':\n warnings.warn('You are using the TensorFlow backend, yet you '\n 'are using the Theano '\n 'image data format convention '\n '(`image_data_format=\"channels_first\"`). '\n 'For best performance, set '\n '`image_data_format=\"channels_last\"` in '\n 'your Keras config '\n 'at ~/.keras/keras.json.')\n\n print(\"Weights for the model were loaded successfully\")\n\n return model\n\n\ndef DenseNetFCN(input_shape, nb_dense_block=5, growth_rate=16, nb_layers_per_block=4,\n reduction=0.0, dropout_rate=0.2, weight_decay=1E-4, init_conv_filters=48,\n include_top=True, weights=None, input_tensor=None, classes=1, activation='sigmoid',\n upsampling_conv=128, upsampling_type='deconv', early_transition=False,\n transition_pooling='max', initial_kernel_size=(3, 3)):\n '''Instantiate the DenseNet FCN architecture.\n Note that when using TensorFlow,\n for best performance you should set\n `image_data_format='channels_last'` in your Keras config\n at ~/.keras/keras.json.\n # Arguments\n nb_dense_block: number of dense blocks to add to end (generally = 3)\n growth_rate: number of filters to add per dense block\n nb_layers_per_block: number of layers in each dense block.\n Can be a positive integer or a list.\n If positive integer, a set number of layers per dense block.\n If list, nb_layer is used as provided. Note that list size must\n be (nb_dense_block + 1)\n reduction: reduction factor of transition blocks.\n Note : reduction value is inverted to compute compression.\n dropout_rate: dropout rate\n weight_decay: weight decay factor\n init_conv_filters: number of layers in the initial convolution layer\n include_top: whether to include the fully-connected\n layer at the top of the network.\n weights: one of `None` (random initialization) or\n 'cifar10' (pre-training on CIFAR-10)..\n input_tensor: optional Keras tensor (i.e. output of `layers.Input()`)\n to use as image input for the model.\n input_shape: optional shape tuple, only to be specified\n if `include_top` is False (otherwise the input shape\n has to be `(32, 32, 3)` (with `channels_last` dim ordering)\n or `(3, 32, 32)` (with `channels_first` dim ordering).\n It should have exactly 3 inputs channels,\n and width and height should be no smaller than 8.\n E.g. `(200, 200, 3)` would be one valid value.\n classes: optional number of classes to classify images\n into, only to be specified if `include_top` is True, and\n if no `weights` argument is specified.\n activation: Type of activation at the top layer. Can be one of 'softmax' or 'sigmoid'.\n Note that if sigmoid is used, classes must be 1.\n upsampling_conv: number of convolutional layers in upsampling via subpixel convolution\n upsampling_type: Can be one of 'deconv', 'upsampling' and\n 'subpixel'. Defines type of upsampling algorithm used.\n batchsize: Fixed batch size. This is a temporary requirement for\n computation of output shape in the case of Deconvolution2D layers.\n Parameter will be removed in next iteration of Keras, which infers\n output shape of deconvolution layers automatically.\n early_transition: Start with an extra initial transition down and end with an extra\n transition up to reduce the network size.\n initial_kernel_size: The first Conv2D kernel might vary in size based on the\n application, this parameter makes it configurable.\n\n # Returns\n A Keras model instance.\n '''\n\n if weights not in {None}:\n raise ValueError('The `weights` argument should be '\n '`None` (random initialization) as no '\n 'model weights are provided.')\n\n upsampling_type = upsampling_type.lower()\n\n if upsampling_type not in ['upsampling', 'deconv', 'subpixel']:\n raise ValueError('Parameter \"upsampling_type\" must be one of \"upsampling\", '\n '\"deconv\" or \"subpixel\".')\n\n if input_shape is None:\n raise ValueError('For fully convolutional models, input shape must be supplied.')\n\n if type(nb_layers_per_block) is not list and nb_dense_block < 1:\n raise ValueError('Number of dense layers per block must be greater than 1. Argument '\n 'value was %d.' % (nb_layers_per_block))\n\n if activation not in ['softmax', 'sigmoid']:\n raise ValueError('activation must be one of \"softmax\" or \"sigmoid\"')\n\n if activation == 'sigmoid' and classes != 1:\n raise ValueError('sigmoid activation can only be used when classes = 1')\n\n # Determine proper input shape\n min_size = 2 ** nb_dense_block\n\n if K.image_data_format() == 'channels_first':\n if input_shape is not None:\n if ((input_shape[1] is not None and input_shape[1] < min_size) or\n (input_shape[2] is not None and input_shape[2] < min_size)):\n raise ValueError('Input size must be at least ' +\n str(min_size) + 'x' + str(min_size) + ', got '\n '`input_shape=' + str(input_shape) + '`')\n else:\n input_shape = (classes, None, None)\n else:\n if input_shape is not None:\n if ((input_shape[0] is not None and input_shape[0] < min_size) or\n (input_shape[1] is not None and input_shape[1] < min_size)):\n raise ValueError('Input size must be at least ' +\n str(min_size) + 'x' + str(min_size) + ', got '\n '`input_shape=' + str(input_shape) + '`')\n else:\n input_shape = (None, None, classes)\n\n if input_tensor is None:\n img_input = Input(shape=input_shape)\n else:\n if not K.is_keras_tensor(input_tensor):\n img_input = Input(tensor=input_tensor, shape=input_shape)\n else:\n img_input = input_tensor\n\n x = __create_fcn_dense_net(classes, img_input, include_top, nb_dense_block, growth_rate,\n reduction, dropout_rate, weight_decay,\n nb_layers_per_block, upsampling_conv, upsampling_type,\n init_conv_filters, input_shape, activation,\n early_transition, transition_pooling, initial_kernel_size)\n\n # Ensure that the model takes into account\n # any potential predecessors of `input_tensor`.\n if input_tensor is not None:\n inputs = get_source_inputs(input_tensor)\n else:\n inputs = img_input\n # Create model.\n model = Model(inputs, x, name='fcn-densenet')\n\n return model\n\n\ndef DenseNetImageNet121(input_shape=None,\n bottleneck=True,\n reduction=0.5,\n dropout_rate=0.0,\n weight_decay=1e-4,\n include_top=True,\n weights='imagenet',\n input_tensor=None,\n pooling=None,\n classes=1000,\n activation='softmax'):\n return DenseNet(input_shape, depth=121, nb_dense_block=4, growth_rate=32, nb_filter=64,\n nb_layers_per_block=[6, 12, 24, 16], bottleneck=bottleneck, reduction=reduction,\n dropout_rate=dropout_rate, weight_decay=weight_decay, subsample_initial_block=True,\n include_top=include_top, weights=weights, input_tensor=input_tensor,\n pooling=pooling, classes=classes, activation=activation)\n\n\ndef DenseNetImageNet169(input_shape=None,\n bottleneck=True,\n reduction=0.5,\n dropout_rate=0.0,\n weight_decay=1e-4,\n include_top=True,\n weights='imagenet',\n input_tensor=None,\n pooling=None,\n classes=1000,\n activation='softmax'):\n return DenseNet(input_shape, depth=169, nb_dense_block=4, growth_rate=32, nb_filter=64,\n nb_layers_per_block=[6, 12, 32, 32], bottleneck=bottleneck, reduction=reduction,\n dropout_rate=dropout_rate, weight_decay=weight_decay, subsample_initial_block=True,\n include_top=include_top, weights=weights, input_tensor=input_tensor,\n pooling=pooling, classes=classes, activation=activation)\n\n\ndef DenseNetImageNet201(input_shape=None,\n bottleneck=True,\n reduction=0.5,\n dropout_rate=0.0,\n weight_decay=1e-4,\n include_top=True,\n weights=None,\n input_tensor=None,\n pooling=None,\n classes=1000,\n activation='softmax'):\n return DenseNet(input_shape, depth=201, nb_dense_block=4, growth_rate=32, nb_filter=64,\n nb_layers_per_block=[6, 12, 48, 32], bottleneck=bottleneck, reduction=reduction,\n dropout_rate=dropout_rate, weight_decay=weight_decay, subsample_initial_block=True,\n include_top=include_top, weights=weights, input_tensor=input_tensor,\n pooling=pooling, classes=classes, activation=activation)\n\n\ndef DenseNetImageNet264(input_shape=None,\n bottleneck=True,\n reduction=0.5,\n dropout_rate=0.0,\n weight_decay=1e-4,\n include_top=True,\n weights=None,\n input_tensor=None,\n pooling=None,\n classes=1000,\n activation='softmax'):\n return DenseNet(input_shape, depth=201, nb_dense_block=4, growth_rate=32, nb_filter=64,\n nb_layers_per_block=[6, 12, 64, 48], bottleneck=bottleneck, reduction=reduction,\n dropout_rate=dropout_rate, weight_decay=weight_decay, subsample_initial_block=True,\n include_top=include_top, weights=weights, input_tensor=input_tensor,\n pooling=pooling, classes=classes, activation=activation)\n\n\ndef DenseNetImageNet161(input_shape=None,\n bottleneck=True,\n reduction=0.5,\n dropout_rate=0.0,\n weight_decay=1e-4,\n include_top=True,\n weights='imagenet',\n input_tensor=None,\n pooling=None,\n classes=1000,\n activation='softmax'):\n return DenseNet(input_shape, depth=161, nb_dense_block=4, growth_rate=48, nb_filter=96,\n nb_layers_per_block=[6, 12, 36, 24], bottleneck=bottleneck, reduction=reduction,\n dropout_rate=dropout_rate, weight_decay=weight_decay, subsample_initial_block=True,\n include_top=include_top, weights=weights, input_tensor=input_tensor,\n pooling=pooling, classes=classes, activation=activation)\n\n\ndef name_or_none(prefix, name):\n return prefix + name if (prefix is not None and name is not None) else None\n\n\ndef __conv_block(ip, nb_filter, bottleneck=False, dropout_rate=None, weight_decay=1e-4, block_prefix=None):\n '''\n Adds a convolution layer (with batch normalization and relu),\n and optionally a bottleneck layer.\n\n # Arguments\n ip: Input tensor\n nb_filter: integer, the dimensionality of the output space\n (i.e. the number output of filters in the convolution)\n bottleneck: if True, adds a bottleneck convolution block\n dropout_rate: dropout rate\n weight_decay: weight decay factor\n block_prefix: str, for unique layer naming\n\n # Input shape\n 4D tensor with shape:\n `(samples, channels, rows, cols)` if data_format='channels_first'\n or 4D tensor with shape:\n `(samples, rows, cols, channels)` if data_format='channels_last'.\n\n # Output shape\n 4D tensor with shape:\n `(samples, filters, new_rows, new_cols)` if data_format='channels_first'\n or 4D tensor with shape:\n `(samples, new_rows, new_cols, filters)` if data_format='channels_last'.\n `rows` and `cols` values might have changed due to stride.\n\n # Returns\n output tensor of block\n '''\n with K.name_scope('ConvBlock'):\n concat_axis = 1 if K.image_data_format() == 'channels_first' else -1\n\n x = BatchNormalization(axis=concat_axis, epsilon=1.1e-5, name=name_or_none(block_prefix, '_bn'))(ip)\n x = Activation('relu')(x)\n\n if bottleneck:\n inter_channel = nb_filter * 4\n\n x = Conv2D(inter_channel, (1, 1), kernel_initializer='he_normal', padding='same', use_bias=False,\n kernel_regularizer=l2(weight_decay), name=name_or_none(block_prefix, '_bottleneck_conv2D'))(x)\n x = BatchNormalization(axis=concat_axis, epsilon=1.1e-5,\n name=name_or_none(block_prefix, '_bottleneck_bn'))(x)\n x = Activation('relu')(x)\n\n x = Conv2D(nb_filter, (3, 3), kernel_initializer='he_normal', padding='same', use_bias=False,\n name=name_or_none(block_prefix, '_conv2D'))(x)\n if dropout_rate:\n x = Dropout(dropout_rate)(x)\n\n return x\n\n\ndef __dense_block(x, nb_layers, nb_filter, growth_rate, bottleneck=False, dropout_rate=None,\n weight_decay=1e-4, grow_nb_filters=True, return_concat_list=False, block_prefix=None):\n '''\n Build a dense_block where the output of each conv_block is fed\n to subsequent ones\n\n # Arguments\n x: input keras tensor\n nb_layers: the number of conv_blocks to append to the model\n nb_filter: integer, the dimensionality of the output space\n (i.e. the number output of filters in the convolution)\n growth_rate: growth rate of the dense block\n bottleneck: if True, adds a bottleneck convolution block to\n each conv_block\n dropout_rate: dropout rate\n weight_decay: weight decay factor\n grow_nb_filters: if True, allows number of filters to grow\n return_concat_list: set to True to return the list of\n feature maps along with the actual output\n block_prefix: str, for block unique naming\n\n # Return\n If return_concat_list is True, returns a list of the output\n keras tensor, the number of filters and a list of all the\n dense blocks added to the keras tensor\n\n If return_concat_list is False, returns a list of the output\n keras tensor and the number of filters\n '''\n with K.name_scope('DenseBlock'):\n concat_axis = 1 if K.image_data_format() == 'channels_first' else -1\n\n x_list = [x]\n\n for i in range(nb_layers):\n cb = __conv_block(x, growth_rate, bottleneck, dropout_rate, weight_decay,\n block_prefix=name_or_none(block_prefix, '_%i' % i))\n x_list.append(cb)\n\n x = concatenate([x, cb], axis=concat_axis)\n\n if grow_nb_filters:\n nb_filter += growth_rate\n\n if return_concat_list:\n return x, nb_filter, x_list\n else:\n return x, nb_filter\n\n\ndef __transition_block(ip, nb_filter, compression=1.0, weight_decay=1e-4, block_prefix=None, transition_pooling='max'):\n '''\n Adds a pointwise convolution layer (with batch normalization and relu),\n and an average pooling layer. The number of output convolution filters\n can be reduced by appropriately reducing the compression parameter.\n\n # Arguments\n ip: input keras tensor\n nb_filter: integer, the dimensionality of the output space\n (i.e. the number output of filters in the convolution)\n compression: calculated as 1 - reduction. Reduces the number\n of feature maps in the transition block.\n weight_decay: weight decay factor\n block_prefix: str, for block unique naming\n\n # Input shape\n 4D tensor with shape:\n `(samples, channels, rows, cols)` if data_format='channels_first'\n or 4D tensor with shape:\n `(samples, rows, cols, channels)` if data_format='channels_last'.\n\n # Output shape\n 4D tensor with shape:\n `(samples, nb_filter * compression, rows / 2, cols / 2)`\n if data_format='channels_first'\n or 4D tensor with shape:\n `(samples, rows / 2, cols / 2, nb_filter * compression)`\n if data_format='channels_last'.\n\n # Returns\n a keras tensor\n '''\n with K.name_scope('Transition'):\n concat_axis = 1 if K.image_data_format() == 'channels_first' else -1\n\n x = BatchNormalization(axis=concat_axis, epsilon=1.1e-5, name=name_or_none(block_prefix, '_bn'))(ip)\n x = Activation('relu')(x)\n x = Conv2D(int(nb_filter * compression), (1, 1), kernel_initializer='he_normal', padding='same',\n use_bias=False, kernel_regularizer=l2(weight_decay), name=name_or_none(block_prefix, '_conv2D'))(x)\n if transition_pooling == 'avg':\n x = AveragePooling2D((2, 2), strides=(2, 2))(x)\n elif transition_pooling == 'max':\n x = MaxPooling2D((2, 2), strides=(2, 2))(x)\n\n return x\n\n\ndef __transition_up_block(ip, nb_filters, type='deconv', weight_decay=1E-4, block_prefix=None):\n '''Adds an upsampling block. Upsampling operation relies on the the type parameter.\n\n # Arguments\n ip: input keras tensor\n nb_filters: integer, the dimensionality of the output space\n (i.e. the number output of filters in the convolution)\n type: can be 'upsampling', 'subpixel', 'deconv'. Determines\n type of upsampling performed\n weight_decay: weight decay factor\n block_prefix: str, for block unique naming\n\n # Input shape\n 4D tensor with shape:\n `(samples, channels, rows, cols)` if data_format='channels_first'\n or 4D tensor with shape:\n `(samples, rows, cols, channels)` if data_format='channels_last'.\n\n # Output shape\n 4D tensor with shape:\n `(samples, nb_filter, rows * 2, cols * 2)` if data_format='channels_first'\n or 4D tensor with shape:\n `(samples, rows * 2, cols * 2, nb_filter)` if data_format='channels_last'.\n\n # Returns\n a keras tensor\n '''\n with K.name_scope('TransitionUp'):\n\n if type == 'upsampling':\n x = UpSampling2D(name=name_or_none(block_prefix, '_upsampling'))(ip)\n elif type == 'subpixel':\n x = Conv2D(nb_filters, (3, 3), activation='relu', padding='same', kernel_regularizer=l2(weight_decay),\n use_bias=False, kernel_initializer='he_normal', name=name_or_none(block_prefix, '_conv2D'))(ip)\n x = SubPixelUpscaling(scale_factor=2, name=name_or_none(block_prefix, '_subpixel'))(x)\n x = Conv2D(nb_filters, (3, 3), activation='relu', padding='same', kernel_regularizer=l2(weight_decay),\n use_bias=False, kernel_initializer='he_normal', name=name_or_none(block_prefix, '_conv2D'))(x)\n else:\n x = Conv2DTranspose(nb_filters, (3, 3), activation='relu', padding='same', strides=(2, 2),\n kernel_initializer='he_normal', kernel_regularizer=l2(weight_decay),\n name=name_or_none(block_prefix, '_conv2DT'))(ip)\n return x\n\n\ndef __create_dense_net(nb_classes, img_input, include_top, depth=40, nb_dense_block=3, growth_rate=12, nb_filter=-1,\n nb_layers_per_block=-1, bottleneck=False, reduction=0.0, dropout_rate=None, weight_decay=1e-4,\n subsample_initial_block=False, pooling=None, activation='sigmoid', transition_pooling='avg'):\n ''' Build the DenseNet model\n\n # Arguments\n nb_classes: number of classes\n img_input: tuple of shape (channels, rows, columns) or (rows, columns, channels)\n include_top: flag to include the final Dense layer\n depth: number or layers\n nb_dense_block: number of dense blocks to add to end (generally = 3)\n growth_rate: number of filters to add per dense block\n nb_filter: initial number of filters. Default -1 indicates initial number of filters is 2 * growth_rate\n nb_layers_per_block: number of layers in each dense block.\n Can be a -1, positive integer or a list.\n If -1, calculates nb_layer_per_block from the depth of the network.\n If positive integer, a set number of layers per dense block.\n If list, nb_layer is used as provided. Note that list size must\n be (nb_dense_block + 1)\n bottleneck: add bottleneck blocks\n reduction: reduction factor of transition blocks. Note : reduction value is inverted to compute compression\n dropout_rate: dropout rate\n weight_decay: weight decay rate\n subsample_initial_block: Changes model type to suit different datasets.\n Should be set to True for ImageNet, and False for CIFAR datasets.\n When set to True, the initial convolution will be strided and\n adds a MaxPooling2D before the initial dense block.\n pooling: Optional pooling mode for feature extraction\n when `include_top` is `False`.\n - `None` means that the output of the model\n will be the 4D tensor output of the\n last convolutional layer.\n - `avg` means that global average pooling\n will be applied to the output of the\n last convolutional layer, and thus\n the output of the model will be a\n 2D tensor.\n - `max` means that global max pooling will\n be applied.\n activation: Type of activation at the top layer. Can be one of 'softmax' or 'sigmoid'.\n Note that if sigmoid is used, classes must be 1.\n transition_pooling: `avg` for avg pooling (default), `max` for max pooling,\n None for no pooling during scale transition blocks. Please note that this\n default differs from the DenseNetFCN paper in accordance with the DenseNet\n paper.\n\n # Returns\n a keras tensor\n\n # Raises\n ValueError: in case of invalid argument for `reduction`\n or `nb_dense_block`\n '''\n with K.name_scope('DenseNet'):\n concat_axis = 1 if K.image_data_format() == 'channels_first' else -1\n\n if reduction != 0.0:\n if not (reduction <= 1.0 and reduction > 0.0):\n raise ValueError('`reduction` value must lie between 0.0 and 1.0')\n\n # layers in each dense block\n if type(nb_layers_per_block) is list or type(nb_layers_per_block) is tuple:\n nb_layers = list(nb_layers_per_block) # Convert tuple to list\n\n if len(nb_layers) != (nb_dense_block):\n raise ValueError('If `nb_dense_block` is a list, its length must match '\n 'the number of layers provided by `nb_layers`.')\n\n final_nb_layer = nb_layers[-1]\n nb_layers = nb_layers[:-1]\n else:\n if nb_layers_per_block == -1:\n assert (depth - 4) % 3 == 0, 'Depth must be 3 N + 4 if nb_layers_per_block == -1'\n count = int((depth - 4) / 3)\n\n if bottleneck:\n count = count // 2\n\n nb_layers = [count for _ in range(nb_dense_block)]\n final_nb_layer = count\n else:\n final_nb_layer = nb_layers_per_block\n nb_layers = [nb_layers_per_block] * nb_dense_block\n\n # compute initial nb_filter if -1, else accept users initial nb_filter\n if nb_filter <= 0:\n nb_filter = 2 * growth_rate\n\n # compute compression factor\n compression = 1.0 - reduction\n\n # Initial convolution\n if subsample_initial_block:\n initial_kernel = (7, 7)\n initial_strides = (2, 2)\n else:\n initial_kernel = (3, 3)\n initial_strides = (1, 1)\n\n x = Conv2D(nb_filter, initial_kernel, kernel_initializer='he_normal', padding='same', name='initial_conv2D',\n strides=initial_strides, use_bias=False, kernel_regularizer=l2(weight_decay))(img_input)\n\n if subsample_initial_block:\n x = BatchNormalization(axis=concat_axis, epsilon=1.1e-5, name='initial_bn')(x)\n x = Activation('relu')(x)\n x = MaxPooling2D((3, 3), strides=(2, 2), padding='same')(x)\n\n # Add dense blocks\n for block_idx in range(nb_dense_block - 1):\n x, nb_filter = __dense_block(x, nb_layers[block_idx], nb_filter, growth_rate, bottleneck=bottleneck,\n dropout_rate=dropout_rate, weight_decay=weight_decay,\n block_prefix='dense_%i' % block_idx)\n # add transition_block\n x = __transition_block(x, nb_filter, compression=compression, weight_decay=weight_decay,\n block_prefix='tr_%i' % block_idx, transition_pooling=transition_pooling)\n nb_filter = int(nb_filter * compression)\n\n # The last dense_block does not have a transition_block\n x, nb_filter = __dense_block(x, final_nb_layer, nb_filter, growth_rate, bottleneck=bottleneck,\n dropout_rate=dropout_rate, weight_decay=weight_decay,\n block_prefix='dense_%i' % (nb_dense_block - 1))\n\n x = BatchNormalization(axis=concat_axis, epsilon=1.1e-5, name='final_bn')(x)\n x = Activation('relu')(x)\n\n if include_top:\n if pooling == 'avg':\n x = GlobalAveragePooling2D()(x)\n elif pooling == 'max':\n x = GlobalMaxPooling2D()(x)\n x = Dense(nb_classes, activation=activation)(x)\n else:\n if pooling == 'avg':\n x = GlobalAveragePooling2D()(x)\n elif pooling == 'max':\n x = GlobalMaxPooling2D()(x)\n\n return x\n\n\ndef __create_fcn_dense_net(nb_classes, img_input, include_top, nb_dense_block=5, growth_rate=12,\n reduction=0.0, dropout_rate=None, weight_decay=1e-4,\n nb_layers_per_block=4, nb_upsampling_conv=128, upsampling_type='deconv',\n init_conv_filters=48, input_shape=None, activation='sigmoid',\n early_transition=False, transition_pooling='max', initial_kernel_size=(3, 3)):\n ''' Build the DenseNet-FCN model\n\n # Arguments\n nb_classes: number of classes\n img_input: tuple of shape (channels, rows, columns) or (rows, columns, channels)\n include_top: flag to include the final Dense layer\n nb_dense_block: number of dense blocks to add to end (generally = 3)\n growth_rate: number of filters to add per dense block\n reduction: reduction factor of transition blocks. Note : reduction value is inverted to compute compression\n dropout_rate: dropout rate\n weight_decay: weight decay\n nb_layers_per_block: number of layers in each dense block.\n Can be a positive integer or a list.\n If positive integer, a set number of layers per dense block.\n If list, nb_layer is used as provided. Note that list size must\n be (nb_dense_block + 1)\n nb_upsampling_conv: number of convolutional layers in upsampling via subpixel convolution\n upsampling_type: Can be one of 'upsampling', 'deconv' and 'subpixel'. Defines\n type of upsampling algorithm used.\n input_shape: Only used for shape inference in fully convolutional networks.\n activation: Type of activation at the top layer. Can be one of 'softmax' or 'sigmoid'.\n Note that if sigmoid is used, classes must be 1.\n early_transition: Start with an extra initial transition down and end with an extra\n transition up to reduce the network size.\n transition_pooling: 'max' for max pooling (default), 'avg' for average pooling,\n None for no pooling. Please note that this default differs from the DenseNet\n paper in accordance with the DenseNetFCN paper.\n initial_kernel_size: The first Conv2D kernel might vary in size based on the\n application, this parameter makes it configurable.\n\n # Returns\n a keras tensor\n\n # Raises\n ValueError: in case of invalid argument for `reduction`,\n `nb_dense_block` or `nb_upsampling_conv`.\n '''\n with K.name_scope('DenseNetFCN'):\n concat_axis = 1 if K.image_data_format() == 'channels_first' else -1\n\n if concat_axis == 1: # channels_first dim ordering\n _, rows, cols = input_shape\n else:\n rows, cols, _ = input_shape\n\n if reduction != 0.0:\n if not (reduction <= 1.0 and reduction > 0.0):\n raise ValueError('`reduction` value must lie between 0.0 and 1.0')\n\n # check if upsampling_conv has minimum number of filters\n # minimum is set to 12, as at least 3 color channels are needed for correct upsampling\n if not (nb_upsampling_conv > 12 and nb_upsampling_conv % 4 == 0):\n raise ValueError('Parameter `nb_upsampling_conv` number of channels must '\n 'be a positive number divisible by 4 and greater than 12')\n\n # layers in each dense block\n if type(nb_layers_per_block) is list or type(nb_layers_per_block) is tuple:\n nb_layers = list(nb_layers_per_block) # Convert tuple to list\n\n if len(nb_layers) != (nb_dense_block + 1):\n raise ValueError('If `nb_dense_block` is a list, its length must be '\n '(`nb_dense_block` + 1)')\n\n bottleneck_nb_layers = nb_layers[-1]\n rev_layers = nb_layers[::-1]\n nb_layers.extend(rev_layers[1:])\n else:\n bottleneck_nb_layers = nb_layers_per_block\n nb_layers = [nb_layers_per_block] * (2 * nb_dense_block + 1)\n\n # compute compression factor\n compression = 1.0 - reduction\n\n # Initial convolution\n x = Conv2D(init_conv_filters, initial_kernel_size, kernel_initializer='he_normal', padding='same', name='initial_conv2D',\n use_bias=False, kernel_regularizer=l2(weight_decay))(img_input)\n x = BatchNormalization(axis=concat_axis, epsilon=1.1e-5, name='initial_bn')(x)\n x = Activation('relu')(x)\n\n nb_filter = init_conv_filters\n\n skip_list = []\n\n if early_transition:\n x = __transition_block(x, nb_filter, compression=compression, weight_decay=weight_decay,\n block_prefix='tr_early', transition_pooling=transition_pooling)\n\n # Add dense blocks and transition down block\n for block_idx in range(nb_dense_block):\n x, nb_filter = __dense_block(x, nb_layers[block_idx], nb_filter, growth_rate, dropout_rate=dropout_rate,\n weight_decay=weight_decay, block_prefix='dense_%i' % block_idx)\n\n # Skip connection\n skip_list.append(x)\n\n # add transition_block\n x = __transition_block(x, nb_filter, compression=compression, weight_decay=weight_decay,\n block_prefix='tr_%i' % block_idx, transition_pooling=transition_pooling)\n\n nb_filter = int(nb_filter * compression) # this is calculated inside transition_down_block\n\n # The last dense_block does not have a transition_down_block\n # return the concatenated feature maps without the concatenation of the input\n _, nb_filter, concat_list = __dense_block(x, bottleneck_nb_layers, nb_filter, growth_rate,\n dropout_rate=dropout_rate, weight_decay=weight_decay,\n return_concat_list=True,\n block_prefix='dense_%i' % nb_dense_block)\n\n skip_list = skip_list[::-1] # reverse the skip list\n\n # Add dense blocks and transition up block\n for block_idx in range(nb_dense_block):\n n_filters_keep = growth_rate * nb_layers[nb_dense_block + block_idx]\n\n # upsampling block must upsample only the feature maps (concat_list[1:]),\n # not the concatenation of the input with the feature maps (concat_list[0].\n l = concatenate(concat_list[1:], axis=concat_axis)\n\n t = __transition_up_block(l, nb_filters=n_filters_keep, type=upsampling_type, weight_decay=weight_decay,\n block_prefix='tr_up_%i' % block_idx)\n\n # concatenate the skip connection with the transition block\n x = concatenate([t, skip_list[block_idx]], axis=concat_axis)\n\n # Dont allow the feature map size to grow in upsampling dense blocks\n x_up, nb_filter, concat_list = __dense_block(x, nb_layers[nb_dense_block + block_idx + 1],\n nb_filter=growth_rate, growth_rate=growth_rate,\n dropout_rate=dropout_rate, weight_decay=weight_decay,\n return_concat_list=True, grow_nb_filters=False,\n block_prefix='dense_%i' % (nb_dense_block + 1 + block_idx))\n\n if early_transition:\n x_up = __transition_up_block(x_up, nb_filters=nb_filter, type=upsampling_type, weight_decay=weight_decay,\n block_prefix='tr_up_early')\n if include_top:\n x = Conv2D(nb_classes, (1, 1), activation='linear', padding='same', use_bias=False)(x_up)\n\n if K.image_data_format() == 'channels_first':\n channel, row, col = input_shape\n else:\n row, col, channel = input_shape\n\n x = Reshape((row * col, nb_classes))(x)\n x = Activation(activation)(x)\n x = Reshape((row, col, nb_classes))(x)\n else:\n x = x_up\n\n return x\n"
},
{
"path": "lib/extract_patches.py",
"content": "import numpy as np\nimport random\nimport configparser\n\nfrom .help_functions import load_hdf5\nfrom .help_functions import visualize\nfrom .help_functions import group_images\n\nfrom .pre_processing import my_PreProc\n\n\n#To select the same images\n# random.seed(10)\n\n#Load the original data and return the extracted patches for training/testing\ndef get_data_training(DRIVE_train_imgs_original,\n DRIVE_train_groudTruth,\n patch_height,\n patch_width,\n N_subimgs,\n inside_FOV):\n train_imgs_original = load_hdf5(DRIVE_train_imgs_original)\n train_masks = load_hdf5(DRIVE_train_groudTruth) #masks always the same\n # visualize(group_images(train_imgs_original[0:20,:,:,:],5),'imgs_train')#.show() #check original imgs train\n\n\n train_imgs = my_PreProc(train_imgs_original)\n train_masks = train_masks/255.\n\n train_imgs = train_imgs[:,:,9:574,:] #cut bottom and top so now it is 565*565\n train_masks = train_masks[:,:,9:574,:] #cut bottom and top so now it is 565*565\n data_consistency_check(train_imgs,train_masks)\n\n #check masks are within 0-1\n assert(np.min(train_masks)==0 and np.max(train_masks)==1)\n\n print(\"\\ntrain images/masks shape:\")\n print(train_imgs.shape)\n print(\"train images range (min-max): \" +str(np.min(train_imgs)) +' - '+str(np.max(train_imgs)))\n print(\"train masks are within 0-1\\n\")\n\n #extract the TRAINING patches from the full images\n patches_imgs_train, patches_masks_train = extract_random(train_imgs,train_masks,patch_height,patch_width,N_subimgs,inside_FOV)\n data_consistency_check(patches_imgs_train, patches_masks_train)\n\n print(\"\\ntrain PATCHES images/masks shape:\")\n print(patches_imgs_train.shape)\n print(\"train PATCHES images range (min-max): \" +str(np.min(patches_imgs_train)) +' - '+str(np.max(patches_imgs_train)))\n\n return patches_imgs_train, patches_masks_train#, patches_imgs_test, patches_masks_test\n\n\n#Load the original data and return the extracted patches for training/testing\ndef get_data_testing(DRIVE_test_imgs_original, DRIVE_test_groudTruth, Imgs_to_test, patch_height, patch_width):\n ### test\n test_imgs_original = load_hdf5(DRIVE_test_imgs_original)\n test_masks = load_hdf5(DRIVE_test_groudTruth)\n\n test_imgs = my_PreProc(test_imgs_original)\n test_masks = test_masks/255.\n\n #extend both images and masks so they can be divided exactly by the patches dimensions\n test_imgs = test_imgs[0:Imgs_to_test,:,:,:]\n test_masks = test_masks[0:Imgs_to_test,:,:,:]\n test_imgs = paint_border(test_imgs,patch_height,patch_width)\n test_masks = paint_border(test_masks,patch_height,patch_width)\n\n data_consistency_check(test_imgs, test_masks)\n\n #check masks are within 0-1\n assert(np.max(test_masks)==1 and np.min(test_masks)==0)\n\n print(\"\\ntest images/masks shape:\")\n print(test_imgs.shape)\n print(\"test images range (min-max): \" +str(np.min(test_imgs)) +' - '+str(np.max(test_imgs)))\n print(\"test masks are within 0-1\\n\")\n\n #extract the TEST patches from the full images\n patches_imgs_test = extract_ordered(test_imgs,patch_height,patch_width)\n patches_masks_test = extract_ordered(test_masks,patch_height,patch_width)\n data_consistency_check(patches_imgs_test, patches_masks_test)\n\n print(\"\\ntest PATCHES images/masks shape:\")\n print(patches_imgs_test.shape)\n print(\"test PATCHES images range (min-max): \" +str(np.min(patches_imgs_test)) +' - '+str(np.max(patches_imgs_test)))\n\n return patches_imgs_test, patches_masks_test\n\n\n\n\n# Load the original data and return the extracted patches for testing\n# return the ground truth in its original shape\ndef get_data_testing_overlap(DRIVE_test_imgs_original, DRIVE_test_groudTruth, Imgs_to_test, patch_height, patch_width, stride_height, stride_width):\n ### test\n test_imgs_original = load_hdf5(DRIVE_test_imgs_original)\n test_masks = load_hdf5(DRIVE_test_groudTruth)\n\n test_imgs = my_PreProc(test_imgs_original)\n test_masks = test_masks/255.\n #extend both images and masks so they can be divided exactly by the patches dimensions\n test_imgs = test_imgs[0:Imgs_to_test,:,:,:]\n test_masks = test_masks[0:Imgs_to_test,:,:,:]\n test_imgs = paint_border_overlap(test_imgs, patch_height, patch_width, stride_height, stride_width)\n\n #check masks are within 0-1\n assert(np.max(test_masks)==1 and np.min(test_masks)==0)\n\n print(\"\\ntest images shape:\")\n print(test_imgs.shape)\n print(\"\\ntest mask shape:\")\n print(test_masks.shape)\n print(\"test images range (min-max): \" +str(np.min(test_imgs)) +' - '+str(np.max(test_imgs)))\n print(\"test masks are within 0-1\\n\")\n\n #extract the TEST patches from the full images\n patches_imgs_test = extract_ordered_overlap(test_imgs,patch_height,patch_width,stride_height,stride_width)\n\n print(\"\\ntest PATCHES images shape:\")\n print(patches_imgs_test.shape)\n print(\"test PATCHES images range (min-max): \" +str(np.min(patches_imgs_test)) +' - '+str(np.max(patches_imgs_test)))\n\n return patches_imgs_test, test_imgs.shape[2], test_imgs.shape[3], test_masks\n\n\n#data consinstency check\ndef data_consistency_check(imgs,masks):\n assert(len(imgs.shape)==len(masks.shape))\n assert(imgs.shape[0]==masks.shape[0])\n assert(imgs.shape[2]==masks.shape[2])\n assert(imgs.shape[3]==masks.shape[3])\n assert(masks.shape[1]==1)\n assert(imgs.shape[1]==1 or imgs.shape[1]==3)\n\n\n#extract patches randomly in the full training images\n# -- Inside OR in full image\ndef extract_random(full_imgs,full_masks, patch_h,patch_w, N_patches, inside=True):\n if (N_patches%full_imgs.shape[0] != 0):\n print(\"N_patches: plase enter a multiple of 20\")\n exit()\n assert (len(full_imgs.shape)==4 and len(full_masks.shape)==4) #4D arrays\n assert (full_imgs.shape[1]==1 or full_imgs.shape[1]==3) #check the channel is 1 or 3\n assert (full_masks.shape[1]==1) #masks only black and white\n assert (full_imgs.shape[2] == full_masks.shape[2] and full_imgs.shape[3] == full_masks.shape[3])\n patches = np.empty((N_patches,full_imgs.shape[1],patch_h,patch_w))\n patches_masks = np.empty((N_patches,full_masks.shape[1],patch_h,patch_w))\n img_h = full_imgs.shape[2] #height of the full image\n img_w = full_imgs.shape[3] #width of the full image\n # (0,0) in the center of the image\n patch_per_img = int(N_patches/full_imgs.shape[0]) #N_patches equally divided in the full images\n print(\"patches per full image: \" +str(patch_per_img))\n iter_tot = 0 #iter over the total numbe rof patches (N_patches)\n for i in range(full_imgs.shape[0]): #loop over the full images\n k=0\n while k division between integers\n N_patches_tot = N_patches_img*full_imgs.shape[0]\n print(\"Number of patches on h : \" +str(((img_h-patch_h)//stride_h+1)))\n print(\"Number of patches on w : \" +str(((img_w-patch_w)//stride_w+1)))\n print(\"number of patches per image: \" +str(N_patches_img) +\", totally for this dataset: \" +str(N_patches_tot))\n patches = np.empty((N_patches_tot,full_imgs.shape[1],patch_h,patch_w))\n iter_tot = 0 #iter over the total number of patches (N_patches)\n for i in range(full_imgs.shape[0]): #loop over the full images\n for h in range((img_h-patch_h)//stride_h+1):\n for w in range((img_w-patch_w)//stride_w+1):\n patch = full_imgs[i,:,h*stride_h:(h*stride_h)+patch_h,w*stride_w:(w*stride_w)+patch_w]\n patches[iter_tot]=patch\n iter_tot +=1 #total\n assert (iter_tot==N_patches_tot)\n return patches #array with all the full_imgs divided in patches\n\n\ndef recompone_overlap(preds, img_h, img_w, stride_h, stride_w):\n assert (len(preds.shape)==4) #4D arrays\n assert (preds.shape[1]==1 or preds.shape[1]==3) #check the channel is 1 or 3\n patch_h = preds.shape[2]\n patch_w = preds.shape[3]\n N_patches_h = (img_h-patch_h)//stride_h+1\n N_patches_w = (img_w-patch_w)//stride_w+1\n N_patches_img = N_patches_h * N_patches_w\n print(\"N_patches_h: \" +str(N_patches_h))\n print(\"N_patches_w: \" +str(N_patches_w))\n print(\"N_patches_img: \" +str(N_patches_img))\n assert (preds.shape[0]%N_patches_img==0)\n N_full_imgs = preds.shape[0]//N_patches_img\n print(\"According to the dimension inserted, there are \" +str(N_full_imgs) +\" full images (of \" +str(img_h)+\"x\" +str(img_w) +\" each)\")\n full_prob = np.zeros((N_full_imgs,preds.shape[1],img_h,img_w)) #itialize to zero mega array with sum of Probabilities\n full_sum = np.zeros((N_full_imgs,preds.shape[1],img_h,img_w))\n\n k = 0 #iterator over all the patches\n for i in range(N_full_imgs):\n for h in range((img_h-patch_h)//stride_h+1):\n for w in range((img_w-patch_w)//stride_w+1):\n full_prob[i,:,h*stride_h:(h*stride_h)+patch_h,w*stride_w:(w*stride_w)+patch_w]+=preds[k]\n full_sum[i,:,h*stride_h:(h*stride_h)+patch_h,w*stride_w:(w*stride_w)+patch_w]+=1\n k+=1\n assert(k==preds.shape[0])\n assert(np.min(full_sum)>=1.0) #at least one\n final_avg = full_prob/full_sum\n print(final_avg.shape)\n assert(np.max(final_avg)<=1.0) #max value for a pixel is 1.0\n assert(np.min(final_avg)>=0.0) #min value for a pixel is 0.0\n return final_avg\n\n\n#Recompone the full images with the patches\ndef recompone(data,N_h,N_w):\n assert (data.shape[1]==1 or data.shape[1]==3) #check the channel is 1 or 3\n assert(len(data.shape)==4)\n N_pacth_per_img = N_w*N_h\n assert(data.shape[0]%N_pacth_per_img == 0)\n N_full_imgs = data.shape[0]/N_pacth_per_img\n patch_h = data.shape[2]\n patch_w = data.shape[3]\n N_pacth_per_img = N_w*N_h\n #define and start full recompone\n full_recomp = np.empty((N_full_imgs,data.shape[1],N_h*patch_h,N_w*patch_w))\n k = 0 #iter full img\n s = 0 #iter single patch\n while (s= DRIVE_masks.shape[3] or y >= DRIVE_masks.shape[2]): #my image bigger than the original\n return False\n\n if (DRIVE_masks[i,0,y,x]>0): #0==black pixels\n # print DRIVE_masks[i,0,y,x] #verify it is working right\n return True\n else:\n return False\n"
},
{
"path": "lib/extract_patches.py.bak",
"content": "import numpy as np\nimport random\nimport ConfigParser\n\nfrom help_functions import load_hdf5\nfrom help_functions import visualize\nfrom help_functions import group_images\n\nfrom pre_processing import my_PreProc\n\n\n#To select the same images\n# random.seed(10)\n\n#Load the original data and return the extracted patches for training/testing\ndef get_data_training(DRIVE_train_imgs_original,\n DRIVE_train_groudTruth,\n patch_height,\n patch_width,\n N_subimgs,\n inside_FOV):\n train_imgs_original = load_hdf5(DRIVE_train_imgs_original)\n train_masks = load_hdf5(DRIVE_train_groudTruth) #masks always the same\n # visualize(group_images(train_imgs_original[0:20,:,:,:],5),'imgs_train')#.show() #check original imgs train\n\n\n train_imgs = my_PreProc(train_imgs_original)\n train_masks = train_masks/255.\n\n train_imgs = train_imgs[:,:,9:574,:] #cut bottom and top so now it is 565*565\n train_masks = train_masks[:,:,9:574,:] #cut bottom and top so now it is 565*565\n data_consistency_check(train_imgs,train_masks)\n\n #check masks are within 0-1\n assert(np.min(train_masks)==0 and np.max(train_masks)==1)\n\n print \"\\ntrain images/masks shape:\"\n print train_imgs.shape\n print \"train images range (min-max): \" +str(np.min(train_imgs)) +' - '+str(np.max(train_imgs))\n print \"train masks are within 0-1\\n\"\n\n #extract the TRAINING patches from the full images\n patches_imgs_train, patches_masks_train = extract_random(train_imgs,train_masks,patch_height,patch_width,N_subimgs,inside_FOV)\n data_consistency_check(patches_imgs_train, patches_masks_train)\n\n print \"\\ntrain PATCHES images/masks shape:\"\n print patches_imgs_train.shape\n print \"train PATCHES images range (min-max): \" +str(np.min(patches_imgs_train)) +' - '+str(np.max(patches_imgs_train))\n\n return patches_imgs_train, patches_masks_train#, patches_imgs_test, patches_masks_test\n\n\n#Load the original data and return the extracted patches for training/testing\ndef get_data_testing(DRIVE_test_imgs_original, DRIVE_test_groudTruth, Imgs_to_test, patch_height, patch_width):\n ### test\n test_imgs_original = load_hdf5(DRIVE_test_imgs_original)\n test_masks = load_hdf5(DRIVE_test_groudTruth)\n\n test_imgs = my_PreProc(test_imgs_original)\n test_masks = test_masks/255.\n\n #extend both images and masks so they can be divided exactly by the patches dimensions\n test_imgs = test_imgs[0:Imgs_to_test,:,:,:]\n test_masks = test_masks[0:Imgs_to_test,:,:,:]\n test_imgs = paint_border(test_imgs,patch_height,patch_width)\n test_masks = paint_border(test_masks,patch_height,patch_width)\n\n data_consistency_check(test_imgs, test_masks)\n\n #check masks are within 0-1\n assert(np.max(test_masks)==1 and np.min(test_masks)==0)\n\n print \"\\ntest images/masks shape:\"\n print test_imgs.shape\n print \"test images range (min-max): \" +str(np.min(test_imgs)) +' - '+str(np.max(test_imgs))\n print \"test masks are within 0-1\\n\"\n\n #extract the TEST patches from the full images\n patches_imgs_test = extract_ordered(test_imgs,patch_height,patch_width)\n patches_masks_test = extract_ordered(test_masks,patch_height,patch_width)\n data_consistency_check(patches_imgs_test, patches_masks_test)\n\n print \"\\ntest PATCHES images/masks shape:\"\n print patches_imgs_test.shape\n print \"test PATCHES images range (min-max): \" +str(np.min(patches_imgs_test)) +' - '+str(np.max(patches_imgs_test))\n\n return patches_imgs_test, patches_masks_test\n\n\n\n\n# Load the original data and return the extracted patches for testing\n# return the ground truth in its original shape\ndef get_data_testing_overlap(DRIVE_test_imgs_original, DRIVE_test_groudTruth, Imgs_to_test, patch_height, patch_width, stride_height, stride_width):\n ### test\n test_imgs_original = load_hdf5(DRIVE_test_imgs_original)\n test_masks = load_hdf5(DRIVE_test_groudTruth)\n\n test_imgs = my_PreProc(test_imgs_original)\n test_masks = test_masks/255.\n #extend both images and masks so they can be divided exactly by the patches dimensions\n test_imgs = test_imgs[0:Imgs_to_test,:,:,:]\n test_masks = test_masks[0:Imgs_to_test,:,:,:]\n test_imgs = paint_border_overlap(test_imgs, patch_height, patch_width, stride_height, stride_width)\n\n #check masks are within 0-1\n assert(np.max(test_masks)==1 and np.min(test_masks)==0)\n\n print \"\\ntest images shape:\"\n print test_imgs.shape\n print \"\\ntest mask shape:\"\n print test_masks.shape\n print \"test images range (min-max): \" +str(np.min(test_imgs)) +' - '+str(np.max(test_imgs))\n print \"test masks are within 0-1\\n\"\n\n #extract the TEST patches from the full images\n patches_imgs_test = extract_ordered_overlap(test_imgs,patch_height,patch_width,stride_height,stride_width)\n\n print \"\\ntest PATCHES images shape:\"\n print patches_imgs_test.shape\n print \"test PATCHES images range (min-max): \" +str(np.min(patches_imgs_test)) +' - '+str(np.max(patches_imgs_test))\n\n return patches_imgs_test, test_imgs.shape[2], test_imgs.shape[3], test_masks\n\n\n#data consinstency check\ndef data_consistency_check(imgs,masks):\n assert(len(imgs.shape)==len(masks.shape))\n assert(imgs.shape[0]==masks.shape[0])\n assert(imgs.shape[2]==masks.shape[2])\n assert(imgs.shape[3]==masks.shape[3])\n assert(masks.shape[1]==1)\n assert(imgs.shape[1]==1 or imgs.shape[1]==3)\n\n\n#extract patches randomly in the full training images\n# -- Inside OR in full image\ndef extract_random(full_imgs,full_masks, patch_h,patch_w, N_patches, inside=True):\n if (N_patches%full_imgs.shape[0] != 0):\n print \"N_patches: plase enter a multiple of 20\"\n exit()\n assert (len(full_imgs.shape)==4 and len(full_masks.shape)==4) #4D arrays\n assert (full_imgs.shape[1]==1 or full_imgs.shape[1]==3) #check the channel is 1 or 3\n assert (full_masks.shape[1]==1) #masks only black and white\n assert (full_imgs.shape[2] == full_masks.shape[2] and full_imgs.shape[3] == full_masks.shape[3])\n patches = np.empty((N_patches,full_imgs.shape[1],patch_h,patch_w))\n patches_masks = np.empty((N_patches,full_masks.shape[1],patch_h,patch_w))\n img_h = full_imgs.shape[2] #height of the full image\n img_w = full_imgs.shape[3] #width of the full image\n # (0,0) in the center of the image\n patch_per_img = int(N_patches/full_imgs.shape[0]) #N_patches equally divided in the full images\n print \"patches per full image: \" +str(patch_per_img)\n iter_tot = 0 #iter over the total numbe rof patches (N_patches)\n for i in range(full_imgs.shape[0]): #loop over the full images\n k=0\n while k division between integers\n N_patches_tot = N_patches_img*full_imgs.shape[0]\n print \"Number of patches on h : \" +str(((img_h-patch_h)//stride_h+1))\n print \"Number of patches on w : \" +str(((img_w-patch_w)//stride_w+1))\n print \"number of patches per image: \" +str(N_patches_img) +\", totally for this dataset: \" +str(N_patches_tot)\n patches = np.empty((N_patches_tot,full_imgs.shape[1],patch_h,patch_w))\n iter_tot = 0 #iter over the total number of patches (N_patches)\n for i in range(full_imgs.shape[0]): #loop over the full images\n for h in range((img_h-patch_h)//stride_h+1):\n for w in range((img_w-patch_w)//stride_w+1):\n patch = full_imgs[i,:,h*stride_h:(h*stride_h)+patch_h,w*stride_w:(w*stride_w)+patch_w]\n patches[iter_tot]=patch\n iter_tot +=1 #total\n assert (iter_tot==N_patches_tot)\n return patches #array with all the full_imgs divided in patches\n\n\ndef recompone_overlap(preds, img_h, img_w, stride_h, stride_w):\n assert (len(preds.shape)==4) #4D arrays\n assert (preds.shape[1]==1 or preds.shape[1]==3) #check the channel is 1 or 3\n patch_h = preds.shape[2]\n patch_w = preds.shape[3]\n N_patches_h = (img_h-patch_h)//stride_h+1\n N_patches_w = (img_w-patch_w)//stride_w+1\n N_patches_img = N_patches_h * N_patches_w\n print \"N_patches_h: \" +str(N_patches_h)\n print \"N_patches_w: \" +str(N_patches_w)\n print \"N_patches_img: \" +str(N_patches_img)\n assert (preds.shape[0]%N_patches_img==0)\n N_full_imgs = preds.shape[0]//N_patches_img\n print \"According to the dimension inserted, there are \" +str(N_full_imgs) +\" full images (of \" +str(img_h)+\"x\" +str(img_w) +\" each)\"\n full_prob = np.zeros((N_full_imgs,preds.shape[1],img_h,img_w)) #itialize to zero mega array with sum of Probabilities\n full_sum = np.zeros((N_full_imgs,preds.shape[1],img_h,img_w))\n\n k = 0 #iterator over all the patches\n for i in range(N_full_imgs):\n for h in range((img_h-patch_h)//stride_h+1):\n for w in range((img_w-patch_w)//stride_w+1):\n full_prob[i,:,h*stride_h:(h*stride_h)+patch_h,w*stride_w:(w*stride_w)+patch_w]+=preds[k]\n full_sum[i,:,h*stride_h:(h*stride_h)+patch_h,w*stride_w:(w*stride_w)+patch_w]+=1\n k+=1\n assert(k==preds.shape[0])\n assert(np.min(full_sum)>=1.0) #at least one\n final_avg = full_prob/full_sum\n print final_avg.shape\n assert(np.max(final_avg)<=1.0) #max value for a pixel is 1.0\n assert(np.min(final_avg)>=0.0) #min value for a pixel is 0.0\n return final_avg\n\n\n#Recompone the full images with the patches\ndef recompone(data,N_h,N_w):\n assert (data.shape[1]==1 or data.shape[1]==3) #check the channel is 1 or 3\n assert(len(data.shape)==4)\n N_pacth_per_img = N_w*N_h\n assert(data.shape[0]%N_pacth_per_img == 0)\n N_full_imgs = data.shape[0]/N_pacth_per_img\n patch_h = data.shape[2]\n patch_w = data.shape[3]\n N_pacth_per_img = N_w*N_h\n #define and start full recompone\n full_recomp = np.empty((N_full_imgs,data.shape[1],N_h*patch_h,N_w*patch_w))\n k = 0 #iter full img\n s = 0 #iter single patch\n while (s= DRIVE_masks.shape[3] or y >= DRIVE_masks.shape[2]): #my image bigger than the original\n return False\n\n if (DRIVE_masks[i,0,y,x]>0): #0==black pixels\n # print DRIVE_masks[i,0,y,x] #verify it is working right\n return True\n else:\n return False\n"
},
{
"path": "lib/help_functions.py",
"content": "import h5py\nimport numpy as np\nfrom PIL import Image\nfrom matplotlib import pyplot as plt\n\ndef load_hdf5(infile):\n with h5py.File(infile,\"r\") as f: #\"with\" close the file after its nested commands\n return f[\"image\"][()]\n\ndef write_hdf5(arr,outfile):\n with h5py.File(outfile,\"w\") as f:\n f.create_dataset(\"image\", data=arr, dtype=arr.dtype)\n\n#convert RGB image in black and white\ndef rgb2gray(rgb):\n assert (len(rgb.shape)==4) #4D arrays\n assert (rgb.shape[1]==3)\n bn_imgs = rgb[:,0,:,:]*0.299 + rgb[:,1,:,:]*0.587 + rgb[:,2,:,:]*0.114\n bn_imgs = np.reshape(bn_imgs,(rgb.shape[0],1,rgb.shape[2],rgb.shape[3]))\n return bn_imgs\n\n#group a set of images row per columns\ndef group_images(data,per_row):\n assert data.shape[0]%per_row==0\n assert (data.shape[1]==1 or data.shape[1]==3)\n data = np.transpose(data,(0,2,3,1)) #corect format for imshow\n all_stripe = []\n for i in range(int(data.shape[0]/per_row)):\n stripe = data[i*per_row]\n for k in range(i*per_row+1, i*per_row+per_row):\n stripe = np.concatenate((stripe,data[k]),axis=1)\n all_stripe.append(stripe)\n totimg = all_stripe[0]\n for i in range(1,len(all_stripe)):\n totimg = np.concatenate((totimg,all_stripe[i]),axis=0)\n return totimg\n\n\n#visualize image (as PIL image, NOT as matplotlib!)\ndef visualize(data,filename):\n assert (len(data.shape)==3) #height*width*channels\n img = None\n if data.shape[2]==1: #in case it is black and white\n data = np.reshape(data,(data.shape[0],data.shape[1]))\n if np.max(data)>1:\n img = Image.fromarray(data.astype(np.uint8)) #the image is already 0-255\n else:\n img = Image.fromarray((data*255).astype(np.uint8)) #the image is between 0-1\n img.save(filename + '.png')\n return img\n\n\n#prepare the mask in the right shape for the Unet\ndef masks_Unet(masks):\n assert (len(masks.shape)==4) #4D arrays\n assert (masks.shape[1]==1 ) #check the channel is 1\n im_h = masks.shape[2]\n im_w = masks.shape[3]\n masks = np.reshape(masks,(masks.shape[0],im_h*im_w))\n new_masks = np.empty((masks.shape[0],im_h*im_w,2))\n #new_masks[np.where(masks==0),0]=1\n #new_masks[np.where(masks==1),1]=1\n\n for i in range(masks.shape[0]):\n for j in range(im_h*im_w):\n if masks[i,j] == 0:\n new_masks[i,j,0]=1\n new_masks[i,j,1]=0\n else:\n new_masks[i,j,0]=0\n new_masks[i,j,1]=1\n\n return new_masks\n\n\ndef pred_to_imgs(pred, patch_height, patch_width, mode=\"original\"):\n assert (len(pred.shape)==3) #3D array: (Npatches,height*width,2)\n assert (pred.shape[2]==2 ) #check the classes are 2\n pred_images = np.empty((pred.shape[0],pred.shape[1])) #(Npatches,height*width)\n if mode==\"original\":\n for i in range(pred.shape[0]):\n for pix in range(pred.shape[1]):\n pred_images[i,pix]=pred[i,pix,1]\n elif mode==\"threshold\":\n for i in range(pred.shape[0]):\n for pix in range(pred.shape[1]):\n if pred[i,pix,1]>=0.5:\n pred_images[i,pix]=1\n else:\n pred_images[i,pix]=0\n else:\n print(\"mode \" +str(mode) +\" not recognized, it can be 'original' or 'threshold'\")\n exit()\n pred_images = np.reshape(pred_images,(pred_images.shape[0],1, patch_height, patch_width))\n return pred_images\n"
},
{
"path": "lib/help_functions.py.bak",
"content": "import h5py\nimport numpy as np\nfrom PIL import Image\nfrom matplotlib import pyplot as plt\n\ndef load_hdf5(infile):\n with h5py.File(infile,\"r\") as f: #\"with\" close the file after its nested commands\n return f[\"image\"][()]\n\ndef write_hdf5(arr,outfile):\n with h5py.File(outfile,\"w\") as f:\n f.create_dataset(\"image\", data=arr, dtype=arr.dtype)\n\n#convert RGB image in black and white\ndef rgb2gray(rgb):\n assert (len(rgb.shape)==4) #4D arrays\n assert (rgb.shape[1]==3)\n bn_imgs = rgb[:,0,:,:]*0.299 + rgb[:,1,:,:]*0.587 + rgb[:,2,:,:]*0.114\n bn_imgs = np.reshape(bn_imgs,(rgb.shape[0],1,rgb.shape[2],rgb.shape[3]))\n return bn_imgs\n\n#group a set of images row per columns\ndef group_images(data,per_row):\n assert data.shape[0]%per_row==0\n assert (data.shape[1]==1 or data.shape[1]==3)\n data = np.transpose(data,(0,2,3,1)) #corect format for imshow\n all_stripe = []\n for i in range(int(data.shape[0]/per_row)):\n stripe = data[i*per_row]\n for k in range(i*per_row+1, i*per_row+per_row):\n stripe = np.concatenate((stripe,data[k]),axis=1)\n all_stripe.append(stripe)\n totimg = all_stripe[0]\n for i in range(1,len(all_stripe)):\n totimg = np.concatenate((totimg,all_stripe[i]),axis=0)\n return totimg\n\n\n#visualize image (as PIL image, NOT as matplotlib!)\ndef visualize(data,filename):\n assert (len(data.shape)==3) #height*width*channels\n img = None\n if data.shape[2]==1: #in case it is black and white\n data = np.reshape(data,(data.shape[0],data.shape[1]))\n if np.max(data)>1:\n img = Image.fromarray(data.astype(np.uint8)) #the image is already 0-255\n else:\n img = Image.fromarray((data*255).astype(np.uint8)) #the image is between 0-1\n img.save(filename + '.png')\n return img\n\n\n#prepare the mask in the right shape for the Unet\ndef masks_Unet(masks):\n assert (len(masks.shape)==4) #4D arrays\n assert (masks.shape[1]==1 ) #check the channel is 1\n im_h = masks.shape[2]\n im_w = masks.shape[3]\n masks = np.reshape(masks,(masks.shape[0],im_h*im_w))\n new_masks = np.empty((masks.shape[0],im_h*im_w,2))\n for i in range(masks.shape[0]):\n for j in range(im_h*im_w):\n if masks[i,j] == 0:\n new_masks[i,j,0]=1\n new_masks[i,j,1]=0\n else:\n new_masks[i,j,0]=0\n new_masks[i,j,1]=1\n return new_masks\n\n\ndef pred_to_imgs(pred, patch_height, patch_width, mode=\"original\"):\n assert (len(pred.shape)==3) #3D array: (Npatches,height*width,2)\n assert (pred.shape[2]==2 ) #check the classes are 2\n pred_images = np.empty((pred.shape[0],pred.shape[1])) #(Npatches,height*width)\n if mode==\"original\":\n for i in range(pred.shape[0]):\n for pix in range(pred.shape[1]):\n pred_images[i,pix]=pred[i,pix,1]\n elif mode==\"threshold\":\n for i in range(pred.shape[0]):\n for pix in range(pred.shape[1]):\n if pred[i,pix,1]>=0.5:\n pred_images[i,pix]=1\n else:\n pred_images[i,pix]=0\n else:\n print \"mode \" +str(mode) +\" not recognized, it can be 'original' or 'threshold'\"\n exit()\n pred_images = np.reshape(pred_images,(pred_images.shape[0],1, patch_height, patch_width))\n return pred_images\n"
},
{
"path": "lib/pre_processing.py",
"content": "###################################################\n#\n# Script to pre-process the original imgs\n#\n##################################################\n\n\nimport numpy as np\nfrom PIL import Image\nimport cv2\n\nfrom .help_functions import *\n\n\n#My pre processing (use for both training and testing!)\ndef my_PreProc(data):\n assert(len(data.shape)==4)\n assert (data.shape[1]==3) #Use the original images\n #black-white conversion\n train_imgs = rgb2gray(data)\n #my preprocessing:\n train_imgs = dataset_normalized(train_imgs)\n train_imgs = clahe_equalized(train_imgs)\n train_imgs = adjust_gamma(train_imgs, 1.2)\n train_imgs = train_imgs/255. #reduce to 0-1 range\n return train_imgs\n\n\n#============================================================\n#========= PRE PROCESSING FUNCTIONS ========================#\n#============================================================\n\n#==== histogram equalization\ndef histo_equalized(imgs):\n assert (len(imgs.shape)==4) #4D arrays\n assert (imgs.shape[1]==1) #check the channel is 1\n imgs_equalized = np.empty(imgs.shape)\n for i in range(imgs.shape[0]):\n imgs_equalized[i,0] = cv2.equalizeHist(np.array(imgs[i,0], dtype = np.uint8))\n return imgs_equalized\n\n\n# CLAHE (Contrast Limited Adaptive Histogram Equalization)\n#adaptive histogram equalization is used. In this, image is divided into small blocks called \"tiles\" (tileSize is 8x8 by default in OpenCV). Then each of these blocks are histogram equalized as usual. So in a small area, histogram would confine to a small region (unless there is noise). If noise is there, it will be amplified. To avoid this, contrast limiting is applied. If any histogram bin is above the specified contrast limit (by default 40 in OpenCV), those pixels are clipped and distributed uniformly to other bins before applying histogram equalization. After equalization, to remove artifacts in tile borders, bilinear interpolation is applied\ndef clahe_equalized(imgs):\n assert (len(imgs.shape)==4) #4D arrays\n assert (imgs.shape[1]==1) #check the channel is 1\n #create a CLAHE object (Arguments are optional).\n clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8))\n imgs_equalized = np.empty(imgs.shape)\n for i in range(imgs.shape[0]):\n imgs_equalized[i,0] = clahe.apply(np.array(imgs[i,0], dtype = np.uint8))\n return imgs_equalized\n\n\n# ===== normalize over the dataset\ndef dataset_normalized(imgs):\n assert (len(imgs.shape)==4) #4D arrays\n assert (imgs.shape[1]==1) #check the channel is 1\n imgs_normalized = np.empty(imgs.shape)\n imgs_std = np.std(imgs)\n imgs_mean = np.mean(imgs)\n imgs_normalized = (imgs-imgs_mean)/imgs_std\n for i in range(imgs.shape[0]):\n imgs_normalized[i] = ((imgs_normalized[i] - np.min(imgs_normalized[i])) / (np.max(imgs_normalized[i])-np.min(imgs_normalized[i])))*255\n return imgs_normalized\n\n\ndef adjust_gamma(imgs, gamma=1.0):\n assert (len(imgs.shape)==4) #4D arrays\n assert (imgs.shape[1]==1) #check the channel is 1\n # build a lookup table mapping the pixel values [0, 255] to\n # their adjusted gamma values\n invGamma = 1.0 / gamma\n table = np.array([((i / 255.0) ** invGamma) * 255 for i in np.arange(0, 256)]).astype(\"uint8\")\n # apply gamma correction using the lookup table\n new_imgs = np.empty(imgs.shape)\n for i in range(imgs.shape[0]):\n new_imgs[i,0] = cv2.LUT(np.array(imgs[i,0], dtype = np.uint8), table)\n return new_imgs\n"
},
{
"path": "lib/pre_processing.py.bak",
"content": "###################################################\n#\n# Script to pre-process the original imgs\n#\n##################################################\n\n\nimport numpy as np\nfrom PIL import Image\nimport cv2\n\nfrom help_functions import *\n\n\n#My pre processing (use for both training and testing!)\ndef my_PreProc(data):\n assert(len(data.shape)==4)\n assert (data.shape[1]==3) #Use the original images\n #black-white conversion\n train_imgs = rgb2gray(data)\n #my preprocessing:\n train_imgs = dataset_normalized(train_imgs)\n train_imgs = clahe_equalized(train_imgs)\n train_imgs = adjust_gamma(train_imgs, 1.2)\n train_imgs = train_imgs/255. #reduce to 0-1 range\n return train_imgs\n\n\n#============================================================\n#========= PRE PROCESSING FUNCTIONS ========================#\n#============================================================\n\n#==== histogram equalization\ndef histo_equalized(imgs):\n assert (len(imgs.shape)==4) #4D arrays\n assert (imgs.shape[1]==1) #check the channel is 1\n imgs_equalized = np.empty(imgs.shape)\n for i in range(imgs.shape[0]):\n imgs_equalized[i,0] = cv2.equalizeHist(np.array(imgs[i,0], dtype = np.uint8))\n return imgs_equalized\n\n\n# CLAHE (Contrast Limited Adaptive Histogram Equalization)\n#adaptive histogram equalization is used. In this, image is divided into small blocks called \"tiles\" (tileSize is 8x8 by default in OpenCV). Then each of these blocks are histogram equalized as usual. So in a small area, histogram would confine to a small region (unless there is noise). If noise is there, it will be amplified. To avoid this, contrast limiting is applied. If any histogram bin is above the specified contrast limit (by default 40 in OpenCV), those pixels are clipped and distributed uniformly to other bins before applying histogram equalization. After equalization, to remove artifacts in tile borders, bilinear interpolation is applied\ndef clahe_equalized(imgs):\n assert (len(imgs.shape)==4) #4D arrays\n assert (imgs.shape[1]==1) #check the channel is 1\n #create a CLAHE object (Arguments are optional).\n clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8))\n imgs_equalized = np.empty(imgs.shape)\n for i in range(imgs.shape[0]):\n imgs_equalized[i,0] = clahe.apply(np.array(imgs[i,0], dtype = np.uint8))\n return imgs_equalized\n\n\n# ===== normalize over the dataset\ndef dataset_normalized(imgs):\n assert (len(imgs.shape)==4) #4D arrays\n assert (imgs.shape[1]==1) #check the channel is 1\n imgs_normalized = np.empty(imgs.shape)\n imgs_std = np.std(imgs)\n imgs_mean = np.mean(imgs)\n imgs_normalized = (imgs-imgs_mean)/imgs_std\n for i in range(imgs.shape[0]):\n imgs_normalized[i] = ((imgs_normalized[i] - np.min(imgs_normalized[i])) / (np.max(imgs_normalized[i])-np.min(imgs_normalized[i])))*255\n return imgs_normalized\n\n\ndef adjust_gamma(imgs, gamma=1.0):\n assert (len(imgs.shape)==4) #4D arrays\n assert (imgs.shape[1]==1) #check the channel is 1\n # build a lookup table mapping the pixel values [0, 255] to\n # their adjusted gamma values\n invGamma = 1.0 / gamma\n table = np.array([((i / 255.0) ** invGamma) * 255 for i in np.arange(0, 256)]).astype(\"uint8\")\n # apply gamma correction using the lookup table\n new_imgs = np.empty(imgs.shape)\n for i in range(imgs.shape[0]):\n new_imgs[i,0] = cv2.LUT(np.array(imgs[i,0], dtype = np.uint8), table)\n return new_imgs\n"
},
{
"path": "prepare_datasets_DRIVE.py",
"content": "#==========================================================\n#\n# This prepare the hdf5 datasets of the DRIVE database\n#\n#============================================================\n\nimport os\nimport h5py\nimport numpy as np\nfrom PIL import Image\n\n\n\ndef write_hdf5(arr,outfile):\n with h5py.File(outfile,\"w\") as f:\n f.create_dataset(\"image\", data=arr, dtype=arr.dtype)\n\n\n#------------Path of the images --------------------------------------------------------------\n#train\noriginal_imgs_train = \"./DRIVE/training/images/\"\ngroundTruth_imgs_train = \"./DRIVE/training/1st_manual/\"\nborderMasks_imgs_train = \"./DRIVE/training/mask/\"\n#test\noriginal_imgs_test = \"./DRIVE/test/images/\"\ngroundTruth_imgs_test = \"./DRIVE/test/1st_manual/\"\nborderMasks_imgs_test = \"./DRIVE/test/mask/\"\n#---------------------------------------------------------------------------------------------\n\nNimgs = 20\nchannels = 3\nheight = 584\nwidth = 565\ndataset_path = \"./DRIVE_datasets_training_testing/\"\n\nif not os.path.isdir(dataset_path):\n os.mkdir(dataset_path)\n\ndef get_datasets(imgs_dir,groundTruth_dir,borderMasks_dir,train_test=\"null\"):\n imgs = np.empty((Nimgs,height,width,channels))\n groundTruth = np.empty((Nimgs,height,width))\n border_masks = np.empty((Nimgs,height,width))\n for path, subdirs, files in os.walk(imgs_dir): #list all files, directories in the path\n for i in range(len(files)):\n #original\n print(\"original image: \" +files[i])\n img = Image.open(imgs_dir+files[i])\n imgs[i] = np.asarray(img)\n #corresponding ground truth\n groundTruth_name = files[i][0:2] + \"_manual1.gif\"\n print(\"ground truth name: \" + groundTruth_name)\n g_truth = Image.open(groundTruth_dir + groundTruth_name)\n groundTruth[i] = np.asarray(g_truth)\n #corresponding border masks\n border_masks_name = \"\"\n if train_test==\"train\":\n border_masks_name = files[i][0:2] + \"_training_mask.gif\"\n elif train_test==\"test\":\n border_masks_name = files[i][0:2] + \"_test_mask.gif\"\n else:\n print(\"specify if train or test!!\")\n exit()\n print(\"border masks name: \" + border_masks_name)\n b_mask = Image.open(borderMasks_dir + border_masks_name)\n border_masks[i] = np.asarray(b_mask)\n\n print(\"imgs max: \" +str(np.max(imgs)))\n print(\"imgs min: \" +str(np.min(imgs)))\n assert(np.max(groundTruth)==255 and np.max(border_masks)==255)\n assert(np.min(groundTruth)==0 and np.min(border_masks)==0)\n print(\"ground truth and border masks are correctly withih pixel value range 0-255 (black-white)\")\n #reshaping for my standard tensors\n imgs = np.transpose(imgs,(0,3,1,2))\n assert(imgs.shape == (Nimgs,channels,height,width))\n groundTruth = np.reshape(groundTruth,(Nimgs,1,height,width))\n border_masks = np.reshape(border_masks,(Nimgs,1,height,width))\n assert(groundTruth.shape == (Nimgs,1,height,width))\n assert(border_masks.shape == (Nimgs,1,height,width))\n return imgs, groundTruth, border_masks\n\n\n#getting the training datasets\nimgs_train, groundTruth_train, border_masks_train = get_datasets(original_imgs_train,groundTruth_imgs_train,borderMasks_imgs_train,\"train\")\nprint(\"saving train datasets\")\nwrite_hdf5(imgs_train, dataset_path + \"DRIVE_dataset_imgs_train.hdf5\")\nwrite_hdf5(groundTruth_train, dataset_path + \"DRIVE_dataset_groundTruth_train.hdf5\")\nwrite_hdf5(border_masks_train,dataset_path + \"DRIVE_dataset_borderMasks_train.hdf5\")\n\n#getting the testing datasets\nimgs_test, groundTruth_test, border_masks_test = get_datasets(original_imgs_test,groundTruth_imgs_test,borderMasks_imgs_test,\"test\")\nprint(\"saving test datasets\")\nwrite_hdf5(imgs_test,dataset_path + \"DRIVE_dataset_imgs_test.hdf5\")\nwrite_hdf5(groundTruth_test, dataset_path + \"DRIVE_dataset_groundTruth_test.hdf5\")\nwrite_hdf5(border_masks_test,dataset_path + \"DRIVE_dataset_borderMasks_test.hdf5\")\n"
},
{
"path": "src/LadderNetv65.py",
"content": "import torch\nimport torch.nn.functional as F\nimport torch.nn as nn\n\ndrop = 0.25\n\ndef conv3x3(in_planes, out_planes, stride=1):\n \"\"\"3x3 convolution with padding\"\"\"\n return nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride,\n padding=1, bias=True)\n\n\nclass BasicBlock(nn.Module):\n expansion = 1\n\n def __init__(self, inplanes, planes, stride=1, downsample=None):\n super(BasicBlock, self).__init__()\n if inplanes!= planes:\n self.conv0 = conv3x3(inplanes,planes)\n\n self.inplanes = inplanes\n self.planes = planes\n\n self.conv1 = conv3x3(planes, planes, stride)\n #self.bn1 = nn.BatchNorm2d(planes)\n self.relu = nn.ReLU(inplace=True)\n #self.conv2 = conv3x3(planes, planes)\n #self.bn2 = nn.BatchNorm2d(planes)\n self.downsample = downsample\n self.stride = stride\n self.drop = nn.Dropout2d(p=drop)\n\n def forward(self, x):\n if self.inplanes != self.planes:\n x = self.conv0(x)\n x = F.relu(x)\n\n out = self.conv1(x)\n #out = self.bn1(out)\n out = self.relu(out)\n\n out = self.drop(out)\n\n out1 = self.conv1(out)\n #out1 = self.relu(out1)\n\n out2 = out1 + x\n\n return F.relu(out2)\n\n\nclass Bottleneck(nn.Module):\n expansion = 4\n\n def __init__(self, inplanes, planes, stride=1, downsample=None):\n super(Bottleneck, self).__init__()\n self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, bias=False)\n self.bn1 = nn.BatchNorm2d(planes)\n self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride,\n padding=1, bias=False)\n self.bn2 = nn.BatchNorm2d(planes)\n self.conv3 = nn.Conv2d(planes, planes * self.expansion, kernel_size=1, bias=False)\n self.bn3 = nn.BatchNorm2d(planes * self.expansion)\n self.relu = nn.ReLU(inplace=True)\n self.downsample = downsample\n self.stride = stride\n\n def forward(self, x):\n residual = x\n\n out = self.conv1(x)\n out = self.bn1(out)\n out = self.relu(out)\n\n out = self.conv2(out)\n out = self.bn2(out)\n out = self.relu(out)\n\n out = self.conv3(out)\n out = self.bn3(out)\n\n if self.downsample is not None:\n residual = self.downsample(x)\n\n out += residual\n out = self.relu(out)\n\n return out\n\nclass Initial_LadderBlock(nn.Module):\n\n def __init__(self,planes,layers,kernel=3,block=BasicBlock,inplanes = 3):\n super().__init__()\n self.planes = planes\n self.layers = layers\n self.kernel = kernel\n\n self.padding = int((kernel-1)/2)\n self.inconv = nn.Conv2d(in_channels=inplanes,out_channels=planes,\n kernel_size=3,stride=1,padding=1,bias=True)\n\n # create module list for down branch\n self.down_module_list = nn.ModuleList()\n for i in range(0,layers):\n self.down_module_list.append(block(planes*(2**i),planes*(2**i)))\n\n # use strided conv instead of poooling\n self.down_conv_list = nn.ModuleList()\n for i in range(0,layers):\n self.down_conv_list.append(nn.Conv2d(planes*2**i,planes*2**(i+1),stride=2,kernel_size=kernel,padding=self.padding))\n\n # create module for bottom block\n self.bottom = block(planes*(2**layers),planes*(2**layers))\n\n # create module list for up branch\n self.up_conv_list = nn.ModuleList()\n self.up_dense_list = nn.ModuleList()\n for i in range(0, layers):\n self.up_conv_list.append(nn.ConvTranspose2d(in_channels=planes*2**(layers-i), out_channels=planes*2**max(0,layers-i-1), kernel_size=3,\n stride=2,padding=1,output_padding=1,bias=True))\n self.up_dense_list.append(block(planes*2**max(0,layers-i-1),planes*2**max(0,layers-i-1)))\n\n\n def forward(self, x):\n out = self.inconv(x)\n out = F.relu(out)\n\n down_out = []\n # down branch\n for i in range(0,self.layers):\n out = self.down_module_list[i](out)\n down_out.append(out)\n out = self.down_conv_list[i](out)\n out = F.relu(out)\n\n # bottom branch\n out = self.bottom(out)\n bottom = out\n\n # up branch\n up_out = []\n up_out.append(bottom)\n\n for j in range(0,self.layers):\n out = self.up_conv_list[j](out) + down_out[self.layers-j-1]\n #out = F.relu(out)\n out = self.up_dense_list[j](out)\n up_out.append(out)\n\n return up_out\n\nclass LadderBlock(nn.Module):\n\n def __init__(self,planes,layers,kernel=3,block=BasicBlock,inplanes = 3):\n super().__init__()\n self.planes = planes\n self.layers = layers\n self.kernel = kernel\n\n self.padding = int((kernel-1)/2)\n self.inconv = block(planes,planes)\n\n # create module list for down branch\n self.down_module_list = nn.ModuleList()\n for i in range(0,layers):\n self.down_module_list.append(block(planes*(2**i),planes*(2**i)))\n\n # use strided conv instead of poooling\n self.down_conv_list = nn.ModuleList()\n for i in range(0,layers):\n self.down_conv_list.append(nn.Conv2d(planes*2**i,planes*2**(i+1),stride=2,kernel_size=kernel,padding=self.padding))\n\n # create module for bottom block\n self.bottom = block(planes*(2**layers),planes*(2**layers))\n\n # create module list for up branch\n self.up_conv_list = nn.ModuleList()\n self.up_dense_list = nn.ModuleList()\n for i in range(0, layers):\n self.up_conv_list.append(nn.ConvTranspose2d(planes*2**(layers-i), planes*2**max(0,layers-i-1), kernel_size=3,\n stride=2,padding=1,output_padding=1,bias=True))\n self.up_dense_list.append(block(planes*2**max(0,layers-i-1),planes*2**max(0,layers-i-1)))\n\n\n def forward(self, x):\n out = self.inconv(x[-1])\n\n down_out = []\n # down branch\n for i in range(0,self.layers):\n out = out + x[-i-1]\n out = self.down_module_list[i](out)\n down_out.append(out)\n\n out = self.down_conv_list[i](out)\n out = F.relu(out)\n\n # bottom branch\n out = self.bottom(out)\n bottom = out\n\n # up branch\n up_out = []\n up_out.append(bottom)\n\n for j in range(0,self.layers):\n out = self.up_conv_list[j](out) + down_out[self.layers-j-1]\n #out = F.relu(out)\n out = self.up_dense_list[j](out)\n up_out.append(out)\n\n return up_out\n\nclass Final_LadderBlock(nn.Module):\n\n def __init__(self,planes,layers,kernel=3,block=BasicBlock,inplanes = 3):\n super().__init__()\n self.block = LadderBlock(planes,layers,kernel=kernel,block=block)\n\n def forward(self, x):\n out = self.block(x)\n return out[-1]\n\nclass LadderNetv6(nn.Module):\n def __init__(self,layers=3,filters=16,num_classes=2,inplanes=3):\n super().__init__()\n self.initial_block = Initial_LadderBlock(planes=filters,layers=layers,inplanes=inplanes)\n #self.middle_block = LadderBlock(planes=filters,layers=layers)\n self.final_block = Final_LadderBlock(planes=filters,layers=layers)\n self.final = nn.Conv2d(in_channels=filters,out_channels=num_classes,kernel_size=1)\n\n def forward(self,x):\n out = self.initial_block(x)\n #out = self.middle_block(out)\n out = self.final_block(out)\n out = self.final(out)\n #out = F.relu(out)\n out = F.log_softmax(out,dim=1)\n return out"
},
{
"path": "src/__init__.py",
"content": ""
},
{
"path": "src/losses.py",
"content": "import torch\nimport numpy as np\nimport torch.nn as nn\n\ndef cuda(x):\n return x.cuda(async=True) if torch.cuda.is_available() else x\n\n\nclass LossMulti:\n def __init__(self, jaccard_weight=0, class_weights=None, num_classes=1):\n if class_weights is not None:\n nll_weight = cuda(\n torch.from_numpy(class_weights.astype(np.float32)))\n else:\n nll_weight = None\n self.nll_loss = nn.NLLLoss2d(weight=nll_weight)\n self.jaccard_weight = jaccard_weight\n self.num_classes = num_classes\n\n def __call__(self, outputs, targets):\n loss = (1 - self.jaccard_weight) * self.nll_loss(outputs, targets)\n\n if self.jaccard_weight:\n eps = 1e-15\n for cls in range(self.num_classes):\n jaccard_target = (targets == cls).float()\n jaccard_output = outputs[:, cls].exp()\n intersection = (jaccard_output * jaccard_target).sum()\n\n union = jaccard_output.sum() + jaccard_target.sum()\n loss -= torch.log((intersection + eps) / (union - intersection + eps)) * self.jaccard_weight\n return loss"
},
{
"path": "src/retinaNN_predict.py",
"content": "###################################################\n#\n# Script to\n# - Calculate prediction of the test dataset\n# - Calculate the parameters to evaluate the prediction\n#\n##################################################\n\n#Python\nimport numpy as np\nimport configparser\nfrom matplotlib import pyplot as plt\nimport torch\nimport os\nimport torch.backends.cudnn as cudnn\nfrom torch.utils.data import DataLoader,Dataset\n\n#scikit learn\nfrom sklearn.metrics import roc_curve\nfrom sklearn.metrics import roc_auc_score\nfrom sklearn.metrics import confusion_matrix\nfrom sklearn.metrics import precision_recall_curve\nfrom sklearn.metrics import jaccard_similarity_score\nfrom sklearn.metrics import f1_score\nimport sys\nsys.path.insert(0, '../')\n# help_functions.py\nfrom lib.help_functions import *\n# extract_patches.py\nfrom lib.extract_patches import recompone\nfrom lib.extract_patches import recompone_overlap\nfrom lib.extract_patches import paint_border\nfrom lib.extract_patches import kill_border\nfrom lib.extract_patches import pred_only_FOV\nfrom lib.extract_patches import get_data_testing\nfrom lib.extract_patches import get_data_testing_overlap\n# pre_processing.py\nfrom lib.pre_processing import my_PreProc\n\n# define pyplot parameters\nimport matplotlib.pylab as pylab\nparams = {'legend.fontsize': 15,\n 'axes.labelsize': 15,\n 'axes.titlesize':15,\n 'xtick.labelsize':15,\n 'ytick.labelsize':15}\npylab.rcParams.update(params)\n\n#========= CONFIG FILE TO READ FROM =======\nconfig = configparser.RawConfigParser()\nconfig.read('../configuration.txt')\n#===========================================\n#run the training on invariant or local\npath_data = config.get('data paths', 'path_local')\n\n#original test images (for FOV selection)\nDRIVE_test_imgs_original = path_data + config.get('data paths', 'test_imgs_original')\ntest_imgs_orig = load_hdf5(DRIVE_test_imgs_original)\nfull_img_height = test_imgs_orig.shape[2]\nfull_img_width = test_imgs_orig.shape[3]\n#the border masks provided by the DRIVE\nDRIVE_test_border_masks = path_data + config.get('data paths', 'test_border_masks')\ntest_border_masks = load_hdf5(DRIVE_test_border_masks)\n# dimension of the patches\npatch_height = int(config.get('data attributes', 'patch_height'))\npatch_width = int(config.get('data attributes', 'patch_width'))\n#the stride in case output with average\nstride_height = int(config.get('testing settings', 'stride_height'))\nstride_width = int(config.get('testing settings', 'stride_width'))\nassert (stride_height < patch_height and stride_width < patch_width)\n#model name\nname_experiment = config.get('experiment name', 'name')\npath_experiment = '../' +name_experiment +'/'\n#N full images to be predicted\nImgs_to_test = int(config.get('testing settings', 'full_images_to_test'))\n#Grouping of the predicted images\nN_visual = int(config.get('testing settings', 'N_group_visual'))\n#====== average mode ===========\naverage_mode = config.getboolean('testing settings', 'average_mode')\n\n\n# #ground truth\n# gtruth= path_data + config.get('data paths', 'test_groundTruth')\n# img_truth= load_hdf5(gtruth)\n# visualize(group_images(test_imgs_orig[0:20,:,:,:],5),'original')#.show()\n# visualize(group_images(test_border_masks[0:20,:,:,:],5),'borders')#.show()\n# visualize(group_images(img_truth[0:20,:,:,:],5),'gtruth')#.show()\n\n\n\n#============ Load the data and divide in patches\npatches_imgs_test = None\nnew_height = None\nnew_width = None\nmasks_test = None\npatches_masks_test = None\nif average_mode == True:\n patches_imgs_test, new_height, new_width, masks_test = get_data_testing_overlap(\n DRIVE_test_imgs_original = DRIVE_test_imgs_original, #original\n DRIVE_test_groudTruth = path_data + config.get('data paths', 'test_groundTruth'), #masks\n Imgs_to_test = int(config.get('testing settings', 'full_images_to_test')),\n patch_height = patch_height,\n patch_width = patch_width,\n stride_height = stride_height,\n stride_width = stride_width\n )\nelse:\n patches_imgs_test, patches_masks_test = get_data_testing(\n DRIVE_test_imgs_original = DRIVE_test_imgs_original, #original\n DRIVE_test_groudTruth = path_data + config.get('data paths', 'test_groundTruth'), #masks\n Imgs_to_test = int(config.get('testing settings', 'full_images_to_test')),\n patch_height = patch_height,\n patch_width = patch_width,\n )\n\n\n\n#================ Run the prediction of the patches ==================================\nbest_last = config.get('testing settings', 'best_last')\n\nlayers= 4\nfilters =10\n\ncheck_path = 'LadderNetv65_layer_%d_filter_%d.pt7'% (layers,filters) #'UNet16.pt7'#'UNet_Resnet101.pt7'\n\nfrom LadderNetv65 import *\n\nnet = LadderNetv6(num_classes=2,layers=layers,filters=filters,inplanes=1)\n\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu'\n\nresume = True\n\nif device == 'cuda':\n net.cuda()\n net = torch.nn.DataParallel(net, device_ids=range(torch.cuda.device_count()))\n cudnn.benchmark = True\n\nif resume:\n # Load checkpoint.\n print('==> Resuming from checkpoint..')\n assert os.path.isdir('checkpoint'), 'Error: no checkpoint directory found!'\n checkpoint = torch.load('./checkpoint/'+check_path)\n net.load_state_dict(checkpoint['net'])\n start_epoch = checkpoint['epoch']\n\nclass TrainDataset(Dataset):\n \"\"\"Endovis 2018 dataset.\"\"\"\n\n def __init__(self, patches_imgs):\n self.imgs = patches_imgs\n\n def __len__(self):\n return self.imgs.shape[0]\n\n def __getitem__(self, idx):\n return torch.from_numpy(self.imgs[idx,...]).float()\n\nbatch_size = 1024\n\ntest_set = TrainDataset(patches_imgs_test)\ntest_loader = DataLoader(test_set, batch_size=batch_size,\n shuffle=False, num_workers=4)\n\npreds = []\nfor batch_idx, inputs in enumerate((test_loader)):\n inputs = inputs.to(device)\n outputs = net(inputs)\n outputs = torch.nn.functional.softmax(outputs,dim=1)\n outputs = outputs.permute(0,2,3,1)\n shape = list(outputs.shape)\n outputs = outputs.view(-1,shape[1]*shape[2],2)\n\n outputs = outputs.data.cpu().numpy()\n preds.append(outputs)\n\npredictions = np.concatenate(preds,axis=0)\nprint(\"Predictions finished\")\n#===== Convert the prediction arrays in corresponding images\npred_patches = pred_to_imgs(predictions, patch_height, patch_width, \"original\")\n\n\n#========== Elaborate and visualize the predicted images ====================\npred_imgs = None\norig_imgs = None\ngtruth_masks = None\nif average_mode == True:\n pred_imgs = recompone_overlap(pred_patches, new_height, new_width, stride_height, stride_width)# predictions\n orig_imgs = my_PreProc(test_imgs_orig[0:pred_imgs.shape[0],:,:,:]) #originals\n gtruth_masks = masks_test #ground truth masks\nelse:\n pred_imgs = recompone(pred_patches,13,12) # predictions\n orig_imgs = recompone(patches_imgs_test,13,12) # originals\n gtruth_masks = recompone(patches_masks_test,13,12) #masks\n# apply the DRIVE masks on the repdictions #set everything outside the FOV to zero!!\nkill_border(pred_imgs, test_border_masks) #DRIVE MASK #only for visualization\n## back to original dimensions\norig_imgs = orig_imgs[:,:,0:full_img_height,0:full_img_width]\npred_imgs = pred_imgs[:,:,0:full_img_height,0:full_img_width]\ngtruth_masks = gtruth_masks[:,:,0:full_img_height,0:full_img_width]\nprint(\"Orig imgs shape: \" +str(orig_imgs.shape))\nprint(\"pred imgs shape: \" +str(pred_imgs.shape))\nprint(\"Gtruth imgs shape: \" +str(gtruth_masks.shape))\nvisualize(group_images(orig_imgs,N_visual),path_experiment+\"all_originals\")#.show()\nvisualize(group_images(pred_imgs,N_visual),path_experiment+\"all_predictions\")#.show()\nvisualize(group_images(gtruth_masks,N_visual),path_experiment+\"all_groundTruths\")#.show()\n#visualize results comparing mask and prediction:\nassert (orig_imgs.shape[0]==pred_imgs.shape[0] and orig_imgs.shape[0]==gtruth_masks.shape[0])\nN_predicted = orig_imgs.shape[0]\ngroup = N_visual\nassert (N_predicted%group==0)\nfor i in range(int(N_predicted/group)):\n orig_stripe = group_images(orig_imgs[i*group:(i*group)+group,:,:,:],group)\n masks_stripe = group_images(gtruth_masks[i*group:(i*group)+group,:,:,:],group)\n pred_stripe = group_images(pred_imgs[i*group:(i*group)+group,:,:,:],group)\n total_img = np.concatenate((orig_stripe,masks_stripe,pred_stripe),axis=0)\n visualize(total_img,path_experiment+name_experiment +\"_Original_GroundTruth_Prediction\"+str(i))#.show()\n\n\n#====== Evaluate the results\nprint(\"\\n\\n======== Evaluate the results =======================\")\n#predictions only inside the FOV\ny_scores, y_true = pred_only_FOV(pred_imgs,gtruth_masks, test_border_masks) #returns data only inside the FOV\nprint(\"Calculating results only inside the FOV:\")\nprint(\"y scores pixels: \" +str(y_scores.shape[0]) +\" (radius 270: 270*270*3.14==228906), including background around retina: \" +str(pred_imgs.shape[0]*pred_imgs.shape[2]*pred_imgs.shape[3]) +\" (584*565==329960)\")\nprint(\"y true pixels: \" +str(y_true.shape[0]) +\" (radius 270: 270*270*3.14==228906), including background around retina: \" +str(gtruth_masks.shape[2]*gtruth_masks.shape[3]*gtruth_masks.shape[0])+\" (584*565==329960)\")\n\n#Area under the ROC curve\nfpr, tpr, thresholds = roc_curve((y_true), y_scores)\nAUC_ROC = roc_auc_score(y_true, y_scores)\n# test_integral = np.trapz(tpr,fpr) #trapz is numpy integration\nprint(\"\\nArea under the ROC curve: \" +str(AUC_ROC))\nroc_curve =plt.figure()\nplt.plot(fpr,tpr,'-',label='Area Under the Curve (AUC = %0.4f)' % AUC_ROC)\nplt.title('ROC curve')\nplt.xlabel(\"FPR (False Positive Rate)\")\nplt.ylabel(\"TPR (True Positive Rate)\")\nplt.legend(loc=\"lower right\")\nplt.savefig(path_experiment+\"ROC.png\")\n\n#Precision-recall curve\nprecision, recall, thresholds = precision_recall_curve(y_true, y_scores)\nprecision = np.fliplr([precision])[0] #so the array is increasing (you won't get negative AUC)\nrecall = np.fliplr([recall])[0] #so the array is increasing (you won't get negative AUC)\nAUC_prec_rec = np.trapz(precision,recall)\nprint(\"\\nArea under Precision-Recall curve: \" +str(AUC_prec_rec))\nprec_rec_curve = plt.figure()\nplt.plot(recall,precision,'-',label='Area Under the Curve (AUC = %0.4f)' % AUC_prec_rec)\nplt.title('Precision - Recall curve')\nplt.xlabel(\"Recall\")\nplt.ylabel(\"Precision\")\nplt.legend(loc=\"lower right\")\nplt.savefig(path_experiment+\"Precision_recall.png\")\n\n#Confusion matrix\nthreshold_confusion = 0.5\nprint(\"\\nConfusion matrix: Costum threshold (for positive) of \" +str(threshold_confusion))\ny_pred = np.empty((y_scores.shape[0]))\nfor i in range(y_scores.shape[0]):\n if y_scores[i]>=threshold_confusion:\n y_pred[i]=1\n else:\n y_pred[i]=0\nconfusion = confusion_matrix(y_true, y_pred)\nprint(confusion)\naccuracy = 0\nif float(np.sum(confusion))!=0:\n accuracy = float(confusion[0,0]+confusion[1,1])/float(np.sum(confusion))\nprint(\"Global Accuracy: \" +str(accuracy))\nspecificity = 0\nif float(confusion[0,0]+confusion[0,1])!=0:\n specificity = float(confusion[0,0])/float(confusion[0,0]+confusion[0,1])\nprint(\"Specificity: \" +str(specificity))\nsensitivity = 0\nif float(confusion[1,1]+confusion[1,0])!=0:\n sensitivity = float(confusion[1,1])/float(confusion[1,1]+confusion[1,0])\nprint(\"Sensitivity: \" +str(sensitivity))\nprecision = 0\nif float(confusion[1,1]+confusion[0,1])!=0:\n precision = float(confusion[1,1])/float(confusion[1,1]+confusion[0,1])\nprint(\"Precision: \" +str(precision))\n\n#Jaccard similarity index\njaccard_index = jaccard_similarity_score(y_true, y_pred, normalize=True)\nprint(\"\\nJaccard similarity score: \" +str(jaccard_index))\n\n#F1 score\nF1_score = f1_score(y_true, y_pred, labels=None, average='binary', sample_weight=None)\nprint(\"\\nF1 score (F-measure): \" +str(F1_score))\n\n#Save the results\nfile_perf = open(path_experiment+'performances.txt', 'w')\nfile_perf.write(\"Area under the ROC curve: \"+str(AUC_ROC)\n + \"\\nArea under Precision-Recall curve: \" +str(AUC_prec_rec)\n + \"\\nJaccard similarity score: \" +str(jaccard_index)\n + \"\\nF1 score (F-measure): \" +str(F1_score)\n +\"\\n\\nConfusion matrix:\"\n +str(confusion)\n +\"\\nACCURACY: \" +str(accuracy)\n +\"\\nSENSITIVITY: \" +str(sensitivity)\n +\"\\nSPECIFICITY: \" +str(specificity)\n +\"\\nPRECISION: \" +str(precision)\n )\nfile_perf.close()\n"
},
{
"path": "src/retinaNN_training.py",
"content": "###################################################\n#\n# Script to:\n# - Load the images and extract the patches\n# - Define the neural network\n# - define the training\n#\n##################################################\n\n\nimport numpy as np\nfrom six.moves import configparser\nimport torch.backends.cudnn as cudnn\n\nimport sys\nsys.path.insert(0, '../')\nfrom lib.help_functions import *\n\n#function to obtain data for training/testing (validation)\nfrom lib.extract_patches import get_data_training\n\nimport os\n\nfrom losses import *\n\nimport torch.optim as optim\n\nfrom torch.utils.data import Dataset, DataLoader\n\nfrom tqdm import tqdm\n\nimport random\n\ndef count_parameters(model):\n return sum(p.numel() for p in model.parameters() if p.requires_grad)\n\n#========= Load settings from Config file\nconfig = configparser.RawConfigParser()\nconfig.read('../configuration.txt')\n\n#patch to the datasets\npath_data = config.get('data paths', 'path_local')\n#Experiment name\nname_experiment = config.get('experiment name', 'name')\n#training settings\nN_epochs = int(config.get('training settings', 'N_epochs'))\nbatch_size = int(config.get('training settings', 'batch_size'))\n\n#========== Define parameters here =============================\n# log file\nif not os.path.exists('./logs'):\n os.mkdir('logs')\n\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu'\n\nstart_epoch = 0 # start from epoch 0 or last checkpoint epoch\ntotal_epoch = 200\n\nval_portion = 0.1\n\nlr_epoch = np.array([150,total_epoch])\nlr_value= np.array([0.001,0.0001])\n\nlayers = 4\nfilters = 10\n\nfrom LadderNetv65 import LadderNetv6\n\nnet = LadderNetv6(num_classes=2,layers=layers,filters=filters,inplanes=1)\nprint(\"Toral number of parameters: \"+str(count_parameters(net)))\n\ncheck_path = 'LadderNetv65_layer_%d_filter_%d.pt7'% (layers,filters) #'UNet16.pt7'#'UNet_Resnet101.pt7'\n\nresume = False\n\ncriterion = LossMulti(jaccard_weight=0)\n#criterion = CrossEntropy2d()\n\n#optimizer = optim.SGD(net.parameters(),\n# lr=lr_schedule[0], momentum=0.9, weight_decay=5e-4, nesterov=True)\n\noptimizer = optim.Adam(net.parameters(),lr=lr_value[0])\n\n#============ Load the data and divided in patches\npatches_imgs_train, patches_masks_train = get_data_training(\n DRIVE_train_imgs_original = path_data + config.get('data paths', 'train_imgs_original'),\n DRIVE_train_groudTruth = path_data + config.get('data paths', 'train_groundTruth'), #masks\n patch_height = int(config.get('data attributes', 'patch_height')),\n patch_width = int(config.get('data attributes', 'patch_width')),\n N_subimgs = int(config.get('training settings', 'N_subimgs')),\n inside_FOV = config.getboolean('training settings', 'inside_FOV') #select the patches only inside the FOV (default == True)\n)\n\nclass TrainDataset(Dataset):\n \"\"\"Endovis 2018 dataset.\"\"\"\n\n def __init__(self, patches_imgs,patches_masks_train):\n self.imgs = patches_imgs\n self.masks = patches_masks_train\n\n def __len__(self):\n return self.imgs.shape[0]\n\n def __getitem__(self, idx):\n tmp = self.masks[idx]\n tmp = np.squeeze(tmp,0)\n return torch.from_numpy(self.imgs[idx,...]).float(), torch.from_numpy(tmp).long()\n\nval_ind = random.sample(range(patches_masks_train.shape[0]),int(np.floor(val_portion*patches_masks_train.shape[0])))\n\ntrain_ind = set(range(patches_masks_train.shape[0])) - set(val_ind)\ntrain_ind = list(train_ind)\n\ntrain_set = TrainDataset(patches_imgs_train[train_ind,...],patches_masks_train[train_ind,...])\ntrain_loader = DataLoader(train_set, batch_size=batch_size,\n shuffle=True, num_workers=4)\n\nval_set = TrainDataset(patches_imgs_train[val_ind,...],patches_masks_train[val_ind,...])\nval_loader = DataLoader(val_set, batch_size=batch_size,\n shuffle=True, num_workers=4)\n\n#========= Save a sample of what you're feeding to the neural network ==========\nN_sample = min(patches_imgs_train.shape[0],40)\nvisualize(group_images(patches_imgs_train[0:N_sample,:,:,:],5),'../'+name_experiment+'/'+\"sample_input_imgs\")#.show()\nvisualize(group_images(patches_masks_train[0:N_sample,:,:,:],5),'../'+name_experiment+'/'+\"sample_input_masks\")#.show()\n\nbest_loss = np.Inf\n\n# create a list of learning rate with epochs\nlr_schedule = np.zeros(total_epoch)\nfor l in range(len(lr_epoch)):\n if l ==0:\n lr_schedule[0:lr_epoch[l]] = lr_value[l]\n else:\n lr_schedule[lr_epoch[l-1]:lr_epoch[l]] = lr_value[l]\n\nif device == 'cuda':\n net.cuda()\n net = torch.nn.DataParallel(net, device_ids=range(torch.cuda.device_count()))\n cudnn.benchmark = True\nif resume:\n # Load checkpoint.\n print('==> Resuming from checkpoint..')\n assert os.path.isdir('checkpoint'), 'Error: no checkpoint directory found!'\n checkpoint = torch.load('./checkpoint/'+check_path)\n net.load_state_dict(checkpoint['net'])\n start_epoch = checkpoint['epoch']\n\ndef train(epoch):\n print('\\nEpoch: %d' % epoch)\n net.train()\n train_loss = 0\n IoU = []\n\n # get learning rate from learing schedule\n lr = lr_schedule[epoch]\n for param_group in optimizer.param_groups:\n param_group['lr'] = lr\n\n print(\"Learning rate = %4f\\n\" % lr)\n\n IU = []\n # train network\n for batch_idx, (inputs, targets) in enumerate(tqdm(train_loader)):\n inputs, targets = inputs.to(device), targets.to(device)\n optimizer.zero_grad()\n\n outputs = net(inputs)\n loss = criterion(outputs, targets)\n loss.backward()\n optimizer.step()\n\n train_loss += loss.item()\n\n print(\"Epoch %d: Train loss %4f\\n\" % (epoch, train_loss / np.float32(len(train_loader))))\n\ndef test(epoch, display=False):\n global best_loss\n net.eval()\n test_loss = 0\n with torch.no_grad():\n\n for batch_idx, (inputs, targets) in enumerate(val_loader):\n inputs, targets = inputs.to(device), targets.to(device)\n outputs = net(inputs)\n loss = criterion(outputs, targets)\n\n test_loss += loss.item()\n\n print(\n 'Valid loss: {:.4f}'.format(test_loss))\n # Save checkpoint.\n if test_loss < best_loss:\n print('Saving..')\n state = {\n 'net': net.state_dict(),\n 'best_loss': best_loss,\n 'epoch': epoch,\n }\n if not os.path.isdir('checkpoint'):\n os.mkdir('checkpoint')\n torch.save(state, './checkpoint/' + check_path)\n best_loss = test_loss\n\nfor epoch in range(start_epoch,total_epoch):\n train(epoch)\n test(epoch,False)\n"
},
{
"path": "src/subpixel_upscaling.py",
"content": "# -*- coding: utf-8 -*-\n\"\"\"\nThis file contains an implementation of Sub-pixel convolutional upscaling layer based on\nthe paper \"Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel\nConvolutional Neural Network\" (https://arxiv.org/abs/1609.05158).\n\"\"\"\n\nfrom __future__ import absolute_import\n\nimport tensorflow as tf\nfrom keras.engine import Layer\nfrom keras.utils.generic_utils import get_custom_objects\nfrom keras.utils.conv_utils import normalize_data_format\n\nclass SubPixelUpscaling(Layer):\n \"\"\" This layer requires a Convolution2D prior to it, having output filters computed according to\n the formula :\n filters = k * (scale_factor * scale_factor)\n where k = a user defined number of filters (generally larger than 32)\n scale_factor = the upscaling factor (generally 2)\n This layer performs the depth to space operation on the convolution filters, and returns a\n tensor with the size as defined below.\n # Example :\n ```python\n # A standard subpixel upscaling block\n x = Convolution2D(256, 3, 3, padding='same', activation='relu')(...)\n u = SubPixelUpscaling(scale_factor=2)(x)\n [Optional]\n x = Convolution2D(256, 3, 3, padding='same', activation='relu')(u)\n ```\n In practice, it is useful to have a second convolution layer after the\n SubPixelUpscaling layer to speed up the learning process.\n However, if you are stacking multiple SubPixelUpscaling blocks, it may increase\n the number of parameters greatly, so the Convolution layer after SubPixelUpscaling\n layer can be removed.\n # Arguments\n scale_factor: Upscaling factor.\n data_format: Can be None, 'channels_first' or 'channels_last'.\n # Input shape\n 4D tensor with shape:\n `(samples, k * (scale_factor * scale_factor) channels, rows, cols)` if data_format='channels_first'\n or 4D tensor with shape:\n `(samples, rows, cols, k * (scale_factor * scale_factor) channels)` if data_format='channels_last'.\n # Output shape\n 4D tensor with shape:\n `(samples, k channels, rows * scale_factor, cols * scale_factor))` if data_format='channels_first'\n or 4D tensor with shape:\n `(samples, rows * scale_factor, cols * scale_factor, k channels)` if data_format='channels_last'.\n \"\"\"\n\n def __init__(self, scale_factor=2, data_format=None, **kwargs):\n super(SubPixelUpscaling, self).__init__(**kwargs)\n\n self.scale_factor = scale_factor\n self.data_format = normalize_data_format(data_format)\n\n def build(self, input_shape):\n pass\n\n def call(self, x, mask=None):\n y = tf.depth_to_space(x, self.scale_factor, self.data_format)\n return y\n\n def compute_output_shape(self, input_shape):\n if self.data_format == 'channels_first':\n b, k, r, c = input_shape\n return (b, k // (self.scale_factor ** 2), r * self.scale_factor, c * self.scale_factor)\n else:\n b, r, c, k = input_shape\n return (b, r * self.scale_factor, c * self.scale_factor, k // (self.scale_factor ** 2))\n\n def get_config(self):\n config = {'scale_factor': self.scale_factor,\n 'data_format': self.data_format}\n base_config = super(SubPixelUpscaling, self).get_config()\n return dict(list(base_config.items()) + list(config.items()))\n\n\nget_custom_objects().update({'SubPixelUpscaling': SubPixelUpscaling})"
},
{
"path": "test/drive_performances.txt",
"content": "Area under the ROC curve: 0.9793486169167707\nArea under Precision-Recall curve: 0.9107415769226219\nJaccard similarity score: 0.9561483628876393\nF1 score (F-measure): 0.8201609835049293\n\nConfusion matrix:[[3885354 75140]\n [ 123865 453784]]\nACCURACY: 0.9561483628876393\nSENSITIVITY: 0.7855704761888275\nSPECIFICITY: 0.981027619281837\nPRECISION: 0.8579380024351324"
},
{
"path": "test/performances.txt",
"content": "Area under the ROC curve: 0.9793257409827817\nArea under Precision-Recall curve: 0.9107420271043044\nJaccard similarity score: 0.9561715001047786\nF1 score (F-measure): 0.8202472616852836\n\nConfusion matrix:[[3885433 75061]\n [ 123839 453810]]\nACCURACY: 0.9561715001047786\nSENSITIVITY: 0.7856154862208712\nSPECIFICITY: 0.9810475662884478\nPRECISION: 0.8580731407091711"
},
{
"path": "test/test_architecture.json",
"content": "{\"class_name\": \"Model\", \"config\": {\"name\": \"model_1\", \"layers\": [{\"name\": \"input_1\", \"class_name\": \"InputLayer\", \"config\": {\"batch_input_shape\": [null, 48, 48, 1], \"dtype\": \"float32\", \"sparse\": false, \"name\": \"input_1\"}, \"inbound_nodes\": []}, {\"name\": \"conv2d_1\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_1\", \"trainable\": true, \"filters\": 10, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"input_1\", 0, 0, {}]]]}, {\"name\": \"conv2d_2\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_2\", \"trainable\": true, \"filters\": 10, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_1\", 0, 0, {}]], [[\"dropout_1\", 0, 0, {}]], [[\"add_1\", 0, 0, {}]]]}, {\"name\": \"dropout_1\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_1\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_2\", 0, 0, {}]]]}, {\"name\": \"dropout_2\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_2\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_2\", 1, 0, {}]]]}, {\"name\": \"add_1\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_1\", \"trainable\": true}, \"inbound_nodes\": [[[\"conv2d_1\", 0, 0, {}], [\"dropout_2\", 0, 0, {}]]]}, {\"name\": \"conv2d_3\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_3\", \"trainable\": true, \"filters\": 20, \"kernel_size\": [3, 3], \"strides\": [2, 2], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_2\", 2, 0, {}]]]}, {\"name\": \"conv2d_4\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_4\", \"trainable\": true, \"filters\": 20, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_3\", 0, 0, {}]], [[\"dropout_3\", 0, 0, {}]], [[\"add_2\", 0, 0, {}]]]}, {\"name\": \"dropout_3\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_3\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_4\", 0, 0, {}]]]}, {\"name\": \"dropout_4\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_4\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_4\", 1, 0, {}]]]}, {\"name\": \"add_2\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_2\", \"trainable\": true}, \"inbound_nodes\": [[[\"conv2d_3\", 0, 0, {}], [\"dropout_4\", 0, 0, {}]]]}, {\"name\": \"conv2d_5\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_5\", \"trainable\": true, \"filters\": 40, \"kernel_size\": [3, 3], \"strides\": [2, 2], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_4\", 2, 0, {}]]]}, {\"name\": \"conv2d_6\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_6\", \"trainable\": true, \"filters\": 40, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_5\", 0, 0, {}]], [[\"dropout_5\", 0, 0, {}]], [[\"add_3\", 0, 0, {}]]]}, {\"name\": \"dropout_5\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_5\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_6\", 0, 0, {}]]]}, {\"name\": \"dropout_6\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_6\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_6\", 1, 0, {}]]]}, {\"name\": \"add_3\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_3\", \"trainable\": true}, \"inbound_nodes\": [[[\"conv2d_5\", 0, 0, {}], [\"dropout_6\", 0, 0, {}]]]}, {\"name\": \"conv2d_7\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_7\", \"trainable\": true, \"filters\": 80, \"kernel_size\": [3, 3], \"strides\": [2, 2], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_6\", 2, 0, {}]]]}, {\"name\": \"conv2d_8\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_8\", \"trainable\": true, \"filters\": 80, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_7\", 0, 0, {}]], [[\"dropout_7\", 0, 0, {}]], [[\"add_4\", 0, 0, {}]]]}, {\"name\": \"dropout_7\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_7\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_8\", 0, 0, {}]]]}, {\"name\": \"dropout_8\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_8\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_8\", 1, 0, {}]]]}, {\"name\": \"add_4\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_4\", \"trainable\": true}, \"inbound_nodes\": [[[\"conv2d_7\", 0, 0, {}], [\"dropout_8\", 0, 0, {}]]]}, {\"name\": \"conv2d_9\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_9\", \"trainable\": true, \"filters\": 160, \"kernel_size\": [3, 3], \"strides\": [2, 2], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_8\", 2, 0, {}]]]}, {\"name\": \"conv2d_10\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_10\", \"trainable\": true, \"filters\": 160, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_9\", 0, 0, {}]], [[\"dropout_9\", 0, 0, {}]], [[\"add_5\", 0, 0, {}]]]}, {\"name\": \"dropout_9\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_9\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_10\", 0, 0, {}]]]}, {\"name\": \"dropout_10\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_10\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_10\", 1, 0, {}]]]}, {\"name\": \"add_5\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_5\", \"trainable\": true}, \"inbound_nodes\": [[[\"conv2d_9\", 0, 0, {}], [\"dropout_10\", 0, 0, {}]]]}, {\"name\": \"conv2d_transpose_1\", \"class_name\": \"Conv2DTranspose\", \"config\": {\"name\": \"conv2d_transpose_1\", \"trainable\": true, \"filters\": 80, \"kernel_size\": [3, 3], \"strides\": [2, 2], \"padding\": \"same\", \"data_format\": \"channels_last\", \"activation\": \"linear\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_10\", 2, 0, {}]]]}, {\"name\": \"add_6\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_6\", \"trainable\": true}, \"inbound_nodes\": [[[\"conv2d_transpose_1\", 0, 0, {}], [\"conv2d_8\", 2, 0, {}]]]}, {\"name\": \"conv2d_11\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_11\", \"trainable\": true, \"filters\": 80, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"add_6\", 0, 0, {}]], [[\"dropout_11\", 0, 0, {}]], [[\"add_7\", 0, 0, {}]]]}, {\"name\": \"dropout_11\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_11\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_11\", 0, 0, {}]]]}, {\"name\": \"dropout_12\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_12\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_11\", 1, 0, {}]]]}, {\"name\": \"add_7\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_7\", \"trainable\": true}, \"inbound_nodes\": [[[\"add_6\", 0, 0, {}], [\"dropout_12\", 0, 0, {}]]]}, {\"name\": \"conv2d_transpose_2\", \"class_name\": \"Conv2DTranspose\", \"config\": {\"name\": \"conv2d_transpose_2\", \"trainable\": true, \"filters\": 40, \"kernel_size\": [3, 3], \"strides\": [2, 2], \"padding\": \"same\", \"data_format\": \"channels_last\", \"activation\": \"linear\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_11\", 2, 0, {}]]]}, {\"name\": \"add_8\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_8\", \"trainable\": true}, \"inbound_nodes\": [[[\"conv2d_transpose_2\", 0, 0, {}], [\"conv2d_6\", 2, 0, {}]]]}, {\"name\": \"conv2d_12\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_12\", \"trainable\": true, \"filters\": 40, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"add_8\", 0, 0, {}]], [[\"dropout_13\", 0, 0, {}]], [[\"add_9\", 0, 0, {}]]]}, {\"name\": \"dropout_13\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_13\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_12\", 0, 0, {}]]]}, {\"name\": \"dropout_14\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_14\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_12\", 1, 0, {}]]]}, {\"name\": \"add_9\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_9\", \"trainable\": true}, \"inbound_nodes\": [[[\"add_8\", 0, 0, {}], [\"dropout_14\", 0, 0, {}]]]}, {\"name\": \"conv2d_transpose_3\", \"class_name\": \"Conv2DTranspose\", \"config\": {\"name\": \"conv2d_transpose_3\", \"trainable\": true, \"filters\": 20, \"kernel_size\": [3, 3], \"strides\": [2, 2], \"padding\": \"same\", \"data_format\": \"channels_last\", \"activation\": \"linear\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_12\", 2, 0, {}]]]}, {\"name\": \"add_10\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_10\", \"trainable\": true}, \"inbound_nodes\": [[[\"conv2d_transpose_3\", 0, 0, {}], [\"conv2d_4\", 2, 0, {}]]]}, {\"name\": \"conv2d_13\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_13\", \"trainable\": true, \"filters\": 20, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"add_10\", 0, 0, {}]], [[\"dropout_15\", 0, 0, {}]], [[\"add_11\", 0, 0, {}]]]}, {\"name\": \"dropout_15\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_15\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_13\", 0, 0, {}]]]}, {\"name\": \"dropout_16\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_16\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_13\", 1, 0, {}]]]}, {\"name\": \"add_11\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_11\", \"trainable\": true}, \"inbound_nodes\": [[[\"add_10\", 0, 0, {}], [\"dropout_16\", 0, 0, {}]]]}, {\"name\": \"conv2d_transpose_4\", \"class_name\": \"Conv2DTranspose\", \"config\": {\"name\": \"conv2d_transpose_4\", \"trainable\": true, \"filters\": 10, \"kernel_size\": [3, 3], \"strides\": [2, 2], \"padding\": \"same\", \"data_format\": \"channels_last\", \"activation\": \"linear\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_13\", 2, 0, {}]]]}, {\"name\": \"add_12\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_12\", \"trainable\": true}, \"inbound_nodes\": [[[\"conv2d_transpose_4\", 0, 0, {}], [\"conv2d_2\", 2, 0, {}]]]}, {\"name\": \"conv2d_14\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_14\", \"trainable\": true, \"filters\": 10, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"add_12\", 0, 0, {}]], [[\"dropout_17\", 0, 0, {}]], [[\"add_13\", 0, 0, {}]]]}, {\"name\": \"dropout_17\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_17\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_14\", 0, 0, {}]]]}, {\"name\": \"dropout_18\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_18\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_14\", 1, 0, {}]]]}, {\"name\": \"add_13\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_13\", \"trainable\": true}, \"inbound_nodes\": [[[\"add_12\", 0, 0, {}], [\"dropout_18\", 0, 0, {}]]]}, {\"name\": \"conv2d_15\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_15\", \"trainable\": true, \"filters\": 10, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_14\", 2, 0, {}]]]}, {\"name\": \"add_14\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_14\", \"trainable\": true}, \"inbound_nodes\": [[[\"conv2d_15\", 0, 0, {}], [\"conv2d_14\", 2, 0, {}]]]}, {\"name\": \"conv2d_16\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_16\", \"trainable\": true, \"filters\": 10, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"add_14\", 0, 0, {}]], [[\"dropout_19\", 0, 0, {}]], [[\"add_15\", 0, 0, {}]]]}, {\"name\": \"dropout_19\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_19\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_16\", 0, 0, {}]]]}, {\"name\": \"dropout_20\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_20\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_16\", 1, 0, {}]]]}, {\"name\": \"add_15\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_15\", \"trainable\": true}, \"inbound_nodes\": [[[\"add_14\", 0, 0, {}], [\"dropout_20\", 0, 0, {}]]]}, {\"name\": \"conv2d_17\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_17\", \"trainable\": true, \"filters\": 20, \"kernel_size\": [3, 3], \"strides\": [2, 2], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_16\", 2, 0, {}]]]}, {\"name\": \"add_16\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_16\", \"trainable\": true}, \"inbound_nodes\": [[[\"conv2d_17\", 0, 0, {}], [\"conv2d_13\", 2, 0, {}]]]}, {\"name\": \"conv2d_18\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_18\", \"trainable\": true, \"filters\": 20, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"add_16\", 0, 0, {}]], [[\"dropout_21\", 0, 0, {}]], [[\"add_17\", 0, 0, {}]]]}, {\"name\": \"dropout_21\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_21\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_18\", 0, 0, {}]]]}, {\"name\": \"dropout_22\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_22\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_18\", 1, 0, {}]]]}, {\"name\": \"add_17\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_17\", \"trainable\": true}, \"inbound_nodes\": [[[\"add_16\", 0, 0, {}], [\"dropout_22\", 0, 0, {}]]]}, {\"name\": \"conv2d_19\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_19\", \"trainable\": true, \"filters\": 40, \"kernel_size\": [3, 3], \"strides\": [2, 2], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_18\", 2, 0, {}]]]}, {\"name\": \"add_18\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_18\", \"trainable\": true}, \"inbound_nodes\": [[[\"conv2d_19\", 0, 0, {}], [\"conv2d_12\", 2, 0, {}]]]}, {\"name\": \"conv2d_20\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_20\", \"trainable\": true, \"filters\": 40, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"add_18\", 0, 0, {}]], [[\"dropout_23\", 0, 0, {}]], [[\"add_19\", 0, 0, {}]]]}, {\"name\": \"dropout_23\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_23\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_20\", 0, 0, {}]]]}, {\"name\": \"dropout_24\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_24\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_20\", 1, 0, {}]]]}, {\"name\": \"add_19\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_19\", \"trainable\": true}, \"inbound_nodes\": [[[\"add_18\", 0, 0, {}], [\"dropout_24\", 0, 0, {}]]]}, {\"name\": \"conv2d_21\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_21\", \"trainable\": true, \"filters\": 80, \"kernel_size\": [3, 3], \"strides\": [2, 2], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_20\", 2, 0, {}]]]}, {\"name\": \"add_20\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_20\", \"trainable\": true}, \"inbound_nodes\": [[[\"conv2d_21\", 0, 0, {}], [\"conv2d_11\", 2, 0, {}]]]}, {\"name\": \"conv2d_22\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_22\", \"trainable\": true, \"filters\": 80, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"add_20\", 0, 0, {}]], [[\"dropout_25\", 0, 0, {}]], [[\"add_21\", 0, 0, {}]]]}, {\"name\": \"dropout_25\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_25\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_22\", 0, 0, {}]]]}, {\"name\": \"dropout_26\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_26\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_22\", 1, 0, {}]]]}, {\"name\": \"add_21\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_21\", \"trainable\": true}, \"inbound_nodes\": [[[\"add_20\", 0, 0, {}], [\"dropout_26\", 0, 0, {}]]]}, {\"name\": \"conv2d_23\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_23\", \"trainable\": true, \"filters\": 160, \"kernel_size\": [3, 3], \"strides\": [2, 2], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_22\", 2, 0, {}]]]}, {\"name\": \"conv2d_24\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_24\", \"trainable\": true, \"filters\": 160, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_23\", 0, 0, {}]], [[\"dropout_27\", 0, 0, {}]], [[\"add_22\", 0, 0, {}]]]}, {\"name\": \"dropout_27\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_27\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_24\", 0, 0, {}]]]}, {\"name\": \"dropout_28\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_28\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_24\", 1, 0, {}]]]}, {\"name\": \"add_22\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_22\", \"trainable\": true}, \"inbound_nodes\": [[[\"conv2d_23\", 0, 0, {}], [\"dropout_28\", 0, 0, {}]]]}, {\"name\": \"conv2d_transpose_5\", \"class_name\": \"Conv2DTranspose\", \"config\": {\"name\": \"conv2d_transpose_5\", \"trainable\": true, \"filters\": 80, \"kernel_size\": [3, 3], \"strides\": [2, 2], \"padding\": \"same\", \"data_format\": \"channels_last\", \"activation\": \"linear\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_24\", 2, 0, {}]]]}, {\"name\": \"add_23\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_23\", \"trainable\": true}, \"inbound_nodes\": [[[\"conv2d_transpose_5\", 0, 0, {}], [\"conv2d_22\", 2, 0, {}]]]}, {\"name\": \"conv2d_25\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_25\", \"trainable\": true, \"filters\": 80, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"add_23\", 0, 0, {}]], [[\"dropout_29\", 0, 0, {}]], [[\"add_24\", 0, 0, {}]]]}, {\"name\": \"dropout_29\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_29\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_25\", 0, 0, {}]]]}, {\"name\": \"dropout_30\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_30\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_25\", 1, 0, {}]]]}, {\"name\": \"add_24\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_24\", \"trainable\": true}, \"inbound_nodes\": [[[\"add_23\", 0, 0, {}], [\"dropout_30\", 0, 0, {}]]]}, {\"name\": \"conv2d_transpose_6\", \"class_name\": \"Conv2DTranspose\", \"config\": {\"name\": \"conv2d_transpose_6\", \"trainable\": true, \"filters\": 40, \"kernel_size\": [3, 3], \"strides\": [2, 2], \"padding\": \"same\", \"data_format\": \"channels_last\", \"activation\": \"linear\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_25\", 2, 0, {}]]]}, {\"name\": \"add_25\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_25\", \"trainable\": true}, \"inbound_nodes\": [[[\"conv2d_transpose_6\", 0, 0, {}], [\"conv2d_20\", 2, 0, {}]]]}, {\"name\": \"conv2d_26\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_26\", \"trainable\": true, \"filters\": 40, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"add_25\", 0, 0, {}]], [[\"dropout_31\", 0, 0, {}]], [[\"add_26\", 0, 0, {}]]]}, {\"name\": \"dropout_31\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_31\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_26\", 0, 0, {}]]]}, {\"name\": \"dropout_32\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_32\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_26\", 1, 0, {}]]]}, {\"name\": \"add_26\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_26\", \"trainable\": true}, \"inbound_nodes\": [[[\"add_25\", 0, 0, {}], [\"dropout_32\", 0, 0, {}]]]}, {\"name\": \"conv2d_transpose_7\", \"class_name\": \"Conv2DTranspose\", \"config\": {\"name\": \"conv2d_transpose_7\", \"trainable\": true, \"filters\": 20, \"kernel_size\": [3, 3], \"strides\": [2, 2], \"padding\": \"same\", \"data_format\": \"channels_last\", \"activation\": \"linear\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_26\", 2, 0, {}]]]}, {\"name\": \"add_27\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_27\", \"trainable\": true}, \"inbound_nodes\": [[[\"conv2d_transpose_7\", 0, 0, {}], [\"conv2d_18\", 2, 0, {}]]]}, {\"name\": \"conv2d_27\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_27\", \"trainable\": true, \"filters\": 20, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"add_27\", 0, 0, {}]], [[\"dropout_33\", 0, 0, {}]], [[\"add_28\", 0, 0, {}]]]}, {\"name\": \"dropout_33\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_33\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_27\", 0, 0, {}]]]}, {\"name\": \"dropout_34\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_34\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_27\", 1, 0, {}]]]}, {\"name\": \"add_28\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_28\", \"trainable\": true}, \"inbound_nodes\": [[[\"add_27\", 0, 0, {}], [\"dropout_34\", 0, 0, {}]]]}, {\"name\": \"conv2d_transpose_8\", \"class_name\": \"Conv2DTranspose\", \"config\": {\"name\": \"conv2d_transpose_8\", \"trainable\": true, \"filters\": 10, \"kernel_size\": [3, 3], \"strides\": [2, 2], \"padding\": \"same\", \"data_format\": \"channels_last\", \"activation\": \"linear\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_27\", 2, 0, {}]]]}, {\"name\": \"add_29\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_29\", \"trainable\": true}, \"inbound_nodes\": [[[\"conv2d_transpose_8\", 0, 0, {}], [\"conv2d_16\", 2, 0, {}]]]}, {\"name\": \"conv2d_28\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_28\", \"trainable\": true, \"filters\": 10, \"kernel_size\": [3, 3], \"strides\": [1, 1], \"padding\": \"same\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"add_29\", 0, 0, {}]], [[\"dropout_35\", 0, 0, {}]], [[\"add_30\", 0, 0, {}]]]}, {\"name\": \"dropout_35\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_35\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_28\", 0, 0, {}]]]}, {\"name\": \"dropout_36\", \"class_name\": \"Dropout\", \"config\": {\"name\": \"dropout_36\", \"trainable\": true, \"rate\": 0.25, \"noise_shape\": null, \"seed\": null}, \"inbound_nodes\": [[[\"conv2d_28\", 1, 0, {}]]]}, {\"name\": \"add_30\", \"class_name\": \"Add\", \"config\": {\"name\": \"add_30\", \"trainable\": true}, \"inbound_nodes\": [[[\"add_29\", 0, 0, {}], [\"dropout_36\", 0, 0, {}]]]}, {\"name\": \"conv2d_29\", \"class_name\": \"Conv2D\", \"config\": {\"name\": \"conv2d_29\", \"trainable\": true, \"filters\": 2, \"kernel_size\": [1, 1], \"strides\": [1, 1], \"padding\": \"valid\", \"data_format\": \"channels_last\", \"dilation_rate\": [1, 1], \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"VarianceScaling\", \"config\": {\"scale\": 1.0, \"mode\": \"fan_avg\", \"distribution\": \"uniform\", \"seed\": null}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}, \"inbound_nodes\": [[[\"conv2d_28\", 2, 0, {}]]]}, {\"name\": \"reshape_1\", \"class_name\": \"Reshape\", \"config\": {\"name\": \"reshape_1\", \"trainable\": true, \"target_shape\": [2304, 2]}, \"inbound_nodes\": [[[\"conv2d_29\", 0, 0, {}]]]}, {\"name\": \"activation_1\", \"class_name\": \"Activation\", \"config\": {\"name\": \"activation_1\", \"trainable\": true, \"activation\": \"softmax\"}, \"inbound_nodes\": [[[\"reshape_1\", 0, 0, {}]]]}], \"input_layers\": [[\"input_1\", 0, 0]], \"output_layers\": [[\"activation_1\", 0, 0]]}, \"keras_version\": \"2.1.5\", \"backend\": \"tensorflow\"}"
},
{
"path": "test/test_configuration.txt",
"content": "[data paths]\npath_local = ../DRIVE_datasets_training_testing/\ntrain_imgs_original = DRIVE_dataset_imgs_train.hdf5\ntrain_groundTruth = DRIVE_dataset_groundTruth_train.hdf5\ntrain_border_masks = DRIVE_dataset_borderMasks_train.hdf5\ntest_imgs_original = DRIVE_dataset_imgs_test.hdf5\ntest_groundTruth = DRIVE_dataset_groundTruth_test.hdf5\ntest_border_masks = DRIVE_dataset_borderMasks_test.hdf5\n\n\n\n[experiment name]\nname = test\n\n\n[data attributes]\n#Dimensions of the patches extracted from the full images\npatch_height = 48\npatch_width = 48\n\n\n[training settings]\n#number of total patches:\nN_subimgs = 190000\n#if patches are extracted only inside the field of view:\ninside_FOV = False\n#Number of training epochs\nN_epochs = 150\nbatch_size = 32\n#if running with nohup\nnohup = True\n\n\n[testing settings]\n#Choose the model to test: best==epoch with min loss, last==last epoch\nbest_last = best\n#number of full images for the test (max 20)\nfull_images_to_test = 20\n#How many original-groundTruth-prediction images are visualized in each image\nN_group_visual = 1\n#Compute average in the prediction, improve results but require more patches to be predicted\naverage_mode = True\n#Only if average_mode==True. Stride for patch extraction, lower value require more patches to be predicted\nstride_height = 5\nstride_width = 5\n#if running with nohup\nnohup = False\n"
},
{
"path": "test/test_training.nohup",
"content": ""
}
]