[
  {
    "path": "README.md",
    "content": "# YOLOv2 Object Detection w/ Keras (in just 20 lines of code)\n\nThis repository presents a quick and simple implementation of YOLOv2 object detection using Keras library with Tensorflow backend.\nCredits goes to [YAD2K Library](https://github.com/allanzelener/YAD2K) on top of which this implementation was built. \n\n![cover01](etc/cover01.jpg)\n![cover02](etc/cover02.jpg)\n###### Note that I do not hold ownership to any of the above pictures. These are merely used for educational purposes to describe the concepts. \n--------------------------------------------------------------------------------\n## Thoughts on the implementation\n\nYOLO is well known technique used to perform fast multiple localizations on a single image.  \n\nA brief algorithm breakdown;\n- Divide the image using a grid (eg: 19x19)\n- Perform image classification and Localization on each grid cell -> Result, a vector for each cell representing the probability of an object detected, the dimensions of the bounding box and class of the detected image.\n- Perform thresholding to remove multiple detected instances \n- Perform Non-max suppression to refine the boxes more\n- Additionally anchor boxes are used to detect several objects in one grid cell\n\n###### If you want to dive down into how these above points are implemented in the code refer yolo_eval function in the keras_yolo.py file from the yad2k/models directory.\n\nPaper reference: [YOLO9000: Better, Faster, Stronger](https://arxiv.org/abs/1612.08242) by Joseph Redmond and Ali Farhadi.\n\n[Keras](https://keras.io/) is a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano. It was developed with a focus on enabling fast experimentation. Being able to go from idea to result with the least possible delay is key to doing good research.\n\nThe use of keras helps to understand the concepts underling a ML technique by reducing the workload on coding. Thus, this implementation becomes a good platform for beginners to core concepts and have a quick implementation giving results. \n\nAlso for anyone who is looking to integrate object detection capabilities in applications, this code can be incorperated with a few lines of code. \n\nBlog post on this: https://medium.com/@miranthaj/quick-implementation-of-yolo-v2-with-keras-ebf6eb40c684\n\n--------------------------------------------------------------------------------\n\n## Quick Start\n\n- Clone this repository to your PC\n- Download any Darknet model cfg and weights from the [official YOLO website](http://pjreddie.com/darknet/yolo/). \n- Convert the dowloaded cfg and weights files into a h5 file using YAD2K library. (This is explained step by step below in the more details section)\n- Copy the generated h5 file to the model_data folder and edit the name of the pretrained model in yolo.py code to the name of your h5 file.\n- Place the input image you want to try object detection in the images folder and copy its file name.\n- Assign your input image file name to input_image_name variable in yolo.py.\n- Open terminal from the repository directory directly and run the yolo.py file\n\t\n\t`python yolo.py`\n\n--------------------------------------------------------------------------------\n\n## More Details\n\nHow to convert cfg and weights files to h5 using YAD2k library (Windows)\n\n- Clone the [YAD2K Library](https://github.com/allanzelener/YAD2K) to your PC\n- Open terminal from the cloned directory\n- Copy and paste the downloaded weights and cfg files to the YAD2K master directory\n- Run `python yad2k.py yolo.cfg yolo.weights model_data/yolo.h5` on the terminal and the h5 file will be generated.\n- Move the generated h5 file to model_data folder of the simpleYOLOwKeras directory\n\n\n\n-------------------------------------------------------------------------------\n"
  },
  {
    "path": "font/SIL+Open+Font+License.txt",
    "content": "Copyright (c) 2014, Mozilla Foundation https://mozilla.org/ with Reserved Font Name Fira Mono.\n\nCopyright (c) 2014, Telefonica S.A.\n\nThis Font Software is licensed under the SIL Open Font License, Version 1.1.\nThis license is copied below, and is also available with a FAQ at: http://scripts.sil.org/OFL\n\n-----------------------------------------------------------\nSIL OPEN FONT LICENSE Version 1.1 - 26 February 2007\n-----------------------------------------------------------\n\nPREAMBLE\nThe goals of the Open Font License (OFL) are to stimulate worldwide development of collaborative font projects, to support the font creation efforts of academic and linguistic communities, and to provide a free and open framework in which fonts may be shared and improved in partnership with others.\n\nThe OFL allows the licensed fonts to be used, studied, modified and redistributed freely as long as they are not sold by themselves. The fonts, including any derivative works, can be bundled, embedded, redistributed and/or sold with any software provided that any reserved names are not used by derivative works. The fonts and derivatives, however, cannot be released under any other type of license. The requirement for fonts to remain under this license does not apply to any document created using the fonts or their derivatives.\n\nDEFINITIONS\n\"Font Software\" refers to the set of files released by the Copyright Holder(s) under this license and clearly marked as such. This may include source files, build scripts and documentation.\n\n\"Reserved Font Name\" refers to any names specified as such after the copyright statement(s).\n\n\"Original Version\" refers to the collection of Font Software components as distributed by the Copyright Holder(s).\n\n\"Modified Version\" refers to any derivative made by adding to, deleting, or substituting -- in part or in whole -- any of the components of the Original Version, by changing formats or by porting the Font Software to a new environment.\n\n\"Author\" refers to any designer, engineer, programmer, technical writer or other person who contributed to the Font Software.\n\nPERMISSION & CONDITIONS\nPermission is hereby granted, free of charge, to any person obtaining a copy of the Font Software, to use, study, copy, merge, embed, modify, redistribute, and sell modified and unmodified copies of the Font Software, subject to the following conditions:\n\n1) Neither the Font Software nor any of its individual components, in Original or Modified Versions, may be sold by itself.\n\n2) Original or Modified Versions of the Font Software may be bundled, redistributed and/or sold with any software, provided that each copy contains the above copyright notice and this license. These can be included either as stand-alone text files, human-readable headers or in the appropriate machine-readable metadata fields within text or binary files as long as those fields can be easily viewed by the user.\n\n3) No Modified Version of the Font Software may use the Reserved Font Name(s) unless explicit written permission is granted by the corresponding Copyright Holder. This restriction only applies to the primary font name as presented to the users.\n\n4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font Software shall not be used to promote, endorse or advertise any Modified Version, except to acknowledge the contribution(s) of the Copyright Holder(s) and the Author(s) or with their explicit written permission.\n\n5) The Font Software, modified or unmodified, in part or in whole, must be distributed entirely under this license, and must not be distributed under any other license. The requirement for fonts to remain under this license does not apply to any document created using the Font Software.\n\nTERMINATION\nThis license becomes null and void if any of the above conditions are not met.\n\nDISCLAIMER\nTHE FONT SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM OTHER DEALINGS IN THE FONT SOFTWARE."
  },
  {
    "path": "model_data/coco_classes.txt",
    "content": "person\nbicycle\ncar\nmotorbike\naeroplane\nbus\ntrain\ntruck\nboat\ntraffic light\nfire hydrant\nstop sign\nparking meter\nbench\nbird\ncat\ndog\nhorse\nsheep\ncow\nelephant\nbear\nzebra\ngiraffe\nbackpack\numbrella\nhandbag\ntie\nsuitcase\nfrisbee\nskis\nsnowboard\nsports ball\nkite\nbaseball bat\nbaseball glove\nskateboard\nsurfboard\ntennis racket\nbottle\nwine glass\ncup\nfork\nknife\nspoon\nbowl\nbanana\napple\nsandwich\norange\nbroccoli\ncarrot\nhot dog\npizza\ndonut\ncake\nchair\nsofa\npottedplant\nbed\ndiningtable\ntoilet\ntvmonitor\nlaptop\nmouse\nremote\nkeyboard\ncell phone\nmicrowave\noven\ntoaster\nsink\nrefrigerator\nbook\nclock\nvase\nscissors\nteddy bear\nhair drier\ntoothbrush\n"
  },
  {
    "path": "model_data/yolo_anchors.txt",
    "content": "0.57273, 0.677385, 1.87446, 2.06253, 3.33843, 5.47434, 7.88282, 3.52778, 9.77052, 9.16828\n"
  },
  {
    "path": "yad2k/models/keras_darknet19.py",
    "content": "\"\"\"Darknet19 Model Defined in Keras.\"\"\"\nimport functools\nfrom functools import partial\n\nfrom keras.layers import Conv2D, MaxPooling2D\nfrom keras.layers.advanced_activations import LeakyReLU\nfrom keras.layers.normalization import BatchNormalization\nfrom keras.models import Model\nfrom keras.regularizers import l2\n\nfrom ..utils import compose\n\n# Partial wrapper for Convolution2D with static default argument.\n_DarknetConv2D = partial(Conv2D, padding='same')\n\n\n@functools.wraps(Conv2D)\ndef DarknetConv2D(*args, **kwargs):\n    \"\"\"Wrapper to set Darknet weight regularizer for Convolution2D.\"\"\"\n    darknet_conv_kwargs = {'kernel_regularizer': l2(5e-4)}\n    darknet_conv_kwargs.update(kwargs)\n    return _DarknetConv2D(*args, **darknet_conv_kwargs)\n\n\ndef DarknetConv2D_BN_Leaky(*args, **kwargs):\n    \"\"\"Darknet Convolution2D followed by BatchNormalization and LeakyReLU.\"\"\"\n    no_bias_kwargs = {'use_bias': False}\n    no_bias_kwargs.update(kwargs)\n    return compose(\n        DarknetConv2D(*args, **no_bias_kwargs),\n        BatchNormalization(),\n        LeakyReLU(alpha=0.1))\n\n\ndef bottleneck_block(outer_filters, bottleneck_filters):\n    \"\"\"Bottleneck block of 3x3, 1x1, 3x3 convolutions.\"\"\"\n    return compose(\n        DarknetConv2D_BN_Leaky(outer_filters, (3, 3)),\n        DarknetConv2D_BN_Leaky(bottleneck_filters, (1, 1)),\n        DarknetConv2D_BN_Leaky(outer_filters, (3, 3)))\n\n\ndef bottleneck_x2_block(outer_filters, bottleneck_filters):\n    \"\"\"Bottleneck block of 3x3, 1x1, 3x3, 1x1, 3x3 convolutions.\"\"\"\n    return compose(\n        bottleneck_block(outer_filters, bottleneck_filters),\n        DarknetConv2D_BN_Leaky(bottleneck_filters, (1, 1)),\n        DarknetConv2D_BN_Leaky(outer_filters, (3, 3)))\n\n\ndef darknet_body():\n    \"\"\"Generate first 18 conv layers of Darknet-19.\"\"\"\n    return compose(\n        DarknetConv2D_BN_Leaky(32, (3, 3)),\n        MaxPooling2D(),\n        DarknetConv2D_BN_Leaky(64, (3, 3)),\n        MaxPooling2D(),\n        bottleneck_block(128, 64),\n        MaxPooling2D(),\n        bottleneck_block(256, 128),\n        MaxPooling2D(),\n        bottleneck_x2_block(512, 256),\n        MaxPooling2D(),\n        bottleneck_x2_block(1024, 512))\n\n\ndef darknet19(inputs):\n    \"\"\"Generate Darknet-19 model for Imagenet classification.\"\"\"\n    body = darknet_body()(inputs)\n    logits = DarknetConv2D(1000, (1, 1), activation='softmax')(body)\n    return Model(inputs, logits)\n"
  },
  {
    "path": "yad2k/models/keras_yolo.py",
    "content": "\"\"\"YOLO_v2 Model Defined in Keras.\"\"\"\nimport sys\n\nimport numpy as np\nimport tensorflow as tf\nfrom keras import backend as K\nfrom keras.layers import Lambda\nfrom keras.layers.merge import concatenate\nfrom keras.models import Model\n\nfrom ..utils import compose\nfrom .keras_darknet19 import (DarknetConv2D, DarknetConv2D_BN_Leaky, darknet_body)\n\nsys.path.append('..')\n\nvoc_anchors = np.array(\n    [[1.08, 1.19], [3.42, 4.41], [6.63, 11.38], [9.42, 5.11], [16.62, 10.52]])\n\nvoc_classes = [\n    \"aeroplane\", \"bicycle\", \"bird\", \"boat\", \"bottle\", \"bus\", \"car\", \"cat\",\n    \"chair\", \"cow\", \"diningtable\", \"dog\", \"horse\", \"motorbike\", \"person\",\n    \"pottedplant\", \"sheep\", \"sofa\", \"train\", \"tvmonitor\"\n]\n\n\ndef space_to_depth_x2(x):\n    \"\"\"Thin wrapper for Tensorflow space_to_depth with block_size=2.\"\"\"\n    # Import currently required to make Lambda work.\n    # See: https://github.com/fchollet/keras/issues/5088#issuecomment-273851273\n    import tensorflow as tf\n    return tf.space_to_depth(x, block_size=2)\n\n\ndef space_to_depth_x2_output_shape(input_shape):\n    \"\"\"Determine space_to_depth output shape for block_size=2.\n\n    Note: For Lambda with TensorFlow backend, output shape may not be needed.\n    \"\"\"\n    return (input_shape[0], input_shape[1] // 2, input_shape[2] // 2, 4 *\n            input_shape[3]) if input_shape[1] else (input_shape[0], None, None,\n                                                    4 * input_shape[3])\n\n\ndef yolo_body(inputs, num_anchors, num_classes):\n    \"\"\"Create YOLO_V2 model CNN body in Keras.\"\"\"\n    darknet = Model(inputs, darknet_body()(inputs))\n    conv20 = compose(\n        DarknetConv2D_BN_Leaky(1024, (3, 3)),\n        DarknetConv2D_BN_Leaky(1024, (3, 3)))(darknet.output)\n\n    conv13 = darknet.layers[43].output\n    conv21 = DarknetConv2D_BN_Leaky(64, (1, 1))(conv13)\n    # TODO: Allow Keras Lambda to use func arguments for output_shape?\n    conv21_reshaped = Lambda(\n        space_to_depth_x2,\n        output_shape=space_to_depth_x2_output_shape,\n        name='space_to_depth')(conv21)\n\n    x = concatenate([conv21_reshaped, conv20])\n    x = DarknetConv2D_BN_Leaky(1024, (3, 3))(x)\n    x = DarknetConv2D(num_anchors * (num_classes + 5), (1, 1))(x)\n    return Model(inputs, x)\n\n\ndef yolo_head(feats, anchors, num_classes):\n    \"\"\"Convert final layer features to bounding box parameters.\n\n    Parameters\n    ----------\n    feats : tensor\n        Final convolutional layer features.\n    anchors : array-like\n        Anchor box widths and heights.\n    num_classes : int\n        Number of target classes.\n\n    Returns\n    -------\n    box_xy : tensor\n        x, y box predictions adjusted by spatial location in conv layer.\n    box_wh : tensor\n        w, h box predictions adjusted by anchors and conv spatial resolution.\n    box_conf : tensor\n        Probability estimate for whether each box contains any object.\n    box_class_pred : tensor\n        Probability distribution estimate for each box over class labels.\n    \"\"\"\n    num_anchors = len(anchors)\n    # Reshape to batch, height, width, num_anchors, box_params.\n    anchors_tensor = K.reshape(K.variable(anchors), [1, 1, 1, num_anchors, 2])\n    # Static implementation for fixed models.\n    # TODO: Remove or add option for static implementation.\n    # _, conv_height, conv_width, _ = K.int_shape(feats)\n    # conv_dims = K.variable([conv_width, conv_height])\n\n    # Dynamic implementation of conv dims for fully convolutional model.\n    conv_dims = K.shape(feats)[1:3]  # assuming channels last\n    # In YOLO the height index is the inner most iteration.\n    conv_height_index = K.arange(0, stop=conv_dims[0])\n    conv_width_index = K.arange(0, stop=conv_dims[1])\n    conv_height_index = K.tile(conv_height_index, [conv_dims[1]])\n\n    # TODO: Repeat_elements and tf.split doesn't support dynamic splits.\n    # conv_width_index = K.repeat_elements(conv_width_index, conv_dims[1], axis=0)\n    conv_width_index = K.tile(K.expand_dims(conv_width_index, 0), [conv_dims[0], 1])\n    conv_width_index = K.flatten(K.transpose(conv_width_index))\n    conv_index = K.transpose(K.stack([conv_height_index, conv_width_index]))\n    conv_index = K.reshape(conv_index, [1, conv_dims[0], conv_dims[1], 1, 2])\n    conv_index = K.cast(conv_index, K.dtype(feats))\n    \n    feats = K.reshape(feats, [-1, conv_dims[0], conv_dims[1], num_anchors, num_classes + 5])\n    conv_dims = K.cast(K.reshape(conv_dims, [1, 1, 1, 1, 2]), K.dtype(feats))\n\n    # Static generation of conv_index:\n    # conv_index = np.array([_ for _ in np.ndindex(conv_width, conv_height)])\n    # conv_index = conv_index[:, [1, 0]]  # swap columns for YOLO ordering.\n    # conv_index = K.variable(\n    #     conv_index.reshape(1, conv_height, conv_width, 1, 2))\n    # feats = Reshape(\n    #     (conv_dims[0], conv_dims[1], num_anchors, num_classes + 5))(feats)\n\n    box_confidence = K.sigmoid(feats[..., 4:5])\n    box_xy = K.sigmoid(feats[..., :2])\n    box_wh = K.exp(feats[..., 2:4])\n    box_class_probs = K.softmax(feats[..., 5:])\n\n    # Adjust preditions to each spatial grid point and anchor size.\n    # Note: YOLO iterates over height index before width index.\n    box_xy = (box_xy + conv_index) / conv_dims\n    box_wh = box_wh * anchors_tensor / conv_dims\n\n    return box_confidence, box_xy, box_wh, box_class_probs\n\n\ndef yolo_boxes_to_corners(box_xy, box_wh):\n    \"\"\"Convert YOLO box predictions to bounding box corners.\"\"\"\n    box_mins = box_xy - (box_wh / 2.)\n    box_maxes = box_xy + (box_wh / 2.)\n\n    return K.concatenate([\n        box_mins[..., 1:2],  # y_min\n        box_mins[..., 0:1],  # x_min\n        box_maxes[..., 1:2],  # y_max\n        box_maxes[..., 0:1]  # x_max\n    ])\n\n\ndef yolo_loss(args,\n              anchors,\n              num_classes,\n              rescore_confidence=False,\n              print_loss=False):\n    \"\"\"YOLO localization loss function.\n\n    Parameters\n    ----------\n    yolo_output : tensor\n        Final convolutional layer features.\n\n    true_boxes : tensor\n        Ground truth boxes tensor with shape [batch, num_true_boxes, 5]\n        containing box x_center, y_center, width, height, and class.\n\n    detectors_mask : array\n        0/1 mask for detector positions where there is a matching ground truth.\n\n    matching_true_boxes : array\n        Corresponding ground truth boxes for positive detector positions.\n        Already adjusted for conv height and width.\n\n    anchors : tensor\n        Anchor boxes for model.\n\n    num_classes : int\n        Number of object classes.\n\n    rescore_confidence : bool, default=False\n        If true then set confidence target to IOU of best predicted box with\n        the closest matching ground truth box.\n\n    print_loss : bool, default=False\n        If True then use a tf.Print() to print the loss components.\n\n    Returns\n    -------\n    mean_loss : float\n        mean localization loss across minibatch\n    \"\"\"\n    (yolo_output, true_boxes, detectors_mask, matching_true_boxes) = args\n    num_anchors = len(anchors)\n    object_scale = 5\n    no_object_scale = 1\n    class_scale = 1\n    coordinates_scale = 1\n    pred_xy, pred_wh, pred_confidence, pred_class_prob = yolo_head(\n        yolo_output, anchors, num_classes)\n\n    # Unadjusted box predictions for loss.\n    # TODO: Remove extra computation shared with yolo_head.\n    yolo_output_shape = K.shape(yolo_output)\n    feats = K.reshape(yolo_output, [\n        -1, yolo_output_shape[1], yolo_output_shape[2], num_anchors,\n        num_classes + 5\n    ])\n    pred_boxes = K.concatenate(\n        (K.sigmoid(feats[..., 0:2]), feats[..., 2:4]), axis=-1)\n\n    # TODO: Adjust predictions by image width/height for non-square images?\n    # IOUs may be off due to different aspect ratio.\n\n    # Expand pred x,y,w,h to allow comparison with ground truth.\n    # batch, conv_height, conv_width, num_anchors, num_true_boxes, box_params\n    pred_xy = K.expand_dims(pred_xy, 4)\n    pred_wh = K.expand_dims(pred_wh, 4)\n\n    pred_wh_half = pred_wh / 2.\n    pred_mins = pred_xy - pred_wh_half\n    pred_maxes = pred_xy + pred_wh_half\n\n    true_boxes_shape = K.shape(true_boxes)\n\n    # batch, conv_height, conv_width, num_anchors, num_true_boxes, box_params\n    true_boxes = K.reshape(true_boxes, [\n        true_boxes_shape[0], 1, 1, 1, true_boxes_shape[1], true_boxes_shape[2]\n    ])\n    true_xy = true_boxes[..., 0:2]\n    true_wh = true_boxes[..., 2:4]\n\n    # Find IOU of each predicted box with each ground truth box.\n    true_wh_half = true_wh / 2.\n    true_mins = true_xy - true_wh_half\n    true_maxes = true_xy + true_wh_half\n\n    intersect_mins = K.maximum(pred_mins, true_mins)\n    intersect_maxes = K.minimum(pred_maxes, true_maxes)\n    intersect_wh = K.maximum(intersect_maxes - intersect_mins, 0.)\n    intersect_areas = intersect_wh[..., 0] * intersect_wh[..., 1]\n\n    pred_areas = pred_wh[..., 0] * pred_wh[..., 1]\n    true_areas = true_wh[..., 0] * true_wh[..., 1]\n\n    union_areas = pred_areas + true_areas - intersect_areas\n    iou_scores = intersect_areas / union_areas\n\n    # Best IOUs for each location.\n    best_ious = K.max(iou_scores, axis=4)  # Best IOU scores.\n    best_ious = K.expand_dims(best_ious)\n\n    # A detector has found an object if IOU > thresh for some true box.\n    object_detections = K.cast(best_ious > 0.6, K.dtype(best_ious))\n\n    # TODO: Darknet region training includes extra coordinate loss for early\n    # training steps to encourage predictions to match anchor priors.\n\n    # Determine confidence weights from object and no_object weights.\n    # NOTE: YOLO does not use binary cross-entropy here.\n    no_object_weights = (no_object_scale * (1 - object_detections) *\n                         (1 - detectors_mask))\n    no_objects_loss = no_object_weights * K.square(-pred_confidence)\n\n    if rescore_confidence:\n        objects_loss = (object_scale * detectors_mask *\n                        K.square(best_ious - pred_confidence))\n    else:\n        objects_loss = (object_scale * detectors_mask *\n                        K.square(1 - pred_confidence))\n    confidence_loss = objects_loss + no_objects_loss\n\n    # Classification loss for matching detections.\n    # NOTE: YOLO does not use categorical cross-entropy loss here.\n    matching_classes = K.cast(matching_true_boxes[..., 4], 'int32')\n    matching_classes = K.one_hot(matching_classes, num_classes)\n    classification_loss = (class_scale * detectors_mask *\n                           K.square(matching_classes - pred_class_prob))\n\n    # Coordinate loss for matching detection boxes.\n    matching_boxes = matching_true_boxes[..., 0:4]\n    coordinates_loss = (coordinates_scale * detectors_mask *\n                        K.square(matching_boxes - pred_boxes))\n\n    confidence_loss_sum = K.sum(confidence_loss)\n    classification_loss_sum = K.sum(classification_loss)\n    coordinates_loss_sum = K.sum(coordinates_loss)\n    total_loss = 0.5 * (\n        confidence_loss_sum + classification_loss_sum + coordinates_loss_sum)\n    if print_loss:\n        total_loss = tf.Print(\n            total_loss, [\n                total_loss, confidence_loss_sum, classification_loss_sum,\n                coordinates_loss_sum\n            ],\n            message='yolo_loss, conf_loss, class_loss, box_coord_loss:')\n\n    return total_loss\n\n\ndef yolo(inputs, anchors, num_classes):\n    \"\"\"Generate a complete YOLO_v2 localization model.\"\"\"\n    num_anchors = len(anchors)\n    body = yolo_body(inputs, num_anchors, num_classes)\n    outputs = yolo_head(body.output, anchors, num_classes)\n    return outputs\n\n\ndef yolo_filter_boxes(box_confidence, boxes, box_class_probs, threshold=.6):\n    \"\"\"Filter YOLO boxes based on object and class confidence.\"\"\"\n\n    box_scores = box_confidence * box_class_probs\n    box_classes = K.argmax(box_scores, axis=-1)\n    box_class_scores = K.max(box_scores, axis=-1)\n    prediction_mask = box_class_scores >= threshold\n\n    # TODO: Expose tf.boolean_mask to Keras backend?\n    boxes = tf.boolean_mask(boxes, prediction_mask)\n    scores = tf.boolean_mask(box_class_scores, prediction_mask)\n    classes = tf.boolean_mask(box_classes, prediction_mask)\n\n    return boxes, scores, classes\n\n\ndef yolo_eval(yolo_outputs,\n              image_shape,\n              max_boxes=10,\n              score_threshold=.6,\n              iou_threshold=.5):\n    \"\"\"Evaluate YOLO model on given input batch and return filtered boxes.\"\"\"\n    box_confidence, box_xy, box_wh, box_class_probs = yolo_outputs\n    boxes = yolo_boxes_to_corners(box_xy, box_wh)\n    boxes, scores, classes = yolo_filter_boxes(\n        box_confidence, boxes, box_class_probs, threshold=score_threshold)\n    \n    # Scale boxes back to original image shape.\n    height = image_shape[0]\n    width = image_shape[1]\n    image_dims = K.stack([height, width, height, width])\n    image_dims = K.reshape(image_dims, [1, 4])\n    boxes = boxes * image_dims\n\n    # TODO: Something must be done about this ugly hack!\n    max_boxes_tensor = K.variable(max_boxes, dtype='int32')\n    K.get_session().run(tf.variables_initializer([max_boxes_tensor]))\n    nms_index = tf.image.non_max_suppression(boxes, scores, max_boxes_tensor, iou_threshold=iou_threshold)\n    # nms_index = tf.image.non_max_suppression(boxes, scores, max_boxes, iou_threshold=iou_threshold)\n    boxes = K.gather(boxes, nms_index)\n    scores = K.gather(scores, nms_index)\n    classes = K.gather(classes, nms_index)\n    \n    return boxes, scores, classes\n\n\ndef preprocess_true_boxes(true_boxes, anchors, image_size):\n    \"\"\"Find detector in YOLO where ground truth box should appear.\n\n    Parameters\n    ----------\n    true_boxes : array\n        List of ground truth boxes in form of relative x, y, w, h, class.\n        Relative coordinates are in the range [0, 1] indicating a percentage\n        of the original image dimensions.\n    anchors : array\n        List of anchors in form of w, h.\n        Anchors are assumed to be in the range [0, conv_size] where conv_size\n        is the spatial dimension of the final convolutional features.\n    image_size : array-like\n        List of image dimensions in form of h, w in pixels.\n\n    Returns\n    -------\n    detectors_mask : array\n        0/1 mask for detectors in [conv_height, conv_width, num_anchors, 1]\n        that should be compared with a matching ground truth box.\n    matching_true_boxes: array\n        Same shape as detectors_mask with the corresponding ground truth box\n        adjusted for comparison with predicted parameters at training time.\n    \"\"\"\n    height, width = image_size\n    num_anchors = len(anchors)\n    # Downsampling factor of 5x 2-stride max_pools == 32.\n    # TODO: Remove hardcoding of downscaling calculations.\n    assert height % 32 == 0, 'Image sizes in YOLO_v2 must be multiples of 32.'\n    assert width % 32 == 0, 'Image sizes in YOLO_v2 must be multiples of 32.'\n    conv_height = height // 32\n    conv_width = width // 32\n    num_box_params = true_boxes.shape[1]\n    detectors_mask = np.zeros(\n        (conv_height, conv_width, num_anchors, 1), dtype=np.float32)\n    matching_true_boxes = np.zeros(\n        (conv_height, conv_width, num_anchors, num_box_params),\n        dtype=np.float32)\n\n    for box in true_boxes:\n        # scale box to convolutional feature spatial dimensions\n        box_class = box[4:5]\n        box = box[0:4] * np.array(\n            [conv_width, conv_height, conv_width, conv_height])\n        i = np.floor(box[1]).astype('int')\n        j = min(np.floor(box[0]).astype('int'),1)\n        best_iou = 0\n        best_anchor = 0\n                \n        for k, anchor in enumerate(anchors):\n            # Find IOU between box shifted to origin and anchor box.\n            box_maxes = box[2:4] / 2.\n            box_mins = -box_maxes\n            anchor_maxes = (anchor / 2.)\n            anchor_mins = -anchor_maxes\n\n            intersect_mins = np.maximum(box_mins, anchor_mins)\n            intersect_maxes = np.minimum(box_maxes, anchor_maxes)\n            intersect_wh = np.maximum(intersect_maxes - intersect_mins, 0.)\n            intersect_area = intersect_wh[0] * intersect_wh[1]\n            box_area = box[2] * box[3]\n            anchor_area = anchor[0] * anchor[1]\n            iou = intersect_area / (box_area + anchor_area - intersect_area)\n            if iou > best_iou:\n                best_iou = iou\n                best_anchor = k\n                \n        if best_iou > 0:\n            detectors_mask[i, j, best_anchor] = 1\n            adjusted_box = np.array(\n                [\n                    box[0] - j, box[1] - i,\n                    np.log(box[2] / anchors[best_anchor][0]),\n                    np.log(box[3] / anchors[best_anchor][1]), box_class\n                ],\n                dtype=np.float32)\n            matching_true_boxes[i, j, best_anchor] = adjusted_box\n    return detectors_mask, matching_true_boxes\n"
  },
  {
    "path": "yad2k/utils/__init__.py",
    "content": "from .utils import *\n"
  },
  {
    "path": "yad2k/utils/utils.py",
    "content": "\"\"\"Miscellaneous utility functions.\"\"\"\n\nfrom functools import reduce\n\n\ndef compose(*funcs):\n    \"\"\"Compose arbitrarily many functions, evaluated left to right.\n\n    Reference: https://mathieularose.com/function-composition-in-python/\n    \"\"\"\n    # return lambda x: reduce(lambda v, f: f(v), funcs, x)\n    if funcs:\n        return reduce(lambda f, g: lambda *a, **kw: g(f(*a, **kw)), funcs)\n    else:\n        raise ValueError('Composition of empty sequence not supported.')\n"
  },
  {
    "path": "yolo.py",
    "content": "# import the needed modules\nimport os\nfrom matplotlib.pyplot import imshow\nimport scipy.io\nimport scipy.misc\nimport numpy as np\nfrom PIL import Image\n\nfrom keras import backend as K\nfrom keras.models import load_model\n\n# The below provided fucntions will be used from yolo_utils.py\nfrom yolo_utils import read_classes, read_anchors, generate_colors, preprocess_image, draw_boxes\n\n# The below functions from the yad2k library will be used\nfrom yad2k.models.keras_yolo import yolo_head, yolo_eval\n\n\n#Provide the name of the image that you saved in the images folder to be fed through the network\ninput_image_name = \"test45.jpg\"\n\n#Obtaining the dimensions of the input image\ninput_image = Image.open(\"images/\" + input_image_name)\nwidth, height = input_image.size\nwidth = np.array(width, dtype=float)\nheight = np.array(height, dtype=float)\n\n#Assign the shape of the input image to image_shapr variable\nimage_shape = (height, width)\n\n\n#Loading the classes and the anchor boxes that are provided in the madel_data folder\nclass_names = read_classes(\"model_data/coco_classes.txt\")\nanchors = read_anchors(\"model_data/yolo_anchors.txt\")\n\n#Load the pretrained model. Please refer the README file to get info on how to obtain the yolo.h5 file\nyolo_model = load_model(\"model_data/yolo.h5\")\n\n#Print the summery of the model\nyolo_model.summary()\n\n#Convert final layer features to bounding box parameters\nyolo_outputs = yolo_head(yolo_model.output, anchors, len(class_names))\n\n#Now yolo_eval function selects the best boxes using filtering and non-max suppression techniques.\n# If you want to dive in more to see how this works, refer keras_yolo.py file in yad2k/models\nboxes, scores, classes = yolo_eval(yolo_outputs, image_shape)\n\n\n# Initiate a session\nsess = K.get_session()\n\n\n#Preprocess the input image before feeding into the convolutional network\nimage, image_data = preprocess_image(\"images/\" + input_image_name, model_image_size = (608, 608))\n\n#Run the session\nout_scores, out_boxes, out_classes = sess.run([scores, boxes, classes],feed_dict={yolo_model.input:image_data,K.learning_phase(): 0})\n\n\n#Print the results\nprint('Found {} boxes for {}'.format(len(out_boxes), input_image_name))\n#Produce the colors for the bounding boxs\ncolors = generate_colors(class_names)\n#Draw the bounding boxes\ndraw_boxes(image, out_scores, out_boxes, out_classes, class_names, colors)\n#Apply the predicted bounding boxes to the image and save it\nimage.save(os.path.join(\"out\", input_image_name), quality=90)\noutput_image = scipy.misc.imread(os.path.join(\"out\", input_image_name))\nimshow(output_image)\n"
  },
  {
    "path": "yolo_utils.py",
    "content": "import colorsys\nimport imghdr\nimport os\nimport random\nfrom keras import backend as K\n\nimport numpy as np\nfrom PIL import Image, ImageDraw, ImageFont\n\ndef read_classes(classes_path):\n    with open(classes_path) as f:\n        class_names = f.readlines()\n    class_names = [c.strip() for c in class_names]\n    return class_names\n\ndef read_anchors(anchors_path):\n    with open(anchors_path) as f:\n        anchors = f.readline()\n        anchors = [float(x) for x in anchors.split(',')]\n        anchors = np.array(anchors).reshape(-1, 2)\n    return anchors\n\ndef generate_colors(class_names):\n    hsv_tuples = [(x / len(class_names), 1., 1.) for x in range(len(class_names))]\n    colors = list(map(lambda x: colorsys.hsv_to_rgb(*x), hsv_tuples))\n    colors = list(map(lambda x: (int(x[0] * 255), int(x[1] * 255), int(x[2] * 255)), colors))\n    random.seed(10101)  # Fixed seed for consistent colors across runs.\n    random.shuffle(colors)  # Shuffle colors to decorrelate adjacent classes.\n    random.seed(None)  # Reset seed to default.\n    return colors\n\ndef scale_boxes(boxes, image_shape):\n    \"\"\" Scales the predicted boxes in order to be drawable on the image\"\"\"\n    height = image_shape[0]\n    width = image_shape[1]\n    image_dims = K.stack([height, width, height, width])\n    image_dims = K.reshape(image_dims, [1, 4])\n    boxes = boxes * image_dims\n    return boxes\n\ndef preprocess_image(img_path, model_image_size):\n    image_type = imghdr.what(img_path)\n    image = Image.open(img_path)\n    resized_image = image.resize(tuple(reversed(model_image_size)), Image.BICUBIC)\n    image_data = np.array(resized_image, dtype='float32')\n    image_data /= 255.\n    image_data = np.expand_dims(image_data, 0)  # Add batch dimension.\n    return image, image_data\n\ndef draw_boxes(image, out_scores, out_boxes, out_classes, class_names, colors):\n    \n    font = ImageFont.truetype(font='font/FiraMono-Medium.otf',size=np.floor(3e-2 * image.size[1] + 0.5).astype('int32'))\n    thickness = (image.size[0] + image.size[1]) // 300\n\n    for i, c in reversed(list(enumerate(out_classes))):\n        predicted_class = class_names[c]\n        box = out_boxes[i]\n        score = out_scores[i]\n\n        label = '{} {:.2f}'.format(predicted_class, score)\n\n        draw = ImageDraw.Draw(image)\n        label_size = draw.textsize(label, font)\n\n        top, left, bottom, right = box\n        top = max(0, np.floor(top + 0.5).astype('int32'))\n        left = max(0, np.floor(left + 0.5).astype('int32'))\n        bottom = min(image.size[1], np.floor(bottom + 0.5).astype('int32'))\n        right = min(image.size[0], np.floor(right + 0.5).astype('int32'))\n        print(label, (left, top), (right, bottom))\n\n        if top - label_size[1] >= 0:\n            text_origin = np.array([left, top - label_size[1]])\n        else:\n            text_origin = np.array([left, top + 1])\n\n        # My kingdom for a good redistributable image drawing library.\n        for i in range(thickness):\n            draw.rectangle([left + i, top + i, right - i, bottom - i], outline=colors[c])\n        draw.rectangle([tuple(text_origin), tuple(text_origin + label_size)], fill=colors[c])\n        draw.text(text_origin, label, fill=(0, 0, 0), font=font)\n        del draw"
  }
]