Repository: miranthajayatilake/YOLOw-Keras Branch: master Commit: d1526fc6a64c Files: 11 Total size: 33.5 KB Directory structure: gitextract_m3h95r1z/ ├── README.md ├── font/ │ ├── FiraMono-Medium.otf │ └── SIL+Open+Font+License.txt ├── model_data/ │ ├── coco_classes.txt │ └── yolo_anchors.txt ├── yad2k/ │ ├── models/ │ │ ├── keras_darknet19.py │ │ └── keras_yolo.py │ └── utils/ │ ├── __init__.py │ └── utils.py ├── yolo.py └── yolo_utils.py ================================================ FILE CONTENTS ================================================ ================================================ FILE: README.md ================================================ # YOLOv2 Object Detection w/ Keras (in just 20 lines of code) This repository presents a quick and simple implementation of YOLOv2 object detection using Keras library with Tensorflow backend. Credits goes to [YAD2K Library](https://github.com/allanzelener/YAD2K) on top of which this implementation was built. ![cover01](etc/cover01.jpg) ![cover02](etc/cover02.jpg) ###### Note that I do not hold ownership to any of the above pictures. These are merely used for educational purposes to describe the concepts. -------------------------------------------------------------------------------- ## Thoughts on the implementation YOLO is well known technique used to perform fast multiple localizations on a single image. A brief algorithm breakdown; - Divide the image using a grid (eg: 19x19) - Perform image classification and Localization on each grid cell -> Result, a vector for each cell representing the probability of an object detected, the dimensions of the bounding box and class of the detected image. - Perform thresholding to remove multiple detected instances - Perform Non-max suppression to refine the boxes more - Additionally anchor boxes are used to detect several objects in one grid cell ###### If you want to dive down into how these above points are implemented in the code refer yolo_eval function in the keras_yolo.py file from the yad2k/models directory. Paper reference: [YOLO9000: Better, Faster, Stronger](https://arxiv.org/abs/1612.08242) by Joseph Redmond and Ali Farhadi. [Keras](https://keras.io/) is a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano. It was developed with a focus on enabling fast experimentation. Being able to go from idea to result with the least possible delay is key to doing good research. The use of keras helps to understand the concepts underling a ML technique by reducing the workload on coding. Thus, this implementation becomes a good platform for beginners to core concepts and have a quick implementation giving results. Also for anyone who is looking to integrate object detection capabilities in applications, this code can be incorperated with a few lines of code. Blog post on this: https://medium.com/@miranthaj/quick-implementation-of-yolo-v2-with-keras-ebf6eb40c684 -------------------------------------------------------------------------------- ## Quick Start - Clone this repository to your PC - Download any Darknet model cfg and weights from the [official YOLO website](http://pjreddie.com/darknet/yolo/). - Convert the dowloaded cfg and weights files into a h5 file using YAD2K library. (This is explained step by step below in the more details section) - Copy the generated h5 file to the model_data folder and edit the name of the pretrained model in yolo.py code to the name of your h5 file. - Place the input image you want to try object detection in the images folder and copy its file name. - Assign your input image file name to input_image_name variable in yolo.py. - Open terminal from the repository directory directly and run the yolo.py file `python yolo.py` -------------------------------------------------------------------------------- ## More Details How to convert cfg and weights files to h5 using YAD2k library (Windows) - Clone the [YAD2K Library](https://github.com/allanzelener/YAD2K) to your PC - Open terminal from the cloned directory - Copy and paste the downloaded weights and cfg files to the YAD2K master directory - Run `python yad2k.py yolo.cfg yolo.weights model_data/yolo.h5` on the terminal and the h5 file will be generated. - Move the generated h5 file to model_data folder of the simpleYOLOwKeras directory ------------------------------------------------------------------------------- ================================================ FILE: font/SIL+Open+Font+License.txt ================================================ Copyright (c) 2014, Mozilla Foundation https://mozilla.org/ with Reserved Font Name Fira Mono. Copyright (c) 2014, Telefonica S.A. This Font Software is licensed under the SIL Open Font License, Version 1.1. This license is copied below, and is also available with a FAQ at: http://scripts.sil.org/OFL ----------------------------------------------------------- SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007 ----------------------------------------------------------- PREAMBLE The goals of the Open Font License (OFL) are to stimulate worldwide development of collaborative font projects, to support the font creation efforts of academic and linguistic communities, and to provide a free and open framework in which fonts may be shared and improved in partnership with others. The OFL allows the licensed fonts to be used, studied, modified and redistributed freely as long as they are not sold by themselves. The fonts, including any derivative works, can be bundled, embedded, redistributed and/or sold with any software provided that any reserved names are not used by derivative works. The fonts and derivatives, however, cannot be released under any other type of license. The requirement for fonts to remain under this license does not apply to any document created using the fonts or their derivatives. DEFINITIONS "Font Software" refers to the set of files released by the Copyright Holder(s) under this license and clearly marked as such. This may include source files, build scripts and documentation. "Reserved Font Name" refers to any names specified as such after the copyright statement(s). "Original Version" refers to the collection of Font Software components as distributed by the Copyright Holder(s). "Modified Version" refers to any derivative made by adding to, deleting, or substituting -- in part or in whole -- any of the components of the Original Version, by changing formats or by porting the Font Software to a new environment. "Author" refers to any designer, engineer, programmer, technical writer or other person who contributed to the Font Software. PERMISSION & CONDITIONS Permission is hereby granted, free of charge, to any person obtaining a copy of the Font Software, to use, study, copy, merge, embed, modify, redistribute, and sell modified and unmodified copies of the Font Software, subject to the following conditions: 1) Neither the Font Software nor any of its individual components, in Original or Modified Versions, may be sold by itself. 2) Original or Modified Versions of the Font Software may be bundled, redistributed and/or sold with any software, provided that each copy contains the above copyright notice and this license. These can be included either as stand-alone text files, human-readable headers or in the appropriate machine-readable metadata fields within text or binary files as long as those fields can be easily viewed by the user. 3) No Modified Version of the Font Software may use the Reserved Font Name(s) unless explicit written permission is granted by the corresponding Copyright Holder. This restriction only applies to the primary font name as presented to the users. 4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font Software shall not be used to promote, endorse or advertise any Modified Version, except to acknowledge the contribution(s) of the Copyright Holder(s) and the Author(s) or with their explicit written permission. 5) The Font Software, modified or unmodified, in part or in whole, must be distributed entirely under this license, and must not be distributed under any other license. The requirement for fonts to remain under this license does not apply to any document created using the Font Software. TERMINATION This license becomes null and void if any of the above conditions are not met. DISCLAIMER THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM OTHER DEALINGS IN THE FONT SOFTWARE. ================================================ FILE: model_data/coco_classes.txt ================================================ person bicycle car motorbike aeroplane bus train truck boat traffic light fire hydrant stop sign parking meter bench bird cat dog horse sheep cow elephant bear zebra giraffe backpack umbrella handbag tie suitcase frisbee skis snowboard sports ball kite baseball bat baseball glove skateboard surfboard tennis racket bottle wine glass cup fork knife spoon bowl banana apple sandwich orange broccoli carrot hot dog pizza donut cake chair sofa pottedplant bed diningtable toilet tvmonitor laptop mouse remote keyboard cell phone microwave oven toaster sink refrigerator book clock vase scissors teddy bear hair drier toothbrush ================================================ FILE: model_data/yolo_anchors.txt ================================================ 0.57273, 0.677385, 1.87446, 2.06253, 3.33843, 5.47434, 7.88282, 3.52778, 9.77052, 9.16828 ================================================ FILE: yad2k/models/keras_darknet19.py ================================================ """Darknet19 Model Defined in Keras.""" import functools from functools import partial from keras.layers import Conv2D, MaxPooling2D from keras.layers.advanced_activations import LeakyReLU from keras.layers.normalization import BatchNormalization from keras.models import Model from keras.regularizers import l2 from ..utils import compose # Partial wrapper for Convolution2D with static default argument. _DarknetConv2D = partial(Conv2D, padding='same') @functools.wraps(Conv2D) def DarknetConv2D(*args, **kwargs): """Wrapper to set Darknet weight regularizer for Convolution2D.""" darknet_conv_kwargs = {'kernel_regularizer': l2(5e-4)} darknet_conv_kwargs.update(kwargs) return _DarknetConv2D(*args, **darknet_conv_kwargs) def DarknetConv2D_BN_Leaky(*args, **kwargs): """Darknet Convolution2D followed by BatchNormalization and LeakyReLU.""" no_bias_kwargs = {'use_bias': False} no_bias_kwargs.update(kwargs) return compose( DarknetConv2D(*args, **no_bias_kwargs), BatchNormalization(), LeakyReLU(alpha=0.1)) def bottleneck_block(outer_filters, bottleneck_filters): """Bottleneck block of 3x3, 1x1, 3x3 convolutions.""" return compose( DarknetConv2D_BN_Leaky(outer_filters, (3, 3)), DarknetConv2D_BN_Leaky(bottleneck_filters, (1, 1)), DarknetConv2D_BN_Leaky(outer_filters, (3, 3))) def bottleneck_x2_block(outer_filters, bottleneck_filters): """Bottleneck block of 3x3, 1x1, 3x3, 1x1, 3x3 convolutions.""" return compose( bottleneck_block(outer_filters, bottleneck_filters), DarknetConv2D_BN_Leaky(bottleneck_filters, (1, 1)), DarknetConv2D_BN_Leaky(outer_filters, (3, 3))) def darknet_body(): """Generate first 18 conv layers of Darknet-19.""" return compose( DarknetConv2D_BN_Leaky(32, (3, 3)), MaxPooling2D(), DarknetConv2D_BN_Leaky(64, (3, 3)), MaxPooling2D(), bottleneck_block(128, 64), MaxPooling2D(), bottleneck_block(256, 128), MaxPooling2D(), bottleneck_x2_block(512, 256), MaxPooling2D(), bottleneck_x2_block(1024, 512)) def darknet19(inputs): """Generate Darknet-19 model for Imagenet classification.""" body = darknet_body()(inputs) logits = DarknetConv2D(1000, (1, 1), activation='softmax')(body) return Model(inputs, logits) ================================================ FILE: yad2k/models/keras_yolo.py ================================================ """YOLO_v2 Model Defined in Keras.""" import sys import numpy as np import tensorflow as tf from keras import backend as K from keras.layers import Lambda from keras.layers.merge import concatenate from keras.models import Model from ..utils import compose from .keras_darknet19 import (DarknetConv2D, DarknetConv2D_BN_Leaky, darknet_body) sys.path.append('..') voc_anchors = np.array( [[1.08, 1.19], [3.42, 4.41], [6.63, 11.38], [9.42, 5.11], [16.62, 10.52]]) voc_classes = [ "aeroplane", "bicycle", "bird", "boat", "bottle", "bus", "car", "cat", "chair", "cow", "diningtable", "dog", "horse", "motorbike", "person", "pottedplant", "sheep", "sofa", "train", "tvmonitor" ] def space_to_depth_x2(x): """Thin wrapper for Tensorflow space_to_depth with block_size=2.""" # Import currently required to make Lambda work. # See: https://github.com/fchollet/keras/issues/5088#issuecomment-273851273 import tensorflow as tf return tf.space_to_depth(x, block_size=2) def space_to_depth_x2_output_shape(input_shape): """Determine space_to_depth output shape for block_size=2. Note: For Lambda with TensorFlow backend, output shape may not be needed. """ return (input_shape[0], input_shape[1] // 2, input_shape[2] // 2, 4 * input_shape[3]) if input_shape[1] else (input_shape[0], None, None, 4 * input_shape[3]) def yolo_body(inputs, num_anchors, num_classes): """Create YOLO_V2 model CNN body in Keras.""" darknet = Model(inputs, darknet_body()(inputs)) conv20 = compose( DarknetConv2D_BN_Leaky(1024, (3, 3)), DarknetConv2D_BN_Leaky(1024, (3, 3)))(darknet.output) conv13 = darknet.layers[43].output conv21 = DarknetConv2D_BN_Leaky(64, (1, 1))(conv13) # TODO: Allow Keras Lambda to use func arguments for output_shape? conv21_reshaped = Lambda( space_to_depth_x2, output_shape=space_to_depth_x2_output_shape, name='space_to_depth')(conv21) x = concatenate([conv21_reshaped, conv20]) x = DarknetConv2D_BN_Leaky(1024, (3, 3))(x) x = DarknetConv2D(num_anchors * (num_classes + 5), (1, 1))(x) return Model(inputs, x) def yolo_head(feats, anchors, num_classes): """Convert final layer features to bounding box parameters. Parameters ---------- feats : tensor Final convolutional layer features. anchors : array-like Anchor box widths and heights. num_classes : int Number of target classes. Returns ------- box_xy : tensor x, y box predictions adjusted by spatial location in conv layer. box_wh : tensor w, h box predictions adjusted by anchors and conv spatial resolution. box_conf : tensor Probability estimate for whether each box contains any object. box_class_pred : tensor Probability distribution estimate for each box over class labels. """ num_anchors = len(anchors) # Reshape to batch, height, width, num_anchors, box_params. anchors_tensor = K.reshape(K.variable(anchors), [1, 1, 1, num_anchors, 2]) # Static implementation for fixed models. # TODO: Remove or add option for static implementation. # _, conv_height, conv_width, _ = K.int_shape(feats) # conv_dims = K.variable([conv_width, conv_height]) # Dynamic implementation of conv dims for fully convolutional model. conv_dims = K.shape(feats)[1:3] # assuming channels last # In YOLO the height index is the inner most iteration. conv_height_index = K.arange(0, stop=conv_dims[0]) conv_width_index = K.arange(0, stop=conv_dims[1]) conv_height_index = K.tile(conv_height_index, [conv_dims[1]]) # TODO: Repeat_elements and tf.split doesn't support dynamic splits. # conv_width_index = K.repeat_elements(conv_width_index, conv_dims[1], axis=0) conv_width_index = K.tile(K.expand_dims(conv_width_index, 0), [conv_dims[0], 1]) conv_width_index = K.flatten(K.transpose(conv_width_index)) conv_index = K.transpose(K.stack([conv_height_index, conv_width_index])) conv_index = K.reshape(conv_index, [1, conv_dims[0], conv_dims[1], 1, 2]) conv_index = K.cast(conv_index, K.dtype(feats)) feats = K.reshape(feats, [-1, conv_dims[0], conv_dims[1], num_anchors, num_classes + 5]) conv_dims = K.cast(K.reshape(conv_dims, [1, 1, 1, 1, 2]), K.dtype(feats)) # Static generation of conv_index: # conv_index = np.array([_ for _ in np.ndindex(conv_width, conv_height)]) # conv_index = conv_index[:, [1, 0]] # swap columns for YOLO ordering. # conv_index = K.variable( # conv_index.reshape(1, conv_height, conv_width, 1, 2)) # feats = Reshape( # (conv_dims[0], conv_dims[1], num_anchors, num_classes + 5))(feats) box_confidence = K.sigmoid(feats[..., 4:5]) box_xy = K.sigmoid(feats[..., :2]) box_wh = K.exp(feats[..., 2:4]) box_class_probs = K.softmax(feats[..., 5:]) # Adjust preditions to each spatial grid point and anchor size. # Note: YOLO iterates over height index before width index. box_xy = (box_xy + conv_index) / conv_dims box_wh = box_wh * anchors_tensor / conv_dims return box_confidence, box_xy, box_wh, box_class_probs def yolo_boxes_to_corners(box_xy, box_wh): """Convert YOLO box predictions to bounding box corners.""" box_mins = box_xy - (box_wh / 2.) box_maxes = box_xy + (box_wh / 2.) return K.concatenate([ box_mins[..., 1:2], # y_min box_mins[..., 0:1], # x_min box_maxes[..., 1:2], # y_max box_maxes[..., 0:1] # x_max ]) def yolo_loss(args, anchors, num_classes, rescore_confidence=False, print_loss=False): """YOLO localization loss function. Parameters ---------- yolo_output : tensor Final convolutional layer features. true_boxes : tensor Ground truth boxes tensor with shape [batch, num_true_boxes, 5] containing box x_center, y_center, width, height, and class. detectors_mask : array 0/1 mask for detector positions where there is a matching ground truth. matching_true_boxes : array Corresponding ground truth boxes for positive detector positions. Already adjusted for conv height and width. anchors : tensor Anchor boxes for model. num_classes : int Number of object classes. rescore_confidence : bool, default=False If true then set confidence target to IOU of best predicted box with the closest matching ground truth box. print_loss : bool, default=False If True then use a tf.Print() to print the loss components. Returns ------- mean_loss : float mean localization loss across minibatch """ (yolo_output, true_boxes, detectors_mask, matching_true_boxes) = args num_anchors = len(anchors) object_scale = 5 no_object_scale = 1 class_scale = 1 coordinates_scale = 1 pred_xy, pred_wh, pred_confidence, pred_class_prob = yolo_head( yolo_output, anchors, num_classes) # Unadjusted box predictions for loss. # TODO: Remove extra computation shared with yolo_head. yolo_output_shape = K.shape(yolo_output) feats = K.reshape(yolo_output, [ -1, yolo_output_shape[1], yolo_output_shape[2], num_anchors, num_classes + 5 ]) pred_boxes = K.concatenate( (K.sigmoid(feats[..., 0:2]), feats[..., 2:4]), axis=-1) # TODO: Adjust predictions by image width/height for non-square images? # IOUs may be off due to different aspect ratio. # Expand pred x,y,w,h to allow comparison with ground truth. # batch, conv_height, conv_width, num_anchors, num_true_boxes, box_params pred_xy = K.expand_dims(pred_xy, 4) pred_wh = K.expand_dims(pred_wh, 4) pred_wh_half = pred_wh / 2. pred_mins = pred_xy - pred_wh_half pred_maxes = pred_xy + pred_wh_half true_boxes_shape = K.shape(true_boxes) # batch, conv_height, conv_width, num_anchors, num_true_boxes, box_params true_boxes = K.reshape(true_boxes, [ true_boxes_shape[0], 1, 1, 1, true_boxes_shape[1], true_boxes_shape[2] ]) true_xy = true_boxes[..., 0:2] true_wh = true_boxes[..., 2:4] # Find IOU of each predicted box with each ground truth box. true_wh_half = true_wh / 2. true_mins = true_xy - true_wh_half true_maxes = true_xy + true_wh_half intersect_mins = K.maximum(pred_mins, true_mins) intersect_maxes = K.minimum(pred_maxes, true_maxes) intersect_wh = K.maximum(intersect_maxes - intersect_mins, 0.) intersect_areas = intersect_wh[..., 0] * intersect_wh[..., 1] pred_areas = pred_wh[..., 0] * pred_wh[..., 1] true_areas = true_wh[..., 0] * true_wh[..., 1] union_areas = pred_areas + true_areas - intersect_areas iou_scores = intersect_areas / union_areas # Best IOUs for each location. best_ious = K.max(iou_scores, axis=4) # Best IOU scores. best_ious = K.expand_dims(best_ious) # A detector has found an object if IOU > thresh for some true box. object_detections = K.cast(best_ious > 0.6, K.dtype(best_ious)) # TODO: Darknet region training includes extra coordinate loss for early # training steps to encourage predictions to match anchor priors. # Determine confidence weights from object and no_object weights. # NOTE: YOLO does not use binary cross-entropy here. no_object_weights = (no_object_scale * (1 - object_detections) * (1 - detectors_mask)) no_objects_loss = no_object_weights * K.square(-pred_confidence) if rescore_confidence: objects_loss = (object_scale * detectors_mask * K.square(best_ious - pred_confidence)) else: objects_loss = (object_scale * detectors_mask * K.square(1 - pred_confidence)) confidence_loss = objects_loss + no_objects_loss # Classification loss for matching detections. # NOTE: YOLO does not use categorical cross-entropy loss here. matching_classes = K.cast(matching_true_boxes[..., 4], 'int32') matching_classes = K.one_hot(matching_classes, num_classes) classification_loss = (class_scale * detectors_mask * K.square(matching_classes - pred_class_prob)) # Coordinate loss for matching detection boxes. matching_boxes = matching_true_boxes[..., 0:4] coordinates_loss = (coordinates_scale * detectors_mask * K.square(matching_boxes - pred_boxes)) confidence_loss_sum = K.sum(confidence_loss) classification_loss_sum = K.sum(classification_loss) coordinates_loss_sum = K.sum(coordinates_loss) total_loss = 0.5 * ( confidence_loss_sum + classification_loss_sum + coordinates_loss_sum) if print_loss: total_loss = tf.Print( total_loss, [ total_loss, confidence_loss_sum, classification_loss_sum, coordinates_loss_sum ], message='yolo_loss, conf_loss, class_loss, box_coord_loss:') return total_loss def yolo(inputs, anchors, num_classes): """Generate a complete YOLO_v2 localization model.""" num_anchors = len(anchors) body = yolo_body(inputs, num_anchors, num_classes) outputs = yolo_head(body.output, anchors, num_classes) return outputs def yolo_filter_boxes(box_confidence, boxes, box_class_probs, threshold=.6): """Filter YOLO boxes based on object and class confidence.""" box_scores = box_confidence * box_class_probs box_classes = K.argmax(box_scores, axis=-1) box_class_scores = K.max(box_scores, axis=-1) prediction_mask = box_class_scores >= threshold # TODO: Expose tf.boolean_mask to Keras backend? boxes = tf.boolean_mask(boxes, prediction_mask) scores = tf.boolean_mask(box_class_scores, prediction_mask) classes = tf.boolean_mask(box_classes, prediction_mask) return boxes, scores, classes def yolo_eval(yolo_outputs, image_shape, max_boxes=10, score_threshold=.6, iou_threshold=.5): """Evaluate YOLO model on given input batch and return filtered boxes.""" box_confidence, box_xy, box_wh, box_class_probs = yolo_outputs boxes = yolo_boxes_to_corners(box_xy, box_wh) boxes, scores, classes = yolo_filter_boxes( box_confidence, boxes, box_class_probs, threshold=score_threshold) # Scale boxes back to original image shape. height = image_shape[0] width = image_shape[1] image_dims = K.stack([height, width, height, width]) image_dims = K.reshape(image_dims, [1, 4]) boxes = boxes * image_dims # TODO: Something must be done about this ugly hack! max_boxes_tensor = K.variable(max_boxes, dtype='int32') K.get_session().run(tf.variables_initializer([max_boxes_tensor])) nms_index = tf.image.non_max_suppression(boxes, scores, max_boxes_tensor, iou_threshold=iou_threshold) # nms_index = tf.image.non_max_suppression(boxes, scores, max_boxes, iou_threshold=iou_threshold) boxes = K.gather(boxes, nms_index) scores = K.gather(scores, nms_index) classes = K.gather(classes, nms_index) return boxes, scores, classes def preprocess_true_boxes(true_boxes, anchors, image_size): """Find detector in YOLO where ground truth box should appear. Parameters ---------- true_boxes : array List of ground truth boxes in form of relative x, y, w, h, class. Relative coordinates are in the range [0, 1] indicating a percentage of the original image dimensions. anchors : array List of anchors in form of w, h. Anchors are assumed to be in the range [0, conv_size] where conv_size is the spatial dimension of the final convolutional features. image_size : array-like List of image dimensions in form of h, w in pixels. Returns ------- detectors_mask : array 0/1 mask for detectors in [conv_height, conv_width, num_anchors, 1] that should be compared with a matching ground truth box. matching_true_boxes: array Same shape as detectors_mask with the corresponding ground truth box adjusted for comparison with predicted parameters at training time. """ height, width = image_size num_anchors = len(anchors) # Downsampling factor of 5x 2-stride max_pools == 32. # TODO: Remove hardcoding of downscaling calculations. assert height % 32 == 0, 'Image sizes in YOLO_v2 must be multiples of 32.' assert width % 32 == 0, 'Image sizes in YOLO_v2 must be multiples of 32.' conv_height = height // 32 conv_width = width // 32 num_box_params = true_boxes.shape[1] detectors_mask = np.zeros( (conv_height, conv_width, num_anchors, 1), dtype=np.float32) matching_true_boxes = np.zeros( (conv_height, conv_width, num_anchors, num_box_params), dtype=np.float32) for box in true_boxes: # scale box to convolutional feature spatial dimensions box_class = box[4:5] box = box[0:4] * np.array( [conv_width, conv_height, conv_width, conv_height]) i = np.floor(box[1]).astype('int') j = min(np.floor(box[0]).astype('int'),1) best_iou = 0 best_anchor = 0 for k, anchor in enumerate(anchors): # Find IOU between box shifted to origin and anchor box. box_maxes = box[2:4] / 2. box_mins = -box_maxes anchor_maxes = (anchor / 2.) anchor_mins = -anchor_maxes intersect_mins = np.maximum(box_mins, anchor_mins) intersect_maxes = np.minimum(box_maxes, anchor_maxes) intersect_wh = np.maximum(intersect_maxes - intersect_mins, 0.) intersect_area = intersect_wh[0] * intersect_wh[1] box_area = box[2] * box[3] anchor_area = anchor[0] * anchor[1] iou = intersect_area / (box_area + anchor_area - intersect_area) if iou > best_iou: best_iou = iou best_anchor = k if best_iou > 0: detectors_mask[i, j, best_anchor] = 1 adjusted_box = np.array( [ box[0] - j, box[1] - i, np.log(box[2] / anchors[best_anchor][0]), np.log(box[3] / anchors[best_anchor][1]), box_class ], dtype=np.float32) matching_true_boxes[i, j, best_anchor] = adjusted_box return detectors_mask, matching_true_boxes ================================================ FILE: yad2k/utils/__init__.py ================================================ from .utils import * ================================================ FILE: yad2k/utils/utils.py ================================================ """Miscellaneous utility functions.""" from functools import reduce def compose(*funcs): """Compose arbitrarily many functions, evaluated left to right. Reference: https://mathieularose.com/function-composition-in-python/ """ # return lambda x: reduce(lambda v, f: f(v), funcs, x) if funcs: return reduce(lambda f, g: lambda *a, **kw: g(f(*a, **kw)), funcs) else: raise ValueError('Composition of empty sequence not supported.') ================================================ FILE: yolo.py ================================================ # import the needed modules import os from matplotlib.pyplot import imshow import scipy.io import scipy.misc import numpy as np from PIL import Image from keras import backend as K from keras.models import load_model # The below provided fucntions will be used from yolo_utils.py from yolo_utils import read_classes, read_anchors, generate_colors, preprocess_image, draw_boxes # The below functions from the yad2k library will be used from yad2k.models.keras_yolo import yolo_head, yolo_eval #Provide the name of the image that you saved in the images folder to be fed through the network input_image_name = "test45.jpg" #Obtaining the dimensions of the input image input_image = Image.open("images/" + input_image_name) width, height = input_image.size width = np.array(width, dtype=float) height = np.array(height, dtype=float) #Assign the shape of the input image to image_shapr variable image_shape = (height, width) #Loading the classes and the anchor boxes that are provided in the madel_data folder class_names = read_classes("model_data/coco_classes.txt") anchors = read_anchors("model_data/yolo_anchors.txt") #Load the pretrained model. Please refer the README file to get info on how to obtain the yolo.h5 file yolo_model = load_model("model_data/yolo.h5") #Print the summery of the model yolo_model.summary() #Convert final layer features to bounding box parameters yolo_outputs = yolo_head(yolo_model.output, anchors, len(class_names)) #Now yolo_eval function selects the best boxes using filtering and non-max suppression techniques. # If you want to dive in more to see how this works, refer keras_yolo.py file in yad2k/models boxes, scores, classes = yolo_eval(yolo_outputs, image_shape) # Initiate a session sess = K.get_session() #Preprocess the input image before feeding into the convolutional network image, image_data = preprocess_image("images/" + input_image_name, model_image_size = (608, 608)) #Run the session out_scores, out_boxes, out_classes = sess.run([scores, boxes, classes],feed_dict={yolo_model.input:image_data,K.learning_phase(): 0}) #Print the results print('Found {} boxes for {}'.format(len(out_boxes), input_image_name)) #Produce the colors for the bounding boxs colors = generate_colors(class_names) #Draw the bounding boxes draw_boxes(image, out_scores, out_boxes, out_classes, class_names, colors) #Apply the predicted bounding boxes to the image and save it image.save(os.path.join("out", input_image_name), quality=90) output_image = scipy.misc.imread(os.path.join("out", input_image_name)) imshow(output_image) ================================================ FILE: yolo_utils.py ================================================ import colorsys import imghdr import os import random from keras import backend as K import numpy as np from PIL import Image, ImageDraw, ImageFont def read_classes(classes_path): with open(classes_path) as f: class_names = f.readlines() class_names = [c.strip() for c in class_names] return class_names def read_anchors(anchors_path): with open(anchors_path) as f: anchors = f.readline() anchors = [float(x) for x in anchors.split(',')] anchors = np.array(anchors).reshape(-1, 2) return anchors def generate_colors(class_names): hsv_tuples = [(x / len(class_names), 1., 1.) for x in range(len(class_names))] colors = list(map(lambda x: colorsys.hsv_to_rgb(*x), hsv_tuples)) colors = list(map(lambda x: (int(x[0] * 255), int(x[1] * 255), int(x[2] * 255)), colors)) random.seed(10101) # Fixed seed for consistent colors across runs. random.shuffle(colors) # Shuffle colors to decorrelate adjacent classes. random.seed(None) # Reset seed to default. return colors def scale_boxes(boxes, image_shape): """ Scales the predicted boxes in order to be drawable on the image""" height = image_shape[0] width = image_shape[1] image_dims = K.stack([height, width, height, width]) image_dims = K.reshape(image_dims, [1, 4]) boxes = boxes * image_dims return boxes def preprocess_image(img_path, model_image_size): image_type = imghdr.what(img_path) image = Image.open(img_path) resized_image = image.resize(tuple(reversed(model_image_size)), Image.BICUBIC) image_data = np.array(resized_image, dtype='float32') image_data /= 255. image_data = np.expand_dims(image_data, 0) # Add batch dimension. return image, image_data def draw_boxes(image, out_scores, out_boxes, out_classes, class_names, colors): font = ImageFont.truetype(font='font/FiraMono-Medium.otf',size=np.floor(3e-2 * image.size[1] + 0.5).astype('int32')) thickness = (image.size[0] + image.size[1]) // 300 for i, c in reversed(list(enumerate(out_classes))): predicted_class = class_names[c] box = out_boxes[i] score = out_scores[i] label = '{} {:.2f}'.format(predicted_class, score) draw = ImageDraw.Draw(image) label_size = draw.textsize(label, font) top, left, bottom, right = box top = max(0, np.floor(top + 0.5).astype('int32')) left = max(0, np.floor(left + 0.5).astype('int32')) bottom = min(image.size[1], np.floor(bottom + 0.5).astype('int32')) right = min(image.size[0], np.floor(right + 0.5).astype('int32')) print(label, (left, top), (right, bottom)) if top - label_size[1] >= 0: text_origin = np.array([left, top - label_size[1]]) else: text_origin = np.array([left, top + 1]) # My kingdom for a good redistributable image drawing library. for i in range(thickness): draw.rectangle([left + i, top + i, right - i, bottom - i], outline=colors[c]) draw.rectangle([tuple(text_origin), tuple(text_origin + label_size)], fill=colors[c]) draw.text(text_origin, label, fill=(0, 0, 0), font=font) del draw