[
  {
    "path": "Attack/README.md",
    "content": "## Preparing an attack\n\n1. First, make a full-face photo of attacked person, a full-face photo in a hat, and a \nfull-face photo in a hat with an example sticker on the hat. To be sure that you use a\nsticker with the correct size follow instructions:\n    1. Download an example.png.\n    2. Open downloaded image with standard Windows print util.\n    3. Choose the regime with 4 photos per page (9 x 13 sm).\n    4. Uncheck the box \"Fit picture to frame\".\n    5. Print page with an example sticker.\n    6. Cut out the sticker and put in on the hat.\n    \n2. Use the next command to prepare photos:\n\n`python3 face_preparation.py PATH_TO_THE_IMAGE`\n\n3. You need to find parameters for the sticker position initialization. Use the next \ncommand to find these parameters:\n\n`python3 face_preparation.py PATH_TO_THE_IMAGE_WITH_HAT_ONLY --mask`\n\nIt will show sticker placement with default parameters. Change parameters until the \nimage looks like a prepared image with the sticker. You can see the parameters using `--help`\nflag.\n\n4. Download TensorFlow ArcFace model \n[here](https://drive.google.com/file/d/1fb70KgMRSmaEUF5cJ67BCD_DmTPCR5uJ/view?usp=sharing).\n\n5. Launch an attack preparation:\n\n`python3 attack.py PATH_TO_THE_PREPARED_IMAGED_WITH_HAT PATH_TO_THE_TF_MODEL --anchor_face \nPATH_TO_THE_PREPARED_IMAGE_WITHOUT_HAT (sticker position parameters in the same format from\nthe third step)`\n\n6. Print the obtained sticker, put it on the hat as before, and make a new photo with the sticker.\n\n7. Use \"face_preparation.py\" again to prepare a new photo and \"cos_tf.py\" to calculate a new similarity.\n\n`python3 cos_tf.py PATH_TO_THE_PREPARED_IMAGE_WITHOUT_HAT PATH_TO_THE_PREPARED_IMAGE_WITH_HAT_ONLY` - baseline similarity\n\n\n`python3 cos_tf.py PATH_TO_THE_PREPARED_IMAGE_WITHOUT_HAT PATH_TO_THE_PREPARED_IMAGE_WITH_THE_NEW_STICKER` - final similarity\n\n### Notes\n\nNote that our printer has good color rendering, that is why NPS-loss does not make influence in our experiments.\nYou may need to add NPS-loss for your printer.\n"
  },
  {
    "path": "Attack/attack.py",
    "content": "import argparse\nimport sys\nimport os\nimport tensorflow as tf\nimport numpy as np\nimport skimage.io as io\nfrom skimage.transform import rescale\nfrom tqdm import tqdm\nfrom stn import spatial_transformer_network as stn\nfrom utils import TVloss, projector\nfrom sklearn.linear_model import LinearRegression as LR\nfrom time import time\nimport datetime\nimport matplotlib.pyplot as plt\n\n# Prepare image to network input format\ndef prep(im):\n    if len(im.shape)==3:\n        return np.transpose(im,[2,0,1]).reshape((1,3,112,112))*2-1\n    elif len(im.shape)==4:\n        return np.transpose(im,[0,3,1,2]).reshape((im.shape[0],3,112,112))*2-1\n\ndef main(args):\n        print(args)\n        now = str(datetime.datetime.now())\n        \n        sess = tf.Session()\n        \n        # Off-plane sticker projection\n        logo = tf.placeholder(tf.float32,shape=[None,400,900,3],name='logo_input')\n        param = tf.placeholder(tf.float32,shape=[None,1],name='param_input')\n        ph = tf.placeholder(tf.float32,shape=[None,1],name='ph_input')\n        result = projector(param,ph,logo)\n\n        # Union of the sticker and face image\n        mask_input = tf.placeholder(tf.float32,shape=[None,900,900,3],name='mask_input')\n        face_input = tf.placeholder(tf.float32,shape=[None,600,600,3],name='face_input')\n        theta = tf.placeholder(tf.float32,shape=[None,6],name='theta_input')\n        prepared = stn(result,theta)\n\t\t\n\t\t# Transformation to ArcFace template\n        theta2 = tf.placeholder(tf.float32,shape=[None,6],name='theta2_input')\n        united = prepared[:,300:,150:750]*mask_input[:,300:,150:750]+\\\n                                        face_input*(1-mask_input[:,300:,150:750])\n        final_crop = tf.clip_by_value(stn(united,theta2,(112,112)),0.,1.)\n        \n        # TV loss and gradients\n        w_tv = tf.placeholder(tf.float32,name='w_tv_input')\n        tv_loss = TVloss(logo,w_tv)\n\n        grads_tv = tf.gradients(tv_loss,logo)\n        grads_input = tf.placeholder(tf.float32,shape=[None,112,112,3],name='grads_input')\n        grads1 = tf.gradients(final_crop,logo,grad_ys=grads_input)\n        \n        # Varios images generator\n        class Imgen(object):\n                def __init__(self):\n                        self.fdict = {ph:[[args.ph]],\\\n                                                  logo:np.ones((1,400,900,3)),\\\n                                                  param:[[args.param]],\\\n                                                  theta:1./args.scale*np.array([[1.,0.,-args.x/450.,0.,1.,-args.y/450.]]),\\\n                                                  theta2:[[1.,0.,0.,0.,1.,0.]],\\\n                                                  w_tv:args.w_tv}\n                        mask = sess.run(prepared,feed_dict=self.fdict)\n                        self.fdict[mask_input] = mask\n                        \n                def gen_fixed(self,im,advhat):\n                        self.fdict[face_input] = np.expand_dims(im,0)\n                        self.fdict[logo] = np.expand_dims(advhat,0)\n                        return self.fdict, sess.run(final_crop,feed_dict=self.fdict)\n                \n                def gen_random(self,im,advhat,batch=args.batch_size):\n                        alpha1 = np.random.uniform(-1.,1.,size=(batch,1))/180.*np.pi\n                        scale1 = np.random.uniform(args.scale-0.02,args.scale+0.02,size=(batch,1))\n                        y1 = np.random.uniform(args.y-600./112.,args.y+600./112.,size=(batch,1))\n                        x1 = np.random.uniform(args.x-600./112.,args.x+600./112.,size=(batch,1))\n                        alpha2 = np.random.uniform(-1.,1.,size=(batch,1))/180.*np.pi\n                        scale2 = np.random.uniform(1./1.04,1.04,size=(batch,1))\n                        y2 = np.random.uniform(-1.,1.,size=(batch,1))/66.\n                        angle = np.random.uniform(args.ph-2.,args.ph+2.,size=(batch,1))\n                        parab = np.random.uniform(args.param-0.0002,args.param+0.0002,size=(batch,1))\n                        fdict = {ph:angle,param:parab,w_tv:args.w_tv,\\\n                                        theta:1./scale1*np.hstack([np.cos(alpha1),np.sin(alpha1),-x1/450.,\\\n                                                                                           -np.sin(alpha1),np.cos(alpha1),-y1/450.]),\\\n                                        theta2:scale2*np.hstack([np.cos(alpha2),np.sin(alpha2),np.zeros((batch,1)),\\\n                                                                                        -np.sin(alpha2),np.cos(alpha2),y2]),\\\n                                        logo:np.ones((batch,400,900,3)),\\\n                                        face_input:np.tile(np.expand_dims(im,0),[batch,1,1,1])}\n                        mask = sess.run(prepared,feed_dict=fdict)\n                        fdict[mask_input] = mask\n                        fdict[logo] = np.tile(np.expand_dims(advhat,0),[batch,1,1,1])\n                        return fdict, sess.run(final_crop,feed_dict=fdict)\n                        \n        gener = Imgen()\n\n        # Initialization of the sticker\n        init_logo = np.ones((400,900,3))*127./255.\n        if args.init_face!=None:\n                init_face = io.imread(args.init_face)/255.\n                init_loss = tv_loss+tf.reduce_sum(tf.abs(init_face-united[0]))\n                init_grads = tf.gradients(init_loss,logo)\n                init_logo = np.ones((400,900,3))*127./255.\n                fdict, _ = gener.gen_fixed(init_face,init_logo)\n                moments = np.zeros((400,900,3))\n                print('Initialization from face, step 1/2')\n                for i in tqdm(range(500)):\n                        fdict[logo] = np.expand_dims(init_logo,0)\n                        grads = moments*0.9+sess.run(init_grads,feed_dict=fdict)[0][0]\n                        moments = moments*0.9 + grads*0.1\n                        init_logo = np.clip(init_logo-1./51.*np.sign(grads),0.,1.)\n                print('Initialization from face, step 2/2')\n                for i in tqdm(range(500)):\n                        fdict[logo] = np.expand_dims(init_logo,0)\n                        grads = moments*0.9+sess.run(init_grads,feed_dict=fdict)[0][0]\n                        moments = moments*0.9 + grads*0.1\n                        init_logo = np.clip(init_logo-1./255.*np.sign(grads),0.,1.)\n                io.imsave(now+'_init_logo.png',init_logo)\n        elif args.init_logo!=None:\n                init_logo[:] = io.imread(args.init_logo)/255.\n                \n                \n        # Embedding model\n        with tf.gfile.GFile(args.model, \"rb\") as f:\n                graph_def = tf.GraphDef()\n                graph_def.ParseFromString(f.read())\n        tf.import_graph_def(graph_def,\n                                          input_map=None,\n                                          return_elements=None,\n                                          name=\"\")\n        image_input = tf.get_default_graph().get_tensor_by_name('image_input:0')\n        keep_prob = tf.get_default_graph().get_tensor_by_name('keep_prob:0')\n        is_train = tf.get_default_graph().get_tensor_by_name('training_mode:0')\n        embedding = tf.get_default_graph().get_tensor_by_name('embedding:0')\n\n        orig_emb = tf.placeholder(tf.float32,shape=[None,512],name='orig_emb_input')\n        cos_loss = tf.reduce_sum(tf.multiply(embedding,orig_emb),axis=1)\n        grads2 = tf.gradients(cos_loss,image_input)\n\n        fdict2 = {keep_prob:1.0,is_train:False}\n        \n        # Anchor embedding calculation\n        if args.anchor_face!=None:\n                anch_im = rescale(io.imread(args.anchor_face)/255.,112./600.,order=5)\n                fdict2[image_input] = prep(anch_im)\n                fdict2[orig_emb] = sess.run(embedding,feed_dict=fdict2)\n        elif args.anchor_emb!=None:\n                fdict2[orig_emb] = np.load(args.anchor_emb)[-1:]\n        else:\n                anch_im = rescale(io.imread(args.image)/255.,112./600.,order=5)\n                fdict2[image_input] = prep(anch_im)\n                fdict2[orig_emb] = sess.run(embedding,feed_dict=fdict2)\n        \n        # Attack constants\n        im0 = io.imread(args.image)/255.\n        regr = LR(n_jobs=4)\n        regr_len = 100\n        regr_coef = -1.\n        moments = np.zeros((400,900,3))\n        moment_val = 0.9\n        step_val = 1./51.\n        stage = 1\n        step = 0\n        lr_thresh = 100\n        ls = []\n        t = time()\n        while True:\n                # Projecting sticker to the face and feeding it to the embedding model\n                fdict,ims = gener.gen_random(im0,init_logo)\n                fdict2[image_input] = prep(ims)\n                grad_tmp = sess.run(grads2,feed_dict=fdict2)\n                \n                fdict_val, im_val = gener.gen_fixed(im0,init_logo)\n                fdict2[image_input] = prep(im_val)\n                ls.append(sess.run(cos_loss,feed_dict=fdict2)[0])\n                \n                # Gradients to the original sticker image\n                fdict[grads_input] = np.transpose(grad_tmp[0],[0,2,3,1])\n                grads_on_logo = np.mean(sess.run(grads1,feed_dict=fdict)[0],0)\n                grads_on_logo += sess.run(grads_tv,feed_dict=fdict)[0][0]\n                moments = moments*moment_val + grads_on_logo*(1.-moment_val)\n                init_logo -= step_val*np.sign(moments)\n                init_logo = np.clip(init_logo,0.,1.)\n                \n                # Logging\n                step += 1\n                if step%20==0:\n                        print('Stage:',stage,'Step:',step,'Av. time:',round((time()-t)/step,2),'Loss:',round(ls[-1],2),'Coef:',regr_coef)\n\n                # Switching to the second stage\n                if step>lr_thresh:\n                        regr.fit(np.expand_dims(np.arange(100),1),np.hstack(ls[-100:]))\n                        regr_coef = regr.coef_[0]\n                        if regr_coef>=0:\n                                if stage==1:\n                                        stage = 2\n                                        moment_val = 0.995\n                                        step_val = 1./255.\n                                        step = 0\n                                        regr_coef = -1.\n                                        lr_thresh = 200\n                                        t = time()\n                                else:\n                                        break\n\n        plt.plot(range(len(ls)),ls)\n        plt.savefig(now+'_cosine.png')\n        io.imsave(now+'_advhat.png',init_logo)\n                   \ndef parse_arguments(argv):\n    parser = argparse.ArgumentParser()\n    \n    parser.add_argument('image', type=str, help='Path to the image for attack.')\n    parser.add_argument('model', type=str, help='Path to the model for attack.')\n    parser.add_argument('--init_face', type=str, default=None, help='Path to the face for sticker inititalization.')\n    parser.add_argument('--init_logo', type=str, default=None, help='Path to the image for inititalization.')\n    parser.add_argument('--anchor_face', type=str, default=None, help='Path to the anchor face.')\n    parser.add_argument('--anchor_emb', type=str, default=None, help='Path to the anchor emb (the last will be used)')\n    parser.add_argument('--w_tv', type=float, default=1e-4, help='Weight of the TV loss')\n    parser.add_argument('--ph', type=float, default=17., help='Angle of the off-plane rotation')\n    parser.add_argument('--param', type=float, default=0.0013, help='Parabola rate for the off-plane parabolic transformation')\n    parser.add_argument('--scale', type=float, default=0.465, help='Scaling parameter for the sticker')\n    parser.add_argument('--x', type=float, default=0., help='Translation of the sticker along x-axis')\n    parser.add_argument('--y', type=float, default=-15., help='Translation of the sticker along y-axis')\n    parser.add_argument('--batch_size', type=int, default=20, help='Batch size for attack')\n    \n    return parser.parse_args(argv)\n\nif __name__ == '__main__':\n    main(parse_arguments(sys.argv[1:]))\n"
  },
  {
    "path": "Attack/cos_mx.py",
    "content": "import argparse\nimport sys\nimport mxnet as mx\nimport mxnet.ndarray as nd\nimport numpy as np\nimport skimage.io as io\nfrom skimage.transform import rescale\nfrom numpy import linalg as LA\n\n# Prepare image to network input format\ndef prep(im):\n    if len(im.shape)==3:\n        return np.transpose(im,[2,0,1]).reshape((1,3,112,112))\n    elif len(im.shape)==4:\n        return np.transpose(im,[0,3,1,2]).reshape((im.shape[0],3,112,112))\n\ndef main(args):\n        print(args)\n        \n        # Embedding model\n        sym, arg_params, aux_params = mx.model.load_checkpoint(args.model, 0)\n        sym = sym.get_internals()['fc1_output']\n        model = mx.mod.Module(symbol=sym, context=mx.gpu(0), label_names = None)\n        model.bind(data_shapes=[('data', (1, 3, 112, 112))])\n        model.set_params(arg_params, aux_params)\n        \n        # Embedding calculation\n        im1 = (prep(rescale(io.imread(args.face1)/255.,112./600.,order=5))*255.).astype(np.uint8)\n        im2 = (prep(rescale(io.imread(args.face2)/255.,112./600.,order=5))*255.).astype(np.uint8)\n        \n        batch = mx.io.DataBatch(data=[nd.array(im1)])\n        model.forward(batch, is_train=False)\n        emb1 = model.get_outputs()[0].asnumpy()[0]\n        batch = mx.io.DataBatch(data=[nd.array(im2)])\n        model.forward(batch, is_train=False)\n        emb2 = model.get_outputs()[0].asnumpy()[0]\n\n        # Normalization\n        emb1 /= LA.norm(emb1)\n        emb2 /= LA.norm(emb2)\n        cos_sim = np.sum(emb1 * emb2)\n\n        # Result\n        print('Cos_sim(face1, face2) =', cos_sim) \n                   \ndef parse_arguments(argv):\n    parser = argparse.ArgumentParser()\n    \n    parser.add_argument('face1', type=str, help='Path to the preprocessed face1.')\n    parser.add_argument('face2', type=str, help='Path to the preprocessed face2.')\n    parser.add_argument('model', type=str, help='Path to the model.')\n    \n    return parser.parse_args(argv)\n\nif __name__ == '__main__':\n    main(parse_arguments(sys.argv[1:]))\n"
  },
  {
    "path": "Attack/cos_tf.py",
    "content": "import argparse\nimport sys\nimport tensorflow as tf\nimport numpy as np\nimport skimage.io as io\nfrom skimage.transform import rescale\n\n# Prepare image to network input format\ndef prep(im):\n    if len(im.shape)==3:\n        return np.transpose(im,[2,0,1]).reshape((1,3,112,112))*2-1\n    elif len(im.shape)==4:\n        return np.transpose(im,[0,3,1,2]).reshape((im.shape[0],3,112,112))*2-1\n\ndef main(args):\n        print(args)\n        \n        sess = tf.Session()\n        \n        # Embedding model\n        with tf.gfile.GFile(args.model, \"rb\") as f:\n                graph_def = tf.GraphDef()\n                graph_def.ParseFromString(f.read())\n        tf.import_graph_def(graph_def,\n                                          input_map=None,\n                                          return_elements=None,\n                                          name=\"\")\n        image_input = tf.get_default_graph().get_tensor_by_name('image_input:0')\n        keep_prob = tf.get_default_graph().get_tensor_by_name('keep_prob:0')\n        is_train = tf.get_default_graph().get_tensor_by_name('training_mode:0')\n        embedding = tf.get_default_graph().get_tensor_by_name('embedding:0')\n\n        tfdict = {keep_prob:1.0, is_train:False}\n        \n        # Embedding calculation\n        im1 = prep(rescale(io.imread(args.face1)/255.,112./600.,order=5))\n        im2 = prep(rescale(io.imread(args.face2)/255.,112./600.,order=5))\n        tfdict[image_input] = im1\n        emb1 = sess.run(embedding,feed_dict=tfdict)\n        tfdict[image_input] = im2\n        emb2 = sess.run(embedding,feed_dict=tfdict)\n\n        # Result\n        cos_sim = np.sum(emb1 * emb2)\n        print('Cos_sim(face1, face2) =', cos_sim) \n                   \ndef parse_arguments(argv):\n    parser = argparse.ArgumentParser()\n    \n    parser.add_argument('face1', type=str, help='Path to the preprocessed face1.')\n    parser.add_argument('face2', type=str, help='Path to the preprocessed face2.')\n    parser.add_argument('model', type=str, help='Path to the model.')\n    \n    return parser.parse_args(argv)\n\nif __name__ == '__main__':\n    main(parse_arguments(sys.argv[1:]))\n"
  },
  {
    "path": "Attack/face_preparation.py",
    "content": "import argparse\nimport os\nimport sys\nsys.path.append(os.path.join(os.path.dirname(__file__), '../Demo/'))\nimport tensorflow as tf\nimport numpy as np\nimport cv2\nimport skimage.io as io\nfrom skimage import transform as trans\nfrom align import detect_face\nfrom stn import spatial_transformer_network as stn\nfrom utils import projector\n\n# Align face as ArcFace template\ndef preprocess(img, landmark):\n    image_size = [600,600]\n    src = 600./112.*np.array([\n\t\t[38.2946, 51.6963],\n\t\t[73.5318, 51.5014],\n\t\t[56.0252, 71.7366],\n\t\t[41.5493, 92.3655],\n\t\t[70.7299, 92.2041] ], dtype=np.float32)\n    dst = landmark.astype(np.float32)\n    tform = trans.SimilarityTransform()\n    tform.estimate(dst, src)\n    M = tform.params[0:2,:]\n\n    warped = cv2.warpAffine(img,M,(image_size[1],image_size[0]), borderValue = 0.0)\n    return warped\n\ndef main(args):\n\tsess = tf.Session()\n\tpnet, rnet, onet = detect_face.create_mtcnn(sess, None)\n\tthreshold = [ 0.6, 0.7, 0.7 ]\n\tfactor = 0.709\n\n\timg = io.imread(args.image)\n\t_minsize = min(min(img.shape[0]//5, img.shape[1]//5),80)\n\tbounding_boxes, points = detect_face.detect_face(img, _minsize, pnet, rnet, onet, threshold, factor)\n\tassert bounding_boxes.size>0\n\tpoints = points[:, 0]\n\tlandmark = points.reshape((2,5)).T\n\twarped = preprocess(img, landmark)\n\n\tio.imsave(args.image[:-4]+'_aligned.png',warped)\n\n\tif args.mask:\n\t\tlogo_mask = np.ones((1,400,900,3),dtype=np.float32)\n\n\t\tlogo = tf.placeholder(tf.float32,shape=[1,400,900,3])\n\t\tparam = tf.placeholder(tf.float32,shape=[1,1])\n\t\tph = tf.placeholder(tf.float32,shape=[1,1])\n\t\tresult = projector(param,ph,logo)\n\n\t\tface_input = tf.placeholder(tf.float32,shape=[1,600,600,3])\n\t\ttheta = tf.placeholder(tf.float32,shape=[1,6])\n\t\tprepared = stn(result,theta)\n\n\t\tunited = prepared[:,300:,150:750]+face_input*(1-prepared[:,300:,150:750])\n\n\t\timg_with_mask = sess.run(united,feed_dict={ph:[[args.ph]],logo:logo_mask,param:[[args.param]],\\\n\t\t\t\t\t\t\t\t\t\tface_input:np.expand_dims(warped/255.,0),\\\n\t\t\t\t\t\t\t\t\t\ttheta:1./args.scale*np.array([[1.,0.,-args.x/450.,0.,1.,-args.y/450.]])})[0]\n\n\t\tio.imsave(args.image[:-4]+'_mask.png',img_with_mask)\n\t\ndef parse_arguments(argv):\n    parser = argparse.ArgumentParser()\n    \n    parser.add_argument('image', type=str, help='Path to the image.')\n    parser.add_argument('--mask', action='store_true', help='Use when search the sticker parameters')\n    parser.add_argument('--ph', type=float, default=17., help='Angle of the off-plane rotation')\n    parser.add_argument('--param', type=float, default=0.0013, help='Parabola rate for the off-plane parabolic transformation')\n    parser.add_argument('--scale', type=float, default=0.465, help='Scaling parameter for the sticker')\n    parser.add_argument('--x', type=float, default=0., help='Translation of the sticker along x-axis')\n    parser.add_argument('--y', type=float, default=-15., help='Translation of the sticker along y-axis')\n    return parser.parse_args(argv)\n\nif __name__ == '__main__':\n    main(parse_arguments(sys.argv[1:]))\n\n"
  },
  {
    "path": "Attack/stn.py",
    "content": "import tensorflow as tf\n\n\ndef spatial_transformer_network(input_fmap, theta, out_dims=None, **kwargs):\n    \"\"\"\n    Spatial Transformer Network layer implementation as described in [1].\n\n    The layer is composed of 3 elements:\n\n    - localization_net: takes the original image as input and outputs\n      the parameters of the affine transformation that should be applied\n      to the input image.\n\n    - affine_grid_generator: generates a grid of (x,y) coordinates that\n      correspond to a set of points where the input should be sampled\n      to produce the transformed output.\n\n    - bilinear_sampler: takes as input the original image and the grid\n      and produces the output image using bilinear interpolation.\n\n    Input\n    -----\n    - input_fmap: output of the previous layer. Can be input if spatial\n      transformer layer is at the beginning of architecture. Should be\n      a tensor of shape (B, H, W, C).\n\n    - theta: affine transform tensor of shape (B, 6). Permits cropping,\n      translation and isotropic scaling. Initialize to identity matrix.\n      It is the output of the localization network.\n\n    Returns\n    -------\n    - out_fmap: transformed input feature map. Tensor of size (B, H, W, C).\n\n    Notes\n    -----\n    [1]: 'Spatial Transformer Networks', Jaderberg et. al,\n         (https://arxiv.org/abs/1506.02025)\n\n    \"\"\"\n    # grab input dimensions\n    B = tf.shape(input_fmap)[0]\n    H = tf.shape(input_fmap)[1]\n    W = tf.shape(input_fmap)[2]\n\n    # reshape theta to (B, 2, 3)\n    theta = tf.reshape(theta, [B, 2, 3])\n\n    # generate grids of same size or upsample/downsample if specified\n    if out_dims:\n        out_H = out_dims[0]\n        out_W = out_dims[1]\n        batch_grids = affine_grid_generator(out_H, out_W, theta)\n    else:\n        batch_grids = affine_grid_generator(H, W, theta)\n\n    x_s = batch_grids[:, 0, :, :]\n    y_s = batch_grids[:, 1, :, :]\n\n    # sample input with grid to get output\n    out_fmap = bilinear_sampler(input_fmap, x_s, y_s)\n\n    return out_fmap\n\n\ndef get_pixel_value(img, x, y):\n    \"\"\"\n    Utility function to get pixel value for coordinate\n    vectors x and y from a  4D tensor image.\n\n    Input\n    -----\n    - img: tensor of shape (B, H, W, C)\n    - x: flattened tensor of shape (B*H*W,)\n    - y: flattened tensor of shape (B*H*W,)\n\n    Returns\n    -------\n    - output: tensor of shape (B, H, W, C)\n    \"\"\"\n    shape = tf.shape(x)\n    batch_size = shape[0]\n    height = shape[1]\n    width = shape[2]\n\n    batch_idx = tf.range(0, batch_size)\n    batch_idx = tf.reshape(batch_idx, (batch_size, 1, 1))\n    b = tf.tile(batch_idx, (1, height, width))\n\n    indices = tf.stack([b, y, x], 3)\n\n    return tf.gather_nd(img, indices)\n\n\ndef affine_grid_generator(height, width, theta):\n    \"\"\"\n    This function returns a sampling grid, which when\n    used with the bilinear sampler on the input feature\n    map, will create an output feature map that is an\n    affine transformation [1] of the input feature map.\n\n    Input\n    -----\n    - height: desired height of grid/output. Used\n      to downsample or upsample.\n\n    - width: desired width of grid/output. Used\n      to downsample or upsample.\n\n    - theta: affine transform matrices of shape (num_batch, 2, 3).\n      For each image in the batch, we have 6 theta parameters of\n      the form (2x3) that define the affine transformation T.\n\n    Returns\n    -------\n    - normalized grid (-1, 1) of shape (num_batch, 2, H, W).\n      The 2nd dimension has 2 components: (x, y) which are the\n      sampling points of the original image for each point in the\n      target image.\n\n    Note\n    ----\n    [1]: the affine transformation allows cropping, translation,\n         and isotropic scaling.\n    \"\"\"\n    num_batch = tf.shape(theta)[0]\n\n    # create normalized 2D grid\n    x = tf.linspace(-1.0, 1.0, width)\n    y = tf.linspace(-1.0, 1.0, height)\n    x_t, y_t = tf.meshgrid(x, y)\n\n    # flatten\n    x_t_flat = tf.reshape(x_t, [-1])\n    y_t_flat = tf.reshape(y_t, [-1])\n\n    # reshape to [x_t, y_t , 1] - (homogeneous form)\n    ones = tf.ones_like(x_t_flat)\n    sampling_grid = tf.stack([x_t_flat, y_t_flat, ones])\n\n    # repeat grid num_batch times\n    sampling_grid = tf.expand_dims(sampling_grid, axis=0)\n    sampling_grid = tf.tile(sampling_grid, tf.stack([num_batch, 1, 1]))\n\n    # cast to float32 (required for matmul)\n    theta = tf.cast(theta, 'float32')\n    sampling_grid = tf.cast(sampling_grid, 'float32')\n\n    # transform the sampling grid - batch multiply\n    batch_grids = tf.matmul(theta, sampling_grid)\n    # batch grid has shape (num_batch, 2, H*W)\n\n    # reshape to (num_batch, H, W, 2)\n    batch_grids = tf.reshape(batch_grids, [num_batch, 2, height, width])\n\n    return batch_grids\n\n\ndef bilinear_sampler(img, x, y):\n    \"\"\"\n    Performs bilinear sampling of the input images according to the\n    normalized coordinates provided by the sampling grid. Note that\n    the sampling is done identically for each channel of the input.\n\n    To test if the function works properly, output image should be\n    identical to input image when theta is initialized to identity\n    transform.\n\n    Input\n    -----\n    - img: batch of images in (B, H, W, C) layout.\n    - grid: x, y which is the output of affine_grid_generator.\n\n    Returns\n    -------\n    - out: interpolated images according to grids. Same size as grid.\n    \"\"\"\n    H = tf.shape(img)[1]\n    W = tf.shape(img)[2]\n    max_y = tf.cast(H - 1, 'int32')\n    max_x = tf.cast(W - 1, 'int32')\n    zero = tf.zeros([], dtype='int32')\n\n    # rescale x and y to [0, W-1/H-1]\n    x = tf.cast(x, 'float32')\n    y = tf.cast(y, 'float32')\n    x = 0.5 * ((x + 1.0) * tf.cast(max_x-1, 'float32'))\n    y = 0.5 * ((y + 1.0) * tf.cast(max_y-1, 'float32'))\n\n    # grab 4 nearest corner points for each (x_i, y_i)\n    x0 = tf.cast(tf.floor(x), 'int32')\n    x1 = x0 + 1\n    y0 = tf.cast(tf.floor(y), 'int32')\n    y1 = y0 + 1\n\n    # clip to range [0, H-1/W-1] to not violate img boundaries\n    x0 = tf.clip_by_value(x0, zero, max_x)\n    x1 = tf.clip_by_value(x1, zero, max_x)\n    y0 = tf.clip_by_value(y0, zero, max_y)\n    y1 = tf.clip_by_value(y1, zero, max_y)\n\n    # get pixel value at corner coords\n    Ia = get_pixel_value(img, x0, y0)\n    Ib = get_pixel_value(img, x0, y1)\n    Ic = get_pixel_value(img, x1, y0)\n    Id = get_pixel_value(img, x1, y1)\n\n    # recast as float for delta calculation\n    x0 = tf.cast(x0, 'float32')\n    x1 = tf.cast(x1, 'float32')\n    y0 = tf.cast(y0, 'float32')\n    y1 = tf.cast(y1, 'float32')\n\n    # calculate deltas\n    wa = (x1-x) * (y1-y)\n    wb = (x1-x) * (y-y0)\n    wc = (x-x0) * (y1-y)\n    wd = (x-x0) * (y-y0)\n\n    # add dimension for addition\n    wa = tf.expand_dims(wa, axis=3)\n    wb = tf.expand_dims(wb, axis=3)\n    wc = tf.expand_dims(wc, axis=3)\n    wd = tf.expand_dims(wd, axis=3)\n\n    # compute output\n    out = tf.add_n([wa*Ia, wb*Ib, wc*Ic, wd*Id])\n\n    return out\n"
  },
  {
    "path": "Attack/utils.py",
    "content": "import numpy as np\nimport tensorflow as tf\n\n\ndef tf_integral(x,a):\n\treturn 0.5*(x*tf.sqrt(x**2+a)+a*tf.log(tf.abs(x+tf.sqrt(x**2+a))))\ndef tf_pre_parabol(x,par):\n\tx = x-450.\n\tprev = 2.*par*(tf_integral(tf.abs(x),0.25/(par**2))-tf_integral(0,0.25/(par**2)))\n\treturn prev+450.\n\t\ndef projector(param,ph,logo):\n\t'''Apply off-plane transformations to the sticker images\n\tparam: parabola rate of the off-plane parabolic tranformation, rank 2 tensor with shape [N, 1]\n\tph:angle of the off-plane rotation, rank 2 tensor with shape [N, 1]\n\tlogo: rank 4 tensor with format NHWC and shape [N, 400, 900, 3]\n\t\n\treturn: rank 4 tensor with format NHWC and shape [N, 900, 900, 3]\n\t'''\n\tright_cumsum = tf.transpose(tf.pad(tf.cumsum(logo[:,:,450:],axis=2),tf.constant([[0,0],[0,0],[1,0],[0,0]])),[0,2,1,3])\n\tleft_cumsum = tf.transpose(tf.pad(tf.cumsum(logo[:,:,:450][:,:,::-1],axis=2),tf.constant([[0,0],[0,0],[1,0],[0,0]])),[0,2,1,3])\n\t\n\tanchors = tf.expand_dims(tf.cast(tf.round(tf.clip_by_value(\\\n\t\t\t\ttf_pre_parabol(tf.expand_dims(tf.constant(np.arange(450,901,dtype=np.float32)),0),\\\n\t\t\t\t  param)-450.,0,450.)),tf.int32),2)\n\tanch_inds = tf.tile(tf.expand_dims(tf.expand_dims(tf.range(tf.shape(param)[0]),1),2),[1,451,1])\n\tnew_anchors = tf.concat([anch_inds,anchors],2)\n\t\n\tanchors_div = tf.expand_dims(tf.cast(tf.clip_by_value(anchors[:,1:]-anchors[:,:-1],1,900),tf.float32),3)\n\tright_anchors_cumsum = tf.gather_nd(right_cumsum,new_anchors)\n\tright_anchors_diffs = right_anchors_cumsum[:,1:]-right_anchors_cumsum[:,:-1]\n\tright = right_anchors_diffs/anchors_div\n\tleft_anchors_cumsum = tf.gather_nd(left_cumsum,new_anchors)\n\tleft_anchors_diffs = left_anchors_cumsum[:,1:]-left_anchors_cumsum[:,:-1]\n\tleft = left_anchors_diffs/anchors_div\n\t\n\ttmp_result = tf.transpose(tf.concat([left[:,::-1],right],axis=1),[0,2,1,3])\n\t\n\tcumsum = tf.pad(tf.cumsum(tmp_result,axis=1),tf.constant([[0,0],[1,0],[0,0],[0,0]]))\n\t\n\tangle = tf.expand_dims(np.pi/180.*ph,2)\n\t\n\tz = param*tf.constant((np.arange(900,dtype=np.float32)-449.5)**2)\n\tz_tile = tf.tile(tf.expand_dims(z,1),tf.constant([1,901,1]))\n\t\n\ty_coord = tf.constant(np.arange(-250,651,dtype=np.float32))\n\ty_tile = tf.tile(tf.expand_dims(tf.expand_dims(y_coord,1),0),[tf.shape(param)[0],1,900])\n\t\n\ty_prev = (y_tile+z_tile*tf.sin(-angle))/tf.cos(angle)\n\ty_round = tf.cast(tf.round(tf.clip_by_value(y_prev,0,400.)),tf.int32)\n\ty_div = tf.clip_by_value(y_round[:,1:]-y_round[:,:-1],1,900)\n\t\n\tx_coord = tf.constant(np.arange(900,dtype=np.int32))\n\tx_tile = tf.tile(tf.expand_dims(tf.expand_dims(x_coord,0),0),[tf.shape(param)[0],901,1])\n\t\n\tb_coord = tf.tile(tf.expand_dims(tf.expand_dims(tf.range(tf.shape(param)[0]),1),2),[1,901,900])\n\t\n\tindices = tf.stack([b_coord,y_round,x_tile],axis=3)\n\t\n\tchosen_cumsum = tf.gather_nd(cumsum,indices)\n\tchosen_cumsum_diffs = chosen_cumsum[:,1:]-chosen_cumsum[:,:-1]\n\tfinal_results = tf.clip_by_value(chosen_cumsum_diffs/tf.expand_dims(tf.cast(y_div,tf.float32),3),0.,1.)\n\t\n\treturn final_results\n\ndef TVloss(logo,w_tv):\n\t'''Calculate TV loss of the sticker image with predefined weight.\n\tlogo: rank 4 tensor with format NHWC\n\tw_tv: weight of the TV loss\n\t\n\treturn: scalar value of the TV loss\n\t'''\n\tvert_diff = logo[:,1:]-logo[:,:-1]\n\thor_diff = logo[:,:,1:]-logo[:,:,:-1]\n\tvert_diff_sq = tf.square(vert_diff)\n\thor_diff_sq = tf.square(hor_diff)\n\tvert_pad = tf.pad(vert_diff_sq,tf.constant([[0,0],[1,0],[0,0],[0,0]]))\n\thor_pad = tf.pad(hor_diff_sq,tf.constant([[0,0],[0,0],[1,0],[0,0]]))\n\ttv_sum = vert_pad+hor_pad\n\ttv = tf.sqrt(tv_sum+1e-5)\n\ttv_final_sum = tf.reduce_sum(tv)\n\ttv_loss = w_tv*tv_final_sum\n\treturn tv_loss\n"
  },
  {
    "path": "Demo/1000_from_CASIA/centroids_names.txt",
    "content": "0000045\n0000099\n0000100\n0000102\n0000103\n0000105\n0000107\n0000108\n0000114\n0000117\n0000119\n0000121\n0000133\n0000137\n0000141\n0000143\n0000144\n0000145\n0000147\n0000156\n0000157\n0000159\n0000166\n0000168\n0000169\n0000170\n0000174\n0000177\n0000183\n0000185\n0000186\n0000188\n0000189\n0000192\n0000195\n0000198\n0000202\n0000204\n0000205\n0000207\n0000208\n0000210\n0000211\n0000212\n0000214\n0000220\n0000225\n0000233\n0000238\n0000240\n0000247\n0000249\n0000254\n0000256\n0000260\n0000262\n0000263\n0000268\n0000270\n0000271\n0000272\n0000275\n0000280\n0000281\n0000282\n0000284\n0000286\n0000287\n0000293\n0000295\n0000296\n0000297\n0000299\n0000301\n0000302\n0000304\n0000307\n0000310\n0000317\n0000318\n0000319\n0000321\n0000324\n0000327\n0000331\n0000332\n0000333\n0000334\n0000335\n0000342\n0000343\n0000344\n0000346\n0000349\n0000350\n0000351\n0000352\n0000353\n0000356\n0000357\n0000360\n0000362\n0000363\n0000364\n0000365\n0000368\n0000373\n0000374\n0000381\n0000383\n0000385\n0000386\n0000387\n0000388\n0000389\n0000391\n0000394\n0000396\n0000397\n0000399\n0000402\n0000405\n0000408\n0000410\n0000411\n0000413\n0000415\n0000420\n0000422\n0000426\n0000427\n0000430\n0000431\n0000433\n0000434\n0000436\n0000437\n0000438\n0000439\n0000442\n0000444\n0000446\n0000447\n0000448\n0000451\n0000452\n0000455\n0000457\n0000459\n0000460\n0000461\n0000462\n0000463\n0000464\n0000465\n0000467\n0000471\n0000473\n0000477\n0000480\n0000481\n0000482\n0000483\n0000484\n0000486\n0000487\n0000492\n0000494\n0000495\n0000498\n0000499\n0000500\n0000505\n0000506\n0000510\n0000512\n0000513\n0000514\n0000515\n0000520\n0000521\n0000524\n0000525\n0000526\n0000529\n0000530\n0000531\n0000532\n0000533\n0000534\n0000535\n0000538\n0000539\n0000541\n0000545\n0000546\n0000547\n0000550\n0000551\n0000552\n0000554\n0000555\n0000562\n0000563\n0000568\n0000570\n0000571\n0000574\n0000575\n0000579\n0000580\n0000582\n0000583\n0000585\n0000588\n0000589\n0000592\n0000593\n0000595\n0000596\n0000597\n0000598\n0000599\n0000600\n0000601\n0000605\n0000606\n0000607\n0000609\n0000610\n0000611\n0000612\n0000613\n0000614\n0000615\n0000616\n0000617\n0000619\n0000620\n0000622\n0000623\n0000624\n0000625\n0000628\n0000630\n0000633\n0000637\n0000642\n0000646\n0000648\n0000651\n0000652\n0000653\n0000655\n0000656\n0000657\n0000662\n0000663\n0000664\n0000665\n0000667\n0000670\n0000672\n0000675\n0000678\n0000679\n0000680\n0000688\n0000689\n0000690\n0000691\n0000693\n0000694\n0000695\n0000696\n0000700\n0000703\n0000705\n0000707\n0000708\n0000709\n0000711\n0000717\n0000725\n0000729\n0000745\n0000749\n0000751\n0000756\n0000759\n0000760\n0000775\n0000776\n0000777\n0000782\n0000792\n0000793\n0000796\n0000800\n0000801\n0000803\n0000809\n0000815\n0000816\n0000830\n0000836\n0000837\n0000838\n0000867\n0000868\n0000871\n0000874\n0000876\n0000880\n0000881\n0000884\n0000889\n0000892\n0000893\n0000902\n0000903\n0000915\n0000916\n0000928\n0000929\n0000933\n0000934\n0000943\n0000944\n0000948\n0000950\n0000952\n0000954\n0000956\n0000959\n0000960\n0000961\n0000962\n0000965\n0000968\n0000977\n0000980\n0000981\n0000982\n0000985\n0000986\n0000991\n0000996\n0000997\n0000998\n0001002\n0001004\n0001005\n0001006\n0001015\n0001018\n0001019\n0001022\n0001026\n0001029\n0001035\n0001037\n0001038\n0001039\n0001040\n0001043\n0001044\n0001046\n0001048\n0001049\n0001053\n0001054\n0001057\n0001061\n0001062\n0001063\n0001064\n0001065\n0001068\n0001069\n0001071\n0001075\n0001081\n0001082\n0001083\n0001084\n0001086\n0001089\n0001092\n0001096\n0001097\n0001099\n0001101\n0001103\n0001104\n0001107\n0001108\n0001110\n0001111\n0001114\n0001116\n0001117\n0001118\n0001126\n0001127\n0001129\n0001131\n0001138\n0001139\n0001143\n0001146\n0001147\n0001151\n0001152\n0001153\n0001154\n0001155\n0001162\n0001165\n0001166\n0001168\n0001170\n0001172\n0001176\n0001180\n0001183\n0001187\n0001194\n0001199\n0001200\n0001201\n0001208\n0001209\n0001210\n0001217\n0001218\n0001223\n0001231\n0001233\n0001235\n0001240\n0001242\n0001250\n0001251\n0001261\n0001263\n0001265\n0001266\n0001267\n0001272\n0001274\n0001277\n0001281\n0001282\n0001286\n0001287\n0001290\n0001292\n0001293\n0001295\n0001298\n0001302\n0001305\n0001307\n0001309\n0001312\n0001314\n0001315\n0001317\n0001319\n0001323\n0001324\n0001325\n0001326\n0001332\n0001334\n0001337\n0001339\n0001344\n0001346\n0001347\n0001348\n0001353\n0001356\n0001364\n0001365\n0001367\n0001368\n0001370\n0001373\n0001377\n0001378\n0001384\n0001387\n0001388\n0001389\n0001390\n0001392\n0001393\n0001398\n0001399\n0001400\n0001403\n0001404\n0001406\n0001407\n0001409\n0001410\n0001412\n0001416\n0001418\n0001424\n0001431\n0001435\n0001436\n0001438\n0001440\n0001441\n0001442\n0001444\n0001457\n0001459\n0001461\n0001462\n0001467\n0001468\n0001469\n0001472\n0001480\n0001484\n0001488\n0001491\n0001492\n0001493\n0001494\n0001496\n0001497\n0001498\n0001499\n0001504\n0001507\n0001508\n0001515\n0001517\n0001518\n0001519\n0001520\n0001521\n0001524\n0001525\n0001527\n0001529\n0001530\n0001533\n0001538\n0001540\n0001541\n0001542\n0001548\n0001550\n0001554\n0001556\n0001557\n0001558\n0001562\n0001564\n0001565\n0001567\n0001568\n0001569\n0001572\n0001573\n0001574\n0001575\n0001578\n0001582\n0001589\n0001590\n0001593\n0001594\n0001595\n0001597\n0001598\n0001599\n0001601\n0001602\n0001604\n0001605\n0001607\n0001608\n0001610\n0001614\n0001617\n0001621\n0001622\n0001624\n0001629\n0001630\n0001631\n0001632\n0001633\n0001634\n0001638\n0001640\n0001641\n0001642\n0001650\n0001652\n0001653\n0001661\n0001662\n0001664\n0001667\n0001668\n0001669\n0001670\n0001675\n0001684\n0001688\n0001694\n0001696\n0001697\n0001698\n0001701\n0001704\n0001709\n0001710\n0001711\n0001713\n0001714\n0001716\n0001720\n0001722\n0001723\n0001728\n0001729\n0001731\n0001732\n0001733\n0001735\n0001736\n0001738\n0001741\n0001742\n0001743\n0001746\n0001748\n0001752\n0001756\n0001758\n0001759\n0001760\n0001762\n0001763\n0001764\n0001767\n0001770\n0001773\n0001774\n0001776\n0001778\n0001780\n0001783\n0001785\n0001787\n0001790\n0001793\n0001794\n0001795\n0001796\n0001797\n0001802\n0001803\n0001804\n0001806\n0001808\n0001810\n0001817\n0001821\n0001823\n0001824\n0001828\n0001832\n0001833\n0001834\n0001835\n0001836\n0001837\n0001838\n0001839\n0001840\n0001841\n0001844\n0001845\n0001848\n0001851\n0001852\n0001857\n0001858\n0001863\n0001868\n0001869\n0001873\n0001877\n0001879\n0001880\n0001882\n0001912\n0001938\n0001942\n0001943\n0001951\n0001952\n0001953\n0001971\n0001973\n0001974\n0001978\n0001993\n0002003\n0002004\n0002006\n0002015\n0002023\n0002026\n0002027\n0002029\n0002033\n0002041\n0002043\n0002055\n0002059\n0002064\n0002065\n0002067\n0002073\n0002076\n0002077\n0002088\n0002090\n0002091\n0002100\n0002102\n0002105\n0002117\n0002119\n0002120\n0002122\n0002124\n0002127\n0002140\n0002142\n0002150\n0002191\n0002217\n0002239\n0002243\n0002253\n0002262\n0002325\n0002332\n0002365\n0002396\n0002436\n0002546\n0002653\n0002657\n0002700\n0002743\n0002773\n0002800\n0002801\n0002871\n0002901\n0002907\n0002913\n0002916\n0002928\n0002936\n0002944\n0002956\n0003067\n0003069\n0003071\n0003072\n0003078\n0003082\n0003115\n0003210\n0003244\n0003265\n0003289\n0003353\n0003354\n0003457\n0003494\n0003506\n0003563\n0003577\n0003620\n0003633\n0003697\n0003777\n0003779\n0003807\n0003888\n0003928\n0003931\n0003941\n0003981\n0004051\n0004056\n0004081\n0004095\n0004109\n0004133\n0004137\n0004147\n0004248\n0004266\n0004284\n0004286\n0004294\n0004303\n0004306\n0004328\n0004349\n0004371\n0004376\n0004426\n0004456\n0004518\n0004539\n0004540\n0004623\n0004645\n0004657\n0004691\n0004692\n0004700\n0004705\n0004710\n0004712\n0004715\n0004716\n0004719\n0004721\n0004724\n0004725\n0004727\n0004728\n0004729\n0004731\n0004734\n0004735\n0004736\n0004739\n0004740\n0004741\n0004743\n0004744\n0004745\n0004747\n0004748\n0004749\n0004751\n0004752\n0004753\n0004757\n0004760\n0004761\n0004763\n0004770\n0004771\n0004774\n0004775\n0004779\n0004782\n0004785\n0004786\n0004787\n0004790\n0004792\n0004793\n0004797\n0004801\n0004802\n0004804\n0004805\n0004806\n0004808\n0004809\n0004810\n0004812\n0004814\n0004818\n0004819\n0004820\n0004822\n0004825\n0004826\n0004827\n0004830\n0004834\n0004838\n0004839\n0004840\n0004841\n0004844\n0004846\n0004849\n0004850\n0004852\n0004853\n0004854\n0004857\n0004859\n0004861\n0004865\n0004866\n0004867\n0004868\n0004871\n0004875\n0004879\n0004880\n0004883\n0004884\n0004886\n0004887\n0004889\n0004892\n0004893\n0004894\n0004895\n0004897\n0004898\n0004899\n0004900\n0004904\n0004906\n0004909\n0004911\n0004912\n0004914\n0004917\n0004918\n0004921\n0004922\n0004923\n0004925\n0004928\n0004929\n0004930\n0004933\n0004936\n0004937\n0004939\n0004940\n0004941\n0004943\n0004947\n0004954\n0004955\n0004956\n0004957\n0004959\n0004960\n0004965\n0004966\n0004967\n0004968\n0004969\n0004971\n0004975\n0004977\n0004978\n0004979\n0004980\n0004981\n0004982\n0004984\n0004985\n0004986\n0004987\n0004988\n0004990\n0004991\n0004993\n0004994\n0004996\n0004999\n0005000\n0005002\n0005006\n0005007\n0005009\n0005010\n0005011\n0005012\n"
  },
  {
    "path": "Demo/README.md",
    "content": "## Demo launch\n\nArcFace@ms1m-refine-v2 transformed to TensorFlow is available [here](https://drive.google.com/file/d/1fb70KgMRSmaEUF5cJ67BCD_DmTPCR5uJ/view?usp=sharing).\n\nThe command for demo launch:\n\n`python3 demo.py PATH_TO_THE_DOWNLOADED_MODEL PATH_TO_THE_DIRECTORY_WITH_CLASS_CENTROIDS`\n\nCentroids for the first 1000 classes of CASIA are in the \"1000_from_CASIA\" directory.\n\n## Preparation of your own centroids\n\n### Alignment\n\nThe dataset of your images has to be arranged in the following way:  \n\n├── Person 1  \n│   ├── Person_1_image_1.png  \n│   ├── Person_1_image_2.png  \n│   ├── Person_1_image_3.png   \n│   └── Person_1_image_4.png  \n├── Person 2  \n│   ├── Person_2_image_1.png  \n│   ├── Person_2_image_2.png  \n│   ├── Person_2_image_3.png   \n│   ├── Person_2_image_4.png  \n│   └── Person_2_image_5.png  \n├── Person 3  \n│   ├── Person_3_image_1.png  \n│   ├── Person_3_image_2.png  \n...  \n\nThe command for images alignment:\n\n`python3 alignment.py PATH_TO_DIRECTIRY_WITH_IMAGES PATH_FOR_THE_ALIGNED_IMAGES`\n\n### Centroids calculation\n\nUsing directory with aligned images from the previous step, you can obtain centroids with the next command:\n\n`python3 dumping.py PATH_TO_DIRECTORY_WITH_ALIGNED_IMAGES PATH_FOR_THE_CENTROIDS PATH_TO_THE_DOWNLOADED_MODEL`\n"
  },
  {
    "path": "Demo/align/.gitignore",
    "content": "*.pyc\n"
  },
  {
    "path": "Demo/align/__init__.py",
    "content": ""
  },
  {
    "path": "Demo/align/detect_face.py",
    "content": "\"\"\" Tensorflow implementation of the face detection / alignment algorithm found at\nhttps://github.com/kpzhang93/MTCNN_face_detection_alignment\n\"\"\"\n# MIT License\n# \n# Copyright (c) 2016 David Sandberg\n# \n# Permission is hereby granted, free of charge, to any person obtaining a copy\n# of this software and associated documentation files (the \"Software\"), to deal\n# in the Software without restriction, including without limitation the rights\n# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n# copies of the Software, and to permit persons to whom the Software is\n# furnished to do so, subject to the following conditions:\n# \n# The above copyright notice and this permission notice shall be included in all\n# copies or substantial portions of the Software.\n# \n# THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n# SOFTWARE.\n\nfrom __future__ import absolute_import\nfrom __future__ import division\nfrom __future__ import print_function\nfrom six import string_types, iteritems\n\nimport numpy as np\nimport tensorflow as tf\n#from math import floor\nimport cv2\nimport os\n\ndef layer(op):\n    '''Decorator for composable network layers.'''\n\n    def layer_decorated(self, *args, **kwargs):\n        # Automatically set a name if not provided.\n        name = kwargs.setdefault('name', self.get_unique_name(op.__name__))\n        # Figure out the layer inputs.\n        if len(self.terminals) == 0:\n            raise RuntimeError('No input variables found for layer %s.' % name)\n        elif len(self.terminals) == 1:\n            layer_input = self.terminals[0]\n        else:\n            layer_input = list(self.terminals)\n        # Perform the operation and get the output.\n        layer_output = op(self, layer_input, *args, **kwargs)\n        # Add to layer LUT.\n        self.layers[name] = layer_output\n        # This output is now the input for the next layer.\n        self.feed(layer_output)\n        # Return self for chained calls.\n        return self\n\n    return layer_decorated\n\nclass Network(object):\n\n    def __init__(self, inputs, trainable=True):\n        # The input nodes for this network\n        self.inputs = inputs\n        # The current list of terminal nodes\n        self.terminals = []\n        # Mapping from layer names to layers\n        self.layers = dict(inputs)\n        # If true, the resulting variables are set as trainable\n        self.trainable = trainable\n\n        self.setup()\n\n    def setup(self):\n        '''Construct the network. '''\n        raise NotImplementedError('Must be implemented by the subclass.')\n\n    def load(self, data_path, session, ignore_missing=False):\n        '''Load network weights.\n        data_path: The path to the numpy-serialized network weights\n        session: The current TensorFlow session\n        ignore_missing: If true, serialized weights for missing layers are ignored.\n        '''\n        data_dict = np.load(data_path, encoding='latin1', allow_pickle=True).item() #pylint: disable=no-member\n\n        for op_name in data_dict:\n            with tf.variable_scope(op_name, reuse=True):\n                for param_name, data in iteritems(data_dict[op_name]):\n                    try:\n                        var = tf.get_variable(param_name)\n                        session.run(var.assign(data))\n                    except ValueError:\n                        if not ignore_missing:\n                            raise\n\n    def feed(self, *args):\n        '''Set the input(s) for the next operation by replacing the terminal nodes.\n        The arguments can be either layer names or the actual layers.\n        '''\n        assert len(args) != 0\n        self.terminals = []\n        for fed_layer in args:\n            if isinstance(fed_layer, string_types):\n                try:\n                    fed_layer = self.layers[fed_layer]\n                except KeyError:\n                    raise KeyError('Unknown layer name fed: %s' % fed_layer)\n            self.terminals.append(fed_layer)\n        return self\n\n    def get_output(self):\n        '''Returns the current network output.'''\n        return self.terminals[-1]\n\n    def get_unique_name(self, prefix):\n        '''Returns an index-suffixed unique name for the given prefix.\n        This is used for auto-generating layer names based on the type-prefix.\n        '''\n        ident = sum(t.startswith(prefix) for t, _ in self.layers.items()) + 1\n        return '%s_%d' % (prefix, ident)\n\n    def make_var(self, name, shape):\n        '''Creates a new TensorFlow variable.'''\n        return tf.get_variable(name, shape, trainable=self.trainable)\n\n    def validate_padding(self, padding):\n        '''Verifies that the padding is one of the supported ones.'''\n        assert padding in ('SAME', 'VALID')\n\n    @layer\n    def conv(self,\n             inp,\n             k_h,\n             k_w,\n             c_o,\n             s_h,\n             s_w,\n             name,\n             relu=True,\n             padding='SAME',\n             group=1,\n             biased=True):\n        # Verify that the padding is acceptable\n        self.validate_padding(padding)\n        # Get the number of channels in the input\n        c_i = int(inp.get_shape()[-1])\n        # Verify that the grouping parameter is valid\n        assert c_i % group == 0\n        assert c_o % group == 0\n        # Convolution for a given input and kernel\n        convolve = lambda i, k: tf.nn.conv2d(i, k, [1, s_h, s_w, 1], padding=padding)\n        with tf.variable_scope(name) as scope:\n            kernel = self.make_var('weights', shape=[k_h, k_w, c_i // group, c_o])\n            # This is the common-case. Convolve the input without any further complications.\n            output = convolve(inp, kernel)\n            # Add the biases\n            if biased:\n                biases = self.make_var('biases', [c_o])\n                output = tf.nn.bias_add(output, biases)\n            if relu:\n                # ReLU non-linearity\n                output = tf.nn.relu(output, name=scope.name)\n            return output\n\n    @layer\n    def prelu(self, inp, name):\n        with tf.variable_scope(name):\n            i = int(inp.get_shape()[-1])\n            alpha = self.make_var('alpha', shape=(i,))\n            output = tf.nn.relu(inp) + tf.multiply(alpha, -tf.nn.relu(-inp))\n        return output\n\n    @layer\n    def max_pool(self, inp, k_h, k_w, s_h, s_w, name, padding='SAME'):\n        self.validate_padding(padding)\n        return tf.nn.max_pool(inp,\n                              ksize=[1, k_h, k_w, 1],\n                              strides=[1, s_h, s_w, 1],\n                              padding=padding,\n                              name=name)\n\n    @layer\n    def fc(self, inp, num_out, name, relu=True):\n        with tf.variable_scope(name):\n            input_shape = inp.get_shape()\n            if input_shape.ndims == 4:\n                # The input is spatial. Vectorize it first.\n                dim = 1\n                for d in input_shape[1:].as_list():\n                    dim *= int(d)\n                feed_in = tf.reshape(inp, [-1, dim])\n            else:\n                feed_in, dim = (inp, input_shape[-1].value)\n            weights = self.make_var('weights', shape=[dim, num_out])\n            biases = self.make_var('biases', [num_out])\n            op = tf.nn.relu_layer if relu else tf.nn.xw_plus_b\n            fc = op(feed_in, weights, biases, name=name)\n            return fc\n\n\n    \"\"\"\n    Multi dimensional softmax,\n    refer to https://github.com/tensorflow/tensorflow/issues/210\n    compute softmax along the dimension of target\n    the native softmax only supports batch_size x dimension\n    \"\"\"\n    @layer\n    def softmax(self, target, axis, name=None):\n        max_axis = tf.reduce_max(target, axis, keep_dims=True)\n        target_exp = tf.exp(target-max_axis)\n        normalize = tf.reduce_sum(target_exp, axis, keep_dims=True)\n        softmax = tf.div(target_exp, normalize, name)\n        return softmax\n    \nclass PNet(Network):\n    def setup(self):\n        (self.feed('data') #pylint: disable=no-value-for-parameter, no-member\n             .conv(3, 3, 10, 1, 1, padding='VALID', relu=False, name='conv1')\n             .prelu(name='PReLU1')\n             .max_pool(2, 2, 2, 2, name='pool1')\n             .conv(3, 3, 16, 1, 1, padding='VALID', relu=False, name='conv2')\n             .prelu(name='PReLU2')\n             .conv(3, 3, 32, 1, 1, padding='VALID', relu=False, name='conv3')\n             .prelu(name='PReLU3')\n             .conv(1, 1, 2, 1, 1, relu=False, name='conv4-1')\n             .softmax(3,name='prob1'))\n\n        (self.feed('PReLU3') #pylint: disable=no-value-for-parameter\n             .conv(1, 1, 4, 1, 1, relu=False, name='conv4-2'))\n        \nclass RNet(Network):\n    def setup(self):\n        (self.feed('data') #pylint: disable=no-value-for-parameter, no-member\n             .conv(3, 3, 28, 1, 1, padding='VALID', relu=False, name='conv1')\n             .prelu(name='prelu1')\n             .max_pool(3, 3, 2, 2, name='pool1')\n             .conv(3, 3, 48, 1, 1, padding='VALID', relu=False, name='conv2')\n             .prelu(name='prelu2')\n             .max_pool(3, 3, 2, 2, padding='VALID', name='pool2')\n             .conv(2, 2, 64, 1, 1, padding='VALID', relu=False, name='conv3')\n             .prelu(name='prelu3')\n             .fc(128, relu=False, name='conv4')\n             .prelu(name='prelu4')\n             .fc(2, relu=False, name='conv5-1')\n             .softmax(1,name='prob1'))\n\n        (self.feed('prelu4') #pylint: disable=no-value-for-parameter\n             .fc(4, relu=False, name='conv5-2'))\n\nclass ONet(Network):\n    def setup(self):\n        (self.feed('data') #pylint: disable=no-value-for-parameter, no-member\n             .conv(3, 3, 32, 1, 1, padding='VALID', relu=False, name='conv1')\n             .prelu(name='prelu1')\n             .max_pool(3, 3, 2, 2, name='pool1')\n             .conv(3, 3, 64, 1, 1, padding='VALID', relu=False, name='conv2')\n             .prelu(name='prelu2')\n             .max_pool(3, 3, 2, 2, padding='VALID', name='pool2')\n             .conv(3, 3, 64, 1, 1, padding='VALID', relu=False, name='conv3')\n             .prelu(name='prelu3')\n             .max_pool(2, 2, 2, 2, name='pool3')\n             .conv(2, 2, 128, 1, 1, padding='VALID', relu=False, name='conv4')\n             .prelu(name='prelu4')\n             .fc(256, relu=False, name='conv5')\n             .prelu(name='prelu5')\n             .fc(2, relu=False, name='conv6-1')\n             .softmax(1, name='prob1'))\n\n        (self.feed('prelu5') #pylint: disable=no-value-for-parameter\n             .fc(4, relu=False, name='conv6-2'))\n\n        (self.feed('prelu5') #pylint: disable=no-value-for-parameter\n             .fc(10, relu=False, name='conv6-3'))\n\ndef create_mtcnn(sess, model_path):\n    if not model_path:\n        model_path,_ = os.path.split(os.path.realpath(__file__))\n\n    with tf.variable_scope('pnet'):\n        data = tf.placeholder(tf.float32, (None,None,None,3), 'input')\n        pnet = PNet({'data':data})\n        pnet.load(os.path.join(model_path, 'det1.npy'), sess)\n    with tf.variable_scope('rnet'):\n        data = tf.placeholder(tf.float32, (None,24,24,3), 'input')\n        rnet = RNet({'data':data})\n        rnet.load(os.path.join(model_path, 'det2.npy'), sess)\n    with tf.variable_scope('onet'):\n        data = tf.placeholder(tf.float32, (None,48,48,3), 'input')\n        onet = ONet({'data':data})\n        onet.load(os.path.join(model_path, 'det3.npy'), sess)\n        \n    pnet_fun = lambda img : sess.run(('pnet/conv4-2/BiasAdd:0', 'pnet/prob1:0'), feed_dict={'pnet/input:0':img})\n    rnet_fun = lambda img : sess.run(('rnet/conv5-2/conv5-2:0', 'rnet/prob1:0'), feed_dict={'rnet/input:0':img})\n    onet_fun = lambda img : sess.run(('onet/conv6-2/conv6-2:0', 'onet/conv6-3/conv6-3:0', 'onet/prob1:0'), feed_dict={'onet/input:0':img})\n    return pnet_fun, rnet_fun, onet_fun\n\ndef detect_face(img, minsize, pnet, rnet, onet, threshold, factor):\n    # im: input image\n    # minsize: minimum of faces' size\n    # pnet, rnet, onet: caffemodel\n    # threshold: threshold=[th1 th2 th3], th1-3 are three steps's threshold\n    # fastresize: resize img from last scale (using in high-resolution images) if fastresize==true\n    factor_count=0\n    total_boxes=np.empty((0,9))\n    points=[]\n    h=img.shape[0]\n    w=img.shape[1]\n    minl=np.amin([h, w])\n    m=12.0/minsize\n    minl=minl*m\n    # creat scale pyramid\n    scales=[]\n    while minl>=12:\n        scales += [m*np.power(factor, factor_count)]\n        minl = minl*factor\n        factor_count += 1\n\n    # first stage\n    for j in range(len(scales)):\n        scale=scales[j]\n        hs=int(np.ceil(h*scale))\n        ws=int(np.ceil(w*scale))\n        im_data = imresample(img, (hs, ws))\n        im_data = (im_data-127.5)*0.0078125\n        img_x = np.expand_dims(im_data, 0)\n        img_y = np.transpose(img_x, (0,2,1,3))\n        out = pnet(img_y)\n        out0 = np.transpose(out[0], (0,2,1,3))\n        out1 = np.transpose(out[1], (0,2,1,3))\n        \n        boxes, _ = generateBoundingBox(out1[0,:,:,1].copy(), out0[0,:,:,:].copy(), scale, threshold[0])\n        \n        # inter-scale nms\n        pick = nms(boxes.copy(), 0.5, 'Union')\n        if boxes.size>0 and pick.size>0:\n            boxes = boxes[pick,:]\n            total_boxes = np.append(total_boxes, boxes, axis=0)\n\n    numbox = total_boxes.shape[0]\n    if numbox>0:\n        pick = nms(total_boxes.copy(), 0.7, 'Union')\n        total_boxes = total_boxes[pick,:]\n        regw = total_boxes[:,2]-total_boxes[:,0]\n        regh = total_boxes[:,3]-total_boxes[:,1]\n        qq1 = total_boxes[:,0]+total_boxes[:,5]*regw\n        qq2 = total_boxes[:,1]+total_boxes[:,6]*regh\n        qq3 = total_boxes[:,2]+total_boxes[:,7]*regw\n        qq4 = total_boxes[:,3]+total_boxes[:,8]*regh\n        total_boxes = np.transpose(np.vstack([qq1, qq2, qq3, qq4, total_boxes[:,4]]))\n        total_boxes = rerec(total_boxes.copy())\n        total_boxes[:,0:4] = np.fix(total_boxes[:,0:4]).astype(np.int32)\n        dy,edy, dx, edx, y, ey, x, ex, tmpw, tmph = pad(total_boxes.copy(), w, h)\n\n    numbox = total_boxes.shape[0]\n    if numbox>0:\n        # second stage\n        tempimg = np.zeros((24,24,3,numbox))\n        for k in range(0,numbox):\n            tmp = np.zeros((int(tmph[k]),int(tmpw[k]),3))\n            tmp[dy[k]-1:edy[k],dx[k]-1:edx[k],:] = img[y[k]-1:ey[k],x[k]-1:ex[k],:]\n            if tmp.shape[0]>0 and tmp.shape[1]>0 or tmp.shape[0]==0 and tmp.shape[1]==0:\n                tempimg[:,:,:,k] = imresample(tmp, (24, 24))\n            else:\n                return np.empty()\n        tempimg = (tempimg-127.5)*0.0078125\n        tempimg1 = np.transpose(tempimg, (3,1,0,2))\n        out = rnet(tempimg1)\n        out0 = np.transpose(out[0])\n        out1 = np.transpose(out[1])\n        score = out1[1,:]\n        ipass = np.where(score>threshold[1])\n        total_boxes = np.hstack([total_boxes[ipass[0],0:4].copy(), np.expand_dims(score[ipass].copy(),1)])\n        mv = out0[:,ipass[0]]\n        if total_boxes.shape[0]>0:\n            pick = nms(total_boxes, 0.7, 'Union')\n            total_boxes = total_boxes[pick,:]\n            total_boxes = bbreg(total_boxes.copy(), np.transpose(mv[:,pick]))\n            total_boxes = rerec(total_boxes.copy())\n\n    numbox = total_boxes.shape[0]\n    if numbox>0:\n        # third stage\n        total_boxes = np.fix(total_boxes).astype(np.int32)\n        dy, edy, dx, edx, y, ey, x, ex, tmpw, tmph = pad(total_boxes.copy(), w, h)\n        tempimg = np.zeros((48,48,3,numbox))\n        for k in range(0,numbox):\n            tmp = np.zeros((int(tmph[k]),int(tmpw[k]),3))\n            tmp[dy[k]-1:edy[k],dx[k]-1:edx[k],:] = img[y[k]-1:ey[k],x[k]-1:ex[k],:]\n            if tmp.shape[0]>0 and tmp.shape[1]>0 or tmp.shape[0]==0 and tmp.shape[1]==0:\n                tempimg[:,:,:,k] = imresample(tmp, (48, 48))\n            else:\n                return np.empty()\n        tempimg = (tempimg-127.5)*0.0078125\n        tempimg1 = np.transpose(tempimg, (3,1,0,2))\n        out = onet(tempimg1)\n        out0 = np.transpose(out[0])\n        out1 = np.transpose(out[1])\n        out2 = np.transpose(out[2])\n        score = out2[1,:]\n        points = out1\n        ipass = np.where(score>threshold[2])\n        points = points[:,ipass[0]]\n        total_boxes = np.hstack([total_boxes[ipass[0],0:4].copy(), np.expand_dims(score[ipass].copy(),1)])\n        mv = out0[:,ipass[0]]\n\n        w = total_boxes[:,2]-total_boxes[:,0]+1\n        h = total_boxes[:,3]-total_boxes[:,1]+1\n        points[0:5,:] = np.tile(w,(5, 1))*points[0:5,:] + np.tile(total_boxes[:,0],(5, 1))-1\n        points[5:10,:] = np.tile(h,(5, 1))*points[5:10,:] + np.tile(total_boxes[:,1],(5, 1))-1\n        if total_boxes.shape[0]>0:\n            total_boxes = bbreg(total_boxes.copy(), np.transpose(mv))\n            pick = nms(total_boxes.copy(), 0.7, 'Min')\n            total_boxes = total_boxes[pick,:]\n            points = points[:,pick]\n                \n    return total_boxes, points\n\ndef detect_face_force(img, bbox, pnet, rnet, onet):\n    total_boxes = np.zeros( (1,5), dtype=np.float32)\n    total_boxes[0,0:4] = bbox\n    threshold = [0.0,0.0,0.0]\n    h=img.shape[0]\n    w=img.shape[1]\n    numbox = total_boxes.shape[0]\n    if numbox>0:\n        dy,edy, dx, edx, y, ey, x, ex, tmpw, tmph = pad(total_boxes.copy(), w, h)\n        # second stage\n        tempimg = np.zeros((24,24,3,numbox))\n        for k in range(0,numbox):\n            tmp = np.zeros((int(tmph[k]),int(tmpw[k]),3))\n            tmp[dy[k]-1:edy[k],dx[k]-1:edx[k],:] = img[y[k]-1:ey[k],x[k]-1:ex[k],:]\n            if tmp.shape[0]>0 and tmp.shape[1]>0 or tmp.shape[0]==0 and tmp.shape[1]==0:\n                tempimg[:,:,:,k] = imresample(tmp, (24, 24))\n            else:\n                return np.empty()\n        tempimg = (tempimg-127.5)*0.0078125\n        tempimg1 = np.transpose(tempimg, (3,1,0,2))\n        out = rnet(tempimg1)\n        out0 = np.transpose(out[0])\n        out1 = np.transpose(out[1])\n        score = out1[1,:]\n        ipass = np.where(score>threshold[1])\n        total_boxes = np.hstack([total_boxes[ipass[0],0:4].copy(), np.expand_dims(score[ipass].copy(),1)])\n        mv = out0[:,ipass[0]]\n        if total_boxes.shape[0]>0:\n            pick = nms(total_boxes, 0.7, 'Union')\n            total_boxes = total_boxes[pick,:]\n            total_boxes = bbreg(total_boxes.copy(), np.transpose(mv[:,pick]))\n            total_boxes = rerec(total_boxes.copy())\n\n    numbox = total_boxes.shape[0]\n    if numbox>0:\n        # third stage\n        total_boxes = np.fix(total_boxes).astype(np.int32)\n        dy, edy, dx, edx, y, ey, x, ex, tmpw, tmph = pad(total_boxes.copy(), w, h)\n        tempimg = np.zeros((48,48,3,numbox))\n        for k in range(0,numbox):\n            tmp = np.zeros((int(tmph[k]),int(tmpw[k]),3))\n            tmp[dy[k]-1:edy[k],dx[k]-1:edx[k],:] = img[y[k]-1:ey[k],x[k]-1:ex[k],:]\n            if tmp.shape[0]>0 and tmp.shape[1]>0 or tmp.shape[0]==0 and tmp.shape[1]==0:\n                tempimg[:,:,:,k] = imresample(tmp, (48, 48))\n            else:\n                return np.empty()\n        tempimg = (tempimg-127.5)*0.0078125\n        tempimg1 = np.transpose(tempimg, (3,1,0,2))\n        out = onet(tempimg1)\n        out0 = np.transpose(out[0])\n        out1 = np.transpose(out[1])\n        out2 = np.transpose(out[2])\n        score = out2[1,:]\n        points = out1\n        ipass = np.where(score>threshold[2])\n        points = points[:,ipass[0]]\n        total_boxes = np.hstack([total_boxes[ipass[0],0:4].copy(), np.expand_dims(score[ipass].copy(),1)])\n        mv = out0[:,ipass[0]]\n\n        w = total_boxes[:,2]-total_boxes[:,0]+1\n        h = total_boxes[:,3]-total_boxes[:,1]+1\n        points[0:5,:] = np.tile(w,(5, 1))*points[0:5,:] + np.tile(total_boxes[:,0],(5, 1))-1\n        points[5:10,:] = np.tile(h,(5, 1))*points[5:10,:] + np.tile(total_boxes[:,1],(5, 1))-1\n        if total_boxes.shape[0]>0:\n            total_boxes = bbreg(total_boxes.copy(), np.transpose(mv))\n            pick = nms(total_boxes.copy(), 0.7, 'Min')\n            total_boxes = total_boxes[pick,:]\n            points = points[:,pick]\n                \n    return total_boxes, points\n\ndef bulk_detect_face(images, detection_window_size_ratio, pnet, rnet, onet, threshold, factor):\n    # im: input image\n    # minsize: minimum of faces' size\n    # pnet, rnet, onet: caffemodel\n    # threshold: threshold=[th1 th2 th3], th1-3 are three steps's threshold [0-1]\n\n    all_scales = [None] * len(images)\n    images_with_boxes = [None] * len(images)\n\n    for i in range(len(images)):\n        images_with_boxes[i] = {'total_boxes': np.empty((0, 9))}\n\n    # create scale pyramid\n    for index, img in enumerate(images):\n        all_scales[index] = []\n        h = img.shape[0]\n        w = img.shape[1]\n        minsize = int(detection_window_size_ratio * np.minimum(w, h))\n        factor_count = 0\n        minl = np.amin([h, w])\n        if minsize <= 12:\n            minsize = 12\n\n        m = 12.0 / minsize\n        minl = minl * m\n        while minl >= 12:\n            all_scales[index].append(m * np.power(factor, factor_count))\n            minl = minl * factor\n            factor_count += 1\n\n    # # # # # # # # # # # # #\n    # first stage - fast proposal network (pnet) to obtain face candidates\n    # # # # # # # # # # # # #\n\n    images_obj_per_resolution = {}\n\n    # TODO: use some type of rounding to number module 8 to increase probability that pyramid images will have the same resolution across input images\n\n    for index, scales in enumerate(all_scales):\n        h = images[index].shape[0]\n        w = images[index].shape[1]\n\n        for scale in scales:\n            hs = int(np.ceil(h * scale))\n            ws = int(np.ceil(w * scale))\n\n            if (ws, hs) not in images_obj_per_resolution:\n                images_obj_per_resolution[(ws, hs)] = []\n\n            im_data = imresample(images[index], (hs, ws))\n            im_data = (im_data - 127.5) * 0.0078125\n            img_y = np.transpose(im_data, (1, 0, 2))  # caffe uses different dimensions ordering\n            images_obj_per_resolution[(ws, hs)].append({'scale': scale, 'image': img_y, 'index': index})\n\n    for resolution in images_obj_per_resolution:\n        images_per_resolution = [i['image'] for i in images_obj_per_resolution[resolution]]\n        outs = pnet(images_per_resolution)\n\n        for index in range(len(outs[0])):\n            scale = images_obj_per_resolution[resolution][index]['scale']\n            image_index = images_obj_per_resolution[resolution][index]['index']\n            out0 = np.transpose(outs[0][index], (1, 0, 2))\n            out1 = np.transpose(outs[1][index], (1, 0, 2))\n\n            boxes, _ = generateBoundingBox(out1[:, :, 1].copy(), out0[:, :, :].copy(), scale, threshold[0])\n\n            # inter-scale nms\n            pick = nms(boxes.copy(), 0.5, 'Union')\n            if boxes.size > 0 and pick.size > 0:\n                boxes = boxes[pick, :]\n                images_with_boxes[image_index]['total_boxes'] = np.append(images_with_boxes[image_index]['total_boxes'],\n                                                                          boxes,\n                                                                          axis=0)\n\n    for index, image_obj in enumerate(images_with_boxes):\n        numbox = image_obj['total_boxes'].shape[0]\n        if numbox > 0:\n            h = images[index].shape[0]\n            w = images[index].shape[1]\n            pick = nms(image_obj['total_boxes'].copy(), 0.7, 'Union')\n            image_obj['total_boxes'] = image_obj['total_boxes'][pick, :]\n            regw = image_obj['total_boxes'][:, 2] - image_obj['total_boxes'][:, 0]\n            regh = image_obj['total_boxes'][:, 3] - image_obj['total_boxes'][:, 1]\n            qq1 = image_obj['total_boxes'][:, 0] + image_obj['total_boxes'][:, 5] * regw\n            qq2 = image_obj['total_boxes'][:, 1] + image_obj['total_boxes'][:, 6] * regh\n            qq3 = image_obj['total_boxes'][:, 2] + image_obj['total_boxes'][:, 7] * regw\n            qq4 = image_obj['total_boxes'][:, 3] + image_obj['total_boxes'][:, 8] * regh\n            image_obj['total_boxes'] = np.transpose(np.vstack([qq1, qq2, qq3, qq4, image_obj['total_boxes'][:, 4]]))\n            image_obj['total_boxes'] = rerec(image_obj['total_boxes'].copy())\n            image_obj['total_boxes'][:, 0:4] = np.fix(image_obj['total_boxes'][:, 0:4]).astype(np.int32)\n            dy, edy, dx, edx, y, ey, x, ex, tmpw, tmph = pad(image_obj['total_boxes'].copy(), w, h)\n\n            numbox = image_obj['total_boxes'].shape[0]\n            tempimg = np.zeros((24, 24, 3, numbox))\n\n            if numbox > 0:\n                for k in range(0, numbox):\n                    tmp = np.zeros((int(tmph[k]), int(tmpw[k]), 3))\n                    tmp[dy[k] - 1:edy[k], dx[k] - 1:edx[k], :] = images[index][y[k] - 1:ey[k], x[k] - 1:ex[k], :]\n                    if tmp.shape[0] > 0 and tmp.shape[1] > 0 or tmp.shape[0] == 0 and tmp.shape[1] == 0:\n                        tempimg[:, :, :, k] = imresample(tmp, (24, 24))\n                    else:\n                        return np.empty()\n\n                tempimg = (tempimg - 127.5) * 0.0078125\n                image_obj['rnet_input'] = np.transpose(tempimg, (3, 1, 0, 2))\n\n    # # # # # # # # # # # # #\n    # second stage - refinement of face candidates with rnet\n    # # # # # # # # # # # # #\n\n    bulk_rnet_input = np.empty((0, 24, 24, 3))\n    for index, image_obj in enumerate(images_with_boxes):\n        if 'rnet_input' in image_obj:\n            bulk_rnet_input = np.append(bulk_rnet_input, image_obj['rnet_input'], axis=0)\n\n    out = rnet(bulk_rnet_input)\n    out0 = np.transpose(out[0])\n    out1 = np.transpose(out[1])\n    score = out1[1, :]\n\n    i = 0\n    for index, image_obj in enumerate(images_with_boxes):\n        if 'rnet_input' not in image_obj:\n            continue\n\n        rnet_input_count = image_obj['rnet_input'].shape[0]\n        score_per_image = score[i:i + rnet_input_count]\n        out0_per_image = out0[:, i:i + rnet_input_count]\n\n        ipass = np.where(score_per_image > threshold[1])\n        image_obj['total_boxes'] = np.hstack([image_obj['total_boxes'][ipass[0], 0:4].copy(),\n                                              np.expand_dims(score_per_image[ipass].copy(), 1)])\n\n        mv = out0_per_image[:, ipass[0]]\n\n        if image_obj['total_boxes'].shape[0] > 0:\n            h = images[index].shape[0]\n            w = images[index].shape[1]\n            pick = nms(image_obj['total_boxes'], 0.7, 'Union')\n            image_obj['total_boxes'] = image_obj['total_boxes'][pick, :]\n            image_obj['total_boxes'] = bbreg(image_obj['total_boxes'].copy(), np.transpose(mv[:, pick]))\n            image_obj['total_boxes'] = rerec(image_obj['total_boxes'].copy())\n\n            numbox = image_obj['total_boxes'].shape[0]\n\n            if numbox > 0:\n                tempimg = np.zeros((48, 48, 3, numbox))\n                image_obj['total_boxes'] = np.fix(image_obj['total_boxes']).astype(np.int32)\n                dy, edy, dx, edx, y, ey, x, ex, tmpw, tmph = pad(image_obj['total_boxes'].copy(), w, h)\n\n                for k in range(0, numbox):\n                    tmp = np.zeros((int(tmph[k]), int(tmpw[k]), 3))\n                    tmp[dy[k] - 1:edy[k], dx[k] - 1:edx[k], :] = images[index][y[k] - 1:ey[k], x[k] - 1:ex[k], :]\n                    if tmp.shape[0] > 0 and tmp.shape[1] > 0 or tmp.shape[0] == 0 and tmp.shape[1] == 0:\n                        tempimg[:, :, :, k] = imresample(tmp, (48, 48))\n                    else:\n                        return np.empty()\n                tempimg = (tempimg - 127.5) * 0.0078125\n                image_obj['onet_input'] = np.transpose(tempimg, (3, 1, 0, 2))\n\n        i += rnet_input_count\n\n    # # # # # # # # # # # # #\n    # third stage - further refinement and facial landmarks positions with onet\n    # # # # # # # # # # # # #\n\n    bulk_onet_input = np.empty((0, 48, 48, 3))\n    for index, image_obj in enumerate(images_with_boxes):\n        if 'onet_input' in image_obj:\n            bulk_onet_input = np.append(bulk_onet_input, image_obj['onet_input'], axis=0)\n\n    out = onet(bulk_onet_input)\n\n    out0 = np.transpose(out[0])\n    out1 = np.transpose(out[1])\n    out2 = np.transpose(out[2])\n    score = out2[1, :]\n    points = out1\n\n    i = 0\n    ret = []\n    for index, image_obj in enumerate(images_with_boxes):\n        if 'onet_input' not in image_obj:\n            ret.append(None)\n            continue\n\n        onet_input_count = image_obj['onet_input'].shape[0]\n\n        out0_per_image = out0[:, i:i + onet_input_count]\n        score_per_image = score[i:i + onet_input_count]\n        points_per_image = points[:, i:i + onet_input_count]\n\n        ipass = np.where(score_per_image > threshold[2])\n        points_per_image = points_per_image[:, ipass[0]]\n\n        image_obj['total_boxes'] = np.hstack([image_obj['total_boxes'][ipass[0], 0:4].copy(),\n                                              np.expand_dims(score_per_image[ipass].copy(), 1)])\n        mv = out0_per_image[:, ipass[0]]\n\n        w = image_obj['total_boxes'][:, 2] - image_obj['total_boxes'][:, 0] + 1\n        h = image_obj['total_boxes'][:, 3] - image_obj['total_boxes'][:, 1] + 1\n        points_per_image[0:5, :] = np.tile(w, (5, 1)) * points_per_image[0:5, :] + np.tile(\n            image_obj['total_boxes'][:, 0], (5, 1)) - 1\n        points_per_image[5:10, :] = np.tile(h, (5, 1)) * points_per_image[5:10, :] + np.tile(\n            image_obj['total_boxes'][:, 1], (5, 1)) - 1\n\n        if image_obj['total_boxes'].shape[0] > 0:\n            image_obj['total_boxes'] = bbreg(image_obj['total_boxes'].copy(), np.transpose(mv))\n            pick = nms(image_obj['total_boxes'].copy(), 0.7, 'Min')\n            image_obj['total_boxes'] = image_obj['total_boxes'][pick, :]\n            points_per_image = points_per_image[:, pick]\n\n            ret.append((image_obj['total_boxes'], points_per_image))\n        else:\n            ret.append(None)\n\n        i += onet_input_count\n\n    return ret\n\n\n# function [boundingbox] = bbreg(boundingbox,reg)\ndef bbreg(boundingbox,reg):\n    # calibrate bounding boxes\n    if reg.shape[1]==1:\n        reg = np.reshape(reg, (reg.shape[2], reg.shape[3]))\n\n    w = boundingbox[:,2]-boundingbox[:,0]+1\n    h = boundingbox[:,3]-boundingbox[:,1]+1\n    b1 = boundingbox[:,0]+reg[:,0]*w\n    b2 = boundingbox[:,1]+reg[:,1]*h\n    b3 = boundingbox[:,2]+reg[:,2]*w\n    b4 = boundingbox[:,3]+reg[:,3]*h\n    boundingbox[:,0:4] = np.transpose(np.vstack([b1, b2, b3, b4 ]))\n    return boundingbox\n \ndef generateBoundingBox(imap, reg, scale, t):\n    # use heatmap to generate bounding boxes\n    stride=2\n    cellsize=12\n\n    imap = np.transpose(imap)\n    dx1 = np.transpose(reg[:,:,0])\n    dy1 = np.transpose(reg[:,:,1])\n    dx2 = np.transpose(reg[:,:,2])\n    dy2 = np.transpose(reg[:,:,3])\n    y, x = np.where(imap >= t)\n    if y.shape[0]==1:\n        dx1 = np.flipud(dx1)\n        dy1 = np.flipud(dy1)\n        dx2 = np.flipud(dx2)\n        dy2 = np.flipud(dy2)\n    score = imap[(y,x)]\n    reg = np.transpose(np.vstack([ dx1[(y,x)], dy1[(y,x)], dx2[(y,x)], dy2[(y,x)] ]))\n    if reg.size==0:\n        reg = np.empty((0,3))\n    bb = np.transpose(np.vstack([y,x]))\n    q1 = np.fix((stride*bb+1)/scale)\n    q2 = np.fix((stride*bb+cellsize-1+1)/scale)\n    boundingbox = np.hstack([q1, q2, np.expand_dims(score,1), reg])\n    return boundingbox, reg\n \n# function pick = nms(boxes,threshold,type)\ndef nms(boxes, threshold, method):\n    if boxes.size==0:\n        return np.empty((0,3))\n    x1 = boxes[:,0]\n    y1 = boxes[:,1]\n    x2 = boxes[:,2]\n    y2 = boxes[:,3]\n    s = boxes[:,4]\n    area = (x2-x1+1) * (y2-y1+1)\n    I = np.argsort(s)\n    pick = np.zeros_like(s, dtype=np.int16)\n    counter = 0\n    while I.size>0:\n        i = I[-1]\n        pick[counter] = i\n        counter += 1\n        idx = I[0:-1]\n        xx1 = np.maximum(x1[i], x1[idx])\n        yy1 = np.maximum(y1[i], y1[idx])\n        xx2 = np.minimum(x2[i], x2[idx])\n        yy2 = np.minimum(y2[i], y2[idx])\n        w = np.maximum(0.0, xx2-xx1+1)\n        h = np.maximum(0.0, yy2-yy1+1)\n        inter = w * h\n        if method == 'Min':\n            o = inter / np.minimum(area[i], area[idx])\n        else:\n            o = inter / (area[i] + area[idx] - inter)\n        I = I[np.where(o<=threshold)]\n    pick = pick[0:counter]\n    return pick\n\n# function [dy edy dx edx y ey x ex tmpw tmph] = pad(total_boxes,w,h)\ndef pad(total_boxes, w, h):\n    # compute the padding coordinates (pad the bounding boxes to square)\n    tmpw = (total_boxes[:,2]-total_boxes[:,0]+1).astype(np.int32)\n    tmph = (total_boxes[:,3]-total_boxes[:,1]+1).astype(np.int32)\n    numbox = total_boxes.shape[0]\n\n    dx = np.ones((numbox), dtype=np.int32)\n    dy = np.ones((numbox), dtype=np.int32)\n    edx = tmpw.copy().astype(np.int32)\n    edy = tmph.copy().astype(np.int32)\n\n    x = total_boxes[:,0].copy().astype(np.int32)\n    y = total_boxes[:,1].copy().astype(np.int32)\n    ex = total_boxes[:,2].copy().astype(np.int32)\n    ey = total_boxes[:,3].copy().astype(np.int32)\n\n    tmp = np.where(ex>w)\n    edx.flat[tmp] = np.expand_dims(-ex[tmp]+w+tmpw[tmp],1)\n    ex[tmp] = w\n    \n    tmp = np.where(ey>h)\n    edy.flat[tmp] = np.expand_dims(-ey[tmp]+h+tmph[tmp],1)\n    ey[tmp] = h\n\n    tmp = np.where(x<1)\n    dx.flat[tmp] = np.expand_dims(2-x[tmp],1)\n    x[tmp] = 1\n\n    tmp = np.where(y<1)\n    dy.flat[tmp] = np.expand_dims(2-y[tmp],1)\n    y[tmp] = 1\n    \n    return dy, edy, dx, edx, y, ey, x, ex, tmpw, tmph\n\n# function [bboxA] = rerec(bboxA)\ndef rerec(bboxA):\n    # convert bboxA to square\n    h = bboxA[:,3]-bboxA[:,1]\n    w = bboxA[:,2]-bboxA[:,0]\n    l = np.maximum(w, h)\n    bboxA[:,0] = bboxA[:,0]+w*0.5-l*0.5\n    bboxA[:,1] = bboxA[:,1]+h*0.5-l*0.5\n    bboxA[:,2:4] = bboxA[:,0:2] + np.transpose(np.tile(l,(2,1)))\n    return bboxA\n\ndef imresample(img, sz):\n    im_data = cv2.resize(img, (sz[1], sz[0]), interpolation=cv2.INTER_AREA) #@UndefinedVariable\n    return im_data\n\n    # This method is kept for debugging purpose\n#     h=img.shape[0]\n#     w=img.shape[1]\n#     hs, ws = sz\n#     dx = float(w) / ws\n#     dy = float(h) / hs\n#     im_data = np.zeros((hs,ws,3))\n#     for a1 in range(0,hs):\n#         for a2 in range(0,ws):\n#             for a3 in range(0,3):\n#                 im_data[a1,a2,a3] = img[int(floor(a1*dy)),int(floor(a2*dx)),a3]\n#     return im_data\n\n"
  },
  {
    "path": "Demo/alignment.py",
    "content": "import argparse\nimport numpy as np\nimport cv2\nfrom skimage import transform as trans\nimport tensorflow as tf\nimport os\nimport skimage.io as io\nimport sys\nfrom tqdm import tqdm\n\nimport align.detect_face as detect_face\n\n# Transform grey image to RGB image\ndef to_rgb(img):\n    w, h = img.shape\n    ret = np.empty((w, h, 3), dtype=np.uint8)\n    ret[:, :, 0] = ret[:, :, 1] = ret[:, :, 2] = img\n    return ret\n\n# Align face as ArcFace template \ndef preprocess(img, landmark):\n    image_size = [112,112]\n    src = np.array([\n\t\t[38.2946, 51.6963],\n\t\t[73.5318, 51.5014],\n\t\t[56.0252, 71.7366],\n\t\t[41.5493, 92.3655],\n\t\t[70.7299, 92.2041] ], dtype=np.float32)\n    dst = landmark.astype(np.float32)\n    tform = trans.SimilarityTransform()\n    tform.estimate(dst, src)\n    M = tform.params[0:2,:]\n\n    warped = cv2.warpAffine(img,M,(image_size[1],image_size[0]), borderValue = 0.0)\n    return warped\n\ndef main(args):\n\n\t# MTCNN\n\twith tf.Graph().as_default():\n\t\tsess = tf.Session()\n\t\twith sess.as_default():\n\t\t\tpnet, rnet, onet = detect_face.create_mtcnn(sess, None)\n\tthreshold = [ 0.6, 0.7, 0.7 ]\n\tfactor = 0.709\n    \n    # Output dirs creation\n\tif not os.path.exists(args.output_dir):\n\t\tos.makedirs(args.output_dir)\n\timages = []\n\tfor path in sorted(os.listdir(args.input_dir)):\n\t\tif not os.path.exists(os.path.join(args.output_dir,path)):\n\t\t\tos.mkdir(os.path.join(args.output_dir,path))\n\t\tfor name in sorted(os.listdir(os.path.join(args.input_dir,path))):\n\t\t\timages.append(os.path.join(path,name))\n\n\t# Alignment procedure\n\tfor path in tqdm(images):\n\t\timg = io.imread(os.path.join(args.input_dir,path))\n\t\tif img.ndim == 2:\n\t\t\timg = to_rgb(img)\n\t\timg = img[:,:,0:3]\n\t\t_minsize = min(min(img.shape[0]//5, img.shape[1]//5),80)\n\t\tbounding_boxes, points = detect_face.detect_face(img, _minsize, pnet, rnet, onet, threshold, factor)\n\t\tif bounding_boxes.size>0:\n\t\t\tbindex = -1\n\t\t\tnrof_faces = bounding_boxes.shape[0]\n\t\t\tif nrof_faces>0:\n\t\t\t\tdet = bounding_boxes[:,0:4]\n\t\t\t\timg_size = np.asarray(img.shape)[0:2]\n\t\t\t\tbindex = 0\n\t\t\t\tif nrof_faces>1:\t\t\t\t\n\t\t\t\t\tbounding_box_size = (det[:,2]-det[:,0])*(det[:,3]-det[:,1])\n\t\t\t\t\timg_center = img_size / 2\n\t\t\t\t\toffsets = np.vstack([ (det[:,0]+det[:,2])/2-img_center[1], (det[:,1]+det[:,3])/2-img_center[0] ])\n\t\t\t\t\toffset_dist_squared = np.sum(np.power(offsets,2.0),0)\n\t\t\t\t\tbindex = np.argmax(bounding_box_size-offset_dist_squared*2.0)\n\t\t\tpoints = points[:, bindex]\n\t\t\tlandmark = points.reshape((2,5)).T\n\t\t\twarped = preprocess(img, landmark)\n\t\t\tio.imsave(os.path.join(args.output_dir,path), warped)\n\t\telse:\n\t\t\tprint(path+' was skipped')\n\n\ndef parse_arguments(argv):\n    parser = argparse.ArgumentParser()\n    \n    parser.add_argument('input_dir', type=str, help='Directory with unaligned images.')\n    parser.add_argument('output_dir', type=str, help='Directory for aligned face thumbnails.')\n    return parser.parse_args(argv)\n\nif __name__ == '__main__':\n    main(parse_arguments(sys.argv[1:]))\n"
  },
  {
    "path": "Demo/demo.py",
    "content": "import sys\r\nimport argparse\r\nimport numpy as np\r\nimport cv2\r\nimport tensorflow as tf\r\nfrom align import detect_face\r\nfrom skimage import transform as trans\r\nfrom skimage.io import imsave\r\nimport os\r\nimport datetime\r\n\r\n# Align face as ArcFace template\r\ndef preprocess(img, landmark):\r\n    image_size = [112,112]\r\n    src = np.array([\r\n\t\t[38.2946, 51.6963],\r\n\t\t[73.5318, 51.5014],\r\n\t\t[56.0252, 71.7366],\r\n\t\t[41.5493, 92.3655],\r\n\t\t[70.7299, 92.2041] ], dtype=np.float32)\r\n    dst = landmark.astype(np.float32)\r\n    tform = trans.SimilarityTransform()\r\n    tform.estimate(dst, src)\r\n    M = tform.params[0:2,:]\r\n\r\n    warped = cv2.warpAffine(img,M,(image_size[1],image_size[0]), borderValue = 0.0)\r\n    return warped\r\n\r\ndef main(args):\r\n\t\r\n\t# Models download\r\n\tfrozen_graph = args.model\r\n\twith tf.gfile.GFile(frozen_graph, \"rb\") as f:\r\n\t\tgraph_def = tf.GraphDef()\r\n\t\tgraph_def.ParseFromString(f.read())\r\n\twith tf.Graph().as_default() as graph:\r\n\t\t  tf.import_graph_def(graph_def,\r\n\t\t\t\t\t\t\t  input_map=None,\r\n\t\t\t\t\t\t\t  return_elements=None,\r\n\t\t\t\t\t\t\t  name=\"\")\r\n\t\t  image_input = graph.get_tensor_by_name('image_input:0')\r\n\t\t  keep_prob = graph.get_tensor_by_name('keep_prob:0')\r\n\t\t  is_train = graph.get_tensor_by_name('training_mode:0')\r\n\t\t  embedding = graph.get_tensor_by_name('embedding:0')\r\n\r\n\t\t  minsize = 100\r\n\t\t  threshold = [ 0.6, 0.7, 0.7 ] \r\n\t\t  factor = 0.709\r\n\t\t  sess = tf.Session(graph=graph)\r\n\t\t  pnet, rnet, onet = detect_face.create_mtcnn(sess, None)\r\n\t\r\n\t# Centroids download\r\n\tanchor = np.load(os.path.join(args.centroids,'centroids.npy'))\r\n\tnames = open(os.path.join(args.centroids,'centroids_names.txt')).read().split('\\n')[:-1]\r\n\r\n\tIDcolor = [255., 255., 255.]\r\n\tIDcolor2 = [255., 0., 0.]\r\n\r\n\tvideo_capture = cv2.VideoCapture(0)\r\n\tvideo_capture.set(3, 1280)\r\n\tvideo_capture.set(4, 1024)\r\n\r\n\twhile(True):\r\n\t\t\r\n\t\t# Start of video sequence processing\r\n\t\tret, frame = video_capture.read()\r\n\t\tframe = cv2.flip(frame[:,:,::-1], 1)\r\n\t\tif not ret:\r\n\t\t\tprint('Cannot access the webcam')\r\n\t\t\tbreak\r\n\t\t\r\n\t\tkey = cv2.waitKey(1)\t\t\t\r\n\t\tif key == ord('q'):\r\n\t\t\tbreak\r\n\t\tif key == ord('s'):\r\n\t\t\timsave('Demo-'+str(datetime.datetime.now())+'.jpg',frame)\r\n\t\t\t\r\n\t\t# Search and preparation of all faces on the frame\r\n\t\tbounding_boxes, points = detect_face.detect_face(frame, minsize, pnet, rnet, onet, threshold, factor)\r\n\t\t\r\n\t\tbatch = np.zeros((bounding_boxes.shape[0],3,112,112),dtype=np.float32)    \r\n\t\tfor i in range(bounding_boxes.shape[0]):\r\n\t\t\tlandmark = points[:,i].reshape((2,5)).T\r\n\t\t\twarped = preprocess(frame, landmark = landmark)\r\n\t\t\twarped = np.transpose(warped,[2,0,1]).reshape((1,3,112,112))\r\n\t\t\tbatch[i] = (warped-127.5)*0.0078125\r\n\t\t\r\n\t\t# Recognition of all faces\r\n\t\tif batch.shape[0]!=0:\r\n\t\t\tembs = sess.run(embedding,feed_dict={image_input:batch,keep_prob:1.0,is_train:False})\r\n\t\t\tfor i in range(bounding_boxes.shape[0]):\r\n\t\t\t\tprobabilities = np.dot(anchor,embs[i])\r\n\t\t\t\tval = np.max(probabilities)\r\n\t\t\t\tpos = np.argmax(probabilities)\r\n\r\n\t\t\t\tpt1 = (int(bounding_boxes[i][0]), int(bounding_boxes[i][1]))\r\n\t\t\t\tpt2 = (int(bounding_boxes[i][2]), int(bounding_boxes[i][3]))\r\n\r\n\t\t\t\tcv2.rectangle(frame, pt1, pt2, IDcolor)\r\n\t\t\t\r\n\t\t\t\tcv2.putText(frame, 'Top-1 class: '+names[pos],\r\n\t\t\t\t\t\t\t\t(int(bounding_boxes[i][0]), int(bounding_boxes[i][1])-5),\r\n\t\t\t\t\t\t\t\tcv2.FONT_HERSHEY_SIMPLEX, 1., IDcolor, 3)\r\n\t\t\t\tcv2.putText(frame, 'Sim. to top-1 class: '+str(round(val,4)),\r\n\t\t\t\t\t\t\t\t(int(bounding_boxes[i][0]), int(bounding_boxes[i][3])+30),\r\n\t\t\t\t\t\t\t\tcv2.FONT_HERSHEY_SIMPLEX, 1., IDcolor, 3)\r\n\r\n\t\tcv2.imshow('Camera (\"q\" to quit, \"s\" to save frame)', frame[:,:,::-1])\r\n\t\t\r\n\t\t\t\t\r\n\tvideo_capture.release()\r\n\tcv2.destroyAllWindows()\r\n\t\r\n\t\r\n\r\ndef parse_arguments(argv):\r\n\tparser = argparse.ArgumentParser()\r\n    \r\n\tparser.add_argument('model',type=str, help='Path to the model.')\r\n\tparser.add_argument('centroids',type=str, help='Dir with centoids of classes for classifier.')\r\n\treturn parser.parse_args(argv)\r\n\r\nif __name__ == '__main__':\r\n\tmain(parse_arguments(sys.argv[1:]))\r\n"
  },
  {
    "path": "Demo/dumping.py",
    "content": "import skimage.io as io\nimport os\nimport numpy as np\nfrom tqdm import tqdm\nimport sys\nimport argparse\n\ndef main(args):\n\t\n\t# Output dirs creation\n\tif not os.path.exists(args.output_dir):\n\t\tos.makedirs(args.output_dir)\n\timages = []\n\tlabels = []\n\tlabel = 0\n\tfor path in sorted(os.listdir(args.input_dir)):\n\t\tfor name in sorted(os.listdir(os.path.join(args.input_dir,path))):\n\t\t\tif args.mx:\n\t\t\t\timages.append([[label],os.path.join(args.input_dir,path,name)])\n\t\t\telse:\n\t\t\t\timages.append(os.path.join(args.input_dir,path,name))\n\t\t\tlabels.append(label)\n\t\tlabel += 1\n\t\n\t\n\tif args.mx:\n\t\t# MXnet model\n\t\timport mxnet as mx\n\t\tsym, arg_params, aux_params = mx.model.load_checkpoint(args.model, 0)\n\t\tsym = sym.get_internals()['fc1_output']\n\t\tmodel = mx.mod.Module(symbol=sym, context=mx.gpu(0), label_names = None)\n\t\tmodel.bind(data_shapes=[('data', (1, 3, 112, 112))])\n\t\tmodel.set_params(arg_params, aux_params)\n\t\titerator = mx.image.ImageIter(batch_size=args.batch,data_shape=(3,112,112),imglist=images,path_root='')\n\telse:\n\t\t# TensorFlow model\n\t\timport tensorflow as tf\n\t\tfrozen_graph = args.model\n\t\twith tf.gfile.GFile(frozen_graph, \"rb\") as f:\n\t\t\tgraph_def = tf.GraphDef()\n\t\t\tgraph_def.ParseFromString(f.read())\n\t\twith tf.Graph().as_default() as graph:\n\t\t\ttf.import_graph_def(graph_def,\n\t\t\t\t\t\t\t  input_map=None,\n\t\t\t\t\t\t\t  return_elements=None,\n\t\t\t\t\t\t\t  name=\"\")\n\t\t\timage_input = graph.get_tensor_by_name('image_input:0')\n\t\t\tkeep_prob = graph.get_tensor_by_name('keep_prob:0')\n\t\t\tis_train = graph.get_tensor_by_name('training_mode:0')\n\t\t\tembedding = graph.get_tensor_by_name('embedding:0')\n\t\t\tsess = tf.Session(graph=graph)\n\t\t\tinp_place = tf.placeholder(np.array(['1','2'],dtype='str').dtype)\n\t\t\tpipeline = tf.data.Dataset.from_tensor_slices(inp_place)\n\t\t\tdef parse(filename):\n\t\t\t\timage_string = tf.read_file(filename)\n\t\t\t\timage = tf.image.decode_jpeg(image_string,dct_method=\"INTEGER_ACCURATE\")\n\t\t\t\timage = tf.cast(image,tf.float32)\n\t\t\t\timage = (image - 127.5)*0.0078125\n\t\t\t\timage = tf.transpose(image,perm=[2,0,1])\n\t\t\t\treturn image\n\t\t\tpipeline = pipeline.map(parse,num_parallel_calls=4)\n\t\t\tpipeline = pipeline.batch(args.batch)\n\t\t\tpipiline = pipeline.prefetch(8)\n\t\t\titerator = pipeline.make_initializable_iterator()\n\t\t\tnext_element = iterator.get_next()\n\t\tsess.run(iterator.initializer,feed_dict={inp_place:images})\t\t\t\t  \n    \n    # Embeddings evaluation\n\tembs = np.zeros((len(images),512),dtype=np.float32)\n\tfor i in tqdm(range(int(np.ceil(len(images)/args.batch)))):\n\t\tif args.mx:\n\t\t\tdb = mx.io.DataBatch(data=iterator.next().data)\n\t\t\tmodel.forward(db, is_train=False)\n\t\t\temb = model.get_outputs()[0].asnumpy()\n\t\t\tlength = min(args.batch,len(images)-i*args.batch)\n\t\t\tembs[i*args.batch:i*args.batch+length] = emb[:length]/np.expand_dims(np.sqrt(np.sum(emb[:length]**2,1)),1)\n\t\telse:\n\t\t\tdb = sess.run(next_element)\n\t\t\tembs[i*args.batch:min((i+1)*args.batch,len(images))] = sess.run(embedding,feed_dict=\\\n\t\t\t\t\t\t\t\t\t\t{image_input:db,keep_prob:1.0,is_train:False})\n\t\n\t# Centroids preparation\n\tanchor = np.zeros((label,512),dtype=np.float32)\n\tlabels = np.array(labels)\n\tfor i in range(label):\n\t\ttmp = np.sum(embs[labels==i],axis=0)\n\t\tanchor[i] = tmp/np.sqrt(np.sum(tmp**2))\n\tnp.save(os.path.join(args.output_dir,'centroids'),anchor)\n\tnames = open(os.path.join(args.output_dir,'centroids_names.txt'),'w')\n\tfor i in sorted(os.listdir(args.input_dir)):\n\t\tnames.write(i+'\\n')\n\tnames.close()            \n\n\ndef parse_arguments(argv):\n    parser = argparse.ArgumentParser()\n    \n    parser.add_argument('input_dir', type=str, help='Directory with aligned images.')\n    parser.add_argument('output_dir', type=str, help='Directory to save embeddings.')\n    parser.add_argument('model',type=str, help='Path to the model.')\n    parser.add_argument('--mx',action='store_true', help='Flag to use the original mxnet model.')\n    parser.add_argument('--batch',type=int, help='Batch size.',default=30)\n    return parser.parse_args(argv)\n\nif __name__ == '__main__':\n    main(parse_arguments(sys.argv[1:]))\n"
  },
  {
    "path": "LICENSE",
    "content": "MIT License\n\nCopyright (c) 2019 Stepan Komkov, Aleksandr Petiushko\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n"
  },
  {
    "path": "README.md",
    "content": "# AdvHat: Real-world adversarial attack on ArcFace Face ID system\n\nBy Stepan Komkov and Aleksandr Petiushko\n\nThis is the code repository for the AdvHat research article. The article is available [here](https://arxiv.org/abs/1908.08705). The video demo is available [here](https://youtu.be/a4iNg0wWBsQ). Code that is used for the article is available right here.\n\n## Abstract\n\nWe propose a novel easily reproducible technique to attack the best public Face ID system ArcFace in different shooting conditions. To create an attack, we print the rectangular paper sticker on a common color printer and put it on the hat. The adversarial sticker is prepared with a novel algorithm for off-plane transformations of the image which imitates sticker location on the hat. Such an approach confuses the state-of-the-art public Face ID model LResNet100E-IR, ArcFace@ms1m-refine-v2 and is transferable to other Face ID models.\n\n## The repository\n\nThe repository is organized as follows:\n\n* In the Attack directory, you can find code and instructions on how to reproduce an attack for your images.\n* In the Demo directory, you can find a demo script which can help you to verify the robustness of the prepared attack to the real-world shooting conditions.\n\n## Built With\n\n* [InsightFace's ArcFace](https://github.com/deepinsight/insightface) - The SOTA public FaceID model\n* [Kevin Zakka's STN](https://github.com/kevinzakka/spatial-transformer-network) - Spatial Transformer realization\n\n## Citation\n\n```\n@article{komkov2019advhat,\n  title={AdvHat: Real-world adversarial attack on ArcFace Face ID system},\n  author={Komkov, Stepan and Petiushko, Aleksandr},\n  journal={arXiv preprint arXiv:1908.08705},\n  year={2019}\n}\n```\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE.md](https://github.com/papermsucode/advhat/blob/master/LICENSE) file for details.\n"
  },
  {
    "path": "Utils/MXtoTF.ipynb",
    "content": "{\n \"cells\": [\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 1,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"import mxnet as mx\\n\",\n    \"from mxnet import ndarray as nd\\n\",\n    \"\\n\",\n    \"from easydict import EasyDict as edict\\n\",\n    \"import numpy as np\\n\",\n    \"import os\\n\",\n    \"from tqdm import tqdm\\n\",\n    \"import skimage.io as io\\n\",\n    \"\\n\",\n    \"import tensorflow as tf\\n\",\n    \"import tensorflow.contrib.slim as slim\\n\",\n    \"gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.3)\\n\",\n    \"sess = tf.Session(config=tf.ConfigProto(gpu_options=gpu_options, log_device_placement=False))\"\n   ]\n  },\n  {\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"### My versions of libraries\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 2,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"data\": {\n      \"text/plain\": [\n       \"'1.4.1'\"\n      ]\n     },\n     \"execution_count\": 2,\n     \"metadata\": {},\n     \"output_type\": \"execute_result\"\n    }\n   ],\n   \"source\": [\n    \"mx.__version__\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 3,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"data\": {\n      \"text/plain\": [\n       \"'1.13.1'\"\n      ]\n     },\n     \"execution_count\": 3,\n     \"metadata\": {},\n     \"output_type\": \"execute_result\"\n    }\n   ],\n   \"source\": [\n    \"tf.__version__\"\n   ]\n  },\n  {\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"## Transformation of the MXNet model weights to NumPy\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 4,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"ctx = mx.gpu(0)\\n\",\n    \"net = edict()\\n\",\n    \"net.ctx = ctx\\n\",\n    \"net.sym, net.arg_params, net.aux_params = mx.model.load_checkpoint('../../Matcher/models/model-r34-amf/model', 0)\\n\",\n    \"all_layers = net.sym.get_internals()\\n\",\n    \"net.sym = all_layers['fc1_output']\\n\",\n    \"net.model = mx.mod.Module(symbol=net.sym, context=net.ctx, label_names = None)\\n\",\n    \"net.model.bind(data_shapes=[('data', (2, 3, 112, 112))])\\n\",\n    \"net.model.set_params(net.arg_params, net.aux_params)\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 5,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"stage1_length = len(set([x.name.split('_')[1] for x in net.sym.get_internals() if 'stage1' in x.name]))\\n\",\n    \"stage2_length = len(set([x.name.split('_')[1] for x in net.sym.get_internals() if 'stage2' in x.name]))\\n\",\n    \"stage3_length = len(set([x.name.split('_')[1] for x in net.sym.get_internals() if 'stage3' in x.name]))\\n\",\n    \"stage4_length = len(set([x.name.split('_')[1] for x in net.sym.get_internals() if 'stage4' in x.name]))\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 6,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"data\": {\n      \"image/svg+xml\": [\n       \"<?xml version=\\\"1.0\\\" encoding=\\\"UTF-8\\\" standalone=\\\"no\\\"?>\\n\",\n       \"<!DOCTYPE svg PUBLIC \\\"-//W3C//DTD SVG 1.1//EN\\\"\\n\",\n       \" \\\"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\\\">\\n\",\n       \"<!-- Generated by graphviz version 2.40.1 (20161225.0304)\\n\",\n       \" -->\\n\",\n       \"<!-- Title: plot Pages: 1 -->\\n\",\n       \"<svg width=\\\"253pt\\\" height=\\\"11534pt\\\"\\n\",\n       \" viewBox=\\\"0.00 0.00 253.00 11534.00\\\" xmlns=\\\"http://www.w3.org/2000/svg\\\" xmlns:xlink=\\\"http://www.w3.org/1999/xlink\\\">\\n\",\n       \"<g id=\\\"graph0\\\" class=\\\"graph\\\" transform=\\\"scale(1 1) rotate(0) translate(4 11530)\\\">\\n\",\n       \"<title>plot</title>\\n\",\n       \"<polygon fill=\\\"#ffffff\\\" stroke=\\\"transparent\\\" points=\\\"-4,4 -4,-11530 249,-11530 249,4 -4,4\\\"/>\\n\",\n       \"<!-- data -->\\n\",\n       \"<g id=\\\"node1\\\" class=\\\"node\\\">\\n\",\n       \"<title>data</title>\\n\",\n       \"<ellipse fill=\\\"#8dd3c7\\\" stroke=\\\"#000000\\\" cx=\\\"155\\\" cy=\\\"-29\\\" rx=\\\"47\\\" ry=\\\"29\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"155\\\" y=\\\"-25.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">data</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- id -->\\n\",\n       \"<g id=\\\"node2\\\" class=\\\"node\\\">\\n\",\n       \"<title>id</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"202,-152 108,-152 108,-94 202,-94 202,-152\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"155\\\" y=\\\"-119.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">id</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- id&#45;&gt;data -->\\n\",\n       \"<g id=\\\"edge1\\\" class=\\\"edge\\\">\\n\",\n       \"<title>id&#45;&gt;data</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M155,-83.6321C155,-75.1148 155,-66.2539 155,-58.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"155,-93.7731 150.5001,-83.773 155,-88.7731 155.0001,-83.7731 155.0001,-83.7731 155.0001,-83.7731 155,-88.7731 159.5001,-83.7731 155,-93.7731 155,-93.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _minusscalar0 -->\\n\",\n       \"<g id=\\\"node3\\\" class=\\\"node\\\">\\n\",\n       \"<title>_minusscalar0</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"202,-246 108,-246 108,-188 202,-188 202,-246\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"155\\\" y=\\\"-213.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">_minusscalar0</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- _minusscalar0&#45;&gt;id -->\\n\",\n       \"<g id=\\\"edge2\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_minusscalar0&#45;&gt;id</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M155,-177.6321C155,-169.1148 155,-160.2539 155,-152.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"155,-187.7731 150.5001,-177.773 155,-182.7731 155.0001,-177.7731 155.0001,-177.7731 155.0001,-177.7731 155,-182.7731 159.5001,-177.7731 155,-187.7731 155,-187.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _mulscalar0 -->\\n\",\n       \"<g id=\\\"node4\\\" class=\\\"node\\\">\\n\",\n       \"<title>_mulscalar0</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"202,-340 108,-340 108,-282 202,-282 202,-340\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"155\\\" y=\\\"-307.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">_mulscalar0</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- _mulscalar0&#45;&gt;_minusscalar0 -->\\n\",\n       \"<g id=\\\"edge3\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_mulscalar0&#45;&gt;_minusscalar0</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M155,-271.6321C155,-263.1148 155,-254.2539 155,-246.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"155,-281.7731 150.5001,-271.773 155,-276.7731 155.0001,-271.7731 155.0001,-271.7731 155.0001,-271.7731 155,-276.7731 159.5001,-271.7731 155,-281.7731 155,-281.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- conv0 -->\\n\",\n       \"<g id=\\\"node5\\\" class=\\\"node\\\">\\n\",\n       \"<title>conv0</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"202,-434 108,-434 108,-376 202,-376 202,-434\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"155\\\" y=\\\"-408.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"155\\\" y=\\\"-393.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 64</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- conv0&#45;&gt;_mulscalar0 -->\\n\",\n       \"<g id=\\\"edge4\\\" class=\\\"edge\\\">\\n\",\n       \"<title>conv0&#45;&gt;_mulscalar0</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M155,-365.6321C155,-357.1148 155,-348.2539 155,-340.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"155,-375.7731 150.5001,-365.773 155,-370.7731 155.0001,-365.7731 155.0001,-365.7731 155.0001,-365.7731 155,-370.7731 159.5001,-365.7731 155,-375.7731 155,-375.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- bn0 -->\\n\",\n       \"<g id=\\\"node6\\\" class=\\\"node\\\">\\n\",\n       \"<title>bn0</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"202,-528 108,-528 108,-470 202,-470 202,-528\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"155\\\" y=\\\"-495.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">bn0</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- bn0&#45;&gt;conv0 -->\\n\",\n       \"<g id=\\\"edge5\\\" class=\\\"edge\\\">\\n\",\n       \"<title>bn0&#45;&gt;conv0</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M155,-459.6321C155,-451.1148 155,-442.2539 155,-434.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"155,-469.7731 150.5001,-459.773 155,-464.7731 155.0001,-459.7731 155.0001,-459.7731 155.0001,-459.7731 155,-464.7731 159.5001,-459.7731 155,-469.7731 155,-469.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- relu0 -->\\n\",\n       \"<g id=\\\"node7\\\" class=\\\"node\\\">\\n\",\n       \"<title>relu0</title>\\n\",\n       \"<polygon fill=\\\"#ffffb3\\\" stroke=\\\"#000000\\\" points=\\\"202,-622 108,-622 108,-564 202,-564 202,-622\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"155\\\" y=\\\"-596.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">LeakyReLU</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"155\\\" y=\\\"-581.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">prelu</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- relu0&#45;&gt;bn0 -->\\n\",\n       \"<g id=\\\"edge6\\\" class=\\\"edge\\\">\\n\",\n       \"<title>relu0&#45;&gt;bn0</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M155,-553.6321C155,-545.1148 155,-536.2539 155,-528.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"155,-563.7731 150.5001,-553.773 155,-558.7731 155.0001,-553.7731 155.0001,-553.7731 155.0001,-553.7731 155,-558.7731 159.5001,-553.7731 155,-563.7731 155,-563.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit1_bn1 -->\\n\",\n       \"<g id=\\\"node8\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit1_bn1</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"165,-716 71,-716 71,-658 165,-658 165,-716\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"118\\\" y=\\\"-683.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage1_unit1_bn1</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit1_bn1&#45;&gt;relu0 -->\\n\",\n       \"<g id=\\\"edge7\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit1_bn1&#45;&gt;relu0</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M133.275,-648.1932C136.695,-639.5047 140.2666,-630.4307 143.5029,-622.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"129.5042,-657.7731 128.9796,-646.8198 131.3356,-653.1205 133.1669,-648.468 133.1669,-648.468 133.1669,-648.468 131.3356,-653.1205 137.3542,-650.1162 129.5042,-657.7731 129.5042,-657.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit1_conv1 -->\\n\",\n       \"<g id=\\\"node9\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit1_conv1</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"160,-810 66,-810 66,-752 160,-752 160,-810\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"113\\\" y=\\\"-784.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"113\\\" y=\\\"-769.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 64</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit1_conv1&#45;&gt;stage1_unit1_bn1 -->\\n\",\n       \"<g id=\\\"edge8\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit1_conv1&#45;&gt;stage1_unit1_bn1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M115.094,-741.6321C115.5471,-733.1148 116.0184,-724.2539 116.4463,-716.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"114.5546,-751.7731 110.5922,-741.5481 114.8203,-746.7801 115.0859,-741.7872 115.0859,-741.7872 115.0859,-741.7872 114.8203,-746.7801 119.5795,-742.0263 114.5546,-751.7731 114.5546,-751.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit1_bn2 -->\\n\",\n       \"<g id=\\\"node10\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit1_bn2</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"151,-904 57,-904 57,-846 151,-846 151,-904\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"104\\\" y=\\\"-871.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage1_unit1_bn2</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit1_bn2&#45;&gt;stage1_unit1_conv1 -->\\n\",\n       \"<g id=\\\"edge9\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit1_bn2&#45;&gt;stage1_unit1_conv1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M107.7693,-835.6321C108.5848,-827.1148 109.4331,-818.2539 110.2034,-810.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"106.7983,-845.7731 103.272,-835.3897 107.2749,-840.7958 107.7515,-835.8186 107.7515,-835.8186 107.7515,-835.8186 107.2749,-840.7958 112.231,-836.2475 106.7983,-845.7731 106.7983,-845.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit1_relu1 -->\\n\",\n       \"<g id=\\\"node11\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit1_relu1</title>\\n\",\n       \"<polygon fill=\\\"#ffffb3\\\" stroke=\\\"#000000\\\" points=\\\"133,-998 39,-998 39,-940 133,-940 133,-998\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"86\\\" y=\\\"-972.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">LeakyReLU</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"86\\\" y=\\\"-957.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">prelu</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit1_relu1&#45;&gt;stage1_unit1_bn2 -->\\n\",\n       \"<g id=\\\"edge10\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit1_relu1&#45;&gt;stage1_unit1_bn2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M93.4848,-929.9128C95.1322,-921.3096 96.8493,-912.3423 98.4068,-904.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"91.5966,-939.7731 89.0577,-929.1052 92.537,-934.8623 93.4774,-929.9515 93.4774,-929.9515 93.4774,-929.9515 92.537,-934.8623 97.8971,-930.7979 91.5966,-939.7731 91.5966,-939.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit1_conv2 -->\\n\",\n       \"<g id=\\\"node12\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit1_conv2</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"133,-1092 39,-1092 39,-1034 133,-1034 133,-1092\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"86\\\" y=\\\"-1066.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"86\\\" y=\\\"-1051.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/2x2, 64</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit1_conv2&#45;&gt;stage1_unit1_relu1 -->\\n\",\n       \"<g id=\\\"edge11\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit1_conv2&#45;&gt;stage1_unit1_relu1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M86,-1023.6321C86,-1015.1148 86,-1006.2539 86,-998.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"86,-1033.7731 81.5001,-1023.773 86,-1028.7731 86.0001,-1023.7731 86.0001,-1023.7731 86.0001,-1023.7731 86,-1028.7731 90.5001,-1023.7731 86,-1033.7731 86,-1033.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit1_bn3 -->\\n\",\n       \"<g id=\\\"node13\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit1_bn3</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"133,-1186 39,-1186 39,-1128 133,-1128 133,-1186\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"86\\\" y=\\\"-1153.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage1_unit1_bn3</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit1_bn3&#45;&gt;stage1_unit1_conv2 -->\\n\",\n       \"<g id=\\\"edge12\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit1_bn3&#45;&gt;stage1_unit1_conv2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M86,-1117.6321C86,-1109.1148 86,-1100.2539 86,-1092.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"86,-1127.7731 81.5001,-1117.773 86,-1122.7731 86.0001,-1117.7731 86.0001,-1117.7731 86.0001,-1117.7731 86,-1122.7731 90.5001,-1117.7731 86,-1127.7731 86,-1127.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit1_conv1sc -->\\n\",\n       \"<g id=\\\"node14\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit1_conv1sc</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"245,-998 151,-998 151,-940 245,-940 245,-998\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"198\\\" y=\\\"-972.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"198\\\" y=\\\"-957.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">1x1/2x2, 64</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit1_conv1sc&#45;&gt;relu0 -->\\n\",\n       \"<g id=\\\"edge13\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit1_conv1sc&#45;&gt;relu0</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M197.5657,-929.5116C196.3257,-870.1237 191.6279,-754.4886 174,-658 171.8237,-646.0875 168.3229,-633.2386 164.9157,-622.1352\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"197.7537,-939.5841 193.0677,-929.6699 197.6603,-934.585 197.567,-929.5859 197.567,-929.5859 197.567,-929.5859 197.6603,-934.585 202.0662,-929.5018 197.7537,-939.5841 197.7537,-939.5841\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit1_sc -->\\n\",\n       \"<g id=\\\"node15\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit1_sc</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"245,-1186 151,-1186 151,-1128 245,-1128 245,-1186\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"198\\\" y=\\\"-1153.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage1_unit1_sc</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit1_sc&#45;&gt;stage1_unit1_conv1sc -->\\n\",\n       \"<g id=\\\"edge14\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit1_sc&#45;&gt;stage1_unit1_conv1sc</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M198,-1117.5685C198,-1081.9118 198,-1030.2305 198,-998.1687\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"198,-1127.625 193.5001,-1117.625 198,-1122.625 198.0001,-1117.625 198.0001,-1117.625 198.0001,-1117.625 198,-1122.625 202.5001,-1117.625 198,-1127.625 198,-1127.625\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus0 -->\\n\",\n       \"<g id=\\\"node16\\\" class=\\\"node\\\">\\n\",\n       \"<title>_plus0</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"189,-1280 95,-1280 95,-1222 189,-1222 189,-1280\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"142\\\" y=\\\"-1247.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">_plus0</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus0&#45;&gt;stage1_unit1_bn3 -->\\n\",\n       \"<g id=\\\"edge15\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus0&#45;&gt;stage1_unit1_bn3</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M119.3812,-1213.0328C114.0542,-1204.091 108.4571,-1194.6959 103.401,-1186.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"124.5882,-1221.7731 115.6042,-1215.4852 122.0292,-1217.4776 119.4701,-1213.1821 119.4701,-1213.1821 119.4701,-1213.1821 122.0292,-1217.4776 123.3361,-1210.8789 124.5882,-1221.7731 124.5882,-1221.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus0&#45;&gt;stage1_unit1_sc -->\\n\",\n       \"<g id=\\\"edge16\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus0&#45;&gt;stage1_unit1_sc</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M164.6188,-1213.0328C169.9458,-1204.091 175.5429,-1194.6959 180.599,-1186.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"159.4118,-1221.7731 160.6639,-1210.8789 161.9708,-1217.4776 164.5299,-1213.1821 164.5299,-1213.1821 164.5299,-1213.1821 161.9708,-1217.4776 168.3958,-1215.4852 159.4118,-1221.7731 159.4118,-1221.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit2_bn1 -->\\n\",\n       \"<g id=\\\"node17\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit2_bn1</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"227,-1374 133,-1374 133,-1316 227,-1316 227,-1374\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"180\\\" y=\\\"-1341.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage1_unit2_bn1</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit2_bn1&#45;&gt;_plus0 -->\\n\",\n       \"<g id=\\\"edge17\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit2_bn1&#45;&gt;_plus0</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M164.4254,-1306.4734C160.8787,-1297.6999 157.1673,-1288.5191 153.8078,-1280.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"168.1849,-1315.7731 160.2649,-1308.1886 166.3109,-1311.1375 164.4369,-1306.502 164.4369,-1306.502 164.4369,-1306.502 166.3109,-1311.1375 168.6089,-1304.8154 168.1849,-1315.7731 168.1849,-1315.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit2_conv1 -->\\n\",\n       \"<g id=\\\"node18\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit2_conv1</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"227,-1468 133,-1468 133,-1410 227,-1410 227,-1468\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"180\\\" y=\\\"-1442.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"180\\\" y=\\\"-1427.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 64</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit2_conv1&#45;&gt;stage1_unit2_bn1 -->\\n\",\n       \"<g id=\\\"edge18\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit2_conv1&#45;&gt;stage1_unit2_bn1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M180,-1399.6321C180,-1391.1148 180,-1382.2539 180,-1374.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"180,-1409.7731 175.5001,-1399.773 180,-1404.7731 180.0001,-1399.7731 180.0001,-1399.7731 180.0001,-1399.7731 180,-1404.7731 184.5001,-1399.7731 180,-1409.7731 180,-1409.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit2_bn2 -->\\n\",\n       \"<g id=\\\"node19\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit2_bn2</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"227,-1562 133,-1562 133,-1504 227,-1504 227,-1562\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"180\\\" y=\\\"-1529.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage1_unit2_bn2</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit2_bn2&#45;&gt;stage1_unit2_conv1 -->\\n\",\n       \"<g id=\\\"edge19\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit2_bn2&#45;&gt;stage1_unit2_conv1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M180,-1493.6321C180,-1485.1148 180,-1476.2539 180,-1468.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"180,-1503.7731 175.5001,-1493.773 180,-1498.7731 180.0001,-1493.7731 180.0001,-1493.7731 180.0001,-1493.7731 180,-1498.7731 184.5001,-1493.7731 180,-1503.7731 180,-1503.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit2_relu1 -->\\n\",\n       \"<g id=\\\"node20\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit2_relu1</title>\\n\",\n       \"<polygon fill=\\\"#ffffb3\\\" stroke=\\\"#000000\\\" points=\\\"227,-1656 133,-1656 133,-1598 227,-1598 227,-1656\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"180\\\" y=\\\"-1630.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">LeakyReLU</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"180\\\" y=\\\"-1615.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">prelu</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit2_relu1&#45;&gt;stage1_unit2_bn2 -->\\n\",\n       \"<g id=\\\"edge20\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit2_relu1&#45;&gt;stage1_unit2_bn2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M180,-1587.6321C180,-1579.1148 180,-1570.2539 180,-1562.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"180,-1597.7731 175.5001,-1587.773 180,-1592.7731 180.0001,-1587.7731 180.0001,-1587.7731 180.0001,-1587.7731 180,-1592.7731 184.5001,-1587.7731 180,-1597.7731 180,-1597.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit2_conv2 -->\\n\",\n       \"<g id=\\\"node21\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit2_conv2</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"227,-1750 133,-1750 133,-1692 227,-1692 227,-1750\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"180\\\" y=\\\"-1724.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"180\\\" y=\\\"-1709.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 64</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit2_conv2&#45;&gt;stage1_unit2_relu1 -->\\n\",\n       \"<g id=\\\"edge21\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit2_conv2&#45;&gt;stage1_unit2_relu1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M180,-1681.6321C180,-1673.1148 180,-1664.2539 180,-1656.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"180,-1691.7731 175.5001,-1681.773 180,-1686.7731 180.0001,-1681.7731 180.0001,-1681.7731 180.0001,-1681.7731 180,-1686.7731 184.5001,-1681.7731 180,-1691.7731 180,-1691.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit2_bn3 -->\\n\",\n       \"<g id=\\\"node22\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit2_bn3</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"227,-1844 133,-1844 133,-1786 227,-1786 227,-1844\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"180\\\" y=\\\"-1811.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage1_unit2_bn3</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit2_bn3&#45;&gt;stage1_unit2_conv2 -->\\n\",\n       \"<g id=\\\"edge22\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit2_bn3&#45;&gt;stage1_unit2_conv2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M180,-1775.6321C180,-1767.1148 180,-1758.2539 180,-1750.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"180,-1785.7731 175.5001,-1775.773 180,-1780.7731 180.0001,-1775.7731 180.0001,-1775.7731 180.0001,-1775.7731 180,-1780.7731 184.5001,-1775.7731 180,-1785.7731 180,-1785.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus1 -->\\n\",\n       \"<g id=\\\"node23\\\" class=\\\"node\\\">\\n\",\n       \"<title>_plus1</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"189,-1938 95,-1938 95,-1880 189,-1880 189,-1938\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"142\\\" y=\\\"-1905.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">_plus1</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus1&#45;&gt;_plus0 -->\\n\",\n       \"<g id=\\\"edge24\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus1&#45;&gt;_plus0</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M129.6305,-1869.6582C118.874,-1831.9243 105,-1773.1468 105,-1721 105,-1721 105,-1721 105,-1439 105,-1381.9505 121.6054,-1316.9649 132.5676,-1280.2799\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"132.5676,-1879.7201 125.4457,-1871.3817 131.1665,-1874.9204 129.7654,-1870.1207 129.7654,-1870.1207 129.7654,-1870.1207 131.1665,-1874.9204 134.0852,-1868.8597 132.5676,-1879.7201 132.5676,-1879.7201\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus1&#45;&gt;stage1_unit2_bn3 -->\\n\",\n       \"<g id=\\\"edge23\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus1&#45;&gt;stage1_unit2_bn3</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M157.5746,-1870.4734C161.1213,-1861.6999 164.8327,-1852.5191 168.1922,-1844.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"153.8151,-1879.7731 153.3911,-1868.8154 155.6891,-1875.1375 157.5631,-1870.502 157.5631,-1870.502 157.5631,-1870.502 155.6891,-1875.1375 161.7351,-1872.1886 153.8151,-1879.7731 153.8151,-1879.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit3_bn1 -->\\n\",\n       \"<g id=\\\"node24\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit3_bn1</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"227,-2032 133,-2032 133,-1974 227,-1974 227,-2032\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"180\\\" y=\\\"-1999.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage1_unit3_bn1</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit3_bn1&#45;&gt;_plus1 -->\\n\",\n       \"<g id=\\\"edge25\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit3_bn1&#45;&gt;_plus1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M164.4254,-1964.4734C160.8787,-1955.6999 157.1673,-1946.5191 153.8078,-1938.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"168.1849,-1973.7731 160.2649,-1966.1886 166.3109,-1969.1375 164.4369,-1964.502 164.4369,-1964.502 164.4369,-1964.502 166.3109,-1969.1375 168.6089,-1962.8154 168.1849,-1973.7731 168.1849,-1973.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit3_conv1 -->\\n\",\n       \"<g id=\\\"node25\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit3_conv1</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"227,-2126 133,-2126 133,-2068 227,-2068 227,-2126\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"180\\\" y=\\\"-2100.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"180\\\" y=\\\"-2085.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 64</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit3_conv1&#45;&gt;stage1_unit3_bn1 -->\\n\",\n       \"<g id=\\\"edge26\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit3_conv1&#45;&gt;stage1_unit3_bn1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M180,-2057.6321C180,-2049.1148 180,-2040.2539 180,-2032.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"180,-2067.7731 175.5001,-2057.773 180,-2062.7731 180.0001,-2057.7731 180.0001,-2057.7731 180.0001,-2057.7731 180,-2062.7731 184.5001,-2057.7731 180,-2067.7731 180,-2067.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit3_bn2 -->\\n\",\n       \"<g id=\\\"node26\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit3_bn2</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"227,-2220 133,-2220 133,-2162 227,-2162 227,-2220\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"180\\\" y=\\\"-2187.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage1_unit3_bn2</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit3_bn2&#45;&gt;stage1_unit3_conv1 -->\\n\",\n       \"<g id=\\\"edge27\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit3_bn2&#45;&gt;stage1_unit3_conv1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M180,-2151.6321C180,-2143.1148 180,-2134.2539 180,-2126.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"180,-2161.7731 175.5001,-2151.773 180,-2156.7731 180.0001,-2151.7731 180.0001,-2151.7731 180.0001,-2151.7731 180,-2156.7731 184.5001,-2151.7731 180,-2161.7731 180,-2161.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit3_relu1 -->\\n\",\n       \"<g id=\\\"node27\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit3_relu1</title>\\n\",\n       \"<polygon fill=\\\"#ffffb3\\\" stroke=\\\"#000000\\\" points=\\\"227,-2314 133,-2314 133,-2256 227,-2256 227,-2314\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"180\\\" y=\\\"-2288.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">LeakyReLU</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"180\\\" y=\\\"-2273.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">prelu</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit3_relu1&#45;&gt;stage1_unit3_bn2 -->\\n\",\n       \"<g id=\\\"edge28\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit3_relu1&#45;&gt;stage1_unit3_bn2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M180,-2245.6321C180,-2237.1148 180,-2228.2539 180,-2220.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"180,-2255.7731 175.5001,-2245.773 180,-2250.7731 180.0001,-2245.7731 180.0001,-2245.7731 180.0001,-2245.7731 180,-2250.7731 184.5001,-2245.7731 180,-2255.7731 180,-2255.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit3_conv2 -->\\n\",\n       \"<g id=\\\"node28\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit3_conv2</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"227,-2408 133,-2408 133,-2350 227,-2350 227,-2408\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"180\\\" y=\\\"-2382.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"180\\\" y=\\\"-2367.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 64</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit3_conv2&#45;&gt;stage1_unit3_relu1 -->\\n\",\n       \"<g id=\\\"edge29\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit3_conv2&#45;&gt;stage1_unit3_relu1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M180,-2339.6321C180,-2331.1148 180,-2322.2539 180,-2314.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"180,-2349.7731 175.5001,-2339.773 180,-2344.7731 180.0001,-2339.7731 180.0001,-2339.7731 180.0001,-2339.7731 180,-2344.7731 184.5001,-2339.7731 180,-2349.7731 180,-2349.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit3_bn3 -->\\n\",\n       \"<g id=\\\"node29\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage1_unit3_bn3</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"227,-2502 133,-2502 133,-2444 227,-2444 227,-2502\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"180\\\" y=\\\"-2469.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage1_unit3_bn3</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage1_unit3_bn3&#45;&gt;stage1_unit3_conv2 -->\\n\",\n       \"<g id=\\\"edge30\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage1_unit3_bn3&#45;&gt;stage1_unit3_conv2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M180,-2433.6321C180,-2425.1148 180,-2416.2539 180,-2408.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"180,-2443.7731 175.5001,-2433.773 180,-2438.7731 180.0001,-2433.7731 180.0001,-2433.7731 180.0001,-2433.7731 180,-2438.7731 184.5001,-2433.7731 180,-2443.7731 180,-2443.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus2 -->\\n\",\n       \"<g id=\\\"node30\\\" class=\\\"node\\\">\\n\",\n       \"<title>_plus2</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"189,-2596 95,-2596 95,-2538 189,-2538 189,-2596\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"142\\\" y=\\\"-2563.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">_plus2</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus2&#45;&gt;_plus1 -->\\n\",\n       \"<g id=\\\"edge32\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus2&#45;&gt;_plus1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M129.6305,-2527.6582C118.874,-2489.9243 105,-2431.1468 105,-2379 105,-2379 105,-2379 105,-2097 105,-2039.9505 121.6054,-1974.9649 132.5676,-1938.2799\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"132.5676,-2537.7201 125.4457,-2529.3817 131.1665,-2532.9204 129.7654,-2528.1207 129.7654,-2528.1207 129.7654,-2528.1207 131.1665,-2532.9204 134.0852,-2526.8597 132.5676,-2537.7201 132.5676,-2537.7201\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus2&#45;&gt;stage1_unit3_bn3 -->\\n\",\n       \"<g id=\\\"edge31\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus2&#45;&gt;stage1_unit3_bn3</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M157.5746,-2528.4734C161.1213,-2519.6999 164.8327,-2510.5191 168.1922,-2502.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"153.8151,-2537.7731 153.3911,-2526.8154 155.6891,-2533.1375 157.5631,-2528.502 157.5631,-2528.502 157.5631,-2528.502 155.6891,-2533.1375 161.7351,-2530.1886 153.8151,-2537.7731 153.8151,-2537.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit1_bn1 -->\\n\",\n       \"<g id=\\\"node31\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit1_bn1</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"152,-2690 58,-2690 58,-2632 152,-2632 152,-2690\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"105\\\" y=\\\"-2657.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage2_unit1_bn1</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit1_bn1&#45;&gt;_plus2 -->\\n\",\n       \"<g id=\\\"edge33\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit1_bn1&#45;&gt;_plus2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M120.275,-2622.1932C123.695,-2613.5047 127.2666,-2604.4307 130.5029,-2596.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"116.5042,-2631.7731 115.9796,-2620.8198 118.3356,-2627.1205 120.1669,-2622.468 120.1669,-2622.468 120.1669,-2622.468 118.3356,-2627.1205 124.3542,-2624.1162 116.5042,-2631.7731 116.5042,-2631.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit1_conv1 -->\\n\",\n       \"<g id=\\\"node32\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit1_conv1</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"147,-2784 53,-2784 53,-2726 147,-2726 147,-2784\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"100\\\" y=\\\"-2758.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"100\\\" y=\\\"-2743.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 128</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit1_conv1&#45;&gt;stage2_unit1_bn1 -->\\n\",\n       \"<g id=\\\"edge34\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit1_conv1&#45;&gt;stage2_unit1_bn1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M102.094,-2715.6321C102.5471,-2707.1148 103.0184,-2698.2539 103.4463,-2690.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"101.5546,-2725.7731 97.5922,-2715.5481 101.8203,-2720.7801 102.0859,-2715.7872 102.0859,-2715.7872 102.0859,-2715.7872 101.8203,-2720.7801 106.5795,-2716.0263 101.5546,-2725.7731 101.5546,-2725.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit1_bn2 -->\\n\",\n       \"<g id=\\\"node33\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit1_bn2</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"138,-2878 44,-2878 44,-2820 138,-2820 138,-2878\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"91\\\" y=\\\"-2845.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage2_unit1_bn2</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit1_bn2&#45;&gt;stage2_unit1_conv1 -->\\n\",\n       \"<g id=\\\"edge35\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit1_bn2&#45;&gt;stage2_unit1_conv1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M94.7693,-2809.6321C95.5848,-2801.1148 96.4331,-2792.2539 97.2034,-2784.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"93.7983,-2819.7731 90.272,-2809.3897 94.2749,-2814.7958 94.7515,-2809.8186 94.7515,-2809.8186 94.7515,-2809.8186 94.2749,-2814.7958 99.231,-2810.2475 93.7983,-2819.7731 93.7983,-2819.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit1_relu1 -->\\n\",\n       \"<g id=\\\"node34\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit1_relu1</title>\\n\",\n       \"<polygon fill=\\\"#ffffb3\\\" stroke=\\\"#000000\\\" points=\\\"120,-2972 26,-2972 26,-2914 120,-2914 120,-2972\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"73\\\" y=\\\"-2946.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">LeakyReLU</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"73\\\" y=\\\"-2931.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">prelu</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit1_relu1&#45;&gt;stage2_unit1_bn2 -->\\n\",\n       \"<g id=\\\"edge36\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit1_relu1&#45;&gt;stage2_unit1_bn2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M80.4848,-2903.9128C82.1322,-2895.3096 83.8493,-2886.3423 85.4068,-2878.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"78.5966,-2913.7731 76.0577,-2903.1052 79.537,-2908.8623 80.4774,-2903.9515 80.4774,-2903.9515 80.4774,-2903.9515 79.537,-2908.8623 84.8971,-2904.7979 78.5966,-2913.7731 78.5966,-2913.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit1_conv2 -->\\n\",\n       \"<g id=\\\"node35\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit1_conv2</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"120,-3066 26,-3066 26,-3008 120,-3008 120,-3066\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"73\\\" y=\\\"-3040.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"73\\\" y=\\\"-3025.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/2x2, 128</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit1_conv2&#45;&gt;stage2_unit1_relu1 -->\\n\",\n       \"<g id=\\\"edge37\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit1_conv2&#45;&gt;stage2_unit1_relu1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M73,-2997.6321C73,-2989.1148 73,-2980.2539 73,-2972.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"73,-3007.7731 68.5001,-2997.773 73,-3002.7731 73.0001,-2997.7731 73.0001,-2997.7731 73.0001,-2997.7731 73,-3002.7731 77.5001,-2997.7731 73,-3007.7731 73,-3007.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit1_bn3 -->\\n\",\n       \"<g id=\\\"node36\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit1_bn3</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"120,-3160 26,-3160 26,-3102 120,-3102 120,-3160\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"73\\\" y=\\\"-3127.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage2_unit1_bn3</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit1_bn3&#45;&gt;stage2_unit1_conv2 -->\\n\",\n       \"<g id=\\\"edge38\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit1_bn3&#45;&gt;stage2_unit1_conv2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M73,-3091.6321C73,-3083.1148 73,-3074.2539 73,-3066.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"73,-3101.7731 68.5001,-3091.773 73,-3096.7731 73.0001,-3091.7731 73.0001,-3091.7731 73.0001,-3091.7731 73,-3096.7731 77.5001,-3091.7731 73,-3101.7731 73,-3101.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit1_conv1sc -->\\n\",\n       \"<g id=\\\"node37\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit1_conv1sc</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"232,-2972 138,-2972 138,-2914 232,-2914 232,-2972\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"185\\\" y=\\\"-2946.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"185\\\" y=\\\"-2931.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">1x1/2x2, 128</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit1_conv1sc&#45;&gt;_plus2 -->\\n\",\n       \"<g id=\\\"edge39\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit1_conv1sc&#45;&gt;_plus2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M184.5657,-2903.5116C183.3257,-2844.1237 178.6279,-2728.4886 161,-2632 158.8237,-2620.0875 155.3229,-2607.2386 151.9157,-2596.1352\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"184.7537,-2913.5841 180.0677,-2903.6699 184.6603,-2908.585 184.567,-2903.5859 184.567,-2903.5859 184.567,-2903.5859 184.6603,-2908.585 189.0662,-2903.5018 184.7537,-2913.5841 184.7537,-2913.5841\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit1_sc -->\\n\",\n       \"<g id=\\\"node38\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit1_sc</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"232,-3160 138,-3160 138,-3102 232,-3102 232,-3160\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"185\\\" y=\\\"-3127.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage2_unit1_sc</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit1_sc&#45;&gt;stage2_unit1_conv1sc -->\\n\",\n       \"<g id=\\\"edge40\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit1_sc&#45;&gt;stage2_unit1_conv1sc</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M185,-3091.5685C185,-3055.9118 185,-3004.2305 185,-2972.1687\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"185,-3101.625 180.5001,-3091.625 185,-3096.625 185.0001,-3091.625 185.0001,-3091.625 185.0001,-3091.625 185,-3096.625 189.5001,-3091.625 185,-3101.625 185,-3101.625\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus3 -->\\n\",\n       \"<g id=\\\"node39\\\" class=\\\"node\\\">\\n\",\n       \"<title>_plus3</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"176,-3254 82,-3254 82,-3196 176,-3196 176,-3254\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"129\\\" y=\\\"-3221.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">_plus3</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus3&#45;&gt;stage2_unit1_bn3 -->\\n\",\n       \"<g id=\\\"edge41\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus3&#45;&gt;stage2_unit1_bn3</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M106.3812,-3187.0328C101.0542,-3178.091 95.4571,-3168.6959 90.401,-3160.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"111.5882,-3195.7731 102.6042,-3189.4852 109.0292,-3191.4776 106.4701,-3187.1821 106.4701,-3187.1821 106.4701,-3187.1821 109.0292,-3191.4776 110.3361,-3184.8789 111.5882,-3195.7731 111.5882,-3195.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus3&#45;&gt;stage2_unit1_sc -->\\n\",\n       \"<g id=\\\"edge42\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus3&#45;&gt;stage2_unit1_sc</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M151.6188,-3187.0328C156.9458,-3178.091 162.5429,-3168.6959 167.599,-3160.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"146.4118,-3195.7731 147.6639,-3184.8789 148.9708,-3191.4776 151.5299,-3187.1821 151.5299,-3187.1821 151.5299,-3187.1821 148.9708,-3191.4776 155.3958,-3189.4852 146.4118,-3195.7731 146.4118,-3195.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit2_bn1 -->\\n\",\n       \"<g id=\\\"node40\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit2_bn1</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"214,-3348 120,-3348 120,-3290 214,-3290 214,-3348\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-3315.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage2_unit2_bn1</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit2_bn1&#45;&gt;_plus3 -->\\n\",\n       \"<g id=\\\"edge43\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit2_bn1&#45;&gt;_plus3</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M151.4254,-3280.4734C147.8787,-3271.6999 144.1673,-3262.5191 140.8078,-3254.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"155.1849,-3289.7731 147.2649,-3282.1886 153.3109,-3285.1375 151.4369,-3280.502 151.4369,-3280.502 151.4369,-3280.502 153.3109,-3285.1375 155.6089,-3278.8154 155.1849,-3289.7731 155.1849,-3289.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit2_conv1 -->\\n\",\n       \"<g id=\\\"node41\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit2_conv1</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"214,-3442 120,-3442 120,-3384 214,-3384 214,-3442\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-3416.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-3401.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 128</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit2_conv1&#45;&gt;stage2_unit2_bn1 -->\\n\",\n       \"<g id=\\\"edge44\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit2_conv1&#45;&gt;stage2_unit2_bn1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M167,-3373.6321C167,-3365.1148 167,-3356.2539 167,-3348.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"167,-3383.7731 162.5001,-3373.773 167,-3378.7731 167.0001,-3373.7731 167.0001,-3373.7731 167.0001,-3373.7731 167,-3378.7731 171.5001,-3373.7731 167,-3383.7731 167,-3383.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit2_bn2 -->\\n\",\n       \"<g id=\\\"node42\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit2_bn2</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"214,-3536 120,-3536 120,-3478 214,-3478 214,-3536\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-3503.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage2_unit2_bn2</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit2_bn2&#45;&gt;stage2_unit2_conv1 -->\\n\",\n       \"<g id=\\\"edge45\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit2_bn2&#45;&gt;stage2_unit2_conv1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M167,-3467.6321C167,-3459.1148 167,-3450.2539 167,-3442.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"167,-3477.7731 162.5001,-3467.773 167,-3472.7731 167.0001,-3467.7731 167.0001,-3467.7731 167.0001,-3467.7731 167,-3472.7731 171.5001,-3467.7731 167,-3477.7731 167,-3477.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit2_relu1 -->\\n\",\n       \"<g id=\\\"node43\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit2_relu1</title>\\n\",\n       \"<polygon fill=\\\"#ffffb3\\\" stroke=\\\"#000000\\\" points=\\\"214,-3630 120,-3630 120,-3572 214,-3572 214,-3630\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-3604.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">LeakyReLU</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-3589.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">prelu</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit2_relu1&#45;&gt;stage2_unit2_bn2 -->\\n\",\n       \"<g id=\\\"edge46\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit2_relu1&#45;&gt;stage2_unit2_bn2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M167,-3561.6321C167,-3553.1148 167,-3544.2539 167,-3536.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"167,-3571.7731 162.5001,-3561.773 167,-3566.7731 167.0001,-3561.7731 167.0001,-3561.7731 167.0001,-3561.7731 167,-3566.7731 171.5001,-3561.7731 167,-3571.7731 167,-3571.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit2_conv2 -->\\n\",\n       \"<g id=\\\"node44\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit2_conv2</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"214,-3724 120,-3724 120,-3666 214,-3666 214,-3724\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-3698.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-3683.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 128</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit2_conv2&#45;&gt;stage2_unit2_relu1 -->\\n\",\n       \"<g id=\\\"edge47\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit2_conv2&#45;&gt;stage2_unit2_relu1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M167,-3655.6321C167,-3647.1148 167,-3638.2539 167,-3630.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"167,-3665.7731 162.5001,-3655.773 167,-3660.7731 167.0001,-3655.7731 167.0001,-3655.7731 167.0001,-3655.7731 167,-3660.7731 171.5001,-3655.7731 167,-3665.7731 167,-3665.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit2_bn3 -->\\n\",\n       \"<g id=\\\"node45\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit2_bn3</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"214,-3818 120,-3818 120,-3760 214,-3760 214,-3818\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-3785.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage2_unit2_bn3</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit2_bn3&#45;&gt;stage2_unit2_conv2 -->\\n\",\n       \"<g id=\\\"edge48\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit2_bn3&#45;&gt;stage2_unit2_conv2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M167,-3749.6321C167,-3741.1148 167,-3732.2539 167,-3724.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"167,-3759.7731 162.5001,-3749.773 167,-3754.7731 167.0001,-3749.7731 167.0001,-3749.7731 167.0001,-3749.7731 167,-3754.7731 171.5001,-3749.7731 167,-3759.7731 167,-3759.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus4 -->\\n\",\n       \"<g id=\\\"node46\\\" class=\\\"node\\\">\\n\",\n       \"<title>_plus4</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"176,-3912 82,-3912 82,-3854 176,-3854 176,-3912\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"129\\\" y=\\\"-3879.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">_plus4</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus4&#45;&gt;_plus3 -->\\n\",\n       \"<g id=\\\"edge50\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus4&#45;&gt;_plus3</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M116.6305,-3843.6582C105.874,-3805.9243 92,-3747.1468 92,-3695 92,-3695 92,-3695 92,-3413 92,-3355.9505 108.6054,-3290.9649 119.5676,-3254.2799\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"119.5676,-3853.7201 112.4457,-3845.3817 118.1665,-3848.9204 116.7654,-3844.1207 116.7654,-3844.1207 116.7654,-3844.1207 118.1665,-3848.9204 121.0852,-3842.8597 119.5676,-3853.7201 119.5676,-3853.7201\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus4&#45;&gt;stage2_unit2_bn3 -->\\n\",\n       \"<g id=\\\"edge49\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus4&#45;&gt;stage2_unit2_bn3</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M144.5746,-3844.4734C148.1213,-3835.6999 151.8327,-3826.5191 155.1922,-3818.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"140.8151,-3853.7731 140.3911,-3842.8154 142.6891,-3849.1375 144.5631,-3844.502 144.5631,-3844.502 144.5631,-3844.502 142.6891,-3849.1375 148.7351,-3846.1886 140.8151,-3853.7731 140.8151,-3853.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit3_bn1 -->\\n\",\n       \"<g id=\\\"node47\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit3_bn1</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"214,-4006 120,-4006 120,-3948 214,-3948 214,-4006\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-3973.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage2_unit3_bn1</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit3_bn1&#45;&gt;_plus4 -->\\n\",\n       \"<g id=\\\"edge51\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit3_bn1&#45;&gt;_plus4</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M151.4254,-3938.4734C147.8787,-3929.6999 144.1673,-3920.5191 140.8078,-3912.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"155.1849,-3947.7731 147.2649,-3940.1886 153.3109,-3943.1375 151.4369,-3938.502 151.4369,-3938.502 151.4369,-3938.502 153.3109,-3943.1375 155.6089,-3936.8154 155.1849,-3947.7731 155.1849,-3947.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit3_conv1 -->\\n\",\n       \"<g id=\\\"node48\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit3_conv1</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"214,-4100 120,-4100 120,-4042 214,-4042 214,-4100\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-4074.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-4059.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 128</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit3_conv1&#45;&gt;stage2_unit3_bn1 -->\\n\",\n       \"<g id=\\\"edge52\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit3_conv1&#45;&gt;stage2_unit3_bn1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M167,-4031.6321C167,-4023.1148 167,-4014.2539 167,-4006.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"167,-4041.7731 162.5001,-4031.773 167,-4036.7731 167.0001,-4031.7731 167.0001,-4031.7731 167.0001,-4031.7731 167,-4036.7731 171.5001,-4031.7731 167,-4041.7731 167,-4041.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit3_bn2 -->\\n\",\n       \"<g id=\\\"node49\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit3_bn2</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"214,-4194 120,-4194 120,-4136 214,-4136 214,-4194\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-4161.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage2_unit3_bn2</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit3_bn2&#45;&gt;stage2_unit3_conv1 -->\\n\",\n       \"<g id=\\\"edge53\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit3_bn2&#45;&gt;stage2_unit3_conv1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M167,-4125.6321C167,-4117.1148 167,-4108.2539 167,-4100.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"167,-4135.7731 162.5001,-4125.773 167,-4130.7731 167.0001,-4125.7731 167.0001,-4125.7731 167.0001,-4125.7731 167,-4130.7731 171.5001,-4125.7731 167,-4135.7731 167,-4135.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit3_relu1 -->\\n\",\n       \"<g id=\\\"node50\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit3_relu1</title>\\n\",\n       \"<polygon fill=\\\"#ffffb3\\\" stroke=\\\"#000000\\\" points=\\\"214,-4288 120,-4288 120,-4230 214,-4230 214,-4288\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-4262.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">LeakyReLU</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-4247.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">prelu</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit3_relu1&#45;&gt;stage2_unit3_bn2 -->\\n\",\n       \"<g id=\\\"edge54\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit3_relu1&#45;&gt;stage2_unit3_bn2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M167,-4219.6321C167,-4211.1148 167,-4202.2539 167,-4194.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"167,-4229.7731 162.5001,-4219.773 167,-4224.7731 167.0001,-4219.7731 167.0001,-4219.7731 167.0001,-4219.7731 167,-4224.7731 171.5001,-4219.7731 167,-4229.7731 167,-4229.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit3_conv2 -->\\n\",\n       \"<g id=\\\"node51\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit3_conv2</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"214,-4382 120,-4382 120,-4324 214,-4324 214,-4382\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-4356.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-4341.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 128</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit3_conv2&#45;&gt;stage2_unit3_relu1 -->\\n\",\n       \"<g id=\\\"edge55\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit3_conv2&#45;&gt;stage2_unit3_relu1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M167,-4313.6321C167,-4305.1148 167,-4296.2539 167,-4288.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"167,-4323.7731 162.5001,-4313.773 167,-4318.7731 167.0001,-4313.7731 167.0001,-4313.7731 167.0001,-4313.7731 167,-4318.7731 171.5001,-4313.7731 167,-4323.7731 167,-4323.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit3_bn3 -->\\n\",\n       \"<g id=\\\"node52\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit3_bn3</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"214,-4476 120,-4476 120,-4418 214,-4418 214,-4476\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-4443.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage2_unit3_bn3</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit3_bn3&#45;&gt;stage2_unit3_conv2 -->\\n\",\n       \"<g id=\\\"edge56\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit3_bn3&#45;&gt;stage2_unit3_conv2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M167,-4407.6321C167,-4399.1148 167,-4390.2539 167,-4382.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"167,-4417.7731 162.5001,-4407.773 167,-4412.7731 167.0001,-4407.7731 167.0001,-4407.7731 167.0001,-4407.7731 167,-4412.7731 171.5001,-4407.7731 167,-4417.7731 167,-4417.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus5 -->\\n\",\n       \"<g id=\\\"node53\\\" class=\\\"node\\\">\\n\",\n       \"<title>_plus5</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"176,-4570 82,-4570 82,-4512 176,-4512 176,-4570\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"129\\\" y=\\\"-4537.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">_plus5</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus5&#45;&gt;_plus4 -->\\n\",\n       \"<g id=\\\"edge58\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus5&#45;&gt;_plus4</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M116.6305,-4501.6582C105.874,-4463.9243 92,-4405.1468 92,-4353 92,-4353 92,-4353 92,-4071 92,-4013.9505 108.6054,-3948.9649 119.5676,-3912.2799\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"119.5676,-4511.7201 112.4457,-4503.3817 118.1665,-4506.9204 116.7654,-4502.1207 116.7654,-4502.1207 116.7654,-4502.1207 118.1665,-4506.9204 121.0852,-4500.8597 119.5676,-4511.7201 119.5676,-4511.7201\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus5&#45;&gt;stage2_unit3_bn3 -->\\n\",\n       \"<g id=\\\"edge57\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus5&#45;&gt;stage2_unit3_bn3</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M144.5746,-4502.4734C148.1213,-4493.6999 151.8327,-4484.5191 155.1922,-4476.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"140.8151,-4511.7731 140.3911,-4500.8154 142.6891,-4507.1375 144.5631,-4502.502 144.5631,-4502.502 144.5631,-4502.502 142.6891,-4507.1375 148.7351,-4504.1886 140.8151,-4511.7731 140.8151,-4511.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit4_bn1 -->\\n\",\n       \"<g id=\\\"node54\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit4_bn1</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"214,-4664 120,-4664 120,-4606 214,-4606 214,-4664\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-4631.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage2_unit4_bn1</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit4_bn1&#45;&gt;_plus5 -->\\n\",\n       \"<g id=\\\"edge59\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit4_bn1&#45;&gt;_plus5</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M151.4254,-4596.4734C147.8787,-4587.6999 144.1673,-4578.5191 140.8078,-4570.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"155.1849,-4605.7731 147.2649,-4598.1886 153.3109,-4601.1375 151.4369,-4596.502 151.4369,-4596.502 151.4369,-4596.502 153.3109,-4601.1375 155.6089,-4594.8154 155.1849,-4605.7731 155.1849,-4605.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit4_conv1 -->\\n\",\n       \"<g id=\\\"node55\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit4_conv1</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"214,-4758 120,-4758 120,-4700 214,-4700 214,-4758\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-4732.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-4717.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 128</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit4_conv1&#45;&gt;stage2_unit4_bn1 -->\\n\",\n       \"<g id=\\\"edge60\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit4_conv1&#45;&gt;stage2_unit4_bn1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M167,-4689.6321C167,-4681.1148 167,-4672.2539 167,-4664.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"167,-4699.7731 162.5001,-4689.773 167,-4694.7731 167.0001,-4689.7731 167.0001,-4689.7731 167.0001,-4689.7731 167,-4694.7731 171.5001,-4689.7731 167,-4699.7731 167,-4699.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit4_bn2 -->\\n\",\n       \"<g id=\\\"node56\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit4_bn2</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"214,-4852 120,-4852 120,-4794 214,-4794 214,-4852\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-4819.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage2_unit4_bn2</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit4_bn2&#45;&gt;stage2_unit4_conv1 -->\\n\",\n       \"<g id=\\\"edge61\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit4_bn2&#45;&gt;stage2_unit4_conv1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M167,-4783.6321C167,-4775.1148 167,-4766.2539 167,-4758.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"167,-4793.7731 162.5001,-4783.773 167,-4788.7731 167.0001,-4783.7731 167.0001,-4783.7731 167.0001,-4783.7731 167,-4788.7731 171.5001,-4783.7731 167,-4793.7731 167,-4793.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit4_relu1 -->\\n\",\n       \"<g id=\\\"node57\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit4_relu1</title>\\n\",\n       \"<polygon fill=\\\"#ffffb3\\\" stroke=\\\"#000000\\\" points=\\\"214,-4946 120,-4946 120,-4888 214,-4888 214,-4946\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-4920.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">LeakyReLU</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-4905.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">prelu</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit4_relu1&#45;&gt;stage2_unit4_bn2 -->\\n\",\n       \"<g id=\\\"edge62\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit4_relu1&#45;&gt;stage2_unit4_bn2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M167,-4877.6321C167,-4869.1148 167,-4860.2539 167,-4852.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"167,-4887.7731 162.5001,-4877.773 167,-4882.7731 167.0001,-4877.7731 167.0001,-4877.7731 167.0001,-4877.7731 167,-4882.7731 171.5001,-4877.7731 167,-4887.7731 167,-4887.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit4_conv2 -->\\n\",\n       \"<g id=\\\"node58\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit4_conv2</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"214,-5040 120,-5040 120,-4982 214,-4982 214,-5040\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-5014.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-4999.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 128</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit4_conv2&#45;&gt;stage2_unit4_relu1 -->\\n\",\n       \"<g id=\\\"edge63\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit4_conv2&#45;&gt;stage2_unit4_relu1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M167,-4971.6321C167,-4963.1148 167,-4954.2539 167,-4946.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"167,-4981.7731 162.5001,-4971.773 167,-4976.7731 167.0001,-4971.7731 167.0001,-4971.7731 167.0001,-4971.7731 167,-4976.7731 171.5001,-4971.7731 167,-4981.7731 167,-4981.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit4_bn3 -->\\n\",\n       \"<g id=\\\"node59\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage2_unit4_bn3</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"214,-5134 120,-5134 120,-5076 214,-5076 214,-5134\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"167\\\" y=\\\"-5101.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage2_unit4_bn3</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage2_unit4_bn3&#45;&gt;stage2_unit4_conv2 -->\\n\",\n       \"<g id=\\\"edge64\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage2_unit4_bn3&#45;&gt;stage2_unit4_conv2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M167,-5065.6321C167,-5057.1148 167,-5048.2539 167,-5040.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"167,-5075.7731 162.5001,-5065.773 167,-5070.7731 167.0001,-5065.7731 167.0001,-5065.7731 167.0001,-5065.7731 167,-5070.7731 171.5001,-5065.7731 167,-5075.7731 167,-5075.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus6 -->\\n\",\n       \"<g id=\\\"node60\\\" class=\\\"node\\\">\\n\",\n       \"<title>_plus6</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"176,-5228 82,-5228 82,-5170 176,-5170 176,-5228\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"129\\\" y=\\\"-5195.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">_plus6</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus6&#45;&gt;_plus5 -->\\n\",\n       \"<g id=\\\"edge66\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus6&#45;&gt;_plus5</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M116.6305,-5159.6582C105.874,-5121.9243 92,-5063.1468 92,-5011 92,-5011 92,-5011 92,-4729 92,-4671.9505 108.6054,-4606.9649 119.5676,-4570.2799\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"119.5676,-5169.7201 112.4457,-5161.3817 118.1665,-5164.9204 116.7654,-5160.1207 116.7654,-5160.1207 116.7654,-5160.1207 118.1665,-5164.9204 121.0852,-5158.8597 119.5676,-5169.7201 119.5676,-5169.7201\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus6&#45;&gt;stage2_unit4_bn3 -->\\n\",\n       \"<g id=\\\"edge65\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus6&#45;&gt;stage2_unit4_bn3</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M144.5746,-5160.4734C148.1213,-5151.6999 151.8327,-5142.5191 155.1922,-5134.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"140.8151,-5169.7731 140.3911,-5158.8154 142.6891,-5165.1375 144.5631,-5160.502 144.5631,-5160.502 144.5631,-5160.502 142.6891,-5165.1375 148.7351,-5162.1886 140.8151,-5169.7731 140.8151,-5169.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit1_bn1 -->\\n\",\n       \"<g id=\\\"node61\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit1_bn1</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"139,-5322 45,-5322 45,-5264 139,-5264 139,-5322\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"92\\\" y=\\\"-5289.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage3_unit1_bn1</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit1_bn1&#45;&gt;_plus6 -->\\n\",\n       \"<g id=\\\"edge67\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit1_bn1&#45;&gt;_plus6</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M107.275,-5254.1932C110.695,-5245.5047 114.2666,-5236.4307 117.5029,-5228.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"103.5042,-5263.7731 102.9796,-5252.8198 105.3356,-5259.1205 107.1669,-5254.468 107.1669,-5254.468 107.1669,-5254.468 105.3356,-5259.1205 111.3542,-5256.1162 103.5042,-5263.7731 103.5042,-5263.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit1_conv1 -->\\n\",\n       \"<g id=\\\"node62\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit1_conv1</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"134,-5416 40,-5416 40,-5358 134,-5358 134,-5416\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"87\\\" y=\\\"-5390.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"87\\\" y=\\\"-5375.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 256</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit1_conv1&#45;&gt;stage3_unit1_bn1 -->\\n\",\n       \"<g id=\\\"edge68\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit1_conv1&#45;&gt;stage3_unit1_bn1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M89.094,-5347.6321C89.5471,-5339.1148 90.0184,-5330.2539 90.4463,-5322.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"88.5546,-5357.7731 84.5922,-5347.5481 88.8203,-5352.7801 89.0859,-5347.7872 89.0859,-5347.7872 89.0859,-5347.7872 88.8203,-5352.7801 93.5795,-5348.0263 88.5546,-5357.7731 88.5546,-5357.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit1_bn2 -->\\n\",\n       \"<g id=\\\"node63\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit1_bn2</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"125,-5510 31,-5510 31,-5452 125,-5452 125,-5510\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"78\\\" y=\\\"-5477.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage3_unit1_bn2</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit1_bn2&#45;&gt;stage3_unit1_conv1 -->\\n\",\n       \"<g id=\\\"edge69\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit1_bn2&#45;&gt;stage3_unit1_conv1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M81.7693,-5441.6321C82.5848,-5433.1148 83.4331,-5424.2539 84.2034,-5416.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"80.7983,-5451.7731 77.272,-5441.3897 81.2749,-5446.7958 81.7515,-5441.8186 81.7515,-5441.8186 81.7515,-5441.8186 81.2749,-5446.7958 86.231,-5442.2475 80.7983,-5451.7731 80.7983,-5451.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit1_relu1 -->\\n\",\n       \"<g id=\\\"node64\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit1_relu1</title>\\n\",\n       \"<polygon fill=\\\"#ffffb3\\\" stroke=\\\"#000000\\\" points=\\\"107,-5604 13,-5604 13,-5546 107,-5546 107,-5604\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"60\\\" y=\\\"-5578.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">LeakyReLU</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"60\\\" y=\\\"-5563.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">prelu</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit1_relu1&#45;&gt;stage3_unit1_bn2 -->\\n\",\n       \"<g id=\\\"edge70\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit1_relu1&#45;&gt;stage3_unit1_bn2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M67.4848,-5535.9128C69.1322,-5527.3096 70.8493,-5518.3423 72.4068,-5510.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"65.5966,-5545.7731 63.0577,-5535.1052 66.537,-5540.8623 67.4774,-5535.9515 67.4774,-5535.9515 67.4774,-5535.9515 66.537,-5540.8623 71.8971,-5536.7979 65.5966,-5545.7731 65.5966,-5545.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit1_conv2 -->\\n\",\n       \"<g id=\\\"node65\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit1_conv2</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"107,-5698 13,-5698 13,-5640 107,-5640 107,-5698\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"60\\\" y=\\\"-5672.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"60\\\" y=\\\"-5657.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/2x2, 256</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit1_conv2&#45;&gt;stage3_unit1_relu1 -->\\n\",\n       \"<g id=\\\"edge71\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit1_conv2&#45;&gt;stage3_unit1_relu1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M60,-5629.6321C60,-5621.1148 60,-5612.2539 60,-5604.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"60,-5639.7731 55.5001,-5629.773 60,-5634.7731 60.0001,-5629.7731 60.0001,-5629.7731 60.0001,-5629.7731 60,-5634.7731 64.5001,-5629.7731 60,-5639.7731 60,-5639.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit1_bn3 -->\\n\",\n       \"<g id=\\\"node66\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit1_bn3</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"107,-5792 13,-5792 13,-5734 107,-5734 107,-5792\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"60\\\" y=\\\"-5759.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage3_unit1_bn3</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit1_bn3&#45;&gt;stage3_unit1_conv2 -->\\n\",\n       \"<g id=\\\"edge72\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit1_bn3&#45;&gt;stage3_unit1_conv2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M60,-5723.6321C60,-5715.1148 60,-5706.2539 60,-5698.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"60,-5733.7731 55.5001,-5723.773 60,-5728.7731 60.0001,-5723.7731 60.0001,-5723.7731 60.0001,-5723.7731 60,-5728.7731 64.5001,-5723.7731 60,-5733.7731 60,-5733.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit1_conv1sc -->\\n\",\n       \"<g id=\\\"node67\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit1_conv1sc</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"219,-5604 125,-5604 125,-5546 219,-5546 219,-5604\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"172\\\" y=\\\"-5578.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"172\\\" y=\\\"-5563.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">1x1/2x2, 256</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit1_conv1sc&#45;&gt;_plus6 -->\\n\",\n       \"<g id=\\\"edge73\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit1_conv1sc&#45;&gt;_plus6</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M171.5657,-5535.5116C170.3257,-5476.1237 165.6279,-5360.4886 148,-5264 145.8237,-5252.0875 142.3229,-5239.2386 138.9157,-5228.1352\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"171.7537,-5545.5841 167.0677,-5535.6699 171.6603,-5540.585 171.567,-5535.5859 171.567,-5535.5859 171.567,-5535.5859 171.6603,-5540.585 176.0662,-5535.5018 171.7537,-5545.5841 171.7537,-5545.5841\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit1_sc -->\\n\",\n       \"<g id=\\\"node68\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit1_sc</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"219,-5792 125,-5792 125,-5734 219,-5734 219,-5792\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"172\\\" y=\\\"-5759.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage3_unit1_sc</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit1_sc&#45;&gt;stage3_unit1_conv1sc -->\\n\",\n       \"<g id=\\\"edge74\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit1_sc&#45;&gt;stage3_unit1_conv1sc</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M172,-5723.5685C172,-5687.9118 172,-5636.2305 172,-5604.1687\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"172,-5733.625 167.5001,-5723.625 172,-5728.625 172.0001,-5723.625 172.0001,-5723.625 172.0001,-5723.625 172,-5728.625 176.5001,-5723.625 172,-5733.625 172,-5733.625\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus7 -->\\n\",\n       \"<g id=\\\"node69\\\" class=\\\"node\\\">\\n\",\n       \"<title>_plus7</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"163,-5886 69,-5886 69,-5828 163,-5828 163,-5886\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"116\\\" y=\\\"-5853.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">_plus7</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus7&#45;&gt;stage3_unit1_bn3 -->\\n\",\n       \"<g id=\\\"edge75\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus7&#45;&gt;stage3_unit1_bn3</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M93.3812,-5819.0328C88.0542,-5810.091 82.4571,-5800.6959 77.401,-5792.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"98.5882,-5827.7731 89.6042,-5821.4852 96.0292,-5823.4776 93.4701,-5819.1821 93.4701,-5819.1821 93.4701,-5819.1821 96.0292,-5823.4776 97.3361,-5816.8789 98.5882,-5827.7731 98.5882,-5827.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus7&#45;&gt;stage3_unit1_sc -->\\n\",\n       \"<g id=\\\"edge76\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus7&#45;&gt;stage3_unit1_sc</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M138.6188,-5819.0328C143.9458,-5810.091 149.5429,-5800.6959 154.599,-5792.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"133.4118,-5827.7731 134.6639,-5816.8789 135.9708,-5823.4776 138.5299,-5819.1821 138.5299,-5819.1821 138.5299,-5819.1821 135.9708,-5823.4776 142.3958,-5821.4852 133.4118,-5827.7731 133.4118,-5827.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit2_bn1 -->\\n\",\n       \"<g id=\\\"node70\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit2_bn1</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"201,-5980 107,-5980 107,-5922 201,-5922 201,-5980\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-5947.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage3_unit2_bn1</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit2_bn1&#45;&gt;_plus7 -->\\n\",\n       \"<g id=\\\"edge77\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit2_bn1&#45;&gt;_plus7</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M138.4254,-5912.4734C134.8787,-5903.6999 131.1673,-5894.5191 127.8078,-5886.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"142.1849,-5921.7731 134.2649,-5914.1886 140.3109,-5917.1375 138.4369,-5912.502 138.4369,-5912.502 138.4369,-5912.502 140.3109,-5917.1375 142.6089,-5910.8154 142.1849,-5921.7731 142.1849,-5921.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit2_conv1 -->\\n\",\n       \"<g id=\\\"node71\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit2_conv1</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"201,-6074 107,-6074 107,-6016 201,-6016 201,-6074\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-6048.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-6033.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 256</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit2_conv1&#45;&gt;stage3_unit2_bn1 -->\\n\",\n       \"<g id=\\\"edge78\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit2_conv1&#45;&gt;stage3_unit2_bn1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-6005.6321C154,-5997.1148 154,-5988.2539 154,-5980.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-6015.7731 149.5001,-6005.773 154,-6010.7731 154.0001,-6005.7731 154.0001,-6005.7731 154.0001,-6005.7731 154,-6010.7731 158.5001,-6005.7731 154,-6015.7731 154,-6015.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit2_bn2 -->\\n\",\n       \"<g id=\\\"node72\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit2_bn2</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"201,-6168 107,-6168 107,-6110 201,-6110 201,-6168\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-6135.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage3_unit2_bn2</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit2_bn2&#45;&gt;stage3_unit2_conv1 -->\\n\",\n       \"<g id=\\\"edge79\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit2_bn2&#45;&gt;stage3_unit2_conv1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-6099.6321C154,-6091.1148 154,-6082.2539 154,-6074.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-6109.7731 149.5001,-6099.773 154,-6104.7731 154.0001,-6099.7731 154.0001,-6099.7731 154.0001,-6099.7731 154,-6104.7731 158.5001,-6099.7731 154,-6109.7731 154,-6109.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit2_relu1 -->\\n\",\n       \"<g id=\\\"node73\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit2_relu1</title>\\n\",\n       \"<polygon fill=\\\"#ffffb3\\\" stroke=\\\"#000000\\\" points=\\\"201,-6262 107,-6262 107,-6204 201,-6204 201,-6262\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-6236.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">LeakyReLU</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-6221.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">prelu</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit2_relu1&#45;&gt;stage3_unit2_bn2 -->\\n\",\n       \"<g id=\\\"edge80\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit2_relu1&#45;&gt;stage3_unit2_bn2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-6193.6321C154,-6185.1148 154,-6176.2539 154,-6168.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-6203.7731 149.5001,-6193.773 154,-6198.7731 154.0001,-6193.7731 154.0001,-6193.7731 154.0001,-6193.7731 154,-6198.7731 158.5001,-6193.7731 154,-6203.7731 154,-6203.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit2_conv2 -->\\n\",\n       \"<g id=\\\"node74\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit2_conv2</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"201,-6356 107,-6356 107,-6298 201,-6298 201,-6356\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-6330.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-6315.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 256</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit2_conv2&#45;&gt;stage3_unit2_relu1 -->\\n\",\n       \"<g id=\\\"edge81\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit2_conv2&#45;&gt;stage3_unit2_relu1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-6287.6321C154,-6279.1148 154,-6270.2539 154,-6262.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-6297.7731 149.5001,-6287.773 154,-6292.7731 154.0001,-6287.7731 154.0001,-6287.7731 154.0001,-6287.7731 154,-6292.7731 158.5001,-6287.7731 154,-6297.7731 154,-6297.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit2_bn3 -->\\n\",\n       \"<g id=\\\"node75\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit2_bn3</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"201,-6450 107,-6450 107,-6392 201,-6392 201,-6450\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-6417.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage3_unit2_bn3</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit2_bn3&#45;&gt;stage3_unit2_conv2 -->\\n\",\n       \"<g id=\\\"edge82\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit2_bn3&#45;&gt;stage3_unit2_conv2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-6381.6321C154,-6373.1148 154,-6364.2539 154,-6356.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-6391.7731 149.5001,-6381.773 154,-6386.7731 154.0001,-6381.7731 154.0001,-6381.7731 154.0001,-6381.7731 154,-6386.7731 158.5001,-6381.7731 154,-6391.7731 154,-6391.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus8 -->\\n\",\n       \"<g id=\\\"node76\\\" class=\\\"node\\\">\\n\",\n       \"<title>_plus8</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"163,-6544 69,-6544 69,-6486 163,-6486 163,-6544\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"116\\\" y=\\\"-6511.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">_plus8</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus8&#45;&gt;_plus7 -->\\n\",\n       \"<g id=\\\"edge84\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus8&#45;&gt;_plus7</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M103.6305,-6475.6582C92.874,-6437.9243 79,-6379.1468 79,-6327 79,-6327 79,-6327 79,-6045 79,-5987.9505 95.6054,-5922.9649 106.5676,-5886.2799\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"106.5676,-6485.7201 99.4457,-6477.3817 105.1665,-6480.9204 103.7654,-6476.1207 103.7654,-6476.1207 103.7654,-6476.1207 105.1665,-6480.9204 108.0852,-6474.8597 106.5676,-6485.7201 106.5676,-6485.7201\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus8&#45;&gt;stage3_unit2_bn3 -->\\n\",\n       \"<g id=\\\"edge83\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus8&#45;&gt;stage3_unit2_bn3</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M131.5746,-6476.4734C135.1213,-6467.6999 138.8327,-6458.5191 142.1922,-6450.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"127.8151,-6485.7731 127.3911,-6474.8154 129.6891,-6481.1375 131.5631,-6476.502 131.5631,-6476.502 131.5631,-6476.502 129.6891,-6481.1375 135.7351,-6478.1886 127.8151,-6485.7731 127.8151,-6485.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit3_bn1 -->\\n\",\n       \"<g id=\\\"node77\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit3_bn1</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"201,-6638 107,-6638 107,-6580 201,-6580 201,-6638\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-6605.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage3_unit3_bn1</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit3_bn1&#45;&gt;_plus8 -->\\n\",\n       \"<g id=\\\"edge85\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit3_bn1&#45;&gt;_plus8</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M138.4254,-6570.4734C134.8787,-6561.6999 131.1673,-6552.5191 127.8078,-6544.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"142.1849,-6579.7731 134.2649,-6572.1886 140.3109,-6575.1375 138.4369,-6570.502 138.4369,-6570.502 138.4369,-6570.502 140.3109,-6575.1375 142.6089,-6568.8154 142.1849,-6579.7731 142.1849,-6579.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit3_conv1 -->\\n\",\n       \"<g id=\\\"node78\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit3_conv1</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"201,-6732 107,-6732 107,-6674 201,-6674 201,-6732\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-6706.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-6691.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 256</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit3_conv1&#45;&gt;stage3_unit3_bn1 -->\\n\",\n       \"<g id=\\\"edge86\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit3_conv1&#45;&gt;stage3_unit3_bn1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-6663.6321C154,-6655.1148 154,-6646.2539 154,-6638.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-6673.7731 149.5001,-6663.773 154,-6668.7731 154.0001,-6663.7731 154.0001,-6663.7731 154.0001,-6663.7731 154,-6668.7731 158.5001,-6663.7731 154,-6673.7731 154,-6673.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit3_bn2 -->\\n\",\n       \"<g id=\\\"node79\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit3_bn2</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"201,-6826 107,-6826 107,-6768 201,-6768 201,-6826\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-6793.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage3_unit3_bn2</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit3_bn2&#45;&gt;stage3_unit3_conv1 -->\\n\",\n       \"<g id=\\\"edge87\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit3_bn2&#45;&gt;stage3_unit3_conv1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-6757.6321C154,-6749.1148 154,-6740.2539 154,-6732.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-6767.7731 149.5001,-6757.773 154,-6762.7731 154.0001,-6757.7731 154.0001,-6757.7731 154.0001,-6757.7731 154,-6762.7731 158.5001,-6757.7731 154,-6767.7731 154,-6767.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit3_relu1 -->\\n\",\n       \"<g id=\\\"node80\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit3_relu1</title>\\n\",\n       \"<polygon fill=\\\"#ffffb3\\\" stroke=\\\"#000000\\\" points=\\\"201,-6920 107,-6920 107,-6862 201,-6862 201,-6920\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-6894.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">LeakyReLU</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-6879.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">prelu</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit3_relu1&#45;&gt;stage3_unit3_bn2 -->\\n\",\n       \"<g id=\\\"edge88\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit3_relu1&#45;&gt;stage3_unit3_bn2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-6851.6321C154,-6843.1148 154,-6834.2539 154,-6826.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-6861.7731 149.5001,-6851.773 154,-6856.7731 154.0001,-6851.7731 154.0001,-6851.7731 154.0001,-6851.7731 154,-6856.7731 158.5001,-6851.7731 154,-6861.7731 154,-6861.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit3_conv2 -->\\n\",\n       \"<g id=\\\"node81\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit3_conv2</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"201,-7014 107,-7014 107,-6956 201,-6956 201,-7014\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-6988.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-6973.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 256</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit3_conv2&#45;&gt;stage3_unit3_relu1 -->\\n\",\n       \"<g id=\\\"edge89\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit3_conv2&#45;&gt;stage3_unit3_relu1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-6945.6321C154,-6937.1148 154,-6928.2539 154,-6920.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-6955.7731 149.5001,-6945.773 154,-6950.7731 154.0001,-6945.7731 154.0001,-6945.7731 154.0001,-6945.7731 154,-6950.7731 158.5001,-6945.7731 154,-6955.7731 154,-6955.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit3_bn3 -->\\n\",\n       \"<g id=\\\"node82\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit3_bn3</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"201,-7108 107,-7108 107,-7050 201,-7050 201,-7108\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-7075.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage3_unit3_bn3</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit3_bn3&#45;&gt;stage3_unit3_conv2 -->\\n\",\n       \"<g id=\\\"edge90\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit3_bn3&#45;&gt;stage3_unit3_conv2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-7039.6321C154,-7031.1148 154,-7022.2539 154,-7014.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-7049.7731 149.5001,-7039.773 154,-7044.7731 154.0001,-7039.7731 154.0001,-7039.7731 154.0001,-7039.7731 154,-7044.7731 158.5001,-7039.7731 154,-7049.7731 154,-7049.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus9 -->\\n\",\n       \"<g id=\\\"node83\\\" class=\\\"node\\\">\\n\",\n       \"<title>_plus9</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"163,-7202 69,-7202 69,-7144 163,-7144 163,-7202\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"116\\\" y=\\\"-7169.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">_plus9</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus9&#45;&gt;_plus8 -->\\n\",\n       \"<g id=\\\"edge92\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus9&#45;&gt;_plus8</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M103.6305,-7133.6582C92.874,-7095.9243 79,-7037.1468 79,-6985 79,-6985 79,-6985 79,-6703 79,-6645.9505 95.6054,-6580.9649 106.5676,-6544.2799\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"106.5676,-7143.7201 99.4457,-7135.3817 105.1665,-7138.9204 103.7654,-7134.1207 103.7654,-7134.1207 103.7654,-7134.1207 105.1665,-7138.9204 108.0852,-7132.8597 106.5676,-7143.7201 106.5676,-7143.7201\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus9&#45;&gt;stage3_unit3_bn3 -->\\n\",\n       \"<g id=\\\"edge91\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus9&#45;&gt;stage3_unit3_bn3</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M131.5746,-7134.4734C135.1213,-7125.6999 138.8327,-7116.5191 142.1922,-7108.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"127.8151,-7143.7731 127.3911,-7132.8154 129.6891,-7139.1375 131.5631,-7134.502 131.5631,-7134.502 131.5631,-7134.502 129.6891,-7139.1375 135.7351,-7136.1886 127.8151,-7143.7731 127.8151,-7143.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit4_bn1 -->\\n\",\n       \"<g id=\\\"node84\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit4_bn1</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"201,-7296 107,-7296 107,-7238 201,-7238 201,-7296\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-7263.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage3_unit4_bn1</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit4_bn1&#45;&gt;_plus9 -->\\n\",\n       \"<g id=\\\"edge93\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit4_bn1&#45;&gt;_plus9</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M138.4254,-7228.4734C134.8787,-7219.6999 131.1673,-7210.5191 127.8078,-7202.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"142.1849,-7237.7731 134.2649,-7230.1886 140.3109,-7233.1375 138.4369,-7228.502 138.4369,-7228.502 138.4369,-7228.502 140.3109,-7233.1375 142.6089,-7226.8154 142.1849,-7237.7731 142.1849,-7237.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit4_conv1 -->\\n\",\n       \"<g id=\\\"node85\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit4_conv1</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"201,-7390 107,-7390 107,-7332 201,-7332 201,-7390\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-7364.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-7349.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 256</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit4_conv1&#45;&gt;stage3_unit4_bn1 -->\\n\",\n       \"<g id=\\\"edge94\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit4_conv1&#45;&gt;stage3_unit4_bn1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-7321.6321C154,-7313.1148 154,-7304.2539 154,-7296.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-7331.7731 149.5001,-7321.773 154,-7326.7731 154.0001,-7321.7731 154.0001,-7321.7731 154.0001,-7321.7731 154,-7326.7731 158.5001,-7321.7731 154,-7331.7731 154,-7331.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit4_bn2 -->\\n\",\n       \"<g id=\\\"node86\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit4_bn2</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"201,-7484 107,-7484 107,-7426 201,-7426 201,-7484\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-7451.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage3_unit4_bn2</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit4_bn2&#45;&gt;stage3_unit4_conv1 -->\\n\",\n       \"<g id=\\\"edge95\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit4_bn2&#45;&gt;stage3_unit4_conv1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-7415.6321C154,-7407.1148 154,-7398.2539 154,-7390.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-7425.7731 149.5001,-7415.773 154,-7420.7731 154.0001,-7415.7731 154.0001,-7415.7731 154.0001,-7415.7731 154,-7420.7731 158.5001,-7415.7731 154,-7425.7731 154,-7425.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit4_relu1 -->\\n\",\n       \"<g id=\\\"node87\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit4_relu1</title>\\n\",\n       \"<polygon fill=\\\"#ffffb3\\\" stroke=\\\"#000000\\\" points=\\\"201,-7578 107,-7578 107,-7520 201,-7520 201,-7578\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-7552.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">LeakyReLU</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-7537.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">prelu</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit4_relu1&#45;&gt;stage3_unit4_bn2 -->\\n\",\n       \"<g id=\\\"edge96\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit4_relu1&#45;&gt;stage3_unit4_bn2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-7509.6321C154,-7501.1148 154,-7492.2539 154,-7484.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-7519.7731 149.5001,-7509.773 154,-7514.7731 154.0001,-7509.7731 154.0001,-7509.7731 154.0001,-7509.7731 154,-7514.7731 158.5001,-7509.7731 154,-7519.7731 154,-7519.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit4_conv2 -->\\n\",\n       \"<g id=\\\"node88\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit4_conv2</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"201,-7672 107,-7672 107,-7614 201,-7614 201,-7672\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-7646.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-7631.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 256</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit4_conv2&#45;&gt;stage3_unit4_relu1 -->\\n\",\n       \"<g id=\\\"edge97\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit4_conv2&#45;&gt;stage3_unit4_relu1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-7603.6321C154,-7595.1148 154,-7586.2539 154,-7578.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-7613.7731 149.5001,-7603.773 154,-7608.7731 154.0001,-7603.7731 154.0001,-7603.7731 154.0001,-7603.7731 154,-7608.7731 158.5001,-7603.7731 154,-7613.7731 154,-7613.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit4_bn3 -->\\n\",\n       \"<g id=\\\"node89\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit4_bn3</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"201,-7766 107,-7766 107,-7708 201,-7708 201,-7766\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-7733.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage3_unit4_bn3</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit4_bn3&#45;&gt;stage3_unit4_conv2 -->\\n\",\n       \"<g id=\\\"edge98\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit4_bn3&#45;&gt;stage3_unit4_conv2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-7697.6321C154,-7689.1148 154,-7680.2539 154,-7672.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-7707.7731 149.5001,-7697.773 154,-7702.7731 154.0001,-7697.7731 154.0001,-7697.7731 154.0001,-7697.7731 154,-7702.7731 158.5001,-7697.7731 154,-7707.7731 154,-7707.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus10 -->\\n\",\n       \"<g id=\\\"node90\\\" class=\\\"node\\\">\\n\",\n       \"<title>_plus10</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"163,-7860 69,-7860 69,-7802 163,-7802 163,-7860\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"116\\\" y=\\\"-7827.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">_plus10</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus10&#45;&gt;_plus9 -->\\n\",\n       \"<g id=\\\"edge100\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus10&#45;&gt;_plus9</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M103.6305,-7791.6582C92.874,-7753.9243 79,-7695.1468 79,-7643 79,-7643 79,-7643 79,-7361 79,-7303.9505 95.6054,-7238.9649 106.5676,-7202.2799\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"106.5676,-7801.7201 99.4457,-7793.3817 105.1665,-7796.9204 103.7654,-7792.1207 103.7654,-7792.1207 103.7654,-7792.1207 105.1665,-7796.9204 108.0852,-7790.8597 106.5676,-7801.7201 106.5676,-7801.7201\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus10&#45;&gt;stage3_unit4_bn3 -->\\n\",\n       \"<g id=\\\"edge99\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus10&#45;&gt;stage3_unit4_bn3</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M131.5746,-7792.4734C135.1213,-7783.6999 138.8327,-7774.5191 142.1922,-7766.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"127.8151,-7801.7731 127.3911,-7790.8154 129.6891,-7797.1375 131.5631,-7792.502 131.5631,-7792.502 131.5631,-7792.502 129.6891,-7797.1375 135.7351,-7794.1886 127.8151,-7801.7731 127.8151,-7801.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit5_bn1 -->\\n\",\n       \"<g id=\\\"node91\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit5_bn1</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"201,-7954 107,-7954 107,-7896 201,-7896 201,-7954\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-7921.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage3_unit5_bn1</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit5_bn1&#45;&gt;_plus10 -->\\n\",\n       \"<g id=\\\"edge101\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit5_bn1&#45;&gt;_plus10</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M138.4254,-7886.4734C134.8787,-7877.6999 131.1673,-7868.5191 127.8078,-7860.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"142.1849,-7895.7731 134.2649,-7888.1886 140.3109,-7891.1375 138.4369,-7886.502 138.4369,-7886.502 138.4369,-7886.502 140.3109,-7891.1375 142.6089,-7884.8154 142.1849,-7895.7731 142.1849,-7895.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit5_conv1 -->\\n\",\n       \"<g id=\\\"node92\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit5_conv1</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"201,-8048 107,-8048 107,-7990 201,-7990 201,-8048\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-8022.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-8007.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 256</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit5_conv1&#45;&gt;stage3_unit5_bn1 -->\\n\",\n       \"<g id=\\\"edge102\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit5_conv1&#45;&gt;stage3_unit5_bn1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-7979.6321C154,-7971.1148 154,-7962.2539 154,-7954.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-7989.7731 149.5001,-7979.773 154,-7984.7731 154.0001,-7979.7731 154.0001,-7979.7731 154.0001,-7979.7731 154,-7984.7731 158.5001,-7979.7731 154,-7989.7731 154,-7989.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit5_bn2 -->\\n\",\n       \"<g id=\\\"node93\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit5_bn2</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"201,-8142 107,-8142 107,-8084 201,-8084 201,-8142\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-8109.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage3_unit5_bn2</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit5_bn2&#45;&gt;stage3_unit5_conv1 -->\\n\",\n       \"<g id=\\\"edge103\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit5_bn2&#45;&gt;stage3_unit5_conv1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-8073.6321C154,-8065.1148 154,-8056.2539 154,-8048.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-8083.7731 149.5001,-8073.773 154,-8078.7731 154.0001,-8073.7731 154.0001,-8073.7731 154.0001,-8073.7731 154,-8078.7731 158.5001,-8073.7731 154,-8083.7731 154,-8083.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit5_relu1 -->\\n\",\n       \"<g id=\\\"node94\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit5_relu1</title>\\n\",\n       \"<polygon fill=\\\"#ffffb3\\\" stroke=\\\"#000000\\\" points=\\\"201,-8236 107,-8236 107,-8178 201,-8178 201,-8236\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-8210.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">LeakyReLU</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-8195.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">prelu</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit5_relu1&#45;&gt;stage3_unit5_bn2 -->\\n\",\n       \"<g id=\\\"edge104\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit5_relu1&#45;&gt;stage3_unit5_bn2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-8167.6321C154,-8159.1148 154,-8150.2539 154,-8142.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-8177.7731 149.5001,-8167.773 154,-8172.7731 154.0001,-8167.7731 154.0001,-8167.7731 154.0001,-8167.7731 154,-8172.7731 158.5001,-8167.7731 154,-8177.7731 154,-8177.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit5_conv2 -->\\n\",\n       \"<g id=\\\"node95\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit5_conv2</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"201,-8330 107,-8330 107,-8272 201,-8272 201,-8330\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-8304.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-8289.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 256</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit5_conv2&#45;&gt;stage3_unit5_relu1 -->\\n\",\n       \"<g id=\\\"edge105\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit5_conv2&#45;&gt;stage3_unit5_relu1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-8261.6321C154,-8253.1148 154,-8244.2539 154,-8236.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-8271.7731 149.5001,-8261.773 154,-8266.7731 154.0001,-8261.7731 154.0001,-8261.7731 154.0001,-8261.7731 154,-8266.7731 158.5001,-8261.7731 154,-8271.7731 154,-8271.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit5_bn3 -->\\n\",\n       \"<g id=\\\"node96\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit5_bn3</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"201,-8424 107,-8424 107,-8366 201,-8366 201,-8424\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-8391.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage3_unit5_bn3</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit5_bn3&#45;&gt;stage3_unit5_conv2 -->\\n\",\n       \"<g id=\\\"edge106\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit5_bn3&#45;&gt;stage3_unit5_conv2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-8355.6321C154,-8347.1148 154,-8338.2539 154,-8330.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-8365.7731 149.5001,-8355.773 154,-8360.7731 154.0001,-8355.7731 154.0001,-8355.7731 154.0001,-8355.7731 154,-8360.7731 158.5001,-8355.7731 154,-8365.7731 154,-8365.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus11 -->\\n\",\n       \"<g id=\\\"node97\\\" class=\\\"node\\\">\\n\",\n       \"<title>_plus11</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"163,-8518 69,-8518 69,-8460 163,-8460 163,-8518\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"116\\\" y=\\\"-8485.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">_plus11</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus11&#45;&gt;_plus10 -->\\n\",\n       \"<g id=\\\"edge108\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus11&#45;&gt;_plus10</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M103.6305,-8449.6582C92.874,-8411.9243 79,-8353.1468 79,-8301 79,-8301 79,-8301 79,-8019 79,-7961.9505 95.6054,-7896.9649 106.5676,-7860.2799\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"106.5676,-8459.7201 99.4457,-8451.3817 105.1665,-8454.9204 103.7654,-8450.1207 103.7654,-8450.1207 103.7654,-8450.1207 105.1665,-8454.9204 108.0852,-8448.8597 106.5676,-8459.7201 106.5676,-8459.7201\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus11&#45;&gt;stage3_unit5_bn3 -->\\n\",\n       \"<g id=\\\"edge107\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus11&#45;&gt;stage3_unit5_bn3</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M131.5746,-8450.4734C135.1213,-8441.6999 138.8327,-8432.5191 142.1922,-8424.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"127.8151,-8459.7731 127.3911,-8448.8154 129.6891,-8455.1375 131.5631,-8450.502 131.5631,-8450.502 131.5631,-8450.502 129.6891,-8455.1375 135.7351,-8452.1886 127.8151,-8459.7731 127.8151,-8459.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit6_bn1 -->\\n\",\n       \"<g id=\\\"node98\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit6_bn1</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"201,-8612 107,-8612 107,-8554 201,-8554 201,-8612\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-8579.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage3_unit6_bn1</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit6_bn1&#45;&gt;_plus11 -->\\n\",\n       \"<g id=\\\"edge109\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit6_bn1&#45;&gt;_plus11</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M138.4254,-8544.4734C134.8787,-8535.6999 131.1673,-8526.5191 127.8078,-8518.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"142.1849,-8553.7731 134.2649,-8546.1886 140.3109,-8549.1375 138.4369,-8544.502 138.4369,-8544.502 138.4369,-8544.502 140.3109,-8549.1375 142.6089,-8542.8154 142.1849,-8553.7731 142.1849,-8553.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit6_conv1 -->\\n\",\n       \"<g id=\\\"node99\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit6_conv1</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"201,-8706 107,-8706 107,-8648 201,-8648 201,-8706\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-8680.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-8665.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 256</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit6_conv1&#45;&gt;stage3_unit6_bn1 -->\\n\",\n       \"<g id=\\\"edge110\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit6_conv1&#45;&gt;stage3_unit6_bn1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-8637.6321C154,-8629.1148 154,-8620.2539 154,-8612.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-8647.7731 149.5001,-8637.773 154,-8642.7731 154.0001,-8637.7731 154.0001,-8637.7731 154.0001,-8637.7731 154,-8642.7731 158.5001,-8637.7731 154,-8647.7731 154,-8647.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit6_bn2 -->\\n\",\n       \"<g id=\\\"node100\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit6_bn2</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"201,-8800 107,-8800 107,-8742 201,-8742 201,-8800\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-8767.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage3_unit6_bn2</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit6_bn2&#45;&gt;stage3_unit6_conv1 -->\\n\",\n       \"<g id=\\\"edge111\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit6_bn2&#45;&gt;stage3_unit6_conv1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-8731.6321C154,-8723.1148 154,-8714.2539 154,-8706.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-8741.7731 149.5001,-8731.773 154,-8736.7731 154.0001,-8731.7731 154.0001,-8731.7731 154.0001,-8731.7731 154,-8736.7731 158.5001,-8731.7731 154,-8741.7731 154,-8741.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit6_relu1 -->\\n\",\n       \"<g id=\\\"node101\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit6_relu1</title>\\n\",\n       \"<polygon fill=\\\"#ffffb3\\\" stroke=\\\"#000000\\\" points=\\\"201,-8894 107,-8894 107,-8836 201,-8836 201,-8894\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-8868.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">LeakyReLU</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-8853.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">prelu</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit6_relu1&#45;&gt;stage3_unit6_bn2 -->\\n\",\n       \"<g id=\\\"edge112\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit6_relu1&#45;&gt;stage3_unit6_bn2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-8825.6321C154,-8817.1148 154,-8808.2539 154,-8800.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-8835.7731 149.5001,-8825.773 154,-8830.7731 154.0001,-8825.7731 154.0001,-8825.7731 154.0001,-8825.7731 154,-8830.7731 158.5001,-8825.7731 154,-8835.7731 154,-8835.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit6_conv2 -->\\n\",\n       \"<g id=\\\"node102\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit6_conv2</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"201,-8988 107,-8988 107,-8930 201,-8930 201,-8988\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-8962.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-8947.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 256</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit6_conv2&#45;&gt;stage3_unit6_relu1 -->\\n\",\n       \"<g id=\\\"edge113\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit6_conv2&#45;&gt;stage3_unit6_relu1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-8919.6321C154,-8911.1148 154,-8902.2539 154,-8894.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-8929.7731 149.5001,-8919.773 154,-8924.7731 154.0001,-8919.7731 154.0001,-8919.7731 154.0001,-8919.7731 154,-8924.7731 158.5001,-8919.7731 154,-8929.7731 154,-8929.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit6_bn3 -->\\n\",\n       \"<g id=\\\"node103\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage3_unit6_bn3</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"201,-9082 107,-9082 107,-9024 201,-9024 201,-9082\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"154\\\" y=\\\"-9049.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage3_unit6_bn3</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage3_unit6_bn3&#45;&gt;stage3_unit6_conv2 -->\\n\",\n       \"<g id=\\\"edge114\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage3_unit6_bn3&#45;&gt;stage3_unit6_conv2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M154,-9013.6321C154,-9005.1148 154,-8996.2539 154,-8988.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"154,-9023.7731 149.5001,-9013.773 154,-9018.7731 154.0001,-9013.7731 154.0001,-9013.7731 154.0001,-9013.7731 154,-9018.7731 158.5001,-9013.7731 154,-9023.7731 154,-9023.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus12 -->\\n\",\n       \"<g id=\\\"node104\\\" class=\\\"node\\\">\\n\",\n       \"<title>_plus12</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"163,-9176 69,-9176 69,-9118 163,-9118 163,-9176\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"116\\\" y=\\\"-9143.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">_plus12</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus12&#45;&gt;_plus11 -->\\n\",\n       \"<g id=\\\"edge116\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus12&#45;&gt;_plus11</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M103.6305,-9107.6582C92.874,-9069.9243 79,-9011.1468 79,-8959 79,-8959 79,-8959 79,-8677 79,-8619.9505 95.6054,-8554.9649 106.5676,-8518.2799\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"106.5676,-9117.7201 99.4457,-9109.3817 105.1665,-9112.9204 103.7654,-9108.1207 103.7654,-9108.1207 103.7654,-9108.1207 105.1665,-9112.9204 108.0852,-9106.8597 106.5676,-9117.7201 106.5676,-9117.7201\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus12&#45;&gt;stage3_unit6_bn3 -->\\n\",\n       \"<g id=\\\"edge115\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus12&#45;&gt;stage3_unit6_bn3</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M131.5746,-9108.4734C135.1213,-9099.6999 138.8327,-9090.5191 142.1922,-9082.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"127.8151,-9117.7731 127.3911,-9106.8154 129.6891,-9113.1375 131.5631,-9108.502 131.5631,-9108.502 131.5631,-9108.502 129.6891,-9113.1375 135.7351,-9110.1886 127.8151,-9117.7731 127.8151,-9117.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit1_bn1 -->\\n\",\n       \"<g id=\\\"node105\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit1_bn1</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"126,-9270 32,-9270 32,-9212 126,-9212 126,-9270\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"79\\\" y=\\\"-9237.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage4_unit1_bn1</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit1_bn1&#45;&gt;_plus12 -->\\n\",\n       \"<g id=\\\"edge117\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit1_bn1&#45;&gt;_plus12</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M94.275,-9202.1932C97.695,-9193.5047 101.2666,-9184.4307 104.5029,-9176.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"90.5042,-9211.7731 89.9796,-9200.8198 92.3356,-9207.1205 94.1669,-9202.468 94.1669,-9202.468 94.1669,-9202.468 92.3356,-9207.1205 98.3542,-9204.1162 90.5042,-9211.7731 90.5042,-9211.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit1_conv1 -->\\n\",\n       \"<g id=\\\"node106\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit1_conv1</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"121,-9364 27,-9364 27,-9306 121,-9306 121,-9364\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"74\\\" y=\\\"-9338.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"74\\\" y=\\\"-9323.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 512</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit1_conv1&#45;&gt;stage4_unit1_bn1 -->\\n\",\n       \"<g id=\\\"edge118\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit1_conv1&#45;&gt;stage4_unit1_bn1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M76.094,-9295.6321C76.5471,-9287.1148 77.0184,-9278.2539 77.4463,-9270.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"75.5546,-9305.7731 71.5922,-9295.5481 75.8203,-9300.7801 76.0859,-9295.7872 76.0859,-9295.7872 76.0859,-9295.7872 75.8203,-9300.7801 80.5795,-9296.0263 75.5546,-9305.7731 75.5546,-9305.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit1_bn2 -->\\n\",\n       \"<g id=\\\"node107\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit1_bn2</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"112,-9458 18,-9458 18,-9400 112,-9400 112,-9458\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"65\\\" y=\\\"-9425.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage4_unit1_bn2</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit1_bn2&#45;&gt;stage4_unit1_conv1 -->\\n\",\n       \"<g id=\\\"edge119\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit1_bn2&#45;&gt;stage4_unit1_conv1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M68.7693,-9389.6321C69.5848,-9381.1148 70.4331,-9372.2539 71.2034,-9364.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"67.7983,-9399.7731 64.272,-9389.3897 68.2749,-9394.7958 68.7515,-9389.8186 68.7515,-9389.8186 68.7515,-9389.8186 68.2749,-9394.7958 73.231,-9390.2475 67.7983,-9399.7731 67.7983,-9399.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit1_relu1 -->\\n\",\n       \"<g id=\\\"node108\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit1_relu1</title>\\n\",\n       \"<polygon fill=\\\"#ffffb3\\\" stroke=\\\"#000000\\\" points=\\\"94,-9552 0,-9552 0,-9494 94,-9494 94,-9552\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"47\\\" y=\\\"-9526.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">LeakyReLU</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"47\\\" y=\\\"-9511.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">prelu</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit1_relu1&#45;&gt;stage4_unit1_bn2 -->\\n\",\n       \"<g id=\\\"edge120\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit1_relu1&#45;&gt;stage4_unit1_bn2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M54.4848,-9483.9128C56.1322,-9475.3096 57.8493,-9466.3423 59.4068,-9458.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"52.5966,-9493.7731 50.0577,-9483.1052 53.537,-9488.8623 54.4774,-9483.9515 54.4774,-9483.9515 54.4774,-9483.9515 53.537,-9488.8623 58.8971,-9484.7979 52.5966,-9493.7731 52.5966,-9493.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit1_conv2 -->\\n\",\n       \"<g id=\\\"node109\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit1_conv2</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"94,-9646 0,-9646 0,-9588 94,-9588 94,-9646\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"47\\\" y=\\\"-9620.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"47\\\" y=\\\"-9605.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/2x2, 512</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit1_conv2&#45;&gt;stage4_unit1_relu1 -->\\n\",\n       \"<g id=\\\"edge121\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit1_conv2&#45;&gt;stage4_unit1_relu1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M47,-9577.6321C47,-9569.1148 47,-9560.2539 47,-9552.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"47,-9587.7731 42.5001,-9577.773 47,-9582.7731 47.0001,-9577.7731 47.0001,-9577.7731 47.0001,-9577.7731 47,-9582.7731 51.5001,-9577.7731 47,-9587.7731 47,-9587.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit1_bn3 -->\\n\",\n       \"<g id=\\\"node110\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit1_bn3</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"94,-9740 0,-9740 0,-9682 94,-9682 94,-9740\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"47\\\" y=\\\"-9707.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage4_unit1_bn3</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit1_bn3&#45;&gt;stage4_unit1_conv2 -->\\n\",\n       \"<g id=\\\"edge122\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit1_bn3&#45;&gt;stage4_unit1_conv2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M47,-9671.6321C47,-9663.1148 47,-9654.2539 47,-9646.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"47,-9681.7731 42.5001,-9671.773 47,-9676.7731 47.0001,-9671.7731 47.0001,-9671.7731 47.0001,-9671.7731 47,-9676.7731 51.5001,-9671.7731 47,-9681.7731 47,-9681.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit1_conv1sc -->\\n\",\n       \"<g id=\\\"node111\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit1_conv1sc</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"206,-9552 112,-9552 112,-9494 206,-9494 206,-9552\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"159\\\" y=\\\"-9526.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"159\\\" y=\\\"-9511.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">1x1/2x2, 512</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit1_conv1sc&#45;&gt;_plus12 -->\\n\",\n       \"<g id=\\\"edge123\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit1_conv1sc&#45;&gt;_plus12</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M158.5657,-9483.5116C157.3257,-9424.1237 152.6279,-9308.4886 135,-9212 132.8237,-9200.0875 129.3229,-9187.2386 125.9157,-9176.1352\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"158.7537,-9493.5841 154.0677,-9483.6699 158.6603,-9488.585 158.567,-9483.5859 158.567,-9483.5859 158.567,-9483.5859 158.6603,-9488.585 163.0662,-9483.5018 158.7537,-9493.5841 158.7537,-9493.5841\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit1_sc -->\\n\",\n       \"<g id=\\\"node112\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit1_sc</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"206,-9740 112,-9740 112,-9682 206,-9682 206,-9740\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"159\\\" y=\\\"-9707.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage4_unit1_sc</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit1_sc&#45;&gt;stage4_unit1_conv1sc -->\\n\",\n       \"<g id=\\\"edge124\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit1_sc&#45;&gt;stage4_unit1_conv1sc</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M159,-9671.5685C159,-9635.9118 159,-9584.2305 159,-9552.1687\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"159,-9681.625 154.5001,-9671.625 159,-9676.625 159.0001,-9671.625 159.0001,-9671.625 159.0001,-9671.625 159,-9676.625 163.5001,-9671.625 159,-9681.625 159,-9681.625\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus13 -->\\n\",\n       \"<g id=\\\"node113\\\" class=\\\"node\\\">\\n\",\n       \"<title>_plus13</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"150,-9834 56,-9834 56,-9776 150,-9776 150,-9834\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"103\\\" y=\\\"-9801.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">_plus13</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus13&#45;&gt;stage4_unit1_bn3 -->\\n\",\n       \"<g id=\\\"edge125\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus13&#45;&gt;stage4_unit1_bn3</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M80.3812,-9767.0328C75.0542,-9758.091 69.4571,-9748.6959 64.401,-9740.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"85.5882,-9775.7731 76.6042,-9769.4852 83.0292,-9771.4776 80.4701,-9767.1821 80.4701,-9767.1821 80.4701,-9767.1821 83.0292,-9771.4776 84.3361,-9764.8789 85.5882,-9775.7731 85.5882,-9775.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus13&#45;&gt;stage4_unit1_sc -->\\n\",\n       \"<g id=\\\"edge126\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus13&#45;&gt;stage4_unit1_sc</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M125.6188,-9767.0328C130.9458,-9758.091 136.5429,-9748.6959 141.599,-9740.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"120.4118,-9775.7731 121.6639,-9764.8789 122.9708,-9771.4776 125.5299,-9767.1821 125.5299,-9767.1821 125.5299,-9767.1821 122.9708,-9771.4776 129.3958,-9769.4852 120.4118,-9775.7731 120.4118,-9775.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit2_bn1 -->\\n\",\n       \"<g id=\\\"node114\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit2_bn1</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"188,-9928 94,-9928 94,-9870 188,-9870 188,-9928\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"141\\\" y=\\\"-9895.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage4_unit2_bn1</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit2_bn1&#45;&gt;_plus13 -->\\n\",\n       \"<g id=\\\"edge127\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit2_bn1&#45;&gt;_plus13</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M125.4254,-9860.4734C121.8787,-9851.6999 118.1673,-9842.5191 114.8078,-9834.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"129.1849,-9869.7731 121.2649,-9862.1886 127.3109,-9865.1375 125.4369,-9860.502 125.4369,-9860.502 125.4369,-9860.502 127.3109,-9865.1375 129.6089,-9858.8154 129.1849,-9869.7731 129.1849,-9869.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit2_conv1 -->\\n\",\n       \"<g id=\\\"node115\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit2_conv1</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"188,-10022 94,-10022 94,-9964 188,-9964 188,-10022\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"141\\\" y=\\\"-9996.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"141\\\" y=\\\"-9981.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 512</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit2_conv1&#45;&gt;stage4_unit2_bn1 -->\\n\",\n       \"<g id=\\\"edge128\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit2_conv1&#45;&gt;stage4_unit2_bn1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M141,-9953.6321C141,-9945.1148 141,-9936.2539 141,-9928.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"141,-9963.7731 136.5001,-9953.773 141,-9958.7731 141.0001,-9953.7731 141.0001,-9953.7731 141.0001,-9953.7731 141,-9958.7731 145.5001,-9953.7731 141,-9963.7731 141,-9963.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit2_bn2 -->\\n\",\n       \"<g id=\\\"node116\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit2_bn2</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"188,-10116 94,-10116 94,-10058 188,-10058 188,-10116\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"141\\\" y=\\\"-10083.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage4_unit2_bn2</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit2_bn2&#45;&gt;stage4_unit2_conv1 -->\\n\",\n       \"<g id=\\\"edge129\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit2_bn2&#45;&gt;stage4_unit2_conv1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M141,-10047.6321C141,-10039.1148 141,-10030.2539 141,-10022.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"141,-10057.7731 136.5001,-10047.773 141,-10052.7731 141.0001,-10047.7731 141.0001,-10047.7731 141.0001,-10047.7731 141,-10052.7731 145.5001,-10047.7731 141,-10057.7731 141,-10057.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit2_relu1 -->\\n\",\n       \"<g id=\\\"node117\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit2_relu1</title>\\n\",\n       \"<polygon fill=\\\"#ffffb3\\\" stroke=\\\"#000000\\\" points=\\\"188,-10210 94,-10210 94,-10152 188,-10152 188,-10210\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"141\\\" y=\\\"-10184.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">LeakyReLU</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"141\\\" y=\\\"-10169.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">prelu</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit2_relu1&#45;&gt;stage4_unit2_bn2 -->\\n\",\n       \"<g id=\\\"edge130\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit2_relu1&#45;&gt;stage4_unit2_bn2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M141,-10141.6321C141,-10133.1148 141,-10124.2539 141,-10116.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"141,-10151.7731 136.5001,-10141.773 141,-10146.7731 141.0001,-10141.7731 141.0001,-10141.7731 141.0001,-10141.7731 141,-10146.7731 145.5001,-10141.7731 141,-10151.7731 141,-10151.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit2_conv2 -->\\n\",\n       \"<g id=\\\"node118\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit2_conv2</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"188,-10304 94,-10304 94,-10246 188,-10246 188,-10304\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"141\\\" y=\\\"-10278.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"141\\\" y=\\\"-10263.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 512</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit2_conv2&#45;&gt;stage4_unit2_relu1 -->\\n\",\n       \"<g id=\\\"edge131\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit2_conv2&#45;&gt;stage4_unit2_relu1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M141,-10235.6321C141,-10227.1148 141,-10218.2539 141,-10210.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"141,-10245.7731 136.5001,-10235.773 141,-10240.7731 141.0001,-10235.7731 141.0001,-10235.7731 141.0001,-10235.7731 141,-10240.7731 145.5001,-10235.7731 141,-10245.7731 141,-10245.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit2_bn3 -->\\n\",\n       \"<g id=\\\"node119\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit2_bn3</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"188,-10398 94,-10398 94,-10340 188,-10340 188,-10398\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"141\\\" y=\\\"-10365.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage4_unit2_bn3</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit2_bn3&#45;&gt;stage4_unit2_conv2 -->\\n\",\n       \"<g id=\\\"edge132\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit2_bn3&#45;&gt;stage4_unit2_conv2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M141,-10329.6321C141,-10321.1148 141,-10312.2539 141,-10304.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"141,-10339.7731 136.5001,-10329.773 141,-10334.7731 141.0001,-10329.7731 141.0001,-10329.7731 141.0001,-10329.7731 141,-10334.7731 145.5001,-10329.7731 141,-10339.7731 141,-10339.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus14 -->\\n\",\n       \"<g id=\\\"node120\\\" class=\\\"node\\\">\\n\",\n       \"<title>_plus14</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"150,-10492 56,-10492 56,-10434 150,-10434 150,-10492\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"103\\\" y=\\\"-10459.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">_plus14</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus14&#45;&gt;_plus13 -->\\n\",\n       \"<g id=\\\"edge134\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus14&#45;&gt;_plus13</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M90.6305,-10423.6582C79.874,-10385.9243 66,-10327.1468 66,-10275 66,-10275 66,-10275 66,-9993 66,-9935.9505 82.6054,-9870.9649 93.5676,-9834.2799\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"93.5676,-10433.7201 86.4457,-10425.3817 92.1665,-10428.9204 90.7654,-10424.1207 90.7654,-10424.1207 90.7654,-10424.1207 92.1665,-10428.9204 95.0852,-10422.8597 93.5676,-10433.7201 93.5676,-10433.7201\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus14&#45;&gt;stage4_unit2_bn3 -->\\n\",\n       \"<g id=\\\"edge133\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus14&#45;&gt;stage4_unit2_bn3</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M118.5746,-10424.4734C122.1213,-10415.6999 125.8327,-10406.5191 129.1922,-10398.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"114.8151,-10433.7731 114.3911,-10422.8154 116.6891,-10429.1375 118.5631,-10424.502 118.5631,-10424.502 118.5631,-10424.502 116.6891,-10429.1375 122.7351,-10426.1886 114.8151,-10433.7731 114.8151,-10433.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit3_bn1 -->\\n\",\n       \"<g id=\\\"node121\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit3_bn1</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"188,-10586 94,-10586 94,-10528 188,-10528 188,-10586\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"141\\\" y=\\\"-10553.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage4_unit3_bn1</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit3_bn1&#45;&gt;_plus14 -->\\n\",\n       \"<g id=\\\"edge135\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit3_bn1&#45;&gt;_plus14</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M125.4254,-10518.4734C121.8787,-10509.6999 118.1673,-10500.5191 114.8078,-10492.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"129.1849,-10527.7731 121.2649,-10520.1886 127.3109,-10523.1375 125.4369,-10518.502 125.4369,-10518.502 125.4369,-10518.502 127.3109,-10523.1375 129.6089,-10516.8154 129.1849,-10527.7731 129.1849,-10527.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit3_conv1 -->\\n\",\n       \"<g id=\\\"node122\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit3_conv1</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"188,-10680 94,-10680 94,-10622 188,-10622 188,-10680\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"141\\\" y=\\\"-10654.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"141\\\" y=\\\"-10639.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 512</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit3_conv1&#45;&gt;stage4_unit3_bn1 -->\\n\",\n       \"<g id=\\\"edge136\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit3_conv1&#45;&gt;stage4_unit3_bn1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M141,-10611.6321C141,-10603.1148 141,-10594.2539 141,-10586.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"141,-10621.7731 136.5001,-10611.773 141,-10616.7731 141.0001,-10611.7731 141.0001,-10611.7731 141.0001,-10611.7731 141,-10616.7731 145.5001,-10611.7731 141,-10621.7731 141,-10621.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit3_bn2 -->\\n\",\n       \"<g id=\\\"node123\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit3_bn2</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"188,-10774 94,-10774 94,-10716 188,-10716 188,-10774\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"141\\\" y=\\\"-10741.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage4_unit3_bn2</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit3_bn2&#45;&gt;stage4_unit3_conv1 -->\\n\",\n       \"<g id=\\\"edge137\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit3_bn2&#45;&gt;stage4_unit3_conv1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M141,-10705.6321C141,-10697.1148 141,-10688.2539 141,-10680.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"141,-10715.7731 136.5001,-10705.773 141,-10710.7731 141.0001,-10705.7731 141.0001,-10705.7731 141.0001,-10705.7731 141,-10710.7731 145.5001,-10705.7731 141,-10715.7731 141,-10715.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit3_relu1 -->\\n\",\n       \"<g id=\\\"node124\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit3_relu1</title>\\n\",\n       \"<polygon fill=\\\"#ffffb3\\\" stroke=\\\"#000000\\\" points=\\\"188,-10868 94,-10868 94,-10810 188,-10810 188,-10868\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"141\\\" y=\\\"-10842.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">LeakyReLU</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"141\\\" y=\\\"-10827.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">prelu</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit3_relu1&#45;&gt;stage4_unit3_bn2 -->\\n\",\n       \"<g id=\\\"edge138\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit3_relu1&#45;&gt;stage4_unit3_bn2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M141,-10799.6321C141,-10791.1148 141,-10782.2539 141,-10774.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"141,-10809.7731 136.5001,-10799.773 141,-10804.7731 141.0001,-10799.7731 141.0001,-10799.7731 141.0001,-10799.7731 141,-10804.7731 145.5001,-10799.7731 141,-10809.7731 141,-10809.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit3_conv2 -->\\n\",\n       \"<g id=\\\"node125\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit3_conv2</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"188,-10962 94,-10962 94,-10904 188,-10904 188,-10962\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"141\\\" y=\\\"-10936.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">Convolution</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"141\\\" y=\\\"-10921.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">3x3/1x1, 512</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit3_conv2&#45;&gt;stage4_unit3_relu1 -->\\n\",\n       \"<g id=\\\"edge139\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit3_conv2&#45;&gt;stage4_unit3_relu1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M141,-10893.6321C141,-10885.1148 141,-10876.2539 141,-10868.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"141,-10903.7731 136.5001,-10893.773 141,-10898.7731 141.0001,-10893.7731 141.0001,-10893.7731 141.0001,-10893.7731 141,-10898.7731 145.5001,-10893.7731 141,-10903.7731 141,-10903.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit3_bn3 -->\\n\",\n       \"<g id=\\\"node126\\\" class=\\\"node\\\">\\n\",\n       \"<title>stage4_unit3_bn3</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"188,-11056 94,-11056 94,-10998 188,-10998 188,-11056\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"141\\\" y=\\\"-11023.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">stage4_unit3_bn3</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- stage4_unit3_bn3&#45;&gt;stage4_unit3_conv2 -->\\n\",\n       \"<g id=\\\"edge140\\\" class=\\\"edge\\\">\\n\",\n       \"<title>stage4_unit3_bn3&#45;&gt;stage4_unit3_conv2</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M141,-10987.6321C141,-10979.1148 141,-10970.2539 141,-10962.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"141,-10997.7731 136.5001,-10987.773 141,-10992.7731 141.0001,-10987.7731 141.0001,-10987.7731 141.0001,-10987.7731 141,-10992.7731 145.5001,-10987.7731 141,-10997.7731 141,-10997.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus15 -->\\n\",\n       \"<g id=\\\"node127\\\" class=\\\"node\\\">\\n\",\n       \"<title>_plus15</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"150,-11150 56,-11150 56,-11092 150,-11092 150,-11150\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"103\\\" y=\\\"-11117.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">_plus15</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus15&#45;&gt;_plus14 -->\\n\",\n       \"<g id=\\\"edge142\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus15&#45;&gt;_plus14</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M90.6305,-11081.6582C79.874,-11043.9243 66,-10985.1468 66,-10933 66,-10933 66,-10933 66,-10651 66,-10593.9505 82.6054,-10528.9649 93.5676,-10492.2799\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"93.5676,-11091.7201 86.4457,-11083.3817 92.1665,-11086.9204 90.7654,-11082.1207 90.7654,-11082.1207 90.7654,-11082.1207 92.1665,-11086.9204 95.0852,-11080.8597 93.5676,-11091.7201 93.5676,-11091.7201\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- _plus15&#45;&gt;stage4_unit3_bn3 -->\\n\",\n       \"<g id=\\\"edge141\\\" class=\\\"edge\\\">\\n\",\n       \"<title>_plus15&#45;&gt;stage4_unit3_bn3</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M118.5746,-11082.4734C122.1213,-11073.6999 125.8327,-11064.5191 129.1922,-11056.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"114.8151,-11091.7731 114.3911,-11080.8154 116.6891,-11087.1375 118.5631,-11082.502 118.5631,-11082.502 118.5631,-11082.502 116.6891,-11087.1375 122.7351,-11084.1886 114.8151,-11091.7731 114.8151,-11091.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- bn1 -->\\n\",\n       \"<g id=\\\"node128\\\" class=\\\"node\\\">\\n\",\n       \"<title>bn1</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"150,-11244 56,-11244 56,-11186 150,-11186 150,-11244\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"103\\\" y=\\\"-11211.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">bn1</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- bn1&#45;&gt;_plus15 -->\\n\",\n       \"<g id=\\\"edge143\\\" class=\\\"edge\\\">\\n\",\n       \"<title>bn1&#45;&gt;_plus15</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M103,-11175.6321C103,-11167.1148 103,-11158.2539 103,-11150.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"103,-11185.7731 98.5001,-11175.773 103,-11180.7731 103.0001,-11175.7731 103.0001,-11175.7731 103.0001,-11175.7731 103,-11180.7731 107.5001,-11175.7731 103,-11185.7731 103,-11185.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- dropout0 -->\\n\",\n       \"<g id=\\\"node129\\\" class=\\\"node\\\">\\n\",\n       \"<title>dropout0</title>\\n\",\n       \"<polygon fill=\\\"#fccde5\\\" stroke=\\\"#000000\\\" points=\\\"150,-11338 56,-11338 56,-11280 150,-11280 150,-11338\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"103\\\" y=\\\"-11305.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">dropout0</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- dropout0&#45;&gt;bn1 -->\\n\",\n       \"<g id=\\\"edge144\\\" class=\\\"edge\\\">\\n\",\n       \"<title>dropout0&#45;&gt;bn1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M103,-11269.6321C103,-11261.1148 103,-11252.2539 103,-11244.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"103,-11279.7731 98.5001,-11269.773 103,-11274.7731 103.0001,-11269.7731 103.0001,-11269.7731 103.0001,-11269.7731 103,-11274.7731 107.5001,-11269.7731 103,-11279.7731 103,-11279.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- pre_fc1 -->\\n\",\n       \"<g id=\\\"node130\\\" class=\\\"node\\\">\\n\",\n       \"<title>pre_fc1</title>\\n\",\n       \"<polygon fill=\\\"#fb8072\\\" stroke=\\\"#000000\\\" points=\\\"150,-11432 56,-11432 56,-11374 150,-11374 150,-11432\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"103\\\" y=\\\"-11406.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">FullyConnected</text>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"103\\\" y=\\\"-11391.8\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">512</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- pre_fc1&#45;&gt;dropout0 -->\\n\",\n       \"<g id=\\\"edge145\\\" class=\\\"edge\\\">\\n\",\n       \"<title>pre_fc1&#45;&gt;dropout0</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M103,-11363.6321C103,-11355.1148 103,-11346.2539 103,-11338.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"103,-11373.7731 98.5001,-11363.773 103,-11368.7731 103.0001,-11363.7731 103.0001,-11363.7731 103.0001,-11363.7731 103,-11368.7731 107.5001,-11363.7731 103,-11373.7731 103,-11373.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"<!-- fc1 -->\\n\",\n       \"<g id=\\\"node131\\\" class=\\\"node\\\">\\n\",\n       \"<title>fc1</title>\\n\",\n       \"<polygon fill=\\\"#bebada\\\" stroke=\\\"#000000\\\" points=\\\"150,-11526 56,-11526 56,-11468 150,-11468 150,-11526\\\"/>\\n\",\n       \"<text text-anchor=\\\"middle\\\" x=\\\"103\\\" y=\\\"-11493.3\\\" font-family=\\\"Times,serif\\\" font-size=\\\"14.00\\\" fill=\\\"#000000\\\">fc1</text>\\n\",\n       \"</g>\\n\",\n       \"<!-- fc1&#45;&gt;pre_fc1 -->\\n\",\n       \"<g id=\\\"edge146\\\" class=\\\"edge\\\">\\n\",\n       \"<title>fc1&#45;&gt;pre_fc1</title>\\n\",\n       \"<path fill=\\\"none\\\" stroke=\\\"#000000\\\" d=\\\"M103,-11457.6321C103,-11449.1148 103,-11440.2539 103,-11432.2088\\\"/>\\n\",\n       \"<polygon fill=\\\"#000000\\\" stroke=\\\"#000000\\\" points=\\\"103,-11467.7731 98.5001,-11457.773 103,-11462.7731 103.0001,-11457.7731 103.0001,-11457.7731 103.0001,-11457.7731 103,-11462.7731 107.5001,-11457.7731 103,-11467.7731 103,-11467.7731\\\"/>\\n\",\n       \"</g>\\n\",\n       \"</g>\\n\",\n       \"</svg>\\n\"\n      ],\n      \"text/plain\": [\n       \"<graphviz.dot.Digraph at 0x7f62d03f7f60>\"\n      ]\n     },\n     \"execution_count\": 6,\n     \"metadata\": {},\n     \"output_type\": \"execute_result\"\n    }\n   ],\n   \"source\": [\n    \"mx.viz.plot_network(net.sym)\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 7,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"name\": \"stderr\",\n     \"output_type\": \"stream\",\n     \"text\": [\n      \"100%|██████████| 110/110 [00:00<00:00, 5757.45it/s]\\n\",\n      \"100%|██████████| 166/166 [00:00<00:00, 1132.65it/s]\\n\"\n     ]\n    }\n   ],\n   \"source\": [\n    \"try:\\n\",\n    \"    os.mkdir('Weights')\\n\",\n    \"except:\\n\",\n    \"    pass\\n\",\n    \"for i in tqdm(net.aux_params.keys()):\\n\",\n    \"    np.save('Weights/'+i,net.aux_params[i].asnumpy())\\n\",\n    \"for i in tqdm(net.arg_params.keys()):\\n\",\n    \"    np.save('Weights/'+i,net.arg_params[i].asnumpy())\"\n   ]\n  },\n  {\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"### Testing batch to compare embeddings\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 8,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"img = io.imread('002.jpg')\\n\",\n    \"img = np.transpose(img, [2,0,1])\\n\",\n    \"b = np.random.uniform(0,255,size=(2,3,112,112))\\n\",\n    \"b[0] = img\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 9,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"data = mx.nd.array(b)\\n\",\n    \"db = mx.io.DataBatch(data=(data,))\\n\",\n    \"net.model.forward(db, is_train=False)\"\n   ]\n  },\n  {\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"## Common functions\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 10,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"def conv(x,name,size,inm,outm,stride=1,pad='SAME'):\\n\",\n    \"    W = tf.get_variable('W_'+name,dtype=tf.float32,shape=[size,size,inm,outm],initializer=tf.constant_initializer(np.transpose(np.load('Weights/'+name+'_weight.npy'),[2,3,1,0])))\\n\",\n    \"    return tf.nn.conv2d(x,W,[1,1,stride,stride],pad,name=name,data_format='NCHW')\\n\",\n    \"def bn(x,name):\\n\",\n    \"    return slim.batch_norm(x,center=True,scale=True,epsilon=2e-5,\\n\",\n    \"                                 param_initializers={\\n\",\n    \"                                     'beta':tf.constant_initializer(np.load('Weights/'+name+'_beta.npy')),\\n\",\n    \"                                     'gamma':tf.constant_initializer(np.load('Weights/'+name+'_gamma.npy')),\\n\",\n    \"                                     'moving_mean':tf.constant_initializer(np.load('Weights/'+name+'_moving_mean.npy')),\\n\",\n    \"                                     'moving_variance':tf.constant_initializer(np.load('Weights/'+name+'_moving_var.npy'))},\\n\",\n    \"                                  is_training=train,data_format='NCHW')\\n\",\n    \"def lrelu(x,name,maps):\\n\",\n    \"    a = tf.get_variable('a_'+name,dtype=tf.float32,shape=[maps],initializer=tf.constant_initializer(np.load('Weights/'+name+'_gamma.npy')))\\n\",\n    \"    a = tf.reshape(a,[maps,1,1])\\n\",\n    \"    return tf.nn.relu(x,name=name+'_pos')-tf.nn.relu(-x,name=name+'_neg')*a\\n\",\n    \"def reduce_block(x,name,mapsin,mapsout):\\n\",\n    \"    v1 = bn(x,name+'_bn1')\\n\",\n    \"    v1 = conv(v1,name+'_conv1',3,mapsin,mapsout)\\n\",\n    \"    v1 = bn(v1,name+'_bn2')\\n\",\n    \"    v1 = lrelu(v1,name+'_relu1',mapsout)\\n\",\n    \"    v1 = tf.pad(v1,[[0,0],[0, 0,], [1, 0],[1,0]])\\n\",\n    \"    v1 = conv(v1,name+'_conv2',3,mapsout,mapsout,2,'VALID')\\n\",\n    \"    v1 = bn(v1,name+'_bn3')\\n\",\n    \"    x = conv(x,name+'_conv1sc',1,mapsin,mapsout,2)\\n\",\n    \"    x = bn(x,name+'_sc')\\n\",\n    \"    return x+v1\\n\",\n    \"def block(x,name,maps):\\n\",\n    \"    v1 = bn(x,name+'_bn1')\\n\",\n    \"    v1 = conv(v1,name+'_conv1',3,maps,maps)\\n\",\n    \"    v1 = bn(v1,name+'_bn2')\\n\",\n    \"    v1 = lrelu(v1,name+'_relu1',maps)\\n\",\n    \"    v1 = conv(v1,name+'_conv2',3,maps,maps)\\n\",\n    \"    v1 = bn(v1,name+'_bn3')\\n\",\n    \"    return x+v1\"\n   ]\n  },\n  {\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"## TensorFlow twin \"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 11,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"x = tf.placeholder(tf.float32, [None,3,112,112], 'image_input')\\n\",\n    \"train = tf.placeholder(tf.bool,name='training_mode')\\n\",\n    \"prob = tf.placeholder(tf.float32,name='keep_prob')\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 12,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"name\": \"stdout\",\n     \"output_type\": \"stream\",\n     \"text\": [\n      \"WARNING:tensorflow:From /home/stepan/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\\n\",\n      \"Instructions for updating:\\n\",\n      \"Colocations handled automatically by placer.\\n\"\n     ]\n    }\n   ],\n   \"source\": [\n    \"conv0 = conv(x,'conv0',3,3,64)\\n\",\n    \"bn0 = bn(conv0,'bn0')\\n\",\n    \"relu0 = lrelu(bn0,'relu0',64)\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 13,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"bl1 = reduce_block(relu0,'stage1_unit1',64,64)\\n\",\n    \"for i in range(stage1_length-1):\\n\",\n    \"    bl1 = block(bl1,'stage1_unit'+str(i+2),64)\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 14,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"bl2 = reduce_block(bl1,'stage2_unit1',64,128)\\n\",\n    \"for i in range(stage2_length-1):\\n\",\n    \"    bl2 = block(bl2,'stage2_unit'+str(i+2),128)\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 15,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"bl3 = reduce_block(bl2,'stage3_unit1',128,256)\\n\",\n    \"for i in range(stage3_length-1):\\n\",\n    \"    bl3 = block(bl3,'stage3_unit'+str(i+2),256)\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 16,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"bl4 = reduce_block(bl3,'stage4_unit1',256,512)\\n\",\n    \"for i in range(stage4_length-1):\\n\",\n    \"    bl4 = block(bl4,'stage4_unit'+str(i+2),512)\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 17,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"name\": \"stdout\",\n     \"output_type\": \"stream\",\n     \"text\": [\n      \"WARNING:tensorflow:From <ipython-input-17-900f5f0593f8>:2: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.\\n\",\n      \"Instructions for updating:\\n\",\n      \"Use keras.layers.flatten instead.\\n\",\n      \"WARNING:tensorflow:From <ipython-input-17-900f5f0593f8>:3: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\\n\",\n      \"Instructions for updating:\\n\",\n      \"Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\\n\"\n     ]\n    }\n   ],\n   \"source\": [\n    \"bn1 = bn(bl4,'bn1')\\n\",\n    \"flat = tf.layers.flatten(bn1)\\n\",\n    \"flat = tf.nn.dropout(flat,prob)\\n\",\n    \"Wfc = tf.get_variable('W_fc',dtype=tf.float32,shape=[25088, 512],initializer=tf.constant_initializer(np.transpose(np.load('Weights/pre_fc1_weight.npy'),[1,0])))\\n\",\n    \"fc = tf.matmul(flat, Wfc)\\n\",\n    \"fc = bn(fc,'fc1')\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 18,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"name\": \"stdout\",\n     \"output_type\": \"stream\",\n     \"text\": [\n      \"L1 difference between embeddings: 3.9339066e-06\\n\"\n     ]\n    }\n   ],\n   \"source\": [\n    \"sess.run(tf.global_variables_initializer())\\n\",\n    \"print('L1 difference between embeddings:',np.max(np.abs(sess.run(fc,feed_dict={x:(b - 127.5)*0.0078125,prob:1.0,train:False})-net.model.get_outputs()[0].asnumpy())))\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 19,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"x_norm = tf.nn.l2_normalize(fc,1,name='embedding')\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 20,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"graph = tf.get_default_graph()\\n\",\n    \"input_graph_def = graph.as_graph_def()\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 21,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"name\": \"stdout\",\n     \"output_type\": \"stream\",\n     \"text\": [\n      \"WARNING:tensorflow:From <ipython-input-21-9b691dcbe87e>:5: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.\\n\",\n      \"Instructions for updating:\\n\",\n      \"Use tf.compat.v1.graph_util.convert_variables_to_constants\\n\",\n      \"WARNING:tensorflow:From /home/stepan/.local/lib/python3.6/site-packages/tensorflow/python/framework/graph_util_impl.py:245: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.\\n\",\n      \"Instructions for updating:\\n\",\n      \"Use tf.compat.v1.graph_util.extract_sub_graph\\n\",\n      \"INFO:tensorflow:Froze 275 variables.\\n\",\n      \"INFO:tensorflow:Converted 275 variables to const ops.\\n\"\n     ]\n    }\n   ],\n   \"source\": [\n    \"output_node_names=\\\"embedding\\\"\\n\",\n    \"output_graph_def = tf.graph_util.convert_variables_to_constants(\\n\",\n    \"            sess, # The session\\n\",\n    \"            input_graph_def, # input_graph_def is useful for retrieving the nodes \\n\",\n    \"            output_node_names.split(\\\",\\\")  \\n\",\n    \")\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 22,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"output_graph=\\\"./r34.pb\\\"\\n\",\n    \"with tf.gfile.GFile(output_graph, \\\"wb\\\") as f:\\n\",\n    \"    f.write(output_graph_def.SerializeToString())\"\n   ]\n  },\n  {\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"## PB check\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 23,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"frozen_graph=\\\"./r34.pb\\\"\\n\",\n    \"with tf.gfile.GFile(frozen_graph, \\\"rb\\\") as f:\\n\",\n    \"    graph_def = tf.GraphDef()\\n\",\n    \"    graph_def.ParseFromString(f.read())\\n\",\n    \"\\n\",\n    \"with tf.Graph().as_default() as graph:\\n\",\n    \"      tf.import_graph_def(graph_def,\\n\",\n    \"                          input_map=None,\\n\",\n    \"                          return_elements=None,\\n\",\n    \"                          name=\\\"\\\")\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 24,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"image_input = graph.get_tensor_by_name('image_input:0')\\n\",\n    \"keep_prob = graph.get_tensor_by_name('keep_prob:0')\\n\",\n    \"is_train = graph.get_tensor_by_name('training_mode:0')\\n\",\n    \"embs = graph.get_tensor_by_name('embedding:0')\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 25,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"name\": \"stdout\",\n     \"output_type\": \"stream\",\n     \"text\": [\n      \"L1 difference between normalized embeddings: 2.6077032e-07\\n\"\n     ]\n    }\n   ],\n   \"source\": [\n    \"c = net.model.get_outputs()[0].asnumpy()\\n\",\n    \"for i in range(2):\\n\",\n    \"    c[i] = c[i]/np.sqrt(np.sum(c[i]**2))\\n\",\n    \"sess = tf.Session(graph=graph)\\n\",\n    \"print('L1 difference between normalized embeddings:',np.max(np.abs(sess.run(embs,feed_dict={image_input:(b - 127.5)*0.0078125,keep_prob:1.0,is_train:False})-c)))\"\n   ]\n  }\n ],\n \"metadata\": {\n  \"kernelspec\": {\n   \"display_name\": \"Python 3\",\n   \"language\": \"python\",\n   \"name\": \"python3\"\n  },\n  \"language_info\": {\n   \"codemirror_mode\": {\n    \"name\": \"ipython\",\n    \"version\": 3\n   },\n   \"file_extension\": \".py\",\n   \"mimetype\": \"text/x-python\",\n   \"name\": \"python\",\n   \"nbconvert_exporter\": \"python\",\n   \"pygments_lexer\": \"ipython3\",\n   \"version\": \"3.6.9\"\n  }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 2\n}\n"
  },
  {
    "path": "Utils/README.md",
    "content": "An example notebook of the MX to TF transformation of the ArcFace models. Can be directly applicable for *LResNet100E-IR,ArcFace@ms1m-refine-v2*, *LResNet50E-IR,ArcFace@ms1m-refine-v1*, *LResNet34E-IR,ArcFace@ms1m-refine-v1*. By analogy, *MobileFaceNet,ArcFace@ms1m-refine-v1* can be transformed by slight modifications.\n\nIn response to [this](https://github.com/papermsucode/advhat/issues/11) issue.\n"
  }
]