[ { "path": "LICENCE", "content": "Yoox Net-a-Porter Dress Code Dataset – Licence Terms\n\nSUMMARY\nThis is a summary of, and not a substitution for, the licence terms set out below.\n- You are free to use and distribute the Dress Code Dataset for the purposes of non-commercial academic research, teaching and publication.\n- You can use the Dress Code Dataset only as provided: you are not permitted to create and distribute alterations to the Dress Code Dataset \nor any part of it.\n- When you share or publish any part of the Dress Code Dataset you must include an attribution.\n- The Dress Code Dataset is provided “as is” and without warranty.\n- Your rights to use the Dress Code Dataset may be revoked. \n\nLICENCE TERMS\nBy making any use of the Dress Code Dataset (as defined below), you accept and agree to comply with the terms and conditions of this Licence. \nYour right to use the Dress Code Dataset is subject to and conditional upon your compliance with those terms and conditions.\n\n*** Definitions *** \n- The Dress Code Dataset means the dataset of image pairs, for image-based virtual try-on, made available by us through Unimore.\n- Unimore means The University of Modena and Reggio Emilia, whose address is at Via Università, 4, 41121 Modena MO, Italy.\n- We \n- or YNAP means Yoox Net-a-Porter Group S.p.A., whose address is at Via Morimondo, 17, 20143 Milan MI, Italy. \n- You means the natural or legal person making use of the Dress Code Dataset\n\n*** What you can do ***\nYou can use the Dress Code Dataset only for the purposes of research, teaching and publication (which must in each case be academic and \nnon-commercial).\n\nFor the purposes of this provision:\n- using the Dress Code Dataset means you can use, copy, publish, distribute and transmit it (in whole or in part), but does not allow you \nto adapt it except as expressly set out below.\n- non-commercial means you cannot use the Dress Code Dataset for purposes which are mainly directed towards payment or some other commercial \nadvantage. \nYou are not prohibited from academic use simply because it has some incidental commercial nature. For example, use of the Dress Code Dataset \nin academic publication is permitted even if the relevant journal is subject to a subscription fee, and use of the Dress Code Dataset in \nacademic teaching is permitted even if that teaching is subject to tuition fees.\n- academic means in connection with education, teaching and research activities undertaken by an accredited not-for-profit academic institution, \nand excludes research undertaken in collaboration with any commercial entity except under terms which prohibit that entity from making \ncommercial use of any results arising from that research.\n\nWe grant to you a worldwide, royalty-free, non-sublicensable, non-exclusive license under our respective rights (including copyright and \ndatabase right) in the Dress Code Dataset for the permitted purposes set out above. \nYou may adapt the Dress Code Dataset only as necessary to make it interoperable with any other systems or technology which you are using \nfor non-commercial academic research, teaching or publication. Otherwise, you may not adapt it without our permission. Any adaptations \nwhich you do make (for interoperability, or with our permission) and any other works you make using the Dress Code Dataset will be subject \nto the same terms and restrictions of this Licence: in particular that means you cannot use them for commercial purposes.\n\nIf you create any adaptations or other works using the Dress Code Dataset, then you hereby grant to us a non-exclusive, royalty free, \nperpetual, irrevocable licence to use, copy, modify, distribute, and otherwise exploit those adaptations or other works for any purposes.\n\n*** Attribution and your downstream obligations ***\nIf you share the Dress Code Dataset (in whole or in part), you must:\n- retain any notices which identify the origins, authors, or rightsholders of the Dress Code Dataset (including without limitation any \ncopyright notice);\n- retain any notices which refer to this Licence in whole or in part, or which contain any disclaimers of warranties in relation to the \nDress Code Dataset;\n- actively notify the recipient that their use of the Dress Code Dataset will be subject to this Licence (and provide them with a link);\n- not seek to impose any additional obligations on the recipient in relation to their use of the Dress Code Dataset which would be \nincompatible with the terms of this Licence.\n\nIf you publish the Dress Code Dataset (in whole or in part), you must include in that publication an attribution in the following form:\n“The Dress Code Dataset is proprietary to and © Yoox Net-a-Porter Group S.p.A., and its licensors. It is distributed by the University of \nModena and Reggio Emilia, and available for non-commercial academic use under licence terms set out at https://github.com/aimagelab/dress-code.” \n\nWhile you must ensure that you acknowledge us as the source of the Dress Code Dataset as described above, you must not in doing so, \nor otherwise, suggest that you or your use of the Dress Code Dataset is endorsed or sponsored by us, or otherwise connected with us, unless \nwe have separately given you permission to do so.\n\n*** What you must not do ***\nYou must not use the Dress Code Dataset except as expressly permitted above. In particular:\n- You may not use the Dress Code Dataset, or any adaptations or other works created using the Dress Code Dataset, for commercial purposes.\n- You may not adapt the Dress Code Dataset except as expressly set out above. In particular, you may not create new works derived from or \nbased upon the Dress Code Dataset in which the Dress Code Dataset is wholly or partially translated, altered, or modified in a manner which \nwould otherwise require permissions from the relevant rightsholder under applicable laws relating to copyright and/or database rights.\n- The images in the Dress Code Dataset depict garments whose designs may be protected by copyright and/or design rights. You are not granted \nany licence under those rights except in connection with your permitted use of the Dress Code Dataset. In particular you are not authorised \nto make or manufacture any of those garments, or to alter their designs in your use of the Dress Code Dataset.\n- The images in the Dress Code Dataset have been cropped so that the models featured in those images are not individually identifiable. \nYou must not attempt to re-identify those models.\n\nIf you become aware of any use of the Dress Code Dataset other than as permitted by this Licence (whether by your own organisation \nor any third party) you must notify us promptly at legalteam_it@ynap.com and segreteria.aimagelab@unimore.it.\n\n*** No warranty ***\nThe Dress Code Dataset is provided to you “as is” and we exclude all representations, warranties, and liabilities in relation to the \nDress Code Dataset (including without limitation as to quality, fitness for purpose, non-infringement and accuracy) to the maximum extent \npermitted by law. \nWithout limiting that exclusion, we will not be liable for any errors or omissions in the Dress Code Dataset or for any loss or damages of any \nkind arising from its use. We do not guarantee that the Dress Code Dataset will remain available, and we may withdraw it at any time. \n\n*** Term and Termination ***\nThis Licence applies for the entire duration of any copyright, database right or other rights we may have in the Dress Code Dataset. \nIf you breach any of your obligations under this Licence, then without limiting our other rights and remedies, your rights under it will \nautomatically terminate. Your rights will be reinstated automatically if you remedy that breach within thirty days after discovery or may be \nreinstated by our written approval.We may withdraw this Licence at any time, and if we do you must stop using the Dress Code Dataset as \nsoon as possible unless we make it available to you on alternative terms and conditions.\n\n*** Other provisions ***\nWe will not be subject to any additional terms or conditions which you may seek to introduce in relation to your use of the Dress Code Dataset.\nThis Licence does not limit your lawful freedoms to use the Dress Code Dataset in reliance on exceptions and exclusions from laws relating to \ncopyright or database rights (such as fair dealing or fair use).\nIf you are in breach of this Licence, we can only waive that breach by written notice. If we do not immediately take action in relation to \nyour breach, we may still do so later.\nThis Licence is the entire agreement between you and us relating to your use of the Dress Code Dataset. You acknowledge that you have not \nentered into this Licence based on any other representation or warranty.\nThis Licence is governed by the laws of Italy. \n\n*** About this Licence ***\nThis licence is © Yoox Net-a-Porter Group S.p.A.. \n" }, { "path": "README.md", "content": "# Dress Code Dataset\n\nThis repository presents the virtual try-on dataset proposed in:\n\n*D. Morelli, M. Fincato, M. Cornia, F. Landi, F. Cesari, R. Cucchiara*
\n**Dress Code: High-Resolution Multi-Category Virtual Try-On**
\n\n**[[Paper](https://arxiv.org/abs/2204.08532)]** **[[Dataset Request Form](https://forms.gle/72Bpeh48P7zQimin7)]** **[[Try-On Demo](https://ailb-web.ing.unimore.it/dress-code)]**\n\n**IMPORTANT!**\n- By making any use of the Dress Code Dataset, you accept and agree to comply with the terms and conditions reported [here](https://github.com/aimagelab/dress-code/blob/main/LICENCE).\n- The dataset will not be released to private companies. \n- When filling the dataset request form, non-institutional emails (e.g. gmail.com, qq.com, etc.) are not allowed.\n- The signed release agreement form is mandatory (see the dataset request form for more details). Incomplete or unsigned release agreement forms are not accepted and will not receive a response. Typed signatures are not allowed.\n\n**Requests are manually validated on a weekly basis. If you do not receive a response, your request does not meet the outlined requirements.**\n\n
\n\nPlease cite with the following BibTeX:\n\n```\n@inproceedings{morelli2022dresscode,\n title={{Dress Code: High-Resolution Multi-Category Virtual Try-On}},\n author={Morelli, Davide and Fincato, Matteo and Cornia, Marcella and Landi, Federico and Cesari, Fabio and Cucchiara, Rita},\n booktitle={Proceedings of the European Conference on Computer Vision},\n year={2022}\n}\n```\n\n

\n \n

\n\n## Dataset\n\nWe collected a new dataset for image-based virtual try-on composed of image pairs coming from different catalogs of YOOX NET-A-PORTER.
\nThe dataset contains more than 50k high resolution model clothing images pairs divided into three different categories (*i.e.* dresses, upper-body clothes, lower-body clothes).\n\n

\n \n

\n\n### Summary\n- 53792 garments\n- 107584 images\n- 3 categories\n - upper body\n - lower body\n - dresses\n- 1024 x 768 image resolution\n- additional info\n - keypoints\n - skeletons\n - human label maps\n - human dense poses\n\n### Additional Info\nAlong with model and garment image pair, we provide also the keypoints, skeleton, human label map, and dense pose. \n\n

\n \n

\n\n
More info\n\n### Keypoints\nFor all image pairs of the dataset, we stored the joint coordinates of human poses.\nIn particular, we used [OpenPose](https://github.com/Hzzone/pytorch-openpose) [1] to extract 18 keypoints for each human body. \n\nFor each image, we provided a json file containing a dictionary with the `keypoints` key.\nThe value of this key is a list of 18 elements, representing the joints of the human body. Each element is a list of 4 values, where the first two indicate the coordinates on the x and y axis respectively.\n\n### Skeletons\nSkeletons are RGB images obtained connecting keypoints with lines.\n\n### Human Label Map\n\nWe employed a human parser to assign each pixel of the image to a specific category thus obtaining a segmentation mask for each target model. \nSpecifically, we used the [SCHP model](https://github.com/PeikeLi/Self-Correction-Human-Parsing) [2] trained on the ATR dataset, a large single person human parsing dataset focused on fashion images with 18 classes.\n\nObtained images are composed of 1 channel filled with the category label value. \nCategories are mapped as follows:\n\n```ruby\n 0 background\n 1 hat\n 2 hair\n 3 sunglasses\n 4 upper_clothes\n 5 skirt\n 6 pants\n 7 dress\n 8 belt\n 9 left_shoe\n10 right_shoe\n11 head\n12 left_leg\n13 right_leg\n14 left_arm\n15 right_arm\n16 bag\n17 scarf\n```\n\n\n### Human Dense Pose\n\nWe also extracted dense label and UV mapping from all the model images using [DensePose](https://github.com/facebookresearch/detectron2/tree/main/projects/DensePose) [3].\n\n
\n\n## Experimental Results\n\n### Low Resolution 256 x 192\n\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
NameSSIMFIDKID
CP-VTON [4]0.80335.162.245
CP-VTON+ [5]0.90225.191.586
CP-VTON* [4]0.87418.991.117
PFAFN [6]0.90214.380.743
VITON-GT [7]0.89913.800.711
WUTON [8]0.90213.280.771
ACGPN [9]0.86813.790.818
OURS0.90611.400.570
\n\n## Code\nDue to a firm collaboration, we cannot release the code. However, we supply an empty Pytorch project to load data.\n## References\n\n[1] Cao, et al. \"OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields.\" IEEE TPAMI, 2019.\n\n[2] Li, et al. \"Self-Correction for Human Parsing.\" arXiv, 2019.\n\n[3] Güler, et al. \"Densepose: Dense human pose estimation in the wild.\" CVPR, 2018.\n\n[4] Wang, et al. \"Toward Characteristic-Preserving Image-based Virtual Try-On Network.\" ECCV, 2018.\n\n[5] Minar, et al. \"CP-VTON+: Clothing Shape and Texture Preserving Image-Based Virtual Try-On.\" CVPR Workshops, 2020.\n\n[6] Ge, et al. \"Parser-Free Virtual Try-On via Distilling Appearance Flows.\" CVPR, 2021.\n\n[7] Fincato, et al. \"VITON-GT: An Image-based Virtual Try-On Model with Geometric Transformations.\" ICPR, 2020.\n\n[8] Issenhuth, el al. \"Do Not Mask What You Do Not Need to Mask: a Parser-Free Virtual Try-On.\" ECCV, 2020.\n\n[9] Yang, et al. \"Towards Photo-Realistic Virtual Try-On by Adaptively Generating-Preserving Image Content.\" CVPR, 2020.\n\n## Contact\n\nIf you have any general doubt about our dataset, please use the [public issues section](https://github.com/aimagelab/dress-code/issues) on this github repo. Alternatively, drop us an e-mail at davide.morelli [at] unimore.it or marcella.cornia [at] unimore.it.\n" }, { "path": "conf.py", "content": "import argparse\n\n\ndef get_conf(train=True):\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--exp_name\", type=str, default=\"\")\n parser.add_argument(\"--category\", default='all', type=str)\n parser.add_argument(\"--dataroot\", type=str, default=\"\")\n parser.add_argument(\"--data_pairs\", default=\"{}_pairs\")\n\n parser.add_argument('--checkpoint_dir', type=str, default='',\n help='save checkpoint infos')\n\n parser.add_argument('-b', '--batch_size', type=int, default=8)\n parser.add_argument('-j', '--workers', type=int, default=0)\n\n parser.add_argument(\"--epochs\", type=int, default=150)\n parser.add_argument(\"--step\", type=int, default=100000)\n parser.add_argument(\"--display_count\", type=int, default=1000)\n parser.add_argument(\"--shuffle\", default=True, action='store_true', help='shuffle input data')\n\n parser.add_argument(\"--height\", type=int, default=256)\n parser.add_argument(\"--width\", type=int, default=192)\n parser.add_argument(\"--radius\", type=int, default=5)\n\n args = parser.parse_args()\n print(args)\n return args\n" }, { "path": "data/__init__.py", "content": "from .dataset import Dataset\nfrom .dataloader import DataLoader" }, { "path": "data/dataloader.py", "content": "import torch\n\n\nclass DataLoader(object):\n def __init__(self, opt, dataset, dist_sampler=False):\n super(DataLoader, self).__init__()\n if dist_sampler:\n train_sampler = torch.utils.data.distributed.DistributedSampler(\n dataset, num_replicas=opt.world_size, rank=opt.rank, shuffle=True)\n else:\n if opt.shuffle:\n train_sampler = torch.utils.data.sampler.RandomSampler(dataset)\n else:\n train_sampler = None\n\n self.sampler = train_sampler\n self.data_loader = torch.utils.data.DataLoader(\n dataset, batch_size=opt.batch_size, shuffle=(train_sampler is None),\n num_workers=opt.workers, pin_memory=True, sampler=train_sampler)\n self.dataset = dataset\n self.data_iter = self.data_loader.__iter__()\n\n def next_batch(self):\n try:\n batch = self.data_iter.__next__()\n except StopIteration:\n self.data_iter = self.data_loader.__iter__()\n batch = self.data_iter.__next__()\n\n return batch\n" }, { "path": "data/dataset.py", "content": "import cv2\nimport torch\nimport torch.utils.data as data\nimport torchvision.transforms as transforms\nfrom PIL import Image, ImageDraw\nimport os\nimport numpy as np\nimport json\nfrom typing import List, Tuple\nfrom data.labelmap import label_map\nfrom numpy.linalg import lstsq\n\n\nclass Dataset(data.Dataset):\n def __init__(self, args, dataroot_path: str,\n phase: str,\n order: str = 'paired',\n category: List[str] = ['dresses', 'upper_body', 'lower_body'],\n size: Tuple[int, int] = (256, 192)):\n \"\"\"\n Initialize the PyTroch Dataset Class\n :param args: argparse parameters\n :type args: argparse\n :param dataroot_path: dataset root folder\n :type dataroot_path: string\n :param phase: phase (train | test)\n :type phase: string\n :param order: setting (paired | unpaired)\n :type order: string\n :param category: clothing category (upper_body | lower_body | dresses)\n :type category: list(str)\n :param size: image size (height, width)\n :type size: tuple(int)\n \"\"\"\n super(Dataset, self).__init__()\n self.args = args\n self.dataroot = dataroot_path\n self.phase = phase\n self.category = category\n self.height = size[0]\n self.width = size[1]\n self.radius = args.radius\n self.transform = transforms.Compose([\n transforms.ToTensor(),\n transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))\n ])\n self.transform2D = transforms.Compose([\n transforms.ToTensor(),\n transforms.Normalize((0.5, ), (0.5, ))\n ])\n\n im_names = []\n c_names = []\n dataroot_names = []\n\n for c in category:\n assert c in ['dresses', 'upper_body', 'lower_body']\n\n dataroot = os.path.join(self.dataroot, c)\n if phase == 'train':\n filename = os.path.join(dataroot, f\"{phase}_pairs.txt\")\n else:\n filename = os.path.join(dataroot, f\"{phase}_pairs_{order}.txt\")\n with open(filename, 'r') as f:\n for line in f.readlines():\n im_name, c_name = line.strip().split()\n im_names.append(im_name)\n c_names.append(c_name)\n dataroot_names.append(dataroot)\n\n self.im_names = im_names\n self.c_names = c_names\n self.dataroot_names = dataroot_names\n\n def __getitem__(self, index):\n \"\"\"\n For each index return the corresponding sample in the dataset\n :param index: data index\n :type index: int\n :return: dict containing dataset samples\n :rtype: dict\n \"\"\"\n c_name = self.c_names[index]\n im_name = self.im_names[index]\n dataroot = self.dataroot_names[index]\n\n # Clothing image\n cloth = Image.open(os.path.join(dataroot, 'images', c_name))\n cloth = cloth.resize((self.width, self.height))\n cloth = self.transform(cloth) # [-1,1]\n\n # Person image\n im = Image.open(os.path.join(dataroot, 'images', im_name))\n im = im.resize((self.width, self.height))\n im = self.transform(im) # [-1,1]\n\n # Skeleton\n skeleton = Image.open(os.path.join(dataroot, 'skeletons', im_name.replace(\"_0\", \"_5\")))\n skeleton = skeleton.resize((self.width, self.height))\n skeleton = self.transform(skeleton)\n\n # Label Map\n parse_name = im_name.replace('_0.jpg', '_4.png')\n im_parse = Image.open(os.path.join(dataroot, 'label_maps', parse_name))\n im_parse = im_parse.resize((self.width, self.height), Image.NEAREST)\n parse_array = np.array(im_parse)\n\n parse_shape = (parse_array > 0).astype(np.float32)\n\n parse_head = (parse_array == 1).astype(np.float32) + \\\n (parse_array == 2).astype(np.float32) + \\\n (parse_array == 3).astype(np.float32) + \\\n (parse_array == 11).astype(np.float32)\n\n parser_mask_fixed = (parse_array == label_map[\"hair\"]).astype(np.float32) + \\\n (parse_array == label_map[\"left_shoe\"]).astype(np.float32) + \\\n (parse_array == label_map[\"right_shoe\"]).astype(np.float32) + \\\n (parse_array == label_map[\"hat\"]).astype(np.float32) + \\\n (parse_array == label_map[\"sunglasses\"]).astype(np.float32) + \\\n (parse_array == label_map[\"scarf\"]).astype(np.float32) + \\\n (parse_array == label_map[\"bag\"]).astype(np.float32)\n\n parser_mask_changeable = (parse_array == label_map[\"background\"]).astype(np.float32)\n\n arms = (parse_array == 14).astype(np.float32) + (parse_array == 15).astype(np.float32)\n\n if dataroot.split('/')[-1] == 'dresses':\n label_cat = 7\n parse_cloth = (parse_array == 7).astype(np.float32)\n parse_mask = (parse_array == 7).astype(np.float32) + \\\n (parse_array == 12).astype(np.float32) + \\\n (parse_array == 13).astype(np.float32)\n parser_mask_changeable += np.logical_and(parse_array, np.logical_not(parser_mask_fixed))\n\n elif dataroot.split('/')[-1] == 'upper_body':\n label_cat = 4\n parse_cloth = (parse_array == 4).astype(np.float32)\n parse_mask = (parse_array == 4).astype(np.float32)\n\n parser_mask_fixed += (parse_array == label_map[\"skirt\"]).astype(np.float32) + \\\n (parse_array == label_map[\"pants\"]).astype(np.float32)\n\n parser_mask_changeable += np.logical_and(parse_array, np.logical_not(parser_mask_fixed))\n elif dataroot.split('/')[-1] == 'lower_body':\n label_cat = 6\n parse_cloth = (parse_array == 6).astype(np.float32)\n parse_mask = (parse_array == 6).astype(np.float32) + \\\n (parse_array == 12).astype(np.float32) + \\\n (parse_array == 13).astype(np.float32)\n\n parser_mask_fixed += (parse_array == label_map[\"upper_clothes\"]).astype(np.float32) + \\\n (parse_array == 14).astype(np.float32) + \\\n (parse_array == 15).astype(np.float32)\n parser_mask_changeable += np.logical_and(parse_array, np.logical_not(parser_mask_fixed))\n\n parse_head = torch.from_numpy(parse_head) # [0,1]\n parse_cloth = torch.from_numpy(parse_cloth) # [0,1]\n parse_mask = torch.from_numpy(parse_mask) # [0,1]\n parser_mask_fixed = torch.from_numpy(parser_mask_fixed)\n parser_mask_changeable = torch.from_numpy(parser_mask_changeable)\n\n # dilation\n parse_without_cloth = np.logical_and(parse_shape, np.logical_not(parse_mask))\n parse_mask = parse_mask.cpu().numpy()\n\n # Masked cloth\n im_head = im * parse_head - (1 - parse_head)\n im_cloth = im * parse_cloth + (1 - parse_cloth)\n\n # Shape\n parse_shape = Image.fromarray((parse_shape * 255).astype(np.uint8))\n parse_shape = parse_shape.resize((self.width // 16, self.height // 16), Image.BILINEAR)\n parse_shape = parse_shape.resize((self.width, self.height), Image.BILINEAR)\n shape = self.transform2D(parse_shape) # [-1,1]\n\n # Load pose points\n pose_name = im_name.replace('_0.jpg', '_2.json')\n with open(os.path.join(dataroot, 'keypoints', pose_name), 'r') as f:\n pose_label = json.load(f)\n pose_data = pose_label['keypoints']\n pose_data = np.array(pose_data)\n pose_data = pose_data.reshape((-1, 4))\n\n point_num = pose_data.shape[0]\n pose_map = torch.zeros(point_num, self.height, self.width)\n r = self.radius * (self.height/512.0)\n im_pose = Image.new('L', (self.width, self.height))\n pose_draw = ImageDraw.Draw(im_pose)\n neck = Image.new('L', (self.width, self.height))\n neck_draw = ImageDraw.Draw(neck)\n for i in range(point_num):\n one_map = Image.new('L', (self.width, self.height))\n draw = ImageDraw.Draw(one_map)\n point_x = np.multiply(pose_data[i, 0], self.width/384.0)\n point_y = np.multiply(pose_data[i, 1], self.height/512.0)\n if point_x > 1 and point_y > 1:\n draw.rectangle((point_x - r, point_y - r, point_x + r, point_y + r), 'white', 'white')\n pose_draw.rectangle((point_x - r, point_y - r, point_x + r, point_y + r), 'white', 'white')\n if i == 2 or i == 5:\n neck_draw.ellipse((point_x - r*4, point_y - r*4, point_x + r*4, point_y + r*4), 'white', 'white')\n one_map = self.transform2D(one_map)\n pose_map[i] = one_map[0]\n\n # just for visualization\n im_pose = self.transform2D(im_pose)\n\n im_arms = Image.new('L', (self.width, self.height))\n arms_draw = ImageDraw.Draw(im_arms)\n if dataroot.split('/')[-1] == 'dresses' or dataroot.split('/')[-1] == 'upper_body':\n with open(os.path.join(dataroot, 'keypoints', pose_name), 'r') as f:\n data = json.load(f)\n shoulder_right = np.multiply(tuple(data['keypoints'][2][:2]), self.height / 512.0)\n shoulder_left = np.multiply(tuple(data['keypoints'][5][:2]), self.height / 512.0)\n elbow_right = np.multiply(tuple(data['keypoints'][3][:2]), self.height / 512.0)\n elbow_left = np.multiply(tuple(data['keypoints'][6][:2]), self.height / 512.0)\n wrist_right = np.multiply(tuple(data['keypoints'][4][:2]), self.height / 512.0)\n wrist_left = np.multiply(tuple(data['keypoints'][7][:2]), self.height / 512.0)\n if wrist_right[0] <= 1. and wrist_right[1] <= 1.:\n if elbow_right[0] <= 1. and elbow_right[1] <= 1.:\n arms_draw.line(np.concatenate((wrist_left, elbow_left, shoulder_left, shoulder_right)).astype(np.uint16).tolist(), 'white', 30, 'curve')\n else:\n arms_draw.line(np.concatenate((wrist_left, elbow_left, shoulder_left, shoulder_right, elbow_right)).astype(np.uint16).tolist(), 'white', 30, 'curve')\n elif wrist_left[0] <= 1. and wrist_left[1] <= 1.:\n if elbow_left[0] <= 1. and elbow_left[1] <= 1.:\n arms_draw.line(np.concatenate((shoulder_left, shoulder_right, elbow_right, wrist_right)).astype(np.uint16).tolist(), 'white', 30, 'curve')\n else:\n arms_draw.line(np.concatenate((elbow_left, shoulder_left, shoulder_right, elbow_right, wrist_right)).astype(np.uint16).tolist(), 'white', 30, 'curve')\n else:\n arms_draw.line(np.concatenate((wrist_left, elbow_left, shoulder_left, shoulder_right, elbow_right, wrist_right)).astype(np.uint16).tolist(),'white', 30, 'curve')\n\n if self.args.height > 512:\n im_arms = cv2.dilate(np.float32(im_arms), np.ones((10, 10), np.uint16), iterations=5)\n # elif self.args.height > 256:\n # im_arms = cv2.dilate(np.float32(im_arms), np.ones((5, 5), np.uint16), iterations=5)\n hands = np.logical_and(np.logical_not(im_arms), arms)\n parse_mask += im_arms\n parser_mask_fixed += hands\n\n #delete neck\n parse_head_2 = torch.clone(parse_head)\n if dataroot.split('/')[-1] == 'dresses' or dataroot.split('/')[-1] == 'upper_body':\n with open(os.path.join(dataroot, 'keypoints', pose_name), 'r') as f:\n data = json.load(f)\n points = []\n points.append(np.multiply(tuple(data['keypoints'][2][:2]), self.height/512.0))\n points.append(np.multiply(tuple(data['keypoints'][5][:2]), self.height/512.0))\n x_coords, y_coords = zip(*points)\n A = np.vstack([x_coords, np.ones(len(x_coords))]).T\n m, c = lstsq(A, y_coords, rcond=None)[0]\n for i in range(parse_array.shape[1]):\n y = i * m + c\n parse_head_2[int(y - 20*(self.height/512.0)):, i] = 0\n\n parser_mask_fixed = np.logical_or(parser_mask_fixed, np.array(parse_head_2, dtype=np.uint16))\n parse_mask += np.logical_or(parse_mask, np.logical_and(np.array(parse_head, dtype=np.uint16), np.logical_not(np.array(parse_head_2, dtype=np.uint16))))\n\n if self.args.height > 512:\n parse_mask = cv2.dilate(parse_mask, np.ones((20, 20), np.uint16), iterations=5)\n # elif self.args.height > 256:\n # parse_mask = cv2.dilate(parse_mask, np.ones((10, 10), np.uint16), iterations=5)\n else:\n parse_mask = cv2.dilate(parse_mask, np.ones((5, 5), np.uint16), iterations=5)\n parse_mask = np.logical_and(parser_mask_changeable, np.logical_not(parse_mask))\n parse_mask_total = np.logical_or(parse_mask, parser_mask_fixed)\n im_mask = im * parse_mask_total\n parse_mask_total = parse_mask_total.numpy()\n parse_mask_total = parse_array * parse_mask_total\n parse_mask_total = torch.from_numpy(parse_mask_total)\n\n uv = np.load(os.path.join(dataroot, 'dense', im_name.replace('_0.jpg', '_5_uv.npz')))\n uv = uv['uv']\n uv = torch.from_numpy(uv)\n uv = transforms.functional.resize(uv, (self.height, self.width))\n\n labels = Image.open(os.path.join(dataroot, 'dense', im_name.replace('_0.jpg', '_5.png')))\n labels = labels.resize((self.width, self.height), Image.NEAREST)\n labels = np.array(labels)\n\n result = {\n 'c_name': c_name, # for visualization\n 'im_name': im_name, # for visualization or ground truth\n 'cloth': cloth, # for input\n 'image': im, # for visualization\n 'im_cloth': im_cloth, # for ground truth\n 'shape': shape, # for visualization\n 'im_head': im_head, # for visualization\n 'im_pose': im_pose, # for visualization\n 'pose_map': pose_map,\n 'parse_array': parse_array,\n 'dense_labels': labels,\n 'dense_uv': uv,\n 'skeleton': skeleton,\n 'm': im_mask, # for input\n 'parse_mask_total': parse_mask_total,\n }\n\n return result\n\n def __len__(self):\n return len(self.c_names)\n" }, { "path": "data/labelmap.py", "content": "label_map={\n \"background\": 0,\n \"hat\": 1,\n \"hair\": 2,\n \"sunglasses\": 3,\n \"upper_clothes\": 4,\n \"skirt\": 5,\n \"pants\": 6,\n \"dress\": 7,\n \"belt\": 8,\n \"left_shoe\": 9,\n \"right_shoe\": 10,\n \"head\": 11,\n \"left_leg\": 12,\n \"right_leg\": 13,\n \"left_arm\": 14,\n \"right_arm\": 15,\n \"bag\": 16,\n \"scarf\": 17,\n}" }, { "path": "main.py", "content": "import torch\n\nfrom data import Dataset, DataLoader\nfrom tqdm import tqdm\nfrom utils import sem2onehot\nimport conf\n\n\ndef test_unpaired(dataloader, model, e, args):\n with tqdm(desc=\"Iteration %d - images extraction\" % e, unit='it', total=len(dataloader.data_loader)) as pbar:\n for step in range(0, len(dataloader.data_loader)):\n inputs = dataloader.next_batch()\n\n with torch.no_grad():\n image_name = inputs['im_name']\n cloth_name = inputs['c_name']\n image = inputs['image'].cuda()\n cloth = inputs['cloth'].cuda()\n cropped_cloth = inputs['im_cloth'].cuda()\n im_head = inputs['im_head'].cuda()\n pose_map = inputs['pose_map'].cuda()\n skeleton = inputs['skeleton'].cuda()\n im_pose = inputs['im_pose'].cuda()\n shape = inputs['shape'].cuda()\n parse_array = inputs['parse_array'].cuda()\n dense_labels = inputs['dense_labels'].cuda()\n dense_uv = inputs['dense_uv'].cuda()\n\n parse_array = sem2onehot(18, parse_array)\n\n # model here\n\n\ndef training_loop(dataloader, model, e, args):\n\n with tqdm(desc=\"Iteration %d - train\" % e, unit='it', total=args.display_count) as pbar:\n for step in range(0, args.display_count):\n inputs = dataloader.next_batch()\n\n image_name = inputs['im_name']\n cloth_name = inputs['c_name']\n image = inputs['image'].cuda()\n cloth = inputs['cloth'].cuda()\n cropped_cloth = inputs['im_cloth'].cuda()\n im_head = inputs['im_head'].cuda()\n pose_map = inputs['pose_map'].cuda()\n skeleton = inputs['skeleton'].cuda()\n im_pose = inputs['im_pose'].cuda()\n shape = inputs['shape'].cuda()\n parse_array = inputs['parse_array'].cuda()\n dense_labels = inputs['dense_labels'].cuda()\n dense_uv = inputs['dense_uv'].cuda()\n\n parse_array = sem2onehot(18, parse_array)\n\n # model here\n\n pbar.update()\n\n\ndef main_worker(args):\n\n # Dataset & Dataloader\n dataset_train = Dataset(args, \n dataroot_path=args.dataroot, \n phase='train',\n order='paired',\n size=(int(args.height), int(args.width)))\n\n dataloader_train = DataLoader(args, dataset_train, dist_sampler=False)\n\n dataset_test_unpaired = Dataset(args,\n dataroot_path=args.dataroot,\n phase='test',\n order='unpaired',\n size=(int(args.height), int(args.width)))\n\n dataloader_test_unpaired = DataLoader(args, dataset_test_unpaired, dist_sampler=False)\n\n # Instance here your model\n model = None\n\n # Loop in epochs\n for e in range(0, args.epochs):\n # Training loop\n training_loop(dataloader_train, model, e, args)\n\n # Test unpaired\n inputs = dataloader_test_unpaired.next_batch()\n test_unpaired(inputs, model, e, args)\n\n\nif __name__ == '__main__':\n # Get argparser configuration\n args = conf.get_conf()\n print(args.exp_name)\n\n # Call main worker\n main_worker(args)\n" }, { "path": "utils/__init__.py", "content": "from .label_map import sem2onehot\n" }, { "path": "utils/label_map.py", "content": "import torch\n\n\ndef sem2onehot(n, labelmap):\n label_map = labelmap.long().unsqueeze(1).cuda()\n bs, _, h, w = label_map.size()\n nc = n\n input_label = torch.FloatTensor(bs, nc, h, w).zero_().cuda()\n input_semantics = input_label.scatter_(1, label_map, 1.0)\n return input_semantics\n" } ]