[ { "path": ".devcontainer/devcontainer.json", "content": "{\n \"name\": \"Python 3\",\n // Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile\n \"image\": \"mcr.microsoft.com/devcontainers/python:1-3.11-bullseye\",\n \"customizations\": {\n \"codespaces\": {\n \"openFiles\": [\n \"README.md\",\n \"gesture_streamlit.py\"\n ]\n },\n \"vscode\": {\n \"settings\": {},\n \"extensions\": [\n \"ms-python.python\",\n \"ms-python.vscode-pylance\"\n ]\n }\n },\n \"updateContentCommand\": \"[ -f packages.txt ] && sudo apt update && sudo apt upgrade -y && sudo xargs apt install -y 1600\n\n- [x] 利用Mosaic数据增强，但是结果不好，之后训练不会采用，除非数据足够多\n\n- [x] 增加yaml文件，利用yaml配置所有参数\n\n- [x] 提高图片的输入shape，从256x256 -> 416x416\n\n- [x] 由于结果不理想，使用部分自制数据集替换，数据集总数不变\n\n- [x] 添加yolov4 tiny 轻量化模型\n\n- [x] 增加注意力机制，可以比轻量化模型得到更不错的结果\n\n \n\n\n\n\n\n## 2. 部分尝试结果\n\n- [x] 使用Mosaic 结果较差\n- [x] 在运行过程中结果十分差，原因是数据集标注出现错误，会重新修改数据集\n- [x] 用SGD的结果没有Adam好\n- [x] front的数据集需要重新修改才能得到更好的结果\n- [x] 使用tiny模型速度更快，结果虽然差一点，但是只是一个速度与精度的trade off\n\n\n\n## 3. 项目整体框架\n\n1、 了解项目研究的背景以及其意义，学习其中的创新点和科研价值。\n\n2、 使用python语言对项目中的代码进行编写。研究项目源代码，理解项目工程的代码结构、原理及其功能。\n\n3、 学习深度学习算法。理解卷积神经网络的相关概念，包括神经元系统、局部感受野、权值共享和卷积神经网络总体结构；了解目前常见的目标检测方法和YOLOv4算法框架，以及基于YOLOv4的手势识别算法。\n\n4、 设计并制作针对本项目手势控制数据集，并使用数据增广的方式对数据集进行扩充，同时使用图像处理的方法包括中值滤波、阈值分割等对数据进行预处理。\n\n5、 训练模型，对目标检测性能进行测试。了解实验环境以及评价标准，测试本项目研究的手势识别算法的实验结果，然后通过采用控制变量方法对手势识别算法进行多组实验，以评估其在不同环境下的识别效果，使用验证集对手势识别算法的精度和速度进行性能测试。\n\n6、 总结本项目的研究工作，对基于无人机的手势识别演剧提供创新点与发展建议。\n\n### 3.1. 数据集构建\n\n1. 设计并制作针对本项目手势控制数据集，对数据集进行分类。\n\n![在这里插入图片描述](https://img-blog.csdnimg.cn/5b3c7cc2c58c404987d54d9a2f5bb68d.png)\n\n\n\n2. 使用Labelimg标注工具设计针对本项目的手势数据集，对数据集进行标注。\n\n![在这里插入图片描述](https://img-blog.csdnimg.cn/3312206223674362b64026836184e3e4.png)\n\n### 3.2. 模型选择\n\n在前期的模型选择中，简单的选择了YOLOv4的模型进行训练和测试\n\n **YOLOv4 = CSPDarknet53（主干） + SPP** **附加模块（颈** **） +** **PANet** **路径聚合（颈** **） + YOLOv3（头部）**\n\n![img](https://pdf.cdn.readpaper.com/parsed/fetch_target/699143cdb334ecfc63caf8192472490c_0_Figure_1.png)\n\n\n\n### 3.3. 代码实现\n\n- [x] 主干特征提取网络：DarkNet53 => CSPDarkNet53\n- [x] 特征金字塔：SPP，PAN\n- [x] 训练用到的小技巧：Mosaic数据增强、Label Smoothing平滑、CIOU、学习率余弦退火衰减\n- [x] 激活函数：使用Mish激活函数\n- [x] 增加yaml配置文件，只需要修改配置文件即可\n- [x] 添加detect.py，利用此进行半自动标注，可以方便标注其他类似于对应👋的数据集\n- [x] 修改成命令行运行的快速模式，很方便，快速运行和理解\n- [x] 利用streamlit部署到服务器上，可以随时使用，在线demo [https://kedreamix-yologesture.streamlit.app/](https://kedreamix-yologesture.streamlit.app/)\n- [ ] ......\n\n\n\n## 4. 实验结果详情\n\n| 训练数据集 | 权值文件名称 | 迭代次数 | Batch-size | 图片shape | 平均准确率 | mAP 0.5 | fps |\n| :--------: | :----------------------------------------------------------: | :------: | :--------: | :-------: | :--------: | :-----: | ----- |\n| Gesture v1 | yolo4_gesture_weights.pth | 150 | 4->8 | 256x256 | 61.65 | 51.66 | |\n| Gesture v2 | yolo4tiny_gesture_SE.pth | 100 | 64->32 | 416x416 | 83.6 | 95.18 | 76.08 |\n| Gesture v2 | yolo4tiny_gesture_CBAM.pth | 100 | 64->32 | 416x416 | 89.35 | 98.85 | 70.01 |\n| Gesture v2 | yolo4tiny_gesture_ECA.pth | 100 | 64->32 | 416x416 | 88.37 | 96.26 | 77.19 |\n| Gesture v2 | yolo4tiny_gesture.pth | 100 | 64->32 | 416x416 | 87.01 | 95.86 | 81.81 |\n| Gesture v2 | yolo4_gesture_weightsv2.pth | 100 | 4->8 | 256x256 | 84.51 | 90.77 | 24.21 |\n| Gesture v3 | [yolov4_tiny.pth](https://github.com/Kedreamix/YoloGesture/releases/download/v1.0/yolov4_tiny.pth) | 150 | 64->32 | 416x416 | 75.05 | 91.30 | |\n| Gesture v3 | [yolov4_SE.pth](https://github.com/Kedreamix/YoloGesture/releases/download/v1.0/yolov4_SE.pth) | 150 | 64->32 | 416x416 | 78.06 | 90.13 | |\n| Gesture v3 | [yolov4_CBAM.pth](https://github.com/Kedreamix/YoloGesture/releases/download/v1.0/yolov4_CBAM.pth) | 150 | 64->32 | 416x416 | 91.09 | 94.97 | |\n| Gesture v3 | [yolov4_ECA.pth](https://github.com/Kedreamix/YoloGesture/releases/download/v1.0/yolov4_ECA.pth) | 150 | 64->32 | 416x416 | 94.58 | 83.24 | |\n| Gesture v3 | [yolov4_weights_ep150_416.pth](https://github.com/Kedreamix/YoloGesture/releases/download/v1.0/yolov4_weights_ep150_416.pth) | 150 | 64->32 | 416x416 | 95.145 | 98.35 | |\n| Gesture v3 | [yolov4_weights_ep150_608.pth](https://github.com/Kedreamix/YoloGesture/releases/download/v1.0/yolov4_weights_ep150_608.pth) | 150 | 64->32 | 608x608 | 93.64 | 97.23 | |\n\n> Gesture v1中存在数据集问题，所以模型结构不好\n>\n> Gesture v2中重新修改数据集\n>\n> Gesture v3中修改front数据集\n\nBatch-Size 64->32是指在进行训练的时候，前半段冻结的时候使用的bs为64，在后续不冻结训练使用bs=32\n\n\n\n### 4.1. 训练权重文件下载\n\n训练所需的yolo4_weights.pth有两种方式下载。（release包含所有过程的权重，百度网盘和奶牛只记录最新的权重）\n\n- 可以从我的release下载权重 [https://github.com/Kedreamix/YoloGesture/releases/tag/v1.0](https://github.com/Kedreamix/YoloGesture/releases/tag/v1.0)\n\n- 可以从我的huggingface的model下载权重 [https://huggingface.co/Kedreamix/YoloGestureWeights](https://huggingface.co/Kedreamix/YoloGestureWeights)\n\n- 也可以百度网盘下载\n\n 链接：https://pan.baidu.com/s/1Pt11VHMaHqSsPjb50W5IeQ\n 提取码：6666\n \n- 由于百度网盘下载速度较慢，这里也给一个不限速的链接 （永久有效）\n\n 传输链接：https://cowtransfer.com/s/dc5e0f7f43a940 或 打开【奶牛快传】cowtransfer.com 使用传输口令：ftyvu0 提取；\n \n \n\n### 4.2. 数据集概况\n\n![在这里插入图片描述](https://img-blog.csdnimg.cn/5b3c7cc2c58c404987d54d9a2f5bb68d.png)\n\n- **Gesture v1** 只有800张图片，数量较少\n- **Gesture** **v2** 增加了800张图片，数量增多，一共1600张图片\n\n在运行过程中结果十分差，原因是数据集标注出现错误，会重新修改数据集\n\n- **Gesture v3** 中修改了front的手势，使得front结果大大提升，平均准确率增大\n\n> 上述展示图是关于Gesture v1的手势，后续数据进行了修改\n\n\n\n整体数据集一共含有1600张，8个类别的手势，我的Gesture v3最后就是8个类别，大概1600张的数据集，类别分别是\n\n- up\n- down\n- left\n- right\n- front\n- back\n- clockwise\n- anticlockwise\n\n数据已经放在release中了，可以下载自用\n\n> 之后我也做了类似的手势识别的任务，里面的数据集有18个类别 HaGRID手势识别数据集，里面的手势结果更多，并且也更大，总共有716G，建议可以缩小以后进行训练增强，如果有机会，我可以拿一个多类别的我也来训练一下\n>\n> 以下是HaGRID的手势识别的类别，支持更多的手势识别的结果，这是官方下载地址：[https://github.com/hukenovs/hagrid](https://github.com/hukenovs/hagrid)\n>\n> [![gestures](https://github.com/hukenovs/hagrid/raw/master/images/gestures.jpg)](https://github.com/hukenovs/hagrid/blob/master/images/gestures.jpg)\n\n\n\n## 5. 环境配置\n\n我用的是torch==1.8.1 torchvision==0.9.1\n\n> 代码在更高的版本也是适配的，我觉得可能去>=1.7的应该都是可以的\n\n相对应的库可以直接利用以下代码在当前路径进行运行，利用清华源进行换源\n\n```bash\npip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple/\n```\n\n\n\n## 6. 快速运行代码\n\n以下可以在命令行中运行，在命令行运行可能会更好一点\n\n
\nInstall\n\n\n```bash\ngit clone https://github.com/Kedreamix/YoloGesture.git \ncd YoloGesture\npip install -r requirements.txt # -i https://pypi.tuna.tsinghua.edu.cn/simple/ # install 可以加上清华源\n```\n\n
\n\n
\nData \n这一部分会生成两个文件，分别是2007_train.txt和2007_val.txt，在每一行包括了图片路径和对应的标签，之后代码会读取文件夹VOCdevkit/VOC2007下的图片和标签\n\n\n```python\npython voc_annotation.py\n```\n\n
\n\n
\nOptional \n可选，yolov4有对anchors进行Kmeans计算，但是用yolov4自带的也可以，这一部分是可选择的，做完有一个可视化的结果\n\n\n```python\npython kmeans_for_anchors.py\n```\n\n![kmeans_for_anchors.jpg](https://github.com/Kedreamix/YoloGesture/blob/main/kmeans_for_anchors.jpg?raw=true)\n\n
\n\n
\nTraining \n\n我们可以在里面设置所需要的参数，phi代表着不同的注意力机制，weights代表着权重，其他的是我们的一些参数的设置，都是可调的，参数的部分解释都可以从python train.py --help看到\n\n```bash\nusage: train.py [-h] [--init INIT] [--epochs EPOCHS] [--weights WEIGHTS]\n [--freeze] [--freeze-epochs FREEZE_EPOCHS]\n [--freeze-size FREEZE_SIZE] [--batch-size BATCH_SIZE]\n [--optimizer {sgd,adam,adamw}] [--num_workers NUM_WORKERS]\n [--lr LR] [--tiny] [--phi PHI] [--weight-decay WEIGHT_DECAY]\n [--momentum MOMENTUM] [--save-period SAVE_PERIOD] [--cuda]\n [--shape SHAPE] [--fp16] [--mosaic]\n [--lr_decay_type {cos,step}] [--distributed]\n\noptional arguments:\n -h, --help show this help message and exit\n --init INIT 从init epoch开始训练\n --epochs EPOCHS epochs for training\n --weights WEIGHTS initial weights path 初始权重的路径\n --freeze 表示是否冻结训练\n --freeze-epochs FREEZE_EPOCHS\n epochs for feeze 冻结训练的迭代次数\n --freeze-size FREEZE_SIZE\n total batch size for Freezeing\n --batch-size BATCH_SIZE\n total batch size for all GPUs\n --optimizer {sgd,adam,adamw}\n 训练使用的optimizer\n --num_workers NUM_WORKERS\n 用于设置是否使用多线程读取数据\n --lr LR Learning Rate 学习率的初始值\n --tiny 使用yolov4-tiny模型\n --phi PHI yolov4-tiny所使用的注意力机制的类型\n --weight-decay WEIGHT_DECAY\n 权值衰减，可防止过拟合\n --momentum MOMENTUM 优化器中的参数\n --save-period SAVE_PERIOD\n 多少个epochs保存一次权重\n --cuda 表示是否使用GPU\n --shape SHAPE 输入图像的shape，一定要是32的倍数\n --fp16 是否使用混合精度训练\n --mosaic Yolov4的tricks应用 马赛克数据增强\n --lr_decay_type {cos,step}\n cos\n --distributed 是否使用多卡运行\n```\n\n这里对一些常用参数进行解释\n\n- fp16\n\n 由于训练神经网络，有时候得到的权重的精度都是64位或者32位的，保存和训练的时候都占了很多显存，但是有时候这些是不必要的，所以可以利用fp16将精度设为16位，这样大概可以减少一半的显存\n\n- phi\n\n 这里解释一下，phi = 0代表的是yolov4_tiny，也就是改进的轻量化yolov4，而phi = 1,2,3分别是加了SE，CBAM，ECA三种注意力机制得到的结果。具体对SE，CBAM，ECA注意力机制不懂的，可以看看这篇博文，我觉得写的蛮好的：[https://blog.csdn.net/weixin_44791964/article/details/121371986](https://blog.csdn.net/weixin_44791964/article/details/121371986)，这里不过多介绍。\n\n- freeze\n\n 除此之外，可以从下面的代码看出，我们可以冻结进行迁移学习，也可以选择不冻结，通过参数freeze来控制，还可以控制冻结次数的冻结时的batch-size，冻结的时候，可以把batch-size调高一点，并且还可以调一下freeze-epochs参数和freeze-size参数\n\n如果对于不同模型训练的不动的，可以看看下面的训练预测细节解释\n\n```python\n# 冻结进行迁移学习，利用已有的yolov4_SE.pth的权重进行\npython train.py --tiny --phi 1 --epochs 100 \\\n --weights model_data/yolov4_SE.pth \\\n --freeze --freeze-epochs 50 --freeze-size 8 \\\n --batch-size 4 --shape 416 \\\n --fp16 --cuda\n\n# 快速运行尝试，重新学习\npython train.py --tiny --phi 1 --epochs 10 \\\n --batch-size 4 --shape 416 \\\n --fp16 --cuda\n```\n\n在后续为了简化操作，不用打那么多的字母，还进行了缩写的修改，把--freeze简化成-f，--weights 简化成 -w, --freeze-epochsj简化成-fe，--freeze-size 简化成fb， --batch-size简化成-bs，这是为了方便运行的时候设置参数\n\n这段代码和上面是等价的\n\n```python\n# 冻结进行迁移学习\npython train.py --tiny --phi 1 --epochs 100 \\\n -w model_data/yolov4_SE.pth \\\n -f -fe 50 -fs 8 \\\n --bs 4 --shape 416 \\\n --fp16 --cuda\n\n# 快速运行尝试，重新学习\npython train.py --tiny --phi 1 --epochs 10 \\\n --batch-size 4 --shape 416 \\\n --fp16 --cuda\n```\n\n
\n\n
\nPredict \n\npredict也有一些参数，比如以什么模式运行，分别有['dir_predict', 'video', 'fps','predict','heatmap']，默认是用predict来推理img文件夹下的所有图片\n\n```python\n# python predict.py --mode dir_predict \\\n# --tiny --phi 1 \\\n# --weights model_data/yolov4_SE.pth \\\n# --cuda --shape 416\npython predict.py --tiny --cuda\n```\n\n
\n\n
\nGet Map \n\n这一部分可以得到召回率和精确率等可视化的图片，可以清晰的看到结果\n\n```python\n# 对验证集进行计算\n# python get_map.py --mode 0 \\\n# --tiny --phi 1 \\\n# --weights model_data/yolov4_SE.pth \\\n# --cuda --shape 416\n# python .\\get_map.py --cuda --mode 0 --tiny --phi 3 --weights model_data/yolotv4_ECA.pth\npython get_map.py --tiny --cuda\n```\n\n
\n\n所有的参数都可以通过，通过help看到解释\n\n```python\npython train.py -h\npython get_map.py -h\npython predict.py -h\n```\n\n除此之外，如果有多个GPU，需要设定指定的GPU，在python前加上配置CUDA_VISIBLE_DEVICES=3，表示使用第四块GPU\n\n```python\n# 比如使用第4块GPU\nCUDA_VISIBLE_DEVICES=3 python train.py ...\n```\n\n或者是多块GPU，比如有0，1两块GPU\n\n```python\n# 比如使用第0,1块GPU\nCUDA_VISIBLE_DEVICES=0,1 python train.py ...\n```\n\n\n\n\n\n## 7. 训练预测细节解释\n\n### 7.1. 训练配置文件（重点）\n\n这个重中之重，在model_data文件夹下，有一个yaml文件，里面包括部分需要运行的参数\n\n只需要调节里面的参数，然后运行就可以得到我们的结果，完全是ok的，只需要改配置文件，其他参数可以在命令行修改，直接运行也是可以使用的，下面会详细介绍，主要是修改train.py的部分，因为这样可以方便我们训练\n\n```yaml\n#------------------------------detect.py--------------------------------#\n# 这一部分是为了半自动标注数据，可以减轻负担，需要提前训练一个权重，以Labelme格式保存\n# dir_origin_path 图片存放位置\n# dir_save_path Annotation保存位置\n# ----------------------------------------------------------------------#\ndir_detect_path: ./JPEGImages \ndetect_save_path: ./Annotation\n\n# ----------------------------- train.py -------------------------------#\nnc: 8 # 类别的数量\nclasses: [\"up\",\"down\",\"left\",\"right\",\"front\",\"back\",\"clockwise\",\"anticlockwise\"] # 类别\nconfidence: 0.5 # 置信度\nnms_iou: 0.3\nletterbox_image: False\n\nlr_decay_type: cos # 使用到的学习率下降方式，可选的有step、cos\n# 用于设置是否使用多线程读取数据\n# 开启后会加快数据读取速度，但是会占用更多内存\n# 内存较小的电脑可以设置为2或者0，win建议设为0\nnum_workers: 4\n```\n\n\n\n### 7.2. 训练自己数据集\n\n1. 数据集的准备 \n 训练前将标签文件放在VOCdevkit文件夹下的VOC2007文件夹下的Annotation中。 \n 训练前将图片文件放在VOCdevkit文件夹下的VOC2007文件夹下的JPEGImages中。 \n\n \n\n2. 数据集的处理 \n 在完成数据集的摆放之后，我们需要利用voc_annotation.py获得训练用的2007_train.txt和2007_val.txt。 \n 修改voc_annotation.py里面的参数。第一次训练可以仅修改classes_path，classes_path用于指向检测类别所对应的txt。 \n\n 然后再前面所说的data.yaml中写清楚自己的类别以及类别的数量\n\n ```bash\n nc: 8 # 类别的数量\n classes: [\"up\",\"down\",\"left\",\"right\",\"front\",\"back\",\"clockwise\",\"anticlockwise\"] # 类别\n ```\n\n \n\n3. 开始网络训练 \n\n 之后根据快速运行train.py，运行train.py开始训练了，在训练多个epoch后，权值会生成在logs文件夹中，可以自己设迭代次数保存权重，如上述快速运行即可。\n\n 这里面我内置了5个模型，分别是最原始的yolov4模型，以及yolov4-tiny，yolov4-SE，yolov4-ECA，yolov4-CBAM四种模型，这四种模型都可以进行训练，其中yolov4-tiny，yolov4-SE，yolov4-ECA，yolov4-CBAM都属于小模型，所以认为是tiny模型，得到的权重也比较小速度也会比较快，这几种方式有不同的参数，我现在简单的介绍，我用tiny和phi的参数对他们进行分开\n\n - phi = 0 yolov4-tiny\n - phi = 1 yolov4-SE\n - phi = 2 yolov4-CBAM\n - phi = 3 yolov4-ECA\n\n ```python\n # yolov4 模型\n python train.py --epochs 10 --shape 416 --cuda --batch-size 4\n # yolov4-tiny\n python train.py --epochs 10 --shape 416 --cuda --batch-size 8 --tiny --phi 0\n # yolov4-SE\n python train.py --epochs 10 --shape 416 --cuda --batch-size 8 --tiny --phi 1\n # yolov4-CBAM\n python train.py --epochs 10 --shape 416 --cuda --batch-size 8 --tiny --phi 2\n # yolov4-ECA\n python train.py --epochs 10 --shape 416 --cuda --batch-size 8 --tiny --phi 3\n ```\n\n 如果还要做其他参数对，也可以看到快速运行代码的训练部分，进行增加一些参数\n\n4. 训练结果预测 \n 训练结果预测需要用到两个文件，分别是yolo.py和predict.py。 \n 完成修改后就可以运行predict.py进行检测了。运行后输入图片路径即可检测。 （可以自己设置模式得到结果）\n\n### 7.3. 使用Tensorboard可视化结果\n\n在我们训练的过程中，我们可以用TensorBoard实时查看训练情况，也可以看到训练的网络模型结构，非常方便\n\n只需要在我们的文件夹的命令行下，运行\n\n```bash\ntensorboard --logdir='logs/'\n```\n\n之后大概我们的6006端口就可以实时看到我们的结果，即是https://localhost:6006\n\n> 如果是使用Ubuntu，有可能会出现一些bug，所以需要进行一些操作，因为会显示无法找到命令\n>\n> 这时候首先需要找到TensorBoard在库的哪里\n>\n> ```bash\n> pip show tensorboard\n> ```\n>\n> 这样子就能看到自己tensorboard下载的路径\n>\n> 然后找到TensorBoard的文件夹下，找到main.py文件，就可以进行了，利用绝对路径就可以了\n>\n> ```\n> python .../python3.8/site-packages/tensorboard/main.py --logdir='logs/' \n> ```\n\n\n\n### 7.4. 预测步骤\n\n1. 下载完库后解压，在百度网盘后者其他地方下载yolo_gesture_weights.pth，放入model_data，运行predict.py，调整权重路径\n\n 在predict.py中事先设置了`dir_predict`表示遍历文件夹进行检测并保存。默认遍历img文件夹，保存img_out文件夹，这样就可以在img_out中得到文件\n\n 有很多种模式，可以通过mode来调节，这一部分还可以设置参数，我们都可以从help里看到\n\n ```bash\n predict.py -h\n usage: predict.py [-h] [--weights WEIGHTS] [--tiny] [--phi PHI]\n [--mode {dir_predict,video,fps,predict,heatmap,export_onnx}]\n [--cuda] [--shape SHAPE] [--video VIDEO]\n [--save-video SAVE_VIDEO] [--confidence CONFIDENCE]\n [--nms_iou NMS_IOU]\n \n optional arguments:\n -h, --help show this help message and exit\n --weights WEIGHTS initial weights path\n --tiny 使用yolotiny模型\n --phi PHI yolov4tiny注意力机制类型\n --mode {dir_predict,video,fps,predict,heatmap,export_onnx}\n 预测的模式\n --cuda 表示是否使用GPU\n --shape SHAPE 输入图像的shape\n --video VIDEO 需要检测的视频文件\n --save-video SAVE_VIDEO\n 保存视频的位置\n --confidence CONFIDENCE\n 只有得分大于置信度的预测框会被保留下来\n --nms_iou NMS_IOU 非极大抑制所用到的nms_iou大小\n ```\n\n 如果下载了权重于路径model_data/yolov4_tiny.pth，默认是文件夹中的图片模式运行，我们就可以直接运行得到结果\n\n ```python\n python predict.py --tiny --phi 0 --weights model_data/yolov4_tiny.pth\n ```\n\n2. 在predict.py里面进行设置可以进行fps测试和video视频检测。 （这一部分可以自己尝试）\n\n 这一部分只要设置一下路径和视频即可，分别有多种模式\n\n\n\n### 7.5. 评估步骤\n\n1. 本文使用VOC格式进行评估。 \n\n2. 如果在训练前已经运行过voc_annotation.py文件，代码会自动将数据集划分成训练集、验证集和测试集。如果想要修改测试集的比例，可以修改voc_annotation.py文件下的trainval_percent。trainval_percent用于指定(训练集+验证集)与测试集的比例，默认情况下 (训练集+验证集):测试集 = 9:1。train_percent用于指定(训练集+验证集)中训练集与验证集的比例，默认情况下 训练集:验证集 = 9:1。\n\n3. 利用voc_annotation.py划分测试集\n\n4. 运行get_map.py即可获得评估结果，评估结果会保存在map_out文件夹中。\n\n\n\n\n\n\n## 8. Streamlit 项目部署\n\n经过上述学习过程，最后我利用streamlit进行了项目部署，可以在本地部署，也可以在云端部署，代码已经上传的了，并且我最后部署到了streamlit的服务器中，大家都可以在线体验 [https://kedreamix-yologesture.streamlit.app/](https://kedreamix-yologesture.streamlit.app/)，然后选择“Run the app”即可，不需要过多的操作，云端服务器会会自动从我的release中下载模型，所以这个不用担心。\n\n> 关于streamlit的一些方法，可以看一下我另一篇博客，也有对应的github链接，那个简单一点[https://redamancy.blog.csdn.net/article/details/121788919](https://redamancy.blog.csdn.net/article/details/121788919)\n\n![在这里插入图片描述](https://img-blog.csdnimg.cn/c62927213dc04faf855ed667d65602bd.png)\n\n\n\n### 8.1. 本地运行\n\n打开命令行运行以下代码，记住，首先进行pip install streamlit\n\n```python\nstreamlit run gesture.streamlit.py\n```\n\n运行之后，打开的 https://localhost:8501 就可以看到自己的streamlit的界面了\n\n![](https://img-blog.csdnimg.cn/45d6624063f649ec8083e75e94afaf28.png)\n\n### 8.2. 检测方法\n\n在这个部署界面中，我一共设了几种方式，分别是\n\n| 测试模型方式 | 测试方式描述 |\n| ------------ | ------------------------------------------------------------ |\n| Example | 已有一部分数据在服务器的文件夹里，可以读取进行检测 |\n| Image | 可以自主上传对应的图片进行检测![在这里插入图片描述](https://img-blog.csdnimg.cn/7db5de05f119415cae6a9cb0ef39d092.png) |\n| Camera | 利用电脑摄像头进行检测，对摄像头进行拍照，然后可以检测fps，heatmap![在这里插入图片描述](https://img-blog.csdnimg.cn/6c1b8f721ebe4e0f86d68bf321e210b2.png) |\n| FPS | 上传一张图片进行FPS |\n| Heatmap | 进行一个热力图的检测，可以看到模型关注哪一部分![在这里插入图片描述](https://img-blog.csdnimg.cn/e2c53a474f764a0f99adb05b39c3bee9.png) |\n| Real Time | 实时检测，可能这一部分在云服务器有点bug，可能要在自己电脑下才能正常运行，云端不可用 |\n| Video | 传入视频进行检测![在这里插入图片描述](https://img-blog.csdnimg.cn/ccde1a9f657d4e7587012bf753f36095.png) |\n\n### 8.3. 选择模型以及参数\n\n并且在下面的部分，也设置了几个参数，首先是使用的模型，根据前面所说的，一共有五种模型，并且可以调节传入的shape，这里注意一下，如果选择tiny模型，要勾选☑️使用tiny模型，要不默认全是yolov4模型，然后tiny模型的shape统一只有416，只有yolov4模型有一个608和416，可以根据自己的情况选择。\n\n除此之外，还可以调节一下confidence和nms的参数，默认分别是0.5和0.3，这个是可以通过滑动杆来修改的\n\n![在这里插入图片描述](https://img-blog.csdnimg.cn/88e387f27b3747c58a6c910281837c03.png)\n\n\n\n\n\n\n\n## 9. 参考Reference\n\n- [https://github.com/bubbliiiing/yolov4-pytorch](https://github.com/bubbliiiing/yolov4-pytorch)\n- [https://github.com/qqwweee/keras-yolo3/](https://github.com/qqwweee/keras-yolo3/) \n- [https://github.com/Ma-Dan/keras-yolo4](https://github.com/Ma-Dan/keras-yolo4) \n\n\n\n\n\n## 10. 代码权重可复现，已开源（求🌟🌟🌟）\n\n所有的上述代码权重全部可复现，已经全部开源，有需要可以自取https://github.com/Kedreamix/YoloGesture\n\n有问题欢迎在issue中讨论，最后创作不易，给我个星星吧哈哈哈star一下，🌟🌟🌟" }, { "path": "VOCdevkit/VOC2007/Annotations/1.xml", "content": "\n\tJPEGImages\n\t1.jpg\n\tE:\\handpose_x_gesture_v2\\JPEGImages\\1.jpg\n\t\n\t\tUnknown\n\t\n\t\n\t\t175\n\t\t223\n\t\t3\n\t\n\t0\n\t\n\t\tdown\n\t\tUnspecified\n\t\t0\n\t\t0\n\t\t\n\t\t\t21\n\t\t\t7\n\t\t\t174\n\t\t\t210\n\t\t\n\t\n\n" }, { "path": "VOCdevkit/VOC2007/Annotations/2.xml", "content": "\n\tJPEGImages\n\t2.jpg\n\tE:\\handpose_x_gesture_v2\\JPEGImages\\2.jpg\n\t\n\t\tUnknown\n\t\n\t\n\t\t274\n\t\t295\n\t\t3\n\t\n\t0\n\t\n\t\tdown\n\t\tUnspecified\n\t\t0\n\t\t0\n\t\t\n\t\t\t44\n\t\t\t20\n\t\t\t259\n\t\t\t264\n\t\t\n\t\n\n" }, { "path": "VOCdevkit/VOC2007/Annotations/3.xml", "content": "\n\tJPEGImages\n\t3.jpg\n\tE:\\handpose_x_gesture_v2\\JPEGImages\\3.jpg\n\t\n\t\tUnknown\n\t\n\t\n\t\t325\n\t\t363\n\t\t3\n\t\n\t0\n\t\n\t\tdown\n\t\tUnspecified\n\t\t0\n\t\t0\n\t\t\n\t\t\t30\n\t\t\t59\n\t\t\t261\n\t\t\t297\n\t\t\n\t\n\n" }, { "path": "VOCdevkit/VOC2007/Annotations/4.xml", "content": "\n\tJPEGImages\n\t4.jpg\n\tE:\\handpose_x_gesture_v2\\JPEGImages\\4.jpg\n\t\n\t\tUnknown\n\t\n\t\n\t\t306\n\t\t299\n\t\t3\n\t\n\t0\n\t\n\t\tdown\n\t\tUnspecified\n\t\t0\n\t\t0\n\t\t\n\t\t\t44\n\t\t\t45\n\t\t\t264\n\t\t\t256\n\t\t\n\t\n\n" }, { "path": "VOCdevkit/VOC2007/Annotations/5.xml", "content": "\n\tJPEGImages\n\t5.jpg\n\tE:\\handpose_x_gesture_v2\\JPEGImages\\5.jpg\n\t\n\t\tUnknown\n\t\n\t\n\t\t191\n\t\t211\n\t\t3\n\t\n\t0\n\t\n\t\tdown\n\t\tUnspecified\n\t\t0\n\t\t0\n\t\t\n\t\t\t31\n\t\t\t19\n\t\t\t152\n\t\t\t167\n\t\t\n\t\n\n" }, { "path": "VOCdevkit/VOC2007/Annotations/README.md", "content": "存放标签文件" }, { "path": "VOCdevkit/VOC2007/ImageSets/Main/README.md", "content": "存放训练索引文件" }, { "path": "VOCdevkit/VOC2007/ImageSets/Main/test.txt", "content": "" }, { "path": "VOCdevkit/VOC2007/ImageSets/Main/train.txt", "content": "10\n100\n1000\n1001\n1002\n1003\n1005\n1006\n1007\n1008\n1009\n101\n1010\n1011\n1012\n1013\n1014\n1015\n1016\n1017\n1019\n102\n1020\n1021\n1022\n1023\n1024\n1025\n1026\n1028\n1029\n103\n1030\n1031\n1032\n1033\n1034\n1035\n1036\n1037\n1038\n1039\n104\n1040\n1041\n1042\n1043\n1044\n1045\n1046\n1047\n1048\n105\n1050\n1051\n1052\n1053\n1054\n1056\n1057\n1058\n1059\n106\n1060\n1061\n1062\n1063\n1064\n1065\n1066\n1067\n1068\n1069\n107\n1070\n1071\n1072\n1074\n1075\n1076\n1077\n1078\n1079\n108\n1080\n1081\n1083\n1084\n1085\n1086\n1088\n1089\n109\n1090\n1091\n1094\n1096\n1098\n11\n110\n1100\n1101\n1102\n1103\n1104\n1105\n1106\n1107\n1108\n1109\n111\n1111\n1112\n1113\n1114\n1115\n1116\n1117\n1118\n1119\n112\n1120\n1121\n1122\n1123\n1124\n1125\n1126\n1127\n1130\n1131\n1132\n1133\n1134\n1135\n1136\n1137\n1138\n1139\n114\n1140\n1141\n1142\n1144\n1145\n1146\n1147\n1148\n1149\n1150\n1151\n1152\n1153\n1155\n1156\n1157\n1158\n116\n1160\n1161\n1162\n1163\n1164\n1165\n1166\n1167\n1168\n1169\n117\n1170\n1171\n1172\n1173\n1174\n1175\n1176\n1178\n1179\n118\n1180\n1181\n1182\n1183\n1184\n1185\n1186\n1187\n1188\n1189\n119\n1190\n1191\n1192\n1193\n1195\n1196\n1197\n1198\n1199\n12\n120\n1202\n1203\n1204\n1205\n1206\n1207\n1208\n1209\n121\n1210\n1211\n1213\n1214\n1216\n1217\n1218\n1219\n122\n1220\n1221\n1222\n1223\n1224\n1225\n1226\n1227\n1228\n1229\n123\n1230\n1231\n1232\n1234\n1235\n1236\n1237\n1238\n1239\n124\n1240\n1241\n1242\n1243\n1244\n1247\n1248\n1249\n1250\n1251\n1252\n1253\n1254\n1255\n1256\n1257\n1259\n126\n1260\n1261\n1263\n1264\n1265\n1266\n1267\n1268\n1269\n127\n1270\n1271\n1272\n1273\n1274\n1275\n1276\n1277\n1278\n1279\n128\n1280\n1281\n1282\n1284\n1285\n1286\n1287\n1288\n1289\n129\n1290\n1291\n1292\n1293\n1295\n1296\n1297\n1298\n1299\n13\n130\n1300\n1301\n1302\n1303\n1304\n1305\n1306\n1307\n1308\n131\n1310\n1311\n1312\n1313\n1314\n1316\n1317\n1318\n1320\n1321\n1322\n1323\n1324\n1325\n1326\n1327\n1328\n1329\n133\n1330\n1331\n1332\n1333\n1334\n1335\n1336\n1338\n134\n1340\n1342\n1343\n1344\n1345\n1346\n1347\n1348\n1349\n135\n1350\n1352\n1353\n1354\n1355\n1356\n1357\n1358\n1359\n136\n1360\n1361\n1362\n1363\n1364\n1365\n1366\n1367\n1369\n137\n1370\n1372\n1373\n1374\n1375\n1376\n1377\n1378\n1379\n138\n1380\n1381\n1382\n1383\n1384\n1385\n1386\n1387\n1388\n1389\n139\n1390\n1391\n1392\n1394\n1395\n1396\n1397\n1398\n1399\n14\n1400\n1401\n1402\n1403\n1404\n1405\n1406\n1407\n1409\n141\n1410\n1411\n1412\n1413\n1414\n1416\n1417\n1418\n142\n1420\n1421\n1422\n1424\n1425\n1426\n1427\n1428\n1429\n143\n1430\n1431\n1432\n1433\n1434\n1435\n1436\n1437\n1439\n1440\n1441\n1442\n1443\n1444\n1445\n1446\n1448\n1449\n145\n1450\n1452\n1454\n1456\n1458\n1459\n146\n1460\n1461\n1462\n1463\n1464\n1465\n1466\n1467\n1468\n147\n1470\n1471\n1472\n1473\n1474\n1475\n1476\n1477\n1478\n1479\n148\n1480\n1482\n1483\n1484\n1485\n1487\n1488\n1489\n149\n1490\n1491\n1492\n1494\n1495\n1496\n1497\n1498\n1499\n15\n150\n1500\n1501\n1502\n1503\n1504\n1506\n1507\n1508\n1509\n151\n1510\n1511\n1512\n1513\n1515\n1516\n1517\n1518\n1519\n152\n1520\n1521\n1522\n1523\n1524\n1525\n1526\n1527\n1528\n1529\n1530\n1532\n1533\n1534\n1535\n1536\n1537\n1539\n154\n1540\n1541\n1542\n1543\n1544\n1545\n1546\n1547\n1548\n1549\n1550\n1551\n1552\n1554\n1555\n1556\n1557\n1558\n1559\n156\n1560\n1561\n1562\n1563\n1564\n1565\n1566\n1567\n1568\n1569\n157\n1571\n1572\n1573\n1574\n1575\n1576\n1577\n1578\n1579\n158\n1580\n1581\n1582\n1583\n1584\n1585\n1586\n1587\n1588\n1589\n159\n1591\n1592\n1593\n1594\n1595\n1596\n1597\n1598\n1599\n16\n160\n1600\n161\n162\n163\n164\n165\n166\n168\n169\n17\n170\n171\n172\n173\n174\n175\n176\n177\n178\n18\n182\n183\n184\n185\n186\n187\n189\n19\n190\n193\n194\n195\n196\n197\n198\n199\n2\n200\n201\n202\n203\n204\n205\n206\n207\n208\n209\n21\n210\n212\n213\n214\n215\n216\n217\n218\n219\n22\n220\n221\n222\n223\n224\n225\n226\n227\n228\n230\n231\n232\n233\n234\n235\n236\n237\n238\n239\n24\n240\n241\n242\n243\n244\n245\n246\n247\n25\n250\n251\n252\n253\n254\n255\n256\n257\n258\n259\n26\n260\n261\n262\n263\n265\n266\n267\n268\n269\n27\n270\n271\n272\n273\n274\n275\n276\n277\n278\n279\n28\n280\n281\n284\n285\n286\n287\n288\n291\n294\n295\n296\n297\n298\n299\n3\n30\n300\n301\n302\n303\n304\n305\n306\n307\n308\n309\n31\n310\n311\n312\n313\n314\n315\n316\n317\n318\n319\n32\n320\n321\n322\n323\n324\n325\n326\n327\n328\n329\n33\n330\n331\n332\n334\n335\n337\n338\n339\n34\n340\n341\n342\n343\n344\n345\n346\n347\n348\n349\n35\n350\n351\n353\n354\n355\n356\n357\n358\n36\n360\n361\n362\n363\n364\n365\n366\n367\n368\n369\n37\n370\n371\n372\n373\n374\n375\n376\n377\n378\n379\n38\n380\n382\n383\n384\n385\n386\n387\n388\n389\n39\n390\n391\n392\n393\n394\n395\n396\n397\n398\n399\n4\n40\n400\n401\n402\n404\n405\n406\n407\n408\n409\n41\n410\n411\n412\n413\n414\n415\n416\n417\n418\n419\n42\n420\n421\n422\n423\n425\n426\n427\n428\n429\n43\n430\n431\n432\n433\n434\n436\n437\n438\n439\n44\n440\n441\n442\n443\n444\n445\n446\n447\n448\n449\n45\n450\n451\n452\n453\n454\n455\n457\n458\n459\n46\n461\n462\n464\n465\n466\n467\n468\n47\n470\n471\n472\n473\n474\n475\n476\n477\n478\n479\n48\n480\n481\n482\n483\n484\n485\n486\n487\n488\n489\n49\n490\n491\n492\n493\n494\n495\n496\n497\n498\n499\n5\n50\n500\n501\n502\n503\n504\n505\n506\n507\n508\n509\n51\n512\n513\n514\n515\n516\n517\n518\n519\n520\n521\n522\n523\n524\n526\n527\n528\n529\n53\n530\n532\n533\n534\n536\n537\n539\n54\n540\n541\n542\n543\n544\n545\n546\n547\n549\n55\n551\n552\n553\n554\n556\n557\n559\n56\n561\n562\n563\n564\n565\n566\n567\n568\n569\n57\n570\n571\n572\n573\n575\n577\n578\n579\n58\n580\n581\n582\n583\n585\n586\n587\n588\n589\n59\n590\n591\n592\n593\n594\n596\n597\n598\n599\n6\n600\n601\n602\n603\n604\n605\n606\n608\n609\n61\n610\n611\n612\n613\n614\n615\n616\n617\n618\n619\n62\n620\n621\n622\n623\n625\n626\n627\n628\n629\n63\n630\n631\n632\n633\n634\n635\n636\n637\n638\n639\n64\n640\n641\n642\n643\n644\n645\n646\n647\n648\n649\n65\n650\n651\n652\n653\n654\n655\n656\n657\n658\n66\n660\n662\n663\n664\n665\n666\n667\n668\n669\n67\n670\n671\n672\n673\n675\n676\n677\n678\n679\n68\n680\n681\n682\n683\n684\n685\n686\n687\n688\n689\n69\n690\n691\n692\n693\n694\n695\n696\n698\n699\n70\n700\n701\n702\n703\n704\n705\n706\n708\n709\n71\n710\n711\n713\n714\n715\n716\n717\n718\n719\n72\n720\n721\n722\n723\n724\n725\n726\n727\n729\n730\n731\n732\n733\n734\n735\n736\n737\n738\n739\n740\n741\n742\n743\n744\n745\n746\n747\n748\n749\n75\n750\n751\n752\n753\n755\n756\n757\n759\n76\n760\n761\n762\n763\n764\n766\n767\n769\n77\n770\n771\n772\n773\n774\n775\n776\n777\n778\n779\n78\n780\n781\n783\n784\n785\n786\n787\n788\n789\n79\n790\n791\n792\n793\n794\n795\n796\n797\n798\n799\n8\n80\n800\n801\n802\n803\n804\n805\n806\n807\n808\n809\n81\n810\n812\n813\n814\n815\n816\n819\n82\n820\n821\n822\n823\n824\n825\n826\n827\n828\n829\n83\n830\n831\n832\n833\n834\n836\n837\n838\n839\n84\n840\n841\n842\n843\n844\n845\n846\n847\n848\n849\n85\n850\n851\n852\n853\n854\n855\n856\n857\n858\n859\n86\n860\n861\n862\n863\n864\n866\n867\n868\n869\n87\n870\n872\n873\n874\n876\n877\n878\n879\n88\n880\n882\n883\n884\n885\n886\n887\n888\n889\n89\n890\n891\n892\n893\n895\n896\n897\n899\n9\n900\n901\n902\n903\n904\n905\n906\n908\n909\n91\n910\n911\n912\n913\n914\n915\n916\n917\n918\n919\n92\n920\n921\n923\n924\n925\n926\n927\n928\n929\n93\n930\n931\n932\n933\n934\n935\n936\n938\n94\n940\n941\n943\n944\n945\n946\n947\n948\n949\n95\n950\n951\n952\n953\n954\n955\n956\n957\n958\n959\n96\n960\n961\n963\n965\n966\n967\n968\n969\n97\n970\n971\n972\n973\n974\n975\n976\n977\n978\n979\n98\n981\n982\n983\n984\n985\n986\n987\n988\n989\n99\n990\n991\n992\n994\n995\n996\n997\n998\n" }, { "path": "VOCdevkit/VOC2007/ImageSets/Main/trainval.txt", "content": "1\n10\n100\n1000\n1001\n1002\n1003\n1004\n1005\n1006\n1007\n1008\n1009\n101\n1010\n1011\n1012\n1013\n1014\n1015\n1016\n1017\n1018\n1019\n102\n1020\n1021\n1022\n1023\n1024\n1025\n1026\n1027\n1028\n1029\n103\n1030\n1031\n1032\n1033\n1034\n1035\n1036\n1037\n1038\n1039\n104\n1040\n1041\n1042\n1043\n1044\n1045\n1046\n1047\n1048\n1049\n105\n1050\n1051\n1052\n1053\n1054\n1055\n1056\n1057\n1058\n1059\n106\n1060\n1061\n1062\n1063\n1064\n1065\n1066\n1067\n1068\n1069\n107\n1070\n1071\n1072\n1073\n1074\n1075\n1076\n1077\n1078\n1079\n108\n1080\n1081\n1082\n1083\n1084\n1085\n1086\n1087\n1088\n1089\n109\n1090\n1091\n1092\n1093\n1094\n1095\n1096\n1097\n1098\n1099\n11\n110\n1100\n1101\n1102\n1103\n1104\n1105\n1106\n1107\n1108\n1109\n111\n1110\n1111\n1112\n1113\n1114\n1115\n1116\n1117\n1118\n1119\n112\n1120\n1121\n1122\n1123\n1124\n1125\n1126\n1127\n1128\n1129\n113\n1130\n1131\n1132\n1133\n1134\n1135\n1136\n1137\n1138\n1139\n114\n1140\n1141\n1142\n1143\n1144\n1145\n1146\n1147\n1148\n1149\n115\n1150\n1151\n1152\n1153\n1154\n1155\n1156\n1157\n1158\n1159\n116\n1160\n1161\n1162\n1163\n1164\n1165\n1166\n1167\n1168\n1169\n117\n1170\n1171\n1172\n1173\n1174\n1175\n1176\n1177\n1178\n1179\n118\n1180\n1181\n1182\n1183\n1184\n1185\n1186\n1187\n1188\n1189\n119\n1190\n1191\n1192\n1193\n1194\n1195\n1196\n1197\n1198\n1199\n12\n120\n1200\n1201\n1202\n1203\n1204\n1205\n1206\n1207\n1208\n1209\n121\n1210\n1211\n1212\n1213\n1214\n1215\n1216\n1217\n1218\n1219\n122\n1220\n1221\n1222\n1223\n1224\n1225\n1226\n1227\n1228\n1229\n123\n1230\n1231\n1232\n1233\n1234\n1235\n1236\n1237\n1238\n1239\n124\n1240\n1241\n1242\n1243\n1244\n1245\n1246\n1247\n1248\n1249\n125\n1250\n1251\n1252\n1253\n1254\n1255\n1256\n1257\n1258\n1259\n126\n1260\n1261\n1262\n1263\n1264\n1265\n1266\n1267\n1268\n1269\n127\n1270\n1271\n1272\n1273\n1274\n1275\n1276\n1277\n1278\n1279\n128\n1280\n1281\n1282\n1283\n1284\n1285\n1286\n1287\n1288\n1289\n129\n1290\n1291\n1292\n1293\n1294\n1295\n1296\n1297\n1298\n1299\n13\n130\n1300\n1301\n1302\n1303\n1304\n1305\n1306\n1307\n1308\n1309\n131\n1310\n1311\n1312\n1313\n1314\n1315\n1316\n1317\n1318\n1319\n132\n1320\n1321\n1322\n1323\n1324\n1325\n1326\n1327\n1328\n1329\n133\n1330\n1331\n1332\n1333\n1334\n1335\n1336\n1337\n1338\n1339\n134\n1340\n1341\n1342\n1343\n1344\n1345\n1346\n1347\n1348\n1349\n135\n1350\n1351\n1352\n1353\n1354\n1355\n1356\n1357\n1358\n1359\n136\n1360\n1361\n1362\n1363\n1364\n1365\n1366\n1367\n1368\n1369\n137\n1370\n1371\n1372\n1373\n1374\n1375\n1376\n1377\n1378\n1379\n138\n1380\n1381\n1382\n1383\n1384\n1385\n1386\n1387\n1388\n1389\n139\n1390\n1391\n1392\n1393\n1394\n1395\n1396\n1397\n1398\n1399\n14\n140\n1400\n1401\n1402\n1403\n1404\n1405\n1406\n1407\n1408\n1409\n141\n1410\n1411\n1412\n1413\n1414\n1415\n1416\n1417\n1418\n1419\n142\n1420\n1421\n1422\n1423\n1424\n1425\n1426\n1427\n1428\n1429\n143\n1430\n1431\n1432\n1433\n1434\n1435\n1436\n1437\n1438\n1439\n144\n1440\n1441\n1442\n1443\n1444\n1445\n1446\n1447\n1448\n1449\n145\n1450\n1451\n1452\n1453\n1454\n1455\n1456\n1457\n1458\n1459\n146\n1460\n1461\n1462\n1463\n1464\n1465\n1466\n1467\n1468\n1469\n147\n1470\n1471\n1472\n1473\n1474\n1475\n1476\n1477\n1478\n1479\n148\n1480\n1481\n1482\n1483\n1484\n1485\n1486\n1487\n1488\n1489\n149\n1490\n1491\n1492\n1493\n1494\n1495\n1496\n1497\n1498\n1499\n15\n150\n1500\n1501\n1502\n1503\n1504\n1505\n1506\n1507\n1508\n1509\n151\n1510\n1511\n1512\n1513\n1514\n1515\n1516\n1517\n1518\n1519\n152\n1520\n1521\n1522\n1523\n1524\n1525\n1526\n1527\n1528\n1529\n153\n1530\n1531\n1532\n1533\n1534\n1535\n1536\n1537\n1538\n1539\n154\n1540\n1541\n1542\n1543\n1544\n1545\n1546\n1547\n1548\n1549\n155\n1550\n1551\n1552\n1553\n1554\n1555\n1556\n1557\n1558\n1559\n156\n1560\n1561\n1562\n1563\n1564\n1565\n1566\n1567\n1568\n1569\n157\n1570\n1571\n1572\n1573\n1574\n1575\n1576\n1577\n1578\n1579\n158\n1580\n1581\n1582\n1583\n1584\n1585\n1586\n1587\n1588\n1589\n159\n1590\n1591\n1592\n1593\n1594\n1595\n1596\n1597\n1598\n1599\n16\n160\n1600\n161\n162\n163\n164\n165\n166\n167\n168\n169\n17\n170\n171\n172\n173\n174\n175\n176\n177\n178\n179\n18\n180\n181\n182\n183\n184\n185\n186\n187\n188\n189\n19\n190\n191\n192\n193\n194\n195\n196\n197\n198\n199\n2\n20\n200\n201\n202\n203\n204\n205\n206\n207\n208\n209\n21\n210\n211\n212\n213\n214\n215\n216\n217\n218\n219\n22\n220\n221\n222\n223\n224\n225\n226\n227\n228\n229\n23\n230\n231\n232\n233\n234\n235\n236\n237\n238\n239\n24\n240\n241\n242\n243\n244\n245\n246\n247\n248\n249\n25\n250\n251\n252\n253\n254\n255\n256\n257\n258\n259\n26\n260\n261\n262\n263\n264\n265\n266\n267\n268\n269\n27\n270\n271\n272\n273\n274\n275\n276\n277\n278\n279\n28\n280\n281\n282\n283\n284\n285\n286\n287\n288\n289\n29\n290\n291\n292\n293\n294\n295\n296\n297\n298\n299\n3\n30\n300\n301\n302\n303\n304\n305\n306\n307\n308\n309\n31\n310\n311\n312\n313\n314\n315\n316\n317\n318\n319\n32\n320\n321\n322\n323\n324\n325\n326\n327\n328\n329\n33\n330\n331\n332\n333 \n333\n334\n335\n336\n337\n338\n339\n34\n340\n341\n342\n343\n344\n345\n346\n347\n348\n349\n35\n350\n351\n352\n353\n354\n355\n356\n357\n358\n359\n36\n360\n361\n362\n363\n364\n365\n366\n367\n368\n369\n37\n370\n371\n372\n373\n374\n375\n376\n377\n378\n379\n38\n380\n381\n382\n383\n384\n385\n386\n387\n388\n389\n39\n390\n391\n392\n393\n394\n395\n396\n397\n398\n399\n4\n40\n400\n401\n402\n403\n404\n405\n406\n407\n408\n409\n41\n410\n411\n412\n413\n414\n415\n416\n417\n418\n419\n42\n420\n421\n422\n423\n424\n425\n426\n427\n428\n429\n43\n430\n431\n432\n433\n434\n435\n436\n437\n438\n439\n44\n440\n441\n442\n443\n444\n445\n446\n447\n448\n449\n45\n450\n451\n452\n453\n454\n455\n456\n457\n458\n459\n46\n460\n461\n462\n463\n464\n465\n466\n467\n468\n469\n47\n470\n471\n472\n473\n474\n475\n476\n477\n478\n479\n48\n480\n481\n482\n483\n484\n485\n486\n487\n488\n489\n49\n490\n491\n492\n493\n494\n495\n496\n497\n498\n499\n5\n50\n500\n501\n502\n503\n504\n505\n506\n507\n508\n509\n51\n510\n511\n512\n513\n514\n515\n516\n517\n518\n519\n52\n520\n521\n522\n523\n524\n525\n526\n527\n528\n529\n53\n530\n531\n532\n533\n534\n535\n536\n537\n538\n539\n54\n540\n541\n542\n543\n544\n545\n546\n547\n548\n549\n55\n550\n551\n552\n553\n554\n555\n556\n557\n558\n559\n56\n560\n561\n562\n563\n564\n565\n566\n567\n568\n569\n57\n570\n571\n572\n573\n574\n575\n576\n577\n578\n579\n58\n580\n581\n582\n583\n584\n585\n586\n587\n588\n589\n59\n590\n591\n592\n593\n594\n595\n596\n597\n598\n599\n6\n60\n600\n601\n602\n603\n604\n605\n606\n607\n608\n609\n61\n610\n611\n612\n613\n614\n615\n616\n617\n618\n619\n62\n620\n621\n622\n623\n624\n625\n626\n627\n628\n629\n63\n630\n631\n632\n633\n634\n635\n636\n637\n638\n639\n64\n640\n641\n642\n643\n644\n645\n646\n647\n648\n649\n65\n650\n651\n652\n653\n654\n655\n656\n657\n658\n659\n66\n660\n661\n662\n663\n664\n665\n666\n667\n668\n669\n67\n670\n671\n672\n673\n674\n675\n676\n677\n678\n679\n68\n680\n681\n682\n683\n684\n685\n686\n687\n688\n689\n69\n690\n691\n692\n693\n694\n695\n696\n697\n698\n699\n7\n70\n700\n701\n702\n703\n704\n705\n706\n707\n708\n709\n71\n710\n711\n712\n713\n714\n715\n716\n717\n718\n719\n72\n720\n721\n722\n723\n724\n725\n726\n727\n728\n729\n73\n730\n731\n732\n733\n734\n735\n736\n737\n738\n739\n74\n740\n741\n742\n743\n744\n745\n746\n747\n748\n749\n75\n750\n751\n752\n753\n754\n755\n756\n757\n758\n759\n76\n760\n761\n762\n763\n764\n765\n766\n767\n768\n769\n77\n770\n771\n772\n773\n774\n775\n776\n777\n778\n779\n78\n780\n781\n782\n783\n784\n785\n786\n787\n788\n789\n79\n790\n791\n792\n793\n794\n795\n796\n797\n798\n799\n8\n80\n800\n801\n802\n803\n804\n805\n806\n807\n808\n809\n81\n810\n811\n812\n813\n814\n815\n816\n817\n818\n819\n82\n820\n821\n822\n823\n824\n825\n826\n827\n828\n829\n83\n830\n831\n832\n833\n834\n835\n836\n837\n838\n839\n84\n840\n841\n842\n843\n844\n845\n846\n847\n848\n849\n85\n850\n851\n852\n853\n854\n855\n856\n857\n858\n859\n86\n860\n861\n862\n863\n864\n865\n866\n867\n868\n869\n87\n870\n871\n872\n873\n874\n875\n876\n877\n878\n879\n88\n880\n881\n882\n883\n884\n885\n886\n887\n888\n889\n89\n890\n891\n892\n893\n894\n895\n896\n897\n898\n899\n9\n90\n900\n901\n902\n903\n904\n905\n906\n907\n908\n909\n91\n910\n911\n912\n913\n914\n915\n916\n917\n918\n919\n92\n920\n921\n922\n923\n924\n925\n926\n927\n928\n929\n93\n930\n931\n932\n933\n934\n935\n936\n937\n938\n939\n94\n940\n941\n942\n943\n944\n945\n946\n947\n948\n949\n95\n950\n951\n952\n953\n954\n955\n956\n957\n958\n959\n96\n960\n961\n962\n963\n964\n965\n966\n967\n968\n969\n97\n970\n971\n972\n973\n974\n975\n976\n977\n978\n979\n98\n980\n981\n982\n983\n984\n985\n986\n987\n988\n989\n99\n990\n991\n992\n993\n994\n995\n996\n997\n998\n999\n" }, { "path": "VOCdevkit/VOC2007/ImageSets/Main/val.txt", "content": "1\n1004\n1018\n1027\n1049\n1055\n1073\n1082\n1087\n1092\n1093\n1095\n1097\n1099\n1110\n1128\n1129\n113\n1143\n115\n1154\n1159\n1177\n1194\n1200\n1201\n1212\n1215\n1233\n1245\n1246\n125\n1258\n1262\n1283\n1294\n1309\n1315\n1319\n132\n1337\n1339\n1341\n1351\n1368\n1371\n1393\n140\n1408\n1415\n1419\n1423\n1438\n144\n1447\n1451\n1453\n1455\n1457\n1469\n1481\n1486\n1493\n1505\n1514\n153\n1531\n1538\n155\n1553\n1570\n1590\n167\n179\n180\n181\n188\n191\n192\n20\n211\n229\n23\n248\n249\n264\n282\n283\n289\n29\n290\n292\n293\n333 \n333\n336\n352\n359\n381\n403\n424\n435\n456\n460\n463\n469\n510\n511\n52\n525\n531\n535\n538\n548\n550\n555\n558\n560\n574\n576\n584\n595\n60\n607\n624\n659\n661\n674\n697\n7\n707\n712\n728\n73\n74\n754\n758\n765\n768\n782\n811\n817\n818\n835\n865\n871\n875\n881\n894\n898\n90\n907\n922\n937\n939\n942\n962\n964\n980\n993\n999\n" }, { "path": "YOLOv4-study学习资料md", "content": "# YOLOv4 学习资料\n\n![在这里插入图片描述](https://img-blog.csdnimg.cn/e3551e344873465d8ad884f856d652ed.png)\n\n[Tianxiaomo](https://github.com/Tianxiaomo)/**[pytorch-YOLOv4](https://github.com/Tianxiaomo/pytorch-YOLOv4)** star 3.5k \n\nPyTorch ,ONNX and TensorRT implementation of *YOLOv4*\n\n[WongKinYiu](https://github.com/WongKinYiu)/**[PyTorch_YOLOv4](https://github.com/WongKinYiu/PyTorch_YOLOv4)** star 1.5k \n\nPyTorch implementation of *YOLOv4*\n\n[argusswift](https://github.com/argusswift)/**[YOLOv4-pytorch ](https://github.com/argusswift/YOLOv4-pytorch)** star 1.4k \n\nThis is a pytorch repository of *YOLOv4*, attentive *YOLOv4* and mobilenet *YOLOv4* with PASCAL VOC and COCO\n\n [bubbliiiing/*yolov4*-pytorch ](https://github.com/bubbliiiing/yolov4-pytorch) star 1.2k\n\n这是一个*YoloV4*-pytorch的源码，可以用于训练自己的模型。\n\n\n\n\n\n## 扩展\n\n [Bil369](https://github.com/Bil369)/**[MaskDetect-YOLOv4-PyTorch](https://github.com/Bil369/MaskDetect-YOLOv4-PyTorch)**\n\n基于*PyTorch*&*YOLOv4*实现的口罩佩戴检测 ⭐ 自建口罩数据集分享\n\n[bobo0810](https://github.com/bobo0810)/**[PytorchNetHub](https://github.com/bobo0810/PytorchNetHub)**\n\n项目注释+论文复现+算法竞赛+Pytorch指北\n\n[Bil369](https://github.com/Bil369)/**[YOLOv4-PyTorch-Simple-Implementation](https://github.com/Bil369/YOLOv4-PyTorch-Simple-Implementation)**\n\n*YOLOv4* *PyTorch* Simple Implementation" }, { "path": "detect.py", "content": "#-----------------------------------------------------------------------#\n# detect.py 是用来尝试利用小模型半自动化进行标注数据\n#-----------------------------------------------------------------------#\nimport numpy as np\nfrom PIL import Image\nfrom get_yaml import get_config\n\nfrom yolo import YOLO\nfrom gen_annotation import GEN_Annotations\n\nif __name__ == \"__main__\":# 配置文件\n # 配置文件\n config = get_config()\n yolo = YOLO()\n\n dir_detect_path = config['dir_detect_path']\n detect_save_path = config['detect_save_path']\n \n import os\n from tqdm import tqdm\n \n img_names = os.listdir(dir_detect_path)\n for img_name in tqdm(img_names):\n \n if img_name.lower().endswith(('.bmp', '.dib', '.png', '.jpg', '.jpeg', '.pbm', '.pgm', '.ppm', '.tif', '.tiff')):\n # if int(img_name.split('.')[0][-4:]) < 355:\n # continue\n image_path = os.path.join(dir_detect_path, img_name)\n image = Image.open(image_path)\n boxes = yolo.get_box(image)\n if not os.path.exists(detect_save_path):\n os.makedirs(detect_save_path)\n\n annotation = GEN_Annotations(img_name)\n w,h = np.array(np.shape(image)[0:2])\n annotation.set_size(w,h,3)\n if boxes:\n for box in boxes:\n label,ymin,xmin,ymax,xmax = box\n annotation.add_pic_attr(label,xmin,ymin,xmax,ymax)\n annotation_path = os.path.join(detect_save_path, img_name.split('.')[0])\n annotation.savefile(\"{}.xml\".format(annotation_path ))\n # print(img_name,'已经半自动标注')" }, { "path": "gen_annotation.py", "content": "from lxml import etree\n \nclass GEN_Annotations:\n def __init__(self, filename):\n self.root = etree.Element(\"annotation\")\n \n child1 = etree.SubElement(self.root, \"folder\")\n child1.text = \"VOC2007\"\n \n child2 = etree.SubElement(self.root, \"filename\")\n child2.text = filename\n \n child3 = etree.SubElement(self.root, \"source\")\n \n child4 = etree.SubElement(child3, \"annotation\")\n child4.text = \"PASCAL VOC2007\"\n child5 = etree.SubElement(child3, \"database\")\n child5.text = \"Unknown\"\n \n## child6 = etree.SubElement(child3, \"image\")\n## child6.text = \"flickr\"\n## child7 = etree.SubElement(child3, \"flickrid\")\n## child7.text = \"35435\"\n \n \n def set_size(self,witdh,height,channel):\n size = etree.SubElement(self.root, \"size\")\n widthn = etree.SubElement(size, \"width\")\n widthn.text = str(witdh)\n heightn = etree.SubElement(size, \"height\")\n heightn.text = str(height)\n channeln = etree.SubElement(size, \"depth\")\n channeln.text = str(channel)\n def savefile(self,filename):\n tree = etree.ElementTree(self.root)\n tree.write(filename, pretty_print=True, xml_declaration=False, encoding='utf-8')\n def add_pic_attr(self,label,xmin,ymin,xmax,ymax):\n object = etree.SubElement(self.root, \"object\")\n namen = etree.SubElement(object, \"name\")\n namen.text = label\n bndbox = etree.SubElement(object, \"bndbox\")\n xminn = etree.SubElement(bndbox, \"xmin\")\n xminn.text = str(xmin)\n yminn = etree.SubElement(bndbox, \"ymin\")\n yminn.text = str(ymin)\n xmaxn = etree.SubElement(bndbox, \"xmax\")\n xmaxn.text = str(xmax)\n ymaxn = etree.SubElement(bndbox, \"ymax\")\n ymaxn.text = str(ymax)\n \n \nif __name__ == '__main__':\n filename=\"000001.jpg\"\n anno= GEN_Annotations(filename)\n anno.set_size(1280,720,3)\n for i in range(3):\n xmin=i+1\n ymin=i+10\n xmax=i+100\n ymax=i+100\n anno.add_pic_attr(\"pikachu\",xmin,ymin,xmax,ymax)\n anno.savefile(\"00001.xml\")\n" }, { "path": "gesture.streamlit.py", "content": "\"\"\"Create an Object Detection Web App using PyTorch and Streamlit.\"\"\"\n# import libraries\nfrom PIL import Image\nfrom torchvision import models, transforms\nimport torch\nimport streamlit as st\nfrom yolo import YOLO\nimport os\nimport urllib\nimport numpy as np\nfrom streamlit_webrtc import webrtc_streamer, WebRtcMode, RTCConfiguration\nimport av\n# 设置网页的icon\nst.set_page_config(page_title='Gesture Detector', page_icon='✌',\n layout='centered', initial_sidebar_state='expanded')\n\nRTC_CONFIGURATION = RTCConfiguration(\n {\n \"RTCIceServer\": [{\n \"urls\": [\"stun:stun.l.google.com:19302\"],\n \"username\": \"pikachu\",\n \"credential\": \"1234\",\n }]\n }\n)\ndef main():\n # Render the readme as markdown using st.markdown.\n readme_text = st.markdown(open(\"instructions.md\",encoding='utf-8').read())\n\n \n # Once we have the dependencies, add a selector for the app mode on the sidebar.\n st.sidebar.title(\"What to do\")\n app_mode = st.sidebar.selectbox(\"Choose the app mode\",\n [\"Show instructions\", \"Run the app\", \"Show the source code\"])\n if app_mode == \"Show instructions\":\n st.sidebar.success('To continue select \"Run the app\".')\n elif app_mode == \"Show the source code\":\n readme_text.empty()\n st.code(open(\"gesture.streamlit.py\",encoding='utf-8').read())\n elif app_mode == \"Run the app\":\n # Download external dependencies.\n for filename in EXTERNAL_DEPENDENCIES.keys():\n download_file(filename)\n\n readme_text.empty()\n run_the_app()\n\n# External files to download.\nEXTERNAL_DEPENDENCIES = {\n \"yolov4_tiny.pth\": {\n \"url\": \"https://github.com/Kedreamix/YoloGesture/releases/download/v1.0/yolov4_tiny.pth\",\n \"size\": 23631189 \n },\n \"yolov4_SE.pth\": {\n \"url\": \"https://github.com/Kedreamix/YoloGesture/releases/download/v1.0/yolov4_SE.pth\",\n \"size\": 23806027\n },\n \"yolov4_CBAM.pth\":{\n \"url\": \"https://github.com/Kedreamix/YoloGesture/releases/download/v1.0/yolov4_CBAM.pth\",\n \"size\": 23981478\n },\n \"yolov4_ECA.pth\":{\n \"url\": \"https://github.com/Kedreamix/YoloGesture/releases/download/v1.0/yolov4_ECA.pth\",\n \"size\": 23632688\n },\n \"yolov4_weights_ep150_608.pth\":{\n \"url\": \"https://github.com/Kedreamix/YoloGesture/releases/download/v1.0/yolov4_weights_ep150_608.pth\",\n \"size\": 256423031\n },\n \"yolov4_weights_ep150_416.pth\":{\n \"url\": \"https://github.com/Kedreamix/YoloGesture/releases/download/v1.0/yolov4_weights_ep150_416.pth\",\n \"size\": 256423031\n },\n}\n\n\n# This file downloader demonstrates Streamlit animation.\ndef download_file(file_path):\n # Don't download the file twice. (If possible, verify the download using the file length.)\n if os.path.exists(file_path):\n if \"size\" not in EXTERNAL_DEPENDENCIES[file_path]:\n return\n elif os.path.getsize(file_path) == EXTERNAL_DEPENDENCIES[file_path][\"size\"]:\n return\n # print(os.path.getsize(file_path))\n # These are handles to two visual elements to animate.\n weights_warning, progress_bar = None, None\n try:\n weights_warning = st.warning(\"Downloading %s...\" % file_path)\n progress_bar = st.progress(0)\n with open(file_path, \"wb\") as output_file:\n with urllib.request.urlopen(EXTERNAL_DEPENDENCIES[file_path][\"url\"]) as response:\n length = int(response.info()[\"Content-Length\"])\n counter = 0.0\n MEGABYTES = 2.0 ** 20.0\n while True:\n data = response.read(8192)\n if not data:\n break\n counter += len(data)\n output_file.write(data)\n\n # We perform animation by overwriting the elements.\n weights_warning.warning(\"Downloading %s... (%6.2f/%6.2f MB)\" %\n (file_path, counter / MEGABYTES, length / MEGABYTES))\n progress_bar.progress(min(counter / length, 1.0))\n except Exception as e:\n print(e)\n # Finally, we remove these visual elements by calling .empty().\n finally:\n if weights_warning is not None:\n weights_warning.empty()\n if progress_bar is not None:\n progress_bar.empty()\n\n# This is the main app app itself, which appears when the user selects \"Run the app\".\ndef run_the_app(): \n class Config():\n def __init__(self, weights = 'yolov4_tiny.pth', tiny = True, phi = 0, shape = 416,nms_iou = 0.3, confidence = 0.5):\n self.weights = weights\n self.tiny = tiny\n self.phi = phi\n self.cuda = False\n self.shape = shape\n self.confidence = confidence\n self.nms_iou = nms_iou\n # set title of app\n st.markdown('

✌ Gesture Detection

',\n unsafe_allow_html=True)\n st.sidebar.markdown(\"# Gesture Detection on?\")\n activities = [\"Example\",\"Image\", \"Camera\", \"FPS\", \"Heatmap\",\"Real Time\", \"Video\"]\n choice = st.sidebar.selectbox(\"Choose among the given options:\", activities)\n phi = st.sidebar.selectbox(\"yolov4-tiny 使用的自注意力模式:\",('0tiny','1SE','2CABM','3ECA'))\n print(\"\")\n\n tiny = st.sidebar.checkbox('是否使用 yolov4 tiny 模型')\n if not tiny:\n shape = st.sidebar.selectbox(\"Choose shape to Input:\", [416,608])\n conf,nms = object_detector_ui()\n @st.cache\n def get_yolo(tiny,phi,conf,nms,shape=416):\n weights = 'yolov4_tiny.pth'\n if tiny:\n if phi == '0tiny':\n weights = 'yolov4_tiny.pth'\n elif phi == '1SE':\n weights = 'yolov4_SE.pth'\n elif phi == '2CABM':\n weights = 'yolov4_CBAM.pth'\n elif phi == '3ECA':\n weights = 'yolov4_ECA.pth'\n else:\n if shape == 608:\n weights = 'yolov4_weights_ep150_608.pth'\n elif shape == 416:\n weights = 'yolov4_weights_ep150_416.pth'\n opt = Config(weights = weights, tiny = tiny , phi = int(phi[0]), shape = shape,nms_iou = nms, confidence = conf)\n yolo = YOLO(opt)\n return yolo\n \n if tiny:\n yolo = get_yolo(tiny, phi, conf, nms)\n st.write(\"YOLOV4 tiny 模型加载完毕\")\n else:\n yolo = get_yolo(tiny, phi, conf, nms, shape)\n st.write(\"YOLOV4 模型加载完毕\")\n \n if choice == 'Image':\n detect_image(yolo)\n elif choice =='Camera':\n detect_camera(yolo)\n elif choice == 'FPS':\n detect_fps(yolo)\n elif choice == \"Heatmap\":\n detect_heatmap(yolo)\n elif choice == \"Example\":\n detect_example(yolo)\n elif choice == \"Real Time\":\n detect_realtime(yolo)\n elif choice == \"Video\":\n detect_video(yolo)\n \n\n\n# This sidebar UI lets the user select parameters for the YOLO object detector.\ndef object_detector_ui():\n st.sidebar.markdown(\"# Model\")\n confidence_threshold = st.sidebar.slider(\"Confidence threshold\", 0.0, 1.0, 0.5, 0.01)\n overlap_threshold = st.sidebar.slider(\"Overlap threshold\", 0.0, 1.0, 0.3, 0.01)\n return confidence_threshold, overlap_threshold\n\ndef predict(image,yolo):\n \"\"\"Return predictions.\n\n Parameters\n ----------\n :param image: uploaded image\n :type image: jpg\n :rtype: list\n :return: none\n \"\"\"\n crop = False\n count = False\n try:\n # image = Image.open(image)\n r_image = yolo.detect_image(image, crop = crop, count=count)\n transform = transforms.Compose([transforms.ToTensor()]) \n result = transform(r_image)\n st.image(result.permute(1,2,0).numpy(), caption = 'Processed Image.', use_column_width = True)\n except Exception as e:\n print(e)\n\ndef fps(image,yolo):\n test_interval = 50\n tact_time = yolo.get_FPS(image, test_interval)\n st.write(str(tact_time) + ' seconds, ', str(1/tact_time),'FPS, @batch_size 1')\n return tact_time\n # print(str(tact_time) + ' seconds, ' + str(1/tact_time) + 'FPS, @batch_size 1')\n\n\ndef detect_image(yolo):\n # enable users to upload images for the model to make predictions\n file_up = st.file_uploader(\"Upload an image\", type = [\"jpg\",\"png\",\"jpeg\"])\n classes = [\"up\",\"down\",\"left\",\"right\",\"front\",\"back\",\"clockwise\",\"anticlockwise\"]\n class_to_idx = {cls: idx for (idx, cls) in enumerate(classes)}\n st.sidebar.markdown(\"See the model preformance and play with it\")\n if file_up is not None:\n with st.spinner(text='Preparing Image'):\n # display image that user uploaded\n image = Image.open(file_up)\n st.image(image, caption = 'Uploaded Image.', use_column_width = True)\n st.balloons()\n detect = st.button(\"开始检测Image\")\n if detect:\n st.write(\"\")\n st.write(\"Just a second ...\")\n predict(image,yolo)\n st.balloons()\n\n\n\ndef detect_camera(yolo):\n picture = st.camera_input(\"Take a picture\")\n if picture:\n filters_to_funcs = {\n \"No filter\": predict,\n \"Heatmap\": heatmap,\n \"FPS\": fps,\n }\n filters = st.selectbox(\"...and now, apply a filter!\", filters_to_funcs.keys())\n image = Image.open(picture)\n with st.spinner(text='Preparing Image'):\n filters_to_funcs[filters](image,yolo)\n st.balloons()\n\ndef detect_fps(yolo):\n file_up = st.file_uploader(\"Upload an image\", type = [\"jpg\",\"png\",\"jpeg\"])\n classes = [\"up\",\"down\",\"left\",\"right\",\"front\",\"back\",\"clockwise\",\"anticlockwise\"]\n class_to_idx = {cls: idx for (idx, cls) in enumerate(classes)}\n st.sidebar.markdown(\"See the model preformance and play with it\")\n if file_up is not None:\n # display image that user uploaded\n image = Image.open(file_up)\n st.image(image, caption = 'Uploaded Image.', use_column_width = True)\n st.balloons()\n detect = st.button(\"开始检测 FPS\")\n if detect:\n with st.spinner(text='Preparing Image'):\n st.write(\"\")\n st.write(\"Just a second ...\")\n tact_time = fps(image,yolo)\n # st.write(str(tact_time) + ' seconds, ', str(1/tact_time),'FPS, @batch_size 1')\n st.balloons()\n\ndef heatmap(image,yolo):\n heatmap_save_path = \"heatmap_vision.png\"\n yolo.detect_heatmap(image, heatmap_save_path)\n img = Image.open(heatmap_save_path)\n transform = transforms.Compose([transforms.ToTensor()]) \n result = transform(img)\n st.image(result.permute(1,2,0).numpy(), caption = 'Processed Image.', use_column_width = True)\n\ndef detect_heatmap(yolo):\n file_up = st.file_uploader(\"Upload an image\", type = [\"jpg\",\"png\",\"jpeg\"])\n classes = [\"up\",\"down\",\"left\",\"right\",\"front\",\"back\",\"clockwise\",\"anticlockwise\"]\n class_to_idx = {cls: idx for (idx, cls) in enumerate(classes)}\n st.sidebar.markdown(\"See the model preformance and play with it\")\n if file_up is not None:\n # display image that user uploaded\n image = Image.open(file_up)\n st.image(image, caption = 'Uploaded Image.', use_column_width = True)\n st.balloons()\n detect = st.button(\"开始检测 heatmap\")\n if detect:\n with st.spinner(text='Preparing Heatmap'):\n st.write(\"\")\n st.write(\"Just a second ...\")\n heatmap(image,yolo)\n st.balloons()\n\ndef detect_example(yolo):\n st.sidebar.title(\"Choose an Image as a example\")\n images = os.listdir('./img')\n images.sort()\n image = st.sidebar.selectbox(\"Image Name\", images)\n st.sidebar.markdown(\"See the model preformance and play with it\")\n image = Image.open(os.path.join('img',image))\n st.image(image, caption = 'Choose Image.', use_column_width = True)\n st.balloons()\n detect = st.button(\"开始检测Image\")\n if detect:\n st.write(\"\")\n st.write(\"Just a second ...\")\n predict(image,yolo)\n st.balloons()\n\ndef detect_realtime(yolo):\n\n class VideoProcessor:\n def recv(self, frame):\n img = frame.to_ndarray(format=\"bgr24\")\n img = Image.fromarray(img)\n crop = False\n count = False\n r_image = yolo.detect_image(img, crop = crop, count=count)\n transform = transforms.Compose([transforms.ToTensor()]) \n result = transform(r_image)\n result = result.permute(1,2,0).numpy()\n result = (result * 255).astype(np.uint8)\n return av.VideoFrame.from_ndarray(result, format=\"bgr24\")\n \n webrtc_ctx = webrtc_streamer(\n key=\"example\",\n mode=WebRtcMode.SENDRECV,\n rtc_configuration=RTC_CONFIGURATION,\n media_stream_constraints={\"video\": True, \"audio\": False},\n async_processing=False,\n video_processor_factory=VideoProcessor\n )\n\nimport cv2\nimport time\ndef detect_video(yolo):\n file_up = st.file_uploader(\"Upload a video\", type = [\"mp4\"])\n print(file_up)\n classes = [\"up\",\"down\",\"left\",\"right\",\"front\",\"back\",\"clockwise\",\"anticlockwise\"]\n \n if file_up is not None:\n video_path = 'video.mp4'\n st.video(file_up)\n with open(video_path, 'wb') as f:\n f.write(file_up.read()) \n detect = st.button(\"开始检测 Video\")\n \n if detect: \n video_save_path = 'video2.mp4'\n # display image that user uploaded\n capture = cv2.VideoCapture(video_path)\n \n video_fps = st.slider(\"Video FPS\", 5, 30, int(capture.get(cv2.CAP_PROP_FPS)), 1)\n fourcc = cv2.VideoWriter_fourcc(*'XVID')\n size = (int(capture.get(cv2.CAP_PROP_FRAME_WIDTH)), int(capture.get(cv2.CAP_PROP_FRAME_HEIGHT)))\n out = cv2.VideoWriter(video_save_path, fourcc, video_fps, size)\n\n\n \n while(True):\n # 读取某一帧\n ref, frame = capture.read()\n if not ref:\n break\n # 转变成Image\n # frame = Image.fromarray(np.uint8(frame))\n # 格式转变，BGRtoRGB\n frame = cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)\n # 转变成Image\n frame = Image.fromarray(np.uint8(frame))\n # 进行检测\n frame = np.array(yolo.detect_image(frame))\n # RGBtoBGR满足opencv显示格式\n frame = cv2.cvtColor(frame,cv2.COLOR_RGB2BGR)\n\n # print(\"fps= %.2f\"%(fps))\n # frame = cv2.putText(frame, \"fps= %.2f\"%(fps), (0, 40), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2)\n out.write(frame)\n \n out.release()\n capture.release()\n print(\"Save processed video to the path :\" + video_save_path)\n \n with open(video_save_path, \"rb\") as file:\n btn = st.download_button(\n label=\"Download Video\",\n data=file,\n file_name=\"video.mp4\",\n )\n st.balloons()\n\nif __name__ == \"__main__\":\n main()" }, { "path": "get_map.py", "content": "import os\nimport xml.etree.ElementTree as ET\n\nfrom PIL import Image\nfrom tqdm import tqdm\nimport yaml\nfrom utils.utils import get_classes\nfrom utils.utils_map import get_coco_map, get_map\nfrom yolo import YOLO\nfrom get_yaml import get_config\nimport argparse\nif __name__ == \"__main__\":\n '''\n Recall和Precision不像AP是一个面积的概念，在门限值不同时，网络的Recall和Precision值是不同的。\n map计算结果中的Recall和Precision代表的是当预测时，门限置信度为0.5时，所对应的Recall和Precision值。\n\n 此处获得的./map_out/detection-results/里面的txt的框的数量会比直接predict多一些，这是因为这里的门限低，\n 目的是为了计算不同门限条件下的Recall和Precision值，从而实现map的计算。\n '''\n parser = argparse.ArgumentParser()\n parser.add_argument('--weights',type=str,default='model_data/yolotiny_SE_ep100.pth',help='initial weights path')\n parser.add_argument('--tiny',action='store_true',help='使用yolotiny模型')\n parser.add_argument('--phi',type=int,default=1,help='yolov4tiny注意力机制类型')\n parser.add_argument('--mode',type=int,default=0,help='get map的模式')\n parser.add_argument('--cuda',action='store_true',help='表示是否使用GPU')\n parser.add_argument('--shape',type=int,default=416,help='输入图像的shape')\n parser.add_argument('--confidence',type=float,default=0.5,help='只有得分大于置信度的预测框会被保留下来')\n parser.add_argument('--nms_iou',type=float,default=0.3,help='非极大抑制所用到的nms_iou大小')\n opt = parser.parse_args()\n print(opt)\n # 配置文件\n config = get_config()\n\n #------------------------------------------------------------------------------------------------------------------#\n # map_mode用于指定该文件运行时计算的内容\n # map_mode为0代表整个map计算流程，包括获得预测结果、获得真实框、计算VOC_map。\n # map_mode为1代表仅仅获得预测结果。\n # map_mode为2代表仅仅获得真实框。\n # map_mode为3代表仅仅计算VOC_map。\n # map_mode为4代表利用COCO工具箱计算当前数据集的0.50:0.95map。需要获得预测结果、获得真实框后并安装pycocotools才行\n #-------------------------------------------------------------------------------------------------------------------#\n map_mode = opt.mode\n #-------------------------------------------------------#\n # MINOVERLAP用于指定想要获得的mAP0.x\n # 比如计算mAP0.75，可以设定MINOVERLAP = 0.75。\n #-------------------------------------------------------#\n MINOVERLAP = 0.5\n #-------------------------------------------------------#\n # map_vis用于指定是否开启VOC_map计算的可视化\n #-------------------------------------------------------#\n map_vis = False\n #-------------------------------------------------------#\n # 指向VOC数据集所在的文件夹\n # 默认指向根目录下的VOC数据集\n #-------------------------------------------------------#\n VOCdevkit_path = 'VOCdevkit'\n #-------------------------------------------------------#\n # 结果输出的文件夹，默认为map_out\n #-------------------------------------------------------#\n map_out_path = 'map_out'\n\n image_ids = open(os.path.join(VOCdevkit_path, \"VOC2007/ImageSets/Main/val.txt\")).read().strip().split()\n\n if not os.path.exists(map_out_path):\n os.makedirs(map_out_path)\n if not os.path.exists(os.path.join(map_out_path, 'ground-truth')):\n os.makedirs(os.path.join(map_out_path, 'ground-truth'))\n if not os.path.exists(os.path.join(map_out_path, 'detection-results')):\n os.makedirs(os.path.join(map_out_path, 'detection-results'))\n if not os.path.exists(os.path.join(map_out_path, 'images-optional')):\n os.makedirs(os.path.join(map_out_path, 'images-optional'))\n\n class_names = config['classes']\n\n if map_mode == 0 or map_mode == 1:\n print(\"Load model.\")\n yolo = YOLO(opt, confidence = 0.001, nms_iou = 0.5)\n print(\"Load model done.\")\n\n print(\"Get predict result.\")\n for image_id in tqdm(image_ids):\n image_path = os.path.join(VOCdevkit_path, \"VOC2007/JPEGImages/\"+image_id+\".jpg\")\n image = Image.open(image_path)\n if map_vis:\n image.save(os.path.join(map_out_path, \"images-optional/\" + image_id + \".jpg\"))\n yolo.get_map_txt(image_id, image, class_names, map_out_path)\n print(\"Get predict result done.\")\n \n if map_mode == 0 or map_mode == 2:\n print(\"Get ground truth result.\")\n for image_id in tqdm(image_ids):\n with open(os.path.join(map_out_path, \"ground-truth/\"+image_id+\".txt\"), \"w\") as new_f:\n root = ET.parse(os.path.join(VOCdevkit_path, \"VOC2007/Annotations/\"+image_id+\".xml\")).getroot()\n for obj in root.findall('object'):\n difficult_flag = False\n if obj.find('difficult')!=None:\n difficult = obj.find('difficult').text\n if int(difficult)==1:\n difficult_flag = True\n obj_name = obj.find('name').text\n if obj_name not in class_names:\n continue\n bndbox = obj.find('bndbox')\n left = bndbox.find('xmin').text\n top = bndbox.find('ymin').text\n right = bndbox.find('xmax').text\n bottom = bndbox.find('ymax').text\n\n if difficult_flag:\n new_f.write(\"%s %s %s %s %s difficult\\n\" % (obj_name, left, top, right, bottom))\n else:\n new_f.write(\"%s %s %s %s %s\\n\" % (obj_name, left, top, right, bottom))\n print(\"Get ground truth result done.\")\n\n if map_mode == 0 or map_mode == 3:\n print(\"Get map.\")\n get_map(MINOVERLAP, True, path = map_out_path)\n print(\"Get map done.\")\n\n if map_mode == 4:\n print(\"Get map.\")\n get_coco_map(class_names = class_names, path = map_out_path)\n print(\"Get map done.\")\n" }, { "path": "get_yaml.py", "content": "import os\nimport sys\nimport yaml\n\ndef get_config():\n yaml_path = 'model_data/gesture.yaml'\n f = open(yaml_path,'r',encoding='utf-8')\n config = yaml.load(f,Loader =yaml.FullLoader)\n f.close()\n return config\n\nif __name__ == \"__main__\":\n config = get_config()\n print(config)" }, { "path": "instructions.md", "content": "# ✌ Gesture Detection\n\n\n这是一个基于无人机视觉图像手势识别控制系统，选择了YOLOv4模型进行训练\n\n **YOLOv4 = CSPDarknet53（主干） + SPP** **附加模块（颈** **） +** **PANet** **路径聚合（颈** **） + YOLOv3（头部）**\n\n![img](https://pdf.cdn.readpaper.com/parsed/fetch_target/699143cdb334ecfc63caf8192472490c_0_Figure_1.png)\n\n" }, { "path": "kmeans_for_anchors.py", "content": "#-------------------------------------------------------------------------------------------------------#\n# kmeans虽然会对数据集中的框进行聚类，但是很多数据集由于框的大小相近，聚类出来的9个框相差不大，\n# 这样的框反而不利于模型的训练。因为不同的特征层适合不同大小的先验框，shape越小的特征层适合越大的先验框\n# 原始网络的先验框已经按大中小比例分配好了，不进行聚类也会有非常好的效果。\n#-------------------------------------------------------------------------------------------------------#\nimport glob\nimport xml.etree.ElementTree as ET\n\nimport matplotlib.pyplot as plt\nimport numpy as np\nfrom tqdm import tqdm\n\n\ndef cas_iou(box, cluster):\n x = np.minimum(cluster[:, 0], box[0])\n y = np.minimum(cluster[:, 1], box[1])\n\n intersection = x * y\n area1 = box[0] * box[1]\n\n area2 = cluster[:,0] * cluster[:,1]\n iou = intersection / (area1 + area2 - intersection)\n\n return iou\n\ndef avg_iou(box, cluster):\n return np.mean([np.max(cas_iou(box[i], cluster)) for i in range(box.shape[0])])\n\ndef kmeans(box, k):\n #-------------------------------------------------------------#\n # 取出一共有多少框\n #-------------------------------------------------------------#\n row = box.shape[0]\n \n #-------------------------------------------------------------#\n # 每个框各个点的位置\n #-------------------------------------------------------------#\n distance = np.empty((row, k))\n \n #-------------------------------------------------------------#\n # 最后的聚类位置\n #-------------------------------------------------------------#\n last_clu = np.zeros((row, ))\n\n np.random.seed()\n\n #-------------------------------------------------------------#\n # 随机选5个当聚类中心\n #-------------------------------------------------------------#\n cluster = box[np.random.choice(row, k, replace = False)]\n\n iter = 0\n while True:\n #-------------------------------------------------------------#\n # 计算当前框和先验框的宽高比例\n #-------------------------------------------------------------#\n for i in range(row):\n distance[i] = 1 - cas_iou(box[i], cluster)\n \n #-------------------------------------------------------------#\n # 取出最小点\n #-------------------------------------------------------------#\n near = np.argmin(distance, axis=1)\n\n if (last_clu == near).all():\n break\n \n #-------------------------------------------------------------#\n # 求每一个类的中位点\n #-------------------------------------------------------------#\n for j in range(k):\n cluster[j] = np.median(\n box[near == j],axis=0)\n\n last_clu = near\n if iter % 5 == 0:\n print('iter: {:d}. avg_iou:{:.2f}'.format(iter, avg_iou(box, cluster)))\n iter += 1\n\n return cluster, near\n\ndef load_data(path):\n data = []\n #-------------------------------------------------------------#\n # 对于每一个xml都寻找box\n #-------------------------------------------------------------#\n for xml_file in tqdm(glob.glob('{}/*xml'.format(path))):\n tree = ET.parse(xml_file)\n height = int(tree.findtext('./size/height'))\n width = int(tree.findtext('./size/width'))\n if height<=0 or width<=0:\n continue\n \n #-------------------------------------------------------------#\n # 对于每一个目标都获得它的宽高\n #-------------------------------------------------------------#\n for obj in tree.iter('object'):\n xmin = int(float(obj.findtext('bndbox/xmin'))) / width\n ymin = int(float(obj.findtext('bndbox/ymin'))) / height\n xmax = int(float(obj.findtext('bndbox/xmax'))) / width\n ymax = int(float(obj.findtext('bndbox/ymax'))) / height\n\n xmin = np.float64(xmin)\n ymin = np.float64(ymin)\n xmax = np.float64(xmax)\n ymax = np.float64(ymax)\n # 得到宽高\n data.append([xmax - xmin, ymax - ymin])\n return np.array(data)\n\nif __name__ == '__main__':\n np.random.seed(0)\n #-------------------------------------------------------------#\n # 运行该程序会计算'./VOCdevkit/VOC2007/Annotations'的xml\n # 会生成yolo_anchors.txt\n #-------------------------------------------------------------#\n input_shape = [224, 224]\n anchors_num = 9\n #-------------------------------------------------------------#\n # 载入数据集，可以使用VOC的xml\n #-------------------------------------------------------------#\n path = 'VOCdevkit/VOC2007/Annotations'\n \n #-------------------------------------------------------------#\n # 载入所有的xml\n # 存储格式为转化为比例后的width,height\n #-------------------------------------------------------------#\n print('Load xmls.')\n data = load_data(path)\n print('Load xmls done.')\n \n #-------------------------------------------------------------#\n # 使用k聚类算法\n #-------------------------------------------------------------#\n print('K-means boxes.')\n cluster, near = kmeans(data, anchors_num)\n print('K-means boxes done.')\n data = data * np.array([input_shape[1], input_shape[0]])\n cluster = cluster * np.array([input_shape[1], input_shape[0]])\n\n #-------------------------------------------------------------#\n # 绘图\n #-------------------------------------------------------------#\n for j in range(anchors_num):\n plt.scatter(data[near == j][:,0], data[near == j][:,1])\n plt.scatter(cluster[j][0], cluster[j][1], marker='x', c='black')\n plt.savefig(\"kmeans_for_anchors.jpg\")\n plt.show()\n print('Save kmeans_for_anchors.jpg in root dir.')\n\n cluster = cluster[np.argsort(cluster[:, 0] * cluster[:, 1])]\n print('avg_ratio:{:.2f}'.format(avg_iou(data, cluster)))\n print(cluster)\n\n f = open(\"yolo_anchors.txt\", 'w')\n row = np.shape(cluster)[0]\n for i in range(row):\n if i == 0:\n x_y = \"%d,%d\" % (cluster[i][0], cluster[i][1])\n else:\n x_y = \", %d,%d\" % (cluster[i][0], cluster[i][1])\n f.write(x_y)\n f.close()\n" }, { "path": "logs/README.md", "content": "用于存放训练好的文件" }, { "path": "logs/gesture_loss_2021_11_14_22_04_00/epoch_loss_2021_11_14_22_04_00.txt", "content": "390.34399642473386\n21.87092101721116\n14.030741856421953\n11.276778338867942\n9.814540598127577\n8.89100271978496\n8.609104168267898\n7.924442773983802\n7.723959027984996\n7.2670195367601185\n7.255199196897907\n6.893556188654016\n6.661026071619104\n6.5294443972316785\n6.535371827490536\n6.529178083678822\n6.403998654565694\n6.439444012112087\n6.092924733220795\n5.926193254965323\n5.9576384785734575\n5.8119951972255\n5.6878520206168846\n5.819804650765878\n5.707105348139633\n5.458082881974585\n5.665041320117903\n5.317585485952872\n5.349038653903538\n5.283199619363855\n5.064980445084749\n5.070186079284291\n4.9681971073150635\n4.793072164794545\n4.973145805759194\n4.918124354915855\n4.663256362632469\n4.837633197690233\n4.743683688434554\n4.616998254516979\n4.524823586146037\n4.4209345593864535\n4.558955289699413\n4.333138801433422\n4.426347941528132\n4.412103137852233\n4.295655697952082\n4.364107617625484\n4.211893027211413\n4.084111590444306\n4.018801480163763\n3.8647101366961443\n3.734398615581018\n3.6937082122873375\n3.793811333032302\n3.429193678093545\n3.6194330038111886\n3.4087822738988898\n3.331124112193967\n3.3782305434162234\n3.3561593158009613\n3.25705443598606\n3.2106575075490973\n3.0107549484129303\n3.0536143231539077\n2.9674469438599953\n3.1300665189822516\n2.909559675204901\n2.9446099194479576\n2.8209132660686236\n2.8917798992292383\n2.815192371238897\n2.861111617566627\n2.9016677490722986\n2.8193857658792427\n2.8216423440126723\n2.777715330874478\n2.725730179820532\n2.589312877184079\n2.670389473438263\n2.626439411331106\n2.57100960759469\n2.6649326178026786\n2.449705180930503\n2.6089335954115715\n2.666015229291386\n2.5139025822281837\n2.4510488511971484\n2.60918134248551\n2.615589211384455\n2.4221341083815067\n2.5034887735490448\n2.3411180855315408\n2.3742799654970934\n2.4252039420383946\n2.5134657593788923\n2.5887757239886273\n2.5031773506859203\n2.3927585335425388\n2.4924555529414874\n2.3816184005987497\n2.3525361067351\n2.35756847280779\n2.4606370890030154\n2.262793848084079\n2.283497501026701\n2.2522216586419095\n2.3806339068177307\n2.345363718767961\n2.305632569723659\n2.1932848855669116\n2.332635486199532\n2.2705356725204138\n2.233249652992796\n2.4728508678115446\n2.3142452859952125\n2.3585592800820314\n2.335805359078042\n2.337391757118849\n2.391327069129473\n2.3404054016242792\n2.3145943543425314\n2.196398460570677\n2.2358641638248056\n2.3038836038774915\n2.2790947368851415\n2.2812541202630525\n2.2533860233278924\n2.3108025224488458\n2.2092323683110284\n2.308551702050515\n2.2422945557369127\n2.1741022714126257\n2.44105933992951\n2.3797168718811905\n2.231722431326354\n2.3973163276174922\n2.1568032256615015\n2.239097781571341\n2.2258979082107544\n2.1682290563612807\n2.2031694714117935\n2.2706658139272973\n2.329095835854978\n2.255610410262037\n2.2977319957665454\n2.3046101513836117\n2.249893919369321\n2.2964354607242123\n2.315463280696192\n" }, { "path": "logs/gesture_loss_2021_11_14_22_04_00/epoch_val_loss_2021_11_14_22_04_00.txt", "content": "28.558996200561523\n15.032766554090712\n11.545120133293999\n9.72215329276191\n8.58862935172187\n8.486469162835014\n7.804132832421197\n7.238262918260363\n6.890773402320014\n6.530833350287543\n6.475247330135769\n6.4751937124464245\n6.239521026611328\n6.0489738782246905\n6.12673372692532\n5.641317420535618\n6.040707217322455\n5.724527147081163\n5.265863656997681\n5.316834555731879\n5.4665877024332685\n5.622564209832086\n5.04600026872423\n5.060362259546916\n5.527375910017225\n5.435662375556098\n5.021538707945082\n5.028834872775608\n4.896508720186022\n4.989696582158406\n5.161070320341322\n5.098267449273004\n4.707995070351495\n4.600137048297459\n4.426739745669895\n4.481476042005751\n4.555791060129802\n4.693203316794501\n4.515556865268284\n4.371145274904039\n4.138098372353448\n4.548380348417494\n4.3106510109371605\n4.320602138837178\n4.131023804346721\n4.0555612511105\n4.217087030410767\n4.128190358479817\n4.032541698879665\n3.99964001443651\n3.741890834437476\n3.749820719162623\n3.6366468982564077\n3.5657983157369824\n3.9311270780033536\n3.6530382368299694\n4.012030104796092\n3.8975768751568265\n3.764561494191488\n3.4476174149248333\n3.535598119099935\n3.998010264502631\n3.88807831870185\n3.810675323009491\n3.8832875225279064\n3.532531124022272\n3.9232571257485285\n3.58525949716568\n3.7238865759637623\n3.7168162133958607\n3.503431843386756\n3.5310314959949918\n3.7993387116326227\n3.5516341394848294\n3.6795931648876934\n3.564246873060862\n3.484692699379391\n3.7236365245448217\n3.7466657956441245\n3.66163033246994\n3.751209259033203\n3.6696145402060614\n3.5883768465783863\n3.853155712286631\n3.4928252498308816\n3.602889382176929\n3.7287648055288525\n3.6207654832137957\n3.610999337500996\n3.8127831634547977\n3.6820534533924527\n3.716387847231494\n3.6561857561270394\n3.703249845239851\n3.686804783013132\n3.687538597318861\n3.8072550859716205\n3.6593143989642463\n3.707283900843726\n3.7246316257450314\n3.8617856800556183\n3.573318580786387\n3.531035871969329\n3.6177483134799533\n3.6122085054715476\n3.5437003208531275\n3.5555910716454187\n3.6909723381201425\n3.5987775524457297\n3.646808198756642\n3.6476809779802957\n3.615621048543188\n3.8375576469633312\n3.7161912678016558\n3.694040416015519\n3.677286409669452\n3.6777902278635235\n3.7830483151806726\n3.707444575097826\n3.7904206779268055\n3.5872142712275186\n3.6864367392328052\n3.7757607218292026\n3.835707320107354\n3.6799587168627315\n3.8233094347847834\n3.6921923756599426\n3.7244893974728055\n3.6797771288288965\n3.711515542533663\n3.8481360466943846\n3.8577410876750946\n3.710074722766876\n3.8249045742882624\n3.7864705423514047\n3.6575047771135965\n3.8352384832170276\n3.7801570263173847\n3.7013448344336615\n3.6655967930952706\n3.657223959763845\n3.722360614273283\n3.772919843594233\n3.7007322708765664\n3.7042017413510218\n3.8934470083978443\n3.8964318566852145\n3.6877589921156564\n3.713595751259062\n3.597744878795412\n" }, { "path": "logs/loss_2022_04_27_08_48_16/epoch_loss.txt", "content": "4.311199968511408\n2.641528855670582\n1.0470811074430293\n0.3173784383318641\n0.1660231321372769\n0.12659757448868317\n0.11646865105087106\n0.1186594499105757\n0.11129742149602283\n0.09524408660151741\n0.09781679036942395\n0.09211275726556778\n0.08542741784317927\n0.08707698925652287\n0.08003932000561194\n0.09124952453103932\n0.07743281058289787\n0.07542280463332479\n0.062316759235479614\n0.07161653380502354\n0.06821535866368901\n0.07083209519359199\n0.07460641437633471\n0.07450477220118046\n0.06487809985198757\n0.050884095443920654\n0.07091375355693427\n0.06433163752610033\n0.0656029749661684\n0.05935167453505776\n0.06459851512177424\n0.06376675008372827\n0.05718133259903301\n0.05716039274226536\n0.05911739483814348\n0.05761603875593706\n0.051265862939709965\n0.047803171148354355\n0.0480937244031917\n0.05439905263483524\n0.058482232080264526\n0.05515999550169164\n0.049258994361893696\n0.050817748277702114\n0.05204320927573876\n0.04787483066320419\n0.050909879194064575\n0.04848571375689723\n0.050943345593457874\n0.04928677469830622\n0.05230807525416215\n0.054047910206847724\n0.04724785503413942\n0.04339685816731718\n0.04393725813262993\n0.04542147194345792\n0.046219487115740775\n0.04159199959701962\n0.0356766721026765\n0.0347878428383006\n0.0335447210404608\n0.03512532735864322\n0.032664823532104495\n0.035281008275018795\n0.027731664727131525\n0.03222298233045472\n0.03146794889536169\n0.02836602210170693\n0.028307923198574118\n0.027572717414134078\n0.026898101448184913\n0.029324432876374987\n0.02880634083929989\n0.024556251760158274\n0.027897736864785354\n0.024288477210534943\n0.022848848750193915\n0.023355903372996385\n0.02707639779481623\n0.022250585506359735\n0.025191593791047732\n0.022139282586673897\n0.02378465121404992\n0.02341305265824\n0.02176100810368856\n0.025529090170231132\n0.023221762292087077\n0.02107305938584937\n0.019723483237127463\n0.027768902087377176\n0.023790666233334277\n0.02183559000906017\n0.019348353561427858\n0.021541342077155908\n0.020851219362682766\n0.01955224501176013\n0.02228688634932041\n0.018856989074912338\n0.01816959279692835\n0.024754421909650166\n" }, { "path": "logs/loss_2022_04_27_08_48_16/epoch_val_loss.txt", "content": "3.5736865997314453\n1.7812694907188416\n0.5147329270839691\n0.15201690793037415\n0.10024188458919525\n0.08380990475416183\n0.07576803863048553\n0.06853799521923065\n0.06467496231198311\n0.060902709141373634\n0.05481202341616154\n0.05164487101137638\n0.046625690534710884\n0.046081338077783585\n0.04508414678275585\n0.046726442873477936\n0.041066285222768784\n0.039722129702568054\n0.0392248947173357\n0.04033488966524601\n0.03738676756620407\n0.0356711745262146\n0.03774934820830822\n0.035463595762848854\n0.03278419189155102\n0.03250573016703129\n0.03182028792798519\n0.031694755889475346\n0.03182463627308607\n0.028715165331959724\n0.03064714837819338\n0.028574727475643158\n0.031066023744642735\n0.028762156143784523\n0.027465523220598698\n0.02787941414862871\n0.02755015157163143\n0.02802269347012043\n0.028581750579178333\n0.026334763504564762\n0.026825452223420143\n0.02670316770672798\n0.02603335492312908\n0.025488858111202717\n0.027477828785777092\n0.02550355065613985\n0.026508965529501438\n0.02424653246998787\n0.02420251350849867\n0.024741491302847862\n0.03815543949604035\n0.024845311418175697\n0.024306144565343857\n0.02493119016289711\n0.024438758194446564\n0.021836227178573607\n0.022118838876485823\n0.02276018038392067\n0.019801595807075502\n0.018804560229182244\n0.01913141254335642\n0.018066196143627165\n0.018252668902277946\n0.017480477318167688\n0.016695075295865537\n0.018235534615814685\n0.016669700480997564\n0.01745656579732895\n0.01661595106124878\n0.014982381090521812\n0.014259136654436589\n0.01617119237780571\n0.01583776492625475\n0.015838896110653877\n0.015466723032295704\n0.014705226197838784\n0.014486565068364144\n0.0142423365265131\n0.013639062829315662\n0.013229098543524742\n0.013664134219288826\n0.014067459665238858\n0.014119291864335536\n0.014162952080368996\n0.014096969552338124\n0.014010479114949704\n0.013855390436947345\n0.01369147039949894\n0.013611100800335407\n0.013387569226324558\n0.013233654387295245\n0.013060701824724675\n0.01311743687838316\n0.013459368608891964\n0.013417618162930012\n0.013188641518354416\n0.013131854124367237\n0.013138605654239655\n0.013040048442780972\n0.013191545940935611\n" }, { "path": "logs/loss_2022_04_27_10_38_48/epoch_loss.txt", "content": "4.417048931121826\n2.7174118811433967\n1.0889532132582231\n0.3425311154939912\n0.17422378638928587\n0.13641497018662366\n0.11632075736468489\n0.11424875665794719\n0.10951222343878313\n0.11042191968722777\n0.0965666960586201\n0.09156128205358982\n0.09250037236647173\n0.09282402846623551\n0.08625757846642625\n0.07673129354688255\n0.07389622215520252\n0.07624811069531874\n0.08134209279986945\n0.08268712799657475\n0.06569299051030116\n0.06593379310586235\n0.07313475605439056\n0.06932794980027458\n0.07105197571218014\n0.05761696923185478\n0.05699523843147538\n0.05502087775279175\n0.056425975635647774\n0.060862130570140754\n0.05275308594784953\n0.05468131161548875\n0.06639060936868191\n0.0586402067406611\n0.05531726946884936\n0.05826686415821314\n0.05614634239199487\n0.060194396329197014\n0.056169633330269295\n0.05521787144243717\n0.05759791826659983\n0.06400778830390084\n0.048669698648154736\n0.05138815820894458\n0.05391152406280691\n0.048903680660507896\n0.05098097136413509\n0.046242827380245384\n0.05179907051338391\n0.0525860372422771\n0.05424936364094416\n0.049993348659740554\n0.04597619854741626\n0.04917745155592759\n0.05255601741373539\n0.04698830768465996\n0.041387100517749784\n0.04129959721532133\n0.04556649559073978\n0.036499715513653226\n0.03981801929573218\n0.04143420826229784\n0.03435336612164974\n0.03496221779949135\n0.03109016865491867\n0.03035914318429099\n0.029583082410196464\n0.03257722655932108\n0.030363482443822754\n0.027382713970210817\n0.03354052487346861\n0.02999182954016659\n0.027540474219454658\n0.03399232141673565\n0.027007617097761897\n0.025914737520118556\n0.0295799125606815\n0.02715012611200412\n0.025495433765980933\n0.0296443536463711\n0.023164296481344434\n0.025637096497747633\n0.024675296164221233\n0.02778547273741828\n0.021970178662902778\n0.023107113461527558\n0.024780070698923535\n0.022441018600430754\n0.023930547055270937\n0.0282184108470877\n0.023034340888261794\n0.024948879559006956\n0.021047428602145778\n0.019247366736332577\n0.019984866658018696\n0.02513700392511156\n0.02460642974409792\n0.0241888129669759\n0.024461371141175428\n0.023433364638023906\n" }, { "path": "logs/loss_2022_04_27_10_38_48/epoch_val_loss.txt", "content": "3.682404637336731\n1.8932517766952515\n0.5478550791740417\n0.1596439927816391\n0.1100359559059143\n0.0877840518951416\n0.07812783867120743\n0.07114855200052261\n0.06861080229282379\n0.059281766414642334\n0.057694293558597565\n0.051728978753089905\n0.052549805492162704\n0.04606110043823719\n0.04738330654799938\n0.04431380145251751\n0.04233948327600956\n0.04040302708745003\n0.038821205496788025\n0.0383895430713892\n0.03584542125463486\n0.03636615164577961\n0.03440128639340401\n0.031500913202762604\n0.03160226531326771\n0.03259335644543171\n0.03182834479957819\n0.03255347441881895\n0.03205320052802563\n0.03115831222385168\n0.030962957069277763\n0.03099967911839485\n0.028362704440951347\n0.029792566783726215\n0.029385950416326523\n0.028081808239221573\n0.02900168113410473\n0.028213596902787685\n0.026003092527389526\n0.029015707783401012\n0.027079648338258266\n0.02746042888611555\n0.026224803179502487\n0.02623423095792532\n0.026428623124957085\n0.025775899179279804\n0.025982394814491272\n0.02434847690165043\n0.027825096622109413\n0.026163294911384583\n0.029283170774579047\n0.025315795838832856\n0.027043038606643678\n0.028298694640398026\n0.024901207908987998\n0.021958087757229804\n0.02251458093523979\n0.022333519905805586\n0.021478286758065224\n0.021176514402031898\n0.018941503018140793\n0.019572099670767784\n0.018108497187495232\n0.018086655251681804\n0.017889507673680784\n0.01727491766214371\n0.01810304317623377\n0.020134907588362692\n0.018655003793537617\n0.018117578141391276\n0.017840097844600677\n0.01779591590166092\n0.016621771082282067\n0.017149972915649413\n0.016952383518218993\n0.015586855821311474\n0.01567951999604702\n0.0161365307867527\n0.01567267570644617\n0.01678410042077303\n0.015898118540644646\n0.01655469797551632\n0.015443072095513344\n0.015269587188959122\n0.015318373404443263\n0.015480193309485912\n0.015252745896577834\n0.015485197678208351\n0.01524040475487709\n0.015235877968370915\n0.015190575830638408\n0.01506870575249195\n0.015268886275589467\n0.015318392775952816\n0.015248116478323937\n0.01509730275720358\n0.015357919968664646\n0.015471475012600423\n0.015338210947811603\n0.015286244638264179\n" }, { "path": "logs/loss_2022_04_27_12_50_47/epoch_loss.txt", "content": "4.458093025467613\n2.7262558070096103\n1.0888537033037706\n0.3306311368942261\n0.1712129498747262\n0.12332972951910713\n0.1077601161192764\n0.10889687660065564\n0.10751076347448608\n0.09971555254676125\n0.09748913144523447\n0.09051749330352653\n0.08674890751188452\n0.09196238592267036\n0.0813636336136948\n0.08286366950381886\n0.07791051878170534\n0.0753517130559141\n0.07469043592837724\n0.07069844498553059\n0.06863954527811571\n0.05802192301912741\n0.07001199353147637\n0.0646351370960474\n0.0635682385076176\n0.06396392174065113\n0.062142887406728485\n0.0702532638203014\n0.056375787339427254\n0.06388939967886968\n0.05778990279544483\n0.06408696647056124\n0.06048921140080148\n0.046278277158059856\n0.05944571127607064\n0.05725045552985235\n0.05380251800472086\n0.053617957894775\n0.053481346842917526\n0.05578712136908011\n0.05615681384436109\n0.0525641811334274\n0.04595534486526793\n0.04221054826947776\n0.0491331076588143\n0.04645225058563731\n0.047417608005079354\n0.045993872325528755\n0.04980102206834338\n0.05388529971241951\n0.04780796766281128\n0.051682502610815896\n0.05296175873114003\n0.04763079182141357\n0.03715274184942245\n0.038538362830877304\n0.03803896543880304\n0.04017537732919057\n0.03992160202728377\n0.03339115016990238\n0.03391021318319771\n0.03317808165318436\n0.033503353450861244\n0.034213335605131255\n0.037453227241834\n0.033429956477549344\n0.032547304261889724\n0.03456145400802294\n0.026851379209094577\n0.029029812270568476\n0.02536299385958248\n0.02381322646720542\n0.02601998903685146\n0.020065840913189782\n0.02312256395816803\n0.028637176213992966\n0.023025286176966295\n0.023644178753925695\n0.024718130793836383\n0.02247788065837489\n0.023494062303668923\n0.025069689253966014\n0.02251974062787162\n0.024839345862468085\n0.021578845319648585\n0.022635220984617867\n0.022249876335263253\n0.01972206729567713\n0.018786311563518312\n0.02083740762124459\n0.02136736027896404\n0.019557259066237342\n0.018951669645806152\n0.020326226308114\n0.021592341653174824\n0.019481366727915075\n0.018176950762669244\n0.02213383706079589\n0.019981356461842854\n0.020978835970163347\n" }, { "path": "logs/loss_2022_04_27_12_50_47/epoch_val_loss.txt", "content": "3.7051011323928833\n1.8262890577316284\n0.5144035518169403\n0.16302762925624847\n0.10760901868343353\n0.09057768434286118\n0.07540924847126007\n0.07146378979086876\n0.06520375981926918\n0.05898746848106384\n0.054325105622410774\n0.05058479495346546\n0.0504811592400074\n0.046029604971408844\n0.04258855804800987\n0.042371716350317\n0.040247365832328796\n0.04038912057876587\n0.03568720445036888\n0.038001520559191704\n0.03973718546330929\n0.035464052110910416\n0.03202499449253082\n0.02998754195868969\n0.032502518966794014\n0.03302299045026302\n0.03285937011241913\n0.029083450324833393\n0.029631994664669037\n0.03396240994334221\n0.029673300683498383\n0.028280221857130527\n0.027639511972665787\n0.028393579646945\n0.027291471138596535\n0.026989608071744442\n0.02653918694704771\n0.027808908373117447\n0.027841621078550816\n0.02570505067706108\n0.025745649822056293\n0.026372630149126053\n0.024600804783403873\n0.026447951793670654\n0.02569119818508625\n0.026840184815227985\n0.024051610380411148\n0.02362955827265978\n0.024365886114537716\n0.024577765725553036\n0.031041909381747244\n0.02641780823469162\n0.02472583018243313\n0.02326701581478119\n0.019615407288074493\n0.021174174174666403\n0.019675580970942973\n0.01869105324149132\n0.018909885734319686\n0.019662134535610675\n0.01899590715765953\n0.016179793514311314\n0.01545619908720255\n0.015423668920993805\n0.018800214119255542\n0.0158102760091424\n0.0158376544713974\n0.01783675402402878\n0.015972125343978405\n0.01454415861517191\n0.014743064902722836\n0.013825051300227643\n0.01407058835029602\n0.013598379865288734\n0.013919505663216114\n0.013623752258718013\n0.014403878897428512\n0.014411385357379913\n0.01337964329868555\n0.013076365552842617\n0.013368507660925389\n0.013667609356343747\n0.013365321420133114\n0.013264597952365875\n0.013465055078268052\n0.01281917616724968\n0.01263135802000761\n0.012750985845923424\n0.01290153805166483\n0.01281326413154602\n0.012850469164550304\n0.012885735556483268\n0.013168741390109063\n0.013198709674179554\n0.0126633545383811\n0.012886124104261399\n0.012797533720731735\n0.012569484673440457\n0.012130422703921794\n0.012647346407175065\n" }, { "path": "logs/loss_2022_04_28_00_40_54/epoch_loss.txt", "content": "4.65520715713501\n3.142860672690652\n1.5020794109864668\n0.5057930661873384\n0.231415910476988\n0.1739024357362227\n0.1501499665054408\n0.13435510004108603\n0.12552000412886793\n0.1170116358182647\n0.1097346202216365\n0.10218094119971449\n0.09653170305219563\n0.09267877211624925\n0.08959556709636342\n0.08778026801618663\n0.0813840397379615\n0.08208547498692166\n0.07795694809068333\n0.0774568762968887\n0.07742892002517526\n0.07316952097144994\n0.0717044398188591\n0.07023497687822039\n0.07019331865012646\n0.06709351390600204\n0.06731417910619215\n0.06743009134449741\n0.06635952317579226\n0.06368578191507947\n0.06163112514398315\n0.06230247410183603\n0.0609466726468368\n0.059141877869313415\n0.059421493925831535\n0.05991599742661823\n0.05664417435499755\n0.05543165823275393\n0.055084149945865975\n0.05501931634816257\n0.05503683621910485\n0.05480257303199985\n0.05537006275897676\n0.05448474125428633\n0.05232419649308378\n0.05311859653077342\n0.05284474231302738\n0.051879515532742844\n0.052160846746780655\n0.048417276787486946\n0.07137971396247546\n0.06579171708888477\n0.06337685022089216\n0.058213022185696496\n0.06011202625102467\n0.05577432778146532\n0.05307989873819881\n0.05232232163349788\n0.047045067904724014\n0.045659234002232554\n0.046541030332446096\n0.041184055474069385\n0.04066362182299296\n0.041569982427689764\n0.03817177605297831\n0.0390163982907931\n0.041840214654803275\n0.038884344117509\n0.03724856765733825\n0.03528667270309395\n0.03439781483676699\n0.03381528837813271\n0.03448933532668485\n0.03202489465475082\n0.03492107921176486\n0.029904662817716598\n0.03170571397576067\n0.03179397972093688\n0.0303279221471813\n0.029197406230701342\n0.02931012755466832\n0.029168612303005326\n0.027595289217101204\n0.02744665356973807\n0.026995969439546266\n0.027659311725033654\n0.02661879969139894\n0.027540806722309855\n0.025905532100134427\n0.0255900744555725\n0.026152818650007247\n0.025521984696388243\n0.025769058614969254\n0.02644038177612755\n0.02754443759719531\n0.024427745077345107\n0.025285613785187403\n0.026757355800105465\n0.02632749622894658\n0.026431108307507303\n" }, { "path": "logs/loss_2022_04_28_00_40_54/epoch_val_loss.txt", "content": "3.979103207588196\n2.2379150390625\n0.7213477790355682\n0.20374882966279984\n0.13149111717939377\n0.10669583082199097\n0.08946957811713219\n0.07844944670796394\n0.07209542766213417\n0.06465885788202286\n0.060964012518525124\n0.05698745884001255\n0.053726550191640854\n0.053231727331876755\n0.05091492086648941\n0.04869535565376282\n0.045929690822958946\n0.043502215296030045\n0.04109686613082886\n0.042073581367731094\n0.03760443814098835\n0.036989014595746994\n0.0369559321552515\n0.03501574695110321\n0.03553796745836735\n0.03463827446103096\n0.03613190911710262\n0.03488997742533684\n0.03165611159056425\n0.03400527499616146\n0.03399870544672012\n0.03354485519230366\n0.030975072644650936\n0.0297493115067482\n0.029600737616419792\n0.02729297336190939\n0.027453931979835033\n0.028598678298294544\n0.027731974609196186\n0.030310326255857944\n0.026450641453266144\n0.027599090710282326\n0.027010041289031506\n0.026624951511621475\n0.027538660913705826\n0.026772234588861465\n0.026853609830141068\n0.027332110330462456\n0.026638195849955082\n0.026076992973685265\n0.029674236476421357\n0.03184238411486149\n0.02579696960747242\n0.026541008800268173\n0.028798045963048934\n0.02365291155874729\n0.024432314187288286\n0.024038903787732123\n0.022221024334430694\n0.022891897335648538\n0.01906990371644497\n0.021012770757079125\n0.020605479553341865\n0.020398029685020448\n0.019171418249607088\n0.01934974603354931\n0.020316287130117416\n0.019410957768559455\n0.018952558375895025\n0.017280998453497887\n0.0177790354937315\n0.018064785189926623\n0.01828454677015543\n0.01720294840633869\n0.01639395747333765\n0.016722467541694642\n0.016642549820244313\n0.01656894329935312\n0.015701821073889732\n0.015975065901875495\n0.016035530529916287\n0.015547602623701095\n0.01571439057588577\n0.01621132455766201\n0.015737788379192354\n0.01545789260417223\n0.015475354716181755\n0.015286277420818806\n0.015320570766925811\n0.015739747881889345\n0.015467294491827488\n0.015462711267173291\n0.015299991890788078\n0.014891423098742963\n0.014959413185715675\n0.015149685740470886\n0.015103902481496335\n0.014999320358037948\n0.015079839341342448\n0.0150094548240304\n" }, { "path": "logs/loss_2022_04_28_14_54_17/epoch_loss.txt", "content": "3.3427013629012636\n0.590641807185279\n0.20623346173928844\n0.13935681179993684\n0.11779505432479911\n0.10669546342558331\n0.0995730339239041\n0.09289641034685903\n0.08960233483877447\n0.08865145291719172\n0.08199652650703987\n0.08332964736554357\n0.08082385785463783\n0.07951261059691508\n0.07187494143015809\n0.07693152552884486\n0.07002928235257665\n0.06805908863122265\n0.06391975372615788\n0.06560571603477001\n0.06688064705166552\n0.062423851289269\n0.06189305805083778\n0.06095021272905999\n0.05913820943484704\n0.05766822151425812\n0.05601171863575776\n0.050846687311099634\n0.0500038359210723\n0.05070198744845887\n0.04995435054620935\n0.04775367355905473\n0.04747431728368004\n0.05075365285803046\n0.049145943324805964\n0.04660840377522012\n0.04236363642849028\n0.04308449916231136\n0.04134128590942257\n0.04134896128024492\n0.040451034003247816\n0.040809157501078316\n0.04189636699027485\n0.03930564734877812\n0.04004836426013046\n0.03825837828529378\n0.03547370240299238\n0.03609677294476165\n0.035196643017439376\n0.03430712522628407\n0.04613391875237641\n0.05915206435084757\n0.045893035898916426\n0.04116026466298434\n0.0429476417420018\n0.03999344222247601\n0.034763063090698175\n0.03578517514720766\n0.03375119598996308\n0.03283411696967151\n0.03579546554893669\n0.03182236654813298\n0.03289871994768166\n0.03093694845964718\n0.028104687105709066\n0.0279214970392382\n0.02814181201522135\n0.026209147684534806\n0.024499411086758807\n0.02420818345660033\n0.02401729004470528\n0.02229024926847261\n0.021894857381832684\n0.021454263018677013\n0.020758730825683518\n0.02169692176976241\n0.019593946940343207\n0.019191343562367062\n0.0194984604876178\n0.02022809916266447\n0.017767922341590747\n0.01808840037944416\n0.018055611812613077\n0.017147960676164885\n0.015863009145121194\n0.015711418758534514\n0.016356725540633003\n0.016216116898512052\n0.015499612758867442\n0.015379458964647104\n0.016735805649982973\n0.014799573211025239\n0.015743958410651734\n0.014708074144113601\n0.014328512709148021\n0.015710317682371373\n0.01542505334622951\n0.014101080921439765\n0.014700241691510503\n0.014981216627832812\n" }, { "path": "logs/loss_2022_04_28_14_54_17/epoch_val_loss.txt", "content": "1.1948505997657777\n0.2769960485398769\n0.1309874437749386\n0.10720247365534305\n0.0823921812698245\n0.06992402952164412\n0.0779087346047163\n0.06684023551642895\n0.06127838855609298\n0.06253754440695047\n0.06560290511697531\n0.05028826054185629\n0.05307867294177413\n0.046788199059665206\n0.05016098273918033\n0.041087670251727104\n0.049103803001344204\n0.04360529286786914\n0.04554138630628586\n0.03290841649286449\n0.04053358295932412\n0.038861811719834806\n0.040706123877316716\n0.03609397481195629\n0.03557254578918219\n0.03464236315339804\n0.03329266821965575\n0.03151600556448102\n0.030487440805882216\n0.03179679936729372\n0.030378894181922078\n0.03546885224059224\n0.028008161624893547\n0.030146837001666427\n0.028426590701565148\n0.030748564330860973\n0.028618200030177832\n0.03007163112051785\n0.02537959101609886\n0.028373095905408263\n0.025091598788276315\n0.027431158255785702\n0.0274854336399585\n0.0238998107612133\n0.024188394332304596\n0.025603410461917518\n0.022463220916688443\n0.021122918161563576\n0.023449525656178593\n0.02241856213659048\n0.030004368303343652\n0.03465683250688016\n0.025661695492453875\n0.025751420808956028\n0.0250759432092309\n0.024298161384649575\n0.023818821809254587\n0.02544179279357195\n0.02248522681184113\n0.02272053265478462\n0.021450468467082828\n0.022059163730591535\n0.01965688676573336\n0.019216149824205785\n0.020135902601759882\n0.02419198288116604\n0.017368705407716335\n0.01844585470389575\n0.015960348234511913\n0.017440078582149\n0.015858469036174938\n0.01589310457929969\n0.01708033775212243\n0.030576034029945732\n0.014990652166306972\n0.020580469502601773\n0.01814356680260971\n0.016363495017867536\n0.016028978914255275\n0.015470803889911622\n0.017227034358074888\n0.016705141763668507\n0.01754759649047628\n0.02099468276137486\n0.02627454571193084\n0.016601535107474773\n0.019520913722226398\n0.016074266715440898\n0.015431905922014266\n0.015508590545505286\n0.013960553548531606\n0.015237966080894694\n0.015095379657577724\n0.01584624971728772\n0.015998882468556984\n0.01559915920952335\n0.01576072332682088\n0.016472871112637223\n0.014691755402600393\n0.014136423316085712\n" }, { "path": "logs/loss_2022_05_02_14_57_57/epoch_loss.txt", "content": "17.101406224568684\n10.8318008740743\n4.240671507517496\n1.0019958794116974\n0.37954812149206796\n0.2687491794427236\n0.22754189471403757\n0.19753684798876445\n0.1771739900112152\n0.16613257378339769\n0.14869885842005412\n0.13755213419596354\n0.13448657716313997\n0.12195368086298307\n0.1128251701593399\n0.10961388771732648\n0.10665635019540787\n0.10061115821202596\n0.0969288428624471\n0.09855932394663493\n0.0889915977915128\n0.08737521395087242\n0.08142138893405597\n0.081571697195371\n0.08513322671254477\n0.0799174178391695\n0.07576848641037941\n0.07407469501097998\n0.07028314856191477\n0.07057048715651035\n0.0709464654326439\n0.07267625791331132\n0.06727536929150423\n0.0662232073644797\n0.06310114165147146\n0.06374188972016176\n0.06626531345148881\n0.05850081816315651\n0.056352414563298224\n0.05607227062185605\n0.057017019018530846\n0.05952403930326303\n0.057178026810288426\n0.051601182545224826\n0.051208433136343955\n0.051774655406673746\n0.050313881536324816\n0.04995381236076355\n0.048258970181147255\n0.04914092607796192\n0.06768884502040844\n0.06370118060149252\n0.05913636611464123\n0.05405666360942026\n0.052676150932287176\n0.04658079737176498\n0.0453374430614834\n0.04464669832183669\n0.04386587947762261\n0.038802354324919484\n0.038647202278176945\n0.03676449179959794\n0.03481319181931516\n0.0347878224371622\n0.03463629183825105\n0.03564592384112378\n0.03169099524772415\n0.03046195216011256\n0.029932656922998527\n0.02693921811878681\n0.02624520653237899\n0.02643638541145871\n0.024267646336617568\n0.02276813123996059\n0.022201836206174146\n0.025956252019386738\n0.022044219623785465\n0.01913531731891756\n0.018665816611610354\n0.020095466733134042\n0.019377306945777186\n0.019703271872519204\n0.017145425283039608\n0.017283631632259735\n0.015655260040269545\n0.017102580536932994\n0.01568767197119693\n0.015433585511830945\n0.01649760961299762\n0.01480112192220986\n0.01458095806495597\n0.01634620662080124\n0.014586444144758086\n0.01412225275610884\n0.014443966598870853\n0.014422304722635696\n0.014611958689056338\n0.01421121487316365\n0.014518235716968775\n0.01446291058867549\n" }, { "path": "logs/loss_2022_05_02_14_57_57/epoch_val_loss.txt", "content": "14.182828585306803\n6.964454015096028\n1.7364161411921184\n0.4160226086775462\n0.23061403135458627\n0.18009933829307556\n0.15316933890183768\n0.12558546662330627\n0.11013514300187428\n0.10292657961448033\n0.09011622269948323\n0.0910362775127093\n0.07362671693166097\n0.06496318926413854\n0.06620646268129349\n0.05724670241276423\n0.05412605529030164\n0.05476600428422292\n0.04998553295930227\n0.04453219473361969\n0.046111090729633965\n0.03964699556430181\n0.04128604009747505\n0.0385576585928599\n0.040300281097491585\n0.036520869781573616\n0.03233897313475609\n0.03402836248278618\n0.029543195540706318\n0.03613479311267535\n0.030847225338220596\n0.03196833903590838\n0.030614140133063\n0.027615018809835117\n0.029661099116007488\n0.028920121490955353\n0.031096385171016056\n0.026975831637779873\n0.02437760556737582\n0.024089227120081585\n0.024140140662590664\n0.02602989909549554\n0.023526831219593685\n0.023234928647677105\n0.02490025262037913\n0.024476055055856705\n0.02195119174818198\n0.02400912468632062\n0.021773086860775948\n0.021737251430749893\n0.03704084885808138\n0.027747553415023364\n0.02609148549918945\n0.027060106253394715\n0.02310138403509672\n0.02209098207262846\n0.019444907299027994\n0.01728303673175665\n0.022116302154385127\n0.017028711091440458\n0.018385969388943452\n0.020397630233604174\n0.017034396529197693\n0.0161269146662492\n0.014033435915525142\n0.015593188958099255\n0.015342251899150701\n0.015232413147504512\n0.01195777920432962\n0.013383755532021705\n0.01376453500527602\n0.012433087345785819\n0.010423123764877137\n0.011021508405414911\n0.010145062186683599\n0.011127662809135823\n0.009687475251177182\n0.010067089210049463\n0.008900713497916093\n0.009318945392106589\n0.008838421199470758\n0.008917749107170563\n0.008874757430301262\n0.00834214468844808\n0.009231974191677112\n0.00839424731496435\n0.00878818673439897\n0.008268425169472512\n0.008394974642075025\n0.008387481507200461\n0.008073390604784856\n0.008447423434028259\n0.007967768595195733\n0.008031251589552714\n0.007093459976693759\n0.0077013208960684445\n0.008188612150171628\n0.008229664276139094\n0.008362234892466893\n0.0081037561624096\n" }, { "path": "model_data/.gitattributes", "content": "*.pth filter=lfs diff=lfs merge=lfs -text\n" }, { "path": "model_data/gesture.yaml", "content": "#------------------------------detect.py--------------------------------#\n# 这一部分是为了半自动标注数据，可以减轻负担，需要提前训练一个权重，以Labelme格式保存\n# dir_origin_path 图片存放位置\n# dir_save_path Annotation保存位置\n# ----------------------------------------------------------------------#\ndir_detect_path: ./JPEGImages \ndetect_save_path: ./Annotation\n\n# ----------------------------- train.py -------------------------------#\nnc: 8 # 类别的数量\nclasses: [\"up\",\"down\",\"left\",\"right\",\"front\",\"back\",\"clockwise\",\"anticlockwise\"] # 类别\nconfidence: 0.5 # 置信度\nnms_iou: 0.3\nletterbox_image: False\n\nlr_decay_type: cos # 使用到的学习率下降方式，可选的有step、cos\n# 用于设置是否使用多线程读取数据\n# 开启后会加快数据读取速度，但是会占用更多内存\n# 内存较小的电脑可以设置为2或者0，win建议设为0\nnum_workers: 4" }, { "path": "model_data/gesture_classes.txt", "content": "up\ndown\nleft\nright\nfront\nback\nclockwise\nanticlockwise" }, { "path": "model_data/yolo_anchors.txt", "content": "12, 16, 19, 36, 40, 28, 36, 75, 76, 55, 72, 146, 142, 110, 192, 243, 459, 401" }, { "path": "model_data/yolotiny_anchors.txt", "content": "10,14, 23,27, 37,58, 81,82, 135,169, 344,319" }, { "path": "nets/CSPdarknet.py", "content": "import math\nfrom collections import OrderedDict\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\n\n#-------------------------------------------------#\n# MISH激活函数\n#-------------------------------------------------#\nclass Mish(nn.Module):\n def __init__(self):\n super(Mish, self).__init__()\n\n def forward(self, x):\n return x * torch.tanh(F.softplus(x))\n\n#---------------------------------------------------#\n# 卷积块 -> 卷积 + 标准化 + 激活函数\n# Conv2d + BatchNormalization + Mish\n#---------------------------------------------------#\nclass BasicConv(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride=1):\n super(BasicConv, self).__init__()\n\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size, stride, kernel_size//2, bias=False)\n self.bn = nn.BatchNorm2d(out_channels)\n self.activation = Mish()\n\n def forward(self, x):\n x = self.conv(x)\n x = self.bn(x)\n x = self.activation(x)\n return x\n\n#---------------------------------------------------#\n# CSPdarknet的结构块的组成部分\n# 内部堆叠的残差块\n#---------------------------------------------------#\nclass Resblock(nn.Module):\n def __init__(self, channels, hidden_channels=None):\n super(Resblock, self).__init__()\n\n if hidden_channels is None:\n hidden_channels = channels\n\n self.block = nn.Sequential(\n BasicConv(channels, hidden_channels, 1),\n BasicConv(hidden_channels, channels, 3)\n )\n\n def forward(self, x):\n return x + self.block(x)\n\n#--------------------------------------------------------------------#\n# CSPdarknet的结构块\n# 首先利用ZeroPadding2D和一个步长为2x2的卷积块进行高和宽的压缩\n# 然后建立一个大的残差边shortconv、这个大残差边绕过了很多的残差结构\n# 主干部分会对num_blocks进行循环，循环内部是残差结构。\n# 对于整个CSPdarknet的结构块，就是一个大残差块+内部多个小残差块\n#--------------------------------------------------------------------#\nclass Resblock_body(nn.Module):\n def __init__(self, in_channels, out_channels, num_blocks, first):\n super(Resblock_body, self).__init__()\n #----------------------------------------------------------------#\n # 利用一个步长为2x2的卷积块进行高和宽的压缩\n #----------------------------------------------------------------#\n self.downsample_conv = BasicConv(in_channels, out_channels, 3, stride=2)\n\n if first:\n #--------------------------------------------------------------------------#\n # 然后建立一个大的残差边self.split_conv0、这个大残差边绕过了很多的残差结构\n #--------------------------------------------------------------------------#\n self.split_conv0 = BasicConv(out_channels, out_channels, 1)\n\n #----------------------------------------------------------------#\n # 主干部分会对num_blocks进行循环，循环内部是残差结构。\n #----------------------------------------------------------------#\n self.split_conv1 = BasicConv(out_channels, out_channels, 1) \n self.blocks_conv = nn.Sequential(\n Resblock(channels=out_channels, hidden_channels=out_channels//2),\n BasicConv(out_channels, out_channels, 1)\n )\n\n self.concat_conv = BasicConv(out_channels*2, out_channels, 1)\n else:\n #--------------------------------------------------------------------------#\n # 然后建立一个大的残差边self.split_conv0、这个大残差边绕过了很多的残差结构\n #--------------------------------------------------------------------------#\n self.split_conv0 = BasicConv(out_channels, out_channels//2, 1)\n\n #----------------------------------------------------------------#\n # 主干部分会对num_blocks进行循环，循环内部是残差结构。\n #----------------------------------------------------------------#\n self.split_conv1 = BasicConv(out_channels, out_channels//2, 1)\n self.blocks_conv = nn.Sequential(\n *[Resblock(out_channels//2) for _ in range(num_blocks)],\n BasicConv(out_channels//2, out_channels//2, 1)\n )\n\n self.concat_conv = BasicConv(out_channels, out_channels, 1)\n\n def forward(self, x):\n x = self.downsample_conv(x)\n\n x0 = self.split_conv0(x)\n\n x1 = self.split_conv1(x)\n x1 = self.blocks_conv(x1)\n\n #------------------------------------#\n # 将大残差边再堆叠回来\n #------------------------------------#\n x = torch.cat([x1, x0], dim=1)\n #------------------------------------#\n # 最后对通道数进行整合\n #------------------------------------#\n x = self.concat_conv(x)\n\n return x\n\n#---------------------------------------------------#\n# CSPdarknet53 的主体部分\n# 输入为一张416x416x3的图片\n# 输出为三个有效特征层\n#---------------------------------------------------#\nclass CSPDarkNet(nn.Module):\n def __init__(self, layers):\n super(CSPDarkNet, self).__init__()\n self.inplanes = 32\n # 416,416,3 -> 416,416,32\n self.conv1 = BasicConv(3, self.inplanes, kernel_size=3, stride=1)\n self.feature_channels = [64, 128, 256, 512, 1024]\n\n self.stages = nn.ModuleList([\n # 416,416,32 -> 208,208,64\n Resblock_body(self.inplanes, self.feature_channels[0], layers[0], first=True),\n # 208,208,64 -> 104,104,128\n Resblock_body(self.feature_channels[0], self.feature_channels[1], layers[1], first=False),\n # 104,104,128 -> 52,52,256\n Resblock_body(self.feature_channels[1], self.feature_channels[2], layers[2], first=False),\n # 52,52,256 -> 26,26,512\n Resblock_body(self.feature_channels[2], self.feature_channels[3], layers[3], first=False),\n # 26,26,512 -> 13,13,1024\n Resblock_body(self.feature_channels[3], self.feature_channels[4], layers[4], first=False)\n ])\n\n self.num_features = 1\n for m in self.modules():\n if isinstance(m, nn.Conv2d):\n n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels\n m.weight.data.normal_(0, math.sqrt(2. / n))\n elif isinstance(m, nn.BatchNorm2d):\n m.weight.data.fill_(1)\n m.bias.data.zero_()\n\n\n def forward(self, x):\n x = self.conv1(x)\n\n x = self.stages[0](x)\n x = self.stages[1](x)\n out3 = self.stages[2](x)\n out4 = self.stages[3](out3)\n out5 = self.stages[4](out4)\n\n return out3, out4, out5\n \ndef darknet53(pretrained):\n model = CSPDarkNet([1, 2, 8, 8, 4])\n if pretrained:\n model.load_state_dict(torch.load(\"model_data/CSPdarknet53_backbone_weights.pth\"))\n return model\n" }, { "path": "nets/CSPdarknet53_tiny.py", "content": "import math\n\nimport torch\nimport torch.nn as nn\n\n\n#-------------------------------------------------#\n# 卷积块\n# Conv2d + BatchNorm2d + LeakyReLU\n#-------------------------------------------------#\nclass BasicConv(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride=1):\n super(BasicConv, self).__init__()\n\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size, stride, kernel_size//2, bias=False)\n self.bn = nn.BatchNorm2d(out_channels)\n self.activation = nn.LeakyReLU(0.1)\n\n def forward(self, x):\n x = self.conv(x)\n x = self.bn(x)\n x = self.activation(x)\n return x\n\n \n'''\n input\n |\n BasicConv\n -----------------------\n | |\n route_group route\n | |\n BasicConv |\n | |\n ------------------- |\n | | |\n route_1 BasicConv |\n | | |\n -----------------cat |\n | |\n ---- BasicConv |\n | | |\n feat cat---------------------\n |\n MaxPooling2D\n'''\n#---------------------------------------------------#\n# CSPdarknet53-tiny的结构块\n# 存在一个大残差边\n# 这个大残差边绕过了很多的残差结构\n#---------------------------------------------------#\nclass Resblock_body(nn.Module):\n def __init__(self, in_channels, out_channels):\n super(Resblock_body, self).__init__()\n self.out_channels = out_channels\n\n self.conv1 = BasicConv(in_channels, out_channels, 3)\n\n self.conv2 = BasicConv(out_channels//2, out_channels//2, 3)\n self.conv3 = BasicConv(out_channels//2, out_channels//2, 3)\n\n self.conv4 = BasicConv(out_channels, out_channels, 1)\n self.maxpool = nn.MaxPool2d([2,2],[2,2])\n\n def forward(self, x):\n # 利用一个3x3卷积进行特征整合\n x = self.conv1(x)\n # 引出一个大的残差边route\n route = x\n \n c = self.out_channels\n # 对特征层的通道进行分割，取第二部分作为主干部分。\n x = torch.split(x, c//2, dim = 1)[1]\n # 对主干部分进行3x3卷积\n x = self.conv2(x)\n # 引出一个小的残差边route_1\n route1 = x\n # 对第主干部分进行3x3卷积\n x = self.conv3(x)\n # 主干部分与残差部分进行相接\n x = torch.cat([x,route1], dim = 1) \n\n # 对相接后的结果进行1x1卷积\n x = self.conv4(x)\n feat = x\n x = torch.cat([route, x], dim = 1)\n \n # 利用最大池化进行高和宽的压缩\n x = self.maxpool(x)\n return x,feat\n\nclass CSPDarkNet(nn.Module):\n def __init__(self):\n super(CSPDarkNet, self).__init__()\n # 首先利用两次步长为2x2的3x3卷积进行高和宽的压缩\n # 416,416,3 -> 208,208,32 -> 104,104,64\n self.conv1 = BasicConv(3, 32, kernel_size=3, stride=2)\n self.conv2 = BasicConv(32, 64, kernel_size=3, stride=2)\n\n # 104,104,64 -> 52,52,128\n self.resblock_body1 = Resblock_body(64, 64)\n # 52,52,128 -> 26,26,256\n self.resblock_body2 = Resblock_body(128, 128)\n # 26,26,256 -> 13,13,512\n self.resblock_body3 = Resblock_body(256, 256)\n # 13,13,512 -> 13,13,512\n self.conv3 = BasicConv(512, 512, kernel_size=3)\n\n self.num_features = 1\n # 进行权值初始化\n for m in self.modules():\n if isinstance(m, nn.Conv2d):\n n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels\n m.weight.data.normal_(0, math.sqrt(2. / n))\n elif isinstance(m, nn.BatchNorm2d):\n m.weight.data.fill_(1)\n m.bias.data.zero_()\n\n\n def forward(self, x):\n # 416,416,3 -> 208,208,32 -> 104,104,64\n x = self.conv1(x)\n x = self.conv2(x)\n\n # 104,104,64 -> 52,52,128\n x, _ = self.resblock_body1(x)\n # 52,52,128 -> 26,26,256\n x, _ = self.resblock_body2(x)\n # 26,26,256 -> x为13,13,512\n # -> feat1为26,26,256\n x, feat1 = self.resblock_body3(x)\n\n # 13,13,512 -> 13,13,512\n x = self.conv3(x)\n feat2 = x\n return feat1,feat2\n\ndef darknet53_tiny(pretrained, **kwargs):\n model = CSPDarkNet()\n if pretrained:\n model.load_state_dict(torch.load(\"model_data/CSPdarknet53_tiny_backbone_weights.pth\"))\n return model\n" }, { "path": "nets/__init__.py", "content": "#" }, { "path": "nets/attention.py", "content": "import torch\nimport torch.nn as nn\nimport math\n\nclass se_block(nn.Module):\n def __init__(self, channel, ratio=16):\n super(se_block, self).__init__()\n self.avg_pool = nn.AdaptiveAvgPool2d(1)\n self.fc = nn.Sequential(\n nn.Linear(channel, channel // ratio, bias=False),\n nn.ReLU(inplace=True),\n nn.Linear(channel // ratio, channel, bias=False),\n nn.Sigmoid()\n )\n\n def forward(self, x):\n b, c, _, _ = x.size()\n y = self.avg_pool(x).view(b, c)\n y = self.fc(y).view(b, c, 1, 1)\n return x * y\n\nclass ChannelAttention(nn.Module):\n def __init__(self, in_planes, ratio=8):\n super(ChannelAttention, self).__init__()\n self.avg_pool = nn.AdaptiveAvgPool2d(1)\n self.max_pool = nn.AdaptiveMaxPool2d(1)\n\n # 利用1x1卷积代替全连接\n self.fc1 = nn.Conv2d(in_planes, in_planes // ratio, 1, bias=False)\n self.relu1 = nn.ReLU()\n self.fc2 = nn.Conv2d(in_planes // ratio, in_planes, 1, bias=False)\n\n self.sigmoid = nn.Sigmoid()\n\n def forward(self, x):\n avg_out = self.fc2(self.relu1(self.fc1(self.avg_pool(x))))\n max_out = self.fc2(self.relu1(self.fc1(self.max_pool(x))))\n out = avg_out + max_out\n return self.sigmoid(out)\n\nclass SpatialAttention(nn.Module):\n def __init__(self, kernel_size=7):\n super(SpatialAttention, self).__init__()\n\n assert kernel_size in (3, 7), 'kernel size must be 3 or 7'\n padding = 3 if kernel_size == 7 else 1\n self.conv1 = nn.Conv2d(2, 1, kernel_size, padding=padding, bias=False)\n self.sigmoid = nn.Sigmoid()\n\n def forward(self, x):\n avg_out = torch.mean(x, dim=1, keepdim=True)\n max_out, _ = torch.max(x, dim=1, keepdim=True)\n x = torch.cat([avg_out, max_out], dim=1)\n x = self.conv1(x)\n return self.sigmoid(x)\n\nclass cbam_block(nn.Module):\n def __init__(self, channel, ratio=8, kernel_size=7):\n super(cbam_block, self).__init__()\n self.channelattention = ChannelAttention(channel, ratio=ratio)\n self.spatialattention = SpatialAttention(kernel_size=kernel_size)\n\n def forward(self, x):\n x = x*self.channelattention(x)\n x = x*self.spatialattention(x)\n return x\n\nclass eca_block(nn.Module):\n def __init__(self, channel, b=1, gamma=2):\n super(eca_block, self).__init__()\n kernel_size = int(abs((math.log(channel, 2) + b) / gamma))\n kernel_size = kernel_size if kernel_size % 2 else kernel_size + 1\n \n self.avg_pool = nn.AdaptiveAvgPool2d(1)\n self.conv = nn.Conv1d(1, 1, kernel_size=kernel_size, padding=(kernel_size - 1) // 2, bias=False) \n self.sigmoid = nn.Sigmoid()\n\n def forward(self, x):\n y = self.avg_pool(x)\n y = self.conv(y.squeeze(-1).transpose(-1, -2)).transpose(-1, -2).unsqueeze(-1)\n y = self.sigmoid(y)\n return x * y.expand_as(x)\n\nclass CA_Block(nn.Module):\n def __init__(self, channel, reduction=16):\n super(CA_Block, self).__init__()\n \n self.conv_1x1 = nn.Conv2d(in_channels=channel, out_channels=channel//reduction, kernel_size=1, stride=1, bias=False)\n \n self.relu = nn.ReLU()\n self.bn = nn.BatchNorm2d(channel//reduction)\n \n self.F_h = nn.Conv2d(in_channels=channel//reduction, out_channels=channel, kernel_size=1, stride=1, bias=False)\n self.F_w = nn.Conv2d(in_channels=channel//reduction, out_channels=channel, kernel_size=1, stride=1, bias=False)\n \n self.sigmoid_h = nn.Sigmoid()\n self.sigmoid_w = nn.Sigmoid()\n \n def forward(self, x):\n _, _, h, w = x.size()\n \n x_h = torch.mean(x, dim = 3, keepdim = True).permute(0, 1, 3, 2)\n x_w = torch.mean(x, dim = 2, keepdim = True)\n \n x_cat_conv_relu = self.relu(self.bn(self.conv_1x1(torch.cat((x_h, x_w), 3))))\n \n x_cat_conv_split_h, x_cat_conv_split_w = x_cat_conv_relu.split([h, w], 3)\n \n s_h = self.sigmoid_h(self.F_h(x_cat_conv_split_h.permute(0, 1, 3, 2)))\n s_w = self.sigmoid_w(self.F_w(x_cat_conv_split_w))\n \n out = x * s_h.expand_as(x) * s_w.expand_as(x)\n return out\n " }, { "path": "nets/yolo.py", "content": "from collections import OrderedDict\n\nimport torch\nimport torch.nn as nn\n\nfrom nets.CSPdarknet import darknet53\n\n\ndef conv2d(filter_in, filter_out, kernel_size, stride=1):\n pad = (kernel_size - 1) // 2 if kernel_size else 0\n return nn.Sequential(OrderedDict([\n (\"conv\", nn.Conv2d(filter_in, filter_out, kernel_size=kernel_size, stride=stride, padding=pad, bias=False)),\n (\"bn\", nn.BatchNorm2d(filter_out)),\n (\"relu\", nn.LeakyReLU(0.1)),\n ]))\n\n#---------------------------------------------------#\n# SPP结构，利用不同大小的池化核进行池化\n# 池化后堆叠\n#---------------------------------------------------#\nclass SpatialPyramidPooling(nn.Module):\n def __init__(self, pool_sizes=[5, 9, 13]):\n super(SpatialPyramidPooling, self).__init__()\n\n self.maxpools = nn.ModuleList([nn.MaxPool2d(pool_size, 1, pool_size//2) for pool_size in pool_sizes])\n\n def forward(self, x):\n features = [maxpool(x) for maxpool in self.maxpools[::-1]]\n features = torch.cat(features + [x], dim=1)\n\n return features\n\n#---------------------------------------------------#\n# 卷积 + 上采样\n#---------------------------------------------------#\nclass Upsample(nn.Module):\n def __init__(self, in_channels, out_channels):\n super(Upsample, self).__init__()\n\n self.upsample = nn.Sequential(\n conv2d(in_channels, out_channels, 1),\n nn.Upsample(scale_factor=2, mode='nearest')\n )\n\n def forward(self, x,):\n x = self.upsample(x)\n return x\n\n#---------------------------------------------------#\n# 三次卷积块\n#---------------------------------------------------#\ndef make_three_conv(filters_list, in_filters):\n m = nn.Sequential(\n conv2d(in_filters, filters_list[0], 1),\n conv2d(filters_list[0], filters_list[1], 3),\n conv2d(filters_list[1], filters_list[0], 1),\n )\n return m\n\n#---------------------------------------------------#\n# 五次卷积块\n#---------------------------------------------------#\ndef make_five_conv(filters_list, in_filters):\n m = nn.Sequential(\n conv2d(in_filters, filters_list[0], 1),\n conv2d(filters_list[0], filters_list[1], 3),\n conv2d(filters_list[1], filters_list[0], 1),\n conv2d(filters_list[0], filters_list[1], 3),\n conv2d(filters_list[1], filters_list[0], 1),\n )\n return m\n\n#---------------------------------------------------#\n# 最后获得yolov4的输出\n#---------------------------------------------------#\ndef yolo_head(filters_list, in_filters):\n m = nn.Sequential(\n conv2d(in_filters, filters_list[0], 3),\n nn.Conv2d(filters_list[0], filters_list[1], 1),\n )\n return m\n\n#---------------------------------------------------#\n# yolo_body\n#---------------------------------------------------#\nclass YoloBody(nn.Module):\n def __init__(self, anchors_mask, num_classes, pretrained = False):\n super(YoloBody, self).__init__()\n #---------------------------------------------------# \n # 生成CSPdarknet53的主干模型\n # 获得三个有效特征层，他们的shape分别是：\n # 52,52,256\n # 26,26,512\n # 13,13,1024\n #---------------------------------------------------#\n self.backbone = darknet53(pretrained)\n\n self.conv1 = make_three_conv([512,1024],1024)\n self.SPP = SpatialPyramidPooling()\n self.conv2 = make_three_conv([512,1024],2048)\n\n self.upsample1 = Upsample(512,256)\n self.conv_for_P4 = conv2d(512,256,1)\n self.make_five_conv1 = make_five_conv([256, 512],512)\n\n self.upsample2 = Upsample(256,128)\n self.conv_for_P3 = conv2d(256,128,1)\n self.make_five_conv2 = make_five_conv([128, 256],256)\n\n # 3*(5+num_classes) = 3*(5+20) = 3*(4+1+20)=75\n self.yolo_head3 = yolo_head([256, len(anchors_mask[0]) * (5 + num_classes)],128)\n\n self.down_sample1 = conv2d(128,256,3,stride=2)\n self.make_five_conv3 = make_five_conv([256, 512],512)\n\n # 3*(5+num_classes) = 3*(5+20) = 3*(4+1+20)=75\n self.yolo_head2 = yolo_head([512, len(anchors_mask[1]) * (5 + num_classes)],256)\n\n self.down_sample2 = conv2d(256,512,3,stride=2)\n self.make_five_conv4 = make_five_conv([512, 1024],1024)\n\n # 3*(5+num_classes)=3*(5+20)=3*(4+1+20)=75\n self.yolo_head1 = yolo_head([1024, len(anchors_mask[2]) * (5 + num_classes)],512)\n\n\n def forward(self, x):\n # backbone\n x2, x1, x0 = self.backbone(x)\n\n # 13,13,1024 -> 13,13,512 -> 13,13,1024 -> 13,13,512 -> 13,13,2048 \n P5 = self.conv1(x0)\n P5 = self.SPP(P5)\n # 13,13,2048 -> 13,13,512 -> 13,13,1024 -> 13,13,512\n P5 = self.conv2(P5)\n\n # 13,13,512 -> 13,13,256 -> 26,26,256\n P5_upsample = self.upsample1(P5)\n # 26,26,512 -> 26,26,256\n P4 = self.conv_for_P4(x1)\n # 26,26,256 + 26,26,256 -> 26,26,512\n P4 = torch.cat([P4,P5_upsample],axis=1)\n # 26,26,512 -> 26,26,256 -> 26,26,512 -> 26,26,256 -> 26,26,512 -> 26,26,256\n P4 = self.make_five_conv1(P4)\n\n # 26,26,256 -> 26,26,128 -> 52,52,128\n P4_upsample = self.upsample2(P4)\n # 52,52,256 -> 52,52,128\n P3 = self.conv_for_P3(x2)\n # 52,52,128 + 52,52,128 -> 52,52,256\n P3 = torch.cat([P3,P4_upsample],axis=1)\n # 52,52,256 -> 52,52,128 -> 52,52,256 -> 52,52,128 -> 52,52,256 -> 52,52,128\n P3 = self.make_five_conv2(P3)\n\n # 52,52,128 -> 26,26,256\n P3_downsample = self.down_sample1(P3)\n # 26,26,256 + 26,26,256 -> 26,26,512\n P4 = torch.cat([P3_downsample,P4],axis=1)\n # 26,26,512 -> 26,26,256 -> 26,26,512 -> 26,26,256 -> 26,26,512 -> 26,26,256\n P4 = self.make_five_conv3(P4)\n\n # 26,26,256 -> 13,13,512\n P4_downsample = self.down_sample2(P4)\n # 13,13,512 + 13,13,512 -> 13,13,1024\n P5 = torch.cat([P4_downsample,P5],axis=1)\n # 13,13,1024 -> 13,13,512 -> 13,13,1024 -> 13,13,512 -> 13,13,1024 -> 13,13,512\n P5 = self.make_five_conv4(P5)\n\n #---------------------------------------------------#\n # 第三个特征层\n # y3=(batch_size,75,52,52)\n #---------------------------------------------------#\n out2 = self.yolo_head3(P3)\n #---------------------------------------------------#\n # 第二个特征层\n # y2=(batch_size,75,26,26)\n #---------------------------------------------------#\n out1 = self.yolo_head2(P4)\n #---------------------------------------------------#\n # 第一个特征层\n # y1=(batch_size,75,13,13)\n #---------------------------------------------------#\n out0 = self.yolo_head1(P5)\n\n return out0, out1, out2\n\n" }, { "path": "nets/yolo_tiny.py", "content": "import torch\nimport torch.nn as nn\n\nfrom nets.CSPdarknet53_tiny import darknet53_tiny\nfrom nets.attention import cbam_block, eca_block, se_block, CA_Block\n\nattention_block = [se_block, cbam_block, eca_block, CA_Block]\n\n#-------------------------------------------------#\n# 卷积块 -> 卷积 + 标准化 + 激活函数\n# Conv2d + BatchNormalization + LeakyReLU\n#-------------------------------------------------#\nclass BasicConv(nn.Module):\n def __init__(self, in_channels, out_channels, kernel_size, stride=1):\n super(BasicConv, self).__init__()\n\n self.conv = nn.Conv2d(in_channels, out_channels, kernel_size, stride, kernel_size//2, bias=False)\n self.bn = nn.BatchNorm2d(out_channels)\n self.activation = nn.LeakyReLU(0.1)\n\n def forward(self, x):\n x = self.conv(x)\n x = self.bn(x)\n x = self.activation(x)\n return x\n\n#---------------------------------------------------#\n# 卷积 + 上采样\n#---------------------------------------------------#\nclass Upsample(nn.Module):\n def __init__(self, in_channels, out_channels):\n super(Upsample, self).__init__()\n\n self.upsample = nn.Sequential(\n BasicConv(in_channels, out_channels, 1),\n nn.Upsample(scale_factor=2, mode='nearest')\n )\n\n def forward(self, x,):\n x = self.upsample(x)\n return x\n\n#---------------------------------------------------#\n# 最后获得yolov4的输出\n#---------------------------------------------------#\ndef yolo_head(filters_list, in_filters):\n m = nn.Sequential(\n BasicConv(in_filters, filters_list[0], 3),\n nn.Conv2d(filters_list[0], filters_list[1], 1),\n )\n return m\n#---------------------------------------------------#\n# yolo_body\n#---------------------------------------------------#\nclass YoloBodytiny(nn.Module):\n def __init__(self, anchors_mask, num_classes, phi=0, pretrained=False):\n super(YoloBodytiny, self).__init__()\n self.phi = phi\n self.backbone = darknet53_tiny(pretrained)\n\n self.conv_for_P5 = BasicConv(512,256,1)\n self.yolo_headP5 = yolo_head([512, len(anchors_mask[0]) * (5 + num_classes)],256)\n\n self.upsample = Upsample(256,128)\n self.yolo_headP4 = yolo_head([256, len(anchors_mask[1]) * (5 + num_classes)],384)\n\n if 1 <= self.phi and self.phi <= 4:\n self.feat1_att = attention_block[self.phi - 1](256)\n self.feat2_att = attention_block[self.phi - 1](512)\n self.upsample_att = attention_block[self.phi - 1](128)\n\n def forward(self, x):\n #---------------------------------------------------#\n # 生成CSPdarknet53_tiny的主干模型\n # feat1的shape为26,26,256\n # feat2的shape为13,13,512\n #---------------------------------------------------#\n feat1, feat2 = self.backbone(x)\n if 1 <= self.phi and self.phi <= 4:\n feat1 = self.feat1_att(feat1)\n feat2 = self.feat2_att(feat2)\n\n # 13,13,512 -> 13,13,256\n P5 = self.conv_for_P5(feat2)\n # 13,13,256 -> 13,13,512 -> 13,13,255\n out0 = self.yolo_headP5(P5) \n\n # 13,13,256 -> 13,13,128 -> 26,26,128\n P5_Upsample = self.upsample(P5)\n # 26,26,256 + 26,26,128 -> 26,26,384\n if 1 <= self.phi and self.phi <= 4:\n P5_Upsample = self.upsample_att(P5_Upsample)\n P4 = torch.cat([P5_Upsample,feat1],axis=1)\n\n # 26,26,384 -> 26,26,256 -> 26,26,255\n out1 = self.yolo_headP4(P4)\n \n return out0, out1\n\n" }, { "path": "nets/yolo_training.py", "content": "import math\nfrom functools import partial\n\nimport numpy as np\nimport torch\nimport torch.nn as nn\n\n\nclass YOLOLoss(nn.Module):\n def __init__(self, anchors, num_classes, input_shape, cuda, anchors_mask = [[6,7,8], [3,4,5], [0,1,2]], label_smoothing = 0, focal_loss = False, alpha = 0.25, gamma = 2):\n super(YOLOLoss, self).__init__()\n #-----------------------------------------------------------#\n # 13x13的特征层对应的anchor是[142, 110],[192, 243],[459, 401]\n # 26x26的特征层对应的anchor是[36, 75],[76, 55],[72, 146]\n # 52x52的特征层对应的anchor是[12, 16],[19, 36],[40, 28]\n #-----------------------------------------------------------#\n self.anchors = anchors\n self.num_classes = num_classes\n self.bbox_attrs = 5 + num_classes\n self.input_shape = input_shape\n self.anchors_mask = anchors_mask\n self.label_smoothing = label_smoothing\n\n self.balance = [0.4, 1.0, 4]\n self.box_ratio = 0.05\n self.obj_ratio = 5 * (input_shape[0] * input_shape[1]) / (416 ** 2)\n self.cls_ratio = 1 * (num_classes / 80)\n \n self.focal_loss = focal_loss\n self.focal_loss_ratio = 10\n self.alpha = alpha\n self.gamma = gamma\n\n self.ignore_threshold = 0.5\n self.cuda = cuda\n\n def clip_by_tensor(self, t, t_min, t_max):\n t = t.float()\n result = (t >= t_min).float() * t + (t < t_min).float() * t_min\n result = (result <= t_max).float() * result + (result > t_max).float() * t_max\n return result\n\n def MSELoss(self, pred, target):\n return torch.pow(pred - target, 2)\n\n def BCELoss(self, pred, target):\n epsilon = 1e-7\n pred = self.clip_by_tensor(pred, epsilon, 1.0 - epsilon)\n output = - target * torch.log(pred) - (1.0 - target) * torch.log(1.0 - pred)\n return output\n \n def box_ciou(self, b1, b2):\n \"\"\"\n 输入为：\n ----------\n b1: tensor, shape=(batch, feat_w, feat_h, anchor_num, 4), xywh\n b2: tensor, shape=(batch, feat_w, feat_h, anchor_num, 4), xywh\n\n 返回为：\n -------\n ciou: tensor, shape=(batch, feat_w, feat_h, anchor_num, 1)\n \"\"\"\n #----------------------------------------------------#\n # 求出预测框左上角右下角\n #----------------------------------------------------#\n b1_xy = b1[..., :2]\n b1_wh = b1[..., 2:4]\n b1_wh_half = b1_wh/2.\n b1_mins = b1_xy - b1_wh_half\n b1_maxes = b1_xy + b1_wh_half\n #----------------------------------------------------#\n # 求出真实框左上角右下角\n #----------------------------------------------------#\n b2_xy = b2[..., :2]\n b2_wh = b2[..., 2:4]\n b2_wh_half = b2_wh/2.\n b2_mins = b2_xy - b2_wh_half\n b2_maxes = b2_xy + b2_wh_half\n\n #----------------------------------------------------#\n # 求真实框和预测框所有的iou\n #----------------------------------------------------#\n intersect_mins = torch.max(b1_mins, b2_mins)\n intersect_maxes = torch.min(b1_maxes, b2_maxes)\n intersect_wh = torch.max(intersect_maxes - intersect_mins, torch.zeros_like(intersect_maxes))\n intersect_area = intersect_wh[..., 0] * intersect_wh[..., 1]\n b1_area = b1_wh[..., 0] * b1_wh[..., 1]\n b2_area = b2_wh[..., 0] * b2_wh[..., 1]\n union_area = b1_area + b2_area - intersect_area\n iou = intersect_area / torch.clamp(union_area,min = 1e-6)\n\n #----------------------------------------------------#\n # 计算中心的差距\n #----------------------------------------------------#\n center_distance = torch.sum(torch.pow((b1_xy - b2_xy), 2), axis=-1)\n \n #----------------------------------------------------#\n # 找到包裹两个框的最小框的左上角和右下角\n #----------------------------------------------------#\n enclose_mins = torch.min(b1_mins, b2_mins)\n enclose_maxes = torch.max(b1_maxes, b2_maxes)\n enclose_wh = torch.max(enclose_maxes - enclose_mins, torch.zeros_like(intersect_maxes))\n #----------------------------------------------------#\n # 计算对角线距离\n #----------------------------------------------------#\n enclose_diagonal = torch.sum(torch.pow(enclose_wh,2), axis=-1)\n ciou = iou - 1.0 * (center_distance) / torch.clamp(enclose_diagonal,min = 1e-6)\n \n v = (4 / (math.pi ** 2)) * torch.pow((torch.atan(b1_wh[..., 0] / torch.clamp(b1_wh[..., 1],min = 1e-6)) - torch.atan(b2_wh[..., 0] / torch.clamp(b2_wh[..., 1], min = 1e-6))), 2)\n alpha = v / torch.clamp((1.0 - iou + v), min=1e-6)\n ciou = ciou - alpha * v\n return ciou\n\n #---------------------------------------------------#\n # 平滑标签\n #---------------------------------------------------#\n def smooth_labels(self, y_true, label_smoothing, num_classes):\n return y_true * (1.0 - label_smoothing) + label_smoothing / num_classes\n\n def forward(self, l, input, targets=None):\n #----------------------------------------------------#\n # l 代表使用的是第几个有效特征层\n # input的shape为 bs, 3*(5+num_classes), 13, 13\n # bs, 3*(5+num_classes), 26, 26\n # bs, 3*(5+num_classes), 52, 52\n # targets 真实框的标签情况 [batch_size, num_gt, 5]\n #----------------------------------------------------#\n #--------------------------------#\n # 获得图片数量，特征层的高和宽\n #--------------------------------#\n bs = input.size(0)\n in_h = input.size(2)\n in_w = input.size(3)\n #-----------------------------------------------------------------------#\n # 计算步长\n # 每一个特征点对应原来的图片上多少个像素点\n # \n # 如果特征层为13x13的话，一个特征点就对应原来的图片上的32个像素点\n # 如果特征层为26x26的话，一个特征点就对应原来的图片上的16个像素点\n # 如果特征层为52x52的话，一个特征点就对应原来的图片上的8个像素点\n # stride_h = stride_w = 32、16、8\n #-----------------------------------------------------------------------#\n stride_h = self.input_shape[0] / in_h\n stride_w = self.input_shape[1] / in_w\n #-------------------------------------------------#\n # 此时获得的scaled_anchors大小是相对于特征层的\n #-------------------------------------------------#\n scaled_anchors = [(a_w / stride_w, a_h / stride_h) for a_w, a_h in self.anchors]\n #-----------------------------------------------#\n # 输入的input一共有三个，他们的shape分别是\n # bs, 3 * (5+num_classes), 13, 13 => bs, 3, 5 + num_classes, 13, 13 => batch_size, 3, 13, 13, 5 + num_classes\n\n # batch_size, 3, 13, 13, 5 + num_classes\n # batch_size, 3, 26, 26, 5 + num_classes\n # batch_size, 3, 52, 52, 5 + num_classes\n #-----------------------------------------------#\n prediction = input.view(bs, len(self.anchors_mask[l]), self.bbox_attrs, in_h, in_w).permute(0, 1, 3, 4, 2).contiguous()\n \n #-----------------------------------------------#\n # 先验框的中心位置的调整参数\n #-----------------------------------------------#\n x = torch.sigmoid(prediction[..., 0])\n y = torch.sigmoid(prediction[..., 1])\n #-----------------------------------------------#\n # 先验框的宽高调整参数\n #-----------------------------------------------#\n w = prediction[..., 2]\n h = prediction[..., 3]\n #-----------------------------------------------#\n # 获得置信度，是否有物体\n #-----------------------------------------------#\n conf = torch.sigmoid(prediction[..., 4])\n #-----------------------------------------------#\n # 种类置信度\n #-----------------------------------------------#\n pred_cls = torch.sigmoid(prediction[..., 5:])\n\n #-----------------------------------------------#\n # 获得网络应该有的预测结果\n #-----------------------------------------------#\n y_true, noobj_mask, box_loss_scale = self.get_target(l, targets, scaled_anchors, in_h, in_w)\n\n #---------------------------------------------------------------#\n # 将预测结果进行解码，判断预测结果和真实值的重合程度\n # 如果重合程度过大则忽略，因为这些特征点属于预测比较准确的特征点\n # 作为负样本不合适\n #----------------------------------------------------------------#\n noobj_mask, pred_boxes = self.get_ignore(l, x, y, h, w, targets, scaled_anchors, in_h, in_w, noobj_mask)\n\n if self.cuda:\n y_true = y_true.type_as(x)\n noobj_mask = noobj_mask.type_as(x)\n box_loss_scale = box_loss_scale.type_as(x)\n #--------------------------------------------------------------------------#\n # box_loss_scale是真实框宽高的乘积，宽高均在0-1之间，因此乘积也在0-1之间。\n # 2-宽高的乘积代表真实框越大，比重越小，小框的比重更大。\n # 使用iou损失时，大中小目标的回归损失不存在比例失衡问题，故弃用\n #--------------------------------------------------------------------------#\n box_loss_scale = 2 - box_loss_scale\n\n loss = 0\n obj_mask = y_true[..., 4] == 1\n n = torch.sum(obj_mask)\n if n != 0:\n #---------------------------------------------------------------#\n # 计算预测结果和真实结果的差距\n # loss_loc ciou回归损失\n # loss_cls 分类损失\n #---------------------------------------------------------------#\n ciou = self.box_ciou(pred_boxes, y_true[..., :4]).type_as(x)\n # loss_loc = torch.mean((1 - ciou)[obj_mask] * box_loss_scale[obj_mask])\n loss_loc = torch.mean((1 - ciou)[obj_mask])\n \n loss_cls = torch.mean(self.BCELoss(pred_cls[obj_mask], y_true[..., 5:][obj_mask]))\n loss += loss_loc * self.box_ratio + loss_cls * self.cls_ratio\n\n #---------------------------------------------------------------#\n # 计算是否包含物体的置信度损失\n #---------------------------------------------------------------#\n if self.focal_loss:\n pos_neg_ratio = torch.where(obj_mask, torch.ones_like(conf) * self.alpha, torch.ones_like(conf) * (1 - self.alpha)) \n hard_easy_ratio = torch.where(obj_mask, torch.ones_like(conf) - conf, conf) ** self.gamma\n loss_conf = torch.mean((self.BCELoss(conf, obj_mask.type_as(conf)) * pos_neg_ratio * hard_easy_ratio)[noobj_mask.bool() | obj_mask]) * self.focal_loss_ratio\n else: \n loss_conf = torch.mean(self.BCELoss(conf, obj_mask.type_as(conf))[noobj_mask.bool() | obj_mask])\n loss += loss_conf * self.balance[l] * self.obj_ratio\n # if n != 0:\n # print(loss_loc * self.box_ratio, loss_cls * self.cls_ratio, loss_conf * self.balance[l] * self.obj_ratio)\n return loss\n\n def calculate_iou(self, _box_a, _box_b):\n #-----------------------------------------------------------#\n # 计算真实框的左上角和右下角\n #-----------------------------------------------------------#\n b1_x1, b1_x2 = _box_a[:, 0] - _box_a[:, 2] / 2, _box_a[:, 0] + _box_a[:, 2] / 2\n b1_y1, b1_y2 = _box_a[:, 1] - _box_a[:, 3] / 2, _box_a[:, 1] + _box_a[:, 3] / 2\n #-----------------------------------------------------------#\n # 计算先验框获得的预测框的左上角和右下角\n #-----------------------------------------------------------#\n b2_x1, b2_x2 = _box_b[:, 0] - _box_b[:, 2] / 2, _box_b[:, 0] + _box_b[:, 2] / 2\n b2_y1, b2_y2 = _box_b[:, 1] - _box_b[:, 3] / 2, _box_b[:, 1] + _box_b[:, 3] / 2\n\n #-----------------------------------------------------------#\n # 将真实框和预测框都转化成左上角右下角的形式\n #-----------------------------------------------------------#\n box_a = torch.zeros_like(_box_a)\n box_b = torch.zeros_like(_box_b)\n box_a[:, 0], box_a[:, 1], box_a[:, 2], box_a[:, 3] = b1_x1, b1_y1, b1_x2, b1_y2\n box_b[:, 0], box_b[:, 1], box_b[:, 2], box_b[:, 3] = b2_x1, b2_y1, b2_x2, b2_y2\n\n #-----------------------------------------------------------#\n # A为真实框的数量，B为先验框的数量\n #-----------------------------------------------------------#\n A = box_a.size(0)\n B = box_b.size(0)\n\n #-----------------------------------------------------------#\n # 计算交的面积\n #-----------------------------------------------------------#\n max_xy = torch.min(box_a[:, 2:].unsqueeze(1).expand(A, B, 2), box_b[:, 2:].unsqueeze(0).expand(A, B, 2))\n min_xy = torch.max(box_a[:, :2].unsqueeze(1).expand(A, B, 2), box_b[:, :2].unsqueeze(0).expand(A, B, 2))\n inter = torch.clamp((max_xy - min_xy), min=0)\n inter = inter[:, :, 0] * inter[:, :, 1]\n #-----------------------------------------------------------#\n # 计算预测框和真实框各自的面积\n #-----------------------------------------------------------#\n area_a = ((box_a[:, 2]-box_a[:, 0]) * (box_a[:, 3]-box_a[:, 1])).unsqueeze(1).expand_as(inter) # [A,B]\n area_b = ((box_b[:, 2]-box_b[:, 0]) * (box_b[:, 3]-box_b[:, 1])).unsqueeze(0).expand_as(inter) # [A,B]\n #-----------------------------------------------------------#\n # 求IOU\n #-----------------------------------------------------------#\n union = area_a + area_b - inter\n return inter / union # [A,B]\n \n def get_target(self, l, targets, anchors, in_h, in_w):\n #-----------------------------------------------------#\n # 计算一共有多少张图片\n #-----------------------------------------------------#\n bs = len(targets)\n #-----------------------------------------------------#\n # 用于选取哪些先验框不包含物体\n #-----------------------------------------------------#\n noobj_mask = torch.ones(bs, len(self.anchors_mask[l]), in_h, in_w, requires_grad = False)\n #-----------------------------------------------------#\n # 让网络更加去关注小目标\n #-----------------------------------------------------#\n box_loss_scale = torch.zeros(bs, len(self.anchors_mask[l]), in_h, in_w, requires_grad = False)\n #-----------------------------------------------------#\n # batch_size, 3, 13, 13, 5 + num_classes\n #-----------------------------------------------------#\n y_true = torch.zeros(bs, len(self.anchors_mask[l]), in_h, in_w, self.bbox_attrs, requires_grad = False)\n for b in range(bs): \n if len(targets[b])==0:\n continue\n batch_target = torch.zeros_like(targets[b])\n #-------------------------------------------------------#\n # 计算出正样本在特征层上的中心点\n #-------------------------------------------------------#\n batch_target[:, [0,2]] = targets[b][:, [0,2]] * in_w\n batch_target[:, [1,3]] = targets[b][:, [1,3]] * in_h\n batch_target[:, 4] = targets[b][:, 4]\n batch_target = batch_target.cpu()\n \n #-------------------------------------------------------#\n # 将真实框转换一个形式\n # num_true_box, 4\n #-------------------------------------------------------#\n gt_box = torch.FloatTensor(torch.cat((torch.zeros((batch_target.size(0), 2)), batch_target[:, 2:4]), 1))\n #-------------------------------------------------------#\n # 将先验框转换一个形式\n # 9, 4\n #-------------------------------------------------------#\n anchor_shapes = torch.FloatTensor(torch.cat((torch.zeros((len(anchors), 2)), torch.FloatTensor(anchors)), 1))\n #-------------------------------------------------------#\n # 计算交并比\n # self.calculate_iou(gt_box, anchor_shapes) = [num_true_box, 9]每一个真实框和9个先验框的重合情况\n # best_ns:\n # [每个真实框最大的重合度max_iou, 每一个真实框最重合的先验框的序号]\n #-------------------------------------------------------#\n best_ns = torch.argmax(self.calculate_iou(gt_box, anchor_shapes), dim=-1)\n\n for t, best_n in enumerate(best_ns):\n if best_n not in self.anchors_mask[l]:\n continue\n #----------------------------------------#\n # 判断这个先验框是当前特征点的哪一个先验框\n #----------------------------------------#\n k = self.anchors_mask[l].index(best_n)\n #----------------------------------------#\n # 获得真实框属于哪个网格点\n #----------------------------------------#\n i = torch.floor(batch_target[t, 0]).long()\n j = torch.floor(batch_target[t, 1]).long()\n #----------------------------------------#\n # 取出真实框的种类\n #----------------------------------------#\n c = batch_target[t, 4].long()\n \n #----------------------------------------#\n # noobj_mask代表无目标的特征点\n #----------------------------------------#\n noobj_mask[b, k, j, i] = 0\n #----------------------------------------#\n # tx、ty代表中心调整参数的真实值\n #----------------------------------------#\n y_true[b, k, j, i, 0] = batch_target[t, 0]\n y_true[b, k, j, i, 1] = batch_target[t, 1]\n y_true[b, k, j, i, 2] = batch_target[t, 2]\n y_true[b, k, j, i, 3] = batch_target[t, 3]\n y_true[b, k, j, i, 4] = 1\n y_true[b, k, j, i, c + 5] = 1\n #----------------------------------------#\n # 用于获得xywh的比例\n # 大目标loss权重小，小目标loss权重大\n #----------------------------------------#\n box_loss_scale[b, k, j, i] = batch_target[t, 2] * batch_target[t, 3] / in_w / in_h\n return y_true, noobj_mask, box_loss_scale\n\n def get_ignore(self, l, x, y, h, w, targets, scaled_anchors, in_h, in_w, noobj_mask):\n #-----------------------------------------------------#\n # 计算一共有多少张图片\n #-----------------------------------------------------#\n bs = len(targets)\n\n #-----------------------------------------------------#\n # 生成网格，先验框中心，网格左上角\n #-----------------------------------------------------#\n grid_x = torch.linspace(0, in_w - 1, in_w).repeat(in_h, 1).repeat(\n int(bs * len(self.anchors_mask[l])), 1, 1).view(x.shape).type_as(x)\n grid_y = torch.linspace(0, in_h - 1, in_h).repeat(in_w, 1).t().repeat(\n int(bs * len(self.anchors_mask[l])), 1, 1).view(y.shape).type_as(x)\n\n # 生成先验框的宽高\n scaled_anchors_l = np.array(scaled_anchors)[self.anchors_mask[l]]\n anchor_w = torch.Tensor(scaled_anchors_l).index_select(1, torch.LongTensor([0])).type_as(x)\n anchor_h = torch.Tensor(scaled_anchors_l).index_select(1, torch.LongTensor([1])).type_as(x)\n \n anchor_w = anchor_w.repeat(bs, 1).repeat(1, 1, in_h * in_w).view(w.shape)\n anchor_h = anchor_h.repeat(bs, 1).repeat(1, 1, in_h * in_w).view(h.shape)\n #-------------------------------------------------------#\n # 计算调整后的先验框中心与宽高\n #-------------------------------------------------------#\n pred_boxes_x = torch.unsqueeze(x + grid_x, -1)\n pred_boxes_y = torch.unsqueeze(y + grid_y, -1)\n pred_boxes_w = torch.unsqueeze(torch.exp(w) * anchor_w, -1)\n pred_boxes_h = torch.unsqueeze(torch.exp(h) * anchor_h, -1)\n pred_boxes = torch.cat([pred_boxes_x, pred_boxes_y, pred_boxes_w, pred_boxes_h], dim = -1)\n \n for b in range(bs): \n #-------------------------------------------------------#\n # 将预测结果转换一个形式\n # pred_boxes_for_ignore num_anchors, 4\n #-------------------------------------------------------#\n pred_boxes_for_ignore = pred_boxes[b].view(-1, 4)\n #-------------------------------------------------------#\n # 计算真实框，并把真实框转换成相对于特征层的大小\n # gt_box num_true_box, 4\n #-------------------------------------------------------#\n if len(targets[b]) > 0:\n batch_target = torch.zeros_like(targets[b])\n #-------------------------------------------------------#\n # 计算出正样本在特征层上的中心点\n #-------------------------------------------------------#\n batch_target[:, [0,2]] = targets[b][:, [0,2]] * in_w\n batch_target[:, [1,3]] = targets[b][:, [1,3]] * in_h\n batch_target = batch_target[:, :4].type_as(x)\n #-------------------------------------------------------#\n # 计算交并比\n # anch_ious num_true_box, num_anchors\n #-------------------------------------------------------#\n anch_ious = self.calculate_iou(batch_target, pred_boxes_for_ignore)\n #-------------------------------------------------------#\n # 每个先验框对应真实框的最大重合度\n # anch_ious_max num_anchors\n #-------------------------------------------------------#\n anch_ious_max, _ = torch.max(anch_ious, dim = 0)\n anch_ious_max = anch_ious_max.view(pred_boxes[b].size()[:3])\n noobj_mask[b][anch_ious_max > self.ignore_threshold] = 0\n return noobj_mask, pred_boxes\n\ndef weights_init(net, init_type='normal', init_gain = 0.02):\n def init_func(m):\n classname = m.__class__.__name__\n if hasattr(m, 'weight') and classname.find('Conv') != -1:\n if init_type == 'normal':\n torch.nn.init.normal_(m.weight.data, 0.0, init_gain)\n elif init_type == 'xavier':\n torch.nn.init.xavier_normal_(m.weight.data, gain=init_gain)\n elif init_type == 'kaiming':\n torch.nn.init.kaiming_normal_(m.weight.data, a=0, mode='fan_in')\n elif init_type == 'orthogonal':\n torch.nn.init.orthogonal_(m.weight.data, gain=init_gain)\n else:\n raise NotImplementedError('initialization method [%s] is not implemented' % init_type)\n elif classname.find('BatchNorm2d') != -1:\n torch.nn.init.normal_(m.weight.data, 1.0, 0.02)\n torch.nn.init.constant_(m.bias.data, 0.0)\n print('initialize network with %s type' % init_type)\n net.apply(init_func)\n\ndef get_lr_scheduler(lr_decay_type, lr, min_lr, total_iters, warmup_iters_ratio = 0.05, warmup_lr_ratio = 0.1, no_aug_iter_ratio = 0.05, step_num = 10):\n def yolox_warm_cos_lr(lr, min_lr, total_iters, warmup_total_iters, warmup_lr_start, no_aug_iter, iters):\n if iters <= warmup_total_iters:\n # lr = (lr - warmup_lr_start) * iters / float(warmup_total_iters) + warmup_lr_start\n lr = (lr - warmup_lr_start) * pow(iters / float(warmup_total_iters), 2) + warmup_lr_start\n elif iters >= total_iters - no_aug_iter:\n lr = min_lr\n else:\n lr = min_lr + 0.5 * (lr - min_lr) * (\n 1.0 + math.cos(math.pi* (iters - warmup_total_iters) / (total_iters - warmup_total_iters - no_aug_iter))\n )\n return lr\n\n def step_lr(lr, decay_rate, step_size, iters):\n if step_size < 1:\n raise ValueError(\"step_size must above 1.\")\n n = iters // step_size\n out_lr = lr * decay_rate ** n\n return out_lr\n\n if lr_decay_type == \"cos\":\n warmup_total_iters = min(max(warmup_iters_ratio * total_iters, 1), 3)\n warmup_lr_start = max(warmup_lr_ratio * lr, 1e-6)\n no_aug_iter = min(max(no_aug_iter_ratio * total_iters, 1), 15)\n func = partial(yolox_warm_cos_lr ,lr, min_lr, total_iters, warmup_total_iters, warmup_lr_start, no_aug_iter)\n else:\n decay_rate = (min_lr / lr) ** (1 / (step_num - 1))\n step_size = total_iters / step_num\n func = partial(step_lr, lr, decay_rate, step_size)\n\n return func\n\ndef set_optimizer_lr(optimizer, lr_scheduler_func, epoch):\n lr = lr_scheduler_func(epoch)\n for param_group in optimizer.param_groups:\n param_group['lr'] = lr\n" }, { "path": "nets/yolotiny_training.py", "content": "import math\nfrom functools import partial\n\nimport numpy as np\nimport torch\nimport torch.nn as nn\n\nclass YOLOLosstiny(nn.Module):\n def __init__(self, anchors, num_classes, input_shape, cuda, anchors_mask = [[6,7,8], [3,4,5], [0,1,2]], label_smoothing = 0):\n super(YOLOLosstiny, self).__init__()\n #-----------------------------------------------------------#\n # 13x13的特征层对应的anchor是[81,82],[135,169],[344,319]\n # 26x26的特征层对应的anchor是[10,14],[23,27],[37,58]\n #-----------------------------------------------------------#\n self.anchors = anchors\n self.num_classes = num_classes\n self.bbox_attrs = 5 + num_classes\n self.input_shape = input_shape\n self.anchors_mask = anchors_mask\n self.label_smoothing = label_smoothing\n\n self.balance = [0.4, 1.0, 4]\n self.box_ratio = 0.05\n self.obj_ratio = 5 * (input_shape[0] * input_shape[1]) / (416 ** 2)\n self.cls_ratio = 1 * (num_classes / 80)\n\n self.ignore_threshold = 0.5\n self.cuda = cuda\n\n def clip_by_tensor(self, t, t_min, t_max):\n t = t.float()\n result = (t >= t_min).float() * t + (t < t_min).float() * t_min\n result = (result <= t_max).float() * result + (result > t_max).float() * t_max\n return result\n\n def MSELoss(self, pred, target):\n return torch.pow(pred - target, 2)\n\n def BCELoss(self, pred, target):\n epsilon = 1e-7\n pred = self.clip_by_tensor(pred, epsilon, 1.0 - epsilon)\n output = - target * torch.log(pred) - (1.0 - target) * torch.log(1.0 - pred)\n return output\n \n def box_ciou(self, b1, b2):\n \"\"\"\n 输入为：\n ----------\n b1: tensor, shape=(batch, feat_w, feat_h, anchor_num, 4), xywh\n b2: tensor, shape=(batch, feat_w, feat_h, anchor_num, 4), xywh\n\n 返回为：\n -------\n ciou: tensor, shape=(batch, feat_w, feat_h, anchor_num, 1)\n \"\"\"\n #----------------------------------------------------#\n # 求出预测框左上角右下角\n #----------------------------------------------------#\n b1_xy = b1[..., :2]\n b1_wh = b1[..., 2:4]\n b1_wh_half = b1_wh/2.\n b1_mins = b1_xy - b1_wh_half\n b1_maxes = b1_xy + b1_wh_half\n #----------------------------------------------------#\n # 求出真实框左上角右下角\n #----------------------------------------------------#\n b2_xy = b2[..., :2]\n b2_wh = b2[..., 2:4]\n b2_wh_half = b2_wh/2.\n b2_mins = b2_xy - b2_wh_half\n b2_maxes = b2_xy + b2_wh_half\n\n #----------------------------------------------------#\n # 求真实框和预测框所有的iou\n #----------------------------------------------------#\n intersect_mins = torch.max(b1_mins, b2_mins)\n intersect_maxes = torch.min(b1_maxes, b2_maxes)\n intersect_wh = torch.max(intersect_maxes - intersect_mins, torch.zeros_like(intersect_maxes))\n intersect_area = intersect_wh[..., 0] * intersect_wh[..., 1]\n b1_area = b1_wh[..., 0] * b1_wh[..., 1]\n b2_area = b2_wh[..., 0] * b2_wh[..., 1]\n union_area = b1_area + b2_area - intersect_area\n iou = intersect_area / torch.clamp(union_area,min = 1e-6)\n\n #----------------------------------------------------#\n # 计算中心的差距\n #----------------------------------------------------#\n center_distance = torch.sum(torch.pow((b1_xy - b2_xy), 2), axis=-1)\n \n #----------------------------------------------------#\n # 找到包裹两个框的最小框的左上角和右下角\n #----------------------------------------------------#\n enclose_mins = torch.min(b1_mins, b2_mins)\n enclose_maxes = torch.max(b1_maxes, b2_maxes)\n enclose_wh = torch.max(enclose_maxes - enclose_mins, torch.zeros_like(intersect_maxes))\n #----------------------------------------------------#\n # 计算对角线距离\n #----------------------------------------------------#\n enclose_diagonal = torch.sum(torch.pow(enclose_wh,2), axis=-1)\n ciou = iou - 1.0 * (center_distance) / torch.clamp(enclose_diagonal,min = 1e-6)\n \n v = (4 / (math.pi ** 2)) * torch.pow((torch.atan(b1_wh[..., 0] / torch.clamp(b1_wh[..., 1],min = 1e-6)) - torch.atan(b2_wh[..., 0] / torch.clamp(b2_wh[..., 1], min = 1e-6))), 2)\n alpha = v / torch.clamp((1.0 - iou + v), min=1e-6)\n ciou = ciou - alpha * v\n return ciou\n\n #---------------------------------------------------#\n # 平滑标签\n #---------------------------------------------------#\n def smooth_labels(self, y_true, label_smoothing, num_classes):\n return y_true * (1.0 - label_smoothing) + label_smoothing / num_classes\n\n def forward(self, l, input, targets=None):\n #----------------------------------------------------#\n # l 代表使用的是第几个有效特征层\n # input的shape为 bs, 3*(5+num_classes), 13, 13\n # bs, 3*(5+num_classes), 26, 26\n # targets 真实框的标签情况 [batch_size, num_gt, 5]\n #----------------------------------------------------#\n #--------------------------------#\n # 获得图片数量，特征层的高和宽\n #--------------------------------#\n bs = input.size(0)\n in_h = input.size(2)\n in_w = input.size(3)\n #-----------------------------------------------------------------------#\n # 计算步长\n # 每一个特征点对应原来的图片上多少个像素点\n # \n # 如果特征层为13x13的话，一个特征点就对应原来的图片上的32个像素点\n # 如果特征层为26x26的话，一个特征点就对应原来的图片上的16个像素点\n # stride_h = stride_w = 32、16\n #-----------------------------------------------------------------------#\n stride_h = self.input_shape[0] / in_h\n stride_w = self.input_shape[1] / in_w\n #-------------------------------------------------#\n # 此时获得的scaled_anchors大小是相对于特征层的\n #-------------------------------------------------#\n scaled_anchors = [(a_w / stride_w, a_h / stride_h) for a_w, a_h in self.anchors]\n #-----------------------------------------------#\n # 输入的input一共有三个，他们的shape分别是\n # bs, 3 * (5+num_classes), 13, 13 => bs, 3, 5 + num_classes, 13, 13 => batch_size, 3, 13, 13, 5 + num_classes\n\n # batch_size, 3, 13, 13, 5 + num_classes\n # batch_size, 3, 26, 26, 5 + num_classes\n #-----------------------------------------------#\n prediction = input.view(bs, len(self.anchors_mask[l]), self.bbox_attrs, in_h, in_w).permute(0, 1, 3, 4, 2).contiguous()\n \n #-----------------------------------------------#\n # 先验框的中心位置的调整参数\n #-----------------------------------------------#\n x = torch.sigmoid(prediction[..., 0])\n y = torch.sigmoid(prediction[..., 1])\n #-----------------------------------------------#\n # 先验框的宽高调整参数\n #-----------------------------------------------#\n w = prediction[..., 2]\n h = prediction[..., 3]\n #-----------------------------------------------#\n # 获得置信度，是否有物体\n #-----------------------------------------------#\n conf = torch.sigmoid(prediction[..., 4])\n #-----------------------------------------------#\n # 种类置信度\n #-----------------------------------------------#\n pred_cls = torch.sigmoid(prediction[..., 5:])\n\n #-----------------------------------------------#\n # 获得网络应该有的预测结果\n #-----------------------------------------------#\n y_true, noobj_mask, box_loss_scale = self.get_target(l, targets, scaled_anchors, in_h, in_w)\n\n #---------------------------------------------------------------#\n # 将预测结果进行解码，判断预测结果和真实值的重合程度\n # 如果重合程度过大则忽略，因为这些特征点属于预测比较准确的特征点\n # 作为负样本不合适\n #----------------------------------------------------------------#\n noobj_mask, pred_boxes = self.get_ignore(l, x, y, h, w, targets, scaled_anchors, in_h, in_w, noobj_mask)\n\n if self.cuda:\n y_true = y_true.type_as(x)\n noobj_mask = noobj_mask.type_as(x)\n box_loss_scale = box_loss_scale.type_as(x)\n #--------------------------------------------------------------------------#\n # box_loss_scale是真实框宽高的乘积，宽高均在0-1之间，因此乘积也在0-1之间。\n # 2-宽高的乘积代表真实框越大，比重越小，小框的比重更大。\n # 使用iou损失时，大中小目标的回归损失不存在比例失衡问题，故弃用\n #--------------------------------------------------------------------------#\n box_loss_scale = 2 - box_loss_scale\n\n loss = 0\n obj_mask = y_true[..., 4] == 1\n n = torch.sum(obj_mask)\n if n != 0:\n #---------------------------------------------------------------#\n # 计算预测结果和真实结果的差距\n # loss_loc ciou回归损失\n # loss_cls 分类损失\n #---------------------------------------------------------------#\n ciou = self.box_ciou(pred_boxes, y_true[..., :4]).type_as(x)\n # loss_loc = torch.mean((1 - ciou)[obj_mask] * box_loss_scale[obj_mask])\n loss_loc = torch.mean((1 - ciou)[obj_mask])\n \n loss_cls = torch.mean(self.BCELoss(pred_cls[obj_mask], y_true[..., 5:][obj_mask]))\n loss += loss_loc * self.box_ratio + loss_cls * self.cls_ratio\n\n loss_conf = torch.mean(self.BCELoss(conf, obj_mask.type_as(conf))[noobj_mask.bool() | obj_mask])\n loss += loss_conf * self.balance[l] * self.obj_ratio\n # if n != 0:\n # print(loss_loc * self.box_ratio, loss_cls * self.cls_ratio, loss_conf * self.balance[l] * self.obj_ratio)\n return loss\n\n def calculate_iou(self, _box_a, _box_b):\n #-----------------------------------------------------------#\n # 计算真实框的左上角和右下角\n #-----------------------------------------------------------#\n b1_x1, b1_x2 = _box_a[:, 0] - _box_a[:, 2] / 2, _box_a[:, 0] + _box_a[:, 2] / 2\n b1_y1, b1_y2 = _box_a[:, 1] - _box_a[:, 3] / 2, _box_a[:, 1] + _box_a[:, 3] / 2\n #-----------------------------------------------------------#\n # 计算先验框获得的预测框的左上角和右下角\n #-----------------------------------------------------------#\n b2_x1, b2_x2 = _box_b[:, 0] - _box_b[:, 2] / 2, _box_b[:, 0] + _box_b[:, 2] / 2\n b2_y1, b2_y2 = _box_b[:, 1] - _box_b[:, 3] / 2, _box_b[:, 1] + _box_b[:, 3] / 2\n\n #-----------------------------------------------------------#\n # 将真实框和预测框都转化成左上角右下角的形式\n #-----------------------------------------------------------#\n box_a = torch.zeros_like(_box_a)\n box_b = torch.zeros_like(_box_b)\n box_a[:, 0], box_a[:, 1], box_a[:, 2], box_a[:, 3] = b1_x1, b1_y1, b1_x2, b1_y2\n box_b[:, 0], box_b[:, 1], box_b[:, 2], box_b[:, 3] = b2_x1, b2_y1, b2_x2, b2_y2\n\n #-----------------------------------------------------------#\n # A为真实框的数量，B为先验框的数量\n #-----------------------------------------------------------#\n A = box_a.size(0)\n B = box_b.size(0)\n\n #-----------------------------------------------------------#\n # 计算交的面积\n #-----------------------------------------------------------#\n max_xy = torch.min(box_a[:, 2:].unsqueeze(1).expand(A, B, 2), box_b[:, 2:].unsqueeze(0).expand(A, B, 2))\n min_xy = torch.max(box_a[:, :2].unsqueeze(1).expand(A, B, 2), box_b[:, :2].unsqueeze(0).expand(A, B, 2))\n inter = torch.clamp((max_xy - min_xy), min=0)\n inter = inter[:, :, 0] * inter[:, :, 1]\n #-----------------------------------------------------------#\n # 计算预测框和真实框各自的面积\n #-----------------------------------------------------------#\n area_a = ((box_a[:, 2]-box_a[:, 0]) * (box_a[:, 3]-box_a[:, 1])).unsqueeze(1).expand_as(inter) # [A,B]\n area_b = ((box_b[:, 2]-box_b[:, 0]) * (box_b[:, 3]-box_b[:, 1])).unsqueeze(0).expand_as(inter) # [A,B]\n #-----------------------------------------------------------#\n # 求IOU\n #-----------------------------------------------------------#\n union = area_a + area_b - inter\n return inter / union # [A,B]\n \n def get_target(self, l, targets, anchors, in_h, in_w):\n #-----------------------------------------------------#\n # 计算一共有多少张图片\n #-----------------------------------------------------#\n bs = len(targets)\n #-----------------------------------------------------#\n # 用于选取哪些先验框不包含物体\n #-----------------------------------------------------#\n noobj_mask = torch.ones(bs, len(self.anchors_mask[l]), in_h, in_w, requires_grad = False)\n #-----------------------------------------------------#\n # 让网络更加去关注小目标\n #-----------------------------------------------------#\n box_loss_scale = torch.zeros(bs, len(self.anchors_mask[l]), in_h, in_w, requires_grad = False)\n #-----------------------------------------------------#\n # batch_size, 3, 13, 13, 5 + num_classes\n #-----------------------------------------------------#\n y_true = torch.zeros(bs, len(self.anchors_mask[l]), in_h, in_w, self.bbox_attrs, requires_grad = False)\n for b in range(bs): \n if len(targets[b])==0:\n continue\n batch_target = torch.zeros_like(targets[b])\n #-------------------------------------------------------#\n # 计算出正样本在特征层上的中心点\n #-------------------------------------------------------#\n batch_target[:, [0,2]] = targets[b][:, [0,2]] * in_w\n batch_target[:, [1,3]] = targets[b][:, [1,3]] * in_h\n batch_target[:, 4] = targets[b][:, 4]\n batch_target = batch_target.cpu()\n \n #-------------------------------------------------------#\n # 将真实框转换一个形式\n # num_true_box, 4\n #-------------------------------------------------------#\n gt_box = torch.FloatTensor(torch.cat((torch.zeros((batch_target.size(0), 2)), batch_target[:, 2:4]), 1))\n #-------------------------------------------------------#\n # 将先验框转换一个形式\n # 9, 4\n #-------------------------------------------------------#\n anchor_shapes = torch.FloatTensor(torch.cat((torch.zeros((len(anchors), 2)), torch.FloatTensor(anchors)), 1))\n #-------------------------------------------------------#\n # 计算交并比\n # self.calculate_iou(gt_box, anchor_shapes) = [num_true_box, 9]每一个真实框和9个先验框的重合情况\n # best_ns:\n # [每个真实框最大的重合度max_iou, 每一个真实框最重合的先验框的序号]\n #-------------------------------------------------------#\n iou = self.calculate_iou(gt_box, anchor_shapes)\n best_ns = torch.argmax(iou, dim=-1)\n sort_ns = torch.argsort(iou, dim=-1, descending=True)\n\n def check_in_anchors_mask(index, anchors_mask):\n for sub_anchors_mask in anchors_mask:\n if index in sub_anchors_mask:\n return True\n return False\n\n for t, best_n in enumerate(best_ns):\n #----------------------------------------#\n # 防止匹配到的先验框不在anchors_mask中\n #----------------------------------------#\n if not check_in_anchors_mask(best_n, self.anchors_mask):\n for index in sort_ns[t]:\n if check_in_anchors_mask(index, self.anchors_mask):\n best_n = index\n break\n\n if best_n not in self.anchors_mask[l]:\n continue\n #----------------------------------------#\n # 判断这个先验框是当前特征点的哪一个先验框\n #----------------------------------------#\n k = self.anchors_mask[l].index(best_n)\n #----------------------------------------#\n # 获得真实框属于哪个网格点\n #----------------------------------------#\n i = torch.floor(batch_target[t, 0]).long()\n j = torch.floor(batch_target[t, 1]).long()\n #----------------------------------------#\n # 取出真实框的种类\n #----------------------------------------#\n c = batch_target[t, 4].long()\n \n #----------------------------------------#\n # noobj_mask代表无目标的特征点\n #----------------------------------------#\n noobj_mask[b, k, j, i] = 0\n #----------------------------------------#\n # tx、ty代表中心调整参数的真实值\n #----------------------------------------#\n y_true[b, k, j, i, 0] = batch_target[t, 0]\n y_true[b, k, j, i, 1] = batch_target[t, 1]\n y_true[b, k, j, i, 2] = batch_target[t, 2]\n y_true[b, k, j, i, 3] = batch_target[t, 3]\n y_true[b, k, j, i, 4] = 1\n y_true[b, k, j, i, c + 5] = 1\n #----------------------------------------#\n # 用于获得xywh的比例\n # 大目标loss权重小，小目标loss权重大\n #----------------------------------------#\n box_loss_scale[b, k, j, i] = batch_target[t, 2] * batch_target[t, 3] / in_w / in_h\n return y_true, noobj_mask, box_loss_scale\n\n def get_ignore(self, l, x, y, h, w, targets, scaled_anchors, in_h, in_w, noobj_mask):\n #-----------------------------------------------------#\n # 计算一共有多少张图片\n #-----------------------------------------------------#\n bs = len(targets)\n\n #-----------------------------------------------------#\n # 生成网格，先验框中心，网格左上角\n #-----------------------------------------------------#\n grid_x = torch.linspace(0, in_w - 1, in_w).repeat(in_h, 1).repeat(\n int(bs * len(self.anchors_mask[l])), 1, 1).view(x.shape).type_as(x)\n grid_y = torch.linspace(0, in_h - 1, in_h).repeat(in_w, 1).t().repeat(\n int(bs * len(self.anchors_mask[l])), 1, 1).view(y.shape).type_as(x)\n\n # 生成先验框的宽高\n scaled_anchors_l = np.array(scaled_anchors)[self.anchors_mask[l]]\n anchor_w = torch.Tensor(scaled_anchors_l).index_select(1, torch.LongTensor([0])).type_as(x)\n anchor_h = torch.Tensor(scaled_anchors_l).index_select(1, torch.LongTensor([1])).type_as(x)\n \n anchor_w = anchor_w.repeat(bs, 1).repeat(1, 1, in_h * in_w).view(w.shape)\n anchor_h = anchor_h.repeat(bs, 1).repeat(1, 1, in_h * in_w).view(h.shape)\n #-------------------------------------------------------#\n # 计算调整后的先验框中心与宽高\n #-------------------------------------------------------#\n pred_boxes_x = torch.unsqueeze(x + grid_x, -1)\n pred_boxes_y = torch.unsqueeze(y + grid_y, -1)\n pred_boxes_w = torch.unsqueeze(torch.exp(w) * anchor_w, -1)\n pred_boxes_h = torch.unsqueeze(torch.exp(h) * anchor_h, -1)\n pred_boxes = torch.cat([pred_boxes_x, pred_boxes_y, pred_boxes_w, pred_boxes_h], dim = -1)\n for b in range(bs): \n #-------------------------------------------------------#\n # 将预测结果转换一个形式\n # pred_boxes_for_ignore num_anchors, 4\n #-------------------------------------------------------#\n pred_boxes_for_ignore = pred_boxes[b].view(-1, 4)\n #-------------------------------------------------------#\n # 计算真实框，并把真实框转换成相对于特征层的大小\n # gt_box num_true_box, 4\n #-------------------------------------------------------#\n if len(targets[b]) > 0:\n batch_target = torch.zeros_like(targets[b])\n #-------------------------------------------------------#\n # 计算出正样本在特征层上的中心点\n #-------------------------------------------------------#\n batch_target[:, [0,2]] = targets[b][:, [0,2]] * in_w\n batch_target[:, [1,3]] = targets[b][:, [1,3]] * in_h\n batch_target = batch_target[:, :4].type_as(x)\n #-------------------------------------------------------#\n # 计算交并比\n # anch_ious num_true_box, num_anchors\n #-------------------------------------------------------#\n anch_ious = self.calculate_iou(batch_target, pred_boxes_for_ignore)\n #-------------------------------------------------------#\n # 每个先验框对应真实框的最大重合度\n # anch_ious_max num_anchors\n #-------------------------------------------------------#\n anch_ious_max, _ = torch.max(anch_ious, dim = 0)\n anch_ious_max = anch_ious_max.view(pred_boxes[b].size()[:3])\n noobj_mask[b][anch_ious_max > self.ignore_threshold] = 0\n return noobj_mask, pred_boxes\n\ndef weights_init(net, init_type='normal', init_gain = 0.02):\n def init_func(m):\n classname = m.__class__.__name__\n if hasattr(m, 'weight') and classname.find('Conv') != -1:\n if init_type == 'normal':\n torch.nn.init.normal_(m.weight.data, 0.0, init_gain)\n elif init_type == 'xavier':\n torch.nn.init.xavier_normal_(m.weight.data, gain=init_gain)\n elif init_type == 'kaiming':\n torch.nn.init.kaiming_normal_(m.weight.data, a=0, mode='fan_in')\n elif init_type == 'orthogonal':\n torch.nn.init.orthogonal_(m.weight.data, gain=init_gain)\n else:\n raise NotImplementedError('initialization method [%s] is not implemented' % init_type)\n elif classname.find('BatchNorm2d') != -1:\n torch.nn.init.normal_(m.weight.data, 1.0, 0.02)\n torch.nn.init.constant_(m.bias.data, 0.0)\n print('initialize network with %s type' % init_type)\n net.apply(init_func)\n\ndef get_lr_scheduler(lr_decay_type, lr, min_lr, total_iters, warmup_iters_ratio = 0.05, warmup_lr_ratio = 0.1, no_aug_iter_ratio = 0.05, step_num = 10):\n def yolox_warm_cos_lr(lr, min_lr, total_iters, warmup_total_iters, warmup_lr_start, no_aug_iter, iters):\n if iters <= warmup_total_iters:\n # lr = (lr - warmup_lr_start) * iters / float(warmup_total_iters) + warmup_lr_start\n lr = (lr - warmup_lr_start) * pow(iters / float(warmup_total_iters), 2) + warmup_lr_start\n elif iters >= total_iters - no_aug_iter:\n lr = min_lr\n else:\n lr = min_lr + 0.5 * (lr - min_lr) * (\n 1.0 + math.cos(math.pi* (iters - warmup_total_iters) / (total_iters - warmup_total_iters - no_aug_iter))\n )\n return lr\n\n def step_lr(lr, decay_rate, step_size, iters):\n if step_size < 1:\n raise ValueError(\"step_size must above 1.\")\n n = iters // step_size\n out_lr = lr * decay_rate ** n\n return out_lr\n\n if lr_decay_type == \"cos\":\n warmup_total_iters = min(max(warmup_iters_ratio * total_iters, 1), 3)\n warmup_lr_start = max(warmup_lr_ratio * lr, 1e-6)\n no_aug_iter = min(max(no_aug_iter_ratio * total_iters, 1), 15)\n func = partial(yolox_warm_cos_lr ,lr, min_lr, total_iters, warmup_total_iters, warmup_lr_start, no_aug_iter)\n else:\n decay_rate = (min_lr / lr) ** (1 / (step_num - 1))\n step_size = total_iters / step_num\n func = partial(step_lr, lr, decay_rate, step_size)\n\n return func\n\ndef set_optimizer_lr(optimizer, lr_scheduler_func, epoch):\n lr = lr_scheduler_func(epoch)\n for param_group in optimizer.param_groups:\n param_group['lr'] = lr\n" }, { "path": "packages.txt", "content": "freeglut3-dev\nlibgtk2.0-dev" }, { "path": "predict.py", "content": "#-----------------------------------------------------------------------#\n# predict.py将单张图片预测、摄像头检测、FPS测试和目录遍历检测等功能\n# 整合到了一个py文件中，通过指定mode进行模式的修改。\n#-----------------------------------------------------------------------#\nimport time\nimport yaml\nimport cv2\nimport numpy as np\nfrom PIL import Image\nfrom get_yaml import get_config\nfrom yolo import YOLO\nimport argparse\nif __name__ == \"__main__\":\n parser = argparse.ArgumentParser()\n parser.add_argument('--weights',type=str,default='model_data/yolotiny_SE_ep100.pth',help='initial weights path')\n parser.add_argument('--tiny',action='store_true',help='使用yolotiny模型')\n parser.add_argument('--phi',type=int,default=1,help='yolov4tiny注意力机制类型')\n parser.add_argument('--mode',type=str,choices=['dir_predict', 'video', 'fps','predict','heatmap','export_onnx'],default=\"dir_predict\",help='预测的模式')\n parser.add_argument('--cuda',action='store_true',help='表示是否使用GPU')\n parser.add_argument('--shape',type=int,default=416,help='输入图像的shape')\n parser.add_argument('--video',type=str,default='',help='需要检测的视频文件')\n parser.add_argument('--save-video',type=str,default='',help='保存视频的位置')\n parser.add_argument('--confidence',type=float,default=0.5,help='只有得分大于置信度的预测框会被保留下来')\n parser.add_argument('--nms_iou',type=float,default=0.3,help='非极大抑制所用到的nms_iou大小')\n opt = parser.parse_args()\n print(opt)\n\n # 配置文件\n config = get_config()\n yolo = YOLO(opt)\n\n #----------------------------------------------------------------------------------------------------------#\n # mode用于指定测试的模式：\n # 'predict' 表示单张图片预测，如果想对预测过程进行修改，如保存图片，截取对象等，可以先看下方详细的注释\n # 'video' 表示视频检测，可调用摄像头或者视频进行检测，详情查看下方注释。\n # 'fps' 表示测试fps，使用的图片是img里面的street.jpg，详情查看下方注释。\n # 'dir_predict' 表示遍历文件夹进行检测并保存。默认遍历img文件夹，保存img_out文件夹，详情查看下方注释。\n # 'heatmap' 表示进行预测结果的热力图可视化，详情查看下方注释。\n # 'export_onnx' 表示将模型导出为onnx，需要pytorch1.7.1以上。\n #----------------------------------------------------------------------------------------------------------#\n mode = opt.mode\n #-------------------------------------------------------------------------#\n # crop 指定了是否在单张图片预测后对目标进行截取\n # count 指定了是否进行目标的计数\n # crop、count仅在mode='predict'时有效\n #-------------------------------------------------------------------------#\n crop = False\n count = False\n #----------------------------------------------------------------------------------------------------------#\n # video_path 用于指定视频的路径，当video_path=0时表示检测摄像头\n # 想要检测视频，则设置如video_path = \"xxx.mp4\"即可，代表读取出根目录下的xxx.mp4文件。\n # video_save_path 表示视频保存的路径，当video_save_path=\"\"时表示不保存\n # 想要保存视频，则设置如video_save_path = \"yyy.mp4\"即可，代表保存为根目录下的yyy.mp4文件。\n # video_fps 用于保存的视频的fps\n #\n # video_path、video_save_path和video_fps仅在mode='video'时有效\n # 保存视频时需要ctrl+c退出或者运行到最后一帧才会完成完整的保存步骤。\n #----------------------------------------------------------------------------------------------------------#\n video_path = 0 if opt.video == '' else opt.video\n video_save_path = opt.save_video\n video_fps = 25.0\n #----------------------------------------------------------------------------------------------------------#\n # test_interval 用于指定测量fps的时候，图片检测的次数。理论上test_interval越大，fps越准确。\n # fps_image_path 用于指定测试的fps图片\n # \n # test_interval和fps_image_path仅在mode='fps'有效\n #----------------------------------------------------------------------------------------------------------#\n test_interval = 100\n fps_image_path = \"img/up.jpg\"\n #-------------------------------------------------------------------------#\n # dir_origin_path 指定了用于检测的图片的文件夹路径\n # dir_save_path 指定了检测完图片的保存路径\n # \n # dir_origin_path和dir_save_path仅在mode='dir_predict'时有效\n #-------------------------------------------------------------------------#\n dir_origin_path = \"img/\"\n dir_save_path = \"img_out/\"\n #-------------------------------------------------------------------------#\n # heatmap_save_path 热力图的保存路径，默认保存在model_data下\n # \n # heatmap_save_path仅在mode='heatmap'有效\n #-------------------------------------------------------------------------#\n heatmap_save_path = \"model_data/heatmap_vision.png\"\n #-------------------------------------------------------------------------#\n # simplify 使用Simplify onnx\n # onnx_save_path 指定了onnx的保存路径\n #-------------------------------------------------------------------------#\n simplify = True\n onnx_save_path = \"model_data/models.onnx\"\n\n if mode == \"predict\":\n '''\n 1、如果想要进行检测完的图片的保存，利用r_image.save(\"img.jpg\")即可保存，直接在predict.py里进行修改即可。 \n 2、如果想要获得预测框的坐标，可以进入yolo.detect_image函数，在绘图部分读取top，left，bottom，right这四个值。\n 3、如果想要利用预测框截取下目标，可以进入yolo.detect_image函数，在绘图部分利用获取到的top，left，bottom，right这四个值\n 在原图上利用矩阵的方式进行截取。\n 4、如果想要在预测图上写额外的字，比如检测到的特定目标的数量，可以进入yolo.detect_image函数，在绘图部分对predicted_class进行判断，\n 比如判断if predicted_class == 'car': 即可判断当前目标是否为车，然后记录数量即可。利用draw.text即可写字。\n '''\n while True:\n img = input('Input image filename:')\n try:\n image = Image.open(img)\n except:\n print('Open Error! Try again!')\n continue\n else:\n r_image = yolo.detect_image(image, crop = crop, count=count)\n r_image.show()\n r_image.save(dir_save_path + 'img_result.jpg')\n\n elif mode == \"video\":\n capture = cv2.VideoCapture(video_path)\n if video_save_path != '':\n fourcc = cv2.VideoWriter_fourcc(*'XVID')\n size = (int(capture.get(cv2.CAP_PROP_FRAME_WIDTH)), int(capture.get(cv2.CAP_PROP_FRAME_HEIGHT)))\n out = cv2.VideoWriter(video_save_path, fourcc, video_fps, size)\n\n ref, frame = capture.read()\n if not ref:\n raise ValueError(\"未能正确读取摄像头（视频），请注意是否正确安装摄像头（是否正确填写视频路径）。\")\n\n fps = 0.0\n while(True):\n t1 = time.time()\n # 读取某一帧\n ref, frame = capture.read()\n if not ref:\n break\n # 格式转变，BGRtoRGB\n frame = cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)\n # 转变成Image\n frame = Image.fromarray(np.uint8(frame))\n # 进行检测\n frame = np.array(yolo.detect_image(frame))\n # RGBtoBGR满足opencv显示格式\n frame = cv2.cvtColor(frame,cv2.COLOR_RGB2BGR)\n \n fps = ( fps + (1./(time.time()-t1)) ) / 2\n print(\"fps= %.2f\"%(fps))\n frame = cv2.putText(frame, \"fps= %.2f\"%(fps), (0, 40), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2)\n \n cv2.imshow(\"video\",frame)\n c= cv2.waitKey(1) & 0xff \n if video_save_path != '':\n out.write(frame)\n\n if c==27:\n capture.release()\n break\n\n print(\"Video Detection Done!\")\n capture.release()\n if video_save_path != '':\n print(\"Save processed video to the path :\" + video_save_path)\n out.release()\n cv2.destroyAllWindows()\n \n elif mode == \"fps\":\n img = Image.open(fps_image_path)\n tact_time = yolo.get_FPS(img, test_interval)\n print(str(tact_time) + ' seconds, ' + str(1/tact_time) + 'FPS, @batch_size 1')\n\n elif mode == \"dir_predict\":\n import os\n\n from tqdm import tqdm\n\n img_names = os.listdir(dir_origin_path)\n for img_name in tqdm(img_names):\n if img_name.lower().endswith(('.bmp', '.dib', '.png', '.jpg', '.jpeg', '.pbm', '.pgm', '.ppm', '.tif', '.tiff')):\n image_path = os.path.join(dir_origin_path, img_name)\n image = Image.open(image_path)\n r_image = yolo.detect_image(image)\n if not os.path.exists(dir_save_path):\n os.makedirs(dir_save_path)\n r_image.save(os.path.join(dir_save_path, img_name.replace(\".jpg\", \".png\")), quality=95, subsampling=0)\n\n elif mode == \"heatmap\":\n while True:\n img = input('Input image filename:')\n try:\n image = Image.open(img)\n except:\n print('Open Error! Try again!')\n continue\n else:\n yolo.detect_heatmap(image, heatmap_save_path)\n \n elif mode == \"export_onnx\":\n yolo.convert_to_onnx(simplify, onnx_save_path)\n \n else:\n raise AssertionError(\"Please specify the correct mode: 'predict', 'video', 'fps', 'heatmap', 'export_onnx', 'dir_predict'.\")\n" }, { "path": "requirements.txt", "content": "scipy\nnumpy\nmatplotlib==3.7.0\nopencv_python\ntorch==1.8.1\ntorchvision==0.9.1\ntqdm==4.60.0\nPillow==8.2.0\nh5py==2.10.0\ntensorboard\npyyaml==6.0\ntorchinfo\nlabelimg==1.8.6\nstreamlit==1.8.1\nopencv-python-headless==4.5.2.52\nstreamlit<=1.11.*\n" }, { "path": "summary.py", "content": "#--------------------------------------------#\n# 该部分代码用于看网络结构\n#--------------------------------------------#\nimport torch\nfrom torchinfo import summary\n\nfrom nets.yolo import YoloBody\nfrom nets.yolo_tiny import YoloBodytiny\nif __name__ == \"__main__\":\n # 需要使用device来指定网络在GPU还是CPU运行\n device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n m = YoloBody([[6, 7, 8], [3, 4, 5], [0, 1, 2]], 80).to(device)\n summary(m, input_size=(1,3, 416, 416))\n\n m = YoloBodytiny([[3, 4, 5], [1, 2, 3]], 80, phi = 1).to(device)\n summary(m, input_size=(1,3, 416, 416))\n" }, { "path": "train.py", "content": "#-------------------------------------#\n# 对数据集进行训练\n#-------------------------------------#\nimport os\n\nimport numpy as np\nimport torch\nimport torch.backends.cudnn as cudnn\nimport torch.distributed as dist\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torch.utils.data import DataLoader\nfrom get_yaml import get_config\n\nfrom nets.yolo import YoloBody\nfrom nets.yolo_tiny import YoloBodytiny\nfrom nets.yolotiny_training import YOLOLosstiny\nfrom nets.yolo_training import (YOLOLoss, get_lr_scheduler, set_optimizer_lr,\n weights_init)\nfrom utils.callbacks import LossHistory\nfrom utils.dataloader import YoloDataset, yolo_dataset_collate\nfrom utils.utils import get_anchors, get_classes\nfrom utils.utils_fit import fit_one_epoch\nfrom utils.utils import get_lr\nimport warnings\nimport yaml\nwarnings.filterwarnings('ignore')\nimport argparse\n'''\n训练自己的目标检测模型一定需要注意以下几点：\n1、训练前仔细检查自己的格式是否满足要求，该库要求数据集格式为VOC格式，需要准备好的内容有输入图片和标签\n 输入图片为.jpg图片，无需固定大小，传入训练前会自动进行resize。\n 灰度图会自动转成RGB图片进行训练，无需自己修改。\n 输入图片如果后缀非jpg，需要自己批量转成jpg后再开始训练。\n\n 标签为.xml格式，文件中会有需要检测的目标信息，标签文件和输入图片文件相对应。\n\n2、训练好的权值文件保存在logs文件夹中，每个epoch都会保存一次，如果只是训练了几个step是不会保存的，epoch和step的概念要捋清楚一下。\n 在训练过程中，该代码并没有设定只保存最低损失的，因此按默认参数训练完会有100个权值，如果空间不够可以自行删除。\n 这个并不是保存越少越好也不是保存越多越好，有人想要都保存、有人想只保存一点，为了满足大多数的需求，还是都保存可选择性高。\n\n3、损失值的大小用于判断是否收敛，比较重要的是有收敛的趋势，即验证集损失不断下降，如果验证集损失基本上不改变的话，模型基本上就收敛了。\n 损失值的具体大小并没有什么意义，大和小只在于损失的计算方式，并不是接近于0才好。如果想要让损失好看点，可以直接到对应的损失函数里面除上10000。\n 训练过程中的损失值会保存在logs文件夹下的loss_%Y_%m_%d_%H_%M_%S文件夹中\n\n4、调参是一门蛮重要的学问，没有什么参数是一定好的，现有的参数是我测试过可以正常训练的参数，因此我会建议用现有的参数。\n 但是参数本身并不是绝对的，比如随着batch的增大学习率也可以增大，效果也会好一些；过深的网络不要用太大的学习率等等。\n 这些都是经验上，只能靠各位同学多查询资料和自己试试了。\n''' \nif __name__ == \"__main__\":\n parser = argparse.ArgumentParser()\n parser.add_argument('--init',type=int,default=0,help='从init epoch开始训练')\n parser.add_argument('--epochs',type=int, default=100,help='epochs for training')\n parser.add_argument('--weights','-w',type=str,default='',help='initial weights path 初始权重的路径')\n parser.add_argument('--freeze','-f',action='store_true',help='表示是否冻结训练')\n parser.add_argument('--freeze-epochs','-fe',type=int,default=50,help='epochs for feeze 冻结训练的迭代次数')\n parser.add_argument('--freeze-size', '-fb',type=int, default=32, help='total batch size for Freezeing')\n parser.add_argument('--batch-size','-bs',type=int, default=10, help='total batch size for all GPUs')\n parser.add_argument('--optimizer',type=str, choices=['sgd', 'adam', 'adamw'], default='adam', help='训练使用的optimizer')\n parser.add_argument('--workers',type=int, default=4, help='用于设置是否使用多线程读取数据')\n parser.add_argument('--lr',type=float,default=0.02,help='Learning Rate 学习率的初始值')\n parser.add_argument('--tiny',action='store_true',help='使用yolov4-tiny模型')\n parser.add_argument('--phi',type=int,default=0,help='yolov4-tiny所使用的注意力机制的类型')\n parser.add_argument('--weight-decay',type=float,default=0,help='权值衰减，可防止过拟合')\n parser.add_argument('--momentum',type=float,default=0.937,help='优化器中的参数')\n parser.add_argument('--save-period','-save',type=int,default=4,help='多少个epochs保存一次权重')\n parser.add_argument('--cuda',action='store_true',help='表示是否使用GPU')\n parser.add_argument('--shape',type=int,default=416,help='输入图像的shape，一定要是32的倍数')\n parser.add_argument('--fp16',action='store_true',help='是否使用混合精度训练')\n parser.add_argument('--mosaic',action='store_true',help='Yolov4的tricks应用 马赛克数据增强')\n parser.add_argument('--lr_decay_type',type=str,default='cos',choices=['cos','step'],help='cos')\n parser.add_argument('--distributed',action='store_true',help='是否使用多卡运行')\n parser.add_argument('--local_rank',type=int)\n parser.add_argument('--resume','-r',action='store_true',help='进行断点续传')\n opt = parser.parse_args()\n print(opt)\n # 配置文件\n config = get_config()\n\n # 断点续传功能\n resume = opt.resume\n if resume:\n try:\n if os.path.exists('logs/opt.yaml'): \n with open('logs/opt.yaml') as f:\n opt = yaml.load(f,Loader =yaml.FullLoader)\n opt = argparse.Namespace(**opt)\n opt.weights = 'logs/last.pth'\n print(\"mode resume :\",opt) \n except Exception as e:\n print(e)\n print(\"断点续传失败，重新训练\")\n #---------------------------------#\n # Cuda 是否使用Cuda\n # 没有GPU可以设置成False\n #---------------------------------#\n Cuda = opt.cuda\n #---------------------------------------------------------------------#\n # distributed 用于指定是否使用单机多卡分布式运行\n # 终端指令仅支持Ubuntu。CUDA_VISIBLE_DEVICES用于在Ubuntu下指定显卡。\n # Windows系统下默认使用DP模式调用所有显卡，不支持DDP。\n # DP模式：\n # 设置 distributed = False\n # 在终端中输入 CUDA_VISIBLE_DEVICES=0,1 python train.py\n # DDP模式：\n # 设置 distributed = True\n # 在终端中输入 CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 train.py\n #---------------------------------------------------------------------#\n distributed = opt.distributed\n #---------------------------------------------------------------------#\n # sync_bn 是否使用sync_bn，DDP模式多卡可用\n #---------------------------------------------------------------------#\n sync_bn = False\n #---------------------------------------------------------------------#\n # fp16 是否使用混合精度训练\n # 可减少约一半的显存、需要pytorch1.7.1以上\n #---------------------------------------------------------------------#\n fp16 = opt.fp16\n\n #---------------------------------------------------------------------#\n # anchors_path 代表先验框对应的txt文件，一般不修改。\n # anchors_mask 用于帮助代码找到对应的先验框，一般不修改。\n #---------------------------------------------------------------------#\n anchors_path = 'model_data/yolo_anchors.txt'\n if not opt.tiny:\n anchors_mask = [[6, 7, 8], [3, 4, 5], [0, 1, 2]]\n elif opt.tiny:\n anchors_mask = [[3,4,5], [1,2,3]]\n anchors_path = 'model_data/yolotiny_anchors.txt'\n #----------------------------------------------------------------------------------------------------------------------------#\n # 权值文件的下载请看README，可以通过网盘下载。模型的 预训练权重 对不同数据集是通用的，因为特征是通用的。\n # 模型的 预训练权重 比较重要的部分是 主干特征提取网络的权值部分，用于进行特征提取。\n # 预训练权重对于99%的情况都必须要用，不用的话主干部分的权值太过随机，特征提取效果不明显，网络训练的结果也不会好\n #\n # 如果训练过程中存在中断训练的操作，可以将model_path设置成logs文件夹下的权值文件，将已经训练了一部分的权值再次载入。\n # 同时修改下方的 冻结阶段 或者 解冻阶段 的参数，来保证模型epoch的连续性。\n # \n # 当model_path = ''的时候不加载整个模型的权值。\n #\n # 此处使用的是整个模型的权重，因此是在train.py进行加载的，下面的pretrain不影响此处的权值加载。\n # 如果想要让模型从主干的预训练权值开始训练，则设置model_path = ''，下面的pretrain = True，此时仅加载主干。\n # 如果想要让模型从0开始训练，则设置model_path = ''，下面的pretrain = Fasle，Freeze_Train = Fasle，此时从0开始训练，且没有冻结主干的过程。\n # \n # 一般来讲，网络从0开始的训练效果会很差，因为权值太过随机，特征提取效果不明显，因此非常、非常、非常不建议大家从0开始训练！\n # 从0开始训练有两个方案：\n # 1、得益于Mosaic数据增强方法强大的数据增强能力，将UnFreeze_Epoch设置的较大（300及以上）、batch较大（16及以上）、数据较多（万以上）的情况下，\n # 可以设置mosaic=True，直接随机初始化参数开始训练，但得到的效果仍然不如有预训练的情况。（像COCO这样的大数据集可以这样做）\n # 2、了解imagenet数据集，首先训练分类模型，获得网络的主干部分权值，分类模型的 主干部分 和该模型通用，基于此进行训练。\n #----------------------------------------------------------------------------------------------------------------------------#\n model_path = opt.weights\n #------------------------------------------------------#\n # input_shape 输入的shape大小，一定要是32的倍数\n #------------------------------------------------------#\n # input_shape = [416, 416]\n input_shape = [opt.shape,opt.shape]\n \n #----------------------------------------------------------------------------------------------------------------------------#\n # pretrained 是否使用主干网络的预训练权重，此处使用的是主干的权重，因此是在模型构建的时候进行加载的。\n # 如果设置了model_path，则主干的权值无需加载，pretrained的值无意义。\n # 如果不设置model_path，pretrained = True，此时仅加载主干开始训练。\n # 如果不设置model_path，pretrained = False，Freeze_Train = Fasle，此时从0开始训练，且没有冻结主干的过程。\n #----------------------------------------------------------------------------------------------------------------------------#\n pretrained = False\n #------------------------------------------------------#\n # Yolov4的tricks应用\n # mosaic 马赛克数据增强\n # 参考YoloX，由于Mosaic生成的训练图片，\n # 远远脱离自然图片的真实分布。\n # 本代码会在训练结束前的N个epoch自动关掉Mosaic\n # 100个世代会关闭30个世代（比例可在dataloader.py调整）\n # label_smoothing 标签平滑。一般0.01以下。如0.01、0.005\n #\n # 余弦退火算法的参数放到下面的lr_decay_type中设置\n #------------------------------------------------------#\n mosaic = opt.mosaic\n label_smoothing = 0\n\n #----------------------------------------------------------------------------------------------------------------------------#\n # 训练分为两个阶段，分别是冻结阶段和解冻阶段。设置冻结阶段是为了满足机器性能不足的同学的训练需求。\n # 冻结训练需要的显存较小，显卡非常差的情况下，可设置Freeze_Epoch等于UnFreeze_Epoch，此时仅仅进行冻结训练。\n # \n # 在此提供若干参数设置建议，各位训练者根据自己的需求进行灵活调整：\n # （一）从整个模型的预训练权重开始训练： \n # Adam：\n # Init_Epoch = 0，Freeze_Epoch = 50，UnFreeze_Epoch = 100，Freeze_Train = True，optimizer_type = 'adam'，Init_lr = 1e-3，weight_decay = 0。（冻结）\n # Init_Epoch = 0，UnFreeze_Epoch = 100，Freeze_Train = False，optimizer_type = 'adam'，Init_lr = 1e-3，weight_decay = 0。（不冻结）\n # SGD：\n # Init_Epoch = 0，Freeze_Epoch = 50，UnFreeze_Epoch = 100，Freeze_Train = True，optimizer_type = 'sgd'，Init_lr = 1e-2，weight_decay = 5e-4。（冻结）\n # Init_Epoch = 0，UnFreeze_Epoch = 100，Freeze_Train = False，optimizer_type = 'sgd'，Init_lr = 1e-2，weight_decay = 5e-4。（不冻结）\n # 其中：UnFreeze_Epoch可以在100-300之间调整。\n # （二）从主干网络的预训练权重开始训练：\n # Adam：\n # Init_Epoch = 0，Freeze_Epoch = 50，UnFreeze_Epoch = 100，Freeze_Train = True，optimizer_type = 'adam'，Init_lr = 1e-3，weight_decay = 0。（冻结）\n # Init_Epoch = 0，UnFreeze_Epoch = 100，Freeze_Train = False，optimizer_type = 'adam'，Init_lr = 1e-3，weight_decay = 0。（不冻结）\n # SGD：\n # Init_Epoch = 0，Freeze_Epoch = 50，UnFreeze_Epoch = 300，Freeze_Train = True，optimizer_type = 'sgd'，Init_lr = 1e-2，weight_decay = 5e-4。（冻结）\n # Init_Epoch = 0，UnFreeze_Epoch = 300，Freeze_Train = False，optimizer_type = 'sgd'，Init_lr = 1e-2，weight_decay = 5e-4。（不冻结）\n # 其中：由于从主干网络的预训练权重开始训练，主干的权值不一定适合目标检测，需要更多的训练跳出局部最优解。\n # UnFreeze_Epoch可以在150-300之间调整，YOLOV5和YOLOX均推荐使用300。\n # Adam相较于SGD收敛的快一些。因此UnFreeze_Epoch理论上可以小一点，但依然推荐更多的Epoch。\n # （三）从0开始训练：\n # Init_Epoch = 0，UnFreeze_Epoch >= 300，Unfreeze_batch_size >= 16，Freeze_Train = False（不冻结训练）\n # 其中：UnFreeze_Epoch尽量不小于300。optimizer_type = 'sgd'，Init_lr = 1e-2，mosaic = True。\n # （四）batch_size的设置：\n # 在显卡能够接受的范围内，以大为好。显存不足与数据集大小无关，提示显存不足（OOM或者CUDA out of memory）请调小batch_size。\n # 受到BatchNorm层影响，batch_size最小为2，不能为1。\n # 正常情况下Freeze_batch_size建议为Unfreeze_batch_size的1-2倍。不建议设置的差距过大，因为关系到学习率的自动调整。\n #----------------------------------------------------------------------------------------------------------------------------#\n #------------------------------------------------------------------#\n # 冻结阶段训练参数\n # 此时模型的主干被冻结了，特征提取网络不发生改变\n # 占用的显存较小，仅对网络进行微调\n # Init_Epoch 模型当前开始的训练世代，其值可以大于Freeze_Epoch，如设置：\n # Init_Epoch = 60、Freeze_Epoch = 50、UnFreeze_Epoch = 100\n # 会跳过冻结阶段，直接从60代开始，并调整对应的学习率。\n # （断点续练时使用）\n # Freeze_Epoch 模型冻结训练的Freeze_Epoch\n # (当Freeze_Train=False时失效)\n # Freeze_batch_size 模型冻结训练的batch_size\n # (当Freeze_Train=False时失效)\n #------------------------------------------------------------------#\n Init_Epoch = opt.init\n Freeze_Epoch = opt.freeze_epochs\n Freeze_batch_size = opt.freeze_size\n #------------------------------------------------------------------#\n # 解冻阶段训练参数\n # 此时模型的主干不被冻结了，特征提取网络会发生改变\n # 占用的显存较大，网络所有的参数都会发生改变\n # UnFreeze_Epoch 模型总共训练的epoch\n # Unfreeze_batch_size 模型在解冻后的batch_size\n #------------------------------------------------------------------#\n UnFreeze_Epoch = opt.epochs\n Unfreeze_batch_size = opt.batch_size\n #------------------------------------------------------------------#\n # Freeze_Train 是否进行冻结训练\n # 默认先冻结主干训练后解冻训练。\n #------------------------------------------------------------------#\n Freeze_Train = opt.freeze\n \n #------------------------------------------------------------------#\n # 其它训练参数：学习率、优化器、学习率下降有关\n #------------------------------------------------------------------#\n #------------------------------------------------------------------#\n # Init_lr 模型的最大学习率\n # Min_lr 模型的最小学习率，默认为最大学习率的0.01\n #------------------------------------------------------------------#\n Init_lr = opt.lr\n Min_lr = Init_lr * 0.01\n #------------------------------------------------------------------#\n # optimizer_type 使用到的优化器种类，可选的有adam、sgd\n # 当使用Adam优化器时建议设置 Init_lr=1e-3\n # 当使用SGD优化器时建议设置 Init_lr=1e-2\n # momentum 优化器内部使用到的momentum参数\n # weight_decay 权值衰减，可防止过拟合\n # adam会导致weight_decay错误，使用adam时建议设置为0。\n #------------------------------------------------------------------#\n optimizer_type = opt.optimizer\n momentum = opt.momentum\n weight_decay = opt.weight_decay\n #------------------------------------------------------------------#\n # lr_decay_type 使用到的学习率下降方式，可选的有step、cos\n #------------------------------------------------------------------#\n lr_decay_type = opt.lr_decay_type\n #------------------------------------------------------------------#\n # focal_loss 是否使用Focal Loss平衡正负样本\n # focal_alpha Focal Loss的正负样本平衡参数\n # focal_gamma Focal Loss的难易分类样本平衡参数\n #------------------------------------------------------------------#\n focal_loss = False\n focal_alpha = 0.25\n focal_gamma = 2\n #------------------------------------------------------------------#\n # save_period 多少个epoch保存一次权值，默认每个世代都保存\n #------------------------------------------------------------------#\n save_period = opt.save_period\n #------------------------------------------------------------------#\n # save_dir 权值与日志文件保存的文件夹\n #------------------------------------------------------------------#\n save_dir = 'logs'\n #------------------------------------------------------------------#\n # num_workers 用于设置是否使用多线程读取数据\n # 开启后会加快数据读取速度，但是会占用更多内存\n # 内存较小的电脑可以设置为2或者0 \n #------------------------------------------------------------------#\n num_workers = opt.workers\n\n #------------------------------------------------------#\n # train_annotation_path 训练图片路径和标签\n # val_annotation_path 验证图片路径和标签\n #------------------------------------------------------#\n train_annotation_path = '2007_train.txt'\n val_annotation_path = '2007_val.txt'\n\n #------------------------------------------------------#\n # 设置用到的显卡\n #------------------------------------------------------#\n ngpus_per_node = torch.cuda.device_count()\n if distributed:\n dist.init_process_group(backend=\"nccl\")\n local_rank = int(os.environ[\"LOCAL_RANK\"])\n rank = int(os.environ[\"RANK\"])\n device = torch.device(\"cuda\", local_rank)\n if local_rank == 0:\n print(f\"[{os.getpid()}] (rank = {rank}, local_rank = {local_rank}) training...\")\n print(\"Gpu Device Count : \", ngpus_per_node)\n else:\n device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n local_rank = 0\n \n #------------------------------------------------------#\n # 获取classes和anchor\n #------------------------------------------------------#\n class_names, num_classes = config['classes'], config['nc']\n anchors, num_anchors = get_anchors(anchors_path)\n \n #------------------------------------------------------#\n # 创建yolo模型\n #------------------------------------------------------#\n if not opt.tiny:\n model = YoloBody(anchors_mask, num_classes, pretrained = pretrained)\n elif opt.tiny:\n model = YoloBodytiny(anchors_mask, num_classes, pretrained = pretrained, phi = opt.phi)\n \n if not pretrained:\n weights_init(model)\n if model_path != \"\":\n if local_rank == 0:\n #------------------------------------------------------#\n # 权值文件请看README，百度网盘下载\n #------------------------------------------------------#\n print('Load weights {}.'.format(model_path))\n model_dict = model.state_dict()\n pretrained_dict = torch.load(model_path, map_location = device)\n pretrained_dict = {k: v for k, v in pretrained_dict.items() if np.shape(model_dict[k]) == np.shape(v)}\n model_dict.update(pretrained_dict)\n model.load_state_dict(model_dict)\n if not opt.tiny:\n yolo_loss = YOLOLoss(anchors, num_classes, input_shape, Cuda, anchors_mask, label_smoothing, focal_loss, focal_alpha, focal_gamma)\n elif opt.tiny:\n yolo_loss = YOLOLosstiny(anchors, num_classes, input_shape, Cuda, anchors_mask, label_smoothing)\n \n if local_rank == 0:\n loss_history = LossHistory(save_dir, model, input_shape=input_shape)\n else:\n loss_history = None\n \n if fp16:\n #------------------------------------------------------------------#\n # torch 1.2不支持amp，建议使用torch 1.7.1及以上正确使用fp16\n # 因此torch1.2这里显示\"could not be resolve\"\n #------------------------------------------------------------------#\n from torch.cuda.amp import GradScaler as GradScaler\n scaler = GradScaler()\n else:\n scaler = None\n\n model_train = model.train()\n #----------------------------#\n # 多卡同步Bn\n #----------------------------#\n if sync_bn and ngpus_per_node > 1 and distributed:\n model_train = torch.nn.SyncBatchNorm.convert_sync_batchnorm(model_train)\n elif sync_bn:\n print(\"Sync_bn is not support in one gpu or not distributed.\")\n\n if Cuda:\n if distributed:\n #----------------------------#\n # 多卡平行运行\n #----------------------------#\n model_train = model_train.cuda(local_rank)\n model_train = torch.nn.parallel.DistributedDataParallel(model_train, device_ids=[local_rank], find_unused_parameters=True)\n else:\n print(\"Gpu Device Count : \", ngpus_per_node)\n devices = [i for i in range(ngpus_per_node)]\n model_train = torch.nn.DataParallel(model, device_ids=devices)\n cudnn.benchmark = True\n model_train = model_train.cuda()\n\n #---------------------------#\n # 读取数据集对应的txt\n #---------------------------#\n with open(train_annotation_path, encoding='utf-8') as f:\n train_lines = f.readlines()\n with open(val_annotation_path, encoding='utf-8') as f:\n val_lines = f.readlines()\n num_train = len(train_lines)\n num_val = len(val_lines)\n\n #------------------------------------------------------#\n # 主干特征提取网络特征通用，冻结训练可以加快训练速度\n # 也可以在训练初期防止权值被破坏。\n # Init_Epoch为起始世代\n # Freeze_Epoch为冻结训练的世代\n # UnFreeze_Epoch总训练世代\n # 提示OOM或者显存不足请调小Batch_size\n #------------------------------------------------------#\n if True:\n UnFreeze_flag = False\n #------------------------------------#\n # 冻结一定部分训练\n #------------------------------------#\n if Freeze_Train:\n for param in model.backbone.parameters():\n param.requires_grad = False\n\n #-------------------------------------------------------------------#\n # 如果不冻结训练的话，直接设置batch_size为Unfreeze_batch_size\n #-------------------------------------------------------------------#\n batch_size = Freeze_batch_size if Freeze_Train else Unfreeze_batch_size\n\n #-------------------------------------------------------------------#\n # 判断当前batch_size，自适应调整学习率\n #-------------------------------------------------------------------#\n nbs = 64\n lr_limit_max = 1e-3 if optimizer_type in ['adam', 'adamw'] else 5e-2\n lr_limit_min = 3e-4 if optimizer_type in ['adam', 'adamw'] else 5e-4\n Init_lr_fit = min(max(batch_size / nbs * Init_lr, lr_limit_min), lr_limit_max)\n Min_lr_fit = min(max(batch_size / nbs * Min_lr, lr_limit_min * 1e-2), lr_limit_max * 1e-2)\n\n #---------------------------------------#\n # 根据optimizer_type选择优化器\n #---------------------------------------#\n pg0, pg1, pg2 = [], [], [] \n for k, v in model.named_modules():\n if hasattr(v, \"bias\") and isinstance(v.bias, nn.Parameter):\n pg2.append(v.bias) \n if isinstance(v, nn.BatchNorm2d) or \"bn\" in k:\n pg0.append(v.weight) \n elif hasattr(v, \"weight\") and isinstance(v.weight, nn.Parameter):\n pg1.append(v.weight) \n optimizer = {\n 'adam' : optim.Adam(pg0, Init_lr_fit, betas = (momentum, 0.999)),\n 'adamw' : optim.AdamW(pg0, Init_lr_fit, betas = (momentum, 0.999)),\n 'sgd' : optim.SGD(pg0, Init_lr_fit, momentum = momentum, nesterov=True)\n }[optimizer_type]\n optimizer.add_param_group({\"params\": pg1, \"weight_decay\": weight_decay})\n optimizer.add_param_group({\"params\": pg2})\n\n #---------------------------------------#\n # 获得学习率下降的公式\n #---------------------------------------#\n lr_scheduler_func = get_lr_scheduler(lr_decay_type, Init_lr_fit, Min_lr_fit, UnFreeze_Epoch)\n \n #---------------------------------------#\n # 判断每一个世代的长度\n #---------------------------------------#\n epoch_step = num_train // batch_size\n epoch_step_val = num_val // batch_size\n \n if epoch_step == 0 or epoch_step_val == 0:\n raise ValueError(\"数据集过小，无法继续进行训练，请扩充数据集。\")\n\n #---------------------------------------#\n # 构建数据集加载器。\n #---------------------------------------#\n train_dataset = YoloDataset(train_lines, input_shape, num_classes, epoch_length = UnFreeze_Epoch, mosaic=mosaic, train = True)\n val_dataset = YoloDataset(val_lines, input_shape, num_classes, epoch_length = UnFreeze_Epoch, mosaic=False, train = False)\n \n if distributed:\n train_sampler = torch.utils.data.distributed.DistributedSampler(train_dataset, shuffle=True,)\n val_sampler = torch.utils.data.distributed.DistributedSampler(val_dataset, shuffle=False,)\n batch_size = batch_size // ngpus_per_node\n shuffle = False\n else:\n train_sampler = None\n val_sampler = None\n shuffle = True\n\n gen = DataLoader(train_dataset, shuffle = shuffle, batch_size = batch_size, num_workers = num_workers, pin_memory=True,\n drop_last=True, collate_fn=yolo_dataset_collate, sampler=train_sampler)\n gen_val = DataLoader(val_dataset , shuffle = shuffle, batch_size = batch_size, num_workers = num_workers, pin_memory=True, \n drop_last=True, collate_fn=yolo_dataset_collate, sampler=val_sampler)\n\n #---------------------------------------#\n # 开始模型训练\n #---------------------------------------#\n for epoch in range(Init_Epoch, UnFreeze_Epoch):\n #---------------------------------------#\n # 如果模型有冻结学习部分\n # 则解冻，并设置参数\n #---------------------------------------#\n if epoch >= Freeze_Epoch and not UnFreeze_flag and Freeze_Train:\n batch_size = Unfreeze_batch_size\n\n #-------------------------------------------------------------------#\n # 判断当前batch_size，自适应调整学习率\n #-------------------------------------------------------------------#\n nbs = 64\n lr_limit_max = 1e-3 if optimizer_type in ['adam', 'adamw'] else 5e-2\n lr_limit_min = 3e-4 if optimizer_type in ['adam', 'adamw'] else 5e-4\n Init_lr_fit = min(max(batch_size / nbs * Init_lr, lr_limit_min), lr_limit_max)\n Min_lr_fit = min(max(batch_size / nbs * Min_lr, lr_limit_min * 1e-2), lr_limit_max * 1e-2)\n #---------------------------------------#\n # 获得学习率下降的公式\n #---------------------------------------#\n lr_scheduler_func = get_lr_scheduler(lr_decay_type, Init_lr_fit, Min_lr_fit, UnFreeze_Epoch)\n \n for param in model.backbone.parameters():\n param.requires_grad = True\n\n epoch_step = num_train // batch_size\n epoch_step_val = num_val // batch_size\n\n if epoch_step == 0 or epoch_step_val == 0:\n raise ValueError(\"数据集过小，无法继续进行训练，请扩充数据集。\")\n\n if distributed:\n batch_size = batch_size // ngpus_per_node\n \n gen = DataLoader(train_dataset, shuffle = shuffle, batch_size = batch_size, num_workers = num_workers, pin_memory=True,\n drop_last=True, collate_fn=yolo_dataset_collate, sampler=train_sampler)\n gen_val = DataLoader(val_dataset , shuffle = shuffle, batch_size = batch_size, num_workers = num_workers, pin_memory=True, \n drop_last=True, collate_fn=yolo_dataset_collate, sampler=val_sampler)\n\n UnFreeze_flag = True\n\n gen.dataset.epoch_now = epoch\n gen_val.dataset.epoch_now = epoch\n\n if distributed:\n train_sampler.set_epoch(epoch)\n\n set_optimizer_lr(optimizer, lr_scheduler_func, epoch)\n\n fit_one_epoch(model_train, model, yolo_loss, loss_history, optimizer, epoch, epoch_step, epoch_step_val, gen, gen_val, UnFreeze_Epoch, Cuda, fp16, scaler, save_period, save_dir, local_rank)\n # 记录\n opt.lr = get_lr(optimizer)\n opt.init = epoch + 1\n with open('logs/opt.yaml','w') as f:\n yaml.dump(vars(opt),f,encoding='utf-8',allow_unicode=True)\n if local_rank == 0:\n loss_history.writer.close()" }, { "path": "utils/__init__.py", "content": "#" }, { "path": "utils/callbacks.py", "content": "import datetime\nimport os\n\nimport torch\nimport matplotlib\nmatplotlib.use('Agg')\nimport scipy.signal\nfrom matplotlib import pyplot as plt\nfrom torch.utils.tensorboard import SummaryWriter\n\n\nclass LossHistory():\n def __init__(self, log_dir, model, input_shape):\n time_str = datetime.datetime.strftime(datetime.datetime.now(),'%Y_%m_%d_%H_%M_%S')\n self.log_dir = os.path.join(log_dir, \"loss_\" + str(time_str))\n self.losses = []\n self.val_loss = []\n \n os.makedirs(self.log_dir)\n self.writer = SummaryWriter(self.log_dir)\n try:\n dummy_input = torch.randn(2, 3, input_shape[0], input_shape[1])\n self.writer.add_graph(model, dummy_input)\n except:\n pass\n\n\n def append_loss(self, epoch, loss, val_loss):\n if not os.path.exists(self.log_dir):\n os.makedirs(self.log_dir)\n\n self.losses.append(loss)\n self.val_loss.append(val_loss)\n\n with open(os.path.join(self.log_dir, \"epoch_loss.txt\"), 'a') as f:\n f.write(str(loss))\n f.write(\"\\n\")\n with open(os.path.join(self.log_dir, \"epoch_val_loss.txt\"), 'a') as f:\n f.write(str(val_loss))\n f.write(\"\\n\")\n\n self.writer.add_scalar('loss', loss, epoch)\n self.writer.add_scalar('val_loss', val_loss, epoch)\n self.loss_plot()\n\n def loss_plot(self):\n iters = range(len(self.losses))\n\n plt.figure()\n plt.plot(iters, self.losses, 'red', linewidth = 2, label='train loss')\n plt.plot(iters, self.val_loss, 'coral', linewidth = 2, label='val loss')\n try:\n if len(self.losses) < 25:\n num = 5\n else:\n num = 15\n \n plt.plot(iters, scipy.signal.savgol_filter(self.losses, num, 3), 'green', linestyle = '--', linewidth = 2, label='smooth train loss')\n plt.plot(iters, scipy.signal.savgol_filter(self.val_loss, num, 3), '#8B4513', linestyle = '--', linewidth = 2, label='smooth val loss')\n except:\n pass\n\n plt.grid(True)\n plt.xlabel('Epoch')\n plt.ylabel('Loss')\n plt.legend(loc=\"upper right\")\n\n plt.savefig(os.path.join(self.log_dir, \"epoch_loss.png\"))\n\n plt.cla()\n plt.close(\"all\")\n" }, { "path": "utils/dataloader.py", "content": "from random import sample, shuffle\n\nimport cv2\nimport numpy as np\nimport torch\nfrom PIL import Image\nfrom torch.utils.data.dataset import Dataset\n\nfrom utils.utils import cvtColor, preprocess_input\n\n\nclass YoloDataset(Dataset):\n def __init__(self, annotation_lines, input_shape, num_classes, epoch_length, mosaic, train, mosaic_ratio = 0.7):\n super(YoloDataset, self).__init__()\n self.annotation_lines = annotation_lines\n self.input_shape = input_shape\n self.num_classes = num_classes\n self.epoch_length = epoch_length\n self.mosaic = mosaic\n self.train = train\n self.mosaic_ratio = mosaic_ratio\n\n self.epoch_now = -1\n self.length = len(self.annotation_lines)\n\n def __len__(self):\n return self.length\n\n def __getitem__(self, index):\n index = index % self.length\n\n #---------------------------------------------------#\n # 训练时进行数据的随机增强\n # 验证时不进行数据的随机增强\n #---------------------------------------------------#\n if self.mosaic:\n if self.rand() < 0.5 and self.epoch_now < self.epoch_length * self.mosaic_ratio:\n lines = sample(self.annotation_lines, 3)\n lines.append(self.annotation_lines[index])\n shuffle(lines)\n image, box = self.get_random_data_with_Mosaic(lines, self.input_shape)\n else:\n image, box = self.get_random_data(self.annotation_lines[index], self.input_shape, random = self.train)\n else:\n image, box = self.get_random_data(self.annotation_lines[index], self.input_shape, random = self.train)\n image = np.transpose(preprocess_input(np.array(image, dtype=np.float32)), (2, 0, 1))\n box = np.array(box, dtype=np.float32)\n if len(box) != 0:\n box[:, [0, 2]] = box[:, [0, 2]] / self.input_shape[1]\n box[:, [1, 3]] = box[:, [1, 3]] / self.input_shape[0]\n\n box[:, 2:4] = box[:, 2:4] - box[:, 0:2]\n box[:, 0:2] = box[:, 0:2] + box[:, 2:4] / 2\n return image, box\n\n def rand(self, a=0, b=1):\n return np.random.rand()*(b-a) + a\n\n def get_random_data(self, annotation_line, input_shape, jitter=.3, hue=.1, sat=0.7, val=0.4, random=True):\n line = annotation_line.split()\n #------------------------------#\n # 读取图像并转换成RGB图像\n #------------------------------#\n image = Image.open(line[0])\n image = cvtColor(image)\n #------------------------------#\n # 获得图像的高宽与目标高宽\n #------------------------------#\n iw, ih = image.size\n h, w = input_shape\n #------------------------------#\n # 获得预测框\n #------------------------------#\n box = np.array([np.array(list(map(int,box.split(',')))) for box in line[1:]])\n\n if not random:\n scale = min(w/iw, h/ih)\n nw = int(iw*scale)\n nh = int(ih*scale)\n dx = (w-nw)//2\n dy = (h-nh)//2\n\n #---------------------------------#\n # 将图像多余的部分加上灰条\n #---------------------------------#\n image = image.resize((nw,nh), Image.BICUBIC)\n new_image = Image.new('RGB', (w,h), (128,128,128))\n new_image.paste(image, (dx, dy))\n image_data = np.array(new_image, np.float32)\n\n #---------------------------------#\n # 对真实框进行调整\n #---------------------------------#\n if len(box)>0:\n np.random.shuffle(box)\n box[:, [0,2]] = box[:, [0,2]]*nw/iw + dx\n box[:, [1,3]] = box[:, [1,3]]*nh/ih + dy\n box[:, 0:2][box[:, 0:2]<0] = 0\n box[:, 2][box[:, 2]>w] = w\n box[:, 3][box[:, 3]>h] = h\n box_w = box[:, 2] - box[:, 0]\n box_h = box[:, 3] - box[:, 1]\n box = box[np.logical_and(box_w>1, box_h>1)] # discard invalid box\n\n return image_data, box\n \n #------------------------------------------#\n # 对图像进行缩放并且进行长和宽的扭曲\n #------------------------------------------#\n new_ar = iw/ih * self.rand(1-jitter,1+jitter) / self.rand(1-jitter,1+jitter)\n scale = self.rand(.25, 2)\n if new_ar < 1:\n nh = int(scale*h)\n nw = int(nh*new_ar)\n else:\n nw = int(scale*w)\n nh = int(nw/new_ar)\n image = image.resize((nw,nh), Image.BICUBIC)\n\n #------------------------------------------#\n # 将图像多余的部分加上灰条\n #------------------------------------------#\n dx = int(self.rand(0, w-nw))\n dy = int(self.rand(0, h-nh))\n new_image = Image.new('RGB', (w,h), (128,128,128))\n new_image.paste(image, (dx, dy))\n image = new_image\n\n #------------------------------------------#\n # 翻转图像\n #------------------------------------------#\n flip = self.rand()<.5\n if flip: image = image.transpose(Image.FLIP_LEFT_RIGHT)\n\n image_data = np.array(image, np.uint8)\n #---------------------------------#\n # 对图像进行色域变换\n # 计算色域变换的参数\n #---------------------------------#\n r = np.random.uniform(-1, 1, 3) * [hue, sat, val] + 1\n #---------------------------------#\n # 将图像转到HSV上\n #---------------------------------#\n hue, sat, val = cv2.split(cv2.cvtColor(image_data, cv2.COLOR_RGB2HSV))\n dtype = image_data.dtype\n #---------------------------------#\n # 应用变换\n #---------------------------------#\n x = np.arange(0, 256, dtype=r.dtype)\n lut_hue = ((x * r[0]) % 180).astype(dtype)\n lut_sat = np.clip(x * r[1], 0, 255).astype(dtype)\n lut_val = np.clip(x * r[2], 0, 255).astype(dtype)\n\n image_data = cv2.merge((cv2.LUT(hue, lut_hue), cv2.LUT(sat, lut_sat), cv2.LUT(val, lut_val)))\n image_data = cv2.cvtColor(image_data, cv2.COLOR_HSV2RGB)\n\n #---------------------------------#\n # 对真实框进行调整\n #---------------------------------#\n if len(box)>0:\n np.random.shuffle(box)\n box[:, [0,2]] = box[:, [0,2]]*nw/iw + dx\n box[:, [1,3]] = box[:, [1,3]]*nh/ih + dy\n if flip: box[:, [0,2]] = w - box[:, [2,0]]\n box[:, 0:2][box[:, 0:2]<0] = 0\n box[:, 2][box[:, 2]>w] = w\n box[:, 3][box[:, 3]>h] = h\n box_w = box[:, 2] - box[:, 0]\n box_h = box[:, 3] - box[:, 1]\n box = box[np.logical_and(box_w>1, box_h>1)] \n \n return image_data, box\n \n def merge_bboxes(self, bboxes, cutx, cuty):\n merge_bbox = []\n for i in range(len(bboxes)):\n for box in bboxes[i]:\n tmp_box = []\n x1, y1, x2, y2 = box[0], box[1], box[2], box[3]\n\n if i == 0:\n if y1 > cuty or x1 > cutx:\n continue\n if y2 >= cuty and y1 <= cuty:\n y2 = cuty\n if x2 >= cutx and x1 <= cutx:\n x2 = cutx\n\n if i == 1:\n if y2 < cuty or x1 > cutx:\n continue\n if y2 >= cuty and y1 <= cuty:\n y1 = cuty\n if x2 >= cutx and x1 <= cutx:\n x2 = cutx\n\n if i == 2:\n if y2 < cuty or x2 < cutx:\n continue\n if y2 >= cuty and y1 <= cuty:\n y1 = cuty\n if x2 >= cutx and x1 <= cutx:\n x1 = cutx\n\n if i == 3:\n if y1 > cuty or x2 < cutx:\n continue\n if y2 >= cuty and y1 <= cuty:\n y2 = cuty\n if x2 >= cutx and x1 <= cutx:\n x1 = cutx\n tmp_box.append(x1)\n tmp_box.append(y1)\n tmp_box.append(x2)\n tmp_box.append(y2)\n tmp_box.append(box[-1])\n merge_bbox.append(tmp_box)\n return merge_bbox\n\n def get_random_data_with_Mosaic(self, annotation_line, input_shape, jitter=0.3, hue=.1, sat=0.7, val=0.4):\n h, w = input_shape\n min_offset_x = self.rand(0.3, 0.7)\n min_offset_y = self.rand(0.3, 0.7)\n\n image_datas = [] \n box_datas = []\n index = 0\n for line in annotation_line:\n #---------------------------------#\n # 每一行进行分割\n #---------------------------------#\n line_content = line.split()\n #---------------------------------#\n # 打开图片\n #---------------------------------#\n image = Image.open(line_content[0])\n image = cvtColor(image)\n \n #---------------------------------#\n # 图片的大小\n #---------------------------------#\n iw, ih = image.size\n #---------------------------------#\n # 保存框的位置\n #---------------------------------#\n box = np.array([np.array(list(map(int,box.split(',')))) for box in line_content[1:]])\n \n #---------------------------------#\n # 是否翻转图片\n #---------------------------------#\n flip = self.rand()<.5\n if flip and len(box)>0:\n image = image.transpose(Image.FLIP_LEFT_RIGHT)\n box[:, [0,2]] = iw - box[:, [2,0]]\n\n #------------------------------------------#\n # 对图像进行缩放并且进行长和宽的扭曲\n #------------------------------------------#\n new_ar = iw/ih * self.rand(1-jitter,1+jitter) / self.rand(1-jitter,1+jitter)\n scale = self.rand(.4, 1)\n if new_ar < 1:\n nh = int(scale*h)\n nw = int(nh*new_ar)\n else:\n nw = int(scale*w)\n nh = int(nw/new_ar)\n image = image.resize((nw, nh), Image.BICUBIC)\n\n #-----------------------------------------------#\n # 将图片进行放置，分别对应四张分割图片的位置\n #-----------------------------------------------#\n if index == 0:\n dx = int(w*min_offset_x) - nw\n dy = int(h*min_offset_y) - nh\n elif index == 1:\n dx = int(w*min_offset_x) - nw\n dy = int(h*min_offset_y)\n elif index == 2:\n dx = int(w*min_offset_x)\n dy = int(h*min_offset_y)\n elif index == 3:\n dx = int(w*min_offset_x)\n dy = int(h*min_offset_y) - nh\n \n new_image = Image.new('RGB', (w,h), (128,128,128))\n new_image.paste(image, (dx, dy))\n image_data = np.array(new_image)\n\n index = index + 1\n box_data = []\n #---------------------------------#\n # 对box进行重新处理\n #---------------------------------#\n if len(box)>0:\n np.random.shuffle(box)\n box[:, [0,2]] = box[:, [0,2]]*nw/iw + dx\n box[:, [1,3]] = box[:, [1,3]]*nh/ih + dy\n box[:, 0:2][box[:, 0:2]<0] = 0\n box[:, 2][box[:, 2]>w] = w\n box[:, 3][box[:, 3]>h] = h\n box_w = box[:, 2] - box[:, 0]\n box_h = box[:, 3] - box[:, 1]\n box = box[np.logical_and(box_w>1, box_h>1)]\n box_data = np.zeros((len(box),5))\n box_data[:len(box)] = box\n \n image_datas.append(image_data)\n box_datas.append(box_data)\n\n #---------------------------------#\n # 将图片分割，放在一起\n #---------------------------------#\n cutx = int(w * min_offset_x)\n cuty = int(h * min_offset_y)\n\n new_image = np.zeros([h, w, 3])\n new_image[:cuty, :cutx, :] = image_datas[0][:cuty, :cutx, :]\n new_image[cuty:, :cutx, :] = image_datas[1][cuty:, :cutx, :]\n new_image[cuty:, cutx:, :] = image_datas[2][cuty:, cutx:, :]\n new_image[:cuty, cutx:, :] = image_datas[3][:cuty, cutx:, :]\n\n new_image = np.array(new_image, np.uint8)\n #---------------------------------#\n # 对图像进行色域变换\n # 计算色域变换的参数\n #---------------------------------#\n r = np.random.uniform(-1, 1, 3) * [hue, sat, val] + 1\n #---------------------------------#\n # 将图像转到HSV上\n #---------------------------------#\n hue, sat, val = cv2.split(cv2.cvtColor(new_image, cv2.COLOR_RGB2HSV))\n dtype = new_image.dtype\n #---------------------------------#\n # 应用变换\n #---------------------------------#\n x = np.arange(0, 256, dtype=r.dtype)\n lut_hue = ((x * r[0]) % 180).astype(dtype)\n lut_sat = np.clip(x * r[1], 0, 255).astype(dtype)\n lut_val = np.clip(x * r[2], 0, 255).astype(dtype)\n\n new_image = cv2.merge((cv2.LUT(hue, lut_hue), cv2.LUT(sat, lut_sat), cv2.LUT(val, lut_val)))\n new_image = cv2.cvtColor(new_image, cv2.COLOR_HSV2RGB)\n\n #---------------------------------#\n # 对框进行进一步的处理\n #---------------------------------#\n new_boxes = self.merge_bboxes(box_datas, cutx, cuty)\n\n return new_image, new_boxes\n\n# DataLoader中collate_fn使用\ndef yolo_dataset_collate(batch):\n images = []\n bboxes = []\n for img, box in batch:\n images.append(img)\n bboxes.append(box)\n images = torch.from_numpy(np.array(images)).type(torch.FloatTensor)\n bboxes = [torch.from_numpy(ann).type(torch.FloatTensor) for ann in bboxes]\n return images, bboxes\n" }, { "path": "utils/utils.py", "content": "import numpy as np\nfrom PIL import Image\n\n#---------------------------------------------------------#\n# 将图像转换成RGB图像，防止灰度图在预测时报错。\n# 代码仅仅支持RGB图像的预测，所有其它类型的图像都会转化成RGB\n#---------------------------------------------------------#\ndef cvtColor(image):\n if len(np.shape(image)) == 3 and np.shape(image)[2] == 3:\n return image \n else:\n image = image.convert('RGB')\n return image \n \n#---------------------------------------------------#\n# 对输入图像进行resize\n#---------------------------------------------------#\ndef resize_image(image, size, letterbox_image):\n iw, ih = image.size\n w, h = size\n if letterbox_image:\n scale = min(w/iw, h/ih)\n nw = int(iw*scale)\n nh = int(ih*scale)\n\n image = image.resize((nw,nh), Image.BICUBIC)\n new_image = Image.new('RGB', size, (128,128,128))\n new_image.paste(image, ((w-nw)//2, (h-nh)//2))\n else:\n new_image = image.resize((w, h), Image.BICUBIC)\n return new_image\n\n#---------------------------------------------------#\n# 获得类\n#---------------------------------------------------#\ndef get_classes(classes_path):\n with open(classes_path, encoding='utf-8') as f:\n class_names = f.readlines()\n class_names = [c.strip() for c in class_names]\n return class_names, len(class_names)\n\n#---------------------------------------------------#\n# 获得先验框\n#---------------------------------------------------#\ndef get_anchors(anchors_path):\n '''loads the anchors from a file'''\n with open(anchors_path, encoding='utf-8') as f:\n anchors = f.readline()\n anchors = [float(x) for x in anchors.split(',')]\n anchors = np.array(anchors).reshape(-1, 2)\n return anchors, len(anchors)\n\n#---------------------------------------------------#\n# 获得学习率\n#---------------------------------------------------#\ndef get_lr(optimizer):\n for param_group in optimizer.param_groups:\n return param_group['lr']\n\ndef preprocess_input(image):\n image /= 255.0\n return image\n" }, { "path": "utils/utils_bbox.py", "content": "import torch\nimport torch.nn as nn\nfrom torchvision.ops import nms\nimport numpy as np\n\nclass DecodeBox():\n def __init__(self, anchors, num_classes, input_shape, anchors_mask = [[6,7,8], [3,4,5], [0,1,2]]):\n super(DecodeBox, self).__init__()\n self.anchors = anchors\n self.num_classes = num_classes\n self.bbox_attrs = 5 + num_classes\n self.input_shape = input_shape\n #-----------------------------------------------------------#\n # 13x13的特征层对应的anchor是[142, 110],[192, 243],[459, 401]\n # 26x26的特征层对应的anchor是[36, 75],[76, 55],[72, 146]\n # 52x52的特征层对应的anchor是[12, 16],[19, 36],[40, 28]\n #-----------------------------------------------------------#\n self.anchors_mask = anchors_mask\n\n def decode_box(self, inputs):\n outputs = []\n for i, input in enumerate(inputs):\n #-----------------------------------------------#\n # 输入的input一共有三个，他们的shape分别是\n # batch_size, 255, 13, 13\n # batch_size, 255, 26, 26\n # batch_size, 255, 52, 52\n #-----------------------------------------------#\n batch_size = input.size(0)\n input_height = input.size(2)\n input_width = input.size(3)\n\n #-----------------------------------------------#\n # 输入为416x416时\n # stride_h = stride_w = 32、16、8\n #-----------------------------------------------#\n stride_h = self.input_shape[0] / input_height\n stride_w = self.input_shape[1] / input_width\n #-------------------------------------------------#\n # 此时获得的scaled_anchors大小是相对于特征层的\n #-------------------------------------------------#\n scaled_anchors = [(anchor_width / stride_w, anchor_height / stride_h) for anchor_width, anchor_height in self.anchors[self.anchors_mask[i]]]\n\n #-----------------------------------------------#\n # 输入的input一共有三个，他们的shape分别是\n # batch_size, 3, 13, 13, 85\n # batch_size, 3, 26, 26, 85\n # batch_size, 3, 52, 52, 85\n #-----------------------------------------------#\n prediction = input.view(batch_size, len(self.anchors_mask[i]),\n self.bbox_attrs, input_height, input_width).permute(0, 1, 3, 4, 2).contiguous()\n\n #-----------------------------------------------#\n # 先验框的中心位置的调整参数\n #-----------------------------------------------#\n x = torch.sigmoid(prediction[..., 0]) \n y = torch.sigmoid(prediction[..., 1])\n #-----------------------------------------------#\n # 先验框的宽高调整参数\n #-----------------------------------------------#\n w = prediction[..., 2]\n h = prediction[..., 3]\n #-----------------------------------------------#\n # 获得置信度，是否有物体\n #-----------------------------------------------#\n conf = torch.sigmoid(prediction[..., 4])\n #-----------------------------------------------#\n # 种类置信度\n #-----------------------------------------------#\n pred_cls = torch.sigmoid(prediction[..., 5:])\n\n FloatTensor = torch.cuda.FloatTensor if x.is_cuda else torch.FloatTensor\n LongTensor = torch.cuda.LongTensor if x.is_cuda else torch.LongTensor\n\n #----------------------------------------------------------#\n # 生成网格，先验框中心，网格左上角 \n # batch_size,3,13,13\n #----------------------------------------------------------#\n grid_x = torch.linspace(0, input_width - 1, input_width).repeat(input_height, 1).repeat(\n batch_size * len(self.anchors_mask[i]), 1, 1).view(x.shape).type(FloatTensor)\n grid_y = torch.linspace(0, input_height - 1, input_height).repeat(input_width, 1).t().repeat(\n batch_size * len(self.anchors_mask[i]), 1, 1).view(y.shape).type(FloatTensor)\n\n #----------------------------------------------------------#\n # 按照网格格式生成先验框的宽高\n # batch_size,3,13,13\n #----------------------------------------------------------#\n anchor_w = FloatTensor(scaled_anchors).index_select(1, LongTensor([0]))\n anchor_h = FloatTensor(scaled_anchors).index_select(1, LongTensor([1]))\n anchor_w = anchor_w.repeat(batch_size, 1).repeat(1, 1, input_height * input_width).view(w.shape)\n anchor_h = anchor_h.repeat(batch_size, 1).repeat(1, 1, input_height * input_width).view(h.shape)\n\n #----------------------------------------------------------#\n # 利用预测结果对先验框进行调整\n # 首先调整先验框的中心，从先验框中心向右下角偏移\n # 再调整先验框的宽高。\n #----------------------------------------------------------#\n pred_boxes = FloatTensor(prediction[..., :4].shape)\n pred_boxes[..., 0] = x.data + grid_x\n pred_boxes[..., 1] = y.data + grid_y\n pred_boxes[..., 2] = torch.exp(w.data) * anchor_w\n pred_boxes[..., 3] = torch.exp(h.data) * anchor_h\n\n #----------------------------------------------------------#\n # 将输出结果归一化成小数的形式\n #----------------------------------------------------------#\n _scale = torch.Tensor([input_width, input_height, input_width, input_height]).type(FloatTensor)\n output = torch.cat((pred_boxes.view(batch_size, -1, 4) / _scale,\n conf.view(batch_size, -1, 1), pred_cls.view(batch_size, -1, self.num_classes)), -1)\n outputs.append(output.data)\n return outputs\n\n def yolo_correct_boxes(self, box_xy, box_wh, input_shape, image_shape, letterbox_image):\n #-----------------------------------------------------------------#\n # 把y轴放前面是因为方便预测框和图像的宽高进行相乘\n #-----------------------------------------------------------------#\n box_yx = box_xy[..., ::-1]\n box_hw = box_wh[..., ::-1]\n input_shape = np.array(input_shape)\n image_shape = np.array(image_shape)\n\n if letterbox_image:\n #-----------------------------------------------------------------#\n # 这里求出来的offset是图像有效区域相对于图像左上角的偏移情况\n # new_shape指的是宽高缩放情况\n #-----------------------------------------------------------------#\n new_shape = np.round(image_shape * np.min(input_shape/image_shape))\n offset = (input_shape - new_shape)/2./input_shape\n scale = input_shape/new_shape\n\n box_yx = (box_yx - offset) * scale\n box_hw *= scale\n\n box_mins = box_yx - (box_hw / 2.)\n box_maxes = box_yx + (box_hw / 2.)\n boxes = np.concatenate([box_mins[..., 0:1], box_mins[..., 1:2], box_maxes[..., 0:1], box_maxes[..., 1:2]], axis=-1)\n boxes *= np.concatenate([image_shape, image_shape], axis=-1)\n return boxes\n\n def non_max_suppression(self, prediction, num_classes, input_shape, image_shape, letterbox_image, conf_thres=0.5, nms_thres=0.4):\n #----------------------------------------------------------#\n # 将预测结果的格式转换成左上角右下角的格式。\n # prediction [batch_size, num_anchors, 85]\n #----------------------------------------------------------#\n box_corner = prediction.new(prediction.shape)\n box_corner[:, :, 0] = prediction[:, :, 0] - prediction[:, :, 2] / 2\n box_corner[:, :, 1] = prediction[:, :, 1] - prediction[:, :, 3] / 2\n box_corner[:, :, 2] = prediction[:, :, 0] + prediction[:, :, 2] / 2\n box_corner[:, :, 3] = prediction[:, :, 1] + prediction[:, :, 3] / 2\n prediction[:, :, :4] = box_corner[:, :, :4]\n\n output = [None for _ in range(len(prediction))]\n for i, image_pred in enumerate(prediction):\n #----------------------------------------------------------#\n # 对种类预测部分取max。\n # class_conf [num_anchors, 1] 种类置信度\n # class_pred [num_anchors, 1] 种类\n #----------------------------------------------------------#\n class_conf, class_pred = torch.max(image_pred[:, 5:5 + num_classes], 1, keepdim=True)\n\n #----------------------------------------------------------#\n # 利用置信度进行第一轮筛选\n #----------------------------------------------------------#\n conf_mask = (image_pred[:, 4] * class_conf[:, 0] >= conf_thres).squeeze()\n\n #----------------------------------------------------------#\n # 根据置信度进行预测结果的筛选\n #----------------------------------------------------------#\n image_pred = image_pred[conf_mask]\n class_conf = class_conf[conf_mask]\n class_pred = class_pred[conf_mask]\n if not image_pred.size(0):\n continue\n #-------------------------------------------------------------------------#\n # detections [num_anchors, 7]\n # 7的内容为：x1, y1, x2, y2, obj_conf, class_conf, class_pred\n #-------------------------------------------------------------------------#\n detections = torch.cat((image_pred[:, :5], class_conf.float(), class_pred.float()), 1)\n\n #------------------------------------------#\n # 获得预测结果中包含的所有种类\n #------------------------------------------#\n unique_labels = detections[:, -1].cpu().unique()\n\n if prediction.is_cuda:\n unique_labels = unique_labels.cuda()\n detections = detections.cuda()\n\n for c in unique_labels:\n #------------------------------------------#\n # 获得某一类得分筛选后全部的预测结果\n #------------------------------------------#\n detections_class = detections[detections[:, -1] == c]\n\n #------------------------------------------#\n # 使用官方自带的非极大抑制会速度更快一些！\n #------------------------------------------#\n keep = nms(\n detections_class[:, :4],\n detections_class[:, 4] * detections_class[:, 5],\n nms_thres\n )\n max_detections = detections_class[keep]\n \n # # 按照存在物体的置信度排序\n # _, conf_sort_index = torch.sort(detections_class[:, 4]*detections_class[:, 5], descending=True)\n # detections_class = detections_class[conf_sort_index]\n # # 进行非极大抑制\n # max_detections = []\n # while detections_class.size(0):\n # # 取出这一类置信度最高的，一步一步往下判断，判断重合程度是否大于nms_thres，如果是则去除掉\n # max_detections.append(detections_class[0].unsqueeze(0))\n # if len(detections_class) == 1:\n # break\n # ious = bbox_iou(max_detections[-1], detections_class[1:])\n # detections_class = detections_class[1:][ious < nms_thres]\n # # 堆叠\n # max_detections = torch.cat(max_detections).data\n \n # Add max detections to outputs\n output[i] = max_detections if output[i] is None else torch.cat((output[i], max_detections))\n \n if output[i] is not None:\n output[i] = output[i].cpu().numpy()\n box_xy, box_wh = (output[i][:, 0:2] + output[i][:, 2:4])/2, output[i][:, 2:4] - output[i][:, 0:2]\n output[i][:, :4] = self.yolo_correct_boxes(box_xy, box_wh, input_shape, image_shape, letterbox_image)\n return output\n" }, { "path": "utils/utils_fit.py", "content": "import os\n\nimport torch\nfrom tqdm import tqdm\n\nfrom utils.utils import get_lr\n\n\ndef fit_one_epoch(model_train, model, yolo_loss, loss_history, optimizer, epoch, epoch_step, epoch_step_val, gen, gen_val, Epoch, cuda, fp16, scaler, save_period, save_dir, local_rank=0):\n loss = 0\n val_loss = 0\n\n if local_rank == 0:\n print('Start Train')\n pbar = tqdm(total=epoch_step,desc=f'Epoch {epoch + 1}/{Epoch}',postfix=dict,mininterval=0.3)\n model_train.train()\n for iteration, batch in enumerate(gen):\n if iteration >= epoch_step:\n break\n\n images, targets = batch[0], batch[1]\n with torch.no_grad():\n if cuda:\n images = images.cuda()\n targets = [ann.cuda() for ann in targets]\n #----------------------#\n # 清零梯度\n #----------------------#\n optimizer.zero_grad()\n if not fp16:\n #----------------------#\n # 前向传播\n #----------------------#\n outputs = model_train(images)\n\n loss_value_all = 0\n #----------------------#\n # 计算损失\n #----------------------#\n for l in range(len(outputs)):\n loss_item = yolo_loss(l, outputs[l], targets)\n loss_value_all += loss_item\n loss_value = loss_value_all\n\n #----------------------#\n # 反向传播\n #----------------------#\n loss_value.backward()\n optimizer.step()\n else:\n from torch.cuda.amp import autocast\n with autocast():\n #----------------------#\n # 前向传播\n #----------------------#\n outputs = model_train(images)\n\n loss_value_all = 0\n #----------------------#\n # 计算损失\n #----------------------#\n for l in range(len(outputs)):\n loss_item = yolo_loss(l, outputs[l], targets)\n loss_value_all += loss_item\n loss_value = loss_value_all\n\n #----------------------#\n # 反向传播\n #----------------------#\n scaler.scale(loss_value).backward()\n scaler.step(optimizer)\n scaler.update()\n\n loss += loss_value.item()\n \n if local_rank == 0:\n pbar.set_postfix(**{'loss' : loss / (iteration + 1), \n 'lr' : get_lr(optimizer)})\n pbar.update(1)\n\n if local_rank == 0:\n pbar.close()\n print('Finish Train')\n print('Start Validation')\n pbar = tqdm(total=epoch_step_val, desc=f'Epoch {epoch + 1}/{Epoch}',postfix=dict,mininterval=0.3)\n\n model_train.eval()\n for iteration, batch in enumerate(gen_val):\n if iteration >= epoch_step_val:\n break\n images, targets = batch[0], batch[1]\n with torch.no_grad():\n if cuda:\n images = images.cuda()\n targets = [ann.cuda() for ann in targets]\n #----------------------#\n # 清零梯度\n #----------------------#\n optimizer.zero_grad()\n #----------------------#\n # 前向传播\n #----------------------#\n outputs = model_train(images)\n\n loss_value_all = 0\n #----------------------#\n # 计算损失\n #----------------------#\n for l in range(len(outputs)):\n loss_item = yolo_loss(l, outputs[l], targets)\n loss_value_all += loss_item\n loss_value = loss_value_all\n\n val_loss += loss_value.item()\n if local_rank == 0:\n pbar.set_postfix(**{'val_loss': val_loss / (iteration + 1)})\n pbar.update(1)\n\n if local_rank == 0:\n pbar.close()\n print('Finish Validation')\n loss_history.append_loss(epoch + 1, loss / epoch_step, val_loss / epoch_step_val)\n print('Epoch:'+ str(epoch + 1) + '/' + str(Epoch))\n print('Total Loss: %.3f || Val Loss: %.3f ' % (loss / epoch_step, val_loss / epoch_step_val))\n if (epoch + 1) % save_period == 0 or epoch + 1 == Epoch:\n torch.save(model.state_dict(), os.path.join(save_dir, \"ep%03d-loss%.3f-val_loss%.3f.pth\" % (epoch + 1, loss / epoch_step, val_loss / epoch_step_val)))\n # 每次保存最后一个权重\n torch.save(model.state_dict(), os.path.join(save_dir, \"last.pth\" ))" }, { "path": "utils/utils_map.py", "content": "import glob\nimport json\nimport math\nimport operator\nimport os\nimport shutil\nimport sys\n\nimport cv2\nimport matplotlib.pyplot as plt\nimport numpy as np\n\n'''\n 0,0 ------> x (width)\n |\n | (Left,Top)\n | *_________\n | | |\n | |\n y |_________|\n (height) *\n (Right,Bottom)\n'''\n\ndef log_average_miss_rate(precision, fp_cumsum, num_images):\n \"\"\"\n log-average miss rate:\n Calculated by averaging miss rates at 9 evenly spaced FPPI points\n between 10e-2 and 10e0, in log-space.\n\n output:\n lamr | log-average miss rate\n mr | miss rate\n fppi | false positives per image\n\n references:\n [1] Dollar, Piotr, et al. \"Pedestrian Detection: An Evaluation of the\n State of the Art.\" Pattern Analysis and Machine Intelligence, IEEE\n Transactions on 34.4 (2012): 743 - 761.\n \"\"\"\n\n if precision.size == 0:\n lamr = 0\n mr = 1\n fppi = 0\n return lamr, mr, fppi\n\n fppi = fp_cumsum / float(num_images)\n mr = (1 - precision)\n\n fppi_tmp = np.insert(fppi, 0, -1.0)\n mr_tmp = np.insert(mr, 0, 1.0)\n\n ref = np.logspace(-2.0, 0.0, num = 9)\n for i, ref_i in enumerate(ref):\n j = np.where(fppi_tmp <= ref_i)[-1][-1]\n ref[i] = mr_tmp[j]\n\n lamr = math.exp(np.mean(np.log(np.maximum(1e-10, ref))))\n\n return lamr, mr, fppi\n\n\"\"\"\n throw error and exit\n\"\"\"\ndef error(msg):\n print(msg)\n sys.exit(0)\n\n\"\"\"\n check if the number is a float between 0.0 and 1.0\n\"\"\"\ndef is_float_between_0_and_1(value):\n try:\n val = float(value)\n if val > 0.0 and val < 1.0:\n return True\n else:\n return False\n except ValueError:\n return False\n\n\"\"\"\n Calculate the AP given the recall and precision array\n 1st) We compute a version of the measured precision/recall curve with\n precision monotonically decreasing\n 2nd) We compute the AP as the area under this curve by numerical integration.\n\"\"\"\ndef voc_ap(rec, prec):\n \"\"\"\n --- Official matlab code VOC2012---\n mrec=[0 ; rec ; 1];\n mpre=[0 ; prec ; 0];\n for i=numel(mpre)-1:-1:1\n mpre(i)=max(mpre(i),mpre(i+1));\n end\n i=find(mrec(2:end)~=mrec(1:end-1))+1;\n ap=sum((mrec(i)-mrec(i-1)).*mpre(i));\n \"\"\"\n rec.insert(0, 0.0) # insert 0.0 at begining of list\n rec.append(1.0) # insert 1.0 at end of list\n mrec = rec[:]\n prec.insert(0, 0.0) # insert 0.0 at begining of list\n prec.append(0.0) # insert 0.0 at end of list\n mpre = prec[:]\n \"\"\"\n This part makes the precision monotonically decreasing\n (goes from the end to the beginning)\n matlab: for i=numel(mpre)-1:-1:1\n mpre(i)=max(mpre(i),mpre(i+1));\n \"\"\"\n for i in range(len(mpre)-2, -1, -1):\n mpre[i] = max(mpre[i], mpre[i+1])\n \"\"\"\n This part creates a list of indexes where the recall changes\n matlab: i=find(mrec(2:end)~=mrec(1:end-1))+1;\n \"\"\"\n i_list = []\n for i in range(1, len(mrec)):\n if mrec[i] != mrec[i-1]:\n i_list.append(i) # if it was matlab would be i + 1\n \"\"\"\n The Average Precision (AP) is the area under the curve\n (numerical integration)\n matlab: ap=sum((mrec(i)-mrec(i-1)).*mpre(i));\n \"\"\"\n ap = 0.0\n for i in i_list:\n ap += ((mrec[i]-mrec[i-1])*mpre[i])\n return ap, mrec, mpre\n\n\n\"\"\"\n Convert the lines of a file to a list\n\"\"\"\ndef file_lines_to_list(path):\n # open txt file lines to a list\n with open(path) as f:\n content = f.readlines()\n # remove whitespace characters like `\\n` at the end of each line\n content = [x.strip() for x in content]\n return content\n\n\"\"\"\n Draws text in image\n\"\"\"\ndef draw_text_in_image(img, text, pos, color, line_width):\n font = cv2.FONT_HERSHEY_PLAIN\n fontScale = 1\n lineType = 1\n bottomLeftCornerOfText = pos\n cv2.putText(img, text,\n bottomLeftCornerOfText,\n font,\n fontScale,\n color,\n lineType)\n text_width, _ = cv2.getTextSize(text, font, fontScale, lineType)[0]\n return img, (line_width + text_width)\n\n\"\"\"\n Plot - adjust axes\n\"\"\"\ndef adjust_axes(r, t, fig, axes):\n # get text width for re-scaling\n bb = t.get_window_extent(renderer=r)\n text_width_inches = bb.width / fig.dpi\n # get axis width in inches\n current_fig_width = fig.get_figwidth()\n new_fig_width = current_fig_width + text_width_inches\n propotion = new_fig_width / current_fig_width\n # get axis limit\n x_lim = axes.get_xlim()\n axes.set_xlim([x_lim[0], x_lim[1]*propotion])\n\n\"\"\"\n Draw plot using Matplotlib\n\"\"\"\ndef draw_plot_func(dictionary, n_classes, window_title, plot_title, x_label, output_path, to_show, plot_color, true_p_bar):\n # sort the dictionary by decreasing value, into a list of tuples\n sorted_dic_by_value = sorted(dictionary.items(), key=operator.itemgetter(1))\n # unpacking the list of tuples into two lists\n sorted_keys, sorted_values = zip(*sorted_dic_by_value)\n # \n if true_p_bar != \"\":\n \"\"\"\n Special case to draw in:\n - green -> TP: True Positives (object detected and matches ground-truth)\n - red -> FP: False Positives (object detected but does not match ground-truth)\n - orange -> FN: False Negatives (object not detected but present in the ground-truth)\n \"\"\"\n fp_sorted = []\n tp_sorted = []\n for key in sorted_keys:\n fp_sorted.append(dictionary[key] - true_p_bar[key])\n tp_sorted.append(true_p_bar[key])\n plt.barh(range(n_classes), fp_sorted, align='center', color='crimson', label='False Positive')\n plt.barh(range(n_classes), tp_sorted, align='center', color='forestgreen', label='True Positive', left=fp_sorted)\n # add legend\n plt.legend(loc='lower right')\n \"\"\"\n Write number on side of bar\n \"\"\"\n fig = plt.gcf() # gcf - get current figure\n axes = plt.gca()\n r = fig.canvas.get_renderer()\n for i, val in enumerate(sorted_values):\n fp_val = fp_sorted[i]\n tp_val = tp_sorted[i]\n fp_str_val = \" \" + str(fp_val)\n tp_str_val = fp_str_val + \" \" + str(tp_val)\n # trick to paint multicolor with offset:\n # first paint everything and then repaint the first number\n t = plt.text(val, i, tp_str_val, color='forestgreen', va='center', fontweight='bold')\n plt.text(val, i, fp_str_val, color='crimson', va='center', fontweight='bold')\n if i == (len(sorted_values)-1): # largest bar\n adjust_axes(r, t, fig, axes)\n else:\n plt.barh(range(n_classes), sorted_values, color=plot_color)\n \"\"\"\n Write number on side of bar\n \"\"\"\n fig = plt.gcf() # gcf - get current figure\n axes = plt.gca()\n r = fig.canvas.get_renderer()\n for i, val in enumerate(sorted_values):\n str_val = \" \" + str(val) # add a space before\n if val < 1.0:\n str_val = \" {0:.2f}\".format(val)\n t = plt.text(val, i, str_val, color=plot_color, va='center', fontweight='bold')\n # re-set axes to show number inside the figure\n if i == (len(sorted_values)-1): # largest bar\n adjust_axes(r, t, fig, axes)\n # set window title\n fig.canvas.manager.set_window_title(window_title)\n # write classes in y axis\n tick_font_size = 12\n plt.yticks(range(n_classes), sorted_keys, fontsize=tick_font_size)\n \"\"\"\n Re-scale height accordingly\n \"\"\"\n init_height = fig.get_figheight()\n # comput the matrix height in points and inches\n dpi = fig.dpi\n height_pt = n_classes * (tick_font_size * 1.4) # 1.4 (some spacing)\n height_in = height_pt / dpi\n # compute the required figure height \n top_margin = 0.15 # in percentage of the figure height\n bottom_margin = 0.05 # in percentage of the figure height\n figure_height = height_in / (1 - top_margin - bottom_margin)\n # set new height\n if figure_height > init_height:\n fig.set_figheight(figure_height)\n\n # set plot title\n plt.title(plot_title, fontsize=14)\n # set axis titles\n # plt.xlabel('classes')\n plt.xlabel(x_label, fontsize='large')\n # adjust size of window\n fig.tight_layout()\n # save the plot\n fig.savefig(output_path)\n # show image\n if to_show:\n plt.show()\n # close the plot\n plt.close()\n\ndef get_map(MINOVERLAP, draw_plot, path = './map_out'):\n GT_PATH = os.path.join(path, 'ground-truth')\n DR_PATH = os.path.join(path, 'detection-results')\n IMG_PATH = os.path.join(path, 'images-optional')\n TEMP_FILES_PATH = os.path.join(path, '.temp_files')\n RESULTS_FILES_PATH = os.path.join(path, 'results')\n\n show_animation = True\n if os.path.exists(IMG_PATH): \n for dirpath, dirnames, files in os.walk(IMG_PATH):\n if not files:\n show_animation = False\n else:\n show_animation = False\n\n if not os.path.exists(TEMP_FILES_PATH):\n os.makedirs(TEMP_FILES_PATH)\n \n if os.path.exists(RESULTS_FILES_PATH):\n shutil.rmtree(RESULTS_FILES_PATH)\n if draw_plot:\n os.makedirs(os.path.join(RESULTS_FILES_PATH, \"AP\"))\n os.makedirs(os.path.join(RESULTS_FILES_PATH, \"F1\"))\n os.makedirs(os.path.join(RESULTS_FILES_PATH, \"Recall\"))\n os.makedirs(os.path.join(RESULTS_FILES_PATH, \"Precision\"))\n if show_animation:\n os.makedirs(os.path.join(RESULTS_FILES_PATH, \"images\", \"detections_one_by_one\"))\n\n ground_truth_files_list = glob.glob(GT_PATH + '/*.txt')\n if len(ground_truth_files_list) == 0:\n error(\"Error: No ground-truth files found!\")\n ground_truth_files_list.sort()\n gt_counter_per_class = {}\n counter_images_per_class = {}\n\n for txt_file in ground_truth_files_list:\n file_id = txt_file.split(\".txt\", 1)[0]\n file_id = os.path.basename(os.path.normpath(file_id))\n temp_path = os.path.join(DR_PATH, (file_id + \".txt\"))\n if not os.path.exists(temp_path):\n error_msg = \"Error. File not found: {}\\n\".format(temp_path)\n error(error_msg)\n lines_list = file_lines_to_list(txt_file)\n bounding_boxes = []\n is_difficult = False\n already_seen_classes = []\n for line in lines_list:\n try:\n if \"difficult\" in line:\n class_name, left, top, right, bottom, _difficult = line.split()\n is_difficult = True\n else:\n class_name, left, top, right, bottom = line.split()\n except:\n if \"difficult\" in line:\n line_split = line.split()\n _difficult = line_split[-1]\n bottom = line_split[-2]\n right = line_split[-3]\n top = line_split[-4]\n left = line_split[-5]\n class_name = \"\"\n for name in line_split[:-5]:\n class_name += name + \" \"\n class_name = class_name[:-1]\n is_difficult = True\n else:\n line_split = line.split()\n bottom = line_split[-1]\n right = line_split[-2]\n top = line_split[-3]\n left = line_split[-4]\n class_name = \"\"\n for name in line_split[:-4]:\n class_name += name + \" \"\n class_name = class_name[:-1]\n\n bbox = left + \" \" + top + \" \" + right + \" \" + bottom\n if is_difficult:\n bounding_boxes.append({\"class_name\":class_name, \"bbox\":bbox, \"used\":False, \"difficult\":True})\n is_difficult = False\n else:\n bounding_boxes.append({\"class_name\":class_name, \"bbox\":bbox, \"used\":False})\n if class_name in gt_counter_per_class:\n gt_counter_per_class[class_name] += 1\n else:\n gt_counter_per_class[class_name] = 1\n\n if class_name not in already_seen_classes:\n if class_name in counter_images_per_class:\n counter_images_per_class[class_name] += 1\n else:\n counter_images_per_class[class_name] = 1\n already_seen_classes.append(class_name)\n\n with open(TEMP_FILES_PATH + \"/\" + file_id + \"_ground_truth.json\", 'w') as outfile:\n json.dump(bounding_boxes, outfile)\n\n gt_classes = list(gt_counter_per_class.keys())\n gt_classes = sorted(gt_classes)\n n_classes = len(gt_classes)\n\n dr_files_list = glob.glob(DR_PATH + '/*.txt')\n dr_files_list.sort()\n for class_index, class_name in enumerate(gt_classes):\n bounding_boxes = []\n for txt_file in dr_files_list:\n file_id = txt_file.split(\".txt\",1)[0]\n file_id = os.path.basename(os.path.normpath(file_id))\n temp_path = os.path.join(GT_PATH, (file_id + \".txt\"))\n if class_index == 0:\n if not os.path.exists(temp_path):\n error_msg = \"Error. File not found: {}\\n\".format(temp_path)\n error(error_msg)\n lines = file_lines_to_list(txt_file)\n for line in lines:\n try:\n tmp_class_name, confidence, left, top, right, bottom = line.split()\n except:\n line_split = line.split()\n bottom = line_split[-1]\n right = line_split[-2]\n top = line_split[-3]\n left = line_split[-4]\n confidence = line_split[-5]\n tmp_class_name = \"\"\n for name in line_split[:-5]:\n tmp_class_name += name + \" \"\n tmp_class_name = tmp_class_name[:-1]\n\n if tmp_class_name == class_name:\n bbox = left + \" \" + top + \" \" + right + \" \" +bottom\n bounding_boxes.append({\"confidence\":confidence, \"file_id\":file_id, \"bbox\":bbox})\n\n bounding_boxes.sort(key=lambda x:float(x['confidence']), reverse=True)\n with open(TEMP_FILES_PATH + \"/\" + class_name + \"_dr.json\", 'w') as outfile:\n json.dump(bounding_boxes, outfile)\n\n sum_AP = 0.0\n ap_dictionary = {}\n lamr_dictionary = {}\n with open(RESULTS_FILES_PATH + \"/results.txt\", 'w') as results_file:\n results_file.write(\"# AP and precision/recall per class\\n\")\n count_true_positives = {}\n\n for class_index, class_name in enumerate(gt_classes):\n count_true_positives[class_name] = 0\n dr_file = TEMP_FILES_PATH + \"/\" + class_name + \"_dr.json\"\n dr_data = json.load(open(dr_file))\n\n nd = len(dr_data)\n tp = [0] * nd\n fp = [0] * nd\n score = [0] * nd\n score05_idx = 0\n for idx, detection in enumerate(dr_data):\n file_id = detection[\"file_id\"]\n score[idx] = float(detection[\"confidence\"])\n if score[idx] > 0.5:\n score05_idx = idx\n\n if show_animation:\n ground_truth_img = glob.glob1(IMG_PATH, file_id + \".*\")\n if len(ground_truth_img) == 0:\n error(\"Error. Image not found with id: \" + file_id)\n elif len(ground_truth_img) > 1:\n error(\"Error. Multiple image with id: \" + file_id)\n else:\n img = cv2.imread(IMG_PATH + \"/\" + ground_truth_img[0])\n img_cumulative_path = RESULTS_FILES_PATH + \"/images/\" + ground_truth_img[0]\n if os.path.isfile(img_cumulative_path):\n img_cumulative = cv2.imread(img_cumulative_path)\n else:\n img_cumulative = img.copy()\n bottom_border = 60\n BLACK = [0, 0, 0]\n img = cv2.copyMakeBorder(img, 0, bottom_border, 0, 0, cv2.BORDER_CONSTANT, value=BLACK)\n\n gt_file = TEMP_FILES_PATH + \"/\" + file_id + \"_ground_truth.json\"\n ground_truth_data = json.load(open(gt_file))\n ovmax = -1\n gt_match = -1\n bb = [float(x) for x in detection[\"bbox\"].split()]\n for obj in ground_truth_data:\n if obj[\"class_name\"] == class_name:\n bbgt = [ float(x) for x in obj[\"bbox\"].split() ]\n bi = [max(bb[0],bbgt[0]), max(bb[1],bbgt[1]), min(bb[2],bbgt[2]), min(bb[3],bbgt[3])]\n iw = bi[2] - bi[0] + 1\n ih = bi[3] - bi[1] + 1\n if iw > 0 and ih > 0:\n ua = (bb[2] - bb[0] + 1) * (bb[3] - bb[1] + 1) + (bbgt[2] - bbgt[0]\n + 1) * (bbgt[3] - bbgt[1] + 1) - iw * ih\n ov = iw * ih / ua\n if ov > ovmax:\n ovmax = ov\n gt_match = obj\n\n if show_animation:\n status = \"NO MATCH FOUND!\" \n \n min_overlap = MINOVERLAP\n if ovmax >= min_overlap:\n if \"difficult\" not in gt_match:\n if not bool(gt_match[\"used\"]):\n tp[idx] = 1\n gt_match[\"used\"] = True\n count_true_positives[class_name] += 1\n with open(gt_file, 'w') as f:\n f.write(json.dumps(ground_truth_data))\n if show_animation:\n status = \"MATCH!\"\n else:\n fp[idx] = 1\n if show_animation:\n status = \"REPEATED MATCH!\"\n else:\n fp[idx] = 1\n if ovmax > 0:\n status = \"INSUFFICIENT OVERLAP\"\n\n \"\"\"\n Draw image to show animation\n \"\"\"\n if show_animation:\n height, widht = img.shape[:2]\n white = (255,255,255)\n light_blue = (255,200,100)\n green = (0,255,0)\n light_red = (30,30,255)\n margin = 10\n # 1nd line\n v_pos = int(height - margin - (bottom_border / 2.0))\n text = \"Image: \" + ground_truth_img[0] + \" \"\n img, line_width = draw_text_in_image(img, text, (margin, v_pos), white, 0)\n text = \"Class [\" + str(class_index) + \"/\" + str(n_classes) + \"]: \" + class_name + \" \"\n img, line_width = draw_text_in_image(img, text, (margin + line_width, v_pos), light_blue, line_width)\n if ovmax != -1:\n color = light_red\n if status == \"INSUFFICIENT OVERLAP\":\n text = \"IoU: {0:.2f}% \".format(ovmax*100) + \"< {0:.2f}% \".format(min_overlap*100)\n else:\n text = \"IoU: {0:.2f}% \".format(ovmax*100) + \">= {0:.2f}% \".format(min_overlap*100)\n color = green\n img, _ = draw_text_in_image(img, text, (margin + line_width, v_pos), color, line_width)\n # 2nd line\n v_pos += int(bottom_border / 2.0)\n rank_pos = str(idx+1)\n text = \"Detection #rank: \" + rank_pos + \" confidence: {0:.2f}% \".format(float(detection[\"confidence\"])*100)\n img, line_width = draw_text_in_image(img, text, (margin, v_pos), white, 0)\n color = light_red\n if status == \"MATCH!\":\n color = green\n text = \"Result: \" + status + \" \"\n img, line_width = draw_text_in_image(img, text, (margin + line_width, v_pos), color, line_width)\n\n font = cv2.FONT_HERSHEY_SIMPLEX\n if ovmax > 0: \n bbgt = [ int(round(float(x))) for x in gt_match[\"bbox\"].split() ]\n cv2.rectangle(img,(bbgt[0],bbgt[1]),(bbgt[2],bbgt[3]),light_blue,2)\n cv2.rectangle(img_cumulative,(bbgt[0],bbgt[1]),(bbgt[2],bbgt[3]),light_blue,2)\n cv2.putText(img_cumulative, class_name, (bbgt[0],bbgt[1] - 5), font, 0.6, light_blue, 1, cv2.LINE_AA)\n bb = [int(i) for i in bb]\n cv2.rectangle(img,(bb[0],bb[1]),(bb[2],bb[3]),color,2)\n cv2.rectangle(img_cumulative,(bb[0],bb[1]),(bb[2],bb[3]),color,2)\n cv2.putText(img_cumulative, class_name, (bb[0],bb[1] - 5), font, 0.6, color, 1, cv2.LINE_AA)\n\n cv2.imshow(\"Animation\", img)\n cv2.waitKey(20) \n output_img_path = RESULTS_FILES_PATH + \"/images/detections_one_by_one/\" + class_name + \"_detection\" + str(idx) + \".jpg\"\n cv2.imwrite(output_img_path, img)\n cv2.imwrite(img_cumulative_path, img_cumulative)\n\n cumsum = 0\n for idx, val in enumerate(fp):\n fp[idx] += cumsum\n cumsum += val\n \n cumsum = 0\n for idx, val in enumerate(tp):\n tp[idx] += cumsum\n cumsum += val\n\n rec = tp[:]\n for idx, val in enumerate(tp):\n rec[idx] = float(tp[idx]) / np.maximum(gt_counter_per_class[class_name], 1)\n\n prec = tp[:]\n for idx, val in enumerate(tp):\n prec[idx] = float(tp[idx]) / np.maximum((fp[idx] + tp[idx]), 1)\n\n ap, mrec, mprec = voc_ap(rec[:], prec[:])\n F1 = np.array(rec)*np.array(prec)*2 / np.where((np.array(prec)+np.array(rec))==0, 1, (np.array(prec)+np.array(rec)))\n\n sum_AP += ap\n text = \"{0:.2f}%\".format(ap*100) + \" = \" + class_name + \" AP \" #class_name + \" AP = {0:.2f}%\".format(ap*100)\n\n if len(prec)>0:\n F1_text = \"{0:.2f}\".format(F1[score05_idx]) + \" = \" + class_name + \" F1 \"\n Recall_text = \"{0:.2f}%\".format(rec[score05_idx]*100) + \" = \" + class_name + \" Recall \"\n Precision_text = \"{0:.2f}%\".format(prec[score05_idx]*100) + \" = \" + class_name + \" Precision \"\n else:\n F1_text = \"0.00\" + \" = \" + class_name + \" F1 \" \n Recall_text = \"0.00%\" + \" = \" + class_name + \" Recall \" \n Precision_text = \"0.00%\" + \" = \" + class_name + \" Precision \" \n\n rounded_prec = [ '%.2f' % elem for elem in prec ]\n rounded_rec = [ '%.2f' % elem for elem in rec ]\n results_file.write(text + \"\\n Precision: \" + str(rounded_prec) + \"\\n Recall :\" + str(rounded_rec) + \"\\n\\n\")\n if len(prec)>0:\n print(text + \"\\t||\\tscore_threhold=0.5 : \" + \"F1=\" + \"{0:.2f}\".format(F1[score05_idx])\\\n + \" ; Recall=\" + \"{0:.2f}%\".format(rec[score05_idx]*100) + \" ; Precision=\" + \"{0:.2f}%\".format(prec[score05_idx]*100))\n else:\n print(text + \"\\t||\\tscore_threhold=0.5 : F1=0.00% ; Recall=0.00% ; Precision=0.00%\")\n ap_dictionary[class_name] = ap\n\n n_images = counter_images_per_class[class_name]\n lamr, mr, fppi = log_average_miss_rate(np.array(rec), np.array(fp), n_images)\n lamr_dictionary[class_name] = lamr\n\n if draw_plot:\n plt.plot(rec, prec, '-o')\n area_under_curve_x = mrec[:-1] + [mrec[-2]] + [mrec[-1]]\n area_under_curve_y = mprec[:-1] + [0.0] + [mprec[-1]]\n plt.fill_between(area_under_curve_x, 0, area_under_curve_y, alpha=0.2, edgecolor='r')\n\n fig = plt.gcf()\n fig.canvas.manager.set_window_title('AP ' + class_name)\n\n plt.title('class: ' + text)\n plt.xlabel('Recall')\n plt.ylabel('Precision')\n axes = plt.gca()\n axes.set_xlim([0.0,1.0])\n axes.set_ylim([0.0,1.05]) \n fig.savefig(RESULTS_FILES_PATH + \"/AP/\" + class_name + \".png\")\n plt.cla()\n\n plt.plot(score, F1, \"-\", color='orangered')\n plt.title('class: ' + F1_text + \"\\nscore_threhold=0.5\")\n plt.xlabel('Score_Threhold')\n plt.ylabel('F1')\n axes = plt.gca()\n axes.set_xlim([0.0,1.0])\n axes.set_ylim([0.0,1.05])\n fig.savefig(RESULTS_FILES_PATH + \"/F1/\" + class_name + \".png\")\n plt.cla()\n\n plt.plot(score, rec, \"-H\", color='gold')\n plt.title('class: ' + Recall_text + \"\\nscore_threhold=0.5\")\n plt.xlabel('Score_Threhold')\n plt.ylabel('Recall')\n axes = plt.gca()\n axes.set_xlim([0.0,1.0])\n axes.set_ylim([0.0,1.05])\n fig.savefig(RESULTS_FILES_PATH + \"/Recall/\" + class_name + \".png\")\n plt.cla()\n\n plt.plot(score, prec, \"-s\", color='palevioletred')\n plt.title('class: ' + Precision_text + \"\\nscore_threhold=0.5\")\n plt.xlabel('Score_Threhold')\n plt.ylabel('Precision')\n axes = plt.gca()\n axes.set_xlim([0.0,1.0])\n axes.set_ylim([0.0,1.05])\n fig.savefig(RESULTS_FILES_PATH + \"/Precision/\" + class_name + \".png\")\n plt.cla()\n \n if show_animation:\n cv2.destroyAllWindows()\n\n results_file.write(\"\\n# mAP of all classes\\n\")\n mAP = sum_AP / n_classes\n text = \"mAP = {0:.2f}%\".format(mAP*100)\n results_file.write(text + \"\\n\")\n print(text)\n\n shutil.rmtree(TEMP_FILES_PATH)\n\n \"\"\"\n Count total of detection-results\n \"\"\"\n det_counter_per_class = {}\n for txt_file in dr_files_list:\n lines_list = file_lines_to_list(txt_file)\n for line in lines_list:\n class_name = line.split()[0]\n if class_name in det_counter_per_class:\n det_counter_per_class[class_name] += 1\n else:\n det_counter_per_class[class_name] = 1\n dr_classes = list(det_counter_per_class.keys())\n\n \"\"\"\n Write number of ground-truth objects per class to results.txt\n \"\"\"\n with open(RESULTS_FILES_PATH + \"/results.txt\", 'a') as results_file:\n results_file.write(\"\\n# Number of ground-truth objects per class\\n\")\n for class_name in sorted(gt_counter_per_class):\n results_file.write(class_name + \": \" + str(gt_counter_per_class[class_name]) + \"\\n\")\n\n \"\"\"\n Finish counting true positives\n \"\"\"\n for class_name in dr_classes:\n if class_name not in gt_classes:\n count_true_positives[class_name] = 0\n\n \"\"\"\n Write number of detected objects per class to results.txt\n \"\"\"\n with open(RESULTS_FILES_PATH + \"/results.txt\", 'a') as results_file:\n results_file.write(\"\\n# Number of detected objects per class\\n\")\n for class_name in sorted(dr_classes):\n n_det = det_counter_per_class[class_name]\n text = class_name + \": \" + str(n_det)\n text += \" (tp:\" + str(count_true_positives[class_name]) + \"\"\n text += \", fp:\" + str(n_det - count_true_positives[class_name]) + \")\\n\"\n results_file.write(text)\n\n \"\"\"\n Plot the total number of occurences of each class in the ground-truth\n \"\"\"\n if draw_plot:\n window_title = \"ground-truth-info\"\n plot_title = \"ground-truth\\n\"\n plot_title += \"(\" + str(len(ground_truth_files_list)) + \" files and \" + str(n_classes) + \" classes)\"\n x_label = \"Number of objects per class\"\n output_path = RESULTS_FILES_PATH + \"/ground-truth-info.png\"\n to_show = False\n plot_color = 'forestgreen'\n draw_plot_func(\n gt_counter_per_class,\n n_classes,\n window_title,\n plot_title,\n x_label,\n output_path,\n to_show,\n plot_color,\n '',\n )\n\n # \"\"\"\n # Plot the total number of occurences of each class in the \"detection-results\" folder\n # \"\"\"\n # if draw_plot:\n # window_title = \"detection-results-info\"\n # # Plot title\n # plot_title = \"detection-results\\n\"\n # plot_title += \"(\" + str(len(dr_files_list)) + \" files and \"\n # count_non_zero_values_in_dictionary = sum(int(x) > 0 for x in list(det_counter_per_class.values()))\n # plot_title += str(count_non_zero_values_in_dictionary) + \" detected classes)\"\n # # end Plot title\n # x_label = \"Number of objects per class\"\n # output_path = RESULTS_FILES_PATH + \"/detection-results-info.png\"\n # to_show = False\n # plot_color = 'forestgreen'\n # true_p_bar = count_true_positives\n # draw_plot_func(\n # det_counter_per_class,\n # len(det_counter_per_class),\n # window_title,\n # plot_title,\n # x_label,\n # output_path,\n # to_show,\n # plot_color,\n # true_p_bar\n # )\n\n \"\"\"\n Draw log-average miss rate plot (Show lamr of all classes in decreasing order)\n \"\"\"\n if draw_plot:\n window_title = \"lamr\"\n plot_title = \"log-average miss rate\"\n x_label = \"log-average miss rate\"\n output_path = RESULTS_FILES_PATH + \"/lamr.png\"\n to_show = False\n plot_color = 'royalblue'\n draw_plot_func(\n lamr_dictionary,\n n_classes,\n window_title,\n plot_title,\n x_label,\n output_path,\n to_show,\n plot_color,\n \"\"\n )\n\n \"\"\"\n Draw mAP plot (Show AP's of all classes in decreasing order)\n \"\"\"\n if draw_plot:\n window_title = \"mAP\"\n plot_title = \"mAP = {0:.2f}%\".format(mAP*100)\n x_label = \"Average Precision\"\n output_path = RESULTS_FILES_PATH + \"/mAP.png\"\n to_show = True\n plot_color = 'royalblue'\n draw_plot_func(\n ap_dictionary,\n n_classes,\n window_title,\n plot_title,\n x_label,\n output_path,\n to_show,\n plot_color,\n \"\"\n )\n\ndef preprocess_gt(gt_path, class_names):\n image_ids = os.listdir(gt_path)\n results = {}\n\n images = []\n bboxes = []\n for i, image_id in enumerate(image_ids):\n lines_list = file_lines_to_list(os.path.join(gt_path, image_id))\n boxes_per_image = []\n image = {}\n image_id = os.path.splitext(image_id)[0]\n image['file_name'] = image_id + '.jpg'\n image['width'] = 1\n image['height'] = 1\n #-----------------------------------------------------------------#\n # 感谢 多学学英语吧 的提醒\n # 解决了'Results do not correspond to current coco set'问题\n #-----------------------------------------------------------------#\n image['id'] = str(image_id)\n\n for line in lines_list:\n difficult = 0 \n if \"difficult\" in line:\n line_split = line.split()\n left, top, right, bottom, _difficult = line_split[-5:]\n class_name = \"\"\n for name in line_split[:-5]:\n class_name += name + \" \"\n class_name = class_name[:-1]\n difficult = 1\n else:\n line_split = line.split()\n left, top, right, bottom = line_split[-4:]\n class_name = \"\"\n for name in line_split[:-4]:\n class_name += name + \" \"\n class_name = class_name[:-1]\n \n left, top, right, bottom = float(left), float(top), float(right), float(bottom)\n cls_id = class_names.index(class_name) + 1\n bbox = [left, top, right - left, bottom - top, difficult, str(image_id), cls_id, (right - left) * (bottom - top) - 10.0]\n boxes_per_image.append(bbox)\n images.append(image)\n bboxes.extend(boxes_per_image)\n results['images'] = images\n\n categories = []\n for i, cls in enumerate(class_names):\n category = {}\n category['supercategory'] = cls\n category['name'] = cls\n category['id'] = i + 1\n categories.append(category)\n results['categories'] = categories\n\n annotations = []\n for i, box in enumerate(bboxes):\n annotation = {}\n annotation['area'] = box[-1]\n annotation['category_id'] = box[-2]\n annotation['image_id'] = box[-3]\n annotation['iscrowd'] = box[-4]\n annotation['bbox'] = box[:4]\n annotation['id'] = i\n annotations.append(annotation)\n results['annotations'] = annotations\n return results\n\ndef preprocess_dr(dr_path, class_names):\n image_ids = os.listdir(dr_path)\n results = []\n for image_id in image_ids:\n lines_list = file_lines_to_list(os.path.join(dr_path, image_id))\n image_id = os.path.splitext(image_id)[0]\n for line in lines_list:\n line_split = line.split()\n confidence, left, top, right, bottom = line_split[-5:]\n class_name = \"\"\n for name in line_split[:-5]:\n class_name += name + \" \"\n class_name = class_name[:-1]\n left, top, right, bottom = float(left), float(top), float(right), float(bottom)\n result = {}\n result[\"image_id\"] = str(image_id)\n result[\"category_id\"] = class_names.index(class_name) + 1\n result[\"bbox\"] = [left, top, right - left, bottom - top]\n result[\"score\"] = float(confidence)\n results.append(result)\n return results\n \ndef get_coco_map(class_names, path):\n from pycocotools.coco import COCO\n from pycocotools.cocoeval import COCOeval\n \n GT_PATH = os.path.join(path, 'ground-truth')\n DR_PATH = os.path.join(path, 'detection-results')\n COCO_PATH = os.path.join(path, 'coco_eval')\n\n if not os.path.exists(COCO_PATH):\n os.makedirs(COCO_PATH)\n\n GT_JSON_PATH = os.path.join(COCO_PATH, 'instances_gt.json')\n DR_JSON_PATH = os.path.join(COCO_PATH, 'instances_dr.json')\n\n with open(GT_JSON_PATH, \"w\") as f:\n results_gt = preprocess_gt(GT_PATH, class_names)\n json.dump(results_gt, f, indent=4)\n\n with open(DR_JSON_PATH, \"w\") as f:\n results_dr = preprocess_dr(DR_PATH, class_names)\n json.dump(results_dr, f, indent=4)\n\n cocoGt = COCO(GT_JSON_PATH)\n cocoDt = cocoGt.loadRes(DR_JSON_PATH)\n cocoEval = COCOeval(cocoGt, cocoDt, 'bbox') \n cocoEval.evaluate()\n cocoEval.accumulate()\n cocoEval.summarize()\n" }, { "path": "utils_coco/coco_annotation.py", "content": "#-------------------------------------------------------#\n# 用于处理COCO数据集，根据json文件生成txt文件用于训练\n#-------------------------------------------------------#\nimport json\nimport os\nfrom collections import defaultdict\n\n#-------------------------------------------------------#\n# 指向了COCO训练集与验证集图片的路径\n#-------------------------------------------------------#\ntrain_datasets_path = \"coco_dataset/train2017\"\nval_datasets_path = \"coco_dataset/val2017\"\n\n#-------------------------------------------------------#\n# 指向了COCO训练集与验证集标签的路径\n#-------------------------------------------------------#\ntrain_annotation_path = \"coco_dataset/annotations/instances_train2017.json\"\nval_annotation_path = \"coco_dataset/annotations/instances_val2017.json\"\n\n#-------------------------------------------------------#\n# 生成的txt文件路径\n#-------------------------------------------------------#\ntrain_output_path = \"coco_train.txt\"\nval_output_path = \"coco_val.txt\"\n\nif __name__ == \"__main__\":\n name_box_id = defaultdict(list)\n id_name = dict()\n f = open(train_annotation_path, encoding='utf-8')\n data = json.load(f)\n\n annotations = data['annotations']\n for ant in annotations:\n id = ant['image_id']\n name = os.path.join(train_datasets_path, '%012d.jpg' % id)\n cat = ant['category_id']\n if cat >= 1 and cat <= 11:\n cat = cat - 1\n elif cat >= 13 and cat <= 25:\n cat = cat - 2\n elif cat >= 27 and cat <= 28:\n cat = cat - 3\n elif cat >= 31 and cat <= 44:\n cat = cat - 5\n elif cat >= 46 and cat <= 65:\n cat = cat - 6\n elif cat == 67:\n cat = cat - 7\n elif cat == 70:\n cat = cat - 9\n elif cat >= 72 and cat <= 82:\n cat = cat - 10\n elif cat >= 84 and cat <= 90:\n cat = cat - 11\n name_box_id[name].append([ant['bbox'], cat])\n\n f = open(train_output_path, 'w')\n for key in name_box_id.keys():\n f.write(key)\n box_infos = name_box_id[key]\n for info in box_infos:\n x_min = int(info[0][0])\n y_min = int(info[0][1])\n x_max = x_min + int(info[0][2])\n y_max = y_min + int(info[0][3])\n\n box_info = \" %d,%d,%d,%d,%d\" % (\n x_min, y_min, x_max, y_max, int(info[1]))\n f.write(box_info)\n f.write('\\n')\n f.close()\n\n name_box_id = defaultdict(list)\n id_name = dict()\n f = open(val_annotation_path, encoding='utf-8')\n data = json.load(f)\n\n annotations = data['annotations']\n for ant in annotations:\n id = ant['image_id']\n name = os.path.join(val_datasets_path, '%012d.jpg' % id)\n cat = ant['category_id']\n if cat >= 1 and cat <= 11:\n cat = cat - 1\n elif cat >= 13 and cat <= 25:\n cat = cat - 2\n elif cat >= 27 and cat <= 28:\n cat = cat - 3\n elif cat >= 31 and cat <= 44:\n cat = cat - 5\n elif cat >= 46 and cat <= 65:\n cat = cat - 6\n elif cat == 67:\n cat = cat - 7\n elif cat == 70:\n cat = cat - 9\n elif cat >= 72 and cat <= 82:\n cat = cat - 10\n elif cat >= 84 and cat <= 90:\n cat = cat - 11\n name_box_id[name].append([ant['bbox'], cat])\n\n f = open(val_output_path, 'w')\n for key in name_box_id.keys():\n f.write(key)\n box_infos = name_box_id[key]\n for info in box_infos:\n x_min = int(info[0][0])\n y_min = int(info[0][1])\n x_max = x_min + int(info[0][2])\n y_max = y_min + int(info[0][3])\n\n box_info = \" %d,%d,%d,%d,%d\" % (\n x_min, y_min, x_max, y_max, int(info[1]))\n f.write(box_info)\n f.write('\\n')\n f.close()\n" }, { "path": "utils_coco/get_map_coco.py", "content": "import json\nimport os\n\nimport numpy as np\nimport torch\nfrom PIL import Image\nfrom pycocotools.coco import COCO\nfrom pycocotools.cocoeval import COCOeval\nfrom tqdm import tqdm\n\nfrom utils.utils import cvtColor, preprocess_input, resize_image\nfrom yolo import YOLO\n\n#---------------------------------------------------------------------------#\n# map_mode用于指定该文件运行时计算的内容\n# map_mode为0代表整个map计算流程，包括获得预测结果、计算map。\n# map_mode为1代表仅仅获得预测结果。\n# map_mode为2代表仅仅获得计算map。\n#---------------------------------------------------------------------------#\nmap_mode = 0\n#-------------------------------------------------------#\n# 指向了验证集标签与图片路径\n#-------------------------------------------------------#\ncocoGt_path = 'coco_dataset/annotations/instances_val2017.json'\ndataset_img_path = 'coco_dataset/val2017'\n#-------------------------------------------------------#\n# 结果输出的文件夹，默认为map_out\n#-------------------------------------------------------#\ntemp_save_path = 'map_out/coco_eval'\n\nclass mAP_YOLO(YOLO):\n #---------------------------------------------------#\n # 检测图片\n #---------------------------------------------------#\n def detect_image(self, image_id, image, results):\n #---------------------------------------------------#\n # 计算输入图片的高和宽\n #---------------------------------------------------#\n image_shape = np.array(np.shape(image)[0:2])\n #---------------------------------------------------------#\n # 在这里将图像转换成RGB图像，防止灰度图在预测时报错。\n # 代码仅仅支持RGB图像的预测，所有其它类型的图像都会转化成RGB\n #---------------------------------------------------------#\n image = cvtColor(image)\n #---------------------------------------------------------#\n # 给图像增加灰条，实现不失真的resize\n # 也可以直接resize进行识别\n #---------------------------------------------------------#\n image_data = resize_image(image, (self.input_shape[1],self.input_shape[0]), self.letterbox_image)\n #---------------------------------------------------------#\n # 添加上batch_size维度\n #---------------------------------------------------------#\n image_data = np.expand_dims(np.transpose(preprocess_input(np.array(image_data, dtype='float32')), (2, 0, 1)), 0)\n\n with torch.no_grad():\n images = torch.from_numpy(image_data)\n if self.cuda:\n images = images.cuda()\n #---------------------------------------------------------#\n # 将图像输入网络当中进行预测！\n #---------------------------------------------------------#\n outputs = self.net(images)\n outputs = self.bbox_util.decode_box(outputs)\n #---------------------------------------------------------#\n # 将预测框进行堆叠，然后进行非极大抑制\n #---------------------------------------------------------#\n outputs = self.bbox_util.non_max_suppression(torch.cat(outputs, 1), self.num_classes, self.input_shape, \n image_shape, self.letterbox_image, conf_thres = self.confidence, nms_thres = self.nms_iou)\n \n if outputs[0] is None: \n return results\n\n top_label = np.array(outputs[0][:, 6], dtype = 'int32')\n top_conf = outputs[0][:, 4] * outputs[0][:, 5]\n top_boxes = outputs[0][:, :4]\n\n for i, c in enumerate(top_label):\n result = {}\n top, left, bottom, right = top_boxes[i]\n\n result[\"image_id\"] = int(image_id)\n result[\"category_id\"] = clsid2catid[c]\n result[\"bbox\"] = [float(left),float(top),float(right-left),float(bottom-top)]\n result[\"score\"] = float(top_conf[i])\n results.append(result)\n return results\n\nif __name__ == \"__main__\":\n if not os.path.exists(temp_save_path):\n os.makedirs(temp_save_path)\n\n cocoGt = COCO(cocoGt_path)\n ids = list(cocoGt.imgToAnns.keys())\n clsid2catid = cocoGt.getCatIds()\n\n if map_mode == 0 or map_mode == 1:\n yolo = mAP_YOLO(confidence = 0.001, nms_iou = 0.65)\n\n with open(os.path.join(temp_save_path, 'eval_results.json'),\"w\") as f:\n results = []\n for image_id in tqdm(ids):\n image_path = os.path.join(dataset_img_path, cocoGt.loadImgs(image_id)[0]['file_name'])\n image = Image.open(image_path)\n results = yolo.detect_image(image_id, image, results)\n json.dump(results, f)\n\n if map_mode == 0 or map_mode == 2:\n cocoDt = cocoGt.loadRes(os.path.join(temp_save_path, 'eval_results.json'))\n cocoEval = COCOeval(cocoGt, cocoDt, 'bbox') \n cocoEval.evaluate()\n cocoEval.accumulate()\n cocoEval.summarize()\n print(\"Get map done.\")\n" }, { "path": "voc_annotation.py", "content": "import os\nimport random\nimport xml.etree.ElementTree as ET\nfrom get_yaml import get_config\nfrom utils.utils import get_classes\n\n#--------------------------------------------------------------------------------------------------------------------------------#\n# annotation_mode用于指定该文件运行时计算的内容\n# annotation_mode为0代表整个标签处理过程，包括获得VOCdevkit/VOC2007/ImageSets里面的txt以及训练用的2007_train.txt、2007_val.txt\n# annotation_mode为1代表获得VOCdevkit/VOC2007/ImageSets里面的txt\n# annotation_mode为2代表获得训练用的2007_train.txt、2007_val.txt\n#--------------------------------------------------------------------------------------------------------------------------------#\nannotation_mode = 0\n#-------------------------------------------------------------------#\n# 必须要修改，用于生成2007_train.txt、2007_val.txt的目标信息\n# 与训练和预测所用的classes_path一致即可\n# 如果生成的2007_train.txt里面没有目标信息\n# 那么就是因为classes没有设定正确\n# 仅在annotation_mode为0和2的时候有效\n#-------------------------------------------------------------------#\n# classes_path = 'model_data/gesture_classes.txt'\n#--------------------------------------------------------------------------------------------------------------------------------#\n# trainval_percent用于指定(训练集+验证集)与测试集的比例，默认情况下 (训练集+验证集):测试集 = 9:1\n# train_percent用于指定(训练集+验证集)中训练集与验证集的比例，默认情况下 训练集:验证集 = 9:1 \n# 仅在annotation_mode为0和1的时候有效\n#--------------------------------------------------------------------------------------------------------------------------------#\ntrainval_percent = 1\ntrain_percent = 0.9\n#-------------------------------------------------------#\n# 指向VOC数据集所在的文件夹\n# 默认指向根目录下的VOC数据集\n#-------------------------------------------------------#\nVOCdevkit_path = 'VOCdevkit'\n\nVOCdevkit_sets = [('2007', 'train'), ('2007', 'val')]\n# classes, _ = get_classes(classes_path)\nconfig = get_config()\nclasses = config['classes']\ndef convert_annotation(year, image_id, list_file):\n in_file = open(os.path.join(VOCdevkit_path, 'VOC%s/Annotations/%s.xml'%(year, image_id)), encoding='utf-8')\n tree=ET.parse(in_file)\n root = tree.getroot()\n\n for obj in root.iter('object'):\n difficult = 0 \n if obj.find('difficult')!=None:\n difficult = obj.find('difficult').text\n cls = obj.find('name').text\n if cls not in classes or int(difficult)==1:\n continue\n cls_id = classes.index(cls)\n xmlbox = obj.find('bndbox')\n b = (int(float(xmlbox.find('xmin').text)), int(float(xmlbox.find('ymin').text)), int(float(xmlbox.find('xmax').text)), int(float(xmlbox.find('ymax').text)))\n list_file.write(\" \" + \",\".join([str(a) for a in b]) + ',' + str(cls_id))\n \nif __name__ == \"__main__\":\n random.seed(0)\n if annotation_mode == 0 or annotation_mode == 1:\n print(\"Generate txt in ImageSets.\")\n xmlfilepath = os.path.join(VOCdevkit_path, 'VOC2007/Annotations')\n saveBasePath = os.path.join(VOCdevkit_path, 'VOC2007/ImageSets/Main')\n temp_xml = os.listdir(xmlfilepath)\n total_xml = []\n for xml in temp_xml:\n if xml.endswith(\".xml\"):\n total_xml.append(xml)\n\n num = len(total_xml) \n list = range(num) \n tv = int(num*trainval_percent) \n tr = int(tv*train_percent) \n trainval= random.sample(list,tv) \n train = random.sample(trainval,tr) \n \n print(\"train and val size\",tv)\n print(\"train size\",tr)\n ftrainval = open(os.path.join(saveBasePath,'trainval.txt'), 'w') \n ftest = open(os.path.join(saveBasePath,'test.txt'), 'w') \n ftrain = open(os.path.join(saveBasePath,'train.txt'), 'w') \n fval = open(os.path.join(saveBasePath,'val.txt'), 'w') \n \n for i in list: \n name=total_xml[i][:-4]+'\\n' \n if i in trainval: \n ftrainval.write(name) \n if i in train: \n ftrain.write(name) \n else: \n fval.write(name) \n else: \n ftest.write(name) \n \n ftrainval.close() \n ftrain.close() \n fval.close() \n ftest.close()\n print(\"Generate txt in ImageSets done.\")\n\n if annotation_mode == 0 or annotation_mode == 2:\n print(\"Generate gesture_train.txt and 2007_val.txt for train.\")\n for year, image_set in VOCdevkit_sets:\n image_ids = open(os.path.join(VOCdevkit_path, 'VOC%s/ImageSets/Main/%s.txt'%(year, image_set)), encoding='utf-8').read().strip().split()\n list_file = open('%s_%s.txt'%(year, image_set), 'w', encoding='utf-8')\n for image_id in image_ids:\n list_file.write('%s/VOC%s/JPEGImages/%s.jpg'%(os.path.abspath(VOCdevkit_path), year, image_id))\n\n convert_annotation(year, image_id, list_file)\n list_file.write('\\n')\n list_file.close()\n print(\"Generate gesture_train.txt and gesture_val.txt for train done.\")\n" }, { "path": "yolo.py", "content": "import colorsys\nimport os\nimport time\n\nimport numpy as np\nimport torch\nimport torch.nn as nn\nfrom PIL import ImageDraw, ImageFont\n\nfrom nets.yolo import YoloBody\nfrom nets.yolo_tiny import YoloBodytiny\nfrom utils.utils import (cvtColor, get_anchors, get_classes, preprocess_input,\n resize_image)\nfrom utils.utils_bbox import DecodeBox\nfrom get_yaml import get_config\nimport argparse\n'''\n训练自己的数据集必看注释！\n'''\nclass YOLO(object):\n # 配置文件\n config = get_config()\n _defaults = {\n #--------------------------------------------------------------------------#\n # 使用自己训练好的模型进行预测一定要修改model_path和classes_path！\n # model_path指向logs文件夹下的权值文件，classes_path指向model_data下的txt\n #\n # 训练好后logs文件夹下存在多个权值文件，选择验证集损失较低的即可。\n # 验证集损失较低不代表mAP较高，仅代表该权值在验证集上泛化性能较好。\n # 如果出现shape不匹配，同时要注意训练时的model_path和classes_path参数的修改\n #--------------------------------------------------------------------------#\n \"class_names\" : config['classes'],\n \"num_classes\" : config['nc'],\n #---------------------------------------------------------------------#\n # anchors_path代表先验框对应的txt文件，一般不修改。\n # anchors_mask用于帮助代码找到对应的先验框，一般不修改。\n #---------------------------------------------------------------------#\n \"anchors_path\" : 'model_data/yolo_anchors.txt',\n \"anchors_mask\" : [[6, 7, 8], [3, 4, 5], [0, 1, 2]],\n #---------------------------------------------------------------------#\n # 只有得分大于置信度的预测框会被保留下来\n #---------------------------------------------------------------------#\n \"confidence\" : 0.5, # 0.5,\n #---------------------------------------------------------------------#\n # 非极大抑制所用到的nms_iou大小\n #---------------------------------------------------------------------#\n \"nms_iou\" : 0.3, # 0.3,\n #---------------------------------------------------------------------#\n # 该变量用于控制是否使用letterbox_image对输入图像进行不失真的resize，\n # 在多次测试后，发现关闭letterbox_image直接resize的效果更好\n #---------------------------------------------------------------------#\n \"letterbox_image\" : config['letterbox_image'], # False,\n }\n\n\n\n @classmethod\n def get_defaults(cls, n):\n if n in cls._defaults:\n return cls._defaults[n]\n else:\n return \"Unrecognized attribute name '\" + n + \"'\"\n\n #---------------------------------------------------#\n # 初始化YOLO\n #---------------------------------------------------#\n def __init__(self, opt, **kwargs):\n self.__dict__.update(self._defaults)\n for name, value in kwargs.items():\n setattr(self, name, value)\n self.phi = opt.phi\n self.tiny = opt.tiny\n self.cuda = opt.cuda\n self.input_shape = [opt.shape,opt.shape]\n self.model_path = opt.weights\n self.phi = opt.phi\n self.confidence = opt.confidence\n self.nms_iou = opt.nms_iou\n if self.tiny:\n self.anchors_mask = [[3,4,5], [1,2,3]]\n self.anchors_path = 'model_data/yolotiny_anchors.txt'\n #---------------------------------------------------#\n # 获得种类和先验框的数量\n #---------------------------------------------------#\n # self.class_names, self.num_classes = get_classes(self.classes_path)\n self.anchors, self.num_anchors = get_anchors(self.anchors_path)\n self.bbox_util = DecodeBox(self.anchors, self.num_classes, (self.input_shape[0], self.input_shape[1]), self.anchors_mask)\n\n #---------------------------------------------------#\n # 画框设置不同的颜色\n #---------------------------------------------------#\n hsv_tuples = [(x / self.num_classes, 1., 1.) for x in range(self.num_classes)]\n self.colors = list(map(lambda x: colorsys.hsv_to_rgb(*x), hsv_tuples))\n self.colors = list(map(lambda x: (int(x[0] * 255), int(x[1] * 255), int(x[2] * 255)), self.colors))\n self.generate()\n\n #---------------------------------------------------#\n # 生成模型\n #---------------------------------------------------#\n def generate(self, onnx=False):\n #---------------------------------------------------#\n # 建立yolo模型，载入yolo模型的权重\n #---------------------------------------------------#\n\n if not self.tiny:\n self.net = YoloBody(self.anchors_mask, self.num_classes)\n elif self.tiny:\n self.net = YoloBodytiny(self.anchors_mask, self.num_classes, self.phi)\n device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n self.net.load_state_dict(torch.load(self.model_path, map_location=device))\n self.net = self.net.eval()\n\n print('{} model, anchors, and classes loaded.'.format(self.model_path))\n if not onnx:\n if self.cuda:\n self.net = nn.DataParallel(self.net)\n self.net = self.net.cuda()\n\n #---------------------------------------------------#\n # 检测图片\n #---------------------------------------------------#\n def detect_image(self, image, crop = False, count = False):\n #---------------------------------------------------#\n # 计算输入图片的高和宽\n #---------------------------------------------------#\n image_shape = np.array(np.shape(image)[0:2])\n #---------------------------------------------------------#\n # 在这里将图像转换成RGB图像，防止灰度图在预测时报错。\n # 代码仅仅支持RGB图像的预测，所有其它类型的图像都会转化成RGB\n #---------------------------------------------------------#\n image = cvtColor(image)\n #---------------------------------------------------------#\n # 给图像增加灰条，实现不失真的resize\n # 也可以直接resize进行识别\n #---------------------------------------------------------#\n image_data = resize_image(image, (self.input_shape[1],self.input_shape[0]), self.letterbox_image)\n #---------------------------------------------------------#\n # 添加上batch_size维度\n #---------------------------------------------------------#\n image_data = np.expand_dims(np.transpose(preprocess_input(np.array(image_data, dtype='float32')), (2, 0, 1)), 0)\n\n with torch.no_grad():\n images = torch.from_numpy(image_data)\n if self.cuda:\n images = images.cuda()\n #---------------------------------------------------------#\n # 将图像输入网络当中进行预测！\n #---------------------------------------------------------#\n outputs = self.net(images)\n outputs = self.bbox_util.decode_box(outputs)\n #---------------------------------------------------------#\n # 将预测框进行堆叠，然后进行非极大抑制\n #---------------------------------------------------------#\n results = self.bbox_util.non_max_suppression(torch.cat(outputs, 1), self.num_classes, self.input_shape, \n image_shape, self.letterbox_image, conf_thres = self.confidence, nms_thres = self.nms_iou)\n \n if results[0] is None: \n return image\n\n top_label = np.array(results[0][:, 6], dtype = 'int32')\n top_conf = results[0][:, 4] * results[0][:, 5]\n top_boxes = results[0][:, :4]\n #---------------------------------------------------------#\n # 设置字体与边框厚度\n #---------------------------------------------------------#\n font = ImageFont.truetype(font='model_data/simhei.ttf', size=np.floor(3e-2 * image.size[1] + 0.5).astype('int32'))\n thickness = int(max((image.size[0] + image.size[1]) // np.mean(self.input_shape), 1))\n #---------------------------------------------------------#\n # 计数\n #---------------------------------------------------------#\n if count:\n print(\"top_label:\", top_label)\n classes_nums = np.zeros([self.num_classes])\n for i in range(self.num_classes):\n num = np.sum(top_label == i)\n if num > 0:\n print(self.class_names[i], \" : \", num)\n classes_nums[i] = num\n print(\"classes_nums:\", classes_nums)\n #---------------------------------------------------------#\n # 是否进行目标的裁剪\n #---------------------------------------------------------#\n if crop:\n for i, c in list(enumerate(top_label)):\n top, left, bottom, right = top_boxes[i]\n top = max(0, np.floor(top).astype('int32'))\n left = max(0, np.floor(left).astype('int32'))\n bottom = min(image.size[1], np.floor(bottom).astype('int32'))\n right = min(image.size[0], np.floor(right).astype('int32'))\n \n dir_save_path = \"img_crop\"\n if not os.path.exists(dir_save_path):\n os.makedirs(dir_save_path)\n crop_image = image.crop([left, top, right, bottom])\n crop_image.save(os.path.join(dir_save_path, \"crop_\" + str(i) + \".png\"), quality=95, subsampling=0)\n print(\"save crop_\" + str(i) + \".png to \" + dir_save_path)\n #---------------------------------------------------------#\n # 图像绘制\n #---------------------------------------------------------#\n for i, c in list(enumerate(top_label)):\n predicted_class = self.class_names[int(c)]\n box = top_boxes[i]\n score = top_conf[i]\n\n top, left, bottom, right = box\n\n top = max(0, np.floor(top).astype('int32'))\n left = max(0, np.floor(left).astype('int32'))\n bottom = min(image.size[1], np.floor(bottom).astype('int32'))\n right = min(image.size[0], np.floor(right).astype('int32'))\n\n label = '{} {:.2f}'.format(predicted_class, score)\n draw = ImageDraw.Draw(image)\n label_size = draw.textsize(label, font)\n label = label.encode('utf-8')\n print(label, top, left, bottom, right)\n \n if top - label_size[1] >= 0:\n text_origin = np.array([left, top - label_size[1]])\n else:\n text_origin = np.array([left, top + 1])\n\n for i in range(thickness):\n draw.rectangle([left + i, top + i, right - i, bottom - i], outline=self.colors[c])\n draw.rectangle([tuple(text_origin), tuple(text_origin + label_size)], fill=self.colors[c])\n draw.text(text_origin, str(label,'UTF-8'), fill=(0, 0, 0), font=font)\n del draw\n\n return image\n\n def get_FPS(self, image, test_interval):\n image_shape = np.array(np.shape(image)[0:2])\n #---------------------------------------------------------#\n # 在这里将图像转换成RGB图像，防止灰度图在预测时报错。\n # 代码仅仅支持RGB图像的预测，所有其它类型的图像都会转化成RGB\n #---------------------------------------------------------#\n image = cvtColor(image)\n #---------------------------------------------------------#\n # 给图像增加灰条，实现不失真的resize\n # 也可以直接resize进行识别\n #---------------------------------------------------------#\n image_data = resize_image(image, (self.input_shape[1],self.input_shape[0]), self.letterbox_image)\n #---------------------------------------------------------#\n # 添加上batch_size维度\n #---------------------------------------------------------#\n image_data = np.expand_dims(np.transpose(preprocess_input(np.array(image_data, dtype='float32')), (2, 0, 1)), 0)\n\n with torch.no_grad():\n images = torch.from_numpy(image_data)\n if self.cuda:\n images = images.cuda()\n #---------------------------------------------------------#\n # 将图像输入网络当中进行预测！\n #---------------------------------------------------------#\n outputs = self.net(images)\n outputs = self.bbox_util.decode_box(outputs)\n #---------------------------------------------------------#\n # 将预测框进行堆叠，然后进行非极大抑制\n #---------------------------------------------------------#\n results = self.bbox_util.non_max_suppression(torch.cat(outputs, 1), self.num_classes, self.input_shape, \n image_shape, self.letterbox_image, conf_thres=self.confidence, nms_thres=self.nms_iou)\n \n t1 = time.time()\n for _ in range(test_interval):\n with torch.no_grad():\n #---------------------------------------------------------#\n # 将图像输入网络当中进行预测！\n #---------------------------------------------------------#\n outputs = self.net(images)\n outputs = self.bbox_util.decode_box(outputs)\n #---------------------------------------------------------#\n # 将预测框进行堆叠，然后进行非极大抑制\n #---------------------------------------------------------#\n results = self.bbox_util.non_max_suppression(torch.cat(outputs, 1), self.num_classes, self.input_shape, \n image_shape, self.letterbox_image, conf_thres=self.confidence, nms_thres=self.nms_iou)\n \n t2 = time.time()\n tact_time = (t2 - t1) / test_interval\n return tact_time\n\n def detect_heatmap(self, image, heatmap_save_path):\n import cv2\n import matplotlib.pyplot as plt\n def sigmoid(x):\n y = 1.0 / (1.0 + np.exp(-x))\n return y\n #---------------------------------------------------------#\n # 在这里将图像转换成RGB图像，防止灰度图在预测时报错。\n # 代码仅仅支持RGB图像的预测，所有其它类型的图像都会转化成RGB\n #---------------------------------------------------------#\n image = cvtColor(image)\n #---------------------------------------------------------#\n # 给图像增加灰条，实现不失真的resize\n # 也可以直接resize进行识别\n #---------------------------------------------------------#\n image_data = resize_image(image, (self.input_shape[1],self.input_shape[0]), self.letterbox_image)\n #---------------------------------------------------------#\n # 添加上batch_size维度\n #---------------------------------------------------------#\n image_data = np.expand_dims(np.transpose(preprocess_input(np.array(image_data, dtype='float32')), (2, 0, 1)), 0)\n\n with torch.no_grad():\n images = torch.from_numpy(image_data)\n if self.cuda:\n images = images.cuda()\n #---------------------------------------------------------#\n # 将图像输入网络当中进行预测！\n #---------------------------------------------------------#\n outputs = self.net(images)\n \n plt.imshow(image, alpha=1)\n plt.axis('off')\n mask = np.zeros((image.size[1], image.size[0]))\n for sub_output in outputs:\n sub_output = sub_output.cpu().numpy()\n b, c, h, w = np.shape(sub_output)\n sub_output = np.transpose(np.reshape(sub_output, [b, 3, -1, h, w]), [0, 3, 4, 1, 2])[0]\n score = np.max(sigmoid(sub_output[..., 4]), -1)\n score = cv2.resize(score, (image.size[0], image.size[1]))\n normed_score = (score * 255).astype('uint8')\n mask = np.maximum(mask, normed_score)\n \n plt.imshow(mask, alpha=0.5, interpolation='nearest', cmap=\"jet\")\n\n plt.axis('off')\n plt.subplots_adjust(top=1, bottom=0, right=1, left=0, hspace=0, wspace=0)\n plt.margins(0, 0)\n plt.savefig(heatmap_save_path, dpi=200, bbox_inches='tight', pad_inches = -0.1)\n print(\"Save to the \" + heatmap_save_path)\n plt.show()\n\n def convert_to_onnx(self, simplify, model_path):\n import onnx\n self.generate(onnx=True)\n\n im = torch.zeros(1, 3, *self.input_shape).to('cpu') # image size(1, 3, 512, 512) BCHW\n input_layer_names = [\"images\"]\n output_layer_names = [\"output\"]\n \n # Export the model\n print(f'Starting export with onnx {onnx.__version__}.')\n torch.onnx.export(self.net,\n im,\n f = model_path,\n verbose = False,\n opset_version = 12,\n training = torch.onnx.TrainingMode.EVAL,\n do_constant_folding = True,\n input_names = input_layer_names,\n output_names = output_layer_names,\n dynamic_axes = None)\n\n # Checks\n model_onnx = onnx.load(model_path) # load onnx model\n onnx.checker.check_model(model_onnx) # check onnx model\n\n # Simplify onnx\n if simplify:\n import onnxsim\n print(f'Simplifying with onnx-simplifier {onnxsim.__version__}.')\n model_onnx, check = onnxsim.simplify(\n model_onnx,\n dynamic_input_shape=False,\n input_shapes=None)\n assert check, 'assert check failed'\n onnx.save(model_onnx, model_path)\n\n print('Onnx model save as {}'.format(model_path))\n\n def get_map_txt(self, image_id, image, class_names, map_out_path):\n f = open(os.path.join(map_out_path, \"detection-results/\"+image_id+\".txt\"),\"w\") \n image_shape = np.array(np.shape(image)[0:2])\n #---------------------------------------------------------#\n # 在这里将图像转换成RGB图像，防止灰度图在预测时报错。\n # 代码仅仅支持RGB图像的预测，所有其它类型的图像都会转化成RGB\n #---------------------------------------------------------#\n image = cvtColor(image)\n #---------------------------------------------------------#\n # 给图像增加灰条，实现不失真的resize\n # 也可以直接resize进行识别\n #---------------------------------------------------------#\n image_data = resize_image(image, (self.input_shape[1],self.input_shape[0]), self.letterbox_image)\n #---------------------------------------------------------#\n # 添加上batch_size维度\n #---------------------------------------------------------#\n image_data = np.expand_dims(np.transpose(preprocess_input(np.array(image_data, dtype='float32')), (2, 0, 1)), 0)\n\n with torch.no_grad():\n images = torch.from_numpy(image_data)\n if self.cuda:\n images = images.cuda()\n #---------------------------------------------------------#\n # 将图像输入网络当中进行预测！\n #---------------------------------------------------------#\n outputs = self.net(images)\n outputs = self.bbox_util.decode_box(outputs)\n #---------------------------------------------------------#\n # 将预测框进行堆叠，然后进行非极大抑制\n #---------------------------------------------------------#\n results = self.bbox_util.non_max_suppression(torch.cat(outputs, 1), self.num_classes, self.input_shape, \n image_shape, self.letterbox_image, conf_thres = self.confidence, nms_thres = self.nms_iou)\n \n if results[0] is None: \n return \n\n top_label = np.array(results[0][:, 6], dtype = 'int32')\n top_conf = results[0][:, 4] * results[0][:, 5]\n top_boxes = results[0][:, :4]\n\n for i, c in list(enumerate(top_label)):\n predicted_class = self.class_names[int(c)]\n box = top_boxes[i]\n score = str(top_conf[i])\n\n top, left, bottom, right = box\n if predicted_class not in class_names:\n continue\n\n f.write(\"%s %s %s %s %s %s\\n\" % (predicted_class, score[:6], str(int(left)), str(int(top)), str(int(right)),str(int(bottom))))\n\n f.close()\n return \n" }, { "path": "yolo_anchors.txt", "content": "105,107, 118,136, 152,122, 114,165, 139,151, 160,156, 152,185, 181,167, 192,197" }, { "path": "yolov4-gesture-tutorial.ipynb", "content": "{\"cells\":[{\"cell_type\":\"markdown\",\"metadata\":{\"id\":\"9MEPFVpX4mRS\"},\"source\":[\"# 挂载Drive （使用colab才做这个操作）\"]},{\"cell_type\":\"code\",\"execution_count\":1,\"metadata\":{\"colab\":{\"base_uri\":\"https://localhost:8080/\"},\"executionInfo\":{\"elapsed\":34099,\"status\":\"ok\",\"timestamp\":1651105771374,\"user\":{\"displayName\":\"KaiJun Deng\",\"userId\":\"04642544504944131029\"},\"user_tz\":-480},\"id\":\"TjAeId9H0zl7\",\"outputId\":\"32dd1d61-331c-46b4-a190-cef10340e732\"},\"outputs\":[{\"output_type\":\"stream\",\"name\":\"stdout\",\"text\":[\"/content\\n\",\"Thu Apr 28 00:28:57 2022 \\n\",\"+-----------------------------------------------------------------------------+\\n\",\"| NVIDIA-SMI 460.32.03 Driver Version: 460.32.03 CUDA Version: 11.2 |\\n\",\"|-------------------------------+----------------------+----------------------+\\n\",\"| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |\\n\",\"| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |\\n\",\"| | | MIG M. |\\n\",\"|===============================+======================+======================|\\n\",\"| 0 Tesla K80 Off | 00000000:00:04.0 Off | 0 |\\n\",\"| N/A 33C P8 29W / 149W | 0MiB / 11441MiB | 0% Default |\\n\",\"| | | N/A |\\n\",\"+-------------------------------+----------------------+----------------------+\\n\",\" \\n\",\"+-----------------------------------------------------------------------------+\\n\",\"| Processes: |\\n\",\"| GPU GI CI PID Type Process name GPU Memory |\\n\",\"| ID ID Usage |\\n\",\"|=============================================================================|\\n\",\"| No running processes found |\\n\",\"+-----------------------------------------------------------------------------+\\n\",\"Mounted at /content/gdrive\\n\"]}],\"source\":[\"%cd /content\\n\",\"!nvidia-smi\\n\",\"from google.colab import drive\\n\",\"drive.mount('/content/gdrive')\"]},{\"cell_type\":\"code\",\"execution_count\":24,\"metadata\":{\"colab\":{\"base_uri\":\"https://localhost:8080/\"},\"executionInfo\":{\"elapsed\":30262,\"status\":\"ok\",\"timestamp\":1651106445511,\"user\":{\"displayName\":\"KaiJun Deng\",\"userId\":\"04642544504944131029\"},\"user_tz\":-480},\"id\":\"1qANLkv-F8Ho\",\"outputId\":\"13ee99dc-9e24-4ab3-8146-1a70322cc750\"},\"outputs\":[{\"output_type\":\"stream\",\"name\":\"stdout\",\"text\":[\"/content\\n\",\"Cloning into 'College-Students-Innovative-Entrepreneurial-Training-Plan-Program'...\\n\",\"warning: redirecting to https://github.com/Dreaming-future/College-Students-Innovative-Entrepreneurial-Training-Plan-Program/\\n\",\"remote: Enumerating objects: 7027, done.\\u001b[K\\n\",\"remote: Counting objects: 100% (12/12), done.\\u001b[K\\n\",\"remote: Compressing objects: 100% (9/9), done.\\u001b[K\\n\",\"remote: Total 7027 (delta 6), reused 9 (delta 3), pack-reused 7015\\u001b[K\\n\",\"Receiving objects: 100% (7027/7027), 326.24 MiB | 12.62 MiB/s, done.\\n\",\"Resolving deltas: 100% (3882/3882), done.\\n\",\"Checking out files: 100% (4448/4448), done.\\n\",\"/content/College-Students-Innovative-Entrepreneurial-Training-Plan-Program/yolov4-gesture\\n\"]}],\"source\":[\"# git clone 一下代码进行colab\\n\",\"%cd /content\\n\",\"!rm -rf /content/College-Students-Innovative-Entrepreneurial-Training-Plan-Program\\n\",\"!git clone http://project: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@github.com/Dreaming-future/College-Students-Innovative-Entrepreneurial-Training-Plan-Program\\n\",\"%cd College-Students-Innovative-Entrepreneurial-Training-Plan-Program/yolov4-gesture\"]},{\"cell_type\":\"markdown\",\"metadata\":{\"id\":\"WZwtRt1N4uEP\"},\"source\":[\"# 连接Drive（使用colab才做这个操作）\"]},{\"cell_type\":\"code\",\"execution_count\":25,\"metadata\":{\"executionInfo\":{\"elapsed\":664,\"status\":\"ok\",\"timestamp\":1651106446164,\"user\":{\"displayName\":\"KaiJun Deng\",\"userId\":\"04642544504944131029\"},\"user_tz\":-480},\"id\":\"LErdhCFizPuU\"},\"outputs\":[],\"source\":[\"!rm -rf logs\\n\",\"# 与drive建立软连接\\n\",\"!ln -s /content/gdrive/MyDrive/weights/logs logs\\n\",\"# # 复制权重文件 \\n\",\"# !cp /content/gdrive/MyDrive/weights/yolo4_gesture_weightsv3.pth \\\\\\n\",\"# /content/College-Students-Innovative-Entrepreneurial-Training-Plan-Program/yolov4-gesture/model_data/yolo4_gesture_weights.pth\"]},{\"cell_type\":\"markdown\",\"metadata\":{\"id\":\"iLMsxSTSGUFi\"},\"source\":[\"# 对数据集进行预处理\"]},{\"cell_type\":\"code\",\"execution_count\":3,\"metadata\":{\"colab\":{\"base_uri\":\"https://localhost:8080/\"},\"executionInfo\":{\"elapsed\":64099,\"status\":\"ok\",\"timestamp\":1651105859081,\"user\":{\"displayName\":\"KaiJun Deng\",\"userId\":\"04642544504944131029\"},\"user_tz\":-480},\"id\":\"l0LFj3WTyaKt\",\"outputId\":\"2d011aa2-668a-4e18-8cad-94db14edfc98\"},\"outputs\":[{\"output_type\":\"stream\",\"name\":\"stdout\",\"text\":[\"Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 1)) (1.4.1)\\n\",\"Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 2)) (1.21.6)\\n\",\"Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 3)) (3.2.2)\\n\",\"Requirement already satisfied: opencv_python in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 4)) (4.1.2.30)\\n\",\"Collecting torch==1.8.1\\n\",\" Downloading torch-1.8.1-cp37-cp37m-manylinux1_x86_64.whl (804.1 MB)\\n\",\"\\u001b[K |████████████████████████████████| 804.1 MB 2.7 kB/s \\n\",\"\\u001b[?25hCollecting torchvision==0.9.1\\n\",\" Downloading torchvision-0.9.1-cp37-cp37m-manylinux1_x86_64.whl (17.4 MB)\\n\",\"\\u001b[K |████████████████████████████████| 17.4 MB 545 kB/s \\n\",\"\\u001b[?25hCollecting tqdm==4.60.0\\n\",\" Downloading tqdm-4.60.0-py2.py3-none-any.whl (75 kB)\\n\",\"\\u001b[K |████████████████████████████████| 75 kB 4.2 MB/s \\n\",\"\\u001b[?25hCollecting Pillow==8.2.0\\n\",\" Downloading Pillow-8.2.0-cp37-cp37m-manylinux1_x86_64.whl (3.0 MB)\\n\",\"\\u001b[K |████████████████████████████████| 3.0 MB 36.0 MB/s \\n\",\"\\u001b[?25hCollecting h5py==2.10.0\\n\",\" Downloading h5py-2.10.0-cp37-cp37m-manylinux1_x86_64.whl (2.9 MB)\\n\",\"\\u001b[K |████████████████████████████████| 2.9 MB 38.9 MB/s \\n\",\"\\u001b[?25hRequirement already satisfied: tensorboard in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 10)) (2.8.0)\\n\",\"Collecting pyyaml==6.0\\n\",\" Downloading PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB)\\n\",\"\\u001b[K |████████████████████████████████| 596 kB 55.7 MB/s \\n\",\"\\u001b[31mERROR: Could not find a version that satisfies the requirement tochinfo (from versions: none)\\u001b[0m\\n\",\"\\u001b[31mERROR: No matching distribution found for tochinfo\\u001b[0m\\n\",\"\\u001b[?25hCollecting tqdm==4.60.0\\n\",\" Using cached tqdm-4.60.0-py2.py3-none-any.whl (75 kB)\\n\",\"Collecting h5py==2.10.0\\n\",\" Using cached h5py-2.10.0-cp37-cp37m-manylinux1_x86_64.whl (2.9 MB)\\n\",\"Collecting pyyaml==6.0\\n\",\" Using cached PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB)\\n\",\"Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from h5py==2.10.0) (1.15.0)\\n\",\"Requirement already satisfied: numpy>=1.7 in /usr/local/lib/python3.7/dist-packages (from h5py==2.10.0) (1.21.6)\\n\",\"Installing collected packages: tqdm, pyyaml, h5py\\n\",\" Attempting uninstall: tqdm\\n\",\" Found existing installation: tqdm 4.64.0\\n\",\" Uninstalling tqdm-4.64.0:\\n\",\" Successfully uninstalled tqdm-4.64.0\\n\",\" Attempting uninstall: pyyaml\\n\",\" Found existing installation: PyYAML 3.13\\n\",\" Uninstalling PyYAML-3.13:\\n\",\" Successfully uninstalled PyYAML-3.13\\n\",\" Attempting uninstall: h5py\\n\",\" Found existing installation: h5py 3.1.0\\n\",\" Uninstalling h5py-3.1.0:\\n\",\" Successfully uninstalled h5py-3.1.0\\n\",\"\\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\\n\",\"tensorflow 2.8.0 requires tf-estimator-nightly==2.8.0.dev2021122109, which is not installed.\\u001b[0m\\n\",\"Successfully installed h5py-2.10.0 pyyaml-6.0 tqdm-4.60.0\\n\"]}],\"source\":[\"# pip 安装包\\n\",\"!pip install -r requirements.txt\\n\",\"!pip install tqdm==4.60.0 h5py==2.10.0 pyyaml==6.0\"]},{\"cell_type\":\"code\",\"execution_count\":26,\"metadata\":{\"colab\":{\"base_uri\":\"https://localhost:8080/\"},\"executionInfo\":{\"elapsed\":4,\"status\":\"ok\",\"timestamp\":1651106446164,\"user\":{\"displayName\":\"KaiJun Deng\",\"userId\":\"04642544504944131029\"},\"user_tz\":-480},\"id\":\"e5j2Q9-zxbuN\",\"outputId\":\"94f411c1-8256-4a02-93c3-9c095e941935\"},\"outputs\":[{\"output_type\":\"stream\",\"name\":\"stdout\",\"text\":[\"Generate txt in ImageSets.\\n\",\"train and val size 1601\\n\",\"train size 1440\\n\",\"Generate txt in ImageSets done.\\n\",\"Generate gesture_train.txt and 2007_val.txt for train.\\n\",\"Generate gesture_train.txt and gesture_val.txt for train done.\\n\"]}],\"source\":[\"!python voc_annotation.py\"]},{\"cell_type\":\"code\",\"execution_count\":null,\"metadata\":{\"id\":\"vX7FkUmiOUQR\"},\"outputs\":[],\"source\":[\"# # 复制最新的权重文件作为backbone\\n\",\"# !cp /content/gdrive/MyDrive/模型权重文件/logs/ep050-loss0.077-val_loss0.052.pth \\\\\\n\",\"# /content/College-Students-Innovative-Entrepreneurial-Training-Plan-Program/yolov4-gesture/model_data/yolo4_gesture_weight.pth\"]},{\"cell_type\":\"code\",\"execution_count\":27,\"metadata\":{\"colab\":{\"base_uri\":\"https://localhost:8080/\"},\"executionInfo\":{\"elapsed\":2408,\"status\":\"ok\",\"timestamp\":1651106448570,\"user\":{\"displayName\":\"KaiJun Deng\",\"userId\":\"04642544504944131029\"},\"user_tz\":-480},\"id\":\"zKn6VU7gnaeU\",\"outputId\":\"0d37fb42-36ac-4db4-d35a-407391460865\"},\"outputs\":[{\"output_type\":\"stream\",\"name\":\"stdout\",\"text\":[\"Load xmls.\\n\",\"\\r 0% 0/1601 [00:00\\n\",\"Save kmeans_for_anchors.jpg in root dir.\\n\",\"avg_ratio:0.89\\n\",\"[[105.91304348 105.77777778]\\n\",\" [ 97.60893855 140.28282828]\\n\",\" [128.99895507 134.10934744]\\n\",\" [159.6 122.79069767]\\n\",\" [121.625 161.66037736]\\n\",\" [147.69230769 153.6 ]\\n\",\" [155.27272727 185.21212121]\\n\",\" [178.21422887 162.90909091]\\n\",\" [192.22695035 196. ]]\\n\"]}],\"source\":[\"!python kmeans_for_anchors.py\"]},{\"cell_type\":\"markdown\",\"metadata\":{\"id\":\"Y-SAO6KltlfF\"},\"source\":[\"# 训练数据\"]},{\"cell_type\":\"markdown\",\"source\":[\"# yolov4 版本\"],\"metadata\":{\"id\":\"efuI36jfn2fL\"}},{\"cell_type\":\"code\",\"source\":[\"!wget -nc https://github.com/bubbliiiing/yolov4-pytorch/releases/download/v1.0/yolo4_weights.pth -O model_data/yolo4_weights.pth\"],\"metadata\":{\"colab\":{\"base_uri\":\"https://localhost:8080/\"},\"id\":\"1K3isfpZn6Kw\",\"executionInfo\":{\"status\":\"ok\",\"timestamp\":1651112763279,\"user_tz\":-480,\"elapsed\":59782,\"user\":{\"displayName\":\"KaiJun Deng\",\"userId\":\"04642544504944131029\"}},\"outputId\":\"23141ca8-580f-4b86-d232-495f277003c2\"},\"execution_count\":33,\"outputs\":[{\"output_type\":\"stream\",\"name\":\"stdout\",\"text\":[\"--2022-04-28 02:25:04-- https://github.com/bubbliiiing/yolov4-pytorch/releases/download/v1.0/yolo4_weights.pth\\n\",\"Resolving github.com (github.com)... 13.114.40.48\\n\",\"Connecting to github.com (github.com)|13.114.40.48|:443... connected.\\n\",\"HTTP request sent, awaiting response... 302 Found\\n\",\"Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/266480370/07efad80-654c-11eb-9fc4-8055eb471ae0?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220428%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220428T022504Z&X-Amz-Expires=300&X-Amz-Signature=c60bad9aed15ce878abe1d659e0ff80078d0064b64d4ff1d470fac80845e84d0&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=266480370&response-content-disposition=attachment%3B%20filename%3Dyolo4_weights.pth&response-content-type=application%2Foctet-stream [following]\\n\",\"--2022-04-28 02:25:04-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/266480370/07efad80-654c-11eb-9fc4-8055eb471ae0?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220428%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220428T022504Z&X-Amz-Expires=300&X-Amz-Signature=c60bad9aed15ce878abe1d659e0ff80078d0064b64d4ff1d470fac80845e84d0&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=266480370&response-content-disposition=attachment%3B%20filename%3Dyolo4_weights.pth&response-content-type=application%2Foctet-stream\\n\",\"Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\\n\",\"Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.108.133|:443... connected.\\n\",\"HTTP request sent, awaiting response... 200 OK\\n\",\"Length: 257863693 (246M) [application/octet-stream]\\n\",\"Saving to: ‘model_data/yolo4_weights.pth’\\n\",\"\\n\",\"model_data/yolo4_we 100%[===================>] 245.92M 3.96MB/s in 57s \\n\",\"\\n\",\"2022-04-28 02:26:03 (4.31 MB/s) - ‘model_data/yolo4_weights.pth’ saved [257863693/257863693]\\n\",\"\\n\"]}]},{\"cell_type\":\"code\",\"source\":[\"# 训练数据，默认为100个epochs\\n\",\"!python train.py --epochs 100 \\\\\\n\",\" --weights model_data/yolo4_weights.pth \\\\\\n\",\" --freeze --freeze-epochs 50 --freeze-size 32 \\\\\\n\",\" --batch-size 10 --shape 416 \\\\\\n\",\" --fp16 --cuda\"],\"metadata\":{\"colab\":{\"base_uri\":\"https://localhost:8080/\"},\"id\":\"lzgTnAW7n8ig\",\"outputId\":\"9f6f2374-3573-430b-a1b1-55af6ad997d8\"},\"execution_count\":null,\"outputs\":[{\"output_type\":\"stream\",\"name\":\"stdout\",\"text\":[\"Namespace(batch_size=10, cuda=True, epochs=100, fp16=True, freeze=True, freeze_epochs=50, freeze_size=32, init=0, lr=0.02, momentum=0.937, mosaic=False, optimizer='adam', phi=0, save_period=4, shape=416, tiny=False, weight_decay=0, weights='model_data/yolo4_weights.pth')\\n\",\"initialize network with normal type\\n\",\"Load weights model_data/yolo4_weights.pth.\\n\",\"Start Train\\n\",\"Epoch 1/100: 100% 45/45 [01:55<00:00, 2.56s/it, loss=14.7, lr=0.0001]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 1/100: 100% 5/5 [00:17<00:00, 3.42s/it, val_loss=10.6]\\n\",\"Finish Validation\\n\",\"Epoch:1/100\\n\",\"Total Loss: 14.700 || Val Loss: 10.551 \\n\",\"Start Train\\n\",\"Epoch 2/100: 100% 45/45 [01:51<00:00, 2.47s/it, loss=7.37, lr=0.0002]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 2/100: 100% 5/5 [00:12<00:00, 2.54s/it, val_loss=4.12]\\n\",\"Finish Validation\\n\",\"Epoch:2/100\\n\",\"Total Loss: 7.371 || Val Loss: 4.121 \\n\",\"Start Train\\n\",\"Epoch 3/100: 100% 45/45 [01:49<00:00, 2.44s/it, loss=1.99, lr=0.0005]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 3/100: 100% 5/5 [00:12<00:00, 2.43s/it, val_loss=0.791]\\n\",\"Finish Validation\\n\",\"Epoch:3/100\\n\",\"Total Loss: 1.995 || Val Loss: 0.791 \\n\",\"Start Train\\n\",\"Epoch 4/100: 100% 45/45 [01:49<00:00, 2.42s/it, loss=0.523, lr=0.001]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 4/100: 100% 5/5 [00:11<00:00, 2.37s/it, val_loss=0.281]\\n\",\"Finish Validation\\n\",\"Epoch:4/100\\n\",\"Total Loss: 0.523 || Val Loss: 0.281 \\n\",\"Start Train\\n\",\"Epoch 5/100: 100% 45/45 [01:51<00:00, 2.48s/it, loss=0.268, lr=0.001]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 5/100: 100% 5/5 [00:11<00:00, 2.36s/it, val_loss=0.176]\\n\",\"Finish Validation\\n\",\"Epoch:5/100\\n\",\"Total Loss: 0.268 || Val Loss: 0.176 \\n\",\"Start Train\\n\",\"Epoch 6/100: 100% 45/45 [01:48<00:00, 2.42s/it, loss=0.209, lr=0.000999]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 6/100: 100% 5/5 [00:12<00:00, 2.42s/it, val_loss=0.132]\\n\",\"Finish Validation\\n\",\"Epoch:6/100\\n\",\"Total Loss: 0.209 || Val Loss: 0.132 \\n\",\"Start Train\\n\",\"Epoch 7/100: 100% 45/45 [01:50<00:00, 2.47s/it, loss=0.179, lr=0.000997]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 7/100: 100% 5/5 [00:12<00:00, 2.53s/it, val_loss=0.108]\\n\",\"Finish Validation\\n\",\"Epoch:7/100\\n\",\"Total Loss: 0.179 || Val Loss: 0.108 \\n\",\"Start Train\\n\",\"Epoch 8/100: 100% 45/45 [01:50<00:00, 2.45s/it, loss=0.158, lr=0.000995]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 8/100: 100% 5/5 [00:12<00:00, 2.52s/it, val_loss=0.0953]\\n\",\"Finish Validation\\n\",\"Epoch:8/100\\n\",\"Total Loss: 0.158 || Val Loss: 0.095 \\n\",\"Start Train\\n\",\"Epoch 9/100: 100% 45/45 [01:51<00:00, 2.47s/it, loss=0.139, lr=0.000993]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 9/100: 100% 5/5 [00:11<00:00, 2.37s/it, val_loss=0.0818]\\n\",\"Finish Validation\\n\",\"Epoch:9/100\\n\",\"Total Loss: 0.139 || Val Loss: 0.082 \\n\",\"Start Train\\n\",\"Epoch 10/100: 100% 45/45 [01:51<00:00, 2.47s/it, loss=0.129, lr=0.00099]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 10/100: 100% 5/5 [00:12<00:00, 2.57s/it, val_loss=0.0734]\\n\",\"Finish Validation\\n\",\"Epoch:10/100\\n\",\"Total Loss: 0.129 || Val Loss: 0.073 \\n\",\"Start Train\\n\",\"Epoch 11/100: 100% 45/45 [01:49<00:00, 2.43s/it, loss=0.126, lr=0.000986]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 11/100: 100% 5/5 [00:11<00:00, 2.35s/it, val_loss=0.0664]\\n\",\"Finish Validation\\n\",\"Epoch:11/100\\n\",\"Total Loss: 0.126 || Val Loss: 0.066 \\n\",\"Start Train\\n\",\"Epoch 12/100: 100% 45/45 [01:48<00:00, 2.41s/it, loss=0.115, lr=0.000982]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 12/100: 100% 5/5 [00:12<00:00, 2.43s/it, val_loss=0.063]\\n\",\"Finish Validation\\n\",\"Epoch:12/100\\n\",\"Total Loss: 0.115 || Val Loss: 0.063 \\n\",\"Start Train\\n\",\"Epoch 13/100: 100% 45/45 [01:49<00:00, 2.44s/it, loss=0.105, lr=0.000977]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 13/100: 100% 5/5 [00:11<00:00, 2.30s/it, val_loss=0.0522]\\n\",\"Finish Validation\\n\",\"Epoch:13/100\\n\",\"Total Loss: 0.105 || Val Loss: 0.052 \\n\",\"Start Train\\n\",\"Epoch 14/100: 100% 45/45 [01:49<00:00, 2.44s/it, loss=0.1, lr=0.000971]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 14/100: 100% 5/5 [00:11<00:00, 2.28s/it, val_loss=0.0489]\\n\",\"Finish Validation\\n\",\"Epoch:14/100\\n\",\"Total Loss: 0.100 || Val Loss: 0.049 \\n\",\"Start Train\\n\",\"Epoch 15/100: 100% 45/45 [01:49<00:00, 2.43s/it, loss=0.0965, lr=0.000965]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 15/100: 100% 5/5 [00:12<00:00, 2.53s/it, val_loss=0.0468]\\n\",\"Finish Validation\\n\",\"Epoch:15/100\\n\",\"Total Loss: 0.097 || Val Loss: 0.047 \\n\",\"Start Train\\n\",\"Epoch 16/100: 100% 45/45 [01:47<00:00, 2.40s/it, loss=0.0992, lr=0.000959]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 16/100: 100% 5/5 [00:12<00:00, 2.43s/it, val_loss=0.045]\\n\",\"Finish Validation\\n\",\"Epoch:16/100\\n\",\"Total Loss: 0.099 || Val Loss: 0.045 \\n\",\"Start Train\\n\",\"Epoch 17/100: 100% 45/45 [01:49<00:00, 2.44s/it, loss=0.0935, lr=0.000952]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 17/100: 100% 5/5 [00:12<00:00, 2.46s/it, val_loss=0.0408]\\n\",\"Finish Validation\\n\",\"Epoch:17/100\\n\",\"Total Loss: 0.094 || Val Loss: 0.041 \\n\",\"Start Train\\n\",\"Epoch 18/100: 100% 45/45 [01:49<00:00, 2.43s/it, loss=0.0881, lr=0.000945]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 18/100: 100% 5/5 [00:11<00:00, 2.36s/it, val_loss=0.0389]\\n\",\"Finish Validation\\n\",\"Epoch:18/100\\n\",\"Total Loss: 0.088 || Val Loss: 0.039 \\n\",\"Start Train\\n\",\"Epoch 19/100: 100% 45/45 [01:48<00:00, 2.42s/it, loss=0.0884, lr=0.000936]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 19/100: 100% 5/5 [00:11<00:00, 2.25s/it, val_loss=0.04]\\n\",\"Finish Validation\\n\",\"Epoch:19/100\\n\",\"Total Loss: 0.088 || Val Loss: 0.040 \\n\",\"Start Train\\n\",\"Epoch 20/100: 100% 45/45 [01:49<00:00, 2.44s/it, loss=0.077, lr=0.000928]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 20/100: 100% 5/5 [00:12<00:00, 2.46s/it, val_loss=0.0346]\\n\",\"Finish Validation\\n\",\"Epoch:20/100\\n\",\"Total Loss: 0.077 || Val Loss: 0.035 \\n\",\"Start Train\\n\",\"Epoch 21/100: 100% 45/45 [01:48<00:00, 2.41s/it, loss=0.0783, lr=0.000919]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 21/100: 100% 5/5 [00:12<00:00, 2.44s/it, val_loss=0.0323]\\n\",\"Finish Validation\\n\",\"Epoch:21/100\\n\",\"Total Loss: 0.078 || Val Loss: 0.032 \\n\",\"Start Train\\n\",\"Epoch 22/100: 100% 45/45 [01:48<00:00, 2.41s/it, loss=0.0769, lr=0.000909]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 22/100: 100% 5/5 [00:11<00:00, 2.37s/it, val_loss=0.0304]\\n\",\"Finish Validation\\n\",\"Epoch:22/100\\n\",\"Total Loss: 0.077 || Val Loss: 0.030 \\n\",\"Start Train\\n\",\"Epoch 23/100: 100% 45/45 [01:47<00:00, 2.40s/it, loss=0.0735, lr=0.000899]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 23/100: 100% 5/5 [00:12<00:00, 2.43s/it, val_loss=0.0338]\\n\",\"Finish Validation\\n\",\"Epoch:23/100\\n\",\"Total Loss: 0.073 || Val Loss: 0.034 \\n\",\"Start Train\\n\",\"Epoch 24/100: 100% 45/45 [01:48<00:00, 2.41s/it, loss=0.0718, lr=0.000889]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 24/100: 100% 5/5 [00:12<00:00, 2.44s/it, val_loss=0.0296]\\n\",\"Finish Validation\\n\",\"Epoch:24/100\\n\",\"Total Loss: 0.072 || Val Loss: 0.030 \\n\",\"Start Train\\n\",\"Epoch 25/100: 100% 45/45 [01:48<00:00, 2.42s/it, loss=0.0707, lr=0.000878]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 25/100: 100% 5/5 [00:12<00:00, 2.47s/it, val_loss=0.0255]\\n\",\"Finish Validation\\n\",\"Epoch:25/100\\n\",\"Total Loss: 0.071 || Val Loss: 0.026 \\n\",\"Start Train\\n\",\"Epoch 26/100: 100% 45/45 [01:47<00:00, 2.39s/it, loss=0.0666, lr=0.000867]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 26/100: 100% 5/5 [00:12<00:00, 2.45s/it, val_loss=0.0256]\\n\",\"Finish Validation\\n\",\"Epoch:26/100\\n\",\"Total Loss: 0.067 || Val Loss: 0.026 \\n\",\"Start Train\\n\",\"Epoch 27/100: 100% 45/45 [01:49<00:00, 2.43s/it, loss=0.0691, lr=0.000855]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 27/100: 100% 5/5 [00:12<00:00, 2.43s/it, val_loss=0.0271]\\n\",\"Finish Validation\\n\",\"Epoch:27/100\\n\",\"Total Loss: 0.069 || Val Loss: 0.027 \\n\",\"Start Train\\n\",\"Epoch 28/100: 100% 45/45 [01:48<00:00, 2.41s/it, loss=0.0636, lr=0.000843]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 28/100: 100% 5/5 [00:11<00:00, 2.40s/it, val_loss=0.0259]\\n\",\"Finish Validation\\n\",\"Epoch:28/100\\n\",\"Total Loss: 0.064 || Val Loss: 0.026 \\n\",\"Start Train\\n\",\"Epoch 29/100: 100% 45/45 [01:50<00:00, 2.46s/it, loss=0.066, lr=0.00083]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 29/100: 100% 5/5 [00:11<00:00, 2.27s/it, val_loss=0.0231]\\n\",\"Finish Validation\\n\",\"Epoch:29/100\\n\",\"Total Loss: 0.066 || Val Loss: 0.023 \\n\",\"Start Train\\n\",\"Epoch 30/100: 100% 45/45 [01:48<00:00, 2.42s/it, loss=0.0672, lr=0.000817]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 30/100: 100% 5/5 [00:12<00:00, 2.45s/it, val_loss=0.0327]\\n\",\"Finish Validation\\n\",\"Epoch:30/100\\n\",\"Total Loss: 0.067 || Val Loss: 0.033 \\n\",\"Start Train\\n\",\"Epoch 31/100: 100% 45/45 [01:49<00:00, 2.43s/it, loss=0.0605, lr=0.000804]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 31/100: 100% 5/5 [00:11<00:00, 2.26s/it, val_loss=0.0269]\\n\",\"Finish Validation\\n\",\"Epoch:31/100\\n\",\"Total Loss: 0.061 || Val Loss: 0.027 \\n\",\"Start Train\\n\",\"Epoch 32/100: 100% 45/45 [01:51<00:00, 2.47s/it, loss=0.0571, lr=0.00079]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 32/100: 100% 5/5 [00:11<00:00, 2.30s/it, val_loss=0.0266]\\n\",\"Finish Validation\\n\",\"Epoch:32/100\\n\",\"Total Loss: 0.057 || Val Loss: 0.027 \\n\",\"Start Train\\n\",\"Epoch 33/100: 33% 15/45 [00:40<01:09, 2.30s/it, loss=0.063, lr=0.000776]\"]}]},{\"cell_type\":\"markdown\",\"source\":[\"## yolov4-tiny 版本\"],\"metadata\":{\"id\":\"6YU6lPwJnfPT\"}},{\"cell_type\":\"markdown\",\"source\":[\"### CSPdarknet53-tiny无注意力机制\"],\"metadata\":{\"id\":\"Zwlh5OyelE4n\"}},{\"cell_type\":\"code\",\"source\":[\"!wget -nc https://github.91chi.fun/https://github.com//bubbliiiing/yolov4-tiny-pytorch/releases/download/v1.0/yolov4_tiny_weights_coco.pth -O model_data/yolov4_tiny_weights_coco.pth\"],\"metadata\":{\"colab\":{\"base_uri\":\"https://localhost:8080/\"},\"id\":\"tFSRGVZBlMd5\",\"executionInfo\":{\"status\":\"ok\",\"timestamp\":1651106450640,\"user_tz\":-480,\"elapsed\":2072,\"user\":{\"displayName\":\"KaiJun Deng\",\"userId\":\"04642544504944131029\"}},\"outputId\":\"ec00320e-6ddf-4a98-f1d6-e70449e90da1\"},\"execution_count\":28,\"outputs\":[{\"output_type\":\"stream\",\"name\":\"stdout\",\"text\":[\"--2022-04-28 00:40:48-- https://github.91chi.fun/https://github.com//bubbliiiing/yolov4-tiny-pytorch/releases/download/v1.0/yolov4_tiny_weights_coco.pth\\n\",\"Resolving github.91chi.fun (github.91chi.fun)... 104.21.48.120, 172.67.151.57, 2606:4700:3037::6815:3078, ...\\n\",\"Connecting to github.91chi.fun (github.91chi.fun)|104.21.48.120|:443... connected.\\n\",\"HTTP request sent, awaiting response... 200 OK\\n\",\"Length: 24274351 (23M) [application/octet-stream]\\n\",\"Saving to: ‘model_data/yolov4_tiny_weights_coco.pth’\\n\",\"\\n\",\"model_data/yolov4_t 100%[===================>] 23.15M 40.3MB/s in 0.6s \\n\",\"\\n\",\"2022-04-28 00:40:50 (40.3 MB/s) - ‘model_data/yolov4_tiny_weights_coco.pth’ saved [24274351/24274351]\\n\",\"\\n\"]}]},{\"cell_type\":\"code\",\"source\":[\"# 训练数据，默认为100个epochs\\n\",\"!python train.py --tiny --phi 0 --epochs 100 \\\\\\n\",\" --weights model_data/yolov4_tiny_weights_coco.pth \\\\\\n\",\" --freeze --freeze-epochs 50 --freeze-size 64 \\\\\\n\",\" --batch-size 32 --shape 416 \\\\\\n\",\" --fp16 --cuda\"],\"metadata\":{\"colab\":{\"base_uri\":\"https://localhost:8080/\"},\"id\":\"BCQ-ywOulHkQ\",\"executionInfo\":{\"status\":\"ok\",\"timestamp\":1651112371419,\"user_tz\":-480,\"elapsed\":5920782,\"user\":{\"displayName\":\"KaiJun Deng\",\"userId\":\"04642544504944131029\"}},\"outputId\":\"df858913-5b20-4a74-d7f9-d30cb4491c4e\"},\"execution_count\":29,\"outputs\":[{\"output_type\":\"stream\",\"name\":\"stdout\",\"text\":[\"Namespace(batch_size=32, cuda=True, epochs=100, fp16=True, freeze=True, freeze_epochs=50, freeze_size=64, init=0, lr=0.02, momentum=0.937, mosaic=False, optimizer='adam', phi=0, save_period=4, shape=416, tiny=True, weight_decay=0, weights='model_data/yolov4_tiny_weights_coco.pth')\\n\",\"initialize network with normal type\\n\",\"Load weights model_data/yolov4_tiny_weights_coco.pth.\\n\",\"Start Train\\n\",\"Epoch 1/100: 100% 22/22 [00:50<00:00, 2.31s/it, loss=4.66, lr=0.0001]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 1/100: 100% 2/2 [00:08<00:00, 4.01s/it, val_loss=3.98]\\n\",\"Finish Validation\\n\",\"Epoch:1/100\\n\",\"Total Loss: 4.655 || Val Loss: 3.979 \\n\",\"Start Train\\n\",\"Epoch 2/100: 100% 22/22 [00:49<00:00, 2.24s/it, loss=3.14, lr=0.0002]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 2/100: 100% 2/2 [00:05<00:00, 2.52s/it, val_loss=2.24]\\n\",\"Finish Validation\\n\",\"Epoch:2/100\\n\",\"Total Loss: 3.143 || Val Loss: 2.238 \\n\",\"Start Train\\n\",\"Epoch 3/100: 100% 22/22 [00:49<00:00, 2.24s/it, loss=1.5, lr=0.0005]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 3/100: 100% 2/2 [00:05<00:00, 2.95s/it, val_loss=0.721]\\n\",\"Finish Validation\\n\",\"Epoch:3/100\\n\",\"Total Loss: 1.502 || Val Loss: 0.721 \\n\",\"Start Train\\n\",\"Epoch 4/100: 100% 22/22 [00:49<00:00, 2.26s/it, loss=0.506, lr=0.001]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 4/100: 100% 2/2 [00:05<00:00, 2.93s/it, val_loss=0.204]\\n\",\"Finish Validation\\n\",\"Epoch:4/100\\n\",\"Total Loss: 0.506 || Val Loss: 0.204 \\n\",\"Start Train\\n\",\"Epoch 5/100: 100% 22/22 [00:50<00:00, 2.30s/it, loss=0.231, lr=0.001]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 5/100: 100% 2/2 [00:06<00:00, 3.10s/it, val_loss=0.131]\\n\",\"Finish Validation\\n\",\"Epoch:5/100\\n\",\"Total Loss: 0.231 || Val Loss: 0.131 \\n\",\"Start Train\\n\",\"Epoch 6/100: 100% 22/22 [00:49<00:00, 2.25s/it, loss=0.174, lr=0.000999]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 6/100: 100% 2/2 [00:05<00:00, 2.99s/it, val_loss=0.107]\\n\",\"Finish Validation\\n\",\"Epoch:6/100\\n\",\"Total Loss: 0.174 || Val Loss: 0.107 \\n\",\"Start Train\\n\",\"Epoch 7/100: 100% 22/22 [00:50<00:00, 2.28s/it, loss=0.15, lr=0.000997]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 7/100: 100% 2/2 [00:05<00:00, 2.79s/it, val_loss=0.0895]\\n\",\"Finish Validation\\n\",\"Epoch:7/100\\n\",\"Total Loss: 0.150 || Val Loss: 0.089 \\n\",\"Start Train\\n\",\"Epoch 8/100: 100% 22/22 [00:49<00:00, 2.27s/it, loss=0.134, lr=0.000995]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 8/100: 100% 2/2 [00:05<00:00, 2.78s/it, val_loss=0.0784]\\n\",\"Finish Validation\\n\",\"Epoch:8/100\\n\",\"Total Loss: 0.134 || Val Loss: 0.078 \\n\",\"Start Train\\n\",\"Epoch 9/100: 100% 22/22 [00:49<00:00, 2.23s/it, loss=0.126, lr=0.000993]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 9/100: 100% 2/2 [00:05<00:00, 2.81s/it, val_loss=0.0721]\\n\",\"Finish Validation\\n\",\"Epoch:9/100\\n\",\"Total Loss: 0.126 || Val Loss: 0.072 \\n\",\"Start Train\\n\",\"Epoch 10/100: 100% 22/22 [00:48<00:00, 2.21s/it, loss=0.117, lr=0.00099]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 10/100: 100% 2/2 [00:05<00:00, 2.92s/it, val_loss=0.0647]\\n\",\"Finish Validation\\n\",\"Epoch:10/100\\n\",\"Total Loss: 0.117 || Val Loss: 0.065 \\n\",\"Start Train\\n\",\"Epoch 11/100: 100% 22/22 [00:49<00:00, 2.24s/it, loss=0.11, lr=0.000986]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 11/100: 100% 2/2 [00:05<00:00, 2.73s/it, val_loss=0.061]\\n\",\"Finish Validation\\n\",\"Epoch:11/100\\n\",\"Total Loss: 0.110 || Val Loss: 0.061 \\n\",\"Start Train\\n\",\"Epoch 12/100: 100% 22/22 [00:48<00:00, 2.22s/it, loss=0.102, lr=0.000982]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 12/100: 100% 2/2 [00:06<00:00, 3.08s/it, val_loss=0.057]\\n\",\"Finish Validation\\n\",\"Epoch:12/100\\n\",\"Total Loss: 0.102 || Val Loss: 0.057 \\n\",\"Start Train\\n\",\"Epoch 13/100: 100% 22/22 [00:50<00:00, 2.29s/it, loss=0.0965, lr=0.000977]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 13/100: 100% 2/2 [00:05<00:00, 2.82s/it, val_loss=0.0537]\\n\",\"Finish Validation\\n\",\"Epoch:13/100\\n\",\"Total Loss: 0.097 || Val Loss: 0.054 \\n\",\"Start Train\\n\",\"Epoch 14/100: 100% 22/22 [00:49<00:00, 2.25s/it, loss=0.0927, lr=0.000971]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 14/100: 100% 2/2 [00:05<00:00, 2.64s/it, val_loss=0.0532]\\n\",\"Finish Validation\\n\",\"Epoch:14/100\\n\",\"Total Loss: 0.093 || Val Loss: 0.053 \\n\",\"Start Train\\n\",\"Epoch 15/100: 100% 22/22 [00:51<00:00, 2.33s/it, loss=0.0896, lr=0.000965]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 15/100: 100% 2/2 [00:05<00:00, 2.64s/it, val_loss=0.0509]\\n\",\"Finish Validation\\n\",\"Epoch:15/100\\n\",\"Total Loss: 0.090 || Val Loss: 0.051 \\n\",\"Start Train\\n\",\"Epoch 16/100: 100% 22/22 [00:50<00:00, 2.29s/it, loss=0.0878, lr=0.000959]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 16/100: 100% 2/2 [00:05<00:00, 2.81s/it, val_loss=0.0487]\\n\",\"Finish Validation\\n\",\"Epoch:16/100\\n\",\"Total Loss: 0.088 || Val Loss: 0.049 \\n\",\"Start Train\\n\",\"Epoch 17/100: 100% 22/22 [00:50<00:00, 2.30s/it, loss=0.0814, lr=0.000952]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 17/100: 100% 2/2 [00:05<00:00, 2.84s/it, val_loss=0.0459]\\n\",\"Finish Validation\\n\",\"Epoch:17/100\\n\",\"Total Loss: 0.081 || Val Loss: 0.046 \\n\",\"Start Train\\n\",\"Epoch 18/100: 100% 22/22 [00:51<00:00, 2.33s/it, loss=0.0821, lr=0.000945]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 18/100: 100% 2/2 [00:06<00:00, 3.15s/it, val_loss=0.0435]\\n\",\"Finish Validation\\n\",\"Epoch:18/100\\n\",\"Total Loss: 0.082 || Val Loss: 0.044 \\n\",\"Start Train\\n\",\"Epoch 19/100: 100% 22/22 [00:50<00:00, 2.30s/it, loss=0.078, lr=0.000936]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 19/100: 100% 2/2 [00:05<00:00, 2.55s/it, val_loss=0.0411]\\n\",\"Finish Validation\\n\",\"Epoch:19/100\\n\",\"Total Loss: 0.078 || Val Loss: 0.041 \\n\",\"Start Train\\n\",\"Epoch 20/100: 100% 22/22 [00:50<00:00, 2.29s/it, loss=0.0775, lr=0.000928]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 20/100: 100% 2/2 [00:05<00:00, 2.78s/it, val_loss=0.0421]\\n\",\"Finish Validation\\n\",\"Epoch:20/100\\n\",\"Total Loss: 0.077 || Val Loss: 0.042 \\n\",\"Start Train\\n\",\"Epoch 21/100: 100% 22/22 [00:51<00:00, 2.35s/it, loss=0.0774, lr=0.000919]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 21/100: 100% 2/2 [00:05<00:00, 2.95s/it, val_loss=0.0376]\\n\",\"Finish Validation\\n\",\"Epoch:21/100\\n\",\"Total Loss: 0.077 || Val Loss: 0.038 \\n\",\"Start Train\\n\",\"Epoch 22/100: 100% 22/22 [00:50<00:00, 2.29s/it, loss=0.0732, lr=0.000909]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 22/100: 100% 2/2 [00:05<00:00, 2.94s/it, val_loss=0.037]\\n\",\"Finish Validation\\n\",\"Epoch:22/100\\n\",\"Total Loss: 0.073 || Val Loss: 0.037 \\n\",\"Start Train\\n\",\"Epoch 23/100: 100% 22/22 [00:50<00:00, 2.30s/it, loss=0.0717, lr=0.000899]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 23/100: 100% 2/2 [00:05<00:00, 2.87s/it, val_loss=0.037]\\n\",\"Finish Validation\\n\",\"Epoch:23/100\\n\",\"Total Loss: 0.072 || Val Loss: 0.037 \\n\",\"Start Train\\n\",\"Epoch 24/100: 100% 22/22 [00:50<00:00, 2.28s/it, loss=0.0702, lr=0.000889]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 24/100: 100% 2/2 [00:05<00:00, 2.64s/it, val_loss=0.035]\\n\",\"Finish Validation\\n\",\"Epoch:24/100\\n\",\"Total Loss: 0.070 || Val Loss: 0.035 \\n\",\"Start Train\\n\",\"Epoch 25/100: 100% 22/22 [00:49<00:00, 2.27s/it, loss=0.0702, lr=0.000878]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 25/100: 100% 2/2 [00:05<00:00, 2.72s/it, val_loss=0.0355]\\n\",\"Finish Validation\\n\",\"Epoch:25/100\\n\",\"Total Loss: 0.070 || Val Loss: 0.036 \\n\",\"Start Train\\n\",\"Epoch 26/100: 100% 22/22 [00:49<00:00, 2.23s/it, loss=0.0671, lr=0.000867]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 26/100: 100% 2/2 [00:05<00:00, 2.86s/it, val_loss=0.0346]\\n\",\"Finish Validation\\n\",\"Epoch:26/100\\n\",\"Total Loss: 0.067 || Val Loss: 0.035 \\n\",\"Start Train\\n\",\"Epoch 27/100: 100% 22/22 [00:49<00:00, 2.26s/it, loss=0.0673, lr=0.000855]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 27/100: 100% 2/2 [00:05<00:00, 2.81s/it, val_loss=0.0361]\\n\",\"Finish Validation\\n\",\"Epoch:27/100\\n\",\"Total Loss: 0.067 || Val Loss: 0.036 \\n\",\"Start Train\\n\",\"Epoch 28/100: 100% 22/22 [00:51<00:00, 2.33s/it, loss=0.0674, lr=0.000843]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 28/100: 100% 2/2 [00:05<00:00, 2.71s/it, val_loss=0.0349]\\n\",\"Finish Validation\\n\",\"Epoch:28/100\\n\",\"Total Loss: 0.067 || Val Loss: 0.035 \\n\",\"Start Train\\n\",\"Epoch 29/100: 100% 22/22 [00:49<00:00, 2.25s/it, loss=0.0664, lr=0.00083]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 29/100: 100% 2/2 [00:05<00:00, 2.88s/it, val_loss=0.0317]\\n\",\"Finish Validation\\n\",\"Epoch:29/100\\n\",\"Total Loss: 0.066 || Val Loss: 0.032 \\n\",\"Start Train\\n\",\"Epoch 30/100: 100% 22/22 [00:49<00:00, 2.25s/it, loss=0.0637, lr=0.000817]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 30/100: 100% 2/2 [00:05<00:00, 2.93s/it, val_loss=0.034]\\n\",\"Finish Validation\\n\",\"Epoch:30/100\\n\",\"Total Loss: 0.064 || Val Loss: 0.034 \\n\",\"Start Train\\n\",\"Epoch 31/100: 100% 22/22 [00:49<00:00, 2.26s/it, loss=0.0616, lr=0.000804]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 31/100: 100% 2/2 [00:05<00:00, 2.85s/it, val_loss=0.034]\\n\",\"Finish Validation\\n\",\"Epoch:31/100\\n\",\"Total Loss: 0.062 || Val Loss: 0.034 \\n\",\"Start Train\\n\",\"Epoch 32/100: 100% 22/22 [00:49<00:00, 2.27s/it, loss=0.0623, lr=0.00079]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 32/100: 100% 2/2 [00:04<00:00, 2.36s/it, val_loss=0.0335]\\n\",\"Finish Validation\\n\",\"Epoch:32/100\\n\",\"Total Loss: 0.062 || Val Loss: 0.034 \\n\",\"Start Train\\n\",\"Epoch 33/100: 100% 22/22 [00:50<00:00, 2.30s/it, loss=0.0609, lr=0.000776]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 33/100: 100% 2/2 [00:05<00:00, 2.78s/it, val_loss=0.031]\\n\",\"Finish Validation\\n\",\"Epoch:33/100\\n\",\"Total Loss: 0.061 || Val Loss: 0.031 \\n\",\"Start Train\\n\",\"Epoch 34/100: 100% 22/22 [00:49<00:00, 2.26s/it, loss=0.0591, lr=0.000762]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 34/100: 100% 2/2 [00:05<00:00, 2.74s/it, val_loss=0.0297]\\n\",\"Finish Validation\\n\",\"Epoch:34/100\\n\",\"Total Loss: 0.059 || Val Loss: 0.030 \\n\",\"Start Train\\n\",\"Epoch 35/100: 100% 22/22 [00:49<00:00, 2.27s/it, loss=0.0594, lr=0.000748]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 35/100: 100% 2/2 [00:05<00:00, 2.97s/it, val_loss=0.0296]\\n\",\"Finish Validation\\n\",\"Epoch:35/100\\n\",\"Total Loss: 0.059 || Val Loss: 0.030 \\n\",\"Start Train\\n\",\"Epoch 36/100: 100% 22/22 [00:49<00:00, 2.27s/it, loss=0.0599, lr=0.000733]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 36/100: 100% 2/2 [00:05<00:00, 2.85s/it, val_loss=0.0273]\\n\",\"Finish Validation\\n\",\"Epoch:36/100\\n\",\"Total Loss: 0.060 || Val Loss: 0.027 \\n\",\"Start Train\\n\",\"Epoch 37/100: 100% 22/22 [00:50<00:00, 2.29s/it, loss=0.0566, lr=0.000718]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 37/100: 100% 2/2 [00:05<00:00, 2.96s/it, val_loss=0.0275]\\n\",\"Finish Validation\\n\",\"Epoch:37/100\\n\",\"Total Loss: 0.057 || Val Loss: 0.027 \\n\",\"Start Train\\n\",\"Epoch 38/100: 100% 22/22 [00:49<00:00, 2.27s/it, loss=0.0554, lr=0.000702]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 38/100: 100% 2/2 [00:06<00:00, 3.11s/it, val_loss=0.0286]\\n\",\"Finish Validation\\n\",\"Epoch:38/100\\n\",\"Total Loss: 0.055 || Val Loss: 0.029 \\n\",\"Start Train\\n\",\"Epoch 39/100: 100% 22/22 [00:49<00:00, 2.27s/it, loss=0.0551, lr=0.000687]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 39/100: 100% 2/2 [00:05<00:00, 2.88s/it, val_loss=0.0277]\\n\",\"Finish Validation\\n\",\"Epoch:39/100\\n\",\"Total Loss: 0.055 || Val Loss: 0.028 \\n\",\"Start Train\\n\",\"Epoch 40/100: 100% 22/22 [00:50<00:00, 2.28s/it, loss=0.055, lr=0.000671]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 40/100: 100% 2/2 [00:06<00:00, 3.16s/it, val_loss=0.0303]\\n\",\"Finish Validation\\n\",\"Epoch:40/100\\n\",\"Total Loss: 0.055 || Val Loss: 0.030 \\n\",\"Start Train\\n\",\"Epoch 41/100: 100% 22/22 [00:51<00:00, 2.34s/it, loss=0.055, lr=0.000655]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 41/100: 100% 2/2 [00:05<00:00, 2.89s/it, val_loss=0.0265]\\n\",\"Finish Validation\\n\",\"Epoch:41/100\\n\",\"Total Loss: 0.055 || Val Loss: 0.026 \\n\",\"Start Train\\n\",\"Epoch 42/100: 100% 22/22 [00:50<00:00, 2.29s/it, loss=0.0548, lr=0.000639]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 42/100: 100% 2/2 [00:05<00:00, 2.91s/it, val_loss=0.0276]\\n\",\"Finish Validation\\n\",\"Epoch:42/100\\n\",\"Total Loss: 0.055 || Val Loss: 0.028 \\n\",\"Start Train\\n\",\"Epoch 43/100: 100% 22/22 [00:50<00:00, 2.30s/it, loss=0.0554, lr=0.000622]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 43/100: 100% 2/2 [00:05<00:00, 2.99s/it, val_loss=0.027]\\n\",\"Finish Validation\\n\",\"Epoch:43/100\\n\",\"Total Loss: 0.055 || Val Loss: 0.027 \\n\",\"Start Train\\n\",\"Epoch 44/100: 100% 22/22 [00:50<00:00, 2.29s/it, loss=0.0545, lr=0.000606]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 44/100: 100% 2/2 [00:05<00:00, 2.64s/it, val_loss=0.0266]\\n\",\"Finish Validation\\n\",\"Epoch:44/100\\n\",\"Total Loss: 0.054 || Val Loss: 0.027 \\n\",\"Start Train\\n\",\"Epoch 45/100: 100% 22/22 [00:49<00:00, 2.23s/it, loss=0.0523, lr=0.000589]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 45/100: 100% 2/2 [00:05<00:00, 2.81s/it, val_loss=0.0275]\\n\",\"Finish Validation\\n\",\"Epoch:45/100\\n\",\"Total Loss: 0.052 || Val Loss: 0.028 \\n\",\"Start Train\\n\",\"Epoch 46/100: 100% 22/22 [00:49<00:00, 2.24s/it, loss=0.0531, lr=0.000572]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 46/100: 100% 2/2 [00:05<00:00, 2.84s/it, val_loss=0.0268]\\n\",\"Finish Validation\\n\",\"Epoch:46/100\\n\",\"Total Loss: 0.053 || Val Loss: 0.027 \\n\",\"Start Train\\n\",\"Epoch 47/100: 100% 22/22 [00:49<00:00, 2.25s/it, loss=0.0528, lr=0.000556]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 47/100: 100% 2/2 [00:05<00:00, 2.87s/it, val_loss=0.0269]\\n\",\"Finish Validation\\n\",\"Epoch:47/100\\n\",\"Total Loss: 0.053 || Val Loss: 0.027 \\n\",\"Start Train\\n\",\"Epoch 48/100: 100% 22/22 [00:48<00:00, 2.20s/it, loss=0.0519, lr=0.000539]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 48/100: 100% 2/2 [00:05<00:00, 2.79s/it, val_loss=0.0273]\\n\",\"Finish Validation\\n\",\"Epoch:48/100\\n\",\"Total Loss: 0.052 || Val Loss: 0.027 \\n\",\"Start Train\\n\",\"Epoch 49/100: 100% 22/22 [00:49<00:00, 2.27s/it, loss=0.0522, lr=0.000522]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 49/100: 100% 2/2 [00:05<00:00, 2.98s/it, val_loss=0.0266]\\n\",\"Finish Validation\\n\",\"Epoch:49/100\\n\",\"Total Loss: 0.052 || Val Loss: 0.027 \\n\",\"Start Train\\n\",\"Epoch 50/100: 100% 22/22 [00:49<00:00, 2.24s/it, loss=0.0484, lr=0.000505]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 50/100: 100% 2/2 [00:05<00:00, 2.68s/it, val_loss=0.0261]\\n\",\"Finish Validation\\n\",\"Epoch:50/100\\n\",\"Total Loss: 0.048 || Val Loss: 0.026 \\n\",\"Start Train\\n\",\"Epoch 51/100: 100% 45/45 [00:56<00:00, 1.26s/it, loss=0.0714, lr=0.000488]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 51/100: 100% 5/5 [00:06<00:00, 1.40s/it, val_loss=0.0297]\\n\",\"Finish Validation\\n\",\"Epoch:51/100\\n\",\"Total Loss: 0.071 || Val Loss: 0.030 \\n\",\"Start Train\\n\",\"Epoch 52/100: 100% 45/45 [00:54<00:00, 1.21s/it, loss=0.0658, lr=0.000471]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 52/100: 100% 5/5 [00:06<00:00, 1.34s/it, val_loss=0.0318]\\n\",\"Finish Validation\\n\",\"Epoch:52/100\\n\",\"Total Loss: 0.066 || Val Loss: 0.032 \\n\",\"Start Train\\n\",\"Epoch 53/100: 100% 45/45 [00:54<00:00, 1.20s/it, loss=0.0634, lr=0.000454]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 53/100: 100% 5/5 [00:06<00:00, 1.38s/it, val_loss=0.0258]\\n\",\"Finish Validation\\n\",\"Epoch:53/100\\n\",\"Total Loss: 0.063 || Val Loss: 0.026 \\n\",\"Start Train\\n\",\"Epoch 54/100: 100% 45/45 [00:54<00:00, 1.21s/it, loss=0.0582, lr=0.000438]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 54/100: 100% 5/5 [00:06<00:00, 1.35s/it, val_loss=0.0265]\\n\",\"Finish Validation\\n\",\"Epoch:54/100\\n\",\"Total Loss: 0.058 || Val Loss: 0.027 \\n\",\"Start Train\\n\",\"Epoch 55/100: 100% 45/45 [00:56<00:00, 1.25s/it, loss=0.0601, lr=0.000421]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 55/100: 100% 5/5 [00:07<00:00, 1.40s/it, val_loss=0.0288]\\n\",\"Finish Validation\\n\",\"Epoch:55/100\\n\",\"Total Loss: 0.060 || Val Loss: 0.029 \\n\",\"Start Train\\n\",\"Epoch 56/100: 100% 45/45 [00:54<00:00, 1.22s/it, loss=0.0558, lr=0.000404]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 56/100: 100% 5/5 [00:06<00:00, 1.31s/it, val_loss=0.0237]\\n\",\"Finish Validation\\n\",\"Epoch:56/100\\n\",\"Total Loss: 0.056 || Val Loss: 0.024 \\n\",\"Start Train\\n\",\"Epoch 57/100: 100% 45/45 [00:54<00:00, 1.21s/it, loss=0.0531, lr=0.000388]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 57/100: 100% 5/5 [00:06<00:00, 1.32s/it, val_loss=0.0244]\\n\",\"Finish Validation\\n\",\"Epoch:57/100\\n\",\"Total Loss: 0.053 || Val Loss: 0.024 \\n\",\"Start Train\\n\",\"Epoch 58/100: 100% 45/45 [00:55<00:00, 1.22s/it, loss=0.0523, lr=0.000371]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 58/100: 100% 5/5 [00:06<00:00, 1.36s/it, val_loss=0.024]\\n\",\"Finish Validation\\n\",\"Epoch:58/100\\n\",\"Total Loss: 0.052 || Val Loss: 0.024 \\n\",\"Start Train\\n\",\"Epoch 59/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.047, lr=0.000355]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 59/100: 100% 5/5 [00:06<00:00, 1.35s/it, val_loss=0.0222]\\n\",\"Finish Validation\\n\",\"Epoch:59/100\\n\",\"Total Loss: 0.047 || Val Loss: 0.022 \\n\",\"Start Train\\n\",\"Epoch 60/100: 100% 45/45 [00:54<00:00, 1.22s/it, loss=0.0457, lr=0.000339]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 60/100: 100% 5/5 [00:06<00:00, 1.35s/it, val_loss=0.0229]\\n\",\"Finish Validation\\n\",\"Epoch:60/100\\n\",\"Total Loss: 0.046 || Val Loss: 0.023 \\n\",\"Start Train\\n\",\"Epoch 61/100: 100% 45/45 [00:55<00:00, 1.22s/it, loss=0.0465, lr=0.000323]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 61/100: 100% 5/5 [00:06<00:00, 1.35s/it, val_loss=0.0191]\\n\",\"Finish Validation\\n\",\"Epoch:61/100\\n\",\"Total Loss: 0.047 || Val Loss: 0.019 \\n\",\"Start Train\\n\",\"Epoch 62/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.0412, lr=0.000308]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 62/100: 100% 5/5 [00:06<00:00, 1.37s/it, val_loss=0.021]\\n\",\"Finish Validation\\n\",\"Epoch:62/100\\n\",\"Total Loss: 0.041 || Val Loss: 0.021 \\n\",\"Start Train\\n\",\"Epoch 63/100: 100% 45/45 [00:54<00:00, 1.21s/it, loss=0.0407, lr=0.000292]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 63/100: 100% 5/5 [00:06<00:00, 1.34s/it, val_loss=0.0206]\\n\",\"Finish Validation\\n\",\"Epoch:63/100\\n\",\"Total Loss: 0.041 || Val Loss: 0.021 \\n\",\"Start Train\\n\",\"Epoch 64/100: 100% 45/45 [00:56<00:00, 1.25s/it, loss=0.0416, lr=0.000277]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 64/100: 100% 5/5 [00:07<00:00, 1.41s/it, val_loss=0.0204]\\n\",\"Finish Validation\\n\",\"Epoch:64/100\\n\",\"Total Loss: 0.042 || Val Loss: 0.020 \\n\",\"Start Train\\n\",\"Epoch 65/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.0382, lr=0.000262]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 65/100: 100% 5/5 [00:07<00:00, 1.41s/it, val_loss=0.0192]\\n\",\"Finish Validation\\n\",\"Epoch:65/100\\n\",\"Total Loss: 0.038 || Val Loss: 0.019 \\n\",\"Start Train\\n\",\"Epoch 66/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.039, lr=0.000248]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 66/100: 100% 5/5 [00:06<00:00, 1.31s/it, val_loss=0.0193]\\n\",\"Finish Validation\\n\",\"Epoch:66/100\\n\",\"Total Loss: 0.039 || Val Loss: 0.019 \\n\",\"Start Train\\n\",\"Epoch 67/100: 100% 45/45 [00:55<00:00, 1.22s/it, loss=0.0418, lr=0.000234]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 67/100: 100% 5/5 [00:07<00:00, 1.41s/it, val_loss=0.0203]\\n\",\"Finish Validation\\n\",\"Epoch:67/100\\n\",\"Total Loss: 0.042 || Val Loss: 0.020 \\n\",\"Start Train\\n\",\"Epoch 68/100: 100% 45/45 [00:55<00:00, 1.22s/it, loss=0.0389, lr=0.00022]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 68/100: 100% 5/5 [00:06<00:00, 1.36s/it, val_loss=0.0194]\\n\",\"Finish Validation\\n\",\"Epoch:68/100\\n\",\"Total Loss: 0.039 || Val Loss: 0.019 \\n\",\"Start Train\\n\",\"Epoch 69/100: 100% 45/45 [00:54<00:00, 1.22s/it, loss=0.0372, lr=0.000206]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 69/100: 100% 5/5 [00:06<00:00, 1.32s/it, val_loss=0.019]\\n\",\"Finish Validation\\n\",\"Epoch:69/100\\n\",\"Total Loss: 0.037 || Val Loss: 0.019 \\n\",\"Start Train\\n\",\"Epoch 70/100: 100% 45/45 [00:54<00:00, 1.22s/it, loss=0.0353, lr=0.000193]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 70/100: 100% 5/5 [00:06<00:00, 1.35s/it, val_loss=0.0173]\\n\",\"Finish Validation\\n\",\"Epoch:70/100\\n\",\"Total Loss: 0.035 || Val Loss: 0.017 \\n\",\"Start Train\\n\",\"Epoch 71/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.0344, lr=0.00018]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 71/100: 100% 5/5 [00:06<00:00, 1.34s/it, val_loss=0.0178]\\n\",\"Finish Validation\\n\",\"Epoch:71/100\\n\",\"Total Loss: 0.034 || Val Loss: 0.018 \\n\",\"Start Train\\n\",\"Epoch 72/100: 100% 45/45 [00:53<00:00, 1.20s/it, loss=0.0338, lr=0.000167]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 72/100: 100% 5/5 [00:06<00:00, 1.34s/it, val_loss=0.0181]\\n\",\"Finish Validation\\n\",\"Epoch:72/100\\n\",\"Total Loss: 0.034 || Val Loss: 0.018 \\n\",\"Start Train\\n\",\"Epoch 73/100: 100% 45/45 [00:54<00:00, 1.20s/it, loss=0.0345, lr=0.000155]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 73/100: 100% 5/5 [00:06<00:00, 1.36s/it, val_loss=0.0183]\\n\",\"Finish Validation\\n\",\"Epoch:73/100\\n\",\"Total Loss: 0.034 || Val Loss: 0.018 \\n\",\"Start Train\\n\",\"Epoch 74/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.032, lr=0.000143]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 74/100: 100% 5/5 [00:06<00:00, 1.35s/it, val_loss=0.0172]\\n\",\"Finish Validation\\n\",\"Epoch:74/100\\n\",\"Total Loss: 0.032 || Val Loss: 0.017 \\n\",\"Start Train\\n\",\"Epoch 75/100: 100% 45/45 [00:53<00:00, 1.20s/it, loss=0.0349, lr=0.000132]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 75/100: 100% 5/5 [00:06<00:00, 1.35s/it, val_loss=0.0164]\\n\",\"Finish Validation\\n\",\"Epoch:75/100\\n\",\"Total Loss: 0.035 || Val Loss: 0.016 \\n\",\"Start Train\\n\",\"Epoch 76/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.0299, lr=0.000121]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 76/100: 100% 5/5 [00:06<00:00, 1.38s/it, val_loss=0.0167]\\n\",\"Finish Validation\\n\",\"Epoch:76/100\\n\",\"Total Loss: 0.030 || Val Loss: 0.017 \\n\",\"Start Train\\n\",\"Epoch 77/100: 100% 45/45 [00:54<00:00, 1.21s/it, loss=0.0317, lr=0.000111]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 77/100: 100% 5/5 [00:06<00:00, 1.35s/it, val_loss=0.0166]\\n\",\"Finish Validation\\n\",\"Epoch:77/100\\n\",\"Total Loss: 0.032 || Val Loss: 0.017 \\n\",\"Start Train\\n\",\"Epoch 78/100: 100% 45/45 [00:54<00:00, 1.21s/it, loss=0.0318, lr=0.000101]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 78/100: 100% 5/5 [00:06<00:00, 1.38s/it, val_loss=0.0166]\\n\",\"Finish Validation\\n\",\"Epoch:78/100\\n\",\"Total Loss: 0.032 || Val Loss: 0.017 \\n\",\"Start Train\\n\",\"Epoch 79/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.0303, lr=9.11e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 79/100: 100% 5/5 [00:06<00:00, 1.35s/it, val_loss=0.0157]\\n\",\"Finish Validation\\n\",\"Epoch:79/100\\n\",\"Total Loss: 0.030 || Val Loss: 0.016 \\n\",\"Start Train\\n\",\"Epoch 80/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.0292, lr=8.21e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 80/100: 100% 5/5 [00:06<00:00, 1.34s/it, val_loss=0.016]\\n\",\"Finish Validation\\n\",\"Epoch:80/100\\n\",\"Total Loss: 0.029 || Val Loss: 0.016 \\n\",\"Start Train\\n\",\"Epoch 81/100: 100% 45/45 [00:54<00:00, 1.22s/it, loss=0.0293, lr=7.35e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 81/100: 100% 5/5 [00:06<00:00, 1.33s/it, val_loss=0.016]\\n\",\"Finish Validation\\n\",\"Epoch:81/100\\n\",\"Total Loss: 0.029 || Val Loss: 0.016 \\n\",\"Start Train\\n\",\"Epoch 82/100: 100% 45/45 [00:55<00:00, 1.24s/it, loss=0.0292, lr=6.55e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 82/100: 100% 5/5 [00:06<00:00, 1.38s/it, val_loss=0.0155]\\n\",\"Finish Validation\\n\",\"Epoch:82/100\\n\",\"Total Loss: 0.029 || Val Loss: 0.016 \\n\",\"Start Train\\n\",\"Epoch 83/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.0276, lr=5.8e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 83/100: 100% 5/5 [00:06<00:00, 1.34s/it, val_loss=0.0157]\\n\",\"Finish Validation\\n\",\"Epoch:83/100\\n\",\"Total Loss: 0.028 || Val Loss: 0.016 \\n\",\"Start Train\\n\",\"Epoch 84/100: 100% 45/45 [00:55<00:00, 1.24s/it, loss=0.0274, lr=5.1e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 84/100: 100% 5/5 [00:07<00:00, 1.42s/it, val_loss=0.0162]\\n\",\"Finish Validation\\n\",\"Epoch:84/100\\n\",\"Total Loss: 0.027 || Val Loss: 0.016 \\n\",\"Start Train\\n\",\"Epoch 85/100: 100% 45/45 [00:54<00:00, 1.22s/it, loss=0.027, lr=4.45e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 85/100: 100% 5/5 [00:06<00:00, 1.37s/it, val_loss=0.0157]\\n\",\"Finish Validation\\n\",\"Epoch:85/100\\n\",\"Total Loss: 0.027 || Val Loss: 0.016 \\n\",\"Start Train\\n\",\"Epoch 86/100: 100% 45/45 [00:53<00:00, 1.20s/it, loss=0.0277, lr=3.86e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 86/100: 100% 5/5 [00:06<00:00, 1.37s/it, val_loss=0.0155]\\n\",\"Finish Validation\\n\",\"Epoch:86/100\\n\",\"Total Loss: 0.028 || Val Loss: 0.015 \\n\",\"Start Train\\n\",\"Epoch 87/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.0266, lr=3.32e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 87/100: 100% 5/5 [00:06<00:00, 1.35s/it, val_loss=0.0155]\\n\",\"Finish Validation\\n\",\"Epoch:87/100\\n\",\"Total Loss: 0.027 || Val Loss: 0.015 \\n\",\"Start Train\\n\",\"Epoch 88/100: 100% 45/45 [00:56<00:00, 1.25s/it, loss=0.0275, lr=2.84e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 88/100: 100% 5/5 [00:06<00:00, 1.40s/it, val_loss=0.0153]\\n\",\"Finish Validation\\n\",\"Epoch:88/100\\n\",\"Total Loss: 0.028 || Val Loss: 0.015 \\n\",\"Start Train\\n\",\"Epoch 89/100: 100% 45/45 [00:56<00:00, 1.25s/it, loss=0.0259, lr=2.41e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 89/100: 100% 5/5 [00:06<00:00, 1.35s/it, val_loss=0.0153]\\n\",\"Finish Validation\\n\",\"Epoch:89/100\\n\",\"Total Loss: 0.026 || Val Loss: 0.015 \\n\",\"Start Train\\n\",\"Epoch 90/100: 100% 45/45 [00:56<00:00, 1.26s/it, loss=0.0256, lr=2.04e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 91/100: 100% 45/45 [00:56<00:00, 1.25s/it, loss=0.0262, lr=1.72e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 91/100: 100% 5/5 [00:06<00:00, 1.36s/it, val_loss=0.0155]\\n\",\"Finish Validation\\n\",\"Epoch:91/100\\n\",\"Total Loss: 0.026 || Val Loss: 0.015 \\n\",\"Start Train\\n\",\"Epoch 92/100: 100% 45/45 [00:56<00:00, 1.25s/it, loss=0.0255, lr=1.46e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 92/100: 100% 5/5 [00:06<00:00, 1.37s/it, val_loss=0.0155]\\n\",\"Finish Validation\\n\",\"Epoch:92/100\\n\",\"Total Loss: 0.026 || Val Loss: 0.015 \\n\",\"Start Train\\n\",\"Epoch 93/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.0258, lr=1.26e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 93/100: 100% 5/5 [00:06<00:00, 1.35s/it, val_loss=0.0153]\\n\",\"Finish Validation\\n\",\"Epoch:93/100\\n\",\"Total Loss: 0.026 || Val Loss: 0.015 \\n\",\"Start Train\\n\",\"Epoch 94/100: 100% 45/45 [00:56<00:00, 1.25s/it, loss=0.0264, lr=1.12e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 94/100: 100% 5/5 [00:06<00:00, 1.31s/it, val_loss=0.0149]\\n\",\"Finish Validation\\n\",\"Epoch:94/100\\n\",\"Total Loss: 0.026 || Val Loss: 0.015 \\n\",\"Start Train\\n\",\"Epoch 95/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.0275, lr=1.03e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 95/100: 100% 5/5 [00:06<00:00, 1.37s/it, val_loss=0.015]\\n\",\"Finish Validation\\n\",\"Epoch:95/100\\n\",\"Total Loss: 0.028 || Val Loss: 0.015 \\n\",\"Start Train\\n\",\"Epoch 96/100: 100% 45/45 [00:55<00:00, 1.22s/it, loss=0.0244, lr=1e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 96/100: 100% 5/5 [00:06<00:00, 1.37s/it, val_loss=0.0151]\\n\",\"Finish Validation\\n\",\"Epoch:96/100\\n\",\"Total Loss: 0.024 || Val Loss: 0.015 \\n\",\"Start Train\\n\",\"Epoch 97/100: 100% 45/45 [00:53<00:00, 1.20s/it, loss=0.0253, lr=1e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 97/100: 100% 5/5 [00:06<00:00, 1.39s/it, val_loss=0.0151]\\n\",\"Finish Validation\\n\",\"Epoch:97/100\\n\",\"Total Loss: 0.025 || Val Loss: 0.015 \\n\",\"Start Train\\n\",\"Epoch 98/100: 100% 45/45 [00:56<00:00, 1.25s/it, loss=0.0268, lr=1e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 98/100: 100% 5/5 [00:06<00:00, 1.35s/it, val_loss=0.015]\\n\",\"Finish Validation\\n\",\"Epoch:98/100\\n\",\"Total Loss: 0.027 || Val Loss: 0.015 \\n\",\"Start Train\\n\",\"Epoch 99/100: 100% 45/45 [00:55<00:00, 1.22s/it, loss=0.0263, lr=1e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 99/100: 100% 5/5 [00:06<00:00, 1.34s/it, val_loss=0.0151]\\n\",\"Finish Validation\\n\",\"Epoch:99/100\\n\",\"Total Loss: 0.026 || Val Loss: 0.015 \\n\",\"Start Train\\n\",\"Epoch 100/100: 100% 45/45 [00:54<00:00, 1.21s/it, loss=0.0264, lr=1e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 100/100: 100% 5/5 [00:06<00:00, 1.37s/it, val_loss=0.015]\\n\",\"Finish Validation\\n\",\"Epoch:100/100\\n\",\"Total Loss: 0.026 || Val Loss: 0.015 \\n\"]}]},{\"cell_type\":\"markdown\",\"metadata\":{\"id\":\"Xdk8w9AgkX9x\"},\"source\":[\"### SE版本\"]},{\"cell_type\":\"code\",\"execution_count\":null,\"metadata\":{\"id\":\"pT9v141CkX9x\"},\"outputs\":[],\"source\":[\"# 训练数据，默认为100个epochs\\n\",\"!python train.py --tiny --phi 1 --epochs 100 \\\\\\n\",\" --weights model_data/yolotiny_SE_ep100.pth \\\\\\n\",\" --freeze --freeze-epochs 50 --freeze-size 64 \\\\\\n\",\" --batch-size 32 --shape 416 \\\\\\n\",\" --fp16 --cuda\"]},{\"cell_type\":\"markdown\",\"metadata\":{\"id\":\"LvfM3YsGkX9y\"},\"source\":[\"### CBAM版本\"]},{\"cell_type\":\"code\",\"execution_count\":null,\"metadata\":{\"colab\":{\"base_uri\":\"https://localhost:8080/\"},\"executionInfo\":{\"elapsed\":5705245,\"status\":\"ok\",\"timestamp\":1651069772733,\"user\":{\"displayName\":\"Pikachu Pika-pika\",\"userId\":\"18122152102369823184\"},\"user_tz\":-480},\"id\":\"0HRM9brRwa7U\",\"outputId\":\"979cb976-476d-4359-ace6-aaf29a99a201\"},\"outputs\":[{\"name\":\"stdout\",\"output_type\":\"stream\",\"text\":[\"Namespace(batch_size=32, cuda=True, epochs=100, fp16=True, freeze=True, freeze_epochs=50, freeze_size=64, lr=0.02, momentum=0.937, optimizer='adam', phi=2, save_period=4, shape=416, tiny=True, weight_decay=0, weights='model_data/yolov4_tiny_weights_voc_CBAM.pth')\\n\",\"initialize network with normal type\\n\",\"Load weights model_data/yolov4_tiny_weights_voc_CBAM.pth.\\n\",\"Start Train\\n\",\"Epoch 1/100: 100% 22/22 [00:52<00:00, 2.40s/it, loss=4.46, lr=0.0001]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 1/100: 100% 2/2 [00:07<00:00, 3.76s/it, val_loss=3.71]\\n\",\"Finish Validation\\n\",\"Epoch:1/100\\n\",\"Total Loss: 4.458 || Val Loss: 3.705 \\n\",\"Start Train\\n\",\"Epoch 2/100: 100% 22/22 [00:52<00:00, 2.38s/it, loss=2.73, lr=0.0002]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 2/100: 100% 2/2 [00:04<00:00, 2.45s/it, val_loss=1.83]\\n\",\"Finish Validation\\n\",\"Epoch:2/100\\n\",\"Total Loss: 2.726 || Val Loss: 1.826 \\n\",\"Start Train\\n\",\"Epoch 3/100: 100% 22/22 [00:51<00:00, 2.36s/it, loss=1.09, lr=0.0005]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 3/100: 100% 2/2 [00:04<00:00, 2.45s/it, val_loss=0.514]\\n\",\"Finish Validation\\n\",\"Epoch:3/100\\n\",\"Total Loss: 1.089 || Val Loss: 0.514 \\n\",\"Start Train\\n\",\"Epoch 4/100: 100% 22/22 [00:50<00:00, 2.30s/it, loss=0.331, lr=0.001]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 4/100: 100% 2/2 [00:05<00:00, 2.60s/it, val_loss=0.163]\\n\",\"Finish Validation\\n\",\"Epoch:4/100\\n\",\"Total Loss: 0.331 || Val Loss: 0.163 \\n\",\"Start Train\\n\",\"Epoch 5/100: 100% 22/22 [00:52<00:00, 2.39s/it, loss=0.171, lr=0.001]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 5/100: 100% 2/2 [00:05<00:00, 2.79s/it, val_loss=0.108]\\n\",\"Finish Validation\\n\",\"Epoch:5/100\\n\",\"Total Loss: 0.171 || Val Loss: 0.108 \\n\",\"Start Train\\n\",\"Epoch 6/100: 100% 22/22 [00:50<00:00, 2.31s/it, loss=0.123, lr=0.000999]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 6/100: 100% 2/2 [00:04<00:00, 2.18s/it, val_loss=0.0906]\\n\",\"Finish Validation\\n\",\"Epoch:6/100\\n\",\"Total Loss: 0.123 || Val Loss: 0.091 \\n\",\"Start Train\\n\",\"Epoch 7/100: 100% 22/22 [00:51<00:00, 2.35s/it, loss=0.108, lr=0.000997]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 7/100: 100% 2/2 [00:04<00:00, 2.42s/it, val_loss=0.0754]\\n\",\"Finish Validation\\n\",\"Epoch:7/100\\n\",\"Total Loss: 0.108 || Val Loss: 0.075 \\n\",\"Start Train\\n\",\"Epoch 8/100: 100% 22/22 [00:52<00:00, 2.38s/it, loss=0.109, lr=0.000995]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 8/100: 100% 2/2 [00:05<00:00, 2.59s/it, val_loss=0.0715]\\n\",\"Finish Validation\\n\",\"Epoch:8/100\\n\",\"Total Loss: 0.109 || Val Loss: 0.071 \\n\",\"Start Train\\n\",\"Epoch 9/100: 100% 22/22 [00:51<00:00, 2.34s/it, loss=0.108, lr=0.000993]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 9/100: 100% 2/2 [00:04<00:00, 2.30s/it, val_loss=0.0652]\\n\",\"Finish Validation\\n\",\"Epoch:9/100\\n\",\"Total Loss: 0.108 || Val Loss: 0.065 \\n\",\"Start Train\\n\",\"Epoch 10/100: 100% 22/22 [00:51<00:00, 2.33s/it, loss=0.0997, lr=0.00099]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 10/100: 100% 2/2 [00:05<00:00, 2.67s/it, val_loss=0.059]\\n\",\"Finish Validation\\n\",\"Epoch:10/100\\n\",\"Total Loss: 0.100 || Val Loss: 0.059 \\n\",\"Start Train\\n\",\"Epoch 11/100: 100% 22/22 [00:52<00:00, 2.38s/it, loss=0.0975, lr=0.000986]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 11/100: 100% 2/2 [00:05<00:00, 2.75s/it, val_loss=0.0543]\\n\",\"Finish Validation\\n\",\"Epoch:11/100\\n\",\"Total Loss: 0.097 || Val Loss: 0.054 \\n\",\"Start Train\\n\",\"Epoch 12/100: 100% 22/22 [00:51<00:00, 2.35s/it, loss=0.0905, lr=0.000982]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 12/100: 100% 2/2 [00:05<00:00, 2.55s/it, val_loss=0.0506]\\n\",\"Finish Validation\\n\",\"Epoch:12/100\\n\",\"Total Loss: 0.091 || Val Loss: 0.051 \\n\",\"Start Train\\n\",\"Epoch 13/100: 100% 22/22 [00:51<00:00, 2.34s/it, loss=0.0867, lr=0.000977]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 13/100: 100% 2/2 [00:04<00:00, 2.50s/it, val_loss=0.0505]\\n\",\"Finish Validation\\n\",\"Epoch:13/100\\n\",\"Total Loss: 0.087 || Val Loss: 0.050 \\n\",\"Start Train\\n\",\"Epoch 14/100: 100% 22/22 [00:50<00:00, 2.31s/it, loss=0.092, lr=0.000971]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 14/100: 100% 2/2 [00:04<00:00, 2.25s/it, val_loss=0.046]\\n\",\"Finish Validation\\n\",\"Epoch:14/100\\n\",\"Total Loss: 0.092 || Val Loss: 0.046 \\n\",\"Start Train\\n\",\"Epoch 15/100: 100% 22/22 [00:50<00:00, 2.32s/it, loss=0.0814, lr=0.000965]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 15/100: 100% 2/2 [00:05<00:00, 2.54s/it, val_loss=0.0426]\\n\",\"Finish Validation\\n\",\"Epoch:15/100\\n\",\"Total Loss: 0.081 || Val Loss: 0.043 \\n\",\"Start Train\\n\",\"Epoch 16/100: 100% 22/22 [00:50<00:00, 2.28s/it, loss=0.0829, lr=0.000959]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 16/100: 100% 2/2 [00:05<00:00, 2.50s/it, val_loss=0.0424]\\n\",\"Finish Validation\\n\",\"Epoch:16/100\\n\",\"Total Loss: 0.083 || Val Loss: 0.042 \\n\",\"Start Train\\n\",\"Epoch 17/100: 100% 22/22 [00:51<00:00, 2.33s/it, loss=0.0779, lr=0.000952]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 17/100: 100% 2/2 [00:05<00:00, 2.55s/it, val_loss=0.0402]\\n\",\"Finish Validation\\n\",\"Epoch:17/100\\n\",\"Total Loss: 0.078 || Val Loss: 0.040 \\n\",\"Start Train\\n\",\"Epoch 18/100: 100% 22/22 [00:49<00:00, 2.27s/it, loss=0.0754, lr=0.000945]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 18/100: 100% 2/2 [00:04<00:00, 2.38s/it, val_loss=0.0404]\\n\",\"Finish Validation\\n\",\"Epoch:18/100\\n\",\"Total Loss: 0.075 || Val Loss: 0.040 \\n\",\"Start Train\\n\",\"Epoch 19/100: 100% 22/22 [00:50<00:00, 2.31s/it, loss=0.0747, lr=0.000936]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 19/100: 100% 2/2 [00:05<00:00, 2.61s/it, val_loss=0.0357]\\n\",\"Finish Validation\\n\",\"Epoch:19/100\\n\",\"Total Loss: 0.075 || Val Loss: 0.036 \\n\",\"Start Train\\n\",\"Epoch 20/100: 100% 22/22 [00:50<00:00, 2.29s/it, loss=0.0707, lr=0.000928]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 20/100: 100% 2/2 [00:05<00:00, 2.63s/it, val_loss=0.038]\\n\",\"Finish Validation\\n\",\"Epoch:20/100\\n\",\"Total Loss: 0.071 || Val Loss: 0.038 \\n\",\"Start Train\\n\",\"Epoch 21/100: 100% 22/22 [00:50<00:00, 2.28s/it, loss=0.0686, lr=0.000919]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 21/100: 100% 2/2 [00:05<00:00, 2.69s/it, val_loss=0.0397]\\n\",\"Finish Validation\\n\",\"Epoch:21/100\\n\",\"Total Loss: 0.069 || Val Loss: 0.040 \\n\",\"Start Train\\n\",\"Epoch 22/100: 100% 22/22 [00:51<00:00, 2.36s/it, loss=0.058, lr=0.000909]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 22/100: 100% 2/2 [00:05<00:00, 2.53s/it, val_loss=0.0355]\\n\",\"Finish Validation\\n\",\"Epoch:22/100\\n\",\"Total Loss: 0.058 || Val Loss: 0.035 \\n\",\"Start Train\\n\",\"Epoch 23/100: 100% 22/22 [00:50<00:00, 2.32s/it, loss=0.07, lr=0.000899]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 23/100: 100% 2/2 [00:05<00:00, 2.72s/it, val_loss=0.032]\\n\",\"Finish Validation\\n\",\"Epoch:23/100\\n\",\"Total Loss: 0.070 || Val Loss: 0.032 \\n\",\"Start Train\\n\",\"Epoch 24/100: 100% 22/22 [00:50<00:00, 2.32s/it, loss=0.0646, lr=0.000889]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 24/100: 100% 2/2 [00:05<00:00, 2.64s/it, val_loss=0.03]\\n\",\"Finish Validation\\n\",\"Epoch:24/100\\n\",\"Total Loss: 0.065 || Val Loss: 0.030 \\n\",\"Start Train\\n\",\"Epoch 25/100: 100% 22/22 [00:49<00:00, 2.27s/it, loss=0.0636, lr=0.000878]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 25/100: 100% 2/2 [00:04<00:00, 2.16s/it, val_loss=0.0325]\\n\",\"Finish Validation\\n\",\"Epoch:25/100\\n\",\"Total Loss: 0.064 || Val Loss: 0.033 \\n\",\"Start Train\\n\",\"Epoch 26/100: 100% 22/22 [00:50<00:00, 2.28s/it, loss=0.064, lr=0.000867]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 26/100: 100% 2/2 [00:05<00:00, 2.70s/it, val_loss=0.033]\\n\",\"Finish Validation\\n\",\"Epoch:26/100\\n\",\"Total Loss: 0.064 || Val Loss: 0.033 \\n\",\"Start Train\\n\",\"Epoch 27/100: 100% 22/22 [00:49<00:00, 2.27s/it, loss=0.0621, lr=0.000855]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 27/100: 100% 2/2 [00:05<00:00, 2.54s/it, val_loss=0.0329]\\n\",\"Finish Validation\\n\",\"Epoch:27/100\\n\",\"Total Loss: 0.062 || Val Loss: 0.033 \\n\",\"Start Train\\n\",\"Epoch 28/100: 100% 22/22 [00:50<00:00, 2.31s/it, loss=0.0703, lr=0.000843]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 28/100: 100% 2/2 [00:05<00:00, 2.67s/it, val_loss=0.0291]\\n\",\"Finish Validation\\n\",\"Epoch:28/100\\n\",\"Total Loss: 0.070 || Val Loss: 0.029 \\n\",\"Start Train\\n\",\"Epoch 29/100: 100% 22/22 [00:51<00:00, 2.34s/it, loss=0.0564, lr=0.00083]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 29/100: 100% 2/2 [00:04<00:00, 2.48s/it, val_loss=0.0296]\\n\",\"Finish Validation\\n\",\"Epoch:29/100\\n\",\"Total Loss: 0.056 || Val Loss: 0.030 \\n\",\"Start Train\\n\",\"Epoch 30/100: 100% 22/22 [00:49<00:00, 2.27s/it, loss=0.0639, lr=0.000817]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 30/100: 100% 2/2 [00:04<00:00, 2.41s/it, val_loss=0.034]\\n\",\"Finish Validation\\n\",\"Epoch:30/100\\n\",\"Total Loss: 0.064 || Val Loss: 0.034 \\n\",\"Start Train\\n\",\"Epoch 31/100: 100% 22/22 [00:50<00:00, 2.31s/it, loss=0.0578, lr=0.000804]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 31/100: 100% 2/2 [00:05<00:00, 2.54s/it, val_loss=0.0297]\\n\",\"Finish Validation\\n\",\"Epoch:31/100\\n\",\"Total Loss: 0.058 || Val Loss: 0.030 \\n\",\"Start Train\\n\",\"Epoch 32/100: 100% 22/22 [00:50<00:00, 2.27s/it, loss=0.0641, lr=0.00079]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 32/100: 100% 2/2 [00:05<00:00, 2.65s/it, val_loss=0.0283]\\n\",\"Finish Validation\\n\",\"Epoch:32/100\\n\",\"Total Loss: 0.064 || Val Loss: 0.028 \\n\",\"Start Train\\n\",\"Epoch 33/100: 100% 22/22 [00:51<00:00, 2.32s/it, loss=0.0605, lr=0.000776]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 33/100: 100% 2/2 [00:04<00:00, 2.32s/it, val_loss=0.0276]\\n\",\"Finish Validation\\n\",\"Epoch:33/100\\n\",\"Total Loss: 0.060 || Val Loss: 0.028 \\n\",\"Start Train\\n\",\"Epoch 34/100: 100% 22/22 [00:50<00:00, 2.30s/it, loss=0.0463, lr=0.000762]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 34/100: 100% 2/2 [00:04<00:00, 2.43s/it, val_loss=0.0284]\\n\",\"Finish Validation\\n\",\"Epoch:34/100\\n\",\"Total Loss: 0.046 || Val Loss: 0.028 \\n\",\"Start Train\\n\",\"Epoch 35/100: 100% 22/22 [00:51<00:00, 2.34s/it, loss=0.0594, lr=0.000748]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 35/100: 100% 2/2 [00:05<00:00, 2.51s/it, val_loss=0.0273]\\n\",\"Finish Validation\\n\",\"Epoch:35/100\\n\",\"Total Loss: 0.059 || Val Loss: 0.027 \\n\",\"Start Train\\n\",\"Epoch 36/100: 100% 22/22 [00:51<00:00, 2.35s/it, loss=0.0573, lr=0.000733]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 36/100: 100% 2/2 [00:05<00:00, 2.51s/it, val_loss=0.027]\\n\",\"Finish Validation\\n\",\"Epoch:36/100\\n\",\"Total Loss: 0.057 || Val Loss: 0.027 \\n\",\"Start Train\\n\",\"Epoch 37/100: 100% 22/22 [00:50<00:00, 2.32s/it, loss=0.0538, lr=0.000718]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 37/100: 100% 2/2 [00:04<00:00, 2.32s/it, val_loss=0.0265]\\n\",\"Finish Validation\\n\",\"Epoch:37/100\\n\",\"Total Loss: 0.054 || Val Loss: 0.027 \\n\",\"Start Train\\n\",\"Epoch 38/100: 100% 22/22 [00:49<00:00, 2.26s/it, loss=0.0536, lr=0.000702]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 38/100: 100% 2/2 [00:05<00:00, 2.57s/it, val_loss=0.0278]\\n\",\"Finish Validation\\n\",\"Epoch:38/100\\n\",\"Total Loss: 0.054 || Val Loss: 0.028 \\n\",\"Start Train\\n\",\"Epoch 39/100: 100% 22/22 [00:50<00:00, 2.30s/it, loss=0.0535, lr=0.000687]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 39/100: 100% 2/2 [00:04<00:00, 2.36s/it, val_loss=0.0278]\\n\",\"Finish Validation\\n\",\"Epoch:39/100\\n\",\"Total Loss: 0.053 || Val Loss: 0.028 \\n\",\"Start Train\\n\",\"Epoch 40/100: 100% 22/22 [00:50<00:00, 2.30s/it, loss=0.0558, lr=0.000671]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 40/100: 100% 2/2 [00:04<00:00, 2.34s/it, val_loss=0.0257]\\n\",\"Finish Validation\\n\",\"Epoch:40/100\\n\",\"Total Loss: 0.056 || Val Loss: 0.026 \\n\",\"Start Train\\n\",\"Epoch 41/100: 100% 22/22 [00:50<00:00, 2.29s/it, loss=0.0562, lr=0.000655]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 41/100: 100% 2/2 [00:04<00:00, 2.46s/it, val_loss=0.0257]\\n\",\"Finish Validation\\n\",\"Epoch:41/100\\n\",\"Total Loss: 0.056 || Val Loss: 0.026 \\n\",\"Start Train\\n\",\"Epoch 42/100: 100% 22/22 [00:50<00:00, 2.29s/it, loss=0.0526, lr=0.000639]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 42/100: 100% 2/2 [00:04<00:00, 2.43s/it, val_loss=0.0264]\\n\",\"Finish Validation\\n\",\"Epoch:42/100\\n\",\"Total Loss: 0.053 || Val Loss: 0.026 \\n\",\"Start Train\\n\",\"Epoch 43/100: 100% 22/22 [00:51<00:00, 2.36s/it, loss=0.046, lr=0.000622]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 43/100: 100% 2/2 [00:05<00:00, 2.54s/it, val_loss=0.0246]\\n\",\"Finish Validation\\n\",\"Epoch:43/100\\n\",\"Total Loss: 0.046 || Val Loss: 0.025 \\n\",\"Start Train\\n\",\"Epoch 44/100: 100% 22/22 [00:50<00:00, 2.27s/it, loss=0.0422, lr=0.000606]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 44/100: 100% 2/2 [00:05<00:00, 2.59s/it, val_loss=0.0264]\\n\",\"Finish Validation\\n\",\"Epoch:44/100\\n\",\"Total Loss: 0.042 || Val Loss: 0.026 \\n\",\"Start Train\\n\",\"Epoch 45/100: 100% 22/22 [00:50<00:00, 2.29s/it, loss=0.0491, lr=0.000589]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 45/100: 100% 2/2 [00:04<00:00, 2.39s/it, val_loss=0.0257]\\n\",\"Finish Validation\\n\",\"Epoch:45/100\\n\",\"Total Loss: 0.049 || Val Loss: 0.026 \\n\",\"Start Train\\n\",\"Epoch 46/100: 100% 22/22 [00:49<00:00, 2.26s/it, loss=0.0465, lr=0.000572]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 46/100: 100% 2/2 [00:05<00:00, 2.68s/it, val_loss=0.0268]\\n\",\"Finish Validation\\n\",\"Epoch:46/100\\n\",\"Total Loss: 0.046 || Val Loss: 0.027 \\n\",\"Start Train\\n\",\"Epoch 47/100: 100% 22/22 [00:50<00:00, 2.29s/it, loss=0.0474, lr=0.000556]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 47/100: 100% 2/2 [00:04<00:00, 2.47s/it, val_loss=0.0241]\\n\",\"Finish Validation\\n\",\"Epoch:47/100\\n\",\"Total Loss: 0.047 || Val Loss: 0.024 \\n\",\"Start Train\\n\",\"Epoch 48/100: 100% 22/22 [00:50<00:00, 2.31s/it, loss=0.046, lr=0.000539]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 48/100: 100% 2/2 [00:04<00:00, 2.43s/it, val_loss=0.0236]\\n\",\"Finish Validation\\n\",\"Epoch:48/100\\n\",\"Total Loss: 0.046 || Val Loss: 0.024 \\n\",\"Start Train\\n\",\"Epoch 49/100: 100% 22/22 [00:50<00:00, 2.30s/it, loss=0.0498, lr=0.000522]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 49/100: 100% 2/2 [00:04<00:00, 2.44s/it, val_loss=0.0244]\\n\",\"Finish Validation\\n\",\"Epoch:49/100\\n\",\"Total Loss: 0.050 || Val Loss: 0.024 \\n\",\"Start Train\\n\",\"Epoch 50/100: 100% 22/22 [00:49<00:00, 2.26s/it, loss=0.0539, lr=0.000505]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 50/100: 100% 2/2 [00:04<00:00, 2.39s/it, val_loss=0.0246]\\n\",\"Finish Validation\\n\",\"Epoch:50/100\\n\",\"Total Loss: 0.054 || Val Loss: 0.025 \\n\",\"Start Train\\n\",\"Epoch 51/100: 100% 45/45 [00:56<00:00, 1.26s/it, loss=0.0478, lr=0.000488]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 51/100: 100% 5/5 [00:06<00:00, 1.35s/it, val_loss=0.031]\\n\",\"Finish Validation\\n\",\"Epoch:51/100\\n\",\"Total Loss: 0.048 || Val Loss: 0.031 \\n\",\"Start Train\\n\",\"Epoch 52/100: 100% 45/45 [00:55<00:00, 1.24s/it, loss=0.0517, lr=0.000471]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 52/100: 100% 5/5 [00:06<00:00, 1.26s/it, val_loss=0.0264]\\n\",\"Finish Validation\\n\",\"Epoch:52/100\\n\",\"Total Loss: 0.052 || Val Loss: 0.026 \\n\",\"Start Train\\n\",\"Epoch 53/100: 100% 45/45 [00:55<00:00, 1.22s/it, loss=0.053, lr=0.000454]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 53/100: 100% 5/5 [00:06<00:00, 1.24s/it, val_loss=0.0247]\\n\",\"Finish Validation\\n\",\"Epoch:53/100\\n\",\"Total Loss: 0.053 || Val Loss: 0.025 \\n\",\"Start Train\\n\",\"Epoch 54/100: 100% 45/45 [00:57<00:00, 1.27s/it, loss=0.0476, lr=0.000438]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 54/100: 100% 5/5 [00:05<00:00, 1.20s/it, val_loss=0.0233]\\n\",\"Finish Validation\\n\",\"Epoch:54/100\\n\",\"Total Loss: 0.048 || Val Loss: 0.023 \\n\",\"Start Train\\n\",\"Epoch 55/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.0372, lr=0.000421]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 55/100: 100% 5/5 [00:06<00:00, 1.20s/it, val_loss=0.0196]\\n\",\"Finish Validation\\n\",\"Epoch:55/100\\n\",\"Total Loss: 0.037 || Val Loss: 0.020 \\n\",\"Start Train\\n\",\"Epoch 56/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.0385, lr=0.000404]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 56/100: 100% 5/5 [00:05<00:00, 1.19s/it, val_loss=0.0212]\\n\",\"Finish Validation\\n\",\"Epoch:56/100\\n\",\"Total Loss: 0.039 || Val Loss: 0.021 \\n\",\"Start Train\\n\",\"Epoch 57/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.038, lr=0.000388]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 57/100: 100% 5/5 [00:06<00:00, 1.24s/it, val_loss=0.0197]\\n\",\"Finish Validation\\n\",\"Epoch:57/100\\n\",\"Total Loss: 0.038 || Val Loss: 0.020 \\n\",\"Start Train\\n\",\"Epoch 58/100: 100% 45/45 [00:55<00:00, 1.22s/it, loss=0.0402, lr=0.000371]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 58/100: 100% 5/5 [00:05<00:00, 1.11s/it, val_loss=0.0187]\\n\",\"Finish Validation\\n\",\"Epoch:58/100\\n\",\"Total Loss: 0.040 || Val Loss: 0.019 \\n\",\"Start Train\\n\",\"Epoch 59/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.0399, lr=0.000355]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 59/100: 100% 5/5 [00:05<00:00, 1.18s/it, val_loss=0.0189]\\n\",\"Finish Validation\\n\",\"Epoch:59/100\\n\",\"Total Loss: 0.040 || Val Loss: 0.019 \\n\",\"Start Train\\n\",\"Epoch 60/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.0334, lr=0.000339]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 60/100: 100% 5/5 [00:06<00:00, 1.25s/it, val_loss=0.0197]\\n\",\"Finish Validation\\n\",\"Epoch:60/100\\n\",\"Total Loss: 0.033 || Val Loss: 0.020 \\n\",\"Start Train\\n\",\"Epoch 61/100: 100% 45/45 [00:55<00:00, 1.24s/it, loss=0.0339, lr=0.000323]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 61/100: 100% 5/5 [00:06<00:00, 1.21s/it, val_loss=0.019]\\n\",\"Finish Validation\\n\",\"Epoch:61/100\\n\",\"Total Loss: 0.034 || Val Loss: 0.019 \\n\",\"Start Train\\n\",\"Epoch 62/100: 100% 45/45 [00:54<00:00, 1.22s/it, loss=0.0332, lr=0.000308]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 62/100: 100% 5/5 [00:06<00:00, 1.23s/it, val_loss=0.0162]\\n\",\"Finish Validation\\n\",\"Epoch:62/100\\n\",\"Total Loss: 0.033 || Val Loss: 0.016 \\n\",\"Start Train\\n\",\"Epoch 63/100: 100% 45/45 [00:54<00:00, 1.22s/it, loss=0.0335, lr=0.000292]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 63/100: 100% 5/5 [00:05<00:00, 1.16s/it, val_loss=0.0155]\\n\",\"Finish Validation\\n\",\"Epoch:63/100\\n\",\"Total Loss: 0.034 || Val Loss: 0.015 \\n\",\"Start Train\\n\",\"Epoch 64/100: 100% 45/45 [00:54<00:00, 1.22s/it, loss=0.0342, lr=0.000277]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 64/100: 100% 5/5 [00:05<00:00, 1.20s/it, val_loss=0.0154]\\n\",\"Finish Validation\\n\",\"Epoch:64/100\\n\",\"Total Loss: 0.034 || Val Loss: 0.015 \\n\",\"Start Train\\n\",\"Epoch 65/100: 100% 45/45 [00:56<00:00, 1.26s/it, loss=0.0375, lr=0.000262]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 65/100: 100% 5/5 [00:05<00:00, 1.18s/it, val_loss=0.0188]\\n\",\"Finish Validation\\n\",\"Epoch:65/100\\n\",\"Total Loss: 0.037 || Val Loss: 0.019 \\n\",\"Start Train\\n\",\"Epoch 66/100: 100% 45/45 [00:54<00:00, 1.22s/it, loss=0.0334, lr=0.000248]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 66/100: 100% 5/5 [00:06<00:00, 1.22s/it, val_loss=0.0158]\\n\",\"Finish Validation\\n\",\"Epoch:66/100\\n\",\"Total Loss: 0.033 || Val Loss: 0.016 \\n\",\"Start Train\\n\",\"Epoch 67/100: 100% 45/45 [00:56<00:00, 1.25s/it, loss=0.0325, lr=0.000234]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 67/100: 100% 5/5 [00:06<00:00, 1.22s/it, val_loss=0.0158]\\n\",\"Finish Validation\\n\",\"Epoch:67/100\\n\",\"Total Loss: 0.033 || Val Loss: 0.016 \\n\",\"Start Train\\n\",\"Epoch 68/100: 100% 45/45 [00:54<00:00, 1.22s/it, loss=0.0346, lr=0.00022]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 68/100: 100% 5/5 [00:06<00:00, 1.21s/it, val_loss=0.0178]\\n\",\"Finish Validation\\n\",\"Epoch:68/100\\n\",\"Total Loss: 0.035 || Val Loss: 0.018 \\n\",\"Start Train\\n\",\"Epoch 69/100: 100% 45/45 [00:57<00:00, 1.27s/it, loss=0.0269, lr=0.000206]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 69/100: 100% 5/5 [00:06<00:00, 1.23s/it, val_loss=0.016]\\n\",\"Finish Validation\\n\",\"Epoch:69/100\\n\",\"Total Loss: 0.027 || Val Loss: 0.016 \\n\",\"Start Train\\n\",\"Epoch 70/100: 100% 45/45 [00:55<00:00, 1.24s/it, loss=0.029, lr=0.000193]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 70/100: 100% 5/5 [00:05<00:00, 1.19s/it, val_loss=0.0145]\\n\",\"Finish Validation\\n\",\"Epoch:70/100\\n\",\"Total Loss: 0.029 || Val Loss: 0.015 \\n\",\"Start Train\\n\",\"Epoch 71/100: 100% 45/45 [00:56<00:00, 1.25s/it, loss=0.0254, lr=0.00018]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 71/100: 100% 5/5 [00:06<00:00, 1.23s/it, val_loss=0.0147]\\n\",\"Finish Validation\\n\",\"Epoch:71/100\\n\",\"Total Loss: 0.025 || Val Loss: 0.015 \\n\",\"Start Train\\n\",\"Epoch 72/100: 100% 45/45 [00:56<00:00, 1.25s/it, loss=0.0238, lr=0.000167]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 72/100: 100% 5/5 [00:06<00:00, 1.29s/it, val_loss=0.0138]\\n\",\"Finish Validation\\n\",\"Epoch:72/100\\n\",\"Total Loss: 0.024 || Val Loss: 0.014 \\n\",\"Start Train\\n\",\"Epoch 73/100: 100% 45/45 [00:55<00:00, 1.24s/it, loss=0.026, lr=0.000155]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 73/100: 100% 5/5 [00:06<00:00, 1.25s/it, val_loss=0.0141]\\n\",\"Finish Validation\\n\",\"Epoch:73/100\\n\",\"Total Loss: 0.026 || Val Loss: 0.014 \\n\",\"Start Train\\n\",\"Epoch 74/100: 100% 45/45 [00:56<00:00, 1.27s/it, loss=0.0201, lr=0.000143]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 74/100: 100% 5/5 [00:06<00:00, 1.23s/it, val_loss=0.0136]\\n\",\"Finish Validation\\n\",\"Epoch:74/100\\n\",\"Total Loss: 0.020 || Val Loss: 0.014 \\n\",\"Start Train\\n\",\"Epoch 75/100: 100% 45/45 [00:56<00:00, 1.26s/it, loss=0.0231, lr=0.000132]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 75/100: 100% 5/5 [00:06<00:00, 1.26s/it, val_loss=0.0139]\\n\",\"Finish Validation\\n\",\"Epoch:75/100\\n\",\"Total Loss: 0.023 || Val Loss: 0.014 \\n\",\"Start Train\\n\",\"Epoch 76/100: 100% 45/45 [00:56<00:00, 1.26s/it, loss=0.0286, lr=0.000121]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 76/100: 100% 5/5 [00:06<00:00, 1.21s/it, val_loss=0.0136]\\n\",\"Finish Validation\\n\",\"Epoch:76/100\\n\",\"Total Loss: 0.029 || Val Loss: 0.014 \\n\",\"Start Train\\n\",\"Epoch 77/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.023, lr=0.000111]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 77/100: 100% 5/5 [00:06<00:00, 1.21s/it, val_loss=0.0144]\\n\",\"Finish Validation\\n\",\"Epoch:77/100\\n\",\"Total Loss: 0.023 || Val Loss: 0.014 \\n\",\"Start Train\\n\",\"Epoch 78/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.0236, lr=0.000101]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 78/100: 100% 5/5 [00:06<00:00, 1.24s/it, val_loss=0.0144]\\n\",\"Finish Validation\\n\",\"Epoch:78/100\\n\",\"Total Loss: 0.024 || Val Loss: 0.014 \\n\",\"Start Train\\n\",\"Epoch 79/100: 100% 45/45 [00:56<00:00, 1.25s/it, loss=0.0247, lr=9.11e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 79/100: 100% 5/5 [00:05<00:00, 1.19s/it, val_loss=0.0134]\\n\",\"Finish Validation\\n\",\"Epoch:79/100\\n\",\"Total Loss: 0.025 || Val Loss: 0.013 \\n\",\"Start Train\\n\",\"Epoch 80/100: 100% 45/45 [00:55<00:00, 1.24s/it, loss=0.0225, lr=8.21e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 80/100: 100% 5/5 [00:05<00:00, 1.17s/it, val_loss=0.0131]\\n\",\"Finish Validation\\n\",\"Epoch:80/100\\n\",\"Total Loss: 0.022 || Val Loss: 0.013 \\n\",\"Start Train\\n\",\"Epoch 81/100: 100% 45/45 [00:55<00:00, 1.24s/it, loss=0.0235, lr=7.35e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 81/100: 100% 5/5 [00:05<00:00, 1.19s/it, val_loss=0.0134]\\n\",\"Finish Validation\\n\",\"Epoch:81/100\\n\",\"Total Loss: 0.023 || Val Loss: 0.013 \\n\",\"Start Train\\n\",\"Epoch 82/100: 100% 45/45 [00:55<00:00, 1.24s/it, loss=0.0251, lr=6.55e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 82/100: 100% 5/5 [00:05<00:00, 1.18s/it, val_loss=0.0137]\\n\",\"Finish Validation\\n\",\"Epoch:82/100\\n\",\"Total Loss: 0.025 || Val Loss: 0.014 \\n\",\"Start Train\\n\",\"Epoch 83/100: 100% 45/45 [00:55<00:00, 1.24s/it, loss=0.0225, lr=5.8e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 83/100: 100% 5/5 [00:05<00:00, 1.18s/it, val_loss=0.0134]\\n\",\"Finish Validation\\n\",\"Epoch:83/100\\n\",\"Total Loss: 0.023 || Val Loss: 0.013 \\n\",\"Start Train\\n\",\"Epoch 84/100: 100% 45/45 [00:56<00:00, 1.25s/it, loss=0.0248, lr=5.1e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 84/100: 100% 5/5 [00:06<00:00, 1.21s/it, val_loss=0.0133]\\n\",\"Finish Validation\\n\",\"Epoch:84/100\\n\",\"Total Loss: 0.025 || Val Loss: 0.013 \\n\",\"Start Train\\n\",\"Epoch 85/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.0216, lr=4.45e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 85/100: 100% 5/5 [00:05<00:00, 1.16s/it, val_loss=0.0135]\\n\",\"Finish Validation\\n\",\"Epoch:85/100\\n\",\"Total Loss: 0.022 || Val Loss: 0.013 \\n\",\"Start Train\\n\",\"Epoch 86/100: 100% 45/45 [00:56<00:00, 1.26s/it, loss=0.0226, lr=3.86e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 86/100: 100% 5/5 [00:05<00:00, 1.18s/it, val_loss=0.0128]\\n\",\"Finish Validation\\n\",\"Epoch:86/100\\n\",\"Total Loss: 0.023 || Val Loss: 0.013 \\n\",\"Start Train\\n\",\"Epoch 87/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.0222, lr=3.32e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 87/100: 100% 5/5 [00:06<00:00, 1.25s/it, val_loss=0.0126]\\n\",\"Finish Validation\\n\",\"Epoch:87/100\\n\",\"Total Loss: 0.022 || Val Loss: 0.013 \\n\",\"Start Train\\n\",\"Epoch 88/100: 100% 45/45 [00:56<00:00, 1.26s/it, loss=0.0197, lr=2.84e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 88/100: 100% 5/5 [00:06<00:00, 1.23s/it, val_loss=0.0128]\\n\",\"Finish Validation\\n\",\"Epoch:88/100\\n\",\"Total Loss: 0.020 || Val Loss: 0.013 \\n\",\"Start Train\\n\",\"Epoch 89/100: 100% 45/45 [00:56<00:00, 1.25s/it, loss=0.0188, lr=2.41e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 89/100: 100% 5/5 [00:05<00:00, 1.18s/it, val_loss=0.0129]\\n\",\"Finish Validation\\n\",\"Epoch:89/100\\n\",\"Total Loss: 0.019 || Val Loss: 0.013 \\n\",\"Start Train\\n\",\"Epoch 90/100: 100% 45/45 [00:56<00:00, 1.25s/it, loss=0.0208, lr=2.04e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 90/100: 100% 5/5 [00:05<00:00, 1.18s/it, val_loss=0.0128]\\n\",\"Finish Validation\\n\",\"Epoch:90/100\\n\",\"Total Loss: 0.021 || Val Loss: 0.013 \\n\",\"Start Train\\n\",\"Epoch 91/100: 100% 45/45 [00:56<00:00, 1.26s/it, loss=0.0214, lr=1.72e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 91/100: 100% 5/5 [00:05<00:00, 1.20s/it, val_loss=0.0129]\\n\",\"Finish Validation\\n\",\"Epoch:91/100\\n\",\"Total Loss: 0.021 || Val Loss: 0.013 \\n\",\"Start Train\\n\",\"Epoch 92/100: 100% 45/45 [00:55<00:00, 1.24s/it, loss=0.0196, lr=1.46e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 92/100: 100% 5/5 [00:06<00:00, 1.22s/it, val_loss=0.0129]\\n\",\"Finish Validation\\n\",\"Epoch:92/100\\n\",\"Total Loss: 0.020 || Val Loss: 0.013 \\n\",\"Start Train\\n\",\"Epoch 93/100: 100% 45/45 [00:56<00:00, 1.25s/it, loss=0.019, lr=1.26e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 93/100: 100% 5/5 [00:05<00:00, 1.18s/it, val_loss=0.0132]\\n\",\"Finish Validation\\n\",\"Epoch:93/100\\n\",\"Total Loss: 0.019 || Val Loss: 0.013 \\n\",\"Start Train\\n\",\"Epoch 94/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.0203, lr=1.12e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 94/100: 100% 5/5 [00:06<00:00, 1.21s/it, val_loss=0.0132]\\n\",\"Finish Validation\\n\",\"Epoch:94/100\\n\",\"Total Loss: 0.020 || Val Loss: 0.013 \\n\",\"Start Train\\n\",\"Epoch 95/100: 100% 45/45 [00:56<00:00, 1.25s/it, loss=0.0216, lr=1.03e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 95/100: 100% 5/5 [00:05<00:00, 1.20s/it, val_loss=0.0127]\\n\",\"Finish Validation\\n\",\"Epoch:95/100\\n\",\"Total Loss: 0.022 || Val Loss: 0.013 \\n\",\"Start Train\\n\",\"Epoch 96/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.0195, lr=1e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 96/100: 100% 5/5 [00:06<00:00, 1.24s/it, val_loss=0.0129]\\n\",\"Finish Validation\\n\",\"Epoch:96/100\\n\",\"Total Loss: 0.019 || Val Loss: 0.013 \\n\",\"Start Train\\n\",\"Epoch 97/100: 100% 45/45 [00:56<00:00, 1.25s/it, loss=0.0182, lr=1e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 97/100: 100% 5/5 [00:06<00:00, 1.20s/it, val_loss=0.0128]\\n\",\"Finish Validation\\n\",\"Epoch:97/100\\n\",\"Total Loss: 0.018 || Val Loss: 0.013 \\n\",\"Start Train\\n\",\"Epoch 98/100: 100% 45/45 [00:56<00:00, 1.25s/it, loss=0.0221, lr=1e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 98/100: 100% 5/5 [00:06<00:00, 1.25s/it, val_loss=0.0126]\\n\",\"Finish Validation\\n\",\"Epoch:98/100\\n\",\"Total Loss: 0.022 || Val Loss: 0.013 \\n\",\"Start Train\\n\",\"Epoch 99/100: 100% 45/45 [00:55<00:00, 1.24s/it, loss=0.02, lr=1e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 99/100: 100% 5/5 [00:05<00:00, 1.19s/it, val_loss=0.0121]\\n\",\"Finish Validation\\n\",\"Epoch:99/100\\n\",\"Total Loss: 0.020 || Val Loss: 0.012 \\n\",\"Start Train\\n\",\"Epoch 100/100: 100% 45/45 [00:55<00:00, 1.23s/it, loss=0.021, lr=1e-5]\\n\",\"Finish Train\\n\",\"Start Validation\\n\",\"Epoch 100/100: 100% 5/5 [00:06<00:00, 1.23s/it, val_loss=0.0126]\\n\",\"Finish Validation\\n\",\"Epoch:100/100\\n\",\"Total Loss: 0.021 || Val Loss: 0.013 \\n\"]}],\"source\":[\"# 训练数据，默认为100个epochs\\n\",\"!python train.py --tiny --phi 2 --epochs 100 \\\\\\n\",\" --weights model_data/yolotiny_CBAM_ep100.pth \\\\\\n\",\" --freeze --freeze-epochs 50 --freeze-size 64 \\\\\\n\",\" --batch-size 32 --shape 416 \\\\\\n\",\" --fp16 --cuda\"]},{\"cell_type\":\"markdown\",\"metadata\":{\"id\":\"FcfrujCdkX9z\"},\"source\":[\"### ECA版本\"]},{\"cell_type\":\"code\",\"execution_count\":null,\"metadata\":{\"id\":\"jaRikD6LkX9z\"},\"outputs\":[],\"source\":[\"# 训练数据，默认为100个epochs\\n\",\"!python train.py --tiny --phi 3 --epochs 100 \\\\\\n\",\" --weights model_data/yolotiny_ECA_ep100.pth \\\\\\n\",\" --freeze --freeze-epochs 50 --freeze-size 64 \\\\\\n\",\" --batch-size 32 --shape 416 \\\\\\n\",\" --fp16 --cuda\"]},{\"cell_type\":\"markdown\",\"metadata\":{\"id\":\"ZdUR507vGaq8\"},\"source\":[\"# 预测结果\"]},{\"cell_type\":\"markdown\",\"source\":[\"### 视频预测或者文件夹预测等\"],\"metadata\":{\"id\":\"l7tb4OUQnpx-\"}},{\"cell_type\":\"code\",\"execution_count\":31,\"metadata\":{\"colab\":{\"base_uri\":\"https://localhost:8080/\"},\"executionInfo\":{\"elapsed\":8279,\"status\":\"ok\",\"timestamp\":1651112609364,\"user\":{\"displayName\":\"KaiJun Deng\",\"userId\":\"04642544504944131029\"},\"user_tz\":-480},\"id\":\"Hb5urvahM-SB\",\"outputId\":\"bfd5b6de-8a07-4a12-d514-7cd88cc60004\"},\"outputs\":[{\"output_type\":\"stream\",\"name\":\"stdout\",\"text\":[\"Namespace(cuda=True, mode='dir_predict', phi=0, shape=416, tiny=True, weights='logs/ep100-loss0.026-val_loss0.015.pth')\\n\",\"logs/ep100-loss0.026-val_loss0.015.pth model, anchors, and classes loaded.\\n\",\" 0% 0/8 [00:00\\n\",\"Get map done.\\n\"]}],\"source\":[\"# # 对验证集进行计算\\n\",\"# !python get_map.py --mode 0 \\\\\\n\",\"# --tiny --phi 1 \\\\\\n\",\"# --weights model_data/yolotiny_SE_ep100.pth \\\\\\n\",\"# --cuda --shape 416\\n\",\"# !python get_map.py --tiny --cuda\\n\",\"!python get_map.py --tiny --cuda --phi 0 --weights logs/ep100-loss0.026-val_loss0.015.pth\"]}],\"metadata\":{\"accelerator\":\"GPU\",\"colab\":{\"collapsed_sections\":[],\"name\":\"yolov4-gesture-tutorial.ipynb\",\"provenance\":[]},\"kernelspec\":{\"display_name\":\"Python 3\",\"name\":\"python3\"},\"language_info\":{\"codemirror_mode\":{\"name\":\"ipython\",\"version\":3},\"file_extension\":\".py\",\"mimetype\":\"text/x-python\",\"name\":\"python\",\"nbconvert_exporter\":\"python\",\"pygments_lexer\":\"ipython3\",\"version\":\"3.8.13\"}},\"nbformat\":4,\"nbformat_minor\":0}" } ]