[ { "path": ".gitignore", "content": "build\ndata\noutput\nerror\nerror.txt\n*.solverstate\n*.caffemodel\n*lmdb" }, { "path": "CMakeLists.txt", "content": "cmake_minimum_required(VERSION 2.8)\nset(ProjName caffe-onclick)\nproject(${ProjName})\nset(CMAKE_CXX_STANDARD 11)\nfind_package(OpenCV REQUIRED)\ninclude_directories(${OpenCV_INCLUDE_DIRS})\n\nfind_package(Caffe REQUIRED)\ninclude_directories(${Caffe_INCLUDE_DIRS})\n\nadd_executable(evaluation cpp/evaluation.cpp cpp/LenetClassifier.cpp)\ntarget_link_libraries(evaluation ${OpenCV_LIBS} ${Caffe_LIBRARIES})\n\nadd_executable(cpp4caffe cpp4caffe/evaluation.cpp)\ntarget_link_libraries(cpp4caffe ${OpenCV_LIBS} ${Caffe_LIBRARIES})" }, { "path": "README.md", "content": "caffe一键式训练评估集成开发环境\n====================================\n**Last Update 2020.07.08**\n\n## 概述\n\n本项目提供一个集成式开发环境，在配好caffe环境的前提下，只需将准备好的图片放入data目录下，便可以一键生成lmdb数据文件、均值文件、标注文件和测试评估模型、找出错误样本、部署模型等所有的操作，更难能可贵的是它是跨平台的，可以无缝的在Windos和Linux间切换。\n\n使用深度学习完成一个特定的任务比如说字符识别、人脸识别等大致可以分为数据准备、定义模型、训练模型、评估模型和部署模型等几个步骤。\n\n### 配置caffe\n\n现在配置caffe十分方便，仅需几行命令即可搞定,确保安装了所需的依赖,这里仅摘录最关键的部分，其余的详细内容可参见参考链接.\n\n**Windows**\n\n\t::为了减少日后不必要的麻烦，建议VS2015,Cuda8.0,cudnn5.1及以上,python2.7\n\tgit clone https://github.com/BVLC/caffe\n\tcd caffe\n\tgit checkout windows\n\tscripts\\build_win.cmd\n\n**Linux** \n\n git clone https://github.com/BVLC/caffe\n\tcd caffe\n\tmkdir build\n\tcd build\n\tcmake ..\n\tmake -j8\n\n### 1.数据准备\n\n首先收集要任务相关的数据，这里准备了一个车牌字符数据（仅包含0-9共10个数字），直接解压[data.zip](https://github.com/imistyrain/caffe-oneclick/releases/download/1.0/data.zip)到当前文件夹即可，格式如下图所示，每类图片对应一个文件夹，放到一个data文件夹下，注意格式一致型（都为.jpg或.png文件），仔细筛查，不要含有其他的非图片文件在里面，你也可以用自己的数据替换这些车牌字符数据。\n\n![structures](https://i.imgur.com/JQmNGYN.png)\n\ncaffe使用了lmdb内存数据库等来加快训练时读取数据的速度，为此，caffe自带的tools里提供了一个工具（可由convert_imageset.cpp编译生成），它的输入是图片路径和标签对组成的文件，每次都手动生成这个文件不胜其烦。\n\n我们希望是自动化的从文件夹读取的功能，因此，本项目通过preprocess/preprocess.py来获取如下图所示的文件夹下所有的文件路径以及对应的文件标签的功能，它输出训练和验证集preprocess/train.txt和preprocess/val.txt以及标签映射文件modef/labels.txt\n\n你也可以直接下载已经转换好的[lmdb.tar.gz](https://github.com/imistyrain/caffe-oneclick/releases/download/1.0/lmdb.tar.gz)文件直接使用\n\n### 2.定义模型\n\n训练定义文件位于models下的plate_train_test.prototxt，部署文件在deploy.prototxt，你可以通过[网络结构可视化](http://ethereon.github.io/netscope/#/editor)对这些网络进行可视化，以便更清晰的理解他们的含义\n\n### 3.训练模型\n\n```\n./train.sh\n```\n\n### 4.评估模型\n\n[evaluation.py](util/evaluation.py)用来对data文件下下的数据进行评估，它会得出迭代次数为10000时模型的错误率，并且打印出误识别图片对应的真值和预测值，并把相应数据保存在error文件夹下，命名格式为字符文件夹/图片在文件夹内的序号_真值类别_预测类别(以0/190_0_4.jpg为例，代表0/190.jpg被误识为4)，这些错误识别的样本需要仔细分析，不断调试参数，以获得期望的结果。\n\n本项目提供了一个训练好的[模型文件](https://github.com/imistyrain/caffe-oneclick/releases/download/1.0/plate999.caffemodel)，其错误率低于0.1%,这就意味着其达到了99.9%以上的准确率。\n\n### 5.部署模型\n\n由于速度原因，实际中多使用C++而不是python进行部署，因此本项目在cpp文件夹下提供了evaluationcpp工程，它使用单例模式来防止每次预测都加载模型，只需使用如下代码即可在你的项目中一行代码使用CNN，此外，该项目也提供了对模型进行评估的功能。\n\n```\n cv::Mat img=cv::imread(\"imagepath.jpg\");\n string result=CnnPredictor::getInstance()->predict(img);\n```\n\n当然，你也可以运行calssification.bat来调用caffe自身进行分类识别\n\n```\n\"../build/examples/cpp_classification/classification\" \"modeldef/deploy.prototxt\" \"trainedmodels/platerecognition_iter_1000.caffemodel\" \"modeldef/mean.binaryproto\" \"modeldef/labels.txt\" \"data/0/4-3.jpg\"\n```\n

\n \n

\n\n其返回了最高的5个类别的相似度，不难看出训练的网络对于data/0/0.jpg有高达93%的概率认为其属于0这个字符，结果还是非常理想的\n\n## 参考\n\n* [Caffe 配置与示例运行](http://blog.csdn.net/guoyk1990/article/details/52909864)\n\n* [图文并解caffe源码](http://blog.csdn.net/mounty_fsc/article/category/6136645)\n\n* [caffe源码解析](http://blog.csdn.net/qq_16055159)\n\n* [从零开始山寨Caffe caffe为什么这么设计？](http://www.cnblogs.com/neopenx/)\n\n* [Caffe代码导读 21天实战caffe作者博客](http://blog.csdn.net/kkk584520/article/category/2620891/2)\n\n* [CNN卷积神经网络推导和实现](http://blog.csdn.net/zouxy09/article/details/9993371)\n\n* [caffe卷积层代码阅读笔记](http://blog.csdn.net/tangwei2014/article/details/47730797)\n\n* [caffe添加新层教程](http://blog.csdn.net/shuzfan/article/details/51322976)\n\n* [caffe中各语言预处理对应方式](http://blog.csdn.net/minstyrain/article/details/78373914)\n\n* [mxnet 训练自己的数据](https://github.com/imistyrain/mxnet-mr)\n\n* [MatconvNet 训练自己的数据](https://github.com/imistyrain/MatConvNet-mr)" }, { "path": "classification.bat", "content": "@echo off\nset CAFFE_DIR=..\nset eval_iter=10000\nset imagepath=data/0/0.jpg\nset trainedmodel=snapshot/plate_iter_%eval_iter%.caffemodel\n::set trainedmodel=platere996.caffemodel\necho %imagepath% %eval_iter%\n\"%CAFFE_DIR%/build/examples/cpp_classification/classification\" \"models/deploy.prototxt\" \"%trainedmodel%\" \"models/mean.binaryproto\" \"models/labels.txt\" \"%imagepath%\"\npause" }, { "path": "cpp/LenetClassifier.cpp", "content": "#include \"LenetClassifier.h\"\nstd::pairCLenetClassifier::predict(const cv::Mat &img)\n{\n\tstd::pairp;\n\tif (!bloaded)\n\t{\n\t\tload();\n\t}\n\telse\n\t{\n cv::Mat input;\n cv::resize(img, input, cv::Size(20, 20));\n cv::Mat inputBlob = blobFromImage(input);// , 255.0f, cv::Size(20, 20), _mean, false);\n\t\tcv::Mat prob;\n _net.setInput(inputBlob, \"data\");\n\t\tprob = _net.forward(\"prob\");\n\t\tcv::Mat probMat = prob.reshape(1, 1);\n\t\tcv::Point classNumber;\n\t\tcv::minMaxLoc(probMat, NULL, &p.second, NULL, &classNumber);\n\t\tp.first = classNumber.x;\n\t}\n\n\treturn p;\n}\n\nbool CLenetClassifier::load(cv::String modelTxt, cv::String modelBin)\n{\n\t_net = cv::dnn::readNetFromCaffe(modelTxt, modelBin);\n// _mean = cv::Scalar(66, 66, 66);\n\tbloaded = !_net.empty();\n\treturn bloaded;\n}" }, { "path": "cpp/LenetClassifier.h", "content": "#pragma once\n#include \"string\"\n#include \"opencv2/opencv.hpp\"\n#include \nusing namespace cv::dnn;\nusing namespace std;\nconst string caffeplatedir = \"../\";\nconst string model_file = caffeplatedir + \"/models/deploy.prototxt\";\nconst string trained_file = caffeplatedir + \"/models/plate999.caffemodel\";\nconst string mean_file = caffeplatedir + \"/models/mean.binaryproto\";\n\nclass CLenetClassifier\n{\npublic:\n\tstatic CLenetClassifier*getInstance()\n\t{\n\t\tstatic CLenetClassifier instance;\n\t\treturn &instance;\n\t}\n\tstd::pairpredict(const cv::Mat &img);\n\tbool load(cv::String modelTxt = model_file, cv::String modelBin = trained_file);\nprivate:\n\tbool bloaded = false;\n Net _net;\n cv::Scalar _mean;\n\tCLenetClassifier() {\n\t}\n};" }, { "path": "cpp/evaluation.cpp", "content": "#include \"mrdir.h\"\n#include \"mrutil.h\"\n#include \"mropencv.h\"\n#include \"LenetClassifier.h\"\nusing namespace std;\nconst string errordir = caffeplatedir + \"/error\";\nconst string platedatadir = caffeplatedir + \"data\";\n\nvoid cleardir(const string dir)\n{\n\tvectorfiles=getAllFilesinDir(dir);\n\tfor (int i = 0; i < files.size(); i++)\n\t{\n\t\tstring filepath = dir + \"/\" + files[i];\n\t\tremove(filepath.c_str());\n\t}\n}\n\nvoid clearerror(const string dir)\n{\n cout << \"clearing\" << dir << endl;\n\tvectorsubdirs=getAllSubdirs(dir);\n\tfor (int i = 0; i < subdirs.size(); i++)\n\t{\n\t\tstring subdir = dir + \"/\" + subdirs[i];\n cout << subdirs[i]<subdirs=getAllSubdirs(platedatadir);\n\tfor (auto sub : subdirs)\n\t{\n\t\tstring subdir = platedatadir + \"/\" + sub;\n\t\tvectorfiles=getAllFilesinDir(subdir);\n\t\tfor (auto file : files)\n\t\t{\n\t\t\tstring fileapth = subdir + \"/\" + file;\n\t\t\tcv::Mat img = cv::imread(fileapth);\n\t\t\tauto ret=CLenetClassifier::getInstance()->predict(img).first;\n\t\t\tif (ret == string2int(sub))\n\t\t\t\trightcount++;\n\t\t\telse\n\t\t\t{\n cout << sub + \"/\" + file.substr(0, file.size() - 4) + \":\" + int2string(ret) << endl;\n\t\t\t\terrorcount++;\n\t\t\t\tstring errorlabeldir = errordir;\n\t\t\t\terrorlabeldir = errorlabeldir + \"/\" + sub;\n\t\t\t\tif (!EXISTS(errorlabeldir.c_str()))\n\t\t\t\t{\n\t\t\t\t\tMKDIR(errorlabeldir.c_str());\n\t\t\t\t}\n\t\t\t\tstring errorfilepath = errorlabeldir + \"/\" + file.substr(0,file.size()-4) + \"_\" + sub + \"_\" + int2string(ret) + \".png\";\n\t\t\t\t//imshow(\"error\", img);\n\t\t\t\timwrite(errorfilepath, img);\n\t\t\t\t//cv::waitKey(1);\n\t\t\t}\n\t\t\ttotal++;\n\t\t}\n\t}\n\tcout << \"acc:\" << rightcount << \"/\" << total << endl;\n\tcout << rightcount*1.0 / total << endl;\n\treturn 0;\n}\n\nint testimg(const std::string imgpath = \"data/0/0.jpg\")\n{\n cv::Mat img = imread(imgpath);\n TickMeter tm;\n tm.start();\n auto p = CLenetClassifier::getInstance()->predict(img);\n tm.stop();\n std::cout << p.first << std::endl;// \" \" << p.second << endl;\n std::cout << tm.getTimeMilli() << \"ms\" << std::endl;\n return 0;\n}\n\nint testdir(const std::string dir = \"img\")\n{\n auto files = getAllFilesinDir(dir);\n for (int i = 0; i < files.size(); i++)\n {\n std::string imgpath = dir + \"/\" + files[i];\n std::cout << files[i] << \":\";\n testimg(imgpath);\n }\n return 0;\n}\n\nint main(int argc,char*argv[])\n{\n\tif (argc==1)\n\t\tevaluation();\n\telse\n\t{\n testimg();\n testdir();\n\t}\n\treturn 0;\n}" }, { "path": "cpp/evaluationcpp.vcxproj", "content": "\n\n \n \n Debug\n Win32\n \n \n Debug\n x64\n \n \n Release\n Win32\n \n \n Release\n x64\n \n \n \n {0A2F0DB8-57C5-4380-93D9-E45110DE3719}\n Win32Proj\n charsrecog\n 8.1\n \n \n \n Application\n true\n v140\n Unicode\n \n \n Application\n true\n v140\n Unicode\n \n \n Application\n false\n v140\n true\n Unicode\n \n \n Application\n false\n v140\n true\n Unicode\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n true\n \n \n true\n D:\\opencv33\\build\\include;$(IncludePath)\n D:\\opencv33\\build\\x64\\vc14\\lib;$(LibraryPath)\n \n \n false\n \n \n false\n D:\\opencv33\\build\\include;$(IncludePath)\n D:\\opencv33\\build\\x64\\vc14\\lib;$(LibraryPath)\n \n \n \n \n \n Level3\n Disabled\n WIN32;_DEBUG;_CONSOLE;_LIB;%(PreprocessorDefinitions)\n true\n \n \n Console\n true\n \n \n \n \n \n \n Level3\n Disabled\n WIN32;_DEBUG;_CONSOLE;_LIB;%(PreprocessorDefinitions)\n true\n $(SolutionDir)include;%(AdditionalIncludeDirectories)\n /D_CRT_SECURE_NO_WARNINGS %(AdditionalOptions)\n \n \n Console\n true\n kernel32.lib;user32.lib;gdi32.lib;winspool.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;comdlg32.lib;advapi32.lib;cudart.lib;cublas.lib;curand.lib;libprotobufd.lib;hdf5_tools.lib;hdf5_hl_fortran.lib;hdf5_fortran.lib;hdf5_hl_f90cstub.lib;hdf5_f90cstub.lib;hdf5_cpp.lib;hdf5_hl_cpp.lib;hdf5_hl.lib;hdf5.lib;zlib.lib;szip.lib;caffelibd.lib;opencv_world300d.lib;shlwapi.lib;leveldbd.lib;cuda.lib;libglog.lib;lmdb.lib;cudnn.lib;libopenblas.dll.a;libgflags.lib;cublas_device.lib;%(AdditionalDependencies)\n \n \n compute_20,sm_20;compute_30,sm_30;compute_35,sm_35;compute_50,sm_50;\n \n \n \n \n Level3\n \n \n MaxSpeed\n true\n true\n WIN32;NDEBUG;_CONSOLE;_LIB;%(PreprocessorDefinitions)\n true\n \n \n Console\n true\n true\n true\n \n \n \n \n Level3\n \n \n Disabled\n true\n true\n WIN32;NDEBUG;_CONSOLE;_LIB;%(PreprocessorDefinitions)\n true\n $(SolutionDir)include;%(AdditionalIncludeDirectories)\n true\n /D_CRT_SECURE_NO_WARNINGS %(AdditionalOptions)\n \n \n Console\n true\n true\n true\n kernel32.lib;user32.lib;gdi32.lib;winspool.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;comdlg32.lib;advapi32.lib;%(AdditionalDependencies)\n \n \n \n \n \n \n Shared\n \n \n 64\n compute_20,sm_20;compute_30,sm_30;compute_35,sm_35;compute_50,sm_50;\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n" }, { "path": "cpp4caffe/cnnpredictor.h", "content": "#pragma once\n#include \n#include \"mrdir.h\"\n#include \"opencv2/opencv.hpp\"\n#include \n#include \n#include \n#include \n#include \n#include \nusing namespace std;\nconst string caffeplatedir = \"../\";\nconst string model_file = caffeplatedir + \"models/deploy.prototxt\";\nconst string trained_file = caffeplatedir + \"models/plate999.caffemodel\";\nconst string mean_file = caffeplatedir + \"models/mean.binaryproto\";\nconst string label_file = caffeplatedir + \"models/labels.txt\";\nusing namespace caffe; // NOLINT(build/namespaces)\nusing std::string;\n\n/* Pair (label, confidence) representing a prediction. */\ntypedef std::pair Prediction;\n\nclass Classifier {\npublic:\n\tClassifier(const string& model_file,\n\t\tconst string& trained_file,\n\t\tconst string& mean_file,\n\t\tconst string& label_file);\n\n\tstd::vector Classify(const cv::Mat& img, int N = 5);\n\nprivate:\n\tvoid SetMean(const string& mean_file);\n\n\tstd::vector Predict(const cv::Mat& img);\n\n\tvoid WrapInputLayer(std::vector* input_channels);\n\n\tvoid Preprocess(const cv::Mat& img,\n\t\tstd::vector* input_channels);\n\nprivate:\n\tstd::shared_ptr > net_;\n\tcv::Size input_geometry_;\n\tint num_channels_;\n\tcv::Mat mean_;\n\tstd::vector labels_;\n};\n\nClassifier::Classifier(const string& model_file,\n\tconst string& trained_file,\n\tconst string& mean_file,\n\tconst string& label_file) {\n#ifdef CPU_ONLY\n\tCaffe::set_mode(Caffe::CPU);\n#else\n\tCaffe::set_mode(Caffe::GPU);\n#endif\n\n\t/* Load the network. */\n\tnet_.reset(new Net(model_file, TEST));\n\tnet_->CopyTrainedLayersFrom(trained_file);\n\n\tCHECK_EQ(net_->num_inputs(), 1) << \"Network should have exactly one input.\";\n\tCHECK_EQ(net_->num_outputs(), 1) << \"Network should have exactly one output.\";\n\n\tBlob* input_layer = net_->input_blobs()[0];\n\tnum_channels_ = input_layer->channels();\n\tCHECK(num_channels_ == 3 || num_channels_ == 1)\n\t\t<< \"Input layer should have 1 or 3 channels.\";\n\tinput_geometry_ = cv::Size(input_layer->width(), input_layer->height());\n\n\t/* Load the binaryproto mean file. */\n\tSetMean(mean_file);\n\n\t/* Load labels. */\n\tstd::ifstream labels(label_file.c_str());\n\tCHECK(labels) << \"Unable to open labels file \" << label_file;\n\tstring line;\n\twhile (std::getline(labels, line))\n\t\tlabels_.push_back(string(line));\n\n\tBlob* output_layer = net_->output_blobs()[0];\n\tCHECK_EQ(labels_.size(), output_layer->channels())\n\t\t<< \"Number of labels is different from the output layer dimension.\";\n}\n\nstatic bool PairCompare(const std::pair& lhs,\n\tconst std::pair& rhs) {\n\treturn lhs.first > rhs.first;\n}\n\n/* Return the indices of the top N values of vector v. */\nstatic std::vector Argmax(const std::vector& v, int N) {\n\tstd::vector > pairs;\n\tfor (size_t i = 0; i < v.size(); ++i)\n\t\tpairs.push_back(std::make_pair(v[i], i));\n\tstd::partial_sort(pairs.begin(), pairs.begin() + N, pairs.end(), PairCompare);\n\n\tstd::vector result;\n\tfor (int i = 0; i < N; ++i)\n\t\tresult.push_back(pairs[i].second);\n\treturn result;\n}\n\n/* Return the top N predictions. */\nstd::vector Classifier::Classify(const cv::Mat& img, int N) {\n\tstd::vector output = Predict(img);\n\n\tN = std::min(labels_.size(), N);\n\tstd::vector maxN = Argmax(output, N);\n\tstd::vector predictions;\n\tfor (int i = 0; i < N; ++i) {\n\t\tint idx = maxN[i];\n\t\tpredictions.push_back(std::make_pair(labels_[idx], output[idx]));\n\t}\n\n\treturn predictions;\n}\n\n/* Load the mean file in binaryproto format. */\nvoid Classifier::SetMean(const string& mean_file) {\n\tBlobProto blob_proto;\n\tReadProtoFromBinaryFileOrDie(mean_file.c_str(), &blob_proto);\n\n\t/* Convert from BlobProto to Blob */\n\tBlob mean_blob;\n\tmean_blob.FromProto(blob_proto);\n\tCHECK_EQ(mean_blob.channels(), num_channels_)\n\t\t<< \"Number of channels of mean file doesn't match input layer.\";\n\n\t/* The format of the mean file is planar 32-bit float BGR or grayscale. */\n\tstd::vector channels;\n\tfloat* data = mean_blob.mutable_cpu_data();\n\tfor (int i = 0; i < num_channels_; ++i) {\n\t\t/* Extract an individual channel. */\n\t\tcv::Mat channel(mean_blob.height(), mean_blob.width(), CV_32FC1, data);\n\t\tchannels.push_back(channel);\n\t\tdata += mean_blob.height() * mean_blob.width();\n\t}\n\n\t/* Merge the separate channels into a single image. */\n\tcv::Mat mean;\n\tcv::merge(channels, mean);\n\n\t/* Compute the global mean pixel value and create a mean image\n\t* filled with this value. */\n\tcv::Scalar channel_mean = cv::mean(mean);\n\tmean_ = cv::Mat(input_geometry_, mean.type(), channel_mean);\n}\n\nstd::vector Classifier::Predict(const cv::Mat& img) {\n\tBlob* input_layer = net_->input_blobs()[0];\n\tinput_layer->Reshape(1, num_channels_,\n\t\tinput_geometry_.height, input_geometry_.width);\n\t/* Forward dimension change to all layers. */\n\tnet_->Reshape();\n\n\tstd::vector input_channels;\n\tWrapInputLayer(&input_channels);\n\n\tPreprocess(img, &input_channels);\n\n\tnet_->Forward();\n\n\t/* Copy the output layer to a std::vector */\n\tBlob* output_layer = net_->output_blobs()[0];\n\tconst float* begin = output_layer->cpu_data();\n\tconst float* end = begin + output_layer->channels();\n\treturn std::vector(begin, end);\n}\n\n/* Wrap the input layer of the network in separate cv::Mat objects\n* (one per channel). This way we save one memcpy operation and we\n* don't need to rely on cudaMemcpy2D. The last preprocessing\n* operation will write the separate channels directly to the input\n* layer. */\nvoid Classifier::WrapInputLayer(std::vector* input_channels) {\n\tBlob* input_layer = net_->input_blobs()[0];\n\n\tint width = input_layer->width();\n\tint height = input_layer->height();\n\tfloat* input_data = input_layer->mutable_cpu_data();\n\tfor (int i = 0; i < input_layer->channels(); ++i) {\n\t\tcv::Mat channel(height, width, CV_32FC1, input_data);\n\t\tinput_channels->push_back(channel);\n\t\tinput_data += width * height;\n\t}\n}\n\nvoid Classifier::Preprocess(const cv::Mat& img,\n\tstd::vector* input_channels) {\n\t/* Convert the input image to the input image format of the network. */\n\tcv::Mat sample;\n\tif (img.channels() == 3 && num_channels_ == 1)\n\t\tcv::cvtColor(img, sample, CV_BGR2GRAY);\n\telse if (img.channels() == 4 && num_channels_ == 1)\n\t\tcv::cvtColor(img, sample, CV_BGRA2GRAY);\n\telse if (img.channels() == 4 && num_channels_ == 3)\n\t\tcv::cvtColor(img, sample, CV_BGRA2BGR);\n\telse if (img.channels() == 1 && num_channels_ == 3)\n\t\tcv::cvtColor(img, sample, CV_GRAY2BGR);\n\telse\n\t\tsample = img;\n\n\tcv::Mat sample_resized;\n\tif (sample.size() != input_geometry_)\n\t\tcv::resize(sample, sample_resized, input_geometry_);\n\telse\n\t\tsample_resized = sample;\n\n\tcv::Mat sample_float;\n\tif (num_channels_ == 3)\n\t\tsample_resized.convertTo(sample_float, CV_32FC3);\n\telse\n\t\tsample_resized.convertTo(sample_float, CV_32FC1);\n\n\tcv::Mat sample_normalized;\n//\tcv::subtract(sample_float, mean_, sample_normalized);\n sample_normalized = sample_float;\n\t/* This operation will write the separate BGR planes directly to the\n\t* input layer of the network because it is wrapped by the cv::Mat\n\t* objects in input_channels. */\n\tcv::split(sample_normalized, *input_channels);\n\n\tCHECK(reinterpret_cast(input_channels->at(0).data)\n\t\t== net_->input_blobs()[0]->cpu_data())\n\t\t<< \"Input channels are not wrapping the input layer of the network.\";\n}\n\nclass CnnPredictor\n{\npublic:\n\tstatic CnnPredictor*getInstance()\n\t{\n\t\tstatic CnnPredictor instance;\n\t\treturn &instance;\n\t}\n\tconst string predict(cv::Mat &img)\n\t{\n\t\tstd::vector predictions = pclassifier->Classify(img);\n\t\treturn predictions[0].first;\n\t}\nprivate:\n\tCnnPredictor()\n\t{\n\t\tpclassifier = new Classifier(model_file, trained_file, mean_file, label_file);\n\t};\n\t~CnnPredictor()\n\t{\n\t\tif (pclassifier)\n\t\t\tdelete pclassifier;\n\t};\n\tClassifier *pclassifier;\n};" }, { "path": "cpp4caffe/cpp4caffe.vcxproj", "content": "\n\n \n \n Debug\n x64\n \n \n Release\n x64\n \n \n \n \n \n \n \n \n \n {D9C1D98F-F412-44F6-A819-D1AFB27D31E7}\n Win32Proj\n x64\n cpp4caffe\n NoUpgrade\n 8.1\n \n \n \n Application\n false\n MultiByte\n v140\n \n \n Application\n false\n MultiByte\n v140\n \n \n \n \n \n \n \n \n \n <_ProjectFileVersion>10.0.20506.1\n Debug\\\n Debug\\\n $(ProjectName)-d\n .exe\n true\n true\n ..\\bin\\\n Release\\\n $(ProjectName)\n .exe\n false\n true\n \n \n \n Debug/\n EnableFastChecks\n CompileAsCpp\n ProgramDatabase\n Sync\n Disabled\n MaxSpeed\n NotUsing\n MultiThreadedDebugDLL\n true\n Level3\n WIN32;_WINDOWS;CAFFE_VERSION=1.0.0;BOOST_ALL_NO_LIB;USE_LMDB;USE_LEVELDB;USE_CUDNN;USE_OPENCV;CMAKE_WINDOWS_BUILD;GLOG_NO_ABBREVIATED_SEVERITIES;GOOGLE_GLOG_DLL_DECL=__declspec(dllimport);GOOGLE_GLOG_DLL_DECL_FOR_UNITTESTS=__declspec(dllimport);H5_BUILT_AS_DYNAMIC_LIB=1;CMAKE_INTDIR=\"Debug\";%(PreprocessorDefinitions)\n $(IntDir)\n \n \n WIN32;_DEBUG;_WINDOWS;CAFFE_VERSION=1.0.0;BOOST_ALL_NO_LIB;USE_LMDB;USE_LEVELDB;USE_CUDNN;USE_OPENCV;CMAKE_WINDOWS_BUILD;GLOG_NO_ABBREVIATED_SEVERITIES;GOOGLE_GLOG_DLL_DECL=__declspec(dllimport);GOOGLE_GLOG_DLL_DECL_FOR_UNITTESTS=__declspec(dllimport);H5_BUILT_AS_DYNAMIC_LIB=1;CMAKE_INTDIR=\\\"Debug\\\";%(PreprocessorDefinitions)\n \n \n $(ProjectDir)/$(IntDir)\n %(Filename).h\n %(Filename).tlb\n %(Filename)_i.c\n %(Filename)_p.c\n \n \n %(AdditionalOptions) /machine:x64\n kernel32.lib;user32.lib;gdi32.lib;winspool.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;comdlg32.lib;advapi32.lib;caffe-d.lib;caffeproto-d.lib;boost_system-vc140-mt-gd-1_61.lib;boost_thread-vc140-mt-gd-1_61.lib;boost_filesystem-vc140-mt-gd-1_61.lib;boost_chrono-vc140-mt-gd-1_61.lib;boost_date_time-vc140-mt-gd-1_61.lib;boost_atomic-vc140-mt-gd-1_61.lib;glogd.lib;gflagsd.lib;shlwapi.lib;libprotobufd.lib;caffehdf5_hl_D.lib;caffehdf5_D.lib;caffezlibd.lib;lmdbd.lib;ntdll.lib;leveldbd.lib;snappy_staticd.lib;caffezlibd.lib;cudart.lib;curand.lib;cublas.lib;cublas_device.lib;cudnn.lib;opencv_highgui310d.lib;opencv_videoio310d.lib;opencv_imgcodecs310d.lib;opencv_imgproc310d.lib;opencv_core310d.lib;libopenblas.dll.a;python27.lib;boost_python-vc140-mt-gd-1_61.lib\n %(AdditionalLibraryDirectories)\n Debug\n %(IgnoreSpecificDefaultLibraries)\n MTCNN-d.pdb\n Console\n \n \n \n \n false\n \n \n \n \n %(AdditionalIncludeDirectories)\n Release/\n CompileAsCpp\n Sync\n AnySuitable\n MaxSpeed\n NotUsing\n MultiThreadedDLL\n true\n Level3\n WIN32;_WINDOWS;NDEBUG;CAFFE_VERSION=1.0.0;BOOST_ALL_NO_LIB;USE_LMDB;USE_LEVELDB;USE_CUDNN;USE_OPENCV;CMAKE_WINDOWS_BUILD;GLOG_NO_ABBREVIATED_SEVERITIES;GOOGLE_GLOG_DLL_DECL=__declspec(dllimport);GOOGLE_GLOG_DLL_DECL_FOR_UNITTESTS=__declspec(dllimport);H5_BUILT_AS_DYNAMIC_LIB=1;CMAKE_INTDIR=\"Release\";%(PreprocessorDefinitions)\n $(IntDir)\n ProgramDatabase\n \n \n WIN32;_WINDOWS;NDEBUG;CAFFE_VERSION=1.0.0;BOOST_ALL_NO_LIB;USE_LMDB;USE_LEVELDB;USE_CUDNN;USE_OPENCV;CMAKE_WINDOWS_BUILD;GLOG_NO_ABBREVIATED_SEVERITIES;GOOGLE_GLOG_DLL_DECL=__declspec(dllimport);GOOGLE_GLOG_DLL_DECL_FOR_UNITTESTS=__declspec(dllimport);H5_BUILT_AS_DYNAMIC_LIB=1;CMAKE_INTDIR=\\\"Release\\\";%(PreprocessorDefinitions)\n \n \n $(ProjectDir)/$(IntDir)\n %(Filename).h\n %(Filename).tlb\n %(Filename)_i.c\n %(Filename)_p.c\n \n \n %(AdditionalOptions) /machine:x64\n kernel32.lib;user32.lib;gdi32.lib;winspool.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;comdlg32.lib;advapi32.lib;boost_system-vc140-mt-1_61.lib;boost_thread-vc140-mt-1_61.lib;boost_filesystem-vc140-mt-1_61.lib;boost_chrono-vc140-mt-1_61.lib;boost_date_time-vc140-mt-1_61.lib;boost_atomic-vc140-mt-1_61.lib;glog.lib;gflags.lib;shlwapi.lib;libprotobuf.lib;caffehdf5_hl.lib;caffehdf5.lib;caffezlib.lib;lmdb.lib;ntdll.lib;leveldb.lib;snappy_static.lib;caffe.lib;caffeproto.lib;cudart.lib;curand.lib;cublas.lib;cublas_device.lib;cudnn.lib;opencv_highgui310.lib;opencv_imgcodecs310.lib;opencv_imgproc310.lib;opencv_core310.lib;libopenblas.dll.a;python27.lib;boost_python-vc140-mt-1_61.lib\n %(AdditionalLibraryDirectories)\n true\n %(IgnoreSpecificDefaultLibraries)\n MTCNN.pdb\n Console\n \n \n \n \n false\n \n \n \n \n \n" }, { "path": "cpp4caffe/evaluation.cpp", "content": "#define REG_USE_CNN 1\n#pragma warning(disable:4996)\n#include \"mrdir.h\"\n#include \"mrutil.h\"\n#include \"cnnpredictor.h\"\nconst string errordir = caffeplatedir + \"error\";\nconst string platedatadir = caffeplatedir+\"data\";\n\nvoid cleardir(const string dir)\n{\n\tvectorfiles=getAllFilesinDir(dir);\n\tfor (int i = 0; i < files.size(); i++)\n\t{\n\t\tstring filepath = dir + \"/\" + files[i];\n\t\tremove(filepath.c_str());\n\t}\n}\n\nvoid clearerror(const string dir)\n{\n\tvectorsubdirs=getAllSubdirs(dir);\n\tfor (int i = 0; i < subdirs.size(); i++)\n\t{\n\t\tstring subdir = dir + \"/\" + subdirs[i];\n\t\tcleardir(subdir);\n\t}\n}\n\nint evaluation()\n{\n\tstring line;\n\tstring label;\n\tint rightcount = 0, errorcount = 0, total = 0;\t\n\tif (!EXISTS(errordir.c_str()))\n\t{\n\t\tcout << \"Error dir not exist\" << endl;\n\t\tMKDIR(errordir.c_str());\n\t}\n\tclearerror(errordir);\n\tvectorsubdirs=getAllSubdirs(platedatadir);\n\tfor (auto sub : subdirs)\n\t{\n\t\tstring subdir = platedatadir + \"/\" + sub;\n\t\tvectorfiles=getAllFilesinDir(subdir);\n\t\tfor (auto file : files)\n\t\t{\n\t\t\tstring fileapth = subdir + \"/\" + file;\n\t\t\tcv::Mat img = cv::imread(fileapth);\n\t\t\tauto ret = split(CnnPredictor::getInstance()->predict(img), \" \")[1];\n\t\t\tif (ret == sub)\n\t\t\t\trightcount++;\n\t\t\telse\n\t\t\t{\n\t\t\t\terrorcount++;\n\t\t\t\tstring errorlabeldir = errordir;\n\t\t\t\terrorlabeldir = errorlabeldir + \"/\" + sub;\n\t\t\t\tif (!EXISTS(errorlabeldir.c_str()))\n\t\t\t\t{\n\t\t\t\t\tMKDIR(errorlabeldir.c_str());\n\t\t\t\t}\n\t\t\t\tstring errorfilepath = errorlabeldir + \"/\" + file.substr(0,file.size()-4) + \"_\" + sub + \"_\" + ret + \".png\";\n\t\t\t\tcout << sub + \"/\" + file.substr(0, file.size() - 4) + \":\" + ret << endl;\n\t\t\t\t//imshow(\"error\", img);\n\t\t\t\timwrite(errorfilepath, img);\n\t\t\t\t//cv::waitKey(1);\n\t\t\t}\n\t\t\ttotal++;\n\t\t}\n\t}\n\tcout << \"acc:\" << rightcount << \"/\" << total << endl;\n\tcout << rightcount*1.0 / total << endl;\n\treturn 0;\n}\n\nint main(int argc,char*argv[])\n{\n\tif (argc==1)\n\t\tevaluation();\n\telse\n\t{\n\t\tcv::Mat img = cv::imread(argv[1]);\n\t\tcout << CnnPredictor::getInstance()->predict(img) << endl;\n\t}\n\treturn 0;\n}" }, { "path": "models/deploy.prototxt", "content": "name: \"LeNet\"\nlayer{\n name:\"data\"\n type:\"Input\"\n top: \"data\"\n input_param { shape: {dim:1 dim:3 dim:20 dim:20 } }\n}\nlayer {\n name: \"conv1\"\n type: \"Convolution\"\n bottom: \"data\"\n top: \"conv1\"\n param {\n lr_mult: 1\n }\n param {\n lr_mult: 2\n }\n convolution_param {\n num_output: 20\n kernel_size: 5\n stride: 1\n weight_filler {\n type: \"xavier\"\n }\n bias_filler {\n type: \"constant\"\n }\n }\n}\nlayer {\n name: \"pool1\"\n type: \"Pooling\"\n bottom: \"conv1\"\n top: \"pool1\"\n pooling_param {\n pool: MAX\n kernel_size: 2\n stride: 2\n }\n}\nlayer {\n name: \"conv2\"\n type: \"Convolution\"\n bottom: \"pool1\"\n top: \"conv2\"\n param {\n lr_mult: 1\n }\n param {\n lr_mult: 2\n }\n convolution_param {\n num_output: 50\n kernel_size: 5\n stride: 1\n weight_filler {\n type: \"xavier\"\n }\n bias_filler {\n type: \"constant\"\n }\n }\n}\nlayer {\n name: \"pool2\"\n type: \"Pooling\"\n bottom: \"conv2\"\n top: \"pool2\"\n pooling_param {\n pool: MAX\n kernel_size: 2\n stride: 2\n }\n}\n\nlayer {\n name: \"fc1\"\n type: \"InnerProduct\"\n bottom: \"pool2\"\n top: \"fc1\"\n param {\n lr_mult: 1\n }\n param {\n lr_mult: 2\n }\n inner_product_param {\n num_output: 500\n weight_filler {\n type: \"xavier\"\n }\n bias_filler {\n type: \"constant\"\n }\n }\n}\nlayer {\n name: \"relu1\"\n type: \"ReLU\"\n bottom: \"fc1\"\n top: \"fc1\"\n}\nlayer {\n name: \"fc2\"\n type: \"InnerProduct\"\n bottom: \"fc1\"\n top: \"fc2\"\n param {\n lr_mult: 1\n }\n param {\n lr_mult: 2\n }\n inner_product_param {\n num_output: 10\n weight_filler {\n type: \"xavier\"\n }\n bias_filler {\n type: \"constant\"\n }\n }\n}\nlayer {\n name: \"prob\"\n type: \"Softmax\"\n bottom: \"fc2\"\n top: \"prob\"\n}" }, { "path": "models/labels.txt", "content": "0 0\n1 1\n2 2\n3 3\n4 4\n5 5\n6 6\n7 7\n8 8\n9 9\n" }, { "path": "models/plate_fromimg.prototxt", "content": "name: \"LeNet\"\nlayer {\n name: \"mnist\"\n type: \"ImageData\"\n top: \"data\"\n top: \"label\"\n image_data_param { \n source: \"util/train.txt\" \n root_folder: \"data/\" \n new_height: 20\n new_width: 20\n batch_size: 64\n shuffle: true \n } \n include: { phase: TRAIN }\n}\nlayer {\n name: \"mnist\"\n type: \"ImageData\"\n top: \"data\"\n top: \"label\"\n image_data_param { \n source: \"util/val.txt\"\n root_folder: \"data/\"\n new_height: 20\n new_width: 20\n batch_size: 64\n } \n include: { phase: TEST }\n}\nlayer {\n name: \"conv1\"\n type: \"Convolution\"\n bottom: \"data\"\n top: \"conv1\"\n param {\n lr_mult: 1\n }\n param {\n lr_mult: 2\n }\n convolution_param {\n num_output: 20\n kernel_size: 5\n stride: 1\n weight_filler {\n type: \"xavier\"\n }\n bias_filler {\n type: \"constant\"\n }\n }\n}\nlayer {\n name: \"pool1\"\n type: \"Pooling\"\n bottom: \"conv1\"\n top: \"pool1\"\n pooling_param {\n pool: MAX\n kernel_size: 2\n stride: 2\n }\n}\nlayer {\n name: \"conv2\"\n type: \"Convolution\"\n bottom: \"pool1\"\n top: \"conv2\"\n param {\n lr_mult: 1\n }\n param {\n lr_mult: 2\n }\n convolution_param {\n num_output: 50\n kernel_size: 5\n stride: 1\n weight_filler {\n type: \"xavier\"\n }\n bias_filler {\n type: \"constant\"\n }\n }\n}\nlayer {\n name: \"pool2\"\n type: \"Pooling\"\n bottom: \"conv2\"\n top: \"pool2\"\n pooling_param {\n pool: MAX\n kernel_size: 2\n stride: 2\n }\n}\nlayer {\n name: \"fc1\"\n type: \"InnerProduct\"\n bottom: \"pool2\"\n top: \"fc1\"\n param {\n lr_mult: 1\n }\n param {\n lr_mult: 2\n }\n inner_product_param {\n num_output: 500\n weight_filler {\n type: \"xavier\"\n }\n bias_filler {\n type: \"constant\"\n }\n }\n}\nlayer {\n name: \"relu1\"\n type: \"ReLU\"\n bottom: \"fc1\"\n top: \"fc1\"\n}\nlayer {\n name: \"fc2\"\n type: \"InnerProduct\"\n bottom: \"fc1\"\n top: \"fc2\"\n param {\n lr_mult: 1\n }\n param {\n lr_mult: 2\n }\n inner_product_param {\n num_output: 10\n weight_filler {\n type: \"xavier\"\n }\n bias_filler {\n type: \"constant\"\n }\n }\n}\nlayer {\n name: \"accuracy\"\n type: \"Accuracy\"\n bottom: \"fc2\"\n bottom: \"label\"\n top: \"accuracy\"\n include {\n phase: TEST\n }\n}\nlayer {\n name: \"loss\"\n type: \"SoftmaxWithLoss\"\n bottom: \"fc2\"\n bottom: \"label\"\n top: \"loss\"\n}" }, { "path": "models/plate_lenet.prototxt", "content": "name: \"Lenet\"\nlayer {\n name: \"Lenet\"\n type: \"Data\"\n top: \"data\"\n top: \"label\"\n transform_param {\n scale: 0.00390625\n }\n data_param {\n source: \"lmdb/train_lmdb\"\n backend: LMDB\n batch_size: 64\n }\n include: { phase: TRAIN }\n}\nlayer {\n name: \"Lenet\"\n type: \"Data\"\n top: \"data\"\n top: \"label\"\n transform_param {\n scale: 0.00390625\n }\n data_param {\n source: \"lmdb/val_lmdb\"\n backend: LMDB\n batch_size: 64\n }\n include: { phase: TEST }\n}\nlayer {\n name: \"conv1\"\n type: \"Convolution\"\n bottom: \"data\"\n top: \"conv1\"\n param {\n lr_mult: 1\n }\n param {\n lr_mult: 2\n }\n convolution_param {\n num_output: 20\n kernel_size: 5\n stride: 1\n weight_filler {\n type: \"xavier\"\n }\n bias_filler {\n type: \"constant\"\n }\n }\n}\nlayer {\n name: \"pool1\"\n type: \"Pooling\"\n bottom: \"conv1\"\n top: \"pool1\"\n pooling_param {\n pool: MAX\n kernel_size: 2\n stride: 2\n }\n}\nlayer {\n name: \"conv2\"\n type: \"Convolution\"\n bottom: \"pool1\"\n top: \"conv2\"\n param {\n lr_mult: 1\n }\n param {\n lr_mult: 2\n }\n convolution_param {\n num_output: 50\n kernel_size: 5\n stride: 1\n weight_filler {\n type: \"xavier\"\n }\n bias_filler {\n type: \"constant\"\n }\n }\n}\nlayer {\n name: \"pool2\"\n type: \"Pooling\"\n bottom: \"conv2\"\n top: \"pool2\"\n pooling_param {\n pool: MAX\n kernel_size: 2\n stride: 2\n }\n}\n\nlayer {\n name: \"fc1\"\n type: \"InnerProduct\"\n bottom: \"pool2\"\n top: \"fc1\"\n param {\n lr_mult: 1\n }\n param {\n lr_mult: 2\n }\n inner_product_param {\n num_output: 500\n weight_filler {\n type: \"xavier\"\n }\n bias_filler {\n type: \"constant\"\n }\n }\n}\nlayer {\n name: \"relu1\"\n type: \"ReLU\"\n bottom: \"fc1\"\n top: \"fc1\"\n}\nlayer {\n name: \"fc2\"\n type: \"InnerProduct\"\n bottom: \"fc1\"\n top: \"fc2\"\n param {\n lr_mult: 1\n }\n param {\n lr_mult: 2\n }\n inner_product_param {\n num_output: 10\n weight_filler {\n type: \"xavier\"\n }\n bias_filler {\n type: \"constant\"\n }\n }\n}\nlayer {\n name: \"accuracy\"\n type: \"Accuracy\"\n bottom: \"fc2\"\n bottom: \"label\"\n top: \"accuracy\"\n include {\n phase: TEST\n }\n}\nlayer {\n name: \"loss\"\n type: \"SoftmaxWithLoss\"\n bottom: \"fc2\"\n bottom: \"label\"\n top: \"loss\"\n}" }, { "path": "models/solver.prototxt", "content": "# The train/test net protocol buffer definition\nnet: \"models/plate_lenet.prototxt\"\n#net: \"models/plate_fromimg.prototxt\"\n# test_iter specifies how many forward passes the test should carry out.\n# In the case of MNIST, we have test batch size 100 and 100 test iterations,\n# covering the full 10,000 testing images.\ntest_iter: 24\n# Carry out testing every 500 training iterations.\ntest_interval: 1000\n# The base learning rate, momentum and the weight decay of the network.\nbase_lr: 0.01\nmomentum: 0.9\n#solver_type: ADAGRAD\nweight_decay: 0.0005\n#weight_decay: 0.0005\n# The learning rate policy\nlr_policy: \"inv\"\ngamma: 0.0001\npower: 0.75\n# Display every 100 iterations\ndisplay: 1000\n# The maximum number of iterations\nmax_iter: 10000\n# snapshot intermediate results\nsnapshot: 5000\nsnapshot_prefix: \"output/plate\"\n# solver mode: CPU or GPU\nsolver_mode: GPU\n" }, { "path": "train.bat", "content": "@echo off\nset CAFFE_DIR=..\nset DATA=data\nset REISZE_DIM=20\nset converttool=%CAFFE_DIR%/build/tools/convert_imageset\nset finetunemodel=models/plate999.caffemodel\nif exist \"lmdb/train_lmdb\" (set regenlmdb=0) else (set regenlmdb=1)\n::set regenlmdb=1\n\nif %regenlmdb% equ 1 goto regeneratelmdb\n\ngoto train\n:regeneratelmdb\necho \"Creating train lmdb...\"\ndel \"lmdb/train_lmdb\\*.*\" /f /s /q\ndel \"lmdb/val_lmdb\\*.*\" /f /s /q\nrd /s /q \"lmdb/train_lmdb\"\nrd /s /q \"lmdb/val_lmdb\"\nrd /s /q lmdb\nmkdir lmdb\n\"%converttool%\" --resize_height=%REISZE_DIM% --resize_width=%REISZE_DIM% --shuffle \"%DATA%/\" \"util/train.txt\" \"lmdb/train_lmdb\"\necho \"Creating val lmdb...\"\n\"%converttool%\" --resize_height=%REISZE_DIM% --resize_width=%REISZE_DIM% --shuffle \"%DATA%/\" \"util/val.txt\" \"lmdb/val_lmdb\"\n\necho \"Computing mean:\"\n\"%CAFFE_DIR%/build/tools/compute_image_mean\" \"lmdb/train_lmdb\" \"models/mean.binaryproto\"\n\n:train\nrem if exist %finetunemodel% (set extra_cmd=\"--weights=%finetunemodel%\")\nrem \"%CAFFE_DIR%/build/tools/caffe\" train --solver=models/solver.prototxt %extra_cmd% \npython3 train.py\n\n:evaluation\npython util/evaluation.py\necho \"Done\"\npause" }, { "path": "train.py", "content": "import os\nimport sys\nsys.path.insert(0, '../python')\nimport caffe\nfrom caffe import layers as L, params as P\nfrom caffe.proto import caffe_pb2\nimport lmdb\nfrom tqdm import tqdm\nimport logging\nimport cv2\nimport numpy as np\n\nroot_folder=\"data/\"\nbatch_size = 64\ntest_batch_size = 100\ninput_size = [20,20]\nG4 = 4*1024*1024*1024\n\ndef remove_if_exists(db):\n if os.path.exists(db):\n logger.info('remove %s'%db)\n shutil.rmtree(db)\n\ndef get_test_num(valpath = \"util/val.txt\"):\n with open(valpath) as f:\n lines = f.readlines()\n return len(lines)\n\ndef make_datum(img,label):\n return caffe_pb2.Datum(channels=3,width=input_size[0],height=input_size[1],label=label,\n data=np.rollaxis(img,2).tobytes())\n\ndef gen_data_layer(phase=\"train\",uselmdb=True):\n if uselmdb:\n source = \"lmdb/\"+phase+\"_lmdb\"\n if not os.path.exists(source):\n print(\"creating \"+source)\n os.makedirs(source)\n db = lmdb.open(source, map_size=G4)\n txn = db.begin(write=True)\n txtfile=\"util/\"+phase+\".txt\"\n with open(txtfile) as f:\n lines = f.readlines()\n for i,line in tqdm(enumerate(lines)):\n items = line.split()\n imgpath = root_folder+\"/\"+items[0]\n img = cv2.imread(imgpath)\n if img is None:\n logging.info(\"cannot read\"+imgpath)\n key = \"%08d_data\"%(i)\n label=int(items[1])\n txn.put(key,make_datum(img,label).SerializeToString())\n if i %1000 == 0:\n txn.commit()\n txn = db.begin(write=True)\n db.close()\n data, label = L.Data(batch_size=batch_size, backend=P.Data.LMDB,source=source,transform_param=dict(scale=1./255), ntop=2)\n else:\n txtfile=\"util/\"+phase+\".txt\"\n data, label = L.ImageData(image_data_param=dict(source=txtfile,root_folder=root_folder,batch_size=batch_size,shuffle=phase==\"train\",new_width=20,new_height=20),ntop=2,transform_param=dict(scale=1./255))\n return data,label\n\ndef lenet(phase=\"train\",batch_size=64):\n n = caffe.NetSpec() \n n.data, n.label = gen_data_layer(phase)\n n.conv1 = L.Convolution(n.data, kernel_size=5, num_output=20, weight_filler=dict(type='xavier'))\n n.pool1 = L.Pooling(n.conv1, kernel_size=2, stride=2, pool=P.Pooling.MAX)\n n.conv2 = L.Convolution(n.pool1, kernel_size=5, num_output=50, weight_filler=dict(type='xavier'))\n n.pool2 = L.Pooling(n.conv2, kernel_size=2, stride=2, pool=P.Pooling.MAX)\n n.fc1 = L.InnerProduct(n.pool2, num_output=500, weight_filler=dict(type='xavier'))\n n.relu1 = L.ReLU(n.fc1, in_place=True)\n n.fc2 = L.InnerProduct(n.relu1, num_output=10, weight_filler=dict(type='xavier'))\n n.acc = L.Accuracy(n.fc2, n.label)\n n.loss = L.SoftmaxWithLoss(n.fc2, n.label)\n return n\n\ndef lenet_deploy(net,deploy_net_file=\"util/deploy.prototxt\"):\n deploy_net = net\n with open(deploy_net_file, 'w') as f:\n net_param = deploy_net.to_proto()\n del net_param.layer[0]\n del net_param.layer[-1]\n del net_param.layer[-1]\n net_param.name = 'lenet'\n net_param.input.extend(['data'])\n net_param.input_shape.extend([\n caffe_pb2.BlobShape(dim=[1, 3, input_size[0], input_size[1]])]) \n f.write(str(net_param))\n\ndef gen_solver_txt(train_net_path, test_net_path):\n s = caffe_pb2.SolverParameter()\n s.train_net = train_net_path\n s.test_net.append(test_net_path)\n s.test_interval = 500\n s.test_iter.append(int(get_test_num()/test_batch_size))\n s.max_iter = 10000\n s.base_lr = 0.01\n s.lr_policy = 'step'\n s.gamma = 0.1\n s.power = 0.75\n s.stepsize = 5000\n s.momentum = 0.9\n s.weight_decay = 5e-4\n s.display = 1000\n s.snapshot = 5000\n s.snapshot_prefix = 'output/plate'\n s.solver_mode = caffe_pb2.SolverParameter.GPU\n return s\n\ndef main():\n train_net_path = 'util/train.prototxt'\n net = lenet('train',batch_size)\n with open(train_net_path, 'w') as f:\n f.write(str(net.to_proto()))\n test_net_path = 'util/test.prototxt'\n net = lenet('val',test_batch_size)\n with open(test_net_path, 'w') as f:\n f.write(str(net.to_proto()))\n lenet_deploy(net)\n solver_path = 'util/solver.prototxt'\n with open(solver_path, 'w') as f:\n f.write(str(gen_solver_txt(train_net_path, test_net_path)))\n caffe.set_mode_gpu()\n solver = caffe.get_solver(solver_path)\n solver.solve()\n\nif __name__==\"__main__\":\n main()" }, { "path": "train.sh", "content": "#!/usr/bin/env sh\nset -e\nDATA=data\nTOOLS=../build/tools\nregeneratelmdb=0\n\nconvert_lmdb(){\n RESIZE_HEIGHT=20\n RESIZE_WIDTH=20\n echo \"Creating train lmdb...\"\n GLOG_logtostderr=1 $TOOLS/convert_imageset \\\n --resize_height=$RESIZE_HEIGHT \\\n --resize_width=$RESIZE_WIDTH \\\n --shuffle $DATA/ \\\n util/train.txt \\\n lmdb/train_lmdb\n echo \"Creating val lmdb...\"\n GLOG_logtostderr=1 $TOOLS/convert_imageset \\\n --resize_height=$RESIZE_HEIGHT \\\n --resize_width=$RESIZE_WIDTH \\\n --shuffle $DATA/ \\\n util/val.txt \\\n lmdb/val_lmdb\n}\ncreate_lmdb(){\n if [ -d lmdb ] ; then\n if [ $regeneratelmdb -eq 1 ] ; then\n rm lmdb -rf\n mkdir lmdb\n convert_lmdb\n fi\n else\n mkdir lmdb\n convert_lmdb\n fi\n}\ntrain(){\n if [ ! -d output ]; then\n mkdir output\n fi\n latest=$(ls -t models/*.caffemodel | head -n 1)\n if [ -f ${latest} ]; then\n echo \"Resume training from ${latest}\"\n $TOOLS/caffe train --solver=models/solver.prototxt --weights=$latest\n else\n echo \"Start Training\"\n $TOOLS/caffe train --solver=models/solver.prototxt\n fi\n echo \"Done\"\n}\n# python evaluate\nevaluate(){\n latest=$(ls -t output/*.caffemodel | head -n 1)\n echo \"Evaluating \"${latest}\n python util/evaluation.py --weights=${latest}\n cd ..\n}\n# c++ evaluate\nrun(){\n if [ not -d build ]; then \n mkdir build\n fi\n cd build\n cmake ..\n make -j8\n ./evaluation\n #./cpp4caffe\n}\n# generate util/train.txt and val.txt for training\n# python3 util/preprocess.py\n#create_lmdb\n# train\npython train.py\nevaluate\n#run" }, { "path": "util/deploy.prototxt", "content": "name: \"lenet\"\ninput: \"data\"\ninput_shape {\n dim: 1\n dim: 3\n dim: 20\n dim: 20\n}\nlayer {\n name: \"conv1\"\n type: \"Convolution\"\n bottom: \"data\"\n top: \"conv1\"\n convolution_param {\n num_output: 20\n kernel_size: 5\n weight_filler {\n type: \"xavier\"\n }\n }\n}\nlayer {\n name: \"pool1\"\n type: \"Pooling\"\n bottom: \"conv1\"\n top: \"pool1\"\n pooling_param {\n pool: MAX\n kernel_size: 2\n stride: 2\n }\n}\nlayer {\n name: \"conv2\"\n type: \"Convolution\"\n bottom: \"pool1\"\n top: \"conv2\"\n convolution_param {\n num_output: 50\n kernel_size: 5\n weight_filler {\n type: \"xavier\"\n }\n }\n}\nlayer {\n name: \"pool2\"\n type: \"Pooling\"\n bottom: \"conv2\"\n top: \"pool2\"\n pooling_param {\n pool: MAX\n kernel_size: 2\n stride: 2\n }\n}\nlayer {\n name: \"fc1\"\n type: \"InnerProduct\"\n bottom: \"pool2\"\n top: \"fc1\"\n inner_product_param {\n num_output: 500\n weight_filler {\n type: \"xavier\"\n }\n }\n}\nlayer {\n name: \"relu1\"\n type: \"ReLU\"\n bottom: \"fc1\"\n top: \"fc1\"\n}\nlayer {\n name: \"fc2\"\n type: \"InnerProduct\"\n bottom: \"fc1\"\n top: \"fc2\"\n inner_product_param {\n num_output: 10\n weight_filler {\n type: \"xavier\"\n }\n }\n}\n" }, { "path": "util/evaluation.py", "content": "import os,argparse,sys,time,shutil\nimport numpy as np\nimport sys\nsys.path.append('../python')\nimport caffe\nimport logging\nfrom tqdm import tqdm\n\ndef create_logger(logdir=\"output\"):\n time_str = time.strftime('%Y%m%d-%H%M%S')\n log_file = '{}/{}.log'.format(logdir, time_str)\n head = '%(asctime)-15s %(message)s'\n logging.basicConfig(filename=str(log_file),format=head)\n logger = logging.getLogger()\n logger.setLevel(logging.INFO)\n console = logging.StreamHandler()\n logging.getLogger('').addHandler(console)\n\ndef clearlasterrors(args):\n if os.path.exists(\"error\"):\n subdirs=os.listdir(args.errordir)\n for subdir in subdirs:\n files=os.listdir(args.errordir+\"/\"+subdir)\n for file in files:\n os.remove(args.errordir+\"/\"+subdir+\"/\"+file)\n os.rmdir(args.errordir+\"/\"+subdir)\n\ndef loadmean(meanprotopath):\n blob = caffe.proto.caffe_pb2.BlobProto()\n blob.ParseFromString(open(meanprotopath, 'rb').read()) \n return np.array(caffe.io.blobproto_to_array(blob))[0]\n\ndef getclassifier(args):\n classifier = caffe.Classifier(args.modeldef, args.weights,image_dims=args.image_dims\n )\n ##mean=loadmean(args.meanfile).mean(1).mean(1),#raw_scale=255,channel_swap=[2,1,0]\n caffe.set_mode_gpu()\n return classifier\n\nclass EvalStatic:\n total = 0\n error = 0\n def __str__(self):\n return str(self.error)+\",\"+str(self.total)+\",\"+str(self.error*1.0/self.total)\n\ndef evaluationonebyone(args):\n labels=[w.split()[1] for w in open(args.labelfile).readlines()]\n classifier = getclassifier(args)\n start = time.time()\n if not os.path.exists(args.errordir):\n os.mkdir(args.errordir)\n subdirs=os.listdir(args.datadir)\n evalstatics=[]\n for subdir in subdirs:\n print(subdir+\":\")\n evalstatic=EvalStatic()\n files=os.listdir(args.datadir+'/'+subdir)\n evalstatic.total=len(files)\n for file in tqdm(files):\n imgpath=args.datadir+'/'+subdir+'/'+file\n inputs = [caffe.io.load_image(imgpath)]\n try:\n predictions = classifier.predict(inputs,oversample=False)\n except Exception as e:\n print(e)\n p=predictions[0,:].argmax()\n label=labels[p]\n if subdir!=label:\n logging.info(subdir+\" \"+file+\":\"+str(label))\n evalstatic.error=evalstatic.error+1\n if not os.path.exists(args.errordir+'/'+subdir):\n os.mkdir(args.errordir+'/'+subdir)\n errorfilepath=args.errordir+'/'+subdir+'/'+file[:-4]+\"_\"+subdir+'_'+label+'.jpg'\n shutil.copy(imgpath,errorfilepath)\n evalstatics.append(evalstatic)\n logging.info(\"Done in %.2f s.\" % (time.time() - start))\n totalcount=0\n error=0\n for i,evalstatic in enumerate(evalstatics):\n error=error+evalstatic.error\n totalcount=totalcount+evalstatic.total\n logging.info(subdirs[i]+\":\"+str(evalstatic))\n logging.info(\"Toal error\")\n logging.info(str(error)+\" \"+str(totalcount)+\" \"+str(error*1.0/totalcount))\n\ndef evaluation_batch(args):\n labels=[w.split()[1] for w in open(args.labelfile).readlines()]\n classifier=getclassifier(args)\n start = time.time()\n if not os.path.exists(args.errordir):\n os.mkdir(args.errordir)\n subdirs=os.listdir(args.datadir)\n evalstatics=[]\n for subdir in subdirs:\n print(subdir)\n evalstatic=EvalStatic()\n files=os.listdir(args.datadir+'/'+subdir)\n evalstatic.total=len(files)\n inputs=[caffe.io.load_image(args.datadir+'/'+subdir+'/'+file) for file in files]\n try:\n predictions = classifier.predict(inputs,oversample=False)\n except Exception as e:\n print(e)\n for i in tqdm(range(len(files))):\n p=predictions[i,:].argmax()\n label=labels[p]\n if subdir!=label:\n logging.info(subdir+\" \"+files[i]+\":\"+str(label))\n evalstatic.error=evalstatic.error+1\n if not os.path.exists(args.errordir+'/'+subdir):\n os.mkdir(args.errordir+'/'+subdir)\n imgpath=args.datadir+\"/\"+subdir+\"/\"+files[i]\n errorfilepath=args.errordir+'/'+subdir+'/'+files[i][:-4]+\"_\"+subdir+'_'+label+'.jpg'\n shutil.copy(imgpath,errorfilepath)\n evalstatics.append(evalstatic)\n logging.info(\"Done in %.2f s.\" % (time.time() - start))\n totalcount=0\n error=0\n for i,evalstatic in enumerate(evalstatics):\n error=error+evalstatic.error\n totalcount=totalcount+evalstatic.total\n logging.info(subdirs[i]+\":\"+str(evalstatic))\n logging.info(\"Toal error\")\n logging.info(str(error)+\" \"+str(totalcount)+\" \"+str(error*1.0/totalcount))\n\ndef get_args():\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--iter\",default=10000,help=\"caffemodel iter to evaluation\")\n parser.add_argument(\"--datadir\",default=\"data\",help=\"datadir\")\n parser.add_argument(\"--image_dims\",default=[20,20],help=\"image_dims\")\n parser.add_argument(\"--modeldef\",default=\"util/deploy.prototxt\",help=\"deploy file\")\n parser.add_argument(\"--weights\",default=\"models/plate999.caffemodel\",help=\"caffemodel\")\n parser.add_argument(\"--labelfile\",default=\"models/labels.txt\",help=\"label file\")\n parser.add_argument(\"--meanfile\",default=\"models/mean.binaryproto\",help=\"meanfile\")\n parser.add_argument(\"--errordir\",default=\"error\",help=\"errordir\")\n parser.add_argument(\"--logfile\",default=\"error.txt\",help=\"log txt\")\n parser.add_argument(\"--evaluationonebyone\",default=True,help=\"log txt\")\n parser.add_argument(\"--imgpath\",default=\"data/0/0.jpg\",help=\"image path\")\n args = parser.parse_args()\n return args\n\ndef classification():\n args = get_args()\n args = parser.parse_args()\n labels=[w.split()[1] for w in open(args.labelfile).readlines()]\n classifier=getclassifier(args)\n inputs=[caffe.io.load_image(args.imgpath)]\n predictions = classifier.predict(inputs,oversample=False)\n p=predictions[0,:].argmax()\n label=labels[p]\n print(label,predictions[0,p])\n top_inds = predictions[0,:].argsort()[::-1][:5]\n\ndef evaluation():\n args = get_args()\n clearlasterrors(args)\n create_logger()\n if args.evaluationonebyone:\n evaluationonebyone(args)\n else:\n evaluation_batch(args)\n\nif __name__=='__main__':\n evaluation()\n #classification()\n" }, { "path": "util/meanfilebinartynpy.py", "content": "import os\nimport sys\nsys.path.append('../../python')\n\nMEAN_BIN = \"../models/mean.binaryproto\"\nMEAN_NPY = \"../models/mean.npy\"\n\nfrom caffe.proto import caffe_pb2\nfrom caffe.io import blobproto_to_array\nprint('generating mean file...')\nmean_blob = caffe_pb2.BlobProto()\nmean_blob.ParseFromString(open(MEAN_BIN, 'rb').read())\n\nimport numpy as np\nmean_npy = blobproto_to_array(mean_blob)\nmean_npy_shape = mean_npy.shape\nmean_npy = mean_npy.reshape(mean_npy_shape[1], mean_npy_shape[2], mean_npy_shape[3])\n\nnp.save(open(MEAN_NPY, 'wb'), mean_npy)\nprint('done...')" }, { "path": "util/plotaccuracy.py", "content": "#coding =utf-8\n#迷若烟雨@Baidu\nimport re\nimport matplotlib.pyplot as plt\nimport os\nimport sys\nimport datetime\n\ndef getlastesttraininfofilefromdir(logdir):\n logfiles=os.listdir(logdir)\n infologfiles=filter(lambda s:s.startswith('INFO'),logfiles)\n infologfiles=filter(lambda s:s.endswith('.txt'),infologfiles)\n if infologfiles:\n lastestfile=infologfiles[0]\n maxtm=0\n for logf in infologfiles:\n path = os.path.join(logdir, logf) \n timestamp = os.path.getmtime(path)\n date = datetime.datetime.fromtimestamp(timestamp)\n if timestamp>maxtm:\n lastestfile=path\n return lastestfile\n else:\n return None\ndef plotaccuarcy():\n logdir='log'\n infologfile=getlastesttraininfofilefromdir(logdir)\n print(infologfile)\n if infologfile:\n # infologfile='../log/INFO2015-11-19T19-45-15.txt'\n f=open(infologfile)\n lines=f.read()\n #print lines\n iterations=re.findall('Iteration \\d*',lines)\n accuracysstrings=re.findall('accuracy = \\d*.\\d*',lines)\n trainlosstrings=re.findall('Train net output #0: loss = \\d*.\\d*',lines)\n testlossstrings=re.findall('Test net output #1: loss = \\d*.\\d*',lines)\n f.close()\n accuracys=[ac[11:] for ac in accuracysstrings]\n trainlosses=[loss[27:-1]for loss in trainlosstrings]\n testlosses=[loss[27:-1]for loss in testlossstrings]\n #for ac in accuracys:\n # print ac\n plt.plot(range(len(accuracys)),accuracys)\n #plt.plot(range(len(trainlosses)),trainlosses)\n #plt.plot(range(len(testlosses)),testlosses)\n plt.show()\nif __name__==\"__main__\":\n plotaccuarcy()" }, { "path": "util/preprocess.py", "content": "import os,argparse,random,shutil\n\ndef main(args):\n datadir=args.rootdir+\"/\"+args.dataname\n print(\"loading data from \"+datadir+\":\")\n trainfile=open(\"util/train.txt\",\"w\")\n valfile=open(\"util/val.txt\",\"w\")\n categoryfile=open(\"models/labels.txt\",'w')\n paths=os.listdir(datadir)\n classindex=0\n trainpaths=[]\n valpaths=[]\n categorys=[]\n for subdir in paths:\n if(os.path.isdir(datadir+\"/\"+subdir)):\n categorys.append(str(classindex)+\" \"+subdir+\"\\n\")\n files=os.listdir(datadir+\"/\"+subdir)\n files=[file for file in files]\n random.shuffle(files)\n print(subdir,len(files))\n num2train=len(files)*args.trainrtaio\n for fileindex,file in enumerate(files):\n if fileindex