\nAll Questions(78) \nQ1:_Some_sample_uses_of_libsvm(2) Q2:_Installation_and_running_the_program(13) Q3:_Data_preparation(7) Q4:_Training_and_prediction(34) Q5:_Probability_outputs(3) Q6:_Graphic_interface(3) Q7:_Java_version_of_libsvm(4) Q8:_Python_interface(1) Q9:_MATLAB_interface(11) \nSome courses which have used libsvm as a tool Some applications/tools which have used libsvm Where can I find documents/videos of libsvm ? Where are change log and earlier versions? How to cite LIBSVM? I would like to use libsvm in my software. Is there any license problem? Is there a repository of additional tools based on libsvm? On unix machines, I got \"error in loading shared libraries\" or \"cannot open shared object file.\" What happened ? I have modified the source and would like to build the graphic interface \"svm-toy\" on MS windows. How should I do it ? I am an MS windows user but why only one (svm-toy) of those precompiled .exe actually runs ? What is the difference between \".\" and \"*\" outputed during training? Why occasionally the program (including MATLAB or other interfaces) crashes and gives a segmentation fault? How to build a dynamic library (.dll file) on MS windows? On some systems (e.g., Ubuntu), compiling LIBSVM gives many warning messages. Is this a problem and how to disable the warning message? In LIBSVM, why you don't use certain C/C++ library functions to make the code shorter? Why sometimes not all attributes of a data appear in the training/model files ? What if my data are non-numerical ? Why do you consider sparse format ? Will the training of dense data be much slower ? Why sometimes the last line of my data is not read by svm-train? Is there a program to check if my data are in the correct format? May I put comments in data files? How to convert other data formats to LIBSVM format? The output of training C-SVM is like the following. What do they mean? Can you explain more about the model file? Should I use float or double to store numbers in the cache ? How do I choose the kernel? Does libsvm have special treatments for linear SVM? The number of free support vectors is large. What should I do? Should I scale training and testing data in a similar way? Does it make a big difference if I scale each attribute to [0,1] instead of [-1,1]? The prediction rate is low. How could I improve it? My data are unbalanced. Could libsvm handle such problems? What is the difference between nu-SVC and C-SVC? The program keeps running (without showing any output). What should I do? The program keeps running (with output, i.e. many dots). What should I do? The training time is too long. What should I do? Does shrinking always help? How do I get the decision value(s)? How do I get the distance between a point and the hyperplane? On 32-bit machines, if I use a large cache (i.e. large -m) on a linux machine, why sometimes I get \"segmentation fault ?\" How do I disable screen output of svm-train? I would like to use my own kernel. Any example? In svm.cpp, there are two subroutines for kernel evaluations: k_function() and kernel_function(). Which one should I modify ? What method does libsvm use for multi-class SVM ? Why don't you use the \"1-against-the rest\" method? How does LIBSVM perform parameter selection for multi-class problems? After doing cross validation, why there is no model file outputted ? Why my cross-validation results are different from those in the Practical Guide? On some systems CV accuracy is the same in several runs. How could I use different data partitions? In other words, how do I set random seed in LIBSVM? I would like to solve L2-loss SVM (i.e., error term is quadratic). How should I modify the code ? How do I choose parameters for one-class svm as training data are in only one class? Why the code gives NaN (not a number) results? Why on windows sometimes grid.py fails? Why grid.py/easy.py sometimes generates the following warning message? Why the sign of predicted labels and decision values are sometimes reversed? I don't know class labels of test data. What should I put in the first column of the test file? How can I use OpenMP to parallelize LIBSVM on a multicore/shared-memory computer? How could I know which training instances are support vectors? Why training a probability model (i.e., -b 1) takes a longer time? Why using the -b option does not give me better accuracy? Why using svm-predict -b 0 and -b 1 gives different accuracy values? How can I save images drawn by svm-toy? I press the \"load\" button to load data points but why svm-toy does not draw them ? I would like svm-toy to handle more than three classes of data, what should I do ? What is the difference between Java version and C++ version of libsvm? Is the Java version significantly slower than the C++ version? While training I get the following error message: java.lang.OutOfMemoryError. What is wrong? Why you have the main source file svm.m4 and then transform it to svm.java? Except the python-C++ interface provided, could I use Jython to call libsvm ? I compile the MATLAB interface without problem, but why errors occur while running it? On 64bit Windows I compile the MATLAB interface without problem, but why errors occur while running it? Does the MATLAB interface provide a function to do scaling? How could I use MATLAB interface for parameter selection? I use MATLAB parallel programming toolbox on a multi-core environment for parameter selection. Why the program is even slower? How do I use LIBSVM with OpenMP under MATLAB? How could I generate the primal variable w of linear SVM? Is there an OCTAVE interface for libsvm? How to handle the name conflict between svmtrain in the libsvm matlab interface and that in MATLAB bioinformatics toolbox? On Windows I got an error message \"Invalid MEX-file: Specific module not found\" when running the pre-built MATLAB interface in the windows sub-directory. What should I do? LIBSVM supports 1-vs-1 multi-class classification. If instead I would like to use 1-vs-rest, how to implement it using MATLAB interface?
\n \nQ: Some courses which have used libsvm as a tool \nInstitute for Computer Science, \nFaculty of Applied Science, University of Freiburg, Germany \n \n \nDivision of Mathematics and Computer Science. \nFaculteit der Exacte Wetenschappen \nVrije Universiteit, The Netherlands. \n \n\nElectrical and Computer Engineering Department, \nUniversity of Wisconsin-Madison \n \n \n\nTechnion (Israel Institute of Technology), Israel.\n\n\nComputer and Information Sciences Dept., University of Florida \n \n\nThe Institute of Computer Science,\nUniversity of Nairobi, Kenya. \n \n\nApplied Mathematics and Computer Science, University of Iceland.\n\n\nSVM tutorial in machine learning\nsummer school, University of Chicago, 2005.\n \n \n\n[Go Top] \n
Q: Some applications/tools which have used libsvm \n[Go Top] \n
Q: Where can I find documents/videos of libsvm ? \n\n
\n\nOfficial implementation document:\npdf , ps.gz ,\nACM digital lib .\n\n\n Instructions for using LIBSVM are in the README files in the main directory and some sub-directories.\n \nA guide for beginners:\n\nA practical guide to support vector classification\n \n An introductory video \nfor windows users.\n\n \n\n[Go Top] \n
Q: Where are change log and earlier versions? See the change log .\n\n
You can download earlier versions \nhere .\n
\n[Go Top] \n
Q: How to cite LIBSVM? \nPlease cite the following paper:\n
\nChih-Chung Chang and Chih-Jen Lin, LIBSVM\n: a library for support vector machines.\nACM Transactions on Intelligent Systems and Technology, 2:27:1--27:27, 2011.\nSoftware available at http://www.csie.ntu.edu.tw/~cjlin/libsvm\n
\nThe bibtex format is \n
\n@article{CC01a,\n author = {Chang, Chih-Chung and Lin, Chih-Jen},\n title = {{LIBSVM}: A library for support vector machines},\n journal = {ACM Transactions on Intelligent Systems and Technology},\n volume = {2},\n issue = {3},\n year = {2011},\n pages = {27:1--27:27},\n note =\t {Software available at \\url{http://www.csie.ntu.edu.tw/~cjlin/libsvm}}\n}\n \n\n[Go Top] \n
Q: I would like to use libsvm in my software. Is there any license problem? \nThe libsvm license (\"the modified BSD license\")\nis compatible with many\nfree software licenses such as GPL. Hence, it is very easy to\nuse libsvm in your software.\nPlease check the COPYRIGHT file in detail. Basically\nyou need to \n
\n\nClearly indicate that LIBSVM is used.\n \n\nRetain the LIBSVM COPYRIGHT file in your software.\n \n \nIt can also be used in commercial products.\n\n[Go Top] \n
Q: Is there a repository of additional tools based on libsvm? \nYes, see libsvm \ntools \n
\n[Go Top] \n
Q: On unix machines, I got \"error in loading shared libraries\" or \"cannot open shared object file.\" What happened ? \nThis usually happens if you compile the code\non one machine and run it on another which has incompatible\nlibraries.\nTry to recompile the program on that machine or use static linking.\n
\n[Go Top] \n
Q: I have modified the source and would like to build the graphic interface \"svm-toy\" on MS windows. How should I do it ? \nBuild it as a project by choosing \"Win32 Project.\"\nOn the other hand, for \"svm-train\" and \"svm-predict\"\nyou want to choose \"Win32 Console Project.\"\nAfter libsvm 2.5, you can also use the file Makefile.win.\nSee details in README.\n\n\n
\nIf you are not using Makefile.win and see the following \nlink error\n
\nLIBCMTD.lib(wwincrt0.obj) : error LNK2001: unresolved external symbol\n_wWinMain@16\n \nyou may have selected a wrong project type.\n\n[Go Top] \n
Q: I am an MS windows user but why only one (svm-toy) of those precompiled .exe actually runs ? \nYou need to open a command window \nand type svmtrain.exe to see all options.\nSome examples are in README file.\n
\n[Go Top] \n
Q: What is the difference between \".\" and \"*\" outputed during training? \n\".\" means every 1,000 iterations (or every #data \niterations is your #data is less than 1,000).\n\"*\" means that after iterations of using\na smaller shrunk problem, \nwe reset to use the whole set. See the \nimplementation document for details.\n
\n[Go Top] \n
Q: Why occasionally the program (including MATLAB or other interfaces) crashes and gives a segmentation fault? \nVery likely the program consumes too much memory than what the \noperating system can provide. Try a smaller data and see if the \nprogram still crashes.\n
\n[Go Top] \n
Q: How to build a dynamic library (.dll file) on MS windows? \n\nThe easiest way is to use Makefile.win.\nSee details in README.\n\nAlternatively, you can use Visual C++. Here is \nthe example using Visual Studio .Net 2008:\n
\nCreate a Win32 empty DLL project and set (in Project->$Project_Name\nProperties...->Configuration) to \"Release.\"\n About how to create a new dynamic link library, please refer to\nhttp://msdn2.microsoft.com/en-us/library/ms235636(VS.80).aspx \n\n Add svm.cpp, svm.h to your project.\n Add __WIN32__ and _CRT_SECURE_NO_DEPRECATE to Preprocessor definitions (in\nProject->$Project_Name Properties...->C/C++->Preprocessor)\n Set Create/Use Precompiled Header to Not Using Precompiled Headers\n(in Project->$Project_Name Properties...->C/C++->Precompiled Headers)\n Set the path for the Modulation Definition File svm.def (in \nProject->$Project_Name Properties...->Linker->input\n Build the DLL.\n Rename the dll file to libsvm.dll and move it to the correct path.\n \n\n\n\n[Go Top] \n
Q: On some systems (e.g., Ubuntu), compiling LIBSVM gives many warning messages. Is this a problem and how to disable the warning message? \nThe warning message is like\n
\nsvm.cpp:2730: warning: ignoring return value of int fscanf(FILE*, const char*, ...), declared with attribute warn_unused_result\n \nThis is not a problem; see this page for more \ndetails of ubuntu systems.\nIn the future we may modify the code\nso that these messages do not appear.\nAt this moment, to disable the warning message you can replace\n\nCFLAGS = -Wall -Wconversion -O3 -fPIC\n \nwith\n\nCFLAGS = -Wall -Wconversion -O3 -fPIC -U_FORTIFY_SOURCE\n \nin Makefile.\n\n[Go Top] \n
Q: In LIBSVM, why you don't use certain C/C++ library functions to make the code shorter? \nFor portability, we use only features defined in ISO C89. Note that features in ISO C99 may not be available everywhere. \nEven the newest gcc lacks some features in C99 (see http://gcc.gnu.org/c99status.html for details).\nIf the situation changes in the future, \nwe might consider using these newer features.\n
\n[Go Top] \n
Q: Why sometimes not all attributes of a data appear in the training/model files ? \nlibsvm uses the so called \"sparse\" format where zero\nvalues do not need to be stored. Hence a data with attributes\n
\n1 0 2 0\n \nis represented as\n\n1:1 3:2\n \n\n[Go Top] \n
Q: What if my data are non-numerical ? \nCurrently libsvm supports only numerical data.\nYou may have to change non-numerical data to \nnumerical. For example, you can use several\nbinary attributes to represent a categorical\nattribute.\n
\n[Go Top] \n
Q: Why do you consider sparse format ? Will the training of dense data be much slower ? \nThis is a controversial issue. The kernel\nevaluation (i.e. inner product) of sparse vectors is slower \nso the total training time can be at least twice or three times\nof that using the dense format.\nHowever, we cannot support only dense format as then we CANNOT\nhandle extremely sparse cases. Simplicity of the code is another\nconcern. Right now we decide to support\nthe sparse format only.\n
\n[Go Top] \n
Q: Why sometimes the last line of my data is not read by svm-train? \nWe assume that you have '\\n' in the end of\neach line. So please press enter in the end\nof your last line.\n
\n[Go Top] \n
Q: Is there a program to check if my data are in the correct format? \nThe svm-train program in libsvm conducts only a simple check of the input data. To do a\ndetailed check, after libsvm 2.85, you can use the python script tools/checkdata.py. See tools/README for details.\n
\n[Go Top] \n
Q: May I put comments in data files? \nWe don't officially support this. But, cureently LIBSVM\nis able to process data in the following\nformat:\n
\n1 1:2 2:1 # your comments\n \nNote that the character \":\" should not appear in your\ncomments.\n\n\n[Go Top] \n
Q: How to convert other data formats to LIBSVM format? \nIt depends on your data format. A simple way is to use\nlibsvmwrite in the libsvm matlab/octave interface.\n\nTake a CSV (comma-separated values) file\nin UCI machine learning repository as an example.\nWe download SPECTF.train . \nLabels are in the first column. The following steps produce\na file in the libsvm format.\n
\nmatlab> SPECTF = csvread('SPECTF.train'); % read a csv file\nmatlab> labels = SPECTF(:, 1); % labels from the 1st column\nmatlab> features = SPECTF(:, 2:end); \nmatlab> features_sparse = sparse(features); % features must be in a sparse matrix\nmatlab> libsvmwrite('SPECTFlibsvm.train', labels, features_sparse);\n \nThe tranformed data are stored in SPECTFlibsvm.train.\n\n\nAlternatively, you can use convert.c \nto convert CSV format to libsvm format.\n
\n[Go Top] \n
Q: The output of training C-SVM is like the following. What do they mean? \nobj is the optimal objective value of the dual SVM problem.\nrho is the bias term in the decision function\nsgn(w^Tx - rho).\nnSV and nBSV are number of support vectors and bounded support\nvectors (i.e., alpha_i = C). nu-svm is a somewhat equivalent\nform of C-SVM where C is replaced by nu. nu simply shows the\ncorresponding parameter. More details are in\n\nlibsvm document .\n
\n[Go Top] \n
Q: Can you explain more about the model file? \nIn the model file, after parameters and other informations such as labels , each line represents a support vector.\nSupport vectors are listed in the order of \"labels\" shown earlier.\n(i.e., those from the first class in the \"labels\" list are\ngrouped first, and so on.) \nIf k is the total number of classes,\nin front of a support vector in class j, there are\nk-1 coefficients \ny*alpha where alpha are dual solution of the\nfollowing two class problems:\n
\n+-+-+-+--------------------+\n|1|1|1| |\n|v|v|v| SVs from class 1 |\n|2|3|4| |\n+-+-+-+--------------------+\n|1|2|2| |\n|v|v|v| SVs from class 2 |\n|2|3|4| |\n+-+-+-+--------------------+\n|1|2|3| |\n|v|v|v| SVs from class 3 |\n|3|3|4| |\n+-+-+-+--------------------+\n|1|2|3| |\n|v|v|v| SVs from class 4 |\n|4|4|4| |\n+-+-+-+--------------------+\n \nSee also\n an illustration using\nMATLAB/OCTAVE. \n\n[Go Top] \n
Q: Should I use float or double to store numbers in the cache ? \nWe have float as the default as you can store more numbers\nin the cache. \nIn general this is good enough but for few difficult\ncases (e.g. C very very large) where solutions are huge\nnumbers, it might be possible that the numerical precision is not\nenough using only float.\n
\n[Go Top] \n
Q: How do I choose the kernel? \nIn general we suggest you to try the RBF kernel first.\nA recent result by Keerthi and Lin\n(\ndownload paper here )\nshows that if RBF is used with model selection,\nthen there is no need to consider the linear kernel.\nThe kernel matrix using sigmoid may not be positive definite\nand in general it's accuracy is not better than RBF.\n(see the paper by Lin and Lin\n(\ndownload paper here ).\nPolynomial kernels are ok but if a high degree is used,\nnumerical difficulties tend to happen\n(thinking about dth power of (<1) goes to 0\nand (>1) goes to infinity).\n
\n[Go Top] \n
Q: Does libsvm have special treatments for linear SVM? \n\nNo, libsvm solves linear/nonlinear SVMs by the\nsame way.\nSome tricks may save training/testing time if the\nlinear kernel is used,\nso libsvm is NOT particularly efficient for linear SVM,\nespecially when\nC is large and\nthe number of data is much larger\nthan the number of attributes.\nYou can either\n
\n\n Please also see our SVM guide \non the discussion of using RBF and linear\nkernels.\n
\n[Go Top] \n
Q: The number of free support vectors is large. What should I do? \nThis usually happens when the data are overfitted.\nIf attributes of your data are in large ranges,\ntry to scale them. Then the region\nof appropriate parameters may be larger.\nNote that there is a scale program\nin libsvm. \n
\n[Go Top] \n
Q: Should I scale training and testing data in a similar way? \nYes, you can do the following:\n
\n> svm-scale -s scaling_parameters train_data > scaled_train_data\n> svm-scale -r scaling_parameters test_data > scaled_test_data\n \n\n[Go Top] \n
Q: Does it make a big difference if I scale each attribute to [0,1] instead of [-1,1]? \nFor the linear scaling method, if the RBF kernel is\nused and parameter selection is conducted, there\nis no difference. Assume Mi and mi are \nrespectively the maximal and minimal values of the\nith attribute. Scaling to [0,1] means\n
\n x'=(x-mi)/(Mi-mi)\n \nFor [-1,1],\n\n x''=2(x-mi)/(Mi-mi)-1.\n \nIn the RBF kernel,\n\n x'-y'=(x-y)/(Mi-mi), x''-y''=2(x-y)/(Mi-mi).\n \nHence, using (C,g) on the [0,1]-scaled data is the\nsame as (C,g/2) on the [-1,1]-scaled data.\n\n Though the performance is the same, the computational\ntime may be different. For data with many zero entries,\n[0,1]-scaling keeps the sparsity of input data and hence\nmay save the time.\n
\n[Go Top] \n
Q: The prediction rate is low. How could I improve it? \nTry to use the model selection tool grid.py in the python\ndirectory find\nout good parameters. To see the importance of model selection,\nplease \nsee my talk:\n\nA practical guide to support vector \nclassification \n \n
\n[Go Top] \n
Q: My data are unbalanced. Could libsvm handle such problems? \nYes, there is a -wi options. For example, if you use\n
\n> svm-train -s 0 -c 10 -w1 1 -w-1 5 data_file\n \n\nthe penalty for class \"-1\" is larger.\nNote that this -w option is for C-SVC only.\n
\n[Go Top] \n
Q: What is the difference between nu-SVC and C-SVC? \nBasically they are the same thing but with different\nparameters. The range of C is from zero to infinity\nbut nu is always between [0,1]. A nice property\nof nu is that it is related to the ratio of \nsupport vectors and the ratio of the training\nerror.\n
\n[Go Top] \n
Q: The program keeps running (without showing any output). What should I do? \nYou may want to check your data. Each training/testing\ndata must be in one line. It cannot be separated.\nIn addition, you have to remove empty lines.\n
\n[Go Top] \n
Q: The program keeps running (with output, i.e. many dots). What should I do? \nIn theory libsvm guarantees to converge.\nTherefore, this means you are\nhandling ill-conditioned situations\n(e.g. too large/small parameters) so numerical\ndifficulties occur.\n
\nYou may get better numerical stability by replacing\n
\ntypedef float Qfloat;\n \nin svm.cpp with\n\ntypedef double Qfloat;\n \nThat is, elements in the kernel cache are stored\nin double instead of single. However, this means fewer elements\ncan be put in the kernel cache.\n\n[Go Top] \n
Q: The training time is too long. What should I do? \nFor large problems, please specify enough cache size (i.e.,\n-m).\nSlow convergence may happen for some difficult cases (e.g. -c is large).\nYou can try to use a looser stopping tolerance with -e.\nIf that still doesn't work, you may train only a subset of the data.\nYou can use the program subset.py in the directory \"tools\" \nto obtain a random subset.\n\n
\nIf you have extremely large data and face this difficulty, please\ncontact us. We will be happy to discuss possible solutions.\n\n
When using large -e, you may want to check if -h 0 (no shrinking) or -h 1 (shrinking) is faster.\nSee a related question below.\n\n
\n[Go Top] \n
Q: Does shrinking always help? \nIf the number of iterations is high, then shrinking\noften helps.\nHowever, if the number of iterations is small\n(e.g., you specify a large -e), then\nprobably using -h 0 (no shrinking) is better.\nSee the \nimplementation document for details.\n
\n[Go Top] \n
Q: How do I get the decision value(s)? \nWe print out decision values for regression. For classification,\nwe solve several binary SVMs for multi-class cases. You\ncan obtain values by easily calling the subroutine\nsvm_predict_values. Their corresponding labels\ncan be obtained from svm_get_labels. \nDetails are in \nREADME of libsvm package. \n\n
\nIf you are using MATLAB/OCTAVE interface, svmpredict can directly\ngive you decision values. Please see matlab/README for details.\n\n
\nWe do not recommend the following. But if you would\nlike to get values for \nTWO-class classification with labels +1 and -1\n(note: +1 and -1 but not things like 5 and 10)\nin the easiest way, simply add \n
\n\t\tprintf(\"%f\\n\", dec_values[0]*model->label[0]);\n \nafter the line\n\n\t\tsvm_predict_values(model, x, dec_values);\n \nof the file svm.cpp.\nPositive (negative)\ndecision values correspond to data predicted as +1 (-1).\n\n\n\n[Go Top] \n
Q: How do I get the distance between a point and the hyperplane? \nThe distance is |decision_value| / |w|. \nWe have |w|^2 = w^Tw = alpha^T Q alpha = 2*(dual_obj + sum alpha_i). \nThus in svm.cpp please find the place \nwhere we calculate the dual objective value\n(i.e., the subroutine Solve())\nand add a statement to print w^Tw.\n\n
\n[Go Top] \n
Q: On 32-bit machines, if I use a large cache (i.e. large -m) on a linux machine, why sometimes I get \"segmentation fault ?\" \n\nOn 32-bit machines, the maximum addressable\nmemory is 4GB. The Linux kernel uses 3:1\nsplit which means user space is 3G and\nkernel space is 1G. Although there are\n3G user space, the maximum dynamic allocation\nmemory is 2G. So, if you specify -m near 2G,\nthe memory will be exhausted. And svm-train\nwill fail when it asks more memory.\nFor more details, please read \n\nthis article .\n
\nThe easiest solution is to switch to a\n 64-bit machine.\nOtherwise, there are two ways to solve this. If your\nmachine supports Intel's PAE (Physical Address\nExtension), you can turn on the option HIGHMEM64G\nin Linux kernel which uses 4G:4G split for\nkernel and user space. If you don't, you can\ntry a software `tub' which can eliminate the 2G\nboundary for dynamic allocated memory. The `tub'\nis available at \nhttp://www.bitwagon.com/tub.html .\n\n\n\n
\n[Go Top] \n
Q: How do I disable screen output of svm-train? \nFor commend-line users, use the -q option:\n
\n> ./svm-train -q heart_scale\n \n\nFor library users, set the global variable\n
\nextern void (*svm_print_string) (const char *);\n \nto specify the output format. You can disable the output by the following steps:\n\n\nDeclare a function to output nothing:\n\nvoid print_null(const char *s) {}\n \n \n\nAssign the output function of libsvm by\n\nsvm_print_string = &print_null;\n \n \n \nFinally, a way used in earlier libsvm\nis by updating svm.cpp from\n\n#if 1\nvoid info(const char *fmt,...)\n \nto\n\n#if 0\nvoid info(const char *fmt,...)\n \n\n[Go Top] \n
Q: I would like to use my own kernel. Any example? In svm.cpp, there are two subroutines for kernel evaluations: k_function() and kernel_function(). Which one should I modify ? \nAn example is \"LIBSVM for string data\" in LIBSVM Tools.\n
\nThe reason why we have two functions is as follows.\nFor the RBF kernel exp(-g |xi - xj|^2), if we calculate\nxi - xj first and then the norm square, there are 3n operations.\nThus we consider exp(-g (|xi|^2 - 2dot(xi,xj) +|xj|^2))\nand by calculating all |xi|^2 in the beginning, \nthe number of operations is reduced to 2n.\nThis is for the training. For prediction we cannot\ndo this so a regular subroutine using that 3n operations is\nneeded.\n\nThe easiest way to have your own kernel is\nto put the same code in these two\nsubroutines by replacing any kernel.\n
\n[Go Top] \n
Q: What method does libsvm use for multi-class SVM ? Why don't you use the \"1-against-the rest\" method? \nIt is one-against-one. We chose it after doing the following\ncomparison:\nC.-W. Hsu and C.-J. Lin.\n\nA comparison of methods \nfor multi-class support vector machines\n , \nIEEE Transactions on Neural Networks , 13(2002), 415-425.\n\n
\n\"1-against-the rest\" is a good method whose performance\nis comparable to \"1-against-1.\" We do the latter\nsimply because its training time is shorter.\n
\n[Go Top] \n
Q: How does LIBSVM perform parameter selection for multi-class problems? \nLIBSVM implements \"one-against-one\" multi-class method, so there are \nk(k-1)/2 binary models, where k is the number of classes.\n\n
\nWe can consider two ways to conduct parameter selection.\n\n
\n\nFor any two classes of data, a parameter selection procedure is conducted. Finally,\neach decision function has its own optimal parameters.\n \n\nThe same parameters are used for all k(k-1)/2 binary classification problems.\nWe select parameters that achieve the highest overall performance.\n \n \n\nEach has its own advantages. A\nsingle parameter set may not be uniformly good for all k(k-1)/2 decision functions.\nHowever, as the overall accuracy is the final consideration, one parameter set \nfor one decision function may lead to over-fitting. In the paper\n\nChen, Lin, and Schölkopf,\n\nA tutorial on nu-support vector machines.\n \nApplied Stochastic Models in Business and Industry, 21(2005), 111-136,\n\n
\nthey have experimentally\nshown that the two methods give similar performance.\nTherefore, currently the parameter selection in LIBSVM\ntakes the second approach by considering the same parameters for\nall k(k-1)/2 models.\n
\n[Go Top] \n
Q: After doing cross validation, why there is no model file outputted ? \nCross validation is used for selecting good parameters.\nAfter finding them, you want to re-train the whole\ndata without the -v option.\n
\n[Go Top] \n
Q: Why my cross-validation results are different from those in the Practical Guide? \n\nDue to random partitions of\nthe data, on different systems CV accuracy values\nmay be different.\n
\n[Go Top] \n
Q: On some systems CV accuracy is the same in several runs. How could I use different data partitions? In other words, how do I set random seed in LIBSVM? \nIf you use GNU C library,\nthe default seed 1 is considered. Thus you always\nget the same result of running svm-train -v.\nTo have different seeds, you can add the following code\nin svm-train.c:\n
\n#include <time.h>\n \nand in the beginning of main(),\n\nsrand(time(0));\n \nAlternatively, if you are not using GNU C library\nand would like to use a fixed seed, you can have\n\nsrand(1);\n \n\n\nFor Java, the random number generator\nis initialized using the time information.\nSo results of two CV runs are different.\nTo fix the seed, after version 3.1 (released\nin mid 2011), you can add\n
\nsvm.rand.setSeed(0);\n \nin the main() function of svm_train.java.\n\n\nIf you use CV to select parameters, it is recommended to use identical folds\nunder different parameters. In this case, you can consider fixing the seed.\n
\n[Go Top] \n
Q: I would like to solve L2-loss SVM (i.e., error term is quadratic). How should I modify the code ? \nIt is extremely easy. Taking c-svc for example, to solve\n
\nmin_w w^Tw/2 + C \\sum max(0, 1- (y_i w^Tx_i+b))^2,\n
\nonly two \nplaces of svm.cpp have to be changed. \nFirst, modify the following line of \nsolve_c_svc from \n
\n\ts.Solve(l, SVC_Q(*prob,*param,y), minus_ones, y,\n\t\talpha, Cp, Cn, param->eps, si, param->shrinking);\n \nto\n\n\ts.Solve(l, SVC_Q(*prob,*param,y), minus_ones, y,\n\t\talpha, INF, INF, param->eps, si, param->shrinking);\n \nSecond, in the class of SVC_Q, declare C as \na private variable:\n\n\tdouble C;\n \nIn the constructor replace\n\n\tfor(int i=0;i<prob.l;i++)\n\t\tQD[i]= (Qfloat)(this->*kernel_function)(i,i);\n \nwith\n\n this->C = param.C;\n\tfor(int i=0;i<prob.l;i++)\n\t\tQD[i]= (Qfloat)(this->*kernel_function)(i,i)+0.5/C;\n \nThen in the subroutine get_Q, after the for loop, add\n\n if(i >= start && i < len) \n\t\tdata[i] += 0.5/C;\n \n\n\nFor one-class svm, the modification is exactly the same. For SVR, you don't need an if statement like the above. Instead, you only need a simple assignment:\n
\n\tdata[real_i] += 0.5/C;\n \n\n\n\nFor large linear L2-loss SVM, please use\nLIBLINEAR .\n
\n[Go Top] \n
Q: How do I choose parameters for one-class svm as training data are in only one class? \nYou have pre-specified true positive rate in mind and then search for\nparameters which achieve similar cross-validation accuracy.\n
\n[Go Top] \n
Q: Why the code gives NaN (not a number) results? \nThis rarely happens, but few users reported the problem.\nIt seems that their \ncomputers for training libsvm have the VPN client\nrunning. The VPN software has some bugs and causes this\nproblem. Please try to close or disconnect the VPN client.\n
\n[Go Top] \n
Q: Why on windows sometimes grid.py fails? \n\nThis problem shouldn't happen after version\n2.85. If you are using earlier versions,\nplease download the latest one.\n\n\n
\n[Go Top] \n
Q: Why grid.py/easy.py sometimes generates the following warning message? \nWarning: empty z range [62.5:62.5], adjusting to [61.875:63.125]\nNotice: cannot contour non grid data!\n \nNothing is wrong and please disregard the \nmessage. It is from gnuplot when drawing\nthe contour. \n
\n[Go Top] \n
Q: Why the sign of predicted labels and decision values are sometimes reversed? Nothing is wrong. Very likely you have two labels +1/-1 and the first instance in your data\nhas -1.\nThink about the case of labels +5/+10. Since\nSVM needs to use +1/-1, internally\nwe map +5/+10 to +1/-1 according to which\nlabel appears first.\nHence a positive decision value implies\nthat we should predict the \"internal\" +1,\nwhich may not be the +1 in the input file.\n\n
\n[Go Top] \n
Q: I don't know class labels of test data. What should I put in the first column of the test file? Any value is ok. In this situation, what you will use is the output file of svm-predict, which gives predicted class labels.\n\n\n
\n[Go Top] \n
Q: How can I use OpenMP to parallelize LIBSVM on a multicore/shared-memory computer? It is very easy if you are using GCC 4.2\nor after. \n\n
In Makefile, add -fopenmp to CFLAGS.\n\n
In class SVC_Q of svm.cpp, modify the for loop\nof get_Q to:\n
\n#pragma omp parallel for private(j) \n\t\t\tfor(j=start;j<len;j++)\n \n In the subroutine svm_predict_values of svm.cpp, add one line to the for loop:\n
\n#pragma omp parallel for private(i) \n\t\tfor(i=0;i<l;i++)\n\t\t\tkvalue[i] = Kernel::k_function(x,model->SV[i],model->param);\n \nFor regression, you need to modify\nclass SVR_Q instead. The loop in svm_predict_values\nis also different because you need\na reduction clause for the variable sum:\n\n#pragma omp parallel for private(i) reduction(+:sum) \n\t\tfor(i=0;i<model->l;i++)\n\t\t\tsum += sv_coef[i] * Kernel::k_function(x,model->SV[i],model->param);\n \n\n Then rebuild the package. Kernel evaluations in training/testing will be parallelized. An example of running this modification on\nan 8-core machine using the data set\nijcnn1 :\n\n
8 cores:\n
\n%setenv OMP_NUM_THREADS 8\n%time svm-train -c 16 -g 4 -m 400 ijcnn1\n27.1sec\n \n1 core:\n\n%setenv OMP_NUM_THREADS 1\n%time svm-train -c 16 -g 4 -m 400 ijcnn1\n79.8sec\n \nFor this data, kernel evaluations take 80% of training time. In the above example, we assume you use csh. For bash, use\n\nexport OMP_NUM_THREADS=8\n \ninstead.\n\n For Python interface, you need to add the -lgomp link option:\n
\n$(CXX) -lgomp -shared -dynamiclib svm.o -o libsvm.so.$(SHVER)\n \n\n For MS Windows, you need to add /openmp in CFLAGS of Makefile.win\n\n
\n[Go Top] \n
Q: How could I know which training instances are support vectors? \nIt's very simple. Since version 3.13, you can use the function\n
\nvoid svm_get_sv_indices(const struct svm_model *model, int *sv_indices)\n \nto get indices of support vectors. For example, in svm-train.c, after\n\n\t\tmodel = svm_train(&prob, ¶m);\n \nyou can add\n\n\t\tint nr_sv = svm_get_nr_sv(model);\n\t\tint *sv_indices = Malloc(int, nr_sv);\n\t\tsvm_get_sv_indices(model, sv_indices);\n\t\tfor (int i=0; i<nr_sv; i++)\n\t\t\tprintf(\"instance %d is a support vector\\n\", sv_indices[i]);\n \n\n If you use matlab interface, you can directly check\n
\nmodel.sv_indices\n \n\n[Go Top] \n
Q: Why training a probability model (i.e., -b 1) takes a longer time? \nTo construct this probability model, we internally conduct a \ncross validation, which is more time consuming than\na regular training.\nHence, in general you do parameter selection first without\n-b 1. You only use -b 1 when good parameters have been\nselected. In other words, you avoid using -b 1 and -v\ntogether.\n
\n[Go Top] \n
Q: Why using the -b option does not give me better accuracy? \nThere is absolutely no reason the probability outputs guarantee\nyou better accuracy. The main purpose of this option is\nto provide you the probability estimates, but not to boost\nprediction accuracy. From our experience, \nafter proper parameter selections, in general with\nand without -b have similar accuracy. Occasionally there\nare some differences.\nIt is not recommended to compare the two under \njust a fixed parameter\nset as more differences will be observed.\n
\n[Go Top] \n
Q: Why using svm-predict -b 0 and -b 1 gives different accuracy values? \nLet's just consider two-class classification here. After probability information is obtained in training,\nwe do not have\n
\nprob > = 0.5 if and only if decision value >= 0.\n
\nSo predictions may be different with -b 0 and 1.\n
\n[Go Top] \n
Q: How can I save images drawn by svm-toy? \nFor Microsoft windows, first press the \"print screen\" key on the keyboard.\nOpen \"Microsoft Paint\" \n(included in Windows) \nand press \"ctrl-v.\" Then you can clip\nthe part of picture which you want.\nFor X windows, you can \nuse the program \"xv\" or \"import\" to grab the picture of the svm-toy window.\n
\n[Go Top] \n
Q: I press the \"load\" button to load data points but why svm-toy does not draw them ? \nThe program svm-toy assumes both attributes (i.e. x-axis and y-axis\nvalues) are in (0,1). Hence you want to scale your \ndata to between a small positive number and \na number less than but very close to 1.\nMoreover, class labels must be 1, 2, or 3\n(not 1.0, 2.0 or anything else).\n
\n[Go Top] \n
Q: I would like svm-toy to handle more than three classes of data, what should I do ? \nTaking windows/svm-toy.cpp as an example, you need to\nmodify it and the difference\nfrom the original file is as the following: (for five classes of\ndata)\n
\n30,32c30\n< \tRGB(200,0,200),\n< \tRGB(0,160,0),\n< \tRGB(160,0,0)\n---\n> \tRGB(200,0,200)\n39c37\n< HBRUSH brush1, brush2, brush3, brush4, brush5;\n---\n> HBRUSH brush1, brush2, brush3;\n113,114d110\n< \tbrush4 = CreateSolidBrush(colors[7]);\n< \tbrush5 = CreateSolidBrush(colors[8]);\n155,157c151\n< \telse if(v==3) return brush3;\n< \telse if(v==4) return brush4;\n< \telse return brush5;\n---\n> \telse return brush3;\n325d318\n< \t int colornum = 5;\n327c320\n< \t\tsvm_node *x_space = new svm_node[colornum * prob.l];\n---\n> \t\tsvm_node *x_space = new svm_node[3 * prob.l];\n333,338c326,331\n< \t\t\tx_space[colornum * i].index = 1;\n< \t\t\tx_space[colornum * i].value = q->x;\n< \t\t\tx_space[colornum * i + 1].index = 2;\n< \t\t\tx_space[colornum * i + 1].value = q->y;\n< \t\t\tx_space[colornum * i + 2].index = -1;\n< \t\t\tprob.x[i] = &x_space[colornum * i];\n---\n> \t\t\tx_space[3 * i].index = 1;\n> \t\t\tx_space[3 * i].value = q->x;\n> \t\t\tx_space[3 * i + 1].index = 2;\n> \t\t\tx_space[3 * i + 1].value = q->y;\n> \t\t\tx_space[3 * i + 2].index = -1;\n> \t\t\tprob.x[i] = &x_space[3 * i];\n397c390\n< \t\t\t\tif(current_value > 5) current_value = 1;\n---\n> \t\t\t\tif(current_value > 3) current_value = 1;\n \n\n[Go Top] \n
Q: What is the difference between Java version and C++ version of libsvm? \nThey are the same thing. We just rewrote the C++ code\nin Java.\n
\n[Go Top] \n
Q: Is the Java version significantly slower than the C++ version? \nThis depends on the VM you used. We have seen good\nVM which leads the Java version to be quite competitive with\nthe C++ code. (though still slower)\n
\n[Go Top] \n
Q: While training I get the following error message: java.lang.OutOfMemoryError. What is wrong? \nYou should try to increase the maximum Java heap size.\nFor example,\n
\njava -Xmx2048m -classpath libsvm.jar svm_train ...\n \nsets the maximum heap size to 2048M.\n\n[Go Top] \n
Q: Why you have the main source file svm.m4 and then transform it to svm.java? \nUnlike C, Java does not have a preprocessor built-in.\nHowever, we need some macros (see first 3 lines of svm.m4).\n\n\n
\n[Go Top] \n
Q: Except the python-C++ interface provided, could I use Jython to call libsvm ? Yes, here are some examples:\n\n
\n$ export CLASSPATH=$CLASSPATH:~/libsvm-2.91/java/libsvm.jar\n$ ./jython\nJython 2.1a3 on java1.3.0 (JIT: jitc)\nType \"copyright\", \"credits\" or \"license\" for more information.\n>>> from libsvm import *\n>>> dir()\n['__doc__', '__name__', 'svm', 'svm_model', 'svm_node', 'svm_parameter',\n'svm_problem']\n>>> x1 = [svm_node(index=1,value=1)]\n>>> x2 = [svm_node(index=1,value=-1)]\n>>> param = svm_parameter(svm_type=0,kernel_type=2,gamma=1,cache_size=40,eps=0.001,C=1,nr_weight=0,shrinking=1)\n>>> prob = svm_problem(l=2,y=[1,-1],x=[x1,x2])\n>>> model = svm.svm_train(prob,param)\n*\noptimization finished, #iter = 1\nnu = 1.0\nobj = -1.018315639346838, rho = 0.0\nnSV = 2, nBSV = 2\nTotal nSV = 2\n>>> svm.svm_predict(model,x1)\n1.0\n>>> svm.svm_predict(model,x2)\n-1.0\n>>> svm.svm_save_model(\"test.model\",model)\n\n \n\n\n[Go Top] \n
Q: I compile the MATLAB interface without problem, but why errors occur while running it? \nYour compiler version may not be supported/compatible for MATLAB.\nPlease check this MATLAB page first and then specify the version\nnumber. For example, if g++ X.Y is supported, replace\n
\nCXX = g++\n \nin the Makefile with\n\nCXX = g++-X.Y\n \n\n[Go Top] \n
Q: On 64bit Windows I compile the MATLAB interface without problem, but why errors occur while running it? \n\n\nPlease make sure that you use\nthe -largeArrayDims option in make.m. For example,\n
\nmex -largeArrayDims -O -c svm.cpp\n \n\nMoreover, if you use Microsoft Visual Studio, \nprobabally it is not properly installed. \nSee the explanation \nhere .\n\n[Go Top] \n
Q: Does the MATLAB interface provide a function to do scaling? \nIt is extremely easy to do scaling under MATLAB.\nThe following one-line code scale each feature to the range\nof [0,1]:\n
\n(data - repmat(min(data,[],1),size(data,1),1))*spdiags(1./(max(data,[],1)-min(data,[],1))',0,size(data,2),size(data,2))\n \n\n[Go Top] \n
Q: How could I use MATLAB interface for parameter selection? \nOne can do this by a simple loop. \nSee the following example:\n
\nbestcv = 0;\nfor log2c = -1:3,\n for log2g = -4:1,\n cmd = ['-v 5 -c ', num2str(2^log2c), ' -g ', num2str(2^log2g)];\n cv = svmtrain(heart_scale_label, heart_scale_inst, cmd);\n if (cv >= bestcv),\n bestcv = cv; bestc = 2^log2c; bestg = 2^log2g;\n end\n fprintf('%g %g %g (best c=%g, g=%g, rate=%g)\\n', log2c, log2g, cv, bestc, bestg, bestcv);\n end\nend\n \nYou may adjust the parameter range in the above loops.\n\n[Go Top] \n
Q: I use MATLAB parallel programming toolbox on a multi-core environment for parameter selection. Why the program is even slower? \nFabrizio Lacalandra of University of Pisa reported this issue.\nIt seems the problem is caused by the screen output.\nIf you disable the info function\nusing
#if 0, then the problem\nmay be solved.\n\n[Go Top] \n
Q: How do I use LIBSVM with OpenMP under MATLAB? \nIn Makefile,\nyou need to add -fopenmp to CFLAGS and -lgomp to MEX_OPTION. For Octave, you need the same modification.\n\n
However, a minor problem is that\nthe number of threads cannot\nbe specified in MATLAB. We tried Version 7.12 (R2011a) and gcc-4.6.1.\n\n
\n% export OMP_NUM_THREADS=4; matlab\n>> setenv('OMP_NUM_THREADS', '1');\n \n\nThen OMP_NUM_THREADS is still 4 while running the program. Please contact us if you \nsee how to solve this problem. You can, however,\nspecify the number in the source code (thanks\nto comments from Ricardo Santiago-mozos):\n\n#pragma omp parallel for private(i) num_threads(4)\n \n\n[Go Top] \n
Q: How could I generate the primal variable w of linear SVM? \nLet's start from the binary class and\nassume you have two labels -1 and +1.\nAfter obtaining the model from calling svmtrain,\ndo the following to have w and b:\n
\nw = model.SVs' * model.sv_coef;\nb = -model.rho;\n\nif model.Label(1) == -1\n w = -w;\n b = -b;\nend\n \nIf you do regression or one-class SVM, then the if statement is not needed.\n\n For multi-class SVM, we illustrate the setting\nin the following example of running the iris\ndata, which have 3 classes\n
\n> [y, x] = libsvmread('../../htdocs/libsvmtools/datasets/multiclass/iris.scale');\n> m = svmtrain(y, x, '-t 0')\n\nm = \n\n Parameters: [5x1 double]\n nr_class: 3\n totalSV: 42\n rho: [3x1 double]\n Label: [3x1 double]\n ProbA: []\n ProbB: []\n nSV: [3x1 double]\n sv_coef: [42x2 double]\n SVs: [42x4 double]\n \nsv_coef is like:\n\n+-+-+--------------------+\n|1|1| |\n|v|v| SVs from class 1 |\n|2|3| |\n+-+-+--------------------+\n|1|2| |\n|v|v| SVs from class 2 |\n|2|3| |\n+-+-+--------------------+\n|1|2| |\n|v|v| SVs from class 3 |\n|3|3| |\n+-+-+--------------------+\n \nso we need to see nSV of each classes.\n \n> m.nSV\n\nans =\n\n 3\n 21\n 18\n \nSuppose the goal is to find the vector w of classes \n1 vs 3. Then\ny_i alpha_i of training 1 vs 3 are\n \n> coef = [m.sv_coef(1:3,2); m.sv_coef(25:42,1)];\n \nand SVs are:\n \n> SVs = [m.SVs(1:3,:); m.SVs(25:42,:)];\n \nHence, w is\n\n> w = SVs'*coef;\n \nFor rho,\n\n> m.rho\n\nans =\n\n 1.1465\n 0.3682\n -1.9969\n> b = -m.rho(2);\n \nbecause rho is arranged by 1vs2 1vs3 2vs3.\n\n\n \n\n[Go Top] \n
Q: Is there an OCTAVE interface for libsvm? \nYes, after libsvm 2.86, the matlab interface\nworks on OCTAVE as well. Please use make.m by typing\n
\n>> make \n \nunder OCTAVE.\n\n[Go Top] \n
Q: How to handle the name conflict between svmtrain in the libsvm matlab interface and that in MATLAB bioinformatics toolbox? \nThe easiest way is to rename the svmtrain binary \nfile (e.g., svmtrain.mexw32 on 32-bit windows) \nto a different\nname (e.g., svmtrain2.mexw32).\n
\n[Go Top] \n
Q: On Windows I got an error message \"Invalid MEX-file: Specific module not found\" when running the pre-built MATLAB interface in the windows sub-directory. What should I do? \n\nThe error usually happens\nwhen there are missing runtime components\nsuch as MSVCR100.dll on your Windows platform.\nYou can use tools such as \nDependency \nWalker to find missing library files.\n\n
\nFor example, if the pre-built MEX files are compiled by\nVisual C++ 2010,\nyou must have installed\nMicrosoft Visual C++ Redistributable Package 2010\n(vcredist_x86.exe). You can easily find the freely\navailable file from Microsoft's web site. \n\n
\nFor 64bit Windows, the situation is similar. If\nthe pre-built files are by\nVisual C++ 2008, then you must have\nMicrosoft Visual C++ Redistributable Package 2008\n(vcredist_x64.exe).\n
\n[Go Top] \n
Q: LIBSVM supports 1-vs-1 multi-class classification. If instead I would like to use 1-vs-rest, how to implement it using MATLAB interface? \nPlease use code in the following directory . The following example shows how to\ntrain and test the problem dna (training and testing ).\n\n
Load, train and predict data:\n
\n[trainY trainX] = libsvmread('./dna.scale');\n[testY testX] = libsvmread('./dna.scale.t');\nmodel = ovrtrain(trainY, trainX, '-c 8 -g 4');\n[pred ac decv] = ovrpredict(testY, testX, model);\nfprintf('Accuracy = %g%%\\n', ac * 100);\n \nConduct CV on a grid of parameters \n\nbestcv = 0;\nfor log2c = -1:2:3,\n for log2g = -4:2:1,\n cmd = ['-q -c ', num2str(2^log2c), ' -g ', num2str(2^log2g)];\n cv = get_cv_ac(trainY, trainX, cmd, 3);\n if (cv >= bestcv),\n bestcv = cv; bestc = 2^log2c; bestg = 2^log2g;\n end\n fprintf('%g %g %g (best c=%g, g=%g, rate=%g)\\n', log2c, log2g, cv, bestc, bestg, bestcv);\n end\nend\n \n\n[Go Top] \n
\nLIBSVM home page \n
\n\n\n"
},
{
"path": "binaries/windows/x86/README",
"content": "Libsvm is a simple, easy-to-use, and efficient software for SVM\nclassification and regression. It solves C-SVM classification, nu-SVM\nclassification, one-class-SVM, epsilon-SVM regression, and nu-SVM\nregression. It also provides an automatic model selection tool for\nC-SVM classification. This document explains the use of libsvm.\n\nLibsvm is available at \nhttp://www.csie.ntu.edu.tw/~cjlin/libsvm\nPlease read the COPYRIGHT file before using libsvm.\n\nTable of Contents\n=================\n\n- Quick Start\n- Installation and Data Format\n- `svm-train' Usage\n- `svm-predict' Usage\n- `svm-scale' Usage\n- Tips on Practical Use\n- Examples\n- Precomputed Kernels \n- Library Usage\n- Java Version\n- Building Windows Binaries\n- Additional Tools: Sub-sampling, Parameter Selection, Format checking, etc.\n- MATLAB/OCTAVE Interface\n- Python Interface\n- Additional Information\n\nQuick Start\n===========\n\nIf you are new to SVM and if the data is not large, please go to \n`tools' directory and use easy.py after installation. It does \neverything automatic -- from data scaling to parameter selection.\n\nUsage: easy.py training_file [testing_file]\n\nMore information about parameter selection can be found in\n`tools/README.'\n\nInstallation and Data Format\n============================\n\nOn Unix systems, type `make' to build the `svm-train' and `svm-predict'\nprograms. Run them without arguments to show the usages of them.\n\nOn other systems, consult `Makefile' to build them (e.g., see\n'Building Windows binaries' in this file) or use the pre-built\nbinaries (Windows binaries are in the directory `windows').\n\nThe format of training and testing data file is:\n\n : : ...\n.\n.\n.\n\nEach line contains an instance and is ended by a '\\n' character. For\nclassification, is an integer indicating the class label\n(multi-class is supported). For regression, is the target\nvalue which can be any real number. For one-class SVM, it's not used\nso can be any number. The pair : gives a feature\n(attribute) value: is an integer starting from 1 and \nis a real number. The only exception is the precomputed kernel, where\n starts from 0; see the section of precomputed kernels. Indices\nmust be in ASCENDING order. Labels in the testing file are only used\nto calculate accuracy or errors. If they are unknown, just fill the\nfirst column with any numbers.\n\nA sample classification data included in this package is\n`heart_scale'. To check if your data is in a correct form, use\n`tools/checkdata.py' (details in `tools/README').\n\nType `svm-train heart_scale', and the program will read the training\ndata and output the model file `heart_scale.model'. If you have a test\nset called heart_scale.t, then type `svm-predict heart_scale.t\nheart_scale.model output' to see the prediction accuracy. The `output'\nfile contains the predicted class labels.\n\nFor classification, if training data are in only one class (i.e., all\nlabels are the same), then `svm-train' issues a warning message:\n`Warning: training data in only one class. See README for details,'\nwhich means the training data is very unbalanced. The label in the\ntraining data is directly returned when testing.\n\nThere are some other useful programs in this package.\n\nsvm-scale:\n\n\tThis is a tool for scaling input data file.\n\nsvm-toy:\n\n\tThis is a simple graphical interface which shows how SVM\n\tseparate data in a plane. You can click in the window to \n\tdraw data points. Use \"change\" button to choose class \n\t1, 2 or 3 (i.e., up to three classes are supported), \"load\"\n\tbutton to load data from a file, \"save\" button to save data to\n\ta file, \"run\" button to obtain an SVM model, and \"clear\"\n\tbutton to clear the window.\n\n\tYou can enter options in the bottom of the window, the syntax of\n\toptions is the same as `svm-train'.\n\n\tNote that \"load\" and \"save\" consider dense data format both in\n\tclassification and the regression cases. For classification,\n\teach data point has one label (the color) that must be 1, 2,\n\tor 3 and two attributes (x-axis and y-axis values) in\n\t[0,1). For regression, each data point has one target value\n\t(y-axis) and one attribute (x-axis values) in [0, 1).\n\n\tType `make' in respective directories to build them.\n\n\tYou need Qt library to build the Qt version.\n\t(available from http://www.trolltech.com)\n\n\tYou need GTK+ library to build the GTK version.\n\t(available from http://www.gtk.org)\n\t\n\tThe pre-built Windows binaries are in the `windows'\n\tdirectory. We use Visual C++ on a 32-bit machine, so the\n\tmaximal cache size is 2GB.\n\n`svm-train' Usage\n=================\n\nUsage: svm-train [options] training_set_file [model_file]\noptions:\n-s svm_type : set type of SVM (default 0)\n\t0 -- C-SVC\t\t(multi-class classification)\n\t1 -- nu-SVC\t\t(multi-class classification)\n\t2 -- one-class SVM\t\n\t3 -- epsilon-SVR\t(regression)\n\t4 -- nu-SVR\t\t(regression)\n-t kernel_type : set type of kernel function (default 2)\n\t0 -- linear: u'*v\n\t1 -- polynomial: (gamma*u'*v + coef0)^degree\n\t2 -- radial basis function: exp(-gamma*|u-v|^2)\n\t3 -- sigmoid: tanh(gamma*u'*v + coef0)\n\t4 -- precomputed kernel (kernel values in training_set_file)\n-d degree : set degree in kernel function (default 3)\n-g gamma : set gamma in kernel function (default 1/num_features)\n-r coef0 : set coef0 in kernel function (default 0)\n-c cost : set the parameter C of C-SVC, epsilon-SVR, and nu-SVR (default 1)\n-n nu : set the parameter nu of nu-SVC, one-class SVM, and nu-SVR (default 0.5)\n-p epsilon : set the epsilon in loss function of epsilon-SVR (default 0.1)\n-m cachesize : set cache memory size in MB (default 100)\n-e epsilon : set tolerance of termination criterion (default 0.001)\n-h shrinking : whether to use the shrinking heuristics, 0 or 1 (default 1)\n-b probability_estimates : whether to train a SVC or SVR model for probability estimates, 0 or 1 (default 0)\n-wi weight : set the parameter C of class i to weight*C, for C-SVC (default 1)\n-v n: n-fold cross validation mode\n-q : quiet mode (no outputs)\n\n\nThe k in the -g option means the number of attributes in the input data.\n\noption -v randomly splits the data into n parts and calculates cross\nvalidation accuracy/mean squared error on them.\n\nSee libsvm FAQ for the meaning of outputs.\n\n`svm-predict' Usage\n===================\n\nUsage: svm-predict [options] test_file model_file output_file\noptions:\n-b probability_estimates: whether to predict probability estimates, 0 or 1 (default 0); for one-class SVM only 0 is supported\n\nmodel_file is the model file generated by svm-train.\ntest_file is the test data you want to predict.\nsvm-predict will produce output in the output_file.\n\n`svm-scale' Usage\n=================\n\nUsage: svm-scale [options] data_filename\noptions:\n-l lower : x scaling lower limit (default -1)\n-u upper : x scaling upper limit (default +1)\n-y y_lower y_upper : y scaling limits (default: no y scaling)\n-s save_filename : save scaling parameters to save_filename\n-r restore_filename : restore scaling parameters from restore_filename\n\nSee 'Examples' in this file for examples.\n\nTips on Practical Use\n=====================\n\n* Scale your data. For example, scale each attribute to [0,1] or [-1,+1].\n* For C-SVC, consider using the model selection tool in the tools directory.\n* nu in nu-SVC/one-class-SVM/nu-SVR approximates the fraction of training\n errors and support vectors.\n* If data for classification are unbalanced (e.g. many positive and\n few negative), try different penalty parameters C by -wi (see\n examples below).\n* Specify larger cache size (i.e., larger -m) for huge problems.\n\nExamples\n========\n\n> svm-scale -l -1 -u 1 -s range train > train.scale\n> svm-scale -r range test > test.scale\n\nScale each feature of the training data to be in [-1,1]. Scaling\nfactors are stored in the file range and then used for scaling the\ntest data.\n\n> svm-train -s 0 -c 5 -t 2 -g 0.5 -e 0.1 data_file \n\nTrain a classifier with RBF kernel exp(-0.5|u-v|^2), C=10, and\nstopping tolerance 0.1.\n\n> svm-train -s 3 -p 0.1 -t 0 data_file\n\nSolve SVM regression with linear kernel u'v and epsilon=0.1\nin the loss function.\n\n> svm-train -c 10 -w1 1 -w-2 5 -w4 2 data_file\n\nTrain a classifier with penalty 10 = 1 * 10 for class 1, penalty 50 =\n5 * 10 for class -2, and penalty 20 = 2 * 10 for class 4.\n\n> svm-train -s 0 -c 100 -g 0.1 -v 5 data_file\n\nDo five-fold cross validation for the classifier using\nthe parameters C = 100 and gamma = 0.1\n\n> svm-train -s 0 -b 1 data_file\n> svm-predict -b 1 test_file data_file.model output_file\n\nObtain a model with probability information and predict test data with\nprobability estimates\n\nPrecomputed Kernels \n===================\n\nUsers may precompute kernel values and input them as training and\ntesting files. Then libsvm does not need the original\ntraining/testing sets.\n\nAssume there are L training instances x1, ..., xL and. \nLet K(x, y) be the kernel\nvalue of two instances x and y. The input formats\nare:\n\nNew training instance for xi:\n\n 0:i 1:K(xi,x1) ... L:K(xi,xL) \n\nNew testing instance for any x:\n\n 0:? 1:K(x,x1) ... L:K(x,xL) \n\nThat is, in the training file the first column must be the \"ID\" of\nxi. In testing, ? can be any value.\n\nAll kernel values including ZEROs must be explicitly provided. Any\npermutation or random subsets of the training/testing files are also\nvalid (see examples below).\n\nNote: the format is slightly different from the precomputed kernel\npackage released in libsvmtools earlier.\n\nExamples:\n\n\tAssume the original training data has three four-feature\n\tinstances and testing data has one instance:\n\n\t15 1:1 2:1 3:1 4:1\n\t45 2:3 4:3\n\t25 3:1\n\n\t15 1:1 3:1\n\n\tIf the linear kernel is used, we have the following new\n\ttraining/testing sets:\n\n\t15 0:1 1:4 2:6 3:1\n\t45 0:2 1:6 2:18 3:0 \n\t25 0:3 1:1 2:0 3:1\n \n\t15 0:? 1:2 2:0 3:1\n\n\t? can be any value.\n\n\tAny subset of the above training file is also valid. For example,\n\n\t25 0:3 1:1 2:0 3:1\n\t45 0:2 1:6 2:18 3:0 \n\n\timplies that the kernel matrix is\n\n\t\t[K(2,2) K(2,3)] = [18 0]\n\t\t[K(3,2) K(3,3)] = [0 1]\n\nLibrary Usage\n=============\n\nThese functions and structures are declared in the header file\n`svm.h'. You need to #include \"svm.h\" in your C/C++ source files and\nlink your program with `svm.cpp'. You can see `svm-train.c' and\n`svm-predict.c' for examples showing how to use them. We define\nLIBSVM_VERSION and declare `extern int libsvm_version; ' in svm.h, so\nyou can check the version number.\n\nBefore you classify test data, you need to construct an SVM model\n(`svm_model') using training data. A model can also be saved in\na file for later use. Once an SVM model is available, you can use it\nto classify new data.\n\n- Function: struct svm_model *svm_train(const struct svm_problem *prob,\n\t\t\t\t\tconst struct svm_parameter *param);\n\n This function constructs and returns an SVM model according to\n the given training data and parameters.\n\n struct svm_problem describes the problem:\n\t\n\tstruct svm_problem\n\t{\n\t\tint l;\n\t\tdouble *y;\n\t\tstruct svm_node **x;\n\t};\n \n where `l' is the number of training data, and `y' is an array containing\n their target values. (integers in classification, real numbers in\n regression) `x' is an array of pointers, each of which points to a sparse\n representation (array of svm_node) of one training vector. \n\n For example, if we have the following training data:\n\n LABEL\tATTR1\tATTR2\tATTR3\tATTR4\tATTR5\n -----\t-----\t-----\t-----\t-----\t-----\n 1\t\t 0\t 0.1\t 0.2\t 0\t 0\n 2\t\t 0\t 0.1\t 0.3\t -1.2\t 0\n 1\t\t 0.4\t 0\t 0\t 0\t 0\n 2\t\t 0\t 0.1\t 0\t 1.4\t 0.5\n 3\t\t -0.1\t -0.2\t 0.1\t 1.1\t 0.1\n\n then the components of svm_problem are:\n\n l = 5\n\n y -> 1 2 1 2 3\n\n x -> [ ] -> (2,0.1) (3,0.2) (-1,?)\n\t [ ] -> (2,0.1) (3,0.3) (4,-1.2) (-1,?)\n\t [ ] -> (1,0.4) (-1,?)\n\t [ ] -> (2,0.1) (4,1.4) (5,0.5) (-1,?)\n\t [ ] -> (1,-0.1) (2,-0.2) (3,0.1) (4,1.1) (5,0.1) (-1,?)\n\n where (index,value) is stored in the structure `svm_node':\n\n\tstruct svm_node\n\t{\n\t\tint index;\n\t\tdouble value;\n\t};\n\n index = -1 indicates the end of one vector. Note that indices must\n be in ASCENDING order.\n \n struct svm_parameter describes the parameters of an SVM model:\n\n\tstruct svm_parameter\n\t{\n\t\tint svm_type;\n\t\tint kernel_type;\n\t\tint degree;\t/* for poly */\n\t\tdouble gamma;\t/* for poly/rbf/sigmoid */\n\t\tdouble coef0;\t/* for poly/sigmoid */\n\n\t\t/* these are for training only */\n\t\tdouble cache_size; /* in MB */\n\t\tdouble eps;\t/* stopping criteria */\n\t\tdouble C;\t/* for C_SVC, EPSILON_SVR, and NU_SVR */\n\t\tint nr_weight;\t\t/* for C_SVC */\n\t\tint *weight_label;\t/* for C_SVC */\n\t\tdouble* weight;\t\t/* for C_SVC */\n\t\tdouble nu;\t/* for NU_SVC, ONE_CLASS, and NU_SVR */\n\t\tdouble p;\t/* for EPSILON_SVR */\n\t\tint shrinking;\t/* use the shrinking heuristics */\n\t\tint probability; /* do probability estimates */\n\t};\n\n svm_type can be one of C_SVC, NU_SVC, ONE_CLASS, EPSILON_SVR, NU_SVR.\n\n C_SVC:\t\tC-SVM classification\n NU_SVC:\t\tnu-SVM classification\n ONE_CLASS:\t\tone-class-SVM\n EPSILON_SVR:\tepsilon-SVM regression\n NU_SVR:\t\tnu-SVM regression\n\n kernel_type can be one of LINEAR, POLY, RBF, SIGMOID.\n\n LINEAR:\tu'*v\n POLY:\t(gamma*u'*v + coef0)^degree\n RBF:\texp(-gamma*|u-v|^2)\n SIGMOID:\ttanh(gamma*u'*v + coef0)\n PRECOMPUTED: kernel values in training_set_file\n\n cache_size is the size of the kernel cache, specified in megabytes.\n C is the cost of constraints violation. \n eps is the stopping criterion. (we usually use 0.00001 in nu-SVC,\n 0.001 in others). nu is the parameter in nu-SVM, nu-SVR, and\n one-class-SVM. p is the epsilon in epsilon-insensitive loss function\n of epsilon-SVM regression. shrinking = 1 means shrinking is conducted;\n = 0 otherwise. probability = 1 means model with probability\n information is obtained; = 0 otherwise.\n\n nr_weight, weight_label, and weight are used to change the penalty\n for some classes (If the weight for a class is not changed, it is\n set to 1). This is useful for training classifier using unbalanced\n input data or with asymmetric misclassification cost.\n\n nr_weight is the number of elements in the array weight_label and\n weight. Each weight[i] corresponds to weight_label[i], meaning that\n the penalty of class weight_label[i] is scaled by a factor of weight[i].\n \n If you do not want to change penalty for any of the classes,\n just set nr_weight to 0.\n\n *NOTE* Because svm_model contains pointers to svm_problem, you can\n not free the memory used by svm_problem if you are still using the\n svm_model produced by svm_train(). \n\n *NOTE* To avoid wrong parameters, svm_check_parameter() should be\n called before svm_train().\n\n struct svm_model stores the model obtained from the training procedure.\n It is not recommended to directly access entries in this structure.\n Programmers should use the interface functions to get the values.\n\n\tstruct svm_model\n\t{\n\t\tstruct svm_parameter param;\t/* parameter */\n\t\tint nr_class;\t\t/* number of classes, = 2 in regression/one class svm */\n\t\tint l;\t\t\t/* total #SV */\n\t\tstruct svm_node **SV;\t\t/* SVs (SV[l]) */\n\t\tdouble **sv_coef;\t/* coefficients for SVs in decision functions (sv_coef[k-1][l]) */\n\t\tdouble *rho;\t\t/* constants in decision functions (rho[k*(k-1)/2]) */\n\t\tdouble *probA;\t\t/* pairwise probability information */\n\t\tdouble *probB;\n\t\tint *sv_indices; /* sv_indices[0,...,nSV-1] are values in [1,...,num_traning_data] to indicate SVs in the training set */\n\n\t\t/* for classification only */\n\n\t\tint *label;\t\t/* label of each class (label[k]) */\n\t\tint *nSV;\t\t/* number of SVs for each class (nSV[k]) */\n\t\t\t\t\t/* nSV[0] + nSV[1] + ... + nSV[k-1] = l */\n\t\t/* XXX */\n\t\tint free_sv;\t\t/* 1 if svm_model is created by svm_load_model*/\n\t\t\t\t\t/* 0 if svm_model is created by svm_train */\n\t};\n\n param describes the parameters used to obtain the model.\n\n nr_class is the number of classes. It is 2 for regression and one-class SVM.\n\n l is the number of support vectors. SV and sv_coef are support\n vectors and the corresponding coefficients, respectively. Assume there are\n k classes. For data in class j, the corresponding sv_coef includes (k-1) y*alpha vectors,\n where alpha's are solutions of the following two class problems:\n 1 vs j, 2 vs j, ..., j-1 vs j, j vs j+1, j vs j+2, ..., j vs k\n and y=1 for the first j-1 vectors, while y=-1 for the remaining k-j \n vectors. For example, if there are 4 classes, sv_coef and SV are like:\n\n +-+-+-+--------------------+\n |1|1|1| |\n |v|v|v| SVs from class 1 |\n |2|3|4| |\n +-+-+-+--------------------+\n |1|2|2| |\n |v|v|v| SVs from class 2 |\n |2|3|4| |\n +-+-+-+--------------------+\n |1|2|3| |\n |v|v|v| SVs from class 3 |\n |3|3|4| |\n +-+-+-+--------------------+\n |1|2|3| |\n |v|v|v| SVs from class 4 |\n |4|4|4| |\n +-+-+-+--------------------+\n\n See svm_train() for an example of assigning values to sv_coef.\n\n rho is the bias term (-b). probA and probB are parameters used in\n probability outputs. If there are k classes, there are k*(k-1)/2\n binary problems as well as rho, probA, and probB values. They are\n aligned in the order of binary problems:\n 1 vs 2, 1 vs 3, ..., 1 vs k, 2 vs 3, ..., 2 vs k, ..., k-1 vs k.\n\n sv_indices[0,...,nSV-1] are values in [1,...,num_traning_data] to\n indicate support vectors in the training set.\n\n label contains labels in the training data.\n\n nSV is the number of support vectors in each class.\n\n free_sv is a flag used to determine whether the space of SV should \n be released in free_model_content(struct svm_model*) and \n free_and_destroy_model(struct svm_model**). If the model is\n generated by svm_train(), then SV points to data in svm_problem\n and should not be removed. For example, free_sv is 0 if svm_model\n is created by svm_train, but is 0 if created by svm_load_model.\n\n- Function: double svm_predict(const struct svm_model *model,\n const struct svm_node *x);\n\n This function does classification or regression on a test vector x\n given a model.\n\n For a classification model, the predicted class for x is returned.\n For a regression model, the function value of x calculated using\n the model is returned. For an one-class model, +1 or -1 is\n returned.\n\n- Function: void svm_cross_validation(const struct svm_problem *prob,\n\tconst struct svm_parameter *param, int nr_fold, double *target);\n\n This function conducts cross validation. Data are separated to\n nr_fold folds. Under given parameters, sequentially each fold is\n validated using the model from training the remaining. Predicted\n labels (of all prob's instances) in the validation process are\n stored in the array called target.\n\n The format of svm_prob is same as that for svm_train(). \n\n- Function: int svm_get_svm_type(const struct svm_model *model);\n\n This function gives svm_type of the model. Possible values of\n svm_type are defined in svm.h.\n\n- Function: int svm_get_nr_class(const svm_model *model);\n\n For a classification model, this function gives the number of\n classes. For a regression or an one-class model, 2 is returned.\n\n- Function: void svm_get_labels(const svm_model *model, int* label)\n \n For a classification model, this function outputs the name of\n labels into an array called label. For regression and one-class\n models, label is unchanged.\n\n- Function: void svm_get_sv_indices(const struct svm_model *model, int *sv_indices)\n\n This function outputs indices of support vectors into an array called sv_indices. \n The size of sv_indices is the number of support vectors and can be obtained by calling svm_get_nr_sv. \n Each sv_indices[i] is in the range of [1, ..., num_traning_data].\n\n- Function: int svm_get_nr_sv(const struct svm_model *model) \n\n This function gives the number of total support vector.\n\n- Function: double svm_get_svr_probability(const struct svm_model *model);\n\n For a regression model with probability information, this function\n outputs a value sigma > 0. For test data, we consider the\n probability model: target value = predicted value + z, z: Laplace\n distribution e^(-|z|/sigma)/(2sigma)\n\n If the model is not for svr or does not contain required\n information, 0 is returned.\n\n- Function: double svm_predict_values(const svm_model *model, \n\t\t\t\t const svm_node *x, double* dec_values)\n\n This function gives decision values on a test vector x given a\n model, and return the predicted label (classification) or\n the function value (regression).\n\n For a classification model with nr_class classes, this function\n gives nr_class*(nr_class-1)/2 decision values in the array\n dec_values, where nr_class can be obtained from the function\n svm_get_nr_class. The order is label[0] vs. label[1], ...,\n label[0] vs. label[nr_class-1], label[1] vs. label[2], ...,\n label[nr_class-2] vs. label[nr_class-1], where label can be\n obtained from the function svm_get_labels. The returned value is\n the predicted class for x. Note that when nr_class = 1, this \n function does not give any decision value.\n\n For a regression model, dec_values[0] and the returned value are\n both the function value of x calculated using the model. For a\n one-class model, dec_values[0] is the decision value of x, while\n the returned value is +1/-1.\n\n- Function: double svm_predict_probability(const struct svm_model *model, \n\t const struct svm_node *x, double* prob_estimates);\n \n This function does classification or regression on a test vector x\n given a model with probability information.\n\n For a classification model with probability information, this\n function gives nr_class probability estimates in the array\n prob_estimates. nr_class can be obtained from the function\n svm_get_nr_class. The class with the highest probability is\n returned. For regression/one-class SVM, the array prob_estimates\n is unchanged and the returned value is the same as that of\n svm_predict.\n\n- Function: const char *svm_check_parameter(const struct svm_problem *prob,\n const struct svm_parameter *param);\n\n This function checks whether the parameters are within the feasible\n range of the problem. This function should be called before calling\n svm_train() and svm_cross_validation(). It returns NULL if the\n parameters are feasible, otherwise an error message is returned.\n\n- Function: int svm_check_probability_model(const struct svm_model *model);\n\n This function checks whether the model contains required\n information to do probability estimates. If so, it returns\n +1. Otherwise, 0 is returned. This function should be called\n before calling svm_get_svr_probability and\n svm_predict_probability.\n\n- Function: int svm_save_model(const char *model_file_name,\n\t\t\t const struct svm_model *model);\n\n This function saves a model to a file; returns 0 on success, or -1\n if an error occurs.\n\n- Function: struct svm_model *svm_load_model(const char *model_file_name);\n\n This function returns a pointer to the model read from the file,\n or a null pointer if the model could not be loaded.\n\n- Function: void svm_free_model_content(struct svm_model *model_ptr);\n\n This function frees the memory used by the entries in a model structure.\n\n- Function: void svm_free_and_destroy_model(struct svm_model **model_ptr_ptr);\n\n This function frees the memory used by a model and destroys the model\n structure. It is equivalent to svm_destroy_model, which\n is deprecated after version 3.0.\n\n- Function: void svm_destroy_param(struct svm_parameter *param);\n\n This function frees the memory used by a parameter set.\n\n- Function: void svm_set_print_string_function(void (*print_func)(const char *));\n\n Users can specify their output format by a function. Use\n svm_set_print_string_function(NULL); \n for default printing to stdout.\n\nJava Version\n============\n\nThe pre-compiled java class archive `libsvm.jar' and its source files are\nin the java directory. To run the programs, use\n\njava -classpath libsvm.jar svm_train \njava -classpath libsvm.jar svm_predict \njava -classpath libsvm.jar svm_toy\njava -classpath libsvm.jar svm_scale \n\nNote that you need Java 1.5 (5.0) or above to run it.\n\nYou may need to add Java runtime library (like classes.zip) to the classpath.\nYou may need to increase maximum Java heap size.\n\nLibrary usages are similar to the C version. These functions are available:\n\npublic class svm {\n\tpublic static final int LIBSVM_VERSION=317; \n\tpublic static svm_model svm_train(svm_problem prob, svm_parameter param);\n\tpublic static void svm_cross_validation(svm_problem prob, svm_parameter param, int nr_fold, double[] target);\n\tpublic static int svm_get_svm_type(svm_model model);\n\tpublic static int svm_get_nr_class(svm_model model);\n\tpublic static void svm_get_labels(svm_model model, int[] label);\n\tpublic static void svm_get_sv_indices(svm_model model, int[] indices);\n\tpublic static int svm_get_nr_sv(svm_model model);\n\tpublic static double svm_get_svr_probability(svm_model model);\n\tpublic static double svm_predict_values(svm_model model, svm_node[] x, double[] dec_values);\n\tpublic static double svm_predict(svm_model model, svm_node[] x);\n\tpublic static double svm_predict_probability(svm_model model, svm_node[] x, double[] prob_estimates);\n\tpublic static void svm_save_model(String model_file_name, svm_model model) throws IOException\n\tpublic static svm_model svm_load_model(String model_file_name) throws IOException\n\tpublic static String svm_check_parameter(svm_problem prob, svm_parameter param);\n\tpublic static int svm_check_probability_model(svm_model model);\n\tpublic static void svm_set_print_string_function(svm_print_interface print_func);\n}\n\nThe library is in the \"libsvm\" package.\nNote that in Java version, svm_node[] is not ended with a node whose index = -1.\n\nUsers can specify their output format by\n\n\tyour_print_func = new svm_print_interface()\n\t{ \n\t\tpublic void print(String s)\n\t\t{\n\t\t\t// your own format\n\t\t}\n\t};\n\tsvm.svm_set_print_string_function(your_print_func);\n\nBuilding Windows Binaries\n=========================\n\nWindows binaries are in the directory `windows'. To build them via\nVisual C++, use the following steps:\n\n1. Open a DOS command box (or Visual Studio Command Prompt) and change\nto libsvm directory. If environment variables of VC++ have not been\nset, type\n\n\"C:\\Program Files\\Microsoft Visual Studio 10.0\\VC\\bin\\vcvars32.bat\"\n\nYou may have to modify the above command according which version of\nVC++ or where it is installed.\n\n2. Type\n\nnmake -f Makefile.win clean all\n\n3. (optional) To build shared library libsvm.dll, type\n\nnmake -f Makefile.win lib\n\nAnother way is to build them from Visual C++ environment. See details\nin libsvm FAQ.\n\n- Additional Tools: Sub-sampling, Parameter Selection, Format checking, etc.\n============================================================================\n\nSee the README file in the tools directory.\n\nMATLAB/OCTAVE Interface\n=======================\n\nPlease check the file README in the directory `matlab'.\n\nPython Interface\n================\n\nSee the README file in python directory.\n\nAdditional Information\n======================\n\nIf you find LIBSVM helpful, please cite it as\n\nChih-Chung Chang and Chih-Jen Lin, LIBSVM : a library for support\nvector machines. ACM Transactions on Intelligent Systems and\nTechnology, 2:27:1--27:27, 2011. Software available at\nhttp://www.csie.ntu.edu.tw/~cjlin/libsvm\n\nLIBSVM implementation document is available at\nhttp://www.csie.ntu.edu.tw/~cjlin/papers/libsvm.pdf\n\nFor any questions and comments, please email cjlin@csie.ntu.edu.tw\n\nAcknowledgments:\nThis work was supported in part by the National Science \nCouncil of Taiwan via the grant NSC 89-2213-E-002-013.\nThe authors thank their group members and users\nfor many helpful discussions and comments. They are listed in\nhttp://www.csie.ntu.edu.tw/~cjlin/libsvm/acknowledgements\n\n"
},
{
"path": "binaries/windows/x86/README-GPU",
"content": "GPU-Accelerated LIBSVM is exploiting the GPU, using the CUDA interface, to\nspeed-up the training process. This package contains a new executable for \ntraining classifiers \"svm-train-gpu.exe\" together with the original one.\nThe use of the new executable is exactly the same as with the original one.\n\nThis executable was built with the CUBLAS API version 2 which is compatible with SDKs from 4.0 and up.\n\nTo test the executable \"svm-train-gpu\" you can run the easy.py script which is located in the \"tools\" folder.\nTo observe speed improvements between CPU and GPU execution we provide a custom relatively large dataset (train_set) which can be used as an input to easy.py.\n\nFEATURES\n\nMode Supported\n\n * c-svc classification with RBF kernel\n\nFunctionality / User interface\n\n * Same as LIBSVM\n\n\nPREREQUISITES\n\n * NVIDIA Graphics card with CUDA support\n * Latest NVIDIA drivers for GPU\n\nAdditional Information\n======================\n\nIf you find GPU-Accelerated LIBSVM helpful, please cite it as\n\nA. Athanasopoulos, A. Dimou, V. Mezaris, I. Kompatsiaris, \"GPU Acceleration for Support Vector Machines\",\nProc. 12th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS 2011), Delft, The Netherlands, April 2011.\n\nSoftware available at http://mklab.iti.gr/project/GPU-LIBSVM\n"
},
{
"path": "binaries/windows/x86/tools/README",
"content": "This directory includes some useful codes:\n\n1. subset selection tools.\n2. parameter selection tools.\n3. LIBSVM format checking tools\n\nPart I: Subset selection tools\n\nIntroduction\n============\n\nTraining large data is time consuming. Sometimes one should work on a\nsmaller subset first. The python script subset.py randomly selects a\nspecified number of samples. For classification data, we provide a\nstratified selection to ensure the same class distribution in the\nsubset.\n\nUsage: subset.py [options] dataset number [output1] [output2]\n\nThis script selects a subset of the given data set.\n\noptions:\n-s method : method of selection (default 0)\n 0 -- stratified selection (classification only)\n 1 -- random selection\n\noutput1 : the subset (optional)\noutput2 : the rest of data (optional)\n\nIf output1 is omitted, the subset will be printed on the screen.\n\nExample\n=======\n\n> python subset.py heart_scale 100 file1 file2\n\nFrom heart_scale 100 samples are randomly selected and stored in\nfile1. All remaining instances are stored in file2.\n\n\nPart II: Parameter Selection Tools\n\nIntroduction\n============\n\ngrid.py is a parameter selection tool for C-SVM classification using\nthe RBF (radial basis function) kernel. It uses cross validation (CV)\ntechnique to estimate the accuracy of each parameter combination in\nthe specified range and helps you to decide the best parameters for\nyour problem.\n\ngrid.py directly executes libsvm binaries (so no python binding is needed)\nfor cross validation and then draw contour of CV accuracy using gnuplot.\nYou must have libsvm and gnuplot installed before using it. The package\ngnuplot is available at http://www.gnuplot.info/\n\nOn Mac OSX, the precompiled gnuplot file needs the library Aquarterm,\nwhich thus must be installed as well. In addition, this version of\ngnuplot does not support png, so you need to change \"set term png\ntransparent small\" and use other image formats. For example, you may\nhave \"set term pbm small color\".\n\nUsage: grid.py [grid_options] [svm_options] dataset\n\ngrid_options :\n-log2c {begin,end,step | \"null\"} : set the range of c (default -5,15,2)\n begin,end,step -- c_range = 2^{begin,...,begin+k*step,...,end}\n \"null\" -- do not grid with c\n-log2g {begin,end,step | \"null\"} : set the range of g (default 3,-15,-2)\n begin,end,step -- g_range = 2^{begin,...,begin+k*step,...,end}\n \"null\" -- do not grid with g\n-v n : n-fold cross validation (default 5)\n-svmtrain pathname : set svm executable path and name\n-gnuplot {pathname | \"null\"} :\n pathname -- set gnuplot executable path and name\n \"null\" -- do not plot \n-out {pathname | \"null\"} : (default dataset.out)\n pathname -- set output file path and name\n \"null\" -- do not output file\n-png pathname : set graphic output file path and name (default dataset.png)\n-resume [pathname] : resume the grid task using an existing output file (default pathname is dataset.out)\n Use this option only if some parameters have been checked for the SAME data.\n\nsvm_options : additional options for svm-train\n\nThe program conducts v-fold cross validation using parameter C (and gamma)\n= 2^begin, 2^(begin+step), ..., 2^end.\n\nYou can specify where the libsvm executable and gnuplot are using the\n-svmtrain and -gnuplot parameters.\n\nFor windows users, please use pgnuplot.exe. If you are using gnuplot\n3.7.1, please upgrade to version 3.7.3 or higher. The version 3.7.1\nhas a bug. If you use cygwin on windows, please use gunplot-x11.\n\nIf the task is terminated accidentally or you would like to change the\nrange of parameters, you can apply '-resume' to save time by re-using\nprevious results. You may specify the output file of a previous run\nor use the default (i.e., dataset.out) without giving a name. Please\nnote that the same condition must be used in two runs. For example,\nyou cannot use '-v 10' earlier and resume the task with '-v 5'.\n\nThe value of some options can be \"null.\" For example, `-log2c -1,0,1\n-log2 \"null\"' means that C=2^-1,2^0,2^1 and g=LIBSVM's default gamma\nvalue. That is, you do not conduct parameter selection on gamma.\n\nExample\n=======\n\n> python grid.py -log2c -5,5,1 -log2g -4,0,1 -v 5 -m 300 heart_scale\n\nUsers (in particular MS Windows users) may need to specify the path of\nexecutable files. You can either change paths in the beginning of\ngrid.py or specify them in the command line. For example,\n\n> grid.py -log2c -5,5,1 -svmtrain \"c:\\Program Files\\libsvm\\windows\\svm-train.exe\" -gnuplot c:\\tmp\\gnuplot\\binary\\pgnuplot.exe -v 10 heart_scale\n\nOutput: two files\ndataset.png: the CV accuracy contour plot generated by gnuplot\ndataset.out: the CV accuracy at each (log2(C),log2(gamma))\n\nThe following example saves running time by loading the output file of a previous run.\n\n> python grid.py -log2c -7,7,1 -log2g -5,2,1 -v 5 -resume heart_scale.out heart_scale\n\nParallel grid search\n====================\n\nYou can conduct a parallel grid search by dispatching jobs to a\ncluster of computers which share the same file system. First, you add\nmachine names in grid.py:\n\nssh_workers = [\"linux1\", \"linux5\", \"linux5\"]\n\nand then setup your ssh so that the authentication works without\nasking a password.\n\nThe same machine (e.g., linux5 here) can be listed more than once if\nit has multiple CPUs or has more RAM. If the local machine is the\nbest, you can also enlarge the nr_local_worker. For example:\n\nnr_local_worker = 2\n\nExample:\n\n> python grid.py heart_scale\n[local] -1 -1 78.8889 (best c=0.5, g=0.5, rate=78.8889)\n[linux5] -1 -7 83.3333 (best c=0.5, g=0.0078125, rate=83.3333)\n[linux5] 5 -1 77.037 (best c=0.5, g=0.0078125, rate=83.3333)\n[linux1] 5 -7 83.3333 (best c=0.5, g=0.0078125, rate=83.3333)\n.\n.\n.\n\nIf -log2c, -log2g, or -v is not specified, default values are used.\n\nIf your system uses telnet instead of ssh, you list the computer names\nin telnet_workers.\n\nCalling grid in Python\n======================\n\nIn addition to using grid.py as a command-line tool, you can use it as a\nPython module. \n\n>>> rate, param = find_parameters(dataset, options)\n\nYou need to specify `dataset' and `options' (default ''). See the following example.\n\n> python\n\n>>> from grid import *\n>>> rate, param = find_parameters('../heart_scale', '-log2c -1,1,1 -log2g -1,1,1')\n[local] 0.0 0.0 rate=74.8148 (best c=1.0, g=1.0, rate=74.8148)\n[local] 0.0 -1.0 rate=77.037 (best c=1.0, g=0.5, rate=77.037)\n.\n.\n[local] -1.0 -1.0 rate=78.8889 (best c=0.5, g=0.5, rate=78.8889)\n.\n.\n>>> rate\n78.8889\n>>> param\n{'c': 0.5, 'g': 0.5}\n\n\nPart III: LIBSVM format checking tools\n\nIntroduction\n============\n\n`svm-train' conducts only a simple check of the input data. To do a\ndetailed check, we provide a python script `checkdata.py.'\n\nUsage: checkdata.py dataset\n\nExit status (returned value): 1 if there are errors, 0 otherwise.\n\nThis tool is written by Rong-En Fan at National Taiwan University.\n\nExample\n=======\n\n> cat bad_data\n1 3:1 2:4\n> python checkdata.py bad_data\nline 1: feature indices must be in an ascending order, previous/current features 3:1 2:4\nFound 1 lines with error.\n\n\n"
},
{
"path": "binaries/windows/x86/tools/checkdata.py",
"content": "#!/usr/bin/env python\n\n#\n# A format checker for LIBSVM\n#\n\n#\n# Copyright (c) 2007, Rong-En Fan\n#\n# All rights reserved.\n#\n# This program is distributed under the same license of the LIBSVM package.\n# \n\nfrom sys import argv, exit\nimport os.path\n\ndef err(line_no, msg):\n\tprint(\"line {0}: {1}\".format(line_no, msg))\n\n# works like float() but does not accept nan and inf\ndef my_float(x):\n\tif x.lower().find(\"nan\") != -1 or x.lower().find(\"inf\") != -1:\n\t\traise ValueError\n\n\treturn float(x)\n\ndef main():\n\tif len(argv) != 2:\n\t\tprint(\"Usage: {0} dataset\".format(argv[0]))\n\t\texit(1)\n\n\tdataset = argv[1]\n\n\tif not os.path.exists(dataset):\n\t\tprint(\"dataset {0} not found\".format(dataset))\n\t\texit(1)\n\n\tline_no = 1\n\terror_line_count = 0\n\tfor line in open(dataset, 'r'):\n\t\tline_error = False\n\n\t\t# each line must end with a newline character\n\t\tif line[-1] != '\\n':\n\t\t\terr(line_no, \"missing a newline character in the end\")\n\t\t\tline_error = True\n\n\t\tnodes = line.split()\n\n\t\t# check label\n\t\ttry:\n\t\t\tlabel = nodes.pop(0)\n\t\t\t\n\t\t\tif label.find(',') != -1:\n\t\t\t\t# multi-label format\n\t\t\t\ttry:\n\t\t\t\t\tfor l in label.split(','):\n\t\t\t\t\t\tl = my_float(l)\n\t\t\t\texcept:\n\t\t\t\t\terr(line_no, \"label {0} is not a valid multi-label form\".format(label))\n\t\t\t\t\tline_error = True\n\t\t\telse:\n\t\t\t\ttry:\n\t\t\t\t\tlabel = my_float(label)\n\t\t\t\texcept:\n\t\t\t\t\terr(line_no, \"label {0} is not a number\".format(label))\n\t\t\t\t\tline_error = True\n\t\texcept:\n\t\t\terr(line_no, \"missing label, perhaps an empty line?\")\n\t\t\tline_error = True\n\n\t\t# check features\n\t\tprev_index = -1\n\t\tfor i in range(len(nodes)):\n\t\t\ttry:\n\t\t\t\t(index, value) = nodes[i].split(':')\n\n\t\t\t\tindex = int(index)\n\t\t\t\tvalue = my_float(value)\n\n\t\t\t\t# precomputed kernel's index starts from 0 and LIBSVM\n\t\t\t\t# checks it. Hence, don't treat index 0 as an error.\n\t\t\t\tif index < 0:\n\t\t\t\t\terr(line_no, \"feature index must be positive; wrong feature {0}\".format(nodes[i]))\n\t\t\t\t\tline_error = True\n\t\t\t\telif index <= prev_index:\n\t\t\t\t\terr(line_no, \"feature indices must be in an ascending order, previous/current features {0} {1}\".format(nodes[i-1], nodes[i]))\n\t\t\t\t\tline_error = True\n\t\t\t\tprev_index = index\n\t\t\texcept:\n\t\t\t\terr(line_no, \"feature '{0}' not an : pair, integer, real number \".format(nodes[i]))\n\t\t\t\tline_error = True\n\n\t\tline_no += 1\n\n\t\tif line_error:\n\t\t\terror_line_count += 1\n\t\n\tif error_line_count > 0:\n\t\tprint(\"Found {0} lines with error.\".format(error_line_count))\n\t\treturn 1\n\telse:\n\t\tprint(\"No error.\")\n\t\treturn 0\n\nif __name__ == \"__main__\":\n\texit(main())\n"
},
{
"path": "binaries/windows/x86/tools/easy.py",
"content": "#!/usr/bin/env python\n\nimport sys\nimport os\nfrom subprocess import *\n\nif len(sys.argv) <= 1:\n\tprint('Usage: {0} training_file [testing_file]'.format(sys.argv[0]))\n\traise SystemExit\n\n# svm, grid, and gnuplot executable files\n\nis_win32 = (sys.platform == 'win32')\nif not is_win32:\n\tsvmscale_exe = \"../svm-scale\"\n\tsvmtrain_exe = \"../svm-train-gpu\"\n\tsvmpredict_exe = \"../svm-predict\"\n\tgrid_py = \"./grid.py\"\n\tgnuplot_exe = \"/usr/bin/gnuplot\"\nelse:\n # example for windows\n\tsvmscale_exe = r\"..\\windows\\svm-scale.exe\"\n\tsvmtrain_exe = r\"..\\windows\\svm-train-gpu.exe\"\n\tsvmpredict_exe = r\"..\\windows\\svm-predict.exe\"\n\tgnuplot_exe = r\"C:\\Program Files (x86)\\gnuplot\\bin\\pgnuplot.exe\"\n\tgrid_py = r\".\\grid.py\"\n\nassert os.path.exists(svmscale_exe),\"svm-scale executable not found\"\nassert os.path.exists(svmtrain_exe),\"svm-train-gpu executable not found\"\nassert os.path.exists(svmpredict_exe),\"svm-predict executable not found\"\nassert os.path.exists(gnuplot_exe),\"gnuplot executable not found\"\nassert os.path.exists(grid_py),\"grid.py not found\"\n\ntrain_pathname = sys.argv[1]\nassert os.path.exists(train_pathname),\"training file not found\"\nfile_name = os.path.split(train_pathname)[1]\nscaled_file = file_name + \".scale\"\nmodel_file = file_name + \".model\"\nrange_file = file_name + \".range\"\n\nif len(sys.argv) > 2:\n\ttest_pathname = sys.argv[2]\n\tfile_name = os.path.split(test_pathname)[1]\n\tassert os.path.exists(test_pathname),\"testing file not found\"\n\tscaled_test_file = file_name + \".scale\"\n\tpredict_test_file = file_name + \".predict\"\n\ncmd = '{0} -s \"{1}\" \"{2}\" > \"{3}\"'.format(svmscale_exe, range_file, train_pathname, scaled_file)\nprint('Scaling training data...')\nPopen(cmd, shell = True, stdout = PIPE).communicate()\t\n\ncmd = '{0} -svmtrain \"{1}\" -gnuplot \"{2}\" \"{3}\"'.format(grid_py, svmtrain_exe, gnuplot_exe, scaled_file)\nprint('Cross validation...')\nf = Popen(cmd, shell = True, stdout = PIPE).stdout\n\nline = ''\nwhile True:\n\tlast_line = line\n\tline = f.readline()\n\tif not line: break\nc,g,rate = map(float,last_line.split())\n\nprint('Best c={0}, g={1} CV rate={2}'.format(c,g,rate))\n\ncmd = '{0} -c {1} -g {2} \"{3}\" \"{4}\"'.format(svmtrain_exe,c,g,scaled_file,model_file)\nprint('Training...')\nPopen(cmd, shell = True, stdout = PIPE).communicate()\n\nprint('Output model: {0}'.format(model_file))\nif len(sys.argv) > 2:\n\tcmd = '{0} -r \"{1}\" \"{2}\" > \"{3}\"'.format(svmscale_exe, range_file, test_pathname, scaled_test_file)\n\tprint('Scaling testing data...')\n\tPopen(cmd, shell = True, stdout = PIPE).communicate()\t\n\n\tcmd = '{0} \"{1}\" \"{2}\" \"{3}\"'.format(svmpredict_exe, scaled_test_file, model_file, predict_test_file)\n\tprint('Testing...')\n\tPopen(cmd, shell = True).communicate()\t\n\n\tprint('Output prediction: {0}'.format(predict_test_file))\n"
},
{
"path": "binaries/windows/x86/tools/grid.py",
"content": "#!/usr/bin/env python\n__all__ = ['find_parameters']\n\nimport os, sys, traceback, getpass, time, re\nfrom threading import Thread\nfrom subprocess import *\n\nif sys.version_info[0] < 3:\n\tfrom Queue import Queue\nelse:\n\tfrom queue import Queue\n\ntelnet_workers = []\nssh_workers = []\nnr_local_worker = 1\n\nclass GridOption:\n\tdef __init__(self, dataset_pathname, options):\n\t\tdirname = os.path.dirname(__file__)\n\t\tif sys.platform != 'win32':\n\t\t\tself.svmtrain_pathname = os.path.join(dirname, '../svm-train')\n\t\t\tself.gnuplot_pathname = '/usr/bin/gnuplot'\n\t\telse:\n\t\t\t# example for windows\n\t\t\tself.svmtrain_pathname = os.path.join(dirname, r'..\\windows\\svm-train.exe')\n\t\t\t# svmtrain_pathname = r'c:\\Program Files\\libsvm\\windows\\svm-train.exe'\n\t\t\tself.gnuplot_pathname = r'C:\\Program Files (x86)\\gnuplot\\bin\\pgnuplot.exe'\n\t\tself.fold = 5\n\t\tself.c_begin, self.c_end, self.c_step = -5, 15, 2\n\t\tself.g_begin, self.g_end, self.g_step = 3, -15, -2\n\t\tself.grid_with_c, self.grid_with_g = True, True\n\t\tself.dataset_pathname = dataset_pathname\n\t\tself.dataset_title = os.path.split(dataset_pathname)[1]\n\t\tself.out_pathname = '{0}.out'.format(self.dataset_title)\n\t\tself.png_pathname = '{0}.png'.format(self.dataset_title)\n\t\tself.pass_through_string = ' '\n\t\tself.resume_pathname = None\n\t\tself.parse_options(options)\n\n\tdef parse_options(self, options):\n\t\tif type(options) == str:\n\t\t\toptions = options.split()\n\t\ti = 0\n\t\tpass_through_options = []\n\t\t\n\t\twhile i < len(options):\n\t\t\tif options[i] == '-log2c':\n\t\t\t\ti = i + 1\n\t\t\t\tif options[i] == 'null':\n\t\t\t\t\tself.grid_with_c = False\n\t\t\t\telse:\n\t\t\t\t\tself.c_begin, self.c_end, self.c_step = map(float,options[i].split(','))\n\t\t\telif options[i] == '-log2g':\n\t\t\t\ti = i + 1\n\t\t\t\tif options[i] == 'null':\n\t\t\t\t\tself.grid_with_g = False\n\t\t\t\telse:\n\t\t\t\t\tself.g_begin, self.g_end, self.g_step = map(float,options[i].split(','))\n\t\t\telif options[i] == '-v':\n\t\t\t\ti = i + 1\n\t\t\t\tself.fold = options[i]\n\t\t\telif options[i] in ('-c','-g'):\n\t\t\t\traise ValueError('Use -log2c and -log2g.')\n\t\t\telif options[i] == '-svmtrain':\n\t\t\t\ti = i + 1\n\t\t\t\tself.svmtrain_pathname = options[i]\n\t\t\telif options[i] == '-gnuplot':\n\t\t\t\ti = i + 1\n\t\t\t\tif options[i] == 'null':\n\t\t\t\t\tself.gnuplot_pathname = None\n\t\t\t\telse:\t\n\t\t\t\t\tself.gnuplot_pathname = options[i]\n\t\t\telif options[i] == '-out':\n\t\t\t\ti = i + 1\n\t\t\t\tif options[i] == 'null':\n\t\t\t\t\tself.out_pathname = None\n\t\t\t\telse:\n\t\t\t\t\tself.out_pathname = options[i]\n\t\t\telif options[i] == '-png':\n\t\t\t\ti = i + 1\n\t\t\t\tself.png_pathname = options[i]\n\t\t\telif options[i] == '-resume':\n\t\t\t\tif i == (len(options)-1) or options[i+1].startswith('-'):\n\t\t\t\t\tself.resume_pathname = self.dataset_title + '.out'\n\t\t\t\telse:\n\t\t\t\t\ti = i + 1\n\t\t\t\t\tself.resume_pathname = options[i]\n\t\t\telse:\n\t\t\t\tpass_through_options.append(options[i])\n\t\t\ti = i + 1\n\n\t\tself.pass_through_string = ' '.join(pass_through_options)\n\t\tif not os.path.exists(self.svmtrain_pathname):\n\t\t\traise IOError('svm-train executable not found')\n\t\tif not os.path.exists(self.dataset_pathname):\n\t\t\traise IOError('dataset not found')\n\t\tif self.resume_pathname and not os.path.exists(self.resume_pathname):\n\t\t\traise IOError('file for resumption not found')\n\t\tif not self.grid_with_c and not self.grid_with_g:\n\t\t\traise ValueError('-log2c and -log2g should not be null simultaneously')\n\t\tif self.gnuplot_pathname and not os.path.exists(self.gnuplot_pathname):\n\t\t\tsys.stderr.write('gnuplot executable not found\\n')\n\t\t\tself.gnuplot_pathname = None\n\ndef redraw(db,best_param,gnuplot,options,tofile=False):\n\tif len(db) == 0: return\n\tbegin_level = round(max(x[2] for x in db)) - 3\n\tstep_size = 0.5\n\n\tbest_log2c,best_log2g,best_rate = best_param\n\n\t# if newly obtained c, g, or cv values are the same,\n\t# then stop redrawing the contour.\n\tif all(x[0] == db[0][0] for x in db): return\n\tif all(x[1] == db[0][1] for x in db): return\n\tif all(x[2] == db[0][2] for x in db): return\n\n\tif tofile:\n\t\tgnuplot.write(b\"set term png transparent small linewidth 2 medium enhanced\\n\")\n\t\tgnuplot.write(\"set output \\\"{0}\\\"\\n\".format(options.png_pathname.replace('\\\\','\\\\\\\\')).encode())\n\t\t#gnuplot.write(b\"set term postscript color solid\\n\")\n\t\t#gnuplot.write(\"set output \\\"{0}.ps\\\"\\n\".format(options.dataset_title).encode().encode())\n\telif sys.platform == 'win32':\n\t\tgnuplot.write(b\"set term windows\\n\")\n\telse:\n\t\tgnuplot.write( b\"set term x11\\n\")\n\tgnuplot.write(b\"set xlabel \\\"log2(C)\\\"\\n\")\n\tgnuplot.write(b\"set ylabel \\\"log2(gamma)\\\"\\n\")\n\tgnuplot.write(\"set xrange [{0}:{1}]\\n\".format(options.c_begin,options.c_end).encode())\n\tgnuplot.write(\"set yrange [{0}:{1}]\\n\".format(options.g_begin,options.g_end).encode())\n\tgnuplot.write(b\"set contour\\n\")\n\tgnuplot.write(\"set cntrparam levels incremental {0},{1},100\\n\".format(begin_level,step_size).encode())\n\tgnuplot.write(b\"unset surface\\n\")\n\tgnuplot.write(b\"unset ztics\\n\")\n\tgnuplot.write(b\"set view 0,0\\n\")\n\tgnuplot.write(\"set title \\\"{0}\\\"\\n\".format(options.dataset_title).encode())\n\tgnuplot.write(b\"unset label\\n\")\n\tgnuplot.write(\"set label \\\"Best log2(C) = {0} log2(gamma) = {1} accuracy = {2}%\\\" \\\n\t\t\t\t at screen 0.5,0.85 center\\n\". \\\n\t\t\t\t format(best_log2c, best_log2g, best_rate).encode())\n\tgnuplot.write(\"set label \\\"C = {0} gamma = {1}\\\"\"\n\t\t\t\t \" at screen 0.5,0.8 center\\n\".format(2**best_log2c, 2**best_log2g).encode())\n\tgnuplot.write(b\"set key at screen 0.9,0.9\\n\")\n\tgnuplot.write(b\"splot \\\"-\\\" with lines\\n\")\n\t\n\tdb.sort(key = lambda x:(x[0], -x[1]))\n\n\tprevc = db[0][0]\n\tfor line in db:\n\t\tif prevc != line[0]:\n\t\t\tgnuplot.write(b\"\\n\")\n\t\t\tprevc = line[0]\n\t\tgnuplot.write(\"{0[0]} {0[1]} {0[2]}\\n\".format(line).encode())\n\tgnuplot.write(b\"e\\n\")\n\tgnuplot.write(b\"\\n\") # force gnuplot back to prompt when term set failure\n\tgnuplot.flush()\n\n\ndef calculate_jobs(options):\n\t\n\tdef range_f(begin,end,step):\n\t\t# like range, but works on non-integer too\n\t\tseq = []\n\t\twhile True:\n\t\t\tif step > 0 and begin > end: break\n\t\t\tif step < 0 and begin < end: break\n\t\t\tseq.append(begin)\n\t\t\tbegin = begin + step\n\t\treturn seq\n\t\n\tdef permute_sequence(seq):\n\t\tn = len(seq)\n\t\tif n <= 1: return seq\n\t\n\t\tmid = int(n/2)\n\t\tleft = permute_sequence(seq[:mid])\n\t\tright = permute_sequence(seq[mid+1:])\n\t\n\t\tret = [seq[mid]]\n\t\twhile left or right:\n\t\t\tif left: ret.append(left.pop(0))\n\t\t\tif right: ret.append(right.pop(0))\n\t\t\t\n\t\treturn ret\t\n\n\t\n\tc_seq = permute_sequence(range_f(options.c_begin,options.c_end,options.c_step))\n\tg_seq = permute_sequence(range_f(options.g_begin,options.g_end,options.g_step))\n\n\tif not options.grid_with_c:\n\t\tc_seq = [None]\n\tif not options.grid_with_g:\n\t\tg_seq = [None] \n\t\n\tnr_c = float(len(c_seq))\n\tnr_g = float(len(g_seq))\n\ti, j = 0, 0\n\tjobs = []\n\n\twhile i < nr_c or j < nr_g:\n\t\tif i/nr_c < j/nr_g:\n\t\t\t# increase C resolution\n\t\t\tline = []\n\t\t\tfor k in range(0,j):\n\t\t\t\tline.append((c_seq[i],g_seq[k]))\n\t\t\ti = i + 1\n\t\t\tjobs.append(line)\n\t\telse:\n\t\t\t# increase g resolution\n\t\t\tline = []\n\t\t\tfor k in range(0,i):\n\t\t\t\tline.append((c_seq[k],g_seq[j]))\n\t\t\tj = j + 1\n\t\t\tjobs.append(line)\n\n\tresumed_jobs = {}\n\t\n\tif options.resume_pathname is None:\n\t\treturn jobs, resumed_jobs\n\n\tfor line in open(options.resume_pathname, 'r'):\n\t\tline = line.strip()\n\t\trst = re.findall(r'rate=([0-9.]+)',line)\n\t\tif not rst: \n\t\t\tcontinue\n\t\trate = float(rst[0])\n\n\t\tc, g = None, None \n\t\trst = re.findall(r'log2c=([0-9.-]+)',line)\n\t\tif rst: \n\t\t\tc = float(rst[0])\n\t\trst = re.findall(r'log2g=([0-9.-]+)',line)\n\t\tif rst: \n\t\t\tg = float(rst[0])\n\n\t\tresumed_jobs[(c,g)] = rate\n\n\treturn jobs, resumed_jobs\n\n\t\nclass WorkerStopToken: # used to notify the worker to stop or if a worker is dead\n\tpass\n\nclass Worker(Thread):\n\tdef __init__(self,name,job_queue,result_queue,options):\n\t\tThread.__init__(self)\n\t\tself.name = name\n\t\tself.job_queue = job_queue\n\t\tself.result_queue = result_queue\n\t\tself.options = options\n\t\t\n\tdef run(self):\n\t\twhile True:\n\t\t\t(cexp,gexp) = self.job_queue.get()\n\t\t\tif cexp is WorkerStopToken:\n\t\t\t\tself.job_queue.put((cexp,gexp))\n\t\t\t\t# print('worker {0} stop.'.format(self.name))\n\t\t\t\tbreak\n\t\t\ttry:\n\t\t\t\tc, g = None, None\n\t\t\t\tif cexp != None:\n\t\t\t\t\tc = 2.0**cexp\n\t\t\t\tif gexp != None:\n\t\t\t\t\tg = 2.0**gexp\n\t\t\t\trate = self.run_one(c,g)\n\t\t\t\tif rate is None: raise RuntimeError('get no rate')\n\t\t\texcept:\n\t\t\t\t# we failed, let others do that and we just quit\n\t\t\t\n\t\t\t\ttraceback.print_exception(sys.exc_info()[0], sys.exc_info()[1], sys.exc_info()[2])\n\t\t\t\t\n\t\t\t\tself.job_queue.put((cexp,gexp))\n\t\t\t\tsys.stderr.write('worker {0} quit.\\n'.format(self.name))\n\t\t\t\tbreak\n\t\t\telse:\n\t\t\t\tself.result_queue.put((self.name,cexp,gexp,rate))\n\n\tdef get_cmd(self,c,g):\n\t\toptions=self.options\n\t\tcmdline = options.svmtrain_pathname\n\t\tif options.grid_with_c: \n\t\t\tcmdline += ' -c {0} '.format(c)\n\t\tif options.grid_with_g: \n\t\t\tcmdline += ' -g {0} '.format(g)\n\t\tcmdline += ' -v {0} {1} {2} '.format\\\n\t\t\t(options.fold,options.pass_through_string,options.dataset_pathname)\n\t\treturn cmdline\n\t\t\nclass LocalWorker(Worker):\n\tdef run_one(self,c,g):\n\t\tcmdline = self.get_cmd(c,g)\n\t\tresult = Popen(cmdline,shell=True,stdout=PIPE,stderr=PIPE,stdin=PIPE).stdout\n\t\tfor line in result.readlines():\n\t\t\tif str(line).find('Cross') != -1:\n\t\t\t\treturn float(line.split()[-1][0:-1])\n\nclass SSHWorker(Worker):\n\tdef __init__(self,name,job_queue,result_queue,host,options):\n\t\tWorker.__init__(self,name,job_queue,result_queue,options)\n\t\tself.host = host\n\t\tself.cwd = os.getcwd()\n\tdef run_one(self,c,g):\n\t\tcmdline = 'ssh -x -t -t {0} \"cd {1}; {2}\"'.format\\\n\t\t\t(self.host,self.cwd,self.get_cmd(c,g))\n\t\tresult = Popen(cmdline,shell=True,stdout=PIPE,stderr=PIPE,stdin=PIPE).stdout\n\t\tfor line in result.readlines():\n\t\t\tif str(line).find('Cross') != -1:\n\t\t\t\treturn float(line.split()[-1][0:-1])\n\nclass TelnetWorker(Worker):\n\tdef __init__(self,name,job_queue,result_queue,host,username,password,options):\n\t\tWorker.__init__(self,name,job_queue,result_queue,options)\n\t\tself.host = host\n\t\tself.username = username\n\t\tself.password = password\t\t\n\tdef run(self):\n\t\timport telnetlib\n\t\tself.tn = tn = telnetlib.Telnet(self.host)\n\t\ttn.read_until('login: ')\n\t\ttn.write(self.username + '\\n')\n\t\ttn.read_until('Password: ')\n\t\ttn.write(self.password + '\\n')\n\n\t\t# XXX: how to know whether login is successful?\n\t\ttn.read_until(self.username)\n\t\t# \n\t\tprint('login ok', self.host)\n\t\ttn.write('cd '+os.getcwd()+'\\n')\n\t\tWorker.run(self)\n\t\ttn.write('exit\\n')\t\t\t \n\tdef run_one(self,c,g):\n\t\tcmdline = self.get_cmd(c,g)\n\t\tresult = self.tn.write(cmdline+'\\n')\n\t\t(idx,matchm,output) = self.tn.expect(['Cross.*\\n'])\n\t\tfor line in output.split('\\n'):\n\t\t\tif str(line).find('Cross') != -1:\n\t\t\t\treturn float(line.split()[-1][0:-1])\n\t\t\t\ndef find_parameters(dataset_pathname, options=''):\n\t\n\tdef update_param(c,g,rate,best_c,best_g,best_rate,worker,resumed):\n\t\tif (rate > best_rate) or (rate==best_rate and g==best_g and c= 3:\n\txrange = range\n\ndef exit_with_help(argv):\n\tprint(\"\"\"\\\nUsage: {0} [options] dataset subset_size [output1] [output2]\n\nThis script randomly selects a subset of the dataset.\n\noptions:\n-s method : method of selection (default 0)\n 0 -- stratified selection (classification only)\n 1 -- random selection\n\noutput1 : the subset (optional)\noutput2 : rest of the data (optional)\nIf output1 is omitted, the subset will be printed on the screen.\"\"\".format(argv[0]))\n\texit(1)\n\ndef process_options(argv):\n\targc = len(argv)\n\tif argc < 3:\n\t\texit_with_help(argv)\n\n\t# default method is stratified selection\n\tmethod = 0 \n\tsubset_file = sys.stdout\n\trest_file = None\n\n\ti = 1\n\twhile i < argc:\n\t\tif argv[i][0] != \"-\":\n\t\t\tbreak\n\t\tif argv[i] == \"-s\":\n\t\t\ti = i + 1\n\t\t\tmethod = int(argv[i])\n\t\t\tif method not in [0,1]:\n\t\t\t\tprint(\"Unknown selection method {0}\".format(method))\n\t\t\t\texit_with_help(argv)\n\t\ti = i + 1\n\n\tdataset = argv[i]\n\tsubset_size = int(argv[i+1])\n\tif i+2 < argc:\n\t\tsubset_file = open(argv[i+2],'w')\n\tif i+3 < argc:\n\t\trest_file = open(argv[i+3],'w')\n\n\treturn dataset, subset_size, method, subset_file, rest_file\n\ndef random_selection(dataset, subset_size):\n\tl = sum(1 for line in open(dataset,'r'))\n\treturn sorted(random.sample(xrange(l), subset_size))\n\ndef stratified_selection(dataset, subset_size):\n\tlabels = [line.split(None,1)[0] for line in open(dataset)]\n\tlabel_linenums = defaultdict(list)\n\tfor i, label in enumerate(labels):\n\t\tlabel_linenums[label] += [i]\n\n\tl = len(labels)\n\tremaining = subset_size\n\tret = []\n\n\t# classes with fewer data are sampled first; otherwise\n\t# some rare classes may not be selected\n\tfor label in sorted(label_linenums, key=lambda x: len(label_linenums[x])):\n\t\tlinenums = label_linenums[label]\n\t\tlabel_size = len(linenums) \n\t\t# at least one instance per class\n\t\ts = int(min(remaining, max(1, math.ceil(label_size*(float(subset_size)/l)))))\n\t\tif s == 0:\n\t\t\tsys.stderr.write('''\\\nError: failed to have at least one instance per class\n 1. You may have regression data.\n 2. Your classification data is unbalanced or too small.\nPlease use -s 1.\n''')\n\t\t\tsys.exit(-1)\n\t\tremaining -= s\n\t\tret += [linenums[i] for i in random.sample(xrange(label_size), s)]\n\treturn sorted(ret)\n\ndef main(argv=sys.argv):\n\tdataset, subset_size, method, subset_file, rest_file = process_options(argv)\n\t#uncomment the following line to fix the random seed \n\t#random.seed(0)\n\tselected_lines = []\n\n\tif method == 0:\n\t\tselected_lines = stratified_selection(dataset, subset_size)\n\telif method == 1:\n\t\tselected_lines = random_selection(dataset, subset_size)\n\n\t#select instances based on selected_lines\n\tdataset = open(dataset,'r')\n\tprev_selected_linenum = -1\n\tfor i in xrange(len(selected_lines)):\n\t\tfor cnt in xrange(selected_lines[i]-prev_selected_linenum-1):\n\t\t\tline = dataset.readline()\n\t\t\tif rest_file: \n\t\t\t\trest_file.write(line)\n\t\tsubset_file.write(dataset.readline())\n\t\tprev_selected_linenum = selected_lines[i]\n\tsubset_file.close()\n\n\tif rest_file:\n\t\tfor line in dataset: \n\t\t\trest_file.write(line)\n\t\trest_file.close()\n\tdataset.close()\n\nif __name__ == '__main__':\n\tmain(sys.argv)\n\n"
},
{
"path": "src/linux/COPYRIGHT",
"content": "\nCopyright (c) 2000-2010 Chih-Chung Chang and Chih-Jen Lin\nAll rights reserved.\n\nRedistribution and use in source and binary forms, with or without\nmodification, are permitted provided that the following conditions\nare met:\n\n1. Redistributions of source code must retain the above copyright\nnotice, this list of conditions and the following disclaimer.\n\n2. Redistributions in binary form must reproduce the above copyright\nnotice, this list of conditions and the following disclaimer in the\ndocumentation and/or other materials provided with the distribution.\n\n3. Neither name of copyright holders nor the names of its contributors\nmay be used to endorse or promote products derived from this software\nwithout specific prior written permission.\n\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\nLIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\nA PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR\nCONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,\nEXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,\nPROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR\nPROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF\nLIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING\nNEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS\nSOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n"
},
{
"path": "src/linux/Makefile",
"content": "################################################################################\n#\n# Copyright 1993-2013 NVIDIA Corporation. All rights reserved.\n#\n# NOTICE TO USER: \n#\n# This source code is subject to NVIDIA ownership rights under U.S. and \n# international Copyright laws. \n#\n# NVIDIA MAKES NO REPRESENTATION ABOUT THE SUITABILITY OF THIS SOURCE \n# CODE FOR ANY PURPOSE. IT IS PROVIDED \"AS IS\" WITHOUT EXPRESS OR \n# IMPLIED WARRANTY OF ANY KIND. NVIDIA DISCLAIMS ALL WARRANTIES WITH \n# REGARD TO THIS SOURCE CODE, INCLUDING ALL IMPLIED WARRANTIES OF \n# MERCHANTABILITY, NONINFRINGEMENT, AND FITNESS FOR A PARTICULAR PURPOSE. \n# IN NO EVENT SHALL NVIDIA BE LIABLE FOR ANY SPECIAL, INDIRECT, INCIDENTAL, \n# OR CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS \n# OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE \n# OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE \n# OR PERFORMANCE OF THIS SOURCE CODE. \n#\n# U.S. Government End Users. This source code is a \"commercial item\" as \n# that term is defined at 48 C.F.R. 2.101 (OCT 1995), consisting of \n# \"commercial computer software\" and \"commercial computer software \n# documentation\" as such terms are used in 48 C.F.R. 12.212 (SEPT 1995) \n# and is provided to the U.S. Government only as a commercial end item. \n# Consistent with 48 C.F.R.12.212 and 48 C.F.R. 227.7202-1 through \n# 227.7202-4 (JUNE 1995), all U.S. Government End Users acquire the \n# source code with only those rights set forth herein.\n#\n################################################################################\n#\n# Makefile project only supported on Mac OS X and Linux Platforms)\n#\n################################################################################\n\ninclude ./findcudalib.mk\n\n# Location of the CUDA Toolkit\nCUDA_PATH ?= \"/usr/local/cuda-5.5\"\n\n# internal flags\nNVCCFLAGS := -m${OS_SIZE} -maxrregcount=16\nCCFLAGS :=\nNVCCLDFLAGS :=\nLDFLAGS :=\n\n# Extra user flags\nEXTRA_NVCCFLAGS ?=\nEXTRA_NVCCLDFLAGS ?=\nEXTRA_LDFLAGS ?=\nEXTRA_CCFLAGS ?=\n\n# OS-specific build flags\nifneq ($(DARWIN),) \n LDFLAGS += -rpath $(CUDA_PATH)/lib\n CCFLAGS += -arch $(OS_ARCH) $(STDLIB) \nelse\n ifeq ($(OS_ARCH),armv7l)\n ifeq ($(abi),gnueabi)\n CCFLAGS += -mfloat-abi=softfp\n else\n # default to gnueabihf\n override abi := gnueabihf\n LDFLAGS += --dynamic-linker=/lib/ld-linux-armhf.so.3\n CCFLAGS += -mfloat-abi=hard\n endif\n endif\nendif\n\nifeq ($(ARMv7),1)\nNVCCFLAGS += -target-cpu-arch ARM\nifneq ($(TARGET_FS),) \nCCFLAGS += --sysroot=$(TARGET_FS)\nLDFLAGS += --sysroot=$(TARGET_FS)\nLDFLAGS += -rpath-link=$(TARGET_FS)/lib\nLDFLAGS += -rpath-link=$(TARGET_FS)/usr/lib\nLDFLAGS += -rpath-link=$(TARGET_FS)/usr/lib/arm-linux-$(abi)\nendif\nendif\n\n# Debug build flags\nifeq ($(dbg),1)\n NVCCFLAGS += -g -G\n TARGET := debug\nelse\n TARGET := release\nendif\n\nALL_CCFLAGS :=\nALL_CCFLAGS += $(NVCCFLAGS)\nALL_CCFLAGS += $(addprefix -Xcompiler ,$(CCFLAGS))\nALL_CCFLAGS += $(EXTRA_NVCCFLAGS)\nALL_CCFLAGS += $(addprefix -Xcompiler ,$(EXTRA_CCFLAGS))\n\nALL_LDFLAGS :=\nALL_LDFLAGS += $(ALL_CCFLAGS)\nALL_LDFLAGS += $(NVCCLDFLAGS)\nALL_LDFLAGS += $(addprefix -Xlinker ,$(LDFLAGS))\nALL_LDFLAGS += $(EXTRA_NVCCLDFLAGS)\nALL_LDFLAGS += $(addprefix -Xlinker ,$(EXTRA_LDFLAGS))\n\n# Common includes and paths for CUDA\nINCLUDES := -I/usr/local/cuda-5.5/include\nINCLUDES := -I/usr/local/cuda-5.5/samples/common/inc\nLIBRARIES := -L/usr/local/cuda-5.5/lib64\n\n################################################################################\n\nLIBRARIES += -lcublas -lcudart\n\n################################################################################\n\n# CUDA code generation flags\nifneq ($(OS_ARCH),armv7l)\nGENCODE_SM10 := -gencode arch=compute_10,code=sm_10\nendif\nGENCODE_SM20 := -gencode arch=compute_20,code=sm_20\nGENCODE_SM30 := -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=\\\"sm_35,compute_35\\\"\nGENCODE_FLAGS := $(GENCODE_SM10) $(GENCODE_SM20) $(GENCODE_SM30)\n\n################################################################################\n\n\n# Target rules\nall: build\n\nbuild: svm-train-gpu\n\nsvm-train-gpu.o: svm.cpp svm-train.o\n\t$(NVCC) $(INCLUDES) $(ALL_CCFLAGS) $(GENCODE_FLAGS) -o $@ -c $<\n\nsvm-train-gpu: svm.o svm-train.o\n\t$(NVCC) $(ALL_LDFLAGS) -o $@ $+ $(LIBRARIES)\n\tmkdir -p /bin/$(OS_ARCH)/$(OSLOWER)/$(TARGET)$(if $(abi),/$(abi))\n\tcp $@ /bin/$(OS_ARCH)/$(OSLOWER)/$(TARGET)$(if $(abi),/$(abi))\n\nrun: build\n\t./svm-train-gpu\n\nclean:\n\trm -f svm-train-gpu.o svm-train-gpu\n\trm -rf /bin/$(OS_ARCH)/$(OSLOWER)/$(TARGET)$(if $(abi),/$(abi))/svm-train-gpu\n\nclobber: clean\n"
},
{
"path": "src/linux/README",
"content": "Libsvm is a simple, easy-to-use, and efficient software for SVM\nclassification and regression. It solves C-SVM classification, nu-SVM\nclassification, one-class-SVM, epsilon-SVM regression, and nu-SVM\nregression. It also provides an automatic model selection tool for\nC-SVM classification. This document explains the use of libsvm.\n\nLibsvm is available at \nhttp://www.csie.ntu.edu.tw/~cjlin/libsvm\nPlease read the COPYRIGHT file before using libsvm.\n\nTable of Contents\n=================\n\n- Quick Start\n- Installation and Data Format\n- `svm-train' Usage\n- `svm-predict' Usage\n- `svm-scale' Usage\n- Tips on Practical Use\n- Examples\n- Precomputed Kernels \n- Library Usage\n- Java Version\n- Building Windows Binaries\n- Additional Tools: Sub-sampling, Parameter Selection, Format checking, etc.\n- Python Interface\n- Additional Information\n\nQuick Start\n===========\n\nIf you are new to SVM and if the data is not large, please go to \n`tools' directory and use easy.py after installation. It does \neverything automatic -- from data scaling to parameter selection.\n\nUsage: easy.py training_file [testing_file]\n\nMore information about parameter selection can be found in\n`tools/README.'\n\nInstallation and Data Format\n============================\n\nOn Unix systems, type `make' to build the `svm-train' and `svm-predict'\nprograms. Run them without arguments to show the usages of them.\n\nOn other systems, consult `Makefile' to build them (e.g., see\n'Building Windows binaries' in this file) or use the pre-built\nbinaries (Windows binaries are in the directory `windows').\n\nThe format of training and testing data file is:\n\n : : ...\n.\n.\n.\n\nEach line contains an instance and is ended by a '\\n' character. For\nclassification, is an integer indicating the class label\n(multi-class is supported). For regression, is the target\nvalue which can be any real number. For one-class SVM, it's not used\nso can be any number. Except using precomputed kernels (explained in\nanother section), : gives a feature (attribute) value.\n is an integer starting from 1 and is a real\nnumber. Indices must be in ASCENDING order. Labels in the testing\nfile are only used to calculate accuracy or errors. If they are\nunknown, just fill the first column with any numbers.\n\nA sample classification data included in this package is\n`heart_scale'. To check if your data is in a correct form, use\n`tools/checkdata.py' (details in `tools/README').\n\nType `svm-train heart_scale', and the program will read the training\ndata and output the model file `heart_scale.model'. If you have a test\nset called heart_scale.t, then type `svm-predict heart_scale.t\nheart_scale.model output' to see the prediction accuracy. The `output'\nfile contains the predicted class labels.\n\nThere are some other useful programs in this package.\n\nsvm-scale:\n\n\tThis is a tool for scaling input data file.\n\nsvm-toy:\n\n\tThis is a simple graphical interface which shows how SVM\n\tseparate data in a plane. You can click in the window to \n\tdraw data points. Use \"change\" button to choose class \n\t1, 2 or 3 (i.e., up to three classes are supported), \"load\"\n\tbutton to load data from a file, \"save\" button to save data to\n\ta file, \"run\" button to obtain an SVM model, and \"clear\"\n\tbutton to clear the window.\n\n\tYou can enter options in the bottom of the window, the syntax of\n\toptions is the same as `svm-train'.\n\n\tNote that \"load\" and \"save\" consider data in the\n\tclassification but not the regression case. Each data point\n\thas one label (the color) which must be 1, 2, or 3 and two\n\tattributes (x-axis and y-axis values) in [0,1].\n\n\tType `make' in respective directories to build them.\n\n\tYou need Qt library to build the Qt version.\n\t(available from http://www.trolltech.com)\n\n\tYou need GTK+ library to build the GTK version.\n\t(available from http://www.gtk.org)\n\t\n\tThe pre-built Windows binaries are in the `windows'\n\tdirectory. We use Visual C++ on a 32-bit machine, so the\n\tmaximal cache size is 2GB.\n\n`svm-train' Usage\n=================\n\nUsage: svm-train [options] training_set_file [model_file]\noptions:\n-s svm_type : set type of SVM (default 0)\n\t0 -- C-SVC\n\t1 -- nu-SVC\n\t2 -- one-class SVM\n\t3 -- epsilon-SVR\n\t4 -- nu-SVR\n-t kernel_type : set type of kernel function (default 2)\n\t0 -- linear: u'*v\n\t1 -- polynomial: (gamma*u'*v + coef0)^degree\n\t2 -- radial basis function: exp(-gamma*|u-v|^2)\n\t3 -- sigmoid: tanh(gamma*u'*v + coef0)\n\t4 -- precomputed kernel (kernel values in training_set_file)\n-d degree : set degree in kernel function (default 3)\n-g gamma : set gamma in kernel function (default 1/num_features)\n-r coef0 : set coef0 in kernel function (default 0)\n-c cost : set the parameter C of C-SVC, epsilon-SVR, and nu-SVR (default 1)\n-n nu : set the parameter nu of nu-SVC, one-class SVM, and nu-SVR (default 0.5)\n-p epsilon : set the epsilon in loss function of epsilon-SVR (default 0.1)\n-m cachesize : set cache memory size in MB (default 100)\n-e epsilon : set tolerance of termination criterion (default 0.001)\n-h shrinking : whether to use the shrinking heuristics, 0 or 1 (default 1)\n-b probability_estimates : whether to train a SVC or SVR model for probability estimates, 0 or 1 (default 0)\n-wi weight : set the parameter C of class i to weight*C, for C-SVC (default 1)\n-v n: n-fold cross validation mode\n-q : quiet mode (no outputs)\n\n\nThe k in the -g option means the number of attributes in the input data.\n\noption -v randomly splits the data into n parts and calculates cross\nvalidation accuracy/mean squared error on them.\n\nSee libsvm FAQ for the meaning of outputs.\n\n`svm-predict' Usage\n===================\n\nUsage: svm-predict [options] test_file model_file output_file\noptions:\n-b probability_estimates: whether to predict probability estimates, 0 or 1 (default 0); for one-class SVM only 0 is supported\n\nmodel_file is the model file generated by svm-train.\ntest_file is the test data you want to predict.\nsvm-predict will produce output in the output_file.\n\n`svm-scale' Usage\n=================\n\nUsage: svm-scale [options] data_filename\noptions:\n-l lower : x scaling lower limit (default -1)\n-u upper : x scaling upper limit (default +1)\n-y y_lower y_upper : y scaling limits (default: no y scaling)\n-s save_filename : save scaling parameters to save_filename\n-r restore_filename : restore scaling parameters from restore_filename\n\nSee 'Examples' in this file for examples.\n\nTips on Practical Use\n=====================\n\n* Scale your data. For example, scale each attribute to [0,1] or [-1,+1].\n* For C-SVC, consider using the model selection tool in the tools directory.\n* nu in nu-SVC/one-class-SVM/nu-SVR approximates the fraction of training\n errors and support vectors.\n* If data for classification are unbalanced (e.g. many positive and\n few negative), try different penalty parameters C by -wi (see\n examples below).\n* Specify larger cache size (i.e., larger -m) for huge problems.\n\nExamples\n========\n\n> svm-scale -l -1 -u 1 -s range train > train.scale\n> svm-scale -r range test > test.scale\n\nScale each feature of the training data to be in [-1,1]. Scaling\nfactors are stored in the file range and then used for scaling the\ntest data.\n\n> svm-train -s 0 -c 5 -t 2 -g 0.5 -e 0.1 data_file \n\nTrain a classifier with RBF kernel exp(-0.5|u-v|^2), C=10, and\nstopping tolerance 0.1.\n\n> svm-train -s 3 -p 0.1 -t 0 data_file\n\nSolve SVM regression with linear kernel u'v and epsilon=0.1\nin the loss function.\n\n> svm-train -c 10 -w1 1 -w2 5 -w4 2 data_file\n\nTrain a classifier with penalty 10 = 1 * 10 for class 1, penalty 50 =\n5 * 10 for class 2, and penalty 20 = 2 * 10 for class 4.\n\n> svm-train -s 0 -c 100 -g 0.1 -v 5 data_file\n\nDo five-fold cross validation for the classifier using\nthe parameters C = 100 and gamma = 0.1\n\n> svm-train -s 0 -b 1 data_file\n> svm-predict -b 1 test_file data_file.model output_file\n\nObtain a model with probability information and predict test data with\nprobability estimates\n\nPrecomputed Kernels \n===================\n\nUsers may precompute kernel values and input them as training and\ntesting files. Then libsvm does not need the original\ntraining/testing sets.\n\nAssume there are L training instances x1, ..., xL and. \nLet K(x, y) be the kernel\nvalue of two instances x and y. The input formats\nare:\n\nNew training instance for xi:\n\n 0:i 1:K(xi,x1) ... L:K(xi,xL) \n\nNew testing instance for any x:\n\n 0:? 1:K(x,x1) ... L:K(x,xL) \n\nThat is, in the training file the first column must be the \"ID\" of\nxi. In testing, ? can be any value.\n\nAll kernel values including ZEROs must be explicitly provided. Any\npermutation or random subsets of the training/testing files are also\nvalid (see examples below).\n\nNote: the format is slightly different from the precomputed kernel\npackage released in libsvmtools earlier.\n\nExamples:\n\n\tAssume the original training data has three four-feature\n\tinstances and testing data has one instance:\n\n\t15 1:1 2:1 3:1 4:1\n\t45 2:3 4:3\n\t25 3:1\n\n\t15 1:1 3:1\n\n\tIf the linear kernel is used, we have the following new\n\ttraining/testing sets:\n\n\t15 0:1 1:4 2:6 3:1\n\t45 0:2 1:6 2:18 3:0 \n\t25 0:3 1:1 2:0 3:1\n \n\t15 0:? 1:2 2:0 3:1\n\n\t? can be any value.\n\n\tAny subset of the above training file is also valid. For example,\n\n\t25 0:3 1:1 2:0 3:1\n\t45 0:2 1:6 2:18 3:0 \n\n\timplies that the kernel matrix is\n\n\t\t[K(2,2) K(2,3)] = [18 0]\n\t\t[K(3,2) K(3,3)] = [0 1]\n\nLibrary Usage\n=============\n\nThese functions and structures are declared in the header file\n`svm.h'. You need to #include \"svm.h\" in your C/C++ source files and\nlink your program with `svm.cpp'. You can see `svm-train.c' and\n`svm-predict.c' for examples showing how to use them. We define\nLIBSVM_VERSION and declare `extern int libsvm_version; ' in svm.h, so\nyou can check the version number.\n\nBefore you classify test data, you need to construct an SVM model\n(`svm_model') using training data. A model can also be saved in\na file for later use. Once an SVM model is available, you can use it\nto classify new data.\n\n- Function: struct svm_model *svm_train(const struct svm_problem *prob,\n\t\t\t\t\tconst struct svm_parameter *param);\n\n This function constructs and returns an SVM model according to\n the given training data and parameters.\n\n struct svm_problem describes the problem:\n\t\n\tstruct svm_problem\n\t{\n\t\tint l;\n\t\tdouble *y;\n\t\tstruct svm_node **x;\n\t};\n \n where `l' is the number of training data, and `y' is an array containing\n their target values. (integers in classification, real numbers in\n regression) `x' is an array of pointers, each of which points to a sparse\n representation (array of svm_node) of one training vector. \n\n For example, if we have the following training data:\n\n LABEL\tATTR1\tATTR2\tATTR3\tATTR4\tATTR5\n -----\t-----\t-----\t-----\t-----\t-----\n 1\t\t 0\t 0.1\t 0.2\t 0\t 0\n 2\t\t 0\t 0.1\t 0.3\t -1.2\t 0\n 1\t\t 0.4\t 0\t 0\t 0\t 0\n 2\t\t 0\t 0.1\t 0\t 1.4\t 0.5\n 3\t\t -0.1\t -0.2\t 0.1\t 1.1\t 0.1\n\n then the components of svm_problem are:\n\n l = 5\n\n y -> 1 2 1 2 3\n\n x -> [ ] -> (2,0.1) (3,0.2) (-1,?)\n\t [ ] -> (2,0.1) (3,0.3) (4,-1.2) (-1,?)\n\t [ ] -> (1,0.4) (-1,?)\n\t [ ] -> (2,0.1) (4,1.4) (5,0.5) (-1,?)\n\t [ ] -> (1,-0.1) (2,-0.2) (3,0.1) (4,1.1) (5,0.1) (-1,?)\n\n where (index,value) is stored in the structure `svm_node':\n\n\tstruct svm_node\n\t{\n\t\tint index;\n\t\tdouble value;\n\t};\n\n index = -1 indicates the end of one vector. Note that indices must\n be in ASCENDING order.\n \n struct svm_parameter describes the parameters of an SVM model:\n\n\tstruct svm_parameter\n\t{\n\t\tint svm_type;\n\t\tint kernel_type;\n\t\tint degree;\t/* for poly */\n\t\tdouble gamma;\t/* for poly/rbf/sigmoid */\n\t\tdouble coef0;\t/* for poly/sigmoid */\n\n\t\t/* these are for training only */\n\t\tdouble cache_size; /* in MB */\n\t\tdouble eps;\t/* stopping criteria */\n\t\tdouble C;\t/* for C_SVC, EPSILON_SVR, and NU_SVR */\n\t\tint nr_weight;\t\t/* for C_SVC */\n\t\tint *weight_label;\t/* for C_SVC */\n\t\tdouble* weight;\t\t/* for C_SVC */\n\t\tdouble nu;\t/* for NU_SVC, ONE_CLASS, and NU_SVR */\n\t\tdouble p;\t/* for EPSILON_SVR */\n\t\tint shrinking;\t/* use the shrinking heuristics */\n\t\tint probability; /* do probability estimates */\n\t};\n\n svm_type can be one of C_SVC, NU_SVC, ONE_CLASS, EPSILON_SVR, NU_SVR.\n\n C_SVC:\t\tC-SVM classification\n NU_SVC:\t\tnu-SVM classification\n ONE_CLASS:\t\tone-class-SVM\n EPSILON_SVR:\tepsilon-SVM regression\n NU_SVR:\t\tnu-SVM regression\n\n kernel_type can be one of LINEAR, POLY, RBF, SIGMOID.\n\n LINEAR:\tu'*v\n POLY:\t(gamma*u'*v + coef0)^degree\n RBF:\texp(-gamma*|u-v|^2)\n SIGMOID:\ttanh(gamma*u'*v + coef0)\n PRECOMPUTED: kernel values in training_set_file\n\n cache_size is the size of the kernel cache, specified in megabytes.\n C is the cost of constraints violation. \n eps is the stopping criterion. (we usually use 0.00001 in nu-SVC,\n 0.001 in others). nu is the parameter in nu-SVM, nu-SVR, and\n one-class-SVM. p is the epsilon in epsilon-insensitive loss function\n of epsilon-SVM regression. shrinking = 1 means shrinking is conducted;\n = 0 otherwise. probability = 1 means model with probability\n information is obtained; = 0 otherwise.\n\n nr_weight, weight_label, and weight are used to change the penalty\n for some classes (If the weight for a class is not changed, it is\n set to 1). This is useful for training classifier using unbalanced\n input data or with asymmetric misclassification cost.\n\n nr_weight is the number of elements in the array weight_label and\n weight. Each weight[i] corresponds to weight_label[i], meaning that\n the penalty of class weight_label[i] is scaled by a factor of weight[i].\n \n If you do not want to change penalty for any of the classes,\n just set nr_weight to 0.\n\n *NOTE* Because svm_model contains pointers to svm_problem, you can\n not free the memory used by svm_problem if you are still using the\n svm_model produced by svm_train(). \n\n *NOTE* To avoid wrong parameters, svm_check_parameter() should be\n called before svm_train().\n\n struct svm_model stores the model obtained from the training procedure.\n It is not recommended to directly access entries in this structure.\n Programmers should use the interface functions to get the values.\n\n\tstruct svm_model\n\t{\n\t\tstruct svm_parameter param;\t/* parameter */\n\t\tint nr_class;\t\t/* number of classes, = 2 in regression/one class svm */\n\t\tint l;\t\t\t/* total #SV */\n\t\tstruct svm_node **SV;\t\t/* SVs (SV[l]) */\n\t\tdouble **sv_coef;\t/* coefficients for SVs in decision functions (sv_coef[k-1][l]) */\n\t\tdouble *rho;\t\t/* constants in decision functions (rho[k*(k-1)/2]) */\n\t\tdouble *probA;\t\t/* pairwise probability information */\n\t\tdouble *probB;\n\n\t\t/* for classification only */\n\n\t\tint *label;\t\t/* label of each class (label[k]) */\n\t\tint *nSV;\t\t/* number of SVs for each class (nSV[k]) */\n\t\t\t\t\t/* nSV[0] + nSV[1] + ... + nSV[k-1] = l */\n\t\t/* XXX */\n\t\tint free_sv;\t\t/* 1 if svm_model is created by svm_load_model*/\n\t\t\t\t\t/* 0 if svm_model is created by svm_train */\n\t};\n\n param describes the parameters used to obtain the model.\n\n nr_class is the number of classes. It is 2 for regression and one-class SVM.\n\n l is the number of support vectors. SV and sv_coef are support\n vectors and the corresponding coefficients, respectively. Assume there are\n k classes. For data in class j, the corresponding sv_coef includes (k-1) y*alpha vectors,\n where alpha's are solutions of the following two class problems:\n 1 vs j, 2 vs j, ..., j-1 vs j, j vs j+1, j vs j+2, ..., j vs k\n and y=1 for the first j-1 vectors, while y=-1 for the remaining k-j \n vectors. For example, if there are 4 classes, sv_coef and SV are like:\n\n +-+-+-+--------------------+\n |1|1|1| |\n |v|v|v| SVs from class 1 |\n |2|3|4| |\n +-+-+-+--------------------+\n |1|2|2| |\n |v|v|v| SVs from class 2 |\n |2|3|4| |\n +-+-+-+--------------------+\n |1|2|3| |\n |v|v|v| SVs from class 3 |\n |3|3|4| |\n +-+-+-+--------------------+\n |1|2|3| |\n |v|v|v| SVs from class 4 |\n |4|4|4| |\n +-+-+-+--------------------+\n\n See svm_train() for an example of assigning values to sv_coef.\n\n rho is the bias term (-b). probA and probB are parameters used in\n probability outputs. If there are k classes, there are k*(k-1)/2\n binary problems as well as rho, probA, and probB values. They are\n aligned in the order of binary problems:\n 1 vs 2, 1 vs 3, ..., 1 vs k, 2 vs 3, ..., 2 vs k, ..., k-1 vs k.\n\n label contains labels in the training data.\n\n nSV is the number of support vectors in each class.\n\n free_sv is a flag used to determine whether the space of SV should \n be released in free_model_content(struct svm_model*) and \n free_and_destroy_model(struct svm_model**). If the model is\n generated by svm_train(), then SV points to data in svm_problem\n and should not be removed. For example, free_sv is 0 if svm_model\n is created by svm_train, but is 0 if created by svm_load_model.\n\n- Function: double svm_predict(const struct svm_model *model,\n const struct svm_node *x);\n\n This function does classification or regression on a test vector x\n given a model.\n\n For a classification model, the predicted class for x is returned.\n For a regression model, the function value of x calculated using\n the model is returned. For an one-class model, +1 or -1 is\n returned.\n\n- Function: void svm_cross_validation(const struct svm_problem *prob,\n\tconst struct svm_parameter *param, int nr_fold, double *target);\n\n This function conducts cross validation. Data are separated to\n nr_fold folds. Under given parameters, sequentially each fold is\n validated using the model from training the remaining. Predicted\n labels (of all prob's instances) in the validation process are\n stored in the array called target.\n\n The format of svm_prob is same as that for svm_train(). \n\n- Function: int svm_get_svm_type(const struct svm_model *model);\n\n This function gives svm_type of the model. Possible values of\n svm_type are defined in svm.h.\n\n- Function: int svm_get_nr_class(const svm_model *model);\n\n For a classification model, this function gives the number of\n classes. For a regression or an one-class model, 2 is returned.\n\n- Function: void svm_get_labels(const svm_model *model, int* label)\n \n For a classification model, this function outputs the name of\n labels into an array called label. For regression and one-class\n models, label is unchanged.\n\n- Function: double svm_get_svr_probability(const struct svm_model *model);\n\n For a regression model with probability information, this function\n outputs a value sigma > 0. For test data, we consider the\n probability model: target value = predicted value + z, z: Laplace\n distribution e^(-|z|/sigma)/(2sigma)\n\n If the model is not for svr or does not contain required\n information, 0 is returned.\n\n- Function: double svm_predict_values(const svm_model *model, \n\t\t\t\t const svm_node *x, double* dec_values)\n\n This function gives decision values on a test vector x given a\n model, and return the predicted label (classification) or\n the function value (regression).\n\n For a classification model with nr_class classes, this function\n gives nr_class*(nr_class-1)/2 decision values in the array\n dec_values, where nr_class can be obtained from the function\n svm_get_nr_class. The order is label[0] vs. label[1], ...,\n label[0] vs. label[nr_class-1], label[1] vs. label[2], ...,\n label[nr_class-2] vs. label[nr_class-1], where label can be\n obtained from the function svm_get_labels. The returned value is\n the predicted class for x.\n\n For a regression model, dec_values[0] and the returned value are\n both the function value of x calculated using the model. For a\n one-class model, dec_values[0] is the decision value of x, while\n the returned value is +1/-1.\n\n- Function: double svm_predict_probability(const struct svm_model *model, \n\t const struct svm_node *x, double* prob_estimates);\n \n This function does classification or regression on a test vector x\n given a model with probability information.\n\n For a classification model with probability information, this\n function gives nr_class probability estimates in the array\n prob_estimates. nr_class can be obtained from the function\n svm_get_nr_class. The class with the highest probability is\n returned. For regression/one-class SVM, the array prob_estimates\n is unchanged and the returned value is the same as that of\n svm_predict.\n\n- Function: const char *svm_check_parameter(const struct svm_problem *prob,\n const struct svm_parameter *param);\n\n This function checks whether the parameters are within the feasible\n range of the problem. This function should be called before calling\n svm_train() and svm_cross_validation(). It returns NULL if the\n parameters are feasible, otherwise an error message is returned.\n\n- Function: int svm_check_probability_model(const struct svm_model *model);\n\n This function checks whether the model contains required\n information to do probability estimates. If so, it returns\n +1. Otherwise, 0 is returned. This function should be called\n before calling svm_get_svr_probability and\n svm_predict_probability.\n\n- Function: int svm_save_model(const char *model_file_name,\n\t\t\t const struct svm_model *model);\n\n This function saves a model to a file; returns 0 on success, or -1\n if an error occurs.\n\n- Function: struct svm_model *svm_load_model(const char *model_file_name);\n\n This function returns a pointer to the model read from the file,\n or a null pointer if the model could not be loaded.\n\n- Function: void svm_free_model_content(struct svm_model *model_ptr);\n\n This function frees the memory used by the entries in a model structure.\n\n- Function: void svm_free_and_destroy_model(struct svm_model **model_ptr_ptr);\n\n This function frees the memory used by a model and destroys the model\n structure. It is equivalent to svm_destroy_model, which\n is deprecated after version 3.0.\n\n- Function: void svm_destroy_param(struct svm_parameter *param);\n\n This function frees the memory used by a parameter set.\n\n- Function: void svm_set_print_string_function(void (*print_func)(const char *));\n\n Users can specify their output format by a function. Use\n svm_set_print_string_function(NULL); \n for default printing to stdout.\n\nJava Version\n============\n\nThe pre-compiled java class archive `libsvm.jar' and its source files are\nin the java directory. To run the programs, use\n\njava -classpath libsvm.jar svm_train \njava -classpath libsvm.jar svm_predict \njava -classpath libsvm.jar svm_toy\njava -classpath libsvm.jar svm_scale \n\nNote that you need Java 1.5 (5.0) or above to run it.\n\nYou may need to add Java runtime library (like classes.zip) to the classpath.\nYou may need to increase maximum Java heap size.\n\nLibrary usages are similar to the C version. These functions are available:\n\npublic class svm {\n\tpublic static final int LIBSVM_VERSION=300; \n\tpublic static svm_model svm_train(svm_problem prob, svm_parameter param);\n\tpublic static void svm_cross_validation(svm_problem prob, svm_parameter param, int nr_fold, double[] target);\n\tpublic static int svm_get_svm_type(svm_model model);\n\tpublic static int svm_get_nr_class(svm_model model);\n\tpublic static void svm_get_labels(svm_model model, int[] label);\n\tpublic static double svm_get_svr_probability(svm_model model);\n\tpublic static double svm_predict_values(svm_model model, svm_node[] x, double[] dec_values);\n\tpublic static double svm_predict(svm_model model, svm_node[] x);\n\tpublic static double svm_predict_probability(svm_model model, svm_node[] x, double[] prob_estimates);\n\tpublic static void svm_save_model(String model_file_name, svm_model model) throws IOException\n\tpublic static svm_model svm_load_model(String model_file_name) throws IOException\n\tpublic static String svm_check_parameter(svm_problem prob, svm_parameter param);\n\tpublic static int svm_check_probability_model(svm_model model);\n\tpublic static void svm_set_print_string_function(svm_print_interface print_func);\n}\n\nThe library is in the \"libsvm\" package.\nNote that in Java version, svm_node[] is not ended with a node whose index = -1.\n\nUsers can specify their output format by\n\n\tyour_print_func = new svm_print_interface()\n\t{ \n\t\tpublic void print(String s)\n\t\t{\n\t\t\t// your own format\n\t\t}\n\t};\n\tsvm.svm_set_print_string_function(your_print_func);\n\nBuilding Windows Binaries\n=========================\n\nWindows binaries are in the directory `windows'. To build them via\nVisual C++, use the following steps:\n\n1. Open a DOS command box (or Visual Studio Command Prompt) and change\nto libsvm directory. If environment variables of VC++ have not been\nset, type\n\n\"C:\\Program Files\\Microsoft Visual Studio 10.0\\VC\\bin\\vcvars32.bat\"\n\nYou may have to modify the above command according which version of\nVC++ or where it is installed.\n\n2. Type\n\nnmake -f Makefile.win clean all\n\n3. (optional) To build shared library libsvm.dll, type\n\nnmake -f Makefile.win lib\n\nAnother way is to build them from Visual C++ environment. See details\nin libsvm FAQ.\n\n- Additional Tools: Sub-sampling, Parameter Selection, Format checking, etc.\n============================================================================\n\nSee the README file in the tools directory.\n\nPython Interface\n================\n\nSee the README file in python directory.\n\nAdditional Information\n======================\n\nIf you find LIBSVM helpful, please cite it as\n\nChih-Chung Chang and Chih-Jen Lin, LIBSVM: a library for \nsupport vector machines, 2001.\nSoftware available at http://www.csie.ntu.edu.tw/~cjlin/libsvm\n\nLIBSVM implementation document is available at\nhttp://www.csie.ntu.edu.tw/~cjlin/papers/libsvm.pdf\n\nFor any questions and comments, please email cjlin@csie.ntu.edu.tw\n\nAcknowledgments:\nThis work was supported in part by the National Science \nCouncil of Taiwan via the grant NSC 89-2213-E-002-013.\nThe authors thank their group members and users\nfor many helpful discussions and comments. They are listed in\nhttp://www.csie.ntu.edu.tw/~cjlin/libsvm/acknowledgements\n\n"
},
{
"path": "src/linux/README-GPU",
"content": "GPU-Accelerated LIBSVM is exploiting the GPU, using the CUDA interface, to\nspeed-up the training process. This package contains a new executable for \ntraining classifiers \"svm-train-gpu.exe\" together with the original one.\nThe use of the new executable is exactly the same as with the original one.\n\n\n\nFEATURES\n\nMode Supported\n\n * c-svc classification with RBF kernel\n\nFunctionality / User interface\n\n * Same as LIBSVM\n\n\nPREREQUISITES\n\n * NVIDIA Graphics card with CUDA support\n * Latest NVIDIA drivers for GPU\n * CUDA toolkit & GPU Computing SDK 5.5 \n\n Download all in one package from: \n\thttps://developer.nvidia.com/cuda-downloads\n\t\n\nINSTRUCTIONS\n\n\t1. Install the NVIDIA drivers, CUDA toolkit and GPU Computing SDK code samples. You can find them all in one package here: \n\n\thttps://developer.nvidia.com/cuda-downloads (Version 5.5)\n\n\tYou may need some additional packets to be installed in order to complete the installation above. \n\n\tA very helpful and descriptive guide is on the CUDA webpage: \n\n\thttp://docs.nvidia.com/cuda/cuda-getting-started-guide-for-linux/index.html\n\n\tMake sure you have followed every step that is relevant to your system, like declaring $PATH and $LD_LIBRARY_PATH on your bash configuration file.\n\n\t2. Copy this folder anywhere you like.\n\n\t3. Use the Makefile found inside this folder.\n\n\t4. Find the \"svm-train-gpu\" executable inside this folder.\n\n\nTroubleshooting\n\n\t1. Nearly all problems are resolved by reading carefully through the nvidia guidelines.\n\n\t2. When making, there is a chance a \"cannot find cublas_v2.h\" or \"cuda_runtime.h\" error to arise. Find where these files are located (Default path is: \"/usr/local/cuda-5.5/include\") and replace the paths on the header files in \"kernel_matrix_calculation.c\" file with your system paths. Also you can change the location of the default CUDA toolkit path on the makefile (CUDA_PATH ?= \"/usr/local/cuda-5.5\") to your cuda toolkit path.\n\n\n\nAdditional Information\n======================\n\n\tIf you find GPU-Accelerated LIBSVM helpful, please cite it as\n\n\tA. Athanasopoulos, A. Dimou, V. Mezaris, I. Kompatsiaris, \"GPU Acceleration for Support Vector Machines\",\n\tProc. 12th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS 2011), Delft, The Netherlands, April 2011.\n\n\tSoftware available at http://mklab.iti.gr/project/GPU-LIBSVM\n"
},
{
"path": "src/linux/cross_validation_with_matrix_precomputation.c",
"content": "void setup_pkm(struct svm_problem *p_km)\n{\n\n\tint i;\n\n\tp_km->l = prob.l;\n\tp_km->x = Malloc(struct svm_node,p_km->l);\n\tp_km->y = Malloc(double,p_km->l);\n\n\tfor(i=0;ix+i)->values = Malloc(double,prob.l+1);\n\t\t(p_km->x+i)->dim = prob.l+1;\n\t}\n\n\tfor( i=0; iy[i] = prob.y[i];\n}\n\nvoid free_pkm(struct svm_problem *p_km)\n{\n\n\tint i;\n\n\tfor(i=0;ix+i)->values);\n\n\tfree( p_km->x );\n\tfree( p_km->y );\n\n}\n\n\ndouble do_crossvalidation(struct svm_problem * p_km)\n{\n\t\t\tdouble rate;\n\n\t\t\tint i;\n\t\t\tint total_correct = 0;\n\t\t\tdouble total_error = 0;\n\t\t\tdouble sumv = 0, sumy = 0, sumvv = 0, sumyy = 0, sumvy = 0;\n\t\t\tdouble *target = Malloc(double,prob.l);\n\n\t\t\tsvm_cross_validation(p_km,¶m,nr_fold,target);\n\n\n\t\t\tif(param.svm_type == EPSILON_SVR ||\n\t\t\t\tparam.svm_type == NU_SVR)\n\t\t\t{\n\t\t\t\tfor(i=0;i/dev/null | tr \"[:lower:]\" \"[:upper:]\")\nOSLOWER = $(shell uname -s 2>/dev/null | tr \"[:upper:]\" \"[:lower:]\")\n\n# Flags to detect 32-bit or 64-bit OS platform\nOS_SIZE = $(shell uname -m | sed -e \"s/i.86/32/\" -e \"s/x86_64/64/\" -e \"s/armv7l/32/\")\nOS_ARCH = $(shell uname -m | sed -e \"s/i386/i686/\")\n\n# Determine OS platform and unix distribution\nifeq (\"$(OSLOWER)\",\"linux\")\n # first search lsb_release\n DISTRO = $(shell lsb_release -i -s 2>/dev/null | tr \"[:upper:]\" \"[:lower:]\")\n DISTVER = $(shell lsb_release -r -s 2>/dev/null)\n ifeq (\"$(DISTRO)\",'') \n # second search and parse /etc/issue\n DISTRO = $(shell more /etc/issue | awk '{print $$1}' | sed '1!d' | sed -e \"/^$$/d\" 2>/dev/null | tr \"[:upper:]\" \"[:lower:]\")\n DISTVER= $(shell more /etc/issue | awk '{print $$2}' | sed '1!d' 2>/dev/null\n endif\n ifeq (\"$(DISTRO)\",'') \n # third, we can search in /etc/os-release or /etc/{distro}-release\n DISTRO = $(shell awk '/ID/' /etc/*-release | sed 's/ID=//' | grep -v \"VERSION\" | grep -v \"ID\" | grep -v \"DISTRIB\")\n DISTVER= $(shell awk '/DISTRIB_RELEASE/' /etc/*-release | sed 's/DISTRIB_RELEASE=//' | grep -v \"DISTRIB_RELEASE\")\n endif\nendif\n\n# search at Darwin (unix based info)\nDARWIN = $(strip $(findstring DARWIN, $(OSUPPER)))\nifneq ($(DARWIN),)\n SNOWLEOPARD = $(strip $(findstring 10.6, $(shell egrep \"10\\.6\" /System/Library/CoreServices/SystemVersion.plist)))\n LION = $(strip $(findstring 10.7, $(shell egrep \"10\\.7\" /System/Library/CoreServices/SystemVersion.plist)))\n MOUNTAIN = $(strip $(findstring 10.8, $(shell egrep \"10\\.8\" /System/Library/CoreServices/SystemVersion.plist)))\n MAVERICKS = $(strip $(findstring 10.9, $(shell egrep \"10\\.9\" /System/Library/CoreServices/SystemVersion.plist)))\nendif \n\n# Common binaries\nGCC ?= g++\nCLANG ?= /usr/bin/clang\n\nifeq (\"$(OSUPPER)\",\"LINUX\")\n NVCC ?= $(CUDA_PATH)/bin/nvcc -ccbin $(GCC)\nelse\n # for some newer versions of XCode, CLANG is the default compiler, so we need to include this\n ifneq ($(MAVERICKS),)\n NVCC ?= $(CUDA_PATH)/bin/nvcc -ccbin $(CLANG)\n STDLIB ?= -stdlib=libstdc++\n else\n NVCC ?= $(CUDA_PATH)/bin/nvcc -ccbin $(GCC)\n endif\nendif\n\n# Take command line flags that override any of these settings\nifeq ($(i386),1)\n\tOS_SIZE = 32\n\tOS_ARCH = i686\nendif\nifeq ($(x86_64),1)\n\tOS_SIZE = 64\n\tOS_ARCH = x86_64\nendif\nifeq ($(ARMv7),1)\n\tOS_SIZE = 32\n\tOS_ARCH = armv7l\nendif\n\nifeq (\"$(OSUPPER)\",\"LINUX\")\n # Each Linux Distribuion has a set of different paths. This applies especially when using the Linux RPM/debian packages\n ifeq (\"$(DISTRO)\",\"ubuntu\")\n CUDAPATH ?= /usr/lib/nvidia-current\n CUDALINK ?= -L/usr/lib/nvidia-current\n DFLT_PATH = /usr/lib\n endif\n ifeq (\"$(DISTRO)\",\"kubuntu\")\n CUDAPATH ?= /usr/lib/nvidia-current\n CUDALINK ?= -L/usr/lib/nvidia-current\n DFLT_PATH = /usr/lib\n endif\n ifeq (\"$(DISTRO)\",\"debian\")\n CUDAPATH ?= /usr/lib/nvidia-current\n CUDALINK ?= -L/usr/lib/nvidia-current\n DFLT_PATH = /usr/lib\n endif\n ifeq (\"$(DISTRO)\",\"suse\")\n ifeq ($(OS_SIZE),64)\n CUDAPATH ?=\n CUDALINK ?=\n DFLT_PATH = /usr/lib64\n else\n CUDAPATH ?=\n CUDALINK ?=\n DFLT_PATH = /usr/lib\n endif\n endif\n ifeq (\"$(DISTRO)\",\"suse linux\")\n ifeq ($(OS_SIZE),64)\n CUDAPATH ?=\n CUDALINK ?=\n DFLT_PATH = /usr/lib64\n else\n CUDAPATH ?=\n CUDALINK ?=\n DFLT_PATH = /usr/lib\n endif\n endif\n ifeq (\"$(DISTRO)\",\"opensuse\")\n ifeq ($(OS_SIZE),64)\n CUDAPATH ?=\n CUDALINK ?=\n DFLT_PATH = /usr/lib64\n else\n CUDAPATH ?=\n CUDALINK ?=\n DFLT_PATH = /usr/lib\n endif\n endif\n ifeq (\"$(DISTRO)\",\"fedora\")\n ifeq ($(OS_SIZE),64)\n CUDAPATH ?= /usr/lib64/nvidia\n CUDALINK ?= -L/usr/lib64/nvidia\n DFLT_PATH = /usr/lib64\n else\n CUDAPATH ?=\n CUDALINK ?=\n DFLT_PATH = /usr/lib\n endif\n endif\n ifeq (\"$(DISTRO)\",\"redhat\")\n ifeq ($(OS_SIZE),64)\n CUDAPATH ?= /usr/lib64/nvidia\n CUDALINK ?= -L/usr/lib64/nvidia\n DFLT_PATH = /usr/lib64\n else\n CUDAPATH ?=\n CUDALINK ?=\n DFLT_PATH = /usr/lib\n endif\n endif\n ifeq (\"$(DISTRO)\",\"red\")\n ifeq ($(OS_SIZE),64)\n CUDAPATH ?= /usr/lib64/nvidia\n CUDALINK ?= -L/usr/lib64/nvidia\n DFLT_PATH = /usr/lib64\n else\n CUDAPATH ?=\n CUDALINK ?=\n DFLT_PATH = /usr/lib\n endif\n endif\n ifeq (\"$(DISTRO)\",\"redhatenterpriseworkstation\")\n ifeq ($(OS_SIZE),64)\n CUDAPATH ?= /usr/lib64/nvidia\n CUDALINK ?= -L/usr/lib64/nvidia\n DFLT_PATH ?= /usr/lib64\n else\n CUDAPATH ?=\n CUDALINK ?=\n DFLT_PATH ?= /usr/lib\n endif\n endif\n ifeq (\"$(DISTRO)\",\"centos\")\n ifeq ($(OS_SIZE),64)\n CUDAPATH ?= /usr/lib64/nvidia\n CUDALINK ?= -L/usr/lib64/nvidia\n DFLT_PATH = /usr/lib64\n else\n CUDAPATH ?=\n CUDALINK ?=\n DFLT_PATH = /usr/lib\n endif\n endif\n \n ifeq ($(ARMv7),1)\n CUDAPATH := /usr/arm-linux-gnueabihf/lib\n CUDALINK := -L/usr/arm-linux-gnueabihf/lib\n ifneq ($(TARGET_FS),) \n CUDAPATH += $(TARGET_FS)/usr/lib/nvidia-current\n CUDALINK += -L$(TARGET_FS)/usr/lib/nvidia-current\n endif \n endif\n\n # Search for Linux distribution path for libcuda.so\n CUDALIB ?= $(shell find $(CUDAPATH) $(DFLT_PATH) -name libcuda.so -print 2>/dev/null)\n\n ifeq (\"$(CUDALIB)\",'')\n $(info >>> WARNING - CUDA Driver libcuda.so is not found. Please check and re-install the NVIDIA driver. <<<)\n EXEC=@echo \"[@]\"\n endif\nelse\n # This would be the Mac OS X path if we had to do anything special\nendif\n\n"
},
{
"path": "src/linux/kernel_matrix_calculation.c",
"content": "#include \"/usr/local/cuda-5.5/include/cuda_runtime.h\"\n#include \"/usr/local/cuda-5.5/include/cublas_v2.h\"\n\n// Scalars\nconst float alpha = 1;\nconst float beta = 0;\n\nvoid ckm( struct svm_problem *prob, struct svm_problem *pecm, float *gamma )\n{\n\tcublasStatus_t status;\n\n\tdouble g_val = *gamma;\n\n\tlong int nfa;\n\t\n\tint len_tv;\n\tint ntv;\n\tint i_v;\n\tint i_el;\n\tint i_r, i_c;\n\tint trvei;\n\n\tdouble *tv_sq;\n\tdouble *v_f_g;\n\n\tfloat *tr_ar;\n\tfloat *tva, *vtm, *DP;\n\tfloat *g_tva = 0, *g_vtm = 0, *g_DotProd = 0;\n\n\tcudaError_t cudaStat; \n\tcublasHandle_t handle;\n\t\n\tstatus = cublasCreate(&handle);\n\n\tlen_tv = prob-> x[0].dim;\n\tntv = prob-> l;\n\n\tnfa = len_tv * ntv; \n\n\ttva = (float*) malloc ( len_tv * ntv* sizeof(float) );\n\tvtm = (float*) malloc ( len_tv * sizeof(float) );\n\tDP = (float*) malloc ( ntv * sizeof(float) );\n\n\ttr_ar = (float*) malloc ( len_tv * ntv* sizeof(float) );\n\n\ttv_sq = (double*) malloc ( ntv * sizeof(double) );\n\n\tv_f_g = (double*) malloc ( ntv * sizeof(double) );\n\n\tfor ( i_r = 0; i_r < ntv ; i_r++ )\n\t{\t\t\t\t \n\t\tfor ( i_c = 0; i_c < len_tv; i_c++ ) \n\t\t\ttva[i_r * len_tv + i_c] = (float)prob-> x[i_r].values[i_c];\n\t}\n\n\tcudaStat = cudaMalloc((void**)&g_tva, len_tv * ntv * sizeof(float));\n\t\n\tif (cudaStat != cudaSuccess) {\n\t\tfree( tva );\n\t\tfree( vtm );\n\t\tfree( DP );\n\n\t\tfree( v_f_g );\n\t\tfree( tv_sq );\n\n\t\tcudaFree( g_tva );\n\t\tcublasDestroy( handle );\t\n\t\n\t\tfprintf (stderr, \"!!!! Device memory allocation error (A)\\n\");\n\t\tgetchar();\n\t\treturn;\n }\n\n\tcudaStat = cudaMalloc((void**)&g_vtm, len_tv * sizeof(float));\n\n\tcudaStat = cudaMalloc((void**)&g_DotProd, ntv * sizeof(float));\n\n\tfor( i_r = 0; i_r < ntv; i_r++ )\n\t\tfor( i_c = 0; i_c < len_tv; i_c++ )\n\t\t\ttr_ar[i_c * ntv + i_r] = tva[i_r * len_tv + i_c];\n\n\t// Copy cpu vector to gpu vector\n\tstatus = cublasSetVector( len_tv * ntv, sizeof(float), tr_ar, 1, g_tva, 1 );\n \n\tfree( tr_ar );\n\n\tfor( i_v = 0; i_v < ntv; i_v++ )\n\t{\n\t\ttv_sq[ i_v ] = 0;\n\t\tfor( i_el = 0; i_el < len_tv; i_el++ )\n\t\t\ttv_sq[i_v] += pow( tva[i_v*len_tv + i_el], (float)2.0 );\n\t}\n\n\n\n\tfor ( trvei = 0; trvei < ntv; trvei++ )\n\t{\n\t\tstatus = cublasSetVector( len_tv, sizeof(float), &tva[trvei * len_tv], 1, g_vtm, 1 );\n\t\t\n\t\tstatus = cublasSgemv( handle, CUBLAS_OP_N, ntv, len_tv, &alpha, g_tva, ntv , g_vtm, 1, &beta, g_DotProd, 1 );\n\n\t\tstatus = cublasGetVector( ntv, sizeof(float), g_DotProd, 1, DP, 1 );\n\n\t\tfor ( i_c = 0; i_c < ntv; i_c++ )\n\t\t\tv_f_g[i_c] = exp( -g_val * (tv_sq[trvei] + tv_sq[i_c]-((double)2.0)* (double)DP[i_c] ));\n\t\t\n\n\t\tpecm-> x[trvei].values[0] = trvei + 1;\n\t\t\n\t\tfor ( i_c = 0; i_c < ntv; i_c++ )\n\t\t\tpecm-> x[trvei].values[i_c + 1] = v_f_g[i_c];\t\t\t\t\n\t\t\n\n\t}\n\n\tfree( tva );\n\tfree( vtm );\n\tfree( DP );\n\tfree( v_f_g );\n\tfree( tv_sq );\n\n\tcudaFree( g_tva );\n\tcudaFree( g_vtm );\n\tcudaFree( g_DotProd );\n\n\tcublasDestroy( handle );\n}\n\nvoid cal_km( struct svm_problem * p_km)\n{\n\tfloat gamma = param.gamma;\n\n\tckm(&prob, p_km, &gamma);\n}\n"
},
{
"path": "src/linux/readme.txt",
"content": "Instructions to compile Linux GPU-Accelerated LIBSVM\n\n1. Install the NVIDIA drivers, CUDA toolkit and GPU Computing SDK code samples. You can find them all in one package here: \n\nhttps://developer.nvidia.com/cuda-downloads (Version 5.5)\n\nYou may need some additional packets to be installed in order to complete the installation above. \n\nA very helpful and descriptive guide is on the CUDA webpage: \n\nhttp://docs.nvidia.com/cuda/cuda-getting-started-guide-for-linux/index.html\n\nMake sure you have followed every step that is relevant to your system, like declaring $PATH and $LD_LIBRARY_PATH on your bash configuration file.\n\n2. Copy this folder anywhere you like.\n\n3. Use the Makefile found inside this folder.\n\n4. Find the \"svm-train-gpu\" executable inside this folder.\n\n\nTroubleshooting\n\n1. Nearly all problems are resolved by reading carefully through the nvidia guidelines.\n\n2. When making, there is a chance a \"cannot find cublas_v2.h\" or \"cuda_runtime.h\" error to arise. Find where these files are located (Default path is: \"/usr/local/cuda-5.5/include\") and replace the paths on the header files in \"kernel_matrix_calculation.c\" file with your system paths.\n\nOr you can change the location of the default CUDA toolkit path on the makefile (CUDA_PATH ?= \"/usr/local/cuda-5.5\") to your cuda toolkit path\n"
},
{
"path": "src/linux/svm-train.c",
"content": "#include \n#include \n#include \n#include \n#include \n#include \"svm.h\"\n#define Malloc(type,n) (type *)malloc((n)*sizeof(type))\n\nvoid print_null(const char *s) {}\n\nvoid exit_with_help()\n{\n\tprintf(\n\t\"Usage: svm-train [options] training_set_file [model_file]\\n\"\n\t\"options:\\n\"\n\t\"-s svm_type : set type of SVM (default 0)\\n\"\n\t\"\t0 -- C-SVC\t\t(multi-class classification)\\n\"\n\t\"\t1 -- nu-SVC\t\t(multi-class classification)\\n\"\n\t\"\t2 -- one-class SVM\\n\"\n\t\"\t3 -- epsilon-SVR\t(regression)\\n\"\n\t\"\t4 -- nu-SVR\t\t(regression)\\n\"\n\t\"-t kernel_type : set type of kernel function (default 2)\\n\"\n\t\"\t0 -- linear: u'*v\\n\"\n\t\"\t1 -- polynomial: (gamma*u'*v + coef0)^degree\\n\"\n\t\"\t2 -- radial basis function: exp(-gamma*|u-v|^2)\\n\"\n\t\"\t3 -- sigmoid: tanh(gamma*u'*v + coef0)\\n\"\n\t\"\t4 -- precomputed kernel (kernel values in training_set_file)\\n\"\n\t\"-d degree : set degree in kernel function (default 3)\\n\"\n\t\"-g gamma : set gamma in kernel function (default 1/num_features)\\n\"\n\t\"-r coef0 : set coef0 in kernel function (default 0)\\n\"\n\t\"-c cost : set the parameter C of C-SVC, epsilon-SVR, and nu-SVR (default 1)\\n\"\n\t\"-n nu : set the parameter nu of nu-SVC, one-class SVM, and nu-SVR (default 0.5)\\n\"\n\t\"-p epsilon : set the epsilon in loss function of epsilon-SVR (default 0.1)\\n\"\n\t\"-m cachesize : set cache memory size in MB (default 100)\\n\"\n\t\"-e epsilon : set tolerance of termination criterion (default 0.001)\\n\"\n\t\"-h shrinking : whether to use the shrinking heuristics, 0 or 1 (default 1)\\n\"\n\t\"-b probability_estimates : whether to train a SVC or SVR model for probability estimates, 0 or 1 (default 0)\\n\"\n\t\"-wi weight : set the parameter C of class i to weight*C, for C-SVC (default 1)\\n\"\n\t\"-v n: n-fold cross validation mode\\n\"\n\t\"-q : quiet mode (no outputs)\\n\"\n\t);\n\texit(1);\n}\n\nvoid exit_input_error(int line_num)\n{\n\tfprintf(stderr,\"Wrong input format at line %d\\n\", line_num);\n\texit(1);\n}\n\nvoid parse_command_line(int argc, char **argv, char *input_file_name, char *model_file_name);\nvoid read_problem(const char *filename);\nvoid do_cross_validation();\n\nstruct svm_parameter param;\t\t// set by parse_command_line\nstruct svm_problem prob;\t\t// set by read_problem\nstruct svm_model *model;\nstruct svm_node *x_space;\nint cross_validation;\nint nr_fold;\n\nstatic char *line = NULL;\nstatic int max_line_len;\n\n#include \"kernel_matrix_calculation.c\"\n#include \"cross_validation_with_matrix_precomputation.c\"\n\nstatic char* readline(FILE *input)\n{\n\tint len;\n\t\n\tif(fgets(line,max_line_len,input) == NULL)\n\t\treturn NULL;\n\n\twhile(strrchr(line,'\\n') == NULL)\n\t{\n\t\tmax_line_len *= 2;\n\t\tline = (char *) realloc(line,max_line_len);\n\t\tlen = (int) strlen(line);\n\t\tif(fgets(line+len,max_line_len-len,input) == NULL)\n\t\t\tbreak;\n\t}\n\treturn line;\n}\n\nint main(int argc, char **argv)\n{\n\tint i;\n\tchar input_file_name[1024];\n\tchar model_file_name[1024];\n\tconst char *error_msg;\n\n\tparse_command_line(argc, argv, input_file_name, model_file_name);\n\tread_problem(input_file_name);\n\terror_msg = svm_check_parameter(&prob,¶m);\n\tif(error_msg)\n\t{\n\t\tfprintf(stderr,\"ERROR: %s\\n\",error_msg);\n\t\texit(1);\n\t}\n\n\tif(cross_validation)\n\t{\n\t\tdo_cross_validation_with_KM_precalculated( );\n\n//\tdo_cross_validation();\n\t}\n\telse\n\t{\n\t\tmodel = svm_train(&prob,¶m);\n\t\tif(svm_save_model(model_file_name,model))\n\t\t{\n\t\t\tfprintf(stderr, \"can't save model to file %s\\n\", model_file_name);\n\t\t\texit(1);\n\t\t}\n\t\tsvm_free_and_destroy_model(&model);\n\t}\n\tsvm_destroy_param(¶m);\n\tfree(prob.y);\n\n#ifdef _DENSE_REP\n\tfor (i = 0; i < prob.l; ++i)\n\t\tfree((prob.x+i)->values);\n#else\n\tfree(x_space);\n#endif\n\tfree(prob.x);\n\tfree(line);\n\n\treturn 0;\n}\n\nvoid do_cross_validation()\n{\n\tint i;\n\tint total_correct = 0;\n\tdouble total_error = 0;\n\tdouble sumv = 0, sumy = 0, sumvv = 0, sumyy = 0, sumvy = 0;\n\tdouble *target = Malloc(double,prob.l);\n\n\tsvm_cross_validation(&prob,¶m,nr_fold,target);\n\tif(param.svm_type == EPSILON_SVR ||\n\t param.svm_type == NU_SVR)\n\t{\n\t\tfor(i=0;i=argc)\n\t\t\texit_with_help();\n\t\tswitch(argv[i-1][1])\n\t\t{\n\t\t\tcase 's':\n\t\t\t\tparam.svm_type = atoi(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 't':\n\t\t\t\tparam.kernel_type = atoi(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'd':\n\t\t\t\tparam.degree = atoi(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'g':\n\t\t\t\tparam.gamma = atof(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'r':\n\t\t\t\tparam.coef0 = atof(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'n':\n\t\t\t\tparam.nu = atof(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'm':\n\t\t\t\tparam.cache_size = atof(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'c':\n\t\t\t\tparam.C = atof(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'e':\n\t\t\t\tparam.eps = atof(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'p':\n\t\t\t\tparam.p = atof(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'h':\n\t\t\t\tparam.shrinking = atoi(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'b':\n\t\t\t\tparam.probability = atoi(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'q':\n\t\t\t\tprint_func = &print_null;\n\t\t\t\ti--;\n\t\t\t\tbreak;\n\t\t\tcase 'v':\n\t\t\t\tcross_validation = 1;\n\t\t\t\tnr_fold = atoi(argv[i]);\n\t\t\t\tif(nr_fold < 2)\n\t\t\t\t{\n\t\t\t\t\tfprintf(stderr,\"n-fold cross validation: n must >= 2\\n\");\n\t\t\t\t\texit_with_help();\n\t\t\t\t}\n\t\t\t\tbreak;\n\t\t\tcase 'w':\n\t\t\t\t++param.nr_weight;\n\t\t\t\tparam.weight_label = (int *)realloc(param.weight_label,sizeof(int)*param.nr_weight);\n\t\t\t\tparam.weight = (double *)realloc(param.weight,sizeof(double)*param.nr_weight);\n\t\t\t\tparam.weight_label[param.nr_weight-1] = atoi(&argv[i-1][2]);\n\t\t\t\tparam.weight[param.nr_weight-1] = atof(argv[i]);\n\t\t\t\tbreak;\n\t\t\tdefault:\n\t\t\t\tfprintf(stderr,\"Unknown option: -%c\\n\", argv[i-1][1]);\n\t\t\t\texit_with_help();\n\t\t}\n\t}\n\n\tsvm_set_print_string_function(print_func);\n\n\t// determine filenames\n\n\tif(i>=argc)\n\t\texit_with_help();\n\n\tstrcpy(input_file_name, argv[i]);\n\n\tif(i line)\n\t\t\t\tp--;\n\t\t\tif(p > line)\n\t\t\t \tmax_index = (int) strtol(p,&endptr,10) + 1;\n\t\t}\n\t\tif(max_index > elements)\n\t\t\telements = max_index;\n\t\t++prob.l;\n\t}\n\n\trewind(fp);\n\n\tprob.y = Malloc(double,prob.l);\n\tprob.x = Malloc(struct svm_node,prob.l);\n\n\tfor(i=0;ivalues = Malloc(double,elements);\n\t\t(prob.x+i)->dim = 0;\n\n\t\tinst_max_index = -1; // strtol gives 0 if wrong format, and precomputed kernel has start from 0\n\t\treadline(fp);\n\n\t\tlabel = strtok(line,\" \\t\");\n\t\tprob.y[i] = strtod(label,&endptr);\n\t\tif(endptr == label)\n\t\t\texit_input_error(i+1);\n\n\t\twhile(1)\n\t\t{\n\t\t\tidx = strtok(NULL,\":\");\n\t\t\tval = strtok(NULL,\" \\t\");\n\n\t\t\tif(val == NULL)\n\t\t\t\tbreak;\n\n\t\t\terrno = 0;\n\t\t\tj = (int) strtol(idx,&endptr,10);\n\t\t\tif(endptr == idx || errno != 0 || *endptr != '\\0' || j <= inst_max_index)\n\t\t\t\texit_input_error(i+1);\n\t\t\telse\n\t\t\t\tinst_max_index = j;\n\n\t\t\terrno = 0;\n\t\t\tvalue = strtod(val,&endptr);\n\t\t\tif(endptr == val || errno != 0 || (*endptr != '\\0' && !isspace(*endptr)))\n\t\t\t\texit_input_error(i+1);\n\n\t\t\td = &((prob.x+i)->dim);\n\t\t\twhile (*d < j)\n\t\t\t\t(prob.x+i)->values[(*d)++] = 0.0;\n\t\t\t(prob.x+i)->values[(*d)++] = value;\n\t\t}\t\n\t}\n\tmax_index = elements-1;\n\n#else\n\twhile(readline(fp)!=NULL)\n\t{\n\t\tchar *p = strtok(line,\" \\t\"); // label\n\n\t\t// features\n\t\twhile(1)\n\t\t{\n\t\t\tp = strtok(NULL,\" \\t\");\n\t\t\tif(p == NULL || *p == '\\n') // check '\\n' as ' ' may be after the last feature\n\t\t\t\tbreak;\n\t\t\t++elements;\n\t\t}\n\t\t++elements;\n\t\t++prob.l;\n\t}\n\trewind(fp);\n\n\tprob.y = Malloc(double,prob.l);\n\tprob.x = Malloc(struct svm_node *,prob.l);\n\tx_space = Malloc(struct svm_node,elements);\n\n\tmax_index = 0;\n\tj=0;\n\tfor(i=0;i start from 0\n\t\treadline(fp);\n\t\tprob.x[i] = &x_space[j];\n\t\tlabel = strtok(line,\" \\t\\n\");\n\t\tif(label == NULL) // empty line\n\t\t\texit_input_error(i+1);\n\t\t\t\n\t\tprob.y[i] = strtod(label,&endptr);\n\t\tif(endptr == label || *endptr != '\\0')\n\t\t\texit_input_error(i+1);\n\n\t\twhile(1)\n\t\t{\n\t\t\tidx = strtok(NULL,\":\");\n\t\t\tval = strtok(NULL,\" \\t\");\n\n\t\t\tif(val == NULL)\n\t\t\t\tbreak;\n\n\t\t\terrno = 0;\n\t\t\tx_space[j].index = (int) strtol(idx,&endptr,10);\n\t\t\tif(endptr == idx || errno != 0 || *endptr != '\\0' || x_space[j].index <= inst_max_index)\n\t\t\t\texit_input_error(i+1);\n\t\t\telse\n\t\t\t\tinst_max_index = x_space[j].index;\n\n\t\t\terrno = 0;\n\t\t\tx_space[j].value = strtod(val,&endptr);\n\t\t\tif(endptr == val || errno != 0 || (*endptr != '\\0' && !isspace(*endptr)))\n\t\t\t\texit_input_error(i+1);\n\n\t\t\t++j;\n\t\t}\n\n\t\tif(inst_max_index > max_index)\n\t\t\tmax_index = inst_max_index;\n\t\tx_space[j++].index = -1;\n\t}\n#endif\n\n\tif(param.gamma == 0 && max_index > 0)\n\t\tparam.gamma = 1.0/max_index;\n\n\tif(param.kernel_type == PRECOMPUTED)\n\t\tfor(i=0;idim == 0 || (prob.x+i)->values[0] == 0.0)\n\t\t\t{\n\t\t\t\tfprintf(stderr,\"Wrong input format: first column must be 0:sample_serial_number\\n\");\n\t\t\t\texit(1);\n\t\t\t}\n\t\t\tif ((int)(prob.x+i)->values[0] < 0 || (int)(prob.x+i)->values[0] > max_index)\n\t\t\t{\n\t\t\t\tfprintf(stderr,\"Wrong input format: sample_serial_number out of range\\n\");\n\t\t\t\texit(1);\n\t\t\t}\n#else\n\t\t\tif (prob.x[i][0].index != 0)\n\t\t\t{\n\t\t\t\tfprintf(stderr,\"Wrong input format: first column must be 0:sample_serial_number\\n\");\n\t\t\t\texit(1);\n\t\t\t}\n\t\t\tif ((int)prob.x[i][0].value <= 0 || (int)prob.x[i][0].value > max_index)\n\t\t\t{\n\t\t\t\tfprintf(stderr,\"Wrong input format: sample_serial_number out of range\\n\");\n\t\t\t\texit(1);\n\t\t\t}\n#endif\n\t\t}\n\tfclose(fp);\n}\n"
},
{
"path": "src/linux/svm.cpp",
"content": "#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n\n#include \"svm.h\"\nint libsvm_version = LIBSVM_VERSION;\ntypedef float Qfloat;\ntypedef signed char schar;\n#ifndef min\ntemplate static inline T min(T x,T y) { return (x static inline T max(T x,T y) { return (x>y)?x:y; }\n#endif\ntemplate static inline void swap(T& x, T& y) { T t=x; x=y; y=t; }\ntemplate static inline void clone(T*& dst, S* src, int n)\n{\n\tdst = new T[n];\n\tmemcpy((void *)dst,(void *)src,sizeof(T)*n);\n}\nstatic inline double powi(double base, int times)\n{\n\tdouble tmp = base, ret = 1.0;\n\n\tfor(int t=times; t>0; t/=2)\n\t{\n\t\tif(t%2==1) ret*=tmp;\n\t\ttmp = tmp * tmp;\n\t}\n\treturn ret;\n}\n#define INF HUGE_VAL\n#define TAU 1e-12\n#define Malloc(type,n) (type *)malloc((n)*sizeof(type))\n\nstatic void print_string_stdout(const char *s)\n{\n\tfputs(s,stdout);\n\tfflush(stdout);\n}\nstatic void (*svm_print_string) (const char *) = &print_string_stdout;\n#if 1\nstatic void info(const char *fmt,...)\n{\n\tchar buf[BUFSIZ];\n\tva_list ap;\n\tva_start(ap,fmt);\n\tvsprintf(buf,fmt,ap);\n\tva_end(ap);\n\t(*svm_print_string)(buf);\n}\n#else\nstatic void info(const char *fmt,...) {}\n#endif\n\n//\n// Kernel Cache\n//\n// l is the number of total data items\n// size is the cache size limit in bytes\n//\nclass Cache\n{\npublic:\n\tCache(int l,long int size);\n\t~Cache();\n\n\t// request data [0,len)\n\t// return some position p where [p,len) need to be filled\n\t// (p >= len if nothing needs to be filled)\n\tint get_data(const int index, Qfloat **data, int len);\n\tvoid swap_index(int i, int j);\t\nprivate:\n\tint l;\n\tlong int size;\n\tstruct head_t\n\t{\n\t\thead_t *prev, *next;\t// a circular list\n\t\tQfloat *data;\n\t\tint len;\t\t// data[0,len) is cached in this entry\n\t};\n\n\thead_t *head;\n\thead_t lru_head;\n\tvoid lru_delete(head_t *h);\n\tvoid lru_insert(head_t *h);\n};\n\nCache::Cache(int l_,long int size_):l(l_),size(size_)\n{\n\thead = (head_t *)calloc(l,sizeof(head_t));\t// initialized to 0\n\tsize /= sizeof(Qfloat);\n\tsize -= l * sizeof(head_t) / sizeof(Qfloat);\n\tsize = max(size, 2 * (long int) l);\t// cache must be large enough for two columns\n\tlru_head.next = lru_head.prev = &lru_head;\n}\n\nCache::~Cache()\n{\n\tfor(head_t *h = lru_head.next; h != &lru_head; h=h->next)\n\t\tfree(h->data);\n\tfree(head);\n}\n\nvoid Cache::lru_delete(head_t *h)\n{\n\t// delete from current location\n\th->prev->next = h->next;\n\th->next->prev = h->prev;\n}\n\nvoid Cache::lru_insert(head_t *h)\n{\n\t// insert to last position\n\th->next = &lru_head;\n\th->prev = lru_head.prev;\n\th->prev->next = h;\n\th->next->prev = h;\n}\n\nint Cache::get_data(const int index, Qfloat **data, int len)\n{\n\thead_t *h = &head[index];\n\tif(h->len) lru_delete(h);\n\tint more = len - h->len;\n\n\tif(more > 0)\n\t{\n\t\t// free old space\n\t\twhile(size < more)\n\t\t{\n\t\t\thead_t *old = lru_head.next;\n\t\t\tlru_delete(old);\n\t\t\tfree(old->data);\n\t\t\tsize += old->len;\n\t\t\told->data = 0;\n\t\t\told->len = 0;\n\t\t}\n\n\t\t// allocate new space\n\t\th->data = (Qfloat *)realloc(h->data,sizeof(Qfloat)*len);\n\t\tsize -= more;\n\t\tswap(h->len,len);\n\t}\n\n\tlru_insert(h);\n\t*data = h->data;\n\treturn len;\n}\n\nvoid Cache::swap_index(int i, int j)\n{\n\tif(i==j) return;\n\n\tif(head[i].len) lru_delete(&head[i]);\n\tif(head[j].len) lru_delete(&head[j]);\n\tswap(head[i].data,head[j].data);\n\tswap(head[i].len,head[j].len);\n\tif(head[i].len) lru_insert(&head[i]);\n\tif(head[j].len) lru_insert(&head[j]);\n\n\tif(i>j) swap(i,j);\n\tfor(head_t *h = lru_head.next; h!=&lru_head; h=h->next)\n\t{\n\t\tif(h->len > i)\n\t\t{\n\t\t\tif(h->len > j)\n\t\t\t\tswap(h->data[i],h->data[j]);\n\t\t\telse\n\t\t\t{\n\t\t\t\t// give up\n\t\t\t\tlru_delete(h);\n\t\t\t\tfree(h->data);\n\t\t\t\tsize += h->len;\n\t\t\t\th->data = 0;\n\t\t\t\th->len = 0;\n\t\t\t}\n\t\t}\n\t}\n}\n\n//\n// Kernel evaluation\n//\n// the static method k_function is for doing single kernel evaluation\n// the constructor of Kernel prepares to calculate the l*l kernel matrix\n// the member function get_Q is for getting one column from the Q Matrix\n//\nclass QMatrix {\npublic:\n\tvirtual Qfloat *get_Q(int column, int len) const = 0;\n\tvirtual double *get_QD() const = 0;\n\tvirtual void swap_index(int i, int j) const = 0;\n\tvirtual ~QMatrix() {}\n};\n\nclass Kernel: public QMatrix {\npublic:\n#ifdef _DENSE_REP\n\tKernel(int l, svm_node * x, const svm_parameter& param);\n#else\n\tKernel(int l, svm_node * const * x, const svm_parameter& param);\n#endif\n\tvirtual ~Kernel();\n\n\tstatic double k_function(const svm_node *x, const svm_node *y,\n\t\t\t\t const svm_parameter& param);\n\tvirtual Qfloat *get_Q(int column, int len) const = 0;\n\tvirtual double *get_QD() const = 0;\n\tvirtual void swap_index(int i, int j) const\t// no so const...\n\t{\n\t\tswap(x[i],x[j]);\n\t\tif(x_square) swap(x_square[i],x_square[j]);\n\t}\nprotected:\n\n\tdouble (Kernel::*kernel_function)(int i, int j) const;\n\nprivate:\n#ifdef _DENSE_REP\n\tsvm_node *x;\n#else\n\tconst svm_node **x;\n#endif\n\tdouble *x_square;\n\n\t// svm_parameter\n\tconst int kernel_type;\n\tconst int degree;\n\tconst double gamma;\n\tconst double coef0;\n\n\tstatic double dot(const svm_node *px, const svm_node *py);\n#ifdef _DENSE_REP\n\tstatic double dot(const svm_node &px, const svm_node &py);\n#endif\n\n\tdouble kernel_linear(int i, int j) const\n\t{\n\t\treturn dot(x[i],x[j]);\n\t}\n\tdouble kernel_poly(int i, int j) const\n\t{\n\t\treturn powi(gamma*dot(x[i],x[j])+coef0,degree);\n\t}\n\tdouble kernel_rbf(int i, int j) const\n\t{\n\t\treturn exp(-gamma*(x_square[i]+x_square[j]-2*dot(x[i],x[j])));\n\t}\n\tdouble kernel_sigmoid(int i, int j) const\n\t{\n\t\treturn tanh(gamma*dot(x[i],x[j])+coef0);\n\t}\n\tdouble kernel_precomputed(int i, int j) const\n\t{\n#ifdef _DENSE_REP\n\t\treturn (x+i)->values[(int)((x+j)->values[0])];\n#else\n\t\treturn x[i][(int)(x[j][0].value)].value;\n#endif\n\t}\n};\n\n#ifdef _DENSE_REP\nKernel::Kernel(int l, svm_node * x_, const svm_parameter& param)\n#else\nKernel::Kernel(int l, svm_node * const * x_, const svm_parameter& param)\n#endif\n:kernel_type(param.kernel_type), degree(param.degree),\n gamma(param.gamma), coef0(param.coef0)\n{\n\tswitch(kernel_type)\n\t{\n\t\tcase LINEAR:\n\t\t\tkernel_function = &Kernel::kernel_linear;\n\t\t\tbreak;\n\t\tcase POLY:\n\t\t\tkernel_function = &Kernel::kernel_poly;\n\t\t\tbreak;\n\t\tcase RBF:\n\t\t\tkernel_function = &Kernel::kernel_rbf;\n\t\t\tbreak;\n\t\tcase SIGMOID:\n\t\t\tkernel_function = &Kernel::kernel_sigmoid;\n\t\t\tbreak;\n\t\tcase PRECOMPUTED:\n\t\t\tkernel_function = &Kernel::kernel_precomputed;\n\t\t\tbreak;\n\t}\n\n\tclone(x,x_,l);\n\n\tif(kernel_type == RBF)\n\t{\n\t\tx_square = new double[l];\n\t\tfor(int i=0;idim, py->dim);\n\tfor (int i = 0; i < dim; i++)\n\t\tsum += (px->values)[i] * (py->values)[i];\n\treturn sum;\n}\n\ndouble Kernel::dot(const svm_node &px, const svm_node &py)\n{\n\tdouble sum = 0;\n\n\tint dim = min(px.dim, py.dim);\n\tfor (int i = 0; i < dim; i++)\n\t\tsum += px.values[i] * py.values[i];\n\treturn sum;\n}\n#else\ndouble Kernel::dot(const svm_node *px, const svm_node *py)\n{\n\tdouble sum = 0;\n\twhile(px->index != -1 && py->index != -1)\n\t{\n\t\tif(px->index == py->index)\n\t\t{\n\t\t\tsum += px->value * py->value;\n\t\t\t++px;\n\t\t\t++py;\n\t\t}\n\t\telse\n\t\t{\n\t\t\tif(px->index > py->index)\n\t\t\t\t++py;\n\t\t\telse\n\t\t\t\t++px;\n\t\t}\t\t\t\n\t}\n\treturn sum;\n}\n#endif\n\ndouble Kernel::k_function(const svm_node *x, const svm_node *y,\n\t\t\t const svm_parameter& param)\n{\n\tswitch(param.kernel_type)\n\t{\n\t\tcase LINEAR:\n\t\t\treturn dot(x,y);\n\t\tcase POLY:\n\t\t\treturn powi(param.gamma*dot(x,y)+param.coef0,param.degree);\n\t\tcase RBF:\n\t\t{\n\t\t\tdouble sum = 0;\n#ifdef _DENSE_REP\n\t\t\tint dim = min(x->dim, y->dim), i;\n\t\t\tfor (i = 0; i < dim; i++)\n\t\t\t{\n\t\t\t\tdouble d = x->values[i] - y->values[i];\n\t\t\t\tsum += d*d;\n\t\t\t}\n\t\t\tfor (; i < x->dim; i++)\n\t\t\t\tsum += x->values[i] * x->values[i];\n\t\t\tfor (; i < y->dim; i++)\n\t\t\t\tsum += y->values[i] * y->values[i];\n#else\n\t\t\twhile(x->index != -1 && y->index !=-1)\n\t\t\t{\n\t\t\t\tif(x->index == y->index)\n\t\t\t\t{\n\t\t\t\t\tdouble d = x->value - y->value;\n\t\t\t\t\tsum += d*d;\n\t\t\t\t\t++x;\n\t\t\t\t\t++y;\n\t\t\t\t}\n\t\t\t\telse\n\t\t\t\t{\n\t\t\t\t\tif(x->index > y->index)\n\t\t\t\t\t{\t\n\t\t\t\t\t\tsum += y->value * y->value;\n\t\t\t\t\t\t++y;\n\t\t\t\t\t}\n\t\t\t\t\telse\n\t\t\t\t\t{\n\t\t\t\t\t\tsum += x->value * x->value;\n\t\t\t\t\t\t++x;\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\n\t\t\twhile(x->index != -1)\n\t\t\t{\n\t\t\t\tsum += x->value * x->value;\n\t\t\t\t++x;\n\t\t\t}\n\n\t\t\twhile(y->index != -1)\n\t\t\t{\n\t\t\t\tsum += y->value * y->value;\n\t\t\t\t++y;\n\t\t\t}\n#endif\n\t\t\treturn exp(-param.gamma*sum);\n\t\t}\n\t\tcase SIGMOID:\n\t\t\treturn tanh(param.gamma*dot(x,y)+param.coef0);\n\t\tcase PRECOMPUTED: //x: test (validation), y: SV\n#ifdef _DENSE_REP\n\t\t\treturn x->values[(int)(y->values[0])];\n#else\n\t\t\treturn x[(int)(y->value)].value;\n#endif\n\t\tdefault:\n\t\t\treturn 0; // Unreachable \n\t}\n}\n\n// An SMO algorithm in Fan et al., JMLR 6(2005), p. 1889--1918\n// Solves:\n//\n//\tmin 0.5(\\alpha^T Q \\alpha) + p^T \\alpha\n//\n//\t\ty^T \\alpha = \\delta\n//\t\ty_i = +1 or -1\n//\t\t0 <= alpha_i <= Cp for y_i = 1\n//\t\t0 <= alpha_i <= Cn for y_i = -1\n//\n// Given:\n//\n//\tQ, p, y, Cp, Cn, and an initial feasible point \\alpha\n//\tl is the size of vectors and matrices\n//\teps is the stopping tolerance\n//\n// solution will be put in \\alpha, objective value will be put in obj\n//\nclass Solver {\npublic:\n\tSolver() {};\n\tvirtual ~Solver() {};\n\n\tstruct SolutionInfo {\n\t\tdouble obj;\n\t\tdouble rho;\n\t\tdouble upper_bound_p;\n\t\tdouble upper_bound_n;\n\t\tdouble r;\t// for Solver_NU\n\t};\n\n\tvoid Solve(int l, const QMatrix& Q, const double *p_, const schar *y_,\n\t\t double *alpha_, double Cp, double Cn, double eps,\n\t\t SolutionInfo* si, int shrinking);\nprotected:\n\tint active_size;\n\tschar *y;\n\tdouble *G;\t\t// gradient of objective function\n\tenum { LOWER_BOUND, UPPER_BOUND, FREE };\n\tchar *alpha_status;\t// LOWER_BOUND, UPPER_BOUND, FREE\n\tdouble *alpha;\n\tconst QMatrix *Q;\n\tconst double *QD;\n\tdouble eps;\n\tdouble Cp,Cn;\n\tdouble *p;\n\tint *active_set;\n\tdouble *G_bar;\t\t// gradient, if we treat free variables as 0\n\tint l;\n\tbool unshrink;\t// XXX\n\n\tdouble get_C(int i)\n\t{\n\t\treturn (y[i] > 0)? Cp : Cn;\n\t}\n\tvoid update_alpha_status(int i)\n\t{\n\t\tif(alpha[i] >= get_C(i))\n\t\t\talpha_status[i] = UPPER_BOUND;\n\t\telse if(alpha[i] <= 0)\n\t\t\talpha_status[i] = LOWER_BOUND;\n\t\telse alpha_status[i] = FREE;\n\t}\n\tbool is_upper_bound(int i) { return alpha_status[i] == UPPER_BOUND; }\n\tbool is_lower_bound(int i) { return alpha_status[i] == LOWER_BOUND; }\n\tbool is_free(int i) { return alpha_status[i] == FREE; }\n\tvoid swap_index(int i, int j);\n\tvoid reconstruct_gradient();\n\tvirtual int select_working_set(int &i, int &j);\n\tvirtual double calculate_rho();\n\tvirtual void do_shrinking();\nprivate:\n\tbool be_shrunk(int i, double Gmax1, double Gmax2);\t\n};\n\nvoid Solver::swap_index(int i, int j)\n{\n\tQ->swap_index(i,j);\n\tswap(y[i],y[j]);\n\tswap(G[i],G[j]);\n\tswap(alpha_status[i],alpha_status[j]);\n\tswap(alpha[i],alpha[j]);\n\tswap(p[i],p[j]);\n\tswap(active_set[i],active_set[j]);\n\tswap(G_bar[i],G_bar[j]);\n}\n\nvoid Solver::reconstruct_gradient()\n{\n\t// reconstruct inactive elements of G from G_bar and free variables\n\n\tif(active_size == l) return;\n\n\tint i,j;\n\tint nr_free = 0;\n\n\tfor(j=active_size;j 2*active_size*(l-active_size))\n\t{\n\t\tfor(i=active_size;iget_Q(i,active_size);\n\t\t\tfor(j=0;jget_Q(i,l);\n\t\t\t\tdouble alpha_i = alpha[i];\n\t\t\t\tfor(j=active_size;jl = l;\n\tthis->Q = &Q;\n\tQD=Q.get_QD();\n\tclone(p, p_,l);\n\tclone(y, y_,l);\n\tclone(alpha,alpha_,l);\n\tthis->Cp = Cp;\n\tthis->Cn = Cn;\n\tthis->eps = eps;\n\tunshrink = false;\n\n\t// initialize alpha_status\n\t{\n\t\talpha_status = new char[l];\n\t\tfor(int i=0;iINT_MAX/100 ? INT_MAX : 100*l);\n\tint counter = min(l,1000)+1;\n\n\twhile(iter < max_iter)\n\t{\n\t\t// show progress and do shrinking\n\n\t\tif(--counter == 0)\n\t\t{\n\t\t\tcounter = min(l,1000);\n\t\t\tif(shrinking) do_shrinking();\n\t\t\tinfo(\".\");\n\t\t}\n\n\t\tint i,j;\n\t\tif(select_working_set(i,j)!=0)\n\t\t{\n\t\t\t// reconstruct the whole gradient\n\t\t\treconstruct_gradient();\n\t\t\t// reset active set size and check\n\t\t\tactive_size = l;\n\t\t\tinfo(\"*\");\n\t\t\tif(select_working_set(i,j)!=0)\n\t\t\t\tbreak;\n\t\t\telse\n\t\t\t\tcounter = 1;\t// do shrinking next iteration\n\t\t}\n\t\t\n\t\t++iter;\n\n\t\t// update alpha[i] and alpha[j], handle bounds carefully\n\t\t\n\t\tconst Qfloat *Q_i = Q.get_Q(i,active_size);\n\t\tconst Qfloat *Q_j = Q.get_Q(j,active_size);\n\n\t\tdouble C_i = get_C(i);\n\t\tdouble C_j = get_C(j);\n\n\t\tdouble old_alpha_i = alpha[i];\n\t\tdouble old_alpha_j = alpha[j];\n\n\t\tif(y[i]!=y[j])\n\t\t{\n\t\t\tdouble quad_coef = QD[i]+QD[j]+2*Q_i[j];\n\t\t\tif (quad_coef <= 0)\n\t\t\t\tquad_coef = TAU;\n\t\t\tdouble delta = (-G[i]-G[j])/quad_coef;\n\t\t\tdouble diff = alpha[i] - alpha[j];\n\t\t\talpha[i] += delta;\n\t\t\talpha[j] += delta;\n\t\t\t\n\t\t\tif(diff > 0)\n\t\t\t{\n\t\t\t\tif(alpha[j] < 0)\n\t\t\t\t{\n\t\t\t\t\talpha[j] = 0;\n\t\t\t\t\talpha[i] = diff;\n\t\t\t\t}\n\t\t\t}\n\t\t\telse\n\t\t\t{\n\t\t\t\tif(alpha[i] < 0)\n\t\t\t\t{\n\t\t\t\t\talpha[i] = 0;\n\t\t\t\t\talpha[j] = -diff;\n\t\t\t\t}\n\t\t\t}\n\t\t\tif(diff > C_i - C_j)\n\t\t\t{\n\t\t\t\tif(alpha[i] > C_i)\n\t\t\t\t{\n\t\t\t\t\talpha[i] = C_i;\n\t\t\t\t\talpha[j] = C_i - diff;\n\t\t\t\t}\n\t\t\t}\n\t\t\telse\n\t\t\t{\n\t\t\t\tif(alpha[j] > C_j)\n\t\t\t\t{\n\t\t\t\t\talpha[j] = C_j;\n\t\t\t\t\talpha[i] = C_j + diff;\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\telse\n\t\t{\n\t\t\tdouble quad_coef = QD[i]+QD[j]-2*Q_i[j];\n\t\t\tif (quad_coef <= 0)\n\t\t\t\tquad_coef = TAU;\n\t\t\tdouble delta = (G[i]-G[j])/quad_coef;\n\t\t\tdouble sum = alpha[i] + alpha[j];\n\t\t\talpha[i] -= delta;\n\t\t\talpha[j] += delta;\n\n\t\t\tif(sum > C_i)\n\t\t\t{\n\t\t\t\tif(alpha[i] > C_i)\n\t\t\t\t{\n\t\t\t\t\talpha[i] = C_i;\n\t\t\t\t\talpha[j] = sum - C_i;\n\t\t\t\t}\n\t\t\t}\n\t\t\telse\n\t\t\t{\n\t\t\t\tif(alpha[j] < 0)\n\t\t\t\t{\n\t\t\t\t\talpha[j] = 0;\n\t\t\t\t\talpha[i] = sum;\n\t\t\t\t}\n\t\t\t}\n\t\t\tif(sum > C_j)\n\t\t\t{\n\t\t\t\tif(alpha[j] > C_j)\n\t\t\t\t{\n\t\t\t\t\talpha[j] = C_j;\n\t\t\t\t\talpha[i] = sum - C_j;\n\t\t\t\t}\n\t\t\t}\n\t\t\telse\n\t\t\t{\n\t\t\t\tif(alpha[i] < 0)\n\t\t\t\t{\n\t\t\t\t\talpha[i] = 0;\n\t\t\t\t\talpha[j] = sum;\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\t// update G\n\n\t\tdouble delta_alpha_i = alpha[i] - old_alpha_i;\n\t\tdouble delta_alpha_j = alpha[j] - old_alpha_j;\n\t\t\n\t\tfor(int k=0;k= max_iter)\n\t{\n\t\tif(active_size < l)\n\t\t{\n\t\t\t// reconstruct the whole gradient to calculate objective value\n\t\t\treconstruct_gradient();\n\t\t\tactive_size = l;\n\t\t\tinfo(\"*\");\n\t\t}\n\t\tfprintf(stderr,\"\\nWARNING: reaching max number of iterations\\n\");\n\t}\n\t\n\t// calculate rho\n\n\tsi->rho = calculate_rho();\n\n\t// calculate objective value\n\t{\n\t\tdouble v = 0;\n\t\tint i;\n\t\tfor(i=0;iobj = v/2;\n\t}\n\n\t// put back the solution\n\t{\n\t\tfor(int i=0;iupper_bound_p = Cp;\n\tsi->upper_bound_n = Cn;\n\n\tinfo(\"\\noptimization finished, #iter = %d\\n\",iter);\n\n\tdelete[] p;\n\tdelete[] y;\n\tdelete[] alpha;\n\tdelete[] alpha_status;\n\tdelete[] active_set;\n\tdelete[] G;\n\tdelete[] G_bar;\n}\n\n// return 1 if already optimal, return 0 otherwise\nint Solver::select_working_set(int &out_i, int &out_j)\n{\n\t// return i,j such that\n\t// i: maximizes -y_i * grad(f)_i, i in I_up(\\alpha)\n\t// j: minimizes the decrease of obj value\n\t// (if quadratic coefficeint <= 0, replace it with tau)\n\t// -y_j*grad(f)_j < -y_i*grad(f)_i, j in I_low(\\alpha)\n\t\n\tdouble Gmax = -INF;\n\tdouble Gmax2 = -INF;\n\tint Gmax_idx = -1;\n\tint Gmin_idx = -1;\n\tdouble obj_diff_min = INF;\n\n\tfor(int t=0;t= Gmax)\n\t\t\t\t{\n\t\t\t\t\tGmax = -G[t];\n\t\t\t\t\tGmax_idx = t;\n\t\t\t\t}\n\t\t}\n\t\telse\n\t\t{\n\t\t\tif(!is_lower_bound(t))\n\t\t\t\tif(G[t] >= Gmax)\n\t\t\t\t{\n\t\t\t\t\tGmax = G[t];\n\t\t\t\t\tGmax_idx = t;\n\t\t\t\t}\n\t\t}\n\n\tint i = Gmax_idx;\n\tconst Qfloat *Q_i = NULL;\n\tif(i != -1) // NULL Q_i not accessed: Gmax=-INF if i=-1\n\t\tQ_i = Q->get_Q(i,active_size);\n\n\tfor(int j=0;j= Gmax2)\n\t\t\t\t\tGmax2 = G[j];\n\t\t\t\tif (grad_diff > 0)\n\t\t\t\t{\n\t\t\t\t\tdouble obj_diff; \n\t\t\t\t\tdouble quad_coef = QD[i]+QD[j]-2.0*y[i]*Q_i[j];\n\t\t\t\t\tif (quad_coef > 0)\n\t\t\t\t\t\tobj_diff = -(grad_diff*grad_diff)/quad_coef;\n\t\t\t\t\telse\n\t\t\t\t\t\tobj_diff = -(grad_diff*grad_diff)/TAU;\n\n\t\t\t\t\tif (obj_diff <= obj_diff_min)\n\t\t\t\t\t{\n\t\t\t\t\t\tGmin_idx=j;\n\t\t\t\t\t\tobj_diff_min = obj_diff;\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\telse\n\t\t{\n\t\t\tif (!is_upper_bound(j))\n\t\t\t{\n\t\t\t\tdouble grad_diff= Gmax-G[j];\n\t\t\t\tif (-G[j] >= Gmax2)\n\t\t\t\t\tGmax2 = -G[j];\n\t\t\t\tif (grad_diff > 0)\n\t\t\t\t{\n\t\t\t\t\tdouble obj_diff; \n\t\t\t\t\tdouble quad_coef = QD[i]+QD[j]+2.0*y[i]*Q_i[j];\n\t\t\t\t\tif (quad_coef > 0)\n\t\t\t\t\t\tobj_diff = -(grad_diff*grad_diff)/quad_coef;\n\t\t\t\t\telse\n\t\t\t\t\t\tobj_diff = -(grad_diff*grad_diff)/TAU;\n\n\t\t\t\t\tif (obj_diff <= obj_diff_min)\n\t\t\t\t\t{\n\t\t\t\t\t\tGmin_idx=j;\n\t\t\t\t\t\tobj_diff_min = obj_diff;\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\tif(Gmax+Gmax2 < eps)\n\t\treturn 1;\n\n\tout_i = Gmax_idx;\n\tout_j = Gmin_idx;\n\treturn 0;\n}\n\nbool Solver::be_shrunk(int i, double Gmax1, double Gmax2)\n{\n\tif(is_upper_bound(i))\n\t{\n\t\tif(y[i]==+1)\n\t\t\treturn(-G[i] > Gmax1);\n\t\telse\n\t\t\treturn(-G[i] > Gmax2);\n\t}\n\telse if(is_lower_bound(i))\n\t{\n\t\tif(y[i]==+1)\n\t\t\treturn(G[i] > Gmax2);\n\t\telse\t\n\t\t\treturn(G[i] > Gmax1);\n\t}\n\telse\n\t\treturn(false);\n}\n\nvoid Solver::do_shrinking()\n{\n\tint i;\n\tdouble Gmax1 = -INF;\t\t// max { -y_i * grad(f)_i | i in I_up(\\alpha) }\n\tdouble Gmax2 = -INF;\t\t// max { y_i * grad(f)_i | i in I_low(\\alpha) }\n\n\t// find maximal violating pair first\n\tfor(i=0;i= Gmax1)\n\t\t\t\t\tGmax1 = -G[i];\n\t\t\t}\n\t\t\tif(!is_lower_bound(i))\t\n\t\t\t{\n\t\t\t\tif(G[i] >= Gmax2)\n\t\t\t\t\tGmax2 = G[i];\n\t\t\t}\n\t\t}\n\t\telse\t\n\t\t{\n\t\t\tif(!is_upper_bound(i))\t\n\t\t\t{\n\t\t\t\tif(-G[i] >= Gmax2)\n\t\t\t\t\tGmax2 = -G[i];\n\t\t\t}\n\t\t\tif(!is_lower_bound(i))\t\n\t\t\t{\n\t\t\t\tif(G[i] >= Gmax1)\n\t\t\t\t\tGmax1 = G[i];\n\t\t\t}\n\t\t}\n\t}\n\n\tif(unshrink == false && Gmax1 + Gmax2 <= eps*10) \n\t{\n\t\tunshrink = true;\n\t\treconstruct_gradient();\n\t\tactive_size = l;\n\t\tinfo(\"*\");\n\t}\n\n\tfor(i=0;i i)\n\t\t\t{\n\t\t\t\tif (!be_shrunk(active_size, Gmax1, Gmax2))\n\t\t\t\t{\n\t\t\t\t\tswap_index(i,active_size);\n\t\t\t\t\tbreak;\n\t\t\t\t}\n\t\t\t\tactive_size--;\n\t\t\t}\n\t\t}\n}\n\ndouble Solver::calculate_rho()\n{\n\tdouble r;\n\tint nr_free = 0;\n\tdouble ub = INF, lb = -INF, sum_free = 0;\n\tfor(int i=0;i0)\n\t\tr = sum_free/nr_free;\n\telse\n\t\tr = (ub+lb)/2;\n\n\treturn r;\n}\n\n//\n// Solver for nu-svm classification and regression\n//\n// additional constraint: e^T \\alpha = constant\n//\nclass Solver_NU: public Solver\n{\npublic:\n\tSolver_NU() {}\n\tvoid Solve(int l, const QMatrix& Q, const double *p, const schar *y,\n\t\t double *alpha, double Cp, double Cn, double eps,\n\t\t SolutionInfo* si, int shrinking)\n\t{\n\t\tthis->si = si;\n\t\tSolver::Solve(l,Q,p,y,alpha,Cp,Cn,eps,si,shrinking);\n\t}\nprivate:\n\tSolutionInfo *si;\n\tint select_working_set(int &i, int &j);\n\tdouble calculate_rho();\n\tbool be_shrunk(int i, double Gmax1, double Gmax2, double Gmax3, double Gmax4);\n\tvoid do_shrinking();\n};\n\n// return 1 if already optimal, return 0 otherwise\nint Solver_NU::select_working_set(int &out_i, int &out_j)\n{\n\t// return i,j such that y_i = y_j and\n\t// i: maximizes -y_i * grad(f)_i, i in I_up(\\alpha)\n\t// j: minimizes the decrease of obj value\n\t// (if quadratic coefficeint <= 0, replace it with tau)\n\t// -y_j*grad(f)_j < -y_i*grad(f)_i, j in I_low(\\alpha)\n\n\tdouble Gmaxp = -INF;\n\tdouble Gmaxp2 = -INF;\n\tint Gmaxp_idx = -1;\n\n\tdouble Gmaxn = -INF;\n\tdouble Gmaxn2 = -INF;\n\tint Gmaxn_idx = -1;\n\n\tint Gmin_idx = -1;\n\tdouble obj_diff_min = INF;\n\n\tfor(int t=0;t= Gmaxp)\n\t\t\t\t{\n\t\t\t\t\tGmaxp = -G[t];\n\t\t\t\t\tGmaxp_idx = t;\n\t\t\t\t}\n\t\t}\n\t\telse\n\t\t{\n\t\t\tif(!is_lower_bound(t))\n\t\t\t\tif(G[t] >= Gmaxn)\n\t\t\t\t{\n\t\t\t\t\tGmaxn = G[t];\n\t\t\t\t\tGmaxn_idx = t;\n\t\t\t\t}\n\t\t}\n\n\tint ip = Gmaxp_idx;\n\tint in = Gmaxn_idx;\n\tconst Qfloat *Q_ip = NULL;\n\tconst Qfloat *Q_in = NULL;\n\tif(ip != -1) // NULL Q_ip not accessed: Gmaxp=-INF if ip=-1\n\t\tQ_ip = Q->get_Q(ip,active_size);\n\tif(in != -1)\n\t\tQ_in = Q->get_Q(in,active_size);\n\n\tfor(int j=0;j= Gmaxp2)\n\t\t\t\t\tGmaxp2 = G[j];\n\t\t\t\tif (grad_diff > 0)\n\t\t\t\t{\n\t\t\t\t\tdouble obj_diff; \n\t\t\t\t\tdouble quad_coef = QD[ip]+QD[j]-2*Q_ip[j];\n\t\t\t\t\tif (quad_coef > 0)\n\t\t\t\t\t\tobj_diff = -(grad_diff*grad_diff)/quad_coef;\n\t\t\t\t\telse\n\t\t\t\t\t\tobj_diff = -(grad_diff*grad_diff)/TAU;\n\n\t\t\t\t\tif (obj_diff <= obj_diff_min)\n\t\t\t\t\t{\n\t\t\t\t\t\tGmin_idx=j;\n\t\t\t\t\t\tobj_diff_min = obj_diff;\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\telse\n\t\t{\n\t\t\tif (!is_upper_bound(j))\n\t\t\t{\n\t\t\t\tdouble grad_diff=Gmaxn-G[j];\n\t\t\t\tif (-G[j] >= Gmaxn2)\n\t\t\t\t\tGmaxn2 = -G[j];\n\t\t\t\tif (grad_diff > 0)\n\t\t\t\t{\n\t\t\t\t\tdouble obj_diff; \n\t\t\t\t\tdouble quad_coef = QD[in]+QD[j]-2*Q_in[j];\n\t\t\t\t\tif (quad_coef > 0)\n\t\t\t\t\t\tobj_diff = -(grad_diff*grad_diff)/quad_coef;\n\t\t\t\t\telse\n\t\t\t\t\t\tobj_diff = -(grad_diff*grad_diff)/TAU;\n\n\t\t\t\t\tif (obj_diff <= obj_diff_min)\n\t\t\t\t\t{\n\t\t\t\t\t\tGmin_idx=j;\n\t\t\t\t\t\tobj_diff_min = obj_diff;\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\tif(max(Gmaxp+Gmaxp2,Gmaxn+Gmaxn2) < eps)\n\t\treturn 1;\n\n\tif (y[Gmin_idx] == +1)\n\t\tout_i = Gmaxp_idx;\n\telse\n\t\tout_i = Gmaxn_idx;\n\tout_j = Gmin_idx;\n\n\treturn 0;\n}\n\nbool Solver_NU::be_shrunk(int i, double Gmax1, double Gmax2, double Gmax3, double Gmax4)\n{\n\tif(is_upper_bound(i))\n\t{\n\t\tif(y[i]==+1)\n\t\t\treturn(-G[i] > Gmax1);\n\t\telse\t\n\t\t\treturn(-G[i] > Gmax4);\n\t}\n\telse if(is_lower_bound(i))\n\t{\n\t\tif(y[i]==+1)\n\t\t\treturn(G[i] > Gmax2);\n\t\telse\t\n\t\t\treturn(G[i] > Gmax3);\n\t}\n\telse\n\t\treturn(false);\n}\n\nvoid Solver_NU::do_shrinking()\n{\n\tdouble Gmax1 = -INF;\t// max { -y_i * grad(f)_i | y_i = +1, i in I_up(\\alpha) }\n\tdouble Gmax2 = -INF;\t// max { y_i * grad(f)_i | y_i = +1, i in I_low(\\alpha) }\n\tdouble Gmax3 = -INF;\t// max { -y_i * grad(f)_i | y_i = -1, i in I_up(\\alpha) }\n\tdouble Gmax4 = -INF;\t// max { y_i * grad(f)_i | y_i = -1, i in I_low(\\alpha) }\n\n\t// find maximal violating pair first\n\tint i;\n\tfor(i=0;i Gmax1) Gmax1 = -G[i];\n\t\t\t}\n\t\t\telse\tif(-G[i] > Gmax4) Gmax4 = -G[i];\n\t\t}\n\t\tif(!is_lower_bound(i))\n\t\t{\n\t\t\tif(y[i]==+1)\n\t\t\t{\t\n\t\t\t\tif(G[i] > Gmax2) Gmax2 = G[i];\n\t\t\t}\n\t\t\telse\tif(G[i] > Gmax3) Gmax3 = G[i];\n\t\t}\n\t}\n\n\tif(unshrink == false && max(Gmax1+Gmax2,Gmax3+Gmax4) <= eps*10) \n\t{\n\t\tunshrink = true;\n\t\treconstruct_gradient();\n\t\tactive_size = l;\n\t}\n\n\tfor(i=0;i i)\n\t\t\t{\n\t\t\t\tif (!be_shrunk(active_size, Gmax1, Gmax2, Gmax3, Gmax4))\n\t\t\t\t{\n\t\t\t\t\tswap_index(i,active_size);\n\t\t\t\t\tbreak;\n\t\t\t\t}\n\t\t\t\tactive_size--;\n\t\t\t}\n\t\t}\n}\n\ndouble Solver_NU::calculate_rho()\n{\n\tint nr_free1 = 0,nr_free2 = 0;\n\tdouble ub1 = INF, ub2 = INF;\n\tdouble lb1 = -INF, lb2 = -INF;\n\tdouble sum_free1 = 0, sum_free2 = 0;\n\n\tfor(int i=0;i 0)\n\t\tr1 = sum_free1/nr_free1;\n\telse\n\t\tr1 = (ub1+lb1)/2;\n\t\n\tif(nr_free2 > 0)\n\t\tr2 = sum_free2/nr_free2;\n\telse\n\t\tr2 = (ub2+lb2)/2;\n\t\n\tsi->r = (r1+r2)/2;\n\treturn (r1-r2)/2;\n}\n\n//\n// Q matrices for various formulations\n//\nclass SVC_Q: public Kernel\n{ \npublic:\n\tSVC_Q(const svm_problem& prob, const svm_parameter& param, const schar *y_)\n\t:Kernel(prob.l, prob.x, param)\n\t{\n\t\tclone(y,y_,prob.l);\n\t\tcache = new Cache(prob.l,(long int)(param.cache_size*(1<<20)));\n\t\tQD = new double[prob.l];\n\t\tfor(int i=0;i*kernel_function)(i,i);\n\t}\n\t\n\tQfloat *get_Q(int i, int len) const\n\t{\n\t\tQfloat *data;\n\t\tint start, j;\n\t\tif((start = cache->get_data(i,&data,len)) < len)\n\t\t{\n\t\t\tfor(j=start;j*kernel_function)(i,j));\n\t\t}\n\t\treturn data;\n\t}\n\n\tdouble *get_QD() const\n\t{\n\t\treturn QD;\n\t}\n\n\tvoid swap_index(int i, int j) const\n\t{\n\t\tcache->swap_index(i,j);\n\t\tKernel::swap_index(i,j);\n\t\tswap(y[i],y[j]);\n\t\tswap(QD[i],QD[j]);\n\t}\n\n\t~SVC_Q()\n\t{\n\t\tdelete[] y;\n\t\tdelete cache;\n\t\tdelete[] QD;\n\t}\nprivate:\n\tschar *y;\n\tCache *cache;\n\tdouble *QD;\n};\n\nclass ONE_CLASS_Q: public Kernel\n{\npublic:\n\tONE_CLASS_Q(const svm_problem& prob, const svm_parameter& param)\n\t:Kernel(prob.l, prob.x, param)\n\t{\n\t\tcache = new Cache(prob.l,(long int)(param.cache_size*(1<<20)));\n\t\tQD = new double[prob.l];\n\t\tfor(int i=0;i*kernel_function)(i,i);\n\t}\n\t\n\tQfloat *get_Q(int i, int len) const\n\t{\n\t\tQfloat *data;\n\t\tint start, j;\n\t\tif((start = cache->get_data(i,&data,len)) < len)\n\t\t{\n\t\t\tfor(j=start;j*kernel_function)(i,j);\n\t\t}\n\t\treturn data;\n\t}\n\n\tdouble *get_QD() const\n\t{\n\t\treturn QD;\n\t}\n\n\tvoid swap_index(int i, int j) const\n\t{\n\t\tcache->swap_index(i,j);\n\t\tKernel::swap_index(i,j);\n\t\tswap(QD[i],QD[j]);\n\t}\n\n\t~ONE_CLASS_Q()\n\t{\n\t\tdelete cache;\n\t\tdelete[] QD;\n\t}\nprivate:\n\tCache *cache;\n\tdouble *QD;\n};\n\nclass SVR_Q: public Kernel\n{ \npublic:\n\tSVR_Q(const svm_problem& prob, const svm_parameter& param)\n\t:Kernel(prob.l, prob.x, param)\n\t{\n\t\tl = prob.l;\n\t\tcache = new Cache(l,(long int)(param.cache_size*(1<<20)));\n\t\tQD = new double[2*l];\n\t\tsign = new schar[2*l];\n\t\tindex = new int[2*l];\n\t\tfor(int k=0;k*kernel_function)(k,k);\n\t\t\tQD[k+l] = QD[k];\n\t\t}\n\t\tbuffer[0] = new Qfloat[2*l];\n\t\tbuffer[1] = new Qfloat[2*l];\n\t\tnext_buffer = 0;\n\t}\n\n\tvoid swap_index(int i, int j) const\n\t{\n\t\tswap(sign[i],sign[j]);\n\t\tswap(index[i],index[j]);\n\t\tswap(QD[i],QD[j]);\n\t}\n\t\n\tQfloat *get_Q(int i, int len) const\n\t{\n\t\tQfloat *data;\n\t\tint j, real_i = index[i];\n\t\tif(cache->get_data(real_i,&data,l) < l)\n\t\t{\n\t\t\tfor(j=0;j*kernel_function)(real_i,j);\n\t\t}\n\n\t\t// reorder and copy\n\t\tQfloat *buf = buffer[next_buffer];\n\t\tnext_buffer = 1 - next_buffer;\n\t\tschar si = sign[i];\n\t\tfor(j=0;jl;\n\tdouble *minus_ones = new double[l];\n\tschar *y = new schar[l];\n\n\tint i;\n\n\tfor(i=0;iy[i] > 0) y[i] = +1; else y[i] = -1;\n\t}\n\n\tSolver s;\n\ts.Solve(l, SVC_Q(*prob,*param,y), minus_ones, y,\n\t\talpha, Cp, Cn, param->eps, si, param->shrinking);\n\n\tdouble sum_alpha=0;\n\tfor(i=0;il));\n\n\tfor(i=0;il;\n\tdouble nu = param->nu;\n\n\tschar *y = new schar[l];\n\n\tfor(i=0;iy[i]>0)\n\t\t\ty[i] = +1;\n\t\telse\n\t\t\ty[i] = -1;\n\n\tdouble sum_pos = nu*l/2;\n\tdouble sum_neg = nu*l/2;\n\n\tfor(i=0;ieps, si, param->shrinking);\n\tdouble r = si->r;\n\n\tinfo(\"C = %f\\n\",1/r);\n\n\tfor(i=0;irho /= r;\n\tsi->obj /= (r*r);\n\tsi->upper_bound_p = 1/r;\n\tsi->upper_bound_n = 1/r;\n\n\tdelete[] y;\n\tdelete[] zeros;\n}\n\nstatic void solve_one_class(\n\tconst svm_problem *prob, const svm_parameter *param,\n\tdouble *alpha, Solver::SolutionInfo* si)\n{\n\tint l = prob->l;\n\tdouble *zeros = new double[l];\n\tschar *ones = new schar[l];\n\tint i;\n\n\tint n = (int)(param->nu*prob->l);\t// # of alpha's at upper bound\n\n\tfor(i=0;il)\n\t\talpha[n] = param->nu * prob->l - n;\n\tfor(i=n+1;ieps, si, param->shrinking);\n\n\tdelete[] zeros;\n\tdelete[] ones;\n}\n\nstatic void solve_epsilon_svr(\n\tconst svm_problem *prob, const svm_parameter *param,\n\tdouble *alpha, Solver::SolutionInfo* si)\n{\n\tint l = prob->l;\n\tdouble *alpha2 = new double[2*l];\n\tdouble *linear_term = new double[2*l];\n\tschar *y = new schar[2*l];\n\tint i;\n\n\tfor(i=0;ip - prob->y[i];\n\t\ty[i] = 1;\n\n\t\talpha2[i+l] = 0;\n\t\tlinear_term[i+l] = param->p + prob->y[i];\n\t\ty[i+l] = -1;\n\t}\n\n\tSolver s;\n\ts.Solve(2*l, SVR_Q(*prob,*param), linear_term, y,\n\t\talpha2, param->C, param->C, param->eps, si, param->shrinking);\n\n\tdouble sum_alpha = 0;\n\tfor(i=0;iC*l));\n\n\tdelete[] alpha2;\n\tdelete[] linear_term;\n\tdelete[] y;\n}\n\nstatic void solve_nu_svr(\n\tconst svm_problem *prob, const svm_parameter *param,\n\tdouble *alpha, Solver::SolutionInfo* si)\n{\n\tint l = prob->l;\n\tdouble C = param->C;\n\tdouble *alpha2 = new double[2*l];\n\tdouble *linear_term = new double[2*l];\n\tschar *y = new schar[2*l];\n\tint i;\n\n\tdouble sum = C * param->nu * l / 2;\n\tfor(i=0;iy[i];\n\t\ty[i] = 1;\n\n\t\tlinear_term[i+l] = prob->y[i];\n\t\ty[i+l] = -1;\n\t}\n\n\tSolver_NU s;\n\ts.Solve(2*l, SVR_Q(*prob,*param), linear_term, y,\n\t\talpha2, C, C, param->eps, si, param->shrinking);\n\n\tinfo(\"epsilon = %f\\n\",-si->r);\n\n\tfor(i=0;il);\n\tSolver::SolutionInfo si;\n\tswitch(param->svm_type)\n\t{\n\t\tcase C_SVC:\n\t\t\tsolve_c_svc(prob,param,alpha,&si,Cp,Cn);\n\t\t\tbreak;\n\t\tcase NU_SVC:\n\t\t\tsolve_nu_svc(prob,param,alpha,&si);\n\t\t\tbreak;\n\t\tcase ONE_CLASS:\n\t\t\tsolve_one_class(prob,param,alpha,&si);\n\t\t\tbreak;\n\t\tcase EPSILON_SVR:\n\t\t\tsolve_epsilon_svr(prob,param,alpha,&si);\n\t\t\tbreak;\n\t\tcase NU_SVR:\n\t\t\tsolve_nu_svr(prob,param,alpha,&si);\n\t\t\tbreak;\n\t}\n\n\tinfo(\"obj = %f, rho = %f\\n\",si.obj,si.rho);\n\n\t// output SVs\n\n\tint nSV = 0;\n\tint nBSV = 0;\n\tfor(int i=0;il;i++)\n\t{\n\t\tif(fabs(alpha[i]) > 0)\n\t\t{\n\t\t\t++nSV;\n\t\t\tif(prob->y[i] > 0)\n\t\t\t{\n\t\t\t\tif(fabs(alpha[i]) >= si.upper_bound_p)\n\t\t\t\t\t++nBSV;\n\t\t\t}\n\t\t\telse\n\t\t\t{\n\t\t\t\tif(fabs(alpha[i]) >= si.upper_bound_n)\n\t\t\t\t\t++nBSV;\n\t\t\t}\n\t\t}\n\t}\n\n\tinfo(\"nSV = %d, nBSV = %d\\n\",nSV,nBSV);\n\n\tdecision_function f;\n\tf.alpha = alpha;\n\tf.rho = si.rho;\n\treturn f;\n}\n\n// Platt's binary SVM Probablistic Output: an improvement from Lin et al.\nstatic void sigmoid_train(\n\tint l, const double *dec_values, const double *labels, \n\tdouble& A, double& B)\n{\n\tdouble prior1=0, prior0 = 0;\n\tint i;\n\n\tfor (i=0;i 0) prior1+=1;\n\t\telse prior0+=1;\n\t\n\tint max_iter=100;\t// Maximal number of iterations\n\tdouble min_step=1e-10;\t// Minimal step taken in line search\n\tdouble sigma=1e-12;\t// For numerically strict PD of Hessian\n\tdouble eps=1e-5;\n\tdouble hiTarget=(prior1+1.0)/(prior1+2.0);\n\tdouble loTarget=1/(prior0+2.0);\n\tdouble *t=Malloc(double,l);\n\tdouble fApB,p,q,h11,h22,h21,g1,g2,det,dA,dB,gd,stepsize;\n\tdouble newA,newB,newf,d1,d2;\n\tint iter; \n\t\n\t// Initial Point and Initial Fun Value\n\tA=0.0; B=log((prior0+1.0)/(prior1+1.0));\n\tdouble fval = 0.0;\n\n\tfor (i=0;i0) t[i]=hiTarget;\n\t\telse t[i]=loTarget;\n\t\tfApB = dec_values[i]*A+B;\n\t\tif (fApB>=0)\n\t\t\tfval += t[i]*fApB + log(1+exp(-fApB));\n\t\telse\n\t\t\tfval += (t[i] - 1)*fApB +log(1+exp(fApB));\n\t}\n\tfor (iter=0;iter= 0)\n\t\t\t{\n\t\t\t\tp=exp(-fApB)/(1.0+exp(-fApB));\n\t\t\t\tq=1.0/(1.0+exp(-fApB));\n\t\t\t}\n\t\t\telse\n\t\t\t{\n\t\t\t\tp=1.0/(1.0+exp(fApB));\n\t\t\t\tq=exp(fApB)/(1.0+exp(fApB));\n\t\t\t}\n\t\t\td2=p*q;\n\t\t\th11+=dec_values[i]*dec_values[i]*d2;\n\t\t\th22+=d2;\n\t\t\th21+=dec_values[i]*d2;\n\t\t\td1=t[i]-p;\n\t\t\tg1+=dec_values[i]*d1;\n\t\t\tg2+=d1;\n\t\t}\n\n\t\t// Stopping Criteria\n\t\tif (fabs(g1)= min_step)\n\t\t{\n\t\t\tnewA = A + stepsize * dA;\n\t\t\tnewB = B + stepsize * dB;\n\n\t\t\t// New function value\n\t\t\tnewf = 0.0;\n\t\t\tfor (i=0;i= 0)\n\t\t\t\t\tnewf += t[i]*fApB + log(1+exp(-fApB));\n\t\t\t\telse\n\t\t\t\t\tnewf += (t[i] - 1)*fApB +log(1+exp(fApB));\n\t\t\t}\n\t\t\t// Check sufficient decrease\n\t\t\tif (newf=max_iter)\n\t\tinfo(\"Reaching maximal iterations in two-class probability estimates\\n\");\n\tfree(t);\n}\n\nstatic double sigmoid_predict(double decision_value, double A, double B)\n{\n\tdouble fApB = decision_value*A+B;\n\t// 1-p used later; avoid catastrophic cancellation\n\tif (fApB >= 0)\n\t\treturn exp(-fApB)/(1.0+exp(-fApB));\n\telse\n\t\treturn 1.0/(1+exp(fApB)) ;\n}\n\n// Method 2 from the multiclass_prob paper by Wu, Lin, and Weng\nstatic void multiclass_probability(int k, double **r, double *p)\n{\n\tint t,j;\n\tint iter = 0, max_iter=max(100,k);\n\tdouble **Q=Malloc(double *,k);\n\tdouble *Qp=Malloc(double,k);\n\tdouble pQp, eps=0.005/k;\n\t\n\tfor (t=0;tmax_error)\n\t\t\t\tmax_error=error;\n\t\t}\n\t\tif (max_error=max_iter)\n\t\tinfo(\"Exceeds max_iter in multiclass_prob\\n\");\n\tfor(t=0;tl);\n\tdouble *dec_values = Malloc(double,prob->l);\n\n\t// random shuffle\n\tfor(i=0;il;i++) perm[i]=i;\n\tfor(i=0;il;i++)\n\t{\n\t\tint j = i+rand()%(prob->l-i);\n\t\tswap(perm[i],perm[j]);\n\t}\n\tfor(i=0;il/nr_fold;\n\t\tint end = (i+1)*prob->l/nr_fold;\n\t\tint j,k;\n\t\tstruct svm_problem subprob;\n\n\t\tsubprob.l = prob->l-(end-begin);\n#ifdef _DENSE_REP\n\t\tsubprob.x = Malloc(struct svm_node,subprob.l);\n#else\n\t\tsubprob.x = Malloc(struct svm_node*,subprob.l);\n#endif\n\t\tsubprob.y = Malloc(double,subprob.l);\n\t\t\t\n\t\tk=0;\n\t\tfor(j=0;jx[perm[j]];\n\t\t\tsubprob.y[k] = prob->y[perm[j]];\n\t\t\t++k;\n\t\t}\n\t\tfor(j=end;jl;j++)\n\t\t{\n\t\t\tsubprob.x[k] = prob->x[perm[j]];\n\t\t\tsubprob.y[k] = prob->y[perm[j]];\n\t\t\t++k;\n\t\t}\n\t\tint p_count=0,n_count=0;\n\t\tfor(j=0;j0)\n\t\t\t\tp_count++;\n\t\t\telse\n\t\t\t\tn_count++;\n\n\t\tif(p_count==0 && n_count==0)\n\t\t\tfor(j=begin;j 0 && n_count == 0)\n\t\t\tfor(j=begin;j 0)\n\t\t\tfor(j=begin;jx+perm[j]),&(dec_values[perm[j]])); \n#else\n\t\t\t\tsvm_predict_values(submodel,prob->x[perm[j]],&(dec_values[perm[j]])); \n#endif\n\t\t\t\t// ensure +1 -1 order; reason not using CV subroutine\n\t\t\t\tdec_values[perm[j]] *= submodel->label[0];\n\t\t\t}\t\t\n\t\t\tsvm_free_and_destroy_model(&submodel);\n\t\t\tsvm_destroy_param(&subparam);\n\t\t}\n\t\tfree(subprob.x);\n\t\tfree(subprob.y);\n\t}\t\t\n\tsigmoid_train(prob->l,dec_values,prob->y,probA,probB);\n\tfree(dec_values);\n\tfree(perm);\n}\n\n// Return parameter of a Laplace distribution \nstatic double svm_svr_probability(\n\tconst svm_problem *prob, const svm_parameter *param)\n{\n\tint i;\n\tint nr_fold = 5;\n\tdouble *ymv = Malloc(double,prob->l);\n\tdouble mae = 0;\n\n\tsvm_parameter newparam = *param;\n\tnewparam.probability = 0;\n\tsvm_cross_validation(prob,&newparam,nr_fold,ymv);\n\tfor(i=0;il;i++)\n\t{\n\t\tymv[i]=prob->y[i]-ymv[i];\n\t\tmae += fabs(ymv[i]);\n\t}\t\t\n\tmae /= prob->l;\n\tdouble std=sqrt(2*mae*mae);\n\tint count=0;\n\tmae=0;\n\tfor(i=0;il;i++)\n\t\tif (fabs(ymv[i]) > 5*std) \n\t\t\tcount=count+1;\n\t\telse \n\t\t\tmae+=fabs(ymv[i]);\n\tmae /= (prob->l-count);\n\tinfo(\"Prob. model for test data: target value = predicted value + z,\\nz: Laplace distribution e^(-|z|/sigma)/(2sigma),sigma= %g\\n\",mae);\n\tfree(ymv);\n\treturn mae;\n}\n\n\n// label: label name, start: begin of each class, count: #data of classes, perm: indices to the original data\n// perm, length l, must be allocated before calling this subroutine\nstatic void svm_group_classes(const svm_problem *prob, int *nr_class_ret, int **label_ret, int **start_ret, int **count_ret, int *perm)\n{\n\tint l = prob->l;\n\tint max_nr_class = 16;\n\tint nr_class = 0;\n\tint *label = Malloc(int,max_nr_class);\n\tint *count = Malloc(int,max_nr_class);\n\tint *data_label = Malloc(int,l);\t\n\tint i;\n\n\tfor(i=0;iy[i];\n\t\tint j;\n\t\tfor(j=0;jparam = *param;\n\tmodel->free_sv = 0;\t// XXX\n\n\tif(param->svm_type == ONE_CLASS ||\n\t param->svm_type == EPSILON_SVR ||\n\t param->svm_type == NU_SVR)\n\t{\n\t\t// regression or one-class-svm\n\t\tmodel->nr_class = 2;\n\t\tmodel->label = NULL;\n\t\tmodel->nSV = NULL;\n\t\tmodel->probA = NULL; model->probB = NULL;\n\t\tmodel->sv_coef = Malloc(double *,1);\n\n\t\tif(param->probability && \n\t\t (param->svm_type == EPSILON_SVR ||\n\t\t param->svm_type == NU_SVR))\n\t\t{\n\t\t\tmodel->probA = Malloc(double,1);\n\t\t\tmodel->probA[0] = svm_svr_probability(prob,param);\n\t\t}\n\n\t\tdecision_function f = svm_train_one(prob,param,0,0);\n\t\tmodel->rho = Malloc(double,1);\n\t\tmodel->rho[0] = f.rho;\n\n\t\tint nSV = 0;\n\t\tint i;\n\t\tfor(i=0;il;i++)\n\t\t\tif(fabs(f.alpha[i]) > 0) ++nSV;\n\t\tmodel->l = nSV;\n#ifdef _DENSE_REP\n\t\tmodel->SV = Malloc(svm_node,nSV);\n#else\n\t\tmodel->SV = Malloc(svm_node *,nSV);\n#endif\n\t\tmodel->sv_coef[0] = Malloc(double,nSV);\n\t\tmodel->sv_indices = Malloc(int,nSV);\n\t\tint j = 0;\n\t\tfor(i=0;il;i++)\n\t\t\tif(fabs(f.alpha[i]) > 0)\n\t\t\t{\n\t\t\t\tmodel->SV[j] = prob->x[i];\n\t\t\t\tmodel->sv_coef[0][j] = f.alpha[i];\n\t\t\t\tmodel->sv_indices[j] = i+1;\n\t\t\t\t++j;\n\t\t\t}\t\t\n\n\t\tfree(f.alpha);\n\t}\n\telse\n\t{\n\t\t// classification\n\t\tint l = prob->l;\n\t\tint nr_class;\n\t\tint *label = NULL;\n\t\tint *start = NULL;\n\t\tint *count = NULL;\n\t\tint *perm = Malloc(int,l);\n\n\t\t// group training data of the same class\n\t\tsvm_group_classes(prob,&nr_class,&label,&start,&count,perm);\t\t\n#ifdef _DENSE_REP\n\t\tsvm_node *x = Malloc(svm_node,l);\n#else\n\t\tsvm_node **x = Malloc(svm_node *,l);\n#endif\n\t\tint i;\n\t\tfor(i=0;ix[perm[i]];\n\n\t\t// calculate weighted C\n\n\t\tdouble *weighted_C = Malloc(double, nr_class);\n\t\tfor(i=0;iC;\n\t\tfor(i=0;inr_weight;i++)\n\t\t{\t\n\t\t\tint j;\n\t\t\tfor(j=0;jweight_label[i] == label[j])\n\t\t\t\t\tbreak;\n\t\t\tif(j == nr_class)\n\t\t\t\tfprintf(stderr,\"warning: class label %d specified in weight is not found\\n\", param->weight_label[i]);\n\t\t\telse\n\t\t\t\tweighted_C[j] *= param->weight[i];\n\t\t}\n\n\t\t// train k*(k-1)/2 models\n\t\t\n\t\tbool *nonzero = Malloc(bool,l);\n\t\tfor(i=0;iprobability)\n\t\t{\n\t\t\tprobA=Malloc(double,nr_class*(nr_class-1)/2);\n\t\t\tprobB=Malloc(double,nr_class*(nr_class-1)/2);\n\t\t}\n\n\t\tint p = 0;\n\t\tfor(i=0;iprobability)\n\t\t\t\t\tsvm_binary_svc_probability(&sub_prob,param,weighted_C[i],weighted_C[j],probA[p],probB[p]);\n\n\t\t\t\tf[p] = svm_train_one(&sub_prob,param,weighted_C[i],weighted_C[j]);\n\t\t\t\tfor(k=0;k 0)\n\t\t\t\t\t\tnonzero[si+k] = true;\n\t\t\t\tfor(k=0;k 0)\n\t\t\t\t\t\tnonzero[sj+k] = true;\n\t\t\t\tfree(sub_prob.x);\n\t\t\t\tfree(sub_prob.y);\n\t\t\t\t++p;\n\t\t\t}\n\n\t\t// build output\n\n\t\tmodel->nr_class = nr_class;\n\t\t\n\t\tmodel->label = Malloc(int,nr_class);\n\t\tfor(i=0;ilabel[i] = label[i];\n\t\t\n\t\tmodel->rho = Malloc(double,nr_class*(nr_class-1)/2);\n\t\tfor(i=0;irho[i] = f[i].rho;\n\n\t\tif(param->probability)\n\t\t{\n\t\t\tmodel->probA = Malloc(double,nr_class*(nr_class-1)/2);\n\t\t\tmodel->probB = Malloc(double,nr_class*(nr_class-1)/2);\n\t\t\tfor(i=0;iprobA[i] = probA[i];\n\t\t\t\tmodel->probB[i] = probB[i];\n\t\t\t}\n\t\t}\n\t\telse\n\t\t{\n\t\t\tmodel->probA=NULL;\n\t\t\tmodel->probB=NULL;\n\t\t}\n\n\t\tint total_sv = 0;\n\t\tint *nz_count = Malloc(int,nr_class);\n\t\tmodel->nSV = Malloc(int,nr_class);\n\t\tfor(i=0;inSV[i] = nSV;\n\t\t\tnz_count[i] = nSV;\n\t\t}\n\t\t\n\t\tinfo(\"Total nSV = %d\\n\",total_sv);\n\n\t\tmodel->l = total_sv;\n#ifdef _DENSE_REP\n\t\tmodel->SV = Malloc(svm_node,total_sv);\n#else\n\t\tmodel->SV = Malloc(svm_node *,total_sv);\n#endif\n\t\tmodel->sv_indices = Malloc(int,total_sv);\n\t\tp = 0;\n\t\tfor(i=0;iSV[p] = x[i];\n\t\t\t\tmodel->sv_indices[p++] = perm[i] + 1;\n\t\t\t}\n\t\t\t\n\t\tint *nz_start = Malloc(int,nr_class);\n\t\tnz_start[0] = 0;\n\t\tfor(i=1;isv_coef = Malloc(double *,nr_class-1);\n\t\tfor(i=0;isv_coef[i] = Malloc(double,total_sv);\n\n\t\tp = 0;\n\t\tfor(i=0;isv_coef[j-1][q++] = f[p].alpha[k];\n\t\t\t\tq = nz_start[j];\n\t\t\t\tfor(k=0;ksv_coef[i][q++] = f[p].alpha[ci+k];\n\t\t\t\t++p;\n\t\t\t}\n\t\t\n\t\tfree(label);\n\t\tfree(probA);\n\t\tfree(probB);\n\t\tfree(count);\n\t\tfree(perm);\n\t\tfree(start);\n\t\tfree(x);\n\t\tfree(weighted_C);\n\t\tfree(nonzero);\n\t\tfor(i=0;il;\n\tint *perm = Malloc(int,l);\n\tint nr_class;\n\tif (nr_fold > l)\n\t{\n\t\tnr_fold = l;\n\t\tfprintf(stderr,\"WARNING: # folds > # data. Will use # folds = # data instead (i.e., leave-one-out cross validation)\\n\");\n\t}\n\tfold_start = Malloc(int,nr_fold+1);\n\t// stratified cv may not give leave-one-out rate\n\t// Each class to l folds -> some folds may have zero elements\n\tif((param->svm_type == C_SVC ||\n\t param->svm_type == NU_SVC) && nr_fold < l)\n\t{\n\t\tint *start = NULL;\n\t\tint *label = NULL;\n\t\tint *count = NULL;\n\t\tsvm_group_classes(prob,&nr_class,&label,&start,&count,perm);\n\n\t\t// random shuffle and then data grouped by fold using the array perm\n\t\tint *fold_count = Malloc(int,nr_fold);\n\t\tint c;\n\t\tint *index = Malloc(int,l);\n\t\tfor(i=0;ix[perm[j]];\n\t\t\tsubprob.y[k] = prob->y[perm[j]];\n\t\t\t++k;\n\t\t}\n\t\tfor(j=end;jx[perm[j]];\n\t\t\tsubprob.y[k] = prob->y[perm[j]];\n\t\t\t++k;\n\t\t}\n\t\tstruct svm_model *submodel = svm_train(&subprob,param);\n\t\tif(param->probability && \n\t\t (param->svm_type == C_SVC || param->svm_type == NU_SVC))\n\t\t{\n\t\t\tdouble *prob_estimates=Malloc(double,svm_get_nr_class(submodel));\n\t\t\tfor(j=begin;jx + perm[j]),prob_estimates);\n#else\n\t\t\t\ttarget[perm[j]] = svm_predict_probability(submodel,prob->x[perm[j]],prob_estimates);\n#endif\n\t\t\tfree(prob_estimates);\t\t\t\n\t\t}\n\t\telse\n\t\t\tfor(j=begin;jx+perm[j]);\n#else\n\t\t\t\ttarget[perm[j]] = svm_predict(submodel,prob->x[perm[j]]);\n#endif\n\t\tsvm_free_and_destroy_model(&submodel);\n\t\tfree(subprob.x);\n\t\tfree(subprob.y);\n\t}\t\t\n\tfree(fold_start);\n\tfree(perm);\t\n}\n\n\nint svm_get_svm_type(const svm_model *model)\n{\n\treturn model->param.svm_type;\n}\n\nint svm_get_nr_class(const svm_model *model)\n{\n\treturn model->nr_class;\n}\n\nvoid svm_get_labels(const svm_model *model, int* label)\n{\n\tif (model->label != NULL)\n\t\tfor(int i=0;inr_class;i++)\n\t\t\tlabel[i] = model->label[i];\n}\n\nvoid svm_get_sv_indices(const svm_model *model, int* indices)\n{\n\tif (model->sv_indices != NULL)\n\t\tfor(int i=0;il;i++)\n\t\t\tindices[i] = model->sv_indices[i];\n}\n\nint svm_get_nr_sv(const svm_model *model)\n{\n\treturn model->l;\n}\n\ndouble svm_get_svr_probability(const svm_model *model)\n{\n\tif ((model->param.svm_type == EPSILON_SVR || model->param.svm_type == NU_SVR) &&\n\t model->probA!=NULL)\n\t\treturn model->probA[0];\n\telse\n\t{\n\t\tfprintf(stderr,\"Model doesn't contain information for SVR probability inference\\n\");\n\t\treturn 0;\n\t}\n}\n\ndouble svm_predict_values(const svm_model *model, const svm_node *x, double* dec_values)\n{\n\tint i;\n\tif(model->param.svm_type == ONE_CLASS ||\n\t model->param.svm_type == EPSILON_SVR ||\n\t model->param.svm_type == NU_SVR)\n\t{\n\t\tdouble *sv_coef = model->sv_coef[0];\n\t\tdouble sum = 0;\n\t\t\n\t\tfor(i=0;il;i++)\n#ifdef _DENSE_REP\n\t\t\tsum += sv_coef[i] * Kernel::k_function(x,model->SV+i,model->param);\n#else\n\t\t\tsum += sv_coef[i] * Kernel::k_function(x,model->SV[i],model->param);\n#endif\n\t\tsum -= model->rho[0];\n\t\t*dec_values = sum;\n\n\t\tif(model->param.svm_type == ONE_CLASS)\n\t\t\treturn (sum>0)?1:-1;\n\t\telse\n\t\t\treturn sum;\n\t}\n\telse\n\t{\n\t\tint i;\n\t\tint nr_class = model->nr_class;\n\t\tint l = model->l;\n\t\t\n\t\tdouble *kvalue = Malloc(double,l);\n\t\tfor(i=0;iSV+i,model->param);\n#else\n\t\t\tkvalue[i] = Kernel::k_function(x,model->SV[i],model->param);\n#endif\n\n\t\tint *start = Malloc(int,nr_class);\n\t\tstart[0] = 0;\n\t\tfor(i=1;inSV[i-1];\n\n\t\tint *vote = Malloc(int,nr_class);\n\t\tfor(i=0;inSV[i];\n\t\t\t\tint cj = model->nSV[j];\n\t\t\t\t\n\t\t\t\tint k;\n\t\t\t\tdouble *coef1 = model->sv_coef[j-1];\n\t\t\t\tdouble *coef2 = model->sv_coef[i];\n\t\t\t\tfor(k=0;krho[p];\n\t\t\t\tdec_values[p] = sum;\n\n\t\t\t\tif(dec_values[p] > 0)\n\t\t\t\t\t++vote[i];\n\t\t\t\telse\n\t\t\t\t\t++vote[j];\n\t\t\t\tp++;\n\t\t\t}\n\n\t\tint vote_max_idx = 0;\n\t\tfor(i=1;i vote[vote_max_idx])\n\t\t\t\tvote_max_idx = i;\n\n\t\tfree(kvalue);\n\t\tfree(start);\n\t\tfree(vote);\n\t\treturn model->label[vote_max_idx];\n\t}\n}\n\ndouble svm_predict(const svm_model *model, const svm_node *x)\n{\n\tint nr_class = model->nr_class;\n\tdouble *dec_values;\n\tif(model->param.svm_type == ONE_CLASS ||\n\t model->param.svm_type == EPSILON_SVR ||\n\t model->param.svm_type == NU_SVR)\n\t\tdec_values = Malloc(double, 1);\n\telse \n\t\tdec_values = Malloc(double, nr_class*(nr_class-1)/2);\n\tdouble pred_result = svm_predict_values(model, x, dec_values);\n\tfree(dec_values);\n\treturn pred_result;\n}\n\ndouble svm_predict_probability(\n\tconst svm_model *model, const svm_node *x, double *prob_estimates)\n{\n\tif ((model->param.svm_type == C_SVC || model->param.svm_type == NU_SVC) &&\n\t model->probA!=NULL && model->probB!=NULL)\n\t{\n\t\tint i;\n\t\tint nr_class = model->nr_class;\n\t\tdouble *dec_values = Malloc(double, nr_class*(nr_class-1)/2);\n\t\tsvm_predict_values(model, x, dec_values);\n\n\t\tdouble min_prob=1e-7;\n\t\tdouble **pairwise_prob=Malloc(double *,nr_class);\n\t\tfor(i=0;iprobA[k],model->probB[k]),min_prob),1-min_prob);\n\t\t\t\tpairwise_prob[j][i]=1-pairwise_prob[i][j];\n\t\t\t\tk++;\n\t\t\t}\n\t\tmulticlass_probability(nr_class,pairwise_prob,prob_estimates);\n\n\t\tint prob_max_idx = 0;\n\t\tfor(i=1;i prob_estimates[prob_max_idx])\n\t\t\t\tprob_max_idx = i;\n\t\tfor(i=0;ilabel[prob_max_idx];\n\t}\n\telse \n\t\treturn svm_predict(model, x);\n}\n\nstatic const char *svm_type_table[] =\n{\n\t\"c_svc\",\"nu_svc\",\"one_class\",\"epsilon_svr\",\"nu_svr\",NULL\n};\n\nstatic const char *kernel_type_table[]=\n{\n\t\"linear\",\"polynomial\",\"rbf\",\"sigmoid\",\"precomputed\",NULL\n};\n\nint svm_save_model(const char *model_file_name, const svm_model *model)\n{\n\tFILE *fp = fopen(model_file_name,\"w\");\n\tif(fp==NULL) return -1;\n\n\tchar *old_locale = strdup(setlocale(LC_ALL, NULL));\n\tsetlocale(LC_ALL, \"C\");\n\t\n\tconst svm_parameter& param = model->param;\n\n\tfprintf(fp,\"svm_type %s\\n\", svm_type_table[param.svm_type]);\n\tfprintf(fp,\"kernel_type %s\\n\", kernel_type_table[param.kernel_type]);\n\n\tif(param.kernel_type == POLY)\n\t\tfprintf(fp,\"degree %d\\n\", param.degree);\n\n\tif(param.kernel_type == POLY || param.kernel_type == RBF || param.kernel_type == SIGMOID)\n\t\tfprintf(fp,\"gamma %g\\n\", param.gamma);\n\n\tif(param.kernel_type == POLY || param.kernel_type == SIGMOID)\n\t\tfprintf(fp,\"coef0 %g\\n\", param.coef0);\n\n\tint nr_class = model->nr_class;\n\tint l = model->l;\n\tfprintf(fp, \"nr_class %d\\n\", nr_class);\n\tfprintf(fp, \"total_sv %d\\n\",l);\n\t\n\t{\n\t\tfprintf(fp, \"rho\");\n\t\tfor(int i=0;irho[i]);\n\t\tfprintf(fp, \"\\n\");\n\t}\n\t\n\tif(model->label)\n\t{\n\t\tfprintf(fp, \"label\");\n\t\tfor(int i=0;ilabel[i]);\n\t\tfprintf(fp, \"\\n\");\n\t}\n\n\tif(model->probA) // regression has probA only\n\t{\n\t\tfprintf(fp, \"probA\");\n\t\tfor(int i=0;iprobA[i]);\n\t\tfprintf(fp, \"\\n\");\n\t}\n\tif(model->probB)\n\t{\n\t\tfprintf(fp, \"probB\");\n\t\tfor(int i=0;iprobB[i]);\n\t\tfprintf(fp, \"\\n\");\n\t}\n\n\tif(model->nSV)\n\t{\n\t\tfprintf(fp, \"nr_sv\");\n\t\tfor(int i=0;inSV[i]);\n\t\tfprintf(fp, \"\\n\");\n\t}\n\n\tfprintf(fp, \"SV\\n\");\n\tconst double * const *sv_coef = model->sv_coef;\n#ifdef _DENSE_REP\n\tconst svm_node *SV = model->SV;\n#else\n\tconst svm_node * const *SV = model->SV;\n#endif\n\n\tfor(int i=0;ivalues[0]));\n\t\telse\n\t\t\tfor (int j = 0; j < p->dim; j++)\n\t\t\t\tif (p->values[j] != 0.0)\n\t\t\t\t\tfprintf(fp,\"%d:%.8g \",j, p->values[j]);\n#else\n\t\tconst svm_node *p = SV[i];\n\n\t\tif(param.kernel_type == PRECOMPUTED)\n\t\t\tfprintf(fp,\"0:%d \",(int)(p->value));\n\t\telse\n\t\t\twhile(p->index != -1)\n\t\t\t{\n\t\t\t\tfprintf(fp,\"%d:%.8g \",p->index,p->value);\n\t\t\t\tp++;\n\t\t\t}\n#endif\n\t\tfprintf(fp, \"\\n\");\n\t}\n\t\n\tsetlocale(LC_ALL, old_locale);\n\tfree(old_locale);\n\t\n\tif (ferror(fp) != 0 || fclose(fp) != 0) return -1;\n\telse return 0;\n}\n\nstatic char *line = NULL;\nstatic int max_line_len;\n\nstatic char* readline(FILE *input)\n{\n\tint len;\n\n\tif(fgets(line,max_line_len,input) == NULL)\n\t\treturn NULL;\n\n\twhile(strrchr(line,'\\n') == NULL)\n\t{\n\t\tmax_line_len *= 2;\n\t\tline = (char *) realloc(line,max_line_len);\n\t\tlen = (int) strlen(line);\n\t\tif(fgets(line+len,max_line_len-len,input) == NULL)\n\t\t\tbreak;\n\t}\n\treturn line;\n}\n\nsvm_model *svm_load_model(const char *model_file_name)\n{\n\tFILE *fp = fopen(model_file_name,\"rb\");\n\tif(fp==NULL) return NULL;\n\t\n\tchar *old_locale = strdup(setlocale(LC_ALL, NULL));\n\tsetlocale(LC_ALL, \"C\");\n\t\n\t// read parameters\n\n\tsvm_model *model = Malloc(svm_model,1);\n\tsvm_parameter& param = model->param;\n\tmodel->rho = NULL;\n\tmodel->probA = NULL;\n\tmodel->probB = NULL;\n\tmodel->sv_indices = NULL;\n\tmodel->label = NULL;\n\tmodel->nSV = NULL;\n\n\tchar cmd[81];\n\twhile(1)\n\t{\n\t\tfscanf(fp,\"%80s\",cmd);\n\n\t\tif(strcmp(cmd,\"svm_type\")==0)\n\t\t{\n\t\t\tfscanf(fp,\"%80s\",cmd);\n\t\t\tint i;\n\t\t\tfor(i=0;svm_type_table[i];i++)\n\t\t\t{\n\t\t\t\tif(strcmp(svm_type_table[i],cmd)==0)\n\t\t\t\t{\n\t\t\t\t\tparam.svm_type=i;\n\t\t\t\t\tbreak;\n\t\t\t\t}\n\t\t\t}\n\t\t\tif(svm_type_table[i] == NULL)\n\t\t\t{\n\t\t\t\tfprintf(stderr,\"unknown svm type.\\n\");\n\t\t\t\t\n\t\t\t\tsetlocale(LC_ALL, old_locale);\n\t\t\t\tfree(old_locale);\n\t\t\t\tfree(model->rho);\n\t\t\t\tfree(model->label);\n\t\t\t\tfree(model->nSV);\n\t\t\t\tfree(model);\n\t\t\t\treturn NULL;\n\t\t\t}\n\t\t}\n\t\telse if(strcmp(cmd,\"kernel_type\")==0)\n\t\t{\t\t\n\t\t\tfscanf(fp,\"%80s\",cmd);\n\t\t\tint i;\n\t\t\tfor(i=0;kernel_type_table[i];i++)\n\t\t\t{\n\t\t\t\tif(strcmp(kernel_type_table[i],cmd)==0)\n\t\t\t\t{\n\t\t\t\t\tparam.kernel_type=i;\n\t\t\t\t\tbreak;\n\t\t\t\t}\n\t\t\t}\n\t\t\tif(kernel_type_table[i] == NULL)\n\t\t\t{\n\t\t\t\tfprintf(stderr,\"unknown kernel function.\\n\");\n\t\t\t\t\n\t\t\t\tsetlocale(LC_ALL, old_locale);\n\t\t\t\tfree(old_locale);\n\t\t\t\tfree(model->rho);\n\t\t\t\tfree(model->label);\n\t\t\t\tfree(model->nSV);\n\t\t\t\tfree(model);\n\t\t\t\treturn NULL;\n\t\t\t}\n\t\t}\n\t\telse if(strcmp(cmd,\"degree\")==0)\n\t\t\tfscanf(fp,\"%d\",¶m.degree);\n\t\telse if(strcmp(cmd,\"gamma\")==0)\n\t\t\tfscanf(fp,\"%lf\",¶m.gamma);\n\t\telse if(strcmp(cmd,\"coef0\")==0)\n\t\t\tfscanf(fp,\"%lf\",¶m.coef0);\n\t\telse if(strcmp(cmd,\"nr_class\")==0)\n\t\t\tfscanf(fp,\"%d\",&model->nr_class);\n\t\telse if(strcmp(cmd,\"total_sv\")==0)\n\t\t\tfscanf(fp,\"%d\",&model->l);\n\t\telse if(strcmp(cmd,\"rho\")==0)\n\t\t{\n\t\t\tint n = model->nr_class * (model->nr_class-1)/2;\n\t\t\tmodel->rho = Malloc(double,n);\n\t\t\tfor(int i=0;irho[i]);\n\t\t}\n\t\telse if(strcmp(cmd,\"label\")==0)\n\t\t{\n\t\t\tint n = model->nr_class;\n\t\t\tmodel->label = Malloc(int,n);\n\t\t\tfor(int i=0;ilabel[i]);\n\t\t}\n\t\telse if(strcmp(cmd,\"probA\")==0)\n\t\t{\n\t\t\tint n = model->nr_class * (model->nr_class-1)/2;\n\t\t\tmodel->probA = Malloc(double,n);\n\t\t\tfor(int i=0;iprobA[i]);\n\t\t}\n\t\telse if(strcmp(cmd,\"probB\")==0)\n\t\t{\n\t\t\tint n = model->nr_class * (model->nr_class-1)/2;\n\t\t\tmodel->probB = Malloc(double,n);\n\t\t\tfor(int i=0;iprobB[i]);\n\t\t}\n\t\telse if(strcmp(cmd,\"nr_sv\")==0)\n\t\t{\n\t\t\tint n = model->nr_class;\n\t\t\tmodel->nSV = Malloc(int,n);\n\t\t\tfor(int i=0;inSV[i]);\n\t\t}\n\t\telse if(strcmp(cmd,\"SV\")==0)\n\t\t{\n\t\t\twhile(1)\n\t\t\t{\n\t\t\t\tint c = getc(fp);\n\t\t\t\tif(c==EOF || c=='\\n') break;\t\n\t\t\t}\n\t\t\tbreak;\n\t\t}\n\t\telse\n\t\t{\n\t\t\tfprintf(stderr,\"unknown text in model file: [%s]\\n\",cmd);\n\t\t\t\n\t\t\tsetlocale(LC_ALL, old_locale);\n\t\t\tfree(old_locale);\n\t\t\tfree(model->rho);\n\t\t\tfree(model->label);\n\t\t\tfree(model->nSV);\n\t\t\tfree(model);\n\t\t\treturn NULL;\n\t\t}\n\t}\n\n\t// read sv_coef and SV\n\n\tint elements = 0;\n\tlong pos = ftell(fp);\n\n\tmax_line_len = 1024;\n\tline = Malloc(char,max_line_len);\n\tchar *p,*endptr,*idx,*val;\n\n#ifdef _DENSE_REP\n\tint max_index = 1;\n\t// read the max dimension of all vectors\n\twhile(readline(fp) != NULL)\n\t{\n\t\tchar *p;\n\t\tp = strrchr(line, ':');\n\t\tif(p != NULL)\n\t\t{\t\t\t\n\t\t\twhile(*p != ' ' && *p != '\\t' && p > line)\n\t\t\t\tp--;\n\t\t\tif(p > line)\n\t\t\t\tmax_index = (int) strtol(p,&endptr,10) + 1;\n\t\t}\t\t\n\t\tif(max_index > elements)\n\t\t\telements = max_index;\n\t}\n#else\n\twhile(readline(fp)!=NULL)\n\t{\n\t\tp = strtok(line,\":\");\n\t\twhile(1)\n\t\t{\n\t\t\tp = strtok(NULL,\":\");\n\t\t\tif(p == NULL)\n\t\t\t\tbreak;\n\t\t\t++elements;\n\t\t}\n\t}\n\telements += model->l;\n\n#endif\n\tfseek(fp,pos,SEEK_SET);\n\n\tint m = model->nr_class - 1;\n\tint l = model->l;\n\tmodel->sv_coef = Malloc(double *,m);\n\tint i;\n\tfor(i=0;isv_coef[i] = Malloc(double,l);\n\n#ifdef _DENSE_REP\n\tint index;\n\tmodel->SV = Malloc(svm_node,l);\n\n\tfor(i=0;iSV[i].values = Malloc(double, elements);\n\t\tmodel->SV[i].dim = 0;\n\n\t\tp = strtok(line, \" \\t\");\n\t\tmodel->sv_coef[0][i] = strtod(p,&endptr);\n\t\tfor(int k=1;ksv_coef[k][i] = strtod(p,&endptr);\n\t\t}\n\n\t\tint *d = &(model->SV[i].dim);\n\t\twhile(1)\n\t\t{\n\t\t\tidx = strtok(NULL, \":\");\n\t\t\tval = strtok(NULL, \" \\t\");\n\n\t\t\tif(val == NULL)\n\t\t\t\tbreak;\n\t\t\tindex = (int) strtol(idx,&endptr,10);\n\t\t\twhile (*d < index)\n\t\t\t\tmodel->SV[i].values[(*d)++] = 0.0;\n\t\t\tmodel->SV[i].values[(*d)++] = strtod(val,&endptr);\n\t\t}\n\t}\n#else\n\tmodel->SV = Malloc(svm_node*,l);\n\tsvm_node *x_space = NULL;\n\tif(l>0) x_space = Malloc(svm_node,elements);\n\tint j=0;\n\tfor(i=0;iSV[i] = &x_space[j];\n\n\t\tp = strtok(line, \" \\t\");\n\t\tmodel->sv_coef[0][i] = strtod(p,&endptr);\n\t\tfor(int k=1;ksv_coef[k][i] = strtod(p,&endptr);\n\t\t}\n\n\t\twhile(1)\n\t\t{\n\t\t\tidx = strtok(NULL, \":\");\n\t\t\tval = strtok(NULL, \" \\t\");\n\n\t\t\tif(val == NULL)\n\t\t\t\tbreak;\n\t\t\tx_space[j].index = (int) strtol(idx,&endptr,10);\n\t\t\tx_space[j].value = strtod(val,&endptr);\n\n\t\t\t++j;\n\t\t}\n\t\tx_space[j++].index = -1;\n\t}\n#endif\n\tfree(line);\n\n\tsetlocale(LC_ALL, old_locale);\n\tfree(old_locale);\n\t\n\tif (ferror(fp) != 0 || fclose(fp) != 0)\n\t\treturn NULL;\n\n\tmodel->free_sv = 1;\t// XXX\n\treturn model;\n}\n\nvoid svm_free_model_content(svm_model* model_ptr)\n{\n\tif(model_ptr->free_sv && model_ptr->l > 0 && model_ptr->SV != NULL)\n#ifdef _DENSE_REP\n\tfor (int i = 0; i < model_ptr->l; i++)\n\t\tfree (model_ptr->SV[i].values);\n#else\n\t\tfree((void *)(model_ptr->SV[0]));\n#endif\n\tif(model_ptr->sv_coef)\n\t{\n\t\tfor(int i=0;inr_class-1;i++)\n\t\t\tfree(model_ptr->sv_coef[i]);\n\t}\n\tfree(model_ptr->SV);\n\tmodel_ptr->SV = NULL;\n\n\tfree(model_ptr->sv_coef);\n\tmodel_ptr->sv_coef = NULL;\n\n\tfree(model_ptr->rho);\n\tmodel_ptr->rho = NULL;\n\n\tfree(model_ptr->label);\n\tmodel_ptr->label= NULL;\n\n\tfree(model_ptr->probA);\n\tmodel_ptr->probA = NULL;\n\n\tfree(model_ptr->probB);\n\tmodel_ptr->probB= NULL;\n\n\tfree(model_ptr->sv_indices);\n\tmodel_ptr->sv_indices = NULL;\n\n\tfree(model_ptr->nSV);\n\tmodel_ptr->nSV = NULL;\n}\n\nvoid svm_free_and_destroy_model(svm_model** model_ptr_ptr)\n{\n\tif(model_ptr_ptr != NULL && *model_ptr_ptr != NULL)\n\t{\n\t\tsvm_free_model_content(*model_ptr_ptr);\n\t\tfree(*model_ptr_ptr);\n\t\t*model_ptr_ptr = NULL;\n\t}\n}\n\nvoid svm_destroy_param(svm_parameter* param)\n{\n\tfree(param->weight_label);\n\tfree(param->weight);\n}\n\nconst char *svm_check_parameter(const svm_problem *prob, const svm_parameter *param)\n{\n\t// svm_type\n\n\tint svm_type = param->svm_type;\n\tif(svm_type != C_SVC &&\n\t svm_type != NU_SVC &&\n\t svm_type != ONE_CLASS &&\n\t svm_type != EPSILON_SVR &&\n\t svm_type != NU_SVR)\n\t\treturn \"unknown svm type\";\n\t\n\t// kernel_type, degree\n\t\n\tint kernel_type = param->kernel_type;\n\tif(kernel_type != LINEAR &&\n\t kernel_type != POLY &&\n\t kernel_type != RBF &&\n\t kernel_type != SIGMOID &&\n\t kernel_type != PRECOMPUTED)\n\t\treturn \"unknown kernel type\";\n\n\tif(param->gamma < 0)\n\t\treturn \"gamma < 0\";\n\n\tif(param->degree < 0)\n\t\treturn \"degree of polynomial kernel < 0\";\n\n\t// cache_size,eps,C,nu,p,shrinking\n\n\tif(param->cache_size <= 0)\n\t\treturn \"cache_size <= 0\";\n\n\tif(param->eps <= 0)\n\t\treturn \"eps <= 0\";\n\n\tif(svm_type == C_SVC ||\n\t svm_type == EPSILON_SVR ||\n\t svm_type == NU_SVR)\n\t\tif(param->C <= 0)\n\t\t\treturn \"C <= 0\";\n\n\tif(svm_type == NU_SVC ||\n\t svm_type == ONE_CLASS ||\n\t svm_type == NU_SVR)\n\t\tif(param->nu <= 0 || param->nu > 1)\n\t\t\treturn \"nu <= 0 or nu > 1\";\n\n\tif(svm_type == EPSILON_SVR)\n\t\tif(param->p < 0)\n\t\t\treturn \"p < 0\";\n\n\tif(param->shrinking != 0 &&\n\t param->shrinking != 1)\n\t\treturn \"shrinking != 0 and shrinking != 1\";\n\n\tif(param->probability != 0 &&\n\t param->probability != 1)\n\t\treturn \"probability != 0 and probability != 1\";\n\n\tif(param->probability == 1 &&\n\t svm_type == ONE_CLASS)\n\t\treturn \"one-class SVM probability output not supported yet\";\n\n\n\t// check whether nu-svc is feasible\n\t\n\tif(svm_type == NU_SVC)\n\t{\n\t\tint l = prob->l;\n\t\tint max_nr_class = 16;\n\t\tint nr_class = 0;\n\t\tint *label = Malloc(int,max_nr_class);\n\t\tint *count = Malloc(int,max_nr_class);\n\n\t\tint i;\n\t\tfor(i=0;iy[i];\n\t\t\tint j;\n\t\t\tfor(j=0;jnu*(n1+n2)/2 > min(n1,n2))\n\t\t\t\t{\n\t\t\t\t\tfree(label);\n\t\t\t\t\tfree(count);\n\t\t\t\t\treturn \"specified nu is infeasible\";\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\tfree(label);\n\t\tfree(count);\n\t}\n\n\treturn NULL;\n}\n\nint svm_check_probability_model(const svm_model *model)\n{\n\treturn ((model->param.svm_type == C_SVC || model->param.svm_type == NU_SVC) &&\n\t\tmodel->probA!=NULL && model->probB!=NULL) ||\n\t\t((model->param.svm_type == EPSILON_SVR || model->param.svm_type == NU_SVR) &&\n\t\t model->probA!=NULL);\n}\n\nvoid svm_set_print_string_function(void (*print_func)(const char *))\n{\n\tif(print_func == NULL)\n\t\tsvm_print_string = &print_string_stdout;\n\telse\n\t\tsvm_print_string = print_func;\n}\n"
},
{
"path": "src/linux/svm.h",
"content": "#ifndef _LIBSVM_H\n#define _LIBSVM_H\n#define _DENSE_REP\n#define LIBSVM_VERSION 317\n\n#ifdef __cplusplus\nextern \"C\" {\n#endif\n\nextern int libsvm_version;\n\n#ifdef _DENSE_REP\nstruct svm_node\n{\n\tint dim;\n\tdouble *values;\n};\n\nstruct svm_problem\n{\n\tint l;\n\tdouble *y;\n\tstruct svm_node *x;\n};\n\n#else\nstruct svm_node\n{\n\tint index;\n\tdouble value;\n};\n\nstruct svm_problem\n{\n\tint l;\n\tdouble *y;\n\tstruct svm_node **x;\n};\n#endif\n\nenum { C_SVC, NU_SVC, ONE_CLASS, EPSILON_SVR, NU_SVR };\t/* svm_type */\nenum { LINEAR, POLY, RBF, SIGMOID, PRECOMPUTED }; /* kernel_type */\n\nstruct svm_parameter\n{\n\tint svm_type;\n\tint kernel_type;\n\tint degree;\t/* for poly */\n\tdouble gamma;\t/* for poly/rbf/sigmoid */\n\tdouble coef0;\t/* for poly/sigmoid */\n\n\t/* these are for training only */\n\tdouble cache_size; /* in MB */\n\tdouble eps;\t/* stopping criteria */\n\tdouble C;\t/* for C_SVC, EPSILON_SVR and NU_SVR */\n\tint nr_weight;\t\t/* for C_SVC */\n\tint *weight_label;\t/* for C_SVC */\n\tdouble* weight;\t\t/* for C_SVC */\n\tdouble nu;\t/* for NU_SVC, ONE_CLASS, and NU_SVR */\n\tdouble p;\t/* for EPSILON_SVR */\n\tint shrinking;\t/* use the shrinking heuristics */\n\tint probability; /* do probability estimates */\n};\n\n//\n// svm_model\n// \nstruct svm_model\n{\n\tstruct svm_parameter param;\t/* parameter */\n\tint nr_class;\t\t/* number of classes, = 2 in regression/one class svm */\n\tint l;\t\t\t/* total #SV */\n#ifdef _DENSE_REP\n\tstruct svm_node *SV;\t\t/* SVs (SV[l]) */\n#else\n\tstruct svm_node **SV;\t\t/* SVs (SV[l]) */\n#endif\n\tdouble **sv_coef;\t/* coefficients for SVs in decision functions (sv_coef[k-1][l]) */\n\tdouble *rho;\t\t/* constants in decision functions (rho[k*(k-1)/2]) */\n\tdouble *probA;\t\t/* pariwise probability information */\n\tdouble *probB;\n\tint *sv_indices; /* sv_indices[0,...,nSV-1] are values in [1,...,num_traning_data] to indicate SVs in the training set */\n\t\n\t/* for classification only */\n\n\tint *label;\t\t/* label of each class (label[k]) */\n\tint *nSV;\t\t/* number of SVs for each class (nSV[k]) */\n\t\t\t\t/* nSV[0] + nSV[1] + ... + nSV[k-1] = l */\n\t/* XXX */\n\tint free_sv;\t\t/* 1 if svm_model is created by svm_load_model*/\n\t\t\t\t/* 0 if svm_model is created by svm_train */\n};\n\nstruct svm_model *svm_train(const struct svm_problem *prob, const struct svm_parameter *param);\nvoid svm_cross_validation(const struct svm_problem *prob, const struct svm_parameter *param, int nr_fold, double *target);\n\nint svm_save_model(const char *model_file_name, const struct svm_model *model);\nstruct svm_model *svm_load_model(const char *model_file_name);\n\nint svm_get_svm_type(const struct svm_model *model);\nint svm_get_nr_class(const struct svm_model *model);\nvoid svm_get_labels(const struct svm_model *model, int *label);\nvoid svm_get_sv_indices(const struct svm_model *model, int *sv_indices);\nint svm_get_nr_sv(const struct svm_model *model);\ndouble svm_get_svr_probability(const struct svm_model *model);\n\ndouble svm_predict_values(const struct svm_model *model, const struct svm_node *x, double* dec_values);\ndouble svm_predict(const struct svm_model *model, const struct svm_node *x);\ndouble svm_predict_probability(const struct svm_model *model, const struct svm_node *x, double* prob_estimates);\n\nvoid svm_free_model_content(struct svm_model *model_ptr);\nvoid svm_free_and_destroy_model(struct svm_model **model_ptr_ptr);\nvoid svm_destroy_param(struct svm_parameter *param);\n\nconst char *svm_check_parameter(const struct svm_problem *prob, const struct svm_parameter *param);\nint svm_check_probability_model(const struct svm_model *model);\n\nvoid svm_set_print_string_function(void (*print_func)(const char *));\n\n#ifdef __cplusplus\n}\n#endif\n\n#endif /* _LIBSVM_H */\n"
},
{
"path": "src/windows/README-GPU",
"content": "GPU-Accelerated LIBSVM is exploiting the GPU, using the CUDA interface, to\nspeed-up the training process. This package contains a new executable for \ntraining classifiers \"svm-train-gpu.exe\" together with the original one.\nThe use of the new executable is exactly the same as with the original one.\n\nThis executable was built with the CUBLAS API version 2 which is compatible with SDKs from 4.0 and up.\n\nFEATURES\n\nMode Supported\n\n * c-svc classification with RBF kernel\n\nFunctionality / User interface\n\n * Same as LIBSVM\n\n\nPREREQUISITES\n\n * NVIDIA Graphics card with CUDA support\n * Latest NVIDIA drivers for GPU\n\t* CUDA toolkit & GPU Computing SDK 5.5 \n\n Download all in one package from: \n\thttps://developer.nvidia.com/cuda-downloads\n\n\t\nINSTRUCTIONS\n\n\t1. Install the NVIDIA drivers, CUDA toolkit and GPU Computing SDK code samples. You can find them all in one package here: \n\n\thttps://developer.nvidia.com/cuda-downloads (Version 5.5)\n\n\t2. Open the Visual studio 2010 project file located inside this folder and build.\n\t\n\t\nAdditional Information\n======================\n\nIf you find GPU-Accelerated LIBSVM helpful, please cite it as\n\nA. Athanasopoulos, A. Dimou, V. Mezaris, I. Kompatsiaris, \"GPU Acceleration for Support Vector Machines\",\nProc. 12th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS 2011), Delft, The Netherlands, April 2011.\n\nSoftware available at http://mklab.iti.gr/project/GPU-LIBSVM\n"
},
{
"path": "src/windows/libsvm_train_dense_gpu/cross_validation_with_matrix_precomputation.c",
"content": "void setup_pkm(struct svm_problem *p_km)\n{\n\n\tint i;\n\n\tp_km->l = prob.l;\n\tp_km->x = Malloc(struct svm_node,p_km->l);\n\tp_km->y = Malloc(double,p_km->l);\n\n\tfor(i=0;ix+i)->values = Malloc(double,prob.l+1);\n\t\t(p_km->x+i)->dim = prob.l+1;\n\t}\n\n\tfor( i=0; iy[i] = prob.y[i];\n}\n\nvoid free_pkm(struct svm_problem *p_km)\n{\n\n\tint i;\n\n\tfor(i=0;ix+i)->values);\n\n\tfree( p_km->x );\n\tfree( p_km->y );\n\n}\n\n\ndouble do_crossvalidation(struct svm_problem * p_km)\n{\n\t\t\tdouble rate;\n\n\t\t\tint i;\n\t\t\tint total_correct = 0;\n\t\t\tdouble total_error = 0;\n\t\t\tdouble sumv = 0, sumy = 0, sumvv = 0, sumyy = 0, sumvy = 0;\n\t\t\tdouble *target = Malloc(double,prob.l);\n\n\t\t\tsvm_cross_validation(p_km,¶m,nr_fold,target);\n\n\n\t\t\tif(param.svm_type == EPSILON_SVR ||\n\t\t\t\tparam.svm_type == NU_SVR)\n\t\t\t{\n\t\t\t\tfor(i=0;i\n#include \"cublas_v2.h\"\n\n// Scalars\nconst float alpha = 1;\nconst float beta = 0;\n\nvoid ckm( struct svm_problem *prob, struct svm_problem *pecm, float *gamma )\n{\n\tcublasStatus_t status;\n\n\tdouble g_val = *gamma;\n\n\tlong int nfa;\n\t\n\tint len_tv;\n\tint ntv;\n\tint i_v;\n\tint i_el;\n\tint i_r, i_c;\n\tint trvei;\n\n\tdouble *tv_sq;\n\tdouble *v_f_g;\n\n\tfloat *tr_ar;\n\tfloat *tva, *vtm, *DP;\n\tfloat *g_tva = 0, *g_vtm = 0, *g_DotProd = 0;\n\n\tcudaError_t cudaStat; \n\tcublasHandle_t handle;\n\t\n\tstatus = cublasCreate(&handle);\n\n\tlen_tv = prob-> x[0].dim;\n\tntv = prob-> l;\n\n\tnfa = len_tv * ntv; \n\n\ttva = (float*) malloc ( len_tv * ntv* sizeof(float) );\n\tvtm = (float*) malloc ( len_tv * sizeof(float) );\n\tDP = (float*) malloc ( ntv * sizeof(float) );\n\n\ttr_ar = (float*) malloc ( len_tv * ntv* sizeof(float) );\n\n\ttv_sq = (double*) malloc ( ntv * sizeof(double) );\n\n\tv_f_g = (double*) malloc ( ntv * sizeof(double) );\n\n\tfor ( i_r = 0; i_r < ntv ; i_r++ )\n\t{\t\t\t\t \n\t\tfor ( i_c = 0; i_c < len_tv; i_c++ ) \n\t\t\ttva[i_r * len_tv + i_c] = (float)prob-> x[i_r].values[i_c];\n\t}\n\n\tcudaStat = cudaMalloc((void**)&g_tva, len_tv * ntv * sizeof(float));\n\t\n\tif (cudaStat != cudaSuccess) {\n\t\tfree( tva );\n\t\tfree( vtm );\n\t\tfree( DP );\n\n\t\tfree( v_f_g );\n\t\tfree( tv_sq );\n\n\t\tcudaFree( g_tva );\n\t\tcublasDestroy( handle );\t\n\t\n\t\tfprintf (stderr, \"!!!! Device memory allocation error (A)\\n\");\n\t\tgetchar();\n\t\treturn;\n }\n\n\tcudaStat = cudaMalloc((void**)&g_vtm, len_tv * sizeof(float));\n\n\tcudaStat = cudaMalloc((void**)&g_DotProd, ntv * sizeof(float));\n\n\tfor( i_r = 0; i_r < ntv; i_r++ )\n\t\tfor( i_c = 0; i_c < len_tv; i_c++ )\n\t\t\ttr_ar[i_c * ntv + i_r] = tva[i_r * len_tv + i_c];\n\n\t// Copy cpu vector to gpu vector\n\tstatus = cublasSetVector( len_tv * ntv, sizeof(float), tr_ar, 1, g_tva, 1 );\n \n\tfree( tr_ar );\n\n\tfor( i_v = 0; i_v < ntv; i_v++ )\n\t{\n\t\ttv_sq[ i_v ] = 0;\n\t\tfor( i_el = 0; i_el < len_tv; i_el++ )\n\t\t\ttv_sq[i_v] += pow( tva[i_v*len_tv + i_el], (float)2.0 );\n\t}\n\n\n\n\tfor ( trvei = 0; trvei < ntv; trvei++ )\n\t{\n\t\tstatus = cublasSetVector( len_tv, sizeof(float), &tva[trvei * len_tv], 1, g_vtm, 1 );\n\t\t\n\t\tstatus = cublasSgemv( handle, CUBLAS_OP_N, ntv, len_tv, &alpha, g_tva, ntv , g_vtm, 1, &beta, g_DotProd, 1 );\n\n\t\tstatus = cublasGetVector( ntv, sizeof(float), g_DotProd, 1, DP, 1 );\n\n\t\tfor ( i_c = 0; i_c < ntv; i_c++ )\n\t\t\tv_f_g[i_c] = exp( -g_val * (tv_sq[trvei] + tv_sq[i_c]-((double)2.0)* (double)DP[i_c] ));\n\t\t\n\n\t\tpecm-> x[trvei].values[0] = trvei + 1;\n\t\t\n\t\tfor ( i_c = 0; i_c < ntv; i_c++ )\n\t\t\tpecm-> x[trvei].values[i_c + 1] = v_f_g[i_c];\t\t\t\t\n\t\t\n\n\t}\n\n\tfree( tva );\n\tfree( vtm );\n\tfree( DP );\n\tfree( v_f_g );\n\tfree( tv_sq );\n\n\tcudaFree( g_tva );\n\tcudaFree( g_vtm );\n\tcudaFree( g_DotProd );\n\n\tcublasDestroy( handle );\n}\n\nvoid cal_km( struct svm_problem * p_km)\n{\n\tfloat gamma = param.gamma;\n\n\tckm(&prob, p_km, &gamma);\n}"
},
{
"path": "src/windows/libsvm_train_dense_gpu/libsvm_train_dense_gpu.vcxproj",
"content": "\n\n \n \n Debug \n Win32 \n \n \n Debug \n x64 \n \n \n Release \n Win32 \n \n \n Release \n x64 \n \n \n \n {80730853-7F34-44C7-BBF7-496977A4C14F} \n libsvm_289_dense \n Win32Proj \n \n \n Application \n Unicode \n true \n \n \n Application \n Unicode \n \n \n Application \n Unicode \n true \n \n \n Application \n Unicode \n \n \n \n \n \n \n \n \n \n \n \n \n <_ProjectFileVersion>10.0.40219.1\n $(SolutionDir)$(Configuration)\\ \n $(Configuration)\\ \n true \n $(SolutionDir)$(Platform)\\$(Configuration)\\ \n $(Platform)\\$(Configuration)\\ \n true \n $(SolutionDir)$(Configuration)\\ \n $(Configuration)\\ \n false \n $(SolutionDir)$(Platform)\\$(Configuration)\\ \n $(Platform)\\$(Configuration)\\ \n false \n $(CUDA_WIN32_LIBS);$(LibraryPath) \n $(CUDA_INC_PATH);C:\\ProgramData\\NVIDIA Corporation\\CUDA Samples\\v5.5\\common\\inc;$(IncludePath) \n \n \n \n Disabled \n $(CUDA_INC_PATH);%(AdditionalIncludeDirectories) \n WIN32;_DEBUG;_CONSOLE;_CONSOLE;_CRT_SECURE_NO_DEPRECATE;_DENSE_REP;%(PreprocessorDefinitions) \n true \n EnableFastChecks \n MultiThreadedDebugDLL \n \n \n Level3 \n EditAndContinue \n \n cublas.lib;cudart.lib;%(AdditionalDependencies) \n $(CUDA_WIN32_LIBS);%(AdditionalLibraryDirectories) \n true \n Console \n false \n \n \n MachineX86 \n \n \n C:\\ProgramData\\NVIDIA Corporation\\CUDA Samples\\v5.5\\common\\inc;%(Include) \n \n \n \n \n X64 \n \n \n Disabled \n $(CUDA_INC_PATH);C:/Documents and Settings/All Users/Application Data/NVIDIA Corporation/NVIDIA GPU Computing SDK/C/common/inc;%(AdditionalIncludeDirectories) \n WIN32;_DEBUG;_CONSOLE;_CONSOLE;_CRT_SECURE_NO_DEPRECATE;_DENSE_REP;%(PreprocessorDefinitions) \n true \n EnableFastChecks \n MultiThreadedDebugDLL \n \n \n Level3 \n ProgramDatabase \n \n cublas.lib;cudart.lib;%(AdditionalDependencies) \n $(CUDA_LIB_PATH);%(AdditionalLibraryDirectories) \n true \n Console \n false \n \n \n MachineX64 \n \n \n \n \n MaxSpeed \n Default \n false \n Neither \n true \n $(CUDA_INC_PATH);%(AdditionalIncludeDirectories) \n WIN32;NDEBUG;_CONSOLE;_CRT_SECURE_NO_DEPRECATE;_DENSE_REP;%(PreprocessorDefinitions) \n Sync \n MultiThreadedDLL \n true \n NotSet \n Precise \n \n \n AssemblyAndMachineCode \n Level3 \n ProgramDatabase \n \n false \n cublas.lib;cudart.lib;cuda.lib;%(AdditionalDependencies) \n $(CUDA_LIB_PATH);%(AdditionalLibraryDirectories) \n true \n Console \n true \n true \n false \n \n \n MachineX86 \n \n \n \n \n X64 \n \n \n /Oa %(AdditionalOptions) \n MaxSpeed \n Default \n false \n Neither \n true \n $(CUDA_INC_PATH);C:/Documents and Settings/All Users/Application Data/NVIDIA Corporation/NVIDIA GPU Computing SDK/C/common/inc;%(AdditionalIncludeDirectories) \n WIN32;NDEBUG;_CONSOLE;_CRT_SECURE_NO_DEPRECATE;_DENSE_REP;%(PreprocessorDefinitions) \n Sync \n MultiThreadedDLL \n true \n NotSet \n Precise \n \n \n AssemblyAndMachineCode \n Level3 \n ProgramDatabase \n \n cublas.lib;cudart.lib;cuda.lib;%(AdditionalDependencies) \n $(CUDA_X64_LIBS);%(AdditionalLibraryDirectories) \n true \n Console \n true \n true \n false \n \n \n MachineX64 \n \n \n \n \n true \n true \n true \n false \n true \n \n \n true \n true \n true \n true \n \n \n \n \n \n \n "
},
{
"path": "src/windows/libsvm_train_dense_gpu/libsvm_train_dense_gpu.vcxproj.filters",
"content": "\n\n \n \n {4FC737F1-C7A5-4376-A066-2A32D752A2FF} \n cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx \n \n \n {93995380-89BD-4b04-88EB-625FBE52EBFB} \n h;hpp;hxx;hm;inl;inc;xsd \n \n \n {67DA6AB6-F800-4c08-8B7A-83BB121AAD01} \n rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx;tiff;tif;png;wav \n \n \n \n \n Source Files \n \n \n Source Files \n \n \n Source Files \n \n \n Source Files \n \n \n \n \n Source Files \n \n \n Header Files \n \n \n "
},
{
"path": "src/windows/libsvm_train_dense_gpu/libsvm_train_dense_gpu.vcxproj.user",
"content": "\n\n "
},
{
"path": "src/windows/libsvm_train_dense_gpu/svm-train.c",
"content": "#include \n#include \n#include \n#include \n#include \n#include \"svm.h\"\n#define Malloc(type,n) (type *)malloc((n)*sizeof(type))\n\nvoid print_null(const char *s) {}\n\nvoid exit_with_help()\n{\n\tprintf(\n\t\"Usage: svm-train [options] training_set_file [model_file]\\n\"\n\t\"options:\\n\"\n\t\"-s svm_type : set type of SVM (default 0)\\n\"\n\t\"\t0 -- C-SVC\t\t(multi-class classification)\\n\"\n\t\"\t1 -- nu-SVC\t\t(multi-class classification)\\n\"\n\t\"\t2 -- one-class SVM\\n\"\n\t\"\t3 -- epsilon-SVR\t(regression)\\n\"\n\t\"\t4 -- nu-SVR\t\t(regression)\\n\"\n\t\"-t kernel_type : set type of kernel function (default 2)\\n\"\n\t\"\t0 -- linear: u'*v\\n\"\n\t\"\t1 -- polynomial: (gamma*u'*v + coef0)^degree\\n\"\n\t\"\t2 -- radial basis function: exp(-gamma*|u-v|^2)\\n\"\n\t\"\t3 -- sigmoid: tanh(gamma*u'*v + coef0)\\n\"\n\t\"\t4 -- precomputed kernel (kernel values in training_set_file)\\n\"\n\t\"-d degree : set degree in kernel function (default 3)\\n\"\n\t\"-g gamma : set gamma in kernel function (default 1/num_features)\\n\"\n\t\"-r coef0 : set coef0 in kernel function (default 0)\\n\"\n\t\"-c cost : set the parameter C of C-SVC, epsilon-SVR, and nu-SVR (default 1)\\n\"\n\t\"-n nu : set the parameter nu of nu-SVC, one-class SVM, and nu-SVR (default 0.5)\\n\"\n\t\"-p epsilon : set the epsilon in loss function of epsilon-SVR (default 0.1)\\n\"\n\t\"-m cachesize : set cache memory size in MB (default 100)\\n\"\n\t\"-e epsilon : set tolerance of termination criterion (default 0.001)\\n\"\n\t\"-h shrinking : whether to use the shrinking heuristics, 0 or 1 (default 1)\\n\"\n\t\"-b probability_estimates : whether to train a SVC or SVR model for probability estimates, 0 or 1 (default 0)\\n\"\n\t\"-wi weight : set the parameter C of class i to weight*C, for C-SVC (default 1)\\n\"\n\t\"-v n: n-fold cross validation mode\\n\"\n\t\"-q : quiet mode (no outputs)\\n\"\n\t);\n\texit(1);\n}\n\nvoid exit_input_error(int line_num)\n{\n\tfprintf(stderr,\"Wrong input format at line %d\\n\", line_num);\n\texit(1);\n}\n\nvoid parse_command_line(int argc, char **argv, char *input_file_name, char *model_file_name);\nvoid read_problem(const char *filename);\nvoid do_cross_validation();\n\nstruct svm_parameter param;\t\t// set by parse_command_line\nstruct svm_problem prob;\t\t// set by read_problem\nstruct svm_model *model;\nstruct svm_node *x_space;\nint cross_validation;\nint nr_fold;\n\nstatic char *line = NULL;\nstatic int max_line_len;\n\n#include \"kernel_matrix_calculation.c\"\n#include \"cross_validation_with_matrix_precomputation.c\"\n\nstatic char* readline(FILE *input)\n{\n\tint len;\n\t\n\tif(fgets(line,max_line_len,input) == NULL)\n\t\treturn NULL;\n\n\twhile(strrchr(line,'\\n') == NULL)\n\t{\n\t\tmax_line_len *= 2;\n\t\tline = (char *) realloc(line,max_line_len);\n\t\tlen = (int) strlen(line);\n\t\tif(fgets(line+len,max_line_len-len,input) == NULL)\n\t\t\tbreak;\n\t}\n\treturn line;\n}\n\nint main(int argc, char **argv)\n{\n\tint i;\n\tchar input_file_name[1024];\n\tchar model_file_name[1024];\n\tconst char *error_msg;\n\n\tparse_command_line(argc, argv, input_file_name, model_file_name);\n\tread_problem(input_file_name);\n\terror_msg = svm_check_parameter(&prob,¶m);\n\tif(error_msg)\n\t{\n\t\tfprintf(stderr,\"ERROR: %s\\n\",error_msg);\n\t\texit(1);\n\t}\n\n\tif(cross_validation)\n\t{\n\t\tdo_cross_validation_with_KM_precalculated( );\n\n//\tdo_cross_validation();\n\t}\n\telse\n\t{\n\t\tmodel = svm_train(&prob,¶m);\n\t\tif(svm_save_model(model_file_name,model))\n\t\t{\n\t\t\tfprintf(stderr, \"can't save model to file %s\\n\", model_file_name);\n\t\t\texit(1);\n\t\t}\n\t\tsvm_free_and_destroy_model(&model);\n\t}\n\tsvm_destroy_param(¶m);\n\tfree(prob.y);\n\n#ifdef _DENSE_REP\n\tfor (i = 0; i < prob.l; ++i)\n\t\tfree((prob.x+i)->values);\n#else\n\tfree(x_space);\n#endif\n\tfree(prob.x);\n\tfree(line);\n\n\treturn 0;\n}\n\nvoid do_cross_validation()\n{\n\tint i;\n\tint total_correct = 0;\n\tdouble total_error = 0;\n\tdouble sumv = 0, sumy = 0, sumvv = 0, sumyy = 0, sumvy = 0;\n\tdouble *target = Malloc(double,prob.l);\n\n\tsvm_cross_validation(&prob,¶m,nr_fold,target);\n\tif(param.svm_type == EPSILON_SVR ||\n\t param.svm_type == NU_SVR)\n\t{\n\t\tfor(i=0;i=argc)\n\t\t\texit_with_help();\n\t\tswitch(argv[i-1][1])\n\t\t{\n\t\t\tcase 's':\n\t\t\t\tparam.svm_type = atoi(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 't':\n\t\t\t\tparam.kernel_type = atoi(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'd':\n\t\t\t\tparam.degree = atoi(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'g':\n\t\t\t\tparam.gamma = atof(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'r':\n\t\t\t\tparam.coef0 = atof(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'n':\n\t\t\t\tparam.nu = atof(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'm':\n\t\t\t\tparam.cache_size = atof(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'c':\n\t\t\t\tparam.C = atof(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'e':\n\t\t\t\tparam.eps = atof(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'p':\n\t\t\t\tparam.p = atof(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'h':\n\t\t\t\tparam.shrinking = atoi(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'b':\n\t\t\t\tparam.probability = atoi(argv[i]);\n\t\t\t\tbreak;\n\t\t\tcase 'q':\n\t\t\t\tprint_func = &print_null;\n\t\t\t\ti--;\n\t\t\t\tbreak;\n\t\t\tcase 'v':\n\t\t\t\tcross_validation = 1;\n\t\t\t\tnr_fold = atoi(argv[i]);\n\t\t\t\tif(nr_fold < 2)\n\t\t\t\t{\n\t\t\t\t\tfprintf(stderr,\"n-fold cross validation: n must >= 2\\n\");\n\t\t\t\t\texit_with_help();\n\t\t\t\t}\n\t\t\t\tbreak;\n\t\t\tcase 'w':\n\t\t\t\t++param.nr_weight;\n\t\t\t\tparam.weight_label = (int *)realloc(param.weight_label,sizeof(int)*param.nr_weight);\n\t\t\t\tparam.weight = (double *)realloc(param.weight,sizeof(double)*param.nr_weight);\n\t\t\t\tparam.weight_label[param.nr_weight-1] = atoi(&argv[i-1][2]);\n\t\t\t\tparam.weight[param.nr_weight-1] = atof(argv[i]);\n\t\t\t\tbreak;\n\t\t\tdefault:\n\t\t\t\tfprintf(stderr,\"Unknown option: -%c\\n\", argv[i-1][1]);\n\t\t\t\texit_with_help();\n\t\t}\n\t}\n\n\tsvm_set_print_string_function(print_func);\n\n\t// determine filenames\n\n\tif(i>=argc)\n\t\texit_with_help();\n\n\tstrcpy(input_file_name, argv[i]);\n\n\tif(i line)\n\t\t\t\tp--;\n\t\t\tif(p > line)\n\t\t\t \tmax_index = (int) strtol(p,&endptr,10) + 1;\n\t\t}\n\t\tif(max_index > elements)\n\t\t\telements = max_index;\n\t\t++prob.l;\n\t}\n\n\trewind(fp);\n\n\tprob.y = Malloc(double,prob.l);\n\tprob.x = Malloc(struct svm_node,prob.l);\n\n\tfor(i=0;ivalues = Malloc(double,elements);\n\t\t(prob.x+i)->dim = 0;\n\n\t\tinst_max_index = -1; // strtol gives 0 if wrong format, and precomputed kernel has start from 0\n\t\treadline(fp);\n\n\t\tlabel = strtok(line,\" \\t\");\n\t\tprob.y[i] = strtod(label,&endptr);\n\t\tif(endptr == label)\n\t\t\texit_input_error(i+1);\n\n\t\twhile(1)\n\t\t{\n\t\t\tidx = strtok(NULL,\":\");\n\t\t\tval = strtok(NULL,\" \\t\");\n\n\t\t\tif(val == NULL)\n\t\t\t\tbreak;\n\n\t\t\terrno = 0;\n\t\t\tj = (int) strtol(idx,&endptr,10);\n\t\t\tif(endptr == idx || errno != 0 || *endptr != '\\0' || j <= inst_max_index)\n\t\t\t\texit_input_error(i+1);\n\t\t\telse\n\t\t\t\tinst_max_index = j;\n\n\t\t\terrno = 0;\n\t\t\tvalue = strtod(val,&endptr);\n\t\t\tif(endptr == val || errno != 0 || (*endptr != '\\0' && !isspace(*endptr)))\n\t\t\t\texit_input_error(i+1);\n\n\t\t\td = &((prob.x+i)->dim);\n\t\t\twhile (*d < j)\n\t\t\t\t(prob.x+i)->values[(*d)++] = 0.0;\n\t\t\t(prob.x+i)->values[(*d)++] = value;\n\t\t}\t\n\t}\n\tmax_index = elements-1;\n\n#else\n\twhile(readline(fp)!=NULL)\n\t{\n\t\tchar *p = strtok(line,\" \\t\"); // label\n\n\t\t// features\n\t\twhile(1)\n\t\t{\n\t\t\tp = strtok(NULL,\" \\t\");\n\t\t\tif(p == NULL || *p == '\\n') // check '\\n' as ' ' may be after the last feature\n\t\t\t\tbreak;\n\t\t\t++elements;\n\t\t}\n\t\t++elements;\n\t\t++prob.l;\n\t}\n\trewind(fp);\n\n\tprob.y = Malloc(double,prob.l);\n\tprob.x = Malloc(struct svm_node *,prob.l);\n\tx_space = Malloc(struct svm_node,elements);\n\n\tmax_index = 0;\n\tj=0;\n\tfor(i=0;i start from 0\n\t\treadline(fp);\n\t\tprob.x[i] = &x_space[j];\n\t\tlabel = strtok(line,\" \\t\\n\");\n\t\tif(label == NULL) // empty line\n\t\t\texit_input_error(i+1);\n\t\t\t\n\t\tprob.y[i] = strtod(label,&endptr);\n\t\tif(endptr == label || *endptr != '\\0')\n\t\t\texit_input_error(i+1);\n\n\t\twhile(1)\n\t\t{\n\t\t\tidx = strtok(NULL,\":\");\n\t\t\tval = strtok(NULL,\" \\t\");\n\n\t\t\tif(val == NULL)\n\t\t\t\tbreak;\n\n\t\t\terrno = 0;\n\t\t\tx_space[j].index = (int) strtol(idx,&endptr,10);\n\t\t\tif(endptr == idx || errno != 0 || *endptr != '\\0' || x_space[j].index <= inst_max_index)\n\t\t\t\texit_input_error(i+1);\n\t\t\telse\n\t\t\t\tinst_max_index = x_space[j].index;\n\n\t\t\terrno = 0;\n\t\t\tx_space[j].value = strtod(val,&endptr);\n\t\t\tif(endptr == val || errno != 0 || (*endptr != '\\0' && !isspace(*endptr)))\n\t\t\t\texit_input_error(i+1);\n\n\t\t\t++j;\n\t\t}\n\n\t\tif(inst_max_index > max_index)\n\t\t\tmax_index = inst_max_index;\n\t\tx_space[j++].index = -1;\n\t}\n#endif\n\n\tif(param.gamma == 0 && max_index > 0)\n\t\tparam.gamma = 1.0/max_index;\n\n\tif(param.kernel_type == PRECOMPUTED)\n\t\tfor(i=0;idim == 0 || (prob.x+i)->values[0] == 0.0)\n\t\t\t{\n\t\t\t\tfprintf(stderr,\"Wrong input format: first column must be 0:sample_serial_number\\n\");\n\t\t\t\texit(1);\n\t\t\t}\n\t\t\tif ((int)(prob.x+i)->values[0] < 0 || (int)(prob.x+i)->values[0] > max_index)\n\t\t\t{\n\t\t\t\tfprintf(stderr,\"Wrong input format: sample_serial_number out of range\\n\");\n\t\t\t\texit(1);\n\t\t\t}\n#else\n\t\t\tif (prob.x[i][0].index != 0)\n\t\t\t{\n\t\t\t\tfprintf(stderr,\"Wrong input format: first column must be 0:sample_serial_number\\n\");\n\t\t\t\texit(1);\n\t\t\t}\n\t\t\tif ((int)prob.x[i][0].value <= 0 || (int)prob.x[i][0].value > max_index)\n\t\t\t{\n\t\t\t\tfprintf(stderr,\"Wrong input format: sample_serial_number out of range\\n\");\n\t\t\t\texit(1);\n\t\t\t}\n#endif\n\t\t}\n\tfclose(fp);\n}\n"
},
{
"path": "src/windows/libsvm_train_dense_gpu/svm.cpp",
"content": "#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include