Repository: lawlite19/MachineLearningEx Branch: master Commit: 44be60fe4d63 Files: 239 Total size: 11.6 MB Directory structure: gitextract_b2kfayth/ ├── machine-learning-ex1/ │ └── ex1/ │ ├── computeCost.m │ ├── computeCostMulti.m │ ├── ex1.m │ ├── ex1_multi.m │ ├── ex1data1.txt │ ├── ex1data2.txt │ ├── featureNormalize.m │ ├── gradientDescent.m │ ├── gradientDescentMulti.m │ ├── lib/ │ │ ├── jsonlab/ │ │ │ ├── AUTHORS.txt │ │ │ ├── ChangeLog.txt │ │ │ ├── LICENSE_BSD.txt │ │ │ ├── README.txt │ │ │ ├── jsonopt.m │ │ │ ├── loadjson.m │ │ │ ├── loadubjson.m │ │ │ ├── mergestruct.m │ │ │ ├── savejson.m │ │ │ ├── saveubjson.m │ │ │ └── varargin2struct.m │ │ ├── makeValidFieldName.m │ │ └── submitWithConfiguration.m │ ├── normalEqn.m │ ├── octave-workspace │ ├── plotData.m │ ├── submit.m │ ├── token.mat │ └── warmUpExercise.m ├── machine-learning-ex2/ │ └── ex2/ │ ├── costFunction.m │ ├── costFunctionReg.m │ ├── ex2.m │ ├── ex2_reg.m │ ├── ex2data1.txt │ ├── ex2data2.txt │ ├── lib/ │ │ ├── jsonlab/ │ │ │ ├── AUTHORS.txt │ │ │ ├── ChangeLog.txt │ │ │ ├── LICENSE_BSD.txt │ │ │ ├── README.txt │ │ │ ├── jsonopt.m │ │ │ ├── loadjson.m │ │ │ ├── loadubjson.m │ │ │ ├── mergestruct.m │ │ │ ├── savejson.m │ │ │ ├── saveubjson.m │ │ │ └── varargin2struct.m │ │ ├── makeValidFieldName.m │ │ └── submitWithConfiguration.m │ ├── mapFeature.m │ ├── plotData.m │ ├── plotDecisionBoundary.m │ ├── predict.m │ ├── sigmoid.m │ ├── submit.m │ └── token.mat ├── machine-learning-ex3/ │ └── ex3/ │ ├── displayData.m │ ├── ex3.m │ ├── ex3_nn.m │ ├── ex3data1.mat │ ├── ex3weights.mat │ ├── fmincg.m │ ├── lib/ │ │ ├── jsonlab/ │ │ │ ├── AUTHORS.txt │ │ │ ├── ChangeLog.txt │ │ │ ├── LICENSE_BSD.txt │ │ │ ├── README.txt │ │ │ ├── jsonopt.m │ │ │ ├── loadjson.m │ │ │ ├── loadubjson.m │ │ │ ├── mergestruct.m │ │ │ ├── savejson.m │ │ │ ├── saveubjson.m │ │ │ └── varargin2struct.m │ │ ├── makeValidFieldName.m │ │ └── submitWithConfiguration.m │ ├── lrCostFunction.m │ ├── oneVsAll.m │ ├── predict.m │ ├── predictOneVsAll.m │ ├── sigmoid.m │ ├── submit.m │ └── token.mat ├── machine-learning-ex4/ │ └── ex4/ │ ├── checkNNGradients.m │ ├── computeNumericalGradient.m │ ├── debugInitializeWeights.m │ ├── displayData.m │ ├── ex4.m │ ├── ex4data1.mat │ ├── ex4weights.mat │ ├── fmincg.m │ ├── lib/ │ │ ├── jsonlab/ │ │ │ ├── AUTHORS.txt │ │ │ ├── ChangeLog.txt │ │ │ ├── LICENSE_BSD.txt │ │ │ ├── README.txt │ │ │ ├── jsonopt.m │ │ │ ├── loadjson.m │ │ │ ├── loadubjson.m │ │ │ ├── mergestruct.m │ │ │ ├── savejson.m │ │ │ ├── saveubjson.m │ │ │ └── varargin2struct.m │ │ ├── makeValidFieldName.m │ │ └── submitWithConfiguration.m │ ├── nnCostFunction.m │ ├── predict.m │ ├── randInitializeWeights.m │ ├── sigmoid.m │ ├── sigmoidGradient.m │ ├── submit.m │ └── token.mat ├── machine-learning-ex5/ │ └── ex5/ │ ├── ex5.m │ ├── ex5data1.mat │ ├── featureNormalize.m │ ├── fmincg.m │ ├── learningCurve.m │ ├── lib/ │ │ ├── jsonlab/ │ │ │ ├── AUTHORS.txt │ │ │ ├── ChangeLog.txt │ │ │ ├── LICENSE_BSD.txt │ │ │ ├── README.txt │ │ │ ├── jsonopt.m │ │ │ ├── loadjson.m │ │ │ ├── loadubjson.m │ │ │ ├── mergestruct.m │ │ │ ├── savejson.m │ │ │ ├── saveubjson.m │ │ │ └── varargin2struct.m │ │ ├── makeValidFieldName.m │ │ └── submitWithConfiguration.m │ ├── linearRegCostFunction.m │ ├── plotFit.m │ ├── polyFeatures.m │ ├── submit.m │ ├── token.mat │ ├── trainLinearReg.m │ └── validationCurve.m ├── machine-learning-ex6/ │ └── ex6/ │ ├── dataset3Params.m │ ├── emailFeatures.m │ ├── emailSample1.txt │ ├── emailSample2.txt │ ├── ex6.m │ ├── ex6_spam.m │ ├── ex6data1.mat │ ├── ex6data2.mat │ ├── ex6data3.mat │ ├── gaussianKernel.m │ ├── getVocabList.m │ ├── lib/ │ │ ├── jsonlab/ │ │ │ ├── AUTHORS.txt │ │ │ ├── ChangeLog.txt │ │ │ ├── LICENSE_BSD.txt │ │ │ ├── README.txt │ │ │ ├── jsonopt.m │ │ │ ├── loadjson.m │ │ │ ├── loadubjson.m │ │ │ ├── mergestruct.m │ │ │ ├── savejson.m │ │ │ ├── saveubjson.m │ │ │ └── varargin2struct.m │ │ ├── makeValidFieldName.m │ │ └── submitWithConfiguration.m │ ├── linearKernel.m │ ├── plotData.m │ ├── porterStemmer.m │ ├── processEmail.m │ ├── readFile.m │ ├── spamSample1.txt │ ├── spamSample2.txt │ ├── spamTest.mat │ ├── spamTrain.mat │ ├── submit.m │ ├── svmPredict.m │ ├── svmTrain.m │ ├── token.mat │ ├── visualizeBoundary.m │ ├── visualizeBoundaryLinear.m │ └── vocab.txt ├── machine-learning-ex7/ │ └── ex7/ │ ├── bird_small.mat │ ├── computeCentroids.m │ ├── displayData.m │ ├── drawLine.m │ ├── ex7.m │ ├── ex7_pca.m │ ├── ex7data1.mat │ ├── ex7data2.mat │ ├── ex7faces.mat │ ├── featureNormalize.m │ ├── findClosestCentroids.m │ ├── kMeansInitCentroids.m │ ├── lib/ │ │ ├── jsonlab/ │ │ │ ├── AUTHORS.txt │ │ │ ├── ChangeLog.txt │ │ │ ├── LICENSE_BSD.txt │ │ │ ├── README.txt │ │ │ ├── jsonopt.m │ │ │ ├── loadjson.m │ │ │ ├── loadubjson.m │ │ │ ├── mergestruct.m │ │ │ ├── savejson.m │ │ │ ├── saveubjson.m │ │ │ └── varargin2struct.m │ │ ├── makeValidFieldName.m │ │ └── submitWithConfiguration.m │ ├── pca.m │ ├── plotDataPoints.m │ ├── plotProgresskMeans.m │ ├── projectData.m │ ├── recoverData.m │ ├── runkMeans.m │ ├── submit.m │ └── token.mat ├── machine-learning-ex8/ │ └── ex8/ │ ├── checkCostFunction.m │ ├── cofiCostFunc.m │ ├── computeNumericalGradient.m │ ├── estimateGaussian.m │ ├── ex8.m │ ├── ex8_cofi.m │ ├── ex8_movieParams.mat │ ├── ex8_movies.mat │ ├── ex8data1.mat │ ├── ex8data2.mat │ ├── fmincg.m │ ├── lib/ │ │ ├── jsonlab/ │ │ │ ├── AUTHORS.txt │ │ │ ├── ChangeLog.txt │ │ │ ├── LICENSE_BSD.txt │ │ │ ├── README.txt │ │ │ ├── jsonopt.m │ │ │ ├── loadjson.m │ │ │ ├── loadubjson.m │ │ │ ├── mergestruct.m │ │ │ ├── savejson.m │ │ │ ├── saveubjson.m │ │ │ └── varargin2struct.m │ │ ├── makeValidFieldName.m │ │ └── submitWithConfiguration.m │ ├── loadMovieList.m │ ├── movie_ids.txt │ ├── multivariateGaussian.m │ ├── normalizeRatings.m │ ├── selectThreshold.m │ ├── submit.m │ ├── token.mat │ └── visualizeFit.m └── readme.md ================================================ FILE CONTENTS ================================================ ================================================ FILE: machine-learning-ex1/ex1/computeCost.m ================================================ function J = computeCost(X, y, theta) %COMPUTECOST Compute cost for linear regression % J = COMPUTECOST(X, y, theta) computes the cost of using theta as the % parameter for linear regression to fit the data points in X and y % Initialize some useful values m = length(y); % number of training examples % You need to return the following variables correctly J = 0; % ====================== YOUR CODE HERE ====================== % Instructions: Compute the cost of a particular choice of theta % You should set J to the cost. temp1=X*theta-y; temp2=sum(temp1.^2); J=1/(2*m)*temp2; %sprintf('%f',J); % ========================================================================= end ================================================ FILE: machine-learning-ex1/ex1/computeCostMulti.m ================================================ function J = computeCostMulti(X, y, theta) %COMPUTECOSTMULTI Compute cost for linear regression with multiple variables % J = COMPUTECOSTMULTI(X, y, theta) computes the cost of using theta as the % parameter for linear regression to fit the data points in X and y % Initialize some useful values m = length(y); % number of training examples % You need to return the following variables correctly J = 0; % ====================== YOUR CODE HERE ====================== % Instructions: Compute the cost of a particular choice of theta % You should set J to the cost. temp=(X*theta-y)'*(X*theta-y); J=1/(2*m)*temp; % ========================================================================= end ================================================ FILE: machine-learning-ex1/ex1/ex1.m ================================================ %% Machine Learning Online Class - Exercise 1: Linear Regression % Instructions % ------------ % % This file contains code that helps you get started on the % linear exercise. You will need to complete the following functions % in this exericse: % % warmUpExercise.m % plotData.m % gradientDescent.m % computeCost.m % gradientDescentMulti.m % computeCostMulti.m % featureNormalize.m % normalEqn.m % % For this exercise, you will not need to change any code in this file, % or any other files other than those mentioned above. % % x refers to the population size in 10,000s % y refers to the profit in $10,000s % %% Initialization clear ; close all; clc %% ==================== Part 1: Basic Function ==================== % Complete warmUpExercise.m fprintf('Running warmUpExercise ... \n'); fprintf('5x5 Identity Matrix: \n'); warmUpExercise() fprintf('Program paused. Press enter to continue.\n'); pause; %% ======================= Part 2: Plotting ======================= fprintf('Plotting Data ...\n') data = load('ex1data1.txt'); X = data(:, 1); y = data(:, 2); m = length(y); % number of training examples % Plot Data % Note: You have to complete the code in plotData.m plotData(X, y); fprintf('Program paused. Press enter to continue.\n'); pause; %% =================== Part 3: Gradient descent =================== fprintf('Running Gradient Descent ...\n') X = [ones(m, 1), data(:,1)]; % Add a column of ones to x theta = zeros(2, 1); % initialize fitting parameters % Some gradient descent settings iterations = 1500; alpha = 0.01; % compute and display initial cost computeCost(X, y, theta) % run gradient descent theta = gradientDescent(X, y, theta, alpha, iterations); % print theta to screen fprintf('Theta found by gradient descent: '); fprintf('%f %f \n', theta(1), theta(2)); % Plot the linear fit hold on; % keep previous plot visible plot(X(:,2), X*theta, '-') legend('Training data', 'Linear regression') hold off % don't overlay any more plots on this figure % Predict values for population sizes of 35,000 and 70,000 predict1 = [1, 3.5] *theta; fprintf('For population = 35,000, we predict a profit of %f\n',... predict1*10000); predict2 = [1, 7] * theta; fprintf('For population = 70,000, we predict a profit of %f\n',... predict2*10000); fprintf('Program paused. Press enter to continue.\n'); pause; %% ============= Part 4: Visualizing J(theta_0, theta_1) ============= fprintf('Visualizing J(theta_0, theta_1) ...\n') % Grid over which we will calculate J theta0_vals = linspace(-10, 10, 100); theta1_vals = linspace(-1, 4, 100); % initialize J_vals to a matrix of 0's J_vals = zeros(length(theta0_vals), length(theta1_vals)); % Fill out J_vals for i = 1:length(theta0_vals) for j = 1:length(theta1_vals) t = [theta0_vals(i); theta1_vals(j)]; J_vals(i,j) = computeCost(X, y, t); end end % Because of the way meshgrids work in the surf command, we need to % transpose J_vals before calling surf, or else the axes will be flipped J_vals = J_vals'; % Surface plot figure; surf(theta0_vals, theta1_vals, J_vals) xlabel('\theta_0'); ylabel('\theta_1'); % Contour plot figure; % Plot J_vals as 15 contours spaced logarithmically between 0.01 and 100 contour(theta0_vals, theta1_vals, J_vals, logspace(-2, 3, 20)) xlabel('\theta_0'); ylabel('\theta_1'); hold on; plot(theta(1), theta(2), 'rx', 'MarkerSize', 10, 'LineWidth', 2); ================================================ FILE: machine-learning-ex1/ex1/ex1_multi.m ================================================ %% Machine Learning Online Class % Exercise 1: Linear regression with multiple variables % % Instructions % ------------ % % This file contains code that helps you get started on the % linear regression exercise. % % You will need to complete the following functions in this % exericse: % % warmUpExercise.m % plotData.m % gradientDescent.m % computeCost.m % gradientDescentMulti.m % computeCostMulti.m % featureNormalize.m % normalEqn.m % % For this part of the exercise, you will need to change some % parts of the code below for various experiments (e.g., changing % learning rates). % %% Initialization %% ================ Part 1: Feature Normalization ================ %% Clear and Close Figures clear ; close all; clc fprintf('Loading data ...\n'); %% Load Data data = load('ex1data2.txt'); X = data(:, 1:2); y = data(:, 3); m = length(y); % Print out some data points fprintf('First 10 examples from the dataset: \n'); fprintf(' x = [%.0f %.0f], y = %.0f \n', [X(1:10,:) y(1:10,:)]'); fprintf('Program paused. Press enter to continue.\n'); pause; % Scale features and set them to zero mean fprintf('Normalizing Features ...\n'); [X mu sigma] = featureNormalize(X);%ע⣬XӦX_norm % Add intercept term to X X = [ones(m, 1) X]; %% ================ Part 2: Gradient Descent ================ % ====================== YOUR CODE HERE ====================== % Instructions: We have provided you with the following starter % code that runs gradient descent with a particular % learning rate (alpha). % % Your task is to first make sure that your functions - % computeCost and gradientDescent already work with % this starter code and support multiple variables. % % After that, try running gradient descent with % different values of alpha and see which one gives % you the best result. % % Finally, you should complete the code at the end % to predict the price of a 1650 sq-ft, 3 br house. % % Hint: By using the 'hold on' command, you can plot multiple % graphs on the same figure. % % Hint: At prediction, make sure you do the same feature normalization. % fprintf('Running gradient descent ...\n'); % Choose some alpha value alpha = 0.01; num_iters = 400; % Init Theta and Run Gradient Descent theta = zeros(3, 1); [theta, J_history] = gradientDescentMulti(X, y, theta, alpha, num_iters); % Plot the convergence graph figure; plot(1:numel(J_history), J_history, '-b', 'LineWidth', 2); xlabel('Number of iterations'); ylabel('Cost J'); % Display gradient descent's result fprintf('Theta computed from gradient descent: \n'); fprintf(' %f \n', theta); fprintf('\n'); % Estimate the price of a 1650 sq-ft, 3 br house % ====================== YOUR CODE HERE ====================== % Recall that the first column of X is all-ones. Thus, it does % not need to be normalized. price = 0; % You should change this predict1=[1650 3]; norm_predict=(predict1-mu)./sigma; final_predict=[1 norm_predict]; price=final_predict*theta; % ============================================================ fprintf(['Predicted price of a 1650 sq-ft, 3 br house ' ... '(using gradient descent):\n $%f\n'], price); fprintf('Program paused. Press enter to continue.\n'); pause; %% ================ Part 3: Normal Equations ================ fprintf('Solving with normal equations...\n'); % ====================== YOUR CODE HERE ====================== % Instructions: The following code computes the closed form % solution for linear regression using the normal % equations. You should complete the code in % normalEqn.m % % After doing so, you should complete this code % to predict the price of a 1650 sq-ft, 3 br house. % %% Load Data data = csvread('ex1data2.txt'); X = data(:, 1:2); y = data(:, 3); m = length(y); % Add intercept term to X X = [ones(m, 1) X]; % Calculate the parameters from the normal equation theta = normalEqn(X, y); % Display normal equation's result fprintf('Theta computed from the normal equations: \n'); fprintf(' %f \n', theta); fprintf('\n'); % Estimate the price of a 1650 sq-ft, 3 br house % ====================== YOUR CODE HERE ====================== price = 0; % You should change this predict2=[1 1650 3]; price=predict2*theta; % ============================================================ fprintf(['Predicted price of a 1650 sq-ft, 3 br house ' ... '(using normal equations):\n $%f\n'], price); ================================================ FILE: machine-learning-ex1/ex1/ex1data1.txt ================================================ 6.1101,17.592 5.5277,9.1302 8.5186,13.662 7.0032,11.854 5.8598,6.8233 8.3829,11.886 7.4764,4.3483 8.5781,12 6.4862,6.5987 5.0546,3.8166 5.7107,3.2522 14.164,15.505 5.734,3.1551 8.4084,7.2258 5.6407,0.71618 5.3794,3.5129 6.3654,5.3048 5.1301,0.56077 6.4296,3.6518 7.0708,5.3893 6.1891,3.1386 20.27,21.767 5.4901,4.263 6.3261,5.1875 5.5649,3.0825 18.945,22.638 12.828,13.501 10.957,7.0467 13.176,14.692 22.203,24.147 5.2524,-1.22 6.5894,5.9966 9.2482,12.134 5.8918,1.8495 8.2111,6.5426 7.9334,4.5623 8.0959,4.1164 5.6063,3.3928 12.836,10.117 6.3534,5.4974 5.4069,0.55657 6.8825,3.9115 11.708,5.3854 5.7737,2.4406 7.8247,6.7318 7.0931,1.0463 5.0702,5.1337 5.8014,1.844 11.7,8.0043 5.5416,1.0179 7.5402,6.7504 5.3077,1.8396 7.4239,4.2885 7.6031,4.9981 6.3328,1.4233 6.3589,-1.4211 6.2742,2.4756 5.6397,4.6042 9.3102,3.9624 9.4536,5.4141 8.8254,5.1694 5.1793,-0.74279 21.279,17.929 14.908,12.054 18.959,17.054 7.2182,4.8852 8.2951,5.7442 10.236,7.7754 5.4994,1.0173 20.341,20.992 10.136,6.6799 7.3345,4.0259 6.0062,1.2784 7.2259,3.3411 5.0269,-2.6807 6.5479,0.29678 7.5386,3.8845 5.0365,5.7014 10.274,6.7526 5.1077,2.0576 5.7292,0.47953 5.1884,0.20421 6.3557,0.67861 9.7687,7.5435 6.5159,5.3436 8.5172,4.2415 9.1802,6.7981 6.002,0.92695 5.5204,0.152 5.0594,2.8214 5.7077,1.8451 7.6366,4.2959 5.8707,7.2029 5.3054,1.9869 8.2934,0.14454 13.394,9.0551 5.4369,0.61705 ================================================ FILE: machine-learning-ex1/ex1/ex1data2.txt ================================================ 2104,3,399900 1600,3,329900 2400,3,369000 1416,2,232000 3000,4,539900 1985,4,299900 1534,3,314900 1427,3,198999 1380,3,212000 1494,3,242500 1940,4,239999 2000,3,347000 1890,3,329999 4478,5,699900 1268,3,259900 2300,4,449900 1320,2,299900 1236,3,199900 2609,4,499998 3031,4,599000 1767,3,252900 1888,2,255000 1604,3,242900 1962,4,259900 3890,3,573900 1100,3,249900 1458,3,464500 2526,3,469000 2200,3,475000 2637,3,299900 1839,2,349900 1000,1,169900 2040,4,314900 3137,3,579900 1811,4,285900 1437,3,249900 1239,3,229900 2132,4,345000 4215,4,549000 2162,4,287000 1664,2,368500 2238,3,329900 2567,4,314000 1200,3,299000 852,2,179900 1852,4,299900 1203,3,239500 ================================================ FILE: machine-learning-ex1/ex1/featureNormalize.m ================================================ function [X_norm, mu, sigma] = featureNormalize(X) %FEATURENORMALIZE Normalizes the features in X % FEATURENORMALIZE(X) returns a normalized version of X where % the mean value of each feature is 0 and the standard deviation % is 1. This is often a good preprocessing step to do when % working with learning algorithms. % You need to set these values correctly X_norm = X; mu = zeros(1, size(X, 2)); sigma = zeros(1, size(X, 2)); % ====================== YOUR CODE HERE ====================== % Instructions: First, for each feature dimension, compute the mean % of the feature and subtract it from the dataset, % storing the mean value in mu. Next, compute the % standard deviation of each feature and divide % each feature by it's standard deviation, storing % the standard deviation in sigma. % % Note that X is a matrix where each column is a % feature and each row is an example. You need % to perform the normalization separately for % each feature. % % Hint: You might find the 'mean' and 'std' functions useful. % mu=mean(X_norm); sigma=std(X_norm); for i=1:size(X,2) X_norm(:,i)=X_norm(:,i)-mu(i); X_norm(:,i)=X_norm(:,i)./sigma(i); end % ============================================================ end ================================================ FILE: machine-learning-ex1/ex1/gradientDescent.m ================================================ function [theta, J_history] = gradientDescent(X, y, theta, alpha, num_iters) %GRADIENTDESCENT Performs gradient descent to learn theta % theta = GRADIENTDESENT(X, y, theta, alpha, num_iters) updates theta by % taking num_iters gradient steps with learning rate alpha % Initialize some useful values m = length(y); % number of training examples J_history = zeros(num_iters, 1); for iter = 1:num_iters % ====================== YOUR CODE HERE ====================== % Instructions: Perform a single gradient step on the parameter vector % theta. % % Hint: While debugging, it can be useful to print out the values % of the cost function (computeCost) and gradient here. % temp1=1/m*alpha*sum(X*theta-y); temp2=1/m*alpha*sum((X*theta-y).*X(:,2)); theta(1)=theta(1)-temp1; theta(2)=theta(2)-temp2; % ============================================================ % Save the cost J in every iteration J_history(iter) = computeCost(X, y, theta); end end ================================================ FILE: machine-learning-ex1/ex1/gradientDescentMulti.m ================================================ function [theta, J_history] = gradientDescentMulti(X, y, theta, alpha, num_iters) %GRADIENTDESCENTMULTI Performs gradient descent to learn theta % theta = GRADIENTDESCENTMULTI(x, y, theta, alpha, num_iters) updates theta by % taking num_iters gradient steps with learning rate alpha % Initialize some useful values m = length(y); % number of training examples n = length(theta); temp = zeros(n, num_iters); J_history = zeros(num_iters, 1); for iter = 1:num_iters % ====================== YOUR CODE HERE ====================== % Instructions: Perform a single gradient step on the parameter vector % theta. % % Hint: While debugging, it can be useful to print out the values % of the cost function (computeCostMulti) and gradient here. % h = X*theta; temp(:,iter) = theta - ((alpha/m)*(X'*(h-y))); theta = temp(:,iter); %temp1=1/m*alpha*sum(X*theta-y); %temp2=1/m*alpha*sum((X*theta-y).*X(:,2)); %temp3=1/m*alpha*sum((X*theta-y).*X(:,3)); %theta(1)=theta(1)-temp1; %theta(2)=theta(2)-temp2; %theta(3)=theta(3)-temp3; % ============================================================ % Save the cost J in every iteration J_history(iter) = computeCostMulti(X, y, theta); end end ================================================ FILE: machine-learning-ex1/ex1/lib/jsonlab/AUTHORS.txt ================================================ The author of "jsonlab" toolbox is Qianqian Fang. Qianqian is currently an Assistant Professor at Massachusetts General Hospital, Harvard Medical School. Address: Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School Bldg 149, 13th St, Charlestown, MA 02129, USA URL: http://nmr.mgh.harvard.edu/~fangq/ Email: or The script loadjson.m was built upon previous works by - Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713 date: 2009/11/02 - François Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393 date: 2009/03/22 - Joel Feenstra: http://www.mathworks.com/matlabcentral/fileexchange/20565 date: 2008/07/03 This toolbox contains patches submitted by the following contributors: - Blake Johnson part of revision 341 - Niclas Borlin various fixes in revision 394, including - loadjson crashes for all-zero sparse matrix. - loadjson crashes for empty sparse matrix. - Non-zero size of 0-by-N and N-by-0 empty matrices is lost after savejson/loadjson. - loadjson crashes for sparse real column vector. - loadjson crashes for sparse complex column vector. - Data is corrupted by savejson for sparse real row vector. - savejson crashes for sparse complex row vector. - Yul Kang patches for svn revision 415. - savejson saves an empty cell array as [] instead of null - loadjson differentiates an empty struct from an empty array ================================================ FILE: machine-learning-ex1/ex1/lib/jsonlab/ChangeLog.txt ================================================ ============================================================================ JSONlab - a toolbox to encode/decode JSON/UBJSON files in MATLAB/Octave ---------------------------------------------------------------------------- JSONlab ChangeLog (key features marked by *): == JSONlab 1.0 (codename: Optimus - Final), FangQ == 2015/01/02 polish help info for all major functions, update examples, finalize 1.0 2014/12/19 fix a bug to strictly respect NoRowBracket in savejson == JSONlab 1.0.0-RC2 (codename: Optimus - RC2), FangQ == 2014/11/22 show progress bar in loadjson ('ShowProgress') 2014/11/17 add Compact option in savejson to output compact JSON format ('Compact') 2014/11/17 add FastArrayParser in loadjson to specify fast parser applicable levels 2014/09/18 start official github mirror: https://github.com/fangq/jsonlab == JSONlab 1.0.0-RC1 (codename: Optimus - RC1), FangQ == 2014/09/17 fix several compatibility issues when running on octave versions 3.2-3.8 2014/09/17 support 2D cell and struct arrays in both savejson and saveubjson 2014/08/04 escape special characters in a JSON string 2014/02/16 fix a bug when saving ubjson files == JSONlab 0.9.9 (codename: Optimus - beta), FangQ == 2014/01/22 use binary read and write in saveubjson and loadubjson == JSONlab 0.9.8-1 (codename: Optimus - alpha update 1), FangQ == 2013/10/07 better round-trip conservation for empty arrays and structs (patch submitted by Yul Kang) == JSONlab 0.9.8 (codename: Optimus - alpha), FangQ == 2013/08/23 *universal Binary JSON (UBJSON) support, including both saveubjson and loadubjson == JSONlab 0.9.1 (codename: Rodimus, update 1), FangQ == 2012/12/18 *handling of various empty and sparse matrices (fixes submitted by Niclas Borlin) == JSONlab 0.9.0 (codename: Rodimus), FangQ == 2012/06/17 *new format for an invalid leading char, unpacking hex code in savejson 2012/06/01 support JSONP in savejson 2012/05/25 fix the empty cell bug (reported by Cyril Davin) 2012/04/05 savejson can save to a file (suggested by Patrick Rapin) == JSONlab 0.8.1 (codename: Sentiel, Update 1), FangQ == 2012/02/28 loadjson quotation mark escape bug, see http://bit.ly/yyk1nS 2012/01/25 patch to handle root-less objects, contributed by Blake Johnson == JSONlab 0.8.0 (codename: Sentiel), FangQ == 2012/01/13 *speed up loadjson by 20 fold when parsing large data arrays in matlab 2012/01/11 remove row bracket if an array has 1 element, suggested by Mykel Kochenderfer 2011/12/22 *accept sequence of 'param',value input in savejson and loadjson 2011/11/18 fix struct array bug reported by Mykel Kochenderfer == JSONlab 0.5.1 (codename: Nexus Update 1), FangQ == 2011/10/21 fix a bug in loadjson, previous code does not use any of the acceleration 2011/10/20 loadjson supports JSON collections - concatenated JSON objects == JSONlab 0.5.0 (codename: Nexus), FangQ == 2011/10/16 package and release jsonlab 0.5.0 2011/10/15 *add json demo and regression test, support cpx numbers, fix double quote bug 2011/10/11 *speed up readjson dramatically, interpret _Array* tags, show data in root level 2011/10/10 create jsonlab project, start jsonlab website, add online documentation 2011/10/07 *speed up savejson by 25x using sprintf instead of mat2str, add options support 2011/10/06 *savejson works for structs, cells and arrays 2011/09/09 derive loadjson from JSON parser from MATLAB Central, draft savejson.m ================================================ FILE: machine-learning-ex1/ex1/lib/jsonlab/LICENSE_BSD.txt ================================================ Copyright 2011-2015 Qianqian Fang . All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. The views and conclusions contained in the software and documentation are those of the authors and should not be interpreted as representing official policies, either expressed or implied, of the copyright holders. ================================================ FILE: machine-learning-ex1/ex1/lib/jsonlab/README.txt ================================================ =============================================================================== = JSONLab = = An open-source MATLAB/Octave JSON encoder and decoder = =============================================================================== *Copyright (C) 2011-2015 Qianqian Fang *License: BSD License, see License_BSD.txt for details *Version: 1.0 (Optimus - Final) ------------------------------------------------------------------------------- Table of Content: I. Introduction II. Installation III.Using JSONLab IV. Known Issues and TODOs V. Contribution and feedback ------------------------------------------------------------------------------- I. Introduction JSON ([http://www.json.org/ JavaScript Object Notation]) is a highly portable, human-readable and "[http://en.wikipedia.org/wiki/JSON fat-free]" text format to represent complex and hierarchical data. It is as powerful as [http://en.wikipedia.org/wiki/XML XML], but less verbose. JSON format is widely used for data-exchange in applications, and is essential for the wild success of [http://en.wikipedia.org/wiki/Ajax_(programming) Ajax] and [http://en.wikipedia.org/wiki/Web_2.0 Web2.0]. UBJSON (Universal Binary JSON) is a binary JSON format, specifically optimized for compact file size and better performance while keeping the semantics as simple as the text-based JSON format. Using the UBJSON format allows to wrap complex binary data in a flexible and extensible structure, making it possible to process complex and large dataset without accuracy loss due to text conversions. We envision that both JSON and its binary version will serve as part of the mainstream data-exchange formats for scientific research in the future. It will provide the flexibility and generality achieved by other popular general-purpose file specifications, such as [http://www.hdfgroup.org/HDF5/whatishdf5.html HDF5], with significantly reduced complexity and enhanced performance. JSONLab is a free and open-source implementation of a JSON/UBJSON encoder and a decoder in the native MATLAB language. It can be used to convert a MATLAB data structure (array, struct, cell, struct array and cell array) into JSON/UBJSON formatted strings, or to decode a JSON/UBJSON file into MATLAB data structure. JSONLab supports both MATLAB and [http://www.gnu.org/software/octave/ GNU Octave] (a free MATLAB clone). ------------------------------------------------------------------------------- II. Installation The installation of JSONLab is no different than any other simple MATLAB toolbox. You only need to download/unzip the JSONLab package to a folder, and add the folder's path to MATLAB/Octave's path list by using the following command: addpath('/path/to/jsonlab'); If you want to add this path permanently, you need to type "pathtool", browse to the jsonlab root folder and add to the list, then click "Save". Then, run "rehash" in MATLAB, and type "which loadjson", if you see an output, that means JSONLab is installed for MATLAB/Octave. ------------------------------------------------------------------------------- III.Using JSONLab JSONLab provides two functions, loadjson.m -- a MATLAB->JSON decoder, and savejson.m -- a MATLAB->JSON encoder, for the text-based JSON, and two equivallent functions -- loadubjson and saveubjson for the binary JSON. The detailed help info for the four functions can be found below: === loadjson.m === 
  data=loadjson(fname,opt)
     or
  data=loadjson(fname,'param1',value1,'param2',value2,...)
 
  parse a JSON (JavaScript Object Notation) file or string
 
  authors:Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2011/09/09, including previous works from 
 
          Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713
             created on 2009/11/02
          Franois Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393
             created on  2009/03/22
          Joel Feenstra:
          http://www.mathworks.com/matlabcentral/fileexchange/20565
             created on 2008/07/03
 
  $Id: loadjson.m 452 2014-11-22 16:43:33Z fangq $
 
  input:
       fname: input file name, if fname contains "{}" or "[]", fname
              will be interpreted as a JSON string
       opt: a struct to store parsing options, opt can be replaced by 
            a list of ('param',value) pairs - the param string is equivallent
            to a field in opt. opt can have the following 
            fields (first in [.|.] is the default)
 
            opt.SimplifyCell [0|1]: if set to 1, loadjson will call cell2mat
                          for each element of the JSON data, and group 
                          arrays based on the cell2mat rules.
            opt.FastArrayParser [1|0 or integer]: if set to 1, use a
                          speed-optimized array parser when loading an 
                          array object. The fast array parser may 
                          collapse block arrays into a single large
                          array similar to rules defined in cell2mat; 0 to 
                          use a legacy parser; if set to a larger-than-1
                          value, this option will specify the minimum
                          dimension to enable the fast array parser. For
                          example, if the input is a 3D array, setting
                          FastArrayParser to 1 will return a 3D array;
                          setting to 2 will return a cell array of 2D
                          arrays; setting to 3 will return to a 2D cell
                          array of 1D vectors; setting to 4 will return a
                          3D cell array.
            opt.ShowProgress [0|1]: if set to 1, loadjson displays a progress bar.
 
  output:
       dat: a cell array, where {...} blocks are converted into cell arrays,
            and [...] are converted to arrays
 
  examples:
       dat=loadjson('{"obj":{"string":"value","array":[1,2,3]}}')
       dat=loadjson(['examples' filesep 'example1.json'])
       dat=loadjson(['examples' filesep 'example1.json'],'SimplifyCell',1)
=== savejson.m === 
  json=savejson(rootname,obj,filename)
     or
  json=savejson(rootname,obj,opt)
  json=savejson(rootname,obj,'param1',value1,'param2',value2,...)
 
  convert a MATLAB object (cell, struct or array) into a JSON (JavaScript
  Object Notation) string
 
  author: Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2011/09/09
 
  $Id: savejson.m 458 2014-12-19 22:17:17Z fangq $
 
  input:
       rootname: the name of the root-object, when set to '', the root name
         is ignored, however, when opt.ForceRootName is set to 1 (see below),
         the MATLAB variable name will be used as the root name.
       obj: a MATLAB object (array, cell, cell array, struct, struct array).
       filename: a string for the file name to save the output JSON data.
       opt: a struct for additional options, ignore to use default values.
         opt can have the following fields (first in [.|.] is the default)
 
         opt.FileName [''|string]: a file name to save the output JSON data
         opt.FloatFormat ['%.10g'|string]: format to show each numeric element
                          of a 1D/2D array;
         opt.ArrayIndent [1|0]: if 1, output explicit data array with
                          precedent indentation; if 0, no indentation
         opt.ArrayToStruct[0|1]: when set to 0, savejson outputs 1D/2D
                          array in JSON array format; if sets to 1, an
                          array will be shown as a struct with fields
                          "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for
                          sparse arrays, the non-zero elements will be
                          saved to _ArrayData_ field in triplet-format i.e.
                          (ix,iy,val) and "_ArrayIsSparse_" will be added
                          with a value of 1; for a complex array, the 
                          _ArrayData_ array will include two columns 
                          (4 for sparse) to record the real and imaginary 
                          parts, and also "_ArrayIsComplex_":1 is added. 
         opt.ParseLogical [0|1]: if this is set to 1, logical array elem
                          will use true/false rather than 1/0.
         opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single
                          numerical element will be shown without a square
                          bracket, unless it is the root object; if 0, square
                          brackets are forced for any numerical arrays.
         opt.ForceRootName [0|1]: when set to 1 and rootname is empty, savejson
                          will use the name of the passed obj variable as the 
                          root object name; if obj is an expression and 
                          does not have a name, 'root' will be used; if this 
                          is set to 0 and rootname is empty, the root level 
                          will be merged down to the lower level.
         opt.Inf ['"$1_Inf_"'|string]: a customized regular expression pattern
                          to represent +/-Inf. The matched pattern is '([-+]*)Inf'
                          and $1 represents the sign. For those who want to use
                          1e999 to represent Inf, they can set opt.Inf to '$11e999'
         opt.NaN ['"_NaN_"'|string]: a customized regular expression pattern
                          to represent NaN
         opt.JSONP [''|string]: to generate a JSONP output (JSON with padding),
                          for example, if opt.JSONP='foo', the JSON data is
                          wrapped inside a function call as 'foo(...);'
         opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson 
                          back to the string form
         opt.SaveBinary [0|1]: 1 - save the JSON file in binary mode; 0 - text mode.
         opt.Compact [0|1]: 1- out compact JSON format (remove all newlines and tabs)
 
         opt can be replaced by a list of ('param',value) pairs. The param 
         string is equivallent to a field in opt and is case sensitive.
  output:
       json: a string in the JSON format (see http://json.org)
 
  examples:
       jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... 
                'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],...
                'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;...
                           2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],...
                'MeshCreator','FangQ','MeshTitle','T6 Cube',...
                'SpecialData',[nan, inf, -inf]);
       savejson('jmesh',jsonmesh)
       savejson('',jsonmesh,'ArrayIndent',0,'FloatFormat','\t%.5g')
=== loadubjson.m === 
  data=loadubjson(fname,opt)
     or
  data=loadubjson(fname,'param1',value1,'param2',value2,...)
 
  parse a JSON (JavaScript Object Notation) file or string
 
  authors:Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2013/08/01
 
  $Id: loadubjson.m 436 2014-08-05 20:51:40Z fangq $
 
  input:
       fname: input file name, if fname contains "{}" or "[]", fname
              will be interpreted as a UBJSON string
       opt: a struct to store parsing options, opt can be replaced by 
            a list of ('param',value) pairs - the param string is equivallent
            to a field in opt. opt can have the following 
            fields (first in [.|.] is the default)
 
            opt.SimplifyCell [0|1]: if set to 1, loadubjson will call cell2mat
                          for each element of the JSON data, and group 
                          arrays based on the cell2mat rules.
            opt.IntEndian [B|L]: specify the endianness of the integer fields
                          in the UBJSON input data. B - Big-Endian format for 
                          integers (as required in the UBJSON specification); 
                          L - input integer fields are in Little-Endian order.
 
  output:
       dat: a cell array, where {...} blocks are converted into cell arrays,
            and [...] are converted to arrays
 
  examples:
       obj=struct('string','value','array',[1 2 3]);
       ubjdata=saveubjson('obj',obj);
       dat=loadubjson(ubjdata)
       dat=loadubjson(['examples' filesep 'example1.ubj'])
       dat=loadubjson(['examples' filesep 'example1.ubj'],'SimplifyCell',1)
=== saveubjson.m === 
  json=saveubjson(rootname,obj,filename)
     or
  json=saveubjson(rootname,obj,opt)
  json=saveubjson(rootname,obj,'param1',value1,'param2',value2,...)
 
  convert a MATLAB object (cell, struct or array) into a Universal 
  Binary JSON (UBJSON) binary string
 
  author: Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2013/08/17
 
  $Id: saveubjson.m 440 2014-09-17 19:59:45Z fangq $
 
  input:
       rootname: the name of the root-object, when set to '', the root name
         is ignored, however, when opt.ForceRootName is set to 1 (see below),
         the MATLAB variable name will be used as the root name.
       obj: a MATLAB object (array, cell, cell array, struct, struct array)
       filename: a string for the file name to save the output UBJSON data
       opt: a struct for additional options, ignore to use default values.
         opt can have the following fields (first in [.|.] is the default)
 
         opt.FileName [''|string]: a file name to save the output JSON data
         opt.ArrayToStruct[0|1]: when set to 0, saveubjson outputs 1D/2D
                          array in JSON array format; if sets to 1, an
                          array will be shown as a struct with fields
                          "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for
                          sparse arrays, the non-zero elements will be
                          saved to _ArrayData_ field in triplet-format i.e.
                          (ix,iy,val) and "_ArrayIsSparse_" will be added
                          with a value of 1; for a complex array, the 
                          _ArrayData_ array will include two columns 
                          (4 for sparse) to record the real and imaginary 
                          parts, and also "_ArrayIsComplex_":1 is added. 
         opt.ParseLogical [1|0]: if this is set to 1, logical array elem
                          will use true/false rather than 1/0.
         opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single
                          numerical element will be shown without a square
                          bracket, unless it is the root object; if 0, square
                          brackets are forced for any numerical arrays.
         opt.ForceRootName [0|1]: when set to 1 and rootname is empty, saveubjson
                          will use the name of the passed obj variable as the 
                          root object name; if obj is an expression and 
                          does not have a name, 'root' will be used; if this 
                          is set to 0 and rootname is empty, the root level 
                          will be merged down to the lower level.
         opt.JSONP [''|string]: to generate a JSONP output (JSON with padding),
                          for example, if opt.JSON='foo', the JSON data is
                          wrapped inside a function call as 'foo(...);'
         opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson 
                          back to the string form
 
         opt can be replaced by a list of ('param',value) pairs. The param 
         string is equivallent to a field in opt and is case sensitive.
  output:
       json: a binary string in the UBJSON format (see http://ubjson.org)
 
  examples:
       jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... 
                'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],...
                'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;...
                           2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],...
                'MeshCreator','FangQ','MeshTitle','T6 Cube',...
                'SpecialData',[nan, inf, -inf]);
       saveubjson('jsonmesh',jsonmesh)
       saveubjson('jsonmesh',jsonmesh,'meshdata.ubj')
=== examples === Under the "examples" folder, you can find several scripts to demonstrate the basic utilities of JSONLab. Running the "demo_jsonlab_basic.m" script, you will see the conversions from MATLAB data structure to JSON text and backward. In "jsonlab_selftest.m", we load complex JSON files downloaded from the Internet and validate the loadjson/savejson functions for regression testing purposes. Similarly, a "demo_ubjson_basic.m" script is provided to test the saveubjson and loadubjson pairs for various matlab data structures. Please run these examples and understand how JSONLab works before you use it to process your data. ------------------------------------------------------------------------------- IV. Known Issues and TODOs JSONLab has several known limitations. We are striving to make it more general and robust. Hopefully in a few future releases, the limitations become less. Here are the known issues: # 3D or higher dimensional cell/struct-arrays will be converted to 2D arrays; # When processing names containing multi-byte characters, Octave and MATLAB \ can give different field-names; you can use feature('DefaultCharacterSet','latin1') \ in MATLAB to get consistant results # savejson can not handle class and dataset. # saveubjson converts a logical array into a uint8 ([U]) array # an unofficial N-D array count syntax is implemented in saveubjson. We are \ actively communicating with the UBJSON spec maintainer to investigate the \ possibility of making it upstream # loadubjson can not parse all UBJSON Specification (Draft 9) compliant \ files, however, it can parse all UBJSON files produced by saveubjson. ------------------------------------------------------------------------------- V. Contribution and feedback JSONLab is an open-source project. This means you can not only use it and modify it as you wish, but also you can contribute your changes back to JSONLab so that everyone else can enjoy the improvement. For anyone who want to contribute, please download JSONLab source code from it's subversion repository by using the following command: svn checkout svn://svn.code.sf.net/p/iso2mesh/code/trunk/jsonlab jsonlab You can make changes to the files as needed. Once you are satisfied with your changes, and ready to share it with others, please cd the root directory of JSONLab, and type svn diff > yourname_featurename.patch You then email the .patch file to JSONLab's maintainer, Qianqian Fang, at the email address shown in the beginning of this file. Qianqian will review the changes and commit it to the subversion if they are satisfactory. We appreciate any suggestions and feedbacks from you. Please use iso2mesh's mailing list to report any questions you may have with JSONLab: http://groups.google.com/group/iso2mesh-users?hl=en&pli=1 (Subscription to the mailing list is needed in order to post messages). ================================================ FILE: machine-learning-ex1/ex1/lib/jsonlab/jsonopt.m ================================================ function val=jsonopt(key,default,varargin) % % val=jsonopt(key,default,optstruct) % % setting options based on a struct. The struct can be produced % by varargin2struct from a list of 'param','value' pairs % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % % $Id: loadjson.m 371 2012-06-20 12:43:06Z fangq $ % % input: % key: a string with which one look up a value from a struct % default: if the key does not exist, return default % optstruct: a struct where each sub-field is a key % % output: % val: if key exists, val=optstruct.key; otherwise val=default % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % val=default; if(nargin<=2) return; end opt=varargin{1}; if(isstruct(opt) && isfield(opt,key)) val=getfield(opt,key); end ================================================ FILE: machine-learning-ex1/ex1/lib/jsonlab/loadjson.m ================================================ function data = loadjson(fname,varargin) % % data=loadjson(fname,opt) % or % data=loadjson(fname,'param1',value1,'param2',value2,...) % % parse a JSON (JavaScript Object Notation) file or string % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2011/09/09, including previous works from % % Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713 % created on 2009/11/02 % François Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393 % created on 2009/03/22 % Joel Feenstra: % http://www.mathworks.com/matlabcentral/fileexchange/20565 % created on 2008/07/03 % % $Id: loadjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % fname: input file name, if fname contains "{}" or "[]", fname % will be interpreted as a JSON string % opt: a struct to store parsing options, opt can be replaced by % a list of ('param',value) pairs - the param string is equivallent % to a field in opt. opt can have the following % fields (first in [.|.] is the default) % % opt.SimplifyCell [0|1]: if set to 1, loadjson will call cell2mat % for each element of the JSON data, and group % arrays based on the cell2mat rules. % opt.FastArrayParser [1|0 or integer]: if set to 1, use a % speed-optimized array parser when loading an % array object. The fast array parser may % collapse block arrays into a single large % array similar to rules defined in cell2mat; 0 to % use a legacy parser; if set to a larger-than-1 % value, this option will specify the minimum % dimension to enable the fast array parser. For % example, if the input is a 3D array, setting % FastArrayParser to 1 will return a 3D array; % setting to 2 will return a cell array of 2D % arrays; setting to 3 will return to a 2D cell % array of 1D vectors; setting to 4 will return a % 3D cell array. % opt.ShowProgress [0|1]: if set to 1, loadjson displays a progress bar. % % output: % dat: a cell array, where {...} blocks are converted into cell arrays, % and [...] are converted to arrays % % examples: % dat=loadjson('{"obj":{"string":"value","array":[1,2,3]}}') % dat=loadjson(['examples' filesep 'example1.json']) % dat=loadjson(['examples' filesep 'example1.json'],'SimplifyCell',1) % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % global pos inStr len esc index_esc len_esc isoct arraytoken if(regexp(fname,'[\{\}\]\[]','once')) string=fname; elseif(exist(fname,'file')) fid = fopen(fname,'rb'); string = fread(fid,inf,'uint8=>char')'; fclose(fid); else error('input file does not exist'); end pos = 1; len = length(string); inStr = string; isoct=exist('OCTAVE_VERSION','builtin'); arraytoken=find(inStr=='[' | inStr==']' | inStr=='"'); jstr=regexprep(inStr,'\\\\',' '); escquote=regexp(jstr,'\\"'); arraytoken=sort([arraytoken escquote]); % String delimiters and escape chars identified to improve speed: esc = find(inStr=='"' | inStr=='\' ); % comparable to: regexp(inStr, '["\\]'); index_esc = 1; len_esc = length(esc); opt=varargin2struct(varargin{:}); if(jsonopt('ShowProgress',0,opt)==1) opt.progressbar_=waitbar(0,'loading ...'); end jsoncount=1; while pos <= len switch(next_char) case '{' data{jsoncount} = parse_object(opt); case '[' data{jsoncount} = parse_array(opt); otherwise error_pos('Outer level structure must be an object or an array'); end jsoncount=jsoncount+1; end % while jsoncount=length(data); if(jsoncount==1 && iscell(data)) data=data{1}; end if(~isempty(data)) if(isstruct(data)) % data can be a struct array data=jstruct2array(data); elseif(iscell(data)) data=jcell2array(data); end end if(isfield(opt,'progressbar_')) close(opt.progressbar_); end %% function newdata=jcell2array(data) len=length(data); newdata=data; for i=1:len if(isstruct(data{i})) newdata{i}=jstruct2array(data{i}); elseif(iscell(data{i})) newdata{i}=jcell2array(data{i}); end end %%------------------------------------------------------------------------- function newdata=jstruct2array(data) fn=fieldnames(data); newdata=data; len=length(data); for i=1:length(fn) % depth-first for j=1:len if(isstruct(getfield(data(j),fn{i}))) newdata(j)=setfield(newdata(j),fn{i},jstruct2array(getfield(data(j),fn{i}))); end end end if(~isempty(strmatch('x0x5F_ArrayType_',fn)) && ~isempty(strmatch('x0x5F_ArrayData_',fn))) newdata=cell(len,1); for j=1:len ndata=cast(data(j).x0x5F_ArrayData_,data(j).x0x5F_ArrayType_); iscpx=0; if(~isempty(strmatch('x0x5F_ArrayIsComplex_',fn))) if(data(j).x0x5F_ArrayIsComplex_) iscpx=1; end end if(~isempty(strmatch('x0x5F_ArrayIsSparse_',fn))) if(data(j).x0x5F_ArrayIsSparse_) if(~isempty(strmatch('x0x5F_ArraySize_',fn))) dim=data(j).x0x5F_ArraySize_; if(iscpx && size(ndata,2)==4-any(dim==1)) ndata(:,end-1)=complex(ndata(:,end-1),ndata(:,end)); end if isempty(ndata) % All-zeros sparse ndata=sparse(dim(1),prod(dim(2:end))); elseif dim(1)==1 % Sparse row vector ndata=sparse(1,ndata(:,1),ndata(:,2),dim(1),prod(dim(2:end))); elseif dim(2)==1 % Sparse column vector ndata=sparse(ndata(:,1),1,ndata(:,2),dim(1),prod(dim(2:end))); else % Generic sparse array. ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3),dim(1),prod(dim(2:end))); end else if(iscpx && size(ndata,2)==4) ndata(:,3)=complex(ndata(:,3),ndata(:,4)); end ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3)); end end elseif(~isempty(strmatch('x0x5F_ArraySize_',fn))) if(iscpx && size(ndata,2)==2) ndata=complex(ndata(:,1),ndata(:,2)); end ndata=reshape(ndata(:),data(j).x0x5F_ArraySize_); end newdata{j}=ndata; end if(len==1) newdata=newdata{1}; end end %%------------------------------------------------------------------------- function object = parse_object(varargin) parse_char('{'); object = []; if next_char ~= '}' while 1 str = parseStr(varargin{:}); if isempty(str) error_pos('Name of value at position %d cannot be empty'); end parse_char(':'); val = parse_value(varargin{:}); eval( sprintf( 'object.%s = val;', valid_field(str) ) ); if next_char == '}' break; end parse_char(','); end end parse_char('}'); %%------------------------------------------------------------------------- function object = parse_array(varargin) % JSON array is written in row-major order global pos inStr isoct parse_char('['); object = cell(0, 1); dim2=[]; arraydepth=jsonopt('JSONLAB_ArrayDepth_',1,varargin{:}); pbar=jsonopt('progressbar_',-1,varargin{:}); if next_char ~= ']' if(jsonopt('FastArrayParser',1,varargin{:})>=1 && arraydepth>=jsonopt('FastArrayParser',1,varargin{:})) [endpos, e1l, e1r, maxlevel]=matching_bracket(inStr,pos); arraystr=['[' inStr(pos:endpos)]; arraystr=regexprep(arraystr,'"_NaN_"','NaN'); arraystr=regexprep(arraystr,'"([-+]*)_Inf_"','$1Inf'); arraystr(arraystr==sprintf('\n'))=[]; arraystr(arraystr==sprintf('\r'))=[]; %arraystr=regexprep(arraystr,'\s*,',','); % this is slow,sometimes needed if(~isempty(e1l) && ~isempty(e1r)) % the array is in 2D or higher D astr=inStr((e1l+1):(e1r-1)); astr=regexprep(astr,'"_NaN_"','NaN'); astr=regexprep(astr,'"([-+]*)_Inf_"','$1Inf'); astr(astr==sprintf('\n'))=[]; astr(astr==sprintf('\r'))=[]; astr(astr==' ')=''; if(isempty(find(astr=='[', 1))) % array is 2D dim2=length(sscanf(astr,'%f,',[1 inf])); end else % array is 1D astr=arraystr(2:end-1); astr(astr==' ')=''; [obj, count, errmsg, nextidx]=sscanf(astr,'%f,',[1,inf]); if(nextidx>=length(astr)-1) object=obj; pos=endpos; parse_char(']'); return; end end if(~isempty(dim2)) astr=arraystr; astr(astr=='[')=''; astr(astr==']')=''; astr(astr==' ')=''; [obj, count, errmsg, nextidx]=sscanf(astr,'%f,',inf); if(nextidx>=length(astr)-1) object=reshape(obj,dim2,numel(obj)/dim2)'; pos=endpos; parse_char(']'); if(pbar>0) waitbar(pos/length(inStr),pbar,'loading ...'); end return; end end arraystr=regexprep(arraystr,'\]\s*,','];'); else arraystr='['; end try if(isoct && regexp(arraystr,'"','once')) error('Octave eval can produce empty cells for JSON-like input'); end object=eval(arraystr); pos=endpos; catch while 1 newopt=varargin2struct(varargin{:},'JSONLAB_ArrayDepth_',arraydepth+1); val = parse_value(newopt); object{end+1} = val; if next_char == ']' break; end parse_char(','); end end end if(jsonopt('SimplifyCell',0,varargin{:})==1) try oldobj=object; object=cell2mat(object')'; if(iscell(oldobj) && isstruct(object) && numel(object)>1 && jsonopt('SimplifyCellArray',1,varargin{:})==0) object=oldobj; elseif(size(object,1)>1 && ndims(object)==2) object=object'; end catch end end parse_char(']'); if(pbar>0) waitbar(pos/length(inStr),pbar,'loading ...'); end %%------------------------------------------------------------------------- function parse_char(c) global pos inStr len skip_whitespace; if pos > len || inStr(pos) ~= c error_pos(sprintf('Expected %c at position %%d', c)); else pos = pos + 1; skip_whitespace; end %%------------------------------------------------------------------------- function c = next_char global pos inStr len skip_whitespace; if pos > len c = []; else c = inStr(pos); end %%------------------------------------------------------------------------- function skip_whitespace global pos inStr len while pos <= len && isspace(inStr(pos)) pos = pos + 1; end %%------------------------------------------------------------------------- function str = parseStr(varargin) global pos inStr len esc index_esc len_esc % len, ns = length(inStr), keyboard if inStr(pos) ~= '"' error_pos('String starting with " expected at position %d'); else pos = pos + 1; end str = ''; while pos <= len while index_esc <= len_esc && esc(index_esc) < pos index_esc = index_esc + 1; end if index_esc > len_esc str = [str inStr(pos:len)]; pos = len + 1; break; else str = [str inStr(pos:esc(index_esc)-1)]; pos = esc(index_esc); end nstr = length(str); switch inStr(pos) case '"' pos = pos + 1; if(~isempty(str)) if(strcmp(str,'_Inf_')) str=Inf; elseif(strcmp(str,'-_Inf_')) str=-Inf; elseif(strcmp(str,'_NaN_')) str=NaN; end end return; case '\' if pos+1 > len error_pos('End of file reached right after escape character'); end pos = pos + 1; switch inStr(pos) case {'"' '\' '/'} str(nstr+1) = inStr(pos); pos = pos + 1; case {'b' 'f' 'n' 'r' 't'} str(nstr+1) = sprintf(['\' inStr(pos)]); pos = pos + 1; case 'u' if pos+4 > len error_pos('End of file reached in escaped unicode character'); end str(nstr+(1:6)) = inStr(pos-1:pos+4); pos = pos + 5; end otherwise % should never happen str(nstr+1) = inStr(pos), keyboard pos = pos + 1; end end error_pos('End of file while expecting end of inStr'); %%------------------------------------------------------------------------- function num = parse_number(varargin) global pos inStr len isoct currstr=inStr(pos:end); numstr=0; if(isoct~=0) numstr=regexp(currstr,'^\s*-?(?:0|[1-9]\d*)(?:\.\d+)?(?:[eE][+\-]?\d+)?','end'); [num, one] = sscanf(currstr, '%f', 1); delta=numstr+1; else [num, one, err, delta] = sscanf(currstr, '%f', 1); if ~isempty(err) error_pos('Error reading number at position %d'); end end pos = pos + delta-1; %%------------------------------------------------------------------------- function val = parse_value(varargin) global pos inStr len true = 1; false = 0; pbar=jsonopt('progressbar_',-1,varargin{:}); if(pbar>0) waitbar(pos/len,pbar,'loading ...'); end switch(inStr(pos)) case '"' val = parseStr(varargin{:}); return; case '[' val = parse_array(varargin{:}); return; case '{' val = parse_object(varargin{:}); if isstruct(val) if(~isempty(strmatch('x0x5F_ArrayType_',fieldnames(val), 'exact'))) val=jstruct2array(val); end elseif isempty(val) val = struct; end return; case {'-','0','1','2','3','4','5','6','7','8','9'} val = parse_number(varargin{:}); return; case 't' if pos+3 <= len && strcmpi(inStr(pos:pos+3), 'true') val = true; pos = pos + 4; return; end case 'f' if pos+4 <= len && strcmpi(inStr(pos:pos+4), 'false') val = false; pos = pos + 5; return; end case 'n' if pos+3 <= len && strcmpi(inStr(pos:pos+3), 'null') val = []; pos = pos + 4; return; end end error_pos('Value expected at position %d'); %%------------------------------------------------------------------------- function error_pos(msg) global pos inStr len poShow = max(min([pos-15 pos-1 pos pos+20],len),1); if poShow(3) == poShow(2) poShow(3:4) = poShow(2)+[0 -1]; % display nothing after end msg = [sprintf(msg, pos) ': ' ... inStr(poShow(1):poShow(2)) '' inStr(poShow(3):poShow(4)) ]; error( ['JSONparser:invalidFormat: ' msg] ); %%------------------------------------------------------------------------- function str = valid_field(str) global isoct % From MATLAB doc: field names must begin with a letter, which may be % followed by any combination of letters, digits, and underscores. % Invalid characters will be converted to underscores, and the prefix % "x0x[Hex code]_" will be added if the first character is not a letter. pos=regexp(str,'^[^A-Za-z]','once'); if(~isempty(pos)) if(~isoct) str=regexprep(str,'^([^A-Za-z])','x0x${sprintf(''%X'',unicode2native($1))}_','once'); else str=sprintf('x0x%X_%s',char(str(1)),str(2:end)); end end if(isempty(regexp(str,'[^0-9A-Za-z_]', 'once' ))) return; end if(~isoct) str=regexprep(str,'([^0-9A-Za-z_])','_0x${sprintf(''%X'',unicode2native($1))}_'); else pos=regexp(str,'[^0-9A-Za-z_]'); if(isempty(pos)) return; end str0=str; pos0=[0 pos(:)' length(str)]; str=''; for i=1:length(pos) str=[str str0(pos0(i)+1:pos(i)-1) sprintf('_0x%X_',str0(pos(i)))]; end if(pos(end)~=length(str)) str=[str str0(pos0(end-1)+1:pos0(end))]; end end %str(~isletter(str) & ~('0' <= str & str <= '9')) = '_'; %%------------------------------------------------------------------------- function endpos = matching_quote(str,pos) len=length(str); while(pos1 && str(pos-1)=='\')) endpos=pos; return; end end pos=pos+1; end error('unmatched quotation mark'); %%------------------------------------------------------------------------- function [endpos, e1l, e1r, maxlevel] = matching_bracket(str,pos) global arraytoken level=1; maxlevel=level; endpos=0; bpos=arraytoken(arraytoken>=pos); tokens=str(bpos); len=length(tokens); pos=1; e1l=[]; e1r=[]; while(pos<=len) c=tokens(pos); if(c==']') level=level-1; if(isempty(e1r)) e1r=bpos(pos); end if(level==0) endpos=bpos(pos); return end end if(c=='[') if(isempty(e1l)) e1l=bpos(pos); end level=level+1; maxlevel=max(maxlevel,level); end if(c=='"') pos=matching_quote(tokens,pos+1); end pos=pos+1; end if(endpos==0) error('unmatched "]"'); end ================================================ FILE: machine-learning-ex1/ex1/lib/jsonlab/loadubjson.m ================================================ function data = loadubjson(fname,varargin) % % data=loadubjson(fname,opt) % or % data=loadubjson(fname,'param1',value1,'param2',value2,...) % % parse a JSON (JavaScript Object Notation) file or string % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2013/08/01 % % $Id: loadubjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % fname: input file name, if fname contains "{}" or "[]", fname % will be interpreted as a UBJSON string % opt: a struct to store parsing options, opt can be replaced by % a list of ('param',value) pairs - the param string is equivallent % to a field in opt. opt can have the following % fields (first in [.|.] is the default) % % opt.SimplifyCell [0|1]: if set to 1, loadubjson will call cell2mat % for each element of the JSON data, and group % arrays based on the cell2mat rules. % opt.IntEndian [B|L]: specify the endianness of the integer fields % in the UBJSON input data. B - Big-Endian format for % integers (as required in the UBJSON specification); % L - input integer fields are in Little-Endian order. % % output: % dat: a cell array, where {...} blocks are converted into cell arrays, % and [...] are converted to arrays % % examples: % obj=struct('string','value','array',[1 2 3]); % ubjdata=saveubjson('obj',obj); % dat=loadubjson(ubjdata) % dat=loadubjson(['examples' filesep 'example1.ubj']) % dat=loadubjson(['examples' filesep 'example1.ubj'],'SimplifyCell',1) % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % global pos inStr len esc index_esc len_esc isoct arraytoken fileendian systemendian if(regexp(fname,'[\{\}\]\[]','once')) string=fname; elseif(exist(fname,'file')) fid = fopen(fname,'rb'); string = fread(fid,inf,'uint8=>char')'; fclose(fid); else error('input file does not exist'); end pos = 1; len = length(string); inStr = string; isoct=exist('OCTAVE_VERSION','builtin'); arraytoken=find(inStr=='[' | inStr==']' | inStr=='"'); jstr=regexprep(inStr,'\\\\',' '); escquote=regexp(jstr,'\\"'); arraytoken=sort([arraytoken escquote]); % String delimiters and escape chars identified to improve speed: esc = find(inStr=='"' | inStr=='\' ); % comparable to: regexp(inStr, '["\\]'); index_esc = 1; len_esc = length(esc); opt=varargin2struct(varargin{:}); fileendian=upper(jsonopt('IntEndian','B',opt)); [os,maxelem,systemendian]=computer; jsoncount=1; while pos <= len switch(next_char) case '{' data{jsoncount} = parse_object(opt); case '[' data{jsoncount} = parse_array(opt); otherwise error_pos('Outer level structure must be an object or an array'); end jsoncount=jsoncount+1; end % while jsoncount=length(data); if(jsoncount==1 && iscell(data)) data=data{1}; end if(~isempty(data)) if(isstruct(data)) % data can be a struct array data=jstruct2array(data); elseif(iscell(data)) data=jcell2array(data); end end %% function newdata=parse_collection(id,data,obj) if(jsoncount>0 && exist('data','var')) if(~iscell(data)) newdata=cell(1); newdata{1}=data; data=newdata; end end %% function newdata=jcell2array(data) len=length(data); newdata=data; for i=1:len if(isstruct(data{i})) newdata{i}=jstruct2array(data{i}); elseif(iscell(data{i})) newdata{i}=jcell2array(data{i}); end end %%------------------------------------------------------------------------- function newdata=jstruct2array(data) fn=fieldnames(data); newdata=data; len=length(data); for i=1:length(fn) % depth-first for j=1:len if(isstruct(getfield(data(j),fn{i}))) newdata(j)=setfield(newdata(j),fn{i},jstruct2array(getfield(data(j),fn{i}))); end end end if(~isempty(strmatch('x0x5F_ArrayType_',fn)) && ~isempty(strmatch('x0x5F_ArrayData_',fn))) newdata=cell(len,1); for j=1:len ndata=cast(data(j).x0x5F_ArrayData_,data(j).x0x5F_ArrayType_); iscpx=0; if(~isempty(strmatch('x0x5F_ArrayIsComplex_',fn))) if(data(j).x0x5F_ArrayIsComplex_) iscpx=1; end end if(~isempty(strmatch('x0x5F_ArrayIsSparse_',fn))) if(data(j).x0x5F_ArrayIsSparse_) if(~isempty(strmatch('x0x5F_ArraySize_',fn))) dim=double(data(j).x0x5F_ArraySize_); if(iscpx && size(ndata,2)==4-any(dim==1)) ndata(:,end-1)=complex(ndata(:,end-1),ndata(:,end)); end if isempty(ndata) % All-zeros sparse ndata=sparse(dim(1),prod(dim(2:end))); elseif dim(1)==1 % Sparse row vector ndata=sparse(1,ndata(:,1),ndata(:,2),dim(1),prod(dim(2:end))); elseif dim(2)==1 % Sparse column vector ndata=sparse(ndata(:,1),1,ndata(:,2),dim(1),prod(dim(2:end))); else % Generic sparse array. ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3),dim(1),prod(dim(2:end))); end else if(iscpx && size(ndata,2)==4) ndata(:,3)=complex(ndata(:,3),ndata(:,4)); end ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3)); end end elseif(~isempty(strmatch('x0x5F_ArraySize_',fn))) if(iscpx && size(ndata,2)==2) ndata=complex(ndata(:,1),ndata(:,2)); end ndata=reshape(ndata(:),data(j).x0x5F_ArraySize_); end newdata{j}=ndata; end if(len==1) newdata=newdata{1}; end end %%------------------------------------------------------------------------- function object = parse_object(varargin) parse_char('{'); object = []; type=''; count=-1; if(next_char == '$') type=inStr(pos+1); % TODO pos=pos+2; end if(next_char == '#') pos=pos+1; count=double(parse_number()); end if next_char ~= '}' num=0; while 1 str = parseStr(varargin{:}); if isempty(str) error_pos('Name of value at position %d cannot be empty'); end %parse_char(':'); val = parse_value(varargin{:}); num=num+1; eval( sprintf( 'object.%s = val;', valid_field(str) ) ); if next_char == '}' || (count>=0 && num>=count) break; end %parse_char(','); end end if(count==-1) parse_char('}'); end %%------------------------------------------------------------------------- function [cid,len]=elem_info(type) id=strfind('iUIlLdD',type); dataclass={'int8','uint8','int16','int32','int64','single','double'}; bytelen=[1,1,2,4,8,4,8]; if(id>0) cid=dataclass{id}; len=bytelen(id); else error_pos('unsupported type at position %d'); end %%------------------------------------------------------------------------- function [data adv]=parse_block(type,count,varargin) global pos inStr isoct fileendian systemendian [cid,len]=elem_info(type); datastr=inStr(pos:pos+len*count-1); if(isoct) newdata=int8(datastr); else newdata=uint8(datastr); end id=strfind('iUIlLdD',type); if(id<=5 && fileendian~=systemendian) newdata=swapbytes(typecast(newdata,cid)); end data=typecast(newdata,cid); adv=double(len*count); %%------------------------------------------------------------------------- function object = parse_array(varargin) % JSON array is written in row-major order global pos inStr isoct parse_char('['); object = cell(0, 1); dim=[]; type=''; count=-1; if(next_char == '$') type=inStr(pos+1); pos=pos+2; end if(next_char == '#') pos=pos+1; if(next_char=='[') dim=parse_array(varargin{:}); count=prod(double(dim)); else count=double(parse_number()); end end if(~isempty(type)) if(count>=0) [object adv]=parse_block(type,count,varargin{:}); if(~isempty(dim)) object=reshape(object,dim); end pos=pos+adv; return; else endpos=matching_bracket(inStr,pos); [cid,len]=elem_info(type); count=(endpos-pos)/len; [object adv]=parse_block(type,count,varargin{:}); pos=pos+adv; parse_char(']'); return; end end if next_char ~= ']' while 1 val = parse_value(varargin{:}); object{end+1} = val; if next_char == ']' break; end %parse_char(','); end end if(jsonopt('SimplifyCell',0,varargin{:})==1) try oldobj=object; object=cell2mat(object')'; if(iscell(oldobj) && isstruct(object) && numel(object)>1 && jsonopt('SimplifyCellArray',1,varargin{:})==0) object=oldobj; elseif(size(object,1)>1 && ndims(object)==2) object=object'; end catch end end if(count==-1) parse_char(']'); end %%------------------------------------------------------------------------- function parse_char(c) global pos inStr len skip_whitespace; if pos > len || inStr(pos) ~= c error_pos(sprintf('Expected %c at position %%d', c)); else pos = pos + 1; skip_whitespace; end %%------------------------------------------------------------------------- function c = next_char global pos inStr len skip_whitespace; if pos > len c = []; else c = inStr(pos); end %%------------------------------------------------------------------------- function skip_whitespace global pos inStr len while pos <= len && isspace(inStr(pos)) pos = pos + 1; end %%------------------------------------------------------------------------- function str = parseStr(varargin) global pos inStr esc index_esc len_esc % len, ns = length(inStr), keyboard type=inStr(pos); if type ~= 'S' && type ~= 'C' && type ~= 'H' error_pos('String starting with S expected at position %d'); else pos = pos + 1; end if(type == 'C') str=inStr(pos); pos=pos+1; return; end bytelen=double(parse_number()); if(length(inStr)>=pos+bytelen-1) str=inStr(pos:pos+bytelen-1); pos=pos+bytelen; else error_pos('End of file while expecting end of inStr'); end %%------------------------------------------------------------------------- function num = parse_number(varargin) global pos inStr len isoct fileendian systemendian id=strfind('iUIlLdD',inStr(pos)); if(isempty(id)) error_pos('expecting a number at position %d'); end type={'int8','uint8','int16','int32','int64','single','double'}; bytelen=[1,1,2,4,8,4,8]; datastr=inStr(pos+1:pos+bytelen(id)); if(isoct) newdata=int8(datastr); else newdata=uint8(datastr); end if(id<=5 && fileendian~=systemendian) newdata=swapbytes(typecast(newdata,type{id})); end num=typecast(newdata,type{id}); pos = pos + bytelen(id)+1; %%------------------------------------------------------------------------- function val = parse_value(varargin) global pos inStr len true = 1; false = 0; switch(inStr(pos)) case {'S','C','H'} val = parseStr(varargin{:}); return; case '[' val = parse_array(varargin{:}); return; case '{' val = parse_object(varargin{:}); if isstruct(val) if(~isempty(strmatch('x0x5F_ArrayType_',fieldnames(val), 'exact'))) val=jstruct2array(val); end elseif isempty(val) val = struct; end return; case {'i','U','I','l','L','d','D'} val = parse_number(varargin{:}); return; case 'T' val = true; pos = pos + 1; return; case 'F' val = false; pos = pos + 1; return; case {'Z','N'} val = []; pos = pos + 1; return; end error_pos('Value expected at position %d'); %%------------------------------------------------------------------------- function error_pos(msg) global pos inStr len poShow = max(min([pos-15 pos-1 pos pos+20],len),1); if poShow(3) == poShow(2) poShow(3:4) = poShow(2)+[0 -1]; % display nothing after end msg = [sprintf(msg, pos) ': ' ... inStr(poShow(1):poShow(2)) '' inStr(poShow(3):poShow(4)) ]; error( ['JSONparser:invalidFormat: ' msg] ); %%------------------------------------------------------------------------- function str = valid_field(str) global isoct % From MATLAB doc: field names must begin with a letter, which may be % followed by any combination of letters, digits, and underscores. % Invalid characters will be converted to underscores, and the prefix % "x0x[Hex code]_" will be added if the first character is not a letter. pos=regexp(str,'^[^A-Za-z]','once'); if(~isempty(pos)) if(~isoct) str=regexprep(str,'^([^A-Za-z])','x0x${sprintf(''%X'',unicode2native($1))}_','once'); else str=sprintf('x0x%X_%s',char(str(1)),str(2:end)); end end if(isempty(regexp(str,'[^0-9A-Za-z_]', 'once' ))) return; end if(~isoct) str=regexprep(str,'([^0-9A-Za-z_])','_0x${sprintf(''%X'',unicode2native($1))}_'); else pos=regexp(str,'[^0-9A-Za-z_]'); if(isempty(pos)) return; end str0=str; pos0=[0 pos(:)' length(str)]; str=''; for i=1:length(pos) str=[str str0(pos0(i)+1:pos(i)-1) sprintf('_0x%X_',str0(pos(i)))]; end if(pos(end)~=length(str)) str=[str str0(pos0(end-1)+1:pos0(end))]; end end %str(~isletter(str) & ~('0' <= str & str <= '9')) = '_'; %%------------------------------------------------------------------------- function endpos = matching_quote(str,pos) len=length(str); while(pos1 && str(pos-1)=='\')) endpos=pos; return; end end pos=pos+1; end error('unmatched quotation mark'); %%------------------------------------------------------------------------- function [endpos e1l e1r maxlevel] = matching_bracket(str,pos) global arraytoken level=1; maxlevel=level; endpos=0; bpos=arraytoken(arraytoken>=pos); tokens=str(bpos); len=length(tokens); pos=1; e1l=[]; e1r=[]; while(pos<=len) c=tokens(pos); if(c==']') level=level-1; if(isempty(e1r)) e1r=bpos(pos); end if(level==0) endpos=bpos(pos); return end end if(c=='[') if(isempty(e1l)) e1l=bpos(pos); end level=level+1; maxlevel=max(maxlevel,level); end if(c=='"') pos=matching_quote(tokens,pos+1); end pos=pos+1; end if(endpos==0) error('unmatched "]"'); end ================================================ FILE: machine-learning-ex1/ex1/lib/jsonlab/mergestruct.m ================================================ function s=mergestruct(s1,s2) % % s=mergestruct(s1,s2) % % merge two struct objects into one % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % date: 2012/12/22 % % input: % s1,s2: a struct object, s1 and s2 can not be arrays % % output: % s: the merged struct object. fields in s1 and s2 will be combined in s. % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(~isstruct(s1) || ~isstruct(s2)) error('input parameters contain non-struct'); end if(length(s1)>1 || length(s2)>1) error('can not merge struct arrays'); end fn=fieldnames(s2); s=s1; for i=1:length(fn) s=setfield(s,fn{i},getfield(s2,fn{i})); end ================================================ FILE: machine-learning-ex1/ex1/lib/jsonlab/savejson.m ================================================ function json=savejson(rootname,obj,varargin) % % json=savejson(rootname,obj,filename) % or % json=savejson(rootname,obj,opt) % json=savejson(rootname,obj,'param1',value1,'param2',value2,...) % % convert a MATLAB object (cell, struct or array) into a JSON (JavaScript % Object Notation) string % % author: Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2011/09/09 % % $Id: savejson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % rootname: the name of the root-object, when set to '', the root name % is ignored, however, when opt.ForceRootName is set to 1 (see below), % the MATLAB variable name will be used as the root name. % obj: a MATLAB object (array, cell, cell array, struct, struct array). % filename: a string for the file name to save the output JSON data. % opt: a struct for additional options, ignore to use default values. % opt can have the following fields (first in [.|.] is the default) % % opt.FileName [''|string]: a file name to save the output JSON data % opt.FloatFormat ['%.10g'|string]: format to show each numeric element % of a 1D/2D array; % opt.ArrayIndent [1|0]: if 1, output explicit data array with % precedent indentation; if 0, no indentation % opt.ArrayToStruct[0|1]: when set to 0, savejson outputs 1D/2D % array in JSON array format; if sets to 1, an % array will be shown as a struct with fields % "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for % sparse arrays, the non-zero elements will be % saved to _ArrayData_ field in triplet-format i.e. % (ix,iy,val) and "_ArrayIsSparse_" will be added % with a value of 1; for a complex array, the % _ArrayData_ array will include two columns % (4 for sparse) to record the real and imaginary % parts, and also "_ArrayIsComplex_":1 is added. % opt.ParseLogical [0|1]: if this is set to 1, logical array elem % will use true/false rather than 1/0. % opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single % numerical element will be shown without a square % bracket, unless it is the root object; if 0, square % brackets are forced for any numerical arrays. % opt.ForceRootName [0|1]: when set to 1 and rootname is empty, savejson % will use the name of the passed obj variable as the % root object name; if obj is an expression and % does not have a name, 'root' will be used; if this % is set to 0 and rootname is empty, the root level % will be merged down to the lower level. % opt.Inf ['"$1_Inf_"'|string]: a customized regular expression pattern % to represent +/-Inf. The matched pattern is '([-+]*)Inf' % and $1 represents the sign. For those who want to use % 1e999 to represent Inf, they can set opt.Inf to '$11e999' % opt.NaN ['"_NaN_"'|string]: a customized regular expression pattern % to represent NaN % opt.JSONP [''|string]: to generate a JSONP output (JSON with padding), % for example, if opt.JSONP='foo', the JSON data is % wrapped inside a function call as 'foo(...);' % opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson % back to the string form % opt.SaveBinary [0|1]: 1 - save the JSON file in binary mode; 0 - text mode. % opt.Compact [0|1]: 1- out compact JSON format (remove all newlines and tabs) % % opt can be replaced by a list of ('param',value) pairs. The param % string is equivallent to a field in opt and is case sensitive. % output: % json: a string in the JSON format (see http://json.org) % % examples: % jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... % 'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],... % 'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;... % 2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],... % 'MeshCreator','FangQ','MeshTitle','T6 Cube',... % 'SpecialData',[nan, inf, -inf]); % savejson('jmesh',jsonmesh) % savejson('',jsonmesh,'ArrayIndent',0,'FloatFormat','\t%.5g') % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(nargin==1) varname=inputname(1); obj=rootname; if(isempty(varname)) varname='root'; end rootname=varname; else varname=inputname(2); end if(length(varargin)==1 && ischar(varargin{1})) opt=struct('FileName',varargin{1}); else opt=varargin2struct(varargin{:}); end opt.IsOctave=exist('OCTAVE_VERSION','builtin'); rootisarray=0; rootlevel=1; forceroot=jsonopt('ForceRootName',0,opt); if((isnumeric(obj) || islogical(obj) || ischar(obj) || isstruct(obj) || iscell(obj)) && isempty(rootname) && forceroot==0) rootisarray=1; rootlevel=0; else if(isempty(rootname)) rootname=varname; end end if((isstruct(obj) || iscell(obj))&& isempty(rootname) && forceroot) rootname='root'; end whitespaces=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); if(jsonopt('Compact',0,opt)==1) whitespaces=struct('tab','','newline','','sep',','); end if(~isfield(opt,'whitespaces_')) opt.whitespaces_=whitespaces; end nl=whitespaces.newline; json=obj2json(rootname,obj,rootlevel,opt); if(rootisarray) json=sprintf('%s%s',json,nl); else json=sprintf('{%s%s%s}\n',nl,json,nl); end jsonp=jsonopt('JSONP','',opt); if(~isempty(jsonp)) json=sprintf('%s(%s);%s',jsonp,json,nl); end % save to a file if FileName is set, suggested by Patrick Rapin if(~isempty(jsonopt('FileName','',opt))) if(jsonopt('SaveBinary',0,opt)==1) fid = fopen(opt.FileName, 'wb'); fwrite(fid,json); else fid = fopen(opt.FileName, 'wt'); fwrite(fid,json,'char'); end fclose(fid); end %%------------------------------------------------------------------------- function txt=obj2json(name,item,level,varargin) if(iscell(item)) txt=cell2json(name,item,level,varargin{:}); elseif(isstruct(item)) txt=struct2json(name,item,level,varargin{:}); elseif(ischar(item)) txt=str2json(name,item,level,varargin{:}); else txt=mat2json(name,item,level,varargin{:}); end %%------------------------------------------------------------------------- function txt=cell2json(name,item,level,varargin) txt=''; if(~iscell(item)) error('input is not a cell'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); ws=jsonopt('whitespaces_',struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')),varargin{:}); padding0=repmat(ws.tab,1,level); padding2=repmat(ws.tab,1,level+1); nl=ws.newline; if(len>1) if(~isempty(name)) txt=sprintf('%s"%s": [%s',padding0, checkname(name,varargin{:}),nl); name=''; else txt=sprintf('%s[%s',padding0,nl); end elseif(len==0) if(~isempty(name)) txt=sprintf('%s"%s": []',padding0, checkname(name,varargin{:})); name=''; else txt=sprintf('%s[]',padding0); end end for j=1:dim(2) if(dim(1)>1) txt=sprintf('%s%s[%s',txt,padding2,nl); end for i=1:dim(1) txt=sprintf('%s%s',txt,obj2json(name,item{i,j},level+(dim(1)>1)+1,varargin{:})); if(i1) txt=sprintf('%s%s%s]',txt,nl,padding2); end if(j1) txt=sprintf('%s%s%s]',txt,nl,padding0); end %%------------------------------------------------------------------------- function txt=struct2json(name,item,level,varargin) txt=''; if(~isstruct(item)) error('input is not a struct'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); ws=struct('tab',sprintf('\t'),'newline',sprintf('\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding0=repmat(ws.tab,1,level); padding2=repmat(ws.tab,1,level+1); padding1=repmat(ws.tab,1,level+(dim(1)>1)+(len>1)); nl=ws.newline; if(~isempty(name)) if(len>1) txt=sprintf('%s"%s": [%s',padding0,checkname(name,varargin{:}),nl); end else if(len>1) txt=sprintf('%s[%s',padding0,nl); end end for j=1:dim(2) if(dim(1)>1) txt=sprintf('%s%s[%s',txt,padding2,nl); end for i=1:dim(1) names = fieldnames(item(i,j)); if(~isempty(name) && len==1) txt=sprintf('%s%s"%s": {%s',txt,padding1, checkname(name,varargin{:}),nl); else txt=sprintf('%s%s{%s',txt,padding1,nl); end if(~isempty(names)) for e=1:length(names) txt=sprintf('%s%s',txt,obj2json(names{e},getfield(item(i,j),... names{e}),level+(dim(1)>1)+1+(len>1),varargin{:})); if(e1) txt=sprintf('%s%s%s]',txt,nl,padding2); end if(j1) txt=sprintf('%s%s%s]',txt,nl,padding0); end %%------------------------------------------------------------------------- function txt=str2json(name,item,level,varargin) txt=''; if(~ischar(item)) error('input is not a string'); end item=reshape(item, max(size(item),[1 0])); len=size(item,1); ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding1=repmat(ws.tab,1,level); padding0=repmat(ws.tab,1,level+1); nl=ws.newline; sep=ws.sep; if(~isempty(name)) if(len>1) txt=sprintf('%s"%s": [%s',padding1,checkname(name,varargin{:}),nl); end else if(len>1) txt=sprintf('%s[%s',padding1,nl); end end isoct=jsonopt('IsOctave',0,varargin{:}); for e=1:len if(isoct) val=regexprep(item(e,:),'\\','\\'); val=regexprep(val,'"','\"'); val=regexprep(val,'^"','\"'); else val=regexprep(item(e,:),'\\','\\\\'); val=regexprep(val,'"','\\"'); val=regexprep(val,'^"','\\"'); end val=escapejsonstring(val); if(len==1) obj=['"' checkname(name,varargin{:}) '": ' '"',val,'"']; if(isempty(name)) obj=['"',val,'"']; end txt=sprintf('%s%s%s%s',txt,padding1,obj); else txt=sprintf('%s%s%s%s',txt,padding0,['"',val,'"']); end if(e==len) sep=''; end txt=sprintf('%s%s',txt,sep); end if(len>1) txt=sprintf('%s%s%s%s',txt,nl,padding1,']'); end %%------------------------------------------------------------------------- function txt=mat2json(name,item,level,varargin) if(~isnumeric(item) && ~islogical(item)) error('input is not an array'); end ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding1=repmat(ws.tab,1,level); padding0=repmat(ws.tab,1,level+1); nl=ws.newline; sep=ws.sep; if(length(size(item))>2 || issparse(item) || ~isreal(item) || ... isempty(item) ||jsonopt('ArrayToStruct',0,varargin{:})) if(isempty(name)) txt=sprintf('%s{%s%s"_ArrayType_": "%s",%s%s"_ArraySize_": %s,%s',... padding1,nl,padding0,class(item),nl,padding0,regexprep(mat2str(size(item)),'\s+',','),nl); else txt=sprintf('%s"%s": {%s%s"_ArrayType_": "%s",%s%s"_ArraySize_": %s,%s',... padding1,checkname(name,varargin{:}),nl,padding0,class(item),nl,padding0,regexprep(mat2str(size(item)),'\s+',','),nl); end else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1 && level>0) numtxt=regexprep(regexprep(matdata2json(item,level+1,varargin{:}),'^\[',''),']',''); else numtxt=matdata2json(item,level+1,varargin{:}); end if(isempty(name)) txt=sprintf('%s%s',padding1,numtxt); else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1) txt=sprintf('%s"%s": %s',padding1,checkname(name,varargin{:}),numtxt); else txt=sprintf('%s"%s": %s',padding1,checkname(name,varargin{:}),numtxt); end end return; end dataformat='%s%s%s%s%s'; if(issparse(item)) [ix,iy]=find(item); data=full(item(find(item))); if(~isreal(item)) data=[real(data(:)),imag(data(:))]; if(size(item,1)==1) % Kludge to have data's 'transposedness' match item's. % (Necessary for complex row vector handling below.) data=data'; end txt=sprintf(dataformat,txt,padding0,'"_ArrayIsComplex_": ','1', sep); end txt=sprintf(dataformat,txt,padding0,'"_ArrayIsSparse_": ','1', sep); if(size(item,1)==1) % Row vector, store only column indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([iy(:),data'],level+2,varargin{:}), nl); elseif(size(item,2)==1) % Column vector, store only row indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([ix,data],level+2,varargin{:}), nl); else % General case, store row and column indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([ix,iy,data],level+2,varargin{:}), nl); end else if(isreal(item)) txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json(item(:)',level+2,varargin{:}), nl); else txt=sprintf(dataformat,txt,padding0,'"_ArrayIsComplex_": ','1', sep); txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([real(item(:)) imag(item(:))],level+2,varargin{:}), nl); end end txt=sprintf('%s%s%s',txt,padding1,'}'); %%------------------------------------------------------------------------- function txt=matdata2json(mat,level,varargin) ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); tab=ws.tab; nl=ws.newline; if(size(mat,1)==1) pre=''; post=''; level=level-1; else pre=sprintf('[%s',nl); post=sprintf('%s%s]',nl,repmat(tab,1,level-1)); end if(isempty(mat)) txt='null'; return; end floatformat=jsonopt('FloatFormat','%.10g',varargin{:}); %if(numel(mat)>1) formatstr=['[' repmat([floatformat ','],1,size(mat,2)-1) [floatformat sprintf('],%s',nl)]]; %else % formatstr=[repmat([floatformat ','],1,size(mat,2)-1) [floatformat sprintf(',\n')]]; %end if(nargin>=2 && size(mat,1)>1 && jsonopt('ArrayIndent',1,varargin{:})==1) formatstr=[repmat(tab,1,level) formatstr]; end txt=sprintf(formatstr,mat'); txt(end-length(nl):end)=[]; if(islogical(mat) && jsonopt('ParseLogical',0,varargin{:})==1) txt=regexprep(txt,'1','true'); txt=regexprep(txt,'0','false'); end %txt=regexprep(mat2str(mat),'\s+',','); %txt=regexprep(txt,';',sprintf('],\n[')); % if(nargin>=2 && size(mat,1)>1) % txt=regexprep(txt,'\[',[repmat(sprintf('\t'),1,level) '[']); % end txt=[pre txt post]; if(any(isinf(mat(:)))) txt=regexprep(txt,'([-+]*)Inf',jsonopt('Inf','"$1_Inf_"',varargin{:})); end if(any(isnan(mat(:)))) txt=regexprep(txt,'NaN',jsonopt('NaN','"_NaN_"',varargin{:})); end %%------------------------------------------------------------------------- function newname=checkname(name,varargin) isunpack=jsonopt('UnpackHex',1,varargin{:}); newname=name; if(isempty(regexp(name,'0x([0-9a-fA-F]+)_','once'))) return end if(isunpack) isoct=jsonopt('IsOctave',0,varargin{:}); if(~isoct) newname=regexprep(name,'(^x|_){1}0x([0-9a-fA-F]+)_','${native2unicode(hex2dec($2))}'); else pos=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','start'); pend=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','end'); if(isempty(pos)) return; end str0=name; pos0=[0 pend(:)' length(name)]; newname=''; for i=1:length(pos) newname=[newname str0(pos0(i)+1:pos(i)-1) char(hex2dec(str0(pos(i)+3:pend(i)-1)))]; end if(pos(end)~=length(name)) newname=[newname str0(pos0(end-1)+1:pos0(end))]; end end end %%------------------------------------------------------------------------- function newstr=escapejsonstring(str) newstr=str; isoct=exist('OCTAVE_VERSION','builtin'); if(isoct) vv=sscanf(OCTAVE_VERSION,'%f'); if(vv(1)>=3.8) isoct=0; end end if(isoct) escapechars={'\a','\f','\n','\r','\t','\v'}; for i=1:length(escapechars); newstr=regexprep(newstr,escapechars{i},escapechars{i}); end else escapechars={'\a','\b','\f','\n','\r','\t','\v'}; for i=1:length(escapechars); newstr=regexprep(newstr,escapechars{i},regexprep(escapechars{i},'\\','\\\\')); end end ================================================ FILE: machine-learning-ex1/ex1/lib/jsonlab/saveubjson.m ================================================ function json=saveubjson(rootname,obj,varargin) % % json=saveubjson(rootname,obj,filename) % or % json=saveubjson(rootname,obj,opt) % json=saveubjson(rootname,obj,'param1',value1,'param2',value2,...) % % convert a MATLAB object (cell, struct or array) into a Universal % Binary JSON (UBJSON) binary string % % author: Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2013/08/17 % % $Id: saveubjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % rootname: the name of the root-object, when set to '', the root name % is ignored, however, when opt.ForceRootName is set to 1 (see below), % the MATLAB variable name will be used as the root name. % obj: a MATLAB object (array, cell, cell array, struct, struct array) % filename: a string for the file name to save the output UBJSON data % opt: a struct for additional options, ignore to use default values. % opt can have the following fields (first in [.|.] is the default) % % opt.FileName [''|string]: a file name to save the output JSON data % opt.ArrayToStruct[0|1]: when set to 0, saveubjson outputs 1D/2D % array in JSON array format; if sets to 1, an % array will be shown as a struct with fields % "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for % sparse arrays, the non-zero elements will be % saved to _ArrayData_ field in triplet-format i.e. % (ix,iy,val) and "_ArrayIsSparse_" will be added % with a value of 1; for a complex array, the % _ArrayData_ array will include two columns % (4 for sparse) to record the real and imaginary % parts, and also "_ArrayIsComplex_":1 is added. % opt.ParseLogical [1|0]: if this is set to 1, logical array elem % will use true/false rather than 1/0. % opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single % numerical element will be shown without a square % bracket, unless it is the root object; if 0, square % brackets are forced for any numerical arrays. % opt.ForceRootName [0|1]: when set to 1 and rootname is empty, saveubjson % will use the name of the passed obj variable as the % root object name; if obj is an expression and % does not have a name, 'root' will be used; if this % is set to 0 and rootname is empty, the root level % will be merged down to the lower level. % opt.JSONP [''|string]: to generate a JSONP output (JSON with padding), % for example, if opt.JSON='foo', the JSON data is % wrapped inside a function call as 'foo(...);' % opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson % back to the string form % % opt can be replaced by a list of ('param',value) pairs. The param % string is equivallent to a field in opt and is case sensitive. % output: % json: a binary string in the UBJSON format (see http://ubjson.org) % % examples: % jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... % 'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],... % 'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;... % 2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],... % 'MeshCreator','FangQ','MeshTitle','T6 Cube',... % 'SpecialData',[nan, inf, -inf]); % saveubjson('jsonmesh',jsonmesh) % saveubjson('jsonmesh',jsonmesh,'meshdata.ubj') % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(nargin==1) varname=inputname(1); obj=rootname; if(isempty(varname)) varname='root'; end rootname=varname; else varname=inputname(2); end if(length(varargin)==1 && ischar(varargin{1})) opt=struct('FileName',varargin{1}); else opt=varargin2struct(varargin{:}); end opt.IsOctave=exist('OCTAVE_VERSION','builtin'); rootisarray=0; rootlevel=1; forceroot=jsonopt('ForceRootName',0,opt); if((isnumeric(obj) || islogical(obj) || ischar(obj) || isstruct(obj) || iscell(obj)) && isempty(rootname) && forceroot==0) rootisarray=1; rootlevel=0; else if(isempty(rootname)) rootname=varname; end end if((isstruct(obj) || iscell(obj))&& isempty(rootname) && forceroot) rootname='root'; end json=obj2ubjson(rootname,obj,rootlevel,opt); if(~rootisarray) json=['{' json '}']; end jsonp=jsonopt('JSONP','',opt); if(~isempty(jsonp)) json=[jsonp '(' json ')']; end % save to a file if FileName is set, suggested by Patrick Rapin if(~isempty(jsonopt('FileName','',opt))) fid = fopen(opt.FileName, 'wb'); fwrite(fid,json); fclose(fid); end %%------------------------------------------------------------------------- function txt=obj2ubjson(name,item,level,varargin) if(iscell(item)) txt=cell2ubjson(name,item,level,varargin{:}); elseif(isstruct(item)) txt=struct2ubjson(name,item,level,varargin{:}); elseif(ischar(item)) txt=str2ubjson(name,item,level,varargin{:}); else txt=mat2ubjson(name,item,level,varargin{:}); end %%------------------------------------------------------------------------- function txt=cell2ubjson(name,item,level,varargin) txt=''; if(~iscell(item)) error('input is not a cell'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); % let's handle 1D cell first if(len>1) if(~isempty(name)) txt=[S_(checkname(name,varargin{:})) '[']; name=''; else txt='['; end elseif(len==0) if(~isempty(name)) txt=[S_(checkname(name,varargin{:})) 'Z']; name=''; else txt='Z'; end end for j=1:dim(2) if(dim(1)>1) txt=[txt '[']; end for i=1:dim(1) txt=[txt obj2ubjson(name,item{i,j},level+(len>1),varargin{:})]; end if(dim(1)>1) txt=[txt ']']; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=struct2ubjson(name,item,level,varargin) txt=''; if(~isstruct(item)) error('input is not a struct'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); if(~isempty(name)) if(len>1) txt=[S_(checkname(name,varargin{:})) '[']; end else if(len>1) txt='['; end end for j=1:dim(2) if(dim(1)>1) txt=[txt '[']; end for i=1:dim(1) names = fieldnames(item(i,j)); if(~isempty(name) && len==1) txt=[txt S_(checkname(name,varargin{:})) '{']; else txt=[txt '{']; end if(~isempty(names)) for e=1:length(names) txt=[txt obj2ubjson(names{e},getfield(item(i,j),... names{e}),level+(dim(1)>1)+1+(len>1),varargin{:})]; end end txt=[txt '}']; end if(dim(1)>1) txt=[txt ']']; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=str2ubjson(name,item,level,varargin) txt=''; if(~ischar(item)) error('input is not a string'); end item=reshape(item, max(size(item),[1 0])); len=size(item,1); if(~isempty(name)) if(len>1) txt=[S_(checkname(name,varargin{:})) '[']; end else if(len>1) txt='['; end end isoct=jsonopt('IsOctave',0,varargin{:}); for e=1:len val=item(e,:); if(len==1) obj=['' S_(checkname(name,varargin{:})) '' '',S_(val),'']; if(isempty(name)) obj=['',S_(val),'']; end txt=[txt,'',obj]; else txt=[txt,'',['',S_(val),'']]; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=mat2ubjson(name,item,level,varargin) if(~isnumeric(item) && ~islogical(item)) error('input is not an array'); end if(length(size(item))>2 || issparse(item) || ~isreal(item) || ... isempty(item) || jsonopt('ArrayToStruct',0,varargin{:})) cid=I_(uint32(max(size(item)))); if(isempty(name)) txt=['{' S_('_ArrayType_'),S_(class(item)),S_('_ArraySize_'),I_a(size(item),cid(1)) ]; else if(isempty(item)) txt=[S_(checkname(name,varargin{:})),'Z']; return; else txt=[S_(checkname(name,varargin{:})),'{',S_('_ArrayType_'),S_(class(item)),S_('_ArraySize_'),I_a(size(item),cid(1))]; end end else if(isempty(name)) txt=matdata2ubjson(item,level+1,varargin{:}); else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1) numtxt=regexprep(regexprep(matdata2ubjson(item,level+1,varargin{:}),'^\[',''),']',''); txt=[S_(checkname(name,varargin{:})) numtxt]; else txt=[S_(checkname(name,varargin{:})),matdata2ubjson(item,level+1,varargin{:})]; end end return; end if(issparse(item)) [ix,iy]=find(item); data=full(item(find(item))); if(~isreal(item)) data=[real(data(:)),imag(data(:))]; if(size(item,1)==1) % Kludge to have data's 'transposedness' match item's. % (Necessary for complex row vector handling below.) data=data'; end txt=[txt,S_('_ArrayIsComplex_'),'T']; end txt=[txt,S_('_ArrayIsSparse_'),'T']; if(size(item,1)==1) % Row vector, store only column indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([iy(:),data'],level+2,varargin{:})]; elseif(size(item,2)==1) % Column vector, store only row indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([ix,data],level+2,varargin{:})]; else % General case, store row and column indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([ix,iy,data],level+2,varargin{:})]; end else if(isreal(item)) txt=[txt,S_('_ArrayData_'),... matdata2ubjson(item(:)',level+2,varargin{:})]; else txt=[txt,S_('_ArrayIsComplex_'),'T']; txt=[txt,S_('_ArrayData_'),... matdata2ubjson([real(item(:)) imag(item(:))],level+2,varargin{:})]; end end txt=[txt,'}']; %%------------------------------------------------------------------------- function txt=matdata2ubjson(mat,level,varargin) if(isempty(mat)) txt='Z'; return; end if(size(mat,1)==1) level=level-1; end type=''; hasnegtive=(mat<0); if(isa(mat,'integer') || isinteger(mat) || (isfloat(mat) && all(mod(mat(:),1) == 0))) if(isempty(hasnegtive)) if(max(mat(:))<=2^8) type='U'; end end if(isempty(type)) % todo - need to consider negative ones separately id= histc(abs(max(mat(:))),[0 2^7 2^15 2^31 2^63]); if(isempty(find(id))) error('high-precision data is not yet supported'); end key='iIlL'; type=key(find(id)); end txt=[I_a(mat(:),type,size(mat))]; elseif(islogical(mat)) logicalval='FT'; if(numel(mat)==1) txt=logicalval(mat+1); else txt=['[$U#' I_a(size(mat),'l') typecast(swapbytes(uint8(mat(:)')),'uint8')]; end else if(numel(mat)==1) txt=['[' D_(mat) ']']; else txt=D_a(mat(:),'D',size(mat)); end end %txt=regexprep(mat2str(mat),'\s+',','); %txt=regexprep(txt,';',sprintf('],[')); % if(nargin>=2 && size(mat,1)>1) % txt=regexprep(txt,'\[',[repmat(sprintf('\t'),1,level) '[']); % end if(any(isinf(mat(:)))) txt=regexprep(txt,'([-+]*)Inf',jsonopt('Inf','"$1_Inf_"',varargin{:})); end if(any(isnan(mat(:)))) txt=regexprep(txt,'NaN',jsonopt('NaN','"_NaN_"',varargin{:})); end %%------------------------------------------------------------------------- function newname=checkname(name,varargin) isunpack=jsonopt('UnpackHex',1,varargin{:}); newname=name; if(isempty(regexp(name,'0x([0-9a-fA-F]+)_','once'))) return end if(isunpack) isoct=jsonopt('IsOctave',0,varargin{:}); if(~isoct) newname=regexprep(name,'(^x|_){1}0x([0-9a-fA-F]+)_','${native2unicode(hex2dec($2))}'); else pos=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','start'); pend=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','end'); if(isempty(pos)) return; end str0=name; pos0=[0 pend(:)' length(name)]; newname=''; for i=1:length(pos) newname=[newname str0(pos0(i)+1:pos(i)-1) char(hex2dec(str0(pos(i)+3:pend(i)-1)))]; end if(pos(end)~=length(name)) newname=[newname str0(pos0(end-1)+1:pos0(end))]; end end end %%------------------------------------------------------------------------- function val=S_(str) if(length(str)==1) val=['C' str]; else val=['S' I_(int32(length(str))) str]; end %%------------------------------------------------------------------------- function val=I_(num) if(~isinteger(num)) error('input is not an integer'); end if(num>=0 && num<255) val=['U' data2byte(swapbytes(cast(num,'uint8')),'uint8')]; return; end key='iIlL'; cid={'int8','int16','int32','int64'}; for i=1:4 if((num>0 && num<2^(i*8-1)) || (num<0 && num>=-2^(i*8-1))) val=[key(i) data2byte(swapbytes(cast(num,cid{i})),'uint8')]; return; end end error('unsupported integer'); %%------------------------------------------------------------------------- function val=D_(num) if(~isfloat(num)) error('input is not a float'); end if(isa(num,'single')) val=['d' data2byte(num,'uint8')]; else val=['D' data2byte(num,'uint8')]; end %%------------------------------------------------------------------------- function data=I_a(num,type,dim,format) id=find(ismember('iUIlL',type)); if(id==0) error('unsupported integer array'); end % based on UBJSON specs, all integer types are stored in big endian format if(id==1) data=data2byte(swapbytes(int8(num)),'uint8'); blen=1; elseif(id==2) data=data2byte(swapbytes(uint8(num)),'uint8'); blen=1; elseif(id==3) data=data2byte(swapbytes(int16(num)),'uint8'); blen=2; elseif(id==4) data=data2byte(swapbytes(int32(num)),'uint8'); blen=4; elseif(id==5) data=data2byte(swapbytes(int64(num)),'uint8'); blen=8; end if(nargin>=3 && length(dim)>=2 && prod(dim)~=dim(2)) format='opt'; end if((nargin<4 || strcmp(format,'opt')) && numel(num)>1) if(nargin>=3 && (length(dim)==1 || (length(dim)>=2 && prod(dim)~=dim(2)))) cid=I_(uint32(max(dim))); data=['$' type '#' I_a(dim,cid(1)) data(:)']; else data=['$' type '#' I_(int32(numel(data)/blen)) data(:)']; end data=['[' data(:)']; else data=reshape(data,blen,numel(data)/blen); data(2:blen+1,:)=data; data(1,:)=type; data=data(:)'; data=['[' data(:)' ']']; end %%------------------------------------------------------------------------- function data=D_a(num,type,dim,format) id=find(ismember('dD',type)); if(id==0) error('unsupported float array'); end if(id==1) data=data2byte(single(num),'uint8'); elseif(id==2) data=data2byte(double(num),'uint8'); end if(nargin>=3 && length(dim)>=2 && prod(dim)~=dim(2)) format='opt'; end if((nargin<4 || strcmp(format,'opt')) && numel(num)>1) if(nargin>=3 && (length(dim)==1 || (length(dim)>=2 && prod(dim)~=dim(2)))) cid=I_(uint32(max(dim))); data=['$' type '#' I_a(dim,cid(1)) data(:)']; else data=['$' type '#' I_(int32(numel(data)/(id*4))) data(:)']; end data=['[' data]; else data=reshape(data,(id*4),length(data)/(id*4)); data(2:(id*4+1),:)=data; data(1,:)=type; data=data(:)'; data=['[' data(:)' ']']; end %%------------------------------------------------------------------------- function bytes=data2byte(varargin) bytes=typecast(varargin{:}); bytes=bytes(:)'; ================================================ FILE: machine-learning-ex1/ex1/lib/jsonlab/varargin2struct.m ================================================ function opt=varargin2struct(varargin) % % opt=varargin2struct('param1',value1,'param2',value2,...) % or % opt=varargin2struct(...,optstruct,...) % % convert a series of input parameters into a structure % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % date: 2012/12/22 % % input: % 'param', value: the input parameters should be pairs of a string and a value % optstruct: if a parameter is a struct, the fields will be merged to the output struct % % output: % opt: a struct where opt.param1=value1, opt.param2=value2 ... % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % len=length(varargin); opt=struct; if(len==0) return; end i=1; while(i<=len) if(isstruct(varargin{i})) opt=mergestruct(opt,varargin{i}); elseif(ischar(varargin{i}) && i or The script loadjson.m was built upon previous works by - Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713 date: 2009/11/02 - François Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393 date: 2009/03/22 - Joel Feenstra: http://www.mathworks.com/matlabcentral/fileexchange/20565 date: 2008/07/03 This toolbox contains patches submitted by the following contributors: - Blake Johnson part of revision 341 - Niclas Borlin various fixes in revision 394, including - loadjson crashes for all-zero sparse matrix. - loadjson crashes for empty sparse matrix. - Non-zero size of 0-by-N and N-by-0 empty matrices is lost after savejson/loadjson. - loadjson crashes for sparse real column vector. - loadjson crashes for sparse complex column vector. - Data is corrupted by savejson for sparse real row vector. - savejson crashes for sparse complex row vector. - Yul Kang patches for svn revision 415. - savejson saves an empty cell array as [] instead of null - loadjson differentiates an empty struct from an empty array ================================================ FILE: machine-learning-ex2/ex2/lib/jsonlab/ChangeLog.txt ================================================ ============================================================================ JSONlab - a toolbox to encode/decode JSON/UBJSON files in MATLAB/Octave ---------------------------------------------------------------------------- JSONlab ChangeLog (key features marked by *): == JSONlab 1.0 (codename: Optimus - Final), FangQ == 2015/01/02 polish help info for all major functions, update examples, finalize 1.0 2014/12/19 fix a bug to strictly respect NoRowBracket in savejson == JSONlab 1.0.0-RC2 (codename: Optimus - RC2), FangQ == 2014/11/22 show progress bar in loadjson ('ShowProgress') 2014/11/17 add Compact option in savejson to output compact JSON format ('Compact') 2014/11/17 add FastArrayParser in loadjson to specify fast parser applicable levels 2014/09/18 start official github mirror: https://github.com/fangq/jsonlab == JSONlab 1.0.0-RC1 (codename: Optimus - RC1), FangQ == 2014/09/17 fix several compatibility issues when running on octave versions 3.2-3.8 2014/09/17 support 2D cell and struct arrays in both savejson and saveubjson 2014/08/04 escape special characters in a JSON string 2014/02/16 fix a bug when saving ubjson files == JSONlab 0.9.9 (codename: Optimus - beta), FangQ == 2014/01/22 use binary read and write in saveubjson and loadubjson == JSONlab 0.9.8-1 (codename: Optimus - alpha update 1), FangQ == 2013/10/07 better round-trip conservation for empty arrays and structs (patch submitted by Yul Kang) == JSONlab 0.9.8 (codename: Optimus - alpha), FangQ == 2013/08/23 *universal Binary JSON (UBJSON) support, including both saveubjson and loadubjson == JSONlab 0.9.1 (codename: Rodimus, update 1), FangQ == 2012/12/18 *handling of various empty and sparse matrices (fixes submitted by Niclas Borlin) == JSONlab 0.9.0 (codename: Rodimus), FangQ == 2012/06/17 *new format for an invalid leading char, unpacking hex code in savejson 2012/06/01 support JSONP in savejson 2012/05/25 fix the empty cell bug (reported by Cyril Davin) 2012/04/05 savejson can save to a file (suggested by Patrick Rapin) == JSONlab 0.8.1 (codename: Sentiel, Update 1), FangQ == 2012/02/28 loadjson quotation mark escape bug, see http://bit.ly/yyk1nS 2012/01/25 patch to handle root-less objects, contributed by Blake Johnson == JSONlab 0.8.0 (codename: Sentiel), FangQ == 2012/01/13 *speed up loadjson by 20 fold when parsing large data arrays in matlab 2012/01/11 remove row bracket if an array has 1 element, suggested by Mykel Kochenderfer 2011/12/22 *accept sequence of 'param',value input in savejson and loadjson 2011/11/18 fix struct array bug reported by Mykel Kochenderfer == JSONlab 0.5.1 (codename: Nexus Update 1), FangQ == 2011/10/21 fix a bug in loadjson, previous code does not use any of the acceleration 2011/10/20 loadjson supports JSON collections - concatenated JSON objects == JSONlab 0.5.0 (codename: Nexus), FangQ == 2011/10/16 package and release jsonlab 0.5.0 2011/10/15 *add json demo and regression test, support cpx numbers, fix double quote bug 2011/10/11 *speed up readjson dramatically, interpret _Array* tags, show data in root level 2011/10/10 create jsonlab project, start jsonlab website, add online documentation 2011/10/07 *speed up savejson by 25x using sprintf instead of mat2str, add options support 2011/10/06 *savejson works for structs, cells and arrays 2011/09/09 derive loadjson from JSON parser from MATLAB Central, draft savejson.m ================================================ FILE: machine-learning-ex2/ex2/lib/jsonlab/LICENSE_BSD.txt ================================================ Copyright 2011-2015 Qianqian Fang . All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. The views and conclusions contained in the software and documentation are those of the authors and should not be interpreted as representing official policies, either expressed or implied, of the copyright holders. ================================================ FILE: machine-learning-ex2/ex2/lib/jsonlab/README.txt ================================================ =============================================================================== = JSONLab = = An open-source MATLAB/Octave JSON encoder and decoder = =============================================================================== *Copyright (C) 2011-2015 Qianqian Fang *License: BSD License, see License_BSD.txt for details *Version: 1.0 (Optimus - Final) ------------------------------------------------------------------------------- Table of Content: I. Introduction II. Installation III.Using JSONLab IV. Known Issues and TODOs V. Contribution and feedback ------------------------------------------------------------------------------- I. Introduction JSON ([http://www.json.org/ JavaScript Object Notation]) is a highly portable, human-readable and "[http://en.wikipedia.org/wiki/JSON fat-free]" text format to represent complex and hierarchical data. It is as powerful as [http://en.wikipedia.org/wiki/XML XML], but less verbose. JSON format is widely used for data-exchange in applications, and is essential for the wild success of [http://en.wikipedia.org/wiki/Ajax_(programming) Ajax] and [http://en.wikipedia.org/wiki/Web_2.0 Web2.0]. UBJSON (Universal Binary JSON) is a binary JSON format, specifically optimized for compact file size and better performance while keeping the semantics as simple as the text-based JSON format. Using the UBJSON format allows to wrap complex binary data in a flexible and extensible structure, making it possible to process complex and large dataset without accuracy loss due to text conversions. We envision that both JSON and its binary version will serve as part of the mainstream data-exchange formats for scientific research in the future. It will provide the flexibility and generality achieved by other popular general-purpose file specifications, such as [http://www.hdfgroup.org/HDF5/whatishdf5.html HDF5], with significantly reduced complexity and enhanced performance. JSONLab is a free and open-source implementation of a JSON/UBJSON encoder and a decoder in the native MATLAB language. It can be used to convert a MATLAB data structure (array, struct, cell, struct array and cell array) into JSON/UBJSON formatted strings, or to decode a JSON/UBJSON file into MATLAB data structure. JSONLab supports both MATLAB and [http://www.gnu.org/software/octave/ GNU Octave] (a free MATLAB clone). ------------------------------------------------------------------------------- II. Installation The installation of JSONLab is no different than any other simple MATLAB toolbox. You only need to download/unzip the JSONLab package to a folder, and add the folder's path to MATLAB/Octave's path list by using the following command: addpath('/path/to/jsonlab'); If you want to add this path permanently, you need to type "pathtool", browse to the jsonlab root folder and add to the list, then click "Save". Then, run "rehash" in MATLAB, and type "which loadjson", if you see an output, that means JSONLab is installed for MATLAB/Octave. ------------------------------------------------------------------------------- III.Using JSONLab JSONLab provides two functions, loadjson.m -- a MATLAB->JSON decoder, and savejson.m -- a MATLAB->JSON encoder, for the text-based JSON, and two equivallent functions -- loadubjson and saveubjson for the binary JSON. The detailed help info for the four functions can be found below: === loadjson.m === 
  data=loadjson(fname,opt)
     or
  data=loadjson(fname,'param1',value1,'param2',value2,...)
 
  parse a JSON (JavaScript Object Notation) file or string
 
  authors:Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2011/09/09, including previous works from 
 
          Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713
             created on 2009/11/02
          Franois Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393
             created on  2009/03/22
          Joel Feenstra:
          http://www.mathworks.com/matlabcentral/fileexchange/20565
             created on 2008/07/03
 
  $Id: loadjson.m 452 2014-11-22 16:43:33Z fangq $
 
  input:
       fname: input file name, if fname contains "{}" or "[]", fname
              will be interpreted as a JSON string
       opt: a struct to store parsing options, opt can be replaced by 
            a list of ('param',value) pairs - the param string is equivallent
            to a field in opt. opt can have the following 
            fields (first in [.|.] is the default)
 
            opt.SimplifyCell [0|1]: if set to 1, loadjson will call cell2mat
                          for each element of the JSON data, and group 
                          arrays based on the cell2mat rules.
            opt.FastArrayParser [1|0 or integer]: if set to 1, use a
                          speed-optimized array parser when loading an 
                          array object. The fast array parser may 
                          collapse block arrays into a single large
                          array similar to rules defined in cell2mat; 0 to 
                          use a legacy parser; if set to a larger-than-1
                          value, this option will specify the minimum
                          dimension to enable the fast array parser. For
                          example, if the input is a 3D array, setting
                          FastArrayParser to 1 will return a 3D array;
                          setting to 2 will return a cell array of 2D
                          arrays; setting to 3 will return to a 2D cell
                          array of 1D vectors; setting to 4 will return a
                          3D cell array.
            opt.ShowProgress [0|1]: if set to 1, loadjson displays a progress bar.
 
  output:
       dat: a cell array, where {...} blocks are converted into cell arrays,
            and [...] are converted to arrays
 
  examples:
       dat=loadjson('{"obj":{"string":"value","array":[1,2,3]}}')
       dat=loadjson(['examples' filesep 'example1.json'])
       dat=loadjson(['examples' filesep 'example1.json'],'SimplifyCell',1)
=== savejson.m === 
  json=savejson(rootname,obj,filename)
     or
  json=savejson(rootname,obj,opt)
  json=savejson(rootname,obj,'param1',value1,'param2',value2,...)
 
  convert a MATLAB object (cell, struct or array) into a JSON (JavaScript
  Object Notation) string
 
  author: Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2011/09/09
 
  $Id: savejson.m 458 2014-12-19 22:17:17Z fangq $
 
  input:
       rootname: the name of the root-object, when set to '', the root name
         is ignored, however, when opt.ForceRootName is set to 1 (see below),
         the MATLAB variable name will be used as the root name.
       obj: a MATLAB object (array, cell, cell array, struct, struct array).
       filename: a string for the file name to save the output JSON data.
       opt: a struct for additional options, ignore to use default values.
         opt can have the following fields (first in [.|.] is the default)
 
         opt.FileName [''|string]: a file name to save the output JSON data
         opt.FloatFormat ['%.10g'|string]: format to show each numeric element
                          of a 1D/2D array;
         opt.ArrayIndent [1|0]: if 1, output explicit data array with
                          precedent indentation; if 0, no indentation
         opt.ArrayToStruct[0|1]: when set to 0, savejson outputs 1D/2D
                          array in JSON array format; if sets to 1, an
                          array will be shown as a struct with fields
                          "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for
                          sparse arrays, the non-zero elements will be
                          saved to _ArrayData_ field in triplet-format i.e.
                          (ix,iy,val) and "_ArrayIsSparse_" will be added
                          with a value of 1; for a complex array, the 
                          _ArrayData_ array will include two columns 
                          (4 for sparse) to record the real and imaginary 
                          parts, and also "_ArrayIsComplex_":1 is added. 
         opt.ParseLogical [0|1]: if this is set to 1, logical array elem
                          will use true/false rather than 1/0.
         opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single
                          numerical element will be shown without a square
                          bracket, unless it is the root object; if 0, square
                          brackets are forced for any numerical arrays.
         opt.ForceRootName [0|1]: when set to 1 and rootname is empty, savejson
                          will use the name of the passed obj variable as the 
                          root object name; if obj is an expression and 
                          does not have a name, 'root' will be used; if this 
                          is set to 0 and rootname is empty, the root level 
                          will be merged down to the lower level.
         opt.Inf ['"$1_Inf_"'|string]: a customized regular expression pattern
                          to represent +/-Inf. The matched pattern is '([-+]*)Inf'
                          and $1 represents the sign. For those who want to use
                          1e999 to represent Inf, they can set opt.Inf to '$11e999'
         opt.NaN ['"_NaN_"'|string]: a customized regular expression pattern
                          to represent NaN
         opt.JSONP [''|string]: to generate a JSONP output (JSON with padding),
                          for example, if opt.JSONP='foo', the JSON data is
                          wrapped inside a function call as 'foo(...);'
         opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson 
                          back to the string form
         opt.SaveBinary [0|1]: 1 - save the JSON file in binary mode; 0 - text mode.
         opt.Compact [0|1]: 1- out compact JSON format (remove all newlines and tabs)
 
         opt can be replaced by a list of ('param',value) pairs. The param 
         string is equivallent to a field in opt and is case sensitive.
  output:
       json: a string in the JSON format (see http://json.org)
 
  examples:
       jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... 
                'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],...
                'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;...
                           2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],...
                'MeshCreator','FangQ','MeshTitle','T6 Cube',...
                'SpecialData',[nan, inf, -inf]);
       savejson('jmesh',jsonmesh)
       savejson('',jsonmesh,'ArrayIndent',0,'FloatFormat','\t%.5g')
=== loadubjson.m === 
  data=loadubjson(fname,opt)
     or
  data=loadubjson(fname,'param1',value1,'param2',value2,...)
 
  parse a JSON (JavaScript Object Notation) file or string
 
  authors:Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2013/08/01
 
  $Id: loadubjson.m 436 2014-08-05 20:51:40Z fangq $
 
  input:
       fname: input file name, if fname contains "{}" or "[]", fname
              will be interpreted as a UBJSON string
       opt: a struct to store parsing options, opt can be replaced by 
            a list of ('param',value) pairs - the param string is equivallent
            to a field in opt. opt can have the following 
            fields (first in [.|.] is the default)
 
            opt.SimplifyCell [0|1]: if set to 1, loadubjson will call cell2mat
                          for each element of the JSON data, and group 
                          arrays based on the cell2mat rules.
            opt.IntEndian [B|L]: specify the endianness of the integer fields
                          in the UBJSON input data. B - Big-Endian format for 
                          integers (as required in the UBJSON specification); 
                          L - input integer fields are in Little-Endian order.
 
  output:
       dat: a cell array, where {...} blocks are converted into cell arrays,
            and [...] are converted to arrays
 
  examples:
       obj=struct('string','value','array',[1 2 3]);
       ubjdata=saveubjson('obj',obj);
       dat=loadubjson(ubjdata)
       dat=loadubjson(['examples' filesep 'example1.ubj'])
       dat=loadubjson(['examples' filesep 'example1.ubj'],'SimplifyCell',1)
=== saveubjson.m === 
  json=saveubjson(rootname,obj,filename)
     or
  json=saveubjson(rootname,obj,opt)
  json=saveubjson(rootname,obj,'param1',value1,'param2',value2,...)
 
  convert a MATLAB object (cell, struct or array) into a Universal 
  Binary JSON (UBJSON) binary string
 
  author: Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2013/08/17
 
  $Id: saveubjson.m 440 2014-09-17 19:59:45Z fangq $
 
  input:
       rootname: the name of the root-object, when set to '', the root name
         is ignored, however, when opt.ForceRootName is set to 1 (see below),
         the MATLAB variable name will be used as the root name.
       obj: a MATLAB object (array, cell, cell array, struct, struct array)
       filename: a string for the file name to save the output UBJSON data
       opt: a struct for additional options, ignore to use default values.
         opt can have the following fields (first in [.|.] is the default)
 
         opt.FileName [''|string]: a file name to save the output JSON data
         opt.ArrayToStruct[0|1]: when set to 0, saveubjson outputs 1D/2D
                          array in JSON array format; if sets to 1, an
                          array will be shown as a struct with fields
                          "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for
                          sparse arrays, the non-zero elements will be
                          saved to _ArrayData_ field in triplet-format i.e.
                          (ix,iy,val) and "_ArrayIsSparse_" will be added
                          with a value of 1; for a complex array, the 
                          _ArrayData_ array will include two columns 
                          (4 for sparse) to record the real and imaginary 
                          parts, and also "_ArrayIsComplex_":1 is added. 
         opt.ParseLogical [1|0]: if this is set to 1, logical array elem
                          will use true/false rather than 1/0.
         opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single
                          numerical element will be shown without a square
                          bracket, unless it is the root object; if 0, square
                          brackets are forced for any numerical arrays.
         opt.ForceRootName [0|1]: when set to 1 and rootname is empty, saveubjson
                          will use the name of the passed obj variable as the 
                          root object name; if obj is an expression and 
                          does not have a name, 'root' will be used; if this 
                          is set to 0 and rootname is empty, the root level 
                          will be merged down to the lower level.
         opt.JSONP [''|string]: to generate a JSONP output (JSON with padding),
                          for example, if opt.JSON='foo', the JSON data is
                          wrapped inside a function call as 'foo(...);'
         opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson 
                          back to the string form
 
         opt can be replaced by a list of ('param',value) pairs. The param 
         string is equivallent to a field in opt and is case sensitive.
  output:
       json: a binary string in the UBJSON format (see http://ubjson.org)
 
  examples:
       jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... 
                'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],...
                'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;...
                           2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],...
                'MeshCreator','FangQ','MeshTitle','T6 Cube',...
                'SpecialData',[nan, inf, -inf]);
       saveubjson('jsonmesh',jsonmesh)
       saveubjson('jsonmesh',jsonmesh,'meshdata.ubj')
=== examples === Under the "examples" folder, you can find several scripts to demonstrate the basic utilities of JSONLab. Running the "demo_jsonlab_basic.m" script, you will see the conversions from MATLAB data structure to JSON text and backward. In "jsonlab_selftest.m", we load complex JSON files downloaded from the Internet and validate the loadjson/savejson functions for regression testing purposes. Similarly, a "demo_ubjson_basic.m" script is provided to test the saveubjson and loadubjson pairs for various matlab data structures. Please run these examples and understand how JSONLab works before you use it to process your data. ------------------------------------------------------------------------------- IV. Known Issues and TODOs JSONLab has several known limitations. We are striving to make it more general and robust. Hopefully in a few future releases, the limitations become less. Here are the known issues: # 3D or higher dimensional cell/struct-arrays will be converted to 2D arrays; # When processing names containing multi-byte characters, Octave and MATLAB \ can give different field-names; you can use feature('DefaultCharacterSet','latin1') \ in MATLAB to get consistant results # savejson can not handle class and dataset. # saveubjson converts a logical array into a uint8 ([U]) array # an unofficial N-D array count syntax is implemented in saveubjson. We are \ actively communicating with the UBJSON spec maintainer to investigate the \ possibility of making it upstream # loadubjson can not parse all UBJSON Specification (Draft 9) compliant \ files, however, it can parse all UBJSON files produced by saveubjson. ------------------------------------------------------------------------------- V. Contribution and feedback JSONLab is an open-source project. This means you can not only use it and modify it as you wish, but also you can contribute your changes back to JSONLab so that everyone else can enjoy the improvement. For anyone who want to contribute, please download JSONLab source code from it's subversion repository by using the following command: svn checkout svn://svn.code.sf.net/p/iso2mesh/code/trunk/jsonlab jsonlab You can make changes to the files as needed. Once you are satisfied with your changes, and ready to share it with others, please cd the root directory of JSONLab, and type svn diff > yourname_featurename.patch You then email the .patch file to JSONLab's maintainer, Qianqian Fang, at the email address shown in the beginning of this file. Qianqian will review the changes and commit it to the subversion if they are satisfactory. We appreciate any suggestions and feedbacks from you. Please use iso2mesh's mailing list to report any questions you may have with JSONLab: http://groups.google.com/group/iso2mesh-users?hl=en&pli=1 (Subscription to the mailing list is needed in order to post messages). ================================================ FILE: machine-learning-ex2/ex2/lib/jsonlab/jsonopt.m ================================================ function val=jsonopt(key,default,varargin) % % val=jsonopt(key,default,optstruct) % % setting options based on a struct. The struct can be produced % by varargin2struct from a list of 'param','value' pairs % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % % $Id: loadjson.m 371 2012-06-20 12:43:06Z fangq $ % % input: % key: a string with which one look up a value from a struct % default: if the key does not exist, return default % optstruct: a struct where each sub-field is a key % % output: % val: if key exists, val=optstruct.key; otherwise val=default % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % val=default; if(nargin<=2) return; end opt=varargin{1}; if(isstruct(opt) && isfield(opt,key)) val=getfield(opt,key); end ================================================ FILE: machine-learning-ex2/ex2/lib/jsonlab/loadjson.m ================================================ function data = loadjson(fname,varargin) % % data=loadjson(fname,opt) % or % data=loadjson(fname,'param1',value1,'param2',value2,...) % % parse a JSON (JavaScript Object Notation) file or string % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2011/09/09, including previous works from % % Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713 % created on 2009/11/02 % François Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393 % created on 2009/03/22 % Joel Feenstra: % http://www.mathworks.com/matlabcentral/fileexchange/20565 % created on 2008/07/03 % % $Id: loadjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % fname: input file name, if fname contains "{}" or "[]", fname % will be interpreted as a JSON string % opt: a struct to store parsing options, opt can be replaced by % a list of ('param',value) pairs - the param string is equivallent % to a field in opt. opt can have the following % fields (first in [.|.] is the default) % % opt.SimplifyCell [0|1]: if set to 1, loadjson will call cell2mat % for each element of the JSON data, and group % arrays based on the cell2mat rules. % opt.FastArrayParser [1|0 or integer]: if set to 1, use a % speed-optimized array parser when loading an % array object. The fast array parser may % collapse block arrays into a single large % array similar to rules defined in cell2mat; 0 to % use a legacy parser; if set to a larger-than-1 % value, this option will specify the minimum % dimension to enable the fast array parser. For % example, if the input is a 3D array, setting % FastArrayParser to 1 will return a 3D array; % setting to 2 will return a cell array of 2D % arrays; setting to 3 will return to a 2D cell % array of 1D vectors; setting to 4 will return a % 3D cell array. % opt.ShowProgress [0|1]: if set to 1, loadjson displays a progress bar. % % output: % dat: a cell array, where {...} blocks are converted into cell arrays, % and [...] are converted to arrays % % examples: % dat=loadjson('{"obj":{"string":"value","array":[1,2,3]}}') % dat=loadjson(['examples' filesep 'example1.json']) % dat=loadjson(['examples' filesep 'example1.json'],'SimplifyCell',1) % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % global pos inStr len esc index_esc len_esc isoct arraytoken if(regexp(fname,'[\{\}\]\[]','once')) string=fname; elseif(exist(fname,'file')) fid = fopen(fname,'rb'); string = fread(fid,inf,'uint8=>char')'; fclose(fid); else error('input file does not exist'); end pos = 1; len = length(string); inStr = string; isoct=exist('OCTAVE_VERSION','builtin'); arraytoken=find(inStr=='[' | inStr==']' | inStr=='"'); jstr=regexprep(inStr,'\\\\',' '); escquote=regexp(jstr,'\\"'); arraytoken=sort([arraytoken escquote]); % String delimiters and escape chars identified to improve speed: esc = find(inStr=='"' | inStr=='\' ); % comparable to: regexp(inStr, '["\\]'); index_esc = 1; len_esc = length(esc); opt=varargin2struct(varargin{:}); if(jsonopt('ShowProgress',0,opt)==1) opt.progressbar_=waitbar(0,'loading ...'); end jsoncount=1; while pos <= len switch(next_char) case '{' data{jsoncount} = parse_object(opt); case '[' data{jsoncount} = parse_array(opt); otherwise error_pos('Outer level structure must be an object or an array'); end jsoncount=jsoncount+1; end % while jsoncount=length(data); if(jsoncount==1 && iscell(data)) data=data{1}; end if(~isempty(data)) if(isstruct(data)) % data can be a struct array data=jstruct2array(data); elseif(iscell(data)) data=jcell2array(data); end end if(isfield(opt,'progressbar_')) close(opt.progressbar_); end %% function newdata=jcell2array(data) len=length(data); newdata=data; for i=1:len if(isstruct(data{i})) newdata{i}=jstruct2array(data{i}); elseif(iscell(data{i})) newdata{i}=jcell2array(data{i}); end end %%------------------------------------------------------------------------- function newdata=jstruct2array(data) fn=fieldnames(data); newdata=data; len=length(data); for i=1:length(fn) % depth-first for j=1:len if(isstruct(getfield(data(j),fn{i}))) newdata(j)=setfield(newdata(j),fn{i},jstruct2array(getfield(data(j),fn{i}))); end end end if(~isempty(strmatch('x0x5F_ArrayType_',fn)) && ~isempty(strmatch('x0x5F_ArrayData_',fn))) newdata=cell(len,1); for j=1:len ndata=cast(data(j).x0x5F_ArrayData_,data(j).x0x5F_ArrayType_); iscpx=0; if(~isempty(strmatch('x0x5F_ArrayIsComplex_',fn))) if(data(j).x0x5F_ArrayIsComplex_) iscpx=1; end end if(~isempty(strmatch('x0x5F_ArrayIsSparse_',fn))) if(data(j).x0x5F_ArrayIsSparse_) if(~isempty(strmatch('x0x5F_ArraySize_',fn))) dim=data(j).x0x5F_ArraySize_; if(iscpx && size(ndata,2)==4-any(dim==1)) ndata(:,end-1)=complex(ndata(:,end-1),ndata(:,end)); end if isempty(ndata) % All-zeros sparse ndata=sparse(dim(1),prod(dim(2:end))); elseif dim(1)==1 % Sparse row vector ndata=sparse(1,ndata(:,1),ndata(:,2),dim(1),prod(dim(2:end))); elseif dim(2)==1 % Sparse column vector ndata=sparse(ndata(:,1),1,ndata(:,2),dim(1),prod(dim(2:end))); else % Generic sparse array. ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3),dim(1),prod(dim(2:end))); end else if(iscpx && size(ndata,2)==4) ndata(:,3)=complex(ndata(:,3),ndata(:,4)); end ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3)); end end elseif(~isempty(strmatch('x0x5F_ArraySize_',fn))) if(iscpx && size(ndata,2)==2) ndata=complex(ndata(:,1),ndata(:,2)); end ndata=reshape(ndata(:),data(j).x0x5F_ArraySize_); end newdata{j}=ndata; end if(len==1) newdata=newdata{1}; end end %%------------------------------------------------------------------------- function object = parse_object(varargin) parse_char('{'); object = []; if next_char ~= '}' while 1 str = parseStr(varargin{:}); if isempty(str) error_pos('Name of value at position %d cannot be empty'); end parse_char(':'); val = parse_value(varargin{:}); eval( sprintf( 'object.%s = val;', valid_field(str) ) ); if next_char == '}' break; end parse_char(','); end end parse_char('}'); %%------------------------------------------------------------------------- function object = parse_array(varargin) % JSON array is written in row-major order global pos inStr isoct parse_char('['); object = cell(0, 1); dim2=[]; arraydepth=jsonopt('JSONLAB_ArrayDepth_',1,varargin{:}); pbar=jsonopt('progressbar_',-1,varargin{:}); if next_char ~= ']' if(jsonopt('FastArrayParser',1,varargin{:})>=1 && arraydepth>=jsonopt('FastArrayParser',1,varargin{:})) [endpos, e1l, e1r, maxlevel]=matching_bracket(inStr,pos); arraystr=['[' inStr(pos:endpos)]; arraystr=regexprep(arraystr,'"_NaN_"','NaN'); arraystr=regexprep(arraystr,'"([-+]*)_Inf_"','$1Inf'); arraystr(arraystr==sprintf('\n'))=[]; arraystr(arraystr==sprintf('\r'))=[]; %arraystr=regexprep(arraystr,'\s*,',','); % this is slow,sometimes needed if(~isempty(e1l) && ~isempty(e1r)) % the array is in 2D or higher D astr=inStr((e1l+1):(e1r-1)); astr=regexprep(astr,'"_NaN_"','NaN'); astr=regexprep(astr,'"([-+]*)_Inf_"','$1Inf'); astr(astr==sprintf('\n'))=[]; astr(astr==sprintf('\r'))=[]; astr(astr==' ')=''; if(isempty(find(astr=='[', 1))) % array is 2D dim2=length(sscanf(astr,'%f,',[1 inf])); end else % array is 1D astr=arraystr(2:end-1); astr(astr==' ')=''; [obj, count, errmsg, nextidx]=sscanf(astr,'%f,',[1,inf]); if(nextidx>=length(astr)-1) object=obj; pos=endpos; parse_char(']'); return; end end if(~isempty(dim2)) astr=arraystr; astr(astr=='[')=''; astr(astr==']')=''; astr(astr==' ')=''; [obj, count, errmsg, nextidx]=sscanf(astr,'%f,',inf); if(nextidx>=length(astr)-1) object=reshape(obj,dim2,numel(obj)/dim2)'; pos=endpos; parse_char(']'); if(pbar>0) waitbar(pos/length(inStr),pbar,'loading ...'); end return; end end arraystr=regexprep(arraystr,'\]\s*,','];'); else arraystr='['; end try if(isoct && regexp(arraystr,'"','once')) error('Octave eval can produce empty cells for JSON-like input'); end object=eval(arraystr); pos=endpos; catch while 1 newopt=varargin2struct(varargin{:},'JSONLAB_ArrayDepth_',arraydepth+1); val = parse_value(newopt); object{end+1} = val; if next_char == ']' break; end parse_char(','); end end end if(jsonopt('SimplifyCell',0,varargin{:})==1) try oldobj=object; object=cell2mat(object')'; if(iscell(oldobj) && isstruct(object) && numel(object)>1 && jsonopt('SimplifyCellArray',1,varargin{:})==0) object=oldobj; elseif(size(object,1)>1 && ndims(object)==2) object=object'; end catch end end parse_char(']'); if(pbar>0) waitbar(pos/length(inStr),pbar,'loading ...'); end %%------------------------------------------------------------------------- function parse_char(c) global pos inStr len skip_whitespace; if pos > len || inStr(pos) ~= c error_pos(sprintf('Expected %c at position %%d', c)); else pos = pos + 1; skip_whitespace; end %%------------------------------------------------------------------------- function c = next_char global pos inStr len skip_whitespace; if pos > len c = []; else c = inStr(pos); end %%------------------------------------------------------------------------- function skip_whitespace global pos inStr len while pos <= len && isspace(inStr(pos)) pos = pos + 1; end %%------------------------------------------------------------------------- function str = parseStr(varargin) global pos inStr len esc index_esc len_esc % len, ns = length(inStr), keyboard if inStr(pos) ~= '"' error_pos('String starting with " expected at position %d'); else pos = pos + 1; end str = ''; while pos <= len while index_esc <= len_esc && esc(index_esc) < pos index_esc = index_esc + 1; end if index_esc > len_esc str = [str inStr(pos:len)]; pos = len + 1; break; else str = [str inStr(pos:esc(index_esc)-1)]; pos = esc(index_esc); end nstr = length(str); switch inStr(pos) case '"' pos = pos + 1; if(~isempty(str)) if(strcmp(str,'_Inf_')) str=Inf; elseif(strcmp(str,'-_Inf_')) str=-Inf; elseif(strcmp(str,'_NaN_')) str=NaN; end end return; case '\' if pos+1 > len error_pos('End of file reached right after escape character'); end pos = pos + 1; switch inStr(pos) case {'"' '\' '/'} str(nstr+1) = inStr(pos); pos = pos + 1; case {'b' 'f' 'n' 'r' 't'} str(nstr+1) = sprintf(['\' inStr(pos)]); pos = pos + 1; case 'u' if pos+4 > len error_pos('End of file reached in escaped unicode character'); end str(nstr+(1:6)) = inStr(pos-1:pos+4); pos = pos + 5; end otherwise % should never happen str(nstr+1) = inStr(pos), keyboard pos = pos + 1; end end error_pos('End of file while expecting end of inStr'); %%------------------------------------------------------------------------- function num = parse_number(varargin) global pos inStr len isoct currstr=inStr(pos:end); numstr=0; if(isoct~=0) numstr=regexp(currstr,'^\s*-?(?:0|[1-9]\d*)(?:\.\d+)?(?:[eE][+\-]?\d+)?','end'); [num, one] = sscanf(currstr, '%f', 1); delta=numstr+1; else [num, one, err, delta] = sscanf(currstr, '%f', 1); if ~isempty(err) error_pos('Error reading number at position %d'); end end pos = pos + delta-1; %%------------------------------------------------------------------------- function val = parse_value(varargin) global pos inStr len true = 1; false = 0; pbar=jsonopt('progressbar_',-1,varargin{:}); if(pbar>0) waitbar(pos/len,pbar,'loading ...'); end switch(inStr(pos)) case '"' val = parseStr(varargin{:}); return; case '[' val = parse_array(varargin{:}); return; case '{' val = parse_object(varargin{:}); if isstruct(val) if(~isempty(strmatch('x0x5F_ArrayType_',fieldnames(val), 'exact'))) val=jstruct2array(val); end elseif isempty(val) val = struct; end return; case {'-','0','1','2','3','4','5','6','7','8','9'} val = parse_number(varargin{:}); return; case 't' if pos+3 <= len && strcmpi(inStr(pos:pos+3), 'true') val = true; pos = pos + 4; return; end case 'f' if pos+4 <= len && strcmpi(inStr(pos:pos+4), 'false') val = false; pos = pos + 5; return; end case 'n' if pos+3 <= len && strcmpi(inStr(pos:pos+3), 'null') val = []; pos = pos + 4; return; end end error_pos('Value expected at position %d'); %%------------------------------------------------------------------------- function error_pos(msg) global pos inStr len poShow = max(min([pos-15 pos-1 pos pos+20],len),1); if poShow(3) == poShow(2) poShow(3:4) = poShow(2)+[0 -1]; % display nothing after end msg = [sprintf(msg, pos) ': ' ... inStr(poShow(1):poShow(2)) '' inStr(poShow(3):poShow(4)) ]; error( ['JSONparser:invalidFormat: ' msg] ); %%------------------------------------------------------------------------- function str = valid_field(str) global isoct % From MATLAB doc: field names must begin with a letter, which may be % followed by any combination of letters, digits, and underscores. % Invalid characters will be converted to underscores, and the prefix % "x0x[Hex code]_" will be added if the first character is not a letter. pos=regexp(str,'^[^A-Za-z]','once'); if(~isempty(pos)) if(~isoct) str=regexprep(str,'^([^A-Za-z])','x0x${sprintf(''%X'',unicode2native($1))}_','once'); else str=sprintf('x0x%X_%s',char(str(1)),str(2:end)); end end if(isempty(regexp(str,'[^0-9A-Za-z_]', 'once' ))) return; end if(~isoct) str=regexprep(str,'([^0-9A-Za-z_])','_0x${sprintf(''%X'',unicode2native($1))}_'); else pos=regexp(str,'[^0-9A-Za-z_]'); if(isempty(pos)) return; end str0=str; pos0=[0 pos(:)' length(str)]; str=''; for i=1:length(pos) str=[str str0(pos0(i)+1:pos(i)-1) sprintf('_0x%X_',str0(pos(i)))]; end if(pos(end)~=length(str)) str=[str str0(pos0(end-1)+1:pos0(end))]; end end %str(~isletter(str) & ~('0' <= str & str <= '9')) = '_'; %%------------------------------------------------------------------------- function endpos = matching_quote(str,pos) len=length(str); while(pos1 && str(pos-1)=='\')) endpos=pos; return; end end pos=pos+1; end error('unmatched quotation mark'); %%------------------------------------------------------------------------- function [endpos, e1l, e1r, maxlevel] = matching_bracket(str,pos) global arraytoken level=1; maxlevel=level; endpos=0; bpos=arraytoken(arraytoken>=pos); tokens=str(bpos); len=length(tokens); pos=1; e1l=[]; e1r=[]; while(pos<=len) c=tokens(pos); if(c==']') level=level-1; if(isempty(e1r)) e1r=bpos(pos); end if(level==0) endpos=bpos(pos); return end end if(c=='[') if(isempty(e1l)) e1l=bpos(pos); end level=level+1; maxlevel=max(maxlevel,level); end if(c=='"') pos=matching_quote(tokens,pos+1); end pos=pos+1; end if(endpos==0) error('unmatched "]"'); end ================================================ FILE: machine-learning-ex2/ex2/lib/jsonlab/loadubjson.m ================================================ function data = loadubjson(fname,varargin) % % data=loadubjson(fname,opt) % or % data=loadubjson(fname,'param1',value1,'param2',value2,...) % % parse a JSON (JavaScript Object Notation) file or string % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2013/08/01 % % $Id: loadubjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % fname: input file name, if fname contains "{}" or "[]", fname % will be interpreted as a UBJSON string % opt: a struct to store parsing options, opt can be replaced by % a list of ('param',value) pairs - the param string is equivallent % to a field in opt. opt can have the following % fields (first in [.|.] is the default) % % opt.SimplifyCell [0|1]: if set to 1, loadubjson will call cell2mat % for each element of the JSON data, and group % arrays based on the cell2mat rules. % opt.IntEndian [B|L]: specify the endianness of the integer fields % in the UBJSON input data. B - Big-Endian format for % integers (as required in the UBJSON specification); % L - input integer fields are in Little-Endian order. % % output: % dat: a cell array, where {...} blocks are converted into cell arrays, % and [...] are converted to arrays % % examples: % obj=struct('string','value','array',[1 2 3]); % ubjdata=saveubjson('obj',obj); % dat=loadubjson(ubjdata) % dat=loadubjson(['examples' filesep 'example1.ubj']) % dat=loadubjson(['examples' filesep 'example1.ubj'],'SimplifyCell',1) % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % global pos inStr len esc index_esc len_esc isoct arraytoken fileendian systemendian if(regexp(fname,'[\{\}\]\[]','once')) string=fname; elseif(exist(fname,'file')) fid = fopen(fname,'rb'); string = fread(fid,inf,'uint8=>char')'; fclose(fid); else error('input file does not exist'); end pos = 1; len = length(string); inStr = string; isoct=exist('OCTAVE_VERSION','builtin'); arraytoken=find(inStr=='[' | inStr==']' | inStr=='"'); jstr=regexprep(inStr,'\\\\',' '); escquote=regexp(jstr,'\\"'); arraytoken=sort([arraytoken escquote]); % String delimiters and escape chars identified to improve speed: esc = find(inStr=='"' | inStr=='\' ); % comparable to: regexp(inStr, '["\\]'); index_esc = 1; len_esc = length(esc); opt=varargin2struct(varargin{:}); fileendian=upper(jsonopt('IntEndian','B',opt)); [os,maxelem,systemendian]=computer; jsoncount=1; while pos <= len switch(next_char) case '{' data{jsoncount} = parse_object(opt); case '[' data{jsoncount} = parse_array(opt); otherwise error_pos('Outer level structure must be an object or an array'); end jsoncount=jsoncount+1; end % while jsoncount=length(data); if(jsoncount==1 && iscell(data)) data=data{1}; end if(~isempty(data)) if(isstruct(data)) % data can be a struct array data=jstruct2array(data); elseif(iscell(data)) data=jcell2array(data); end end %% function newdata=parse_collection(id,data,obj) if(jsoncount>0 && exist('data','var')) if(~iscell(data)) newdata=cell(1); newdata{1}=data; data=newdata; end end %% function newdata=jcell2array(data) len=length(data); newdata=data; for i=1:len if(isstruct(data{i})) newdata{i}=jstruct2array(data{i}); elseif(iscell(data{i})) newdata{i}=jcell2array(data{i}); end end %%------------------------------------------------------------------------- function newdata=jstruct2array(data) fn=fieldnames(data); newdata=data; len=length(data); for i=1:length(fn) % depth-first for j=1:len if(isstruct(getfield(data(j),fn{i}))) newdata(j)=setfield(newdata(j),fn{i},jstruct2array(getfield(data(j),fn{i}))); end end end if(~isempty(strmatch('x0x5F_ArrayType_',fn)) && ~isempty(strmatch('x0x5F_ArrayData_',fn))) newdata=cell(len,1); for j=1:len ndata=cast(data(j).x0x5F_ArrayData_,data(j).x0x5F_ArrayType_); iscpx=0; if(~isempty(strmatch('x0x5F_ArrayIsComplex_',fn))) if(data(j).x0x5F_ArrayIsComplex_) iscpx=1; end end if(~isempty(strmatch('x0x5F_ArrayIsSparse_',fn))) if(data(j).x0x5F_ArrayIsSparse_) if(~isempty(strmatch('x0x5F_ArraySize_',fn))) dim=double(data(j).x0x5F_ArraySize_); if(iscpx && size(ndata,2)==4-any(dim==1)) ndata(:,end-1)=complex(ndata(:,end-1),ndata(:,end)); end if isempty(ndata) % All-zeros sparse ndata=sparse(dim(1),prod(dim(2:end))); elseif dim(1)==1 % Sparse row vector ndata=sparse(1,ndata(:,1),ndata(:,2),dim(1),prod(dim(2:end))); elseif dim(2)==1 % Sparse column vector ndata=sparse(ndata(:,1),1,ndata(:,2),dim(1),prod(dim(2:end))); else % Generic sparse array. ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3),dim(1),prod(dim(2:end))); end else if(iscpx && size(ndata,2)==4) ndata(:,3)=complex(ndata(:,3),ndata(:,4)); end ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3)); end end elseif(~isempty(strmatch('x0x5F_ArraySize_',fn))) if(iscpx && size(ndata,2)==2) ndata=complex(ndata(:,1),ndata(:,2)); end ndata=reshape(ndata(:),data(j).x0x5F_ArraySize_); end newdata{j}=ndata; end if(len==1) newdata=newdata{1}; end end %%------------------------------------------------------------------------- function object = parse_object(varargin) parse_char('{'); object = []; type=''; count=-1; if(next_char == '$') type=inStr(pos+1); % TODO pos=pos+2; end if(next_char == '#') pos=pos+1; count=double(parse_number()); end if next_char ~= '}' num=0; while 1 str = parseStr(varargin{:}); if isempty(str) error_pos('Name of value at position %d cannot be empty'); end %parse_char(':'); val = parse_value(varargin{:}); num=num+1; eval( sprintf( 'object.%s = val;', valid_field(str) ) ); if next_char == '}' || (count>=0 && num>=count) break; end %parse_char(','); end end if(count==-1) parse_char('}'); end %%------------------------------------------------------------------------- function [cid,len]=elem_info(type) id=strfind('iUIlLdD',type); dataclass={'int8','uint8','int16','int32','int64','single','double'}; bytelen=[1,1,2,4,8,4,8]; if(id>0) cid=dataclass{id}; len=bytelen(id); else error_pos('unsupported type at position %d'); end %%------------------------------------------------------------------------- function [data adv]=parse_block(type,count,varargin) global pos inStr isoct fileendian systemendian [cid,len]=elem_info(type); datastr=inStr(pos:pos+len*count-1); if(isoct) newdata=int8(datastr); else newdata=uint8(datastr); end id=strfind('iUIlLdD',type); if(id<=5 && fileendian~=systemendian) newdata=swapbytes(typecast(newdata,cid)); end data=typecast(newdata,cid); adv=double(len*count); %%------------------------------------------------------------------------- function object = parse_array(varargin) % JSON array is written in row-major order global pos inStr isoct parse_char('['); object = cell(0, 1); dim=[]; type=''; count=-1; if(next_char == '$') type=inStr(pos+1); pos=pos+2; end if(next_char == '#') pos=pos+1; if(next_char=='[') dim=parse_array(varargin{:}); count=prod(double(dim)); else count=double(parse_number()); end end if(~isempty(type)) if(count>=0) [object adv]=parse_block(type,count,varargin{:}); if(~isempty(dim)) object=reshape(object,dim); end pos=pos+adv; return; else endpos=matching_bracket(inStr,pos); [cid,len]=elem_info(type); count=(endpos-pos)/len; [object adv]=parse_block(type,count,varargin{:}); pos=pos+adv; parse_char(']'); return; end end if next_char ~= ']' while 1 val = parse_value(varargin{:}); object{end+1} = val; if next_char == ']' break; end %parse_char(','); end end if(jsonopt('SimplifyCell',0,varargin{:})==1) try oldobj=object; object=cell2mat(object')'; if(iscell(oldobj) && isstruct(object) && numel(object)>1 && jsonopt('SimplifyCellArray',1,varargin{:})==0) object=oldobj; elseif(size(object,1)>1 && ndims(object)==2) object=object'; end catch end end if(count==-1) parse_char(']'); end %%------------------------------------------------------------------------- function parse_char(c) global pos inStr len skip_whitespace; if pos > len || inStr(pos) ~= c error_pos(sprintf('Expected %c at position %%d', c)); else pos = pos + 1; skip_whitespace; end %%------------------------------------------------------------------------- function c = next_char global pos inStr len skip_whitespace; if pos > len c = []; else c = inStr(pos); end %%------------------------------------------------------------------------- function skip_whitespace global pos inStr len while pos <= len && isspace(inStr(pos)) pos = pos + 1; end %%------------------------------------------------------------------------- function str = parseStr(varargin) global pos inStr esc index_esc len_esc % len, ns = length(inStr), keyboard type=inStr(pos); if type ~= 'S' && type ~= 'C' && type ~= 'H' error_pos('String starting with S expected at position %d'); else pos = pos + 1; end if(type == 'C') str=inStr(pos); pos=pos+1; return; end bytelen=double(parse_number()); if(length(inStr)>=pos+bytelen-1) str=inStr(pos:pos+bytelen-1); pos=pos+bytelen; else error_pos('End of file while expecting end of inStr'); end %%------------------------------------------------------------------------- function num = parse_number(varargin) global pos inStr len isoct fileendian systemendian id=strfind('iUIlLdD',inStr(pos)); if(isempty(id)) error_pos('expecting a number at position %d'); end type={'int8','uint8','int16','int32','int64','single','double'}; bytelen=[1,1,2,4,8,4,8]; datastr=inStr(pos+1:pos+bytelen(id)); if(isoct) newdata=int8(datastr); else newdata=uint8(datastr); end if(id<=5 && fileendian~=systemendian) newdata=swapbytes(typecast(newdata,type{id})); end num=typecast(newdata,type{id}); pos = pos + bytelen(id)+1; %%------------------------------------------------------------------------- function val = parse_value(varargin) global pos inStr len true = 1; false = 0; switch(inStr(pos)) case {'S','C','H'} val = parseStr(varargin{:}); return; case '[' val = parse_array(varargin{:}); return; case '{' val = parse_object(varargin{:}); if isstruct(val) if(~isempty(strmatch('x0x5F_ArrayType_',fieldnames(val), 'exact'))) val=jstruct2array(val); end elseif isempty(val) val = struct; end return; case {'i','U','I','l','L','d','D'} val = parse_number(varargin{:}); return; case 'T' val = true; pos = pos + 1; return; case 'F' val = false; pos = pos + 1; return; case {'Z','N'} val = []; pos = pos + 1; return; end error_pos('Value expected at position %d'); %%------------------------------------------------------------------------- function error_pos(msg) global pos inStr len poShow = max(min([pos-15 pos-1 pos pos+20],len),1); if poShow(3) == poShow(2) poShow(3:4) = poShow(2)+[0 -1]; % display nothing after end msg = [sprintf(msg, pos) ': ' ... inStr(poShow(1):poShow(2)) '' inStr(poShow(3):poShow(4)) ]; error( ['JSONparser:invalidFormat: ' msg] ); %%------------------------------------------------------------------------- function str = valid_field(str) global isoct % From MATLAB doc: field names must begin with a letter, which may be % followed by any combination of letters, digits, and underscores. % Invalid characters will be converted to underscores, and the prefix % "x0x[Hex code]_" will be added if the first character is not a letter. pos=regexp(str,'^[^A-Za-z]','once'); if(~isempty(pos)) if(~isoct) str=regexprep(str,'^([^A-Za-z])','x0x${sprintf(''%X'',unicode2native($1))}_','once'); else str=sprintf('x0x%X_%s',char(str(1)),str(2:end)); end end if(isempty(regexp(str,'[^0-9A-Za-z_]', 'once' ))) return; end if(~isoct) str=regexprep(str,'([^0-9A-Za-z_])','_0x${sprintf(''%X'',unicode2native($1))}_'); else pos=regexp(str,'[^0-9A-Za-z_]'); if(isempty(pos)) return; end str0=str; pos0=[0 pos(:)' length(str)]; str=''; for i=1:length(pos) str=[str str0(pos0(i)+1:pos(i)-1) sprintf('_0x%X_',str0(pos(i)))]; end if(pos(end)~=length(str)) str=[str str0(pos0(end-1)+1:pos0(end))]; end end %str(~isletter(str) & ~('0' <= str & str <= '9')) = '_'; %%------------------------------------------------------------------------- function endpos = matching_quote(str,pos) len=length(str); while(pos1 && str(pos-1)=='\')) endpos=pos; return; end end pos=pos+1; end error('unmatched quotation mark'); %%------------------------------------------------------------------------- function [endpos e1l e1r maxlevel] = matching_bracket(str,pos) global arraytoken level=1; maxlevel=level; endpos=0; bpos=arraytoken(arraytoken>=pos); tokens=str(bpos); len=length(tokens); pos=1; e1l=[]; e1r=[]; while(pos<=len) c=tokens(pos); if(c==']') level=level-1; if(isempty(e1r)) e1r=bpos(pos); end if(level==0) endpos=bpos(pos); return end end if(c=='[') if(isempty(e1l)) e1l=bpos(pos); end level=level+1; maxlevel=max(maxlevel,level); end if(c=='"') pos=matching_quote(tokens,pos+1); end pos=pos+1; end if(endpos==0) error('unmatched "]"'); end ================================================ FILE: machine-learning-ex2/ex2/lib/jsonlab/mergestruct.m ================================================ function s=mergestruct(s1,s2) % % s=mergestruct(s1,s2) % % merge two struct objects into one % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % date: 2012/12/22 % % input: % s1,s2: a struct object, s1 and s2 can not be arrays % % output: % s: the merged struct object. fields in s1 and s2 will be combined in s. % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(~isstruct(s1) || ~isstruct(s2)) error('input parameters contain non-struct'); end if(length(s1)>1 || length(s2)>1) error('can not merge struct arrays'); end fn=fieldnames(s2); s=s1; for i=1:length(fn) s=setfield(s,fn{i},getfield(s2,fn{i})); end ================================================ FILE: machine-learning-ex2/ex2/lib/jsonlab/savejson.m ================================================ function json=savejson(rootname,obj,varargin) % % json=savejson(rootname,obj,filename) % or % json=savejson(rootname,obj,opt) % json=savejson(rootname,obj,'param1',value1,'param2',value2,...) % % convert a MATLAB object (cell, struct or array) into a JSON (JavaScript % Object Notation) string % % author: Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2011/09/09 % % $Id: savejson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % rootname: the name of the root-object, when set to '', the root name % is ignored, however, when opt.ForceRootName is set to 1 (see below), % the MATLAB variable name will be used as the root name. % obj: a MATLAB object (array, cell, cell array, struct, struct array). % filename: a string for the file name to save the output JSON data. % opt: a struct for additional options, ignore to use default values. % opt can have the following fields (first in [.|.] is the default) % % opt.FileName [''|string]: a file name to save the output JSON data % opt.FloatFormat ['%.10g'|string]: format to show each numeric element % of a 1D/2D array; % opt.ArrayIndent [1|0]: if 1, output explicit data array with % precedent indentation; if 0, no indentation % opt.ArrayToStruct[0|1]: when set to 0, savejson outputs 1D/2D % array in JSON array format; if sets to 1, an % array will be shown as a struct with fields % "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for % sparse arrays, the non-zero elements will be % saved to _ArrayData_ field in triplet-format i.e. % (ix,iy,val) and "_ArrayIsSparse_" will be added % with a value of 1; for a complex array, the % _ArrayData_ array will include two columns % (4 for sparse) to record the real and imaginary % parts, and also "_ArrayIsComplex_":1 is added. % opt.ParseLogical [0|1]: if this is set to 1, logical array elem % will use true/false rather than 1/0. % opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single % numerical element will be shown without a square % bracket, unless it is the root object; if 0, square % brackets are forced for any numerical arrays. % opt.ForceRootName [0|1]: when set to 1 and rootname is empty, savejson % will use the name of the passed obj variable as the % root object name; if obj is an expression and % does not have a name, 'root' will be used; if this % is set to 0 and rootname is empty, the root level % will be merged down to the lower level. % opt.Inf ['"$1_Inf_"'|string]: a customized regular expression pattern % to represent +/-Inf. The matched pattern is '([-+]*)Inf' % and $1 represents the sign. For those who want to use % 1e999 to represent Inf, they can set opt.Inf to '$11e999' % opt.NaN ['"_NaN_"'|string]: a customized regular expression pattern % to represent NaN % opt.JSONP [''|string]: to generate a JSONP output (JSON with padding), % for example, if opt.JSONP='foo', the JSON data is % wrapped inside a function call as 'foo(...);' % opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson % back to the string form % opt.SaveBinary [0|1]: 1 - save the JSON file in binary mode; 0 - text mode. % opt.Compact [0|1]: 1- out compact JSON format (remove all newlines and tabs) % % opt can be replaced by a list of ('param',value) pairs. The param % string is equivallent to a field in opt and is case sensitive. % output: % json: a string in the JSON format (see http://json.org) % % examples: % jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... % 'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],... % 'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;... % 2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],... % 'MeshCreator','FangQ','MeshTitle','T6 Cube',... % 'SpecialData',[nan, inf, -inf]); % savejson('jmesh',jsonmesh) % savejson('',jsonmesh,'ArrayIndent',0,'FloatFormat','\t%.5g') % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(nargin==1) varname=inputname(1); obj=rootname; if(isempty(varname)) varname='root'; end rootname=varname; else varname=inputname(2); end if(length(varargin)==1 && ischar(varargin{1})) opt=struct('FileName',varargin{1}); else opt=varargin2struct(varargin{:}); end opt.IsOctave=exist('OCTAVE_VERSION','builtin'); rootisarray=0; rootlevel=1; forceroot=jsonopt('ForceRootName',0,opt); if((isnumeric(obj) || islogical(obj) || ischar(obj) || isstruct(obj) || iscell(obj)) && isempty(rootname) && forceroot==0) rootisarray=1; rootlevel=0; else if(isempty(rootname)) rootname=varname; end end if((isstruct(obj) || iscell(obj))&& isempty(rootname) && forceroot) rootname='root'; end whitespaces=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); if(jsonopt('Compact',0,opt)==1) whitespaces=struct('tab','','newline','','sep',','); end if(~isfield(opt,'whitespaces_')) opt.whitespaces_=whitespaces; end nl=whitespaces.newline; json=obj2json(rootname,obj,rootlevel,opt); if(rootisarray) json=sprintf('%s%s',json,nl); else json=sprintf('{%s%s%s}\n',nl,json,nl); end jsonp=jsonopt('JSONP','',opt); if(~isempty(jsonp)) json=sprintf('%s(%s);%s',jsonp,json,nl); end % save to a file if FileName is set, suggested by Patrick Rapin if(~isempty(jsonopt('FileName','',opt))) if(jsonopt('SaveBinary',0,opt)==1) fid = fopen(opt.FileName, 'wb'); fwrite(fid,json); else fid = fopen(opt.FileName, 'wt'); fwrite(fid,json,'char'); end fclose(fid); end %%------------------------------------------------------------------------- function txt=obj2json(name,item,level,varargin) if(iscell(item)) txt=cell2json(name,item,level,varargin{:}); elseif(isstruct(item)) txt=struct2json(name,item,level,varargin{:}); elseif(ischar(item)) txt=str2json(name,item,level,varargin{:}); else txt=mat2json(name,item,level,varargin{:}); end %%------------------------------------------------------------------------- function txt=cell2json(name,item,level,varargin) txt=''; if(~iscell(item)) error('input is not a cell'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); ws=jsonopt('whitespaces_',struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')),varargin{:}); padding0=repmat(ws.tab,1,level); padding2=repmat(ws.tab,1,level+1); nl=ws.newline; if(len>1) if(~isempty(name)) txt=sprintf('%s"%s": [%s',padding0, checkname(name,varargin{:}),nl); name=''; else txt=sprintf('%s[%s',padding0,nl); end elseif(len==0) if(~isempty(name)) txt=sprintf('%s"%s": []',padding0, checkname(name,varargin{:})); name=''; else txt=sprintf('%s[]',padding0); end end for j=1:dim(2) if(dim(1)>1) txt=sprintf('%s%s[%s',txt,padding2,nl); end for i=1:dim(1) txt=sprintf('%s%s',txt,obj2json(name,item{i,j},level+(dim(1)>1)+1,varargin{:})); if(i1) txt=sprintf('%s%s%s]',txt,nl,padding2); end if(j1) txt=sprintf('%s%s%s]',txt,nl,padding0); end %%------------------------------------------------------------------------- function txt=struct2json(name,item,level,varargin) txt=''; if(~isstruct(item)) error('input is not a struct'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); ws=struct('tab',sprintf('\t'),'newline',sprintf('\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding0=repmat(ws.tab,1,level); padding2=repmat(ws.tab,1,level+1); padding1=repmat(ws.tab,1,level+(dim(1)>1)+(len>1)); nl=ws.newline; if(~isempty(name)) if(len>1) txt=sprintf('%s"%s": [%s',padding0,checkname(name,varargin{:}),nl); end else if(len>1) txt=sprintf('%s[%s',padding0,nl); end end for j=1:dim(2) if(dim(1)>1) txt=sprintf('%s%s[%s',txt,padding2,nl); end for i=1:dim(1) names = fieldnames(item(i,j)); if(~isempty(name) && len==1) txt=sprintf('%s%s"%s": {%s',txt,padding1, checkname(name,varargin{:}),nl); else txt=sprintf('%s%s{%s',txt,padding1,nl); end if(~isempty(names)) for e=1:length(names) txt=sprintf('%s%s',txt,obj2json(names{e},getfield(item(i,j),... names{e}),level+(dim(1)>1)+1+(len>1),varargin{:})); if(e1) txt=sprintf('%s%s%s]',txt,nl,padding2); end if(j1) txt=sprintf('%s%s%s]',txt,nl,padding0); end %%------------------------------------------------------------------------- function txt=str2json(name,item,level,varargin) txt=''; if(~ischar(item)) error('input is not a string'); end item=reshape(item, max(size(item),[1 0])); len=size(item,1); ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding1=repmat(ws.tab,1,level); padding0=repmat(ws.tab,1,level+1); nl=ws.newline; sep=ws.sep; if(~isempty(name)) if(len>1) txt=sprintf('%s"%s": [%s',padding1,checkname(name,varargin{:}),nl); end else if(len>1) txt=sprintf('%s[%s',padding1,nl); end end isoct=jsonopt('IsOctave',0,varargin{:}); for e=1:len if(isoct) val=regexprep(item(e,:),'\\','\\'); val=regexprep(val,'"','\"'); val=regexprep(val,'^"','\"'); else val=regexprep(item(e,:),'\\','\\\\'); val=regexprep(val,'"','\\"'); val=regexprep(val,'^"','\\"'); end val=escapejsonstring(val); if(len==1) obj=['"' checkname(name,varargin{:}) '": ' '"',val,'"']; if(isempty(name)) obj=['"',val,'"']; end txt=sprintf('%s%s%s%s',txt,padding1,obj); else txt=sprintf('%s%s%s%s',txt,padding0,['"',val,'"']); end if(e==len) sep=''; end txt=sprintf('%s%s',txt,sep); end if(len>1) txt=sprintf('%s%s%s%s',txt,nl,padding1,']'); end %%------------------------------------------------------------------------- function txt=mat2json(name,item,level,varargin) if(~isnumeric(item) && ~islogical(item)) error('input is not an array'); end ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding1=repmat(ws.tab,1,level); padding0=repmat(ws.tab,1,level+1); nl=ws.newline; sep=ws.sep; if(length(size(item))>2 || issparse(item) || ~isreal(item) || ... isempty(item) ||jsonopt('ArrayToStruct',0,varargin{:})) if(isempty(name)) txt=sprintf('%s{%s%s"_ArrayType_": "%s",%s%s"_ArraySize_": %s,%s',... padding1,nl,padding0,class(item),nl,padding0,regexprep(mat2str(size(item)),'\s+',','),nl); else txt=sprintf('%s"%s": {%s%s"_ArrayType_": "%s",%s%s"_ArraySize_": %s,%s',... padding1,checkname(name,varargin{:}),nl,padding0,class(item),nl,padding0,regexprep(mat2str(size(item)),'\s+',','),nl); end else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1 && level>0) numtxt=regexprep(regexprep(matdata2json(item,level+1,varargin{:}),'^\[',''),']',''); else numtxt=matdata2json(item,level+1,varargin{:}); end if(isempty(name)) txt=sprintf('%s%s',padding1,numtxt); else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1) txt=sprintf('%s"%s": %s',padding1,checkname(name,varargin{:}),numtxt); else txt=sprintf('%s"%s": %s',padding1,checkname(name,varargin{:}),numtxt); end end return; end dataformat='%s%s%s%s%s'; if(issparse(item)) [ix,iy]=find(item); data=full(item(find(item))); if(~isreal(item)) data=[real(data(:)),imag(data(:))]; if(size(item,1)==1) % Kludge to have data's 'transposedness' match item's. % (Necessary for complex row vector handling below.) data=data'; end txt=sprintf(dataformat,txt,padding0,'"_ArrayIsComplex_": ','1', sep); end txt=sprintf(dataformat,txt,padding0,'"_ArrayIsSparse_": ','1', sep); if(size(item,1)==1) % Row vector, store only column indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([iy(:),data'],level+2,varargin{:}), nl); elseif(size(item,2)==1) % Column vector, store only row indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([ix,data],level+2,varargin{:}), nl); else % General case, store row and column indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([ix,iy,data],level+2,varargin{:}), nl); end else if(isreal(item)) txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json(item(:)',level+2,varargin{:}), nl); else txt=sprintf(dataformat,txt,padding0,'"_ArrayIsComplex_": ','1', sep); txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([real(item(:)) imag(item(:))],level+2,varargin{:}), nl); end end txt=sprintf('%s%s%s',txt,padding1,'}'); %%------------------------------------------------------------------------- function txt=matdata2json(mat,level,varargin) ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); tab=ws.tab; nl=ws.newline; if(size(mat,1)==1) pre=''; post=''; level=level-1; else pre=sprintf('[%s',nl); post=sprintf('%s%s]',nl,repmat(tab,1,level-1)); end if(isempty(mat)) txt='null'; return; end floatformat=jsonopt('FloatFormat','%.10g',varargin{:}); %if(numel(mat)>1) formatstr=['[' repmat([floatformat ','],1,size(mat,2)-1) [floatformat sprintf('],%s',nl)]]; %else % formatstr=[repmat([floatformat ','],1,size(mat,2)-1) [floatformat sprintf(',\n')]]; %end if(nargin>=2 && size(mat,1)>1 && jsonopt('ArrayIndent',1,varargin{:})==1) formatstr=[repmat(tab,1,level) formatstr]; end txt=sprintf(formatstr,mat'); txt(end-length(nl):end)=[]; if(islogical(mat) && jsonopt('ParseLogical',0,varargin{:})==1) txt=regexprep(txt,'1','true'); txt=regexprep(txt,'0','false'); end %txt=regexprep(mat2str(mat),'\s+',','); %txt=regexprep(txt,';',sprintf('],\n[')); % if(nargin>=2 && size(mat,1)>1) % txt=regexprep(txt,'\[',[repmat(sprintf('\t'),1,level) '[']); % end txt=[pre txt post]; if(any(isinf(mat(:)))) txt=regexprep(txt,'([-+]*)Inf',jsonopt('Inf','"$1_Inf_"',varargin{:})); end if(any(isnan(mat(:)))) txt=regexprep(txt,'NaN',jsonopt('NaN','"_NaN_"',varargin{:})); end %%------------------------------------------------------------------------- function newname=checkname(name,varargin) isunpack=jsonopt('UnpackHex',1,varargin{:}); newname=name; if(isempty(regexp(name,'0x([0-9a-fA-F]+)_','once'))) return end if(isunpack) isoct=jsonopt('IsOctave',0,varargin{:}); if(~isoct) newname=regexprep(name,'(^x|_){1}0x([0-9a-fA-F]+)_','${native2unicode(hex2dec($2))}'); else pos=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','start'); pend=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','end'); if(isempty(pos)) return; end str0=name; pos0=[0 pend(:)' length(name)]; newname=''; for i=1:length(pos) newname=[newname str0(pos0(i)+1:pos(i)-1) char(hex2dec(str0(pos(i)+3:pend(i)-1)))]; end if(pos(end)~=length(name)) newname=[newname str0(pos0(end-1)+1:pos0(end))]; end end end %%------------------------------------------------------------------------- function newstr=escapejsonstring(str) newstr=str; isoct=exist('OCTAVE_VERSION','builtin'); if(isoct) vv=sscanf(OCTAVE_VERSION,'%f'); if(vv(1)>=3.8) isoct=0; end end if(isoct) escapechars={'\a','\f','\n','\r','\t','\v'}; for i=1:length(escapechars); newstr=regexprep(newstr,escapechars{i},escapechars{i}); end else escapechars={'\a','\b','\f','\n','\r','\t','\v'}; for i=1:length(escapechars); newstr=regexprep(newstr,escapechars{i},regexprep(escapechars{i},'\\','\\\\')); end end ================================================ FILE: machine-learning-ex2/ex2/lib/jsonlab/saveubjson.m ================================================ function json=saveubjson(rootname,obj,varargin) % % json=saveubjson(rootname,obj,filename) % or % json=saveubjson(rootname,obj,opt) % json=saveubjson(rootname,obj,'param1',value1,'param2',value2,...) % % convert a MATLAB object (cell, struct or array) into a Universal % Binary JSON (UBJSON) binary string % % author: Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2013/08/17 % % $Id: saveubjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % rootname: the name of the root-object, when set to '', the root name % is ignored, however, when opt.ForceRootName is set to 1 (see below), % the MATLAB variable name will be used as the root name. % obj: a MATLAB object (array, cell, cell array, struct, struct array) % filename: a string for the file name to save the output UBJSON data % opt: a struct for additional options, ignore to use default values. % opt can have the following fields (first in [.|.] is the default) % % opt.FileName [''|string]: a file name to save the output JSON data % opt.ArrayToStruct[0|1]: when set to 0, saveubjson outputs 1D/2D % array in JSON array format; if sets to 1, an % array will be shown as a struct with fields % "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for % sparse arrays, the non-zero elements will be % saved to _ArrayData_ field in triplet-format i.e. % (ix,iy,val) and "_ArrayIsSparse_" will be added % with a value of 1; for a complex array, the % _ArrayData_ array will include two columns % (4 for sparse) to record the real and imaginary % parts, and also "_ArrayIsComplex_":1 is added. % opt.ParseLogical [1|0]: if this is set to 1, logical array elem % will use true/false rather than 1/0. % opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single % numerical element will be shown without a square % bracket, unless it is the root object; if 0, square % brackets are forced for any numerical arrays. % opt.ForceRootName [0|1]: when set to 1 and rootname is empty, saveubjson % will use the name of the passed obj variable as the % root object name; if obj is an expression and % does not have a name, 'root' will be used; if this % is set to 0 and rootname is empty, the root level % will be merged down to the lower level. % opt.JSONP [''|string]: to generate a JSONP output (JSON with padding), % for example, if opt.JSON='foo', the JSON data is % wrapped inside a function call as 'foo(...);' % opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson % back to the string form % % opt can be replaced by a list of ('param',value) pairs. The param % string is equivallent to a field in opt and is case sensitive. % output: % json: a binary string in the UBJSON format (see http://ubjson.org) % % examples: % jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... % 'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],... % 'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;... % 2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],... % 'MeshCreator','FangQ','MeshTitle','T6 Cube',... % 'SpecialData',[nan, inf, -inf]); % saveubjson('jsonmesh',jsonmesh) % saveubjson('jsonmesh',jsonmesh,'meshdata.ubj') % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(nargin==1) varname=inputname(1); obj=rootname; if(isempty(varname)) varname='root'; end rootname=varname; else varname=inputname(2); end if(length(varargin)==1 && ischar(varargin{1})) opt=struct('FileName',varargin{1}); else opt=varargin2struct(varargin{:}); end opt.IsOctave=exist('OCTAVE_VERSION','builtin'); rootisarray=0; rootlevel=1; forceroot=jsonopt('ForceRootName',0,opt); if((isnumeric(obj) || islogical(obj) || ischar(obj) || isstruct(obj) || iscell(obj)) && isempty(rootname) && forceroot==0) rootisarray=1; rootlevel=0; else if(isempty(rootname)) rootname=varname; end end if((isstruct(obj) || iscell(obj))&& isempty(rootname) && forceroot) rootname='root'; end json=obj2ubjson(rootname,obj,rootlevel,opt); if(~rootisarray) json=['{' json '}']; end jsonp=jsonopt('JSONP','',opt); if(~isempty(jsonp)) json=[jsonp '(' json ')']; end % save to a file if FileName is set, suggested by Patrick Rapin if(~isempty(jsonopt('FileName','',opt))) fid = fopen(opt.FileName, 'wb'); fwrite(fid,json); fclose(fid); end %%------------------------------------------------------------------------- function txt=obj2ubjson(name,item,level,varargin) if(iscell(item)) txt=cell2ubjson(name,item,level,varargin{:}); elseif(isstruct(item)) txt=struct2ubjson(name,item,level,varargin{:}); elseif(ischar(item)) txt=str2ubjson(name,item,level,varargin{:}); else txt=mat2ubjson(name,item,level,varargin{:}); end %%------------------------------------------------------------------------- function txt=cell2ubjson(name,item,level,varargin) txt=''; if(~iscell(item)) error('input is not a cell'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); % let's handle 1D cell first if(len>1) if(~isempty(name)) txt=[S_(checkname(name,varargin{:})) '[']; name=''; else txt='['; end elseif(len==0) if(~isempty(name)) txt=[S_(checkname(name,varargin{:})) 'Z']; name=''; else txt='Z'; end end for j=1:dim(2) if(dim(1)>1) txt=[txt '[']; end for i=1:dim(1) txt=[txt obj2ubjson(name,item{i,j},level+(len>1),varargin{:})]; end if(dim(1)>1) txt=[txt ']']; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=struct2ubjson(name,item,level,varargin) txt=''; if(~isstruct(item)) error('input is not a struct'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); if(~isempty(name)) if(len>1) txt=[S_(checkname(name,varargin{:})) '[']; end else if(len>1) txt='['; end end for j=1:dim(2) if(dim(1)>1) txt=[txt '[']; end for i=1:dim(1) names = fieldnames(item(i,j)); if(~isempty(name) && len==1) txt=[txt S_(checkname(name,varargin{:})) '{']; else txt=[txt '{']; end if(~isempty(names)) for e=1:length(names) txt=[txt obj2ubjson(names{e},getfield(item(i,j),... names{e}),level+(dim(1)>1)+1+(len>1),varargin{:})]; end end txt=[txt '}']; end if(dim(1)>1) txt=[txt ']']; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=str2ubjson(name,item,level,varargin) txt=''; if(~ischar(item)) error('input is not a string'); end item=reshape(item, max(size(item),[1 0])); len=size(item,1); if(~isempty(name)) if(len>1) txt=[S_(checkname(name,varargin{:})) '[']; end else if(len>1) txt='['; end end isoct=jsonopt('IsOctave',0,varargin{:}); for e=1:len val=item(e,:); if(len==1) obj=['' S_(checkname(name,varargin{:})) '' '',S_(val),'']; if(isempty(name)) obj=['',S_(val),'']; end txt=[txt,'',obj]; else txt=[txt,'',['',S_(val),'']]; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=mat2ubjson(name,item,level,varargin) if(~isnumeric(item) && ~islogical(item)) error('input is not an array'); end if(length(size(item))>2 || issparse(item) || ~isreal(item) || ... isempty(item) || jsonopt('ArrayToStruct',0,varargin{:})) cid=I_(uint32(max(size(item)))); if(isempty(name)) txt=['{' S_('_ArrayType_'),S_(class(item)),S_('_ArraySize_'),I_a(size(item),cid(1)) ]; else if(isempty(item)) txt=[S_(checkname(name,varargin{:})),'Z']; return; else txt=[S_(checkname(name,varargin{:})),'{',S_('_ArrayType_'),S_(class(item)),S_('_ArraySize_'),I_a(size(item),cid(1))]; end end else if(isempty(name)) txt=matdata2ubjson(item,level+1,varargin{:}); else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1) numtxt=regexprep(regexprep(matdata2ubjson(item,level+1,varargin{:}),'^\[',''),']',''); txt=[S_(checkname(name,varargin{:})) numtxt]; else txt=[S_(checkname(name,varargin{:})),matdata2ubjson(item,level+1,varargin{:})]; end end return; end if(issparse(item)) [ix,iy]=find(item); data=full(item(find(item))); if(~isreal(item)) data=[real(data(:)),imag(data(:))]; if(size(item,1)==1) % Kludge to have data's 'transposedness' match item's. % (Necessary for complex row vector handling below.) data=data'; end txt=[txt,S_('_ArrayIsComplex_'),'T']; end txt=[txt,S_('_ArrayIsSparse_'),'T']; if(size(item,1)==1) % Row vector, store only column indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([iy(:),data'],level+2,varargin{:})]; elseif(size(item,2)==1) % Column vector, store only row indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([ix,data],level+2,varargin{:})]; else % General case, store row and column indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([ix,iy,data],level+2,varargin{:})]; end else if(isreal(item)) txt=[txt,S_('_ArrayData_'),... matdata2ubjson(item(:)',level+2,varargin{:})]; else txt=[txt,S_('_ArrayIsComplex_'),'T']; txt=[txt,S_('_ArrayData_'),... matdata2ubjson([real(item(:)) imag(item(:))],level+2,varargin{:})]; end end txt=[txt,'}']; %%------------------------------------------------------------------------- function txt=matdata2ubjson(mat,level,varargin) if(isempty(mat)) txt='Z'; return; end if(size(mat,1)==1) level=level-1; end type=''; hasnegtive=(mat<0); if(isa(mat,'integer') || isinteger(mat) || (isfloat(mat) && all(mod(mat(:),1) == 0))) if(isempty(hasnegtive)) if(max(mat(:))<=2^8) type='U'; end end if(isempty(type)) % todo - need to consider negative ones separately id= histc(abs(max(mat(:))),[0 2^7 2^15 2^31 2^63]); if(isempty(find(id))) error('high-precision data is not yet supported'); end key='iIlL'; type=key(find(id)); end txt=[I_a(mat(:),type,size(mat))]; elseif(islogical(mat)) logicalval='FT'; if(numel(mat)==1) txt=logicalval(mat+1); else txt=['[$U#' I_a(size(mat),'l') typecast(swapbytes(uint8(mat(:)')),'uint8')]; end else if(numel(mat)==1) txt=['[' D_(mat) ']']; else txt=D_a(mat(:),'D',size(mat)); end end %txt=regexprep(mat2str(mat),'\s+',','); %txt=regexprep(txt,';',sprintf('],[')); % if(nargin>=2 && size(mat,1)>1) % txt=regexprep(txt,'\[',[repmat(sprintf('\t'),1,level) '[']); % end if(any(isinf(mat(:)))) txt=regexprep(txt,'([-+]*)Inf',jsonopt('Inf','"$1_Inf_"',varargin{:})); end if(any(isnan(mat(:)))) txt=regexprep(txt,'NaN',jsonopt('NaN','"_NaN_"',varargin{:})); end %%------------------------------------------------------------------------- function newname=checkname(name,varargin) isunpack=jsonopt('UnpackHex',1,varargin{:}); newname=name; if(isempty(regexp(name,'0x([0-9a-fA-F]+)_','once'))) return end if(isunpack) isoct=jsonopt('IsOctave',0,varargin{:}); if(~isoct) newname=regexprep(name,'(^x|_){1}0x([0-9a-fA-F]+)_','${native2unicode(hex2dec($2))}'); else pos=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','start'); pend=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','end'); if(isempty(pos)) return; end str0=name; pos0=[0 pend(:)' length(name)]; newname=''; for i=1:length(pos) newname=[newname str0(pos0(i)+1:pos(i)-1) char(hex2dec(str0(pos(i)+3:pend(i)-1)))]; end if(pos(end)~=length(name)) newname=[newname str0(pos0(end-1)+1:pos0(end))]; end end end %%------------------------------------------------------------------------- function val=S_(str) if(length(str)==1) val=['C' str]; else val=['S' I_(int32(length(str))) str]; end %%------------------------------------------------------------------------- function val=I_(num) if(~isinteger(num)) error('input is not an integer'); end if(num>=0 && num<255) val=['U' data2byte(swapbytes(cast(num,'uint8')),'uint8')]; return; end key='iIlL'; cid={'int8','int16','int32','int64'}; for i=1:4 if((num>0 && num<2^(i*8-1)) || (num<0 && num>=-2^(i*8-1))) val=[key(i) data2byte(swapbytes(cast(num,cid{i})),'uint8')]; return; end end error('unsupported integer'); %%------------------------------------------------------------------------- function val=D_(num) if(~isfloat(num)) error('input is not a float'); end if(isa(num,'single')) val=['d' data2byte(num,'uint8')]; else val=['D' data2byte(num,'uint8')]; end %%------------------------------------------------------------------------- function data=I_a(num,type,dim,format) id=find(ismember('iUIlL',type)); if(id==0) error('unsupported integer array'); end % based on UBJSON specs, all integer types are stored in big endian format if(id==1) data=data2byte(swapbytes(int8(num)),'uint8'); blen=1; elseif(id==2) data=data2byte(swapbytes(uint8(num)),'uint8'); blen=1; elseif(id==3) data=data2byte(swapbytes(int16(num)),'uint8'); blen=2; elseif(id==4) data=data2byte(swapbytes(int32(num)),'uint8'); blen=4; elseif(id==5) data=data2byte(swapbytes(int64(num)),'uint8'); blen=8; end if(nargin>=3 && length(dim)>=2 && prod(dim)~=dim(2)) format='opt'; end if((nargin<4 || strcmp(format,'opt')) && numel(num)>1) if(nargin>=3 && (length(dim)==1 || (length(dim)>=2 && prod(dim)~=dim(2)))) cid=I_(uint32(max(dim))); data=['$' type '#' I_a(dim,cid(1)) data(:)']; else data=['$' type '#' I_(int32(numel(data)/blen)) data(:)']; end data=['[' data(:)']; else data=reshape(data,blen,numel(data)/blen); data(2:blen+1,:)=data; data(1,:)=type; data=data(:)'; data=['[' data(:)' ']']; end %%------------------------------------------------------------------------- function data=D_a(num,type,dim,format) id=find(ismember('dD',type)); if(id==0) error('unsupported float array'); end if(id==1) data=data2byte(single(num),'uint8'); elseif(id==2) data=data2byte(double(num),'uint8'); end if(nargin>=3 && length(dim)>=2 && prod(dim)~=dim(2)) format='opt'; end if((nargin<4 || strcmp(format,'opt')) && numel(num)>1) if(nargin>=3 && (length(dim)==1 || (length(dim)>=2 && prod(dim)~=dim(2)))) cid=I_(uint32(max(dim))); data=['$' type '#' I_a(dim,cid(1)) data(:)']; else data=['$' type '#' I_(int32(numel(data)/(id*4))) data(:)']; end data=['[' data]; else data=reshape(data,(id*4),length(data)/(id*4)); data(2:(id*4+1),:)=data; data(1,:)=type; data=data(:)'; data=['[' data(:)' ']']; end %%------------------------------------------------------------------------- function bytes=data2byte(varargin) bytes=typecast(varargin{:}); bytes=bytes(:)'; ================================================ FILE: machine-learning-ex2/ex2/lib/jsonlab/varargin2struct.m ================================================ function opt=varargin2struct(varargin) % % opt=varargin2struct('param1',value1,'param2',value2,...) % or % opt=varargin2struct(...,optstruct,...) % % convert a series of input parameters into a structure % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % date: 2012/12/22 % % input: % 'param', value: the input parameters should be pairs of a string and a value % optstruct: if a parameter is a struct, the fields will be merged to the output struct % % output: % opt: a struct where opt.param1=value1, opt.param2=value2 ... % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % len=length(varargin); opt=struct; if(len==0) return; end i=1; while(i<=len) if(isstruct(varargin{i})) opt=mergestruct(opt,varargin{i}); elseif(ischar(varargin{i}) && i3 matrix, where the first column is all-ones % Plot Data plotData(X(:,2:3), y); hold on if size(X, 2) <= 3 % Only need 2 points to define a line, so choose two endpoints plot_x = [min(X(:,2))-2, max(X(:,2))+2]; % Calculate the decision boundary line plot_y = (-1./theta(3)).*(theta(2).*plot_x + theta(1)); % plot_xΪX1plot_yΪX2 % Plot, and adjust axes for better viewing plot(plot_x, plot_y) % Legend, specific for the exercise legend('Admitted', 'Not admitted', 'Decision Boundary') axis([30, 100, 30, 100]) else % Here is the grid range u = linspace(-1, 1.5, 50); v = linspace(-1, 1.5, 50); z = zeros(length(u), length(v)); % Evaluate z = theta*x over the grid for i = 1:length(u) for j = 1:length(v) z(i,j) = mapFeature(u(i), v(j))*theta; end end z = z'; % important to transpose z before calling contour % Plot z = 0 % Notice you need to specify the range [0, 0] contour(u, v, z, [0, 0], 'LineWidth', 2) %ֵΪ0ĵֵ end hold off end ================================================ FILE: machine-learning-ex2/ex2/predict.m ================================================ function p = predict(theta, X) %PREDICT Predict whether the label is 0 or 1 using learned logistic %regression parameters theta % p = PREDICT(theta, X) computes the predictions for X using a % threshold at 0.5 (i.e., if sigmoid(theta'*x) >= 0.5, predict 1) m = size(X, 1); % Number of training examples % You need to return the following variables correctly p = zeros(m, 1); % ====================== YOUR CODE HERE ====================== % Instructions: Complete the following code to make predictions using % your learned logistic regression parameters. % You should set p to a vector of 0's and 1's % p = sigmoid(X*theta); for i=1:m if p(i)>0.5 p(i)=1; else p(i)=0; end end % ========================================================================= end ================================================ FILE: machine-learning-ex2/ex2/sigmoid.m ================================================ function g = sigmoid(z) %SIGMOID Compute sigmoid functoon % J = SIGMOID(z) computes the sigmoid of z. % You need to return the following variables correctly g = zeros(size(z)); % ====================== YOUR CODE HERE ====================== % Instructions: Compute the sigmoid of each value of z (z can be a matrix, % vector or scalar). g = 1./(1+exp(-z)); % ============================================================= end ================================================ FILE: machine-learning-ex2/ex2/submit.m ================================================ function submit() addpath('./lib'); conf.assignmentSlug = 'logistic-regression'; conf.itemName = 'Logistic Regression'; conf.partArrays = { ... { ... '1', ... { 'sigmoid.m' }, ... 'Sigmoid Function', ... }, ... { ... '2', ... { 'costFunction.m' }, ... 'Logistic Regression Cost', ... }, ... { ... '3', ... { 'costFunction.m' }, ... 'Logistic Regression Gradient', ... }, ... { ... '4', ... { 'predict.m' }, ... 'Predict', ... }, ... { ... '5', ... { 'costFunctionReg.m' }, ... 'Regularized Logistic Regression Cost', ... }, ... { ... '6', ... { 'costFunctionReg.m' }, ... 'Regularized Logistic Regression Gradient', ... }, ... }; conf.output = @output; submitWithConfiguration(conf); end function out = output(partId, auxstring) % Random Test Cases X = [ones(20,1) (exp(1) * sin(1:1:20))' (exp(0.5) * cos(1:1:20))']; y = sin(X(:,1) + X(:,2)) > 0; if partId == '1' out = sprintf('%0.5f ', sigmoid(X)); elseif partId == '2' out = sprintf('%0.5f ', costFunction([0.25 0.5 -0.5]', X, y)); elseif partId == '3' [cost, grad] = costFunction([0.25 0.5 -0.5]', X, y); out = sprintf('%0.5f ', grad); elseif partId == '4' out = sprintf('%0.5f ', predict([0.25 0.5 -0.5]', X)); elseif partId == '5' out = sprintf('%0.5f ', costFunctionReg([0.25 0.5 -0.5]', X, y, 0.1)); elseif partId == '6' [cost, grad] = costFunctionReg([0.25 0.5 -0.5]', X, y, 0.1); out = sprintf('%0.5f ', grad); end end ================================================ FILE: machine-learning-ex3/ex3/displayData.m ================================================ function [h, display_array] = displayData(X, example_width) %DISPLAYDATA Display 2D data in a nice grid % [h, display_array] = DISPLAYDATA(X, example_width) displays 2D data % stored in X in a nice grid. It returns the figure handle h and the % displayed array if requested. % Set example_width automatically if not passed in if ~exist('example_width', 'var') || isempty(example_width) example_width = round(sqrt(size(X, 2))); end % Gray Image colormap(gray); % Compute rows, cols [m n] = size(X); example_height = (n / example_width); % Compute number of items to display display_rows = floor(sqrt(m)); display_cols = ceil(m / display_rows); % Between images padding pad = 1; % Setup blank display display_array = - ones(pad + display_rows * (example_height + pad), ... pad + display_cols * (example_width + pad)); % Copy each example into a patch on the display array curr_ex = 1; for j = 1:display_rows for i = 1:display_cols if curr_ex > m, break; end % Copy the patch % Get the max value of the patch max_val = max(abs(X(curr_ex, :))); display_array(pad + (j - 1) * (example_height + pad) + (1:example_height), ... pad + (i - 1) * (example_width + pad) + (1:example_width)) = ... reshape(X(curr_ex, :), example_height, example_width) / max_val; curr_ex = curr_ex + 1; end if curr_ex > m, break; end end % Display Image h = imagesc(display_array, [-1 1]); % Do not show axis axis image off drawnow; end ================================================ FILE: machine-learning-ex3/ex3/ex3.m ================================================ %% Machine Learning Online Class - Exercise 3 | Part 1: One-vs-all % Instructions % ------------ % % This file contains code that helps you get started on the % linear exercise. You will need to complete the following functions % in this exericse: % % lrCostFunction.m (logistic regression cost function) % oneVsAll.m % predictOneVsAll.m % predict.m % % For this exercise, you will not need to change any code in this file, % or any other files other than those mentioned above. % %% Initialization clear ; close all; clc %% Setup the parameters you will use for this part of the exercise input_layer_size = 400; % 20x20 Input Images of Digits num_labels = 10; % 10 labels, from 1 to 10 % (note that we have mapped "0" to label 10) %% =========== Part 1: Loading and Visualizing Data ============= % We start the exercise by first loading and visualizing the dataset. % You will be working with a dataset that contains handwritten digits. % % Load Training Data fprintf('Loading and Visualizing Data ...\n') load('ex3data1.mat'); % training data stored in arrays X, y m = size(X, 1); % Randomly select 100 data points to display rand_indices = randperm(m); sel = X(rand_indices(1:100), :); displayData(sel); fprintf('Program paused. Press enter to continue.\n'); pause; %% ============ Part 2: Vectorize Logistic Regression ============ % In this part of the exercise, you will reuse your logistic regression % code from the last exercise. You task here is to make sure that your % regularized logistic regression implementation is vectorized. After % that, you will implement one-vs-all classification for the handwritten % digit dataset. % fprintf('\nTraining One-vs-All Logistic Regression...\n') lambda = 0.1; [all_theta] = oneVsAll(X, y, num_labels, lambda); fprintf('Program paused. Press enter to continue.\n'); pause; %% ================ Part 3: Predict for One-Vs-All ================ % After ... pred = predictOneVsAll(all_theta, X); fprintf('\nTraining Set Accuracy: %f\n', mean(double(pred == y)) * 100); ================================================ FILE: machine-learning-ex3/ex3/ex3_nn.m ================================================ %% Machine Learning Online Class - Exercise 3 | Part 2: Neural Networks % Instructions % ------------ % % This file contains code that helps you get started on the % linear exercise. You will need to complete the following functions % in this exericse: % % lrCostFunction.m (logistic regression cost function) % oneVsAll.m % predictOneVsAll.m % predict.m % % For this exercise, you will not need to change any code in this file, % or any other files other than those mentioned above. % %% Initialization clear ; close all; clc %% Setup the parameters you will use for this exercise input_layer_size = 400; % 20x20 Input Images of Digits hidden_layer_size = 25; % 25 hidden units num_labels = 10; % 10 labels, from 1 to 10 % (note that we have mapped "0" to label 10) %% =========== Part 1: Loading and Visualizing Data ============= % We start the exercise by first loading and visualizing the dataset. % You will be working with a dataset that contains handwritten digits. % % Load Training Data fprintf('Loading and Visualizing Data ...\n') load('ex3data1.mat'); m = size(X, 1); % Randomly select 100 data points to display sel = randperm(size(X, 1)); sel = sel(1:100); displayData(X(sel, :)); fprintf('Program paused. Press enter to continue.\n'); pause; %% ================ Part 2: Loading Pameters ================ % In this part of the exercise, we load some pre-initialized % neural network parameters. fprintf('\nLoading Saved Neural Network Parameters ...\n') % Load the weights into variables Theta1 and Theta2 load('ex3weights.mat'); %% ================= Part 3: Implement Predict ================= % After training the neural network, we would like to use it to predict % the labels. You will now implement the "predict" function to use the % neural network to predict the labels of the training set. This lets % you compute the training set accuracy. pred = predict(Theta1, Theta2, X); fprintf('\nTraining Set Accuracy: %f\n', mean(double(pred == y)) * 100); fprintf('Program paused. Press enter to continue.\n'); pause; % To give you an idea of the network's output, you can also run % through the examples one at the a time to see what it is predicting. % Randomly permute examples rp = randperm(m); for i = 1:m % Display fprintf('\nDisplaying Example Image\n'); displayData(X(rp(i), :)); pred = predict(Theta1, Theta2, X(rp(i),:)); fprintf('\nNeural Network Prediction: %d (digit %d)\n', pred, mod(pred, 10));%mod࣬Ϊ0ӳ10 % Pause fprintf('Program paused. Press enter to continue.\n'); pause; end ================================================ FILE: machine-learning-ex3/ex3/fmincg.m ================================================ function [X, fX, i] = fmincg(f, X, options, P1, P2, P3, P4, P5) % Minimize a continuous differentialble multivariate function. Starting point % is given by "X" (D by 1), and the function named in the string "f", must % return a function value and a vector of partial derivatives. The Polack- % Ribiere flavour of conjugate gradients is used to compute search directions, % and a line search using quadratic and cubic polynomial approximations and the % Wolfe-Powell stopping criteria is used together with the slope ratio method % for guessing initial step sizes. Additionally a bunch of checks are made to % make sure that exploration is taking place and that extrapolation will not % be unboundedly large. The "length" gives the length of the run: if it is % positive, it gives the maximum number of line searches, if negative its % absolute gives the maximum allowed number of function evaluations. You can % (optionally) give "length" a second component, which will indicate the % reduction in function value to be expected in the first line-search (defaults % to 1.0). The function returns when either its length is up, or if no further % progress can be made (ie, we are at a minimum, or so close that due to % numerical problems, we cannot get any closer). If the function terminates % within a few iterations, it could be an indication that the function value % and derivatives are not consistent (ie, there may be a bug in the % implementation of your "f" function). The function returns the found % solution "X", a vector of function values "fX" indicating the progress made % and "i" the number of iterations (line searches or function evaluations, % depending on the sign of "length") used. % % Usage: [X, fX, i] = fmincg(f, X, options, P1, P2, P3, P4, P5) % % See also: checkgrad % % Copyright (C) 2001 and 2002 by Carl Edward Rasmussen. Date 2002-02-13 % % % (C) Copyright 1999, 2000 & 2001, Carl Edward Rasmussen % % Permission is granted for anyone to copy, use, or modify these % programs and accompanying documents for purposes of research or % education, provided this copyright notice is retained, and note is % made of any changes that have been made. % % These programs and documents are distributed without any warranty, % express or implied. As the programs were written for research % purposes only, they have not been tested to the degree that would be % advisable in any important application. All use of these programs is % entirely at the user's own risk. % % [ml-class] Changes Made: % 1) Function name and argument specifications % 2) Output display % % Read options if exist('options', 'var') && ~isempty(options) && isfield(options, 'MaxIter') length = options.MaxIter; else length = 100; end RHO = 0.01; % a bunch of constants for line searches SIG = 0.5; % RHO and SIG are the constants in the Wolfe-Powell conditions INT = 0.1; % don't reevaluate within 0.1 of the limit of the current bracket EXT = 3.0; % extrapolate maximum 3 times the current bracket MAX = 20; % max 20 function evaluations per line search RATIO = 100; % maximum allowed slope ratio argstr = ['feval(f, X']; % compose string used to call function for i = 1:(nargin - 3) argstr = [argstr, ',P', int2str(i)]; end argstr = [argstr, ')']; if max(size(length)) == 2, red=length(2); length=length(1); else red=1; end S=['Iteration ']; i = 0; % zero the run length counter ls_failed = 0; % no previous line search has failed fX = []; [f1 df1] = eval(argstr); % get function value and gradient i = i + (length<0); % count epochs?! s = -df1; % search direction is steepest d1 = -s'*s; % this is the slope z1 = red/(1-d1); % initial step is red/(|s|+1) while i < abs(length) % while not finished i = i + (length>0); % count iterations?! X0 = X; f0 = f1; df0 = df1; % make a copy of current values X = X + z1*s; % begin line search [f2 df2] = eval(argstr); i = i + (length<0); % count epochs?! d2 = df2'*s; f3 = f1; d3 = d1; z3 = -z1; % initialize point 3 equal to point 1 if length>0, M = MAX; else M = min(MAX, -length-i); end success = 0; limit = -1; % initialize quanteties while 1 while ((f2 > f1+z1*RHO*d1) || (d2 > -SIG*d1)) && (M > 0) limit = z1; % tighten the bracket if f2 > f1 z2 = z3 - (0.5*d3*z3*z3)/(d3*z3+f2-f3); % quadratic fit else A = 6*(f2-f3)/z3+3*(d2+d3); % cubic fit B = 3*(f3-f2)-z3*(d3+2*d2); z2 = (sqrt(B*B-A*d2*z3*z3)-B)/A; % numerical error possible - ok! end if isnan(z2) || isinf(z2) z2 = z3/2; % if we had a numerical problem then bisect end z2 = max(min(z2, INT*z3),(1-INT)*z3); % don't accept too close to limits z1 = z1 + z2; % update the step X = X + z2*s; [f2 df2] = eval(argstr); M = M - 1; i = i + (length<0); % count epochs?! d2 = df2'*s; z3 = z3-z2; % z3 is now relative to the location of z2 end if f2 > f1+z1*RHO*d1 || d2 > -SIG*d1 break; % this is a failure elseif d2 > SIG*d1 success = 1; break; % success elseif M == 0 break; % failure end A = 6*(f2-f3)/z3+3*(d2+d3); % make cubic extrapolation B = 3*(f3-f2)-z3*(d3+2*d2); z2 = -d2*z3*z3/(B+sqrt(B*B-A*d2*z3*z3)); % num. error possible - ok! if ~isreal(z2) || isnan(z2) || isinf(z2) || z2 < 0 % num prob or wrong sign? if limit < -0.5 % if we have no upper limit z2 = z1 * (EXT-1); % the extrapolate the maximum amount else z2 = (limit-z1)/2; % otherwise bisect end elseif (limit > -0.5) && (z2+z1 > limit) % extraplation beyond max? z2 = (limit-z1)/2; % bisect elseif (limit < -0.5) && (z2+z1 > z1*EXT) % extrapolation beyond limit z2 = z1*(EXT-1.0); % set to extrapolation limit elseif z2 < -z3*INT z2 = -z3*INT; elseif (limit > -0.5) && (z2 < (limit-z1)*(1.0-INT)) % too close to limit? z2 = (limit-z1)*(1.0-INT); end f3 = f2; d3 = d2; z3 = -z2; % set point 3 equal to point 2 z1 = z1 + z2; X = X + z2*s; % update current estimates [f2 df2] = eval(argstr); M = M - 1; i = i + (length<0); % count epochs?! d2 = df2'*s; end % end of line search if success % if line search succeeded f1 = f2; fX = [fX' f1]'; fprintf('%s %4i | Cost: %4.6e\r', S, i, f1); s = (df2'*df2-df1'*df2)/(df1'*df1)*s - df2; % Polack-Ribiere direction tmp = df1; df1 = df2; df2 = tmp; % swap derivatives d2 = df1'*s; if d2 > 0 % new slope must be negative s = -df1; % otherwise use steepest direction d2 = -s'*s; end z1 = z1 * min(RATIO, d1/(d2-realmin)); % slope ratio but max RATIO d1 = d2; ls_failed = 0; % this line search did not fail else X = X0; f1 = f0; df1 = df0; % restore point from before failed line search if ls_failed || i > abs(length) % line search failed twice in a row break; % or we ran out of time, so we give up end tmp = df1; df1 = df2; df2 = tmp; % swap derivatives s = -df1; % try steepest d1 = -s'*s; z1 = 1/(1-d1); ls_failed = 1; % this line search failed end if exist('OCTAVE_VERSION') fflush(stdout); end end fprintf('\n'); ================================================ FILE: machine-learning-ex3/ex3/lib/jsonlab/AUTHORS.txt ================================================ The author of "jsonlab" toolbox is Qianqian Fang. Qianqian is currently an Assistant Professor at Massachusetts General Hospital, Harvard Medical School. Address: Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School Bldg 149, 13th St, Charlestown, MA 02129, USA URL: http://nmr.mgh.harvard.edu/~fangq/ Email: or The script loadjson.m was built upon previous works by - Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713 date: 2009/11/02 - François Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393 date: 2009/03/22 - Joel Feenstra: http://www.mathworks.com/matlabcentral/fileexchange/20565 date: 2008/07/03 This toolbox contains patches submitted by the following contributors: - Blake Johnson part of revision 341 - Niclas Borlin various fixes in revision 394, including - loadjson crashes for all-zero sparse matrix. - loadjson crashes for empty sparse matrix. - Non-zero size of 0-by-N and N-by-0 empty matrices is lost after savejson/loadjson. - loadjson crashes for sparse real column vector. - loadjson crashes for sparse complex column vector. - Data is corrupted by savejson for sparse real row vector. - savejson crashes for sparse complex row vector. - Yul Kang patches for svn revision 415. - savejson saves an empty cell array as [] instead of null - loadjson differentiates an empty struct from an empty array ================================================ FILE: machine-learning-ex3/ex3/lib/jsonlab/ChangeLog.txt ================================================ ============================================================================ JSONlab - a toolbox to encode/decode JSON/UBJSON files in MATLAB/Octave ---------------------------------------------------------------------------- JSONlab ChangeLog (key features marked by *): == JSONlab 1.0 (codename: Optimus - Final), FangQ == 2015/01/02 polish help info for all major functions, update examples, finalize 1.0 2014/12/19 fix a bug to strictly respect NoRowBracket in savejson == JSONlab 1.0.0-RC2 (codename: Optimus - RC2), FangQ == 2014/11/22 show progress bar in loadjson ('ShowProgress') 2014/11/17 add Compact option in savejson to output compact JSON format ('Compact') 2014/11/17 add FastArrayParser in loadjson to specify fast parser applicable levels 2014/09/18 start official github mirror: https://github.com/fangq/jsonlab == JSONlab 1.0.0-RC1 (codename: Optimus - RC1), FangQ == 2014/09/17 fix several compatibility issues when running on octave versions 3.2-3.8 2014/09/17 support 2D cell and struct arrays in both savejson and saveubjson 2014/08/04 escape special characters in a JSON string 2014/02/16 fix a bug when saving ubjson files == JSONlab 0.9.9 (codename: Optimus - beta), FangQ == 2014/01/22 use binary read and write in saveubjson and loadubjson == JSONlab 0.9.8-1 (codename: Optimus - alpha update 1), FangQ == 2013/10/07 better round-trip conservation for empty arrays and structs (patch submitted by Yul Kang) == JSONlab 0.9.8 (codename: Optimus - alpha), FangQ == 2013/08/23 *universal Binary JSON (UBJSON) support, including both saveubjson and loadubjson == JSONlab 0.9.1 (codename: Rodimus, update 1), FangQ == 2012/12/18 *handling of various empty and sparse matrices (fixes submitted by Niclas Borlin) == JSONlab 0.9.0 (codename: Rodimus), FangQ == 2012/06/17 *new format for an invalid leading char, unpacking hex code in savejson 2012/06/01 support JSONP in savejson 2012/05/25 fix the empty cell bug (reported by Cyril Davin) 2012/04/05 savejson can save to a file (suggested by Patrick Rapin) == JSONlab 0.8.1 (codename: Sentiel, Update 1), FangQ == 2012/02/28 loadjson quotation mark escape bug, see http://bit.ly/yyk1nS 2012/01/25 patch to handle root-less objects, contributed by Blake Johnson == JSONlab 0.8.0 (codename: Sentiel), FangQ == 2012/01/13 *speed up loadjson by 20 fold when parsing large data arrays in matlab 2012/01/11 remove row bracket if an array has 1 element, suggested by Mykel Kochenderfer 2011/12/22 *accept sequence of 'param',value input in savejson and loadjson 2011/11/18 fix struct array bug reported by Mykel Kochenderfer == JSONlab 0.5.1 (codename: Nexus Update 1), FangQ == 2011/10/21 fix a bug in loadjson, previous code does not use any of the acceleration 2011/10/20 loadjson supports JSON collections - concatenated JSON objects == JSONlab 0.5.0 (codename: Nexus), FangQ == 2011/10/16 package and release jsonlab 0.5.0 2011/10/15 *add json demo and regression test, support cpx numbers, fix double quote bug 2011/10/11 *speed up readjson dramatically, interpret _Array* tags, show data in root level 2011/10/10 create jsonlab project, start jsonlab website, add online documentation 2011/10/07 *speed up savejson by 25x using sprintf instead of mat2str, add options support 2011/10/06 *savejson works for structs, cells and arrays 2011/09/09 derive loadjson from JSON parser from MATLAB Central, draft savejson.m ================================================ FILE: machine-learning-ex3/ex3/lib/jsonlab/LICENSE_BSD.txt ================================================ Copyright 2011-2015 Qianqian Fang . All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. The views and conclusions contained in the software and documentation are those of the authors and should not be interpreted as representing official policies, either expressed or implied, of the copyright holders. ================================================ FILE: machine-learning-ex3/ex3/lib/jsonlab/README.txt ================================================ =============================================================================== = JSONLab = = An open-source MATLAB/Octave JSON encoder and decoder = =============================================================================== *Copyright (C) 2011-2015 Qianqian Fang *License: BSD License, see License_BSD.txt for details *Version: 1.0 (Optimus - Final) ------------------------------------------------------------------------------- Table of Content: I. Introduction II. Installation III.Using JSONLab IV. Known Issues and TODOs V. Contribution and feedback ------------------------------------------------------------------------------- I. Introduction JSON ([http://www.json.org/ JavaScript Object Notation]) is a highly portable, human-readable and "[http://en.wikipedia.org/wiki/JSON fat-free]" text format to represent complex and hierarchical data. It is as powerful as [http://en.wikipedia.org/wiki/XML XML], but less verbose. JSON format is widely used for data-exchange in applications, and is essential for the wild success of [http://en.wikipedia.org/wiki/Ajax_(programming) Ajax] and [http://en.wikipedia.org/wiki/Web_2.0 Web2.0]. UBJSON (Universal Binary JSON) is a binary JSON format, specifically optimized for compact file size and better performance while keeping the semantics as simple as the text-based JSON format. Using the UBJSON format allows to wrap complex binary data in a flexible and extensible structure, making it possible to process complex and large dataset without accuracy loss due to text conversions. We envision that both JSON and its binary version will serve as part of the mainstream data-exchange formats for scientific research in the future. It will provide the flexibility and generality achieved by other popular general-purpose file specifications, such as [http://www.hdfgroup.org/HDF5/whatishdf5.html HDF5], with significantly reduced complexity and enhanced performance. JSONLab is a free and open-source implementation of a JSON/UBJSON encoder and a decoder in the native MATLAB language. It can be used to convert a MATLAB data structure (array, struct, cell, struct array and cell array) into JSON/UBJSON formatted strings, or to decode a JSON/UBJSON file into MATLAB data structure. JSONLab supports both MATLAB and [http://www.gnu.org/software/octave/ GNU Octave] (a free MATLAB clone). ------------------------------------------------------------------------------- II. Installation The installation of JSONLab is no different than any other simple MATLAB toolbox. You only need to download/unzip the JSONLab package to a folder, and add the folder's path to MATLAB/Octave's path list by using the following command: addpath('/path/to/jsonlab'); If you want to add this path permanently, you need to type "pathtool", browse to the jsonlab root folder and add to the list, then click "Save". Then, run "rehash" in MATLAB, and type "which loadjson", if you see an output, that means JSONLab is installed for MATLAB/Octave. ------------------------------------------------------------------------------- III.Using JSONLab JSONLab provides two functions, loadjson.m -- a MATLAB->JSON decoder, and savejson.m -- a MATLAB->JSON encoder, for the text-based JSON, and two equivallent functions -- loadubjson and saveubjson for the binary JSON. The detailed help info for the four functions can be found below: === loadjson.m === 
  data=loadjson(fname,opt)
     or
  data=loadjson(fname,'param1',value1,'param2',value2,...)
 
  parse a JSON (JavaScript Object Notation) file or string
 
  authors:Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2011/09/09, including previous works from 
 
          Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713
             created on 2009/11/02
          Franois Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393
             created on  2009/03/22
          Joel Feenstra:
          http://www.mathworks.com/matlabcentral/fileexchange/20565
             created on 2008/07/03
 
  $Id: loadjson.m 452 2014-11-22 16:43:33Z fangq $
 
  input:
       fname: input file name, if fname contains "{}" or "[]", fname
              will be interpreted as a JSON string
       opt: a struct to store parsing options, opt can be replaced by 
            a list of ('param',value) pairs - the param string is equivallent
            to a field in opt. opt can have the following 
            fields (first in [.|.] is the default)
 
            opt.SimplifyCell [0|1]: if set to 1, loadjson will call cell2mat
                          for each element of the JSON data, and group 
                          arrays based on the cell2mat rules.
            opt.FastArrayParser [1|0 or integer]: if set to 1, use a
                          speed-optimized array parser when loading an 
                          array object. The fast array parser may 
                          collapse block arrays into a single large
                          array similar to rules defined in cell2mat; 0 to 
                          use a legacy parser; if set to a larger-than-1
                          value, this option will specify the minimum
                          dimension to enable the fast array parser. For
                          example, if the input is a 3D array, setting
                          FastArrayParser to 1 will return a 3D array;
                          setting to 2 will return a cell array of 2D
                          arrays; setting to 3 will return to a 2D cell
                          array of 1D vectors; setting to 4 will return a
                          3D cell array.
            opt.ShowProgress [0|1]: if set to 1, loadjson displays a progress bar.
 
  output:
       dat: a cell array, where {...} blocks are converted into cell arrays,
            and [...] are converted to arrays
 
  examples:
       dat=loadjson('{"obj":{"string":"value","array":[1,2,3]}}')
       dat=loadjson(['examples' filesep 'example1.json'])
       dat=loadjson(['examples' filesep 'example1.json'],'SimplifyCell',1)
=== savejson.m === 
  json=savejson(rootname,obj,filename)
     or
  json=savejson(rootname,obj,opt)
  json=savejson(rootname,obj,'param1',value1,'param2',value2,...)
 
  convert a MATLAB object (cell, struct or array) into a JSON (JavaScript
  Object Notation) string
 
  author: Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2011/09/09
 
  $Id: savejson.m 458 2014-12-19 22:17:17Z fangq $
 
  input:
       rootname: the name of the root-object, when set to '', the root name
         is ignored, however, when opt.ForceRootName is set to 1 (see below),
         the MATLAB variable name will be used as the root name.
       obj: a MATLAB object (array, cell, cell array, struct, struct array).
       filename: a string for the file name to save the output JSON data.
       opt: a struct for additional options, ignore to use default values.
         opt can have the following fields (first in [.|.] is the default)
 
         opt.FileName [''|string]: a file name to save the output JSON data
         opt.FloatFormat ['%.10g'|string]: format to show each numeric element
                          of a 1D/2D array;
         opt.ArrayIndent [1|0]: if 1, output explicit data array with
                          precedent indentation; if 0, no indentation
         opt.ArrayToStruct[0|1]: when set to 0, savejson outputs 1D/2D
                          array in JSON array format; if sets to 1, an
                          array will be shown as a struct with fields
                          "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for
                          sparse arrays, the non-zero elements will be
                          saved to _ArrayData_ field in triplet-format i.e.
                          (ix,iy,val) and "_ArrayIsSparse_" will be added
                          with a value of 1; for a complex array, the 
                          _ArrayData_ array will include two columns 
                          (4 for sparse) to record the real and imaginary 
                          parts, and also "_ArrayIsComplex_":1 is added. 
         opt.ParseLogical [0|1]: if this is set to 1, logical array elem
                          will use true/false rather than 1/0.
         opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single
                          numerical element will be shown without a square
                          bracket, unless it is the root object; if 0, square
                          brackets are forced for any numerical arrays.
         opt.ForceRootName [0|1]: when set to 1 and rootname is empty, savejson
                          will use the name of the passed obj variable as the 
                          root object name; if obj is an expression and 
                          does not have a name, 'root' will be used; if this 
                          is set to 0 and rootname is empty, the root level 
                          will be merged down to the lower level.
         opt.Inf ['"$1_Inf_"'|string]: a customized regular expression pattern
                          to represent +/-Inf. The matched pattern is '([-+]*)Inf'
                          and $1 represents the sign. For those who want to use
                          1e999 to represent Inf, they can set opt.Inf to '$11e999'
         opt.NaN ['"_NaN_"'|string]: a customized regular expression pattern
                          to represent NaN
         opt.JSONP [''|string]: to generate a JSONP output (JSON with padding),
                          for example, if opt.JSONP='foo', the JSON data is
                          wrapped inside a function call as 'foo(...);'
         opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson 
                          back to the string form
         opt.SaveBinary [0|1]: 1 - save the JSON file in binary mode; 0 - text mode.
         opt.Compact [0|1]: 1- out compact JSON format (remove all newlines and tabs)
 
         opt can be replaced by a list of ('param',value) pairs. The param 
         string is equivallent to a field in opt and is case sensitive.
  output:
       json: a string in the JSON format (see http://json.org)
 
  examples:
       jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... 
                'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],...
                'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;...
                           2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],...
                'MeshCreator','FangQ','MeshTitle','T6 Cube',...
                'SpecialData',[nan, inf, -inf]);
       savejson('jmesh',jsonmesh)
       savejson('',jsonmesh,'ArrayIndent',0,'FloatFormat','\t%.5g')
=== loadubjson.m === 
  data=loadubjson(fname,opt)
     or
  data=loadubjson(fname,'param1',value1,'param2',value2,...)
 
  parse a JSON (JavaScript Object Notation) file or string
 
  authors:Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2013/08/01
 
  $Id: loadubjson.m 436 2014-08-05 20:51:40Z fangq $
 
  input:
       fname: input file name, if fname contains "{}" or "[]", fname
              will be interpreted as a UBJSON string
       opt: a struct to store parsing options, opt can be replaced by 
            a list of ('param',value) pairs - the param string is equivallent
            to a field in opt. opt can have the following 
            fields (first in [.|.] is the default)
 
            opt.SimplifyCell [0|1]: if set to 1, loadubjson will call cell2mat
                          for each element of the JSON data, and group 
                          arrays based on the cell2mat rules.
            opt.IntEndian [B|L]: specify the endianness of the integer fields
                          in the UBJSON input data. B - Big-Endian format for 
                          integers (as required in the UBJSON specification); 
                          L - input integer fields are in Little-Endian order.
 
  output:
       dat: a cell array, where {...} blocks are converted into cell arrays,
            and [...] are converted to arrays
 
  examples:
       obj=struct('string','value','array',[1 2 3]);
       ubjdata=saveubjson('obj',obj);
       dat=loadubjson(ubjdata)
       dat=loadubjson(['examples' filesep 'example1.ubj'])
       dat=loadubjson(['examples' filesep 'example1.ubj'],'SimplifyCell',1)
=== saveubjson.m === 
  json=saveubjson(rootname,obj,filename)
     or
  json=saveubjson(rootname,obj,opt)
  json=saveubjson(rootname,obj,'param1',value1,'param2',value2,...)
 
  convert a MATLAB object (cell, struct or array) into a Universal 
  Binary JSON (UBJSON) binary string
 
  author: Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2013/08/17
 
  $Id: saveubjson.m 440 2014-09-17 19:59:45Z fangq $
 
  input:
       rootname: the name of the root-object, when set to '', the root name
         is ignored, however, when opt.ForceRootName is set to 1 (see below),
         the MATLAB variable name will be used as the root name.
       obj: a MATLAB object (array, cell, cell array, struct, struct array)
       filename: a string for the file name to save the output UBJSON data
       opt: a struct for additional options, ignore to use default values.
         opt can have the following fields (first in [.|.] is the default)
 
         opt.FileName [''|string]: a file name to save the output JSON data
         opt.ArrayToStruct[0|1]: when set to 0, saveubjson outputs 1D/2D
                          array in JSON array format; if sets to 1, an
                          array will be shown as a struct with fields
                          "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for
                          sparse arrays, the non-zero elements will be
                          saved to _ArrayData_ field in triplet-format i.e.
                          (ix,iy,val) and "_ArrayIsSparse_" will be added
                          with a value of 1; for a complex array, the 
                          _ArrayData_ array will include two columns 
                          (4 for sparse) to record the real and imaginary 
                          parts, and also "_ArrayIsComplex_":1 is added. 
         opt.ParseLogical [1|0]: if this is set to 1, logical array elem
                          will use true/false rather than 1/0.
         opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single
                          numerical element will be shown without a square
                          bracket, unless it is the root object; if 0, square
                          brackets are forced for any numerical arrays.
         opt.ForceRootName [0|1]: when set to 1 and rootname is empty, saveubjson
                          will use the name of the passed obj variable as the 
                          root object name; if obj is an expression and 
                          does not have a name, 'root' will be used; if this 
                          is set to 0 and rootname is empty, the root level 
                          will be merged down to the lower level.
         opt.JSONP [''|string]: to generate a JSONP output (JSON with padding),
                          for example, if opt.JSON='foo', the JSON data is
                          wrapped inside a function call as 'foo(...);'
         opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson 
                          back to the string form
 
         opt can be replaced by a list of ('param',value) pairs. The param 
         string is equivallent to a field in opt and is case sensitive.
  output:
       json: a binary string in the UBJSON format (see http://ubjson.org)
 
  examples:
       jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... 
                'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],...
                'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;...
                           2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],...
                'MeshCreator','FangQ','MeshTitle','T6 Cube',...
                'SpecialData',[nan, inf, -inf]);
       saveubjson('jsonmesh',jsonmesh)
       saveubjson('jsonmesh',jsonmesh,'meshdata.ubj')
=== examples === Under the "examples" folder, you can find several scripts to demonstrate the basic utilities of JSONLab. Running the "demo_jsonlab_basic.m" script, you will see the conversions from MATLAB data structure to JSON text and backward. In "jsonlab_selftest.m", we load complex JSON files downloaded from the Internet and validate the loadjson/savejson functions for regression testing purposes. Similarly, a "demo_ubjson_basic.m" script is provided to test the saveubjson and loadubjson pairs for various matlab data structures. Please run these examples and understand how JSONLab works before you use it to process your data. ------------------------------------------------------------------------------- IV. Known Issues and TODOs JSONLab has several known limitations. We are striving to make it more general and robust. Hopefully in a few future releases, the limitations become less. Here are the known issues: # 3D or higher dimensional cell/struct-arrays will be converted to 2D arrays; # When processing names containing multi-byte characters, Octave and MATLAB \ can give different field-names; you can use feature('DefaultCharacterSet','latin1') \ in MATLAB to get consistant results # savejson can not handle class and dataset. # saveubjson converts a logical array into a uint8 ([U]) array # an unofficial N-D array count syntax is implemented in saveubjson. We are \ actively communicating with the UBJSON spec maintainer to investigate the \ possibility of making it upstream # loadubjson can not parse all UBJSON Specification (Draft 9) compliant \ files, however, it can parse all UBJSON files produced by saveubjson. ------------------------------------------------------------------------------- V. Contribution and feedback JSONLab is an open-source project. This means you can not only use it and modify it as you wish, but also you can contribute your changes back to JSONLab so that everyone else can enjoy the improvement. For anyone who want to contribute, please download JSONLab source code from it's subversion repository by using the following command: svn checkout svn://svn.code.sf.net/p/iso2mesh/code/trunk/jsonlab jsonlab You can make changes to the files as needed. Once you are satisfied with your changes, and ready to share it with others, please cd the root directory of JSONLab, and type svn diff > yourname_featurename.patch You then email the .patch file to JSONLab's maintainer, Qianqian Fang, at the email address shown in the beginning of this file. Qianqian will review the changes and commit it to the subversion if they are satisfactory. We appreciate any suggestions and feedbacks from you. Please use iso2mesh's mailing list to report any questions you may have with JSONLab: http://groups.google.com/group/iso2mesh-users?hl=en&pli=1 (Subscription to the mailing list is needed in order to post messages). ================================================ FILE: machine-learning-ex3/ex3/lib/jsonlab/jsonopt.m ================================================ function val=jsonopt(key,default,varargin) % % val=jsonopt(key,default,optstruct) % % setting options based on a struct. The struct can be produced % by varargin2struct from a list of 'param','value' pairs % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % % $Id: loadjson.m 371 2012-06-20 12:43:06Z fangq $ % % input: % key: a string with which one look up a value from a struct % default: if the key does not exist, return default % optstruct: a struct where each sub-field is a key % % output: % val: if key exists, val=optstruct.key; otherwise val=default % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % val=default; if(nargin<=2) return; end opt=varargin{1}; if(isstruct(opt) && isfield(opt,key)) val=getfield(opt,key); end ================================================ FILE: machine-learning-ex3/ex3/lib/jsonlab/loadjson.m ================================================ function data = loadjson(fname,varargin) % % data=loadjson(fname,opt) % or % data=loadjson(fname,'param1',value1,'param2',value2,...) % % parse a JSON (JavaScript Object Notation) file or string % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2011/09/09, including previous works from % % Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713 % created on 2009/11/02 % François Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393 % created on 2009/03/22 % Joel Feenstra: % http://www.mathworks.com/matlabcentral/fileexchange/20565 % created on 2008/07/03 % % $Id: loadjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % fname: input file name, if fname contains "{}" or "[]", fname % will be interpreted as a JSON string % opt: a struct to store parsing options, opt can be replaced by % a list of ('param',value) pairs - the param string is equivallent % to a field in opt. opt can have the following % fields (first in [.|.] is the default) % % opt.SimplifyCell [0|1]: if set to 1, loadjson will call cell2mat % for each element of the JSON data, and group % arrays based on the cell2mat rules. % opt.FastArrayParser [1|0 or integer]: if set to 1, use a % speed-optimized array parser when loading an % array object. The fast array parser may % collapse block arrays into a single large % array similar to rules defined in cell2mat; 0 to % use a legacy parser; if set to a larger-than-1 % value, this option will specify the minimum % dimension to enable the fast array parser. For % example, if the input is a 3D array, setting % FastArrayParser to 1 will return a 3D array; % setting to 2 will return a cell array of 2D % arrays; setting to 3 will return to a 2D cell % array of 1D vectors; setting to 4 will return a % 3D cell array. % opt.ShowProgress [0|1]: if set to 1, loadjson displays a progress bar. % % output: % dat: a cell array, where {...} blocks are converted into cell arrays, % and [...] are converted to arrays % % examples: % dat=loadjson('{"obj":{"string":"value","array":[1,2,3]}}') % dat=loadjson(['examples' filesep 'example1.json']) % dat=loadjson(['examples' filesep 'example1.json'],'SimplifyCell',1) % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % global pos inStr len esc index_esc len_esc isoct arraytoken if(regexp(fname,'[\{\}\]\[]','once')) string=fname; elseif(exist(fname,'file')) fid = fopen(fname,'rb'); string = fread(fid,inf,'uint8=>char')'; fclose(fid); else error('input file does not exist'); end pos = 1; len = length(string); inStr = string; isoct=exist('OCTAVE_VERSION','builtin'); arraytoken=find(inStr=='[' | inStr==']' | inStr=='"'); jstr=regexprep(inStr,'\\\\',' '); escquote=regexp(jstr,'\\"'); arraytoken=sort([arraytoken escquote]); % String delimiters and escape chars identified to improve speed: esc = find(inStr=='"' | inStr=='\' ); % comparable to: regexp(inStr, '["\\]'); index_esc = 1; len_esc = length(esc); opt=varargin2struct(varargin{:}); if(jsonopt('ShowProgress',0,opt)==1) opt.progressbar_=waitbar(0,'loading ...'); end jsoncount=1; while pos <= len switch(next_char) case '{' data{jsoncount} = parse_object(opt); case '[' data{jsoncount} = parse_array(opt); otherwise error_pos('Outer level structure must be an object or an array'); end jsoncount=jsoncount+1; end % while jsoncount=length(data); if(jsoncount==1 && iscell(data)) data=data{1}; end if(~isempty(data)) if(isstruct(data)) % data can be a struct array data=jstruct2array(data); elseif(iscell(data)) data=jcell2array(data); end end if(isfield(opt,'progressbar_')) close(opt.progressbar_); end %% function newdata=jcell2array(data) len=length(data); newdata=data; for i=1:len if(isstruct(data{i})) newdata{i}=jstruct2array(data{i}); elseif(iscell(data{i})) newdata{i}=jcell2array(data{i}); end end %%------------------------------------------------------------------------- function newdata=jstruct2array(data) fn=fieldnames(data); newdata=data; len=length(data); for i=1:length(fn) % depth-first for j=1:len if(isstruct(getfield(data(j),fn{i}))) newdata(j)=setfield(newdata(j),fn{i},jstruct2array(getfield(data(j),fn{i}))); end end end if(~isempty(strmatch('x0x5F_ArrayType_',fn)) && ~isempty(strmatch('x0x5F_ArrayData_',fn))) newdata=cell(len,1); for j=1:len ndata=cast(data(j).x0x5F_ArrayData_,data(j).x0x5F_ArrayType_); iscpx=0; if(~isempty(strmatch('x0x5F_ArrayIsComplex_',fn))) if(data(j).x0x5F_ArrayIsComplex_) iscpx=1; end end if(~isempty(strmatch('x0x5F_ArrayIsSparse_',fn))) if(data(j).x0x5F_ArrayIsSparse_) if(~isempty(strmatch('x0x5F_ArraySize_',fn))) dim=data(j).x0x5F_ArraySize_; if(iscpx && size(ndata,2)==4-any(dim==1)) ndata(:,end-1)=complex(ndata(:,end-1),ndata(:,end)); end if isempty(ndata) % All-zeros sparse ndata=sparse(dim(1),prod(dim(2:end))); elseif dim(1)==1 % Sparse row vector ndata=sparse(1,ndata(:,1),ndata(:,2),dim(1),prod(dim(2:end))); elseif dim(2)==1 % Sparse column vector ndata=sparse(ndata(:,1),1,ndata(:,2),dim(1),prod(dim(2:end))); else % Generic sparse array. ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3),dim(1),prod(dim(2:end))); end else if(iscpx && size(ndata,2)==4) ndata(:,3)=complex(ndata(:,3),ndata(:,4)); end ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3)); end end elseif(~isempty(strmatch('x0x5F_ArraySize_',fn))) if(iscpx && size(ndata,2)==2) ndata=complex(ndata(:,1),ndata(:,2)); end ndata=reshape(ndata(:),data(j).x0x5F_ArraySize_); end newdata{j}=ndata; end if(len==1) newdata=newdata{1}; end end %%------------------------------------------------------------------------- function object = parse_object(varargin) parse_char('{'); object = []; if next_char ~= '}' while 1 str = parseStr(varargin{:}); if isempty(str) error_pos('Name of value at position %d cannot be empty'); end parse_char(':'); val = parse_value(varargin{:}); eval( sprintf( 'object.%s = val;', valid_field(str) ) ); if next_char == '}' break; end parse_char(','); end end parse_char('}'); %%------------------------------------------------------------------------- function object = parse_array(varargin) % JSON array is written in row-major order global pos inStr isoct parse_char('['); object = cell(0, 1); dim2=[]; arraydepth=jsonopt('JSONLAB_ArrayDepth_',1,varargin{:}); pbar=jsonopt('progressbar_',-1,varargin{:}); if next_char ~= ']' if(jsonopt('FastArrayParser',1,varargin{:})>=1 && arraydepth>=jsonopt('FastArrayParser',1,varargin{:})) [endpos, e1l, e1r, maxlevel]=matching_bracket(inStr,pos); arraystr=['[' inStr(pos:endpos)]; arraystr=regexprep(arraystr,'"_NaN_"','NaN'); arraystr=regexprep(arraystr,'"([-+]*)_Inf_"','$1Inf'); arraystr(arraystr==sprintf('\n'))=[]; arraystr(arraystr==sprintf('\r'))=[]; %arraystr=regexprep(arraystr,'\s*,',','); % this is slow,sometimes needed if(~isempty(e1l) && ~isempty(e1r)) % the array is in 2D or higher D astr=inStr((e1l+1):(e1r-1)); astr=regexprep(astr,'"_NaN_"','NaN'); astr=regexprep(astr,'"([-+]*)_Inf_"','$1Inf'); astr(astr==sprintf('\n'))=[]; astr(astr==sprintf('\r'))=[]; astr(astr==' ')=''; if(isempty(find(astr=='[', 1))) % array is 2D dim2=length(sscanf(astr,'%f,',[1 inf])); end else % array is 1D astr=arraystr(2:end-1); astr(astr==' ')=''; [obj, count, errmsg, nextidx]=sscanf(astr,'%f,',[1,inf]); if(nextidx>=length(astr)-1) object=obj; pos=endpos; parse_char(']'); return; end end if(~isempty(dim2)) astr=arraystr; astr(astr=='[')=''; astr(astr==']')=''; astr(astr==' ')=''; [obj, count, errmsg, nextidx]=sscanf(astr,'%f,',inf); if(nextidx>=length(astr)-1) object=reshape(obj,dim2,numel(obj)/dim2)'; pos=endpos; parse_char(']'); if(pbar>0) waitbar(pos/length(inStr),pbar,'loading ...'); end return; end end arraystr=regexprep(arraystr,'\]\s*,','];'); else arraystr='['; end try if(isoct && regexp(arraystr,'"','once')) error('Octave eval can produce empty cells for JSON-like input'); end object=eval(arraystr); pos=endpos; catch while 1 newopt=varargin2struct(varargin{:},'JSONLAB_ArrayDepth_',arraydepth+1); val = parse_value(newopt); object{end+1} = val; if next_char == ']' break; end parse_char(','); end end end if(jsonopt('SimplifyCell',0,varargin{:})==1) try oldobj=object; object=cell2mat(object')'; if(iscell(oldobj) && isstruct(object) && numel(object)>1 && jsonopt('SimplifyCellArray',1,varargin{:})==0) object=oldobj; elseif(size(object,1)>1 && ndims(object)==2) object=object'; end catch end end parse_char(']'); if(pbar>0) waitbar(pos/length(inStr),pbar,'loading ...'); end %%------------------------------------------------------------------------- function parse_char(c) global pos inStr len skip_whitespace; if pos > len || inStr(pos) ~= c error_pos(sprintf('Expected %c at position %%d', c)); else pos = pos + 1; skip_whitespace; end %%------------------------------------------------------------------------- function c = next_char global pos inStr len skip_whitespace; if pos > len c = []; else c = inStr(pos); end %%------------------------------------------------------------------------- function skip_whitespace global pos inStr len while pos <= len && isspace(inStr(pos)) pos = pos + 1; end %%------------------------------------------------------------------------- function str = parseStr(varargin) global pos inStr len esc index_esc len_esc % len, ns = length(inStr), keyboard if inStr(pos) ~= '"' error_pos('String starting with " expected at position %d'); else pos = pos + 1; end str = ''; while pos <= len while index_esc <= len_esc && esc(index_esc) < pos index_esc = index_esc + 1; end if index_esc > len_esc str = [str inStr(pos:len)]; pos = len + 1; break; else str = [str inStr(pos:esc(index_esc)-1)]; pos = esc(index_esc); end nstr = length(str); switch inStr(pos) case '"' pos = pos + 1; if(~isempty(str)) if(strcmp(str,'_Inf_')) str=Inf; elseif(strcmp(str,'-_Inf_')) str=-Inf; elseif(strcmp(str,'_NaN_')) str=NaN; end end return; case '\' if pos+1 > len error_pos('End of file reached right after escape character'); end pos = pos + 1; switch inStr(pos) case {'"' '\' '/'} str(nstr+1) = inStr(pos); pos = pos + 1; case {'b' 'f' 'n' 'r' 't'} str(nstr+1) = sprintf(['\' inStr(pos)]); pos = pos + 1; case 'u' if pos+4 > len error_pos('End of file reached in escaped unicode character'); end str(nstr+(1:6)) = inStr(pos-1:pos+4); pos = pos + 5; end otherwise % should never happen str(nstr+1) = inStr(pos), keyboard pos = pos + 1; end end error_pos('End of file while expecting end of inStr'); %%------------------------------------------------------------------------- function num = parse_number(varargin) global pos inStr len isoct currstr=inStr(pos:end); numstr=0; if(isoct~=0) numstr=regexp(currstr,'^\s*-?(?:0|[1-9]\d*)(?:\.\d+)?(?:[eE][+\-]?\d+)?','end'); [num, one] = sscanf(currstr, '%f', 1); delta=numstr+1; else [num, one, err, delta] = sscanf(currstr, '%f', 1); if ~isempty(err) error_pos('Error reading number at position %d'); end end pos = pos + delta-1; %%------------------------------------------------------------------------- function val = parse_value(varargin) global pos inStr len true = 1; false = 0; pbar=jsonopt('progressbar_',-1,varargin{:}); if(pbar>0) waitbar(pos/len,pbar,'loading ...'); end switch(inStr(pos)) case '"' val = parseStr(varargin{:}); return; case '[' val = parse_array(varargin{:}); return; case '{' val = parse_object(varargin{:}); if isstruct(val) if(~isempty(strmatch('x0x5F_ArrayType_',fieldnames(val), 'exact'))) val=jstruct2array(val); end elseif isempty(val) val = struct; end return; case {'-','0','1','2','3','4','5','6','7','8','9'} val = parse_number(varargin{:}); return; case 't' if pos+3 <= len && strcmpi(inStr(pos:pos+3), 'true') val = true; pos = pos + 4; return; end case 'f' if pos+4 <= len && strcmpi(inStr(pos:pos+4), 'false') val = false; pos = pos + 5; return; end case 'n' if pos+3 <= len && strcmpi(inStr(pos:pos+3), 'null') val = []; pos = pos + 4; return; end end error_pos('Value expected at position %d'); %%------------------------------------------------------------------------- function error_pos(msg) global pos inStr len poShow = max(min([pos-15 pos-1 pos pos+20],len),1); if poShow(3) == poShow(2) poShow(3:4) = poShow(2)+[0 -1]; % display nothing after end msg = [sprintf(msg, pos) ': ' ... inStr(poShow(1):poShow(2)) '' inStr(poShow(3):poShow(4)) ]; error( ['JSONparser:invalidFormat: ' msg] ); %%------------------------------------------------------------------------- function str = valid_field(str) global isoct % From MATLAB doc: field names must begin with a letter, which may be % followed by any combination of letters, digits, and underscores. % Invalid characters will be converted to underscores, and the prefix % "x0x[Hex code]_" will be added if the first character is not a letter. pos=regexp(str,'^[^A-Za-z]','once'); if(~isempty(pos)) if(~isoct) str=regexprep(str,'^([^A-Za-z])','x0x${sprintf(''%X'',unicode2native($1))}_','once'); else str=sprintf('x0x%X_%s',char(str(1)),str(2:end)); end end if(isempty(regexp(str,'[^0-9A-Za-z_]', 'once' ))) return; end if(~isoct) str=regexprep(str,'([^0-9A-Za-z_])','_0x${sprintf(''%X'',unicode2native($1))}_'); else pos=regexp(str,'[^0-9A-Za-z_]'); if(isempty(pos)) return; end str0=str; pos0=[0 pos(:)' length(str)]; str=''; for i=1:length(pos) str=[str str0(pos0(i)+1:pos(i)-1) sprintf('_0x%X_',str0(pos(i)))]; end if(pos(end)~=length(str)) str=[str str0(pos0(end-1)+1:pos0(end))]; end end %str(~isletter(str) & ~('0' <= str & str <= '9')) = '_'; %%------------------------------------------------------------------------- function endpos = matching_quote(str,pos) len=length(str); while(pos1 && str(pos-1)=='\')) endpos=pos; return; end end pos=pos+1; end error('unmatched quotation mark'); %%------------------------------------------------------------------------- function [endpos, e1l, e1r, maxlevel] = matching_bracket(str,pos) global arraytoken level=1; maxlevel=level; endpos=0; bpos=arraytoken(arraytoken>=pos); tokens=str(bpos); len=length(tokens); pos=1; e1l=[]; e1r=[]; while(pos<=len) c=tokens(pos); if(c==']') level=level-1; if(isempty(e1r)) e1r=bpos(pos); end if(level==0) endpos=bpos(pos); return end end if(c=='[') if(isempty(e1l)) e1l=bpos(pos); end level=level+1; maxlevel=max(maxlevel,level); end if(c=='"') pos=matching_quote(tokens,pos+1); end pos=pos+1; end if(endpos==0) error('unmatched "]"'); end ================================================ FILE: machine-learning-ex3/ex3/lib/jsonlab/loadubjson.m ================================================ function data = loadubjson(fname,varargin) % % data=loadubjson(fname,opt) % or % data=loadubjson(fname,'param1',value1,'param2',value2,...) % % parse a JSON (JavaScript Object Notation) file or string % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2013/08/01 % % $Id: loadubjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % fname: input file name, if fname contains "{}" or "[]", fname % will be interpreted as a UBJSON string % opt: a struct to store parsing options, opt can be replaced by % a list of ('param',value) pairs - the param string is equivallent % to a field in opt. opt can have the following % fields (first in [.|.] is the default) % % opt.SimplifyCell [0|1]: if set to 1, loadubjson will call cell2mat % for each element of the JSON data, and group % arrays based on the cell2mat rules. % opt.IntEndian [B|L]: specify the endianness of the integer fields % in the UBJSON input data. B - Big-Endian format for % integers (as required in the UBJSON specification); % L - input integer fields are in Little-Endian order. % % output: % dat: a cell array, where {...} blocks are converted into cell arrays, % and [...] are converted to arrays % % examples: % obj=struct('string','value','array',[1 2 3]); % ubjdata=saveubjson('obj',obj); % dat=loadubjson(ubjdata) % dat=loadubjson(['examples' filesep 'example1.ubj']) % dat=loadubjson(['examples' filesep 'example1.ubj'],'SimplifyCell',1) % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % global pos inStr len esc index_esc len_esc isoct arraytoken fileendian systemendian if(regexp(fname,'[\{\}\]\[]','once')) string=fname; elseif(exist(fname,'file')) fid = fopen(fname,'rb'); string = fread(fid,inf,'uint8=>char')'; fclose(fid); else error('input file does not exist'); end pos = 1; len = length(string); inStr = string; isoct=exist('OCTAVE_VERSION','builtin'); arraytoken=find(inStr=='[' | inStr==']' | inStr=='"'); jstr=regexprep(inStr,'\\\\',' '); escquote=regexp(jstr,'\\"'); arraytoken=sort([arraytoken escquote]); % String delimiters and escape chars identified to improve speed: esc = find(inStr=='"' | inStr=='\' ); % comparable to: regexp(inStr, '["\\]'); index_esc = 1; len_esc = length(esc); opt=varargin2struct(varargin{:}); fileendian=upper(jsonopt('IntEndian','B',opt)); [os,maxelem,systemendian]=computer; jsoncount=1; while pos <= len switch(next_char) case '{' data{jsoncount} = parse_object(opt); case '[' data{jsoncount} = parse_array(opt); otherwise error_pos('Outer level structure must be an object or an array'); end jsoncount=jsoncount+1; end % while jsoncount=length(data); if(jsoncount==1 && iscell(data)) data=data{1}; end if(~isempty(data)) if(isstruct(data)) % data can be a struct array data=jstruct2array(data); elseif(iscell(data)) data=jcell2array(data); end end %% function newdata=parse_collection(id,data,obj) if(jsoncount>0 && exist('data','var')) if(~iscell(data)) newdata=cell(1); newdata{1}=data; data=newdata; end end %% function newdata=jcell2array(data) len=length(data); newdata=data; for i=1:len if(isstruct(data{i})) newdata{i}=jstruct2array(data{i}); elseif(iscell(data{i})) newdata{i}=jcell2array(data{i}); end end %%------------------------------------------------------------------------- function newdata=jstruct2array(data) fn=fieldnames(data); newdata=data; len=length(data); for i=1:length(fn) % depth-first for j=1:len if(isstruct(getfield(data(j),fn{i}))) newdata(j)=setfield(newdata(j),fn{i},jstruct2array(getfield(data(j),fn{i}))); end end end if(~isempty(strmatch('x0x5F_ArrayType_',fn)) && ~isempty(strmatch('x0x5F_ArrayData_',fn))) newdata=cell(len,1); for j=1:len ndata=cast(data(j).x0x5F_ArrayData_,data(j).x0x5F_ArrayType_); iscpx=0; if(~isempty(strmatch('x0x5F_ArrayIsComplex_',fn))) if(data(j).x0x5F_ArrayIsComplex_) iscpx=1; end end if(~isempty(strmatch('x0x5F_ArrayIsSparse_',fn))) if(data(j).x0x5F_ArrayIsSparse_) if(~isempty(strmatch('x0x5F_ArraySize_',fn))) dim=double(data(j).x0x5F_ArraySize_); if(iscpx && size(ndata,2)==4-any(dim==1)) ndata(:,end-1)=complex(ndata(:,end-1),ndata(:,end)); end if isempty(ndata) % All-zeros sparse ndata=sparse(dim(1),prod(dim(2:end))); elseif dim(1)==1 % Sparse row vector ndata=sparse(1,ndata(:,1),ndata(:,2),dim(1),prod(dim(2:end))); elseif dim(2)==1 % Sparse column vector ndata=sparse(ndata(:,1),1,ndata(:,2),dim(1),prod(dim(2:end))); else % Generic sparse array. ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3),dim(1),prod(dim(2:end))); end else if(iscpx && size(ndata,2)==4) ndata(:,3)=complex(ndata(:,3),ndata(:,4)); end ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3)); end end elseif(~isempty(strmatch('x0x5F_ArraySize_',fn))) if(iscpx && size(ndata,2)==2) ndata=complex(ndata(:,1),ndata(:,2)); end ndata=reshape(ndata(:),data(j).x0x5F_ArraySize_); end newdata{j}=ndata; end if(len==1) newdata=newdata{1}; end end %%------------------------------------------------------------------------- function object = parse_object(varargin) parse_char('{'); object = []; type=''; count=-1; if(next_char == '$') type=inStr(pos+1); % TODO pos=pos+2; end if(next_char == '#') pos=pos+1; count=double(parse_number()); end if next_char ~= '}' num=0; while 1 str = parseStr(varargin{:}); if isempty(str) error_pos('Name of value at position %d cannot be empty'); end %parse_char(':'); val = parse_value(varargin{:}); num=num+1; eval( sprintf( 'object.%s = val;', valid_field(str) ) ); if next_char == '}' || (count>=0 && num>=count) break; end %parse_char(','); end end if(count==-1) parse_char('}'); end %%------------------------------------------------------------------------- function [cid,len]=elem_info(type) id=strfind('iUIlLdD',type); dataclass={'int8','uint8','int16','int32','int64','single','double'}; bytelen=[1,1,2,4,8,4,8]; if(id>0) cid=dataclass{id}; len=bytelen(id); else error_pos('unsupported type at position %d'); end %%------------------------------------------------------------------------- function [data adv]=parse_block(type,count,varargin) global pos inStr isoct fileendian systemendian [cid,len]=elem_info(type); datastr=inStr(pos:pos+len*count-1); if(isoct) newdata=int8(datastr); else newdata=uint8(datastr); end id=strfind('iUIlLdD',type); if(id<=5 && fileendian~=systemendian) newdata=swapbytes(typecast(newdata,cid)); end data=typecast(newdata,cid); adv=double(len*count); %%------------------------------------------------------------------------- function object = parse_array(varargin) % JSON array is written in row-major order global pos inStr isoct parse_char('['); object = cell(0, 1); dim=[]; type=''; count=-1; if(next_char == '$') type=inStr(pos+1); pos=pos+2; end if(next_char == '#') pos=pos+1; if(next_char=='[') dim=parse_array(varargin{:}); count=prod(double(dim)); else count=double(parse_number()); end end if(~isempty(type)) if(count>=0) [object adv]=parse_block(type,count,varargin{:}); if(~isempty(dim)) object=reshape(object,dim); end pos=pos+adv; return; else endpos=matching_bracket(inStr,pos); [cid,len]=elem_info(type); count=(endpos-pos)/len; [object adv]=parse_block(type,count,varargin{:}); pos=pos+adv; parse_char(']'); return; end end if next_char ~= ']' while 1 val = parse_value(varargin{:}); object{end+1} = val; if next_char == ']' break; end %parse_char(','); end end if(jsonopt('SimplifyCell',0,varargin{:})==1) try oldobj=object; object=cell2mat(object')'; if(iscell(oldobj) && isstruct(object) && numel(object)>1 && jsonopt('SimplifyCellArray',1,varargin{:})==0) object=oldobj; elseif(size(object,1)>1 && ndims(object)==2) object=object'; end catch end end if(count==-1) parse_char(']'); end %%------------------------------------------------------------------------- function parse_char(c) global pos inStr len skip_whitespace; if pos > len || inStr(pos) ~= c error_pos(sprintf('Expected %c at position %%d', c)); else pos = pos + 1; skip_whitespace; end %%------------------------------------------------------------------------- function c = next_char global pos inStr len skip_whitespace; if pos > len c = []; else c = inStr(pos); end %%------------------------------------------------------------------------- function skip_whitespace global pos inStr len while pos <= len && isspace(inStr(pos)) pos = pos + 1; end %%------------------------------------------------------------------------- function str = parseStr(varargin) global pos inStr esc index_esc len_esc % len, ns = length(inStr), keyboard type=inStr(pos); if type ~= 'S' && type ~= 'C' && type ~= 'H' error_pos('String starting with S expected at position %d'); else pos = pos + 1; end if(type == 'C') str=inStr(pos); pos=pos+1; return; end bytelen=double(parse_number()); if(length(inStr)>=pos+bytelen-1) str=inStr(pos:pos+bytelen-1); pos=pos+bytelen; else error_pos('End of file while expecting end of inStr'); end %%------------------------------------------------------------------------- function num = parse_number(varargin) global pos inStr len isoct fileendian systemendian id=strfind('iUIlLdD',inStr(pos)); if(isempty(id)) error_pos('expecting a number at position %d'); end type={'int8','uint8','int16','int32','int64','single','double'}; bytelen=[1,1,2,4,8,4,8]; datastr=inStr(pos+1:pos+bytelen(id)); if(isoct) newdata=int8(datastr); else newdata=uint8(datastr); end if(id<=5 && fileendian~=systemendian) newdata=swapbytes(typecast(newdata,type{id})); end num=typecast(newdata,type{id}); pos = pos + bytelen(id)+1; %%------------------------------------------------------------------------- function val = parse_value(varargin) global pos inStr len true = 1; false = 0; switch(inStr(pos)) case {'S','C','H'} val = parseStr(varargin{:}); return; case '[' val = parse_array(varargin{:}); return; case '{' val = parse_object(varargin{:}); if isstruct(val) if(~isempty(strmatch('x0x5F_ArrayType_',fieldnames(val), 'exact'))) val=jstruct2array(val); end elseif isempty(val) val = struct; end return; case {'i','U','I','l','L','d','D'} val = parse_number(varargin{:}); return; case 'T' val = true; pos = pos + 1; return; case 'F' val = false; pos = pos + 1; return; case {'Z','N'} val = []; pos = pos + 1; return; end error_pos('Value expected at position %d'); %%------------------------------------------------------------------------- function error_pos(msg) global pos inStr len poShow = max(min([pos-15 pos-1 pos pos+20],len),1); if poShow(3) == poShow(2) poShow(3:4) = poShow(2)+[0 -1]; % display nothing after end msg = [sprintf(msg, pos) ': ' ... inStr(poShow(1):poShow(2)) '' inStr(poShow(3):poShow(4)) ]; error( ['JSONparser:invalidFormat: ' msg] ); %%------------------------------------------------------------------------- function str = valid_field(str) global isoct % From MATLAB doc: field names must begin with a letter, which may be % followed by any combination of letters, digits, and underscores. % Invalid characters will be converted to underscores, and the prefix % "x0x[Hex code]_" will be added if the first character is not a letter. pos=regexp(str,'^[^A-Za-z]','once'); if(~isempty(pos)) if(~isoct) str=regexprep(str,'^([^A-Za-z])','x0x${sprintf(''%X'',unicode2native($1))}_','once'); else str=sprintf('x0x%X_%s',char(str(1)),str(2:end)); end end if(isempty(regexp(str,'[^0-9A-Za-z_]', 'once' ))) return; end if(~isoct) str=regexprep(str,'([^0-9A-Za-z_])','_0x${sprintf(''%X'',unicode2native($1))}_'); else pos=regexp(str,'[^0-9A-Za-z_]'); if(isempty(pos)) return; end str0=str; pos0=[0 pos(:)' length(str)]; str=''; for i=1:length(pos) str=[str str0(pos0(i)+1:pos(i)-1) sprintf('_0x%X_',str0(pos(i)))]; end if(pos(end)~=length(str)) str=[str str0(pos0(end-1)+1:pos0(end))]; end end %str(~isletter(str) & ~('0' <= str & str <= '9')) = '_'; %%------------------------------------------------------------------------- function endpos = matching_quote(str,pos) len=length(str); while(pos1 && str(pos-1)=='\')) endpos=pos; return; end end pos=pos+1; end error('unmatched quotation mark'); %%------------------------------------------------------------------------- function [endpos e1l e1r maxlevel] = matching_bracket(str,pos) global arraytoken level=1; maxlevel=level; endpos=0; bpos=arraytoken(arraytoken>=pos); tokens=str(bpos); len=length(tokens); pos=1; e1l=[]; e1r=[]; while(pos<=len) c=tokens(pos); if(c==']') level=level-1; if(isempty(e1r)) e1r=bpos(pos); end if(level==0) endpos=bpos(pos); return end end if(c=='[') if(isempty(e1l)) e1l=bpos(pos); end level=level+1; maxlevel=max(maxlevel,level); end if(c=='"') pos=matching_quote(tokens,pos+1); end pos=pos+1; end if(endpos==0) error('unmatched "]"'); end ================================================ FILE: machine-learning-ex3/ex3/lib/jsonlab/mergestruct.m ================================================ function s=mergestruct(s1,s2) % % s=mergestruct(s1,s2) % % merge two struct objects into one % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % date: 2012/12/22 % % input: % s1,s2: a struct object, s1 and s2 can not be arrays % % output: % s: the merged struct object. fields in s1 and s2 will be combined in s. % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(~isstruct(s1) || ~isstruct(s2)) error('input parameters contain non-struct'); end if(length(s1)>1 || length(s2)>1) error('can not merge struct arrays'); end fn=fieldnames(s2); s=s1; for i=1:length(fn) s=setfield(s,fn{i},getfield(s2,fn{i})); end ================================================ FILE: machine-learning-ex3/ex3/lib/jsonlab/savejson.m ================================================ function json=savejson(rootname,obj,varargin) % % json=savejson(rootname,obj,filename) % or % json=savejson(rootname,obj,opt) % json=savejson(rootname,obj,'param1',value1,'param2',value2,...) % % convert a MATLAB object (cell, struct or array) into a JSON (JavaScript % Object Notation) string % % author: Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2011/09/09 % % $Id: savejson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % rootname: the name of the root-object, when set to '', the root name % is ignored, however, when opt.ForceRootName is set to 1 (see below), % the MATLAB variable name will be used as the root name. % obj: a MATLAB object (array, cell, cell array, struct, struct array). % filename: a string for the file name to save the output JSON data. % opt: a struct for additional options, ignore to use default values. % opt can have the following fields (first in [.|.] is the default) % % opt.FileName [''|string]: a file name to save the output JSON data % opt.FloatFormat ['%.10g'|string]: format to show each numeric element % of a 1D/2D array; % opt.ArrayIndent [1|0]: if 1, output explicit data array with % precedent indentation; if 0, no indentation % opt.ArrayToStruct[0|1]: when set to 0, savejson outputs 1D/2D % array in JSON array format; if sets to 1, an % array will be shown as a struct with fields % "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for % sparse arrays, the non-zero elements will be % saved to _ArrayData_ field in triplet-format i.e. % (ix,iy,val) and "_ArrayIsSparse_" will be added % with a value of 1; for a complex array, the % _ArrayData_ array will include two columns % (4 for sparse) to record the real and imaginary % parts, and also "_ArrayIsComplex_":1 is added. % opt.ParseLogical [0|1]: if this is set to 1, logical array elem % will use true/false rather than 1/0. % opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single % numerical element will be shown without a square % bracket, unless it is the root object; if 0, square % brackets are forced for any numerical arrays. % opt.ForceRootName [0|1]: when set to 1 and rootname is empty, savejson % will use the name of the passed obj variable as the % root object name; if obj is an expression and % does not have a name, 'root' will be used; if this % is set to 0 and rootname is empty, the root level % will be merged down to the lower level. % opt.Inf ['"$1_Inf_"'|string]: a customized regular expression pattern % to represent +/-Inf. The matched pattern is '([-+]*)Inf' % and $1 represents the sign. For those who want to use % 1e999 to represent Inf, they can set opt.Inf to '$11e999' % opt.NaN ['"_NaN_"'|string]: a customized regular expression pattern % to represent NaN % opt.JSONP [''|string]: to generate a JSONP output (JSON with padding), % for example, if opt.JSONP='foo', the JSON data is % wrapped inside a function call as 'foo(...);' % opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson % back to the string form % opt.SaveBinary [0|1]: 1 - save the JSON file in binary mode; 0 - text mode. % opt.Compact [0|1]: 1- out compact JSON format (remove all newlines and tabs) % % opt can be replaced by a list of ('param',value) pairs. The param % string is equivallent to a field in opt and is case sensitive. % output: % json: a string in the JSON format (see http://json.org) % % examples: % jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... % 'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],... % 'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;... % 2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],... % 'MeshCreator','FangQ','MeshTitle','T6 Cube',... % 'SpecialData',[nan, inf, -inf]); % savejson('jmesh',jsonmesh) % savejson('',jsonmesh,'ArrayIndent',0,'FloatFormat','\t%.5g') % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(nargin==1) varname=inputname(1); obj=rootname; if(isempty(varname)) varname='root'; end rootname=varname; else varname=inputname(2); end if(length(varargin)==1 && ischar(varargin{1})) opt=struct('FileName',varargin{1}); else opt=varargin2struct(varargin{:}); end opt.IsOctave=exist('OCTAVE_VERSION','builtin'); rootisarray=0; rootlevel=1; forceroot=jsonopt('ForceRootName',0,opt); if((isnumeric(obj) || islogical(obj) || ischar(obj) || isstruct(obj) || iscell(obj)) && isempty(rootname) && forceroot==0) rootisarray=1; rootlevel=0; else if(isempty(rootname)) rootname=varname; end end if((isstruct(obj) || iscell(obj))&& isempty(rootname) && forceroot) rootname='root'; end whitespaces=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); if(jsonopt('Compact',0,opt)==1) whitespaces=struct('tab','','newline','','sep',','); end if(~isfield(opt,'whitespaces_')) opt.whitespaces_=whitespaces; end nl=whitespaces.newline; json=obj2json(rootname,obj,rootlevel,opt); if(rootisarray) json=sprintf('%s%s',json,nl); else json=sprintf('{%s%s%s}\n',nl,json,nl); end jsonp=jsonopt('JSONP','',opt); if(~isempty(jsonp)) json=sprintf('%s(%s);%s',jsonp,json,nl); end % save to a file if FileName is set, suggested by Patrick Rapin if(~isempty(jsonopt('FileName','',opt))) if(jsonopt('SaveBinary',0,opt)==1) fid = fopen(opt.FileName, 'wb'); fwrite(fid,json); else fid = fopen(opt.FileName, 'wt'); fwrite(fid,json,'char'); end fclose(fid); end %%------------------------------------------------------------------------- function txt=obj2json(name,item,level,varargin) if(iscell(item)) txt=cell2json(name,item,level,varargin{:}); elseif(isstruct(item)) txt=struct2json(name,item,level,varargin{:}); elseif(ischar(item)) txt=str2json(name,item,level,varargin{:}); else txt=mat2json(name,item,level,varargin{:}); end %%------------------------------------------------------------------------- function txt=cell2json(name,item,level,varargin) txt=''; if(~iscell(item)) error('input is not a cell'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); ws=jsonopt('whitespaces_',struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')),varargin{:}); padding0=repmat(ws.tab,1,level); padding2=repmat(ws.tab,1,level+1); nl=ws.newline; if(len>1) if(~isempty(name)) txt=sprintf('%s"%s": [%s',padding0, checkname(name,varargin{:}),nl); name=''; else txt=sprintf('%s[%s',padding0,nl); end elseif(len==0) if(~isempty(name)) txt=sprintf('%s"%s": []',padding0, checkname(name,varargin{:})); name=''; else txt=sprintf('%s[]',padding0); end end for j=1:dim(2) if(dim(1)>1) txt=sprintf('%s%s[%s',txt,padding2,nl); end for i=1:dim(1) txt=sprintf('%s%s',txt,obj2json(name,item{i,j},level+(dim(1)>1)+1,varargin{:})); if(i1) txt=sprintf('%s%s%s]',txt,nl,padding2); end if(j1) txt=sprintf('%s%s%s]',txt,nl,padding0); end %%------------------------------------------------------------------------- function txt=struct2json(name,item,level,varargin) txt=''; if(~isstruct(item)) error('input is not a struct'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); ws=struct('tab',sprintf('\t'),'newline',sprintf('\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding0=repmat(ws.tab,1,level); padding2=repmat(ws.tab,1,level+1); padding1=repmat(ws.tab,1,level+(dim(1)>1)+(len>1)); nl=ws.newline; if(~isempty(name)) if(len>1) txt=sprintf('%s"%s": [%s',padding0,checkname(name,varargin{:}),nl); end else if(len>1) txt=sprintf('%s[%s',padding0,nl); end end for j=1:dim(2) if(dim(1)>1) txt=sprintf('%s%s[%s',txt,padding2,nl); end for i=1:dim(1) names = fieldnames(item(i,j)); if(~isempty(name) && len==1) txt=sprintf('%s%s"%s": {%s',txt,padding1, checkname(name,varargin{:}),nl); else txt=sprintf('%s%s{%s',txt,padding1,nl); end if(~isempty(names)) for e=1:length(names) txt=sprintf('%s%s',txt,obj2json(names{e},getfield(item(i,j),... names{e}),level+(dim(1)>1)+1+(len>1),varargin{:})); if(e1) txt=sprintf('%s%s%s]',txt,nl,padding2); end if(j1) txt=sprintf('%s%s%s]',txt,nl,padding0); end %%------------------------------------------------------------------------- function txt=str2json(name,item,level,varargin) txt=''; if(~ischar(item)) error('input is not a string'); end item=reshape(item, max(size(item),[1 0])); len=size(item,1); ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding1=repmat(ws.tab,1,level); padding0=repmat(ws.tab,1,level+1); nl=ws.newline; sep=ws.sep; if(~isempty(name)) if(len>1) txt=sprintf('%s"%s": [%s',padding1,checkname(name,varargin{:}),nl); end else if(len>1) txt=sprintf('%s[%s',padding1,nl); end end isoct=jsonopt('IsOctave',0,varargin{:}); for e=1:len if(isoct) val=regexprep(item(e,:),'\\','\\'); val=regexprep(val,'"','\"'); val=regexprep(val,'^"','\"'); else val=regexprep(item(e,:),'\\','\\\\'); val=regexprep(val,'"','\\"'); val=regexprep(val,'^"','\\"'); end val=escapejsonstring(val); if(len==1) obj=['"' checkname(name,varargin{:}) '": ' '"',val,'"']; if(isempty(name)) obj=['"',val,'"']; end txt=sprintf('%s%s%s%s',txt,padding1,obj); else txt=sprintf('%s%s%s%s',txt,padding0,['"',val,'"']); end if(e==len) sep=''; end txt=sprintf('%s%s',txt,sep); end if(len>1) txt=sprintf('%s%s%s%s',txt,nl,padding1,']'); end %%------------------------------------------------------------------------- function txt=mat2json(name,item,level,varargin) if(~isnumeric(item) && ~islogical(item)) error('input is not an array'); end ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding1=repmat(ws.tab,1,level); padding0=repmat(ws.tab,1,level+1); nl=ws.newline; sep=ws.sep; if(length(size(item))>2 || issparse(item) || ~isreal(item) || ... isempty(item) ||jsonopt('ArrayToStruct',0,varargin{:})) if(isempty(name)) txt=sprintf('%s{%s%s"_ArrayType_": "%s",%s%s"_ArraySize_": %s,%s',... padding1,nl,padding0,class(item),nl,padding0,regexprep(mat2str(size(item)),'\s+',','),nl); else txt=sprintf('%s"%s": {%s%s"_ArrayType_": "%s",%s%s"_ArraySize_": %s,%s',... padding1,checkname(name,varargin{:}),nl,padding0,class(item),nl,padding0,regexprep(mat2str(size(item)),'\s+',','),nl); end else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1 && level>0) numtxt=regexprep(regexprep(matdata2json(item,level+1,varargin{:}),'^\[',''),']',''); else numtxt=matdata2json(item,level+1,varargin{:}); end if(isempty(name)) txt=sprintf('%s%s',padding1,numtxt); else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1) txt=sprintf('%s"%s": %s',padding1,checkname(name,varargin{:}),numtxt); else txt=sprintf('%s"%s": %s',padding1,checkname(name,varargin{:}),numtxt); end end return; end dataformat='%s%s%s%s%s'; if(issparse(item)) [ix,iy]=find(item); data=full(item(find(item))); if(~isreal(item)) data=[real(data(:)),imag(data(:))]; if(size(item,1)==1) % Kludge to have data's 'transposedness' match item's. % (Necessary for complex row vector handling below.) data=data'; end txt=sprintf(dataformat,txt,padding0,'"_ArrayIsComplex_": ','1', sep); end txt=sprintf(dataformat,txt,padding0,'"_ArrayIsSparse_": ','1', sep); if(size(item,1)==1) % Row vector, store only column indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([iy(:),data'],level+2,varargin{:}), nl); elseif(size(item,2)==1) % Column vector, store only row indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([ix,data],level+2,varargin{:}), nl); else % General case, store row and column indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([ix,iy,data],level+2,varargin{:}), nl); end else if(isreal(item)) txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json(item(:)',level+2,varargin{:}), nl); else txt=sprintf(dataformat,txt,padding0,'"_ArrayIsComplex_": ','1', sep); txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([real(item(:)) imag(item(:))],level+2,varargin{:}), nl); end end txt=sprintf('%s%s%s',txt,padding1,'}'); %%------------------------------------------------------------------------- function txt=matdata2json(mat,level,varargin) ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); tab=ws.tab; nl=ws.newline; if(size(mat,1)==1) pre=''; post=''; level=level-1; else pre=sprintf('[%s',nl); post=sprintf('%s%s]',nl,repmat(tab,1,level-1)); end if(isempty(mat)) txt='null'; return; end floatformat=jsonopt('FloatFormat','%.10g',varargin{:}); %if(numel(mat)>1) formatstr=['[' repmat([floatformat ','],1,size(mat,2)-1) [floatformat sprintf('],%s',nl)]]; %else % formatstr=[repmat([floatformat ','],1,size(mat,2)-1) [floatformat sprintf(',\n')]]; %end if(nargin>=2 && size(mat,1)>1 && jsonopt('ArrayIndent',1,varargin{:})==1) formatstr=[repmat(tab,1,level) formatstr]; end txt=sprintf(formatstr,mat'); txt(end-length(nl):end)=[]; if(islogical(mat) && jsonopt('ParseLogical',0,varargin{:})==1) txt=regexprep(txt,'1','true'); txt=regexprep(txt,'0','false'); end %txt=regexprep(mat2str(mat),'\s+',','); %txt=regexprep(txt,';',sprintf('],\n[')); % if(nargin>=2 && size(mat,1)>1) % txt=regexprep(txt,'\[',[repmat(sprintf('\t'),1,level) '[']); % end txt=[pre txt post]; if(any(isinf(mat(:)))) txt=regexprep(txt,'([-+]*)Inf',jsonopt('Inf','"$1_Inf_"',varargin{:})); end if(any(isnan(mat(:)))) txt=regexprep(txt,'NaN',jsonopt('NaN','"_NaN_"',varargin{:})); end %%------------------------------------------------------------------------- function newname=checkname(name,varargin) isunpack=jsonopt('UnpackHex',1,varargin{:}); newname=name; if(isempty(regexp(name,'0x([0-9a-fA-F]+)_','once'))) return end if(isunpack) isoct=jsonopt('IsOctave',0,varargin{:}); if(~isoct) newname=regexprep(name,'(^x|_){1}0x([0-9a-fA-F]+)_','${native2unicode(hex2dec($2))}'); else pos=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','start'); pend=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','end'); if(isempty(pos)) return; end str0=name; pos0=[0 pend(:)' length(name)]; newname=''; for i=1:length(pos) newname=[newname str0(pos0(i)+1:pos(i)-1) char(hex2dec(str0(pos(i)+3:pend(i)-1)))]; end if(pos(end)~=length(name)) newname=[newname str0(pos0(end-1)+1:pos0(end))]; end end end %%------------------------------------------------------------------------- function newstr=escapejsonstring(str) newstr=str; isoct=exist('OCTAVE_VERSION','builtin'); if(isoct) vv=sscanf(OCTAVE_VERSION,'%f'); if(vv(1)>=3.8) isoct=0; end end if(isoct) escapechars={'\a','\f','\n','\r','\t','\v'}; for i=1:length(escapechars); newstr=regexprep(newstr,escapechars{i},escapechars{i}); end else escapechars={'\a','\b','\f','\n','\r','\t','\v'}; for i=1:length(escapechars); newstr=regexprep(newstr,escapechars{i},regexprep(escapechars{i},'\\','\\\\')); end end ================================================ FILE: machine-learning-ex3/ex3/lib/jsonlab/saveubjson.m ================================================ function json=saveubjson(rootname,obj,varargin) % % json=saveubjson(rootname,obj,filename) % or % json=saveubjson(rootname,obj,opt) % json=saveubjson(rootname,obj,'param1',value1,'param2',value2,...) % % convert a MATLAB object (cell, struct or array) into a Universal % Binary JSON (UBJSON) binary string % % author: Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2013/08/17 % % $Id: saveubjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % rootname: the name of the root-object, when set to '', the root name % is ignored, however, when opt.ForceRootName is set to 1 (see below), % the MATLAB variable name will be used as the root name. % obj: a MATLAB object (array, cell, cell array, struct, struct array) % filename: a string for the file name to save the output UBJSON data % opt: a struct for additional options, ignore to use default values. % opt can have the following fields (first in [.|.] is the default) % % opt.FileName [''|string]: a file name to save the output JSON data % opt.ArrayToStruct[0|1]: when set to 0, saveubjson outputs 1D/2D % array in JSON array format; if sets to 1, an % array will be shown as a struct with fields % "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for % sparse arrays, the non-zero elements will be % saved to _ArrayData_ field in triplet-format i.e. % (ix,iy,val) and "_ArrayIsSparse_" will be added % with a value of 1; for a complex array, the % _ArrayData_ array will include two columns % (4 for sparse) to record the real and imaginary % parts, and also "_ArrayIsComplex_":1 is added. % opt.ParseLogical [1|0]: if this is set to 1, logical array elem % will use true/false rather than 1/0. % opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single % numerical element will be shown without a square % bracket, unless it is the root object; if 0, square % brackets are forced for any numerical arrays. % opt.ForceRootName [0|1]: when set to 1 and rootname is empty, saveubjson % will use the name of the passed obj variable as the % root object name; if obj is an expression and % does not have a name, 'root' will be used; if this % is set to 0 and rootname is empty, the root level % will be merged down to the lower level. % opt.JSONP [''|string]: to generate a JSONP output (JSON with padding), % for example, if opt.JSON='foo', the JSON data is % wrapped inside a function call as 'foo(...);' % opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson % back to the string form % % opt can be replaced by a list of ('param',value) pairs. The param % string is equivallent to a field in opt and is case sensitive. % output: % json: a binary string in the UBJSON format (see http://ubjson.org) % % examples: % jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... % 'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],... % 'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;... % 2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],... % 'MeshCreator','FangQ','MeshTitle','T6 Cube',... % 'SpecialData',[nan, inf, -inf]); % saveubjson('jsonmesh',jsonmesh) % saveubjson('jsonmesh',jsonmesh,'meshdata.ubj') % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(nargin==1) varname=inputname(1); obj=rootname; if(isempty(varname)) varname='root'; end rootname=varname; else varname=inputname(2); end if(length(varargin)==1 && ischar(varargin{1})) opt=struct('FileName',varargin{1}); else opt=varargin2struct(varargin{:}); end opt.IsOctave=exist('OCTAVE_VERSION','builtin'); rootisarray=0; rootlevel=1; forceroot=jsonopt('ForceRootName',0,opt); if((isnumeric(obj) || islogical(obj) || ischar(obj) || isstruct(obj) || iscell(obj)) && isempty(rootname) && forceroot==0) rootisarray=1; rootlevel=0; else if(isempty(rootname)) rootname=varname; end end if((isstruct(obj) || iscell(obj))&& isempty(rootname) && forceroot) rootname='root'; end json=obj2ubjson(rootname,obj,rootlevel,opt); if(~rootisarray) json=['{' json '}']; end jsonp=jsonopt('JSONP','',opt); if(~isempty(jsonp)) json=[jsonp '(' json ')']; end % save to a file if FileName is set, suggested by Patrick Rapin if(~isempty(jsonopt('FileName','',opt))) fid = fopen(opt.FileName, 'wb'); fwrite(fid,json); fclose(fid); end %%------------------------------------------------------------------------- function txt=obj2ubjson(name,item,level,varargin) if(iscell(item)) txt=cell2ubjson(name,item,level,varargin{:}); elseif(isstruct(item)) txt=struct2ubjson(name,item,level,varargin{:}); elseif(ischar(item)) txt=str2ubjson(name,item,level,varargin{:}); else txt=mat2ubjson(name,item,level,varargin{:}); end %%------------------------------------------------------------------------- function txt=cell2ubjson(name,item,level,varargin) txt=''; if(~iscell(item)) error('input is not a cell'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); % let's handle 1D cell first if(len>1) if(~isempty(name)) txt=[S_(checkname(name,varargin{:})) '[']; name=''; else txt='['; end elseif(len==0) if(~isempty(name)) txt=[S_(checkname(name,varargin{:})) 'Z']; name=''; else txt='Z'; end end for j=1:dim(2) if(dim(1)>1) txt=[txt '[']; end for i=1:dim(1) txt=[txt obj2ubjson(name,item{i,j},level+(len>1),varargin{:})]; end if(dim(1)>1) txt=[txt ']']; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=struct2ubjson(name,item,level,varargin) txt=''; if(~isstruct(item)) error('input is not a struct'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); if(~isempty(name)) if(len>1) txt=[S_(checkname(name,varargin{:})) '[']; end else if(len>1) txt='['; end end for j=1:dim(2) if(dim(1)>1) txt=[txt '[']; end for i=1:dim(1) names = fieldnames(item(i,j)); if(~isempty(name) && len==1) txt=[txt S_(checkname(name,varargin{:})) '{']; else txt=[txt '{']; end if(~isempty(names)) for e=1:length(names) txt=[txt obj2ubjson(names{e},getfield(item(i,j),... names{e}),level+(dim(1)>1)+1+(len>1),varargin{:})]; end end txt=[txt '}']; end if(dim(1)>1) txt=[txt ']']; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=str2ubjson(name,item,level,varargin) txt=''; if(~ischar(item)) error('input is not a string'); end item=reshape(item, max(size(item),[1 0])); len=size(item,1); if(~isempty(name)) if(len>1) txt=[S_(checkname(name,varargin{:})) '[']; end else if(len>1) txt='['; end end isoct=jsonopt('IsOctave',0,varargin{:}); for e=1:len val=item(e,:); if(len==1) obj=['' S_(checkname(name,varargin{:})) '' '',S_(val),'']; if(isempty(name)) obj=['',S_(val),'']; end txt=[txt,'',obj]; else txt=[txt,'',['',S_(val),'']]; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=mat2ubjson(name,item,level,varargin) if(~isnumeric(item) && ~islogical(item)) error('input is not an array'); end if(length(size(item))>2 || issparse(item) || ~isreal(item) || ... isempty(item) || jsonopt('ArrayToStruct',0,varargin{:})) cid=I_(uint32(max(size(item)))); if(isempty(name)) txt=['{' S_('_ArrayType_'),S_(class(item)),S_('_ArraySize_'),I_a(size(item),cid(1)) ]; else if(isempty(item)) txt=[S_(checkname(name,varargin{:})),'Z']; return; else txt=[S_(checkname(name,varargin{:})),'{',S_('_ArrayType_'),S_(class(item)),S_('_ArraySize_'),I_a(size(item),cid(1))]; end end else if(isempty(name)) txt=matdata2ubjson(item,level+1,varargin{:}); else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1) numtxt=regexprep(regexprep(matdata2ubjson(item,level+1,varargin{:}),'^\[',''),']',''); txt=[S_(checkname(name,varargin{:})) numtxt]; else txt=[S_(checkname(name,varargin{:})),matdata2ubjson(item,level+1,varargin{:})]; end end return; end if(issparse(item)) [ix,iy]=find(item); data=full(item(find(item))); if(~isreal(item)) data=[real(data(:)),imag(data(:))]; if(size(item,1)==1) % Kludge to have data's 'transposedness' match item's. % (Necessary for complex row vector handling below.) data=data'; end txt=[txt,S_('_ArrayIsComplex_'),'T']; end txt=[txt,S_('_ArrayIsSparse_'),'T']; if(size(item,1)==1) % Row vector, store only column indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([iy(:),data'],level+2,varargin{:})]; elseif(size(item,2)==1) % Column vector, store only row indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([ix,data],level+2,varargin{:})]; else % General case, store row and column indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([ix,iy,data],level+2,varargin{:})]; end else if(isreal(item)) txt=[txt,S_('_ArrayData_'),... matdata2ubjson(item(:)',level+2,varargin{:})]; else txt=[txt,S_('_ArrayIsComplex_'),'T']; txt=[txt,S_('_ArrayData_'),... matdata2ubjson([real(item(:)) imag(item(:))],level+2,varargin{:})]; end end txt=[txt,'}']; %%------------------------------------------------------------------------- function txt=matdata2ubjson(mat,level,varargin) if(isempty(mat)) txt='Z'; return; end if(size(mat,1)==1) level=level-1; end type=''; hasnegtive=(mat<0); if(isa(mat,'integer') || isinteger(mat) || (isfloat(mat) && all(mod(mat(:),1) == 0))) if(isempty(hasnegtive)) if(max(mat(:))<=2^8) type='U'; end end if(isempty(type)) % todo - need to consider negative ones separately id= histc(abs(max(mat(:))),[0 2^7 2^15 2^31 2^63]); if(isempty(find(id))) error('high-precision data is not yet supported'); end key='iIlL'; type=key(find(id)); end txt=[I_a(mat(:),type,size(mat))]; elseif(islogical(mat)) logicalval='FT'; if(numel(mat)==1) txt=logicalval(mat+1); else txt=['[$U#' I_a(size(mat),'l') typecast(swapbytes(uint8(mat(:)')),'uint8')]; end else if(numel(mat)==1) txt=['[' D_(mat) ']']; else txt=D_a(mat(:),'D',size(mat)); end end %txt=regexprep(mat2str(mat),'\s+',','); %txt=regexprep(txt,';',sprintf('],[')); % if(nargin>=2 && size(mat,1)>1) % txt=regexprep(txt,'\[',[repmat(sprintf('\t'),1,level) '[']); % end if(any(isinf(mat(:)))) txt=regexprep(txt,'([-+]*)Inf',jsonopt('Inf','"$1_Inf_"',varargin{:})); end if(any(isnan(mat(:)))) txt=regexprep(txt,'NaN',jsonopt('NaN','"_NaN_"',varargin{:})); end %%------------------------------------------------------------------------- function newname=checkname(name,varargin) isunpack=jsonopt('UnpackHex',1,varargin{:}); newname=name; if(isempty(regexp(name,'0x([0-9a-fA-F]+)_','once'))) return end if(isunpack) isoct=jsonopt('IsOctave',0,varargin{:}); if(~isoct) newname=regexprep(name,'(^x|_){1}0x([0-9a-fA-F]+)_','${native2unicode(hex2dec($2))}'); else pos=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','start'); pend=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','end'); if(isempty(pos)) return; end str0=name; pos0=[0 pend(:)' length(name)]; newname=''; for i=1:length(pos) newname=[newname str0(pos0(i)+1:pos(i)-1) char(hex2dec(str0(pos(i)+3:pend(i)-1)))]; end if(pos(end)~=length(name)) newname=[newname str0(pos0(end-1)+1:pos0(end))]; end end end %%------------------------------------------------------------------------- function val=S_(str) if(length(str)==1) val=['C' str]; else val=['S' I_(int32(length(str))) str]; end %%------------------------------------------------------------------------- function val=I_(num) if(~isinteger(num)) error('input is not an integer'); end if(num>=0 && num<255) val=['U' data2byte(swapbytes(cast(num,'uint8')),'uint8')]; return; end key='iIlL'; cid={'int8','int16','int32','int64'}; for i=1:4 if((num>0 && num<2^(i*8-1)) || (num<0 && num>=-2^(i*8-1))) val=[key(i) data2byte(swapbytes(cast(num,cid{i})),'uint8')]; return; end end error('unsupported integer'); %%------------------------------------------------------------------------- function val=D_(num) if(~isfloat(num)) error('input is not a float'); end if(isa(num,'single')) val=['d' data2byte(num,'uint8')]; else val=['D' data2byte(num,'uint8')]; end %%------------------------------------------------------------------------- function data=I_a(num,type,dim,format) id=find(ismember('iUIlL',type)); if(id==0) error('unsupported integer array'); end % based on UBJSON specs, all integer types are stored in big endian format if(id==1) data=data2byte(swapbytes(int8(num)),'uint8'); blen=1; elseif(id==2) data=data2byte(swapbytes(uint8(num)),'uint8'); blen=1; elseif(id==3) data=data2byte(swapbytes(int16(num)),'uint8'); blen=2; elseif(id==4) data=data2byte(swapbytes(int32(num)),'uint8'); blen=4; elseif(id==5) data=data2byte(swapbytes(int64(num)),'uint8'); blen=8; end if(nargin>=3 && length(dim)>=2 && prod(dim)~=dim(2)) format='opt'; end if((nargin<4 || strcmp(format,'opt')) && numel(num)>1) if(nargin>=3 && (length(dim)==1 || (length(dim)>=2 && prod(dim)~=dim(2)))) cid=I_(uint32(max(dim))); data=['$' type '#' I_a(dim,cid(1)) data(:)']; else data=['$' type '#' I_(int32(numel(data)/blen)) data(:)']; end data=['[' data(:)']; else data=reshape(data,blen,numel(data)/blen); data(2:blen+1,:)=data; data(1,:)=type; data=data(:)'; data=['[' data(:)' ']']; end %%------------------------------------------------------------------------- function data=D_a(num,type,dim,format) id=find(ismember('dD',type)); if(id==0) error('unsupported float array'); end if(id==1) data=data2byte(single(num),'uint8'); elseif(id==2) data=data2byte(double(num),'uint8'); end if(nargin>=3 && length(dim)>=2 && prod(dim)~=dim(2)) format='opt'; end if((nargin<4 || strcmp(format,'opt')) && numel(num)>1) if(nargin>=3 && (length(dim)==1 || (length(dim)>=2 && prod(dim)~=dim(2)))) cid=I_(uint32(max(dim))); data=['$' type '#' I_a(dim,cid(1)) data(:)']; else data=['$' type '#' I_(int32(numel(data)/(id*4))) data(:)']; end data=['[' data]; else data=reshape(data,(id*4),length(data)/(id*4)); data(2:(id*4+1),:)=data; data(1,:)=type; data=data(:)'; data=['[' data(:)' ']']; end %%------------------------------------------------------------------------- function bytes=data2byte(varargin) bytes=typecast(varargin{:}); bytes=bytes(:)'; ================================================ FILE: machine-learning-ex3/ex3/lib/jsonlab/varargin2struct.m ================================================ function opt=varargin2struct(varargin) % % opt=varargin2struct('param1',value1,'param2',value2,...) % or % opt=varargin2struct(...,optstruct,...) % % convert a series of input parameters into a structure % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % date: 2012/12/22 % % input: % 'param', value: the input parameters should be pairs of a string and a value % optstruct: if a parameter is a struct, the fields will be merged to the output struct % % output: % opt: a struct where opt.param1=value1, opt.param2=value2 ... % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % len=length(varargin); opt=struct; if(len==0) return; end i=1; while(i<=len) if(isstruct(varargin{i})) opt=mergestruct(opt,varargin{i}); elseif(ischar(varargin{i}) && i 0; Xm = [ -1 -1 ; -1 -2 ; -2 -1 ; -2 -2 ; ... 1 1 ; 1 2 ; 2 1 ; 2 2 ; ... -1 1 ; -1 2 ; -2 1 ; -2 2 ; ... 1 -1 ; 1 -2 ; -2 -1 ; -2 -2 ]; ym = [ 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 ]'; t1 = sin(reshape(1:2:24, 4, 3)); t2 = cos(reshape(1:2:40, 4, 5)); if partId == '1' [J, grad] = lrCostFunction([0.25 0.5 -0.5]', X, y, 0.1); out = sprintf('%0.5f ', J); out = [out sprintf('%0.5f ', grad)]; elseif partId == '2' out = sprintf('%0.5f ', oneVsAll(Xm, ym, 4, 0.1)); elseif partId == '3' out = sprintf('%0.5f ', predictOneVsAll(t1, Xm)); elseif partId == '4' out = sprintf('%0.5f ', predict(t1, t2, Xm)); end end ================================================ FILE: machine-learning-ex4/ex4/checkNNGradients.m ================================================ function checkNNGradients(lambda) %CHECKNNGRADIENTS Creates a small neural network to check the %backpropagation gradients % CHECKNNGRADIENTS(lambda) Creates a small neural network to check the % backpropagation gradients, it will output the analytical gradients % produced by your backprop code and the numerical gradients (computed % using computeNumericalGradient). These two gradient computations should % result in very similar values. % if ~exist('lambda', 'var') || isempty(lambda) lambda = 0; end input_layer_size = 3; hidden_layer_size = 5; num_labels = 3; m = 5; % We generate some 'random' test data Theta1 = debugInitializeWeights(hidden_layer_size, input_layer_size); Theta2 = debugInitializeWeights(num_labels, hidden_layer_size); % Reusing debugInitializeWeights to generate X X = debugInitializeWeights(m, input_layer_size - 1); y = 1 + mod(1:m, num_labels)'; % Unroll parameters nn_params = [Theta1(:) ; Theta2(:)]; % Short hand for cost function costFunc = @(p) nnCostFunction(p, input_layer_size, hidden_layer_size, ... num_labels, X, y, lambda); [cost, grad] = costFunc(nn_params); numgrad = computeNumericalGradient(costFunc, nn_params); % Visually examine the two gradient computations. The two columns % you get should be very similar. disp([numgrad grad]); fprintf(['The above two columns you get should be very similar.\n' ... '(Left-Your Numerical Gradient, Right-Analytical Gradient)\n\n']); % Evaluate the norm of the difference between two solutions. % If you have a correct implementation, and assuming you used EPSILON = 0.0001 % in computeNumericalGradient.m, then diff below should be less than 1e-9 diff = norm(numgrad-grad)/norm(numgrad+grad); fprintf(['If your backpropagation implementation is correct, then \n' ... 'the relative difference will be small (less than 1e-9). \n' ... '\nRelative Difference: %g\n'], diff); end ================================================ FILE: machine-learning-ex4/ex4/computeNumericalGradient.m ================================================ function numgrad = computeNumericalGradient(J, theta) %COMPUTENUMERICALGRADIENT Computes the gradient using "finite differences" %and gives us a numerical estimate of the gradient. % numgrad = COMPUTENUMERICALGRADIENT(J, theta) computes the numerical % gradient of the function J around theta. Calling y = J(theta) should % return the function value at theta. % Notes: The following code implements numerical gradient checking, and % returns the numerical gradient.It sets numgrad(i) to (a numerical % approximation of) the partial derivative of J with respect to the % i-th input argument, evaluated at theta. (i.e., numgrad(i) should % be the (approximately) the partial derivative of J with respect % to theta(i).) % numgrad = zeros(size(theta)); perturb = zeros(size(theta)); e = 1e-4; for p = 1:numel(theta) % Set perturbation vector perturb(p) = e; loss1 = J(theta - perturb); loss2 = J(theta + perturb); % Compute Numerical Gradient numgrad(p) = (loss2 - loss1) / (2*e); perturb(p) = 0; end end ================================================ FILE: machine-learning-ex4/ex4/debugInitializeWeights.m ================================================ function W = debugInitializeWeights(fan_out, fan_in) %DEBUGINITIALIZEWEIGHTS Initialize the weights of a layer with fan_in %incoming connections and fan_out outgoing connections using a fixed %strategy, this will help you later in debugging % W = DEBUGINITIALIZEWEIGHTS(fan_in, fan_out) initializes the weights % of a layer with fan_in incoming connections and fan_out outgoing % connections using a fix set of values % % Note that W should be set to a matrix of size(1 + fan_in, fan_out) as % the first row of W handles the "bias" terms % % Set W to zeros W = zeros(fan_out, 1 + fan_in); % Initialize W using "sin", this ensures that W is always of the same % values and will be useful for debugging W = reshape(sin(1:numel(W)), size(W)) / 10; % ========================================================================= end ================================================ FILE: machine-learning-ex4/ex4/displayData.m ================================================ function [h, display_array] = displayData(X, example_width) %DISPLAYDATA Display 2D data in a nice grid % [h, display_array] = DISPLAYDATA(X, example_width) displays 2D data % stored in X in a nice grid. It returns the figure handle h and the % displayed array if requested. % Set example_width automatically if not passed in if ~exist('example_width', 'var') || isempty(example_width) example_width = round(sqrt(size(X, 2))); end % Gray Image colormap(gray); % Compute rows, cols [m n] = size(X); example_height = (n / example_width); % Compute number of items to display display_rows = floor(sqrt(m)); display_cols = ceil(m / display_rows); % Between images padding pad = 1; % Setup blank display display_array = - ones(pad + display_rows * (example_height + pad), ... pad + display_cols * (example_width + pad)); % Copy each example into a patch on the display array curr_ex = 1; for j = 1:display_rows for i = 1:display_cols if curr_ex > m, break; end % Copy the patch % Get the max value of the patch max_val = max(abs(X(curr_ex, :))); display_array(pad + (j - 1) * (example_height + pad) + (1:example_height), ... pad + (i - 1) * (example_width + pad) + (1:example_width)) = ... reshape(X(curr_ex, :), example_height, example_width) / max_val; curr_ex = curr_ex + 1; end if curr_ex > m, break; end end % Display Image h = imagesc(display_array, [-1 1]); % Do not show axis axis image off drawnow; end ================================================ FILE: machine-learning-ex4/ex4/ex4.m ================================================ %% Machine Learning Online Class - Exercise 4 Neural Network Learning % Instructions % ------------ % % This file contains code that helps you get started on the % linear exercise. You will need to complete the following functions % in this exericse: % % sigmoidGradient.m % randInitializeWeights.m % nnCostFunction.m % % For this exercise, you will not need to change any code in this file, % or any other files other than those mentioned above. % %% Initialization clear ; close all; clc %% Setup the parameters you will use for this exercise input_layer_size = 400; % 20x20 Input Images of Digits hidden_layer_size = 25; % 25 hidden units num_labels = 10; % 10 labels, from 1 to 10 % (note that we have mapped "0" to label 10) %% =========== Part 1: Loading and Visualizing Data ============= % We start the exercise by first loading and visualizing the dataset. % You will be working with a dataset that contains handwritten digits. % % Load Training Data fprintf('Loading and Visualizing Data ...\n') load('ex4data1.mat'); m = size(X, 1); % Randomly select 100 data points to display sel = randperm(size(X, 1)); sel = sel(1:100); displayData(X(sel, :)); fprintf('Program paused. Press enter to continue.\n'); pause; %% ================ Part 2: Loading Parameters ================ % In this part of the exercise, we load some pre-initialized % neural network parameters. fprintf('\nLoading Saved Neural Network Parameters ...\n') % Load the weights into variables Theta1 and Theta2 load('ex4weights.mat'); % Unroll parameters nn_params = [Theta1(:) ; Theta2(:)]; %% ================ Part 3: Compute Cost (Feedforward) ================ % To the neural network, you should first start by implementing the % feedforward part of the neural network that returns the cost only. You % should complete the code in nnCostFunction.m to return cost. After % implementing the feedforward to compute the cost, you can verify that % your implementation is correct by verifying that you get the same cost % as us for the fixed debugging parameters. % % We suggest implementing the feedforward cost *without* regularization % first so that it will be easier for you to debug. Later, in part 4, you % will get to implement the regularized cost. % fprintf('\nFeedforward Using Neural Network ...\n') % Weight regularization parameter (we set this to 0 here). lambda = 0.1; J = nnCostFunction(nn_params, input_layer_size, hidden_layer_size, ... num_labels, X, y, lambda); fprintf(['Cost at parameters (loaded from ex4weights): %f '... '\n(this value should be about 0.287629)\n'], J); fprintf('\nProgram paused. Press enter to continue.\n'); pause; %% =============== Part 4: Implement Regularization =============== % Once your cost function implementation is correct, you should now % continue to implement the regularization with the cost. % fprintf('\nChecking Cost Function (w/ Regularization) ... \n') % Weight regularization parameter (we set this to 1 here). lambda = 1; J = nnCostFunction(nn_params, input_layer_size, hidden_layer_size, ... num_labels, X, y, lambda); fprintf(['Cost at parameters (loaded from ex4weights): %f '... '\n(this value should be about 0.383770)\n'], J); fprintf('Program paused. Press enter to continue.\n'); pause; %% ================ Part 5: Sigmoid Gradient ================ % Before you start implementing the neural network, you will first % implement the gradient for the sigmoid function. You should complete the % code in the sigmoidGradient.m file. % fprintf('\nEvaluating sigmoid gradient...\n') g = sigmoidGradient([1 -0.5 0 0.5 1]); fprintf('Sigmoid gradient evaluated at [1 -0.5 0 0.5 1]:\n '); fprintf('%f ', g); fprintf('\n\n'); fprintf('Program paused. Press enter to continue.\n'); pause; %% ================ Part 6: Initializing Pameters ================ % In this part of the exercise, you will be starting to implment a two % layer neural network that classifies digits. You will start by % implementing a function to initialize the weights of the neural network % (randInitializeWeights.m) fprintf('\nInitializing Neural Network Parameters ...\n') initial_Theta1 = randInitializeWeights(input_layer_size, hidden_layer_size); initial_Theta2 = randInitializeWeights(hidden_layer_size, num_labels); % Unroll parameters initial_nn_params = [initial_Theta1(:) ; initial_Theta2(:)]; %% =============== Part 7: Implement Backpropagation =============== % Once your cost matches up with ours, you should proceed to implement the % backpropagation algorithm for the neural network. You should add to the % code you've written in nnCostFunction.m to return the partial % derivatives of the parameters. % fprintf('\nChecking Backpropagation... \n'); % Check gradients by running checkNNGradients checkNNGradients; fprintf('\nProgram paused. Press enter to continue.\n'); pause; %% =============== Part 8: Implement Regularization =============== % Once your backpropagation implementation is correct, you should now % continue to implement the regularization with the cost and gradient. % fprintf('\nChecking Backpropagation (w/ Regularization) ... \n') % Check gradients by running checkNNGradients lambda = 3; checkNNGradients(lambda); % Also output the costFunction debugging values debug_J = nnCostFunction(nn_params, input_layer_size, ... hidden_layer_size, num_labels, X, y, lambda); fprintf(['\n\nCost at (fixed) debugging parameters (w/ lambda = 10): %f ' ... '\n(this value should be about 0.576051)\n\n'], debug_J); fprintf('Program paused. Press enter to continue.\n'); pause; %% =================== Part 8: Training NN =================== % You have now implemented all the code necessary to train a neural % network. To train your neural network, we will now use "fmincg", which % is a function which works similarly to "fminunc". Recall that these % advanced optimizers are able to train our cost functions efficiently as % long as we provide them with the gradient computations. % fprintf('\nTraining Neural Network... \n') % After you have completed the assignment, change the MaxIter to a larger % value to see how more training helps. options = optimset('MaxIter', 50); % You should also try different values of lambda lambda = 1; % Create "short hand" for the cost function to be minimized costFunction = @(p) nnCostFunction(p, ... input_layer_size, ... hidden_layer_size, ... num_labels, X, y, lambda); % Now, costFunction is a function that takes in only one argument (the % neural network parameters) [nn_params, cost] = fmincg(costFunction, initial_nn_params, options); % Obtain Theta1 and Theta2 back from nn_params Theta1 = reshape(nn_params(1:hidden_layer_size * (input_layer_size + 1)), ... hidden_layer_size, (input_layer_size + 1)); Theta2 = reshape(nn_params((1 + (hidden_layer_size * (input_layer_size + 1))):end), ... num_labels, (hidden_layer_size + 1)); fprintf('Program paused. Press enter to continue.\n'); pause; %% ================= Part 9: Visualize Weights ================= % You can now "visualize" what the neural network is learning by % displaying the hidden units to see what features they are capturing in % the data. fprintf('\nVisualizing Neural Network... \n') displayData(Theta1(:, 2:end)); fprintf('\nProgram paused. Press enter to continue.\n'); pause; %% ================= Part 10: Implement Predict ================= % After training the neural network, we would like to use it to predict % the labels. You will now implement the "predict" function to use the % neural network to predict the labels of the training set. This lets % you compute the training set accuracy. pred = predict(Theta1, Theta2, X); fprintf('\nTraining Set Accuracy: %f\n', mean(double(pred == y)) * 100); ================================================ FILE: machine-learning-ex4/ex4/fmincg.m ================================================ function [X, fX, i] = fmincg(f, X, options, P1, P2, P3, P4, P5) % Minimize a continuous differentialble multivariate function. Starting point % is given by "X" (D by 1), and the function named in the string "f", must % return a function value and a vector of partial derivatives. The Polack- % Ribiere flavour of conjugate gradients is used to compute search directions, % and a line search using quadratic and cubic polynomial approximations and the % Wolfe-Powell stopping criteria is used together with the slope ratio method % for guessing initial step sizes. Additionally a bunch of checks are made to % make sure that exploration is taking place and that extrapolation will not % be unboundedly large. The "length" gives the length of the run: if it is % positive, it gives the maximum number of line searches, if negative its % absolute gives the maximum allowed number of function evaluations. You can % (optionally) give "length" a second component, which will indicate the % reduction in function value to be expected in the first line-search (defaults % to 1.0). The function returns when either its length is up, or if no further % progress can be made (ie, we are at a minimum, or so close that due to % numerical problems, we cannot get any closer). If the function terminates % within a few iterations, it could be an indication that the function value % and derivatives are not consistent (ie, there may be a bug in the % implementation of your "f" function). The function returns the found % solution "X", a vector of function values "fX" indicating the progress made % and "i" the number of iterations (line searches or function evaluations, % depending on the sign of "length") used. % % Usage: [X, fX, i] = fmincg(f, X, options, P1, P2, P3, P4, P5) % % See also: checkgrad % % Copyright (C) 2001 and 2002 by Carl Edward Rasmussen. Date 2002-02-13 % % % (C) Copyright 1999, 2000 & 2001, Carl Edward Rasmussen % % Permission is granted for anyone to copy, use, or modify these % programs and accompanying documents for purposes of research or % education, provided this copyright notice is retained, and note is % made of any changes that have been made. % % These programs and documents are distributed without any warranty, % express or implied. As the programs were written for research % purposes only, they have not been tested to the degree that would be % advisable in any important application. All use of these programs is % entirely at the user's own risk. % % [ml-class] Changes Made: % 1) Function name and argument specifications % 2) Output display % % Read options if exist('options', 'var') && ~isempty(options) && isfield(options, 'MaxIter') length = options.MaxIter; else length = 100; end RHO = 0.01; % a bunch of constants for line searches SIG = 0.5; % RHO and SIG are the constants in the Wolfe-Powell conditions INT = 0.1; % don't reevaluate within 0.1 of the limit of the current bracket EXT = 3.0; % extrapolate maximum 3 times the current bracket MAX = 20; % max 20 function evaluations per line search RATIO = 100; % maximum allowed slope ratio argstr = ['feval(f, X']; % compose string used to call function for i = 1:(nargin - 3) argstr = [argstr, ',P', int2str(i)]; end argstr = [argstr, ')']; if max(size(length)) == 2, red=length(2); length=length(1); else red=1; end S=['Iteration ']; i = 0; % zero the run length counter ls_failed = 0; % no previous line search has failed fX = []; [f1 df1] = eval(argstr); % get function value and gradient i = i + (length<0); % count epochs?! s = -df1; % search direction is steepest d1 = -s'*s; % this is the slope z1 = red/(1-d1); % initial step is red/(|s|+1) while i < abs(length) % while not finished i = i + (length>0); % count iterations?! X0 = X; f0 = f1; df0 = df1; % make a copy of current values X = X + z1*s; % begin line search [f2 df2] = eval(argstr); i = i + (length<0); % count epochs?! d2 = df2'*s; f3 = f1; d3 = d1; z3 = -z1; % initialize point 3 equal to point 1 if length>0, M = MAX; else M = min(MAX, -length-i); end success = 0; limit = -1; % initialize quanteties while 1 while ((f2 > f1+z1*RHO*d1) || (d2 > -SIG*d1)) && (M > 0) limit = z1; % tighten the bracket if f2 > f1 z2 = z3 - (0.5*d3*z3*z3)/(d3*z3+f2-f3); % quadratic fit else A = 6*(f2-f3)/z3+3*(d2+d3); % cubic fit B = 3*(f3-f2)-z3*(d3+2*d2); z2 = (sqrt(B*B-A*d2*z3*z3)-B)/A; % numerical error possible - ok! end if isnan(z2) || isinf(z2) z2 = z3/2; % if we had a numerical problem then bisect end z2 = max(min(z2, INT*z3),(1-INT)*z3); % don't accept too close to limits z1 = z1 + z2; % update the step X = X + z2*s; [f2 df2] = eval(argstr); M = M - 1; i = i + (length<0); % count epochs?! d2 = df2'*s; z3 = z3-z2; % z3 is now relative to the location of z2 end if f2 > f1+z1*RHO*d1 || d2 > -SIG*d1 break; % this is a failure elseif d2 > SIG*d1 success = 1; break; % success elseif M == 0 break; % failure end A = 6*(f2-f3)/z3+3*(d2+d3); % make cubic extrapolation B = 3*(f3-f2)-z3*(d3+2*d2); z2 = -d2*z3*z3/(B+sqrt(B*B-A*d2*z3*z3)); % num. error possible - ok! if ~isreal(z2) || isnan(z2) || isinf(z2) || z2 < 0 % num prob or wrong sign? if limit < -0.5 % if we have no upper limit z2 = z1 * (EXT-1); % the extrapolate the maximum amount else z2 = (limit-z1)/2; % otherwise bisect end elseif (limit > -0.5) && (z2+z1 > limit) % extraplation beyond max? z2 = (limit-z1)/2; % bisect elseif (limit < -0.5) && (z2+z1 > z1*EXT) % extrapolation beyond limit z2 = z1*(EXT-1.0); % set to extrapolation limit elseif z2 < -z3*INT z2 = -z3*INT; elseif (limit > -0.5) && (z2 < (limit-z1)*(1.0-INT)) % too close to limit? z2 = (limit-z1)*(1.0-INT); end f3 = f2; d3 = d2; z3 = -z2; % set point 3 equal to point 2 z1 = z1 + z2; X = X + z2*s; % update current estimates [f2 df2] = eval(argstr); M = M - 1; i = i + (length<0); % count epochs?! d2 = df2'*s; end % end of line search if success % if line search succeeded f1 = f2; fX = [fX' f1]'; fprintf('%s %4i | Cost: %4.6e\r', S, i, f1); s = (df2'*df2-df1'*df2)/(df1'*df1)*s - df2; % Polack-Ribiere direction tmp = df1; df1 = df2; df2 = tmp; % swap derivatives d2 = df1'*s; if d2 > 0 % new slope must be negative s = -df1; % otherwise use steepest direction d2 = -s'*s; end z1 = z1 * min(RATIO, d1/(d2-realmin)); % slope ratio but max RATIO d1 = d2; ls_failed = 0; % this line search did not fail else X = X0; f1 = f0; df1 = df0; % restore point from before failed line search if ls_failed || i > abs(length) % line search failed twice in a row break; % or we ran out of time, so we give up end tmp = df1; df1 = df2; df2 = tmp; % swap derivatives s = -df1; % try steepest d1 = -s'*s; z1 = 1/(1-d1); ls_failed = 1; % this line search failed end if exist('OCTAVE_VERSION') fflush(stdout); end end fprintf('\n'); ================================================ FILE: machine-learning-ex4/ex4/lib/jsonlab/AUTHORS.txt ================================================ The author of "jsonlab" toolbox is Qianqian Fang. Qianqian is currently an Assistant Professor at Massachusetts General Hospital, Harvard Medical School. Address: Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School Bldg 149, 13th St, Charlestown, MA 02129, USA URL: http://nmr.mgh.harvard.edu/~fangq/ Email: or The script loadjson.m was built upon previous works by - Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713 date: 2009/11/02 - François Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393 date: 2009/03/22 - Joel Feenstra: http://www.mathworks.com/matlabcentral/fileexchange/20565 date: 2008/07/03 This toolbox contains patches submitted by the following contributors: - Blake Johnson part of revision 341 - Niclas Borlin various fixes in revision 394, including - loadjson crashes for all-zero sparse matrix. - loadjson crashes for empty sparse matrix. - Non-zero size of 0-by-N and N-by-0 empty matrices is lost after savejson/loadjson. - loadjson crashes for sparse real column vector. - loadjson crashes for sparse complex column vector. - Data is corrupted by savejson for sparse real row vector. - savejson crashes for sparse complex row vector. - Yul Kang patches for svn revision 415. - savejson saves an empty cell array as [] instead of null - loadjson differentiates an empty struct from an empty array ================================================ FILE: machine-learning-ex4/ex4/lib/jsonlab/ChangeLog.txt ================================================ ============================================================================ JSONlab - a toolbox to encode/decode JSON/UBJSON files in MATLAB/Octave ---------------------------------------------------------------------------- JSONlab ChangeLog (key features marked by *): == JSONlab 1.0 (codename: Optimus - Final), FangQ == 2015/01/02 polish help info for all major functions, update examples, finalize 1.0 2014/12/19 fix a bug to strictly respect NoRowBracket in savejson == JSONlab 1.0.0-RC2 (codename: Optimus - RC2), FangQ == 2014/11/22 show progress bar in loadjson ('ShowProgress') 2014/11/17 add Compact option in savejson to output compact JSON format ('Compact') 2014/11/17 add FastArrayParser in loadjson to specify fast parser applicable levels 2014/09/18 start official github mirror: https://github.com/fangq/jsonlab == JSONlab 1.0.0-RC1 (codename: Optimus - RC1), FangQ == 2014/09/17 fix several compatibility issues when running on octave versions 3.2-3.8 2014/09/17 support 2D cell and struct arrays in both savejson and saveubjson 2014/08/04 escape special characters in a JSON string 2014/02/16 fix a bug when saving ubjson files == JSONlab 0.9.9 (codename: Optimus - beta), FangQ == 2014/01/22 use binary read and write in saveubjson and loadubjson == JSONlab 0.9.8-1 (codename: Optimus - alpha update 1), FangQ == 2013/10/07 better round-trip conservation for empty arrays and structs (patch submitted by Yul Kang) == JSONlab 0.9.8 (codename: Optimus - alpha), FangQ == 2013/08/23 *universal Binary JSON (UBJSON) support, including both saveubjson and loadubjson == JSONlab 0.9.1 (codename: Rodimus, update 1), FangQ == 2012/12/18 *handling of various empty and sparse matrices (fixes submitted by Niclas Borlin) == JSONlab 0.9.0 (codename: Rodimus), FangQ == 2012/06/17 *new format for an invalid leading char, unpacking hex code in savejson 2012/06/01 support JSONP in savejson 2012/05/25 fix the empty cell bug (reported by Cyril Davin) 2012/04/05 savejson can save to a file (suggested by Patrick Rapin) == JSONlab 0.8.1 (codename: Sentiel, Update 1), FangQ == 2012/02/28 loadjson quotation mark escape bug, see http://bit.ly/yyk1nS 2012/01/25 patch to handle root-less objects, contributed by Blake Johnson == JSONlab 0.8.0 (codename: Sentiel), FangQ == 2012/01/13 *speed up loadjson by 20 fold when parsing large data arrays in matlab 2012/01/11 remove row bracket if an array has 1 element, suggested by Mykel Kochenderfer 2011/12/22 *accept sequence of 'param',value input in savejson and loadjson 2011/11/18 fix struct array bug reported by Mykel Kochenderfer == JSONlab 0.5.1 (codename: Nexus Update 1), FangQ == 2011/10/21 fix a bug in loadjson, previous code does not use any of the acceleration 2011/10/20 loadjson supports JSON collections - concatenated JSON objects == JSONlab 0.5.0 (codename: Nexus), FangQ == 2011/10/16 package and release jsonlab 0.5.0 2011/10/15 *add json demo and regression test, support cpx numbers, fix double quote bug 2011/10/11 *speed up readjson dramatically, interpret _Array* tags, show data in root level 2011/10/10 create jsonlab project, start jsonlab website, add online documentation 2011/10/07 *speed up savejson by 25x using sprintf instead of mat2str, add options support 2011/10/06 *savejson works for structs, cells and arrays 2011/09/09 derive loadjson from JSON parser from MATLAB Central, draft savejson.m ================================================ FILE: machine-learning-ex4/ex4/lib/jsonlab/LICENSE_BSD.txt ================================================ Copyright 2011-2015 Qianqian Fang . All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. The views and conclusions contained in the software and documentation are those of the authors and should not be interpreted as representing official policies, either expressed or implied, of the copyright holders. ================================================ FILE: machine-learning-ex4/ex4/lib/jsonlab/README.txt ================================================ =============================================================================== = JSONLab = = An open-source MATLAB/Octave JSON encoder and decoder = =============================================================================== *Copyright (C) 2011-2015 Qianqian Fang *License: BSD License, see License_BSD.txt for details *Version: 1.0 (Optimus - Final) ------------------------------------------------------------------------------- Table of Content: I. Introduction II. Installation III.Using JSONLab IV. Known Issues and TODOs V. Contribution and feedback ------------------------------------------------------------------------------- I. Introduction JSON ([http://www.json.org/ JavaScript Object Notation]) is a highly portable, human-readable and "[http://en.wikipedia.org/wiki/JSON fat-free]" text format to represent complex and hierarchical data. It is as powerful as [http://en.wikipedia.org/wiki/XML XML], but less verbose. JSON format is widely used for data-exchange in applications, and is essential for the wild success of [http://en.wikipedia.org/wiki/Ajax_(programming) Ajax] and [http://en.wikipedia.org/wiki/Web_2.0 Web2.0]. UBJSON (Universal Binary JSON) is a binary JSON format, specifically optimized for compact file size and better performance while keeping the semantics as simple as the text-based JSON format. Using the UBJSON format allows to wrap complex binary data in a flexible and extensible structure, making it possible to process complex and large dataset without accuracy loss due to text conversions. We envision that both JSON and its binary version will serve as part of the mainstream data-exchange formats for scientific research in the future. It will provide the flexibility and generality achieved by other popular general-purpose file specifications, such as [http://www.hdfgroup.org/HDF5/whatishdf5.html HDF5], with significantly reduced complexity and enhanced performance. JSONLab is a free and open-source implementation of a JSON/UBJSON encoder and a decoder in the native MATLAB language. It can be used to convert a MATLAB data structure (array, struct, cell, struct array and cell array) into JSON/UBJSON formatted strings, or to decode a JSON/UBJSON file into MATLAB data structure. JSONLab supports both MATLAB and [http://www.gnu.org/software/octave/ GNU Octave] (a free MATLAB clone). ------------------------------------------------------------------------------- II. Installation The installation of JSONLab is no different than any other simple MATLAB toolbox. You only need to download/unzip the JSONLab package to a folder, and add the folder's path to MATLAB/Octave's path list by using the following command: addpath('/path/to/jsonlab'); If you want to add this path permanently, you need to type "pathtool", browse to the jsonlab root folder and add to the list, then click "Save". Then, run "rehash" in MATLAB, and type "which loadjson", if you see an output, that means JSONLab is installed for MATLAB/Octave. ------------------------------------------------------------------------------- III.Using JSONLab JSONLab provides two functions, loadjson.m -- a MATLAB->JSON decoder, and savejson.m -- a MATLAB->JSON encoder, for the text-based JSON, and two equivallent functions -- loadubjson and saveubjson for the binary JSON. The detailed help info for the four functions can be found below: === loadjson.m === 
  data=loadjson(fname,opt)
     or
  data=loadjson(fname,'param1',value1,'param2',value2,...)
 
  parse a JSON (JavaScript Object Notation) file or string
 
  authors:Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2011/09/09, including previous works from 
 
          Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713
             created on 2009/11/02
          Franois Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393
             created on  2009/03/22
          Joel Feenstra:
          http://www.mathworks.com/matlabcentral/fileexchange/20565
             created on 2008/07/03
 
  $Id: loadjson.m 452 2014-11-22 16:43:33Z fangq $
 
  input:
       fname: input file name, if fname contains "{}" or "[]", fname
              will be interpreted as a JSON string
       opt: a struct to store parsing options, opt can be replaced by 
            a list of ('param',value) pairs - the param string is equivallent
            to a field in opt. opt can have the following 
            fields (first in [.|.] is the default)
 
            opt.SimplifyCell [0|1]: if set to 1, loadjson will call cell2mat
                          for each element of the JSON data, and group 
                          arrays based on the cell2mat rules.
            opt.FastArrayParser [1|0 or integer]: if set to 1, use a
                          speed-optimized array parser when loading an 
                          array object. The fast array parser may 
                          collapse block arrays into a single large
                          array similar to rules defined in cell2mat; 0 to 
                          use a legacy parser; if set to a larger-than-1
                          value, this option will specify the minimum
                          dimension to enable the fast array parser. For
                          example, if the input is a 3D array, setting
                          FastArrayParser to 1 will return a 3D array;
                          setting to 2 will return a cell array of 2D
                          arrays; setting to 3 will return to a 2D cell
                          array of 1D vectors; setting to 4 will return a
                          3D cell array.
            opt.ShowProgress [0|1]: if set to 1, loadjson displays a progress bar.
 
  output:
       dat: a cell array, where {...} blocks are converted into cell arrays,
            and [...] are converted to arrays
 
  examples:
       dat=loadjson('{"obj":{"string":"value","array":[1,2,3]}}')
       dat=loadjson(['examples' filesep 'example1.json'])
       dat=loadjson(['examples' filesep 'example1.json'],'SimplifyCell',1)
=== savejson.m === 
  json=savejson(rootname,obj,filename)
     or
  json=savejson(rootname,obj,opt)
  json=savejson(rootname,obj,'param1',value1,'param2',value2,...)
 
  convert a MATLAB object (cell, struct or array) into a JSON (JavaScript
  Object Notation) string
 
  author: Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2011/09/09
 
  $Id: savejson.m 458 2014-12-19 22:17:17Z fangq $
 
  input:
       rootname: the name of the root-object, when set to '', the root name
         is ignored, however, when opt.ForceRootName is set to 1 (see below),
         the MATLAB variable name will be used as the root name.
       obj: a MATLAB object (array, cell, cell array, struct, struct array).
       filename: a string for the file name to save the output JSON data.
       opt: a struct for additional options, ignore to use default values.
         opt can have the following fields (first in [.|.] is the default)
 
         opt.FileName [''|string]: a file name to save the output JSON data
         opt.FloatFormat ['%.10g'|string]: format to show each numeric element
                          of a 1D/2D array;
         opt.ArrayIndent [1|0]: if 1, output explicit data array with
                          precedent indentation; if 0, no indentation
         opt.ArrayToStruct[0|1]: when set to 0, savejson outputs 1D/2D
                          array in JSON array format; if sets to 1, an
                          array will be shown as a struct with fields
                          "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for
                          sparse arrays, the non-zero elements will be
                          saved to _ArrayData_ field in triplet-format i.e.
                          (ix,iy,val) and "_ArrayIsSparse_" will be added
                          with a value of 1; for a complex array, the 
                          _ArrayData_ array will include two columns 
                          (4 for sparse) to record the real and imaginary 
                          parts, and also "_ArrayIsComplex_":1 is added. 
         opt.ParseLogical [0|1]: if this is set to 1, logical array elem
                          will use true/false rather than 1/0.
         opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single
                          numerical element will be shown without a square
                          bracket, unless it is the root object; if 0, square
                          brackets are forced for any numerical arrays.
         opt.ForceRootName [0|1]: when set to 1 and rootname is empty, savejson
                          will use the name of the passed obj variable as the 
                          root object name; if obj is an expression and 
                          does not have a name, 'root' will be used; if this 
                          is set to 0 and rootname is empty, the root level 
                          will be merged down to the lower level.
         opt.Inf ['"$1_Inf_"'|string]: a customized regular expression pattern
                          to represent +/-Inf. The matched pattern is '([-+]*)Inf'
                          and $1 represents the sign. For those who want to use
                          1e999 to represent Inf, they can set opt.Inf to '$11e999'
         opt.NaN ['"_NaN_"'|string]: a customized regular expression pattern
                          to represent NaN
         opt.JSONP [''|string]: to generate a JSONP output (JSON with padding),
                          for example, if opt.JSONP='foo', the JSON data is
                          wrapped inside a function call as 'foo(...);'
         opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson 
                          back to the string form
         opt.SaveBinary [0|1]: 1 - save the JSON file in binary mode; 0 - text mode.
         opt.Compact [0|1]: 1- out compact JSON format (remove all newlines and tabs)
 
         opt can be replaced by a list of ('param',value) pairs. The param 
         string is equivallent to a field in opt and is case sensitive.
  output:
       json: a string in the JSON format (see http://json.org)
 
  examples:
       jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... 
                'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],...
                'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;...
                           2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],...
                'MeshCreator','FangQ','MeshTitle','T6 Cube',...
                'SpecialData',[nan, inf, -inf]);
       savejson('jmesh',jsonmesh)
       savejson('',jsonmesh,'ArrayIndent',0,'FloatFormat','\t%.5g')
=== loadubjson.m === 
  data=loadubjson(fname,opt)
     or
  data=loadubjson(fname,'param1',value1,'param2',value2,...)
 
  parse a JSON (JavaScript Object Notation) file or string
 
  authors:Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2013/08/01
 
  $Id: loadubjson.m 436 2014-08-05 20:51:40Z fangq $
 
  input:
       fname: input file name, if fname contains "{}" or "[]", fname
              will be interpreted as a UBJSON string
       opt: a struct to store parsing options, opt can be replaced by 
            a list of ('param',value) pairs - the param string is equivallent
            to a field in opt. opt can have the following 
            fields (first in [.|.] is the default)
 
            opt.SimplifyCell [0|1]: if set to 1, loadubjson will call cell2mat
                          for each element of the JSON data, and group 
                          arrays based on the cell2mat rules.
            opt.IntEndian [B|L]: specify the endianness of the integer fields
                          in the UBJSON input data. B - Big-Endian format for 
                          integers (as required in the UBJSON specification); 
                          L - input integer fields are in Little-Endian order.
 
  output:
       dat: a cell array, where {...} blocks are converted into cell arrays,
            and [...] are converted to arrays
 
  examples:
       obj=struct('string','value','array',[1 2 3]);
       ubjdata=saveubjson('obj',obj);
       dat=loadubjson(ubjdata)
       dat=loadubjson(['examples' filesep 'example1.ubj'])
       dat=loadubjson(['examples' filesep 'example1.ubj'],'SimplifyCell',1)
=== saveubjson.m === 
  json=saveubjson(rootname,obj,filename)
     or
  json=saveubjson(rootname,obj,opt)
  json=saveubjson(rootname,obj,'param1',value1,'param2',value2,...)
 
  convert a MATLAB object (cell, struct or array) into a Universal 
  Binary JSON (UBJSON) binary string
 
  author: Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2013/08/17
 
  $Id: saveubjson.m 440 2014-09-17 19:59:45Z fangq $
 
  input:
       rootname: the name of the root-object, when set to '', the root name
         is ignored, however, when opt.ForceRootName is set to 1 (see below),
         the MATLAB variable name will be used as the root name.
       obj: a MATLAB object (array, cell, cell array, struct, struct array)
       filename: a string for the file name to save the output UBJSON data
       opt: a struct for additional options, ignore to use default values.
         opt can have the following fields (first in [.|.] is the default)
 
         opt.FileName [''|string]: a file name to save the output JSON data
         opt.ArrayToStruct[0|1]: when set to 0, saveubjson outputs 1D/2D
                          array in JSON array format; if sets to 1, an
                          array will be shown as a struct with fields
                          "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for
                          sparse arrays, the non-zero elements will be
                          saved to _ArrayData_ field in triplet-format i.e.
                          (ix,iy,val) and "_ArrayIsSparse_" will be added
                          with a value of 1; for a complex array, the 
                          _ArrayData_ array will include two columns 
                          (4 for sparse) to record the real and imaginary 
                          parts, and also "_ArrayIsComplex_":1 is added. 
         opt.ParseLogical [1|0]: if this is set to 1, logical array elem
                          will use true/false rather than 1/0.
         opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single
                          numerical element will be shown without a square
                          bracket, unless it is the root object; if 0, square
                          brackets are forced for any numerical arrays.
         opt.ForceRootName [0|1]: when set to 1 and rootname is empty, saveubjson
                          will use the name of the passed obj variable as the 
                          root object name; if obj is an expression and 
                          does not have a name, 'root' will be used; if this 
                          is set to 0 and rootname is empty, the root level 
                          will be merged down to the lower level.
         opt.JSONP [''|string]: to generate a JSONP output (JSON with padding),
                          for example, if opt.JSON='foo', the JSON data is
                          wrapped inside a function call as 'foo(...);'
         opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson 
                          back to the string form
 
         opt can be replaced by a list of ('param',value) pairs. The param 
         string is equivallent to a field in opt and is case sensitive.
  output:
       json: a binary string in the UBJSON format (see http://ubjson.org)
 
  examples:
       jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... 
                'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],...
                'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;...
                           2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],...
                'MeshCreator','FangQ','MeshTitle','T6 Cube',...
                'SpecialData',[nan, inf, -inf]);
       saveubjson('jsonmesh',jsonmesh)
       saveubjson('jsonmesh',jsonmesh,'meshdata.ubj')
=== examples === Under the "examples" folder, you can find several scripts to demonstrate the basic utilities of JSONLab. Running the "demo_jsonlab_basic.m" script, you will see the conversions from MATLAB data structure to JSON text and backward. In "jsonlab_selftest.m", we load complex JSON files downloaded from the Internet and validate the loadjson/savejson functions for regression testing purposes. Similarly, a "demo_ubjson_basic.m" script is provided to test the saveubjson and loadubjson pairs for various matlab data structures. Please run these examples and understand how JSONLab works before you use it to process your data. ------------------------------------------------------------------------------- IV. Known Issues and TODOs JSONLab has several known limitations. We are striving to make it more general and robust. Hopefully in a few future releases, the limitations become less. Here are the known issues: # 3D or higher dimensional cell/struct-arrays will be converted to 2D arrays; # When processing names containing multi-byte characters, Octave and MATLAB \ can give different field-names; you can use feature('DefaultCharacterSet','latin1') \ in MATLAB to get consistant results # savejson can not handle class and dataset. # saveubjson converts a logical array into a uint8 ([U]) array # an unofficial N-D array count syntax is implemented in saveubjson. We are \ actively communicating with the UBJSON spec maintainer to investigate the \ possibility of making it upstream # loadubjson can not parse all UBJSON Specification (Draft 9) compliant \ files, however, it can parse all UBJSON files produced by saveubjson. ------------------------------------------------------------------------------- V. Contribution and feedback JSONLab is an open-source project. This means you can not only use it and modify it as you wish, but also you can contribute your changes back to JSONLab so that everyone else can enjoy the improvement. For anyone who want to contribute, please download JSONLab source code from it's subversion repository by using the following command: svn checkout svn://svn.code.sf.net/p/iso2mesh/code/trunk/jsonlab jsonlab You can make changes to the files as needed. Once you are satisfied with your changes, and ready to share it with others, please cd the root directory of JSONLab, and type svn diff > yourname_featurename.patch You then email the .patch file to JSONLab's maintainer, Qianqian Fang, at the email address shown in the beginning of this file. Qianqian will review the changes and commit it to the subversion if they are satisfactory. We appreciate any suggestions and feedbacks from you. Please use iso2mesh's mailing list to report any questions you may have with JSONLab: http://groups.google.com/group/iso2mesh-users?hl=en&pli=1 (Subscription to the mailing list is needed in order to post messages). ================================================ FILE: machine-learning-ex4/ex4/lib/jsonlab/jsonopt.m ================================================ function val=jsonopt(key,default,varargin) % % val=jsonopt(key,default,optstruct) % % setting options based on a struct. The struct can be produced % by varargin2struct from a list of 'param','value' pairs % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % % $Id: loadjson.m 371 2012-06-20 12:43:06Z fangq $ % % input: % key: a string with which one look up a value from a struct % default: if the key does not exist, return default % optstruct: a struct where each sub-field is a key % % output: % val: if key exists, val=optstruct.key; otherwise val=default % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % val=default; if(nargin<=2) return; end opt=varargin{1}; if(isstruct(opt) && isfield(opt,key)) val=getfield(opt,key); end ================================================ FILE: machine-learning-ex4/ex4/lib/jsonlab/loadjson.m ================================================ function data = loadjson(fname,varargin) % % data=loadjson(fname,opt) % or % data=loadjson(fname,'param1',value1,'param2',value2,...) % % parse a JSON (JavaScript Object Notation) file or string % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2011/09/09, including previous works from % % Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713 % created on 2009/11/02 % François Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393 % created on 2009/03/22 % Joel Feenstra: % http://www.mathworks.com/matlabcentral/fileexchange/20565 % created on 2008/07/03 % % $Id: loadjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % fname: input file name, if fname contains "{}" or "[]", fname % will be interpreted as a JSON string % opt: a struct to store parsing options, opt can be replaced by % a list of ('param',value) pairs - the param string is equivallent % to a field in opt. opt can have the following % fields (first in [.|.] is the default) % % opt.SimplifyCell [0|1]: if set to 1, loadjson will call cell2mat % for each element of the JSON data, and group % arrays based on the cell2mat rules. % opt.FastArrayParser [1|0 or integer]: if set to 1, use a % speed-optimized array parser when loading an % array object. The fast array parser may % collapse block arrays into a single large % array similar to rules defined in cell2mat; 0 to % use a legacy parser; if set to a larger-than-1 % value, this option will specify the minimum % dimension to enable the fast array parser. For % example, if the input is a 3D array, setting % FastArrayParser to 1 will return a 3D array; % setting to 2 will return a cell array of 2D % arrays; setting to 3 will return to a 2D cell % array of 1D vectors; setting to 4 will return a % 3D cell array. % opt.ShowProgress [0|1]: if set to 1, loadjson displays a progress bar. % % output: % dat: a cell array, where {...} blocks are converted into cell arrays, % and [...] are converted to arrays % % examples: % dat=loadjson('{"obj":{"string":"value","array":[1,2,3]}}') % dat=loadjson(['examples' filesep 'example1.json']) % dat=loadjson(['examples' filesep 'example1.json'],'SimplifyCell',1) % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % global pos inStr len esc index_esc len_esc isoct arraytoken if(regexp(fname,'[\{\}\]\[]','once')) string=fname; elseif(exist(fname,'file')) fid = fopen(fname,'rb'); string = fread(fid,inf,'uint8=>char')'; fclose(fid); else error('input file does not exist'); end pos = 1; len = length(string); inStr = string; isoct=exist('OCTAVE_VERSION','builtin'); arraytoken=find(inStr=='[' | inStr==']' | inStr=='"'); jstr=regexprep(inStr,'\\\\',' '); escquote=regexp(jstr,'\\"'); arraytoken=sort([arraytoken escquote]); % String delimiters and escape chars identified to improve speed: esc = find(inStr=='"' | inStr=='\' ); % comparable to: regexp(inStr, '["\\]'); index_esc = 1; len_esc = length(esc); opt=varargin2struct(varargin{:}); if(jsonopt('ShowProgress',0,opt)==1) opt.progressbar_=waitbar(0,'loading ...'); end jsoncount=1; while pos <= len switch(next_char) case '{' data{jsoncount} = parse_object(opt); case '[' data{jsoncount} = parse_array(opt); otherwise error_pos('Outer level structure must be an object or an array'); end jsoncount=jsoncount+1; end % while jsoncount=length(data); if(jsoncount==1 && iscell(data)) data=data{1}; end if(~isempty(data)) if(isstruct(data)) % data can be a struct array data=jstruct2array(data); elseif(iscell(data)) data=jcell2array(data); end end if(isfield(opt,'progressbar_')) close(opt.progressbar_); end %% function newdata=jcell2array(data) len=length(data); newdata=data; for i=1:len if(isstruct(data{i})) newdata{i}=jstruct2array(data{i}); elseif(iscell(data{i})) newdata{i}=jcell2array(data{i}); end end %%------------------------------------------------------------------------- function newdata=jstruct2array(data) fn=fieldnames(data); newdata=data; len=length(data); for i=1:length(fn) % depth-first for j=1:len if(isstruct(getfield(data(j),fn{i}))) newdata(j)=setfield(newdata(j),fn{i},jstruct2array(getfield(data(j),fn{i}))); end end end if(~isempty(strmatch('x0x5F_ArrayType_',fn)) && ~isempty(strmatch('x0x5F_ArrayData_',fn))) newdata=cell(len,1); for j=1:len ndata=cast(data(j).x0x5F_ArrayData_,data(j).x0x5F_ArrayType_); iscpx=0; if(~isempty(strmatch('x0x5F_ArrayIsComplex_',fn))) if(data(j).x0x5F_ArrayIsComplex_) iscpx=1; end end if(~isempty(strmatch('x0x5F_ArrayIsSparse_',fn))) if(data(j).x0x5F_ArrayIsSparse_) if(~isempty(strmatch('x0x5F_ArraySize_',fn))) dim=data(j).x0x5F_ArraySize_; if(iscpx && size(ndata,2)==4-any(dim==1)) ndata(:,end-1)=complex(ndata(:,end-1),ndata(:,end)); end if isempty(ndata) % All-zeros sparse ndata=sparse(dim(1),prod(dim(2:end))); elseif dim(1)==1 % Sparse row vector ndata=sparse(1,ndata(:,1),ndata(:,2),dim(1),prod(dim(2:end))); elseif dim(2)==1 % Sparse column vector ndata=sparse(ndata(:,1),1,ndata(:,2),dim(1),prod(dim(2:end))); else % Generic sparse array. ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3),dim(1),prod(dim(2:end))); end else if(iscpx && size(ndata,2)==4) ndata(:,3)=complex(ndata(:,3),ndata(:,4)); end ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3)); end end elseif(~isempty(strmatch('x0x5F_ArraySize_',fn))) if(iscpx && size(ndata,2)==2) ndata=complex(ndata(:,1),ndata(:,2)); end ndata=reshape(ndata(:),data(j).x0x5F_ArraySize_); end newdata{j}=ndata; end if(len==1) newdata=newdata{1}; end end %%------------------------------------------------------------------------- function object = parse_object(varargin) parse_char('{'); object = []; if next_char ~= '}' while 1 str = parseStr(varargin{:}); if isempty(str) error_pos('Name of value at position %d cannot be empty'); end parse_char(':'); val = parse_value(varargin{:}); eval( sprintf( 'object.%s = val;', valid_field(str) ) ); if next_char == '}' break; end parse_char(','); end end parse_char('}'); %%------------------------------------------------------------------------- function object = parse_array(varargin) % JSON array is written in row-major order global pos inStr isoct parse_char('['); object = cell(0, 1); dim2=[]; arraydepth=jsonopt('JSONLAB_ArrayDepth_',1,varargin{:}); pbar=jsonopt('progressbar_',-1,varargin{:}); if next_char ~= ']' if(jsonopt('FastArrayParser',1,varargin{:})>=1 && arraydepth>=jsonopt('FastArrayParser',1,varargin{:})) [endpos, e1l, e1r, maxlevel]=matching_bracket(inStr,pos); arraystr=['[' inStr(pos:endpos)]; arraystr=regexprep(arraystr,'"_NaN_"','NaN'); arraystr=regexprep(arraystr,'"([-+]*)_Inf_"','$1Inf'); arraystr(arraystr==sprintf('\n'))=[]; arraystr(arraystr==sprintf('\r'))=[]; %arraystr=regexprep(arraystr,'\s*,',','); % this is slow,sometimes needed if(~isempty(e1l) && ~isempty(e1r)) % the array is in 2D or higher D astr=inStr((e1l+1):(e1r-1)); astr=regexprep(astr,'"_NaN_"','NaN'); astr=regexprep(astr,'"([-+]*)_Inf_"','$1Inf'); astr(astr==sprintf('\n'))=[]; astr(astr==sprintf('\r'))=[]; astr(astr==' ')=''; if(isempty(find(astr=='[', 1))) % array is 2D dim2=length(sscanf(astr,'%f,',[1 inf])); end else % array is 1D astr=arraystr(2:end-1); astr(astr==' ')=''; [obj, count, errmsg, nextidx]=sscanf(astr,'%f,',[1,inf]); if(nextidx>=length(astr)-1) object=obj; pos=endpos; parse_char(']'); return; end end if(~isempty(dim2)) astr=arraystr; astr(astr=='[')=''; astr(astr==']')=''; astr(astr==' ')=''; [obj, count, errmsg, nextidx]=sscanf(astr,'%f,',inf); if(nextidx>=length(astr)-1) object=reshape(obj,dim2,numel(obj)/dim2)'; pos=endpos; parse_char(']'); if(pbar>0) waitbar(pos/length(inStr),pbar,'loading ...'); end return; end end arraystr=regexprep(arraystr,'\]\s*,','];'); else arraystr='['; end try if(isoct && regexp(arraystr,'"','once')) error('Octave eval can produce empty cells for JSON-like input'); end object=eval(arraystr); pos=endpos; catch while 1 newopt=varargin2struct(varargin{:},'JSONLAB_ArrayDepth_',arraydepth+1); val = parse_value(newopt); object{end+1} = val; if next_char == ']' break; end parse_char(','); end end end if(jsonopt('SimplifyCell',0,varargin{:})==1) try oldobj=object; object=cell2mat(object')'; if(iscell(oldobj) && isstruct(object) && numel(object)>1 && jsonopt('SimplifyCellArray',1,varargin{:})==0) object=oldobj; elseif(size(object,1)>1 && ndims(object)==2) object=object'; end catch end end parse_char(']'); if(pbar>0) waitbar(pos/length(inStr),pbar,'loading ...'); end %%------------------------------------------------------------------------- function parse_char(c) global pos inStr len skip_whitespace; if pos > len || inStr(pos) ~= c error_pos(sprintf('Expected %c at position %%d', c)); else pos = pos + 1; skip_whitespace; end %%------------------------------------------------------------------------- function c = next_char global pos inStr len skip_whitespace; if pos > len c = []; else c = inStr(pos); end %%------------------------------------------------------------------------- function skip_whitespace global pos inStr len while pos <= len && isspace(inStr(pos)) pos = pos + 1; end %%------------------------------------------------------------------------- function str = parseStr(varargin) global pos inStr len esc index_esc len_esc % len, ns = length(inStr), keyboard if inStr(pos) ~= '"' error_pos('String starting with " expected at position %d'); else pos = pos + 1; end str = ''; while pos <= len while index_esc <= len_esc && esc(index_esc) < pos index_esc = index_esc + 1; end if index_esc > len_esc str = [str inStr(pos:len)]; pos = len + 1; break; else str = [str inStr(pos:esc(index_esc)-1)]; pos = esc(index_esc); end nstr = length(str); switch inStr(pos) case '"' pos = pos + 1; if(~isempty(str)) if(strcmp(str,'_Inf_')) str=Inf; elseif(strcmp(str,'-_Inf_')) str=-Inf; elseif(strcmp(str,'_NaN_')) str=NaN; end end return; case '\' if pos+1 > len error_pos('End of file reached right after escape character'); end pos = pos + 1; switch inStr(pos) case {'"' '\' '/'} str(nstr+1) = inStr(pos); pos = pos + 1; case {'b' 'f' 'n' 'r' 't'} str(nstr+1) = sprintf(['\' inStr(pos)]); pos = pos + 1; case 'u' if pos+4 > len error_pos('End of file reached in escaped unicode character'); end str(nstr+(1:6)) = inStr(pos-1:pos+4); pos = pos + 5; end otherwise % should never happen str(nstr+1) = inStr(pos), keyboard pos = pos + 1; end end error_pos('End of file while expecting end of inStr'); %%------------------------------------------------------------------------- function num = parse_number(varargin) global pos inStr len isoct currstr=inStr(pos:end); numstr=0; if(isoct~=0) numstr=regexp(currstr,'^\s*-?(?:0|[1-9]\d*)(?:\.\d+)?(?:[eE][+\-]?\d+)?','end'); [num, one] = sscanf(currstr, '%f', 1); delta=numstr+1; else [num, one, err, delta] = sscanf(currstr, '%f', 1); if ~isempty(err) error_pos('Error reading number at position %d'); end end pos = pos + delta-1; %%------------------------------------------------------------------------- function val = parse_value(varargin) global pos inStr len true = 1; false = 0; pbar=jsonopt('progressbar_',-1,varargin{:}); if(pbar>0) waitbar(pos/len,pbar,'loading ...'); end switch(inStr(pos)) case '"' val = parseStr(varargin{:}); return; case '[' val = parse_array(varargin{:}); return; case '{' val = parse_object(varargin{:}); if isstruct(val) if(~isempty(strmatch('x0x5F_ArrayType_',fieldnames(val), 'exact'))) val=jstruct2array(val); end elseif isempty(val) val = struct; end return; case {'-','0','1','2','3','4','5','6','7','8','9'} val = parse_number(varargin{:}); return; case 't' if pos+3 <= len && strcmpi(inStr(pos:pos+3), 'true') val = true; pos = pos + 4; return; end case 'f' if pos+4 <= len && strcmpi(inStr(pos:pos+4), 'false') val = false; pos = pos + 5; return; end case 'n' if pos+3 <= len && strcmpi(inStr(pos:pos+3), 'null') val = []; pos = pos + 4; return; end end error_pos('Value expected at position %d'); %%------------------------------------------------------------------------- function error_pos(msg) global pos inStr len poShow = max(min([pos-15 pos-1 pos pos+20],len),1); if poShow(3) == poShow(2) poShow(3:4) = poShow(2)+[0 -1]; % display nothing after end msg = [sprintf(msg, pos) ': ' ... inStr(poShow(1):poShow(2)) '' inStr(poShow(3):poShow(4)) ]; error( ['JSONparser:invalidFormat: ' msg] ); %%------------------------------------------------------------------------- function str = valid_field(str) global isoct % From MATLAB doc: field names must begin with a letter, which may be % followed by any combination of letters, digits, and underscores. % Invalid characters will be converted to underscores, and the prefix % "x0x[Hex code]_" will be added if the first character is not a letter. pos=regexp(str,'^[^A-Za-z]','once'); if(~isempty(pos)) if(~isoct) str=regexprep(str,'^([^A-Za-z])','x0x${sprintf(''%X'',unicode2native($1))}_','once'); else str=sprintf('x0x%X_%s',char(str(1)),str(2:end)); end end if(isempty(regexp(str,'[^0-9A-Za-z_]', 'once' ))) return; end if(~isoct) str=regexprep(str,'([^0-9A-Za-z_])','_0x${sprintf(''%X'',unicode2native($1))}_'); else pos=regexp(str,'[^0-9A-Za-z_]'); if(isempty(pos)) return; end str0=str; pos0=[0 pos(:)' length(str)]; str=''; for i=1:length(pos) str=[str str0(pos0(i)+1:pos(i)-1) sprintf('_0x%X_',str0(pos(i)))]; end if(pos(end)~=length(str)) str=[str str0(pos0(end-1)+1:pos0(end))]; end end %str(~isletter(str) & ~('0' <= str & str <= '9')) = '_'; %%------------------------------------------------------------------------- function endpos = matching_quote(str,pos) len=length(str); while(pos1 && str(pos-1)=='\')) endpos=pos; return; end end pos=pos+1; end error('unmatched quotation mark'); %%------------------------------------------------------------------------- function [endpos, e1l, e1r, maxlevel] = matching_bracket(str,pos) global arraytoken level=1; maxlevel=level; endpos=0; bpos=arraytoken(arraytoken>=pos); tokens=str(bpos); len=length(tokens); pos=1; e1l=[]; e1r=[]; while(pos<=len) c=tokens(pos); if(c==']') level=level-1; if(isempty(e1r)) e1r=bpos(pos); end if(level==0) endpos=bpos(pos); return end end if(c=='[') if(isempty(e1l)) e1l=bpos(pos); end level=level+1; maxlevel=max(maxlevel,level); end if(c=='"') pos=matching_quote(tokens,pos+1); end pos=pos+1; end if(endpos==0) error('unmatched "]"'); end ================================================ FILE: machine-learning-ex4/ex4/lib/jsonlab/loadubjson.m ================================================ function data = loadubjson(fname,varargin) % % data=loadubjson(fname,opt) % or % data=loadubjson(fname,'param1',value1,'param2',value2,...) % % parse a JSON (JavaScript Object Notation) file or string % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2013/08/01 % % $Id: loadubjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % fname: input file name, if fname contains "{}" or "[]", fname % will be interpreted as a UBJSON string % opt: a struct to store parsing options, opt can be replaced by % a list of ('param',value) pairs - the param string is equivallent % to a field in opt. opt can have the following % fields (first in [.|.] is the default) % % opt.SimplifyCell [0|1]: if set to 1, loadubjson will call cell2mat % for each element of the JSON data, and group % arrays based on the cell2mat rules. % opt.IntEndian [B|L]: specify the endianness of the integer fields % in the UBJSON input data. B - Big-Endian format for % integers (as required in the UBJSON specification); % L - input integer fields are in Little-Endian order. % % output: % dat: a cell array, where {...} blocks are converted into cell arrays, % and [...] are converted to arrays % % examples: % obj=struct('string','value','array',[1 2 3]); % ubjdata=saveubjson('obj',obj); % dat=loadubjson(ubjdata) % dat=loadubjson(['examples' filesep 'example1.ubj']) % dat=loadubjson(['examples' filesep 'example1.ubj'],'SimplifyCell',1) % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % global pos inStr len esc index_esc len_esc isoct arraytoken fileendian systemendian if(regexp(fname,'[\{\}\]\[]','once')) string=fname; elseif(exist(fname,'file')) fid = fopen(fname,'rb'); string = fread(fid,inf,'uint8=>char')'; fclose(fid); else error('input file does not exist'); end pos = 1; len = length(string); inStr = string; isoct=exist('OCTAVE_VERSION','builtin'); arraytoken=find(inStr=='[' | inStr==']' | inStr=='"'); jstr=regexprep(inStr,'\\\\',' '); escquote=regexp(jstr,'\\"'); arraytoken=sort([arraytoken escquote]); % String delimiters and escape chars identified to improve speed: esc = find(inStr=='"' | inStr=='\' ); % comparable to: regexp(inStr, '["\\]'); index_esc = 1; len_esc = length(esc); opt=varargin2struct(varargin{:}); fileendian=upper(jsonopt('IntEndian','B',opt)); [os,maxelem,systemendian]=computer; jsoncount=1; while pos <= len switch(next_char) case '{' data{jsoncount} = parse_object(opt); case '[' data{jsoncount} = parse_array(opt); otherwise error_pos('Outer level structure must be an object or an array'); end jsoncount=jsoncount+1; end % while jsoncount=length(data); if(jsoncount==1 && iscell(data)) data=data{1}; end if(~isempty(data)) if(isstruct(data)) % data can be a struct array data=jstruct2array(data); elseif(iscell(data)) data=jcell2array(data); end end %% function newdata=parse_collection(id,data,obj) if(jsoncount>0 && exist('data','var')) if(~iscell(data)) newdata=cell(1); newdata{1}=data; data=newdata; end end %% function newdata=jcell2array(data) len=length(data); newdata=data; for i=1:len if(isstruct(data{i})) newdata{i}=jstruct2array(data{i}); elseif(iscell(data{i})) newdata{i}=jcell2array(data{i}); end end %%------------------------------------------------------------------------- function newdata=jstruct2array(data) fn=fieldnames(data); newdata=data; len=length(data); for i=1:length(fn) % depth-first for j=1:len if(isstruct(getfield(data(j),fn{i}))) newdata(j)=setfield(newdata(j),fn{i},jstruct2array(getfield(data(j),fn{i}))); end end end if(~isempty(strmatch('x0x5F_ArrayType_',fn)) && ~isempty(strmatch('x0x5F_ArrayData_',fn))) newdata=cell(len,1); for j=1:len ndata=cast(data(j).x0x5F_ArrayData_,data(j).x0x5F_ArrayType_); iscpx=0; if(~isempty(strmatch('x0x5F_ArrayIsComplex_',fn))) if(data(j).x0x5F_ArrayIsComplex_) iscpx=1; end end if(~isempty(strmatch('x0x5F_ArrayIsSparse_',fn))) if(data(j).x0x5F_ArrayIsSparse_) if(~isempty(strmatch('x0x5F_ArraySize_',fn))) dim=double(data(j).x0x5F_ArraySize_); if(iscpx && size(ndata,2)==4-any(dim==1)) ndata(:,end-1)=complex(ndata(:,end-1),ndata(:,end)); end if isempty(ndata) % All-zeros sparse ndata=sparse(dim(1),prod(dim(2:end))); elseif dim(1)==1 % Sparse row vector ndata=sparse(1,ndata(:,1),ndata(:,2),dim(1),prod(dim(2:end))); elseif dim(2)==1 % Sparse column vector ndata=sparse(ndata(:,1),1,ndata(:,2),dim(1),prod(dim(2:end))); else % Generic sparse array. ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3),dim(1),prod(dim(2:end))); end else if(iscpx && size(ndata,2)==4) ndata(:,3)=complex(ndata(:,3),ndata(:,4)); end ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3)); end end elseif(~isempty(strmatch('x0x5F_ArraySize_',fn))) if(iscpx && size(ndata,2)==2) ndata=complex(ndata(:,1),ndata(:,2)); end ndata=reshape(ndata(:),data(j).x0x5F_ArraySize_); end newdata{j}=ndata; end if(len==1) newdata=newdata{1}; end end %%------------------------------------------------------------------------- function object = parse_object(varargin) parse_char('{'); object = []; type=''; count=-1; if(next_char == '$') type=inStr(pos+1); % TODO pos=pos+2; end if(next_char == '#') pos=pos+1; count=double(parse_number()); end if next_char ~= '}' num=0; while 1 str = parseStr(varargin{:}); if isempty(str) error_pos('Name of value at position %d cannot be empty'); end %parse_char(':'); val = parse_value(varargin{:}); num=num+1; eval( sprintf( 'object.%s = val;', valid_field(str) ) ); if next_char == '}' || (count>=0 && num>=count) break; end %parse_char(','); end end if(count==-1) parse_char('}'); end %%------------------------------------------------------------------------- function [cid,len]=elem_info(type) id=strfind('iUIlLdD',type); dataclass={'int8','uint8','int16','int32','int64','single','double'}; bytelen=[1,1,2,4,8,4,8]; if(id>0) cid=dataclass{id}; len=bytelen(id); else error_pos('unsupported type at position %d'); end %%------------------------------------------------------------------------- function [data adv]=parse_block(type,count,varargin) global pos inStr isoct fileendian systemendian [cid,len]=elem_info(type); datastr=inStr(pos:pos+len*count-1); if(isoct) newdata=int8(datastr); else newdata=uint8(datastr); end id=strfind('iUIlLdD',type); if(id<=5 && fileendian~=systemendian) newdata=swapbytes(typecast(newdata,cid)); end data=typecast(newdata,cid); adv=double(len*count); %%------------------------------------------------------------------------- function object = parse_array(varargin) % JSON array is written in row-major order global pos inStr isoct parse_char('['); object = cell(0, 1); dim=[]; type=''; count=-1; if(next_char == '$') type=inStr(pos+1); pos=pos+2; end if(next_char == '#') pos=pos+1; if(next_char=='[') dim=parse_array(varargin{:}); count=prod(double(dim)); else count=double(parse_number()); end end if(~isempty(type)) if(count>=0) [object adv]=parse_block(type,count,varargin{:}); if(~isempty(dim)) object=reshape(object,dim); end pos=pos+adv; return; else endpos=matching_bracket(inStr,pos); [cid,len]=elem_info(type); count=(endpos-pos)/len; [object adv]=parse_block(type,count,varargin{:}); pos=pos+adv; parse_char(']'); return; end end if next_char ~= ']' while 1 val = parse_value(varargin{:}); object{end+1} = val; if next_char == ']' break; end %parse_char(','); end end if(jsonopt('SimplifyCell',0,varargin{:})==1) try oldobj=object; object=cell2mat(object')'; if(iscell(oldobj) && isstruct(object) && numel(object)>1 && jsonopt('SimplifyCellArray',1,varargin{:})==0) object=oldobj; elseif(size(object,1)>1 && ndims(object)==2) object=object'; end catch end end if(count==-1) parse_char(']'); end %%------------------------------------------------------------------------- function parse_char(c) global pos inStr len skip_whitespace; if pos > len || inStr(pos) ~= c error_pos(sprintf('Expected %c at position %%d', c)); else pos = pos + 1; skip_whitespace; end %%------------------------------------------------------------------------- function c = next_char global pos inStr len skip_whitespace; if pos > len c = []; else c = inStr(pos); end %%------------------------------------------------------------------------- function skip_whitespace global pos inStr len while pos <= len && isspace(inStr(pos)) pos = pos + 1; end %%------------------------------------------------------------------------- function str = parseStr(varargin) global pos inStr esc index_esc len_esc % len, ns = length(inStr), keyboard type=inStr(pos); if type ~= 'S' && type ~= 'C' && type ~= 'H' error_pos('String starting with S expected at position %d'); else pos = pos + 1; end if(type == 'C') str=inStr(pos); pos=pos+1; return; end bytelen=double(parse_number()); if(length(inStr)>=pos+bytelen-1) str=inStr(pos:pos+bytelen-1); pos=pos+bytelen; else error_pos('End of file while expecting end of inStr'); end %%------------------------------------------------------------------------- function num = parse_number(varargin) global pos inStr len isoct fileendian systemendian id=strfind('iUIlLdD',inStr(pos)); if(isempty(id)) error_pos('expecting a number at position %d'); end type={'int8','uint8','int16','int32','int64','single','double'}; bytelen=[1,1,2,4,8,4,8]; datastr=inStr(pos+1:pos+bytelen(id)); if(isoct) newdata=int8(datastr); else newdata=uint8(datastr); end if(id<=5 && fileendian~=systemendian) newdata=swapbytes(typecast(newdata,type{id})); end num=typecast(newdata,type{id}); pos = pos + bytelen(id)+1; %%------------------------------------------------------------------------- function val = parse_value(varargin) global pos inStr len true = 1; false = 0; switch(inStr(pos)) case {'S','C','H'} val = parseStr(varargin{:}); return; case '[' val = parse_array(varargin{:}); return; case '{' val = parse_object(varargin{:}); if isstruct(val) if(~isempty(strmatch('x0x5F_ArrayType_',fieldnames(val), 'exact'))) val=jstruct2array(val); end elseif isempty(val) val = struct; end return; case {'i','U','I','l','L','d','D'} val = parse_number(varargin{:}); return; case 'T' val = true; pos = pos + 1; return; case 'F' val = false; pos = pos + 1; return; case {'Z','N'} val = []; pos = pos + 1; return; end error_pos('Value expected at position %d'); %%------------------------------------------------------------------------- function error_pos(msg) global pos inStr len poShow = max(min([pos-15 pos-1 pos pos+20],len),1); if poShow(3) == poShow(2) poShow(3:4) = poShow(2)+[0 -1]; % display nothing after end msg = [sprintf(msg, pos) ': ' ... inStr(poShow(1):poShow(2)) '' inStr(poShow(3):poShow(4)) ]; error( ['JSONparser:invalidFormat: ' msg] ); %%------------------------------------------------------------------------- function str = valid_field(str) global isoct % From MATLAB doc: field names must begin with a letter, which may be % followed by any combination of letters, digits, and underscores. % Invalid characters will be converted to underscores, and the prefix % "x0x[Hex code]_" will be added if the first character is not a letter. pos=regexp(str,'^[^A-Za-z]','once'); if(~isempty(pos)) if(~isoct) str=regexprep(str,'^([^A-Za-z])','x0x${sprintf(''%X'',unicode2native($1))}_','once'); else str=sprintf('x0x%X_%s',char(str(1)),str(2:end)); end end if(isempty(regexp(str,'[^0-9A-Za-z_]', 'once' ))) return; end if(~isoct) str=regexprep(str,'([^0-9A-Za-z_])','_0x${sprintf(''%X'',unicode2native($1))}_'); else pos=regexp(str,'[^0-9A-Za-z_]'); if(isempty(pos)) return; end str0=str; pos0=[0 pos(:)' length(str)]; str=''; for i=1:length(pos) str=[str str0(pos0(i)+1:pos(i)-1) sprintf('_0x%X_',str0(pos(i)))]; end if(pos(end)~=length(str)) str=[str str0(pos0(end-1)+1:pos0(end))]; end end %str(~isletter(str) & ~('0' <= str & str <= '9')) = '_'; %%------------------------------------------------------------------------- function endpos = matching_quote(str,pos) len=length(str); while(pos1 && str(pos-1)=='\')) endpos=pos; return; end end pos=pos+1; end error('unmatched quotation mark'); %%------------------------------------------------------------------------- function [endpos e1l e1r maxlevel] = matching_bracket(str,pos) global arraytoken level=1; maxlevel=level; endpos=0; bpos=arraytoken(arraytoken>=pos); tokens=str(bpos); len=length(tokens); pos=1; e1l=[]; e1r=[]; while(pos<=len) c=tokens(pos); if(c==']') level=level-1; if(isempty(e1r)) e1r=bpos(pos); end if(level==0) endpos=bpos(pos); return end end if(c=='[') if(isempty(e1l)) e1l=bpos(pos); end level=level+1; maxlevel=max(maxlevel,level); end if(c=='"') pos=matching_quote(tokens,pos+1); end pos=pos+1; end if(endpos==0) error('unmatched "]"'); end ================================================ FILE: machine-learning-ex4/ex4/lib/jsonlab/mergestruct.m ================================================ function s=mergestruct(s1,s2) % % s=mergestruct(s1,s2) % % merge two struct objects into one % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % date: 2012/12/22 % % input: % s1,s2: a struct object, s1 and s2 can not be arrays % % output: % s: the merged struct object. fields in s1 and s2 will be combined in s. % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(~isstruct(s1) || ~isstruct(s2)) error('input parameters contain non-struct'); end if(length(s1)>1 || length(s2)>1) error('can not merge struct arrays'); end fn=fieldnames(s2); s=s1; for i=1:length(fn) s=setfield(s,fn{i},getfield(s2,fn{i})); end ================================================ FILE: machine-learning-ex4/ex4/lib/jsonlab/savejson.m ================================================ function json=savejson(rootname,obj,varargin) % % json=savejson(rootname,obj,filename) % or % json=savejson(rootname,obj,opt) % json=savejson(rootname,obj,'param1',value1,'param2',value2,...) % % convert a MATLAB object (cell, struct or array) into a JSON (JavaScript % Object Notation) string % % author: Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2011/09/09 % % $Id: savejson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % rootname: the name of the root-object, when set to '', the root name % is ignored, however, when opt.ForceRootName is set to 1 (see below), % the MATLAB variable name will be used as the root name. % obj: a MATLAB object (array, cell, cell array, struct, struct array). % filename: a string for the file name to save the output JSON data. % opt: a struct for additional options, ignore to use default values. % opt can have the following fields (first in [.|.] is the default) % % opt.FileName [''|string]: a file name to save the output JSON data % opt.FloatFormat ['%.10g'|string]: format to show each numeric element % of a 1D/2D array; % opt.ArrayIndent [1|0]: if 1, output explicit data array with % precedent indentation; if 0, no indentation % opt.ArrayToStruct[0|1]: when set to 0, savejson outputs 1D/2D % array in JSON array format; if sets to 1, an % array will be shown as a struct with fields % "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for % sparse arrays, the non-zero elements will be % saved to _ArrayData_ field in triplet-format i.e. % (ix,iy,val) and "_ArrayIsSparse_" will be added % with a value of 1; for a complex array, the % _ArrayData_ array will include two columns % (4 for sparse) to record the real and imaginary % parts, and also "_ArrayIsComplex_":1 is added. % opt.ParseLogical [0|1]: if this is set to 1, logical array elem % will use true/false rather than 1/0. % opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single % numerical element will be shown without a square % bracket, unless it is the root object; if 0, square % brackets are forced for any numerical arrays. % opt.ForceRootName [0|1]: when set to 1 and rootname is empty, savejson % will use the name of the passed obj variable as the % root object name; if obj is an expression and % does not have a name, 'root' will be used; if this % is set to 0 and rootname is empty, the root level % will be merged down to the lower level. % opt.Inf ['"$1_Inf_"'|string]: a customized regular expression pattern % to represent +/-Inf. The matched pattern is '([-+]*)Inf' % and $1 represents the sign. For those who want to use % 1e999 to represent Inf, they can set opt.Inf to '$11e999' % opt.NaN ['"_NaN_"'|string]: a customized regular expression pattern % to represent NaN % opt.JSONP [''|string]: to generate a JSONP output (JSON with padding), % for example, if opt.JSONP='foo', the JSON data is % wrapped inside a function call as 'foo(...);' % opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson % back to the string form % opt.SaveBinary [0|1]: 1 - save the JSON file in binary mode; 0 - text mode. % opt.Compact [0|1]: 1- out compact JSON format (remove all newlines and tabs) % % opt can be replaced by a list of ('param',value) pairs. The param % string is equivallent to a field in opt and is case sensitive. % output: % json: a string in the JSON format (see http://json.org) % % examples: % jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... % 'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],... % 'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;... % 2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],... % 'MeshCreator','FangQ','MeshTitle','T6 Cube',... % 'SpecialData',[nan, inf, -inf]); % savejson('jmesh',jsonmesh) % savejson('',jsonmesh,'ArrayIndent',0,'FloatFormat','\t%.5g') % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(nargin==1) varname=inputname(1); obj=rootname; if(isempty(varname)) varname='root'; end rootname=varname; else varname=inputname(2); end if(length(varargin)==1 && ischar(varargin{1})) opt=struct('FileName',varargin{1}); else opt=varargin2struct(varargin{:}); end opt.IsOctave=exist('OCTAVE_VERSION','builtin'); rootisarray=0; rootlevel=1; forceroot=jsonopt('ForceRootName',0,opt); if((isnumeric(obj) || islogical(obj) || ischar(obj) || isstruct(obj) || iscell(obj)) && isempty(rootname) && forceroot==0) rootisarray=1; rootlevel=0; else if(isempty(rootname)) rootname=varname; end end if((isstruct(obj) || iscell(obj))&& isempty(rootname) && forceroot) rootname='root'; end whitespaces=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); if(jsonopt('Compact',0,opt)==1) whitespaces=struct('tab','','newline','','sep',','); end if(~isfield(opt,'whitespaces_')) opt.whitespaces_=whitespaces; end nl=whitespaces.newline; json=obj2json(rootname,obj,rootlevel,opt); if(rootisarray) json=sprintf('%s%s',json,nl); else json=sprintf('{%s%s%s}\n',nl,json,nl); end jsonp=jsonopt('JSONP','',opt); if(~isempty(jsonp)) json=sprintf('%s(%s);%s',jsonp,json,nl); end % save to a file if FileName is set, suggested by Patrick Rapin if(~isempty(jsonopt('FileName','',opt))) if(jsonopt('SaveBinary',0,opt)==1) fid = fopen(opt.FileName, 'wb'); fwrite(fid,json); else fid = fopen(opt.FileName, 'wt'); fwrite(fid,json,'char'); end fclose(fid); end %%------------------------------------------------------------------------- function txt=obj2json(name,item,level,varargin) if(iscell(item)) txt=cell2json(name,item,level,varargin{:}); elseif(isstruct(item)) txt=struct2json(name,item,level,varargin{:}); elseif(ischar(item)) txt=str2json(name,item,level,varargin{:}); else txt=mat2json(name,item,level,varargin{:}); end %%------------------------------------------------------------------------- function txt=cell2json(name,item,level,varargin) txt=''; if(~iscell(item)) error('input is not a cell'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); ws=jsonopt('whitespaces_',struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')),varargin{:}); padding0=repmat(ws.tab,1,level); padding2=repmat(ws.tab,1,level+1); nl=ws.newline; if(len>1) if(~isempty(name)) txt=sprintf('%s"%s": [%s',padding0, checkname(name,varargin{:}),nl); name=''; else txt=sprintf('%s[%s',padding0,nl); end elseif(len==0) if(~isempty(name)) txt=sprintf('%s"%s": []',padding0, checkname(name,varargin{:})); name=''; else txt=sprintf('%s[]',padding0); end end for j=1:dim(2) if(dim(1)>1) txt=sprintf('%s%s[%s',txt,padding2,nl); end for i=1:dim(1) txt=sprintf('%s%s',txt,obj2json(name,item{i,j},level+(dim(1)>1)+1,varargin{:})); if(i1) txt=sprintf('%s%s%s]',txt,nl,padding2); end if(j1) txt=sprintf('%s%s%s]',txt,nl,padding0); end %%------------------------------------------------------------------------- function txt=struct2json(name,item,level,varargin) txt=''; if(~isstruct(item)) error('input is not a struct'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); ws=struct('tab',sprintf('\t'),'newline',sprintf('\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding0=repmat(ws.tab,1,level); padding2=repmat(ws.tab,1,level+1); padding1=repmat(ws.tab,1,level+(dim(1)>1)+(len>1)); nl=ws.newline; if(~isempty(name)) if(len>1) txt=sprintf('%s"%s": [%s',padding0,checkname(name,varargin{:}),nl); end else if(len>1) txt=sprintf('%s[%s',padding0,nl); end end for j=1:dim(2) if(dim(1)>1) txt=sprintf('%s%s[%s',txt,padding2,nl); end for i=1:dim(1) names = fieldnames(item(i,j)); if(~isempty(name) && len==1) txt=sprintf('%s%s"%s": {%s',txt,padding1, checkname(name,varargin{:}),nl); else txt=sprintf('%s%s{%s',txt,padding1,nl); end if(~isempty(names)) for e=1:length(names) txt=sprintf('%s%s',txt,obj2json(names{e},getfield(item(i,j),... names{e}),level+(dim(1)>1)+1+(len>1),varargin{:})); if(e1) txt=sprintf('%s%s%s]',txt,nl,padding2); end if(j1) txt=sprintf('%s%s%s]',txt,nl,padding0); end %%------------------------------------------------------------------------- function txt=str2json(name,item,level,varargin) txt=''; if(~ischar(item)) error('input is not a string'); end item=reshape(item, max(size(item),[1 0])); len=size(item,1); ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding1=repmat(ws.tab,1,level); padding0=repmat(ws.tab,1,level+1); nl=ws.newline; sep=ws.sep; if(~isempty(name)) if(len>1) txt=sprintf('%s"%s": [%s',padding1,checkname(name,varargin{:}),nl); end else if(len>1) txt=sprintf('%s[%s',padding1,nl); end end isoct=jsonopt('IsOctave',0,varargin{:}); for e=1:len if(isoct) val=regexprep(item(e,:),'\\','\\'); val=regexprep(val,'"','\"'); val=regexprep(val,'^"','\"'); else val=regexprep(item(e,:),'\\','\\\\'); val=regexprep(val,'"','\\"'); val=regexprep(val,'^"','\\"'); end val=escapejsonstring(val); if(len==1) obj=['"' checkname(name,varargin{:}) '": ' '"',val,'"']; if(isempty(name)) obj=['"',val,'"']; end txt=sprintf('%s%s%s%s',txt,padding1,obj); else txt=sprintf('%s%s%s%s',txt,padding0,['"',val,'"']); end if(e==len) sep=''; end txt=sprintf('%s%s',txt,sep); end if(len>1) txt=sprintf('%s%s%s%s',txt,nl,padding1,']'); end %%------------------------------------------------------------------------- function txt=mat2json(name,item,level,varargin) if(~isnumeric(item) && ~islogical(item)) error('input is not an array'); end ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding1=repmat(ws.tab,1,level); padding0=repmat(ws.tab,1,level+1); nl=ws.newline; sep=ws.sep; if(length(size(item))>2 || issparse(item) || ~isreal(item) || ... isempty(item) ||jsonopt('ArrayToStruct',0,varargin{:})) if(isempty(name)) txt=sprintf('%s{%s%s"_ArrayType_": "%s",%s%s"_ArraySize_": %s,%s',... padding1,nl,padding0,class(item),nl,padding0,regexprep(mat2str(size(item)),'\s+',','),nl); else txt=sprintf('%s"%s": {%s%s"_ArrayType_": "%s",%s%s"_ArraySize_": %s,%s',... padding1,checkname(name,varargin{:}),nl,padding0,class(item),nl,padding0,regexprep(mat2str(size(item)),'\s+',','),nl); end else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1 && level>0) numtxt=regexprep(regexprep(matdata2json(item,level+1,varargin{:}),'^\[',''),']',''); else numtxt=matdata2json(item,level+1,varargin{:}); end if(isempty(name)) txt=sprintf('%s%s',padding1,numtxt); else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1) txt=sprintf('%s"%s": %s',padding1,checkname(name,varargin{:}),numtxt); else txt=sprintf('%s"%s": %s',padding1,checkname(name,varargin{:}),numtxt); end end return; end dataformat='%s%s%s%s%s'; if(issparse(item)) [ix,iy]=find(item); data=full(item(find(item))); if(~isreal(item)) data=[real(data(:)),imag(data(:))]; if(size(item,1)==1) % Kludge to have data's 'transposedness' match item's. % (Necessary for complex row vector handling below.) data=data'; end txt=sprintf(dataformat,txt,padding0,'"_ArrayIsComplex_": ','1', sep); end txt=sprintf(dataformat,txt,padding0,'"_ArrayIsSparse_": ','1', sep); if(size(item,1)==1) % Row vector, store only column indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([iy(:),data'],level+2,varargin{:}), nl); elseif(size(item,2)==1) % Column vector, store only row indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([ix,data],level+2,varargin{:}), nl); else % General case, store row and column indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([ix,iy,data],level+2,varargin{:}), nl); end else if(isreal(item)) txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json(item(:)',level+2,varargin{:}), nl); else txt=sprintf(dataformat,txt,padding0,'"_ArrayIsComplex_": ','1', sep); txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([real(item(:)) imag(item(:))],level+2,varargin{:}), nl); end end txt=sprintf('%s%s%s',txt,padding1,'}'); %%------------------------------------------------------------------------- function txt=matdata2json(mat,level,varargin) ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); tab=ws.tab; nl=ws.newline; if(size(mat,1)==1) pre=''; post=''; level=level-1; else pre=sprintf('[%s',nl); post=sprintf('%s%s]',nl,repmat(tab,1,level-1)); end if(isempty(mat)) txt='null'; return; end floatformat=jsonopt('FloatFormat','%.10g',varargin{:}); %if(numel(mat)>1) formatstr=['[' repmat([floatformat ','],1,size(mat,2)-1) [floatformat sprintf('],%s',nl)]]; %else % formatstr=[repmat([floatformat ','],1,size(mat,2)-1) [floatformat sprintf(',\n')]]; %end if(nargin>=2 && size(mat,1)>1 && jsonopt('ArrayIndent',1,varargin{:})==1) formatstr=[repmat(tab,1,level) formatstr]; end txt=sprintf(formatstr,mat'); txt(end-length(nl):end)=[]; if(islogical(mat) && jsonopt('ParseLogical',0,varargin{:})==1) txt=regexprep(txt,'1','true'); txt=regexprep(txt,'0','false'); end %txt=regexprep(mat2str(mat),'\s+',','); %txt=regexprep(txt,';',sprintf('],\n[')); % if(nargin>=2 && size(mat,1)>1) % txt=regexprep(txt,'\[',[repmat(sprintf('\t'),1,level) '[']); % end txt=[pre txt post]; if(any(isinf(mat(:)))) txt=regexprep(txt,'([-+]*)Inf',jsonopt('Inf','"$1_Inf_"',varargin{:})); end if(any(isnan(mat(:)))) txt=regexprep(txt,'NaN',jsonopt('NaN','"_NaN_"',varargin{:})); end %%------------------------------------------------------------------------- function newname=checkname(name,varargin) isunpack=jsonopt('UnpackHex',1,varargin{:}); newname=name; if(isempty(regexp(name,'0x([0-9a-fA-F]+)_','once'))) return end if(isunpack) isoct=jsonopt('IsOctave',0,varargin{:}); if(~isoct) newname=regexprep(name,'(^x|_){1}0x([0-9a-fA-F]+)_','${native2unicode(hex2dec($2))}'); else pos=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','start'); pend=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','end'); if(isempty(pos)) return; end str0=name; pos0=[0 pend(:)' length(name)]; newname=''; for i=1:length(pos) newname=[newname str0(pos0(i)+1:pos(i)-1) char(hex2dec(str0(pos(i)+3:pend(i)-1)))]; end if(pos(end)~=length(name)) newname=[newname str0(pos0(end-1)+1:pos0(end))]; end end end %%------------------------------------------------------------------------- function newstr=escapejsonstring(str) newstr=str; isoct=exist('OCTAVE_VERSION','builtin'); if(isoct) vv=sscanf(OCTAVE_VERSION,'%f'); if(vv(1)>=3.8) isoct=0; end end if(isoct) escapechars={'\a','\f','\n','\r','\t','\v'}; for i=1:length(escapechars); newstr=regexprep(newstr,escapechars{i},escapechars{i}); end else escapechars={'\a','\b','\f','\n','\r','\t','\v'}; for i=1:length(escapechars); newstr=regexprep(newstr,escapechars{i},regexprep(escapechars{i},'\\','\\\\')); end end ================================================ FILE: machine-learning-ex4/ex4/lib/jsonlab/saveubjson.m ================================================ function json=saveubjson(rootname,obj,varargin) % % json=saveubjson(rootname,obj,filename) % or % json=saveubjson(rootname,obj,opt) % json=saveubjson(rootname,obj,'param1',value1,'param2',value2,...) % % convert a MATLAB object (cell, struct or array) into a Universal % Binary JSON (UBJSON) binary string % % author: Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2013/08/17 % % $Id: saveubjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % rootname: the name of the root-object, when set to '', the root name % is ignored, however, when opt.ForceRootName is set to 1 (see below), % the MATLAB variable name will be used as the root name. % obj: a MATLAB object (array, cell, cell array, struct, struct array) % filename: a string for the file name to save the output UBJSON data % opt: a struct for additional options, ignore to use default values. % opt can have the following fields (first in [.|.] is the default) % % opt.FileName [''|string]: a file name to save the output JSON data % opt.ArrayToStruct[0|1]: when set to 0, saveubjson outputs 1D/2D % array in JSON array format; if sets to 1, an % array will be shown as a struct with fields % "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for % sparse arrays, the non-zero elements will be % saved to _ArrayData_ field in triplet-format i.e. % (ix,iy,val) and "_ArrayIsSparse_" will be added % with a value of 1; for a complex array, the % _ArrayData_ array will include two columns % (4 for sparse) to record the real and imaginary % parts, and also "_ArrayIsComplex_":1 is added. % opt.ParseLogical [1|0]: if this is set to 1, logical array elem % will use true/false rather than 1/0. % opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single % numerical element will be shown without a square % bracket, unless it is the root object; if 0, square % brackets are forced for any numerical arrays. % opt.ForceRootName [0|1]: when set to 1 and rootname is empty, saveubjson % will use the name of the passed obj variable as the % root object name; if obj is an expression and % does not have a name, 'root' will be used; if this % is set to 0 and rootname is empty, the root level % will be merged down to the lower level. % opt.JSONP [''|string]: to generate a JSONP output (JSON with padding), % for example, if opt.JSON='foo', the JSON data is % wrapped inside a function call as 'foo(...);' % opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson % back to the string form % % opt can be replaced by a list of ('param',value) pairs. The param % string is equivallent to a field in opt and is case sensitive. % output: % json: a binary string in the UBJSON format (see http://ubjson.org) % % examples: % jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... % 'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],... % 'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;... % 2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],... % 'MeshCreator','FangQ','MeshTitle','T6 Cube',... % 'SpecialData',[nan, inf, -inf]); % saveubjson('jsonmesh',jsonmesh) % saveubjson('jsonmesh',jsonmesh,'meshdata.ubj') % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(nargin==1) varname=inputname(1); obj=rootname; if(isempty(varname)) varname='root'; end rootname=varname; else varname=inputname(2); end if(length(varargin)==1 && ischar(varargin{1})) opt=struct('FileName',varargin{1}); else opt=varargin2struct(varargin{:}); end opt.IsOctave=exist('OCTAVE_VERSION','builtin'); rootisarray=0; rootlevel=1; forceroot=jsonopt('ForceRootName',0,opt); if((isnumeric(obj) || islogical(obj) || ischar(obj) || isstruct(obj) || iscell(obj)) && isempty(rootname) && forceroot==0) rootisarray=1; rootlevel=0; else if(isempty(rootname)) rootname=varname; end end if((isstruct(obj) || iscell(obj))&& isempty(rootname) && forceroot) rootname='root'; end json=obj2ubjson(rootname,obj,rootlevel,opt); if(~rootisarray) json=['{' json '}']; end jsonp=jsonopt('JSONP','',opt); if(~isempty(jsonp)) json=[jsonp '(' json ')']; end % save to a file if FileName is set, suggested by Patrick Rapin if(~isempty(jsonopt('FileName','',opt))) fid = fopen(opt.FileName, 'wb'); fwrite(fid,json); fclose(fid); end %%------------------------------------------------------------------------- function txt=obj2ubjson(name,item,level,varargin) if(iscell(item)) txt=cell2ubjson(name,item,level,varargin{:}); elseif(isstruct(item)) txt=struct2ubjson(name,item,level,varargin{:}); elseif(ischar(item)) txt=str2ubjson(name,item,level,varargin{:}); else txt=mat2ubjson(name,item,level,varargin{:}); end %%------------------------------------------------------------------------- function txt=cell2ubjson(name,item,level,varargin) txt=''; if(~iscell(item)) error('input is not a cell'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); % let's handle 1D cell first if(len>1) if(~isempty(name)) txt=[S_(checkname(name,varargin{:})) '[']; name=''; else txt='['; end elseif(len==0) if(~isempty(name)) txt=[S_(checkname(name,varargin{:})) 'Z']; name=''; else txt='Z'; end end for j=1:dim(2) if(dim(1)>1) txt=[txt '[']; end for i=1:dim(1) txt=[txt obj2ubjson(name,item{i,j},level+(len>1),varargin{:})]; end if(dim(1)>1) txt=[txt ']']; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=struct2ubjson(name,item,level,varargin) txt=''; if(~isstruct(item)) error('input is not a struct'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); if(~isempty(name)) if(len>1) txt=[S_(checkname(name,varargin{:})) '[']; end else if(len>1) txt='['; end end for j=1:dim(2) if(dim(1)>1) txt=[txt '[']; end for i=1:dim(1) names = fieldnames(item(i,j)); if(~isempty(name) && len==1) txt=[txt S_(checkname(name,varargin{:})) '{']; else txt=[txt '{']; end if(~isempty(names)) for e=1:length(names) txt=[txt obj2ubjson(names{e},getfield(item(i,j),... names{e}),level+(dim(1)>1)+1+(len>1),varargin{:})]; end end txt=[txt '}']; end if(dim(1)>1) txt=[txt ']']; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=str2ubjson(name,item,level,varargin) txt=''; if(~ischar(item)) error('input is not a string'); end item=reshape(item, max(size(item),[1 0])); len=size(item,1); if(~isempty(name)) if(len>1) txt=[S_(checkname(name,varargin{:})) '[']; end else if(len>1) txt='['; end end isoct=jsonopt('IsOctave',0,varargin{:}); for e=1:len val=item(e,:); if(len==1) obj=['' S_(checkname(name,varargin{:})) '' '',S_(val),'']; if(isempty(name)) obj=['',S_(val),'']; end txt=[txt,'',obj]; else txt=[txt,'',['',S_(val),'']]; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=mat2ubjson(name,item,level,varargin) if(~isnumeric(item) && ~islogical(item)) error('input is not an array'); end if(length(size(item))>2 || issparse(item) || ~isreal(item) || ... isempty(item) || jsonopt('ArrayToStruct',0,varargin{:})) cid=I_(uint32(max(size(item)))); if(isempty(name)) txt=['{' S_('_ArrayType_'),S_(class(item)),S_('_ArraySize_'),I_a(size(item),cid(1)) ]; else if(isempty(item)) txt=[S_(checkname(name,varargin{:})),'Z']; return; else txt=[S_(checkname(name,varargin{:})),'{',S_('_ArrayType_'),S_(class(item)),S_('_ArraySize_'),I_a(size(item),cid(1))]; end end else if(isempty(name)) txt=matdata2ubjson(item,level+1,varargin{:}); else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1) numtxt=regexprep(regexprep(matdata2ubjson(item,level+1,varargin{:}),'^\[',''),']',''); txt=[S_(checkname(name,varargin{:})) numtxt]; else txt=[S_(checkname(name,varargin{:})),matdata2ubjson(item,level+1,varargin{:})]; end end return; end if(issparse(item)) [ix,iy]=find(item); data=full(item(find(item))); if(~isreal(item)) data=[real(data(:)),imag(data(:))]; if(size(item,1)==1) % Kludge to have data's 'transposedness' match item's. % (Necessary for complex row vector handling below.) data=data'; end txt=[txt,S_('_ArrayIsComplex_'),'T']; end txt=[txt,S_('_ArrayIsSparse_'),'T']; if(size(item,1)==1) % Row vector, store only column indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([iy(:),data'],level+2,varargin{:})]; elseif(size(item,2)==1) % Column vector, store only row indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([ix,data],level+2,varargin{:})]; else % General case, store row and column indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([ix,iy,data],level+2,varargin{:})]; end else if(isreal(item)) txt=[txt,S_('_ArrayData_'),... matdata2ubjson(item(:)',level+2,varargin{:})]; else txt=[txt,S_('_ArrayIsComplex_'),'T']; txt=[txt,S_('_ArrayData_'),... matdata2ubjson([real(item(:)) imag(item(:))],level+2,varargin{:})]; end end txt=[txt,'}']; %%------------------------------------------------------------------------- function txt=matdata2ubjson(mat,level,varargin) if(isempty(mat)) txt='Z'; return; end if(size(mat,1)==1) level=level-1; end type=''; hasnegtive=(mat<0); if(isa(mat,'integer') || isinteger(mat) || (isfloat(mat) && all(mod(mat(:),1) == 0))) if(isempty(hasnegtive)) if(max(mat(:))<=2^8) type='U'; end end if(isempty(type)) % todo - need to consider negative ones separately id= histc(abs(max(mat(:))),[0 2^7 2^15 2^31 2^63]); if(isempty(find(id))) error('high-precision data is not yet supported'); end key='iIlL'; type=key(find(id)); end txt=[I_a(mat(:),type,size(mat))]; elseif(islogical(mat)) logicalval='FT'; if(numel(mat)==1) txt=logicalval(mat+1); else txt=['[$U#' I_a(size(mat),'l') typecast(swapbytes(uint8(mat(:)')),'uint8')]; end else if(numel(mat)==1) txt=['[' D_(mat) ']']; else txt=D_a(mat(:),'D',size(mat)); end end %txt=regexprep(mat2str(mat),'\s+',','); %txt=regexprep(txt,';',sprintf('],[')); % if(nargin>=2 && size(mat,1)>1) % txt=regexprep(txt,'\[',[repmat(sprintf('\t'),1,level) '[']); % end if(any(isinf(mat(:)))) txt=regexprep(txt,'([-+]*)Inf',jsonopt('Inf','"$1_Inf_"',varargin{:})); end if(any(isnan(mat(:)))) txt=regexprep(txt,'NaN',jsonopt('NaN','"_NaN_"',varargin{:})); end %%------------------------------------------------------------------------- function newname=checkname(name,varargin) isunpack=jsonopt('UnpackHex',1,varargin{:}); newname=name; if(isempty(regexp(name,'0x([0-9a-fA-F]+)_','once'))) return end if(isunpack) isoct=jsonopt('IsOctave',0,varargin{:}); if(~isoct) newname=regexprep(name,'(^x|_){1}0x([0-9a-fA-F]+)_','${native2unicode(hex2dec($2))}'); else pos=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','start'); pend=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','end'); if(isempty(pos)) return; end str0=name; pos0=[0 pend(:)' length(name)]; newname=''; for i=1:length(pos) newname=[newname str0(pos0(i)+1:pos(i)-1) char(hex2dec(str0(pos(i)+3:pend(i)-1)))]; end if(pos(end)~=length(name)) newname=[newname str0(pos0(end-1)+1:pos0(end))]; end end end %%------------------------------------------------------------------------- function val=S_(str) if(length(str)==1) val=['C' str]; else val=['S' I_(int32(length(str))) str]; end %%------------------------------------------------------------------------- function val=I_(num) if(~isinteger(num)) error('input is not an integer'); end if(num>=0 && num<255) val=['U' data2byte(swapbytes(cast(num,'uint8')),'uint8')]; return; end key='iIlL'; cid={'int8','int16','int32','int64'}; for i=1:4 if((num>0 && num<2^(i*8-1)) || (num<0 && num>=-2^(i*8-1))) val=[key(i) data2byte(swapbytes(cast(num,cid{i})),'uint8')]; return; end end error('unsupported integer'); %%------------------------------------------------------------------------- function val=D_(num) if(~isfloat(num)) error('input is not a float'); end if(isa(num,'single')) val=['d' data2byte(num,'uint8')]; else val=['D' data2byte(num,'uint8')]; end %%------------------------------------------------------------------------- function data=I_a(num,type,dim,format) id=find(ismember('iUIlL',type)); if(id==0) error('unsupported integer array'); end % based on UBJSON specs, all integer types are stored in big endian format if(id==1) data=data2byte(swapbytes(int8(num)),'uint8'); blen=1; elseif(id==2) data=data2byte(swapbytes(uint8(num)),'uint8'); blen=1; elseif(id==3) data=data2byte(swapbytes(int16(num)),'uint8'); blen=2; elseif(id==4) data=data2byte(swapbytes(int32(num)),'uint8'); blen=4; elseif(id==5) data=data2byte(swapbytes(int64(num)),'uint8'); blen=8; end if(nargin>=3 && length(dim)>=2 && prod(dim)~=dim(2)) format='opt'; end if((nargin<4 || strcmp(format,'opt')) && numel(num)>1) if(nargin>=3 && (length(dim)==1 || (length(dim)>=2 && prod(dim)~=dim(2)))) cid=I_(uint32(max(dim))); data=['$' type '#' I_a(dim,cid(1)) data(:)']; else data=['$' type '#' I_(int32(numel(data)/blen)) data(:)']; end data=['[' data(:)']; else data=reshape(data,blen,numel(data)/blen); data(2:blen+1,:)=data; data(1,:)=type; data=data(:)'; data=['[' data(:)' ']']; end %%------------------------------------------------------------------------- function data=D_a(num,type,dim,format) id=find(ismember('dD',type)); if(id==0) error('unsupported float array'); end if(id==1) data=data2byte(single(num),'uint8'); elseif(id==2) data=data2byte(double(num),'uint8'); end if(nargin>=3 && length(dim)>=2 && prod(dim)~=dim(2)) format='opt'; end if((nargin<4 || strcmp(format,'opt')) && numel(num)>1) if(nargin>=3 && (length(dim)==1 || (length(dim)>=2 && prod(dim)~=dim(2)))) cid=I_(uint32(max(dim))); data=['$' type '#' I_a(dim,cid(1)) data(:)']; else data=['$' type '#' I_(int32(numel(data)/(id*4))) data(:)']; end data=['[' data]; else data=reshape(data,(id*4),length(data)/(id*4)); data(2:(id*4+1),:)=data; data(1,:)=type; data=data(:)'; data=['[' data(:)' ']']; end %%------------------------------------------------------------------------- function bytes=data2byte(varargin) bytes=typecast(varargin{:}); bytes=bytes(:)'; ================================================ FILE: machine-learning-ex4/ex4/lib/jsonlab/varargin2struct.m ================================================ function opt=varargin2struct(varargin) % % opt=varargin2struct('param1',value1,'param2',value2,...) % or % opt=varargin2struct(...,optstruct,...) % % convert a series of input parameters into a structure % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % date: 2012/12/22 % % input: % 'param', value: the input parameters should be pairs of a string and a value % optstruct: if a parameter is a struct, the fields will be merged to the output struct % % output: % opt: a struct where opt.param1=value1, opt.param2=value2 ... % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % len=length(varargin); opt=struct; if(len==0) return; end i=1; while(i<=len) if(isstruct(varargin{i})) opt=mergestruct(opt,varargin{i}); elseif(ischar(varargin{i}) && i0); % count iterations?! X0 = X; f0 = f1; df0 = df1; % make a copy of current values X = X + z1*s; % begin line search [f2 df2] = eval(argstr); i = i + (length<0); % count epochs?! d2 = df2'*s; f3 = f1; d3 = d1; z3 = -z1; % initialize point 3 equal to point 1 if length>0, M = MAX; else M = min(MAX, -length-i); end success = 0; limit = -1; % initialize quanteties while 1 while ((f2 > f1+z1*RHO*d1) || (d2 > -SIG*d1)) && (M > 0) limit = z1; % tighten the bracket if f2 > f1 z2 = z3 - (0.5*d3*z3*z3)/(d3*z3+f2-f3); % quadratic fit else A = 6*(f2-f3)/z3+3*(d2+d3); % cubic fit B = 3*(f3-f2)-z3*(d3+2*d2); z2 = (sqrt(B*B-A*d2*z3*z3)-B)/A; % numerical error possible - ok! end if isnan(z2) || isinf(z2) z2 = z3/2; % if we had a numerical problem then bisect end z2 = max(min(z2, INT*z3),(1-INT)*z3); % don't accept too close to limits z1 = z1 + z2; % update the step X = X + z2*s; [f2 df2] = eval(argstr); M = M - 1; i = i + (length<0); % count epochs?! d2 = df2'*s; z3 = z3-z2; % z3 is now relative to the location of z2 end if f2 > f1+z1*RHO*d1 || d2 > -SIG*d1 break; % this is a failure elseif d2 > SIG*d1 success = 1; break; % success elseif M == 0 break; % failure end A = 6*(f2-f3)/z3+3*(d2+d3); % make cubic extrapolation B = 3*(f3-f2)-z3*(d3+2*d2); z2 = -d2*z3*z3/(B+sqrt(B*B-A*d2*z3*z3)); % num. error possible - ok! if ~isreal(z2) || isnan(z2) || isinf(z2) || z2 < 0 % num prob or wrong sign? if limit < -0.5 % if we have no upper limit z2 = z1 * (EXT-1); % the extrapolate the maximum amount else z2 = (limit-z1)/2; % otherwise bisect end elseif (limit > -0.5) && (z2+z1 > limit) % extraplation beyond max? z2 = (limit-z1)/2; % bisect elseif (limit < -0.5) && (z2+z1 > z1*EXT) % extrapolation beyond limit z2 = z1*(EXT-1.0); % set to extrapolation limit elseif z2 < -z3*INT z2 = -z3*INT; elseif (limit > -0.5) && (z2 < (limit-z1)*(1.0-INT)) % too close to limit? z2 = (limit-z1)*(1.0-INT); end f3 = f2; d3 = d2; z3 = -z2; % set point 3 equal to point 2 z1 = z1 + z2; X = X + z2*s; % update current estimates [f2 df2] = eval(argstr); M = M - 1; i = i + (length<0); % count epochs?! d2 = df2'*s; end % end of line search if success % if line search succeeded f1 = f2; fX = [fX' f1]'; fprintf('%s %4i | Cost: %4.6e\r', S, i, f1); s = (df2'*df2-df1'*df2)/(df1'*df1)*s - df2; % Polack-Ribiere direction tmp = df1; df1 = df2; df2 = tmp; % swap derivatives d2 = df1'*s; if d2 > 0 % new slope must be negative s = -df1; % otherwise use steepest direction d2 = -s'*s; end z1 = z1 * min(RATIO, d1/(d2-realmin)); % slope ratio but max RATIO d1 = d2; ls_failed = 0; % this line search did not fail else X = X0; f1 = f0; df1 = df0; % restore point from before failed line search if ls_failed || i > abs(length) % line search failed twice in a row break; % or we ran out of time, so we give up end tmp = df1; df1 = df2; df2 = tmp; % swap derivatives s = -df1; % try steepest d1 = -s'*s; z1 = 1/(1-d1); ls_failed = 1; % this line search failed end if exist('OCTAVE_VERSION') fflush(stdout); end end fprintf('\n'); ================================================ FILE: machine-learning-ex5/ex5/learningCurve.m ================================================ function [error_train, error_val] = ... learningCurve(X, y, Xval, yval, lambda) %LEARNINGCURVE Generates the train and cross validation set errors needed %to plot a learning curve % [error_train, error_val] = ... % LEARNINGCURVE(X, y, Xval, yval, lambda) returns the train and % cross validation set errors for a learning curve. In particular, % it returns two vectors of the same length - error_train and % error_val. Then, error_train(i) contains the training error for % i examples (and similarly for error_val(i)). % % In this function, you will compute the train and test errors for % dataset sizes from 1 up to m. In practice, when working with larger % datasets, you might want to do this in larger intervals. % % Number of training examples m = size(X, 1); % You need to return these values correctly error_train = zeros(m, 1); error_val = zeros(m, 1); % ====================== YOUR CODE HERE ====================== % Instructions: Fill in this function to return training errors in % error_train and the cross validation errors in error_val. % i.e., error_train(i) and % error_val(i) should give you the errors % obtained after training on i examples. % % Note: You should evaluate the training error on the first i training % examples (i.e., X(1:i, :) and y(1:i)). % % For the cross-validation error, you should instead evaluate on % the _entire_ cross validation set (Xval and yval). % % Note: If you are using your cost function (linearRegCostFunction) % to compute the training and cross validation error, you should % call the function with the lambda argument set to 0. % Do note that you will still need to use lambda when running % the training to obtain the theta parameters. % % Hint: You can loop over the examples with the following: % % for i = 1:m % % Compute train/cross validation errors using training examples % % X(1:i, :) and y(1:i), storing the result in % % error_train(i) and error_val(i) % .... % % end % % ---------------------- Sample Solution ---------------------- for i = 1:m [theta] = trainLinearReg(X(1:i,:),y(1:i),lambda); h_train(1:i,i) = X(1:i,:)*theta; h_val(:,i) = Xval*theta; error_train(i) = (((h_train(1:i,i)-y(1:i))'*(h_train(1:i,i)-y(1:i)))/(2*i)); error_val(i) = (((h_val(:,i)-yval)'*(h_val(:,i)-yval))/(2*length(yval))); end % ------------------------------------------------------------- % ========================================================================= end ================================================ FILE: machine-learning-ex5/ex5/lib/jsonlab/AUTHORS.txt ================================================ The author of "jsonlab" toolbox is Qianqian Fang. Qianqian is currently an Assistant Professor at Massachusetts General Hospital, Harvard Medical School. Address: Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School Bldg 149, 13th St, Charlestown, MA 02129, USA URL: http://nmr.mgh.harvard.edu/~fangq/ Email: or The script loadjson.m was built upon previous works by - Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713 date: 2009/11/02 - François Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393 date: 2009/03/22 - Joel Feenstra: http://www.mathworks.com/matlabcentral/fileexchange/20565 date: 2008/07/03 This toolbox contains patches submitted by the following contributors: - Blake Johnson part of revision 341 - Niclas Borlin various fixes in revision 394, including - loadjson crashes for all-zero sparse matrix. - loadjson crashes for empty sparse matrix. - Non-zero size of 0-by-N and N-by-0 empty matrices is lost after savejson/loadjson. - loadjson crashes for sparse real column vector. - loadjson crashes for sparse complex column vector. - Data is corrupted by savejson for sparse real row vector. - savejson crashes for sparse complex row vector. - Yul Kang patches for svn revision 415. - savejson saves an empty cell array as [] instead of null - loadjson differentiates an empty struct from an empty array ================================================ FILE: machine-learning-ex5/ex5/lib/jsonlab/ChangeLog.txt ================================================ ============================================================================ JSONlab - a toolbox to encode/decode JSON/UBJSON files in MATLAB/Octave ---------------------------------------------------------------------------- JSONlab ChangeLog (key features marked by *): == JSONlab 1.0 (codename: Optimus - Final), FangQ == 2015/01/02 polish help info for all major functions, update examples, finalize 1.0 2014/12/19 fix a bug to strictly respect NoRowBracket in savejson == JSONlab 1.0.0-RC2 (codename: Optimus - RC2), FangQ == 2014/11/22 show progress bar in loadjson ('ShowProgress') 2014/11/17 add Compact option in savejson to output compact JSON format ('Compact') 2014/11/17 add FastArrayParser in loadjson to specify fast parser applicable levels 2014/09/18 start official github mirror: https://github.com/fangq/jsonlab == JSONlab 1.0.0-RC1 (codename: Optimus - RC1), FangQ == 2014/09/17 fix several compatibility issues when running on octave versions 3.2-3.8 2014/09/17 support 2D cell and struct arrays in both savejson and saveubjson 2014/08/04 escape special characters in a JSON string 2014/02/16 fix a bug when saving ubjson files == JSONlab 0.9.9 (codename: Optimus - beta), FangQ == 2014/01/22 use binary read and write in saveubjson and loadubjson == JSONlab 0.9.8-1 (codename: Optimus - alpha update 1), FangQ == 2013/10/07 better round-trip conservation for empty arrays and structs (patch submitted by Yul Kang) == JSONlab 0.9.8 (codename: Optimus - alpha), FangQ == 2013/08/23 *universal Binary JSON (UBJSON) support, including both saveubjson and loadubjson == JSONlab 0.9.1 (codename: Rodimus, update 1), FangQ == 2012/12/18 *handling of various empty and sparse matrices (fixes submitted by Niclas Borlin) == JSONlab 0.9.0 (codename: Rodimus), FangQ == 2012/06/17 *new format for an invalid leading char, unpacking hex code in savejson 2012/06/01 support JSONP in savejson 2012/05/25 fix the empty cell bug (reported by Cyril Davin) 2012/04/05 savejson can save to a file (suggested by Patrick Rapin) == JSONlab 0.8.1 (codename: Sentiel, Update 1), FangQ == 2012/02/28 loadjson quotation mark escape bug, see http://bit.ly/yyk1nS 2012/01/25 patch to handle root-less objects, contributed by Blake Johnson == JSONlab 0.8.0 (codename: Sentiel), FangQ == 2012/01/13 *speed up loadjson by 20 fold when parsing large data arrays in matlab 2012/01/11 remove row bracket if an array has 1 element, suggested by Mykel Kochenderfer 2011/12/22 *accept sequence of 'param',value input in savejson and loadjson 2011/11/18 fix struct array bug reported by Mykel Kochenderfer == JSONlab 0.5.1 (codename: Nexus Update 1), FangQ == 2011/10/21 fix a bug in loadjson, previous code does not use any of the acceleration 2011/10/20 loadjson supports JSON collections - concatenated JSON objects == JSONlab 0.5.0 (codename: Nexus), FangQ == 2011/10/16 package and release jsonlab 0.5.0 2011/10/15 *add json demo and regression test, support cpx numbers, fix double quote bug 2011/10/11 *speed up readjson dramatically, interpret _Array* tags, show data in root level 2011/10/10 create jsonlab project, start jsonlab website, add online documentation 2011/10/07 *speed up savejson by 25x using sprintf instead of mat2str, add options support 2011/10/06 *savejson works for structs, cells and arrays 2011/09/09 derive loadjson from JSON parser from MATLAB Central, draft savejson.m ================================================ FILE: machine-learning-ex5/ex5/lib/jsonlab/LICENSE_BSD.txt ================================================ Copyright 2011-2015 Qianqian Fang . All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. The views and conclusions contained in the software and documentation are those of the authors and should not be interpreted as representing official policies, either expressed or implied, of the copyright holders. ================================================ FILE: machine-learning-ex5/ex5/lib/jsonlab/README.txt ================================================ =============================================================================== = JSONLab = = An open-source MATLAB/Octave JSON encoder and decoder = =============================================================================== *Copyright (C) 2011-2015 Qianqian Fang *License: BSD License, see License_BSD.txt for details *Version: 1.0 (Optimus - Final) ------------------------------------------------------------------------------- Table of Content: I. Introduction II. Installation III.Using JSONLab IV. Known Issues and TODOs V. Contribution and feedback ------------------------------------------------------------------------------- I. Introduction JSON ([http://www.json.org/ JavaScript Object Notation]) is a highly portable, human-readable and "[http://en.wikipedia.org/wiki/JSON fat-free]" text format to represent complex and hierarchical data. It is as powerful as [http://en.wikipedia.org/wiki/XML XML], but less verbose. JSON format is widely used for data-exchange in applications, and is essential for the wild success of [http://en.wikipedia.org/wiki/Ajax_(programming) Ajax] and [http://en.wikipedia.org/wiki/Web_2.0 Web2.0]. UBJSON (Universal Binary JSON) is a binary JSON format, specifically optimized for compact file size and better performance while keeping the semantics as simple as the text-based JSON format. Using the UBJSON format allows to wrap complex binary data in a flexible and extensible structure, making it possible to process complex and large dataset without accuracy loss due to text conversions. We envision that both JSON and its binary version will serve as part of the mainstream data-exchange formats for scientific research in the future. It will provide the flexibility and generality achieved by other popular general-purpose file specifications, such as [http://www.hdfgroup.org/HDF5/whatishdf5.html HDF5], with significantly reduced complexity and enhanced performance. JSONLab is a free and open-source implementation of a JSON/UBJSON encoder and a decoder in the native MATLAB language. It can be used to convert a MATLAB data structure (array, struct, cell, struct array and cell array) into JSON/UBJSON formatted strings, or to decode a JSON/UBJSON file into MATLAB data structure. JSONLab supports both MATLAB and [http://www.gnu.org/software/octave/ GNU Octave] (a free MATLAB clone). ------------------------------------------------------------------------------- II. Installation The installation of JSONLab is no different than any other simple MATLAB toolbox. You only need to download/unzip the JSONLab package to a folder, and add the folder's path to MATLAB/Octave's path list by using the following command: addpath('/path/to/jsonlab'); If you want to add this path permanently, you need to type "pathtool", browse to the jsonlab root folder and add to the list, then click "Save". Then, run "rehash" in MATLAB, and type "which loadjson", if you see an output, that means JSONLab is installed for MATLAB/Octave. ------------------------------------------------------------------------------- III.Using JSONLab JSONLab provides two functions, loadjson.m -- a MATLAB->JSON decoder, and savejson.m -- a MATLAB->JSON encoder, for the text-based JSON, and two equivallent functions -- loadubjson and saveubjson for the binary JSON. The detailed help info for the four functions can be found below: === loadjson.m === 
  data=loadjson(fname,opt)
     or
  data=loadjson(fname,'param1',value1,'param2',value2,...)
 
  parse a JSON (JavaScript Object Notation) file or string
 
  authors:Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2011/09/09, including previous works from 
 
          Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713
             created on 2009/11/02
          Franois Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393
             created on  2009/03/22
          Joel Feenstra:
          http://www.mathworks.com/matlabcentral/fileexchange/20565
             created on 2008/07/03
 
  $Id: loadjson.m 452 2014-11-22 16:43:33Z fangq $
 
  input:
       fname: input file name, if fname contains "{}" or "[]", fname
              will be interpreted as a JSON string
       opt: a struct to store parsing options, opt can be replaced by 
            a list of ('param',value) pairs - the param string is equivallent
            to a field in opt. opt can have the following 
            fields (first in [.|.] is the default)
 
            opt.SimplifyCell [0|1]: if set to 1, loadjson will call cell2mat
                          for each element of the JSON data, and group 
                          arrays based on the cell2mat rules.
            opt.FastArrayParser [1|0 or integer]: if set to 1, use a
                          speed-optimized array parser when loading an 
                          array object. The fast array parser may 
                          collapse block arrays into a single large
                          array similar to rules defined in cell2mat; 0 to 
                          use a legacy parser; if set to a larger-than-1
                          value, this option will specify the minimum
                          dimension to enable the fast array parser. For
                          example, if the input is a 3D array, setting
                          FastArrayParser to 1 will return a 3D array;
                          setting to 2 will return a cell array of 2D
                          arrays; setting to 3 will return to a 2D cell
                          array of 1D vectors; setting to 4 will return a
                          3D cell array.
            opt.ShowProgress [0|1]: if set to 1, loadjson displays a progress bar.
 
  output:
       dat: a cell array, where {...} blocks are converted into cell arrays,
            and [...] are converted to arrays
 
  examples:
       dat=loadjson('{"obj":{"string":"value","array":[1,2,3]}}')
       dat=loadjson(['examples' filesep 'example1.json'])
       dat=loadjson(['examples' filesep 'example1.json'],'SimplifyCell',1)
=== savejson.m === 
  json=savejson(rootname,obj,filename)
     or
  json=savejson(rootname,obj,opt)
  json=savejson(rootname,obj,'param1',value1,'param2',value2,...)
 
  convert a MATLAB object (cell, struct or array) into a JSON (JavaScript
  Object Notation) string
 
  author: Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2011/09/09
 
  $Id: savejson.m 458 2014-12-19 22:17:17Z fangq $
 
  input:
       rootname: the name of the root-object, when set to '', the root name
         is ignored, however, when opt.ForceRootName is set to 1 (see below),
         the MATLAB variable name will be used as the root name.
       obj: a MATLAB object (array, cell, cell array, struct, struct array).
       filename: a string for the file name to save the output JSON data.
       opt: a struct for additional options, ignore to use default values.
         opt can have the following fields (first in [.|.] is the default)
 
         opt.FileName [''|string]: a file name to save the output JSON data
         opt.FloatFormat ['%.10g'|string]: format to show each numeric element
                          of a 1D/2D array;
         opt.ArrayIndent [1|0]: if 1, output explicit data array with
                          precedent indentation; if 0, no indentation
         opt.ArrayToStruct[0|1]: when set to 0, savejson outputs 1D/2D
                          array in JSON array format; if sets to 1, an
                          array will be shown as a struct with fields
                          "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for
                          sparse arrays, the non-zero elements will be
                          saved to _ArrayData_ field in triplet-format i.e.
                          (ix,iy,val) and "_ArrayIsSparse_" will be added
                          with a value of 1; for a complex array, the 
                          _ArrayData_ array will include two columns 
                          (4 for sparse) to record the real and imaginary 
                          parts, and also "_ArrayIsComplex_":1 is added. 
         opt.ParseLogical [0|1]: if this is set to 1, logical array elem
                          will use true/false rather than 1/0.
         opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single
                          numerical element will be shown without a square
                          bracket, unless it is the root object; if 0, square
                          brackets are forced for any numerical arrays.
         opt.ForceRootName [0|1]: when set to 1 and rootname is empty, savejson
                          will use the name of the passed obj variable as the 
                          root object name; if obj is an expression and 
                          does not have a name, 'root' will be used; if this 
                          is set to 0 and rootname is empty, the root level 
                          will be merged down to the lower level.
         opt.Inf ['"$1_Inf_"'|string]: a customized regular expression pattern
                          to represent +/-Inf. The matched pattern is '([-+]*)Inf'
                          and $1 represents the sign. For those who want to use
                          1e999 to represent Inf, they can set opt.Inf to '$11e999'
         opt.NaN ['"_NaN_"'|string]: a customized regular expression pattern
                          to represent NaN
         opt.JSONP [''|string]: to generate a JSONP output (JSON with padding),
                          for example, if opt.JSONP='foo', the JSON data is
                          wrapped inside a function call as 'foo(...);'
         opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson 
                          back to the string form
         opt.SaveBinary [0|1]: 1 - save the JSON file in binary mode; 0 - text mode.
         opt.Compact [0|1]: 1- out compact JSON format (remove all newlines and tabs)
 
         opt can be replaced by a list of ('param',value) pairs. The param 
         string is equivallent to a field in opt and is case sensitive.
  output:
       json: a string in the JSON format (see http://json.org)
 
  examples:
       jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... 
                'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],...
                'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;...
                           2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],...
                'MeshCreator','FangQ','MeshTitle','T6 Cube',...
                'SpecialData',[nan, inf, -inf]);
       savejson('jmesh',jsonmesh)
       savejson('',jsonmesh,'ArrayIndent',0,'FloatFormat','\t%.5g')
=== loadubjson.m === 
  data=loadubjson(fname,opt)
     or
  data=loadubjson(fname,'param1',value1,'param2',value2,...)
 
  parse a JSON (JavaScript Object Notation) file or string
 
  authors:Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2013/08/01
 
  $Id: loadubjson.m 436 2014-08-05 20:51:40Z fangq $
 
  input:
       fname: input file name, if fname contains "{}" or "[]", fname
              will be interpreted as a UBJSON string
       opt: a struct to store parsing options, opt can be replaced by 
            a list of ('param',value) pairs - the param string is equivallent
            to a field in opt. opt can have the following 
            fields (first in [.|.] is the default)
 
            opt.SimplifyCell [0|1]: if set to 1, loadubjson will call cell2mat
                          for each element of the JSON data, and group 
                          arrays based on the cell2mat rules.
            opt.IntEndian [B|L]: specify the endianness of the integer fields
                          in the UBJSON input data. B - Big-Endian format for 
                          integers (as required in the UBJSON specification); 
                          L - input integer fields are in Little-Endian order.
 
  output:
       dat: a cell array, where {...} blocks are converted into cell arrays,
            and [...] are converted to arrays
 
  examples:
       obj=struct('string','value','array',[1 2 3]);
       ubjdata=saveubjson('obj',obj);
       dat=loadubjson(ubjdata)
       dat=loadubjson(['examples' filesep 'example1.ubj'])
       dat=loadubjson(['examples' filesep 'example1.ubj'],'SimplifyCell',1)
=== saveubjson.m === 
  json=saveubjson(rootname,obj,filename)
     or
  json=saveubjson(rootname,obj,opt)
  json=saveubjson(rootname,obj,'param1',value1,'param2',value2,...)
 
  convert a MATLAB object (cell, struct or array) into a Universal 
  Binary JSON (UBJSON) binary string
 
  author: Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2013/08/17
 
  $Id: saveubjson.m 440 2014-09-17 19:59:45Z fangq $
 
  input:
       rootname: the name of the root-object, when set to '', the root name
         is ignored, however, when opt.ForceRootName is set to 1 (see below),
         the MATLAB variable name will be used as the root name.
       obj: a MATLAB object (array, cell, cell array, struct, struct array)
       filename: a string for the file name to save the output UBJSON data
       opt: a struct for additional options, ignore to use default values.
         opt can have the following fields (first in [.|.] is the default)
 
         opt.FileName [''|string]: a file name to save the output JSON data
         opt.ArrayToStruct[0|1]: when set to 0, saveubjson outputs 1D/2D
                          array in JSON array format; if sets to 1, an
                          array will be shown as a struct with fields
                          "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for
                          sparse arrays, the non-zero elements will be
                          saved to _ArrayData_ field in triplet-format i.e.
                          (ix,iy,val) and "_ArrayIsSparse_" will be added
                          with a value of 1; for a complex array, the 
                          _ArrayData_ array will include two columns 
                          (4 for sparse) to record the real and imaginary 
                          parts, and also "_ArrayIsComplex_":1 is added. 
         opt.ParseLogical [1|0]: if this is set to 1, logical array elem
                          will use true/false rather than 1/0.
         opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single
                          numerical element will be shown without a square
                          bracket, unless it is the root object; if 0, square
                          brackets are forced for any numerical arrays.
         opt.ForceRootName [0|1]: when set to 1 and rootname is empty, saveubjson
                          will use the name of the passed obj variable as the 
                          root object name; if obj is an expression and 
                          does not have a name, 'root' will be used; if this 
                          is set to 0 and rootname is empty, the root level 
                          will be merged down to the lower level.
         opt.JSONP [''|string]: to generate a JSONP output (JSON with padding),
                          for example, if opt.JSON='foo', the JSON data is
                          wrapped inside a function call as 'foo(...);'
         opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson 
                          back to the string form
 
         opt can be replaced by a list of ('param',value) pairs. The param 
         string is equivallent to a field in opt and is case sensitive.
  output:
       json: a binary string in the UBJSON format (see http://ubjson.org)
 
  examples:
       jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... 
                'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],...
                'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;...
                           2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],...
                'MeshCreator','FangQ','MeshTitle','T6 Cube',...
                'SpecialData',[nan, inf, -inf]);
       saveubjson('jsonmesh',jsonmesh)
       saveubjson('jsonmesh',jsonmesh,'meshdata.ubj')
=== examples === Under the "examples" folder, you can find several scripts to demonstrate the basic utilities of JSONLab. Running the "demo_jsonlab_basic.m" script, you will see the conversions from MATLAB data structure to JSON text and backward. In "jsonlab_selftest.m", we load complex JSON files downloaded from the Internet and validate the loadjson/savejson functions for regression testing purposes. Similarly, a "demo_ubjson_basic.m" script is provided to test the saveubjson and loadubjson pairs for various matlab data structures. Please run these examples and understand how JSONLab works before you use it to process your data. ------------------------------------------------------------------------------- IV. Known Issues and TODOs JSONLab has several known limitations. We are striving to make it more general and robust. Hopefully in a few future releases, the limitations become less. Here are the known issues: # 3D or higher dimensional cell/struct-arrays will be converted to 2D arrays; # When processing names containing multi-byte characters, Octave and MATLAB \ can give different field-names; you can use feature('DefaultCharacterSet','latin1') \ in MATLAB to get consistant results # savejson can not handle class and dataset. # saveubjson converts a logical array into a uint8 ([U]) array # an unofficial N-D array count syntax is implemented in saveubjson. We are \ actively communicating with the UBJSON spec maintainer to investigate the \ possibility of making it upstream # loadubjson can not parse all UBJSON Specification (Draft 9) compliant \ files, however, it can parse all UBJSON files produced by saveubjson. ------------------------------------------------------------------------------- V. Contribution and feedback JSONLab is an open-source project. This means you can not only use it and modify it as you wish, but also you can contribute your changes back to JSONLab so that everyone else can enjoy the improvement. For anyone who want to contribute, please download JSONLab source code from it's subversion repository by using the following command: svn checkout svn://svn.code.sf.net/p/iso2mesh/code/trunk/jsonlab jsonlab You can make changes to the files as needed. Once you are satisfied with your changes, and ready to share it with others, please cd the root directory of JSONLab, and type svn diff > yourname_featurename.patch You then email the .patch file to JSONLab's maintainer, Qianqian Fang, at the email address shown in the beginning of this file. Qianqian will review the changes and commit it to the subversion if they are satisfactory. We appreciate any suggestions and feedbacks from you. Please use iso2mesh's mailing list to report any questions you may have with JSONLab: http://groups.google.com/group/iso2mesh-users?hl=en&pli=1 (Subscription to the mailing list is needed in order to post messages). ================================================ FILE: machine-learning-ex5/ex5/lib/jsonlab/jsonopt.m ================================================ function val=jsonopt(key,default,varargin) % % val=jsonopt(key,default,optstruct) % % setting options based on a struct. The struct can be produced % by varargin2struct from a list of 'param','value' pairs % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % % $Id: loadjson.m 371 2012-06-20 12:43:06Z fangq $ % % input: % key: a string with which one look up a value from a struct % default: if the key does not exist, return default % optstruct: a struct where each sub-field is a key % % output: % val: if key exists, val=optstruct.key; otherwise val=default % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % val=default; if(nargin<=2) return; end opt=varargin{1}; if(isstruct(opt) && isfield(opt,key)) val=getfield(opt,key); end ================================================ FILE: machine-learning-ex5/ex5/lib/jsonlab/loadjson.m ================================================ function data = loadjson(fname,varargin) % % data=loadjson(fname,opt) % or % data=loadjson(fname,'param1',value1,'param2',value2,...) % % parse a JSON (JavaScript Object Notation) file or string % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2011/09/09, including previous works from % % Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713 % created on 2009/11/02 % François Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393 % created on 2009/03/22 % Joel Feenstra: % http://www.mathworks.com/matlabcentral/fileexchange/20565 % created on 2008/07/03 % % $Id: loadjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % fname: input file name, if fname contains "{}" or "[]", fname % will be interpreted as a JSON string % opt: a struct to store parsing options, opt can be replaced by % a list of ('param',value) pairs - the param string is equivallent % to a field in opt. opt can have the following % fields (first in [.|.] is the default) % % opt.SimplifyCell [0|1]: if set to 1, loadjson will call cell2mat % for each element of the JSON data, and group % arrays based on the cell2mat rules. % opt.FastArrayParser [1|0 or integer]: if set to 1, use a % speed-optimized array parser when loading an % array object. The fast array parser may % collapse block arrays into a single large % array similar to rules defined in cell2mat; 0 to % use a legacy parser; if set to a larger-than-1 % value, this option will specify the minimum % dimension to enable the fast array parser. For % example, if the input is a 3D array, setting % FastArrayParser to 1 will return a 3D array; % setting to 2 will return a cell array of 2D % arrays; setting to 3 will return to a 2D cell % array of 1D vectors; setting to 4 will return a % 3D cell array. % opt.ShowProgress [0|1]: if set to 1, loadjson displays a progress bar. % % output: % dat: a cell array, where {...} blocks are converted into cell arrays, % and [...] are converted to arrays % % examples: % dat=loadjson('{"obj":{"string":"value","array":[1,2,3]}}') % dat=loadjson(['examples' filesep 'example1.json']) % dat=loadjson(['examples' filesep 'example1.json'],'SimplifyCell',1) % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % global pos inStr len esc index_esc len_esc isoct arraytoken if(regexp(fname,'[\{\}\]\[]','once')) string=fname; elseif(exist(fname,'file')) fid = fopen(fname,'rb'); string = fread(fid,inf,'uint8=>char')'; fclose(fid); else error('input file does not exist'); end pos = 1; len = length(string); inStr = string; isoct=exist('OCTAVE_VERSION','builtin'); arraytoken=find(inStr=='[' | inStr==']' | inStr=='"'); jstr=regexprep(inStr,'\\\\',' '); escquote=regexp(jstr,'\\"'); arraytoken=sort([arraytoken escquote]); % String delimiters and escape chars identified to improve speed: esc = find(inStr=='"' | inStr=='\' ); % comparable to: regexp(inStr, '["\\]'); index_esc = 1; len_esc = length(esc); opt=varargin2struct(varargin{:}); if(jsonopt('ShowProgress',0,opt)==1) opt.progressbar_=waitbar(0,'loading ...'); end jsoncount=1; while pos <= len switch(next_char) case '{' data{jsoncount} = parse_object(opt); case '[' data{jsoncount} = parse_array(opt); otherwise error_pos('Outer level structure must be an object or an array'); end jsoncount=jsoncount+1; end % while jsoncount=length(data); if(jsoncount==1 && iscell(data)) data=data{1}; end if(~isempty(data)) if(isstruct(data)) % data can be a struct array data=jstruct2array(data); elseif(iscell(data)) data=jcell2array(data); end end if(isfield(opt,'progressbar_')) close(opt.progressbar_); end %% function newdata=jcell2array(data) len=length(data); newdata=data; for i=1:len if(isstruct(data{i})) newdata{i}=jstruct2array(data{i}); elseif(iscell(data{i})) newdata{i}=jcell2array(data{i}); end end %%------------------------------------------------------------------------- function newdata=jstruct2array(data) fn=fieldnames(data); newdata=data; len=length(data); for i=1:length(fn) % depth-first for j=1:len if(isstruct(getfield(data(j),fn{i}))) newdata(j)=setfield(newdata(j),fn{i},jstruct2array(getfield(data(j),fn{i}))); end end end if(~isempty(strmatch('x0x5F_ArrayType_',fn)) && ~isempty(strmatch('x0x5F_ArrayData_',fn))) newdata=cell(len,1); for j=1:len ndata=cast(data(j).x0x5F_ArrayData_,data(j).x0x5F_ArrayType_); iscpx=0; if(~isempty(strmatch('x0x5F_ArrayIsComplex_',fn))) if(data(j).x0x5F_ArrayIsComplex_) iscpx=1; end end if(~isempty(strmatch('x0x5F_ArrayIsSparse_',fn))) if(data(j).x0x5F_ArrayIsSparse_) if(~isempty(strmatch('x0x5F_ArraySize_',fn))) dim=data(j).x0x5F_ArraySize_; if(iscpx && size(ndata,2)==4-any(dim==1)) ndata(:,end-1)=complex(ndata(:,end-1),ndata(:,end)); end if isempty(ndata) % All-zeros sparse ndata=sparse(dim(1),prod(dim(2:end))); elseif dim(1)==1 % Sparse row vector ndata=sparse(1,ndata(:,1),ndata(:,2),dim(1),prod(dim(2:end))); elseif dim(2)==1 % Sparse column vector ndata=sparse(ndata(:,1),1,ndata(:,2),dim(1),prod(dim(2:end))); else % Generic sparse array. ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3),dim(1),prod(dim(2:end))); end else if(iscpx && size(ndata,2)==4) ndata(:,3)=complex(ndata(:,3),ndata(:,4)); end ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3)); end end elseif(~isempty(strmatch('x0x5F_ArraySize_',fn))) if(iscpx && size(ndata,2)==2) ndata=complex(ndata(:,1),ndata(:,2)); end ndata=reshape(ndata(:),data(j).x0x5F_ArraySize_); end newdata{j}=ndata; end if(len==1) newdata=newdata{1}; end end %%------------------------------------------------------------------------- function object = parse_object(varargin) parse_char('{'); object = []; if next_char ~= '}' while 1 str = parseStr(varargin{:}); if isempty(str) error_pos('Name of value at position %d cannot be empty'); end parse_char(':'); val = parse_value(varargin{:}); eval( sprintf( 'object.%s = val;', valid_field(str) ) ); if next_char == '}' break; end parse_char(','); end end parse_char('}'); %%------------------------------------------------------------------------- function object = parse_array(varargin) % JSON array is written in row-major order global pos inStr isoct parse_char('['); object = cell(0, 1); dim2=[]; arraydepth=jsonopt('JSONLAB_ArrayDepth_',1,varargin{:}); pbar=jsonopt('progressbar_',-1,varargin{:}); if next_char ~= ']' if(jsonopt('FastArrayParser',1,varargin{:})>=1 && arraydepth>=jsonopt('FastArrayParser',1,varargin{:})) [endpos, e1l, e1r, maxlevel]=matching_bracket(inStr,pos); arraystr=['[' inStr(pos:endpos)]; arraystr=regexprep(arraystr,'"_NaN_"','NaN'); arraystr=regexprep(arraystr,'"([-+]*)_Inf_"','$1Inf'); arraystr(arraystr==sprintf('\n'))=[]; arraystr(arraystr==sprintf('\r'))=[]; %arraystr=regexprep(arraystr,'\s*,',','); % this is slow,sometimes needed if(~isempty(e1l) && ~isempty(e1r)) % the array is in 2D or higher D astr=inStr((e1l+1):(e1r-1)); astr=regexprep(astr,'"_NaN_"','NaN'); astr=regexprep(astr,'"([-+]*)_Inf_"','$1Inf'); astr(astr==sprintf('\n'))=[]; astr(astr==sprintf('\r'))=[]; astr(astr==' ')=''; if(isempty(find(astr=='[', 1))) % array is 2D dim2=length(sscanf(astr,'%f,',[1 inf])); end else % array is 1D astr=arraystr(2:end-1); astr(astr==' ')=''; [obj, count, errmsg, nextidx]=sscanf(astr,'%f,',[1,inf]); if(nextidx>=length(astr)-1) object=obj; pos=endpos; parse_char(']'); return; end end if(~isempty(dim2)) astr=arraystr; astr(astr=='[')=''; astr(astr==']')=''; astr(astr==' ')=''; [obj, count, errmsg, nextidx]=sscanf(astr,'%f,',inf); if(nextidx>=length(astr)-1) object=reshape(obj,dim2,numel(obj)/dim2)'; pos=endpos; parse_char(']'); if(pbar>0) waitbar(pos/length(inStr),pbar,'loading ...'); end return; end end arraystr=regexprep(arraystr,'\]\s*,','];'); else arraystr='['; end try if(isoct && regexp(arraystr,'"','once')) error('Octave eval can produce empty cells for JSON-like input'); end object=eval(arraystr); pos=endpos; catch while 1 newopt=varargin2struct(varargin{:},'JSONLAB_ArrayDepth_',arraydepth+1); val = parse_value(newopt); object{end+1} = val; if next_char == ']' break; end parse_char(','); end end end if(jsonopt('SimplifyCell',0,varargin{:})==1) try oldobj=object; object=cell2mat(object')'; if(iscell(oldobj) && isstruct(object) && numel(object)>1 && jsonopt('SimplifyCellArray',1,varargin{:})==0) object=oldobj; elseif(size(object,1)>1 && ndims(object)==2) object=object'; end catch end end parse_char(']'); if(pbar>0) waitbar(pos/length(inStr),pbar,'loading ...'); end %%------------------------------------------------------------------------- function parse_char(c) global pos inStr len skip_whitespace; if pos > len || inStr(pos) ~= c error_pos(sprintf('Expected %c at position %%d', c)); else pos = pos + 1; skip_whitespace; end %%------------------------------------------------------------------------- function c = next_char global pos inStr len skip_whitespace; if pos > len c = []; else c = inStr(pos); end %%------------------------------------------------------------------------- function skip_whitespace global pos inStr len while pos <= len && isspace(inStr(pos)) pos = pos + 1; end %%------------------------------------------------------------------------- function str = parseStr(varargin) global pos inStr len esc index_esc len_esc % len, ns = length(inStr), keyboard if inStr(pos) ~= '"' error_pos('String starting with " expected at position %d'); else pos = pos + 1; end str = ''; while pos <= len while index_esc <= len_esc && esc(index_esc) < pos index_esc = index_esc + 1; end if index_esc > len_esc str = [str inStr(pos:len)]; pos = len + 1; break; else str = [str inStr(pos:esc(index_esc)-1)]; pos = esc(index_esc); end nstr = length(str); switch inStr(pos) case '"' pos = pos + 1; if(~isempty(str)) if(strcmp(str,'_Inf_')) str=Inf; elseif(strcmp(str,'-_Inf_')) str=-Inf; elseif(strcmp(str,'_NaN_')) str=NaN; end end return; case '\' if pos+1 > len error_pos('End of file reached right after escape character'); end pos = pos + 1; switch inStr(pos) case {'"' '\' '/'} str(nstr+1) = inStr(pos); pos = pos + 1; case {'b' 'f' 'n' 'r' 't'} str(nstr+1) = sprintf(['\' inStr(pos)]); pos = pos + 1; case 'u' if pos+4 > len error_pos('End of file reached in escaped unicode character'); end str(nstr+(1:6)) = inStr(pos-1:pos+4); pos = pos + 5; end otherwise % should never happen str(nstr+1) = inStr(pos), keyboard pos = pos + 1; end end error_pos('End of file while expecting end of inStr'); %%------------------------------------------------------------------------- function num = parse_number(varargin) global pos inStr len isoct currstr=inStr(pos:end); numstr=0; if(isoct~=0) numstr=regexp(currstr,'^\s*-?(?:0|[1-9]\d*)(?:\.\d+)?(?:[eE][+\-]?\d+)?','end'); [num, one] = sscanf(currstr, '%f', 1); delta=numstr+1; else [num, one, err, delta] = sscanf(currstr, '%f', 1); if ~isempty(err) error_pos('Error reading number at position %d'); end end pos = pos + delta-1; %%------------------------------------------------------------------------- function val = parse_value(varargin) global pos inStr len true = 1; false = 0; pbar=jsonopt('progressbar_',-1,varargin{:}); if(pbar>0) waitbar(pos/len,pbar,'loading ...'); end switch(inStr(pos)) case '"' val = parseStr(varargin{:}); return; case '[' val = parse_array(varargin{:}); return; case '{' val = parse_object(varargin{:}); if isstruct(val) if(~isempty(strmatch('x0x5F_ArrayType_',fieldnames(val), 'exact'))) val=jstruct2array(val); end elseif isempty(val) val = struct; end return; case {'-','0','1','2','3','4','5','6','7','8','9'} val = parse_number(varargin{:}); return; case 't' if pos+3 <= len && strcmpi(inStr(pos:pos+3), 'true') val = true; pos = pos + 4; return; end case 'f' if pos+4 <= len && strcmpi(inStr(pos:pos+4), 'false') val = false; pos = pos + 5; return; end case 'n' if pos+3 <= len && strcmpi(inStr(pos:pos+3), 'null') val = []; pos = pos + 4; return; end end error_pos('Value expected at position %d'); %%------------------------------------------------------------------------- function error_pos(msg) global pos inStr len poShow = max(min([pos-15 pos-1 pos pos+20],len),1); if poShow(3) == poShow(2) poShow(3:4) = poShow(2)+[0 -1]; % display nothing after end msg = [sprintf(msg, pos) ': ' ... inStr(poShow(1):poShow(2)) '' inStr(poShow(3):poShow(4)) ]; error( ['JSONparser:invalidFormat: ' msg] ); %%------------------------------------------------------------------------- function str = valid_field(str) global isoct % From MATLAB doc: field names must begin with a letter, which may be % followed by any combination of letters, digits, and underscores. % Invalid characters will be converted to underscores, and the prefix % "x0x[Hex code]_" will be added if the first character is not a letter. pos=regexp(str,'^[^A-Za-z]','once'); if(~isempty(pos)) if(~isoct) str=regexprep(str,'^([^A-Za-z])','x0x${sprintf(''%X'',unicode2native($1))}_','once'); else str=sprintf('x0x%X_%s',char(str(1)),str(2:end)); end end if(isempty(regexp(str,'[^0-9A-Za-z_]', 'once' ))) return; end if(~isoct) str=regexprep(str,'([^0-9A-Za-z_])','_0x${sprintf(''%X'',unicode2native($1))}_'); else pos=regexp(str,'[^0-9A-Za-z_]'); if(isempty(pos)) return; end str0=str; pos0=[0 pos(:)' length(str)]; str=''; for i=1:length(pos) str=[str str0(pos0(i)+1:pos(i)-1) sprintf('_0x%X_',str0(pos(i)))]; end if(pos(end)~=length(str)) str=[str str0(pos0(end-1)+1:pos0(end))]; end end %str(~isletter(str) & ~('0' <= str & str <= '9')) = '_'; %%------------------------------------------------------------------------- function endpos = matching_quote(str,pos) len=length(str); while(pos1 && str(pos-1)=='\')) endpos=pos; return; end end pos=pos+1; end error('unmatched quotation mark'); %%------------------------------------------------------------------------- function [endpos, e1l, e1r, maxlevel] = matching_bracket(str,pos) global arraytoken level=1; maxlevel=level; endpos=0; bpos=arraytoken(arraytoken>=pos); tokens=str(bpos); len=length(tokens); pos=1; e1l=[]; e1r=[]; while(pos<=len) c=tokens(pos); if(c==']') level=level-1; if(isempty(e1r)) e1r=bpos(pos); end if(level==0) endpos=bpos(pos); return end end if(c=='[') if(isempty(e1l)) e1l=bpos(pos); end level=level+1; maxlevel=max(maxlevel,level); end if(c=='"') pos=matching_quote(tokens,pos+1); end pos=pos+1; end if(endpos==0) error('unmatched "]"'); end ================================================ FILE: machine-learning-ex5/ex5/lib/jsonlab/loadubjson.m ================================================ function data = loadubjson(fname,varargin) % % data=loadubjson(fname,opt) % or % data=loadubjson(fname,'param1',value1,'param2',value2,...) % % parse a JSON (JavaScript Object Notation) file or string % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2013/08/01 % % $Id: loadubjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % fname: input file name, if fname contains "{}" or "[]", fname % will be interpreted as a UBJSON string % opt: a struct to store parsing options, opt can be replaced by % a list of ('param',value) pairs - the param string is equivallent % to a field in opt. opt can have the following % fields (first in [.|.] is the default) % % opt.SimplifyCell [0|1]: if set to 1, loadubjson will call cell2mat % for each element of the JSON data, and group % arrays based on the cell2mat rules. % opt.IntEndian [B|L]: specify the endianness of the integer fields % in the UBJSON input data. B - Big-Endian format for % integers (as required in the UBJSON specification); % L - input integer fields are in Little-Endian order. % % output: % dat: a cell array, where {...} blocks are converted into cell arrays, % and [...] are converted to arrays % % examples: % obj=struct('string','value','array',[1 2 3]); % ubjdata=saveubjson('obj',obj); % dat=loadubjson(ubjdata) % dat=loadubjson(['examples' filesep 'example1.ubj']) % dat=loadubjson(['examples' filesep 'example1.ubj'],'SimplifyCell',1) % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % global pos inStr len esc index_esc len_esc isoct arraytoken fileendian systemendian if(regexp(fname,'[\{\}\]\[]','once')) string=fname; elseif(exist(fname,'file')) fid = fopen(fname,'rb'); string = fread(fid,inf,'uint8=>char')'; fclose(fid); else error('input file does not exist'); end pos = 1; len = length(string); inStr = string; isoct=exist('OCTAVE_VERSION','builtin'); arraytoken=find(inStr=='[' | inStr==']' | inStr=='"'); jstr=regexprep(inStr,'\\\\',' '); escquote=regexp(jstr,'\\"'); arraytoken=sort([arraytoken escquote]); % String delimiters and escape chars identified to improve speed: esc = find(inStr=='"' | inStr=='\' ); % comparable to: regexp(inStr, '["\\]'); index_esc = 1; len_esc = length(esc); opt=varargin2struct(varargin{:}); fileendian=upper(jsonopt('IntEndian','B',opt)); [os,maxelem,systemendian]=computer; jsoncount=1; while pos <= len switch(next_char) case '{' data{jsoncount} = parse_object(opt); case '[' data{jsoncount} = parse_array(opt); otherwise error_pos('Outer level structure must be an object or an array'); end jsoncount=jsoncount+1; end % while jsoncount=length(data); if(jsoncount==1 && iscell(data)) data=data{1}; end if(~isempty(data)) if(isstruct(data)) % data can be a struct array data=jstruct2array(data); elseif(iscell(data)) data=jcell2array(data); end end %% function newdata=parse_collection(id,data,obj) if(jsoncount>0 && exist('data','var')) if(~iscell(data)) newdata=cell(1); newdata{1}=data; data=newdata; end end %% function newdata=jcell2array(data) len=length(data); newdata=data; for i=1:len if(isstruct(data{i})) newdata{i}=jstruct2array(data{i}); elseif(iscell(data{i})) newdata{i}=jcell2array(data{i}); end end %%------------------------------------------------------------------------- function newdata=jstruct2array(data) fn=fieldnames(data); newdata=data; len=length(data); for i=1:length(fn) % depth-first for j=1:len if(isstruct(getfield(data(j),fn{i}))) newdata(j)=setfield(newdata(j),fn{i},jstruct2array(getfield(data(j),fn{i}))); end end end if(~isempty(strmatch('x0x5F_ArrayType_',fn)) && ~isempty(strmatch('x0x5F_ArrayData_',fn))) newdata=cell(len,1); for j=1:len ndata=cast(data(j).x0x5F_ArrayData_,data(j).x0x5F_ArrayType_); iscpx=0; if(~isempty(strmatch('x0x5F_ArrayIsComplex_',fn))) if(data(j).x0x5F_ArrayIsComplex_) iscpx=1; end end if(~isempty(strmatch('x0x5F_ArrayIsSparse_',fn))) if(data(j).x0x5F_ArrayIsSparse_) if(~isempty(strmatch('x0x5F_ArraySize_',fn))) dim=double(data(j).x0x5F_ArraySize_); if(iscpx && size(ndata,2)==4-any(dim==1)) ndata(:,end-1)=complex(ndata(:,end-1),ndata(:,end)); end if isempty(ndata) % All-zeros sparse ndata=sparse(dim(1),prod(dim(2:end))); elseif dim(1)==1 % Sparse row vector ndata=sparse(1,ndata(:,1),ndata(:,2),dim(1),prod(dim(2:end))); elseif dim(2)==1 % Sparse column vector ndata=sparse(ndata(:,1),1,ndata(:,2),dim(1),prod(dim(2:end))); else % Generic sparse array. ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3),dim(1),prod(dim(2:end))); end else if(iscpx && size(ndata,2)==4) ndata(:,3)=complex(ndata(:,3),ndata(:,4)); end ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3)); end end elseif(~isempty(strmatch('x0x5F_ArraySize_',fn))) if(iscpx && size(ndata,2)==2) ndata=complex(ndata(:,1),ndata(:,2)); end ndata=reshape(ndata(:),data(j).x0x5F_ArraySize_); end newdata{j}=ndata; end if(len==1) newdata=newdata{1}; end end %%------------------------------------------------------------------------- function object = parse_object(varargin) parse_char('{'); object = []; type=''; count=-1; if(next_char == '$') type=inStr(pos+1); % TODO pos=pos+2; end if(next_char == '#') pos=pos+1; count=double(parse_number()); end if next_char ~= '}' num=0; while 1 str = parseStr(varargin{:}); if isempty(str) error_pos('Name of value at position %d cannot be empty'); end %parse_char(':'); val = parse_value(varargin{:}); num=num+1; eval( sprintf( 'object.%s = val;', valid_field(str) ) ); if next_char == '}' || (count>=0 && num>=count) break; end %parse_char(','); end end if(count==-1) parse_char('}'); end %%------------------------------------------------------------------------- function [cid,len]=elem_info(type) id=strfind('iUIlLdD',type); dataclass={'int8','uint8','int16','int32','int64','single','double'}; bytelen=[1,1,2,4,8,4,8]; if(id>0) cid=dataclass{id}; len=bytelen(id); else error_pos('unsupported type at position %d'); end %%------------------------------------------------------------------------- function [data adv]=parse_block(type,count,varargin) global pos inStr isoct fileendian systemendian [cid,len]=elem_info(type); datastr=inStr(pos:pos+len*count-1); if(isoct) newdata=int8(datastr); else newdata=uint8(datastr); end id=strfind('iUIlLdD',type); if(id<=5 && fileendian~=systemendian) newdata=swapbytes(typecast(newdata,cid)); end data=typecast(newdata,cid); adv=double(len*count); %%------------------------------------------------------------------------- function object = parse_array(varargin) % JSON array is written in row-major order global pos inStr isoct parse_char('['); object = cell(0, 1); dim=[]; type=''; count=-1; if(next_char == '$') type=inStr(pos+1); pos=pos+2; end if(next_char == '#') pos=pos+1; if(next_char=='[') dim=parse_array(varargin{:}); count=prod(double(dim)); else count=double(parse_number()); end end if(~isempty(type)) if(count>=0) [object adv]=parse_block(type,count,varargin{:}); if(~isempty(dim)) object=reshape(object,dim); end pos=pos+adv; return; else endpos=matching_bracket(inStr,pos); [cid,len]=elem_info(type); count=(endpos-pos)/len; [object adv]=parse_block(type,count,varargin{:}); pos=pos+adv; parse_char(']'); return; end end if next_char ~= ']' while 1 val = parse_value(varargin{:}); object{end+1} = val; if next_char == ']' break; end %parse_char(','); end end if(jsonopt('SimplifyCell',0,varargin{:})==1) try oldobj=object; object=cell2mat(object')'; if(iscell(oldobj) && isstruct(object) && numel(object)>1 && jsonopt('SimplifyCellArray',1,varargin{:})==0) object=oldobj; elseif(size(object,1)>1 && ndims(object)==2) object=object'; end catch end end if(count==-1) parse_char(']'); end %%------------------------------------------------------------------------- function parse_char(c) global pos inStr len skip_whitespace; if pos > len || inStr(pos) ~= c error_pos(sprintf('Expected %c at position %%d', c)); else pos = pos + 1; skip_whitespace; end %%------------------------------------------------------------------------- function c = next_char global pos inStr len skip_whitespace; if pos > len c = []; else c = inStr(pos); end %%------------------------------------------------------------------------- function skip_whitespace global pos inStr len while pos <= len && isspace(inStr(pos)) pos = pos + 1; end %%------------------------------------------------------------------------- function str = parseStr(varargin) global pos inStr esc index_esc len_esc % len, ns = length(inStr), keyboard type=inStr(pos); if type ~= 'S' && type ~= 'C' && type ~= 'H' error_pos('String starting with S expected at position %d'); else pos = pos + 1; end if(type == 'C') str=inStr(pos); pos=pos+1; return; end bytelen=double(parse_number()); if(length(inStr)>=pos+bytelen-1) str=inStr(pos:pos+bytelen-1); pos=pos+bytelen; else error_pos('End of file while expecting end of inStr'); end %%------------------------------------------------------------------------- function num = parse_number(varargin) global pos inStr len isoct fileendian systemendian id=strfind('iUIlLdD',inStr(pos)); if(isempty(id)) error_pos('expecting a number at position %d'); end type={'int8','uint8','int16','int32','int64','single','double'}; bytelen=[1,1,2,4,8,4,8]; datastr=inStr(pos+1:pos+bytelen(id)); if(isoct) newdata=int8(datastr); else newdata=uint8(datastr); end if(id<=5 && fileendian~=systemendian) newdata=swapbytes(typecast(newdata,type{id})); end num=typecast(newdata,type{id}); pos = pos + bytelen(id)+1; %%------------------------------------------------------------------------- function val = parse_value(varargin) global pos inStr len true = 1; false = 0; switch(inStr(pos)) case {'S','C','H'} val = parseStr(varargin{:}); return; case '[' val = parse_array(varargin{:}); return; case '{' val = parse_object(varargin{:}); if isstruct(val) if(~isempty(strmatch('x0x5F_ArrayType_',fieldnames(val), 'exact'))) val=jstruct2array(val); end elseif isempty(val) val = struct; end return; case {'i','U','I','l','L','d','D'} val = parse_number(varargin{:}); return; case 'T' val = true; pos = pos + 1; return; case 'F' val = false; pos = pos + 1; return; case {'Z','N'} val = []; pos = pos + 1; return; end error_pos('Value expected at position %d'); %%------------------------------------------------------------------------- function error_pos(msg) global pos inStr len poShow = max(min([pos-15 pos-1 pos pos+20],len),1); if poShow(3) == poShow(2) poShow(3:4) = poShow(2)+[0 -1]; % display nothing after end msg = [sprintf(msg, pos) ': ' ... inStr(poShow(1):poShow(2)) '' inStr(poShow(3):poShow(4)) ]; error( ['JSONparser:invalidFormat: ' msg] ); %%------------------------------------------------------------------------- function str = valid_field(str) global isoct % From MATLAB doc: field names must begin with a letter, which may be % followed by any combination of letters, digits, and underscores. % Invalid characters will be converted to underscores, and the prefix % "x0x[Hex code]_" will be added if the first character is not a letter. pos=regexp(str,'^[^A-Za-z]','once'); if(~isempty(pos)) if(~isoct) str=regexprep(str,'^([^A-Za-z])','x0x${sprintf(''%X'',unicode2native($1))}_','once'); else str=sprintf('x0x%X_%s',char(str(1)),str(2:end)); end end if(isempty(regexp(str,'[^0-9A-Za-z_]', 'once' ))) return; end if(~isoct) str=regexprep(str,'([^0-9A-Za-z_])','_0x${sprintf(''%X'',unicode2native($1))}_'); else pos=regexp(str,'[^0-9A-Za-z_]'); if(isempty(pos)) return; end str0=str; pos0=[0 pos(:)' length(str)]; str=''; for i=1:length(pos) str=[str str0(pos0(i)+1:pos(i)-1) sprintf('_0x%X_',str0(pos(i)))]; end if(pos(end)~=length(str)) str=[str str0(pos0(end-1)+1:pos0(end))]; end end %str(~isletter(str) & ~('0' <= str & str <= '9')) = '_'; %%------------------------------------------------------------------------- function endpos = matching_quote(str,pos) len=length(str); while(pos1 && str(pos-1)=='\')) endpos=pos; return; end end pos=pos+1; end error('unmatched quotation mark'); %%------------------------------------------------------------------------- function [endpos e1l e1r maxlevel] = matching_bracket(str,pos) global arraytoken level=1; maxlevel=level; endpos=0; bpos=arraytoken(arraytoken>=pos); tokens=str(bpos); len=length(tokens); pos=1; e1l=[]; e1r=[]; while(pos<=len) c=tokens(pos); if(c==']') level=level-1; if(isempty(e1r)) e1r=bpos(pos); end if(level==0) endpos=bpos(pos); return end end if(c=='[') if(isempty(e1l)) e1l=bpos(pos); end level=level+1; maxlevel=max(maxlevel,level); end if(c=='"') pos=matching_quote(tokens,pos+1); end pos=pos+1; end if(endpos==0) error('unmatched "]"'); end ================================================ FILE: machine-learning-ex5/ex5/lib/jsonlab/mergestruct.m ================================================ function s=mergestruct(s1,s2) % % s=mergestruct(s1,s2) % % merge two struct objects into one % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % date: 2012/12/22 % % input: % s1,s2: a struct object, s1 and s2 can not be arrays % % output: % s: the merged struct object. fields in s1 and s2 will be combined in s. % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(~isstruct(s1) || ~isstruct(s2)) error('input parameters contain non-struct'); end if(length(s1)>1 || length(s2)>1) error('can not merge struct arrays'); end fn=fieldnames(s2); s=s1; for i=1:length(fn) s=setfield(s,fn{i},getfield(s2,fn{i})); end ================================================ FILE: machine-learning-ex5/ex5/lib/jsonlab/savejson.m ================================================ function json=savejson(rootname,obj,varargin) % % json=savejson(rootname,obj,filename) % or % json=savejson(rootname,obj,opt) % json=savejson(rootname,obj,'param1',value1,'param2',value2,...) % % convert a MATLAB object (cell, struct or array) into a JSON (JavaScript % Object Notation) string % % author: Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2011/09/09 % % $Id: savejson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % rootname: the name of the root-object, when set to '', the root name % is ignored, however, when opt.ForceRootName is set to 1 (see below), % the MATLAB variable name will be used as the root name. % obj: a MATLAB object (array, cell, cell array, struct, struct array). % filename: a string for the file name to save the output JSON data. % opt: a struct for additional options, ignore to use default values. % opt can have the following fields (first in [.|.] is the default) % % opt.FileName [''|string]: a file name to save the output JSON data % opt.FloatFormat ['%.10g'|string]: format to show each numeric element % of a 1D/2D array; % opt.ArrayIndent [1|0]: if 1, output explicit data array with % precedent indentation; if 0, no indentation % opt.ArrayToStruct[0|1]: when set to 0, savejson outputs 1D/2D % array in JSON array format; if sets to 1, an % array will be shown as a struct with fields % "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for % sparse arrays, the non-zero elements will be % saved to _ArrayData_ field in triplet-format i.e. % (ix,iy,val) and "_ArrayIsSparse_" will be added % with a value of 1; for a complex array, the % _ArrayData_ array will include two columns % (4 for sparse) to record the real and imaginary % parts, and also "_ArrayIsComplex_":1 is added. % opt.ParseLogical [0|1]: if this is set to 1, logical array elem % will use true/false rather than 1/0. % opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single % numerical element will be shown without a square % bracket, unless it is the root object; if 0, square % brackets are forced for any numerical arrays. % opt.ForceRootName [0|1]: when set to 1 and rootname is empty, savejson % will use the name of the passed obj variable as the % root object name; if obj is an expression and % does not have a name, 'root' will be used; if this % is set to 0 and rootname is empty, the root level % will be merged down to the lower level. % opt.Inf ['"$1_Inf_"'|string]: a customized regular expression pattern % to represent +/-Inf. The matched pattern is '([-+]*)Inf' % and $1 represents the sign. For those who want to use % 1e999 to represent Inf, they can set opt.Inf to '$11e999' % opt.NaN ['"_NaN_"'|string]: a customized regular expression pattern % to represent NaN % opt.JSONP [''|string]: to generate a JSONP output (JSON with padding), % for example, if opt.JSONP='foo', the JSON data is % wrapped inside a function call as 'foo(...);' % opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson % back to the string form % opt.SaveBinary [0|1]: 1 - save the JSON file in binary mode; 0 - text mode. % opt.Compact [0|1]: 1- out compact JSON format (remove all newlines and tabs) % % opt can be replaced by a list of ('param',value) pairs. The param % string is equivallent to a field in opt and is case sensitive. % output: % json: a string in the JSON format (see http://json.org) % % examples: % jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... % 'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],... % 'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;... % 2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],... % 'MeshCreator','FangQ','MeshTitle','T6 Cube',... % 'SpecialData',[nan, inf, -inf]); % savejson('jmesh',jsonmesh) % savejson('',jsonmesh,'ArrayIndent',0,'FloatFormat','\t%.5g') % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(nargin==1) varname=inputname(1); obj=rootname; if(isempty(varname)) varname='root'; end rootname=varname; else varname=inputname(2); end if(length(varargin)==1 && ischar(varargin{1})) opt=struct('FileName',varargin{1}); else opt=varargin2struct(varargin{:}); end opt.IsOctave=exist('OCTAVE_VERSION','builtin'); rootisarray=0; rootlevel=1; forceroot=jsonopt('ForceRootName',0,opt); if((isnumeric(obj) || islogical(obj) || ischar(obj) || isstruct(obj) || iscell(obj)) && isempty(rootname) && forceroot==0) rootisarray=1; rootlevel=0; else if(isempty(rootname)) rootname=varname; end end if((isstruct(obj) || iscell(obj))&& isempty(rootname) && forceroot) rootname='root'; end whitespaces=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); if(jsonopt('Compact',0,opt)==1) whitespaces=struct('tab','','newline','','sep',','); end if(~isfield(opt,'whitespaces_')) opt.whitespaces_=whitespaces; end nl=whitespaces.newline; json=obj2json(rootname,obj,rootlevel,opt); if(rootisarray) json=sprintf('%s%s',json,nl); else json=sprintf('{%s%s%s}\n',nl,json,nl); end jsonp=jsonopt('JSONP','',opt); if(~isempty(jsonp)) json=sprintf('%s(%s);%s',jsonp,json,nl); end % save to a file if FileName is set, suggested by Patrick Rapin if(~isempty(jsonopt('FileName','',opt))) if(jsonopt('SaveBinary',0,opt)==1) fid = fopen(opt.FileName, 'wb'); fwrite(fid,json); else fid = fopen(opt.FileName, 'wt'); fwrite(fid,json,'char'); end fclose(fid); end %%------------------------------------------------------------------------- function txt=obj2json(name,item,level,varargin) if(iscell(item)) txt=cell2json(name,item,level,varargin{:}); elseif(isstruct(item)) txt=struct2json(name,item,level,varargin{:}); elseif(ischar(item)) txt=str2json(name,item,level,varargin{:}); else txt=mat2json(name,item,level,varargin{:}); end %%------------------------------------------------------------------------- function txt=cell2json(name,item,level,varargin) txt=''; if(~iscell(item)) error('input is not a cell'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); ws=jsonopt('whitespaces_',struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')),varargin{:}); padding0=repmat(ws.tab,1,level); padding2=repmat(ws.tab,1,level+1); nl=ws.newline; if(len>1) if(~isempty(name)) txt=sprintf('%s"%s": [%s',padding0, checkname(name,varargin{:}),nl); name=''; else txt=sprintf('%s[%s',padding0,nl); end elseif(len==0) if(~isempty(name)) txt=sprintf('%s"%s": []',padding0, checkname(name,varargin{:})); name=''; else txt=sprintf('%s[]',padding0); end end for j=1:dim(2) if(dim(1)>1) txt=sprintf('%s%s[%s',txt,padding2,nl); end for i=1:dim(1) txt=sprintf('%s%s',txt,obj2json(name,item{i,j},level+(dim(1)>1)+1,varargin{:})); if(i1) txt=sprintf('%s%s%s]',txt,nl,padding2); end if(j1) txt=sprintf('%s%s%s]',txt,nl,padding0); end %%------------------------------------------------------------------------- function txt=struct2json(name,item,level,varargin) txt=''; if(~isstruct(item)) error('input is not a struct'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); ws=struct('tab',sprintf('\t'),'newline',sprintf('\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding0=repmat(ws.tab,1,level); padding2=repmat(ws.tab,1,level+1); padding1=repmat(ws.tab,1,level+(dim(1)>1)+(len>1)); nl=ws.newline; if(~isempty(name)) if(len>1) txt=sprintf('%s"%s": [%s',padding0,checkname(name,varargin{:}),nl); end else if(len>1) txt=sprintf('%s[%s',padding0,nl); end end for j=1:dim(2) if(dim(1)>1) txt=sprintf('%s%s[%s',txt,padding2,nl); end for i=1:dim(1) names = fieldnames(item(i,j)); if(~isempty(name) && len==1) txt=sprintf('%s%s"%s": {%s',txt,padding1, checkname(name,varargin{:}),nl); else txt=sprintf('%s%s{%s',txt,padding1,nl); end if(~isempty(names)) for e=1:length(names) txt=sprintf('%s%s',txt,obj2json(names{e},getfield(item(i,j),... names{e}),level+(dim(1)>1)+1+(len>1),varargin{:})); if(e1) txt=sprintf('%s%s%s]',txt,nl,padding2); end if(j1) txt=sprintf('%s%s%s]',txt,nl,padding0); end %%------------------------------------------------------------------------- function txt=str2json(name,item,level,varargin) txt=''; if(~ischar(item)) error('input is not a string'); end item=reshape(item, max(size(item),[1 0])); len=size(item,1); ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding1=repmat(ws.tab,1,level); padding0=repmat(ws.tab,1,level+1); nl=ws.newline; sep=ws.sep; if(~isempty(name)) if(len>1) txt=sprintf('%s"%s": [%s',padding1,checkname(name,varargin{:}),nl); end else if(len>1) txt=sprintf('%s[%s',padding1,nl); end end isoct=jsonopt('IsOctave',0,varargin{:}); for e=1:len if(isoct) val=regexprep(item(e,:),'\\','\\'); val=regexprep(val,'"','\"'); val=regexprep(val,'^"','\"'); else val=regexprep(item(e,:),'\\','\\\\'); val=regexprep(val,'"','\\"'); val=regexprep(val,'^"','\\"'); end val=escapejsonstring(val); if(len==1) obj=['"' checkname(name,varargin{:}) '": ' '"',val,'"']; if(isempty(name)) obj=['"',val,'"']; end txt=sprintf('%s%s%s%s',txt,padding1,obj); else txt=sprintf('%s%s%s%s',txt,padding0,['"',val,'"']); end if(e==len) sep=''; end txt=sprintf('%s%s',txt,sep); end if(len>1) txt=sprintf('%s%s%s%s',txt,nl,padding1,']'); end %%------------------------------------------------------------------------- function txt=mat2json(name,item,level,varargin) if(~isnumeric(item) && ~islogical(item)) error('input is not an array'); end ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding1=repmat(ws.tab,1,level); padding0=repmat(ws.tab,1,level+1); nl=ws.newline; sep=ws.sep; if(length(size(item))>2 || issparse(item) || ~isreal(item) || ... isempty(item) ||jsonopt('ArrayToStruct',0,varargin{:})) if(isempty(name)) txt=sprintf('%s{%s%s"_ArrayType_": "%s",%s%s"_ArraySize_": %s,%s',... padding1,nl,padding0,class(item),nl,padding0,regexprep(mat2str(size(item)),'\s+',','),nl); else txt=sprintf('%s"%s": {%s%s"_ArrayType_": "%s",%s%s"_ArraySize_": %s,%s',... padding1,checkname(name,varargin{:}),nl,padding0,class(item),nl,padding0,regexprep(mat2str(size(item)),'\s+',','),nl); end else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1 && level>0) numtxt=regexprep(regexprep(matdata2json(item,level+1,varargin{:}),'^\[',''),']',''); else numtxt=matdata2json(item,level+1,varargin{:}); end if(isempty(name)) txt=sprintf('%s%s',padding1,numtxt); else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1) txt=sprintf('%s"%s": %s',padding1,checkname(name,varargin{:}),numtxt); else txt=sprintf('%s"%s": %s',padding1,checkname(name,varargin{:}),numtxt); end end return; end dataformat='%s%s%s%s%s'; if(issparse(item)) [ix,iy]=find(item); data=full(item(find(item))); if(~isreal(item)) data=[real(data(:)),imag(data(:))]; if(size(item,1)==1) % Kludge to have data's 'transposedness' match item's. % (Necessary for complex row vector handling below.) data=data'; end txt=sprintf(dataformat,txt,padding0,'"_ArrayIsComplex_": ','1', sep); end txt=sprintf(dataformat,txt,padding0,'"_ArrayIsSparse_": ','1', sep); if(size(item,1)==1) % Row vector, store only column indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([iy(:),data'],level+2,varargin{:}), nl); elseif(size(item,2)==1) % Column vector, store only row indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([ix,data],level+2,varargin{:}), nl); else % General case, store row and column indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([ix,iy,data],level+2,varargin{:}), nl); end else if(isreal(item)) txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json(item(:)',level+2,varargin{:}), nl); else txt=sprintf(dataformat,txt,padding0,'"_ArrayIsComplex_": ','1', sep); txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([real(item(:)) imag(item(:))],level+2,varargin{:}), nl); end end txt=sprintf('%s%s%s',txt,padding1,'}'); %%------------------------------------------------------------------------- function txt=matdata2json(mat,level,varargin) ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); tab=ws.tab; nl=ws.newline; if(size(mat,1)==1) pre=''; post=''; level=level-1; else pre=sprintf('[%s',nl); post=sprintf('%s%s]',nl,repmat(tab,1,level-1)); end if(isempty(mat)) txt='null'; return; end floatformat=jsonopt('FloatFormat','%.10g',varargin{:}); %if(numel(mat)>1) formatstr=['[' repmat([floatformat ','],1,size(mat,2)-1) [floatformat sprintf('],%s',nl)]]; %else % formatstr=[repmat([floatformat ','],1,size(mat,2)-1) [floatformat sprintf(',\n')]]; %end if(nargin>=2 && size(mat,1)>1 && jsonopt('ArrayIndent',1,varargin{:})==1) formatstr=[repmat(tab,1,level) formatstr]; end txt=sprintf(formatstr,mat'); txt(end-length(nl):end)=[]; if(islogical(mat) && jsonopt('ParseLogical',0,varargin{:})==1) txt=regexprep(txt,'1','true'); txt=regexprep(txt,'0','false'); end %txt=regexprep(mat2str(mat),'\s+',','); %txt=regexprep(txt,';',sprintf('],\n[')); % if(nargin>=2 && size(mat,1)>1) % txt=regexprep(txt,'\[',[repmat(sprintf('\t'),1,level) '[']); % end txt=[pre txt post]; if(any(isinf(mat(:)))) txt=regexprep(txt,'([-+]*)Inf',jsonopt('Inf','"$1_Inf_"',varargin{:})); end if(any(isnan(mat(:)))) txt=regexprep(txt,'NaN',jsonopt('NaN','"_NaN_"',varargin{:})); end %%------------------------------------------------------------------------- function newname=checkname(name,varargin) isunpack=jsonopt('UnpackHex',1,varargin{:}); newname=name; if(isempty(regexp(name,'0x([0-9a-fA-F]+)_','once'))) return end if(isunpack) isoct=jsonopt('IsOctave',0,varargin{:}); if(~isoct) newname=regexprep(name,'(^x|_){1}0x([0-9a-fA-F]+)_','${native2unicode(hex2dec($2))}'); else pos=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','start'); pend=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','end'); if(isempty(pos)) return; end str0=name; pos0=[0 pend(:)' length(name)]; newname=''; for i=1:length(pos) newname=[newname str0(pos0(i)+1:pos(i)-1) char(hex2dec(str0(pos(i)+3:pend(i)-1)))]; end if(pos(end)~=length(name)) newname=[newname str0(pos0(end-1)+1:pos0(end))]; end end end %%------------------------------------------------------------------------- function newstr=escapejsonstring(str) newstr=str; isoct=exist('OCTAVE_VERSION','builtin'); if(isoct) vv=sscanf(OCTAVE_VERSION,'%f'); if(vv(1)>=3.8) isoct=0; end end if(isoct) escapechars={'\a','\f','\n','\r','\t','\v'}; for i=1:length(escapechars); newstr=regexprep(newstr,escapechars{i},escapechars{i}); end else escapechars={'\a','\b','\f','\n','\r','\t','\v'}; for i=1:length(escapechars); newstr=regexprep(newstr,escapechars{i},regexprep(escapechars{i},'\\','\\\\')); end end ================================================ FILE: machine-learning-ex5/ex5/lib/jsonlab/saveubjson.m ================================================ function json=saveubjson(rootname,obj,varargin) % % json=saveubjson(rootname,obj,filename) % or % json=saveubjson(rootname,obj,opt) % json=saveubjson(rootname,obj,'param1',value1,'param2',value2,...) % % convert a MATLAB object (cell, struct or array) into a Universal % Binary JSON (UBJSON) binary string % % author: Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2013/08/17 % % $Id: saveubjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % rootname: the name of the root-object, when set to '', the root name % is ignored, however, when opt.ForceRootName is set to 1 (see below), % the MATLAB variable name will be used as the root name. % obj: a MATLAB object (array, cell, cell array, struct, struct array) % filename: a string for the file name to save the output UBJSON data % opt: a struct for additional options, ignore to use default values. % opt can have the following fields (first in [.|.] is the default) % % opt.FileName [''|string]: a file name to save the output JSON data % opt.ArrayToStruct[0|1]: when set to 0, saveubjson outputs 1D/2D % array in JSON array format; if sets to 1, an % array will be shown as a struct with fields % "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for % sparse arrays, the non-zero elements will be % saved to _ArrayData_ field in triplet-format i.e. % (ix,iy,val) and "_ArrayIsSparse_" will be added % with a value of 1; for a complex array, the % _ArrayData_ array will include two columns % (4 for sparse) to record the real and imaginary % parts, and also "_ArrayIsComplex_":1 is added. % opt.ParseLogical [1|0]: if this is set to 1, logical array elem % will use true/false rather than 1/0. % opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single % numerical element will be shown without a square % bracket, unless it is the root object; if 0, square % brackets are forced for any numerical arrays. % opt.ForceRootName [0|1]: when set to 1 and rootname is empty, saveubjson % will use the name of the passed obj variable as the % root object name; if obj is an expression and % does not have a name, 'root' will be used; if this % is set to 0 and rootname is empty, the root level % will be merged down to the lower level. % opt.JSONP [''|string]: to generate a JSONP output (JSON with padding), % for example, if opt.JSON='foo', the JSON data is % wrapped inside a function call as 'foo(...);' % opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson % back to the string form % % opt can be replaced by a list of ('param',value) pairs. The param % string is equivallent to a field in opt and is case sensitive. % output: % json: a binary string in the UBJSON format (see http://ubjson.org) % % examples: % jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... % 'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],... % 'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;... % 2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],... % 'MeshCreator','FangQ','MeshTitle','T6 Cube',... % 'SpecialData',[nan, inf, -inf]); % saveubjson('jsonmesh',jsonmesh) % saveubjson('jsonmesh',jsonmesh,'meshdata.ubj') % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(nargin==1) varname=inputname(1); obj=rootname; if(isempty(varname)) varname='root'; end rootname=varname; else varname=inputname(2); end if(length(varargin)==1 && ischar(varargin{1})) opt=struct('FileName',varargin{1}); else opt=varargin2struct(varargin{:}); end opt.IsOctave=exist('OCTAVE_VERSION','builtin'); rootisarray=0; rootlevel=1; forceroot=jsonopt('ForceRootName',0,opt); if((isnumeric(obj) || islogical(obj) || ischar(obj) || isstruct(obj) || iscell(obj)) && isempty(rootname) && forceroot==0) rootisarray=1; rootlevel=0; else if(isempty(rootname)) rootname=varname; end end if((isstruct(obj) || iscell(obj))&& isempty(rootname) && forceroot) rootname='root'; end json=obj2ubjson(rootname,obj,rootlevel,opt); if(~rootisarray) json=['{' json '}']; end jsonp=jsonopt('JSONP','',opt); if(~isempty(jsonp)) json=[jsonp '(' json ')']; end % save to a file if FileName is set, suggested by Patrick Rapin if(~isempty(jsonopt('FileName','',opt))) fid = fopen(opt.FileName, 'wb'); fwrite(fid,json); fclose(fid); end %%------------------------------------------------------------------------- function txt=obj2ubjson(name,item,level,varargin) if(iscell(item)) txt=cell2ubjson(name,item,level,varargin{:}); elseif(isstruct(item)) txt=struct2ubjson(name,item,level,varargin{:}); elseif(ischar(item)) txt=str2ubjson(name,item,level,varargin{:}); else txt=mat2ubjson(name,item,level,varargin{:}); end %%------------------------------------------------------------------------- function txt=cell2ubjson(name,item,level,varargin) txt=''; if(~iscell(item)) error('input is not a cell'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); % let's handle 1D cell first if(len>1) if(~isempty(name)) txt=[S_(checkname(name,varargin{:})) '[']; name=''; else txt='['; end elseif(len==0) if(~isempty(name)) txt=[S_(checkname(name,varargin{:})) 'Z']; name=''; else txt='Z'; end end for j=1:dim(2) if(dim(1)>1) txt=[txt '[']; end for i=1:dim(1) txt=[txt obj2ubjson(name,item{i,j},level+(len>1),varargin{:})]; end if(dim(1)>1) txt=[txt ']']; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=struct2ubjson(name,item,level,varargin) txt=''; if(~isstruct(item)) error('input is not a struct'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); if(~isempty(name)) if(len>1) txt=[S_(checkname(name,varargin{:})) '[']; end else if(len>1) txt='['; end end for j=1:dim(2) if(dim(1)>1) txt=[txt '[']; end for i=1:dim(1) names = fieldnames(item(i,j)); if(~isempty(name) && len==1) txt=[txt S_(checkname(name,varargin{:})) '{']; else txt=[txt '{']; end if(~isempty(names)) for e=1:length(names) txt=[txt obj2ubjson(names{e},getfield(item(i,j),... names{e}),level+(dim(1)>1)+1+(len>1),varargin{:})]; end end txt=[txt '}']; end if(dim(1)>1) txt=[txt ']']; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=str2ubjson(name,item,level,varargin) txt=''; if(~ischar(item)) error('input is not a string'); end item=reshape(item, max(size(item),[1 0])); len=size(item,1); if(~isempty(name)) if(len>1) txt=[S_(checkname(name,varargin{:})) '[']; end else if(len>1) txt='['; end end isoct=jsonopt('IsOctave',0,varargin{:}); for e=1:len val=item(e,:); if(len==1) obj=['' S_(checkname(name,varargin{:})) '' '',S_(val),'']; if(isempty(name)) obj=['',S_(val),'']; end txt=[txt,'',obj]; else txt=[txt,'',['',S_(val),'']]; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=mat2ubjson(name,item,level,varargin) if(~isnumeric(item) && ~islogical(item)) error('input is not an array'); end if(length(size(item))>2 || issparse(item) || ~isreal(item) || ... isempty(item) || jsonopt('ArrayToStruct',0,varargin{:})) cid=I_(uint32(max(size(item)))); if(isempty(name)) txt=['{' S_('_ArrayType_'),S_(class(item)),S_('_ArraySize_'),I_a(size(item),cid(1)) ]; else if(isempty(item)) txt=[S_(checkname(name,varargin{:})),'Z']; return; else txt=[S_(checkname(name,varargin{:})),'{',S_('_ArrayType_'),S_(class(item)),S_('_ArraySize_'),I_a(size(item),cid(1))]; end end else if(isempty(name)) txt=matdata2ubjson(item,level+1,varargin{:}); else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1) numtxt=regexprep(regexprep(matdata2ubjson(item,level+1,varargin{:}),'^\[',''),']',''); txt=[S_(checkname(name,varargin{:})) numtxt]; else txt=[S_(checkname(name,varargin{:})),matdata2ubjson(item,level+1,varargin{:})]; end end return; end if(issparse(item)) [ix,iy]=find(item); data=full(item(find(item))); if(~isreal(item)) data=[real(data(:)),imag(data(:))]; if(size(item,1)==1) % Kludge to have data's 'transposedness' match item's. % (Necessary for complex row vector handling below.) data=data'; end txt=[txt,S_('_ArrayIsComplex_'),'T']; end txt=[txt,S_('_ArrayIsSparse_'),'T']; if(size(item,1)==1) % Row vector, store only column indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([iy(:),data'],level+2,varargin{:})]; elseif(size(item,2)==1) % Column vector, store only row indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([ix,data],level+2,varargin{:})]; else % General case, store row and column indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([ix,iy,data],level+2,varargin{:})]; end else if(isreal(item)) txt=[txt,S_('_ArrayData_'),... matdata2ubjson(item(:)',level+2,varargin{:})]; else txt=[txt,S_('_ArrayIsComplex_'),'T']; txt=[txt,S_('_ArrayData_'),... matdata2ubjson([real(item(:)) imag(item(:))],level+2,varargin{:})]; end end txt=[txt,'}']; %%------------------------------------------------------------------------- function txt=matdata2ubjson(mat,level,varargin) if(isempty(mat)) txt='Z'; return; end if(size(mat,1)==1) level=level-1; end type=''; hasnegtive=(mat<0); if(isa(mat,'integer') || isinteger(mat) || (isfloat(mat) && all(mod(mat(:),1) == 0))) if(isempty(hasnegtive)) if(max(mat(:))<=2^8) type='U'; end end if(isempty(type)) % todo - need to consider negative ones separately id= histc(abs(max(mat(:))),[0 2^7 2^15 2^31 2^63]); if(isempty(find(id))) error('high-precision data is not yet supported'); end key='iIlL'; type=key(find(id)); end txt=[I_a(mat(:),type,size(mat))]; elseif(islogical(mat)) logicalval='FT'; if(numel(mat)==1) txt=logicalval(mat+1); else txt=['[$U#' I_a(size(mat),'l') typecast(swapbytes(uint8(mat(:)')),'uint8')]; end else if(numel(mat)==1) txt=['[' D_(mat) ']']; else txt=D_a(mat(:),'D',size(mat)); end end %txt=regexprep(mat2str(mat),'\s+',','); %txt=regexprep(txt,';',sprintf('],[')); % if(nargin>=2 && size(mat,1)>1) % txt=regexprep(txt,'\[',[repmat(sprintf('\t'),1,level) '[']); % end if(any(isinf(mat(:)))) txt=regexprep(txt,'([-+]*)Inf',jsonopt('Inf','"$1_Inf_"',varargin{:})); end if(any(isnan(mat(:)))) txt=regexprep(txt,'NaN',jsonopt('NaN','"_NaN_"',varargin{:})); end %%------------------------------------------------------------------------- function newname=checkname(name,varargin) isunpack=jsonopt('UnpackHex',1,varargin{:}); newname=name; if(isempty(regexp(name,'0x([0-9a-fA-F]+)_','once'))) return end if(isunpack) isoct=jsonopt('IsOctave',0,varargin{:}); if(~isoct) newname=regexprep(name,'(^x|_){1}0x([0-9a-fA-F]+)_','${native2unicode(hex2dec($2))}'); else pos=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','start'); pend=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','end'); if(isempty(pos)) return; end str0=name; pos0=[0 pend(:)' length(name)]; newname=''; for i=1:length(pos) newname=[newname str0(pos0(i)+1:pos(i)-1) char(hex2dec(str0(pos(i)+3:pend(i)-1)))]; end if(pos(end)~=length(name)) newname=[newname str0(pos0(end-1)+1:pos0(end))]; end end end %%------------------------------------------------------------------------- function val=S_(str) if(length(str)==1) val=['C' str]; else val=['S' I_(int32(length(str))) str]; end %%------------------------------------------------------------------------- function val=I_(num) if(~isinteger(num)) error('input is not an integer'); end if(num>=0 && num<255) val=['U' data2byte(swapbytes(cast(num,'uint8')),'uint8')]; return; end key='iIlL'; cid={'int8','int16','int32','int64'}; for i=1:4 if((num>0 && num<2^(i*8-1)) || (num<0 && num>=-2^(i*8-1))) val=[key(i) data2byte(swapbytes(cast(num,cid{i})),'uint8')]; return; end end error('unsupported integer'); %%------------------------------------------------------------------------- function val=D_(num) if(~isfloat(num)) error('input is not a float'); end if(isa(num,'single')) val=['d' data2byte(num,'uint8')]; else val=['D' data2byte(num,'uint8')]; end %%------------------------------------------------------------------------- function data=I_a(num,type,dim,format) id=find(ismember('iUIlL',type)); if(id==0) error('unsupported integer array'); end % based on UBJSON specs, all integer types are stored in big endian format if(id==1) data=data2byte(swapbytes(int8(num)),'uint8'); blen=1; elseif(id==2) data=data2byte(swapbytes(uint8(num)),'uint8'); blen=1; elseif(id==3) data=data2byte(swapbytes(int16(num)),'uint8'); blen=2; elseif(id==4) data=data2byte(swapbytes(int32(num)),'uint8'); blen=4; elseif(id==5) data=data2byte(swapbytes(int64(num)),'uint8'); blen=8; end if(nargin>=3 && length(dim)>=2 && prod(dim)~=dim(2)) format='opt'; end if((nargin<4 || strcmp(format,'opt')) && numel(num)>1) if(nargin>=3 && (length(dim)==1 || (length(dim)>=2 && prod(dim)~=dim(2)))) cid=I_(uint32(max(dim))); data=['$' type '#' I_a(dim,cid(1)) data(:)']; else data=['$' type '#' I_(int32(numel(data)/blen)) data(:)']; end data=['[' data(:)']; else data=reshape(data,blen,numel(data)/blen); data(2:blen+1,:)=data; data(1,:)=type; data=data(:)'; data=['[' data(:)' ']']; end %%------------------------------------------------------------------------- function data=D_a(num,type,dim,format) id=find(ismember('dD',type)); if(id==0) error('unsupported float array'); end if(id==1) data=data2byte(single(num),'uint8'); elseif(id==2) data=data2byte(double(num),'uint8'); end if(nargin>=3 && length(dim)>=2 && prod(dim)~=dim(2)) format='opt'; end if((nargin<4 || strcmp(format,'opt')) && numel(num)>1) if(nargin>=3 && (length(dim)==1 || (length(dim)>=2 && prod(dim)~=dim(2)))) cid=I_(uint32(max(dim))); data=['$' type '#' I_a(dim,cid(1)) data(:)']; else data=['$' type '#' I_(int32(numel(data)/(id*4))) data(:)']; end data=['[' data]; else data=reshape(data,(id*4),length(data)/(id*4)); data(2:(id*4+1),:)=data; data(1,:)=type; data=data(:)'; data=['[' data(:)' ']']; end %%------------------------------------------------------------------------- function bytes=data2byte(varargin) bytes=typecast(varargin{:}); bytes=bytes(:)'; ================================================ FILE: machine-learning-ex5/ex5/lib/jsonlab/varargin2struct.m ================================================ function opt=varargin2struct(varargin) % % opt=varargin2struct('param1',value1,'param2',value2,...) % or % opt=varargin2struct(...,optstruct,...) % % convert a series of input parameters into a structure % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % date: 2012/12/22 % % input: % 'param', value: the input parameters should be pairs of a string and a value % optstruct: if a parameter is a struct, the fields will be merged to the output struct % % output: % opt: a struct where opt.param1=value1, opt.param2=value2 ... % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % len=length(varargin); opt=struct; if(len==0) return; end i=1; while(i<=len) if(isstruct(varargin{i})) opt=mergestruct(opt,varargin{i}); elseif(ischar(varargin{i}) && i Anyone knows how much it costs to host a web portal ? > Well, it depends on how many visitors you're expecting. This can be anywhere from less than 10 bucks a month to a couple of $100. You should checkout http://www.rackspace.com/ or perhaps Amazon EC2 if youre running something big.. To unsubscribe yourself from this mailing list, send an email to: groupname-unsubscribe@egroups.com ================================================ FILE: machine-learning-ex6/ex6/emailSample2.txt ================================================ Folks, my first time posting - have a bit of Unix experience, but am new to Linux. Just got a new PC at home - Dell box with Windows XP. Added a second hard disk for Linux. Partitioned the disk and have installed Suse 7.2 from CD, which went fine except it didn't pick up my monitor. I have a Dell branded E151FPp 15" LCD flat panel monitor and a nVidia GeForce4 Ti4200 video card, both of which are probably too new to feature in Suse's default set. I downloaded a driver from the nVidia website and installed it using RPM. Then I ran Sax2 (as was recommended in some postings I found on the net), but it still doesn't feature my video card in the available list. What next? Another problem. I have a Dell branded keyboard and if I hit Caps-Lock twice, the whole machine crashes (in Linux, not Windows) - even the on/off switch is inactive, leaving me to reach for the power cable instead. If anyone can help me in any way with these probs., I'd be really grateful - I've searched the 'net but have run out of ideas. Or should I be going for a different version of Linux such as RedHat? Opinions welcome. Thanks a lot, Peter -- Irish Linux Users' Group: ilug@linux.ie http://www.linux.ie/mailman/listinfo/ilug for (un)subscription information. List maintainer: listmaster@linux.ie ================================================ FILE: machine-learning-ex6/ex6/ex6.m ================================================ %% Machine Learning Online Class % Exercise 6 | Support Vector Machines % % Instructions % ------------ % % This file contains code that helps you get started on the % exercise. You will need to complete the following functions: % % gaussianKernel.m % dataset3Params.m % processEmail.m % emailFeatures.m % % For this exercise, you will not need to change any code in this file, % or any other files other than those mentioned above. % %% Initialization clear ; close all; clc %% =============== Part 1: Loading and Visualizing Data ================ % We start the exercise by first loading and visualizing the dataset. % The following code will load the dataset into your environment and plot % the data. % fprintf('Loading and Visualizing Data ...\n') % Load from ex6data1: % You will have X, y in your environment load('ex6data1.mat'); % Plot training data plotData(X, y); fprintf('Program paused. Press enter to continue.\n'); pause; %% ==================== Part 2: Training Linear SVM ==================== % The following code will train a linear SVM on the dataset and plot the % decision boundary learned. % % Load from ex6data1: % You will have X, y in your environment load('ex6data1.mat'); fprintf('\nTraining Linear SVM ...\n') % You should try to change the C value below and see how the decision % boundary varies (e.g., try C = 1000) C = 1; model = svmTrain(X, y, C, @linearKernel, 1e-3, 20); visualizeBoundaryLinear(X, y, model); fprintf('Program paused. Press enter to continue.\n'); pause; %% =============== Part 3: Implementing Gaussian Kernel =============== % You will now implement the Gaussian kernel to use % with the SVM. You should complete the code in gaussianKernel.m % fprintf('\nEvaluating the Gaussian Kernel ...\n') x1 = [1 2 1]; x2 = [0 4 -1]; sigma = 2; sim = gaussianKernel(x1, x2, sigma); fprintf(['Gaussian Kernel between x1 = [1; 2; 1], x2 = [0; 4; -1], sigma = 0.5 :' ... '\n\t%f\n(this value should be about 0.324652)\n'], sim); fprintf('Program paused. Press enter to continue.\n'); pause; %% =============== Part 4: Visualizing Dataset 2 ================ % The following code will load the next dataset into your environment and % plot the data. % fprintf('Loading and Visualizing Data ...\n') % Load from ex6data2: % You will have X, y in your environment load('ex6data2.mat'); % Plot training data plotData(X, y); fprintf('Program paused. Press enter to continue.\n'); pause; %% ========== Part 5: Training SVM with RBF Kernel (Dataset 2) ========== % After you have implemented the kernel, we can now use it to train the % SVM classifier. % fprintf('\nTraining SVM with RBF Kernel (this may take 1 to 2 minutes) ...\n'); % Load from ex6data2: % You will have X, y in your environment load('ex6data2.mat'); % SVM Parameters C = 1; sigma = 0.1; % We set the tolerance and max_passes lower here so that the code will run % faster. However, in practice, you will want to run the training to % convergence. model= svmTrain(X, y, C, @(x1, x2) gaussianKernel(x1, x2, sigma)); visualizeBoundary(X, y, model); fprintf('Program paused. Press enter to continue.\n'); pause; %% =============== Part 6: Visualizing Dataset 3 ================ % The following code will load the next dataset into your environment and % plot the data. % fprintf('Loading and Visualizing Data ...\n') % Load from ex6data3: % You will have X, y in your environment load('ex6data3.mat'); % Plot training data plotData(X, y); fprintf('Program paused. Press enter to continue.\n'); pause; %% ========== Part 7: Training SVM with RBF Kernel (Dataset 3) ========== % This is a different dataset that you can use to experiment with. Try % different values of C and sigma here. % % Load from ex6data3: % You will have X, y in your environment load('ex6data3.mat'); % Try different SVM Parameters here [C, sigma] = dataset3Params(X, y, Xval, yval); % Train the SVM model= svmTrain(X, y, C, @(x1, x2) gaussianKernel(x1, x2, sigma)); visualizeBoundary(X, y, model); fprintf('Program paused. Press enter to continue.\n'); pause; ================================================ FILE: machine-learning-ex6/ex6/ex6_spam.m ================================================ %% Machine Learning Online Class % Exercise 6 | Spam Classification with SVMs % % Instructions % ------------ % % This file contains code that helps you get started on the % exercise. You will need to complete the following functions: % % gaussianKernel.m % dataset3Params.m % processEmail.m % emailFeatures.m % % For this exercise, you will not need to change any code in this file, % or any other files other than those mentioned above. % %% Initialization clear ; close all; clc %% ==================== Part 1: Email Preprocessing ==================== % To use an SVM to classify emails into Spam v.s. Non-Spam, you first need % to convert each email into a vector of features. In this part, you will % implement the preprocessing steps for each email. You should % complete the code in processEmail.m to produce a word indices vector % for a given email. fprintf('\nPreprocessing sample email (emailSample1.txt)\n'); % Extract Features file_contents = readFile('emailSample1.txt'); word_indices = processEmail(file_contents); % Print Stats fprintf('Word Indices: \n'); fprintf(' %d', word_indices); fprintf('\n\n'); fprintf('Program paused. Press enter to continue.\n'); pause; %% ==================== Part 2: Feature Extraction ==================== % Now, you will convert each email into a vector of features in R^n. % You should complete the code in emailFeatures.m to produce a feature % vector for a given email. fprintf('\nExtracting features from sample email (emailSample1.txt)\n'); % Extract Features file_contents = readFile('emailSample1.txt'); word_indices = processEmail(file_contents); features = emailFeatures(word_indices); % Print Stats fprintf('Length of feature vector: %d\n', length(features)); fprintf('Number of non-zero entries: %d\n', sum(features > 0)); fprintf('Program paused. Press enter to continue.\n'); pause; %% =========== Part 3: Train Linear SVM for Spam Classification ======== % In this section, you will train a linear classifier to determine if an % email is Spam or Not-Spam. % Load the Spam Email dataset % You will have X, y in your environment load('spamTrain.mat'); fprintf('\nTraining Linear SVM (Spam Classification)\n') fprintf('(this may take 1 to 2 minutes) ...\n') C = 0.1; model = svmTrain(X, y, C, @linearKernel); p = svmPredict(model, X); fprintf('Training Accuracy: %f\n', mean(double(p == y)) * 100); %% =================== Part 4: Test Spam Classification ================ % After training the classifier, we can evaluate it on a test set. We have % included a test set in spamTest.mat % Load the test dataset % You will have Xtest, ytest in your environment load('spamTest.mat'); fprintf('\nEvaluating the trained Linear SVM on a test set ...\n') p = svmPredict(model, Xtest); fprintf('Test Accuracy: %f\n', mean(double(p == ytest)) * 100); pause; %% ================= Part 5: Top Predictors of Spam ==================== % Since the model we are training is a linear SVM, we can inspect the % weights learned by the model to understand better how it is determining % whether an email is spam or not. The following code finds the words with % the highest weights in the classifier. Informally, the classifier % 'thinks' that these words are the most likely indicators of spam. % % Sort the weights and obtin the vocabulary list [weight, idx] = sort(model.w, 'descend'); vocabList = getVocabList(); fprintf('\nTop predictors of spam: \n'); for i = 1:15 fprintf(' %-15s (%f) \n', vocabList{idx(i)}, weight(i)); end fprintf('\n\n'); fprintf('\nProgram paused. Press enter to continue.\n'); pause; %% =================== Part 6: Try Your Own Emails ===================== % Now that you've trained the spam classifier, you can use it on your own % emails! In the starter code, we have included spamSample1.txt, % spamSample2.txt, emailSample1.txt and emailSample2.txt as examples. % The following code reads in one of these emails and then uses your % learned SVM classifier to determine whether the email is Spam or % Not Spam % Set the file to be read in (change this to spamSample2.txt, % emailSample1.txt or emailSample2.txt to see different predictions on % different emails types). Try your own emails as well! filename = 'spamSample1.txt'; % Read and predict file_contents = readFile(filename); word_indices = processEmail(file_contents); x = emailFeatures(word_indices); p = svmPredict(model, x); fprintf('\nProcessed %s\n\nSpam Classification: %d\n', filename, p); fprintf('(1 indicates spam, 0 indicates not spam)\n\n'); ================================================ FILE: machine-learning-ex6/ex6/gaussianKernel.m ================================================ function sim = gaussianKernel(x1, x2, sigma) %RBFKERNEL returns a radial basis function kernel between x1 and x2 % sim = gaussianKernel(x1, x2) returns a gaussian kernel between x1 and x2 % and returns the value in sim % Ensure that x1 and x2 are column vectors x1 = x1(:); x2 = x2(:); % You need to return the following variables correctly. sim = 0; % ====================== YOUR CODE HERE ====================== % Instructions: Fill in this function to return the similarity between x1 % and x2 computed using a Gaussian kernel with bandwidth % sigma % % temp = (x1-x2)'*(x1-x2); sim = exp(-temp/(2*(sigma^2))); % ============================================================= end ================================================ FILE: machine-learning-ex6/ex6/getVocabList.m ================================================ function vocabList = getVocabList() %GETVOCABLIST reads the fixed vocabulary list in vocab.txt and returns a %cell array of the words % vocabList = GETVOCABLIST() reads the fixed vocabulary list in vocab.txt % and returns a cell array of the words in vocabList. %% Read the fixed vocabulary list fid = fopen('vocab.txt'); % Store all dictionary words in cell array vocab{} n = 1899; % Total number of words in the dictionary % For ease of implementation, we use a struct to map the strings => integers % In practice, you'll want to use some form of hashmap vocabList = cell(n, 1); for i = 1:n % Word Index (can ignore since it will be = i) fscanf(fid, '%d', 1); % Actual Word vocabList{i} = fscanf(fid, '%s', 1); end fclose(fid); end ================================================ FILE: machine-learning-ex6/ex6/lib/jsonlab/AUTHORS.txt ================================================ The author of "jsonlab" toolbox is Qianqian Fang. Qianqian is currently an Assistant Professor at Massachusetts General Hospital, Harvard Medical School. Address: Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School Bldg 149, 13th St, Charlestown, MA 02129, USA URL: http://nmr.mgh.harvard.edu/~fangq/ Email: or The script loadjson.m was built upon previous works by - Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713 date: 2009/11/02 - François Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393 date: 2009/03/22 - Joel Feenstra: http://www.mathworks.com/matlabcentral/fileexchange/20565 date: 2008/07/03 This toolbox contains patches submitted by the following contributors: - Blake Johnson part of revision 341 - Niclas Borlin various fixes in revision 394, including - loadjson crashes for all-zero sparse matrix. - loadjson crashes for empty sparse matrix. - Non-zero size of 0-by-N and N-by-0 empty matrices is lost after savejson/loadjson. - loadjson crashes for sparse real column vector. - loadjson crashes for sparse complex column vector. - Data is corrupted by savejson for sparse real row vector. - savejson crashes for sparse complex row vector. - Yul Kang patches for svn revision 415. - savejson saves an empty cell array as [] instead of null - loadjson differentiates an empty struct from an empty array ================================================ FILE: machine-learning-ex6/ex6/lib/jsonlab/ChangeLog.txt ================================================ ============================================================================ JSONlab - a toolbox to encode/decode JSON/UBJSON files in MATLAB/Octave ---------------------------------------------------------------------------- JSONlab ChangeLog (key features marked by *): == JSONlab 1.0 (codename: Optimus - Final), FangQ == 2015/01/02 polish help info for all major functions, update examples, finalize 1.0 2014/12/19 fix a bug to strictly respect NoRowBracket in savejson == JSONlab 1.0.0-RC2 (codename: Optimus - RC2), FangQ == 2014/11/22 show progress bar in loadjson ('ShowProgress') 2014/11/17 add Compact option in savejson to output compact JSON format ('Compact') 2014/11/17 add FastArrayParser in loadjson to specify fast parser applicable levels 2014/09/18 start official github mirror: https://github.com/fangq/jsonlab == JSONlab 1.0.0-RC1 (codename: Optimus - RC1), FangQ == 2014/09/17 fix several compatibility issues when running on octave versions 3.2-3.8 2014/09/17 support 2D cell and struct arrays in both savejson and saveubjson 2014/08/04 escape special characters in a JSON string 2014/02/16 fix a bug when saving ubjson files == JSONlab 0.9.9 (codename: Optimus - beta), FangQ == 2014/01/22 use binary read and write in saveubjson and loadubjson == JSONlab 0.9.8-1 (codename: Optimus - alpha update 1), FangQ == 2013/10/07 better round-trip conservation for empty arrays and structs (patch submitted by Yul Kang) == JSONlab 0.9.8 (codename: Optimus - alpha), FangQ == 2013/08/23 *universal Binary JSON (UBJSON) support, including both saveubjson and loadubjson == JSONlab 0.9.1 (codename: Rodimus, update 1), FangQ == 2012/12/18 *handling of various empty and sparse matrices (fixes submitted by Niclas Borlin) == JSONlab 0.9.0 (codename: Rodimus), FangQ == 2012/06/17 *new format for an invalid leading char, unpacking hex code in savejson 2012/06/01 support JSONP in savejson 2012/05/25 fix the empty cell bug (reported by Cyril Davin) 2012/04/05 savejson can save to a file (suggested by Patrick Rapin) == JSONlab 0.8.1 (codename: Sentiel, Update 1), FangQ == 2012/02/28 loadjson quotation mark escape bug, see http://bit.ly/yyk1nS 2012/01/25 patch to handle root-less objects, contributed by Blake Johnson == JSONlab 0.8.0 (codename: Sentiel), FangQ == 2012/01/13 *speed up loadjson by 20 fold when parsing large data arrays in matlab 2012/01/11 remove row bracket if an array has 1 element, suggested by Mykel Kochenderfer 2011/12/22 *accept sequence of 'param',value input in savejson and loadjson 2011/11/18 fix struct array bug reported by Mykel Kochenderfer == JSONlab 0.5.1 (codename: Nexus Update 1), FangQ == 2011/10/21 fix a bug in loadjson, previous code does not use any of the acceleration 2011/10/20 loadjson supports JSON collections - concatenated JSON objects == JSONlab 0.5.0 (codename: Nexus), FangQ == 2011/10/16 package and release jsonlab 0.5.0 2011/10/15 *add json demo and regression test, support cpx numbers, fix double quote bug 2011/10/11 *speed up readjson dramatically, interpret _Array* tags, show data in root level 2011/10/10 create jsonlab project, start jsonlab website, add online documentation 2011/10/07 *speed up savejson by 25x using sprintf instead of mat2str, add options support 2011/10/06 *savejson works for structs, cells and arrays 2011/09/09 derive loadjson from JSON parser from MATLAB Central, draft savejson.m ================================================ FILE: machine-learning-ex6/ex6/lib/jsonlab/LICENSE_BSD.txt ================================================ Copyright 2011-2015 Qianqian Fang . All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. The views and conclusions contained in the software and documentation are those of the authors and should not be interpreted as representing official policies, either expressed or implied, of the copyright holders. ================================================ FILE: machine-learning-ex6/ex6/lib/jsonlab/README.txt ================================================ =============================================================================== = JSONLab = = An open-source MATLAB/Octave JSON encoder and decoder = =============================================================================== *Copyright (C) 2011-2015 Qianqian Fang *License: BSD License, see License_BSD.txt for details *Version: 1.0 (Optimus - Final) ------------------------------------------------------------------------------- Table of Content: I. Introduction II. Installation III.Using JSONLab IV. Known Issues and TODOs V. Contribution and feedback ------------------------------------------------------------------------------- I. Introduction JSON ([http://www.json.org/ JavaScript Object Notation]) is a highly portable, human-readable and "[http://en.wikipedia.org/wiki/JSON fat-free]" text format to represent complex and hierarchical data. It is as powerful as [http://en.wikipedia.org/wiki/XML XML], but less verbose. JSON format is widely used for data-exchange in applications, and is essential for the wild success of [http://en.wikipedia.org/wiki/Ajax_(programming) Ajax] and [http://en.wikipedia.org/wiki/Web_2.0 Web2.0]. UBJSON (Universal Binary JSON) is a binary JSON format, specifically optimized for compact file size and better performance while keeping the semantics as simple as the text-based JSON format. Using the UBJSON format allows to wrap complex binary data in a flexible and extensible structure, making it possible to process complex and large dataset without accuracy loss due to text conversions. We envision that both JSON and its binary version will serve as part of the mainstream data-exchange formats for scientific research in the future. It will provide the flexibility and generality achieved by other popular general-purpose file specifications, such as [http://www.hdfgroup.org/HDF5/whatishdf5.html HDF5], with significantly reduced complexity and enhanced performance. JSONLab is a free and open-source implementation of a JSON/UBJSON encoder and a decoder in the native MATLAB language. It can be used to convert a MATLAB data structure (array, struct, cell, struct array and cell array) into JSON/UBJSON formatted strings, or to decode a JSON/UBJSON file into MATLAB data structure. JSONLab supports both MATLAB and [http://www.gnu.org/software/octave/ GNU Octave] (a free MATLAB clone). ------------------------------------------------------------------------------- II. Installation The installation of JSONLab is no different than any other simple MATLAB toolbox. You only need to download/unzip the JSONLab package to a folder, and add the folder's path to MATLAB/Octave's path list by using the following command: addpath('/path/to/jsonlab'); If you want to add this path permanently, you need to type "pathtool", browse to the jsonlab root folder and add to the list, then click "Save". Then, run "rehash" in MATLAB, and type "which loadjson", if you see an output, that means JSONLab is installed for MATLAB/Octave. ------------------------------------------------------------------------------- III.Using JSONLab JSONLab provides two functions, loadjson.m -- a MATLAB->JSON decoder, and savejson.m -- a MATLAB->JSON encoder, for the text-based JSON, and two equivallent functions -- loadubjson and saveubjson for the binary JSON. The detailed help info for the four functions can be found below: === loadjson.m === 
  data=loadjson(fname,opt)
     or
  data=loadjson(fname,'param1',value1,'param2',value2,...)
 
  parse a JSON (JavaScript Object Notation) file or string
 
  authors:Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2011/09/09, including previous works from 
 
          Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713
             created on 2009/11/02
          Franois Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393
             created on  2009/03/22
          Joel Feenstra:
          http://www.mathworks.com/matlabcentral/fileexchange/20565
             created on 2008/07/03
 
  $Id: loadjson.m 452 2014-11-22 16:43:33Z fangq $
 
  input:
       fname: input file name, if fname contains "{}" or "[]", fname
              will be interpreted as a JSON string
       opt: a struct to store parsing options, opt can be replaced by 
            a list of ('param',value) pairs - the param string is equivallent
            to a field in opt. opt can have the following 
            fields (first in [.|.] is the default)
 
            opt.SimplifyCell [0|1]: if set to 1, loadjson will call cell2mat
                          for each element of the JSON data, and group 
                          arrays based on the cell2mat rules.
            opt.FastArrayParser [1|0 or integer]: if set to 1, use a
                          speed-optimized array parser when loading an 
                          array object. The fast array parser may 
                          collapse block arrays into a single large
                          array similar to rules defined in cell2mat; 0 to 
                          use a legacy parser; if set to a larger-than-1
                          value, this option will specify the minimum
                          dimension to enable the fast array parser. For
                          example, if the input is a 3D array, setting
                          FastArrayParser to 1 will return a 3D array;
                          setting to 2 will return a cell array of 2D
                          arrays; setting to 3 will return to a 2D cell
                          array of 1D vectors; setting to 4 will return a
                          3D cell array.
            opt.ShowProgress [0|1]: if set to 1, loadjson displays a progress bar.
 
  output:
       dat: a cell array, where {...} blocks are converted into cell arrays,
            and [...] are converted to arrays
 
  examples:
       dat=loadjson('{"obj":{"string":"value","array":[1,2,3]}}')
       dat=loadjson(['examples' filesep 'example1.json'])
       dat=loadjson(['examples' filesep 'example1.json'],'SimplifyCell',1)
=== savejson.m === 
  json=savejson(rootname,obj,filename)
     or
  json=savejson(rootname,obj,opt)
  json=savejson(rootname,obj,'param1',value1,'param2',value2,...)
 
  convert a MATLAB object (cell, struct or array) into a JSON (JavaScript
  Object Notation) string
 
  author: Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2011/09/09
 
  $Id: savejson.m 458 2014-12-19 22:17:17Z fangq $
 
  input:
       rootname: the name of the root-object, when set to '', the root name
         is ignored, however, when opt.ForceRootName is set to 1 (see below),
         the MATLAB variable name will be used as the root name.
       obj: a MATLAB object (array, cell, cell array, struct, struct array).
       filename: a string for the file name to save the output JSON data.
       opt: a struct for additional options, ignore to use default values.
         opt can have the following fields (first in [.|.] is the default)
 
         opt.FileName [''|string]: a file name to save the output JSON data
         opt.FloatFormat ['%.10g'|string]: format to show each numeric element
                          of a 1D/2D array;
         opt.ArrayIndent [1|0]: if 1, output explicit data array with
                          precedent indentation; if 0, no indentation
         opt.ArrayToStruct[0|1]: when set to 0, savejson outputs 1D/2D
                          array in JSON array format; if sets to 1, an
                          array will be shown as a struct with fields
                          "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for
                          sparse arrays, the non-zero elements will be
                          saved to _ArrayData_ field in triplet-format i.e.
                          (ix,iy,val) and "_ArrayIsSparse_" will be added
                          with a value of 1; for a complex array, the 
                          _ArrayData_ array will include two columns 
                          (4 for sparse) to record the real and imaginary 
                          parts, and also "_ArrayIsComplex_":1 is added. 
         opt.ParseLogical [0|1]: if this is set to 1, logical array elem
                          will use true/false rather than 1/0.
         opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single
                          numerical element will be shown without a square
                          bracket, unless it is the root object; if 0, square
                          brackets are forced for any numerical arrays.
         opt.ForceRootName [0|1]: when set to 1 and rootname is empty, savejson
                          will use the name of the passed obj variable as the 
                          root object name; if obj is an expression and 
                          does not have a name, 'root' will be used; if this 
                          is set to 0 and rootname is empty, the root level 
                          will be merged down to the lower level.
         opt.Inf ['"$1_Inf_"'|string]: a customized regular expression pattern
                          to represent +/-Inf. The matched pattern is '([-+]*)Inf'
                          and $1 represents the sign. For those who want to use
                          1e999 to represent Inf, they can set opt.Inf to '$11e999'
         opt.NaN ['"_NaN_"'|string]: a customized regular expression pattern
                          to represent NaN
         opt.JSONP [''|string]: to generate a JSONP output (JSON with padding),
                          for example, if opt.JSONP='foo', the JSON data is
                          wrapped inside a function call as 'foo(...);'
         opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson 
                          back to the string form
         opt.SaveBinary [0|1]: 1 - save the JSON file in binary mode; 0 - text mode.
         opt.Compact [0|1]: 1- out compact JSON format (remove all newlines and tabs)
 
         opt can be replaced by a list of ('param',value) pairs. The param 
         string is equivallent to a field in opt and is case sensitive.
  output:
       json: a string in the JSON format (see http://json.org)
 
  examples:
       jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... 
                'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],...
                'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;...
                           2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],...
                'MeshCreator','FangQ','MeshTitle','T6 Cube',...
                'SpecialData',[nan, inf, -inf]);
       savejson('jmesh',jsonmesh)
       savejson('',jsonmesh,'ArrayIndent',0,'FloatFormat','\t%.5g')
=== loadubjson.m === 
  data=loadubjson(fname,opt)
     or
  data=loadubjson(fname,'param1',value1,'param2',value2,...)
 
  parse a JSON (JavaScript Object Notation) file or string
 
  authors:Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2013/08/01
 
  $Id: loadubjson.m 436 2014-08-05 20:51:40Z fangq $
 
  input:
       fname: input file name, if fname contains "{}" or "[]", fname
              will be interpreted as a UBJSON string
       opt: a struct to store parsing options, opt can be replaced by 
            a list of ('param',value) pairs - the param string is equivallent
            to a field in opt. opt can have the following 
            fields (first in [.|.] is the default)
 
            opt.SimplifyCell [0|1]: if set to 1, loadubjson will call cell2mat
                          for each element of the JSON data, and group 
                          arrays based on the cell2mat rules.
            opt.IntEndian [B|L]: specify the endianness of the integer fields
                          in the UBJSON input data. B - Big-Endian format for 
                          integers (as required in the UBJSON specification); 
                          L - input integer fields are in Little-Endian order.
 
  output:
       dat: a cell array, where {...} blocks are converted into cell arrays,
            and [...] are converted to arrays
 
  examples:
       obj=struct('string','value','array',[1 2 3]);
       ubjdata=saveubjson('obj',obj);
       dat=loadubjson(ubjdata)
       dat=loadubjson(['examples' filesep 'example1.ubj'])
       dat=loadubjson(['examples' filesep 'example1.ubj'],'SimplifyCell',1)
=== saveubjson.m === 
  json=saveubjson(rootname,obj,filename)
     or
  json=saveubjson(rootname,obj,opt)
  json=saveubjson(rootname,obj,'param1',value1,'param2',value2,...)
 
  convert a MATLAB object (cell, struct or array) into a Universal 
  Binary JSON (UBJSON) binary string
 
  author: Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2013/08/17
 
  $Id: saveubjson.m 440 2014-09-17 19:59:45Z fangq $
 
  input:
       rootname: the name of the root-object, when set to '', the root name
         is ignored, however, when opt.ForceRootName is set to 1 (see below),
         the MATLAB variable name will be used as the root name.
       obj: a MATLAB object (array, cell, cell array, struct, struct array)
       filename: a string for the file name to save the output UBJSON data
       opt: a struct for additional options, ignore to use default values.
         opt can have the following fields (first in [.|.] is the default)
 
         opt.FileName [''|string]: a file name to save the output JSON data
         opt.ArrayToStruct[0|1]: when set to 0, saveubjson outputs 1D/2D
                          array in JSON array format; if sets to 1, an
                          array will be shown as a struct with fields
                          "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for
                          sparse arrays, the non-zero elements will be
                          saved to _ArrayData_ field in triplet-format i.e.
                          (ix,iy,val) and "_ArrayIsSparse_" will be added
                          with a value of 1; for a complex array, the 
                          _ArrayData_ array will include two columns 
                          (4 for sparse) to record the real and imaginary 
                          parts, and also "_ArrayIsComplex_":1 is added. 
         opt.ParseLogical [1|0]: if this is set to 1, logical array elem
                          will use true/false rather than 1/0.
         opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single
                          numerical element will be shown without a square
                          bracket, unless it is the root object; if 0, square
                          brackets are forced for any numerical arrays.
         opt.ForceRootName [0|1]: when set to 1 and rootname is empty, saveubjson
                          will use the name of the passed obj variable as the 
                          root object name; if obj is an expression and 
                          does not have a name, 'root' will be used; if this 
                          is set to 0 and rootname is empty, the root level 
                          will be merged down to the lower level.
         opt.JSONP [''|string]: to generate a JSONP output (JSON with padding),
                          for example, if opt.JSON='foo', the JSON data is
                          wrapped inside a function call as 'foo(...);'
         opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson 
                          back to the string form
 
         opt can be replaced by a list of ('param',value) pairs. The param 
         string is equivallent to a field in opt and is case sensitive.
  output:
       json: a binary string in the UBJSON format (see http://ubjson.org)
 
  examples:
       jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... 
                'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],...
                'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;...
                           2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],...
                'MeshCreator','FangQ','MeshTitle','T6 Cube',...
                'SpecialData',[nan, inf, -inf]);
       saveubjson('jsonmesh',jsonmesh)
       saveubjson('jsonmesh',jsonmesh,'meshdata.ubj')
=== examples === Under the "examples" folder, you can find several scripts to demonstrate the basic utilities of JSONLab. Running the "demo_jsonlab_basic.m" script, you will see the conversions from MATLAB data structure to JSON text and backward. In "jsonlab_selftest.m", we load complex JSON files downloaded from the Internet and validate the loadjson/savejson functions for regression testing purposes. Similarly, a "demo_ubjson_basic.m" script is provided to test the saveubjson and loadubjson pairs for various matlab data structures. Please run these examples and understand how JSONLab works before you use it to process your data. ------------------------------------------------------------------------------- IV. Known Issues and TODOs JSONLab has several known limitations. We are striving to make it more general and robust. Hopefully in a few future releases, the limitations become less. Here are the known issues: # 3D or higher dimensional cell/struct-arrays will be converted to 2D arrays; # When processing names containing multi-byte characters, Octave and MATLAB \ can give different field-names; you can use feature('DefaultCharacterSet','latin1') \ in MATLAB to get consistant results # savejson can not handle class and dataset. # saveubjson converts a logical array into a uint8 ([U]) array # an unofficial N-D array count syntax is implemented in saveubjson. We are \ actively communicating with the UBJSON spec maintainer to investigate the \ possibility of making it upstream # loadubjson can not parse all UBJSON Specification (Draft 9) compliant \ files, however, it can parse all UBJSON files produced by saveubjson. ------------------------------------------------------------------------------- V. Contribution and feedback JSONLab is an open-source project. This means you can not only use it and modify it as you wish, but also you can contribute your changes back to JSONLab so that everyone else can enjoy the improvement. For anyone who want to contribute, please download JSONLab source code from it's subversion repository by using the following command: svn checkout svn://svn.code.sf.net/p/iso2mesh/code/trunk/jsonlab jsonlab You can make changes to the files as needed. Once you are satisfied with your changes, and ready to share it with others, please cd the root directory of JSONLab, and type svn diff > yourname_featurename.patch You then email the .patch file to JSONLab's maintainer, Qianqian Fang, at the email address shown in the beginning of this file. Qianqian will review the changes and commit it to the subversion if they are satisfactory. We appreciate any suggestions and feedbacks from you. Please use iso2mesh's mailing list to report any questions you may have with JSONLab: http://groups.google.com/group/iso2mesh-users?hl=en&pli=1 (Subscription to the mailing list is needed in order to post messages). ================================================ FILE: machine-learning-ex6/ex6/lib/jsonlab/jsonopt.m ================================================ function val=jsonopt(key,default,varargin) % % val=jsonopt(key,default,optstruct) % % setting options based on a struct. The struct can be produced % by varargin2struct from a list of 'param','value' pairs % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % % $Id: loadjson.m 371 2012-06-20 12:43:06Z fangq $ % % input: % key: a string with which one look up a value from a struct % default: if the key does not exist, return default % optstruct: a struct where each sub-field is a key % % output: % val: if key exists, val=optstruct.key; otherwise val=default % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % val=default; if(nargin<=2) return; end opt=varargin{1}; if(isstruct(opt) && isfield(opt,key)) val=getfield(opt,key); end ================================================ FILE: machine-learning-ex6/ex6/lib/jsonlab/loadjson.m ================================================ function data = loadjson(fname,varargin) % % data=loadjson(fname,opt) % or % data=loadjson(fname,'param1',value1,'param2',value2,...) % % parse a JSON (JavaScript Object Notation) file or string % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2011/09/09, including previous works from % % Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713 % created on 2009/11/02 % François Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393 % created on 2009/03/22 % Joel Feenstra: % http://www.mathworks.com/matlabcentral/fileexchange/20565 % created on 2008/07/03 % % $Id: loadjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % fname: input file name, if fname contains "{}" or "[]", fname % will be interpreted as a JSON string % opt: a struct to store parsing options, opt can be replaced by % a list of ('param',value) pairs - the param string is equivallent % to a field in opt. opt can have the following % fields (first in [.|.] is the default) % % opt.SimplifyCell [0|1]: if set to 1, loadjson will call cell2mat % for each element of the JSON data, and group % arrays based on the cell2mat rules. % opt.FastArrayParser [1|0 or integer]: if set to 1, use a % speed-optimized array parser when loading an % array object. The fast array parser may % collapse block arrays into a single large % array similar to rules defined in cell2mat; 0 to % use a legacy parser; if set to a larger-than-1 % value, this option will specify the minimum % dimension to enable the fast array parser. For % example, if the input is a 3D array, setting % FastArrayParser to 1 will return a 3D array; % setting to 2 will return a cell array of 2D % arrays; setting to 3 will return to a 2D cell % array of 1D vectors; setting to 4 will return a % 3D cell array. % opt.ShowProgress [0|1]: if set to 1, loadjson displays a progress bar. % % output: % dat: a cell array, where {...} blocks are converted into cell arrays, % and [...] are converted to arrays % % examples: % dat=loadjson('{"obj":{"string":"value","array":[1,2,3]}}') % dat=loadjson(['examples' filesep 'example1.json']) % dat=loadjson(['examples' filesep 'example1.json'],'SimplifyCell',1) % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % global pos inStr len esc index_esc len_esc isoct arraytoken if(regexp(fname,'[\{\}\]\[]','once')) string=fname; elseif(exist(fname,'file')) fid = fopen(fname,'rb'); string = fread(fid,inf,'uint8=>char')'; fclose(fid); else error('input file does not exist'); end pos = 1; len = length(string); inStr = string; isoct=exist('OCTAVE_VERSION','builtin'); arraytoken=find(inStr=='[' | inStr==']' | inStr=='"'); jstr=regexprep(inStr,'\\\\',' '); escquote=regexp(jstr,'\\"'); arraytoken=sort([arraytoken escquote]); % String delimiters and escape chars identified to improve speed: esc = find(inStr=='"' | inStr=='\' ); % comparable to: regexp(inStr, '["\\]'); index_esc = 1; len_esc = length(esc); opt=varargin2struct(varargin{:}); if(jsonopt('ShowProgress',0,opt)==1) opt.progressbar_=waitbar(0,'loading ...'); end jsoncount=1; while pos <= len switch(next_char) case '{' data{jsoncount} = parse_object(opt); case '[' data{jsoncount} = parse_array(opt); otherwise error_pos('Outer level structure must be an object or an array'); end jsoncount=jsoncount+1; end % while jsoncount=length(data); if(jsoncount==1 && iscell(data)) data=data{1}; end if(~isempty(data)) if(isstruct(data)) % data can be a struct array data=jstruct2array(data); elseif(iscell(data)) data=jcell2array(data); end end if(isfield(opt,'progressbar_')) close(opt.progressbar_); end %% function newdata=jcell2array(data) len=length(data); newdata=data; for i=1:len if(isstruct(data{i})) newdata{i}=jstruct2array(data{i}); elseif(iscell(data{i})) newdata{i}=jcell2array(data{i}); end end %%------------------------------------------------------------------------- function newdata=jstruct2array(data) fn=fieldnames(data); newdata=data; len=length(data); for i=1:length(fn) % depth-first for j=1:len if(isstruct(getfield(data(j),fn{i}))) newdata(j)=setfield(newdata(j),fn{i},jstruct2array(getfield(data(j),fn{i}))); end end end if(~isempty(strmatch('x0x5F_ArrayType_',fn)) && ~isempty(strmatch('x0x5F_ArrayData_',fn))) newdata=cell(len,1); for j=1:len ndata=cast(data(j).x0x5F_ArrayData_,data(j).x0x5F_ArrayType_); iscpx=0; if(~isempty(strmatch('x0x5F_ArrayIsComplex_',fn))) if(data(j).x0x5F_ArrayIsComplex_) iscpx=1; end end if(~isempty(strmatch('x0x5F_ArrayIsSparse_',fn))) if(data(j).x0x5F_ArrayIsSparse_) if(~isempty(strmatch('x0x5F_ArraySize_',fn))) dim=data(j).x0x5F_ArraySize_; if(iscpx && size(ndata,2)==4-any(dim==1)) ndata(:,end-1)=complex(ndata(:,end-1),ndata(:,end)); end if isempty(ndata) % All-zeros sparse ndata=sparse(dim(1),prod(dim(2:end))); elseif dim(1)==1 % Sparse row vector ndata=sparse(1,ndata(:,1),ndata(:,2),dim(1),prod(dim(2:end))); elseif dim(2)==1 % Sparse column vector ndata=sparse(ndata(:,1),1,ndata(:,2),dim(1),prod(dim(2:end))); else % Generic sparse array. ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3),dim(1),prod(dim(2:end))); end else if(iscpx && size(ndata,2)==4) ndata(:,3)=complex(ndata(:,3),ndata(:,4)); end ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3)); end end elseif(~isempty(strmatch('x0x5F_ArraySize_',fn))) if(iscpx && size(ndata,2)==2) ndata=complex(ndata(:,1),ndata(:,2)); end ndata=reshape(ndata(:),data(j).x0x5F_ArraySize_); end newdata{j}=ndata; end if(len==1) newdata=newdata{1}; end end %%------------------------------------------------------------------------- function object = parse_object(varargin) parse_char('{'); object = []; if next_char ~= '}' while 1 str = parseStr(varargin{:}); if isempty(str) error_pos('Name of value at position %d cannot be empty'); end parse_char(':'); val = parse_value(varargin{:}); eval( sprintf( 'object.%s = val;', valid_field(str) ) ); if next_char == '}' break; end parse_char(','); end end parse_char('}'); %%------------------------------------------------------------------------- function object = parse_array(varargin) % JSON array is written in row-major order global pos inStr isoct parse_char('['); object = cell(0, 1); dim2=[]; arraydepth=jsonopt('JSONLAB_ArrayDepth_',1,varargin{:}); pbar=jsonopt('progressbar_',-1,varargin{:}); if next_char ~= ']' if(jsonopt('FastArrayParser',1,varargin{:})>=1 && arraydepth>=jsonopt('FastArrayParser',1,varargin{:})) [endpos, e1l, e1r, maxlevel]=matching_bracket(inStr,pos); arraystr=['[' inStr(pos:endpos)]; arraystr=regexprep(arraystr,'"_NaN_"','NaN'); arraystr=regexprep(arraystr,'"([-+]*)_Inf_"','$1Inf'); arraystr(arraystr==sprintf('\n'))=[]; arraystr(arraystr==sprintf('\r'))=[]; %arraystr=regexprep(arraystr,'\s*,',','); % this is slow,sometimes needed if(~isempty(e1l) && ~isempty(e1r)) % the array is in 2D or higher D astr=inStr((e1l+1):(e1r-1)); astr=regexprep(astr,'"_NaN_"','NaN'); astr=regexprep(astr,'"([-+]*)_Inf_"','$1Inf'); astr(astr==sprintf('\n'))=[]; astr(astr==sprintf('\r'))=[]; astr(astr==' ')=''; if(isempty(find(astr=='[', 1))) % array is 2D dim2=length(sscanf(astr,'%f,',[1 inf])); end else % array is 1D astr=arraystr(2:end-1); astr(astr==' ')=''; [obj, count, errmsg, nextidx]=sscanf(astr,'%f,',[1,inf]); if(nextidx>=length(astr)-1) object=obj; pos=endpos; parse_char(']'); return; end end if(~isempty(dim2)) astr=arraystr; astr(astr=='[')=''; astr(astr==']')=''; astr(astr==' ')=''; [obj, count, errmsg, nextidx]=sscanf(astr,'%f,',inf); if(nextidx>=length(astr)-1) object=reshape(obj,dim2,numel(obj)/dim2)'; pos=endpos; parse_char(']'); if(pbar>0) waitbar(pos/length(inStr),pbar,'loading ...'); end return; end end arraystr=regexprep(arraystr,'\]\s*,','];'); else arraystr='['; end try if(isoct && regexp(arraystr,'"','once')) error('Octave eval can produce empty cells for JSON-like input'); end object=eval(arraystr); pos=endpos; catch while 1 newopt=varargin2struct(varargin{:},'JSONLAB_ArrayDepth_',arraydepth+1); val = parse_value(newopt); object{end+1} = val; if next_char == ']' break; end parse_char(','); end end end if(jsonopt('SimplifyCell',0,varargin{:})==1) try oldobj=object; object=cell2mat(object')'; if(iscell(oldobj) && isstruct(object) && numel(object)>1 && jsonopt('SimplifyCellArray',1,varargin{:})==0) object=oldobj; elseif(size(object,1)>1 && ndims(object)==2) object=object'; end catch end end parse_char(']'); if(pbar>0) waitbar(pos/length(inStr),pbar,'loading ...'); end %%------------------------------------------------------------------------- function parse_char(c) global pos inStr len skip_whitespace; if pos > len || inStr(pos) ~= c error_pos(sprintf('Expected %c at position %%d', c)); else pos = pos + 1; skip_whitespace; end %%------------------------------------------------------------------------- function c = next_char global pos inStr len skip_whitespace; if pos > len c = []; else c = inStr(pos); end %%------------------------------------------------------------------------- function skip_whitespace global pos inStr len while pos <= len && isspace(inStr(pos)) pos = pos + 1; end %%------------------------------------------------------------------------- function str = parseStr(varargin) global pos inStr len esc index_esc len_esc % len, ns = length(inStr), keyboard if inStr(pos) ~= '"' error_pos('String starting with " expected at position %d'); else pos = pos + 1; end str = ''; while pos <= len while index_esc <= len_esc && esc(index_esc) < pos index_esc = index_esc + 1; end if index_esc > len_esc str = [str inStr(pos:len)]; pos = len + 1; break; else str = [str inStr(pos:esc(index_esc)-1)]; pos = esc(index_esc); end nstr = length(str); switch inStr(pos) case '"' pos = pos + 1; if(~isempty(str)) if(strcmp(str,'_Inf_')) str=Inf; elseif(strcmp(str,'-_Inf_')) str=-Inf; elseif(strcmp(str,'_NaN_')) str=NaN; end end return; case '\' if pos+1 > len error_pos('End of file reached right after escape character'); end pos = pos + 1; switch inStr(pos) case {'"' '\' '/'} str(nstr+1) = inStr(pos); pos = pos + 1; case {'b' 'f' 'n' 'r' 't'} str(nstr+1) = sprintf(['\' inStr(pos)]); pos = pos + 1; case 'u' if pos+4 > len error_pos('End of file reached in escaped unicode character'); end str(nstr+(1:6)) = inStr(pos-1:pos+4); pos = pos + 5; end otherwise % should never happen str(nstr+1) = inStr(pos), keyboard pos = pos + 1; end end error_pos('End of file while expecting end of inStr'); %%------------------------------------------------------------------------- function num = parse_number(varargin) global pos inStr len isoct currstr=inStr(pos:end); numstr=0; if(isoct~=0) numstr=regexp(currstr,'^\s*-?(?:0|[1-9]\d*)(?:\.\d+)?(?:[eE][+\-]?\d+)?','end'); [num, one] = sscanf(currstr, '%f', 1); delta=numstr+1; else [num, one, err, delta] = sscanf(currstr, '%f', 1); if ~isempty(err) error_pos('Error reading number at position %d'); end end pos = pos + delta-1; %%------------------------------------------------------------------------- function val = parse_value(varargin) global pos inStr len true = 1; false = 0; pbar=jsonopt('progressbar_',-1,varargin{:}); if(pbar>0) waitbar(pos/len,pbar,'loading ...'); end switch(inStr(pos)) case '"' val = parseStr(varargin{:}); return; case '[' val = parse_array(varargin{:}); return; case '{' val = parse_object(varargin{:}); if isstruct(val) if(~isempty(strmatch('x0x5F_ArrayType_',fieldnames(val), 'exact'))) val=jstruct2array(val); end elseif isempty(val) val = struct; end return; case {'-','0','1','2','3','4','5','6','7','8','9'} val = parse_number(varargin{:}); return; case 't' if pos+3 <= len && strcmpi(inStr(pos:pos+3), 'true') val = true; pos = pos + 4; return; end case 'f' if pos+4 <= len && strcmpi(inStr(pos:pos+4), 'false') val = false; pos = pos + 5; return; end case 'n' if pos+3 <= len && strcmpi(inStr(pos:pos+3), 'null') val = []; pos = pos + 4; return; end end error_pos('Value expected at position %d'); %%------------------------------------------------------------------------- function error_pos(msg) global pos inStr len poShow = max(min([pos-15 pos-1 pos pos+20],len),1); if poShow(3) == poShow(2) poShow(3:4) = poShow(2)+[0 -1]; % display nothing after end msg = [sprintf(msg, pos) ': ' ... inStr(poShow(1):poShow(2)) '' inStr(poShow(3):poShow(4)) ]; error( ['JSONparser:invalidFormat: ' msg] ); %%------------------------------------------------------------------------- function str = valid_field(str) global isoct % From MATLAB doc: field names must begin with a letter, which may be % followed by any combination of letters, digits, and underscores. % Invalid characters will be converted to underscores, and the prefix % "x0x[Hex code]_" will be added if the first character is not a letter. pos=regexp(str,'^[^A-Za-z]','once'); if(~isempty(pos)) if(~isoct) str=regexprep(str,'^([^A-Za-z])','x0x${sprintf(''%X'',unicode2native($1))}_','once'); else str=sprintf('x0x%X_%s',char(str(1)),str(2:end)); end end if(isempty(regexp(str,'[^0-9A-Za-z_]', 'once' ))) return; end if(~isoct) str=regexprep(str,'([^0-9A-Za-z_])','_0x${sprintf(''%X'',unicode2native($1))}_'); else pos=regexp(str,'[^0-9A-Za-z_]'); if(isempty(pos)) return; end str0=str; pos0=[0 pos(:)' length(str)]; str=''; for i=1:length(pos) str=[str str0(pos0(i)+1:pos(i)-1) sprintf('_0x%X_',str0(pos(i)))]; end if(pos(end)~=length(str)) str=[str str0(pos0(end-1)+1:pos0(end))]; end end %str(~isletter(str) & ~('0' <= str & str <= '9')) = '_'; %%------------------------------------------------------------------------- function endpos = matching_quote(str,pos) len=length(str); while(pos1 && str(pos-1)=='\')) endpos=pos; return; end end pos=pos+1; end error('unmatched quotation mark'); %%------------------------------------------------------------------------- function [endpos, e1l, e1r, maxlevel] = matching_bracket(str,pos) global arraytoken level=1; maxlevel=level; endpos=0; bpos=arraytoken(arraytoken>=pos); tokens=str(bpos); len=length(tokens); pos=1; e1l=[]; e1r=[]; while(pos<=len) c=tokens(pos); if(c==']') level=level-1; if(isempty(e1r)) e1r=bpos(pos); end if(level==0) endpos=bpos(pos); return end end if(c=='[') if(isempty(e1l)) e1l=bpos(pos); end level=level+1; maxlevel=max(maxlevel,level); end if(c=='"') pos=matching_quote(tokens,pos+1); end pos=pos+1; end if(endpos==0) error('unmatched "]"'); end ================================================ FILE: machine-learning-ex6/ex6/lib/jsonlab/loadubjson.m ================================================ function data = loadubjson(fname,varargin) % % data=loadubjson(fname,opt) % or % data=loadubjson(fname,'param1',value1,'param2',value2,...) % % parse a JSON (JavaScript Object Notation) file or string % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2013/08/01 % % $Id: loadubjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % fname: input file name, if fname contains "{}" or "[]", fname % will be interpreted as a UBJSON string % opt: a struct to store parsing options, opt can be replaced by % a list of ('param',value) pairs - the param string is equivallent % to a field in opt. opt can have the following % fields (first in [.|.] is the default) % % opt.SimplifyCell [0|1]: if set to 1, loadubjson will call cell2mat % for each element of the JSON data, and group % arrays based on the cell2mat rules. % opt.IntEndian [B|L]: specify the endianness of the integer fields % in the UBJSON input data. B - Big-Endian format for % integers (as required in the UBJSON specification); % L - input integer fields are in Little-Endian order. % % output: % dat: a cell array, where {...} blocks are converted into cell arrays, % and [...] are converted to arrays % % examples: % obj=struct('string','value','array',[1 2 3]); % ubjdata=saveubjson('obj',obj); % dat=loadubjson(ubjdata) % dat=loadubjson(['examples' filesep 'example1.ubj']) % dat=loadubjson(['examples' filesep 'example1.ubj'],'SimplifyCell',1) % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % global pos inStr len esc index_esc len_esc isoct arraytoken fileendian systemendian if(regexp(fname,'[\{\}\]\[]','once')) string=fname; elseif(exist(fname,'file')) fid = fopen(fname,'rb'); string = fread(fid,inf,'uint8=>char')'; fclose(fid); else error('input file does not exist'); end pos = 1; len = length(string); inStr = string; isoct=exist('OCTAVE_VERSION','builtin'); arraytoken=find(inStr=='[' | inStr==']' | inStr=='"'); jstr=regexprep(inStr,'\\\\',' '); escquote=regexp(jstr,'\\"'); arraytoken=sort([arraytoken escquote]); % String delimiters and escape chars identified to improve speed: esc = find(inStr=='"' | inStr=='\' ); % comparable to: regexp(inStr, '["\\]'); index_esc = 1; len_esc = length(esc); opt=varargin2struct(varargin{:}); fileendian=upper(jsonopt('IntEndian','B',opt)); [os,maxelem,systemendian]=computer; jsoncount=1; while pos <= len switch(next_char) case '{' data{jsoncount} = parse_object(opt); case '[' data{jsoncount} = parse_array(opt); otherwise error_pos('Outer level structure must be an object or an array'); end jsoncount=jsoncount+1; end % while jsoncount=length(data); if(jsoncount==1 && iscell(data)) data=data{1}; end if(~isempty(data)) if(isstruct(data)) % data can be a struct array data=jstruct2array(data); elseif(iscell(data)) data=jcell2array(data); end end %% function newdata=parse_collection(id,data,obj) if(jsoncount>0 && exist('data','var')) if(~iscell(data)) newdata=cell(1); newdata{1}=data; data=newdata; end end %% function newdata=jcell2array(data) len=length(data); newdata=data; for i=1:len if(isstruct(data{i})) newdata{i}=jstruct2array(data{i}); elseif(iscell(data{i})) newdata{i}=jcell2array(data{i}); end end %%------------------------------------------------------------------------- function newdata=jstruct2array(data) fn=fieldnames(data); newdata=data; len=length(data); for i=1:length(fn) % depth-first for j=1:len if(isstruct(getfield(data(j),fn{i}))) newdata(j)=setfield(newdata(j),fn{i},jstruct2array(getfield(data(j),fn{i}))); end end end if(~isempty(strmatch('x0x5F_ArrayType_',fn)) && ~isempty(strmatch('x0x5F_ArrayData_',fn))) newdata=cell(len,1); for j=1:len ndata=cast(data(j).x0x5F_ArrayData_,data(j).x0x5F_ArrayType_); iscpx=0; if(~isempty(strmatch('x0x5F_ArrayIsComplex_',fn))) if(data(j).x0x5F_ArrayIsComplex_) iscpx=1; end end if(~isempty(strmatch('x0x5F_ArrayIsSparse_',fn))) if(data(j).x0x5F_ArrayIsSparse_) if(~isempty(strmatch('x0x5F_ArraySize_',fn))) dim=double(data(j).x0x5F_ArraySize_); if(iscpx && size(ndata,2)==4-any(dim==1)) ndata(:,end-1)=complex(ndata(:,end-1),ndata(:,end)); end if isempty(ndata) % All-zeros sparse ndata=sparse(dim(1),prod(dim(2:end))); elseif dim(1)==1 % Sparse row vector ndata=sparse(1,ndata(:,1),ndata(:,2),dim(1),prod(dim(2:end))); elseif dim(2)==1 % Sparse column vector ndata=sparse(ndata(:,1),1,ndata(:,2),dim(1),prod(dim(2:end))); else % Generic sparse array. ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3),dim(1),prod(dim(2:end))); end else if(iscpx && size(ndata,2)==4) ndata(:,3)=complex(ndata(:,3),ndata(:,4)); end ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3)); end end elseif(~isempty(strmatch('x0x5F_ArraySize_',fn))) if(iscpx && size(ndata,2)==2) ndata=complex(ndata(:,1),ndata(:,2)); end ndata=reshape(ndata(:),data(j).x0x5F_ArraySize_); end newdata{j}=ndata; end if(len==1) newdata=newdata{1}; end end %%------------------------------------------------------------------------- function object = parse_object(varargin) parse_char('{'); object = []; type=''; count=-1; if(next_char == '$') type=inStr(pos+1); % TODO pos=pos+2; end if(next_char == '#') pos=pos+1; count=double(parse_number()); end if next_char ~= '}' num=0; while 1 str = parseStr(varargin{:}); if isempty(str) error_pos('Name of value at position %d cannot be empty'); end %parse_char(':'); val = parse_value(varargin{:}); num=num+1; eval( sprintf( 'object.%s = val;', valid_field(str) ) ); if next_char == '}' || (count>=0 && num>=count) break; end %parse_char(','); end end if(count==-1) parse_char('}'); end %%------------------------------------------------------------------------- function [cid,len]=elem_info(type) id=strfind('iUIlLdD',type); dataclass={'int8','uint8','int16','int32','int64','single','double'}; bytelen=[1,1,2,4,8,4,8]; if(id>0) cid=dataclass{id}; len=bytelen(id); else error_pos('unsupported type at position %d'); end %%------------------------------------------------------------------------- function [data adv]=parse_block(type,count,varargin) global pos inStr isoct fileendian systemendian [cid,len]=elem_info(type); datastr=inStr(pos:pos+len*count-1); if(isoct) newdata=int8(datastr); else newdata=uint8(datastr); end id=strfind('iUIlLdD',type); if(id<=5 && fileendian~=systemendian) newdata=swapbytes(typecast(newdata,cid)); end data=typecast(newdata,cid); adv=double(len*count); %%------------------------------------------------------------------------- function object = parse_array(varargin) % JSON array is written in row-major order global pos inStr isoct parse_char('['); object = cell(0, 1); dim=[]; type=''; count=-1; if(next_char == '$') type=inStr(pos+1); pos=pos+2; end if(next_char == '#') pos=pos+1; if(next_char=='[') dim=parse_array(varargin{:}); count=prod(double(dim)); else count=double(parse_number()); end end if(~isempty(type)) if(count>=0) [object adv]=parse_block(type,count,varargin{:}); if(~isempty(dim)) object=reshape(object,dim); end pos=pos+adv; return; else endpos=matching_bracket(inStr,pos); [cid,len]=elem_info(type); count=(endpos-pos)/len; [object adv]=parse_block(type,count,varargin{:}); pos=pos+adv; parse_char(']'); return; end end if next_char ~= ']' while 1 val = parse_value(varargin{:}); object{end+1} = val; if next_char == ']' break; end %parse_char(','); end end if(jsonopt('SimplifyCell',0,varargin{:})==1) try oldobj=object; object=cell2mat(object')'; if(iscell(oldobj) && isstruct(object) && numel(object)>1 && jsonopt('SimplifyCellArray',1,varargin{:})==0) object=oldobj; elseif(size(object,1)>1 && ndims(object)==2) object=object'; end catch end end if(count==-1) parse_char(']'); end %%------------------------------------------------------------------------- function parse_char(c) global pos inStr len skip_whitespace; if pos > len || inStr(pos) ~= c error_pos(sprintf('Expected %c at position %%d', c)); else pos = pos + 1; skip_whitespace; end %%------------------------------------------------------------------------- function c = next_char global pos inStr len skip_whitespace; if pos > len c = []; else c = inStr(pos); end %%------------------------------------------------------------------------- function skip_whitespace global pos inStr len while pos <= len && isspace(inStr(pos)) pos = pos + 1; end %%------------------------------------------------------------------------- function str = parseStr(varargin) global pos inStr esc index_esc len_esc % len, ns = length(inStr), keyboard type=inStr(pos); if type ~= 'S' && type ~= 'C' && type ~= 'H' error_pos('String starting with S expected at position %d'); else pos = pos + 1; end if(type == 'C') str=inStr(pos); pos=pos+1; return; end bytelen=double(parse_number()); if(length(inStr)>=pos+bytelen-1) str=inStr(pos:pos+bytelen-1); pos=pos+bytelen; else error_pos('End of file while expecting end of inStr'); end %%------------------------------------------------------------------------- function num = parse_number(varargin) global pos inStr len isoct fileendian systemendian id=strfind('iUIlLdD',inStr(pos)); if(isempty(id)) error_pos('expecting a number at position %d'); end type={'int8','uint8','int16','int32','int64','single','double'}; bytelen=[1,1,2,4,8,4,8]; datastr=inStr(pos+1:pos+bytelen(id)); if(isoct) newdata=int8(datastr); else newdata=uint8(datastr); end if(id<=5 && fileendian~=systemendian) newdata=swapbytes(typecast(newdata,type{id})); end num=typecast(newdata,type{id}); pos = pos + bytelen(id)+1; %%------------------------------------------------------------------------- function val = parse_value(varargin) global pos inStr len true = 1; false = 0; switch(inStr(pos)) case {'S','C','H'} val = parseStr(varargin{:}); return; case '[' val = parse_array(varargin{:}); return; case '{' val = parse_object(varargin{:}); if isstruct(val) if(~isempty(strmatch('x0x5F_ArrayType_',fieldnames(val), 'exact'))) val=jstruct2array(val); end elseif isempty(val) val = struct; end return; case {'i','U','I','l','L','d','D'} val = parse_number(varargin{:}); return; case 'T' val = true; pos = pos + 1; return; case 'F' val = false; pos = pos + 1; return; case {'Z','N'} val = []; pos = pos + 1; return; end error_pos('Value expected at position %d'); %%------------------------------------------------------------------------- function error_pos(msg) global pos inStr len poShow = max(min([pos-15 pos-1 pos pos+20],len),1); if poShow(3) == poShow(2) poShow(3:4) = poShow(2)+[0 -1]; % display nothing after end msg = [sprintf(msg, pos) ': ' ... inStr(poShow(1):poShow(2)) '' inStr(poShow(3):poShow(4)) ]; error( ['JSONparser:invalidFormat: ' msg] ); %%------------------------------------------------------------------------- function str = valid_field(str) global isoct % From MATLAB doc: field names must begin with a letter, which may be % followed by any combination of letters, digits, and underscores. % Invalid characters will be converted to underscores, and the prefix % "x0x[Hex code]_" will be added if the first character is not a letter. pos=regexp(str,'^[^A-Za-z]','once'); if(~isempty(pos)) if(~isoct) str=regexprep(str,'^([^A-Za-z])','x0x${sprintf(''%X'',unicode2native($1))}_','once'); else str=sprintf('x0x%X_%s',char(str(1)),str(2:end)); end end if(isempty(regexp(str,'[^0-9A-Za-z_]', 'once' ))) return; end if(~isoct) str=regexprep(str,'([^0-9A-Za-z_])','_0x${sprintf(''%X'',unicode2native($1))}_'); else pos=regexp(str,'[^0-9A-Za-z_]'); if(isempty(pos)) return; end str0=str; pos0=[0 pos(:)' length(str)]; str=''; for i=1:length(pos) str=[str str0(pos0(i)+1:pos(i)-1) sprintf('_0x%X_',str0(pos(i)))]; end if(pos(end)~=length(str)) str=[str str0(pos0(end-1)+1:pos0(end))]; end end %str(~isletter(str) & ~('0' <= str & str <= '9')) = '_'; %%------------------------------------------------------------------------- function endpos = matching_quote(str,pos) len=length(str); while(pos1 && str(pos-1)=='\')) endpos=pos; return; end end pos=pos+1; end error('unmatched quotation mark'); %%------------------------------------------------------------------------- function [endpos e1l e1r maxlevel] = matching_bracket(str,pos) global arraytoken level=1; maxlevel=level; endpos=0; bpos=arraytoken(arraytoken>=pos); tokens=str(bpos); len=length(tokens); pos=1; e1l=[]; e1r=[]; while(pos<=len) c=tokens(pos); if(c==']') level=level-1; if(isempty(e1r)) e1r=bpos(pos); end if(level==0) endpos=bpos(pos); return end end if(c=='[') if(isempty(e1l)) e1l=bpos(pos); end level=level+1; maxlevel=max(maxlevel,level); end if(c=='"') pos=matching_quote(tokens,pos+1); end pos=pos+1; end if(endpos==0) error('unmatched "]"'); end ================================================ FILE: machine-learning-ex6/ex6/lib/jsonlab/mergestruct.m ================================================ function s=mergestruct(s1,s2) % % s=mergestruct(s1,s2) % % merge two struct objects into one % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % date: 2012/12/22 % % input: % s1,s2: a struct object, s1 and s2 can not be arrays % % output: % s: the merged struct object. fields in s1 and s2 will be combined in s. % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(~isstruct(s1) || ~isstruct(s2)) error('input parameters contain non-struct'); end if(length(s1)>1 || length(s2)>1) error('can not merge struct arrays'); end fn=fieldnames(s2); s=s1; for i=1:length(fn) s=setfield(s,fn{i},getfield(s2,fn{i})); end ================================================ FILE: machine-learning-ex6/ex6/lib/jsonlab/savejson.m ================================================ function json=savejson(rootname,obj,varargin) % % json=savejson(rootname,obj,filename) % or % json=savejson(rootname,obj,opt) % json=savejson(rootname,obj,'param1',value1,'param2',value2,...) % % convert a MATLAB object (cell, struct or array) into a JSON (JavaScript % Object Notation) string % % author: Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2011/09/09 % % $Id: savejson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % rootname: the name of the root-object, when set to '', the root name % is ignored, however, when opt.ForceRootName is set to 1 (see below), % the MATLAB variable name will be used as the root name. % obj: a MATLAB object (array, cell, cell array, struct, struct array). % filename: a string for the file name to save the output JSON data. % opt: a struct for additional options, ignore to use default values. % opt can have the following fields (first in [.|.] is the default) % % opt.FileName [''|string]: a file name to save the output JSON data % opt.FloatFormat ['%.10g'|string]: format to show each numeric element % of a 1D/2D array; % opt.ArrayIndent [1|0]: if 1, output explicit data array with % precedent indentation; if 0, no indentation % opt.ArrayToStruct[0|1]: when set to 0, savejson outputs 1D/2D % array in JSON array format; if sets to 1, an % array will be shown as a struct with fields % "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for % sparse arrays, the non-zero elements will be % saved to _ArrayData_ field in triplet-format i.e. % (ix,iy,val) and "_ArrayIsSparse_" will be added % with a value of 1; for a complex array, the % _ArrayData_ array will include two columns % (4 for sparse) to record the real and imaginary % parts, and also "_ArrayIsComplex_":1 is added. % opt.ParseLogical [0|1]: if this is set to 1, logical array elem % will use true/false rather than 1/0. % opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single % numerical element will be shown without a square % bracket, unless it is the root object; if 0, square % brackets are forced for any numerical arrays. % opt.ForceRootName [0|1]: when set to 1 and rootname is empty, savejson % will use the name of the passed obj variable as the % root object name; if obj is an expression and % does not have a name, 'root' will be used; if this % is set to 0 and rootname is empty, the root level % will be merged down to the lower level. % opt.Inf ['"$1_Inf_"'|string]: a customized regular expression pattern % to represent +/-Inf. The matched pattern is '([-+]*)Inf' % and $1 represents the sign. For those who want to use % 1e999 to represent Inf, they can set opt.Inf to '$11e999' % opt.NaN ['"_NaN_"'|string]: a customized regular expression pattern % to represent NaN % opt.JSONP [''|string]: to generate a JSONP output (JSON with padding), % for example, if opt.JSONP='foo', the JSON data is % wrapped inside a function call as 'foo(...);' % opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson % back to the string form % opt.SaveBinary [0|1]: 1 - save the JSON file in binary mode; 0 - text mode. % opt.Compact [0|1]: 1- out compact JSON format (remove all newlines and tabs) % % opt can be replaced by a list of ('param',value) pairs. The param % string is equivallent to a field in opt and is case sensitive. % output: % json: a string in the JSON format (see http://json.org) % % examples: % jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... % 'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],... % 'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;... % 2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],... % 'MeshCreator','FangQ','MeshTitle','T6 Cube',... % 'SpecialData',[nan, inf, -inf]); % savejson('jmesh',jsonmesh) % savejson('',jsonmesh,'ArrayIndent',0,'FloatFormat','\t%.5g') % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(nargin==1) varname=inputname(1); obj=rootname; if(isempty(varname)) varname='root'; end rootname=varname; else varname=inputname(2); end if(length(varargin)==1 && ischar(varargin{1})) opt=struct('FileName',varargin{1}); else opt=varargin2struct(varargin{:}); end opt.IsOctave=exist('OCTAVE_VERSION','builtin'); rootisarray=0; rootlevel=1; forceroot=jsonopt('ForceRootName',0,opt); if((isnumeric(obj) || islogical(obj) || ischar(obj) || isstruct(obj) || iscell(obj)) && isempty(rootname) && forceroot==0) rootisarray=1; rootlevel=0; else if(isempty(rootname)) rootname=varname; end end if((isstruct(obj) || iscell(obj))&& isempty(rootname) && forceroot) rootname='root'; end whitespaces=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); if(jsonopt('Compact',0,opt)==1) whitespaces=struct('tab','','newline','','sep',','); end if(~isfield(opt,'whitespaces_')) opt.whitespaces_=whitespaces; end nl=whitespaces.newline; json=obj2json(rootname,obj,rootlevel,opt); if(rootisarray) json=sprintf('%s%s',json,nl); else json=sprintf('{%s%s%s}\n',nl,json,nl); end jsonp=jsonopt('JSONP','',opt); if(~isempty(jsonp)) json=sprintf('%s(%s);%s',jsonp,json,nl); end % save to a file if FileName is set, suggested by Patrick Rapin if(~isempty(jsonopt('FileName','',opt))) if(jsonopt('SaveBinary',0,opt)==1) fid = fopen(opt.FileName, 'wb'); fwrite(fid,json); else fid = fopen(opt.FileName, 'wt'); fwrite(fid,json,'char'); end fclose(fid); end %%------------------------------------------------------------------------- function txt=obj2json(name,item,level,varargin) if(iscell(item)) txt=cell2json(name,item,level,varargin{:}); elseif(isstruct(item)) txt=struct2json(name,item,level,varargin{:}); elseif(ischar(item)) txt=str2json(name,item,level,varargin{:}); else txt=mat2json(name,item,level,varargin{:}); end %%------------------------------------------------------------------------- function txt=cell2json(name,item,level,varargin) txt=''; if(~iscell(item)) error('input is not a cell'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); ws=jsonopt('whitespaces_',struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')),varargin{:}); padding0=repmat(ws.tab,1,level); padding2=repmat(ws.tab,1,level+1); nl=ws.newline; if(len>1) if(~isempty(name)) txt=sprintf('%s"%s": [%s',padding0, checkname(name,varargin{:}),nl); name=''; else txt=sprintf('%s[%s',padding0,nl); end elseif(len==0) if(~isempty(name)) txt=sprintf('%s"%s": []',padding0, checkname(name,varargin{:})); name=''; else txt=sprintf('%s[]',padding0); end end for j=1:dim(2) if(dim(1)>1) txt=sprintf('%s%s[%s',txt,padding2,nl); end for i=1:dim(1) txt=sprintf('%s%s',txt,obj2json(name,item{i,j},level+(dim(1)>1)+1,varargin{:})); if(i1) txt=sprintf('%s%s%s]',txt,nl,padding2); end if(j1) txt=sprintf('%s%s%s]',txt,nl,padding0); end %%------------------------------------------------------------------------- function txt=struct2json(name,item,level,varargin) txt=''; if(~isstruct(item)) error('input is not a struct'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); ws=struct('tab',sprintf('\t'),'newline',sprintf('\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding0=repmat(ws.tab,1,level); padding2=repmat(ws.tab,1,level+1); padding1=repmat(ws.tab,1,level+(dim(1)>1)+(len>1)); nl=ws.newline; if(~isempty(name)) if(len>1) txt=sprintf('%s"%s": [%s',padding0,checkname(name,varargin{:}),nl); end else if(len>1) txt=sprintf('%s[%s',padding0,nl); end end for j=1:dim(2) if(dim(1)>1) txt=sprintf('%s%s[%s',txt,padding2,nl); end for i=1:dim(1) names = fieldnames(item(i,j)); if(~isempty(name) && len==1) txt=sprintf('%s%s"%s": {%s',txt,padding1, checkname(name,varargin{:}),nl); else txt=sprintf('%s%s{%s',txt,padding1,nl); end if(~isempty(names)) for e=1:length(names) txt=sprintf('%s%s',txt,obj2json(names{e},getfield(item(i,j),... names{e}),level+(dim(1)>1)+1+(len>1),varargin{:})); if(e1) txt=sprintf('%s%s%s]',txt,nl,padding2); end if(j1) txt=sprintf('%s%s%s]',txt,nl,padding0); end %%------------------------------------------------------------------------- function txt=str2json(name,item,level,varargin) txt=''; if(~ischar(item)) error('input is not a string'); end item=reshape(item, max(size(item),[1 0])); len=size(item,1); ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding1=repmat(ws.tab,1,level); padding0=repmat(ws.tab,1,level+1); nl=ws.newline; sep=ws.sep; if(~isempty(name)) if(len>1) txt=sprintf('%s"%s": [%s',padding1,checkname(name,varargin{:}),nl); end else if(len>1) txt=sprintf('%s[%s',padding1,nl); end end isoct=jsonopt('IsOctave',0,varargin{:}); for e=1:len if(isoct) val=regexprep(item(e,:),'\\','\\'); val=regexprep(val,'"','\"'); val=regexprep(val,'^"','\"'); else val=regexprep(item(e,:),'\\','\\\\'); val=regexprep(val,'"','\\"'); val=regexprep(val,'^"','\\"'); end val=escapejsonstring(val); if(len==1) obj=['"' checkname(name,varargin{:}) '": ' '"',val,'"']; if(isempty(name)) obj=['"',val,'"']; end txt=sprintf('%s%s%s%s',txt,padding1,obj); else txt=sprintf('%s%s%s%s',txt,padding0,['"',val,'"']); end if(e==len) sep=''; end txt=sprintf('%s%s',txt,sep); end if(len>1) txt=sprintf('%s%s%s%s',txt,nl,padding1,']'); end %%------------------------------------------------------------------------- function txt=mat2json(name,item,level,varargin) if(~isnumeric(item) && ~islogical(item)) error('input is not an array'); end ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding1=repmat(ws.tab,1,level); padding0=repmat(ws.tab,1,level+1); nl=ws.newline; sep=ws.sep; if(length(size(item))>2 || issparse(item) || ~isreal(item) || ... isempty(item) ||jsonopt('ArrayToStruct',0,varargin{:})) if(isempty(name)) txt=sprintf('%s{%s%s"_ArrayType_": "%s",%s%s"_ArraySize_": %s,%s',... padding1,nl,padding0,class(item),nl,padding0,regexprep(mat2str(size(item)),'\s+',','),nl); else txt=sprintf('%s"%s": {%s%s"_ArrayType_": "%s",%s%s"_ArraySize_": %s,%s',... padding1,checkname(name,varargin{:}),nl,padding0,class(item),nl,padding0,regexprep(mat2str(size(item)),'\s+',','),nl); end else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1 && level>0) numtxt=regexprep(regexprep(matdata2json(item,level+1,varargin{:}),'^\[',''),']',''); else numtxt=matdata2json(item,level+1,varargin{:}); end if(isempty(name)) txt=sprintf('%s%s',padding1,numtxt); else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1) txt=sprintf('%s"%s": %s',padding1,checkname(name,varargin{:}),numtxt); else txt=sprintf('%s"%s": %s',padding1,checkname(name,varargin{:}),numtxt); end end return; end dataformat='%s%s%s%s%s'; if(issparse(item)) [ix,iy]=find(item); data=full(item(find(item))); if(~isreal(item)) data=[real(data(:)),imag(data(:))]; if(size(item,1)==1) % Kludge to have data's 'transposedness' match item's. % (Necessary for complex row vector handling below.) data=data'; end txt=sprintf(dataformat,txt,padding0,'"_ArrayIsComplex_": ','1', sep); end txt=sprintf(dataformat,txt,padding0,'"_ArrayIsSparse_": ','1', sep); if(size(item,1)==1) % Row vector, store only column indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([iy(:),data'],level+2,varargin{:}), nl); elseif(size(item,2)==1) % Column vector, store only row indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([ix,data],level+2,varargin{:}), nl); else % General case, store row and column indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([ix,iy,data],level+2,varargin{:}), nl); end else if(isreal(item)) txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json(item(:)',level+2,varargin{:}), nl); else txt=sprintf(dataformat,txt,padding0,'"_ArrayIsComplex_": ','1', sep); txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([real(item(:)) imag(item(:))],level+2,varargin{:}), nl); end end txt=sprintf('%s%s%s',txt,padding1,'}'); %%------------------------------------------------------------------------- function txt=matdata2json(mat,level,varargin) ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); tab=ws.tab; nl=ws.newline; if(size(mat,1)==1) pre=''; post=''; level=level-1; else pre=sprintf('[%s',nl); post=sprintf('%s%s]',nl,repmat(tab,1,level-1)); end if(isempty(mat)) txt='null'; return; end floatformat=jsonopt('FloatFormat','%.10g',varargin{:}); %if(numel(mat)>1) formatstr=['[' repmat([floatformat ','],1,size(mat,2)-1) [floatformat sprintf('],%s',nl)]]; %else % formatstr=[repmat([floatformat ','],1,size(mat,2)-1) [floatformat sprintf(',\n')]]; %end if(nargin>=2 && size(mat,1)>1 && jsonopt('ArrayIndent',1,varargin{:})==1) formatstr=[repmat(tab,1,level) formatstr]; end txt=sprintf(formatstr,mat'); txt(end-length(nl):end)=[]; if(islogical(mat) && jsonopt('ParseLogical',0,varargin{:})==1) txt=regexprep(txt,'1','true'); txt=regexprep(txt,'0','false'); end %txt=regexprep(mat2str(mat),'\s+',','); %txt=regexprep(txt,';',sprintf('],\n[')); % if(nargin>=2 && size(mat,1)>1) % txt=regexprep(txt,'\[',[repmat(sprintf('\t'),1,level) '[']); % end txt=[pre txt post]; if(any(isinf(mat(:)))) txt=regexprep(txt,'([-+]*)Inf',jsonopt('Inf','"$1_Inf_"',varargin{:})); end if(any(isnan(mat(:)))) txt=regexprep(txt,'NaN',jsonopt('NaN','"_NaN_"',varargin{:})); end %%------------------------------------------------------------------------- function newname=checkname(name,varargin) isunpack=jsonopt('UnpackHex',1,varargin{:}); newname=name; if(isempty(regexp(name,'0x([0-9a-fA-F]+)_','once'))) return end if(isunpack) isoct=jsonopt('IsOctave',0,varargin{:}); if(~isoct) newname=regexprep(name,'(^x|_){1}0x([0-9a-fA-F]+)_','${native2unicode(hex2dec($2))}'); else pos=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','start'); pend=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','end'); if(isempty(pos)) return; end str0=name; pos0=[0 pend(:)' length(name)]; newname=''; for i=1:length(pos) newname=[newname str0(pos0(i)+1:pos(i)-1) char(hex2dec(str0(pos(i)+3:pend(i)-1)))]; end if(pos(end)~=length(name)) newname=[newname str0(pos0(end-1)+1:pos0(end))]; end end end %%------------------------------------------------------------------------- function newstr=escapejsonstring(str) newstr=str; isoct=exist('OCTAVE_VERSION','builtin'); if(isoct) vv=sscanf(OCTAVE_VERSION,'%f'); if(vv(1)>=3.8) isoct=0; end end if(isoct) escapechars={'\a','\f','\n','\r','\t','\v'}; for i=1:length(escapechars); newstr=regexprep(newstr,escapechars{i},escapechars{i}); end else escapechars={'\a','\b','\f','\n','\r','\t','\v'}; for i=1:length(escapechars); newstr=regexprep(newstr,escapechars{i},regexprep(escapechars{i},'\\','\\\\')); end end ================================================ FILE: machine-learning-ex6/ex6/lib/jsonlab/saveubjson.m ================================================ function json=saveubjson(rootname,obj,varargin) % % json=saveubjson(rootname,obj,filename) % or % json=saveubjson(rootname,obj,opt) % json=saveubjson(rootname,obj,'param1',value1,'param2',value2,...) % % convert a MATLAB object (cell, struct or array) into a Universal % Binary JSON (UBJSON) binary string % % author: Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2013/08/17 % % $Id: saveubjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % rootname: the name of the root-object, when set to '', the root name % is ignored, however, when opt.ForceRootName is set to 1 (see below), % the MATLAB variable name will be used as the root name. % obj: a MATLAB object (array, cell, cell array, struct, struct array) % filename: a string for the file name to save the output UBJSON data % opt: a struct for additional options, ignore to use default values. % opt can have the following fields (first in [.|.] is the default) % % opt.FileName [''|string]: a file name to save the output JSON data % opt.ArrayToStruct[0|1]: when set to 0, saveubjson outputs 1D/2D % array in JSON array format; if sets to 1, an % array will be shown as a struct with fields % "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for % sparse arrays, the non-zero elements will be % saved to _ArrayData_ field in triplet-format i.e. % (ix,iy,val) and "_ArrayIsSparse_" will be added % with a value of 1; for a complex array, the % _ArrayData_ array will include two columns % (4 for sparse) to record the real and imaginary % parts, and also "_ArrayIsComplex_":1 is added. % opt.ParseLogical [1|0]: if this is set to 1, logical array elem % will use true/false rather than 1/0. % opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single % numerical element will be shown without a square % bracket, unless it is the root object; if 0, square % brackets are forced for any numerical arrays. % opt.ForceRootName [0|1]: when set to 1 and rootname is empty, saveubjson % will use the name of the passed obj variable as the % root object name; if obj is an expression and % does not have a name, 'root' will be used; if this % is set to 0 and rootname is empty, the root level % will be merged down to the lower level. % opt.JSONP [''|string]: to generate a JSONP output (JSON with padding), % for example, if opt.JSON='foo', the JSON data is % wrapped inside a function call as 'foo(...);' % opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson % back to the string form % % opt can be replaced by a list of ('param',value) pairs. The param % string is equivallent to a field in opt and is case sensitive. % output: % json: a binary string in the UBJSON format (see http://ubjson.org) % % examples: % jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... % 'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],... % 'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;... % 2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],... % 'MeshCreator','FangQ','MeshTitle','T6 Cube',... % 'SpecialData',[nan, inf, -inf]); % saveubjson('jsonmesh',jsonmesh) % saveubjson('jsonmesh',jsonmesh,'meshdata.ubj') % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(nargin==1) varname=inputname(1); obj=rootname; if(isempty(varname)) varname='root'; end rootname=varname; else varname=inputname(2); end if(length(varargin)==1 && ischar(varargin{1})) opt=struct('FileName',varargin{1}); else opt=varargin2struct(varargin{:}); end opt.IsOctave=exist('OCTAVE_VERSION','builtin'); rootisarray=0; rootlevel=1; forceroot=jsonopt('ForceRootName',0,opt); if((isnumeric(obj) || islogical(obj) || ischar(obj) || isstruct(obj) || iscell(obj)) && isempty(rootname) && forceroot==0) rootisarray=1; rootlevel=0; else if(isempty(rootname)) rootname=varname; end end if((isstruct(obj) || iscell(obj))&& isempty(rootname) && forceroot) rootname='root'; end json=obj2ubjson(rootname,obj,rootlevel,opt); if(~rootisarray) json=['{' json '}']; end jsonp=jsonopt('JSONP','',opt); if(~isempty(jsonp)) json=[jsonp '(' json ')']; end % save to a file if FileName is set, suggested by Patrick Rapin if(~isempty(jsonopt('FileName','',opt))) fid = fopen(opt.FileName, 'wb'); fwrite(fid,json); fclose(fid); end %%------------------------------------------------------------------------- function txt=obj2ubjson(name,item,level,varargin) if(iscell(item)) txt=cell2ubjson(name,item,level,varargin{:}); elseif(isstruct(item)) txt=struct2ubjson(name,item,level,varargin{:}); elseif(ischar(item)) txt=str2ubjson(name,item,level,varargin{:}); else txt=mat2ubjson(name,item,level,varargin{:}); end %%------------------------------------------------------------------------- function txt=cell2ubjson(name,item,level,varargin) txt=''; if(~iscell(item)) error('input is not a cell'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); % let's handle 1D cell first if(len>1) if(~isempty(name)) txt=[S_(checkname(name,varargin{:})) '[']; name=''; else txt='['; end elseif(len==0) if(~isempty(name)) txt=[S_(checkname(name,varargin{:})) 'Z']; name=''; else txt='Z'; end end for j=1:dim(2) if(dim(1)>1) txt=[txt '[']; end for i=1:dim(1) txt=[txt obj2ubjson(name,item{i,j},level+(len>1),varargin{:})]; end if(dim(1)>1) txt=[txt ']']; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=struct2ubjson(name,item,level,varargin) txt=''; if(~isstruct(item)) error('input is not a struct'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); if(~isempty(name)) if(len>1) txt=[S_(checkname(name,varargin{:})) '[']; end else if(len>1) txt='['; end end for j=1:dim(2) if(dim(1)>1) txt=[txt '[']; end for i=1:dim(1) names = fieldnames(item(i,j)); if(~isempty(name) && len==1) txt=[txt S_(checkname(name,varargin{:})) '{']; else txt=[txt '{']; end if(~isempty(names)) for e=1:length(names) txt=[txt obj2ubjson(names{e},getfield(item(i,j),... names{e}),level+(dim(1)>1)+1+(len>1),varargin{:})]; end end txt=[txt '}']; end if(dim(1)>1) txt=[txt ']']; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=str2ubjson(name,item,level,varargin) txt=''; if(~ischar(item)) error('input is not a string'); end item=reshape(item, max(size(item),[1 0])); len=size(item,1); if(~isempty(name)) if(len>1) txt=[S_(checkname(name,varargin{:})) '[']; end else if(len>1) txt='['; end end isoct=jsonopt('IsOctave',0,varargin{:}); for e=1:len val=item(e,:); if(len==1) obj=['' S_(checkname(name,varargin{:})) '' '',S_(val),'']; if(isempty(name)) obj=['',S_(val),'']; end txt=[txt,'',obj]; else txt=[txt,'',['',S_(val),'']]; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=mat2ubjson(name,item,level,varargin) if(~isnumeric(item) && ~islogical(item)) error('input is not an array'); end if(length(size(item))>2 || issparse(item) || ~isreal(item) || ... isempty(item) || jsonopt('ArrayToStruct',0,varargin{:})) cid=I_(uint32(max(size(item)))); if(isempty(name)) txt=['{' S_('_ArrayType_'),S_(class(item)),S_('_ArraySize_'),I_a(size(item),cid(1)) ]; else if(isempty(item)) txt=[S_(checkname(name,varargin{:})),'Z']; return; else txt=[S_(checkname(name,varargin{:})),'{',S_('_ArrayType_'),S_(class(item)),S_('_ArraySize_'),I_a(size(item),cid(1))]; end end else if(isempty(name)) txt=matdata2ubjson(item,level+1,varargin{:}); else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1) numtxt=regexprep(regexprep(matdata2ubjson(item,level+1,varargin{:}),'^\[',''),']',''); txt=[S_(checkname(name,varargin{:})) numtxt]; else txt=[S_(checkname(name,varargin{:})),matdata2ubjson(item,level+1,varargin{:})]; end end return; end if(issparse(item)) [ix,iy]=find(item); data=full(item(find(item))); if(~isreal(item)) data=[real(data(:)),imag(data(:))]; if(size(item,1)==1) % Kludge to have data's 'transposedness' match item's. % (Necessary for complex row vector handling below.) data=data'; end txt=[txt,S_('_ArrayIsComplex_'),'T']; end txt=[txt,S_('_ArrayIsSparse_'),'T']; if(size(item,1)==1) % Row vector, store only column indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([iy(:),data'],level+2,varargin{:})]; elseif(size(item,2)==1) % Column vector, store only row indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([ix,data],level+2,varargin{:})]; else % General case, store row and column indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([ix,iy,data],level+2,varargin{:})]; end else if(isreal(item)) txt=[txt,S_('_ArrayData_'),... matdata2ubjson(item(:)',level+2,varargin{:})]; else txt=[txt,S_('_ArrayIsComplex_'),'T']; txt=[txt,S_('_ArrayData_'),... matdata2ubjson([real(item(:)) imag(item(:))],level+2,varargin{:})]; end end txt=[txt,'}']; %%------------------------------------------------------------------------- function txt=matdata2ubjson(mat,level,varargin) if(isempty(mat)) txt='Z'; return; end if(size(mat,1)==1) level=level-1; end type=''; hasnegtive=(mat<0); if(isa(mat,'integer') || isinteger(mat) || (isfloat(mat) && all(mod(mat(:),1) == 0))) if(isempty(hasnegtive)) if(max(mat(:))<=2^8) type='U'; end end if(isempty(type)) % todo - need to consider negative ones separately id= histc(abs(max(mat(:))),[0 2^7 2^15 2^31 2^63]); if(isempty(find(id))) error('high-precision data is not yet supported'); end key='iIlL'; type=key(find(id)); end txt=[I_a(mat(:),type,size(mat))]; elseif(islogical(mat)) logicalval='FT'; if(numel(mat)==1) txt=logicalval(mat+1); else txt=['[$U#' I_a(size(mat),'l') typecast(swapbytes(uint8(mat(:)')),'uint8')]; end else if(numel(mat)==1) txt=['[' D_(mat) ']']; else txt=D_a(mat(:),'D',size(mat)); end end %txt=regexprep(mat2str(mat),'\s+',','); %txt=regexprep(txt,';',sprintf('],[')); % if(nargin>=2 && size(mat,1)>1) % txt=regexprep(txt,'\[',[repmat(sprintf('\t'),1,level) '[']); % end if(any(isinf(mat(:)))) txt=regexprep(txt,'([-+]*)Inf',jsonopt('Inf','"$1_Inf_"',varargin{:})); end if(any(isnan(mat(:)))) txt=regexprep(txt,'NaN',jsonopt('NaN','"_NaN_"',varargin{:})); end %%------------------------------------------------------------------------- function newname=checkname(name,varargin) isunpack=jsonopt('UnpackHex',1,varargin{:}); newname=name; if(isempty(regexp(name,'0x([0-9a-fA-F]+)_','once'))) return end if(isunpack) isoct=jsonopt('IsOctave',0,varargin{:}); if(~isoct) newname=regexprep(name,'(^x|_){1}0x([0-9a-fA-F]+)_','${native2unicode(hex2dec($2))}'); else pos=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','start'); pend=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','end'); if(isempty(pos)) return; end str0=name; pos0=[0 pend(:)' length(name)]; newname=''; for i=1:length(pos) newname=[newname str0(pos0(i)+1:pos(i)-1) char(hex2dec(str0(pos(i)+3:pend(i)-1)))]; end if(pos(end)~=length(name)) newname=[newname str0(pos0(end-1)+1:pos0(end))]; end end end %%------------------------------------------------------------------------- function val=S_(str) if(length(str)==1) val=['C' str]; else val=['S' I_(int32(length(str))) str]; end %%------------------------------------------------------------------------- function val=I_(num) if(~isinteger(num)) error('input is not an integer'); end if(num>=0 && num<255) val=['U' data2byte(swapbytes(cast(num,'uint8')),'uint8')]; return; end key='iIlL'; cid={'int8','int16','int32','int64'}; for i=1:4 if((num>0 && num<2^(i*8-1)) || (num<0 && num>=-2^(i*8-1))) val=[key(i) data2byte(swapbytes(cast(num,cid{i})),'uint8')]; return; end end error('unsupported integer'); %%------------------------------------------------------------------------- function val=D_(num) if(~isfloat(num)) error('input is not a float'); end if(isa(num,'single')) val=['d' data2byte(num,'uint8')]; else val=['D' data2byte(num,'uint8')]; end %%------------------------------------------------------------------------- function data=I_a(num,type,dim,format) id=find(ismember('iUIlL',type)); if(id==0) error('unsupported integer array'); end % based on UBJSON specs, all integer types are stored in big endian format if(id==1) data=data2byte(swapbytes(int8(num)),'uint8'); blen=1; elseif(id==2) data=data2byte(swapbytes(uint8(num)),'uint8'); blen=1; elseif(id==3) data=data2byte(swapbytes(int16(num)),'uint8'); blen=2; elseif(id==4) data=data2byte(swapbytes(int32(num)),'uint8'); blen=4; elseif(id==5) data=data2byte(swapbytes(int64(num)),'uint8'); blen=8; end if(nargin>=3 && length(dim)>=2 && prod(dim)~=dim(2)) format='opt'; end if((nargin<4 || strcmp(format,'opt')) && numel(num)>1) if(nargin>=3 && (length(dim)==1 || (length(dim)>=2 && prod(dim)~=dim(2)))) cid=I_(uint32(max(dim))); data=['$' type '#' I_a(dim,cid(1)) data(:)']; else data=['$' type '#' I_(int32(numel(data)/blen)) data(:)']; end data=['[' data(:)']; else data=reshape(data,blen,numel(data)/blen); data(2:blen+1,:)=data; data(1,:)=type; data=data(:)'; data=['[' data(:)' ']']; end %%------------------------------------------------------------------------- function data=D_a(num,type,dim,format) id=find(ismember('dD',type)); if(id==0) error('unsupported float array'); end if(id==1) data=data2byte(single(num),'uint8'); elseif(id==2) data=data2byte(double(num),'uint8'); end if(nargin>=3 && length(dim)>=2 && prod(dim)~=dim(2)) format='opt'; end if((nargin<4 || strcmp(format,'opt')) && numel(num)>1) if(nargin>=3 && (length(dim)==1 || (length(dim)>=2 && prod(dim)~=dim(2)))) cid=I_(uint32(max(dim))); data=['$' type '#' I_a(dim,cid(1)) data(:)']; else data=['$' type '#' I_(int32(numel(data)/(id*4))) data(:)']; end data=['[' data]; else data=reshape(data,(id*4),length(data)/(id*4)); data(2:(id*4+1),:)=data; data(1,:)=type; data=data(:)'; data=['[' data(:)' ']']; end %%------------------------------------------------------------------------- function bytes=data2byte(varargin) bytes=typecast(varargin{:}); bytes=bytes(:)'; ================================================ FILE: machine-learning-ex6/ex6/lib/jsonlab/varargin2struct.m ================================================ function opt=varargin2struct(varargin) % % opt=varargin2struct('param1',value1,'param2',value2,...) % or % opt=varargin2struct(...,optstruct,...) % % convert a series of input parameters into a structure % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % date: 2012/12/22 % % input: % 'param', value: the input parameters should be pairs of a string and a value % optstruct: if a parameter is a struct, the fields will be merged to the output struct % % output: % opt: a struct where opt.param1=value1, opt.param2=value2 ... % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % len=length(varargin); opt=struct; if(len==0) return; end i=1; while(i<=len) if(isstruct(varargin{i})) opt=mergestruct(opt,varargin{i}); elseif(ischar(varargin{i}) && i 2 % Output displays per step are commented out. %disp(sprintf('Word to stem: %s', b)); x = step1ab(b, k, k0); %disp(sprintf('Steps 1A and B yield: %s', x{1})); x = step1c(x{1}, x{2}, k0); %disp(sprintf('Step 1C yields: %s', x{1})); x = step2(x{1}, x{2}, k0); %disp(sprintf('Step 2 yields: %s', x{1})); x = step3(x{1}, x{2}, k0); %disp(sprintf('Step 3 yields: %s', x{1})); x = step4(x{1}, x{2}, k0); %disp(sprintf('Step 4 yields: %s', x{1})); x = step5(x{1}, x{2}, k0); %disp(sprintf('Step 5 yields: %s', x{1})); stem = x{1}; end % cons(j) is TRUE <=> b[j] is a consonant. function c = cons(i, b, k0) c = true; switch(b(i)) case {'a', 'e', 'i', 'o', 'u'} c = false; case 'y' if i == k0 c = true; else c = ~cons(i - 1, b, k0); end end % mseq() measures the number of consonant sequences between k0 and j. If % c is a consonant sequence and v a vowel sequence, and <..> indicates % arbitrary presence, % gives 0 % vc gives 1 % vcvc gives 2 % vcvcvc gives 3 % .... function n = measure(b, k0) global j; n = 0; i = k0; while true if i > j return end if ~cons(i, b, k0) break; end i = i + 1; end i = i + 1; while true while true if i > j return end if cons(i, b, k0) break; end i = i + 1; end i = i + 1; n = n + 1; while true if i > j return end if ~cons(i, b, k0) break; end i = i + 1; end i = i + 1; end % vowelinstem() is TRUE <=> k0,...j contains a vowel function vis = vowelinstem(b, k0) global j; for i = k0:j, if ~cons(i, b, k0) vis = true; return end end vis = false; %doublec(i) is TRUE <=> i,(i-1) contain a double consonant. function dc = doublec(i, b, k0) if i < k0+1 dc = false; return end if b(i) ~= b(i-1) dc = false; return end dc = cons(i, b, k0); % cvc(j) is TRUE <=> j-2,j-1,j has the form consonant - vowel - consonant % and also if the second c is not w,x or y. this is used when trying to % restore an e at the end of a short word. e.g. % % cav(e), lov(e), hop(e), crim(e), but % snow, box, tray. function c1 = cvc(i, b, k0) if ((i < (k0+2)) || ~cons(i, b, k0) || cons(i-1, b, k0) || ~cons(i-2, b, k0)) c1 = false; else if (b(i) == 'w' || b(i) == 'x' || b(i) == 'y') c1 = false; return end c1 = true; end % ends(s) is TRUE <=> k0,...k ends with the string s. function s = ends(str, b, k) global j; if (str(length(str)) ~= b(k)) s = false; return end % tiny speed-up if (length(str) > k) s = false; return end if strcmp(b(k-length(str)+1:k), str) s = true; j = k - length(str); return else s = false; end % setto(s) sets (j+1),...k to the characters in the string s, readjusting % k accordingly. function so = setto(s, b, k) global j; for i = j+1:(j+length(s)) b(i) = s(i-j); end if k > j+length(s) b((j+length(s)+1):k) = ''; end k = length(b); so = {b, k}; % rs(s) is used further down. % [Note: possible null/value for r if rs is called] function r = rs(str, b, k, k0) r = {b, k}; if measure(b, k0) > 0 r = setto(str, b, k); end % step1ab() gets rid of plurals and -ed or -ing. e.g. % caresses -> caress % ponies -> poni % ties -> ti % caress -> caress % cats -> cat % feed -> feed % agreed -> agree % disabled -> disable % matting -> mat % mating -> mate % meeting -> meet % milling -> mill % messing -> mess % meetings -> meet function s1ab = step1ab(b, k, k0) global j; if b(k) == 's' if ends('sses', b, k) k = k-2; elseif ends('ies', b, k) retVal = setto('i', b, k); b = retVal{1}; k = retVal{2}; elseif (b(k-1) ~= 's') k = k-1; end end if ends('eed', b, k) if measure(b, k0) > 0; k = k-1; end elseif (ends('ed', b, k) || ends('ing', b, k)) && vowelinstem(b, k0) k = j; retVal = {b, k}; if ends('at', b, k) retVal = setto('ate', b(k0:k), k); elseif ends('bl', b, k) retVal = setto('ble', b(k0:k), k); elseif ends('iz', b, k) retVal = setto('ize', b(k0:k), k); elseif doublec(k, b, k0) retVal = {b, k-1}; if b(retVal{2}) == 'l' || b(retVal{2}) == 's' || ... b(retVal{2}) == 'z' retVal = {retVal{1}, retVal{2}+1}; end elseif measure(b, k0) == 1 && cvc(k, b, k0) retVal = setto('e', b(k0:k), k); end k = retVal{2}; b = retVal{1}(k0:k); end j = k; s1ab = {b(k0:k), k}; % step1c() turns terminal y to i when there is another vowel in the stem. function s1c = step1c(b, k, k0) global j; if ends('y', b, k) && vowelinstem(b, k0) b(k) = 'i'; end j = k; s1c = {b, k}; % step2() maps double suffices to single ones. so -ization ( = -ize plus % -ation) maps to -ize etc. note that the string before the suffix must give % m() > 0. function s2 = step2(b, k, k0) global j; s2 = {b, k}; switch b(k-1) case {'a'} if ends('ational', b, k) s2 = rs('ate', b, k, k0); elseif ends('tional', b, k) s2 = rs('tion', b, k, k0); end; case {'c'} if ends('enci', b, k) s2 = rs('ence', b, k, k0); elseif ends('anci', b, k) s2 = rs('ance', b, k, k0); end; case {'e'} if ends('izer', b, k) s2 = rs('ize', b, k, k0); end; case {'l'} if ends('bli', b, k) s2 = rs('ble', b, k, k0); elseif ends('alli', b, k) s2 = rs('al', b, k, k0); elseif ends('entli', b, k) s2 = rs('ent', b, k, k0); elseif ends('eli', b, k) s2 = rs('e', b, k, k0); elseif ends('ousli', b, k) s2 = rs('ous', b, k, k0); end; case {'o'} if ends('ization', b, k) s2 = rs('ize', b, k, k0); elseif ends('ation', b, k) s2 = rs('ate', b, k, k0); elseif ends('ator', b, k) s2 = rs('ate', b, k, k0); end; case {'s'} if ends('alism', b, k) s2 = rs('al', b, k, k0); elseif ends('iveness', b, k) s2 = rs('ive', b, k, k0); elseif ends('fulness', b, k) s2 = rs('ful', b, k, k0); elseif ends('ousness', b, k) s2 = rs('ous', b, k, k0); end; case {'t'} if ends('aliti', b, k) s2 = rs('al', b, k, k0); elseif ends('iviti', b, k) s2 = rs('ive', b, k, k0); elseif ends('biliti', b, k) s2 = rs('ble', b, k, k0); end; case {'g'} if ends('logi', b, k) s2 = rs('log', b, k, k0); end; end j = s2{2}; % step3() deals with -ic-, -full, -ness etc. similar strategy to step2. function s3 = step3(b, k, k0) global j; s3 = {b, k}; switch b(k) case {'e'} if ends('icate', b, k) s3 = rs('ic', b, k, k0); elseif ends('ative', b, k) s3 = rs('', b, k, k0); elseif ends('alize', b, k) s3 = rs('al', b, k, k0); end; case {'i'} if ends('iciti', b, k) s3 = rs('ic', b, k, k0); end; case {'l'} if ends('ical', b, k) s3 = rs('ic', b, k, k0); elseif ends('ful', b, k) s3 = rs('', b, k, k0); end; case {'s'} if ends('ness', b, k) s3 = rs('', b, k, k0); end; end j = s3{2}; % step4() takes off -ant, -ence etc., in context vcvc. function s4 = step4(b, k, k0) global j; switch b(k-1) case {'a'} if ends('al', b, k) end; case {'c'} if ends('ance', b, k) elseif ends('ence', b, k) end; case {'e'} if ends('er', b, k) end; case {'i'} if ends('ic', b, k) end; case {'l'} if ends('able', b, k) elseif ends('ible', b, k) end; case {'n'} if ends('ant', b, k) elseif ends('ement', b, k) elseif ends('ment', b, k) elseif ends('ent', b, k) end; case {'o'} if ends('ion', b, k) if j == 0 elseif ~(strcmp(b(j),'s') || strcmp(b(j),'t')) j = k; end elseif ends('ou', b, k) end; case {'s'} if ends('ism', b, k) end; case {'t'} if ends('ate', b, k) elseif ends('iti', b, k) end; case {'u'} if ends('ous', b, k) end; case {'v'} if ends('ive', b, k) end; case {'z'} if ends('ize', b, k) end; end if measure(b, k0) > 1 s4 = {b(k0:j), j}; else s4 = {b(k0:k), k}; end % step5() removes a final -e if m() > 1, and changes -ll to -l if m() > 1. function s5 = step5(b, k, k0) global j; j = k; if b(k) == 'e' a = measure(b, k0); if (a > 1) || ((a == 1) && ~cvc(k-1, b, k0)) k = k-1; end end if (b(k) == 'l') && doublec(k, b, k0) && (measure(b, k0) > 1) k = k-1; end s5 = {b(k0:k), k}; ================================================ FILE: machine-learning-ex6/ex6/processEmail.m ================================================ function word_indices = processEmail(email_contents) %PROCESSEMAIL preprocesses a the body of an email and %returns a list of word_indices % word_indices = PROCESSEMAIL(email_contents) preprocesses % the body of an email and returns a list of indices of the % words contained in the email. % % Load Vocabulary vocabList = getVocabList(); % Init return value word_indices = []; % ========================== Preprocess Email =========================== % Find the Headers ( \n\n and remove ) % Uncomment the following lines if you are working with raw emails with the % full headers % hdrstart = strfind(email_contents, ([char(10) char(10)])); % email_contents = email_contents(hdrstart(1):end); % Lower case email_contents = lower(email_contents); % Strip all HTML % Looks for any expression that starts with < and ends with > and replace % and does not have any < or > in the tag it with a space email_contents = regexprep(email_contents, '<[^<>]+>', ' '); % Handle Numbers % Look for one or more characters between 0-9 email_contents = regexprep(email_contents, '[0-9]+', 'number'); % Handle URLS % Look for strings starting with http:// or https:// email_contents = regexprep(email_contents, ... '(http|https)://[^\s]*', 'httpaddr'); % Handle Email Addresses % Look for strings with @ in the middle email_contents = regexprep(email_contents, '[^\s]+@[^\s]+', 'emailaddr'); % Handle $ sign email_contents = regexprep(email_contents, '[$]+', 'dollar'); % ========================== Tokenize Email =========================== % Output the email to screen as well fprintf('\n==== Processed Email ====\n\n'); % Process file l = 0; while ~isempty(email_contents) % Tokenize and also get rid of any punctuation [str, email_contents] = ... strtok(email_contents, ... [' @$/#.-:&*+=[]?!(){},''">_<;%' char(10) char(13)]); % Remove any non alphanumeric characters str = regexprep(str, '[^a-zA-Z0-9]', ''); % Stem the word % (the porterStemmer sometimes has issues, so we use a try catch block) try str = porterStemmer(strtrim(str)); catch str = ''; continue; end; % Skip the word if it is too short if length(str) < 1 continue; end % Look up the word in the dictionary and add to word_indices if % found % ====================== YOUR CODE HERE ====================== % Instructions: Fill in this function to add the index of str to % word_indices if it is in the vocabulary. At this point % of the code, you have a stemmed word from the email in % the variable str. You should look up str in the % vocabulary list (vocabList). If a match exists, you % should add the index of the word to the word_indices % vector. Concretely, if str = 'action', then you should % look up the vocabulary list to find where in vocabList % 'action' appears. For example, if vocabList{18} = % 'action', then, you should add 18 to the word_indices % vector (e.g., word_indices = [word_indices ; 18]; ). % % Note: vocabList{idx} returns a the word with index idx in the % vocabulary list. % % Note: You can use strcmp(str1, str2) to compare two strings (str1 and % str2). It will return 1 only if the two strings are equivalent. % for i = 1:size(vocabList) if strcmp(str,vocabList{i}) == 1 word_indices = [word_indices;i]; end end % ============================================================= % Print to screen, ensuring that the output lines are not too long if (l + length(str) + 1) > 78 fprintf('\n'); l = 0; end fprintf('%s ', str); l = l + length(str) + 1; end % Print footer fprintf('\n\n=========================\n'); end ================================================ FILE: machine-learning-ex6/ex6/readFile.m ================================================ function file_contents = readFile(filename) %READFILE reads a file and returns its entire contents % file_contents = READFILE(filename) reads a file and returns its entire % contents in file_contents % % Load File fid = fopen(filename); if fid file_contents = fscanf(fid, '%c', inf); fclose(fid); else file_contents = ''; fprintf('Unable to open %s\n', filename); end end ================================================ FILE: machine-learning-ex6/ex6/spamSample1.txt ================================================ Do You Want To Make $1000 Or More Per Week? If you are a motivated and qualified individual - I will personally demonstrate to you a system that will make you $1,000 per week or more! This is NOT mlm. Call our 24 hour pre-recorded number to get the details. 000-456-789 I need people who want to make serious money. Make the call and get the facts. Invest 2 minutes in yourself now! 000-456-789 Looking forward to your call and I will introduce you to people like yourself who are currently making $10,000 plus per week! 000-456-789 3484lJGv6-241lEaN9080lRmS6-271WxHo7524qiyT5-438rjUv5615hQcf0-662eiDB9057dMtVl72 ================================================ FILE: machine-learning-ex6/ex6/spamSample2.txt ================================================ Best Buy Viagra Generic Online Viagra 100mg x 60 Pills $125, Free Pills & Reorder Discount, Top Selling 100% Quality & Satisfaction guaranteed! We accept VISA, Master & E-Check Payments, 90000+ Satisfied Customers! http://medphysitcstech.ru ================================================ FILE: machine-learning-ex6/ex6/submit.m ================================================ function submit() addpath('./lib'); conf.assignmentSlug = 'support-vector-machines'; conf.itemName = 'Support Vector Machines'; conf.partArrays = { ... { ... '1', ... { 'gaussianKernel.m' }, ... 'Gaussian Kernel', ... }, ... { ... '2', ... { 'dataset3Params.m' }, ... 'Parameters (C, sigma) for Dataset 3', ... }, ... { ... '3', ... { 'processEmail.m' }, ... 'Email Preprocessing', ... }, ... { ... '4', ... { 'emailFeatures.m' }, ... 'Email Feature Extraction', ... }, ... }; conf.output = @output; submitWithConfiguration(conf); end function out = output(partId, auxstring) % Random Test Cases x1 = sin(1:10)'; x2 = cos(1:10)'; ec = 'the quick brown fox jumped over the lazy dog'; wi = 1 + abs(round(x1 * 1863)); wi = [wi ; wi]; if partId == '1' sim = gaussianKernel(x1, x2, 2); out = sprintf('%0.5f ', sim); elseif partId == '2' load('ex6data3.mat'); [C, sigma] = dataset3Params(X, y, Xval, yval); out = sprintf('%0.5f ', C); out = [out sprintf('%0.5f ', sigma)]; elseif partId == '3' word_indices = processEmail(ec); out = sprintf('%d ', word_indices); elseif partId == '4' x = emailFeatures(wi); out = sprintf('%d ', x); end end ================================================ FILE: machine-learning-ex6/ex6/svmPredict.m ================================================ function pred = svmPredict(model, X) %SVMPREDICT returns a vector of predictions using a trained SVM model %(svmTrain). % pred = SVMPREDICT(model, X) returns a vector of predictions using a % trained SVM model (svmTrain). X is a mxn matrix where there each % example is a row. model is a svm model returned from svmTrain. % predictions pred is a m x 1 column of predictions of {0, 1} values. % % Check if we are getting a column vector, if so, then assume that we only % need to do prediction for a single example if (size(X, 2) == 1) % Examples should be in rows X = X'; end % Dataset m = size(X, 1); p = zeros(m, 1); pred = zeros(m, 1); if strcmp(func2str(model.kernelFunction), 'linearKernel') % We can use the weights and bias directly if working with the % linear kernel p = X * model.w + model.b; elseif strfind(func2str(model.kernelFunction), 'gaussianKernel') % Vectorized RBF Kernel % This is equivalent to computing the kernel on every pair of examples X1 = sum(X.^2, 2); X2 = sum(model.X.^2, 2)'; K = bsxfun(@plus, X1, bsxfun(@plus, X2, - 2 * X * model.X')); K = model.kernelFunction(1, 0) .^ K; K = bsxfun(@times, model.y', K); K = bsxfun(@times, model.alphas', K); p = sum(K, 2); else % Other Non-linear kernel for i = 1:m prediction = 0; for j = 1:size(model.X, 1) prediction = prediction + ... model.alphas(j) * model.y(j) * ... model.kernelFunction(X(i,:)', model.X(j,:)'); end p(i) = prediction + model.b; end end % Convert predictions into 0 / 1 pred(p >= 0) = 1; pred(p < 0) = 0; end ================================================ FILE: machine-learning-ex6/ex6/svmTrain.m ================================================ function [model] = svmTrain(X, Y, C, kernelFunction, ... tol, max_passes) %SVMTRAIN Trains an SVM classifier using a simplified version of the SMO %algorithm. % [model] = SVMTRAIN(X, Y, C, kernelFunction, tol, max_passes) trains an % SVM classifier and returns trained model. X is the matrix of training % examples. Each row is a training example, and the jth column holds the % jth feature. Y is a column matrix containing 1 for positive examples % and 0 for negative examples. C is the standard SVM regularization % parameter. tol is a tolerance value used for determining equality of % floating point numbers. max_passes controls the number of iterations % over the dataset (without changes to alpha) before the algorithm quits. % % Note: This is a simplified version of the SMO algorithm for training % SVMs. In practice, if you want to train an SVM classifier, we % recommend using an optimized package such as: % % LIBSVM (http://www.csie.ntu.edu.tw/~cjlin/libsvm/) % SVMLight (http://svmlight.joachims.org/) % % if ~exist('tol', 'var') || isempty(tol) tol = 1e-3; end if ~exist('max_passes', 'var') || isempty(max_passes) max_passes = 5; end % Data parameters m = size(X, 1); n = size(X, 2); % Map 0 to -1 Y(Y==0) = -1; % Variables alphas = zeros(m, 1); b = 0; E = zeros(m, 1); passes = 0; eta = 0; L = 0; H = 0; % Pre-compute the Kernel Matrix since our dataset is small % (in practice, optimized SVM packages that handle large datasets % gracefully will _not_ do this) % % We have implemented optimized vectorized version of the Kernels here so % that the svm training will run faster. if strcmp(func2str(kernelFunction), 'linearKernel') % Vectorized computation for the Linear Kernel % This is equivalent to computing the kernel on every pair of examples K = X*X'; elseif strfind(func2str(kernelFunction), 'gaussianKernel') % Vectorized RBF Kernel % This is equivalent to computing the kernel on every pair of examples X2 = sum(X.^2, 2); K = bsxfun(@plus, X2, bsxfun(@plus, X2', - 2 * (X * X'))); K = kernelFunction(1, 0) .^ K; else % Pre-compute the Kernel Matrix % The following can be slow due to the lack of vectorization K = zeros(m); for i = 1:m for j = i:m K(i,j) = kernelFunction(X(i,:)', X(j,:)'); K(j,i) = K(i,j); %the matrix is symmetric end end end % Train fprintf('\nTraining ...'); dots = 12; while passes < max_passes, num_changed_alphas = 0; for i = 1:m, % Calculate Ei = f(x(i)) - y(i) using (2). % E(i) = b + sum (X(i, :) * (repmat(alphas.*Y,1,n).*X)') - Y(i); E(i) = b + sum (alphas.*Y.*K(:,i)) - Y(i); if ((Y(i)*E(i) < -tol && alphas(i) < C) || (Y(i)*E(i) > tol && alphas(i) > 0)), % In practice, there are many heuristics one can use to select % the i and j. In this simplified code, we select them randomly. j = ceil(m * rand()); while j == i, % Make sure i \neq j j = ceil(m * rand()); end % Calculate Ej = f(x(j)) - y(j) using (2). E(j) = b + sum (alphas.*Y.*K(:,j)) - Y(j); % Save old alphas alpha_i_old = alphas(i); alpha_j_old = alphas(j); % Compute L and H by (10) or (11). if (Y(i) == Y(j)), L = max(0, alphas(j) + alphas(i) - C); H = min(C, alphas(j) + alphas(i)); else L = max(0, alphas(j) - alphas(i)); H = min(C, C + alphas(j) - alphas(i)); end if (L == H), % continue to next i. continue; end % Compute eta by (14). eta = 2 * K(i,j) - K(i,i) - K(j,j); if (eta >= 0), % continue to next i. continue; end % Compute and clip new value for alpha j using (12) and (15). alphas(j) = alphas(j) - (Y(j) * (E(i) - E(j))) / eta; % Clip alphas(j) = min (H, alphas(j)); alphas(j) = max (L, alphas(j)); % Check if change in alpha is significant if (abs(alphas(j) - alpha_j_old) < tol), % continue to next i. % replace anyway alphas(j) = alpha_j_old; continue; end % Determine value for alpha i using (16). alphas(i) = alphas(i) + Y(i)*Y(j)*(alpha_j_old - alphas(j)); % Compute b1 and b2 using (17) and (18) respectively. b1 = b - E(i) ... - Y(i) * (alphas(i) - alpha_i_old) * K(i,j)' ... - Y(j) * (alphas(j) - alpha_j_old) * K(i,j)'; b2 = b - E(j) ... - Y(i) * (alphas(i) - alpha_i_old) * K(i,j)' ... - Y(j) * (alphas(j) - alpha_j_old) * K(j,j)'; % Compute b by (19). if (0 < alphas(i) && alphas(i) < C), b = b1; elseif (0 < alphas(j) && alphas(j) < C), b = b2; else b = (b1+b2)/2; end num_changed_alphas = num_changed_alphas + 1; end end if (num_changed_alphas == 0), passes = passes + 1; else passes = 0; end fprintf('.'); dots = dots + 1; if dots > 78 dots = 0; fprintf('\n'); end if exist('OCTAVE_VERSION') fflush(stdout); end end fprintf(' Done! \n\n'); % Save the model idx = alphas > 0; model.X= X(idx,:); model.y= Y(idx); model.kernelFunction = kernelFunction; model.b= b; model.alphas= alphas(idx); model.w = ((alphas.*Y)'*X)'; end ================================================ FILE: machine-learning-ex6/ex6/visualizeBoundary.m ================================================ function visualizeBoundary(X, y, model, varargin) %VISUALIZEBOUNDARY plots a non-linear decision boundary learned by the SVM % VISUALIZEBOUNDARYLINEAR(X, y, model) plots a non-linear decision % boundary learned by the SVM and overlays the data on it % Plot the training data on top of the boundary plotData(X, y) % Make classification predictions over a grid of values x1plot = linspace(min(X(:,1)), max(X(:,1)), 100)'; x2plot = linspace(min(X(:,2)), max(X(:,2)), 100)'; [X1, X2] = meshgrid(x1plot, x2plot); vals = zeros(size(X1)); for i = 1:size(X1, 2) this_X = [X1(:, i), X2(:, i)]; vals(:, i) = svmPredict(model, this_X); end % Plot the SVM boundary hold on contour(X1, X2, vals, [0 1], 'Color', 'b'); %ԭһ󣬷ΧӦ[0 1] hold off; end ================================================ FILE: machine-learning-ex6/ex6/visualizeBoundaryLinear.m ================================================ function visualizeBoundaryLinear(X, y, model) %VISUALIZEBOUNDARYLINEAR plots a linear decision boundary learned by the %SVM % VISUALIZEBOUNDARYLINEAR(X, y, model) plots a linear decision boundary % learned by the SVM and overlays the data on it w = model.w; b = model.b; xp = linspace(min(X(:,1)), max(X(:,1)), 100); yp = - (w(1)*xp + b)/w(2); plotData(X, y); hold on; plot(xp, yp, '-b'); hold off end ================================================ FILE: machine-learning-ex6/ex6/vocab.txt ================================================ 1 aa 2 ab 3 abil 4 abl 5 about 6 abov 7 absolut 8 abus 9 ac 10 accept 11 access 12 accord 13 account 14 achiev 15 acquir 16 across 17 act 18 action 19 activ 20 actual 21 ad 22 adam 23 add 24 addit 25 address 26 administr 27 adult 28 advanc 29 advantag 30 advertis 31 advic 32 advis 33 ae 34 af 35 affect 36 affili 37 afford 38 africa 39 after 40 ag 41 again 42 against 43 agenc 44 agent 45 ago 46 agre 47 agreement 48 aid 49 air 50 al 51 alb 52 align 53 all 54 allow 55 almost 56 alon 57 along 58 alreadi 59 alsa 60 also 61 altern 62 although 63 alwai 64 am 65 amaz 66 america 67 american 68 among 69 amount 70 amp 71 an 72 analysi 73 analyst 74 and 75 ani 76 anim 77 announc 78 annual 79 annuiti 80 anoth 81 answer 82 anti 83 anumb 84 anybodi 85 anymor 86 anyon 87 anyth 88 anywai 89 anywher 90 aol 91 ap 92 apolog 93 app 94 appar 95 appear 96 appl 97 appli 98 applic 99 appreci 100 approach 101 approv 102 apt 103 ar 104 archiv 105 area 106 aren 107 argument 108 arial 109 arm 110 around 111 arrai 112 arriv 113 art 114 articl 115 artist 116 as 117 ascii 118 ask 119 asset 120 assist 121 associ 122 assum 123 assur 124 at 125 atol 126 attach 127 attack 128 attempt 129 attent 130 attornei 131 attract 132 audio 133 aug 134 august 135 author 136 auto 137 autom 138 automat 139 avail 140 averag 141 avoid 142 awai 143 awar 144 award 145 ba 146 babi 147 back 148 background 149 backup 150 bad 151 balanc 152 ban 153 bank 154 bar 155 base 156 basenumb 157 basi 158 basic 159 bb 160 bc 161 bd 162 be 163 beat 164 beberg 165 becaus 166 becom 167 been 168 befor 169 begin 170 behalf 171 behavior 172 behind 173 believ 174 below 175 benefit 176 best 177 beta 178 better 179 between 180 bf 181 big 182 bill 183 billion 184 bin 185 binari 186 bit 187 black 188 blank 189 block 190 blog 191 blood 192 blue 193 bnumber 194 board 195 bodi 196 boi 197 bonu 198 book 199 boot 200 border 201 boss 202 boston 203 botan 204 both 205 bottl 206 bottom 207 boundari 208 box 209 brain 210 brand 211 break 212 brian 213 bring 214 broadcast 215 broker 216 browser 217 bug 218 bui 219 build 220 built 221 bulk 222 burn 223 bush 224 busi 225 but 226 button 227 by 228 byte 229 ca 230 cabl 231 cach 232 calcul 233 california 234 call 235 came 236 camera 237 campaign 238 can 239 canada 240 cannot 241 canon 242 capabl 243 capillari 244 capit 245 car 246 card 247 care 248 career 249 carri 250 cartridg 251 case 252 cash 253 cat 254 catch 255 categori 256 caus 257 cb 258 cc 259 cd 260 ce 261 cell 262 cent 263 center 264 central 265 centuri 266 ceo 267 certain 268 certainli 269 cf 270 challeng 271 chanc 272 chang 273 channel 274 char 275 charact 276 charg 277 charset 278 chat 279 cheap 280 check 281 cheer 282 chief 283 children 284 china 285 chip 286 choic 287 choos 288 chri 289 citi 290 citizen 291 civil 292 claim 293 class 294 classifi 295 clean 296 clear 297 clearli 298 click 299 client 300 close 301 clue 302 cnet 303 cnumber 304 co 305 code 306 collect 307 colleg 308 color 309 com 310 combin 311 come 312 comfort 313 command 314 comment 315 commentari 316 commerci 317 commiss 318 commit 319 common 320 commun 321 compani 322 compar 323 comparison 324 compat 325 compet 326 competit 327 compil 328 complet 329 comprehens 330 comput 331 concentr 332 concept 333 concern 334 condit 335 conf 336 confer 337 confid 338 confidenti 339 config 340 configur 341 confirm 342 conflict 343 confus 344 congress 345 connect 346 consid 347 consolid 348 constitut 349 construct 350 consult 351 consum 352 contact 353 contain 354 content 355 continu 356 contract 357 contribut 358 control 359 conveni 360 convers 361 convert 362 cool 363 cooper 364 copi 365 copyright 366 core 367 corpor 368 correct 369 correspond 370 cost 371 could 372 couldn 373 count 374 countri 375 coupl 376 cours 377 court 378 cover 379 coverag 380 crash 381 creat 382 creativ 383 credit 384 critic 385 cross 386 cultur 387 current 388 custom 389 cut 390 cv 391 da 392 dagga 393 dai 394 daili 395 dan 396 danger 397 dark 398 data 399 databas 400 datapow 401 date 402 dave 403 david 404 dc 405 de 406 dead 407 deal 408 dear 409 death 410 debt 411 decad 412 decid 413 decis 414 declar 415 declin 416 decor 417 default 418 defend 419 defens 420 defin 421 definit 422 degre 423 delai 424 delet 425 deliv 426 deliveri 427 dell 428 demand 429 democrat 430 depart 431 depend 432 deposit 433 describ 434 descript 435 deserv 436 design 437 desir 438 desktop 439 despit 440 detail 441 detect 442 determin 443 dev 444 devel 445 develop 446 devic 447 di 448 dial 449 did 450 didn 451 diet 452 differ 453 difficult 454 digit 455 direct 456 directli 457 director 458 directori 459 disabl 460 discount 461 discov 462 discoveri 463 discuss 464 disk 465 displai 466 disposit 467 distanc 468 distribut 469 dn 470 dnumber 471 do 472 doc 473 document 474 doe 475 doer 476 doesn 477 dollar 478 dollarac 479 dollarnumb 480 domain 481 don 482 done 483 dont 484 doubl 485 doubt 486 down 487 download 488 dr 489 draw 490 dream 491 drive 492 driver 493 drop 494 drug 495 due 496 dure 497 dvd 498 dw 499 dynam 500 ea 501 each 502 earli 503 earlier 504 earn 505 earth 506 easi 507 easier 508 easili 509 eat 510 eb 511 ebai 512 ec 513 echo 514 econom 515 economi 516 ed 517 edg 518 edit 519 editor 520 educ 521 eff 522 effect 523 effici 524 effort 525 either 526 el 527 electron 528 elimin 529 els 530 email 531 emailaddr 532 emerg 533 empir 534 employ 535 employe 536 en 537 enabl 538 encod 539 encourag 540 end 541 enemi 542 enenkio 543 energi 544 engin 545 english 546 enhanc 547 enjoi 548 enough 549 ensur 550 enter 551 enterpris 552 entertain 553 entir 554 entri 555 enumb 556 environ 557 equal 558 equip 559 equival 560 error 561 especi 562 essenti 563 establish 564 estat 565 estim 566 et 567 etc 568 euro 569 europ 570 european 571 even 572 event 573 eventu 574 ever 575 everi 576 everyon 577 everyth 578 evid 579 evil 580 exactli 581 exampl 582 excel 583 except 584 exchang 585 excit 586 exclus 587 execut 588 exercis 589 exist 590 exmh 591 expand 592 expect 593 expens 594 experi 595 expert 596 expir 597 explain 598 explor 599 express 600 extend 601 extens 602 extra 603 extract 604 extrem 605 ey 606 fa 607 face 608 fact 609 factor 610 fail 611 fair 612 fall 613 fals 614 famili 615 faq 616 far 617 fast 618 faster 619 fastest 620 fat 621 father 622 favorit 623 fax 624 fb 625 fd 626 featur 627 feder 628 fee 629 feed 630 feedback 631 feel 632 femal 633 few 634 ffffff 635 ffnumber 636 field 637 fight 638 figur 639 file 640 fill 641 film 642 filter 643 final 644 financ 645 financi 646 find 647 fine 648 finish 649 fire 650 firewal 651 firm 652 first 653 fit 654 five 655 fix 656 flag 657 flash 658 flow 659 fnumber 660 focu 661 folder 662 folk 663 follow 664 font 665 food 666 for 667 forc 668 foreign 669 forev 670 forget 671 fork 672 form 673 format 674 former 675 fortun 676 forward 677 found 678 foundat 679 four 680 franc 681 free 682 freedom 683 french 684 freshrpm 685 fri 686 fridai 687 friend 688 from 689 front 690 ftoc 691 ftp 692 full 693 fulli 694 fun 695 function 696 fund 697 further 698 futur 699 ga 700 gain 701 game 702 gari 703 garrigu 704 gave 705 gcc 706 geek 707 gener 708 get 709 gif 710 gift 711 girl 712 give 713 given 714 global 715 gnome 716 gnu 717 gnupg 718 go 719 goal 720 god 721 goe 722 gold 723 gone 724 good 725 googl 726 got 727 govern 728 gpl 729 grand 730 grant 731 graphic 732 great 733 greater 734 ground 735 group 736 grow 737 growth 738 gt 739 guarante 740 guess 741 gui 742 guid 743 ha 744 hack 745 had 746 half 747 ham 748 hand 749 handl 750 happen 751 happi 752 hard 753 hardwar 754 hat 755 hate 756 have 757 haven 758 he 759 head 760 header 761 headlin 762 health 763 hear 764 heard 765 heart 766 heaven 767 hei 768 height 769 held 770 hello 771 help 772 helvetica 773 her 774 herba 775 here 776 hermio 777 hettinga 778 hi 779 high 780 higher 781 highli 782 highlight 783 him 784 histori 785 hit 786 hold 787 home 788 honor 789 hope 790 host 791 hot 792 hour 793 hous 794 how 795 howev 796 hp 797 html 798 http 799 httpaddr 800 huge 801 human 802 hundr 803 ibm 804 id 805 idea 806 ident 807 identifi 808 idnumb 809 ie 810 if 811 ignor 812 ii 813 iii 814 iiiiiiihnumberjnumberhnumberjnumberhnumb 815 illeg 816 im 817 imag 818 imagin 819 immedi 820 impact 821 implement 822 import 823 impress 824 improv 825 in 826 inc 827 includ 828 incom 829 increas 830 incred 831 inde 832 independ 833 index 834 india 835 indian 836 indic 837 individu 838 industri 839 info 840 inform 841 initi 842 inlin 843 innov 844 input 845 insert 846 insid 847 instal 848 instanc 849 instant 850 instead 851 institut 852 instruct 853 insur 854 int 855 integr 856 intel 857 intellig 858 intend 859 interact 860 interest 861 interfac 862 intern 863 internet 864 interview 865 into 866 intro 867 introduc 868 inumb 869 invest 870 investig 871 investor 872 invok 873 involv 874 ip 875 ireland 876 irish 877 is 878 island 879 isn 880 iso 881 isp 882 issu 883 it 884 item 885 itself 886 jabber 887 jame 888 java 889 jim 890 jnumberiiiiiiihepihepihf 891 job 892 joe 893 john 894 join 895 journal 896 judg 897 judgment 898 jul 899 juli 900 jump 901 june 902 just 903 justin 904 keep 905 kei 906 kept 907 kernel 908 kevin 909 keyboard 910 kid 911 kill 912 kind 913 king 914 kingdom 915 knew 916 know 917 knowledg 918 known 919 la 920 lack 921 land 922 languag 923 laptop 924 larg 925 larger 926 largest 927 laser 928 last 929 late 930 later 931 latest 932 launch 933 law 934 lawrenc 935 le 936 lead 937 leader 938 learn 939 least 940 leav 941 left 942 legal 943 lender 944 length 945 less 946 lesson 947 let 948 letter 949 level 950 lib 951 librari 952 licens 953 life 954 lifetim 955 light 956 like 957 limit 958 line 959 link 960 linux 961 list 962 listen 963 littl 964 live 965 ll 966 lo 967 load 968 loan 969 local 970 locat 971 lock 972 lockergnom 973 log 974 long 975 longer 976 look 977 lose 978 loss 979 lost 980 lot 981 love 982 low 983 lower 984 lowest 985 lt 986 ma 987 mac 988 machin 989 made 990 magazin 991 mai 992 mail 993 mailer 994 main 995 maintain 996 major 997 make 998 maker 999 male 1000 man 1001 manag 1002 mani 1003 manual 1004 manufactur 1005 map 1006 march 1007 margin 1008 mark 1009 market 1010 marshal 1011 mass 1012 master 1013 match 1014 materi 1015 matter 1016 matthia 1017 mayb 1018 me 1019 mean 1020 measur 1021 mechan 1022 media 1023 medic 1024 meet 1025 member 1026 membership 1027 memori 1028 men 1029 mention 1030 menu 1031 merchant 1032 messag 1033 method 1034 mh 1035 michael 1036 microsoft 1037 middl 1038 might 1039 mike 1040 mile 1041 militari 1042 million 1043 mime 1044 mind 1045 mine 1046 mini 1047 minimum 1048 minut 1049 miss 1050 mistak 1051 mobil 1052 mode 1053 model 1054 modem 1055 modifi 1056 modul 1057 moment 1058 mon 1059 mondai 1060 monei 1061 monitor 1062 month 1063 monthli 1064 more 1065 morn 1066 mortgag 1067 most 1068 mostli 1069 mother 1070 motiv 1071 move 1072 movi 1073 mpnumber 1074 mr 1075 ms 1076 msg 1077 much 1078 multi 1079 multipart 1080 multipl 1081 murphi 1082 music 1083 must 1084 my 1085 myself 1086 name 1087 nation 1088 natur 1089 nbsp 1090 near 1091 nearli 1092 necessari 1093 need 1094 neg 1095 net 1096 netscap 1097 network 1098 never 1099 new 1100 newslett 1101 next 1102 nextpart 1103 nice 1104 nigeria 1105 night 1106 no 1107 nobodi 1108 non 1109 none 1110 nor 1111 normal 1112 north 1113 not 1114 note 1115 noth 1116 notic 1117 now 1118 nt 1119 null 1120 number 1121 numbera 1122 numberam 1123 numberanumb 1124 numberb 1125 numberbit 1126 numberc 1127 numbercb 1128 numbercbr 1129 numbercfont 1130 numbercli 1131 numbercnumb 1132 numbercp 1133 numberctd 1134 numberd 1135 numberdari 1136 numberdnumb 1137 numberenumb 1138 numberf 1139 numberfb 1140 numberff 1141 numberffont 1142 numberfp 1143 numberftd 1144 numberk 1145 numberm 1146 numbermb 1147 numberp 1148 numberpd 1149 numberpm 1150 numberpx 1151 numberst 1152 numberth 1153 numbertnumb 1154 numberx 1155 object 1156 oblig 1157 obtain 1158 obvious 1159 occur 1160 oct 1161 octob 1162 of 1163 off 1164 offer 1165 offic 1166 offici 1167 often 1168 oh 1169 ok 1170 old 1171 on 1172 onc 1173 onli 1174 onlin 1175 open 1176 oper 1177 opinion 1178 opportun 1179 opt 1180 optim 1181 option 1182 or 1183 order 1184 org 1185 organ 1186 origin 1187 os 1188 osdn 1189 other 1190 otherwis 1191 our 1192 out 1193 outlook 1194 output 1195 outsid 1196 over 1197 own 1198 owner 1199 oz 1200 pacif 1201 pack 1202 packag 1203 page 1204 pai 1205 paid 1206 pain 1207 palm 1208 panel 1209 paper 1210 paragraph 1211 parent 1212 part 1213 parti 1214 particip 1215 particular 1216 particularli 1217 partit 1218 partner 1219 pass 1220 password 1221 past 1222 patch 1223 patent 1224 path 1225 pattern 1226 paul 1227 payment 1228 pc 1229 peac 1230 peopl 1231 per 1232 percent 1233 percentag 1234 perfect 1235 perfectli 1236 perform 1237 perhap 1238 period 1239 perl 1240 perman 1241 permiss 1242 person 1243 pgp 1244 phone 1245 photo 1246 php 1247 phrase 1248 physic 1249 pick 1250 pictur 1251 piec 1252 piiiiiiii 1253 pipe 1254 pjnumber 1255 place 1256 plai 1257 plain 1258 plan 1259 planet 1260 plant 1261 planta 1262 platform 1263 player 1264 pleas 1265 plu 1266 plug 1267 pm 1268 pocket 1269 point 1270 polic 1271 polici 1272 polit 1273 poor 1274 pop 1275 popul 1276 popular 1277 port 1278 posit 1279 possibl 1280 post 1281 potenti 1282 pound 1283 powel 1284 power 1285 powershot 1286 practic 1287 pre 1288 predict 1289 prefer 1290 premium 1291 prepar 1292 present 1293 presid 1294 press 1295 pretti 1296 prevent 1297 previou 1298 previous 1299 price 1300 principl 1301 print 1302 printabl 1303 printer 1304 privaci 1305 privat 1306 prize 1307 pro 1308 probabl 1309 problem 1310 procedur 1311 process 1312 processor 1313 procmail 1314 produc 1315 product 1316 profession 1317 profil 1318 profit 1319 program 1320 programm 1321 progress 1322 project 1323 promis 1324 promot 1325 prompt 1326 properti 1327 propos 1328 proprietari 1329 prospect 1330 protect 1331 protocol 1332 prove 1333 proven 1334 provid 1335 proxi 1336 pub 1337 public 1338 publish 1339 pudg 1340 pull 1341 purchas 1342 purpos 1343 put 1344 python 1345 qnumber 1346 qualifi 1347 qualiti 1348 quarter 1349 question 1350 quick 1351 quickli 1352 quit 1353 quot 1354 radio 1355 ragga 1356 rais 1357 random 1358 rang 1359 rate 1360 rather 1361 ratio 1362 razor 1363 razornumb 1364 re 1365 reach 1366 read 1367 reader 1368 readi 1369 real 1370 realiz 1371 realli 1372 reason 1373 receiv 1374 recent 1375 recipi 1376 recommend 1377 record 1378 red 1379 redhat 1380 reduc 1381 refer 1382 refin 1383 reg 1384 regard 1385 region 1386 regist 1387 regul 1388 regular 1389 rel 1390 relat 1391 relationship 1392 releas 1393 relev 1394 reliabl 1395 remain 1396 rememb 1397 remot 1398 remov 1399 replac 1400 repli 1401 report 1402 repositori 1403 repres 1404 republ 1405 request 1406 requir 1407 research 1408 reserv 1409 resid 1410 resourc 1411 respect 1412 respond 1413 respons 1414 rest 1415 result 1416 retail 1417 return 1418 reveal 1419 revenu 1420 revers 1421 review 1422 revok 1423 rh 1424 rich 1425 right 1426 risk 1427 road 1428 robert 1429 rock 1430 role 1431 roll 1432 rom 1433 roman 1434 room 1435 root 1436 round 1437 rpm 1438 rss 1439 rule 1440 run 1441 sa 1442 safe 1443 sai 1444 said 1445 sale 1446 same 1447 sampl 1448 san 1449 saou 1450 sat 1451 satellit 1452 save 1453 saw 1454 scan 1455 schedul 1456 school 1457 scienc 1458 score 1459 screen 1460 script 1461 se 1462 search 1463 season 1464 second 1465 secret 1466 section 1467 secur 1468 see 1469 seed 1470 seek 1471 seem 1472 seen 1473 select 1474 self 1475 sell 1476 seminar 1477 send 1478 sender 1479 sendmail 1480 senior 1481 sens 1482 sensit 1483 sent 1484 sep 1485 separ 1486 septemb 1487 sequenc 1488 seri 1489 serif 1490 seriou 1491 serv 1492 server 1493 servic 1494 set 1495 setup 1496 seven 1497 seventh 1498 sever 1499 sex 1500 sexual 1501 sf 1502 shape 1503 share 1504 she 1505 shell 1506 ship 1507 shop 1508 short 1509 shot 1510 should 1511 show 1512 side 1513 sign 1514 signatur 1515 signific 1516 similar 1517 simpl 1518 simpli 1519 sinc 1520 sincer 1521 singl 1522 sit 1523 site 1524 situat 1525 six 1526 size 1527 skeptic 1528 skill 1529 skin 1530 skip 1531 sleep 1532 slow 1533 small 1534 smart 1535 smoke 1536 smtp 1537 snumber 1538 so 1539 social 1540 societi 1541 softwar 1542 sold 1543 solut 1544 solv 1545 some 1546 someon 1547 someth 1548 sometim 1549 son 1550 song 1551 soni 1552 soon 1553 sorri 1554 sort 1555 sound 1556 sourc 1557 south 1558 space 1559 spain 1560 spam 1561 spamassassin 1562 spamd 1563 spammer 1564 speak 1565 spec 1566 special 1567 specif 1568 specifi 1569 speech 1570 speed 1571 spend 1572 sponsor 1573 sport 1574 spot 1575 src 1576 ssh 1577 st 1578 stabl 1579 staff 1580 stai 1581 stand 1582 standard 1583 star 1584 start 1585 state 1586 statement 1587 statu 1588 step 1589 steve 1590 still 1591 stock 1592 stop 1593 storag 1594 store 1595 stori 1596 strategi 1597 stream 1598 street 1599 string 1600 strip 1601 strong 1602 structur 1603 studi 1604 stuff 1605 stupid 1606 style 1607 subject 1608 submit 1609 subscrib 1610 subscript 1611 substanti 1612 success 1613 such 1614 suffer 1615 suggest 1616 suit 1617 sum 1618 summari 1619 summer 1620 sun 1621 super 1622 suppli 1623 support 1624 suppos 1625 sure 1626 surpris 1627 suse 1628 suspect 1629 sweet 1630 switch 1631 system 1632 tab 1633 tabl 1634 tablet 1635 tag 1636 take 1637 taken 1638 talk 1639 tape 1640 target 1641 task 1642 tax 1643 teach 1644 team 1645 tech 1646 technic 1647 techniqu 1648 technolog 1649 tel 1650 telecom 1651 telephon 1652 tell 1653 temperatur 1654 templ 1655 ten 1656 term 1657 termin 1658 terror 1659 terrorist 1660 test 1661 texa 1662 text 1663 than 1664 thank 1665 that 1666 the 1667 thei 1668 their 1669 them 1670 themselv 1671 then 1672 theori 1673 there 1674 therefor 1675 these 1676 thi 1677 thing 1678 think 1679 thinkgeek 1680 third 1681 those 1682 though 1683 thought 1684 thousand 1685 thread 1686 threat 1687 three 1688 through 1689 thu 1690 thursdai 1691 ti 1692 ticket 1693 tim 1694 time 1695 tip 1696 tire 1697 titl 1698 tm 1699 to 1700 todai 1701 togeth 1702 token 1703 told 1704 toll 1705 tom 1706 toner 1707 toni 1708 too 1709 took 1710 tool 1711 top 1712 topic 1713 total 1714 touch 1715 toward 1716 track 1717 trade 1718 tradit 1719 traffic 1720 train 1721 transact 1722 transfer 1723 travel 1724 treat 1725 tree 1726 tri 1727 trial 1728 trick 1729 trip 1730 troubl 1731 true 1732 truli 1733 trust 1734 truth 1735 try 1736 tue 1737 tuesdai 1738 turn 1739 tv 1740 two 1741 type 1742 uk 1743 ultim 1744 un 1745 under 1746 understand 1747 unfortun 1748 uniqu 1749 unison 1750 unit 1751 univers 1752 unix 1753 unless 1754 unlik 1755 unlimit 1756 unseen 1757 unsolicit 1758 unsubscrib 1759 until 1760 up 1761 updat 1762 upgrad 1763 upon 1764 urgent 1765 url 1766 us 1767 usa 1768 usag 1769 usb 1770 usd 1771 usdollarnumb 1772 useless 1773 user 1774 usr 1775 usual 1776 util 1777 vacat 1778 valid 1779 valu 1780 valuabl 1781 var 1782 variabl 1783 varieti 1784 variou 1785 ve 1786 vendor 1787 ventur 1788 veri 1789 verifi 1790 version 1791 via 1792 video 1793 view 1794 virtual 1795 visa 1796 visit 1797 visual 1798 vnumber 1799 voic 1800 vote 1801 vs 1802 vulner 1803 wa 1804 wai 1805 wait 1806 wake 1807 walk 1808 wall 1809 want 1810 war 1811 warm 1812 warn 1813 warranti 1814 washington 1815 wasn 1816 wast 1817 watch 1818 water 1819 we 1820 wealth 1821 weapon 1822 web 1823 weblog 1824 websit 1825 wed 1826 wednesdai 1827 week 1828 weekli 1829 weight 1830 welcom 1831 well 1832 went 1833 were 1834 west 1835 what 1836 whatev 1837 when 1838 where 1839 whether 1840 which 1841 while 1842 white 1843 whitelist 1844 who 1845 whole 1846 whose 1847 why 1848 wi 1849 wide 1850 width 1851 wife 1852 will 1853 william 1854 win 1855 window 1856 wing 1857 winner 1858 wireless 1859 wish 1860 with 1861 within 1862 without 1863 wnumberp 1864 woman 1865 women 1866 won 1867 wonder 1868 word 1869 work 1870 worker 1871 world 1872 worldwid 1873 worri 1874 worst 1875 worth 1876 would 1877 wouldn 1878 write 1879 written 1880 wrong 1881 wrote 1882 www 1883 ximian 1884 xml 1885 xp 1886 yahoo 1887 ye 1888 yeah 1889 year 1890 yesterdai 1891 yet 1892 york 1893 you 1894 young 1895 your 1896 yourself 1897 zdnet 1898 zero 1899 zip ================================================ FILE: machine-learning-ex7/ex7/computeCentroids.m ================================================ function centroids = computeCentroids(X, idx, K) %COMPUTECENTROIDS returs the new centroids by computing the means of the %data points assigned to each centroid. % centroids = COMPUTECENTROIDS(X, idx, K) returns the new centroids by % computing the means of the data points assigned to each centroid. It is % given a dataset X where each row is a single data point, a vector % idx of centroid assignments (i.e. each entry in range [1..K]) for each % example, and K, the number of centroids. You should return a matrix % centroids, where each row of centroids is the mean of the data points % assigned to it. % % Useful variables [m n] = size(X); % You need to return the following variables correctly. centroids = zeros(K, n); % ====================== YOUR CODE HERE ====================== % Instructions: Go over every centroid and compute mean of all points that % belong to it. Concretely, the row vector centroids(i, :) % should contain the mean of the data points assigned to % centroid i. % % Note: You can use a for-loop over the centroids to compute this. % for i = 1:K centroids(i,:) = mean(X(idx==i,:),1); end % ============================================================= end ================================================ FILE: machine-learning-ex7/ex7/displayData.m ================================================ function [h, display_array] = displayData(X, example_width) %DISPLAYDATA Display 2D data in a nice grid % [h, display_array] = DISPLAYDATA(X, example_width) displays 2D data % stored in X in a nice grid. It returns the figure handle h and the % displayed array if requested. % Set example_width automatically if not passed in if ~exist('example_width', 'var') || isempty(example_width) example_width = round(sqrt(size(X, 2))); end % Gray Image colormap(gray); % Compute rows, cols [m n] = size(X); example_height = (n / example_width); % Compute number of items to display display_rows = floor(sqrt(m)); display_cols = ceil(m / display_rows); % Between images padding pad = 1; % Setup blank display display_array = - ones(pad + display_rows * (example_height + pad), ... pad + display_cols * (example_width + pad)); % Copy each example into a patch on the display array curr_ex = 1; for j = 1:display_rows for i = 1:display_cols if curr_ex > m, break; end % Copy the patch % Get the max value of the patch max_val = max(abs(X(curr_ex, :))); display_array(pad + (j - 1) * (example_height + pad) + (1:example_height), ... pad + (i - 1) * (example_width + pad) + (1:example_width)) = ... reshape(X(curr_ex, :), example_height, example_width) / max_val; curr_ex = curr_ex + 1; end if curr_ex > m, break; end end % Display Image h = imagesc(display_array, [-1 1]); % Do not show axis axis image off drawnow; end ================================================ FILE: machine-learning-ex7/ex7/drawLine.m ================================================ function drawLine(p1, p2, varargin) %DRAWLINE Draws a line from point p1 to point p2 % DRAWLINE(p1, p2) Draws a line from point p1 to point p2 and holds the % current figure plot([p1(1) p2(1)], [p1(2) p2(2)], varargin{:}); end ================================================ FILE: machine-learning-ex7/ex7/ex7.m ================================================ %% Machine Learning Online Class % Exercise 7 | Principle Component Analysis and K-Means Clustering % % Instructions % ------------ % % This file contains code that helps you get started on the % exercise. You will need to complete the following functions: % % pca.m % projectData.m % recoverData.m % computeCentroids.m % findClosestCentroids.m % kMeansInitCentroids.m % % For this exercise, you will not need to change any code in this file, % or any other files other than those mentioned above. % %% Initialization clear ; close all; clc %% ================= Part 1: Find Closest Centroids ==================== % To help you implement K-Means, we have divided the learning algorithm % into two functions -- findClosestCentroids and computeCentroids. In this % part, you shoudl complete the code in the findClosestCentroids function. % fprintf('Finding closest centroids.\n\n'); % Load an example dataset that we will be using load('ex7data2.mat'); % Select an initial set of centroids K = 3; % 3 Centroids initial_centroids = [3 3; 6 2; 8 5]; % Find the closest centroids for the examples using the % initial_centroids idx = findClosestCentroids(X, initial_centroids); fprintf('Closest centroids for the first 3 examples: \n') fprintf(' %d', idx(1:3)); fprintf('\n(the closest centroids should be 1, 3, 2 respectively)\n'); fprintf('Program paused. Press enter to continue.\n'); pause; %% ===================== Part 2: Compute Means ========================= % After implementing the closest centroids function, you should now % complete the computeCentroids function. % fprintf('\nComputing centroids means.\n\n'); % Compute means based on the closest centroids found in the previous part. centroids = computeCentroids(X, idx, K); fprintf('Centroids computed after initial finding of closest centroids: \n') fprintf(' %f %f \n' , centroids'); fprintf('\n(the centroids should be\n'); fprintf(' [ 2.428301 3.157924 ]\n'); fprintf(' [ 5.813503 2.633656 ]\n'); fprintf(' [ 7.119387 3.616684 ]\n\n'); fprintf('Program paused. Press enter to continue.\n'); pause; %% =================== Part 3: K-Means Clustering ====================== % After you have completed the two functions computeCentroids and % findClosestCentroids, you have all the necessary pieces to run the % kMeans algorithm. In this part, you will run the K-Means algorithm on % the example dataset we have provided. % fprintf('\nRunning K-Means clustering on example dataset.\n\n'); % Load an example dataset load('ex7data2.mat'); % Settings for running K-Means K = 3; max_iters = 10; % For consistency, here we set centroids to specific values % but in practice you want to generate them automatically, such as by % settings them to be random examples (as can be seen in % kMeansInitCentroids). initial_centroids = [3 3; 6 2; 8 5]; % Run K-Means algorithm. The 'true' at the end tells our function to plot % the progress of K-Means [centroids, idx] = runkMeans(X, initial_centroids, max_iters, true); fprintf('\nK-Means Done.\n\n'); fprintf('Program paused. Press enter to continue.\n'); pause; %% ============= Part 4: K-Means Clustering on Pixels =============== % In this exercise, you will use K-Means to compress an image. To do this, % you will first run K-Means on the colors of the pixels in the image and % then you will map each pixel on to it's closest centroid. % % You should now complete the code in kMeansInitCentroids.m % fprintf('\nRunning K-Means clustering on pixels from an image.\n\n'); % Load an image of a bird A = double(imread('bird_small.png')); % If imread does not work for you, you can try instead % load ('bird_small.mat'); A = A / 255; % Divide by 255 so that all values are in the range 0 - 1 % Size of the image img_size = size(A); % Reshape the image into an Nx3 matrix where N = number of pixels. % Each row will contain the Red, Green and Blue pixel values % This gives us our dataset matrix X that we will use K-Means on. X = reshape(A, img_size(1) * img_size(2), 3); % Run your K-Means algorithm on this data % You should try different values of K and max_iters here K = 16; max_iters = 10; % When using K-Means, it is important the initialize the centroids % randomly. % You should complete the code in kMeansInitCentroids.m before proceeding initial_centroids = kMeansInitCentroids(X, K); % Run K-Means [centroids, idx] = runkMeans(X, initial_centroids, max_iters); fprintf('Program paused. Press enter to continue.\n'); pause; %% ================= Part 5: Image Compression ====================== % In this part of the exercise, you will use the clusters of K-Means to % compress an image. To do this, we first find the closest clusters for % each example. After that, we fprintf('\nApplying K-Means to compress an image.\n\n'); % Find closest cluster members idx = findClosestCentroids(X, centroids); % Essentially, now we have represented the image X as in terms of the % indices in idx. % We can now recover the image from the indices (idx) by mapping each pixel % (specified by it's index in idx) to the centroid value X_recovered = centroids(idx,:); % Reshape the recovered image into proper dimensions X_recovered = reshape(X_recovered, img_size(1), img_size(2), 3); % Display the original image subplot(1, 2, 1); imagesc(A); title('Original'); % Display compressed image side by side subplot(1, 2, 2); imagesc(X_recovered) title(sprintf('Compressed, with %d colors.', K)); fprintf('Program paused. Press enter to continue.\n'); pause; ================================================ FILE: machine-learning-ex7/ex7/ex7_pca.m ================================================ %% Machine Learning Online Class % Exercise 7 | Principle Component Analysis and K-Means Clustering % % Instructions % ------------ % % This file contains code that helps you get started on the % exercise. You will need to complete the following functions: % % pca.m % projectData.m % recoverData.m % computeCentroids.m % findClosestCentroids.m % kMeansInitCentroids.m % % For this exercise, you will not need to change any code in this file, % or any other files other than those mentioned above. % %% Initialization clear ; close all; clc %% ================== Part 1: Load Example Dataset =================== % We start this exercise by using a small dataset that is easily to % visualize % fprintf('Visualizing example dataset for PCA.\n\n'); % The following command loads the dataset. You should now have the % variable X in your environment load ('ex7data1.mat'); % Visualize the example dataset plot(X(:, 1), X(:, 2), 'bo'); axis([0.5 6.5 2 8]); axis square; fprintf('Program paused. Press enter to continue.\n'); pause; %% =============== Part 2: Principal Component Analysis =============== % You should now implement PCA, a dimension reduction technique. You % should complete the code in pca.m % fprintf('\nRunning PCA on example dataset.\n\n'); % Before running PCA, it is important to first normalize X [X_norm, mu, sigma] = featureNormalize(X); % Run PCA [U, S] = pca(X_norm); % Compute mu, the mean of the each feature % Draw the eigenvectors centered at mean of data. These lines show the % directions of maximum variations in the dataset. hold on; drawLine(mu, mu + 1.5 * S(1,1) * U(:,1)', '-k', 'LineWidth', 2); drawLine(mu, mu + 1.5 * S(2,2) * U(:,2)', '-k', 'LineWidth', 2); hold off; fprintf('Top eigenvector: \n'); fprintf(' U(:,1) = %f %f \n', U(1,1), U(2,1)); fprintf('\n(you should expect to see -0.707107 -0.707107)\n'); fprintf('Program paused. Press enter to continue.\n'); pause; %% =================== Part 3: Dimension Reduction =================== % You should now implement the projection step to map the data onto the % first k eigenvectors. The code will then plot the data in this reduced % dimensional space. This will show you what the data looks like when % using only the corresponding eigenvectors to reconstruct it. % % You should complete the code in projectData.m % fprintf('\nDimension reduction on example dataset.\n\n'); % Plot the normalized dataset (returned from pca) plot(X_norm(:, 1), X_norm(:, 2), 'bo'); axis([-4 3 -4 3]); axis square % Project the data onto K = 1 dimension K = 1; Z = projectData(X_norm, U, K); fprintf('Projection of the first example: %f\n', Z(1)); fprintf('\n(this value should be about 1.481274)\n\n'); X_rec = recoverData(Z, U, K); fprintf('Approximation of the first example: %f %f\n', X_rec(1, 1), X_rec(1, 2)); fprintf('\n(this value should be about -1.047419 -1.047419)\n\n'); % Draw lines connecting the projected points to the original points hold on; plot(X_rec(:, 1), X_rec(:, 2), 'ro'); for i = 1:size(X_norm, 1) drawLine(X_norm(i,:), X_rec(i,:), '--k', 'LineWidth', 1); end hold off fprintf('Program paused. Press enter to continue.\n'); pause; %% =============== Part 4: Loading and Visualizing Face Data ============= % We start the exercise by first loading and visualizing the dataset. % The following code will load the dataset into your environment % fprintf('\nLoading face dataset.\n\n'); % Load Face dataset load ('ex7faces.mat') % Display the first 100 faces in the dataset displayData(X(1:100, :)); fprintf('Program paused. Press enter to continue.\n'); pause; %% =========== Part 5: PCA on Face Data: Eigenfaces =================== % Run PCA and visualize the eigenvectors which are in this case eigenfaces % We display the first 36 eigenfaces. % fprintf(['\nRunning PCA on face dataset.\n' ... '(this mght take a minute or two ...)\n\n']); % Before running PCA, it is important to first normalize X by subtracting % the mean value from each feature [X_norm, mu, sigma] = featureNormalize(X); % Run PCA [U, S] = pca(X_norm); % Visualize the top 36 eigenvectors found displayData(U(:, 1:36)'); fprintf('Program paused. Press enter to continue.\n'); pause; %% ============= Part 6: Dimension Reduction for Faces ================= % Project images to the eigen space using the top k eigenvectors % If you are applying a machine learning algorithm fprintf('\nDimension reduction for face dataset.\n\n'); K = 100; Z = projectData(X_norm, U, K); fprintf('The projected data Z has a size of: ') fprintf('%d ', size(Z)); fprintf('\n\nProgram paused. Press enter to continue.\n'); pause; %% ==== Part 7: Visualization of Faces after PCA Dimension Reduction ==== % Project images to the eigen space using the top K eigen vectors and % visualize only using those K dimensions % Compare to the original input, which is also displayed fprintf('\nVisualizing the projected (reduced dimension) faces.\n\n'); K = 100; X_rec = recoverData(Z, U, K); % Display normalized data subplot(1, 2, 1); displayData(X_norm(1:100,:)); title('Original faces'); axis square; % Display reconstructed data from only k eigenfaces subplot(1, 2, 2); displayData(X_rec(1:100,:)); title('Recovered faces'); axis square; fprintf('Program paused. Press enter to continue.\n'); pause; %% === Part 8(a): Optional (ungraded) Exercise: PCA for Visualization === % One useful application of PCA is to use it to visualize high-dimensional % data. In the last K-Means exercise you ran K-Means on 3-dimensional % pixel colors of an image. We first visualize this output in 3D, and then % apply PCA to obtain a visualization in 2D. close all; close all; clc % Re-load the image from the previous exercise and run K-Means on it % For this to work, you need to complete the K-Means assignment first A = double(imread('bird_small.png')); % If imread does not work for you, you can try instead % load ('bird_small.mat'); A = A / 255; img_size = size(A); X = reshape(A, img_size(1) * img_size(2), 3); K = 16; max_iters = 10; initial_centroids = kMeansInitCentroids(X, K); [centroids, idx] = runkMeans(X, initial_centroids, max_iters); % Sample 1000 random indexes (since working with all the data is % too expensive. If you have a fast computer, you may increase this. sel = floor(rand(1000, 1) * size(X, 1)) + 1; % Setup Color Palette palette = hsv(K); colors = palette(idx(sel), :); % Visualize the data and centroid memberships in 3D figure; scatter3(X(sel, 1), X(sel, 2), X(sel, 3), 10, colors); title('Pixel dataset plotted in 3D. Color shows centroid memberships'); fprintf('Program paused. Press enter to continue.\n'); pause; %% === Part 8(b): Optional (ungraded) Exercise: PCA for Visualization === % Use PCA to project this cloud to 2D for visualization % Subtract the mean to use PCA [X_norm, mu, sigma] = featureNormalize(X); % PCA and project the data to 2D [U, S] = pca(X_norm); Z = projectData(X_norm, U, 2); % Plot in 2D figure; plotDataPoints(Z(sel, :), idx(sel), K); title('Pixel dataset plotted in 2D, using PCA for dimensionality reduction'); fprintf('Program paused. Press enter to continue.\n'); pause; ================================================ FILE: machine-learning-ex7/ex7/ex7faces.mat ================================================ [File too large to display: 10.5 MB] ================================================ FILE: machine-learning-ex7/ex7/featureNormalize.m ================================================ function [X_norm, mu, sigma] = featureNormalize(X) %FEATURENORMALIZE Normalizes the features in X % FEATURENORMALIZE(X) returns a normalized version of X where % the mean value of each feature is 0 and the standard deviation % is 1. This is often a good preprocessing step to do when % working with learning algorithms. mu = mean(X); X_norm = bsxfun(@minus, X, mu); sigma = std(X_norm); X_norm = bsxfun(@rdivide, X_norm, sigma); % ============================================================ end ================================================ FILE: machine-learning-ex7/ex7/findClosestCentroids.m ================================================ function idx = findClosestCentroids(X, centroids) %FINDCLOSESTCENTROIDS computes the centroid memberships for every example % idx = FINDCLOSESTCENTROIDS (X, centroids) returns the closest centroids % in idx for a dataset X where each row is a single example. idx = m x 1 % vector of centroid assignments (i.e. each entry in range [1..K]) % % Set K K = size(centroids, 1); % You need to return the following variables correctly. idx = zeros(size(X,1), 1); % ====================== YOUR CODE HERE ====================== % Instructions: Go over every example, find its closest centroid, and store % the index inside idx at the appropriate location. % Concretely, idx(i) should contain the index of the centroid % closest to example i. Hence, it should be a value in the % range 1..K % % Note: You can use a for-loop over the examples to compute this. % m = size(X,1); for i = 1:m for j = 1:K idx(i,j) = (X(i,:)-centroids(j,:))*(X(i,:)-centroids(j,:))'; end end [dummy idx] = min(idx,[],2); % ============================================================= end ================================================ FILE: machine-learning-ex7/ex7/kMeansInitCentroids.m ================================================ function centroids = kMeansInitCentroids(X, K) %KMEANSINITCENTROIDS This function initializes K centroids that are to be %used in K-Means on the dataset X % centroids = KMEANSINITCENTROIDS(X, K) returns K initial centroids to be % used with the K-Means on the dataset X % % You should return this values correctly centroids = zeros(K, size(X, 2)); % ====================== YOUR CODE HERE ====================== % Instructions: You should set centroids to randomly chosen examples from % the dataset X % randidx = randperm(size(X, 1)); centroids = X(randidx(1:K), :); % ============================================================= end ================================================ FILE: machine-learning-ex7/ex7/lib/jsonlab/AUTHORS.txt ================================================ The author of "jsonlab" toolbox is Qianqian Fang. Qianqian is currently an Assistant Professor at Massachusetts General Hospital, Harvard Medical School. Address: Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School Bldg 149, 13th St, Charlestown, MA 02129, USA URL: http://nmr.mgh.harvard.edu/~fangq/ Email: or The script loadjson.m was built upon previous works by - Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713 date: 2009/11/02 - François Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393 date: 2009/03/22 - Joel Feenstra: http://www.mathworks.com/matlabcentral/fileexchange/20565 date: 2008/07/03 This toolbox contains patches submitted by the following contributors: - Blake Johnson part of revision 341 - Niclas Borlin various fixes in revision 394, including - loadjson crashes for all-zero sparse matrix. - loadjson crashes for empty sparse matrix. - Non-zero size of 0-by-N and N-by-0 empty matrices is lost after savejson/loadjson. - loadjson crashes for sparse real column vector. - loadjson crashes for sparse complex column vector. - Data is corrupted by savejson for sparse real row vector. - savejson crashes for sparse complex row vector. - Yul Kang patches for svn revision 415. - savejson saves an empty cell array as [] instead of null - loadjson differentiates an empty struct from an empty array ================================================ FILE: machine-learning-ex7/ex7/lib/jsonlab/ChangeLog.txt ================================================ ============================================================================ JSONlab - a toolbox to encode/decode JSON/UBJSON files in MATLAB/Octave ---------------------------------------------------------------------------- JSONlab ChangeLog (key features marked by *): == JSONlab 1.0 (codename: Optimus - Final), FangQ == 2015/01/02 polish help info for all major functions, update examples, finalize 1.0 2014/12/19 fix a bug to strictly respect NoRowBracket in savejson == JSONlab 1.0.0-RC2 (codename: Optimus - RC2), FangQ == 2014/11/22 show progress bar in loadjson ('ShowProgress') 2014/11/17 add Compact option in savejson to output compact JSON format ('Compact') 2014/11/17 add FastArrayParser in loadjson to specify fast parser applicable levels 2014/09/18 start official github mirror: https://github.com/fangq/jsonlab == JSONlab 1.0.0-RC1 (codename: Optimus - RC1), FangQ == 2014/09/17 fix several compatibility issues when running on octave versions 3.2-3.8 2014/09/17 support 2D cell and struct arrays in both savejson and saveubjson 2014/08/04 escape special characters in a JSON string 2014/02/16 fix a bug when saving ubjson files == JSONlab 0.9.9 (codename: Optimus - beta), FangQ == 2014/01/22 use binary read and write in saveubjson and loadubjson == JSONlab 0.9.8-1 (codename: Optimus - alpha update 1), FangQ == 2013/10/07 better round-trip conservation for empty arrays and structs (patch submitted by Yul Kang) == JSONlab 0.9.8 (codename: Optimus - alpha), FangQ == 2013/08/23 *universal Binary JSON (UBJSON) support, including both saveubjson and loadubjson == JSONlab 0.9.1 (codename: Rodimus, update 1), FangQ == 2012/12/18 *handling of various empty and sparse matrices (fixes submitted by Niclas Borlin) == JSONlab 0.9.0 (codename: Rodimus), FangQ == 2012/06/17 *new format for an invalid leading char, unpacking hex code in savejson 2012/06/01 support JSONP in savejson 2012/05/25 fix the empty cell bug (reported by Cyril Davin) 2012/04/05 savejson can save to a file (suggested by Patrick Rapin) == JSONlab 0.8.1 (codename: Sentiel, Update 1), FangQ == 2012/02/28 loadjson quotation mark escape bug, see http://bit.ly/yyk1nS 2012/01/25 patch to handle root-less objects, contributed by Blake Johnson == JSONlab 0.8.0 (codename: Sentiel), FangQ == 2012/01/13 *speed up loadjson by 20 fold when parsing large data arrays in matlab 2012/01/11 remove row bracket if an array has 1 element, suggested by Mykel Kochenderfer 2011/12/22 *accept sequence of 'param',value input in savejson and loadjson 2011/11/18 fix struct array bug reported by Mykel Kochenderfer == JSONlab 0.5.1 (codename: Nexus Update 1), FangQ == 2011/10/21 fix a bug in loadjson, previous code does not use any of the acceleration 2011/10/20 loadjson supports JSON collections - concatenated JSON objects == JSONlab 0.5.0 (codename: Nexus), FangQ == 2011/10/16 package and release jsonlab 0.5.0 2011/10/15 *add json demo and regression test, support cpx numbers, fix double quote bug 2011/10/11 *speed up readjson dramatically, interpret _Array* tags, show data in root level 2011/10/10 create jsonlab project, start jsonlab website, add online documentation 2011/10/07 *speed up savejson by 25x using sprintf instead of mat2str, add options support 2011/10/06 *savejson works for structs, cells and arrays 2011/09/09 derive loadjson from JSON parser from MATLAB Central, draft savejson.m ================================================ FILE: machine-learning-ex7/ex7/lib/jsonlab/LICENSE_BSD.txt ================================================ Copyright 2011-2015 Qianqian Fang . All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. The views and conclusions contained in the software and documentation are those of the authors and should not be interpreted as representing official policies, either expressed or implied, of the copyright holders. ================================================ FILE: machine-learning-ex7/ex7/lib/jsonlab/README.txt ================================================ =============================================================================== = JSONLab = = An open-source MATLAB/Octave JSON encoder and decoder = =============================================================================== *Copyright (C) 2011-2015 Qianqian Fang *License: BSD License, see License_BSD.txt for details *Version: 1.0 (Optimus - Final) ------------------------------------------------------------------------------- Table of Content: I. Introduction II. Installation III.Using JSONLab IV. Known Issues and TODOs V. Contribution and feedback ------------------------------------------------------------------------------- I. Introduction JSON ([http://www.json.org/ JavaScript Object Notation]) is a highly portable, human-readable and "[http://en.wikipedia.org/wiki/JSON fat-free]" text format to represent complex and hierarchical data. It is as powerful as [http://en.wikipedia.org/wiki/XML XML], but less verbose. JSON format is widely used for data-exchange in applications, and is essential for the wild success of [http://en.wikipedia.org/wiki/Ajax_(programming) Ajax] and [http://en.wikipedia.org/wiki/Web_2.0 Web2.0]. UBJSON (Universal Binary JSON) is a binary JSON format, specifically optimized for compact file size and better performance while keeping the semantics as simple as the text-based JSON format. Using the UBJSON format allows to wrap complex binary data in a flexible and extensible structure, making it possible to process complex and large dataset without accuracy loss due to text conversions. We envision that both JSON and its binary version will serve as part of the mainstream data-exchange formats for scientific research in the future. It will provide the flexibility and generality achieved by other popular general-purpose file specifications, such as [http://www.hdfgroup.org/HDF5/whatishdf5.html HDF5], with significantly reduced complexity and enhanced performance. JSONLab is a free and open-source implementation of a JSON/UBJSON encoder and a decoder in the native MATLAB language. It can be used to convert a MATLAB data structure (array, struct, cell, struct array and cell array) into JSON/UBJSON formatted strings, or to decode a JSON/UBJSON file into MATLAB data structure. JSONLab supports both MATLAB and [http://www.gnu.org/software/octave/ GNU Octave] (a free MATLAB clone). ------------------------------------------------------------------------------- II. Installation The installation of JSONLab is no different than any other simple MATLAB toolbox. You only need to download/unzip the JSONLab package to a folder, and add the folder's path to MATLAB/Octave's path list by using the following command: addpath('/path/to/jsonlab'); If you want to add this path permanently, you need to type "pathtool", browse to the jsonlab root folder and add to the list, then click "Save". Then, run "rehash" in MATLAB, and type "which loadjson", if you see an output, that means JSONLab is installed for MATLAB/Octave. ------------------------------------------------------------------------------- III.Using JSONLab JSONLab provides two functions, loadjson.m -- a MATLAB->JSON decoder, and savejson.m -- a MATLAB->JSON encoder, for the text-based JSON, and two equivallent functions -- loadubjson and saveubjson for the binary JSON. The detailed help info for the four functions can be found below: === loadjson.m === 
  data=loadjson(fname,opt)
     or
  data=loadjson(fname,'param1',value1,'param2',value2,...)
 
  parse a JSON (JavaScript Object Notation) file or string
 
  authors:Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2011/09/09, including previous works from 
 
          Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713
             created on 2009/11/02
          Franois Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393
             created on  2009/03/22
          Joel Feenstra:
          http://www.mathworks.com/matlabcentral/fileexchange/20565
             created on 2008/07/03
 
  $Id: loadjson.m 452 2014-11-22 16:43:33Z fangq $
 
  input:
       fname: input file name, if fname contains "{}" or "[]", fname
              will be interpreted as a JSON string
       opt: a struct to store parsing options, opt can be replaced by 
            a list of ('param',value) pairs - the param string is equivallent
            to a field in opt. opt can have the following 
            fields (first in [.|.] is the default)
 
            opt.SimplifyCell [0|1]: if set to 1, loadjson will call cell2mat
                          for each element of the JSON data, and group 
                          arrays based on the cell2mat rules.
            opt.FastArrayParser [1|0 or integer]: if set to 1, use a
                          speed-optimized array parser when loading an 
                          array object. The fast array parser may 
                          collapse block arrays into a single large
                          array similar to rules defined in cell2mat; 0 to 
                          use a legacy parser; if set to a larger-than-1
                          value, this option will specify the minimum
                          dimension to enable the fast array parser. For
                          example, if the input is a 3D array, setting
                          FastArrayParser to 1 will return a 3D array;
                          setting to 2 will return a cell array of 2D
                          arrays; setting to 3 will return to a 2D cell
                          array of 1D vectors; setting to 4 will return a
                          3D cell array.
            opt.ShowProgress [0|1]: if set to 1, loadjson displays a progress bar.
 
  output:
       dat: a cell array, where {...} blocks are converted into cell arrays,
            and [...] are converted to arrays
 
  examples:
       dat=loadjson('{"obj":{"string":"value","array":[1,2,3]}}')
       dat=loadjson(['examples' filesep 'example1.json'])
       dat=loadjson(['examples' filesep 'example1.json'],'SimplifyCell',1)
=== savejson.m === 
  json=savejson(rootname,obj,filename)
     or
  json=savejson(rootname,obj,opt)
  json=savejson(rootname,obj,'param1',value1,'param2',value2,...)
 
  convert a MATLAB object (cell, struct or array) into a JSON (JavaScript
  Object Notation) string
 
  author: Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2011/09/09
 
  $Id: savejson.m 458 2014-12-19 22:17:17Z fangq $
 
  input:
       rootname: the name of the root-object, when set to '', the root name
         is ignored, however, when opt.ForceRootName is set to 1 (see below),
         the MATLAB variable name will be used as the root name.
       obj: a MATLAB object (array, cell, cell array, struct, struct array).
       filename: a string for the file name to save the output JSON data.
       opt: a struct for additional options, ignore to use default values.
         opt can have the following fields (first in [.|.] is the default)
 
         opt.FileName [''|string]: a file name to save the output JSON data
         opt.FloatFormat ['%.10g'|string]: format to show each numeric element
                          of a 1D/2D array;
         opt.ArrayIndent [1|0]: if 1, output explicit data array with
                          precedent indentation; if 0, no indentation
         opt.ArrayToStruct[0|1]: when set to 0, savejson outputs 1D/2D
                          array in JSON array format; if sets to 1, an
                          array will be shown as a struct with fields
                          "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for
                          sparse arrays, the non-zero elements will be
                          saved to _ArrayData_ field in triplet-format i.e.
                          (ix,iy,val) and "_ArrayIsSparse_" will be added
                          with a value of 1; for a complex array, the 
                          _ArrayData_ array will include two columns 
                          (4 for sparse) to record the real and imaginary 
                          parts, and also "_ArrayIsComplex_":1 is added. 
         opt.ParseLogical [0|1]: if this is set to 1, logical array elem
                          will use true/false rather than 1/0.
         opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single
                          numerical element will be shown without a square
                          bracket, unless it is the root object; if 0, square
                          brackets are forced for any numerical arrays.
         opt.ForceRootName [0|1]: when set to 1 and rootname is empty, savejson
                          will use the name of the passed obj variable as the 
                          root object name; if obj is an expression and 
                          does not have a name, 'root' will be used; if this 
                          is set to 0 and rootname is empty, the root level 
                          will be merged down to the lower level.
         opt.Inf ['"$1_Inf_"'|string]: a customized regular expression pattern
                          to represent +/-Inf. The matched pattern is '([-+]*)Inf'
                          and $1 represents the sign. For those who want to use
                          1e999 to represent Inf, they can set opt.Inf to '$11e999'
         opt.NaN ['"_NaN_"'|string]: a customized regular expression pattern
                          to represent NaN
         opt.JSONP [''|string]: to generate a JSONP output (JSON with padding),
                          for example, if opt.JSONP='foo', the JSON data is
                          wrapped inside a function call as 'foo(...);'
         opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson 
                          back to the string form
         opt.SaveBinary [0|1]: 1 - save the JSON file in binary mode; 0 - text mode.
         opt.Compact [0|1]: 1- out compact JSON format (remove all newlines and tabs)
 
         opt can be replaced by a list of ('param',value) pairs. The param 
         string is equivallent to a field in opt and is case sensitive.
  output:
       json: a string in the JSON format (see http://json.org)
 
  examples:
       jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... 
                'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],...
                'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;...
                           2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],...
                'MeshCreator','FangQ','MeshTitle','T6 Cube',...
                'SpecialData',[nan, inf, -inf]);
       savejson('jmesh',jsonmesh)
       savejson('',jsonmesh,'ArrayIndent',0,'FloatFormat','\t%.5g')
=== loadubjson.m === 
  data=loadubjson(fname,opt)
     or
  data=loadubjson(fname,'param1',value1,'param2',value2,...)
 
  parse a JSON (JavaScript Object Notation) file or string
 
  authors:Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2013/08/01
 
  $Id: loadubjson.m 436 2014-08-05 20:51:40Z fangq $
 
  input:
       fname: input file name, if fname contains "{}" or "[]", fname
              will be interpreted as a UBJSON string
       opt: a struct to store parsing options, opt can be replaced by 
            a list of ('param',value) pairs - the param string is equivallent
            to a field in opt. opt can have the following 
            fields (first in [.|.] is the default)
 
            opt.SimplifyCell [0|1]: if set to 1, loadubjson will call cell2mat
                          for each element of the JSON data, and group 
                          arrays based on the cell2mat rules.
            opt.IntEndian [B|L]: specify the endianness of the integer fields
                          in the UBJSON input data. B - Big-Endian format for 
                          integers (as required in the UBJSON specification); 
                          L - input integer fields are in Little-Endian order.
 
  output:
       dat: a cell array, where {...} blocks are converted into cell arrays,
            and [...] are converted to arrays
 
  examples:
       obj=struct('string','value','array',[1 2 3]);
       ubjdata=saveubjson('obj',obj);
       dat=loadubjson(ubjdata)
       dat=loadubjson(['examples' filesep 'example1.ubj'])
       dat=loadubjson(['examples' filesep 'example1.ubj'],'SimplifyCell',1)
=== saveubjson.m === 
  json=saveubjson(rootname,obj,filename)
     or
  json=saveubjson(rootname,obj,opt)
  json=saveubjson(rootname,obj,'param1',value1,'param2',value2,...)
 
  convert a MATLAB object (cell, struct or array) into a Universal 
  Binary JSON (UBJSON) binary string
 
  author: Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2013/08/17
 
  $Id: saveubjson.m 440 2014-09-17 19:59:45Z fangq $
 
  input:
       rootname: the name of the root-object, when set to '', the root name
         is ignored, however, when opt.ForceRootName is set to 1 (see below),
         the MATLAB variable name will be used as the root name.
       obj: a MATLAB object (array, cell, cell array, struct, struct array)
       filename: a string for the file name to save the output UBJSON data
       opt: a struct for additional options, ignore to use default values.
         opt can have the following fields (first in [.|.] is the default)
 
         opt.FileName [''|string]: a file name to save the output JSON data
         opt.ArrayToStruct[0|1]: when set to 0, saveubjson outputs 1D/2D
                          array in JSON array format; if sets to 1, an
                          array will be shown as a struct with fields
                          "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for
                          sparse arrays, the non-zero elements will be
                          saved to _ArrayData_ field in triplet-format i.e.
                          (ix,iy,val) and "_ArrayIsSparse_" will be added
                          with a value of 1; for a complex array, the 
                          _ArrayData_ array will include two columns 
                          (4 for sparse) to record the real and imaginary 
                          parts, and also "_ArrayIsComplex_":1 is added. 
         opt.ParseLogical [1|0]: if this is set to 1, logical array elem
                          will use true/false rather than 1/0.
         opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single
                          numerical element will be shown without a square
                          bracket, unless it is the root object; if 0, square
                          brackets are forced for any numerical arrays.
         opt.ForceRootName [0|1]: when set to 1 and rootname is empty, saveubjson
                          will use the name of the passed obj variable as the 
                          root object name; if obj is an expression and 
                          does not have a name, 'root' will be used; if this 
                          is set to 0 and rootname is empty, the root level 
                          will be merged down to the lower level.
         opt.JSONP [''|string]: to generate a JSONP output (JSON with padding),
                          for example, if opt.JSON='foo', the JSON data is
                          wrapped inside a function call as 'foo(...);'
         opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson 
                          back to the string form
 
         opt can be replaced by a list of ('param',value) pairs. The param 
         string is equivallent to a field in opt and is case sensitive.
  output:
       json: a binary string in the UBJSON format (see http://ubjson.org)
 
  examples:
       jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... 
                'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],...
                'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;...
                           2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],...
                'MeshCreator','FangQ','MeshTitle','T6 Cube',...
                'SpecialData',[nan, inf, -inf]);
       saveubjson('jsonmesh',jsonmesh)
       saveubjson('jsonmesh',jsonmesh,'meshdata.ubj')
=== examples === Under the "examples" folder, you can find several scripts to demonstrate the basic utilities of JSONLab. Running the "demo_jsonlab_basic.m" script, you will see the conversions from MATLAB data structure to JSON text and backward. In "jsonlab_selftest.m", we load complex JSON files downloaded from the Internet and validate the loadjson/savejson functions for regression testing purposes. Similarly, a "demo_ubjson_basic.m" script is provided to test the saveubjson and loadubjson pairs for various matlab data structures. Please run these examples and understand how JSONLab works before you use it to process your data. ------------------------------------------------------------------------------- IV. Known Issues and TODOs JSONLab has several known limitations. We are striving to make it more general and robust. Hopefully in a few future releases, the limitations become less. Here are the known issues: # 3D or higher dimensional cell/struct-arrays will be converted to 2D arrays; # When processing names containing multi-byte characters, Octave and MATLAB \ can give different field-names; you can use feature('DefaultCharacterSet','latin1') \ in MATLAB to get consistant results # savejson can not handle class and dataset. # saveubjson converts a logical array into a uint8 ([U]) array # an unofficial N-D array count syntax is implemented in saveubjson. We are \ actively communicating with the UBJSON spec maintainer to investigate the \ possibility of making it upstream # loadubjson can not parse all UBJSON Specification (Draft 9) compliant \ files, however, it can parse all UBJSON files produced by saveubjson. ------------------------------------------------------------------------------- V. Contribution and feedback JSONLab is an open-source project. This means you can not only use it and modify it as you wish, but also you can contribute your changes back to JSONLab so that everyone else can enjoy the improvement. For anyone who want to contribute, please download JSONLab source code from it's subversion repository by using the following command: svn checkout svn://svn.code.sf.net/p/iso2mesh/code/trunk/jsonlab jsonlab You can make changes to the files as needed. Once you are satisfied with your changes, and ready to share it with others, please cd the root directory of JSONLab, and type svn diff > yourname_featurename.patch You then email the .patch file to JSONLab's maintainer, Qianqian Fang, at the email address shown in the beginning of this file. Qianqian will review the changes and commit it to the subversion if they are satisfactory. We appreciate any suggestions and feedbacks from you. Please use iso2mesh's mailing list to report any questions you may have with JSONLab: http://groups.google.com/group/iso2mesh-users?hl=en&pli=1 (Subscription to the mailing list is needed in order to post messages). ================================================ FILE: machine-learning-ex7/ex7/lib/jsonlab/jsonopt.m ================================================ function val=jsonopt(key,default,varargin) % % val=jsonopt(key,default,optstruct) % % setting options based on a struct. The struct can be produced % by varargin2struct from a list of 'param','value' pairs % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % % $Id: loadjson.m 371 2012-06-20 12:43:06Z fangq $ % % input: % key: a string with which one look up a value from a struct % default: if the key does not exist, return default % optstruct: a struct where each sub-field is a key % % output: % val: if key exists, val=optstruct.key; otherwise val=default % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % val=default; if(nargin<=2) return; end opt=varargin{1}; if(isstruct(opt) && isfield(opt,key)) val=getfield(opt,key); end ================================================ FILE: machine-learning-ex7/ex7/lib/jsonlab/loadjson.m ================================================ function data = loadjson(fname,varargin) % % data=loadjson(fname,opt) % or % data=loadjson(fname,'param1',value1,'param2',value2,...) % % parse a JSON (JavaScript Object Notation) file or string % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2011/09/09, including previous works from % % Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713 % created on 2009/11/02 % François Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393 % created on 2009/03/22 % Joel Feenstra: % http://www.mathworks.com/matlabcentral/fileexchange/20565 % created on 2008/07/03 % % $Id: loadjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % fname: input file name, if fname contains "{}" or "[]", fname % will be interpreted as a JSON string % opt: a struct to store parsing options, opt can be replaced by % a list of ('param',value) pairs - the param string is equivallent % to a field in opt. opt can have the following % fields (first in [.|.] is the default) % % opt.SimplifyCell [0|1]: if set to 1, loadjson will call cell2mat % for each element of the JSON data, and group % arrays based on the cell2mat rules. % opt.FastArrayParser [1|0 or integer]: if set to 1, use a % speed-optimized array parser when loading an % array object. The fast array parser may % collapse block arrays into a single large % array similar to rules defined in cell2mat; 0 to % use a legacy parser; if set to a larger-than-1 % value, this option will specify the minimum % dimension to enable the fast array parser. For % example, if the input is a 3D array, setting % FastArrayParser to 1 will return a 3D array; % setting to 2 will return a cell array of 2D % arrays; setting to 3 will return to a 2D cell % array of 1D vectors; setting to 4 will return a % 3D cell array. % opt.ShowProgress [0|1]: if set to 1, loadjson displays a progress bar. % % output: % dat: a cell array, where {...} blocks are converted into cell arrays, % and [...] are converted to arrays % % examples: % dat=loadjson('{"obj":{"string":"value","array":[1,2,3]}}') % dat=loadjson(['examples' filesep 'example1.json']) % dat=loadjson(['examples' filesep 'example1.json'],'SimplifyCell',1) % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % global pos inStr len esc index_esc len_esc isoct arraytoken if(regexp(fname,'[\{\}\]\[]','once')) string=fname; elseif(exist(fname,'file')) fid = fopen(fname,'rb'); string = fread(fid,inf,'uint8=>char')'; fclose(fid); else error('input file does not exist'); end pos = 1; len = length(string); inStr = string; isoct=exist('OCTAVE_VERSION','builtin'); arraytoken=find(inStr=='[' | inStr==']' | inStr=='"'); jstr=regexprep(inStr,'\\\\',' '); escquote=regexp(jstr,'\\"'); arraytoken=sort([arraytoken escquote]); % String delimiters and escape chars identified to improve speed: esc = find(inStr=='"' | inStr=='\' ); % comparable to: regexp(inStr, '["\\]'); index_esc = 1; len_esc = length(esc); opt=varargin2struct(varargin{:}); if(jsonopt('ShowProgress',0,opt)==1) opt.progressbar_=waitbar(0,'loading ...'); end jsoncount=1; while pos <= len switch(next_char) case '{' data{jsoncount} = parse_object(opt); case '[' data{jsoncount} = parse_array(opt); otherwise error_pos('Outer level structure must be an object or an array'); end jsoncount=jsoncount+1; end % while jsoncount=length(data); if(jsoncount==1 && iscell(data)) data=data{1}; end if(~isempty(data)) if(isstruct(data)) % data can be a struct array data=jstruct2array(data); elseif(iscell(data)) data=jcell2array(data); end end if(isfield(opt,'progressbar_')) close(opt.progressbar_); end %% function newdata=jcell2array(data) len=length(data); newdata=data; for i=1:len if(isstruct(data{i})) newdata{i}=jstruct2array(data{i}); elseif(iscell(data{i})) newdata{i}=jcell2array(data{i}); end end %%------------------------------------------------------------------------- function newdata=jstruct2array(data) fn=fieldnames(data); newdata=data; len=length(data); for i=1:length(fn) % depth-first for j=1:len if(isstruct(getfield(data(j),fn{i}))) newdata(j)=setfield(newdata(j),fn{i},jstruct2array(getfield(data(j),fn{i}))); end end end if(~isempty(strmatch('x0x5F_ArrayType_',fn)) && ~isempty(strmatch('x0x5F_ArrayData_',fn))) newdata=cell(len,1); for j=1:len ndata=cast(data(j).x0x5F_ArrayData_,data(j).x0x5F_ArrayType_); iscpx=0; if(~isempty(strmatch('x0x5F_ArrayIsComplex_',fn))) if(data(j).x0x5F_ArrayIsComplex_) iscpx=1; end end if(~isempty(strmatch('x0x5F_ArrayIsSparse_',fn))) if(data(j).x0x5F_ArrayIsSparse_) if(~isempty(strmatch('x0x5F_ArraySize_',fn))) dim=data(j).x0x5F_ArraySize_; if(iscpx && size(ndata,2)==4-any(dim==1)) ndata(:,end-1)=complex(ndata(:,end-1),ndata(:,end)); end if isempty(ndata) % All-zeros sparse ndata=sparse(dim(1),prod(dim(2:end))); elseif dim(1)==1 % Sparse row vector ndata=sparse(1,ndata(:,1),ndata(:,2),dim(1),prod(dim(2:end))); elseif dim(2)==1 % Sparse column vector ndata=sparse(ndata(:,1),1,ndata(:,2),dim(1),prod(dim(2:end))); else % Generic sparse array. ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3),dim(1),prod(dim(2:end))); end else if(iscpx && size(ndata,2)==4) ndata(:,3)=complex(ndata(:,3),ndata(:,4)); end ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3)); end end elseif(~isempty(strmatch('x0x5F_ArraySize_',fn))) if(iscpx && size(ndata,2)==2) ndata=complex(ndata(:,1),ndata(:,2)); end ndata=reshape(ndata(:),data(j).x0x5F_ArraySize_); end newdata{j}=ndata; end if(len==1) newdata=newdata{1}; end end %%------------------------------------------------------------------------- function object = parse_object(varargin) parse_char('{'); object = []; if next_char ~= '}' while 1 str = parseStr(varargin{:}); if isempty(str) error_pos('Name of value at position %d cannot be empty'); end parse_char(':'); val = parse_value(varargin{:}); eval( sprintf( 'object.%s = val;', valid_field(str) ) ); if next_char == '}' break; end parse_char(','); end end parse_char('}'); %%------------------------------------------------------------------------- function object = parse_array(varargin) % JSON array is written in row-major order global pos inStr isoct parse_char('['); object = cell(0, 1); dim2=[]; arraydepth=jsonopt('JSONLAB_ArrayDepth_',1,varargin{:}); pbar=jsonopt('progressbar_',-1,varargin{:}); if next_char ~= ']' if(jsonopt('FastArrayParser',1,varargin{:})>=1 && arraydepth>=jsonopt('FastArrayParser',1,varargin{:})) [endpos, e1l, e1r, maxlevel]=matching_bracket(inStr,pos); arraystr=['[' inStr(pos:endpos)]; arraystr=regexprep(arraystr,'"_NaN_"','NaN'); arraystr=regexprep(arraystr,'"([-+]*)_Inf_"','$1Inf'); arraystr(arraystr==sprintf('\n'))=[]; arraystr(arraystr==sprintf('\r'))=[]; %arraystr=regexprep(arraystr,'\s*,',','); % this is slow,sometimes needed if(~isempty(e1l) && ~isempty(e1r)) % the array is in 2D or higher D astr=inStr((e1l+1):(e1r-1)); astr=regexprep(astr,'"_NaN_"','NaN'); astr=regexprep(astr,'"([-+]*)_Inf_"','$1Inf'); astr(astr==sprintf('\n'))=[]; astr(astr==sprintf('\r'))=[]; astr(astr==' ')=''; if(isempty(find(astr=='[', 1))) % array is 2D dim2=length(sscanf(astr,'%f,',[1 inf])); end else % array is 1D astr=arraystr(2:end-1); astr(astr==' ')=''; [obj, count, errmsg, nextidx]=sscanf(astr,'%f,',[1,inf]); if(nextidx>=length(astr)-1) object=obj; pos=endpos; parse_char(']'); return; end end if(~isempty(dim2)) astr=arraystr; astr(astr=='[')=''; astr(astr==']')=''; astr(astr==' ')=''; [obj, count, errmsg, nextidx]=sscanf(astr,'%f,',inf); if(nextidx>=length(astr)-1) object=reshape(obj,dim2,numel(obj)/dim2)'; pos=endpos; parse_char(']'); if(pbar>0) waitbar(pos/length(inStr),pbar,'loading ...'); end return; end end arraystr=regexprep(arraystr,'\]\s*,','];'); else arraystr='['; end try if(isoct && regexp(arraystr,'"','once')) error('Octave eval can produce empty cells for JSON-like input'); end object=eval(arraystr); pos=endpos; catch while 1 newopt=varargin2struct(varargin{:},'JSONLAB_ArrayDepth_',arraydepth+1); val = parse_value(newopt); object{end+1} = val; if next_char == ']' break; end parse_char(','); end end end if(jsonopt('SimplifyCell',0,varargin{:})==1) try oldobj=object; object=cell2mat(object')'; if(iscell(oldobj) && isstruct(object) && numel(object)>1 && jsonopt('SimplifyCellArray',1,varargin{:})==0) object=oldobj; elseif(size(object,1)>1 && ndims(object)==2) object=object'; end catch end end parse_char(']'); if(pbar>0) waitbar(pos/length(inStr),pbar,'loading ...'); end %%------------------------------------------------------------------------- function parse_char(c) global pos inStr len skip_whitespace; if pos > len || inStr(pos) ~= c error_pos(sprintf('Expected %c at position %%d', c)); else pos = pos + 1; skip_whitespace; end %%------------------------------------------------------------------------- function c = next_char global pos inStr len skip_whitespace; if pos > len c = []; else c = inStr(pos); end %%------------------------------------------------------------------------- function skip_whitespace global pos inStr len while pos <= len && isspace(inStr(pos)) pos = pos + 1; end %%------------------------------------------------------------------------- function str = parseStr(varargin) global pos inStr len esc index_esc len_esc % len, ns = length(inStr), keyboard if inStr(pos) ~= '"' error_pos('String starting with " expected at position %d'); else pos = pos + 1; end str = ''; while pos <= len while index_esc <= len_esc && esc(index_esc) < pos index_esc = index_esc + 1; end if index_esc > len_esc str = [str inStr(pos:len)]; pos = len + 1; break; else str = [str inStr(pos:esc(index_esc)-1)]; pos = esc(index_esc); end nstr = length(str); switch inStr(pos) case '"' pos = pos + 1; if(~isempty(str)) if(strcmp(str,'_Inf_')) str=Inf; elseif(strcmp(str,'-_Inf_')) str=-Inf; elseif(strcmp(str,'_NaN_')) str=NaN; end end return; case '\' if pos+1 > len error_pos('End of file reached right after escape character'); end pos = pos + 1; switch inStr(pos) case {'"' '\' '/'} str(nstr+1) = inStr(pos); pos = pos + 1; case {'b' 'f' 'n' 'r' 't'} str(nstr+1) = sprintf(['\' inStr(pos)]); pos = pos + 1; case 'u' if pos+4 > len error_pos('End of file reached in escaped unicode character'); end str(nstr+(1:6)) = inStr(pos-1:pos+4); pos = pos + 5; end otherwise % should never happen str(nstr+1) = inStr(pos), keyboard pos = pos + 1; end end error_pos('End of file while expecting end of inStr'); %%------------------------------------------------------------------------- function num = parse_number(varargin) global pos inStr len isoct currstr=inStr(pos:end); numstr=0; if(isoct~=0) numstr=regexp(currstr,'^\s*-?(?:0|[1-9]\d*)(?:\.\d+)?(?:[eE][+\-]?\d+)?','end'); [num, one] = sscanf(currstr, '%f', 1); delta=numstr+1; else [num, one, err, delta] = sscanf(currstr, '%f', 1); if ~isempty(err) error_pos('Error reading number at position %d'); end end pos = pos + delta-1; %%------------------------------------------------------------------------- function val = parse_value(varargin) global pos inStr len true = 1; false = 0; pbar=jsonopt('progressbar_',-1,varargin{:}); if(pbar>0) waitbar(pos/len,pbar,'loading ...'); end switch(inStr(pos)) case '"' val = parseStr(varargin{:}); return; case '[' val = parse_array(varargin{:}); return; case '{' val = parse_object(varargin{:}); if isstruct(val) if(~isempty(strmatch('x0x5F_ArrayType_',fieldnames(val), 'exact'))) val=jstruct2array(val); end elseif isempty(val) val = struct; end return; case {'-','0','1','2','3','4','5','6','7','8','9'} val = parse_number(varargin{:}); return; case 't' if pos+3 <= len && strcmpi(inStr(pos:pos+3), 'true') val = true; pos = pos + 4; return; end case 'f' if pos+4 <= len && strcmpi(inStr(pos:pos+4), 'false') val = false; pos = pos + 5; return; end case 'n' if pos+3 <= len && strcmpi(inStr(pos:pos+3), 'null') val = []; pos = pos + 4; return; end end error_pos('Value expected at position %d'); %%------------------------------------------------------------------------- function error_pos(msg) global pos inStr len poShow = max(min([pos-15 pos-1 pos pos+20],len),1); if poShow(3) == poShow(2) poShow(3:4) = poShow(2)+[0 -1]; % display nothing after end msg = [sprintf(msg, pos) ': ' ... inStr(poShow(1):poShow(2)) '' inStr(poShow(3):poShow(4)) ]; error( ['JSONparser:invalidFormat: ' msg] ); %%------------------------------------------------------------------------- function str = valid_field(str) global isoct % From MATLAB doc: field names must begin with a letter, which may be % followed by any combination of letters, digits, and underscores. % Invalid characters will be converted to underscores, and the prefix % "x0x[Hex code]_" will be added if the first character is not a letter. pos=regexp(str,'^[^A-Za-z]','once'); if(~isempty(pos)) if(~isoct) str=regexprep(str,'^([^A-Za-z])','x0x${sprintf(''%X'',unicode2native($1))}_','once'); else str=sprintf('x0x%X_%s',char(str(1)),str(2:end)); end end if(isempty(regexp(str,'[^0-9A-Za-z_]', 'once' ))) return; end if(~isoct) str=regexprep(str,'([^0-9A-Za-z_])','_0x${sprintf(''%X'',unicode2native($1))}_'); else pos=regexp(str,'[^0-9A-Za-z_]'); if(isempty(pos)) return; end str0=str; pos0=[0 pos(:)' length(str)]; str=''; for i=1:length(pos) str=[str str0(pos0(i)+1:pos(i)-1) sprintf('_0x%X_',str0(pos(i)))]; end if(pos(end)~=length(str)) str=[str str0(pos0(end-1)+1:pos0(end))]; end end %str(~isletter(str) & ~('0' <= str & str <= '9')) = '_'; %%------------------------------------------------------------------------- function endpos = matching_quote(str,pos) len=length(str); while(pos1 && str(pos-1)=='\')) endpos=pos; return; end end pos=pos+1; end error('unmatched quotation mark'); %%------------------------------------------------------------------------- function [endpos, e1l, e1r, maxlevel] = matching_bracket(str,pos) global arraytoken level=1; maxlevel=level; endpos=0; bpos=arraytoken(arraytoken>=pos); tokens=str(bpos); len=length(tokens); pos=1; e1l=[]; e1r=[]; while(pos<=len) c=tokens(pos); if(c==']') level=level-1; if(isempty(e1r)) e1r=bpos(pos); end if(level==0) endpos=bpos(pos); return end end if(c=='[') if(isempty(e1l)) e1l=bpos(pos); end level=level+1; maxlevel=max(maxlevel,level); end if(c=='"') pos=matching_quote(tokens,pos+1); end pos=pos+1; end if(endpos==0) error('unmatched "]"'); end ================================================ FILE: machine-learning-ex7/ex7/lib/jsonlab/loadubjson.m ================================================ function data = loadubjson(fname,varargin) % % data=loadubjson(fname,opt) % or % data=loadubjson(fname,'param1',value1,'param2',value2,...) % % parse a JSON (JavaScript Object Notation) file or string % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2013/08/01 % % $Id: loadubjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % fname: input file name, if fname contains "{}" or "[]", fname % will be interpreted as a UBJSON string % opt: a struct to store parsing options, opt can be replaced by % a list of ('param',value) pairs - the param string is equivallent % to a field in opt. opt can have the following % fields (first in [.|.] is the default) % % opt.SimplifyCell [0|1]: if set to 1, loadubjson will call cell2mat % for each element of the JSON data, and group % arrays based on the cell2mat rules. % opt.IntEndian [B|L]: specify the endianness of the integer fields % in the UBJSON input data. B - Big-Endian format for % integers (as required in the UBJSON specification); % L - input integer fields are in Little-Endian order. % % output: % dat: a cell array, where {...} blocks are converted into cell arrays, % and [...] are converted to arrays % % examples: % obj=struct('string','value','array',[1 2 3]); % ubjdata=saveubjson('obj',obj); % dat=loadubjson(ubjdata) % dat=loadubjson(['examples' filesep 'example1.ubj']) % dat=loadubjson(['examples' filesep 'example1.ubj'],'SimplifyCell',1) % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % global pos inStr len esc index_esc len_esc isoct arraytoken fileendian systemendian if(regexp(fname,'[\{\}\]\[]','once')) string=fname; elseif(exist(fname,'file')) fid = fopen(fname,'rb'); string = fread(fid,inf,'uint8=>char')'; fclose(fid); else error('input file does not exist'); end pos = 1; len = length(string); inStr = string; isoct=exist('OCTAVE_VERSION','builtin'); arraytoken=find(inStr=='[' | inStr==']' | inStr=='"'); jstr=regexprep(inStr,'\\\\',' '); escquote=regexp(jstr,'\\"'); arraytoken=sort([arraytoken escquote]); % String delimiters and escape chars identified to improve speed: esc = find(inStr=='"' | inStr=='\' ); % comparable to: regexp(inStr, '["\\]'); index_esc = 1; len_esc = length(esc); opt=varargin2struct(varargin{:}); fileendian=upper(jsonopt('IntEndian','B',opt)); [os,maxelem,systemendian]=computer; jsoncount=1; while pos <= len switch(next_char) case '{' data{jsoncount} = parse_object(opt); case '[' data{jsoncount} = parse_array(opt); otherwise error_pos('Outer level structure must be an object or an array'); end jsoncount=jsoncount+1; end % while jsoncount=length(data); if(jsoncount==1 && iscell(data)) data=data{1}; end if(~isempty(data)) if(isstruct(data)) % data can be a struct array data=jstruct2array(data); elseif(iscell(data)) data=jcell2array(data); end end %% function newdata=parse_collection(id,data,obj) if(jsoncount>0 && exist('data','var')) if(~iscell(data)) newdata=cell(1); newdata{1}=data; data=newdata; end end %% function newdata=jcell2array(data) len=length(data); newdata=data; for i=1:len if(isstruct(data{i})) newdata{i}=jstruct2array(data{i}); elseif(iscell(data{i})) newdata{i}=jcell2array(data{i}); end end %%------------------------------------------------------------------------- function newdata=jstruct2array(data) fn=fieldnames(data); newdata=data; len=length(data); for i=1:length(fn) % depth-first for j=1:len if(isstruct(getfield(data(j),fn{i}))) newdata(j)=setfield(newdata(j),fn{i},jstruct2array(getfield(data(j),fn{i}))); end end end if(~isempty(strmatch('x0x5F_ArrayType_',fn)) && ~isempty(strmatch('x0x5F_ArrayData_',fn))) newdata=cell(len,1); for j=1:len ndata=cast(data(j).x0x5F_ArrayData_,data(j).x0x5F_ArrayType_); iscpx=0; if(~isempty(strmatch('x0x5F_ArrayIsComplex_',fn))) if(data(j).x0x5F_ArrayIsComplex_) iscpx=1; end end if(~isempty(strmatch('x0x5F_ArrayIsSparse_',fn))) if(data(j).x0x5F_ArrayIsSparse_) if(~isempty(strmatch('x0x5F_ArraySize_',fn))) dim=double(data(j).x0x5F_ArraySize_); if(iscpx && size(ndata,2)==4-any(dim==1)) ndata(:,end-1)=complex(ndata(:,end-1),ndata(:,end)); end if isempty(ndata) % All-zeros sparse ndata=sparse(dim(1),prod(dim(2:end))); elseif dim(1)==1 % Sparse row vector ndata=sparse(1,ndata(:,1),ndata(:,2),dim(1),prod(dim(2:end))); elseif dim(2)==1 % Sparse column vector ndata=sparse(ndata(:,1),1,ndata(:,2),dim(1),prod(dim(2:end))); else % Generic sparse array. ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3),dim(1),prod(dim(2:end))); end else if(iscpx && size(ndata,2)==4) ndata(:,3)=complex(ndata(:,3),ndata(:,4)); end ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3)); end end elseif(~isempty(strmatch('x0x5F_ArraySize_',fn))) if(iscpx && size(ndata,2)==2) ndata=complex(ndata(:,1),ndata(:,2)); end ndata=reshape(ndata(:),data(j).x0x5F_ArraySize_); end newdata{j}=ndata; end if(len==1) newdata=newdata{1}; end end %%------------------------------------------------------------------------- function object = parse_object(varargin) parse_char('{'); object = []; type=''; count=-1; if(next_char == '$') type=inStr(pos+1); % TODO pos=pos+2; end if(next_char == '#') pos=pos+1; count=double(parse_number()); end if next_char ~= '}' num=0; while 1 str = parseStr(varargin{:}); if isempty(str) error_pos('Name of value at position %d cannot be empty'); end %parse_char(':'); val = parse_value(varargin{:}); num=num+1; eval( sprintf( 'object.%s = val;', valid_field(str) ) ); if next_char == '}' || (count>=0 && num>=count) break; end %parse_char(','); end end if(count==-1) parse_char('}'); end %%------------------------------------------------------------------------- function [cid,len]=elem_info(type) id=strfind('iUIlLdD',type); dataclass={'int8','uint8','int16','int32','int64','single','double'}; bytelen=[1,1,2,4,8,4,8]; if(id>0) cid=dataclass{id}; len=bytelen(id); else error_pos('unsupported type at position %d'); end %%------------------------------------------------------------------------- function [data adv]=parse_block(type,count,varargin) global pos inStr isoct fileendian systemendian [cid,len]=elem_info(type); datastr=inStr(pos:pos+len*count-1); if(isoct) newdata=int8(datastr); else newdata=uint8(datastr); end id=strfind('iUIlLdD',type); if(id<=5 && fileendian~=systemendian) newdata=swapbytes(typecast(newdata,cid)); end data=typecast(newdata,cid); adv=double(len*count); %%------------------------------------------------------------------------- function object = parse_array(varargin) % JSON array is written in row-major order global pos inStr isoct parse_char('['); object = cell(0, 1); dim=[]; type=''; count=-1; if(next_char == '$') type=inStr(pos+1); pos=pos+2; end if(next_char == '#') pos=pos+1; if(next_char=='[') dim=parse_array(varargin{:}); count=prod(double(dim)); else count=double(parse_number()); end end if(~isempty(type)) if(count>=0) [object adv]=parse_block(type,count,varargin{:}); if(~isempty(dim)) object=reshape(object,dim); end pos=pos+adv; return; else endpos=matching_bracket(inStr,pos); [cid,len]=elem_info(type); count=(endpos-pos)/len; [object adv]=parse_block(type,count,varargin{:}); pos=pos+adv; parse_char(']'); return; end end if next_char ~= ']' while 1 val = parse_value(varargin{:}); object{end+1} = val; if next_char == ']' break; end %parse_char(','); end end if(jsonopt('SimplifyCell',0,varargin{:})==1) try oldobj=object; object=cell2mat(object')'; if(iscell(oldobj) && isstruct(object) && numel(object)>1 && jsonopt('SimplifyCellArray',1,varargin{:})==0) object=oldobj; elseif(size(object,1)>1 && ndims(object)==2) object=object'; end catch end end if(count==-1) parse_char(']'); end %%------------------------------------------------------------------------- function parse_char(c) global pos inStr len skip_whitespace; if pos > len || inStr(pos) ~= c error_pos(sprintf('Expected %c at position %%d', c)); else pos = pos + 1; skip_whitespace; end %%------------------------------------------------------------------------- function c = next_char global pos inStr len skip_whitespace; if pos > len c = []; else c = inStr(pos); end %%------------------------------------------------------------------------- function skip_whitespace global pos inStr len while pos <= len && isspace(inStr(pos)) pos = pos + 1; end %%------------------------------------------------------------------------- function str = parseStr(varargin) global pos inStr esc index_esc len_esc % len, ns = length(inStr), keyboard type=inStr(pos); if type ~= 'S' && type ~= 'C' && type ~= 'H' error_pos('String starting with S expected at position %d'); else pos = pos + 1; end if(type == 'C') str=inStr(pos); pos=pos+1; return; end bytelen=double(parse_number()); if(length(inStr)>=pos+bytelen-1) str=inStr(pos:pos+bytelen-1); pos=pos+bytelen; else error_pos('End of file while expecting end of inStr'); end %%------------------------------------------------------------------------- function num = parse_number(varargin) global pos inStr len isoct fileendian systemendian id=strfind('iUIlLdD',inStr(pos)); if(isempty(id)) error_pos('expecting a number at position %d'); end type={'int8','uint8','int16','int32','int64','single','double'}; bytelen=[1,1,2,4,8,4,8]; datastr=inStr(pos+1:pos+bytelen(id)); if(isoct) newdata=int8(datastr); else newdata=uint8(datastr); end if(id<=5 && fileendian~=systemendian) newdata=swapbytes(typecast(newdata,type{id})); end num=typecast(newdata,type{id}); pos = pos + bytelen(id)+1; %%------------------------------------------------------------------------- function val = parse_value(varargin) global pos inStr len true = 1; false = 0; switch(inStr(pos)) case {'S','C','H'} val = parseStr(varargin{:}); return; case '[' val = parse_array(varargin{:}); return; case '{' val = parse_object(varargin{:}); if isstruct(val) if(~isempty(strmatch('x0x5F_ArrayType_',fieldnames(val), 'exact'))) val=jstruct2array(val); end elseif isempty(val) val = struct; end return; case {'i','U','I','l','L','d','D'} val = parse_number(varargin{:}); return; case 'T' val = true; pos = pos + 1; return; case 'F' val = false; pos = pos + 1; return; case {'Z','N'} val = []; pos = pos + 1; return; end error_pos('Value expected at position %d'); %%------------------------------------------------------------------------- function error_pos(msg) global pos inStr len poShow = max(min([pos-15 pos-1 pos pos+20],len),1); if poShow(3) == poShow(2) poShow(3:4) = poShow(2)+[0 -1]; % display nothing after end msg = [sprintf(msg, pos) ': ' ... inStr(poShow(1):poShow(2)) '' inStr(poShow(3):poShow(4)) ]; error( ['JSONparser:invalidFormat: ' msg] ); %%------------------------------------------------------------------------- function str = valid_field(str) global isoct % From MATLAB doc: field names must begin with a letter, which may be % followed by any combination of letters, digits, and underscores. % Invalid characters will be converted to underscores, and the prefix % "x0x[Hex code]_" will be added if the first character is not a letter. pos=regexp(str,'^[^A-Za-z]','once'); if(~isempty(pos)) if(~isoct) str=regexprep(str,'^([^A-Za-z])','x0x${sprintf(''%X'',unicode2native($1))}_','once'); else str=sprintf('x0x%X_%s',char(str(1)),str(2:end)); end end if(isempty(regexp(str,'[^0-9A-Za-z_]', 'once' ))) return; end if(~isoct) str=regexprep(str,'([^0-9A-Za-z_])','_0x${sprintf(''%X'',unicode2native($1))}_'); else pos=regexp(str,'[^0-9A-Za-z_]'); if(isempty(pos)) return; end str0=str; pos0=[0 pos(:)' length(str)]; str=''; for i=1:length(pos) str=[str str0(pos0(i)+1:pos(i)-1) sprintf('_0x%X_',str0(pos(i)))]; end if(pos(end)~=length(str)) str=[str str0(pos0(end-1)+1:pos0(end))]; end end %str(~isletter(str) & ~('0' <= str & str <= '9')) = '_'; %%------------------------------------------------------------------------- function endpos = matching_quote(str,pos) len=length(str); while(pos1 && str(pos-1)=='\')) endpos=pos; return; end end pos=pos+1; end error('unmatched quotation mark'); %%------------------------------------------------------------------------- function [endpos e1l e1r maxlevel] = matching_bracket(str,pos) global arraytoken level=1; maxlevel=level; endpos=0; bpos=arraytoken(arraytoken>=pos); tokens=str(bpos); len=length(tokens); pos=1; e1l=[]; e1r=[]; while(pos<=len) c=tokens(pos); if(c==']') level=level-1; if(isempty(e1r)) e1r=bpos(pos); end if(level==0) endpos=bpos(pos); return end end if(c=='[') if(isempty(e1l)) e1l=bpos(pos); end level=level+1; maxlevel=max(maxlevel,level); end if(c=='"') pos=matching_quote(tokens,pos+1); end pos=pos+1; end if(endpos==0) error('unmatched "]"'); end ================================================ FILE: machine-learning-ex7/ex7/lib/jsonlab/mergestruct.m ================================================ function s=mergestruct(s1,s2) % % s=mergestruct(s1,s2) % % merge two struct objects into one % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % date: 2012/12/22 % % input: % s1,s2: a struct object, s1 and s2 can not be arrays % % output: % s: the merged struct object. fields in s1 and s2 will be combined in s. % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(~isstruct(s1) || ~isstruct(s2)) error('input parameters contain non-struct'); end if(length(s1)>1 || length(s2)>1) error('can not merge struct arrays'); end fn=fieldnames(s2); s=s1; for i=1:length(fn) s=setfield(s,fn{i},getfield(s2,fn{i})); end ================================================ FILE: machine-learning-ex7/ex7/lib/jsonlab/savejson.m ================================================ function json=savejson(rootname,obj,varargin) % % json=savejson(rootname,obj,filename) % or % json=savejson(rootname,obj,opt) % json=savejson(rootname,obj,'param1',value1,'param2',value2,...) % % convert a MATLAB object (cell, struct or array) into a JSON (JavaScript % Object Notation) string % % author: Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2011/09/09 % % $Id: savejson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % rootname: the name of the root-object, when set to '', the root name % is ignored, however, when opt.ForceRootName is set to 1 (see below), % the MATLAB variable name will be used as the root name. % obj: a MATLAB object (array, cell, cell array, struct, struct array). % filename: a string for the file name to save the output JSON data. % opt: a struct for additional options, ignore to use default values. % opt can have the following fields (first in [.|.] is the default) % % opt.FileName [''|string]: a file name to save the output JSON data % opt.FloatFormat ['%.10g'|string]: format to show each numeric element % of a 1D/2D array; % opt.ArrayIndent [1|0]: if 1, output explicit data array with % precedent indentation; if 0, no indentation % opt.ArrayToStruct[0|1]: when set to 0, savejson outputs 1D/2D % array in JSON array format; if sets to 1, an % array will be shown as a struct with fields % "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for % sparse arrays, the non-zero elements will be % saved to _ArrayData_ field in triplet-format i.e. % (ix,iy,val) and "_ArrayIsSparse_" will be added % with a value of 1; for a complex array, the % _ArrayData_ array will include two columns % (4 for sparse) to record the real and imaginary % parts, and also "_ArrayIsComplex_":1 is added. % opt.ParseLogical [0|1]: if this is set to 1, logical array elem % will use true/false rather than 1/0. % opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single % numerical element will be shown without a square % bracket, unless it is the root object; if 0, square % brackets are forced for any numerical arrays. % opt.ForceRootName [0|1]: when set to 1 and rootname is empty, savejson % will use the name of the passed obj variable as the % root object name; if obj is an expression and % does not have a name, 'root' will be used; if this % is set to 0 and rootname is empty, the root level % will be merged down to the lower level. % opt.Inf ['"$1_Inf_"'|string]: a customized regular expression pattern % to represent +/-Inf. The matched pattern is '([-+]*)Inf' % and $1 represents the sign. For those who want to use % 1e999 to represent Inf, they can set opt.Inf to '$11e999' % opt.NaN ['"_NaN_"'|string]: a customized regular expression pattern % to represent NaN % opt.JSONP [''|string]: to generate a JSONP output (JSON with padding), % for example, if opt.JSONP='foo', the JSON data is % wrapped inside a function call as 'foo(...);' % opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson % back to the string form % opt.SaveBinary [0|1]: 1 - save the JSON file in binary mode; 0 - text mode. % opt.Compact [0|1]: 1- out compact JSON format (remove all newlines and tabs) % % opt can be replaced by a list of ('param',value) pairs. The param % string is equivallent to a field in opt and is case sensitive. % output: % json: a string in the JSON format (see http://json.org) % % examples: % jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... % 'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],... % 'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;... % 2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],... % 'MeshCreator','FangQ','MeshTitle','T6 Cube',... % 'SpecialData',[nan, inf, -inf]); % savejson('jmesh',jsonmesh) % savejson('',jsonmesh,'ArrayIndent',0,'FloatFormat','\t%.5g') % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(nargin==1) varname=inputname(1); obj=rootname; if(isempty(varname)) varname='root'; end rootname=varname; else varname=inputname(2); end if(length(varargin)==1 && ischar(varargin{1})) opt=struct('FileName',varargin{1}); else opt=varargin2struct(varargin{:}); end opt.IsOctave=exist('OCTAVE_VERSION','builtin'); rootisarray=0; rootlevel=1; forceroot=jsonopt('ForceRootName',0,opt); if((isnumeric(obj) || islogical(obj) || ischar(obj) || isstruct(obj) || iscell(obj)) && isempty(rootname) && forceroot==0) rootisarray=1; rootlevel=0; else if(isempty(rootname)) rootname=varname; end end if((isstruct(obj) || iscell(obj))&& isempty(rootname) && forceroot) rootname='root'; end whitespaces=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); if(jsonopt('Compact',0,opt)==1) whitespaces=struct('tab','','newline','','sep',','); end if(~isfield(opt,'whitespaces_')) opt.whitespaces_=whitespaces; end nl=whitespaces.newline; json=obj2json(rootname,obj,rootlevel,opt); if(rootisarray) json=sprintf('%s%s',json,nl); else json=sprintf('{%s%s%s}\n',nl,json,nl); end jsonp=jsonopt('JSONP','',opt); if(~isempty(jsonp)) json=sprintf('%s(%s);%s',jsonp,json,nl); end % save to a file if FileName is set, suggested by Patrick Rapin if(~isempty(jsonopt('FileName','',opt))) if(jsonopt('SaveBinary',0,opt)==1) fid = fopen(opt.FileName, 'wb'); fwrite(fid,json); else fid = fopen(opt.FileName, 'wt'); fwrite(fid,json,'char'); end fclose(fid); end %%------------------------------------------------------------------------- function txt=obj2json(name,item,level,varargin) if(iscell(item)) txt=cell2json(name,item,level,varargin{:}); elseif(isstruct(item)) txt=struct2json(name,item,level,varargin{:}); elseif(ischar(item)) txt=str2json(name,item,level,varargin{:}); else txt=mat2json(name,item,level,varargin{:}); end %%------------------------------------------------------------------------- function txt=cell2json(name,item,level,varargin) txt=''; if(~iscell(item)) error('input is not a cell'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); ws=jsonopt('whitespaces_',struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')),varargin{:}); padding0=repmat(ws.tab,1,level); padding2=repmat(ws.tab,1,level+1); nl=ws.newline; if(len>1) if(~isempty(name)) txt=sprintf('%s"%s": [%s',padding0, checkname(name,varargin{:}),nl); name=''; else txt=sprintf('%s[%s',padding0,nl); end elseif(len==0) if(~isempty(name)) txt=sprintf('%s"%s": []',padding0, checkname(name,varargin{:})); name=''; else txt=sprintf('%s[]',padding0); end end for j=1:dim(2) if(dim(1)>1) txt=sprintf('%s%s[%s',txt,padding2,nl); end for i=1:dim(1) txt=sprintf('%s%s',txt,obj2json(name,item{i,j},level+(dim(1)>1)+1,varargin{:})); if(i1) txt=sprintf('%s%s%s]',txt,nl,padding2); end if(j1) txt=sprintf('%s%s%s]',txt,nl,padding0); end %%------------------------------------------------------------------------- function txt=struct2json(name,item,level,varargin) txt=''; if(~isstruct(item)) error('input is not a struct'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); ws=struct('tab',sprintf('\t'),'newline',sprintf('\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding0=repmat(ws.tab,1,level); padding2=repmat(ws.tab,1,level+1); padding1=repmat(ws.tab,1,level+(dim(1)>1)+(len>1)); nl=ws.newline; if(~isempty(name)) if(len>1) txt=sprintf('%s"%s": [%s',padding0,checkname(name,varargin{:}),nl); end else if(len>1) txt=sprintf('%s[%s',padding0,nl); end end for j=1:dim(2) if(dim(1)>1) txt=sprintf('%s%s[%s',txt,padding2,nl); end for i=1:dim(1) names = fieldnames(item(i,j)); if(~isempty(name) && len==1) txt=sprintf('%s%s"%s": {%s',txt,padding1, checkname(name,varargin{:}),nl); else txt=sprintf('%s%s{%s',txt,padding1,nl); end if(~isempty(names)) for e=1:length(names) txt=sprintf('%s%s',txt,obj2json(names{e},getfield(item(i,j),... names{e}),level+(dim(1)>1)+1+(len>1),varargin{:})); if(e1) txt=sprintf('%s%s%s]',txt,nl,padding2); end if(j1) txt=sprintf('%s%s%s]',txt,nl,padding0); end %%------------------------------------------------------------------------- function txt=str2json(name,item,level,varargin) txt=''; if(~ischar(item)) error('input is not a string'); end item=reshape(item, max(size(item),[1 0])); len=size(item,1); ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding1=repmat(ws.tab,1,level); padding0=repmat(ws.tab,1,level+1); nl=ws.newline; sep=ws.sep; if(~isempty(name)) if(len>1) txt=sprintf('%s"%s": [%s',padding1,checkname(name,varargin{:}),nl); end else if(len>1) txt=sprintf('%s[%s',padding1,nl); end end isoct=jsonopt('IsOctave',0,varargin{:}); for e=1:len if(isoct) val=regexprep(item(e,:),'\\','\\'); val=regexprep(val,'"','\"'); val=regexprep(val,'^"','\"'); else val=regexprep(item(e,:),'\\','\\\\'); val=regexprep(val,'"','\\"'); val=regexprep(val,'^"','\\"'); end val=escapejsonstring(val); if(len==1) obj=['"' checkname(name,varargin{:}) '": ' '"',val,'"']; if(isempty(name)) obj=['"',val,'"']; end txt=sprintf('%s%s%s%s',txt,padding1,obj); else txt=sprintf('%s%s%s%s',txt,padding0,['"',val,'"']); end if(e==len) sep=''; end txt=sprintf('%s%s',txt,sep); end if(len>1) txt=sprintf('%s%s%s%s',txt,nl,padding1,']'); end %%------------------------------------------------------------------------- function txt=mat2json(name,item,level,varargin) if(~isnumeric(item) && ~islogical(item)) error('input is not an array'); end ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding1=repmat(ws.tab,1,level); padding0=repmat(ws.tab,1,level+1); nl=ws.newline; sep=ws.sep; if(length(size(item))>2 || issparse(item) || ~isreal(item) || ... isempty(item) ||jsonopt('ArrayToStruct',0,varargin{:})) if(isempty(name)) txt=sprintf('%s{%s%s"_ArrayType_": "%s",%s%s"_ArraySize_": %s,%s',... padding1,nl,padding0,class(item),nl,padding0,regexprep(mat2str(size(item)),'\s+',','),nl); else txt=sprintf('%s"%s": {%s%s"_ArrayType_": "%s",%s%s"_ArraySize_": %s,%s',... padding1,checkname(name,varargin{:}),nl,padding0,class(item),nl,padding0,regexprep(mat2str(size(item)),'\s+',','),nl); end else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1 && level>0) numtxt=regexprep(regexprep(matdata2json(item,level+1,varargin{:}),'^\[',''),']',''); else numtxt=matdata2json(item,level+1,varargin{:}); end if(isempty(name)) txt=sprintf('%s%s',padding1,numtxt); else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1) txt=sprintf('%s"%s": %s',padding1,checkname(name,varargin{:}),numtxt); else txt=sprintf('%s"%s": %s',padding1,checkname(name,varargin{:}),numtxt); end end return; end dataformat='%s%s%s%s%s'; if(issparse(item)) [ix,iy]=find(item); data=full(item(find(item))); if(~isreal(item)) data=[real(data(:)),imag(data(:))]; if(size(item,1)==1) % Kludge to have data's 'transposedness' match item's. % (Necessary for complex row vector handling below.) data=data'; end txt=sprintf(dataformat,txt,padding0,'"_ArrayIsComplex_": ','1', sep); end txt=sprintf(dataformat,txt,padding0,'"_ArrayIsSparse_": ','1', sep); if(size(item,1)==1) % Row vector, store only column indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([iy(:),data'],level+2,varargin{:}), nl); elseif(size(item,2)==1) % Column vector, store only row indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([ix,data],level+2,varargin{:}), nl); else % General case, store row and column indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([ix,iy,data],level+2,varargin{:}), nl); end else if(isreal(item)) txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json(item(:)',level+2,varargin{:}), nl); else txt=sprintf(dataformat,txt,padding0,'"_ArrayIsComplex_": ','1', sep); txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([real(item(:)) imag(item(:))],level+2,varargin{:}), nl); end end txt=sprintf('%s%s%s',txt,padding1,'}'); %%------------------------------------------------------------------------- function txt=matdata2json(mat,level,varargin) ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); tab=ws.tab; nl=ws.newline; if(size(mat,1)==1) pre=''; post=''; level=level-1; else pre=sprintf('[%s',nl); post=sprintf('%s%s]',nl,repmat(tab,1,level-1)); end if(isempty(mat)) txt='null'; return; end floatformat=jsonopt('FloatFormat','%.10g',varargin{:}); %if(numel(mat)>1) formatstr=['[' repmat([floatformat ','],1,size(mat,2)-1) [floatformat sprintf('],%s',nl)]]; %else % formatstr=[repmat([floatformat ','],1,size(mat,2)-1) [floatformat sprintf(',\n')]]; %end if(nargin>=2 && size(mat,1)>1 && jsonopt('ArrayIndent',1,varargin{:})==1) formatstr=[repmat(tab,1,level) formatstr]; end txt=sprintf(formatstr,mat'); txt(end-length(nl):end)=[]; if(islogical(mat) && jsonopt('ParseLogical',0,varargin{:})==1) txt=regexprep(txt,'1','true'); txt=regexprep(txt,'0','false'); end %txt=regexprep(mat2str(mat),'\s+',','); %txt=regexprep(txt,';',sprintf('],\n[')); % if(nargin>=2 && size(mat,1)>1) % txt=regexprep(txt,'\[',[repmat(sprintf('\t'),1,level) '[']); % end txt=[pre txt post]; if(any(isinf(mat(:)))) txt=regexprep(txt,'([-+]*)Inf',jsonopt('Inf','"$1_Inf_"',varargin{:})); end if(any(isnan(mat(:)))) txt=regexprep(txt,'NaN',jsonopt('NaN','"_NaN_"',varargin{:})); end %%------------------------------------------------------------------------- function newname=checkname(name,varargin) isunpack=jsonopt('UnpackHex',1,varargin{:}); newname=name; if(isempty(regexp(name,'0x([0-9a-fA-F]+)_','once'))) return end if(isunpack) isoct=jsonopt('IsOctave',0,varargin{:}); if(~isoct) newname=regexprep(name,'(^x|_){1}0x([0-9a-fA-F]+)_','${native2unicode(hex2dec($2))}'); else pos=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','start'); pend=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','end'); if(isempty(pos)) return; end str0=name; pos0=[0 pend(:)' length(name)]; newname=''; for i=1:length(pos) newname=[newname str0(pos0(i)+1:pos(i)-1) char(hex2dec(str0(pos(i)+3:pend(i)-1)))]; end if(pos(end)~=length(name)) newname=[newname str0(pos0(end-1)+1:pos0(end))]; end end end %%------------------------------------------------------------------------- function newstr=escapejsonstring(str) newstr=str; isoct=exist('OCTAVE_VERSION','builtin'); if(isoct) vv=sscanf(OCTAVE_VERSION,'%f'); if(vv(1)>=3.8) isoct=0; end end if(isoct) escapechars={'\a','\f','\n','\r','\t','\v'}; for i=1:length(escapechars); newstr=regexprep(newstr,escapechars{i},escapechars{i}); end else escapechars={'\a','\b','\f','\n','\r','\t','\v'}; for i=1:length(escapechars); newstr=regexprep(newstr,escapechars{i},regexprep(escapechars{i},'\\','\\\\')); end end ================================================ FILE: machine-learning-ex7/ex7/lib/jsonlab/saveubjson.m ================================================ function json=saveubjson(rootname,obj,varargin) % % json=saveubjson(rootname,obj,filename) % or % json=saveubjson(rootname,obj,opt) % json=saveubjson(rootname,obj,'param1',value1,'param2',value2,...) % % convert a MATLAB object (cell, struct or array) into a Universal % Binary JSON (UBJSON) binary string % % author: Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2013/08/17 % % $Id: saveubjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % rootname: the name of the root-object, when set to '', the root name % is ignored, however, when opt.ForceRootName is set to 1 (see below), % the MATLAB variable name will be used as the root name. % obj: a MATLAB object (array, cell, cell array, struct, struct array) % filename: a string for the file name to save the output UBJSON data % opt: a struct for additional options, ignore to use default values. % opt can have the following fields (first in [.|.] is the default) % % opt.FileName [''|string]: a file name to save the output JSON data % opt.ArrayToStruct[0|1]: when set to 0, saveubjson outputs 1D/2D % array in JSON array format; if sets to 1, an % array will be shown as a struct with fields % "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for % sparse arrays, the non-zero elements will be % saved to _ArrayData_ field in triplet-format i.e. % (ix,iy,val) and "_ArrayIsSparse_" will be added % with a value of 1; for a complex array, the % _ArrayData_ array will include two columns % (4 for sparse) to record the real and imaginary % parts, and also "_ArrayIsComplex_":1 is added. % opt.ParseLogical [1|0]: if this is set to 1, logical array elem % will use true/false rather than 1/0. % opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single % numerical element will be shown without a square % bracket, unless it is the root object; if 0, square % brackets are forced for any numerical arrays. % opt.ForceRootName [0|1]: when set to 1 and rootname is empty, saveubjson % will use the name of the passed obj variable as the % root object name; if obj is an expression and % does not have a name, 'root' will be used; if this % is set to 0 and rootname is empty, the root level % will be merged down to the lower level. % opt.JSONP [''|string]: to generate a JSONP output (JSON with padding), % for example, if opt.JSON='foo', the JSON data is % wrapped inside a function call as 'foo(...);' % opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson % back to the string form % % opt can be replaced by a list of ('param',value) pairs. The param % string is equivallent to a field in opt and is case sensitive. % output: % json: a binary string in the UBJSON format (see http://ubjson.org) % % examples: % jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... % 'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],... % 'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;... % 2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],... % 'MeshCreator','FangQ','MeshTitle','T6 Cube',... % 'SpecialData',[nan, inf, -inf]); % saveubjson('jsonmesh',jsonmesh) % saveubjson('jsonmesh',jsonmesh,'meshdata.ubj') % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(nargin==1) varname=inputname(1); obj=rootname; if(isempty(varname)) varname='root'; end rootname=varname; else varname=inputname(2); end if(length(varargin)==1 && ischar(varargin{1})) opt=struct('FileName',varargin{1}); else opt=varargin2struct(varargin{:}); end opt.IsOctave=exist('OCTAVE_VERSION','builtin'); rootisarray=0; rootlevel=1; forceroot=jsonopt('ForceRootName',0,opt); if((isnumeric(obj) || islogical(obj) || ischar(obj) || isstruct(obj) || iscell(obj)) && isempty(rootname) && forceroot==0) rootisarray=1; rootlevel=0; else if(isempty(rootname)) rootname=varname; end end if((isstruct(obj) || iscell(obj))&& isempty(rootname) && forceroot) rootname='root'; end json=obj2ubjson(rootname,obj,rootlevel,opt); if(~rootisarray) json=['{' json '}']; end jsonp=jsonopt('JSONP','',opt); if(~isempty(jsonp)) json=[jsonp '(' json ')']; end % save to a file if FileName is set, suggested by Patrick Rapin if(~isempty(jsonopt('FileName','',opt))) fid = fopen(opt.FileName, 'wb'); fwrite(fid,json); fclose(fid); end %%------------------------------------------------------------------------- function txt=obj2ubjson(name,item,level,varargin) if(iscell(item)) txt=cell2ubjson(name,item,level,varargin{:}); elseif(isstruct(item)) txt=struct2ubjson(name,item,level,varargin{:}); elseif(ischar(item)) txt=str2ubjson(name,item,level,varargin{:}); else txt=mat2ubjson(name,item,level,varargin{:}); end %%------------------------------------------------------------------------- function txt=cell2ubjson(name,item,level,varargin) txt=''; if(~iscell(item)) error('input is not a cell'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); % let's handle 1D cell first if(len>1) if(~isempty(name)) txt=[S_(checkname(name,varargin{:})) '[']; name=''; else txt='['; end elseif(len==0) if(~isempty(name)) txt=[S_(checkname(name,varargin{:})) 'Z']; name=''; else txt='Z'; end end for j=1:dim(2) if(dim(1)>1) txt=[txt '[']; end for i=1:dim(1) txt=[txt obj2ubjson(name,item{i,j},level+(len>1),varargin{:})]; end if(dim(1)>1) txt=[txt ']']; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=struct2ubjson(name,item,level,varargin) txt=''; if(~isstruct(item)) error('input is not a struct'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); if(~isempty(name)) if(len>1) txt=[S_(checkname(name,varargin{:})) '[']; end else if(len>1) txt='['; end end for j=1:dim(2) if(dim(1)>1) txt=[txt '[']; end for i=1:dim(1) names = fieldnames(item(i,j)); if(~isempty(name) && len==1) txt=[txt S_(checkname(name,varargin{:})) '{']; else txt=[txt '{']; end if(~isempty(names)) for e=1:length(names) txt=[txt obj2ubjson(names{e},getfield(item(i,j),... names{e}),level+(dim(1)>1)+1+(len>1),varargin{:})]; end end txt=[txt '}']; end if(dim(1)>1) txt=[txt ']']; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=str2ubjson(name,item,level,varargin) txt=''; if(~ischar(item)) error('input is not a string'); end item=reshape(item, max(size(item),[1 0])); len=size(item,1); if(~isempty(name)) if(len>1) txt=[S_(checkname(name,varargin{:})) '[']; end else if(len>1) txt='['; end end isoct=jsonopt('IsOctave',0,varargin{:}); for e=1:len val=item(e,:); if(len==1) obj=['' S_(checkname(name,varargin{:})) '' '',S_(val),'']; if(isempty(name)) obj=['',S_(val),'']; end txt=[txt,'',obj]; else txt=[txt,'',['',S_(val),'']]; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=mat2ubjson(name,item,level,varargin) if(~isnumeric(item) && ~islogical(item)) error('input is not an array'); end if(length(size(item))>2 || issparse(item) || ~isreal(item) || ... isempty(item) || jsonopt('ArrayToStruct',0,varargin{:})) cid=I_(uint32(max(size(item)))); if(isempty(name)) txt=['{' S_('_ArrayType_'),S_(class(item)),S_('_ArraySize_'),I_a(size(item),cid(1)) ]; else if(isempty(item)) txt=[S_(checkname(name,varargin{:})),'Z']; return; else txt=[S_(checkname(name,varargin{:})),'{',S_('_ArrayType_'),S_(class(item)),S_('_ArraySize_'),I_a(size(item),cid(1))]; end end else if(isempty(name)) txt=matdata2ubjson(item,level+1,varargin{:}); else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1) numtxt=regexprep(regexprep(matdata2ubjson(item,level+1,varargin{:}),'^\[',''),']',''); txt=[S_(checkname(name,varargin{:})) numtxt]; else txt=[S_(checkname(name,varargin{:})),matdata2ubjson(item,level+1,varargin{:})]; end end return; end if(issparse(item)) [ix,iy]=find(item); data=full(item(find(item))); if(~isreal(item)) data=[real(data(:)),imag(data(:))]; if(size(item,1)==1) % Kludge to have data's 'transposedness' match item's. % (Necessary for complex row vector handling below.) data=data'; end txt=[txt,S_('_ArrayIsComplex_'),'T']; end txt=[txt,S_('_ArrayIsSparse_'),'T']; if(size(item,1)==1) % Row vector, store only column indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([iy(:),data'],level+2,varargin{:})]; elseif(size(item,2)==1) % Column vector, store only row indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([ix,data],level+2,varargin{:})]; else % General case, store row and column indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([ix,iy,data],level+2,varargin{:})]; end else if(isreal(item)) txt=[txt,S_('_ArrayData_'),... matdata2ubjson(item(:)',level+2,varargin{:})]; else txt=[txt,S_('_ArrayIsComplex_'),'T']; txt=[txt,S_('_ArrayData_'),... matdata2ubjson([real(item(:)) imag(item(:))],level+2,varargin{:})]; end end txt=[txt,'}']; %%------------------------------------------------------------------------- function txt=matdata2ubjson(mat,level,varargin) if(isempty(mat)) txt='Z'; return; end if(size(mat,1)==1) level=level-1; end type=''; hasnegtive=(mat<0); if(isa(mat,'integer') || isinteger(mat) || (isfloat(mat) && all(mod(mat(:),1) == 0))) if(isempty(hasnegtive)) if(max(mat(:))<=2^8) type='U'; end end if(isempty(type)) % todo - need to consider negative ones separately id= histc(abs(max(mat(:))),[0 2^7 2^15 2^31 2^63]); if(isempty(find(id))) error('high-precision data is not yet supported'); end key='iIlL'; type=key(find(id)); end txt=[I_a(mat(:),type,size(mat))]; elseif(islogical(mat)) logicalval='FT'; if(numel(mat)==1) txt=logicalval(mat+1); else txt=['[$U#' I_a(size(mat),'l') typecast(swapbytes(uint8(mat(:)')),'uint8')]; end else if(numel(mat)==1) txt=['[' D_(mat) ']']; else txt=D_a(mat(:),'D',size(mat)); end end %txt=regexprep(mat2str(mat),'\s+',','); %txt=regexprep(txt,';',sprintf('],[')); % if(nargin>=2 && size(mat,1)>1) % txt=regexprep(txt,'\[',[repmat(sprintf('\t'),1,level) '[']); % end if(any(isinf(mat(:)))) txt=regexprep(txt,'([-+]*)Inf',jsonopt('Inf','"$1_Inf_"',varargin{:})); end if(any(isnan(mat(:)))) txt=regexprep(txt,'NaN',jsonopt('NaN','"_NaN_"',varargin{:})); end %%------------------------------------------------------------------------- function newname=checkname(name,varargin) isunpack=jsonopt('UnpackHex',1,varargin{:}); newname=name; if(isempty(regexp(name,'0x([0-9a-fA-F]+)_','once'))) return end if(isunpack) isoct=jsonopt('IsOctave',0,varargin{:}); if(~isoct) newname=regexprep(name,'(^x|_){1}0x([0-9a-fA-F]+)_','${native2unicode(hex2dec($2))}'); else pos=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','start'); pend=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','end'); if(isempty(pos)) return; end str0=name; pos0=[0 pend(:)' length(name)]; newname=''; for i=1:length(pos) newname=[newname str0(pos0(i)+1:pos(i)-1) char(hex2dec(str0(pos(i)+3:pend(i)-1)))]; end if(pos(end)~=length(name)) newname=[newname str0(pos0(end-1)+1:pos0(end))]; end end end %%------------------------------------------------------------------------- function val=S_(str) if(length(str)==1) val=['C' str]; else val=['S' I_(int32(length(str))) str]; end %%------------------------------------------------------------------------- function val=I_(num) if(~isinteger(num)) error('input is not an integer'); end if(num>=0 && num<255) val=['U' data2byte(swapbytes(cast(num,'uint8')),'uint8')]; return; end key='iIlL'; cid={'int8','int16','int32','int64'}; for i=1:4 if((num>0 && num<2^(i*8-1)) || (num<0 && num>=-2^(i*8-1))) val=[key(i) data2byte(swapbytes(cast(num,cid{i})),'uint8')]; return; end end error('unsupported integer'); %%------------------------------------------------------------------------- function val=D_(num) if(~isfloat(num)) error('input is not a float'); end if(isa(num,'single')) val=['d' data2byte(num,'uint8')]; else val=['D' data2byte(num,'uint8')]; end %%------------------------------------------------------------------------- function data=I_a(num,type,dim,format) id=find(ismember('iUIlL',type)); if(id==0) error('unsupported integer array'); end % based on UBJSON specs, all integer types are stored in big endian format if(id==1) data=data2byte(swapbytes(int8(num)),'uint8'); blen=1; elseif(id==2) data=data2byte(swapbytes(uint8(num)),'uint8'); blen=1; elseif(id==3) data=data2byte(swapbytes(int16(num)),'uint8'); blen=2; elseif(id==4) data=data2byte(swapbytes(int32(num)),'uint8'); blen=4; elseif(id==5) data=data2byte(swapbytes(int64(num)),'uint8'); blen=8; end if(nargin>=3 && length(dim)>=2 && prod(dim)~=dim(2)) format='opt'; end if((nargin<4 || strcmp(format,'opt')) && numel(num)>1) if(nargin>=3 && (length(dim)==1 || (length(dim)>=2 && prod(dim)~=dim(2)))) cid=I_(uint32(max(dim))); data=['$' type '#' I_a(dim,cid(1)) data(:)']; else data=['$' type '#' I_(int32(numel(data)/blen)) data(:)']; end data=['[' data(:)']; else data=reshape(data,blen,numel(data)/blen); data(2:blen+1,:)=data; data(1,:)=type; data=data(:)'; data=['[' data(:)' ']']; end %%------------------------------------------------------------------------- function data=D_a(num,type,dim,format) id=find(ismember('dD',type)); if(id==0) error('unsupported float array'); end if(id==1) data=data2byte(single(num),'uint8'); elseif(id==2) data=data2byte(double(num),'uint8'); end if(nargin>=3 && length(dim)>=2 && prod(dim)~=dim(2)) format='opt'; end if((nargin<4 || strcmp(format,'opt')) && numel(num)>1) if(nargin>=3 && (length(dim)==1 || (length(dim)>=2 && prod(dim)~=dim(2)))) cid=I_(uint32(max(dim))); data=['$' type '#' I_a(dim,cid(1)) data(:)']; else data=['$' type '#' I_(int32(numel(data)/(id*4))) data(:)']; end data=['[' data]; else data=reshape(data,(id*4),length(data)/(id*4)); data(2:(id*4+1),:)=data; data(1,:)=type; data=data(:)'; data=['[' data(:)' ']']; end %%------------------------------------------------------------------------- function bytes=data2byte(varargin) bytes=typecast(varargin{:}); bytes=bytes(:)'; ================================================ FILE: machine-learning-ex7/ex7/lib/jsonlab/varargin2struct.m ================================================ function opt=varargin2struct(varargin) % % opt=varargin2struct('param1',value1,'param2',value2,...) % or % opt=varargin2struct(...,optstruct,...) % % convert a series of input parameters into a structure % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % date: 2012/12/22 % % input: % 'param', value: the input parameters should be pairs of a string and a value % optstruct: if a parameter is a struct, the fields will be merged to the output struct % % output: % opt: a struct where opt.param1=value1, opt.param2=value2 ... % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % len=length(varargin); opt=struct; if(len==0) return; end i=1; while(i<=len) if(isstruct(varargin{i})) opt=mergestruct(opt,varargin{i}); elseif(ischar(varargin{i}) && i 0.5) = 0; R = zeros(size(Y)); R(Y ~= 0) = 1; %% Run Gradient Checking X = randn(size(X_t)); Theta = randn(size(Theta_t)); num_users = size(Y, 2); num_movies = size(Y, 1); num_features = size(Theta_t, 2); numgrad = computeNumericalGradient( ... @(t) cofiCostFunc(t, Y, R, num_users, num_movies, ... num_features, lambda), [X(:); Theta(:)]); [cost, grad] = cofiCostFunc([X(:); Theta(:)], Y, R, num_users, ... num_movies, num_features, lambda); disp([numgrad grad]); fprintf(['The above two columns you get should be very similar.\n' ... '(Left-Your Numerical Gradient, Right-Analytical Gradient)\n\n']); diff = norm(numgrad-grad)/norm(numgrad+grad); fprintf(['If your backpropagation implementation is correct, then \n' ... 'the relative difference will be small (less than 1e-9). \n' ... '\nRelative Difference: %g\n'], diff); end ================================================ FILE: machine-learning-ex8/ex8/cofiCostFunc.m ================================================ function [J, grad] = cofiCostFunc(params, Y, R, num_users, num_movies, ... num_features, lambda) %COFICOSTFUNC Collaborative filtering cost function % [J, grad] = COFICOSTFUNC(params, Y, R, num_users, num_movies, ... % num_features, lambda) returns the cost and gradient for the % collaborative filtering problem. % % Unfold the U and W matrices from params X = reshape(params(1:num_movies*num_features), num_movies, num_features); Theta = reshape(params(num_movies*num_features+1:end), ... num_users, num_features); % You need to return the following values correctly J = 0; X_grad = zeros(size(X)); Theta_grad = zeros(size(Theta)); % ====================== YOUR CODE HERE ====================== % Instructions: Compute the cost function and gradient for collaborative % filtering. Concretely, you should first implement the cost % function (without regularization) and make sure it is % matches our costs. After that, you should implement the % gradient and use the checkCostFunction routine to check % that the gradient is correct. Finally, you should implement % regularization. % % Notes: X - num_movies x num_features matrix of movie features % Theta - num_users x num_features matrix of user features % Y - num_movies x num_users matrix of user ratings of movies % R - num_movies x num_users matrix, where R(i, j) = 1 if the % i-th movie was rated by the j-th user % % You should set the following variables correctly: % % X_grad - num_movies x num_features matrix, containing the % partial derivatives w.r.t. to each element of X % Theta_grad - num_users x num_features matrix, containing the % partial derivatives w.r.t. to each element of Theta % term = X*Theta'.*R-Y.*R; J = (term(:)'*term(:)+lambda*Theta(:)'*Theta(:)+lambda*X(:)'*X(:))/2; X_grad = term*Theta+lambda*X; Theta_grad = term'*X+lambda*Theta; % ============================================================= grad = [X_grad(:); Theta_grad(:)]; end ================================================ FILE: machine-learning-ex8/ex8/computeNumericalGradient.m ================================================ function numgrad = computeNumericalGradient(J, theta) %COMPUTENUMERICALGRADIENT Computes the gradient using "finite differences" %and gives us a numerical estimate of the gradient. % numgrad = COMPUTENUMERICALGRADIENT(J, theta) computes the numerical % gradient of the function J around theta. Calling y = J(theta) should % return the function value at theta. % Notes: The following code implements numerical gradient checking, and % returns the numerical gradient.It sets numgrad(i) to (a numerical % approximation of) the partial derivative of J with respect to the % i-th input argument, evaluated at theta. (i.e., numgrad(i) should % be the (approximately) the partial derivative of J with respect % to theta(i).) % numgrad = zeros(size(theta)); perturb = zeros(size(theta)); e = 1e-4; for p = 1:numel(theta) % Set perturbation vector perturb(p) = e; loss1 = J(theta - perturb); loss2 = J(theta + perturb); % Compute Numerical Gradient numgrad(p) = (loss2 - loss1) / (2*e); perturb(p) = 0; end end ================================================ FILE: machine-learning-ex8/ex8/estimateGaussian.m ================================================ function [mu sigma2] = estimateGaussian(X) %ESTIMATEGAUSSIAN This function estimates the parameters of a %Gaussian distribution using the data in X % [mu sigma2] = estimateGaussian(X), % The input X is the dataset with each n-dimensional data point in one row % The output is an n-dimensional vector mu, the mean of the data set % and the variances sigma^2, an n x 1 vector % % Useful variables [m, n] = size(X); % You should return these values correctly mu = zeros(n, 1); sigma2 = zeros(n, 1); % ====================== YOUR CODE HERE ====================== % Instructions: Compute the mean of the data and the variances % In particular, mu(i) should contain the mean of % the data for the i-th feature and sigma2(i) % should contain variance of the i-th feature. % mu = mean(X,1)'; %ÿеƽֵתΪ sigma2 = ((m-1)*var(X)/m)'; %var(X)ÿеķ,ҪעmĹϵ % ============================================================= end ================================================ FILE: machine-learning-ex8/ex8/ex8.m ================================================ %% Machine Learning Online Class % Exercise 8 | Anomaly Detection and Collaborative Filtering % % Instructions % ------------ % % This file contains code that helps you get started on the % exercise. You will need to complete the following functions: % % estimateGaussian.m % selectThreshold.m % cofiCostFunc.m % % For this exercise, you will not need to change any code in this file, % or any other files other than those mentioned above. % %% Initialization clear ; close all; clc %% ================== Part 1: Load Example Dataset =================== % We start this exercise by using a small dataset that is easy to % visualize. % % Our example case consists of 2 network server statistics across % several machines: the latency and throughput of each machine. % This exercise will help us find possibly faulty (or very fast) machines. % fprintf('Visualizing example dataset for outlier detection.\n\n'); % The following command loads the dataset. You should now have the % variables X, Xval, yval in your environment load('ex8data1.mat'); % Visualize the example dataset plot(X(:, 1), X(:, 2), 'bx'); axis([0 30 0 30]); xlabel('Latency (ms)'); ylabel('Throughput (mb/s)'); fprintf('Program paused. Press enter to continue.\n'); pause %% ================== Part 2: Estimate the dataset statistics =================== % For this exercise, we assume a Gaussian distribution for the dataset. % % We first estimate the parameters of our assumed Gaussian distribution, % then compute the probabilities for each of the points and then visualize % both the overall distribution and where each of the points falls in % terms of that distribution. % fprintf('Visualizing Gaussian fit.\n\n'); % Estimate my and sigma2 [mu sigma2] = estimateGaussian(X); % Returns the density of the multivariate normal at each data point (row) % of X p = multivariateGaussian(X, mu, sigma2); % Visualize the fit visualizeFit(X, mu, sigma2); xlabel('Latency (ms)'); ylabel('Throughput (mb/s)'); fprintf('Program paused. Press enter to continue.\n'); pause; %% ================== Part 3: Find Outliers =================== % Now you will find a good epsilon threshold using a cross-validation set % probabilities given the estimated Gaussian distribution % pval = multivariateGaussian(Xval, mu, sigma2); [epsilon F1] = selectThreshold(yval, pval); fprintf('Best epsilon found using cross-validation: %e\n', epsilon); fprintf('Best F1 on Cross Validation Set: %f\n', F1); fprintf(' (you should see a value epsilon of about 8.99e-05)\n\n'); % Find the outliers in the training set and plot the outliers = find(p < epsilon); % Draw a red circle around those outliers hold on plot(X(outliers, 1), X(outliers, 2), 'ro', 'LineWidth', 2, 'MarkerSize', 10); hold off fprintf('Program paused. Press enter to continue.\n'); pause; %% ================== Part 4: Multidimensional Outliers =================== % We will now use the code from the previous part and apply it to a % harder problem in which more features describe each datapoint and only % some features indicate whether a point is an outlier. % % Loads the second dataset. You should now have the % variables X, Xval, yval in your environment load('ex8data2.mat'); % Apply the same steps to the larger dataset [mu sigma2] = estimateGaussian(X); % Training set p = multivariateGaussian(X, mu, sigma2); % Cross-validation set pval = multivariateGaussian(Xval, mu, sigma2); % Find the best threshold [epsilon F1] = selectThreshold(yval, pval); fprintf('Best epsilon found using cross-validation: %e\n', epsilon); fprintf('Best F1 on Cross Validation Set: %f\n', F1); fprintf('# Outliers found: %d\n', sum(p < epsilon)); fprintf(' (you should see a value epsilon of about 1.38e-18)\n\n'); pause ================================================ FILE: machine-learning-ex8/ex8/ex8_cofi.m ================================================ %% Machine Learning Online Class % Exercise 8 | Anomaly Detection and Collaborative Filtering % % Instructions % ------------ % % This file contains code that helps you get started on the % exercise. You will need to complete the following functions: % % estimateGaussian.m % selectThreshold.m % cofiCostFunc.m % % For this exercise, you will not need to change any code in this file, % or any other files other than those mentioned above. % %% =============== Part 1: Loading movie ratings dataset ================ % You will start by loading the movie ratings dataset to understand the % structure of the data. % fprintf('Loading movie ratings dataset.\n\n'); % Load data load ('ex8_movies.mat'); % Y is a 1682x943 matrix, containing ratings (1-5) of 1682 movies on % 943 users % % R is a 1682x943 matrix, where R(i,j) = 1 if and only if user j gave a % rating to movie i % From the matrix, we can compute statistics like average rating. fprintf('Average rating for movie 1 (Toy Story): %f / 5\n\n', ... mean(Y(1, R(1, :)))); % We can "visualize" the ratings matrix by plotting it with imagesc imagesc(Y); ylabel('Movies'); xlabel('Users'); fprintf('\nProgram paused. Press enter to continue.\n'); pause; %% ============ Part 2: Collaborative Filtering Cost Function =========== % You will now implement the cost function for collaborative filtering. % To help you debug your cost function, we have included set of weights % that we trained on that. Specifically, you should complete the code in % cofiCostFunc.m to return J. % Load pre-trained weights (X, Theta, num_users, num_movies, num_features) load ('ex8_movieParams.mat'); % Reduce the data set size so that this runs faster num_users = 4; num_movies = 5; num_features = 3; X = X(1:num_movies, 1:num_features); Theta = Theta(1:num_users, 1:num_features); Y = Y(1:num_movies, 1:num_users); R = R(1:num_movies, 1:num_users); % Evaluate cost function J = cofiCostFunc([X(:) ; Theta(:)], Y, R, num_users, num_movies, ... num_features, 0); fprintf(['Cost at loaded parameters: %f '... '\n(this value should be about 22.22)\n'], J); fprintf('\nProgram paused. Press enter to continue.\n'); pause; %% ============== Part 3: Collaborative Filtering Gradient ============== % Once your cost function matches up with ours, you should now implement % the collaborative filtering gradient function. Specifically, you should % complete the code in cofiCostFunc.m to return the grad argument. % fprintf('\nChecking Gradients (without regularization) ... \n'); % Check gradients by running checkNNGradients checkCostFunction; fprintf('\nProgram paused. Press enter to continue.\n'); pause; %% ========= Part 4: Collaborative Filtering Cost Regularization ======== % Now, you should implement regularization for the cost function for % collaborative filtering. You can implement it by adding the cost of % regularization to the original cost computation. % % Evaluate cost function J = cofiCostFunc([X(:) ; Theta(:)], Y, R, num_users, num_movies, ... num_features, 1.5); fprintf(['Cost at loaded parameters (lambda = 1.5): %f '... '\n(this value should be about 31.34)\n'], J); fprintf('\nProgram paused. Press enter to continue.\n'); pause; %% ======= Part 5: Collaborative Filtering Gradient Regularization ====== % Once your cost matches up with ours, you should proceed to implement % regularization for the gradient. % % fprintf('\nChecking Gradients (with regularization) ... \n'); % Check gradients by running checkNNGradients checkCostFunction(1.5); fprintf('\nProgram paused. Press enter to continue.\n'); pause; %% ============== Part 6: Entering ratings for a new user =============== % Before we will train the collaborative filtering model, we will first % add ratings that correspond to a new user that we just observed. This % part of the code will also allow you to put in your own ratings for the % movies in our dataset! % movieList = loadMovieList(); % Initialize my ratings my_ratings = zeros(1682, 1); % Check the file movie_idx.txt for id of each movie in our dataset % For example, Toy Story (1995) has ID 1, so to rate it "4", you can set my_ratings(1) = 4; % Or suppose did not enjoy Silence of the Lambs (1991), you can set my_ratings(98) = 2; % We have selected a few movies we liked / did not like and the ratings we % gave are as follows: my_ratings(7) = 3; my_ratings(12)= 5; my_ratings(54) = 4; my_ratings(64)= 5; my_ratings(66)= 3; my_ratings(69) = 5; my_ratings(183) = 4; my_ratings(226) = 5; my_ratings(355)= 5; fprintf('\n\nNew user ratings:\n'); for i = 1:length(my_ratings) if my_ratings(i) > 0 fprintf('Rated %d for %s\n', my_ratings(i), ... movieList{i}); end end fprintf('\nProgram paused. Press enter to continue.\n'); pause; %% ================== Part 7: Learning Movie Ratings ==================== % Now, you will train the collaborative filtering model on a movie rating % dataset of 1682 movies and 943 users % fprintf('\nTraining collaborative filtering...\n'); % Load data load('ex8_movies.mat'); % Y is a 1682x943 matrix, containing ratings (1-5) of 1682 movies by % 943 users % % R is a 1682x943 matrix, where R(i,j) = 1 if and only if user j gave a % rating to movie i % Add our own ratings to the data matrix Y = [my_ratings Y]; R = [(my_ratings ~= 0) R]; % Normalize Ratings [Ynorm, Ymean] = normalizeRatings(Y, R); % Useful Values num_users = size(Y, 2); num_movies = size(Y, 1); num_features = 10; % Set Initial Parameters (Theta, X) X = randn(num_movies, num_features); Theta = randn(num_users, num_features); initial_parameters = [X(:); Theta(:)]; % Set options for fmincg options = optimset('GradObj', 'on', 'MaxIter', 100); % Set Regularization lambda = 10; theta = fmincg (@(t)(cofiCostFunc(t, Y, R, num_users, num_movies, ... num_features, lambda)), ... initial_parameters, options); % Unfold the returned theta back into U and W X = reshape(theta(1:num_movies*num_features), num_movies, num_features); Theta = reshape(theta(num_movies*num_features+1:end), ... num_users, num_features); fprintf('Recommender system learning completed.\n'); fprintf('\nProgram paused. Press enter to continue.\n'); pause; %% ================== Part 8: Recommendation for you ==================== % After training the model, you can now make recommendations by computing % the predictions matrix. % p = X * Theta'; my_predictions = p(:,1) + Ymean; movieList = loadMovieList(); [r, ix] = sort(my_predictions, 'descend'); fprintf('\nTop recommendations for you:\n'); for i=1:10 j = ix(i); fprintf('Predicting rating %.1f for movie %s\n', my_predictions(j), ... movieList{j}); end fprintf('\n\nOriginal ratings provided:\n'); for i = 1:length(my_ratings) if my_ratings(i) > 0 fprintf('Rated %d for %s\n', my_ratings(i), ... movieList{i}); end end ================================================ FILE: machine-learning-ex8/ex8/fmincg.m ================================================ function [X, fX, i] = fmincg(f, X, options, P1, P2, P3, P4, P5) % Minimize a continuous differentialble multivariate function. Starting point % is given by "X" (D by 1), and the function named in the string "f", must % return a function value and a vector of partial derivatives. The Polack- % Ribiere flavour of conjugate gradients is used to compute search directions, % and a line search using quadratic and cubic polynomial approximations and the % Wolfe-Powell stopping criteria is used together with the slope ratio method % for guessing initial step sizes. Additionally a bunch of checks are made to % make sure that exploration is taking place and that extrapolation will not % be unboundedly large. The "length" gives the length of the run: if it is % positive, it gives the maximum number of line searches, if negative its % absolute gives the maximum allowed number of function evaluations. You can % (optionally) give "length" a second component, which will indicate the % reduction in function value to be expected in the first line-search (defaults % to 1.0). The function returns when either its length is up, or if no further % progress can be made (ie, we are at a minimum, or so close that due to % numerical problems, we cannot get any closer). If the function terminates % within a few iterations, it could be an indication that the function value % and derivatives are not consistent (ie, there may be a bug in the % implementation of your "f" function). The function returns the found % solution "X", a vector of function values "fX" indicating the progress made % and "i" the number of iterations (line searches or function evaluations, % depending on the sign of "length") used. % % Usage: [X, fX, i] = fmincg(f, X, options, P1, P2, P3, P4, P5) % % See also: checkgrad % % Copyright (C) 2001 and 2002 by Carl Edward Rasmussen. Date 2002-02-13 % % % (C) Copyright 1999, 2000 & 2001, Carl Edward Rasmussen % % Permission is granted for anyone to copy, use, or modify these % programs and accompanying documents for purposes of research or % education, provided this copyright notice is retained, and note is % made of any changes that have been made. % % These programs and documents are distributed without any warranty, % express or implied. As the programs were written for research % purposes only, they have not been tested to the degree that would be % advisable in any important application. All use of these programs is % entirely at the user's own risk. % % [ml-class] Changes Made: % 1) Function name and argument specifications % 2) Output display % % Read options if exist('options', 'var') && ~isempty(options) && isfield(options, 'MaxIter') length = options.MaxIter; else length = 100; end RHO = 0.01; % a bunch of constants for line searches SIG = 0.5; % RHO and SIG are the constants in the Wolfe-Powell conditions INT = 0.1; % don't reevaluate within 0.1 of the limit of the current bracket EXT = 3.0; % extrapolate maximum 3 times the current bracket MAX = 20; % max 20 function evaluations per line search RATIO = 100; % maximum allowed slope ratio argstr = ['feval(f, X']; % compose string used to call function for i = 1:(nargin - 3) argstr = [argstr, ',P', int2str(i)]; end argstr = [argstr, ')']; if max(size(length)) == 2, red=length(2); length=length(1); else red=1; end S=['Iteration ']; i = 0; % zero the run length counter ls_failed = 0; % no previous line search has failed fX = []; [f1 df1] = eval(argstr); % get function value and gradient i = i + (length<0); % count epochs?! s = -df1; % search direction is steepest d1 = -s'*s; % this is the slope z1 = red/(1-d1); % initial step is red/(|s|+1) while i < abs(length) % while not finished i = i + (length>0); % count iterations?! X0 = X; f0 = f1; df0 = df1; % make a copy of current values X = X + z1*s; % begin line search [f2 df2] = eval(argstr); i = i + (length<0); % count epochs?! d2 = df2'*s; f3 = f1; d3 = d1; z3 = -z1; % initialize point 3 equal to point 1 if length>0, M = MAX; else M = min(MAX, -length-i); end success = 0; limit = -1; % initialize quanteties while 1 while ((f2 > f1+z1*RHO*d1) || (d2 > -SIG*d1)) && (M > 0) limit = z1; % tighten the bracket if f2 > f1 z2 = z3 - (0.5*d3*z3*z3)/(d3*z3+f2-f3); % quadratic fit else A = 6*(f2-f3)/z3+3*(d2+d3); % cubic fit B = 3*(f3-f2)-z3*(d3+2*d2); z2 = (sqrt(B*B-A*d2*z3*z3)-B)/A; % numerical error possible - ok! end if isnan(z2) || isinf(z2) z2 = z3/2; % if we had a numerical problem then bisect end z2 = max(min(z2, INT*z3),(1-INT)*z3); % don't accept too close to limits z1 = z1 + z2; % update the step X = X + z2*s; [f2 df2] = eval(argstr); M = M - 1; i = i + (length<0); % count epochs?! d2 = df2'*s; z3 = z3-z2; % z3 is now relative to the location of z2 end if f2 > f1+z1*RHO*d1 || d2 > -SIG*d1 break; % this is a failure elseif d2 > SIG*d1 success = 1; break; % success elseif M == 0 break; % failure end A = 6*(f2-f3)/z3+3*(d2+d3); % make cubic extrapolation B = 3*(f3-f2)-z3*(d3+2*d2); z2 = -d2*z3*z3/(B+sqrt(B*B-A*d2*z3*z3)); % num. error possible - ok! if ~isreal(z2) || isnan(z2) || isinf(z2) || z2 < 0 % num prob or wrong sign? if limit < -0.5 % if we have no upper limit z2 = z1 * (EXT-1); % the extrapolate the maximum amount else z2 = (limit-z1)/2; % otherwise bisect end elseif (limit > -0.5) && (z2+z1 > limit) % extraplation beyond max? z2 = (limit-z1)/2; % bisect elseif (limit < -0.5) && (z2+z1 > z1*EXT) % extrapolation beyond limit z2 = z1*(EXT-1.0); % set to extrapolation limit elseif z2 < -z3*INT z2 = -z3*INT; elseif (limit > -0.5) && (z2 < (limit-z1)*(1.0-INT)) % too close to limit? z2 = (limit-z1)*(1.0-INT); end f3 = f2; d3 = d2; z3 = -z2; % set point 3 equal to point 2 z1 = z1 + z2; X = X + z2*s; % update current estimates [f2 df2] = eval(argstr); M = M - 1; i = i + (length<0); % count epochs?! d2 = df2'*s; end % end of line search if success % if line search succeeded f1 = f2; fX = [fX' f1]'; fprintf('%s %4i | Cost: %4.6e\r', S, i, f1); s = (df2'*df2-df1'*df2)/(df1'*df1)*s - df2; % Polack-Ribiere direction tmp = df1; df1 = df2; df2 = tmp; % swap derivatives d2 = df1'*s; if d2 > 0 % new slope must be negative s = -df1; % otherwise use steepest direction d2 = -s'*s; end z1 = z1 * min(RATIO, d1/(d2-realmin)); % slope ratio but max RATIO d1 = d2; ls_failed = 0; % this line search did not fail else X = X0; f1 = f0; df1 = df0; % restore point from before failed line search if ls_failed || i > abs(length) % line search failed twice in a row break; % or we ran out of time, so we give up end tmp = df1; df1 = df2; df2 = tmp; % swap derivatives s = -df1; % try steepest d1 = -s'*s; z1 = 1/(1-d1); ls_failed = 1; % this line search failed end if exist('OCTAVE_VERSION') fflush(stdout); end end fprintf('\n'); ================================================ FILE: machine-learning-ex8/ex8/lib/jsonlab/AUTHORS.txt ================================================ The author of "jsonlab" toolbox is Qianqian Fang. Qianqian is currently an Assistant Professor at Massachusetts General Hospital, Harvard Medical School. Address: Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School Bldg 149, 13th St, Charlestown, MA 02129, USA URL: http://nmr.mgh.harvard.edu/~fangq/ Email: or The script loadjson.m was built upon previous works by - Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713 date: 2009/11/02 - François Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393 date: 2009/03/22 - Joel Feenstra: http://www.mathworks.com/matlabcentral/fileexchange/20565 date: 2008/07/03 This toolbox contains patches submitted by the following contributors: - Blake Johnson part of revision 341 - Niclas Borlin various fixes in revision 394, including - loadjson crashes for all-zero sparse matrix. - loadjson crashes for empty sparse matrix. - Non-zero size of 0-by-N and N-by-0 empty matrices is lost after savejson/loadjson. - loadjson crashes for sparse real column vector. - loadjson crashes for sparse complex column vector. - Data is corrupted by savejson for sparse real row vector. - savejson crashes for sparse complex row vector. - Yul Kang patches for svn revision 415. - savejson saves an empty cell array as [] instead of null - loadjson differentiates an empty struct from an empty array ================================================ FILE: machine-learning-ex8/ex8/lib/jsonlab/ChangeLog.txt ================================================ ============================================================================ JSONlab - a toolbox to encode/decode JSON/UBJSON files in MATLAB/Octave ---------------------------------------------------------------------------- JSONlab ChangeLog (key features marked by *): == JSONlab 1.0 (codename: Optimus - Final), FangQ == 2015/01/02 polish help info for all major functions, update examples, finalize 1.0 2014/12/19 fix a bug to strictly respect NoRowBracket in savejson == JSONlab 1.0.0-RC2 (codename: Optimus - RC2), FangQ == 2014/11/22 show progress bar in loadjson ('ShowProgress') 2014/11/17 add Compact option in savejson to output compact JSON format ('Compact') 2014/11/17 add FastArrayParser in loadjson to specify fast parser applicable levels 2014/09/18 start official github mirror: https://github.com/fangq/jsonlab == JSONlab 1.0.0-RC1 (codename: Optimus - RC1), FangQ == 2014/09/17 fix several compatibility issues when running on octave versions 3.2-3.8 2014/09/17 support 2D cell and struct arrays in both savejson and saveubjson 2014/08/04 escape special characters in a JSON string 2014/02/16 fix a bug when saving ubjson files == JSONlab 0.9.9 (codename: Optimus - beta), FangQ == 2014/01/22 use binary read and write in saveubjson and loadubjson == JSONlab 0.9.8-1 (codename: Optimus - alpha update 1), FangQ == 2013/10/07 better round-trip conservation for empty arrays and structs (patch submitted by Yul Kang) == JSONlab 0.9.8 (codename: Optimus - alpha), FangQ == 2013/08/23 *universal Binary JSON (UBJSON) support, including both saveubjson and loadubjson == JSONlab 0.9.1 (codename: Rodimus, update 1), FangQ == 2012/12/18 *handling of various empty and sparse matrices (fixes submitted by Niclas Borlin) == JSONlab 0.9.0 (codename: Rodimus), FangQ == 2012/06/17 *new format for an invalid leading char, unpacking hex code in savejson 2012/06/01 support JSONP in savejson 2012/05/25 fix the empty cell bug (reported by Cyril Davin) 2012/04/05 savejson can save to a file (suggested by Patrick Rapin) == JSONlab 0.8.1 (codename: Sentiel, Update 1), FangQ == 2012/02/28 loadjson quotation mark escape bug, see http://bit.ly/yyk1nS 2012/01/25 patch to handle root-less objects, contributed by Blake Johnson == JSONlab 0.8.0 (codename: Sentiel), FangQ == 2012/01/13 *speed up loadjson by 20 fold when parsing large data arrays in matlab 2012/01/11 remove row bracket if an array has 1 element, suggested by Mykel Kochenderfer 2011/12/22 *accept sequence of 'param',value input in savejson and loadjson 2011/11/18 fix struct array bug reported by Mykel Kochenderfer == JSONlab 0.5.1 (codename: Nexus Update 1), FangQ == 2011/10/21 fix a bug in loadjson, previous code does not use any of the acceleration 2011/10/20 loadjson supports JSON collections - concatenated JSON objects == JSONlab 0.5.0 (codename: Nexus), FangQ == 2011/10/16 package and release jsonlab 0.5.0 2011/10/15 *add json demo and regression test, support cpx numbers, fix double quote bug 2011/10/11 *speed up readjson dramatically, interpret _Array* tags, show data in root level 2011/10/10 create jsonlab project, start jsonlab website, add online documentation 2011/10/07 *speed up savejson by 25x using sprintf instead of mat2str, add options support 2011/10/06 *savejson works for structs, cells and arrays 2011/09/09 derive loadjson from JSON parser from MATLAB Central, draft savejson.m ================================================ FILE: machine-learning-ex8/ex8/lib/jsonlab/LICENSE_BSD.txt ================================================ Copyright 2011-2015 Qianqian Fang . All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. The views and conclusions contained in the software and documentation are those of the authors and should not be interpreted as representing official policies, either expressed or implied, of the copyright holders. ================================================ FILE: machine-learning-ex8/ex8/lib/jsonlab/README.txt ================================================ =============================================================================== = JSONLab = = An open-source MATLAB/Octave JSON encoder and decoder = =============================================================================== *Copyright (C) 2011-2015 Qianqian Fang *License: BSD License, see License_BSD.txt for details *Version: 1.0 (Optimus - Final) ------------------------------------------------------------------------------- Table of Content: I. Introduction II. Installation III.Using JSONLab IV. Known Issues and TODOs V. Contribution and feedback ------------------------------------------------------------------------------- I. Introduction JSON ([http://www.json.org/ JavaScript Object Notation]) is a highly portable, human-readable and "[http://en.wikipedia.org/wiki/JSON fat-free]" text format to represent complex and hierarchical data. It is as powerful as [http://en.wikipedia.org/wiki/XML XML], but less verbose. JSON format is widely used for data-exchange in applications, and is essential for the wild success of [http://en.wikipedia.org/wiki/Ajax_(programming) Ajax] and [http://en.wikipedia.org/wiki/Web_2.0 Web2.0]. UBJSON (Universal Binary JSON) is a binary JSON format, specifically optimized for compact file size and better performance while keeping the semantics as simple as the text-based JSON format. Using the UBJSON format allows to wrap complex binary data in a flexible and extensible structure, making it possible to process complex and large dataset without accuracy loss due to text conversions. We envision that both JSON and its binary version will serve as part of the mainstream data-exchange formats for scientific research in the future. It will provide the flexibility and generality achieved by other popular general-purpose file specifications, such as [http://www.hdfgroup.org/HDF5/whatishdf5.html HDF5], with significantly reduced complexity and enhanced performance. JSONLab is a free and open-source implementation of a JSON/UBJSON encoder and a decoder in the native MATLAB language. It can be used to convert a MATLAB data structure (array, struct, cell, struct array and cell array) into JSON/UBJSON formatted strings, or to decode a JSON/UBJSON file into MATLAB data structure. JSONLab supports both MATLAB and [http://www.gnu.org/software/octave/ GNU Octave] (a free MATLAB clone). ------------------------------------------------------------------------------- II. Installation The installation of JSONLab is no different than any other simple MATLAB toolbox. You only need to download/unzip the JSONLab package to a folder, and add the folder's path to MATLAB/Octave's path list by using the following command: addpath('/path/to/jsonlab'); If you want to add this path permanently, you need to type "pathtool", browse to the jsonlab root folder and add to the list, then click "Save". Then, run "rehash" in MATLAB, and type "which loadjson", if you see an output, that means JSONLab is installed for MATLAB/Octave. ------------------------------------------------------------------------------- III.Using JSONLab JSONLab provides two functions, loadjson.m -- a MATLAB->JSON decoder, and savejson.m -- a MATLAB->JSON encoder, for the text-based JSON, and two equivallent functions -- loadubjson and saveubjson for the binary JSON. The detailed help info for the four functions can be found below: === loadjson.m === 
  data=loadjson(fname,opt)
     or
  data=loadjson(fname,'param1',value1,'param2',value2,...)
 
  parse a JSON (JavaScript Object Notation) file or string
 
  authors:Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2011/09/09, including previous works from 
 
          Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713
             created on 2009/11/02
          Franois Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393
             created on  2009/03/22
          Joel Feenstra:
          http://www.mathworks.com/matlabcentral/fileexchange/20565
             created on 2008/07/03
 
  $Id: loadjson.m 452 2014-11-22 16:43:33Z fangq $
 
  input:
       fname: input file name, if fname contains "{}" or "[]", fname
              will be interpreted as a JSON string
       opt: a struct to store parsing options, opt can be replaced by 
            a list of ('param',value) pairs - the param string is equivallent
            to a field in opt. opt can have the following 
            fields (first in [.|.] is the default)
 
            opt.SimplifyCell [0|1]: if set to 1, loadjson will call cell2mat
                          for each element of the JSON data, and group 
                          arrays based on the cell2mat rules.
            opt.FastArrayParser [1|0 or integer]: if set to 1, use a
                          speed-optimized array parser when loading an 
                          array object. The fast array parser may 
                          collapse block arrays into a single large
                          array similar to rules defined in cell2mat; 0 to 
                          use a legacy parser; if set to a larger-than-1
                          value, this option will specify the minimum
                          dimension to enable the fast array parser. For
                          example, if the input is a 3D array, setting
                          FastArrayParser to 1 will return a 3D array;
                          setting to 2 will return a cell array of 2D
                          arrays; setting to 3 will return to a 2D cell
                          array of 1D vectors; setting to 4 will return a
                          3D cell array.
            opt.ShowProgress [0|1]: if set to 1, loadjson displays a progress bar.
 
  output:
       dat: a cell array, where {...} blocks are converted into cell arrays,
            and [...] are converted to arrays
 
  examples:
       dat=loadjson('{"obj":{"string":"value","array":[1,2,3]}}')
       dat=loadjson(['examples' filesep 'example1.json'])
       dat=loadjson(['examples' filesep 'example1.json'],'SimplifyCell',1)
=== savejson.m === 
  json=savejson(rootname,obj,filename)
     or
  json=savejson(rootname,obj,opt)
  json=savejson(rootname,obj,'param1',value1,'param2',value2,...)
 
  convert a MATLAB object (cell, struct or array) into a JSON (JavaScript
  Object Notation) string
 
  author: Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2011/09/09
 
  $Id: savejson.m 458 2014-12-19 22:17:17Z fangq $
 
  input:
       rootname: the name of the root-object, when set to '', the root name
         is ignored, however, when opt.ForceRootName is set to 1 (see below),
         the MATLAB variable name will be used as the root name.
       obj: a MATLAB object (array, cell, cell array, struct, struct array).
       filename: a string for the file name to save the output JSON data.
       opt: a struct for additional options, ignore to use default values.
         opt can have the following fields (first in [.|.] is the default)
 
         opt.FileName [''|string]: a file name to save the output JSON data
         opt.FloatFormat ['%.10g'|string]: format to show each numeric element
                          of a 1D/2D array;
         opt.ArrayIndent [1|0]: if 1, output explicit data array with
                          precedent indentation; if 0, no indentation
         opt.ArrayToStruct[0|1]: when set to 0, savejson outputs 1D/2D
                          array in JSON array format; if sets to 1, an
                          array will be shown as a struct with fields
                          "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for
                          sparse arrays, the non-zero elements will be
                          saved to _ArrayData_ field in triplet-format i.e.
                          (ix,iy,val) and "_ArrayIsSparse_" will be added
                          with a value of 1; for a complex array, the 
                          _ArrayData_ array will include two columns 
                          (4 for sparse) to record the real and imaginary 
                          parts, and also "_ArrayIsComplex_":1 is added. 
         opt.ParseLogical [0|1]: if this is set to 1, logical array elem
                          will use true/false rather than 1/0.
         opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single
                          numerical element will be shown without a square
                          bracket, unless it is the root object; if 0, square
                          brackets are forced for any numerical arrays.
         opt.ForceRootName [0|1]: when set to 1 and rootname is empty, savejson
                          will use the name of the passed obj variable as the 
                          root object name; if obj is an expression and 
                          does not have a name, 'root' will be used; if this 
                          is set to 0 and rootname is empty, the root level 
                          will be merged down to the lower level.
         opt.Inf ['"$1_Inf_"'|string]: a customized regular expression pattern
                          to represent +/-Inf. The matched pattern is '([-+]*)Inf'
                          and $1 represents the sign. For those who want to use
                          1e999 to represent Inf, they can set opt.Inf to '$11e999'
         opt.NaN ['"_NaN_"'|string]: a customized regular expression pattern
                          to represent NaN
         opt.JSONP [''|string]: to generate a JSONP output (JSON with padding),
                          for example, if opt.JSONP='foo', the JSON data is
                          wrapped inside a function call as 'foo(...);'
         opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson 
                          back to the string form
         opt.SaveBinary [0|1]: 1 - save the JSON file in binary mode; 0 - text mode.
         opt.Compact [0|1]: 1- out compact JSON format (remove all newlines and tabs)
 
         opt can be replaced by a list of ('param',value) pairs. The param 
         string is equivallent to a field in opt and is case sensitive.
  output:
       json: a string in the JSON format (see http://json.org)
 
  examples:
       jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... 
                'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],...
                'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;...
                           2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],...
                'MeshCreator','FangQ','MeshTitle','T6 Cube',...
                'SpecialData',[nan, inf, -inf]);
       savejson('jmesh',jsonmesh)
       savejson('',jsonmesh,'ArrayIndent',0,'FloatFormat','\t%.5g')
=== loadubjson.m === 
  data=loadubjson(fname,opt)
     or
  data=loadubjson(fname,'param1',value1,'param2',value2,...)
 
  parse a JSON (JavaScript Object Notation) file or string
 
  authors:Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2013/08/01
 
  $Id: loadubjson.m 436 2014-08-05 20:51:40Z fangq $
 
  input:
       fname: input file name, if fname contains "{}" or "[]", fname
              will be interpreted as a UBJSON string
       opt: a struct to store parsing options, opt can be replaced by 
            a list of ('param',value) pairs - the param string is equivallent
            to a field in opt. opt can have the following 
            fields (first in [.|.] is the default)
 
            opt.SimplifyCell [0|1]: if set to 1, loadubjson will call cell2mat
                          for each element of the JSON data, and group 
                          arrays based on the cell2mat rules.
            opt.IntEndian [B|L]: specify the endianness of the integer fields
                          in the UBJSON input data. B - Big-Endian format for 
                          integers (as required in the UBJSON specification); 
                          L - input integer fields are in Little-Endian order.
 
  output:
       dat: a cell array, where {...} blocks are converted into cell arrays,
            and [...] are converted to arrays
 
  examples:
       obj=struct('string','value','array',[1 2 3]);
       ubjdata=saveubjson('obj',obj);
       dat=loadubjson(ubjdata)
       dat=loadubjson(['examples' filesep 'example1.ubj'])
       dat=loadubjson(['examples' filesep 'example1.ubj'],'SimplifyCell',1)
=== saveubjson.m === 
  json=saveubjson(rootname,obj,filename)
     or
  json=saveubjson(rootname,obj,opt)
  json=saveubjson(rootname,obj,'param1',value1,'param2',value2,...)
 
  convert a MATLAB object (cell, struct or array) into a Universal 
  Binary JSON (UBJSON) binary string
 
  author: Qianqian Fang (fangq nmr.mgh.harvard.edu)
  created on 2013/08/17
 
  $Id: saveubjson.m 440 2014-09-17 19:59:45Z fangq $
 
  input:
       rootname: the name of the root-object, when set to '', the root name
         is ignored, however, when opt.ForceRootName is set to 1 (see below),
         the MATLAB variable name will be used as the root name.
       obj: a MATLAB object (array, cell, cell array, struct, struct array)
       filename: a string for the file name to save the output UBJSON data
       opt: a struct for additional options, ignore to use default values.
         opt can have the following fields (first in [.|.] is the default)
 
         opt.FileName [''|string]: a file name to save the output JSON data
         opt.ArrayToStruct[0|1]: when set to 0, saveubjson outputs 1D/2D
                          array in JSON array format; if sets to 1, an
                          array will be shown as a struct with fields
                          "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for
                          sparse arrays, the non-zero elements will be
                          saved to _ArrayData_ field in triplet-format i.e.
                          (ix,iy,val) and "_ArrayIsSparse_" will be added
                          with a value of 1; for a complex array, the 
                          _ArrayData_ array will include two columns 
                          (4 for sparse) to record the real and imaginary 
                          parts, and also "_ArrayIsComplex_":1 is added. 
         opt.ParseLogical [1|0]: if this is set to 1, logical array elem
                          will use true/false rather than 1/0.
         opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single
                          numerical element will be shown without a square
                          bracket, unless it is the root object; if 0, square
                          brackets are forced for any numerical arrays.
         opt.ForceRootName [0|1]: when set to 1 and rootname is empty, saveubjson
                          will use the name of the passed obj variable as the 
                          root object name; if obj is an expression and 
                          does not have a name, 'root' will be used; if this 
                          is set to 0 and rootname is empty, the root level 
                          will be merged down to the lower level.
         opt.JSONP [''|string]: to generate a JSONP output (JSON with padding),
                          for example, if opt.JSON='foo', the JSON data is
                          wrapped inside a function call as 'foo(...);'
         opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson 
                          back to the string form
 
         opt can be replaced by a list of ('param',value) pairs. The param 
         string is equivallent to a field in opt and is case sensitive.
  output:
       json: a binary string in the UBJSON format (see http://ubjson.org)
 
  examples:
       jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... 
                'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],...
                'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;...
                           2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],...
                'MeshCreator','FangQ','MeshTitle','T6 Cube',...
                'SpecialData',[nan, inf, -inf]);
       saveubjson('jsonmesh',jsonmesh)
       saveubjson('jsonmesh',jsonmesh,'meshdata.ubj')
=== examples === Under the "examples" folder, you can find several scripts to demonstrate the basic utilities of JSONLab. Running the "demo_jsonlab_basic.m" script, you will see the conversions from MATLAB data structure to JSON text and backward. In "jsonlab_selftest.m", we load complex JSON files downloaded from the Internet and validate the loadjson/savejson functions for regression testing purposes. Similarly, a "demo_ubjson_basic.m" script is provided to test the saveubjson and loadubjson pairs for various matlab data structures. Please run these examples and understand how JSONLab works before you use it to process your data. ------------------------------------------------------------------------------- IV. Known Issues and TODOs JSONLab has several known limitations. We are striving to make it more general and robust. Hopefully in a few future releases, the limitations become less. Here are the known issues: # 3D or higher dimensional cell/struct-arrays will be converted to 2D arrays; # When processing names containing multi-byte characters, Octave and MATLAB \ can give different field-names; you can use feature('DefaultCharacterSet','latin1') \ in MATLAB to get consistant results # savejson can not handle class and dataset. # saveubjson converts a logical array into a uint8 ([U]) array # an unofficial N-D array count syntax is implemented in saveubjson. We are \ actively communicating with the UBJSON spec maintainer to investigate the \ possibility of making it upstream # loadubjson can not parse all UBJSON Specification (Draft 9) compliant \ files, however, it can parse all UBJSON files produced by saveubjson. ------------------------------------------------------------------------------- V. Contribution and feedback JSONLab is an open-source project. This means you can not only use it and modify it as you wish, but also you can contribute your changes back to JSONLab so that everyone else can enjoy the improvement. For anyone who want to contribute, please download JSONLab source code from it's subversion repository by using the following command: svn checkout svn://svn.code.sf.net/p/iso2mesh/code/trunk/jsonlab jsonlab You can make changes to the files as needed. Once you are satisfied with your changes, and ready to share it with others, please cd the root directory of JSONLab, and type svn diff > yourname_featurename.patch You then email the .patch file to JSONLab's maintainer, Qianqian Fang, at the email address shown in the beginning of this file. Qianqian will review the changes and commit it to the subversion if they are satisfactory. We appreciate any suggestions and feedbacks from you. Please use iso2mesh's mailing list to report any questions you may have with JSONLab: http://groups.google.com/group/iso2mesh-users?hl=en&pli=1 (Subscription to the mailing list is needed in order to post messages). ================================================ FILE: machine-learning-ex8/ex8/lib/jsonlab/jsonopt.m ================================================ function val=jsonopt(key,default,varargin) % % val=jsonopt(key,default,optstruct) % % setting options based on a struct. The struct can be produced % by varargin2struct from a list of 'param','value' pairs % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % % $Id: loadjson.m 371 2012-06-20 12:43:06Z fangq $ % % input: % key: a string with which one look up a value from a struct % default: if the key does not exist, return default % optstruct: a struct where each sub-field is a key % % output: % val: if key exists, val=optstruct.key; otherwise val=default % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % val=default; if(nargin<=2) return; end opt=varargin{1}; if(isstruct(opt) && isfield(opt,key)) val=getfield(opt,key); end ================================================ FILE: machine-learning-ex8/ex8/lib/jsonlab/loadjson.m ================================================ function data = loadjson(fname,varargin) % % data=loadjson(fname,opt) % or % data=loadjson(fname,'param1',value1,'param2',value2,...) % % parse a JSON (JavaScript Object Notation) file or string % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2011/09/09, including previous works from % % Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713 % created on 2009/11/02 % François Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393 % created on 2009/03/22 % Joel Feenstra: % http://www.mathworks.com/matlabcentral/fileexchange/20565 % created on 2008/07/03 % % $Id: loadjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % fname: input file name, if fname contains "{}" or "[]", fname % will be interpreted as a JSON string % opt: a struct to store parsing options, opt can be replaced by % a list of ('param',value) pairs - the param string is equivallent % to a field in opt. opt can have the following % fields (first in [.|.] is the default) % % opt.SimplifyCell [0|1]: if set to 1, loadjson will call cell2mat % for each element of the JSON data, and group % arrays based on the cell2mat rules. % opt.FastArrayParser [1|0 or integer]: if set to 1, use a % speed-optimized array parser when loading an % array object. The fast array parser may % collapse block arrays into a single large % array similar to rules defined in cell2mat; 0 to % use a legacy parser; if set to a larger-than-1 % value, this option will specify the minimum % dimension to enable the fast array parser. For % example, if the input is a 3D array, setting % FastArrayParser to 1 will return a 3D array; % setting to 2 will return a cell array of 2D % arrays; setting to 3 will return to a 2D cell % array of 1D vectors; setting to 4 will return a % 3D cell array. % opt.ShowProgress [0|1]: if set to 1, loadjson displays a progress bar. % % output: % dat: a cell array, where {...} blocks are converted into cell arrays, % and [...] are converted to arrays % % examples: % dat=loadjson('{"obj":{"string":"value","array":[1,2,3]}}') % dat=loadjson(['examples' filesep 'example1.json']) % dat=loadjson(['examples' filesep 'example1.json'],'SimplifyCell',1) % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % global pos inStr len esc index_esc len_esc isoct arraytoken if(regexp(fname,'[\{\}\]\[]','once')) string=fname; elseif(exist(fname,'file')) fid = fopen(fname,'rb'); string = fread(fid,inf,'uint8=>char')'; fclose(fid); else error('input file does not exist'); end pos = 1; len = length(string); inStr = string; isoct=exist('OCTAVE_VERSION','builtin'); arraytoken=find(inStr=='[' | inStr==']' | inStr=='"'); jstr=regexprep(inStr,'\\\\',' '); escquote=regexp(jstr,'\\"'); arraytoken=sort([arraytoken escquote]); % String delimiters and escape chars identified to improve speed: esc = find(inStr=='"' | inStr=='\' ); % comparable to: regexp(inStr, '["\\]'); index_esc = 1; len_esc = length(esc); opt=varargin2struct(varargin{:}); if(jsonopt('ShowProgress',0,opt)==1) opt.progressbar_=waitbar(0,'loading ...'); end jsoncount=1; while pos <= len switch(next_char) case '{' data{jsoncount} = parse_object(opt); case '[' data{jsoncount} = parse_array(opt); otherwise error_pos('Outer level structure must be an object or an array'); end jsoncount=jsoncount+1; end % while jsoncount=length(data); if(jsoncount==1 && iscell(data)) data=data{1}; end if(~isempty(data)) if(isstruct(data)) % data can be a struct array data=jstruct2array(data); elseif(iscell(data)) data=jcell2array(data); end end if(isfield(opt,'progressbar_')) close(opt.progressbar_); end %% function newdata=jcell2array(data) len=length(data); newdata=data; for i=1:len if(isstruct(data{i})) newdata{i}=jstruct2array(data{i}); elseif(iscell(data{i})) newdata{i}=jcell2array(data{i}); end end %%------------------------------------------------------------------------- function newdata=jstruct2array(data) fn=fieldnames(data); newdata=data; len=length(data); for i=1:length(fn) % depth-first for j=1:len if(isstruct(getfield(data(j),fn{i}))) newdata(j)=setfield(newdata(j),fn{i},jstruct2array(getfield(data(j),fn{i}))); end end end if(~isempty(strmatch('x0x5F_ArrayType_',fn)) && ~isempty(strmatch('x0x5F_ArrayData_',fn))) newdata=cell(len,1); for j=1:len ndata=cast(data(j).x0x5F_ArrayData_,data(j).x0x5F_ArrayType_); iscpx=0; if(~isempty(strmatch('x0x5F_ArrayIsComplex_',fn))) if(data(j).x0x5F_ArrayIsComplex_) iscpx=1; end end if(~isempty(strmatch('x0x5F_ArrayIsSparse_',fn))) if(data(j).x0x5F_ArrayIsSparse_) if(~isempty(strmatch('x0x5F_ArraySize_',fn))) dim=data(j).x0x5F_ArraySize_; if(iscpx && size(ndata,2)==4-any(dim==1)) ndata(:,end-1)=complex(ndata(:,end-1),ndata(:,end)); end if isempty(ndata) % All-zeros sparse ndata=sparse(dim(1),prod(dim(2:end))); elseif dim(1)==1 % Sparse row vector ndata=sparse(1,ndata(:,1),ndata(:,2),dim(1),prod(dim(2:end))); elseif dim(2)==1 % Sparse column vector ndata=sparse(ndata(:,1),1,ndata(:,2),dim(1),prod(dim(2:end))); else % Generic sparse array. ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3),dim(1),prod(dim(2:end))); end else if(iscpx && size(ndata,2)==4) ndata(:,3)=complex(ndata(:,3),ndata(:,4)); end ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3)); end end elseif(~isempty(strmatch('x0x5F_ArraySize_',fn))) if(iscpx && size(ndata,2)==2) ndata=complex(ndata(:,1),ndata(:,2)); end ndata=reshape(ndata(:),data(j).x0x5F_ArraySize_); end newdata{j}=ndata; end if(len==1) newdata=newdata{1}; end end %%------------------------------------------------------------------------- function object = parse_object(varargin) parse_char('{'); object = []; if next_char ~= '}' while 1 str = parseStr(varargin{:}); if isempty(str) error_pos('Name of value at position %d cannot be empty'); end parse_char(':'); val = parse_value(varargin{:}); eval( sprintf( 'object.%s = val;', valid_field(str) ) ); if next_char == '}' break; end parse_char(','); end end parse_char('}'); %%------------------------------------------------------------------------- function object = parse_array(varargin) % JSON array is written in row-major order global pos inStr isoct parse_char('['); object = cell(0, 1); dim2=[]; arraydepth=jsonopt('JSONLAB_ArrayDepth_',1,varargin{:}); pbar=jsonopt('progressbar_',-1,varargin{:}); if next_char ~= ']' if(jsonopt('FastArrayParser',1,varargin{:})>=1 && arraydepth>=jsonopt('FastArrayParser',1,varargin{:})) [endpos, e1l, e1r, maxlevel]=matching_bracket(inStr,pos); arraystr=['[' inStr(pos:endpos)]; arraystr=regexprep(arraystr,'"_NaN_"','NaN'); arraystr=regexprep(arraystr,'"([-+]*)_Inf_"','$1Inf'); arraystr(arraystr==sprintf('\n'))=[]; arraystr(arraystr==sprintf('\r'))=[]; %arraystr=regexprep(arraystr,'\s*,',','); % this is slow,sometimes needed if(~isempty(e1l) && ~isempty(e1r)) % the array is in 2D or higher D astr=inStr((e1l+1):(e1r-1)); astr=regexprep(astr,'"_NaN_"','NaN'); astr=regexprep(astr,'"([-+]*)_Inf_"','$1Inf'); astr(astr==sprintf('\n'))=[]; astr(astr==sprintf('\r'))=[]; astr(astr==' ')=''; if(isempty(find(astr=='[', 1))) % array is 2D dim2=length(sscanf(astr,'%f,',[1 inf])); end else % array is 1D astr=arraystr(2:end-1); astr(astr==' ')=''; [obj, count, errmsg, nextidx]=sscanf(astr,'%f,',[1,inf]); if(nextidx>=length(astr)-1) object=obj; pos=endpos; parse_char(']'); return; end end if(~isempty(dim2)) astr=arraystr; astr(astr=='[')=''; astr(astr==']')=''; astr(astr==' ')=''; [obj, count, errmsg, nextidx]=sscanf(astr,'%f,',inf); if(nextidx>=length(astr)-1) object=reshape(obj,dim2,numel(obj)/dim2)'; pos=endpos; parse_char(']'); if(pbar>0) waitbar(pos/length(inStr),pbar,'loading ...'); end return; end end arraystr=regexprep(arraystr,'\]\s*,','];'); else arraystr='['; end try if(isoct && regexp(arraystr,'"','once')) error('Octave eval can produce empty cells for JSON-like input'); end object=eval(arraystr); pos=endpos; catch while 1 newopt=varargin2struct(varargin{:},'JSONLAB_ArrayDepth_',arraydepth+1); val = parse_value(newopt); object{end+1} = val; if next_char == ']' break; end parse_char(','); end end end if(jsonopt('SimplifyCell',0,varargin{:})==1) try oldobj=object; object=cell2mat(object')'; if(iscell(oldobj) && isstruct(object) && numel(object)>1 && jsonopt('SimplifyCellArray',1,varargin{:})==0) object=oldobj; elseif(size(object,1)>1 && ndims(object)==2) object=object'; end catch end end parse_char(']'); if(pbar>0) waitbar(pos/length(inStr),pbar,'loading ...'); end %%------------------------------------------------------------------------- function parse_char(c) global pos inStr len skip_whitespace; if pos > len || inStr(pos) ~= c error_pos(sprintf('Expected %c at position %%d', c)); else pos = pos + 1; skip_whitespace; end %%------------------------------------------------------------------------- function c = next_char global pos inStr len skip_whitespace; if pos > len c = []; else c = inStr(pos); end %%------------------------------------------------------------------------- function skip_whitespace global pos inStr len while pos <= len && isspace(inStr(pos)) pos = pos + 1; end %%------------------------------------------------------------------------- function str = parseStr(varargin) global pos inStr len esc index_esc len_esc % len, ns = length(inStr), keyboard if inStr(pos) ~= '"' error_pos('String starting with " expected at position %d'); else pos = pos + 1; end str = ''; while pos <= len while index_esc <= len_esc && esc(index_esc) < pos index_esc = index_esc + 1; end if index_esc > len_esc str = [str inStr(pos:len)]; pos = len + 1; break; else str = [str inStr(pos:esc(index_esc)-1)]; pos = esc(index_esc); end nstr = length(str); switch inStr(pos) case '"' pos = pos + 1; if(~isempty(str)) if(strcmp(str,'_Inf_')) str=Inf; elseif(strcmp(str,'-_Inf_')) str=-Inf; elseif(strcmp(str,'_NaN_')) str=NaN; end end return; case '\' if pos+1 > len error_pos('End of file reached right after escape character'); end pos = pos + 1; switch inStr(pos) case {'"' '\' '/'} str(nstr+1) = inStr(pos); pos = pos + 1; case {'b' 'f' 'n' 'r' 't'} str(nstr+1) = sprintf(['\' inStr(pos)]); pos = pos + 1; case 'u' if pos+4 > len error_pos('End of file reached in escaped unicode character'); end str(nstr+(1:6)) = inStr(pos-1:pos+4); pos = pos + 5; end otherwise % should never happen str(nstr+1) = inStr(pos), keyboard pos = pos + 1; end end error_pos('End of file while expecting end of inStr'); %%------------------------------------------------------------------------- function num = parse_number(varargin) global pos inStr len isoct currstr=inStr(pos:end); numstr=0; if(isoct~=0) numstr=regexp(currstr,'^\s*-?(?:0|[1-9]\d*)(?:\.\d+)?(?:[eE][+\-]?\d+)?','end'); [num, one] = sscanf(currstr, '%f', 1); delta=numstr+1; else [num, one, err, delta] = sscanf(currstr, '%f', 1); if ~isempty(err) error_pos('Error reading number at position %d'); end end pos = pos + delta-1; %%------------------------------------------------------------------------- function val = parse_value(varargin) global pos inStr len true = 1; false = 0; pbar=jsonopt('progressbar_',-1,varargin{:}); if(pbar>0) waitbar(pos/len,pbar,'loading ...'); end switch(inStr(pos)) case '"' val = parseStr(varargin{:}); return; case '[' val = parse_array(varargin{:}); return; case '{' val = parse_object(varargin{:}); if isstruct(val) if(~isempty(strmatch('x0x5F_ArrayType_',fieldnames(val), 'exact'))) val=jstruct2array(val); end elseif isempty(val) val = struct; end return; case {'-','0','1','2','3','4','5','6','7','8','9'} val = parse_number(varargin{:}); return; case 't' if pos+3 <= len && strcmpi(inStr(pos:pos+3), 'true') val = true; pos = pos + 4; return; end case 'f' if pos+4 <= len && strcmpi(inStr(pos:pos+4), 'false') val = false; pos = pos + 5; return; end case 'n' if pos+3 <= len && strcmpi(inStr(pos:pos+3), 'null') val = []; pos = pos + 4; return; end end error_pos('Value expected at position %d'); %%------------------------------------------------------------------------- function error_pos(msg) global pos inStr len poShow = max(min([pos-15 pos-1 pos pos+20],len),1); if poShow(3) == poShow(2) poShow(3:4) = poShow(2)+[0 -1]; % display nothing after end msg = [sprintf(msg, pos) ': ' ... inStr(poShow(1):poShow(2)) '' inStr(poShow(3):poShow(4)) ]; error( ['JSONparser:invalidFormat: ' msg] ); %%------------------------------------------------------------------------- function str = valid_field(str) global isoct % From MATLAB doc: field names must begin with a letter, which may be % followed by any combination of letters, digits, and underscores. % Invalid characters will be converted to underscores, and the prefix % "x0x[Hex code]_" will be added if the first character is not a letter. pos=regexp(str,'^[^A-Za-z]','once'); if(~isempty(pos)) if(~isoct) str=regexprep(str,'^([^A-Za-z])','x0x${sprintf(''%X'',unicode2native($1))}_','once'); else str=sprintf('x0x%X_%s',char(str(1)),str(2:end)); end end if(isempty(regexp(str,'[^0-9A-Za-z_]', 'once' ))) return; end if(~isoct) str=regexprep(str,'([^0-9A-Za-z_])','_0x${sprintf(''%X'',unicode2native($1))}_'); else pos=regexp(str,'[^0-9A-Za-z_]'); if(isempty(pos)) return; end str0=str; pos0=[0 pos(:)' length(str)]; str=''; for i=1:length(pos) str=[str str0(pos0(i)+1:pos(i)-1) sprintf('_0x%X_',str0(pos(i)))]; end if(pos(end)~=length(str)) str=[str str0(pos0(end-1)+1:pos0(end))]; end end %str(~isletter(str) & ~('0' <= str & str <= '9')) = '_'; %%------------------------------------------------------------------------- function endpos = matching_quote(str,pos) len=length(str); while(pos1 && str(pos-1)=='\')) endpos=pos; return; end end pos=pos+1; end error('unmatched quotation mark'); %%------------------------------------------------------------------------- function [endpos, e1l, e1r, maxlevel] = matching_bracket(str,pos) global arraytoken level=1; maxlevel=level; endpos=0; bpos=arraytoken(arraytoken>=pos); tokens=str(bpos); len=length(tokens); pos=1; e1l=[]; e1r=[]; while(pos<=len) c=tokens(pos); if(c==']') level=level-1; if(isempty(e1r)) e1r=bpos(pos); end if(level==0) endpos=bpos(pos); return end end if(c=='[') if(isempty(e1l)) e1l=bpos(pos); end level=level+1; maxlevel=max(maxlevel,level); end if(c=='"') pos=matching_quote(tokens,pos+1); end pos=pos+1; end if(endpos==0) error('unmatched "]"'); end ================================================ FILE: machine-learning-ex8/ex8/lib/jsonlab/loadubjson.m ================================================ function data = loadubjson(fname,varargin) % % data=loadubjson(fname,opt) % or % data=loadubjson(fname,'param1',value1,'param2',value2,...) % % parse a JSON (JavaScript Object Notation) file or string % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2013/08/01 % % $Id: loadubjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % fname: input file name, if fname contains "{}" or "[]", fname % will be interpreted as a UBJSON string % opt: a struct to store parsing options, opt can be replaced by % a list of ('param',value) pairs - the param string is equivallent % to a field in opt. opt can have the following % fields (first in [.|.] is the default) % % opt.SimplifyCell [0|1]: if set to 1, loadubjson will call cell2mat % for each element of the JSON data, and group % arrays based on the cell2mat rules. % opt.IntEndian [B|L]: specify the endianness of the integer fields % in the UBJSON input data. B - Big-Endian format for % integers (as required in the UBJSON specification); % L - input integer fields are in Little-Endian order. % % output: % dat: a cell array, where {...} blocks are converted into cell arrays, % and [...] are converted to arrays % % examples: % obj=struct('string','value','array',[1 2 3]); % ubjdata=saveubjson('obj',obj); % dat=loadubjson(ubjdata) % dat=loadubjson(['examples' filesep 'example1.ubj']) % dat=loadubjson(['examples' filesep 'example1.ubj'],'SimplifyCell',1) % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % global pos inStr len esc index_esc len_esc isoct arraytoken fileendian systemendian if(regexp(fname,'[\{\}\]\[]','once')) string=fname; elseif(exist(fname,'file')) fid = fopen(fname,'rb'); string = fread(fid,inf,'uint8=>char')'; fclose(fid); else error('input file does not exist'); end pos = 1; len = length(string); inStr = string; isoct=exist('OCTAVE_VERSION','builtin'); arraytoken=find(inStr=='[' | inStr==']' | inStr=='"'); jstr=regexprep(inStr,'\\\\',' '); escquote=regexp(jstr,'\\"'); arraytoken=sort([arraytoken escquote]); % String delimiters and escape chars identified to improve speed: esc = find(inStr=='"' | inStr=='\' ); % comparable to: regexp(inStr, '["\\]'); index_esc = 1; len_esc = length(esc); opt=varargin2struct(varargin{:}); fileendian=upper(jsonopt('IntEndian','B',opt)); [os,maxelem,systemendian]=computer; jsoncount=1; while pos <= len switch(next_char) case '{' data{jsoncount} = parse_object(opt); case '[' data{jsoncount} = parse_array(opt); otherwise error_pos('Outer level structure must be an object or an array'); end jsoncount=jsoncount+1; end % while jsoncount=length(data); if(jsoncount==1 && iscell(data)) data=data{1}; end if(~isempty(data)) if(isstruct(data)) % data can be a struct array data=jstruct2array(data); elseif(iscell(data)) data=jcell2array(data); end end %% function newdata=parse_collection(id,data,obj) if(jsoncount>0 && exist('data','var')) if(~iscell(data)) newdata=cell(1); newdata{1}=data; data=newdata; end end %% function newdata=jcell2array(data) len=length(data); newdata=data; for i=1:len if(isstruct(data{i})) newdata{i}=jstruct2array(data{i}); elseif(iscell(data{i})) newdata{i}=jcell2array(data{i}); end end %%------------------------------------------------------------------------- function newdata=jstruct2array(data) fn=fieldnames(data); newdata=data; len=length(data); for i=1:length(fn) % depth-first for j=1:len if(isstruct(getfield(data(j),fn{i}))) newdata(j)=setfield(newdata(j),fn{i},jstruct2array(getfield(data(j),fn{i}))); end end end if(~isempty(strmatch('x0x5F_ArrayType_',fn)) && ~isempty(strmatch('x0x5F_ArrayData_',fn))) newdata=cell(len,1); for j=1:len ndata=cast(data(j).x0x5F_ArrayData_,data(j).x0x5F_ArrayType_); iscpx=0; if(~isempty(strmatch('x0x5F_ArrayIsComplex_',fn))) if(data(j).x0x5F_ArrayIsComplex_) iscpx=1; end end if(~isempty(strmatch('x0x5F_ArrayIsSparse_',fn))) if(data(j).x0x5F_ArrayIsSparse_) if(~isempty(strmatch('x0x5F_ArraySize_',fn))) dim=double(data(j).x0x5F_ArraySize_); if(iscpx && size(ndata,2)==4-any(dim==1)) ndata(:,end-1)=complex(ndata(:,end-1),ndata(:,end)); end if isempty(ndata) % All-zeros sparse ndata=sparse(dim(1),prod(dim(2:end))); elseif dim(1)==1 % Sparse row vector ndata=sparse(1,ndata(:,1),ndata(:,2),dim(1),prod(dim(2:end))); elseif dim(2)==1 % Sparse column vector ndata=sparse(ndata(:,1),1,ndata(:,2),dim(1),prod(dim(2:end))); else % Generic sparse array. ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3),dim(1),prod(dim(2:end))); end else if(iscpx && size(ndata,2)==4) ndata(:,3)=complex(ndata(:,3),ndata(:,4)); end ndata=sparse(ndata(:,1),ndata(:,2),ndata(:,3)); end end elseif(~isempty(strmatch('x0x5F_ArraySize_',fn))) if(iscpx && size(ndata,2)==2) ndata=complex(ndata(:,1),ndata(:,2)); end ndata=reshape(ndata(:),data(j).x0x5F_ArraySize_); end newdata{j}=ndata; end if(len==1) newdata=newdata{1}; end end %%------------------------------------------------------------------------- function object = parse_object(varargin) parse_char('{'); object = []; type=''; count=-1; if(next_char == '$') type=inStr(pos+1); % TODO pos=pos+2; end if(next_char == '#') pos=pos+1; count=double(parse_number()); end if next_char ~= '}' num=0; while 1 str = parseStr(varargin{:}); if isempty(str) error_pos('Name of value at position %d cannot be empty'); end %parse_char(':'); val = parse_value(varargin{:}); num=num+1; eval( sprintf( 'object.%s = val;', valid_field(str) ) ); if next_char == '}' || (count>=0 && num>=count) break; end %parse_char(','); end end if(count==-1) parse_char('}'); end %%------------------------------------------------------------------------- function [cid,len]=elem_info(type) id=strfind('iUIlLdD',type); dataclass={'int8','uint8','int16','int32','int64','single','double'}; bytelen=[1,1,2,4,8,4,8]; if(id>0) cid=dataclass{id}; len=bytelen(id); else error_pos('unsupported type at position %d'); end %%------------------------------------------------------------------------- function [data adv]=parse_block(type,count,varargin) global pos inStr isoct fileendian systemendian [cid,len]=elem_info(type); datastr=inStr(pos:pos+len*count-1); if(isoct) newdata=int8(datastr); else newdata=uint8(datastr); end id=strfind('iUIlLdD',type); if(id<=5 && fileendian~=systemendian) newdata=swapbytes(typecast(newdata,cid)); end data=typecast(newdata,cid); adv=double(len*count); %%------------------------------------------------------------------------- function object = parse_array(varargin) % JSON array is written in row-major order global pos inStr isoct parse_char('['); object = cell(0, 1); dim=[]; type=''; count=-1; if(next_char == '$') type=inStr(pos+1); pos=pos+2; end if(next_char == '#') pos=pos+1; if(next_char=='[') dim=parse_array(varargin{:}); count=prod(double(dim)); else count=double(parse_number()); end end if(~isempty(type)) if(count>=0) [object adv]=parse_block(type,count,varargin{:}); if(~isempty(dim)) object=reshape(object,dim); end pos=pos+adv; return; else endpos=matching_bracket(inStr,pos); [cid,len]=elem_info(type); count=(endpos-pos)/len; [object adv]=parse_block(type,count,varargin{:}); pos=pos+adv; parse_char(']'); return; end end if next_char ~= ']' while 1 val = parse_value(varargin{:}); object{end+1} = val; if next_char == ']' break; end %parse_char(','); end end if(jsonopt('SimplifyCell',0,varargin{:})==1) try oldobj=object; object=cell2mat(object')'; if(iscell(oldobj) && isstruct(object) && numel(object)>1 && jsonopt('SimplifyCellArray',1,varargin{:})==0) object=oldobj; elseif(size(object,1)>1 && ndims(object)==2) object=object'; end catch end end if(count==-1) parse_char(']'); end %%------------------------------------------------------------------------- function parse_char(c) global pos inStr len skip_whitespace; if pos > len || inStr(pos) ~= c error_pos(sprintf('Expected %c at position %%d', c)); else pos = pos + 1; skip_whitespace; end %%------------------------------------------------------------------------- function c = next_char global pos inStr len skip_whitespace; if pos > len c = []; else c = inStr(pos); end %%------------------------------------------------------------------------- function skip_whitespace global pos inStr len while pos <= len && isspace(inStr(pos)) pos = pos + 1; end %%------------------------------------------------------------------------- function str = parseStr(varargin) global pos inStr esc index_esc len_esc % len, ns = length(inStr), keyboard type=inStr(pos); if type ~= 'S' && type ~= 'C' && type ~= 'H' error_pos('String starting with S expected at position %d'); else pos = pos + 1; end if(type == 'C') str=inStr(pos); pos=pos+1; return; end bytelen=double(parse_number()); if(length(inStr)>=pos+bytelen-1) str=inStr(pos:pos+bytelen-1); pos=pos+bytelen; else error_pos('End of file while expecting end of inStr'); end %%------------------------------------------------------------------------- function num = parse_number(varargin) global pos inStr len isoct fileendian systemendian id=strfind('iUIlLdD',inStr(pos)); if(isempty(id)) error_pos('expecting a number at position %d'); end type={'int8','uint8','int16','int32','int64','single','double'}; bytelen=[1,1,2,4,8,4,8]; datastr=inStr(pos+1:pos+bytelen(id)); if(isoct) newdata=int8(datastr); else newdata=uint8(datastr); end if(id<=5 && fileendian~=systemendian) newdata=swapbytes(typecast(newdata,type{id})); end num=typecast(newdata,type{id}); pos = pos + bytelen(id)+1; %%------------------------------------------------------------------------- function val = parse_value(varargin) global pos inStr len true = 1; false = 0; switch(inStr(pos)) case {'S','C','H'} val = parseStr(varargin{:}); return; case '[' val = parse_array(varargin{:}); return; case '{' val = parse_object(varargin{:}); if isstruct(val) if(~isempty(strmatch('x0x5F_ArrayType_',fieldnames(val), 'exact'))) val=jstruct2array(val); end elseif isempty(val) val = struct; end return; case {'i','U','I','l','L','d','D'} val = parse_number(varargin{:}); return; case 'T' val = true; pos = pos + 1; return; case 'F' val = false; pos = pos + 1; return; case {'Z','N'} val = []; pos = pos + 1; return; end error_pos('Value expected at position %d'); %%------------------------------------------------------------------------- function error_pos(msg) global pos inStr len poShow = max(min([pos-15 pos-1 pos pos+20],len),1); if poShow(3) == poShow(2) poShow(3:4) = poShow(2)+[0 -1]; % display nothing after end msg = [sprintf(msg, pos) ': ' ... inStr(poShow(1):poShow(2)) '' inStr(poShow(3):poShow(4)) ]; error( ['JSONparser:invalidFormat: ' msg] ); %%------------------------------------------------------------------------- function str = valid_field(str) global isoct % From MATLAB doc: field names must begin with a letter, which may be % followed by any combination of letters, digits, and underscores. % Invalid characters will be converted to underscores, and the prefix % "x0x[Hex code]_" will be added if the first character is not a letter. pos=regexp(str,'^[^A-Za-z]','once'); if(~isempty(pos)) if(~isoct) str=regexprep(str,'^([^A-Za-z])','x0x${sprintf(''%X'',unicode2native($1))}_','once'); else str=sprintf('x0x%X_%s',char(str(1)),str(2:end)); end end if(isempty(regexp(str,'[^0-9A-Za-z_]', 'once' ))) return; end if(~isoct) str=regexprep(str,'([^0-9A-Za-z_])','_0x${sprintf(''%X'',unicode2native($1))}_'); else pos=regexp(str,'[^0-9A-Za-z_]'); if(isempty(pos)) return; end str0=str; pos0=[0 pos(:)' length(str)]; str=''; for i=1:length(pos) str=[str str0(pos0(i)+1:pos(i)-1) sprintf('_0x%X_',str0(pos(i)))]; end if(pos(end)~=length(str)) str=[str str0(pos0(end-1)+1:pos0(end))]; end end %str(~isletter(str) & ~('0' <= str & str <= '9')) = '_'; %%------------------------------------------------------------------------- function endpos = matching_quote(str,pos) len=length(str); while(pos1 && str(pos-1)=='\')) endpos=pos; return; end end pos=pos+1; end error('unmatched quotation mark'); %%------------------------------------------------------------------------- function [endpos e1l e1r maxlevel] = matching_bracket(str,pos) global arraytoken level=1; maxlevel=level; endpos=0; bpos=arraytoken(arraytoken>=pos); tokens=str(bpos); len=length(tokens); pos=1; e1l=[]; e1r=[]; while(pos<=len) c=tokens(pos); if(c==']') level=level-1; if(isempty(e1r)) e1r=bpos(pos); end if(level==0) endpos=bpos(pos); return end end if(c=='[') if(isempty(e1l)) e1l=bpos(pos); end level=level+1; maxlevel=max(maxlevel,level); end if(c=='"') pos=matching_quote(tokens,pos+1); end pos=pos+1; end if(endpos==0) error('unmatched "]"'); end ================================================ FILE: machine-learning-ex8/ex8/lib/jsonlab/mergestruct.m ================================================ function s=mergestruct(s1,s2) % % s=mergestruct(s1,s2) % % merge two struct objects into one % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % date: 2012/12/22 % % input: % s1,s2: a struct object, s1 and s2 can not be arrays % % output: % s: the merged struct object. fields in s1 and s2 will be combined in s. % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(~isstruct(s1) || ~isstruct(s2)) error('input parameters contain non-struct'); end if(length(s1)>1 || length(s2)>1) error('can not merge struct arrays'); end fn=fieldnames(s2); s=s1; for i=1:length(fn) s=setfield(s,fn{i},getfield(s2,fn{i})); end ================================================ FILE: machine-learning-ex8/ex8/lib/jsonlab/savejson.m ================================================ function json=savejson(rootname,obj,varargin) % % json=savejson(rootname,obj,filename) % or % json=savejson(rootname,obj,opt) % json=savejson(rootname,obj,'param1',value1,'param2',value2,...) % % convert a MATLAB object (cell, struct or array) into a JSON (JavaScript % Object Notation) string % % author: Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2011/09/09 % % $Id: savejson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % rootname: the name of the root-object, when set to '', the root name % is ignored, however, when opt.ForceRootName is set to 1 (see below), % the MATLAB variable name will be used as the root name. % obj: a MATLAB object (array, cell, cell array, struct, struct array). % filename: a string for the file name to save the output JSON data. % opt: a struct for additional options, ignore to use default values. % opt can have the following fields (first in [.|.] is the default) % % opt.FileName [''|string]: a file name to save the output JSON data % opt.FloatFormat ['%.10g'|string]: format to show each numeric element % of a 1D/2D array; % opt.ArrayIndent [1|0]: if 1, output explicit data array with % precedent indentation; if 0, no indentation % opt.ArrayToStruct[0|1]: when set to 0, savejson outputs 1D/2D % array in JSON array format; if sets to 1, an % array will be shown as a struct with fields % "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for % sparse arrays, the non-zero elements will be % saved to _ArrayData_ field in triplet-format i.e. % (ix,iy,val) and "_ArrayIsSparse_" will be added % with a value of 1; for a complex array, the % _ArrayData_ array will include two columns % (4 for sparse) to record the real and imaginary % parts, and also "_ArrayIsComplex_":1 is added. % opt.ParseLogical [0|1]: if this is set to 1, logical array elem % will use true/false rather than 1/0. % opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single % numerical element will be shown without a square % bracket, unless it is the root object; if 0, square % brackets are forced for any numerical arrays. % opt.ForceRootName [0|1]: when set to 1 and rootname is empty, savejson % will use the name of the passed obj variable as the % root object name; if obj is an expression and % does not have a name, 'root' will be used; if this % is set to 0 and rootname is empty, the root level % will be merged down to the lower level. % opt.Inf ['"$1_Inf_"'|string]: a customized regular expression pattern % to represent +/-Inf. The matched pattern is '([-+]*)Inf' % and $1 represents the sign. For those who want to use % 1e999 to represent Inf, they can set opt.Inf to '$11e999' % opt.NaN ['"_NaN_"'|string]: a customized regular expression pattern % to represent NaN % opt.JSONP [''|string]: to generate a JSONP output (JSON with padding), % for example, if opt.JSONP='foo', the JSON data is % wrapped inside a function call as 'foo(...);' % opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson % back to the string form % opt.SaveBinary [0|1]: 1 - save the JSON file in binary mode; 0 - text mode. % opt.Compact [0|1]: 1- out compact JSON format (remove all newlines and tabs) % % opt can be replaced by a list of ('param',value) pairs. The param % string is equivallent to a field in opt and is case sensitive. % output: % json: a string in the JSON format (see http://json.org) % % examples: % jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... % 'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],... % 'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;... % 2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],... % 'MeshCreator','FangQ','MeshTitle','T6 Cube',... % 'SpecialData',[nan, inf, -inf]); % savejson('jmesh',jsonmesh) % savejson('',jsonmesh,'ArrayIndent',0,'FloatFormat','\t%.5g') % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(nargin==1) varname=inputname(1); obj=rootname; if(isempty(varname)) varname='root'; end rootname=varname; else varname=inputname(2); end if(length(varargin)==1 && ischar(varargin{1})) opt=struct('FileName',varargin{1}); else opt=varargin2struct(varargin{:}); end opt.IsOctave=exist('OCTAVE_VERSION','builtin'); rootisarray=0; rootlevel=1; forceroot=jsonopt('ForceRootName',0,opt); if((isnumeric(obj) || islogical(obj) || ischar(obj) || isstruct(obj) || iscell(obj)) && isempty(rootname) && forceroot==0) rootisarray=1; rootlevel=0; else if(isempty(rootname)) rootname=varname; end end if((isstruct(obj) || iscell(obj))&& isempty(rootname) && forceroot) rootname='root'; end whitespaces=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); if(jsonopt('Compact',0,opt)==1) whitespaces=struct('tab','','newline','','sep',','); end if(~isfield(opt,'whitespaces_')) opt.whitespaces_=whitespaces; end nl=whitespaces.newline; json=obj2json(rootname,obj,rootlevel,opt); if(rootisarray) json=sprintf('%s%s',json,nl); else json=sprintf('{%s%s%s}\n',nl,json,nl); end jsonp=jsonopt('JSONP','',opt); if(~isempty(jsonp)) json=sprintf('%s(%s);%s',jsonp,json,nl); end % save to a file if FileName is set, suggested by Patrick Rapin if(~isempty(jsonopt('FileName','',opt))) if(jsonopt('SaveBinary',0,opt)==1) fid = fopen(opt.FileName, 'wb'); fwrite(fid,json); else fid = fopen(opt.FileName, 'wt'); fwrite(fid,json,'char'); end fclose(fid); end %%------------------------------------------------------------------------- function txt=obj2json(name,item,level,varargin) if(iscell(item)) txt=cell2json(name,item,level,varargin{:}); elseif(isstruct(item)) txt=struct2json(name,item,level,varargin{:}); elseif(ischar(item)) txt=str2json(name,item,level,varargin{:}); else txt=mat2json(name,item,level,varargin{:}); end %%------------------------------------------------------------------------- function txt=cell2json(name,item,level,varargin) txt=''; if(~iscell(item)) error('input is not a cell'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); ws=jsonopt('whitespaces_',struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')),varargin{:}); padding0=repmat(ws.tab,1,level); padding2=repmat(ws.tab,1,level+1); nl=ws.newline; if(len>1) if(~isempty(name)) txt=sprintf('%s"%s": [%s',padding0, checkname(name,varargin{:}),nl); name=''; else txt=sprintf('%s[%s',padding0,nl); end elseif(len==0) if(~isempty(name)) txt=sprintf('%s"%s": []',padding0, checkname(name,varargin{:})); name=''; else txt=sprintf('%s[]',padding0); end end for j=1:dim(2) if(dim(1)>1) txt=sprintf('%s%s[%s',txt,padding2,nl); end for i=1:dim(1) txt=sprintf('%s%s',txt,obj2json(name,item{i,j},level+(dim(1)>1)+1,varargin{:})); if(i1) txt=sprintf('%s%s%s]',txt,nl,padding2); end if(j1) txt=sprintf('%s%s%s]',txt,nl,padding0); end %%------------------------------------------------------------------------- function txt=struct2json(name,item,level,varargin) txt=''; if(~isstruct(item)) error('input is not a struct'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); ws=struct('tab',sprintf('\t'),'newline',sprintf('\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding0=repmat(ws.tab,1,level); padding2=repmat(ws.tab,1,level+1); padding1=repmat(ws.tab,1,level+(dim(1)>1)+(len>1)); nl=ws.newline; if(~isempty(name)) if(len>1) txt=sprintf('%s"%s": [%s',padding0,checkname(name,varargin{:}),nl); end else if(len>1) txt=sprintf('%s[%s',padding0,nl); end end for j=1:dim(2) if(dim(1)>1) txt=sprintf('%s%s[%s',txt,padding2,nl); end for i=1:dim(1) names = fieldnames(item(i,j)); if(~isempty(name) && len==1) txt=sprintf('%s%s"%s": {%s',txt,padding1, checkname(name,varargin{:}),nl); else txt=sprintf('%s%s{%s',txt,padding1,nl); end if(~isempty(names)) for e=1:length(names) txt=sprintf('%s%s',txt,obj2json(names{e},getfield(item(i,j),... names{e}),level+(dim(1)>1)+1+(len>1),varargin{:})); if(e1) txt=sprintf('%s%s%s]',txt,nl,padding2); end if(j1) txt=sprintf('%s%s%s]',txt,nl,padding0); end %%------------------------------------------------------------------------- function txt=str2json(name,item,level,varargin) txt=''; if(~ischar(item)) error('input is not a string'); end item=reshape(item, max(size(item),[1 0])); len=size(item,1); ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding1=repmat(ws.tab,1,level); padding0=repmat(ws.tab,1,level+1); nl=ws.newline; sep=ws.sep; if(~isempty(name)) if(len>1) txt=sprintf('%s"%s": [%s',padding1,checkname(name,varargin{:}),nl); end else if(len>1) txt=sprintf('%s[%s',padding1,nl); end end isoct=jsonopt('IsOctave',0,varargin{:}); for e=1:len if(isoct) val=regexprep(item(e,:),'\\','\\'); val=regexprep(val,'"','\"'); val=regexprep(val,'^"','\"'); else val=regexprep(item(e,:),'\\','\\\\'); val=regexprep(val,'"','\\"'); val=regexprep(val,'^"','\\"'); end val=escapejsonstring(val); if(len==1) obj=['"' checkname(name,varargin{:}) '": ' '"',val,'"']; if(isempty(name)) obj=['"',val,'"']; end txt=sprintf('%s%s%s%s',txt,padding1,obj); else txt=sprintf('%s%s%s%s',txt,padding0,['"',val,'"']); end if(e==len) sep=''; end txt=sprintf('%s%s',txt,sep); end if(len>1) txt=sprintf('%s%s%s%s',txt,nl,padding1,']'); end %%------------------------------------------------------------------------- function txt=mat2json(name,item,level,varargin) if(~isnumeric(item) && ~islogical(item)) error('input is not an array'); end ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); padding1=repmat(ws.tab,1,level); padding0=repmat(ws.tab,1,level+1); nl=ws.newline; sep=ws.sep; if(length(size(item))>2 || issparse(item) || ~isreal(item) || ... isempty(item) ||jsonopt('ArrayToStruct',0,varargin{:})) if(isempty(name)) txt=sprintf('%s{%s%s"_ArrayType_": "%s",%s%s"_ArraySize_": %s,%s',... padding1,nl,padding0,class(item),nl,padding0,regexprep(mat2str(size(item)),'\s+',','),nl); else txt=sprintf('%s"%s": {%s%s"_ArrayType_": "%s",%s%s"_ArraySize_": %s,%s',... padding1,checkname(name,varargin{:}),nl,padding0,class(item),nl,padding0,regexprep(mat2str(size(item)),'\s+',','),nl); end else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1 && level>0) numtxt=regexprep(regexprep(matdata2json(item,level+1,varargin{:}),'^\[',''),']',''); else numtxt=matdata2json(item,level+1,varargin{:}); end if(isempty(name)) txt=sprintf('%s%s',padding1,numtxt); else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1) txt=sprintf('%s"%s": %s',padding1,checkname(name,varargin{:}),numtxt); else txt=sprintf('%s"%s": %s',padding1,checkname(name,varargin{:}),numtxt); end end return; end dataformat='%s%s%s%s%s'; if(issparse(item)) [ix,iy]=find(item); data=full(item(find(item))); if(~isreal(item)) data=[real(data(:)),imag(data(:))]; if(size(item,1)==1) % Kludge to have data's 'transposedness' match item's. % (Necessary for complex row vector handling below.) data=data'; end txt=sprintf(dataformat,txt,padding0,'"_ArrayIsComplex_": ','1', sep); end txt=sprintf(dataformat,txt,padding0,'"_ArrayIsSparse_": ','1', sep); if(size(item,1)==1) % Row vector, store only column indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([iy(:),data'],level+2,varargin{:}), nl); elseif(size(item,2)==1) % Column vector, store only row indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([ix,data],level+2,varargin{:}), nl); else % General case, store row and column indices. txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([ix,iy,data],level+2,varargin{:}), nl); end else if(isreal(item)) txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json(item(:)',level+2,varargin{:}), nl); else txt=sprintf(dataformat,txt,padding0,'"_ArrayIsComplex_": ','1', sep); txt=sprintf(dataformat,txt,padding0,'"_ArrayData_": ',... matdata2json([real(item(:)) imag(item(:))],level+2,varargin{:}), nl); end end txt=sprintf('%s%s%s',txt,padding1,'}'); %%------------------------------------------------------------------------- function txt=matdata2json(mat,level,varargin) ws=struct('tab',sprintf('\t'),'newline',sprintf('\n'),'sep',sprintf(',\n')); ws=jsonopt('whitespaces_',ws,varargin{:}); tab=ws.tab; nl=ws.newline; if(size(mat,1)==1) pre=''; post=''; level=level-1; else pre=sprintf('[%s',nl); post=sprintf('%s%s]',nl,repmat(tab,1,level-1)); end if(isempty(mat)) txt='null'; return; end floatformat=jsonopt('FloatFormat','%.10g',varargin{:}); %if(numel(mat)>1) formatstr=['[' repmat([floatformat ','],1,size(mat,2)-1) [floatformat sprintf('],%s',nl)]]; %else % formatstr=[repmat([floatformat ','],1,size(mat,2)-1) [floatformat sprintf(',\n')]]; %end if(nargin>=2 && size(mat,1)>1 && jsonopt('ArrayIndent',1,varargin{:})==1) formatstr=[repmat(tab,1,level) formatstr]; end txt=sprintf(formatstr,mat'); txt(end-length(nl):end)=[]; if(islogical(mat) && jsonopt('ParseLogical',0,varargin{:})==1) txt=regexprep(txt,'1','true'); txt=regexprep(txt,'0','false'); end %txt=regexprep(mat2str(mat),'\s+',','); %txt=regexprep(txt,';',sprintf('],\n[')); % if(nargin>=2 && size(mat,1)>1) % txt=regexprep(txt,'\[',[repmat(sprintf('\t'),1,level) '[']); % end txt=[pre txt post]; if(any(isinf(mat(:)))) txt=regexprep(txt,'([-+]*)Inf',jsonopt('Inf','"$1_Inf_"',varargin{:})); end if(any(isnan(mat(:)))) txt=regexprep(txt,'NaN',jsonopt('NaN','"_NaN_"',varargin{:})); end %%------------------------------------------------------------------------- function newname=checkname(name,varargin) isunpack=jsonopt('UnpackHex',1,varargin{:}); newname=name; if(isempty(regexp(name,'0x([0-9a-fA-F]+)_','once'))) return end if(isunpack) isoct=jsonopt('IsOctave',0,varargin{:}); if(~isoct) newname=regexprep(name,'(^x|_){1}0x([0-9a-fA-F]+)_','${native2unicode(hex2dec($2))}'); else pos=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','start'); pend=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','end'); if(isempty(pos)) return; end str0=name; pos0=[0 pend(:)' length(name)]; newname=''; for i=1:length(pos) newname=[newname str0(pos0(i)+1:pos(i)-1) char(hex2dec(str0(pos(i)+3:pend(i)-1)))]; end if(pos(end)~=length(name)) newname=[newname str0(pos0(end-1)+1:pos0(end))]; end end end %%------------------------------------------------------------------------- function newstr=escapejsonstring(str) newstr=str; isoct=exist('OCTAVE_VERSION','builtin'); if(isoct) vv=sscanf(OCTAVE_VERSION,'%f'); if(vv(1)>=3.8) isoct=0; end end if(isoct) escapechars={'\a','\f','\n','\r','\t','\v'}; for i=1:length(escapechars); newstr=regexprep(newstr,escapechars{i},escapechars{i}); end else escapechars={'\a','\b','\f','\n','\r','\t','\v'}; for i=1:length(escapechars); newstr=regexprep(newstr,escapechars{i},regexprep(escapechars{i},'\\','\\\\')); end end ================================================ FILE: machine-learning-ex8/ex8/lib/jsonlab/saveubjson.m ================================================ function json=saveubjson(rootname,obj,varargin) % % json=saveubjson(rootname,obj,filename) % or % json=saveubjson(rootname,obj,opt) % json=saveubjson(rootname,obj,'param1',value1,'param2',value2,...) % % convert a MATLAB object (cell, struct or array) into a Universal % Binary JSON (UBJSON) binary string % % author: Qianqian Fang (fangq nmr.mgh.harvard.edu) % created on 2013/08/17 % % $Id: saveubjson.m 460 2015-01-03 00:30:45Z fangq $ % % input: % rootname: the name of the root-object, when set to '', the root name % is ignored, however, when opt.ForceRootName is set to 1 (see below), % the MATLAB variable name will be used as the root name. % obj: a MATLAB object (array, cell, cell array, struct, struct array) % filename: a string for the file name to save the output UBJSON data % opt: a struct for additional options, ignore to use default values. % opt can have the following fields (first in [.|.] is the default) % % opt.FileName [''|string]: a file name to save the output JSON data % opt.ArrayToStruct[0|1]: when set to 0, saveubjson outputs 1D/2D % array in JSON array format; if sets to 1, an % array will be shown as a struct with fields % "_ArrayType_", "_ArraySize_" and "_ArrayData_"; for % sparse arrays, the non-zero elements will be % saved to _ArrayData_ field in triplet-format i.e. % (ix,iy,val) and "_ArrayIsSparse_" will be added % with a value of 1; for a complex array, the % _ArrayData_ array will include two columns % (4 for sparse) to record the real and imaginary % parts, and also "_ArrayIsComplex_":1 is added. % opt.ParseLogical [1|0]: if this is set to 1, logical array elem % will use true/false rather than 1/0. % opt.NoRowBracket [1|0]: if this is set to 1, arrays with a single % numerical element will be shown without a square % bracket, unless it is the root object; if 0, square % brackets are forced for any numerical arrays. % opt.ForceRootName [0|1]: when set to 1 and rootname is empty, saveubjson % will use the name of the passed obj variable as the % root object name; if obj is an expression and % does not have a name, 'root' will be used; if this % is set to 0 and rootname is empty, the root level % will be merged down to the lower level. % opt.JSONP [''|string]: to generate a JSONP output (JSON with padding), % for example, if opt.JSON='foo', the JSON data is % wrapped inside a function call as 'foo(...);' % opt.UnpackHex [1|0]: conver the 0x[hex code] output by loadjson % back to the string form % % opt can be replaced by a list of ('param',value) pairs. The param % string is equivallent to a field in opt and is case sensitive. % output: % json: a binary string in the UBJSON format (see http://ubjson.org) % % examples: % jsonmesh=struct('MeshNode',[0 0 0;1 0 0;0 1 0;1 1 0;0 0 1;1 0 1;0 1 1;1 1 1],... % 'MeshTetra',[1 2 4 8;1 3 4 8;1 2 6 8;1 5 6 8;1 5 7 8;1 3 7 8],... % 'MeshTri',[1 2 4;1 2 6;1 3 4;1 3 7;1 5 6;1 5 7;... % 2 8 4;2 8 6;3 8 4;3 8 7;5 8 6;5 8 7],... % 'MeshCreator','FangQ','MeshTitle','T6 Cube',... % 'SpecialData',[nan, inf, -inf]); % saveubjson('jsonmesh',jsonmesh) % saveubjson('jsonmesh',jsonmesh,'meshdata.ubj') % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of JSONLab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % if(nargin==1) varname=inputname(1); obj=rootname; if(isempty(varname)) varname='root'; end rootname=varname; else varname=inputname(2); end if(length(varargin)==1 && ischar(varargin{1})) opt=struct('FileName',varargin{1}); else opt=varargin2struct(varargin{:}); end opt.IsOctave=exist('OCTAVE_VERSION','builtin'); rootisarray=0; rootlevel=1; forceroot=jsonopt('ForceRootName',0,opt); if((isnumeric(obj) || islogical(obj) || ischar(obj) || isstruct(obj) || iscell(obj)) && isempty(rootname) && forceroot==0) rootisarray=1; rootlevel=0; else if(isempty(rootname)) rootname=varname; end end if((isstruct(obj) || iscell(obj))&& isempty(rootname) && forceroot) rootname='root'; end json=obj2ubjson(rootname,obj,rootlevel,opt); if(~rootisarray) json=['{' json '}']; end jsonp=jsonopt('JSONP','',opt); if(~isempty(jsonp)) json=[jsonp '(' json ')']; end % save to a file if FileName is set, suggested by Patrick Rapin if(~isempty(jsonopt('FileName','',opt))) fid = fopen(opt.FileName, 'wb'); fwrite(fid,json); fclose(fid); end %%------------------------------------------------------------------------- function txt=obj2ubjson(name,item,level,varargin) if(iscell(item)) txt=cell2ubjson(name,item,level,varargin{:}); elseif(isstruct(item)) txt=struct2ubjson(name,item,level,varargin{:}); elseif(ischar(item)) txt=str2ubjson(name,item,level,varargin{:}); else txt=mat2ubjson(name,item,level,varargin{:}); end %%------------------------------------------------------------------------- function txt=cell2ubjson(name,item,level,varargin) txt=''; if(~iscell(item)) error('input is not a cell'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); % let's handle 1D cell first if(len>1) if(~isempty(name)) txt=[S_(checkname(name,varargin{:})) '[']; name=''; else txt='['; end elseif(len==0) if(~isempty(name)) txt=[S_(checkname(name,varargin{:})) 'Z']; name=''; else txt='Z'; end end for j=1:dim(2) if(dim(1)>1) txt=[txt '[']; end for i=1:dim(1) txt=[txt obj2ubjson(name,item{i,j},level+(len>1),varargin{:})]; end if(dim(1)>1) txt=[txt ']']; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=struct2ubjson(name,item,level,varargin) txt=''; if(~isstruct(item)) error('input is not a struct'); end dim=size(item); if(ndims(squeeze(item))>2) % for 3D or higher dimensions, flatten to 2D for now item=reshape(item,dim(1),numel(item)/dim(1)); dim=size(item); end len=numel(item); if(~isempty(name)) if(len>1) txt=[S_(checkname(name,varargin{:})) '[']; end else if(len>1) txt='['; end end for j=1:dim(2) if(dim(1)>1) txt=[txt '[']; end for i=1:dim(1) names = fieldnames(item(i,j)); if(~isempty(name) && len==1) txt=[txt S_(checkname(name,varargin{:})) '{']; else txt=[txt '{']; end if(~isempty(names)) for e=1:length(names) txt=[txt obj2ubjson(names{e},getfield(item(i,j),... names{e}),level+(dim(1)>1)+1+(len>1),varargin{:})]; end end txt=[txt '}']; end if(dim(1)>1) txt=[txt ']']; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=str2ubjson(name,item,level,varargin) txt=''; if(~ischar(item)) error('input is not a string'); end item=reshape(item, max(size(item),[1 0])); len=size(item,1); if(~isempty(name)) if(len>1) txt=[S_(checkname(name,varargin{:})) '[']; end else if(len>1) txt='['; end end isoct=jsonopt('IsOctave',0,varargin{:}); for e=1:len val=item(e,:); if(len==1) obj=['' S_(checkname(name,varargin{:})) '' '',S_(val),'']; if(isempty(name)) obj=['',S_(val),'']; end txt=[txt,'',obj]; else txt=[txt,'',['',S_(val),'']]; end end if(len>1) txt=[txt ']']; end %%------------------------------------------------------------------------- function txt=mat2ubjson(name,item,level,varargin) if(~isnumeric(item) && ~islogical(item)) error('input is not an array'); end if(length(size(item))>2 || issparse(item) || ~isreal(item) || ... isempty(item) || jsonopt('ArrayToStruct',0,varargin{:})) cid=I_(uint32(max(size(item)))); if(isempty(name)) txt=['{' S_('_ArrayType_'),S_(class(item)),S_('_ArraySize_'),I_a(size(item),cid(1)) ]; else if(isempty(item)) txt=[S_(checkname(name,varargin{:})),'Z']; return; else txt=[S_(checkname(name,varargin{:})),'{',S_('_ArrayType_'),S_(class(item)),S_('_ArraySize_'),I_a(size(item),cid(1))]; end end else if(isempty(name)) txt=matdata2ubjson(item,level+1,varargin{:}); else if(numel(item)==1 && jsonopt('NoRowBracket',1,varargin{:})==1) numtxt=regexprep(regexprep(matdata2ubjson(item,level+1,varargin{:}),'^\[',''),']',''); txt=[S_(checkname(name,varargin{:})) numtxt]; else txt=[S_(checkname(name,varargin{:})),matdata2ubjson(item,level+1,varargin{:})]; end end return; end if(issparse(item)) [ix,iy]=find(item); data=full(item(find(item))); if(~isreal(item)) data=[real(data(:)),imag(data(:))]; if(size(item,1)==1) % Kludge to have data's 'transposedness' match item's. % (Necessary for complex row vector handling below.) data=data'; end txt=[txt,S_('_ArrayIsComplex_'),'T']; end txt=[txt,S_('_ArrayIsSparse_'),'T']; if(size(item,1)==1) % Row vector, store only column indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([iy(:),data'],level+2,varargin{:})]; elseif(size(item,2)==1) % Column vector, store only row indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([ix,data],level+2,varargin{:})]; else % General case, store row and column indices. txt=[txt,S_('_ArrayData_'),... matdata2ubjson([ix,iy,data],level+2,varargin{:})]; end else if(isreal(item)) txt=[txt,S_('_ArrayData_'),... matdata2ubjson(item(:)',level+2,varargin{:})]; else txt=[txt,S_('_ArrayIsComplex_'),'T']; txt=[txt,S_('_ArrayData_'),... matdata2ubjson([real(item(:)) imag(item(:))],level+2,varargin{:})]; end end txt=[txt,'}']; %%------------------------------------------------------------------------- function txt=matdata2ubjson(mat,level,varargin) if(isempty(mat)) txt='Z'; return; end if(size(mat,1)==1) level=level-1; end type=''; hasnegtive=(mat<0); if(isa(mat,'integer') || isinteger(mat) || (isfloat(mat) && all(mod(mat(:),1) == 0))) if(isempty(hasnegtive)) if(max(mat(:))<=2^8) type='U'; end end if(isempty(type)) % todo - need to consider negative ones separately id= histc(abs(max(mat(:))),[0 2^7 2^15 2^31 2^63]); if(isempty(find(id))) error('high-precision data is not yet supported'); end key='iIlL'; type=key(find(id)); end txt=[I_a(mat(:),type,size(mat))]; elseif(islogical(mat)) logicalval='FT'; if(numel(mat)==1) txt=logicalval(mat+1); else txt=['[$U#' I_a(size(mat),'l') typecast(swapbytes(uint8(mat(:)')),'uint8')]; end else if(numel(mat)==1) txt=['[' D_(mat) ']']; else txt=D_a(mat(:),'D',size(mat)); end end %txt=regexprep(mat2str(mat),'\s+',','); %txt=regexprep(txt,';',sprintf('],[')); % if(nargin>=2 && size(mat,1)>1) % txt=regexprep(txt,'\[',[repmat(sprintf('\t'),1,level) '[']); % end if(any(isinf(mat(:)))) txt=regexprep(txt,'([-+]*)Inf',jsonopt('Inf','"$1_Inf_"',varargin{:})); end if(any(isnan(mat(:)))) txt=regexprep(txt,'NaN',jsonopt('NaN','"_NaN_"',varargin{:})); end %%------------------------------------------------------------------------- function newname=checkname(name,varargin) isunpack=jsonopt('UnpackHex',1,varargin{:}); newname=name; if(isempty(regexp(name,'0x([0-9a-fA-F]+)_','once'))) return end if(isunpack) isoct=jsonopt('IsOctave',0,varargin{:}); if(~isoct) newname=regexprep(name,'(^x|_){1}0x([0-9a-fA-F]+)_','${native2unicode(hex2dec($2))}'); else pos=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','start'); pend=regexp(name,'(^x|_){1}0x([0-9a-fA-F]+)_','end'); if(isempty(pos)) return; end str0=name; pos0=[0 pend(:)' length(name)]; newname=''; for i=1:length(pos) newname=[newname str0(pos0(i)+1:pos(i)-1) char(hex2dec(str0(pos(i)+3:pend(i)-1)))]; end if(pos(end)~=length(name)) newname=[newname str0(pos0(end-1)+1:pos0(end))]; end end end %%------------------------------------------------------------------------- function val=S_(str) if(length(str)==1) val=['C' str]; else val=['S' I_(int32(length(str))) str]; end %%------------------------------------------------------------------------- function val=I_(num) if(~isinteger(num)) error('input is not an integer'); end if(num>=0 && num<255) val=['U' data2byte(swapbytes(cast(num,'uint8')),'uint8')]; return; end key='iIlL'; cid={'int8','int16','int32','int64'}; for i=1:4 if((num>0 && num<2^(i*8-1)) || (num<0 && num>=-2^(i*8-1))) val=[key(i) data2byte(swapbytes(cast(num,cid{i})),'uint8')]; return; end end error('unsupported integer'); %%------------------------------------------------------------------------- function val=D_(num) if(~isfloat(num)) error('input is not a float'); end if(isa(num,'single')) val=['d' data2byte(num,'uint8')]; else val=['D' data2byte(num,'uint8')]; end %%------------------------------------------------------------------------- function data=I_a(num,type,dim,format) id=find(ismember('iUIlL',type)); if(id==0) error('unsupported integer array'); end % based on UBJSON specs, all integer types are stored in big endian format if(id==1) data=data2byte(swapbytes(int8(num)),'uint8'); blen=1; elseif(id==2) data=data2byte(swapbytes(uint8(num)),'uint8'); blen=1; elseif(id==3) data=data2byte(swapbytes(int16(num)),'uint8'); blen=2; elseif(id==4) data=data2byte(swapbytes(int32(num)),'uint8'); blen=4; elseif(id==5) data=data2byte(swapbytes(int64(num)),'uint8'); blen=8; end if(nargin>=3 && length(dim)>=2 && prod(dim)~=dim(2)) format='opt'; end if((nargin<4 || strcmp(format,'opt')) && numel(num)>1) if(nargin>=3 && (length(dim)==1 || (length(dim)>=2 && prod(dim)~=dim(2)))) cid=I_(uint32(max(dim))); data=['$' type '#' I_a(dim,cid(1)) data(:)']; else data=['$' type '#' I_(int32(numel(data)/blen)) data(:)']; end data=['[' data(:)']; else data=reshape(data,blen,numel(data)/blen); data(2:blen+1,:)=data; data(1,:)=type; data=data(:)'; data=['[' data(:)' ']']; end %%------------------------------------------------------------------------- function data=D_a(num,type,dim,format) id=find(ismember('dD',type)); if(id==0) error('unsupported float array'); end if(id==1) data=data2byte(single(num),'uint8'); elseif(id==2) data=data2byte(double(num),'uint8'); end if(nargin>=3 && length(dim)>=2 && prod(dim)~=dim(2)) format='opt'; end if((nargin<4 || strcmp(format,'opt')) && numel(num)>1) if(nargin>=3 && (length(dim)==1 || (length(dim)>=2 && prod(dim)~=dim(2)))) cid=I_(uint32(max(dim))); data=['$' type '#' I_a(dim,cid(1)) data(:)']; else data=['$' type '#' I_(int32(numel(data)/(id*4))) data(:)']; end data=['[' data]; else data=reshape(data,(id*4),length(data)/(id*4)); data(2:(id*4+1),:)=data; data(1,:)=type; data=data(:)'; data=['[' data(:)' ']']; end %%------------------------------------------------------------------------- function bytes=data2byte(varargin) bytes=typecast(varargin{:}); bytes=bytes(:)'; ================================================ FILE: machine-learning-ex8/ex8/lib/jsonlab/varargin2struct.m ================================================ function opt=varargin2struct(varargin) % % opt=varargin2struct('param1',value1,'param2',value2,...) % or % opt=varargin2struct(...,optstruct,...) % % convert a series of input parameters into a structure % % authors:Qianqian Fang (fangq nmr.mgh.harvard.edu) % date: 2012/12/22 % % input: % 'param', value: the input parameters should be pairs of a string and a value % optstruct: if a parameter is a struct, the fields will be merged to the output struct % % output: % opt: a struct where opt.param1=value1, opt.param2=value2 ... % % license: % BSD, see LICENSE_BSD.txt files for details % % -- this function is part of jsonlab toolbox (http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab) % len=length(varargin); opt=struct; if(len==0) return; end i=1; while(i<=len) if(isstruct(varargin{i})) opt=mergestruct(opt,varargin{i}); elseif(ischar(varargin{i}) && i bestF1 bestF1 = F1; bestEpsilon = epsilon; end end end ================================================ FILE: machine-learning-ex8/ex8/submit.m ================================================ function submit() addpath('./lib'); conf.assignmentSlug = 'anomaly-detection-and-recommender-systems'; conf.itemName = 'Anomaly Detection and Recommender Systems'; conf.partArrays = { ... { ... '1', ... { 'estimateGaussian.m' }, ... 'Estimate Gaussian Parameters', ... }, ... { ... '2', ... { 'selectThreshold.m' }, ... 'Select Threshold', ... }, ... { ... '3', ... { 'cofiCostFunc.m' }, ... 'Collaborative Filtering Cost', ... }, ... { ... '4', ... { 'cofiCostFunc.m' }, ... 'Collaborative Filtering Gradient', ... }, ... { ... '5', ... { 'cofiCostFunc.m' }, ... 'Regularized Cost', ... }, ... { ... '6', ... { 'cofiCostFunc.m' }, ... 'Regularized Gradient', ... }, ... }; conf.output = @output; submitWithConfiguration(conf); end function out = output(partId, auxstring) % Random Test Cases n_u = 3; n_m = 4; n = 5; X = reshape(sin(1:n_m*n), n_m, n); Theta = reshape(cos(1:n_u*n), n_u, n); Y = reshape(sin(1:2:2*n_m*n_u), n_m, n_u); R = Y > 0.5; pval = [abs(Y(:)) ; 0.001; 1]; yval = [R(:) ; 1; 0]; params = [X(:); Theta(:)]; if partId == '1' [mu sigma2] = estimateGaussian(X); out = sprintf('%0.5f ', [mu(:); sigma2(:)]); elseif partId == '2' [bestEpsilon bestF1] = selectThreshold(yval, pval); out = sprintf('%0.5f ', [bestEpsilon(:); bestF1(:)]); elseif partId == '3' [J] = cofiCostFunc(params, Y, R, n_u, n_m, ... n, 0); out = sprintf('%0.5f ', J(:)); elseif partId == '4' [J, grad] = cofiCostFunc(params, Y, R, n_u, n_m, ... n, 0); out = sprintf('%0.5f ', grad(:)); elseif partId == '5' [J] = cofiCostFunc(params, Y, R, n_u, n_m, ... n, 1.5); out = sprintf('%0.5f ', J(:)); elseif partId == '6' [J, grad] = cofiCostFunc(params, Y, R, n_u, n_m, ... n, 1.5); out = sprintf('%0.5f ', grad(:)); end end ================================================ FILE: machine-learning-ex8/ex8/visualizeFit.m ================================================ function visualizeFit(X, mu, sigma2) %VISUALIZEFIT Visualize the dataset and its estimated distribution. % VISUALIZEFIT(X, p, mu, sigma2) This visualization shows you the % probability density function of the Gaussian distribution. Each example % has a location (x1, x2) that depends on its feature values. % [X1,X2] = meshgrid(0:.5:35); Z = multivariateGaussian([X1(:) X2(:)],mu,sigma2); Z = reshape(Z,size(X1)); plot(X(:, 1), X(:, 2),'bx'); hold on; % Do not plot if there are infinities if (sum(isinf(Z)) == 0) contour(X1, X2, Z, 10.^(-20:3:0)'); end hold off; end ================================================ FILE: readme.md ================================================ Machine Learning Exercises ====== Coursera上斯坦福大学的机器学习编程作业 ### 1. [machine-learning-ex1][1] - 线性回归 - 梯度下降 - 正规方程 ![machine-learning-ex1][1.1] ### 2. [machine-learning-ex2][2] - 逻辑回归 - 正则化 ![machine-learning-ex2][2.1] ### 3. [machine-learning-ex3][3] - 多分类逻辑回归 - 神经网络预测(权值weights已知) ![machine-learning-ex3][3.1] ### 4. [machine-learning-ex4][4] - bp神经网络识别手写数字 ![machine-learning-ex4][4.1] ### 5. [machine-learning-ex5][5] - 学习曲线 - 验证集 ![machine-learning-ex5][5.1] ### 6. [machine-learning-ex6][6] - 支持向量SVM - 高斯核函数 - 线性核函数 - 垃圾邮件分类 ![machine-learning-ex6][6.1] ### 7. [machine-learning-ex7][7] - k-means - 降维PCA（主成分分析） ![machine-learning-ex7][7.1] ### 8. [machine-learning-ex8][8] - 异常检测（高斯分布） - 协同过滤 ![machine-learning-ex8][8.1] [1]:https://github.com/lawlite19/MachineLearningEx/tree/master/machine-learning-ex1 [1.1]: ./images/machine-learning-ex1_01.png "machine-learning-ex1_01.png" [2]:https://github.com/lawlite19/MachineLearningEx/tree/master/machine-learning-ex2 [2.1]: ./images/machine-learning-ex2_01.png "machine-learning-ex2_01.png" [3]:https://github.com/lawlite19/MachineLearningEx/blob/master/machine-learning-ex3 [3.1]: ./images/machine-learning-ex3_01.png "machine-learning-ex3_01.png" [4]:https://github.com/lawlite19/MachineLearningEx/blob/master/machine-learning-ex4 [4.1]: ./images/machine-learning-ex4_01.png "machine-learning-ex4_01.png" [5]:https://github.com/lawlite19/MachineLearningEx/blob/master/machine-learning-ex5 [5.1]: ./images/machine-learning-ex5_01.png "machine-learning-ex5_01.png" [6]:https://github.com/lawlite19/MachineLearningEx/blob/master/machine-learning-ex6 [6.1]: ./images/machine-learning-ex6_01.png "machine-learning-ex6_01.png" [7]:https://github.com/lawlite19/MachineLearningEx/blob/master/machine-learning-ex7 [7.1]: ./images/machine-learning-ex7_01.png "machine-learning-ex7_01.png" [8]:https://github.com/lawlite19/MachineLearningEx/blob/master/machine-learning-ex8 [8.1]: ./images/machine-learning-ex8_01.png "machine-learning-ex8_01.png"