[ { "path": ".gitignore", "content": "node_modules/\ndist/\n*.log\n.vscode\n" }, { "path": "CONTRIBUTING.md", "content": "Want to contribute? Great! First, read this page (including the small print at the end).\n\n### Before you contribute\nBefore we can use your code, you must sign the\n[Google Individual Contributor License Agreement]\n(https://cla.developers.google.com/about/google-individual)\n(CLA), which you can do online. The CLA is necessary mainly because you own the\ncopyright to your changes, even after your contribution becomes part of our\ncodebase, so we need your permission to use and distribute your code. We also\nneed to be sure of various other things—for instance that you'll tell us if you\nknow that your code infringes on other people's patents. You don't have to sign\nthe CLA until after you've submitted your code for review and a member has\napproved it, but you must do it before we can put your code into our codebase.\nBefore you start working on a larger contribution, you should get in touch with\nus first through the issue tracker with your idea so that we can help out and\npossibly guide you. Coordinating up front makes it much easier to avoid\nfrustration later on.\n\n### Code reviews\nAll submissions, including submissions by project members, require review. We\nuse Github pull requests for this purpose.\n\n### The small print\nContributions made by corporations are covered by a different agreement than\nthe one above, the\n[Software Grant and Corporate Contributor License Agreement]\n(https://cla.developers.google.com/about/google-corporate).\n" }, { "path": "LICENSE", "content": "\n Apache License\n Version 2.0, January 2004\n http://www.apache.org/licenses/\n\n TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n 1. Definitions.\n\n \"License\" shall mean the terms and conditions for use, reproduction,\n and distribution as defined by Sections 1 through 9 of this document.\n\n \"Licensor\" shall mean the copyright owner or entity authorized by\n the copyright owner that is granting the License.\n\n \"Legal Entity\" shall mean the union of the acting entity and all\n other entities that control, are controlled by, or are under common\n control with that entity. For the purposes of this definition,\n \"control\" means (i) the power, direct or indirect, to cause the\n direction or management of such entity, whether by contract or\n otherwise, or (ii) ownership of fifty percent (50%) or more of the\n outstanding shares, or (iii) beneficial ownership of such entity.\n\n \"You\" (or \"Your\") shall mean an individual or Legal Entity\n exercising permissions granted by this License.\n\n \"Source\" form shall mean the preferred form for making modifications,\n including but not limited to software source code, documentation\n source, and configuration files.\n\n \"Object\" form shall mean any form resulting from mechanical\n transformation or translation of a Source form, including but\n not limited to compiled object code, generated documentation,\n and conversions to other media types.\n\n \"Work\" shall mean the work of authorship, whether in Source or\n Object form, made available under the License, as indicated by a\n copyright notice that is included in or attached to the work\n (an example is provided in the Appendix below).\n\n \"Derivative Works\" shall mean any work, whether in Source or Object\n form, that is based on (or derived from) the Work and for which the\n editorial revisions, annotations, elaborations, or other modifications\n represent, as a whole, an original work of authorship. For the purposes\n of this License, Derivative Works shall not include works that remain\n separable from, or merely link (or bind by name) to the interfaces of,\n the Work and Derivative Works thereof.\n\n \"Contribution\" shall mean any work of authorship, including\n the original version of the Work and any modifications or additions\n to that Work or Derivative Works thereof, that is intentionally\n submitted to Licensor for inclusion in the Work by the copyright owner\n or by an individual or Legal Entity authorized to submit on behalf of\n the copyright owner. For the purposes of this definition, \"submitted\"\n means any form of electronic, verbal, or written communication sent\n to the Licensor or its representatives, including but not limited to\n communication on electronic mailing lists, source code control systems,\n and issue tracking systems that are managed by, or on behalf of, the\n Licensor for the purpose of discussing and improving the Work, but\n excluding communication that is conspicuously marked or otherwise\n designated in writing by the copyright owner as \"Not a Contribution.\"\n\n \"Contributor\" shall mean Licensor and any individual or Legal Entity\n on behalf of whom a Contribution has been received by Licensor and\n subsequently incorporated within the Work.\n\n 2. Grant of Copyright License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n copyright license to reproduce, prepare Derivative Works of,\n publicly display, publicly perform, sublicense, and distribute the\n Work and such Derivative Works in Source or Object form.\n\n 3. Grant of Patent License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n (except as stated in this section) patent license to make, have made,\n use, offer to sell, sell, import, and otherwise transfer the Work,\n where such license applies only to those patent claims licensable\n by such Contributor that are necessarily infringed by their\n Contribution(s) alone or by combination of their Contribution(s)\n with the Work to which such Contribution(s) was submitted. If You\n institute patent litigation against any entity (including a\n cross-claim or counterclaim in a lawsuit) alleging that the Work\n or a Contribution incorporated within the Work constitutes direct\n or contributory patent infringement, then any patent licenses\n granted to You under this License for that Work shall terminate\n as of the date such litigation is filed.\n\n 4. Redistribution. You may reproduce and distribute copies of the\n Work or Derivative Works thereof in any medium, with or without\n modifications, and in Source or Object form, provided that You\n meet the following conditions:\n\n (a) You must give any other recipients of the Work or\n Derivative Works a copy of this License; and\n\n (b) You must cause any modified files to carry prominent notices\n stating that You changed the files; and\n\n (c) You must retain, in the Source form of any Derivative Works\n that You distribute, all copyright, patent, trademark, and\n attribution notices from the Source form of the Work,\n excluding those notices that do not pertain to any part of\n the Derivative Works; and\n\n (d) If the Work includes a \"NOTICE\" text file as part of its\n distribution, then any Derivative Works that You distribute must\n include a readable copy of the attribution notices contained\n within such NOTICE file, excluding those notices that do not\n pertain to any part of the Derivative Works, in at least one\n of the following places: within a NOTICE text file distributed\n as part of the Derivative Works; within the Source form or\n documentation, if provided along with the Derivative Works; or,\n within a display generated by the Derivative Works, if and\n wherever such third-party notices normally appear. The contents\n of the NOTICE file are for informational purposes only and\n do not modify the License. You may add Your own attribution\n notices within Derivative Works that You distribute, alongside\n or as an addendum to the NOTICE text from the Work, provided\n that such additional attribution notices cannot be construed\n as modifying the License.\n\n You may add Your own copyright statement to Your modifications and\n may provide additional or different license terms and conditions\n for use, reproduction, or distribution of Your modifications, or\n for any such Derivative Works as a whole, provided Your use,\n reproduction, and distribution of the Work otherwise complies with\n the conditions stated in this License.\n\n 5. Submission of Contributions. Unless You explicitly state otherwise,\n any Contribution intentionally submitted for inclusion in the Work\n by You to the Licensor shall be under the terms and conditions of\n this License, without any additional terms or conditions.\n Notwithstanding the above, nothing herein shall supersede or modify\n the terms of any separate license agreement you may have executed\n with Licensor regarding such Contributions.\n\n 6. Trademarks. This License does not grant permission to use the trade\n names, trademarks, service marks, or product names of the Licensor,\n except as required for reasonable and customary use in describing the\n origin of the Work and reproducing the content of the NOTICE file.\n\n 7. Disclaimer of Warranty. Unless required by applicable law or\n agreed to in writing, Licensor provides the Work (and each\n Contributor provides its Contributions) on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n implied, including, without limitation, any warranties or conditions\n of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n PARTICULAR PURPOSE. You are solely responsible for determining the\n appropriateness of using or redistributing the Work and assume any\n risks associated with Your exercise of permissions under this License.\n\n 8. Limitation of Liability. In no event and under no legal theory,\n whether in tort (including negligence), contract, or otherwise,\n unless required by applicable law (such as deliberate and grossly\n negligent acts) or agreed to in writing, shall any Contributor be\n liable to You for damages, including any direct, indirect, special,\n incidental, or consequential damages of any character arising as a\n result of this License or out of the use or inability to use the\n Work (including but not limited to damages for loss of goodwill,\n work stoppage, computer failure or malfunction, or any and all\n other commercial damages or losses), even if such Contributor\n has been advised of the possibility of such damages.\n\n 9. Accepting Warranty or Additional Liability. While redistributing\n the Work or Derivative Works thereof, You may choose to offer,\n and charge a fee for, acceptance of support, warranty, indemnity,\n or other liability obligations and/or rights consistent with this\n License. However, in accepting such obligations, You may act only\n on Your own behalf and on Your sole responsibility, not on behalf\n of any other Contributor, and only if You agree to indemnify,\n defend, and hold each Contributor harmless for any liability\n incurred by, or claims asserted against, such Contributor by reason\n of your accepting any such warranty or additional liability.\n\n END OF TERMS AND CONDITIONS\n\n APPENDIX: How to apply the Apache License to your work.\n\n To apply the Apache License to your work, attach the following\n boilerplate notice, with the fields enclosed by brackets \"[]\"\n replaced with your own identifying information. (Don't include\n the brackets!) The text should be enclosed in the appropriate\n comment syntax for the file format. We also recommend that a\n file or class name and description of purpose be included on the\n same \"printed page\" as the copyright notice for easier\n identification within third-party archives.\n\n Copyright [yyyy] [name of copyright owner]\n\n Licensed under the Apache License, Version 2.0 (the \"License\");\n you may not use this file except in compliance with the License.\n You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software\n distributed under the License is distributed on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n See the License for the specific language governing permissions and\n limitations under the License.\n" }, { "path": "README.md", "content": "# Deep playground\n\nDeep playground is an interactive visualization of neural networks, written in\nTypeScript using d3.js. We use GitHub issues for tracking new requests and bugs.\nYour feedback is highly appreciated!\n\n**If you'd like to contribute, be sure to review the [contribution guidelines](CONTRIBUTING.md).**\n\n## Development\n\nTo run the visualization locally, run:\n- `npm i` to install dependencies\n- `npm run build` to compile the app and place it in the `dist/` directory\n- `npm run serve` to serve from the `dist/` directory and open a page on your browser.\n\nFor a fast edit-refresh cycle when developing run `npm run serve-watch`.\nThis will start an http server and automatically re-compile the TypeScript,\nHTML and CSS files whenever they change.\n\n## For owners\nTo push to production: `git subtree push --prefix dist origin gh-pages`.\n\nThis is not an official Google product.\n" }, { "path": "analytics.js", "content": "(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){\n(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),\nm=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)\n})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');\n\nvar ANALYTICS_ID = 'Add your own analytics ID here';\nga('create', ANALYTICS_ID, 'auto');" }, { "path": "index.html", "content": "\n\n\n\n \n \n \n \n\n \n \n \n \n \n\n \n \n \n \n \n \n \n\n \n A Neural Network Playground\n \n \n \n\n\n \n \n \n \n \n \n \n \n
\n

Tinker With a Neural Network Right Here in Your Browser.
Don’t Worry, You Can’t Break It. We Promise.

\n
\n\n \n
\n
\n
\n \n \n \n
\n
\n Epoch\n \n
\n
\n \n
\n \n
\n
\n
\n \n
\n \n
\n
\n
\n \n
\n \n
\n
\n
\n \n
\n \n
\n
\n
\n \n
\n \n
\n
\n
\n
\n\n \n
\n\n \n
\n

\n Data\n

\n
\n

Which dataset do you want to use?

\n
\n
\n \n
\n
\n \n
\n
\n \n
\n
\n \n
\n
\n \n
\n
\n \n
\n
\n
\n
\n
\n \n

\n \n

\n
\n
\n \n

\n \n

\n
\n
\n \n

\n \n

\n
\n \n
\n
\n\n \n
\n

Features

\n

Which properties do you want to feed in?

\n
\n \n \n \n \n \n \n \n \n
\n
Click anywhere to edit.
\n
Weight/Bias is 0.2.
\n
\n
\n \n \n \n \n \n \n \n \n
\n This is the output from one neuron. Hover to see it larger.\n
\n
\n
\n \n \n \n \n \n \n \n \n
\n The outputs are mixed with varying weights, shown by the thickness of the lines.\n
\n
\n
\n
\n\n \n
\n

\n
\n \n \n
\n \n \n

\n
\n
\n\n \n
\n

Output

\n
\n
\n Test loss\n
\n
\n
\n Training loss\n
\n
\n
\n
\n
\n
\n
\n\n \n
\n Colors shows data, neuron and weight values.\n
\n \n \n \n \n \n \n \n \n \n \n \n \n
\n
\n
\n \n \n
\n
\n
\n\n
\n\n \n
\n \n \n
\n \n\n
\n
\n

Um, What Is a Neural Network?

\n

It’s a technique for building a computer program that learns from data. It is based very loosely on how we think the human brain works. First, a collection of software “neurons” are created and connected together, allowing them to send messages to each other. Next, the network is asked to solve a problem, which it attempts to do over and over, each time strengthening the connections that lead to success and diminishing those that lead to failure. For a more detailed introduction to neural networks, Michael Nielsen’s Neural Networks and Deep Learning is a good place to start. For a more technical overview, try Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville.

\n
\n\n
\n

This Is Cool, Can I Repurpose It?

\n

Please do! We’ve open sourced it on GitHub with the hope that it can make neural networks a little more accessible and easier to learn. You’re free to use it in any way that follows our Apache License. And if you have any suggestions for additions or changes, please let us know.

\n

We’ve also provided some controls below to enable you tailor the playground to a specific topic or lesson. Just choose which features you’d like to be visible below then save this link, or refresh the page.

\n
\n
\n\n
\n

What Do All the Colors Mean?

\n

Orange and blue are used throughout the visualization in slightly different ways, but in general orange shows negative values while blue shows positive values.

\n

The data points (represented by small circles) are initially colored orange or blue, which correspond to positive one and negative one.

\n

In the hidden layers, the lines are colored by the weights of the connections between neurons. Blue shows a positive weight, which means the network is using that output of the neuron as given. An orange line shows that the network is assiging a negative weight.

\n

In the output layer, the dots are colored orange or blue depending on their original values. The background color shows what the network is predicting for a particular area. The intensity of the color shows how confident that prediction is.

\n
\n\n
\n

What Library Are You Using?

\n

We wrote a tiny neural network library\n that meets the demands of this educational visualization. For real-world applications, consider the\n TensorFlow library.\n

\n
\n\n
\n

Credits

\n

\n This was created by Daniel Smilkov and Shan Carter.\n This is a continuation of many people’s previous work — most notably Andrej Karpathy’s convnet.js demo\n and Chris Olah’s articles about neural networks.\n Many thanks also to D. Sculley for help with the original idea and to Fernanda Viégas and Martin Wattenberg and the rest of the\n Big Picture and Google Brain teams for feedback and guidance.\n

\n
\n
\n\n \n \n \n \n \n\n\n" }, { "path": "package.json", "content": "{\n \"name\": \"deep-playground-prototype\",\n \"version\": \"2016.3.10\",\n \"description\": \"\",\n \"private\": true,\n \"scripts\": {\n \"clean\": \"rimraf dist\",\n \"start\": \"npm run serve-watch\",\n \"prep\": \"copyfiles analytics.js dist && concat node_modules/material-design-lite/material.min.js node_modules/seedrandom/seedrandom.min.js > dist/lib.js\",\n \"build-css\": \"concat node_modules/material-design-lite/material.min.css styles.css > dist/bundle.css\",\n \"watch-css\": \"concat node_modules/material-design-lite/material.min.css styles.css -o dist/bundle.css\",\n \"build-html\": \"copyfiles index.html dist\",\n \"watch-html\": \"concat index.html -o dist/index.html\",\n \"build-js\": \"browserify src/playground.ts -p [tsify] | uglifyjs -c > dist/bundle.js\",\n \"watch-js\": \"watchify src/playground.ts -p [tsify] -v --debug -o dist/bundle.js\",\n \"build\": \"npm run prep && npm run build-js && npm run build-css && npm run build-html\",\n \"watch\": \"npm run prep && concurrently \\\"npm run watch-js\\\" \\\"npm run watch-css\\\" \\\"npm run watch-html\\\"\",\n \"serve\": \"npx serve dist/\",\n \"serve-watch\": \"concurrently \\\"npx serve dist/\\\" \\\"npm run watch\\\"\"\n },\n \"devDependencies\": {\n \"@types/d3\": \"^3.5.34\",\n \"concat\": \"^1.0.3\",\n \"concurrently\": \"3.1.0\",\n \"copyfiles\": \"1.0.0\",\n \"rimraf\": \"2.5.4\",\n \"serve\": \"^11.3.0\",\n \"tsify\": \"^4.0.0\",\n \"typescript\": \"^2.9\",\n \"uglify-js\": \"^2.8.29\",\n \"watchify\": \"^4.0.0\"\n },\n \"dependencies\": {\n \"d3\": \"^3.5.16\",\n \"material-design-lite\": \"^1.3.0\",\n \"seedrandom\": \"^2.4.3\"\n }\n}\n" }, { "path": "src/dataset.ts", "content": "/* Copyright 2016 Google Inc. All Rights Reserved.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n==============================================================================*/\n\nimport * as d3 from 'd3';\n\n/**\n * A two dimensional example: x and y coordinates with the label.\n */\nexport type Example2D = {\n x: number,\n y: number,\n label: number\n};\n\ntype Point = {\n x: number,\n y: number\n};\n\n/**\n * Shuffles the array using Fisher-Yates algorithm. Uses the seedrandom\n * library as the random generator.\n */\nexport function shuffle(array: any[]): void {\n let counter = array.length;\n let temp = 0;\n let index = 0;\n // While there are elements in the array\n while (counter > 0) {\n // Pick a random index\n index = Math.floor(Math.random() * counter);\n // Decrease counter by 1\n counter--;\n // And swap the last element with it\n temp = array[counter];\n array[counter] = array[index];\n array[index] = temp;\n }\n}\n\nexport type DataGenerator = (numSamples: number, noise: number) => Example2D[];\n\nexport function classifyTwoGaussData(numSamples: number, noise: number):\n Example2D[] {\n let points: Example2D[] = [];\n\n let varianceScale = d3.scale.linear().domain([0, .5]).range([0.5, 4]);\n let variance = varianceScale(noise);\n\n function genGauss(cx: number, cy: number, label: number) {\n for (let i = 0; i < numSamples / 2; i++) {\n let x = normalRandom(cx, variance);\n let y = normalRandom(cy, variance);\n points.push({x, y, label});\n }\n }\n\n genGauss(2, 2, 1); // Gaussian with positive examples.\n genGauss(-2, -2, -1); // Gaussian with negative examples.\n return points;\n}\n\nexport function regressPlane(numSamples: number, noise: number):\n Example2D[] {\n let radius = 6;\n let labelScale = d3.scale.linear()\n .domain([-10, 10])\n .range([-1, 1]);\n let getLabel = (x, y) => labelScale(x + y);\n\n let points: Example2D[] = [];\n for (let i = 0; i < numSamples; i++) {\n let x = randUniform(-radius, radius);\n let y = randUniform(-radius, radius);\n let noiseX = randUniform(-radius, radius) * noise;\n let noiseY = randUniform(-radius, radius) * noise;\n let label = getLabel(x + noiseX, y + noiseY);\n points.push({x, y, label});\n }\n return points;\n}\n\nexport function regressGaussian(numSamples: number, noise: number):\n Example2D[] {\n let points: Example2D[] = [];\n\n let labelScale = d3.scale.linear()\n .domain([0, 2])\n .range([1, 0])\n .clamp(true);\n\n let gaussians = [\n [-4, 2.5, 1],\n [0, 2.5, -1],\n [4, 2.5, 1],\n [-4, -2.5, -1],\n [0, -2.5, 1],\n [4, -2.5, -1]\n ];\n\n function getLabel(x, y) {\n // Choose the one that is maximum in abs value.\n let label = 0;\n gaussians.forEach(([cx, cy, sign]) => {\n let newLabel = sign * labelScale(dist({x, y}, {x: cx, y: cy}));\n if (Math.abs(newLabel) > Math.abs(label)) {\n label = newLabel;\n }\n });\n return label;\n }\n let radius = 6;\n for (let i = 0; i < numSamples; i++) {\n let x = randUniform(-radius, radius);\n let y = randUniform(-radius, radius);\n let noiseX = randUniform(-radius, radius) * noise;\n let noiseY = randUniform(-radius, radius) * noise;\n let label = getLabel(x + noiseX, y + noiseY);\n points.push({x, y, label});\n };\n return points;\n}\n\nexport function classifySpiralData(numSamples: number, noise: number):\n Example2D[] {\n let points: Example2D[] = [];\n let n = numSamples / 2;\n\n function genSpiral(deltaT: number, label: number) {\n for (let i = 0; i < n; i++) {\n let r = i / n * 5;\n let t = 1.75 * i / n * 2 * Math.PI + deltaT;\n let x = r * Math.sin(t) + randUniform(-1, 1) * noise;\n let y = r * Math.cos(t) + randUniform(-1, 1) * noise;\n points.push({x, y, label});\n }\n }\n\n genSpiral(0, 1); // Positive examples.\n genSpiral(Math.PI, -1); // Negative examples.\n return points;\n}\n\nexport function classifyCircleData(numSamples: number, noise: number):\n Example2D[] {\n let points: Example2D[] = [];\n let radius = 5;\n function getCircleLabel(p: Point, center: Point) {\n return (dist(p, center) < (radius * 0.5)) ? 1 : -1;\n }\n\n // Generate positive points inside the circle.\n for (let i = 0; i < numSamples / 2; i++) {\n let r = randUniform(0, radius * 0.5);\n let angle = randUniform(0, 2 * Math.PI);\n let x = r * Math.sin(angle);\n let y = r * Math.cos(angle);\n let noiseX = randUniform(-radius, radius) * noise;\n let noiseY = randUniform(-radius, radius) * noise;\n let label = getCircleLabel({x: x + noiseX, y: y + noiseY}, {x: 0, y: 0});\n points.push({x, y, label});\n }\n\n // Generate negative points outside the circle.\n for (let i = 0; i < numSamples / 2; i++) {\n let r = randUniform(radius * 0.7, radius);\n let angle = randUniform(0, 2 * Math.PI);\n let x = r * Math.sin(angle);\n let y = r * Math.cos(angle);\n let noiseX = randUniform(-radius, radius) * noise;\n let noiseY = randUniform(-radius, radius) * noise;\n let label = getCircleLabel({x: x + noiseX, y: y + noiseY}, {x: 0, y: 0});\n points.push({x, y, label});\n }\n return points;\n}\n\nexport function classifyXORData(numSamples: number, noise: number):\n Example2D[] {\n function getXORLabel(p: Point) { return p.x * p.y >= 0 ? 1 : -1; }\n\n let points: Example2D[] = [];\n for (let i = 0; i < numSamples; i++) {\n let x = randUniform(-5, 5);\n let padding = 0.3;\n x += x > 0 ? padding : -padding; // Padding.\n let y = randUniform(-5, 5);\n y += y > 0 ? padding : -padding;\n let noiseX = randUniform(-5, 5) * noise;\n let noiseY = randUniform(-5, 5) * noise;\n let label = getXORLabel({x: x + noiseX, y: y + noiseY});\n points.push({x, y, label});\n }\n return points;\n}\n\n/**\n * Returns a sample from a uniform [a, b] distribution.\n * Uses the seedrandom library as the random generator.\n */\nfunction randUniform(a: number, b: number) {\n return Math.random() * (b - a) + a;\n}\n\n/**\n * Samples from a normal distribution. Uses the seedrandom library as the\n * random generator.\n *\n * @param mean The mean. Default is 0.\n * @param variance The variance. Default is 1.\n */\nfunction normalRandom(mean = 0, variance = 1): number {\n let v1: number, v2: number, s: number;\n do {\n v1 = 2 * Math.random() - 1;\n v2 = 2 * Math.random() - 1;\n s = v1 * v1 + v2 * v2;\n } while (s > 1);\n\n let result = Math.sqrt(-2 * Math.log(s) / s) * v1;\n return mean + Math.sqrt(variance) * result;\n}\n\n/** Returns the eucledian distance between two points in space. */\nfunction dist(a: Point, b: Point): number {\n let dx = a.x - b.x;\n let dy = a.y - b.y;\n return Math.sqrt(dx * dx + dy * dy);\n}\n" }, { "path": "src/heatmap.ts", "content": "/* Copyright 2016 Google Inc. All Rights Reserved.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n==============================================================================*/\n\nimport {Example2D} from \"./dataset\";\nimport * as d3 from 'd3';\n\nexport interface HeatMapSettings {\n [key: string]: any;\n showAxes?: boolean;\n noSvg?: boolean;\n}\n\n/** Number of different shades (colors) when drawing a gradient heatmap */\nconst NUM_SHADES = 30;\n\n/**\n * Draws a heatmap using canvas. Used for showing the learned decision\n * boundary of the classification algorithm. Can also draw data points\n * using an svg overlayed on top of the canvas heatmap.\n */\nexport class HeatMap {\n private settings: HeatMapSettings = {\n showAxes: false,\n noSvg: false\n };\n private xScale;\n private yScale;\n private numSamples: number;\n private color;\n private canvas;\n private svg;\n\n constructor(\n width: number, numSamples: number, xDomain: [number, number],\n yDomain: [number, number], container,\n userSettings?: HeatMapSettings) {\n this.numSamples = numSamples;\n let height = width;\n let padding = userSettings.showAxes ? 20 : 0;\n\n if (userSettings != null) {\n // overwrite the defaults with the user-specified settings.\n for (let prop in userSettings) {\n this.settings[prop] = userSettings[prop];\n }\n }\n\n this.xScale = d3.scale.linear()\n .domain(xDomain)\n .range([0, width - 2 * padding]);\n\n this.yScale = d3.scale.linear()\n .domain(yDomain)\n .range([height - 2 * padding, 0]);\n\n // Get a range of colors.\n let tmpScale = d3.scale.linear()\n .domain([0, .5, 1])\n .range([\"#f59322\", \"#e8eaeb\", \"#0877bd\"])\n .clamp(true);\n // Due to numerical error, we need to specify\n // d3.range(0, end + small_epsilon, step)\n // in order to guarantee that we will have end/step entries with\n // the last element being equal to end.\n let colors = d3.range(0, 1 + 1E-9, 1 / NUM_SHADES).map(a => {\n return tmpScale(a);\n });\n this.color = d3.scale.quantize()\n .domain([-1, 1])\n .range(colors);\n\n container = container.append(\"div\")\n .style({\n width: `${width}px`,\n height: `${height}px`,\n position: \"relative\",\n top: `-${padding}px`,\n left: `-${padding}px`\n });\n this.canvas = container.append(\"canvas\")\n .attr(\"width\", numSamples)\n .attr(\"height\", numSamples)\n .style(\"width\", (width - 2 * padding) + \"px\")\n .style(\"height\", (height - 2 * padding) + \"px\")\n .style(\"position\", \"absolute\")\n .style(\"top\", `${padding}px`)\n .style(\"left\", `${padding}px`);\n\n if (!this.settings.noSvg) {\n this.svg = container.append(\"svg\").attr({\n \"width\": width,\n \"height\": height\n }).style({\n // Overlay the svg on top of the canvas.\n \"position\": \"absolute\",\n \"left\": \"0\",\n \"top\": \"0\"\n }).append(\"g\")\n .attr(\"transform\", `translate(${padding},${padding})`);\n\n this.svg.append(\"g\").attr(\"class\", \"train\");\n this.svg.append(\"g\").attr(\"class\", \"test\");\n }\n\n if (this.settings.showAxes) {\n let xAxis = d3.svg.axis()\n .scale(this.xScale)\n .orient(\"bottom\");\n\n let yAxis = d3.svg.axis()\n .scale(this.yScale)\n .orient(\"right\");\n\n this.svg.append(\"g\")\n .attr(\"class\", \"x axis\")\n .attr(\"transform\", `translate(0,${height - 2 * padding})`)\n .call(xAxis);\n\n this.svg.append(\"g\")\n .attr(\"class\", \"y axis\")\n .attr(\"transform\", \"translate(\" + (width - 2 * padding) + \",0)\")\n .call(yAxis);\n }\n }\n\n updateTestPoints(points: Example2D[]): void {\n if (this.settings.noSvg) {\n throw Error(\"Can't add points since noSvg=true\");\n }\n this.updateCircles(this.svg.select(\"g.test\"), points);\n }\n\n updatePoints(points: Example2D[]): void {\n if (this.settings.noSvg) {\n throw Error(\"Can't add points since noSvg=true\");\n }\n this.updateCircles(this.svg.select(\"g.train\"), points);\n }\n\n updateBackground(data: number[][], discretize: boolean): void {\n let dx = data[0].length;\n let dy = data.length;\n\n if (dx !== this.numSamples || dy !== this.numSamples) {\n throw new Error(\n \"The provided data matrix must be of size \" +\n \"numSamples X numSamples\");\n }\n\n // Compute the pixel colors; scaled by CSS.\n let context = (this.canvas.node() as HTMLCanvasElement).getContext(\"2d\");\n let image = context.createImageData(dx, dy);\n\n for (let y = 0, p = -1; y < dy; ++y) {\n for (let x = 0; x < dx; ++x) {\n let value = data[x][y];\n if (discretize) {\n value = (value >= 0 ? 1 : -1);\n }\n let c = d3.rgb(this.color(value));\n image.data[++p] = c.r;\n image.data[++p] = c.g;\n image.data[++p] = c.b;\n image.data[++p] = 160;\n }\n }\n context.putImageData(image, 0, 0);\n }\n\n private updateCircles(container, points: Example2D[]) {\n // Keep only points that are inside the bounds.\n let xDomain = this.xScale.domain();\n let yDomain = this.yScale.domain();\n points = points.filter(p => {\n return p.x >= xDomain[0] && p.x <= xDomain[1]\n && p.y >= yDomain[0] && p.y <= yDomain[1];\n });\n\n // Attach data to initially empty selection.\n let selection = container.selectAll(\"circle\").data(points);\n\n // Insert elements to match length of points array.\n selection.enter().append(\"circle\").attr(\"r\", 3);\n\n // Update points to be in the correct position.\n selection\n .attr({\n cx: (d: Example2D) => this.xScale(d.x),\n cy: (d: Example2D) => this.yScale(d.y),\n })\n .style(\"fill\", d => this.color(d.label));\n\n // Remove points if the length has gone down.\n selection.exit().remove();\n }\n} // Close class HeatMap.\n\nexport function reduceMatrix(matrix: number[][], factor: number): number[][] {\n if (matrix.length !== matrix[0].length) {\n throw new Error(\"The provided matrix must be a square matrix\");\n }\n if (matrix.length % factor !== 0) {\n throw new Error(\"The width/height of the matrix must be divisible by \" +\n \"the reduction factor\");\n }\n let result: number[][] = new Array(matrix.length / factor);\n for (let i = 0; i < matrix.length; i += factor) {\n result[i / factor] = new Array(matrix.length / factor);\n for (let j = 0; j < matrix.length; j += factor) {\n let avg = 0;\n // Sum all the values in the neighborhood.\n for (let k = 0; k < factor; k++) {\n for (let l = 0; l < factor; l++) {\n avg += matrix[i + k][j + l];\n }\n }\n avg /= (factor * factor);\n result[i / factor][j / factor] = avg;\n }\n }\n return result;\n}\n" }, { "path": "src/linechart.ts", "content": "/* Copyright 2016 Google Inc. All Rights Reserved.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n==============================================================================*/\n\nimport * as d3 from 'd3';\n\ntype DataPoint = {\n x: number;\n y: number[];\n};\n\n/**\n * A multi-series line chart that allows you to append new data points\n * as data becomes available.\n */\nexport class AppendingLineChart {\n private numLines: number;\n private data: DataPoint[] = [];\n private svg;\n private xScale;\n private yScale;\n private paths;\n private lineColors: string[];\n\n private minY = Number.MAX_VALUE;\n private maxY = Number.MIN_VALUE;\n\n constructor(container, lineColors: string[]) {\n this.lineColors = lineColors;\n this.numLines = lineColors.length;\n let node = container.node() as HTMLElement;\n let totalWidth = node.offsetWidth;\n let totalHeight = node.offsetHeight;\n let margin = {top: 2, right: 0, bottom: 2, left: 2};\n let width = totalWidth - margin.left - margin.right;\n let height = totalHeight - margin.top - margin.bottom;\n\n this.xScale = d3.scale.linear()\n .domain([0, 0])\n .range([0, width]);\n\n this.yScale = d3.scale.linear()\n .domain([0, 0])\n .range([height, 0]);\n\n this.svg = container.append(\"svg\")\n .attr(\"width\", width + margin.left + margin.right)\n .attr(\"height\", height + margin.top + margin.bottom)\n .append(\"g\")\n .attr(\"transform\", `translate(${margin.left},${margin.top})`);\n\n this.paths = new Array(this.numLines);\n for (let i = 0; i < this.numLines; i++) {\n this.paths[i] = this.svg.append(\"path\")\n .attr(\"class\", \"line\")\n .style({\n \"fill\": \"none\",\n \"stroke\": lineColors[i],\n \"stroke-width\": \"1.5px\"\n });\n }\n }\n\n reset() {\n this.data = [];\n this.redraw();\n this.minY = Number.MAX_VALUE;\n this.maxY = Number.MIN_VALUE;\n }\n\n addDataPoint(dataPoint: number[]) {\n if (dataPoint.length !== this.numLines) {\n throw Error(\"Length of dataPoint must equal number of lines\");\n }\n dataPoint.forEach(y => {\n this.minY = Math.min(this.minY, y);\n this.maxY = Math.max(this.maxY, y);\n });\n\n this.data.push({x: this.data.length + 1, y: dataPoint});\n this.redraw();\n }\n\n private redraw() {\n // Adjust the x and y domain.\n this.xScale.domain([1, this.data.length]);\n this.yScale.domain([this.minY, this.maxY]);\n // Adjust all the elements (lines).\n let getPathMap = (lineIndex: number) => {\n return d3.svg.line<{x: number, y:number}>()\n .x(d => this.xScale(d.x))\n .y(d => this.yScale(d.y[lineIndex]));\n };\n for (let i = 0; i < this.numLines; i++) {\n this.paths[i].datum(this.data).attr(\"d\", getPathMap(i));\n }\n }\n}\n" }, { "path": "src/nn.ts", "content": "/* Copyright 2016 Google Inc. All Rights Reserved.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n==============================================================================*/\n\n/**\n * A node in a neural network. Each node has a state\n * (total input, output, and their respectively derivatives) which changes\n * after every forward and back propagation run.\n */\nexport class Node {\n id: string;\n /** List of input links. */\n inputLinks: Link[] = [];\n bias = 0.1;\n /** List of output links. */\n outputs: Link[] = [];\n totalInput: number;\n output: number;\n /** Error derivative with respect to this node's output. */\n outputDer = 0;\n /** Error derivative with respect to this node's total input. */\n inputDer = 0;\n /**\n * Accumulated error derivative with respect to this node's total input since\n * the last update. This derivative equals dE/db where b is the node's\n * bias term.\n */\n accInputDer = 0;\n /**\n * Number of accumulated err. derivatives with respect to the total input\n * since the last update.\n */\n numAccumulatedDers = 0;\n /** Activation function that takes total input and returns node's output */\n activation: ActivationFunction;\n\n /**\n * Creates a new node with the provided id and activation function.\n */\n constructor(id: string, activation: ActivationFunction, initZero?: boolean) {\n this.id = id;\n this.activation = activation;\n if (initZero) {\n this.bias = 0;\n }\n }\n\n /** Recomputes the node's output and returns it. */\n updateOutput(): number {\n // Stores total input into the node.\n this.totalInput = this.bias;\n for (let j = 0; j < this.inputLinks.length; j++) {\n let link = this.inputLinks[j];\n this.totalInput += link.weight * link.source.output;\n }\n this.output = this.activation.output(this.totalInput);\n return this.output;\n }\n}\n\n/**\n * An error function and its derivative.\n */\nexport interface ErrorFunction {\n error: (output: number, target: number) => number;\n der: (output: number, target: number) => number;\n}\n\n/** A node's activation function and its derivative. */\nexport interface ActivationFunction {\n output: (input: number) => number;\n der: (input: number) => number;\n}\n\n/** Function that computes a penalty cost for a given weight in the network. */\nexport interface RegularizationFunction {\n output: (weight: number) => number;\n der: (weight: number) => number;\n}\n\n/** Built-in error functions */\nexport class Errors {\n public static SQUARE: ErrorFunction = {\n error: (output: number, target: number) =>\n 0.5 * Math.pow(output - target, 2),\n der: (output: number, target: number) => output - target\n };\n}\n\n/** Polyfill for TANH */\n(Math as any).tanh = (Math as any).tanh || function(x) {\n if (x === Infinity) {\n return 1;\n } else if (x === -Infinity) {\n return -1;\n } else {\n let e2x = Math.exp(2 * x);\n return (e2x - 1) / (e2x + 1);\n }\n};\n\n/** Built-in activation functions */\nexport class Activations {\n public static TANH: ActivationFunction = {\n output: x => (Math as any).tanh(x),\n der: x => {\n let output = Activations.TANH.output(x);\n return 1 - output * output;\n }\n };\n public static RELU: ActivationFunction = {\n output: x => Math.max(0, x),\n der: x => x <= 0 ? 0 : 1\n };\n public static SIGMOID: ActivationFunction = {\n output: x => 1 / (1 + Math.exp(-x)),\n der: x => {\n let output = Activations.SIGMOID.output(x);\n return output * (1 - output);\n }\n };\n public static LINEAR: ActivationFunction = {\n output: x => x,\n der: x => 1\n };\n}\n\n/** Build-in regularization functions */\nexport class RegularizationFunction {\n public static L1: RegularizationFunction = {\n output: w => Math.abs(w),\n der: w => w < 0 ? -1 : (w > 0 ? 1 : 0)\n };\n public static L2: RegularizationFunction = {\n output: w => 0.5 * w * w,\n der: w => w\n };\n}\n\n/**\n * A link in a neural network. Each link has a weight and a source and\n * destination node. Also it has an internal state (error derivative\n * with respect to a particular input) which gets updated after\n * a run of back propagation.\n */\nexport class Link {\n id: string;\n source: Node;\n dest: Node;\n weight = Math.random() - 0.5;\n isDead = false;\n /** Error derivative with respect to this weight. */\n errorDer = 0;\n /** Accumulated error derivative since the last update. */\n accErrorDer = 0;\n /** Number of accumulated derivatives since the last update. */\n numAccumulatedDers = 0;\n regularization: RegularizationFunction;\n\n /**\n * Constructs a link in the neural network initialized with random weight.\n *\n * @param source The source node.\n * @param dest The destination node.\n * @param regularization The regularization function that computes the\n * penalty for this weight. If null, there will be no regularization.\n */\n constructor(source: Node, dest: Node,\n regularization: RegularizationFunction, initZero?: boolean) {\n this.id = source.id + \"-\" + dest.id;\n this.source = source;\n this.dest = dest;\n this.regularization = regularization;\n if (initZero) {\n this.weight = 0;\n }\n }\n}\n\n/**\n * Builds a neural network.\n *\n * @param networkShape The shape of the network. E.g. [1, 2, 3, 1] means\n * the network will have one input node, 2 nodes in first hidden layer,\n * 3 nodes in second hidden layer and 1 output node.\n * @param activation The activation function of every hidden node.\n * @param outputActivation The activation function for the output nodes.\n * @param regularization The regularization function that computes a penalty\n * for a given weight (parameter) in the network. If null, there will be\n * no regularization.\n * @param inputIds List of ids for the input nodes.\n */\nexport function buildNetwork(\n networkShape: number[], activation: ActivationFunction,\n outputActivation: ActivationFunction,\n regularization: RegularizationFunction,\n inputIds: string[], initZero?: boolean): Node[][] {\n let numLayers = networkShape.length;\n let id = 1;\n /** List of layers, with each layer being a list of nodes. */\n let network: Node[][] = [];\n for (let layerIdx = 0; layerIdx < numLayers; layerIdx++) {\n let isOutputLayer = layerIdx === numLayers - 1;\n let isInputLayer = layerIdx === 0;\n let currentLayer: Node[] = [];\n network.push(currentLayer);\n let numNodes = networkShape[layerIdx];\n for (let i = 0; i < numNodes; i++) {\n let nodeId = id.toString();\n if (isInputLayer) {\n nodeId = inputIds[i];\n } else {\n id++;\n }\n let node = new Node(nodeId,\n isOutputLayer ? outputActivation : activation, initZero);\n currentLayer.push(node);\n if (layerIdx >= 1) {\n // Add links from nodes in the previous layer to this node.\n for (let j = 0; j < network[layerIdx - 1].length; j++) {\n let prevNode = network[layerIdx - 1][j];\n let link = new Link(prevNode, node, regularization, initZero);\n prevNode.outputs.push(link);\n node.inputLinks.push(link);\n }\n }\n }\n }\n return network;\n}\n\n/**\n * Runs a forward propagation of the provided input through the provided\n * network. This method modifies the internal state of the network - the\n * total input and output of each node in the network.\n *\n * @param network The neural network.\n * @param inputs The input array. Its length should match the number of input\n * nodes in the network.\n * @return The final output of the network.\n */\nexport function forwardProp(network: Node[][], inputs: number[]): number {\n let inputLayer = network[0];\n if (inputs.length !== inputLayer.length) {\n throw new Error(\"The number of inputs must match the number of nodes in\" +\n \" the input layer\");\n }\n // Update the input layer.\n for (let i = 0; i < inputLayer.length; i++) {\n let node = inputLayer[i];\n node.output = inputs[i];\n }\n for (let layerIdx = 1; layerIdx < network.length; layerIdx++) {\n let currentLayer = network[layerIdx];\n // Update all the nodes in this layer.\n for (let i = 0; i < currentLayer.length; i++) {\n let node = currentLayer[i];\n node.updateOutput();\n }\n }\n return network[network.length - 1][0].output;\n}\n\n/**\n * Runs a backward propagation using the provided target and the\n * computed output of the previous call to forward propagation.\n * This method modifies the internal state of the network - the error\n * derivatives with respect to each node, and each weight\n * in the network.\n */\nexport function backProp(network: Node[][], target: number,\n errorFunc: ErrorFunction): void {\n // The output node is a special case. We use the user-defined error\n // function for the derivative.\n let outputNode = network[network.length - 1][0];\n outputNode.outputDer = errorFunc.der(outputNode.output, target);\n\n // Go through the layers backwards.\n for (let layerIdx = network.length - 1; layerIdx >= 1; layerIdx--) {\n let currentLayer = network[layerIdx];\n // Compute the error derivative of each node with respect to:\n // 1) its total input\n // 2) each of its input weights.\n for (let i = 0; i < currentLayer.length; i++) {\n let node = currentLayer[i];\n node.inputDer = node.outputDer * node.activation.der(node.totalInput);\n node.accInputDer += node.inputDer;\n node.numAccumulatedDers++;\n }\n\n // Error derivative with respect to each weight coming into the node.\n for (let i = 0; i < currentLayer.length; i++) {\n let node = currentLayer[i];\n for (let j = 0; j < node.inputLinks.length; j++) {\n let link = node.inputLinks[j];\n if (link.isDead) {\n continue;\n }\n link.errorDer = node.inputDer * link.source.output;\n link.accErrorDer += link.errorDer;\n link.numAccumulatedDers++;\n }\n }\n if (layerIdx === 1) {\n continue;\n }\n let prevLayer = network[layerIdx - 1];\n for (let i = 0; i < prevLayer.length; i++) {\n let node = prevLayer[i];\n // Compute the error derivative with respect to each node's output.\n node.outputDer = 0;\n for (let j = 0; j < node.outputs.length; j++) {\n let output = node.outputs[j];\n node.outputDer += output.weight * output.dest.inputDer;\n }\n }\n }\n}\n\n/**\n * Updates the weights of the network using the previously accumulated error\n * derivatives.\n */\nexport function updateWeights(network: Node[][], learningRate: number,\n regularizationRate: number) {\n for (let layerIdx = 1; layerIdx < network.length; layerIdx++) {\n let currentLayer = network[layerIdx];\n for (let i = 0; i < currentLayer.length; i++) {\n let node = currentLayer[i];\n // Update the node's bias.\n if (node.numAccumulatedDers > 0) {\n node.bias -= learningRate * node.accInputDer / node.numAccumulatedDers;\n node.accInputDer = 0;\n node.numAccumulatedDers = 0;\n }\n // Update the weights coming into this node.\n for (let j = 0; j < node.inputLinks.length; j++) {\n let link = node.inputLinks[j];\n if (link.isDead) {\n continue;\n }\n let regulDer = link.regularization ?\n link.regularization.der(link.weight) : 0;\n if (link.numAccumulatedDers > 0) {\n // Update the weight based on dE/dw.\n link.weight = link.weight -\n (learningRate / link.numAccumulatedDers) * link.accErrorDer;\n // Further update the weight based on regularization.\n let newLinkWeight = link.weight -\n (learningRate * regularizationRate) * regulDer;\n if (link.regularization === RegularizationFunction.L1 &&\n link.weight * newLinkWeight < 0) {\n // The weight crossed 0 due to the regularization term. Set it to 0.\n link.weight = 0;\n link.isDead = true;\n } else {\n link.weight = newLinkWeight;\n }\n link.accErrorDer = 0;\n link.numAccumulatedDers = 0;\n }\n }\n }\n }\n}\n\n/** Iterates over every node in the network/ */\nexport function forEachNode(network: Node[][], ignoreInputs: boolean,\n accessor: (node: Node) => any) {\n for (let layerIdx = ignoreInputs ? 1 : 0;\n layerIdx < network.length;\n layerIdx++) {\n let currentLayer = network[layerIdx];\n for (let i = 0; i < currentLayer.length; i++) {\n let node = currentLayer[i];\n accessor(node);\n }\n }\n}\n\n/** Returns the output node in the network. */\nexport function getOutputNode(network: Node[][]) {\n return network[network.length - 1][0];\n}\n" }, { "path": "src/playground.ts", "content": "/* Copyright 2016 Google Inc. All Rights Reserved.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n==============================================================================*/\n\nimport * as nn from \"./nn\";\nimport {HeatMap, reduceMatrix} from \"./heatmap\";\nimport {\n State,\n datasets,\n regDatasets,\n activations,\n problems,\n regularizations,\n getKeyFromValue,\n Problem\n} from \"./state\";\nimport {Example2D, shuffle} from \"./dataset\";\nimport {AppendingLineChart} from \"./linechart\";\nimport * as d3 from 'd3';\n\nlet mainWidth;\n\n// More scrolling\nd3.select(\".more button\").on(\"click\", function() {\n let position = 800;\n d3.transition()\n .duration(1000)\n .tween(\"scroll\", scrollTween(position));\n});\n\nfunction scrollTween(offset) {\n return function() {\n let i = d3.interpolateNumber(window.pageYOffset ||\n document.documentElement.scrollTop, offset);\n return function(t) { scrollTo(0, i(t)); };\n };\n}\n\nconst RECT_SIZE = 30;\nconst BIAS_SIZE = 5;\nconst NUM_SAMPLES_CLASSIFY = 500;\nconst NUM_SAMPLES_REGRESS = 1200;\nconst DENSITY = 100;\n\nenum HoverType {\n BIAS, WEIGHT\n}\n\ninterface InputFeature {\n f: (x: number, y: number) => number;\n label?: string;\n}\n\nlet INPUTS: {[name: string]: InputFeature} = {\n \"x\": {f: (x, y) => x, label: \"X_1\"},\n \"y\": {f: (x, y) => y, label: \"X_2\"},\n \"xSquared\": {f: (x, y) => x * x, label: \"X_1^2\"},\n \"ySquared\": {f: (x, y) => y * y, label: \"X_2^2\"},\n \"xTimesY\": {f: (x, y) => x * y, label: \"X_1X_2\"},\n \"sinX\": {f: (x, y) => Math.sin(x), label: \"sin(X_1)\"},\n \"sinY\": {f: (x, y) => Math.sin(y), label: \"sin(X_2)\"},\n};\n\nlet HIDABLE_CONTROLS = [\n [\"Show test data\", \"showTestData\"],\n [\"Discretize output\", \"discretize\"],\n [\"Play button\", \"playButton\"],\n [\"Step button\", \"stepButton\"],\n [\"Reset button\", \"resetButton\"],\n [\"Learning rate\", \"learningRate\"],\n [\"Activation\", \"activation\"],\n [\"Regularization\", \"regularization\"],\n [\"Regularization rate\", \"regularizationRate\"],\n [\"Problem type\", \"problem\"],\n [\"Which dataset\", \"dataset\"],\n [\"Ratio train data\", \"percTrainData\"],\n [\"Noise level\", \"noise\"],\n [\"Batch size\", \"batchSize\"],\n [\"# of hidden layers\", \"numHiddenLayers\"],\n];\n\nclass Player {\n private timerIndex = 0;\n private isPlaying = false;\n private callback: (isPlaying: boolean) => void = null;\n\n /** Plays/pauses the player. */\n playOrPause() {\n if (this.isPlaying) {\n this.isPlaying = false;\n this.pause();\n } else {\n this.isPlaying = true;\n if (iter === 0) {\n simulationStarted();\n }\n this.play();\n }\n }\n\n onPlayPause(callback: (isPlaying: boolean) => void) {\n this.callback = callback;\n }\n\n play() {\n this.pause();\n this.isPlaying = true;\n if (this.callback) {\n this.callback(this.isPlaying);\n }\n this.start(this.timerIndex);\n }\n\n pause() {\n this.timerIndex++;\n this.isPlaying = false;\n if (this.callback) {\n this.callback(this.isPlaying);\n }\n }\n\n private start(localTimerIndex: number) {\n d3.timer(() => {\n if (localTimerIndex < this.timerIndex) {\n return true; // Done.\n }\n oneStep();\n return false; // Not done.\n }, 0);\n }\n}\n\nlet state = State.deserializeState();\n\n// Filter out inputs that are hidden.\nstate.getHiddenProps().forEach(prop => {\n if (prop in INPUTS) {\n delete INPUTS[prop];\n }\n});\n\nlet boundary: {[id: string]: number[][]} = {};\nlet selectedNodeId: string = null;\n// Plot the heatmap.\nlet xDomain: [number, number] = [-6, 6];\nlet heatMap =\n new HeatMap(300, DENSITY, xDomain, xDomain, d3.select(\"#heatmap\"),\n {showAxes: true});\nlet linkWidthScale = d3.scale.linear()\n .domain([0, 5])\n .range([1, 10])\n .clamp(true);\nlet colorScale = d3.scale.linear()\n .domain([-1, 0, 1])\n .range([\"#f59322\", \"#e8eaeb\", \"#0877bd\"])\n .clamp(true);\nlet iter = 0;\nlet trainData: Example2D[] = [];\nlet testData: Example2D[] = [];\nlet network: nn.Node[][] = null;\nlet lossTrain = 0;\nlet lossTest = 0;\nlet player = new Player();\nlet lineChart = new AppendingLineChart(d3.select(\"#linechart\"),\n [\"#777\", \"black\"]);\n\nfunction makeGUI() {\n d3.select(\"#reset-button\").on(\"click\", () => {\n reset();\n userHasInteracted();\n d3.select(\"#play-pause-button\");\n });\n\n d3.select(\"#play-pause-button\").on(\"click\", function () {\n // Change the button's content.\n userHasInteracted();\n player.playOrPause();\n });\n\n player.onPlayPause(isPlaying => {\n d3.select(\"#play-pause-button\").classed(\"playing\", isPlaying);\n });\n\n d3.select(\"#next-step-button\").on(\"click\", () => {\n player.pause();\n userHasInteracted();\n if (iter === 0) {\n simulationStarted();\n }\n oneStep();\n });\n\n d3.select(\"#data-regen-button\").on(\"click\", () => {\n generateData();\n parametersChanged = true;\n });\n\n let dataThumbnails = d3.selectAll(\"canvas[data-dataset]\");\n dataThumbnails.on(\"click\", function() {\n let newDataset = datasets[this.dataset.dataset];\n if (newDataset === state.dataset) {\n return; // No-op.\n }\n state.dataset = newDataset;\n dataThumbnails.classed(\"selected\", false);\n d3.select(this).classed(\"selected\", true);\n generateData();\n parametersChanged = true;\n reset();\n });\n\n let datasetKey = getKeyFromValue(datasets, state.dataset);\n // Select the dataset according to the current state.\n d3.select(`canvas[data-dataset=${datasetKey}]`)\n .classed(\"selected\", true);\n\n let regDataThumbnails = d3.selectAll(\"canvas[data-regDataset]\");\n regDataThumbnails.on(\"click\", function() {\n let newDataset = regDatasets[this.dataset.regdataset];\n if (newDataset === state.regDataset) {\n return; // No-op.\n }\n state.regDataset = newDataset;\n regDataThumbnails.classed(\"selected\", false);\n d3.select(this).classed(\"selected\", true);\n generateData();\n parametersChanged = true;\n reset();\n });\n\n let regDatasetKey = getKeyFromValue(regDatasets, state.regDataset);\n // Select the dataset according to the current state.\n d3.select(`canvas[data-regDataset=${regDatasetKey}]`)\n .classed(\"selected\", true);\n\n d3.select(\"#add-layers\").on(\"click\", () => {\n if (state.numHiddenLayers >= 6) {\n return;\n }\n state.networkShape[state.numHiddenLayers] = 2;\n state.numHiddenLayers++;\n parametersChanged = true;\n reset();\n });\n\n d3.select(\"#remove-layers\").on(\"click\", () => {\n if (state.numHiddenLayers <= 0) {\n return;\n }\n state.numHiddenLayers--;\n state.networkShape.splice(state.numHiddenLayers);\n parametersChanged = true;\n reset();\n });\n\n let showTestData = d3.select(\"#show-test-data\").on(\"change\", function() {\n state.showTestData = this.checked;\n state.serialize();\n userHasInteracted();\n heatMap.updateTestPoints(state.showTestData ? testData : []);\n });\n // Check/uncheck the checkbox according to the current state.\n showTestData.property(\"checked\", state.showTestData);\n\n let discretize = d3.select(\"#discretize\").on(\"change\", function() {\n state.discretize = this.checked;\n state.serialize();\n userHasInteracted();\n updateUI();\n });\n // Check/uncheck the checbox according to the current state.\n discretize.property(\"checked\", state.discretize);\n\n let percTrain = d3.select(\"#percTrainData\").on(\"input\", function() {\n state.percTrainData = this.value;\n d3.select(\"label[for='percTrainData'] .value\").text(this.value);\n generateData();\n parametersChanged = true;\n reset();\n });\n percTrain.property(\"value\", state.percTrainData);\n d3.select(\"label[for='percTrainData'] .value\").text(state.percTrainData);\n\n let noise = d3.select(\"#noise\").on(\"input\", function() {\n state.noise = this.value;\n d3.select(\"label[for='noise'] .value\").text(this.value);\n generateData();\n parametersChanged = true;\n reset();\n });\n let currentMax = parseInt(noise.property(\"max\"));\n if (state.noise > currentMax) {\n if (state.noise <= 80) {\n noise.property(\"max\", state.noise);\n } else {\n state.noise = 50;\n }\n } else if (state.noise < 0) {\n state.noise = 0;\n }\n noise.property(\"value\", state.noise);\n d3.select(\"label[for='noise'] .value\").text(state.noise);\n\n let batchSize = d3.select(\"#batchSize\").on(\"input\", function() {\n state.batchSize = this.value;\n d3.select(\"label[for='batchSize'] .value\").text(this.value);\n parametersChanged = true;\n reset();\n });\n batchSize.property(\"value\", state.batchSize);\n d3.select(\"label[for='batchSize'] .value\").text(state.batchSize);\n\n let activationDropdown = d3.select(\"#activations\").on(\"change\", function() {\n state.activation = activations[this.value];\n parametersChanged = true;\n reset();\n });\n activationDropdown.property(\"value\",\n getKeyFromValue(activations, state.activation));\n\n let learningRate = d3.select(\"#learningRate\").on(\"change\", function() {\n state.learningRate = +this.value;\n state.serialize();\n userHasInteracted();\n parametersChanged = true;\n });\n learningRate.property(\"value\", state.learningRate);\n\n let regularDropdown = d3.select(\"#regularizations\").on(\"change\",\n function() {\n state.regularization = regularizations[this.value];\n parametersChanged = true;\n reset();\n });\n regularDropdown.property(\"value\",\n getKeyFromValue(regularizations, state.regularization));\n\n let regularRate = d3.select(\"#regularRate\").on(\"change\", function() {\n state.regularizationRate = +this.value;\n parametersChanged = true;\n reset();\n });\n regularRate.property(\"value\", state.regularizationRate);\n\n let problem = d3.select(\"#problem\").on(\"change\", function() {\n state.problem = problems[this.value];\n generateData();\n drawDatasetThumbnails();\n parametersChanged = true;\n reset();\n });\n problem.property(\"value\", getKeyFromValue(problems, state.problem));\n\n // Add scale to the gradient color map.\n let x = d3.scale.linear().domain([-1, 1]).range([0, 144]);\n let xAxis = d3.svg.axis()\n .scale(x)\n .orient(\"bottom\")\n .tickValues([-1, 0, 1])\n .tickFormat(d3.format(\"d\"));\n d3.select(\"#colormap g.core\").append(\"g\")\n .attr(\"class\", \"x axis\")\n .attr(\"transform\", \"translate(0,10)\")\n .call(xAxis);\n\n // Listen for css-responsive changes and redraw the svg network.\n\n window.addEventListener(\"resize\", () => {\n let newWidth = document.querySelector(\"#main-part\")\n .getBoundingClientRect().width;\n if (newWidth !== mainWidth) {\n mainWidth = newWidth;\n drawNetwork(network);\n updateUI(true);\n }\n });\n\n // Hide the text below the visualization depending on the URL.\n if (state.hideText) {\n d3.select(\"#article-text\").style(\"display\", \"none\");\n d3.select(\"div.more\").style(\"display\", \"none\");\n d3.select(\"header\").style(\"display\", \"none\");\n }\n}\n\nfunction updateBiasesUI(network: nn.Node[][]) {\n nn.forEachNode(network, true, node => {\n d3.select(`rect#bias-${node.id}`).style(\"fill\", colorScale(node.bias));\n });\n}\n\nfunction updateWeightsUI(network: nn.Node[][], container) {\n for (let layerIdx = 1; layerIdx < network.length; layerIdx++) {\n let currentLayer = network[layerIdx];\n // Update all the nodes in this layer.\n for (let i = 0; i < currentLayer.length; i++) {\n let node = currentLayer[i];\n for (let j = 0; j < node.inputLinks.length; j++) {\n let link = node.inputLinks[j];\n container.select(`#link${link.source.id}-${link.dest.id}`)\n .style({\n \"stroke-dashoffset\": -iter / 3,\n \"stroke-width\": linkWidthScale(Math.abs(link.weight)),\n \"stroke\": colorScale(link.weight)\n })\n .datum(link);\n }\n }\n }\n}\n\nfunction drawNode(cx: number, cy: number, nodeId: string, isInput: boolean,\n container, node?: nn.Node) {\n let x = cx - RECT_SIZE / 2;\n let y = cy - RECT_SIZE / 2;\n\n let nodeGroup = container.append(\"g\")\n .attr({\n \"class\": \"node\",\n \"id\": `node${nodeId}`,\n \"transform\": `translate(${x},${y})`\n });\n\n // Draw the main rectangle.\n nodeGroup.append(\"rect\")\n .attr({\n x: 0,\n y: 0,\n width: RECT_SIZE,\n height: RECT_SIZE,\n });\n let activeOrNotClass = state[nodeId] ? \"active\" : \"inactive\";\n if (isInput) {\n let label = INPUTS[nodeId].label != null ?\n INPUTS[nodeId].label : nodeId;\n // Draw the input label.\n let text = nodeGroup.append(\"text\").attr({\n class: \"main-label\",\n x: -10,\n y: RECT_SIZE / 2, \"text-anchor\": \"end\"\n });\n if (/[_^]/.test(label)) {\n let myRe = /(.*?)([_^])(.)/g;\n let myArray;\n let lastIndex;\n while ((myArray = myRe.exec(label)) != null) {\n lastIndex = myRe.lastIndex;\n let prefix = myArray[1];\n let sep = myArray[2];\n let suffix = myArray[3];\n if (prefix) {\n text.append(\"tspan\").text(prefix);\n }\n text.append(\"tspan\")\n .attr(\"baseline-shift\", sep === \"_\" ? \"sub\" : \"super\")\n .style(\"font-size\", \"9px\")\n .text(suffix);\n }\n if (label.substring(lastIndex)) {\n text.append(\"tspan\").text(label.substring(lastIndex));\n }\n } else {\n text.append(\"tspan\").text(label);\n }\n nodeGroup.classed(activeOrNotClass, true);\n }\n if (!isInput) {\n // Draw the node's bias.\n nodeGroup.append(\"rect\")\n .attr({\n id: `bias-${nodeId}`,\n x: -BIAS_SIZE - 2,\n y: RECT_SIZE - BIAS_SIZE + 3,\n width: BIAS_SIZE,\n height: BIAS_SIZE,\n }).on(\"mouseenter\", function() {\n updateHoverCard(HoverType.BIAS, node, d3.mouse(container.node()));\n }).on(\"mouseleave\", function() {\n updateHoverCard(null);\n });\n }\n\n // Draw the node's canvas.\n let div = d3.select(\"#network\").insert(\"div\", \":first-child\")\n .attr({\n \"id\": `canvas-${nodeId}`,\n \"class\": \"canvas\"\n })\n .style({\n position: \"absolute\",\n left: `${x + 3}px`,\n top: `${y + 3}px`\n })\n .on(\"mouseenter\", function() {\n selectedNodeId = nodeId;\n div.classed(\"hovered\", true);\n nodeGroup.classed(\"hovered\", true);\n updateDecisionBoundary(network, false);\n heatMap.updateBackground(boundary[nodeId], state.discretize);\n })\n .on(\"mouseleave\", function() {\n selectedNodeId = null;\n div.classed(\"hovered\", false);\n nodeGroup.classed(\"hovered\", false);\n updateDecisionBoundary(network, false);\n heatMap.updateBackground(boundary[nn.getOutputNode(network).id],\n state.discretize);\n });\n if (isInput) {\n div.on(\"click\", function() {\n state[nodeId] = !state[nodeId];\n parametersChanged = true;\n reset();\n });\n div.style(\"cursor\", \"pointer\");\n }\n if (isInput) {\n div.classed(activeOrNotClass, true);\n }\n let nodeHeatMap = new HeatMap(RECT_SIZE, DENSITY / 10, xDomain,\n xDomain, div, {noSvg: true});\n div.datum({heatmap: nodeHeatMap, id: nodeId});\n\n}\n\n// Draw network\nfunction drawNetwork(network: nn.Node[][]): void {\n let svg = d3.select(\"#svg\");\n // Remove all svg elements.\n svg.select(\"g.core\").remove();\n // Remove all div elements.\n d3.select(\"#network\").selectAll(\"div.canvas\").remove();\n d3.select(\"#network\").selectAll(\"div.plus-minus-neurons\").remove();\n\n // Get the width of the svg container.\n let padding = 3;\n let co = d3.select(\".column.output\").node() as HTMLDivElement;\n let cf = d3.select(\".column.features\").node() as HTMLDivElement;\n let width = co.offsetLeft - cf.offsetLeft;\n svg.attr(\"width\", width);\n\n // Map of all node coordinates.\n let node2coord: {[id: string]: {cx: number, cy: number}} = {};\n let container = svg.append(\"g\")\n .classed(\"core\", true)\n .attr(\"transform\", `translate(${padding},${padding})`);\n // Draw the network layer by layer.\n let numLayers = network.length;\n let featureWidth = 118;\n let layerScale = d3.scale.ordinal()\n .domain(d3.range(1, numLayers - 1))\n .rangePoints([featureWidth, width - RECT_SIZE], 0.7);\n let nodeIndexScale = (nodeIndex: number) => nodeIndex * (RECT_SIZE + 25);\n\n\n let calloutThumb = d3.select(\".callout.thumbnail\").style(\"display\", \"none\");\n let calloutWeights = d3.select(\".callout.weights\").style(\"display\", \"none\");\n let idWithCallout = null;\n let targetIdWithCallout = null;\n\n // Draw the input layer separately.\n let cx = RECT_SIZE / 2 + 50;\n let nodeIds = Object.keys(INPUTS);\n let maxY = nodeIndexScale(nodeIds.length);\n nodeIds.forEach((nodeId, i) => {\n let cy = nodeIndexScale(i) + RECT_SIZE / 2;\n node2coord[nodeId] = {cx, cy};\n drawNode(cx, cy, nodeId, true, container);\n });\n\n // Draw the intermediate layers.\n for (let layerIdx = 1; layerIdx < numLayers - 1; layerIdx++) {\n let numNodes = network[layerIdx].length;\n let cx = layerScale(layerIdx) + RECT_SIZE / 2;\n maxY = Math.max(maxY, nodeIndexScale(numNodes));\n addPlusMinusControl(layerScale(layerIdx), layerIdx);\n for (let i = 0; i < numNodes; i++) {\n let node = network[layerIdx][i];\n let cy = nodeIndexScale(i) + RECT_SIZE / 2;\n node2coord[node.id] = {cx, cy};\n drawNode(cx, cy, node.id, false, container, node);\n\n // Show callout to thumbnails.\n let numNodes = network[layerIdx].length;\n let nextNumNodes = network[layerIdx + 1].length;\n if (idWithCallout == null &&\n i === numNodes - 1 &&\n nextNumNodes <= numNodes) {\n calloutThumb.style({\n display: null,\n top: `${20 + 3 + cy}px`,\n left: `${cx}px`\n });\n idWithCallout = node.id;\n }\n\n // Draw links.\n for (let j = 0; j < node.inputLinks.length; j++) {\n let link = node.inputLinks[j];\n let path: SVGPathElement = drawLink(link, node2coord, network,\n container, j === 0, j, node.inputLinks.length).node() as any;\n // Show callout to weights.\n let prevLayer = network[layerIdx - 1];\n let lastNodePrevLayer = prevLayer[prevLayer.length - 1];\n if (targetIdWithCallout == null &&\n i === numNodes - 1 &&\n link.source.id === lastNodePrevLayer.id &&\n (link.source.id !== idWithCallout || numLayers <= 5) &&\n link.dest.id !== idWithCallout &&\n prevLayer.length >= numNodes) {\n let midPoint = path.getPointAtLength(path.getTotalLength() * 0.7);\n calloutWeights.style({\n display: null,\n top: `${midPoint.y + 5}px`,\n left: `${midPoint.x + 3}px`\n });\n targetIdWithCallout = link.dest.id;\n }\n }\n }\n }\n\n // Draw the output node separately.\n cx = width + RECT_SIZE / 2;\n let node = network[numLayers - 1][0];\n let cy = nodeIndexScale(0) + RECT_SIZE / 2;\n node2coord[node.id] = {cx, cy};\n // Draw links.\n for (let i = 0; i < node.inputLinks.length; i++) {\n let link = node.inputLinks[i];\n drawLink(link, node2coord, network, container, i === 0, i,\n node.inputLinks.length);\n }\n // Adjust the height of the svg.\n svg.attr(\"height\", maxY);\n\n // Adjust the height of the features column.\n let height = Math.max(\n getRelativeHeight(calloutThumb),\n getRelativeHeight(calloutWeights),\n getRelativeHeight(d3.select(\"#network\"))\n );\n d3.select(\".column.features\").style(\"height\", height + \"px\");\n}\n\nfunction getRelativeHeight(selection) {\n let node = selection.node() as HTMLAnchorElement;\n return node.offsetHeight + node.offsetTop;\n}\n\nfunction addPlusMinusControl(x: number, layerIdx: number) {\n let div = d3.select(\"#network\").append(\"div\")\n .classed(\"plus-minus-neurons\", true)\n .style(\"left\", `${x - 10}px`);\n\n let i = layerIdx - 1;\n let firstRow = div.append(\"div\").attr(\"class\", `ui-numNodes${layerIdx}`);\n firstRow.append(\"button\")\n .attr(\"class\", \"mdl-button mdl-js-button mdl-button--icon\")\n .on(\"click\", () => {\n let numNeurons = state.networkShape[i];\n if (numNeurons >= 8) {\n return;\n }\n state.networkShape[i]++;\n parametersChanged = true;\n reset();\n })\n .append(\"i\")\n .attr(\"class\", \"material-icons\")\n .text(\"add\");\n\n firstRow.append(\"button\")\n .attr(\"class\", \"mdl-button mdl-js-button mdl-button--icon\")\n .on(\"click\", () => {\n let numNeurons = state.networkShape[i];\n if (numNeurons <= 1) {\n return;\n }\n state.networkShape[i]--;\n parametersChanged = true;\n reset();\n })\n .append(\"i\")\n .attr(\"class\", \"material-icons\")\n .text(\"remove\");\n\n let suffix = state.networkShape[i] > 1 ? \"s\" : \"\";\n div.append(\"div\").text(\n state.networkShape[i] + \" neuron\" + suffix\n );\n}\n\nfunction updateHoverCard(type: HoverType, nodeOrLink?: nn.Node | nn.Link,\n coordinates?: [number, number]) {\n let hovercard = d3.select(\"#hovercard\");\n if (type == null) {\n hovercard.style(\"display\", \"none\");\n d3.select(\"#svg\").on(\"click\", null);\n return;\n }\n d3.select(\"#svg\").on(\"click\", () => {\n hovercard.select(\".value\").style(\"display\", \"none\");\n let input = hovercard.select(\"input\");\n input.style(\"display\", null);\n input.on(\"input\", function() {\n if (this.value != null && this.value !== \"\") {\n if (type === HoverType.WEIGHT) {\n (nodeOrLink as nn.Link).weight = +this.value;\n } else {\n (nodeOrLink as nn.Node).bias = +this.value;\n }\n updateUI();\n }\n });\n input.on(\"keypress\", () => {\n if ((d3.event as any).keyCode === 13) {\n updateHoverCard(type, nodeOrLink, coordinates);\n }\n });\n (input.node() as HTMLInputElement).focus();\n });\n let value = (type === HoverType.WEIGHT) ?\n (nodeOrLink as nn.Link).weight :\n (nodeOrLink as nn.Node).bias;\n let name = (type === HoverType.WEIGHT) ? \"Weight\" : \"Bias\";\n hovercard.style({\n \"left\": `${coordinates[0] + 20}px`,\n \"top\": `${coordinates[1]}px`,\n \"display\": \"block\"\n });\n hovercard.select(\".type\").text(name);\n hovercard.select(\".value\")\n .style(\"display\", null)\n .text(value.toPrecision(2));\n hovercard.select(\"input\")\n .property(\"value\", value.toPrecision(2))\n .style(\"display\", \"none\");\n}\n\nfunction drawLink(\n input: nn.Link, node2coord: {[id: string]: {cx: number, cy: number}},\n network: nn.Node[][], container,\n isFirst: boolean, index: number, length: number) {\n let line = container.insert(\"path\", \":first-child\");\n let source = node2coord[input.source.id];\n let dest = node2coord[input.dest.id];\n let datum = {\n source: {\n y: source.cx + RECT_SIZE / 2 + 2,\n x: source.cy\n },\n target: {\n y: dest.cx - RECT_SIZE / 2,\n x: dest.cy + ((index - (length - 1) / 2) / length) * 12\n }\n };\n let diagonal = d3.svg.diagonal().projection(d => [d.y, d.x]);\n line.attr({\n \"marker-start\": \"url(#markerArrow)\",\n class: \"link\",\n id: \"link\" + input.source.id + \"-\" + input.dest.id,\n d: diagonal(datum, 0)\n });\n\n // Add an invisible thick link that will be used for\n // showing the weight value on hover.\n container.append(\"path\")\n .attr(\"d\", diagonal(datum, 0))\n .attr(\"class\", \"link-hover\")\n .on(\"mouseenter\", function() {\n updateHoverCard(HoverType.WEIGHT, input, d3.mouse(this));\n }).on(\"mouseleave\", function() {\n updateHoverCard(null);\n });\n return line;\n}\n\n/**\n * Given a neural network, it asks the network for the output (prediction)\n * of every node in the network using inputs sampled on a square grid.\n * It returns a map where each key is the node ID and the value is a square\n * matrix of the outputs of the network for each input in the grid respectively.\n */\nfunction updateDecisionBoundary(network: nn.Node[][], firstTime: boolean) {\n if (firstTime) {\n boundary = {};\n nn.forEachNode(network, true, node => {\n boundary[node.id] = new Array(DENSITY);\n });\n // Go through all predefined inputs.\n for (let nodeId in INPUTS) {\n boundary[nodeId] = new Array(DENSITY);\n }\n }\n let xScale = d3.scale.linear().domain([0, DENSITY - 1]).range(xDomain);\n let yScale = d3.scale.linear().domain([DENSITY - 1, 0]).range(xDomain);\n\n let i = 0, j = 0;\n for (i = 0; i < DENSITY; i++) {\n if (firstTime) {\n nn.forEachNode(network, true, node => {\n boundary[node.id][i] = new Array(DENSITY);\n });\n // Go through all predefined inputs.\n for (let nodeId in INPUTS) {\n boundary[nodeId][i] = new Array(DENSITY);\n }\n }\n for (j = 0; j < DENSITY; j++) {\n // 1 for points inside the circle, and 0 for points outside the circle.\n let x = xScale(i);\n let y = yScale(j);\n let input = constructInput(x, y);\n nn.forwardProp(network, input);\n nn.forEachNode(network, true, node => {\n boundary[node.id][i][j] = node.output;\n });\n if (firstTime) {\n // Go through all predefined inputs.\n for (let nodeId in INPUTS) {\n boundary[nodeId][i][j] = INPUTS[nodeId].f(x, y);\n }\n }\n }\n }\n}\n\nfunction getLoss(network: nn.Node[][], dataPoints: Example2D[]): number {\n let loss = 0;\n for (let i = 0; i < dataPoints.length; i++) {\n let dataPoint = dataPoints[i];\n let input = constructInput(dataPoint.x, dataPoint.y);\n let output = nn.forwardProp(network, input);\n loss += nn.Errors.SQUARE.error(output, dataPoint.label);\n }\n return loss / dataPoints.length;\n}\n\nfunction updateUI(firstStep = false) {\n // Update the links visually.\n updateWeightsUI(network, d3.select(\"g.core\"));\n // Update the bias values visually.\n updateBiasesUI(network);\n // Get the decision boundary of the network.\n updateDecisionBoundary(network, firstStep);\n let selectedId = selectedNodeId != null ?\n selectedNodeId : nn.getOutputNode(network).id;\n heatMap.updateBackground(boundary[selectedId], state.discretize);\n\n // Update all decision boundaries.\n d3.select(\"#network\").selectAll(\"div.canvas\")\n .each(function(data: {heatmap: HeatMap, id: string}) {\n data.heatmap.updateBackground(reduceMatrix(boundary[data.id], 10),\n state.discretize);\n });\n\n function zeroPad(n: number): string {\n let pad = \"000000\";\n return (pad + n).slice(-pad.length);\n }\n\n function addCommas(s: string): string {\n return s.replace(/\\B(?=(\\d{3})+(?!\\d))/g, \",\");\n }\n\n function humanReadable(n: number): string {\n return n.toFixed(3);\n }\n\n // Update loss and iteration number.\n d3.select(\"#loss-train\").text(humanReadable(lossTrain));\n d3.select(\"#loss-test\").text(humanReadable(lossTest));\n d3.select(\"#iter-number\").text(addCommas(zeroPad(iter)));\n lineChart.addDataPoint([lossTrain, lossTest]);\n}\n\nfunction constructInputIds(): string[] {\n let result: string[] = [];\n for (let inputName in INPUTS) {\n if (state[inputName]) {\n result.push(inputName);\n }\n }\n return result;\n}\n\nfunction constructInput(x: number, y: number): number[] {\n let input: number[] = [];\n for (let inputName in INPUTS) {\n if (state[inputName]) {\n input.push(INPUTS[inputName].f(x, y));\n }\n }\n return input;\n}\n\nfunction oneStep(): void {\n iter++;\n trainData.forEach((point, i) => {\n let input = constructInput(point.x, point.y);\n nn.forwardProp(network, input);\n nn.backProp(network, point.label, nn.Errors.SQUARE);\n if ((i + 1) % state.batchSize === 0) {\n nn.updateWeights(network, state.learningRate, state.regularizationRate);\n }\n });\n // Compute the loss.\n lossTrain = getLoss(network, trainData);\n lossTest = getLoss(network, testData);\n updateUI();\n}\n\nexport function getOutputWeights(network: nn.Node[][]): number[] {\n let weights: number[] = [];\n for (let layerIdx = 0; layerIdx < network.length - 1; layerIdx++) {\n let currentLayer = network[layerIdx];\n for (let i = 0; i < currentLayer.length; i++) {\n let node = currentLayer[i];\n for (let j = 0; j < node.outputs.length; j++) {\n let output = node.outputs[j];\n weights.push(output.weight);\n }\n }\n }\n return weights;\n}\n\nfunction reset(onStartup=false) {\n lineChart.reset();\n state.serialize();\n if (!onStartup) {\n userHasInteracted();\n }\n player.pause();\n\n let suffix = state.numHiddenLayers !== 1 ? \"s\" : \"\";\n d3.select(\"#layers-label\").text(\"Hidden layer\" + suffix);\n d3.select(\"#num-layers\").text(state.numHiddenLayers);\n\n // Make a simple network.\n iter = 0;\n let numInputs = constructInput(0 , 0).length;\n let shape = [numInputs].concat(state.networkShape).concat([1]);\n let outputActivation = (state.problem === Problem.REGRESSION) ?\n nn.Activations.LINEAR : nn.Activations.TANH;\n network = nn.buildNetwork(shape, state.activation, outputActivation,\n state.regularization, constructInputIds(), state.initZero);\n lossTrain = getLoss(network, trainData);\n lossTest = getLoss(network, testData);\n drawNetwork(network);\n updateUI(true);\n};\n\nfunction initTutorial() {\n if (state.tutorial == null || state.tutorial === '' || state.hideText) {\n return;\n }\n // Remove all other text.\n d3.selectAll(\"article div.l--body\").remove();\n let tutorial = d3.select(\"article\").append(\"div\")\n .attr(\"class\", \"l--body\");\n // Insert tutorial text.\n d3.html(`tutorials/${state.tutorial}.html`, (err, htmlFragment) => {\n if (err) {\n throw err;\n }\n tutorial.node().appendChild(htmlFragment);\n // If the tutorial has a tag, set the page title to that.\n let title = tutorial.select(\"title\");\n if (title.size()) {\n d3.select(\"header h1\").style({\n \"margin-top\": \"20px\",\n \"margin-bottom\": \"20px\",\n })\n .text(title.text());\n document.title = title.text();\n }\n });\n}\n\nfunction drawDatasetThumbnails() {\n function renderThumbnail(canvas, dataGenerator) {\n let w = 100;\n let h = 100;\n canvas.setAttribute(\"width\", w);\n canvas.setAttribute(\"height\", h);\n let context = canvas.getContext(\"2d\");\n let data = dataGenerator(200, 0);\n data.forEach(function(d) {\n context.fillStyle = colorScale(d.label);\n context.fillRect(w * (d.x + 6) / 12, h * (d.y + 6) / 12, 4, 4);\n });\n d3.select(canvas.parentNode).style(\"display\", null);\n }\n d3.selectAll(\".dataset\").style(\"display\", \"none\");\n\n if (state.problem === Problem.CLASSIFICATION) {\n for (let dataset in datasets) {\n let canvas: any =\n document.querySelector(`canvas[data-dataset=${dataset}]`);\n let dataGenerator = datasets[dataset];\n renderThumbnail(canvas, dataGenerator);\n }\n }\n if (state.problem === Problem.REGRESSION) {\n for (let regDataset in regDatasets) {\n let canvas: any =\n document.querySelector(`canvas[data-regDataset=${regDataset}]`);\n let dataGenerator = regDatasets[regDataset];\n renderThumbnail(canvas, dataGenerator);\n }\n }\n}\n\nfunction hideControls() {\n // Set display:none to all the UI elements that are hidden.\n let hiddenProps = state.getHiddenProps();\n hiddenProps.forEach(prop => {\n let controls = d3.selectAll(`.ui-${prop}`);\n if (controls.size() === 0) {\n console.warn(`0 html elements found with class .ui-${prop}`);\n }\n controls.style(\"display\", \"none\");\n });\n\n // Also add checkbox for each hidable control in the \"use it in classrom\"\n // section.\n let hideControls = d3.select(\".hide-controls\");\n HIDABLE_CONTROLS.forEach(([text, id]) => {\n let label = hideControls.append(\"label\")\n .attr(\"class\", \"mdl-checkbox mdl-js-checkbox mdl-js-ripple-effect\");\n let input = label.append(\"input\")\n .attr({\n type: \"checkbox\",\n class: \"mdl-checkbox__input\",\n });\n if (hiddenProps.indexOf(id) === -1) {\n input.attr(\"checked\", \"true\");\n }\n input.on(\"change\", function() {\n state.setHideProperty(id, !this.checked);\n state.serialize();\n userHasInteracted();\n d3.select(\".hide-controls-link\")\n .attr(\"href\", window.location.href);\n });\n label.append(\"span\")\n .attr(\"class\", \"mdl-checkbox__label label\")\n .text(text);\n });\n d3.select(\".hide-controls-link\")\n .attr(\"href\", window.location.href);\n}\n\nfunction generateData(firstTime = false) {\n if (!firstTime) {\n // Change the seed.\n state.seed = Math.random().toFixed(5);\n state.serialize();\n userHasInteracted();\n }\n Math.seedrandom(state.seed);\n let numSamples = (state.problem === Problem.REGRESSION) ?\n NUM_SAMPLES_REGRESS : NUM_SAMPLES_CLASSIFY;\n let generator = state.problem === Problem.CLASSIFICATION ?\n state.dataset : state.regDataset;\n let data = generator(numSamples, state.noise / 100);\n // Shuffle the data in-place.\n shuffle(data);\n // Split into train and test data.\n let splitIndex = Math.floor(data.length * state.percTrainData / 100);\n trainData = data.slice(0, splitIndex);\n testData = data.slice(splitIndex);\n heatMap.updatePoints(trainData);\n heatMap.updateTestPoints(state.showTestData ? testData : []);\n}\n\nlet firstInteraction = true;\nlet parametersChanged = false;\n\nfunction userHasInteracted() {\n if (!firstInteraction) {\n return;\n }\n firstInteraction = false;\n let page = 'index';\n if (state.tutorial != null && state.tutorial !== '') {\n page = `/v/tutorials/${state.tutorial}`;\n }\n ga('set', 'page', page);\n ga('send', 'pageview', {'sessionControl': 'start'});\n}\n\nfunction simulationStarted() {\n ga('send', {\n hitType: 'event',\n eventCategory: 'Starting Simulation',\n eventAction: parametersChanged ? 'changed' : 'unchanged',\n eventLabel: state.tutorial == null ? '' : state.tutorial\n });\n parametersChanged = false;\n}\n\ndrawDatasetThumbnails();\ninitTutorial();\nmakeGUI();\ngenerateData(true);\nreset(true);\nhideControls();\n" }, { "path": "src/seedrandom.d.ts", "content": "/* Copyright 2016 Google Inc. All Rights Reserved.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n==============================================================================*/\n\ninterface Math {\n seedrandom: (seed: string) => void;\n}\n\ndeclare let ga: any;" }, { "path": "src/state.ts", "content": "/* Copyright 2016 Google Inc. All Rights Reserved.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n==============================================================================*/\n\nimport * as nn from \"./nn\";\nimport * as dataset from \"./dataset\";\n\n/** Suffix added to the state when storing if a control is hidden or not. */\nconst HIDE_STATE_SUFFIX = \"_hide\";\n\n/** A map between names and activation functions. */\nexport let activations: {[key: string]: nn.ActivationFunction} = {\n \"relu\": nn.Activations.RELU,\n \"tanh\": nn.Activations.TANH,\n \"sigmoid\": nn.Activations.SIGMOID,\n \"linear\": nn.Activations.LINEAR\n};\n\n/** A map between names and regularization functions. */\nexport let regularizations: {[key: string]: nn.RegularizationFunction} = {\n \"none\": null,\n \"L1\": nn.RegularizationFunction.L1,\n \"L2\": nn.RegularizationFunction.L2\n};\n\n/** A map between dataset names and functions that generate classification data. */\nexport let datasets: {[key: string]: dataset.DataGenerator} = {\n \"circle\": dataset.classifyCircleData,\n \"xor\": dataset.classifyXORData,\n \"gauss\": dataset.classifyTwoGaussData,\n \"spiral\": dataset.classifySpiralData,\n};\n\n/** A map between dataset names and functions that generate regression data. */\nexport let regDatasets: {[key: string]: dataset.DataGenerator} = {\n \"reg-plane\": dataset.regressPlane,\n \"reg-gauss\": dataset.regressGaussian\n};\n\nexport function getKeyFromValue(obj: any, value: any): string {\n for (let key in obj) {\n if (obj[key] === value) {\n return key;\n }\n }\n return undefined;\n}\n\nfunction endsWith(s: string, suffix: string): boolean {\n return s.substr(-suffix.length) === suffix;\n}\n\nfunction getHideProps(obj: any): string[] {\n let result: string[] = [];\n for (let prop in obj) {\n if (endsWith(prop, HIDE_STATE_SUFFIX)) {\n result.push(prop);\n }\n }\n return result;\n}\n\n/**\n * The data type of a state variable. Used for determining the\n * (de)serialization method.\n */\nexport enum Type {\n STRING,\n NUMBER,\n ARRAY_NUMBER,\n ARRAY_STRING,\n BOOLEAN,\n OBJECT\n}\n\nexport enum Problem {\n CLASSIFICATION,\n REGRESSION\n}\n\nexport let problems = {\n \"classification\": Problem.CLASSIFICATION,\n \"regression\": Problem.REGRESSION\n};\n\nexport interface Property {\n name: string;\n type: Type;\n keyMap?: {[key: string]: any};\n};\n\n// Add the GUI state.\nexport class State {\n\n private static PROPS: Property[] = [\n {name: \"activation\", type: Type.OBJECT, keyMap: activations},\n {name: \"regularization\", type: Type.OBJECT, keyMap: regularizations},\n {name: \"batchSize\", type: Type.NUMBER},\n {name: \"dataset\", type: Type.OBJECT, keyMap: datasets},\n {name: \"regDataset\", type: Type.OBJECT, keyMap: regDatasets},\n {name: \"learningRate\", type: Type.NUMBER},\n {name: \"regularizationRate\", type: Type.NUMBER},\n {name: \"noise\", type: Type.NUMBER},\n {name: \"networkShape\", type: Type.ARRAY_NUMBER},\n {name: \"seed\", type: Type.STRING},\n {name: \"showTestData\", type: Type.BOOLEAN},\n {name: \"discretize\", type: Type.BOOLEAN},\n {name: \"percTrainData\", type: Type.NUMBER},\n {name: \"x\", type: Type.BOOLEAN},\n {name: \"y\", type: Type.BOOLEAN},\n {name: \"xTimesY\", type: Type.BOOLEAN},\n {name: \"xSquared\", type: Type.BOOLEAN},\n {name: \"ySquared\", type: Type.BOOLEAN},\n {name: \"cosX\", type: Type.BOOLEAN},\n {name: \"sinX\", type: Type.BOOLEAN},\n {name: \"cosY\", type: Type.BOOLEAN},\n {name: \"sinY\", type: Type.BOOLEAN},\n {name: \"collectStats\", type: Type.BOOLEAN},\n {name: \"tutorial\", type: Type.STRING},\n {name: \"problem\", type: Type.OBJECT, keyMap: problems},\n {name: \"initZero\", type: Type.BOOLEAN},\n {name: \"hideText\", type: Type.BOOLEAN}\n ];\n\n [key: string]: any;\n learningRate = 0.03;\n regularizationRate = 0;\n showTestData = false;\n noise = 0;\n batchSize = 10;\n discretize = false;\n tutorial: string = null;\n percTrainData = 50;\n activation = nn.Activations.TANH;\n regularization: nn.RegularizationFunction = null;\n problem = Problem.CLASSIFICATION;\n initZero = false;\n hideText = false;\n collectStats = false;\n numHiddenLayers = 1;\n hiddenLayerControls: any[] = [];\n networkShape: number[] = [4, 2];\n x = true;\n y = true;\n xTimesY = false;\n xSquared = false;\n ySquared = false;\n cosX = false;\n sinX = false;\n cosY = false;\n sinY = false;\n dataset: dataset.DataGenerator = dataset.classifyCircleData;\n regDataset: dataset.DataGenerator = dataset.regressPlane;\n seed: string;\n\n /**\n * Deserializes the state from the url hash.\n */\n static deserializeState(): State {\n let map: {[key: string]: string} = {};\n for (let keyvalue of window.location.hash.slice(1).split(\"&\")) {\n let [name, value] = keyvalue.split(\"=\");\n map[name] = value;\n }\n let state = new State();\n\n function hasKey(name: string): boolean {\n return name in map && map[name] != null && map[name].trim() !== \"\";\n }\n\n function parseArray(value: string): string[] {\n return value.trim() === \"\" ? [] : value.split(\",\");\n }\n\n // Deserialize regular properties.\n State.PROPS.forEach(({name, type, keyMap}) => {\n switch (type) {\n case Type.OBJECT:\n if (keyMap == null) {\n throw Error(\"A key-value map must be provided for state \" +\n \"variables of type Object\");\n }\n if (hasKey(name) && map[name] in keyMap) {\n state[name] = keyMap[map[name]];\n }\n break;\n case Type.NUMBER:\n if (hasKey(name)) {\n // The + operator is for converting a string to a number.\n state[name] = +map[name];\n }\n break;\n case Type.STRING:\n if (hasKey(name)) {\n state[name] = map[name];\n }\n break;\n case Type.BOOLEAN:\n if (hasKey(name)) {\n state[name] = (map[name] === \"false\" ? false : true);\n }\n break;\n case Type.ARRAY_NUMBER:\n if (name in map) {\n state[name] = parseArray(map[name]).map(Number);\n }\n break;\n case Type.ARRAY_STRING:\n if (name in map) {\n state[name] = parseArray(map[name]);\n }\n break;\n default:\n throw Error(\"Encountered an unknown type for a state variable\");\n }\n });\n\n // Deserialize state properties that correspond to hiding UI controls.\n getHideProps(map).forEach(prop => {\n state[prop] = (map[prop] === \"true\") ? true : false;\n });\n state.numHiddenLayers = state.networkShape.length;\n if (state.seed == null) {\n state.seed = Math.random().toFixed(5);\n }\n Math.seedrandom(state.seed);\n return state;\n }\n\n /**\n * Serializes the state into the url hash.\n */\n serialize() {\n // Serialize regular properties.\n let props: string[] = [];\n State.PROPS.forEach(({name, type, keyMap}) => {\n let value = this[name];\n // Don't serialize missing values.\n if (value == null) {\n return;\n }\n if (type === Type.OBJECT) {\n value = getKeyFromValue(keyMap, value);\n } else if (type === Type.ARRAY_NUMBER ||\n type === Type.ARRAY_STRING) {\n value = value.join(\",\");\n }\n props.push(`${name}=${value}`);\n });\n // Serialize properties that correspond to hiding UI controls.\n getHideProps(this).forEach(prop => {\n props.push(`${prop}=${this[prop]}`);\n });\n window.location.hash = props.join(\"&\");\n }\n\n /** Returns all the hidden properties. */\n getHiddenProps(): string[] {\n let result: string[] = [];\n for (let prop in this) {\n if (endsWith(prop, HIDE_STATE_SUFFIX) && String(this[prop]) === \"true\") {\n result.push(prop.replace(HIDE_STATE_SUFFIX, \"\"));\n }\n }\n return result;\n }\n\n setHideProperty(name: string, hidden: boolean) {\n this[name + HIDE_STATE_SUFFIX] = hidden;\n }\n}\n" }, { "path": "styles.css", "content": "/* Copyright 2016 Google Inc. All Rights Reserved.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n==============================================================================*/\n\n/* General Type */\n\nbody {\n font-family: \"Helvetica\", \"Arial\", sans-serif;\n background-color: #f7f7f7;\n}\n\nh1 {\n font-size: 34px;\n}\n\nheader h1 {\n line-height: 1.45em;\n font-weight: 300;\n color: rgba(255, 255, 255, 0.7);\n}\n\nh1 b {\n font-weight: 400;\n color: rgba(255, 255, 255, 1);\n}\n\nh2 {\n margin: 5px 0;\n font-weight: 300;\n font-size: 18px;\n}\n\nh3 {\n margin: 10px 0;\n}\n\np a {\n color: #0D658C;\n}\n\n/* Layout */\n\nbody {\n margin: 0;\n}\n\n.l--body {\n width: 550px;\n margin-left: auto;\n margin-right: auto;\n}\n\n.l--page {\n width: 944px;\n margin-left: auto;\n margin-right: auto;\n}\n\n@media (min-width: 1180px) {\n .l--page {\n width: 1100px;\n }\n}\n\n@media (min-width: 1400px) {\n .l--page {\n width: 1220px;\n }\n}\n\n/* Buttons */\n\n#main-part .mdl-button {\n background-color: rgba(158,158,158,.1);\n width: 28px;\n height: 28px;\n min-width: 28px;\n}\n\n#main-part .mdl-button:hover {\n background-color: rgba(158,158,158,.3);\n}\n\n#main-part .mdl-button:focus:not(:active) {\n background-color: rgba(158,158,158,.4);\n}\n\n#main-part .mdl-button:active {\n background-color: rgba(158,158,158,.5);\n}\n\n#main-part .mdl-button .material-icons {\n font-size: 20px;\n color: rgba(0, 0, 0, 0.7);\n}\n\n\n.button {\n cursor: pointer;\n display: -webkit-box;\n display: -moz-box;\n display: -ms-flexbox;\n display: -webkit-flex;\n display: flex;\n align-items: center;\n -webkit-justify-content: center;\n justify-content: center;\n width: 24px;\n height: 24px;\n font-size: 18px;\n border-radius: 50%;\n margin: 0 1px;\n background-color: rgba(0,0,0,0.05);\n outline: none;\n border: none;\n padding: 0;\n color: #666;\n transition: background-color 0.3s, color 0.3s;\n}\n\n.button:hover {\n background-color: rgba(0,0,0,0.1);\n\n}\n\n.button:active {\n background-color: rgba(0,0,0,0.15);\n color: #333;\n}\n\n.button i {\n font-size: 16px;\n}\n\n.hide-button {\n cursor: pointer;\n padding: 6px 4px 8px 4px;\n border-left: 1px solid #2c2c2c;\n border-bottom: 1px solid #2c2c2c;\n position: fixed;\n right: 0px;\n background: #1a1a1a;\n color: #eee;\n font: 11px 'Lucida Grande', sans-serif;\n display: table;\n}\n\n/* Header */\n\n.github-link {\n width: 60px;\n height: 60px;\n position: absolute;\n display: block;\n top: 0;\n right: 0;\n z-index: 1000;\n}\n\n.github-link .bg {\n fill: #fff;\n fill-opacity: 0.2;\n}\n\n.github-link:hover .bg {\n fill-opacity: 0.3;\n}\n\n.github-link .icon {\n fill: #fff;\n fill-opacity: 0.6;\n}\n\n.github-link:hover .icon {\n fill-opacity: 0.7;\n}\n\nheader {\n border-bottom: solid 1px rgba(0,0,0,0.4);\n background-color: #183D4E;\n color: white;\n overflow: hidden;\n box-shadow: 0 2px 4px rgba(0,0,0,0.2);\n position: relative;\n}\n\nheader h1 {\n font-size: 30px;\n text-align: center;\n margin-top: 30px;\n margin-bottom: 30px;\n -webkit-font-smoothing: antialiased;\n}\n\nheader h1 .optional {\n display: none;\n}\n\n@media (min-width: 1064px) {\n header h1 .optional {\n display: inline;\n }\n}\n\n@media (min-height: 700px) {\n header h1 {\n margin-top: 40px;\n margin-bottom: 40px;\n }\n}\n\n@media (min-height: 800px) {\n header h1 {\n font-size: 34px;\n margin-top: 60px;\n margin-bottom: 60px;\n }\n}\n\n/* Top Controls */\n\n#top-controls {\n border-bottom: 1px solid #ddd;\n padding: 18px 0;\n box-shadow: 0 1px 4px rgba(0,0,0,0.08);\n background: white;\n}\n\n@media (min-height: 700px) {\n #top-controls {\n padding: 24px 0;\n }\n}\n\n#top-controls .container {\n display: -webkit-box;\n display: -moz-box;\n display: -ms-flexbox;\n display: -webkit-flex;\n display: flex;\n -webkit-justify-content: space-betweenspace-between;\n justify-content: space-between;\n}\n\n#top-controls .timeline-controls {\n display: -webkit-box;\n display: -moz-box;\n display: -ms-flexbox;\n display: -webkit-flex;\n display: flex;\n align-items: center;\n margin-right: 20px;\n width: 140px;\n}\n\n#play-pause-button .material-icons {\n color: white;\n font-size: 36px;\n transform: translate(-18px,-12px);\n}\n\n\n#play-pause-button .material-icons:nth-of-type(2) {\n display: none;\n}\n\n#play-pause-button.playing .material-icons:nth-of-type(1) {\n display: none;\n}\n\n#play-pause-button.playing .material-icons:nth-of-type(2) {\n display: inherit;\n}\n\n#top-controls .control {\n flex-grow: 1;\n max-width: 180px;\n min-width: 110px;\n margin-left: 30px;\n margin-top: 6px;\n}\n\n#top-controls .control .label,\n#top-controls .control label {\n color: #777;\n font-size: 13px;\n display: block;\n margin-bottom: 6px;\n font-weight: 300;\n}\n\n#top-controls .control .value {\n font-size: 24px;\n margin: 0;\n font-weight: 300;\n}\n\n#top-controls .control .select {\n position: relative;\n}\n\n#top-controls .control select {\n -webkit-appearance: none;\n -moz-appearance: none;\n appearance: none;\n display: block;\n background: none;\n border: none;\n border-radius: 0;\n padding: 6px 0;\n width: 100%;\n font-size: 14px;\n border-bottom: solid 1px #ccc;\n color: #333;\n outline: none;\n}\n\n#top-controls .control select:focus {\n border-bottom-color: #183D4E;\n}\n\n#top-controls .control .select::after {\n class: \"material-icons\";\n content: \"arrow_drop_down\";\n color: #999;\n font-family: 'Material Icons';\n font-weight: normal;\n font-style: normal;\n font-size: 18px;\n line-height: 1;\n letter-spacing: normal;\n text-transform: none;\n display: inline-block;\n white-space: nowrap;\n word-wrap: normal;\n direction: ltr;\n position: absolute;\n right: 0;\n top: 6px;\n pointer-events: none;\n}\n\n/* Hover card */\n#hovercard {\n display: none;\n position: absolute;\n padding: 5px;\n border: 1px solid #aaa;\n z-index: 1000;\n background: #fff;\n cursor: default;\n border-radius: 5px;\n left: 240px;\n width: 150px;\n top: -20px;\n}\n\n#hovercard input {\n width: 60px;\n}\n\n/* Main Part*/\n\n#main-part {\n display: -webkit-box;\n display: -moz-box;\n display: -ms-flexbox;\n display: -webkit-flex;\n display: flex;\n -webkit-justify-content: space-between;\n justify-content: space-between;\n margin-top: 30px;\n margin-bottom: 50px;\n padding-top: 2px;\n position: relative;\n}\n\n@media (min-height: 700px) {\n #main-part {\n margin-top: 50px;\n }\n}\n\n#main-part h4 {\n display: -webkit-box;\n display: -moz-box;\n display: -ms-flexbox;\n display: -webkit-flex;\n display: flex;\n align-items: center;\n font-weight: 400;\n font-size: 16px;\n text-transform: uppercase;\n position: relative;\n padding-bottom: 8px;\n margin: 0;\n line-height: 1.4em;\n}\n\n#main-part p,\n#main-part .column .label,\n#main-part .column label {\n font-weight: 300;\n line-height: 1.38em;\n margin: 0;\n color: #777;\n font-size: 13px;\n}\n\n.more {\n position: absolute;\n left: 50%;\n}\n\n.more button {\n position: absolute;\n left: -28px;\n top: -28px;\n background: white;\n}\n\n.more button:hover,\n.more button:active,\n.more button:focus,\n.more button:focus:not(:active) {\n background: white;\n}\n\nsvg text {\n dominant-baseline: middle;\n}\n\ncanvas {\n display: block;\n}\n\n.link {\n fill: none;\n stroke: #aaa;\n stroke-width: 1;\n}\n\ng.column rect {\n stroke: none;\n}\n\n#heatmap {\n position: relative;\n float: left;\n margin-top: 10px;\n}\n\n#heatmap .tick line {\n stroke: #ddd;\n}\n\n#heatmap .tick text {\n fill: #bbb;\n dominant-baseline: auto;\n}\n\n#heatmap .tick:nth-child(7) text {\n fill: #333;\n}\n\n#heatmap .tick:nth-child(7) line {\n stroke: #999;\n}\n\n/* Data column */\n\n.vcenter {\n display: -webkit-box;\n display: -moz-box;\n display: -ms-flexbox;\n display: -webkit-flex;\n display: flex;\n align-items: center;\n}\n\n.data.column {\n width: 10%;\n}\n\n.data.column .dataset-list {\n margin: 20px 0 10px;\n overflow: hidden;\n}\n\n.data.column .dataset {\n position: relative;\n float: left;\n width: 34px;\n height: 34px;\n margin: 0 14px 14px 0;\n}\n\n.data.column .dataset:nth-of-type(2n) {\n margin-right: 0;\n}\n\n.data.column .data-thumbnail {\n cursor: pointer;\n width: 100%;\n height: 100%;\n opacity: 0.2;\n border: 2px solid rgba(0,0,0,0.1);\n border-radius: 3px;\n}\n\n/*.data.column .dataset:first-of-type {\n margin-top: 8px;\n}\n\n.data.column .dataset:last-of-type {\n margin-bottom: 20px;\n}*/\n\n.data.column .data-thumbnail:hover {\n border: 2px solid #999;\n}\n\n.data.column .data-thumbnail.selected {\n border: 2px solid black;\n opacity: 1;\n box-shadow: 0 1px 5px rgba(0,0,0,0.2);\n background-color: white;\n}\n\n#main-part .data.column .dataset .label {\n position: absolute;\n left: 48px;\n top: calc(50% - 9px);\n display: none;\n}\n\n#main-part .data.column p.slider {\n margin: 0 -25px 20px;\n}\n\n#main-part .basic-button {\n font-family: \"Roboto\", \"Helvetica\", \"Arial\", sans-serif;\n margin-top: 25px;\n height: 34px;\n margin-right: 0;\n width: 100%;\n display: block;\n color: rgba(0, 0, 0, 0.5);\n border: none;\n background: rgba(158,158,158,.1);\n border-radius: 3px;\n padding: 5px;\n font-size: 12px;\n text-transform: uppercase;\n font-weight: 500;\n outline: none;\n transition: background 0.3s linear;\n cursor: pointer;\n}\n\n#main-part .basic-button:hover {\n background: rgba(158,158,158,.3);\n color: rgba(0, 0, 0, 0.6);\n}\n\n#main-part .basic-button:focus {\n background: rgba(158,158,158,.4);\n color: rgba(0, 0, 0, 0.7);\n}\n\n#main-part .basic-button:active {\n background: rgba(158,158,158,.5);\n color: rgba(0, 0, 0, 0.8);\n}\n\n/* Features column */\n\n.features.column {\n width: 10%;\n position: relative;\n}\n\n.features.column .plus-minus-neurons {\n position: absolute;\n text-align: center;\n line-height: 28px;\n top: -58px;\n width: 65px;\n height: 44px;\n font-size: 12px;\n z-index: 100;\n}\n\n.plus-minus-neurons .mdl-button:first-of-type {\n margin-right: 5px;\n}\n\n.features.column .callout {\n position: absolute;\n width: 95px;\n font-style: italic;\n}\n\n.features.column .callout svg {\n position: absolute;\n left: -15px;\n width: 30px;\n height: 30px;\n}\n\n.features.column .callout svg path {\n fill: none;\n stroke: black;\n stroke-opacity: 0.4;\n}\n\n.features.column .callout svg defs path {\n fill: black;\n stroke: none;\n fill-opacity: 0.4;\n}\n\n#main-part .features.column .callout .label {\n position: absolute;\n top: 24px;\n left: 3px;\n font-size: 11px;\n}\n\n/* Network (inside features column) */\n\n#network {\n position: absolute;\n top: 110px;\n left: 0;\n z-index: 100;\n}\n\n#network svg .main-label {\n font-size: 13px;\n fill: #333;\n font-weight: 300;\n}\n\n.axis line {\n fill: none;\n stroke: #777;\n shape-rendering: crispEdges;\n}\n\n.axis text {\n fill: #777;\n font-size: 10px;\n}\n\n.axis path {\n display: none;\n}\n\n#network svg .active .main-label {\n fill: #333\n}\n\n#network svg #markerArrow {\n fill: black;\n stroke: black;\n stroke-opacity: 0.2;\n}\n\n#network .node {\n cursor: default;\n}\n\n#network .node rect {\n fill: white;\n stroke-width: 0;\n}\n\n#network .node.inactive {\n opacity: 0.5;\n}\n\n#network .node.hovered {\n opacity: 1.0;\n}\n\n@-webkit-keyframes flowing {\n from { stroke-dashoffset: 0; } to { stroke-dashoffset: -10; }\n}\n\n#network .core .link {\n stroke-dasharray: 9 1;\n stroke-dashoffset: 1;\n /*-webkit-animation: 0.5s linear 0s infinite flowing;*/\n}\n\n/** Invisible thick links used for showing weight values on mouse hover. */\n#network .core .link-hover {\n stroke-width: 8;\n stroke: black;\n fill: none;\n opacity: 0;\n}\n\n#network .canvas canvas {\n position: absolute;\n top: -2px;\n left: -2px;\n border: 2px solid black;\n border-radius: 3px;\n box-shadow: 0 2px 5px rgba(0,0,0,0.2);\n}\n\n#network .canvas.inactive canvas {\n box-shadow: inherit;\n}\n\n#network .canvas.inactive canvas {\n opacity: 0.4;\n border: 0;\n top: 0;\n left: 0;\n}\n\n#network .canvas.hovered canvas {\n opacity: 1.0;\n border: 2px solid #666;\n top: -2px;\n left: -2px;\n}\n\n/* Hidden layers column */\n\n.hidden-layers.column {\n width: 40%;\n}\n\n#main-part .hidden-layers h4 {\n -webkit-justify-content: center;\n justify-content: center;\n margin-top: -5px;\n}\n\n.hidden-layers #layers-label {\n width: 125px;\n display: inline-block;\n}\n\n.hidden-layers #num-layers {\n margin: 0 10px;\n width: 10px;\n display: inline-block;\n}\n\n.hidden-layers h4 .mdl-button {\n margin-right: 5px;\n}\n\n.bracket {\n margin-top: 5px;\n border: solid 1px rgba(0, 0, 0, 0.2);\n border-bottom: 0;\n height: 4px;\n}\n\n.bracket.reverse {\n border-bottom: solid 1px rgba(0, 0, 0, 0.2);\n border-top: 0;\n margin-top: 0;\n margin-bottom: 5px;\n}\n\n/* Output column */\n\n.output.column {\n width: 275px;\n}\n\n.metrics {\n position: relative;\n font-weight: 300;\n font-size: 13px;\n height: 60px;\n}\n\n#linechart {\n position: absolute;\n top: 0;\n right: 0;\n width: 50%;\n height: 55px;\n}\n.metrics .train {\n color: #777;\n}\n\n#loss-test {\n color: black;\n}\n\n.output .output-stats .value {\n color: rgba(0, 0, 0, 0.6);\n /*font-size: 20px;*/\n font-weight: 300;\n display: inline;\n\n}\n\ng.train circle {\n stroke: white;\n stroke-width: 1;\n stroke-opacity: 0.8;\n fill-opacity: 0.9;\n}\n\n g.test circle {\n stroke-width: 1;\n stroke: black;\n stroke-opacity: 0.6;\n fill-opacity: 0.9;\n}\n\n#main-part .output .mdl-checkbox__label.label {\n line-height: 1.7em;\n}\n\n/* Article */\n\narticle {\n background: white;\n padding: 80px 0;\n box-shadow: 0 0px 4px rgba(0, 0, 0, 0.1);\n /*border-top: 1px solid rgba(0, 0, 0, 0.08);*/\n}\n\narticle h2, article h3 {\n margin: 60px 0 20px 0;\n font-size: 22px;\n font-weight: 500;\n line-height: 1.45em;\n color: rgba(0, 0, 0, 0.7);\n}\n\narticle h3 {\n margin: 40px 0 20px 0;\n font-size: 18px;\n}\n\narticle :first-child h2 {\n margin-top: 0;\n}\n\narticle p {\n font-weight: 400;\n font-size: 17px;\n line-height: 1.6;\n color: rgba(0, 0, 0, 0.7);\n\n}\n\n/* Footer */\n\nfooter {\n border-top: solid 1px #eee;\n color: #ccc;\n font-weight: 300;\n padding: 40px 0;\n height: 30px;\n}\n\nfooter svg {\n margin-top: 2px;\n float: left;\n width: 110px;\n height: 18px;\n fill: #aaa;\n}\n\nfooter .links {\n float: right;\n font-size: 13px;\n height: 30px;\n line-height: 30px;\n margin-left: 20px;\n}\n\nfooter .links a {\n margin-right: 15px;\n text-decoration: none;\n color: #999;\n}\n\nfooter .links a:hover {\n text-decoration: underline;\n}\n\n.hide-controls {\n display: -webkit-box;\n display: -moz-box;\n display: -ms-flexbox;\n display: -webkit-flex;\n display: flex;\n flex-wrap: wrap;\n -webkit-justify-content: space-between;\n justify-content: space-between;\n}\n\n.hide-controls label.mdl-checkbox {\n width: 145px;\n height: 30px;\n}\n\n.hide-controls .mdl-checkbox__label {\n font-size: 13px;\n}\n\n/* Material Overrides */\n\n/* Buttons */\n\n.mdl-button--fab.mdl-button--colored,\n.mdl-button--fab.mdl-button--colored:hover,\n.mdl-button--fab.mdl-button--colored:active,\n.mdl-button--fab.mdl-button--colored:focus,\n.mdl-button--fab.mdl-button--colored:focus:not(:active) {\n background: #183D4E;\n}\n\n/* Checkbox */\n\n.mdl-checkbox__box-outline {\n border-color: rgba(0, 0, 0, 0.5);\n}\n\n.mdl-checkbox.is-checked .mdl-checkbox__tick-outline {\n background-color: #183D4E;\n}\n\n.mdl-checkbox.is-checked .mdl-checkbox__tick-outline {\n background-color: #183D4E;\n}\n\n.mdl-checkbox.is-checked .mdl-checkbox__box-outline {\n border-color: #183D4E;\n}\n\n.mdl-checkbox__ripple-container .mdl-ripple {\n background-color: #183D4E;\n}\n\n/* Slider */\n\n#main-part .mdl-slider.is-upgraded {\n color: #183D4E;\n}\n\n#main-part .mdl-slider__background-lower {\n background-color: #183D4E;\n}\n\n#main-part .mdl-slider.is-upgraded::-webkit-slider-thumb {\n background-color: #183D4E;\n}\n\n#main-part .mdl-slider.is-upgraded::-moz-range-thumb {\n background-color: #183D4E;\n}\n\n#main-part .mdl-slider.is-upgraded::-ms-thumb {\n background-color: #183D4E;\n}\n\n#main-part .mdl-slider.is-upgraded.is-lowest-value::-webkit-slider-thumb {\n border-color: #183D4E;\n}\n\n#main-part .mdl-slider.is-upgraded.is-lowest-value::-moz-range-thumb {\n border-color: #183D4E;\n}\n/* Keep grey focus circle for non-start values */\n#main-part .mdl-slider.is-upgraded:focus:not(:active)::-webkit-slider-thumb {\n box-shadow: 0 0 0 10px rgba(0,0,0, 0.12);\n}\n" }, { "path": "tsconfig.json", "content": "{\n \"compilerOptions\": {\n \"module\": \"commonjs\",\n \"removeComments\": true,\n \"preserveConstEnums\": true\n },\n \"exclude\": [\n \"node_modules\" \n ]\n}\n" }, { "path": "tslint.json", "content": "{\n \"rules\": {\n \"class-name\": true,\n \"comment-format\": [\n true,\n \"check-space\"\n ],\n \"max-line-length\": [true, 80],\n \"indent\": [\n true,\n \"spaces\"\n ],\n \"no-duplicate-variable\": true,\n \"no-eval\": true,\n \"no-internal-module\": true,\n \"no-trailing-whitespace\": true,\n \"no-var-keyword\": true,\n \"no-unused-variable\": true,\n \"no-unused-expression\": true,\n \"no-switch-case-fall-through\": true,\n \"no-unreachable\": true,\n \"one-line\": [\n true,\n \"check-open-brace\",\n \"check-whitespace\"\n ],\n \"forin\": false,\n \"quotemark\": [\n true,\n \"double\"\n ],\n \"semicolon\": [\n true,\n \"always\"\n ],\n \"triple-equals\": false,\n \"typedef-whitespace\": [\n true,\n {\n \"call-signature\": \"nospace\",\n \"index-signature\": \"nospace\",\n \"parameter\": \"nospace\",\n \"property-declaration\": \"nospace\",\n \"variable-declaration\": \"nospace\"\n }\n ],\n \"variable-name\": [\n true,\n \"ban-keywords\"\n ],\n \"whitespace\": [\n true,\n \"check-branch\",\n \"check-decl\",\n \"check-operator\",\n \"check-separator\",\n \"check-type\"\n ]\n }\n}\n" } ]