Repository: poloclub/cnn-explainer Branch: master Commit: d0971f9447ed Files: 47 Total size: 649.6 KB Directory structure: gitextract_939_80p5/ ├── .github/ │ └── workflows/ │ └── build.yml ├── .gitignore ├── LICENSE ├── README.md ├── deploy-gh-page.sh ├── package.json ├── public/ │ ├── assets/ │ │ └── data/ │ │ ├── model.json │ │ └── nn_10.json │ ├── global.css │ └── index.html ├── rollup.config.js ├── rollup_start_dev.js ├── src/ │ ├── App.svelte │ ├── Explainer.svelte │ ├── Header.svelte │ ├── article/ │ │ ├── Article.svelte │ │ └── Youtube.svelte │ ├── config.js │ ├── detail-view/ │ │ ├── ActivationAnimator.svelte │ │ ├── Activationview.svelte │ │ ├── ConvolutionAnimator.svelte │ │ ├── Convolutionview.svelte │ │ ├── Dataview.svelte │ │ ├── DetailviewUtils.js │ │ ├── HyperparameterAnimator.svelte │ │ ├── HyperparameterDataview.svelte │ │ ├── Hyperparameterview.svelte │ │ ├── KernelMathView.svelte │ │ ├── PoolAnimator.svelte │ │ ├── Poolview.svelte │ │ └── Softmaxview.svelte │ ├── main.js │ ├── overview/ │ │ ├── Modal.svelte │ │ ├── Overview.svelte │ │ ├── draw-utils.js │ │ ├── flatten-draw.js │ │ ├── intermediate-draw.js │ │ ├── intermediate-utils.js │ │ └── overview-draw.js │ ├── stores.js │ └── utils/ │ ├── cnn-tf.js │ ├── cnn.js │ ├── deploy-to-gh-pages.sh │ └── utlis.py └── tiny-vgg/ ├── README.md ├── environment.yaml └── tiny-vgg.py ================================================ FILE CONTENTS ================================================ ================================================ FILE: .github/workflows/build.yml ================================================ name: build on: # Triggers the workflow on push or pull request events but only for the master branch push: branches: [ master ] pull_request: branches: [ master ] schedule: - cron: "0 0 * * *" jobs: build: runs-on: ${{ matrix.os }} strategy: matrix: node-version: [16] os: [ubuntu-latest, macos-latest, windows-latest] steps: - uses: actions/checkout@v3 - uses: actions/setup-node@v3 with: node-version: ${{ matrix.node-version }} - name: Install dependencies run: npm install - name: Build run: npm run build ================================================ FILE: .gitignore ================================================ .DS_Store node_modules public/bundle.* package-lock.json public/assets/js/bundle.* public/assets/js/ public/assets/css/bundle.* *.swp yarn.lock dist/ local-build/ tiny-vgg/data/ pnpm-lock.yaml ================================================ FILE: LICENSE ================================================ MIT License Copyright (c) 2020 Polo Club of Data Science Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ================================================ FILE: README.md ================================================ # CNN Explainer An interactive visualization system designed to help non-experts learn about Convolutional Neural Networks (CNNs) [![build](https://github.com/poloclub/cnn-explainer/workflows/build/badge.svg)](https://github.com/poloclub/cnn-explainer/actions) [![arxiv badge](https://img.shields.io/badge/arXiv-2004.15004-red)](http://arxiv.org/abs/2004.15004) [![DOI:10.1109/TVCG.2020.3030418](https://img.shields.io/badge/DOI-10.1109/TVCG.2020.3030418-blue)](https://doi.org/10.1109/TVCG.2020.3030418) For more information, check out our manuscript: [**CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization**](https://arxiv.org/abs/2004.15004). Wang, Zijie J., Robert Turko, Omar Shaikh, Haekyu Park, Nilaksh Das, Fred Hohman, Minsuk Kahng, and Duen Horng Chau. *IEEE Transactions on Visualization and Computer Graphics (TVCG), 2020.* ## Live Demo For a live demo, visit: http://poloclub.github.io/cnn-explainer/ ## Running Locally Clone or download this repository: ```bash git clone git@github.com:poloclub/cnn-explainer.git # use degit if you don't want to download commit histories degit poloclub/cnn-explainer ``` Install the dependencies: ```bash npm install ``` Then run CNN Explainer: ```bash npm run dev ``` Navigate to [localhost:3000](https://localhost:3000). You should see CNN Explainer running in your broswer :) To see how we trained the CNN, visit the directory [`./tiny-vgg/`](tiny-vgg). If you want to use CNN Explainer with your own CNN model or image classes, see [#8](/../../issues/8) and [#14](/../../issues/14). ## Credits CNN Explainer was created by Jay Wang, Robert Turko, Omar Shaikh, Haekyu Park, Nilaksh Das, Fred Hohman, Minsuk Kahng, and Polo Chau, which was the result of a research collaboration between Georgia Tech and Oregon State. We thank [Anmol Chhabria](https://www.linkedin.com/in/anmolchhabria), [Kaan Sancak](https://kaansancak.com), [Kantwon Rogers](https://www.kantwon.com), and the [Georgia Tech Visualization Lab](http://vis.gatech.edu) for their support and constructive feedback. ## Citation ```bibTeX @article{wangCNNExplainerLearning2020, title = {{{CNN Explainer}}: {{Learning Convolutional Neural Networks}} with {{Interactive Visualization}}}, shorttitle = {{{CNN Explainer}}}, author = {Wang, Zijie J. and Turko, Robert and Shaikh, Omar and Park, Haekyu and Das, Nilaksh and Hohman, Fred and Kahng, Minsuk and Chau, Duen Horng}, journal={IEEE Transactions on Visualization and Computer Graphics (TVCG)}, year={2020}, publisher={IEEE} } ``` ## License The software is available under the [MIT License](https://github.com/poloclub/cnn-explainer/blob/master/LICENSE). ## Contact If you have any questions, feel free to [open an issue](https://github.com/poloclub/cnn-explainer/issues/new/choose) or contact [Jay Wang](https://zijie.wang). ================================================ FILE: deploy-gh-page.sh ================================================ npm run build cp -r ./public/assets ./dist cp -r ./public/bundle* ./dist cp -r ./public/global.css ./dist npx gh-pages -m "Deploy $(git log '--format=format:%H' master -1)" -d ./dist ================================================ FILE: package.json ================================================ { "name": "svelte-app", "version": "1.0.0", "devDependencies": { "@rollup/plugin-replace": "^2.3.2", "gh-pages": "^6.0.0", "rollup": "^1.27.13", "rollup-plugin-commonjs": "^10.0.0", "rollup-plugin-livereload": "^1.0.0", "rollup-plugin-node-resolve": "^5.2.0", "rollup-plugin-svelte": "~6.1.1", "rollup-plugin-terser": "^5.1.3", "svelte": "^3.31.0" }, "dependencies": { "@tensorflow/tfjs": "^1.4.0", "sirv-cli": "^0.4.4" }, "scripts": { "build": "rollup -c", "dev": "rollup -c -w", "start": "sirv public --single", "start:dev": "sirv public --single --dev --port 3000", "deploy": "npx" } } ================================================ FILE: public/assets/data/model.json ================================================ {"format": "layers-model", "generatedBy": "keras v2.2.4-tf", "convertedBy": "TensorFlow.js Converter v1.3.1.1", "modelTopology": {"keras_version": "2.2.4-tf", "backend": "tensorflow", "model_config": {"class_name": "Sequential", "config": {"name": "sequential", "layers": [{"class_name": "Conv2D", "config": {"name": "conv_1_1", "trainable": true, "batch_input_shape": [null, 64, 64, 3], "dtype": "float32", "filters": 10, "kernel_size": [3, 3], "strides": [1, 1], "padding": "valid", "data_format": "channels_last", "dilation_rate": [1, 1], "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "GlorotUniform", "config": {"seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}}, {"class_name": "Activation", "config": {"name": "relu_1_1", "trainable": true, "dtype": "float32", "activation": "relu"}}, {"class_name": "Conv2D", "config": {"name": "conv_1_2", "trainable": true, "dtype": "float32", "filters": 10, "kernel_size": [3, 3], "strides": [1, 1], "padding": "valid", "data_format": "channels_last", "dilation_rate": [1, 1], "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "GlorotUniform", "config": {"seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}}, {"class_name": "Activation", "config": {"name": "relu_1_2", "trainable": true, "dtype": "float32", "activation": "relu"}}, {"class_name": "MaxPooling2D", "config": {"name": "max_pool_1", "trainable": true, "dtype": "float32", "pool_size": [2, 2], "padding": "valid", "strides": [2, 2], "data_format": "channels_last"}}, {"class_name": "Conv2D", "config": {"name": "conv_2_1", "trainable": true, "dtype": "float32", "filters": 10, "kernel_size": [3, 3], "strides": [1, 1], "padding": "valid", "data_format": "channels_last", "dilation_rate": [1, 1], "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "GlorotUniform", "config": {"seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}}, {"class_name": "Activation", "config": {"name": "relu_2_1", "trainable": true, "dtype": "float32", "activation": "relu"}}, {"class_name": "Conv2D", "config": {"name": "conv_2_2", "trainable": true, "dtype": "float32", "filters": 10, "kernel_size": [3, 3], "strides": [1, 1], "padding": "valid", "data_format": "channels_last", "dilation_rate": [1, 1], "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "GlorotUniform", "config": {"seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}}, {"class_name": "Activation", "config": {"name": "relu_2_2", "trainable": true, "dtype": "float32", "activation": "relu"}}, {"class_name": "MaxPooling2D", "config": {"name": "max_pool_2", "trainable": true, "dtype": "float32", "pool_size": [2, 2], "padding": "valid", "strides": [2, 2], "data_format": "channels_last"}}, {"class_name": "Flatten", "config": {"name": "flatten", "trainable": true, "dtype": "float32", "data_format": "channels_last"}}, {"class_name": "Dense", "config": {"name": "output", "trainable": true, "dtype": "float32", "units": 10, "activation": "softmax", "use_bias": true, "kernel_initializer": {"class_name": "GlorotUniform", "config": {"seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}}]}}}, "weightsManifest": [{"paths": ["group1-shard1of1.bin"], "weights": [{"name": "conv_1_1/kernel", "shape": [3, 3, 3, 10], "dtype": "float32"}, {"name": "conv_1_1/bias", "shape": [10], "dtype": "float32"}, {"name": "conv_1_2/kernel", "shape": [3, 3, 10, 10], "dtype": "float32"}, {"name": "conv_1_2/bias", "shape": [10], "dtype": "float32"}, {"name": "conv_2_1/kernel", "shape": [3, 3, 10, 10], "dtype": "float32"}, {"name": "conv_2_1/bias", "shape": [10], "dtype": "float32"}, {"name": "conv_2_2/kernel", "shape": [3, 3, 10, 10], "dtype": "float32"}, {"name": "conv_2_2/bias", "shape": [10], "dtype": "float32"}, {"name": "output/kernel", "shape": [1690, 10], "dtype": "float32"}, {"name": "output/bias", "shape": [10], "dtype": "float32"}]}]} ================================================ FILE: public/assets/data/nn_10.json ================================================ [ { "name": "conv_1_1", "input_shape": [ 64, 64, 3 ], "output_shape": [ 62, 62, 10 ], "num_neurons": 10, "weights": [ { "bias": 0.05529399216175079, "weights": [ [ [ 0.005686734337359667, 0.5303338170051575, 0.4508902430534363 ], [ 0.04920876398682594, -0.029017755761742592, 0.23914846777915955 ], [ -0.06062067300081253, 0.10217977315187454, 0.2789939343929291 ] ], [ [ -0.11416204273700714, 0.2790183424949646, 0.14520207047462463 ], [ -0.14735761284828186, -0.21102763712406158, -0.15575028955936432 ], [ -0.10717931389808655, -0.28811952471733093, -0.0018726304406300187 ] ], [ [ -0.06842527538537979, 0.14463859796524048, -0.012262370437383652 ], [ -0.36216601729393005, -0.14219844341278076, -0.1687535047531128 ], [ 0.13701502978801727, 0.10424327105283737, 0.0899757370352745 ] ] ] }, { "bias": 0.15698345005512238, "weights": [ [ [ 0.11193452030420303, -0.02605212852358818, -0.17117753624916077 ], [ -0.1265387237071991, 0.11026564985513687, -0.23662784695625305 ], [ -0.113533616065979, -0.06741151958703995, 0.00405964907258749 ] ], [ [ -0.20665371417999268, 0.09713144600391388, 0.18439456820487976 ], [ 0.22104552388191223, 0.06683074682950974, 0.08409177511930466 ], [ -0.1461413949728012, -0.0339348129928112, 0.09842409938573837 ] ], [ [ -0.2126864641904831, -0.24490225315093994, 0.10998662561178207 ], [ 0.09896539151668549, -0.2154281884431839, -0.04586761072278023 ], [ 0.12391256541013718, -0.06876956671476364, -0.17699216306209564 ] ] ] }, { "bias": 0.1929195076227188, "weights": [ [ [ 0.303184449672699, -0.22255642712116241, 0.15304619073867798 ], [ -0.3187333941459656, -0.15920908749103546, 0.33759790658950806 ], [ 0.19879762828350067, 0.30086207389831543, -0.14565040171146393 ] ], [ [ -0.24475222826004028, -0.11444883793592453, -0.03977220505475998 ], [ -0.5766695141792297, -0.26573851704597473, 0.2617243826389313 ], [ -0.059220075607299805, 0.05921584367752075, -0.23321840167045593 ] ], [ [ 0.12728002667427063, -0.22710849344730377, 0.34297189116477966 ], [ -0.3018116354942322, 0.054087359458208084, 0.4344978928565979 ], [ 0.0469631627202034, 0.14949560165405273, -0.1706211119890213 ] ] ] }, { "bias": 0.13662371039390564, "weights": [ [ [ -0.2468211054801941, -0.07405976951122284, -0.056588608771562576 ], [ -0.0507829450070858, 0.18406149744987488, 0.11345680058002472 ], [ -0.28884875774383545, 0.1144554391503334, -0.20539431273937225 ] ], [ [ 0.1392928510904312, 0.2163618952035904, -0.0069107115268707275 ], [ -0.04500287026166916, 0.18057486414909363, 0.3434883654117584 ], [ 0.09264994412660599, 0.17778418958187103, -0.20051360130310059 ] ], [ [ 0.18443480134010315, 0.18599331378936768, -0.2483920156955719 ], [ 0.17116613686084747, 0.015662359073758125, 0.24437673389911652 ], [ 0.08613235503435135, -0.18919849395751953, 0.14660470187664032 ] ] ] }, { "bias": -0.01665310189127922, "weights": [ [ [ 0.0513838492333889, -0.18428930640220642, -0.13631688058376312 ], [ -0.020188838243484497, 0.014654036611318588, -0.2090645283460617 ], [ 0.19347725808620453, 0.008561060763895512, 0.12752579152584076 ] ], [ [ 0.10648118704557419, -0.09161978214979172, 0.15131479501724243 ], [ -0.07589973509311676, 0.03778020665049553, 0.07684637606143951 ], [ -0.21972328424453735, -0.06632781028747559, -0.21909500658512115 ] ], [ [ -0.11479348689317703, -0.10508662462234497, 0.07523884624242783 ], [ -0.14848960936069489, -0.04849429056048393, -0.14041589200496674 ], [ -0.18590395152568817, 0.031492143869400024, 0.10233276337385178 ] ] ] }, { "bias": 0.10964040458202362, "weights": [ [ [ -0.12156059592962265, 0.053360339254140854, 0.1554896980524063 ], [ 0.08920006453990936, -0.12873634696006775, -0.03641781583428383 ], [ -0.26828017830848694, 0.07801597565412521, 0.004753198008984327 ] ], [ [ -0.17305849492549896, -0.18035148084163666, -0.05541511997580528 ], [ 0.06097719073295593, -0.1959034502506256, -0.18717913329601288 ], [ -0.20423458516597748, 0.21905024349689484, -0.11647790670394897 ] ], [ [ 0.24856887757778168, 0.24108672142028809, -0.180223748087883 ], [ -0.10905743390321732, -0.15688706934452057, 0.07556945085525513 ], [ 0.060350243002176285, -0.07040304690599442, 0.21788306534290314 ] ] ] }, { "bias": 0.029825814068317413, "weights": [ [ [ -0.19718821346759796, -0.16889642179012299, 0.12187362462282181 ], [ 0.06039876490831375, -0.14378444850444794, 0.07948035001754761 ], [ -0.025484031066298485, -0.16321809589862823, -0.019062098115682602 ] ], [ [ -0.032011669129133224, -0.14139215648174286, 0.11273358762264252 ], [ -0.10067833960056305, -0.20742711424827576, 0.13296671211719513 ], [ 0.1675073355436325, -0.1571367383003235, 0.22434785962104797 ] ], [ [ -0.10166903585195541, -0.19185051321983337, 0.08926276117563248 ], [ 0.09333138912916183, 0.037962570786476135, 0.0007648404571227729 ], [ -0.05245523899793625, -0.1298869550228119, 0.03659629821777344 ] ] ] }, { "bias": 0.11516544222831726, "weights": [ [ [ -0.14063894748687744, 0.3289564251899719, 0.2985765039920807 ], [ 0.09437553584575653, -0.1446211189031601, -0.3673877418041229 ], [ -0.2984912395477295, -0.11701186001300812, 0.22971859574317932 ] ], [ [ 0.09642521291971207, 0.413046658039093, 0.3844977021217346 ], [ 0.2882622480392456, 0.1406082808971405, -0.2174244523048401 ], [ -0.24284584820270538, 0.041832320392131805, 0.24873341619968414 ] ], [ [ -0.0859166830778122, 0.11321970075368881, 0.13290554285049438 ], [ -0.09440730512142181, -0.26801732182502747, -0.4094286561012268 ], [ -0.36698707938194275, -0.018707290291786194, 0.08283202350139618 ] ] ] }, { "bias": 0.07734016329050064, "weights": [ [ [ -0.03253738954663277, -0.026013268157839775, 0.016034334897994995 ], [ -0.22023527324199677, 0.1287398338317871, -0.04254328832030296 ], [ -0.008483637124300003, -0.016512051224708557, 0.21947342157363892 ] ], [ [ -0.22642101347446442, 0.20579710602760315, 0.11521881073713303 ], [ -0.10988859832286835, 0.052845388650894165, 0.13670620322227478 ], [ -0.12536796927452087, 0.04135845974087715, -0.1554696261882782 ] ], [ [ -0.18598666787147522, 0.12068557739257812, 0.12700338661670685 ], [ 0.1888495832681656, -0.15277566015720367, -0.147127166390419 ], [ -0.1856405884027481, 0.07612188160419464, -0.2162964791059494 ] ] ] }, { "bias": -0.03841400146484375, "weights": [ [ [ -0.23960189521312714, -0.16422033309936523, 0.09344270825386047 ], [ -0.22473567724227905, -0.025265153497457504, -0.3128132224082947 ], [ -0.05619398131966591, 0.14468322694301605, -0.23834674060344696 ] ], [ [ 0.15905526280403137, 0.39751049876213074, 0.40521639585494995 ], [ 0.07429175078868866, 0.2791271209716797, 0.15926817059516907 ], [ 0.3116576373577118, 0.47250816226005554, 0.27831581234931946 ] ], [ [ -0.14721138775348663, 0.11997095495462418, -0.10465282946825027 ], [ -0.10158450901508331, -0.20062342286109924, -0.3983880579471588 ], [ -0.352329283952713, -0.22438788414001465, -0.14405152201652527 ] ] ] } ] }, { "name": "relu_1_1", "input_shape": [ 62, 62, 10 ], "output_shape": [ 62, 62, 10 ], "num_neurons": 10 }, { "name": "conv_1_2", "input_shape": [ 62, 62, 10 ], "output_shape": [ 60, 60, 10 ], "num_neurons": 10, "weights": [ { "bias": -0.015000003390014172, "weights": [ [ [ 0.06984533369541168, 0.12250980734825134, -0.06276966631412506 ], [ 0.027698174118995667, 0.28825634717941284, 0.11451877653598785 ], [ 0.1613343358039856, -0.07120175659656525, -0.02248406782746315 ] ], [ [ 0.06695717573165894, 0.004763114266097546, 0.03319444879889488 ], [ 0.04770096018910408, -0.07627807557582855, -0.1811506152153015 ], [ -0.09521480649709702, -0.020181972533464432, -0.14486107230186462 ] ], [ [ -0.061954572796821594, -0.07911273837089539, -0.3499084711074829 ], [ -0.24773694574832916, -0.21033312380313873, -0.01210795622318983 ], [ 0.053867269307374954, -0.07325703650712967, -0.2212786078453064 ] ], [ [ -0.13988210260868073, 0.06434164941310883, -0.09834592044353485 ], [ -0.026962799951434135, 0.12251508235931396, -0.08030775189399719 ], [ -0.22049900889396667, -0.03990422561764717, -0.13152480125427246 ] ], [ [ -0.17654503881931305, 0.1668800264596939, -0.138304203748703 ], [ 0.03783787786960602, -0.18137794733047485, -0.10964040458202362 ], [ 0.08009066432714462, -0.028887981548905373, -0.08581449836492538 ] ], [ [ 0.12367407232522964, -0.04227888956665993, -0.12297787517309189 ], [ -0.03683138266205788, -0.07924527674913406, 0.07216089963912964 ], [ 0.09089690446853638, -0.14476703107357025, -0.07134923338890076 ] ], [ [ 0.05118329077959061, 0.013351107016205788, 0.12198783457279205 ], [ 0.15751278400421143, -0.048728566616773605, -0.17966294288635254 ], [ 0.11106358468532562, -0.05154714733362198, 0.15185466408729553 ] ], [ [ -0.002612997544929385, -0.05618676543235779, 0.04629763215780258 ], [ 0.20909954607486725, 0.2826078534126282, 0.06790830194950104 ], [ -0.24769099056720734, -0.018110016360878944, 0.03419424593448639 ] ], [ [ -0.20570960640907288, 0.16505253314971924, 0.010972544550895691 ], [ -0.07593987882137299, 0.18975234031677246, 0.12766748666763306 ], [ -0.16967566311359406, -0.14139984548091888, -0.06672513484954834 ] ], [ [ -0.09351741522550583, -0.05367553234100342, 0.006337949074804783 ], [ 0.09447716921567917, 0.05620996654033661, -0.1393093317747116 ], [ 0.09124802052974701, -0.008008070290088654, -0.08058559894561768 ] ] ] }, { "bias": 0.03633357957005501, "weights": [ [ [ -0.028699593618512154, 0.01408400945365429, 0.053278084844350815 ], [ 0.17656444013118744, 0.020243415609002113, -0.27064037322998047 ], [ -0.03375490754842758, -0.07598341256380081, -0.054536327719688416 ] ], [ [ -0.016911577433347702, 0.011905811727046967, -0.004597058519721031 ], [ -0.15162792801856995, -0.040205202996730804, 0.029262922704219818 ], [ 0.12659160792827606, 0.1467726081609726, 0.13518129289150238 ] ], [ [ 0.08571384847164154, 0.04543592408299446, 0.1619890332221985 ], [ 0.04412376508116722, -0.20871470868587494, 0.13537569344043732 ], [ -0.11269881576299667, 0.316618412733078, -0.1193186566233635 ] ], [ [ 0.1737903505563736, -0.04851733520627022, 0.053338903933763504 ], [ 0.2689688801765442, 0.18783660233020782, 0.0049596684984862804 ], [ 0.08433520793914795, 0.2502553164958954, -0.10368360579013824 ] ], [ [ 0.03098849020898342, -0.16898685693740845, -0.14307470619678497 ], [ -0.04081837087869644, 0.1253221333026886, -0.175978884100914 ], [ -0.040029484778642654, -0.028231589123606682, 0.1737002283334732 ] ], [ [ 0.007191381882876158, 0.10486532747745514, 0.09948095679283142 ], [ 0.08876245468854904, 0.04661615192890167, 0.055014386773109436 ], [ 0.11458297818899155, 0.14941437542438507, 0.14682719111442566 ] ], [ [ -0.029932519420981407, 0.06707341223955154, 0.0011027005966752768 ], [ 0.013172822073101997, -0.027043195441365242, -0.07705768197774887 ], [ -0.1878139227628708, -0.10642603039741516, 0.15161097049713135 ] ], [ [ -0.24196398258209229, 0.2093714028596878, 0.016427693888545036 ], [ 0.18196193873882294, 0.15705186128616333, -0.16816706955432892 ], [ 0.2550819218158722, -0.012771722860634327, -0.03235609456896782 ] ], [ [ 0.05405253916978836, 0.04333319142460823, 0.18497972190380096 ], [ 0.040979642421007156, 0.04391923174262047, 0.003401109715923667 ], [ 0.06960181146860123, -0.16919413208961487, -0.01900867559015751 ] ], [ [ -0.12346938252449036, -0.0845215693116188, 0.03198058158159256 ], [ -0.09701372683048248, -0.31881439685821533, 0.05198657885193825 ], [ 0.0014515378279611468, -0.0015912456437945366, -0.13556411862373352 ] ] ] }, { "bias": 0.26450011134147644, "weights": [ [ [ 0.11858198046684265, 0.08620305359363556, -0.22812843322753906 ], [ 0.0737232193350792, -0.1764031946659088, -0.10877794027328491 ], [ -0.04786592349410057, 0.06122968718409538, -0.22324822843074799 ] ], [ [ 0.0688820481300354, -0.12272901833057404, -0.11515816301107407 ], [ -0.1755588799715042, -0.18296514451503754, 0.005947907920926809 ], [ 0.05370483547449112, -0.17687486112117767, -0.07346649467945099 ] ], [ [ -0.28695711493492126, -0.14942891895771027, 0.22552116215229034 ], [ 0.14913678169250488, 0.25072580575942993, -0.13216601312160492 ], [ 0.25203147530555725, 0.013846390880644321, 0.07630975544452667 ] ], [ [ 0.14870744943618774, 0.26395735144615173, 0.07945573329925537 ], [ 0.02941100113093853, -0.06789202243089676, -0.09479090571403503 ], [ -0.13743452727794647, 0.0012073514517396688, -0.19661739468574524 ] ], [ [ -0.01009973231703043, -0.1716722846031189, 0.1032852828502655 ], [ -0.005336238536983728, -0.015667693689465523, 0.01772109791636467 ], [ 0.15534654259681702, 0.030132949352264404, 0.059307970106601715 ] ], [ [ -0.17051032185554504, -0.03172222524881363, 0.017660612240433693 ], [ 0.06734868139028549, -0.16958904266357422, 0.0980442464351654 ], [ 0.08474505692720413, -0.16626432538032532, -0.14391560852527618 ] ], [ [ -0.047566138207912445, -0.12840814888477325, 0.07308086007833481 ], [ -0.10277002304792404, 0.03899145871400833, -0.08504311740398407 ], [ 0.08088894188404083, 0.07359552383422852, -0.1757054328918457 ] ], [ [ 0.2511260211467743, 0.16259819269180298, -0.2731759548187256 ], [ -0.15424731373786926, -0.2542499899864197, 0.1603444516658783 ], [ -0.03646847605705261, -0.02279706671833992, -0.0198900755494833 ] ], [ [ 0.1734747290611267, 0.03886084258556366, 0.013219695538282394 ], [ -0.04848269373178482, 0.12644779682159424, -0.13532158732414246 ], [ 0.17683625221252441, 0.13510529696941376, 0.06751107424497604 ] ], [ [ -0.1525471806526184, -0.1715271919965744, 0.09087955206632614 ], [ -0.030184892937541008, 0.008808514103293419, -0.14952726662158966 ], [ -0.12190154939889908, -0.08360952883958817, -0.27969890832901 ] ] ] }, { "bias": 0.16449642181396484, "weights": [ [ [ 0.01370085310190916, -0.003117413492873311, 0.018573859706521034 ], [ 0.02700856886804104, 0.0726592019200325, -0.16934508085250854 ], [ -0.013026679866015911, -0.06702490895986557, -0.02738550864160061 ] ], [ [ 0.21202805638313293, 0.1058105081319809, -0.07208364456892014 ], [ -0.06420744210481644, 0.011604615487158298, 0.027805227786302567 ], [ 0.00818086601793766, 0.13248521089553833, 0.1713949739933014 ] ], [ [ 0.15266969799995422, 0.07938865572214127, 0.1686062216758728 ], [ 0.21102824807167053, 0.20970553159713745, 0.04996765777468681 ], [ 0.25311100482940674, 0.120435930788517, 0.0902794823050499 ] ], [ [ 0.17864742875099182, 0.14377900958061218, -0.08969362825155258 ], [ 0.0035156651865690947, -0.1548624336719513, -0.038364484906196594 ], [ 0.04785368591547012, 0.010373460128903389, -0.048020996153354645 ] ], [ [ -0.014619680121541023, 0.15819314122200012, -0.14283199608325958 ], [ -0.1338491141796112, -0.027468981221318245, -0.09391266107559204 ], [ -0.11103012412786484, -0.16180108487606049, -0.02236912027001381 ] ], [ [ 0.12564800679683685, -0.10915327072143555, -0.053323157131671906 ], [ 0.06013397499918938, 0.07818517088890076, 0.04390117898583412 ], [ 0.05919521674513817, -0.05086110159754753, 0.11842437833547592 ] ], [ [ 0.009561538696289062, 0.13100337982177734, -0.0885411724448204 ], [ 0.0943290963768959, 0.007461306173354387, -0.08001946657896042 ], [ 0.12241971492767334, 0.1626623272895813, -0.17123454809188843 ] ], [ [ 0.006828694604337215, 6.963577470742166e-05, -0.17743292450904846 ], [ -0.18081223964691162, 0.020155085250735283, -0.12944959104061127 ], [ -0.10669642686843872, -0.1092572808265686, -0.01022006943821907 ] ], [ [ 0.08811519294977188, -0.010638792999088764, 0.037426482886075974 ], [ 0.029551351442933083, 0.10919863730669022, -0.17812275886535645 ], [ -0.01921534352004528, -0.020105307921767235, -0.017948994413018227 ] ], [ [ -0.13676193356513977, -0.06529264152050018, -0.03060491383075714 ], [ -0.16058501601219177, 0.22819195687770844, 0.17382000386714935 ], [ 0.02250857651233673, 0.1810695230960846, 0.21420879662036896 ] ] ] }, { "bias": 0.3483165502548218, "weights": [ [ [ 0.032918818295001984, 0.16698546707630157, 0.18808385729789734 ], [ -0.18842415511608124, -0.19741851091384888, -0.3181258738040924 ], [ 0.14529986679553986, -0.0633912906050682, -0.1235964298248291 ] ], [ [ -0.17358721792697906, -0.14975450932979584, -0.12765149772167206 ], [ -0.1116621121764183, 0.061584655195474625, -0.0584794245660305 ], [ -0.0361844040453434, -0.0742720440030098, -0.08337784558534622 ] ], [ [ -0.16112861037254333, -0.20642298460006714, 0.18931011855602264 ], [ -0.12355746328830719, -0.1905362457036972, 0.071263886988163 ], [ 0.05011044070124626, 0.07991564273834229, 0.07828167080879211 ] ], [ [ -0.06476299464702606, 0.05114796757698059, -0.1346709281206131 ], [ 0.11902449280023575, -0.16854026913642883, 0.12135276943445206 ], [ -0.19429254531860352, 0.16592955589294434, 0.11022176593542099 ] ], [ [ 0.08290845155715942, -0.08158741891384125, 0.1433313637971878 ], [ 3.642140654847026e-05, -0.07293203473091125, 0.07498873025178909 ], [ 0.14000684022903442, -0.16949677467346191, -0.12858793139457703 ] ], [ [ 0.14515793323516846, -0.11156363785266876, 0.18032991886138916 ], [ 0.0346059650182724, 0.022723345085978508, 0.1322818100452423 ], [ 0.14743609726428986, -0.030711062252521515, 0.03786199167370796 ] ], [ [ 0.10720982402563095, -0.11306586116552353, 0.0857093334197998 ], [ -0.11355438083410263, -0.14333486557006836, 0.03453052043914795 ], [ -0.004926359746605158, 0.039875756949186325, -0.11913741379976273 ] ], [ [ 0.294634073972702, 0.2950073778629303, 0.21420356631278992 ], [ -0.21549411118030548, -0.017663640901446342, -0.14800578355789185 ], [ 0.1540672779083252, 0.13856224715709686, -0.07062943279743195 ] ], [ [ 0.09850916266441345, -0.024218950420618057, -0.05085296928882599 ], [ -0.1842269003391266, 0.048921141773462296, -0.15639637410640717 ], [ 0.15068410336971283, 0.1781669557094574, 0.13751555979251862 ] ], [ [ -0.3126057982444763, -0.3406130373477936, -0.03664514049887657 ], [ -0.19378718733787537, -0.2823072373867035, -0.24569031596183777 ], [ -0.011888631619513035, -0.2881462574005127, -0.16774912178516388 ] ] ] }, { "bias": 0.04205586388707161, "weights": [ [ [ -0.050872303545475006, -0.05784498527646065, 0.08709856122732162 ], [ -0.02384408563375473, -0.09397155791521072, 0.03822978958487511 ], [ 0.07812274247407913, -0.041850049048662186, 0.0914970338344574 ] ], [ [ 0.20612703263759613, 0.10431799292564392, 0.09797626733779907 ], [ -0.08422691375017166, -0.12570352852344513, 0.03475036844611168 ], [ 0.10499346256256104, 0.17909479141235352, -0.07103309035301208 ] ], [ [ -0.0803171768784523, 0.14055944979190826, -0.0013921728823333979 ], [ -0.05157731845974922, 0.14367206394672394, 0.16281574964523315 ], [ 0.1779191792011261, 0.17450547218322754, -0.02203560434281826 ] ], [ [ -0.17883086204528809, -0.04414394870400429, -0.06294535100460052 ], [ 0.0421249195933342, -0.15779680013656616, 0.08037769794464111 ], [ -0.2165309339761734, 0.14304062724113464, -0.13399317860603333 ] ], [ [ 0.15507706999778748, 0.132803812623024, 0.011086942628026009 ], [ 0.17226529121398926, 0.12231457978487015, 0.011460813693702221 ], [ -0.05953741818666458, 0.009429561905562878, 0.17142777144908905 ] ], [ [ 0.06041353940963745, 0.05031156539916992, 0.12597262859344482 ], [ 0.10787922143936157, -0.04708215966820717, 0.046055953949689865 ], [ -0.1617310792207718, 0.16448339819908142, 0.10370638221502304 ] ], [ [ -0.07778611779212952, -0.07144667208194733, 0.021185168996453285 ], [ -0.014000819995999336, -0.1030643954873085, 0.0849083736538887 ], [ -0.14547759294509888, -0.13143889605998993, -0.14898383617401123 ] ], [ [ -0.10427332669496536, 0.013692626729607582, -0.041272081434726715 ], [ 0.04340681806206703, 0.011249948292970657, -0.06556360423564911 ], [ 0.1761879026889801, -0.22885175049304962, 0.22462105751037598 ] ], [ [ -0.09201741963624954, -0.12168310582637787, 0.02565200999379158 ], [ 0.004468440543860197, -0.10384641587734222, 0.08242103457450867 ], [ -0.10442862659692764, 0.1434611976146698, -0.0901908352971077 ] ], [ [ 0.13533282279968262, 0.22042447328567505, -0.0668957456946373 ], [ -0.07557129114866257, 0.05515684932470322, 0.23244601488113403 ], [ 0.19463591277599335, 0.07740461826324463, 0.13794946670532227 ] ] ] }, { "bias": 0.014336160384118557, "weights": [ [ [ 0.0215923935174942, -0.16690650582313538, 0.015022673644125462 ], [ 0.2356853038072586, -0.16496746242046356, 0.13733810186386108 ], [ -0.013798801228404045, 0.12202651798725128, -0.09092853218317032 ] ], [ [ 0.1390375792980194, 0.054428163915872574, -0.16656725108623505 ], [ -0.09568659216165543, 0.07450474798679352, 0.10316252708435059 ], [ -0.05642378702759743, 0.11634775251150131, 0.040672823786735535 ] ], [ [ -0.09930086135864258, 0.14880722761154175, -0.022999756038188934 ], [ 0.07259044796228409, -0.20595431327819824, 0.10381084680557251 ], [ -0.0962037593126297, 0.057415831834077835, 0.11492513865232468 ] ], [ [ -0.07774700969457626, 0.17856010794639587, -0.060527559369802475 ], [ 0.025360064581036568, 0.03321215882897377, 0.024507902562618256 ], [ -0.10944100469350815, -0.02191023901104927, -0.15557676553726196 ] ], [ [ -0.11318005621433258, -0.10387733578681946, 0.016798585653305054 ], [ -0.1566755175590515, -0.05321263521909714, -0.058401837944984436 ], [ 0.019487643614411354, 0.12416136264801025, 0.16156084835529327 ] ], [ [ 0.17321249842643738, 0.15694253146648407, -0.1380874663591385 ], [ 0.08091419190168381, -0.026859184727072716, 0.055830758064985275 ], [ -0.08057769387960434, -0.03078553080558777, -0.14363344013690948 ] ], [ [ 0.051236167550086975, 0.167289599776268, 0.02166205458343029 ], [ 0.0026116727385669947, 0.14677225053310394, -0.13095352053642273 ], [ -0.005355099681764841, -0.14963918924331665, 0.16254675388336182 ] ], [ [ 0.09966745227575302, 0.1953197717666626, 0.1834649294614792 ], [ -0.04643874615430832, 0.003983447328209877, -0.1375250369310379 ], [ 0.13853110373020172, 0.02780631184577942, -0.021691713482141495 ] ], [ [ -0.17471736669540405, -0.003995680715888739, 0.0908963605761528 ], [ 0.05729997158050537, -0.03710588812828064, 0.14208091795444489 ], [ 0.017629601061344147, 0.14826329052448273, -0.04206182062625885 ] ], [ [ 0.04621823504567146, 0.19993871450424194, 0.12714999914169312 ], [ 0.2284945249557495, 0.06086286902427673, -0.09190674126148224 ], [ -0.014598064124584198, 0.09851402789354324, -0.038618091493844986 ] ] ] }, { "bias": 0.10236959904432297, "weights": [ [ [ 0.13946866989135742, 0.09682520478963852, 0.1726500242948532 ], [ 0.1549411416053772, 0.20717331767082214, 0.15288777649402618 ], [ 0.2388300597667694, 0.20189401507377625, 0.2861592471599579 ] ], [ [ 0.1471264809370041, -0.15974710881710052, -0.15733277797698975 ], [ 0.12168208509683609, 0.046046722680330276, 0.06857374310493469 ], [ 0.09625067561864853, 0.02841225638985634, -0.18588554859161377 ] ], [ [ 0.21173368394374847, -0.1557256281375885, 0.19390174746513367 ], [ 0.09824199974536896, -0.03420214727520943, -0.008039628155529499 ], [ -0.07244376838207245, 0.12665802240371704, -0.14270643889904022 ] ], [ [ -0.10829263925552368, -0.21082808077335358, -0.018086524680256844 ], [ -0.18155768513679504, -0.11288563162088394, -0.27771344780921936 ], [ -0.07960627973079681, -0.23326782882213593, 0.029522554948925972 ] ], [ [ 0.1645551472902298, 0.07067253440618515, -0.15274055302143097 ], [ 0.09972383826971054, 0.05966971442103386, 0.05424782633781433 ], [ 0.09747873246669769, 0.17227253317832947, 0.07710618525743484 ] ], [ [ -0.142201766371727, 0.06311608850955963, -0.09166456758975983 ], [ 0.036629579961299896, -0.16250286996364594, -0.103266641497612 ], [ 0.14275093376636505, -0.09976468980312347, -0.11892399936914444 ] ], [ [ -0.062094107270240784, -0.07857262343168259, -0.06865912675857544 ], [ 0.14247825741767883, 0.017749130725860596, -0.17575152218341827 ], [ 0.034957896918058395, -0.18680624663829803, -0.07339288294315338 ] ], [ [ 0.15000995993614197, 0.15952040255069733, 0.025392794981598854 ], [ 0.008648335002362728, 0.05714226886630058, 0.10262136161327362 ], [ -0.02552706189453602, 0.24016767740249634, -0.07566685974597931 ] ], [ [ -0.0037647292483597994, -0.07180869579315186, -0.10357300192117691 ], [ -0.13562867045402527, -0.18133904039859772, 0.10665327310562134 ], [ -0.1242426410317421, -0.08196112513542175, 0.015137712471187115 ] ], [ [ -0.08718607574701309, -0.11071056127548218, -0.19932056963443756 ], [ 0.12107475847005844, 0.031282905489206314, -0.008140921592712402 ], [ -0.10637729614973068, 0.05876592919230461, -0.14555838704109192 ] ] ] }, { "bias": 0.09469860047101974, "weights": [ [ [ -0.2105904221534729, 0.0822480246424675, 0.1040528193116188 ], [ -0.4372435510158539, -0.1394747942686081, -0.13298483192920685 ], [ 0.07332262396812439, 0.3908269703388214, 0.38406169414520264 ] ], [ [ -0.045297324657440186, 0.09923484921455383, 0.009942020289599895 ], [ -0.10473351180553436, 0.1444244235754013, 0.036649953573942184 ], [ 0.05448807030916214, -0.1387009620666504, -0.09983030706644058 ] ], [ [ -0.16015511751174927, 0.11032894253730774, 0.07853087782859802 ], [ -0.3843325972557068, -0.12732912600040436, -0.13995136320590973 ], [ -0.12522496283054352, 0.1591143161058426, -0.1607891321182251 ] ], [ [ -0.18878810107707977, -0.02027733065187931, -0.17432159185409546 ], [ 0.09949029237031937, 0.1345290243625641, 0.33093181252479553 ], [ -0.0773019939661026, -0.12286946922540665, -0.07437783479690552 ] ], [ [ -0.05076870322227478, -0.17143796384334564, -0.02844293601810932 ], [ -0.056532178074121475, -0.05581158027052879, 0.13571591675281525 ], [ -0.0011971743078902364, 0.1389237940311432, 0.045508045703172684 ] ], [ [ -0.14648418128490448, 0.008594298735260963, 0.06316842883825302 ], [ -0.1327832192182541, 0.05533916503190994, 0.0405786857008934 ], [ 0.016748623922467232, -0.0036282374057918787, -0.06971527636051178 ] ], [ [ -0.06318449974060059, -0.04867660999298096, 0.0033905047457665205 ], [ -0.14766040444374084, 0.07115902006626129, -0.1082526445388794 ], [ 0.18094147741794586, 0.075848788022995, 0.010570191778242588 ] ], [ [ -0.4461718201637268, -0.12685231864452362, -0.09470995515584946 ], [ -0.37245672941207886, 0.06401273608207703, -0.10587650537490845 ], [ 0.1323092132806778, 0.2841692864894867, 0.3288729190826416 ] ], [ [ 0.11332128196954727, 0.07668281346559525, 0.17105501890182495 ], [ 0.07676290720701218, 0.18461892008781433, -0.030073756352066994 ], [ -0.058217138051986694, 0.06935951113700867, -0.10205817967653275 ] ], [ [ 0.0596625953912735, 0.11606793105602264, -0.03409811481833458 ], [ -0.15589463710784912, -0.03166348114609718, -0.09652978181838989 ], [ -0.03323217108845711, 0.14403294026851654, 0.047793734818696976 ] ] ] }, { "bias": 0.01194368302822113, "weights": [ [ [ -0.018009334802627563, 0.03910083323717117, -0.12360009551048279 ], [ -0.16260863840579987, 0.06422073394060135, 0.1096610575914383 ], [ -0.09977766126394272, 0.08435740321874619, -0.15145544707775116 ] ], [ [ -0.15748460590839386, -0.02257942594587803, -0.12646594643592834 ], [ -0.17101167142391205, -0.10232622921466827, 0.060039620846509933 ], [ -0.037509892135858536, 0.13630206882953644, 0.13288167119026184 ] ], [ [ 0.17928721010684967, 0.06221248582005501, -0.1642381101846695 ], [ 0.15033742785453796, -0.12057711184024811, -0.1177884042263031 ], [ -0.029275977984070778, 0.13740703463554382, 0.0059226457960903645 ] ], [ [ -0.08302946388721466, -0.09800400584936142, -0.10539329797029495 ], [ -0.10979030281305313, 0.12786464393138885, 0.15422867238521576 ], [ 0.1178753450512886, 0.037778858095407486, -0.13282115757465363 ] ], [ [ 0.049398407340049744, -0.09233351051807404, 0.020540276542305946 ], [ -0.07510533183813095, -0.0051919217221438885, 0.03071579709649086 ], [ 0.06637870520353317, 0.14142747223377228, 0.0987064316868782 ] ], [ [ -0.06529685854911804, 0.15087558329105377, 0.1440766304731369 ], [ -0.09470537304878235, 0.0037186474073678255, 0.06637263298034668 ], [ -0.12547937035560608, 0.16972284018993378, 0.02478511817753315 ] ], [ [ 0.1591171771287918, 0.13206999003887177, -0.128485769033432 ], [ -0.06297139078378677, 0.07761325687170029, 0.1717512309551239 ], [ 0.006974450778216124, 0.10103538632392883, 0.15904146432876587 ] ], [ [ -0.0991336777806282, 0.10918153822422028, -0.1413096934556961 ], [ 0.12189608812332153, -0.030078722164034843, 0.16664233803749084 ], [ -0.03848503902554512, -0.044872891157865524, -0.05667019262909889 ] ], [ [ -0.05376162379980087, 0.12212453037500381, 0.001853001187555492 ], [ 0.007966096512973309, -0.052032239735126495, 0.08911513537168503 ], [ 0.10235605388879776, -0.1754452884197235, 0.11022020131349564 ] ], [ [ 0.0523245595395565, -0.08961309492588043, 0.028888678178191185 ], [ -0.182817280292511, -0.14101825654506683, 0.07048813253641129 ], [ -0.08123685419559479, 0.020566679537296295, 0.1554270088672638 ] ] ] } ] }, { "name": "relu_1_2", "input_shape": [ 60, 60, 10 ], "output_shape": [ 60, 60, 10 ], "num_neurons": 10 }, { "name": "max_pool_1", "input_shape": [ 60, 60, 10 ], "output_shape": [ 30, 30, 10 ], "num_neurons": 10 }, { "name": "conv_2_1", "input_shape": [ 30, 30, 10 ], "output_shape": [ 28, 28, 10 ], "num_neurons": 10, "weights": [ { "bias": 0.23078683018684387, "weights": [ [ [ 0.012396186590194702, 0.106514573097229, -0.059165868908166885 ], [ 0.0473598875105381, -0.06922586262226105, -0.010398129001259804 ], [ -0.10683242231607437, 0.16645176708698273, -0.027350030839443207 ] ], [ [ 0.10285908728837967, 0.01539174560457468, 0.2610231935977936 ], [ 0.09296361356973648, 0.18699924647808075, 0.16006100177764893 ], [ 0.23022422194480896, 0.10022423416376114, 0.12351842224597931 ] ], [ [ -0.11672525107860565, 0.04385579749941826, -0.1581880748271942 ], [ 0.06049090251326561, -0.09956669807434082, -0.054375164210796356 ], [ 0.02838168479502201, 0.19887058436870575, 0.07822155952453613 ] ], [ [ -0.011373915709555149, 0.23057536780834198, -0.05637764558196068 ], [ 0.12049947679042816, 0.23012472689151764, -0.07205697149038315 ], [ 0.13936983048915863, 0.16423775255680084, 0.1870001256465912 ] ], [ [ 0.25594502687454224, 0.2258918136358261, -0.029594261199235916 ], [ 0.21196569502353668, 0.015812525525689125, 0.0857214406132698 ], [ 0.08068202435970306, -0.11682596802711487, -0.14986300468444824 ] ], [ [ -0.03351468965411186, -0.027632812038064003, 0.2219613939523697 ], [ 0.23095473647117615, 0.2543334364891052, 0.11295383423566818 ], [ 0.2238248735666275, 0.14109322428703308, 0.28218019008636475 ] ], [ [ -0.0726231187582016, 0.15335825085639954, -0.1863558143377304 ], [ -0.04176076874136925, -0.07777108252048492, -0.13225321471691132 ], [ 0.11718687415122986, 0.2275283932685852, -0.09361421316862106 ] ], [ [ -0.2195308804512024, -0.06082876771688461, 0.03180938959121704 ], [ -0.04475773125886917, -0.1764926314353943, -0.19074630737304688 ], [ 0.02116229757666588, -0.10730506479740143, -0.12850704789161682 ] ], [ [ -0.18486733734607697, -0.1645059436559677, -0.011868911795318127 ], [ -0.2320302575826645, -0.2014266848564148, -0.0683027133345604 ], [ -0.052557531744241714, -0.09631044417619705, -0.05061513930559158 ] ], [ [ -0.12093799561262131, -0.02727758139371872, -0.003536903066560626 ], [ -0.17244760692119598, 0.15485739707946777, 0.08143389225006104 ], [ 0.16484986245632172, -0.09964308887720108, -0.08044561743736267 ] ] ] }, { "bias": -0.057308949530124664, "weights": [ [ [ -0.09050483256578445, 0.12815767526626587, 0.16287687420845032 ], [ 0.183803990483284, 0.20328572392463684, 0.12396799027919769 ], [ -9.452683298150077e-05, -0.08443509787321091, 0.10762670636177063 ] ], [ [ -0.1335304081439972, -0.13950252532958984, -0.21879605948925018 ], [ -0.24052183330059052, 0.03482829034328461, -0.2282852828502655 ], [ -0.22395604848861694, -0.0027982203755527735, 0.060258373618125916 ] ], [ [ 0.07878094166517258, 0.01724150963127613, 0.0008939485996961594 ], [ 0.11977328360080719, -0.18965256214141846, 0.030577486380934715 ], [ 0.13316135108470917, 0.12767747044563293, 0.10731510818004608 ] ], [ [ 0.183749720454216, -0.06252773851156235, -0.1400405764579773 ], [ -0.14373596012592316, -0.09674247354269028, -0.13893984258174896 ], [ -0.06367962062358856, 0.14348967373371124, -0.010082925669848919 ] ], [ [ 0.050250597298145294, -0.09106949716806412, 0.043723855167627335 ], [ -0.22382380068302155, -0.06815892457962036, -0.24148085713386536 ], [ -0.22215908765792847, 0.13405835628509521, -0.03889786824584007 ] ], [ [ -0.029541397467255592, 0.039604585617780685, -0.06830139458179474 ], [ 0.08915697783231735, -0.14642280340194702, 0.010759814642369747 ], [ 0.1909075230360031, 0.06867215782403946, 0.19335366785526276 ] ], [ [ -0.14572195708751678, 0.08272267132997513, -0.06870009750127792 ], [ -0.07135029137134552, -0.08967690169811249, 0.17823578417301178 ], [ -0.016790935769677162, -0.0723540410399437, -0.12720520794391632 ] ], [ [ -0.07198317348957062, -0.08225284516811371, -0.06538373976945877 ], [ 0.009972434490919113, 0.08435807377099991, 0.21338078379631042 ], [ 0.21013794839382172, 0.27571287751197815, 0.16767624020576477 ] ], [ [ -0.09659308195114136, -0.08892834931612015, 0.04783710464835167 ], [ 0.07307176291942596, 0.1534288227558136, 0.08913584798574448 ], [ -0.09973634779453278, -0.15064027905464172, -0.046413201838731766 ] ], [ [ 0.06449321657419205, -0.06896071135997772, 0.02535885013639927 ], [ 0.12327052652835846, -0.053086407482624054, 0.07709107547998428 ], [ 0.17837822437286377, 0.03317869082093239, -0.1522664576768875 ] ] ] }, { "bias": -0.026059698313474655, "weights": [ [ [ -0.15007951855659485, -0.0826105996966362, 0.06387073546648026 ], [ 0.0058092703111469746, -0.14636841416358948, -0.149959534406662 ], [ 0.13165725767612457, -0.040089916437864304, -0.17642652988433838 ] ], [ [ 0.10313130170106888, -0.07163388282060623, -0.14289669692516327 ], [ -0.1887245625257492, 0.12385864555835724, 0.1596568524837494 ], [ -0.08863251656293869, -0.013152513653039932, -0.0032642274163663387 ] ], [ [ 0.149405837059021, -0.11012585461139679, 0.019091080874204636 ], [ 0.009260197170078754, 0.13162799179553986, -0.017952140420675278 ], [ -0.14289560914039612, -0.11887412518262863, -0.0405728742480278 ] ], [ [ -0.04376418888568878, -0.11715403199195862, 0.0011360639473423362 ], [ -0.16810937225818634, 0.144540473818779, -0.1474577635526657 ], [ 0.0034044382628053427, -0.14074952900409698, 0.007462761830538511 ] ], [ [ 0.1627795696258545, 0.024526961147785187, 0.063499316573143 ], [ 0.151484414935112, -0.03054751269519329, -0.15957514941692352 ], [ 0.07261790335178375, 0.07631902396678925, 0.057671498507261276 ] ], [ [ 0.04817947372794151, 0.04407208412885666, 0.112233966588974 ], [ 0.10323022305965424, 0.08272760361433029, -0.17297306656837463 ], [ -0.10106699913740158, -0.011175623163580894, 0.08249827474355698 ] ], [ [ 0.02647663652896881, -0.17062802612781525, 0.1257237046957016 ], [ 0.04216809570789337, -0.10751289874315262, 0.02213270217180252 ], [ -0.17760638892650604, 0.058295294642448425, 0.015924541279673576 ] ], [ [ -0.024231404066085815, -0.03134108707308769, 0.16960833966732025 ], [ -0.042533110827207565, -0.13449126482009888, 0.0006716327625326812 ], [ -0.18493595719337463, 0.09931904077529907, -0.08988532423973083 ] ], [ [ -0.1356506645679474, -0.06827053427696228, 0.17023585736751556 ], [ -0.028375260531902313, -0.08702178299427032, -0.1335667222738266 ], [ -0.061089497059583664, -0.09718924760818481, -0.08918967097997665 ] ], [ [ 0.10196224600076675, -0.18089181184768677, 0.14458014070987701 ], [ -0.009191855788230896, 0.14005054533481598, -0.08165542781352997 ], [ -0.1418958604335785, 0.0806783139705658, -0.030212635174393654 ] ] ] }, { "bias": -0.00868820771574974, "weights": [ [ [ -0.15327812731266022, -0.15684106945991516, -0.09862995147705078 ], [ -0.1392715722322464, 0.02297556959092617, -0.049294907599687576 ], [ -0.016246501356363297, -0.0766921415925026, -0.12175631523132324 ] ], [ [ 0.16165898740291595, -0.047843266278505325, -0.03394342213869095 ], [ 0.16841118037700653, -0.1869945377111435, 0.015181371010839939 ], [ -0.04449327290058136, -0.04168952628970146, -0.02938416227698326 ] ], [ [ -0.09043948352336884, -0.12080211937427521, 0.13188543915748596 ], [ 0.18012335896492004, -0.08274321258068085, -0.1497965157032013 ], [ -0.10115096718072891, 0.002968881046399474, -0.15274283289909363 ] ], [ [ 0.12413588166236877, 0.15294747054576874, -0.1342247724533081 ], [ 0.015276852063834667, -0.1625896394252777, -0.15562738478183746 ], [ -0.15663205087184906, -0.1001645028591156, -0.15538941323757172 ] ], [ [ 0.09208566695451736, -0.011178149841725826, -0.16122280061244965 ], [ 0.10234502702951431, 0.10364406555891037, -0.08907386660575867 ], [ 0.0834117978811264, 0.005532237701117992, 0.06636765599250793 ] ], [ [ 0.16093949973583221, -0.02300426922738552, -0.1140693873167038 ], [ -0.1308731585741043, 0.08393826335668564, -0.15887396037578583 ], [ 0.04560978710651398, 0.07385460287332535, 0.16902343928813934 ] ], [ [ -0.07676400244235992, 0.07403796911239624, 0.022018754854798317 ], [ 0.002957345684990287, 0.1573934704065323, 0.036665670573711395 ], [ 0.010597487911581993, -0.12240803241729736, -0.032673951238393784 ] ], [ [ 0.04240453243255615, -0.11351556330919266, -0.04438462108373642 ], [ 0.006118774879723787, -0.015009335242211819, -0.08410405367612839 ], [ -0.0913618803024292, 0.03194887191057205, 0.08386318385601044 ] ], [ [ -0.15381479263305664, -0.11809832602739334, 0.12145192921161652 ], [ -0.05429745092988014, 0.1322549730539322, 0.03329390659928322 ], [ -0.13567295670509338, -0.10557886958122253, 0.09567809104919434 ] ], [ [ 0.10477942228317261, -0.0323621928691864, -0.052374619990587234 ], [ -0.1321583241224289, -0.056856222450733185, 0.003001779317855835 ], [ 0.13107536733150482, 0.1097632497549057, -0.04634663462638855 ] ] ] }, { "bias": 0.10796646773815155, "weights": [ [ [ -0.1507304608821869, 0.026182429865002632, 0.1444566398859024 ], [ -0.08902022987604141, 0.10641860216856003, -0.13652002811431885 ], [ -0.039226654917001724, -0.022110262885689735, 0.12704786658287048 ] ], [ [ 0.3481782078742981, 0.047287702560424805, 0.04006631299853325 ], [ 0.011805927380919456, 0.01431749016046524, -0.09476961940526962 ], [ 0.1401517689228058, -0.13059474527835846, 0.11255893856287003 ] ], [ [ 0.11205146461725235, 0.206292986869812, 0.377199649810791 ], [ -0.028613677248358727, 0.1723896861076355, 0.0018812445923686028 ], [ 0.11386403441429138, -0.06011408194899559, 0.02250262349843979 ] ], [ [ -0.05323943868279457, 0.0508950911462307, 0.29620838165283203 ], [ 0.17404206097126007, -0.13460589945316315, 0.09607753902673721 ], [ 0.030427904799580574, 0.10790881514549255, -0.0872771218419075 ] ], [ [ -0.2658837139606476, -0.1246415376663208, -0.1741020381450653 ], [ 0.009110961109399796, -0.2966923415660858, 0.01331463735550642 ], [ -0.3045470714569092, -0.0006362893618643284, -0.3392437696456909 ] ], [ [ 0.058949440717697144, 0.048762403428554535, 0.09043963253498077 ], [ -0.14432676136493683, 0.1221139207482338, 0.07445020973682404 ], [ -0.0939536914229393, 0.11804372817277908, 0.11689592897891998 ] ], [ [ 0.0008243181509897113, 0.20011498034000397, -0.0012665813555940986 ], [ -0.030231043696403503, 0.03821808844804764, 0.18870511651039124 ], [ 0.12854346632957458, 0.10788316279649734, 0.23676030337810516 ] ], [ [ -0.04490512236952782, -0.1087682768702507, -0.04493439570069313 ], [ 0.19902750849723816, 0.11878892034292221, 0.34698039293289185 ], [ -0.0190410278737545, 0.15304304659366608, 0.2643662095069885 ] ], [ [ -0.0857265517115593, -0.07916845381259918, -0.09775075316429138 ], [ -0.19191114604473114, 0.02176794782280922, 0.05157535523176193 ], [ 0.04070887342095375, 0.03840877115726471, -0.2585756778717041 ] ], [ [ 0.043157365173101425, -0.1509348303079605, -0.004716380499303341 ], [ -0.03060164675116539, 0.0992133617401123, -0.1614258885383606 ], [ -0.0033713087905198336, 0.10944762825965881, -0.16037793457508087 ] ] ] }, { "bias": 0.05496266856789589, "weights": [ [ [ -0.13509221374988556, -0.08933935314416885, 0.14709274470806122 ], [ 0.05901208147406578, 0.06928139925003052, 0.07829902321100235 ], [ 0.002249462064355612, -0.18047143518924713, 0.03880225494503975 ] ], [ [ 0.030934272333979607, -0.02754145674407482, 0.054108597338199615 ], [ 0.25470981001853943, 0.10047401487827301, 0.14475074410438538 ], [ 0.11149384081363678, -0.048276420682668686, 0.00662165367975831 ] ], [ [ 0.05291430652141571, 0.19186468422412872, 0.07003743201494217 ], [ -0.11174073815345764, 0.1881842464208603, 0.1360943466424942 ], [ 0.07475761324167252, -0.07336270064115524, -0.10000384598970413 ] ], [ [ -0.07975286990404129, -0.24706263840198517, -0.26596394181251526 ], [ -0.23612700402736664, -0.01565210521221161, 0.07619105279445648 ], [ -0.04895380511879921, -0.2918459177017212, -0.18951596319675446 ] ], [ [ 0.14410583674907684, -0.035015467554330826, 0.1238742396235466 ], [ -0.11094899475574493, 0.2838386595249176, 0.1437252163887024 ], [ -0.026335369795560837, -0.02288086898624897, 0.10518255829811096 ] ], [ [ -0.1502123922109604, -0.1990748941898346, -0.12756723165512085 ], [ 0.03942368924617767, -0.11737325042486191, 0.08303513377904892 ], [ -0.05691595748066902, 0.05863361805677414, 0.018958356231451035 ] ], [ [ 0.01500424649566412, -0.13895905017852783, -0.019347142428159714 ], [ 0.13329140841960907, 0.1639011800289154, 0.009016373194754124 ], [ -0.13765521347522736, -0.084888756275177, 0.06924717873334885 ] ], [ [ -0.20339468121528625, 0.0933724194765091, 0.043660301715135574 ], [ -0.06258143484592438, -0.012316839769482613, 0.1963297426700592 ], [ 0.10155200958251953, 0.2652052640914917, 0.3104066252708435 ] ], [ [ -0.3128395676612854, -0.29480090737342834, -0.24699248373508453 ], [ -0.12206172943115234, -0.19278986752033234, -0.3234478533267975 ], [ 0.03785733878612518, -0.26259225606918335, -0.30989569425582886 ] ], [ [ -0.09871251881122589, -0.12416747212409973, 0.047914400696754456 ], [ 0.16389912366867065, 0.011755961924791336, -0.017375703901052475 ], [ 0.08810996264219284, -0.03147492557764053, -0.08156110346317291 ] ] ] }, { "bias": -0.03608671948313713, "weights": [ [ [ 0.07521034777164459, 0.032540932297706604, -0.1476229876279831 ], [ -0.10402955114841461, -0.10514653474092484, 0.10495533049106598 ], [ 0.1492186337709427, -0.12216785550117493, 0.07365602254867554 ] ], [ [ -0.03775200992822647, -0.09271999448537827, -0.13640622794628143 ], [ 0.007787204813212156, 0.13475951552391052, 0.08501224964857101 ], [ 0.038410719484090805, -0.18841730058193207, -0.07372402399778366 ] ], [ [ -0.06593351066112518, -0.06819465011358261, 0.10519562661647797 ], [ -0.07410232722759247, 0.09198575466871262, 0.17219869792461395 ], [ 0.060195110738277435, -0.015554492361843586, 0.15317052602767944 ] ], [ [ -0.10770566016435623, -0.0653575137257576, 0.16414234042167664 ], [ -0.10187935829162598, -0.04773027449846268, 0.038777489215135574 ], [ 0.05748850852251053, 0.041632648557424545, 0.062476158142089844 ] ], [ [ -0.025755811482667923, -0.07449911534786224, 0.032723233103752136 ], [ -0.1973990947008133, -0.1767793446779251, -0.06521757692098618 ], [ 0.029782965779304504, 0.02546435222029686, -0.03705110400915146 ] ], [ [ -0.09324422478675842, -0.14537101984024048, 0.1318194717168808 ], [ 0.00044559736852534115, -0.12903304398059845, -0.12332028895616531 ], [ 0.14647816121578217, -0.10266920924186707, 0.14999932050704956 ] ], [ [ -0.06336613744497299, 0.08409980684518814, -0.08580014109611511 ], [ 0.1320841759443283, -0.1531745195388794, -0.1763831228017807 ], [ 0.04841432720422745, -0.013923467136919498, 0.08944347500801086 ] ], [ [ -0.16080684959888458, 0.04525016248226166, 0.0414198562502861 ], [ -0.14402882754802704, -0.1412883847951889, 0.055425647646188736 ], [ 0.017574450001120567, 0.18039840459823608, -0.009708801284432411 ] ], [ [ -0.13676346838474274, 0.014800326898694038, 0.05060349404811859 ], [ -0.04449760541319847, 0.025106903165578842, 0.0704495757818222 ], [ 0.10736533999443054, -0.050611693412065506, -0.014067563228309155 ] ], [ [ 0.037774667143821716, -0.1306772530078888, -0.13443945348262787 ], [ 0.005202122963964939, -0.09896185249090195, 0.16476544737815857 ], [ -0.057337626814842224, 0.060669589787721634, -0.07003173232078552 ] ] ] }, { "bias": 0.009175595827400684, "weights": [ [ [ 0.1372649222612381, 0.07310942560434341, 0.09293381869792938 ], [ -0.01409513596445322, 0.18146024644374847, 0.19604869186878204 ], [ -0.008923155255615711, -0.19196783006191254, -0.13125421106815338 ] ], [ [ 0.014929680153727531, -0.09620988368988037, -0.15984515845775604 ], [ 0.09972062706947327, -0.022798551246523857, -0.055318545550107956 ], [ -0.17736919224262238, -0.1110055148601532, -0.1602240949869156 ] ], [ [ 0.19213220477104187, 0.09443316608667374, 0.16093586385250092 ], [ 0.0033481072168797255, -0.037993114441633224, -0.12192386388778687 ], [ -0.1685992330312729, -0.14172862470149994, 0.07944667339324951 ] ], [ [ -0.12170552462339401, -0.13767358660697937, 0.07502470165491104 ], [ 0.03369561582803726, -0.15253625810146332, -0.11817822605371475 ], [ -0.100308857858181, -0.016297699883580208, -0.10435254126787186 ] ], [ [ 0.04112238064408302, 0.0213188286870718, 0.11308953911066055 ], [ -0.0010636821389198303, 0.08405053615570068, 0.17462211847305298 ], [ -0.09219279885292053, 0.06761076301336288, -0.008089696988463402 ] ], [ [ -0.061231330037117004, -0.17982029914855957, -0.21145972609519958 ], [ -0.10267185419797897, -0.10252882540225983, 0.08570924401283264 ], [ -0.025686895474791527, 0.16276656091213226, 0.022308165207505226 ] ], [ [ -0.016727356240153313, 0.14001750946044922, 0.12016607820987701 ], [ 0.141238272190094, -0.09624945372343063, -0.1236259713768959 ], [ 0.08482365310192108, -0.150004044175148, 0.09036954492330551 ] ], [ [ 0.08908596634864807, 0.13462431728839874, -0.05469755828380585 ], [ 0.09516725689172745, -0.1146475225687027, 0.20659831166267395 ], [ -0.04286177083849907, 0.029708532616496086, 0.007774696219712496 ] ], [ [ 0.009699777700006962, -0.09435168653726578, 0.06956082582473755 ], [ 0.08944177627563477, -0.14151452481746674, 0.08015299588441849 ], [ 0.1195245310664177, 0.06250646710395813, -0.15702283382415771 ] ], [ [ 0.1309179812669754, -0.05220411345362663, 0.12045758962631226 ], [ -0.13330580294132233, -0.0012973755365237594, 0.1349809318780899 ], [ -0.027604976668953896, -0.16998353600502014, 0.1366959810256958 ] ] ] }, { "bias": -0.003776360535994172, "weights": [ [ [ 0.17279647290706635, -0.06964075565338135, 0.04323156550526619 ], [ -0.025604162365198135, 0.14047738909721375, -0.004479078575968742 ], [ 0.029341932386159897, 0.030494775623083115, -0.1335068941116333 ] ], [ [ 0.09240026026964188, -0.11094912886619568, 0.08517055958509445 ], [ 0.012049105018377304, -0.07688391953706741, 0.1787722110748291 ], [ -0.023024313151836395, -0.1682383269071579, -0.178447887301445 ] ], [ [ -0.1495608389377594, 0.16619254648685455, 0.1342262476682663 ], [ 0.1576603502035141, -0.10944052785634995, -0.01419719960540533 ], [ -0.03726334124803543, -0.04629863426089287, -0.10657647252082825 ] ], [ [ -0.17824728786945343, -0.08168867975473404, 0.025643901899456978 ], [ -0.12964068353176117, 0.09894271939992905, -0.10827117413282394 ], [ -0.16557833552360535, -0.0790194496512413, 0.13079170882701874 ] ], [ [ -0.02129005640745163, -0.14314498007297516, -0.056888509541749954 ], [ 0.135353684425354, 0.08285944163799286, -0.12849228084087372 ], [ 0.07882042229175568, -0.16718867421150208, -0.1280491203069687 ] ], [ [ 0.17641016840934753, -0.09271540492773056, 0.03547276556491852 ], [ -0.11888423562049866, 0.0356455072760582, -0.1658187210559845 ], [ -0.1298179030418396, 0.01567753776907921, 0.054983582347631454 ] ], [ [ -0.01863483153283596, 0.1717471331357956, -0.07820325344800949 ], [ -0.17182089388370514, 0.16218425333499908, -0.018255263566970825 ], [ -0.0573904886841774, -0.07738706469535828, -0.18235725164413452 ] ], [ [ 0.0639929473400116, -0.11592573672533035, 0.060891155153512955 ], [ -0.10414804518222809, 0.11736100167036057, -0.11977871507406235 ], [ 0.051007796078920364, 0.022053631022572517, -0.07404788583517075 ] ], [ [ -0.146500825881958, 0.1362224519252777, 0.02524641901254654 ], [ -0.16801369190216064, -0.02728229947388172, -0.18054360151290894 ], [ 0.16980679333209991, 0.15864145755767822, 0.052760049700737 ] ], [ [ -0.020358631387352943, -0.06838512420654297, -0.09648600220680237 ], [ 0.1290944218635559, 0.11064353585243225, 0.014462131075561047 ], [ 0.047761350870132446, -0.14560990035533905, -0.08718659728765488 ] ] ] }, { "bias": -0.07763895392417908, "weights": [ [ [ 0.2694922685623169, 0.24197611212730408, 0.08383424580097198 ], [ 0.0990048423409462, -0.0034763102885335684, 0.15709368884563446 ], [ 0.13681627810001373, 0.07826804369688034, 0.07874651998281479 ] ], [ [ 0.11787533760070801, 0.23093080520629883, 0.2037479430437088 ], [ 0.01225980930030346, -0.014696580357849598, 0.21061557531356812 ], [ -0.18517103791236877, 0.06154485419392586, -0.10908146947622299 ] ], [ [ 0.32395485043525696, 0.2772344648838043, 0.2637268304824829 ], [ 0.31862667202949524, 0.20011520385742188, -0.022607995197176933 ], [ 0.27067869901657104, 0.25704577565193176, 0.0024352900218218565 ] ], [ [ 0.00820037443190813, 0.1509440690279007, -0.17756050825119019 ], [ -0.15615518391132355, -0.11637229472398758, 0.0665382444858551 ], [ 0.006587250158190727, 0.10080977529287338, -0.036548878997564316 ] ], [ [ 0.2496289610862732, 0.2784920930862427, 0.06407802551984787 ], [ 0.06363044679164886, 0.038830626755952835, 0.0685986652970314 ], [ 0.23036327958106995, 0.08188434690237045, 0.13114432990550995 ] ], [ [ 0.13804379105567932, -0.06537020951509476, 0.07749289274215698 ], [ 0.06075216084718704, -0.10845176130533218, -0.13249245285987854 ], [ -0.28506502509117126, 0.0771365612745285, 0.03315943852066994 ] ], [ [ 0.10850785672664642, 0.19386376440525055, 0.04043261706829071 ], [ -0.11965324729681015, 0.15540647506713867, 0.054148077964782715 ], [ -0.08049509674310684, -0.09168446809053421, -0.03272629529237747 ] ], [ [ -0.15657858550548553, -0.1601806879043579, 0.08438631892204285 ], [ 0.14753501117229462, 0.07282920926809311, 0.08307341486215591 ], [ 0.11728155612945557, -0.2642086446285248, -0.043825458735227585 ] ], [ [ -0.047177258878946304, 0.19377855956554413, 0.23226378858089447 ], [ 0.11053099483251572, -0.07336296886205673, 0.19029884040355682 ], [ -0.07901981472969055, 0.16959023475646973, 0.20988865196704865 ] ], [ [ -0.1105785146355629, 0.022895105183124542, -0.05301348865032196 ], [ -0.11357446014881134, 0.12780886888504028, 0.10286659002304077 ], [ 0.18008482456207275, 0.057069484144449234, -0.1692861020565033 ] ] ] } ] }, { "name": "relu_2_1", "input_shape": [ 28, 28, 10 ], "output_shape": [ 28, 28, 10 ], "num_neurons": 10 }, { "name": "conv_2_2", "input_shape": [ 28, 28, 10 ], "output_shape": [ 26, 26, 10 ], "num_neurons": 10, "weights": [ { "bias": -0.06928090751171112, "weights": [ [ [ 0.08598040044307709, -0.15661418437957764, 0.10458510369062424 ], [ -0.11801018565893173, 0.0066466866992414, -0.07464653998613358 ], [ -0.0566081739962101, -0.07635676115751266, 0.016953084617853165 ] ], [ [ -0.04266742616891861, 0.15755142271518707, -0.08596903830766678 ], [ 0.03977324068546295, 0.17321203649044037, 0.16583320498466492 ], [ -0.1584334522485733, -0.11751523613929749, 0.15463431179523468 ] ], [ [ -0.17958573997020721, 0.1346961259841919, 0.07698938250541687 ], [ -0.17861729860305786, 0.023941045626997948, 0.016261372715234756 ], [ 0.05962218716740608, 0.04584416374564171, -0.11381524801254272 ] ], [ [ 0.11156799644231796, -0.012116417288780212, 0.1565047651529312 ], [ 0.007633236236870289, 0.13187412917613983, -0.15669921040534973 ], [ -0.1256016343832016, -0.04784016311168671, -0.10156258195638657 ] ], [ [ 0.23189041018486023, 0.18846964836120605, 0.09803152829408646 ], [ 0.10253984481096268, 0.2656000554561615, -0.08241341263055801 ], [ 0.10929454118013382, 0.20914490520954132, 0.2639399468898773 ] ], [ [ 0.11014262586832047, 0.12597136199474335, -0.062423381954431534 ], [ 0.23459452390670776, 0.18304942548274994, -0.00461974460631609 ], [ 0.19337813556194305, 0.17780911922454834, 0.15392877161502838 ] ], [ [ 0.16176021099090576, 0.08786827325820923, -0.16233132779598236 ], [ 0.1456843465566635, 0.17537158727645874, 0.13992878794670105 ], [ 0.09141495823860168, 0.1278669387102127, 0.07846090197563171 ] ], [ [ 0.09421935677528381, 0.008094422519207, 0.07347014546394348 ], [ -0.049914367496967316, -0.09917126595973969, 0.031234242022037506 ], [ -0.14336086809635162, 0.15580561757087708, 0.0460781529545784 ] ], [ [ -0.11763220280408859, 0.13204853236675262, 0.04893353208899498 ], [ -0.012086287140846252, 0.018663855269551277, -0.04036339744925499 ], [ 0.005858162883669138, 0.04900277405977249, -0.0608283169567585 ] ], [ [ -0.09456668049097061, 0.004962638486176729, 0.13074404001235962 ], [ 0.016134988516569138, -0.20001815259456635, 0.10673729330301285 ], [ -0.0930432379245758, 0.11103270202875137, -0.05254105478525162 ] ] ] }, { "bias": -0.016896897926926613, "weights": [ [ [ 0.010515295900404453, 0.07174450904130936, 0.03518962115049362 ], [ -0.15565933287143707, 0.17348918318748474, -0.046614233404397964 ], [ -0.10957509279251099, -0.07884623855352402, -0.08424291759729385 ] ], [ [ -0.04069344699382782, -0.018266776576638222, -0.052818767726421356 ], [ -0.12165004014968872, 0.16037918627262115, 0.0028885514475405216 ], [ 0.09751822054386139, -0.008691028691828251, 0.05092262104153633 ] ], [ [ -0.1637870818376541, -0.11834993958473206, 0.03644406422972679 ], [ 0.17490926384925842, -0.16657812893390656, -0.1686854362487793 ], [ -0.00875480379909277, 0.10025767982006073, 0.079607293009758 ] ], [ [ -0.16998125612735748, 0.011822454631328583, 0.10799288004636765 ], [ 0.1708323359489441, 0.11254071444272995, -0.023379087448120117 ], [ -0.09230536222457886, 0.13267692923545837, 0.06022457033395767 ] ], [ [ -0.039116840809583664, -0.09745854884386063, 0.10001355409622192 ], [ -0.058901846408843994, -0.0633816346526146, 0.25829941034317017 ], [ -0.0033515046816319227, -0.10516081750392914, 0.1933746188879013 ] ], [ [ -0.0191354937851429, 0.033260591328144073, 0.10637985169887543 ], [ -0.06287524849176407, -0.16584312915802002, -0.178731769323349 ], [ -0.11020718514919281, -0.26101186871528625, -0.27676376700401306 ] ], [ [ 0.12988366186618805, -0.09975159913301468, -0.03594619408249855 ], [ 0.013879583217203617, -0.014339292421936989, -0.12960761785507202 ], [ -0.029267409816384315, 0.0656919926404953, -0.06838901340961456 ] ], [ [ -0.16180476546287537, 0.13674040138721466, -0.14401094615459442 ], [ 0.12005572766065598, 0.05233161896467209, 0.02275143563747406 ], [ -0.044939711689949036, 0.1748659610748291, -0.027873260900378227 ] ], [ [ 0.14486064016819, 0.11711373925209045, -0.13860833644866943 ], [ 0.003040055511519313, 0.09735389053821564, -0.03913307562470436 ], [ 0.14987336099147797, -0.16979654133319855, 0.05046921595931053 ] ], [ [ 0.33600518107414246, 0.23251022398471832, 0.17040139436721802 ], [ 0.014293010346591473, -0.04002862796187401, 0.18930195271968842 ], [ 0.08930770307779312, 0.11070305109024048, 0.16571563482284546 ] ] ] }, { "bias": 0.0003043923934455961, "weights": [ [ [ -0.03225057199597359, -0.1675012707710266, 0.1059720367193222 ], [ -0.1665038913488388, -0.04912973567843437, -0.062342386692762375 ], [ -0.10942523181438446, -0.12790238857269287, -0.17531943321228027 ] ], [ [ -0.12098591774702072, -0.14656835794448853, 0.12434645742177963 ], [ -0.1253986656665802, -0.06918247044086456, -0.08831476420164108 ], [ -0.10906275361776352, 0.0639200359582901, -0.018018294125795364 ] ], [ [ -0.13535581529140472, -0.025706501677632332, 0.07317788898944855 ], [ -0.1467301845550537, 9.577369200997055e-05, -0.10828758031129837 ], [ 0.0753730982542038, -0.018032625317573547, -0.08529771864414215 ] ], [ [ -0.1071033701300621, -0.14495904743671417, 0.16617079079151154 ], [ -0.05282296612858772, -0.16811032593250275, 0.15499021112918854 ], [ -0.014911152422428131, 0.04684891551733017, -0.02959609217941761 ] ], [ [ 0.06498781591653824, -0.04017537459731102, -0.10274551808834076 ], [ 0.07708664238452911, -0.17044679820537567, 0.009095845744013786 ], [ -0.0026617608964443207, -0.05690290406346321, -0.13626231253147125 ] ], [ [ -0.02382458932697773, -0.16046307981014252, -0.04977470263838768 ], [ 0.027230404317378998, -0.13205501437187195, 0.08445755392313004 ], [ 0.17525137960910797, -0.01425952184945345, 0.05640949308872223 ] ], [ [ -0.011486219242215157, 0.03407638147473335, 0.06504546105861664 ], [ 0.07462088763713837, 0.11084514111280441, -0.17557725310325623 ], [ -0.00040298295789398253, -0.053970303386449814, -0.044276539236307144 ] ], [ [ 0.06198526918888092, 0.13931012153625488, 0.12043576687574387 ], [ -0.1402665078639984, 0.05957089737057686, 0.003116732696071267 ], [ -0.10933493077754974, -0.04451589658856392, -0.11461278051137924 ] ], [ [ 0.17624329030513763, 0.1451093852519989, -0.01907532475888729 ], [ -0.16605983674526215, 0.10669174790382385, -0.0948493480682373 ], [ -0.1267155110836029, -0.12291537970304489, 0.05642882362008095 ] ], [ [ 0.17027094960212708, -0.08252441138029099, -0.10560049116611481 ], [ -0.13899169862270355, -0.01087633054703474, 0.015592963434755802 ], [ -0.05057007446885109, -0.1585644632577896, 0.06018302962183952 ] ] ] }, { "bias": 0.025450965389609337, "weights": [ [ [ -0.14883872866630554, -0.1539710909128189, 0.030757807195186615 ], [ -0.13624624907970428, -0.017219746485352516, 0.0077850897796452045 ], [ 0.07874865084886551, -0.03389166668057442, 0.18048839271068573 ] ], [ [ 0.0376468263566494, 0.11783362925052643, 0.09815100580453873 ], [ 0.10770928859710693, 0.1612078994512558, -0.12880432605743408 ], [ -0.010542704723775387, 0.1984904557466507, -0.11256863176822662 ] ], [ [ 0.06596554070711136, 0.11651577055454254, -0.11643873155117035 ], [ 0.15006743371486664, -0.1437334269285202, 0.1470753401517868 ], [ -0.13797885179519653, -0.018675317987799644, 0.15400108695030212 ] ], [ [ 0.12248498201370239, -0.06398702412843704, 0.18048100173473358 ], [ 0.04396796599030495, 0.002043294021859765, -0.08613097667694092 ], [ 0.1374073177576065, 0.005020493175834417, -0.13388139009475708 ] ], [ [ -0.06050415709614754, -0.11082898080348969, 0.09905088692903519 ], [ 0.143870010972023, 0.15556935966014862, 0.01168390829116106 ], [ -0.030719488859176636, -0.162771075963974, -0.0974450558423996 ] ], [ [ -0.296623170375824, 0.09341170638799667, 0.05299468711018562 ], [ 0.12450848519802094, -0.1708381175994873, -0.05469423532485962 ], [ -0.11729957908391953, -0.13073882460594177, -0.09305202215909958 ] ], [ [ 0.14354491233825684, -0.07621651887893677, 0.1559310257434845 ], [ 0.16547739505767822, 0.04346732795238495, -0.16540241241455078 ], [ -0.14048179984092712, 0.1427590399980545, 0.12544023990631104 ] ], [ [ -0.05909881368279457, 0.17422327399253845, 0.19445350766181946 ], [ 0.054037272930145264, -0.1322638839483261, 0.07989992946386337 ], [ -0.09926623106002808, 0.09489045292139053, 0.0921379029750824 ] ], [ [ -0.13270999491214752, -0.042694009840488434, -0.020755372941493988 ], [ -0.021717816591262817, -0.1382790207862854, 0.12393639981746674 ], [ -0.03843085840344429, -0.052387405186891556, -0.03458016365766525 ] ], [ [ 0.260768860578537, 0.11117256432771683, 0.18648739159107208 ], [ 0.27562054991722107, 0.19844390451908112, 0.2226613461971283 ], [ 0.25097647309303284, 0.11254110932350159, 0.11099715530872345 ] ] ] }, { "bias": 0.08187747746706009, "weights": [ [ [ 0.10404898971319199, 0.08964062482118607, 0.02767273783683777 ], [ -0.21951045095920563, -0.05491212010383606, -0.23909711837768555 ], [ -0.12873464822769165, -0.12541238963603973, -0.016261311247944832 ] ], [ [ 0.05838247761130333, -0.040936172008514404, 0.08199704438447952 ], [ -0.1735961139202118, -0.13730625808238983, 0.17307841777801514 ], [ -0.12099836766719818, -0.10004908591508865, 0.07688338309526443 ] ], [ [ -0.18259228765964508, 0.15632861852645874, 0.006600016262382269 ], [ -0.023014860227704048, 0.07120193541049957, -0.09667934477329254 ], [ 0.16096729040145874, 0.14017324149608612, -0.11965309828519821 ] ], [ [ -0.12984591722488403, 0.11726689338684082, 0.16403205692768097 ], [ -0.08682794868946075, 0.004607963841408491, 0.052352651953697205 ], [ 0.13781727850437164, -0.033284109085798264, -0.13342851400375366 ] ], [ [ 0.17575091123580933, -0.05865838751196861, 0.3193773031234741 ], [ 0.22509527206420898, 0.09300155937671661, 0.16211801767349243 ], [ 0.2365456372499466, 0.09619361162185669, 0.035926446318626404 ] ], [ [ -0.0903039500117302, 0.2372758835554123, 0.23922602832317352 ], [ -0.0025516848545521498, -0.05170881748199463, -0.003260094905272126 ], [ -0.013456434942781925, 0.2358999252319336, 0.027249205857515335 ] ], [ [ 0.06481575965881348, 0.04652019962668419, 0.17423945665359497 ], [ 0.041122306138277054, 0.06162906438112259, -0.12034609913825989 ], [ -0.0683126226067543, 0.05408501252532005, 0.009873728267848492 ] ], [ [ -0.023901041597127914, 0.16978727281093597, -0.13701239228248596 ], [ -0.15532830357551575, 0.19634653627872467, -0.06749705225229263 ], [ 0.06686746329069138, -0.04560288041830063, 0.1947353482246399 ] ], [ [ 0.09764733165502548, 0.11716950684785843, -0.05606110766530037 ], [ 0.0921536460518837, -0.17302048206329346, -0.05728591978549957 ], [ -0.039875198155641556, -0.15957432985305786, 0.1714814454317093 ] ], [ [ -0.09091954678297043, 0.1893412321805954, 0.2223561853170395 ], [ -0.09084659069776535, -0.090516097843647, 0.25710973143577576 ], [ 0.015550807118415833, -0.0536077618598938, -0.07631698250770569 ] ] ] }, { "bias": -0.0007078819326125085, "weights": [ [ [ 0.10348378121852875, 0.1658703088760376, 0.21937958896160126 ], [ -0.1605636179447174, -0.10375072062015533, 0.0317123606801033 ], [ -0.08316430449485779, 0.10123857110738754, -0.13323435187339783 ] ], [ [ 0.046227842569351196, 0.25546056032180786, 0.2341330200433731 ], [ 0.24970883131027222, 0.026491131633520126, 0.20424413681030273 ], [ 0.05459621176123619, 0.022550098598003387, 0.017616553232073784 ] ], [ [ 0.12702660262584686, 0.05750229209661484, 0.12593555450439453 ], [ 0.09516311436891556, -0.021120281890034676, 0.17927201092243195 ], [ -0.15829746425151825, 0.08277112990617752, -0.02489512600004673 ] ], [ [ -0.09650641679763794, -0.16835880279541016, 0.06058763340115547 ], [ -0.17324136197566986, 0.05352270230650902, 0.13822206854820251 ], [ -0.05745401233434677, 0.1021539568901062, 0.04325346648693085 ] ], [ [ 0.08210805058479309, 0.17616593837738037, 0.15967094898223877 ], [ 0.3011374771595001, 0.20852616429328918, 0.1891082376241684 ], [ 0.07962772250175476, -0.10633142292499542, -0.01192254014313221 ] ], [ [ -0.07665342092514038, -0.0029840569477528334, -0.10289561003446579 ], [ -0.11194054782390594, -0.2018415778875351, -0.03850949928164482 ], [ -0.10226105898618698, 0.027894427999854088, -0.33567458391189575 ] ], [ [ -0.05404815077781677, -0.1340097039937973, -0.04483448714017868 ], [ 0.10605424642562866, 0.033482909202575684, -0.03591015562415123 ], [ 0.050270434468984604, -0.08913973718881607, 0.12602296471595764 ] ], [ [ 0.05558621138334274, -0.015462568961083889, -0.011693321168422699 ], [ -0.030964285135269165, -0.12944014370441437, 0.009920598939061165 ], [ 0.09361882507801056, -0.06596191972494125, 0.02901480719447136 ] ], [ [ -0.16552084684371948, 0.07774512469768524, -0.1292998492717743 ], [ -0.17971312999725342, -0.1001170203089714, 0.022619865834712982 ], [ 0.01972196064889431, 0.11501184105873108, 0.12538138031959534 ] ], [ [ 0.019463486969470978, -0.0983787402510643, 0.13546645641326904 ], [ -0.16044984757900238, 0.12485826760530472, 0.05440671741962433 ], [ -0.1767689734697342, 0.08979399502277374, -0.13887923955917358 ] ] ] }, { "bias": 0.08010556548833847, "weights": [ [ [ 0.1133585199713707, -0.00997573509812355, 0.14980067312717438 ], [ 0.2416609227657318, 0.2957213819026947, 0.31225094199180603 ], [ 0.15922629833221436, 0.30081695318222046, 0.2270483672618866 ] ], [ [ -0.1329600065946579, -0.07163365930318832, 0.0458776131272316 ], [ -0.09909605979919434, -0.06709331274032593, 0.037862274795770645 ], [ -0.18296410143375397, -0.1338837444782257, -0.26663830876350403 ] ], [ [ 0.06541728228330612, -0.13355036079883575, -0.020394109189510345 ], [ -0.11164810508489609, 0.1490645706653595, 0.017893286421895027 ], [ -0.09118642657995224, -0.1515958458185196, -0.03566630929708481 ] ], [ [ 0.07353078573942184, -0.16447250545024872, 0.17671015858650208 ], [ -0.04292967915534973, -0.05507134273648262, 0.14150682091712952 ], [ 0.10491141676902771, 0.011298518627882004, 0.10192622989416122 ] ], [ [ 0.18014416098594666, -0.07646407932043076, -0.05285857990384102 ], [ -0.1046762764453888, 0.01189336646348238, 0.033325064927339554 ], [ -0.18640351295471191, -0.10287334024906158, 0.1082197055220604 ] ], [ [ 0.19771923124790192, 0.169350266456604, 0.14940451085567474 ], [ 0.18463142216205597, 0.11183974891901016, -0.0458606481552124 ], [ -0.04137343913316727, 0.24504254758358002, 0.2552352547645569 ] ], [ [ 0.03883717954158783, 0.018807582557201385, -0.10296282172203064 ], [ 0.07945653796195984, 0.04317868500947952, -0.1169438287615776 ], [ 0.07721126079559326, 0.05919022485613823, 0.05295775085687637 ] ], [ [ 0.07933920621871948, -0.11539453268051147, -0.14242030680179596 ], [ -0.09015672653913498, 0.0723484605550766, 0.11390630900859833 ], [ -0.20067369937896729, 0.12236489355564117, -0.00815193448215723 ] ], [ [ 0.1511957347393036, 0.04844445362687111, 0.016093147918581963 ], [ -0.02125474624335766, 0.016247756779193878, -0.1110120490193367 ], [ 0.12395704537630081, 0.09948017448186874, -0.04832472652196884 ] ], [ [ 0.20658090710639954, 0.22431017458438873, 0.1386585384607315 ], [ 0.022130215540528297, 0.2725863456726074, 0.01770108751952648 ], [ -0.17148301005363464, 0.12963199615478516, 0.02227788418531418 ] ] ] }, { "bias": -0.00032840375206433237, "weights": [ [ [ 0.14926579594612122, -0.1658632755279541, -0.006891955621540546 ], [ 0.018407020717859268, 0.11032596230506897, -0.09037923067808151 ], [ 0.054062776267528534, -0.024696068838238716, -0.13622203469276428 ] ], [ [ 0.04308362305164337, 0.011902612634003162, -0.14171727001667023 ], [ -0.029846854507923126, -0.10324931889772415, -0.060329001396894455 ], [ 0.18067266047000885, -0.17709118127822876, -0.017955072224140167 ] ], [ [ -0.07637296617031097, -0.04532130807638168, 0.09160733222961426 ], [ -0.07682713866233826, 0.11216340959072113, -0.08164174854755402 ], [ -0.15361127257347107, -0.12153203040361404, 0.07293781638145447 ] ], [ [ -0.06226680055260658, -0.020958133041858673, -0.17354628443717957 ], [ 0.005389615427702665, -0.021011322736740112, -0.12607698142528534 ], [ -0.02782212570309639, -0.10399992018938065, -0.14286470413208008 ] ], [ [ -0.0819178894162178, 0.05096708983182907, 0.0012606605887413025 ], [ -0.05451017990708351, 0.1276327669620514, -0.15054619312286377 ], [ -0.12790453433990479, 0.04455510899424553, -0.11942426860332489 ] ], [ [ 0.07089143991470337, -0.027805596590042114, -0.0731336772441864 ], [ -0.15991254150867462, 0.0080441078171134, -0.01506744883954525 ], [ 0.09929340332746506, 0.031093627214431763, 0.07412277162075043 ] ], [ [ 0.03911280632019043, 0.004630458075553179, -0.15607154369354248 ], [ -0.03609822317957878, -0.0874827653169632, 0.04198885336518288 ], [ 0.003667711280286312, 0.12561820447444916, 0.05015202611684799 ] ], [ [ -0.17954029142856598, -0.17719751596450806, -0.12609604001045227 ], [ 0.06174042820930481, -0.12848852574825287, -0.03465423732995987 ], [ -0.009152312763035297, 0.06498356908559799, 0.08579559624195099 ] ], [ [ 0.15141189098358154, 0.043854519724845886, 0.0866377055644989 ], [ -0.02429567277431488, 0.02457781881093979, 0.10732681304216385 ], [ -0.17606911063194275, 0.18131668865680695, -0.11436362564563751 ] ], [ [ -0.06993728131055832, -0.07894193381071091, -0.03506617620587349 ], [ -0.04411335662007332, -0.10679899901151657, 0.03913060203194618 ], [ 0.15648718178272247, -0.09655686467885971, 0.11565577238798141 ] ] ] }, { "bias": 0.10102199018001556, "weights": [ [ [ -0.18843390047550201, -0.17197029292583466, 0.09857185930013657 ], [ -0.06710131466388702, 0.042346883565187454, 0.12859860062599182 ], [ -0.08121371269226074, -0.03670134395360947, 0.0045297881588339806 ] ], [ [ -0.14061777293682098, -0.1551135927438736, 0.12329231202602386 ], [ 0.12051475793123245, 0.20856688916683197, 0.11334879696369171 ], [ -0.12097702920436859, 0.13367164134979248, -0.16495303809642792 ] ], [ [ 0.12965843081474304, 0.18333521485328674, 0.08880039304494858 ], [ 0.12541590631008148, 0.06118476018309593, -0.06615698337554932 ], [ 0.044610921293497086, -0.002378442557528615, -0.1364840716123581 ] ], [ [ 0.07500725239515305, 0.15382656455039978, -0.18037930130958557 ], [ -0.09707397222518921, 0.025962140411138535, 0.17580506205558777 ], [ -0.05261111259460449, 0.04544889181852341, 0.1407015472650528 ] ], [ [ -0.00922468863427639, 0.15449784696102142, 0.12286806851625443 ], [ 0.26022836565971375, 0.29797008633613586, 0.12426729500293732 ], [ 0.028880679979920387, -0.10370765626430511, -0.1462513953447342 ] ], [ [ -0.11443484574556351, 0.09595445543527603, -0.0535745806992054 ], [ 0.03697243332862854, -0.13069505989551544, -0.2539665400981903 ], [ -0.2158351093530655, -0.3199607729911804, -0.16416803002357483 ] ], [ [ -0.11691446602344513, 0.030756477266550064, -0.09193025529384613 ], [ -0.07388360798358917, 0.046113599091768265, -0.17120300233364105 ], [ 0.1160215511918068, 0.034703608602285385, 0.18252792954444885 ] ], [ [ -0.07212908565998077, 0.09425552934408188, -0.020474182441830635 ], [ -0.14558999240398407, -0.2292071431875229, -0.09061917662620544 ], [ -0.13141657412052155, -0.06997330486774445, -0.12621599435806274 ] ], [ [ -0.16734322905540466, -0.1640625, 0.17294487357139587 ], [ -0.03534962981939316, -0.03587022051215172, -0.1575644314289093 ], [ 0.003622462972998619, -0.17602811753749847, -0.06479363888502121 ] ], [ [ 0.10049576312303543, -0.13191525638103485, 0.17221088707447052 ], [ 0.08992503583431244, 0.09434428811073303, 0.01953481324017048 ], [ 0.027625583112239838, -0.13099995255470276, 0.22338591516017914 ] ] ] }, { "bias": 0.1839410364627838, "weights": [ [ [ 0.33975979685783386, -0.0007806850480847061, 0.008579856716096401 ], [ 0.3656090497970581, 0.2688677906990051, 0.016921566799283028 ], [ 0.35365620255470276, 0.12918882071971893, -0.05961879342794418 ] ], [ [ -0.14423567056655884, -0.05012332648038864, 0.07619207352399826 ], [ -0.09011676162481308, -0.04360014945268631, -0.07565093785524368 ], [ 0.019539909437298775, 0.13974043726921082, -0.14280207455158234 ] ], [ [ 0.03198204189538956, 0.06887462735176086, -0.1125723198056221 ], [ 0.14379338920116425, -0.07409057766199112, 0.14650262892246246 ], [ -0.17403532564640045, 0.07053810358047485, 0.03149620443582535 ] ], [ [ -0.02583300694823265, -0.16200023889541626, -0.0300496444106102 ], [ -0.1711486428976059, -0.07888854295015335, 0.07103282958269119 ], [ 0.031202886253595352, 0.008295580744743347, -0.07996819913387299 ] ], [ [ 0.1111307144165039, 0.01955331861972809, 0.2893529236316681 ], [ -0.15778030455112457, 0.1852802336215973, 0.24673998355865479 ], [ -0.13873955607414246, -0.1221257820725441, 0.020849088206887245 ] ], [ [ 0.09436196833848953, -0.11874820291996002, 0.13310982286930084 ], [ -0.18267351388931274, -0.16806723177433014, -0.10300038754940033 ], [ 0.03245091810822487, 0.02065262384712696, -0.02199680171906948 ] ], [ [ -0.14088568091392517, -0.07974132150411606, 0.07132813334465027 ], [ 0.10476739704608917, -0.1766519844532013, 0.054840538650751114 ], [ -0.16788484156131744, 0.03572266548871994, -0.09109844267368317 ] ], [ [ -0.08281754702329636, 0.0322098508477211, 0.17384164035320282 ], [ -0.07534745335578918, 0.02286914363503456, -0.07090599089860916 ], [ -0.06827270239591599, -0.008776373229920864, 0.06894057989120483 ] ], [ [ -0.16268222033977509, -0.08499103039503098, 0.15647783875465393 ], [ -0.15771383047103882, 0.15894384682178497, 0.15144884586334229 ], [ 0.07181107997894287, -0.06984170526266098, -0.07709267735481262 ] ], [ [ 0.006052928511053324, -0.146978959441185, -0.041797246783971786 ], [ -0.14079707860946655, -0.22883114218711853, -0.11258888989686966 ], [ -0.16370557248592377, -0.10180635750293732, -0.016936233267188072 ] ] ] } ] }, { "name": "relu_2_2", "input_shape": [ 26, 26, 10 ], "output_shape": [ 26, 26, 10 ], "num_neurons": 10 }, { "name": "max_pool_2", "input_shape": [ 26, 26, 10 ], "output_shape": [ 13, 13, 10 ], "num_neurons": 10 }, { "name": "conv_3_1", "input_shape": [ 13, 13, 10 ], "output_shape": [ 11, 11, 10 ], "num_neurons": 10, "weights": [ { "bias": 0.0007923436933197081, "weights": [ [ [ 0.014084209688007832, 0.12834112346172333, 0.13770976662635803 ], [ -0.15767118334770203, -0.14443932473659515, 0.020143991336226463 ], [ -0.137760728597641, 0.09652291983366013, -0.05444719269871712 ] ], [ [ -0.09999548643827438, 0.11078786849975586, -0.022936243563890457 ], [ -0.1495421975851059, -0.18033678829669952, 0.0147013608366251 ], [ 0.11255548149347305, -0.08196849375963211, -0.020989680662751198 ] ], [ [ -0.020475145429372787, 0.10974124073982239, 0.13217176496982574 ], [ 0.1812802404165268, 0.1309806853532791, 0.165183424949646 ], [ 0.032678816467523575, 0.11972841620445251, 0.11467933654785156 ] ], [ [ -0.01285762619227171, -0.04902748763561249, -0.17654164135456085 ], [ -0.03163326531648636, 0.08867398649454117, -0.14534570276737213 ], [ 0.05154038593173027, -0.11657627671957016, -0.1699831187725067 ] ], [ [ -0.05665671080350876, -0.04876944422721863, 0.04310169443488121 ], [ -0.07165641337633133, 0.02466914989054203, -0.1543319970369339 ], [ -0.07406008988618851, 0.09902089834213257, -0.0853695422410965 ] ], [ [ -0.09680116176605225, 0.0020944171119481325, 0.06861642003059387 ], [ 0.06426360458135605, 0.0014486954314634204, -0.06736121326684952 ], [ -0.02081463299691677, 0.010254998691380024, -0.1277466118335724 ] ], [ [ 0.13085784018039703, 0.01865667849779129, 0.11534454673528671 ], [ -0.16589294373989105, 0.15188102424144745, -0.02093818411231041 ], [ -0.1733652651309967, -0.005625993013381958, -0.07835599780082703 ] ], [ [ -0.0005362312076613307, -0.14726924896240234, 0.11500425636768341 ], [ 0.04323713108897209, -0.02632644958794117, -0.12503141164779663 ], [ -0.08531049638986588, 0.03978920727968216, 0.026031920686364174 ] ], [ [ -0.09447162598371506, 0.08095337450504303, -0.0033416757360100746 ], [ 0.07817487418651581, -0.08524847030639648, 0.15410764515399933 ], [ 0.07341686636209488, -0.08353333920240402, 0.09512470662593842 ] ], [ [ -0.1630215346813202, -0.048441022634506226, -0.13386407494544983 ], [ 0.03796912357211113, -0.12130269408226013, 0.10480397939682007 ], [ -0.09396984428167343, 0.14544925093650818, 0.13277694582939148 ] ] ] }, { "bias": -0.03995319455862045, "weights": [ [ [ 0.10615699738264084, 0.1713346689939499, -0.20744158327579498 ], [ 0.12267278879880905, 0.11937644332647324, 0.00038562045665457845 ], [ 0.03341822326183319, 0.11432299017906189, -0.05625610798597336 ] ], [ [ 0.09210916608572006, -0.040792591869831085, -0.02658102475106716 ], [ 0.08646120131015778, 0.12138230353593826, -0.1291978806257248 ], [ 0.02896762639284134, -0.13562193512916565, -0.21209317445755005 ] ], [ [ 0.11792007833719254, -0.08320023119449615, -0.04069831594824791 ], [ 0.15743722021579742, -0.1775541752576828, -0.10747846961021423 ], [ 0.15329301357269287, 0.1677314192056656, 0.12893836200237274 ] ], [ [ 0.12554802000522614, -0.17939364910125732, 0.11124496906995773 ], [ 0.09351976215839386, -0.1847659945487976, -0.10156165063381195 ], [ 0.05262088030576706, -0.03287182003259659, -0.1808682680130005 ] ], [ [ 0.024454670026898384, -0.012735440395772457, -0.12574359774589539 ], [ 0.07756925374269485, -0.010186216793954372, 0.010992603376507759 ], [ -0.0616394467651844, 0.13827480375766754, -0.001830886583775282 ] ], [ [ -0.18649210035800934, -0.06740047037601471, -0.15099391341209412 ], [ -0.07808662205934525, 0.14133435487747192, 0.17570528388023376 ], [ -0.1506563127040863, 0.1270168423652649, 0.0695977583527565 ] ], [ [ -0.07790328562259674, -0.11900027841329575, -0.14872725307941437 ], [ 0.010531118139624596, -0.06965688616037369, -0.1263745129108429 ], [ -0.09708639979362488, -0.09562280029058456, -0.049736279994249344 ] ], [ [ 0.1256728172302246, -0.11736968904733658, 0.12513695657253265 ], [ -0.023180535063147545, 0.1055678203701973, 0.0043745096772909164 ], [ -0.05807867273688316, 0.10425814986228943, -0.020190350711345673 ] ], [ [ -0.06526564061641693, -0.07953085005283356, 0.06664080917835236 ], [ -0.14328138530254364, 0.0676945149898529, -0.16580939292907715 ], [ -0.094744972884655, -0.027153242379426956, -0.0967867374420166 ] ], [ [ 0.03862609341740608, -0.10124414414167404, 0.14332693815231323 ], [ 0.16817833483219147, 0.024794770404696465, 0.06339366734027863 ], [ -0.0812276303768158, 0.19088798761367798, -0.07698020339012146 ] ] ] }, { "bias": -0.1186998263001442, "weights": [ [ [ -0.1698475033044815, -0.052456241101026535, 0.0413820780813694 ], [ -0.06934557110071182, -0.10775590687990189, 0.09477849304676056 ], [ -0.09385121613740921, -0.0356290377676487, 0.04569070041179657 ] ], [ [ 0.22217205166816711, 0.07174063473939896, -0.11545348912477493 ], [ 0.21276743710041046, 0.1291293352842331, 0.01896503008902073 ], [ 0.14763914048671722, 0.1861116588115692, 0.08590038865804672 ] ], [ [ 0.026769159361720085, -0.045213937759399414, 0.075718954205513 ], [ -0.14487768709659576, 0.0035822070203721523, 0.14938460290431976 ], [ -0.06984766572713852, -0.07488036900758743, 0.10114625096321106 ] ], [ [ 0.23332081735134125, 0.09064438939094543, -0.036316052079200745 ], [ -0.16097210347652435, 0.005021014250814915, 0.06244467943906784 ], [ -0.03754465654492378, 0.045506302267313004, -0.042660053819417953 ] ], [ [ -0.21556663513183594, -0.026264134794473648, 0.0604860857129097 ], [ 0.2277263104915619, 0.19397714734077454, 0.0751928836107254 ], [ 0.03162621706724167, -0.10699104517698288, -0.004026363138109446 ] ], [ [ 0.08232510834932327, -0.13078849017620087, 0.03585424646735191 ], [ 0.21168948709964752, 0.07903169840574265, 0.05411357805132866 ], [ 0.16992908716201782, 0.0022068489342927933, 0.10664689540863037 ] ], [ [ 0.08053668588399887, -0.030569858849048615, 0.051013581454753876 ], [ -0.1376090943813324, -0.1279177963733673, 0.08727388828992844 ], [ -0.17089048027992249, 0.055040109902620316, 0.09938357025384903 ] ], [ [ 0.049769479781389236, -0.10225320607423782, -0.06922544538974762 ], [ 0.010851606726646423, 0.0875324010848999, -0.11066267639398575 ], [ 0.1668866127729416, 0.014141902327537537, 0.038830194622278214 ] ], [ [ 0.06571630388498306, 0.15494510531425476, -0.06427671760320663 ], [ 0.11934246122837067, 0.10940946638584137, 0.2175624519586563 ], [ 0.33665385842323303, 0.08218508213758469, 0.2736110985279083 ] ], [ [ -0.0974932312965393, -0.19408659636974335, -0.12665219604969025 ], [ -0.11315779387950897, -0.21975912153720856, -0.1136365607380867 ], [ -0.05188465490937233, 0.02909829467535019, -0.08100558072328568 ] ] ] }, { "bias": -0.09210537374019623, "weights": [ [ [ -0.03278696909546852, -0.017309516668319702, 0.08832841366529465 ], [ 0.10297974944114685, 0.13024617731571198, -0.020145995542407036 ], [ 0.19648101925849915, 0.05248822271823883, -0.07985596358776093 ] ], [ [ -0.054609447717666626, 0.2046888768672943, 0.06291133165359497 ], [ -0.16105268895626068, 0.2436676025390625, 0.08801200240850449 ], [ 0.11915095895528793, 0.20166395604610443, 0.037480369210243225 ] ], [ [ 0.1711929589509964, -0.16010943055152893, -0.12285618484020233 ], [ -0.17299924790859222, -0.07008962333202362, 0.14374801516532898 ], [ 0.09877115488052368, -0.1352241039276123, 0.04064025729894638 ] ], [ [ 0.08953512459993362, 0.16134575009346008, 0.19984757900238037 ], [ -0.004337814636528492, -0.11732278764247894, 0.0945245549082756 ], [ 0.20016124844551086, 0.20509259402751923, -0.009458988904953003 ] ], [ [ 0.011065728031098843, 0.11924783140420914, 0.2751632630825043 ], [ -0.1809077262878418, 0.04392337054014206, -0.10610227286815643 ], [ 0.11697299778461456, -0.10463783890008926, 0.10434195399284363 ] ], [ [ 0.13397638499736786, 0.23100727796554565, 0.2506504952907562 ], [ 0.21284300088882446, -0.09303019195795059, 0.04000652953982353 ], [ 0.13550232350826263, -0.05390007048845291, 0.15633025765419006 ] ], [ [ -0.035088714212179184, 0.011019790545105934, -0.15191912651062012 ], [ -0.0970035269856453, -0.20510318875312805, -0.2798207104206085 ], [ 0.22112753987312317, 0.173477441072464, -0.07703046500682831 ] ], [ [ -0.1316671073436737, 0.11739280819892883, -0.12437774240970612 ], [ 0.14049214124679565, 0.018660126253962517, -0.008317839354276657 ], [ 0.10293585807085037, 0.12097896635532379, -0.18470950424671173 ] ], [ [ 0.07217487692832947, 0.17033891379833221, -0.05641436204314232 ], [ 0.05039115250110626, -0.09965449571609497, -0.0016430896939709783 ], [ 0.12333089858293533, 0.07186176627874374, -0.021624935790896416 ] ], [ [ 0.05988701060414314, -0.09962324053049088, -0.047765620052814484 ], [ 0.004566616844385862, 0.14760108292102814, -0.049695611000061035 ], [ 0.08817210793495178, 0.09744234383106232, -0.1272374391555786 ] ] ] }, { "bias": 0.08263328671455383, "weights": [ [ [ 0.05567118898034096, 0.06958293914794922, 0.29363101720809937 ], [ -0.15386274456977844, -0.1892932951450348, 0.09584794193506241 ], [ -0.16181093454360962, -0.26432499289512634, 0.09839805960655212 ] ], [ [ -0.1616799533367157, -0.011068828403949738, 0.09977106750011444 ], [ 0.0686490461230278, -0.03881068900227547, -0.05779111757874489 ], [ 0.12616251409053802, -0.10615657269954681, 0.007902081124484539 ] ], [ [ -0.051640547811985016, -0.06663563847541809, 0.05050617456436157 ], [ -0.17704543471336365, -0.07125069200992584, 0.07076115161180496 ], [ -0.08993498235940933, 0.12576580047607422, 0.15912599861621857 ] ], [ [ -0.16754917800426483, 0.07261624932289124, 0.04180620238184929 ], [ 0.0661192536354065, -0.04214955493807793, 0.034119412302970886 ], [ -0.08485493063926697, -0.17403626441955566, -0.1516757309436798 ] ], [ [ 0.15753023326396942, 0.14854496717453003, 0.1365014761686325 ], [ 0.07514838874340057, -0.12848225235939026, 0.020275237038731575 ], [ -0.07221581786870956, 0.0022716999519616365, -0.19164001941680908 ] ], [ [ 0.08286779373884201, -0.013372530229389668, -0.021720657125115395 ], [ -0.21152161061763763, 0.026690226048231125, 0.06342209130525589 ], [ 0.21802379190921783, 0.08808977156877518, 0.0848800539970398 ] ], [ [ 0.17877766489982605, 0.02883482351899147, 0.16922372579574585 ], [ -0.09378396719694138, -0.006063633132725954, -0.09353656321763992 ], [ -0.04188361391425133, 0.1893092840909958, 0.13593453168869019 ] ], [ [ 0.019957734271883965, 0.17231403291225433, 0.11049658805131912 ], [ -0.07806538790464401, 0.0027042622677981853, 0.11312343180179596 ], [ -0.1269344538450241, -0.0074524227529764175, -0.025511115789413452 ] ], [ [ 0.07176779955625534, 0.044990986585617065, 0.1393134891986847 ], [ 0.14584551751613617, -0.11557573080062866, -0.05252663046121597 ], [ 0.33578360080718994, 0.16844221949577332, 0.19836050271987915 ] ], [ [ 0.20327317714691162, 0.08193209767341614, 0.09176541119813919 ], [ 0.022141672670841217, 0.2393805980682373, 0.24619236588478088 ], [ 0.2033611685037613, 0.2690250277519226, 0.07805047184228897 ] ] ] }, { "bias": 0.0020236384589225054, "weights": [ [ [ 0.10053439438343048, -0.11898089200258255, -0.02185264602303505 ], [ 0.07545827329158783, -0.004810892511159182, 0.06351689249277115 ], [ 0.10038071125745773, 0.256062388420105, 0.14542889595031738 ] ], [ [ 0.11477155238389969, 0.08614219725131989, -0.10725229978561401 ], [ -0.2381533682346344, -0.18247275054454803, 0.08615956455469131 ], [ 0.11388798803091049, -0.22101150453090668, -0.06669963151216507 ] ], [ [ -0.04353608563542366, 0.1748303472995758, 0.00718834949657321 ], [ -0.15852247178554535, 0.025131087750196457, 0.07791343331336975 ], [ 0.047718364745378494, 0.03452707454562187, -0.13101045787334442 ] ], [ [ -0.13334953784942627, -0.04534310847520828, 0.19090056419372559 ], [ -0.04243757203221321, 0.13102532923221588, -0.000876651203725487 ], [ -0.11044615507125854, -0.010847765021026134, -0.28389787673950195 ] ], [ [ 0.09492381662130356, 0.09689532965421677, 0.07702462375164032 ], [ 0.09333714842796326, 0.09936945885419846, 0.15126819908618927 ], [ 0.11038269847631454, 0.03863901272416115, 0.17338909208774567 ] ], [ [ -0.2164311558008194, 0.12255308777093887, -0.22648969292640686 ], [ -0.1568596065044403, -0.14963537454605103, -0.01591780036687851 ], [ -0.07670484483242035, -0.05227745696902275, -0.10527215898036957 ] ], [ [ 0.25787675380706787, 0.2774695158004761, 0.2681882381439209 ], [ -0.21463291347026825, -0.17441409826278687, 0.17279638350009918 ], [ 0.07751452922821045, 0.1427261233329773, 0.0029161006677895784 ] ], [ [ -0.023808851838111877, -0.1338464319705963, -0.16531069576740265 ], [ 0.09184430539608002, -0.10058215260505676, 0.10079031437635422 ], [ -0.09520717710256577, -0.12539193034172058, -0.0013041305355727673 ] ], [ [ -0.202018141746521, -0.13908347487449646, 0.09860333055257797 ], [ -0.012914903461933136, 0.030028915032744408, -0.05219478905200958 ], [ -0.06688292324542999, -0.1209234818816185, -0.02122826874256134 ] ], [ [ -0.03583580628037453, 0.019045300781726837, -0.09886214882135391 ], [ 0.15042924880981445, 0.16495978832244873, -0.050434794276952744 ], [ 0.056220296770334244, -0.06817059218883514, -0.040640488266944885 ] ] ] }, { "bias": 0.017033971846103668, "weights": [ [ [ -0.20809075236320496, -0.13181592524051666, -0.12393893301486969 ], [ -0.06774646043777466, 0.03121250309050083, -0.07943106442689896 ], [ -0.10797509551048279, 0.0999375507235527, 0.1254819631576538 ] ], [ [ 0.10122480243444443, 0.09813398867845535, 0.052496425807476044 ], [ 0.16393233835697174, -0.09034610539674759, 0.05845034122467041 ], [ -0.02693263627588749, -0.2084280252456665, 0.006234470289200544 ] ], [ [ 0.18094633519649506, -0.05955606698989868, 0.11186791211366653 ], [ 0.08737774938344955, 0.17498096823692322, 0.0400092788040638 ], [ -0.17215411365032196, -0.1591709554195404, 0.17329218983650208 ] ], [ [ 0.14523540437221527, 0.21107657253742218, 0.27582937479019165 ], [ -0.035530272871255875, 0.11021815985441208, -0.11280883103609085 ], [ 0.12257345765829086, -0.2404092252254486, -0.09588366001844406 ] ], [ [ -0.17064805328845978, -0.21269570291042328, -0.09245115518569946 ], [ -0.002536489861086011, -0.04389023408293724, 0.11974436789751053 ], [ 0.01879681646823883, -0.05463852360844612, 0.0689098984003067 ] ], [ [ -0.09446363896131516, -0.2626676857471466, 0.037944354116916656 ], [ -0.028313560411334038, 0.08721524477005005, 0.12055739760398865 ], [ 0.07355979830026627, 0.18201249837875366, 0.10116560012102127 ] ], [ [ 0.3293829560279846, 0.32655951380729675, 0.24378590285778046 ], [ -0.07051360607147217, -0.10129305720329285, -0.0892932265996933 ], [ -0.23557034134864807, -0.31763985753059387, -0.3160083293914795 ] ], [ [ -0.03236345574259758, -0.10293465107679367, -0.1326933354139328 ], [ -0.03448130562901497, -0.035801827907562256, 0.14836330711841583 ], [ -0.06528409570455551, 0.012504453770816326, 0.009019790217280388 ] ], [ [ 0.07146022468805313, 0.17739997804164886, -0.14866097271442413 ], [ -0.01479280460625887, 0.001622064271941781, 0.1060650423169136 ], [ -0.0778941810131073, -0.12459131330251694, 0.1424666792154312 ] ], [ [ -0.15287674963474274, -0.057680219411849976, -0.11161118000745773 ], [ 0.020092297345399857, 0.056244343519210815, 0.0034972724970430136 ], [ -0.01507959607988596, -0.10725513100624084, -0.005411537829786539 ] ] ] }, { "bias": -0.0487380176782608, "weights": [ [ [ 0.013004053384065628, 0.1438901126384735, 0.2067205160856247 ], [ 0.2778065502643585, 0.035558704286813736, 0.15905006229877472 ], [ 0.1556456834077835, -0.055424440652132034, -0.196225106716156 ] ], [ [ -0.06706154346466064, -0.06031229719519615, 0.1522347778081894 ], [ -0.1272343546152115, -0.024298088625073433, -0.17963898181915283 ], [ -0.296866238117218, 0.05329625681042671, -0.08038515597581863 ] ], [ [ 0.08246057480573654, 0.11964347958564758, -0.15256546437740326 ], [ 0.04718828201293945, -0.036873333156108856, 0.11580844223499298 ], [ -0.02689993940293789, 0.14799445867538452, 0.0425935722887516 ] ], [ [ -0.2321205884218216, 0.07651403546333313, 0.06711816787719727 ], [ -0.24553300440311432, -0.19699813425540924, -0.10530014336109161 ], [ -0.3381549119949341, -0.23991091549396515, 0.08259432017803192 ] ], [ [ 0.1878867745399475, 0.0027300918009132147, 0.13993220031261444 ], [ -0.006729230750352144, 0.04196777939796448, 0.12541988492012024 ], [ -0.18064618110656738, -0.02939745783805847, -0.12110479176044464 ] ], [ [ -0.02033473551273346, 0.1171392872929573, 0.1683845967054367 ], [ 0.04123875871300697, 0.12270105630159378, 0.15763184428215027 ], [ 0.010796451941132545, 0.1692696511745453, -0.012512930668890476 ] ], [ [ 0.01538323238492012, 0.14960414171218872, 0.2853028476238251 ], [ -0.029109112918376923, -0.11643902212381363, -0.055375710129737854 ], [ -0.004981658887118101, -0.09975367784500122, 0.01719915121793747 ] ], [ [ -0.08597826957702637, -0.1395689994096756, -0.0871158018708229 ], [ 0.07820675522089005, 0.06476256996393204, 0.08447449654340744 ], [ 0.04280698299407959, 0.13454754650592804, -0.14602670073509216 ] ], [ [ -0.2120448648929596, -0.10349173098802567, -0.01867772452533245 ], [ -0.13450013101100922, 0.03246336057782173, 0.008800385519862175 ], [ -0.06651842594146729, 0.0027631367556750774, 0.1084757074713707 ] ], [ [ 0.024234263226389885, 0.035659898072481155, 0.07447188347578049 ], [ 0.14663217961788177, 0.12224237620830536, 0.11024618148803711 ], [ -0.017633860930800438, 0.11866504698991776, 0.031190453097224236 ] ] ] }, { "bias": 0.06205393746495247, "weights": [ [ [ 0.10403458029031754, -0.04516236484050751, 0.007944069802761078 ], [ 0.027522817254066467, -0.06080198660492897, 0.17122933268547058 ], [ 0.031839076429605484, -0.13542407751083374, 0.147073432803154 ] ], [ [ -0.07265862077474594, 0.07584702968597412, 0.0826362743973732 ], [ -0.12504354119300842, 0.10610494017601013, -0.05252160131931305 ], [ -0.1802300661802292, 0.13104572892189026, 0.1587398797273636 ] ], [ [ -0.01467291358858347, -0.1736292839050293, -0.06778385490179062 ], [ -0.08540039509534836, 0.1718093454837799, 0.037517379969358444 ], [ -0.11343583464622498, -0.09185007214546204, 0.04145399108529091 ] ], [ [ -0.0252825990319252, 0.13854925334453583, 0.1342930644750595 ], [ -0.18445101380348206, -0.06430703401565552, -0.1338326334953308 ], [ 0.044733159244060516, 0.1775427758693695, 0.18956460058689117 ] ], [ [ 0.05329892411828041, 0.14391566812992096, 0.17773744463920593 ], [ -0.13861612975597382, 0.05440625175833702, 0.18248164653778076 ], [ 0.054199911653995514, 0.10067229717969894, 0.03818938881158829 ] ], [ [ 0.15098167955875397, -0.10688918083906174, 0.10139582306146622 ], [ -0.030306799337267876, -0.08467961102724075, -0.024463560432195663 ], [ -0.004675365053117275, 0.06831468641757965, 0.10519678145647049 ] ], [ [ 0.005631213542073965, 0.024092094972729683, -0.2629964053630829 ], [ 0.02316136285662651, -0.1372058093547821, -0.0011137331603094935 ], [ 0.08635002374649048, 0.12336578965187073, -0.20182885229587555 ] ], [ [ -0.12374411523342133, 0.06976250559091568, 0.0033627478405833244 ], [ -0.06103638559579849, -0.01258084923028946, -0.16220147907733917 ], [ 0.036776743829250336, -0.043156035244464874, -0.002188880927860737 ] ], [ [ 0.008888236247003078, -0.20282362401485443, -0.00037311940104700625 ], [ -0.06386574357748032, 0.1379087269306183, -0.1674935668706894 ], [ -0.06518663465976715, 0.11533026397228241, 0.04993900656700134 ] ], [ [ 0.13230127096176147, 0.14546886086463928, 0.025895265862345695 ], [ 0.2527938485145569, 0.23856063187122345, 0.2568085789680481 ], [ -0.045192964375019073, 0.02460247091948986, -0.09280352294445038 ] ] ] }, { "bias": -0.0025097853504121304, "weights": [ [ [ -0.22498294711112976, 0.09176423400640488, -0.2085607796907425 ], [ -0.1700950711965561, -0.1335902363061905, -0.10471974313259125 ], [ -0.01865343004465103, 0.05609619617462158, 0.061930976808071136 ] ], [ [ 0.1914878934621811, 0.1001431941986084, -0.1320115625858307 ], [ -0.03730002045631409, -0.023550791665911674, 0.1155635267496109 ], [ -0.11461998522281647, -0.16998285055160522, -0.0012773670023307204 ] ], [ [ -0.17897191643714905, 0.17310252785682678, -0.08422792702913284 ], [ -0.07960566878318787, 0.0966135635972023, -0.017904765903949738 ], [ 0.09933783113956451, -0.024894703179597855, -0.07360128313302994 ] ], [ [ 0.15306206047534943, 0.038106270134449005, -0.01117273885756731 ], [ -0.04237715154886246, -0.14435125887393951, 0.13693749904632568 ], [ 0.0670156180858612, 0.08106747269630432, 0.17583781480789185 ] ], [ [ -0.0670420229434967, -0.08985667675733566, -0.21924220025539398 ], [ -0.19975325465202332, -0.02605069987475872, -0.15807080268859863 ], [ -0.26512908935546875, -0.11320961266756058, -0.020481813699007034 ] ], [ [ -0.126532182097435, 0.0574677474796772, 0.13956378400325775 ], [ -0.2515821158885956, 0.1006961390376091, -0.014027233235538006 ], [ -0.22177976369857788, 0.057052042335271835, -0.15844808518886566 ] ], [ [ 0.14358428120613098, 0.18655778467655182, -0.06682707369327545 ], [ 0.09867151081562042, 0.20803463459014893, -0.07671579718589783 ], [ 0.26640254259109497, 0.18211060762405396, 0.21034330129623413 ] ], [ [ 0.08930578082799911, 0.13615724444389343, -0.13849537074565887 ], [ -0.1625012904405594, 0.11744222044944763, -0.1308753341436386 ], [ 0.17427915334701538, -0.11424800753593445, -0.0840780958533287 ] ], [ [ -0.103568896651268, -0.03933466225862503, 0.1908055990934372 ], [ -0.040733352303504944, 0.1285048872232437, -0.13059766590595245 ], [ -0.12315960228443146, -0.15618792176246643, 0.14209264516830444 ] ], [ [ 0.05338658019900322, -0.06080073490738869, -0.05633510276675224 ], [ -0.17862318456172943, -0.11959339678287506, 0.04116475209593773 ], [ 0.16246318817138672, -0.030260881409049034, -0.16533328592777252 ] ] ] } ] }, { "name": "relu_3_1", "input_shape": [ 11, 11, 10 ], "output_shape": [ 11, 11, 10 ], "num_neurons": 10 }, { "name": "conv_3_2", "input_shape": [ 11, 11, 10 ], "output_shape": [ 9, 9, 10 ], "num_neurons": 10, "weights": [ { "bias": -0.10277897119522095, "weights": [ [ [ 0.09267803281545639, -0.14078417420387268, -0.07605091482400894 ], [ 0.14614148437976837, 0.08831778913736343, -0.12347656488418579 ], [ -0.13764387369155884, 0.17568658292293549, 0.0465690940618515 ] ], [ [ -0.07284963130950928, 0.10043230652809143, -0.08929219841957092 ], [ 0.14805518090724945, 0.0652356818318367, 0.10943868011236191 ], [ 0.018983790650963783, 0.183913916349411, 0.1163867861032486 ] ], [ [ 0.08620022237300873, 0.13983801007270813, 0.10129179805517197 ], [ 0.0601688027381897, 0.01619243435561657, -0.1599501520395279 ], [ 0.0444798469543457, -0.15351161360740662, -0.0690305083990097 ] ], [ [ -0.03898674622178078, -0.020412325859069824, -0.165174663066864 ], [ 0.2007649689912796, 0.08091966062784195, 0.08275861293077469 ], [ -0.3116636276245117, -0.040311604738235474, -0.25993087887763977 ] ], [ [ 0.24938461184501648, 0.004057968035340309, 0.22963069379329681 ], [ 0.15706834197044373, 0.1954222023487091, -0.07087729126214981 ], [ -0.20342469215393066, 0.03749361261725426, -0.09815824776887894 ] ], [ [ 0.29192686080932617, -0.058532387018203735, 0.03931589797139168 ], [ -0.21860815584659576, 0.15099529922008514, -0.15310262143611908 ], [ -0.1616145670413971, -0.2389601618051529, -0.04281521961092949 ] ], [ [ 0.12723246216773987, 0.1764863282442093, -0.13758128881454468 ], [ 0.07308806478977203, -0.08522924035787582, 0.08005966991186142 ], [ 0.05577504634857178, 0.24023334681987762, 0.21399162709712982 ] ], [ [ 0.2390051931142807, 0.0032715487759560347, 0.030026260763406754 ], [ 0.2931196987628937, 0.23040874302387238, 0.02019393816590309 ], [ 0.26748859882354736, 0.043368130922317505, 0.18693090975284576 ] ], [ [ 0.11633669584989548, -0.060911625623703, -0.1735755205154419 ], [ 0.14914847910404205, 0.04549966752529144, 0.10831913352012634 ], [ -0.16951464116573334, -0.19501762092113495, -0.19189445674419403 ] ], [ [ -0.04856371879577637, -0.13896895945072174, -0.15048551559448242 ], [ 0.04664619266986847, 0.20016220211982727, 0.20906642079353333 ], [ 0.15420468151569366, 0.14150410890579224, 0.1315082162618637 ] ] ] }, { "bias": 0.19701802730560303, "weights": [ [ [ -0.143484964966774, 0.04895861819386482, -0.1301470696926117 ], [ 0.009600421413779259, 0.13975095748901367, 0.03641190379858017 ], [ 0.1698870062828064, 0.15469446778297424, -0.012752104550600052 ] ], [ [ -0.1372455209493637, 0.11356484889984131, 0.14451460540294647 ], [ -0.07270649820566177, 0.16737598180770874, -0.1741221398115158 ], [ 0.0332883819937706, -0.08568745106458664, 0.16088034212589264 ] ], [ [ -0.008876948617398739, -0.0006492537213489413, -0.2033199816942215 ], [ 0.14547814428806305, 0.09163498133420944, 0.005591763183474541 ], [ -0.20547902584075928, -0.15030084550380707, 0.17562271654605865 ] ], [ [ 0.26435554027557373, -0.02550823986530304, -0.0867445170879364 ], [ -0.07698290050029755, 0.19865825772285461, 0.1312965452671051 ], [ -0.1653180867433548, 0.003725706599652767, -0.08473141491413116 ] ], [ [ 0.2102108597755432, 0.16618561744689941, 0.21938714385032654 ], [ -0.07313138246536255, 0.03749938681721687, 0.036075253039598465 ], [ -0.08825419843196869, -0.06802782416343689, 0.27731508016586304 ] ], [ [ 0.07379036396741867, -0.048663150519132614, 0.06394065916538239 ], [ 0.1504632532596588, -0.15331095457077026, 0.039276234805583954 ], [ 0.029437465593218803, -0.1090429499745369, 0.1631459891796112 ] ], [ [ -0.07403174042701721, -0.20716427266597748, -0.2277587205171585 ], [ -0.07353825122117996, 0.10423948615789413, -0.04673963785171509 ], [ -0.12316188961267471, 0.04170292615890503, -0.09625712037086487 ] ], [ [ 0.15695101022720337, 0.03164668008685112, -0.07265935093164444 ], [ 0.038059987127780914, 0.19235441088676453, -0.17337316274642944 ], [ 0.10923129320144653, 0.12453075498342514, 0.10390300303697586 ] ], [ [ 0.24312619864940643, -0.07914382964372635, -0.07341685146093369 ], [ -0.002172311767935753, 0.013023238629102707, -0.01515197940170765 ], [ 0.16983307898044586, 0.15501026809215546, -0.019847286865115166 ] ], [ [ -0.0009645269601605833, -0.13425539433956146, -0.1456792801618576 ], [ -0.043353136628866196, -0.08142224699258804, -0.19224217534065247 ], [ 0.039527688175439835, -0.03303470090031624, -0.25063997507095337 ] ] ] }, { "bias": -0.03129541128873825, "weights": [ [ [ -0.04401397705078125, -0.1026557981967926, -0.15873579680919647 ], [ 0.14723625779151917, -0.02761595882475376, -0.11629337817430496 ], [ -0.10660669207572937, 0.06266634911298752, -0.12126072496175766 ] ], [ [ -0.051345374435186386, -0.12373384833335876, 0.01398497074842453 ], [ 0.13865694403648376, -0.16774369776248932, -0.1940239816904068 ], [ -0.11738843470811844, -0.16516463458538055, -0.032126326113939285 ] ], [ [ 0.19980014860630035, -0.012881172820925713, 0.02587236277759075 ], [ -0.0648706927895546, -0.03064793534576893, 0.17219239473342896 ], [ 0.2922405004501343, 0.11596475541591644, -0.05196569487452507 ] ], [ [ -0.08409712463617325, -0.2780201733112335, -0.1252819001674652 ], [ -0.11839822679758072, 0.07058580219745636, -0.10253388434648514 ], [ 0.0600438117980957, 0.3053727149963379, 0.28478389978408813 ] ], [ [ 0.1946101039648056, 0.03956221789121628, -0.13357213139533997 ], [ -0.17230957746505737, -0.21701259911060333, 0.027316274121403694 ], [ -0.3401566445827484, -0.11001702398061752, -0.15215662121772766 ] ], [ [ 0.16042929887771606, 0.1576676219701767, -0.04330100119113922 ], [ -0.13132819533348083, 0.12381011247634888, 0.19731402397155762 ], [ 0.0245810654014349, 0.09865681827068329, -0.15396825969219208 ] ], [ [ 0.004712874535471201, -0.06420975923538208, 0.007638312876224518 ], [ 0.10823718458414078, 0.044939979910850525, -0.12299510091543198 ], [ -0.07070112228393555, 0.19507981836795807, 0.12318171560764313 ] ], [ [ -0.1818331927061081, 0.15946221351623535, 0.04939134046435356 ], [ -0.13904240727424622, 0.049535155296325684, -0.13799390196800232 ], [ 0.006305249873548746, 0.0588289350271225, 0.12431934475898743 ] ], [ [ -0.07872401922941208, 0.047131527215242386, -0.17851923406124115 ], [ 0.10127964615821838, 0.09443499892950058, 0.007255954202264547 ], [ 0.00549765769392252, 0.1199440211057663, 0.1588352471590042 ] ], [ [ 0.3437877297401428, 0.07055503875017166, 0.010348151437938213 ], [ 0.10796968638896942, -0.13699431717395782, -0.24103495478630066 ], [ -0.12173858284950256, -0.11904724687337875, -0.11231179535388947 ] ] ] }, { "bias": 0.17929309606552124, "weights": [ [ [ 0.0019429587991908193, -0.09248022735118866, 0.007708424236625433 ], [ 0.1692398488521576, 0.16336318850517273, 0.11352494359016418 ], [ -0.040179528295993805, 0.07438023388385773, 0.13233979046344757 ] ], [ [ -0.17890037596225739, -0.08385209739208221, -0.15496759116649628 ], [ 0.10870101302862167, -0.04719959944486618, 0.06616398692131042 ], [ 0.1379544734954834, 0.08445236086845398, 0.14320217072963715 ] ], [ [ 0.03291373327374458, 0.039768971502780914, -0.10941271483898163 ], [ 0.09658505767583847, -0.25947511196136475, 0.1890028566122055 ], [ -0.03597365692257881, 0.02223622426390648, -0.03105202130973339 ] ], [ [ -0.09644536674022675, 0.13919652998447418, 0.18691404163837433 ], [ -0.06257228553295135, -0.12506872415542603, 0.2528545558452606 ], [ -0.273127943277359, -0.21213173866271973, 0.23635731637477875 ] ], [ [ 0.19669513404369354, 0.06016920506954193, 0.21170251071453094 ], [ -0.126479834318161, 0.038581881672143936, -0.02009897492825985 ], [ 0.17705003917217255, 0.2454189658164978, 0.12400642782449722 ] ], [ [ 0.06749136745929718, 0.07600580900907516, 0.22043104469776154 ], [ -0.10949134826660156, -0.04233156889677048, 0.029990488663315773 ], [ -0.3844650685787201, -0.23563167452812195, 0.05225079134106636 ] ], [ [ 0.13738933205604553, 0.04688725993037224, 0.015981517732143402 ], [ -0.06326860189437866, 0.16815994679927826, 0.05785881355404854 ], [ 0.14164455235004425, -0.032385967671871185, 0.06069803610444069 ] ], [ [ -0.0930677205324173, -0.2240021824836731, -0.08354826271533966 ], [ -0.28700146079063416, -0.16311541199684143, -0.08450518548488617 ], [ -0.32040783762931824, 0.06595692038536072, -0.042691562324762344 ] ], [ [ -0.12971054017543793, -0.06245476379990578, 0.27206432819366455 ], [ -0.11758383363485336, -0.16736234724521637, 0.19137832522392273 ], [ 0.0216483436524868, -0.06711260974407196, 0.13713283836841583 ] ], [ [ -0.07027512788772583, 0.08385137468576431, -0.15525342524051666 ], [ -0.018946722149848938, 0.13567037880420685, 0.0418560765683651 ], [ -0.03601307421922684, -0.015821611508727074, -0.08597856014966965 ] ] ] }, { "bias": -0.03919683396816254, "weights": [ [ [ -0.12174763530492783, 0.13658182322978973, -0.1651463508605957 ], [ -0.028811035677790642, -0.015335196629166603, -0.02954046055674553 ], [ 0.11986204236745834, -0.15042239427566528, 0.0074767074547708035 ] ], [ [ -0.1640665978193283, -0.04325765371322632, -0.05339731648564339 ], [ -0.016034750267863274, -0.022160634398460388, -0.13991817831993103 ], [ 0.025412749499082565, -0.16299155354499817, 0.010081378743052483 ] ], [ [ -0.20143799483776093, -0.11384277045726776, -0.10117499530315399 ], [ 0.06904461979866028, 0.050809185951948166, 0.08019406348466873 ], [ 0.22358885407447815, -0.010466795414686203, 0.21184107661247253 ] ], [ [ -0.10086043179035187, -0.15338215231895447, -0.10359030216932297 ], [ -0.061026688665151596, -0.09536128491163254, 0.11404836177825928 ], [ -0.019127026200294495, 0.14049454033374786, -0.12202276289463043 ] ], [ [ -0.07664071768522263, 0.015995649620890617, 0.15433888137340546 ], [ 0.06485911458730698, 0.09001573175191879, 0.06948885321617126 ], [ 0.04386594146490097, 0.038411740213632584, 0.11713247001171112 ] ], [ [ 0.09215138107538223, 0.12106510251760483, 0.23122583329677582 ], [ 0.14584136009216309, 0.16980870068073273, 0.16628189384937286 ], [ -0.04187764227390289, -0.10224214196205139, 0.019382622092962265 ] ], [ [ -0.05033596232533455, 0.12056105583906174, 0.22138570249080658 ], [ 0.10856883227825165, 0.1487428843975067, 0.0575917586684227 ], [ -0.00046111567644402385, 0.042668137699365616, 0.05538313090801239 ] ], [ [ 0.13978596031665802, 0.02549193613231182, 0.1446741670370102 ], [ 0.08918158710002899, -0.043298862874507904, 0.03623894974589348 ], [ 0.17429907619953156, 0.03104049526154995, -0.11564658582210541 ] ], [ [ 0.16242437064647675, -0.05185018479824066, -0.0121151739731431 ], [ -0.10923708975315094, -0.1006595641374588, -0.08410155028104782 ], [ 0.042500089854002, -0.17756427824497223, -0.15757030248641968 ] ], [ [ 0.1477101892232895, 0.13517220318317413, 0.16390906274318695 ], [ -0.09088374674320221, -0.03356750309467316, -0.11755392700433731 ], [ 0.029441041871905327, 0.08585233241319656, 0.0896700844168663 ] ] ] }, { "bias": 0.01446094922721386, "weights": [ [ [ -0.16259777545928955, -0.07919269800186157, -0.10109077394008636 ], [ 0.07295500487089157, -0.13824179768562317, -0.055377405136823654 ], [ 0.043624382466077805, -0.1642560511827469, 0.035963453352451324 ] ], [ [ 0.12694010138511658, 0.1857878863811493, 0.02139131911098957 ], [ -0.1294437199831009, -0.1224006712436676, 0.04927557706832886 ], [ 0.1591484695672989, 0.007865636609494686, -0.04981435835361481 ] ], [ [ 0.025919122621417046, -0.12693646550178528, 0.1762840300798416 ], [ 0.13712717592716217, -0.07527372986078262, 0.13072159886360168 ], [ 0.04369989410042763, -0.15994201600551605, -0.07772780954837799 ] ], [ [ 0.08389417082071304, 0.02161312662065029, 0.11244728416204453 ], [ 0.04562941938638687, -0.1850823312997818, -0.19985713064670563 ], [ 0.013847727328538895, 0.07941315323114395, -0.07868022471666336 ] ], [ [ -0.1346278041601181, -0.09600866585969925, -0.24615386128425598 ], [ -0.13599228858947754, -0.027469327673316002, -0.13686475157737732 ], [ 0.11155768483877182, 0.20709258317947388, 0.055043429136276245 ] ], [ [ -0.09350954741239548, -0.2341221421957016, -0.06315312534570694 ], [ 0.15690560638904572, 0.05876573547720909, 0.2008129358291626 ], [ 0.09133657813072205, 0.07357358932495117, 0.04054557904601097 ] ], [ [ 0.02289535477757454, 0.05094471201300621, 0.022777413949370384 ], [ 0.1911495476961136, 0.1095840260386467, 0.1875132918357849 ], [ 0.10030882805585861, 0.0299226064234972, 0.2583174705505371 ] ], [ [ -0.04467569664120674, -0.15514443814754486, 0.012017147615551949 ], [ -0.1360175758600235, 0.04701819643378258, 0.01685551553964615 ], [ -0.015009362250566483, 0.15808776021003723, 0.1380142867565155 ] ], [ [ 0.0014395368052646518, 0.10229988396167755, -0.08579538762569427 ], [ -0.04380276799201965, -0.003118126653134823, 0.07287978380918503 ], [ -0.050696540623903275, -0.010711082257330418, -0.05901684984564781 ] ], [ [ 0.12797345221042633, -0.041508764028549194, 0.05089915543794632 ], [ 0.08462511003017426, 0.15349045395851135, 0.06033384054899216 ], [ -0.17805145680904388, -0.09504542499780655, -0.11060364544391632 ] ] ] }, { "bias": -0.18813537061214447, "weights": [ [ [ -0.1358455866575241, 0.020784197375178337, 0.012531528249382973 ], [ -0.17718467116355896, 0.1581783890724182, -0.10088316351175308 ], [ -0.03216350078582764, -0.04750040918588638, 0.046315405517816544 ] ], [ [ -0.10508772730827332, -0.07013152539730072, -0.0639897957444191 ], [ 0.12097233533859253, -0.013156263157725334, -0.08829545974731445 ], [ 0.11504996567964554, 0.10036181658506393, -0.048247672617435455 ] ], [ [ 0.22100888192653656, 0.16265073418617249, 0.09039779752492905 ], [ 0.20307500660419464, 0.07338210940361023, 0.10677074640989304 ], [ 0.12762710452079773, 0.12233605980873108, 0.16212224960327148 ] ], [ [ 0.2026071697473526, 0.03163035959005356, 0.05191440507769585 ], [ 0.05101798474788666, -0.08335137367248535, 0.12224773317575455 ], [ -0.026638077571988106, -0.009244870394468307, 0.08258619904518127 ] ], [ [ 0.1353917270898819, 0.11299196630716324, 0.14517857134342194 ], [ 0.013255714438855648, 0.0006891486700624228, 0.0024036001414060593 ], [ 0.011934444308280945, -0.04438449814915657, -0.07786228507757187 ] ], [ [ 0.12286064773797989, 0.1729215383529663, 0.03957889601588249 ], [ 0.13248325884342194, -0.14979279041290283, 0.0712703987956047 ], [ -0.03144574910402298, 0.07466572523117065, -0.15872295200824738 ] ], [ [ 0.0500401109457016, 0.11128253489732742, -0.0670473501086235 ], [ 0.053425371646881104, -0.0975981280207634, -0.1515394151210785 ], [ -0.018341612070798874, 0.08227913826704025, -0.07649536430835724 ] ], [ [ -0.013060234487056732, 0.10545924305915833, -0.04617242142558098 ], [ -0.17272469401359558, -0.12658539414405823, -0.15309768915176392 ], [ -0.059895988553762436, 0.010617214255034924, -0.20278626680374146 ] ], [ [ -0.0945475772023201, 0.048694733530282974, -0.12032745778560638 ], [ -0.04117373004555702, -0.0721011757850647, 0.2760360538959503 ], [ -0.23647838830947876, 0.09139523655176163, 0.07709531486034393 ] ], [ [ 0.08168414980173111, -0.049900736659765244, -0.017324190586805344 ], [ -0.12146569043397903, 0.18098480999469757, 0.13150836527347565 ], [ -0.19960907101631165, -0.06940629333257675, -0.12759949266910553 ] ] ] }, { "bias": 0.032049767673015594, "weights": [ [ [ 0.07276657968759537, 0.17987872660160065, 0.015256171114742756 ], [ -0.12216264009475708, -0.06664621829986572, -0.16023167967796326 ], [ 0.06514883786439896, -0.16295680403709412, 0.042405229061841965 ] ], [ [ -0.14318525791168213, 0.13009771704673767, 0.0518876276910305 ], [ 0.1567896604537964, -0.051100775599479675, 0.14236561954021454 ], [ 0.0707058385014534, -0.029033901169896126, 0.2330576330423355 ] ], [ [ 0.06189902126789093, -0.25021445751190186, -0.1533758044242859 ], [ -0.05761691555380821, 0.11707664281129837, 0.10397264361381531 ], [ -0.10958359390497208, 0.19651801884174347, 0.13396427035331726 ] ], [ [ -0.06290310621261597, 0.008595671504735947, -0.19669604301452637 ], [ -0.21231496334075928, -0.0584346279501915, 0.10230404138565063 ], [ 0.013176584616303444, 0.1945866346359253, 0.20213302969932556 ] ], [ [ -0.15669439733028412, -0.1461036652326584, 0.031452495604753494 ], [ -0.1582546830177307, 0.11258459836244583, 0.2544539272785187 ], [ -0.053227197378873825, 0.25587764382362366, 0.1207578256726265 ] ], [ [ -0.09621194005012512, -0.013983515091240406, -0.042230118066072464 ], [ -0.13541610538959503, -0.21901044249534607, 0.03681187331676483 ], [ 0.015613908879458904, -0.018824201077222824, 0.014227106235921383 ] ], [ [ 0.29758620262145996, 0.30296194553375244, 0.08188604563474655 ], [ 0.2363891303539276, 0.14506536722183228, -0.0023157803807407618 ], [ 0.12809130549430847, -0.042837560176849365, 0.09612684696912766 ] ], [ [ 0.01854560151696205, -0.061136938631534576, 0.0714552104473114 ], [ 0.10207103937864304, -0.01072518341243267, 0.12656676769256592 ], [ -0.040705688297748566, 0.09779739379882812, 0.023327942937612534 ] ], [ [ -0.022386737167835236, 0.12966154515743256, 0.04666091501712799 ], [ 0.01978493109345436, -0.11666550487279892, -0.06739896535873413 ], [ -0.06784651428461075, 0.059927359223365784, 0.2724410891532898 ] ], [ [ -0.2819039225578308, -0.17425957322120667, -0.0023961211554706097 ], [ 0.06943542510271072, 0.010751917958259583, 0.20004703104496002 ], [ -0.04761410504579544, 0.0035902871750295162, 0.2276211827993393 ] ] ] }, { "bias": -0.1071610152721405, "weights": [ [ [ 0.16179943084716797, -0.15223224461078644, 0.05603885278105736 ], [ 0.03607629984617233, 0.04757857695221901, -0.13878557085990906 ], [ -0.1390153467655182, -0.09328515827655792, 0.15812215209007263 ] ], [ [ 0.1543763130903244, -0.03176015987992287, 0.08444511145353317 ], [ 0.06713499128818512, 0.1576194316148758, -0.03991593047976494 ], [ -0.0672399029135704, 0.052136920392513275, 0.06228874623775482 ] ], [ [ 0.05517008900642395, 0.032260771840810776, 0.32103991508483887 ], [ -0.15375973284244537, -0.0356624498963356, -0.09302377700805664 ], [ -0.2430153340101242, -0.0408388189971447, 0.009174891747534275 ] ], [ [ 0.24025054275989532, 0.14598426222801208, 0.1818782240152359 ], [ 0.20681455731391907, -0.019322190433740616, 0.22610841691493988 ], [ 0.003642961150035262, -0.08128095418214798, -0.1269708275794983 ] ], [ [ 0.048292554914951324, -0.2193034142255783, -0.003415939398109913 ], [ 0.07603807002305984, -0.1078244149684906, 0.0187393631786108 ], [ -0.08292911946773529, -0.25053516030311584, 0.07760937511920929 ] ], [ [ 0.06579913944005966, -0.1253804713487625, 0.15515655279159546 ], [ -0.09566143155097961, -0.19516333937644958, -0.19508473575115204 ], [ 0.01030671875923872, -0.177727609872818, 0.015511158853769302 ] ], [ [ 0.21972207725048065, 0.09551124274730682, 0.05152615159749985 ], [ 0.08212753385305405, 0.26710039377212524, 0.17063234746456146 ], [ 0.18379761278629303, -0.008331249468028545, 0.032313983887434006 ] ], [ [ 0.21882523596286774, 0.04904114082455635, 0.03894847258925438 ], [ -0.06760572642087936, -0.12773454189300537, 0.2312447726726532 ], [ -0.07193315774202347, -0.014704928733408451, 0.11695306748151779 ] ], [ [ 0.22869613766670227, 0.11457665264606476, -0.13291311264038086 ], [ 0.08686745911836624, -0.12870098650455475, -0.05958646908402443 ], [ -0.006725900340825319, 0.04420801252126694, -0.05865180864930153 ] ], [ [ -0.014937112107872963, -0.2035684734582901, -0.1690637618303299 ], [ -0.028141602873802185, 0.02778678573668003, -0.2195882499217987 ], [ 0.215241476893425, 0.09540508687496185, 0.11833357065916061 ] ] ] }, { "bias": 0.061665862798690796, "weights": [ [ [ -0.0025511737912893295, 0.020746171474456787, -0.06292499601840973 ], [ -0.13648031651973724, 0.05221004784107208, 0.11034466326236725 ], [ 0.04746418446302414, 0.043327465653419495, -0.12029217183589935 ] ], [ [ -0.12141206860542297, -0.021659456193447113, -0.1135789230465889 ], [ -0.19066378474235535, -0.20452120900154114, 0.0035310890525579453 ], [ -0.017933499068021774, 0.04559336602687836, 0.014429604634642601 ] ], [ [ 0.10998199135065079, 0.15145368874073029, -0.10743904858827591 ], [ 0.00623478926718235, -0.05017784237861633, 0.011177558451890945 ], [ -0.06357812881469727, -0.16046376526355743, 0.13748908042907715 ] ], [ [ 0.12345323711633682, -0.03187406808137894, 0.11310245841741562 ], [ 0.16261525452136993, 0.14970175921916962, -0.06283792853355408 ], [ 0.17747862637043, 0.1026889979839325, -0.15795530378818512 ] ], [ [ -0.031117647886276245, 0.05445241183042526, 0.1802845150232315 ], [ 0.2290922999382019, -0.007870027795433998, 0.012697769328951836 ], [ 0.1058914065361023, -0.07278214395046234, -0.0018795863725245 ] ], [ [ 0.15368755161762238, 0.2867761552333832, -0.09185421466827393 ], [ 0.0020685032941401005, 0.20404541492462158, 0.19803866744041443 ], [ 0.2220652550458908, 0.13969124853610992, 0.04406684264540672 ] ], [ [ -0.1418893188238144, 0.028295258060097694, -0.10037285834550858 ], [ -0.07259444892406464, -0.03481004014611244, 0.003599332645535469 ], [ 0.05741039291024208, -0.13806051015853882, 0.012858081609010696 ] ], [ [ 0.06935367733240128, -0.20636332035064697, 0.03540106117725372 ], [ 0.07383903115987778, 0.03785941004753113, -0.09802525490522385 ], [ -0.058929651975631714, -0.13806124031543732, 0.006437752395868301 ] ], [ [ -0.03544848412275314, 0.041394755244255066, 0.13945014774799347 ], [ 0.1043098196387291, 0.10708555579185486, -0.08619974553585052 ], [ -0.16758239269256592, 0.06004820391535759, 0.12389624863862991 ] ], [ [ -0.043459489941596985, 0.0891413688659668, -0.029467688873410225 ], [ 0.1968507170677185, 0.01814853958785534, -0.0600057914853096 ], [ -0.005719579290598631, -0.048423200845718384, 0.17595772445201874 ] ] ] } ] }, { "name": "relu_3_2", "input_shape": [ 9, 9, 10 ], "output_shape": [ 9, 9, 10 ], "num_neurons": 10 }, { "name": "max_pool_3", "input_shape": [ 9, 9, 10 ], "output_shape": [ 4, 4, 10 ], "num_neurons": 10 }, { "name": "flatten", "input_shape": [ 4, 4, 10 ], "output_shape": [ 160 ], "num_neurons": 160 }, { "name": "output", "input_shape": [ 160 ], "output_shape": [ 10 ], "num_neurons": 10, "weights": [ { "bias": -0.2513352632522583, "weights": [ 0.037237223237752914, -0.2085798680782318, 0.22239898145198822, -0.018747668713331223, 0.19035179913043976, -0.18876996636390686, -0.25303611159324646, -0.030314432457089424, 0.15639150142669678, -0.008488400839269161, 0.1312105506658554, -0.1418263167142868, 0.1837759017944336, 0.0009167467942461371, 0.19680120050907135, 0.01619894616305828, 0.13799647986888885, -0.17226719856262207, 0.2117895931005478, 0.05933043360710144, 0.12771391868591309, 0.01596875675022602, 0.0722590759396553, -0.04783660173416138, -0.05229617655277252, -0.07667139917612076, 0.08286578208208084, -0.22329497337341309, 0.1616649478673935, -0.04890842363238335, 0.16079849004745483, -0.24089208245277405, 0.04068567976355553, -0.15328344702720642, 0.24964410066604614, -0.045488741248846054, -0.06377352029085159, -0.28701427578926086, 0.033565498888492584, -0.18892920017242432, -0.03639240562915802, -0.2350505292415619, 0.11429664492607117, 0.02341393753886223, -0.11650383472442627, -0.06349196285009384, -0.11841116100549698, 0.0019236041698604822, 0.0964500904083252, -0.14897628128528595, 0.11839055269956589, -0.015549846924841404, 0.13918085396289825, 0.014651562087237835, 0.031121404841542244, 0.12348775565624237, -0.01968524605035782, -0.14411517977714539, -0.03137340396642685, 0.15658320486545563, 0.0929691344499588, -0.07419539988040924, 0.0623166449368, 0.14762182533740997, 0.01012086495757103, 0.11676504462957382, 0.0023846919648349285, 0.04194874316453934, 0.2167632281780243, 0.107383131980896, -0.02139628306031227, -0.1818522959947586, 0.0939592644572258, 0.06153164803981781, 0.02344508282840252, 0.05982291325926781, 0.03424457833170891, -0.03409535437822342, 0.06347865611314774, 0.1078614890575409, 0.17487934231758118, -0.19930392503738403, 0.2562271058559418, 0.03345225751399994, -0.06433529406785965, 0.06729006767272949, -0.03861946240067482, -0.15650668740272522, 0.028838133439421654, 0.006899332627654076, 0.18134081363677979, 0.008289948105812073, 0.2117832452058792, -0.11967208236455917, -0.1511869728565216, 0.03577937185764313, 0.13548101484775543, -0.12569761276245117, 0.1621827483177185, 0.040863316506147385, 0.0857875868678093, 0.09568747133016586, 0.24893949925899506, 0.10296972095966339, 0.07670018821954727, -0.16938653588294983, -0.05782659351825714, -0.1479375809431076, 0.004151251167058945, -0.036444827914237976, 0.13446637988090515, -0.08143768459558487, 0.18580955266952515, -0.13741815090179443, -0.0870201364159584, 0.061781659722328186, 0.13522882759571075, 0.06917691230773926, -0.08724942058324814, -0.13882677257061005, 0.04797711968421936, 0.004900653380900621, 0.037656333297491074, 0.1827707439661026, 0.14588098227977753, 0.10867556929588318, -0.07463699579238892, 0.026831157505512238, 0.0734119638800621, -0.3187274634838104, 0.14199024438858032, -0.15632282197475433, -0.04299023002386093, 0.004703454673290253, 0.05190644413232803, -0.0728243887424469, 0.04847163334488869, -0.05418686196208, 0.09738653153181076, -0.10706260800361633, 0.13013815879821777, -0.15482038259506226, 0.018822895362973213, 0.050359707325696945, 0.22491790354251862, 0.003010277636349201, -0.04754158854484558, -0.0860431045293808, 0.021679630503058434, -0.14216278493404388, 0.1486811339855194, -0.23144711554050446, -0.09288418292999268, 0.031953901052474976, -0.03763307258486748, 0.053416907787323, 0.015163575299084187, 0.01830022782087326, 0.019117359071969986, 0.03853989765048027 ] }, { "bias": 0.06519734114408493, "weights": [ 0.02079402282834053, -0.11754846572875977, -0.19642002880573273, -0.24779118597507477, 0.03460664302110672, 0.04089893400669098, -0.36112111806869507, 0.20083148777484894, 0.033368732780218124, -0.12880755960941315, -0.15078726410865784, -0.2041657567024231, 0.04551428556442261, -0.24495230615139008, -0.055047597736120224, -0.011546147055923939, -0.11893712729215622, -0.1647050678730011, -0.029689233750104904, 0.010487670078873634, 0.07262550294399261, 0.12508803606033325, -0.020908445119857788, -0.23473933339118958, -0.1036846935749054, -0.08660558611154556, -0.04750441759824753, 0.10566245019435883, -0.09703733772039413, -0.003489075228571892, -0.03992157801985741, -0.05630800500512123, -0.20488663017749786, -0.058865614235401154, -0.0873987227678299, 0.014982353895902634, -0.2316000908613205, 0.07593020051717758, 0.04896589368581772, -0.19822505116462708, -0.05083863064646721, 0.15500585734844208, -0.10749870538711548, 0.08721087872982025, -0.05278895050287247, -0.015682891011238098, -0.07422029972076416, 0.17600804567337036, 0.0158672071993351, 0.22179925441741943, -0.08887603133916855, 0.02084948681294918, 0.008970390073955059, 0.11157546937465668, 0.007794892881065607, 0.09842189401388168, 0.1446964293718338, 0.14059989154338837, 0.23860600590705872, 0.12240061163902283, 0.1566430777311325, 0.06698709726333618, 0.14156758785247803, 0.20906579494476318, 0.04928354173898697, -0.1434832662343979, 0.12041249871253967, -0.07896735519170761, -0.029586967080831528, 0.062060169875621796, 0.01035227533429861, 0.08735382556915283, -0.05076536163687706, -0.044588249176740646, -0.035106439143419266, -0.1356794834136963, 0.07217061519622803, -0.024540981277823448, -0.06691071391105652, -0.05676655098795891, 0.10740158706903458, 0.04756231978535652, -0.11898987740278244, -0.030191099271178246, 0.09835253655910492, -0.09163330495357513, 0.1027570515871048, -0.02512558549642563, -0.016787171363830566, -0.12739400565624237, -0.13635672628879547, -0.08522175252437592, 0.10673031210899353, 0.016609013080596924, 0.07704536616802216, -0.0576607845723629, 0.2912350594997406, 0.14694605767726898, 0.0024474398232996464, 0.10420805215835571, 0.15038296580314636, 0.1361939162015915, -0.03876202553510666, -0.007628224324434996, 0.24061846733093262, -0.18753314018249512, 0.15079623460769653, -0.030442936345934868, -0.14139853417873383, 0.21154966950416565, 0.0838024914264679, 0.05419695004820824, -0.02998300828039646, -0.011226293630897999, -0.04138803854584694, 0.04061892628669739, 0.1276482343673706, 0.1606159657239914, 0.037188563495874405, -0.1970548778772354, 0.09592647105455399, 0.10023986548185349, -0.1526363492012024, -0.2351713627576828, 0.10576551407575607, -0.06816257536411285, -0.12890216708183289, 0.07298823446035385, -0.14376012980937958, 0.03043556958436966, -0.04120851680636406, -0.00672142906114459, 0.09243026375770569, -0.12206718325614929, 0.1518348604440689, 0.09600169211626053, -0.035856325179338455, -0.09588852524757385, -0.05772422254085541, 0.08547937124967575, 0.0068990495055913925, 0.12570695579051971, -0.19614481925964355, -0.10259822010993958, 0.01009683683514595, -0.13089512288570404, 0.10345528274774551, 0.05156470835208893, -0.15156997740268707, -0.07541146874427795, -0.11222236603498459, 0.21292605996131897, -0.13383132219314575, -0.2018393576145172, 0.03841444477438927, -0.005980853457003832, -0.20552890002727509, -0.023261312395334244, -0.038892123848199844, -0.01693912036716938 ] }, { "bias": 0.04120241478085518, "weights": [ 0.08600084483623505, -0.15282900631427765, -0.027479447424411774, -0.04215899482369423, -0.2327398955821991, -0.00358211575075984, 0.08998187631368637, 0.13590587675571442, 0.040852587670087814, 0.07764560729265213, 0.18494489789009094, 0.013980111107230186, 0.03266307711601257, 0.07039047032594681, 0.03717689961194992, 0.20998527109622955, 0.03674859181046486, 0.03126056492328644, 0.027937866747379303, -0.0662318766117096, 0.14224347472190857, -0.011425723321735859, -0.03436950221657753, -0.17810063064098358, -0.0452599823474884, 0.024546818807721138, -0.06436756253242493, -0.09777086228132248, -0.1455928385257721, 0.02671065181493759, 0.11764450371265411, -0.08235208690166473, 0.1531396210193634, 0.12281898409128189, -0.24636049568653107, -0.02294488251209259, 0.052015628665685654, -0.12787996232509613, -0.10134577006101608, 0.22447580099105835, -0.08638007938861847, -0.11863862723112106, 0.011148069985210896, 0.016892261803150177, -0.0375971756875515, 0.00924800056964159, 0.05850135162472725, -0.0026676931884139776, -0.1703430712223053, 0.07496099919080734, -0.18020468950271606, -0.03123406320810318, -0.035994257777929306, -0.03662952780723572, 0.05994078144431114, 0.006410139612853527, -0.11289823055267334, 0.07897420972585678, 0.03905794396996498, 0.04750785604119301, -0.1678929328918457, -0.01354603935033083, -0.06797380745410919, -0.12825189530849457, -0.191240593791008, 0.0038869220297783613, -0.08653810620307922, 0.04066885635256767, -0.12487009912729263, 0.06414137780666351, -0.037770166993141174, 0.09392068535089493, -0.05836348980665207, 0.047921646386384964, -0.05376650393009186, 0.05610073357820511, 0.13865968585014343, 0.0442340262234211, -0.02996636927127838, 0.09017479419708252, -0.14603812992572784, -0.15702925622463226, 0.09836402535438538, -0.034204814583063126, -0.21223345398902893, 0.08547372370958328, 0.03568708151578903, -0.18350523710250854, -0.04710116609930992, -0.006407922599464655, -0.039264336228370667, -0.026584159582853317, -0.19885477423667908, -0.026866992935538292, 0.02649182453751564, -0.07081405818462372, -0.12021728605031967, -0.045769963413476944, -0.045715611428022385, 0.010395940393209457, 0.14889976382255554, 0.08393600583076477, -0.013625028543174267, -0.184380441904068, -0.22028972208499908, -0.05215242877602577, -0.07085873931646347, -0.2045580893754959, -0.07469934970140457, -0.04117636755108833, -0.1794842928647995, -0.05560043454170227, 0.03548799827694893, -0.08768782764673233, -0.15210869908332825, -0.03236064314842224, 0.11876364797353745, -0.1098104789853096, -0.005126004572957754, -0.042576082050800323, 0.1425704061985016, 0.1461399495601654, 0.01607448048889637, -0.2623697817325592, -0.04019223526120186, 0.07856371253728867, 0.15838487446308136, -0.27415820956230164, -0.16817432641983032, 0.09030286967754364, -0.0840967521071434, 0.17270471155643463, 0.18276546895503998, -0.1929769366979599, -0.2359301745891571, 0.0933491438627243, 0.11974931508302689, 0.07347185164690018, 0.03694111481308937, 0.04725728556513786, -0.05635693669319153, -0.11006646603345871, 0.006177674047648907, 0.01241665706038475, -0.123158298432827, -0.14310431480407715, -0.1627674251794815, -0.2512933611869812, 0.06060843914747238, 0.18860188126564026, 0.15036679804325104, -0.06769870221614838, 0.07621799409389496, -0.08008167892694473, 0.0892903134226799, -0.18899571895599365, 0.07670537382364273, -0.09369657188653946, 0.049252886325120926, 0.04767722636461258 ] }, { "bias": 0.03328557312488556, "weights": [ -0.08469369262456894, -0.07368701696395874, -0.1512986570596695, 0.2565789222717285, -0.013534227386116982, 0.011298569850623608, 0.05071094632148743, -0.1242319792509079, 0.07724617421627045, 0.11585883051156998, -0.08613696694374084, 0.01709127426147461, 0.14706312119960785, 0.03544112294912338, -0.03719817101955414, 0.09248726814985275, -0.20743463933467865, 0.10114835202693939, -0.03204192593693733, 0.04098070040345192, 0.010841119103133678, -0.1099933311343193, -0.12631045281887054, -0.048705197870731354, 0.06112825870513916, -0.0438520610332489, -0.10077061504125595, -0.019756561145186424, 0.05961857736110687, -0.09528839588165283, -0.009388601407408714, 0.20330873131752014, 0.0077501204796135426, 0.13931621611118317, 0.1991031914949417, -0.06700847297906876, 0.08266443014144897, -0.07973328232765198, 0.038616426289081573, 0.0636761337518692, 0.1014070212841034, -0.11712313443422318, 0.03464946523308754, 0.06275692582130432, -0.03704669326543808, -0.14066182076931, -0.13146544992923737, 0.05711666867136955, 0.23695756494998932, -0.09217248857021332, 0.21224084496498108, 0.06584629416465759, 0.021800076588988304, 0.09231077879667282, 0.07490959763526917, 0.15343786776065826, 0.16477355360984802, -0.12703096866607666, 0.03442982956767082, -0.11825532466173172, 0.032010845839977264, -0.04941524937748909, -0.03632350638508797, -0.1250848025083542, 0.09015306085348129, -0.16828550398349762, 0.13186994194984436, 0.007492118515074253, -0.020117156207561493, -0.1650894582271576, 0.12328720837831497, 0.018113315105438232, -0.04145171865820885, -0.00039698652108199894, 0.07897568494081497, 0.14171190559864044, -0.005400299560278654, -0.007740440778434277, -0.052659858018159866, -0.0036274211015552282, 0.15316137671470642, 0.13744430243968964, -0.02076825313270092, -0.06474146246910095, 0.004920308478176594, -0.008472095243632793, 0.08498229086399078, -0.08411326259374619, 0.034755319356918335, 0.145986407995224, -0.03708430007100105, 0.13377876579761505, 0.05018776282668114, 0.04652559012174606, 0.023020414635539055, -0.020148739218711853, -0.08832806348800659, -0.08817742019891739, -0.13774582743644714, 0.14065656065940857, 0.11330980062484741, 0.23775307834148407, -0.13410240411758423, 0.041749853640794754, -0.049154892563819885, 0.13007943332195282, -0.12735864520072937, -0.04375014081597328, -0.14050902426242828, 0.0044339741580188274, 0.14104455709457397, -0.04526012763381004, 0.029577406123280525, 0.03591854125261307, -0.18595625460147858, 0.16793856024742126, -0.1735418438911438, 0.11090452969074249, 0.08248969167470932, 0.031469836831092834, -0.20477627217769623, 0.08334281295537949, 0.03731105849146843, -0.018131623044610023, 0.10176900774240494, 0.04530928283929825, -0.20170095562934875, 0.07753785699605942, 0.005386087577790022, 0.0879441350698471, 0.11917395144701004, 0.10809807479381561, -0.03257959708571434, -0.08768340945243835, -0.15129457414150238, 0.13132545351982117, -0.05518629401922226, 0.1352892816066742, -0.1272444874048233, -0.1279493123292923, 0.11846945434808731, 0.13123778998851776, 0.02546725608408451, -0.18402616679668427, -0.25343838334083557, 0.1608027219772339, 0.14363862574100494, 0.095777228474617, 0.12361801415681839, -0.14778845012187958, -0.0717211589217186, 0.17189612984657288, -0.03806455805897713, 0.06086942180991173, -0.19953981041908264, -0.17191222310066223, -0.18540118634700775, 0.1942734569311142, 0.03480803221464157, 0.13359607756137848 ] }, { "bias": -0.049977295100688934, "weights": [ -0.05572054535150528, -0.1551874876022339, 0.020352210849523544, 0.09161262959241867, 0.10101374983787537, -0.022689171135425568, 0.1960146725177765, 0.022763028740882874, 0.12757505476474762, 0.17756937444210052, 0.022784404456615448, 0.18289858102798462, 0.08029237389564514, 0.10200183838605881, 0.20415280759334564, -0.21716825664043427, -0.017852002754807472, -0.06801117956638336, 0.2604348063468933, 0.19233711063861847, 0.07736406475305557, -0.08600379526615143, -0.000648561748676002, 0.08696174621582031, 0.24168306589126587, -0.17914535105228424, -0.019220460206270218, 0.010958154685795307, 0.18479633331298828, 0.1812601238489151, 0.12014281004667282, 0.07065637409687042, 0.13548587262630463, 0.21402156352996826, 0.16186054050922394, 0.028483539819717407, 0.1447264850139618, 0.11166881769895554, 0.03180659934878349, -0.06799910217523575, 0.1424344778060913, -0.16308742761611938, 0.12037429213523865, 0.08241201937198639, -0.006381646264344454, -0.1439223289489746, 0.018708644434809685, 0.17588558793067932, 0.10204339027404785, 0.03225938603281975, -0.16779983043670654, -0.1852608621120453, 0.12777948379516602, 0.14605174958705902, 0.10819468647241592, 0.18260523676872253, 0.039845060557127, 0.1529584378004074, 0.04988490045070648, 0.0747872069478035, 0.16076454520225525, -0.24388034641742706, 0.1855817288160324, -0.05500513315200806, 0.05364350229501724, -0.04310567304491997, -0.12908554077148438, 0.04129517823457718, -0.09934262931346893, -0.12845230102539062, -0.06754323840141296, -0.21861182153224945, 0.00018565059872344136, -0.07219569385051727, 0.26546522974967957, 0.03840245306491852, 0.14276957511901855, 0.14317616820335388, 0.02138613536953926, -0.010423548519611359, -0.12960036098957062, 0.028418630361557007, 0.17652270197868347, 0.1768396496772766, -0.03356723487377167, -0.09414089471101761, -0.06974460929632187, 0.24123646318912506, 0.08451832830905914, -0.1976855844259262, -0.1876990795135498, 0.04215606674551964, 0.1136370375752449, -0.12485310435295105, -0.052624695003032684, -0.004095508251339197, 0.10575198382139206, 0.0005804583197459579, -0.05689302831888199, -0.1730596274137497, -0.10368581116199493, -0.14030338823795319, 0.038566138595342636, -0.0693182647228241, -0.088661789894104, 0.07880612462759018, -0.02643081173300743, 0.20525892078876495, 0.14734841883182526, -0.04572084918618202, -0.16984106600284576, 0.05686834454536438, -0.04002545028924942, 0.19997598230838776, 0.13044387102127075, 0.10726533830165863, 0.022250255569815636, 0.0871301144361496, 0.07369503378868103, -0.2776543200016022, -0.009098000824451447, -0.149514839053154, 0.0821806862950325, 0.13884590566158295, 0.13742420077323914, 0.0424516536295414, 0.007404979784041643, 0.3190549612045288, 0.07036988437175751, -0.1615973263978958, -0.06823543459177017, -0.07634284347295761, -0.030366120859980583, 0.09335874766111374, -0.0037275233771651983, -0.12423108518123627, -0.11606099456548691, 0.2158622443675995, -0.10634196549654007, -0.22440743446350098, -0.09882140159606934, -0.20353126525878906, 0.039445336908102036, -0.04115420952439308, 0.2158142477273941, -0.14689353108406067, -0.18669050931930542, 0.004791576880961657, -0.03494725376367569, -0.16598285734653473, -0.024321993812918663, 0.007548720110207796, 0.04897141456604004, 0.151950404047966, -0.12424694746732712, 0.11363803595304489, -0.029262833297252655, -0.22700084745883942, 0.18970008194446564, -0.27406176924705505 ] }, { "bias": 0.10740397870540619, "weights": [ -0.09817632287740707, 0.09360858052968979, -0.2206110954284668, -0.03642928972840309, 0.019186800345778465, 0.009339817799627781, 0.0678006112575531, -0.12545500695705414, -0.17405526340007782, 0.1787732094526291, 0.016165463253855705, -0.1535927951335907, -0.23425143957138062, 0.10757003724575043, -0.06270799785852432, -0.04213930293917656, -0.00785924680531025, -0.0915057510137558, -0.1977023184299469, 0.18485864996910095, -0.08709600567817688, -0.07833657413721085, -0.23701629042625427, -0.02431904524564743, -0.05959422141313553, -0.12913313508033752, 0.29110148549079895, -0.07331027090549469, -0.0945194661617279, 0.014398790895938873, -0.20398154854774475, -0.0639834851026535, -0.28068962693214417, -0.07411859184503555, 0.17561130225658417, 0.03084406815469265, 0.13568758964538574, -0.17196929454803467, -0.17531254887580872, 0.24685105681419373, -0.07949556410312653, -0.06590619683265686, -0.08142048120498657, 0.06871413439512253, 0.10955297201871872, -0.09832993894815445, 0.2631336450576782, -0.12286641448736191, -0.2420239895582199, 0.17391496896743774, -0.10369700938463211, 0.08312420547008514, 0.028955940157175064, -0.05447767302393913, -0.10312772542238235, 0.008984916843473911, -0.10846628993749619, 0.011468200944364071, -0.04736282303929329, 0.019867198541760445, -0.24239042401313782, -0.13549824059009552, -0.2936073839664459, -0.22102601826190948, 0.06334040313959122, -0.13097910583019257, 0.06423677504062653, -0.2156268060207367, -0.06296639889478683, 0.0018850264605134726, -0.19911891222000122, -0.1331825703382492, -0.17773933708667755, -0.07899640500545502, -0.07640063017606735, -0.17165009677410126, -0.0670141652226448, 0.1168694868683815, -0.08239220082759857, 0.10499954223632812, 0.13124549388885498, 0.16127580404281616, -0.27503979206085205, -0.10864947736263275, 0.046090465039014816, -0.021556736901402473, -0.027808692306280136, 0.013691266998648643, -0.09099084138870239, 0.11974895745515823, -0.1061704084277153, 0.1164446473121643, -0.02484109252691269, -0.12037941068410873, -0.02865842543542385, 0.014138509519398212, -0.08974955230951309, -0.10594083368778229, -0.15397924184799194, -0.017448142170906067, -0.3773985803127289, -0.13716748356819153, -0.1757728010416031, 0.08123417943716049, -0.04952746257185936, -0.10134467482566833, -0.1725357621908188, -0.009008901193737984, -0.19613268971443176, -0.1252610832452774, -0.18627676367759705, 0.11160551756620407, -0.05294667184352875, -0.1735457181930542, -0.048068828880786896, 0.03661644086241722, 0.10665728151798248, 0.05310671776533127, -0.05112740397453308, -0.04014366492629051, -0.2204791009426117, -0.1124662235379219, -0.14420171082019806, -0.12129344791173935, -0.045645374804735184, 0.04447237029671669, -0.005466880742460489, 0.05493749678134918, -0.2751377522945404, 0.35105863213539124, -0.19648021459579468, -0.040038101375103, -0.05119453743100166, -0.057337936013936996, -0.09457419067621231, -0.25313904881477356, 0.06937748193740845, -0.18291690945625305, -0.2244023233652115, -0.14007000625133514, -0.2888190746307373, 0.010960889980196953, -0.25302475690841675, -0.08481965959072113, -0.05915164574980736, 0.030427807942032814, 0.04273337498307228, -0.09530351310968399, -0.16611160337924957, 0.11970489472150803, -0.3337106704711914, -0.1330523043870926, -0.08704158663749695, 0.10914306342601776, -0.0006350909825414419, -0.3267476260662079, 0.014995367266237736, 0.10203836113214493, -0.20047084987163544, 0.04865623638033867 ] }, { "bias": 0.1981860101222992, "weights": [ -0.27284112572669983, 0.0540563128888607, 0.0998094454407692, 0.00199695467017591, -0.01917155645787716, 0.036235369741916656, -0.21463792026042938, 0.03745388612151146, 0.004020120482891798, -0.03348729759454727, -0.18775486946105957, 0.04625977948307991, -0.08220256119966507, -0.02491600625216961, -0.14655394852161407, 0.04158813878893852, -0.023304522037506104, -0.10217802226543427, 0.029488958418369293, 0.07126743346452713, -0.19182774424552917, -0.20424877107143402, -0.1188196912407875, 0.017517048865556717, -0.1786332130432129, 0.2456461787223816, -0.10822447389364243, -0.029559070244431496, 0.0666329637169838, -0.13657335937023163, 0.054573904722929, 0.11644081771373749, 0.020247742533683777, -0.001992663834244013, 0.15066377818584442, 0.20816725492477417, 0.15670248866081238, 0.014836751855909824, -0.050185926258563995, 0.09624837338924408, -0.07556010782718658, 0.20634761452674866, 0.02330104447901249, 0.1292237490415573, 0.0666479766368866, 0.12132132798433304, -0.0023121866397559643, -0.007688293233513832, -0.07181555032730103, 0.23851683735847473, 0.03026459738612175, -0.12228409945964813, 0.1331338882446289, 0.044146109372377396, 0.08712542057037354, 0.23654964566230774, -0.08287416398525238, 0.05321338772773743, 0.14882305264472961, -0.017454281449317932, 0.09899231046438217, -0.06365566700696945, -0.05781932175159454, -0.029267551377415657, -0.05585131794214249, -0.051691435277462006, 0.03675556182861328, -0.008843853138387203, 0.07112447917461395, -0.12925799190998077, -0.03820851817727089, 0.22050751745700836, 0.19380871951580048, -0.14906930923461914, 0.16994433104991913, -0.0308974739164114, -0.0971590057015419, 0.04362250119447708, -0.03274126723408699, 0.08698645979166031, -0.09849371761083603, 0.04764360189437866, -0.017240118235349655, 0.08262528479099274, 0.0040047550573945045, 0.1820852905511856, -0.24041011929512024, -0.05690561234951019, -0.07320531457662582, 0.07184021919965744, -0.11587467044591904, 0.06611733138561249, 0.04691815376281738, -0.09590533375740051, -0.07177133858203888, 0.2021479606628418, -0.029307160526514053, 0.0022750445641577244, -0.13753801584243774, 0.07947014272212982, -0.018867958337068558, 0.02580343745648861, -0.11479823291301727, 0.08818550407886505, -0.05761965364217758, 0.29467448592185974, -0.020573871210217476, 0.10719917714595795, 0.0674787238240242, -0.21910299360752106, -0.10424784570932388, -0.05166369676589966, 0.07482614368200302, 0.09548874944448471, 0.16565583646297455, 0.06946326047182083, -0.15518254041671753, 0.15225088596343994, -0.059569139033555984, 0.17331576347351074, -0.15469063818454742, -0.011736918240785599, -0.0743703693151474, -0.02521207369863987, 0.21097147464752197, 0.015317046083509922, -0.10350048542022705, -0.20174823701381683, -0.027751153334975243, 0.34808358550071716, -0.1496248096227646, -0.09757106006145477, -0.07120630890130997, -0.04872015118598938, -0.1251097172498703, 0.19881290197372437, -0.12300252169370651, -0.1502315253019333, -0.0363832451403141, -0.08398815989494324, 0.012706249952316284, 0.060364462435245514, 0.06915893405675888, 0.03780388832092285, -0.01439569890499115, 0.08950001001358032, -0.1820613294839859, 0.10769841074943542, 0.13988932967185974, -0.01807638816535473, -0.06515968590974808, -0.2280558943748474, -0.10001692920923233, -0.09029679745435715, 0.16249671578407288, 0.06676555424928665, -0.053001731634140015, 0.02137565053999424, -0.06049026548862457, 0.162782222032547 ] }, { "bias": 0.0370665043592453, "weights": [ -0.26121050119400024, 0.10680586099624634, 0.0461772084236145, -0.16212515532970428, 0.003976386971771717, 0.07709202915430069, -0.12171993404626846, -0.038415055721998215, -0.17564398050308228, 0.01424784492701292, 0.09240981191396713, 0.05835915356874466, 0.2542452812194824, -0.0135171664878726, -0.3632211983203888, 0.05839724838733673, 0.02638164721429348, -0.05304727330803871, -0.00911666452884674, 0.18669220805168152, -0.2187773585319519, 0.12940119206905365, 0.15737488865852356, 0.01647847890853882, -0.06713542342185974, -0.07137651741504669, -0.13015934824943542, -0.2209329456090927, -0.0966019555926323, 0.25791457295417786, -0.21341100335121155, -0.035787537693977356, -0.14084142446517944, 0.0818888247013092, -0.18205320835113525, 0.1392735093832016, 0.007235927972942591, -0.03819909319281578, -0.12506785988807678, 0.008972321636974812, 0.024128643795847893, 0.07486371695995331, 0.0830928161740303, 0.10497559607028961, -0.09628425538539886, 0.1274452954530716, 0.09825670719146729, -0.0626559853553772, -0.11526541411876678, 0.026937123388051987, -0.18119166791439056, 0.0732833594083786, 0.07145573943853378, -0.11325812339782715, 0.11196794360876083, 0.13995517790317535, -0.04731713607907295, -0.01690071076154709, -0.3084547221660614, -0.010559958405792713, 0.02768581360578537, -0.15354549884796143, -0.05657998472452164, -0.0037228295113891363, -0.16291484236717224, -0.1428159922361374, -0.048418737947940826, -0.008436597883701324, 0.017381368204951286, 0.17255446314811707, -0.0776229053735733, -0.03611063212156296, -0.12281385809183121, -0.07541469484567642, -0.1406850516796112, -0.055661462247371674, -0.16544950008392334, -0.014977896586060524, -0.30811694264411926, 0.13425123691558838, -0.10105883330106735, -0.011093179695308208, 0.02761003002524376, 0.1308542788028717, 0.14720095694065094, 0.06768316775560379, -0.12713392078876495, 0.08260264992713928, -0.12877966463565826, 0.04395975172519684, -0.0896310806274414, -0.09982939809560776, -0.04898013547062874, 0.1607513129711151, 0.07599145919084549, 0.04804789274930954, -0.09473568201065063, -0.042790886014699936, -0.05863460153341293, -0.02368701808154583, -0.10920858383178711, 0.02913752570748329, 0.10707156360149384, 0.2899341881275177, -0.0159510038793087, -0.09972861409187317, -0.025558440014719963, 0.008900151588022709, -0.20323482155799866, 0.13385596871376038, -0.03748917579650879, -0.03837504982948303, -0.17934364080429077, 0.20230770111083984, -0.196036696434021, -0.00694623775780201, -0.1302284449338913, -0.08840484172105789, -0.10602930933237076, 0.12468640506267548, -0.11644363403320312, 0.025978771969676018, -0.005548249464482069, 0.2190469205379486, 0.023829400539398193, 0.08599808812141418, 0.05084935575723648, 0.2142515927553177, -0.22909900546073914, 0.006318274885416031, -0.21812517940998077, 0.004667122382670641, -0.11009600013494492, 0.10025832056999207, 0.07672252506017685, 0.05106695368885994, 0.05383049324154854, 0.045482657849788666, -0.2300644963979721, 0.0935501977801323, -0.06428369134664536, -0.1584600806236267, -0.19616837799549103, 0.30590179562568665, 0.06762436032295227, -0.03887428715825081, -0.11756851524114609, 0.22885584831237793, -0.10290771722793579, 0.04584900289773941, 0.014222455210983753, -0.0825687050819397, -0.2224753350019455, -0.03995548188686371, -0.07024715095758438, -0.031246084719896317, 0.12813539803028107, 0.12576213479042053, -0.1577892154455185, -0.10511001199483871 ] }, { "bias": 0.03854462504386902, "weights": [ -0.09666208177804947, 0.029394706711173058, -0.1254337877035141, 0.11446832120418549, 0.08943864703178406, 0.04882677271962166, 0.0020381093490868807, -0.09975072741508484, 0.09895923733711243, -0.13372060656547546, -0.1110915094614029, -0.07699407637119293, -0.08014658838510513, -0.0859091728925705, 0.03857537731528282, 0.10735710710287094, -0.0371607281267643, -0.12063384801149368, 0.07624576985836029, 0.038483791053295135, -0.10902794450521469, 0.022159377112984657, 0.13537147641181946, -0.06897123157978058, 0.18731948733329773, 0.06551744788885117, -0.033693280071020126, -0.1659877449274063, 0.00724980840459466, -0.00790323968976736, 0.15806816518306732, -0.003911122214049101, -0.11398297548294067, 0.08504553884267807, -0.011943445540964603, -0.0021410456392914057, -0.25380584597587585, 0.09608065336942673, 0.16827677190303802, -0.18283317983150482, -0.16612882912158966, 0.018003389239311218, 0.08439077436923981, 0.03611861914396286, -0.04194595664739609, -0.14725127816200256, -0.1306677907705307, 0.0406867153942585, 0.0784306675195694, -0.13322554528713226, -0.05561885982751846, 0.0038844668306410313, 0.021333331242203712, -0.0033197312150150537, 0.12906287610530853, -0.10337760299444199, 0.07833029329776764, 0.08704245835542679, 0.17515510320663452, 0.09370747208595276, -0.12477362900972366, 0.16433316469192505, -0.11719950288534164, 0.2011999934911728, -0.02053927257657051, 0.10993778705596924, -0.07146348804235458, 0.2645721137523651, 0.1467093527317047, 0.1399621069431305, -0.1954391449689865, -0.025107722729444504, 0.14887191355228424, 0.10378609597682953, 0.001424496527761221, -0.11814416944980621, -0.06833596527576447, -0.07151030749082565, 0.07245617359876633, 0.15705761313438416, 0.042368583381175995, 0.030194761231541634, -0.06286198645830154, 0.004694900941103697, -0.018568556755781174, -0.10706563293933868, 0.11104834079742432, -0.12686151266098022, -0.04825436323881149, 0.2427487075328827, -0.028469813987612724, 0.041383400559425354, 0.22761604189872742, 0.11543919891119003, 0.07864727824926376, -0.02435583993792534, -0.144240140914917, -0.08225312829017639, 0.12744276225566864, -0.09187740832567215, 0.06431471556425095, -0.07465823739767075, 0.05263656750321388, 0.09415699541568756, 0.021273881196975708, 0.13154473900794983, -0.17318099737167358, -0.10128828883171082, 0.181057408452034, -0.010079347528517246, -0.08375677466392517, 0.026337042450904846, -0.026335477828979492, 0.11287073791027069, 0.04385574162006378, 0.07973909378051758, -0.025874832645058632, -0.023699169978499413, -0.01767638884484768, -0.05727190524339676, -0.14186827838420868, -0.04676585644483566, 0.14679542183876038, 0.11656296253204346, -0.08118315786123276, -0.09438435733318329, -0.27734941244125366, -0.05376912280917168, 0.15657515823841095, -0.004638702142983675, 0.13610883057117462, -0.04935608431696892, 0.12849493324756622, 0.03019319288432598, 0.08441486209630966, -0.07159248739480972, -0.1660023182630539, -0.05452330410480499, 0.004930651281028986, 0.017215538769960403, -0.043742891401052475, 0.23605124652385712, 0.028741635382175446, -0.14261862635612488, -0.15088991820812225, 0.01460717711597681, -0.06834860891103745, -0.08679278194904327, -0.21159447729587555, 0.1392872929573059, -0.03119533881545067, 0.20758967101573944, 0.019692832604050636, 0.02798999845981598, 0.023634497076272964, -0.07321225851774216, -0.10218268632888794, 0.09177958220243454, 0.1011536568403244, -0.02782781608402729 ] }, { "bias": -0.21957211196422577, "weights": [ -0.08631733804941177, -0.09169624745845795, -0.0868954285979271, -0.2362481951713562, -0.01671820506453514, 0.19936442375183105, 0.025327913463115692, 0.18400217592716217, -0.050545986741781235, 0.024359596893191338, 0.09668876975774765, -0.2641465663909912, 0.0683712363243103, -0.06481257826089859, -0.1372736543416977, -0.06165653467178345, 0.09512709826231003, -0.007064086385071278, 0.02610013075172901, -0.11298362910747528, 0.13367030024528503, -0.09786675125360489, 0.030519912019371986, 0.09724839776754379, 0.12115022540092468, 0.2557755410671234, -0.03782111778855324, 0.16955135762691498, 0.01413385197520256, 0.030685709789395332, -0.10797213017940521, -0.15391862392425537, -0.00028278533136472106, 0.053137414157390594, 0.006760092917829752, 0.16912689805030823, 0.10216201096773148, 0.017426462844014168, 0.044181615114212036, -0.2847808003425598, 0.09711595624685287, 0.012986400164663792, -0.12308140099048615, -0.1203189417719841, -0.084618479013443, 0.16808536648750305, -0.003849987406283617, 0.1495172381401062, -0.07037550210952759, -0.12532570958137512, 0.1818188577890396, -0.09584223479032516, -0.15509819984436035, -0.07505150884389877, 0.005048006772994995, 0.19233135879039764, 0.018545635044574738, 0.18574918806552887, -0.21432122588157654, -0.20721225440502167, 0.18027199804782867, -0.08862737566232681, -0.11541688442230225, -0.32148972153663635, 0.15248200297355652, 0.17131578922271729, 0.11279366165399551, -0.020592020824551582, 0.09154266864061356, -0.12984587252140045, 0.07451387494802475, -0.12150340527296066, 0.1821117401123047, -0.06719622015953064, -0.06629617512226105, 0.05715804919600487, -0.12847685813903809, 0.22482816874980927, 0.13104964792728424, 0.04575655981898308, -0.061889708042144775, 0.07622509449720383, -0.0012407713802531362, -0.24739132821559906, 0.1261616200208664, 0.21985629200935364, -0.02537592127919197, 0.07340868562459946, 0.11279113590717316, 0.20221973955631256, 0.11219894140958786, 0.168039932847023, -0.12270182371139526, -0.261880099773407, -8.49431671667844e-06, -0.07801557332277298, -0.07929310947656631, 0.1982761174440384, 0.13327869772911072, -0.0644545927643776, 0.018913015723228455, 0.06036343425512314, 0.12980662286281586, -0.01493810210376978, 0.16211330890655518, 0.04009845852851868, -0.1935233473777771, 0.27241766452789307, 0.05454820767045021, 0.06039433181285858, 0.013994371518492699, -0.06063270568847656, -0.11903529614210129, 0.07067041844129562, 0.11669784784317017, 0.16885024309158325, 0.00928397849202156, 0.1473366767168045, 0.13088810443878174, 0.17934522032737732, -0.004325420595705509, -0.027260515838861465, -0.15746904909610748, 0.2920587956905365, 0.10150379687547684, 0.27921855449676514, 0.3468317985534668, 0.2332412302494049, 0.016965227201581, -0.02529432810842991, 0.09169992059469223, -0.03665655851364136, -0.12456502765417099, 0.15592020750045776, 0.06345918029546738, -0.03025202266871929, 0.011918516829609871, 0.14057765901088715, -0.0870533436536789, 0.03265158459544182, -0.004262031055986881, 0.10487637668848038, -0.031096341088414192, -0.03632301092147827, -0.007921137847006321, 0.09567331522703171, 0.09852716326713562, 0.028825316578149796, -0.14790326356887817, -0.09557581692934036, -0.03719637170433998, -0.059132400900125504, -0.15941011905670166, 0.058650240302085876, 0.05947893485426903, 0.30133211612701416, 0.020923253148794174, 0.030720166862010956, 0.16891704499721527, -0.04065362364053726 ] } ] } ] ================================================ FILE: public/global.css ================================================ :root { --outer-shadow: 0 .5rem 1rem rgba(0,0,0,.15); --outer-shadow-sm: 0 .125rem .25rem rgba(0,0,0,.075); --outer-shadow-lg: 0 1rem 3rem rgba(0,0,0,.175); --red: rgb(255, 59, 48); --blue: rgb(0, 122, 255); --green: rgb(52, 199, 89); --teal: rgb(90, 200, 250); --light-gray: rgb(250, 250, 250); --middle-gray: rgb(190, 190, 190); --dark-gray: rgb(155, 155, 155); --deep-gray: rgb(100, 100, 100); } html, body { position: relative; width: 100%; height: 100%; scroll-behavior: smooth; } body { color: #333; margin: 0; padding: 0; box-sizing: border-box; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen-Sans, Ubuntu, Cantarell, "Helvetica Neue", sans-serif; } ================================================ FILE: public/index.html ================================================ CNN Explainer ================================================ FILE: rollup.config.js ================================================ import svelte from 'rollup-plugin-svelte'; import resolve from 'rollup-plugin-node-resolve'; import commonjs from 'rollup-plugin-commonjs'; import livereload from 'rollup-plugin-livereload'; import { terser } from 'rollup-plugin-terser'; import rollup_start_dev from './rollup_start_dev'; import replace from '@rollup/plugin-replace'; const production = !process.env.ROLLUP_WATCH; export default { input: 'src/main.js', output: { sourcemap: true, format: 'iife', name: 'app', file: 'public/bundle.js' }, plugins: [ svelte({ // enable run-time checks when not in production dev: !production, // we'll extract any component CSS out into // a separate file — better for performance css: css => { css.write('bundle.css'); } }), replace({PUBLIC_URL: production ? '/cnn-explainer' : ''}), // If you have external dependencies installed from // npm, you'll most likely need these plugins. In // some cases you'll need additional configuration — // consult the documentation for details: // https://github.com/rollup/rollup-plugin-commonjs resolve({ browser: true, dedupe: importee => importee === 'svelte' || importee.startsWith('svelte/') }), commonjs(), // In dev mode, call `npm run start:dev` once // the bundle has been generated !production && rollup_start_dev, // Watch the `public` directory and refresh the // browser on changes when not in production !production && livereload('public'), // If we're building for production (npm run build // instead of npm run dev), minify production && terser() ], watch: { clearScreen: false } }; ================================================ FILE: rollup_start_dev.js ================================================ import * as child_process from 'child_process'; let running_dev_server = false; export default { writeBundle() { if (!running_dev_server) { running_dev_server = true; child_process.spawn('npm', ['run', 'start:dev'], { stdio: ['ignore', 'inherit', 'inherit'], shell: true }); } } }; ================================================ FILE: src/App.svelte ================================================
================================================ FILE: src/Explainer.svelte ================================================
================================================ FILE: src/Header.svelte ================================================ ================================================ FILE: src/article/Article.svelte ================================================

What is a Convolutional Neural Network?

In machine learning, a classifier assigns a class label to a data point. For example, an image classifier produces a class label (e.g, bird, plane) for what objects exist within an image. A convolutional neural network, or CNN for short, is a type of classifier, which excels at solving this problem!

A CNN is a neural network: an algorithm used to recognize patterns in data. Neural Networks in general are composed of a collection of neurons that are organized in layers, each with their own learnable weights and biases. Let’s break down a CNN into its basic building blocks.

  1. A tensor can be thought of as an n-dimensional matrix. In the CNN above, tensors will be 3-dimensional with the exception of the output layer.
  2. A neuron can be thought of as a function that takes in multiple inputs and yields a single output. The outputs of neurons are represented above as the red → blue activation maps.
  3. A layer is simply a collection of neurons with the same operation, including the same hyperparameters.
  4. Kernel weights and biases, while unique to each neuron, are tuned during the training phase, and allow the classifier to adapt to the problem and dataset provided. They are encoded in the visualization with a yellow → green diverging colorscale. The specific values can be viewed in the Interactive Formula View by clicking a neuron or by hovering over the kernel/bias in the Convolutional Elastic Explanation View.
  5. A CNN conveys a differentiable score function, which is represented as class scores in the visualization on the output layer.

If you have studied neural networks before, these terms may sound familiar to you. So what makes a CNN different? CNNs utilize a special type of layer, aptly named a convolutional layer, that makes them well-positioned to learn from image and image-like data. Regarding image data, CNNs can be used for many different computer vision tasks, such as image processing, classification, segmentation, and object detection.

In CNN Explainer, you can see how a simple CNN can be used for image classification. Because of the network’s simplicity, its performance isn’t perfect, but that’s okay! The network architecture, Tiny VGG, used in CNN Explainer contains many of the same layers and operations used in state-of-the-art CNNs today, but on a smaller scale. This way, it will be easier to understand getting started.

What does each layer of the network do?

Let’s walk through each layer in the network. Feel free to interact with the visualization above by clicking and hovering over various parts of it as you read.

Input Layer

The input layer (leftmost layer) represents the input image into the CNN. Because we use RGB images as input, the input layer has three channels, corresponding to the red, green, and blue channels, respectively, which are shown in this layer. Use the color scale when you click on the network details icon icon above to display detailed information (on this layer, and others).

Convolutional Layers

The convolutional layers are the foundation of CNN, as they contain the learned kernels (weights), which extract features that distinguish different images from one another—this is what we want for classification! As you interact with the convolutional layer, you will notice links between the previous layers and the convolutional layers. Each link represents a unique kernel, which is used for the convolution operation to produce the current convolutional neuron’s output or activation map.

The convolutional neuron performs an elementwise dot product with a unique kernel and the output of the previous layer’s corresponding neuron. This will yield as many intermediate results as there are unique kernels. The convolutional neuron is the result of all of the intermediate results summed together with the learned bias.

For example, let’s look at the first convolutional layer in the Tiny VGG architecture above. Notice that there are 10 neurons in this layer, but only 3 neurons in the previous layer. In the Tiny VGG architecture, convolutional layers are fully-connected, meaning each neuron is connected to every other neuron in the previous layer. Focusing on the output of the topmost convolutional neuron from the first convolutional layer, we see that there are 3 unique kernels when we hover over the activation map.

clicking on topmost first conv. layer activation map
Figure 1. As you hover over the activation map of the topmost node from the first convolutional layer, you can see that 3 kernels were applied to yield this activation map. After clicking this activation map, you can see the convolution operation occuring with each unique kernel.

The size of these kernels is a hyper-parameter specified by the designers of the network architecture. In order to produce the output of the convolutional neuron (activation map), we must perform an elementwise dot product with the output of the previous layer and the unique kernel learned by the network. In TinyVGG, the dot product operation uses a stride of 1, which means that the kernel is shifted over 1 pixel per dot product, but this is a hyperparameter that the network architecture designer can adjust to better fit their dataset. We must do this for all 3 kernels, which will yield 3 intermediate results.

clicking on topmost first conv. layer activation map
Figure 2. The kernel being applied to yield the topmost intermediate result for the discussed activation map.

Then, an elementwise sum is performed containing all 3 intermediate results along with the bias the network has learned. After this, the resulting 2-dimensional tensor will be the activation map viewable on the interface above for the topmost neuron in the first convolutional layer. This same operation must be applied to produce each neuron’s activation map.

With some simple math, we are able to deduce that there are 3 x 10 = 30 unique kernels, each of size 3x3, applied in the first convolutional layer. The connectivity between the convolutional layer and the previous layer is a design decision when building a network architecture, which will affect the number of kernels per convolutional layer. Click around the visualization to better understand the operations behind the convolutional layer. See if you can follow the example above!

Understanding Hyperparameters

  1. Padding is often necessary when the kernel extends beyond the activation map. Padding conserves data at the borders of activation maps, which leads to better performance, and it can help preserve the input's spatial size, which allows an architecture designer to build deeper, higher performing networks. There exist many padding techniques, but the most commonly used approach is zero-padding because of its performance, simplicity, and computational efficiency. The technique involves adding zeros symmetrically around the edges of an input. This approach is adopted by many high-performing CNNs such as AlexNet.
  2. Kernel size, often also referred to as filter size, refers to the dimensions of the sliding window over the input. Choosing this hyperparameter has a massive impact on the image classification task. For example, small kernel sizes are able to extract a much larger amount of information containing highly local features from the input. As you can see on the visualization above, a smaller kernel size also leads to a smaller reduction in layer dimensions, which allows for a deeper architecture. Conversely, a large kernel size extracts less information, which leads to a faster reduction in layer dimensions, often leading to worse performance. Large kernels are better suited to extract features that are larger. At the end of the day, choosing an appropriate kernel size will be dependent on your task and dataset, but generally, smaller kernel sizes lead to better performance for the image classification task because an architecture designer is able to stack more and more layers together to learn more and more complex features!
  3. Stride indicates how many pixels the kernel should be shifted over at a time. For example, as described in the convolutional layer example above, Tiny VGG uses a stride of 1 for its convolutional layers, which means that the dot product is performed on a 3x3 window of the input to yield an output value, then is shifted to the right by one pixel for every subsequent operation. The impact stride has on a CNN is similar to kernel size. As stride is decreased, more features are learned because more data is extracted, which also leads to larger output layers. On the contrary, as stride is increased, this leads to more limited feature extraction and smaller output layer dimensions. One responsibility of the architecture designer is to ensure that the kernel slides across the input symmetrically when implementing a CNN. Use the hyperparameter visualization above to alter stride on various input/kernel dimensions to understand this constraint!

Activation Functions

ReLU

Neural networks are extremely prevalent in modern technology—because they are so accurate! The highest performing CNNs today consist of an absurd amount of layers, which are able to learn more and more features. Part of the reason these groundbreaking CNNs are able to achieve such tremendous accuracies is because of their non-linearity. ReLU applies much-needed non-linearity into the model. Non-linearity is necessary to produce non-linear decision boundaries, so that the output cannot be written as a linear combination of the inputs. If a non-linear activation function was not present, deep CNN architectures would devolve into a single, equivalent convolutional layer, which would not perform nearly as well. The ReLU activation function is specifically used as a non-linear activation function, as opposed to other non-linear functions such as Sigmoid because it has been empirically observed that CNNs using ReLU are faster to train than their counterparts.

The ReLU activation function is an elementwise mathematical operation: {reluEquation}

relu graph
Figure 3. The ReLU activation function graphed, which disregards all negative data.

This activation function is applied elementwise on every value from the input tensor. For example, if applied ReLU on the value 2.24, the result would be 2.24, since 2.24 is larger than 0. You can observe how this activation function is applied by clicking a ReLU neuron in the network above. The Rectified Linear Activation function (ReLU) is performed after every convolutional layer in the network architecture outlined above. Notice the impact this layer has on the activation map of various neurons throughout the network!

Softmax

{softmaxEquation} A softmax operation serves a key purpose: making sure the CNN outputs sum to 1. Because of this, softmax operations are useful to scale model outputs into probabilities. Clicking on the last layer reveals the softmax operation in the network. Notice how the logits after flatten aren’t scaled between zero to one. For a visual indication of the impact of each logit (unscaled scalar value), they are encoded using a light orange → dark orange color scale. After passing through the softmax function, each class now corresponds to an appropriate probability!

You might be thinking what the difference between standard normalization and softmax is—after all, both rescale the logits between 0 and 1. Remember that backpropagation is a key aspect of training neural networks—we want the correct answer to have the largest “signal.” By using softmax, we are effectively “approximating” argmax while gaining differentiability. Rescaling doesn’t weigh the max significantly higher than other logits, whereas softmax does. Simply put, softmax is a “softer” argmax—see what we did there?

softmax interactive formula view
Figure 4. The Softmax Interactive Formula View allows a user to interact with both the color encoded logits and formula to understand how the prediction scores after the flatten layer are normalized to yield classification scores.

Pooling Layers

There are many types of pooling layers in different CNN architectures, but they all have the purpose of gradually decreasing the spatial extent of the network, which reduces the parameters and overall computation of the network. The type of pooling used in the Tiny VGG architecture above is Max-Pooling.

The Max-Pooling operation requires selecting a kernel size and a stride length during architecture design. Once selected, the operation slides the kernel with the specified stride over the input while only selecting the largest value at each kernel slice from the input to yield a value for the output. This process can be viewed by clicking a pooling neuron in the network above.

In the Tiny VGG architecture above, the pooling layers use a 2x2 kernel and a stride of 2. This operation with these specifications results in the discarding of 75% of activations. By discarding so many values, Tiny VGG is more computationally efficient and avoids overfitting.

Flatten Layer

This layer converts a three-dimensional layer in the network into a one-dimensional vector to fit the input of a fully-connected layer for classification. For example, a 5x5x2 tensor would be converted into a vector of size 50. The previous convolutional layers of the network extracted the features from the input image, but now it is time to classify the features. We use the softmax function to classify these features, which requires a 1-dimensional input. This is why the flatten layer is necessary. This layer can be viewed by clicking any output class.

Interactive features

  1. Upload your own image by selecting upload image icon to understand how your image is classified into the 10 classes. By analyzing the neurons throughout the network, you can understand the activations maps and extracted features.
  2. Change the activation map colorscale to better understand the impact of activations at different levels of abstraction by adjusting heatmap.
  3. Understand network details such as layer dimensions and colorscales by clicking the network details icon icon.
  4. Simulate network operations by clicking the play icon button or interact with the layer slice in the Interactive Formula View by hovering over portions of the input or output to understand the mappings and underlying operations.
  5. Learn layer functions by clicking info icon from the Interactive Formula View to read layer details from the article.

Video Tutorial

How is CNN Explainer implemented?

CNN Explainer uses TensorFlow.js, an in-browser GPU-accelerated deep learning library to load the pretrained model for visualization. The entire interactive system is written in Javascript using Svelte as a framework and D3.js for visualizations. You only need a web browser to get started learning CNNs today!

Who developed CNN Explainer?

CNN Explainer was created by Jay Wang, Robert Turko, Omar Shaikh, Haekyu Park, Nilaksh Das, Fred Hohman, Minsuk Kahng, and Polo Chau, which was the result of a research collaboration between Georgia Tech and Oregon State. We thank Anmol Chhabria, Kaan Sancak, Kantwon Rogers, and the Georgia Tech Visualization Lab for their support and constructive feedback. This work was supported in part by NSF grants IIS-1563816, CNS-1704701, NASA NSTRF, DARPA GARD, gifts from Intel, NVIDIA, Google, Amazon.

================================================ FILE: src/article/Youtube.svelte ================================================
================================================ FILE: src/config.js ================================================ /* global d3 */ const layerColorScales = { input: [d3.interpolateGreys, d3.interpolateGreys, d3.interpolateGreys], conv: d3.interpolateRdBu, relu: d3.interpolateRdBu, pool: d3.interpolateRdBu, fc: d3.interpolateGreys, weight: d3.interpolateBrBG, logit: d3.interpolateOranges }; let nodeLength = 40; export const overviewConfig = { nodeLength : nodeLength, plusSymbolRadius : nodeLength / 5, numLayers : 12, edgeOpacity : 0.8, edgeInitColor : 'rgb(230, 230, 230)', edgeHoverColor : 'rgb(130, 130, 130)', edgeHoverOuting : false, edgeStrokeWidth : 0.7, intermediateColor : 'gray', layerColorScales: layerColorScales, svgPaddings: {top: 25, bottom: 25, left: 50, right: 50}, kernelRectLength: 8/3, gapRatio: 4, overlayRectOffset: 12, classLists: ['lifeboat', 'ladybug', 'pizza', 'bell pepper', 'school bus', 'koala', 'espresso', 'red panda', 'orange', 'sport car'] }; ================================================ FILE: src/detail-view/ActivationAnimator.svelte ================================================
Input ({image.length}, {image[0].length})
max( , ) =
Output ({output.length}, {output[0].length})
================================================ FILE: src/detail-view/Activationview.svelte ================================================ {#if !isExited}
ReLU Activation
{@html isPaused ? '' : ''}
pointer icon
Hover over the matrices to change pixel.
{/if} ================================================ FILE: src/detail-view/ConvolutionAnimator.svelte ================================================
Input ({image.length}, {image[0].length})
Output ({output.length}, {output[0].length})
================================================ FILE: src/detail-view/Convolutionview.svelte ================================================ {#if !isExited}
Convolution
{@html isPaused ? '' : ''}
pointer icon
Hover over the matrices to change kernel position.
{/if} ================================================ FILE: src/detail-view/Dataview.svelte ================================================
================================================ FILE: src/detail-view/DetailviewUtils.js ================================================ import { matrixSlice } from '../utils/cnn.js'; export function array1d(length, f) { return Array.from({length: length}, f ? ((v, i) => f(i)) : undefined); } function array2d(height, width, f) { return Array.from({length: height}, (v, i) => Array.from({length: width}, f ? ((w, j) => f(i, j)) : undefined)); } export function generateOutputMappings(stride, output, kernelLength, padded_input_size, dilation) { const outputMapping = array2d(output.length, output.length, (i, j) => array2d(kernelLength, kernelLength)); for (let h_out = 0; h_out < output.length; h_out++) { for (let w_out = 0; w_out < output.length; w_out++) { for (let h_kern = 0; h_kern < kernelLength; h_kern++) { for (let w_kern = 0; w_kern < kernelLength; w_kern++) { const h_im = h_out * stride + h_kern * dilation; const w_im = w_out * stride + w_kern * dilation; outputMapping[h_out][w_out][h_kern][w_kern] = h_im * padded_input_size + w_im; } } } } return outputMapping; } export function compute_input_multiplies_with_weight(hoverH, hoverW, padded_input_size, weight_dims, outputMappings, kernelLength) { const [h_weight, w_weight] = weight_dims; const input_multiplies_with_weight = array1d(padded_input_size * padded_input_size); for (let h_weight = 0; h_weight < kernelLength; h_weight++) { for (let w_weight = 0; w_weight < kernelLength; w_weight++) { const flat_input = outputMappings[hoverH][hoverW][h_weight][w_weight]; if (typeof flat_input === "undefined") continue; input_multiplies_with_weight[flat_input] = [h_weight, w_weight]; } } return input_multiplies_with_weight; } export function getMatrixSliceFromInputHighlights(matrix, highlights, kernelLength) { var indices = highlights.reduce((total, value, index) => { if (value != undefined) total.push(index); return total; }, []); return matrixSlice(matrix, Math.floor(indices[0] / matrix.length), Math.floor(indices[0] / matrix.length) + kernelLength, indices[0] % matrix.length, indices[0] % matrix.length + kernelLength); } export function getMatrixSliceFromOutputHighlights(matrix, highlights) { var indices = highlights.reduce((total, value, index) => { if (value != false) total.push(index); return total; }, []); return matrixSlice(matrix, Math.floor(indices[0] / matrix.length), Math.floor(indices[0] / matrix.length) + 1, indices[0] % matrix.length, indices[0] % matrix.length + 1); } // Edit these values to change size of low-level conv visualization. export function getVisualizationSizeConstraint(imageLength) { let sizeOfGrid = 150; let maxSizeOfGridCell = 20; return sizeOfGrid / imageLength > maxSizeOfGridCell ? maxSizeOfGridCell : sizeOfGrid / imageLength; } export function getDataRange(image) { let maxRow = image.map(function(row){ return Math.max.apply(Math, row); }); let max = Math.max.apply(null, maxRow); let minRow = image.map(function(row){ return Math.min.apply(Math, row); }); let min = Math.min.apply(null, minRow); let range = { range: 2 * Math.max(Math.abs(min), Math.abs(max)), min: min, max: max }; return range; } export function gridData(image, constraint=getVisualizationSizeConstraint(image.length)) { // Constrain grids based on input image size. var data = new Array(); var xpos = 1; var ypos = 1; var width = constraint; var height = constraint; for (var row = 0; row < image.length; row++) { data.push( new Array() ); for (var column = 0; column < image[0].length; column++) { data[row].push({ text: Math.round(image[row][column] * 100) / 100, row: row, col: column, x: xpos, y: ypos, width: width, height: height }) xpos += width; } xpos = 1; ypos += height; } return data; } ================================================ FILE: src/detail-view/HyperparameterAnimator.svelte ================================================
Input ({image.length - 2 * padding}, {image.length - 2 * padding})
After-padding ({image.length}, {image.length})
Output ({output.length}, {output.length})
 
================================================ FILE: src/detail-view/HyperparameterDataview.svelte ================================================
================================================ FILE: src/detail-view/Hyperparameterview.svelte ================================================
{@html isPaused ? '' : ''}
pointer icon
Hover over the matrices to change kernel position.
================================================ FILE: src/detail-view/KernelMathView.svelte ================================================
================================================ FILE: src/detail-view/PoolAnimator.svelte ================================================
Input ({testImage.length}, {testImage[0].length})
max( ) =
Output ({testOutput.length}, {testOutput[0].length})
================================================ FILE: src/detail-view/Poolview.svelte ================================================ {#if !isExited}
Max Pooling
{@html isPaused ? '' : ''}
pointer icon
Hover over the matrices to change kernel position.
{/if} ================================================ FILE: src/detail-view/Softmaxview.svelte ================================================
Softmax Score for "{outputName}"
pointer icon
Hover over the numbers to highlight logit circles.
================================================ FILE: src/main.js ================================================ import App from './App.svelte'; const app = new App({ target: document.body, props: {} }); export default app; ================================================ FILE: src/overview/Modal.svelte ================================================
================================================ FILE: src/overview/Overview.svelte ================================================
{#each imageOptions as image, i}
{} : imageOptionClicked} class:inactive={selectedImage !== image.file} class:disabled={disableControl} data-imageName={image.file}> image option
{/each}
{} : customImageClicked}> plus button
{#if selectedNode.data && selectedNode.data.type === 'conv' && selectedNodeIndex != -1} {:else if selectedNode.data && selectedNode.data.type === 'relu'} {:else if selectedNode.data && selectedNode.data.type === 'pool'} {:else if softmaxDetailViewInfo.show} {/if}
================================================ FILE: src/overview/draw-utils.js ================================================ import { overviewConfig } from '../config.js'; // Configs const nodeLength = overviewConfig.nodeLength; /** * Compute the [minimum, maximum] of a 1D or 2D array. * @param {[number]} array */ export const getExtent = (array) => { let min = Infinity; let max = -Infinity; // Scalar if (array.length === undefined) { return [array, array]; } // 1D array if (array[0].length === undefined) { for (let i = 0; i < array[0].length; i++) { if (array[i] < min) { min = array[i]; } else if (array[i] > max) { max = array[i]; } } return [min, max]; } // 2D array for (let i = 0; i < array.length; i++) { for (let j = 0; j < array[0].length; j++) { if (array[i][j] < min) { min = array[i][j]; } else if (array[i][j] > max) { max = array[i][j]; } } } return [min, max]; } /** * Convert the svg element center coord to document absolute value * // Inspired by https://github.com/caged/d3-tip/blob/master/index.js#L286 * @param {elem} elem */ export const getMidCoords = (svg, elem) => { if (svg !== undefined) { let targetel = elem; while (targetel.getScreenCTM == null && targetel.parentNode != null) { targetel = targetel.parentNode; } // Get the absolute coordinate of the E point of element bbox let point = svg.node().ownerSVGElement.createSVGPoint(); let matrix = targetel.getScreenCTM(); let tbbox = targetel.getBBox(); // let width = tbbox.width; let height = tbbox.height; point.x += 0; point.y -= height / 2; let bbox = point.matrixTransform(matrix); return { top: bbox.y, left: bbox.x }; } } /** * Return the output knot (right boundary center) * @param {object} point {x: x, y:y} */ export const getOutputKnot = (point) => { return { x: point.x + nodeLength, y: point.y + nodeLength / 2 }; } /** * Return the output knot (left boundary center) * @param {object} point {x: x, y:y} */ export const getInputKnot = (point) => { return { x: point.x, y: point.y + nodeLength / 2 } } /** * Compute edge data * @param {[[[number, number]]]} nodeCoordinate Constructed neuron svg locations * @param {[object]} cnn Constructed CNN model */ export const getLinkData = (nodeCoordinate, cnn) => { let linkData = []; // Create links backward (starting for the first conv layer) for (let l = 1; l < cnn.length; l++) { for (let n = 0; n < cnn[l].length; n++) { let isOutput = cnn[l][n].layerName === 'output'; let curTarget = getInputKnot(nodeCoordinate[l][n]); for (let p = 0; p < cnn[l][n].inputLinks.length; p++) { // Specially handle output layer (since we are ignoring the flatten) let inputNodeIndex = cnn[l][n].inputLinks[p].source.index; if (isOutput) { let flattenDimension = cnn[l-1][0].output.length * cnn[l-1][0].output.length; if (inputNodeIndex % flattenDimension !== 0){ continue; } inputNodeIndex = Math.floor(inputNodeIndex / flattenDimension); } let curSource = getOutputKnot(nodeCoordinate[l-1][inputNodeIndex]); let curWeight = cnn[l][n].inputLinks[p].weight; linkData.push({ source: curSource, target: curTarget, weight: curWeight, targetLayerIndex: l, targetNodeIndex: n, sourceNodeIndex: inputNodeIndex }); } } } return linkData; } /** * Color scale wrapper (support artificially lighter color!) * @param {function} colorScale D3 color scale function * @param {number} range Color range (max - min) * @param {number} value Color value * @param {number} gap Tail of the color scale to skip */ export const gappedColorScale = (colorScale, range, value, gap) => { if (gap === undefined) { gap = 0; } let normalizedValue = (value + range / 2) / range; return colorScale(normalizedValue * (1 - 2 * gap) + gap); } ================================================ FILE: src/overview/flatten-draw.js ================================================ /* global d3, SmoothScroll */ import { svgStore, vSpaceAroundGapStore, hSpaceAroundGapStore, cnnStore, nodeCoordinateStore, selectedScaleLevelStore, cnnLayerRangesStore, cnnLayerMinMaxStore, isInSoftmaxStore, softmaxDetailViewStore, hoverInfoStore, allowsSoftmaxAnimationStore, detailedModeStore } from '../stores.js'; import { getOutputKnot, getInputKnot, gappedColorScale, getMidCoords } from './draw-utils.js'; import { drawIntermediateLayerLegend, moveLayerX, addOverlayGradient, drawArrow } from './intermediate-utils.js'; import { overviewConfig } from '../config.js'; // Configs const layerColorScales = overviewConfig.layerColorScales; const nodeLength = overviewConfig.nodeLength; const plusSymbolRadius = overviewConfig.plusSymbolRadius; const intermediateColor = overviewConfig.intermediateColor; const kernelRectLength = overviewConfig.kernelRectLength; const svgPaddings = overviewConfig.svgPaddings; const gapRatio = overviewConfig.gapRatio; const classList = overviewConfig.classLists; const formater = d3.format('.4f'); // Shared variables let svg = undefined; svgStore.subscribe( value => {svg = value;} ) let vSpaceAroundGap = undefined; vSpaceAroundGapStore.subscribe( value => {vSpaceAroundGap = value;} ) let hSpaceAroundGap = undefined; hSpaceAroundGapStore.subscribe( value => {hSpaceAroundGap = value;} ) let cnn = undefined; cnnStore.subscribe( value => {cnn = value;} ) let nodeCoordinate = undefined; nodeCoordinateStore.subscribe( value => {nodeCoordinate = value;} ) let selectedScaleLevel = undefined; selectedScaleLevelStore.subscribe( value => {selectedScaleLevel = value;} ) let cnnLayerRanges = undefined; cnnLayerRangesStore.subscribe( value => {cnnLayerRanges = value;} ) let cnnLayerMinMax = undefined; cnnLayerMinMaxStore.subscribe( value => {cnnLayerMinMax = value;} ) let isInSoftmax = undefined; isInSoftmaxStore.subscribe( value => {isInSoftmax = value;} ) let allowsSoftmaxAnimation = undefined; allowsSoftmaxAnimationStore.subscribe( value => {allowsSoftmaxAnimation = value;} ) let softmaxDetailViewInfo = undefined; softmaxDetailViewStore.subscribe( value => {softmaxDetailViewInfo = value;} ) let hoverInfo = undefined; hoverInfoStore.subscribe( value => {hoverInfo = value;} ) let detailedMode = undefined; detailedModeStore.subscribe( value => {detailedMode = value;} ) let layerIndexDict = { 'input': 0, 'conv_1_1': 1, 'relu_1_1': 2, 'conv_1_2': 3, 'relu_1_2': 4, 'max_pool_1': 5, 'conv_2_1': 6, 'relu_2_1': 7, 'conv_2_2': 8, 'relu_2_2': 9, 'max_pool_2': 10, 'output': 11 } let hasInitialized = false; let logits = []; let flattenFactoredFDict = {}; const moveLegend = (d, i, g, moveX, duration, restore) => { let legend = d3.select(g[i]); if (!restore) { let previousTransform = legend.attr('transform'); let previousLegendX = +previousTransform.replace(/.*\(([\d\.]+),.*/, '$1'); let previousLegendY = +previousTransform.replace(/.*,\s([\d\.]+)\)/, '$1'); legend.transition('softmax') .duration(duration) .ease(d3.easeCubicInOut) .attr('transform', `translate(${previousLegendX - moveX}, ${previousLegendY})`); // If not in restore mode, we register the previous location to the DOM element legend.attr('data-preX', previousLegendX); legend.attr('data-preY', previousLegendY); } else { // Restore the recorded location let previousLegendX = +legend.attr('data-preX'); let previousLegendY = +legend.attr('data-preY'); legend.transition('softmax') .duration(duration) .ease(d3.easeCubicInOut) .attr('transform', `translate(${previousLegendX}, ${previousLegendY})`); } } const logitCircleMouseOverHandler = (i) => { // Update the hover info UI hoverInfoStore.set({ show: true, text: `Logit: ${formater(logits[i])}` }) // Highlight the text in the detail view softmaxDetailViewInfo.highlightI = i; softmaxDetailViewStore.set(softmaxDetailViewInfo); let logitLayer = svg.select('.logit-layer'); let logitLayerLower = svg.select('.underneath'); let intermediateLayer = svg.select('.intermediate-layer'); // Highlight the circle logitLayer.select(`#logit-circle-${i}`) .style('stroke-width', 2); // Highlight the associated plus symbol intermediateLayer.select(`#plus-symbol-clone-${i}`) .style('opacity', 1) .select('circle') .style('fill', d => d.fill); // Raise the associated edge group logitLayerLower.select(`#logit-lower-${i}`).raise(); // Highlight the associated edges logitLayerLower.selectAll(`.softmax-abstract-edge-${i}`) .style('stroke-width', 0.8) .style('stroke', '#E0E0E0'); logitLayerLower.selectAll(`.softmax-edge-${i}`) .style('stroke-width', 1) .style('stroke', '#E0E0E0'); logitLayerLower.selectAll(`.logit-output-edge-${i}`) .style('stroke-width', 3) .style('stroke', '#E0E0E0'); logitLayer.selectAll(`.logit-output-edge-${i}`) .style('stroke-width', 3) .style('stroke', '#E0E0E0'); } const logitCircleMouseLeaveHandler = (i) => { // screenshot // return; // Update the hover info UI hoverInfoStore.set({ show: false, text: `Logit: ${formater(logits[i])}` }) // Dehighlight the text in the detail view softmaxDetailViewInfo.highlightI = -1; softmaxDetailViewStore.set(softmaxDetailViewInfo); let logitLayer = svg.select('.logit-layer'); let logitLayerLower = svg.select('.underneath'); let intermediateLayer = svg.select('.intermediate-layer'); // Restore the circle logitLayer.select(`#logit-circle-${i}`) .style('stroke-width', 1); // Restore the associated plus symbol intermediateLayer.select(`#plus-symbol-clone-${i}`) .style('opacity', 0.2); // Restore the associated edges logitLayerLower.selectAll(`.softmax-abstract-edge-${i}`) .style('stroke-width', 0.2) .style('stroke', '#EDEDED'); logitLayerLower.selectAll(`.softmax-edge-${i}`) .style('stroke-width', 0.2) .style('stroke', '#F1F1F1'); logitLayerLower.selectAll(`.logit-output-edge-${i}`) .style('stroke-width', 1.2) .style('stroke', '#E5E5E5'); logitLayer.selectAll(`.logit-output-edge-${i}`) .style('stroke-width', 1.2) .style('stroke', '#E5E5E5'); } // This function is binded to the detail view in Overview.svelte export const softmaxDetailViewMouseOverHandler = (event) => { logitCircleMouseOverHandler(event.detail.curI); } // This function is binded to the detail view in Overview.svelte export const softmaxDetailViewMouseLeaveHandler = (event) => { logitCircleMouseLeaveHandler(event.detail.curI); } const drawLogitLayer = (arg) => { let curLayerIndex = arg.curLayerIndex, moveX = arg.moveX, softmaxLeftMid = arg.softmaxLeftMid, selectedI = arg.selectedI, intermediateX1 = arg.intermediateX1, intermediateX2 = arg.intermediateX2, pixelWidth = arg.pixelWidth, pixelHeight = arg.pixelHeight, topY = arg.topY, bottomY = arg.bottomY, softmaxX = arg.softmaxX, middleGap = arg.middleGap, middleRectHeight = arg.middleRectHeight, symbolGroup = arg.symbolGroup, symbolX = arg.symbolX, flattenRange = arg.flattenRange; let logitLayer = svg.select('.intermediate-layer') .append('g') .attr('class', 'logit-layer') .raise(); // Minotr layer ordering change let tempClone = svg.select('.intermediate-layer') .select('.flatten-layer') .select('.plus-symbol') .clone(true) .attr('class', 'temp-clone-plus-symbol') .attr('transform', `translate(${symbolX - moveX}, ${nodeCoordinate[curLayerIndex][selectedI].y + nodeLength / 2})`) // Cool hack -> d3 clone doesnt clone events, make the front object pointer // event transparent so users can trigger the underlying object's event! .style('pointer-events', 'none') .remove(); let tempPlusSymbol = logitLayer.append(() => tempClone.node()); svg.select('.softmax-symbol').raise(); let logitLayerLower = svg.select('.underneath') .append('g') .attr('class', 'logit-layer-lower') .lower(); // Use circles to encode logit values let centerX = softmaxLeftMid - moveX * 4 / 5; // Get all logits logits = []; for (let i = 0; i < cnn[layerIndexDict['output']].length; i++) { logits.push(cnn[layerIndexDict['output']][i].logit); } // Construct a color scale for the logit values let logitColorScale = d3.scaleLinear() .domain(d3.extent(logits)) .range([0.2, 1]); // Draw the current logit circle before animation let logitRadius = 8; logitLayer.append('circle') .attr('class', 'logit-circle') .attr('id', `logit-circle-${selectedI}`) .attr('cx', centerX) .attr('cy', nodeCoordinate[curLayerIndex - 1][selectedI].y + nodeLength / 2) .attr('r', logitRadius) .style('fill', layerColorScales.logit(logitColorScale(logits[selectedI]))) .style('cursor', 'crosshair') .style('pointer-events', 'all') .style('stroke', intermediateColor) .on('mouseover', () => logitCircleMouseOverHandler(selectedI)) .on('mouseleave', () => logitCircleMouseLeaveHandler(selectedI)) .on('click', () => { d3.event.stopPropagation() }); // Show the logit circle corresponding label let softmaxDetailAnnotation = svg.select('.intermediate-layer-annotation') .select('.softmax-detail-annoataion'); softmaxDetailAnnotation.select(`#logit-text-${selectedI}`) .style('opacity', 1); tempPlusSymbol.raise(); // Draw another line from plus symbol to softmax symbol logitLayer.append('line') .attr('class', `logit-output-edge-${selectedI}`) .attr('x1', intermediateX2 - moveX + plusSymbolRadius * 2) .attr('x2', softmaxX) .attr('y1', nodeCoordinate[curLayerIndex - 1][selectedI].y + nodeLength / 2) .attr('y2', nodeCoordinate[curLayerIndex - 1][selectedI].y + nodeLength / 2) .style('fill', 'none') .style('stroke', '#EAEAEA') .style('stroke-width', '1.2') .lower(); // Add the flatten to logit links let linkData = []; let flattenLength = cnn.flatten.length / cnn[1].length; let underneathIs = [...Array(cnn[layerIndexDict['output']].length).keys()] .filter(d => d != selectedI); let curIIndex = 0; let linkGen = d3.linkHorizontal() .x(d => d.x) .y(d => d.y); const drawOneEdgeGroup = () => { // Only draw the new group if it is in the softmax mode if (!allowsSoftmaxAnimation) { svg.select('.underneath') .selectAll(`.logit-lower`) .remove(); return; } let curI = underneathIs[curIIndex]; let curEdgeGroup = svg.select('.underneath') .select(`#logit-lower-${curI}`); if (curEdgeGroup.empty()) { curEdgeGroup = svg.select('.underneath') .append('g') .attr('class', 'logit-lower') .attr('id', `logit-lower-${curI}`) .style('opacity', 0); // Hack: now show all edges, only draw 1/3 of the actual edges for (let f = 0; f < flattenLength; f += 3) { let loopFactors = [0, 9]; loopFactors.forEach(l => { let factoredF = f + l * flattenLength; // Flatten -> output linkData.push({ source: {x: intermediateX1 + pixelWidth + 3 - moveX, y: l === 0 ? topY + f * pixelHeight : bottomY + f * pixelHeight}, target: {x: intermediateX2 - moveX, y: nodeCoordinate[curLayerIndex][curI].y + nodeLength / 2}, index: factoredF, weight: cnn.flatten[factoredF].outputLinks[curI].weight, color: '#F1F1F1', width: 0.5, opacity: 1, class: `softmax-edge-${curI}` }); }); } // Draw middle rect to logits for (let vi = 0; vi < cnn[layerIndexDict['output']].length - 2; vi++) { linkData.push({ source: {x: intermediateX1 + pixelWidth + 3 - moveX, y: topY + flattenLength * pixelHeight + middleGap * (vi + 1) + middleRectHeight * (vi + 0.5)}, target: {x: intermediateX2 - moveX, y: nodeCoordinate[curLayerIndex][curI].y + nodeLength / 2}, index: -1, color: '#EDEDED', width: 0.5, opacity: 1, class: `softmax-abstract-edge-${curI}` }); } // Render the edges on the underneath layer curEdgeGroup.selectAll(`path.softmax-edge-${curI}`) .data(linkData) .enter() .append('path') .attr('class', d => d.class) .attr('id', d => `edge-${d.name}`) .attr('d', d => linkGen({source: d.source, target: d.target})) .style('fill', 'none') .style('stroke-width', d => d.width) .style('stroke', d => d.color === undefined ? intermediateColor : d.color) .style('opacity', d => d.opacity) .style('pointer-events', 'none'); } let curNodeGroup = logitLayer.append('g') .attr('class', `logit-layer-${curI}`) .style('opacity', 0); // Draw the plus symbol let symbolClone = symbolGroup.clone(true) .style('opacity', 0); // Change the style of the clone symbolClone.attr('class', 'plus-symbol-clone') .attr('id', `plus-symbol-clone-${curI}`) .select('circle') .datum({fill: gappedColorScale(layerColorScales.weight, flattenRange, cnn[layerIndexDict['output']][curI].bias, 0.35)}) .style('pointer-events', 'none') .style('fill', '#E5E5E5'); symbolClone.attr('transform', `translate(${symbolX}, ${nodeCoordinate[curLayerIndex][curI].y + nodeLength / 2})`); // Draw the outter link using only merged path let outputEdgeD1 = linkGen({ source: { x: intermediateX2 - moveX + plusSymbolRadius * 2, y: nodeCoordinate[curLayerIndex][curI].y + nodeLength / 2 }, target: { x: centerX + logitRadius, y: nodeCoordinate[curLayerIndex][curI].y + nodeLength / 2 } }); let outputEdgeD2 = linkGen({ source: { x: centerX + logitRadius, y: nodeCoordinate[curLayerIndex][curI].y + nodeLength / 2 }, target: { x: softmaxX, y: nodeCoordinate[curLayerIndex][selectedI].y + nodeLength / 2 } }); // There are ways to combine these two paths into one. However, the animation // for merged path is not continuous, so we use two saperate paths here. let outputEdge1 = logitLayerLower.append('path') .attr('class', `logit-output-edge-${curI}`) .attr('d', outputEdgeD1) .style('fill', 'none') .style('stroke', '#EAEAEA') .style('stroke-width', '1.2'); let outputEdge2 = logitLayerLower.append('path') .attr('class', `logit-output-edge-${curI}`) .attr('d', outputEdgeD2) .style('fill', 'none') .style('stroke', '#EAEAEA') .style('stroke-width', '1.2'); let outputEdgeLength1 = outputEdge1.node().getTotalLength(); let outputEdgeLength2 = outputEdge2.node().getTotalLength(); let totalLength = outputEdgeLength1 + outputEdgeLength2; let totalDuration = hasInitialized ? 500 : 800; let opacityDuration = hasInitialized ? 400 : 600; outputEdge1.attr('stroke-dasharray', outputEdgeLength1 + ' ' + outputEdgeLength1) .attr('stroke-dashoffset', outputEdgeLength1); outputEdge2.attr('stroke-dasharray', outputEdgeLength2 + ' ' + outputEdgeLength2) .attr('stroke-dashoffset', outputEdgeLength2); outputEdge1.transition('softmax-output-edge') .duration(outputEdgeLength1 / totalLength * totalDuration) .attr('stroke-dashoffset', 0); outputEdge2.transition('softmax-output-edge') .delay(outputEdgeLength1 / totalLength * totalDuration) .duration(outputEdgeLength2 / totalLength * totalDuration) .attr('stroke-dashoffset', 0); // Draw the logit circle curNodeGroup.append('circle') .attr('class', 'logit-circle') .attr('id', `logit-circle-${curI}`) .attr('cx', centerX) .attr('cy', nodeCoordinate[curLayerIndex - 1][curI].y + nodeLength / 2) .attr('r', 7) .style('fill', layerColorScales.logit(logitColorScale(logits[curI]))) .style('stroke', intermediateColor) .style('cursor', 'crosshair') .on('mouseover', () => logitCircleMouseOverHandler(curI)) .on('mouseleave', () => logitCircleMouseLeaveHandler(curI)) .on('click', () => { d3.event.stopPropagation() }); // Show the element in the detailed view softmaxDetailViewInfo.startAnimation = { i: curI, duration: opacityDuration, // Always show the animation hasInitialized: false }; softmaxDetailViewStore.set(softmaxDetailViewInfo); // Show the elements with animation curNodeGroup.transition('softmax-edge') .duration(opacityDuration) .style('opacity', 1); if ((selectedI < 3 && curI == 9) || (selectedI >= 3 && curI == 0)) { // Show the hover text softmaxDetailAnnotation.select('.softmax-detail-hover-annotation') .transition('softmax-edge') .duration(opacityDuration) .style('opacity', 1); } softmaxDetailAnnotation.select(`#logit-text-${curI}`) .transition('softmax-edge') .duration(opacityDuration) .style('opacity', 1); curEdgeGroup.transition('softmax-edge') .duration(opacityDuration) .style('opacity', 1) .on('end', () => { // Recursive animaiton curIIndex ++; if (curIIndex < underneathIs.length) { linkData = []; drawOneEdgeGroup(); } else { hasInitialized = true; softmaxDetailViewInfo.hasInitialized = true; softmaxDetailViewStore.set(softmaxDetailViewInfo); } }); symbolClone.transition('softmax-edge') .duration(opacityDuration) .style('opacity', 0.2); } // Show the softmax detail view let anchorElement = svg.select('.intermediate-layer') .select('.layer-label').node(); let pos = getMidCoords(svg, anchorElement); let wholeSvg = d3.select('#cnn-svg'); let svgYMid = +wholeSvg.style('height').replace('px', '') / 2; let detailViewTop = 100 + svgYMid - 192 / 2; const detailview = document.getElementById('detailview'); detailview.style.top = `${detailViewTop}px`; detailview.style.left = `${pos.left - 490 - 50}px`; detailview.style.position = 'absolute'; softmaxDetailViewStore.set({ show: true, logits: logits, logitColors: logits.map(d => layerColorScales.logit(logitColorScale(d))), selectedI: selectedI, highlightI: -1, outputName: classList[selectedI], outputValue: cnn[layerIndexDict['output']][selectedI].output, startAnimation: {i: -1, duration: 0, hasInitialized: hasInitialized} }) drawOneEdgeGroup(); // Draw logit circle color scale drawIntermediateLayerLegend({ legendHeight: 5, curLayerIndex: curLayerIndex, range: d3.extent(logits)[1] - d3.extent(logits)[0], minMax: {min: d3.extent(logits)[0], max: d3.extent(logits)[1]}, group: logitLayer, width: softmaxX - (intermediateX2 + plusSymbolRadius * 2 - moveX + 5), gradientAppendingName: 'flatten-logit-gradient', gradientGap: 0.1, colorScale: layerColorScales.logit, x: intermediateX2 + plusSymbolRadius * 2 - moveX + 5, y: svgPaddings.top + vSpaceAroundGap * (10) + vSpaceAroundGap + nodeLength * 10 }); // Draw logit layer label let logitLabel = logitLayer.append('g') .attr('class', 'layer-label') .classed('hidden', detailedMode) .attr('transform', () => { let x = centerX; let y = (svgPaddings.top + vSpaceAroundGap) / 2 + 5; return `translate(${x}, ${y})`; }); logitLabel.append('text') .style('text-anchor', 'middle') .style('dominant-baseline', 'middle') .style('opacity', 0.8) .style('font-weight', 800) .text('logit'); } const removeLogitLayer = () => { svg.select('.logit-layer').remove(); svg.select('.logit-layer-lower').remove(); svg.selectAll('.plus-symbol-clone').remove(); // Instead of removing the paths, we hide them, so it is faster to load in // the future svg.select('.underneath') .selectAll('.logit-lower') .style('opacity', 0); softmaxDetailViewStore.set({ show: false, logits: [] }) } const softmaxClicked = (arg) => { let curLayerIndex = arg.curLayerIndex, moveX = arg.moveX, symbolX = arg.symbolX, symbolY = arg.symbolY, outputX = arg.outputX, outputY = arg.outputY, softmaxLeftMid = arg.softmaxLeftMid, selectedI = arg.selectedI, intermediateX1 = arg.intermediateX1, intermediateX2 = arg.intermediateX2, pixelWidth = arg.pixelWidth, pixelHeight = arg.pixelHeight, topY = arg.topY, bottomY = arg.bottomY, middleGap = arg.middleGap, middleRectHeight = arg.middleRectHeight, softmaxX = arg.softmaxX, softmaxTextY = arg.softmaxTextY, softmaxWidth = arg.softmaxWidth, symbolGroup = arg.symbolGroup, flattenRange = arg.flattenRange; let duration = 600; let centerX = softmaxLeftMid - moveX * 4 / 5; d3.event.stopPropagation(); // Clean up the logit elemends before moving anything if (isInSoftmax) { allowsSoftmaxAnimationStore.set(false); removeLogitLayer(); } else { allowsSoftmaxAnimationStore.set(true); } // Move the overlay gradient svg.select('.intermediate-layer-overlay') .select('rect.overlay') .transition('softmax') .ease(d3.easeCubicInOut) .duration(duration) .attr('transform', `translate(${isInSoftmax ? 0 : -moveX}, ${0})`); // Move the legends svg.selectAll(`.intermediate-legend-${curLayerIndex - 1}`) .each((d, i, g) => moveLegend(d, i, g, moveX, duration, isInSoftmax)); svg.select('.intermediate-layer') .select(`.layer-label`) .each((d, i, g) => moveLegend(d, i, g, moveX, duration, isInSoftmax)); svg.select('.intermediate-layer') .select(`.layer-detailed-label`) .each((d, i, g) => moveLegend(d, i, g, moveX, duration, isInSoftmax)); // Also move all layers on the left for (let i = curLayerIndex - 1; i >= 0; i--) { let curLayer = svg.select(`g#cnn-layer-group-${i}`); let previousX = +curLayer.select('image').attr('x'); let newX = isInSoftmax ? previousX + moveX : previousX - moveX; moveLayerX({ layerIndex: i, targetX: newX, disable: true, delay: 0, transitionName: 'softmax', duration: duration }); } // Hide the sum up annotation svg.select('.plus-annotation') .transition('softmax') .duration(duration) .style('opacity', isInSoftmax ? 1 : 0) .style('pointer-events', isInSoftmax ? 'all' : 'none'); // Hide the softmax annotation let softmaxAnnotation = svg.select('.softmax-annotation') .style('pointer-events', isInSoftmax ? 'all' : 'none'); let softmaxDetailAnnotation = softmaxAnnotation.selectAll('.softmax-detail-annoataion') .data([0]) .enter() .append('g') .attr('class', 'softmax-detail-annoataion'); // Remove the detailed annoatioan when quitting the detail view if (isInSoftmax) { softmaxAnnotation.selectAll('.softmax-detail-annoataion').remove(); } softmaxAnnotation.select('.arrow-group') .transition('softmax') .duration(duration) .style('opacity', isInSoftmax ? 1 : 0); softmaxAnnotation.select('.annotation-text') .style('cursor', 'help') .style('pointer-events', 'all') .on('click', () => { d3.event.stopPropagation(); // Scroll to the article element document.querySelector(`#article-softmax`).scrollIntoView({ behavior: 'smooth' }); }) .transition('softmax') .duration(duration) .style('opacity', isInSoftmax ? 1 : 0) .on('end', () => { if (!isInSoftmax) { // Add new annotation for the softmax button let textX = softmaxX + softmaxWidth / 2; let textY = softmaxTextY - 10; if (selectedI === 0) { textY = softmaxTextY + 70; } let text = softmaxDetailAnnotation.append('text') .attr('x', textX) .attr('y', textY) .attr('class', 'annotation-text softmax-detail-text') .style('dominant-baseline', 'baseline') .style('text-anchor', 'middle') .text('Normalize '); text.append('tspan') .attr('dx', 1) .style('fill', '#E56014') .text('logits'); text.append('tspan') .attr('dx', 1) .text(' into'); text.append('tspan') .attr('x', textX) .attr('dy', '1.1em') .text('class probabilities'); if (selectedI === 0) { drawArrow({ group: softmaxDetailAnnotation, sx: softmaxX + softmaxWidth / 2 - 5, sy: softmaxTextY + 44, tx: softmaxX + softmaxWidth / 2, ty: textY - 12, dr: 50, hFlip: true, marker: 'marker-alt' }); } else { drawArrow({ group: softmaxDetailAnnotation, sx: softmaxX + softmaxWidth / 2 - 5, sy: softmaxTextY + 4, tx: softmaxX + softmaxWidth / 2, ty: symbolY - plusSymbolRadius - 4, dr: 50, hFlip: true, marker: 'marker-alt' }); } // Add annotation for the logit layer label textX = centerX + 45; textY = (svgPaddings.top + vSpaceAroundGap) / 2 + 5; let arrowTX = centerX + 20; let arrowTY = (svgPaddings.top + vSpaceAroundGap) / 2 + 5; softmaxDetailAnnotation.append('g') .attr('class', 'layer-detailed-label') .attr('transform', () => { let x = centerX; let y = (svgPaddings.top + vSpaceAroundGap) / 2 - 5; return `translate(${x}, ${y})`; }) .classed('hidden', !detailedMode) .append('text') // .attr('x', centerX) // .attr('y', (svgPaddings.top + vSpaceAroundGap) / 2 - 6) .style('opacity', 0.7) .style('dominant-baseline', 'middle') .style('font-size', '12px') .style('font-weight', '800') .append('tspan') .attr('x', 0) .text('logit') .append('tspan') .attr('x', 0) .style('font-size', '8px') .style('font-weight', 'normal') .attr('dy', '1.5em') .text('(10)'); softmaxDetailAnnotation.append('text') .attr('class', 'annotation-text') .attr('x', textX) .attr('y', (svgPaddings.top + vSpaceAroundGap) / 2 + 3) .style('text-anchor', 'start') .text('Before') .append('tspan') .attr('x', textX) .attr('dy', '1em') .text('normalization') drawArrow({ group: softmaxDetailAnnotation, tx: arrowTX, ty: arrowTY, sx: textX - 6, sy: textY + 2, dr: 60, hFlip: false, marker: 'marker-alt' }); softmaxDetailAnnotation.append('text') .attr('class', 'annotation-text') .attr('x', nodeCoordinate[layerIndexDict['output']][0].x - 35) .attr('y', (svgPaddings.top + vSpaceAroundGap) / 2 + 3) .style('text-anchor', 'end') .text('After') .append('tspan') .attr('x', nodeCoordinate[layerIndexDict['output']][0].x - 35) .attr('dy', '1em') .text('normalization') drawArrow({ group: softmaxDetailAnnotation, tx: nodeCoordinate[layerIndexDict['output']][0].x - 8, ty: arrowTY, sx: nodeCoordinate[layerIndexDict['output']][0].x - 27, sy: textY + 2, dr: 60, hFlip: true, marker: 'marker-alt' }); // Add annotation for the logit circle for (let i = 0; i < 10; i++) { softmaxDetailAnnotation.append('text') .attr('x', centerX) .attr('y', nodeCoordinate[curLayerIndex - 1][i].y + nodeLength / 2 + 8) .attr('class', 'annotation-text softmax-detail-text') .attr('id', `logit-text-${i}`) .style('text-anchor', 'middle') .style('dominant-baseline', 'hanging') .style('opacity', 0) .text(`${classList[i]}`); } let hoverTextGroup = softmaxDetailAnnotation.append('g') .attr('class', 'softmax-detail-hover-annotation') .style('opacity', 0); textX = centerX + 50; textY = nodeCoordinate[curLayerIndex - 1][0].y + nodeLength / 2; if (selectedI < 3) { textY = nodeCoordinate[curLayerIndex - 1][9].y + nodeLength / 2; } // Add annotation to prompt user to check the logit value let hoverText = hoverTextGroup.append('text') .attr('x', textX) .attr('y', textY) .attr('class', 'annotation-text softmax-detail-text softmax-hover-text') .style('text-anchor', 'start') .style('dominant-baseline', 'baseline') .append('tspan') .style('font-weight', 700) .style('dominant-baseline', 'baseline') .text(`Hover over `) .append('tspan') .style('font-weight', 400) .style('dominant-baseline', 'baseline') .text('to see'); hoverText.append('tspan') .style('dominant-baseline', 'baseline') .attr('x', textX) .attr('dy', '1em') .text('its '); hoverText.append('tspan') .style('dominant-baseline', 'baseline') .attr('dx', 1) .style('fill', '#E56014') .text('logit'); hoverText.append('tspan') .style('dominant-baseline', 'baseline') .attr('dx', 1) .text(' value'); drawArrow({ group: hoverTextGroup, tx: centerX + 15, ty: textY, sx: textX - 8, sy: textY + 2, dr: 60, hFlip: false }); } }) // Hide the annotation svg.select('.flatten-annotation') .transition('softmax') .duration(duration) .style('opacity', isInSoftmax ? 1 : 0) .style('pointer-events', isInSoftmax ? 'all' : 'none'); // Move the left part of faltten layer elements let flattenLeftPart = svg.select('.flatten-layer-left'); flattenLeftPart.transition('softmax') .duration(duration) .ease(d3.easeCubicInOut) .attr('transform', `translate(${isInSoftmax ? 0 : -moveX}, ${0})`) .on('end', () => { // Add the logit layer if (!isInSoftmax) { let logitArg = { curLayerIndex: curLayerIndex, moveX: moveX, softmaxLeftMid: softmaxLeftMid, selectedI: selectedI, intermediateX1: intermediateX1, intermediateX2: intermediateX2, pixelWidth: pixelWidth, pixelHeight: pixelHeight, topY: topY, bottomY: bottomY, middleGap: middleGap, middleRectHeight: middleRectHeight, softmaxX: softmaxX, symbolGroup: symbolGroup, symbolX: symbolX, flattenRange: flattenRange }; drawLogitLayer(logitArg); } // Redraw the line from the plus symbol to the output node if (!isInSoftmax) { let newLine = flattenLeftPart.select('.edge-group') .append('line') .attr('class', 'symbol-output-line') .attr('x1', symbolX) .attr('y1', symbolY) .attr('x2', outputX + moveX) .attr('y2', outputY) .style('stroke-width', 1.2) .style('stroke', '#E5E5E5') .style('opacity', 0); newLine.transition('softmax') .delay(duration / 3) .duration(duration * 2 / 3) .style('opacity', 1); } else { flattenLeftPart.select('.symbol-output-line').remove(); } isInSoftmax = !isInSoftmax; isInSoftmaxStore.set(isInSoftmax); }) } /** * Draw the flatten layer before output layer * @param {number} curLayerIndex Index of the selected layer * @param {object} d Bounded d3 data * @param {number} i Index of the selected node * @param {number} width CNN group width * @param {number} height CNN group height */ export const drawFlatten = (curLayerIndex, d, i, width, height) => { // Show the output legend svg.selectAll('.output-legend') .classed('hidden', false); let pixelWidth = nodeLength / 2; let pixelHeight = 1.1; let totalLength = (2 * nodeLength + 5.5 * hSpaceAroundGap * gapRatio + pixelWidth); let leftX = nodeCoordinate[curLayerIndex][0].x - totalLength; let intermediateGap = (hSpaceAroundGap * gapRatio * 4) / 2; const minimumGap = 20; let linkGen = d3.linkHorizontal() .x(d => d.x) .y(d => d.y); // Hide the edges svg.select('g.edge-group') .style('visibility', 'hidden'); // Move the previous layer moveLayerX({layerIndex: curLayerIndex - 1, targetX: leftX, disable: true, delay: 0}); // Disable the current layer (output layer) moveLayerX({layerIndex: curLayerIndex, targetX: nodeCoordinate[curLayerIndex][0].x, disable: true, delay: 0, opacity: 0.15, specialIndex: i}); // Compute the gap in the left shrink region let leftEnd = leftX - hSpaceAroundGap; let leftGap = (leftEnd - nodeCoordinate[0][0].x - 10 * nodeLength) / 10; // Different from other intermediate view, we push the left part dynamically // 1. If there is enough space, we fix the first layer position and move all // other layers; // 2. If there is not enough space, we maintain the minimum gap and push all // left layers to the left (could be out-of-screen) if (leftGap > minimumGap) { // Move the left layers for (let i = 0; i < curLayerIndex - 1; i++) { let curX = nodeCoordinate[0][0].x + i * (nodeLength + leftGap); moveLayerX({layerIndex: i, targetX: curX, disable: true, delay: 0}); } } else { leftGap = minimumGap; let curLeftBound = leftX - leftGap * 2 - nodeLength; // Move the left layers for (let i = curLayerIndex - 2; i >= 0; i--) { moveLayerX({layerIndex: i, targetX: curLeftBound, disable: true, delay: 0}); curLeftBound = curLeftBound - leftGap - nodeLength; } } // Add an overlay let stops = [{offset: '0%', color: 'rgb(250, 250, 250)', opacity: 1}, {offset: '50%', color: 'rgb(250, 250, 250)', opacity: 0.95}, {offset: '100%', color: 'rgb(250, 250, 250)', opacity: 0.85}]; addOverlayGradient('overlay-gradient-left', stops); let intermediateLayerOverlay = svg.append('g') .attr('class', 'intermediate-layer-overlay'); intermediateLayerOverlay.append('rect') .attr('class', 'overlay') .style('fill', 'url(#overlay-gradient-left)') .style('stroke', 'none') .attr('width', leftX + svgPaddings.left - (leftGap * 2) + 3) .attr('height', height + svgPaddings.top + svgPaddings.bottom) .attr('x', -svgPaddings.left) .attr('y', 0) .style('opacity', 0); intermediateLayerOverlay.selectAll('rect.overlay') .transition('move') .duration(800) .ease(d3.easeCubicInOut) .style('opacity', 1); // Add the intermediate layer let intermediateLayer = svg.append('g') .attr('class', 'intermediate-layer') .style('opacity', 0); let intermediateX1 = leftX + nodeLength + intermediateGap; let intermediateX2 = intermediateX1 + intermediateGap + pixelWidth; let range = cnnLayerRanges[selectedScaleLevel][curLayerIndex - 1]; let colorScale = layerColorScales.conv; let flattenLength = cnn.flatten.length / cnn[1].length; let linkData = []; let flattenLayer = intermediateLayer.append('g') .attr('class', 'flatten-layer'); let flattenLayerLeftPart = flattenLayer.append('g') .attr('class', 'flatten-layer-left'); let topY = nodeCoordinate[curLayerIndex - 1][0].y; let bottomY = nodeCoordinate[curLayerIndex - 1][9].y + nodeLength - flattenLength * pixelHeight; // Compute the pre-layer gap let preLayerDimension = cnn[curLayerIndex - 1][0].output.length; let preLayerGap = nodeLength / (2 * preLayerDimension); // Compute bounding box length let boundingBoxLength = nodeLength / preLayerDimension; // Compute the weight color scale let flattenExtent = d3.extent(cnn.flatten.slice(flattenLength) .map(d => d.outputLinks[i].weight) .concat(cnn.flatten.slice(9 * flattenLength, 10 * flattenLength) .map(d => d.outputLinks[i].weight))); let flattenRange = 2 * (Math.round( Math.max(...flattenExtent.map(Math.abs)) * 1000) / 1000); let flattenMouseOverHandler = (d) => { let index = d.index; // Screenshot // console.log(index); // Update the hover info UI if (d.weight === undefined) { hoverInfo = { show: true, text: `Pixel value: ${formater(flattenFactoredFDict[index])}` }; } else { hoverInfo = { show: true, text: `Weight: ${formater(d.weight)}` }; } hoverInfoStore.set(hoverInfo); flattenLayerLeftPart.select(`#edge-flatten-${index}`) .raise() .style('stroke', intermediateColor) .style('stroke-width', 1); flattenLayerLeftPart.select(`#edge-flatten-${index}-output`) .raise() .style('stroke-width', 1) .style('stroke', da => gappedColorScale(layerColorScales.weight, flattenRange, da.weight, 0.1)); flattenLayerLeftPart.select(`#bounding-${index}`) .raise() .style('opacity', 1); } let flattenMouseLeaveHandler = (d) => { let index = d.index; // screenshot // if (index === 32) {return;} // Update the hover info UI if (d.weight === undefined) { hoverInfo = { show: false, text: `Pixel value: ${formater(flattenFactoredFDict[index])}` }; } else { hoverInfo = { show: false, text: `Weight: ${formater(d.weight)}` }; } hoverInfoStore.set(hoverInfo); flattenLayerLeftPart.select(`#edge-flatten-${index}`) .style('stroke-width', 0.6) .style('stroke', '#E5E5E5') flattenLayerLeftPart.select(`#edge-flatten-${index}-output`) .style('stroke-width', 0.6) .style('stroke', da => gappedColorScale(layerColorScales.weight, flattenRange, da.weight, 0.35)); flattenLayerLeftPart.select(`#bounding-${index}`) .raise() .style('opacity', 0); } flattenFactoredFDict = {}; for (let f = 0; f < flattenLength; f++) { let loopFactors = [0, 9]; loopFactors.forEach(l => { let factoredF = f + l * flattenLength; flattenFactoredFDict[factoredF] = cnn.flatten[factoredF].output; flattenLayerLeftPart.append('rect') .attr('x', intermediateX1) .attr('y', l === 0 ? topY + f * pixelHeight : bottomY + f * pixelHeight) .attr('width', pixelWidth) .attr('height', pixelHeight) .style('cursor', 'crosshair') .style('fill', colorScale((cnn.flatten[factoredF].output + range / 2) / range)) .on('mouseover', () => flattenMouseOverHandler({index: factoredF})) .on('mouseleave', () => flattenMouseLeaveHandler({index: factoredF})) .on('click', () => { d3.event.stopPropagation() }); // Flatten -> output linkData.push({ source: {x: intermediateX1 + pixelWidth + 3, y: l === 0 ? topY + f * pixelHeight : bottomY + f * pixelHeight}, target: {x: intermediateX2, //nodeCoordinate[curLayerIndex][i].x - nodeLength, y: nodeCoordinate[curLayerIndex][i].y + nodeLength / 2}, index: factoredF, weight: cnn.flatten[factoredF].outputLinks[i].weight, name: `flatten-${factoredF}-output`, color: gappedColorScale(layerColorScales.weight, flattenRange, cnn.flatten[factoredF].outputLinks[i].weight, 0.35), width: 0.6, opacity: 1, class: `flatten-output` }); // Pre-layer -> flatten let row = Math.floor(f / preLayerDimension); linkData.push({ target: {x: intermediateX1 - 3, y: l === 0 ? topY + f * pixelHeight : bottomY + f * pixelHeight}, source: {x: leftX + nodeLength + 3, y: nodeCoordinate[curLayerIndex - 1][l].y + (2 * row + 1) * preLayerGap}, index: factoredF, name: `flatten-${factoredF}`, color: '#E5E5E5', // color: gappedColorScale(layerColorScales.conv, // 2 * Math.max(Math.abs(cnnLayerMinMax[10].max), Math.abs(cnnLayerMinMax[10].min)), // cnn.flatten[factoredF].output, 0.2), width: 0.6, opacity: 1, class: `flatten` }); // Add original pixel bounding box let loc = cnn.flatten[factoredF].inputLinks[0].weight; flattenLayerLeftPart.append('rect') .attr('id', `bounding-${factoredF}`) .attr('class', 'flatten-bounding') .attr('x', leftX + loc[1] * boundingBoxLength) .attr('y', nodeCoordinate[curLayerIndex - 1][l].y + loc[0] * boundingBoxLength) .attr('width', boundingBoxLength) .attr('height', boundingBoxLength) .style('fill', 'none') .style('stroke', intermediateColor) .style('stroke-length', '0.5') .style('pointer-events', 'all') .style('cursor', 'crosshair') .style('opacity', 0) .on('mouseover', () => flattenMouseOverHandler({index: factoredF})) .on('mouseleave', () => flattenMouseLeaveHandler({index: factoredF})) .on('click', () => {d3.event.stopPropagation()}); }) } // Use abstract symbol to represent the flatten nodes in between (between // the first and the last nodes) // Compute the average value of input node and weights let meanValues = []; for (let n = 1; n < cnn[curLayerIndex - 1].length - 1; n++) { /* let meanOutput = d3.mean(cnn.flatten.slice(flattenLength * n, flattenLength * (n + 1)).map(d => d.output)); let meanWeight= d3.mean(cnn.flatten.slice(flattenLength * n, flattenLength * (n + 1)).map(d => d.outputLinks[i].weight)); meanValues.push({index: n, output: meanOutput, weight: meanWeight}); */ meanValues.push({index: n}); } // Compute the middle gap let middleGap = 5; let middleRectHeight = (10 * nodeLength + (10 - 1) * vSpaceAroundGap - pixelHeight * flattenLength * 2 - 5 * (8 + 1)) / 8; // Add middle nodes meanValues.forEach((v, vi) => { // Add a small rectangle flattenLayerLeftPart.append('rect') .attr('x', intermediateX1 + pixelWidth / 4) .attr('y', topY + flattenLength * pixelHeight + middleGap * (vi + 1) + middleRectHeight * vi) .attr('width', pixelWidth / 2) .attr('height', middleRectHeight) // .style('fill', colorScale((v.output + range / 2) / range)); .style('fill', '#E5E5E5'); // Add a triangle next to the input node flattenLayerLeftPart.append('polyline') .attr('points', `${leftX + nodeLength + 3} ${nodeCoordinate[curLayerIndex - 1][v.index].y}, ${leftX + nodeLength + 10} ${nodeCoordinate[curLayerIndex - 1][v.index].y + nodeLength / 2}, ${leftX + nodeLength + 3} ${nodeCoordinate[curLayerIndex - 1][v.index].y + nodeLength}`) .style('fill', '#E5E5E5') .style('opacity', 1); // Input -> flatten linkData.push({ source: {x: leftX + nodeLength + 10, y: nodeCoordinate[curLayerIndex - 1][v.index].y + nodeLength / 2}, target: {x: intermediateX1 - 3, y: topY + flattenLength * pixelHeight + middleGap * (vi + 1) + middleRectHeight * (vi + 0.5)}, index: -1, width: 1, opacity: 1, name: `flatten-abstract-${v.index}`, color: '#E5E5E5', class: `flatten-abstract` }); // Flatten -> output linkData.push({ source: {x: intermediateX1 + pixelWidth + 3, y: topY + flattenLength * pixelHeight + middleGap * (vi + 1) + middleRectHeight * (vi + 0.5)}, target: {x: intermediateX2, y: nodeCoordinate[curLayerIndex][i].y + nodeLength / 2}, index: -1, name: `flatten-abstract-${v.index}-output`, // color: gappedColorScale(layerColorScales.weight, flattenRange, // v.weight, 0.35), color: '#E5E5E5', weight: v.weight, width: 1, opacity: 1, class: `flatten-abstract-output` }); }) // Draw the plus operation symbol let symbolX = intermediateX2 + plusSymbolRadius; let symbolY = nodeCoordinate[curLayerIndex][i].y + nodeLength / 2; let symbolRectHeight = 1; let symbolGroup = flattenLayerLeftPart.append('g') .attr('class', 'plus-symbol') .attr('transform', `translate(${symbolX}, ${symbolY})`); symbolGroup.append('rect') .attr('x', -plusSymbolRadius) .attr('y', -plusSymbolRadius) .attr('width', plusSymbolRadius * 2) .attr('height', plusSymbolRadius * 2) .attr('rx', 3) .attr('ry', 3) .style('fill', 'none') .style('stroke', intermediateColor); symbolGroup.append('rect') .attr('x', -(plusSymbolRadius - 3)) .attr('y', -symbolRectHeight / 2) .attr('width', 2 * (plusSymbolRadius - 3)) .attr('height', symbolRectHeight) .style('fill', intermediateColor); symbolGroup.append('rect') .attr('x', -symbolRectHeight / 2) .attr('y', -(plusSymbolRadius - 3)) .attr('width', symbolRectHeight) .attr('height', 2 * (plusSymbolRadius - 3)) .style('fill', intermediateColor); // Place the bias rectangle below the plus sign if user clicks the first // conv node (no need now, since we added annotaiton for softmax to make it // look better aligned) // Add bias symbol to the plus symbol symbolGroup.append('circle') .attr('cx', 0) .attr('cy', -nodeLength / 2 - 0.5 * kernelRectLength) .attr('r', kernelRectLength * 1.5) .style('stroke', intermediateColor) .style('cursor', 'crosshair') .style('fill', gappedColorScale(layerColorScales.weight, flattenRange, d.bias, 0.35)) .on('mouseover', () => { hoverInfoStore.set( {show: true, text: `Bias: ${formater(d.bias)}`} ); }) .on('mouseleave', () => { hoverInfoStore.set( {show: false, text: `Bias: ${formater(d.bias)}`} ); }) .on('click', () => { d3.event.stopPropagation(); }); // Link from bias to the plus symbol symbolGroup.append('path') .attr('d', linkGen({ source: { x: 0, y: 0 }, target: { x: 0, y: -nodeLength / 2 - 0.5 * kernelRectLength } })) .attr('id', 'bias-plus') .attr('stroke-width', 1.2) .attr('stroke', '#E5E5E5') .lower(); // Link from the plus symbol to the output linkData.push({ source: getOutputKnot({x: intermediateX2 + 2 * plusSymbolRadius - nodeLength, y: nodeCoordinate[curLayerIndex][i].y}), target: getInputKnot({x: nodeCoordinate[curLayerIndex][i].x - 3, y: nodeCoordinate[curLayerIndex][i].y}), name: `symbol-output`, width: 1.2, color: '#E5E5E5' }); // Draw softmax operation symbol let softmaxWidth = 55; let emptySpace = ((totalLength - 2 * nodeLength - 2 * intermediateGap) - softmaxWidth) / 2; let symbolEndX = intermediateX2 + plusSymbolRadius * 2; let softmaxX = emptySpace + symbolEndX; let softmaxLeftMid = emptySpace / 2 + symbolEndX; let softmaxTextY = nodeCoordinate[curLayerIndex][i].y - 2 * kernelRectLength - 6; let moveX = (intermediateX2 - (intermediateX1 + pixelWidth + 3)) * 2 / 3; let softmaxArg = { curLayerIndex: curLayerIndex, moveX: moveX, symbolX: symbolX, symbolY: symbolY, outputX: nodeCoordinate[curLayerIndex][i].x, outputY: symbolY, softmaxLeftMid: softmaxLeftMid, selectedI: i, intermediateX1: intermediateX1, intermediateX2: intermediateX2, pixelWidth: pixelWidth, pixelHeight: pixelHeight, topY: topY, bottomY: bottomY, middleGap: middleGap, middleRectHeight: middleRectHeight, softmaxX: softmaxX, softmaxWidth: softmaxWidth, softmaxTextY: softmaxTextY, symbolGroup: symbolGroup, flattenRange: flattenRange }; let softmaxSymbol = intermediateLayer.append('g') .attr('class', 'softmax-symbol') .attr('transform', `translate(${softmaxX}, ${symbolY})`) .style('pointer-event', 'all') .style('cursor', 'pointer') .on('click', () => softmaxClicked(softmaxArg)); softmaxSymbol.append('rect') .attr('x', 0) .attr('y', -plusSymbolRadius) .attr('width', softmaxWidth) .attr('height', plusSymbolRadius * 2) .attr('stroke', intermediateColor) .attr('rx', 2) .attr('ry', 2) .attr('fill', '#FAFAFA'); softmaxSymbol.append('text') .attr('x', 5) .attr('y', 1) .style('dominant-baseline', 'middle') .style('font-size', '12px') .style('opacity', 0.5) .text('softmax'); // Draw the layer label let layerLabel = intermediateLayer.append('g') .attr('class', 'layer-label') .classed('hidden', detailedMode) .attr('transform', () => { let x = leftX + nodeLength + (4 * hSpaceAroundGap * gapRatio + pixelWidth) / 2; let y = (svgPaddings.top + vSpaceAroundGap) / 2 + 5; return `translate(${x}, ${y})`; }) .style('cursor', 'help') .on('click', () => { d3.event.stopPropagation(); // Scroll to the article element document.querySelector(`#article-flatten`).scrollIntoView({ behavior: 'smooth' }); }); layerLabel.append('text') .style('dominant-baseline', 'middle') .style('opacity', 0.8) .style('font-weight', 800) .text('flatten'); let svgHeight = Number(d3.select('#cnn-svg').style('height').replace('px', '')) + 150; let scroll = new SmoothScroll('a[href*="#"]', {offset: -svgHeight}); let detailedLabelGroup = intermediateLayer.append('g') .attr('transform', () => { let x = leftX + nodeLength + (4 * hSpaceAroundGap * gapRatio + pixelWidth) / 2; let y = (svgPaddings.top + vSpaceAroundGap) / 2 - 5; return `translate(${x}, ${y})`; }) .attr('class', 'layer-detailed-label') .classed('hidden', !detailedMode) .style('cursor', 'help') .on('click', () => { d3.event.stopPropagation(); // Scroll to the article element let anchor = document.querySelector(`#article-flatten`); scroll.animateScroll(anchor); }); detailedLabelGroup.append('title') .text('Move to article section'); let detailedLabelText = detailedLabelGroup.append('text') .style('text-anchor', 'middle') .style('dominant-baseline', 'middle') .style('opacity', '0.7') .style('font-weight', 800) .append('tspan') .text('flatten'); let dimension = cnn[layerIndexDict['max_pool_2']].length * cnn[layerIndexDict['max_pool_2']][0].output.length * cnn[layerIndexDict['max_pool_2']][0].output[0].length; detailedLabelText.append('tspan') .attr('x', 0) .attr('dy', '1.5em') .style('font-size', '8px') .style('font-weight', 'normal') .text(`(${dimension})`); // Add edges between nodes let edgeGroup = flattenLayerLeftPart.append('g') .attr('class', 'edge-group') .lower(); edgeGroup.selectAll('path') .data(linkData) .enter() .append('path') .attr('class', d => d.class) .attr('id', d => `edge-${d.name}`) .attr('d', d => linkGen({source: d.source, target: d.target})) .style('fill', 'none') .style('stroke-width', d => d.width) .style('stroke', d => d.color === undefined ? intermediateColor : d.color) .style('opacity', d => d.opacity); edgeGroup.selectAll('path.flatten-abstract-output') .lower(); edgeGroup.selectAll('path.flatten,path.flatten-output') .style('cursor', 'crosshair') .style('pointer-events', 'all') .on('mouseover', flattenMouseOverHandler) .on('mouseleave', flattenMouseLeaveHandler) .on('click', () => { d3.event.stopPropagation() }); // Add legend drawIntermediateLayerLegend({ legendHeight: 5, curLayerIndex: curLayerIndex, range: range, minMax: cnnLayerMinMax[10], group: intermediateLayer, width: intermediateGap + nodeLength - 3, x: leftX, y: svgPaddings.top + vSpaceAroundGap * (10) + vSpaceAroundGap + nodeLength * 10 }); drawIntermediateLayerLegend({ legendHeight: 5, curLayerIndex: curLayerIndex, range: flattenRange, minMax: {min: flattenExtent[0], max: flattenExtent[1]}, group: intermediateLayer, width: intermediateGap - 3 - 5, gradientAppendingName: 'flatten-weight-gradient', gradientGap: 0.1, colorScale: layerColorScales.weight, x: leftX + intermediateGap + nodeLength + pixelWidth + 3, y: svgPaddings.top + vSpaceAroundGap * (10) + vSpaceAroundGap + nodeLength * 10 }); // Add annotation to the intermediate layer let intermediateLayerAnnotation = svg.append('g') .attr('class', 'intermediate-layer-annotation') .style('opacity', 0); // Add annotation for the sum operation let plusAnnotation = intermediateLayerAnnotation.append('g') .attr('class', 'plus-annotation'); // let textX = nodeCoordinate[curLayerIndex][i].x - 50; let textX = intermediateX2; let textY = nodeCoordinate[curLayerIndex][i].y + nodeLength + kernelRectLength * 3; let arrowSY = nodeCoordinate[curLayerIndex][i].y + nodeLength + kernelRectLength * 2; let arrowTY = nodeCoordinate[curLayerIndex][i].y + nodeLength / 2 + plusSymbolRadius; if (i == 9) { textY -= 110; arrowSY -= 70; arrowTY -= 18; } let plusText = plusAnnotation.append('text') .attr('x', textX) .attr('y', textY) .attr('class', 'annotation-text') .style('dominant-baseline', 'hanging') .style('text-anchor', 'middle'); plusText.append('tspan') .style('dominant-baseline', 'hanging') .text('Add up all products'); plusText.append('tspan') .attr('x', textX) .attr('dy', '1em') .style('dominant-baseline', 'hanging') .text('('); plusText.append('tspan') .style('fill', '#66a3c8') .style('dominant-baseline', 'hanging') .text('element'); plusText.append('tspan') .style('dominant-baseline', 'hanging') .text(' × '); plusText.append('tspan') .style('dominant-baseline', 'hanging') .style('fill', '#b58946') .text('weight'); plusText.append('tspan') .style('dominant-baseline', 'hanging') .text(')'); plusText.append('tspan') .attr('x', textX) .attr('dy', '1em') .style('dominant-baseline', 'hanging') .text('and then '); plusText.append('tspan') .style('dominant-baseline', 'hanging') .style('fill', '#479d94') .text('bias'); drawArrow({ group: plusAnnotation, sx: intermediateX2 - 2 * plusSymbolRadius - 3, sy: arrowSY, tx: intermediateX2 - 5, ty: arrowTY, dr: 30, hFlip: i === 9, marker: 'marker-alt' }); // Add annotation for the bias let biasTextY = nodeCoordinate[curLayerIndex][i].y; biasTextY -= 2 * kernelRectLength + 4; flattenLayerLeftPart.append('text') .attr('class', 'annotation-text') .attr('x', intermediateX2 + plusSymbolRadius) .attr('y', biasTextY) .style('text-anchor', 'middle') .style('dominant-baseline', 'baseline') .text('Bias'); // Add annotation for the softmax symbol let softmaxAnnotation = intermediateLayerAnnotation.append('g') .attr('class', 'softmax-annotation'); softmaxAnnotation.append('text') .attr('x', softmaxX + softmaxWidth / 2) .attr('y', softmaxTextY) .attr('class', 'annotation-text') .style('dominant-baseline', 'baseline') .style('text-anchor', 'middle') .style('font-weight', 700) .text('Click ') .append('tspan') .attr('dx', 1) .style('font-weight', 400) .text('to learn more'); drawArrow({ group: softmaxAnnotation, sx: softmaxX + softmaxWidth / 2 - 5, sy: softmaxTextY + 4, tx: softmaxX + softmaxWidth / 2, ty: symbolY - plusSymbolRadius - 4, dr: 50, hFlip: true }); // Add annotation for the flatten layer let flattenAnnotation = intermediateLayerAnnotation.append('g') .attr('class', 'flatten-annotation'); textX = leftX - 80; textY = nodeCoordinate[curLayerIndex - 1][0].y; let flattenText = flattenAnnotation.append('text') .attr('x', textX) .attr('y', textY) .attr('class', 'annotation-text') .style('dominant-baseline', 'hanging') .style('text-anchor', 'middle'); let tempTspan = flattenText.append('tspan') .style('dominant-baseline', 'hanging') .style('font-weight', 700) .text('Hover over '); tempTspan.append('tspan') .attr('dx', 1) .style('font-weight', 400) .style('dominant-baseline', 'hanging') .text('matrix to'); flattenText.append('tspan') .style('dominant-baseline', 'hanging') .attr('x', textX) .attr('dy', '1em') .text('see how it is flattened'); flattenText.append('tspan') .style('dominant-baseline', 'hanging') .attr('x', textX) .attr('dy', '1em') .text('into a 1D array!'); drawArrow({ group: flattenAnnotation, sx: textX + 45, sy: textY + nodeLength * 0.4 + 12, tx: leftX - 10, ty: textY + nodeLength / 2, dr: 80, hFlip: true }); // Add annotation to explain the middle images textY = nodeCoordinate[curLayerIndex - 1][1].y; let middleText = flattenAnnotation.append('text') .attr('x', textX) .attr('y', textY) .attr('class', 'annotation-text') .style('dominant-baseline', 'hanging') .style('text-anchor', 'middle'); middleText.append('tspan') .style('dominant-baseline', 'hanging') .text('Same flattening'); middleText.append('tspan') .style('dominant-baseline', 'hanging') .attr('x', textX) .attr('dy', '1em') .text('operation for'); middleText.append('tspan') .style('dominant-baseline', 'hanging') .attr('x', textX) .attr('dy', '1em') .text('each neuron'); drawArrow({ group: flattenAnnotation, sx: textX + 39, sy: textY + 25, tx: leftX - 10, ty: textY + nodeLength / 2 - 2, dr: 80, hFlip: true, marker: 'marker-alt' }); // Add annotation for the output neuron let outputAnnotation = intermediateLayerAnnotation.append('g') .attr('class', 'output-annotation'); outputAnnotation.append('text') .attr('x', nodeCoordinate[layerIndexDict['output']][i].x) .attr('y', nodeCoordinate[layerIndexDict['output']][i].y + 10) .attr('class', 'annotation-text') .text(`(${d3.format('.4f')(cnn[layerIndexDict['output']][i].output)})`); /* Prototype of using arc to represent the flatten layer (future) let pie = d3.pie() .padAngle(0) .sort(null) .value(d => d.output) .startAngle(0) .endAngle(-Math.PI); let radius = 490 / 2; let arc = d3.arc() .innerRadius(radius - 20) .outerRadius(radius); let arcs = pie(cnn.flatten); console.log(arcs); let test = svg.append('g') .attr('class', 'test') .attr('transform', 'translate(500, 250)'); test.selectAll("path") .data(arcs) .join("path") .attr('class', 'arc') .attr("fill", d => colorScale((d.value + range/2) / range)) .attr("d", arc); */ // Show everything svg.selectAll('g.intermediate-layer, g.intermediate-layer-annotation') .transition() .delay(500) .duration(500) .ease(d3.easeCubicInOut) .style('opacity', 1); } ================================================ FILE: src/overview/intermediate-draw.js ================================================ /* global d3 */ import { svgStore, vSpaceAroundGapStore, hSpaceAroundGapStore, cnnStore, nodeCoordinateStore, selectedScaleLevelStore, cnnLayerRangesStore, needRedrawStore, cnnLayerMinMaxStore, shouldIntermediateAnimateStore, hoverInfoStore, detailedModeStore, intermediateLayerPositionStore } from '../stores.js'; import { getExtent, getOutputKnot, getInputKnot, gappedColorScale } from './draw-utils.js'; import { drawOutput } from './overview-draw.js'; import { drawIntermediateLayerLegend, moveLayerX, addOverlayGradient, drawArrow } from './intermediate-utils.js'; import { singleConv } from '../utils/cnn.js'; import { overviewConfig } from '../config.js'; // Configs const layerColorScales = overviewConfig.layerColorScales; const nodeLength = overviewConfig.nodeLength; const plusSymbolRadius = overviewConfig.plusSymbolRadius; const numLayers = overviewConfig.numLayers; const intermediateColor = overviewConfig.intermediateColor; const kernelRectLength = overviewConfig.kernelRectLength; const svgPaddings = overviewConfig.svgPaddings; const gapRatio = overviewConfig.gapRatio; const overlayRectOffset = overviewConfig.overlayRectOffset; const formater = d3.format('.4f'); let isEndOfAnimation = false; // Shared variables let svg = undefined; svgStore.subscribe( value => {svg = value;} ) let vSpaceAroundGap = undefined; vSpaceAroundGapStore.subscribe( value => {vSpaceAroundGap = value;} ) let hSpaceAroundGap = undefined; hSpaceAroundGapStore.subscribe( value => {hSpaceAroundGap = value;} ) let cnn = undefined; cnnStore.subscribe( value => {cnn = value;} ) let nodeCoordinate = undefined; nodeCoordinateStore.subscribe( value => {nodeCoordinate = value;} ) let selectedScaleLevel = undefined; selectedScaleLevelStore.subscribe( value => {selectedScaleLevel = value;} ) let cnnLayerRanges = undefined; cnnLayerRangesStore.subscribe( value => {cnnLayerRanges = value;} ) let cnnLayerMinMax = undefined; cnnLayerMinMaxStore.subscribe( value => {cnnLayerMinMax = value;} ) let needRedraw = [undefined, undefined]; needRedrawStore.subscribe( value => {needRedraw = value;} ) let shouldIntermediateAnimate = undefined; shouldIntermediateAnimateStore.subscribe(value => { shouldIntermediateAnimate = value; }) let detailedMode = undefined; detailedModeStore.subscribe( value => {detailedMode = value;} ) let intermediateLayerPosition = undefined; intermediateLayerPositionStore.subscribe ( value => {intermediateLayerPosition = value;} ) // let curRightX = 0; /** * Draw the intermediate layer activation heatmaps * @param {element} image Neuron heatmap image * @param {number} range Colormap range * @param {function} colorScale Colormap * @param {number} length Image length * @param {[[number]]} dataMatrix Heatmap matrix */ const drawIntermidiateImage = (image, range, colorScale, length, dataMatrix) => { // Set up a buffer convas in order to resize image let imageLength = length; let bufferCanvas = document.createElement("canvas"); let bufferContext = bufferCanvas.getContext("2d"); bufferCanvas.width = imageLength; bufferCanvas.height = imageLength; // Fill image pixel array let imageSingle = bufferContext.getImageData(0, 0, imageLength, imageLength); let imageSingleArray = imageSingle.data; for (let i = 0; i < imageSingleArray.length; i+=4) { let pixeIndex = Math.floor(i / 4); let row = Math.floor(pixeIndex / imageLength); let column = pixeIndex % imageLength; let color = d3.rgb(colorScale((dataMatrix[row][column] + range / 2) / range)); imageSingleArray[i] = color.r; imageSingleArray[i + 1] = color.g; imageSingleArray[i + 2] = color.b; imageSingleArray[i + 3] = 255; } // canvas.toDataURL() only exports image in 96 DPI, so we can hack it to have // higher DPI by rescaling the image using canvas magic let largeCanvas = document.createElement('canvas'); largeCanvas.width = nodeLength * 3; largeCanvas.height = nodeLength * 3; let largeCanvasContext = largeCanvas.getContext('2d'); // Use drawImage to resize the original pixel array, and put the new image // (canvas) into corresponding canvas bufferContext.putImageData(imageSingle, 0, 0); largeCanvasContext.drawImage(bufferCanvas, 0, 0, imageLength, imageLength, 0, 0, nodeLength * 3, nodeLength * 3); let imageDataURL = largeCanvas.toDataURL(); image.attr('xlink:href', imageDataURL); // Destory the buffer canvas bufferCanvas.remove(); largeCanvas.remove(); } /** * Create a node group for the intermediate layer * @param {number} curLayerIndex Intermediate layer index * @param {number} selectedI Clicked node index * @param {element} groupLayer Group element * @param {number} x Node's x * @param {number} y Node's y * @param {number} nodeIndex Node's index * @param {function} intermediateNodeMouseOverHandler Mouse over handler * @param {function} intermediateNodeMouseLeaveHandler Mouse leave handler * @param {function} intermediateNodeClicked Mouse click handler * @param {bool} interaction Whether support interaction */ const createIntermediateNode = (curLayerIndex, selectedI, groupLayer, x, y, nodeIndex, stride, intermediateNodeMouseOverHandler, intermediateNodeMouseLeaveHandler, intermediateNodeClicked, interaction) => { let newNode = groupLayer.append('g') .datum(cnn[curLayerIndex - 1][nodeIndex]) .attr('class', 'intermediate-node') .attr('cursor', interaction ? 'pointer': 'default') .attr('pointer-events', interaction ? 'all': 'none') .attr('node-index', nodeIndex) .on('mouseover', intermediateNodeMouseOverHandler) .on('mouseleave', intermediateNodeMouseLeaveHandler) .on('click', (d, g, i) => intermediateNodeClicked(d, g, i, selectedI, curLayerIndex)); newNode.append('image') .attr('width', nodeLength) .attr('height', nodeLength) .attr('x', x) .attr('y', y); // Overlay the image with a mask of many small rectangles let strideTime = Math.floor(nodeLength / stride); let overlayGroup = newNode.append('g') .attr('class', 'overlay-group') .attr('transform', `translate(${x}, ${y})`); for (let i = 0; i < strideTime; i++) { for (let j = 0; j < strideTime; j++) { overlayGroup.append('rect') .attr('class', `mask-overlay mask-${i}-${j}`) .attr('width', stride) .attr('height', stride) .attr('x', i * stride) .attr('y', j * stride) .style('fill', 'var(--light-gray)') .style('stroke', 'var(--light-gray)') .style('opacity', 1); } } // Add a rectangle to show the border newNode.append('rect') .attr('class', 'bounding') .attr('width', nodeLength) .attr('height', nodeLength) .attr('x', x) .attr('y', y) .style('fill', 'none') .style('stroke', intermediateColor) .style('stroke-width', 1); return newNode; } const startOutputAnimation = (kernelGroup, tickTime1D, stride, delay, curLayerIndex) => { const slidingAnimation = () => { let originX = +kernelGroup.attr('data-origin-x'); let originY = +kernelGroup.attr('data-origin-y'); let oldTick = +kernelGroup.attr('data-tick'); let i = (oldTick) % tickTime1D; let j = Math.floor((oldTick) / tickTime1D); let x = originX + i * stride; let y = originY + j * stride; let newTick = (oldTick + 1) % (tickTime1D * tickTime1D); // Remove one mask rect at each tick svg.selectAll(`rect.mask-${i}-${j}`) .transition('window-sliding-mask') .delay(delay + 100) .duration(300) .style('opacity', 0); kernelGroup.attr('data-tick', newTick) .transition('window-sliding-input') .delay(delay) .duration(200) .attr('transform', `translate(${x}, ${y})`) .on('end', () => { if (newTick === 0) { /* Uncomment to wrap the sliding svg.selectAll(`rect.mask-overlay`) .transition('window-sliding-mask') .delay(delay - 200) .duration(300) .style('opacity', 1); */ // Stop the animation // Be careful with animation racing so call this function here instead // of under selectALL if (!isEndOfAnimation) { animationButtonClicked(curLayerIndex); } } if (shouldIntermediateAnimate) { slidingAnimation(); } }); } slidingAnimation(); } const startIntermediateAnimation = (kernelGroupInput, kernelGroupResult, tickTime1D, stride) => { let delay = 200; const slidingAnimation = () => { let originX = +kernelGroupInput.attr('data-origin-x'); let originY = +kernelGroupInput.attr('data-origin-y'); let originXResult = +kernelGroupResult.attr('data-origin-x'); let oldTick = +kernelGroupInput.attr('data-tick'); let i = (oldTick) % tickTime1D; let j = Math.floor((oldTick) / tickTime1D); let x = originX + i * stride; let y = originY + j * stride; let xResult = originXResult + (oldTick % tickTime1D) * stride; let newTick = (oldTick + 1) % (tickTime1D * tickTime1D); // Remove one mask rect at each tick svg.selectAll(`rect.mask-${i}-${j}`) .transition('window-sliding-mask') .delay(delay + 100) .duration(300) .style('opacity', 0); kernelGroupInput.attr('data-tick', newTick) .transition('window-sliding-input') .delay(delay) .duration(200) .attr('transform', `translate(${x}, ${y})`); kernelGroupResult.attr('data-tick', newTick) .transition('window-sliding-result') .delay(delay) .duration(200) .attr('transform', `translate(${xResult}, ${y})`) .on('end', () => { /* Uncomment to wrap the sliding if (newTick === 0) { svg.selectAll(`rect.mask-overlay`) .transition('window-sliding-mask') .delay(delay - 200) .duration(300) .style('opacity', 1); } */ if (shouldIntermediateAnimate) { slidingAnimation(); } }); } slidingAnimation(); } const animationButtonClicked = (curLayerIndex) => { if (d3.event !== null) { d3.event.stopPropagation(); } let delay = 200; let tickTime1D = nodeLength / (kernelRectLength * 3); let stride = kernelRectLength * 3; if (isEndOfAnimation) { // Start the animation shouldIntermediateAnimateStore.set(true); // Show kernel svg.selectAll('.kernel-clone') .transition() .duration(300) .style('opacity', 1) // Restore the mask svg.selectAll(`rect.mask-overlay`) .transition() .duration(300) .style('opacity', 1); // Start the intermediate animation for (let i = 0; i < nodeCoordinate[curLayerIndex - 1].length; i++) { startIntermediateAnimation(d3.select(`.kernel-input-${i}`), d3.select(`.kernel-result-${i}`), tickTime1D, stride); } // Start the output animation startOutputAnimation(d3.select('.kernel-output'), tickTime1D, stride, delay, curLayerIndex); // Change the flow edge style svg.selectAll('path.flow-edge') .attr('stroke-dasharray', '4 2') .attr('stroke-dashoffset', 0) .each((d, i, g) => animateEdge(d, i, g, 0 - 1000)); // Change button icon svg.select('.animation-control-button') .attr('xlink:href', 'PUBLIC_URL/assets/img/fast_forward.svg'); isEndOfAnimation = false; } else { // End the animation shouldIntermediateAnimateStore.set(false); // Show all intermediate and output results svg.selectAll(`rect.mask-overlay`) .transition('skip') .duration(600) .style('opacity', 0); // Move kernel to the beginning to prepare for the next animation let kernelClones = svg.selectAll('.kernel-clone'); kernelClones.attr('data-tick', 0) .transition('skip') .duration(300) .style('opacity', 0) .on('end', (d, i, g) => { let element = d3.select(g[i]); let originX = +element.attr('data-origin-x'); let originY = +element.attr('data-origin-y'); element.attr('transform', `translate(${originX}, ${originY})`); }); // Change flow edge style svg.selectAll('path.flow-edge') .interrupt() .attr('stroke-dasharray', '0 0'); // Change button icon svg.select('.animation-control-button') .attr('xlink:href', 'PUBLIC_URL/assets/img/redo.svg'); isEndOfAnimation = true; } } const animateEdge = (d, i, g, dashoffset) => { let curPath = d3.select(g[i]); curPath.transition() .duration(60000) .ease(d3.easeLinear) .attr('stroke-dashoffset', dashoffset) .on('end', (d, i, g) => { if (shouldIntermediateAnimate) { animateEdge(d, i, g, dashoffset - 2000); } }); } /** * Draw one intermediate layer * @param {number} curLayerIndex * @param {number} leftX X value of intermediate layer left border * @param {number} rightX X value of intermediate layer right border * @param {number} rightStart X value of right component starting anchor * @param {number} intermediateGap The inner gap * @param {number} d Clicked node bounded data * @param {number} i Clicked node index * @param {function} intermediateNodeMouseOverHandler Mouse over handler * @param {function} intermediateNodeMouseLeaveHandler Mouse leave handler * @param {function} intermediateNodeClicked Mouse click handler */ const drawIntermediateLayer = (curLayerIndex, leftX, rightX, rightStart, intermediateGap, d, i, intermediateNodeMouseOverHandler, intermediateNodeMouseLeaveHandler, intermediateNodeClicked) => { // curRightX = rightStart; // Add the intermediate layer let intermediateLayer = svg.append('g') .attr('class', 'intermediate-layer') .style('opacity', 0); // Recovert the animation counter isEndOfAnimation = false; // Tried to add a rectangle to block the intermediate because of webkit's // horrible support (decade old bug) for foreignObject. It doesnt work either. // https://bugs.webkit.org/show_bug.cgi?id=23113 // (1). ForeignObject's inside position is wrong on webkit // (2). 'opacity' of ForeignObject doesn't work on webkit // (3). ForeignObject always show up at the front regardless the svg // stacking order on webkit let intermediateX1 = leftX + nodeLength + intermediateGap; let intermediateX2 = intermediateX1 + nodeLength + intermediateGap * 1.5; let range = cnnLayerRanges[selectedScaleLevel][curLayerIndex]; let colorScale = layerColorScales[d.type]; let intermediateMinMax = []; // Copy the previsious layer to construct foreignObject placeholder // Also add edges from/to the intermediate layer in this loop let linkData = []; // Accumulate the intermediate sum // let itnermediateSumMatrix = init2DArray(d.output.length, // d.output.length, 0); // Compute the min max of all kernel weights in the intermediate layer let kernelExtents = d.inputLinks.map(link => getExtent(link.weight)); let kernelExtent = kernelExtents.reduce((acc, cur) => { return [Math.min(acc[0], cur[0]), Math.max(acc[1], cur[1])]; }) let kernelRange = 2 * (Math.round( Math.max(...kernelExtent.map(Math.abs)) * 1000) / 1000); let kernelColorGap = 0.2; // Compute stride for the kernel animation let stride = kernelRectLength * 3; // Also add the overlay mask on the output node let outputY = nodeCoordinate[curLayerIndex][i].y; let curNode = svg.select(`#layer-${curLayerIndex}-node-${i}`); let outputOverlayGroup = curNode.append('g') .attr('class', 'overlay-group') .attr('transform', `translate(${rightX}, ${outputY})`); let strideTime = Math.floor(nodeLength / stride); for (let i = 0; i < strideTime; i++) { for (let j = 0; j < strideTime; j++) { outputOverlayGroup.append('rect') .attr('class', `mask-overlay mask-${i}-${j}`) .attr('width', stride) .attr('height', stride) .attr('x', i * stride) .attr('y', j * stride) .style('fill', 'var(--light-gray)') .style('stroke', 'var(--light-gray)') .style('opacity', 1); } } // Make sure the bounding box is on top of other things curNode.select('rect.bounding').raise(); // Add sliding kernel for the output node let kernelGroup = intermediateLayer.append('g') .attr('class', `kernel kernel-output kernel-clone`) .attr('transform', `translate(${rightX}, ${outputY})`); kernelGroup.append('rect') .attr('x', 0) .attr('y', 0) .attr('width', kernelRectLength * 3) .attr('height', kernelRectLength * 3) .attr('fill', 'none') .attr('stroke', intermediateColor); kernelGroup.attr('data-tick', 0) .attr('data-origin-x', rightX) .attr('data-origin-y', outputY); let delay = 200; let tickTime1D = nodeLength / (kernelRectLength * 3); startOutputAnimation(kernelGroup, tickTime1D, stride, delay, curLayerIndex); // First intermediate layer nodeCoordinate[curLayerIndex - 1].forEach((n, ni) => { // Compute the intermediate value let inputMatrix = cnn[curLayerIndex - 1][ni].output; let kernelMatrix = cnn[curLayerIndex][i].inputLinks[ni].weight; let interMatrix = singleConv(inputMatrix, kernelMatrix); // Compute the intermediate layer min max intermediateMinMax.push(getExtent(interMatrix)); // Update the intermediate sum // itnermediateSumMatrix = matrixAdd(itnermediateSumMatrix, interMatrix); // Layout the canvas and rect let newNode = createIntermediateNode(curLayerIndex, i, intermediateLayer, intermediateX1, n.y, ni, stride, intermediateNodeMouseOverHandler, intermediateNodeMouseLeaveHandler, intermediateNodeClicked, true); // Draw the image let image = newNode.select('image'); drawIntermidiateImage(image, range, colorScale, d.output.length, interMatrix); // Edge: input -> intermediate1 linkData.push({ source: getOutputKnot({x: leftX, y: n.y}), target: getInputKnot({x: intermediateX1, y: n.y}), name: `input-${ni}-inter1-${ni}` }); // Edge: intermediate1 -> intermediate2-1 linkData.push({ source: getOutputKnot({x: intermediateX1, y: n.y}), target: getInputKnot({x: intermediateX2, y: nodeCoordinate[curLayerIndex][i].y}), name: `inter1-${ni}-inter2-1` }); // Create a small kernel illustration // Here we minus 2 because of no padding // let tickTime1D = nodeLength / (kernelRectLength) - 2; let kernelRectX = leftX - kernelRectLength * 3 * 2; let kernelGroup = intermediateLayer.append('g') .attr('class', `kernel kernel-${ni}`) .attr('transform', `translate(${kernelRectX}, ${n.y})`); let weightText = 'Kernel weights: ['; let f2 = d3.format('.2f'); for (let r = 0; r < kernelMatrix.length; r++) { for (let c = 0; c < kernelMatrix[0].length; c++) { kernelGroup.append('rect') .attr('class', 'kernel') .attr('x', kernelRectLength * c) .attr('y', kernelRectLength * r) .attr('width', kernelRectLength) .attr('height', kernelRectLength) .attr('fill', gappedColorScale(layerColorScales.weight, kernelRange, kernelMatrix[r][c], kernelColorGap)); let sep = ''; if (c === 0 && r == 0) { sep = ''; } else if (c === 0) { sep = '; '; } else { sep = ', '; } weightText = weightText.concat(sep, `${f2(kernelMatrix[r][c])}`); } } weightText = weightText.concat(']'); kernelGroup.append('rect') .attr('x', 0) .attr('y', 0) .attr('width', kernelRectLength * 3) .attr('height', kernelRectLength * 3) .attr('fill', 'none') .attr('stroke', intermediateColor); kernelGroup.style('pointer-events', 'all') .style('cursor', 'crosshair') .on('mouseover', () => { hoverInfoStore.set( {show: true, text: weightText} ); }) .on('mouseleave', () => { hoverInfoStore.set( {show: false, text: weightText} ); }) .on('click', () => {d3.event.stopPropagation()}); // Sliding the kernel on the input channel and result channel at the same // time let kernelGroupInput = kernelGroup.clone(true) .style('pointer-events', 'none') .style('cursor', 'pointer') .classed('kernel-clone', true) .classed(`kernel-input-${ni}`, true); kernelGroupInput.style('opacity', 0.9) .selectAll('rect.kernel') .style('opacity', 0.7); kernelGroupInput.attr('transform', `translate(${leftX}, ${n.y})`) .attr('data-tick', 0) .attr('data-origin-x', leftX) .attr('data-origin-y', n.y); let kernelGroupResult = kernelGroup.clone(true) .style('pointer-events', 'none') .style('cursor', 'pointer') .classed('kernel-clone', true) .classed(`kernel-result-${ni}`, true); kernelGroupResult.style('opacity', 0.9) .selectAll('rect.kernel') .style('fill', 'none'); kernelGroupResult.attr('transform', `translate(${intermediateX1}, ${n.y})`) .attr('data-origin-x', intermediateX1) .attr('data-origin-y', n.y); startIntermediateAnimation(kernelGroupInput, kernelGroupResult, tickTime1D, stride); }); // Aggregate the intermediate min max let aggregatedExtent = intermediateMinMax.reduce((acc, cur) => { return [Math.min(acc[0], cur[0]), Math.max(acc[1], cur[1])]; }) let aggregatedMinMax = {min: aggregatedExtent[0], max: aggregatedExtent[1]}; // Draw the plus operation symbol let symbolY = nodeCoordinate[curLayerIndex][i].y + nodeLength / 2; let symbolRectHeight = 1; let symbolGroup = intermediateLayer.append('g') .attr('class', 'plus-symbol') .attr('transform', `translate(${intermediateX2 + plusSymbolRadius}, ${symbolY})`); symbolGroup.append('rect') .attr('x', -plusSymbolRadius) .attr('y', -plusSymbolRadius) .attr('width', 2 * plusSymbolRadius) .attr('height', 2 * plusSymbolRadius) .attr('rx', 3) .attr('ry', 3) .style('fill', 'none') .style('stroke', intermediateColor); symbolGroup.append('rect') .attr('x', -(plusSymbolRadius - 3)) .attr('y', -symbolRectHeight / 2) .attr('width', 2 * (plusSymbolRadius - 3)) .attr('height', symbolRectHeight) .style('fill', intermediateColor); symbolGroup.append('rect') .attr('x', -symbolRectHeight / 2) .attr('y', -(plusSymbolRadius - 3)) .attr('width', symbolRectHeight) .attr('height', 2 * (plusSymbolRadius - 3)) .style('fill', intermediateColor); // Place the bias rectangle below the plus sign if user clicks the firrst // conv node if (i == 0) { // Add bias symbol to the plus symbol symbolGroup.append('circle') .attr('cx', 0) .attr('cy', nodeLength / 2 + kernelRectLength) .attr('r', 4) .style('stroke', intermediateColor) .style('cursor', 'crosshair') .style('fill', gappedColorScale(layerColorScales.weight, kernelRange, d.bias, kernelColorGap)) .on('mouseover', () => { hoverInfoStore.set( {show: true, text: `Bias: ${formater(d.bias)}`} ); }) .on('mouseleave', () => { hoverInfoStore.set( {show: false, text: `Bias: ${formater(d.bias)}`} ); }); // Link from bias to the plus symbol linkData.push({ source: {x: intermediateX2 + plusSymbolRadius, y: nodeCoordinate[curLayerIndex][i].y + nodeLength}, target: {x: intermediateX2 + plusSymbolRadius, y: nodeCoordinate[curLayerIndex][i].y + nodeLength / 2 + plusSymbolRadius}, name: `bias-plus` }); } else { // Add bias symbol to the plus symbol symbolGroup.append('circle') .attr('cx', 0) .attr('cy', -nodeLength / 2 - kernelRectLength) .attr('r', 4) .style('stroke', intermediateColor) .style('cursor', 'crosshair') .style('fill', gappedColorScale(layerColorScales.weight, kernelRange, d.bias, kernelColorGap)) .on('mouseover', () => { hoverInfoStore.set( {show: true, text: `Bias: ${formater(d.bias)}`} ); }) .on('mouseleave', () => { hoverInfoStore.set( {show: false, text: `Bias: ${formater(d.bias)}`} ); }); // Link from bias to the plus symbol linkData.push({ source: {x: intermediateX2 + plusSymbolRadius, y: nodeCoordinate[curLayerIndex][i].y}, target: {x: intermediateX2 + plusSymbolRadius, y: nodeCoordinate[curLayerIndex][i].y + nodeLength / 2 - plusSymbolRadius}, name: `bias-plus` }); } // Link from the plus symbol to the output linkData.push({ source: getOutputKnot({x: intermediateX2 + 2 * plusSymbolRadius - nodeLength, y: nodeCoordinate[curLayerIndex][i].y}), target: getInputKnot({x: rightX, y: nodeCoordinate[curLayerIndex][i].y}), name: `symbol-output` }); // Output -> next layer linkData.push({ source: getOutputKnot({x: rightX, y: nodeCoordinate[curLayerIndex][i].y}), target: getInputKnot({x: rightStart, y: nodeCoordinate[curLayerIndex][i].y}), name: `output-next` }); // Draw the layer label intermediateLayer.append('g') .attr('class', 'layer-intermediate-label layer-label') .attr('transform', () => { let x = intermediateX1 + nodeLength / 2; let y = (svgPaddings.top + vSpaceAroundGap) / 2 + 5; return `translate(${x}, ${y})`; }) .classed('hidden', detailedMode) .append('text') .style('text-anchor', 'middle') .style('dominant-baseline', 'middle') .style('font-weight', 800) .style('opacity', '0.8') .text('intermediate'); intermediateLayer.append('g') .attr('class', 'animation-control') .attr('transform', () => { let x = intermediateX1 + nodeLength / 2; let y = (svgPaddings.top + vSpaceAroundGap) / 2 - 4; return `translate(${x}, ${y})`; }) .on('click', () => animationButtonClicked(curLayerIndex)) .append('image') .attr('class', 'animation-control-button') .attr('xlink:href', 'PUBLIC_URL/assets/img/fast_forward.svg') .attr('x', 50) .attr('y', 0) .attr('height', 13) .attr('width', 13); // Draw the detailed model layer label intermediateLayer.append('g') .attr('class', 'layer-intermediate-label layer-detailed-label') .attr('transform', () => { let x = intermediateX1 + nodeLength / 2; let y = (svgPaddings.top + vSpaceAroundGap) / 2 - 5; return `translate(${x}, ${y})`; }) .classed('hidden', !detailedMode) .append('text') .style('text-anchor', 'middle') .style('dominant-baseline', 'middle') .style('opacity', '0.7') .style('font-weight', 800) .append('tspan') .text('intermediate') .append('tspan') .style('font-size', '8px') .style('font-weight', 'normal') .attr('x', 0) .attr('dy', '1.5em') .text(`(${cnn[curLayerIndex][0].output.length}, ${cnn[curLayerIndex][0].output[0].length}, ${cnn[curLayerIndex].length})`); // Draw the edges let linkGen = d3.linkHorizontal() .x(d => d.x) .y(d => d.y); let edgeGroup = intermediateLayer.append('g') .attr('class', 'edge-group') .lower(); let dashoffset = 0; edgeGroup.selectAll('path') .data(linkData) .enter() .append('path') .classed('flow-edge', d => d.name !== 'output-next') .attr('id', d => `edge-${d.name}`) .attr('d', d => linkGen({source: d.source, target: d.target})) .style('fill', 'none') .style('stroke-width', 1) .style('stroke', intermediateColor); edgeGroup.select('#edge-output-next') .style('opacity', 0.1); edgeGroup.selectAll('path.flow-edge') .attr('stroke-dasharray', '4 2') .attr('stroke-dashoffset', 0) .each((d, i, g) => animateEdge(d, i, g, dashoffset - 1000)); return {intermediateLayer: intermediateLayer, intermediateMinMax: aggregatedMinMax, kernelRange: kernelRange, kernelMinMax: {min: kernelExtent[0], max: kernelExtent[1]}}; } /** * Add an annotation for the kernel and the sliding * @param {object} arg * { * leftX: X value of the left border of intermedaite layer * group: element group * intermediateGap: the inner gap of intermediate layer * isFirstConv: if this intermediate layer is after the first layer * i: index of the selected node * } */ const drawIntermediateLayerAnnotation = (arg) => { let leftX = arg.leftX, curLayerIndex = arg.curLayerIndex, group = arg.group, intermediateGap = arg.intermediateGap, isFirstConv = arg.isFirstConv, i = arg.i; let kernelAnnotation = group.append('g') .attr('class', 'kernel-annotation'); kernelAnnotation.append('text') .text('Kernel') .attr('class', 'annotation-text') .attr('x', leftX - 2.5 * kernelRectLength * 3) .attr('y', nodeCoordinate[curLayerIndex - 1][0].y + kernelRectLength * 3) .style('dominant-baseline', 'baseline') .style('text-anchor', 'end'); let sliderX, sliderY, arrowSX, arrowSY, dr; let sliderX2, sliderY2, arrowSX2, arrowSY2, dr2, arrowTX2, arrowTY2; if (isFirstConv) { sliderX = leftX; sliderY = nodeCoordinate[curLayerIndex - 1][0].y + nodeLength + kernelRectLength * 3; arrowSX = leftX - 5; arrowSY = nodeCoordinate[curLayerIndex - 1][0].y + nodeLength + kernelRectLength * 3 + 5; dr = 20; sliderX2 = leftX; sliderY2 = nodeCoordinate[curLayerIndex - 1][1].y + nodeLength + kernelRectLength * 3; arrowSX2 = leftX - kernelRectLength * 3; arrowSY2 = nodeCoordinate[curLayerIndex - 1][1].y + nodeLength + 15; arrowTX2 = leftX - 13; arrowTY2 = nodeCoordinate[curLayerIndex - 1][1].y + 15; dr2 = 35; } else { sliderX = leftX - 3 * kernelRectLength * 3; sliderY = nodeCoordinate[curLayerIndex - 1][0].y + nodeLength / 3; arrowSX = leftX - 2 * kernelRectLength * 3 - 5; arrowSY = nodeCoordinate[curLayerIndex - 1][0].y + nodeLength - 10; dr = 50; sliderX2 = leftX - 3 * kernelRectLength * 3; sliderY2 = nodeCoordinate[curLayerIndex - 1][2].y - 3; arrowTX2 = leftX - kernelRectLength * 3 - 4; arrowTY2 = nodeCoordinate[curLayerIndex - 1][2].y + kernelRectLength * 3 + 6; arrowSX2 = leftX - kernelRectLength * 3 - 13; arrowSY2 = nodeCoordinate[curLayerIndex - 1][2].y + 26; dr2 = 20; } let slideText = kernelAnnotation.append('text') .attr('x', sliderX) .attr('y', sliderY) .attr('class', 'annotation-text') .style('dominant-baseline', 'hanging') .style('text-anchor', isFirstConv ? 'start' : 'end'); slideText.append('tspan') .style('dominant-baseline', 'hanging') .text('Slide kernel over input channel'); slideText.append('tspan') .attr('x', sliderX) .attr('dy', '1em') .style('dominant-baseline', 'hanging') .text('to get intermediate result'); // slideText.append('tspan') // .attr('x', sliderX) // .attr('dy', '1em') // .style('dominant-baseline', 'hanging') // .text(''); slideText.append('tspan') .attr('x', sliderX) .attr('dy', '1.2em') .style('dominant-baseline', 'hanging') .style('font-weight', 700) .text('Click '); slideText.append('tspan') .style('dominant-baseline', 'hanging') .style('font-weight', 400) .text('to learn more') drawArrow({ group: group, tx: leftX - 7, ty: nodeCoordinate[curLayerIndex - 1][0].y + nodeLength / 2, sx: arrowSX, sy: arrowSY, hFlip: !isFirstConv, dr: dr, marker: 'marker' }); // Add kernel annotation let slideText2 = kernelAnnotation.append('text') .attr('x', sliderX2) .attr('y', sliderY2) .attr('class', 'annotation-text') .style('dominant-baseline', 'hanging') .style('text-anchor', isFirstConv ? 'start' : 'end'); slideText2.append('tspan') .style('dominant-baseline', 'hanging') .text('Each input chanel'); slideText2.append('tspan') .attr('x', sliderX) .attr('dy', '1em') .style('dominant-baseline', 'hanging') .text('gets a different kernel'); slideText2.append('tspan') .attr('x', sliderX) .attr('dy', '1.3em') .style('font-weight', 700) .style('dominant-baseline', 'hanging') .text('Hover over '); slideText2.append('tspan') .style('font-weight', 400) .style('dominant-baseline', 'hanging') .text('to see value!') drawArrow({ group: group, tx: arrowTX2, ty: arrowTY2, sx: arrowSX2, sy: arrowSY2, dr: dr2, hFlip: !isFirstConv, marker: 'marker' }); // Add annotation for the sum operation let plusAnnotation = group.append('g') .attr('class', 'plus-annotation'); let intermediateX2 = leftX + 2 * nodeLength + 2.5 * intermediateGap; let textX = intermediateX2; let textY = nodeCoordinate[curLayerIndex][i].y + nodeLength + kernelRectLength * 3; // Special case 1: first node if (i === 0) { textX += 30; } // Special case 2: last node if (i === 9) { textX = intermediateX2 + plusSymbolRadius - 10; textY -= 2.5 * nodeLength; } let plusText = plusAnnotation.append('text') .attr('x', textX) .attr('y', textY) .attr('class', 'annotation-text') .style('dominant-baseline', 'hanging') .style('text-anchor', 'start'); plusText.append('tspan') .style('dominant-baseline', 'hanging') .text('Add up all intermediate'); plusText.append('tspan') .attr('x', textX) .attr('dy', '1em') .style('dominant-baseline', 'hanging') .text('results and then add bias'); if (i === 9) { drawArrow({ group: group, sx: intermediateX2 + 50, sy: nodeCoordinate[curLayerIndex][i].y - (nodeLength / 2 + kernelRectLength * 2), tx: intermediateX2 + 2 * plusSymbolRadius + 5, ty: nodeCoordinate[curLayerIndex][i].y + nodeLength / 2 - plusSymbolRadius, dr: 50, hFlip: false, marker: 'marker-alt' }); } else { drawArrow({ group: group, sx: intermediateX2 + 35, sy: nodeCoordinate[curLayerIndex][i].y + nodeLength + kernelRectLength * 2, tx: intermediateX2 + 2 * plusSymbolRadius + 5, ty: nodeCoordinate[curLayerIndex][i].y + nodeLength / 2 + plusSymbolRadius, dr: 30, hFlip: true, marker: 'marker-alt' }); } // Add annotation for the bias let biasTextY = nodeCoordinate[curLayerIndex][i].y; if (i === 0) { biasTextY += nodeLength + 3 * kernelRectLength; } else { biasTextY -= 2 * kernelRectLength + 5; } plusAnnotation.append('text') .attr('class', 'annotation-text') .attr('x', intermediateX2 + plusSymbolRadius) .attr('y', biasTextY) .style('text-anchor', 'middle') .style('dominant-baseline', i === 0 ? 'hanging' : 'baseline') .text('Bias'); } /** * Append a filled rectangle under a pair of nodes. * @param {number} curLayerIndex Index of the selected layer * @param {number} i Index of the selected node * @param {number} leftX X value of the left border of intermediate layer * @param {number} intermediateGap Inner gap of this intermediate layer * @param {number} padding Padding around the rect * @param {function} intermediateNodeMouseOverHandler Mouse over handler * @param {function} intermediateNodeMouseLeaveHandler Mouse leave handler * @param {function} intermediateNodeClicked Mouse click handler */ const addUnderneathRect = (curLayerIndex, i, leftX, intermediateGap, padding, intermediateNodeMouseOverHandler, intermediateNodeMouseLeaveHandler, intermediateNodeClicked) => { // Add underneath rects let underGroup = svg.select('g.underneath'); for (let n = 0; n < cnn[curLayerIndex - 1].length; n++) { underGroup.append('rect') .attr('class', 'underneath-gateway') .attr('id', `underneath-gateway-${n}`) .attr('x', leftX - padding) .attr('y', nodeCoordinate[curLayerIndex - 1][n].y - padding) .attr('width', (2 * nodeLength + intermediateGap) + 2 * padding) .attr('height', nodeLength + 2 * padding) .attr('rx', 10) .style('fill', 'rgba(160, 160, 160, 0.2)') .style('opacity', 0); // Register new events for input layer nodes svg.select(`g#layer-${curLayerIndex - 1}-node-${n}`) .style('pointer-events', 'all') .style('cursor', 'pointer') .on('mouseover', intermediateNodeMouseOverHandler) .on('mouseleave', intermediateNodeMouseLeaveHandler) .on('click', (d, ni, g) => intermediateNodeClicked(d, ni, g, i, curLayerIndex)); // .on('click', (d, i) => {console.log(i)}); } underGroup.lower(); } /** * Add an overlaying rect * @param {string} gradientName Gradient name of overlay rect * @param {number} x X value of the overlaying rect * @param {number} y Y value of the overlaying rect * @param {number} width Rect width * @param {number} height Rect height */ export const addOverlayRect = (gradientName, x, y, width, height) => { if (svg.select('.intermediate-layer-overlay').empty()) { svg.append('g').attr('class', 'intermediate-layer-overlay'); } let intermediateLayerOverlay = svg.select('.intermediate-layer-overlay'); let overlayRect = intermediateLayerOverlay.append('rect') .attr('class', 'overlay') .style('fill', `url(#${gradientName})`) .style('stroke', 'none') .attr('width', width) .attr('height', height) .attr('x', x) .attr('y', y) .style('opacity', 0); overlayRect.transition('move') .duration(800) .ease(d3.easeCubicInOut) .style('opacity', 1); } /** * Redraw the layer if needed (entering the intermediate view to make sure * all layers have the same color scale) * @param {number} curLayerIndex Index of the selected layer * @param {number} i Index of the selected node */ const redrawLayerIfNeeded = (curLayerIndex, i) => { // Determine the range for this layerview, and redraw the layer with // smaller range so all layers have the same range let rangePre = cnnLayerRanges[selectedScaleLevel][curLayerIndex - 1]; let rangeCur = cnnLayerRanges[selectedScaleLevel][curLayerIndex]; let range = Math.max(rangePre, rangeCur); if (rangePre > rangeCur) { // Redraw the current layer (selected node) svg.select(`g#layer-${curLayerIndex}-node-${i}`) .select('image.node-image') .each((d, g, i) => drawOutput(d, g, i, range)); // Record the change so we will re-redraw the layer when user quits // the intermediate view needRedraw = [curLayerIndex, i]; needRedrawStore.set(needRedraw); } else if (rangePre < rangeCur) { // Redraw the previous layer (whole layer) svg.select(`g#cnn-layer-group-${curLayerIndex - 1}`) .selectAll('image.node-image') .each((d, g, i) => drawOutput(d, g, i, range)); // Record the change so we will re-redraw the layer when user quits // the intermediate view needRedraw = [curLayerIndex - 1, undefined]; needRedrawStore.set(needRedraw); } // Compute the min, max value of all nodes in pre-layer and the selected // node of cur-layer let min = cnnLayerMinMax[curLayerIndex - 1].min, max = cnnLayerMinMax[curLayerIndex - 1].max; // Selected node let n = cnn[curLayerIndex][i]; for (let r = 0; r < n.output.length; r++) { for (let c = 0; c < n.output[0].length; c++) { if (n.output[r][c] < min) { min = n.output[r][c]; } if (n.output[r][c] > max) { max = n.output[r][c]; } } } return {range: range, minMax: {min: min, max: max}}; } /** * Draw the intermediate layer before conv_1_1 * @param {number} curLayerIndex Index of the selected layer * @param {object} d Bounded d3 data * @param {number} i Index of the selected node * @param {number} width CNN group width * @param {number} height CNN group height * @param {function} intermediateNodeMouseOverHandler mouse over handler * @param {function} intermediateNodeMouseLeaveHandler mouse leave handler * @param {function} intermediateNodeClicked node clicking handler */ export const drawConv1 = (curLayerIndex, d, i, width, height, intermediateNodeMouseOverHandler, intermediateNodeMouseLeaveHandler, intermediateNodeClicked) => { // Compute the target location let targetX = nodeCoordinate[curLayerIndex - 1][0].x + 2 * nodeLength + 2 * hSpaceAroundGap * gapRatio + plusSymbolRadius * 2; let intermediateGap = (hSpaceAroundGap * gapRatio * 2) / 3; let leftX = nodeCoordinate[curLayerIndex - 1][0].x; // Record the left x position for dynamic detial view positioning intermediateLayerPosition['conv_1_1'] = targetX + nodeLength; intermediateLayerPositionStore.set(intermediateLayerPosition); // Hide the edges svg.select('g.edge-group') .style('visibility', 'hidden'); // Move the selected layer moveLayerX({layerIndex: curLayerIndex, targetX: targetX, disable: true, delay: 0, opacity: 0.15, specialIndex: i}); // Compute the gap in the right shrink region let rightStart = targetX + nodeLength + hSpaceAroundGap * gapRatio; let rightGap = (width - rightStart - 10 * nodeLength) / 10; // Move the right layers for (let i = curLayerIndex + 1; i < numLayers; i++) { let curX = rightStart + (i - (curLayerIndex + 1)) * (nodeLength + rightGap); moveLayerX({layerIndex: i, targetX: curX, disable: true, delay: 0}); } // Add an overlay gradient and rect let stops = [{offset: '0%', color: 'rgb(250, 250, 250)', opacity: 0.85}, {offset: '50%', color: 'rgb(250, 250, 250)', opacity: 0.95}, {offset: '100%', color: 'rgb(250, 250, 250)', opacity: 1}]; addOverlayGradient('overlay-gradient', stops); addOverlayRect('overlay-gradient', rightStart - overlayRectOffset / 2, 0, width - rightStart + overlayRectOffset, height + svgPaddings.top + svgPaddings.bottom); // Draw the intermediate layer let {intermediateLayer, intermediateMinMax, kernelRange, kernelMinMax} = drawIntermediateLayer(curLayerIndex, leftX, targetX, rightStart, intermediateGap, d, i, intermediateNodeMouseOverHandler, intermediateNodeMouseLeaveHandler, intermediateNodeClicked); addUnderneathRect(curLayerIndex, i, leftX, intermediateGap, 8, intermediateNodeMouseOverHandler, intermediateNodeMouseLeaveHandler, intermediateNodeClicked); // Compute the selected node's min max // Selected node let min = Infinity, max = -Infinity; let n = cnn[curLayerIndex][i]; for (let r = 0; r < n.output.length; r++) { for (let c = 0; c < n.output[0].length; c++) { if (n.output[r][c] < min) { min = n.output[r][c]; } if (n.output[r][c] > max) { max = n.output[r][c]; } } } let finalMinMax = { min: Math.min(min, intermediateMinMax.min), max: Math.max(max, intermediateMinMax.max) } // Add annotation to the intermediate layer let intermediateLayerAnnotation = svg.append('g') .attr('class', 'intermediate-layer-annotation') .style('opacity', 0); drawIntermediateLayerAnnotation({ leftX: leftX, curLayerIndex: curLayerIndex, group: intermediateLayerAnnotation, intermediateGap: intermediateGap, isFirstConv: true, i: i }); let range = cnnLayerRanges.local[curLayerIndex]; drawIntermediateLayerLegend({ legendHeight: 5, curLayerIndex: curLayerIndex, range: 1, group: intermediateLayer, width: 2 * nodeLength + intermediateGap, isInput: true, x: leftX, y: svgPaddings.top + vSpaceAroundGap * (10) + vSpaceAroundGap + nodeLength * 10 - 25 }); drawIntermediateLayerLegend({ legendHeight: 5, curLayerIndex: curLayerIndex, range: range, minMax: finalMinMax, group: intermediateLayer, width: 2 * nodeLength + intermediateGap, x: nodeCoordinate[curLayerIndex - 1][2].x, y: svgPaddings.top + vSpaceAroundGap * (10) + vSpaceAroundGap + nodeLength * 10 }); drawIntermediateLayerLegend({ legendHeight: 5, curLayerIndex: curLayerIndex, range: kernelRange, minMax: kernelMinMax, group: intermediateLayer, width: 2 * nodeLength + intermediateGap, x: targetX + nodeLength - (2 * nodeLength + intermediateGap), y: svgPaddings.top + vSpaceAroundGap * (10) + vSpaceAroundGap + nodeLength * 10, gradientAppendingName: 'kernelColorGradient', colorScale: layerColorScales.weight, gradientGap: 0.2 }); // Show everything svg.selectAll('g.intermediate-layer, g.intermediate-layer-annotation') .transition() .delay(500) .duration(500) .ease(d3.easeCubicInOut) .style('opacity', 1); } /** * Draw the intermediate layer before conv_1_2 * @param {number} curLayerIndex Index of the selected layer * @param {object} d Bounded d3 data * @param {number} i Index of the selected node * @param {number} width CNN group width * @param {number} height CNN group height * @param {function} intermediateNodeMouseOverHandler mouse over handler * @param {function} intermediateNodeMouseLeaveHandler mouse leave handler * @param {function} intermediateNodeClicked node clicking handler */ export const drawConv2 = (curLayerIndex, d, i, width, height, intermediateNodeMouseOverHandler, intermediateNodeMouseLeaveHandler, intermediateNodeClicked) => { let targetX = nodeCoordinate[curLayerIndex - 1][0].x + 2 * nodeLength + 2 * hSpaceAroundGap * gapRatio + plusSymbolRadius * 2; let intermediateGap = (hSpaceAroundGap * gapRatio * 2) / 3; // Record the left x position for dynamic detial view positioning intermediateLayerPosition['conv_1_2'] = targetX + nodeLength; intermediateLayerPositionStore.set(intermediateLayerPosition); // Make sure two layers have the same range let {range, minMax} = redrawLayerIfNeeded(curLayerIndex, i); // Hide the edges svg.select('g.edge-group') .style('visibility', 'hidden'); // Move the selected layer moveLayerX({layerIndex: curLayerIndex, targetX: targetX, disable: true, delay: 0, opacity: 0.15, specialIndex: i}); // Compute the gap in the right shrink region let rightStart = targetX + nodeLength + hSpaceAroundGap * gapRatio; let rightGap = (width - rightStart - 8 * nodeLength) / 8; // Move the right layers for (let i = curLayerIndex + 1; i < numLayers; i++) { let curX = rightStart + (i - (curLayerIndex + 1)) * (nodeLength + rightGap); moveLayerX({layerIndex: i, targetX: curX, disable: true, delay: 0}); } // Add an overlay let stops = [{offset: '0%', color: 'rgb(250, 250, 250)', opacity: 0.85}, {offset: '50%', color: 'rgb(250, 250, 250)', opacity: 0.95}, {offset: '100%', color: 'rgb(250, 250, 250)', opacity: 1}]; addOverlayGradient('overlay-gradient-right', stops); let leftRightRatio = (2 * nodeLength + hSpaceAroundGap * gapRatio) / (8 * nodeLength + intermediateGap * 7); let endingGradient = 0.85 + (0.95 - 0.85) * leftRightRatio; stops = [{offset: '0%', color: 'rgb(250, 250, 250)', opacity: endingGradient}, {offset: '100%', color: 'rgb(250, 250, 250)', opacity: 0.85}]; addOverlayGradient('overlay-gradient-left', stops); addOverlayRect('overlay-gradient-right', rightStart - overlayRectOffset / 2, 0, width - rightStart + overlayRectOffset, height + svgPaddings.top + svgPaddings.bottom); addOverlayRect('overlay-gradient-left', nodeCoordinate[0][0].x - overlayRectOffset / 2, 0, nodeLength * 2 + hSpaceAroundGap * gapRatio + overlayRectOffset, height + svgPaddings.top + svgPaddings.bottom); // Draw the intermediate layer let leftX = nodeCoordinate[curLayerIndex - 1][0].x; let {intermediateLayer, intermediateMinMax, kernelRange, kernelMinMax} = drawIntermediateLayer(curLayerIndex, leftX, targetX, rightStart, intermediateGap, d, i, intermediateNodeMouseOverHandler, intermediateNodeMouseLeaveHandler, intermediateNodeClicked); addUnderneathRect(curLayerIndex, i, leftX, intermediateGap, 5, intermediateNodeMouseOverHandler, intermediateNodeMouseLeaveHandler, intermediateNodeClicked); // After getting the intermediateMinMax, we can finally aggregate it with // the preLayer minmax, curLayer minmax let finalMinMax = { min: Math.min(minMax.min, intermediateMinMax.min), max: Math.max(minMax.max, intermediateMinMax.max) } // Add annotation to the intermediate layer let intermediateLayerAnnotation = svg.append('g') .attr('class', 'intermediate-layer-annotation') .style('opacity', 0); drawIntermediateLayerAnnotation({ leftX: leftX, curLayerIndex: curLayerIndex, group: intermediateLayerAnnotation, intermediateGap: intermediateGap, i: i }); drawIntermediateLayerLegend({ legendHeight: 5, curLayerIndex: curLayerIndex, range: range, minMax: finalMinMax, group: intermediateLayer, width: 2 * nodeLength + intermediateGap, x: leftX, y: svgPaddings.top + vSpaceAroundGap * (10) + vSpaceAroundGap + nodeLength * 10 }); drawIntermediateLayerLegend({ legendHeight: 5, curLayerIndex: curLayerIndex, range: kernelRange, minMax: kernelMinMax, group: intermediateLayer, width: 2 * nodeLength + intermediateGap, x: targetX + nodeLength - (2 * nodeLength + intermediateGap), y: svgPaddings.top + vSpaceAroundGap * (10) + vSpaceAroundGap + nodeLength * 10, gradientAppendingName: 'kernelColorGradient', colorScale: layerColorScales.weight, gradientGap: 0.2 }); // Show everything svg.selectAll('g.intermediate-layer, g.intermediate-layer-annotation') .transition() .delay(500) .duration(500) .ease(d3.easeCubicInOut) .style('opacity', 1); } /** * Draw the intermediate layer before conv_2_1 * @param {number} curLayerIndex Index of the selected layer * @param {object} d Bounded d3 data * @param {number} i Index of the selected node * @param {number} width CNN group width * @param {number} height CNN group height * @param {function} intermediateNodeMouseOverHandler mouse over handler * @param {function} intermediateNodeMouseLeaveHandler mouse leave handler * @param {function} intermediateNodeClicked node clicking handler */ export const drawConv3 = (curLayerIndex, d, i, width, height, intermediateNodeMouseOverHandler, intermediateNodeMouseLeaveHandler, intermediateNodeClicked) => { let targetX = nodeCoordinate[curLayerIndex][0].x; let leftX = targetX - (2 * nodeLength + 2 * hSpaceAroundGap * gapRatio + plusSymbolRadius * 2); let intermediateGap = (hSpaceAroundGap * gapRatio * 2) / 3; // Record the left x position for dynamic detial view positioning intermediateLayerPosition['conv_2_1'] = targetX + nodeLength; intermediateLayerPositionStore.set(intermediateLayerPosition); // Hide the edges svg.select('g.edge-group') .style('visibility', 'hidden'); // Make sure two layers have the same range let {range, minMax} = redrawLayerIfNeeded(curLayerIndex, i); // Move the previous layer moveLayerX({layerIndex: curLayerIndex - 1, targetX: leftX, disable: true, delay: 0}); moveLayerX({layerIndex: curLayerIndex, targetX: targetX, disable: true, delay: 0, opacity: 0.15, specialIndex: i}); // Compute the gap in the left shrink region let leftEnd = leftX - hSpaceAroundGap; let leftGap = (leftEnd - nodeCoordinate[0][0].x - 5 * nodeLength) / 5; let rightStart = nodeCoordinate[curLayerIndex][0].x + nodeLength + hSpaceAroundGap; // Move the left layers for (let i = 0; i < curLayerIndex - 1; i++) { let curX = nodeCoordinate[0][0].x + i * (nodeLength + leftGap); moveLayerX({layerIndex: i, targetX: curX, disable: true, delay: 0}); } // Add an overlay let stops = [{offset: '0%', color: 'rgb(250, 250, 250)', opacity: 1}, {offset: '50%', color: 'rgb(250, 250, 250)', opacity: 0.9}, {offset: '100%', color: 'rgb(250, 250, 250)', opacity: 0.85}]; addOverlayGradient('overlay-gradient-left', stops); stops = [{offset: '0%', color: 'rgb(250, 250, 250)', opacity: 0.85}, {offset: '50%', color: 'rgb(250, 250, 250)', opacity: 0.95}, {offset: '100%', color: 'rgb(250, 250, 250)', opacity: 1}]; addOverlayGradient('overlay-gradient-right', stops); addOverlayRect('overlay-gradient-left', nodeCoordinate[0][0].x - overlayRectOffset / 2, 0, leftEnd - nodeCoordinate[0][0].x + overlayRectOffset, height + svgPaddings.top + svgPaddings.bottom); addOverlayRect('overlay-gradient-right', rightStart - overlayRectOffset / 2, 0, width - rightStart + overlayRectOffset, height + svgPaddings.top + svgPaddings.bottom); // Draw the intermediate layer let {intermediateLayer, intermediateMinMax, kernelRange, kernelMinMax} = drawIntermediateLayer(curLayerIndex, leftX, nodeCoordinate[curLayerIndex][0].x, rightStart, intermediateGap, d, i, intermediateNodeMouseOverHandler, intermediateNodeMouseLeaveHandler, intermediateNodeClicked); addUnderneathRect(curLayerIndex, i, leftX, intermediateGap, 5, intermediateNodeMouseOverHandler, intermediateNodeMouseLeaveHandler, intermediateNodeClicked); // After getting the intermediateMinMax, we can finally aggregate it with // the preLayer minmax, curLayer minmax let finalMinMax = { min: Math.min(minMax.min, intermediateMinMax.min), max: Math.max(minMax.max, intermediateMinMax.max) } // Add annotation to the intermediate layer let intermediateLayerAnnotation = svg.append('g') .attr('class', 'intermediate-layer-annotation') .style('opacity', 0); drawIntermediateLayerAnnotation({ leftX: leftX, curLayerIndex: curLayerIndex, group: intermediateLayerAnnotation, intermediateGap: intermediateGap, i: i }); drawIntermediateLayerLegend({ legendHeight: 5, curLayerIndex: curLayerIndex, range: range, group: intermediateLayer, width: 2 * nodeLength + intermediateGap, minMax: finalMinMax, x: leftX, y: svgPaddings.top + vSpaceAroundGap * (10) + vSpaceAroundGap + nodeLength * 10 }); drawIntermediateLayerLegend({ legendHeight: 5, curLayerIndex: curLayerIndex, range: kernelRange, minMax: kernelMinMax, group: intermediateLayer, width: 2 * nodeLength + intermediateGap, x: targetX + nodeLength - (2 * nodeLength + intermediateGap), y: svgPaddings.top + vSpaceAroundGap * (10) + vSpaceAroundGap + nodeLength * 10, gradientAppendingName: 'kernelColorGradient', colorScale: layerColorScales.weight, gradientGap: 0.2 }); // Show everything svg.selectAll('g.intermediate-layer, g.intermediate-layer-annotation') .transition() .delay(500) .duration(500) .ease(d3.easeCubicInOut) .style('opacity', 1); } /** * Draw the intermediate layer before conv_2_2 * @param {number} curLayerIndex Index of the selected layer * @param {object} d Bounded d3 data * @param {number} i Index of the selected node * @param {number} width CNN group width * @param {number} height CNN group height * @param {function} intermediateNodeMouseOverHandler mouse over handler * @param {function} intermediateNodeMouseLeaveHandler mouse leave handler * @param {function} intermediateNodeClicked node clicking handler */ export const drawConv4 = (curLayerIndex, d, i, width, height, intermediateNodeMouseOverHandler, intermediateNodeMouseLeaveHandler, intermediateNodeClicked) => { let targetX = nodeCoordinate[curLayerIndex][0].x; let leftX = targetX - (2 * nodeLength + 2 * hSpaceAroundGap * gapRatio + plusSymbolRadius * 2); let intermediateGap = (hSpaceAroundGap * gapRatio * 2) / 3; // Record the left x position for dynamic detial view positioning intermediateLayerPosition['conv_2_2'] = leftX; intermediateLayerPositionStore.set(intermediateLayerPosition); // Hide the edges svg.select('g.edge-group') .style('visibility', 'hidden'); // Make sure two layers have the same range let {range, minMax} = redrawLayerIfNeeded(curLayerIndex, i); // Move the previous layer moveLayerX({layerIndex: curLayerIndex - 1, targetX: leftX, disable: true, delay: 0}); moveLayerX({layerIndex: curLayerIndex, targetX: targetX, disable: true, delay: 0, opacity: 0.15, specialIndex: i}); // Compute the gap in the left shrink region let leftEnd = leftX - hSpaceAroundGap; let leftGap = (leftEnd - nodeCoordinate[0][0].x - 7 * nodeLength) / 7; let rightStart = targetX + nodeLength + hSpaceAroundGap; // Move the left layers for (let i = 0; i < curLayerIndex - 1; i++) { let curX = nodeCoordinate[0][0].x + i * (nodeLength + leftGap); moveLayerX({layerIndex: i, targetX: curX, disable: true, delay: 0}); } // Add an overlay let stops = [{offset: '0%', color: 'rgb(250, 250, 250)', opacity: 1}, {offset: '50%', color: 'rgb(250, 250, 250)', opacity: 0.95}, {offset: '100%', color: 'rgb(250, 250, 250)', opacity: 0.85}]; addOverlayGradient('overlay-gradient-left', stops); stops = [{offset: '0%', color: 'rgb(250, 250, 250)', opacity: 0.85}, {offset: '50%', color: 'rgb(250, 250, 250)', opacity: 0.95}, {offset: '100%', color: 'rgb(250, 250, 250)', opacity: 1}]; addOverlayGradient('overlay-gradient-right', stops); addOverlayRect('overlay-gradient-left', nodeCoordinate[0][0].x - overlayRectOffset / 2, 0, leftEnd - nodeCoordinate[0][0].x + overlayRectOffset, height + svgPaddings.top + svgPaddings.bottom); addOverlayRect('overlay-gradient-right', rightStart - overlayRectOffset / 2, 0, width - rightStart + overlayRectOffset, height + svgPaddings.top + svgPaddings.bottom); // Draw the intermediate layer let {intermediateLayer, intermediateMinMax, kernelRange, kernelMinMax} = drawIntermediateLayer(curLayerIndex, leftX, nodeCoordinate[curLayerIndex][0].x, rightStart, intermediateGap, d, i, intermediateNodeMouseOverHandler, intermediateNodeMouseLeaveHandler, intermediateNodeClicked); addUnderneathRect(curLayerIndex, i, leftX, intermediateGap, 5, intermediateNodeMouseOverHandler, intermediateNodeMouseLeaveHandler, intermediateNodeClicked); // After getting the intermediateMinMax, we can finally aggregate it with // the preLayer minmax, curLayer minmax let finalMinMax = { min: Math.min(minMax.min, intermediateMinMax.min), max: Math.max(minMax.max, intermediateMinMax.max) } // Add annotation to the intermediate layer let intermediateLayerAnnotation = svg.append('g') .attr('class', 'intermediate-layer-annotation') .style('opacity', 0); drawIntermediateLayerAnnotation({ leftX: leftX, curLayerIndex: curLayerIndex, group: intermediateLayerAnnotation, intermediateGap: intermediateGap, i: i }); drawIntermediateLayerLegend({ legendHeight: 5, curLayerIndex: curLayerIndex, range: range, group: intermediateLayer, minMax: finalMinMax, width: 2 * nodeLength + intermediateGap, x: leftX, y: svgPaddings.top + vSpaceAroundGap * (10) + vSpaceAroundGap + nodeLength * 10 }); drawIntermediateLayerLegend({ legendHeight: 5, curLayerIndex: curLayerIndex, range: kernelRange, minMax: kernelMinMax, group: intermediateLayer, width: 2 * nodeLength + intermediateGap, x: targetX + nodeLength - (2 * nodeLength + intermediateGap), y: svgPaddings.top + vSpaceAroundGap * (10) + vSpaceAroundGap + nodeLength * 10, gradientAppendingName: 'kernelColorGradient', colorScale: layerColorScales.weight, gradientGap: 0.2 }); // Show everything svg.selectAll('g.intermediate-layer, g.intermediate-layer-annotation') .transition() .delay(500) .duration(500) .ease(d3.easeCubicInOut) .style('opacity', 1); } ================================================ FILE: src/overview/intermediate-utils.js ================================================ /* global d3 */ import { svgStore, vSpaceAroundGapStore } from '../stores.js'; import { overviewConfig } from '../config.js'; // Configs const layerColorScales = overviewConfig.layerColorScales; const nodeLength = overviewConfig.nodeLength; const intermediateColor = overviewConfig.intermediateColor; const svgPaddings = overviewConfig.svgPaddings; // Shared variables let svg = undefined; svgStore.subscribe( value => {svg = value;} ) let vSpaceAroundGap = undefined; vSpaceAroundGapStore.subscribe( value => {vSpaceAroundGap = value;} ) /** * Move one layer horizontally * @param {object} arg Multiple arguments { * layerIndex: current layer index * targetX: destination x * disable: make this layer unresponsible * delay: animation delay * opacity: change the current layer's opacity * specialIndex: avoid manipulating `specialIndex`th node * onEndFunc: call this function when animation finishes * transitionName: animation ID * } */ export const moveLayerX = (arg) => { let layerIndex = arg.layerIndex; let targetX = arg.targetX; let disable = arg.disable; let delay = arg.delay; let opacity = arg.opacity; let specialIndex = arg.specialIndex; let onEndFunc = arg.onEndFunc; let transitionName = arg.transitionName === undefined ? 'move' : arg.transitionName; let duration = arg.duration === undefined ? 500 : arg.duration; // Move the selected layer let curLayer = svg.select(`g#cnn-layer-group-${layerIndex}`); curLayer.selectAll('g.node-group').each((d, i, g) => { d3.select(g[i]) .style('cursor', disable && i !== specialIndex ? 'default' : 'pointer') .style('pointer-events', disable && i !== specialIndex ? 'none' : 'all') .select('image') .transition(transitionName) .ease(d3.easeCubicInOut) .delay(delay) .duration(duration) .attr('x', targetX); d3.select(g[i]) .select('rect.bounding') .transition(transitionName) .ease(d3.easeCubicInOut) .delay(delay) .duration(duration) .attr('x', targetX); if (opacity !== undefined && i !== specialIndex) { d3.select(g[i]) .select('image') .style('opacity', opacity); } }); // Also move the layer labels svg.selectAll(`g#layer-label-${layerIndex}`) .transition(transitionName) .ease(d3.easeCubicInOut) .delay(delay) .duration(duration) .attr('transform', () => { let x = targetX + nodeLength / 2; let y = (svgPaddings.top + vSpaceAroundGap) / 2 + 5; return `translate(${x}, ${y})`; }) .on('end', onEndFunc); svg.selectAll(`g#layer-detailed-label-${layerIndex}`) .transition(transitionName) .ease(d3.easeCubicInOut) .delay(delay) .duration(duration) .attr('transform', () => { let x = targetX + nodeLength / 2; let y = (svgPaddings.top + vSpaceAroundGap) / 2 - 6; return `translate(${x}, ${y})`; }) .on('end', onEndFunc); } /** * Append a gradient definition to `group` * @param {string} gradientID CSS ID for the gradient def * @param {[{offset: number, color: string, opacity: number}]} stops Gradient stops * @param {element} group Element to append def to */ export const addOverlayGradient = (gradientID, stops, group) => { if (group === undefined) { group = svg; } // Create a gradient let defs = group.append("defs") .attr('class', 'overlay-gradient'); let gradient = defs.append("linearGradient") .attr("id", gradientID) .attr("x1", "0%") .attr("x2", "100%") .attr("y1", "100%") .attr("y2", "100%"); stops.forEach(s => { gradient.append('stop') .attr('offset', s.offset) .attr('stop-color', s.color) .attr('stop-opacity', s.opacity); }) } /** * Draw the legend for intermediate layer * @param {object} arg * { * legendHeight: height of the legend rectangle * curLayerIndex: the index of selected layer * range: colormap range * group: group to append the legend * minMax: {min: min value, max: max value} * width: width of the legend * x: x position of the legend * y: y position of the legend * isInput: if the legend is for the input layer (special handle black to * white color scale) * colorScale: d3 color scale * gradientAppendingName: name of the appending gradient * gradientGap: gap to make the color lighter * } */ export const drawIntermediateLayerLegend = (arg) => { let legendHeight = arg.legendHeight, curLayerIndex = arg.curLayerIndex, range = arg.range, group = arg.group, minMax = arg.minMax, width = arg.width, x = arg.x, y = arg.y, isInput = arg.isInput, colorScale = arg.colorScale, gradientAppendingName = arg.gradientAppendingName, gradientGap = arg.gradientGap; if (colorScale === undefined) { colorScale = layerColorScales.conv; } if (gradientGap === undefined) { gradientGap = 0; } // Add a legend color gradient let gradientName = 'url(#inputGradient)'; let normalizedColor = v => colorScale(v * (1 - 2 * gradientGap) + gradientGap); if (!isInput) { let leftValue = (minMax.min + range / 2) / range, zeroValue = (0 + range / 2) / range, rightValue = (minMax.max + range / 2) / range, totalRange = minMax.max - minMax.min, zeroLocation = (0 - minMax.min) / totalRange, leftMidValue = leftValue + (zeroValue - leftValue)/2, rightMidValue = zeroValue + (rightValue - zeroValue)/2; let stops = [ {offset: 0, color: normalizedColor(leftValue), opacity: 1}, {offset: zeroLocation / 2, color: normalizedColor(leftMidValue), opacity: 1}, {offset: zeroLocation, color: normalizedColor(zeroValue), opacity: 1}, {offset: zeroLocation + (1 - zeroValue) / 2, color: normalizedColor(rightMidValue), opacity: 1}, {offset: 1, color: normalizedColor(rightValue), opacity: 1} ]; if (gradientAppendingName === undefined) { addOverlayGradient('intermediate-legend-gradient', stops, group); gradientName = 'url(#intermediate-legend-gradient)'; } else { addOverlayGradient(`${gradientAppendingName}`, stops, group); gradientName = `url(#${gradientAppendingName})`; } } let legendScale = d3.scaleLinear() .range([0, width - 1.2]) .domain(isInput ? [0, range] : [minMax.min, minMax.max]); let legendAxis = d3.axisBottom() .scale(legendScale) .tickFormat(d3.format(isInput ? 'd' : '.2f')) .tickValues(isInput ? [0, range] : [minMax.min, 0, minMax.max]); let intermediateLegend = group.append('g') .attr('class', `intermediate-legend-${curLayerIndex - 1}`) .attr('transform', `translate(${x}, ${y})`); let legendGroup = intermediateLegend.append('g') .attr('transform', `translate(0, ${legendHeight - 3})`) .call(legendAxis); legendGroup.selectAll('text') .style('font-size', '9px') .style('fill', intermediateColor); legendGroup.selectAll('path, line') .style('stroke', intermediateColor); intermediateLegend.append('rect') .attr('width', width) .attr('height', legendHeight) .attr('transform', `rotate(${isInput ? 180 : 0}, ${width / 2}, ${legendHeight / 2})`) .style('fill', gradientName); } /** * Draw an very neat arrow! * @param {object} arg * { * group: element to append this arrow to * sx: source x * sy: source y * tx: target x * ty: target y * dr: radius of curve (I'm using a circle) * hFlip: the direction to choose the circle (there are always two ways) * } */ export const drawArrow = (arg) => { let group = arg.group, sx = arg.sx, sy = arg.sy, tx = arg.tx, ty = arg.ty, dr = arg.dr, hFlip = arg.hFlip, marker = arg.marker === undefined ? 'marker' : arg.marker; /* Cool graphics trick -> merge translate and scale together translateX = (1 - scaleX) * tx, translateY = (1 - scaleY) * ty; */ let arrow = group.append('g') .attr('class', 'arrow-group'); arrow.append('path') .attr("d", `M${sx},${sy}A${dr},${dr} 0 0,${hFlip ? 0 : 1} ${tx},${ty}`) .attr('marker-end', `url(#${marker})`) .style('stroke', 'gray') .style('fill', 'none'); } ================================================ FILE: src/overview/overview-draw.js ================================================ /* global d3, SmoothScroll */ import { svgStore, vSpaceAroundGapStore, hSpaceAroundGapStore, cnnStore, nodeCoordinateStore, selectedScaleLevelStore, cnnLayerRangesStore, detailedModeStore, cnnLayerMinMaxStore, hoverInfoStore } from '../stores.js'; import { getExtent, getLinkData } from './draw-utils.js'; import { overviewConfig } from '../config.js'; // Configs const layerColorScales = overviewConfig.layerColorScales; const nodeLength = overviewConfig.nodeLength; const numLayers = overviewConfig.numLayers; const edgeOpacity = overviewConfig.edgeOpacity; const edgeInitColor = overviewConfig.edgeInitColor; const edgeStrokeWidth = overviewConfig.edgeStrokeWidth; const svgPaddings = overviewConfig.svgPaddings; const gapRatio = overviewConfig.gapRatio; const classLists = overviewConfig.classLists; const formater = d3.format('.4f'); // Shared variables let svg = undefined; svgStore.subscribe( value => {svg = value;} ) let vSpaceAroundGap = undefined; vSpaceAroundGapStore.subscribe( value => {vSpaceAroundGap = value;} ) let hSpaceAroundGap = undefined; hSpaceAroundGapStore.subscribe( value => {hSpaceAroundGap = value;} ) let cnn = undefined; cnnStore.subscribe( value => {cnn = value;} ) let nodeCoordinate = undefined; nodeCoordinateStore.subscribe( value => {nodeCoordinate = value;} ) let selectedScaleLevel = undefined; selectedScaleLevelStore.subscribe( value => {selectedScaleLevel = value;} ) let cnnLayerRanges = undefined; cnnLayerRangesStore.subscribe( value => {cnnLayerRanges = value;} ) let cnnLayerMinMax = undefined; cnnLayerMinMaxStore.subscribe( value => {cnnLayerMinMax = value;} ) let detailedMode = undefined; detailedModeStore.subscribe( value => {detailedMode = value;} ) /** * Use bounded d3 data to draw one canvas * @param {object} d d3 data * @param {index} i d3 data index * @param {[object]} g d3 group * @param {number} range color range map (max - min) */ export const drawOutput = (d, i, g, range) => { let image = g[i]; let colorScale = layerColorScales[d.type]; if (d.type === 'input') { colorScale = colorScale[d.index]; } // Set up a second convas in order to resize image let imageLength = d.output.length === undefined ? 1 : d.output.length; let bufferCanvas = document.createElement("canvas"); let bufferContext = bufferCanvas.getContext("2d"); bufferCanvas.width = imageLength; bufferCanvas.height = imageLength; // Fill image pixel array let imageSingle = bufferContext.getImageData(0, 0, imageLength, imageLength); let imageSingleArray = imageSingle.data; if (imageLength === 1) { imageSingleArray[0] = d.output; } else { for (let i = 0; i < imageSingleArray.length; i+=4) { let pixeIndex = Math.floor(i / 4); let row = Math.floor(pixeIndex / imageLength); let column = pixeIndex % imageLength; let color = undefined; if (d.type === 'input' || d.type === 'fc' ) { color = d3.rgb(colorScale(1 - d.output[row][column])) } else { color = d3.rgb(colorScale((d.output[row][column] + range / 2) / range)); } imageSingleArray[i] = color.r; imageSingleArray[i + 1] = color.g; imageSingleArray[i + 2] = color.b; imageSingleArray[i + 3] = 255; } } // canvas.toDataURL() only exports image in 96 DPI, so we can hack it to have // higher DPI by rescaling the image using canvas magic let largeCanvas = document.createElement('canvas'); largeCanvas.width = nodeLength * 3; largeCanvas.height = nodeLength * 3; let largeCanvasContext = largeCanvas.getContext('2d'); // Use drawImage to resize the original pixel array, and put the new image // (canvas) into corresponding canvas bufferContext.putImageData(imageSingle, 0, 0); largeCanvasContext.drawImage(bufferCanvas, 0, 0, imageLength, imageLength, 0, 0, nodeLength * 3, nodeLength * 3); let imageDataURL = largeCanvas.toDataURL(); d3.select(image).attr('xlink:href', imageDataURL); // Destory the buffer canvas bufferCanvas.remove(); largeCanvas.remove(); } /** * Draw bar chart to encode the output value * @param {object} d d3 data * @param {index} i d3 data index * @param {[object]} g d3 group * @param {function} scale map value to length */ const drawOutputScore = (d, i, g, scale) => { let group = d3.select(g[i]); group.select('rect.output-rect') .transition('output') .delay(500) .duration(800) .ease(d3.easeCubicIn) .attr('width', scale(d.output)) } export const drawCustomImage = (image, inputLayer) => { let imageWidth = image.width; // Set up a second convas in order to resize image let imageLength = inputLayer[0].output.length; let bufferCanvas = document.createElement("canvas"); let bufferContext = bufferCanvas.getContext("2d"); bufferCanvas.width = imageLength; bufferCanvas.height = imageLength; // Fill image pixel array let imageSingle = bufferContext.getImageData(0, 0, imageLength, imageLength); let imageSingleArray = imageSingle.data; for (let i = 0; i < imageSingleArray.length; i+=4) { let pixeIndex = Math.floor(i / 4); let row = Math.floor(pixeIndex / imageLength); let column = pixeIndex % imageLength; let red = inputLayer[0].output[row][column]; let green = inputLayer[1].output[row][column]; let blue = inputLayer[2].output[row][column]; imageSingleArray[i] = red * 255; imageSingleArray[i + 1] = green * 255; imageSingleArray[i + 2] = blue * 255; imageSingleArray[i + 3] = 255; } // canvas.toDataURL() only exports image in 96 DPI, so we can hack it to have // higher DPI by rescaling the image using canvas magic let largeCanvas = document.createElement('canvas'); largeCanvas.width = imageWidth * 3; largeCanvas.height = imageWidth * 3; let largeCanvasContext = largeCanvas.getContext('2d'); // Use drawImage to resize the original pixel array, and put the new image // (canvas) into corresponding canvas bufferContext.putImageData(imageSingle, 0, 0); largeCanvasContext.drawImage(bufferCanvas, 0, 0, imageLength, imageLength, 0, 0, imageWidth * 3, imageWidth * 3); let imageDataURL = largeCanvas.toDataURL(); // d3.select(image).attr('xlink:href', imageDataURL); image.src = imageDataURL; // Destory the buffer canvas bufferCanvas.remove(); largeCanvas.remove(); } /** * Create color gradient for the legend * @param {[object]} g d3 group * @param {function} colorScale Colormap * @param {string} gradientName Label for gradient def * @param {number} min Min of legend value * @param {number} max Max of legend value */ const getLegendGradient = (g, colorScale, gradientName, min, max) => { if (min === undefined) { min = 0; } if (max === undefined) { max = 1; } let gradient = g.append('defs') .append('svg:linearGradient') .attr('id', `${gradientName}`) .attr('x1', '0%') .attr('y1', '100%') .attr('x2', '100%') .attr('y2', '100%') .attr('spreadMethod', 'pad'); let interpolation = 10 for (let i = 0; i < interpolation; i++) { let curProgress = i / (interpolation - 1); let curColor = colorScale(curProgress * (max - min) + min); gradient.append('stop') .attr('offset', `${curProgress * 100}%`) .attr('stop-color', curColor) .attr('stop-opacity', 1); } } /** * Draw all legends * @param {object} legends Parent group * @param {number} legendHeight Height of the legend element */ const drawLegends = (legends, legendHeight) => { // Add local legends for (let i = 0; i < 2; i++){ let start = 1 + i * 5; let range1 = cnnLayerRanges.local[start]; let range2 = cnnLayerRanges.local[start + 2]; let localLegendScale1 = d3.scaleLinear() .range([0, 2 * nodeLength + hSpaceAroundGap - 1.2]) .domain([-range1 / 2, range1 / 2]); let localLegendScale2 = d3.scaleLinear() .range([0, 3 * nodeLength + 2 * hSpaceAroundGap - 1.2]) .domain([-range2 / 2, range2 / 2]); let localLegendAxis1 = d3.axisBottom() .scale(localLegendScale1) .tickFormat(d3.format('.2f')) .tickValues([-range1 / 2, 0, range1 / 2]); let localLegendAxis2 = d3.axisBottom() .scale(localLegendScale2) .tickFormat(d3.format('.2f')) .tickValues([-range2 / 2, 0, range2 / 2]); let localLegend1 = legends.append('g') .attr('class', 'legend local-legend') .attr('id', `local-legend-${i}-1`) .classed('hidden', !detailedMode || selectedScaleLevel !== 'local') .attr('transform', `translate(${nodeCoordinate[start][0].x}, ${0})`); localLegend1.append('g') .attr('transform', `translate(0, ${legendHeight - 3})`) .call(localLegendAxis1) localLegend1.append('rect') .attr('width', 2 * nodeLength + hSpaceAroundGap) .attr('height', legendHeight) .style('fill', 'url(#convGradient)'); let localLegend2 = legends.append('g') .attr('class', 'legend local-legend') .attr('id', `local-legend-${i}-2`) .classed('hidden', !detailedMode || selectedScaleLevel !== 'local') .attr('transform', `translate(${nodeCoordinate[start + 2][0].x}, ${0})`); localLegend2.append('g') .attr('transform', `translate(0, ${legendHeight - 3})`) .call(localLegendAxis2) localLegend2.append('rect') .attr('width', 3 * nodeLength + 2 * hSpaceAroundGap) .attr('height', legendHeight) .style('fill', 'url(#convGradient)'); } // Add module legends for (let i = 0; i < 2; i++){ let start = 1 + i * 5; let range = cnnLayerRanges.module[start]; let moduleLegendScale = d3.scaleLinear() .range([0, 5 * nodeLength + 3 * hSpaceAroundGap + 1 * hSpaceAroundGap * gapRatio - 1.2]) .domain([-range / 2, range / 2]); let moduleLegendAxis = d3.axisBottom() .scale(moduleLegendScale) .tickFormat(d3.format('.2f')) .tickValues([-range / 2, -(range / 4), 0, range / 4, range / 2]); let moduleLegend = legends.append('g') .attr('class', 'legend module-legend') .attr('id', `module-legend-${i}`) .classed('hidden', !detailedMode || selectedScaleLevel !== 'module') .attr('transform', `translate(${nodeCoordinate[start][0].x}, ${0})`); moduleLegend.append('g') .attr('transform', `translate(0, ${legendHeight - 3})`) .call(moduleLegendAxis) moduleLegend.append('rect') .attr('width', 5 * nodeLength + 3 * hSpaceAroundGap + 1 * hSpaceAroundGap * gapRatio) .attr('height', legendHeight) .style('fill', 'url(#convGradient)'); } // Add global legends let start = 1; let range = cnnLayerRanges.global[start]; let globalLegendScale = d3.scaleLinear() .range([0, 10 * nodeLength + 6 * hSpaceAroundGap + 3 * hSpaceAroundGap * gapRatio - 1.2]) .domain([-range / 2, range / 2]); let globalLegendAxis = d3.axisBottom() .scale(globalLegendScale) .tickFormat(d3.format('.2f')) .tickValues([-range / 2, -(range / 4), 0, range / 4, range / 2]); let globalLegend = legends.append('g') .attr('class', 'legend global-legend') .attr('id', 'global-legend') .classed('hidden', !detailedMode || selectedScaleLevel !== 'global') .attr('transform', `translate(${nodeCoordinate[start][0].x}, ${0})`); globalLegend.append('g') .attr('transform', `translate(0, ${legendHeight - 3})`) .call(globalLegendAxis) globalLegend.append('rect') .attr('width', 10 * nodeLength + 6 * hSpaceAroundGap + 3 * hSpaceAroundGap * gapRatio) .attr('height', legendHeight) .style('fill', 'url(#convGradient)'); // Add output legend let outputRectScale = d3.scaleLinear() .domain(cnnLayerRanges.output) .range([0, nodeLength - 1.2]); let outputLegendAxis = d3.axisBottom() .scale(outputRectScale) .tickFormat(d3.format('.1f')) .tickValues([0, cnnLayerRanges.output[1]]) let outputLegend = legends.append('g') .attr('class', 'legend output-legend') .attr('id', 'output-legend') .classed('hidden', !detailedMode) .attr('transform', `translate(${nodeCoordinate[11][0].x}, ${0})`); outputLegend.append('g') .attr('transform', `translate(0, ${legendHeight - 3})`) .call(outputLegendAxis); outputLegend.append('rect') .attr('width', nodeLength) .attr('height', legendHeight) .style('fill', 'gray'); // Add input image legend let inputScale = d3.scaleLinear() .range([0, nodeLength - 1.2]) .domain([0, 1]); let inputLegendAxis = d3.axisBottom() .scale(inputScale) .tickFormat(d3.format('.1f')) .tickValues([0, 0.5, 1]); let inputLegend = legends.append('g') .attr('class', 'legend input-legend') .classed('hidden', !detailedMode) .attr('transform', `translate(${nodeCoordinate[0][0].x}, ${0})`); inputLegend.append('g') .attr('transform', `translate(0, ${legendHeight - 3})`) .call(inputLegendAxis); inputLegend.append('rect') .attr('x', 0.3) .attr('width', nodeLength - 0.3) .attr('height', legendHeight) .attr('transform', `rotate(180, ${nodeLength/2}, ${legendHeight/2})`) .style('stroke', 'rgb(20, 20, 20)') .style('stroke-width', 0.3) .style('fill', 'url(#inputGradient)'); } /** * Draw the overview * @param {number} width Width of the cnn group * @param {number} height Height of the cnn group * @param {object} cnnGroup Group to appen cnn elements to * @param {function} nodeMouseOverHandler Callback func for mouseOver * @param {function} nodeMouseLeaveHandler Callback func for mouseLeave * @param {function} nodeClickHandler Callback func for click */ export const drawCNN = (width, height, cnnGroup, nodeMouseOverHandler, nodeMouseLeaveHandler, nodeClickHandler) => { // Draw the CNN // There are 8 short gaps and 5 long gaps hSpaceAroundGap = (width - nodeLength * numLayers) / (8 + 5 * gapRatio); hSpaceAroundGapStore.set(hSpaceAroundGap); let leftAccuumulatedSpace = 0; // Iterate through the cnn to draw nodes in each layer for (let l = 0; l < cnn.length; l++) { let curLayer = cnn[l]; let isOutput = curLayer[0].layerName === 'output'; nodeCoordinate.push([]); // Compute the x coordinate of the whole layer // Output layer and conv layer has long gaps if (isOutput || curLayer[0].type === 'conv') { leftAccuumulatedSpace += hSpaceAroundGap * gapRatio; } else { leftAccuumulatedSpace += hSpaceAroundGap; } // All nodes share the same x coordiante (left in div style) let left = leftAccuumulatedSpace; let layerGroup = cnnGroup.append('g') .attr('class', 'cnn-layer-group') .attr('id', `cnn-layer-group-${l}`); vSpaceAroundGap = (height - nodeLength * curLayer.length) / (curLayer.length + 1); vSpaceAroundGapStore.set(vSpaceAroundGap); let nodeGroups = layerGroup.selectAll('g.node-group') .data(curLayer, d => d.index) .enter() .append('g') .attr('class', 'node-group') .style('cursor', 'pointer') .style('pointer-events', 'all') .on('click', nodeClickHandler) .on('mouseover', nodeMouseOverHandler) .on('mouseleave', nodeMouseLeaveHandler) .classed('node-output', isOutput) .attr('id', (d, i) => { // Compute the coordinate // Not using transform on the group object because of a decade old // bug on webkit (safari) // https://bugs.webkit.org/show_bug.cgi?id=23113 let top = i * nodeLength + (i + 1) * vSpaceAroundGap; top += svgPaddings.top; nodeCoordinate[l].push({x: left, y: top}); return `layer-${l}-node-${i}` }); // Overwrite the mouseover and mouseleave function for output nodes to show // hover info in the UI layerGroup.selectAll('g.node-output') .on('mouseover', (d, i, g) => { nodeMouseOverHandler(d, i, g); hoverInfoStore.set( {show: true, text: `Output value: ${formater(d.output)}`} ); }) .on('mouseleave', (d, i, g) => { nodeMouseLeaveHandler(d, i, g); hoverInfoStore.set( {show: false, text: `Output value: ${formater(d.output)}`} ); }); if (curLayer[0].layerName !== 'output') { // Embed raster image in these groups nodeGroups.append('image') .attr('class', 'node-image') .attr('width', nodeLength) .attr('height', nodeLength) .attr('x', left) .attr('y', (d, i) => nodeCoordinate[l][i].y); // Add a rectangle to show the border nodeGroups.append('rect') .attr('class', 'bounding') .attr('width', nodeLength) .attr('height', nodeLength) .attr('x', left) .attr('y', (d, i) => nodeCoordinate[l][i].y) .style('fill', 'none') .style('stroke', 'gray') .style('stroke-width', 1) .classed('hidden', true); } else { nodeGroups.append('rect') .attr('class', 'output-rect') .attr('x', left) .attr('y', (d, i) => nodeCoordinate[l][i].y + nodeLength / 2 + 8) .attr('height', nodeLength / 4) .attr('width', 0) .style('fill', 'gray'); nodeGroups.append('text') .attr('class', 'output-text') .attr('x', left) .attr('y', (d, i) => nodeCoordinate[l][i].y + nodeLength / 2) .style('dominant-baseline', 'middle') .style('font-size', '11px') .style('fill', 'black') .style('opacity', 0.5) .text((d, i) => classLists[i]); // Add annotation text to tell readers the exact output probability // nodeGroups.append('text') // .attr('class', 'annotation-text') // .attr('id', (d, i) => `output-prob-${i}`) // .attr('x', left) // .attr('y', (d, i) => nodeCoordinate[l][i].y + 10) // .text(d => `(${d3.format('.4f')(d.output)})`); } leftAccuumulatedSpace += nodeLength; } // Share the nodeCoordinate nodeCoordinateStore.set(nodeCoordinate) // Compute the scale of the output score width (mapping the the node // width to the max output score) let outputRectScale = d3.scaleLinear() .domain(cnnLayerRanges.output) .range([0, nodeLength]); // Draw the canvas for (let l = 0; l < cnn.length; l++) { let range = cnnLayerRanges[selectedScaleLevel][l]; svg.select(`g#cnn-layer-group-${l}`) .selectAll('image.node-image') .each((d, i, g) => drawOutput(d, i, g, range)); } svg.selectAll('g.node-output').each( (d, i, g) => drawOutputScore(d, i, g, outputRectScale) ); // Add layer label let layerNames = cnn.map(d => { if (d[0].layerName === 'output') { return { name: d[0].layerName, dimension: `(${d.length})` } } else { return { name: d[0].layerName, dimension: `(${d[0].output.length}, ${d[0].output.length}, ${d.length})` } } }); let svgHeight = Number(d3.select('#cnn-svg').style('height').replace('px', '')) + 150; let scroll = new SmoothScroll('a[href*="#"]', {offset: -svgHeight}); let detailedLabels = svg.selectAll('g.layer-detailed-label') .data(layerNames) .enter() .append('g') .attr('class', 'layer-detailed-label') .attr('id', (d, i) => `layer-detailed-label-${i}`) .classed('hidden', !detailedMode) .attr('transform', (d, i) => { let x = nodeCoordinate[i][0].x + nodeLength / 2; let y = (svgPaddings.top + vSpaceAroundGap) / 2 - 6; return `translate(${x}, ${y})`; }) .style('cursor', d => d.name.includes('output') ? 'default' : 'help') .on('click', (d) => { let target = ''; if (d.name.includes('conv')) { target = 'convolution' } if (d.name.includes('relu')) { target = 'relu' } if (d.name.includes('max_pool')) { target = 'pooling'} if (d.name.includes('input')) { target = 'input'} // Scroll to a article element let anchor = document.querySelector(`#article-${target}`); scroll.animateScroll(anchor); }); detailedLabels.append('title') .text('Move to article section'); detailedLabels.append('text') .style('opacity', 0.7) .style('dominant-baseline', 'middle') .append('tspan') .style('font-size', '12px') .text(d => d.name) .append('tspan') .style('font-size', '8px') .style('font-weight', 'normal') .attr('x', 0) .attr('dy', '1.5em') .text(d => d.dimension); let labels = svg.selectAll('g.layer-label') .data(layerNames) .enter() .append('g') .attr('class', 'layer-label') .attr('id', (d, i) => `layer-label-${i}`) .classed('hidden', detailedMode) .attr('transform', (d, i) => { let x = nodeCoordinate[i][0].x + nodeLength / 2; let y = (svgPaddings.top + vSpaceAroundGap) / 2 + 5; return `translate(${x}, ${y})`; }) .style('cursor', d => d.name.includes('output') ? 'default' : 'help') .on('click', (d) => { let target = ''; if (d.name.includes('conv')) { target = 'convolution' } if (d.name.includes('relu')) { target = 'relu' } if (d.name.includes('max_pool')) { target = 'pooling'} if (d.name.includes('input')) { target = 'input'} // Scroll to a article element let anchor = document.querySelector(`#article-${target}`); scroll.animateScroll(anchor); }); labels.append('title') .text('Move to article section'); labels.append('text') .style('dominant-baseline', 'middle') .style('opacity', 0.8) .text(d => { if (d.name.includes('conv')) { return 'conv' } if (d.name.includes('relu')) { return 'relu' } if (d.name.includes('max_pool')) { return 'max_pool'} return d.name }); // Add layer color scale legends getLegendGradient(svg, layerColorScales.conv, 'convGradient'); getLegendGradient(svg, layerColorScales.input[0], 'inputGradient'); let legendHeight = 5; let legends = svg.append('g') .attr('class', 'color-legend') .attr('transform', `translate(${0}, ${ svgPaddings.top + vSpaceAroundGap * (10) + vSpaceAroundGap + nodeLength * 10 })`); drawLegends(legends, legendHeight); // Add edges between nodes let linkGen = d3.linkHorizontal() .x(d => d.x) .y(d => d.y); let linkData = getLinkData(nodeCoordinate, cnn); let edgeGroup = cnnGroup.append('g') .attr('class', 'edge-group'); edgeGroup.selectAll('path.edge') .data(linkData) .enter() .append('path') .attr('class', d => `edge edge-${d.targetLayerIndex} edge-${d.targetLayerIndex}-${d.targetNodeIndex}`) .attr('id', d => `edge-${d.targetLayerIndex}-${d.targetNodeIndex}-${d.sourceNodeIndex}`) .attr('d', d => linkGen({source: d.source, target: d.target})) .style('fill', 'none') .style('stroke-width', edgeStrokeWidth) .style('opacity', edgeOpacity) .style('stroke', edgeInitColor); // Add input channel annotations let inputAnnotation = cnnGroup.append('g') .attr('class', 'input-annotation'); let redChannel = inputAnnotation.append('text') .attr('x', nodeCoordinate[0][0].x + nodeLength / 2) .attr('y', nodeCoordinate[0][0].y + nodeLength + 5) .attr('class', 'annotation-text') .style('dominant-baseline', 'hanging') .style('text-anchor', 'middle'); redChannel.append('tspan') .style('dominant-baseline', 'hanging') .style('fill', '#C95E67') .text('Red'); redChannel.append('tspan') .style('dominant-baseline', 'hanging') .text(' channel'); inputAnnotation.append('text') .attr('x', nodeCoordinate[0][1].x + nodeLength / 2) .attr('y', nodeCoordinate[0][1].y + nodeLength + 5) .attr('class', 'annotation-text') .style('dominant-baseline', 'hanging') .style('text-anchor', 'middle') .style('fill', '#3DB665') .text('Green'); inputAnnotation.append('text') .attr('x', nodeCoordinate[0][2].x + nodeLength / 2) .attr('y', nodeCoordinate[0][2].y + nodeLength + 5) .attr('class', 'annotation-text') .style('dominant-baseline', 'hanging') .style('text-anchor', 'middle') .style('fill', '#3F7FBC') .text('Blue'); } /** * Update canvas values when user changes input image */ export const updateCNN = () => { // Compute the scale of the output score width (mapping the the node // width to the max output score) let outputRectScale = d3.scaleLinear() .domain(cnnLayerRanges.output) .range([0, nodeLength]); // Rebind the cnn data to layer groups layer by layer for (let l = 0; l < cnn.length; l++) { let curLayer = cnn[l]; let range = cnnLayerRanges[selectedScaleLevel][l]; let layerGroup = svg.select(`g#cnn-layer-group-${l}`); let nodeGroups = layerGroup.selectAll('g.node-group') .data(curLayer); if (l < cnn.length - 1) { // Redraw the canvas and output node nodeGroups.transition('disappear') .duration(300) .ease(d3.easeCubicOut) .style('opacity', 0) .on('end', function() { d3.select(this) .select('image.node-image') .each((d, i, g) => drawOutput(d, i, g, range)); d3.select(this).transition('appear') .duration(700) .ease(d3.easeCubicIn) .style('opacity', 1); }); } else { nodeGroups.each( (d, i, g) => drawOutputScore(d, i, g, outputRectScale) ); } } // Update the color scale legend // Local legends for (let i = 0; i < 2; i++){ let start = 1 + i * 5; let range1 = cnnLayerRanges.local[start]; let range2 = cnnLayerRanges.local[start + 2]; let localLegendScale1 = d3.scaleLinear() .range([0, 2 * nodeLength + hSpaceAroundGap]) .domain([-range1 / 2, range1 / 2]); let localLegendScale2 = d3.scaleLinear() .range([0, 3 * nodeLength + 2 * hSpaceAroundGap]) .domain([-range2 / 2, range2 / 2]); let localLegendAxis1 = d3.axisBottom() .scale(localLegendScale1) .tickFormat(d3.format('.2f')) .tickValues([-range1 / 2, 0, range1 / 2]); let localLegendAxis2 = d3.axisBottom() .scale(localLegendScale2) .tickFormat(d3.format('.2f')) .tickValues([-range2 / 2, 0, range2 / 2]); svg.select(`g#local-legend-${i}-1`).select('g').call(localLegendAxis1); svg.select(`g#local-legend-${i}-2`).select('g').call(localLegendAxis2); } // Module legend for (let i = 0; i < 2; i++){ let start = 1 + i * 5; let range = cnnLayerRanges.local[start]; let moduleLegendScale = d3.scaleLinear() .range([0, 5 * nodeLength + 3 * hSpaceAroundGap + 1 * hSpaceAroundGap * gapRatio - 1.2]) .domain([-range, range]); let moduleLegendAxis = d3.axisBottom() .scale(moduleLegendScale) .tickFormat(d3.format('.2f')) .tickValues([-range, -(range / 2), 0, range/2, range]); svg.select(`g#module-legend-${i}`).select('g').call(moduleLegendAxis); } // Global legend let start = 1; let range = cnnLayerRanges.global[start]; let globalLegendScale = d3.scaleLinear() .range([0, 10 * nodeLength + 6 * hSpaceAroundGap + 3 * hSpaceAroundGap * gapRatio - 1.2]) .domain([-range, range]); let globalLegendAxis = d3.axisBottom() .scale(globalLegendScale) .tickFormat(d3.format('.2f')) .tickValues([-range, -(range / 2), 0, range/2, range]); svg.select(`g#global-legend`).select('g').call(globalLegendAxis); // Output legend let outputLegendAxis = d3.axisBottom() .scale(outputRectScale) .tickFormat(d3.format('.1f')) .tickValues([0, cnnLayerRanges.output[1]]); svg.select('g#output-legend').select('g').call(outputLegendAxis); } /** * Update the ranges for current CNN layers */ export const updateCNNLayerRanges = () => { // Iterate through all nodes to find a output ranges for each layer let cnnLayerRangesLocal = [1]; let curRange = undefined; // Also track the min/max of each layer (avoid computing during intermediate // layer) cnnLayerMinMax = []; for (let l = 0; l < cnn.length - 1; l++) { let curLayer = cnn[l]; // Compute the min max let outputExtents = curLayer.map(l => getExtent(l.output)); let aggregatedExtent = outputExtents.reduce((acc, cur) => { return [Math.min(acc[0], cur[0]), Math.max(acc[1], cur[1])]; }) cnnLayerMinMax.push({min: aggregatedExtent[0], max: aggregatedExtent[1]}); // conv layer refreshes curRange counting if (curLayer[0].type === 'conv' || curLayer[0].type === 'fc') { aggregatedExtent = aggregatedExtent.map(Math.abs); // Plus 0.1 to offset the rounding error (avoid black color) curRange = 2 * (0.1 + Math.round(Math.max(...aggregatedExtent) * 1000) / 1000); } if (curRange !== undefined){ cnnLayerRangesLocal.push(curRange); } } // Finally, add the output layer range cnnLayerRangesLocal.push(1); cnnLayerMinMax.push({min: 0, max: 1}); // Support different levels of scales (1) lcoal, (2) component, (3) global let cnnLayerRangesComponent = [1]; let numOfComponent = (numLayers - 2) / 5; for (let i = 0; i < numOfComponent; i++) { let curArray = cnnLayerRangesLocal.slice(1 + 5 * i, 1 + 5 * i + 5); let maxRange = Math.max(...curArray); for (let j = 0; j < 5; j++) { cnnLayerRangesComponent.push(maxRange); } } cnnLayerRangesComponent.push(1); let cnnLayerRangesGlobal = [1]; let maxRange = Math.max(...cnnLayerRangesLocal.slice(1, cnnLayerRangesLocal.length - 1)); for (let i = 0; i < numLayers - 2; i++) { cnnLayerRangesGlobal.push(maxRange); } cnnLayerRangesGlobal.push(1); // Update the ranges dictionary cnnLayerRanges.local = cnnLayerRangesLocal; cnnLayerRanges.module = cnnLayerRangesComponent; cnnLayerRanges.global = cnnLayerRangesGlobal; cnnLayerRanges.output = [0, d3.max(cnn[cnn.length - 1].map(d => d.output))]; cnnLayerRangesStore.set(cnnLayerRanges); cnnLayerMinMaxStore.set(cnnLayerMinMax); } ================================================ FILE: src/stores.js ================================================ import { writable } from 'svelte/store'; export const cnnStore = writable([]); export const svgStore = writable(undefined); export const vSpaceAroundGapStore = writable(undefined); export const hSpaceAroundGapStore = writable(undefined); export const nodeCoordinateStore = writable([]); export const selectedScaleLevelStore = writable(undefined); export const cnnLayerRangesStore = writable({}); export const cnnLayerMinMaxStore = writable([]); export const needRedrawStore = writable([undefined, undefined]); export const detailedModeStore = writable(true); export const shouldIntermediateAnimateStore = writable(false); export const isInSoftmaxStore = writable(false); export const softmaxDetailViewStore = writable({}); export const allowsSoftmaxAnimationStore = writable(false); export const hoverInfoStore = writable({}); export const modalStore = writable({}); export const intermediateLayerPositionStore = writable({}); ================================================ FILE: src/utils/cnn-tf.js ================================================ /* global tf */ // Network input image size const networkInputSize = 64; // Enum of node types const nodeType = { INPUT: 'input', CONV: 'conv', POOL: 'pool', RELU: 'relu', FC: 'fc', FLATTEN: 'flatten' } class Node { /** * Class structure for each neuron node. * * @param {string} layerName Name of the node's layer. * @param {int} index Index of this node in its layer. * @param {string} type Node type {input, conv, pool, relu, fc}. * @param {number} bias The bias assocated to this node. * @param {number[]} output Output of this node. */ constructor(layerName, index, type, bias, output) { this.layerName = layerName; this.index = index; this.type = type; this.bias = bias; this.output = output; // Weights are stored in the links this.inputLinks = []; this.outputLinks = []; } } class Link { /** * Class structure for each link between two nodes. * * @param {Node} source Source node. * @param {Node} dest Target node. * @param {number} weight Weight associated to this link. It can be a number, * 1D array, or 2D array. */ constructor(source, dest, weight) { this.source = source; this.dest = dest; this.weight = weight; } } /** * Construct a CNN with given extracted outputs from every layer. * * @param {number[][]} allOutputs Array of outputs for each layer. * allOutputs[i][j] is the output for layer i node j. * @param {Model} model Loaded tf.js model. * @param {Tensor} inputImageTensor Loaded input image tensor. */ const constructCNNFromOutputs = (allOutputs, model, inputImageTensor) => { let cnn = []; // Add the first layer (input layer) let inputLayer = []; let inputShape = model.layers[0].batchInputShape.slice(1); let inputImageArray = inputImageTensor.transpose([2, 0, 1]).arraySync(); // First layer's three nodes' outputs are the channels of inputImageArray for (let i = 0; i < inputShape[2]; i++) { let node = new Node('input', i, nodeType.INPUT, 0, inputImageArray[i]); inputLayer.push(node); } cnn.push(inputLayer); let curLayerIndex = 1; for (let l = 0; l < model.layers.length; l++) { let layer = model.layers[l]; // Get the current output let outputs = allOutputs[l].squeeze(); outputs = outputs.arraySync(); let curLayerNodes = []; let curLayerType; // Identify layer type based on the layer name if (layer.name.includes('conv')) { curLayerType = nodeType.CONV; } else if (layer.name.includes('pool')) { curLayerType = nodeType.POOL; } else if (layer.name.includes('relu')) { curLayerType = nodeType.RELU; } else if (layer.name.includes('output')) { curLayerType = nodeType.FC; } else if (layer.name.includes('flatten')) { curLayerType = nodeType.FLATTEN; } else { console.log('Find unknown type'); } // Construct this layer based on its layer type switch (curLayerType) { case nodeType.CONV: { let biases = layer.bias.val.arraySync(); // The new order is [output_depth, input_depth, height, width] let weights = layer.kernel.val.transpose([3, 2, 0, 1]).arraySync(); // Add nodes into this layer for (let i = 0; i < outputs.length; i++) { let node = new Node(layer.name, i, curLayerType, biases[i], outputs[i]); // Connect this node to all previous nodes (create links) // CONV layers have weights in links. Links are one-to-multiple. for (let j = 0; j < cnn[curLayerIndex - 1].length; j++) { let preNode = cnn[curLayerIndex - 1][j]; let curLink = new Link(preNode, node, weights[i][j]); preNode.outputLinks.push(curLink); node.inputLinks.push(curLink); } curLayerNodes.push(node); } break; } case nodeType.FC: { let biases = layer.bias.val.arraySync(); // The new order is [output_depth, input_depth] let weights = layer.kernel.val.transpose([1, 0]).arraySync(); // Add nodes into this layer for (let i = 0; i < outputs.length; i++) { let node = new Node(layer.name, i, curLayerType, biases[i], outputs[i]); // Connect this node to all previous nodes (create links) // FC layers have weights in links. Links are one-to-multiple. // Since we are visualizing the logit values, we need to track // the raw value before softmax let curLogit = 0; for (let j = 0; j < cnn[curLayerIndex - 1].length; j++) { let preNode = cnn[curLayerIndex - 1][j]; let curLink = new Link(preNode, node, weights[i][j]); preNode.outputLinks.push(curLink); node.inputLinks.push(curLink); curLogit += preNode.output * weights[i][j]; } curLogit += biases[i]; node.logit = curLogit; curLayerNodes.push(node); } // Sort flatten layer based on the node TF index cnn[curLayerIndex - 1].sort((a, b) => a.realIndex - b.realIndex); break; } case nodeType.RELU: case nodeType.POOL: { // RELU and POOL have no bias nor weight let bias = 0; let weight = null; // Add nodes into this layer for (let i = 0; i < outputs.length; i++) { let node = new Node(layer.name, i, curLayerType, bias, outputs[i]); // RELU and POOL layers have no weights. Links are one-to-one let preNode = cnn[curLayerIndex - 1][i]; let link = new Link(preNode, node, weight); preNode.outputLinks.push(link); node.inputLinks.push(link); curLayerNodes.push(node); } break; } case nodeType.FLATTEN: { // Flatten layer has no bias nor weights. let bias = 0; for (let i = 0; i < outputs.length; i++) { // Flatten layer has no weights. Links are multiple-to-one. // Use dummy weights to store the corresponding entry in the previsou // node as (row, column) // The flatten() in tf2.keras has order: channel -> row -> column let preNodeWidth = cnn[curLayerIndex - 1][0].output.length, preNodeNum = cnn[curLayerIndex - 1].length, preNodeIndex = i % preNodeNum, preNodeRow = Math.floor(Math.floor(i / preNodeNum) / preNodeWidth), preNodeCol = Math.floor(i / preNodeNum) % preNodeWidth, // Use channel, row, colume to compute the real index with order // row -> column -> channel curNodeRealIndex = preNodeIndex * (preNodeWidth * preNodeWidth) + preNodeRow * preNodeWidth + preNodeCol; let node = new Node(layer.name, i, curLayerType, bias, outputs[i]); // TF uses the (i) index for computation, but the real order should // be (curNodeRealIndex). We will sort the nodes using the real order // after we compute the logits in the output layer. node.realIndex = curNodeRealIndex; let link = new Link(cnn[curLayerIndex - 1][preNodeIndex], node, [preNodeRow, preNodeCol]); cnn[curLayerIndex - 1][preNodeIndex].outputLinks.push(link); node.inputLinks.push(link); curLayerNodes.push(node); } // Sort flatten layer based on the node TF index curLayerNodes.sort((a, b) => a.index - b.index); break; } default: console.error('Encounter unknown layer type'); break; } // Add current layer to the NN cnn.push(curLayerNodes); curLayerIndex++; } return cnn; } /** * Construct a CNN with given model and input. * * @param {string} inputImageFile filename of input image. * @param {Model} model Loaded tf.js model. */ export const constructCNN = async (inputImageFile, model) => { // Load the image file let inputImageTensor = await getInputImageArray(inputImageFile, true); // Need to feed the model with a batch let inputImageTensorBatch = tf.stack([inputImageTensor]); // To get intermediate layer outputs, we will iterate through all layers in // the model, and sequencially apply transformations. let preTensor = inputImageTensorBatch; let outputs = []; // Iterate through all layers, and build one model with that layer as output for (let l = 0; l < model.layers.length; l++) { let curTensor = model.layers[l].apply(preTensor); // Record the output tensor // Because there is only one element in the batch, we use squeeze() // We also want to use CHW order here let output = curTensor.squeeze(); if (output.shape.length === 3) { output = output.transpose([2, 0, 1]); } outputs.push(output); // Update preTensor for next nesting iteration preTensor = curTensor; } let cnn = constructCNNFromOutputs(outputs, model, inputImageTensor); return cnn; } // Helper functions /** * Crop the largest central square of size 64x64x3 of a 3d array. * * @param {[int8]} arr array that requires cropping and padding (if a 64x64 crop * is not present) * @returns 64x64x3 array */ const cropCentralSquare = (arr) => { let width = arr.length; let height = arr[0].length; let croppedArray; // Crop largest square from image if the image is smaller than 64x64 and pad the // cropped image. if (width < networkInputSize || height < networkInputSize) { // TODO(robert): Finish the padding logic. Pushing now for Omar to work on when he is ready. let cropDimensions = Math.min(width, height); let startXIdx = Math.floor(width / 2) - (cropDimensions / 2); let startYIdx = Math.floor(height / 2) - (cropDimensions / 2); let unpaddedSubarray = arr.slice(startXIdx, startXIdx + cropDimensions).map(i => i.slice(startYIdx, startYIdx + cropDimensions)); } else { let startXIdx = Math.floor(width / 2) - Math.floor(networkInputSize / 2); let startYIdx = Math.floor(height / 2) - Math.floor(networkInputSize / 2); croppedArray = arr.slice(startXIdx, startXIdx + networkInputSize).map(i => i.slice(startYIdx, startYIdx + networkInputSize)); } return croppedArray; } /** * Convert canvas image data into a 3D tensor with dimension [height, width, 3]. * Recall that tensorflow uses NHWC order (batch, height, width, channel). * Each pixel is in 0-255 scale. * * @param {[int8]} imageData Canvas image data * @param {int} width Canvas image width * @param {int} height Canvas image height */ const imageDataTo3DTensor = (imageData, width, height, normalize=true) => { // Create array placeholder for the 3d array let imageArray = tf.fill([width, height, 3], 0).arraySync(); // Iterate through the data to fill out channel arrays above for (let i = 0; i < imageData.length; i++) { let pixelIndex = Math.floor(i / 4), channelIndex = i % 4, row = width === height ? Math.floor(pixelIndex / width) : pixelIndex % width, column = width === height ? pixelIndex % width : Math.floor(pixelIndex / width); if (channelIndex < 3) { let curEntry = imageData[i]; // Normalize the original pixel value from [0, 255] to [0, 1] if (normalize) { curEntry /= 255; } imageArray[row][column][channelIndex] = curEntry; } } // If the image is not 64x64, crop and or pad the image appropriately. if (width != networkInputSize && height != networkInputSize) { imageArray = cropCentralSquare(imageArray) } let tensor = tf.tensor3d(imageArray); return tensor; } /** * Get the 3D pixel value array of the given image file. * * @param {string} imgFile File path to the image file * @returns A promise with the corresponding 3D array */ const getInputImageArray = (imgFile, normalize=true) => { let canvas = document.createElement('canvas'); canvas.style.cssText = 'display:none;'; document.getElementsByTagName('body')[0].appendChild(canvas); let context = canvas.getContext('2d'); return new Promise((resolve, reject) => { let inputImage = new Image(); inputImage.crossOrigin = "Anonymous"; inputImage.src = imgFile; let canvasImage; inputImage.onload = () => { canvas.width = inputImage.width; canvas.height = inputImage.height; // Resize the input image of the network if it is too large to simply crop // the center 64x64 portion in order to still provide a representative // input image into the network. if (inputImage.width > networkInputSize || inputImage.height > networkInputSize) { // Step 1 - Resize using smaller dimension to scale the image down. let resizeCanvas = document.createElement('canvas'), resizeContext = resizeCanvas.getContext('2d'); let smallerDimension = Math.min(inputImage.width, inputImage.height); const resizeFactor = (networkInputSize + 1) / smallerDimension; resizeCanvas.width = inputImage.width * resizeFactor; resizeCanvas.height = inputImage.height * resizeFactor; resizeContext.drawImage(inputImage, 0, 0, resizeCanvas.width, resizeCanvas.height); // Step 2 - Flip non-square images horizontally and rotate them 90deg since // non-square images are not stored upright. if (inputImage.width != inputImage.height) { context.translate(resizeCanvas.width, 0); context.scale(-1, 1); context.translate(resizeCanvas.width / 2, resizeCanvas.height / 2); context.rotate(90 * Math.PI / 180); } // Step 3 - Draw resized image on original canvas. if (inputImage.width != inputImage.height) { context.drawImage(resizeCanvas, -resizeCanvas.width / 2, -resizeCanvas.height / 2); } else { context.drawImage(resizeCanvas, 0, 0); } canvasImage = context.getImageData(0, 0, resizeCanvas.width, resizeCanvas.height); } else { context.drawImage(inputImage, 0, 0); canvasImage = context.getImageData(0, 0, inputImage.width, inputImage.height); } // Get image data and convert it to a 3D array let imageData = canvasImage.data; let imageWidth = canvasImage.width; let imageHeight = canvasImage.height; // Remove this newly created canvas element canvas.parentNode.removeChild(canvas); resolve(imageDataTo3DTensor(imageData, imageWidth, imageHeight, normalize)); } inputImage.onerror = reject; }) } /** * Wrapper to load a model. * * @param {string} modelFile Filename of converted (through tensorflowjs.py) * model json file. */ export const loadTrainedModel = (modelFile) => { return tf.loadLayersModel(modelFile); } ================================================ FILE: src/utils/cnn.js ================================================ // Enum of node types const nodeType = { INPUT: 'input', CONV: 'conv', POOL: 'pool', RELU: 'relu', FC: 'fc', FLATTEN: 'flatten' } class Node { /** * Class structure for each neuron node. * * @param {string} layerName Name of the node's layer. * @param {int} index Index of this node in its layer. * @param {string} type Node type {input, conv, pool, relu, fc}. * @param {number} bias The bias assocated to this node. * @param {[[number]]} output Output of this node. */ constructor(layerName, index, type, bias, output) { this.layerName = layerName; this.index = index; this.type = type; this.bias = bias; this.output = output; // Weights are stored in the links this.inputLinks = []; this.outputLinks = []; } } class Link { constructor(source, dest, weight) { this.source = source; this.dest = dest; this.weight = weight; } } const constructNNFromJSON = (nnJSON, inputImageArray) => { console.log(nnJSON); console.log(inputImageArray); let nn = []; // Add the first layer (input layer) let inputLayer = []; let inputShape = nnJSON[0].input_shape; // First layer's three nodes' outputs are the channels of inputImageArray for (let i = 0; i < inputShape[2]; i++) { let node = new Node('input', i, nodeType.INPUT, 0, inputImageArray[i]); inputLayer.push(node); } nn.push(inputLayer); let curLayerIndex = 1; nnJSON.forEach(layer => { let curLayerNodes = []; let curLayerType; if (layer.name.includes('conv')) { curLayerType = nodeType.CONV; } else if (layer.name.includes('pool')) { curLayerType = nodeType.POOL; } else if (layer.name.includes('relu')) { curLayerType = nodeType.RELU; } else if (layer.name.includes('output')) { curLayerType = nodeType.FC; } else if (layer.name.includes('flatten')) { curLayerType = nodeType.FLATTEN; } else { console.log('Find unknown type'); } let shape = layer.output_shape.slice(0, 2); let bias = 0; let output; if (curLayerType === nodeType.FLATTEN || curLayerType === nodeType.FC) { output = 0; } else { output = init2DArray(shape[0], shape[1], 0); } // Add neurons into this layer for (let i = 0; i < layer.num_neurons; i++) { if (curLayerType === nodeType.CONV || curLayerType === nodeType.FC) { bias = layer.weights[i].bias; } let node = new Node(layer.name, i, curLayerType, bias, output) // Connect this node to all previous nodes (create links) if (curLayerType === nodeType.CONV || curLayerType === nodeType.FC) { // CONV and FC layers have weights in links. Links are one-to-multiple for (let j = 0; j < nn[curLayerIndex - 1].length; j++) { let preNode = nn[curLayerIndex - 1][j]; let curLink = new Link(preNode, node, layer.weights[i].weights[j]); preNode.outputLinks.push(curLink); node.inputLinks.push(curLink); } } else if (curLayerType === nodeType.RELU || curLayerType === nodeType.POOL) { // RELU and POOL layers have no weights. Links are one-to-one let preNode = nn[curLayerIndex - 1][i]; let link = new Link(preNode, node, null); preNode.outputLinks.push(link); node.inputLinks.push(link); } else if (curLayerType === nodeType.FLATTEN) { // Flatten layer has no weights. Links are multiple-to-one. // Use dummy weights to store the corresponding entry in the previsou // node as (row, column) // The flatten() in tf2.keras has order: channel -> row -> column let preNodeWidth = nn[curLayerIndex - 1][0].output.length, preNodeNum = nn[curLayerIndex - 1].length, preNodeIndex = i % preNodeNum, preNodeRow = Math.floor(Math.floor(i / preNodeNum) / preNodeWidth), preNodeCol = Math.floor(i / preNodeNum) % preNodeWidth, link = new Link(nn[curLayerIndex - 1][preNodeIndex], node, [preNodeRow, preNodeCol]); nn[curLayerIndex - 1][preNodeIndex].outputLinks.push(link); node.inputLinks.push(link); } curLayerNodes.push(node); } // Add current layer to the NN nn.push(curLayerNodes); curLayerIndex++; }); return nn; } export const constructNN = (inputImageFile) => { // Load the saved model file return new Promise((resolve, reject) => { fetch('PUBLIC_URL/assets/data/nn_10.json') .then(response => { response.json().then(nnJSON => { getInputImageArray(inputImageFile) .then(inputImageArray => { let nn = constructNNFromJSON(nnJSON, inputImageArray); resolve(nn); }) }); }) .catch(error => { reject(error); }); }); } // Helper functions /** * Create a 2D array (matrix) with given size and default value. * * @param {int} height Height (number of rows) for the matrix * @param {int} width Width (number of columns) for the matrix * @param {int} fill Default value to fill this matrix */ export const init2DArray = (height, width, fill) => { let array = []; // Itereate through rows for (let r = 0; r < height; r++) { let row = new Array(width).fill(fill); array.push(row); } return array; } /** * Dot product of two matrices. * @param {[[number]]} mat1 Matrix 1 * @param {[[number]]} mat2 Matrix 2 */ const matrixDot = (mat1, mat2) => { console.assert(mat1.length === mat2.length, 'Dimension not matching'); console.assert(mat1[0].length === mat2[0].length, 'Dimension not matching'); let result = 0; for (let i = 0; i < mat1.length; i++){ for (let j = 0; j < mat1[0].length; j++){ result += mat1[i][j] * mat2[i][j]; } } return result; } /** * Matrix elementwise addition. * @param {[[number]]} mat1 Matrix 1 * @param {[[number]]} mat2 Matrix 2 */ export const matrixAdd = (mat1, mat2) => { console.assert(mat1.length === mat2.length, 'Dimension not matching'); console.assert(mat1[0].length === mat2[0].length, 'Dimension not matching'); let result = init2DArray(mat1.length, mat1.length, 0); for (let i = 0; i < mat1.length; i++) { for (let j = 0; j < mat1.length; j++) { result[i][j] = mat1[i][j] + mat2[i][j]; } } return result; } /** * 2D slice on a matrix. * @param {[[number]]} mat Matrix * @param {int} xs First dimension (row) starting index * @param {int} xe First dimension (row) ending index * @param {int} ys Second dimension (column) starting index * @param {int} ye Second dimension (column) ending index */ export const matrixSlice = (mat, xs, xe, ys, ye) => { return mat.slice(xs, xe).map(s => s.slice(ys, ye)); } /** * Compute the maximum of a matrix. * @param {[[number]]} mat Matrix */ const matrixMax = (mat) => { let curMax = -Infinity; for (let i = 0; i < mat.length; i++) { for (let j = 0; j < mat[0].length; j++) { if (mat[i][j] > curMax) { curMax = mat[i][j]; } } } return curMax; } /** * Convert canvas image data into a 3D array with dimension [height, width, 3]. * Each pixel is in 0-255 scale. * @param {[int8]} imageData Canvas image data */ const imageDataTo3DArray = (imageData) => { // Get image dimension (assume square image) let width = Math.sqrt(imageData.length / 4); // Create array placeholder for each channel let imageArray = [init2DArray(width, width, 0), init2DArray(width, width, 0), init2DArray(width, width, 0)]; // Iterate through the data to fill out channel arrays above for (let i = 0; i < imageData.length; i++) { let pixelIndex = Math.floor(i / 4), channelIndex = i % 4, row = Math.floor(pixelIndex / width), column = pixelIndex % width; if (channelIndex < 3) { imageArray[channelIndex][row][column] = imageData[i]; } } return imageArray; } /** * Get the 3D pixel value array of the given image file. * @param {string} imgFile File path to the image file * @returns A promise with the corresponding 3D array */ const getInputImageArray = (imgFile) => { let canvas = document.createElement('canvas'); canvas.style.cssText = 'display:none;'; document.getElementsByTagName('body')[0].appendChild(canvas); let context = canvas.getContext('2d'); return new Promise((resolve, reject) => { let inputImage = new Image(); inputImage.src = imgFile; inputImage.onload = () => { context.drawImage(inputImage, 0, 0,); // Get image data and convert it to a 3D array let imageData = context.getImageData(0, 0, inputImage.width, inputImage.height).data; // Remove this newly created canvas element canvas.parentNode.removeChild(canvas); console.log(imageDataTo3DArray(imageData)); resolve(imageDataTo3DArray(imageData)); } inputImage.onerror = reject; }) } /** * Compute convolutions of one kernel on one matrix (one slice of a tensor). * @param {[[number]]} input Input, square matrix * @param {[[number]]} kernel Kernel weights, square matrix * @param {int} stride Stride size * @param {int} padding Padding size */ export const singleConv = (input, kernel, stride=1, padding=0) => { // TODO: implement padding // Only support square input and kernel console.assert(input.length === input[0].length, 'Conv input is not square'); console.assert(kernel.length === kernel[0].length, 'Conv kernel is not square'); let stepSize = (input.length - kernel.length) / stride + 1; let result = init2DArray(stepSize, stepSize, 0); // Window sliding for (let r = 0; r < stepSize; r++) { for (let c = 0; c < stepSize; c++) { let curWindow = matrixSlice(input, r * stride, r * stride + kernel.length, c * stride, c * stride + kernel.length); let dot = matrixDot(curWindow, kernel); result[r][c] = dot; } } return result; } /** * Convolution operation. This function update the outputs property of all nodes * in the given layer. Previous layer is accessed by the reference in nodes' * links. * @param {[Node]} curLayer Conv layer. */ const convolute = (curLayer) => { console.assert(curLayer[0].type === 'conv', 'Wrong layer type'); // Itereate through all nodes in curLayer to update their outputs curLayer.forEach(node => { /* * Accumulate the single conv result matrices from previous channels. * Previous channels (node) are accessed by the reference in Link objects. */ let newOutput = init2DArray(node.output.length, node.output.length, 0); for (let i = 0; i < node.inputLinks.length; i++) { let curLink = node.inputLinks[i]; let curConvResult = singleConv(curLink.source.output, curLink.weight); newOutput = matrixAdd(newOutput, curConvResult); } // Add bias to all element in the output let biasMatrix = init2DArray(newOutput.length, newOutput.length, node.bias); newOutput = matrixAdd(newOutput, biasMatrix); node.output = newOutput; }) } /** * Activate matrix mat using ReLU (max(0, x)). * @param {[[number]]} mat Matrix */ const singleRelu = (mat) => { // Only support square matrix console.assert(mat.length === mat[0].length, 'Activating non-square matrix!'); let width = mat.length; let result = init2DArray(width, width, 0); for (let i = 0; i < width; i++) { for (let j = 0; j < width; j++) { result[i][j] = Math.max(0, mat[i][j]); } } return result; } /** * Update outputs of all nodes in the current ReLU layer. Values of previous * layer nodes are accessed by the links stored in the current layer. * @param {[Node]} curLayer ReLU layer */ const relu = (curLayer) => { console.assert(curLayer[0].type === 'relu', 'Wrong layer type'); // Itereate through all nodes in curLayer to update their outputs for (let i = 0; i < curLayer.length; i++) { let curNode = curLayer[i]; let preNode = curNode.inputLinks[0].source; curNode.output = singleRelu(preNode.output); } } /** * Max pool one matrix. * @param {[[number]]} mat Matrix * @param {int} kernelWidth Pooling kernel length (only supports 2) * @param {int} stride Pooling sliding stride (only supports 2) * @param {string} padding Pading method when encountering odd number mat, * currently this function only supports 'VALID' */ export const singleMaxPooling = (mat, kernelWidth=2, stride=2, padding='VALID') => { console.assert(kernelWidth === 2, 'Only supports kernen = [2,2]'); console.assert(stride === 2, 'Only supports stride = 2'); console.assert(padding === 'VALID', 'Only support valid padding'); // Handle odd length mat // 'VALID': ignore edge rows and columns // 'SAME': add zero padding to make the mat have even length if (mat.length % 2 === 1 && padding === 'VALID') { mat = matrixSlice(mat, 0, mat.length - 1, 0, mat.length - 1); } let stepSize = (mat.length - kernelWidth) / stride + 1; let result = init2DArray(stepSize, stepSize, 0); for (let r = 0; r < stepSize; r++) { for (let c = 0; c < stepSize; c++) { let curWindow = matrixSlice(mat, r * stride, r * stride + kernelWidth, c * stride, c * stride + kernelWidth); result[r][c] = matrixMax(curWindow); } } return result; } /** * Max pooling one layer. * @param {[Node]} curLayer MaxPool layer */ const maxPooling = (curLayer) => { console.assert(curLayer[0].type === 'pool', 'Wrong layer type'); // Itereate through all nodes in curLayer to update their outputs for (let i = 0; i < curLayer.length; i++) { let curNode = curLayer[i]; let preNode = curNode.inputLinks[0].source; curNode.output = singleMaxPooling(preNode.output); } } /** * Flatten a previous 2D layer (conv2d or maxpool2d). The flatten order matches * tf2.keras' implementation: channel -> row -> column. * @param {[Node]} curLayer Flatten layer */ const flatten = (curLayer) => { console.assert(curLayer[0].type === 'flatten', 'Wrong layer type'); // Itereate through all nodes in curLayer to update their outputs for (let i = 0; i < curLayer.length; i++) { let curNode = curLayer[i]; let preNode = curNode.inputLinks[0].source; let coordinate = curNode.inputLinks[0].weight; // Take advantage of the dummy weights curNode.output = preNode.output[coordinate[0]][coordinate[1]]; } } const fullyConnect = (curLayer) => { console.assert(curLayer[0].type === 'fc', 'Wrong layer type'); // TODO } export const tempMain = async () => { let nn = await constructNN('PUBLIC_URL/assets/img/koala.jpeg'); convolute(nn[1]); relu(nn[2]) convolute(nn[3]); relu(nn[4]); maxPooling(nn[5]); convolute(nn[6]); relu(nn[7]) convolute(nn[8]); relu(nn[9]); maxPooling(nn[10]); convolute(nn[11]); relu(nn[12]) convolute(nn[13]); relu(nn[14]); maxPooling(nn[15]); flatten(nn[16]); console.log(nn[16].map(d => d.output)); } ================================================ FILE: src/utils/deploy-to-gh-pages.sh ================================================ #!/bin/bash set -o errexit # config git config --global user.email "xiao.hk1997@gmail.com" git config --global user.name "xiaohk" # build git clone git@github.com:poloclub/cnn-explainer.git cd cnn-explainer npm install npm run build mkdir dist copy -r ./public/* ./dist sed -i 's/\/assets/\/cnn-explainer\/assets/g' ./dist/index.html git add dist git commit -m "Deploy gh-pages from Travis" git subtree push --prefix dist origin gh-pages ================================================ FILE: src/utils/utlis.py ================================================ import tensorflow as tf from json import dump assert(int(tf.__version__.split('.')[0]) == 2) def convert_h5_to_json(model_h5_file, model_json_file): """ Helper function to convert tf2 stored model h5 file to a customized json format. Args: model_h5_file(string): filename of the stored h5 file model_json_file(string): filename of the output json file """ model = tf.keras.models.load_model(model_h5_file) json_dict = {} for l in model.layers: json_dict[l.name] = { 'input_shape': l.input_shape[1:], 'output_shape': l.output_shape[1:], 'num_neurons': l.output_shape[-1] } if 'conv' in l.name: all_weights = l.weights[0] neuron_weights = [] # Iterate through neurons in that layer for n in range(all_weights.shape[3]): cur_neuron_dict = {} cur_neuron_dict['bias'] = l.bias.numpy()[n].item() # Get the current weights cur_weights = all_weights[:, :, :, n].numpy().astype(float) # Reshape the weights from (height, width, input_c) to # (input_c, height, width) cur_weights = cur_weights.transpose((2, 0, 1)).tolist() cur_neuron_dict['weights'] = cur_weights neuron_weights.append(cur_neuron_dict) json_dict[l.name]['weights'] = neuron_weights elif 'output' in l.name: all_weights = l.weights[0] neuron_weights = [] # Iterate through neurons in that layer for n in range(all_weights.shape[1]): cur_neuron_dict = {} cur_neuron_dict['bias'] = l.bias.numpy()[n].item() # Get the current weights cur_weights = all_weights[:, n].numpy().astype(float).tolist() cur_neuron_dict['weights'] = cur_weights neuron_weights.append(cur_neuron_dict) json_dict[l.name]['weights'] = neuron_weights dump(json_dict, open(model_json_file, 'w'), indent=2) ================================================ FILE: tiny-vgg/README.md ================================================ # Train a Tiny VGG This directory includes code and data to train a Tiny VGG model (inspired by the demo CNN in [Stanford CS231n class](http://cs231n.stanford.edu)) on 10 everyday classes from the [Tiny ImageNet](https://tiny-imagenet.herokuapp.com). ## Installation First, you want to unzip `data.zip`. The file structure would be something like: ``` . ├── data │   ├── class_10_train │   │   ├── n01882714 │   │   │   ├── images [500 entries exceeds filelimit, not opening dir] │   │   │   └── n01882714_boxes.txt │   │   ├── n02165456 │   │   │   ├── images [500 entries exceeds filelimit, not opening dir] │   │   │   └── n02165456_boxes.txt │   │   ├── n02509815 │   │   │   ├── images [500 entries exceeds filelimit, not opening dir] │   │   │   └── n02509815_boxes.txt │   │   ├── n03662601 │   │   │   ├── images [500 entries exceeds filelimit, not opening dir] │   │   │   └── n03662601_boxes.txt │   │   ├── n04146614 │   │   │   ├── images [500 entries exceeds filelimit, not opening dir] │   │   │   └── n04146614_boxes.txt │   │   ├── n04285008 │   │   │   ├── images [500 entries exceeds filelimit, not opening dir] │   │   │   └── n04285008_boxes.txt │   │   ├── n07720875 │   │   │   ├── images [500 entries exceeds filelimit, not opening dir] │   │   │   └── n07720875_boxes.txt │   │   ├── n07747607 │   │   │   ├── images [500 entries exceeds filelimit, not opening dir] │   │   │   └── n07747607_boxes.txt │   │   ├── n07873807 │   │   │   ├── images [500 entries exceeds filelimit, not opening dir] │   │   │   └── n07873807_boxes.txt │   │   └── n07920052 │   │   ├── images [500 entries exceeds filelimit, not opening dir] │   │   └── n07920052_boxes.txt │   ├── class_10_val │   │   ├── test_images [250 entries exceeds filelimit, not opening dir] │   │   └── val_images [250 entries exceeds filelimit, not opening dir] │   ├── class_dict_10.json │   └── val_class_dict_10.json ├── data.zip ├── environment.yaml └── tiny-vgg.py ``` To install all dependencies, run the following code ``` conda env create --file environment.yaml ``` ## Training To train Tiny VGG on these 10 classes, run the following code ``` python tiny-vgg.py ``` After training, you will get two saved models in Keras format: `trained_tiny_vgg.h5` and `trained_vgg_best.h5`. The first file is the final model after training, and `trained_vgg_best.h5` is the model having the best validation performance. You can use either one for CNN Explainer. ## Convert Model Format Before loading the model using *tensorflow.js*, you want to convert the model file from Keras `h5` format to [tensorflow.js format](https://www.tensorflow.org/js/tutorials/conversion/import_keras). ``` tensorflowjs_converter --input_format keras trained_vgg_best.h5 ./ ``` Then you can put the output file `group1-shard1of1.bin` in `/public/data` and use *tensorflow.js* to load the trained model. ================================================ FILE: tiny-vgg/environment.yaml ================================================ name: tiny-vgg channels: - defaults dependencies: - pip=20.0.2 - python=3.6.10 - pip: - h5py==2.10.0 - numpy==1.18.3 - pandas==1.0.3 - scipy==1.4.1 - tensorflow==2.1.0 - tensorflow-cpu==2.1.0 - tensorflowjs==1.7.4 ================================================ FILE: tiny-vgg/tiny-vgg.py ================================================ import tensorflow as tf import numpy as np import pandas as pd import re from shutil import copyfile from glob import glob from json import load, dump from tensorflow.keras.layers import Dense, Flatten, Conv2D, MaxPool2D,\ Activation from tensorflow.keras import Model, Sequential from os.path import basename from time import time print(tf.__version__) def create_class_dict(): # Create a new version only including tiny 200 classes df = pd.read_csv('./tiny-imagenet-200/words.txt', sep='\t', header=None) keys, classes = df[0], df[1] class_dict = dict(zip(keys, classes)) tiny_class_dict = {} cur_index = 0 for directory in glob('./tiny-imagenet-200/train/*'): cur_key = basename(directory) tiny_class_dict[cur_key] = {'class': class_dict[cur_key], 'index': cur_index} cur_index += 1 dump(tiny_class_dict, open('./tiny-imagenet-200/class_dict.json', 'w'), indent=2) def create_val_class_dict(): tiny_class_dict = load(open('./tiny-imagenet-200/class_dict.json', 'r')) tiny_val_class_dict = {} # Create a dictionary for validation images df = pd.read_csv('./tiny-imagenet-200/val/val_annotations.txt', sep='\t', header=None) image_names = df[0] image_classes = df[1] for i in range(len(image_names)): tiny_val_class_dict[image_names[i]] = { 'class': tiny_class_dict[image_classes[i]]['class'], 'index': tiny_class_dict[image_classes[i]]['index'], } dump(tiny_val_class_dict, open('./tiny-imagenet-200/val_class_dict.json', 'w'), indent=2) def split_val_data(): # Split validation images to 50% early stopping and 50% hold-out testing val_images = glob('./tiny-imagenet-200/val/images/*.JPEG') np.random.shuffle(val_images) for i in range(len(val_images)): if i < len(val_images) // 2: copyfile(val_images[i], val_images[i].replace('images', 'val_images')) else: copyfile(val_images[i], val_images[i].replace('images', 'test_images')) def process_path_train(path): """ Get the (class label, processed image) pair of the given image path. This funciton uses python primitives, so you need to use tf.py_funciton wrapper. This function uses global variables: WIDTH(int): the width of the targeting image HEIGHT(int): the height of the targeting iamge NUM_CLASS(int): number of classes Args: path(string): path to an image file """ # Get the class path = path.numpy() image_name = basename(path.decode('ascii')) label_name = re.sub(r'(.+)_\d+\.JPEG', r'\1', image_name) label_index = tiny_class_dict[label_name]['index'] # Convert label to one-hot encoding label = tf.one_hot(indices=[label_index], depth=NUM_CLASS) label = tf.reshape(label, [NUM_CLASS]) # Read image and convert the image to [0, 1] range 3d tensor img = tf.io.read_file(path) img = tf.image.decode_jpeg(img, channels=3) img = tf.image.convert_image_dtype(img, tf.float32) img = tf.image.resize(img, [WIDTH, HEIGHT]) return(img, label) def process_path_test(path): """ Get the (class label, processed image) pair of the given image path. This funciton uses python primitives, so you need to use tf.py_funciton wrapper. This function uses global variables: WIDTH(int): the width of the targeting image HEIGHT(int): the height of the targeting iamge NUM_CLASS(int): number of classes The filepath encoding for test images is different from training images. Args: path(string): path to an image file """ # Get the class path = path.numpy() image_name = basename(path.decode('ascii')) label_index = tiny_val_class_dict[image_name]['index'] # Convert label to one-hot encoding label = tf.one_hot(indices=[label_index], depth=NUM_CLASS) label = tf.reshape(label, [NUM_CLASS]) # Read image and convert the image to [0, 1] range 3d tensor img = tf.io.read_file(path) img = tf.image.decode_jpeg(img, channels=3) img = tf.image.convert_image_dtype(img, tf.float32) img = tf.image.resize(img, [WIDTH, HEIGHT]) return(img, label) def prepare_for_training(dataset, batch_size=32, cache=True, shuffle_buffer_size=1000): if cache: if isinstance(cache, str): dataset = dataset.cache(cache) else: dataset = dataset.cache() # Only shuffle elements in the buffer size dataset = dataset.shuffle(buffer_size=shuffle_buffer_size) # Pre featch batches in the background dataset = dataset.batch(batch_size) dataset = dataset.prefetch(buffer_size=tf.data.experimental.AUTOTUNE) return dataset def prepare_for_testing(dataset, batch_size=32, cache=True): if cache: if isinstance(cache, str): dataset = dataset.cache(cache) else: dataset = dataset.cache() # Pre featch batches in the background dataset = dataset.batch(batch_size) dataset = dataset.prefetch(buffer_size=tf.data.experimental.AUTOTUNE) return dataset class TinyVGG(Model): """ Tiny VGG structure is adapted from http://cs231n.stanford.edu: > This particular network is classifying CIFAR-10 images into one of 10 > classes and was trained with ConvNetJS. Its exact architecture is > [conv-relu-conv-relu-pool]x3-fc-softmax, for a total of 17 layers and > 7000 parameters. It uses 3x3 convolutions and 2x2 pooling regions. """ def __init__(self, filters=10): super(TinyVGG, self).__init__() self.conv_1_1 = Conv2D(filters, (3, 3), name='conv_1_1') self.relu_1_1 = Activation('relu', name='relu_1_1') self.conv_1_2 = Conv2D(filters, (3, 3), name='conv_1_2') self.relu_1_2 = Activation('relu', name='relu_1_2') self.max_pool_1 = MaxPool2D((2, 2), name='max_pool_1') self.conv_2_1 = Conv2D(filters, (3, 3), name='conv_2_1') self.relu_2_1 = Activation('relu', name='relu_2_1') self.conv_2_2 = Conv2D(filters, (3, 3), name='conv_2_2') self.relu_2_2 = Activation('relu', name='relu_2_2') self.max_pool_2 = MaxPool2D((2, 2), name='max_pool_2') self.flatten = Flatten() self.fc = Dense(NUM_CLASS, activation='softmax') def call(self, x): x = self.conv_1_1(x) x = self.relu_1_1(x) x = self.conv_1_2(x) x = self.relu_1_2(x) x = self.max_pool_1(x) x = self.conv_2_1(x) x = self.relu_2_1(x) x = self.conv_2_2(x) x = self.relu_2_2(x) x = self.max_pool_2(x) x = self.conv_3_1(x) x = self.relu_3_1(x) x = self.conv_3_2(x) x = self.relu_3_2(x) x = self.max_pool_3(x) x = self.flatten(x) return self.fc(x) @tf.function def train_step(image_batch, label_batch): with tf.GradientTape() as tape: # Predict predictions = tiny_vgg(image_batch) # Update gradient loss = loss_object(label_batch, predictions) gradients = tape.gradient(loss, tiny_vgg.trainable_variables) optimizer.apply_gradients(zip(gradients, tiny_vgg.trainable_variables)) train_mean_loss(loss) train_accuracy(label_batch, predictions) @tf.function def vali_step(image_batch, label_batch): predictions = tiny_vgg(image_batch) vali_loss = loss_object(label_batch, predictions) vali_mean_loss(vali_loss) vali_accuracy(label_batch, predictions) @tf.function def test_step(image_batch, label_batch): predictions = tiny_vgg(image_batch) test_loss = loss_object(label_batch, predictions) test_mean_loss(test_loss) test_accuracy(label_batch, predictions) WIDTH = 64 HEIGHT = 64 EPOCHS = 1000 PATIENCE = 50 LR = 0.001 NUM_CLASS = 10 BATCH_SIZE = 32 # Create training and validation dataset tiny_class_dict = load(open('./data/class_dict_10.json', 'r')) tiny_val_class_dict = load(open('./data/val_class_dict_10.json', 'r')) training_images = './data/class_10_train/*/images/*.JPEG' vali_images = './data/class_10_val/val_images/*.JPEG' test_images = './data/class_10_val/test_images/*.JPEG' # Create training dataset train_path_dataset = tf.data.Dataset.list_files(training_images) train_labeld_dataset = train_path_dataset.map( lambda path: tf.py_function( process_path_train, [path], [tf.float32, tf.float32] ) ) # Create vali dataset vali_path_dataset = tf.data.Dataset.list_files(vali_images) vali_labeld_dataset = vali_path_dataset.map( lambda path: tf.py_function( process_path_test, [path], [tf.float32, tf.float32] ) ) # Create test dataset test_path_dataset = tf.data.Dataset.list_files(test_images) test_labeld_dataset = test_path_dataset.map( lambda path: tf.py_function( process_path_test, [path], [tf.float32, tf.float32] ) ) train_dataset = prepare_for_training(train_labeld_dataset, batch_size=BATCH_SIZE) vali_dataset = prepare_for_training(vali_labeld_dataset, batch_size=BATCH_SIZE) test_dataset = prepare_for_training(test_labeld_dataset, batch_size=BATCH_SIZE) # Create an instance of the model # tiny_vgg = TinyVGG() # Use Keras Sequential API instead, since it is easy to save the model filters = 10 tiny_vgg = Sequential([ Conv2D(filters, (3, 3), input_shape=(64, 64, 3), name='conv_1_1'), Activation('relu', name='relu_1_1'), Conv2D(filters, (3, 3), name='conv_1_2'), Activation('relu', name='relu_1_2'), MaxPool2D((2, 2), name='max_pool_1'), Conv2D(filters, (3, 3), name='conv_2_1'), Activation('relu', name='relu_2_1'), Conv2D(filters, (3, 3), name='conv_2_2'), Activation('relu', name='relu_2_2'), MaxPool2D((2, 2), name='max_pool_2'), Flatten(name='flatten'), Dense(NUM_CLASS, activation='softmax', name='output') ]) # "Compile" the model with loss function and optimizer loss_object = tf.keras.losses.CategoricalCrossentropy() # optimizer = tf.keras.optimizers.Adam(learning_rate=LR) optimizer = tf.keras.optimizers.SGD(learning_rate=LR) train_mean_loss = tf.keras.metrics.Mean(name='train_mean_loss') train_accuracy = tf.keras.metrics.CategoricalAccuracy(name='train_accuracy') vali_mean_loss = tf.keras.metrics.Mean(name='vali_mean_loss') vali_accuracy = tf.keras.metrics.CategoricalAccuracy(name='vali_accuracy') # Initialize early stopping parameters no_improvement_epochs = 0 best_vali_loss = np.inf start_time = time() print('Start training.\n') for epoch in range(EPOCHS): # Train for image_batch, label_batch in train_dataset: train_step(image_batch, label_batch) # Predict on the test dataset for image_batch, label_batch in vali_dataset: vali_step(image_batch, label_batch) template = 'epoch: {}, train loss: {:.4f}, train accuracy: {:.4f}, ' template += 'vali loss: {:.4f}, vali accuracy: {:.4f}' print(template.format(epoch + 1, train_mean_loss.result(), train_accuracy.result() * 100, vali_mean_loss.result(), vali_accuracy.result() * 100)) # Early stopping if vali_mean_loss.result() < best_vali_loss: no_improvement_epochs = 0 best_vali_loss = vali_mean_loss.result() # Save the best model tiny_vgg.save('trained_vgg_best.h5') else: no_improvement_epochs += 1 if no_improvement_epochs >= PATIENCE: print('Early stopping at epoch = {}'.format(epoch)) break # Reset evaluation metrics train_mean_loss.reset_states() train_accuracy.reset_states() vali_mean_loss.reset_states() vali_accuracy.reset_states() print('\nFinished training, used {:.4f} mins.'.format((time() - start_time) / 60)) # Save trained model tiny_vgg.save('trained_tiny_vgg.h5') tiny_vgg = tf.keras.models.load_model('trained_vgg_best.h5') # Test on hold-out test images test_mean_loss = tf.keras.metrics.Mean(name='test_mean_loss') test_accuracy = tf.keras.metrics.CategoricalAccuracy(name='test_accuracy') for image_batch, label_batch in test_dataset: test_step(image_batch, label_batch) template = '\ntest loss: {:.4f}, test accuracy: {:.4f}' print(template.format(test_mean_loss.result(), test_accuracy.result() * 100))