Repository: neuro-ml/resnet_cnn_mri_adni
Branch: master
Commit: 66204c80e893
Files: 16
Total size: 99.7 KB
Directory structure:
gitextract_1lj34b5c/
├── .gitignore
├── .theanorc
├── Dockerfile
├── README.md
├── jupyter/
│ └── jupyter_notebook_config.py
├── scripts/
│ ├── README.md
│ ├── data/
│ │ ├── FirstADSubject.nii
│ │ └── metadata.csv
│ ├── requirements.txt
│ ├── resnet_plot_heatmap.ipynb
│ ├── resnet_plot_results.ipynb
│ ├── resnet_train.ipynb
│ ├── utils.py
│ ├── vgg_like_plot_results.ipynb
│ └── vgg_like_train.ipynb
└── test_scripts/
└── test_theano.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*,cover
.hypothesis/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# IPython Notebook
.ipynb_checkpoints
# pyenv
.python-version
# celery beat schedule file
celerybeat-schedule
# dotenv
.env
# virtualenv
venv/
ENV/
# Spyder project settings
.spyderproject
# Rope project settings
.ropeproject
================================================
FILE: .theanorc
================================================
[global]
device=gpu
floatX=float32
================================================
FILE: Dockerfile
================================================
FROM nvidia/cuda:7.5-cudnn5-devel
MAINTAINER Safiullin Amir amir147@rambler.ru
RUN apt-get clean && apt-get update
RUN apt-get install -yqq python3 python3-pip python3-dev build-essential \
python3-setuptools python3-numpy python3-scipy \
libatlas-dev libatlas3gf-base \
git wget gfortran libatlas-base-dev libatlas3-base libhdf5-dev \
libfreetype6-dev libpng12-dev pkg-config libxml2-dev libxslt-dev \
libboost-program-options-dev zlib1g-dev libboost-python-dev
ADD scripts /scripts
RUN pip3 install -U pip cython numpy
RUN pip3 install -U -r scripts/requirements.txt
EXPOSE 8888
VOLUME ["/notebook", "/scripts"]
WORKDIR /scripts
ADD test_scripts /test_scripts
COPY .theanorc /root/.theanorc
ADD jupyter /jupyter
ENV JUPYTER_CONFIG_DIR="/jupyter"
CMD ["jupyter", "notebook", "--ip=localhost"]
================================================
FILE: README.md
================================================
## Residual and Plain Convolutional Neural Networks for 3D Brain MRI Classification
https://arxiv.org/abs/1701.06643
Authors:
[Sergey Korolev](https://github.com/libfun), [Amir Safiullin](https://github.com/amigoml), [Mikhail Belyaev](https://github.com/mibel), [Yulia Dodonova](https://github.com/YuliaD)
____
Scripts for training without Docker image are located in `scripts` folder
____
### Install docker-jupyter
Install Docker https://docs.docker.com/engine/installation/
Install nvidia-docker https://github.com/NVIDIA/nvidia-docker/wiki/Installation
### Howto run docker-jupyter
Clone:
``` bash
git clone https://github.com/neuro-ml/resnet_cnn_mri.git
cd resnet_cnn_mri
```
Build:
```bash
[sudo] docker build -t dl_isbi:repr -f Dockerfile .
```
Run container:
```bash
[sudo] nvidia-docker run -it -p 8809:8888 -v ~/absolute/path/to_data:/scripts/data/ dl_isbi:repr bash
```
where ```~/absolute/path/to_data``` is absolute path on your local machine to folder with adni data.
Run notebook inside the container:
```bash
jupyter notebook
```
Open `http://localhost:8809` on your local machine.
_______
#### Data
Place all the .nii files and metadata.csv inside ```~/absolute/path/to_data``` (on your local machine).
metadata.csv should have `Label` and `Path` columns for file retrieval and class labels (you can find sample /scripts/data/metadata.csv).
`Path` value shoult be `data/some/path/to_scan.nii`
================================================
FILE: jupyter/jupyter_notebook_config.py
================================================
# Configuration file for jupyter-notebook.
#------------------------------------------------------------------------------
# Configurable configuration
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
# LoggingConfigurable configuration
#------------------------------------------------------------------------------
# A parent class for Configurables that log.
#
# Subclasses have a log trait, and the default behavior is to get the logger
# from the currently running Application.
#------------------------------------------------------------------------------
# SingletonConfigurable configuration
#------------------------------------------------------------------------------
# A configurable that only allows one instance.
#
# This class is for classes that should only have one instance of itself or
# *any* subclass. To create and retrieve such a class use the
# :meth:`SingletonConfigurable.instance` method.
#------------------------------------------------------------------------------
# Application configuration
#------------------------------------------------------------------------------
# This is an application.
# The date format used by logging formatters for %(asctime)s
# c.Application.log_datefmt = '%Y-%m-%d %H:%M:%S'
# The Logging format template
# c.Application.log_format = '[%(name)s]%(highlevel)s %(message)s'
# Set the log level by value or name.
# c.Application.log_level = 30
#------------------------------------------------------------------------------
# JupyterApp configuration
#------------------------------------------------------------------------------
# Base class for Jupyter applications
# Answer yes to any prompts.
# c.JupyterApp.answer_yes = False
# Full path of a config file.
# c.JupyterApp.config_file = u''
# Specify a config file to load.
# c.JupyterApp.config_file_name = u''
# Generate default config file.
# c.JupyterApp.generate_config = False
#------------------------------------------------------------------------------
# NotebookApp configuration
#------------------------------------------------------------------------------
# Set the Access-Control-Allow-Credentials: true header
# c.NotebookApp.allow_credentials = False
# Set the Access-Control-Allow-Origin header
#
# Use '*' to allow any origin to access your server.
#
# Takes precedence over allow_origin_pat.
# c.NotebookApp.allow_origin = ''
# Use a regular expression for the Access-Control-Allow-Origin header
#
# Requests from an origin matching the expression will get replies with:
#
# Access-Control-Allow-Origin: origin
#
# where `origin` is the origin of the request.
#
# Ignored if allow_origin is set.
# c.NotebookApp.allow_origin_pat = ''
# DEPRECATED use base_url
# c.NotebookApp.base_project_url = '/'
# The base URL for the notebook server.
#
# Leading and trailing slashes can be omitted, and will automatically be added.
# c.NotebookApp.base_url = '/'
# Specify what command to use to invoke a web browser when opening the notebook.
# If not specified, the default browser will be determined by the `webbrowser`
# standard library module, which allows setting of the BROWSER environment
# variable to override it.
# c.NotebookApp.browser = u''
# The full path to an SSL/TLS certificate file.
# c.NotebookApp.certfile = u'/jupyter/certs/server.crt'
# The full path to a certificate authority certifificate for SSL/TLS client
# authentication.
# c.NotebookApp.client_ca = u''
# The config manager class to use
# c.NotebookApp.config_manager_class = 'notebook.services.config.manager.ConfigManager'
# The notebook manager class to use.
# c.NotebookApp.contents_manager_class = 'notebook.services.contents.filemanager.FileContentsManager'
# Extra keyword arguments to pass to `set_secure_cookie`. See tornado's
# set_secure_cookie docs for details.
# c.NotebookApp.cookie_options = {}
# The random bytes used to secure cookies. By default this is a new random
# number every time you start the Notebook. Set it to a value in a config file
# to enable logins to persist across server sessions.
#
# Note: Cookie secrets should be kept private, do not share config files with
# cookie_secret stored in plaintext (you can read the value from a file).
# c.NotebookApp.cookie_secret = ''
# The file where the cookie secret is stored.
# c.NotebookApp.cookie_secret_file = u''
# The default URL to redirect to from `/`
# c.NotebookApp.default_url = '/tree'
# Whether to enable MathJax for typesetting math/TeX
#
# MathJax is the javascript library Jupyter uses to render math/LaTeX. It is
# very large, so you may want to disable it if you have a slow internet
# connection, or for offline use of the notebook.
#
# When disabled, equations etc. will appear as their untransformed TeX source.
# c.NotebookApp.enable_mathjax = True
# extra paths to look for Javascript notebook extensions
# c.NotebookApp.extra_nbextensions_path = []
# Extra paths to search for serving static files.
#
# This allows adding javascript/css to be available from the notebook server
# machine, or overriding individual files in the IPython
# c.NotebookApp.extra_static_paths = []
# Extra paths to search for serving jinja templates.
#
# Can be used to override templates from notebook.templates.
# c.NotebookApp.extra_template_paths = []
#
# c.NotebookApp.file_to_run = ''
# Use minified JS file or not, mainly use during dev to avoid JS recompilation
# c.NotebookApp.ignore_minified_js = False
# (bytes/sec) Maximum rate at which messages can be sent on iopub before they
# are limited.
# c.NotebookApp.iopub_data_rate_limit = 0
# (msg/sec) Maximum rate at which messages can be sent on iopub before they are
# limited.
# c.NotebookApp.iopub_msg_rate_limit = 0
# The IP address the notebook server will listen on.
c.NotebookApp.ip = '*'
# Supply extra arguments that will be passed to Jinja environment.
# c.NotebookApp.jinja_environment_options = {}
# Extra variables to supply to jinja templates when rendering.
# c.NotebookApp.jinja_template_vars = {}
# The kernel manager class to use.
# c.NotebookApp.kernel_manager_class = 'notebook.services.kernels.kernelmanager.MappingKernelManager'
# The kernel spec manager class to use. Should be a subclass of
# `jupyter_client.kernelspec.KernelSpecManager`.
#
# The Api of KernelSpecManager is provisional and might change without warning
# between this version of Jupyter and the next stable one.
# c.NotebookApp.kernel_spec_manager_class = 'jupyter_client.kernelspec.KernelSpecManager'
# The full path to a private key file for usage with SSL/TLS.
# c.NotebookApp.keyfile = u'/jupyter/certs/server.key'
# The login handler class to use.
# c.NotebookApp.login_handler_class = 'notebook.auth.login.LoginHandler'
# The logout handler class to use.
# c.NotebookApp.logout_handler_class = 'notebook.auth.logout.LogoutHandler'
# The url for MathJax.js.
# c.NotebookApp.mathjax_url = ''
# Dict of Python modules to load as notebook server extensions.Entry values can
# be used to enable and disable the loading ofthe extensions.
# c.NotebookApp.nbserver_extensions = {}
# The directory to use for notebooks and kernels.
# c.NotebookApp.notebook_dir = u''
# Whether to open in a browser after starting. The specific browser used is
# platform dependent and determined by the python standard library `webbrowser`
# module, unless it is overridden using the --browser (NotebookApp.browser)
# configuration option.
# c.NotebookApp.open_browser = False
# Hashed password to use for web authentication.
#
# To generate, type in a python/IPython shell:
#
# from notebook.auth import passwd; passwd()
#
# The string should be of the form type:salt:hashed-password.
# c.NotebookApp.password = u''
# The port the notebook server will listen on.
# c.NotebookApp.port = 8888
# The number of additional ports to try if the specified port is not available.
# c.NotebookApp.port_retries = 50
# DISABLED: use %pylab or %matplotlib in the notebook to enable matplotlib.
#c.NotebookApp.pylab = 'inline'
# (sec) Time window used to check the message and data rate limits.
# c.NotebookApp.rate_limit_window = 1.0
# Reraise exceptions encountered loading server extensions?
# c.NotebookApp.reraise_server_extension_failures = False
# DEPRECATED use the nbserver_extensions dict instead
# c.NotebookApp.server_extensions = []
# The session manager class to use.
# c.NotebookApp.session_manager_class = 'notebook.services.sessions.sessionmanager.SessionManager'
# Supply SSL options for the tornado HTTPServer. See the tornado docs for
# details.
# c.NotebookApp.ssl_options = {}
# Supply overrides for the tornado.web.Application that the Jupyter notebook
# uses.
# c.NotebookApp.tornado_settings = {}
# Whether to trust or not X-Scheme/X-Forwarded-Proto and X-Real-Ip/X-Forwarded-
# For headerssent by the upstream reverse proxy. Necessary if the proxy handles
# SSL
# c.NotebookApp.trust_xheaders = False
# DEPRECATED, use tornado_settings
# c.NotebookApp.webapp_settings = {}
# The base URL for websockets, if it differs from the HTTP server (hint: it
# almost certainly doesn't).
#
# Should be in the form of an HTTP origin: ws[s]://hostname[:port]
# c.NotebookApp.websocket_url = ''
#------------------------------------------------------------------------------
# ConnectionFileMixin configuration
#------------------------------------------------------------------------------
# Mixin for configurable classes that work with connection files
# JSON file in which to store connection info [default: kernel-<pid>.json]
#
# This file will contain the IP, ports, and authentication key needed to connect
# clients to this kernel. By default, this file will be created in the security
# dir of the current profile, but can be specified by absolute path.
# c.ConnectionFileMixin.connection_file = ''
# set the control (ROUTER) port [default: random]
# c.ConnectionFileMixin.control_port = 0
# set the heartbeat port [default: random]
# c.ConnectionFileMixin.hb_port = 0
# set the iopub (PUB) port [default: random]
# c.ConnectionFileMixin.iopub_port = 0
# Set the kernel's IP address [default localhost]. If the IP address is
# something other than localhost, then Consoles on other machines will be able
# to connect to the Kernel, so be careful!
# c.ConnectionFileMixin.ip = u''
# set the shell (ROUTER) port [default: random]
# c.ConnectionFileMixin.shell_port = 0
# set the stdin (ROUTER) port [default: random]
# c.ConnectionFileMixin.stdin_port = 0
#
# c.ConnectionFileMixin.transport = 'tcp'
#------------------------------------------------------------------------------
# KernelManager configuration
#------------------------------------------------------------------------------
# Manages a single kernel in a subprocess on this host.
#
# This version starts kernels with Popen.
# Should we autorestart the kernel if it dies.
# c.KernelManager.autorestart = True
# DEPRECATED: Use kernel_name instead.
#
# The Popen Command to launch the kernel. Override this if you have a custom
# kernel. If kernel_cmd is specified in a configuration file, Jupyter does not
# pass any arguments to the kernel, because it cannot make any assumptions about
# the arguments that the kernel understands. In particular, this means that the
# kernel does not receive the option --debug if it given on the Jupyter command
# line.
# c.KernelManager.kernel_cmd = []
#------------------------------------------------------------------------------
# Session configuration
#------------------------------------------------------------------------------
# Object for handling serialization and sending of messages.
#
# The Session object handles building messages and sending them with ZMQ sockets
# or ZMQStream objects. Objects can communicate with each other over the
# network via Session objects, and only need to work with the dict-based IPython
# message spec. The Session will handle serialization/deserialization, security,
# and metadata.
#
# Sessions support configurable serialization via packer/unpacker traits, and
# signing with HMAC digests via the key/keyfile traits.
#
# Parameters ----------
#
# debug : bool
# whether to trigger extra debugging statements
# packer/unpacker : str : 'json', 'pickle' or import_string
# importstrings for methods to serialize message parts. If just
# 'json' or 'pickle', predefined JSON and pickle packers will be used.
# Otherwise, the entire importstring must be used.
#
# The functions must accept at least valid JSON input, and output *bytes*.
#
# For example, to use msgpack:
# packer = 'msgpack.packb', unpacker='msgpack.unpackb'
# pack/unpack : callables
# You can also set the pack/unpack callables for serialization directly.
# session : bytes
# the ID of this Session object. The default is to generate a new UUID.
# username : unicode
# username added to message headers. The default is to ask the OS.
# key : bytes
# The key used to initialize an HMAC signature. If unset, messages
# will not be signed or checked.
# keyfile : filepath
# The file containing a key. If this is set, `key` will be initialized
# to the contents of the file.
# Threshold (in bytes) beyond which an object's buffer should be extracted to
# avoid pickling.
# c.Session.buffer_threshold = 1024
# Whether to check PID to protect against calls after fork.
#
# This check can be disabled if fork-safety is handled elsewhere.
# c.Session.check_pid = True
# Threshold (in bytes) beyond which a buffer should be sent without copying.
# c.Session.copy_threshold = 65536
# Debug output in the Session
# c.Session.debug = False
# The maximum number of digests to remember.
#
# The digest history will be culled when it exceeds this value.
# c.Session.digest_history_size = 65536
# The maximum number of items for a container to be introspected for custom
# serialization. Containers larger than this are pickled outright.
# c.Session.item_threshold = 64
# execution key, for signing messages.
# c.Session.key = ''
# path to file containing execution key.
# c.Session.keyfile = ''
# Metadata dictionary, which serves as the default top-level metadata dict for
# each message.
# c.Session.metadata = {}
# The name of the packer for serializing messages. Should be one of 'json',
# 'pickle', or an import name for a custom callable serializer.
# c.Session.packer = 'json'
# The UUID identifying this session.
# c.Session.session = u''
# The digest scheme used to construct the message signatures. Must have the form
# 'hmac-HASH'.
# c.Session.signature_scheme = 'hmac-sha256'
# The name of the unpacker for unserializing messages. Only used with custom
# functions for `packer`.
# c.Session.unpacker = 'json'
# Username for the Session. Default is your system username.
# c.Session.username = u'exuser'
#------------------------------------------------------------------------------
# MultiKernelManager configuration
#------------------------------------------------------------------------------
# A class for managing multiple kernels.
# The name of the default kernel to start
# c.MultiKernelManager.default_kernel_name = 'python2'
# The kernel manager class. This is configurable to allow subclassing of the
# KernelManager for customized behavior.
# c.MultiKernelManager.kernel_manager_class = 'jupyter_client.ioloop.IOLoopKernelManager'
#------------------------------------------------------------------------------
# MappingKernelManager configuration
#------------------------------------------------------------------------------
# A KernelManager that handles notebook mapping and HTTP error handling
#
# c.MappingKernelManager.root_dir = u''
#------------------------------------------------------------------------------
# ContentsManager configuration
#------------------------------------------------------------------------------
# Base class for serving files and directories.
#
# This serves any text or binary file, as well as directories, with special
# handling for JSON notebook documents.
#
# Most APIs take a path argument, which is always an API-style unicode path, and
# always refers to a directory.
#
# - unicode, not url-escaped
# - '/'-separated
# - leading and trailing '/' will be stripped
# - if unspecified, path defaults to '',
# indicating the root path.
#
# c.ContentsManager.checkpoints = None
#
# c.ContentsManager.checkpoints_class = 'notebook.services.contents.checkpoints.Checkpoints'
#
# c.ContentsManager.checkpoints_kwargs = {}
# Glob patterns to hide in file and directory listings.
# c.ContentsManager.hide_globs = [u'__pycache__', '*.pyc', '*.pyo', '.DS_Store', '*.so', '*.dylib', '*~']
# Python callable or importstring thereof
#
# To be called on a contents model prior to save.
#
# This can be used to process the structure, such as removing notebook outputs
# or other side effects that should not be saved.
#
# It will be called as (all arguments passed by keyword)::
#
# hook(path=path, model=model, contents_manager=self)
#
# - model: the model to be saved. Includes file contents.
# Modifying this dict will affect the file that is stored.
# - path: the API path of the save destination
# - contents_manager: this ContentsManager instance
# c.ContentsManager.pre_save_hook = None
# The base name used when creating untitled directories.
# c.ContentsManager.untitled_directory = 'Untitled Folder'
# The base name used when creating untitled files.
# c.ContentsManager.untitled_file = 'untitled'
# The base name used when creating untitled notebooks.
# c.ContentsManager.untitled_notebook = 'Untitled'
#------------------------------------------------------------------------------
# FileManagerMixin configuration
#------------------------------------------------------------------------------
# Mixin for ContentsAPI classes that interact with the filesystem.
#
# Provides facilities for reading, writing, and copying both notebooks and
# generic files.
#
# Shared by FileContentsManager and FileCheckpoints.
#
# Note ---- Classes using this mixin must provide the following attributes:
#
# root_dir : unicode
# A directory against against which API-style paths are to be resolved.
#
# log : logging.Logger
# By default notebooks are saved on disk on a temporary file and then if
# succefully written, it replaces the old ones. This procedure, namely
# 'atomic_writing', causes some bugs on file system whitout operation order
# enforcement (like some networked fs). If set to False, the new notebook is
# written directly on the old one which could fail (eg: full filesystem or quota
# )
# c.FileManagerMixin.use_atomic_writing = True
#------------------------------------------------------------------------------
# FileContentsManager configuration
#------------------------------------------------------------------------------
# Python callable or importstring thereof
#
# to be called on the path of a file just saved.
#
# This can be used to process the file on disk, such as converting the notebook
# to a script or HTML via nbconvert.
#
# It will be called as (all arguments passed by keyword)::
#
# hook(os_path=os_path, model=model, contents_manager=instance)
#
# - path: the filesystem path to the file just written - model: the model
# representing the file - contents_manager: this ContentsManager instance
# c.FileContentsManager.post_save_hook = None
#
# c.FileContentsManager.root_dir = u''
# DEPRECATED, use post_save_hook. Will be removed in Notebook 5.0
# c.FileContentsManager.save_script = False
#------------------------------------------------------------------------------
# NotebookNotary configuration
#------------------------------------------------------------------------------
# A class for computing and verifying notebook signatures.
# The hashing algorithm used to sign notebooks.
# c.NotebookNotary.algorithm = 'sha256'
# The number of notebook signatures to cache. When the number of signatures
# exceeds this value, the oldest 25% of signatures will be culled.
# c.NotebookNotary.cache_size = 65535
# The sqlite file in which to store notebook signatures. By default, this will
# be in your Jupyter runtime directory. You can set it to ':memory:' to disable
# sqlite writing to the filesystem.
# c.NotebookNotary.db_file = u''
# The secret key with which notebooks are signed.
# c.NotebookNotary.secret = ''
# The file where the secret key is stored.
# c.NotebookNotary.secret_file = u''
#------------------------------------------------------------------------------
# KernelSpecManager configuration
#------------------------------------------------------------------------------
# If there is no Python kernelspec registered and the IPython kernel is
# available, ensure it is added to the spec list.
# c.KernelSpecManager.ensure_native_kernel = True
# The kernel spec class. This is configurable to allow subclassing of the
# KernelSpecManager for customized behavior.
# c.KernelSpecManager.kernel_spec_class = 'jupyter_client.kernelspec.KernelSpec'
# Whitelist of allowed kernel names.
#
# By default, all installed kernels are allowed.
# c.KernelSpecManager.whitelist = set([])
================================================
FILE: scripts/README.md
================================================
## Residual and Plain Convolutional Neural Networks for 3D Brain MRI Classification
### Rerun experiments for training
1. Install `cuda==7.5` and `cudnn==5.0`.
2. Install the necessary python packages including Lasagne and Theano with `pip install -r requirements.txt`
3. Put your `metadata.csv` in `data` folder. It should have `Label` and `Path` columns for file retrieval and class labels.
4. Run `resnet_train.ipynb` or `vgg_like_train.ipynb` notebook to get classification results in `results_resnet` or `results_cnn` folder.
Note: Resnet code takes up about 2.7GB of video memory per sample in batch.
### Plot results of training
Use `resnet_plot_results.ipynb` or `vgg_like_plot_results.ipynb`
### Plot heatmap of resnet attention
Use `resnet_plot_heatmap.ipynb`
================================================
FILE: scripts/data/FirstADSubject.nii
================================================
================================================
FILE: scripts/data/metadata.csv
================================================
Label,Path
"AD","data/FirstADSubject.nii"
================================================
FILE: scripts/requirements.txt
================================================
ipykernel==4.5.0
ipython==5.1.0
ipython-genutils==0.1.0
ipywidgets==5.2.2
jupyter==1.0.0
jupyter-client==4.4.0
jupyter-console==5.0.0
jupyter-core==4.2.0
-e git+https://github.com/Lasagne/Lasagne.git@996bf64c0aec6d481044495800b461cc62040041#egg=Lasagne
matplotlib==1.5.3
nbconvert==4.2.0
nbformat==4.1.0
nibabel==2.1.0
nipype==0.12.1
notebook==4.2.3
numpy==1.11.2
pandas==0.19.0
scikit-image==0.12.3
scikit-learn==0.18
scipy==0.18.1
seaborn==0.7.1
https://github.com/Theano/Theano/archive/master.zip
tqdm==4.8.4
================================================
FILE: scripts/resnet_plot_heatmap.ipynb
================================================
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import pandas as pd\n",
"import numpy as np\n",
"import nibabel as nib\n",
"from tqdm import tqdm\n",
"from sklearn.cross_validation import StratifiedKFold\n",
"import lasagne\n",
"import theano\n",
"from lasagne.layers import InputLayer\n",
"from lasagne.layers.dnn import Conv3DDNNLayer\n",
"from lasagne.layers.dnn import Pool3DDNNLayer\n",
"from lasagne.layers import BatchNormLayer\n",
"from lasagne.layers import NonlinearityLayer\n",
"from lasagne.layers import ElemwiseSumLayer\n",
"from lasagne.layers import DenseLayer\n",
"from lasagne.nonlinearities import identity, softmax\n",
"import theano.tensor as T\n",
"import pickle\n",
"from skimage.transform import rotate\n",
"import matplotlib.pyplot as plt\n",
"%matplotlib inline"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"input_var = T.tensor5(name='input', dtype='float32')\n",
"target_var = T.ivector()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Function for heatmap calculation"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def occlusion_heatmap(net, x, label, square_size=7, batchsize=1):\n",
" \"\"\"\n",
" Parameters\n",
" ----------\n",
" net : Lasagne Layer\n",
" The neural net to test.\n",
" x : np.array\n",
" The input data, should be of shape (1, c, x, y, z).\n",
" label : int\n",
" Sample label\n",
" square_size : int, optional (default=7)\n",
" The length of the side of the square that occludes the image.\n",
" batchsize : int, optional (default=1)\n",
" Number of images in batch for inference pass.\n",
" \n",
" Results\n",
" -------\n",
" np.array\n",
" 3D np.array that at each point (i, j) contains the predicted\n",
" probability of the correct class if the image is occluded by a\n",
" square with center (i, j).\n",
" \"\"\"\n",
" if (x.ndim != 5) or x.shape[0] != 1:\n",
" raise ValueError(\"This function requires the input data to be of shape\"\n",
" \" (1, c, x, y, z), instead got {}\".format(x.shape))\n",
" if square_size % 2 == 0:\n",
" raise ValueError(\"Square length has to be an odd number, instead \"\n",
" \"got {}.\".format(square_size))\n",
"\n",
" num_classes = 2\n",
" img = x[0].copy()\n",
" bs, col, s0, s1, s2 = x.shape\n",
"\n",
" pad = square_size // 2 + 1\n",
" x_occluded = np.zeros((s2, col, s0, s1, s2), dtype=img.dtype)\n",
" probs = np.zeros((s0, s1, s2, num_classes))\n",
"\n",
" for i in range(s0):\n",
" for j in tqdm(range(s1), desc='x = {}'.format(i)):\n",
" batch_iter = 1\n",
" batch = []\n",
" for k in range(s2):\n",
" x_pad = np.pad(img, ((0, 0), (pad, pad),\n",
" (pad, pad), (pad, pad)), 'constant')\n",
" x_pad[:,\n",
" i:i + square_size,\n",
" j:j + square_size,\n",
" k:k + square_size] = 0.\n",
" x_occluded = x_pad[:, pad:-pad, pad:-pad, pad:-pad]\n",
" batch.append(x_occluded)\n",
" if batch_iter % batchsize == 0:\n",
" y_proba = test_fn(np.array(batch).reshape((-1, 1, 110, 110, 110)))\n",
" probs[i, j, k - batchsize + 1:k + 1, :] = y_proba\n",
" batch_iter = 0\n",
" batch = []\n",
" batch_iter += 1\n",
"\n",
" return probs"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def build_net():\n",
" \"\"\"Method for VoxResNet Building.\n",
"\n",
" Returns\n",
" -------\n",
" dictionary\n",
" Network dictionary.\n",
" \"\"\"\n",
" net = {}\n",
" net['input'] = InputLayer((None, 1, 110, 110, 110), input_var=input_var)\n",
" net['conv1a'] = Conv3DDNNLayer(net['input'], 32, 3, pad='same',\n",
" nonlinearity=identity)\n",
" net['bn1a'] = BatchNormLayer(net['conv1a'])\n",
" net['relu1a'] = NonlinearityLayer(net['bn1a'])\n",
" net['conv1b'] = Conv3DDNNLayer(net['relu1a'], 32, 3, pad='same',\n",
" nonlinearity=identity)\n",
" net['bn1b'] = BatchNormLayer(net['conv1b'])\n",
" net['relu1b'] = NonlinearityLayer(net['bn1b'])\n",
" net['conv1c'] = Conv3DDNNLayer(net['relu1b'], 64, 3, stride=(2, 2, 2),\n",
" pad='same', nonlinearity=identity)\n",
" # VoxRes block 2\n",
" net['voxres2_bn1'] = BatchNormLayer(net['conv1c'])\n",
" net['voxres2_relu1'] = NonlinearityLayer(net['voxres2_bn1'])\n",
" net['voxres2_conv1'] = Conv3DDNNLayer(net['voxres2_relu1'], 64, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres2_bn2'] = BatchNormLayer(net['voxres2_conv1'])\n",
" net['voxres2_relu2'] = NonlinearityLayer(net['voxres2_bn2'])\n",
" net['voxres2_conv2'] = Conv3DDNNLayer(net['voxres2_relu2'], 64, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres2_out'] = ElemwiseSumLayer([net['conv1c'],\n",
" net['voxres2_conv2']])\n",
" # VoxRes block 3\n",
" net['voxres3_bn1'] = BatchNormLayer(net['voxres2_out'])\n",
" net['voxres3_relu1'] = NonlinearityLayer(net['voxres3_bn1'])\n",
" net['voxres3_conv1'] = Conv3DDNNLayer(net['voxres3_relu1'], 64, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres3_bn2'] = BatchNormLayer(net['voxres3_conv1'])\n",
" net['voxres3_relu2'] = NonlinearityLayer(net['voxres3_bn2'])\n",
" net['voxres3_conv2'] = Conv3DDNNLayer(net['voxres3_relu2'], 64, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres3_out'] = ElemwiseSumLayer([net['voxres2_out'],\n",
" net['voxres3_conv2']])\n",
"\n",
" net['bn4'] = BatchNormLayer(net['voxres3_out'])\n",
" net['relu4'] = NonlinearityLayer(net['bn4'])\n",
" net['conv4'] = Conv3DDNNLayer(net['relu4'], 64, 3, stride=(2, 2, 2),\n",
" pad='same', nonlinearity=identity)\n",
" # VoxRes block 5\n",
" net['voxres5_bn1'] = BatchNormLayer(net['conv4'])\n",
" net['voxres5_relu1'] = NonlinearityLayer(net['voxres5_bn1'])\n",
" net['voxres5_conv1'] = Conv3DDNNLayer(net['voxres5_relu1'], 64, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres5_bn2'] = BatchNormLayer(net['voxres5_conv1'])\n",
" net['voxres5_relu2'] = NonlinearityLayer(net['voxres5_bn2'])\n",
" net['voxres5_conv2'] = Conv3DDNNLayer(net['voxres5_relu2'], 64, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres5_out'] = ElemwiseSumLayer([net['conv4'], net['voxres5_conv2']])\n",
" # VoxRes block 6\n",
" net['voxres6_bn1'] = BatchNormLayer(net['voxres5_out'])\n",
" net['voxres6_relu1'] = NonlinearityLayer(net['voxres6_bn1'])\n",
" net['voxres6_conv1'] = Conv3DDNNLayer(net['voxres6_relu1'], 64, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres6_bn2'] = BatchNormLayer(net['voxres6_conv1'])\n",
" net['voxres6_relu2'] = NonlinearityLayer(net['voxres6_bn2'])\n",
" net['voxres6_conv2'] = Conv3DDNNLayer(net['voxres6_relu2'], 64, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres6_out'] = ElemwiseSumLayer([net['voxres5_out'],\n",
" net['voxres6_conv2']])\n",
"\n",
" net['bn7'] = BatchNormLayer(net['voxres6_out'])\n",
" net['relu7'] = NonlinearityLayer(net['bn7'])\n",
" net['conv7'] = Conv3DDNNLayer(net['relu7'], 128, 3, stride=(2, 2, 2),\n",
" pad='same', nonlinearity=identity)\n",
"\n",
" # VoxRes block 8\n",
" net['voxres8_bn1'] = BatchNormLayer(net['conv7'])\n",
" net['voxres8_relu1'] = NonlinearityLayer(net['voxres8_bn1'])\n",
" net['voxres8_conv1'] = Conv3DDNNLayer(net['voxres8_relu1'], 128, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres8_bn2'] = BatchNormLayer(net['voxres8_conv1'])\n",
" net['voxres8_relu2'] = NonlinearityLayer(net['voxres8_bn2'])\n",
" net['voxres8_conv2'] = Conv3DDNNLayer(net['voxres8_relu2'], 128, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres8_out'] = ElemwiseSumLayer([net['conv7'], net['voxres8_conv2']])\n",
" # VoxRes block 9\n",
" net['voxres9_bn1'] = BatchNormLayer(net['voxres8_out'])\n",
" net['voxres9_relu1'] = NonlinearityLayer(net['voxres9_bn1'])\n",
" net['voxres9_conv1'] = Conv3DDNNLayer(net['voxres9_relu1'], 128, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres9_bn2'] = BatchNormLayer(net['voxres9_conv1'])\n",
" net['voxres9_relu2'] = NonlinearityLayer(net['voxres9_bn2'])\n",
" net['voxres9_conv2'] = Conv3DDNNLayer(net['voxres9_relu2'], 128, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres9_out'] = ElemwiseSumLayer([net['voxres8_out'],\n",
" net['voxres9_conv2']])\n",
"\n",
" net['pool10'] = Pool3DDNNLayer(net['voxres9_out'], 7)\n",
" net['fc11'] = DenseLayer(net['pool10'], 128)\n",
" net['prob'] = DenseLayer(net['fc11'], 2, nonlinearity=softmax)\n",
" \n",
" return net"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Network initialization with pretrained weights"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"net = build_net()\n",
"test_prediction = lasagne.layers.get_output(net['prob'], deterministic=True)\n",
"test_fn = theano.function([input_var], test_prediction)\n",
"\n",
"with open('data/resnet_weights.pkl', 'rb') as f:\n",
" weights = pickle.load(f)\n",
" lasagne.layers.set_all_param_values(net['prob'], weights)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load data and split into the same validation folds"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"metadata = pd.read_csv('data/metadata.csv')\n",
"smc_mask = ((metadata.Label == 'Normal') | (\n",
" metadata.Label == 'AD')).values.astype('bool')\n",
"data = np.zeros((smc_mask.sum(), 1, 110, 110, 110), dtype='float32')\n",
"\n",
"for it, im in tqdm(enumerate(metadata[smc_mask].Path.values),\n",
" total=smc_mask.sum(), desc='Reading MRI to memory'):\n",
" mx = nib.load(im).get_data().max(axis=0).max(axis=0).max(axis=0)\n",
" data[it, 0, :, :, :] = np.array(nib.load(im).get_data()) / mx\n",
"\n",
"target = (metadata[smc_mask].Label != 'Normal').values.astype('int32')\n",
"\n",
"cv = StratifiedKFold(target, n_folds=5, random_state=0, shuffle=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Select sample for heatmap plot"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"for fold, (train_index, test_index) in enumerate(cv):\n",
" X_train, y_train = data[train_index], target[train_index]\n",
" X_test, y_test = data[test_index], target[test_index]\n",
"\n",
" for it, img in enumerate(X_test):\n",
" print(test_fn(img.reshape((1, 1, 110, 110, 110))).reshape(-1,),\n",
" y_test[it])\n",
" break"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"res = occlusion_heatmap(net, X_test[2].reshape(1, 1, 110, 110, 110), 0)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Plot heatmap as overlay"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"plt.figure(figsize=(14, 5))\n",
"plt.subplot(1, 2, 1)\n",
"plt.axis('off')\n",
"matr = rotate(X_test[2][:, :, :, 47].reshape(110, 110), 90)\n",
"plt.imshow(matr, cmap=plt.cm.Greys_r, interpolation=None,\n",
" vmax=1., vmin=0.)\n",
"plt.hold(True)\n",
"matr = rotate(res[:, :, 47, 0].reshape(110, 110), 90)\n",
"plt.imshow(200. * (1 - np.ma.masked_where(matr > .999, matr)),\n",
" interpolation=None, vmax=1., vmin=.0, alpha=.8,\n",
" cmap=plt.cm.viridis_r)\n",
"plt.subplot(1, 2, 2)\n",
"plt.axis('off')\n",
"matr = rotate(X_test[2][:, 57, :, :].reshape(110, 110), 90)\n",
"plt.imshow(matr, cmap=plt.cm.Greys_r, interpolation=None,\n",
" vmax=1., vmin=0.)\n",
"plt.hold(True)\n",
"matr = rotate(res[57, :, :, 0].reshape(110, 110), 90)\n",
"plt.imshow(200. * (1 - np.ma.masked_where(matr > .999, matr)),\n",
" interpolation=None, vmax=1., vmin=.0, alpha=.8,\n",
" cmap=plt.cm.viridis_r)\n",
"cbar = plt.colorbar()\n",
"cbar.ax.tick_params(labelsize=20)\n",
"plt.show()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.4.3"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: scripts/resnet_plot_results.ipynb
================================================
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import seaborn as sns; sns.set(style='whitegrid')\n",
"%matplotlib inline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Plot all folds and splits learning curves"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def plot_curves(results_folder):\n",
" \"\"\"Plot learning curves, accuracies and ROC AUC from epoch.\n",
"\n",
" Parameters\n",
" ----------\n",
" folder : string\n",
" Location of the results .npy files.\n",
" \"\"\"\n",
" for cvrand in range(5):\n",
" for fold in range(5):\n",
" eps = np.load('./{}/{}_{}nm_eps.npy'.format(results_folder,\n",
" cvrand, fold))\n",
" tr_loss = np.load('./{}/{}_{}nm_tr_loss.npy'.format(results_folder,\n",
" cvrand, fold))\n",
" vl_loss = np.load('./{}/{}_{}nm_vl_loss.npy'.format(results_folder,\n",
" cvrand, fold))\n",
" vl_acc = np.load('./{}/{}_{}nm_vl_accs.npy'.format(results_folder,\n",
" cvrand, fold))\n",
" vl_roc = np.load('./{}/{}_{}nm_vl_rocs.npy'.format(results_folder,\n",
" cvrand, fold))\n",
" plt.figure(figsize=(15, 7))\n",
" plt.plot(eps, tr_loss, label='train_loss')\n",
" plt.plot(eps, vl_loss, label='val_loss')\n",
" plt.plot(eps, vl_acc, label='val_accuracy')\n",
" plt.plot(eps, vl_roc, label='val_rocauc')\n",
" plt.ylim((0., 1.))\n",
" plt.title('Nesterov')\n",
" plt.legend(loc=0, frameon=True, framealpha=1)\n",
" plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"plot_curves('./results_resnet/ad_vs_norm')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"plot_curves('./results_resnet/ad_vs_emci')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"plot_curves('./results_resnet/ad_vs_lmci')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"plot_curves('./results_resnet/emci_vs_norm')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"plot_curves('./results_resnet/lmci_vs_norm')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"plot_curves('./results_resnet/lmci_vs_emci')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Plot ROC AUC curves with mean and std"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def plot_auc(results_folder):\n",
" \"\"\"Plot ROC AUC curves with mean and std from epoch.\n",
"\n",
" Parameters\n",
" ----------\n",
" folder : string\n",
" Location of the results .npy files.\n",
" \"\"\"\n",
" eps = np.load('./{}/{}_{}nm_eps.npy'.format(results_folder,\n",
" 0, 0))\n",
" vl_roc = np.zeros(eps.shape)\n",
" for cvrand in range(5):\n",
" for fold in range(5):\n",
" eps = np.load('./{}/{}_{}nm_eps.npy'.format(results_folder,\n",
" cvrand, fold))\n",
" vl_roc = np.vstack((vl_roc,\n",
" np.load('./{}/{}_{}nm_vl_rocs.npy'.format(\n",
" results_folder, cvrand, fold))))\n",
" plt.figure(figsize=(15, 7))\n",
" plt.plot(eps, vl_roc[1:].mean(axis=0), label='val_rocauc')\n",
" plt.fill_between(eps, vl_roc[1:].mean(axis=0) + vl_roc[1:].std(axis=0),\n",
" vl_roc[1:].mean(axis=0) - vl_roc[1:].std(axis=0),\n",
" alpha=.3)\n",
" plt.ylim((0., 1.))\n",
" plt.xlabel('Epoch')\n",
" plt.legend(loc=0, frameon=True, framealpha=1)\n",
" plt.show()\n",
" print('AUC = {:.5f} +/- {:.5f}'.format(vl_roc[1:].mean(axis=0)[-1], vl_roc[1:].std(axis=0)[-1]))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"plot_auc('./results_resnet/ad_vs_norm')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"plot_auc('./results_resnet/ad_vs_emci')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"plot_auc('./results_resnet/ad_vs_lmci')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"plot_auc('./results_resnet/emci_vs_norm')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"plot_auc('./results_resnet/lmci_vs_norm')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"plot_auc('./results_resnet/lmci_vs_emci')"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.4.3"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: scripts/resnet_train.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Notebook for network training"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import pandas as pd\n",
"import numpy as np\n",
"import nibabel as nib\n",
"from tqdm import tqdm\n",
"import logging\n",
"from sklearn.cross_validation import StratifiedKFold\n",
"import lasagne\n",
"import theano\n",
"from lasagne.layers import InputLayer\n",
"from lasagne.layers.dnn import Conv3DDNNLayer\n",
"from lasagne.layers.dnn import Pool3DDNNLayer\n",
"from lasagne.layers import BatchNormLayer\n",
"from lasagne.layers import NonlinearityLayer\n",
"from lasagne.layers import ElemwiseSumLayer\n",
"from lasagne.layers import DenseLayer\n",
"from lasagne.nonlinearities import identity, softmax\n",
"from lasagne.objectives import categorical_crossentropy\n",
"import theano.tensor as T\n",
"import time\n",
"from sklearn.metrics import accuracy_score, roc_auc_score, confusion_matrix\n",
"import sys\n",
"import os"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Batch iteration functions"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"from utils import iterate_minibatches, iterate_minibatches_train"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"input_var = T.tensor5(name='input', dtype='float32')\n",
"target_var = T.ivector()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Network definition"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def build_net():\n",
" \"\"\"Method for VoxResNet Building.\n",
"\n",
" Returns\n",
" -------\n",
" dictionary\n",
" Network dictionary.\n",
" \"\"\"\n",
" net = {}\n",
" net['input'] = InputLayer((None, 1, 110, 110, 110), input_var=input_var)\n",
" net['conv1a'] = Conv3DDNNLayer(net['input'], 32, 3, pad='same',\n",
" nonlinearity=identity)\n",
" net['bn1a'] = BatchNormLayer(net['conv1a'])\n",
" net['relu1a'] = NonlinearityLayer(net['bn1a'])\n",
" net['conv1b'] = Conv3DDNNLayer(net['relu1a'], 32, 3, pad='same',\n",
" nonlinearity=identity)\n",
" net['bn1b'] = BatchNormLayer(net['conv1b'])\n",
" net['relu1b'] = NonlinearityLayer(net['bn1b'])\n",
" net['conv1c'] = Conv3DDNNLayer(net['relu1b'], 64, 3, stride=(2, 2, 2),\n",
" pad='same', nonlinearity=identity)\n",
" # VoxRes block 2\n",
" net['voxres2_bn1'] = BatchNormLayer(net['conv1c'])\n",
" net['voxres2_relu1'] = NonlinearityLayer(net['voxres2_bn1'])\n",
" net['voxres2_conv1'] = Conv3DDNNLayer(net['voxres2_relu1'], 64, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres2_bn2'] = BatchNormLayer(net['voxres2_conv1'])\n",
" net['voxres2_relu2'] = NonlinearityLayer(net['voxres2_bn2'])\n",
" net['voxres2_conv2'] = Conv3DDNNLayer(net['voxres2_relu2'], 64, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres2_out'] = ElemwiseSumLayer([net['conv1c'],\n",
" net['voxres2_conv2']])\n",
" # VoxRes block 3\n",
" net['voxres3_bn1'] = BatchNormLayer(net['voxres2_out'])\n",
" net['voxres3_relu1'] = NonlinearityLayer(net['voxres3_bn1'])\n",
" net['voxres3_conv1'] = Conv3DDNNLayer(net['voxres3_relu1'], 64, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres3_bn2'] = BatchNormLayer(net['voxres3_conv1'])\n",
" net['voxres3_relu2'] = NonlinearityLayer(net['voxres3_bn2'])\n",
" net['voxres3_conv2'] = Conv3DDNNLayer(net['voxres3_relu2'], 64, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres3_out'] = ElemwiseSumLayer([net['voxres2_out'],\n",
" net['voxres3_conv2']])\n",
"\n",
" net['bn4'] = BatchNormLayer(net['voxres3_out'])\n",
" net['relu4'] = NonlinearityLayer(net['bn4'])\n",
" net['conv4'] = Conv3DDNNLayer(net['relu4'], 64, 3, stride=(2, 2, 2),\n",
" pad='same', nonlinearity=identity)\n",
" # VoxRes block 5\n",
" net['voxres5_bn1'] = BatchNormLayer(net['conv4'])\n",
" net['voxres5_relu1'] = NonlinearityLayer(net['voxres5_bn1'])\n",
" net['voxres5_conv1'] = Conv3DDNNLayer(net['voxres5_relu1'], 64, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres5_bn2'] = BatchNormLayer(net['voxres5_conv1'])\n",
" net['voxres5_relu2'] = NonlinearityLayer(net['voxres5_bn2'])\n",
" net['voxres5_conv2'] = Conv3DDNNLayer(net['voxres5_relu2'], 64, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres5_out'] = ElemwiseSumLayer([net['conv4'], net['voxres5_conv2']])\n",
" # VoxRes block 6\n",
" net['voxres6_bn1'] = BatchNormLayer(net['voxres5_out'])\n",
" net['voxres6_relu1'] = NonlinearityLayer(net['voxres6_bn1'])\n",
" net['voxres6_conv1'] = Conv3DDNNLayer(net['voxres6_relu1'], 64, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres6_bn2'] = BatchNormLayer(net['voxres6_conv1'])\n",
" net['voxres6_relu2'] = NonlinearityLayer(net['voxres6_bn2'])\n",
" net['voxres6_conv2'] = Conv3DDNNLayer(net['voxres6_relu2'], 64, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres6_out'] = ElemwiseSumLayer([net['voxres5_out'],\n",
" net['voxres6_conv2']])\n",
"\n",
" net['bn7'] = BatchNormLayer(net['voxres6_out'])\n",
" net['relu7'] = NonlinearityLayer(net['bn7'])\n",
" net['conv7'] = Conv3DDNNLayer(net['relu7'], 128, 3, stride=(2, 2, 2),\n",
" pad='same', nonlinearity=identity)\n",
"\n",
" # VoxRes block 8\n",
" net['voxres8_bn1'] = BatchNormLayer(net['conv7'])\n",
" net['voxres8_relu1'] = NonlinearityLayer(net['voxres8_bn1'])\n",
" net['voxres8_conv1'] = Conv3DDNNLayer(net['voxres8_relu1'], 128, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres8_bn2'] = BatchNormLayer(net['voxres8_conv1'])\n",
" net['voxres8_relu2'] = NonlinearityLayer(net['voxres8_bn2'])\n",
" net['voxres8_conv2'] = Conv3DDNNLayer(net['voxres8_relu2'], 128, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres8_out'] = ElemwiseSumLayer([net['conv7'], net['voxres8_conv2']])\n",
" # VoxRes block 9\n",
" net['voxres9_bn1'] = BatchNormLayer(net['voxres8_out'])\n",
" net['voxres9_relu1'] = NonlinearityLayer(net['voxres9_bn1'])\n",
" net['voxres9_conv1'] = Conv3DDNNLayer(net['voxres9_relu1'], 128, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres9_bn2'] = BatchNormLayer(net['voxres9_conv1'])\n",
" net['voxres9_relu2'] = NonlinearityLayer(net['voxres9_bn2'])\n",
" net['voxres9_conv2'] = Conv3DDNNLayer(net['voxres9_relu2'], 128, 3,\n",
" pad='same', nonlinearity=identity)\n",
" net['voxres9_out'] = ElemwiseSumLayer([net['voxres8_out'],\n",
" net['voxres9_conv2']])\n",
"\n",
" net['pool10'] = Pool3DDNNLayer(net['voxres9_out'], 7)\n",
" net['fc11'] = DenseLayer(net['pool10'], 128)\n",
" net['prob'] = DenseLayer(net['fc11'], 2, nonlinearity=softmax)\n",
"\n",
" return net"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Logging setup\n",
"logging.basicConfig(format='[%(asctime)s] %(message)s',\n",
" datefmt='%d.%m %H:%M:%S',\n",
" level=logging.DEBUG)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Training function definition"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def run_training(first_class, second_class, results_folder,\n",
" num_epochs=70, batchsize=3):\n",
" \"\"\"Iterate minibatches on train subset.\n",
"\n",
" Parameters\n",
" ----------\n",
" first_class : {'AD', 'LMCI', 'EMCI', 'Normal'}\n",
" String label for target == 0.\n",
" second_class : {'AD', 'LMCI', 'EMCI', 'Normal'}\n",
" String label for target == 1.\n",
" results_folder : string\n",
" Folder to store results.\n",
" num_epochs : integer\n",
" Number of epochs for all of the experiments. Default is 70.\n",
" batchsize : integer\n",
" Batchsize for network training. Default is 3.\n",
" \"\"\"\n",
" \n",
" if first_class not in {'AD', 'LMCI', 'EMCI', 'Normal'}:\n",
" msg = \"First class must be 'AD', 'LMCI', 'EMCI' or 'Normal', not {0}\"\n",
" raise ValueError(msg.format(first_class))\n",
" \n",
" if second_class not in {'AD', 'LMCI', 'EMCI', 'Normal'}:\n",
" msg = \"Second class must be 'AD', 'LMCI', 'EMCI' or 'Normal', not {0}\"\n",
" raise ValueError(msg.format(second_class))\n",
" \n",
" if first_class == second_class:\n",
" raise ValueError(\"Class labels should be different\")\n",
" \n",
" if not os.path.exists(results_folder):\n",
" os.makedirs(results_folder)\n",
" \n",
" metadata = pd.read_csv('data/metadata.csv')\n",
" smc_mask = ((metadata.Label == first_class) |\n",
" (metadata.Label == second_class)).values.astype('bool')\n",
" data = np.zeros((smc_mask.sum(), 1, 110, 110, 110), dtype='float32')\n",
"\n",
" for it, im in tqdm(enumerate(metadata[smc_mask].Path.values),\n",
" total=smc_mask.sum(), desc='Reading MRI to memory'):\n",
" mx = nib.load(im).get_data().max(axis=0).max(axis=0).max(axis=0)\n",
" data[it, 0, :, :, :] = np.array(nib.load(im).get_data()) / mx\n",
"\n",
" target = (metadata[smc_mask].Label == second_class).values.astype('int32')\n",
" \n",
" for cvrand in range(5):\n",
" cv = StratifiedKFold(target, n_folds=5, random_state=42 * cvrand,\n",
" shuffle=True)\n",
"\n",
" for fold, (train_index, test_index) in enumerate(cv):\n",
" logging.debug('Starting fold {}'.format(fold))\n",
" X_train, y_train = data[train_index], target[train_index]\n",
" X_test, y_test = data[test_index], target[test_index]\n",
"\n",
" net = build_net()\n",
"\n",
" prediction = lasagne.layers.get_output(net['prob'])\n",
" loss = lasagne.objectives.categorical_crossentropy(prediction,\n",
" target_var)\n",
" loss = loss.mean()\n",
"\n",
" params = lasagne.layers.get_all_params(net['prob'], trainable=True)\n",
" updates = lasagne.updates.nesterov_momentum(loss, params, 0.001)\n",
"\n",
" test_prediction = lasagne.layers.get_output(net['prob'],\n",
" deterministic=True)\n",
" test_loss = categorical_crossentropy(test_prediction, target_var)\n",
" test_loss = test_loss.mean()\n",
"\n",
" train_fn = theano.function([input_var, target_var], loss,\n",
" updates=updates)\n",
" val_fn = theano.function([input_var, target_var], test_loss)\n",
" test_fn = theano.function([input_var], test_prediction)\n",
"\n",
" logging.debug(\"Done building net\")\n",
"\n",
" eps = []\n",
" tr_losses = []\n",
" val_losses = []\n",
" val_accs = []\n",
" val_rocs = []\n",
"\n",
" logging.debug(\"Starting training...\")\n",
" den = X_train.shape[0] / batchsize\n",
" for epoch in range(num_epochs):\n",
" train_err = 0\n",
" train_batches = 0\n",
" start_time = time.time()\n",
" t = tqdm(iterate_minibatches_train(X_train, y_train, batchsize,\n",
" shuffle=True),\n",
" total=int(den),\n",
" desc='Epoch {}, Loss = inf'.format(epoch + 1))\n",
" for batch in t:\n",
" inputs, targets = batch\n",
" train_err += train_fn(inputs, targets)\n",
" train_batches += 1\n",
" ls = train_err / train_batches\n",
" t.set_description('Epoch {}, Loss = {:.5f}'.format(epoch +\n",
" 1, ls))\n",
"\n",
" val_err = 0\n",
" val_batches = 0\n",
" preds = []\n",
" targ = []\n",
" for batch in iterate_minibatches(X_test, y_test, batchsize,\n",
" shuffle=False):\n",
" inputs, targets = batch\n",
" err = val_fn(inputs, targets)\n",
" val_err += err\n",
" val_batches += 1\n",
" out = test_fn(inputs)\n",
" [preds.append(i) for i in out]\n",
" [targ.append(i) for i in targets]\n",
"\n",
" logging.debug(\"Epoch {} done\".format(epoch + 1))\n",
" print(\"Epoch {} of {} took {:.3f}s\".format(\n",
" epoch + 1, num_epochs, time.time() - start_time),\n",
" flush=True)\n",
" print(\" training loss:\\t\\t{:.7f}\".format(\n",
" train_err / train_batches), flush=True)\n",
" print(\" validation loss:\\t\\t{:.7f}\".format(\n",
" val_err / val_batches), flush=True)\n",
" print(\" validation accuracy:\\t\\t{:.7f}\".format(\n",
" accuracy_score(np.array(targ),\n",
" np.array(preds).argmax(axis=1))),\n",
" flush=True)\n",
" print(\" validation auc:\\t\\t{:.7f}\".format(\n",
" roc_auc_score(np.array(targ),\n",
" np.array(preds)[:, 1])), flush=True)\n",
"\n",
" eps.append(epoch)\n",
" tr_losses.append(train_err / train_batches)\n",
" val_losses.append(val_err / val_batches)\n",
" val_accs.append(accuracy_score(np.array(targ),\n",
" np.array(preds).argmax(axis=1)))\n",
" val_rocs.append(roc_auc_score(np.array(targ),\n",
" np.array(preds)[:, 1]))\n",
"\n",
"\n",
" np.save('./{}/{}_{}nm_eps.npy'.format(results_folder, \n",
" cvrand, fold),\n",
" np.array(eps))\n",
" np.save('./{}/{}_{}nm_tr_loss.npy'.format(results_folder, \n",
" cvrand, fold),\n",
" np.array(tr_losses))\n",
" np.save('./{}/{}_{}nm_vl_loss.npy'.format(results_folder, \n",
" cvrand, fold),\n",
" np.array(val_losses))\n",
" np.save('./{}/{}_{}nm_vl_accs.npy'.format(results_folder, \n",
" cvrand, fold),\n",
" np.array(val_accs))\n",
" np.save('./{}/{}_{}nm_vl_rocs.npy'.format(results_folder, \n",
" cvrand, fold),\n",
" np.array(val_rocs))\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Run training and save results"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"run_training('AD', 'Normal', './results_resnet/ad_vs_norm')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"run_training('AD', 'EMCI', './results_resnet/ad_vs_emci')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"run_training('AD', 'LMCI', './results_resnet/ad_vs_lmci')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"run_training('EMCI', 'Normal', './results_resnet/emci_vs_norm')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"run_training('LMCI', 'Normal', './results_resnet/lmci_vs_norm')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"run_training('LMCI', 'EMCI', './results_resnet/lmci_vs_emci')"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.4.3"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: scripts/utils.py
================================================
import numpy as np
def iterate_minibatches_train(inputs, targets, batchsize, shuffle=False):
"""Iterate minibatches on train subset.
Parameters
----------
inputs : numpy.ndarray
Numpy array of input images.
targets : numpy.ndarray
Numpy array of binary labels.
batchsize : integer
Size of the output array batches.
shuffle : bool, optional
Whether to shuffle input before sampling. Default is False.
Returns
-------
numpy.ndarray, numpy.ndarray
inputs, targets for given batch.
"""
assert len(inputs) == len(targets)
indices = np.arange(len(inputs))
if shuffle:
np.random.shuffle(indices)
m_len = np.min([sum(targets == 1), sum(targets == 0)])
targets = targets[indices]
pos = inputs[indices][np.where(targets == 1)[0][:m_len]]
neg = inputs[indices][np.where(targets == 0)[0][:m_len]]
pos_t = targets[np.where(targets == 1)[0][:m_len]]
neg_t = targets[np.where(targets == 0)[0][:m_len]]
inputs = np.insert(pos, np.arange(len(neg)), neg, axis=0)
targets = np.insert(pos_t, np.arange(len(neg_t)), neg_t, axis=0)
assert len(inputs) == len(targets)
indices = np.arange(len(inputs))
if shuffle:
np.random.shuffle(indices)
if batchsize > len(indices):
sys.stderr.write('BatchSize out of index size')
batchsize = len(indices)
for start_idx in range(0, len(inputs) - batchsize + 1, batchsize):
if shuffle:
excerpt = indices[start_idx:start_idx + batchsize]
else:
excerpt = slice(start_idx, start_idx + batchsize)
yield inputs[excerpt], targets[excerpt]
def iterate_minibatches(inputs, targets, batchsize, shuffle=False):
"""Iterate minibatches.
Parameters
----------
inputs : numpy.ndarray
Numpy array of input images.
targets : numpy.ndarray
Numpy array of class labels.
batchsize : integer
Size of the output array batches.
shuffle : bool, optional
Whether to shuffle input before sampling. Default is False.
Returns
-------
numpy.ndarray, numpy.ndarray
inputs, targets for given batch.
"""
if shuffle:
indices = np.arange(len(inputs))
np.random.shuffle(indices)
for start_idx in range(0, len(inputs) - batchsize + 1, batchsize):
if shuffle:
excerpt = indices[start_idx:start_idx + batchsize]
else:
excerpt = slice(start_idx, start_idx + batchsize)
yield inputs[excerpt], targets[excerpt]
================================================
FILE: scripts/vgg_like_plot_results.ipynb
================================================
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import os\n",
"\n",
"import numpy as np\n",
"\n",
"import matplotlib.pyplot as plt\n",
"import seaborn as sns\n",
"%matplotlib inline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"-----"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## AD vs Normal"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def plot(sm_folder, s):\n",
" cv_results = []\n",
" for i in range(5):\n",
" # loading cv results\n",
" with np.load(sm_folder + s + str(i) + '.npz') as f:\n",
" cv_results.append([f['arr_%d' % i]\n",
" for i in range(len(f.files))][0])\n",
" cv_results = np.array(cv_results)\n",
"\n",
" tmplate = 2 # accuracy\n",
" plt.figure()\n",
" plt.plot(cv_results[:, :, tmplate, :].mean(axis=0).mean(axis=0))\n",
" y1 = cv_results[:, :, tmplate, :].mean(axis=0).mean(\n",
" axis=0) + cv_results[:, :, tmplate, :].reshape(25, -1).std(axis=0)\n",
" y2 = cv_results[:, :, tmplate, :].mean(axis=0).mean(\n",
" axis=0) - cv_results[:, :, tmplate, :].reshape(25, -1).std(axis=0)\n",
" plt.fill_between(np.arange(len(y1)), y1, y2, alpha=0.2)\n",
" plt.title('mean accuracy ' + s[11:-1])\n",
" plt.ylabel('accuracy')\n",
" plt.xlabel('epoch')\n",
" # plt.savefig('mean_roc_auc__5_fold_'+ LABEL_1+'_vs_'+ LABEL_2 +'_'+str(i)+'.png')\n",
" print('Accuracy: ' + str(cv_results[:, :, tmplate, :].mean(axis=0).mean(\n",
" axis=0)[-1]) + ' ± ' + str(\n",
" cv_results[:, :, tmplate, :].reshape(25, -1).std(axis=0)[-1]))\n",
"\n",
" tmplate = 3 # roc auc\n",
" plt.figure()\n",
" plt.plot(np.array(cv_results)[:, :, tmplate, :].mean(axis=0).mean(axis=0))\n",
" y1 = np.array(cv_results)[:, :, tmplate, :].mean(axis=0).mean(\n",
" axis=0) + np.array(cv_results)[:, :, tmplate, :].reshape(25, -1).std(\n",
" axis=0)\n",
" y2 = np.array(cv_results)[:, :, tmplate, :].mean(axis=0).mean(\n",
" axis=0) - np.array(cv_results)[:, :, tmplate, :].reshape(25, -1).std(\n",
" axis=0)\n",
" plt.fill_between(np.arange(len(y1)), y1, y2, alpha=0.2)\n",
" plt.title('mean roc_auc ' + s[11:-1])\n",
" plt.ylabel('roc_auc')\n",
" plt.xlabel('epoch')\n",
" # plt.savefig('mean_roc_auc__5_fold_'+ LABEL_1+'_vs_'+ LABEL_2 +'_'+str(i)+'.png')\n",
" print('ROC AUC: ' + str(\n",
" np.array(cv_results)[:, :, tmplate, :].mean(axis=0).mean(\n",
" axis=0)[-1]) + ' ± ' + str(\n",
" np.array(cv_results)[:, :, tmplate, :].reshape(25, -1).std(axis=0)[-1]))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"_____"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"### AD vs Normal"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"sm_folder = './results/ad_vs_norm/'\n",
"s = 'cv_results_AD_vs_Normal_'\n",
"plot(sm_folder, s)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"_____"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### AD LMCI"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"sm_folder = './results_cnn/ad_vs_lmci/'\n",
"s = 'cv_results_AD_vs_LMCI_'\n",
"plot(sm_folder, s)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"______"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### AD EMCI"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"s = 'cv_results_AD_vs_EMCI_'\n",
"sm_folder = './results_cnn/ad_vs_emci/'\n",
"plot(sm_folder, s)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"_______"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Normal vs EMCI"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"s = 'cv_results_Normal_vs_EMCI_'\n",
"sm_folder = './results_cnn/norm_vs_emci/'\n",
"plot(sm_folder, s)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"_______"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Normal vs LMCI"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"s = 'cv_results_Normal_vs_LMCI_'\n",
"sm_folder = './results_cnn/norm_vs_lmci/'\n",
"plot(sm_folder, s)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"______"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### EMCI vs LMCI"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"s = 'cv_results_EMCI_vs_LMCI_'\n",
"sm_folder = './results_cnn/emci_vs_lmci/'\n",
"plot(sm_folder, s)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.4.3"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: scripts/vgg_like_train.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Notebook for reproduce paper results\n",
"\n",
"##### content\n",
"\n",
"- [Batch iteration functions](#Batch-iteration-functions)\n",
"- [Train functions](#Train-functions)\n",
"- [Network architecture](#network-architecture)\n",
"- [Cross_validation function](#cross-validation)\n",
"- [Cross-validation one_vs_one - run](#Cross-validation-one_vs_one)\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import os\n",
"import gc\n",
"import sys\n",
"import time\n",
"import datetime\n",
"import traceback\n",
"from collections import OrderedDict\n",
"\n",
"import numpy as np\n",
"import pandas as pd\n",
"import nibabel as nib\n",
"from tqdm import *\n",
"\n",
"from sklearn.metrics import confusion_matrix, accuracy_score, roc_auc_score\n",
"from sklearn.cross_validation import train_test_split\n",
"from sklearn.cross_validation import StratifiedKFold\n",
"\n",
"import theano\n",
"import theano.tensor as T\n",
"import lasagne\n",
"from lasagne.layers import InputLayer\n",
"from lasagne.layers.dnn import Conv3DDNNLayer\n",
"from lasagne.layers.dnn import Pool3DDNNLayer\n",
"from lasagne.layers import BatchNormLayer\n",
"from lasagne.layers import NonlinearityLayer\n",
"from lasagne.layers import ElemwiseSumLayer\n",
"from lasagne.layers import DenseLayer\n",
"from lasagne.nonlinearities import rectify, softmax, identity\n",
"from lasagne.layers import set_all_param_values\n",
"from lasagne.layers import DropoutLayer\n",
"\n",
"import matplotlib.pyplot as plt\n",
"import seaborn as sns"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"PATH_TO_REP = 'data/' # adni_data\n",
"\n",
"inp_shape = (None, 1, 110, 110, 110)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"input_var = T.TensorType('float32', (False,) * 5)('inputs')\n",
"target_var = T.ivector('targets')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"____"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Batch iteration functions\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"from utils import iterate_minibatches, iterate_minibatches_train"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Train functions"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def get_train_functions(nn, updates_method=lasagne.updates.nesterov_momentum,\n",
" _lr=0.00001):\n",
" \"\"\"\n",
" Return functions for training, validation network and predicting answers.\n",
"\n",
" Parameters\n",
" ----------\n",
" nn : lasagne.Layer\n",
" network last layer\n",
"\n",
" updates_method : function\n",
" like in lasagne.objectives or function from there\n",
"\n",
" _lr : float\n",
" learning rate which relate with the updates_method\n",
"\n",
" Returns\n",
" -------\n",
" train_fn : theano.function\n",
" Train network function.\n",
" val_fn : theano.function\n",
" Validation function.\n",
" pred_fn : theano.function\n",
" Function for get predicts from network.\n",
" \"\"\"\n",
" prediction = lasagne.layers.get_output(nn)\n",
" loss = lasagne.objectives.categorical_crossentropy(prediction, target_var)\n",
" loss = loss.mean()\n",
" params = lasagne.layers.get_all_params(nn, trainable=True)\n",
" updates = updates_method(loss, params, learning_rate=_lr)\n",
"\n",
" test_prediction = lasagne.layers.get_output(nn, deterministic=True)\n",
" test_loss = lasagne.objectives.categorical_crossentropy(test_prediction,\n",
" target_var)\n",
" test_loss = test_loss.mean()\n",
"\n",
" train_fn = theano.function([input_var, target_var], loss, updates=updates)\n",
" val_fn = theano.function([input_var, target_var], test_loss)\n",
" pred_fn = theano.function([input_var], test_prediction)\n",
"\n",
" return train_fn, val_fn, pred_fn"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def train(train_fn, val_fn, test_fn,\n",
" X_train, y_train,\n",
" X_test, y_test,\n",
" LABEL_1, LABEL_2, # labels of the y.\n",
" num_epochs=100, batchsize=5,\n",
" dict_of_paths={'output': '1.txt', 'picture': '1.png',\n",
" 'report': 'report.txt'},\n",
" report='''trained next architecture, used some\n",
" optimizstion method with learning rate...''',\n",
" architecture='nn=...'):\n",
" \"\"\"\n",
" Iterate minibatches on train subset and validate results on test subset.\n",
"\n",
" Parameters\n",
" ----------\n",
" train_fn : theano.function\n",
" Train network function.\n",
" val_fn : theano.function\n",
" Validation network function.\n",
" test_fn : theano.function\n",
" Function for get predicts from network.\n",
" X_train : numpy array\n",
" X train subset.\n",
" y_train : numpy array\n",
" Y train subset.\n",
" X_test : numpy array\n",
" X test subset.\n",
" y_test : numpy array\n",
" Y test subset.\n",
" LABEL_1 : {'AD', 'LMCI', 'EMCI', 'Normal'}\n",
" String label for target == 0.\n",
" LABEL_2 : {'AD', 'LMCI', 'EMCI', 'Normal'}\n",
" String label for target == 1.\n",
" dict_of_paths : dictionary\n",
" Names of files to store results.\n",
" report : string\n",
" Some comments which will saved into report after ending of training.\n",
" num_epochs : integer\n",
" Number of epochs for all of the experiments. Default is 100.\n",
" batchsize : integer\n",
" Batchsize for network training. Default is 5.\n",
"\n",
" Returns\n",
" -------\n",
" tr_losses : numpy.array\n",
" Array with loss values on train.\n",
" val_losses : numpy.array\n",
" Array with loss values on test.\n",
" val_accs : numpy.array\n",
" Array with accuracy values on test.\n",
" rocs : numpy.array\n",
" Array with roc auc values on test.\n",
"\n",
" \"\"\"\n",
"\n",
" eps = []\n",
" tr_losses = []\n",
" val_losses = []\n",
" val_accs = []\n",
" rocs = []\n",
"\n",
" FILE_PATH = dict_of_paths['output']\n",
" PICTURE_PATH = dict_of_paths['picture']\n",
" REPORT_PATH = dict_of_paths['report']\n",
"\n",
" # here we written outputs on each step (val and train losses, accuracy, auc)\n",
" with open(FILE_PATH, 'w') as f:\n",
" f.write('\\n----------\\n\\n' + str(datetime.datetime.now())[:19])\n",
" f.write('\\n' + LABEL_1 + '-' + LABEL_2 + '\\n')\n",
" f.close()\n",
"\n",
" # starting training\n",
" print(\"Starting training...\", flush=True)\n",
" den = X_train.shape[0] / batchsize\n",
" for epoch in range(num_epochs):\n",
" train_err = 0\n",
" train_batches = 0\n",
" start_time = time.time()\n",
" for batch in iterate_minibatches_train(X_train, y_train, batchsize,\n",
" shuffle=True):\n",
" inputs, targets = batch\n",
" train_err += train_fn(inputs, targets)\n",
" train_batches += 1\n",
"\n",
" val_err = 0\n",
" val_batches = 0\n",
" preds = []\n",
" targ = []\n",
" for batch in iterate_minibatches(X_test, y_test, batchsize,\n",
" shuffle=False):\n",
" inputs, targets = batch\n",
" err = val_fn(inputs, targets)\n",
" val_err += err\n",
" val_batches += 1\n",
" out = test_fn(inputs)\n",
" [preds.append(i) for i in out]\n",
" [targ.append(i) for i in targets]\n",
"\n",
" preds_tst = np.array(preds).argmax(axis=1)\n",
" ##\n",
" ## output\n",
" print(\"Epoch {} of {} took {:.3f}s\".format(epoch + 1,\n",
" num_epochs,\n",
" time.time() - start_time),\n",
" flush=True)\n",
" print(\" training loss:\\t\\t{:.7f}\".format(train_err / train_batches),\n",
" flush=True)\n",
" print(\" validation loss:\\t\\t{:.7f}\".format(val_err / val_batches),\n",
" flush=True)\n",
" print(' validation accuracy:\\t\\t{:.7f}'.format(\n",
" accuracy_score(np.array(targ),\n",
" preds_tst)), flush=True)\n",
" print('Confusion matrix for test:', flush=True)\n",
" print(confusion_matrix(np.array(targ), np.array(preds).argmax(axis=1)),\n",
" flush=True)\n",
" rcs = roc_auc_score(np.array(targ), np.array(preds)[:, 1])\n",
" sys.stderr.write('Pairwise ROC_AUCs: ' + str(rcs))\n",
" print('')\n",
"\n",
" with open(FILE_PATH, 'a') as f:\n",
" f.write(\"\\nEpoch {} of {} took {:.3f}s\".format(epoch + 1,\n",
" num_epochs,\n",
" time.time() - start_time))\n",
" f.write(\n",
" \"\\n training loss:\\t\\t{:.7f}\".format(train_err / train_batches))\n",
" f.write(\n",
" \"\\n validation loss:\\t\\t{:.7f}\".format(val_err / val_batches))\n",
" f.write('\\n validation accuracy:\\t\\t{:.7f}'.format(\n",
" accuracy_score(np.array(targ),\n",
" np.array(preds).argmax(axis=1))))\n",
"\n",
" f.write('\\n Pairwise ROC_AUCs:' + str(rcs) + '\\n')\n",
" f.close()\n",
" ## output\n",
" ## saving results\n",
" eps.append(epoch + 1)\n",
" tr_losses.append(train_err / train_batches)\n",
" val_losses.append(val_err / val_batches)\n",
" val_accs.append(\n",
" accuracy_score(np.array(targ), np.array(preds).argmax(axis=1)))\n",
" rocs.append(rcs)\n",
"\n",
" print('ended!')\n",
"\n",
" ### and save plots\n",
" plt.figure(figsize=(15, 10))\n",
" plt.subplot(2, 2, 1)\n",
" plt.title('Loss ' + LABEL_1 + ' vs ' + LABEL_2)\n",
" plt.xlabel('Epoch')\n",
" plt.ylim((0, 3))\n",
" plt.ylabel('Loss')\n",
" plt.plot(eps, tr_losses, label='train')\n",
" plt.plot(eps, val_losses, label='validation')\n",
" plt.legend(loc=0)\n",
" #\n",
" plt.subplot(2, 2, 2)\n",
" plt.title('Accuracy ' + LABEL_1 + ' vs ' + LABEL_2)\n",
" plt.xlabel('Epoch')\n",
" plt.ylabel('Accuracy')\n",
" plt.plot(eps, val_accs, label='validation accuracy')\n",
" plt.legend(loc=0)\n",
" #\n",
" plt.subplot(2, 2, 3)\n",
" plt.title('AUC ' + LABEL_1 + ' vs ' + LABEL_2)\n",
" plt.xlabel('Epoch')\n",
" plt.ylabel('AUC')\n",
" plt.plot(eps, np.array(rocs), label='validation auc')\n",
" plt.legend(loc=0)\n",
" #\n",
" plt.subplot(2, 2, 4)\n",
" plt.title('architecture')\n",
" plt.axis('off')\n",
" plt.text(0, -0.1, architecture, fontsize=7, )\n",
" plt.savefig(PICTURE_PATH)\n",
" ###########\n",
"\n",
" # write that trainig was ended\n",
" with open(FILE_PATH, 'a') as f:\n",
" f.write('\\nended at ' + str(datetime.datetime.now())[:19] + '\\n \\n')\n",
" f.close()\n",
"\n",
" # write report\n",
" with open(REPORT_PATH, 'a') as f:\n",
" f.write(\n",
" '\\n классификация ' + LABEL_1 + ' vs ' + LABEL_2 + '\\n' + report)\n",
" # f.write(architecture)\n",
" f.write('final results are:')\n",
" f.write('\\n tr_loss: ' + str(tr_losses[-1]) + '\\n val_loss: ' + \\\n",
" str(val_losses[-1]) + '\\n val_acc; ' + str(val_accs[-1]) + \\\n",
" '\\n val_roc_auc: ' + str(rocs[-1]))\n",
" f.write('\\nresults has been saved in files:\\n')\n",
" f.write(FILE_PATH + '\\n')\n",
" f.write(PICTURE_PATH + '\\n')\n",
" f.write('\\n ___________________ \\n\\n\\n')\n",
" f.close()\n",
"\n",
" return tr_losses, val_losses, val_accs, rocs"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"______"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### network architecture"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def build_net():\n",
" \"\"\"Method for VGG like net Building.\n",
"\n",
" Returns\n",
" -------\n",
" nn : lasagne.layer\n",
" Network.\n",
" \"\"\"\n",
" nn = InputLayer(inp_shape, input_var=input_var)\n",
"\n",
" nn = Conv3DDNNLayer(nn, 8, 3)\n",
" nn = Conv3DDNNLayer(nn, 8, 3, nonlinearity=identity)\n",
" nn = NonlinearityLayer(nn)\n",
" nn = Pool3DDNNLayer(nn, 2)\n",
"\n",
" nn = Conv3DDNNLayer(nn, 16, 3)\n",
" nn = Conv3DDNNLayer(nn, 16, 3, nonlinearity=identity)\n",
" nn = NonlinearityLayer(nn)\n",
" nn = Pool3DDNNLayer(nn, 2)\n",
"\n",
" nn = Conv3DDNNLayer(nn, 32, 3)\n",
" nn = Conv3DDNNLayer(nn, 32, 3)\n",
" nn = Conv3DDNNLayer(nn, 32, 3, nonlinearity=identity)\n",
" nn = NonlinearityLayer(nn)\n",
" nn = Pool3DDNNLayer(nn, 2)\n",
"\n",
" nn = Conv3DDNNLayer(nn, 64, 3)\n",
" nn = Conv3DDNNLayer(nn, 64, 3)\n",
" nn = Conv3DDNNLayer(nn, 64, 3, nonlinearity=identity)\n",
" nn = NonlinearityLayer(nn)\n",
" nn = Pool3DDNNLayer(nn, 2)\n",
"\n",
" nn = DenseLayer(nn, num_units=128)\n",
" nn = BatchNormLayer(nn)\n",
" nn = DropoutLayer(nn, p=0.7)\n",
"\n",
" nn = DenseLayer(nn, num_units=64)\n",
"\n",
" nn = DenseLayer(nn, num_units=2,\n",
" nonlinearity=lasagne.nonlinearities.softmax)\n",
" return nn\n",
"\n",
"\n",
"# writing architecture in report\n",
"architecture = '''\n",
"nn = InputLayer(inp_shape, input_var=input_var)\n",
"\n",
"nn = Conv3DDNNLayer(nn, 8, 3)\n",
"nn = Conv3DDNNLayer(nn, 8, 3, nonlinearity=identity)\n",
"nn = NonlinearityLayer(nn)\n",
"nn = Pool3DDNNLayer(nn, 2)\n",
"\n",
"nn = Conv3DDNNLayer(nn, 16, 3)\n",
"nn = Conv3DDNNLayer(nn, 16, 3, nonlinearity=identity)\n",
"nn = NonlinearityLayer(nn)\n",
"nn = Pool3DDNNLayer(nn, 2)\n",
"\n",
"nn = Conv3DDNNLayer(nn, 32, 3)\n",
"nn = Conv3DDNNLayer(nn, 32, 3)\n",
"nn = Conv3DDNNLayer(nn, 32, 3, nonlinearity=identity)\n",
"nn = NonlinearityLayer(nn)\n",
"nn = Pool3DDNNLayer(nn, 2)\n",
"\n",
"nn = Conv3DDNNLayer(nn, 64, 3)\n",
"nn = Conv3DDNNLayer(nn, 64, 3)\n",
"nn = Conv3DDNNLayer(nn, 64, 3, nonlinearity=identity)\n",
"nn = NonlinearityLayer(nn)\n",
"nn = Pool3DDNNLayer(nn, 2)\n",
"\n",
"nn = DenseLayer(nn, num_units=128)\n",
"nn = BatchNormLayer(nn)\n",
"nn = DropoutLayer(nn, p=0.7)\n",
"\n",
"nn = DenseLayer(nn, num_units=64)\n",
"\n",
"nn = DenseLayer(nn, num_units=2, nonlinearity=lasagne.nonlinearities.softmax)\n",
"\n",
"'''"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"_____"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### cross validation"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"rnd_states = [14, 11, 1993, 19931411, 14111993]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def run_cross_validation(LABEL_1, LABEL_2, results_folder):\n",
" \"\"\"\n",
" Method for cross-validation.\n",
" Takes two labels, reading data, prepair data with this labels for trainig.\n",
"\n",
" Parameters\n",
" ----------\n",
" LABEL_1 : {'AD', 'LMCI', 'EMCI', 'Normal'}\n",
" String label for target == 0.\n",
" LABEL_2 : {'AD', 'LMCI', 'EMCI', 'Normal'}\n",
" String label for target == 1.\n",
" results_folder : string\n",
" Folder to store results.\n",
"\n",
" Returns\n",
" -------\n",
" None.\n",
" \"\"\"\n",
" if not os.path.exists(results_folder):\n",
" os.makedirs(results_folder)\n",
"\n",
" # reading data\n",
" gc.collect()\n",
" metadata = pd.read_csv(PATH_TO_REP + 'metadata.csv')\n",
" smc_mask = (\n",
" (metadata.Label == LABEL_1) | (metadata.Label == LABEL_2)).values.astype(\n",
" 'bool')\n",
" y = (metadata[smc_mask].Label == LABEL_1).astype(np.int32).values\n",
" data = np.zeros((smc_mask.sum(), 1, 110, 110, 110), dtype='float32')\n",
" # into memory\n",
" for it, im in tqdm(enumerate(metadata[smc_mask].Path.values),\n",
" total=smc_mask.sum(), desc='Reading MRI to memory'):\n",
" mx = nib.load(im).get_data().max(axis=0).max(axis=0).max(axis=0)\n",
" data[it, 0, :, :, :] = np.array(nib.load(im).get_data()) / mx\n",
"\n",
" # loop by random states (different splitting)\n",
" for i in range(len(rnd_states)):\n",
" counter = 1\n",
" cv_results = []\n",
" skf = StratifiedKFold(y, n_folds=5, random_state=rnd_states[i])\n",
"\n",
" for tr, ts in skf:\n",
" X_train = data[tr]\n",
" X_test = data[ts]\n",
" y_train = y[tr]\n",
" y_test = y[ts]\n",
" # creating folder for random states\n",
" rnd_state_folder = results_folder + 'rnd_state_' + str(i) + '/'\n",
" if not os.path.exists(rnd_state_folder):\n",
" os.makedirs(rnd_state_folder)\n",
"\n",
" dict_of_paths = {\n",
" 'output': rnd_state_folder + 'Exp_CV_' + str(\n",
" counter) + '_' + LABEL_1 + '_vs_' + \\\n",
" LABEL_2 + '_.txt',\n",
" 'picture': rnd_state_folder + 'Exp_CV_' + str(\n",
" counter) + '_' + LABEL_1 + '_vs_' + \\\n",
" LABEL_2 + '_.png',\n",
" 'report': 'report.txt'\n",
" }\n",
"\n",
" report = '\\n' + LABEL_1 + '_vs_' + LABEL_2 + 'cv_fold ' + \\\n",
" str(counter) + ' random state ' + str(i) + \\\n",
" '_\\n' + 'adam, lr=0.000027' + '\\n '\n",
" # building net and training\n",
" nn = build_net()\n",
" train_fn, val_fn, test_fn = get_train_functions(nn,\n",
" updates_method=lasagne.updates.adam,\n",
" _lr=0.000027)\n",
"\n",
" try:\n",
" tr_losses, val_losses, val_accs, rocs = train(train_fn, val_fn,\n",
" test_fn, X_train,\n",
" y_train, X_test,\n",
" y_test, LABEL_1,\n",
" LABEL_2,\n",
" num_epochs=150,\n",
" batchsize=5,\n",
" dict_of_paths=dict_of_paths,\n",
" report=report,\n",
" architecture=architecture)\n",
" cv_results.append((tr_losses, val_losses, val_accs, rocs))\n",
" except Exception as e:\n",
" with open('errors_msg.txt', 'a') as f:\n",
" f.write('Time: ' + str(datetime.datetime.now())[:19] + \\\n",
" '\\n' + str(e) + traceback.format_exc())\n",
"\n",
" counter += 1\n",
" # saving network params\n",
" # np.savez('net_weights'+ str(counter) + str(i) +'.npz',\n",
" # *lasagne.layers.get_all_param_values(nn))\n",
"\n",
" # saving losses, aucs, accuracies\n",
" np.savez(results_folder + 'cv_results_' + LABEL_1 + \\\n",
" '_vs_' + LABEL_2 + '_' + str(i) + '.npz', np.array(cv_results))\n",
"\n",
" # plotting mean roc_auc and with losses by random_state\n",
" plt.figure()\n",
" plt.plot(np.array(cv_results)[:, 3, :].mean(axis=0))\n",
" y1 = np.array(cv_results)[:, 3, :].mean(axis=0) + np.array(cv_results)[\n",
" :, 3, :].std(axis=0)\n",
" y2 = np.array(cv_results)[:, 3, :].mean(axis=0) - np.array(cv_results)[\n",
" :, 3, :].std(axis=0)\n",
" plt.fill_between(np.arange(len(y1)), y1, y2, alpha=0.4)\n",
" plt.title(\n",
" 'mean roc auc' + '_' + str(i) + '_ ' + LABEL_1 + ' vs ' + LABEL_2)\n",
" plt.ylabel('roc_auc')\n",
" plt.xlabel('epoch')\n",
" plt.savefig(\n",
" rnd_state_folder + 'mean_roc_auc_5_fold_cv_' + LABEL_1 + '_vs_' + LABEL_2 + \\\n",
" '_for_rnd_state_' + str(i) + '.png')"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"-------"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Cross-validation one_vs_one"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"run_cross_validation('AD', 'Normal', './results_cnn/ad_vs_norm/')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"run_cross_validation('AD', 'LMCI', './results_cnn/ad_vs_lmci/')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"run_cross_validation('AD', 'EMCI', './results_cnn/ad_vs_emci/')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"run_cross_validation('Normal', 'EMCI', './results_cnn/norm_vs_emci/')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"run_cross_validation('Normal', 'LMCI', './results_cnn/norm_vs_lmci/')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"run_cross_validation('EMCI', 'LMCI', './results_cnn/emci_vs_lmci/')"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.0"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: test_scripts/test_theano.py
================================================
#!/usr/bin/env python
from theano import function, config, shared, sandbox
import theano.tensor as T
import numpy
import time
vlen = 10 * 30 * 768 # 10 x #cores x # threads per core
iters = 1000
rng = numpy.random.RandomState(22)
x = shared(numpy.asarray(rng.rand(vlen), config.floatX))
f = function([], T.exp(x))
print(f.maker.fgraph.toposort())
t0 = time.time()
for i in range(iters):
r = f()
t1 = time.time()
print("Looping %d times took %f seconds" % (iters, t1 - t0))
print("Result is %s" % (r,))
if numpy.any([isinstance(x.op, T.Elemwise) for x in f.maker.fgraph.toposort()]):
print('Used the cpu')
else:
print('Used the gpu')
gitextract_1lj34b5c/
├── .gitignore
├── .theanorc
├── Dockerfile
├── README.md
├── jupyter/
│ └── jupyter_notebook_config.py
├── scripts/
│ ├── README.md
│ ├── data/
│ │ ├── FirstADSubject.nii
│ │ └── metadata.csv
│ ├── requirements.txt
│ ├── resnet_plot_heatmap.ipynb
│ ├── resnet_plot_results.ipynb
│ ├── resnet_train.ipynb
│ ├── utils.py
│ ├── vgg_like_plot_results.ipynb
│ └── vgg_like_train.ipynb
└── test_scripts/
└── test_theano.py
SYMBOL INDEX (2 symbols across 1 files) FILE: scripts/utils.py function iterate_minibatches_train (line 3) | def iterate_minibatches_train(inputs, targets, batchsize, shuffle=False): function iterate_minibatches (line 50) | def iterate_minibatches(inputs, targets, batchsize, shuffle=False):
Condensed preview — 16 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (111K chars).
[
{
"path": ".gitignore",
"chars": 1045,
"preview": "# Byte-compiled / optimized / DLL files\n__pycache__/\n*.py[cod]\n*$py.class\n\n# C extensions\n*.so\n\n# Distribution / packagi"
},
{
"path": ".theanorc",
"chars": 35,
"preview": "[global]\ndevice=gpu\nfloatX=float32\n"
},
{
"path": "Dockerfile",
"chars": 821,
"preview": "FROM nvidia/cuda:7.5-cudnn5-devel\n\nMAINTAINER Safiullin Amir amir147@rambler.ru\n\nRUN apt-get clean && apt-get update\nRUN"
},
{
"path": "README.md",
"chars": 1430,
"preview": "## Residual and Plain Convolutional Neural Networks for 3D Brain MRI Classification\n\nhttps://arxiv.org/abs/1701.06643\n\nA"
},
{
"path": "jupyter/jupyter_notebook_config.py",
"chars": 21300,
"preview": "# Configuration file for jupyter-notebook.\n\n#---------------------------------------------------------------------------"
},
{
"path": "scripts/README.md",
"chars": 774,
"preview": "## Residual and Plain Convolutional Neural Networks for 3D Brain MRI Classification\n\n### Rerun experiments for training\n"
},
{
"path": "scripts/data/FirstADSubject.nii",
"chars": 0,
"preview": ""
},
{
"path": "scripts/data/metadata.csv",
"chars": 42,
"preview": "Label,Path\n\"AD\",\"data/FirstADSubject.nii\"\n"
},
{
"path": "scripts/requirements.txt",
"chars": 512,
"preview": "ipykernel==4.5.0\nipython==5.1.0\nipython-genutils==0.1.0\nipywidgets==5.2.2\njupyter==1.0.0\njupyter-client==4.4.0\njupyter-c"
},
{
"path": "scripts/resnet_plot_heatmap.ipynb",
"chars": 15147,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {\n \"collapsed\": false\n },\n \""
},
{
"path": "scripts/resnet_plot_results.ipynb",
"chars": 6755,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {\n \"collapsed\": true\n },\n \"o"
},
{
"path": "scripts/resnet_train.ipynb",
"chars": 19294,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Notebook for network training\"\n "
},
{
"path": "scripts/utils.py",
"chars": 2574,
"preview": "import numpy as np\n\ndef iterate_minibatches_train(inputs, targets, batchsize, shuffle=False):\n \"\"\"Iterate minibatches"
},
{
"path": "scripts/vgg_like_plot_results.ipynb",
"chars": 6180,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {\n \"collapsed\": false\n },\n \""
},
{
"path": "scripts/vgg_like_train.ipynb",
"chars": 25541,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Notebook for reproduce paper res"
},
{
"path": "test_scripts/test_theano.py",
"chars": 649,
"preview": "#!/usr/bin/env python\n\nfrom theano import function, config, shared, sandbox\nimport theano.tensor as T\nimport numpy\nimpor"
}
]
About this extraction
This page contains the full source code of the neuro-ml/resnet_cnn_mri_adni GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 16 files (99.7 KB), approximately 29.1k tokens, and a symbol index with 2 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.