[ { "path": "CHANGELOG.md", "content": "# 24 December 2019\n* With Satoshi Teraski's instructions it is possible to get Julia 1.3.0 / IJulia working on Raspberry Pis with ARM v6 CPU\n* His repository can be found [here](https://github.com/Julia-Embedded/jlcross).\n* Response to [issue #21](https://github.com/Julia-Embedded/jlcross/issues/21) outlines the workaround to get IJulia working.\n* The IJulia installtion workaround should also work with other Julia versions including 1.0.3, the version that can be apt installed on Raspbian Buster\n\n# 23 December 2019\n* If we exclude Julia for a moment, the setup works fine on Buster as well.\n* But when it comes to Julia, it is getting trickier and trickier to support ARM v6 based Raspberry Pi boards as official ARM v6 Julia binaries do no longer exist. Whilst Terasaki Satoshi cross-compiled several Julia versions including 1.3.0 using Docker and his binaries work fine, I cannot get IJulia to work with any of them on ARM v6. IJulia builds ok but the resulting kernels die. The same by the way happens with apt-get installed Julia. I searched the web intensively without finding a solution. If you do, kindly get in touch. Thanks & Merry Christmas!\n\n# 1 June 2019\n* added R-3.6.0 + IRkernel installer\n* added Julia-1.1.0 + IJulia installer\n* commented out a couple of optional installs in inst_jns.sh\n* updated requierments.txt\n* updated README.md - may still require some tweaks. Will attendd to it again as I find the time\n\n# 11 May 2019\n* updated requirements\n* removed line sqlite3-kernel==1.0 from requirements as the kernel is pulled from GitHub\n* this fixes issue #34 raised by #chalgand\n\n# 20 April 2019\n* updated requirements as GitHub urllib3 < 1.24.2 to be vulnerable\n\n# 6 April 2019\n* updated requirements\n* added a line to inst_labextensions.sh to use ipyleaflet in lab\n* updated README.md accordingly\n\n# 3 April 2019\n* updated requirements as GitHub reported the notebook verions < 5.7,9 to be vulnerable\n\n# 31 March 2019\n* updated requirements\n\n# 23 March 2019\n* updated requirements\n\n# 12 March 2019\n* corrected inst_lab_ext.sh as the line that activates the environment. This fixes issue #33 filed by @dwblair. Thanks for reporting this Don! \n\n# 9 March 2019\n* updated requirements\n\n# 6 March 2019\n* downgraded tornado to 5.1.1 to fix issue #32 as suggested by @Kevin--R - Thanks Kevin!\n\n# 4 March 2019\n* updated requirements.txt again\n* quite a challenge to keep up with the pace of the python community\n\n# 3 March 2019\n* updated requirements\n* was notified by GitHub in PyYAML before 4.1, the yaml.load() API could execute arbitrary code. In other words, yaml.safe_load is not used.\n* updated PyYAML\n\n# 30 January 2019\n* updated requirements.txt\n* downgraded vega to version 1.4 to resolve issue #31\n* credits go to @jakevdp who made this suggestion\n\n# 20 December 2018\n* updated requirements\n\n# 27 November 2018\n* updated requirements\n* fixed inst_opencv.sh as suggested by @mt08\n* corrected README.md accordingly\n\n# 10 November 2018\n* updated requirements\n\n# 27 October 2018\n* updated requirements\n\n# 20 October 2018\n* updated requirements\n\n# 15 October 2018\n* updated requirements\n\n# 6 October 2018\n* updated requirements\n\n# 4 October 2018\n* updated requirements\n* both ipython and jupyter-console are now compatible with prompt-toolkit 2.0.5 \n* merged 2 pull requests from @bennuttall - THANKS BEN! \n\n# 1 October 2018\n* added tensorflow\n* updated requirements\n\n# 23 September 2018\n* updated requirements\n* modified conf_jupyter.sh to fix issue #26 by setting ```c.NotebookApp.allow_remote_access = True``` in the configuration file located at ```~/.jupyter/jupyter_notebook_config.py```. In existing installtions just edit the configuration file manually\n\n# 15 September 2018\n* updated requirements\n* simpified requirements using pipdeptree -f --warn silence | grep -P '^[\\w0-9\\-=.]+'\n\n# 14 Septmber 2018\n* updated requirements.txt\n=======\n# 13 September 2018\n* created new branch dev\n* shifted bash scripts into `scripts` folder\n* adjusted `README.md' accordingly \n>>>>>>> create dev branch\n\n# 12 September 2018\n* updated requirements\n* removed service.sh as it is renamed to conf_servivice.sh\n\n# 9 September 2018\n* installed from scratch on a Raspberry Pi 3 and in the process made some fixes\n* updated requirements\n\n# 28 August 2018\n* updated requirements\n* made minor corrections to README.md\n\n# 18 August 2018\n* updated requirements\n\n# 12 August 2018\n* updated requirements\n* added SQLite3 kernel installation script\n* correted prep.sh - thanks mt08xx\n* adopted mt08xx's nice solution to start the server as a service\n\n# 7 August 2018\n* updated requirements\n\n# 31 July 2018\n* updated requirements\n\n# 20 July 2018\n* upgated requirements\n\n# 8 July 2018\n* updated requirements\n* added snowballstemmer back in\n\n# 1 July 2018\n* updated requirements\n\n# 12 June 2018\n* updated requirements\n\n# 3 June 2018\n* made minor changes to `README.md`\n* added dependencies for `python-opencv-headless` to `prep.sh`\n* updated requirements\n\n# 1 June 2018\n* fixed typo in `README.md`\n* closed issue # 23\n* updated requirements\n\n# 31 May 2018\n* removed `inst_node.sh`, modified ```conf_jupyter.sh` and added `inst_lab_ext.sh` as proposad by @Kevin--R to address issue #23 - Thanks Kevin!\n* modified `README.md` to refelct the changes above \n\n# 26 May 2018\n* updated requirements\n* corrected a typo\n\n# 21 May 2018\n* Thanks to comments received form @Kevin--R issue 22 should now be fixed \n\n# 20 May 2018\n* ***fixed further issue*** buy modifying ```inst_node.sh``` - see issue #22\n* ***fixed issue 22*** by modifying ```conf_jupyter.sh``` in accordance with a hint found [here](https://stackoverflow.com/questions/43659084/source-bashrc-in-a-script-not-working).\n* if you encountered this issue, please run the modified version of ```conf_jupyter.sh```\n\n# 19 May 2018\n* updated requirements\n\n# 5 May 2018\n* updated requirements\n* added altair + vega_datasets - see https://altair-viz.github.io/index.html for details\n\n# 25 April 2018\n* updated requirements\n\n# 17 April 2018\n* updated requirements\n* removed `format = 'legacy'` in README.md because it is deprecated in the latest version of `pip`\n* temporarily removed the altair package from requirements.txt due to incompatibility \n\n# 09 April 2018\n* closed issue #21\n\n# 07 April 2018\n* updated requirements\n* fixed typo in README.md - thanks for reporting it @m-r-white \n* ran a fresh install based on the DESKTOP version of Raspbian Stretch to fix issue #21 opened by @gusdrawn and independently confirmed by @m-r-white (Thanks to both!) and updated `README.md` and `conf_jupyter` to clarify what possibly went wrong in their installations. The short answer: `nodejs 5+` is required for `conf_jupyter.sh` to run properly. \n\n# 01 April 2018\n* updated requirements\n\n# 27 March 2018\n* updated requirements\n* in the wake of recent abuse of datascience tools: ***DO NO EVIL!***\n\n# 19 March 2018\n* updated ```requiremnets```\n* fixed ```inst_julia.sh```\n\n# 15 March 2018\n* added note to the end of ```README.md``` on how to stop the server if launched on boot\n\n# 14 March 2018\n* closed issue #17\n* added the solution as an option to ```README.md``` as this might be useful to some\n\n# 12 March 2018\n* removed necessity to reboot after changing ```swap_size``` by stopping and starting the service\n* updated ```requirements``` \n\n# 11 March 2018\n* closed issues #15 and #16\n* issue #15 is personal preference - I may pick up on this at a later stage\n* issue #16 is fixed by installing ```nodejs``` with a separate script. For further info please refer to the issue itself\n* cut down Julia installation as the JuliaBerry packages lead to segmentation faults\n* simplified TeX installtion as suggested in the documentation of ```nbconvert```\n\n# 22 February 2018\n* updated requirements\n* manually installed the bokeh jupyter lab extension for testing by running: ```jupyter labextension install jupyterlab_bokeh``` Took some time to build but seems to work fine\n\n# 21 Febuary 2017\n* updated requirements \n\n# 17 February 2017\n* updated requirements\n* it is really amazing how quickly updated packages become available on piwheels...\n\n# 14 February 2018\n* HAPPY VALENTINE'S DAY\n* updated requirements \n\n# 10 February 2018\n* updated requirements\n* if you installed prior to this update, just run:\n\n```bash\nsource /home/pi/.venv/jns/bin/activate\npip list --outdated --format='legacy'\npip install -U package1 package2...\n```\nwhere ```package1 package2...``` is a list of package names of the packages you want to update\n \n\n# 26 January 2018\n* added support for Sphinx for new installations \n* in case you want to add this manually because you already have a working jns installation just run:\n\n```bash\nsudo apt install latexmk\nsource ~/.venv/jns/bin/activate\npip3 install Sphinx docutils \n```\n\n# 24 January 2018\n* corrected `inst_jns.sh` and corresponding entry in `README.md`\n\n\n# 22 January 2018\n* set size of swap_file to 2048MB\n\n\n# 21 January 2018\n* modified `conf_jupyter.sh`in order to facilitate use of `ipywidgets` and `bqplot` in JupyterLab\n\n\n# 20 January 2018\n* new version on GitHub\n\n\n# 15 January 2018\n## IMPORTANT NOTIFICATION: I plan to release a new version of jns within this week\n\n***KEY CHANGES***\n* Rather than installing the latest version of Python, the new version will use the latest Python 3 version supported in Raspbian - as of this writing Python 3.5.3.\n* Whilst this seems to be a step backwards, it is a in fact a giant step forward as we benefit from significant installation speedups made possible by the recently released ***piwheels*** project.\n* The scripts will work across the entire range of Raspberry Pis - perhaps with the exception of the early models with just 256MB of memory - I tested Pi 1, 2, 3 and ZeroW.\n* The new version provides Python support for gpio, sensehat and picamera without the worries of breaking things on the system level as all Python modules will be pip installed into a virtual environment using a requirements.txt file.\n* This achieves a more maintainable setup and opens up more possibilities.\n* Pyhon 3, Julia and Bash kernels will be installed and configured. R, Ruby and Octave kernels are likely to follow as I find the time.\n* JupyterLab will be installed and ready to use.\n\n" }, { "path": "LICENSE.md", "content": "The MIT License (MIT)\n\nCopyright (c) 2015,2016,2017,2018 Eckhard Kleine\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "README.md", "content": "\n[![hardware](https://img.shields.io/badge/hardware-Raspberry_Pi-red.svg)](https://www.raspberrypi.org/products/)\n[![os](https://img.shields.io/badge/OS-Raspbian_Stretch_Lite-red.svg)](https://www.raspberrypi.org/downloads/raspbian/)\n\n![python](https://img.shields.io/badge/python-3.5.3-yellow.svg)\n![nodejs](https://img.shields.io/badge/nodejs-v8.11.4-green.svg)\n![Julia](https://img.shields.io/badge/julia-v1.1.0-magenta.svg)\n![R](https://img.shields.io/badge/R-v3.6.0-blue.svg)\n\n[![jupyter](https://img.shields.io/badge/more_info-jupyter-black.svg)](http://jupyter.org)\n[![piwheels](https://img.shields.io/badge/more_info-piwheels-black.svg)](https://www.piwheels.org)\n\n# Jupyter Notebook & Lab Server on Raspberry Pi\n## Intro\n\n[Project Jupyter](https://jupyter.org) not only revolutionizes data-heavy research across domains - it also boosts personal productivity for problems on a much smaller scale. Due to openness it is an amazing platform for exploring concepts and learning new things.\n\nI started setting up a Jupyter Notebook Server on a [Raspberry Pi](https://www.raspberrypi.org) following [this blog post](https://arundurvasula.wordpress.com/2014/04/01/remote-ipython-notebook-with-raspberry-pi/) by Arun Durvasula. Convinced of the potential of the platform I followed the development. \n\nMy personal exercise soon taught me a great deal about the underlying architcture. Given Jupyter's complexity, speed of growth and scale, it is remarkable that such a system runs fine on a Raspberry Pi.\n\nThis repository isn't really anything genuine: I owe big thanks to many contributors in the Jupyter, Raspberry Pi, Linux, Python, Julia and greater Open Source communities for providing all the beautiful building blocks - small and large.\n\n### What is new?\n\n* Rather than installing the latest version of Python as I did the past, I decided that the new version would use the latest Python 3 version supported in Raspbian - as of this writing Python 3.5.3.\n* Whilst this seems to be a step backwards, it is a in fact a giant step forward as you benefit from significant installation speedups made possible by the recently released [piwheels](https://www.piwheels.org) project.\n* The scripts work across the entire range of Raspberry Pis - including the early models with just 256MB of memory.\n* Python support for GPIO, Sense HAT and PICAMERA is installed without the earlier worries of breaking things on system level.\n* All Python modules are pip installed into a virtual environment following advice found online: ***You should never use `sudo pip install` -NEVER***. Well I did this in the past and it certainly had me and users confused. We have to learn certain things the hard way to really appreciate the benefits of doing them right. It is worth reading up on this in [this blogpost](http://jakevdp.github.io/blog/2017/12/05/installing-python-packages-from-jupyter/).\n* You now install Python packages into a virtual environment created with `venv` using a `requirements.txt` file. This really achieves a more maintainable setup, opens up more possibilities and hopefully makes this project more useful for the Raspberry Pi and Jupyter communities.\n* Python 3, Julia and Bash kernels are installed and configured during installation.\n* JupyterLab is installed and ready to use.\n\n## What do you need to follow along?\n\n* a ***Raspberry Pi model of your choice*** complete with micro-usb power-supply - I recommend a Raspberry Pi 3 but the setup should work across the entire range of Pi models, perhaps with the exception of the very early models that featured only 256MB of memory. I tested on a ZeroW, 1, 2 and 3. \n* a (micro) SD card with 16GB capacity or more to suit your Pi model with [Raspbian Stretch Lite](https://www.raspberrypi.org/downloads/raspbian/) installed and configured to permit access via ssh as user ***pi***.\n* an ethernet or wifi connection for the Pi\n* internet access on the Pi\n* a computer to carry out the installation connected to the same network as the Pi\n* ***LESS TIME THAN EVER BEFORE*** due to the recent release of [piwheels](https://www.piwheels.org). Users new to this project might argue that the setup is still time-consuming. Believe me: In the past 6 hours+ were not uncommon and installing the system on a Raspberry Pi 1 was not impossible but required quite some patience and time. Note that some packages listed in `requirements.txt` may not yet be available as Python wheels. Such packages are then built from source and this takes some time... \n\n## Installation\n\n### IMPORTANT NOTE on fresh installations\n* An increasing number of users seem to install on top of images that have 'nodejs' already installed.\n* The scripts in this repository were initially designed to work based on ***Raspbian Stretch Lite*** as a starting point with the intention to run the server headless in order to maximise memory available for data analysis.\n* One such starting point is the desktop version of ***Raspbian Stretch*** which comes with`nodejs` (and `git`) pre-installed. `conf_jupyter.sh` explained later now checks for the existence of `node` and only installs it if not yet present on the system.\n \n* ***For the scripts to run properly on the desktop version of Raspbian or any other startingpoint with `node` installed, it is necessary that `node` is version 5 or higher !!!***\n\n* If you start with a fresh Raspbian Stretch Desktop image, you can uninstall `node` using `apt purge nodejs` and then execute the scripts.\n\n### First boot with fresh SD card\n* ssh into your Raspberry Pi with the the fresh install of Raspbian Stretch Lite as user ***pi***. Then run `sudo raspi-config` and set the memory split to 16MB, expand the file-system and set a new password for the user pi. When done, reboot and log in again via ssh.\n\n* If not yet present, install `git`:\n\n```bash\nsudo apt install -y git\n```\n\n* To increase the size of swap_file to 2048MB run:\n```bash\nsudo sed -i -e 's/CONF_SWAPSIZE=100/CONF_SWAPSIZE=2048/' /etc/dphys-swapfile\nsudo /etc/init.d/dphys-swapfile stop\nsudo /etc/init.d/dphys-swapfile start\n```\n\n* With preparations out of the way clone this repository into the home directory of user ***pi***\n\n```bash\ngit clone https://github.com/kleinee/jns\n```\n* Change into the new directory `~/jns/scripts` just created with `git`:\n\n```bash\ncd ~/jns/scripts\n```\n\nTechnically you can now run `sudo ./inst_jns.sh` which is the installer script that combines the steps described below. If you follow along I assume that you run all scripts from inside the directory `~/jns/scripts`.\n\nYou might not want all features on your system. Feel free to edit `inst_jns.sh' to suit your requirements.\n\n## Install required Raspbian packages with apt\n\n```bash\nsudo ./prep.sh\n```\n\nA couple of packages from the Raspbian repository are required during installation and later for a some Python packages to work properly. The script just fetches these packages and installs them.\n\n```bash\n#!/bin/bash\n# script name: prep.sh\n# last modified: 2018/09/09\n# sudo: yes\n\nscript_name=$(basename -- \"$0\")\n\nif ! [ $(id -u) = 0 ]; then\n echo \"usage: sudo ./$script_name\"\n exit 1\nfi\n\napt update && apt -y upgrade\napt -y install pandoc\napt -y install libxml2-dev libxslt-dev\napt -y install libblas-dev liblapack-dev\napt -y install libatlas-base-dev gfortran\napt -y install libtiff5-dev libjpeg62-turbo-dev\napt -y install zlib1g-dev libfreetype6-dev liblcms2-dev\napt -y install libwebp-dev tcl8.5-dev tk8.5-dev\napt -y install libharfbuzz-dev libfribidi-dev\napt -y install libhdf5-dev\napt -y install libnetcdf-dev\napt -y install python3-pip\napt -y install python3-venv\napt -y install libzmq3-dev\napt -y install sqlite3 \n```\n\n## Install required Python 3 packages with pip\n\n```bash\n./inst_stack.sh\n```\n* This creates a virtual Python 3 environment '/home/pi/.venv/jns' and activates it temporarily\n* It then updates `pip3` to the latest version available from the Python package repository before it processes the `requirements.txt` file line by line. \n* This is a workaround to prevent `pip` from failing if one or more requirements listed fail to install.\n\n```bash\n#!/bin/bash\n# script name: inst_stack.sh\n# last modified: 2018/01/14\n# sudo: no\n\nscript_name=$(basename -- \"$0\")\nenv=\"/home/pi/.venv/jns\"\n\nif [ $(id -u) = 0 ]\nthen\n echo \"usage: ./$script_name\"\n exit 1\nfi\n\nif [ ! -d \"$venv\" ]; then\n python3 -m venv $env\nfi\n\n# activate virtual environment\nsource $env/bin/activate\n\npip3 install pip==9.0.0\npip3 install setuptools\npip3 install -U pip\n\ncat requirements.txt | xargs -n 1 pip3 install\n```\n\n## Configure Jupyter\n\n```bash\n./conf_jupyter.sh\n```\n\nWith this script you generate a jupyter notebook configuration directory and in it a file called `jupyter_notebook_config.py` that holds the configuration settings for your notebook / lab server. You also create a folder notebooks in the home directory of user `pi` as the `notebook_dir` for your server. In the configuration file, you apply the following changes:\n\n* tell jupyter not to sart a browser upon start - we access the server from a remote machine on the same network\n* set the IP address to '*' \n* set the port for the notebook server to listen to 8888\n* enable `mathjax` for rendering math in notebooks\n* set the notebook_dir to `~/notebooks`\n* use the password hash for the default server password `jns`\n\nNOTE: This setup still uses password authentication. If you prefer token-based authentication, you have to change settings in the config file `/home/pi/.jupyter/jupyter_notebook_config.py`. Documentation of possible configuration settings can be found [here](https://jupyter-notebook.readthedocs.io/en/stable/index.html).\n\nAfter the basic configuration the script activates the bash kernel and activates extensions for Jupyter Notebook and JupyterLab. At the JupyterLab end this requires intstallation of `node` followed by installation of the underlying JS infrastructure which is a bit time-consuming but ultimately allows you to use `ipywidgets`, `bqplot` and potentially other extensions.\n\n```bash\n#!/bin/bash\n# script name: conf_jupyter.sh\n# last modified: 2018/09/09\n# sudo: no\n\nscript_name=$(basename -- \"$0\")\nenv=\"/home/pi/.venv/jns\"\n\nif [ $(id -u) = 0 ]\nthen\n echo \"usage: ./$script_name\"\n exit 1\nfi\n\n# activate virtual environment\nsource $env/bin/activate\n\n# generate config and create notebook directory\n# if notebook directory exists, we keep it (-p)\n# if configuration file exeists, we overwrite it (-y)\n\njupyter notebook -y --generate-config\ncd $home\nmkdir -p notebooks\n\ntarget=~/.jupyter/jupyter_notebook_config.py\n\n# set up dictionary of changes for jupyter_config.py\ndeclare -A arr\napp='c.NotebookApp'\narr+=([\"$app.open_browser\"]=\"$app.open_browser = False\")\narr+=([\"$app.ip\"]=\"$app.ip ='*'\")\narr+=([\"$app.port\"]=\"$app.port = 8888\")\narr+=([\"$app.enable_mathjax\"]=\"$app.enable_mathjax = True\")\narr+=([\"$app.notebook_dir\"]=\"$app.notebook_dir = '/home/pi/notebooks'\")\narr+=([\"$app.password\"]=\"$app.password = 'sha1:5815fb7ca805:f09ed218dfcc908acb3e29c3b697079fea37486a'\")\n\n# apply changes to jupyter_notebook_config.py\n\nfor key in ${!arr[@]};do\n if grep -qF $key ${target}; then\n # key found -> replace line\n sed -i \"/${key}/c ${arr[${key}]}\" $target\n else\n # key not found -> append line\n echo \"${arr[${key}]}\" >> $target\n fi\ndone\n\n# install bash kernel\npython3 -m bash_kernel.install\n\n# install extensions\njupyter serverextension enable --py jupyterlab\njupyter nbextension enable --py widgetsnbextension --sys-prefix\njupyter nbextension enable --py --sys-prefix bqplot\n\n# activate clusters tab in notebook interface\n/home/pi/.venv/jns/bin/ipcluster nbextension enable --user\n\n# install nodejs and node version manager n\n# if node is not yet installed\nif which node > /dev/null\n then\n echo \"node is installed, skipping...\"\n else\n # install nodejs and node version manager n\n cd ~/jns\n # fix for issue #22\n # install nodejs and node version manager n\n # see: https://github.com/mklement0/n-install\n curl -L https://git.io/n-install | bash -s -- -y lts\nfi\n\n# install jupyter lab extensions\nbash -i ./inst_lab_ext.sh\n```\n\nThe script ```inst_lab_ext.sh``` - introduced by @Kevin--R to fix issue#23 has the following content:\n\n```bash\n#!/bin/bash\n# script name: inst_lab_ext.sh\n# last modified: 2019/04/06\n# sudo: no\n\nscript_name=$(basename -- \"$0\")\nenv=\"/home/pi/.venv/jns\"\n\nif [ $(id -u) = 0 ]\nthen\n echo \"usage: ./$script_name\"\n exit 1\nfi\n\n. /home/pi/.bashrc\n. $env/bin/activate\njupyter lab clean\njupyter labextension install @jupyter-widgets/jupyterlab-manager --no-build\njupyter labextension install bqplot --no-build\njupyter labextension install jupyterlab_bokeh --no-build\njupyter labextension install jupyter-leaflet --no-build\njupyter lab build\n```\n\n## Start and access your server\n\n### Activate the virtual environment\nSince you used a virtual environment to install Python modules, you need to activate this environment before you can start your server:\n\n\n```bash\nsource /home/pi/.venv/jns/bin/activate\n```\n\nThe prompt will change to indicate successfull activation preceding `pi@hostname:` with the envireonment name - in case pf this setup `(jns)`. With hostname set to `zerow` it looks like this:\n\n```bash\n(jns) pi@zerow:~ $\n```\n\n### Before you proceed\nAfter installation completes, you will still need to activate the change made to `~\\.bashrc` when node was installed before doing anything that requires node.\n\nYou can be accomplish this by any of the following:\n- reboot\n- logout and log back in\n- call `. ~/.bashrc` from the command line\n\nThat's the reason for this warning during node installation:\n```\n IMPORTANT: OPEN A NEW TERMINAL TAB/WINDOW or run `. /home/pi/.bashrc`\n before using n and Node.js.\n```\nYou can see this by running the following commands after your installation completes:\n```bash\npi@test-pi:~/jns $ echo $PATH\n/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games\npi@test-pi:~/jns $ . ~/.bashrc\npi@test-pi:~/jns $ echo $PATH\n/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games:/home/pi/n/bin\npi@test-pi:~/jns $ \n```\nIf you look at your **$PATH** environment variable and see **/home/pi/n/bin** you are ready to use node.\n\nAlso note that if you uninstall node with `n-uninstall` **/home/pi/n/bin** will remain in your **$PATH** environment variable until you reboot or logout and log back in.\n\n\n### Start the server\nTo start your server just type `jupyter notebook` or `jupyter lab` \n\n### Access the server\nTo access your server form a webbrowser on a computer running on the same network as your Raspberry Pi, just open a browser and use the Pi's IP address / port 8888 as the url. \n\n```bash\nxxx.xxx.xxx.xxx:8888\n```\n\nChange `xxx.xxx.xxx.xxx' to the IP address of the Raspberry Pi.\n\n### Login\nDuring the configuration the default password for the server was set to `jns`. You can change this by typing:\n\n```bash\n(jns) pi@zerow:~ $ jupyter notebook password\nEnter password: ****\nVerify password: ****\n```\n\n## Install TeX (optional)\n\n```bash\nsudo ./inst_tex.sh\n```\n\n* [TeX](https://en.wikipedia.org/wiki/TeX) (and [Pandoc](http://pandoc.org)) are used under the hood to convert Jupyter notebooks to other formats including PDF.\n* Whilst not strictly necessary if no PDF export is rquired, I still recommend to run this step.\n\n```bash\n#!/bin/bash\n# script name: inst_tex.sh\n# last modified: 2018/03/11\n# sudo: yes\n\nscript_name=$(basename -- \"$0\")\n\nif ! [ $(id -u) = 0 ]; then\n echo \"usage: sudo ./$script_name\"\n exit 1\nfi\n\n#------------------------------------------------------\napt install -y texlive-xetex\napt install -y latexmk\n#------------------------------------------------------\n```\n\n## Install Julia and the IJulia kernel (optional)\n* [Julia](https://julialang.org) is a relatively new high-level, high-performance dynamic programming language for numerical computing trying to combine the ease of Python with the speed of C. Thanks to the efforts of the Raspberry Pi community `Julia 0.6.0` is available in the Raspbian Stretch Repository. It is really worth a try as the language is a rising star in scientific computing.\n\n* [IJulia](https://github.com/JuliaLang/IJulia.jl) is the kernel required for Jupyter Notebook / JupyterLab. Backgroud information on Julia on the Raspberry Pi can be found [here](https://www.raspberrypi.org/blog/julia-language-raspberry-pi/).\n\n### Alternative 1: Julia 1.1.0 (RECOMMENDED)\n\n```bash\nsudo ./inst_julia-1.1.0.sh\n```\n* ***NOTE*** tha the installer assumes that *** Julia IS NOT yet installed***. If it is and you want to proceed with installing Julia-1.1.0, I suggest to remove the existing Julia installation before you proceed.\n\n* ```Julia 1.1.0``` binaries were cross-compiled by Mr Satoshi Terasaki (for more information follow [this link](https://gist.github.com/terasakisatoshi/3f8a55391b1fc22a5db4a43da8d92c98)) and Mr Satoshi thankfully hosts his binaries [here](https://drive.google.com/file/d/1fj6pNAJgmUD7bsSXqh8ocC1wESx8jkRh/view).\n\n* We need to download a large file from Google Drive and need to install the right binary based on the CPU architecture. The download contains binaries for both architectures.\n\n#### The Download Helper\nAs per comments in the script this is literally a 1:1 copy of code found on stack overflow - adjustments were only necessary to set the ***FILE_ID*** and the ***DESTINATION*** as required in the context of this repository. The helper is called by the installer script and is not meant to be exceuted manually.\n\n```Python\n#!/home/pi/.venv/jns/bin/python\n\n#\n# last modified 2019/05/26\n#\n# Python helper script to download Julia 1.1.0 binaries\n# not meant to be executed manually\n# https://stackoverflow.com/questions/38511444/python-download-files-from-google-drive-using-url\n#\n\nFILE_ID = '1fj6pNAJgmUD7bsSXqh8ocC1wESx8jkRh'\nDESTINATION = './julia-1.1.0-arm32bit.zip'\n\nimport requests\n\ndef download_file_from_google_drive(id, destination):\n URL = \"https://docs.google.com/uc?export=download\"\n\n session = requests.Session()\n\n response = session.get(URL, params = { 'id' : id }, stream = True)\n token = get_confirm_token(response)\n\n if token:\n params = { 'id' : id, 'confirm' : token }\n response = session.get(URL, params = params, stream = True)\n\n save_response_content(response, destination) \n\ndef get_confirm_token(response):\n for key, value in response.cookies.items():\n if key.startswith('download_warning'):\n return value\n\n return None\n\ndef save_response_content(response, destination):\n CHUNK_SIZE = 32768\n\n with open(destination, \"wb\") as f:\n for chunk in response.iter_content(CHUNK_SIZE):\n if chunk: # filter out keep-alive new chunks\n f.write(chunk)\n\nif __name__ == \"__main__\":\n file_id = FILE_ID\n destination = DESTINATION\n download_file_from_google_drive(file_id, destination)\n```\n\n#### The Installer\n\nNote that the code assumes that julia is not present. \n\n* installs necessary dependencies as suggested by Mr Terasaki\n* uses the downlaod helper to downlaod his binaries\n* detects the CPU architecture and installs the matching julia binary\n* creates a soft link in ```/usr/local/bin```\n* installs the the IJulia kernel\n\n```bash\n#!/bin/bash\n# script name: inst_julia-1.1.0.sh\n# last modified: 2019/05/26\n# sudo: yes\n\nSCRIPT_NAME=$(basename -- \"$0\")\nJNS_USER='pi'\nHOME_DIR=\"/home/$JNS_USER\"\nENV=\"$HOME_DIR/.venv/jns\"\n\nJULIA_HOME=$HOME_DIR/julia/\n\nif ! [ $(id -u) = 0 ]; then\n echo \"usage: sudo ./$SCRIPT_NAME\"\n exit 1\nfi\n\n#\n# apt install dependencies\n#\napt install -y build-essential \napt install -y libatomic1\napt install -y gfortran \napt install -y perl \napt install -y wget\napt install -y m4\napt install -y cmake \napt install -y pkg-config\napt install -y libopenblas-base libopenblas-dev\napt install -y libatlas3-base libatlas-base-dev\napt install -y liblapack-dev\napt install -y libmpfr-dev libgmp3-dev\napt install -y libgfortran3\n\n#\n# download and install julia based on architecture\n#\nsu pi < /home/pi/jupyter_start.sh && chmod a+x /home/pi/jupyter_start.sh\n#!/bin/bash\n. /home/pi/.venv/jns/bin/activate\njupyter lab\n#jupyter notebook\nONE\n\ncat << 'TWO' | sudo tee /etc/systemd/system/jupyter.service\n[Unit]\nDescription=Jupyter\n\n[Service]\nType=simple\nExecStart=/home/pi/jupyter_start.sh\nUser=pi\nGroup=pi\nWorkingDirectory=/home/pi/notebooks\nRestart=always\nRestartSec=10\n\n[Install]\nWantedBy=multi-user.target\nTWO\n\n# start jupyter\nsystemctl daemon-reload\nsystemctl start jupyter\nsystemctl enable jupyter\n```\n\n* Next time you boot your Pi, the service is stared automatically.\n* To stop the service for system updates run:\n\n```bash\nsudo systemctl stop jupyter\n```\n\n## Put it all together\n\nThis script is just convenience - it executes the individual steps described above in the order necessary.\nNote that installation of additinal languages and their respective kernels as well as installtion of opnencv is deactivated by default as not all users may need this functionality. I recommend to run ```inst_jns.sh``` as is and install additional functionality using the individual scripts.\n\n+ inst_sqlite.sh\n+ inst_R-3.6.0.sh\n+ inst_julia-0.6.0.sh\n+ inst_julia-1.1.0.sh\n\n```bash\n#!/bin/bash\n# script name: inst_jns.sh\n# last modified: 2019/05/26\n# sudo: yes\n\nscript_name=$(basename -- \"$0\")\n\nif ! [ $(id -u) = 0 ]; then\n echo \"usage: sudo ./$script_name\"\n exit 1\nfi\n\n#-----------------------------------------------\n# MANDATORY\n#-----------------------------------------------\n\n# make necessary preparations\n./prep.sh\n\n# install Python packages \nsudo -u pi ./inst_stack.sh\n\n# configure server\nsudo -u pi ./conf_jupyter.sh\n\n\n#-----------------------------------------------\n# OPTIONAL, RECOMMENDED\n#-----------------------------------------------\n\n# install TeX\n./inst_tex.sh\n\n# install support for Pi hardware\nsudo -u pi ./inst_pi_hardware.sh\n\n# set up service to start the server on boot\n./conf_service.sh\n\n\n#-----------------------------------------------\n# OPTIONAL, DISABLED BY DEFAULT\n#-----------------------------------------------\n\n# install Julia 0.6.0 and the IJulia kernel NOT RECOMMENDED\n# ./inst_julia-0.6.0.sh\n\n# install Julia 1.1.0 and the IJulia kernel\n# ./inst_julia-1.1.0.sh\n\n# install R 3.6.0 and the IRkernel\n# ./inst_R-3.6.0.sh\n\n# install the SQLite3 kernel\n# sudo -u pi ./inst_sqlite.sh\n\n# install opencv\n# ./inst_opencv.sh\n\n```\n\n## Keep your installation up to date\n### Raspbian operating system\n\n* just run `sudo apt update && sudo apt -y upgrade`\n\n### Python 3 packages\n\n* activate the virtual environment with `source /home/pi/.venv/jns/bin/activate`\n\n* list outdated packages with `pip3 list --outdated`\n\n* Update `package` with `pip3 install -U package` where `package` is the name of package you want to update.\n" }, { "path": "scripts/conf_jupyter.sh", "content": "#!/bin/bash\n# script name: conf_jupyter.sh\n# last modified: 2018/09/23\n# sudo: no\n\nscript_name=$(basename -- \"$0\")\nenv=\"/home/pi/.venv/jns\"\n\nif [ $(id -u) = 0 ]\nthen\n echo \"usage: ./$script_name\"\n exit 1\nfi\n\n# activate virtual environment\nsource $env/bin/activate\n\n# generate config and create notebook directory\n# if notebook directory exists, we keep it (-p)\n# if configuration file exeists, we overwrite it (-y)\n\njupyter notebook -y --generate-config\ncd $home\nmkdir -p notebooks\n\ntarget=~/.jupyter/jupyter_notebook_config.py\n\n# set up dictionary of changes for jupyter_config.py\ndeclare -A arr\napp='c.NotebookApp'\narr+=([\"$app.open_browser\"]=\"$app.open_browser = False\")\narr+=([\"$app.ip\"]=\"$app.ip ='*'\")\narr+=([\"$app.port\"]=\"$app.port = 8888\")\narr+=([\"$app.enable_mathjax\"]=\"$app.enable_mathjax = True\")\narr+=([\"$app.notebook_dir\"]=\"$app.notebook_dir = '/home/pi/notebooks'\")\narr+=([\"$app.password\"]=\"$app.password = 'sha1:5815fb7ca805:f09ed218dfcc908acb3e29c3b697079fea37486a'\")\narr+=([\"$app.allow_remote_access\"]=\"$app.allow_remote_access = True\")\n\n# apply changes to jupyter_notebook_config.py\n\nfor key in ${!arr[@]};do\n if grep -qF $key ${target}; then\n # key found -> replace line\n sed -i \"/${key}/c ${arr[${key}]}\" $target\n else\n # key not found -> append line\n echo \"${arr[${key}]}\" >> $target\n fi\ndone\n\n# install bash kernel\npython3 -m bash_kernel.install\n\n# install extensions\njupyter serverextension enable --py jupyterlab\njupyter nbextension enable --py widgetsnbextension --sys-prefix\njupyter nbextension enable --py --sys-prefix bqplot\n\n# activate clusters tab in notebook interface\n/home/pi/.venv/jns/bin/ipcluster nbextension enable --user\n\n# install nodejs and node version manager n\n# if node is not yet installed\nif which node > /dev/null\n then\n echo \"node is installed, skipping...\"\n else\n # install nodejs and node version manager n\n cd ~/jns\n # fix for issue #22\n # install nodejs and node version manager n\n # see: https://github.com/mklement0/n-install\n curl -L https://git.io/n-install | bash -s -- -y lts\nfi\n\n# install jupyter lab extensions\nbash -i ./inst_lab_ext.sh\n\n" }, { "path": "scripts/conf_service.sh", "content": "#!/bin/bash\n# script name: conf_service.sh\n# last modified: 2018/08/12\n# credits: mt08xx\n# sudo: yes\n\nscript_name=$(basename -- \"$0\")\n\nif ! [ $(id -u) = 0 ]; then\n echo \"usage: sudo ./$script_name\"\n exit 1\nfi\n\n# create jupyter.sh in /home/pi and make it executable\ncat << 'ONE' > /home/pi/jupyter_start.sh && chmod a+x /home/pi/jupyter_start.sh\n#!/bin/bash\n. /home/pi/.venv/jns/bin/activate\njupyter lab\n#jupyter notebook\nONE\n\ncat << 'TWO' | sudo tee /etc/systemd/system/jupyter.service\n[Unit]\nDescription=Jupyter\n\n[Service]\nType=simple\nExecStart=/home/pi/jupyter_start.sh\nUser=pi\nGroup=pi\nWorkingDirectory=/home/pi/notebooks\nRestart=always\nRestartSec=10\n\n[Install]\nWantedBy=multi-user.target\nTWO\n\n# start jupyter\nsystemctl daemon-reload\nsystemctl start jupyter\nsystemctl enable jupyter\n" }, { "path": "scripts/dnld_julia-1.1.0-arm32bit.py", "content": "#!/home/pi/.venv/jns/bin/python\n\n#\n# last modified 2019/05/26\n#\n# Python helper script to download Julia 1.1.0 binaries\n# not meant to be executed manually\n# https://stackoverflow.com/questions/38511444/python-download-files-from-google-drive-using-url\n#\n\nFILE_ID = '1fj6pNAJgmUD7bsSXqh8ocC1wESx8jkRh'\nDESTINATION = './julia-1.1.0-arm32bit.zip'\n\nimport requests\n\ndef download_file_from_google_drive(id, destination):\n URL = \"https://docs.google.com/uc?export=download\"\n\n session = requests.Session()\n\n response = session.get(URL, params = { 'id' : id }, stream = True)\n token = get_confirm_token(response)\n\n if token:\n params = { 'id' : id, 'confirm' : token }\n response = session.get(URL, params = params, stream = True)\n\n save_response_content(response, destination) \n\ndef get_confirm_token(response):\n for key, value in response.cookies.items():\n if key.startswith('download_warning'):\n return value\n\n return None\n\ndef save_response_content(response, destination):\n CHUNK_SIZE = 32768\n\n with open(destination, \"wb\") as f:\n for chunk in response.iter_content(CHUNK_SIZE):\n if chunk: # filter out keep-alive new chunks\n f.write(chunk)\n\nif __name__ == \"__main__\":\n file_id = FILE_ID\n destination = DESTINATION\n download_file_from_google_drive(file_id, destination)\n" }, { "path": "scripts/inst_R-3.6.0.sh", "content": "#!/bin/bash\n# script name: inst_R-3.6.0.sh\n# last modified: 2019/05/19\n# sudo: yes\n\nSCRIPT_NAME=$(basename -- \"$0\")\nJNS_USER='pi'\nHOME_DIR=\"/home/$JNS_USER\"\nENV=\"$HOME_DIR/.venv/jns\"\n\nR_VERSION=\"R-3.6.0\"\nR_DOWNLOAD_URL=\"http://mirrors.psu.ac.th/pub/cran/src/base/R-3/$R_VERSION.tar.gz\"\nR_EXEC=$(which R)\nR_HOME=\"$HOME_DIR/R\"\n\nif ! [ $(id -u) = 0 ]; then\n echo \"usage: sudo ./$SCRIPT_NAME\"\n exit 1\nfi\n\ncd $HOME_DIR\n\n#\n# apt install additional packages\n#\napt install -y libreadline-dev\napt install -y libbz2-dev\n\n#\n# download R source and compile\n# if R is not yet present \n#\nsu pi <