[
{
"path": ".gitignore",
"content": "*.DS_Store\n.vscode/\ndata/PyCon\npy/.ipynb_checkpoints/\n__*\npy/logs/"
},
{
"path": "LICENSE",
"content": "MIT License\n\nCopyright (c) 2018 Jens Leitloff & Felix Riese\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n"
},
{
"path": "README.md",
"content": "[](https://doi.org/10.5281/zenodo.3268451)\n[](LICENSE)\n[](https://www.codacy.com/manual/jensleitloff/CNN-Sentinel?utm_source=github.com&utm_medium=referral&utm_content=jensleitloff/CNN-Sentinel&utm_campaign=Badge_Grade)\n\n# Analyzing Sentinel-2 satellite data in Python with TensorFlow.Keras\n\nOverview about state-of-the-art land-use classification from satellite data\nwith CNNs based on an open dataset\n\n## Outline\n\n* [Scripts you will find here](#scripts-you-will-find-here)\n* [Requirements (what we used):](#requirements--what-we-used--)\n* [Setup Environment](#setup-environment)\n* [Our talks about this topic](#our-talks-about-this-topic)\n* [Resources](#resources)\n* [How to get Sentinel-2 data](#how-to-get-sentinel-2-data)\n* [Citation](#citation)\n\n## Scripts you will find here\n\n* `01_split_data_to_train_and_validation.py`: split complete dataset into train\n and validation\n* `02_train_rgb_finetuning.py`: train VGG16 or DenseNet201 using RGB data with\n pre-trained weights on ImageNet\n* `03_train_rgb_from_scratch.py`: train VGG16 or DenseNet201 from scratch using\n RGB data\n* `04_train_ms_finetuning.py`: train VGG16 or DenseNet201 using multispectral\n data with pre-trained weights on ImageNet\n* `04_train_ms_finetuning_alternative.py`: an alternative way to train VGG16 or\n DenseNet201 using multispectral data with pre-trained weights on ImageNet\n* `05_train_ms_from_scratch.py`: train VGG16 or DenseNet201 from scratch using\n multispectral data\n* `06_classify_image.py`: a simple implementation to classify images with\n trained models\n* `image_functions.py`: functions for image normalization and a simple\n generator for training data augmentation\n* `statistics.py`: a simple implementation to calculate normalization\n parameters (i.e. mean and std of training data)\n\nAdditionally you will find the following notebooks:\n\n* `Image_functions.ipynb`: notebook of `image_functions.py`\n* `Train_from_Scratch.ipynb`: notebook of `05_train_ms_from_scratch.py`\n* `Transfer_learning.ipynb`: notebook of `02_train_rgb_finetuning.py`\n\n## Requirements (what we used)\n\nWe have defined the requirements in [requirements.txt](requirements.txt).\nWe used:\n\n* python 3.6.x\n* tensorflow 2.2\n* scikit-image (0.14.1)\n* gdal (2.2.4) for `06_classify_image.py`\n\n## Frequently asked questions (FAQs)\n\n* **How can I interpret the classification results?** - Please have a look at our answers\n [#3](https://github.com/jensleitloff/CNN-Sentinel/issues/3),\n [#4](https://github.com/jensleitloff/CNN-Sentinel/issues/4), and\n [#6](https://github.com/jensleitloff/CNN-Sentinel/issues/6).\n* **Is there a paper I can cite for this repository?** - Please have a look at [Citation](#citation)\n\n## Setup environment\n\nAppend conda-forge to your Anaconda channels:\n\n```bash\nconda config --append channels conda-forge\n```\n\nCreate new environment:\n\n```bash\nconda create -n pycon scikit-image gdal tqdm\nconda activate pycon\npip install tensorflow-gpu\npip install keras\n```\n\n(or use tensorflow version of keras, i.e. `from tensorflow import keras`)\n\nSee also:\n\n* [Keras](https://keras.io/)\n\n## Our talks about this topic\n\n### Podcast episode @ TechTiefen\n\n* **Title:** \"Fernerkundung mit multispektralen Satellitenbildern\"\n* **Episode:** [Episode 18](https://techtiefen.de/18-fernerkundung-mit-multispektralen-satellitenbildern/)\n* **Podcast:** [TechTiefen](https://techtiefen.de) by Nico Kreiling\n* **Language:** German (Deutsch)\n* **Date:** July 2019\n\nAbstract
\n Jens Leitloff und Felix Riese berichten in Folge 18 von ihrer Forschung am “Institut für Photogrammetrie und Fernerkundung” des Karlsruher Instituts für Technologie. Mit der Bestrebung Nachhaltigkeit zu stärken erforschen die beiden etwa Verfahren, um Wasserqualität anhand von Satellitenaufnahmen zu bewerten oder die Nutzung landwirtschaftlicher Flächen zu kartografieren. Hierfür kommen unterschiedlichste Verfahren zum Einsatz wie Radaraufnahmen oder multispektrale Bilderdaten, die mehr als die drei von Menschen wahrnehmbaren Farbkanäle erfassen. Außerdem geht es um Drohnen, Satelliten und zahlreiche ML-Verfahren wie Transfer- und Aktive Learning. Persönliche Erfahrungen von Jens und Felix im Umgang mit unterschiedlichen Datenmengen runden eine thematisch Breite und anschauliche Folge ab.\nAbstract
\n> Im Forschungsfeld des Maschinellen Lernens werden zunehmend leicht zugängliche Framework wie Keras, Tensorflow oder Pytorch verwendet. Hierdurch ist ein Austausch und die Wiederverwendung bestehender (trainierter) neuronaler Netze möglich.\n>\n> Wir am Institut für Photogrammetrie und Fernerkundung (IPF) des Karlsruher Institut für Technologie (KIT) beschäftigen uns unter anderem mit der Analyse von optischen Satellitendaten. Satellitenprogramme wie Sentinel-2 von Copernicus liefern wöchentliche, weltweite und dabei frei zugängliche multispektrale Bilder, die eine Vielzahl neuartiger Anwendungen ermöglichen. Wir nehmen das zum Anlass, eine interaktive Einführung in die Auswertung dieser Satellitendaten mit Learnings aus unserer täglichen Forschung zu geben. Wir sprechen unter anderem über die folgenden Themen:\n>\n> * Einfacher Umgang mit georeferenzierten Bilddaten\n> * Einführung in Learning-From-Scratch und Transfer Learning mit Keras\n> * Anpassung von fertigen Netzen an neue Eingangsdaten (RGB → multispektral)\n> * Anschauliche Interpretation von Klassifikationsergebnissen\n> * Best Practices aus unserer Forschung, die die Arbeit mit Neuronalen Netzen wesentlich vereinfachen und beschleunigen\n> * Code und Daten für die ersten Schritte mit CNNs mit Keras in Python, welche in einem GitHub Repository zur Verfügung gestellt werden\nAbstract
\n> The largest earth observation programme Copernicus (http://copernicus.eu) makes it possible to perform terrestrial observations providing data for all kinds of purposes. One important objective is to monitor the land-use and land-cover changes with the Sentinel-2 satellite mission. These satellites measure the sun reflectance on the earth surface with multispectral cameras (13 channels between 440 nm to 2190 nm). Machine learning techniques like convolutional neural networks (CNN) are able to learn the link between the satellite image (spectrum) and the ground truth (land use class). In this talk, we give an overview about the state-of-the-art land-use classification with CNNs based on an open dataset.\n>\n> We use different out-of-box CNNs for the Keras deep learning library (https://keras.io/). All networks are either included in Keras itself or are available from Github repositories. We show the process of transfer learning for the RGB datasets. Furthermore, the minimal changes required to apply commonly used CNNs to multispectral data are demonstrated. Thus, the interested audience will be able to perform their own classification of remote sensing data within a very short time. Results of different network structures are visually compared. Especially the differences of transfer learning and learning from scratch are demonstrated. This also includes the amount of necessary training epochs, progress of training and validation error and visual comparison of the results of the trained networks. Finally, we give a quick overview about the current research topics including recurrent neural networks for spatio-temporal land-use classification and further applications of multi- and hyperspectral data, e.g. for the estimation of water parameters and soil characteristics.\n