Repository: jensleitloff/CNN-Sentinel
Branch: master
Commit: 094b8d3ff897
Files: 17
Total size: 87.0 KB
Directory structure:
gitextract_jvujk87r/
├── .gitignore
├── LICENSE
├── README.md
├── bibliography.bib
├── py/
│ ├── 01_split_data_to_train_and_validation.py
│ ├── 02_train_rgb_finetuning.py
│ ├── 03_train_rgb_from_scratch.py
│ ├── 04_train_ms_finetuning.py
│ ├── 04_train_ms_finetuning_alternative.py
│ ├── 05_train_ms_from_scratch.py
│ ├── 06_classify_image.py
│ ├── Image_functions.ipynb
│ ├── Train_from_Scratch.ipynb
│ ├── Transfer_learning.ipynb
│ ├── image_functions.py
│ └── statistics.py
└── requirements.txt
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
*.DS_Store
.vscode/
data/PyCon
py/.ipynb_checkpoints/
__*
py/logs/
================================================
FILE: LICENSE
================================================
MIT License
Copyright (c) 2018 Jens Leitloff & Felix Riese
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: README.md
================================================
[](https://doi.org/10.5281/zenodo.3268451)
[](LICENSE)
[](https://www.codacy.com/manual/jensleitloff/CNN-Sentinel?utm_source=github.com&utm_medium=referral&utm_content=jensleitloff/CNN-Sentinel&utm_campaign=Badge_Grade)
# Analyzing Sentinel-2 satellite data in Python with TensorFlow.Keras
Overview about state-of-the-art land-use classification from satellite data
with CNNs based on an open dataset
## Outline
* [Scripts you will find here](#scripts-you-will-find-here)
* [Requirements (what we used):](#requirements--what-we-used--)
* [Setup Environment](#setup-environment)
* [Our talks about this topic](#our-talks-about-this-topic)
* [Resources](#resources)
* [How to get Sentinel-2 data](#how-to-get-sentinel-2-data)
* [Citation](#citation)
## Scripts you will find here
* `01_split_data_to_train_and_validation.py`: split complete dataset into train
and validation
* `02_train_rgb_finetuning.py`: train VGG16 or DenseNet201 using RGB data with
pre-trained weights on ImageNet
* `03_train_rgb_from_scratch.py`: train VGG16 or DenseNet201 from scratch using
RGB data
* `04_train_ms_finetuning.py`: train VGG16 or DenseNet201 using multispectral
data with pre-trained weights on ImageNet
* `04_train_ms_finetuning_alternative.py`: an alternative way to train VGG16 or
DenseNet201 using multispectral data with pre-trained weights on ImageNet
* `05_train_ms_from_scratch.py`: train VGG16 or DenseNet201 from scratch using
multispectral data
* `06_classify_image.py`: a simple implementation to classify images with
trained models
* `image_functions.py`: functions for image normalization and a simple
generator for training data augmentation
* `statistics.py`: a simple implementation to calculate normalization
parameters (i.e. mean and std of training data)
Additionally you will find the following notebooks:
* `Image_functions.ipynb`: notebook of `image_functions.py`
* `Train_from_Scratch.ipynb`: notebook of `05_train_ms_from_scratch.py`
* `Transfer_learning.ipynb`: notebook of `02_train_rgb_finetuning.py`
## Requirements (what we used)
We have defined the requirements in [requirements.txt](requirements.txt).
We used:
* python 3.6.x
* tensorflow 2.2
* scikit-image (0.14.1)
* gdal (2.2.4) for `06_classify_image.py`
## Frequently asked questions (FAQs)
* **How can I interpret the classification results?** - Please have a look at our answers
[#3](https://github.com/jensleitloff/CNN-Sentinel/issues/3),
[#4](https://github.com/jensleitloff/CNN-Sentinel/issues/4), and
[#6](https://github.com/jensleitloff/CNN-Sentinel/issues/6).
* **Is there a paper I can cite for this repository?** - Please have a look at [Citation](#citation)
## Setup environment
Append conda-forge to your Anaconda channels:
```bash
conda config --append channels conda-forge
```
Create new environment:
```bash
conda create -n pycon scikit-image gdal tqdm
conda activate pycon
pip install tensorflow-gpu
pip install keras
```
(or use tensorflow version of keras, i.e. `from tensorflow import keras`)
See also:
* [Keras](https://keras.io/)
## Our talks about this topic
### Podcast episode @ TechTiefen
* **Title:** "Fernerkundung mit multispektralen Satellitenbildern"
* **Episode:** [Episode 18](https://techtiefen.de/18-fernerkundung-mit-multispektralen-satellitenbildern/)
* **Podcast:** [TechTiefen](https://techtiefen.de) by Nico Kreiling
* **Language:** German (Deutsch)
* **Date:** July 2019
<details><summary>Abstract</summary>
Jens Leitloff und Felix Riese berichten in Folge 18 von ihrer Forschung am “Institut für Photogrammetrie und Fernerkundung” des Karlsruher Instituts für Technologie. Mit der Bestrebung Nachhaltigkeit zu stärken erforschen die beiden etwa Verfahren, um Wasserqualität anhand von Satellitenaufnahmen zu bewerten oder die Nutzung landwirtschaftlicher Flächen zu kartografieren. Hierfür kommen unterschiedlichste Verfahren zum Einsatz wie Radaraufnahmen oder multispektrale Bilderdaten, die mehr als die drei von Menschen wahrnehmbaren Farbkanäle erfassen. Außerdem geht es um Drohnen, Satelliten und zahlreiche ML-Verfahren wie Transfer- und Aktive Learning. Persönliche Erfahrungen von Jens und Felix im Umgang mit unterschiedlichen Datenmengen runden eine thematisch Breite und anschauliche Folge ab.
</details>
### M3 Minds mastering machines 2019 @ Mannheim
* **Title:** "Satellite Computer Vision mit Keras und Tensorflow - Best practices und beispiele aus der Forschung"
* **Slides:** [Slides](slides/M3-2019_RieseLeitloff_SatelliteCV.pdf)
* **Language:** German (Deutsch)
* **Date:** 15 - 16 May 2019
* **DOI:** [](https://doi.org/10.5281/zenodo.4056744)
* **URL:** [m3-konferenz.de](https://m3-konferenz.de/2019/)
<details><summary>Abstract</summary>
> Im Forschungsfeld des Maschinellen Lernens werden zunehmend leicht zugängliche Framework wie Keras, Tensorflow oder Pytorch verwendet. Hierdurch ist ein Austausch und die Wiederverwendung bestehender (trainierter) neuronaler Netze möglich.
>
> Wir am Institut für Photogrammetrie und Fernerkundung (IPF) des Karlsruher Institut für Technologie (KIT) beschäftigen uns unter anderem mit der Analyse von optischen Satellitendaten. Satellitenprogramme wie Sentinel-2 von Copernicus liefern wöchentliche, weltweite und dabei frei zugängliche multispektrale Bilder, die eine Vielzahl neuartiger Anwendungen ermöglichen. Wir nehmen das zum Anlass, eine interaktive Einführung in die Auswertung dieser Satellitendaten mit Learnings aus unserer täglichen Forschung zu geben. Wir sprechen unter anderem über die folgenden Themen:
>
> * Einfacher Umgang mit georeferenzierten Bilddaten
> * Einführung in Learning-From-Scratch und Transfer Learning mit Keras
> * Anpassung von fertigen Netzen an neue Eingangsdaten (RGB → multispektral)
> * Anschauliche Interpretation von Klassifikationsergebnissen
> * Best Practices aus unserer Forschung, die die Arbeit mit Neuronalen Netzen wesentlich vereinfachen und beschleunigen
> * Code und Daten für die ersten Schritte mit CNNs mit Keras in Python, welche in einem GitHub Repository zur Verfügung gestellt werden
</details>
### PyCon.DE 2018 @ Karlsruhe
* **Title:** "Satellite data is for everyone: insights into modern remote sensing research with open data and Python"
* **Slides:** [Slides](slides/PyCon2018_LeitloffRiese_SatelliteData.pdf)
* **Video:** [youtube.com/watch?v=tKRoMcBeWjQ](https://www.youtube.com/watch?v=tKRoMcBeWjQ)
* **Language:** English
* **Date:** 24 - 28 October 2018
* **DOI:** [](https://doi.org/10.5281/zenodo.4056516)
* **URL:** [de.pycon.org](https://de.pycon.org)
<details><summary>Abstract</summary>
> The largest earth observation programme Copernicus (http://copernicus.eu) makes it possible to perform terrestrial observations providing data for all kinds of purposes. One important objective is to monitor the land-use and land-cover changes with the Sentinel-2 satellite mission. These satellites measure the sun reflectance on the earth surface with multispectral cameras (13 channels between 440 nm to 2190 nm). Machine learning techniques like convolutional neural networks (CNN) are able to learn the link between the satellite image (spectrum) and the ground truth (land use class). In this talk, we give an overview about the state-of-the-art land-use classification with CNNs based on an open dataset.
>
> We use different out-of-box CNNs for the Keras deep learning library (https://keras.io/). All networks are either included in Keras itself or are available from Github repositories. We show the process of transfer learning for the RGB datasets. Furthermore, the minimal changes required to apply commonly used CNNs to multispectral data are demonstrated. Thus, the interested audience will be able to perform their own classification of remote sensing data within a very short time. Results of different network structures are visually compared. Especially the differences of transfer learning and learning from scratch are demonstrated. This also includes the amount of necessary training epochs, progress of training and validation error and visual comparison of the results of the trained networks. Finally, we give a quick overview about the current research topics including recurrent neural networks for spatio-temporal land-use classification and further applications of multi- and hyperspectral data, e.g. for the estimation of water parameters and soil characteristics.
</details>
## Resources
**This talk:**
* EuroSAT Data (Sentinel-2, [Link](http://madm.dfki.de/downloads))
**Platforms for datasets:**
* HyperLabelMe: a Web Platform for Benchmarking Remote Sensing Image Classifiers ([Link](http://hyperlabelme.uv.es/))
* GRSS Data and Algorithm Standard Evaluation (DASE) website ([Link](http://dase.ticinumaerospace.com/))
**Datasets:**
* ISPRS 2D labeling challenge ([Link](http://www2.isprs.org/commissions/comm3/wg4/semantic-labeling.html))
* UC Merced Land Use Dataset ([Link](http://weegee.vision.ucmerced.edu/datasets/landuse.html))
* AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classification ([Link](https://captain-whu.github.io/AID/))
* NWPU-RESISC45 (RGB, [Link](http://www.escience.cn/people/JunweiHan/NWPU-RESISC45.html))
* Zurich Summer Dataset (RGB, [Link](https://sites.google.com/site/michelevolpiresearch/data/zurich-dataset))
* **Note**: Many German state authorities offer free geodata (high resolution images, land use/cover vector data, ...) over their geoportals. You can find an overview of all geoportals here ([geoportals](https://www.geoportal.nrw/geoportale_bundeslaender_nachbarstaaten))
**Image Segmentation Resources:**
* More than 100 combinations for image segmentation routines with Keras and pretrained weights for endcoding phase ([Segmentation Models](https://github.com/qubvel/segmentation_models))
* Another source for image segmentation with Keras including pretrained weights ([Keras-FCN](https://github.com/aurora95/Keras-FCN))
* Great link collection of image segmantation networks and datasets ([Link](https://github.com/mrgloom/awesome-semantic-segmentation))
* Free land use vector data of NRW ([BasisDLM](https://www.bezreg-koeln.nrw.de/brk_internet/geobasis/landschaftsmodelle/basis_dlm/index.html) or [openNRW](https://open.nrw/en/node/154))
**Other:**
* DeepHyperX - Deep learning for Hyperspectral imagery: [gitlab.inria.fr/naudeber/DeepHyperX/](https://gitlab.inria.fr/naudeber/DeepHyperX/)
## How to get Sentinel-2 data
1. Register at Copernicus [Open Access Hub](https://scihub.copernicus.eu/dhus/#/home) or [EarthExplorer](https://earthexplorer.usgs.gov/)
2. Find your region
3. Choose tile(s) (→ area) and date
* Less tiles makes things easier
* Less clouds in the image are better
* Consider multiple dates for classes like “annual crop”
4. Download L1C data
5. Decide of you want to apply L2A atmospheric corrections
* Your CNN might be able to do this by itself
* If you want to correct, use [Sen2Cor](http://step.esa.int/main/third-party-plugins-2/sen2cor/)
6. Have fun with the data
## Citation
Jens Leitloff and Felix M. Riese, "Examples for CNN training and classification on Sentinel-2 data", Zenodo, [10.5281/zenodo.3268451](http://doi.org/10.5281/zenodo.3268451), 2018.
```tex
@misc{leitloff2018examples,
author = {Leitloff, Jens and Riese, Felix~M.},
title = {{Examples for CNN training and classification on Sentinel-2 data}},
year = {2018},
DOI = {10.5281/zenodo.3268451},
publisher = {Zenodo},
howpublished = {\href{http://doi.org/10.5281/zenodo.3268451}{http://doi.org/10.5281/zenodo.3268451}}
}
```
================================================
FILE: bibliography.bib
================================================
@misc{leitloff2018examples,
author = {Leitloff, Jens and Riese, Felix~M.},
title = {{Examples for CNN training and classification on Sentinel-2 data}},
year = {2018},
DOI = {10.5281/zenodo.3268451},
publisher = {Zenodo},
howpublished = {\href{http://doi.org/10.5281/zenodo.3268451}{http://doi.org/10.5281/zenodo.3268451}}
}
@inproceedings{leitloff2018satellite,
author = {Leitloff, Jens and Riese, Felix~M.},
title = {{Satellite data is for everyone: insights into modern remote sensing research with open data and Python}},
year = {2018},
booktitle = {PyCon.DE 2018},
address = {Karlsruhe, Germany},
DOI = {10.5281/zenodo.4056516},
publisher = {Zenodo},
howpublished = {\href{http://doi.org/10.5281/zenodo.4056516}{http://doi.org/10.5281/zenodo.4056516}}
}
@inproceedings{leitloff2019satellite,
author = {Leitloff, Jens and Riese, Felix~M.},
title = {{Satellite Computer Vision mit Keras und Tensorflow - Best practices und beispiele aus der Forschung}},
year = {2019},
booktitle = {Minds Mastering Machines (M3)},
address = {Mannheim, Germany},
DOI = {10.5281/zenodo.4056744},
publisher = {Zenodo},
howpublished = {\href{http://doi.org/10.5281/zenodo.4056744}{http://doi.org/10.5281/zenodo.4056744}}
}
================================================
FILE: py/01_split_data_to_train_and_validation.py
================================================
# -*- coding: utf-8 -*-
"""
Code for the PyCon.DE 2018 talk by Jens Leitloff and Felix M. Riese.
PyCon 2018 talk: Satellite data is for everyone: insights into modern remote
sensing research with open data and Python.
License: MIT
"""
import os
import random
import shutil
random.seed(42)
# variables
# root path to folders "AnnualCrop, Forest ..." in home ("~")
path_to_all_images = "~/Documents/Data/EuroSAT/AllBands"
path_to_all_images = r'C:\Users\Jens\Downloads\EuroSAT_RGB'
# path to new created folders "train" and "validation" with subfolders
# "AnnualCrop, Forest ..." in home ("~")
path_to_split_datasets = "~/Documents/Data/PyCon/AllBands"
path_to_split_datasets = r'C:\Users\Jens\Downloads\PyCon\RGB'
# percentage of validation data (between 0 an 1)
percentage_validation = 0.3
# !!! If "True", complete "path_to_split_datasets" tree will be deleted !!!
delete_old_path_to_split_datasets = True
# contruct path
path_to_home = os.path.expanduser("~")
path_to_all_images = path_to_all_images.replace("~", path_to_home)
path_to_split_datasets = path_to_split_datasets.replace("~", path_to_home)
# create directories if necessary
if delete_old_path_to_split_datasets and os.path.isdir(path_to_split_datasets):
shutil.rmtree(path_to_split_datasets)
path_to_train = os.path.join(path_to_split_datasets, "train")
path_to_validation = os.path.join(path_to_split_datasets, "validation")
if not os.path.isdir(path_to_train):
os.makedirs(path_to_train)
if not os.path.isdir(path_to_validation):
os.makedirs(path_to_validation)
# copy files
sub_dirs = [sub_dir for sub_dir in os.listdir(path_to_all_images)
if os.path.isdir(os.path.join(path_to_all_images, sub_dir))]
for sub_dir in sub_dirs:
# list and shuffle images in class directories
current_dir = os.path.join(path_to_all_images, sub_dir)
files = os.listdir(current_dir)
random.shuffle(files)
# split files into train and validation set
split_idx = int(len(files)*percentage_validation)
files_for_validation = files[:split_idx]
files_for_train = files[split_idx:]
# copy files to path_to_split_datasets
if not os.path.isdir(os.path.join(path_to_train, sub_dir)):
os.makedirs(os.path.join(path_to_train, sub_dir))
if not os.path.isdir(os.path.join(path_to_validation, sub_dir)):
os.makedirs(os.path.join(path_to_validation, sub_dir))
for file in files_for_train:
shutil.copy2(os.path.join(current_dir, file),
os.path.join(path_to_train, sub_dir))
for file in files_for_validation:
shutil.copy2(os.path.join(current_dir, file),
os.path.join(path_to_validation, sub_dir))
================================================
FILE: py/02_train_rgb_finetuning.py
================================================
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Code for the PyCon.DE 2018 talk by Jens Leitloff and Felix M. Riese.
PyCon 2018 talk: Satellite data is for everyone: insights into modern remote
sensing research with open data and Python.
License: MIT
"""
import os
from tensorflow.keras.applications.densenet import DenseNet201 as DenseNet
from tensorflow.keras.applications.vgg16 import VGG16 as VGG
from tensorflow.keras.callbacks import (EarlyStopping, ModelCheckpoint,
TensorBoard)
from tensorflow.keras.layers import Dense, GlobalAveragePooling2D
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import SGD
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from image_functions import preprocessing_image_rgb
# variables
path_to_split_datasets = "~/Documents/Data/PyCon/RGB"
use_vgg = True
batch_size = 64
# contruct path
path_to_home = os.path.expanduser("~")
path_to_split_datasets = path_to_split_datasets.replace("~", path_to_home)
path_to_train = os.path.join(path_to_split_datasets, "train")
path_to_validation = os.path.join(path_to_split_datasets, "validation")
# get number of classes
sub_dirs = [sub_dir for sub_dir in os.listdir(path_to_train)
if os.path.isdir(os.path.join(path_to_train, sub_dir))]
num_classes = len(sub_dirs)
# parameters for CNN
if use_vgg:
base_model = VGG(include_top=False,
weights='imagenet',
input_shape=(64, 64, 3))
else:
base_model = DenseNet(include_top=False,
weights='imagenet',
input_shape=(64, 64, 3))
# add a global spatial average pooling layer
top_model = base_model.output
top_model = GlobalAveragePooling2D()(top_model)
# or just flatten the layers
# top_model = Flatten()(top_model)
# let's add a fully-connected layer
if use_vgg:
# only in VGG19 a fully connected nn is added for classfication
# DenseNet tends to overfitting if using additionally dense layers
top_model = Dense(2048, activation='relu')(top_model)
top_model = Dense(2048, activation='relu')(top_model)
# and a logistic layer
predictions = Dense(num_classes, activation='softmax')(top_model)
# this is the model we will train
model = Model(inputs=base_model.input, outputs=predictions)
# print network structure
model.summary()
# defining ImageDataGenerators
# ... initialization for training
train_datagen = ImageDataGenerator(
fill_mode="reflect",
rotation_range=45,
horizontal_flip=True,
vertical_flip=True,
preprocessing_function=preprocessing_image_rgb)
# ... initialization for validation
test_datagen = ImageDataGenerator(
preprocessing_function=preprocessing_image_rgb)
# ... definition for training
train_generator = train_datagen.flow_from_directory(path_to_train,
target_size=(64, 64),
batch_size=batch_size,
class_mode='categorical')
# just for information
class_indices = train_generator.class_indices
print(class_indices)
# ... definition for validation
validation_generator = test_datagen.flow_from_directory(
path_to_validation,
target_size=(64, 64),
batch_size=batch_size,
class_mode='categorical')
# first: train only the top layers (which were randomly initialized)
# i.e. freeze all convolutional layers
for layer in base_model.layers:
layer.trainable = False
# compile the model (should be done *after* setting layers to non-trainable)
model.compile(optimizer='adadelta', loss='categorical_crossentropy',
metrics=['categorical_accuracy'])
# generate callback to save best model w.r.t val_categorical_accuracy
if use_vgg:
file_name = "vgg"
else:
file_name = "dense"
checkpointer = ModelCheckpoint("../data/models/" + file_name +
"_rgb_transfer_init." +
"{epoch:02d}-{val_categorical_accuracy:.3f}." +
"hdf5",
monitor='val_categorical_accuracy',
verbose=1,
save_best_only=True,
mode='max')
earlystopper = EarlyStopping(monitor='val_categorical_accuracy',
patience=10,
mode='max',
restore_best_weights=True)
tensorboard = TensorBoard(log_dir='./logs', write_graph=True,
write_images=True, update_freq='epoch')
history = model.fit(
train_generator,
steps_per_epoch=1000,
epochs=10000,
callbacks=[checkpointer, earlystopper,
tensorboard],
validation_data=validation_generator,
validation_steps=500)
initial_epoch = len(history.history['loss'])+1
# at this point, the top layers are well trained and we can start fine-tuning
# convolutional layers. We will freeze the bottom N layers
# and train the remaining top layers.
# let's visualize layer names and layer indices to see how many layers
# we should freeze:
names = []
for i, layer in enumerate(model.layers):
names.append([i, layer.name, layer.trainable])
print(names)
if use_vgg:
# we will freaze the first convolutional block and train all
# remaining blocks, including top layers.
for layer in model.layers[:4]:
layer.trainable = False
for layer in model.layers[4:]:
layer.trainable = True
else:
for layer in model.layers[:7]:
layer.trainable = False
for layer in model.layers[7:]:
layer.trainable = True
# we need to recompile the model for these modifications to take effect
# we use SGD with a low learning rate
model.compile(optimizer=SGD(lr=0.0001, momentum=0.9),
loss='categorical_crossentropy',
metrics=['categorical_accuracy'])
# generate callback to save best model w.r.t val_categorical_accuracy
if use_vgg:
file_name = "vgg"
else:
file_name = "dense"
checkpointer = ModelCheckpoint("../data/models/" + file_name +
"_rgb_transfer_final." +
"{epoch:02d}-{val_categorical_accuracy:.3f}" +
".hdf5",
monitor='val_categorical_accuracy',
verbose=1,
save_best_only=True,
mode='max')
earlystopper = EarlyStopping(monitor='val_categorical_accuracy',
patience=50,
mode='max')
model.fit(
train_generator,
steps_per_epoch=1000,
epochs=10000,
callbacks=[checkpointer, earlystopper, tensorboard],
validation_data=validation_generator,
validation_steps=500,
initial_epoch=initial_epoch)
================================================
FILE: py/03_train_rgb_from_scratch.py
================================================
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Code for the PyCon.DE 2018 talk by Jens Leitloff and Felix M. Riese.
PyCon 2018 talk: Satellite data is for everyone: insights into modern remote
sensing research with open data and Python.
License: MIT
"""
import os
from tensorflow.keras.applications.densenet import DenseNet201 as DenseNet
from tensorflow.keras.applications.vgg16 import VGG16 as VGG
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint
from tensorflow.keras.layers import Dense, GlobalAveragePooling2D
from tensorflow.keras.models import Model
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from image_functions import preprocessing_image_rgb
# variables
path_to_split_datasets = "~/Documents/Data/PyCon/RGB"
use_vgg = False
batch_size = 64
# contruct path
path_to_home = os.path.expanduser("~")
path_to_split_datasets = path_to_split_datasets.replace("~", path_to_home)
path_to_train = os.path.join(path_to_split_datasets, "train")
path_to_validation = os.path.join(path_to_split_datasets, "validation")
# get number of classes
sub_dirs = [sub_dir for sub_dir in os.listdir(path_to_train)
if os.path.isdir(os.path.join(path_to_train, sub_dir))]
num_classes = len(sub_dirs)
# parameters for CNN
if use_vgg:
base_model = VGG(include_top=False,
weights=None,
input_shape=(64, 64, 3))
else:
base_model = DenseNet(include_top=False,
weights=None,
input_shape=(64, 64, 3))
# add a global spatial average pooling layer
top_model = base_model.output
top_model = GlobalAveragePooling2D()(top_model)
# or just flatten the layers
# top_model = Flatten()(top_model)
# let's add a fully-connected layer
if use_vgg:
# only in VGG19 a fully connected nn is added for classfication
# DenseNet tends to overfitting if using additionally dense layers
top_model = Dense(2048, activation='relu')(top_model)
top_model = Dense(2048, activation='relu')(top_model)
# and a logistic layer
predictions = Dense(num_classes, activation='softmax')(top_model)
# this is the model we will train
model = Model(inputs=base_model.input, outputs=predictions)
# print network structure
model.summary()
# defining ImageDataGenerators
# ... initialization for training
train_datagen = ImageDataGenerator(
fill_mode="reflect",
rotation_range=45,
horizontal_flip=True,
vertical_flip=True,
preprocessing_function=preprocessing_image_rgb)
# ... initialization for validation
test_datagen = ImageDataGenerator(
preprocessing_function=preprocessing_image_rgb)
# ... definition for training
train_generator = train_datagen.flow_from_directory(path_to_train,
target_size=(64, 64),
batch_size=batch_size,
class_mode='categorical')
print(train_generator.class_indices)
# ... definition for validation
validation_generator = test_datagen.flow_from_directory(
path_to_validation,
target_size=(64, 64),
batch_size=batch_size,
class_mode='categorical')
# compile the model (should be done *after* setting layers to non-trainable)
model.compile(optimizer='adadelta', loss='categorical_crossentropy',
metrics=['categorical_accuracy'])
# generate callback to save best model w.r.t val_categorical_accuracy
if use_vgg:
file_name = "vgg"
else:
file_name = "dense"
checkpointer = ModelCheckpoint("../data/models/" + file_name +
"_rgb_from_scratch." +
"{epoch:02d}-{val_categorical_accuracy:.3f}" +
".hdf5",
monitor='val_categorical_accuracy',
verbose=1,
save_best_only=True,
mode='max')
earlystopper = EarlyStopping(monitor='val_categorical_accuracy',
patience=50,
mode='max')
model.fit(
train_generator,
steps_per_epoch=1000,
epochs=10000,
callbacks=[checkpointer, earlystopper],
validation_data=validation_generator,
validation_steps=500)
================================================
FILE: py/04_train_ms_finetuning.py
================================================
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Code for the PyCon.DE 2018 talk by Jens Leitloff and Felix M. Riese.
PyCon 2018 talk: Satellite data is for everyone: insights into modern remote
sensing research with open data and Python.
License: MIT
"""
import os
from glob import glob
from tensorflow.keras.applications.densenet import DenseNet201 as DenseNet
from tensorflow.keras.applications.vgg16 import VGG16 as VGG
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint
from tensorflow.keras.layers import (Conv2D, Dense, GlobalAveragePooling2D,
Input)
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import SGD
from image_functions import simple_image_generator
# variables
path_to_split_datasets = "~/Documents/Data/PyCon/AllBands"
use_vgg = False
batch_size = 64
class_indices = {'AnnualCrop': 0, 'Forest': 1, 'HerbaceousVegetation': 2,
'Highway': 3, 'Industrial': 4, 'Pasture': 5,
'PermanentCrop': 6, 'Residential': 7, 'River': 8,
'SeaLake': 9}
num_classes = len(class_indices)
# contruct path
path_to_home = os.path.expanduser("~")
path_to_split_datasets = path_to_split_datasets.replace("~", path_to_home)
path_to_train = os.path.join(path_to_split_datasets, "train")
path_to_validation = os.path.join(path_to_split_datasets, "validation")
# parameters for CNN
input_tensor = Input(shape=(64, 64, 13))
# introduce a additional layer to get from 13 to 3 input channels
input_tensor = Conv2D(3, (1, 1))(input_tensor)
if use_vgg:
base_model_imagenet = VGG(include_top=False,
weights='imagenet',
input_shape=(64, 64, 3))
base_model = VGG(include_top=False,
weights=None,
input_tensor=input_tensor)
for i, layer in enumerate(base_model_imagenet.layers):
# we must skip input layer, which has no weights
if i == 0:
continue
base_model.layers[i+1].set_weights(layer.get_weights())
else:
base_model_imagenet = DenseNet(include_top=False,
weights='imagenet',
input_shape=(64, 64, 3))
base_model = DenseNet(include_top=False,
weights=None,
input_tensor=input_tensor)
for i, layer in enumerate(base_model_imagenet.layers):
# we must skip input layer, which has no weights
if i == 0:
continue
base_model.layers[i+1].set_weights(layer.get_weights())
# add a global spatial average pooling layer
top_model = base_model.output
top_model = GlobalAveragePooling2D()(top_model)
# or just flatten the layers
# top_model = Flatten()(top_model)
# let's add a fully-connected layer
if use_vgg:
# only in VGG19 a fully connected nn is added for classfication
# DenseNet tends to overfitting if using additionally dense layers
top_model = Dense(2048, activation='relu')(top_model)
top_model = Dense(2048, activation='relu')(top_model)
# and a logistic layer
predictions = Dense(num_classes, activation='softmax')(top_model)
# this is the model we will train
model = Model(inputs=base_model.input, outputs=predictions)
# print network structure
model.summary()
# defining ImageDataGenerators
# ... initialization for training
training_files = glob(path_to_train + "/**/*.tif")
train_generator = simple_image_generator(training_files, class_indices,
batch_size=batch_size,
rotation_range=45,
horizontal_flip=True,
vertical_flip=True)
# ... initialization for validation
validation_files = glob(path_to_validation + "/**/*.tif")
validation_generator = simple_image_generator(validation_files, class_indices,
batch_size=batch_size)
# first: train only the top layers (which were randomly initialized)
# i.e. freeze all convolutional layers
for layer in base_model.layers:
layer.trainable = False
# set convolution block for reducing 13 to 3 layers trainable
for layer in model.layers[:2]:
layer.trainable = True
# compile the model (should be done *after* setting layers to non-trainable)
model.compile(optimizer='adam', loss='categorical_crossentropy',
metrics=['categorical_accuracy'])
# generate callback to save best model w.r.t val_categorical_accuracy
if use_vgg:
file_name = "vgg"
else:
file_name = "dense"
checkpointer = ModelCheckpoint("../data/models/" + file_name +
"_ms_transfer_init." +
"{epoch:02d}-{val_categorical_accuracy:.3f}." +
"hdf5",
monitor='val_categorical_accuracy',
verbose=1,
save_best_only=True,
mode='max')
earlystopper = EarlyStopping(monitor='val_categorical_accuracy',
patience=10,
mode='max',
restore_best_weights=True)
history = model.fit(
train_generator,
steps_per_epoch=1000,
epochs=10000,
callbacks=[checkpointer, earlystopper],
validation_data=validation_generator,
validation_steps=500)
initial_epoch = len(history.history['loss'])+1
# at this point, the top layers are well trained and we can start fine-tuning
# convolutional layers. We will freeze the bottom N layers
# and train the remaining top layers.
# let's visualize layer names and layer indices to see how many layers
# we should freeze:
names = []
for i, layer in enumerate(model.layers):
names.append([i, layer.name, layer.trainable])
print(names)
if use_vgg:
# we will freaze the first convolutional block and train all
# remaining blocks, including top layers.
for layer in model.layers[:2]:
layer.trainable = True
for layer in model.layers[2:5]:
layer.trainable = False
for layer in model.layers[5:]:
layer.trainable = True
else:
for layer in model.layers[:2]:
layer.trainable = True
for layer in model.layers[2:8]:
layer.trainable = False
for layer in model.layers[8:]:
layer.trainable = True
# we need to recompile the model for these modifications to take effect
# we use SGD with a low learning rate
model.compile(optimizer=SGD(lr=0.0001, momentum=0.9),
loss='categorical_crossentropy',
metrics=['categorical_accuracy'])
# generate callback to save best model w.r.t val_categorical_accuracy
if use_vgg:
file_name = "vgg"
else:
file_name = "dense"
checkpointer = ModelCheckpoint("../data/models/" + file_name +
"_ms_transfer_final." +
"{epoch:02d}-{val_categorical_accuracy:.3f}" +
".hdf5",
monitor='val_categorical_accuracy',
verbose=1,
save_best_only=True,
mode='max')
earlystopper = EarlyStopping(monitor='val_categorical_accuracy',
patience=10,
mode='max')
model.fit(
train_generator,
steps_per_epoch=1000,
epochs=10000,
callbacks=[checkpointer, earlystopper],
validation_data=validation_generator,
validation_steps=500,
initial_epoch=initial_epoch)
================================================
FILE: py/04_train_ms_finetuning_alternative.py
================================================
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Code for the PyCon.DE 2018 talk by Jens Leitloff and Felix M. Riese.
PyCon 2018 talk: Satellite data is for everyone: insights into modern remote
sensing research with open data and Python.
License: MIT
"""
import os
from glob import glob
# from tensorflow.keras.applications.vgg19 import VGG19 as VGG
# from tensorflow.keras.applications.densenet import DenseNet121 as DenseNet
from tensorflow.keras.applications.densenet import DenseNet201 as DenseNet
from tensorflow.keras.applications.vgg16 import VGG16 as VGG
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint
from tensorflow.keras.layers import Dense, GlobalAveragePooling2D
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import SGD
from image_functions import simple_image_generator
# variables
path_to_split_datasets = "~/Dokumente/Data/PyCon/AllBands"
use_vgg = False
batch_size = 64
class_indices = {'AnnualCrop': 0, 'Forest': 1, 'HerbaceousVegetation': 2,
'Highway': 3, 'Industrial': 4, 'Pasture': 5,
'PermanentCrop': 6, 'Residential': 7, 'River': 8,
'SeaLake': 9}
num_classes = len(class_indices)
# contruct path
path_to_home = os.path.expanduser("~")
path_to_split_datasets = path_to_split_datasets.replace("~", path_to_home)
path_to_train = os.path.join(path_to_split_datasets, "train")
path_to_validation = os.path.join(path_to_split_datasets, "validation")
# parameters for CNN
if use_vgg:
base_model_imagenet = VGG(include_top=False,
weights='imagenet',
input_shape=(64, 64, 3))
base_model = VGG(include_top=False,
weights=None,
input_shape=(64, 64, 13))
for i, layer in enumerate(base_model_imagenet.layers):
# we must skip input layer and first convolutional layer
if i < 2:
continue
base_model.layers[i].set_weights(layer.get_weights())
else:
base_model_imagenet = DenseNet(include_top=False,
weights='imagenet',
input_shape=(64, 64, 3))
base_model = DenseNet(include_top=False,
weights=None,
input_shape=(64, 64, 13))
for i, layer in enumerate(base_model_imagenet.layers):
# we must skip input layer, zeropadding and first convolutional layer
if i < 3:
continue
base_model.layers[i].set_weights(layer.get_weights())
# add a global spatial average pooling layer
top_model = base_model.output
top_model = GlobalAveragePooling2D()(top_model)
# or just flatten the layers
# top_model = Flatten()(top_model)
# let's add a fully-connected layer
if use_vgg:
# only in VGG19 a fully connected nn is added for classfication
# DenseNet tends to overfitting if using additionally dense layers
top_model = Dense(2048, activation='relu')(top_model)
top_model = Dense(2048, activation='relu')(top_model)
# and a logistic layer
predictions = Dense(num_classes, activation='softmax')(top_model)
# this is the model we will train
model = Model(inputs=base_model.input, outputs=predictions)
# print network structure
model.summary()
# defining ImageDataGenerators
# ... initialization for training
training_files = glob(path_to_train + "/**/*.tif")
train_generator = simple_image_generator(training_files, class_indices,
batch_size=batch_size,
rotation_range=45,
horizontal_flip=True,
vertical_flip=True)
# ... initialization for validation
validation_files = glob(path_to_validation + "/**/*.tif")
validation_generator = simple_image_generator(validation_files, class_indices,
batch_size=batch_size)
# first: train only the top layers (which were randomly initialized)
# i.e. freeze all convolutional layers
for layer in base_model.layers:
layer.trainable = False
# set first convolution block for reducing 13 to 3 layers trainable
for layer in model.layers[:3]:
layer.trainable = True
# compile the model (should be done *after* setting layers to non-trainable)
model.compile(optimizer='adam', loss='categorical_crossentropy',
metrics=['categorical_accuracy'])
# generate callback to save best model w.r.t val_categorical_accuracy
if use_vgg:
file_name = "vgg"
else:
file_name = "dense"
checkpointer = ModelCheckpoint("../data/models/" + file_name +
"_ms_transfer_alternative_init." +
"{epoch:02d}-{val_categorical_accuracy:.3f}." +
"hdf5",
monitor='val_categorical_accuracy',
verbose=1,
save_best_only=True,
mode='max')
earlystopper = EarlyStopping(monitor='val_categorical_accuracy',
patience=10,
mode='max',
restore_best_weights=True)
history = model.fit(
train_generator,
steps_per_epoch=1000,
epochs=10000,
callbacks=[checkpointer, earlystopper],
validation_data=validation_generator,
validation_steps=500)
initial_epoch = len(history.history['loss'])+1
# at this point, the top layers are well trained and we can start fine-tuning
# convolutional layers. We will freeze the bottom N layers
# and train the remaining top layers.
# let's visualize layer names and layer indices to see how many layers
# we should freeze:
names = []
for i, layer in enumerate(model.layers):
names.append([i, layer.name, layer.trainable])
print(names)
if use_vgg:
# we will freaze the first convolutional block and train all
# remaining blocks, including top layers.
for layer in model.layers[:2]:
layer.trainable = True
for layer in model.layers[2:5]:
layer.trainable = False
for layer in model.layers[5:]:
layer.trainable = True
else:
for layer in model.layers[:3]:
layer.trainable = True
for layer in model.layers[3:7]:
layer.trainable = False
for layer in model.layers[7:]:
layer.trainable = True
# we need to recompile the model for these modifications to take effect
# we use SGD with a low learning rate
model.compile(optimizer=SGD(lr=0.0001, momentum=0.9),
loss='categorical_crossentropy',
metrics=['categorical_accuracy'])
# generate callback to save best model w.r.t val_categorical_accuracy
if use_vgg:
file_name = "vgg"
else:
file_name = "dense"
checkpointer = ModelCheckpoint("../data/models/" + file_name +
"_ms_transfer_alternative_final." +
"{epoch:02d}-{val_categorical_accuracy:.3f}" +
".hdf5",
monitor='val_categorical_accuracy',
verbose=1,
save_best_only=True,
mode='max')
earlystopper = EarlyStopping(monitor='val_categorical_accuracy',
patience=10,
mode='max')
model.fit(
train_generator,
steps_per_epoch=1000,
epochs=10000,
callbacks=[checkpointer, earlystopper],
validation_data=validation_generator,
validation_steps=500,
initial_epoch=initial_epoch)
================================================
FILE: py/05_train_ms_from_scratch.py
================================================
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Code for the PyCon.DE 2018 talk by Jens Leitloff and Felix M. Riese.
PyCon 2018 talk: Satellite data is for everyone: insights into modern remote
sensing research with open data and Python.
License: MIT
"""
import os
from glob import glob
from tensorflow.keras.applications.densenet import DenseNet201 as DenseNet
from tensorflow.keras.applications.vgg16 import VGG16 as VGG
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint
from tensorflow.keras.layers import Dense, GlobalAveragePooling2D
from tensorflow.keras.models import Model
from image_functions import simple_image_generator
# variables
path_to_split_datasets = "~/Documents/Data/PyCon/AllBands"
use_vgg = False
batch_size = 64
class_indices = {'AnnualCrop': 0, 'Forest': 1, 'HerbaceousVegetation': 2,
'Highway': 3, 'Industrial': 4, 'Pasture': 5,
'PermanentCrop': 6, 'Residential': 7, 'River': 8,
'SeaLake': 9}
num_classes = len(class_indices)
# contruct path
path_to_home = os.path.expanduser("~")
path_to_split_datasets = path_to_split_datasets.replace("~", path_to_home)
path_to_train = os.path.join(path_to_split_datasets, "train")
path_to_validation = os.path.join(path_to_split_datasets, "validation")
# parameters for CNN
if use_vgg:
base_model = VGG(include_top=False,
weights=None,
input_shape=(64, 64, 13))
else:
base_model = DenseNet(include_top=False,
weights=None,
input_shape=(64, 64, 13))
# add a global spatial average pooling layer
top_model = base_model.output
top_model = GlobalAveragePooling2D()(top_model)
# or just flatten the layers
# top_model = Flatten()(top_model)
# let's add a fully-connected layer
if use_vgg:
# only in VGG19 a fully connected nn is added for classfication
# DenseNet tends to overfitting if using additionally dense layers
top_model = Dense(2048, activation='relu')(top_model)
top_model = Dense(2048, activation='relu')(top_model)
# and a logistic layer
predictions = Dense(num_classes, activation='softmax')(top_model)
# this is the model we will train
model = Model(inputs=base_model.input, outputs=predictions)
# print network structure
model.summary()
# defining ImageDataGenerators
# ... initialization for training
training_files = glob(path_to_train + "/**/*.tif")
train_generator = simple_image_generator(training_files, class_indices,
batch_size=batch_size,
rotation_range=45,
horizontal_flip=True,
vertical_flip=True)
# ... initialization for validation
validation_files = glob(path_to_validation + "/**/*.tif")
validation_generator = simple_image_generator(validation_files, class_indices,
batch_size=batch_size)
# compile the model
model.compile(optimizer='adam', loss='categorical_crossentropy',
metrics=['categorical_accuracy'])
# generate callback to save best model w.r.t val_categorical_accuracy
if use_vgg:
file_name = "vgg"
else:
file_name = "dense"
checkpointer = ModelCheckpoint("../data/models/" + file_name +
"_ms_from_scratch." +
"{epoch:02d}-{val_categorical_accuracy:.3f}." +
"hdf5",
monitor='val_categorical_accuracy',
verbose=1,
save_best_only=True,
mode='max')
earlystopper = EarlyStopping(monitor='val_categorical_accuracy',
patience=50,
mode='max',
restore_best_weights=True)
model.fit(
train_generator,
steps_per_epoch=1000,
epochs=10000,
callbacks=[checkpointer, earlystopper],
validation_data=validation_generator,
validation_steps=500)
================================================
FILE: py/06_classify_image.py
================================================
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Code for the PyCon.DE 2018 talk by Jens Leitloff and Felix M. Riese.
PyCon 2018 talk: Satellite data is for everyone: insights into modern remote
sensing research with open data and Python.
License: MIT
"""
import gdal
import numpy as np
from skimage.io import imread
from skimage.util import pad
from tensorflow.keras.models import load_model
from tqdm import tqdm
# input files
path_to_image = "../data/karlsruhe/2018_zugeschnitten.tif"
path_to_model = "../data/models/vgg/vgg_ms_transfer_alternative_final.27-0.985.hdf5"
# output files
path_to_label_image = "../data/karlsruhe/2018_zugeschnitten_10m_vgg_ms_label.tif"
path_to_prob_image = "../data/karlsruhe/2018_zugeschnitten_10m_vgg_ms_prob.tif"
# read image and model
image = np.array(imread(path_to_image), dtype=float)
_, num_cols_unpadded, _ = image.shape
model = load_model(path_to_model)
# get input shape of model
_, input_rows, input_cols, input_channels = model.layers[0].input_shape
_, output_classes = model.layers[-1].output_shape
in_rows_half = int(input_rows/2)
in_cols_half = int(input_cols/2)
# import correct preprocessing
if input_channels is 3:
from image_functions import preprocessing_image_rgb as preprocessing_image
else:
from image_functions import preprocessing_image_ms as preprocessing_image
# pad image
image = pad(image, ((input_rows, input_rows),
(input_cols, input_cols),
(0, 0)), 'symmetric')
# don't forget to preprocess
image = preprocessing_image(image)
num_rows, num_cols, _ = image.shape
# sliding window over image
image_classified_prob = np.zeros((num_rows, num_cols, output_classes))
row_images = np.zeros((num_cols_unpadded, input_rows,
input_cols, input_channels))
for row in tqdm(range(input_rows, num_rows-input_rows), desc="Processing..."):
# get all images along one row
for idx, col in enumerate(range(input_cols, num_cols-input_cols)):
# cut smal image patch
row_images[idx, ...] = image[row-in_rows_half:row+in_rows_half,
col-in_cols_half:col+in_cols_half, :]
# classify images
row_classified = model.predict(row_images, batch_size=1024, verbose=0)
# put them to final image
image_classified_prob[row, input_cols:num_cols-input_cols, : ] = row_classified
# crop padded final image
image_classified_prob = image_classified_prob[input_rows:num_rows-input_rows,
input_cols:num_cols-input_cols, :]
image_classified_label = np.argmax(image_classified_prob, axis=-1)
image_classified_prob = np.sort(image_classified_prob, axis=-1)[..., -1]
# write image as Geotiff for correct georeferencing
# read geotransformation
image = gdal.Open(path_to_image, gdal.GA_ReadOnly)
geotransform = image.GetGeoTransform()
# create image driver
driver = gdal.GetDriverByName('GTiff')
# create destination for label file
file = driver.Create(path_to_label_image,
image_classified_label.shape[1],
image_classified_label.shape[0],
1,
gdal.GDT_Byte,
['TFW=YES', 'NUM_THREADS=1'])
file.SetGeoTransform(geotransform)
file.SetProjection(image.GetProjection())
# write label file
file.GetRasterBand(1).WriteArray(image_classified_label)
file = None
# create destination for probability file
file = driver.Create(path_to_prob_image,
image_classified_prob.shape[1],
image_classified_prob.shape[0],
1,
gdal.GDT_Float32,
['TFW=YES', 'NUM_THREADS=1'])
file.SetGeoTransform(geotransform)
file.SetProjection(image.GetProjection())
# write label file
file.GetRasterBand(1).WriteArray(image_classified_prob)
file = None
image = None
================================================
FILE: py/Image_functions.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Notebook for image_functions.py"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from random import choice, sample\n",
"\n",
"import numpy as np\n",
"from skimage.io import imread\n",
"from skimage.transform import rotate\n",
"from tensorflow.keras.utils import to_categorical"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Preprocessing"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### RGB data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def preprocessing_image_rgb(x):\n",
" # define mean and std values\n",
" mean = [87.845, 96.965, 103.947]\n",
" std = [23.657, 16.474, 13.793]\n",
" # loop over image channels\n",
" for idx, mean_value in enumerate(mean):\n",
" x[..., idx] -= mean_value\n",
" x[..., idx] /= std[idx]\n",
" return x"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### MS data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def preprocessing_image_ms(x):\n",
" # define mean and std values\n",
" mean = [1353.036, 1116.468, 1041.475, 945.344, 1198.498, 2004.878,\n",
" 2376.699, 2303.738, 732.957, 12.092, 1818.820, 1116.271, 2602.579]\n",
" std = [65.479, 154.008, 187.997, 278.508, 228.122, 356.598, 456.035,\n",
" 531.570, 98.947, 1.188, 378.993, 303.851, 503.181]\n",
" # loop over image channels\n",
" for idx, mean_value in enumerate(mean):\n",
" x[..., idx] -= mean_value\n",
" x[..., idx] /= std[idx]\n",
" return x"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Image Generator for MS data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Get label from file name"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def categorical_label_from_full_file_name(files, class_indices):\n",
" # file basename without extension\n",
" base_name = [os.path.splitext(os.path.basename(i))[0] for i in files]\n",
" # class label from filename\n",
" base_name = [i.split(\"_\")[0] for i in base_name]\n",
" # label to indices\n",
" image_class = [class_indices[i] for i in base_name]\n",
" # class indices to one-hot-label\n",
" return to_categorical(image_class, num_classes=len(class_indices))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Generate images"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def simple_image_generator(files, class_indices, batch_size=32,\n",
" rotation_range=0, horizontal_flip=False,\n",
" vertical_flip=False):\n",
"\n",
" while True:\n",
" # select batch_size number of samples without replacement\n",
" batch_files = sample(files, batch_size)\n",
" # get one_hot_label\n",
" batch_Y = categorical_label_from_full_file_name(batch_files,\n",
" class_indices)\n",
" # array for images\n",
" batch_X = []\n",
" # loop over images of the current batch\n",
" for idx, input_path in enumerate(batch_files):\n",
" image = np.array(imread(input_path), dtype=float)\n",
" image = preprocessing_image_ms(image)\n",
" # process image\n",
" if horizontal_flip:\n",
" # randomly flip image up/down\n",
" if choice([True, False]):\n",
" image = np.flipud(image)\n",
" if vertical_flip:\n",
" # randomly flip image left/right\n",
" if choice([True, False]):\n",
" image = np.fliplr(image)\n",
" # rotate image by random angle between\n",
" # -rotation_range <= angle < rotation_range\n",
" if rotation_range is not 0:\n",
" angle = np.random.uniform(low=-abs(rotation_range),\n",
" high=abs(rotation_range))\n",
" image = rotate(image, angle, mode='reflect',\n",
" order=1, preserve_range=True)\n",
" # put all together\n",
" batch_X += [image]\n",
" # convert lists to np.array\n",
" X = np.array(batch_X)\n",
" Y = np.array(batch_Y)\n",
"\n",
" yield(X, Y)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.7"
},
"varInspector": {
"cols": {
"lenName": 16,
"lenType": 16,
"lenVar": 40
},
"kernels_config": {
"python": {
"delete_cmd_postfix": "",
"delete_cmd_prefix": "del ",
"library": "var_list.py",
"varRefreshCmd": "print(var_dic_list())"
},
"r": {
"delete_cmd_postfix": ") ",
"delete_cmd_prefix": "rm(",
"library": "var_list.r",
"varRefreshCmd": "cat(var_dic_list()) "
}
},
"types_to_exclude": [
"module",
"function",
"builtin_function_or_method",
"instance",
"_Feature"
],
"window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 2
}
================================================
FILE: py/Train_from_Scratch.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Notebook for 05_train_ms_from_scratch.py"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Import libaries"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from glob import glob\n",
"\n",
"from tensorflow.keras.applications.vgg16 import VGG16 as VGG\n",
"from tensorflow.keras.applications.densenet import DenseNet201 as DenseNet\n",
"from tensorflow.keras.layers import GlobalAveragePooling2D, Dense\n",
"from tensorflow.keras.models import Model\n",
"from tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping, TensorBoard\n",
"\n",
"from image_functions import simple_image_generator"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### define path to training and validation data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# variables\n",
"path_to_split_datasets = \"~/Documents/Data/PyCon/AllBands\"\n",
"use_vgg = False\n",
"batch_size = 64\n",
"\n",
"# contruct path\n",
"path_to_home = os.path.expanduser(\"~\")\n",
"path_to_split_datasets = path_to_split_datasets.replace(\"~\", path_to_home)\n",
"path_to_train = os.path.join(path_to_split_datasets, \"train\")\n",
"path_to_validation = os.path.join(path_to_split_datasets, \"validation\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![tree](images_for_notebook/tree_files.png \"file_tree\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### define classes"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"class_indices = {'AnnualCrop': 0, 'Forest': 1, 'HerbaceousVegetation': 2,\n",
" 'Highway': 3, 'Industrial': 4, 'Pasture': 5,\n",
" 'PermanentCrop': 6, 'Residential': 7, 'River': 8,\n",
" 'SeaLake': 9}\n",
"num_classes = len(class_indices)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Training from scratch"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![vgg16](images_for_notebook/vgg16.png \"Original VGG\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1. Initialize network model without top layers"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![vgg16_no_top](images_for_notebook/vgg16_no_top.png \"VGG no top\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# parameters for CNN\n",
"if use_vgg:\n",
" base_model = VGG(include_top=False,\n",
" weights=None,\n",
" input_shape=(64, 64, 13))\n",
"else:\n",
" base_model = DenseNet(include_top=False,\n",
" weights=None,\n",
" input_shape=(64, 64, 13))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 2. define new top layers"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![vgg16_sentinel_rgb](images_for_notebook/vgg16_sentinel_rgb.png \"VGG RGB Sentinel\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# add a global spatial average pooling layer\n",
"top_model = base_model.output\n",
"top_model = GlobalAveragePooling2D()(top_model)\n",
"# or just flatten the layers\n",
"# top_model = Flatten()(top_model)\n",
"# let's add a fully-connected layer\n",
"if use_vgg:\n",
" # only in VGG19 a fully connected nn is added for classfication\n",
" # DenseNet tends to overfitting if using additionally dense layers\n",
" top_model = Dense(2048, activation='relu')(top_model)\n",
" top_model = Dense(2048, activation='relu')(top_model)\n",
"# and a logistic layer\n",
"predictions = Dense(num_classes, activation='softmax')(top_model)\n",
"# this is the model we will train\n",
"model = Model(inputs=base_model.input, outputs=predictions)\n",
"# print network structure\n",
"model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3. define data augmentation"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# defining ImageDataGenerators\n",
"# ... initialization for training\n",
"training_files = glob(path_to_train + \"/**/*.tif\")\n",
"train_generator = simple_image_generator(training_files, class_indices,\n",
" batch_size=batch_size,\n",
" rotation_range=45,\n",
" horizontal_flip=True,\n",
" vertical_flip=True)\n",
"\n",
"# ... initialization for validation\n",
"validation_files = glob(path_to_validation + \"/**/*.tif\")\n",
"validation_generator = simple_image_generator(validation_files, class_indices,\n",
" batch_size=batch_size)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 4. define callbacks"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# generate callback to save best model w.r.t val_categorical_accuracy\n",
"if use_vgg:\n",
" file_name = \"vgg\"\n",
"else:\n",
" file_name = \"dense\"\n",
"checkpointer = ModelCheckpoint(\"../data/models/\" + file_name +\n",
" \"_ms_from_scratch.\" +\n",
" \"{epoch:02d}-{val_categorical_accuracy:.3f}.\" +\n",
" \"hdf5\",\n",
" monitor='val_categorical_accuracy',\n",
" verbose=1,\n",
" save_best_only=True,\n",
" mode='max')\n",
"earlystopper = EarlyStopping(monitor='val_categorical_accuracy',\n",
" patience=50,\n",
" mode='max',\n",
" restore_best_weights=True)\n",
"\n",
"tensorboard = TensorBoard(log_dir='./logs', write_graph=True,\n",
" write_images=True, update_freq='epoch')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![tensorflow](images_for_notebook/tensorflow.png \"VGG RGB Sentinel\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 8. fit model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# compile the model\n",
"model.compile(optimizer='adam', loss='categorical_crossentropy',\n",
" metrics=['categorical_accuracy'])\n",
"\n",
"model.fit(\n",
" train_generator,\n",
" steps_per_epoch=100,\n",
" epochs=5,\n",
" callbacks=[checkpointer, earlystopper, tensorboard],\n",
" validation_data=validation_generator,\n",
" validation_steps=500)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
},
"varInspector": {
"cols": {
"lenName": 16,
"lenType": 16,
"lenVar": 40
},
"kernels_config": {
"python": {
"delete_cmd_postfix": "",
"delete_cmd_prefix": "del ",
"library": "var_list.py",
"varRefreshCmd": "print(var_dic_list())"
},
"r": {
"delete_cmd_postfix": ") ",
"delete_cmd_prefix": "rm(",
"library": "var_list.r",
"varRefreshCmd": "cat(var_dic_list()) "
}
},
"types_to_exclude": [
"module",
"function",
"builtin_function_or_method",
"instance",
"_Feature"
],
"window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 2
}
================================================
FILE: py/Transfer_learning.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Note for 02_train_rgb_finetuning.py"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Import libaries"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Using TensorFlow backend.\n"
]
}
],
"source": [
"import os\n",
"\n",
"from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
"from tensorflow.keras.applications.vgg16 import VGG16 as VGG\n",
"from tensorflow.keras.applications.densenet import DenseNet201 as DenseNet\n",
"from tensorflow.keras.optimizers import SGD\n",
"from tensorflow.keras.layers import GlobalAveragePooling2D, Dense\n",
"from tensorflow.keras.models import Model\n",
"from tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping, TensorBoard\n",
"\n",
"from image_functions import preprocessing_image_rgb"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### define path to training and validation data"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"# variables\n",
"path_to_split_datasets = \"~/Documents/Data/PyCon/RGB\"\n",
"use_vgg = True\n",
"batch_size = 64\n",
"\n",
"# contruct path\n",
"path_to_home = os.path.expanduser(\"~\")\n",
"path_to_split_datasets = path_to_split_datasets.replace(\"~\", path_to_home)\n",
"path_to_train = os.path.join(path_to_split_datasets, \"train\")\n",
"path_to_validation = os.path.join(path_to_split_datasets, \"validation\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![tree](images_for_notebook/tree_files.png \"file_tree\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### determine number of classes from data"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"# get number of classes\n",
"sub_dirs = [sub_dir for sub_dir in os.listdir(path_to_train)\n",
" if os.path.isdir(os.path.join(path_to_train, sub_dir))]\n",
"num_classes = len(sub_dirs)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Transfer-learning "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![vgg16](images_for_notebook/vgg16.png \"Original VGG\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1. Pretrained network model without top layers"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![vgg16_no_top](images_for_notebook/vgg16_no_top.png \"VGG no top\")"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"# parameters for CNN\n",
"if use_vgg:\n",
" base_model = VGG(include_top=False,\n",
" weights='imagenet',\n",
" input_shape=(64, 64, 3))\n",
"else:\n",
" base_model = DenseNet(include_top=False,\n",
" weights='imagenet',\n",
" input_shape=(64, 64, 3))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 2. define new top layers"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![vgg16_sentinel_rgb](images_for_notebook/vgg16_sentinel_rgb.png \"VGG RGB Sentinel\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# add a global spatial average pooling layer\n",
"top_model = base_model.output\n",
"top_model = GlobalAveragePooling2D()(top_model)\n",
"# or just flatten the layers\n",
"# top_model = Flatten()(top_model)\n",
"# let's add a fully-connected layer\n",
"if use_vgg:\n",
" # only in VGG19 a fully connected nn is added for classfication\n",
" # DenseNet tends to overfitting if using additionally dense layers\n",
" top_model = Dense(2048, activation='relu')(top_model)\n",
" top_model = Dense(2048, activation='relu')(top_model)\n",
"# and a logistic layer\n",
"predictions = Dense(num_classes, activation='softmax')(top_model)\n",
"\n",
"# this is the model we will train\n",
"model = Model(inputs=base_model.input, outputs=predictions)\n",
"\n",
"# print network structure\n",
"model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3. define data augmentation"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Found 18900 images belonging to 10 classes.\n",
"{'AnnualCrop': 0, 'Forest': 1, 'HerbaceousVegetation': 2, 'Highway': 3, 'Industrial': 4, 'Pasture': 5, 'PermanentCrop': 6, 'Residential': 7, 'River': 8, 'SeaLake': 9}\n",
"Found 8100 images belonging to 10 classes.\n"
]
}
],
"source": [
"# defining ImageDataGenerators\n",
"# ... initialization for training\n",
"train_datagen = ImageDataGenerator(fill_mode=\"reflect\",\n",
" rotation_range=45,\n",
" horizontal_flip=True,\n",
" vertical_flip=True,\n",
" preprocessing_function=preprocessing_image_rgb)\n",
"# ... initialization for validation\n",
"test_datagen = ImageDataGenerator(preprocessing_function=preprocessing_image_rgb)\n",
"# ... definition for training\n",
"train_generator = train_datagen.flow_from_directory(path_to_train,\n",
" target_size=(64, 64),\n",
" batch_size=batch_size,\n",
" class_mode='categorical')\n",
"# just for information\n",
"class_indices = train_generator.class_indices\n",
"print(class_indices)\n",
"\n",
"# ... definition for validation\n",
"validation_generator = test_datagen.flow_from_directory(path_to_validation,\n",
" target_size=(64, 64),\n",
" batch_size=batch_size,\n",
" class_mode='categorical')\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 4. define callbacks"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"# generate callback to save best model w.r.t val_categorical_accuracy\n",
"if use_vgg:\n",
" file_name = \"vgg\"\n",
"else:\n",
" file_name = \"dense\"\n",
"\n",
"checkpointer = ModelCheckpoint(\"../data/models/\" + file_name +\n",
" \"_rgb_transfer_init.\" +\n",
" \"{epoch:02d}-{val_categorical_accuracy:.3f}.\" +\n",
" \"hdf5\",\n",
" monitor='val_categorical_accuracy',\n",
" verbose=1,\n",
" save_best_only=True,\n",
" mode='max')\n",
"\n",
"earlystopper = EarlyStopping(monitor='val_categorical_accuracy',\n",
" patience=10,\n",
" mode='max',\n",
" restore_best_weights=True)\n",
"\n",
"tensorboard = TensorBoard(log_dir='./logs', write_graph=True, \n",
" write_images=True, update_freq='epoch')\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![tensorflow](images_for_notebook/tensorflow.png \"VGG RGB Sentinel\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 5. set base layers non trainable "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![vgg16_rgb_init](images_for_notebook/vgg16_rgb_init.png \"VGG RGB Sentinel\")"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"# first: train only the top layers (which were randomly initialized)\n",
"# i.e. freeze all convolutional layers\n",
"for layer in base_model.layers:\n",
" layer.trainable = False\n",
"\n",
"# compile the model (should be done *after* setting layers to non-trainable)\n",
"model.compile(optimizer='adadelta', loss='categorical_crossentropy',\n",
" metrics=['categorical_accuracy'])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 6. fit model (train new top layers)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/5\n",
"100/100 [==============================] - 18s 182ms/step - loss: 0.6681 - categorical_accuracy: 0.7841 - val_loss: 0.3120 - val_categorical_accuracy: 0.8931\n",
"\n",
"Epoch 00001: val_categorical_accuracy improved from -inf to 0.89309, saving model to ../data/models/vgg_rgb_transfer_init.01-0.893.hdf5\n",
"Epoch 2/5\n",
"100/100 [==============================] - 17s 174ms/step - loss: 0.3477 - categorical_accuracy: 0.8808 - val_loss: 0.2733 - val_categorical_accuracy: 0.9039\n",
"\n",
"Epoch 00002: val_categorical_accuracy improved from 0.89309 to 0.90388, saving model to ../data/models/vgg_rgb_transfer_init.02-0.904.hdf5\n",
"Epoch 3/5\n",
"100/100 [==============================] - 17s 172ms/step - loss: 0.3098 - categorical_accuracy: 0.8961 - val_loss: 0.2515 - val_categorical_accuracy: 0.9126\n",
"\n",
"Epoch 00003: val_categorical_accuracy improved from 0.90388 to 0.91263, saving model to ../data/models/vgg_rgb_transfer_init.03-0.913.hdf5\n",
"Epoch 4/5\n",
"100/100 [==============================] - 17s 174ms/step - loss: 0.2719 - categorical_accuracy: 0.9106 - val_loss: 0.2330 - val_categorical_accuracy: 0.9200\n",
"\n",
"Epoch 00004: val_categorical_accuracy improved from 0.91263 to 0.92003, saving model to ../data/models/vgg_rgb_transfer_init.04-0.920.hdf5\n",
"Epoch 5/5\n",
"100/100 [==============================] - 17s 172ms/step - loss: 0.2601 - categorical_accuracy: 0.9136 - val_loss: 0.3209 - val_categorical_accuracy: 0.8893\n",
"\n",
"Epoch 00005: val_categorical_accuracy did not improve from 0.92003\n"
]
}
],
"source": [
"history = model.fit(train_generator,\n",
" steps_per_epoch=100,\n",
" epochs=5,\n",
" callbacks=[checkpointer, earlystopper,\n",
" tensorboard],\n",
" validation_data=validation_generator,\n",
" validation_steps=500)\n",
"initial_epoch = len(history.history['loss'])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 7. set (some) base layers trainable"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# at this point, the top layers are well trained and we can start fine-tuning\n",
"# convolutional layers. We will freeze the bottom N layers\n",
"# and train the remaining top layers.\n",
"\n",
"# let's visualize layer names and layer indices to see how many layers\n",
"# we should freeze:\n",
"names = []\n",
"for i, layer in enumerate(model.layers):\n",
" print([i, layer.name, layer.trainable])\n",
"\n",
"if use_vgg:\n",
" # we will freaze the first convolutional block and train all\n",
" # remaining blocks, including top layers.\n",
" for layer in model.layers[:4]:\n",
" layer.trainable = False\n",
" for layer in model.layers[4:]:\n",
" layer.trainable = True\n",
"else:\n",
" for layer in model.layers[:7]:\n",
" layer.trainable = False\n",
" for layer in model.layers[7:]:\n",
" layer.trainable = True\n",
"\n",
"# we need to recompile the model for these modifications to take effect\n",
"# we use SGD with a low learning rate\n",
"model.compile(optimizer=SGD(lr=0.0001, momentum=0.9),\n",
" loss='categorical_crossentropy',\n",
" metrics=['categorical_accuracy'])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 8. fit model (fine-tune base and top layers)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 6/10\n",
"100/100 [==============================] - 18s 181ms/step - loss: 0.1926 - categorical_accuracy: 0.9299 - val_loss: 0.1883 - val_categorical_accuracy: 0.9356\n",
"\n",
"Epoch 00006: val_categorical_accuracy improved from -inf to 0.93564, saving model to ../data/models/vgg_rgb_transfer_final.06-0.936.hdf5\n",
"Epoch 7/10\n",
"100/100 [==============================] - 18s 178ms/step - loss: 0.1697 - categorical_accuracy: 0.9409 - val_loss: 0.1771 - val_categorical_accuracy: 0.9399\n",
"\n",
"Epoch 00007: val_categorical_accuracy improved from 0.93564 to 0.93985, saving model to ../data/models/vgg_rgb_transfer_final.07-0.940.hdf5\n",
"Epoch 8/10\n",
"100/100 [==============================] - 18s 176ms/step - loss: 0.1614 - categorical_accuracy: 0.9436 - val_loss: 0.1674 - val_categorical_accuracy: 0.9430\n",
"\n",
"Epoch 00008: val_categorical_accuracy improved from 0.93985 to 0.94299, saving model to ../data/models/vgg_rgb_transfer_final.08-0.943.hdf5\n",
"Epoch 9/10\n",
"100/100 [==============================] - 17s 174ms/step - loss: 0.1489 - categorical_accuracy: 0.9515 - val_loss: 0.1619 - val_categorical_accuracy: 0.9466\n",
"\n",
"Epoch 00009: val_categorical_accuracy improved from 0.94299 to 0.94663, saving model to ../data/models/vgg_rgb_transfer_final.09-0.947.hdf5\n",
"Epoch 10/10\n",
"100/100 [==============================] - 17s 174ms/step - loss: 0.1279 - categorical_accuracy: 0.9544 - val_loss: 0.1482 - val_categorical_accuracy: 0.9483\n",
"\n",
"Epoch 00010: val_categorical_accuracy improved from 0.94663 to 0.94829, saving model to ../data/models/vgg_rgb_transfer_final.10-0.948.hdf5\n"
]
},
{
"data": {
"text/plain": [
"<keras.callbacks.History at 0x7fdff804c978>"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# generate callback to save best model w.r.t val_categorical_accuracy\n",
"if use_vgg:\n",
" file_name = \"vgg\"\n",
"else:\n",
" file_name = \"dense\"\n",
"checkpointer = ModelCheckpoint(\"../data/models/\" + file_name +\n",
" \"_rgb_transfer_final.\" +\n",
" \"{epoch:02d}-{val_categorical_accuracy:.3f}\" +\n",
" \".hdf5\",\n",
" monitor='val_categorical_accuracy',\n",
" verbose=1,\n",
" save_best_only=True,\n",
" mode='max')\n",
"earlystopper = EarlyStopping(monitor='val_categorical_accuracy',\n",
" patience=50,\n",
" mode='max')\n",
"model.fit(train_generator,\n",
" steps_per_epoch=100,\n",
" epochs=initial_epoch+5,\n",
" callbacks=[checkpointer, earlystopper, tensorboard],\n",
" validation_data=validation_generator,\n",
" validation_steps=500,\n",
" initial_epoch=initial_epoch)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
},
"varInspector": {
"cols": {
"lenName": 16,
"lenType": 16,
"lenVar": 40
},
"kernels_config": {
"python": {
"delete_cmd_postfix": "",
"delete_cmd_prefix": "del ",
"library": "var_list.py",
"varRefreshCmd": "print(var_dic_list())"
},
"r": {
"delete_cmd_postfix": ") ",
"delete_cmd_prefix": "rm(",
"library": "var_list.r",
"varRefreshCmd": "cat(var_dic_list()) "
}
},
"types_to_exclude": [
"module",
"function",
"builtin_function_or_method",
"instance",
"_Feature"
],
"window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 2
}
================================================
FILE: py/image_functions.py
================================================
# -*- coding: utf-8 -*-
import os
from random import choice, sample
import numpy as np
from skimage.io import imread
from skimage.transform import rotate
from tensorflow.keras.utils import to_categorical
def preprocessing_image_rgb(x):
# define mean and std values
mean = [87.845, 96.965, 103.947]
std = [23.657, 16.474, 13.793]
# loop over image channels
for idx, mean_value in enumerate(mean):
x[..., idx] -= mean_value
x[..., idx] /= std[idx]
return x
def preprocessing_image_ms(x):
# define mean and std values
mean = [1353.036, 1116.468, 1041.475, 945.344, 1198.498, 2004.878,
2376.699, 2303.738, 732.957, 12.092, 1818.820, 1116.271, 2602.579]
std = [65.479, 154.008, 187.997, 278.508, 228.122, 356.598, 456.035,
531.570, 98.947, 1.188, 378.993, 303.851, 503.181]
# loop over image channels
for idx, mean_value in enumerate(mean):
x[..., idx] -= mean_value
x[..., idx] /= std[idx]
return x
def categorical_label_from_full_file_name(files, class_indices):
# file basename without extension
base_name = [os.path.splitext(os.path.basename(i))[0] for i in files]
# class label from filename
base_name = [i.split("_")[0] for i in base_name]
# label to indices
image_class = [class_indices[i] for i in base_name]
# class indices to one-hot-label
return to_categorical(image_class, num_classes=len(class_indices))
def simple_image_generator(files, class_indices, batch_size=32,
rotation_range=0, horizontal_flip=False,
vertical_flip=False):
while True:
# select batch_size number of samples without replacement
batch_files = sample(files, batch_size)
# get one_hot_label
batch_Y = categorical_label_from_full_file_name(batch_files,
class_indices)
# array for images
batch_X = []
# loop over images of the current batch
for idx, input_path in enumerate(batch_files):
image = np.array(imread(input_path), dtype=float)
image = preprocessing_image_ms(image)
# process image
if horizontal_flip:
# randomly flip image up/down
if choice([True, False]):
image = np.flipud(image)
if vertical_flip:
# randomly flip image left/right
if choice([True, False]):
image = np.fliplr(image)
# rotate image by random angle between
# -rotation_range <= angle < rotation_range
if rotation_range is not 0:
angle = np.random.uniform(low=-abs(rotation_range),
high=abs(rotation_range))
image = rotate(image, angle, mode='reflect',
order=1, preserve_range=True)
# put all together
batch_X += [image]
# convert lists to np.array
X = np.array(batch_X)
Y = np.array(batch_Y)
yield(X, Y)
================================================
FILE: py/statistics.py
================================================
# -*- coding: utf-8 -*-
"""Some statistics about the EuroSAT dataset."""
import glob
import os
import numpy as np
from osgeo import gdal
def getMeanStd(path, n_bands=3, n_max=-1):
"""Get mean and standard deviation from images.
Parameters
----------
path : str
Path to training images
n_bands : int
Number of spectral bands (3 for RGB, 13 for Sentinel-2)
n_max : int
Maximum number of iterations (-1 = all)
Return
------
"""
if not os.path.isdir(path):
print("Error: Directory does not exist.")
return 0
mean_array = [[] for _ in range(n_bands)]
std_array = [[] for _ in range(n_bands)]
# iterate over the images
i = 0
for tif in glob.glob(path+"*/*.*"):
if (i < n_max) or (n_max == -1):
ds = gdal.Open(tif)
for band in range(n_bands):
mean_array[band].append(
np.mean(ds.GetRasterBand(band+1).ReadAsArray()))
std_array[band].append(
np.std(ds.GetRasterBand(band+1).ReadAsArray()))
i+=1
else:
break
# results
res_mean = [np.mean(mean_array[band]) for band in range(n_bands)]
res_std = [np.mean(std_array[band]) for band in range(n_bands)]
# print results table
print("Band | Mean | Std")
print("-"*28)
for band in range(n_bands):
print("{band:4d} | {mean:8.3f} | {std:8.3f}".format(
band=band, mean=res_mean[band], std=res_std[band]))
return res_mean, res_std
if __name__ == "__main__":
getMeanStd(path="data/PyCon/RGB/train/", n_bands=3)
================================================
FILE: requirements.txt
================================================
tensorflow>=2.1.0
skimage>=0.14.1
gdal==2.2.4
tqdm>=4.0.0
numpy>=1.17.4
gitextract_jvujk87r/ ├── .gitignore ├── LICENSE ├── README.md ├── bibliography.bib ├── py/ │ ├── 01_split_data_to_train_and_validation.py │ ├── 02_train_rgb_finetuning.py │ ├── 03_train_rgb_from_scratch.py │ ├── 04_train_ms_finetuning.py │ ├── 04_train_ms_finetuning_alternative.py │ ├── 05_train_ms_from_scratch.py │ ├── 06_classify_image.py │ ├── Image_functions.ipynb │ ├── Train_from_Scratch.ipynb │ ├── Transfer_learning.ipynb │ ├── image_functions.py │ └── statistics.py └── requirements.txt
SYMBOL INDEX (5 symbols across 2 files) FILE: py/image_functions.py function preprocessing_image_rgb (line 11) | def preprocessing_image_rgb(x): function preprocessing_image_ms (line 22) | def preprocessing_image_ms(x): function categorical_label_from_full_file_name (line 35) | def categorical_label_from_full_file_name(files, class_indices): function simple_image_generator (line 46) | def simple_image_generator(files, class_indices, batch_size=32, FILE: py/statistics.py function getMeanStd (line 12) | def getMeanStd(path, n_bands=3, n_max=-1):
Condensed preview — 17 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (95K chars).
[
{
"path": ".gitignore",
"chars": 66,
"preview": "*.DS_Store\n.vscode/\ndata/PyCon\npy/.ipynb_checkpoints/\n__*\npy/logs/"
},
{
"path": "LICENSE",
"chars": 1084,
"preview": "MIT License\n\nCopyright (c) 2018 Jens Leitloff & Felix Riese\n\nPermission is hereby granted, free of charge, to any person"
},
{
"path": "README.md",
"chars": 11951,
"preview": "[](https://doi.org/10.5281/zenodo.3268451)\n[![License: MI"
},
{
"path": "bibliography.bib",
"chars": 1294,
"preview": "@misc{leitloff2018examples,\n author = {Leitloff, Jens and Riese, Felix~M.},\n title = {{Examples for CNN training a"
},
{
"path": "py/01_split_data_to_train_and_validation.py",
"chars": 2682,
"preview": "# -*- coding: utf-8 -*-\n\"\"\"\nCode for the PyCon.DE 2018 talk by Jens Leitloff and Felix M. Riese.\n\nPyCon 2018 talk: Satel"
},
{
"path": "py/02_train_rgb_finetuning.py",
"chars": 6915,
"preview": "#!/usr/bin/env python3\n# -*- coding: utf-8 -*-\n\"\"\"\nCode for the PyCon.DE 2018 talk by Jens Leitloff and Felix M. Riese.\n"
},
{
"path": "py/03_train_rgb_from_scratch.py",
"chars": 4306,
"preview": "#!/usr/bin/env python3\n# -*- coding: utf-8 -*-\n\"\"\"\nCode for the PyCon.DE 2018 talk by Jens Leitloff and Felix M. Riese.\n"
},
{
"path": "py/04_train_ms_finetuning.py",
"chars": 7633,
"preview": "#!/usr/bin/env python3\n# -*- coding: utf-8 -*-\n\"\"\"\nCode for the PyCon.DE 2018 talk by Jens Leitloff and Felix M. Riese.\n"
},
{
"path": "py/04_train_ms_finetuning_alternative.py",
"chars": 7619,
"preview": "#!/usr/bin/env python3\n# -*- coding: utf-8 -*-\n\"\"\"\nCode for the PyCon.DE 2018 talk by Jens Leitloff and Felix M. Riese.\n"
},
{
"path": "py/05_train_ms_from_scratch.py",
"chars": 4119,
"preview": "#!/usr/bin/env python3\n# -*- coding: utf-8 -*-\n\"\"\"\nCode for the PyCon.DE 2018 talk by Jens Leitloff and Felix M. Riese.\n"
},
{
"path": "py/06_classify_image.py",
"chars": 3866,
"preview": "#!/usr/bin/env python3\n# -*- coding: utf-8 -*-\n\"\"\"\nCode for the PyCon.DE 2018 talk by Jens Leitloff and Felix M. Riese.\n"
},
{
"path": "py/Image_functions.ipynb",
"chars": 6137,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Notebook for image_functions.py\"\n"
},
{
"path": "py/Train_from_Scratch.ipynb",
"chars": 8844,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Notebook for 05_train_ms_from_scr"
},
{
"path": "py/Transfer_learning.ipynb",
"chars": 17757,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Note for 02_train_rgb_finetuning."
},
{
"path": "py/image_functions.py",
"chars": 3142,
"preview": "# -*- coding: utf-8 -*-\nimport os\nfrom random import choice, sample\n\nimport numpy as np\nfrom skimage.io import imread\nfr"
},
{
"path": "py/statistics.py",
"chars": 1651,
"preview": "# -*- coding: utf-8 -*-\n\n\"\"\"Some statistics about the EuroSAT dataset.\"\"\"\n\nimport glob\nimport os\n\nimport numpy as np\nfro"
},
{
"path": "requirements.txt",
"chars": 72,
"preview": "tensorflow>=2.1.0\nskimage>=0.14.1\ngdal==2.2.4\ntqdm>=4.0.0\nnumpy>=1.17.4\n"
}
]
About this extraction
This page contains the full source code of the jensleitloff/CNN-Sentinel GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 17 files (87.0 KB), approximately 22.7k tokens, and a symbol index with 5 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.