Showing preview only (1,824K chars total). Download the full file or copy to clipboard to get everything.
Repository: ZFTurbo/MobileNet-in-FPGA
Branch: master
Commit: 24eb350e48c0
Files: 61
Total size: 1.7 MB
Directory structure:
gitextract_rr5i2nao/
├── README.md
├── a00_common_functions.py
├── a01_oid_utils.py
├── docs/
│ └── Writing_weights_to_memory_using_UART.md
├── r01_prepare_open_images_dataset.py
├── r02_train_mobilenet.py
├── r03_mobilenet_v1_reduce_and_scale_model.py
├── r03_remove_batchnorm_layers.py
├── r04_find_optimal_bit_for_weights.py
├── r05_gen_weights_in_verilog_format.py
├── r06_generate_debug_data.py
├── r07_generate_verilog_for_mobilenet.py
├── r08_generate_weights_file_for_FPGA.py
├── utils/
│ └── data_uart_to_fpga.py
└── verilog/
├── CAMERA/
│ ├── camera_controller.v
│ ├── cmos_i2c_ov5640/
│ │ ├── CMOS_Capture.v
│ │ ├── i2c_com.v
│ │ ├── ov5640_cfg.v
│ │ ├── power_on_delay.v
│ │ └── reg_config.v
│ ├── sdram_ov5640_vga.v
│ └── system_ctrl.v
├── GENERAL.qsf
├── GENERAL.qws
├── GENERAL.v
├── MobileNet_v3_conv_8_3x1/
│ ├── RAM.v
│ ├── RAMtoMEM.v
│ ├── TOP.v
│ ├── addressRAM.v
│ ├── border.v
│ ├── conv.v
│ ├── conv_TOP.v
│ ├── dense.v
│ └── result.v
├── OpenVino_MobileNet.qpf
├── RAM.v
├── Seg7.v
├── UART/
│ ├── async.v
│ └── serialGPIO.v
├── ili9341/
│ ├── tft_ili9341.sv
│ └── tft_ili9341_spi.sv
├── pll_24_100/
│ ├── pll_24_100_0002.qip
│ └── pll_24_100_0002.v
├── pll_24_100.bsf
├── pll_24_100.cmp
├── pll_24_100.ppf
├── pll_24_100.qip
├── pll_24_100.sip
├── pll_24_100.spd
├── pll_24_100.v
├── pll_24_100_sim/
│ ├── aldec/
│ │ └── rivierapro_setup.tcl
│ ├── cadence/
│ │ ├── cds.lib
│ │ ├── hdl.var
│ │ └── ncsim_setup.sh
│ ├── mentor/
│ │ └── msim_setup.tcl
│ ├── pll_24_100.vo
│ └── synopsys/
│ ├── vcs/
│ │ └── vcs_setup.sh
│ └── vcsmx/
│ ├── synopsys_sim.setup
│ └── vcsmx_setup.sh
├── pll_24_100_sim.f
└── scale_picture.v
================================================
FILE CONTENTS
================================================
================================================
FILE: README.md
================================================
# MobileNet in FPGA
Generator of verilog description for FPGA MobileNet implementation.
There are several pre-trained models available for frequent tasks like detection of people, cars and animals.
You can train your own model easily on your dataset using code from this repository and have the same very fast detector on FPGA working in real time for your own task.
## Software requirements
Python 3.*, keras 2.2.4, tensorflow, kito
## Hardware requirements
1) TFT-screen ILI9341 Size: 2.8", Resolution: 240x320, Interface: SPI
2) Camera OV5640. Active array size: 2592 x 1944
3) OpenVINO Starter Kit. Cyclone V (301K LE, 13,917 Kbits embedded memory)
## Demo
[](https://www.youtube.com/watch?v=EQ9MJnWeHlo)
## How to run
1) `python3 r01_prepare_open_images_dataset.py` - it will create training files using [Open Images Dataset (OID)](https://storage.googleapis.com/openimages/web/index.html).
2) `python3 r02_train_mobilenet.py` - run training process. Will create weights for model and output accuracy of model.
3) `python3 r03_mobilenet_v1_reduce_and_scale_model.py` - batchnorm fusion and rescale model on range (0, 1) instead of (0, 6). Returns new rescaled model
Note: You can skip part 1, 2 and 3 if you use our pretrained weight files below
4) `python3 r04_find_optimal_bit_for_weights.py` - code to find optimal bit for feature maps, weights and biases, also returns maximum overflow for weights and biases over 1.0 value.
5) `python3 r05_gen_weights_in_verilog_format.py` - generate weights in verliog format using optimal bits from previous step
6) `python3 r06_generate_debug_data.py` - generate intermediate feature maps for each layer and details about first pixel calculation (can be used for debug)
7) `python3 r07_generate_verilog_for_mobilenet.py` - generate verilog based on given model and parameters like number of convolution blocks
## Updates
* **2019.10.04** We greatly improved speed of image reading and preprocessing. Now it takes only 5% of total time instead of 77% earlier. Speed for 8 convolution version of device increased from ~10 FPS up to ~ 40 FPS.
## Pre-trained models
| | People detector (128px) | Cars detector (128px) | Animals detector (128px) |
| --- | --- | --- | --- |
| Accuracy (%) | 84.42 | 96.31 | 89.67 |
| Init model (can be used for training and fine-tuning) | [people.h5](https://github.com/ZFTurbo/MobileNet-in-FPGA/releases/download/v1.0/weights_mobilenet_1_0.25_128px_people_loss_0.3600_acc_0.8442_epoch_38.h5) | [cars.h5](https://github.com/ZFTurbo/MobileNet-in-FPGA/releases/download/v1.0/weights_mobilenet_1_0.25_128px_cars_loss_0.1088_acc_0.9631_epoch_67.h5) | [animals.h5](https://github.com/ZFTurbo/MobileNet-in-FPGA/releases/download/v1.0/weights_mobilenet_1_0.25_128px_animals_loss_0.2486_acc_0.8967_epoch_33.h5) |
| Reduced and rescaled model | [people.h5](https://github.com/ZFTurbo/MobileNet-in-FPGA/releases/download/v1.0/weights_mobilenet_1_0.25_128px_people_loss_0.3600_acc_0.8442_epoch_38_reduced_rescaled.h5) | [cars.h5](https://github.com/ZFTurbo/MobileNet-in-FPGA/releases/download/v1.0/weights_mobilenet_1_0.25_128px_cars_loss_0.1088_acc_0.9631_epoch_67_reduced_rescaled.h5) | [animals.h5](https://github.com/ZFTurbo/MobileNet-in-FPGA/releases/download/v1.0/weights_mobilenet_1_0.25_128px_animals_loss_0.2486_acc_0.8967_epoch_33_reduced_rescaled.h5) |
| Optimal bits found | 12, 11, 10, 7, 3 | 10, 9, 8, 7, 3 | 12, 11, 10, 7, 3 |
| Quartus project (verilog) | [link](https://github.com/ZFTurbo/MobileNet-in-FPGA/releases/download/v2.0/OpenVino_MobileNet_verilog_project_people.zip) | [link](https://github.com/ZFTurbo/MobileNet-in-FPGA/releases/download/v2.0/OpenVino_MobileNet_verilog_project_cars.zip) | [link](https://github.com/ZFTurbo/MobileNet-in-FPGA/releases/download/v2.0/OpenVino_MobileNet_verilog_project_animals.zip) |
## Connection of peripherals
## Writing weights in memory
[See guide](https://github.com/ZFTurbo/MobileNet-in-FPGA/blob/master/docs/Writing_weights_to_memory_using_UART.md)
## Description of method
[Innovate FPGA](http://www.innovatefpga.com/cgi-bin/innovate/teams.pl?Id=EM031)
================================================
FILE: a00_common_functions.py
================================================
# Util functions
import pickle
import gzip
import cv2
import numpy as np
import pandas as pd
import os
import glob
import random
ROOT_PATH = os.path.dirname(os.path.realpath(__file__)) + '/'
MODEL_PATH = ROOT_PATH + 'models/'
if not os.path.isdir(MODEL_PATH):
os.mkdir(MODEL_PATH)
CACHE_PATH = ROOT_PATH + 'cache/'
if not os.path.isdir(CACHE_PATH):
os.mkdir(CACHE_PATH)
def save_in_file(arr, file_name):
pickle.dump(arr, gzip.open(file_name, 'wb+', compresslevel=3))
def load_from_file(file_name):
return pickle.load(gzip.open(file_name, 'rb'))
def show_image(im, name='image'):
cv2.imshow(name, im.astype(np.uint8))
cv2.waitKey(0)
cv2.destroyAllWindows()
def show_resized_image(P, w=1000, h=1000):
res = cv2.resize(P.astype(np.uint8), (w, h), interpolation=cv2.INTER_CUBIC)
show_image(res)
def relu_1(x):
from keras.activations import relu
return relu(x, max_value=1.0)
def save_history(history, path, columns=('loss', 'val_loss')):
import matplotlib.pyplot as plt
import pandas as pd
s = pd.DataFrame(history.history)
s.to_csv(path + '.csv')
plt.plot(s[list(columns)])
plt.savefig(path + '.png')
plt.close()
def get_model(weights_path):
from keras.models import load_model
print('Load: {}'.format(weights_path))
model = load_model(weights_path, custom_objects={'relu_1': relu_1})
print('Number of layers: {}'.format(len(model.layers)))
return model
def get_model_memory_usage(batch_size, model):
import numpy as np
from keras import backend as K
shapes_mem_count = 0
for l in model.layers:
single_layer_mem = 1
for s in l.output_shape:
if s is None:
continue
single_layer_mem *= s
shapes_mem_count += single_layer_mem
trainable_count = np.sum([K.count_params(p) for p in set(model.trainable_weights)])
non_trainable_count = np.sum([K.count_params(p) for p in set(model.non_trainable_weights)])
number_size = 4.0
if K.floatx() == 'float16':
number_size = 2.0
if K.floatx() == 'float64':
number_size = 8.0
total_memory = number_size*(batch_size*shapes_mem_count + trainable_count + non_trainable_count)
gbytes = np.round(total_memory / (1024.0 ** 3), 3)
return gbytes
================================================
FILE: a01_oid_utils.py
================================================
# coding: utf-8
__author__ = 'Roman Solovyev (ZFTurbo), IPPM RAS'
import platform
from PIL import Image
from a00_common_functions import *
# Paths and constants
if platform.processor() == 'Intel64 Family 6 Model 79 Stepping 1, GenuineIntel':
DATASET_PATH = 'E:/Projects_M2/2019_06_Google_Open_Images/input/'
else:
DATASET_PATH = 'E:/Projects_2TB/2019_06_Google_Open_Images/input/'
STORAGE_PATH_TRAIN = DATASET_PATH + 'train/'
STORAGE_PATH_TEST = DATASET_PATH + 'test/'
STORAGE_PATH_VALID = DATASET_PATH + 'validation/'
OID_CLASS_DESCRIPTION = DATASET_PATH + 'data_detection/challenge-2019-classes-description-500.csv'
OID_ANNOTATIONS_TRAIN = DATASET_PATH + 'data_detection/challenge-2019-train-detection-bbox.csv'
OID_ANNOTATIONS_VALID = DATASET_PATH + 'data_detection/challenge-2019-validation-detection-bbox.csv'
def get_model_memory_usage(batch_size, model):
import numpy as np
from keras import backend as K
shapes_mem_count = 0
for l in model.layers:
single_layer_mem = 1
for s in l.output_shape:
if s is None:
continue
single_layer_mem *= s
shapes_mem_count += single_layer_mem
trainable_count = np.sum([K.count_params(p) for p in set(model.trainable_weights)])
non_trainable_count = np.sum([K.count_params(p) for p in set(model.non_trainable_weights)])
number_size = 4.0
if K.floatx() == 'float16':
number_size = 2.0
if K.floatx() == 'float64':
number_size = 8.0
total_memory = number_size*(batch_size*shapes_mem_count + trainable_count + non_trainable_count)
gbytes = np.round(total_memory / (1024.0 ** 3), 3)
return gbytes
def get_description_for_labels():
out = open(OID_CLASS_DESCRIPTION)
lines = out.readlines()
ret_1, ret_2 = dict(), dict()
for l in lines:
arr = l.strip().split(',')
ret_1[arr[0]] = arr[1]
ret_2[arr[1]] = arr[0]
return ret_1, ret_2
def random_intensity_change(img, max_change):
img = img.astype(np.float32)
for j in range(3):
delta = random.randint(-max_change, max_change)
img[:, :, j] += delta
img[img < 0] = 0
img[img > 255] = 255
return img
def random_rotate(image, max_angle):
cols = image.shape[1]
rows = image.shape[0]
angle = random.uniform(-max_angle, max_angle)
M = cv2.getRotationMatrix2D((cols // 2, rows // 2), angle, 1)
dst = cv2.warpAffine(image, M, (cols, rows), borderMode=cv2.BORDER_REFLECT)
return dst
def read_single_image(path):
try:
img = np.array(Image.open(path))
except:
try:
img = cv2.cvtColor(cv2.imread(path), cv2.COLOR_BGR2RGB)
except:
print('Fail')
return None
if len(img.shape) == 2:
img = cv2.cvtColor(img, cv2.COLOR_GRAY2RGB)
if img.shape[2] == 2:
img = img[:, :, :1]
if img.shape[2] == 1:
img = np.concatenate((img, img, img), axis=2)
if img.shape[2] > 3:
img = img[:, :, :3]
return img
def read_image_bgr_fast(path):
img2 = cv2.imread(path)
return img2
def prepare_training_csv(type, true_labels_enc, output_path, side_size=128, min_class_size=5):
print('Go for: {} True labels: {}'.format(type, true_labels_enc))
if type == 'train':
boxes = pd.read_csv(OID_ANNOTATIONS_TRAIN)
else:
boxes = pd.read_csv(OID_ANNOTATIONS_VALID)
print('Initial boxes: {}'.format(len(boxes)))
image_ids = boxes['ImageID'].unique()
print('Unique images: {}'.format(len(image_ids)))
boxes_part = boxes[boxes['LabelName'].isin(true_labels_enc)]
print('Potential needed class boxes: {}'.format(len(boxes_part)))
print('Potential images with class: {}'.format(len(boxes_part['ImageID'].unique())))
images_with_needed_class = set()
for index, row in boxes_part.iterrows():
x1 = row['XMin']
x2 = row['XMax']
y1 = row['YMin']
y2 = row['YMax']
if (x2-x1)*side_size >= min_class_size and (y2-y1)*side_size >= min_class_size:
images_with_needed_class |= {row['ImageID']}
print('Images with class reduced: {}'.format(len(images_with_needed_class)))
no_class = list(set(image_ids) - set(images_with_needed_class))
print('Images without class: {}'.format(len(no_class)))
out = open(output_path, 'w')
out.write('id,target\n')
for id in sorted(list(images_with_needed_class)):
out.write(id + ',1\n')
for id in sorted(list(no_class)):
out.write(id + ',0\n')
out.close()
def check_validation_set(input_csv):
s = pd.read_csv(input_csv)
print('Go for true')
s_true = s[s['target'] == 1]
ids_true = list(s_true['id'].values)
for id in ids_true[:10]:
img = cv2.imread(STORAGE_PATH_VALID + id + '.jpg')
show_image(img)
print('Go for false')
s_true = s[s['target'] == 0]
ids_true = list(s_true['id'].values)
for id in ids_true[:10]:
img = cv2.imread(STORAGE_PATH_VALID + id + '.jpg')
show_image(img)
def check_train_set(input_csv):
s = pd.read_csv(input_csv)
print('Go for true')
s_true = s[s['target'] == 1]
ids_true = list(s_true['id'].values)
for id in ids_true[:10]:
img = cv2.imread(STORAGE_PATH_TRAIN + id[:3] + '/' + id + '.jpg')
show_image(img)
print('Go for false')
s_true = s[s['target'] == 0]
ids_true = list(s_true['id'].values)
for id in ids_true[:10]:
img = cv2.imread(STORAGE_PATH_TRAIN + id[:3] + '/' + id + '.jpg')
show_image(img)
def get_class_labels(true_labels):
d1, d2 = get_description_for_labels()
arr = []
for t in true_labels:
arr.append(d2[t])
print(arr)
return arr
================================================
FILE: docs/Writing_weights_to_memory_using_UART.md
================================================
The [UART](https://en.wikipedia.org/wiki/Universal_asynchronous_receiver-transmitter) port is used to write the neural net weights to the FPGA memory. And the weights are transferred directly from
the PC using the Python language. First you need to connect all the necessary wires to OpenVino. See picture below.
Next, turn on the device. Then launch Quartus Prime (we used version 18.0) and flash the entire project
using Programmer. Next, run a special Python script:
[data_uart_to_fpga.py](https://github.com/ZFTurbo/MobileNet-in-FPGA/blob/master/utils/data_uart_to_fpga.py).
Make sure that your folder with weights is next to the executable file and has the correct name.
Depending on what weights you need - for people, animals or cars - select the appropriate file.
It must be written in the code [WEIGHT_FILE_TO_USE](https://github.com/ZFTurbo/MobileNet-in-FPGA/blob/master/utils/data_uart_to_fpga.py#L4).
If everything is done correctly then progress will go.
Upon completion of loading weights, an image from the camera will appear on the screen and the
neural network will start to recognize it. The result will be displayed in the upper left corner
in red (there is an required object) or in green (the required object is missing).
================================================
FILE: r01_prepare_open_images_dataset.py
================================================
# coding: utf-8
__author__ = 'Roman Solovyev (ZFTurbo), IPPM RAS'
'''
Prepare dataset for different classes from Open Images Dataset (OID) from google
1) Class must be at least 5px size (for 128x128 image)
'''
if __name__ == '__main__':
import os
gpu_use = 0
os.environ["KERAS_BACKEND"] = "tensorflow"
os.environ["CUDA_VISIBLE_DEVICES"] = "{}".format(gpu_use)
from a01_oid_utils import *
# Definition for people
TRUE_LABELS_PEOPLE = ['Person', 'Man', 'Woman', 'Boy', 'Girl', 'Human body', 'Human eye', 'Skull', 'Human head', 'Human face',
'Human mouth', 'Human ear', 'Human nose', 'Human hair', 'Human hand', 'Human foot', 'Human arm', 'Human leg',
'Human beard']
TRUE_LABELS_PEOPLE_ENC = ['/m/01g317', '/m/04yx4', '/m/03bt1vf', '/m/01bl7v', '/m/05r655', '/m/02p0tk3',
'/m/014sv8', '/m/016m2d', '/m/04hgtk', '/m/0dzct', '/m/0283dt1', '/m/039xj_',
'/m/0k0pj', '/m/03q69', '/m/0k65p', '/m/031n1', '/m/0dzf4', '/m/035r7c', '/m/015h_t']
# Definition for cars
TRUE_LABELS_CAR = ['Car', 'Van', 'Taxi', 'Limousine', 'Truck', 'Bus', 'Ambulance']
TRUE_LABELS_CAR_ENC = ['/m/0k4j', '/m/0h2r6', '/m/0pg52', '/m/01lcw4', '/m/07r04', '/m/01bjv', '/m/012n7d']
# Definition for animals
TRUE_LABELS_ANIMAL = ['Animal', 'Bird', 'Woodpecker', 'Blue jay', 'Ostrich', 'Penguin', 'Raven', 'Chicken',
'Eagle', 'Owl', 'Duck', 'Canary', 'Goose', 'Swan', 'Falcon', 'Parrot', 'Sparrow',
'Turkey', 'Invertebrate', 'Tick', 'Centipede', 'Marine invertebrates', 'Starfish',
'Lobster', 'Jellyfish', 'Shrimp', 'Crab', 'Insect', 'Bee', 'Beetle', 'Ladybug',
'Ant', 'Moths and butterflies', 'Caterpillar', 'Butterfly', 'Dragonfly', 'Spider',
'Oyster', 'Snail', 'Bat', 'Carnivore', 'Bear', 'Brown bear', 'Polar bear', 'Cat',
'Fox', 'Jaguar', 'Lynx', 'Tiger', 'Lion', 'Dog', 'Leopard', 'Cheetah', 'Otter',
'Raccoon', 'Camel', 'Cattle', 'Giraffe', 'Rhinoceros', 'Goat', 'Horse', 'Hamster',
'Kangaroo', 'Mouse', 'Pig', 'Rabbit', 'Squirrel', 'Sheep', 'Zebra', 'Monkey', 'Deer',
'Elephant', 'Porcupine', 'Bull', 'Antelope', 'Mule', 'Marine mammal', 'Dolphin', 'Whale',
'Sea lion', 'Harbor seal', 'Alpaca', 'Reptile', 'Dinosaur', 'Lizard', 'Snake', 'Turtle',
'Tortoise', 'Sea turtle', 'Crocodile', 'Frog', 'Fish', 'Goldfish', 'Shark', 'Seahorse',
'Shellfish']
TRUE_LABELS_ANIMAL_ENC = ['/m/0jbk', '/m/015p6', '/m/01dy8n', '/m/01f8m5', '/m/05n4y', '/m/05z6w', '/m/06j2d',
'/m/09b5t', '/m/09csl', '/m/09d5_', '/m/09ddx', '/m/0ccs93', '/m/0dbvp', '/m/0dftk',
'/m/0f6wt', '/m/0gv1x', '/m/0h23m', '/m/0jly1', '/m/03xxp', '/m/0175cv', '/m/019h78',
'/m/03hl4l9', '/m/01h8tj', '/m/0cjq5', '/m/0d8zb', '/m/0ll1f78', '/m/0n28_', '/m/03vt0',
'/m/01h3n', '/m/020jm', '/m/0gj37', '/m/0_k2', '/m/0d_2m', '/m/0cydv', '/m/0cyf8',
'/m/0ft9s', '/m/09kmb', '/m/0_cp5', '/m/0f9_l', '/m/01h44', '/m/01lrl', '/m/01dws',
'/m/01dxs', '/m/0633h', '/m/01yrx', '/m/0306r', '/m/0449p', '/m/04g2r', '/m/07dm6',
'/m/096mb', '/m/0bt9lr', '/m/0c29q', '/m/0cd4d', '/m/0cn6p', '/m/0dq75', '/m/01x_v',
'/m/01xq0k1', '/m/03bk1', '/m/03d443', '/m/03fwl', '/m/03k3r', '/m/03qrc', '/m/04c0y',
'/m/04rmv', '/m/068zj', '/m/06mf6', '/m/071qp', '/m/07bgp', '/m/0898b', '/m/08pbxl',
'/m/09kx5', '/m/0bwd_0j', '/m/0c568', '/m/0cnyhnx', '/m/0czz2', '/m/0dbzx', '/m/0gd2v',
'/m/02hj4', '/m/084zz', '/m/0gd36', '/m/02l8p9', '/m/0pcr', '/m/06bt6', '/m/029tx',
'/m/04m9y', '/m/078jl', '/m/09dzg', '/m/011k07', '/m/0120dh', '/m/09f_2', '/m/09ld4',
'/m/0ch_cf', '/m/03fj2', '/m/0by6g', '/m/0nybt', '/m/0fbdv']
SIDE_SIZE = 128
MIN_CLASS_SIZE = 5
if __name__ == '__main__':
# Prepare people CSV
prepare_training_csv('validation', TRUE_LABELS_PEOPLE_ENC, CACHE_PATH + 'oid_validation_people.csv', SIDE_SIZE, MIN_CLASS_SIZE)
prepare_training_csv('train', TRUE_LABELS_PEOPLE_ENC, CACHE_PATH + 'oid_train_people.csv', SIDE_SIZE, MIN_CLASS_SIZE)
# Prepare cars CSV
prepare_training_csv('validation', TRUE_LABELS_CAR_ENC, CACHE_PATH + 'oid_validation_cars.csv', SIDE_SIZE, MIN_CLASS_SIZE)
prepare_training_csv('train', TRUE_LABELS_CAR_ENC, CACHE_PATH + 'oid_train_cars.csv', SIDE_SIZE, MIN_CLASS_SIZE)
# Prepare animals CSV
prepare_training_csv('validation', TRUE_LABELS_ANIMAL_ENC, CACHE_PATH + 'oid_validation_animals.csv', SIDE_SIZE, MIN_CLASS_SIZE)
prepare_training_csv('train', TRUE_LABELS_ANIMAL_ENC, CACHE_PATH + 'oid_train_animals.csv', SIDE_SIZE, MIN_CLASS_SIZE)
================================================
FILE: r02_train_mobilenet.py
================================================
# coding: utf-8
__author__ = 'Roman Solovyev (ZFTurbo), IPPM RAS'
# Train MobileNet with batch generator and augmentations
# Made for training with tensorflow only
import os
import glob
if __name__ == '__main__':
# Block to choose GPU
gpu_use = 2
print('GPU use: {}'.format(gpu_use))
os.environ["KERAS_BACKEND"] = "tensorflow"
os.environ["CUDA_VISIBLE_DEVICES"] = "{}".format(gpu_use)
import tensorflow as tf
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
session = tf.Session(config=config)
MOBILENET_VERSION = 1
MOBILENET_ALFA = 0.25
MOBILENET_INPUT_SIZE = 128
CHANNEL_TYPE = 'RGB'
from functools import partial
from keras import backend as K
from keras.optimizers import SGD, Adam
if MOBILENET_VERSION == 1:
from keras.applications.mobilenet import MobileNet, preprocess_input
else:
from keras.applications.mobilenet_v2 import MobileNetV2, preprocess_input
from keras.layers.core import Dense
from keras.models import Model
from a01_oid_utils import *
from a00_common_functions import *
from albumentations import *
import pandas as pd
from r01_prepare_open_images_dataset import DATASET_PATH
from multiprocessing.pool import ThreadPool
from multiprocessing import cpu_count
import random
def strong_aug(p=.5):
return Compose([
# RandomRotate90(),
HorizontalFlip(p=0.5),
# Transpose(),
OneOf([
IAAAdditiveGaussianNoise(),
GaussNoise(),
], p=0.1),
OneOf([
MotionBlur(p=.2),
MedianBlur(blur_limit=3, p=.1),
Blur(blur_limit=3, p=.1),
], p=0.1),
ShiftScaleRotate(shift_limit=0.0625, scale_limit=0.2, rotate_limit=10, p=0.1),
OneOf([
OpticalDistortion(p=0.3),
GridDistortion(p=0.1),
IAAPiecewiseAffine(p=0.3),
], p=0.1),
OneOf([
CLAHE(clip_limit=2),
IAASharpen(),
IAAEmboss(),
], p=0.3),
RGBShift(p=0.1, r_shift_limit=(-30, 30), g_shift_limit=(-30, 30), b_shift_limit=(-30, 30)),
RandomBrightnessContrast(p=0.05),
HueSaturationValue(p=0.05),
ToGray(p=0.05),
JpegCompression(p=0.05, quality_lower=55, quality_upper=99),
ElasticTransform(p=0.05),
], p=p)
GLOBAL_AUG = strong_aug(p=1.0)
def process_single_item(id, box_size, validation=True):
global global_aug
# Important: RGB order!
if validation is not True:
file_path = DATASET_PATH + 'train/' + id[:3] + '/' + id + '.jpg'
else:
file_path = DATASET_PATH + 'validation/' + id + '.jpg'
if CHANNEL_TYPE == 'RGB':
img = read_single_image(file_path)
else:
img = read_image_bgr_fast(file_path)
if img is None:
img = np.zeros((box_size, box_size, 3), dtype=np.uint8)
if validation is not True:
# img = GLOBAL_AUG(image=img)['image']
if 1:
img = random_intensity_change(img, 10)
img = random_rotate(img, 10)
if random.randint(0, 1) == 0:
# fliplr
img = img[:, ::-1, :]
if img.shape[0] != box_size:
img = cv2.resize(img, (box_size, box_size), interpolation=cv2.INTER_LINEAR)
return img
def batch_generator(X_train, Y_train, batch_size, input_size, prep_input, validation):
threads = cpu_count() - 1
p = ThreadPool(threads)
process_item_func = partial(process_single_item, validation=validation, box_size=input_size)
# Do around 50% of batch to have required class
X_train_no_class = X_train[Y_train[:, 1] == 0]
X_train_with_class = X_train[Y_train[:, 1] == 1]
Y_train_no_class = Y_train[Y_train[:, 1] == 0]
Y_train_with_class = Y_train[Y_train[:, 1] == 1]
print('Use threads: {}'.format(threads))
print(X_train_no_class.shape, X_train_with_class.shape, Y_train_no_class.shape, Y_train_with_class.shape)
b1 = batch_size // 2
b2 = batch_size - b1
while True:
batch_indexes_no_cars = np.random.choice(X_train_no_class.shape[0], b1)
batch_indexes_with_cars = np.random.choice(X_train_with_class.shape[0], b2)
batch_image_files = np.concatenate(
(X_train_no_class[batch_indexes_no_cars].copy(), X_train_with_class[batch_indexes_with_cars].copy())
)
batch_classes = np.concatenate(
(Y_train_no_class[batch_indexes_no_cars].copy(), Y_train_with_class[batch_indexes_with_cars].copy())
)
batch_images = p.map(process_item_func, batch_image_files)
batch_images = np.array(batch_images, np.float32)
batch_images = prep_input(batch_images)
yield batch_images, batch_classes
def evaluate_generator(X_test, Y_test, batch_size, input_size, prep_input):
number_of_batches = X_test.shape[0] // batch_size
target_size = input_size
i = 0
while 1:
batch_images = np.zeros((batch_size, target_size, target_size, 3))
if i >= number_of_batches:
print('Current {}'.format(i))
batch_image_files = X_test[-batch_size:]
batch_classes = Y_test[-batch_size:]
else:
batch_image_files = X_test[i*batch_size:(i+1)*batch_size]
batch_classes = Y_test[i*batch_size:(i+1)*batch_size]
# Rescale to 128x128
for j in range(batch_size):
path = DATASET_PATH + 'validation/' + batch_image_files[j] + '.jpg'
if CHANNEL_TYPE == 'RGB':
img = read_single_image(path)
else:
img = read_image_bgr_fast(path)
if img.shape[0] != target_size:
img = cv2.resize(img, (target_size, target_size), interpolation=cv2.INTER_LINEAR)
batch_images[j, :, :, :] = img
batch_images = prep_input(batch_images)
i += 1
yield batch_images, batch_classes
def load_train_valid_data(train_csv, valid_csv):
from keras.utils import to_categorical
valid = pd.read_csv(valid_csv)
train = pd.read_csv(train_csv)
X_train = train['id'].values
Y_train = to_categorical(train['target'].values, num_classes=2)
X_valid = valid['id'].values
Y_valid = to_categorical(valid['target'].values, num_classes=2)
return X_train, Y_train, X_valid, Y_valid
def train_mobile_net_v1(input_size, train_csv, valid_csv, type):
from keras.callbacks import EarlyStopping, ModelCheckpoint, CSVLogger, ReduceLROnPlateau
batch_size = 1024
nb_classes = 2
nb_epoch = 1000
patience = 50
optimizer = 'Adam'
learning_rate = 0.0001
restore = 1
print('Train MobileNet version: {} Input size: {}'.format(MOBILENET_VERSION, input_size))
print('Train for {} epochs with patience {}. Batch size: {}. Optimizer: {} Learing rate: {}'.
format(nb_epoch, patience, batch_size, optimizer, learning_rate))
X_train, Y_train, X_valid, Y_valid = load_train_valid_data(train_csv, valid_csv)
print('Train shape: {}'.format(X_train.shape))
print('Valid shape: {}'.format(X_valid.shape))
print('Dim ordering:', K.image_dim_ordering())
if MOBILENET_VERSION == 1:
alpha = MOBILENET_ALFA
base_model = MobileNet((input_size, input_size, 3), depth_multiplier=1, alpha=alpha,
include_top=False, pooling='avg', weights='imagenet')
else:
alpha = 0.35
base_model = MobileNetV2((input_size, input_size, 3), depth_multiplier=1, alpha=alpha,
include_top=False, pooling='avg', weights='imagenet')
x = base_model.output
x = Dense(nb_classes, activation='softmax', name='predictions', use_bias=False)(x)
model = Model(inputs=base_model.input, outputs=x)
print(model.summary())
if optimizer == 'SGD':
optim = SGD(lr=learning_rate, decay=1e-6, momentum=0.9, nesterov=True)
else:
optim = Adam(lr=learning_rate)
model.compile(optimizer=optim, loss='categorical_crossentropy', metrics=['accuracy'])
print('Model memory usage: {:.3f} GB'.format(get_model_memory_usage(batch_size, model)))
if not os.path.isdir('cache'):
os.mkdir('cache')
prefix = 'mobilenet_{}_{:.2f}_{}px_{}'.format(MOBILENET_VERSION, alpha, input_size, type)
cache_model_path = os.path.join(MODEL_PATH, 'weights_{}.h5'.format(prefix))
cache_model_path_score = MODEL_PATH + 'weights_{}_'.format(prefix) + 'loss_{val_loss:.4f}_acc_{val_acc:.4f}_epoch_{epoch:02d}_' + '{}.h5'.format(CHANNEL_TYPE)
if os.path.isfile(cache_model_path) and restore:
print('Restore weights from cache: {}'.format(cache_model_path))
model.load_weights(cache_model_path)
history_path = os.path.join(MODEL_PATH,
'weights_mobilenet_{}_{:.2f}_{}px_people_v2.csv'.format(MOBILENET_VERSION, alpha,
input_size))
callbacks = [
EarlyStopping(monitor='val_loss', patience=patience, verbose=0),
ModelCheckpoint(cache_model_path, monitor='val_loss', save_best_only=True, verbose=0),
ModelCheckpoint(cache_model_path_score, monitor='val_loss', save_best_only=False, verbose=0),
CSVLogger(MODEL_PATH + 'history_{}_lr_{}_optim_{}_v2.csv'.format(type, learning_rate, optimizer), append=True),
ReduceLROnPlateau(monitor='val_loss', factor=0.9, patience=5, min_lr=1e-9, min_delta=0.00001, verbose=1, mode='min'),
]
steps_per_epoch = 100
validation_steps = X_valid.shape[0] // batch_size
history = model.fit_generator(generator=batch_generator(X_train, Y_train, batch_size, input_size, preprocess_input, validation=False),
epochs=nb_epoch,
steps_per_epoch=steps_per_epoch,
validation_data=batch_generator(X_valid, Y_valid, batch_size, input_size, preprocess_input, validation=True),
validation_steps=validation_steps,
verbose=1,
max_queue_size=16,
initial_epoch=0,
callbacks=callbacks)
pd.DataFrame(history.history).to_csv(history_path, index=False)
score = model.evaluate_generator(generator=evaluate_generator(X_valid, Y_valid, batch_size, input_size, preprocess_input),
steps=X_valid.shape[0] // batch_size,
max_queue_size=1)
print('Full validation loss: {:.4f} Full validation accuracy: {:.4f} (For best model)'.format(score[0], score[1]))
print('Best model stored in {}'.format(cache_model_path))
return cache_model_path
def evaluate_model(model_path, input_size, train_csv, valid_csv):
from keras.models import load_model
print('Load model: {}'.format(model_path))
model = load_model(model_path)
batch_size = 1024
X_train, Y_train, X_valid, Y_valid = load_train_valid_data(train_csv, valid_csv)
print('Train shape: {}'.format(X_train.shape))
print('Valid shape: {}'.format(X_valid.shape))
score = model.evaluate_generator(
generator=evaluate_generator(X_valid, Y_valid, batch_size, input_size, preprocess_input),
steps=X_valid.shape[0] // batch_size,
max_queue_size=10, verbose=1)
print('Full validation loss: {:.4f} Full validation accuracy: {:.4f} (For best model)'.format(score[0], score[1]))
if __name__ == '__main__':
type = 'people'
# type = 'cars'
# type = 'animals'
train_csv = CACHE_PATH + 'oid_train_{}.csv'.format(type)
valid_csv = CACHE_PATH + 'oid_validation_{}.csv'.format(type)
# best_model_path = train_mobile_net_v1(MOBILENET_INPUT_SIZE, train_csv, valid_csv, type)
best_model_path = MODEL_PATH + 'best/weights_mobilenet_1_0.25_128px_people_loss_0.3600_acc_0.8442_epoch_38.h5'
evaluate_model(best_model_path, MOBILENET_INPUT_SIZE, train_csv, valid_csv)
'''
Animals MobileNet v1 (0.25, 128px):
Ep 1: 1563s 16s/step - loss: 0.3970 - acc: 0.8251 - val_loss: 0.4029 - val_acc: 0.8228
Ep 2: 1413s 14s/step - loss: 0.3499 - acc: 0.8453 - val_loss: 0.4436 - val_acc: 0.7914
Ep 3: 1435s 14s/step - loss: 0.3365 - acc: 0.8518 - val_loss: 0.3098 - val_acc: 0.8671
Ep 4: 1425s 14s/step - loss: 0.3279 - acc: 0.8563 - val_loss: 0.3208 - val_acc: 0.8587
Ep 5: 1441s 14s/step - loss: 0.3204 - acc: 0.8597 - val_loss: 0.3832 - val_acc: 0.8334
Ep 6: 1501s 15s/step - loss: 0.3198 - acc: 0.8610 - val_loss: 0.5252 - val_acc: 0.7404
Ep 7: 1506s 15s/step - loss: 0.3170 - acc: 0.8621 - val_loss: 0.3017 - val_acc: 0.8725
Ep 8: 1502s 15s/step - loss: 0.3124 - acc: 0.8643 - val_loss: 0.2960 - val_acc: 0.8740
Ep 14:1464s 15s/step - loss: 0.3020 - acc: 0.8687 - val_loss: 0.2765 - val_acc: 0.8811
Ep 17:1504s 15s/step - loss: 0.2951 - acc: 0.8712 - val_loss: 0.2786 - val_acc: 0.8853
Ep 18:1495s 15s/step - loss: 0.2944 - acc: 0.8733 - val_loss: 0.3067 - val_acc: 0.8671
Ep 19:1547s 15s/step - loss: 0.2923 - acc: 0.8738 - val_loss: 0.3065 - val_acc: 0.8663
Ep 33:1687s 17s/step - loss: 0.2748 - acc: 0.8826 - val_loss: 0.2486 - val_acc: 0.8967 - best
Ep 35:1750s 18s/step - loss: 0.2751 - acc: 0.8817 - val_loss: 0.2686 - val_acc: 0.8831
Ep 38:1789s 18s/step - loss: 0.2729 - acc: 0.8822 - val_loss: 0.2726 - val_acc: 0.8883
Ep 39:1814s 18s/step - loss: 0.2722 - acc: 0.8837 - val_loss: 0.2951 - val_acc: 0.8752
Full validation loss: 0.2788 Full validation accuracy: 0.8866 (For best model)
Cars MobileNet v1 (0.25, 128px):
Ep 1: 1566s 16s/step - loss: 0.3212 - acc: 0.8624 - val_loss: 0.1829 - val_acc: 0.9324
Ep 2: 1429s 14s/step - loss: 0.2557 - acc: 0.8907 - val_loss: 0.1463 - val_acc: 0.9472
Ep 3: 1564s 16s/step - loss: 0.2438 - acc: 0.8968 - val_loss: 0.1586 - val_acc: 0.9424
Ep 4: 1566s 16s/step - loss: 0.2360 - acc: 0.9003 - val_loss: 0.1453 - val_acc: 0.9473
Ep 17:1664s 17s/step - loss: 0.1947 - acc: 0.9208 - val_loss: 0.1188 - val_acc: 0.9586 LR: 0.00090
Ep 21:1726s 17s/step - loss: 0.1908 - acc: 0.9214 - val_loss: 0.1240 - val_acc: 0.9552
Ep 24:1739s 17s/step - loss: 0.1819 - acc: 0.9261 - val_loss: 0.1183 - val_acc: 0.9567 LR: 0.00081
Ep 25:1747s 17s/step - loss: 0.1841 - acc: 0.9246 - val_loss: 0.1406 - val_acc: 0.9500
Ep 47:2232s 22s/step - loss: 0.1657 - acc: 0.9336 - val_loss: 0.1188 - val_acc: 0.9597
Ep 48:1713s 17s/step - loss: 0.1720 - acc: 0.9299 - val_loss: 0.1183 - val_acc: 0.9556
Ep 49:1544s 15s/step - loss: 0.1662 - acc: 0.9321 - val_loss: 0.1265 - val_acc: 0.9564
Ep 55:1558s 16s/step - loss: 0.1659 - acc: 0.9334 - val_loss: 0.1134 - val_acc: 0.9613 LR: 0.00045
Ep 67:1584s 16s/step - loss: 0.1576 - acc: 0.9355 - val_loss: 0.1088 - val_acc: 0.9631 LR: 0.0003645 - Best
Ep 72:1625s 16s/step - loss: 0.1528 - acc: 0.9393 - val_loss: 0.1273 - val_acc: 0.9594 LR: 0.00032805
Full validation loss: 0.0993 Full validation accuracy: 0.9662 (For best model)
People MobileNet v1 (0.25, 128px):
Ep 1: 1716s 17s/step - loss: 0.4718 - acc: 0.7887 - val_loss: 0.7406 - val_acc: 0.7069
Ep 2: 1550s 15s/step - loss: 0.3771 - acc: 0.8368 - val_loss: 0.5902 - val_acc: 0.7573
Ep 20:1625s 16s/step - loss: 0.3022 - acc: 0.8749 - val_loss: 0.3756 - val_acc: 0.8370
Ep 25:1710s 17s/step - loss: 0.2953 - acc: 0.8794 - val_loss: 0.3769 - val_acc: 0.8374
Ep 30:1726s 17s/step - loss: 0.2953 - acc: 0.8775 - val_loss: 0.3741 - val_acc: 0.8411
Ep 32:1748s 17s/step - loss: 0.2911 - acc: 0.8813 - val_loss: 0.3645 - val_acc: 0.8442
Ep 38:1818s 18s/step - loss: 0.2844 - acc: 0.8844 - val_loss: 0.3600 - val_acc: 0.8442 - best
Ep 41:1871s 19s/step - loss: 0.2891 - acc: 0.8810 - val_loss: 0.3963 - val_acc: 0.8322 LR: 8.100000122794882e-05
Ep 43:1895s 19s/step - loss: 0.2851 - acc: 0.8838 - val_loss: 0.3851 - val_acc: 0.8361 LR: 7.289999848580919e-05
Ep 52:1999s 20s/step - loss: 0.2805 - acc: 0.8852 - val_loss: 0.3790 - val_acc: 0.8433
Ep 68:2276s 23s/step - loss: 0.2795 - acc: 0.8862 - val_loss: 0.4114 - val_acc: 0.8298 LR: 4.304672074795235e-05
Full validation loss: 0.3053 Full validation accuracy: 0.8739 (For best model)
'''
================================================
FILE: r03_mobilenet_v1_reduce_and_scale_model.py
================================================
# coding: utf-8
__author__ = 'Roman Solovyev (ZFTurbo), IPPM RAS'
'''
Code to find reduction coefficients for fixed point representation of weights.
It run some images from validation part of dataset to find maximum ranges of values.
Then convert RELU6 -> RELU1 and rescale some weights and biases.
At the end code checks that initial and rescaled models gives totally same result.
'''
import os
import glob
from a01_oid_utils import read_single_image, DATASET_PATH
if __name__ == '__main__':
# Block to choose backend
gpu_use = 4
os.environ["KERAS_BACKEND"] = "tensorflow"
os.environ["CUDA_VISIBLE_DEVICES"] = "{}".format(gpu_use)
print('GPU use: {}'.format(gpu_use))
from keras import backend as K
from a00_common_functions import *
# Coefficient to make safe gap for found range to prevent overflow. Lower - less safe, higher - more rounding error.
GAP_COEFF = 1.0
def preproc_input_mathmodel(x):
x /= 127.5
x -= 1.
return x
def rescale_weights(model, layer_num, coeff):
w = model.layers[layer_num].get_weights()
model.layers[layer_num].set_weights(w / coeff)
return model
def rescale_weights_with_bias(model, layer_num, coeff, current_scale):
w, b = model.layers[layer_num].get_weights()
w_new = w / coeff
b_new = b / (coeff * current_scale)
model.layers[layer_num].set_weights((w_new, b_new))
return model
def rescale_only_bias(model, layer_num, coeff, current_scale):
w, b = model.layers[layer_num].get_weights()
w_new = w.copy()
b_new = b / (coeff * current_scale)
model.layers[layer_num].set_weights((w_new, b_new))
return model
def rescale_batch_norm_weights_initital_v1(model, layer_num, coeff, current_scale):
eps = 0.001
gamma, beta, run_mean, run_std = model.layers[layer_num].get_weights()
gamma /= (coeff * current_scale)
beta /= (coeff * current_scale)
run_mean /= (coeff * current_scale)
# Из за квадратного корня и EPS тут не всё так просто. Надо пересчитывать по формуле
# после решения уравнения sqrt(Mx+e) = sqrt(x+e)/K
# M = 1/(K*K) + e*(1-K*K)/(K*K*x)
c2 = (coeff*coeff*current_scale*current_scale)
# print('Run std: {}'.format(run_std))
run_std = run_std/c2 + eps*(1-c2)/c2
model.layers[layer_num].set_weights((gamma, beta, run_mean, run_std))
return model
def rescale_batch_norm_weights_initital(model, layer_num, coeff, current_scale):
gamma, beta, run_mean, run_std = model.layers[layer_num].get_weights()
beta /= (coeff * current_scale)
run_mean /= current_scale
gamma /= coeff
model.layers[layer_num].set_weights((gamma, beta, run_mean, run_std))
return model
def rescale_dense_weights(model, layer_num, current_scale, coeff):
weights = model.layers[layer_num].get_weights()
if len(weights) == 2:
w, b = weights
w /= coeff
b /= (current_scale*coeff)
model.layers[layer_num].set_weights((w, b))
else:
w = weights
w /= coeff
model.layers[layer_num].set_weights(w)
return model
def is_next_relu6(model, layer_id):
if layer_id >= len(model.layers) - 1:
return False
layer = model.layers[layer_id + 1]
layer_type = layer.__class__.__name__
if layer_type == 'Activation':
config = layer.get_config()
activation = config['activation']
if activation == 'relu6':
return True
return False
def replace_intermediate_layer_in_keras(model, layer_id, new_layer):
from keras.models import Model
layers = [l for l in model.layers]
x = layers[0].output
for i in range(1, len(layers)):
if i == layer_id:
x = new_layer(x)
else:
x = layers[i](x)
new_model = Model(inputs=layers[0].input, outputs=x)
return new_model
def get_min_max_for_model(model, img_list):
from keras.models import Model
from keras.layers import ReLU
from keras.models import load_model
from keras.optimizers import Adam
reduction_koeffs = dict()
current_six_value = 1.0
current_scale = 6.0
eps = 0.001
first_rescale = True
for i in range(len(model.layers)):
class_name = model.layers[i].__class__.__name__
layer = model.layers[i]
print('Layer {}: {} Name {}'.format(i, class_name, layer.name))
print('In nodes: {}'.format(len(layer._inbound_nodes)))
w1 = layer.get_weights()
red_coeff = 1.0
if len(w1) > 0:
submodel = Model(inputs=model.inputs, outputs=layer.output)
print(submodel.summary())
# out = submodel.predict(img_list)
if class_name == 'Conv2D':
config = layer.get_config()
use_bias = config['use_bias']
print('Min weights value: {} Max weights value: {}'.format(w1[0].min(), w1[0].max()))
print('Min bias value: {} Max bias value: {}'.format(w1[1].min(), w1[1].max()))
if first_rescale is True:
model = rescale_weights_with_bias(model, i, 6.0, 1.0)
first_rescale = False
else:
model = rescale_only_bias(model, i, red_coeff, current_scale)
elif class_name == 'DepthwiseConv2D':
config = layer.get_config()
print(config)
use_bias = config['use_bias']
print('Min weights value: {} Max weights value: {}'.format(w1[0].min(), w1[0].max()))
print('Min bias value: {} Max bias value: {}'.format(w1[1].min(), w1[1].max()))
model = rescale_only_bias(model, i, red_coeff, current_scale)
elif class_name == 'Dense':
config = layer.get_config()
use_bias = config['use_bias']
print('Bias state: {}'.format(use_bias))
if use_bias == False:
print('We dont need to rescale Dense')
else:
print('Bias not supported yet!')
exit()
else:
continue
reduction_koeffs[i] = red_coeff
print('Layer: {} Scale: {} Reduction coeff: {} Six value: {}'.format(i, current_scale, red_coeff, current_six_value))
if class_name == 'Activation' or class_name == 'ReLU':
print(layer.get_config())
# Replace model with new activation
# model = replace_intermediate_layer_in_keras(model, i, Activation(lambda x: relu(x, max_value=current_six_value), name='custom_relu_{}'.format(i)))
print('Activation six value: {}'.format(current_six_value))
if abs(current_six_value - 1.0) > 0.0000001:
print('Not expected six value!')
exit()
# We always add relu_1 activation (due to scaling algorithm)
model = replace_intermediate_layer_in_keras(model, i, ReLU(max_value=1.0, name='custom_relu_{}'.format(i)))
model.compile(optimizer=Adam(), loss='categorical_crossentropy', metrics=['accuracy'])
model.save(MODEL_PATH + 'debug.h5')
model = load_model(MODEL_PATH + 'debug.h5')
print(model.summary)
if i == 0:
continue
# Check new min, max
layer = model.layers[i]
o = layer.output
submodel = Model(inputs=model.inputs, outputs=o)
print(submodel.summary())
out = submodel.predict(img_list)
print('Rescaled submodel: {} Min out value: {} Max out value: {}'.format(out.shape, out.min(), out.max()))
print('Reduction koeffs: ', reduction_koeffs)
return model, reduction_koeffs
def load_oid_data(type):
from keras.utils import to_categorical
valid = pd.read_csv(CACHE_PATH + 'oid_validation_{}.csv'.format(type))
train = pd.read_csv(CACHE_PATH + 'oid_train_{}.csv'.format(type))
X_train = train['id'].values
Y_train = to_categorical(train['target'].values, num_classes=2)
X_valid = valid['id'].values
Y_valid = to_categorical(valid['target'].values, num_classes=2)
return X_train, Y_train, X_valid, Y_valid
def process_single_item(id, box_size):
img = read_single_image(DATASET_PATH + 'validation/' + id + '.jpg')
img = cv2.resize(img, (box_size, box_size), interpolation=cv2.INTER_LINEAR)
return img
def check_results_are_the_same(model_path1, model_path2, img_list):
from keras.models import load_model
modelA = load_model(model_path1)
modelB = load_model(model_path2)
resA = modelA.predict(img_list)
resB = modelB.predict(img_list)
print(resA)
print(resB)
print('Probabilities shape: {}'.format(resA.shape))
maxA = resA.argmax(axis=1)
maxB = resB.argmax(axis=1)
print(maxA)
print(maxB)
print('Answer shape: {}'.format(maxA.shape))
print(np.unique(maxA, return_counts=True))
print(np.unique(maxB, return_counts=True))
diff = len(maxA[maxA != maxB])
print('Answer difference: {}'.format(diff))
print((maxA - maxB).sum())
if __name__ == '__main__':
from kito import reduce_keras_model
from keras.models import load_model
from keras.applications.mobilenet import preprocess_input
# Params
image_limit = 10000
input_size = 128
model_type = 'animals'
model_path = MODEL_PATH + 'best/weights_mobilenet_1_0.25_128px_animals_loss_0.2486_acc_0.8967_epoch_33.h5'
model_path_reduced = model_path[:-3] + '_reduced.h5'
model_path_rescaled = model_path[:-3] + '_reduced_rescaled.h5'
if not os.path.isfile(model_path_reduced):
model = load_model(model_path)
model = reduce_keras_model(model, verbose=True)
model.save(model_path_reduced)
else:
model = load_model(model_path_reduced)
print(model.summary())
print('Number of layers: {}'.format(len(model.layers)))
X_train, Y_train, X_test, Y_test = load_oid_data(model_type)
print(X_train.shape, X_test.shape)
X_test = X_test[:image_limit]
Y_test = Y_test[:image_limit]
uni = np.unique(Y_test, return_counts=True)
print(uni[0].sum())
img_list = []
for i in range(len(X_test)):
img = process_single_item(X_test[i], input_size)
img_list.append(img)
img_list = np.array(img_list, dtype=np.float32)
img_list = preprocess_input(img_list)
print("Image limit: {} Images shape: {}".format(image_limit, img_list.shape))
model, reduction_koeffs = get_min_max_for_model(model, img_list)
overall_reduction_rate = 1.0
for i in sorted(reduction_koeffs.keys()):
print('Layer {} reduction coeff: {}'.format(i, reduction_koeffs[i]))
overall_reduction_rate *= reduction_koeffs[i]
print('Overall scale change: {}'.format(overall_reduction_rate))
print('Save model in {}'.format(model_path_rescaled))
model.save(model_path_rescaled)
check_results_are_the_same(model_path, model_path_rescaled, img_list)
================================================
FILE: r03_remove_batchnorm_layers.py
================================================
# coding: utf-8
__author__ = 'Roman Solovyev (ZFTurbo), IPPM RAS'
# Remove layers which is not needed for inference using KITO script
import os
import glob
if __name__ == '__main__':
# Block to choose GPU
gpu_use = 4
print('GPU use: {}'.format(gpu_use))
os.environ["KERAS_BACKEND"] = "tensorflow"
os.environ["CUDA_VISIBLE_DEVICES"] = "{}".format(gpu_use)
from kito import reduce_keras_model
from keras.models import load_model
from a00_common_functions import *
if __name__ == '__main__':
model_path_in = MODEL_PATH + 'best/weights_mobilenet_1_0.25_128px_animals_loss_0.2486_acc_0.8967_epoch_33.h5'
model_path_out = MODEL_PATH + 'best/weights_mobilenet_1_0.25_128px_animals_loss_0.2486_acc_0.8967_epoch_33_bnfused.h5'
model = load_model(model_path_in, custom_objects={'relu_1': relu_1})
model_reduced = reduce_keras_model(model, verbose=True)
print(model_reduced.summary())
print('Initial layers: {}'.format(len(model.layers)))
print('Reduced layers: {}'.format(len(model_reduced.layers)))
model_reduced.save(model_path_out)
'''
MobileNet V1 (Keras 2.2.4)
Initial layers: 89
Reduced layers: 62
'''
================================================
FILE: r04_find_optimal_bit_for_weights.py
================================================
# coding: utf-8
__author__ = 'Roman Solovyev (ZFTurbo), IPPM RAS'
'''
This code finds out which bit size for weight lead to zero classification error on fixed point test data
comparing with floating point test data. Start search from 8 bits up to 32 bits.
'''
if __name__ == '__main__':
import os
# Block to choose backend
gpu_use = 2
os.environ["KERAS_BACKEND"] = "tensorflow"
os.environ["CUDA_VISIBLE_DEVICES"] = "{}".format(gpu_use)
from a00_common_functions import *
from scipy.signal import convolve2d
import math
import tensorflow as tf
sess = tf.Session()
sess.run(tf.global_variables_initializer())
tf.logging.set_verbosity(tf.logging.ERROR)
# Note: We suppose that every Conv2D layer has type "same"
# In Tensorflow weight matrices already transposed
def my_convolve(input, kernel):
output = np.zeros((input.shape[0], input.shape[1]))
zero_pad = np.zeros((input.shape[0] + 2, input.shape[1] + 2))
zero_pad[1:-1, 1:-1] = input
# kernel = np.flipud(kernel)
# kernel = np.fliplr(kernel)
for i in range(1, zero_pad.shape[0] - 1):
for j in range(1, zero_pad.shape[1] - 1):
sub = zero_pad[i-1:i+2, j-1:j+2]
output[i-1, j-1] = np.sum(sub*kernel)
return output
def my_convolve_fixed_point(input, kernel, bit):
output = np.zeros((input.shape[0], input.shape[1]))
zero_pad = np.zeros((input.shape[0] + 2, input.shape[1] + 2))
zero_pad[1:-1, 1:-1] = input
# kernel = np.flipud(kernel)
# kernel = np.fliplr(kernel)
for i in range(1, zero_pad.shape[0] - 1):
for j in range(1, zero_pad.shape[1] - 1):
sub = zero_pad[i-1:i+2, j-1:j+2]
output[i-1, j-1] = np.sum((sub*kernel).astype(np.int64))
return output
def preprocess_forward(arr, val):
arr1 = arr.copy().astype(np.float32)
arr1 /= val
return arr1
def convert_to_fix_point(arr1, bit):
arr2 = arr1.copy().astype(np.float32)
arr2[arr2 < 0] = 0.0
arr2 = np.round(np.abs(arr2) * (2 ** bit))
arr3 = arr1.copy().astype(np.float32)
arr3[arr3 > 0] = 0.0
arr3 = -np.round(np.abs(-arr3) * (2 ** bit))
arr4 = arr2 + arr3
return arr4.astype(np.int64)
def from_fix_point_to_float(arr, bit):
return arr / (2 ** bit)
def compare_outputs(s1, s2, debug_info=True):
if s1.shape != s2.shape:
print('Shape of arrays is different! {} != {}'.format(s1.shape, s2.shape))
s = np.abs(s1 - s2)
size = 1
for dim in np.shape(s): size *= dim
if debug_info:
print('Max difference: {}'.format(s.max()))
print('Avg difference: {}'.format(s.mean()))
print('Value range float: {} - {}'.format(s1.min(), s1.max()))
print('Value range fixed: {} - {}'.format(s2.min(), s2.max()))
def print_first_pixel_detailed_calculation_dense(previous_layer_output, wgt_bit, bit_precizion):
i = 10
conv_my = 0
for j in range(0, previous_layer_output.shape[0]):
print('Pixel {}: {}'.format(j, int(previous_layer_output[j])))
print('Weight {}: {}'.format(j, wgt_bit[j][i]))
conv_my += np.right_shift((previous_layer_output[j]*wgt_bit[j][i]).astype(np.int64), bit_precizion)
if j > 0 and j % 9 == 8:
print('Current conv_my: {}'.format(conv_my))
print('Result first pixel: {}'.format(conv_my))
exit()
def print_first_pixel_detailed_calculation(previous_layer_output, wgt_bit, bit_precizion):
i = 0
x = 0
y = 0
conv_my = 0
print('Point: {} X: {} Y: {}'.format(i, x, y))
print('Weights shape: {}'.format(wgt_bit.shape))
for j in range(wgt_bit.shape[2]):
full_image = previous_layer_output[:, :, j]
zero_pad = np.zeros((full_image.shape[0] + 2, full_image.shape[1] + 2))
zero_pad[1:-1, 1:-1] = full_image
pics = zero_pad[x+1-1:x+1+2, y+1-1:y+1+2].astype(np.int64)
print('Pixel area 3x3 for [{}, {}]:'.format(x, y), pics)
kernel = wgt_bit[:, :, j, i].copy()
# Не надо переворачивать для TensorFlow
# kernel = np.flipud(kernel)
# kernel = np.fliplr(kernel)
print('Weights {}: {}'.format(j, kernel))
res = np.sum(np.right_shift((pics*kernel).astype(np.int64), bit_precizion))
print('Convolution result {}: {}'.format(j, res))
conv_my += res
print('Overall result: {}'.format(conv_my))
if conv_my[conv_my > 2 ** bit_precizion].any() or conv_my[conv_my < - 2 ** bit_precizion].any():
print('Overflow! {}'.format(conv_my[conv_my > 2 ** bit_precizion]))
exit()
if conv_my < 0:
conv_my = 0
exit()
def mmZeroPadding2D_floating_point(layer, img):
config = layer.get_config()
print(config)
if len(config['padding']) == 1:
padding1_start = config['padding']
padding1_end = config['padding']
padding2_start = config['padding']
padding2_end = config['padding']
elif len(config['padding']) == 2:
padding1_start = config['padding'][0][0]
padding1_end = config['padding'][0][1]
padding2_start = config['padding'][1][0]
padding2_end = config['padding'][1][1]
out = np.zeros((img.shape[0],
img.shape[1] + padding1_start + padding1_end,
img.shape[2] + padding2_start + padding2_end,
img.shape[3]), dtype=np.float64)
out[:, padding1_start:out.shape[1] - padding1_end, padding2_start:out.shape[2] - padding2_end, :] = img.copy()
return out
def mmZeroPadding2D_fixed_point(layer, img):
config = layer.get_config()
print(config)
if len(config['padding']) == 1:
padding1_start = config['padding']
padding1_end = config['padding']
padding2_start = config['padding']
padding2_end = config['padding']
elif len(config['padding']) == 2:
padding1_start = config['padding'][0][0]
padding1_end = config['padding'][0][1]
padding2_start = config['padding'][1][0]
padding2_end = config['padding'][1][1]
out = np.zeros((img.shape[0],
img.shape[1] + padding1_start + padding1_end,
img.shape[2] + padding2_start + padding2_end,
img.shape[3]), dtype=np.int64)
out[:, padding1_start:out.shape[1] - padding1_end, padding2_start:out.shape[2] - padding2_end, :] = img.copy()
return out
def run_TF_Conv2D(img, w, b, strides, padding, type='float'):
global sess
in1 = tf.Variable(img.astype(np.float64))
w1 = tf.Variable(w.astype(np.float64))
b1 = tf.Variable(b.astype(np.float64))
data = tf.nn.conv2d(in1, w1, (1,) + strides + (1,), str(padding).upper())
data = tf.nn.bias_add(data, b1)
sess.run(tf.global_variables_initializer())
out = sess.run(data)
if type == 'float':
out = out.astype(np.float64)
else:
out = out.astype(np.int64)
tf.reset_default_graph()
sess = tf.Session()
return out
def run_TF_Depthwise_Conv2D(img, w, b, strides, padding, type='float'):
global sess
in1 = tf.Variable(img.astype(np.float64))
w1 = tf.Variable(w.astype(np.float64))
b1 = tf.Variable(b.astype(np.float64))
data = tf.nn.depthwise_conv2d(in1, w1, (1,) + strides + (1,), str(padding).upper())
data = tf.nn.bias_add(data, b1)
sess.run(tf.global_variables_initializer())
out = sess.run(data)
if type == 'float':
out = out.astype(np.float64)
else:
out = out.astype(np.int64)
tf.reset_default_graph()
sess = tf.Session()
return out
def mmConv2D_floating_point(layer, img, debug_info):
global sess
calc_type = 'tf'
config = layer.get_config()
filters = config['filters']
use_bias = config['use_bias']
strides = config['strides']
padding = config['padding']
kernel_size = config['kernel_size']
if debug_info and 0:
print(config)
sh1 = img.shape[1]
sh2 = img.shape[2]
if padding == 'valid':
sh1 -= 2 - (img.shape[1] % 2)
sh2 -= 2 - (img.shape[2] % 2)
if strides == (1, 1):
out = np.zeros((img.shape[0], sh1, sh2, filters), dtype=np.float64)
elif strides == (2, 2):
out = np.zeros((img.shape[0], sh1 // 2, sh2 // 2, filters), dtype=np.float64)
# calc_type = 'slow'
else:
print('Not supported conditions yet!')
exit()
if kernel_size != (3, 3) and kernel_size != (1, 1):
print('Unsupported kernel size: {}'.format(kernel_size))
exit()
(w, b) = layer.get_weights()
if debug_info:
print(w.shape, b.shape, out.shape)
if calc_type == 'slow':
# Cycle by different batch images
for sh0 in range(img.shape[0]):
# output filters cycle
for wi in range(w.shape[-1]):
# input filters cycle
for wj in range(w.shape[-2]):
kernel = w[:, :, wj, wi].copy()
slice = img[sh0, :, :, wj]
if padding == 'same':
zero_pad = np.zeros((slice.shape[0] + 2, slice.shape[1] + 2))
zero_pad[1:-1, 1:-1] = slice
elif padding == 'valid':
zero_pad = slice.copy()
else:
print('Unknown padding: {}'.format(padding))
exit()
# convolution
for i in range(1, zero_pad.shape[0] - 1, strides[0]):
for j in range(1, zero_pad.shape[1] - 1, strides[0]):
if kernel_size == (3, 3):
sub = zero_pad[i - 1:i + 2, j - 1:j + 2]
out[sh0, (i - 1) // strides[0], (j - 1) // strides[1], wi] += np.sum(sub * kernel)
elif kernel_size == (1, 1):
sub = zero_pad[i, j]
out[sh0, (i - 1) // strides[0], (j - 1) // strides[1], wi] += sub * kernel[0, 0]
out[sh0, :, :, wi] += b[wi]
elif calc_type == 'fast':
# Cycle by different batch images
for sh0 in range(img.shape[0]):
# output filters cycle
for wi in range(w.shape[-1]):
# input filters cycle
for wj in range(w.shape[-2]):
kernel = w[:, :, wj, wi].copy()
slice = img[sh0, :, :, wj].copy()
conv_my = convolve2d(slice, kernel, mode=padding)
out[sh0, :, :, wi] += conv_my
out[sh0, :, :, wi] += b[wi]
elif calc_type == 'tf':
out[...] = run_TF_Conv2D(img, w, b, strides, padding, 'float')
return out
def mmConv2D_fixed_point(layer, img, bit_precizion, bit_precizion_weights, bit_precizion_bias, debug_info):
global sess
calc_type = 'tf'
# Convolution with fixed point
config = layer.get_config()
filters = config['filters']
use_bias = config['use_bias']
strides = config['strides']
padding = config['padding']
kernel_size = config['kernel_size']
if debug_info and 0:
print(config)
sh1 = img.shape[1]
sh2 = img.shape[2]
if padding == 'valid':
sh1 -= 2 - (img.shape[1] % 2)
sh2 -= 2 - (img.shape[2] % 2)
if strides == (1, 1):
out = np.zeros((img.shape[0], sh1, sh2, filters), dtype=np.int64)
elif strides == (2, 2):
out = np.zeros((img.shape[0], sh1 // 2, sh2 // 2, filters), dtype=np.int64)
else:
print('Not supported conditions yet!')
exit()
if kernel_size != (3, 3) and kernel_size != (1, 1):
print('Unsupported kernel size: {}'.format(kernel_size))
exit()
(w, b) = layer.get_weights()
if debug_info:
print(w.shape, b.shape, out.shape)
w = convert_to_fix_point(w.copy(), bit_precizion_weights)
b = convert_to_fix_point(b.copy(), bit_precizion_bias)
# We need to shift it to sum with result of multiplication
b <<= bit_precizion_weights + (bit_precizion - bit_precizion_bias)
if calc_type == 'slow':
# Cycle by different batch images
for sh0 in range(img.shape[0]):
# output filters cycle
for wi in range(w.shape[-1]):
# input filters cycle
for wj in range(w.shape[-2]):
kernel = w[:, :, wj, wi].copy()
slice = img[sh0, :, :, wj]
if padding == 'same':
zero_pad = np.zeros((slice.shape[0] + 2, slice.shape[1] + 2))
zero_pad[1:-1, 1:-1] = slice
elif padding == 'valid':
zero_pad = slice.copy()
else:
print('Unknown padding: {}'.format(padding))
exit()
# convolution
for i in range(1, zero_pad.shape[0] - 1, strides[0]):
for j in range(1, zero_pad.shape[1] - 1, strides[0]):
if kernel_size == (3, 3):
sub = zero_pad[i - 1:i + 2, j - 1:j + 2]
out[sh0, (i - 1) // strides[0], (j - 1) // strides[1], wi] += np.sum((sub*kernel).astype(np.int64))
elif kernel_size == (1, 1):
sub = zero_pad[i, j]
out[sh0, (i - 1) // strides[0], (j - 1) // strides[1], wi] += (sub*kernel[0, 0]).astype(np.int64)
out[sh0, :, :, wi] += b[wi]
elif calc_type == 'fast':
# Cycle by different batch images
for sh0 in range(img.shape[0]):
# output filters cycle
for wi in range(w.shape[-1]):
# input filters cycle
for wj in range(w.shape[-2]):
kernel = w[:, :, wj, wi].copy()
slice = img[sh0, :, :, wj].copy()
conv_my = convolve2d(slice, kernel, mode=padding)
out[sh0, :, :, wi] += conv_my
out[sh0, :, :, wi] += b[wi]
elif calc_type == 'tf':
out[...] = run_TF_Conv2D(img, w, b, strides, padding, 'int')
# Shift it back to initial scale
out = np.right_shift(out.astype(np.int64), bit_precizion_weights)
return out
def mmGlobalAveragePooling2D_floating_point(img):
# Standard glob pool
result = np.zeros((img.shape[0], img.shape[-1]))
for j in range(img.shape[0]):
for i in range(img.shape[-1]):
result[j, i] = img[j, :, :, i].mean()
return result
def mmGlobalAveragePooling2D_fixed_point(img):
# Standard glob pool
result = np.zeros((img.shape[0], img.shape[-1]), dtype=np.int64)
block_size = img.shape[1] * img.shape[2]
for j in range(img.shape[0]):
for i in range(img.shape[-1]):
value = img[j, :, :, i].sum() // block_size
result[j, i] = value
return result
def mmActivation_floating_point(layer, img, one_value=1.0, debug_info=False):
config = layer.get_config()
activation = config['activation']
if activation != 'relu_1':
print('Unsupported activation {}!'.format(activation))
exit()
result = img.copy()
result[result < 0] = 0.
result[result > one_value] = one_value
return result
def mmActivation_fixed_point(layer, img, bit_precizion, debug_info=False):
config = layer.get_config()
activation = config['activation']
if activation != 'relu_1':
print('Unsupported activation {}!'.format(activation))
exit()
result = img.copy()
result[result < 0] = 0.
result[result >= 2 ** bit_precizion] = 2 ** bit_precizion - 1
return result
def mmReLU_floating_point(layer, img, one_value=1.0, debug_info=False):
config = layer.get_config()
max_value = config['max_value']
if max_value != 1:
print('Unsupported value for ReLU activation {}!'.format(max_value))
exit()
result = img.copy()
result[result < 0] = 0.
result[result > one_value] = one_value
return result
def mmReLU_fixed_point(layer, img, bit_precizion, debug_info=False):
config = layer.get_config()
max_value = config['max_value']
if max_value != 1:
print('Unsupported value for ReLU activation {}!'.format(max_value))
exit()
result = img.copy()
result[result < 0] = 0.
result[result >= 2 ** bit_precizion] = 2 ** bit_precizion - 1
return result
def mmDepthwiseConv2D_floating_point(layer, img, debug_info):
config = layer.get_config()
calc_type = 'tf'
# print(config)
use_bias = config['use_bias']
strides = config['strides']
padding = config['padding']
kernel_size = config['kernel_size']
filters = img.shape[3]
sh1 = img.shape[1]
sh2 = img.shape[2]
if padding == 'valid':
sh1 -= 2 - (img.shape[1] % 2)
sh2 -= 2 - (img.shape[2] % 2)
if strides == (1, 1):
out = np.zeros((img.shape[0], sh1, sh2, filters), dtype=np.float64)
elif strides == (2, 2):
out = np.zeros((img.shape[0], sh1 // 2, sh2 // 2, filters), dtype=np.float64)
else:
print('Not supported strides yet: {}'.format(strides))
exit()
if kernel_size != (3, 3):
print('Unsupported kernel size: {}'.format(kernel_size))
exit()
(w, b) = layer.get_weights()
print(w.shape, b.shape, out.shape)
if calc_type == 'slow':
# Cycle by different batch images
for sh0 in range(img.shape[0]):
# input filters cycle
for wj in range(w.shape[-2]):
kernel = w[:, :, wj, 0].copy()
slice = img[sh0, :, :, wj]
if padding == 'same':
zero_pad = np.zeros((slice.shape[0] + 2, slice.shape[1] + 2))
zero_pad[1:-1, 1:-1] = slice
elif padding == 'valid':
zero_pad = slice.copy()
else:
print('Unknown padding: {}'.format(padding))
exit()
# kernel = np.flipud(kernel)
# kernel = np.fliplr(kernel)
# convolution
for i in range(1, zero_pad.shape[0] - 1, strides[0]):
for j in range(1, zero_pad.shape[1] - 1, strides[0]):
sub = zero_pad[i - 1:i + 2, j - 1:j + 2]
# print((i - 1) // strides[0], (j - 1) // strides[1], wi)
out[sh0, (i - 1) // strides[0], (j - 1) // strides[1], wj] = np.sum(sub * kernel)
out[sh0, :, :, wj] += b[wj]
elif calc_type == 'tf':
out[...] = run_TF_Depthwise_Conv2D(img, w, b, strides, padding, 'float')
return out
def mmDepthwiseConv2D_fixed_point(layer, img, bit_precizion, bit_precizion_weights, bit_precizion_bias, debug_info):
config = layer.get_config()
calc_type = 'tf'
# print(config)
use_bias = config['use_bias']
strides = config['strides']
padding = config['padding']
kernel_size = config['kernel_size']
filters = img.shape[3]
sh1 = img.shape[1]
sh2 = img.shape[2]
if padding == 'valid':
sh1 -= 2 - (img.shape[1] % 2)
sh2 -= 2 - (img.shape[2] % 2)
if strides == (1, 1):
out = np.zeros((img.shape[0], sh1, sh2, filters), dtype=np.float64)
elif strides == (2, 2):
out = np.zeros((img.shape[0], sh1 // 2, sh2 // 2, filters), dtype=np.float64)
else:
print('Not supported strides yet: {}'.format(strides))
exit()
if kernel_size != (3, 3):
print('Unsupported kernel size: {}'.format(kernel_size))
exit()
(w, b) = layer.get_weights()
print(w.shape, b.shape, out.shape)
w = convert_to_fix_point(w.copy(), bit_precizion_weights)
b = convert_to_fix_point(b.copy(), bit_precizion_bias)
# We need to shift it to sum with result of multiplication
b <<= bit_precizion_weights + (bit_precizion - bit_precizion_bias)
if calc_type == 'slow':
# Cycle by different batch images
for sh0 in range(img.shape[0]):
# input filters cycle
for wj in range(w.shape[-2]):
kernel = w[:, :, wj, 0].copy()
slice = img[sh0, :, :, wj]
if padding == 'same':
zero_pad = np.zeros((slice.shape[0] + 2, slice.shape[1] + 2))
zero_pad[1:-1, 1:-1] = slice
elif padding == 'valid':
zero_pad = slice.copy()
else:
print('Unknown padding: {}'.format(padding))
exit()
# kernel = np.flipud(kernel)
# kernel = np.fliplr(kernel)
# convolution
for i in range(1, zero_pad.shape[0] - 1, strides[0]):
for j in range(1, zero_pad.shape[1] - 1, strides[0]):
sub = zero_pad[i - 1:i + 2, j - 1:j + 2]
# print((i - 1) // strides[0], (j - 1) // strides[1], wi)
out[sh0, (i - 1) // strides[0], (j - 1) // strides[1], wj] = np.sum((sub*kernel).astype(np.int64))
out[sh0, :, :, wj] += b[wj]
elif calc_type == 'tf':
out[...] = run_TF_Depthwise_Conv2D(img, w, b, strides, padding, 'int')
# Shift it back to initial scale
out = np.right_shift(out.astype(np.int64), bit_precizion_weights)
return out
def mmDense_floating_point(layer, img, debug_info):
config = layer.get_config()
print(config)
use_bias = config['use_bias']
activation = config['activation']
units = config['units']
batch_size = img.shape[0]
if use_bias:
(w, b) = layer.get_weights()
else:
(w,) = layer.get_weights()
print('Dense weights shape: {}'.format(w.shape))
if activation != 'softmax':
print('Activation {} is not supported'.format(activation))
exit()
if use_bias is True:
print('Bias currently not supported!')
exit()
out = np.zeros((batch_size, units))
for sh0 in range(batch_size):
for i in range(w.shape[1]):
for j in range(w.shape[0]):
out[sh0, i] += img[sh0, j] * w[j, i]
# Softmax activation part
# We skip it here because we will use max at the end
if 0:
for sh0 in range(batch_size):
maxy = out[sh0].max()
out[sh0] = np.exp(out[sh0] - maxy)
sum = out[sh0].sum()
out[sh0] /= sum
return out
def mmDense_fixed_point(layer, img, bit_precizion, bit_precizion_weights, debug_info):
config = layer.get_config()
if debug_info is True:
print(config)
use_bias = config['use_bias']
activation = config['activation']
units = config['units']
batch_size = img.shape[0]
if use_bias:
(w, b) = layer.get_weights()
else:
(w,) = layer.get_weights()
if use_bias is True:
print('Bias currently not supported!')
exit()
if debug_info is True:
print('Dense weights shape: {}'.format(w.shape))
if activation != 'softmax':
print('Activation {} is not supported'.format(activation))
exit()
w = convert_to_fix_point(w.copy(), bit_precizion_weights)
out = np.zeros((batch_size, units))
for sh0 in range(batch_size):
for i in range(w.shape[1]):
for j in range(w.shape[0]):
out[sh0, i] += img[sh0, j] * w[j, i]
# Divide by 2^bp
out = np.right_shift(out.astype(np.int64), bit_precizion_weights)
if out[out > 2 ** bit_precizion].any() or out[out < - 2 ** bit_precizion].any():
if out[out > 2 ** bit_precizion].any():
print('Warning overflow on current level! {}'.format(out[out > 2 ** bit_precizion]))
else:
print('Warning overflow on current level! {}'.format(out[out < - 2 ** bit_precizion]))
print('Max is {}'.format(2 ** bit_precizion))
# We don't need to find softmax here, since we only need the
# position of max value, which will be the same
return out
# bit_precizion - fixed point accuracy in bits
def go_mat_model(model, images, bit_precizion, bit_precizion_weights, bit_precizion_bias, debug_info=True):
level_out = dict()
level_out_reduced = dict()
print_pixel_calc = False
# Hack before we solve problem with exact 1.0 value
one_value = (2 ** bit_precizion - 1) / (2 ** bit_precizion)
for level_id in range(len(model.layers)):
layer = model.layers[level_id]
layer_type = layer.__class__.__name__
if debug_info:
print('Layer name: {} Layer type: {}'.format(layer.name, layer_type))
if level_id > 0:
print('Input shape: {}'.format(level_out[level_id-1].shape))
if layer_type == 'InputLayer':
level_out[level_id] = images.copy()
level_out_reduced[level_id] = convert_to_fix_point(images.copy(), bit_precizion)
elif layer_type == 'ZeroPadding2D':
level_out[level_id] = mmZeroPadding2D_floating_point(layer, level_out[level_id - 1].copy())
level_out_reduced[level_id] = mmZeroPadding2D_fixed_point(layer, level_out_reduced[level_id - 1].copy())
elif layer_type == 'Conv2D':
level_out[level_id] = mmConv2D_floating_point(layer, level_out[level_id - 1].copy(), debug_info)
level_out_reduced[level_id] = mmConv2D_fixed_point(layer, level_out_reduced[level_id - 1].copy(), bit_precizion, bit_precizion_weights, bit_precizion_bias, debug_info)
elif layer_type == 'DepthwiseConv2D':
level_out[level_id] = mmDepthwiseConv2D_floating_point(layer, level_out[level_id - 1].copy(), debug_info)
level_out_reduced[level_id] = mmDepthwiseConv2D_fixed_point(layer, level_out_reduced[level_id - 1].copy(), bit_precizion, bit_precizion_weights, bit_precizion_bias, debug_info)
elif layer_type == 'Activation':
level_out[level_id] = mmActivation_floating_point(layer, level_out[level_id - 1].copy(), one_value=one_value, debug_info=debug_info)
level_out_reduced[level_id] = mmActivation_fixed_point(layer, level_out_reduced[level_id - 1].copy(), bit_precizion, debug_info)
elif layer_type == 'ReLU':
level_out[level_id] = mmReLU_floating_point(layer, level_out[level_id - 1].copy(), one_value=one_value, debug_info=debug_info)
level_out_reduced[level_id] = mmReLU_fixed_point(layer, level_out_reduced[level_id - 1].copy(), bit_precizion, debug_info)
elif layer_type == 'GlobalAveragePooling2D':
level_out[level_id] = mmGlobalAveragePooling2D_floating_point(level_out[level_id - 1].copy())
level_out_reduced[level_id] = mmGlobalAveragePooling2D_fixed_point(level_out_reduced[level_id - 1].copy())
elif layer_type == 'Dense':
level_out[level_id] = mmDense_floating_point(layer, level_out[level_id - 1].copy(), debug_info)
level_out_reduced[level_id] = mmDense_fixed_point(layer, level_out_reduced[level_id - 1].copy(),
bit_precizion, bit_precizion_weights, debug_info)
# Convert back to float for comparison
checker_tmp = from_fix_point_to_float(level_out_reduced[level_id], bit_precizion)
compare_outputs(level_out[level_id], checker_tmp, debug_info)
if debug_info:
print('')
if level_id > 1000:
exit()
if layer.name == 'conv_dw_2_bn_':
exit()
print(level_out[len(model.layers) - 1].shape)
print(level_out[len(model.layers) - 1])
print(level_out_reduced[len(model.layers) - 1].shape)
print(level_out_reduced[len(model.layers) - 1])
pred_float = np.argmax(level_out[len(model.layers) - 1], axis=1)
pred_fixed = np.argmax(level_out_reduced[len(model.layers) - 1], axis=1)
error_rate = (pred_float != pred_fixed).sum() / level_out[len(model.layers) - 1].shape[0]
return error_rate, pred_float, pred_fixed
def get_error_rate(a1, a2):
miss = 0
for i in range(len(a1)):
if a1[i] != a2[i]:
miss += 1
print('Error rate: {}%'.format(round(100*miss/len(a1), 2)))
return miss
def preproc_input_mathmodel(x):
x -= 127.5
x /= 128.
return x
def load_oid_data_optimal(type):
valid = pd.read_csv(CACHE_PATH + 'oid_validation_{}.csv'.format(type))
X_valid = valid['id'].values
Y_valid = valid['target'].values
return X_valid, Y_valid
def get_image_set(type, image_limit, preproc_type='keras'):
from keras.applications.mobilenet import preprocess_input
from r03_mobilenet_v1_reduce_and_scale_model import process_single_item
from a01_oid_utils import read_single_image, DATASET_PATH
input_size = 128
X_test, Y_test = load_oid_data_optimal(type)
condition1 = (Y_test == 0)
print(X_test.shape, Y_test.shape)
X_test = np.concatenate((
X_test[condition1][:image_limit // 2],
X_test[~condition1][:image_limit // 2],
))
Y_test = np.concatenate((
Y_test[condition1][:image_limit // 2],
Y_test[~condition1][:image_limit // 2],
))
print(X_test.shape)
uni = np.unique(Y_test, return_counts=True)
print('Targets: {}'.format(uni))
img_list = []
for i in range(len(X_test)):
img = process_single_item(X_test[i], input_size)
img_list.append(img)
img_list = np.array(img_list, dtype=np.float32)
if preproc_type == 'keras':
img_list = preprocess_input(img_list)
else:
img_list = preproc_input_mathmodel(img_list)
print("Image limit: {} Images shape: {}".format(image_limit, img_list.shape))
return img_list, Y_test
def find_conv_overflow_bit_values(model):
max_w = -1000000000
max_b = -1000000000
for level_id in range(len(model.layers)):
layer = model.layers[level_id]
layer_type = layer.__class__.__name__
if layer_type == 'Conv2D' or layer_type == 'DepthwiseConv2D':
print('Go for layer: {}'.format(layer.name))
config = layer.get_config()
w, b = layer.get_weights()
print('Weights range: {} - {}'.format(w.min(), w.max()))
print('Bias range: {} - {}'.format(w.min(), w.max()))
if w.max() > max_w:
max_w = w.max()
if np.abs(w.min()) > max_w:
max_w = np.abs(w.min())
if b.max() > max_b:
max_b = b.max()
if np.abs(b.min()) > max_b:
max_b = np.abs(b.min())
print('Maximum weight in covolution overall: {}'.format(max_w))
print('Maximum bias in covolution overall: {}'.format(max_b))
max_w_bit = math.ceil(math.log(max_w, 2))
max_b_bit = math.ceil(math.log(max_b, 2))
print('Overflow for conv weights w: {} bits b: {} bits'.format(max_w_bit, max_b_bit))
return max_w_bit, max_b_bit
# This function works slow, so it should be run once to find optimal bit
def get_optimal_bit_for_weights(type, model_path, image_limit, acceptable_error_rate, use_cache):
cache_path = CACHE_PATH + 'optimal_bit_{}_{}.pklz'.format(type, image_limit)
if not os.path.isfile(cache_path) or use_cache is not True:
print('Read model...')
# We read already reduced weights. We don't need to fix them any way
model = get_model(model_path)
print(model.summary())
convW, convB = find_conv_overflow_bit_values(model)
# We doing preprocessing a little bit different because values shouldn't goes to -1 and 1 values (it will lead to overflow).
# It can reduce accuracy a little bit. Probably we should initially train with this preproc
images, answers = get_image_set(type, image_limit, 'math')
print('Classify images...')
keras_out = model.predict(images)
res_keras_array = []
acc = 0.
for i in range(keras_out.shape[0]):
res_keras_array.append(np.argmax(keras_out[i]))
if res_keras_array[-1] == answers[i]:
acc += 1.
print('Keras result raw: ', keras_out)
print('Keras result pos: ', res_keras_array)
print('Accuracy: {}'.format(acc / keras_out.shape[0]))
image_bit_precision = 8
weight_bit_precision = 16
bias_bit_precision = 16
if 1:
print('First run')
while 1:
print('\nStart image bit precision: {} Weights precision: {} Bias precision: {}'.format(image_bit_precision, weight_bit_precision, bias_bit_precision))
error_rate, pred_float, pred_fixed = go_mat_model(model, images, image_bit_precision, weight_bit_precision, bias_bit_precision, debug_info=True)
print('Error rate: {:.6f}'.format(error_rate))
print(res_keras_array)
print(pred_float)
print(pred_fixed)
image_bit_precision += 1
if error_rate < acceptable_error_rate or image_bit_precision > 36:
break
if image_bit_precision > 32:
return -1, -1, -1, -1, -1
print('Second run. Decrease weights bitsize')
while 1:
weight_bit_precision -= 1
# bias_bit_precision = image_bit_precision
print('\nStart image bit precision: {} Weights precision: {} Bias precision: {}'.format(image_bit_precision, weight_bit_precision, bias_bit_precision))
error_rate, pred_float, pred_fixed = go_mat_model(model, images, image_bit_precision, weight_bit_precision, bias_bit_precision, debug_info=True)
print('Error rate: {:.6f}'.format(error_rate))
print(res_keras_array)
print(pred_float)
print(pred_fixed)
if error_rate > acceptable_error_rate:
weight_bit_precision += 1
break
print('Third run. Decrease bias bitsize')
while 1:
bias_bit_precision -= 1
print('\nStart image bit precision: {} Weights precision: {} Bias precision: {}'.format(image_bit_precision,
weight_bit_precision,
bias_bit_precision))
error_rate, pred_float, pred_fixed = go_mat_model(model, images, image_bit_precision,
weight_bit_precision, bias_bit_precision,
debug_info=True)
print('Error rate: {:.6f}'.format(error_rate))
print(res_keras_array)
print(pred_float)
print(pred_fixed)
if error_rate > acceptable_error_rate:
bias_bit_precision += 1
break
if 0:
print('Single debug run')
print('\nStart error precision: {} Weights precision: {} Bias precision: {}'.format(image_bit_precision,
weight_bit_precision,
bias_bit_precision))
error_rate, pred_float, pred_fixed = go_mat_model(model, images, image_bit_precision,
weight_bit_precision, bias_bit_precision,
debug_info=True)
print('Error rate: {:.6f}'.format(error_rate))
print(res_keras_array)
print(pred_float)
print(pred_fixed)
save_in_file((image_bit_precision, weight_bit_precision, bias_bit_precision, convW, convB), cache_path)
return image_bit_precision, weight_bit_precision, bias_bit_precision, convW, convB
else:
return load_from_file(cache_path)
if __name__ == '__main__':
if 0:
use_cache = False
acceptable_error_rate = 0.005 # 0.5%
image_limit = 3000
type = 'people'
model_path_rescaled = MODEL_PATH + 'best/weights_mobilenet_1_0.25_128px_people_loss_0.3600_acc_0.8442_epoch_38_reduced_rescaled.h5'
if 0:
use_cache = False
acceptable_error_rate = 0.005 # 0.5%
image_limit = 3000
type = 'cars'
model_path_rescaled = MODEL_PATH + 'best/weights_mobilenet_1_0.25_128px_cars_loss_0.1088_acc_0.9631_epoch_67_reduced_rescaled.h5'
if 1:
use_cache = False
acceptable_error_rate = 0.005 # 0.5%
image_limit = 3000
type = 'animals'
model_path_rescaled = MODEL_PATH + 'best/weights_mobilenet_1_0.25_128px_animals_loss_0.2486_acc_0.8967_epoch_33_reduced_rescaled.h5'
image_bit_precision, weight_bit_precision, bias_bit_precision, convW, convB = get_optimal_bit_for_weights(type, model_path_rescaled, image_limit, acceptable_error_rate, use_cache)
if image_bit_precision > 0:
print('Optimal bit size for image and feature maps (sign bit is not included) is: {}'.format(image_bit_precision))
print('Optimal bit size for weights: {}'.format(weight_bit_precision))
print('Optimal bit size for bias: {}'.format(bias_bit_precision))
print('Bit overflows. Weights {} Bias: {}'.format(convW, convB))
else:
print('Impossible to find optimal bit!')
sess.close()
'''
Max error rate: 0.5%
weights_mobilenet_1_0.25_128px_people_loss_0.3600_acc_0.8442_epoch_38_reduced_rescaled.h5
Optimal 12, 11, 10, 7, 3
weights_mobilenet_1_0.25_128px_cars_loss_0.1088_acc_0.9631_epoch_67_reduced_rescaled.h5
Optimal 10, 9, 8, 7, 3
weights_mobilenet_1_0.25_128px_animals_loss_0.2486_acc_0.8967_epoch_33_reduced_rescaled.h5
Optimal 12, 11, 10, 7, 3
'''
================================================
FILE: r05_gen_weights_in_verilog_format.py
================================================
# coding: utf-8
__author__ = 'Roman Solovyev (ZFTurbo), IPPM RAS'
'''
Generate weights with optimal bit size in verilog format
'''
if __name__ == '__main__':
import os
# Block to choose backend
gpu_use = 4
os.environ["KERAS_BACKEND"] = "tensorflow"
os.environ["CUDA_VISIBLE_DEVICES"] = "{}".format(gpu_use)
from r04_find_optimal_bit_for_weights import *
STORAGE_COUNT_WEIGHTS = 0
STORAGE_COUNT_BIAS = 0
# Note: We suppose that every Conv2D layer has type "same"
# In Tensorflow weight matrices already transposed
def my_convolve(input, kernel):
output = np.zeros((input.shape[0], input.shape[1]))
zero_pad = np.zeros((input.shape[0] + 2, input.shape[1] + 2))
zero_pad[1:-1, 1:-1] = input
# kernel = np.flipud(kernel)
# kernel = np.fliplr(kernel)
for i in range(1, zero_pad.shape[0] - 1):
for j in range(1, zero_pad.shape[1] - 1):
sub = zero_pad[i-1:i+2, j-1:j+2]
output[i-1, j-1] = np.sum(sub*kernel)
return output
def my_convolve_fixed_point(input, kernel, bit):
output = np.zeros((input.shape[0], input.shape[1]))
zero_pad = np.zeros((input.shape[0] + 2, input.shape[1] + 2))
zero_pad[1:-1, 1:-1] = input
# kernel = np.flipud(kernel)
# kernel = np.fliplr(kernel)
for i in range(1, zero_pad.shape[0] - 1):
for j in range(1, zero_pad.shape[1] - 1):
sub = zero_pad[i-1:i+2, j-1:j+2]
output[i-1, j-1] = np.sum((sub*kernel).astype(np.int64))
return output
def preprocess_forward(arr, val):
arr1 = arr.copy().astype(np.float32)
arr1 /= val
return arr1
def convert_to_fix_point(arr1, bit):
arr2 = arr1.copy().astype(np.float32)
arr2[arr2 < 0] = 0.0
arr2 = np.round(np.abs(arr2) * (2 ** bit))
arr3 = arr1.copy().astype(np.float32)
arr3[arr3 > 0] = 0.0
arr3 = -np.round(np.abs(-arr3) * (2 ** bit))
arr4 = arr2 + arr3
return arr4.astype(np.int64)
def from_fix_point_to_float(arr, bit):
return arr / (2 ** bit)
def compare_outputs(s1, s2, debug_info=True):
if s1.shape != s2.shape:
print('Shape of arrays is different! {} != {}'.format(s1.shape, s2.shape))
s = np.abs(s1 - s2)
size = 1
for dim in np.shape(s): size *= dim
if debug_info:
print('Max difference: {}'.format(s.max()))
print('Avg difference: {}'.format(s.mean()/size))
print('Value range float: {} - {}'.format(s1.min(), s1.max()))
print('Value range fixed: {} - {}'.format(s2.min(), s2.max()))
def dump_memory_structure_conv(arr, out_file):
print('Dump memory structure in file: {}'.format(out_file))
out = open(out_file, "w")
total = 0
for a in range(arr.shape[2]):
for i in range(arr.shape[0]):
for j in range(arr.shape[1]):
out.write(str(total) + " LVL: {} X: {} Y: {} ".format(a, i, j) + str(arr[i, j, a]) + '\n')
total += 1
out.close()
def dump_memory_structure_dense(arr, out_file):
print('Dump memory structure for dense layer in file: {}'.format(out_file))
out = open(out_file, "w")
total = 0
print('Shape:', arr.shape)
for j in range(arr.shape[0]):
out.write(str(total) + " POS: {} ".format(j) + str(arr[j]) + '\n')
total += 1
out.close()
def print_first_pixel_detailed_calculation_dense(previous_layer_output, wgt_bit, bit_precizion):
i = 10
conv_my = 0
for j in range(0, previous_layer_output.shape[0]):
print('Pixel {}: {}'.format(j, int(previous_layer_output[j])))
print('Weight {}: {}'.format(j, wgt_bit[j][i]))
conv_my += np.right_shift((previous_layer_output[j]*wgt_bit[j][i]).astype(np.int64), bit_precizion)
if j > 0 and j % 9 == 8:
print('Current conv_my: {}'.format(conv_my))
print('Result first pixel: {}'.format(conv_my))
exit()
def print_first_pixel_detailed_calculation(previous_layer_output, wgt_bit, bit_precizion):
i = 0
x = 0
y = 0
conv_my = 0
print('Point: {} X: {} Y: {}'.format(i, x, y))
print('Weights shape: {}'.format(wgt_bit.shape))
for j in range(wgt_bit.shape[2]):
full_image = previous_layer_output[:, :, j]
zero_pad = np.zeros((full_image.shape[0] + 2, full_image.shape[1] + 2))
zero_pad[1:-1, 1:-1] = full_image
pics = zero_pad[x+1-1:x+1+2, y+1-1:y+1+2].astype(np.int64)
print('Pixel area 3x3 for [{}, {}]:'.format(x, y), pics)
kernel = wgt_bit[:, :, j, i].copy()
# Не надо переворачивать для TensorFlow
# kernel = np.flipud(kernel)
# kernel = np.fliplr(kernel)
print('Weights {}: {}'.format(j, kernel))
res = np.sum(np.right_shift((pics*kernel).astype(np.int64), bit_precizion))
print('Convolution result {}: {}'.format(j, res))
conv_my += res
print('Overall result: {}'.format(conv_my))
if conv_my[conv_my > 2 ** bit_precizion].any() or conv_my[conv_my < - 2 ** bit_precizion].any():
print('Overflow! {}'.format(conv_my[conv_my > 2 ** bit_precizion]))
exit()
if conv_my < 0:
conv_my = 0
exit()
def convert_to_normalized_form(value, precision, required_precision=None):
sign = 0
ret = value
if ret < 0:
sign = 1
ret = abs(ret)
# down = ret - math.floor(ret)
# print(ret, down)
normed = int(round(ret * 2**(precision-1)))
#if sign == 1 and normed != 0:
# Complement code for negative numbers
#normed = 2**(precision) - normed
down_binary_str = "{:0b}".format(normed)
if required_precision is None:
required_precision = precision
for j in range(len(down_binary_str), required_precision):
down_binary_str = '0' + down_binary_str
return sign, down_binary_str
def convert_to_normalized_form_array(value, precision):
ret = np.abs(value)
normed = np.round(ret * 2**(precision - 1)).astype(np.int64)
return normed
def convert_to_normalized_form_v2(value, precision):
sign = 0
ret = value
if ret < 0:
sign = 1
ret = abs(ret)
normed = ret
#if sign == 1 and normed != 0:
# Complement code for negative numbers
#normed = 2**(precision) - normed
down_binary_str = "{:b}".format(normed)
for j in range(len(down_binary_str), precision):
down_binary_str = '0' + down_binary_str
return sign, down_binary_str
def get_shape_string(w):
r = str(w.shape)[1:-1]
r = r.replace(',', '')
r = r.replace(' ', '_')
return r
def gen_convolution_weights(level_id, layer, bit_precizion, weight_bit_precision, bias_bit_precision, convW, convB, out_weights, out_bias):
global STORAGE_COUNT_WEIGHTS, STORAGE_COUNT_BIAS
# Convolution with fixed point
config = layer.get_config()
use_bias = config['use_bias']
kernel_size = config['kernel_size']
requred_mem_in_bits = 0
if kernel_size != (3, 3) and kernel_size != (1, 1):
print('Unsupported kernel size: {}'.format(kernel_size))
exit()
(w, b) = layer.get_weights()
# w = convert_to_fix_point(w.copy(), bit_precizion)
# Check that everything is fine with weights
if w[w > 2 ** convW].any() or w[w < -2 ** convW].any():
print('Overflow for conv weights!')
exit()
# Check that everything is fine with bias
if b[b > 2 ** convB].any() or b[b < -2 ** convB].any():
print('Overflow for conv bias!')
exit()
precisionW = weight_bit_precision + 1 + convW
precisionB = bias_bit_precision + 1 + convB
print('Initial bits weights: {} bias: {}'.format(precisionW, precisionB))
w_check = convert_to_normalized_form_array(w, weight_bit_precision + 1)
w_check_max = w_check.max()
precisionW = np.log2(w_check_max).astype(np.int64) + 1 + 1
b_check = convert_to_normalized_form_array(b, bias_bit_precision + 1)
b_check_max = b_check.max()
precisionB = np.log2(b_check_max).astype(np.int64) + 1 + 1
print('Max value to store weights: {} bias: {}'.format(w_check_max, b_check_max))
print('Reduced bits weights: {} bias: {}'.format(precisionW, precisionB))
print('Go for: {} Shape: {}'.format(layer.name, w.shape))
tp1 = 'bin'
s1 = '// Level: {:02d} Name: {} Type: {} BP Set: {} {} {} Shape: {}\n\n'.format(level_id, layer.name, layer.__class__.__name__,
bit_precizion + 1,
weight_bit_precision + 1 + convW,
bias_bit_precision + 1 + convB,
get_shape_string(w))
out_weights.write(s1)
out_bias.write(s1)
# Cycle by outputs
for i in range(w.shape[3]):
# Cycle by inputs
for j in range(w.shape[2]):
# Cycle by conv 3x3
for k in range(w.shape[1]):
for l in range(w.shape[0]):
sign, bin1 = convert_to_normalized_form(w[k, l, j, i].copy(), weight_bit_precision + 1, precisionW)
sgn = ' '
if sign == 1:
sgn = '-'
dec_verilog = int(bin1, 2)
if sign == 1:
dec_verilog = -dec_verilog
if tp1 == 'hex':
hx = hex(int(bin1, 2))[2:].upper()
out_weights.write(
"storage[{}] = {}{}'h{}; // {} {}\n".format(STORAGE_COUNT_WEIGHTS, sgn, precisionW, hx, dec_verilog, w[k, l, j, i]))
else:
out_weights.write(
"storage[{}] = {}{}'b{}; // {} {}\n".format(STORAGE_COUNT_WEIGHTS, sgn, precisionW, bin1, dec_verilog, w[k, l, j, i]))
requred_mem_in_bits += precisionW
STORAGE_COUNT_WEIGHTS += 1
if w.shape[1] > 1:
out_weights.write('\n')
out_weights.write('\n')
# Cycle by outputs
for i in range(w.shape[3]):
sign, bin1 = convert_to_normalized_form(b[i].copy(), bias_bit_precision + 1, precisionB)
sgn = ' '
if sign == 1:
sgn = '-'
dec_verilog = int(bin1, 2)
if sign == 1:
dec_verilog = -dec_verilog
if tp1 == 'hex':
hx = hex(int(bin1, 2))[2:].upper()
out_bias.write(
"storage_bias[{}] = {}{}'h{}; // {} {}\n".format(STORAGE_COUNT_BIAS, sgn, precisionB, hx, dec_verilog, b[i]))
else:
out_bias.write(
"storage_bias[{}] = {}{}'b{}; // {} {}\n".format(STORAGE_COUNT_BIAS, sgn, precisionB, bin1, dec_verilog, b[i]))
requred_mem_in_bits += precisionB
STORAGE_COUNT_BIAS += 1
out_bias.write('\n')
return requred_mem_in_bits
def gen_depthwise_convolution_weights(level_id, layer, bit_precizion, weight_bit_precision, bias_bit_precision, convW, convB, out_weights, out_bias):
global STORAGE_COUNT_WEIGHTS, STORAGE_COUNT_BIAS
config = layer.get_config()
use_bias = config['use_bias']
kernel_size = config['kernel_size']
requred_mem_in_bits = 0
if kernel_size != (3, 3) and kernel_size != (1, 1):
print('Unsupported kernel size: {}'.format(kernel_size))
exit()
(w, b) = layer.get_weights()
# Check that everything is fine with weights
if w[w > 2 ** convW].any() or w[w < -2 ** convW].any():
print('Overflow for conv weights!')
exit()
# Check that everything is fine with bias
if b[b > 2 ** convB].any() or b[b < -2 ** convB].any():
print('Overflow for conv bias!')
exit()
precisionW = weight_bit_precision + 1 + convW
precisionB = bias_bit_precision + 1 + convB
print('Initial bits weights: {} bias: {}'.format(precisionW, precisionB))
w_check = convert_to_normalized_form_array(w, weight_bit_precision + 1)
w_check_max = w_check.max()
precisionW = np.log2(w_check_max).astype(np.int64) + 1 + 1
b_check = convert_to_normalized_form_array(b, bias_bit_precision + 1)
b_check_max = b_check.max()
precisionB = np.log2(b_check_max).astype(np.int64) + 1 + 1
print('Max value to store weights: {} bias: {}'.format(w_check_max, b_check_max))
print('Reduced bits weights: {} bias: {}'.format(precisionW, precisionB))
print('Go for: {} Shape: {}'.format(layer.name, w.shape))
s1 = '// Level: {:02d} Name: {} Type: {} BP Set: {} {} {} Shape: {}\n\n'.format(level_id, layer.name, layer.__class__.__name__,
bit_precizion + 1,
weight_bit_precision + 1 + convW,
bias_bit_precision + 1 + convB,
get_shape_string(w))
out_weights.write(s1)
out_bias.write(s1)
tp1 = 'bin'
# Cycle by inputs. Output is always 1
for i in range(w.shape[2]):
# Cycle by conv 3x3
for k in range(w.shape[1]):
for l in range(w.shape[0]):
sign, bin1 = convert_to_normalized_form(w[k, l, i, 0].copy(), weight_bit_precision + 1, precisionW)
sgn = ' '
if sign == 1:
sgn = '-'
dec_verilog = int(bin1, 2)
if sign == 1:
dec_verilog = -dec_verilog
if tp1 == 'hex':
hx = hex(int(bin1, 2))[2:].upper()
out_weights.write(
"storage[{}] = {}{}'h{}; // {} {}\n".format(STORAGE_COUNT_WEIGHTS, sgn, precisionW, hx, dec_verilog, w[k, l, i, 0]))
else:
out_weights.write(
"storage[{}] = {}{}'b{}; // {} {}\n".format(STORAGE_COUNT_WEIGHTS, sgn, precisionW, bin1, dec_verilog, w[k, l, i, 0]))
requred_mem_in_bits += precisionW
STORAGE_COUNT_WEIGHTS += 1
out_weights.write('\n')
# Cycle by inputs. Output is always 1
for i in range(w.shape[2]):
sign, bin1 = convert_to_normalized_form(b[i].copy(), bias_bit_precision + 1, precisionB)
sgn = ' '
if sign == 1:
sgn = '-'
dec_verilog = int(bin1, 2)
if sign == 1:
dec_verilog = -dec_verilog
if tp1 == 'hex':
hx = hex(int(bin1, 2))[2:].upper()
out_bias.write(
"storage_bias[{}] = {}{}'h{}; // {} {}\n".format(STORAGE_COUNT_BIAS, sgn, precisionB, hx, dec_verilog,
w[k, l, i, 0]))
else:
out_bias.write(
"storage_bias[{}] = {}{}'b{}; // {} {}\n".format(STORAGE_COUNT_BIAS, sgn, precisionB, bin1, dec_verilog,
w[k, l, i, 0]))
requred_mem_in_bits += precisionB
STORAGE_COUNT_BIAS += 1
out_bias.write('\n')
return requred_mem_in_bits
def gen_dense_weights(level_id, layer, bit_precizion, out_weights):
global STORAGE_COUNT_WEIGHTS, STORAGE_COUNT_BIAS
config = layer.get_config()
use_bias = config['use_bias']
requred_mem_in_bits = 0
if use_bias:
print('Bias currently unsupported!')
exit()
(w,) = layer.get_weights()
# Check that everything is fine with weights
if w[w > 1].any() or w[w < -1].any():
print('Overflow for depthwise conv weights!')
exit()
print('Go for: {} Shape: {}'.format(layer.name, w.shape))
s1 = '// Level: {:02d} Name: {} Type: {} BP Set: {} Shape: {}\n\n'.format(level_id, layer.name, layer.__class__.__name__,
bit_precizion + 1,
get_shape_string(w))
out_weights.write(s1)
tp1 = 'bin'
precision = bit_precizion + 1
# Cycle by outputs
for i in range(w.shape[1]):
# Cycle by inputs
for j in range(w.shape[0]):
sign, bin1 = convert_to_normalized_form(w[j, i].copy(), precision)
sgn = ' '
if sign == 1:
sgn = '-'
dec_verilog = int(bin1, 2)
if sign == 1:
dec_verilog = -dec_verilog
if tp1 == 'hex':
hx = hex(int(bin1, 2))[2:].upper()
out_weights.write(
"storage[{}] = {}{}'h{}; // {} {}\n".format(STORAGE_COUNT_WEIGHTS, sgn, precision, hx, dec_verilog, w[j, i]))
else:
out_weights.write(
"storage[{}] = {}{}'b{}; // {} {}\n".format(STORAGE_COUNT_WEIGHTS, sgn, precision, bin1, dec_verilog, w[j, i]))
STORAGE_COUNT_WEIGHTS += 1
requred_mem_in_bits += precision
out_weights.write('\n')
return requred_mem_in_bits
def generate_weights_for_layers(model, bp, weight_bit_precision, bias_bit_precision, convW, convB, out_dir):
global STORAGE_COUNT_WEIGHTS, STORAGE_COUNT_BIAS
STORAGE_COUNT_WEIGHTS = 0
STORAGE_COUNT_BIAS = 0
weights_required_memory = 0
out_weights = open(out_dir + 'storage.v', 'w')
out_bias = open(out_dir + 'storage_bias.v', 'w')
for level_id in range(len(model.layers)):
layer = model.layers[level_id]
layer_type = layer.__class__.__name__
req_mem = 0
if layer_type == 'Conv2D':
req_mem = gen_convolution_weights(level_id, layer, bp, weight_bit_precision, bias_bit_precision, convW, convB, out_weights, out_bias)
elif layer_type == 'DepthwiseConv2D':
req_mem = gen_depthwise_convolution_weights(level_id, layer, bp, weight_bit_precision, bias_bit_precision, convW, convB, out_weights, out_bias)
elif layer_type == 'Dense':
req_mem = gen_dense_weights(level_id, layer, weight_bit_precision, out_weights)
else:
continue
print('Required weights memory: {} bit'.format(req_mem))
weights_required_memory += req_mem
out_weights.close()
out_bias.close()
print('Overall weights memory requirements: {} bit ({:.2f} MB)'.format(weights_required_memory, weights_required_memory / (1024*1024)))
if __name__ == '__main__':
type = 'animals'
model_path = MODEL_PATH + 'best/weights_mobilenet_1_0.25_128px_animals_loss_0.2486_acc_0.8967_epoch_33_reduced_rescaled.h5'
acceptable_error_rate = 0.005 # 0.5%
image_limit = 3000
if 0:
image_bit_precision, weight_bit_precision, bias_bit_precision, convW, convB = get_optimal_bit_for_weights(type, model_path, image_limit, acceptable_error_rate, use_cache=True)
else:
image_bit_precision, weight_bit_precision, bias_bit_precision, convW, convB = 12, 11, 10, 7, 3
out_dir = CACHE_PATH + type + '/'
if not os.path.isdir(out_dir):
os.mkdir(out_dir)
model = get_model(model_path)
generate_weights_for_layers(model, image_bit_precision, weight_bit_precision, bias_bit_precision, convW, convB, out_dir)
================================================
FILE: r06_generate_debug_data.py
================================================
# coding: utf-8
__author__ = 'Roman Solovyev (ZFTurbo), IPPM RAS'
'''
This code takes one image run it through the network and store all intermediate feature maps
in fixed point representation in separate files. Also detailed first pixel calculation is
generated. It used later to check generated verilog on correctness.
'''
if __name__ == '__main__':
import os
# Block to choose backend and GPU to run
gpu_use = 4
os.environ["KERAS_BACKEND"] = "tensorflow"
os.environ["CUDA_VISIBLE_DEVICES"] = "{}".format(gpu_use)
from r04_find_optimal_bit_for_weights import *
import math
def convert_to_normalized_form_v2(value, precision):
sign = 0
ret = value
if ret < 0:
sign = 1
ret = abs(ret)
normed = ret
#if sign == 1 and normed != 0:
# Complement code for negative numbers
#normed = 2**(precision) - normed
down_binary_str = "{:b}".format(normed)
for j in range(len(down_binary_str), precision):
down_binary_str = '0' + down_binary_str
return sign, down_binary_str
def store_layer_result(level_id, layer, layer_type, bp, res):
r = str(res.shape[1:])[1:-1]
r = r.replace(',', '')
r = r.replace(' ', '_')
out_file = INTERMEDIATE_OUTPUT_PATH + 'level_{:02d}_name_{}_bp_{}_shape_{}.txt'.format(level_id, layer.name, bp+1, r)
print('Write to {}'.format(out_file))
out = open(out_file, 'w')
if layer_type != 'Conv2D' and layer_type != 'DepthwiseConv2D':
if np.abs(res).max() >= 2 ** bp:
print('Some layer result problem here! ({} > {})'.format(np.abs(res).max(), 2 ** bp))
exit()
else:
if np.abs(res).max() >= 2 ** bp:
print('Overflow on {} layer! ({} > {}). It is expected, increase bit space!'.format(layer_type, np.abs(res).max(), 2 ** bp))
precision = bp + 1
# Possible overflow. It's fine
bit_max = math.ceil(math.log(np.abs(res).max() + 1, 2)) + 1
if bit_max > precision:
precision = bit_max
total = 0
if len(res.shape) == 4:
if 1:
# Start from channels
for i in range(res.shape[3]):
for j in range(res.shape[1]):
for k in range(res.shape[2]):
sign, bin1 = convert_to_normalized_form_v2(res[0, j, k, i].copy(), precision)
sgn = ' '
if sign == 1:
sgn = '-'
out.write("pixel[{}] = {}{}'b{}; // {}\n".format(total, sgn, precision, bin1, res[0, j, k, i]))
total += 1
out.write('\n')
elif len(res.shape) == 2:
for i in range(res.shape[1]):
sign, bin1 = convert_to_normalized_form_v2(res[0, i].copy(), precision)
sgn = ' '
if sign == 1:
sgn = '-'
out.write("pixel[{}] = {}{}'b{}; // {}\n".format(total, sgn, precision, bin1, res[0, i]))
total += 1
else:
print('Shape problem!')
exit()
out.close()
def print_convolution_detailed_first_pixel_calculation(level_id, layer, img, image_bit_precizion, weight_bit_precision, bias_bit_precision):
config = layer.get_config()
filters = config['filters']
use_bias = config['use_bias']
strides = config['strides']
padding = config['padding']
kernel_size = config['kernel_size']
sh1 = img.shape[1]
sh2 = img.shape[2]
if padding == 'valid':
sh1 -= 2
sh2 -= 2
if kernel_size != (3, 3) and kernel_size != (1, 1):
print('Unsupported kernel size: {}'.format(kernel_size))
exit()
(w, b) = layer.get_weights()
out_file = FIRST_PIXEL_OUTPUT_PATH + 'level_{:02d}_name_{}_bp_{}.txt'.format(level_id, layer.name, image_bit_precizion + 1)
print('Write to {}'.format(out_file))
out = open(out_file, 'w')
w = convert_to_fix_point(w.copy(), weight_bit_precision)
b = convert_to_fix_point(b.copy(), bias_bit_precision)
i = 0
x = 0
y = 0
# Output filter number
wi = i
# Batch image number
sh0 = 0
out.write('Point: {} X: {} Y: {}\n'.format(i, x, y))
# input filters cycle
value = 0
for wj in range(w.shape[-2]):
kernel = w[:, :, wj, wi].copy()
slice = img[sh0, :, :, wj]
if padding == 'same':
zero_pad = np.zeros((slice.shape[0] + 2, slice.shape[1] + 2))
zero_pad[1:-1, 1:-1] = slice
elif padding == 'valid':
zero_pad = slice.copy()
else:
print('Unknown padding: {}'.format(padding))
exit()
# Convolution for single output pixel
i = x*strides[0] + 1
j = y*strides[1] + 1
if kernel_size == (3, 3):
sub = zero_pad[i - 1:i + 2, j - 1:j + 2]
vv = np.sum((sub * kernel).astype(np.int64))
elif kernel_size == (1, 1):
sub = zero_pad[i, j]
vv = (sub * kernel[0, 0]).astype(np.int64)
out.write('Input kernel number: {}\n'.format(wj))
out.write('Kernel:\n{}\n'.format(kernel))
out.write('Part:\n{}\n'.format(sub))
out.write('Current result: {}\n\n'.format(vv))
value += vv
b[wi] <<= weight_bit_precision + (image_bit_precizion - bias_bit_precision)
out.write('Add bias: {}\n'.format(b[wi]))
value += b[wi]
out.write('Overall result before shift: {}\n'.format(value))
# Divide by 2^bp
value = np.right_shift(value, weight_bit_precision)
out.write('Overall result after shift: {}\n'.format(value))
out.close()
def print_depthwise_conv_detailed_first_pixel_calculation(level_id, layer, img, image_bit_precizion, weight_bit_precision, bias_bit_precision):
config = layer.get_config()
use_bias = config['use_bias']
strides = config['strides']
padding = config['padding']
kernel_size = config['kernel_size']
sh1 = img.shape[1]
sh2 = img.shape[2]
if padding == 'valid':
sh1 -= 2
sh2 -= 2
if kernel_size != (3, 3):
print('Unsupported kernel size: {}'.format(kernel_size))
exit()
(w, b) = layer.get_weights()
out_file = FIRST_PIXEL_OUTPUT_PATH + 'level_{:02d}_name_{}_bp_{}.txt'.format(level_id, layer.name, image_bit_precizion+1)
print('Write to {}'.format(out_file))
out = open(out_file, 'w')
w = convert_to_fix_point(w.copy(), weight_bit_precision)
b = convert_to_fix_point(b.copy(), bias_bit_precision)
i = 0
x = 0
y = 0
# Output filter number
wj = i
wi = 0
# Batch image number
sh0 = 0
out.write('Point: {} X: {} Y: {}\n'.format(i, x, y))
# input filters cycle
value = 0
kernel = w[:, :, wj, wi].copy()
slice = img[sh0, :, :, wj]
if padding == 'same':
zero_pad = np.zeros((slice.shape[0] + 2, slice.shape[1] + 2))
zero_pad[1:-1, 1:-1] = slice
elif padding == 'valid':
zero_pad = slice.copy()
else:
print('Unknown padding: {}'.format(padding))
exit()
# Convolution for single output pixel
i = x*strides[0] + 1
j = y*strides[1] + 1
if kernel_size == (3, 3):
sub = zero_pad[i - 1:i + 2, j - 1:j + 2]
vv = np.sum((sub * kernel).astype(np.int64))
elif kernel_size == (1, 1):
sub = zero_pad[i, j]
vv = (sub * kernel[0, 0]).astype(np.int64)
value = vv
out.write('Input kernel number: {}\n'.format(wj))
out.write('Kernel:\n{}\n'.format(kernel))
out.write('Part:\n{}\n'.format(sub))
b[wj] <<= weight_bit_precision + (image_bit_precizion - bias_bit_precision)
out.write('Add bias: {}\n'.format(b[wj]))
value += b[wj]
out.write('Overall result before shift: {}\n'.format(value))
# Divide by 2^bp
value = np.right_shift(value, weight_bit_precision)
out.write('Overall result after shift: {}\n'.format(value))
out.close()
def print_dense_detailed_first_pixel_calculation(level_id, layer, img, image_bit_precizion, weight_bit_precision):
config = layer.get_config()
use_bias = config['use_bias']
activation = config['activation']
if use_bias:
(w, b) = layer.get_weights()
else:
(w,) = layer.get_weights()
if use_bias is True:
print('Bias currently not supported!')
exit()
if activation != 'softmax':
print('Activation {} is not supported'.format(activation))
exit()
w = convert_to_fix_point(w.copy(), weight_bit_precision)
out_file = FIRST_PIXEL_OUTPUT_PATH + 'level_{:02d}_name_{}_bp_{}.txt'.format(level_id, layer.name,
weight_bit_precision + 1)
print('Write to {}'.format(out_file))
out = open(out_file, 'w')
i = 0
x = 0
# Batch image number
sh0 = 0
out.write('Point: {} X: {}\n'.format(i, x))
value = 0
for j in range(w.shape[0]):
out.write('Weight {}: {}\n'.format(j, w[j, i]))
vv = img[sh0, j] * w[j, i]
value += vv
out.write('Current intermediate result: {} [Accumulate: {}]\n'.format(vv, value))
out.write('Overall result before shift: {}\n'.format(value))
# Divide by 2^bp
value = np.right_shift(value.astype(np.int64), weight_bit_precision)
out.write('Overall result after shift: {}\n'.format(value))
out.close()
def get_filters_size(arr):
a = np.prod(np.array(arr.shape).astype(np.int64))
return a
def generate_layer_results(model, images, image_bit_precizion, weight_bit_precision, bias_bit_precision, convW, convB):
if images.shape[0] > 1:
print('Only one image must be in batch for debug!')
exit()
level_out_reduced = dict()
debug_info = False
prev_filters_space = -1
next_filters_space = -1
max_filter_space = -1
critical_layer = -1
for level_id in range(len(model.layers)):
layer = model.layers[level_id]
layer_type = layer.__class__.__name__
print('Layer num: {} Layer name: {} Layer type: {}'.format(level_id, layer.name, layer_type))
if level_id == 0:
next_filters_space = get_filters_size(images[0]) * (image_bit_precizion + 1)
if level_id > 0:
print('Input shape: {}'.format(level_out_reduced[level_id-1].shape))
prev_filters_space = next_filters_space.copy()
next_filters_space = get_filters_size(level_out_reduced[level_id-1][0]) * (image_bit_precizion + 1)
if prev_filters_space + next_filters_space > max_filter_space:
max_filter_space = prev_filters_space + next_filters_space
critical_layer = level_id
if layer_type == 'InputLayer':
level_out_reduced[level_id] = convert_to_fix_point(images.copy(), image_bit_precizion)
store_layer_result(level_id, layer, layer_type, image_bit_precizion, level_out_reduced[level_id])
elif layer_type == 'ZeroPadding2D':
level_out_reduced[level_id] = mmZeroPadding2D_fixed_point(layer, level_out_reduced[level_id - 1].copy())
elif layer_type == 'Conv2D':
level_out_reduced[level_id] = mmConv2D_fixed_point(layer, level_out_reduced[level_id - 1].copy(), image_bit_precizion, weight_bit_precision, bias_bit_precision, debug_info)
print_convolution_detailed_first_pixel_calculation(level_id, layer, level_out_reduced[level_id - 1].copy(), image_bit_precizion, weight_bit_precision, bias_bit_precision)
store_layer_result(level_id, layer, layer_type, image_bit_precizion, level_out_reduced[level_id])
elif layer_type == 'DepthwiseConv2D':
level_out_reduced[level_id] = mmDepthwiseConv2D_fixed_point(layer, level_out_reduced[level_id - 1].copy(), image_bit_precizion, weight_bit_precision, bias_bit_precision, debug_info)
print_depthwise_conv_detailed_first_pixel_calculation(level_id, layer, level_out_reduced[level_id - 1].copy(), image_bit_precizion, weight_bit_precision, bias_bit_precision)
store_layer_result(level_id, layer, layer_type, image_bit_precizion, level_out_reduced[level_id])
elif layer_type == 'Activation':
level_out_reduced[level_id] = mmActivation_fixed_point(layer, level_out_reduced[level_id - 1].copy(), image_bit_precizion, debug_info)
store_layer_result(level_id, layer, layer_type, image_bit_precizion, level_out_reduced[level_id])
elif layer_type == 'ReLU':
level_out_reduced[level_id] = mmReLU_fixed_point(layer, level_out_reduced[level_id - 1].copy(), image_bit_precizion, debug_info)
store_layer_result(level_id, layer, layer_type, image_bit_precizion, level_out_reduced[level_id])
elif layer_type == 'GlobalAveragePooling2D':
level_out_reduced[level_id] = mmGlobalAveragePooling2D_fixed_point(level_out_reduced[level_id - 1].copy())
store_layer_result(level_id, layer, layer_type, image_bit_precizion, level_out_reduced[level_id])
elif layer_type == 'Dense':
level_out_reduced[level_id] = mmDense_fixed_point(layer, level_out_reduced[level_id - 1].copy(), image_bit_precizion, weight_bit_precision, debug_info)
print_dense_detailed_first_pixel_calculation(level_id, layer, level_out_reduced[level_id - 1].copy(), image_bit_precizion, weight_bit_precision)
store_layer_result(level_id, layer, layer_type, image_bit_precizion, level_out_reduced[level_id])
print('Required space to store intermediate results of calculations: {} bits ({:.2f} MB)'.format(max_filter_space, max_filter_space / (1024 * 1024)))
print('Critical layer number: {}'.format(critical_layer))
def get_debug_image():
img = cv2.imread(CACHE_PATH + 'image.png')
img_list = []
img_list.append(img.copy())
img_list = np.array(img_list, dtype=np.float32)
img_list = preproc_input_mathmodel(img_list)
print(img_list.shape, img_list.max(), img_list.min())
return img_list
def generate_layer_results_for_image(type, model, image_bit_precision, weight_bit_precision, bias_bit_precision, convW, convB):
print(model.summary())
# Get only one image
try:
a = 10/0
# If OID dataset exists
images, answers = get_image_set(type, 2, 'math')
images = images[0:1]
print('Use OID images')
except:
images = np.zeros((1, 128, 128, 3), dtype=np.float32)
images[...] = 255
images = preproc_input_mathmodel(images)
print('No OID images found. Use generated image')
generate_layer_results(model, images, image_bit_precision, weight_bit_precision, bias_bit_precision, convW, convB)
if __name__ == '__main__':
problem_type = 'people'
INTERMEDIATE_OUTPUT_PATH = CACHE_PATH + 'intermediate_{}/'.format(problem_type)
if not os.path.isdir(INTERMEDIATE_OUTPUT_PATH):
os.mkdir(INTERMEDIATE_OUTPUT_PATH)
FIRST_PIXEL_OUTPUT_PATH = CACHE_PATH + 'first_pixel_{}/'.format(problem_type)
if not os.path.isdir(FIRST_PIXEL_OUTPUT_PATH):
os.mkdir(FIRST_PIXEL_OUTPUT_PATH)
if problem_type == 'people':
model = get_model(MODEL_PATH + 'best/weights_mobilenet_1_0.25_128px_people_loss_0.3600_acc_0.8442_epoch_38_reduced_rescaled.h5')
# bit_precision - without sign, so we need to add 1 to it to store sign as well
# image_bit_precision, weight_bit_precision, bias_bit_precision, convW, convB = get_optimal_bit_for_weights()
image_bit_precision, weight_bit_precision, bias_bit_precision, convW, convB = 12, 11, 10, 7, 3
elif problem_type == 'cars':
model = get_model(MODEL_PATH + 'best/weights_mobilenet_1_0.25_128px_cars_loss_0.1088_acc_0.9631_epoch_67_reduced_rescaled.h5')
# bit_precision - without sign, so we need to add 1 to it to store sign as well
# image_bit_precision, weight_bit_precision, bias_bit_precision, convW, convB = get_optimal_bit_for_weights()
image_bit_precision, weight_bit_precision, bias_bit_precision, convW, convB = 10, 9, 8, 7, 3
elif problem_type == 'animals':
model = get_model(MODEL_PATH + 'best/weights_mobilenet_1_0.25_128px_animals_loss_0.2486_acc_0.8967_epoch_33_reduced_rescaled.h5')
# bit_precision - without sign, so we need to add 1 to it to store sign as well
# image_bit_precision, weight_bit_precision, bias_bit_precision, convW, convB = get_optimal_bit_for_weights()
image_bit_precision, weight_bit_precision, bias_bit_precision, convW, convB = 12, 11, 10, 7, 3
generate_layer_results_for_image(problem_type, model, image_bit_precision, weight_bit_precision, bias_bit_precision, convW, convB)
================================================
FILE: r07_generate_verilog_for_mobilenet.py
================================================
# coding: utf-8
__author__ = 'Alex Kustov, IPPM RAS'
import os
gpu_use = 0
os.environ["KERAS_BACKEND"] = "tensorflow"
os.environ["CUDA_VISIBLE_DEVICES"] = "{}".format(gpu_use)
from r04_find_optimal_bit_for_weights import *
def addressRAM(directory, steps_count, max_address_value):
f = open(directory + "addressRAM.v", 'w')
bit_steps_count = len(bin(steps_count)) - 2
bit_max_address_value = len(bin(max_address_value)) - 2
f.write("module addressRAM(\n")
f.write(" input ["+str(bit_steps_count-1)+":0] step,\n")
f.write(" output reg re_weights,\n")
f.write(" output reg re_bias,\n")
f.write(" output reg ["+str(bit_max_address_value-1)+":0] firstaddr, lastaddr\n")
f.write(");\n")
f.write("parameter convolution_size = 9;\n")
f.write("parameter conv1 = 1*8*3 * convolution_size;\n")
f.write("parameter conv2_1 = 8 * convolution_size + conv1;\n")
f.write("parameter conv2_2 = (8*8*2) + conv2_1;\n")
f.write("parameter conv3_1 = 16 * convolution_size + conv2_2;\n")
f.write("parameter conv3_2 = (16*16*2) + conv3_1;\n")
f.write("parameter conv4_1 = 32 * convolution_size + conv3_2;\n")
f.write("parameter conv4_2 = (32*32) + conv4_1;\n")
f.write("parameter conv5_1 = 32 * convolution_size + conv4_2;\n")
f.write("parameter conv5_2 = (32*32*2) + conv5_1;\n")
f.write("parameter conv6_1 = 64 * convolution_size + conv5_2;\n")
f.write("parameter conv6_2 = (64*64) + conv6_1;\n")
f.write("parameter conv7_1 = 64 * convolution_size + conv6_2;\n")
f.write("parameter conv7_2 = (64*64*2) + conv7_1;\n")
f.write("parameter conv8_1 = 128 * convolution_size + conv7_2;\n")
f.write("parameter conv8_2 = (128*128) + conv8_1;\n")
f.write("parameter conv9_1 = 128 * convolution_size + conv8_2;\n")
f.write("parameter conv9_2 = (128*128) + conv9_1;\n")
f.write("parameter conv10_1 = 128 * convolution_size + conv9_2;\n")
f.write("parameter conv10_2 = (128*128) + conv10_1;\n")
f.write("parameter conv11_1 = 128 * convolution_size + conv10_2;\n")
f.write("parameter conv11_2 = (128*128) + conv11_1;\n")
f.write("parameter conv12_1 = 128 * convolution_size + conv11_2;\n")
f.write("parameter conv12_2 = (128*128) + conv12_1;\n")
f.write("parameter conv13_1 = 128 * convolution_size + conv12_2;\n")
f.write("parameter conv13_2 = (128*128*2) + conv13_1;\n")
f.write("parameter conv14_1 = 256 * convolution_size + conv13_2;\n")
f.write("parameter conv14_2_1 = ((256*256)>>1) + conv14_1;\n")
f.write("parameter conv14_2_2 = ((256*256)>>1) + conv14_2_1;\n")
f.write("parameter predict = 512 + conv14_2_2;\n")
f.write("\n")
f.write("\n")
f.write("parameter bias1 = 8;\n")
f.write("parameter bias2_1 = (8)+8;\n")
f.write("parameter bias2_2 = (16)+16;\n")
f.write("parameter bias3_1 = (32)+16;\n")
f.write("parameter bias3_2 = (48)+32;\n")
f.write("parameter bias4_1 = (80)+32;\n")
f.write("parameter bias4_2 = (112)+32;\n")
f.write("parameter bias5_1 = (144)+32;\n")
f.write("parameter bias5_2 = (176)+64;\n")
f.write("parameter bias6_1 = (240)+64;\n")
f.write("parameter bias6_2 = (304)+64;\n")
f.write("parameter bias7_1 = (368)+64;\n")
f.write("parameter bias7_2 = (432)+128;\n")
f.write("parameter bias8_1 = (560)+128;\n")
f.write("parameter bias8_2 = (688)+128;\n")
f.write("parameter bias9_1 = (816)+128;\n")
f.write("parameter bias9_2 = (944)+128;\n")
f.write("parameter bias10_1 = (1072)+128;\n")
f.write("parameter bias10_2 = (1200)+128;\n")
f.write("parameter bias11_1 = (1328)+128;\n")
f.write("parameter bias11_2 = (1456)+128;\n")
f.write("parameter bias12_1 = (1584)+128;\n")
f.write("parameter bias12_2 = (1712)+128;\n")
f.write("parameter bias13_1 = (1840)+128;\n")
f.write("parameter bias13_2 = (1968)+256;\n")
f.write("parameter bias14_1 = (2224)+256;\n")
f.write("parameter bias14_2_1 = (2480)+(256>>1);\n")
f.write("parameter bias14_2_2 = (2608)+(256>>1);\n")
f.write("\n")
f.write("\n")
f.write("always @(step)\n")
f.write("case (step) \n")
f.write("8'd1: begin //weights conv1 \n")
f.write(" firstaddr = 0;\n")
f.write(" lastaddr = conv1;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd2: begin //bias conv1\n")
f.write(" firstaddr = 0;\n")
f.write(" lastaddr = bias1;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd4: begin //weights conv2 dw 1\n")
f.write(" firstaddr = conv1;\n")
f.write(" lastaddr = conv2_1;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd5: begin //bias conv2 dw\n")
f.write(" firstaddr = bias1;\n")
f.write(" lastaddr = bias2_1;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd7: begin //weights conv2 1x1\n")
f.write(" firstaddr = conv2_1;\n")
f.write(" lastaddr = conv2_2;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd8: begin //bias conv2 1x1\n")
f.write(" firstaddr = bias2_1;\n")
f.write(" lastaddr = bias2_2;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd10: begin //weights conv3 dw 2\n")
f.write(" firstaddr = conv2_2;\n")
f.write(" lastaddr = conv3_1;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd11: begin //bias conv3 DW\n")
f.write(" firstaddr = bias2_2;\n")
f.write(" lastaddr = bias3_1;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd13: begin //weights conv3 1x1\n")
f.write(" firstaddr = conv3_1;\n")
f.write(" lastaddr = conv3_2;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd14: begin //bias conv\n")
f.write(" firstaddr = bias3_1;\n")
f.write(" lastaddr = bias3_2;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd16: begin\n")
f.write(" firstaddr = conv3_2; // dw 3\n")
f.write(" lastaddr = conv4_1;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd17: begin\n")
f.write(" firstaddr = bias3_2;\n")
f.write(" lastaddr = bias4_1;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd19: begin\n")
f.write(" firstaddr = conv4_1;\n")
f.write(" lastaddr = conv4_2;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd20: begin\n")
f.write(" firstaddr = bias4_1;\n")
f.write(" lastaddr = bias4_2;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd22: begin\n")
f.write(" firstaddr = conv4_2; // dw 4\n")
f.write(" lastaddr = conv5_1;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd23: begin\n")
f.write(" firstaddr = bias4_2;\n")
f.write(" lastaddr = bias5_1;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd25: begin\n")
f.write(" firstaddr = conv5_1;\n")
f.write(" lastaddr = conv5_2;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd26: begin\n")
f.write(" firstaddr = bias5_1;\n")
f.write(" lastaddr = bias5_2;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd28: begin\n")
f.write(" firstaddr = conv5_2; // dw 5\n")
f.write(" lastaddr = conv6_1;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd29: begin\n")
f.write(" firstaddr = bias5_2;\n")
f.write(" lastaddr = bias6_1;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd31: begin\n")
f.write(" firstaddr = conv6_1;\n")
f.write(" lastaddr = conv6_2;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd32: begin\n")
f.write(" firstaddr = bias6_1;\n")
f.write(" lastaddr = bias6_2;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd34: begin\n")
f.write(" firstaddr = conv6_2; // dw 6\n")
f.write(" lastaddr = conv7_1;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd35: begin\n")
f.write(" firstaddr = bias6_2;\n")
f.write(" lastaddr = bias7_1;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd37: begin\n")
f.write(" firstaddr = conv7_1;\n")
f.write(" lastaddr = conv7_2;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd38: begin\n")
f.write(" firstaddr = bias7_1;\n")
f.write(" lastaddr = bias7_2;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd40: begin\n")
f.write(" firstaddr = conv7_2; // dw 7\n")
f.write(" lastaddr = conv8_1;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd41: begin\n")
f.write(" firstaddr = bias7_2;\n")
f.write(" lastaddr = bias8_1;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd43: begin\n")
f.write(" firstaddr = conv8_1;\n")
f.write(" lastaddr = conv8_2;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd44: begin\n")
f.write(" firstaddr = bias8_1;\n")
f.write(" lastaddr = bias8_2;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd46: begin\n")
f.write(" firstaddr = conv8_2; // dw 8\n")
f.write(" lastaddr = conv9_1;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd47: begin\n")
f.write(" firstaddr = bias8_2;\n")
f.write(" lastaddr = bias9_1;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd49: begin\n")
f.write(" firstaddr = conv9_1;\n")
f.write(" lastaddr = conv9_2;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd50: begin\n")
f.write(" firstaddr = bias9_1;\n")
f.write(" lastaddr = bias9_2;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd52: begin\n")
f.write(" firstaddr = conv9_2; // dw 9\n")
f.write(" lastaddr = conv10_1;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd53: begin\n")
f.write(" firstaddr = bias9_2;\n")
f.write(" lastaddr = bias10_1;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd55: begin\n")
f.write(" firstaddr = conv10_1;\n")
f.write(" lastaddr = conv10_2;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd56: begin\n")
f.write(" firstaddr = bias10_1;\n")
f.write(" lastaddr = bias10_2;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd58: begin\n")
f.write(" firstaddr = conv10_2; // dw 10\n")
f.write(" lastaddr = conv11_1;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd59: begin\n")
f.write(" firstaddr = bias10_2;\n")
f.write(" lastaddr = bias11_1;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd61: begin\n")
f.write(" firstaddr = conv11_1;\n")
f.write(" lastaddr = conv11_2;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd62: begin\n")
f.write(" firstaddr = bias11_1;\n")
f.write(" lastaddr = bias11_2;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd64: begin\n")
f.write(" firstaddr = conv11_2; // dw 11\n")
f.write(" lastaddr = conv12_1;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd65: begin\n")
f.write(" firstaddr = bias11_2;\n")
f.write(" lastaddr = bias12_1;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd67: begin\n")
f.write(" firstaddr = conv12_1;\n")
f.write(" lastaddr = conv12_2;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd68: begin\n")
f.write(" firstaddr = bias12_1;\n")
f.write(" lastaddr = bias12_2;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd70: begin\n")
f.write(" firstaddr = conv12_2; // dw 12\n")
f.write(" lastaddr = conv13_1;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd71: begin\n")
f.write(" firstaddr = bias12_2;\n")
f.write(" lastaddr = bias13_1;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd73: begin\n")
f.write(" firstaddr = conv13_1;\n")
f.write(" lastaddr = conv13_2;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd74: begin\n")
f.write(" firstaddr = bias13_1;\n")
f.write(" lastaddr = bias13_2;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd76: begin\n")
f.write(" firstaddr = conv13_2; // dw 13\n")
f.write(" lastaddr = conv14_1;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd77: begin\n")
f.write(" firstaddr = bias13_2;\n")
f.write(" lastaddr = bias14_1;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd79: begin\n")
f.write(" firstaddr = conv14_1;\n")
f.write(" lastaddr = conv14_2_1;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd80: begin\n")
f.write(" firstaddr = bias14_1;\n")
f.write(" lastaddr = bias14_2_1;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd82: begin\n")
f.write(" firstaddr = conv14_2_1;\n")
f.write(" lastaddr = conv14_2_2;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("8'd83: begin\n")
f.write(" firstaddr = bias14_2_1;\n")
f.write(" lastaddr = bias14_2_2;\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 1;\n")
f.write(" end\n")
f.write("8'd85: begin\n")
f.write(" firstaddr = conv14_2_2;\n")
f.write(" lastaddr = predict;\n")
f.write(" re_weights = 1;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("default:\n")
f.write(" begin\n")
f.write(" re_weights = 0;\n")
f.write(" re_bias = 0;\n")
f.write(" end\n")
f.write("endcase\n")
f.write("endmodule\n")
f.close()
def border(directory, razmer):
f = open(directory + "border.v",'w')
bit_matrix = len(bin(razmer))-2
bit_matrix_2=len(bin(razmer*razmer))-2
f.write("module border(\n")
f.write(" input clk, go,\n")
f.write(" input ["+str(bit_matrix_2-1)+":0] i,\n")
f.write(" input ["+str(bit_matrix-1)+":0] matrix,\n")
f.write(" output reg [1:0] prov\n")
f.write(");\n")
f.write(" always @(posedge clk)\n")
f.write(" begin \n")
f.write(" if (go == 1)\n")
f.write(" begin\n")
f.write(" prov = 0;\n")
for i in range(128):
f.write(" if ((i == "+str(i+1)+"*matrix-1'b1) && (prov != 2'b10)) prov = 2'b10;\n")
f.write("\n")
for i in range(128+1):
f.write(" if ((i == "+str(i)+"*matrix) && (prov != 2'b11)) prov = 2'b11;\n")
f.write(" end\n")
f.write(" else\n")
f.write(" prov = 0;\n")
f.write(" end\n")
f.write("endmodule\n")
f.close()
def conv_block(directory,razmer):
f = open(directory + "conv.v",'w')
bit_matrix = len(bin(razmer)) - 2
bit_matrix_2 = len(bin(razmer*razmer)) - 2
f.write("module conv(clk,Y1,prov,matrix,matrix2,i,up_perm,down_perm,p1,p2,p3,w1,w2,w3,conv_en,dense_en,stride_plus_prov);\n")
f.write("\n")
f.write("parameter SIZE=0;\n")
f.write("parameter SIZE_address_pix=18;\n")
f.write("parameter SIZE_weights=0;\n")
f.write("\n")
f.write("input clk;\n")
f.write("output reg signed [32-1:0] Y1;\n")
f.write("input [1:0] prov;\n")
f.write("input ["+str(bit_matrix-1)+":0] matrix;\n")
f.write("input ["+str(bit_matrix_2-1)+":0] matrix2;\n")
f.write("input ["+str(bit_matrix_2-1)+":0] i;\n")
f.write("input up_perm,down_perm;\n")
f.write("input signed [SIZE-1:0] p1,p2,p3;\n")
f.write("input signed [SIZE_weights-1:0] w1,w2,w3;\n")
f.write("input conv_en;\n")
f.write("input dense_en;\n")
f.write("input [SIZE_address_pix-1:0] stride_plus_prov;\n")
f.write("\n")
f.write("wire up,down;\n")
f.write("\n")
f.write("assign up = (((i+stride_plus_prov)<=matrix-1'b1)&&(up_perm))?1'b1:1'b0;\n")
f.write("assign down = (((i+stride_plus_prov)>=matrix2-matrix)&&(down_perm))?1'b1:1'b0;\n")
f.write("\n")
f.write("always @(posedge clk)\n")
f.write(" begin\n")
f.write(" if (conv_en==1)\n")
f.write(" begin\n")
f.write(" Y1=0;\n")
f.write(" if ((prov!=2'b11)&&(!up)&&(!down)) Y1 = Y1+(p1*w1);\n")
f.write(" if ((!up)&&(!down)) Y1 = Y1+(p2*w2);\n")
f.write(" if ((prov!=2'b10)&&(!up)&&(!down)) Y1 = Y1+(p3*w3);\n")
f.write(" end\n")
f.write(" end\n")
f.write("\n")
f.write("endmodule\n")
f.close()
def conv_TOP(directory, razmer, max_conv_input_size, max_conv_output_size, num_conv, steps_count, sizeI, sizeW):
f = open(directory + "conv_TOP.v", 'w')
bit_matrix=len(bin(razmer))-2
bit_matrix_2=len(bin(razmer*razmer))-2
bit_max_conv_input_size = len(bin(max_conv_input_size)) - 2
bit_max_conv_output_size = len(bin(max_conv_output_size)) - 2
bit_razmer_2 = len(bin(razmer * razmer)) - 2
bit_steps_count = len(bin(steps_count)) - 2
Y=''
w=''
p=''
res_out=''
res=''
res_old=''
glob_average_perem=''
glob_average_perem_1=''
res_bias_check=''
data_bias=''
for i in range(num_conv):
Y = Y + "Y" + str(i + 1) + ","
w = w + "w"+str(i+1)+"1,w"+str(i+1)+"2,w"+str(i+1)+"3,"
p = p + "p" + str(i) + "_1,p" + str(i) + "_2,p" + str(i) + "_3,"
res_out = res_out + "res_out_" +str(i+1) + ","
res = res + "res"+ str(i+1) + ","
res_old = res_old + "res_old_" + str(i+1) + ","
glob_average_perem = glob_average_perem + "glob_average_perem_" + str(i+1) + ","
glob_average_perem_1 = glob_average_perem_1 + "glob_average_perem_" + str(i + 1) + "_1,"
res_bias_check = res_bias_check + "res_bias_check_" + str(i+1) + ","
data_bias = data_bias + "data_bias_" + str(i+1) + ","
f.write("module conv_TOP(clk,conv_en,STOP,memstartp,memstartw,memstartzap,read_addressp,write_addressp,read_addresstp,write_addresstp,read_addressw,we,re_wb,re,we_t,re_t,qp,qtp,qw,dp,dtp,prov,matrix,matrix2,i_to_prov,lvl,slvl,mem,")
f.write(Y)
f.write(w)
f.write(p)
f.write("go,up_perm,down_perm,num,filt,bias,glob_average_en,step,stride,depthwise,onexone,q_bias,read_addressb,memstartb,stride_plus_prov);\n")
f.write("\n")
for i in range(num_conv):
f.write("parameter SIZE_"+str(i+1)+"=0;\n")
f.write("parameter SIZE_address_pix=13;\n")
f.write("parameter SIZE_address_pix_t=12;\n")
f.write("parameter SIZE_address_wei=13;\n")
f.write("parameter SIZE_weights=0;\n")
f.write("parameter SIZE_bias=0;\n")
f.write("\n")
f.write("input clk,conv_en,glob_average_en;\n")
f.write("input [1:0] prov;\n")
f.write("input ["+str(bit_matrix-1)+":0] matrix;\n")
f.write("input ["+str(bit_matrix_2-1)+":0] matrix2;\n")
f.write("input [SIZE_address_pix-1:0] memstartp;\n")
f.write("input [SIZE_address_wei-1:0] memstartw;\n")
f.write("input [SIZE_address_pix-1:0] memstartzap;\n")
f.write("input [10:0] memstartb;\n")
f.write("input ["+str(bit_max_conv_input_size-1)+":0] lvl;\n")
f.write("input [8:0] slvl;\n")
f.write("output reg [SIZE_address_pix-1:0] read_addressp;\n")
f.write("output reg [SIZE_address_pix_t-1:0] read_addresstp;\n")
f.write("output reg [SIZE_address_wei-1:0] read_addressw;\n")
f.write("output reg [10:0] read_addressb;\n")
f.write("output reg [SIZE_address_pix-1:0] write_addressp;\n")
f.write("output reg [SIZE_address_pix_t-1:0] write_addresstp;\n")
f.write("output reg we,re,re_wb;\n")
f.write("output reg we_t,re_t;\n")
f.write("input signed [SIZE_"+str(num_conv)+"-1:0] qp;\n")
f.write("input signed [32*"+str(num_conv)+"-1:0] qtp;\n")
f.write("input signed [SIZE_weights*9-1:0] qw;\n")
f.write("input signed [SIZE_bias-1:0] q_bias;\n")
f.write("output signed [SIZE_"+str(num_conv)+"-1:0] dp;\n")
f.write("output signed [32*"+str(num_conv)+"-1:0] dtp;\n")
f.write("output reg STOP;\n")
f.write("output reg ["+str(bit_razmer_2-1)+":0] i_to_prov;\n")
f.write("input signed [32-1:0] "+Y[:-1]+";\n")
f.write("output reg signed [SIZE_weights-1:0] "+w[:-1]+";\n")
f.write("output reg signed [SIZE_1-1:0] "+p[:-1]+";\n")
f.write("output reg go;\n")
f.write("output reg up_perm,down_perm;\n")
f.write("input [2:0] num;\n")
f.write("input ["+str(bit_max_conv_output_size-1)+":0] mem;\n")
f.write("input ["+str(bit_max_conv_input_size-1)+":0] filt;\n")
f.write("input bias;\n")
f.write("input ["+str(bit_steps_count-1)+":0] step;\n")
f.write("input [1:0] stride;\n")
f.write("output reg [SIZE_address_pix-1:0] stride_plus_prov;\n")
f.write("\n")
f.write("input depthwise,onexone;\n")
f.write("\n")
for i in range(num_conv):
f.write("reg signed [SIZE_weights-1:0] w" + str(i + 1) + "1_pre,w" + str(i + 1) + "2_pre,w" + str(i + 1) + "3_pre,w" + str(i + 1) + "4_pre,w" + str(i + 1) + "5_pre,w" + str(i + 1) + "6_pre,w" + str(i + 1) + "7_pre,w" + str(i + 1) + "8_pre,w" + str(i + 1) + "9_pre;\n")
f.write("reg signed [SIZE_1-1:0]")
for i in range(int(num_conv/4)):
f.write("p" + str(0+8*i) + "_pre,p" + str(1+8*i) + "_pre,p" + str(2+8*i) + "_pre,p" + str(3+8*i) + "_pre,p" + str(4+8*i) + "_pre,p" + str(5+8*i) + "_pre,p" + str(6+8*i) + "_pre,p" + str(7+8*i) + "_pre")
if ((i+1)==(int(num_conv/4))): f.write(";")
else: f.write(",")
f.write("\n")
f.write("reg signed [SIZE_1-1:0] "+res_out[:-1]+";\n")
f.write("reg signed [32-1:0] "+res[:-1]+";\n")
f.write("reg signed [32-1:0] "+res_old[:-1]+";\n")
f.write("reg signed [21:0] "+glob_average_perem[:-1]+";\n")
f.write("wire signed [SIZE_1-1:0] "+glob_average_perem_1[:-1]+";\n")
f.write("\n")
f.write("reg signed [SIZE_1-1:0]")
for i in range(num_conv):
f.write("buff" + str(i) + "_0 [2:0]")
if (i != (num_conv-1)): f.write(", ")
else: f.write(";\n")
f.write("reg signed [SIZE_1-1:0]")
for i in range(num_conv):
f.write("buff" + str(i) + "_1 [2:0]")
if (i != (num_conv-1)): f.write(", ")
else: f.write(";\n")
f.write("reg signed [SIZE_1-1:0]")
for i in range(num_conv):
f.write("buff" + str(i) + "_2 [2:0]")
if (i != (num_conv-1)): f.write(", ")
else: f.write(";\n")
f.write("\n")
f.write("reg [4:0] marker;\n")
f.write("reg zagryzka_weight;\n")
f.write("reg [15:0] i;\n")
f.write("reg [15:0] i_onexone,i_onexone_1;\n")
f.write("wire [15:0] i_onexone_plus1;\n")
f.write("assign i_onexone_plus1 = i_onexone + 1'b1;\n")
f.write("reg [SIZE_address_pix-1:0] stride_plus,next_number,next_number_prov;\n")
f.write("\n")
f.write("reg signed [19-1:0] "+res_bias_check[:-1]+";\n")
f.write("\n")
f.write("reg signed [SIZE_bias-1:0] " + data_bias[:-1]+";\n")
f.write("\n")
f.write("initial zagryzka_weight=0;\n")
f.write("initial marker=0;\n")
f.write("\n")
f.write("wire [15:0] line_stride;\n")
f.write("\n")
f.write("assign line_stride=matrix>>(stride-1);\n")
f.write("\n")
f.write("always @(posedge clk)\n")
f.write("begin\n")
f.write("if (conv_en==1)\n")
f.write(" begin\n")
f.write(" if (zagryzka_weight==0)\n")
f.write(" begin\n")
f.write(" next_number = matrix;\n")
f.write(" next_number_prov = matrix;\n")
f.write(" if ((step!=3)&&(step!=12)&&(step!=24)&&(step!=36)&&(step!=72)) stride_plus=0;\n")
f.write(" else stride_plus=matrix;\n")
f.write(" if ((step!=3)&&(step!=12)&&(step!=24)&&(step!=36)&&(step!=72)) stride_plus_prov=0;\n")
f.write(" else stride_plus_prov=matrix;\n")
f.write(" case (marker)\n")
for i in range(num_conv+2):
f.write(" "+str(i)+": begin\n")
if (i==0): f.write(" re_wb=1;\n")
if (i < num_conv):
f.write(" read_addressw=memstartw+" + str(i) + "*((depthwise)?1:((onexone)?((mem+1)>>3):(filt+1)));\n")
f.write(" read_addressb=memstartb+" + str(i) + ";\n")
if ((i<num_conv+2)&(i>=2)):
for j in range(9):
f.write(" w"+str(i-1)+str(j+1)+"_pre=qw[SIZE_weights*"+str(j+1)+"-1:")
if (j==0): f.write("0")
else: f.write("SIZE_weights*"+str(j))
f.write("]; \n")
f.write("\n")
f.write(" data_bias_" + str(i-1) + " = q_bias;\n")
if (i == num_conv + 1): f.write(" zagryzka_weight=1; re_wb=0; marker=-1;\n")
f.write(" end\n")
f.write(" default: \n")
f.write(" begin\n")
f.write(" read_addressw=0;\n")
f.write(" read_addressb=0;\n")
f.write(" re_wb=0;\n")
f.write(" $display(\"Check zagryzka_weight\");\n")
f.write(" end\n")
f.write(" endcase\n")
f.write(" marker=marker+1;\n")
f.write(" end\n")
f.write(" else\n")
f.write(" begin\n")
f.write(" re=1;\n")
f.write(" case (marker)\n")
f.write(" 0: begin \n")
f.write(" re_t=0;\n")
f.write(" if ((stride==2)&&(i==next_number))\n")
f.write(" begin\n")
f.write(" stride_plus=stride_plus+matrix;\n")
f.write(" next_number = matrix+next_number;\n")
f.write(" end\n")
f.write(" if (onexone) read_addressp = memstartp+(matrix*matrix)*(3*i_onexone_1+marker)+i_onexone-1;\n")
f.write(" else read_addressp=i+memstartp+stride_plus;\n")
f.write("\n")
f.write(" if (onexone)\n")
f.write(" begin\n")
for i in range(num_conv):
f.write(" p" + str(i) + "_1=p6_pre;\n")
f.write(" p" + str(i) + "_2=p7_pre;\n")
f.write(" p" + str(i) + "_3=0;\n")
f.write(" end\n")
f.write(" else\n")
f.write(" begin\n")
f.write(" if (depthwise)\n")
f.write(" begin\n")
for i in range(num_conv):
f.write(" buff" + str(i) + "_2[2]=qp[SIZE_" + str(num_conv-i) + "-1:")
if (num_conv==(i+1)): f.write("0")
else: f.write("SIZE_"+ str(num_conv-i-1))
f.write("];\n")
f.write(" end\n")
f.write(" else\n")
f.write(" begin\n")
f.write(" if (((i+stride_plus-1)<matrix2-matrix)||(onexone))\n")
f.write(" begin\n")
lvl=''
for i in range(len(bin(num_conv))-3):
lvl = ",lvl["+str(i)+"]" + lvl
for i in range(num_conv):
one_size="SIZE_"+str(num_conv-i)
if ((num_conv-i-1)==0): two_size="0"
else: two_size="SIZE_"+str(num_conv-i-1)
if (num_conv>1):
if (i==0):
f.write(" if")
else:
f.write(" else if")
f.write(" ({"+lvl[1:]+"}=="+str(len(bin(num_conv))-3)+"'d"+str(i)+") \n")
f.write(" begin\n")
for i in range(num_conv):
f.write(" buff" + str(i) + "_2[2]=qp["+one_size+"-1:"+two_size+"];\n")
f.write(" end\n")
else:
f.write(" begin\n")
for i in range(num_conv):
f.write(" buff" + str(i) + "2[2]=qp[SIZE_1-1:0];\n")
f.write(" end\n")
f.write(" end\n")
f.write(" else\n")
f.write(" begin\n")
for i in range(num_conv):
f.write(" buff" + str(i) + "_2[2]=0;\n")
f.write(" end\n")
f.write(" end\n")
for i in range(num_conv):
f.write(" p" + str(i) + "_1=buff" + str(i) + "_2[0];\n")
f.write(" p" + str(i) + "_2=buff" + str(i) + "_2[1];\n")
f.write(" p" + str(i) + "_3=buff" + str(i) + "_2[2];\n")
f.write("\n")
f.write(" end\n")
f.write("\n")
for i in range(num_conv):
f.write(" w" + str(i+1) + "1=(onexone)?w" + str(i+1) + "3_pre:w" + str(i+1) + "3_pre;\n")
f.write(" w" + str(i+1) + "2=(onexone)?w" + str(i+1) + "2_pre:w" + str(i+1) + "2_pre;\n")
f.write(" w" + str(i+1) + "3=(onexone)?w" + str(i+1) + "1_pre:w" + str(i+1) + "1_pre;\n")
f.write(" up_perm=0;\n")
f.write(" if (onexone) down_perm=0; else down_perm=1;\n")
for i in range(num_conv):
f.write(" res" + str(i+1) + "=Y" + str(i+1) + ";\n")
f.write(" end\n")
f.write(" 1: begin\n")
f.write(" if (onexone) read_addressp = memstartp+(matrix*matrix)*(3*i_onexone_1+marker)+i_onexone-1;\n")
f.write(" else if ((i+stride_plus)>=matrix-1) read_addressp=i-matrix+memstartp+stride_plus;\n")
f.write("\n")
for i in range(num_conv):
f.write(" res" + str(i+1) + "=res" + str(i+1) + "+Y" + str(i+1) + ";\n")
f.write(" if ((i>=2)&&(((stride==2)&&((((step==3)||(step==12)||(step==24)||(step==36)||(step==72))&&(i[0]==1))||(((step!=3)&&(step!=12)&&(step!=24)&&(step!=36)&&(step!=72))&&(i[0]==0))))||(stride==1))) \n")
f.write(" begin\n")
for i in range(num_conv):
f.write(" res_old_" + str(i+1) + "=qtp[32*" + str(num_conv-i) + "-1:32*")
if (num_conv==(i+1)): f.write("0")
else: f.write(str(num_conv-i-1))
f.write("];\n")
f.write(" end\n")
f.write(" go=0;\n")
f.write(" i_to_prov=i_to_prov+1'b1;\n")
f.write(" if ((stride==2)&&(i_to_prov==next_number_prov)) \n")
f.write(" begin\n")
f.write(" stride_plus_prov=stride_plus_prov+matrix;\n")
f.write(" next_number_prov = matrix+next_number_prov;\n")
f.write(" end\n")
f.write("\n")
for i in range(num_conv):
f.write(" buff" + str(i) + "_2[0]=buff" + str(i) + "_2[1];\n")
f.write(" buff" + str(i) + "_1[0]=buff" + str(i) + "_1[1];\n")
f.write(" buff" + str(i) + "_0[0]=buff" + str(i) + "_0[1];\n")
f.write(" buff" + str(i) + "_2[1]=buff" + str(i) + "_2[2];\n")
f.write(" buff" + str(i) + "_1[1]=buff" + str(i) + "_1[2];\n")
f.write(" buff" + str(i) + "_0[1]=buff" + str(i) + "_0[2];\n")
f.write("\n")
f.write(" end\n")
f.write(" 2: begin\n")
f.write(" if (onexone) read_addressp = memstartp+(matrix*matrix)*(3*i_onexone_1+marker)+i_onexone-1;\n")
f.write(" else if ((i+stride_plus)<matrix2-matrix) read_addressp=i+matrix+memstartp+stride_plus;\n")
f.write("\n")
f.write(" if (onexone)\n")
f.write(" begin\n")
for i in range(num_conv):
f.write(" p" + str(i) + "_pre = qp[SIZE_" + str(num_conv-i) + "-1:")
if (num_conv==(i+1)): f.write("0")
else: f.write("SIZE_" + str(num_conv-i-1))
f.write("];\n")
for i in range(num_conv):
f.write(" p" + str(i) + "_1=p0_pre;\n")
f.write(" p" + str(i) + "_2=p1_pre;\n")
f.write(" p" + str(i) + "_3=p2_pre;\n")
f.write("\n")
f.write(" end\n")
f.write(" else\n")
f.write(" begin\n")
f.write(" if (depthwise)\n")
f.write(" begin\n")
for i in range(num_conv):
f.write(" buff" + str(i) + "_1[2]=qp[SIZE_" + str(num_conv-i) + "-1:")
if (num_conv==(i+1)): f.write("0")
else: f.write("SIZE_" + str(num_conv-i-1))
f.write("];\n")
f.write(" end\n")
f.write(" else\n")
f.write(" begin\n")
lvl=''
for i in range(len(bin(num_conv))-3):
lvl = ",lvl["+str(i)+"]" + lvl
for i in range(num_conv):
one_size="SIZE_"+str(num_conv-i)
if ((num_conv-i-1)==0): two_size="0"
else: two_size="SIZE_"+str(num_conv-i-1)
if (num_conv>1):
if (i==0):
f.write(" if")
else:
f.write(" else if")
f.write(" ({"+lvl[1:]+"}=="+str(len(bin(num_conv))-3)+"'d"+str(i)+") \n")
f.write(" begin\n")
for i in range(num_conv):
f.write(" buff" + str(i) + "_1[2]=qp["+one_size+"-1:"+two_size+"];\n")
f.write(" end\n")
else:
f.write(" begin\n")
for i in range(num_conv):
f.write(" buff" + str(i) + "1[2]=qp[SIZE_1-1:0];\n")
f.write(" end\n")
f.write(" end\n")
for i in range(num_conv):
f.write(" p" + str(i) + "_1=buff" + str(i) + "_1[0];\n")
f.write(" p" + str(i) + "_2=buff" + str(i) + "_1[1];\n")
f.write(" p" + str(i) + "_3=buff" + str(i) + "_1[2];\n")
f.write("\n")
f.write(" end\n")
for i in range(num_conv):
f.write(" w" + str(i+1) + "1=(onexone)?w" + str(i+1) + "9_pre:w" + str(i+1) + "6_pre;\n")
f.write(" w" + str(i+1) + "2=(onexone)?w" + str(i+1) + "8_pre:w" + str(i+1) + "5_pre;\n")
f.write(" w" + str(i+1) + "3=(onexone)?w" + str(i+1) + "7_pre:w" + str(i+1) + "4_pre;\n")
f.write("\n")
f.write(" go=1;\n")
f.write(" up_perm=0;\n")
f.write(" down_perm=0;\n")
f.write(" if ((i>=2)&&(((stride==2)&&((((step==3)||(step==12)||(step==24)||(step==36)||(step==72))&&(i[0]==1))||(((step!=3)&&(step!=12)&&(step!=24)&&(step!=36)&&(step!=72))&&(i[0]==0))))||(stride==1)))\n")
f.write(" begin\n")
f.write(" if (onexone) write_addresstp=i_onexone-2;\n")
f.write(" else write_addresstp=(i>>(stride-1))-1;\n")
f.write(" if (glob_average_en) write_addressp=memstartzap;\n")
f.write(" else\n")
f.write(" begin\n")
f.write(" if (onexone) write_addressp=memstartzap+i_onexone-2;\n")
f.write(" else write_addressp=memstartzap+((i-2)>>(stride-1));\n")
f.write(" end\n")
f.write("\n")
f.write(" if (((onexone && (i_onexone_1 == 0)) || !onexone)&&(!bias)) we_t=1;\n")
f.write("\n")
for i in range(num_conv):
f.write(" res" + str(i+1) + "=res" + str(i+1) + "+Y" + str(i+1) + ";\n")
f.write("\n")
f.write(" if ((lvl!=0)&&(!depthwise))\n")
f.write(" begin\n")
for i in range(num_conv):
f.write(" res" + str(i+1) + "=res" + str(i+1) + "+res_old_" + str(i+1) + ";\n")
f.write(" end\n")
f.write(" if (bias)\n")
f.write(" begin\n")
for i in range(num_conv):
f.write(" res" + str(i+1) + "=res" + str(i+1) + "+(data_bias_" + str(i+1) + "<<"+str(sizeI+1)+");\n")
f.write("\n")
for i in range(num_conv):
f.write(" if (res" + str(i+1) + "<0) res" + str(i+1) + "=0; //RELU\n")
f.write("\n")
for i in range(num_conv):
f.write(" res_bias_check_" + str(i+1) + "=res" + str(i+1) + "["+ str(sizeI+sizeW+2) +"-1-2:SIZE_1-2];\n")
f.write("\n")
for i in range(num_conv):
f.write(" if (res_bias_check_" + str(i+1) + ">(2**(SIZE_1-1))-1) res_out_" + str(i+1) + "=(2**(SIZE_1-1))-1;\n")
f.write(" else res_out_" + str(i+1) + "=res" + str(i+1) + "[SIZE_1+SIZE_1-2-2:SIZE_1-2];\n")
f.write("\n")
f.write(" if ((glob_average_en)&&(i_onexone_1 == 0))\n")
f.write(" begin\n")
for i in range(num_conv):
f.write(" glob_average_perem_" + str(i+1) + " = glob_average_perem_" + str(i+1) + " + res_out_" + str(i+1) + ";\n")
f.write(" end\n")
f.write(" if ((onexone && (i_onexone_1 == 0)) || !onexone) we=1;\n")
f.write(" end\n")
f.write(" end\n")
f.write(" end\n")
f.write(" 3: begin\n")
f.write(" re_t=1;\n")
f.write(" if (onexone) read_addresstp=i_onexone-1;\n")
f.write(" else read_addresstp=(i>>(stride-1))-1;\n")
f.write("\n")
f.write(" if (onexone)\n")
f.write(" begin\n")
if (num_conv<8):
for i in range(num_conv):
f.write(" p" + str(num_conv+i) + "_pre = qp[SIZE_" + str(num_conv-i) + "-1:")
if (num_conv==(i+1)): f.write("0")
else: f.write("SIZE_" + str(num_conv-i-1))
f.write("];\n")
f.write("\n")
for i in range(num_conv):
f.write(" p" + str(i) + "_1=p3_pre;\n")
f.write(" p" + str(i) + "_2=p4_pre;\n")
f.write(" p" + str(i) + "_3=p5_pre;\n")
f.write("\n")
f.write(" end\n")
f.write(" else\n")
f.write(" begin\n")
f.write(" if (depthwise)\n")
f.write(" begin\n")
for i in range(num_conv):
f.write(" buff" + str(i) + "_0[2]=qp[SIZE_" + str(num_conv-i) + "-1:")
if (num_conv==(i+1)): f.write("0")
else: f.write("SIZE_" + str(num_conv-i-1))
f.write("];\n")
f.write(" end\n")
f.write(" else\n")
f.write(" begin\n")
f.write(" if ((i+stride_plus)>=matrix-1)\n")
f.write(" begin\n")
lvl=''
for i in range(len(bin(num_conv))-3):
lvl = ",lvl["+str(i)+"]" + lvl
for i in range(num_conv):
one_size="SIZE_"+str(num_conv-i)
if ((num_conv-i-1)==0): two_size="0"
else: two_size="SIZE_"+str(num_conv-i-1)
if (num_conv>1):
if (i==0):
f.write(" if")
else:
f.write(" else if")
f.write(" ({"+lvl[1:]+"}=="+str(len(bin(num_conv))-3)+"'d"+str(i)+") \n")
f.write(" begin\n")
for i in range(num_conv):
f.write(" buff" + str(i) + "_0[2]=qp["+one_size+"-1:"+two_size+"];\n")
f.write(" end\n")
else:
f.write(" begin\n")
for i in range(num_conv):
f.write(" buff" + str(i) + "0[2]=qp[SIZE_1-1:0];\n")
f.write(" end\n")
f.write(" end\n")
f.write(" else\n")
f.write(" begin\n")
for i in range(num_conv):
f.write(" buff" + str(i) + "_0[2]=0;\n")
f.write(" end\n")
f.write(" end\n")
for i in range(num_conv):
f.write(" p" + str(i) + "_1=buff" + str(i) + "_0[0];\n")
f.write(" p" + str(i) + "_2=buff" + str(i) + "_0[1];\n")
f.write(" p" + str(i) + "_3=buff" + str(i) + "_0[2];\n")
f.write("\n")
f.write(" end\n")
for i in range(num_conv):
f.write(" w" + str(i+1) + "1=(onexone)?w" + str(i+1) + "6_pre:w" + str(i+1) + "9_pre;\n")
f.write(" w" + str(i+1) + "2=(onexone)?w" + str(i+1) + "5_pre:w" + str(i+1) + "8_pre;\n")
f.write(" w" + str(i+1) + "3=(onexone)?w" + str(i+1) + "4_pre:w" + str(i+1) + "7_pre;\n")
f.write("\n")
f.write(" if (onexone) up_perm=0; else up_perm=1;\n")
f.write(" down_perm=0;\n")
f.write(" we_t=0;\n")
f.write(" we=0;\n")
f.write(" end \n")
f.write(" default: $display(\"Check case conv_TOP\");\n")
f.write(" endcase\n")
f.write("\n")
f.write(" if (marker!=3) marker=marker+1;\n")
f.write(" else begin \n")
f.write(" marker=0; \n")
f.write(" if (((i<matrix*line_stride+1)&&(!onexone))||((onexone)&&(i_onexone_plus1<(matrix*line_stride)+2)))\n")
f.write(" begin\n")
f.write(" i=i+1; \n")
f.write(" if (onexone)\n")
f.write(" begin\n")
f.write(" if (i_onexone_1 == 2>>2)\n")
f.write(" begin\n")
f.write(" i_onexone = i_onexone + 1;\n")
f.write(" i_onexone_1 = 0;\n")
f.write(" end\n")
f.write(" else i_onexone_1 = i_onexone_1 + 1;\n")
f.write(" end\n")
f.write(" end\n")
f.write(" else STOP=1; \n")
f.write(" end\n")
f.write(" end\n")
f.write(" end\n")
f.write("else \n")
f.write(" begin\n")
f.write(" i=0;\n")
f.write(" i_to_prov=-2;\n")
f.write(" stride_plus=0;\n")
f.write(" next_number=matrix;\n")
f.write(" zagryzka_weight=0;\n")
f.write(" STOP=0;\n")
f.write(" re=0;\n")
f.write(" re_t=0;\n")
f.write(" go=0;\n")
f.write(" marker=0;\n")
for i in range(num_conv):
f.write(" glob_average_perem_"+str(i+1)+"=0;\n")
f.write(" i_onexone = 0;\n")
f.write(" i_onexone_1 = 0;\n")
f.write(" read_addressw=0;\n")
f.write(" read_addressb=0;\n")
f.write(" re_wb=0;\n")
f.write(" end\n")
f.write("end\n")
for i in range(num_conv):
f.write("assign glob_average_perem_" + str(i+1) + "_1=glob_average_perem_" + str(i+1) + ">>4;\n")
f.write("assign dp={")
for i in range(num_conv):
f.write("(glob_average_en?glob_average_perem_" + str(i+1) + "_1:res_out_" + str(i+1) + ")")
if ((i+1)!=num_conv): f.write(",")
f.write("\n")
f.write("};\n")
f.write("assign dtp={" + str(res[:-1]) + "};\n")
f.write("endmodule\n")
f.close()
def dense(directory, in_dense_razmer, out_dense_razmer, num_conv, sizeI, sizeW):
file = open(directory + "dense.v", 'w')
bit_in = len(bin(in_dense_razmer)) - 2
bit_out = len(bin(out_dense_razmer)) - 2
Y=''
w=''
p=''
Ypl=''
Y_use=''
Y_use_pl=''
for i in range(num_conv):
Y=Y+" Y"+str(i+1)+","
Y_use=Y_use+" Y"+str(i+1)+"_use,"
Y_use_pl = Y_use_pl + " Y" + str(i + 1) + "_use+"
Ypl = Ypl + " (Y" + str(i + 1) + "_use?Y" + str(i + 1) + ":0)+"
w=w+" w"+str(i+1)+"1, w"+str(i+1)+"2, w"+str(i+1)+"3,"
p=p+" p"+str(i+1)+"1, p"+str(i+1)+"2, p"+str(i+1)+"3,"
file.write("module dense(clk, dense_en, STOP, in, out, we, re_p, re_w, read_addressp, read_addressw, write_addressp, memstartp, memstartzap, qp, qw, res,"+Y+w+p+" go, nozero);\n\n")
file.write("parameter num_conv=0;\n")
file.write("\n")
for i in range(num_conv):
file.write("parameter SIZE_"+str(i+1)+"=0;\n")
file.write("parameter SIZE_address_pix=0;\n")
file.write("parameter SIZE_address_wei=0;\n")
file.write("parameter SIZE_weights=0;\n")
file.write("\n")
file.write("input clk,dense_en;\n")
file.write("output reg STOP;\n")
file.write("input ["+str(bit_in-1)+":0] in;\n")
file.write("input ["+str(bit_out-1)+":0] out;\n")
file.write("output reg we,re_p,re_w;\n")
file.write("output reg [SIZE_address_pix-1:0] read_addressp;\n")
file.write("output reg [SIZE_address_wei-1:0] read_addressw;\n")
file.write("output reg [SIZE_address_pix-1:0] write_addressp;\n")
file.write("input [SIZE_address_pix-1:0] memstartp,memstartzap;\n")
file.write("input signed [SIZE_"+str(num_conv)+"-1:0] qp;\n")
file.write("input signed [SIZE_weights*9-1:0] qw;\n")
file.write("output reg signed [SIZE_"+str(num_conv)+"-1:0] res;\n")
file.write("input signed [32-1:0]"+Y[:-1]+";\n")
file.write("output reg signed [SIZE_weights - 1:0]"+w[:-1]+";\n")
file.write("output reg signed [SIZE_1-1:0]"+p[:-1]+";\n")
file.write("output reg go;\n")
file.write("input nozero;\n")
file.write("\n")
for i in range(num_conv):
file.write("reg signed[SIZE_weights - 1:0] w"+str(i+1)+"1_pre, w"+str(i+1)+"2_pre, w"+str(i+1)+"3_pre, w"+str(i+1)+"4_pre, w"+str(i+1)+"5_pre, w"+str(i+1)+"6_pre, w"+str(i+1)+"7_pre, w"+str(i+1)+"8_pre, w"+str(i+1)+"9_pre;\n")
file.write("\n")
for i in range(num_conv):
file.write("reg signed[SIZE_1 - 1:0] p"+str(i+1)+"1_pre, p"+str(i+1)+"2_pre, p"+str(i+1)+"3_pre, p"+str(i+1)+"4_pre, p"+str(i+1)+"5_pre, p"+str(i+1)+"6_pre, p"+str(i+1)+"7_pre, p"+str(i+1)+"8_pre, p"+str(i+1)+"9_pre;\n")
file.write("reg [3:0] marker;\n")
file.write("reg [6:0] lvl;\n")
file.write("reg [8:0] i;\n")
file.write("reg [8:0] j;\n")
bit_sh=len(bin(num_conv)) - 4
if (bit_sh>0): sh="["+str(bit_sh)+":0]"
else: sh=''
file.write("reg "+sh+" sh;\n")
file.write("reg signed [32-1:0] dp;\n") ###[(SIZE_2)*"+str(num_conv)+"-1:0]
file.write("reg signed [SIZE_1-1:0] dp_shift;\n")
file.write("reg signed [19-1:0]dp_check;\n\n")
file.write("always @(posedge clk)\n")
file.write("begin\n")
file.write(" if (dense_en==1)\n")
file.write(" begin\n")
file.write(" re_p=1;\n")
file.write(" case (marker)\n")
k1=1
k2=1
k3=1
k4=1
k7=1
k8=1
k9=1
k10=1
i=2
STOP=0
while (STOP==0):
file.write(str(i)+":begin\n")
file.write(" if (i>(in>>"+str(len(bin(num_conv))-3)+")+1) begin\n")
file.write(" ")
for j in range(num_conv):
file.write(" p"+str(k1)+str(k2)+"_pre = 0;")
if (k2==9):
k2=1
k1+=1
else: k2+=1
file.write("\n end\n")
file.write(" else begin\n")
file.write(" ")
for j in range(num_conv):
one="SIZE_"+str(num_conv-j)
if (num_conv-j-1==0): two="0"
else: two = "SIZE_"+str(num_conv - j - 1)
file.write(" p"+str(k3)+str(k4)+"_pre = qp["+one+" - 1:"+two+"];")
if (k4==9):
k4=1
k3+=1
else: k4+=1
file.write("\n end\n")
file.write(" ")
if (((i>=2)|((i<2)&((i+num_conv)<=num_conv)))&(i<(num_conv+2))):
k6=9
for k5 in range(9):
if (i<2): m=(num_conv*1+1)
else: m=0
file.write("w"+str(i-1+m)+str(k5+1)+"_pre=qw[SIZE_weights*"+str(k6)+"-1:SIZE_weights*")
if ((k6-1)==0): file.write("0]; ")
else: file.write(str(k6-1)+"]; ")
k6-=1
file.write("\n")
if (i<num_conv): file.write(" read_addressw = lvl*29 + " + str(i) + " + j*" + str(num_conv) + ";\n")
if (i==0):
file.write(" we=0;\n")
file.write(" re_w=1;\n")
if (i==1):
STOP=1
if ((i==2)|(i==5)|(i==8)): file.write(" go=0;\n")
if (i==num_conv+1): file.write(" re_w=0;\n")
if ((i==3)|(i==6)|(i==0)):
if ((i==3)|(i==0)): file.write(" if (i!="+str(i)+") ")
else: file.write(" ")
file.write("dp=")
for j in range(num_conv):
file.write("Y"+str(j+1)+"+")
file.write("dp;\n")
if ((i==1)|(i==4)|(i==7)):
file.write(" ")
if (i==1): file.write("if (i!=1) ")
file.write("go=1;\n")
for j in range(num_conv):
file.write(" ")
for k in range(3):
file.write("p" + str(j + 1) + str(k + 1) + "=")
file.write("p" + str(k7) + str(k8) + "_pre; ")
if (k8 == 9):
k8 = 1
k7 += 1
else:
k8 += 1
file.write("\n")
file.write("\n")
for j in range(num_conv):
file.write(" ")
for k in range(3):
file.write("w" + str(j + 1) + str(k + 1) + "=")
file.write("w" + str(k9) + str(k10) + "_pre; ")
if (k10 == 9):
k10 = 1
k9 += 1
else:
k10 += 1
file.write("\n")
file.write("\n")
if (i==num_conv): file.write(" j=j+1;\n")
file.write(" end\n")
if (i!=8): i=i+1
else: i=0
file.write(" default: $display(\"Check case dense\");\n")
file.write(" endcase\n\n")
file.write(" read_addressp=memstartp+i;\n\n")
file.write(" if (marker!=8) marker=marker+1; else marker=0;\n")
file.write(" i=i+1;\n")
file.write(" if ((i>(in>>"+str(len(bin(num_conv))-3)+")+4)&&(marker==4))\n")
file.write(" begin\n")
file.write(" write_addressp=memstartzap+(lvl>>(num_conv>>1));\n")
file.write(" dp_check=dp["+str(sizeI+sizeW+2)+"-1-2:SIZE_1-2];\n")
file.write(" if ((dp_shift<0)&&(nozero==0)) dp_shift=0;\n")
file.write(" if (dp_check>2**(SIZE_1-1)-1) dp_shift=2**(SIZE_1-1)-1;\n")
file.write(" else dp_shift=dp_check;\n")
for i in range(num_conv):
file.write(" if (sh == "+str(i)+") begin")
if (i==0): file.write(" res=0;")
file.write(" res[SIZE_"+str(num_conv-i)+"-1:")
if (num_conv-i-1==0): file.write("0")
else: file.write("SIZE_"+str(num_conv-i-1))
file.write("]=dp_shift; end\n")
file.write(" lvl=lvl+1;\n")
file.write(" i=0; \n")
file.write(" j=0; \n")
file.write(" dp=0; \n")
file.write(" marker=0;\n")
file.write(" sh=sh+1; if (sh==num_conv) sh=0; \n")
file.write(" if ((sh==0)||(lvl==out)) we=1;\n")
file.write(" if (lvl==out) STOP=1;\n")
file.write(" end\n")
file.write("end\n")
file.write("else\n")
file.write("begin\n")
file.write(" marker=0;\n")
file.write(" i=0;\n")
file.write(" j=0;\n")
file.write(" sh=0;\n")
file.write(" we=0;\n")
file.write(" dp=0;\n")
file.write(" res=0;\n")
file.write(" re_p=0;\n")
file.write(" re_w=0;\n")
file.write(" STOP=0;\n")
file.write(" lvl=0;\n")
file.write("end\n")
file.write("end\n")
file.write("endmodule\n")
file.close()
def RAM(directory, max_weights_per_layer, num_conv):
f = open(directory + "RAM.v", 'w')
f.write("module RAM(qp,qtp,qw,dp,dtp,dw,write_addressp,read_addressp,write_addresstp,read_addresstp,write_addressw,read_addressw,we_p,we_tp,we_w,re_p,re_tp,re_w,clk,clk_RAM_w,q_bias,d_bias,we_bias,re_bias,write_address_bias,read_address_bias);\n")
f.write("parameter picture_size=0; \n")
for i in range(num_conv):
f.write("parameter SIZE_"+str(i+1)+"=0;\n")
f.write("parameter SIZE_address_pix=13;\n")
f.write("parameter SIZE_address_pix_t=12;\n")
f.write("parameter SIZE_address_wei=13;\n")
f.write("parameter SIZE_address_image=16;\n")
f.write("parameter SIZE_weights=0;\n")
f.write("parameter SIZE_bias=0;\n")
f.write("\n")
f.write("output reg signed [SIZE_"+str(num_conv)+"-1:0] qp; //read data\n")
f.write("output reg signed [32*"+str(num_conv)+"-1:0] qtp; //read data\n")
f.write("output reg signed [SIZE_weights*9-1:0] qw; //read weight\n")
f.write("output reg signed [SIZE_bias-1:0] q_bias;\n")
f.write("input signed [SIZE_1*"+str(num_conv)+"-1:0] dp; //write data\n")
f.write("input signed [32*"+str(num_conv)+"-1:0] dtp; //write data\n")
f.write("input signed [SIZE_weights*9-1:0] dw; //write weight\n")
f.write("input signed [SIZE_bias-1:0] d_bias;\n")
f.write("input [SIZE_address_pix-1:0] write_addressp, read_addressp;\n")
f.write("input [SIZE_address_pix_t-1:0] write_addresstp, read_addresstp;\n")
f.write("input [SIZE_address_wei-1:0] write_addressw, read_addressw;\n")
f.write("input [10:0] write_address_bias,read_address_bias;\n")
f.write("input we_p;\n")
f.write("input we_tp;\n")
f.write("input we_w;\n")
f.write("input we_bias;\n")
f.write("input re_p;\n")
f.write("input re_tp;\n")
f.write("input re_w;\n")
f.write("input re_bias;\n")
f.write("input clk,clk_RAM_w;\n")
f.write("\n")
f.write("reg signed [SIZE_1*"+str(num_conv)+"-1:0] mem [0:128*128*"+str(int(4/num_conv))+"+4096*"+str(num_conv)+"-1];\n")
f.write("reg signed [32*"+str(num_conv)+"-1:0] mem_t [0:4096-1];\n")
f.write("reg signed [SIZE_weights*9-1:0] weight [0:4095]; \n")
f.write("reg signed [SIZE_bias-1:0] mem_bias [0:256];\n")
f.write("always @ (posedge clk) \n")
f.write(" begin\n")
f.write(" if (we_p) mem[write_addressp] <= dp;\n")
f.write(" if (we_tp) mem_t[write_addresstp] <= dtp;\n")
f.write(" end\n")
f.write("always @ (posedge clk_RAM_w)\n")
f.write(" begin\n")
f.write(" if (we_w) weight[write_addressw] <= dw;\n")
f.write(" if (we_bias) mem_bias[write_address_bias] <= d_bias;\n")
f.write(" end\n")
f.write("always @ (posedge clk)\n")
f.write(" begin\n")
f.write(" if (re_p) qp <= mem[read_addressp];\n")
f.write(" if (re_tp)qtp <= mem_t[read_addresstp];\n")
f.write(" if (re_w) qw <= weight[read_addressw];\n")
f.write(" if (re_bias) q_bias <= mem_bias[read_address_bias];\n")
f.write(" end\n")
f.write("\n")
f.write("endmodule\n")
f.close()
def RAMtoMEM(directory, max_address_value, steps_count, in_dense_razmer, conv_block_size, num_conv):
f = open(directory + "RAMtoMEM.v", 'w')
bit_max_address_value = len(bin(max_address_value)) - 2
bit_weight_case = len(bin(conv_block_size*conv_block_size)) - 2
bit_steps_count = len(bin(steps_count)) - 2
bit_in_dense_razmer = len(bin(in_dense_razmer)) - 2
f.write("module memorywork(clk_RAM_w,data,data_bias,address,we_w,re_weights,re_bias,nextstep,dw,addrw,step_out,GO,in_dense,load_weights,onexone,address_bias,d_bias,load_bias,we_bias,write_address_bias);\n")
f.write("\n")
f.write("parameter num_conv=0;\n")
f.write("\n")
for i in range(num_conv):
f.write("parameter SIZE_"+str(i+1)+"=0;\n")
f.write("parameter SIZE_address_pix=0;\n")
f.write("parameter SIZE_address_wei=0;\n")
f.write("parameter SIZE_weights=0;\n")
f.write("parameter SIZE_bias=0;\n")
f.write("\n")
f.write("input clk_RAM_w;\n")
f.write("input signed [SIZE_weights-1:0] data;\n")
f.write("input signed [SIZE_bias-1:0] data_bias;\n")
f.write("output [23:0] address;\n")
f.write("output reg we_w;\n")
f.write("output re_weights,re_bias;\n")
f.write("input nextstep;\n")
f.write("output reg signed [SIZE_weights*9-1:0] dw;\n")
f.write("output reg [SIZE_address_wei-1:0] addrw;\n")
f.write("output ["+str(bit_steps_count-1)+":0] step_out;\n")
f.write("input GO;\n")
f.write("input ["+str(bit_in_dense_razmer-1)+":0] in_dense;\n")
f.write("input load_weights,load_bias;\n")
f.write("\n")
f.write("output reg signed [SIZE_bias-1:0] d_bias;\n")
f.write("output reg we_bias;\n")
f.write("output reg [10:0] write_address_bias;\n")
f.write("output [11:0] address_bias;\n")
f.write("\n")
f.write("input onexone; \n")
f.write("\n")
f.write("reg [SIZE_address_pix-1:0] addr;\n")
f.write("wire ["+str(bit_max_address_value-1)+":0] firstaddr,lastaddr;\n")
f.write("\n")
f.write("wire [18:0] razn_addr;\n")
f.write("assign razn_addr = lastaddr-firstaddr;\n")
f.write("\n")
f.write("reg ["+str(bit_steps_count-1)+":0] step;\n")
f.write("reg ["+str(bit_steps_count-1)+":0] step_n;\n")
f.write("reg ["+str(bit_weight_case-1)+":0] weight_case;\n")
f.write("reg [SIZE_weights*9-1:0] buff;\n")
f.write("reg ["+str(bit_max_address_value-1)+":0] i;\n")
f.write("reg ["+str(bit_max_address_value-1)+":0] i_d;\n")
f.write("reg ["+str(bit_max_address_value-1)+":0] i1;\n")
f.write("addressRAM inst_1(.step(step_out),.re_weights(re_weights),.re_bias(re_bias),.firstaddr(firstaddr),.lastaddr(lastaddr)); \n")
f.write("initial weight_case=0;\n")
f.write("initial i=0;\n")
f.write("initial i_d=0;\n")
f.write("initial i1=0;\n")
f.write("\n")
f.write("always @(negedge clk_RAM_w)\n")
f.write(" if ( (step_out==1)||(step_out==2)\n")
f.write(" ||(step_out==4)||(step_out==5)\n")
f.write(" ||(step_out==7)||(step_out==8)\n")
f.write(" ||(step_out==10)||(step_out==11)\n")
f.write(" ||(step_out==13)||(step_out==14)\n")
f.write(" ||(step_out==16)||(step_out==17)\n")
f.write(" ||(step_out==19)||(step_out==20)\n")
f.write(" ||(step_out==22)||(step_out==23)\n")
f.write(" ||(step_out==25)||(step_out==26)\n")
f.write(" ||(step_out==28)||(step_out==29)\n")
f.write(" ||(step_out==31)||(step_out==32)\n")
f.write(" ||(step_out==34)||(step_out==35)\n")
f.write(" ||(step_out==37)||(step_out==38)\n")
f.write(" ||(step_out==40)||(step_out==41)\n")
f.write(" ||(step_out==43)||(step_out==44)\n")
f.write(" ||(step_out==46)||(step_out==47)\n")
f.write(" ||(step_out==49)||(step_out==50)\n")
f.write(" ||(step_out==52)||(step_out==53)\n")
f.write(" ||(step_out==55)||(step_out==56)\n")
f.write(" ||(step_out==58)||(step_out==59)\n")
f.write(" ||(step_out==61)||(step_out==62)\n")
f.write(" ||(step_out==64)||(step_out==65)\n")
f.write(" ||(step_out==67)||(step_out==68)\n")
f.write(" ||(step_out==70)||(step_out==71)\n")
f.write(" ||(step_out==73)||(step_out==74)\n")
f.write(" ||(step_out==76)||(step_out==77)\n")
f.write(" ||(step_out==79)||(step_out==80)\n")
f.write(" ||(step_out==82)||(step_out==83)\n")
f.write(" ||(step_out==85)\n")
f.write(" )\n")
f.write(" begin\n")
f.write(" if ((i<=razn_addr+1)&&(re_weights)) addr=i1;\n")
f.write(" if ((i<=razn_addr+1)&&(re_bias)) addr=i;\n")
f.write(" end\n")
f.write("\n")
f.write("always @(posedge clk_RAM_w or posedge GO)\n")
f.write(" if (GO) step=1;\n")
f.write(" else\n")
f.write(" begin\n")
f.write(" case (step_out)\n")
f.write(" 8'd1,8'd4,8'd7,8'd10,8'd13,8'd16,8'd19,8'd22,8'd25,8'd28,8'd31,8'd34,8'd37,8'd40,8'd43,8'd46,8'd49,8'd52,8'd55,8'd58,8'd61,8'd64,8'd67,8'd70,8'd73,8'd76,8'd79,8'd82,8'd85:\n")
f.write(" begin\n")
f.write(" if (i<=razn_addr+3)\n")
f.write(" begin\n")
f.write(" we_w=0;\n")
f.write(" addrw=addr;\n")
f.write(" if (load_weights==1'b1) i=i+1; \n")
f.write(" if (step_out==85) if (i_d==((in_dense)+1)) begin dw=buff; we_w=1; weight_case=1; i_d=1; i1=i1+1; end\n")
f.write(" case (weight_case)\n")
f.write(" 0: ;\n")
f.write(" 1: begin buff=0; buff[SIZE_weights*9-1:SIZE_weights*8]=data[SIZE_weights-1:0]; end \n")
f.write(" 2: buff[SIZE_weights*8-1:SIZE_weights*7]=data[SIZE_weights-1:0]; \n")
f.write(" 3: buff[SIZE_weights*7-1:SIZE_weights*6]=data[SIZE_weights-1:0]; \n")
f.write(" 4: buff[SIZE_weights*6-1:SIZE_weights*5]=data[SIZE_weights-1:0]; \n")
f.write(" 5: buff[SIZE_weights*5-1:SIZE_weights*4]=data[SIZE_weights-1:0]; \n")
f.write(" 6: buff[SIZE_weights*4-1:SIZE_weights*3]=data[SIZE_weights-1:0]; \n")
f.write(" 7: buff[SIZE_weights*3-1:SIZE_weights*2]=data[SIZE_weights-1:0]; \n")
f.write(" 8: buff[SIZE_weights*2-1:SIZE_weights]=data[SIZE_weights-1:0]; \n")
f.write(" 9: begin buff[SIZE_weights-1:0]=data[SIZE_weights-1:0]; end\n")
f.write(" default: $display(\"Check weight_case\");\n")
f.write(" endcase\n")
f.write(" if (load_weights==1'b1) i_d=i_d+1;\n")
f.write(" if (load_weights==1'b1)\n")
f.write(" begin\n")
f.write(" if ((weight_case==9)||((onexone)&&(weight_case==8))) \n")
f.write(" begin \n")
f.write(" weight_case=1; \n")
f.write(" dw=buff; \n")
f.write(" we_w=1; \n")
f.write(" i1=i1+1;\n")
f.write(" end \n")
f.write(" else \n")
f.write(" begin\n")
f.write(" weight_case=weight_case+1;\n")
f.write(" end\n")
f.write(" end\n")
f.write(" end\n")
f.write(" if (i>razn_addr+3)\n")
f.write(" begin\n")
f.write(" step=step+1; //next step\n")
f.write(" i=0;\n")
f.write(" i_d=0;\n")
f.write(" i1=0;\n")
f.write(" weight_case=0;\n")
f.write(" end\n")
f.write(" end\n")
f.write(" 8'd2,8'd5,8'd8,8'd11,8'd14,8'd17,8'd20,8'd23,8'd26,8'd29,8'd32,8'd35,8'd38,8'd41,8'd44,8'd47,8'd50,8'd53,8'd56,8'd59,8'd62,8'd65,8'd68,8'd71,8'd74,8'd77,8'd80,8'd83:\n")
f.write(" begin\n")
f.write(" if (i<=razn_addr)\n")
f.write(" begin\n")
f.write(" we_bias=1;\n")
f.write(" write_address_bias=addr;\n")
f.write(" if (load_bias==1'b1) i=i+1;\n")
f.write(" d_bias=data_bias;\n")
f.write(" end\n")
f.write(" else \n")
f.write(" begin\n")
f.write(" step=step+1;\n")
f.write(" i=0;\n")
f.write(" we_bias=0;\n")
f.write(" end\n")
f.write(" end\n")
f.write(" default:\n")
f.write(" begin\n")
f.write(" we_w=0;\n")
f.write(" we_bias=0;\n")
f.write(" i=0;\n")
f.write(" i_d=0;\n")
f.write(" i1=0;\n")
f.write(" end\n")
f.write(" endcase\n")
f.write(" end\n")
f.write("always @(posedge nextstep) if (GO==1) step_n=0; else step_n=step_n+1;\n")
f.write("assign step_out=step+step_n;\n")
f.write("assign address=(re_weights)?(firstaddr+i):0;\n")
f.write("assign address_bias=(re_bias)?(firstaddr+i):0;\n")
f.write("endmodule\n")
f.close()
def result(directory,output_neurons_count,num_conv):
f = open(directory + "result.v", 'w')
bit_output_neurons_count = len(bin(output_neurons_count))-2
bit_marker_chislo = len(bin(output_neurons_count+2)) - 2
f.write("module result(clk,enable,STOP,memstartp,read_addressp,qp,re,RESULT);\n")
f.write("\n")
for i in range(num_conv):
f.write("parameter SIZE_"+str(i+1)+"=0;\n")
f.write("parameter SIZE_address_pix=0;\n")
f.write("\n")
f.write("input clk,enable;\n")
f.write("output reg STOP;\n")
f.write("input [SIZE_address_pix-1:0] memstartp;\n")
f.write("input [SIZE_"+str(num_conv)+"-1:0] qp;\n")
f.write("output reg re;\n")
f.write("output reg [SIZE_address_pix-1:0] read_addressp;\n")
f.write("output reg ["+str(bit_output_neurons_count-1)+":0] RESULT;\n")
f.write("\n")
f.write("reg ["+str(bit_marker_chislo-1)+":0] marker;\n")
f.write("wire signed [SIZE_1-1:0] p1,p2;\n")
f.write("always @(posedge clk)\n")
f.write("begin\n")
f.write("if (enable==1)\n")
f.write("begin\n")
f.write("re=1;\n")
f.write("case (marker)\n")
f.write(" 0: read_addressp=memstartp;\n")
f.write(" 1: ;\n")
f.write(" 2: begin\n")
f.write(" RESULT=0; \n")
f.write(" if (p2>=p1) RESULT=1; \n")
f.write(" else RESULT=0; \n")
f.write(" STOP=1; \n")
f.write(" end\n")
f.write(" default: $display(\"Check case result\");\n")
f.write("endcase\n")
f.write("marker=marker+1;\n")
f.write("end\n")
f.write("else \n")
f.write("begin\n")
f.write("re=0;\n")
f.write("marker=0;\n")
f.write("STOP=0;\n")
f.write("end\n")
f.write("end\n")
f.write("\n")
f.write("assign p1=qp[SIZE_" + str(num_conv) + "-1:SIZE_" + str(num_conv-1) + "];\n")
f.write("assign p2=qp[SIZE_" + str(num_conv-1) + "-1:")
if (num_conv-2!=0): f.write("SIZE_" + str(num_conv-2))
else: f.write("0")
f.write("];\n")
f.write("endmodule\n")
def TOP(directory, sizeI, sizeW, sizeB, razmer, max_address_value, output_neurons_count, max_weights_per_layer,
total_conv_layers_number, max_conv_input_size, in_dense_razmer,
out_dense_razmer, max_conv_output_size, layers, num_conv, steps_count):
f = open(directory + "TOP.v", 'w')
bit_max_address_value = len(bin(max_address_value)) - 2
bit_output_neurons_count = len(bin(output_neurons_count)) - 2
bit_address_pix = len(bin(razmer*razmer*8+razmer*razmer)) - 2
bit_address_pix_t = len(bin(razmer*razmer*4)) - 2
bit_max_weights_per_layer = len(bin(max_weights_per_layer)) - 2
bit_total_conv_layers_number = len(bin(total_conv_layers_number)) - 2
bit_max_conv_input_size = len(bin(max_conv_input_size)) - 2
bit_razmer = len(bin(razmer)) - 2
bit_razmer_2 = len(bin(razmer*razmer)) - 2
bit_in_dense_razmer = len(bin(in_dense_razmer)) - 2
bit_out_dense_razmer = len(bin(out_dense_razmer)) - 2
bit_max_conv_output_size = len(bin(max_conv_output_size)) - 2
bit_steps_count = len(bin(steps_count)) - 2
Y=''
p=''
w=''
p_d=''
w_c=''
w_d=''
for i in range(num_conv):
Y=Y+"Y"+str(i+1)+","
w_c=w_c+"w"+str(i+1)+"1_c,w"+str(i+1)+"2_c,w"+str(i+1)+"3_c,w"+str(i+1)+"4_c,w"+str(i+1)+"5_c,w"+str(i+1)+"6_c,w"+str(i+1)+"7_c,w"+str(i+1)+"8_c,w"+str(i+1)+"9_c,"
w_d=w_d+"w"+str(i+1)+"1_d,w"+str(i+1)+"2_d,w"+str(i+1)+"3_d,w"+str(i+1)+"4_d,w"+str(i+1)+"5_d,w"+str(i+1)+"6_d,w"+str(i+1)+"7_d,w"+str(i+1)+"8_d,w"+str(i+1)+"9_d,"
p_d=p_d+"p"+str(i+1)+"1_d,p"+str(i+1)+"2_d,p"+str(i+1)+"3_d,p"+str(i+1)+"4_d,p"+str(i+1)+"5_d,p"+str(i+1)+"6_d,p"+str(i+1)+"7_d,p"+str(i+1)+"8_d,p"+str(i+1)+"9_d,"
p = p + "p" + str(i + 1) + "1,p" + str(i + 1) + "2,p" + str(i + 1) + "3,p" + str(i + 1) + "4,p" + str(i + 1) + "5,p" + str(i + 1) + "6,p" + str(i + 1) + "7,p" + str(i + 1) + "8,p" + str(i + 1) + "9,"
w = w + "w" + str(i + 1) + "1,w" + str(i + 1) + "2,w" + str(i + 1) + "3,w" + str(i + 1) + "4,w" + str(i + 1) + "5,w" + str(i + 1) + "6,w" + str(i + 1) + "7,w" + str(i + 1) + "8,w" + str(i + 1) + "9,"
f.write("module TOP(\n")
f.write("clk,\n")
f.write("clk_RAM_w,\n")
f.write("clk_RAM_p,\n")
f.write("GO,\n")
f.write("RESULT,\n")
f.write("STOP,\n")
f.write("\n")
f.write("re_weights,\n")
f.write("load_weights,\n")
f.write("dp_weights,\n")
f.write("address_weights,\n")
f.write("\n")
f.write("re_bias,\n")
f.write("load_bias,\n")
f.write("dp_bias,\n")
f.write("address_bias,\n")
f.write("\n")
f.write("we_image,\n")
f.write("dp_image,\n")
f.write("address_image,\n")
f.write("step\n")
f.write(");\n")
f.write("\n")
f.write("parameter num_conv="+str(num_conv)+";\n")
f.write("parameter SIZE_weights = "+str(sizeW+1)+";\n")
f.write("parameter SIZE_bias = "+str(sizeB+1)+";\n")
for i in range(num_conv):
f.write("parameter SIZE_"+ str(i+1) +"=" + str((sizeI + 1)*(i+1)) + ";\n")
f.write("parameter SIZE_address_pix="+str(bit_address_pix)+";\n")
f.write("parameter SIZE_address_pix_t="+str(bit_address_pix_t)+";\n")
f.write("parameter SIZE_address_wei="+str(bit_max_weights_per_layer)+";\n")
f.write("parameter SIZE_address_image=16;\n")
f.write("parameter picture_size = "+str(razmer)+";\n")
f.write("parameter picture_storage_limit = 0;\n")
f.write("parameter razmpar = picture_size >> 1;\n")
f.write("parameter razmpar2 = picture_size >> 2;\n")
f.write("parameter picture_storage_limit_2 = picture_size*picture_size;\n")
f.write("input clk,clk_RAM_w,clk_RAM_p;\n")
f.write("input GO;\n")
f.write("output ["+str(bit_output_neurons_count-1)+":0] RESULT;\n")
f.write("input signed [SIZE_weights-1:0] dp_weights;\n")
f.write("input signed [SIZE_bias-1:0] dp_bias;\n")
f.write("output [23:0] address_weights;\n")
f.write("output [11:0] address_bias;\n")
f.write("input load_weights,load_bias;\n")
f.write("input signed [SIZE_1-1:0] dp_image;\n")
f.write("input [SIZE_address_image-1:0] address_image;\n")
f.write("input we_image;\n")
f.write("output reg STOP;\n")
f.write("output re_weights,re_bias;\n")
f.write("output ["+str(bit_steps_count-1)+":0] step;\n")
f.write("\n")
f.write("wire [SIZE_address_image-1:0] address_image_1;\n")
f.write("\n")
f.write("reg conv_en;\n")
f.write("wire STOP_conv;\n")
f.write("\n")
f.write("reg dense_en;\n")
f.write("wire STOP_dense;\n")
f.write("\n")
f.write("reg result_en;\n")
f.write("wire STOP_res; \n")
f.write("wire ["+str(bit_output_neurons_count-1)+":0] res_out;\n")
f.write("\n")
f.write("reg bias,glob_average_en;\n")
f.write("\n")
f.write("reg ["+str(bit_total_conv_layers_number-1)+":0] TOPlvl_conv;\n")
f.write("wire ["+str(bit_total_conv_layers_number-1)+":0] TOPlvl;\n")
f.write("reg [8:0] lvl;\n")
f.write("reg [8:0] slvl;\n")
f.write("reg [2:0] num;\n")
f.write("reg [SIZE_address_pix-1:0] memstartp;\n")
f.write("wire [SIZE_address_pix-1:0] memstartp_lvl;\n")
f.write("reg [SIZE_address_wei-1:0] memstartw;\n")
f.write("wire [SIZE_address_wei-1:0] memstartw_lvl;\n")
f.write("reg [SIZE_address_pix-1:0] memstartzap;\n")
f.write("wire [SIZE_address_pix-1:0] memstartzap_num;\n")
f.write("wire [10:0] memstartb;\n")
f.write("wire [SIZE_address_pix-1:0] read_addressp;\n")
f.write("wire [SIZE_address_image-1:0] read_addressp_init;\n")
f.write("wire [SIZE_address_pix_t-1:0] read_addresstp;\n")
f.write("wire [SIZE_address_wei-1:0] read_addressw;\n")
f.write("wire [10:0] read_address_bias; \n")
f.write("wire [SIZE_address_pix-1:0] read_addressp_conv;\n")
f.write("wire [SIZE_address_pix-1:0] read_addressp_dense;\n")
f.write("wire [SIZE_address_pix-1:0] read_addressp_res;\n")
f.write("wire [SIZE_address_wei-1:0] read_addressw_conv;\n")
f.write("wire [SIZE_address_wei-1:0] read_addressw_dense;\n")
f.write("wire [SIZE_address_pix-1:0] write_addressp;\n")
f.write("wire [SIZE_address_pix_t-1:0] write_addresstp;\n")
f.write("wire [SIZE_address_wei-1:0] write_addressw;\n")
f.write("wire [10:0] write_address_bias; \n")
f.write("wire [SIZE_address_pix-1:0] write_addressp_zagr;\n")
f.write("wire [SIZE_address_pix-1:0] write_addressp_conv;\n")
f.write("wire [SIZE_address_pix-1:0] write_addressp_dense;\n")
f.write("wire we_p,we_tp,we_w;\n")
f.write("wire re_p,re_tp,re_w,re_p_init;\n")
f.write("wire re_bias_RAM;\n")
f.write("wire we_p_zagr;\n")
f.write("wire we_conv,re_wb_conv,re_conv;\n")
f.write("wire we_dense,re_p_dense,re_w_dense;\n")
f.write("wire we_bias;\n")
f.write("wire re_p_res;\n")
f.write("wire signed [SIZE_"+str(num_conv)+"-1:0] qp;\n")
f.write("wire signed [32*"+str(num_conv)+"-1:0] qtp;\n")
f.write("wire signed [SIZE_weights*9-1:0] qw;\n")
f.write("wire signed [SIZE_bias-1:0] q_bias;\n")
f.write("wire signed [SIZE_"+str(num_conv)+"-1:0] dp;\n")
f.write("wire signed [32*"+str(num_conv)+"-1:0] dtp;\n")
f.write("wire signed [SIZE_weights*9-1:0] dw;\n")
f.write("wire signed [SIZE_"+str(num_conv)+"-1:0] dp_conv;\n")
f.write("wire signed [SIZE_"+str(num_conv)+"-1:0] dp_dense;\n")
f.write("wire signed [SIZE_"+str(num_conv)+"-1:0] dp_zagr;\n")
f.write("wire signed [SIZE_bias-1:0] d_bias;\n")
f.write("\n")
f.write("wire [1:0] prov;\n")
f.write("wire ["+str(bit_razmer_2-1)+":0] i_conv;\n")
f.write("wire signed [32-1:0] ")
for i in range(num_conv):
f.write("Y"+str(i+1))
if (i != num_conv - 1): f.write(",")
f.write(";\n")
f.write("\n")
f.write("wire signed [SIZE_weights-1:0] ")
for i in range(num_conv):
f.write("w"+str(i+1)+"1,w"+str(i+1)+"2,w"+str(i+1)+"3")
if (i!=num_conv-1): f.write(",")
f.write(";\n")
f.write("wire signed [SIZE_weights-1:0] ")
for i in range(num_conv):
f.write("w"+str(i+1)+"1_c,w"+str(i+1)+"2_c,w"+str(i+1)+"3_c")
if (i != num_conv - 1): f.write(",")
f.write(";\n")
f.write("wire signed [SIZE_weights-1:0] ")
for i in range(num_conv):
f.write("w"+str(i+1)+"1_d,w"+str(i+1)+"2_d,w"+str(i+1)+"3_d")
if (i != num_conv - 1): f.write(",")
f.write(";\n")
f.write("wire signed [SIZE_1-1:0] ")
for i in range(num_conv):
f.write("p"+str(i+1)+"1,p"+str(i+1)+"2,p"+str(i+1)+"3")
if (i != num_conv - 1): f.write(",")
f.write(";\n")
f.write("wire signed [SIZE_1-1:0] ")
for i in range(num_conv):
f.write("p"+str(i+1)+"1_c,p"+str(i+1)+"2_c,p"+str(i+1)+"3_c")
if (i != num_conv - 1): f.write(",")
f.write(";\n")
f.write("wire signed [SIZE_1-1:0] ")
for i in range(num_conv):
f.write("p"+str(i+1)+"1_d,p"+str(i+1)+"2_d,p"+str(i+1)+"3_d")
if (i != num_conv - 1): f.write(",")
f.write(";\n")
f.write("wire go_conv;\n")
f.write("wire go_conv_TOP;\n")
f.write("wire go_dense;\n")
f.write("\n")
f.write("reg nextstep;\n")
f.write("\n")
f.write("reg ["+str(bit_razmer-1)+":0] matrix;\n")
f.write("wire ["+str(bit_razmer_2-1)+":0] matrix2; //razmer*razmer\n")
f.write("\n")
f.write("reg ["+str(bit_max_conv_output_size-1)+":0] mem;\n")
f.write("reg ["+str(bit_max_conv_input_size-1)+":0] filt;\n")
f.write("reg [1:0] stride;\n")
f.write("reg depthwise;\n")
f.write("reg onexone;\n")
f.write("\n")
f.write("reg ["+str(bit_in_dense_razmer-1)+":0] in_dense;\n")
f.write("reg ["+str(bit_out_dense_razmer-1)+":0] out_dense;\n")
f.write("reg nozero_dense;\n")
f.write("\n")
f.write("wire clk_RAM;\n")
f.write("\n")
f.write("wire up_perm,down_perm;\n")
f.write("wire [SIZE_address_pix-1:0] stride_plus_prov;\n")
f.write("\n")
f.write("conv_TOP #(\n")
for i in range(num_conv):
f.write(" SIZE_"+ str(i+1) +",\n")
f.write(" SIZE_address_pix,\n")
f.write(" SIZE_address_pix_t,\n")
f.write(" SIZE_address_wei,\n")
f.write(" SIZE_weights,\n")
f.write(" SIZE_bias\n")
f.write(") conv_TOP (\n")
f.write(" .clk (clk),\n")
f.write(" .conv_en (conv_en),\n")
f.write(" .STOP (STOP_co
gitextract_rr5i2nao/
├── README.md
├── a00_common_functions.py
├── a01_oid_utils.py
├── docs/
│ └── Writing_weights_to_memory_using_UART.md
├── r01_prepare_open_images_dataset.py
├── r02_train_mobilenet.py
├── r03_mobilenet_v1_reduce_and_scale_model.py
├── r03_remove_batchnorm_layers.py
├── r04_find_optimal_bit_for_weights.py
├── r05_gen_weights_in_verilog_format.py
├── r06_generate_debug_data.py
├── r07_generate_verilog_for_mobilenet.py
├── r08_generate_weights_file_for_FPGA.py
├── utils/
│ └── data_uart_to_fpga.py
└── verilog/
├── CAMERA/
│ ├── camera_controller.v
│ ├── cmos_i2c_ov5640/
│ │ ├── CMOS_Capture.v
│ │ ├── i2c_com.v
│ │ ├── ov5640_cfg.v
│ │ ├── power_on_delay.v
│ │ └── reg_config.v
│ ├── sdram_ov5640_vga.v
│ └── system_ctrl.v
├── GENERAL.qsf
├── GENERAL.qws
├── GENERAL.v
├── MobileNet_v3_conv_8_3x1/
│ ├── RAM.v
│ ├── RAMtoMEM.v
│ ├── TOP.v
│ ├── addressRAM.v
│ ├── border.v
│ ├── conv.v
│ ├── conv_TOP.v
│ ├── dense.v
│ └── result.v
├── OpenVino_MobileNet.qpf
├── RAM.v
├── Seg7.v
├── UART/
│ ├── async.v
│ └── serialGPIO.v
├── ili9341/
│ ├── tft_ili9341.sv
│ └── tft_ili9341_spi.sv
├── pll_24_100/
│ ├── pll_24_100_0002.qip
│ └── pll_24_100_0002.v
├── pll_24_100.bsf
├── pll_24_100.cmp
├── pll_24_100.ppf
├── pll_24_100.qip
├── pll_24_100.sip
├── pll_24_100.spd
├── pll_24_100.v
├── pll_24_100_sim/
│ ├── aldec/
│ │ └── rivierapro_setup.tcl
│ ├── cadence/
│ │ ├── cds.lib
│ │ ├── hdl.var
│ │ └── ncsim_setup.sh
│ ├── mentor/
│ │ └── msim_setup.tcl
│ ├── pll_24_100.vo
│ └── synopsys/
│ ├── vcs/
│ │ └── vcs_setup.sh
│ └── vcsmx/
│ ├── synopsys_sim.setup
│ └── vcsmx_setup.sh
├── pll_24_100_sim.f
└── scale_picture.v
SYMBOL INDEX (106 symbols across 9 files)
FILE: a00_common_functions.py
function save_in_file (line 20) | def save_in_file(arr, file_name):
function load_from_file (line 24) | def load_from_file(file_name):
function show_image (line 28) | def show_image(im, name='image'):
function show_resized_image (line 34) | def show_resized_image(P, w=1000, h=1000):
function relu_1 (line 39) | def relu_1(x):
function save_history (line 44) | def save_history(history, path, columns=('loss', 'val_loss')):
function get_model (line 54) | def get_model(weights_path):
function get_model_memory_usage (line 62) | def get_model_memory_usage(batch_size, model):
FILE: a01_oid_utils.py
function get_model_memory_usage (line 21) | def get_model_memory_usage(batch_size, model):
function get_description_for_labels (line 48) | def get_description_for_labels():
function random_intensity_change (line 59) | def random_intensity_change(img, max_change):
function random_rotate (line 69) | def random_rotate(image, max_angle):
function read_single_image (line 79) | def read_single_image(path):
function read_image_bgr_fast (line 104) | def read_image_bgr_fast(path):
function prepare_training_csv (line 109) | def prepare_training_csv(type, true_labels_enc, output_path, side_size=1...
function check_validation_set (line 143) | def check_validation_set(input_csv):
function check_train_set (line 161) | def check_train_set(input_csv):
function get_class_labels (line 179) | def get_class_labels(true_labels):
FILE: r02_train_mobilenet.py
function strong_aug (line 48) | def strong_aug(p=.5):
function process_single_item (line 85) | def process_single_item(id, box_size, validation=True):
function batch_generator (line 114) | def batch_generator(X_train, Y_train, batch_size, input_size, prep_input...
function evaluate_generator (line 144) | def evaluate_generator(X_test, Y_test, batch_size, input_size, prep_input):
function load_train_valid_data (line 174) | def load_train_valid_data(train_csv, valid_csv):
function train_mobile_net_v1 (line 185) | def train_mobile_net_v1(input_size, train_csv, valid_csv, type):
function evaluate_model (line 265) | def evaluate_model(model_path, input_size, train_csv, valid_csv):
FILE: r03_mobilenet_v1_reduce_and_scale_model.py
function preproc_input_mathmodel (line 32) | def preproc_input_mathmodel(x):
function rescale_weights (line 38) | def rescale_weights(model, layer_num, coeff):
function rescale_weights_with_bias (line 44) | def rescale_weights_with_bias(model, layer_num, coeff, current_scale):
function rescale_only_bias (line 52) | def rescale_only_bias(model, layer_num, coeff, current_scale):
function rescale_batch_norm_weights_initital_v1 (line 60) | def rescale_batch_norm_weights_initital_v1(model, layer_num, coeff, curr...
function rescale_batch_norm_weights_initital (line 76) | def rescale_batch_norm_weights_initital(model, layer_num, coeff, current...
function rescale_dense_weights (line 85) | def rescale_dense_weights(model, layer_num, current_scale, coeff):
function is_next_relu6 (line 99) | def is_next_relu6(model, layer_id):
function replace_intermediate_layer_in_keras (line 112) | def replace_intermediate_layer_in_keras(model, layer_id, new_layer):
function get_min_max_for_model (line 128) | def get_min_max_for_model(model, img_list):
function load_oid_data (line 222) | def load_oid_data(type):
function process_single_item (line 233) | def process_single_item(id, box_size):
function check_results_are_the_same (line 239) | def check_results_are_the_same(model_path1, model_path2, img_list):
FILE: r04_find_optimal_bit_for_weights.py
function my_convolve (line 31) | def my_convolve(input, kernel):
function my_convolve_fixed_point (line 44) | def my_convolve_fixed_point(input, kernel, bit):
function preprocess_forward (line 57) | def preprocess_forward(arr, val):
function convert_to_fix_point (line 63) | def convert_to_fix_point(arr1, bit):
function from_fix_point_to_float (line 74) | def from_fix_point_to_float(arr, bit):
function compare_outputs (line 78) | def compare_outputs(s1, s2, debug_info=True):
function print_first_pixel_detailed_calculation_dense (line 91) | def print_first_pixel_detailed_calculation_dense(previous_layer_output, ...
function print_first_pixel_detailed_calculation (line 104) | def print_first_pixel_detailed_calculation(previous_layer_output, wgt_bi...
function mmZeroPadding2D_floating_point (line 135) | def mmZeroPadding2D_floating_point(layer, img):
function mmZeroPadding2D_fixed_point (line 156) | def mmZeroPadding2D_fixed_point(layer, img):
function run_TF_Conv2D (line 177) | def run_TF_Conv2D(img, w, b, strides, padding, type='float'):
function run_TF_Depthwise_Conv2D (line 195) | def run_TF_Depthwise_Conv2D(img, w, b, strides, padding, type='float'):
function mmConv2D_floating_point (line 213) | def mmConv2D_floating_point(layer, img, debug_info):
function mmConv2D_fixed_point (line 295) | def mmConv2D_fixed_point(layer, img, bit_precizion, bit_precizion_weight...
function mmGlobalAveragePooling2D_floating_point (line 390) | def mmGlobalAveragePooling2D_floating_point(img):
function mmGlobalAveragePooling2D_fixed_point (line 399) | def mmGlobalAveragePooling2D_fixed_point(img):
function mmActivation_floating_point (line 410) | def mmActivation_floating_point(layer, img, one_value=1.0, debug_info=Fa...
function mmActivation_fixed_point (line 424) | def mmActivation_fixed_point(layer, img, bit_precizion, debug_info=False):
function mmReLU_floating_point (line 438) | def mmReLU_floating_point(layer, img, one_value=1.0, debug_info=False):
function mmReLU_fixed_point (line 452) | def mmReLU_fixed_point(layer, img, bit_precizion, debug_info=False):
function mmDepthwiseConv2D_floating_point (line 466) | def mmDepthwiseConv2D_floating_point(layer, img, debug_info):
function mmDepthwiseConv2D_fixed_point (line 529) | def mmDepthwiseConv2D_fixed_point(layer, img, bit_precizion, bit_precizi...
function mmDense_floating_point (line 601) | def mmDense_floating_point(layer, img, debug_info):
function mmDense_fixed_point (line 642) | def mmDense_fixed_point(layer, img, bit_precizion, bit_precizion_weights...
function go_mat_model (line 692) | def go_mat_model(model, images, bit_precizion, bit_precizion_weights, bi...
function get_error_rate (line 766) | def get_error_rate(a1, a2):
function preproc_input_mathmodel (line 775) | def preproc_input_mathmodel(x):
function load_oid_data_optimal (line 781) | def load_oid_data_optimal(type):
function get_image_set (line 788) | def get_image_set(type, image_limit, preproc_type='keras'):
function find_conv_overflow_bit_values (line 823) | def find_conv_overflow_bit_values(model):
function get_optimal_bit_for_weights (line 853) | def get_optimal_bit_for_weights(type, model_path, image_limit, acceptabl...
FILE: r05_gen_weights_in_verilog_format.py
function my_convolve (line 26) | def my_convolve(input, kernel):
function my_convolve_fixed_point (line 39) | def my_convolve_fixed_point(input, kernel, bit):
function preprocess_forward (line 52) | def preprocess_forward(arr, val):
function convert_to_fix_point (line 58) | def convert_to_fix_point(arr1, bit):
function from_fix_point_to_float (line 69) | def from_fix_point_to_float(arr, bit):
function compare_outputs (line 73) | def compare_outputs(s1, s2, debug_info=True):
function dump_memory_structure_conv (line 86) | def dump_memory_structure_conv(arr, out_file):
function dump_memory_structure_dense (line 99) | def dump_memory_structure_dense(arr, out_file):
function print_first_pixel_detailed_calculation_dense (line 111) | def print_first_pixel_detailed_calculation_dense(previous_layer_output, ...
function print_first_pixel_detailed_calculation (line 124) | def print_first_pixel_detailed_calculation(previous_layer_output, wgt_bi...
function convert_to_normalized_form (line 155) | def convert_to_normalized_form(value, precision, required_precision=None):
function convert_to_normalized_form_array (line 176) | def convert_to_normalized_form_array(value, precision):
function convert_to_normalized_form_v2 (line 182) | def convert_to_normalized_form_v2(value, precision):
function get_shape_string (line 199) | def get_shape_string(w):
function gen_convolution_weights (line 206) | def gen_convolution_weights(level_id, layer, bit_precizion, weight_bit_p...
function gen_depthwise_convolution_weights (line 307) | def gen_depthwise_convolution_weights(level_id, layer, bit_precizion, we...
function gen_dense_weights (line 403) | def gen_dense_weights(level_id, layer, bit_precizion, out_weights):
function generate_weights_for_layers (line 455) | def generate_weights_for_layers(model, bp, weight_bit_precision, bias_bi...
FILE: r06_generate_debug_data.py
function convert_to_normalized_form_v2 (line 24) | def convert_to_normalized_form_v2(value, precision):
function store_layer_result (line 41) | def store_layer_result(level_id, layer, layer_type, bp, res):
function print_convolution_detailed_first_pixel_calculation (line 92) | def print_convolution_detailed_first_pixel_calculation(level_id, layer, ...
function print_depthwise_conv_detailed_first_pixel_calculation (line 168) | def print_depthwise_conv_detailed_first_pixel_calculation(level_id, laye...
function print_dense_detailed_first_pixel_calculation (line 241) | def print_dense_detailed_first_pixel_calculation(level_id, layer, img, i...
function get_filters_size (line 285) | def get_filters_size(arr):
function generate_layer_results (line 290) | def generate_layer_results(model, images, image_bit_precizion, weight_bi...
function get_debug_image (line 356) | def get_debug_image():
function generate_layer_results_for_image (line 366) | def generate_layer_results_for_image(type, model, image_bit_precision, w...
FILE: r07_generate_verilog_for_mobilenet.py
function addressRAM (line 13) | def addressRAM(directory, steps_count, max_address_value):
function border (line 441) | def border(directory, razmer):
function conv_block (line 471) | def conv_block(directory,razmer):
function conv_TOP (line 516) | def conv_TOP(directory, razmer, max_conv_input_size, max_conv_output_siz...
function dense (line 1054) | def dense(directory, in_dense_razmer, out_dense_razmer, num_conv, sizeI,...
function RAM (line 1274) | def RAM(directory, max_weights_per_layer, num_conv):
function RAMtoMEM (line 1336) | def RAMtoMEM(directory, max_address_value, steps_count, in_dense_razmer,...
function result (line 1516) | def result(directory,output_neurons_count,num_conv):
function TOP (line 1571) | def TOP(directory, sizeI, sizeW, sizeB, razmer, max_address_value, outpu...
FILE: r08_generate_weights_file_for_FPGA.py
function load_cache_file (line 4) | def load_cache_file(f):
Condensed preview — 61 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (1,914K chars).
[
{
"path": "README.md",
"chars": 4327,
"preview": "# MobileNet in FPGA\nGenerator of verilog description for FPGA MobileNet implementation.\nThere are several pre-trained mo"
},
{
"path": "a00_common_functions.py",
"chars": 2310,
"preview": "# Util functions\nimport pickle\nimport gzip\nimport cv2\nimport numpy as np\nimport pandas as pd\nimport os\nimport glob\nimpor"
},
{
"path": "a01_oid_utils.py",
"chars": 5745,
"preview": "# coding: utf-8\n__author__ = 'Roman Solovyev (ZFTurbo), IPPM RAS'\n\nimport platform\nfrom PIL import Image\nfrom a00_common"
},
{
"path": "docs/Writing_weights_to_memory_using_UART.md",
"chars": 1428,
"preview": "The [UART](https://en.wikipedia.org/wiki/Universal_asynchronous_receiver-transmitter) port is used to write the neural n"
},
{
"path": "r01_prepare_open_images_dataset.py",
"chars": 4996,
"preview": "# coding: utf-8\n__author__ = 'Roman Solovyev (ZFTurbo), IPPM RAS'\n\n'''\nPrepare dataset for different classes from Open I"
},
{
"path": "r02_train_mobilenet.py",
"chars": 16081,
"preview": "# coding: utf-8\n__author__ = 'Roman Solovyev (ZFTurbo), IPPM RAS'\n\n# Train MobileNet with batch generator and augmentati"
},
{
"path": "r03_mobilenet_v1_reduce_and_scale_model.py",
"chars": 10952,
"preview": "# coding: utf-8\n__author__ = 'Roman Solovyev (ZFTurbo), IPPM RAS'\n\n'''\nCode to find reduction coefficients for fixed poi"
},
{
"path": "r03_remove_batchnorm_layers.py",
"chars": 1158,
"preview": "# coding: utf-8\n__author__ = 'Roman Solovyev (ZFTurbo), IPPM RAS'\n\n# Remove layers which is not needed for inference usi"
},
{
"path": "r04_find_optimal_bit_for_weights.py",
"chars": 38070,
"preview": "# coding: utf-8\n__author__ = 'Roman Solovyev (ZFTurbo), IPPM RAS'\n\n\n'''\nThis code finds out which bit size for weight le"
},
{
"path": "r05_gen_weights_in_verilog_format.py",
"chars": 19351,
"preview": "# coding: utf-8\n__author__ = 'Roman Solovyev (ZFTurbo), IPPM RAS'\n\n'''\nGenerate weights with optimal bit size in verilog"
},
{
"path": "r06_generate_debug_data.py",
"chars": 16648,
"preview": "# coding: utf-8\n__author__ = 'Roman Solovyev (ZFTurbo), IPPM RAS'\n\n\n'''\nThis code takes one image run it through the net"
},
{
"path": "r07_generate_verilog_for_mobilenet.py",
"chars": 101669,
"preview": "# coding: utf-8\n__author__ = 'Alex Kustov, IPPM RAS'\n\nimport os\n\ngpu_use = 0\nos.environ[\"KERAS_BACKEND\"] = \"tensorflow\"\n"
},
{
"path": "r08_generate_weights_file_for_FPGA.py",
"chars": 873,
"preview": "# coding: utf-8\n__author__ = 'Alex Kustov, IPPM RAS'\n\ndef load_cache_file(f):\n file = open(f,'r')\n result_list = ["
},
{
"path": "utils/data_uart_to_fpga.py",
"chars": 1664,
"preview": "import serial\nfrom tqdm import tqdm\n\nWEIGHT_FILE_TO_USE = 'weights/weights_cars.txt'\n\n\nif __name__ == '__main__':\n se"
},
{
"path": "verilog/CAMERA/camera_controller.v",
"chars": 3149,
"preview": "module camera_contoller(\n\toutput\t\t\tCMOS_SCLK,\t//cmos i2c clock\n\tinout\t\t\t\tCMOS_SDAT,\t//cmos i2c data\n\tinput\t\t\t\tCMOS_VSYNC"
},
{
"path": "verilog/CAMERA/cmos_i2c_ov5640/CMOS_Capture.v",
"chars": 5521,
"preview": "/*-------------------------------------------------------------------------\nDescription\t\t\t:\t\tsdram test with uart interf"
},
{
"path": "verilog/CAMERA/cmos_i2c_ov5640/i2c_com.v",
"chars": 3176,
"preview": " //sclksdinݴʱ루i2cдƴ룩\nmodule i2c_com(clock_i2c, //i2cƽӿڴʱӣ0-400khz˴Ϊ20khz\n camera_rstn, \n "
},
{
"path": "verilog/CAMERA/cmos_i2c_ov5640/ov5640_cfg.v",
"chars": 44,
"preview": "module ov5640_cfg(\n \n);\n\n\n\nendmodule"
},
{
"path": "verilog/CAMERA/cmos_i2c_ov5640/power_on_delay.v",
"chars": 1313,
"preview": "//camera power on timing requirement\nmodule power_on_delay(clk_50M,reset_n,camera_rstn,camera_pwnd,initial_en); "
},
{
"path": "verilog/CAMERA/cmos_i2c_ov5640/reg_config.v",
"chars": 17323,
"preview": "//camera�мĴ��������ó���\n module reg_config( \n\t\t input clk_25M,\n\t\t input camera_rstn,\n\t\t input initial_en,\n\t\t out"
},
{
"path": "verilog/CAMERA/sdram_ov5640_vga.v",
"chars": 3921,
"preview": "/*-------------------------------------------------------------------------\nFilename\t\t\t:\t\tsdram_ov5640_vga.v\nDescription"
},
{
"path": "verilog/CAMERA/system_ctrl.v",
"chars": 2254,
"preview": "/*-------------------------------------------------------------------------\nDescription\t\t\t:\t\tsdram vga controller with o"
},
{
"path": "verilog/GENERAL.qsf",
"chars": 10363,
"preview": "# -------------------------------------------------------------------------- #\n#\n# Copyright (C) 2018 Intel Corporation"
},
{
"path": "verilog/GENERAL.v",
"chars": 10306,
"preview": "module GENERAL(\ninput CLOCK_50,\n\n//////////// ILI9341 //////////////\n\ninput \t\t\t\t\t\t\ttft_sdo, \noutput \t\t\t\t\t\t\ttft_sck, \nou"
},
{
"path": "verilog/MobileNet_v3_conv_8_3x1/RAM.v",
"chars": 2045,
"preview": "module RAM(qp,qtp,qw,dp,dtp,dw,write_addressp,read_addressp,write_addresstp,read_addresstp,write_addressw,read_addressw,"
},
{
"path": "verilog/MobileNet_v3_conv_8_3x1/RAMtoMEM.v",
"chars": 5305,
"preview": "module memorywork(clk_RAM_w,data,data_bias,address,we_w,re_weights,re_bias,nextstep,dw,addrw,step_out,GO,in_dense,load_w"
},
{
"path": "verilog/MobileNet_v3_conv_8_3x1/TOP.v",
"chars": 33865,
"preview": "module TOP(\nclk,\nclk_RAM_w,\nclk_RAM_p,\nGO,\nRESULT,\nSTOP,\n\nre_weights,\nload_weights,\ndp_weights,\naddress_weights,\n\nre_bia"
},
{
"path": "verilog/MobileNet_v3_conv_8_3x1/addressRAM.v",
"chars": 9141,
"preview": "module addressRAM(\n\tinput [6:0] step,\n\toutput reg re_weights,\n\toutput reg re_bias,\n\toutput reg [17:0] firstaddr, lastadd"
},
{
"path": "verilog/MobileNet_v3_conv_8_3x1/border.v",
"chars": 18012,
"preview": "module border(\n input clk, go,\n input [14:0] i,\n input [7:0] matrix,\n output reg [1:0] prov\n);\n\talways @(pos"
},
{
"path": "verilog/MobileNet_v3_conv_8_3x1/conv.v",
"chars": 918,
"preview": "module conv(clk,Y1,prov,matrix,matrix2,i,up_perm,down_perm,p1,p2,p3,w1,w2,w3,conv_en,dense_en,stride_plus_prov);\n\nparame"
},
{
"path": "verilog/MobileNet_v3_conv_8_3x1/conv_TOP.v",
"chars": 41244,
"preview": "module conv_TOP(clk,conv_en,STOP,memstartp,memstartw,memstartzap,read_addressp,write_addressp,read_addresstp,write_addre"
},
{
"path": "verilog/MobileNet_v3_conv_8_3x1/dense.v",
"chars": 14966,
"preview": "module dense(clk, dense_en, STOP, in, out, we, re_p, re_w, read_addressp, read_addressw, write_addressp, memstartp, mems"
},
{
"path": "verilog/MobileNet_v3_conv_8_3x1/result.v",
"chars": 892,
"preview": "module result(clk,enable,STOP,memstartp,read_addressp,qp,re,RESULT);\n\nparameter SIZE_1=0;\nparameter SIZE_2=0;\nparameter "
},
{
"path": "verilog/OpenVino_MobileNet.qpf",
"chars": 1282,
"preview": "# -------------------------------------------------------------------------- #\n#\n# Copyright (C) 2018 Intel Corporation"
},
{
"path": "verilog/RAM.v",
"chars": 1366,
"preview": "module RAM_general(\ninput clk_in,\ninput clk_out,\ninput clk_in_im,\ninput clk_out_im,\ninput clk_in_im_scale,\ninput clk_out"
},
{
"path": "verilog/Seg7.v",
"chars": 1177,
"preview": "module Seg7 (data,hex);\n\ninput [3:0] data;\n\noutput [6:0] hex;\n\nassign hex[0] = !(((data==0)||(data==2)||(data==3)||(data"
},
{
"path": "verilog/UART/async.v",
"chars": 7465,
"preview": "////////////////////////////////////////////////////////\n// RS-232 RX and TX module\n// (c) fpga4fun.com & KNJN LLC - 200"
},
{
"path": "verilog/UART/serialGPIO.v",
"chars": 1503,
"preview": "module serialGPIO(\n input clk25,\n input RxD,\n output TxD,\n\t \n\t input reset,\n\t output\treg [23:0] address,\n\t outp"
},
{
"path": "verilog/ili9341/tft_ili9341.sv",
"chars": 4199,
"preview": "/** Simple frame-buffer based driver for the ILI9341 TFT module */\nmodule tft_ili9341(\n\t\tinput clk,\n\t\tinput tft_sdo, out"
},
{
"path": "verilog/ili9341/tft_ili9341_spi.sv",
"chars": 1617,
"preview": "// --- Byte-wise SPI + DC implementation\n// * Will copy data into internal buffer\n// * 'Idle' will be set to 0 once buff"
},
{
"path": "verilog/pll_24_100/pll_24_100_0002.qip",
"chars": 319,
"preview": "set_instance_assignment -name PLL_COMPENSATION_MODE DIRECT -to \"*pll_24_100_0002*|altera_pll:altera_pll_i*|*\"\n \nset_inst"
},
{
"path": "verilog/pll_24_100/pll_24_100_0002.v",
"chars": 2209,
"preview": "`timescale 1ns/10ps\nmodule pll_24_100_0002(\n\n\t// interface 'refclk'\n\tinput wire refclk,\n\n\t// interface 'reset'\n\tinput w"
},
{
"path": "verilog/pll_24_100.bsf",
"chars": 4348,
"preview": "/*\nWARNING: Do NOT edit the input and output ports in this file in a text\neditor if you plan to continue editing the blo"
},
{
"path": "verilog/pll_24_100.cmp",
"chars": 330,
"preview": "\tcomponent pll_24_100 is\n\t\tport (\n\t\t\trefclk : in std_logic := 'X'; -- clk\n\t\t\trst : in std_logic := 'X'; -- rese"
},
{
"path": "verilog/pll_24_100.ppf",
"chars": 555,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<pinplan\n variation_name=\"pll_24_100\"\n megafunction_name=\"ALTERA_PLL\"\n intended_f"
},
{
"path": "verilog/pll_24_100.qip",
"chars": 56256,
"preview": "set_global_assignment -entity \"pll_24_100\" -library \"pll_24_100\" -name IP_TOOL_NAME \"altera_pll\"\nset_global_assignment -"
},
{
"path": "verilog/pll_24_100.sip",
"chars": 536,
"preview": "set_global_assignment -entity \"pll_24_100\" -library \"lib_pll_24_100\" -name IP_TOOL_NAME \"altera_pll\"\nset_global_assignme"
},
{
"path": "verilog/pll_24_100.spd",
"chars": 193,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<simPackage>\n <file path=\"pll_24_100_sim/pll_24_100.vo\" type=\"VERILOG\" />\n <topLe"
},
{
"path": "verilog/pll_24_100.v",
"chars": 17272,
"preview": "// megafunction wizard: %PLL Intel FPGA IP v18.0%\n// GENERATION: XML\n// pll_24_100.v\n\n// Generated using ACDS version 18"
},
{
"path": "verilog/pll_24_100_sim/aldec/rivierapro_setup.tcl",
"chars": 12560,
"preview": "\n# (C) 2001-2019 Altera Corporation. All rights reserved.\n# Your use of Altera Corporation's design tools, logic functio"
},
{
"path": "verilog/pll_24_100_sim/cadence/cds.lib",
"chars": 1297,
"preview": "\nDEFINE std $CDS_ROOT/tools/inca/files/STD/ \nDEFINE synopsys $CDS_ROOT/tools/in"
},
{
"path": "verilog/pll_24_100_sim/cadence/hdl.var",
"chars": 18,
"preview": "\nDEFINE WORK work\n"
},
{
"path": "verilog/pll_24_100_sim/cadence/ncsim_setup.sh",
"chars": 9562,
"preview": "\n# (C) 2001-2019 Altera Corporation. All rights reserved.\n# Your use of Altera Corporation's design tools, logic functio"
},
{
"path": "verilog/pll_24_100_sim/mentor/msim_setup.tcl",
"chars": 12657,
"preview": "\n# (C) 2001-2019 Altera Corporation. All rights reserved.\n# Your use of Altera Corporation's design tools, logic functio"
},
{
"path": "verilog/pll_24_100_sim/pll_24_100.vo",
"chars": 18415,
"preview": "//IP Functional Simulation Model\n//VERSION_BEGIN 18.0 cbx_mgl 2018:04:24:18:08:49:SJ cbx_simgen 2018:04:24:18:04:18:SJ "
},
{
"path": "verilog/pll_24_100_sim/synopsys/vcs/vcs_setup.sh",
"chars": 6356,
"preview": "\n# (C) 2001-2019 Altera Corporation. All rights reserved.\n# Your use of Altera Corporation's design tools, logic functio"
},
{
"path": "verilog/pll_24_100_sim/synopsys/vcsmx/synopsys_sim.setup",
"chars": 616,
"preview": "\nWORK > DEFAULT\nDEFAULT: ./libraries/work/ \nwork: ./libraries/work/ "
},
{
"path": "verilog/pll_24_100_sim/synopsys/vcsmx/vcsmx_setup.sh",
"chars": 9639,
"preview": "\n# (C) 2001-2019 Altera Corporation. All rights reserved.\n# Your use of Altera Corporation's design tools, logic functio"
},
{
"path": "verilog/pll_24_100_sim.f",
"chars": 29,
"preview": "pll_24_100_sim/pll_24_100.vo\n"
},
{
"path": "verilog/scale_picture.v",
"chars": 1230790,
"preview": "module scale_picture(\nclk,\nrst,\nvalid_data,\nr,\ng,\nb,\nx,\ny,\nr_out,\ng_out,\nb_out,\naddr_out,\nvalid_data_out,\ntest\n);\ninput "
}
]
// ... and 1 more files (download for full content)
About this extraction
This page contains the full source code of the ZFTurbo/MobileNet-in-FPGA GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 61 files (1.7 MB), approximately 786.7k tokens, and a symbol index with 106 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.