IC的帆哥 - 量化bit https://icfgblog.com/index.php/tag/%E9%87%8F%E5%8C%96bit/ zh-CN Fri, 26 Mar 2021 09:24:00 +0800 Fri, 26 Mar 2021 09:24:00 +0800 CNN训练MNIST数据集tenflow2(下) https://icfgblog.com/index.php/NN/169.html https://icfgblog.com/index.php/NN/169.html Fri, 26 Mar 2021 09:24:00 +0800 icfg66 接下来利用tflite做量化,其实中间的数据流到底有没量化,我也不清楚。。。

一、数据准备

# 1.数据准备
import tensorflow as tf
import numpy as np

mnist = tf.keras.datasets.mnist
img_rows,img_cols = 28,28
(x_train_, y_train_), (x_test_, y_test_) = mnist.load_data()
x_train = x_train_.reshape(x_train_.shape[0],img_rows,img_cols,1)
x_test = x_test_.reshape(x_test_.shape[0],img_rows,img_cols,1)

x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train = x_train / 256
x_test = x_test / 256
y_train_onehot = tf.keras.utils.to_categorical(y_train_)
y_test_onehot = tf.keras.utils.to_categorical(y_test_)

二、初始模型导入

model = tf.keras.Sequential()
model = tf.keras.models.load_model('models/mnist_tf2_fw.h5')
score = model.evaluate(x_test, y_test_onehot, verbose=0)
print('Test accuracy:', "{:.5f}".format(score[1]))

三、量化模型导入

interpreter= tf.lite.Interpreter(model_path="models/tflite_tf2_8.tflite")
#interpreter= tf.lite.Interpreter(model_path="models/tflite_tf2_dy.tflite")
#interpreter= tf.lite.Interpreter(model_path="models/tflite_tf2_16.tflite")
#interpreter= tf.lite.Interpreter(model_path="models/tflite_tf2_32.tflite")

interpreter.allocate_tensors()

input_details=interpreter.get_input_details()
output_details=interpreter.get_output_details()

#print(input_details)
#print(output_details)
y_lebal=y_test_onehot.argmax(1)

# 量化模型推理
pre = []
# tflite针对安卓端运用,每次只能推理一个数据
for i in range (len(x_test)):
    # .astpe 非常重要,找了很久才解决这个bug
    x_test1 = x_test[i].reshape(1,28,28,1).astype(np.float32)
    interpreter.set_tensor(input_details[0]['index'],x_test1)
    interpreter.invoke()
    output_data = interpreter.get_tensor(output_details[0]['index'])
    pre.append(output_data.argmax())

temp = (pre==y_lebal)
acc=sum(temp)/10000
print(acc)
32bit 16bit 8bit 默认
98.500 98.500 98.440 98.440

这结果似乎和权重量化的差不多。数据流量化确实不太容易,需要更加底层、细粒度的操作。只有rram的训练有可能是4bit和8bit的混合,所以低精度数据流的训练还是挺重要的。

后期计划,做低比特、数据流量化的调用,看看别人的工作,为rram高性能计算打下理论基础。 github 代码

]]>
0 https://icfgblog.com/index.php/NN/169.html#comments https://icfgblog.com/index.php/feed/tag/%E9%87%8F%E5%8C%96bit/
CNN训练MNIST数据集tenflow2(中) https://icfgblog.com/index.php/NN/166.html https://icfgblog.com/index.php/NN/166.html Fri, 26 Mar 2021 08:38:00 +0800 icfg66 紧接上文(上),我们已经做好了训练,分别保存了32bit、16bit、8bit的量化模型。本篇的工作是加载模型,查看增加噪声、权重量化时的精度变化。数据流量化的结果将在下篇介绍。

一、加载数据集

# 1. 加载数据集
import tensorflow as tf
import numpy as np
minst = tf.keras.datasets.mnist
img_rows,img_cols = 28,28
(x_train, y_train), (x_test, y_test) = minst.load_data()
x_train = x_train.reshape(x_train.shape[0],img_rows,img_cols,1)
x_test = x_test.reshape(x_test.shape[0],img_rows,img_cols,1)

x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train = x_train / 255
x_test = x_test / 255
y_train_onehot = tf.keras.utils.to_categorical(y_train)
y_test_onehot = tf.keras.utils.to_categorical(y_test)

二、加载模型和权重,测试准确率

# 2. 加载模型和权重,测试准确率
model = tf.keras.Sequential()
model = tf.keras.models.load_model('models/mnist_tf2_fw.h5')
score = model.evaluate(x_test, y_test_onehot, verbose=0)
print('Test accuracy:', "{:.5f}".format(score[1]))

三、增加噪声,测试准确率

model = tf.keras.models.load_model('models/mnist_tf2_fw.h5')
# 这里0,2,5层分别表示conv1,conv2,fc层的权重
for i in  [0,2,5]:
    # 通过get_weights 和setweights修改权重
    temp = model.layers[i].get_weights()[0]
    # 0.15表征噪声的标准差
    temp = temp + np.random.rand(*temp.shape)*0.15
    model.layers[i].set_weights([temp])
score = model.evaluate(x_test, y_test_onehot, verbose=0)
print('Test accuracy:', "{:.5f}".format(score[1]))

四、权重量化,测试准确率

# 8bit量化
bit = 8
model = tf.keras.models.load_model('models/mnist_tf2_fw.h5')
# 这里0,2,5层分别表示conv1,conv2,fc层的权重
for i in  [0,2,5]:
    temp = model.layers[i].get_weights()[0]
    temp = np.rint(temp * np.power(2,bit))/np.power(2,bit)
    model.layers[i].set_weights([temp])
score = model.evaluate(x_test, y_test_onehot, verbose=0)
print('Test accuracy:', "{:.5f}".format(score[1]))

统计结果如下:

噪声标准差 0 0.04 0.06 0.08 0.09 0.10 0.16
准确率 98.500 96.870 92.590 87.510 78.590 70.180 52.800
量化精度 32bit 16bit 8bit 4bit 3bit 2bit 1bit
准确率 98.500 98.500 98.490 98.410 97.380 89.400 66.250

特别说明的是:这里只是对权重做量化,推理计算过程中的数据还是32bit的浮点数,所以结果上看,3bit量化的精度下降不多,只有约1%。

有意思的一点是:3bit量化的准确率和0.04噪声接近;2bit量化准确率和0.08噪声相近,1bit量化的准确率和0.16的噪声相当。

分析原因:3bit量化相当于最小精度是0.125,可以将其等效为$3\sigma$的偏差,求得标准差为0.041,即为0.04的噪声。

github 代码

]]>
0 https://icfgblog.com/index.php/NN/166.html#comments https://icfgblog.com/index.php/feed/tag/%E9%87%8F%E5%8C%96bit/
CNN训练MNIST数据集tenflow2(上) https://icfgblog.com/index.php/NN/165.html https://icfgblog.com/index.php/NN/165.html Thu, 25 Mar 2021 22:06:00 +0800 icfg66

背景

还是那片nature的文章Fully hardware-implemented memristor convolutional neural network。忆阻器的精度很低,虽然外围的ADC是8bit,但忆阻器实际的精度有4bit就不错了。希望先用32bit浮点数训练网络,然后做量化,比较准确率的下降情况:

  • 搭建网络
  • 增加噪声
  • 量化权重
  • 量化数据流

测试的环境如下:

python 3.7.4, tensorflow 2.0.0

一、 搭建网络

1.数据集准备

import tensorflow as tf
import numpy as np

# 1.数据集准备
minst = tf.keras.datasets.mnist
img_rows,img_cols = 28,28
(x_train, y_train), (x_test, y_test) = minst.load_data()
# 卷积运算需要将原来的(6000,28,28)变成(6000,28,28,1) ,即增加通道的参数
x_train = x_train.reshape(x_train.shape[0],img_rows,img_cols,1)
x_test = x_test.reshape(x_test.shape[0],img_rows,img_cols,1)
# 需要将原来的0-255的整数转为32bit单精度浮点数,再归一化为[0,1]
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train = x_train / 255
x_test = x_test / 255
# 标签为独热码的形式,方便交叉熵运算
y_train_onehot = tf.keras.utils.to_categorical(y_train)
y_test_onehot = tf.keras.utils.to_categorical(y_test)

2. 模型搭建

keras搭建模型很方便:

# 2. 模型搭建
model = tf.keras.Sequential()
# 注意这里padding的类型“valid”,表示不填充外围,输出shape为(26,26,8)。并且没有设置bias,因为忆阻器硬件里不好添加
model.add(tf.keras.layers.Conv2D(8, kernel_size=(3, 3),padding="valid", activation='relu',use_bias=False,input_shape=(28, 28, 1)))
# 同样注意这里的padding类型“sample”,表示除不尽须填充而不是舍去,输出shape为(9,9,8)
model.add(tf.keras.layers.MaxPooling2D(pool_size=(3, 3),padding="same"))
# 同理,注意padding,输出shape=(7,7,12)
model.add(tf.keras.layers.Conv2D(12, kernel_size=(3, 3),padding="valid", activation='relu',use_bias=False,))
# 同理,注意padding,输出shape=(4,4,12)
model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2),padding="same"))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(10, use_bias=False,activation='softmax'))
model.summary()

model

三、开始训练

开始训练,数据集很小,cpu训练一两分钟左右就能达到98%准确率

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

from time import time
startTime = time()
history = model.fit(x_train, y_train_onehot, batch_size = 100, epochs = 10, verbose=2, validation_data = (x_test, y_test_onehot))
duration = time()-startTime
print("Train Finished! takes:","{:.2f}".format(duration))

score = model.evaluate(x_test, y_test_onehot, verbose=0)
print('Test accuracy:', score[1])

image.png

四、模型的保存和加载

如果还要后续操作,模型的保存和加载还是相当重要的。

# 1.保存框架,权重,优化器,损失函数
model.save('models/mnist_tf2_fw.h5')
# 2.保存框架,不含权重和配置
yaml_tring = model.to_yaml()
with open('./models/mnist_tf2_f.yaml','w') as model_file:
    model_file.write(yaml_tring)
# 3.保存权重,不含框架和配置
model.save_weights('models/mnist_tf2_w.h5')

保存量化后的tflite格式,找了好久才搞定的!tflite格式主要是为了安卓端推理应用推出的:

#1 未量化-32bit
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
open("models/tflite_model_32.tflite","wb").write(tflite_model)

# 2.float16量化
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_types = [tf.float16]
tflite_model = converter.convert()
open("models/tflite_model_16.tflite", "wb").write(tflite_model)

# 3.8bit量化
def representative_data_gen():
# *这里需要注意:函数需要获得x_train的shape信息
    data = tf.data.Dataset.from_tensor_slices(x_train).batch(1).take(100)
    for input_value in data:
        yield [input_value]

converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_data_gen
# Restricting supported target op specification to INT8
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
# Set the input and output tensors to uint8 
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
# Convert and Save the model
tflite_model = converter.convert()
open("models/tflite_model_8.tflite", "wb").write(tflite_model)

# 4.动态量化
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_model = converter.convert()
open("models/tflite_model_dy.tflite", "wb").write(tflite_model)

五、小结

截止目前,我们已经做好了训练,分别保存了32bit、16bit、8bit的量化模型。接下来的工作是加载模型,查看增加噪声、量化后对精度的损失。

github 代码

]]>
0 https://icfgblog.com/index.php/NN/165.html#comments https://icfgblog.com/index.php/feed/tag/%E9%87%8F%E5%8C%96bit/