IC的帆哥 - 量化bit https://icfgblog.com/index.php/tag/%E9%87%8F%E5%8C%96bit/ CNN训练MNIST数据集tenflow2(下) https://icfgblog.com/index.php/NN/169.html 2021-03-26T09:24:00+08:00 接下来利用tflite做量化,其实中间的数据流到底有没量化,我也不清楚。。。 一、数据准备 # 1.数据准备 import tensorflow as tf import numpy as np mnist = tf.keras.datasets.mnist img_rows,img_cols = 28,28 (x_train_, y_train_), (x_test_, y_test_) = mnist.load_data() x_train = x_train_.reshape(x_train_.shape[0],img_rows,img_cols,1) x_test = x_test_.reshape(x_test_.shape[0],img_rows,img_cols,1) x_train = x_train.astype('float32') x_test = x_test.astype('float32') x_train = x_train / 256 x_test = x_test / 256 y_train_onehot = tf.keras.utils.to_categorical(y_train_) y_test_onehot = tf.keras.utils.to_categorical(y_test_) 二、初始模型导入 model = tf.keras.Sequential() model = tf.keras.models.load_model('models/mnist_tf2_fw.h5') score = model.evaluate(x_test, y_test_onehot, verbose=0) print('Test accuracy:', "{:.5f}".format(score[1])) 三、量化模型导入 interpreter= tf.lite.Interpreter(model_path="models/tflite_tf2_8.tflite") #interpreter= tf.lite.Interpreter(model_path="models/tflite_tf2_dy.tflite") #interpreter= tf.lite.Interpreter(model_path="models/tflite_tf2_16.tflite") #interpreter= tf.lite.Interpreter(model_path="models/tflite_tf2_32.tflite") interpreter.allocate_tensors() input_details=interpreter.get_input_details() output_details=interpreter.get_output_details() #print(input_details) #print(output_details) y_lebal=y_test_onehot.argmax(1) # 量化模型推理 pre = [] # tflite针对安卓端运用,每次只能推理一个数据 for i in range (len(x_test)): # .astpe 非常重要,找了很久才解决这个bug x_test1 = x_test[i].reshape(1,28,28,1).astype(np.float32) interpreter.set_tensor(input_details[0]['index'],x_test1) interpreter.invoke() output_data = interpreter.get_tensor(output_details[0]['index']) pre.append(output_data.argmax()) temp = (pre==y_lebal) acc=sum(temp)/10000 print(acc) 32bit 16bit 8bit 默认 98.500 98.500 98.440 98.440 这结果似乎和权重量化的差不多。数据流量化确实不太容易,需要更加底层、细粒度的操作。只有rram的训练有可能是4bit和8bit的混合,所以低精度数据流的训练还是挺重要的。 后期计划,做低比特、数据流量化的调用,看看别人的工作,为rram高性能计算打下理论基础。 github 代码 CNN训练MNIST数据集tenflow2(中) https://icfgblog.com/index.php/NN/166.html 2021-03-26T08:38:00+08:00 紧接上文(上),我们已经做好了训练,分别保存了32bit、16bit、8bit的量化模型。本篇的工作是加载模型,查看增加噪声、权重量化时的精度变化。数据流量化的结果将在下篇介绍。 一、加载数据集 # 1. 加载数据集 import tensorflow as tf import numpy as np minst = tf.keras.datasets.mnist img_rows,img_cols = 28,28 (x_train, y_train), (x_test, y_test) = minst.load_data() x_train = x_train.reshape(x_train.shape[0],img_rows,img_cols,1) x_test = x_test.reshape(x_test.shape[0],img_rows,img_cols,1) x_train = x_train.astype('float32') x_test = x_test.astype('float32') x_train = x_train / 255 x_test = x_test / 255 y_train_onehot = tf.keras.utils.to_categorical(y_train) y_test_onehot = tf.keras.utils.to_categorical(y_test) 二、加载模型和权重,测试准确率 # 2. 加载模型和权重,测试准确率 model = tf.keras.Sequential() model = tf.keras.models.load_model('models/mnist_tf2_fw.h5') score = model.evaluate(x_test, y_test_onehot, verbose=0) print('Test accuracy:', "{:.5f}".format(score[1])) 三、增加噪声,测试准确率 model = tf.keras.models.load_model('models/mnist_tf2_fw.h5') # 这里0,2,5层分别表示conv1,conv2,fc层的权重 for i in [0,2,5]: # 通过get_weights 和setweights修改权重 temp = model.layers[i].get_weights()[0] # 0.15表征噪声的标准差 temp = temp + np.random.rand(*temp.shape)*0.15 model.layers[i].set_weights([temp]) score = model.evaluate(x_test, y_test_onehot, verbose=0) print('Test accuracy:', "{:.5f}".format(score[1])) 四、权重量化,测试准确率 # 8bit量化 bit = 8 model = tf.keras.models.load_model('models/mnist_tf2_fw.h5') # 这里0,2,5层分别表示conv1,conv2,fc层的权重 for i in [0,2,5]: temp = model.layers[i].get_weights()[0] temp = np.rint(temp * np.power(2,bit))/np.power(2,bit) model.layers[i].set_weights([temp]) score = model.evaluate(x_test, y_test_onehot, verbose=0) print('Test accuracy:', "{:.5f}".format(score[1])) 统计结果如下: 噪声标准差 0 0.04 0.06 0.08 0.09 0.10 0.16 准确率 98.500 96.870 92.590 87.510 78.590 70.180 52.800 量化精度 32bit 16bit 8bit 4bit 3bit 2bit 1bit 准确率 98.500 98.500 98.490 98.410 97.380 89.400 66.250 特别说明的是:这里只是对权重做量化,推理计算过程中的数据还是32bit的浮点数,所以结果上看,3bit量化的精度下降不多,只有约1%。 有意思的一点是:3bit量化的准确率和0.04噪声接近;2bit量化准确率和0.08噪声相近,1bit量化的准确率和0.16的噪声相当。 分析原因:3bit量化相当于最小精度是0.125,可以将其等效为$3\sigma$的偏差,求得标准差为0.041,即为0.04的噪声。 github 代码 CNN训练MNIST数据集tenflow2(上) https://icfgblog.com/index.php/NN/165.html 2021-03-25T22:06:00+08:00 背景 还是那片nature的文章Fully hardware-implemented memristor convolutional neural network。忆阻器的精度很低,虽然外围的ADC是8bit,但忆阻器实际的精度有4bit就不错了。希望先用32bit浮点数训练网络,然后做量化,比较准确率的下降情况: 搭建网络 增加噪声 量化权重 量化数据流 测试的环境如下: python 3.7.4, tensorflow 2.0.0 一、 搭建网络 1.数据集准备 import tensorflow as tf import numpy as np # 1.数据集准备 minst = tf.keras.datasets.mnist img_rows,img_cols = 28,28 (x_train, y_train), (x_test, y_test) = minst.load_data() # 卷积运算需要将原来的(6000,28,28)变成(6000,28,28,1) ,即增加通道的参数 x_train = x_train.reshape(x_train.shape[0],img_rows,img_cols,1) x_test = x_test.reshape(x_test.shape[0],img_rows,img_cols,1) # 需要将原来的0-255的整数转为32bit单精度浮点数,再归一化为[0,1] x_train = x_train.astype('float32') x_test = x_test.astype('float32') x_train = x_train / 255 x_test = x_test / 255 # 标签为独热码的形式,方便交叉熵运算 y_train_onehot = tf.keras.utils.to_categorical(y_train) y_test_onehot = tf.keras.utils.to_categorical(y_test) 2. 模型搭建 keras搭建模型很方便: # 2. 模型搭建 model = tf.keras.Sequential() # 注意这里padding的类型“valid”,表示不填充外围,输出shape为(26,26,8)。并且没有设置bias,因为忆阻器硬件里不好添加 model.add(tf.keras.layers.Conv2D(8, kernel_size=(3, 3),padding="valid", activation='relu',use_bias=False,input_shape=(28, 28, 1))) # 同样注意这里的padding类型“sample”,表示除不尽须填充而不是舍去,输出shape为(9,9,8) model.add(tf.keras.layers.MaxPooling2D(pool_size=(3, 3),padding="same")) # 同理,注意padding,输出shape=(7,7,12) model.add(tf.keras.layers.Conv2D(12, kernel_size=(3, 3),padding="valid", activation='relu',use_bias=False,)) # 同理,注意padding,输出shape=(4,4,12) model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2),padding="same")) model.add(tf.keras.layers.Flatten()) model.add(tf.keras.layers.Dense(10, use_bias=False,activation='softmax')) model.summary() 三、开始训练 开始训练,数据集很小,cpu训练一两分钟左右就能达到98%准确率 model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy']) from time import time startTime = time() history = model.fit(x_train, y_train_onehot, batch_size = 100, epochs = 10, verbose=2, validation_data = (x_test, y_test_onehot)) duration = time()-startTime print("Train Finished! takes:","{:.2f}".format(duration)) score = model.evaluate(x_test, y_test_onehot, verbose=0) print('Test accuracy:', score[1]) 四、模型的保存和加载 如果还要后续操作,模型的保存和加载还是相当重要的。 # 1.保存框架,权重,优化器,损失函数 model.save('models/mnist_tf2_fw.h5') # 2.保存框架,不含权重和配置 yaml_tring = model.to_yaml() with open('./models/mnist_tf2_f.yaml','w') as model_file: model_file.write(yaml_tring) # 3.保存权重,不含框架和配置 model.save_weights('models/mnist_tf2_w.h5') 保存量化后的tflite格式,找了好久才搞定的!tflite格式主要是为了安卓端推理应用推出的: #1 未量化-32bit converter = tf.lite.TFLiteConverter.from_keras_model(model) tflite_model = converter.convert() open("models/tflite_model_32.tflite","wb").write(tflite_model) # 2.float16量化 converter = tf.lite.TFLiteConverter.from_keras_model(model) converter.optimizations = [tf.lite.Optimize.DEFAULT] converter.target_spec.supported_types = [tf.float16] tflite_model = converter.convert() open("models/tflite_model_16.tflite", "wb").write(tflite_model) # 3.8bit量化 def representative_data_gen(): # *这里需要注意:函数需要获得x_train的shape信息 data = tf.data.Dataset.from_tensor_slices(x_train).batch(1).take(100) for input_value in data: yield [input_value] converter = tf.lite.TFLiteConverter.from_keras_model(model) converter.optimizations = [tf.lite.Optimize.DEFAULT] converter.representative_dataset = representative_data_gen # Restricting supported target op specification to INT8 converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8] # Set the input and output tensors to uint8 converter.inference_input_type = tf.uint8 converter.inference_output_type = tf.uint8 # Convert and Save the model tflite_model = converter.convert() open("models/tflite_model_8.tflite", "wb").write(tflite_model) # 4.动态量化 converter = tf.lite.TFLiteConverter.from_keras_model(model) converter.optimizations = [tf.lite.Optimize.DEFAULT] tflite_model = converter.convert() open("models/tflite_model_dy.tflite", "wb").write(tflite_model) 五、小结 截止目前,我们已经做好了训练,分别保存了32bit、16bit、8bit的量化模型。接下来的工作是加载模型,查看增加噪声、量化后对精度的损失。 github 代码