“AI” How to Use MLP Model with only numpy. By Matrix products and Sums.

 

The deep learning libraries keras and pytorch are frequently used to build AI models.

In the case of keras, when we use the model, below code is used.

model.predict ( input data )

 

However, it is only with environment of keras(or pytorch) .

 

After building an AI model on a keras environment, it is frequently occur that I cannot use keras environment on actual adapting condition.

 

So, in this page, I introduce how to use AI model without keras environment.

The method uses only numpy.

 

*I’m not familiar with pytorch, but may be almost same method can adopt to pytorch too.

 

OK, let’s get started!!

 

About Inside Calculation of Neural Network

What is conducted in neural network?

The answer is “matrix products and sums”.

 

It is easy to do if we can get the information that is used in the matrix calculation.

 

And how to get the information that neural network has is introduced below link.

 

By using this method, I will demonstrate the calculation of matrix products and sums that is conducted in neural network.

 

Build Sample Neural Network Model by Using Keras

Since I can’t explain it without a sample neural network model, I build the sample MLP model by using keras first.

 

The data for MLP model is shown below.

output data “y” is made by below equation.

y=1.2*x1-2.1*x2+0.7*x3^2-0.5*x4^2+random number

 

And x1, x2, x3, x4 is used as input for MLP model.

This file is saved as “sample_data.csv”.

 

The MLP model is build by below code.

#Import Library
import matplotlib.pyplot as plt 
import numpy as np
import pandas as pd
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.optimizers import Adam

#Data Preparation
data_file='sample_data.csv'
data=pd.read_csv(data_file).values
Xdata=data[:,:4]
Ydata=data[:,-1]

#Model Build and Learning
model = Sequential()
model.add(Dense(20, input_dim=4))
model.add(Dense(10))
model.add(Dense(1))
model.compile(loss="mean_absolute_error", optimizer=Adam(lr=0.001))
model.summary()
validation_split_rate=0.2
history=model.fit(Xdata,Ydata,batch_size=16,epochs=100,validation_split=validation_split_rate)

 

Learned MLP model is stored in variable “model”.

 

Get Weight and Bias from MLP Model

As I introduced before, inside calculation of MLP model is just only matrix products and sums.

The value used for matrix products calculation is called “weights”.

And used for matrix sums calculation is called “bias”.

 

Since There is three layer in this MLP model, weight and bias is also three exist respectively.

And how to get these information is below.

#First Layer's Process
layer1 = model.layers[0]
L1_weights=layer1.get_weights()[0]
L1_bias=layer1.get_weights()[1]
#Second Layer's Process
layer2 = model.layers[1]
L2_weights=layer2.get_weights()[0]
L2_bias=layer2.get_weights()[1]
#Third Layer's Process
layer3 = model.layers[2]
L3_weights=layer3.get_weights()[0]
L3_bias=layer3.get_weights()[1]

 

Let’s check the size of each variables.

print('First_Weight_Shape=', L1_weights.shape)
print('First_Bias_Shape=', L1_bias.shape)
print('Second_Weight_Shape=', L2_weights.shape) 
print('Second_Bias_Shape=', L2_bias.shape)
print('Third_Weight_Shape=', L3_weights.shape) 
print('Third_Bias_Shape=', L3_bias.shape)

 

The Result is below.

 

As I introduced before, the MLP model has below layers.

model.add(Dense(20, input_dim=4))
model.add(Dense(10))
model.add(Dense(1))

It is confirmed the weight and bias shape is correspond to these model information.

You can check these method more detail from below link.

“AI” How to Get Weight and Bias of Learned MLP Model by Using Keras.

 

Anyway, preliminary preparations are now complete.

 

Matrix Calculation By Weight & Bias of Neural Network

I demonstrate how to calculate the inside calculation of MLP model by using weight & bias that we get before.

 

The sample code is below.

#List for output
Pred_mat=[]
#Calculation of matrix products and sums
for i in range(len(X_validation)):
    input_data=X_validation[i,:].reshape(1,4)
    L1P=np.dot(input_data,L1_weights)+L1_bias
    L2P=np.dot(L1P,L2_weights)+L2_bias
    L3P=np.dot(L2P,L3_weights)+L3_bias
    Pred_mat.append(L3P)

AI model calculation is likely difficult, but actually very easy as shown above.

 

Result Comparison with Keras Predict

Let’s compare the result with keras predict result.

 

Although detail of confirming is omitted, the comparison result is below.

Horizontal axis value is result of Keras.Predict.

And Vertical axis value is result from weight and bias.

Each value is completely same!!

 

So, if you save weight & bias of learned MLP model as numpy array data in keras environment and upload it another environment that cannot use keras but numpy can use, and by usind this method, you can use MLP model easily.