AIfES®-Express Tutorial (feedforward neural Network-float32)

Introdution

In this tutorial we present AIfES®-Express (AIfES®-E), a simplified API that is already included in the AIfES® library. In the AIfES® standard version the user has all setting options and can describe each layer of his neural network in detail. This means that a relatively large number of lines of code must be created and is intended more for AI experts. AIfES®-E tries to make the inference and the training as simple as possible. The code becomes much more compact and easier to understand. AIfES®-E are functions that do the model building for you and also allocate the memory directly. Only the important parameters have to be passed and the rest is done by the functions. This tutorial is for a feedforward neural network (FNN) with floatingpoint (float32) weights. After the AIfES® update, quantization of the weights is also possible. There will be a separate tutorial for this.

Advantages and disadvantages of AIfES®-E:

• + Very easy to use

• + Clear and only a few lines of code

• + Error handling

• + Fast switching between different FNNs

• - For each inference the model must be built and compiled (slightly slower)

• - More memory is needed because every possible layer must be present

We will show you the following points in this tutorial:

• How to create a FNN

• How to train it

• How do I perform the inference

• How to perform an inference with an already trained FNN

If you want to know how AIfES® works by default and how the basic structure is, have a look at our tutorial.

More infos about AIfES®:

• www.aifes.ai

• AIfES for Arduino - GitHub

This tutorial is for version 2.1.0 or higher!

Install AIfES® in the Arduino IDE

To follow up the examples you have to download and install AIfES® (search for aifes) with the Arduino library manager. (Version 2.1.0 or higher)

Hardware

AIfES® should run on any Arduino or Arduino compatible board. From the UNO to the Portenta H7. With very large FNNs, you may not have enough memory. Of course this depends on the used microcontroller.

The XOR example

We use the XOR problem as an example. More details about the XOR Problem can be found in this article. An XOR gate is replicated and trained with an ANN. The XOR truth table is shown in the image below.

The used ANN structure for the replication is shown in the picture below. The sigmoid function is used as the activation function in the entire ANN.

Create the FNN

Describe the network structure:

We start by describing the structure of the FNN. To describe the number of neurons for each layer we use an uint32_t array. The size of the array corresponds to the total number of layers including the input layer. For example, in our XOR example we have 2 input neurons, a hidden layer with 3 neurons and 1 output neuron. The description in AIfES®-Express would look like this:

#define FNN_3_LAYERS 3
uint32_t FNN_structure[FNN_3_LAYERS] = {2,3,1};

Next, we select the desired activation function for each layer. For this purpose, an enumeration with the name AIFES_E_activations was created. Again an array is created. Since the input layer has no activation function, the array size is reduced by one. Different activation functions were implemented but in our example a sigmoid activation was chosen for both layers.

#define FNN_3_LAYERS 3
uint32_t FNN_structure[FNN_3_LAYERS] = {2,3,1};
//
// select the activation functions for the dense layer
AIFES_E_activations FNN_activations[FNN_3_LAYERS - 1];
FNN_activations[0] = AIfES_E_sigmoid; //Sigmoid for hidden (dense) layer
FNN_activations[1] = AIfES_E_sigmoid; //Sigmoid for output (dense) layer
//
/* possible activation functions
AIfES_E_relu
AIfES_E_sigmoid
AIfES_E_softmax
AIfES_E_leaky_relu
AIfES_E_elu
AIfES_E_tanh
AIfES_E_softsign
AIfES_E_linear
*/

That's it. This describes the network structure.

Weights arrangement:

In AIfES® there are two methods to bring the weights into the FNN (a detailed description can be found here):

• LayeredWeights: The weights are transferred layer wise

• FlatWeights: All weights are passed in one array

AIfES®-E uses only the FlatWeights method. It is easier to store the weights and it is easier to pass them.

If you already have a trained FNN, you will of course have to pass the weights in the correct order. Many AI frameworks already have a coordinated order here and AIfES® has this as well.

The following figure shows the arrangement of the weights and the bias weights. The weights of the hidden layer are named >Wh< and the bias weights are named >Bh<. The output layer has the identifiers >Wout< and >Bout<. The sigmoid activation function was chosen for the entire FNN.

Calculate the required number of weights:

If you want to train the FNN with AIfES®-E from scratch you have to know the number of weights your FNN needs. For this purpose, there is already a predefined function that does the calculation for you. You only have to pass the already created array with the FNN structure and the total number of layers. Once the number of weights required is known, a suitable float array can be created to store the weights. If there is already a trained weight set, you can check the number of weights.

uint32_t weight_number = AIFES_E_flat_weights_number_fnn_f32(FNN_structure,FNN_3_LAYERS);
float FlatWeights[weight_number];

Pack your FNN:

To make it easier for you to use your FNN, we put all components together in one struct. The AIFES_E_model_parameter_fnn_f32 struct was developed for this purpose. Here the total number of layers, the layer structure, the activation functions and the weights are passed.

AIFES_E_model_parameter_fnn_f32 FNN;
FNN.layer_count = FNN_3_LAYERS;
FNN.fnn_structure = FNN_structure;
FNN.fnn_activations = FNN_activations;
FNN.flat_weights = FlatWeights;

Full code as summary:

#define FNN_3_LAYERS 3
uint32_t FNN_structure[FNN_3_LAYERS] = {2,3,1};
AIFES_E_activations FNN_activations[FNN_3_LAYERS - 1];
FNN_activations[0] = AIfES_E_sigmoid;
FNN_activations[1] = AIfES_E_sigmoid;
uint32_t weight_number = AIFES_E_flat_weights_number_fnn_f32(FNN_structure,FNN_3_LAYERS);
float FlatWeights[weight_number];
AIFES_E_model_parameter_fnn_f32 FNN;
FNN.layer_count = FNN_3_LAYERS;
FNN.fnn_structure = FNN_structure;
FNN.fnn_activations = FNN_activations;
FNN.flat_weights = FlatWeights;

Tensors

In order to use your FNN you need tensors in which the data is stored or results are stored. Detailed information about the AIfES® tensors can be found here.

We have simplified the handling of the tensors with the update. For the creation you can now use macros that can also be used in the normal AIfES® version.

Since we want to train the XOR gate in the example shown here, we need a tensor for the input / training data. A tensor consists of a memory area where the data is stored and a description of the shape.

To insert the training data we declare a 2D float array in which the 4 XOR combinations are stored. You could also use a 1D array as shown in the 2nd example. But we stay with the 2D variant.

float input_data[4][2] = {
{0.0f, 0.0f},
{0.0f, 1.0f},
{1.0f, 0.0f},
{1.0f, 1.0f}
};
// 1D
float input_data[] = {0.0f, 0.0f, 0.0f, 1.0f, 1.0f, 0.0f, 1.0f, 1.0f};

We have 4 data sets and our FNN has 2 inputs. This corresponds to the shape of our tensor. To represent this we use an uint16_t array. The first number of our array corresponds to the number of datasets (4) and the second number corresponds to the number of inputs (2).

#define DATASETS 4
uint16_t input_shape[] = {DATASETS, 2};
//Or
uint16_t input_shape[] = {DATASETS, (uint16_t)FNN_structure[0]};

To make an AIfES® tensor out of this we can use the new macro.

#define DATASETS 4
aitensor_t input_tensor = AITENSOR_2D_F32(input_shape, input_data);

That's it, the input tensor is ready.

Next we need the target tensor, where the expected results for the training are stored. Our example FNN has only one output, so only a 1D array is needed. The shape of our tensor changes accordingly. We still have 4 datasets but only one output. The order of the data must match the input data. For example, if we input a 0/0 into the FNN we expect a 0 and if we input a 0/1 we expect a 1 and so on.

#define DATASETS 4
float target_data[] = {0.0f, 1.0f, 1.0f, 0.0f};
uint16_t target_shape[] = {DATASETS, 1};
aitensor_t target_tensor = AITENSOR_2D_F32(target_shape, target_data);

The output tensor for the calculated results of the FNN has the same structure as the target tensor but does not yet have any data in the memory area.

float output_data[DATASETS];
uint16_t output_shape[] = {DATASETS, 1};
aitensor_t output_tensor = AITENSOR_2D_F32(output_shape, output_data); 

If you only want to perform inference, you would not need a target tensor.

Training

Choose the method for initializing the weights:

If you want to train your FNN from scratch you have to initialize the weights.Of course, for this to work, you still need to consider the random seed. We like to use the noise at a pin for this. This should already be done in the setup() routine of your sketch. AIfES currently offers two methods for initializing the weights. The init_uniform method dices the weights in a range of values you specify. The second initialization is the glorot_uniform where random numbers are uniformly diced within a certain range, according to Glorot et al. To make this easy to implement the AIFES_E_init_weights_parameter_fnn_f32 struct was created.

If you already have a trained FNN that you want to train further or you have your own method for weight initialization, you can use no_init.

In the example we use the init_uniform in a value range from -2 to 2. For the glorot_uniform, the min max values are not taken into account.

//In setup()
//-------------------
srand(analogRead(A5));
//-------------------
AIFES_E_init_weights_parameter_fnn_f32  FNN_INIT_WEIGHTS;
FNN_INIT_WEIGHTS.init_weights_method = AIfES_E_init_uniform;
FNN_INIT_WEIGHTS.min_init_uniform = -2
FNN_INIT_WEIGHTS.max_init_uniform = 2;
//Glorot example
AIFES_E_init_weights_parameter_fnn_f32  FNN_INIT_WEIGHTS;
FNN_INIT_WEIGHTS.init_weights_method = AIfES_E_init_glorot_uniform;

Set training parameters:

The last step is to set the training parameters. Here, the learning method, loss, learning rate, epochs, etc. are defined for the training. All training parameters are passed to the AIFES_E_training_parameter_fnn_f32 struct.

First we declare a variable for the training parameters:

AIFES_E_training_parameter_fnn_f32  FNN_TRAIN;

At the beginning we choose the optimizer. AIfES has integrated the ADAM and the SGD optimizer. For the loss, the (Mean Squared Error) MSE and the crossentropy are available.

FNN_TRAIN.optimizer = AIfES_E_adam;
/* optimizers
AIfES_E_adam
AIfES_E_sgd
//
*/
FNN_TRAIN.loss = AIfES_E_mse;
/* loss
AIfES_E_mse,
AIfES_E_crossentropy
*/

After that, the learning rate is determined, which is used in ADAM and SGD. The additional parameter sgd_momentum is only used with SGD and determines the momentum. In addition, the batch size can be selected. In our example we use a full batch.

FNN_TRAIN.learn_rate = 0.05f;
FNN_TRAIN.sgd_momentum = 0.0;
FNN_TRAIN.batch_size = DATASETS;

The next step is to specify the number of desired epochs. The print_interval can be used to specify how often the loss should be calculated and given out. In the AIfES®-E training function, the training is performed fully automatically. AIfES®-E uses a "sum over batch size" loss reduction similar to Keras. But the loss is always calculated for the entire data set.

FNN_TRAIN.epochs = 1000;
FNN_TRAIN.epochs_loss_print_interval = PRINT_INTERVAL;

For the loss output an own individual function can be created. The function must be a void with a float as passing value (void YourFunctionName(float x)). In the function you can process the loss individually. In our example below we just print the loss.

void printLoss(float loss)
{
   Serial.print(F("Loss: "));
   Serial.println(loss,5);
}

In order for AIfES®-E to use the function, the function pointer must be passed to the struct.

FNN_TRAIN.loss_print_function = printLoss

It is possible to activate early stopping and specify a target loss. The training is then automatically stopped if the target loss is reached before the maximum number of epochs has been reached. The loss is only checked in the context of the print interval and not for each epoch.

FNN_TRAIN.early_stopping = AIfES_E_early_stopping_on;
/* early_stopping
AIfES_E_early_stopping_off,
AIfES_E_early_stopping_on
*/
//Define your target loss
FNN_TRAIN.early_stopping_target_loss = 0.004;

Trainingfunction:

The actual training is done with a function call. The function creates and compiles the model. After that the training is executed. The function also has error handling that is returned. An explanation of the possible errors is available in the sketch.

int8_t error = 0;
error = AIFES_E_training_fnn_f32(&input_tensor,&target_tensor,&FNN,&FNN_TRAIN,&FNN_INIT_WEIGHTS,&output_tensor);

Inference

After the training the inference can be performed directly with one function call and error handling is also available here.

int8_t error = 0;
error = AIFES_E_inference_fnn_f32(&input_tensor,&FNN,&output_tensor);

Inference with an already trained FNN

If you already have a trained FNN the inference is very easy. You have to transfer the structure of your FNN into AIfES, import the weights, declare the appropriate tensors and run the inference. Here is the source code for an inference with the XOR example.

#define DATASETS 4
#define FNN_3_LAYERS 3
uint32_t FNN_structure[FNN_3_LAYERS] = {2,3,1};
AIFES_E_activations FNN_activations[FNN_3_LAYERS - 1];
FNN_activations[0] = AIfES_E_sigmoid;
FNN_activations[1] = AIfES_E_sigmoid;
// FlatWeights from the trained FNN
float FlatWeights[] = {-10.1164f, -8.4212f, 5.4396f, 7.297f, -7.6482f,
-9.0155f, -2.9653f,  2.3677f, -1.5968f, 12.0305f, -6.5858f, 11.9371f,
-5.4247f};
// fill the AIfES Express struct
AIFES_E_model_parameter_fnn_f32 FNN;
FNN.layer_count = FNN_3_LAYERS;
FNN.fnn_structure = FNN_structure;
FNN.fnn_activations = FNN_activations;
FNN.flat_weights = FlatWeights;
// -------------------- create the tensors -----------------
float input_data[] = {0.0f, 0.0f, 0.0f, 1.0f, 1.0f, 0.0f, 1.0f, 1.0f};
uint16_t input_shape[] = {DATASETS, (uint16_t)FNN_structure[0]};
aitensor_t input_tensor = AITENSOR_2D_F32(input_shape, input_data);
float output_data[DATASETS];
uint16_t output_shape[] = {DATASETS, (uint16_t)FNN_structure[2]};
aitensor_t output_tensor = AITENSOR_2D_F32(output_shape, output_data);   
// ------------------- do the inference -------------------
error = AIFES_E_inference_fnn_f32(&input_tensor,&FNN,&output_tensor);

Where can I find this examples?

On GitHub or in your Arduino IDE at:

Examples->AIfES for Arduino->0_Universal->2_AIfES_Express_XOR_F32

Posted by: aifes team