Published September 4, 2021 © MIT

Codecraft: Graphical Programming for Embedded ML

Seeed Studio's Codecraft now gets powered by Edge Impulse leading to the development of next-generation of developing embedded ML projects!

BeginnerProtip1 hour1,023

Codecraft: Graphical Programming for Embedded ML

Things used in this project

Hardware components

Seeed Studio Wio Terminal

Software apps and online services

Seeed Studio Codecraft

Edge Impulse Studio

Story

"Developing embedded ML prototypes is hard!"

In the recent years embedded ML has been one of the things that developers want to get their hands on. But for developing embedded ML projects have to be proficient in Python and C/C++. Not all embedded developers use Python. And not all ML developers use C++.

"Things are changing though"

With the support from tons of industry giants and startups, the TinyML world is changing. Edge Impulse is helping developers to quickly go from development to production of embedded ML prototypes. You can now develop anything you want with super user-friendly TinyML software such as Edge Impulse within minutes.

Although Edge Impulse doesn't require you to have a Ph.D. for developing TinyML models BUT it isn't easy and simple yet to be used by middle/high schoolers especially students who are learning STEAM/STEM.

"Codecraft comes to the rescue"

Seeed Studio's new TinkerGen IDE is mind-blowing. And the good thing is that it now has support for developing embedded ML prototypes right inside the web IDE. No need for using different tabs/software or even using the Edge Impulse Studio.

Let me show you a quick overview and walkthrough of the platform.

Head over to ide.tinkergen.com and sign up with your email address as shown below.

Moving on let us select the hardware that we're gonna be using for today i.e. the Seeed Studio Wio Terminal.

Select the Wio Terminal from the hardware catalog

Next up we need to install Codecraft Assistant which will communicate with the Wio Terminal and Codecraft's Web IDE.

Click on the UPLOAD option and a PROMPT will get displayed as seen here.

1 / 2 • Downloading CodecraftAssistant (Windows)

Click on the Download Device Assistant and wait until it completes downloading it. Until then have a quick glance out of your window and enjoy the fresh air!

Alright since we're done with the downloading and installation let us move to try out a simple project using CodeCraft and our Wio Terminal.

So now connect the Wio Terminal using a Type-C USB Cable and click on the upload button in the Codecraft IDE.

I am on Linux so its /dev/tty. It should be the same for Mac and COM for Windows.

Once it has successfully paired the device you see the following screen.

Roger that! We're now moving to try out some experiments with the Embedded Machine Learning feature of this IDE.

Clicking on the ModelCreation tab let us select the MotionRecognition model for this demonstration.

Now click on it and type a name for your project as shown below.

Upon clicking OK you get to see a page with Scratch-like programming as seen here.

Default data collection script.

But I have modified it a bit to look COOLER!

1 / 2 • Looking far better now! :D

So here are some key points you need to remember, button A is the last button from left, B is the middle one and C is the first one.

Follow this for a quick understanding :

And now let us collect some DATA.

Firstly let us have an understanding of the motion we need to perform for the data collection process through the following gif's.

Flip

Wave

Idle

And perform each of the steps at least 6-7 times for getting good data for training the ML model (i.e. for 30-35 seconds).

I have collected 6 samples for each type with 5 seconds in each sample as shown below.

Sample Data Collected

Next up : Model Training and Deployment

Click on the Training and Deployment tab choose the NN selection to be the default one i.e. Small. Higher NN will increase the training time and higher accuracy as well.

Let us keep the training cycles (known as Epochs) to be 20 for a faster model creation.

After all the changes it will look like this :

Model Selection

Just hit "Start training" and come back after a quick refreshment break.

This option lowers down usage of 3-4 steps that is done in EI Studio, but here you do it in just 1 click.

After 2-3 minutes :

Model training complete!

You can also check out the Raw Data tab wherein it shows the data collected in a graphical representation.

Nice...I like graphs.

Next up click on Model Training Report and VOILA!

Model Training Report

So we got an accuracy of 95.8% (pretty good for the first attempt). Plus we can even see the inferencing time and Peak RAM and FLASH MEMORY usage.

Now click on Model deployment and be sure to check that you're deploying an int8 (quantized) model.

Now you will have to click on the Programming tab and simply drag and drop according to the program I've designed as shown below.

Model Deployment becomes easy as well.

And just hit upload!

After waiting for 4-5 minutes you will see the following screen. Our device is now activated with the new program.

YAY!

Now do a wave pattern it will print out a wave and similarly for flip and idle.

So we've actually made a complete ML project in less than an hour. From model selection to data collection and even deployment all without writing a single line of code scripts.

You can do anything you want with Codecraft and Edge Impulse, tons and tons more than the examples.

That's it for now. Feel free to comment below if you have any questions.

Signing off,

Arijit

Code

Final Code

#include"TFT_eSPI.h"
#include <project_46617_inferencing.h>
#include"LIS3DHTR.h"

TFT_eSPI tft;
LIS3DHTR<TwoWire> lis;
#define CONVERT_G_TO_MS2    9.80665f
ei_impulse_result_classification_t currentClassification[EI_CLASSIFIER_LABEL_COUNT];
const char* maxConfidenceLabel;
void ei_printf(const char *format, ...) {
    static char print_buf[1024] = { 0 };

    va_list args;
    va_start(args, format);
    int r = vsnprintf(print_buf, sizeof(print_buf), format, args);
    va_end(args);

    if (r > 0) {
        Serial.write(print_buf);
    }
}

void runClassifier()
{
    float buffer[EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE] = { 0 };
    for (size_t ix = 0; ix < EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE; ix += 3) {
        // Determine the next tick (and then sleep later)
        uint64_t next_tick = micros() + (EI_CLASSIFIER_INTERVAL_MS * 1000);
        lis.getAcceleration(&buffer[ix], &buffer[ix + 1], &buffer[ix + 2]);
        buffer[ix + 0] *= CONVERT_G_TO_MS2;
        buffer[ix + 1] *= CONVERT_G_TO_MS2;
        buffer[ix + 2] *= CONVERT_G_TO_MS2;
        delayMicroseconds(next_tick - micros());
    }
    signal_t signal;
    int err = numpy:: signal_from_buffer(buffer, EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE, &signal);
    ei_impulse_result_t result = { 0 };

    err = run_classifier(&signal, &result, false);
    float maxValue = 0;
    for (size_t ix = 0; ix < EI_CLASSIFIER_LABEL_COUNT; ix++) {
        ei_impulse_result_classification_t classification_t = result.classification[ix];
        ei_printf("    %s: %.5f\n", classification_t.label, classification_t.value);
        float value = classification_t.value;
        if (value > maxValue) {
            maxValue = value;
            maxConfidenceLabel = classification_t.label;
        }
        currentClassification[ix] = classification_t;
    }
}

void setup(){
    tft.begin();
    lis.begin(Wire1);
    lis.setOutputDataRate(LIS3DHTR_DATARATE_100HZ);
    lis.setFullScaleRange(LIS3DHTR_RANGE_4G);

  tft.setRotation(3);
  tft.fillScreen(0xFD00);
  tft.setTextSize(4);
  tft.setTextColor(0x0);

}



void loop(){

  runClassifier();
  tft.drawRect(10,10,100,50,0x4FF);
  tft.fillRect(10,10,100,50,0x4FF);
  tft.drawString((String)maxConfidenceLabel, 10, 10);
  Serial.println(maxConfidenceLabel);

}

Credits

Arijit Das

10 projects • 60 followers

16 y/o • Community Expert @Edge Impulse • Co-Organizer @tinyML India • Speaker @GitHub, @Docker, @Linaro

Thanks to Seeed Studio.

Codecraft: Graphical Programming for Embedded ML

Things used in this project

Hardware components

Software apps and online services

Story

"Developing embedded ML prototypes is hard!"

"Things are changing though"

"Codecraft comes to the rescue"

Schematics

Final Outcome

Code

Final Code

Credits

Arijit Das

Comments

Embed the widget on your own site

Codecraft: Graphical Programming for Embedded ML

Codecraft: Graphical Programming for Embedded ML

Things used in this project

Hardware components

Software apps and online services

Story

"Developing embedded ML prototypes is hard!"

"Things are changing though"

"Codecraft comes to the rescue"

Schematics

Final Outcome

Code

Final Code

Credits

Arijit Das

Comments

Related channels and tags