Published October 1, 2021

Escape the Pandemic

What if... the challenge would be to escape the pandemic ?

213

Things used in this project

Hardware components

Amazon Web Services AWS IoT EduKit

Ultrasonic Sensor - HC-SR04 (Generic)

Seeed Studio Grove CO2 Sensor

Story

Everybody knows what an "escape room" is. You gather some friends, go to place where you will be locked up and then.. you have to find a way to escape What if.... instead of bringing the players to the game, we would bring the game to the players ? What if... the challenge would be to 'escape the pandemic" ?

This project aims to design and build a kit. The kit is used by people used to gather in a common (and usually crowded) place. Once deployed, players will monitor the place and help people to escape a potential risky situation.

There are 3 roles:

The "Registrar" : it is located at the entry. It allows visitors to register themselves, in order to be notified later.
The "Counter": its mission is to figure out how crowded the place is
The "Sniffer": its mission is to measure CO2 levels and to figure out how "closed" the space is

Data is uploaded to AWS, along with metrics (local temperature,..) Participants will get notifications as well, like "run, go way, this is not a safe place !

Let's have a look under the hood:

All items are based on the AWS Edukit. The initial configuration has been done by following the tutorials here
The "Counter": an ultrasonic sensor (HR04) is connected to port B. It detects any object up to 5m and as close as 2cm. The best is to place it an angle (90 degrees), so you can count people intercepting the beam. People count is uploaded to to AWS (using Device Shadow). If count exceeds a threshold, a notification is sent to registered visitors.
The "Sniffer": a CO2 sensor is connected to port C. Communication is based on UART protocol. Measurements are uploaded to AWS (using Device Shadow). If value exceeds a threshold, a notification is sent to registered visitors.
The "Registrar" : a QR code is displayed on the panel at the entrance. Visitor scan the QR code using their smartphone and enter a web page (not implemented for this initial prototype, due to SNS SMS sandbox limitations) where they can enter their name and phone number. Once registered, they can be notified by text message any time later if something goes wrong.

Here are some rules: (there is absolutely no science backing this, this is just a demo)

CO2 level over 1000 will trigger an immediate alarm but only if people count is > 0. CO2 level over 5000 will trigger an immediate alarm regardless of the people count
As soon as people count is > 10, masks/noMasks are taken into account. An alarm is triggered when the proportion is "more than 20% without masks"

Design notes

Measurements and user-input are captured on the device and sent to the Cloud. Then, decisions are taken (based on the data coming from all devices) and an action is triggered back to the device (if needed)

The "shadow" looks like:

{
  "state": {
    "reported": {
      "temperature": 55.99939,
      "sound": 5,
      "masksCount": 0,
      "noMasksCount": 0,
      "peopleCount": 5,
      "emergency": 0,
      "CO2": 0,
      "mustEscape": false
    }
    "desired: {
      "mustEscape": true
    }
  }
}

Temperature/ sound: not used but it was there, so I kept it, for future use
PeopleCount : each time the beam is crossed, it increases by 1
Emergency: turns into 1 when the button is pressed, 0 otherwise
CO2 : as mesured by the CO2 sensor
masks/noMasks values based on how many times the observer clicked on those choices
mustEscape: when value is set to 1 (by the rules running on the Cloud), AWS Edukit is instructed to display an alarm message, buzz and to turn the LEDs red.

Implementation (see code in attachement)

Start from the "Smart thermostat" code sample

People detection (see attached video)

Add support for GPIO based peripherals (i.e the ultrasonic sensor and the mushroom button)

Add 2 components (ultrasonic and esp-idf_lib_helpers)
Update the existing code to add the new measured values and to update the message sent to the back-end
Update the code to implement a callback (similar to the hvac_Callback) that triggers when the "mustEscape" desired state changes. Turn the LEDs an Buzzer On/Off accordingly

CO2 measurement (see attached video)

Add support for Support UART based peripherals (i.e the CO2 meter). See the library here. Add that component to your project.

See here for a tutorial on science on CO2

350~450ppm: General outdoor environment
350~1000ppm：The air is fresh and breathing smooth
1000~2000ppm：The air was stagnant and feel asleep
5000ppm：Permissible exposure limit for an 8h work day

Manual crowd check (see attached video)

Start from an image like the following one and resize the image to 320 by 240 pixels

Convert it to a True Color C array using this tool
Copy the masks.c file to your main folder and update your makefile to include it
In a C file of your application declare the image as: LV_IMG_DECLARE(my_image_name);
Load the image:

lv_obj_t * img = lv_img_create(lv_scr_act(), NULL);
lv_img_set_src(img, &masks);

See here for more information on the topic

Server-side

On the Cloud side, data is gathered from the various connected devices. Each device has a digital twin ("shadow") with the most up-to-date data.

{
  "current": {
    "state": {
      "reported": {
        "temperature": 57.40942,
        "sound": 4,
        "masksCount": 2,
        "noMasksCount": 4,
        "peopleCount": 8,
        "emergency": 0,
        "CO2": 0,
        "mustEscape": false
      }
    }
  }
}

A "Rule" is created to forward the data to AWS IoT Events as an "Input"

Then we have a model...

When there are more than 10 people in the room, we want to send an alarm to participants (Note: since we are using SNS SMS sandbox, it is limited to a number we hard coded but in a real system, people could register when then enter the monitored room)

This is our Model. There are two states (Idle/Alarm). Transition between the states is based on the "peopleCount" input parameter value

Here are the Actions triggered when we enter in "Alarm" state

And here is the notification on my smartphone

What could be improved...

People counting : a better solution should be to have 2 sensors at 1m distance to count people entering (i.e crossing beam #1 and then beam #2) and people leaving (i.e crossing beam #2 and then beam #1) the area.
Fleet management: currently, the demo is based on two devices. If we had more devices, it would be better to leverage fleet management features
A "snooze/cancel" button could make sense... Currently, the only way to get rid of an alarm is to restart the devices
Text message: a custom format would be nicer..

What went wrong...

When you follow AWS instructions, you will be asked somewhere to start an app (menuconfig) to... configure things. For unknown reasons, arrows (up/down) do not work. It took me a while to figure out you have to use "j" and "k" keys instead. Counterintuitive, isn't?
There is limited documentation on the "ports' (there are 3 ports on the AWS Edukit. Do not forget to "enable" them using menuconfig

port A is located on the yellow part is is intended to connect I2C devices
port B and C are on the black part. I used port B to connect generic GPIOs (like the HR-04 ultrasonic sensor or the mushroom button) and port C to connect UART device (the CO2 sensor)

/*
 * AWS IoT EduKit - Core2 for AWS IoT EduKit
 * Smart Thermostat v1.2.0
 * main.c
 * 
 * Copyright 2010-2015 Amazon.com, Inc. or its affiliates. All Rights Reserved.
 * Additions Copyright 2016 Espressif Systems (Shanghai) PTE LTD
 *
 * Licensed under the Apache License, Version 2.0 (the "License").
 * You may not use this file except in compliance with the License.
 * A copy of the License is located at
 *
 *  http://aws.amazon.com/apache2.0
 *
 * or in the "license" file accompanying this file. This file is distributed
 * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
 * express or implied. See the License for the specific language governing
 * permissions and limitations under the License.
 */
/**
 * @file main.c
 * @brief simple MQTT publish, subscribe, and device shadows for use with AWS IoT EduKit reference hardware.
 *
 * This example takes the parameters from the build configuration and establishes a connection to AWS IoT Core over MQTT.
 *
 * Some configuration is required. Visit https://edukit.workshop.aws
 *
 */
#include <stdio.h>
#include <stdlib.h>
#include <ctype.h>
#include <unistd.h>
#include <limits.h>
#include <string.h>
#include <math.h>

#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/semphr.h"
#include "freertos/event_groups.h"
#include "esp_log.h"

#include "aws_iot_config.h"
#include "aws_iot_log.h"
#include "aws_iot_version.h"
#include "aws_iot_mqtt_client_interface.h"
#include "aws_iot_shadow_interface.h"

#include "core2forAWS.h"

#include "wifi.h"
#include "fft.h"
#include "ui.h"

#include "mhz19b.h"

#include <ultrasonic.h>
#define MAX_DISTANCE_CM 50       // 0.5m max
#define TRIGGER_GPIO 26          // PORT B
#define MUSHROOM_PIN 36          // PORT B - yellow  (when mushroom connected)
#define ECHO_GPIO 36             // PORT B - yellow  (when ultrasonic sensor is connected)
float distance;
uint8_t peopleCount = 0; 

static const char *TAG = "MAIN";

// Number of slices to split the microphone sample into
#define AUDIO_TIME_SLICES 60

#define MAX_LENGTH_OF_UPDATE_JSON_BUFFER 220

/* CA Root certificate */
extern const uint8_t aws_root_ca_pem_start[] asm("_binary_aws_root_ca_pem_start");
extern const uint8_t aws_root_ca_pem_end[] asm("_binary_aws_root_ca_pem_end");

/* Default MQTT HOST URL is pulled from the aws_iot_config.h */
char HostAddress[255] = AWS_IOT_MQTT_HOST;

/* Default MQTT port is pulled from the aws_iot_config.h */
uint32_t port = AWS_IOT_MQTT_PORT;

/* Semaphore for sound levels */
SemaphoreHandle_t xMaxNoiseSemaphore;
/* Semaphore for people count */
SemaphoreHandle_t xPeopleCountSemaphore;

void iot_subscribe_callback_handler(AWS_IoT_Client *pClient, char *topicName, uint16_t topicNameLen,
                                    IoT_Publish_Message_Params *params, void *pData) {
    ESP_LOGI(TAG, "Subscribe callback");
    ESP_LOGI(TAG, "%.*s\t%.*s", topicNameLen, topicName, (int) params->payloadLen, (char *)params->payload);
}

void disconnect_callback_handler(AWS_IoT_Client *pClient, void *data) {
    ESP_LOGW(TAG, "MQTT Disconnect");
    ESP_LOGI(TAG, "Disconnected from AWS IoT Core...");

    IoT_Error_t rc = FAILURE;

    if(NULL == pClient) {
        return;
    }

    if(aws_iot_is_autoreconnect_enabled(pClient)) {
        ESP_LOGI(TAG, "Auto Reconnect is enabled, Reconnecting attempt will start now");
    } else {
        ESP_LOGW(TAG, "Auto Reconnect not enabled. Starting manual reconnect...");
        rc = aws_iot_mqtt_attempt_reconnect(pClient);
        if(NETWORK_RECONNECTED == rc) {
            ESP_LOGW(TAG, "Manual Reconnect Successful");
        } else {
            ESP_LOGW(TAG, "Manual Reconnect Failed - %d", rc);
        }
    }
}

static bool shadowUpdateInProgress;

void ShadowUpdateStatusCallback(const char *pThingName, ShadowActions_t action, Shadow_Ack_Status_t status,
                                const char *pReceivedJsonDocument, void *pContextData) {
    IOT_UNUSED(pThingName);
    IOT_UNUSED(action);
    IOT_UNUSED(pReceivedJsonDocument);
    IOT_UNUSED(pContextData);

    shadowUpdateInProgress = false;

    if(SHADOW_ACK_TIMEOUT == status) {
        ESP_LOGE(TAG, "Update timed out");
    } else if(SHADOW_ACK_REJECTED == status) {
        ESP_LOGE(TAG, "Update rejected");
    } else if(SHADOW_ACK_ACCEPTED == status) {
        ESP_LOGI(TAG, "Update accepted");
    }
}

float temperature = 0;
uint8_t soundBuffer = 0;
uint8_t reportedSound = 0;
uint8_t masksCount = 0;
uint8_t noMasksCount = 0;
uint8_t emergency = 0;
int16_t co2 = 0;

bool mustEscape = 0;

// helper function for working with audio data
long map(long x, long in_min, long in_max, long out_min, long out_max) {
    long divisor = (in_max - in_min);
    if(divisor == 0){
        return -1; //AVR returns -1, SAM returns 0
    }
    return (x - in_min) * (out_max - out_min) / divisor + out_min;
}

void microphone_task(void *arg) {
    static int8_t i2s_readraw_buff[1024];
    size_t bytesread;
    int16_t *buffptr;
    double data = 0;

    Microphone_Init();
    uint8_t maxSound = 0x00;
    uint8_t currentSound = 0x00;

    for (;;) {
        maxSound = 0x00;
        fft_config_t *real_fft_plan = fft_init(512, FFT_REAL, FFT_FORWARD, NULL, NULL);
        i2s_read(I2S_NUM_0, (char *)i2s_readraw_buff, 1024, &bytesread, pdMS_TO_TICKS(100));
        buffptr = (int16_t *)i2s_readraw_buff;
        for (uint16_t count_n = 0; count_n < real_fft_plan->size; count_n++) {
            real_fft_plan->input[count_n] = (float)map(buffptr[count_n], INT16_MIN, INT16_MAX, -1000, 1000);
        }
        fft_execute(real_fft_plan);

        for (uint16_t count_n = 1; count_n < AUDIO_TIME_SLICES; count_n++) {
            data = sqrt(real_fft_plan->output[2 * count_n] * real_fft_plan->output[2 * count_n] + real_fft_plan->output[2 * count_n + 1] * real_fft_plan->output[2 * count_n + 1]);
            currentSound = map(data, 0, 2000, 0, 256);
            if(currentSound > maxSound) {
                maxSound = currentSound;
            }
        }
        fft_destroy(real_fft_plan);

        // store max of sample in semaphore
        xSemaphoreTake(xMaxNoiseSemaphore, portMAX_DELAY);
        soundBuffer = maxSound;
        xSemaphoreGive(xMaxNoiseSemaphore);
    }
}

void ultrasonic_task(void *pvParameters)
{
    ultrasonic_sensor_t sensor = {
        .trigger_pin = TRIGGER_GPIO,
        .echo_pin = ECHO_GPIO
    };
    ultrasonic_init(&sensor);
    for (;;) {
        //float distance;
        esp_err_t res = ultrasonic_measure(&sensor, MAX_DISTANCE_CM, &distance);
        if (res != ESP_OK)
        {
            printf("Error %d: ", res);
            switch (res)
            {
                case ESP_ERR_ULTRASONIC_PING:
                    printf("Cannot ping (device is in invalid state)\n");
                    break;
                case ESP_ERR_ULTRASONIC_PING_TIMEOUT:
                    printf("Ping timeout (no device found)\n");
                    break;
                case ESP_ERR_ULTRASONIC_ECHO_TIMEOUT:
                    printf("Echo timeout (i.e. distance too big)\n");
                    break;
                default:
                    printf("%s\n", esp_err_to_name(res));
            }
        }
        else {
            // printf("Distance: %0.04f m\n", distance);
            if (distance > 0.05 && distance < 0.50) {
                ESP_LOGI(TAG,"Beam crossed");
                ui_display_scanned();                                
                xSemaphoreTake(xPeopleCountSemaphore, portMAX_DELAY);
                peopleCount++;
                xSemaphoreGive(xPeopleCountSemaphore);
                vTaskDelay(pdMS_TO_TICKS(500));
                ui_display_danbo();
                continue;
            }
        }
        vTaskDelay(pdMS_TO_TICKS(500));
    }
}

void alarm_task(void *pvParameters) {
    if (mustEscape) {
        ui_display_alarm();
        ESP_LOGI(TAG, "Setting side LEDs to red");
        Core2ForAWS_Sk6812_SetSideColor(SK6812_SIDE_LEFT, 0xFF0000);
        Core2ForAWS_Sk6812_SetSideColor(SK6812_SIDE_RIGHT, 0xFF0000);
        Core2ForAWS_Sk6812_Show();
        Core2ForAWS_Motor_SetStrength(80);
    }
    else
    {
        ui_init();
        ESP_LOGI(TAG, "clearing side LEDs");
        Core2ForAWS_Sk6812_Clear();
        Core2ForAWS_Sk6812_Show();
        Core2ForAWS_Motor_SetStrength(0);
    }
    vTaskDelete(NULL);
}

void escape_Callback(const char *pJsonString, uint32_t JsonStringDataLen, jsonStruct_t *pContext) {
    IOT_UNUSED(pJsonString);
    IOT_UNUSED(JsonStringDataLen);

    if(pContext != NULL) {
        ESP_LOGI(TAG, "Delta - alarm state changed to %d", *(bool *) (pContext->pData));
        xTaskCreatePinnedToCore(&alarm_task, "alarm_task", 4096, NULL, 1, NULL, 1);
    }   
}

// TODO   this code does not work 
void mushroom_read_task(){
    for(;;){
        Core2ForAWS_Port_PinMode(MUSHROOM_PIN, INPUT);
        bool isHigh = Core2ForAWS_Port_Read(MUSHROOM_PIN);       
        if (isHigh){
            emergency = 1;
            ESP_LOGI(TAG, "Mushroom pressed ");
        } else {
            emergency = 0;
        }        
        vTaskDelay(pdMS_TO_TICKS(1000));
    }
}

#define MHZ19B_TX 12
#define MHZ19B_RX 13
static void CO2_sensor_task(void *arg){
    //ESP_LOGI(TAG, "UART - Tx task start");
   
	mhz19b_dev_t dev;
	char version[6];
	uint16_t range;
	bool autocal;

	ESP_ERROR_CHECK(mhz19b_init(&dev, UART_NUM_1, PORT_C_UART_TX_PIN, PORT_C_UART_RX_PIN));  

	while (!mhz19b_detect(&dev))
	{
		ESP_LOGI(TAG, "MHZ-19B not detected, waiting...");
		vTaskDelay(1000 / portTICK_RATE_MS);
	}

	mhz19b_get_version(&dev, version);
	ESP_LOGI(TAG, "MHZ-19B firmware version: %s", version);
	ESP_LOGI(TAG, "MHZ-19B set range and autocal");

	mhz19b_set_range(&dev, MHZ19B_RANGE_5000);
	mhz19b_set_auto_calibration(&dev, false);

	mhz19b_get_range(&dev, &range);
	ESP_LOGI(TAG, "  range: %d", range);

	mhz19b_get_auto_calibration(&dev, &autocal);
	ESP_LOGI(TAG, "  autocal: %s", autocal ? "ON" : "OFF");

	while (mhz19b_is_warming_up(&dev, true))  // use smart warming up detection
	{
		//ESP_LOGI(TAG, "MHZ-19B is warming up");
		vTaskDelay(1000 / portTICK_RATE_MS);
	}

    while (1) {
		mhz19b_read_co2(&dev, &co2);
		//ESP_LOGI(TAG, "CO2: %d", co2);
		vTaskDelay(5000 / portTICK_RATE_MS);
    }    
 }

void aws_iot_task(void *param) {
    IoT_Error_t rc = FAILURE;

    char JsonDocumentBuffer[MAX_LENGTH_OF_UPDATE_JSON_BUFFER];
    size_t sizeOfJsonDocumentBuffer = sizeof(JsonDocumentBuffer) / sizeof(JsonDocumentBuffer[0]);

    jsonStruct_t temperatureHandler;
    temperatureHandler.cb = NULL;
    temperatureHandler.pKey = "temperature";
    temperatureHandler.pData = &temperature;
    temperatureHandler.type = SHADOW_JSON_FLOAT;
    temperatureHandler.dataLength = sizeof(float);

    jsonStruct_t soundHandler;
    soundHandler.cb = NULL;
    soundHandler.pKey = "sound";
    soundHandler.pData = &reportedSound;
    soundHandler.type = SHADOW_JSON_UINT8;
    soundHandler.dataLength = sizeof(uint8_t);

    jsonStruct_t maskCountHandler;
    maskCountHandler.cb = NULL;
    maskCountHandler.pKey = "masksCount";
    maskCountHandler.pData = &masksCount;
    maskCountHandler.type = SHADOW_JSON_UINT8;
    maskCountHandler.dataLength = sizeof(uint8_t);

    jsonStruct_t noMaskCountHandler;
    noMaskCountHandler.cb = NULL;
    noMaskCountHandler.pKey = "noMasksCount";
    noMaskCountHandler.pData = &noMasksCount;
    noMaskCountHandler.type = SHADOW_JSON_UINT8;
    noMaskCountHandler.dataLength = sizeof(uint8_t);

    jsonStruct_t peopleCountHandler;
    peopleCountHandler.cb = NULL;
    peopleCountHandler.pKey = "peopleCount";
    peopleCountHandler.pData = &peopleCount;
    peopleCountHandler.type = SHADOW_JSON_UINT8;
    peopleCountHandler.dataLength = sizeof(uint8_t);

    jsonStruct_t mushroomHandler;
    mushroomHandler.cb = NULL;
    mushroomHandler.pKey = "emergency";
    mushroomHandler.pData = &emergency;
    mushroomHandler.type = SHADOW_JSON_UINT8;
    mushroomHandler.dataLength = sizeof(uint8_t);

    jsonStruct_t co2Handler;
    co2Handler.cb = NULL;
    co2Handler.pKey = "CO2";
    co2Handler.pData = &co2;
    co2Handler.type = SHADOW_JSON_UINT8;
    co2Handler.dataLength = sizeof(uint8_t);

    jsonStruct_t mustEscapeActuator;
    mustEscapeActuator.cb = escape_Callback;
    mustEscapeActuator.pKey = "mustEscape";
    mustEscapeActuator.pData = &mustEscape;
    mustEscapeActuator.type = SHADOW_JSON_BOOL;
    mustEscapeActuator.dataLength = sizeof(bool);

    ESP_LOGI(TAG, "AWS IoT SDK Version %d.%d.%d-%s", VERSION_MAJOR, VERSION_MINOR, VERSION_PATCH, VERSION_TAG);

    // initialize the mqtt client
    AWS_IoT_Client iotCoreClient;

    ShadowInitParameters_t sp = ShadowInitParametersDefault;
    sp.pHost = HostAddress;
    sp.port = port;
    sp.enableAutoReconnect = false;
    sp.disconnectHandler = disconnect_callback_handler;

    sp.pRootCA = (const char *)aws_root_ca_pem_start;
    sp.pClientCRT = "#";
    sp.pClientKey = "#0";
    
    #define CLIENT_ID_LEN (ATCA_SERIAL_NUM_SIZE * 2)
    char *client_id = malloc(CLIENT_ID_LEN + 1);
    ATCA_STATUS ret = Atecc608_GetSerialString(client_id);
    if (ret != ATCA_SUCCESS){
        ESP_LOGE(TAG, "Failed to get device serial from secure element. Error: %i", ret);
        abort();
    }

    ESP_LOGI(TAG, "Device client Id: %s", client_id);

    /* Wait for WiFI to show as connected */
    xEventGroupWaitBits(wifi_event_group, CONNECTED_BIT,
                        false, true, portMAX_DELAY);

    ESP_LOGI(TAG, "Shadow Init");

    rc = aws_iot_shadow_init(&iotCoreClient, &sp);
    if(SUCCESS != rc) {
        ESP_LOGE(TAG, "aws_iot_shadow_init returned error %d, aborting...", rc);
        abort();
    }

    ShadowConnectParameters_t scp = ShadowConnectParametersDefault;
    scp.pMyThingName = client_id;
    scp.pMqttClientId = client_id;
    scp.mqttClientIdLen = CLIENT_ID_LEN;

    ESP_LOGI(TAG, "Shadow Connect");
    rc = aws_iot_shadow_connect(&iotCoreClient, &scp);
    if(SUCCESS != rc) {
        ESP_LOGE(TAG, "aws_iot_shadow_connect returned error %d, aborting...", rc);
        abort();
    }
    ESP_LOGI(TAG, "Connected to AWS IoT Device Shadow service");

    // this task will perform sound capture
    xTaskCreatePinnedToCore(&microphone_task, "microphone_task", 4096, NULL, 1, NULL, 1);

    // this task will perform people detection using ultrasound
    xTaskCreatePinnedToCore(&ultrasonic_task, "ultrasonic_task", 4096, NULL, 1, NULL, 1);

    // this task will monitor the mushroom state    
    // xTaskCreatePinnedToCore(&mushroom_read_task, "mushroom_read", 1024*4, NULL, 1, NULL, 1);

    // this task will manage the comm with the CO2 sensor
    esp_err_t err = Core2ForAWS_Port_PinMode(PORT_C_UART_TX_PIN, UART);
    if (err == ESP_OK){
        Core2ForAWS_Port_C_UART_Begin(115200);   
        xTaskCreatePinnedToCore(&CO2_sensor_task, "CO2_sensor_task", 4096*2, NULL, 5, NULL, 1);
    }

    /*
     * Enable Auto Reconnect functionality. Minimum and Maximum time of Exponential backoff are set in aws_iot_config.h
     *  #AWS_IOT_MQTT_MIN_RECONNECT_WAIT_INTERVAL
     *  #AWS_IOT_MQTT_MAX_RECONNECT_WAIT_INTERVAL
     */
    rc = aws_iot_shadow_set_autoreconnect_status(&iotCoreClient, true);
    if(SUCCESS != rc) {
        ESP_LOGE(TAG, "Unable to set Auto Reconnect to true - %d, aborting...", rc);
        abort();
    }

    // register delta callback for alarm
    rc = aws_iot_shadow_register_delta(&iotCoreClient, &mustEscapeActuator);
    if(SUCCESS != rc) {
        ESP_LOGE(TAG, "Shadow Register Delta Error");
    }
 
    // loop and publish changes
    while(NETWORK_ATTEMPTING_RECONNECT == rc || NETWORK_RECONNECTED == rc || SUCCESS == rc) {
        rc = aws_iot_shadow_yield(&iotCoreClient, 200);
        if(NETWORK_ATTEMPTING_RECONNECT == rc || shadowUpdateInProgress) {
            rc = aws_iot_shadow_yield(&iotCoreClient, 1000);
            // If the client is attempting to reconnect, or already waiting on a shadow update,
            // we will skip the rest of the loop.
            continue;
        }

        // START get sensor readings
        // sample temperature, convert to fahrenheit
        MPU6886_GetTempData(&temperature);
        temperature = (temperature * 1.8)  + 32 - 50;

        // sample from soundBuffer (latest reading from microphone)
        xSemaphoreTake(xMaxNoiseSemaphore, portMAX_DELAY);
        reportedSound = soundBuffer;
        xSemaphoreGive(xMaxNoiseSemaphore);

        // get masks counts
        masksCount = ui_countMasks();
        noMasksCount = ui_countNoMasks();

        // END get sensor readings

        ESP_LOGI(TAG, "*****************************************************************************************");      
        ESP_LOGI(TAG, "On Device: temperature %f", temperature);
        ESP_LOGI(TAG, "On Device: sound %d", reportedSound);
        ESP_LOGI(TAG, "On Device: mask count %d", masksCount);
        ESP_LOGI(TAG, "On Device: no mask count %d", noMasksCount);
        ESP_LOGI(TAG, "On Device: people count %d", peopleCount); 
        ESP_LOGI(TAG, "On Device: emergency %d", emergency); 
        ESP_LOGI(TAG, "On Device: CO2 %d", co2); 

        rc = aws_iot_shadow_init_json_document(JsonDocumentBuffer, sizeOfJsonDocumentBuffer);
        if(SUCCESS == rc) {
            rc = aws_iot_shadow_add_reported(JsonDocumentBuffer, sizeOfJsonDocumentBuffer, 8, 
                                             &temperatureHandler, &soundHandler, &maskCountHandler, &noMaskCountHandler,
                                             &peopleCountHandler, &mushroomHandler, &co2Handler, &mustEscapeActuator
                                             );
            if(SUCCESS == rc) {
                rc = aws_iot_finalize_json_document(JsonDocumentBuffer, sizeOfJsonDocumentBuffer);
                if(SUCCESS == rc) {
                    ESP_LOGI(TAG, "Update Shadow: %s", JsonDocumentBuffer);
                    rc = aws_iot_shadow_update(&iotCoreClient, client_id, JsonDocumentBuffer,
                                               ShadowUpdateStatusCallback, NULL, 4, true);
                    shadowUpdateInProgress = true;
                }
            }
        }
        ESP_LOGI(TAG, "*****************************************************************************************");
        ESP_LOGI(TAG, "Stack remaining for task '%s' is %d bytes", pcTaskGetTaskName(NULL), uxTaskGetStackHighWaterMark(NULL));

        vTaskDelay(pdMS_TO_TICKS(5000));
    }

    if(SUCCESS != rc) {
        ESP_LOGE(TAG, "An error occurred in the loop %d", rc);
    }

    ESP_LOGI(TAG, "Disconnecting");
    rc = aws_iot_shadow_disconnect(&iotCoreClient);

    if(SUCCESS != rc) {
        ESP_LOGE(TAG, "Disconnect error %d", rc);
    }

    vTaskDelete(NULL);
}

void app_main()
{   
    Core2ForAWS_Init();
    Core2ForAWS_Display_SetBrightness(80);
    Core2ForAWS_LED_Enable(1);   

    xMaxNoiseSemaphore = xSemaphoreCreateMutex();
    xPeopleCountSemaphore = xSemaphoreCreateMutex();  

    ui_init();
    initialise_wifi();

    xTaskCreatePinnedToCore(&aws_iot_task, "aws_iot_task", 4096*2, NULL, 5, NULL, 1);    
}

/*
 * AWS IoT EduKit - Core2 for AWS IoT EduKit
 * Smart Thermostat v1.2.0
 * ui.c
 * 
 * Copyright (C) 2020 Amazon.com, Inc. or its affiliates.  All Rights Reserved.
 *
 * Permission is hereby granted, free of charge, to any person obtaining a copy of
 * this software and associated documentation files (the "Software"), to deal in
 * the Software without restriction, including without limitation the rights to
 * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
 * the Software, and to permit persons to whom the Software is furnished to do so,
 * subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in all
 * copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
 * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
 * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
 * IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
 * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 */

#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/semphr.h"
#include "freertos/queue.h"
#include "freertos/event_groups.h"

#include "esp_log.h"

#include "core2forAWS.h"
#include "ui.h"

#define MAX_TEXTAREA_LENGTH 256

LV_IMG_DECLARE(masks);
LV_IMG_DECLARE(alarm);
LV_IMG_DECLARE(detected);
LV_IMG_DECLARE(danbo);

static lv_obj_t *out_txtarea;
static lv_obj_t *out_txtarea2;
static lv_obj_t *wifi_label;

static u_int8_t m1 = 0;
static u_int8_t m2 = 0;
static u_int8_t m3 = 0;
static u_int8_t m4 = 0;
static u_int8_t m5 = 0;
static u_int8_t m6 = 0;

static char *TAG = "UI";

static void ui_textarea_prune(size_t new_text_length){
    const char * current_text = lv_textarea_get_text(out_txtarea);
    size_t current_text_len = strlen(current_text);
    if(current_text_len + new_text_length >= MAX_TEXTAREA_LENGTH){
        for(int i = 0; i < new_text_length; i++){
            lv_textarea_set_cursor_pos(out_txtarea, 0);
            lv_textarea_del_char_forward(out_txtarea);
        }
        lv_textarea_set_cursor_pos(out_txtarea, LV_TEXTAREA_CURSOR_LAST);
    }
}

void ui_textarea_add(char *baseTxt, char *param, size_t paramLen) {
    if( baseTxt != NULL ){
        xSemaphoreTake(xGuiSemaphore, portMAX_DELAY);
        if (param != NULL && paramLen != 0){
            size_t baseTxtLen = strlen(baseTxt);
            ui_textarea_prune(paramLen);
            size_t bufLen = baseTxtLen + paramLen;
            char buf[(int) bufLen];
            sprintf(buf, baseTxt, param);
            lv_textarea_add_text(out_txtarea, buf);
        } 
        else{
            lv_textarea_add_text(out_txtarea, baseTxt); 
        }
        xSemaphoreGive(xGuiSemaphore);
    } 
    else{
        ESP_LOGE(TAG, "Textarea baseTxt is NULL!");
    }
}

void ui_textarea_clear() {
    lv_textarea_set_cursor_pos(out_txtarea, 0);
    lv_textarea_del_char_forward(out_txtarea);
}

void ui_wifi_label_update(bool state){
    xSemaphoreTake(xGuiSemaphore, portMAX_DELAY);
    if (state == false) {
        lv_label_set_text(wifi_label, LV_SYMBOL_WIFI);
    } 
    else{
        char buffer[25];
        sprintf (buffer, "#0000ff %s #", LV_SYMBOL_WIFI);
        lv_label_set_text(wifi_label, buffer);
    }
    xSemaphoreGive(xGuiSemaphore);
}

void buzzer_task(void *param) {
    Core2ForAWS_Motor_SetStrength(100);
    ets_delay_us(100000);
    Core2ForAWS_Motor_SetStrength(0);
    vTaskDelete(NULL);
}

static void image_mask_callback(lv_obj_t * obj, lv_event_t event)
{
    uint16_t x, y;
    bool press;
    //char label_stash[200];
    switch(event) {    
        case LV_EVENT_CLICKED:
            //xSemaphoreTake(xGuiSemaphore, portMAX_DELAY);
            //printf("Clicked\n");             
            FT6336U_GetTouch(&x, &y, &press);
            //sprintf(label_stash, "Touch x: %d, y: %d, press: %d\r\n", x, y, press);
            //printf(label_stash);
            if ((x > 50) && (x < 100) & (y > 50) && (y < 100) ) m1++;
            if ((x > 125) && (x < 175) & (y > 50) && (y < 100) ) m2++;
            if ((x > 220) && (x < 250) & (y > 50) && (y < 100) ) m3++;
            if ((x > 50) && (x < 100) & (y > 125) && (y < 175)) m4++;
            if ((x > 125) && (x < 175) & (y > 125) && (y < 175) ) m5++;
            if ((x > 220) && (x < 250) & (y > 125) && (y < 175) ) m6++;
            xTaskCreatePinnedToCore(&buzzer_task, "buzzer_task", 4096*2, NULL, 5, NULL, 1);             
            //xSemaphoreGive(xGuiSemaphore);         
            break;        
    }      
}

static void image_danbo_callback(lv_obj_t * obj, lv_event_t event)
{
    ui_display_masks();
}

u_int8_t ui_countMasks() {    
    return m1;   
}

u_int8_t ui_countNoMasks() {
    return m2 + m3 + m4 + m5 + m6;
}

void ui_init() {
    xSemaphoreTake(xGuiSemaphore, portMAX_DELAY);
    lv_obj_t * img = lv_img_create(lv_scr_act(), NULL);  
    lv_obj_set_click(img, true);
    lv_obj_set_event_cb(img, image_danbo_callback); 
    lv_img_set_src(img, &danbo);    
    wifi_label = lv_label_create(lv_scr_act(), NULL);
    lv_obj_align(wifi_label,NULL,LV_ALIGN_IN_TOP_RIGHT, 0, 6);
    lv_label_set_text(wifi_label, LV_SYMBOL_WIFI);
    lv_label_set_recolor(wifi_label, true);
    /* instructions = lv_label_create(lv_scr_act(), NULL);
    lv_label_set_long_mode(instructions, LV_LABEL_LONG_SROLL_CIRC);     
    lv_obj_set_width(instructions, 230);
    lv_label_set_recolor(instructions, true);
    lv_label_set_text(instructions, "Scroll  test.. ");
    lv_obj_align(instructions, NULL, LV_ALIGN_IN_TOP_MID, 0, 5); */    
    out_txtarea = lv_textarea_create(lv_scr_act(), NULL);
    lv_obj_set_size(out_txtarea, 300, 45);
    lv_obj_align(out_txtarea, NULL, LV_ALIGN_IN_BOTTOM_MID, 0, -3);
    lv_textarea_set_max_length(out_txtarea, MAX_TEXTAREA_LENGTH);
    lv_textarea_set_text_sel(out_txtarea, false);
    lv_textarea_set_cursor_hidden(out_txtarea, true);
    lv_textarea_set_text(out_txtarea, "Hi! I'm Danbo ...");
    lv_textarea_add_text(out_txtarea, "Welcome to Escape the Pandemic !!"); 
    xSemaphoreGive(xGuiSemaphore);
}

void ui_display_alarm() {  
    xSemaphoreTake(xGuiSemaphore, portMAX_DELAY); 
    lv_obj_t * img = lv_img_create(lv_scr_act(), NULL);  
    lv_obj_set_click(img, false);
    lv_img_set_src(img, &alarm);  
    lv_obj_set_size(out_txtarea, 1, 1); // make it dissapear
    xSemaphoreGive(xGuiSemaphore);  
}

void ui_display_danbo() {  
    xSemaphoreTake(xGuiSemaphore, portMAX_DELAY); 
    lv_obj_t * img = lv_img_create(lv_scr_act(), NULL);  
    lv_obj_set_click(img, false);
    lv_img_set_src(img, &danbo);  
    lv_obj_set_size(out_txtarea, 1, 1); // make it dissapear
    xSemaphoreGive(xGuiSemaphore);  
}

void ui_display_scanned() {  
    xSemaphoreTake(xGuiSemaphore, portMAX_DELAY); 
    lv_obj_t * img = lv_img_create(lv_scr_act(), NULL);  
    lv_img_set_src(img, &detected);  
    lv_obj_set_size(out_txtarea, 1, 1); // make it dissapear
    xSemaphoreGive(xGuiSemaphore);  
}

void ui_display_masks() {       
    lv_obj_t * img = lv_img_create(lv_scr_act(), NULL);
    lv_obj_set_click(img, true);
    lv_obj_set_event_cb(img, image_mask_callback); 
    lv_img_set_src(img, &masks);  
    out_txtarea2 = lv_textarea_create(lv_scr_act(), NULL);
    lv_obj_set_size(out_txtarea2, 300, 45);
    lv_obj_align(out_txtarea2, NULL, LV_ALIGN_IN_BOTTOM_MID, 0, -3);
    lv_textarea_set_max_length(out_txtarea2, MAX_TEXTAREA_LENGTH);
    lv_textarea_set_text_sel(out_txtarea2, false);
    lv_textarea_set_cursor_hidden(out_txtarea2, true);
    lv_textarea_set_text(out_txtarea2, "Look around you and click... ");
}

Credits

Philippe Libioulle

7 projects • 49 followers

Escape the Pandemic

Things used in this project

Hardware components

Story

Design notes

Implementation (see code in attachement)

What could be improved...

What went wrong...

Code

main.c

ui.c

Credits

Philippe Libioulle

Comments

Embed the widget on your own site

Escape the Pandemic

Escape the Pandemic

Things used in this project

Hardware components

Story

Design notes

Implementation (see code in attachement)

What could be improved...

What went wrong...

Code

main.c

ui.c

Credits

Philippe Libioulle

Comments

Related channels and tags