Azure Powered AI Freezer Monitor

The Azure Powered AI Freezer Monitor is a maker hardware based IoT monitor that uses machine learning, or ML, to provide early warnings of potential equipment failure. This guide covers building the device, collecting training data, setting up email alerts, training a custom auto encoder machine learning model, and deploying the model to an ESP32 development board.

The project is designed to be functional for low temperature scientific freezers (-60 C), with the goal of reducing catastrophic failure and the need to keep backup freezers running full-time. However, please note that this project is primarily for demonstrative and educational purposes and has not gone through extensive testing.

This project takes about one hour to fully complete.

Azure Setup

There are details below about the cost and architecture of this sample, but if you just want to get it running right away here are the steps to get started.

Deploy Resources

1. Log in to your Azure account

2. Click the Deploy to Azure link above to provision all the resources for this project

As an alternative you can deploy the template by using the Deploy Custom Template service in the Azure portal, and selecting Build your own template in the editor and uploading the azuredeploy.json file from this repo.

3. Create a new resource group for the project

4. Select a region for the your resources, choose one that near you for the best performance

Note: Some resources aren't available in all regions

5. Provide unique names for all the resources

Note: Some resources require globally unique names

Setup Azure Functions

1. Once your deployment is complete use the left navigation to open you new functions app

2. Select Functions from the left navigation

3. Select Add in the top left

4. Select the following options in the window:

Development environment: Develop in portal

Template: Timer trigger

New Function: dataSaver

you can leave any other settings as they are

5. Once the function is created select Code + Test from the left navigation

6. In run.csx, replace all the existing code with:

// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.

#r "Newtonsoft.Json"

using System;
using Newtonsoft.Json;

public static void Run(string myIoTHubMessage, ICollector<outTable> outputTable, ILogger log)
{
    log.LogInformation($"C# IoT Hub trigger function processed a message: {myIoTHubMessage}");
    dynamic input = JsonConvert.DeserializeObject(myIoTHubMessage);
    Guid guid = Guid.NewGuid();
    log.LogInformation($"Message guid: {guid}");
    outputTable.Add(
            new outTable() { 
                PartitionKey = "test", 
                RowKey = guid.ToString(), 
                deviceId = input.deviceId.ToString(),
                temperature = input.Temperature}
            );
}

public class outTable
{
    public string PartitionKey { get; set; }
    public string RowKey { get; set; }
    public string deviceId { get; set; }
    public float temperature {get; set;}

}

Navigate to function.json and replace all the existing code with:

{
    "bindings": [
      {
        "type": "eventHubTrigger",
        "name": "myIoTHubMessage",
        "direction": "in",
        "eventHubName": "samples-workitems",
        "connection": "ai-freezer-hub_events_IOTHUB",
        "consumerGroup": "$Default"
      },
      {
        "name": "outputTable",
        "direction": "out",
        "type": "table",
        "tableName": "tempTable",
        "connection": "AzureWebJobsStorage"
      }
    ]
  }

8. Repeat these steps for the anomaly detector function using the options below:

Development environment: Develop in portal

Template: IoT Hub (Event Hub)

New Function: anomalyDetector

you can leave any other settings as they are

run.csx:

// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.

#r "Microsoft.WindowsAzure.Storage"
#r "Newtonsoft.Json"
#r "System.Text.Json"

using Microsoft.Extensions.Logging;
using Microsoft.WindowsAzure.Storage.Table;
using System.Threading.Tasks;
using System;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using System.Text.Json;
using System.Text.Json.Serialization;

public static readonly string emailAlertUrl = Environment.GetEnvironmentVariable("EMAIL_ALERT_URL");
public static readonly HttpClient client1 = new HttpClient();

// Anomaly detection API secrets
public static readonly string subscriptionKey = Environment.GetEnvironmentVariable("ANOMALY_DETECTOR_KEY");
public static readonly string endpoint = Environment.GetEnvironmentVariable("ANOMALY_DETECTOR_ENDPOINT");

const string latestPointDetectionUrl = "/anomalydetector/v1.0/timeseries/last/detect";
public const string batchDetectionUrl = "/anomalydetector/v1.0/timeseries/entire/detect";

public static DateTimeOffset targetTime;

public static async Task Run(TimerInfo myTimer, CloudTable inputTable, ILogger log)
{
    log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
    
    // Get traget time from when to start reading the data
    targetTime = DateTime.UtcNow;
    targetTime = targetTime.AddHours(-6);
    log.LogInformation($"Target start time is: {targetTime}");

    TableQuery<DataPoint> rangeQuery = new TableQuery<DataPoint>().Where(
        TableQuery.GenerateFilterConditionForDate("Timestamp", QueryComparisons.GreaterThan, targetTime));

    // Execute the query and loop through the results
    List<DataPoint> data = new List<DataPoint>();
    foreach (DataPoint entity in 
    await inputTable.ExecuteQuerySegmentedAsync(rangeQuery, null))
    {
        data.Add(new DataPoint() {Timestamp = entity.Timestamp, temperature = entity.temperature});
    }

    // Sort data by Timestamp
    data.Sort((dp1, dp2) => DateTimeOffset.Compare(dp1.Timestamp, dp2.Timestamp));
    
    List<FormatedData> formatedData = new List<FormatedData>();
    data.ForEach( delegate(DataPoint point)
    {
        formatedData.Add(new FormatedData() { timestamp = point.Timestamp.ToString("yyyy-MM-ddTHH:mm:00Z"), value = point.temperature});
    });

    var options = new JsonSerializerOptions
    {
        IgnoreNullValues = true,
        // PropertyNamingPolicy = new LowerCaseNamingPolicy()
    };

    List<JsonFormat> jsonFormat = new List<JsonFormat>();
    jsonFormat.Add(new JsonFormat() {series = formatedData, granularity = "minutely", customInterval = 1, period = 90, sensitivity = 85});
    string dataToSend = JsonSerializer.Serialize(jsonFormat, options);

    // Call anomaly detection API
    var anomalies = detectAnomaliesBatch(dataToSend, log);

    if (anomalies != null){
        var json = JsonSerializer.Serialize(anomalies);
        var content = new StringContent(json, Encoding.UTF8, "application/json");
        var response = await client1.PostAsync(emailAlertUrl, content);
        log.LogInformation(response.ToString());
    }
}



static async Task<string> Request(string apiAddress, string endpoint, string subscriptionKey, string requestData)
{
    using (HttpClient client = new HttpClient { BaseAddress = new Uri(apiAddress) })
    {
        System.Net.ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;
        client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
        client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", subscriptionKey);

        var content = new StringContent(requestData, Encoding.UTF8, "application/json");
        var res = await client.PostAsync(endpoint, content);
        return await res.Content.ReadAsStringAsync();
    }
}

static string detectAnomaliesBatch(string requestData, ILogger log)
{
    log.LogInformation("Detecting anomalies as a batch");
   
    requestData = requestData.TrimEnd(']').TrimStart('[');

    //construct the request
    var result = Request(
        endpoint,
        batchDetectionUrl,
        subscriptionKey,
        requestData).Result;

    //deserialize the JSON object, and display it
    dynamic jsonObj = Newtonsoft.Json.JsonConvert.DeserializeObject(result);
    System.Console.WriteLine(jsonObj);

    string foundAnomalies = "Anomalies detected in the following data positions: ";

    if (jsonObj["code"] != null)
    {
        System.Console.WriteLine($"Detection failed. ErrorCode:{jsonObj["code"]}, ErrorMessage:{jsonObj["message"]}");
        
        log.LogInformation($"Detection failed. ErrorCode:{jsonObj["code"]}, ErrorMessage:{jsonObj["message"]}");
    }
    else
    {
        // log.LogInformation(result);
        //Find and display the positions of anomalies in the data set
        bool[] anomalies = jsonObj["isAnomaly"].ToObject<bool[]>();
        System.Console.WriteLine("\nAnomalies detected in the following data positions:");
        log.LogInformation("\nAnomalies detected in the following data positions:");
        for (var i = 0; i < anomalies.Length; i++)
        {
            if (anomalies[i])
            {
                System.Console.Write(i + ", ");
                log.LogInformation(i + ", ");
                foundAnomalies += i;
                foundAnomalies += ", ";
            }
        }
        if (anomalies.Any(item => item == true))
        {
            return foundAnomalies;
        }
    }
    return null;
}

public class FormatedData
{
    public string timestamp { get; set; }
    public string value { get; set; }
}

public class DataPoint : TableEntity
{
    [JsonPropertyName("value")]
    public string temperature { get; set;}
    public string timestamp { get; set; }
    
}

public class JsonFormat
{
    public List<FormatedData> series { get; set; }
    public string granularity { get; set; }
    public int customInterval { get; set; }
    public int period { get; set; }
    // public float maxAnomalyRatio { get; set; }
    public int sensitivity { get; set; }
}

public class LowerCaseNamingPolicy : JsonNamingPolicy
{
    public override string ConvertName(string name) =>
        name.ToLower();
}

function.json:

// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.

#r "Microsoft.WindowsAzure.Storage"
#r "Newtonsoft.Json"
#r "System.Text.Json"

using Microsoft.Extensions.Logging;
using Microsoft.WindowsAzure.Storage.Table;
using System.Threading.Tasks;
using System;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using System.Text.Json;
using System.Text.Json.Serialization;

public static readonly string emailAlertUrl = Environment.GetEnvironmentVariable("EMAIL_ALERT_URL");
public static readonly HttpClient client1 = new HttpClient();

// Anomaly detection API secrets
public static readonly string subscriptionKey = Environment.GetEnvironmentVariable("ANOMALY_DETECTOR_KEY");
public static readonly string endpoint = Environment.GetEnvironmentVariable("ANOMALY_DETECTOR_ENDPOINT");

const string latestPointDetectionUrl = "/anomalydetector/v1.0/timeseries/last/detect";
public const string batchDetectionUrl = "/anomalydetector/v1.0/timeseries/entire/detect";

public static DateTimeOffset targetTime;

public static async Task Run(TimerInfo myTimer, CloudTable inputTable, ILogger log)
{
    log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
    
    // Get traget time from when to start reading the data
    targetTime = DateTime.UtcNow;
    targetTime = targetTime.AddHours(-6);
    log.LogInformation($"Target start time is: {targetTime}");

    TableQuery<DataPoint> rangeQuery = new TableQuery<DataPoint>().Where(
        TableQuery.GenerateFilterConditionForDate("Timestamp", QueryComparisons.GreaterThan, targetTime));

    // Execute the query and loop through the results
    List<DataPoint> data = new List<DataPoint>();
    foreach (DataPoint entity in 
    await inputTable.ExecuteQuerySegmentedAsync(rangeQuery, null))
    {
        data.Add(new DataPoint() {Timestamp = entity.Timestamp, temperature = entity.temperature});
    }

    // Sort data by Timestamp
    data.Sort((dp1, dp2) => DateTimeOffset.Compare(dp1.Timestamp, dp2.Timestamp));
    
    List<FormatedData> formatedData = new List<FormatedData>();
    data.ForEach( delegate(DataPoint point)
    {
        formatedData.Add(new FormatedData() { timestamp = point.Timestamp.ToString("yyyy-MM-ddTHH:mm:00Z"), value = point.temperature});
    });

    var options = new JsonSerializerOptions
    {
        IgnoreNullValues = true,
        // PropertyNamingPolicy = new LowerCaseNamingPolicy()
    };

    List<JsonFormat> jsonFormat = new List<JsonFormat>();
    jsonFormat.Add(new JsonFormat() {series = formatedData, granularity = "minutely", customInterval = 1, period = 90, sensitivity = 85});
    string dataToSend = JsonSerializer.Serialize(jsonFormat, options);

    // Call anomaly detection API
    var anomalies = detectAnomaliesBatch(dataToSend, log);

    if (anomalies != null){
        var json = JsonSerializer.Serialize(anomalies);
        var content = new StringContent(json, Encoding.UTF8, "application/json");
        var response = await client1.PostAsync(emailAlertUrl, content);
        log.LogInformation(response.ToString());
    }
}



static async Task<string> Request(string apiAddress, string endpoint, string subscriptionKey, string requestData)
{
    using (HttpClient client = new HttpClient { BaseAddress = new Uri(apiAddress) })
    {
        System.Net.ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;
        client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
        client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", subscriptionKey);

        var content = new StringContent(requestData, Encoding.UTF8, "application/json");
        var res = await client.PostAsync(endpoint, content);
        return await res.Content.ReadAsStringAsync();
    }
}

static string detectAnomaliesBatch(string requestData, ILogger log)
{
    log.LogInformation("Detecting anomalies as a batch");
   
    requestData = requestData.TrimEnd(']').TrimStart('[');

    //construct the request
    var result = Request(
        endpoint,
        batchDetectionUrl,
        subscriptionKey,
        requestData).Result;

    //deserialize the JSON object, and display it
    dynamic jsonObj = Newtonsoft.Json.JsonConvert.DeserializeObject(result);
    System.Console.WriteLine(jsonObj);

    string foundAnomalies = "Anomalies detected in the following data positions: ";

    if (jsonObj["code"] != null)
    {
        System.Console.WriteLine($"Detection failed. ErrorCode:{jsonObj["code"]}, ErrorMessage:{jsonObj["message"]}");
        
        log.LogInformation($"Detection failed. ErrorCode:{jsonObj["code"]}, ErrorMessage:{jsonObj["message"]}");
    }
    else
    {
        // log.LogInformation(result);
        //Find and display the positions of anomalies in the data set
        bool[] anomalies = jsonObj["isAnomaly"].ToObject<bool[]>();
        System.Console.WriteLine("\nAnomalies detected in the following data positions:");
        log.LogInformation("\nAnomalies detected in the following data positions:");
        for (var i = 0; i < anomalies.Length; i++)
        {
            if (anomalies[i])
            {
                System.Console.Write(i + ", ");
                log.LogInformation(i + ", ");
                foundAnomalies += i;
                foundAnomalies += ", ";
            }
        }
        if (anomalies.Any(item => item == true))
        {
            return foundAnomalies;
        }
    }
    return null;
}

public class FormatedData
{
    public string timestamp { get; set; }
    public string value { get; set; }
}

public class DataPoint : TableEntity
{
    [JsonPropertyName("value")]
    public string temperature { get; set;}
    public string timestamp { get; set; }
    
}

public class JsonFormat
{
    public List<FormatedData> series { get; set; }
    public string granularity { get; set; }
    public int customInterval { get; set; }
    public int period { get; set; }
    // public float maxAnomalyRatio { get; set; }
    public int sensitivity { get; set; }
}

public class LowerCaseNamingPolicy : JsonNamingPolicy
{
    public override string ConvertName(string name) =>
        name.ToLower();
}

Configure Logic App

1. Once the deployment is complete use the left navigation to open the newly created Logic App

2. Select Logic app designer from the left navigation

3. Select + New step

4. Search for the email client you'd like to use (Office 365 Outlook, Gmail, and Outlook.com)

5. Select the Send an email action

Note: This will be a little different depending on which email client you use

6. Log in with your email account

7. Customize your message, this email will be sent any time an anomaly is detected.

Set up IoT Device

1. Next you'll need to get the connection string for you device, navigate to the IoT Hub you created earlier

2. In the left navigation select IoT devices

3. At the top left of the page select + New

4. Give the device an ID

5. Press Save at the bottom of the screen

6. Select the device you created

7. Copy the Primary Connection String you'll use this string in the next section

Device Build

1. Solder the screw terminal to the top side of your MCP9600.

2. Solder the pins to the bottom side of the MCP9600.

Tip: Put the pins in the breadboard to hold them in place when soldering.

3. Insert the ESP32 and thermocouple amplifier into the breadboard.

4. Follow the wiring diagram below to connect the thermocouple amplifier to the ESP32 using your jumper wires.

5. Connect the Thermocouple to the screw terminals on the MCP9600

The picture below use a generic ESP32 Dev board, new picture with the Adafruit Huzzah32 coming soon!

Device Code

1. If you haven't already, clone this repo to your computer

2. Open the AiFreezer folder with VS Code

3. Create a new file in this folder and name it config.h

4. Paste the code below in config.h

const char* ssid     = "<YOUR-NETWORK-NAME>";
const char* password = "<YOUR-WIFI-PASSWORD>";
static const char* connectionString = "<YOUR-CONNECTION-STRING>";

5. Fill in your network credentials

6. Paste the connection string from IoT Hub

7. Follow the first section of this [guide](https://randomnerdtutorials.com/installing-the-esp32-board-in-arduino-ide-windows-instructions/) to add the ESP32 extension to the Arduino IDE.

8. Install the library listed below using Arduino's Library manager. If you have used the library manager before [here](https://www.arduino.cc/en/guide/libraries) is a helpful guide.

9. Adafruit MCP9600

Note: If you're prompted to install other dependencies for these libraries, select Install All

10. With FreezerTempAlert.ino open in the VS Code open the command palette (CTL+SHIFT+P) and type in Arduino: Change Board Type then search Adafruit ESP32 Feather

11. Next select the the active serial port, open the command palette and type in Arduino: Select Serial Port

12. Finally upload your code to your Feather board, open the command palette and type in Arduino: Upload