Mr. Dhar owns a company which fabricates glass products. In the manufacturing of such products, the lighting conditions in the workplace plays a very crucial role. If the potency of the light is too stubby the workers cannot work properly. Similarly if the potency of the light is too lofty, it would ensure high electricity usage thereby acquiring a loss for his company.
Hence to ensure polished productivity -
- The light intensity should be balanced.
- An alert message should be sent to Mr. Dhar whenever the light intensity is unbalanced.
To solve this problem, Mr. Dhar decided to implement an Iot device so that it sets an alarm whenever abnormality in light conditions occur as well as he remains informed regarding the lighting conditions and their anomaly.
To set threshold values Mr. Dhar applied anomaly detection using Z-score analysis.
ANOMALYAnomaly is something that deviates from what is standard, normal, or expected.
In other words an anomaly is an abnormality, a blip on the screen of life that doesn’t fit with the rest of the pattern.
Let us consider the following graph -
Here the red mark is known as anomaly because that point deviates from the rest of the pattern.
We can declare that red point as an anomaly by setting some threshold values around the pattern by using Z - score analysis.
Z -SCORE ANALYSISZ-score analysis is technique to detect anomalies in certain projects.
In the above graph we can point out the anomaly by setting some threshold values.
- The green line signifies high threshold
- The red line indicates low threshold
We can set those threshold values by using below formulas -
- Mean Formula -
- Z-Score Formula -
- Threshold Formula -
In the above formulas -
- r = Frame Size
- C = Multiplication Factor
- Complete the circuit connections as per the schematics provided.
- Once connections are secured, log in to your DigitalOcean droplet and create a new folder to hold the files for this project.
Note: If you are using VirtualBox or VMWare or a Linux system, log in to the system accordingly.
- You can create a new folder using the following command -
mkdir Anomaly_Detection;
cd Anomaly_Detection;
- Create a configurations file for this project, using the following command -
nano conf.py
- After the editor opens, type in the following configuration parameters -
SSID = 'You can find SSID in your Twilio Dashboard'
AUTH_TOKEN = 'You can find on your Twilio Dashboard'
FROM_NUMBER = 'This is the no. generated by Twilio. You can find this on your Twilio Dashboard'
TO_NUMBER = 'This is your number. Make sure you are adding +91 in beginning'
API_KEY = 'This is your Bolt Cloud account API key'
DEVICE_ID = 'This is the ID of your Bolt device'
FRAME_SIZE = 10
MUL_FACTOR = 3
Note: You have to replace all the above value with your credentials. You can find the first four values in Twilio dashboard and the last two in Bolt Cloud dashboard.
You can set the FRAME_SIZE to 10, and the MUL_FACTOR to 3 for now. Once done, you can save the configurations file by pressing 'CTRL+X'
- Now create another file named anomaly_detction.py, using the following command -
sudo nano anomaly_detection.py
This file will contain the main code. The algorithm for the code can be broken down into the following steps:
- Fetch the latest sensor value from the Bolt device.
- Store the sensor value in a list, that will be used for computing z-score.
- Compute the z-score and upper and lower threshold bounds for normal and anomalous readings.
- Check if the sensor reading is within the range of normal readings.
- If it is not in range, send the SMS and trigger the buzzer as well as indicating LED.
- Wait for 10 seconds.
- Repeat.
- Now we will start writing the code. Let us start with the imports.
import conf, json, time, math, statistics
from boltiot import Sms, Bolt
The math and statistics libraries will be required for calculating the Z-score and the threshold boundaries.
The following code helps define a function which calculates the Z-score and using it further calculates the boundaries required for anomaly detection.
def compute_bounds(history_data,frame_size,factor):
The above line helps define a function, which takes 3 input variables : history_data, frame_size and factor.
if len(history_data)<frame_size :
return None
if len(history_data)>frame_size :
del history_data[0:len(history_data)-frame_size]
The above code checks whether enough data has been accumulated to calculate the Z-score, and if there is too muchdata, then the code deletes the older data.
Mn=statistics.mean(history_data)
The above code calculates the mean ( Mn ) value of the collected data points.
Variance=0
for data in history_data :
Variance += math.pow((data-Mn),2)
The above code calculates the Variance of the data points.
Zn = factor * math.sqrt(Variance / frame_size)
High_bound = history_data[frame_size-1]+Zn
Low_bound = history_data[frame_size-1]-Zn
return [High_bound,Low_bound]
Here we calculate the Z score ( Zn ) for the data which is used to calculate the upper and lower threshold bounds required to check if a new data point is normal or anomalous.
The next code is used to initialize the Bolt and SMS variables, which is used to collect data and send SMS alerts. Here we also initialize an empty list with the name 'history_data' which we will use to store older data, so that we can calculate the Z-score.
mybolt = Bolt(conf.API_KEY, conf.DEVICE_ID)
sms = Sms(conf.SSID, conf.AUTH_TOKEN, conf.TO_NUMBER, conf.FROM_NUMBER)
history_data=[]
The following while loop contains the code required to run the algorithm of anomaly detection.
while True:
response = mybolt.analogRead('A0')
data = json.loads(response)
if data['success'] != 1:
print("There was an error while retriving the data.")
print("This is the error:"+data['value'])
time.sleep(10)
continue
print ("This is the value "+data['value'])
sensor_value=0
try:
sensor_value = int(data['value'])
except e:
print("There was an error while parsing the response: ",e)
continue
bound = compute_bounds(history_data,conf.FRAME_SIZE,conf.MUL_FACTOR)
if not bound:
required_data_count=conf.FRAME_SIZE-len(history_data)
print("Not enough data to compute Z-score. Need ",required_data_count," more data points")
history_data.append(int(data['value']))
time.sleep(10)
continue
try:
if sensor_value > bound[0] :
print ("The light level increased suddenly. Sending an SMS")
response = sms.send_sms("Someone turned on the lights")
print("This is the response ",response)
response = mybolt.analogRead('0','255')
elif sensor_value < bound[1]:
print ("The light level decreased suddenly. Sending an SMS")
response = sms.send_sms("Someone turned off the lights")
print("This is the response ",response)
response = mybolt.analogRead('0','0')
history_data.append(sensor_value);
except Exception as e:
print ("Error",e)
time.sleep(10)
- Save the file by pressing 'Ctrl+X'.
Once that is done, run the anomaly detection code using the following command -
python3 anomaly_detection.py
The code will initially start printing the following -
After about 100 seconds (10 seconds delay with a frame size of 10), the system will start printing the light intensity values, as per the following -
Note: The values may change depending on the lighting conditions.
Note: If you are not getting these results, try changing the FRAME_SIZE and MUL_FACTOR values in the conf.py file.
We are using the Z-score algorithm to dynamically change the bounds at which an alert is sent.
So when you move the light source close to or away from the LDR slowly, the bounds also start changing slowly.
But when you move the light source close to or away from the LDR very fast, the bounds do not change fast enough, so the system detects an anomaly and sends an SMS alert while triggering the Buzzer as well as the indicator Led.
The SMS alert will be something like this -
The Final Output of the product as follows -
The light intensity data can visualized through Bolt Cloud as follows -
- Just Log in to Bolt Cloud -
- Go to Products tab and Create a Product -
- Configure the product and save it -
- Link the product with your Bolt device -
- Finally view this device -
Thank you.
Enjoy and share the awesomeness. All the very best.
Comments