Mr.Tony has a company which manufactures house hold products. In the making of the house hold products, the lights in the workplace plays a crucial role. If the light's brightness is too low the workers cannot manufacture the products. If the light brightness is too high, the electricity usage will be much higher causing a loss for Tony's company.
Here there are two problems:
1)The light brightness should not be too low.
2)The light brightness should not be too high.
Tony needs to get an alert message on his mail whenever the light's brightness is too high or too low in his company's workplace in order to respond to the problem.
To solve this problem, Tony decided to use an iot circuit connected to a light intensity sensor so that he could be alerted whenever the lights reaches their threshold values.
To set some threshold values Tony uses some anomaly detection using Z-score analysis.
we shall discuss what is anomaly and Z-score analysis
ANOMALYAnomaly is something that deviates from what is standard, normal, or expected.
In other words an anomaly is an abnormality, a blip on the screen of life that doesn’t fit with the rest of the pattern.
Let us consider the following graph.
Here the red mark is known as anomaly because that point deviates from the rest of the pattern.
We can declare that red point as an anomaly by setting some threshold values around the pattern by using Z - score analysis.
Z -SCORE ANALYSISZ-score analysis is technique to detect anomalies in certain projects.
In the above graph we can point out the anomaly by setting some threshold values. (The green high threshold and red low threshold lines)
We can set those threshold values by using some formulas.
1) MEAN FORMULA
2) Z - SCORE FORMULA
3)THRESHOLDS FORMULA
In the above formulas
Now we shall build the circuit.
COMPONENTS REQUIRED1)BOLT IOT WIFI MODULE
2) LIGHT INTENSITY MONITORING SENSOR (LDR)
3)USB-A TO MICRO-USB CABLE
4)RESISTOR 10K OHM
Connect the LDR to Bolt as shown in image below. Note: There is no positive or negative for this and the 10k Ohm resistor. Also, make sure the Bolt module is not powered on while making connections. Always make it a habit to power off the circuit while making connections for your own and the circuit's safety. Double-check all connections before turning it on.
Here are the steps for making the hardware connections:
Step 1: Insert one lead of the LDR into the Bolt Module's 3v3 Pin.
Step 2: Insert other lead of the LDR into the A0 pin.
Step 3: Insert one leg of the 10k Ohm resistor into the GND pin
Step 4: Insert the other leg of the resistor also into the A0 pin.
Warning!! Make sure that at no point do the 3.3V (or even 5V) and GND pins or wires coming out of them touch each other. If you short power to Ground without a resistor even accidentally, the current drawn might be high enough to destroy the Bolt module
Thus, we are effectively measuring the voltage across the 10k Ohm Resistor and the final circuit should look like the image below:
Step 1:Complete the circuit connections correctly
Step 2: Once you are done with that, log in to your DigitalOcean droplet and create a new folder to hold the files for this topic.
Note: If you are using VirtualBox or VMWare or a Linux system, log in to the system accordingly.
Step 3: You can create a new folder and enter it using the following command.
mkdir Anomaly_Detection;
cd Anomaly_Detection;
Step 4: Create a configurations file for this project, using the following command.
nano conf.py
After the editor is open, type in the following configuration parameters
SSID = 'You can find SSID in your Twilio Dashboard'
AUTH_TOKEN = 'You can find on your Twilio Dashboard'
FROM_NUMBER = 'This is the no. generated by Twilio. You can find this on your Twilio Dashboard'
TO_NUMBER = 'This is your number. Make sure you are adding +91 in beginning'
API_KEY = 'This is your Bolt Cloud account API key'
DEVICE_ID = 'This is the ID of your Bolt device'
FRAME_SIZE = 10
MUL_FACTOR = 6
Note: You have to replace all the above value with your credentials. You can find the first four values in Twilio dashboard and the last two in Bolt Cloud dashboard.
You can set the FRAME_SIZE to 10, and the MUL_FACTOR to 3 for now.Once done, you can save the configurations file by pressing 'CTRL+x'
Step 5: Now create one more file named anomaly_detction.py, using the following command
sudo nano anomaly_detection.py
This file will contain the main code. The algorithm for the code can be broken down into the following steps:
1) Fetch the latest sensor value from the Bolt device.
2) Store the sensor value in a list, that will be used for computing z-score.
3) Compute the z-score and upper and lower threshold bounds for normal and anomalous readings.
4) Check if the sensor reading is within the range for normal readings.
5) If it is not in range, send the SMS.
6) Wait for 10 seconds.
7) Repeat from step 1.
Step 6: Here we will start writing the code. Let us start with the imports.
import conf, json, time, math, statistics
from boltiot import Sms, Bolt
The math and statistics libraries will be required for calculating the Z-score and the threshold boundaries.
The following lies code helps define a function which calculates the Z-score and the using the Z-score calculates the boundaries required for anomaly detection.
def compute_bounds(history_data,frame_size,factor):
The above line helps define a function, which takes 3 input variables: hisotry_data, frame_size and factor.
if len(history_data)<frame_size :
return None
if len(history_data)>frame_size :
del history_data[0:len(history_data)-frame_size]
The above code checks whether enough data has been accumulated to calculate the Z-score, and if there is too much data, then the code deletes the older data.
Mn=statistics.mean(history_data)
The above code calculates the mean (Mn) value of the collected data points.
Variance=0
for data in history_data :
Variance += math.pow((data-Mn),2)
This code helps to calculate the Variance of the data points. You can read more about variance here.
Zn = factor * math.sqrt(Variance / frame_size)
High_bound = history_data[frame_size-1]+Zn
Low_bound = history_data[frame_size-1]-Zn
return [High_bound,Low_bound]
Here we calculate the Z score (Zn) for the data and use it to calculate the upper and lower threshold bounds required to check if a new data point is normal or anomalous.
The next code is used to initialize the Bolt and SMS variables, which we will use to collect data and send SMS alerts. Here we also initialize an empty list with the name 'history_data' which we will use to store older data, so that we can calculate the Z-score.
mybolt = Bolt(conf.API_KEY, conf.DEVICE_ID)
sms = Sms(conf.SSID, conf.AUTH_TOKEN, conf.TO_NUMBER, conf.FROM_NUMBER)
history_data=[]
The following while loop contains the code required to run the algorithm of anomaly detection.
while True:
response = mybolt.analogRead('A0')
data = json.loads(response)
if data['success'] != 1:
print("There was an error while retriving the data.")
print("This is the error:"+data['value'])
time.sleep(10)
continue
print ("This is the value "+data['value'])
sensor_value=0
try:
sensor_value = int(data['value'])
except e:
print("There was an error while parsing the response: ",e)
continue
bound = compute_bounds(history_data,conf.FRAME_SIZE,conf.MUL_FACTOR)
if not bound:
required_data_count=conf.FRAME_SIZE-len(history_data)
print("Not enough data to compute Z-score. Need ",required_data_count," more data points")
history_data.append(int(data['value']))
time.sleep(10)
continue
try:
if sensor_value > bound[0] :
print ("The light level increased suddenly. Sending an SMS")
response = sms.send_sms("Someone turned on the lights")
print("This is the response ",response)
elif sensor_value < bound[1]:
print ("The light level decreased suddenly. Sending an SMS")
response = sms.send_sms("Someone turned off the lights")
print("This is the response ",response)
history_data.append(sensor_value);
except Exception as e:
print ("Error",e)
time.sleep(10)
Here is the full code in one frame, you can copy and paste this code into your anomaly_detection.py file.
Step 7: Save the file by pressing Ctrl+x.
Once that is done, run the anomaly detection code using the following command
python3 anomaly_detection.py
The code will initially start printing the following.
After about 100 seconds (10 seconds delay with a frame size of 10), the system will start printing the light intensity values, as per the following image.
Note: The values may change depending on the lighting conditions.
Note: If you are not getting these results, try changing the FRAME_SIZE and MUL_FACTOR values in the conf.py file.
we are using the Z-score algorithm to dynamically change the bounds at which an alert is sent.
So when you move the light source close to or away from the LDR slowly, the bounds also start changing slowly.
But when you move the light source close to or away from the LDR very fast, the bounds do not change fast enough, and the system detects an anomaly and sends an SMS alert.
This project can be used at many circumstances to detect anomalies by using a light intensity monitoring sensor.
HAVE A GOOD DAY.
Comments