Introducing the Smart Fridge

Creating the Smart Fridge Assistant project involves integrating hardware components with software applications. Below is a step-by-step guide to create this project:

Hardware Components Needed:

1. Raspberry Pi (or similar single-board computer)

https://www.raspberrypi.com/products/

2. Bluetooth Low Energy (BLE) peripheral sensor(s) for temperature and humidity monitoring

https://core-electronics.com.au/nano-v3-0-board.html?gad_source=1&gclid=CjwKCAjwx-CyBhAqEiwAeOcTdbsw_6o_lAprggG9Kws1kk5rbHOiRYloH3Lnb8DMO5B4gd1-JgA_ZBoC9x0QAvD_BwE

3. Raspberry Pi camera module for capturing images

https://core-electronics.com.au/raspberry-pi-camera-3.html?gad_source=1&gclid=CjwKCAjwx-CyBhAqEiwAeOcTdekX9Mrgtw0DJnqV3zMz1uYyoMoDrm_PDJvoHZ7YrDmCczERlJ3u9hoCdisQAvD_BwE

4. Servo motor for adjusting the camera angle

https://core-electronics.com.au/feetech-fs90-1-5kgcm-micro-servo-9g.html?gad_source=1&gclid=CjwKCAjwx-CyBhAqEiwAeOcTdbsWyalQmRreHG_T6Kmo3MIvyCsNqHvyLbuutjjAQHauK40vZYJbxRoCRFgQAvD_BwE

5. Internet connectivity (Wi-Fi or Ethernet) for communication

6. Power source for Raspberry Pi and peripherals

Software Components Needed:

1. Raspberry Pi OS installed on the Raspberry Pi

2. Python programming environment

3. Flask web framework for creating the web application

4. Flask-SocketIO for real-time communication between the web app and server

5. Bluepy library for BLE communication

Step-by-Step Guide:

1. Setting Up Raspberry Pi:

1. Install Raspberry Pi OS on the Raspberry Pi.

• For this you need a formatted SD card with 32GB storage ( or you can just use a used one and format it using Windows default formatter)
• Download latest Raspberry Pi imager (https://www.raspberrypi.com/software/)
• Select your device and OS and upload it on the SD card.
• Connect to a monitor, keyboard and mouse.

2. Connect the peripherals (USB camera, servo motor) to the Raspberry Pi.

I also created a makeshift camera holder out of carboard to mount the camera on the server motor. as shown in the image

3. Connect the Raspberry Pi to the internet via Wi-Fi or Ethernet.

2. Install Required Software Libraries:

Pre-req: Make a virtual env in a selected folder as latest raspberry pi OS does not allow external pip installations so this is the simpler way, this can be done simply by

pip -m venv "Path to venv"and then activate using "path to venv/scripts/activate"

1. Install Python and necessary libraries like Flask, Flask-SocketIO, and Bluepy, using pip.

sudo apt update

sudo apt install python3

sudo apt install python3-pip

pip3 install Flask

pip3 install flask-socketio

sudo apt install libglib2.0-dev

sudo pip3 install bluepy

2. Ensure the required dependencies for the Bluepy library are installed on the Raspberry Pi.

To install the required dependencies for the Bluepy library on a Raspberry Pi, you can follow these steps:

2.1. Update Package Lists:

First, ensure that the package lists for the system's package manager (apt) are up-to-date. Open a terminal window on the Raspberry Pi and run the following command:

```

sudo apt update

```

2.2. Install Required Packages:

Bluepy requires some dependencies to be installed on the Raspberry Pi before it can be used. Install these dependencies by running the following command:

```

sudo apt install libglib2.0-dev

```

This command installs the `libglib2.0-dev` package, which provides the GLib development files necessary for Bluepy.

2.3. Install Bluepy:

After installing the dependencies, you can install Bluepy itself. Bluepy can be installed using pip, the Python package manager. Run the following command to install Bluepy:

```

sudo pip install bluepy

```

This command installs the Bluepy library and its dependencies from the Python Package Index (PyPI).

2.4. Verify Installation:

Once the installation is complete, you can verify that Bluepy is installed correctly by importing it in a Python script or in a Python interpreter session. Open a Python interpreter session by typing `python` in the terminal, and then enter the following commands:

```

import bluepy

```

If no errors occur and the prompt returns without any messages, Bluepy is installed successfully.

By following these steps, you can ensure that the required dependencies for the Bluepy library are installed on the Raspberry Pi, allowing you to use Bluepy for Bluetooth Low Energy communication.

3. Develop the Python Scripts (Reference in GitHub link: https://github.com/Daiyan-Khan/SmartFridgeV2 ):

1. Write Python scripts to interact with the peripherals:

• Script for capturing images using the USB camera. Use Picamera2 module and Image module.
• Script for controlling the servo motor to adjust the camera angle. Use Rpi.GPIO
• Script for interacting with the BLE peripheral sensors to read temperature and humidity data. Use Adafruit DHT library.

4.Connecting DHT11 Sensors to Arduino and Arduino Code

• Components Needed:

- Arduino board (e.g., Arduino Uno)

- 2 DHT11 temperature and humidity sensors

- Breadboard and jumper wires

• Step-by-Step Guide to Connect DHT11 Sensor to Arduino:

1. Hardware Connections:

- DHT11 Pin Configuration:

- Pin 1: VCC (5V)

- Pin 2: Data (Signal)

- Pin 3: Not connected (NC)

- Pin 4: Ground (GND)

- Connecting DHT11 to Arduino:

1. Connect DHT11 pin 1 (VCC) to the 5V pin on the Arduino.

2. Connect DHT11 pin 4 (GND) to the GND pin on the Arduino.

3. Connect DHT11 pin 2 (Data) to a digital pin on the Arduino (e.g., digital pin 2).

(Source: https://www.google.com/url?sa=i&url=https%3A%2F%2Fnewbiely.com%2Ftutorials%2Farduino-nano%2Farduino-nano-dht11&psig=AOvVaw0pwgy5IsU2hbNgrY18ww7b&ust=1717214932816000&source=images&cd=vfe&opi=89978449&ved=0CBIQjRxqFwoTCIjRo6SCt4YDFQAAAAAdAAAAABAE)

Connect the 2nd DHT sensor with same VCC and GND pin but use different Digital Pin connection (For E.g. digital pin 3)

2. Install DHT and BLE Library:

- Open the Arduino IDE.

- Go to `Sketch` -> `Include Library` -> `Manage Libraries`.

- In the Library Manager, search for `DHT` and install the "DHT sensor library" by Adafruit.

- Similarly download Arduino BLE library.

3. Arduino Code (in GitHub Repository): Connectivity with Bluetooth and DHT sensor data reading

4. Uploading Code to Arduino:

- Connect your Arduino to your computer using a USB cable.

- Select your Arduino board and port in the Arduino IDE: `Tools` -> `Board` -> select your Arduino model (e.g., Arduino Uno).

- Select the correct COM port: `Tools` -> `Port` -> select the COM port that your Arduino is connected to.

- Click the upload button (right arrow) in the Arduino IDE to upload the code to the Arduino.

5. Testing the Sensor:

- Open the Serial Monitor in the Arduino IDE (`Tools` -> `Serial Monitor`) to see the temperature and humidity readings.

- Ensure the baud rate in the Serial Monitor matches the baud rate set in the `Serial.begin()` function in your code (9600 in this case).

By following these steps, you can successfully connect a DHT11 sensor to an Arduino and read temperature and humidity data. The Arduino code provided handles the initialization of the sensor, reading data, and outputting the results to the Serial Monitor.

NOTE: You will need to check Mac Address of your arduino to connect with raspberry pi.

To get the MAC address of an Arduino from a Raspberry Pi, you can follow these steps:

1. Establish a Bluetooth Connection:

- Make sure that the Arduino and Raspberry Pi are paired and connected via Bluetooth.

2. Scan for Bluetooth Devices:

- On the Raspberry Pi, use a Bluetooth scanning tool to discover nearby Bluetooth devices, including the Arduino.

- You can use the `hcitool` command-line tool to scan for Bluetooth devices. Run the following command:

sudo hcitool scan

- This command will list all Bluetooth devices in range along with their MAC addresses.

3. Identify Arduino MAC Address:

- Look for the entry corresponding to your Arduino device in the list of scanned devices.

- The MAC address (also known as the Bluetooth address) of the Arduino will be listed next to its name.

4. Record the MAC Address:

- Note down the MAC address of the Arduino for future reference.

- You will need this MAC address to establish Bluetooth connections or communicate with the Arduino from the Raspberry Pi.

By following these steps, you can discover and retrieve the MAC address of the Arduino from the Raspberry Pi using Bluetooth scanning tools. Make sure that the Arduino is powered on and in Bluetooth discoverable mode for it to appear in the list of scanned devices.

4. Create the Flask Web Application:

1. Initialize a Flask web application.

2. Create routes for different functionalities such as capturing images, refreshing sensor data, and controlling the servo motor.

3. Implement socket.io integration for real-time communication between the server and the web app.

5. Implement Fault Tolerance Mechanisms:

1. Add error handling in Python scripts to handle exceptions gracefully, especially during BLE communication and peripheral disconnections.

2. Implement retry mechanisms for reconnecting to the BLE peripheral in case of connection failures.

3. Implement alert system in app for Bluetooth connectivity issues.

6. Design User Interface:

1. Create HTML templates for the web application to display images, sensor data, and control buttons.

2. Use CSS for styling the web interface to make it user-friendly and visually appealing.

7. Testing and Debugging:

1. Test each component individually to ensure they work as expected.

2. Debug any issues encountered during testing, such as connectivity issues, sensor data accuracy, or camera angle adjustments.

3. Test using hardcoded values to check alert systems.

8. Integration and Final Testing:

1. Integrate all components together and test the complete system.

2. Verify fault tolerance mechanisms by simulating various failure scenarios such as BLE disconnection or sensor data retrieval failures.

3. Ensure the web application provides real-time updates and responds correctly to user inputs.

9. Deployment:

1. Deploy the Flask web application on the Raspberry Pi.

2. Ensure the system is accessible over the network and can be used remotely.

3. Monitor the system for any issues and perform periodic maintenance as needed.

By following these steps, you can create a Smart Fridge Assistant project that monitors temperature and humidity, captures images, and provides remote control capabilities via a web interface. This is a project submitted to Deakin, School of Information Technology, 210 Embedded Systems Development.