EESTech Challenge Final Round

Idea

The Smart Office Environment Monitoring System is designed to improve the working conditions in an office by monitoring various environmental parameters and providing actionable recommendations to optimize energy consumption and comfort. The system integrates sensors to measure atmospheric pressure, CO2 levels, light intensity, humidity, and temperature across different office areas. Using this data, along with weather forecasts, the system generates recommendations to adjust environmental controls and enhance energy efficiency.

Step-by-Step Guide on Technical Implementation

Prerequisites

Hardware Requirements

• Environmental sensors (CO2, temperature, humidity, light, pressure).
• Home Assistant setup for sensor data collection.

Software Requirements

• Python 3.x
• Flask
• Llama_cpp
• Transformers library (Hugging Face)
• Home Assistant API
• gTTS (Google Text-to-Speech)
• Numpy

Installation

Set up Home Assistant

• Install Home Assistant on a local server or Raspberry Pi.
• Connect environmental sensors to Home Assistant.
• Obtain Home Assistant access token and API details.

Install Python packages

pip install flask llama_cpp transformers homeassistant-api gtts numpy

Model and Tokenizer

• Download the fine-tuned LLaMA model and tokenizer from the specified repository.

Code Documentation

Flask App

• Importing the libraries and packages

from flask import Flask, render_template, request, send_file
from llama_cpp import Llama
from homeassistant_api import Client
import secret
from datetime import datetime, timedelta
import numpy as np
from gtts import gTTS
import os

app = Flask(__name__)

• Load the Fine-Tuned LLaMA model

LLM = Llama(model_path='llama-2-7b-chat.Q5_K_M.gguf', n_ctx=1024)

• Load model directly using transformers

from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct")
model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct")

• Home Assistant details

HA_IP_ADDR = '10.11.22.52'

SENSOR_IDS = {
"outside": {
"pressure": "sensor.psoc6_micropython_sensornode_outside_atmospheric_pressure",
"co2": "sensor.psoc6_micropython_sensornode_outside_co2_ppm",
"temperature": "sensor.psoc6_micropython_sensornode_outside_temperature"
},
"working_space_2": {
"pressure": "sensor.psoc6_micropython_sensornode_working_space_2_atmospheric_pressure",
"co2": "sensor.psoc6_micropython_sensornode_working_space_2_co2_ppm",
"light": "sensor.psoc6_micropython_sensornode_working_space_2_light",
"humidity": "sensor.psoc6_micropython_sensornode_working_space_2_relative_humidity",
"temperature": "sensor.psoc6_micropython_sensornode_working_space_2_temperature"
},
"open_space": {
"pressure": "sensor.psoc6_micropython_sensornode_open_space_atmospheric_pressure",
"co2": "sensor.psoc6_micropython_sensornode_open_space_co2_ppm",
"light": "sensor.psoc6_micropython_sensornode_open_space_light",
"humidity": "sensor.psoc6_micropython_sensornode_open_space_relative_humidity",
"temperature": "sensor.psoc6_micropython_sensornode_open_space_temperature"
},
"small_meeting_room": {
"pressure": "sensor.psoc6_micropython_sensornode_small_meeting_room_atmospheric_pressure",
"co2": "sensor.psoc6_micropython_sensornode_small_meeting_room_co2_ppm",
"temperature": "sensor.psoc6_micropython_sensornode_small_meeting_room_temperature"
},
"office_box": {
"pressure": "sensor.psoc6_micropython_sensornode_office_box_atmospheric_pressure",
"co2": "sensor.psoc6_micropython_sensornode_office_box_co2_ppm",
"light": "sensor.psoc6_micropython_sensornode_office_box_light",
"humidity": "sensor.psoc6_micropython_sensornode_office_box_relative_humidity",
"temperature": "sensor.psoc6_micropython_sensornode_office_box_temperature"
},
"kitchen": {
"pressure": "sensor.psoc6_micropython_sensornode_kitchen_atmospheric_pressure",
"co2": "sensor.psoc6_micropython_sensornode_kitchen_co2_ppm",
"temperature": "sensor.psoc6_micropython_sensornode_kitchen_temperature"
},
"eestec_office": {
"pressure": "sensor.psoc6_micropython_sensornode_eestec_office_atmospheric_pressure",
"co2": "sensor.psoc6_micropython_sensornode_eestec_office_co2_ppm",
"temperature": "sensor.psoc6_micropython_sensornode_eestec_office_temperature"
},
}

HISTORY_MINUTES = 5
ALERT_THRESHOLDS = {
"co2": 1000,  # ppm
"temperature": 30,  # Celsius
"humidity": 70,  # %
"pressure": 1020,  # hPa
"light": 3000  # Lux
}

• Function to fetch sensor data from Home Assistant

def fetch_data(sensor_id):
try:
    with Client(f'http://{HA_IP_ADDR}:8123/api', secret.ha_access_token) as client:
    entity = client.get_entity(entity_id=sensor_id)
    start = datetime.now() - timedelta(minutes=HISTORY_MINUTES)
    history = client.get_entity_histories(entities=[entity], start_timestamp=start)
    values = [float(x.state) for entry in history for x in entry.states if x.state is not None]
    return values
except Exception as e:
    print(f"Error fetching data for sensor {sensor_id}: {e}")
    return None

• Function to fetch occupancy status from Home Assistant

def fetch_occupancy_status(sensor_id):
try:
with Client(f'http://{HA_IP_ADDR}:8123/api', secret.ha_access_token) as client:
entity = client.get_entity(entity_id=sensor_id)
return entity.state == 'on'
except Exception as e:
print(f"Error fetching occupancy status for sensor {sensor_id}: {e}")
return None

• Function to fetch weather forecast data from Home Assistant

def fetch_weather_forecast():
try:
    with Client(f'http://{HA_IP_ADDR}:8123/api', secret.ha_access_token) as client:
    all_states = client.get_states()
    for state in all_states:
        if state.entity_id == 'weather.forecast_home':
            forecast_attr = state.attributes
            forecast_time = forecast_attr.get('datetime', 'N/A')
            forecast_condition = forecast_attr.get('condition', 'N/A')
            forecast_temp = forecast_attr.get('temperature', None)
            forecast_pressure = forecast_attr.get('pressure', None)
            forecast_humidity = forecast_attr.get('humidity', None)
            forecast_wind_speed = forecast_attr.get('wind_speed', None)
            forecast_str = f"Forecast for {forecast_time}: {forecast_condition}\n"
            if forecast_temp:
                forecast_str += f"Temperature: {forecast_temp}°C\n"
            if forecast_pressure:
                forecast_str += f"Atmospheric pressure: {forecast_pressure} hPa\n"
            if forecast_humidity:
                forecast_str += f"Humidity: {forecast_humidity}%\n"
            if forecast_wind_speed:
                forecast_str += f"Wind speed: {forecast_wind_speed} km/h\n"
            return forecast_str
        return "No forecast data available"
    except Exception as e:
        print(f"Error fetching weather forecast: {e}")
        return None

• Function to generate recommendations

def generate_recommendation(sensor_data, forecast_data=None):
    if sensor_data:
        prompt = "Based on the average measurements from the last 5 minutes:\n"
        for sensor_type, value in sensor_data.items():
        unit = ""
        if "pressure" in sensor_type.lower():
            unit = "hPa"
        elif "temperature" in sensor_type.lower():
            unit = "°C"
        elif "co2" in sensor_type.lower():
            unit = "ppm"
        elif "light" in sensor_type.lower():
            unit = "lx"
        elif "humidity" in sensor_type.lower():
            unit = "%"
            prompt += f"{sensor_type}: {value:.2f} {unit}\n"

    if forecast_data:
        prompt += f"\nForecast weather:\n{forecast_data}\n"
        prompt += "\nWhat recommendations would you suggest to my smart office for lower energy consumption and better room conditions. Also take actions according to the forecast weather?"
        print('Prompt:')
        print(prompt)

• Generate a response from the LLaMA model

try:
    output = LLM(prompt, max_tokens=2000)
    recommendation = output["choices"][0]["text"]
    print('Model output:')
    print(recommendation)
    return recommendation
except Exception as e:
    print(f"Error generating recommendation: {e}")
    return None
else:
    print("No sensor data available")

• Function to convert text to speech

def text_to_speech(text, filename='static/recommendation.mp3'):
    tts = gTTS(text)
    tts.save(filename)

• Define the index route

@app.route('/', methods=['GET', 'POST'])
def index():
    sensor_type = None
    recommendation = None
    forecast_data = fetch_weather_forecast()
    sensor_data = {}
    if request.method == 'POST':
        sensor_type = request.form.get('sensor_type')
        if sensor_type:
            for room, sensors in SENSOR_IDS.items():
                if sensor_type in sensors:
                    sensor_id = sensors[sensor_type]
                    if sensor_id:
                        values = fetch_data(sensor_id)
                        if values:
                        average_value = np.mean(values)
                        sensor_data[f"{sensor_type} in {room}"] = average_value
                        else:
                        print(f"No data fetched for {sensor_type} in {room}")
                    else:
                    print(f"No sensor ID available for {sensor_type} in {room}")
                recommendation = generate_recommendation(sensor_data, forecast_data)
                if recommendation:
                    text_to_speech(recommendation)
                return render_template('index.html', recommendation=recommendation)

• Run the Flask app

if __name__ == '__main__':
    app.run(debug=True) 

Explanation of Functions

• fetch_data(sensor_id): Fetches historical sensor data for the specified sensor ID from Home Assistant.
• fetch_occupancy_status(sensor_id): Fetches the occupancy status of a given sensor from Home Assistant.
• fetch_weather_forecast(): Retrieves the weather forecast from Home Assistant.
• generate_recommendation(sensor_data, forecast_data=None): Generates recommendations based on sensor data and optional weather forecast data using the LLaMA model.
• text_to_speech(text, filename='static/recommendation.mp3'): Converts text to speech and saves it as an MP3 file.
• index(): The main route for the Flask app. It handles form submissions, fetches sensor data, generates recommendations, and renders the web page.

HTML Template (index.html)

Create an index.html file in the templates directory for the Flask app:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Smart Office Recommendations</title>
</head>
<body>
    <h1>Smart Office Environment Monitoring</h1>
    <form method="POST">
        <label for="sensor_type">Select Sensor Type:</label>
        <select name="sensor_type" id="sensor_type">
            <option value="pressure">Pressure</option>
            <option value="co2">CO2</option>
            <option value="temperature">Temperature</option>
            <option value="light">Light</option>
            <option value="humidity">Humidity</option>
        </select>
        <button type="submit">Get Recommendation</button>
    </form>
    {% if recommendation %}
        <h2>Recommendation</h2>
        <p>{{ recommendation }}</p>
        <audio controls>
            <source src="{{ url_for('static', filename='recommendation.mp3') }}" type="audio/mpeg">
            Your browser does not support the audio element.
        </audio>
    {% endif %}
</body>
</html>

Running the App

Start the Flask app using the command:

python app.py

Access the app in your web browser at http://127.0.0.1:5000/.

Overview of Project Outcome

The project successfully created a web application that:

• Fetches real-time and historical data from environmental sensors.
• Provides weather forecast data.
• Generates recommendations for optimizing office conditions using a fine-tuned LLaMA model.
• Converts the generated recommendations to speech for easy access.

Outlook for Future Improvement

With more time, the project could be improved by:

• Adding more sensors for additional environmental parameters.
• Implementing real-time control actions based on recommendations.
• Enhancing the user interface for better usability.
• Incorporating machine learning algorithms to predict future conditions and proactively manage the office environment.
• Apply improved security and privacy protocols.

Real-Time Control Actions

Automating real-time control actions involves implementing systems that can autonomously adjust the office environment based on the data collected from sensors and the recommendations generated by the model.

• Connect the system with Heating, Ventilation, and Air Conditioning (HVAC) systems to automatically control temperature and humidity levels.
• Integrate with smart lighting systems to adjust light intensity based on ambient light sensor readings and occupancy status.
• Implement systems to control air purifiers or ventilation based on CO2 levels and other air quality indicators. Smart air purifiers with API support can be integrated.
• Develop a notification system to ensure that important recommendations generated by the model are immediately communicated to users. This system will send alerts to users through various channels (such as email, SMS or mobile app notifications) based on the importance of the recommendations.

User Interface

• Use libraries like D3.js or Plotly to create interactive charts and graphs that visualize sensor data trends, real-time updates, and historical comparisons.
• Implement a notification system to alert users of important events or changes in the office environment, such as high CO2 levels or temperature anomalies.
• Ensure the web application is fully responsive and accessible on various devices, including desktops, tablets, and smartphones.

Advanced Predictive Analytics

Develop advanced predictive analytics capabilities to forecast future environmental conditions and proactively manage the office environment.

• Train advanced machine learning models using historical sensor data to predict trends in temperature, humidity, CO2 levels, etc. This could involve time series analysis and regression models.
• Implement anomaly detection algorithms to identify unusual patterns or potential issues in the sensor data, allowing for early intervention.

Scalability and Robustness

• Migrate data processing and storage to cloud platforms like AWS, Azure, or Google Cloud to handle large datasets and provide high availability.
• Implement a distributed system architecture to distribute the load across multiple servers or microservices, ensuring better performance and reliability.
• Optimize database queries and use scalable database solutions like Amazon RDS, Google Cloud SQL, or NoSQL databases for efficient data storage and retrieval.

Expanded Sensor Network

Incorporate additional types of sensors to provide a more comprehensive environmental monitoring system, covering more aspects of the office environment.

• Add sensors for volatile organic compounds (VOCs) and particulate matter (PM2.5 and PM10) to monitor air quality more comprehensively.
• Integrate noise level sensors to monitor and manage sound levels in different office areas, ensuring a comfortable and productive work environment.
• Implement advanced occupancy detection systems using motion sensors, infrared sensors, or even camera-based solutions to accurately monitor space usage.

Security and Privacy

Implement robust security and privacy measures to protect sensitive data and ensure compliance with relevant regulations.

• Use encryption for data at rest and in transit to ensure that sensitive information is protected from unauthorized access.
• Implement role-based access control to manage user permissions and restrict access to critical functions and data.

AI-Powered Insights

Leverage artificial intelligence to provide deeper insights and more personalized recommendations for office management and energy efficiency.

• Personalize recommendations for specific office spaces, times of day and usage patterns for more effective energy management and environmental control.
• Use Natural Language Processing to allow users to interact with the system through voice commands or natural language queries, making the system more accessible.

Conclusion

The Smart Office Environment Monitoring System project has laid a strong foundation for creating a smart, energy-efficient, and comfortable office environment. The achievements in integrating sensors, generating recommendations, and developing a user-friendly web interface demonstrate the system's potential. By addressing the current limitations and pursuing future goals, the system can evolve into a more advanced and comprehensive solution for smart office management.

Team Members (NNN)

• Berkay ÜZER
• İrem Gül ER
• Alperen SÖNMEZ