Smart Building using oneM2M protocol

Introduction

Imagine that you are arriving at your home by car and, when you stop in front of the parking door, automatically it gets open. After that, you park at your parking place and when you go to the Elevator, it is already opened waiting for you to come in and go up. Then, when you enter at home, you find the kitchen light turned on and the coffee recently made.

All this is possible thanks to a coordination between different Application Entities (AE at OneM2M notation) in an Smart Building. The Parking module is able to detect an inhabitant of the building, and calls to the Elevator Module to go to pick up the user. It also notifies to your home devices, if you want, to activate them while you are arriving.

Overview

To perform this scenario, we support on the following AE's:

• Proximity Sensor AE
• Camera AE
• License Plate Recognition AE
• Motor AE
• Semaphore AE
• Monitor AE
• Elevator AE
• Elevator Motor AE
• Elevator Proximity Sensor AE
• Elevator Screen AE
• database wrapper to API REST
• A simple elevator APP to control the Elevator
• An app to create an account at the Smart Building
• A frontend for the Monitor AE

The proximity sensor detects the presence of a user, which is communicated to the Camera and The proximity sensor detects the presence of a user, which is communicated to the camera to take a picture and send it to the neural network to perform the number plate analysis. This number plate is checked against the database of allowed users in the building to know whether to open the garage door, call the lift, and notify your devices if you link an e-weLink account.

To provide the OneM2M Services, we have decided to use Mobius Solution.

Mobius is an implementation of the OneM2M protocol using javascript, so you will need NodeJS to run the server. You will also need MySQL, which is used by Mobius to stored its data, and MQTT Server, which is used to perform the comunication between AEs.

Taking advantage of the fact that we have to use that software, we can use MySQL also for storing data for some AEs if needed, and NodeJS for creating AEs in javascrit.

Implementation

For our aproach, we decided to use the Monitor Architecture, so that all our AEs are independent and all the bussines logic is just in the MonitorAE.

All the AEs contains contains the following containers (CNT):

• DESCRIPTION: with a description of the AE
• DATA: To store information
• COMMAND: if the AE is an actuator. To trigger its actions.

So all our codes contains a function called

registerModule(String AEname, bool isActuator, String initialDesc, String initialData);

1. Garage Door

1.1 Proximity Sensor

An AE (actuator) which detects the presence of something and published on its DATA container "Detected":

With a NodeMCU we can make a simple program in arduino to control a proximity sensor module HC-SR04. In our case we use the library NewPing.

You should change the global variables to set up your CSE and MQTT endpoints

void init_ProximitySensor(){
    String desc = "Name=ProximitySensor;Location=Parking;Desc=it 
        publishes 'Detected' when detects something.";
    String initialData = "";
    originator = "CProximitySensor";
    bool isActuator = false
    registerModule("ProximitySensor", isActuator, desc, initialData);
}

void setup() {
  // intialize the serial port
  Serial.begin(SERIAL_SPEED);
  
  // Connect to WiFi network
  init_WiFi();
  
  // register sensors and actuators
  init_ProximitySensor();
}

// Main loop of the µController
void loop() {
  int distance=0;
  // 50 ms between each detection
  delay(50);
  distance=sonar.ping_cm();

  // 0 means no detection
  if(distance<10 && distance!=0){
    Serial.print("Something detected at ");
    Serial.print(distance);
    Serial.println("cm");
    createCI("ProximitySensor", DATA_CNT_NAME,"Detected");
    // Delay to avoid detect the same object twice
    delay(2000);
  }
}

1.2 Camera

The AE that makes a shoot when triggered a Shoot command, and returns an URL to access to the photo:

AE_NAME = "Camera"
description="Name=Camera;Location=Parking;Desc=This is 
a camera which takes pictures when receives Shoot command"
isActuator = True
registerAE(AE_NAME, isActuator, description)

In our case, we have used a raspberry pi, and we have connected a camera which is controlled with a library in python called PiCamera, so when a Shoot command is sent, the camera takes a photo and stores it in a path that is served by an apache server:

# Get content of oneM2M request
con = jsonP["pc"]["m2m:sgn"]["nev"]["rep"]["m2m:cin"]["con"]
if con=="Shoot": # If the command is "Shoot"
    # Generate the next id of the photo
    global_counter= (global_counter + 1) % 10

    # Path where the image will be stored
    path_img=IMAGE_PATH+"{}.jpg".format(global_counter)
    # Make the camera capture an image and save on the path
    camera.capture(path_img)
    # Create a CI at container data to store the url where the image is available
    module.createCI(AE_NAME, DATA_CONTAINER, 
        "http://"+my_IP+"/{}.jpg".format(global_counter))

1.3 Neuronal Network

This AE receives an URL as a command, verifies whether the URL has a photo on it or not. If so, it downloads the photo and runs a detection algorithm to check if there're any licenses available, and returns in the DATA container the license plate number. Otherwise it'll return "none"

AE_NAME = "LicensePlateRecog"
DESCRIPTION = "This is a neuronal network for number-plate recognition"
isActuator = True
registerAE(AE_NAME, isActuator, DESCRIPTION)

As seen in the photo below, the algorithm has detected a number plate and surrounds it with a green rectangle. Afterwards, it runs other character recognition algorithm in order to extract the letters of the license plate. Once this's finished, it'll return the number plate characters

# Get content of oneM2M request
con = jsonP["pc"]["m2m:sgn"]["nev"]["rep"]["m2m:cin"]["con"]
if con:
    # the content has got an URL to download the image
    response = requests.get(con)

    # The downloaded image is stored at a folder
    if not os.path.isdir(DOWNLOADED_IMAGE_PATH):
        os.mkdir(DOWNLOADED_IMAGE_PATH)
    with open("{}/image.jpg".format(DOWNLOADED_IMAGE_PATH), "wb") as file:
        file.write(response.content)

    # Call the Neuronal Network to perform the detection
    numberPlate = ANPR.runDetection("{}/image.jpg".format(DOWNLOADED_IMAGE_PATH), export=False)    

    # Create the CI with the result of the Network or "none" if no car detected
    if numberPlate != False:
        module.createCI(AE_NAME, DATA_CONTAINER, numberPlate[0])
    else:
        module.createCI(AE_NAME, DATA_CONTAINER, "none")

1.4 Motor

This AE opens it doors when you send "open" command, then waits some time and then it closes.

void init_Motor(){
   String initialDescription = "Name=Motor;Location=Parking;
       Desc=A Motor which opens or closes a garage door";
   String initialData = "closed";
   originator = "CMotor";
   bool isActuator = true
   registerModule("Motor", isActuator, initialDescription, 
                   initialData);
}

void setup() {
   // intialize the serial liaison
   Serial.begin(SERIAL_SPEED);
   //Set up mqtt client
   client.setServer(mqtt_server, 1883);
   client.setCallback(callback);
   client.setBufferSize(1024);

   // set Pins for motor
   pinMode(IN1, OUTPUT);
   pinMode(IN2, OUTPUT);
   pinMode(IN3, OUTPUT);
   pinMode(IN4, OUTPUT);
   // set stepper motor speed to 10rpm
   stepper.setSpeed(10);
   // Connect to WiFi network
   init_WiFi();
   // register sensors and actuators
   init_Motor();
}
void callback(char* topic, byte* payload, unsigned int length) {
    ...
   // At MQTT callback
   if(data == "open") {
      stepper.step(105);
      createCI("Motor",DATA_CNT_NAME,"opened");
      delay(1000);
      stepper.step(-105);
      createCI("Motor",DATA_CNT_NAME,"closed");
   }
}

1.5 Semaphore

This AE is a semaphore with 3 LEDs, one red, one yellow and one green.

void init_Semaphore(){
   String initialDescription = "Name=Semaphore;Location=Parking;
      Desc=A Semaphore which can be turned on in red, green or yellow";
   String initialData = "red";
   originator = "CSemaphore";
   bool isActuator = true
   registerModule("Semaphore", isActuator, initialDescription, initialData);
}
void setup() {
   // intialize the serial liaison
   Serial.begin(SERIAL_SPEED);

   // Connect to WiFi network
   init_WiFi();
   // register sensors and actuators
   init_Semaphore();

   // Set led pins to output
   pinMode(GREEN_PIN, OUTPUT);
   pinMode(YELLOW_PIN, OUTPUT);
   pinMode(RED_PIN, OUTPUT);
}

When "red" command is sent, the red LED turns on, and so on with the green and yellow LEDs.

void callback(char* topic, byte* payload, unsigned int length) {
   ...
   // At MQTT callback
   if(data == "red") {  
      isYellow = 0;
      Serial.println("a");
      digitalWrite(GREEN_PIN,LOW);
      digitalWrite(YELLOW_PIN,LOW);
      digitalWrite(RED_PIN,HIGH);
   } else if(data == "green") {
      isYellow = 0;
      digitalWrite(YELLOW_PIN,LOW);
      digitalWrite(GREEN_PIN,HIGH); 
      digitalWrite(RED_PIN,LOW);
   }else if(data=="yellow"){
      isYellow = 1;
      digitalWrite(RED_PIN,LOW);
      digitalWrite(GREEN_PIN,LOW);
   }
}

// Main loop of the µController
void loop() {
   // The yellow LED is separated from the rest to make the flashing effect.
   if(isYellow == 1) {
      digitalWrite(YELLOW_PIN,HIGH);
      delay(420);
      digitalWrite(YELLOW_PIN,LOW);
      delay(420);
   }
   ...
}

1.6 Monitor

This AE is the one which contains all the bussiness logic of the application. It is subscribed to all AE's to handle the next step of the chain of events.

// Called when receives an MQTT req. 
// "originator" is the name of the broadcaster and "data" is the ci value.
switch(originator) {
    case "ProximitySensor":
        // Save the detection at database
        monitorService.onProximitySensor();
        // tell the camera to make a shoot
        cseService.instanciate("Camera","COMMAND","Shoot");
        // Tell the semaphore to change to yellow 
        // (as a feedback to the user to know that the system is processing)
        cseService.instanciate("Semaphore", "COMMAND", "yellow");
        break;
    case "Camera":
        console.log("Camara AE made a shoot and published it here:", data);
        // Tell to the ANPR which is the photo
        cseService.instanciate("LicensePlateRecog","COMMAND", data);
        break;
    case "LicensePlateRecog":
        console.log("License Plate Recognition AE detected this plate:", data);
        if(data == "none") {
            // Store at database a false detection
            dbService.detectedFalse();
            // Put the semaphore in red, because no-one is there
            cseService.instanciate("Semaphore", "COMMAND", "red");
        } else {
            // Store at database a success detection
            dbService.detectedTrue();
            const {belongs, userId} = dbService.checkPlate(data);
            if(belongs){
                console.log("This plate belongs to the building");
                // Store at database the number of cars which belongs to the building
                dbService.detectedBelong();
                // Tell the motor to open
                cseService.instanciate("Motor","COMMAND", "open");
                // Tell the Mobile App which user has arrived to turn on its devices
                cseService.instanciate("MobileApp", "COMMAND", userId);
                // Tell the semaphore to be yellow
                cseService.instanciate("Semaphore", "COMMAND", "yellow");
                // Tell the elevator to go to the Parking floor (Floor 0)
                cseService.instanciate("Elevator", "COMMAND", 0);
            } else{
                console.log("This plate does not belongs to the building");
                dbService.detectedNotBelong();
                cseService.instanciate("Semaphore", "COMMAND", "red");
            }
        }
        break;
    case "Motor":
        if(data == "opened") {
            // When the motor is open, tell the semaphore to change to green
            cseService.instanciate("Semaphore", "COMMAND", "green");
        } else if(data == "closed") {
            // When the motor is close, tell the semaphore to change to red
            cseService.instanciate("Semaphore", "COMMAND", "red");
        }
        break;
}

The Monitor has some instructions to store data at a database. With this information we can make a frontend web showing the stadistics of the website.

From this frontend we can also show the list of users which have tried to register at that building so that an administrator can confirm that actually that user can access to the Parking.

2. App

This AE will trigger the Home Devices of a user.

const isActuator = true
registerModule('MobileApp', isActuator, 'Mobile app for controlling elevator and ewelink devices', '')

It will receive on its COMMAND container the id of the user who has entered through the parking door.

// data is the COMMAND container instance
con.query(`SELECT * FROM USER WHERE USER.id='${data}'`, (err, m) => {
    databaseInfo = m[0]
    // If the user has got an e-WeLink account linked and 
    // the building administrator has verified the account, connect to e-weLink
    if (databaseInfo.Ewelink == 1 && databaseInfo.verified == 1){
        const connect = new ewelinkAPI({
            email: databaseInfo.Ewemail,
            password: databaseInfo.Ewpassword,
        });
        const devicesONData = JSON.parse(databaseInfo.devicesON)
        let devices2On = []
        for (let i = 0; i < devicesONData.n; i++){
            if (devicesONData.data[Object.keys(devicesONData.data)[i]] == 1){
            devices2On.push(Object.keys(devicesONData.data)[i])
        }
    }
    if (devices2On != ""){
        for (let i = 0; i < devices2On.length; i++){
            connect.setDevicePowerState(devices2On[i], 'on')
        }
    }
}

The AE has got a frontend to make possible for the user to select the devices (ewelink devices) that they want to turn on when they enter the building. Also, it brings the possibility of calling the elevator whenever needed, even if the user it's outside the building.

It is connected to the frontend through a Web Socket.

When the user want to call the elevator, sends a message to the backend, and it commands the elevator

case "setFloor":
    const floor = data.floorNumber;
     module.createCI('ElevatorAE', 'COMMAND', floor)
     resolve(JSON.stringify({return: true}))
     break;

On the same way, to know the user credentials, to login in the app or to get the devices to turn on, has to ask to the backend:

case "login":
    // Ask to the db if the password is the same
    break;
case "register":
    // add user to the database
    break;
case "set":
    // set devices to turn on at database
case "getDevices":
    // ask the devices to the database

The app requires a backend running, the IP of the computer which is running the backend written on the app logic (needs to be changed in every different network) and the mobius server as well as other dependencies that mobius requires

wsConnection.js file in mybuildingapp/app/services/

const IP = "ws://BACKEND_IP:BACKEND_PORT";

Screens

The backend implements the oneM2M protocol through which it communicates the client id. Once it has received the id, it proceeds to turn the previously selected devices on.

3. Elevator

The elevator module is under a Middle Node in the OneM2M resource tree. So we have used &Cube Rosemary to create the CSE-MN. As its requirements are the same as Mobius, we don't need to install any software apart from &Cube Rosemary.

The Elevator has two main use cases:

• Allows users to operate the elevator via the building's mobile application directly. (Though its COMMAND container)
• For the users which doesn't belong to the building, they can access to it via an URL, protected with a One Time Password (OTP) to confirm that the user is in front of the elevator.

The Elevator AE acts as the monitor of that module. It controls three AEs: A motor, a proximity sensor and a screen.

The proximity sensor detects the presence of the user and notifies the Screen to make an OTP, to show it and send to the Elevator Monitor. When the user enters in the web the OTP, is also sent to the Elevator Monitor to be compared and trigger the elevator if correct.

// Called when receives an MQTT req. 
// "ae" is the name of the broadcaster and "data" is the ci value.
switch(ae) {
    case AE_NAME:
        // Tell the elevator what floor to go
        // (1st use case: Triggeder from App Module or from the Building Monitor)
        rosemary.instanciate("Elevator_Motor", "COMMAND", data);
        break;
    case "Elevator_ProximitySensor":
        // Tell the Screen that someone is in front the elevator
        rosemary.instanciate("Elevator_Screen", "COMMAND", "detect");
        // It could be added an instruction to store the 
        // number of detections at database
        break;
    case "Elevator_Screen":
        // The screen tells an otp, which is stored at database
        elevatorService.setOTP(1,data);
        // When the user send the OTP from the mobile frontend (Through HTTP),
        // It will be contrasted with the one at database
        break;
    case "Elevator_Motor":
        console.log("The elevator is at " + data + " floor");
        // It could be added an instruction to store 
        // the floors where the elevator goes
        break;
}

The Elevator AE has a REST API to receive the OTP that the user has entered and to register more lifts if necessary.

// Add an elevator resource
// POST /elevator/add
router.post('/add',express.json(), async (req, res) => {
    try {
        const floors = req.body.floors;
        if(!req.body.floors) {
            res.send(400);
        } else {
            const r = await add(floors);
            res.send({id:r.insertId}).status(200);
        }
    }catch(error) {
        res.status(500);
    }
});

// retrieve a elevator by its id
// GET /elevator/:id
router.get('/:id', async (req, res) => {
    try{
        const id = req.params.id;
        const r = await get(id);
        if(r.length === 0){
            console.error(`ID ${id} does not exists`);
            res.status(404).end();
        } else {
            res.send({floors:JSON.parse(r[0].FLOORS)}).status(200);
        }
    } catch(error) {
        console.error(error);
        res.status(500);
    }
});

// Check if the OPT of an elevator ID 
// POST /elevator/:id/check-otp
// body: {currentFloor: Int}
router.post("/:id/check-otp", express.json(), async (req, res) =>{
    try {
        const id = req.params.id;
        const otp = req.body.otp;
        const currentFloor = req.body.currentFloor;
        // Compare it with the OTP that the Screen sent to this AE.
        const r = await checkOTP(id, otp);
        if(r) {
            rosemary.instanciate("Elevator_Motor", "COMMAND", currentFloor);
            rosemary.instanciate("Elevator_Screen", "COMMAND","correct_password")
        } else {
            rosemary.instanciate("Elevator_Screen", "COMMAND","incorrect_password")
        }
        res.send({correct:r}).status(200);
    } catch(error) {
        console.error(error);
        res.status(500);
    }
});

router.post("/:id/go-to", express.json(), async (req, res) =>{
    try {
        const id = req.params.id;
        const desiredFloor = req.body.desiredFloor;
        rosemary.instanciate("Elevator_Motor", "COMMAND", desiredFloor);
        res.status(204).end();
    } catch(error) {
        console.error(error);
        res.status(500);
    }
})

3.1 Motor

This AE moves up or down the elevator in order to reach the designed floor.

int currentFloor;

void init_Elevator_Motor(){
    String initialDescription = "Name=Elevator_Motor;Location=Elevator";
    String initialData = "0";
    originator = "CElevator_Motor";
    registerModule("Elevator_Motor", true, initialDescription, initialData);
}
void setup() {
    // initialize the serial port
    Serial.begin(SERIAL_SPEED);
    // initialize motor pins
    pinMode(IN1, OUTPUT);
    pinMode(IN2, OUTPUT);
    pinMode(IN3, OUTPUT);
    pinMode(IN4, OUTPUT);
    // Connect to WiFi network
    init_WiFi();
    // set stepper motor speed to 10rpm
    stepper.setSpeed(10);
    currentFloor=0;
    // set up mqtt client
    client.setServer(mqtt_server, 1883);
    client.setCallback(callback);
    client.setBufferSize(1024);
    // register elevator motor AE
    init_Elevator_Motor();
}

void callback(char* topic, byte* payload, unsigned int length) {
    ...
    if(data != NULL){
        if(data != "") {
           long floor = data.toInt();
           // Check how many floors have to go up or down
           long diff= floor - currentFloor;
           currentFloor=floor;
           // Create a content instance in the data container 
           // with the destination floor
           createCI("Elevator_Motor", DATA_CNT_NAME,String(currentFloor));
           // Check if the elevator has to go up or down
           if(diff>0){
              for(int i = 0; i < diff; i++){
                stepper.step(500);
                yield();
              }
           }else if(diff<0){
             for(int i = 0; i > diff; i--){
               stepper.step(-500);
               yield();
             }
           }
        }
     }
}

3.2 Screen

This AE uses an LCD display to communicate with the user in case someone who doesn't belong to the building wants to use the elevator.

void init_Elevator_screen(){
  String initialDescription = "Name=Elevator_Screen;Location=Elevator";
  String initialData = "Nothing yet";
  originator = "CElevator_Screen";
  registerModule("Elevator_Screen", true, initialDescription, initialData);
}

//function that generates the password to access the elevator control
void generateOTP(){
  otp = (String)random(0,10)+(String)random(0,10)+(String)random(0,10)
        + (String)random(0,10);
}

void setup() {
  
  // intialize the serial liaison
  Serial.begin(SERIAL_SPEED); 
  
  // Connect to WiFi network
  init_WiFi();
  //Set up mqtt client
  client.setServer(mqtt_server, 1883);
  client.setCallback(callback);
  client.setBufferSize(1024);
  
  Wire.begin(D2,D1);

  // register elevator screen AE
  init_Elevator_screen();
  
  lcd.init();
  lcd.backlight();
}

If the display is notified of someone's presence, it generates an OTP, publishes it in its data and displays it to the user. ("DETECT")

The AE in charge of verifying that the user has entered the correct password, will notify the Display with "correct_password" or "incorrect_password".

In both cases, the display will show messages to let the user know the status of the elevator.

void callback(char* topic, byte* payload, unsigned int length) {
  ...

  if(data != NULL){                                            
    if(data == "detect") {                                                             
      //if the proximity sensor detects something, the screen generate a 
      //password and shows it
      lcd.clear();
      lcd.setCursor(0,0);
      lcd.print("Scan the QR code");
      lcd.setCursor(5,1);
      generateOTP();
      // Publish the OTP at container data
      createCI("Elevator_Screen", DATA_CNT_NAME, otp);
      lcd.print(otp);                                                  
    } else if(data == "correct_password") {  
      //if the user 
      lcd.clear();
      lcd.setCursor(0,0);
      lcd.print("Password correct");
      lcd.setCursor(0,1);
      lcd.print("Elevator coming");
    }else if(data=="incorrect_password"){
      lcd.clear();
      lcd.setCursor(0,0);
      lcd.print("Incorrect");
      lcd.setCursor(0,1);
      lcd.print("password");
      delay(5000);
      lcd.clear();
      lcd.setCursor(0,0);
      lcd.print("Scan the QR code");
      lcd.setCursor(5,1);
      // Generate another OTP
      generateOTP();
      createCI("Elevator_Screen", DATA_CNT_NAME, otp);
      lcd.print(otp);
    }else{
      lcd.clear();
      lcd.setCursor(0, 0);
    }
  }

}

3.3 Proximity Sensor

This AE is same as the Parking Proximity Sensor. Is just an actuator which detects the presence of a user trying to scan the QR, so as to tell the Screen to generate an OTP.