INTERACTIVE - Smart Museum

Idea

Near an artwork, the visitor will find a hologram of another work of the same author and can rotate it to admire it from every point of view simply using a mobile phone. In fact, moving the mobile phone in one direction, the hologram will rotate in the same direction. Moreover, only by shaking the phone, the visitor will discover other hologram-artworks of that author! Finally, the visitor will be able to leave his feedback to make shine his favorite artwork. In fact, the best-rated one will have a light on!

Thanks to INTERACTIVE the problem of artworks temporarily transferred is solved because with the holograms the visitor can always find all the artworks, and, even if the spaces of the museum are limited, the artworks exhibited can be infinite!

Moreover, through feedbacks, the curator of the museum can easily decide the artworks for the next exhibition to satisfy visitors and increase the museum's success!

Architecture

Using the Generic Sensor API we collect the data of the device sensors (accelerometer and gyroscope) and transfer them via MQTT to the Mosquitto MQTT broker connected to ThingsBoard. Finally, a web page receives the values of the sensors passed by Thingsboard and use them to dynamically rotate the 3D graphic models of the sculpture (created with Three.js) in order to realize the right images to be projected as a hologram.

Always using the MQTT bridge, the values of the feedbacks are sent to ThingsBoard and analyzed through a Rule Chain.

Using an ESP8266 WiFi module, connected to an Arduino UNO board, the values analyzed by ThingsBoard are received, and a yellow LED, connected to the board, lights up when the user is observing the most appreciated artwork.

IoT elements

Generic Sensor API and MQTT protocol

Thanks to Generic Sensor API the data of the orientation and linear acceleration sensors of the mobile phone are collected, and then, through Eclipse Paho, an MQTT client library that accesses the MQTT broker over WebSockets, the messages with the data information are sent to Mosquitto MQTT broker that is connected with ThingsBoard.

This is the Mosquitto .conf file:

# thingsBoard IoT endpoint
connection bridge-01
address cloud.thingsboard.io:1883

# Connection to MQTT Client
listener 9001
protocol websockets

# Specifying which topics are bridged
topic v1/devices/me/telemetry both 1

# Setting protocol version explicitly
#bridge_protocol_version mqttv311
#bridge_insecure false

# Bridge connection name and MQTT client Id,
# enabling the connection automatically when the broker starts.
try_private false
bridge_attempt_unsubscribe false
cleansession true
#clientid qb8UMVi4J4WhcWpxmgZJ
start_type automatic
notifications false
log_type all

remote_username your_access_token

# =================================================================
# Certificate based SSL/TLS support
# -----------------------------------------------------------------
#Path to the rootCA
#bridge_cafile /etc/mosquitto/certs/rootCA.pem

# Path to the PEM encoded client certificate
#bridge_certfile /etc/mosquitto/certs/cert.crt

# Path to the PEM encoded client private key
#bridge_keyfile /etc/mosquitto/certs/private.key

ThingsBoard Rule Chain

We used a ThingsBoard Rule Chain to analyze the feedback of a work of art and calculate the average that must be compared with that of other works of art to find out which is the best rated. In particular, to calculate the average we used the aggregate stream Analytics, and finally, we saved the value, with a new key, through the save timeseries Action.

Arduino

The LED used to identify the best-rated hologram, is connected to a GPIO pin of the ESP8266 WiFi module, and the power is taken from the Arduino Uno board.

This is how the connection is made:

Evaluation

User experience evaluation

From the evaluation of the user experience that we have conducted, it turns out that our system has a high task success rate (≃ 89%), a low task time, and that it is accessible to almost everyone (except for blind people).

Through the SUS (System Usability Scale), that consists of 10 statements to which users rate their level of agreement on a five-point scale, we also have quantified the qualitative data, like the user’s perceived satisfaction of the product, and these are the results:

Technical evaluation

About the technical evaluation, the required software is not so difficult to satisfy: the main request is the use of a browser compatible with the mobile phone sensors.

We also have evaluated the latency of our system and turns out that the time between the rotation of the smartphone and the hologram rotation is few milliseconds, while the time between the shaking of the mobile phone and the appearance of the new hologram is 1-2 seconds.

Finally, the costs are the following:

Initial investment costs

• Hologram projector structure ≃ 20 €
• Device with sensors ≃ 100 €
• Device to display 3D graphics ≃ 50 €
• Arduino Uno board ≃ 25 €
• ESP8266 WiFi module ≃ 5 €

Maintenance costs

The cost of WiFi connection (but this is supposed to be already present in the museum) and the cost related to the energy consumed by the devices.

Demo

About us

This project has been developed as part of my MSc degree at Sapienza University of Rome for the 2019/2020 Internet of Things course.

Alessandro Migliore

Lorella Landi