This project started out as Project VAL, an intelligent voice activated lamp which uses Microsoft’s Speech Recognition AI service. The lamp was a fun and quirky addition in my office, and was often put on display in events and showcases during the University’s open days and roadshows.
After posting about this project on Facebook, I was contacted by a friend of mine, Richard, a maker in the University of Greenwich who wanted to collaborate. Initially, we wanted to make a pair of internet connected VAL, so I can control his lamp and he can control mine. However, we realized that for a lot of long distance friends and couples, this product could be very useful!
Richard eventually created CALVIN which runs on Android. It was an exciting time for us as we collaborate across the globe to build something together!
To simplify the project, we decided that Twitter would be the easiest way to communicate, as lovers would often tweet to each other. (WeChat and Whatsapp would be used more frequently by couples, but we have no experience accessing those APIs, so we decided to stick with Twitter!). Thus, after 2 weeks of discussion, the development plan for Project Flame was born!
We started off trying to link-up the modules using a pair of ESP8266 Modules courtesy of SeeedStudio. However, due to the complexity of using the ESP8266 on Twitter, we decided to switch over to the TweetInvi C# library for Twitter access. The idea is that the user will have the lamp connected to their home PC for power, so the lamp will be able to communicate with a small background program running on the PC.
To test the lamp, we created 2 Twitter accounts, @projectflame_my and @projectflame_uk, and configured our lamps to follow each other! So now, I can change the lamp on Richard's side by tweeting on @projectflame_my and Richard can change the color of the lamp on my side by tweeting on @projectflame_uk.
Well, it seems cool to be able to change colors on each other's lamp, but what if we could do more? Well, Richard managed to find a great reference to convert "emotions" into colors. More specifically, we aim to encode emotions implied by the tweet into color values to be displayed on the lamp.
To encode them, we first create an SQL database with all the emotion keywords, and the respective color value. After that, an AI system was trained to iddentify tweets which corresponds to the user's feeling. This would allow the AI to guess the author's emotion based on the tweet he/she is sending. To do this, we used Microsoft's LUIS (Language Understanding Intelligent Service) to classify and iddentify emotions from Tweets.
Since these are all webservices, we implemented simple web service calls to get results from the web. The idea is simple, a small.NET application is used as the client, which connects to the Seeeduino Lotus via serial communication. The Seeeduino acts as the hub that controls the LEDs on the lamp. The.NET application also connects to Twitter and checks for new tweets every 2 seconds. We plan to let the user edit the refresh time in the future. If there's a new Tweet, it then sends the tweet to LUIS to determine the emotion. If an "emotional" tweet was detected, then LUIS will return the emotion. The emotion will be used to query the corresponding color from the SQL database created earlier, based on wefeelfine.org's feelings chart.
Imagine this: Coming home to your apartment, and realizing the lamp is blue. You instantly know that your partner is feeling sad or lonely, hence, you can give him/her a call. It certaintly adds a new dimension to your relationship!
Attached here is the preliminary design, and codes for a basic version of Project Flame. We are keeping the codes for emotion detection and the emotion-color database for our future production. If you are interested in investing in our ideas, please do contact us on our twitter, or via email: ivanling92@gmail.com.
Thank you very much for following our story!
Comments