Magic VR Hat

Introduction

Last August 8-11, we were fortunate to participate at the 2017 China-US Young Maker Competition. It was a great event, the Chinese Maker community is vibrant and growing! It's awesome to see great projects and young Makers in one event! https://www.chinaus-maker.org. Thanks Intel for sponsoring the competition.

The entry project for the competition is the IoT VR SnowGlobe. Snow Globe hosting a Virtual Reality Website. Play w/ snow in VR via HTC Vive, Tilt the Snow Globe and snow starts falling.

During our stay, we were able to visit Great Wall of China. While we were there, our friend Mike gave us a Chinese straw hat. It was very useful, lots of walking in the heat so I used it as a fan. It was hot up there.

The competition started with a 24-hour Hackathon, we decided to build a prototype of a VR Hat. The goal is to re-experience the event (Great Wall) thru Virtual Reality. Think of it as a slideshow viewer with a twist; using the hat to transport through different 360° photos of the event.

Video

Step 1

The participant wears a VR head mounted device. Each time the participant wears the hat, it loads a random 360° photo of the event. To be transported into another experience, the participants removes the hat and wears it back. The image on the laptop is what you will see inside VR.

It's a simple interaction paradigm to shift different VR experiences.

What we did was to re-use all the parts we have for the IoT VR SnowGlobe and transferred it to the hat. All we need is the accelerometer data from Arduino 101 and connect it to the ESP8266. The ESP8266 servers as a VR webserver.

This is the Arduino 101 code, we use it to send Accelerometer data, detects "up" or "down" and sends signal to pin 13.

/*
  IMU orentation + temperature rise detection
  for the Intel Arduino 101
  http://electronhacks.com
  http://dagdag.net
  https://www.hackster.io/33351/let-it-snow-iot-snow-globe-with-virtual-reality-web-5123a6
*/
#include "CurieIMU.h"
void setup() 
{
 pinMode(13, OUTPUT);
  Serial.begin(115200);
  Serial.println("Initializing IMU device...");
  CurieIMU.begin();
  CurieIMU.setAccelerometerRange(2);
}

Loop - read accelerometer data and send to pin 13.

void loop() 
{
 int orientation = - 1;   // the board's orientation
  String orientationString; // string for printing description of orientation
 // read accelerometer:
 int x = CurieIMU.readAccelerometer(X_AXIS);
 int y = CurieIMU.readAccelerometer(Y_AXIS);
 int z = CurieIMU.readAccelerometer(Z_AXIS);
 // calculate the absolute values, to determine the largest
 int absX = abs(x);
 int absY = abs(y);
 int absZ = abs(z);
 if ( (absZ > absX) && (absZ > absY)) {
 // base orientation on Z
 if (z > 0) {
      orientationString = "up";
      orientation = 0;
 digitalWrite(13, HIGH);  
    } else {
      orientationString = "down";
      orientation = 1;
 digitalWrite(13, LOW);
    }
  }
 // if the orientation has changed, print out a description:
 if (orientation != lastOrientation) {
    Serial.println(orientationString);
    lastOrientation = orientation;
  }
}

ESP8266 - code initialization. We used ESP8266 Wemos to host the Hat VR WebServer. Other Javascript files and images are hosted on different server.

/*
    IoT Snow Globe
    Ron Dagdag @rondagdag
     WiFi VR Web Server
*/
#include <ESP8266WiFi.h>
#include <WiFiClient.h>
#include <ESP8266WebServer.h>
#include <ESP8266mDNS.h>
#include "WifiAuth.h"
#include <Timer.h>
Timer t;     
int accelTrig = 0;
int fog = 0;
const int led = 13;
int anlgSamples[9];
int rawUpperRange = 450;
int rawLowerRange = 430;
int ctr = 0;
int keyIndex = 0;                 // your network key Index number (needed only for WEP)
int status = WL_IDLE_STATUS;
ESP8266WebServer server ( 80 );

Step 2

Setup - Connect to WiFi, start Webserver on Port 80. D3 is an input connected to Arduino 101 via pin 13.

void setup() {
 pinMode(D3, INPUT);
 pinMode(A0, INPUT);
 randomSeed(analogRead(0));
 pinMode ( led, OUTPUT );
 digitalWrite ( led, 0 );
  Serial.begin ( 115200 );
  WiFi.begin ( WIFI_SSID, WIFI_PASS );
  Serial.println ( "" );
 // Wait for connection
 while ( WiFi.status() != WL_CONNECTED ) {
 delay ( 500 );
    Serial.print ( "." );
  }
  Serial.println ( "" );
  Serial.print ( "Connected to " );
  Serial.println ( WIFI_SSID );
  Serial.print ( "IP address: " );
  Serial.println ( WiFi.localIP() );
 if ( MDNS.begin ( "esp8266" ) ) {
    Serial.println ( "MDNS responder started" );
  }
  server.on("/", handleRoot);
  server.on("/vr", handleVR);
  server.on("/updates", handleUpdates);
  server.onNotFound(handleNotFound);
  server.begin();
 // you're connected now, so print out the status:
  Serial.println ( "HTTP server started" );
  t.every(5000, averaging);
  rawUpperRange = analogRead(A0);
 for (int i = 0; i < 9; i++){
    anlgSamples[i] = rawUpperRange;
  }
  rawLowerRange = (rawUpperRange - 22);
}

• Handle Root - if the user goes to main site, it will show "hello from esp8266".

void handleRoot() {
 digitalWrite(led, 1);
  server.send(200, "text/plain", "hello from esp8266!");
 digitalWrite(led, 0);
}

• Handle Not Found - if the route is not found, it will show message 'file not found'.

void handleNotFound(){
 digitalWrite(led, 1);
  String message = "File Not Found\n\n";
  message += "URI: ";
  message += server.uri();
  message += "\nMethod: ";
  message += (server.method() == HTTP_GET)?"GET":"POST";
  message += "\nArguments: ";
  message += server.args();
  message += "\n";
 for (uint8_t i=0; i<server.args(); i++){
    message += " " + server.argName(i) + ": " + server.arg(i) + "\n";
  }
  server.send(404, "text/plain", message);
 digitalWrite(led, 0);
}

• Handle Updates - when /updates is requested, it would return a json containing accel, either the hat is up or down.

void handleUpdates() {
 char temp[50];
 //int accel = random(2);
 int accel;
  accel = digitalRead(D3);
  fog = map(analogRead(A0), rawLowerRange, rawUpperRange, 255, 0);
 snprintf ( temp, 50,
"{\"accel\":%d,\"fog\":%d }",
accel, fog);
  server.send ( 200, "application/json", temp );
  Serial.println(accel);
  Serial.println(fog);
}

• Handle VR Requests - This is the main website that will be loaded when user requests.

void handleVR() {
 digitalWrite ( led, 1 );
 ///char temp[3500];
 //snprintf ( temp, 3500,
String html = "<html>\
<head>\
<meta charset='utf-8'>\
<title>VR SnowGlobe</title>\
<meta name='description' content='VR SnowGlobe'>\
<script src='http://192.168.137.1:3000/components/aframe.min.js'></script>\
<script src='http://192.168.137.1:3000/components/aframe-vid-shader.min.js'></script>\
<script src='http://192.168.137.1:3000/components/aframe-extras.min.js'></script>\
<script src='http://192.168.137.1:3000/components/aframe-particle-system-component.min.js'></script>\
\
</head>\
<body>\
<a-scene>\
  <a-assets>\
    <a-mixin id='snowTemplate' position='0 2.25 -15' particle-system='preset: snow;particleCount:500; texture: http://192.168.137.1:3000/images/star.png'></a-mixin>\
\
  </a-assets>\
  <a-entity id='snowList'>\
    <a-entity id='snow' mixin='snowTemplate'></a-entity>\
  </a-entity>\
    <a-entity light='type: ambient; color: #405e94'></a-entity>\
    <a-entity light='type: directional; color: #FFF; intensity: 0.8' position='5 5 10'></a-entity>\
    <a-entity id='leftController' static-body='shape: sphere; sphereRadius: 0.02;' hand-controls='left' sphere-collider='objects: .throwable' grab></a-entity>\
    <a-entity id='rightController' static-body='shape: sphere; sphereRadius: 0.02;' hand-controls='right' sphere-collider='objects: .throwable' grab></a-entity>\
\
    <a-entity id='message' text='wrapCount:15;width:20;height:20;value:Please wear Hat;alphaTest:0;opacity:0.7' position='0 0 -6' rotation='0 0 0'></a-entity>\
    <a-sky id='sky' set-sky src='http://192.168.137.1:3000/assets/greatwall.jpg' geometry='thetaStart:2;radius:25'></a-sky>";
String   last = "</a-scene>\
<script src='http://192.168.137.1:3000/hat.js'></script>\
</body>\
</html>"; //);
  html = html + last;
  server.send ( 200, "text/html", html );
 digitalWrite ( led, 0 );
}

Notice that the a-sky tag has 'set-sky' attribute. That's defined in hat.js. A-Sky is the background 360° image to be loaded.

 <a-sky id='sky' set-sky src='http://192.168.137.1:3000/assets/greatwall.jpg' geometry='thetaStart:2;radius:25'></a-sky>";

Also noticed that I stored all the images and javascript in a different server. I specifically did this to load faster since the ESP8266 does not have enough horsepower.

Step 3

Loop - just calls handleClient for routing.

void loop(void){
  server.handleClient();
  t.update();
}

• hat.js - this is stored in a different server, so I can easily control the hat. This register a component - set-sky inside set-sky init. It runs the timeout interval every 3 seconds and runs the checkStatus function.

• The checkStatus function calls /updates to get the json result, parses it and pass to updateSky function.

• UpdateSky function checks for accelerometer data, this will set the attribute of a-sky to change background image. It randomly loads pictures from this server http://192.168.0.101:3000/assets/image[0-6].jpg

var snowList = document.querySelector('#snowList'); 
var scene = document.querySelector('a-scene'); 
var carbonOverload = false; 
AFRAME.registerComponent('set-sky', { 
  schema: {default:''}, 
  init: function() { 
    this.timeout = setInterval(this.checkStatus.bind(this), 3000); 
    this.material = this.el.getAttribute('material'); 
     this.el.setAttribute('material',{ src: '', color: 'black'}) 
    this.skyvisible = false; 
  }, 
  remove: function() { 
    clearInterval(this.timeout); 
    this.el.removeObject3D(this.object3D); 
  }, 
 updateSky: function(result) { 
       if (result.accel >= 1 ) { 
                   // snow.components['particle-system'].data['particleCount'] = 5000; 
                   // enableSnow(snow); 
               if (!this.skyvisible) {  
                 return; 
               } else { 
                   this.el.setAttribute('material',{ src: '', color: 'black'}) 
                   message.setAttribute('visible', true); 
                   this.skyvisible = false; 
               } 
             } else { 
               if (this.skyvisible) {  
                 return; 
               } else { 
                 var filename = 'http://192.168.0.101:3000/assets/image' + Math.floor(Math.random()*6) + '.jpg';// + "?" + Math.random(); 
                 this.el.setAttribute('src',filename) 
                 this.material.src = filename 
                 this.el.setAttribute('material', this.material); 
                 message.setAttribute('visible', false); 
                 this.skyvisible = true; 
               } 
             }    
  }, 
  checkStatus: function() { 
    var self = this; 
       var xhr = new XMLHttpRequest(); 
       xhr.open('GET', "/updates"); 
       xhr.addEventListener('readystatechange', function(data) 
		    {  
         if(xhr.readyState !== XMLHttpRequest.DONE) return; 
         if(xhr.status !== 200 && xhr.status !== 304){ 
           console.warn('Could not fetch avatar info'); 
           return; 
         } 
         var result = JSON.parse(xhr.responseText);   
         self.updateSky(result);   
       }); 
       xhr.send(); 
  } 
}); 

That's it. Feel free to reach out if you have questions.

Conclusions

Use Samsung Internet to access the VR Webserver. In Windows Mixed Reality Headset, use the Edge Browser. Use Mozilla Firefox if you have Oculus Rift or HTC Vive.

If this project made you interested in learning more about AFrameVR, Arduino 101, ESP8266 Wemos, click on the "Add Respect" and follow me.