AVATOUR by Imeve - Real-time remote presence with Theta V

AVATOUR: Introduction

Founded in 2017 by key members of the Nokia OZO team, Imeve develops tools and solutions for live and real-time immersive applications. Our newest product is the AVATOUR Remote Presence platform.

AVATOUR was designed to solve a simple but very important problem: you can't always get the right people to the right place.

• Real estate agents can’t always get their clients to the property.
• Construction crews can’t always get needed expertise on site.
• Museums can’t sell tickets to people who aren’t physically present.
• Loved ones can’t always make the wedding.

One-to-many streaming doesn’t solve these problems. Neither does standard videoconferencing. AVATOUR Remote Presence does.

With AVATOUR, multiple remote guests can join a local host in a real place. We capture a space in real-time using a 360-degree camera, deliver a low-latency high-resolution stream, and allow up to 5 users to join using a VR headset or on any standard laptop or mobile device. Our goal is to create the Zoom of XR.

But it's easier to show than tell. Check out the below demo video, showing the view from inside a headset, of an AVATOUR of a property led in real time by a property agent:

We believe this platform has a wide range of applications for businesses and consumers and are eager to bring it to the world in conjunction with Ricoh and their Theta hardware line.

AVATOUR: Platform

The platform involves several components:

• The Host app, which runs on an Android mobile device, controls the Ricoh Theta 360 camera, and shows avatars and voice chat to the local host;
• The cloud platform, which provides ultra-low-latency 4k video streaming worldwide, along with account management, voice chat and metadata serialization;
• And the Guest app, which runs on Oculus VR devices and enables the remote immersive experience.

Prerequisites

Host App hardware

• Camera: Ricoh Theta V (for full integration) or any RTMP source (for manual configuration).
• Until the plug-in is approved on the Ricoh app store, you must register as a Developer with Ricoh in order to access our pre-release "plug-in" for the camera. Fill out the form at https://www8.webcas.net/db/pub/ricoh/thetaplugin/create/input
• AVATOUR is incompatible with 3.0.1 firmware (although it does work with earlier versions). Be sure to update your Theta to the latest version.
• Host device: Android 8.x (Oreo) phone/tablet. iOS devices will be supported in a future release.
• Audio Headset: We recommend the Host use a Bluetooth or wired headset with a boom microphone, as our full-duplex speakerphone capability is not yet implemented. If you do not use a headset, there will be echo.
• Mount: Tablet or phone should be mounted directly under the camera. We recommend the Ulanzi mounts as they are compact, all-metal, and inexpensive. Check out the tablet mount or phone mount.
• Tripod/Selfie Stick: For a stationary setup we recommend this Manfrotto tripod; for a mobile (handheld) setup we recommend this adjustable selfie stick. Of course any tripod or selfie stick with a standard 1/4-20 mount will suffice.

Guest App hardware

• VR Headset: Oculus Go or GearVR. You need to know your Oculus ID and the email you use for your Oculus account for each Guest headset. You may assign more than one device to a single ID, but then those devices will share the same avatar.
• Audio Headset: If you intend to have multiple Guests in the same physical space, you should equip each Guest with a wired audio headset; otherwise, there will be audio crosstalk as one users' mic picks up other users' speech. We like this inexpensive model. If Guests are in different locations, this is not a requirement, although headphones are still recommended to avoid audio leakage from the user's speakers into their own mic.

Installation

The installation process for the current Alpha version is quite complex, as the various components are not available in public app stores. Future releases will be considerably easier to install!

After you have your prerequisites, please follow these steps in order to install the three apps required to run the full end-to-end system:

• Email support@imeve.com with the following information: (1) The Google Play account email which will download the Host app; (2) The Oculus ID's and Oculus account emails you intend to use as Guests
• Support will respond by granting you access to our private Alpha distribution channels, and provisioning your Avatour Host account. You will receive two separate email invitations, one from Google Play for the Host app, and one from Oculus for the Guest app. Respond to both invitations and accept them.
• Support will also provide user credentials for the Avatour platform. You will need this after you install the apps to launch an Avatour.
• Installing the Theta plugin: Currently the Theta plugin is not quite live on the Theta Plugin Store. If it's unavailable from the store, install as follows: Download from https://avatour.s3-us-west-1.amazonaws.com/v0.1/theta-plugin/com.imeve.avatour.cloudstreaming-1.0.14-alpha-release.apk and run`adb install -r -g com.imeve.avatour.cloudstreaming-1.0.14-alpha-release.apk` from a command line
• Installing the Host app: after accepting the Google Play invite, launch Google Play on your Android device. Choose "My Apps & Games" from the left menu, then "Library." Find "Avatour Host" in the list, and click Install.
• Installing the Guest app: after accepting the Oculus Store invite, launch the Oculus app on the mobile device that you paired with the Oculus device(s) you want to use as Guests. Choose "Store" and then scroll all the way to the bottom to find the section labeled "My Preview Apps." Choose the "Avatour" app and click Install. (Alternatively you can do the same thing from your user account on the Oculus website.)

Once everything is installed, you're ready to launch your first AVATOUR.

Operation

To start an Avatour, follow this procedure.

HOST SIDE

• Connect your Theta to the same network that your Android device is on. (The Green WiFi light should be on.)
• Launch the Avatour plugin on your Theta following the standard plugin procedure.
• Launch the Avatour Host app on your Android device.
• Login to the Avatour Host app using the credentials provided to you by Imeve Support.

• Click "Setup" to discover and connect to your Theta V.

• Press "Find" to discover the camera.

• If the camera is not automatically discovered, you may need to manually enter the IP address, camera ID and password, by choosing "Manual" from the drop-down box. (Ways to find camera IP of your Theta - http://theta360.guide/blog/plugin/2018/08/01/find-theta-ip-address.html.)
• The camera ID usually tends to be 'THETAV' followed by the serial number at the bottom of the camera beneath the USB port. (So, if the THETA V serial number on the bottom of the camera is THETAYL123456789 then the password is 123456789.)
• Click 'Connect' to establish a connection with the camera. The live streaming settings should appear

Creating an event and inviting headset users to join an event

• By default, all events are private and only accessible by headset users you invite. Invitations are by oculus id. E-mail field (as of this moment) is not relevant because we do not require headset users to register their emails with the Avatour platform.
• Adding the oculus usernames will automatically make those events visible to the guest when they launch the Avatour client app for Go/ Gear VR.
• The passcode field needs to be shared with users that would like to access the event from a web browser.

Joining/starting an event

The hard work is over ... now you can join your event. Select the event from the Upcoming Events list, and choose Start Event. Any Guests currently logged in for the Event will appear on your screen as avatars, and you will be able to enable or disable the live stream using the small "camera" button at the bottom of your screen.