Since v2.5 Babylon.js support WebVR using the WebVRFreeCamera.
In Babylon v3.0 we fully support the WebVR 1.1 specifications (https://w3c.github.io/webvr/) which is supported by the latest version of chromium and firefox nightly, and planning to support WebVR 1.0 for legacy systems, such as GearVR.
The WebVR camera is Babylon's simple interface to interaction with the HTC Vive and Oculus Rift.
Babylon.js also supports the VR devices' controllers - The HTC vive's controllers and the Oculus touch - using the gamepad extension. Further details below.
WebVR 1.1 is enabled in specific versions of chromium, firefox nightly, and soon Microsoft Edge. To get constant status updates, please visit WebVR rocks at https://webvr.rocks/ . We support any browser that implements WebVR 1.1.
The WebVR controllers are offered in browsers that support the WebVR gamepad extensions - https://w3c.github.io/gamepad/extensions.html . In Chromium you must enable this API in chrome://flags in order to get it working. Make sure to visit https://mozvr.com/ for full installation instructions.
The WebVRFreeCamera is being initialized the same as a standard free camera:
var camera = new BABYLON.WebVRFreeCamera("camera1", new BABYLON.Vector3(0, 0, 0), scene);
This will initialize a new WebVR camera and will enable WebVR in the engine.
Just like any other camera, to get the camera working correctly with user input and interactions, we will need to attach the camera (and the VR device) to the canvas and the scene. To do that we use the same method we know from the free camera:
camera.attachControl(canvas, true);
This will, however, only work on all browsers. Chromium (and probably the rest of the browsers soon) only support attaching the VR device to the scene during a user interaction (a mouse click, for example). To get that working correctly, a simple solution would be:
scene.onPointerDown = function () {
scene.onPointerDown = undefined
camera.attachControl(canvas, true);
}
What it does is attach control once the user click on the canvas, and disables the onPointerDown callback.
This can be done with an HTML or a Canvas2D button as well, and using vanilla javascript event listeners. Any intentional user interaction is allowed. A mouse-move event will not trigger it, so don't bother trying. A simple example would be:
// after creating a button with vrButton as ID:
let button = document.getElementById('vrButton');
function attachWebVR() {
camera.attachControl(canvas, true);
window.removeEventListener('click', attachWebVR, false);
}
button.addEventListener('click', attachWebVR, false );
Don't forget to remove the event listener, other wise any click on the button will trigger the attach function. It won't attach again, but it a waste of function calls and is not needed.
You should now be able to see your scene in the WebVR device. If not, go to troubleshooting!
The WebVR camera is an extended FreeCamera. Apart from all of the abilities a standard FreeCamera has, the WebVR camera has 2 major extensions - an extra position and an extra rotation, which are the pose data broadcasted by the VR device connected to the browser. This means that the camera has actually two transformation - one is controlled by you, and the other by the device. They are accumulated - position is being added and rotation multiplied - in order to combine the developer's input and the VR device's pose data.
To understand that think of your head and your body. Without moving your body, your head can move in all directions, and rotate in all directions. The WebVR device is your head. Your body is the regular position and rotationQuaternion we all know and love. If you rotate your body, the head rotates with it. But if you move the head, the body stays in the same position.
This is exactly how you should see the WebVR extra transformation - your head position is set by the VR device (and cannot be interfered with). Your body (or position in the world) is fully controlled by you.
This allows you to use the same code you use for a game based on the FreeCamera with the WebVR camera. the only difference is that the user will have the ability to rotate the camera locally using the VR device and not the mouse.
This also allows the WebVR to be controlled by the same input devices that control the FreeCamera - keyboard, mouse (with rotation exception), XBOX controller and so on.
The device's "front" position is set by the device itself (it is set during the device's setup and has not a lot to do with WebVR directly). The developer, however, has the ability to change the "front" rotation with a simple function call:
camera.resetToCurrentRotation().
This will set the current Y axis (and Y axis direction only!!) to be the current front rotation of the user.
camera._vrDevice, a public hidden member in the camera.camera.rawPose. The rawPose has the following interface (a dream for physics lovers!):export interface DevicePose {
readonly position?: Float32Array;
readonly linearVelocity?: Float32Array;
readonly linearAcceleration?: Float32Array;
readonly orientation?: Float32Array;
readonly angularVelocity?: Float32Array;
readonly angularAcceleration?: Float32Array;
}
Each VR device currently available (Oculus rift and vive) has controllers that complement its usage. Both the vive controllers and the oculus touch controllers are supported by using the gamepad extensions.
During the WebVRFreeCamera initialization it will attempt to attach the controllers and detect them if found. If found, the controllers will be located at camera.controllers which is an array that will either have a length of 2 or 0. If the controllers are attached and were not detected, you could also try to manually call camera.initControllers() at a future time.
To fire a callback when the controllers are found you can use the optional camera.onControllersAttached callback:
onControllersAttached = function(controllers) {
console.log(controllers.length === 2); // outputs true;
}
Initializing the controllers using the camera will also attach them to the camera, which will allow moving the controllers together with the WebVR camera, if moved by the user.
There are two high level implementations that are automatically assigned to a WebVR controller:
OculusTouchController for the oculus touch
ViveController for the vive controllers.
Both extend the WebVRController class, which extends the PoseEnabledController.
To cut a long story short - Each controller is assigned the same set of functions, with the only different being the button mappings. The type of the device can be retrieved using controller.controllerType, which has the following values:
export enum PoseEnabledControllerType {
VIVE,
OCULUS,
GENERIC
}
This enum will be extended when needed.
A controller button has the following set of data:
interface ExtendedGamepadButton extends GamepadButton {
readonly pressed: boolean;
readonly touched: boolean;
readonly value: number;
}
These values will be sent to the observers of any specific button when either on of them was changed.
The controllers also have Axes-data, which can be compared to the stick value of an x-box controller. They consist of a 2D vector (with x and y values). Both the oculus (the touchpad) and the oculus touch (the center trackpad) can omit stick values. Stick values (SHOULD BE) are between -1, -1 and 1, 1, with 0,0 being the default value.
Abstract mapping
The following observables exist on all types of WebVR controllers, in case you wish to develop an abstract solution to all VR devices and not focus on a specific device:
onTriggerStateChangedObservable is the main trigger observableonMainButtonStateChangedObservable the main button observable onSecondaryButtonStateChangedObservable - you get the point...onPadStateChangedObservable - stick-button observable (NOT the Stick Values)onPadValuesChangedObservable - stick values changed observableTo use any of them, simple register a new function with the desired observable. For example, to monitor the trigger and observe pad value changes:
controller.onPadValuesChangedObservable.add(function (stateObject) {
console.log(stateObject); // {x: 0.1, y: -0.3}
});
controller.onTriggerStateChangedObservable.add(function (stateObject) {
let value = stateObject.value;
console.log(value);
});
Vive Controller mapping
The vive supports:
onPadStateChangedObservableonTriggerStateChangedObservableonMainButtonStateChangedObservable, onRightButtonStateChangedObservable and onLeftButtonStateChangedObservable (aliases to the same observable object!);onSecondaryButtonStateChangedObservable and onMenuButtonStateChangedObservable (aliases).Oculus touch mapping
The oculus touch supports 6 different buttons:
onPadStateChangedObservable.onTriggerStateChangedObservable.onSecondaryTriggerStateChangedObservable.onMainButtonStateChangedObservable, onAButtonStateChangedObservable on the right hand and onXButtonStateChangedObservable on the left hand.onSecondaryButtonStateChangedObservable, onBButtonStateChangedObservable on the right hand and onYButtonStateChangedObservable on the left hand.onThumbRestChangedObservable .Instead of forcing you to use the controller meshes (which will prevent you from implementing a single app for many types of devices), we have decided to allow you to attach the controller to a mesh. This will make the controller the mesh's "parent" (but not using the parenting system! As a controller is not a node). The controller's actions (rotation and position changes) will reflect directly to the mesh.
To attach the controller to a mesh:
controller.attachToMesh(mesh);
Note that this will create a new quaternion to the mesh .
Controllers without WebVR camera
The controllers can also be initialized without using a WebVR camera, which means - you can use them to control your regular WebGL game or 3D application.
To do that, simply initialize the Gamepads Class:
new BABYLON.Gamepads((gp) => {
if (gp.type === BABYLON.Gamepad.POSE_ENABLED) {
// Do something with the controller!
}
});
Note that the position will be relative to the initial VR Device that is related to those controllers.
Pose data
Just like the WebVR camera, the controllers export their (right handed!!) raw pose data. The data is updated each frame at controller.rawPose.
https://www.babylonjs-playground.com/#5MV04 -
Enjoy!
My WebVR camera is not working!!
Seems like a very common problem - a WebVRFreeCamera class is initialized, but you can't see a thing in the device.
navigator.getVRDevices().then((vrs) => {console.log(vrs.length)}) . If you got 0 or an error, the device is not properly connected.The camera's rotation is changing, but i can't see a thing in my display
This error occurs when you didn't attach control to the VR device.
My (Vive) controllers are not detected!! HELP!!!!
Ah, I know this problem.
camera.initControllers() !navigator.getGamepads() in your console. Is the list empty? are there controllers in the list? what controllers?