UTS Data Arena
UTS Data Arena
projectsdive inaboutcalendar

Unreal Engine

Follow our ongoing process of integrating Unreal Engine and nDisplay into the Data Arena.

9 Dec 2020 16:12

Live interaction with our Motion Capture trackers

Pickup, move, rotate and drop with one hand

Since our previous update, we've achieved fluent interaction with dynamic objects with our Motion Capture "Tracking Markers" (aka "tracker"). Each tracker - a plastic 3D Mouse - controls a cursor on screen. A flip of the tracker "grabs" an object. Move around the Data Arena and the object's position is moved in the scene, with realtime lighting and shadows.

Move towards or away from the screen to push objects further or closer in the virtual room. Rotate your tracker like a jar lid. The object turns accordingly. Flip your tracker to "select" and "release" objects. Lifted furniture drops to the floor in its new place.

We have written this functionality into a DA_Marker class, to be shared with Unreal Engine users, for multi-user interaction. This class handles most of the work connecting to our system. It provides simple functions such as Get Location in DA, Get Relative Rotation, as well as a trigger-event for when a tracker is flipped.

This class will be in the UTS Data Arena Unreal Engine Content Pack - available soon.

9 Dec 2020 16:12

Realtime layout designing in the Cozy Living Room

Live input and interaction

The video above shows recent progress, using the Cozy Living Room model. We demonstrate realtime object manipulation with dynamic lighting. When viewed in the Data Arena we seem to be standing in the room. With Active Shutter Glasses the view is 3D Stereoscopic in 360 degrees.

A Sony DualShock PlayStation-4 controller has been integrated. It moves a 3D cursor to select, pickup, turn and place furniture. Lifted-objects cast shadows on the floor and walls. Our linux version of nDisplay runs this demonstration across the Data Arena's multi-computer cluster (7 Computers, 15 GPUs). Multi-computer means higher display performance.

Several 3D cursors are also on display. Each is controlled by a plastic Optitrack marker held by a user in the theatre. The markers operate like a magic wand. In the next development, particular marker-gestures will make these cursors select and move furniture. Then everyone in the Data Arena can interact with the model.

This is a significant milestone towards our goal of multi-user participation in real time visualisations. Immediate applications include existing UE4 Architectural Model Walkthroughs and Interior Design/Layout composition.

People can design, layout, and decide as a group.

9 Dec 2020 16:12

Progress with nDisplay

Running the latest 4.25.2 build

We had some huge success and progress with our UE4 implementation a few weeks ago (we've been busy and haven't had time to document!). Darren has been working on modifying the latest version of 4.25 so we can use nDisplay on our Linux cluster. Although it's not particularly pretty, here it is in action.

darren_slack.png
PANO_20200730_154502.vr.jpg

What am I looking at here?

Each "screen" (with nDisplay Screen Preview written) is being computed and drawn from a separate Linux machine in our cluster. On top of this, each machine is also drawing in stereo, so a separate image for the left and right eye. UE4 doesn't have official support for nDisplay on Linux (Windows only) at the time of writing, so this has been a major achievement and a big step towards an eventual Pull Request into the official Epic Games Unreal Engine repo.

Going forward from here we'll be able to continue using the latest version of UE4, starting new projects and displaying them in the Data Arena. Here's the Cozy Living Room project we've been working on running in real-time stereo in the Data Arena.

IMG_20200904_114927.jpg
IMG_20200904_114903.jpg
Pages
1234567