JReality Virtual Environment Tutorial

From JReality Wiki
Jump to: navigation, search

Overview

Here we explain how to set up jreality for a VE (virtual environment), with multiple screens and tracking devices.

We start with distributed rendering, and assume the following set up:

There is one master, a computer wich manages/synchronizes the clients, several computers that are responsable for rendering to the individual screens of the VE. These machines do not need to be distinct. For instance, there might be only one single machine that does all the work. In this case, we will still have a seperate process for each screen and also a master process which keeps all in synch and reads from tracking devices etc.


There are two kinds of backends:


Synchronized SceneGraph

There is one jReality application running on the master, including a ToolSystem (reading from Tracking devices etc.) and maybe also java GUI on the masters desktop. On the clients, there is just a jreality viewer, which renders a copy of the master application scene graph. The only additional thing (except for rendering) done on the client side is the computation of the camera matrix depending on the head transformation as well as the client screen orientation and size.

PRO: This backend has the advantage that any jreality application can be run out-of-the-box in a VE.

CON: Performance. All changes to geometries and appearances are sent over the ethernet. This becomes an issue when large geometries or textures are frequently changed.


Synchronized ToolSystem

The jReality application is running on each of the clients, using a special ToolSystem. The application may also include java GUI, but the GUI needs to to be embedded into the 3D scene. On master, there is just a ToolSystem running, which reads from the tracking devices and broadcasts the ToolEvents to the clients ToolSystems. This backend is obviously faster, since there is much less traffic.

PRO: Performance. Network traffic is independent of the complexity of the scene, since only the data of the input devices is sent over the ethernet.

CON: Only immersed GUI is possible. Animations/timers need to be synchronized between the machines (The ToolSystem offers everything that is needed for this).


Both backends use the same low-level library:


SMRJ

SMRJ is a java library with similar functionality as RMI. It allows to create objects (Proxies) on the master which broadcast all method calls to objects (Clients) living on different client machines. In contrast to RMI, which always maps one local object to one remote object, SMRJ allows to map one local object to n remote objects. SMRJ also allows to call every interface method remotely, in contrast to RMI which for instance requires the remote methods to throw an RMIException.


On the client computers, a single java process (de.smrj.tcp.management.ClientService) is running. It will create a connection to a de.smrj.tcp.TCPBroadCaster, running on the master machine. Via the broadcaster, one can create Proxies, corresponding to Clients that are created on each client computer.


SMRJ also takes care of the classpath: Besides the BroadCaster, there is also a JarServer, which provides missing classpath entries to the client processes.


A simple SMRJ example

Interface:

public interface SysOut {
    void println(String str);
}

The implementation:

public class SysOutImpl implements SysOut {
    public void println(String str) {
        System.out.println(str);
    }
}


On each client (or in each shell if you use a single machine), run the following command:

 $> java -jar smrj-client.jar

This runs de.smrj.tcp.managemenbt.ClientService, which listens for UDP messages with information to which BroadCaster it should connect.


On the master, run the following main method:

import de.smrj.Broadcaster;
import de.smrj.ClientFactory;
import de.smrj.tcp.TCPBroadcasterNIO;
import de.smrj.tcp.management.Local;
import de.smrj.tcp.management.JarServer;
 
import java.net.ServerSocket;
 
public class SMRJDemo {
 
    public static void main(String[] args) throws Exception {
        // set up communication:
        int port = 8868;
        int cpPort = 8869;
        JarServer js = new JarServer(new ServerSocket(cpPort));
        Broadcaster bc = new TCPBroadcasterNIO(port, Broadcaster.RESPONSE_TYPE_EXCEPTION);
        Local.sendStart(port, cpPort, Broadcaster.RESPONSE_TYPE_EXCEPTION, ClientFactory.class);
        js.waitForDownloads();
 
        // create a local proxy and the corresponding client objects:
        SysOut proxy = bc.getRemoteFactory().createRemote(SysOutImpl.class);
 
        proxy.println("Hi there!");
    }
}


Compile and run the demo on the master machine:

 $> javac SysOut.java SysOutImpl.java
 $> javac -cp smrj.jar:. SMRJDemo.java
 $> java -cp smrj.jar:. SMRJDemo

If there are no networking issues each ClientService should have done the print out.


This is just to illustrate how the low level stuff works. Back to jReality:

VR-Demo for single Desktop with Synchronized SceneGraph

In this Demo, a master applicaton for the toolsystem and six slaves displaying a synchronized cubic scene around the avatar are started.


First you need to setup jReality with Eclipse. Afterwards go to the cave-demo folder in Eclipse and open the startup script:


Image-startupfile.jpg


Adapt your workspace path:


 JR=PATH_TO_YOUR_WORKSPACE/jreality


Now, open the "Run Configuartions" of a tutorial file, e.g. de.jreality.tutorial.viewer.JRViewerVR:

Navigate to src-tutorial/de/jreality/tutorial/viewer -> right click on "JRViewerVR.java" -> "Run As" -> left click on "Run Configurations"


Check if the Classname at the top matches the tutorial you're starting. At our case the name should be, "JRViewerVR". Below the Name, click on the tab "(x)= Arguments" and add the following three options to the "VM arguments"


  -Dde.jreality.scene.Viewer=de.jreality.portal.PortalServerViewer
  -Dde.jreality.portal.localViewer=de.jreality.jogl.Viewer
  -Dde.jreality.viewerapp.env=portal


Run configurations.jpg


Open a terminal, navigate to the cave-demo folder and start the startup script:


  cd PATH_TO_YOUR_WORKSPACE/jreality/cave-demo
  ./startup


Now there should be running six smrj-clients. You can check that with:


  ps x | grep smrj


You will get a result like this:


 574 pts/0    Sl     0:00 java -Djreality.config=demo.left -Xms128M -Xmx256M -jar ../lib/smrj-client.jar
 575 pts/0    Sl     0:00 java -Djreality.config=demo.center -Xms128M -Xmx256M -jar ../lib/smrj-client.jar
 576 pts/0    Sl     0:00 java -Djreality.config=demo.right -Xms128M -Xmx256M -jar ../lib/smrj-client.jar
 577 pts/0    Sl     0:00 java -Djreality.config=demo.floor-square -Xms128M -Xmx256M -jar ../lib/smrj-client.jar
 578 pts/0    Sl     0:00 java -Djreality.config=demo.ceiling-square -Xms128M -Xmx256M -jar ../lib/smrj-client.jar
 579 pts/0    Sl     0:00 java -Djreality.config=demo.back -Xms128M -Xmx256M -jar ../lib/smrj-client.jar
 739 pts/4    S+     0:00 grep smrj


Now run the JRViewerVR Tutorial: right click on "JRViewerVR.java" -> "Run As" -> left click "Java Application" and you should get the following results:


Results stereo.jpg


To get a better understanding of the displayed geometry toggle stereo mode:

At the MasterViewer "jReality" click at "Camera" -> "Toggle Stereo". Now you should see an Icosahedron. The toolsystem just works on the Masterviewer and the six other viewers are updated as soon as a change to the master-scene is happening.


Toggle stereo.jpg

VR-Demo for separate X with Synchronized SceneGraph

The setup right now is quite similar to the setup above for a single Desktop. But now there are two displays with separate X installed. You start again one mainviewer for the toolsystem and couple of clients. But instead of a "cubic" scene you start at the left display the screen for the left viewer and at the right the center viewer. Of course you could choose each other viewer config combination, e.g. floor and right etc. aswell. If there are more than two separate X-Screens attached you can also simple increase the number of used config files. But now back to the two separate X left/center setup


First check if the old smrj-clients are still running:


  ps x | grep smrj


If the old smrj-clients are still running, then kill their process


  kill PROCESS_ID


Now go to the cave-demo folder and edit the startup file again. Write before demo.left "DISPLAY=:0.1" and before demo.center "DISPLAY=:0.0" or, depending on your X-configuration, the other way around. Further "Comment" the other demo-configs. Afterwards the script should look like this:


  #!/bin/bash
  JR=PATH_TO_YOUR_WORKSPACE/jreality
  ARGS="-Xms128M -Xmx256M"
  DISPLAY=:0.1 java -Djreality.config=demo.left $ARGS -jar ../lib/smrj-client.jar &
  DISPLAY=:0.0 java -Djreality.config=demo.center $ARGS -jar ../lib/smrj-client.jar &
  #java -Djreality.config=demo.right $ARGS -jar ../lib/smrj-client.jar &
  #java -Djreality.config=demo.floor $ARGS -jar ../lib/smrj-client.jar &
  #java -Djreality.config=demo.ceiling $ARGS -jar ../lib/smrj-client.jar &
  #java -Djreality.config=demo.floor-square $ARGS -jar ../lib/smrj-client.jar &
  #java -Djreality.config=demo.ceiling-square $ARGS -jar ../lib/smrj-client.jar &
  #java -Djreality.config=demo.back $ARGS -jar ../lib/smrj-client.jar &


start the script und JRViewerVR with the above VR-Arguments


 -Dde.jreality.scene.Viewer=de.jreality.portal.PortalServerViewer
 -Dde.jreality.portal.localViewer=de.jreality.jogl.Viewer
 -Dde.jreality.viewerapp.env=portal


As result you'll get a mainviewer and two frames, one on your left and one on your right screen.


To change the framesize to fullscreen you have to kill the smrj-clients first.


  ps x | grep smrj
  kill PROCESS_ID


Then edit the demo.left and demo.center config-files. Change their offset positions to:


  frame.offset.x: 0
  frame.offset.y: 0


Afterwards edit the demo.props config-file. Set the frame to fullscreen (frame.fullscreen: true) and the resolution for your "viewer inside the frame" to your fullscreen resolution, e.g. ( frame.width: 1280 frame.height: 1024)


  frame.fullscreen: true
  frame.undecorated: true
  frame.width: 1280
  frame.height: 1024


start the smrj-clients again


  cd  PATH_TO_WORKSPACE/jreality/cave-demo
  ./startup


As result, now your clientviewers will have fullscreen resolution.

Running ViewerVR with Synchronized SceneGraph (OLD)

To run ViewerVR with the Synchronized SceneGraph backend, we first start the ClientServices on the client computers (which may also be the same computer as the master). The command is the same as for the simple demo, but we need to set a couple of properties for the correct initialization of ViewerVR:


 $> JR_PROPS="-Xms1024M -Xmx1536M \  
   -Djreality.jogl.quadBufferedStereo=true"
 $> java -Djreality.config={path-to-your-config-files (e.g: home/user/cave/workspace/jreality/cave-demo/portal.left} $JR_PROPS 
   -jar {path-to-smrj-client.jar (e.g.:home/user/cave/workspace/jreality/lib/smrj-client.jar)} tcp 4444


The properties have the following effects:


  • jreality.jogl.quadBufferedStereo: just what it says: do/do not use quad buffered Stereo
  • jreality.config: pass a config file that contains information on screen orientation etc.


Also start up the other Clients with config their config-files.


Now we will start the ViewerVR on the master machine, such that it uses a Viewer class that:

  • creates a BroadCaster, to which the clients will connect
  • creates a Proxy for the three copies of the scene graph on the clients
  • creates a Proxy for three viewers on the clients
  • provides a local viewer for mouse and keyboard interaction


The corresponding java command is simply


 $> java -Djava.library.path={path-to-jogl-native-libs(e.g.: home/user/cave/workspace/jreality/jni/linux64)} 
    -cp $CLASSPATH -Xms1024M -Xmx1536M $JR_MASTER_PROPS de.jreality.vr.ViewerVR


with the following properties:


 $> JR_MASTER_PROPS="-Dde.jreality.scene.Viewer=de.jreality.portal.PortalServerViewer \
    -Dde.jreality.viewerapp.env=portal \
    -Dde.jreality.scene.tool.Config=portal \ 
    -Dde.jreality.portal.localViewer=de.jreality.jogl.Viewer\
    -Dde.smrj.clients="{name of your clients (e.g: left center right)}" \
    -Dde.smrj.clients.port=4444 "


The properties have the following effects:


  • de.jreality.scene.Viewer=de.jreality.portal.PortalServerViewer: use a Viewer class which does all we have mentioned above
  • de.jreality.viewerapp.env=portal: create a Scene Graph adapted for use in a VE, for instance adds a PointerDisplayTool etc.
  • de.jreality.portal.localViewer=de.jreality.jogl.Viewer: for the local display use a jogl viewer


Note that you still need to set-up the classpath here, which has to include jReality and the 3rd-party-libs that jReality needs. The easiest thing is to run this from Eclipse, see Eclipse set-up tutorial. After eclipse has compiled the code you can use the following code for your classpath-setup


  $> for d in $(ls -d {path-to-your/workspace (e.g: /home/user/cave/workspace)}/*/); do   
     for f in $(ls ${d}lib/*.jar); do CLASSPATH=$f:$CLASSPATH; done;   CLASSPATH=${d}bin:$CLASSPATH; done


When running this on a single machine, the first thing to do would be turning off stereo in the ViewerVR Camera Menu (Toggle stereo).

Running ViewerVR with Synchronized Toolsystem (OLD)

To run ViewerVR with the Synchronized SceneGraph backend, we first need to start the ClientServices on the client computers (which may also be the same computer as the master). The command is the same as for the simple demo, but we need to set a couple of properties for the correct initialization of the ViewerVR:


 $> JR_PROPS="-Xms256M -Xmx512M \ 
   -Djava.library.path={path-to-jogl-native-libs (e.g.: home/user/cave/workspace/jreality/jni/linux64)} \ 
   -Djreality.jogl.quadBufferedStereo=true"
 $> java -Djreality.config={path-to-your-config-files (e.g: home/user/cave/workspace/jreality/cave-demo/portal.left} $JR_PROPS 
   -jar {path-to-smrj-client.jar (e.g.:home/user/cave/workspace/jreality/lib/smrj-client.jar)} tcp 4444


The properties have the following effects:


  • jreality.jogl.quadBufferedStereo: just what it says: do/do not use quad buffered Stereo
  • jreality.config: pass a config file that contains information on screen orientation etc.


Also start up the other Clients with config their config-files.


Now we will start the ViewerVR on the master machine, such that it uses a Viewer class that:

  • creates a BroadCaster, to which the clients will connect
  • creates a Proxy for the three copies of the scene graph on the clients
  • creates a Proxy for three viewers on the clients
  • provides a local viewer for mouse and keyboard interaction

The corresponding java command is simply


 $> java -Djava.library.path={path-to-jogl-native-libs(e.g.: home/user/cave/workspace/jreality/jni/linux64)} 
    -cp $CLASSPATH -Xms256M -Xmx512M $JR_MASTER_PROPS de.jreality.vr.ViewerVR


with the following properties:


 $> JR_MASTER_PROPS="-Dde.jreality.scene.Viewer=de.jreality.portal.PortalServerViewer \
    -Dde.jreality.viewerapp.env=portal \
    -Dde.jreality.scene.tool.Config=portal \ 
    -Dde.jreality.portal.localViewer=de.jreality.jogl.Viewer\
    -Dde.smrj.clients="{name of your clients (e.g: left center right)}" \
    -Dde.smrj.clients.port=4444 "


The properties have the following effects:


  • de.jreality.scene.Viewer=de.jreality.portal.PortalServerViewer: use a Viewer class which does all we have mentioned above
  • de.jreality.viewerapp.env=portal: create a Scene Graph adapted for use in a VE, for instance adds a PointerDisplayTool etc.
  • de.jreality.portal.localViewer=de.jreality.jogl.Viewer: for the local display use a jogl viewer


Note that you still need to set-up the classpath here, which has to include jReality and the 3rd-party-libs that jReality needs. The easiest thing is to run this from Eclipse, see Eclipse set-up tutorial. After eclipse has compiled the code you can use the following code for your classpath-setup


  $> for d in $(ls -d {path-to-your/workspace (e.g: /home/user/cave/workspace)}/*/); do   
     for f in $(ls ${d}lib/*.jar); do CLASSPATH=$f:$CLASSPATH; done;   CLASSPATH=${d}bin:$CLASSPATH; done


When running this on a single machine, the first thing to do would be turning off stereo in the ViewerVR Camera Menu (Toggle stereo).

How to use our CAVE-settings for synchronized ToolSystem (OLD)

For this tutorial you need a 64bit and a 32bit-version of Java6 and in your CAVE-setting should be involved 3 servers (one for left, center, right)

Create a folder in your HOME for the CAVE-setup, go and create there a second folder for the workspace.


$> mkdir cave
$> cd cave
$> mkdir workspace


To get jreality compiled and with a working classpath, setup jreality with Eclipse, see Eclipse set-up tutorial. Use ~/cave/workspace as your eclipse-workspace. Unzip our Settings


$> unzip workspace/jreality/jreality_portal_settings/jreality_portal_settings.zip 
$>  
$> cp -r jreality_portal_settings/* .
$> rm -r jreality_portal_settings


Now we will setup our configs for your system. Open the 'jrSettings'-file


$> gedit bin/jrSettings &


Change the line `export BASE=/central/home/ptldemo/cave` to `export BASE=~/cave`. Check the lines JDK_64 and JDK_32 and change, if necessary, their PATHs to your System-setup. Save and close the 'jrSettings'-file.

Now open the file 'jrFromWorkspace'-file


$> gedit bin/jrFromWorkspace &


Change the line 'killall javaJR;' to 'killall java;' and also '$JDK_32/bin/javaJR -cp $CLASSPATH $VR_ARGS $*' to '$JDK_32/bin/java -cp $CLASSPATH $VR_ARGS $*'

Now we have to toggle some lines at the configfiles:


$> gedit workspace/jreality/cave-demo/portal.* &


Follow the instructions, which are written in the files.

Finally we have to setup your involved servers.

$> gedit bin/clients.dat &

At the file 'bin/clients.dat' you find a mapping for our involved servers (l -> left; r right; c center etc.). Don't change the shortcuts (l,r,c,..), otherwise you have to change all the scripts, too. Change the name of the servers which are involved e.g `left -> leftserver`, `right -> rightserver` etc.

Save and close all files.


$> cd


now you should be able to start applications with

$> ~/cave/bin/runRemoteApp de.jreality.vr.ViewerVR


an kill them with

$> ~/cave/bin/killallclients