JReality Virtual Environment Tutorial
Contents
Overview
Here we explain how to set up jreality for a VE (virtual environment), with multiple screens and tracking devices.
We start with distributed rendering, and assume the following set up:
There is one master, a computer wich manages/synchronizes the clients, several computers that are responsable for rendering to the individual screens of the VE. These machines do not need to be distinct. For instance, there might be only one single machine that does all the work. In this case, we will still have a seperate process for each screen and also a master process which keeps all in synch and reads from tracking devices etc.
There are two kinds of backends:
Synchronized SceneGraph
There is one jReality application running on the master, including a ToolSystem (reading from Tracking devices etc.) and maybe also java GUI on the masters desktop. On the clients, there is just a jreality viewer, which renders a copy of the master application scene graph. The only additional thing (except for rendering) done on the client side is the computation of the camera matrix depending on the head transformation as well as the client screen orientation and size.
PRO: This backend has the advantage that any jreality application can be run out-of-the-box in a VE.
CON: Performance. All changes to geometries and appearances are sent over the ethernet. This becomes an issue when large geometries or textures are frequently changed.
Synchronized ToolSystem
The jReality application is running on each of the clients, using a special ToolSystem. The application may also include java GUI, but the GUI needs to to be embedded into the 3D scene. On master, there is just a ToolSystem running, which reads from the tracking devices and broadcasts the ToolEvents to the clients ToolSystems. This backend is obviously faster, since there is much less traffic.
PRO: Performance. Network traffic is independent of the complexity of the scene, since only the data of the input devices is sent over the ethernet.
CON: Only immersed GUI is possible. Animations/timers need to be synchronized between the machines (The ToolSystem offers everything that is needed for this).
Both backends use the same low-level library:
SMRJ
SMRJ is a java library with similar functionality as RMI. It allows to create objects (Proxies) on the master which broadcast all method calls to objects (Clients) living on different client machines. In contrast to RMI, which always maps one local object to one remote object, SMRJ allows to map one local object to n remote objects. SMRJ also allows to call every interface method remotely, in contrast to RMI which for instance requires the remote methods to throw an RMIException.
On the client computers, a single java process (de.smrj.tcp.management.ClientService) is running. It will create a connection to a de.smrj.tcp.TCPBroadCaster, running on the master machine. Via the broadcaster, one can create Proxies, corresponding to Clients that are created on each client computer.
SMRJ also takes care of the classpath: Besides the BroadCaster, there is also a JarServer, which provides missing classpath entries to the client processes.
A simple SMRJ example
Interface:
public interface SysOut { void println(String str); }
The implementation:
public class SysOutImpl implements SysOut { public void println(String str) { System.out.println(str); } }
On each client (or in each shell if you use a single machine), run the following command:
$> java -jar smrj-client.jar
This runs de.smrj.tcp.managemenbt.ClientService, which listens for UDP messages with information to which BroadCaster it should connect.
On the master, run the following main method:
import de.smrj.Broadcaster; import de.smrj.ClientFactory; import de.smrj.tcp.TCPBroadcasterNIO; import de.smrj.tcp.management.Local; import de.smrj.tcp.management.JarServer; import java.net.ServerSocket; public class SMRJDemo { public static void main(String[] args) throws Exception { // set up communication: int port = 8868; int cpPort = 8869; JarServer js = new JarServer(new ServerSocket(cpPort)); Broadcaster bc = new TCPBroadcasterNIO(port, Broadcaster.RESPONSE_TYPE_EXCEPTION); Local.sendStart(port, cpPort, Broadcaster.RESPONSE_TYPE_EXCEPTION, ClientFactory.class); js.waitForDownloads(); // create a local proxy and the corresponding client objects: SysOut proxy = bc.getRemoteFactory().createRemote(SysOutImpl.class); proxy.println("Hi there!"); } }
Compile and run the demo on the master machine:
$> javac SysOut.java SysOutImpl.java $> javac -cp smrj.jar:. SMRJDemo.java $> java -cp smrj.jar:. SMRJDemo
If there are no networking issues each ClientService should have done the print out.
This is just to illustrate how the low level stuff works. Back to jReality:
VR-Demo for single Desktop with Synchronized SceneGraph
In this Demo, a master applicaton for the toolsystem and six slaves displaying a synchronized cubic scene around the avatar are started.
First you need to setup jReality with Eclipse. Afterwards go to the cave-demo folder in Eclipse and open the startup script:
Adapt your workspace path:
JR=PATH_TO_YOUR_WORKSPACE/jreality
Now, open the "Run Configuartions" of a tutorial file, e.g. de.jreality.tutorial.viewer.JRViewerVR:
Navigate to src-tutorial/de/jreality/tutorial/viewer -> right click on "JRViewerVR.java" -> "Run As" -> left click on "Run Configurations"
Check if the Classname at the top matches the tutorial you're starting. At our case the name should be, "JRViewerVR". Below the Name, click on the tab "(x)= Arguments" and add the following three options to the "VM arguments"
-Dde.jreality.scene.Viewer=de.jreality.portal.PortalServerViewer -Dde.jreality.portal.localViewer=de.jreality.jogl.Viewer -Dde.jreality.viewerapp.env=portal
Open a terminal, navigate to the cave-demo folder and start the startup script:
cd PATH_TO_YOUR_WORKSPACE/jreality/cave-demo ./startup
Now there should be running six smrj-clients. You can check that with:
ps x | grep smrj
You will get a result like this:
574 pts/0 Sl 0:00 java -Djreality.config=demo.left -Xms128M -Xmx256M -jar ../lib/smrj-client.jar 575 pts/0 Sl 0:00 java -Djreality.config=demo.center -Xms128M -Xmx256M -jar ../lib/smrj-client.jar 576 pts/0 Sl 0:00 java -Djreality.config=demo.right -Xms128M -Xmx256M -jar ../lib/smrj-client.jar 577 pts/0 Sl 0:00 java -Djreality.config=demo.floor-square -Xms128M -Xmx256M -jar ../lib/smrj-client.jar 578 pts/0 Sl 0:00 java -Djreality.config=demo.ceiling-square -Xms128M -Xmx256M -jar ../lib/smrj-client.jar 579 pts/0 Sl 0:00 java -Djreality.config=demo.back -Xms128M -Xmx256M -jar ../lib/smrj-client.jar 739 pts/4 S+ 0:00 grep smrj
Now run the JRViewerVR Tutorial: right click on "JRViewerVR.java" -> "Run As" -> left click "Java Application" and you should get the following results:
To get a better understanding of the displayed geometry toggle stereo mode:
At the MasterViewer "jReality" click at "Camera" -> "Toggle Stereo". Now you should see an Icosahedron. The toolsystem just works on the Masterviewer and the six other viewers are updated as soon as a change to the master-scene is happening.
VR-Demo for separate X with Synchronized SceneGraph
The setup right now is quite similar to the setup above for a single Desktop. But now there are two displays with separate X installed. You start again one mainviewer for the toolsystem and couple of clients. But instead of a "cubic" scene you start at the left display the screen for the left viewer and at the right the center viewer. Of course you could choose each other viewer config combination, e.g. floor and right etc. aswell. If there are more than two separate X-Screens attached you can also simple increase the number of used config files. But now back to the two separate X left/center setup
First check if the old smrj-clients are still running:
ps x | grep smrj
If the old smrj-clients are still running, then kill their process
kill PROCESS_ID
Now go to the cave-demo folder and edit the startup file again. Write before demo.left "DISPLAY=:0.1" and before demo.center "DISPLAY=:0.0" or, depending on your X-configuration, the other way around. Further "Comment" the other demo-configs. Afterwards the script should look like this:
#!/bin/bash JR=PATH_TO_YOUR_WORKSPACE/jreality ARGS="-Xms128M -Xmx256M" DISPLAY=:0.1 java -Djreality.config=demo.left $ARGS -jar ../lib/smrj-client.jar & DISPLAY=:0.0 java -Djreality.config=demo.center $ARGS -jar ../lib/smrj-client.jar & #java -Djreality.config=demo.right $ARGS -jar ../lib/smrj-client.jar & #java -Djreality.config=demo.floor $ARGS -jar ../lib/smrj-client.jar & #java -Djreality.config=demo.ceiling $ARGS -jar ../lib/smrj-client.jar & #java -Djreality.config=demo.floor-square $ARGS -jar ../lib/smrj-client.jar & #java -Djreality.config=demo.ceiling-square $ARGS -jar ../lib/smrj-client.jar & #java -Djreality.config=demo.back $ARGS -jar ../lib/smrj-client.jar &
start the script und JRViewerVR with the above VR-Arguments
-Dde.jreality.scene.Viewer=de.jreality.portal.PortalServerViewer -Dde.jreality.portal.localViewer=de.jreality.jogl.Viewer -Dde.jreality.viewerapp.env=portal
As result you'll get a mainviewer and two frames, one on your left and one on your right screen.
To change the framesize to fullscreen you have to kill the smrj-clients first.
ps x | grep smrj kill PROCESS_ID
Then edit the demo.left and demo.center config-files. Change their offset positions to:
frame.offset.x: 0 frame.offset.y: 0
Afterwards edit the demo.props config-file. Set the frame to fullscreen (frame.fullscreen: true) and the resolution for your "viewer inside the frame" to your fullscreen resolution, e.g. ( frame.width: 1280 frame.height: 1024)
frame.fullscreen: true frame.undecorated: true frame.width: 1280 frame.height: 1024
start the smrj-clients again
cd PATH_TO_WORKSPACE/jreality/cave-demo ./startup
As result, now your clientviewers will have fullscreen resolution.
Running ViewerVR with Synchronized SceneGraph (OLD)
To run ViewerVR with the Synchronized SceneGraph backend, we first start the ClientServices on the client computers (which may also be the same computer as the master). The command is the same as for the simple demo, but we need to set a couple of properties for the correct initialization of ViewerVR:
$> JR_PROPS="-Xms1024M -Xmx1536M \ -Djreality.jogl.quadBufferedStereo=true" $> java -Djreality.config={path-to-your-config-files (e.g: home/user/cave/workspace/jreality/cave-demo/portal.left} $JR_PROPS -jar {path-to-smrj-client.jar (e.g.:home/user/cave/workspace/jreality/lib/smrj-client.jar)} tcp 4444
The properties have the following effects:
- jreality.jogl.quadBufferedStereo: just what it says: do/do not use quad buffered Stereo
- jreality.config: pass a config file that contains information on screen orientation etc.
Also start up the other Clients with config their config-files.
Now we will start the ViewerVR on the master machine, such that it uses a Viewer class that:
- creates a BroadCaster, to which the clients will connect
- creates a Proxy for the three copies of the scene graph on the clients
- creates a Proxy for three viewers on the clients
- provides a local viewer for mouse and keyboard interaction
The corresponding java command is simply
$> java -Djava.library.path={path-to-jogl-native-libs(e.g.: home/user/cave/workspace/jreality/jni/linux64)} -cp $CLASSPATH -Xms1024M -Xmx1536M $JR_MASTER_PROPS de.jreality.vr.ViewerVR
with the following properties:
$> JR_MASTER_PROPS="-Dde.jreality.scene.Viewer=de.jreality.portal.PortalServerViewer \ -Dde.jreality.viewerapp.env=portal \ -Dde.jreality.scene.tool.Config=portal \ -Dde.jreality.portal.localViewer=de.jreality.jogl.Viewer\ -Dde.smrj.clients="{name of your clients (e.g: left center right)}" \ -Dde.smrj.clients.port=4444 "
The properties have the following effects:
- de.jreality.scene.Viewer=de.jreality.portal.PortalServerViewer: use a Viewer class which does all we have mentioned above
- de.jreality.viewerapp.env=portal: create a Scene Graph adapted for use in a VE, for instance adds a PointerDisplayTool etc.
- de.jreality.portal.localViewer=de.jreality.jogl.Viewer: for the local display use a jogl viewer
Note that you still need to set-up the classpath here, which has to include jReality and the 3rd-party-libs that jReality needs. The easiest thing is to run this from Eclipse, see Eclipse set-up tutorial. After eclipse has compiled the code you can use the following code for your classpath-setup
$> for d in $(ls -d {path-to-your/workspace (e.g: /home/user/cave/workspace)}/*/); do for f in $(ls ${d}lib/*.jar); do CLASSPATH=$f:$CLASSPATH; done; CLASSPATH=${d}bin:$CLASSPATH; done
When running this on a single machine, the first thing to do would be turning off stereo in the ViewerVR Camera Menu (Toggle stereo).
Running ViewerVR with Synchronized Toolsystem (OLD)
To run ViewerVR with the Synchronized SceneGraph backend, we first need to start the ClientServices on the client computers (which may also be the same computer as the master). The command is the same as for the simple demo, but we need to set a couple of properties for the correct initialization of the ViewerVR:
$> JR_PROPS="-Xms256M -Xmx512M \ -Djava.library.path={path-to-jogl-native-libs (e.g.: home/user/cave/workspace/jreality/jni/linux64)} \ -Djreality.jogl.quadBufferedStereo=true" $> java -Djreality.config={path-to-your-config-files (e.g: home/user/cave/workspace/jreality/cave-demo/portal.left} $JR_PROPS -jar {path-to-smrj-client.jar (e.g.:home/user/cave/workspace/jreality/lib/smrj-client.jar)} tcp 4444
The properties have the following effects:
- jreality.jogl.quadBufferedStereo: just what it says: do/do not use quad buffered Stereo
- jreality.config: pass a config file that contains information on screen orientation etc.
Also start up the other Clients with config their config-files.
Now we will start the ViewerVR on the master machine, such that it uses a Viewer class that:
- creates a BroadCaster, to which the clients will connect
- creates a Proxy for the three copies of the scene graph on the clients
- creates a Proxy for three viewers on the clients
- provides a local viewer for mouse and keyboard interaction
The corresponding java command is simply
$> java -Djava.library.path={path-to-jogl-native-libs(e.g.: home/user/cave/workspace/jreality/jni/linux64)} -cp $CLASSPATH -Xms256M -Xmx512M $JR_MASTER_PROPS de.jreality.vr.ViewerVR
with the following properties:
$> JR_MASTER_PROPS="-Dde.jreality.scene.Viewer=de.jreality.portal.PortalServerViewer \ -Dde.jreality.viewerapp.env=portal \ -Dde.jreality.scene.tool.Config=portal \ -Dde.jreality.portal.localViewer=de.jreality.jogl.Viewer\ -Dde.smrj.clients="{name of your clients (e.g: left center right)}" \ -Dde.smrj.clients.port=4444 "
The properties have the following effects:
- de.jreality.scene.Viewer=de.jreality.portal.PortalServerViewer: use a Viewer class which does all we have mentioned above
- de.jreality.viewerapp.env=portal: create a Scene Graph adapted for use in a VE, for instance adds a PointerDisplayTool etc.
- de.jreality.portal.localViewer=de.jreality.jogl.Viewer: for the local display use a jogl viewer
Note that you still need to set-up the classpath here, which has to include jReality and the 3rd-party-libs that jReality needs. The easiest thing is to run this from Eclipse, see Eclipse set-up tutorial. After eclipse has compiled the code you can use the following code for your classpath-setup
$> for d in $(ls -d {path-to-your/workspace (e.g: /home/user/cave/workspace)}/*/); do for f in $(ls ${d}lib/*.jar); do CLASSPATH=$f:$CLASSPATH; done; CLASSPATH=${d}bin:$CLASSPATH; done
When running this on a single machine, the first thing to do would be turning off stereo in the ViewerVR Camera Menu (Toggle stereo).
How to use our CAVE-settings for synchronized ToolSystem (OLD)
For this tutorial you need a 64bit and a 32bit-version of Java6 and in your CAVE-setting should be involved 3 servers (one for left, center, right)
Create a folder in your HOME for the CAVE-setup, go and create there a second folder for the workspace.
$> mkdir cave $> cd cave $> mkdir workspace
To get jreality compiled and with a working classpath, setup jreality with Eclipse, see Eclipse set-up tutorial. Use ~/cave/workspace as your eclipse-workspace. Unzip our Settings
$> unzip workspace/jreality/jreality_portal_settings/jreality_portal_settings.zip $> $> cp -r jreality_portal_settings/* . $> rm -r jreality_portal_settings
Now we will setup our configs for your system. Open the 'jrSettings'-file
$> gedit bin/jrSettings &
Change the line `export BASE=/central/home/ptldemo/cave` to `export BASE=~/cave`.
Check the lines JDK_64 and JDK_32 and change, if necessary, their PATHs to your System-setup. Save and close the 'jrSettings'-file.
Now open the file 'jrFromWorkspace'-file
$> gedit bin/jrFromWorkspace &
Change the line 'killall javaJR;' to 'killall java;' and also '$JDK_32/bin/javaJR -cp $CLASSPATH $VR_ARGS $*' to '$JDK_32/bin/java -cp $CLASSPATH $VR_ARGS $*'
Now we have to toggle some lines at the configfiles:
$> gedit workspace/jreality/cave-demo/portal.* &
Follow the instructions, which are written in the files.
Finally we have to setup your involved servers.
$> gedit bin/clients.dat &
At the file 'bin/clients.dat' you find a mapping for our involved servers (l -> left; r right; c center etc.). Don't change the shortcuts (l,r,c,..), otherwise you have to change all the scripts, too. Change the name of the servers which are involved e.g `left -> leftserver`, `right -> rightserver` etc.
Save and close all files.
$> cd
now you should be able to start applications with
$> ~/cave/bin/runRemoteApp de.jreality.vr.ViewerVR
an kill them with
$> ~/cave/bin/killallclients