Tools introduction

Prelimilaries

To understand jreality tools it is necessary to explain some basic ideas and notation first:

A Tool is an object which is attached to the jreality scene graph in oder to manipulate it, the manipulation usually depends on user input, or time for animation tasks.

User input can come for instance from mouse, keyboard, joysticks, wand, space mouse, space navigator, speech recognition software, guesture recognition, tracking systems, ...

The jreality tool system abstracts all available input devices and maps them to a set of virtual devices. These virtual devices are usually available in every setup/environment, no matter if you run jreality on

• a desktop with mouse/keyboard
• a desktop with a joystick or a space navigator or a wii remote
• an immersive virtual environment (CAVE).

AxisStates and DoubleArrays

This set of virtual devices consists of AxisStates and DoubleArrays.

An AxisState represents a floating point value (ranging from -1 to 1), usually from a button or a joystick axis. Buttons are special AxisStates, which take only the values 0 and 1.

A DoubleArray always represents a 4x4 transformation matrix. 2- and 3-vectors are also represented as DoubleArrays, as the transformation's translation part.

InputSlots

Every virtual device is mapped to an InputSlot, which makes it availabe for the tools under a meaningful name.

PointerTransformation

The most important device to understand is the pointer transformation. Interaction with the 3D scene in jreality is usually pointer-based, that means the user can point to some geometry and then interact with it. For instance, if you want to drag a control point of a polygon, you point on that vertex (usually with the mouse pointer), then you press a button and drag the point. In a CAVE the user may have a wand which he uses to drag that vertex, or he might have a SensAble 6DOF input device.

To make this pointer interaction independent of the available devices, the tools will be driven by a virtual Pointer device, mapped to the input slot PointerTransformation. This transformation has the following meaning:

• translation part: the point in the 3D scene, where the pointer starts
• z-axis: the pointer direction in the 3D scene
• x- and y-axis: form together with the z-axis an ONB

From a mouse pointer, this transformation is obtained as follows:

• translation part: the mouse point on the near-clipping plane in scene coordinates
• z-axis: the direction in which the mouse pointer points
• x- and y-axis: computed in such a way that they are as close as possible to the screen coordinate system in scene coordinates

Querying the state of a virtual device

Querying the state of a device is done by method calls like:

// get an AxisState value:
AxisState as = context.getAxisState(InputSlot.getDevice("SlotName")); 
// get a DoubleArray value:
DoubleArray da = context.getTransformationMatrix(InputSlot.getDevice("SlotName"));

Tools

Conceptually, we destinguish two sorts of tools, that are:

• Pointer Tools: Tools that perform Pointer-based interaction (like dragging, rotating)
• Always-active Tools: Tools that perform interaction that is not associated to pointing at something (like navigation, animation)

Therefore, a tool defines a list of activation slots. This list may be empty, then we have an always-active tool.

Pointer tools are only activated when the user points to some geometry in the scene with the pointer device.

For the interaction with an active tool it defines also another list of slots, the current slots.

The list of activation slots of a tool is fixed, the list of current slots may be changed as desired. For instance, a tool may require the SystemTime slot for doing some animation, and when the animation is finished it removes this slot from its current slot list.

Activation slots always need to correspond to an AxisState. The tool becomes active as soon as one of its activation slots changes its state to AxisState.PRESSED. It gets deactivated again as soon as none of its activation slots corresponds to an axis state with value AxisState.PRESSED.

Current slots can be both AxisStates and DoubleArrays. Whenever the value corresponding to a current slot changes, the tool will be notified (but only if the tool is active).

The basic API of a tool are the three methods

• activate(ToolContext context): called as soon as the tool gets activated
• perform(ToolContext context): called when a tool is active and the value of one of the current slots has changed
• deactivate(ToolContext context): called as soon as the tool gets deactivated

The ToolContext gives access to all information a tool may require. It allows querying the states of all InputSlots, it provides information about the current pick result and gives the SceneGraphPaths a tool may require.

For instance, we can read the value of some AxisState:

AxisState axisState = context.getAxisState("ForwardBackwardAxis");

This axis is for instance mapped to the two keys W and S on a desktop, or to a joystick. On a desktop, the value will be either 1, 0 or -1 (as double value, the int value will be between -Integer.MAX_VALUE and Integer.MAX_VALUE). For an analog joystick it may take any value between 0 and 1. To read the value of the axis state, you can use

axisState.getDoubleValue();
axisState.isPressed();
axisState.isReleased();
axisState.intValue();
axisState.doubleValue();

Or we can get the current pointer tranformation by

DoubleArray da = tc.getTransformationMatrix(InputSlot.getDevice("PointerTransformation");

Now we can access all matrix entries by da.getValue(idx);, but it is usually more useful to initialize a matrix from that DoubleArray:

Matrix mat = new Matrix(da);
FactoredMatrix mat = new FactoredMatrix(da);

The FactoredMatrix provides things such as the rotation axis, the scaling etc. of the given matrix, which may be very useful when writing a tool...

The ToolContext also gives access to the location of the tool inside the scene graph. Since the same tool may occur in different places of the scene, it is necessary to obtain the information from the ToolContext. The corresponding information is always a SceneGraphPath, starting at the scene root. There are always two paths, one is to the component where the tool is located, the other one to the component which was picked for activation (for always active tools both paths are the same). The code is

// path to the picked component at activation:
SceneGraphPath pathToActivationComponent = context.getRootToLocal();
// path to the tool's component
SceneGraphPath pathToTool = context.getRootToToolComponent();

Since the root to local path corresponds to the activation path, it will not change during successive calls of the perform method.