Adding Points Interactively Outside SceneGraphComponents

[karunaMaitri]

Hello,

For the application I am developing, I need to allow the user to enter data points that are outside the existing SceneGraphComponents. I do not know how to do this. These points can also be inside or outside the existing bounding box (it is very constraining to restrict the user to a prespecified bounding box). An example is allowing the user to enter a new control point for a NURBS curve/ surface. Another example is for the user to enter data by clicking on the panel and then my application interpolates through the data points to generate a surface or a curve. From the data points entered, my application creates a new SceneGraphComponent or extends an existing SceneGraphComponent.

I have looked at various TOOLS and their demos in tutorials section of jReality. As far as I have seen, the TOOLS support the user interactions with the SceneGraphComponent. I think I have three options.

1. I create a TOOL that works outside the SceneGraphComponents by extending AbstractTool. If this can be done, can you suggest me how I can do this?

2. I create SceneGraphComponents for a 2D curve or a 3D surface as follows: I create a plane (a single face) for 2D curve and draw the curve on the face. Then I attach the Tool to the SceneGraphComponent containing the face. In case of a 3D surface, I embed the surface in a bounded 3D space - an invisible cube. Is there a simple method than doing these?

3. I access the panel on which rendering is done, and use JOGL methods to listen to user inputs. Are there better ways of doing things than this?

I shall appreciate your suggestions.

Karuna.

[steffen]

Hi, I have just added a simple example to the SVN. It is under src-tutorial, the classname is de.jreality.tutorial.tool.AddPointsExample

Just post any questions here. Run the example and shift+left mouse will add points.

[karunaMaitri]

Thank you very much for the example. It is very helpful! I will post messages if I find problems using it or building upon it in my application.

Karuna

[karunaMaitri]

I have couple of questions using tools in jReality. It took me some time to continue this thread as I have been trying to interface the jReality Tools with my programs in a particular way. I need "application centric" and "object group centric" tools. These are slightly higher level tools using the jReality tools. An example is to consider a curve and a control polygon as a configuration of geometric objects, and associate a tool with the configuration for a specific behavior.

An example of behavior is changing the curve shape by dragging the control polygon points. Here, I have encountered a number of problems. I have an elaborate group of java classes that set up the object configurations for applications. I tried to use DragEventTool and PointDragListener. I put my code in the pointDragged method (similar to DragEventTool01 in tutorials). The code not only modifies pointSet of the SceneGraphComponent for control polygon, but also searches for the SceneGraphComponent used for curve and then updates it similar to updateContent method of ModelExample (Tools tutorial).

This is causing problems - I am getting some multi-threading error messages. I am using complex algorithms for curve (and surface) modifications and they take time.

I am not sure why I am getting this error. I found one problem due to the way pointDragged method works.My algorithm expects the start and end points of the drag event. But the pointDragged method generates several points, one for each small movement of the mouse.

One solution seems to be to use pointDragStart and pointDragEnd methods. But pointDragEnd is producing same coordinates as pointDragStart. There may be a bug in jReality code.

I would like to know if there are any (such as time of computation) issues in putting complex code in event listeners?

I am trying to use perform method to take care of drag events. I am not sure how to declare InputSlots for the click of left mouse button and release of left mouse button. Can you let me know.

Thank you very much!
Karuna

[STRESS]

I am trying to use perform method to take care of drag events. I am not sure how to declare InputSlots for the click of left mouse button and release of left mouse button. Can you let me know.

Thank you very much!
Karuna

Have a look at toolconfig.xml all the input slots and how they are connected is configured in there, it should be easy to find out the different mouse button.

Actually let me for the left mouse button the InputSlot is called PrimaryAction

[steffen]

Hi, just some remarks:
1. We have added the constants InputSlot.LEFT_MOUSE etc. to make life easier... ;-)
2. We are currently splitting the toolconfig.xml file into smaller parts to make them more readable, I think we can switch to these files the next two weeks.
3. Your issue with the DragEventTool: The wrong end-point coordinates sounds really like a bug. Can you just save the coordinates in pointDragged and then apply the last point in dragEnd? I have no time right now to debug the DragEventTool...

[karunaMaitri]

Thank you for the prompt replies and suggestions.
I did look at toolconfig.xml before. From the source code and tutorial examples I understand that the way jReality captures "released" or "pressed" is as an AxisState of the specific type of InputSlot. As far as I can see, these are defined inside AxisState class, not in toolconfig.xml. Thank you for suggesting that I can use InputSlot.LEFT_BUTTON.

I tried to write a program that exhibits behavior I stated in my previous message. This is a simplification. It sets up two curves with points. I used several ideas from your AddPointsExample tutorial. I used AbstractTool class and perform method. The behavior is as follows - if we click (shift left click) on a point (vertex) on the second curve, and drag it and release it, both curves should be reshaped.

But it is not working. I cannot even enter perform method. I do not understand. I included the code here. I shall appreciate if you can tell me what the problem is.

Code: Select all

import de.jreality.math.Matrix;
import de.jreality.math.Rn;
import de.jreality.plugin.JRViewer;
import de.jreality.scene.Geometry;
import de.jreality.scene.pick.PickResult;
import de.jreality.scene.tool.AbstractTool;
import de.jreality.scene.tool.InputSlot;
import de.jreality.scene.tool.ToolContext;
import de.jreality.toolsystem.ToolUtility;

public class DragExample4 extends AbstractTool {
    private double[] startPoint;
    private int startIndex;
    private double[] endPoint;
    private int endIndex;
    private boolean mousePressed = false;
    private boolean mouseReleased = true;
    private double[][] controlPoints;
    private GeometricObjectSceneGraphNode curve1SGC;
    private GeometricObjectSceneGraphNode curve2SGC;
    private double[][] curvePoints;

    public static void main(String[] args) {
        DragExample4 test = new DragExample4();
    }

    public DragExample4() {
        addCurrentSlot(InputSlot.SHIFT_LEFT_BUTTON, "add a new point");
        curvePoints = new double[][]{{0.0, 0.0, 0.0}, {10.0, 10.0, 0.0}, {10.0, 20.0, 0.0}, {20.0, 20.0, 0.0}, {20.0, 10.0, 0.0}, {30.0, 10.0, 0.0}};
        curve1SGC = new GeometricObjectSceneGraphNode();
        CurveGeometryObject curveObject = new CurveGeometryObject();
        curveObject.updateGeometry(curvePoints);
        curve1SGC.updateGeometry(curveObject);

        controlPoints = new double[][]{{5.0, 5.0, 0.0}, {15.0, 15.0, 0.0}, {15.0, 25.0, 0.0}, {25.0, 25.0, 0.0}, {25.0, 15.0, 0.0}, {35.0, 15.0, 0.0}};
        curve2SGC = new GeometricObjectSceneGraphNode();
        CurveGeometryObject curveObject2 = new CurveGeometryObject();
        curveObject2.updateGeometry(controlPoints);
        curve2SGC.updateGeometry(curveObject2);

        curve1SGC.addChild(curve2SGC);
        JRViewer.display(curve1SGC);
    }

    @Override
    public void perform(ToolContext tc) {
        System.out.println("Entering perform method");
        if ((!tc.getAxisState(InputSlot.SHIFT_LEFT_BUTTON).isPressed()) ||
                (!tc.getAxisState(InputSlot.SHIFT_LEFT_BUTTON).isReleased())) {
            System.out.println("Entering perform method");
            return;
        }

        // determine the pointer transformation:
        // translation is the mouse pointer on the near clipping plane
        // z-axis is the direction of the mouse ray out of the screen
        // for a 6DOF input device, it is the position/orientation of the device
        Matrix m = new Matrix(tc.getTransformationMatrix(InputSlot.POINTER_TRANSFORMATION));

        // we compute the coordinates of the new point in world coordinates
        double[] foot = m.getColumn(3);
        double[] dir = m.getColumn(2);
        double[] offset = Rn.times(null, -5, dir);
        double[] newPoint = Rn.add(null, foot, offset);

        // now we transform the world coordinates to the coordinate system of the tool component
        if (tc.getAxisState(InputSlot.SHIFT_LEFT_BUTTON).isPressed() && !mousePressed) {
            startPoint = ToolUtility.worldToLocal(tc, newPoint);
            if (!oneOfControlPoints(startPoint))
                return;
            else {
                mousePressed = true;
               PickResult currentPick = tc.getCurrentPick();
                startIndex = currentPick.getIndex();
                System.out.println("Index of Picked Point: " + startIndex);
                Geometry geom = (Geometry) currentPick.getPickPath().getLastElement();
                double[] pickPoint = currentPick.getObjectCoordinates();
                System.out.println("Point Picked: [" + pickPoint[0] + ", " + pickPoint[1] + ", " + pickPoint[2] + "]");
            }
        }
        if (tc.getAxisState(InputSlot.SHIFT_LEFT_BUTTON).isReleased() && mousePressed && !mouseReleased) {
            endPoint = ToolUtility.worldToLocal(tc, newPoint);
            mousePressed = false;
            mouseReleased = true;
            updateGeometry();
        }
    }

    public void updateGeometry() {
        controlPoints[startIndex] = endPoint;
        CurveGeometryObject geomObject = new CurveGeometryObject();
        geomObject.updateGeometry(controlPoints);
        curve2SGC.updateGeometry(geomObject);

        endPoint[0] += 10;
        endPoint[1] += 10;
        curvePoints[startIndex] = endPoint;
        CurveGeometryObject geomObject2 = new CurveGeometryObject();
        geomObject2.updateGeometry(curvePoints);
        curve1SGC.updateGeometry(geomObject2);
    }

    public boolean oneOfControlPoints(double[] point) {
        boolean cond = false;
        for (int i = 0; i < controlPoints.length; i++) {
            if (controlPoints[i][0] == point[0] &&
                    controlPoints[i][1] == point[1] &&
                    controlPoints[i][2] == point[2])
                return true;
        }
        return cond;
    }
}


public class CurveGeometryObject implements GeometryObject {
    private double[][] points;

    public void updateGeometry(double[][] points) {
        this.points = points;
    }

    public GeometryObject createGeometryObject(double[][] points) {
        this.points = points;
        return this;
    }

    public double[][] getPoints() {
        return points;
    }
}


public class GeometricObjectSceneGraphNode extends SceneGraphComponent {
    private IndexedLineSetFactory lsf;
    private double[][] points;
    private DefaultGeometryShader dgs;
    private Color[] colors;

    public GeometricObjectSceneGraphNode() {
        lsf = new IndexedLineSetFactory();
        colors = createColorList(10);
        // getAppearance().setAttribute(LINE_SHADER+"."+POLYGON_SHADER+"."+SMOOTH_SHADING, true);

    }

    public void updateGeometry(GeometryObject gObject) {
        CurveGeometryObject curveObject = (CurveGeometryObject) gObject;
        double[][] points = curveObject.getPoints();
        int n = points.length;

        // set new vertices
        /* lsf.setVertexCount(n);
        if (n > 0) lsf.setVertexCoordinates(points);

        if (n > 1) {
            // compute and set new edge indices:
            int[][] idx = new int[n - 1][2];
            for (int i = 1; i < n; i++) {
                idx[i - 1][0] = i - 1;
                idx[i - 1][1] = i;
            }

            lsf.setEdgeCount(n - 1);
            lsf.setEdgeIndices(idx);
        }*/

        double[][] vertices = points;
        int[][] edgeIndices = createEdgeIndices(vertices);

        lsf.setVertexCount(vertices.length);
        lsf.setVertexCoordinates(vertices);
        lsf.setVertexNormals(vertices);
        lsf.setEdgeCount(edgeIndices.length);
        lsf.setEdgeIndices(edgeIndices);

        lsf.update();

        setGeometry(lsf.getPointSet());
        setGeometry(lsf.getIndexedLineSet());

        Appearance ap = new Appearance();
        setupAppearance(ap);
        setAppearance(ap);
    }

    public void setUpAppearance(Appearance ap) {
        ap = getAppearance();
        ap.setAttribute(LINE_SHADER+"."+POLYGON_SHADER+"."+SMOOTH_SHADING, true);
        dgs = ShaderUtility.createDefaultGeometryShader(ap, true);
        dgs.setShowPoints(false);
        dgs.setShowLines(true);
        RenderingHintsShader rhs = ShaderUtility.createDefaultRenderingHintsShader(ap, true);
        DefaultLineShader dls = (DefaultLineShader) dgs.createLineShader("default");
        dls.setTubeDraw(false);
        dls.setLineWidth(3.0);
        dls.setDiffuseColor(Color.BLUE);
    }

    private void setupAppearance(Appearance ap) {
        DefaultGeometryShader dgs;
        //DefaultPolygonShader dps;
        DefaultLineShader dls;
        DefaultPointShader dpts;
        dgs = ShaderUtility.createDefaultGeometryShader(ap, true);
        //dgs.setShowFaces(true);
        dgs.setShowLines(true);
        dgs.setShowPoints(true);
        //dps = (DefaultPolygonShader) dgs.createPolygonShader("default");
        //dps.setDiffuseColor(Color.blue);
        dls = (DefaultLineShader) dgs.createLineShader("default");
        dls.setDiffuseColor(colors[0]);
        dls.setTubeRadius(.05);
        dpts = (DefaultPointShader) dgs.createPointShader("default");
        dpts.setDiffuseColor(Color.red);
        dpts.setPointRadius(.5);
    }

    public int[][] createEdgeIndices(double[][] points) {
        int[][] edges = new int[points.length - 1][2];

        for (int i = 0; i < points.length - 1; i++) {
            edges[i][0] = i;
            edges[i][1] = i + 1;
        }
        return edges;
    }

     //Colors range from 0 to 255
    public Color[] createColorList(int n){
        Color[] colors = new Color[n];
        int red = 0;
        int green = 0;
        int blue = 0;
        for (int i = 0; i < n ; i++) {
            red = generateNewColorValue(red);
            green = generateNewColorValue(green);
            blue = generateNewColorValue(blue);
            colors[i] = new Color(red, green, blue);
        }
        return colors;
    }

    private int generateNewColorValue(int col) {
        Random ran = new Random();
        int n = ran.nextInt(255);
        col += col + 256 * n;
        while ( col > 255){
            col = col/256;
        }
        System.out.println("Color: " + col);
        return col;
    }
}

public interface GeometryObject {
    public void updateGeometry(double[][] points);

    public GeometryObject createGeometryObject(double[][] points);
}

Any comments and suggestions you have will help me a lot.

Thanks,
Karuna

[karunaMaitri]

Hi, I had some success in making the program work. I attach the (modified) main program below. Please let me know if there is a simpler way of handling the problem in jReality.

I have a question. As I understand there are two ways to retrieve the coordinates of the point clicked. (a) By using ToolContext as used in AddPointsExample tutorial and (b) using PickResult of the ToolContext. In my case, they were giving two different values. I chose to use the first method when clicking on the background and second method when clicking on the points. I am not sure why there is a difference in their values.

Code: Select all

public class DragExample4 extends AbstractTool {
    private double[] startPoint;
    private int startIndex;
    private double[] endPoint;
    private int endIndex;
    private boolean mousePressed = false;
    private boolean mouseReleased = true;
    private double[][] controlPoints;
    private GeometricObjectSceneGraphNode curve1SGC;
    private GeometricObjectSceneGraphNode curve2SGC;
    private double[][] curvePoints;

    public static void main(String[] args) {
        DragExample4 test = new DragExample4();
    }

    public DragExample4() {
        addCurrentSlot(InputSlot.SHIFT_LEFT_BUTTON, "add a new point");
        curvePoints = new double[][]{{0.0, 0.0, 0.0}, {10.0, 10.0, 0.0}, {10.0, 20.0, 0.0}, {20.0, 20.0, 0.0}, {20.0, 10.0, 0.0}, {30.0, 10.0, 0.0}};
        curve1SGC = new GeometricObjectSceneGraphNode();
        CurveGeometryObject curveObject = new CurveGeometryObject();
        curveObject.updateGeometry(curvePoints);
        curve1SGC.updateGeometry(curveObject);

        controlPoints = new double[][]{{5.0, 5.0, 0.0}, {15.0, 15.0, 0.0}, {15.0, 25.0, 0.0}, {25.0, 25.0, 0.0}, {25.0, 15.0, 0.0}, {35.0, 15.0, 0.0}};
        curve2SGC = new GeometricObjectSceneGraphNode();
        CurveGeometryObject curveObject2 = new CurveGeometryObject();
        curveObject2.updateGeometry(controlPoints);
        curve2SGC.updateGeometry(curveObject2);

        curve1SGC.addChild(curve2SGC);
        curve1SGC.addTool(this);
        JRViewer.display(curve1SGC);
    }

    @Override
    public void perform(ToolContext tc) {
        System.out.println("Entering perform method");

        if (! mousePressed) {
            if (tc.getCurrentPick() == null) {
                System.out.println("Exiting perform method: 1_1");
                return;
            }
        }

        // now we transform the world coordinates to the coordinate system of the tool component
        if (tc.getAxisState(InputSlot.SHIFT_LEFT_BUTTON).isPressed() && !mousePressed) {

            PickResult currentPick = tc.getCurrentPick();
            startIndex = currentPick.getIndex();
            startPoint = controlPoints[startIndex];
            System.out.println("Start point from Current Pick Index: [" + startPoint[0] + ", " + startPoint[1] + ", " + startPoint[2] + "]");
            if (!oneOfControlPoints(startPoint)) {
                System.out.println("Exiting perform method : 2");
                return;
            } else {
                mousePressed = true;
                mouseReleased = false;
                System.out.println("Exiting condition: 1");
            }
        } else if (tc.getAxisState(InputSlot.SHIFT_LEFT_BUTTON).isReleased()) {
            System.out.println("Entered mouse released condition");
            if (mousePressed && !mouseReleased) {
                // determine the pointer transformation:
                // translation is the mouse pointer on the near clipping plane
                // z-axis is the direction of the mouse ray out of the screen
                // for a 6DOF input device, it is the position/orientation of the device
                Matrix m = new Matrix(tc.getTransformationMatrix(InputSlot.POINTER_TRANSFORMATION));

                // we compute the coordinates of the new point in world coordinates
                double[] foot = m.getColumn(3);
                double[] dir = m.getColumn(2);
                double[] offset = Rn.times(null, -5, dir);
                double[] newPoint = Rn.add(null, foot, offset);
                endPoint = ToolUtility.worldToLocal(tc, newPoint);
                mousePressed = false;
                mouseReleased = true;
                System.out.println("About to update the geometry");
                updateGeometry();
            } else {
                System.out.println("Exiting perform method : 3");
                return;
            }
        }
        System.out.println("Exiting perform method : 4");
    }

    public void updateGeometry() {
        controlPoints[startIndex] = endPoint;
        CurveGeometryObject geomObject = new CurveGeometryObject();
        geomObject.updateGeometry(controlPoints);
        curve2SGC.updateGeometry(geomObject);

        endPoint[0] += 5;
        endPoint[1] += 5;
        curvePoints[startIndex] = endPoint;
        CurveGeometryObject geomObject2 = new CurveGeometryObject();
        geomObject2.updateGeometry(curvePoints);
        curve1SGC.updateGeometry(geomObject2);
    }

    public boolean oneOfControlPoints(double[] point) {
        boolean cond = false;
        for (int i = 0; i < controlPoints.length; i++) {
            if (controlPoints[i][0] == point[0] &&
                    controlPoints[i][1] == point[1] &&
                    controlPoints[i][2] == point[2])
                return true;
        }
        return cond;
    }
}

Thanking you!
Karuna

[steffen]

I have a question. As I understand there are two ways to retrieve the coordinates of the point clicked. (a) By using ToolContext as used in AddPointsExample tutorial and (b) using PickResult of the ToolContext. In my case, they were giving two different values. I chose to use the first method when clicking on the background and second method when clicking on the points. I am not sure why there is a difference in their values.

The PickResult gives the coordinates of the intersection of the pointer ray and the scene, but in the AddPointsExample we are adding new points, so there is no meaningful PickResult. Instead, we take the pointer foot point (for mouse the point on the near clipping plane) and place a point on the ray at a certain distance from the foot point.

For your code I would suggest you use the DragEventTool (as I have written before: remember the point position in pointDragged and update your geometry in pointDragEnd).

[karunaMaitri]

Thanks Steffen!

I have used DragEventTool and your suggestion. It is working! You are right, it is much simpler than my design. That gives me a lot of confidence! Thanks again for that.

Thanks also for the explanation about data differences. That makes sense.

Karuna