GetCameraPathImage

[ted]

JReality renders very nicely. Thanks for all your hard work in it.

One thing it seems to lack: cameras taking individual pictures on demand.

I propose adding a method

Code: Select all

BufferedImage viewer.GetCameraPathImage(SceneGraphPath cameraPath, int width,int height)

or

Code: Select all

BufferedImage scene.GetCameraPathImage(SceneGraphPath cameraPath, int width,int height)

(You might also consider defining similar asynchronous methods)

The camera's view would be rendered off-screen immediately into a BufferedImage having the specified dimensions.

Alternatively an empty BufferedImage could be passed to the functions and they could render into it using its preset height and width.

This functionality seems to be missing. The viewer's captureScreenShot method currently suffers from several problems.

* The view has to already be displayed and not moving to get a rendering.
-- Otherwise you get null or a corrupted image capturing other windows or the desktop, etc.
-- Users may need to display one view while capturing another, or not display any view at all.

I'm not sure how this would be implemented, but it might have something to do with either using the soft renderer into a buffer off of the screen, or using a GL Off-Screen Frame Buffer Object using maybe glReadBuffer & glReadPixels. U3D looks like it may have related code, but I'm sort-of clueless about details like that at this point.

_-T

[gunn]

I touched on this briefly in a reply I posted this morning to another thread you've initiated, but I'll repeat the relevant part here.

There is no tutorial on off-screen rendering but the feature is supported and how to use it can be studied in the class de.jreality.ui.viewerapp.actions.file.ExportImage, included in the jReality distribution in the source fold src-ui. You of course don't need to use an instance of Expression to call the method; this is done to avoid a dependence to the JOGL backend in the core jreality classes.

If you're using an instance of JRViewer, this method is available via the menu item "File->Export->Image...".

FYI, it uses an OpenGL FrameBufferObject to implement the offscreen rendering, allowing antialiased images of up to 8K x 8K, at least on my machine. It sounds like you're interested in more modest sizes based on the current window on your display.

[ted]

If you're using an instance of JRViewer, this method is available via the menu item "File->Export->Image...".

FYI, it uses an OpenGL FrameBufferObject to implement the offscreen rendering, allowing antialiased images of up to 8K x 8K, at least on my machine. It sounds like you're interested in more modest sizes based on the current window on your display.

Yes, 8kx8k will be just fine.

However, the ExportImage raises an exception with the newest code from the repository. Maybe you can take a look at it. I posted details on Bugs topic.
http://www3.math.tu-berlin.de/jreality/ ... ?f=7&t=620

Aloha,

_-Ted

[ted]

After a lot of chopping down to isolate the required code, this is what I came up with:

Code: Select all

package de.jreality.ui.viewerapp;
import java.awt.Component;
import java.awt.image.BufferedImage;

import javax.swing.ImageIcon;
import javax.swing.JFrame;
import javax.swing.JLabel;

import de.jreality.geometry.Primitives;
import de.jreality.io.JrSceneFactory;

public class OffscreenRenderApp {

	public static void main(String[] args) {
		
		final de.jreality.jogl.Viewer v = new de.jreality.jogl.Viewer();
		v.setSceneRoot(Primitives.wireframeSphere());		
		v.setCameraPath(JrSceneFactory.getDefaultDesktopScene().getPath("cameraPath"));
	
		JFrame frame = new JFrame();
		frame.getContentPane().add((Component) v.getViewingComponent());	
		frame.setSize(1,1);  
		frame.setVisible(true);
		frame.setVisible(false);
		final javax.swing.JFrame jf = new javax.swing.JFrame();
		
		java.awt.EventQueue.invokeLater(new Runnable() {public void run(){
			BufferedImage bi = v.renderOffscreen(1000,1000); 
                        // we have a buffered image to do with what we want.
                        // to write to a file, serve in a servlet, send to image processing, etc...
                        // but we'll just display here to show that we have the image.
			jf.getContentPane().add(bi==null ? new JLabel("NULL") : new JLabel(new ImageIcon(bi)));
			jf.setVisible(true);
			jf.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
			jf.pack();
		}});
	}
}

This gets the job done - most of the time - (there's a race condition).

I'm not completely satisfied with some kludges still in my code:

1. There seems to be a race condition at startup even with the invokeLater.
I could add a sleep but that doesn't seem to guarantee anything.
Is there something I should do to ensure it's ready (and sleep if not 'till it is)?
2. It seems frame should not be necessary or setVisible, though you can immediately hide it again.
3. I don't understand why SoftViewer can't render into a BufferedImage. It seems the bytes would be available if it's done in software even more readily than through JOGL.

Am I doing it correctly? or is there a simple workaround for some of these?

Perhaps I'm making it too complicated somehow. If this is the way it's supposed to be done, I can write up a few lines of explanation and drop it in the programmer tutorials so nobody else will have to go through the trouble of dissecting ViewerApp.

[gunn]

I was able to remove the invokeLater() by replacing the frame.setVisible(false) with v.render(). The latter call guarantees that the JOGL renderer is initialized properly. It forces a render into the viewing component of the Viewer.

Ideally one would be able to do offscreen renderer without having a running JOGL Viewer attached to a component. I think with a bit of work that should be possible, since the offscreen rendering is now (since about two months) done with FBOs. I see no reason why it shouldn't be possible to render a scene into an FBO without having to render the scene first (with the same GL context) into a "normal" framebuffer. But that, in (j)reality, is how it is right now:(

The Softviewer could certainly be extended to output a BufferedImage. However there has been no requests for this feature and we apply our limited resources to the requests which come in.

[gunn]

Correction regarding offscreen rendering with SoftViewer: there is a method renderOffscreen(int width, int height) in the viewer, which returns a BufferedImage. However, for some reason, the actual call to render() (line 453) is commented out in the version in the repository. Try un-commenting this out and see what you get.

[ted]

Thanks for the quick response!

I was able to remove the invokeLater() by replacing the frame.setVisible(false) with v.render(). The latter call guarantees that the JOGL renderer is initialized properly. It forces a render into the viewing component of the Viewer.

Sounds simple eough, but I get a null BufferedImage with or without the invokeLater when I try:

Code: Select all

		frame.getContentPane().add((Component) v.getViewingComponent());	
		frame.setSize(1,1);  
		//frame.setVisible(true);
		/////////////
		v.render();//
		frame.setVisible(false);

What does work is this:

Code: Select all

package de.jreality.ui.viewerapp;
import java.awt.Component;
import java.awt.image.BufferedImage;

import javax.swing.ImageIcon;
import javax.swing.JFrame;
import javax.swing.JLabel;

import de.jreality.geometry.Primitives;
import de.jreality.io.JrSceneFactory;

public class CopyOfViewerApp {

	public static void main(String[] args) throws InterruptedException {
		
		final de.jreality.jogl.Viewer v = new de.jreality.jogl.Viewer();
		v.setSceneRoot(Primitives.wireframeSphere());		
		v.setCameraPath(JrSceneFactory.getDefaultDesktopScene().getPath("cameraPath"));
	
		JFrame frame = new JFrame();
		frame.getContentPane().add((Component) v.getViewingComponent());	
		frame.setVisible(true);
		frame.setVisible(false);
		v.render();
		BufferedImage bi = v.renderOffscreen(1000,1000);
		
		JFrame jf = new JFrame();
		jf.getContentPane().add(bi==null ? new JLabel("NULL") : new JLabel(new ImageIcon(bi)));
		jf.setVisible(true);
		jf.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
		jf.pack();
	}
}

[ted]

Although that works to capture an image, if I want to capture a snapshot of an image from a different camera than the one that the GUI is displaying, I will have to switch to the snapshot view, take the snapshot, then switch back, which will cause the GUI display to flicker. Please correct me if I'm wrong. Is this currently the best option?

Thanks.

_-T

[gunn]

Essentially what you say is correct. The offscreen rendering in the JOGL backend is, as noted already in the posts in this thread, tied to the window component associated to the instance of de.jreality.jogl.AbstractViewer. This is due to the nature of the rendering process in JOGL. Rendering occurs on a separate thread, and is governed by the interface javax.media.opengl.GLEventListener. As a result, offscreen rendering requests are awkward to handle, since one has to generate a call to the method

Code: Select all

    public abstract void display(GLAutoDrawable glautodrawable);

in order to have a valid GL context for rendering. When this callback occurs, the backend renderer has to know that the size of the desired image comes from the offscreen renderer, not from the onscreen component. Any other different settings, such as another camera path, have to be handled in the same way.

I've gone ahead and added this functionality and checked it in to SVN, since I think it's something useful for a more general audience. To see how to use the new functionality, I've also added a tutorial example de.jreality.tutorial.viewer.OffscreenRenderAlternateCameraPath.

Note that the current API for offscreen rendering will likely be changed in the future. With the addition of one more parameter, the use of static methods to implement offscreen rendering becomes too awkward. Instead, the offscreen rendering should be encapsulated in a "factory", which has setters for its parameters (antialiasing factor, save alpha channel, camera path, etc.), and an update() method, followed by a getBufferedImage() method for the result.

[ted]

Thanks.

One small suggestion. Either resolve the jni paths at runtime or catch this in the Viewer and display the contents of jni/README

Code: Select all

Exception in thread "main" java.lang.UnsatisfiedLinkError: no jogl in java.library.path
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1758)
	at java.lang.Runtime.loadLibrary0(Runtime.java:823)
	at java.lang.System.loadLibrary(System.java:1045)
	at com.sun.opengl.impl.NativeLibLoader.loadLibraryInternal(NativeLibLoader.java:189)
	at com.sun.opengl.impl.NativeLibLoader.access$000(NativeLibLoader.java:49)
	at com.sun.opengl.impl.NativeLibLoader$DefaultAction.loadLibrary(NativeLibLoader.java:80)
	at com.sun.opengl.impl.NativeLibLoader.loadLibrary(NativeLibLoader.java:103)
	at com.sun.opengl.impl.NativeLibLoader.access$200(NativeLibLoader.java:49)
	at com.sun.opengl.impl.NativeLibLoader$1.run(NativeLibLoader.java:111)
	at java.security.AccessController.doPrivileged(Native Method)
	at com.sun.opengl.impl.NativeLibLoader.loadCore(NativeLibLoader.java:109)
	at com.sun.opengl.impl.macosx.MacOSXGLDrawableFactory.<clinit>(MacOSXGLDrawableFactory.java:53)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:169)
	at javax.media.opengl.GLDrawableFactory.getFactory(GLDrawableFactory.java:108)
	at javax.media.opengl.GLCanvas.chooseGraphicsConfiguration(GLCanvas.java:520)
	at javax.media.opengl.GLCanvas.<init>(GLCanvas.java:131)
	at de.jreality.jogl.Viewer.initializeFrom(Viewer.java:81)
	at de.jreality.jogl.AbstractViewer.<init>(AbstractViewer.java:105)
	at de.jreality.jogl.AbstractViewer.<init>(AbstractViewer.java:100)
	at de.jreality.jogl.Viewer.<init>(Viewer.java:63)
	at de.jreality.jogl.Viewer.<init>(Viewer.java:60)
	at de.jreality.tutorial.viewer.OffscreenRenderAlternateCameraPath.main(OffscreenRenderAlternateCameraPath.java:115)

I think I posted a patch for that before...

[ted]

OK, it works, intermittently. (race condition or initialization problem?)
Sometimes if I start it up, and click in the middle as it's rendering, the window renders all black.

I also tried waiting 5 to 10 seconds instead of one second and popping up a new JWindow to display the image. Often it would not render the background and left it gray in the buffered image, but in the live viewer, it would reduce to 1/2 scale and draw in the lower left corner of the window. If I do it in a loop 3x (repeating starting from the sleep) over 15-30 seconds, maybe one in 3 will fail in this manner.

One time it rendered chunks from my screen into the window. I could not reproduce that problem again and don't know what happened.

My configuration is build 5517 from the jreality trunk, running on OSX Snow Leopard on a MacBook Pro, with Eclipse Juno.

[gunn]

Hi Ted,

Regarding your previous post: the current SVN repository contains the patch you contributed to display the jni/README when it detects a problem.

I'm sorry you're having problems with the improved offscreen rendering. Are you having problems with your own code, or with the tutorial example de.jreality.tutorial.viewer.OffscreenRenderAlternateCameraPath?

We're in the midst of an upgrade to a newer version of JOGL so please be forewarned that this feature is not slated to get further attention in the near future.

[ted]

No, not in my own code. This was working with the tutorial itself as delivered, and modifying it to pop up the image or change the delay, and for debugging, adding a loop to repeat the last part of the code a few times from the delay onwards.

I was just evaluating and trying to understand but before getting far I found these problems & decided not to convert to this method until it's more reliable.

Thanks
_-T

[ted]

This reproduces the problem

Code: Select all

        rt.addViewer(viewer);
        // have to give the viewer time to initialize itself before doing offscreen rendering
        for (int i=0;i<3;i++) {
        	try {
        		Thread.sleep(5000);
        	} catch (InterruptedException e) {
        		// TODO Auto-generated catch block
        		e.printStackTrace();
        	}
        	// note that the size of the image is "unantialiased"; final size will be reduced by a factor 4
        	BufferedImage bi = viewer.getRenderer().getOffscreenRenderer().renderOffscreen(
        			null, 1600, 1200, 4, viewer.getDrawable(), camPath2);
        	File file = new File("/tmp/foo.png");
        	ImageUtility.writeBufferedImage(file, bi);

        	JFrame snapshot = new JFrame("snapshot");
        	snapshot.getContentPane().add(new JLabel(new ImageIcon(bi)));
        	snapshot.pack();
        	snapshot.setVisible(true);
        }
    }
}

The defect is shown in the attached screen dump image. Use the mouse to spin it and then watch it take the snapshots. This symptom occurs fairly often when doing that.

[gunn]

Thanks for the sample code.

I tried it out here (jreality 5517, eclipse 3.5 (Galileo?), Mac OS 10.7, new Macbook Pro w/ NVidia GeForce 650 M), and experienced no problems. I decreased the sleep time to 1000, then to 500, then to 100, finally to 10, while increasing the number of screen shots to 15 and tried it several times, without seeing any of the problems you are having. So, my tentative conclusion -- given the similarity of the configurations -- is that it may have to do with the hardware. How old is your Macbook Pro and what is the graphics configuration?

This is a good test -- perhaps other forum readers who have a spare minute can run the sample code on their machines and report the results?