Problems with releasing memory

Have jReality programming problems or questions? Post them here.
Joe
Posts: 61
Joined: Fri 11. Jul 2008, 19:29

Problems with releasing memory

Post by Joe » Sun 12. Apr 2009, 17:55

Hi,
I have an application, which allows to load CAD files, visualized with jReality. If one of these files is closed, the allocated memory isn't released. I examined the problem and came to the conclusion, that the ViewerApp class doesn't release the memory. Could this be correct or is there an official way to release the memory, allocated by jReality?

I broke it down to this little example. You can see, that the memory grows from 0 MB to 19 MB. The Garbage Collector can free up 2 MB, so that it ends up in 17 MB.

If the ViewerApp class isn't instantiated, the memory goes up to 17 MB and can completly be removed by the Garbage Collector.

Thanks and best regards, Joe

Code

Code: Select all

import java.awt.Component;

import javax.swing.JFrame;

import de.jreality.geometry.Primitives;
import de.jreality.scene.SceneGraphComponent;
import de.jreality.ui.viewerapp.ViewerApp;

public class SimpleViewer {
	private static void printMemoryUsage(String message){
		long memoryMaximum = Runtime.getRuntime().maxMemory();
		long memoryTotal = Runtime.getRuntime().totalMemory();
		long memoryUsed = memoryTotal - Runtime.getRuntime().freeMemory();
		System.out.println(message + ": " + (memoryUsed >> 20) + " / " + (memoryTotal >> 20) + " / " + (memoryMaximum >> 20) + " MB");
	}

	public static void main(String[] arguments){
		printMemoryUsage("Before creating geometry");
		SceneGraphComponent sceneGraphComponent = new SceneGraphComponent();
		sceneGraphComponent.setGeometry(Primitives.torus(3.0, 5.0, 200, 200));
		ViewerApp viewerApp = new ViewerApp(sceneGraphComponent);
		printMemoryUsage("After creating geometry");
        sceneGraphComponent = null;
        viewerApp.dispose();
        viewerApp = null;
        System.gc();
        try {
        	Thread.sleep(5000);
        } catch(Exception exception){
        	exception.printStackTrace();
        }
        printMemoryUsage("After garbage collection");
	}
}
Output

Code: Select all

Before creating geometry: 0 / 4 / 63 MB
scene root is root
After creating geometry: 19 / 23 / 63 MB
After garbage collection: 17 / 31 / 63 MB
Results without instantiating the VieweApp class

Code: Select all

Before creating geometry: 0 / 4 / 63 MB
After creating geometry: 17 / 23 / 63 MB
After garbage collection: 0 / 23 / 63 MB

STRESS
Posts: 141
Joined: Mon 19. Jan 2009, 12:10

Post by STRESS » Tue 14. Apr 2009, 10:47

Which backend are you using? Does it happens both with OpenGL (jogl) and software backend? There could be some NIO stuff through JOGL that is hanging onto the memory.

Joe
Posts: 61
Joined: Fri 11. Jul 2008, 19:29

Post by Joe » Tue 14. Apr 2009, 17:55

Hi STRESS,
there's no difference between the viewers and their memory usage:

Softviewer:
Before creating geometry: 0 / 4 / 63 MB
scene root is root
Viewer: de.jreality.softviewer.SoftViewer[,0,0,0x0,invalid]
After creating geometry: 19 / 23 / 63 MB
After garbage collection: 17 / 31 / 63 MB

JOGL:
Before creating geometry: 0 / 4 / 63 MB
scene root is root
Viewer: de.jreality.jogl.Viewer@c9ba38
After creating geometry: 19 / 23 / 63 MB
After garbage collection: 17 / 31 / 63 MB

STRESS
Posts: 141
Joined: Mon 19. Jan 2009, 12:10

Post by STRESS » Fri 17. Apr 2009, 10:50

Hmm okay was worth a try :oops: . So well my shot in the dark here would be probably there are some Lists or Hashtables that are hanging around some references that do not get collected. If you look at the source code there are quite a lot of these used.

Joe
Posts: 61
Joined: Fri 11. Jul 2008, 19:29

Post by Joe » Thu 23. Apr 2009, 22:23

OK, I created a little fix (see bottom of this post), here are my results:

The memory leak occured, because the SceneGraphComponents still were referenced and couldn't be removed by the GarbageCollector. My little helper method fixes this issue for some cases.

If the SceneGraphComponents are disconnected from each other and the geometry is set to null, the memory will be released completly if the softviewer has been used for rendering. The JOGL viewer still keeps some memory.
If the model is rotated for example by the RotateTool, some of the memory can't be released, even with the softviewer. In this case some classes still reference the instances of the used SceneGraphComponents, so that they can't be removed by the GarbageCollector.

Code: Select all

              (a)  (b)  (c)  (d)
JOGL viewer    5	51	18	30
Soft viewer    5	42	5	19
(a) Used memory in MB before loading geometry
(b) Used memory in MB after loading geometry
(c) Used memory in MB after garbage collection (without mouse interaction)
(d) Used memory in MB after garbage collection (with mouse interaction)

Results
The JOGL viewer seems to keep some references which forbid the GarbageCollector the removal of these objects:
  • The classloader/component "sun.misc.Launcher$AppClassLoader @ 0x910c040" occupies 13.844.248 (50,15%) bytes. The memory is accumulated in class "de.jreality.geometry.SphereUtility", loaded by "sun.misc.Launcher$AppClassLoader @ 0x910c040".
  • One instance of "de.jreality.jogl.JOGLRenderer" loaded by "sun.misc.Launcher$AppClassLoader @ 0x910c040" occupies 7.767.368 (28,13%) bytes. The memory is accumulated in one instance of "de.jreality.scene.IndexedFaceSet" loaded by "sun.misc.Launcher$AppClassLoader @ 0x910c040".
If some tools are used (e. g.RotateTool, DraggingTool, ...) they seem to keep also references to the used IndexedFaceSet (or SceneGraphComponents):
  • One instance of "de.jreality.toolsystem.ToolSystem" loaded by "sun.misc.Launcher$AppClassLoader @ 0x910c150" occupies 8.111.216 (48,87%) bytes. The memory is accumulated in one instance of "de.jreality.scene.IndexedFaceSet" loaded by "sun.misc.Launcher$AppClassLoader @ 0x910c150".
  • One instance of "de.jreality.softviewer.SoftViewer" loaded by "sun.misc.Launcher$AppClassLoader @ 0x910c150" occupies 1.809.896 (10,91%) bytes. The memory is accumulated in one instance of "de.jreality.softviewer.Renderer" loaded by "sun.misc.Launcher$AppClassLoader @ 0x910c150".
Here is the code, which helps cleaning the memory. I hope somebody finds workarounds for releasing all other unused references. I tried to enhance my cleaning method, but I think the problems have to be solved in the jReality code.

Code: Select all

	private void prepareMemoryCleanUp(SceneGraphComponent sceneGraphComponent){
		sceneGraphComponent.setGeometry(null);

		while(sceneGraphComponent.getChildComponentCount() > 0){
			SceneGraphComponent sgcChild = sceneGraphComponent.getChildComponent(0);
			sceneGraphComponent.removeChild(sgcChild);
			prepareMemoryCleanUp(sgcChild);
		}
	}

	public void dispose(){
		prepareMemoryCleanUp(_viewerApp.getSceneRoot());
		_viewerApp.dispose();
	}

Joe
Posts: 61
Joined: Fri 11. Jul 2008, 19:29

Post by Joe » Fri 24. Apr 2009, 17:20

Hi,
I also fixed the JOGL memory leak. Here is the code:

The enhanced helper method from previous post:

Code: Select all

/**
 * Start preparing the objects to be removable by teh garbage collector.
 */
public void dispose(){
	prepareMemoryCleanUp(_viewerApp.getSceneRoot());

	// Help the JOGL viewer to release memory
	if(_viewerApp.getViewerSwitch().getCurrentViewer() instanceof Viewer){
		JOGLSphereHelper.disposeSphereDLists(((Viewer)_viewerApp.getViewerSwitch().getCurrentViewer()).getRenderer());
	}

	_viewerApp.dispose();
}
This has to be integrated into JOGLSphereHelper.disposeSphereDLists(), which has never been called:

Code: Select all

public static void disposeSphereDLists(JOGLRenderer jr) {
	int[] dlists = getSphereDLists(jr);
	if (dlists == null)	{
		throw new IllegalStateException("No such gl context");
	}
	// probably don't need to actually delete them since the context

	// This helps the garbage collector
	sphereDListsTable.clear();
	for(int i = 0; i < tessellatedCubes.length; i++){
		tessellatedCubes[i].setGeometry(null);
		tessellatedCubes[i] = null;
	}
	for(int i = 0; i < cubePanels.length; i++){
		cubePanels[i] = null;
	}
	// This helps the garbage collector
}
Now only the problem with the references in the tools has to be fixed (more details, see previous post). It would be nice, if this code changes could be transferred into the jReality project.

User avatar
steffen
Posts: 186
Joined: Fri 16. Jun 2006, 13:30
Location: TU Berlin
Contact:

Post by steffen » Sat 25. Apr 2009, 20:21

Joe, thanks for pointing this out! I will figure out where the references are kept the next days... I hope Charles will include your fix into the jogl backend soon, I will not check it in since he knows better if there might be any side effects to take care of.

Steffen.

Joe
Posts: 61
Joined: Fri 11. Jul 2008, 19:29

Post by Joe » Sun 26. Apr 2009, 20:28

Hi Steffen,
I played around with the profiler and found the other memory leaks (when the mouse tools are used). Here is a complete code, which helps the GarbageCollector to remove all of the objects, creating memory leaks. I think it would make sense to put the code into ViewerApp.dispose(). It also handles the leaks, appearing after switching the viewers (softviewe / JOGL) while displaying the geometries.

One important question: Is jReality designed for having multiple ViewerApp instances? Because in my Application you can load different geometry files, which are hold by JInternalFrames. Since the memory leaks have been removed in my local jreality installation, it's not possible to close one model (call viewerApp.dispose()) without violating the other viewerApp instances in the other internal frames. For example, when I start to disconnect the SceneGraphComponents, the geometries in the other internal frames disappears. It seems that there are some static objects, which cover all ViewerApp instances.
Is there a way to handle multiple instances of ViewerApp, with the possibility of removing one some of them? Maybe you can modify my proposal in a way, allowing the fix the memory leaks and to dispose only some ViewerApps, whithout violating the other ViewerApp instances.

Heres the code, which removes the memory leaks, but violates the handling of multiple ViewerApp instances:

Code: Select all

private void prepareMemoryCleanUp(SceneGraphComponent sceneGraphComponent){
	sceneGraphComponent.setGeometry(null);

	while(sceneGraphComponent.getChildComponentCount() > 0){
		SceneGraphComponent sgcChild = sceneGraphComponent.getChildComponent(0);
		sceneGraphComponent.removeChild(sgcChild);
		prepareMemoryCleanUp(sgcChild);
	}
}

public void dispose(){
	// Disconnect the SceneGraphComponent tree and set the geometries to "null"
	prepareMemoryCleanUp(_viewerApp.getSceneRoot());

	// Help the viewers to release their memory
	Viewer[] viewers = _viewerApp.getViewerSwitch().getViewers();
	for(int i = 0; i < viewers.length; i++){
		if(viewers[i] instanceof de.jreality.softviewer.SoftViewer){
			((de.jreality.softviewer.SoftViewer)viewers[i]).dispose();
		} else if(viewers[i] instanceof de.jreality.jogl.Viewer){
			JOGLRenderer joglRenderer = ((de.jreality.jogl.Viewer)viewers[i]).getRenderer();
			if(joglRenderer != null){
				JOGLSphereHelper.disposeSphereDLists(joglRenderer);
				JOGLRenderingState joglRenderingState = joglRenderer.getJOGLRenderingState();
				if(joglRenderingState.currentGeometry != null){
					joglRenderingState.currentGeometry = null;
				}
			}
		}
	}

	// Release the memory, occupied by the pick system
	ToolSystem toolSystem = _viewerApp.getToolSystem();
	PickSystem pickSystem = toolSystem.getPickSystem();
	if(pickSystem instanceof AABBPickSystem){
		((AABBPickSystem)pickSystem).dispose();
	}

	// Release the memory, occupied by the pick results
	List<PickResult> pickResults = toolSystem.getPickResults();
	for(Iterator<PickResult> iterator = pickResults.iterator(); iterator.hasNext();){
		PickResult pickResult = iterator.next();
		if(pickResult instanceof Hit){
			Hit hit = (Hit)pickResult;
			SceneGraphPath sceneGraphPath = hit.getPickPath();
			for(Iterator<SceneGraphNode> iterator2 = sceneGraphPath.iterator(); iterator2.hasNext();){
				SceneGraphNode sceneGraphNode = iterator2.next();
				if(sceneGraphNode instanceof SceneGraphComponent){
					((SceneGraphComponent)sceneGraphNode).setGeometry(null);
				}
			}
			sceneGraphPath.clear();
		}
	}

	// Release the memory, occupied by the tool path
	HashMap<Tool, List<SceneGraphPath>> toolToPath = toolSystem.getToolToPath();
	for(Iterator<List<SceneGraphPath>> iterator = toolToPath.values().iterator(); iterator.hasNext();){
		List<SceneGraphPath> sceneGraphPathList = iterator.next();
		for(Iterator<SceneGraphPath> iterator2 = sceneGraphPathList.iterator(); iterator2.hasNext();){
			SceneGraphPath sceneGraphPath = iterator2.next();
			for(Iterator<SceneGraphNode> iterator3 = sceneGraphPath.iterator(); iterator3.hasNext();){
				SceneGraphNode sceneGraphNode = iterator3.next();
				if(sceneGraphNode instanceof SceneGraphComponent){
					((SceneGraphComponent)sceneGraphNode).setGeometry(null);
				}
			}
			sceneGraphPath.clear();
		}
	}

	// Call the predefined disposing stuff
	_viewerApp.dispose();
}

JOGLSphereHelper:
	public static void disposeSphereDLists(JOGLRenderer jr) {
		int[] dlists = getSphereDLists(jr);
		if (dlists == null)	{
			throw new IllegalStateException("No such gl context");
		}
		// probably don't need to actually delete them since the context

		// New stuff to help releasing the memory
		jr.renderingState.currentGeometry = null;
		sphereDListsTable.clear();
		for(int i = 0; i < tessellatedCubes.length; i++){
			tessellatedCubes[i].setGeometry(null);
			tessellatedCubes[i] = null;
		}
		for(int i = 0; i < cubePanels.length; i++){
			cubePanels[i] = null;
		}
		// New stuff to help releasing the memory
	}

JOGLRenderer:
	public JOGLRenderingState getJOGLRenderingState(){
		return renderingState;
	}

AABBPickSystem:
	public void dispose(){
		aabbTreeExists.clear();
		isPickableMap.clear();
	}

ToolSystem:
	public List getPickResults(){
    	return pickResults;
    }

    public HashMap<Tool, List<SceneGraphPath>> getToolToPath(){
		return toolToPath;
	}

	public ToolUpdateProxy getToolUpdateProxy(){
		return updater;
	}

SoftViewer:
	public void dispose() {
        synchronized (renderAsyncLock) {
            disposed = true;
            renderAsyncLock.notify();
        }
        // New stuff to help releasing the memory
        renderer = null;
        // New stuff to help releasing the memory
    }

User avatar
gunn
Posts: 323
Joined: Thu 14. Dec 2006, 09:56
Location: TU Berlin
Contact:

Post by gunn » Mon 27. Apr 2009, 12:34

Thanks for diagnosing the problem involving memory problems with spheres.

The main memory sink here is not in the class JOGLSphereHelper but in de.jreality.geometry.SphereUtility. The code you've suggested for JOGLSphereHelper is basically removing memory which was allocated by SphereUtility.

Since I think it's clearer to remove references in the same class which allocates them, I've introduced a dispose() method to SphereUtility, and would recommend that you call this method directly, since the JOGLSphereHelper itself only manages display lists, which first of all don't take up much memory on the host and secondly by default are shared among all GL contexts, so shouldn't be removed until all contexts are disposed.

You can safely call SphereUtility.dispose() whenever you like; subsequent calls to SphereUtility which require the removed objects will simply recreate the objects it needs. It is possible to avoid having the objects retained in the first place by setting the boolean parameter sharedInstance to false when calling the static methods in SphereUtility. Perhaps JOGLSpbereHelper should also do so when it sets up its sphere display lists; we can discuss this in the developer's meeting and decide if that's the right course. In the meantime, just call SphereUtility.dispose().
jReality core developer

User avatar
steffen
Posts: 186
Joined: Fri 16. Jun 2006, 13:30
Location: TU Berlin
Contact:

Post by steffen » Mon 27. Apr 2009, 20:42

it's not possible to close one model (call viewerApp.dispose()) without violating the other viewerApp instances in the other internal frames
Sounds like you display the same scene graph (or parts of it) in different viewer apps. This should be ok, but you will of course see the stuff vanishing in the other frames, when you call your method prepareMemoryCleanUp.

The basic problem is that there exist some references to the scene graph components and other objects, which prevent the garbage collector from doing its job.
I have fixed this so far for the tool system and the SoftViewer, but I am failing in figuring out where the references are kept in the jogl backend... Please test with software viewer to see if your stuff is ok so far, without the prepareMemoryCleanUp-call.
Note that calling System.gc() once may not be enough, I use the follwoing loop:

Code: Select all

for (int i=0; i<25; i++) {
    System.gc();
    Thread.sleep(250);
    printMemoryUsage(i+". After gc");
}
I hope we find the problems in the jogl-backend soon...

User avatar
gunn
Posts: 323
Joined: Thu 14. Dec 2006, 09:56
Location: TU Berlin
Contact:

Post by gunn » Tue 28. Apr 2009, 14:51

Just a quick note to update on the problem with JOGLSphereHelper/SpherreUtility and memory usage: the JOGL backend now does not retain the geometry used to create its sphere display lists. So there's no need anymore to call SphereUtility.dispose() unless you have called the methods in the class yourself with sharedInstance set to true.
jReality core developer

Joe
Posts: 61
Joined: Fri 11. Jul 2008, 19:29

Post by Joe » Tue 28. Apr 2009, 21:34

Hmm ... mysterious. If I use a JTabbedPane instead of JInternalFrames, I can close the loaded models without affecting the other models. So the bug has to be searched in my code (or it has something to do with the JInternalFrames).

I checked out the current sources. The memory is released in all my use cases. Many many thanks.

But sometimes the call of ViewerApp.dispose() leads to a NullPointerException:

Code: Select all

Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
	at de.jreality.scene.proxy.tree.UpToDateSceneProxyBuilder.disposeEntity(UpToDateSceneProxyBuilder.java:180)
	at de.jreality.scene.proxy.tree.UpToDateSceneProxyBuilder.access$1(UpToDateSceneProxyBuilder.java:177)
	at de.jreality.scene.proxy.tree.UpToDateSceneProxyBuilder$3.visit(UpToDateSceneProxyBuilder.java:208)
	at de.jreality.scene.SceneGraphNode.accept(SceneGraphNode.java:176)
	at de.jreality.scene.Appearance.superAccept(Appearance.java:217)
	at de.jreality.scene.Appearance.superAccept(Appearance.java:214)TEQ shut down.

	at de.jreality.scene.SceneGraphVisitor.visit(SceneGraphVisitor.java:74)
	at de.jreality.scene.Appearance.accept(Appearance.java:208)
	at de.jreality.scene.SceneGraphComponent.childrenAccept(SceneGraphComponent.java:393)
	at de.jreality.scene.proxy.tree.UpToDateSceneProxyBuilder$3.visit(UpToDateSceneProxyBuilder.java:212)
	at de.jreality.scene.SceneGraphComponent.accept(SceneGraphComponent.java:364)
	at de.jreality.scene.SceneGraphComponent.childrenAccept(SceneGraphComponent.java:399)
	at de.jreality.scene.proxy.tree.UpToDateSceneProxyBuilder$3.visit(UpToDateSceneProxyBuilder.java:212)
	at de.jreality.scene.SceneGraphComponent.accept(SceneGraphComponent.java:364)
	at de.jreality.scene.SceneGraphComponent.childrenAccept(SceneGraphComponent.java:399)
	at de.jreality.scene.proxy.tree.UpToDateSceneProxyBuilder$3.visit(UpToDateSceneProxyBuilder.java:212)
	at de.jreality.scene.SceneGraphComponent.accept(SceneGraphComponent.java:364)
	at de.jreality.scene.SceneGraphComponent.childrenAccept(SceneGraphComponent.java:399)
	at de.jreality.scene.proxy.tree.UpToDateSceneProxyBuilder$3.visit(UpToDateSceneProxyBuilder.java:212)
	at de.jreality.scene.SceneGraphComponent.accept(SceneGraphComponent.java:364)
	at de.jreality.scene.SceneGraphComponent.childrenAccept(SceneGraphComponent.java:399)
	at de.jreality.scene.proxy.tree.UpToDateSceneProxyBuilder$3.visit(UpToDateSceneProxyBuilder.java:212)
	at de.jreality.scene.SceneGraphComponent.accept(SceneGraphComponent.java:364)
	at de.jreality.scene.SceneGraphComponent.childrenAccept(SceneGraphComponent.java:399)
	at de.jreality.scene.proxy.tree.UpToDateSceneProxyBuilder$3.visit(UpToDateSceneProxyBuilder.java:212)
	at de.jreality.scene.SceneGraphComponent.accept(SceneGraphComponent.java:364)
	at de.jreality.scene.proxy.tree.UpToDateSceneProxyBuilder.dispose(UpToDateSceneProxyBuilder.java:215)
	at de.jreality.toolsystem.ToolUpdateProxy.dispose(ToolUpdateProxy.java:65)
	at de.jreality.toolsystem.ToolSystem.dispose(ToolSystem.java:620)
	at de.jreality.ui.viewerapp.ViewerApp.dispose(ViewerApp.java:1089)

User avatar
steffen
Posts: 186
Joined: Fri 16. Jun 2006, 13:30
Location: TU Berlin
Contact:

Post by steffen » Wed 29. Apr 2009, 01:12

The memory is released in all my use cases. Many many thanks.
Great! Thanks for pointing us to that problem...
But sometimes the call of ViewerApp.dispose() leads to a NullPointerException:
Should be fixed now, was an obvious bug.

STRESS
Posts: 141
Joined: Mon 19. Jan 2009, 12:10

Re: Problems with releasing memory

Post by STRESS » Thu 17. Sep 2009, 17:26

Has any of this incooperated in the code by now I wonder? Since I have some strange memory leaks in my RCP program basically every time I open / close a RCP view I loose about 10-15Mb of memory when the view uses jReality at least with the software render backend even when there are only empty scene nodes. With the JOGL one it doesn't seem to be that much or at least not that noticeable and that might be due to the revived SWT JOGL backend I put in.

User avatar
steffen
Posts: 186
Joined: Fri 16. Jun 2006, 13:30
Location: TU Berlin
Contact:

Re: Problems with releasing memory

Post by steffen » Fri 18. Sep 2009, 07:58

What do you mean by closing a RCP view with software render backend? Calling ViewerApp.dispose()?

Post Reply