Any work on cutting down the memory usage?

[STRESS]

Just wondering are their any moves from the team on cuting down the amount of memory jReality uses.

Basically I mostly run in the 2Gig memory wall with any geometry that is more than half to a full million vertices. Unfortunately lot of the workstation in use here are 32bit :(. Besides half a million isn't that much.

Looking at the code there are very easy targets to do that. For example I don't think that storing geometry data for display in doubles does makes much sense (unless you try to render large-scale terrain data which already is impossible anyway) especially not texture coordinates and colours, there is no gain at all.

Even vertex coordinates do not really need to be in double when using JOGL (GPU's internally does not run 64bit anyway unless you maybe use OpenGL4.0).

By moving to float that would already half the amount of total memory used.

I have no idea how painful that is. But I can imagine there is quite some work to do especially on the software backend.

[gunn]

One idea for you is to make sure that you don't hold pointers to geometry factories any longer than you have to. The geometry factory has lots of copies of information that gets garbage collected once the factory is no longer referenced.

I can't offer much encouragement personally for the idea of converting from doubles to floats, or of offering a float option instead of doubles. Doubles are very much entrenched in the code base at this point. That's just MHO, however.

[STRESS]

Actually I just realized what an even bigger issue is besides the double is the data allignment is the wrong way round. Read on how Java allocates memory for arrays. If you have a double array the minor dim should be the first member not the second. So for example instead of using

double[][] coords = new double[1000][3] you should use

double[][] coords = new double[3][1000]

you will notice a significant difference in memory usage. Unfortunately I think internally it seems to expect the first one though so you can not turn it around. :(.

[steffen]

It should be possible to use inlined arrays:

Code: Select all

double[] coords = new double[3*1000]

The factories support that too. There might be internal code that converts to double[1000][3] again, but this is probably a bug and should be fixed.

[STRESS]

I think it does! I haven't fully grasped yet what is happening in the back. But this whole StorageModel system is quite thick and not that easy to understand. I think I am missing a bit the big picture behind or what is the motivation of doing it in this manner.

It looks quite complicated for what it is trying to achieve. But then I don't really know what is it trying to achieve?

[gunn]

He who understands the StorageModel receives a lifetime subscription to "Obfuscated Code"

Seriously though, the inlined feature was always defended as an important memory saver by those responsible for coding the StorageModel class in the first place. Although I can read the code about as well as I can read sanskrit, I believe that it does not make 2D copies of the inlined arrays and so should, theoretically, use less memory. But compared to the actual double data stored in the array, not more than a 33% saving.

[STRESS]

He who understands the StorageModel receives a lifetime subscription to "Obfuscated Code"

Thanks gunn :) , from this response I gather that the original author of this piece of code is no longer around?

Seriously though, the inlined feature was always defended as an important memory saver by those responsible for coding the StorageModel class in the first place. Although I can read the code about as well as I can read sanskrit, I believe that it does not make 2D copies of the inlined arrays and so should, theoretically, use less memory. But compared to the actual double data stored in the array, not more than a 33% saving.

Well I am going seriously debug through this code now to find out if that's the case or not and will report back my findings.

I assume the only way to make it use less memory is to replace this whole StorageModel system that is used behind with something else. That's a very high risk task and lots of work from what I understand. I am bit reluctant to do that since that might divert me massively from your codebase but I might have no other choice.

[STRESS]

Well here is some interesting bit.

If you look at DoubleArrayArray the inlined static class

Code: Select all

private transient DoubleArray daView;
private transient final double[] data;
private transient final int offset, length, entryLength;
private transient final DoubleArray[] arrays;
    
	  
public Inlined(final double[] initialData, int numPerEntry) {
      this(initialData, numPerEntry, 0, initialData.length/numPerEntry);
}
    
    
public Inlined(final double[] initialData, int numPerEntry,
                   int firstEntry, int numEntries) {
    super(StorageModel.DOUBLE_ARRAY.inlined(numPerEntry), initialData,
             firstEntry, numEntries);
    data=initialData;
    if(numPerEntry<1)
        throw new IllegalArgumentException("numPerEntry="+numPerEntry);
    entryLength=numPerEntry;
    offset=firstEntry;
    length= numEntries;
    arrays=new DoubleArray[length];
}

There are three interesting things. First it keeps the original data around, Second it created a StorageModel wherever that data goes (don't know yet) and third it keeps a cached version called arrays which I assume is basically the individual [numPerEntry] Arrays.

That looks like it is keeping three copies of the data ??? Or am I seeing things wrong?

[paul peters]

Having extensively investigated the code of the Data/StorageModel a year ago I suspect that:

1. the

   Code: Select all

   StorageModel

   is just a description of the array and does not know the actual data array (something that seems to be invented because there where no generics around)

2. Code: Select all

   DoubleArray[] arrays

   is an array of wrappers that points to the "subarray" (although inlined, probably with the some offset) and should not duplicate data. But your remark on the ordering of indices seems to apply here, which may imply that there are a lot of such wrapper objects around. But probably they are never created in normal use.

I agree with gunns remark on obfuscated code, but still I would be surprised if the code would fail on its main goal of keeping memory footprint low. If I'm wrong and actual performance data shows that this goal is not achieved (I do not mean here the possible gain from switching to float arrays) then this and the obfuscation of the code would really demand a refactoring of that part of jReality.

[STRESS]

Having extensively investigated the code of the Data/StorageModel a year ago I suspect that:

1. the

   Code: Select all

   StorageModel

   is just a description of the array and does not know the actual data array (something that seems to be invented because there where no generics around)

Yes I came to the same conclusion it is just a complicated way to basically tell what the format of the storage is, there is some magic offseting going on which I did not quite grasp but I don't think this is the culprit anyway so I stopped looking deeper into this.

[*]

Code: Select all

DoubleArray[] arrays

is an array of wrappers that points to the "subarray" (although inlined, probably with the some offset) and should not duplicate data. But your remark on the ordering of indices seems to apply here, which may imply that there are a lot of such wrapper objects around. But probably they are never created in normal use.[/list]

Unfortunately this is not true. It is called every frame at least in the JOGL backend since the JOGLHelper classes when it renders the geometry uses a lot DataList.item() which then results into filling up this subarray (in it's private member called subArray) and it will keep it around for obvious caching reasons. So it provides a second view on the original data so it is actually not that bad as I original thought but still a few megs that can be saved for sure. :). Since this will happen basically with any VertexAttribute. There is also a second callee which calls getValueAt() which does the same, I haven't found yet who is the originator of this.

So I am still a bit puzzled where the majority of the memory gets lost the Profiler shows numbers that do not make much sense to me.

If I'm wrong and actual performance data shows that this goal is not achieved (I do not mean here the possible gain from switching to float arrays) then this and the obfuscation of the code would really demand a refactoring of that part of jReality.

Well I have some initial numbers which do not show very good memory behaviour (it sky rockets at some point) but I have to crunch this again through the mem profiler to get a much clearer view of the actual amount of memory consumed.

[paul peters]

Well I have some initial numbers which do not show very good memory behaviour (it sky rockets at some point) but I have to crunch this again through the mem profiler to get a much clearer view of the actual amount of memory consumed.

I also experienced to much memory usage, only I never managed to find the culprit.

[steffen]

STRESS, can you modify the subArray(..) methods in DoubleArray (maybe also IntArray) to always create a new instance instead of caching it? This will hit perfromance, but it will also show if this is a serious memory issue...

[STRESS]

Okay here some numbers from my profiling session:

I used a modified/simplified version of the QuadMeshExample code in tutorial/Geom here is the code:

Code: Select all

	public static PointSet createSurface( int x, int y ) {
		double [] coords = new double [x*y*3];
		double [] colours = new double[x*y*3];
		
		for( int i=0; i<x; i++) {
			for (int j = 0; j<y; ++j)	{
				double u = -15 + Math.random() * 30;
				double v = -15 + Math.random() * 30;
				double w = -15 + Math.random() * 30;
				
				coords[(i+j*x)*3] = u;
				coords[(i+j*x)*3+1]= v;
				coords[(i+j*x)*3+2]= w;
				colours[(i+j*x)*3] = Math.random();
				colours[(i+j*x)*3+1] = Math.random();
				colours[(i+j*x)*3+2] = Math.random();
			}
		}
		PointSetFactory factory = new PointSetFactory();
		factory.setVertexCount(x*y);
		factory.setVertexCoordinates(coords);
		factory.setVertexColors(colours);
		factory.update();

		
		return factory.getPointSet();
	}
	
	public static void main(String[] args) {
		SceneGraphComponent sgc = SceneGraphUtility.createFullSceneGraphComponent("world");
		sgc.setGeometry(createSurface(512,512));
		System.gc();
		Appearance ap = sgc.getAppearance();
		ap.setAttribute(CommonAttributes.POINT_SHADER+ "." + CommonAttributes.SPHERES_DRAW,false);
		ap.setAttribute(CommonAttributes.LINE_SHADER + "." + CommonAttributes.TUBES_DRAW, false);
		ap.setAttribute(CommonAttributes.LIGHTING_ENABLED, false);
		ap.setAttribute(CommonAttributes.ATTENUATE_POINT_SIZE,false);
		ap.setAttribute(CommonAttributes.POINT_SHADER+ "." + CommonAttributes.POINT_SIZE,2.0);
		JRViewer.display(sgc );
	}

I ran this through JProfiler 5.3.2 on Windows Xp using JOGL backend. And this is what I got:

createSurface size 64,64:
Total mem usage according to JProfile 9.9 Mbytes
Top 3
1. java.lang.Class 1.9MB
2. double[] 1.4MB
3.byte[] 1.4MB

createSurface size 128,128
Total mem usage 15.4 Mbytes
Top 3
1. double[] 4.3MB
2. java.lang.Class 1.9MB
3. int[] 1.8MB

createSurface size 256,256
Total mem usage 27 Mbytes
Top 3
1. double[] 10MB
2. DoubleArray 5MB
3. int[] 2.4MB

createSurface size 512,512
Total mem usage 77Mbytes
Top 3
1. double[] 40MB
2. DoubleArray 21MB
3. DoubleArray[] 4.2MB

createSurface size 1024,1024
Total mem usage 254Mbytes
Top 3
1. double[] 145MB
2. DoubleArray 82MB
3. DoubleArray[] 16.4MB

Now DoubleArray can be completed avoided when you change the code slightly I managed to do just for the PointFactory example by changing DoubleArrayArray.inlined, JOGLRenderHelper and BruteforcePicking so it got rid of DoubleArray showing up completely. And the new total is now about the size of DoubleArray + a bit smaller.

But more disturbing is the double[] size since that doesn't make any sense. If you check I've got only colour and coordinates so it should be

1024x1024x3x2x8 Bytes ~= 48MB but it is nearly 4 times more than what it should be, so even with crappy padding and a 4 Byte overhead for class identifier + some extra shenigans that number maybe could be 70-80 MB. But this much more just doesn't make any sense to me.

[gunn]

Make sure the reference to the quadmesh factory isn't hanging around after you get the geometry. The factories may keep copies of the data you hand them, too.

[STRESS]

Thanks gunn for your suggestion but if you look at the example code you will see the factory is a local method variable and after the block is executed it should be gone. I even call a System.gc() so it is unlikely to be the problem.