library_webgl.js wrong(?) assumption

[caiiiycuk]

glGen* in library_webgl.js implemented with assumption that it called rarely:

emscripten/src/library_webgl.js

Lines 303 to 311 in 619f08d

	// Get a new ID for a texture/buffer/etc., while keeping the table dense and
	// fast. Creation is fairly rare so it is worth optimizing lookups later.
	getNewId: (table) => {
	var ret = GL.counter++;
	for (var i = table.length; i < ret; i++) {
	table[i] = null;
	}
	return ret;
	},

I'm not a 3D developer, so I really don't know, but I think this assumption is correct for most quality made games.

However recently we ported 3d game which have fully changable terrain. It generates a lot of buffers for each frame. The GL.textures/buffers/programs/etc arrays grows very fast. Usually after ~1 hour plaing game took 2GB memory and then crash because can't create more.

Maybe this is ridiculous behavior for modern 3D (e.g. it shouldn't generate new buffers in every frame), but at least it works natively just fine. Is it also a requirement of GL? Fortunately, we have control over the sources and we can implement buffer pooling. And we will.

BUT, I also checked random Unity game that we ported, and for Unity games situation is same, the pools is constantly growing (not so fast). Not sure how much this affects performance on mobile, but from my experience Unity games degrade performance while playing (maybe related, maybe not)

I looked at the sources, it seems that arrays can be easily converted to hashmaps with minimal changes. Is it worth doing, what are the drawbacks?

Examples:

• run the game
• evalute this in dev tools:

console.log("counter:", Module.GL.counter, "textures:", Module.GL.textures.length, "buffers:", Module.GL.buffers.length, "programs:", Module.GL.programs.length, "syncs:", Module.GL.syncs.length)

1st. The PERIMETER game:

https://d13m0wt5um9hyu.cloudfront.net/repository/perimeter/v7/web/release/index.html

• on start: counter: 39547 textures: 39507 buffers: 39547 programs: 35 syncs: 0
• after 30 sec: counter: 460222 textures: 91414 buffers: 460222 programs: 35 syncs: 0
• on mission start: counter: 1174529 textures: 1111667 buffers: 1174529 programs: 1111173 syncs: 0

it grows very fast

2nd. City Police (typical unity game):

https://d146hdyl1gi3hn.cloudfront.net/repository/city-police-simulation/v2/web/release/index.html

• on start: counter: 1338 textures: 568 buffers: 1251 programs: 930 syncs: 1338
• after pressing play: counter: 7699 textures: 5226 buffers: 7667 programs: 7545 syncs: 7699
• after 3 restarts: counter: 22620 textures: 19209 buffers: 22605 programs: 14420 syncs: 22620
• after 6 restarts: counter: 30679 textures: 30284 buffers: 30678 programs: 24063 syncs: 30679

[sbc100]

I have a WIP PR convert GL to using the handle allocator: #18874. I think that would solve the problem here.

[caiiiycuk]

#18874 looks like massive change and it's not clear why it is not landed yet. It need more testing or what the blocker?

Actually I have something like #18668 in mind. Hashmap have same semantic so they can replace each other, only need to introduce freeId and use it in place where we do table[id] = null;

The idea to add link time option which contianer to use. And use arrays as default container, if you provide -s GL_TABLES=MAP then map will be used. We can also introduce -s GL_TABLES=ALLOCATORS. Btw, I don't care much about perfomrance degradation regarding to switching from array to map, I think implementing caching layer will reduce performance more.