Fix GPU usage (Makie issue #678)

[pbouffard]

So this is actually the PR #107 from @ffreyer with merge conflicts resolved. I does seem to solve the GPU usage issue on Mac, and 2D and 3D scenes appear to work properly, at least, with very minimal testing. However widgets appear to be broken: they appear as expected but don't respond at all.

Full disclosure: I do not know anything about the GLMakie internals and just did some educated guesses on resolving the merge conflicts. Submitting this PR in the hopes that there's just a minor delta from it to what will actually resolve MakieOrg/Makie.jl#678.

Also, I have no idea whether corresponding changes in other packages!

Apologies if this isn't quite the right process here, but I'm hoping this is helpful!

[ffreyer]

https://github.com/pbouffard/GLMakie.jl/blob/2c9a542030ae04d41423ef5b235e9064b4aa99ec/src/screen.jl#L455

This line needs to use fb.buffers[:objectid] as well.

[ffreyer]

The reason I stopped working on this and made it a draft was that I wanted to do something more fancy. Specifically something like nodes in Blender to select which shaders are applied and how data flows. I didn't get very far with that though.

If this makes things significantly better for macos maybe we should just merge it and make something better later

[pbouffard]

Thanks @ffreyer, though unfortunately that doesn't seem to resolve it.

[pbouffard]

New info: I just noticed that at least one widget (Textbox) does work..

[pbouffard]

Some progress.. I've been able to make other widgets 'work' again, some surgery in AbstractPlotting was required. By 'work' I mean that since I really have no business performing this kind of surgery without a medical license, it could be that really they just appear to work and something is totally borked under the hood.

Actually looking closer certainly something isn't working quite correctly: sliders do respond to the mouse but the slider handle jumps away from the mouse so it isn't under it once you start dragging. At least sometimes--changing the window size seems to have something to do with it. Buttons seem OK. I haven't tried to apply my 'fixes' to the other widgets yet.

Interestingly the GPU % as reported by Activity Monitor is still close to 100%, but overall graphics performance doesn't seem to be negatively affected the way it is with current GLMakie master.

But, I feel this is close. I need to put a pin in this for now but in case it's useful here are the branches of GLMakie and AbstractPlotting that represent where I'm at:

https://github.com/pbouffard/GLMakie.jl/tree/work-in-progress
https://github.com/pbouffard/AbstractPlotting.jl/tree/wip

When I get a chance later this week I'll try to clean this up.

[ffreyer]

To look at the objectid buffer:

using GLMakie, ModernGL
using GLMakie: GLVisualize, GLAbstraction

fig, ax, pl = scatter(1:10)
scene = fig.scene

img = let 
    screen = scene.current_screens[1]
    window_size = widths(screen)
    fb = screen.framebuffer
    buff = fb.buffers[:objectid]
    glBindFramebuffer(GL_FRAMEBUFFER, fb.id[1])
    glReadBuffer(GL_COLOR_ATTACHMENT1)
    sid = zeros(GLMakie.SelectionID{UInt32}, window_size...)
    glReadPixels(0, 0, window_size..., buff.format, buff.pixeltype, sid)
    sid
end

image(first.(img))

That looks like this for me:

You can also get this to display directly, without copying from the GPU and feeding it back. For that change assets/shader/fragment_output.frag to use fragment_color = vec4(float(id.x) / 100., float(id.y) / 100., 0, 1); or something similar. Without the copying it looks fine to me, so maybe there's something wrong about buffer sizes that messes up the copy? Did you make any changes similar to #145?

There are also some calls to framebuffer.color and framebuffer.depth which should access framebuffer.buffers[...] in screen.jl

[pbouffard]

Ok, I had to also add using ModernGL, but otherwise got identical output to yours. I'm not sure what that means, though.

I don't think I did anything similar to #145, not intentionally, at least.

[ffreyer]

This needs to be Gluint because of #126.

[ffreyer]

I fixed things in my pr

[pbouffard]

Thanks @ffreyer!!

[georgematheos]

Hi! To clarify--am I correct to understand that the changes in this PR were added to #107 and then #107 was merged?

[ffreyer]

Unless I missed something this pr just had some changes to make #107 merge-able. Those changes should also be in #107

[georgematheos]

Okay, awesome! I just tried #107 and it seems to significantly improve performance on my Mac. Thank you for this!