Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Emscripten support #8

Closed
wants to merge 8 commits into from
Closed

[WIP] Emscripten support #8

wants to merge 8 commits into from

Conversation

decahedron1
Copy link
Contributor

WIP HTML5 target using Emscripten.

Currently, I can't get this to build properly w/ emcmake; no matter what I do, CMake will always build for Linux/x86_64. Might be something to do with cmakefied?

But I believe this should work once its built properly, all you would need to get it working is an HTML file with <canvas id="canvas" width="400" height="400"></canvas>.

Copy link
Owner

@pr0g pr0g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @sudo-carson, thanks for opening the PR!

Definitely looks interesting, I don't think cmakefied is going to be the cause, all that does is pull in the Dear ImGui library to build with CMake (though I could totally be wrong, it might be something we need to change with ExternalProject_Add?).

I'm not really familiar with emcmake, do you have a link for trying it out? I'll see if I can try out the branch this weekend.

It would totally be cool to support this as an option though! Hopefully we can get it working 👍 Cheers!

configure-make.sh Show resolved Hide resolved
file-ops.h Outdated Show resolved Hide resolved
main.cpp Show resolved Hide resolved
file-ops.h Outdated Show resolved Hide resolved
@decahedron1
Copy link
Contributor Author

emcmake is part of emsdk, link to install instructions here 👍

@decahedron1
Copy link
Contributor Author

Searching ExternalProject_Add emscripten led me here: https://github.com/ethereum/cpp-dependencies/blob/master/cryptopp.cmake#L4,L9 & https://github.com/ethereum/cpp-dependencies/blob/master/cryptopp.cmake#L163

Seems like we do need to change ExternalProject_Add, but that doesn't really explain why I wasn't getting linker errors or anything building the whole project... hmm...

@decahedron1
Copy link
Contributor Author

Hmm, adding CMAKE_COMMAND to the external projects does build everything as Emscripten-linkable, but it gives me this error when building texturev & geometryv:

error: undefined symbol: emscripten_webgl2_get_proc_address (referenced by top-level compiled C/C++ code)
warning: Link with `-s LLD_REPORT_UNDEFINED` to get more information on undefined symbols
warning: To disable errors for undefined symbols use `-s ERROR_ON_UNDEFINED_SYMBOLS=0`
warning: _emscripten_webgl2_get_proc_address may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
Error: Aborting compilation due to previous errors
em++: error: '/home/carson/emsdk/node/14.15.5_64bit/bin/node /home/carson/emsdk/upstream/emscripten/src/compiler.js /tmp/tmphmgppfbl.txt' failed (1)
make[5]: *** [CMakeFiles/texturev.dir/build.make:102: texturev.js] Error 1
make[4]: *** [CMakeFiles/Makefile2:420: CMakeFiles/texturev.dir/all] Error 2

Copy link
Owner

@pr0g pr0g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making the updates, this is looking pretty cool now! I just have two minor nits and then I'll see if I can try it out later. You mentioned there's still an issue right now? I'll see if I hit it too.

bgfx-emscripten/bgfx_emscripten_utils.hpp Outdated Show resolved Hide resolved
main.cpp Outdated Show resolved Hide resolved
@pr0g
Copy link
Owner

pr0g commented Aug 14, 2021

Hmm, adding CMAKE_COMMAND to the external projects does build everything as Emscripten-linkable, but it gives me this error when building texturev & geometryv:

error: undefined symbol: emscripten_webgl2_get_proc_address (referenced by top-level compiled C/C++ code)
warning: Link with `-s LLD_REPORT_UNDEFINED` to get more information on undefined symbols
warning: To disable errors for undefined symbols use `-s ERROR_ON_UNDEFINED_SYMBOLS=0`
warning: _emscripten_webgl2_get_proc_address may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
Error: Aborting compilation due to previous errors
em++: error: '/home/carson/emsdk/node/14.15.5_64bit/bin/node /home/carson/emsdk/upstream/emscripten/src/compiler.js /tmp/tmphmgppfbl.txt' failed (1)
make[5]: *** [CMakeFiles/texturev.dir/build.make:102: texturev.js] Error 1
make[4]: *** [CMakeFiles/Makefile2:420: CMakeFiles/texturev.dir/all] Error 2

Hmm so do all the third-party dependencies build? I take it you'll need to add the change to CMakeLists.txt in the third-party folder too (From looking at that earlier link)?

Does there need to be an emscripten configure script too that uses emcmake? Is that causing a problem? Again later today I'll see if I can test it out and let you know 👍

@decahedron1
Copy link
Contributor Author

Just pushed the changes I made to the third-party CMakeLists 👍

The third-party dependencies don't build, that's what's giving me the emscripten_webgl2_get_proc_address error. But it seems to be a problem with my (seemingly cursed) environment, as just building bgfx.cmake alone with emcmake gave me the same error. emscripten_webgl2_get_proc_address is declared in both my Emscripten headers and in bgfx's glcontext_html5.cpp so I'm very confused why this happens.

@pr0g
Copy link
Owner

pr0g commented Aug 14, 2021

Hey @sudo-carson,

I was able to sync your branch and try building bgfx but ran into the same issue you described...

error: undefined symbol: emscripten_webgl2_get_proc_address (referenced by top-level compiled C/C++ code)
warning: Link with `-s LLD_REPORT_UNDEFINED` to get more information on undefined symbols
warning: To disable errors for undefined symbols use `-s ERROR_ON_UNDEFINED_SYMBOLS=0`
warning: _emscripten_webgl2_get_proc_address may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
Error: Aborting compilation due to previous errors
em++: error: '/.../Documents/Projects/emsdk/node/14.15.5_64bit/bin/node /.../Documents/Projects/emsdk/upstream/emscripten/src/compiler.js /var/folders/7q/9jvc3pcs59s9y6y_rx7wrpg00000gn/T/tmpsxf9chsv.txt' failed (returned 1)
make[5]: *** [texturev.js] Error 1
make[4]: *** [CMakeFiles/texturev.dir/all] Error 2
make[3]: *** [all] Error 2
make[2]: *** [bgfx-prefix/src/bgfx-stamp/bgfx-build] Error 2
make[1]: *** [CMakeFiles/bgfx.dir/all] Error 2
make: *** [all] Error 2
emcmake: error: 'cmake --build build -DCMAKE_TOOLCHAIN_FILE=/.../Documents/Projects/emsdk/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake -DCMAKE_CROSSCOMPILING_EMULATOR=/.../Documents/Projects/emsdk/node/14.15.5_64bit/bin/node' failed (returned 2)

So I don't think it's a local environment issue. I'm not really sure why it's failing either. Is there any info on the bgfx repo about people building using Emscripten and hitting this?

@decahedron1
Copy link
Contributor Author

No issues about this specific error on bgfx. I opened a discussion there but haven't got a response yet. Super strange... Maybe it's a problem with emscripten? Hmm

@decahedron1
Copy link
Contributor Author

The only mention of an undefined reference to emscripten_webgl2_get_proc_address is this Unity post, doesn't seem to have anything that'll help us here though. I'll try building on Windows to see if that fixes it at all.

@decahedron1
Copy link
Contributor Author

🤔

configure: cmake -S . -B build-vs -G "Visual Studio 16 2019" -A x64 -DCMAKE_PREFIX_PATH=E:\Misc\sdl-bgfx-imgui-starter/third-party/build -DCMAKE_TOOLCHAIN_FILE=E:\Programs\emsdk\upstream\emscripten\cmake\Modules\Platform\Emscripten.cmake -DCMAKE_CROSSCOMPILING_EMULATOR="E:/Programs/emsdk/node/14.15.5_64bit/bin/node"
CMake Error at CMakeLists.txt:5 (find_package):
  Could not find a package configuration file provided by "SDL2" with any of
  the following names:

    SDL2Config.cmake
    sdl2-config.cmake

  Add the installation prefix of "SDL2" to CMAKE_PREFIX_PATH or set
  "SDL2_DIR" to a directory containing one of the above files.  If "SDL2"
  provides a separate development package or SDK, be sure it has been
  installed.


-- Configuring incomplete, errors occurred!
See also "E:/Misc/sdl-bgfx-imgui-starter/build-vs/CMakeFiles/CMakeOutput.log".

@decahedron1
Copy link
Contributor Author

decahedron1 commented Aug 14, 2021

Got it built on Unix!

set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -s MAX_WEBGL_VERSION=2")
set(CMAKE_CXX_FLAGS "${CMAKE_C_FLAGS} -s MAX_WEBGL_VERSION=2")

Well... sort of...

SDL_SysWMinfo could not be retrieved. SDL_Error: That operation is not supported

🥲

@pr0g
Copy link
Owner

pr0g commented Aug 14, 2021

Hey there just catching up on things, hmm interesting. Not sure about that SDL error on Windows, actually looking at the command you have a mix of forward and backward slashes that might be confusing it so it doesn't find the SDL library (with -DCMAKE_PREFIX_PATH). Also, I think with the flags thing it's better to use target_compile_options instead of setting the flags globally if possible. I'll see if I can have a try on Windows tomorrow (the test I had earlier was macOS). Sounds like it's getting closer though! 🙂

@pr0g
Copy link
Owner

pr0g commented Aug 14, 2021

Oh the other thing I forgot earlier sorry is for this PR the one other important thing we should add is an update to the README (docs) to let people know emscripten is available and how to build for it (once everything is working that is 🙂)

@decahedron1
Copy link
Contributor Author

Oh the other thing I forgot earlier sorry is for this PR the one other important thing we should add is an update to the README (docs) to let people know emscripten is available and how to build for it (once everything is working that is 🙂)

Yep, just focusing on getting it built and running first 👍

Adding -DCMAKE_C_FLAGS="-s MAX_WEBGL_VERSION=2" to the third-party external project CMake args gave me a bunch of errors like clang: unknown option -s.

I'm also trying to build this project with as little things as possible + without ImGui and third-party downloading stuff (just add_subdirectory and target_link_libraries) because that's been the biggest challenge so far and it seems promising; it builds and runs with no errors but doesn't render anything. One thing I noticed is that both -s MAX_WEBGL_VERSION=2 and -s ALLOW_MEMORY_GROWTH=1 need to be enabled for it to run without errors... but not without warnings. I get tons of WebGL: INVALID_ENUM: getInternalformatParameter: invalid internalformat and WebGL: INVALID_ENUM: getInternalformatParameter: invalid target warnings in the console and no image. Good news is SDL initializes properly though so this is a bgfx thing. Maybe we're not initializing something right...?

Also, thanks for the heads up about target_compile_options - I'm a CMake noob and just used the first thing I found on Google 😅

@decahedron1
Copy link
Contributor Author

decahedron1 commented Aug 14, 2021

Well, I'm not getting warnings running it anymore - but it just hangs at 100% CPU. No graphical or console output. 🥲

@pr0g
Copy link
Owner

pr0g commented Aug 14, 2021

Thanks for the update, I tried the third-party stuff on Windows but ran into problems unfortunately. I could use...

emcmake cmake -B build

which seemed to work okay, but then...

emcmake cmake --build build wasn't recognized correctly

and cmake --build build got confused with the git-clone.sh file (which works fine normally 🤔)

So not sure I'm afraid 😬 Let me know if you have any more luck!

No worries about the target_compile_options thing, sometimes the global flags are a good idea (say with a project you might want to enable C++ 17 for or something) but as with most things limiting scope is usually good 🙂

I'm not a CMake expert but did write up a bunch about it here (mainly to help myself remember how to use it 😝 https://github.com/pr0g/cmake-examples

@decahedron1
Copy link
Contributor Author

@pr0g I believe the commit I just pushed fixes all problems with the code. Now all I have to do is figure out how to get -s MAX_WEBGL_VERSION=2 to the external projects and it should work (no guarantees 😅)

@pr0g
Copy link
Owner

pr0g commented Aug 17, 2021

Hey @sudo-carson, sorry I'm just getting back to you, work's been pretty hectic recently.

That's great to hear, I'll try and pull the latest changes soon and give it a whirl.

With the -s MAX_WEBGL_VERSION=2 argument try and see if there's any info out there about passing arguments to ExternalProject_Add, I'm sure it'll be possible (is that for the configure step?). I'll have a noodle to see if I can get it to work too at some point. I'll let you know if I have any more info 👍

Cheers!

@decahedron1
Copy link
Contributor Author

Adding the args to ExternalProject_Add with something like CMAKE_ARGS "-DCMAKE_C_FLAGS=\"-s MAX_WEBGL_VERSION=2\"" gave me clang errors about unknown options... Hmm

@pr0g
Copy link
Owner

pr0g commented Aug 19, 2021

Hmm interesting... not sure what that'd happen. I'll try have a look later this week/weekend to see if I can find the right incantation 😝 Cheers!

@pr0g
Copy link
Owner

pr0g commented Aug 22, 2021

Hey there, I did noodle around with this again for a little while this evening (on Mac) but still kept hitting the error: undefined symbol: emscripten_webgl2_get_proc_address issue.

I tried passing -s MAX_WEBGL_VERSION=2 to CMAKE_ARGS in the ExternalProject_Add command but it didn't seem to fix it. I tried with CMAKE_EXE_LINKER_FLAGS as well as CMAKE_CXX_FLAGS (I think the error you described earlier might have been because you weren't appending to CMAKE_CXX_FLAGS, you need to include the existing ${CMAKE_CXX_FLAGS} variable if I remember correctly).

I did some googling and found a few other people having issues with bgfx and emscripten in general...

So not sure, basically from third-party I'm running this...

source <path/to/emsdk>/emsdk_env.sh
emcmake cmake -B build
emcmake cmake --build build

And then I hit the link error when linking texturev.js...

It's possible something is missing in bgfx.cmake? Might be worth looking at the genie build files in the main bgfx repo to see If the cmake build is missing something.

@decahedron1
Copy link
Contributor Author

bgfx.cmake and bgfx genie files both omit MAX_WEBGL_VERSION... hm. Only thing I can think of now is to use git submodules & add_subdirectory instead of ExternalProject_Add, don't know why CMake won't let us change those variables with external projects...

@pr0g
Copy link
Owner

pr0g commented Aug 22, 2021

I'm afraid I don't want to switch to submodules sorry... are you able to just build bgfx using CMake with emscripten on its own?

@pr0g
Copy link
Owner

pr0g commented Aug 22, 2021

The problem seems to be the definition of emscripten_webgl2_ext_get_proc_address is not found (and bgfx should link against it). It seems that happens based on the environment? It's not explicit in the CMakeLists.txt file...

You can see the definition here - https://git.sprintf.io/emscripten-ports/emscripten/commit/f8d9751855d78e36a78a30d007eeadebf01cc610?style=split, I'm not sure why specifying MAX_WEBGL_VERSION fixes it (it didn't when I tried to build)

@pr0g
Copy link
Owner

pr0g commented Aug 23, 2021

I just built bgfx.cmake using emscripten on its own and got the exact same issue, so it has nothing to do with ExternalProject_Add. There's likely an issue in one of the CMake scripts for bgfx missing some dependency. It's likely worth studying the GENie .lua build files to see how they differ.

@pr0g
Copy link
Owner

pr0g commented Aug 23, 2021

I made a new fork off bkaradzic/bgfx.cmake here pr0g/bgfx.cmake and updated it to work when building with emscripten - pr0g/bgfx.cmake@5e7ec5f

I had to make this local change to third-party/CMakeLists.txt

ExternalProject_Add(
    bgfx
    GIT_REPOSITORY https://github.com/pr0g/bgfx.cmake.git # <-------- updated repo
    GIT_TAG        5e7ec5f7d783fb348608abc6f932a34b937f0115 # <------ updated commit
    BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR}/bgfx/build/${build_type_dir}
    INSTALL_DIR ${CMAKE_CURRENT_BINARY_DIR}
    CMAKE_COMMAND ${THIRD_PARTY_CMAKE_COMMAND}
    CMAKE_ARGS ${build_type_arg}
               -D CMAKE_INSTALL_PREFIX=<INSTALL_DIR>
               "$<$<CONFIG:Debug>:-DCMAKE_DEBUG_POSTFIX=d>"
               -D BGFX_BUILD_EXAMPLES=OFF)

With this change, things built fine in third-party, might be worth giving it a try your end.

@pr0g
Copy link
Owner

pr0g commented Aug 23, 2021

Argh but now when trying to build the actual starter demo I'm hitting an issue where emscripten does not respect CMAKE_PREFIX_PATH - there are people mentioning the same issue on this thread - emscripten-core/emscripten#6595. Looks like it might be possible to use CMAKE_FIND_ROOT_PATH_BOTH with the find_package command, I'll give that a shot and see.

@pr0g
Copy link
Owner

pr0g commented Aug 23, 2021

I managed to get everything building and linking but it required quite a few changes that I'm not a massive fan of...

As Emscripten is quite different to the normal building approach I think it's actually introducing quite a lot of complexity for the 'normal' case so I'm leaning towards this PR being moved to a separate repo that's Emscripten only. That way we don't have to jump through hoops to support Mac/Windows/Linux/Enscripten at the same time.

In the latest version of this PR there were a lot of compile errors with things not being hooked up correctly. I have pushed a branch with my fixes here - https://github.com/pr0g/sdl-bgfx-imgui-starter/tree/sudo-carson-platform/emscripten (4a4cab8).

To build use these commands from the root of the repo (this is after building third-party above).

emcmake cmake -S . -B build -DCMAKE_PREFIX_PATH="$(pwd)/third-party/build"
emcmake cmake --build build

It still doesn't actually work (I think it might be related to the fetch_file function, though I'm not sure) - you can open the app in a browser but at the moment it hangs on 'Preparing'.

I'll wait to hear back from you but I think at this point it might be worth taking what we've learned to another repo that deals with Emscripten exclusively (we don't need SDL2 in third-party then for example).

Thanks,

Tom

@pr0g
Copy link
Owner

pr0g commented Aug 23, 2021

I made another small update (0213a3f) that actually does see bgfx get initialised (I realised I had to start a local web server with python3 -m http.server) but I then see lots of warnings about WebGL Invalid Enum getInternalFormatParameter: invalid internal format. Not 100% sure what's that about... there's still just a black screen unfortunately but it is 'running'. Giving up for now, but if you do have any more luck give me a shout 👍

@decahedron1
Copy link
Contributor Author

... but I then see lots of warnings about WebGL Invalid Enum getInternalFormatParameter: invalid internal format. Not 100% sure what's that about...

That's (apparently) normal with bgfx - I get the same thing on a simple bgfx example and it renders just fine. Not sure why the black screen - probably has something to do with the shaders because I made that loader code completely from memory and without testing it -- I'm surprised it compiled at all 😅

Here's a super simple working example that I can confirm works, hope this is useful in figuring out the black screen problem:

#include <SDL.h>
#include <SDL_syswm.h>
#include <bgfx/bgfx.h>
#include <bgfx/platform.h>
#include <bx/math.h>
#include <bx/bx.h>
#include <stdio.h>

#include <emscripten.h>
#include <emscripten/html5.h>

void loop(void) {
    SDL_Event currentEvent;
    while (SDL_PollEvent(&currentEvent) != 0) {
        if (currentEvent.type == SDL_QUIT) {
            return;
        }
    }

    bgfx::setViewRect(0, 0, 0, 800, 600);

    bgfx::touch(0);

    bgfx::dbgTextClear();
    bgfx::dbgTextPrintf(0, 1, 0x0f, "it works!");

    bgfx::frame();
}

int main(int argc, char** argv)
{
    if (SDL_Init(SDL_INIT_VIDEO) < 0) {
        printf("SDL could not initialize. SDL_Error: %s\n", SDL_GetError());
        return 1;
    }

    const int width = 800;
    const int height = 600;
    SDL_Window* window = SDL_CreateWindow(
        "bgfx", SDL_WINDOWPOS_UNDEFINED, SDL_WINDOWPOS_UNDEFINED, width,
        height, SDL_WINDOW_SHOWN);

    if (window == nullptr) {
        printf("Window could not be created. SDL_Error: %s\n", SDL_GetError());
        return 1;
    }

    static const char *canvas = "#canvas";

    bgfx::PlatformData pd;
    bx::memSet(&pd, 0, sizeof(pd));
    pd.nwh = (void*)canvas;
    bgfx::setPlatformData(pd);

    bgfx::Init bgfx_init;
    bgfx_init.type = bgfx::RendererType::Count; // auto choose renderer
    bgfx_init.resolution.width = width;
    bgfx_init.resolution.height = height;
    bgfx_init.resolution.reset = BGFX_RESET_VSYNC;
    bgfx::init(bgfx_init);

    bgfx::setDebug(BGFX_DEBUG_TEXT);

    bgfx::setViewClear(
        0, BGFX_CLEAR_COLOR | BGFX_CLEAR_DEPTH, 0x6495EDFF, 1.0f, 0);
    bgfx::setViewRect(0, 0, 0, width, height);

    bgfx::frame();

    emscripten_set_main_loop(loop, -1, 1);

    bgfx::shutdown();

    SDL_DestroyWindow(window);
    SDL_Quit();

    return 0;
}

@pr0g
Copy link
Owner

pr0g commented Aug 24, 2021

Haha okay understood. Back at work today so won't have as much time to noodle with this but I'll see if I can get back to it later in the week, thanks for the example, it'll be useful to narrow down what the problem is.

@pr0g
Copy link
Owner

pr0g commented Aug 30, 2021

I had another little noodle around with this, this evening. The Dear ImGui stuff works fine, but the fetch_file stuff still isn't working. I updated the call to use polling (emscripten_fetch_wait) which got a bit closer, the other option is to build with --proxy-to-worker and -s USE_PTHREADS=1 to support synchronous loading.

Screenshot 2021-08-30 at 22 26 07

The other option might be to embed the shaders, but that's a bit of a pain... I'll update if I push any more commits

@pr0g
Copy link
Owner

pr0g commented Sep 1, 2021

It lives! 😄

Screenshot 2021-09-01 at 21 15 24

I pushed the latest changes to get it working here - 47cd548

I did some more digging and found you can still use C/C++ file loading APIs with Emscripten, you just have to let Emscripten know ahead of time which files you want to load with --preload-file=<filename> (more info here). For this to work I had to copy the compiled shader files (v_simple.bin and f_simple.bin) to the build/ folder. I also used the compile-shaders-linux.sh script to compile the shaders for OpenGL. To have an executable of shaderc.exe I compiled bgfx normally so I could use this to compile the shaders.

I removed the bgfx_emscripten_utils.hpp file as it was no longer needed and one other change I had to make was to use SDL_GetMouseState instead of SDL_GetGlobalMouseState which isn't supported in Emscripten.

Let me know if you're able to try this updated branch and get it working. I'll have another think about the best way to integrate it, whether that's a separate repo or as part of this one, I'm still on the fence right now...

@decahedron1
Copy link
Contributor Author

I was able to get it working on my end with those changes, so glad to finally see it working after all this time! 😄

@pr0g
Copy link
Owner

pr0g commented Sep 1, 2021

Great to hear! 😄 Thanks for letting me know 👍

@pr0g
Copy link
Owner

pr0g commented Sep 12, 2021

I'm closing this PR as it is now replaced by #9.

The core work by @sudo-carson is still there but there have been additional updates to ensure everything works cross-platform and the docs reflect the Emscripten addition.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants