NervLand DevLog #25: Introducing support for IMGUI

Hello hello! In our previous devlog article we introduced some “additional features” on the text rendering system I'm building in NervLand. One of the main interest when introducing such support for text rendering, if of course to get ready to builf some user interfaces, and eventually, I think I will get to this. But for this new episode, I'm rather going to try to take a shortcut for once, and figure out how to integrate IMGUI in our engine 😎!

From what I've read already, imgui should provide us with “backends” for both SDL2 and webgpu, which are exactly what we need, so, what go wrong really ?? 🤣🤣🤣

Youtube video for this article available at:

Hmmm… first thing already: I have already forked the imgui repository on github (cf. https://github.com/ocornut/imgui), but I forked the master branch, and now, I can't find a to change to the docking branch 😲, come on github, really ?!

And in the end I could find a way to have 2 different branches forked on github directly, so the easiest solution for me here is to delete my existing fork. For that, you need to go on the forked repository Settings tab:

Then you navigate down to the Danger zone:

And now, I can create a new fork, using the branch docking as source 😉! But this time I'm making sure that I'm not only forking the master branch by unchecking the checkbox below (⚠️ it is checked by default):

Now checking out the repo ans switching to the docking branch:

git clone git@github.com:roche-emmanuel/imgui.git
cd imgui
git fetch
git branch -r
git checkout -t origin/docking

Now I need to integrate the imgui sources into my own project:

Then I just had to extend my list of input files and add the include directory for SDL2, and the build of the nvGPU module will go just fine 👍!

Using this file as template: https://github.com/ocornut/imgui/blob/master/examples/example_emscripten_wgpu/main.cpp

I now everything setup properly and added a dedicated render pass to process the imgui data:

    // Add a dedicated render pass to display the imgui data:
    auto& rpass2 = eng->create_render_pass(width, height, {true, 0});
    rpass2.set_encode_func([](RenderPassEncoder& pass) {
        ImGui_ImplWGPU_RenderDrawData(ImGui::GetDrawData(), pass.Get());
    });

but unfortunately, this doesn't work 😭… some validation error from wgpu… let's see…

Hmmm, I have this error message:

2023-07-27 13:30:39.115464 [ERROR] Dawn: Validation error: Attachment state of [RenderPipeline] is not compatible with [RenderPassEncoder].
[RenderPassEncoder] expects an attachment state of { colorFormats: [TextureFormat::BGRA8Unorm], depthStencilFormat: TextureFormat::Depth24PlusStencil8, sampleCount: 1 }.
[RenderPipeline] has an attachment state of { colorFormats: [TextureFormat::RGBA8Unorm], sampleCount: 1 }.

So it could simply be that I need to pass the depth stencil format too when initializing imgui. Trying that with:

    ImGui_ImplWGPU_Init(eng->get_device(), 3, WGPUTextureFormat_RGBA8Unorm,
                        WGPUTextureFormat_Depth24PlusStencil8);

Nope 😭😭, still not good enough:

2023-07-27 13:35:51.843629 [ERROR] Dawn: Validation error: Attachment state of [RenderPipeline] is not compatible with [RenderPassEncoder].
[RenderPassEncoder] expects an attachment state of { colorFormats: [TextureFormat::BGRA8Unorm], depthStencilFormat: TextureFormat::Depth24PlusStencil8, sampleCount: 1 }.
[RenderPipeline] has an attachment state of { colorFormats: [TextureFormat::RGBA8Unorm], depthStencilFormat: TextureFormat::Depth24PlusStencil8, sampleCount: 1 }.

Ohhh, that's the color attachment format in fact, I'm using BRGA myself but configured imgui to use RGBA, stupid me. Fixing it… Cooll! Now it's starting to work a little bit more, here is what I see:

Mouse movements are detected properly since the hover color is changed when I move on different elements, but I can't click any widget, or drag anything. So could be the mouse button handling is not working yet ? Also it's strange that the window seems to be pushed on the right, and also strange that I cannot see my background cubemap 🤔. Let's start investigating.

Oh, I see, this is what I was missing:

// You can read the io.WantCaptureMouse, io.WantCaptureKeyboard flags to tell if dear imgui wants to use your inputs.
// - When io.WantCaptureMouse is true, do not dispatch mouse input data to your main application, or clear/overwrite your copy of the mouse data.
// - When io.WantCaptureKeyboard is true, do not dispatch keyboard input data to your main application, or clear/overwrite your copy of the keyboard data.
// Generally you may always pass all inputs to dear imgui, and hide them from your application based on those two flags.
// If you have multiple SDL events and some of them are not meant to be used by dear imgui, you may need to filter events based on their windowID field.
bool ImGui_ImplSDL2_ProcessEvent(const SDL_Event* event)

I have thus introduced support to add custom global event handlers in the SDLWindowManager:

    auto* wman = (SDLWindowManager*)SDLWindowManager::instance();
    wman->add_event_handler([](SDL_Event& evt) {
        ImGui_ImplSDL2_ProcessEvent(&evt);
        // ImGuiIO& io = ImGui::GetIO();
        return false;
    });

And now the display of the imgui demo window is OK!

But I still have this black background issue… or could this just be because of my render pass setup 🤔? Checking this… And sure enough, this was it: my second render pass was configured to clear its target, fixing it:

    auto& rpass2 = eng->create_render_pass(
        width, height,
        {.with_depth = true, .swapchain_idx = 0, .clear_color = false});
    rpass2.set_encode_func([](RenderPassEncoder& pass) {
        ImGui_ImplWGPU_RenderDrawData(ImGui::GetDrawData(), pass.Get());
    });

And now I have this nice display result:

⇒ That is so amazing 🤩!

Yet, there is a final change I need to make: as mentioned in the C++ comment above I should prevent the handling my the mouse events if IMGUI is currently “using” the mouse, otherwise I'm interacting with the scene when moving the GUI itself:

    auto* wman = (SDLWindowManager*)SDLWindowManager::instance();
    wman->add_event_handler([](SDL_Event& evt) {
        ImGui_ImplSDL2_ProcessEvent(&evt);
        ImGuiIO& io = ImGui::GetIO();

        return io.WantCaptureMouse || io.WantCaptureKeyboard;
    });

⇒ Allright, with that small change I can now drag/interact with the GUI without intefering on the scene itself.

Just activated the support for docking/multi-viewports:

    ImGui::CreateContext();
    ImGuiIO& io = ImGui::GetIO();
    (void)io;
    io.ConfigFlags |=
        ImGuiConfigFlags_NavEnableKeyboard; // Enable Keyboard Controls
    io.ConfigFlags |=
        ImGuiConfigFlags_NavEnableGamepad;            // Enable Gamepad Controls
    io.ConfigFlags |= ImGuiConfigFlags_DockingEnable; // Enable Docking
    io.ConfigFlags |= ImGuiConfigFlags_ViewportsEnable; // Enable Multi-Viewport
                                                        // / Platform Windows
    // io.ConfigViewportsNoAutoMerge = true;
    // io.ConfigViewportsNoTaskBarIcon = true;

And docking seems to work, even if that's not fully what I was expecting (I was expecting to be able to dock at the border of the actual SDL window, but that is not how it works ):

Unfortunately, the “multi-viewports” part doesn't seem to work 🤔… Not real idea why, maybe because of the webgpu backend not really supporting that yet ? I could try to investigate this further one day, but for now I think this is not a priority anyway.

Ohhh, but in fact, I just noticed in the SDL2/OpenGL3 example that there is another dedicated code section for the multi-viewports that I don't have yet:

        // Update and Render additional Platform Windows
        // (Platform functions may change the current OpenGL context, so we save/restore it to make it easier to paste this code elsewhere.
        //  For this specific demo app we could also call SDL_GL_MakeCurrent(window, gl_context) directly)
        if (io.ConfigFlags & ImGuiConfigFlags_ViewportsEnable)
        {
            SDL_Window* backup_current_window = SDL_GL_GetCurrentWindow();
            SDL_GLContext backup_current_context = SDL_GL_GetCurrentContext();
            ImGui::UpdatePlatformWindows();
            ImGui::RenderPlatformWindowsDefault();
            SDL_GL_MakeCurrent(backup_current_window, backup_current_context);
        }

⇒ Let's try to add something like that:

    // Update and Render additional Platform Windows
    // (Platform functions may change the current OpenGL context, so we
    // save/restore it to make it easier to paste this code elsewhere.
    // For this specific demo app we could also call SDL_GL_MakeCurrent(window,
    // gl_context) directly)
    if ((io.ConfigFlags & ImGuiConfigFlags_ViewportsEnable) != 0) {
        ImGui::UpdatePlatformWindows();
        ImGui::RenderPlatformWindowsDefault();
    }

Hmmm, nope, not really changing anything. And I eventually realized that, even if I request this feature, the flag will be removed from the ConfigFlags. And searching in the imgui code I found this section:

        if ((g.IO.BackendFlags & ImGuiBackendFlags_PlatformHasViewports) && (g.IO.BackendFlags & ImGuiBackendFlags_RendererHasViewports))
        {
            IM_ASSERT((g.FrameCount == 0 || g.FrameCount == g.FrameCountPlatformEnded) && "Forgot to call UpdatePlatformWindows() in main loop after EndFrame()? Check examples/ applications for reference.");
            IM_ASSERT(g.PlatformIO.Platform_CreateWindow  != NULL && "Platform init didn't install handlers?");
            IM_ASSERT(g.PlatformIO.Platform_DestroyWindow != NULL && "Platform init didn't install handlers?");
            IM_ASSERT(g.PlatformIO.Platform_GetWindowPos  != NULL && "Platform init didn't install handlers?");
            IM_ASSERT(g.PlatformIO.Platform_SetWindowPos  != NULL && "Platform init didn't install handlers?");
            IM_ASSERT(g.PlatformIO.Platform_GetWindowSize != NULL && "Platform init didn't install handlers?");
            IM_ASSERT(g.PlatformIO.Platform_SetWindowSize != NULL && "Platform init didn't install handlers?");
            IM_ASSERT(g.PlatformIO.Monitors.Size > 0 && "Platform init didn't setup Monitors list?");
            IM_ASSERT((g.Viewports[0]->PlatformUserData != NULL || g.Viewports[0]->PlatformHandle != NULL) && "Platform init didn't setup main viewport.");
            if (g.IO.ConfigDockingTransparentPayload && (g.IO.ConfigFlags & ImGuiConfigFlags_DockingEnable))
                IM_ASSERT(g.PlatformIO.Platform_SetWindowAlpha != NULL && "Platform_SetWindowAlpha handler is required to use io.ConfigDockingTransparent!");
        }
        else
        {
            // Disable feature, our backends do not support it
            g.IO.ConfigFlags &= ~ImGuiConfigFlags_ViewportsEnable;
        }

So yeah, pretty sure that WebGPU doesn't support multiviewport yet (as this is probably implemented with on rendering in browser with a “single window” in mind 😉). As I said: never mind for now.