Bindings generation for QT6

In this post I'm covering the compilation of QT6 on linux, and then I continue with more lua bindings on windows: that second part was a lot of fun actually, and I ended up with a working minimal QT window built completely from lua 😁! There is still a lot to do here, but who knows, maybe someday I'm gonig to replace pyQT6 this way ?! 🤣. Just keep reading below if you're interested in some painful errors or brief moment of joy on this journey

• We have already built the QT6 library successfully on windows (but only with the MSVC compiler so far), so now it's time to also build that dependency on linux:
• First, we prepare the build_on_linux() method in the QT6 NVP builder:

      def build_on_linux(self, build_dir, prefix, _desc):
          """Build method for QT6 on linux"""
  
          # Next we call cmake to generate the config.opt file:
          cmd = [
              self.tools.get_cmake_path(),
              "-DIN_FILE=config.opt.in",
              "-DOUT_FILE=config.opt",
              "-DIGNORE_ARGS=-top-level",
              "-P",
              f"{build_dir}/qtbase/cmake/QtWriteArgsFile.cmake",
          ]
  
          self.check_execute(cmd, cwd=build_dir, env=self.env)
  
          logger.info("Done generating config.opt file.")
  
          # prepare the python env:
          pyenv = self.ctx.get_component("pyenvs")
          pdesc = {"inherit": "default_env", "packages": ["html5lib"]}
  
          pyenv.add_py_env_desc("qt6_env", pdesc)
  
          pyenv.setup_py_env("qt6_env")
          py_dir = pyenv.get_py_env_dir("qt6_env")
          py_dir = self.get_path(py_dir, "qt6_env")
  
          # Prepare a nodejs env:
          nodejs = self.ctx.get_component("nodejs")
  
          nodejs_dir = self.get_path(build_dir, "qt6_env")
          ndesc = {"nodejs_version": "18.13.0", "packages": [], "install_dir": build_dir}
  
          nodejs.setup_nodejs_env("qt6_env", env_dir=build_dir, desc=ndesc, update_npm=True)
  
          dirs = [
              self.get_path(build_dir, "qtbase", "bin"),
              py_dir,
              nodejs_dir,
              # gperf_dir,
              # bison_dir,
              # flex_dir,
              # self.get_path(perl_dir, "perl", "site", "bin"),
              # self.get_path(perl_dir, "perl", "bin"),
              # self.get_path(perl_dir, "c", "bin"),
          ]
          logger.info("Adding additional paths: %s", dirs)
  
          self.env = self.append_env_list(dirs, self.env)
  
          logger.info("Environment paths: %s", self.env["PATH"])
  
          # Configuration step:
          cmd = [
              self.tools.get_cmake_path(),
              "-DOPTFILE=config.opt",
              "-DTOP_LEVEL=TRUE",
              "-P",
              f"{build_dir}/qtbase/cmake/QtProcessConfigureArgs.cmake",
              "-Wno-dev",
          ]
  
          self.check_execute(cmd, cwd=build_dir, env=self.env)
  
          # Building the library now:
          logger.info("Building QT6 libraries...")
          cmd = [self.tools.get_cmake_path(), "--build", ".", "--parallel"]
          self.check_execute(cmd, cwd=build_dir, env=self.env)
  
          # Testing direct execution of ninja to get mode debug outputs:
          # self.exec_ninja(build_dir, ["-v"])
  
          logger.info("Installing QT6 libraries...")
          cmd = [self.tools.get_cmake_path(), "--install", "."]
          self.check_execute(cmd, cwd=build_dir, env=self.env)
  
          pyenv.remove_py_env("qt6_env")
  

• On linux it seems we also need the gperf/bison/flex tools, so just added a mechanism to be able to compile those tools from sources:

    - name: gperf
      sub_path: bin/gperf
      version: 3.1
      urls:
        - http://files.nervtech.org/nvp_packages/tools/gperf-3.1-linux-clang.tar.xz
      # For building:
      # url: http://ftp.gnu.org/pub/gnu/gperf/gperf-3.1.tar.gz
      # build_mode: "std"
    - name: bison
      sub_path: bin/bison
      version: 3.8.2
      urls:
        - http://files.nervtech.org/nvp_packages/tools/bison-3.8.2-linux-clang.tar.xz
      # For building:
      # url: http://ftp.gnu.org/gnu/bison/bison-3.8.2.tar.xz
      # build_mode: "std"
    - name: flex
      sub_path: bin/flex
      version: 2.6.4
      urls:
        - http://files.nervtech.org/nvp_packages/tools/flex-2.6.4-linux-clang.tar.xz
      # For building:
      # url: https://github.com/westes/flex/files/981163/flex-2.6.4.tar.gz
      # build_mode: "std"

• And we now have a dedicated method in the tools component to “build a tool”:

      def build_tool(self, full_name, desc):
          """Build a tool package from sources"""
          # Get the build directory:
          base_build_dir = self.make_folder(self.ctx.get_root_dir(), "build", "tools")
          prefix = self.get_path(self.tools_dir, full_name)
  
          # get the build manager:
          bman = self.get_component("builder")
  
          # Prepare the build folder:
          build_dir, _, _ = bman.setup_build_context(desc, False, base_build_dir)
  
          # Run the build system:
          bmode = desc['build_mode']
          if bmode == "std":
              # Run configure/make std commands:
              builder = NVPBuilder(bman)
              builder.init_env()
              builder.run_configure(build_dir, prefix)
              builder.run_make(build_dir)
          else:
              self.throw("Unsupported build mode: %s", bmode)
  
          pkgname = bman.get_library_package_name(full_name)
          self.create_package(prefix, self.tools_dir, pkgname)

• Now it also seems we need the NSS library to build the QtWebEngine on linux:

  -- Support check for QtWebEngine failed: Build requires nss >= 3.26.

• I think this should rather be built as a full library instead of a simple tool, so we will write a builder for it.
• For the nspr library we can use a simple builder:

  class Builder(NVPBuilder):
      """nspr builder class."""
  
      def build_on_windows(self, build_dir, prefix, _desc):
          """Build on windows method"""
          raise NotImplementedError()
  
      def build_on_linux(self, build_dir, prefix, _desc):
          """Build on linux method"""
          self.run_configure(build_dir, prefix, ["--enable-optimize", "--enable-64bit"])
  
          self.run_make(build_dir)

• But for the NSS library itself, it seems we need a python environment with gyp-next at least.
• ⇒ And in the end providing the nss library this way did not work, so I had to install the requiredd packages directly instead:

  sudo apt-get install libnss3-dev libdbus-1-dev

• Then I finally got the QtWebEngine to start compiling but faced another error:

  /mnt/data1/dev/projects/NervProj/build/libraries/QT6-6.4.2/qtwebengine/src/3rdparty/chromium/third_party/devtools-frontend/src/node_modules/yargs/node_modules/yargs-parser/build/index.cjs:1013
          throw Error(`yargs parser supports a minimum Node.js version of ${minNodeVersion}. Read our version support policy: https://github.com/yargs/yargs-parser#supported-nodejs-versions`);
          ^
  
  Error: yargs parser supports a minimum Node.js version of 12. Read our version support policy: https://github.com/yargs/yargs-parser#supported-nodejs-versions

• ⇒ trying with node “v12.22.9” instead. Nope, same error.

• Actually checking the error context a bit more we have this command executed from python apparently:

  FAILED: gen/third_party/devtools-frontend/src/front_end/devtools_app.html gen/third_party/devtools-frontend/src/front_end/inspector.html gen/third_party/devtools-frontend/src/front_end/js_app.html gen/third_party/devtools-frontend/src/front_end/ndb_app.html gen/third_party/devtools-frontend/src/front_end/node_app.html gen/third_party/devtools-frontend/src/front_end/worker_app.html gen/third_party/devtools-frontend/src/front_end/device_mode_emulation_frame.html 
  /mnt/data1/dev/projects/NervProj/.pyenvs/qt6_env/bin/python3 ../../../3rdparty/chromium/third_party/node/node.py ../../../3rdparty/chromium/third_party/devtools-frontend/src/scripts/build/generate_html_entrypoint.js --template ../../../3rdparty/chromium/third_party/devtools-frontend/src/front_end/entrypoint_template.html --out-directory gen/third_party/devtools-frontend/src/front_end --entrypoints devtools_app --entrypoints inspector --entrypoints js_app --entrypoints ndb_app --entrypoints node_app --entrypoints worker_app --entrypoints device_mode_emulation_frame
  Traceback (most recent call last):
    File "/mnt/data1/dev/projects/NervProj/build/libraries/QT6-6.4.2/qtwebengine/src/core/Release/x86_64/../../../3rdparty/chromium/third_party/node/node.py", line 58, in <module>
      RunNode(sys.argv[1:])
    File "/mnt/data1/dev/projects/NervProj/build/libraries/QT6-6.4.2/qtwebengine/src/core/Release/x86_64/../../../3rdparty/chromium/third_party/node/node.py", line 53, in RunNode
      raise RuntimeError('Command \'%s\' failed\n%s' % (' '.join(cmd), err))
  RuntimeError: Command '/usr/bin/nodejs ../../../3rdparty/chromium/third_party/devtools-frontend/src/scripts/build/generate_html_entrypoint.js --template ../../../3rdparty/chromium/third_party/devtools-frontend/src/front_end/entrypoint_template.html --out-directory gen/third_party/devtools-frontend/src/front_end --entrypoints devtools_app --entrypoints inspector --entrypoints js_app --entrypoints ndb_app --entrypoints node_app --entrypoints worker_app --entrypoints device_mode_emulation_frame' failed
  /mnt/data1/dev/projects/NervProj/build/libraries/QT6-6.4.2/qtwebengine/src/3rdparty/chromium/third_party/devtools-frontend/src/node_modules/yargs/node_modules/yargs-parser/build/index.cjs:1013
          throw Error(`yargs parser supports a minimum Node.js version of ${minNodeVersion}. Read our version support policy: https://github.com/yargs/yargs-parser#supported-nodejs-versions`);
          ^

• ⇒ So let's try to patch that python file. OK: I simply enforce the correct path for node with this change:

          # patch the node.py file to use our nodejs binary:
          tgt_file = f"{build_dir}/qtwebengine/src/3rdparty/chromium/third_party/node/node.py"
          self.patch_file(tgt_file, "nodejs = which('nodejs')", f"nodejs = '{nodejs_dir}/node'")

• With the previous change it seems I can build the QtWebEngine module properly, but then I get an error while build the QtPdf module:

  CMake Error at cmake/Gn.cmake:72 (message):
    
  
    -- GN FAILED
  
    ERROR at //printing/BUILD.gn:401:14: Script returned non-zero exit code.
  
          libs = exec_script("cups_config_helper.py",
                 ^----------
  
    Current dir:
    /mnt/data1/dev/projects/NervProj/build/libraries/QT6-6.4.2/qtwebengine/src/pdf/Release/x86_64/
  
  
    Command: /mnt/data1/dev/projects/NervProj/.pyenvs/qt6_env/bin/python3
    /mnt/data1/dev/projects/NervProj/build/libraries/QT6-6.4.2/qtwebengine/src/3rdparty/chromium/printing/cups_config_helper.py
    --libs-for-gn
  
    Returned 1.
  
    stderr:
  
    
  
    Traceback (most recent call last):
  
      File "/mnt/data1/dev/projects/NervProj/build/libraries/QT6-6.4.2/qtwebengine/src/3rdparty/chromium/printing/cups_config_helper.py", line 108, in <module>
        sys.exit(main())
      File "/mnt/data1/dev/projects/NervProj/build/libraries/QT6-6.4.2/qtwebengine/src/3rdparty/chromium/printing/cups_config_helper.py", line 92, in main
        flags = run_cups_config(cups_config, mode)
      File "/mnt/data1/dev/projects/NervProj/build/libraries/QT6-6.4.2/qtwebengine/src/3rdparty/chromium/printing/cups_config_helper.py", line 35, in run_cups_config
        cups = subprocess.Popen([cups_config, '--cflags', '--ldflags', '--libs'],
      File "/mnt/data1/dev/projects/NervProj/.pyenvs/qt6_env/lib/python3.10/subprocess.py", line 966, in __init__
        self._execute_child(args, executable, preexec_fn, close_fds,
      File "/mnt/data1/dev/projects/NervProj/.pyenvs/qt6_env/lib/python3.10/subprocess.py", line 1842, in _execute_child
        raise child_exception_type(errno_num, err_msg, err_filename)
  
    FileNotFoundError: [Errno 2] No such file or directory: 'cups-config'
  
    
  
    See //BUILD.gn:287:15: which caused the file to be included.
  
        deps += [ "//printing:printing_unittests" ]
                  ^------------------------------
  
    

• If I understand what's happening here then it seems we are trying to get the compile details from cups-config and I don't have that program installed ? ⇒ It seems I should install libcups2-dev to fix that. OK This will do the trick, but then I get another error, this time with a missing xkbcommon package:

  Traceback (most recent call last):
    Package xkbcommon was not found in the pkg-config search path.
  
    Perhaps you should add the directory containing `xkbcommon.pc'
  
    File "/mnt/data1/dev/projects/NervProj/nvp/core/build_manager.py", line 421, in <module>
    to the PKG_CONFIG_PATH environment variable
  
    No package 'xkbcommon' found
  
    Could not run pkg-config.
  
  
  
    See //ui/events/ozone/layout/BUILD.gn:11:3: whence it was called.
  
      pkg_config("xkbcommon") {
      ^------------------------
  
    See //ui/views/BUILD.gn:1387:7: which caused the file to be included.
  
          "//ui/events/ozone/layout:test_support",
          ^--------------------------------------

• ⇒ So now also installing the libxkbcommon-dev package.
• To save time for the compilation test I'm using the existing build dir for now with the command:

  nvp build libs qt6 -k -u

• Yeepeee 🥳! This is finally compiling correctly!

• Next I'm trying to build the minimal test QT app from NervLand. To get this to compile, I add to specify the libc++ as stdlib for clang:

  if(UNIX)
  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -stdlib=libc++")
  # set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -lc++ ")
  endif()

• Now the other issue with this is that I built the QT6 library with dynamic linkage to the libpng12 library, and on my other ubuntu system I rather have libpng16. So I need to sort this mismatch somehow.

• OK: I updated the builder for QT6 to build all dependencies internally:

      def build_on_linux(self, build_dir, prefix, _desc):
          """Build method for QT6 on linux"""
  
          cmake_args = '-DCMAKE_SUPPRESS_DEVELOPER_WARNINGS=1 -DCMAKE_CXX_FLAGS="-Wno-ignored-pragmas -Wno-deprecated-builtins"'
          args = [
              "-optimize-full", "-opensource", "-confirm-license",
              "-qt-doubleconversion", "-qt-pcre", "-qt-zlib", "-qt-freetype",
              "-qt-harfbuzz", "-qt-libpng", "-qt-libjpeg", "-qt-sqlite",
              "-qt-tiff", "-qt-webp", "-openssl-runtime"
          ]
          # "-qt-assimp", "-webengine-icu=qt", "-qt-webengine-ffmpeg", "-qt-webengine-opus", "-qt-webengine-webp",
          args = " ".join(args)

• With that previous change it seems I can build the QT test app properly on my other linux system (no more libpng12 dependency ?)

• Next I'm preparing to run that test app, so I'm copying the required dependency libraries: libQT6Widgets.so.6, libQT6Core.so.6, libQT6Gui.so.6, etc
• But now I'm facing a problem with the “xcb plugin” for QT, since I get this error message on start:

  kenshin@rog:~/projects/NervLand/dist$ ./test_qt_window 
  qt.qpa.plugin: Could not find the Qt platform plugin "xcb" in ""
  This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

• As indicated on this page, let's try to enable debuggin for the plugins:

  export QT_DEBUG_PLUGINS=1
  ./test_qt_window

• Arrff, first error: I should place those platform plugins in platforms/, not plugins/platforms apparently.
• Second point is, with debug outputs we get the list of available platforms:

  qt.qpa.plugin: Could not find the Qt platform plugin "xcb" in ""
  This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
  
  Available platform plugins are: eglfs, minimalegl, wayland, offscreen, linuxfb, wayland-egl, minimal, vnc.

• ⇒ So we don't seem to have xcb in there. Could we maybe try wayland insted ?
• Setting the environment variable:

  export QT_QPA_PLATFORM=wayland

• Nahh, not quite working… and I think it's better to stick to the default first, which should be xcb.
• So now rebuilding again with support for xcb:

          # Note: really need to install the following packages:
          # libnss3-dev libdbus-1-dev libcups2-dev libxkbcommon-dev libx11-xcb-dev
          # Also perl is already available on my system.
  
          cmake_args = '-DCMAKE_SUPPRESS_DEVELOPER_WARNINGS=1 -DCMAKE_CXX_FLAGS="-Wno-ignored-pragmas -Wno-deprecated-builtins"'
          args = [
              "-optimize-full", "-opensource", "-confirm-license",
              "-qt-doubleconversion", "-qt-pcre", "-qt-zlib", "-qt-freetype",
              "-qt-harfbuzz", "-qt-libpng", "-qt-libjpeg", "-qt-sqlite",
              "-qt-tiff", "-qt-webp", "-openssl-runtime", "-xcb-xlib", "-xcb"
          ]
          # "-qt-assimp", "-webengine-icu=qt", "-qt-webengine-ffmpeg", "-qt-webengine-opus", "-qt-webengine-webp",
          args = " ".join(args)

• Found this page with infos on the requirements for QT6 on linux: https://doc.qt.io/qt-6/linux-requirements.html

• OK: I could then successfully build the xcb plugin for QT6 on linux.

• Another kind of annoying issue I'm facing with th QT app test now is on the deployment of the binary files on linux, which feels like too much pain to me 😅.
• So I'm thinking: let's automate this in our configuration file…
• Here is the new sections I added in the config to control this deployment:

      windows_deps:
        QT6:
          bin/liQt6Core.dll: dist/libQt6Core.dll
          bin/liQt6Gui.dll: dist/libQt6Gui.dll
          bin/liQt6Widgets.dll: dist/libQt6Widgets.dll
          plugins/platforms/qwindows.dll: dist/plugins/platforms/qwindows.dll
        LuaJIT:
          bin/lua51.dll: dist/lua51.dll
        SDL2:
          bin/SDL2.dll: dist/SDL2.dll
  
      linux_deps:
        QT6:
          lib/liQt6Core.so.6.4.2: dist/libQt6Core.so.6
          lib/liQt6DBus.so.6.4.2: dist/libQt6DBus.so.6
          lib/liQt6Gui.so.6.4.2: dist/libQt6Gui.so.6
          lib/liQt6Widgets.so.6.4.2: dist/libQt6Widgets.so.6
          plugins/platforms/libqxcb.so: dist/platforms/libqxcb.so
        LuaJIT:
          lib/libluajit-5.1.so.2.1.0: dist/libluajit-5.1.so.2
        LLVM:
          lib/libc++.so.1.0: dist/libc++.so.1
          lib/libc++abi.so.1.0: dist/libc++abi.so.1

• And now we need to implement the logic to perform this deployment step (in the cmake_manager I guess)… And here is the function I created for that:

      def install_dep_modules(self, proj_name, install_dir):
          """Install all the dependencies for a given project"""
          desc = self.cmake_projects[proj_name]
  
          if install_dir is None:
              install_dir = desc["install_dir"]
  
          bman = self.get_component("builder")
          tool = self.get_component("tools")
  
          key = f"{self.platform}_dep_modules"
          mods = desc.get(key, {})
  
          for lib_name, file_map in mods.items():
              # logger.info("Should install modules for %s: %s", lib_name, file_map)
              # get the root path of that dependency:
              if bman.has_library(lib_name):
                  root_dir = bman.get_library_root_dir(lib_name)
              else:
                  root_dir = tool.get_tool_root_dir(lib_name)
  
              # Iterate on each file to check if it's already installed or not:
              for src_file, dst_file in file_map.items():
                  src_path = self.get_path(root_dir, src_file)
                  dst_path = self.get_path(install_dir, dst_file)
                  copy_needed = False
  
                  if self.file_exists(dst_path):
                      # Check if the hash will match:
                      hash1 = self.compute_file_hash(src_path)
                      hash2 = self.compute_file_hash(dst_path)
                      if hash1 != hash2:
                          logger.info("Updating dep module %s...", dst_file)
                          self.remove_file(dst_file)
                          copy_needed = True
                  else:
                      # The destination file doesn't exist yet, we simply install it:
                      logger.info("Installing dep module %s...", dst_file)
                      copy_needed = True
  
                  if copy_needed:
                      # Check that the source file exists:
                      self.check(self.file_exists(src_path), "Invalid source file: %s", src_path)
                      folder = self.get_parent_folder(dst_path)
                      self.make_folder(folder)
                      self.copy_file(src_path, dst_path)

• ⇒ Calling this at the end of the build process for a given cmake project works like a charm (on windows at least, now time to check on linux)

• OK, so now let's continue with the QT bindings in lua! So far I have only started to provide the bindings for the QObject class, but the idea now is to extend this to be able to run our current test_qt_window app written in C++ directly from lua:

  #include <QtWidgets/QApplication>
  #include <QtWidgets/QWidget>
  
  auto main(int argc, char* argv[]) -> int {
      QApplication app(argc, argv);
      QWidget window;
      window.resize(800, 600);
      window.show();
      return QApplication::exec();
  }

• So we will need the QApplication, which inherits from QGuiApplication, which itself inherit from QCoreApplication, which inherits from QObject 😆
• ⇒ Let's thus try to generate the bindings for QCoreApplication too. [crunching… crunching…] ANd in the end, this was relatively easy, just updating a bit tne Nervbind.lua file:

  opts.ignoredHeaders = {
      "_impl%.h$"
  }
  
  opts.ignoredClasses = {
      "QPrivate",
      "QtPrivate",
  }
  
  opts.allowedClasses = {
      "QObject",
      "QCoreApplication",
      "QApplication",
  }

• And then including the header of interest in bind_context.h:

  #include <QtCore/qcoreapplication.h>

• And introducing some minor fixes to ignore all the “_impl.h” files, like here:

              relfile = relfile:gsub("\\", "/")
              local ignored = false
              for _, p in ipairs(ignoredHeaders) do
                  if relfile:find(p) then
                      ignored = true
                  end
              end
  
              if ignored then
                  logDEBUG("Ignoring header ", relfile)
              else
                  self:addHeader(relfile)
              end

• Next we thus continue with the QGuiApplication class, again, nothing too tricky here, just a simple fix because we have an “EnumDecl” in the “QVariant::Type” enum (don't ask me why exactly lol):

      -- We retrieve all the values inside that enum:
      cursor:visitChildren(function(cur, _)
          local ckind = cur:getKind()
  
          if ckind ~= clang.CursorKind.EnumConstantDecl then
              logWARN("Unexpected cursor type in enum: ", cursor:getKindSpelling(), ", name: ", cur:getSpelling(), ", in: ",
                  enum:getFullName())
              return clang.VisitResult.Continue
          end
  
          local cname = cur:getSpelling()
          enum:addConstant(cname)
          return clang.VisitResult.Continue
      end)

• Also, I faced a strange issue with a binding function trying to push an enum class as integer:

  static auto _bind_highDpiScaleFactorRoundingPolicy_sig1(lua_State* L) -> int {
  
  	Qt::HighDpiScaleFactorRoundingPolicy res = (Qt::HighDpiScaleFactorRoundingPolicy)QGuiApplication::highDpiScaleFactorRoundingPolicy();
  	lua_pushinteger(L,res);
  
  	return 1;
  }

• ⇒ First I tried to fix that with a custom additional helper function in the compile context:

  inline void lua_pushinteger(lua_State* L,
                              Qt::HighDpiScaleFactorRoundingPolicy val) {
      lua_pushinteger(L, (int)val);
  }

• But then I decided to introduce a dedicated EnumConverter instead, explicitly casting enums to int (and this works fine.)

• Oh, and, we need to link to the QT6Gui library for this additional binding of course.

• And finally the QApplication itself: A piece of cake! Just need to add the Qt6Widgets library 😁

• Next we have the QWidget class which inherits from QObject and QPaintDevice: OK! this went fine, just igoring the QWidget::RenderFlags class, where there is still some kind of templating issue (to be investigated one day.. maybe lol)

• No let's try to create that famous QApplication now 😊
• I start with adding a very minimal lua app class:

  ---@class app.QTApp: app.AppBase
  local Class = createClass { name = "QTApp", bases = "app.AppBase" }
  
  local bm = import "base.BindingsManager"
  
  function Class:__init(args)
      Class.super.__init(self, args)
  
      logDEBUG("Loading QT bindings...")
      bm:loadBindings("QT", "luaQT")
  end
  
  function Class:run()
      self:init()
      self:runLoop()
      self:uninit()
      logDEBUG("Done running app.")
  end
  
  function Class:init(resizable)
      logDEBUG("Initializing QT app")
  end
  
  function Class:uninit()
      logDEBUG("Uninitializing QT app")
  end
  
  -- Implementation of a default run loop in lua:
  function Class:runLoop()
      logDEBUG("Running QT app")
  end
  
  return Class
  

• And now trying to run this:

  nvp nvl qtapp

• BUt this produced an error:

  2023-03-03 22:50:20.615805 [DEBUG] lib file: modules/luaQT.dll
  2023-03-03 22:50:20.621912 [DEBUG] Registering bindings for QT
  2023-03-03 22:50:20.621937 [DEBUG] Opened DynamicLibrary modules/luaQT.dll
  2023-03-03 22:50:20.622004 [DEBUG] Luna: Creating module Qt
  2023-03-03 22:50:20.622220 [DEBUG] Luna: Creating module QtGlobalStatic
  2023-03-03 22:50:20.622230 [DEBUG] Luna: Creating module QtMetaContainerPrivate
  2023-03-03 22:50:20.622236 [DEBUG] Luna: Creating module QtSharedPointer
  2023-03-03 22:50:20.622506 [ERROR] Luna error: error in script:
  
  _luna.copyAllParents(Qt);
  
  stack :
  2023-03-03 22:50:20.622520 [DEBUG] Luna: Luna stacktrace:stack trace: top 2
    1: [string] = "QT"
    2: [string] = "[string "..."]:30: bad argument #1 to 'pairs' (table expected, got nil)"

• ⇒ Investigating this. OK ⇒ This was due to incorrect usage of the default namespace for classes with no parent namespace. Now fixed in getLuaName:

  function Class:getLuaName()
      local fname = self:getFullName(".")
  
      if not fname:find("%.") then
          -- Add the default namespace:
          local luna = import "bind.LunaManager"
          local defSpace = luna:getDefaultNamespaceName()
          fname = defSpace .. "." .. fname
      end
  
      -- template class instances may still contains colons:
      return self:sanitizeName(fname)
  end

• OK, error now fixed and I also added a first simple extension to be able to create an app with command line arguments:

  namespace Qt {
  
  auto _lunactr_QApplication(const nv::StringList& args) -> QApplication*;
  
  } // namespace Qt

  /

• Now implementing that function:

  auto _lunactr_QApplication(const nv::StringList& args) -> QApplication* {
  
      nv::Vector<char*> strs;
      for (const auto& str : args) {
          strs.push_back((char*)str.c_str());
      }
  
      // Next we create the QApplication:
      int nargs = (int)args.size();
      // logDEBUG("Creating app with {} args.", nargs);
      auto* app = new QApplication(nargs, strs.data());
  
      return app;
  }

• ⇒ This works just fine (but will not display anything yet of course).

• To get access to the QWidget constructor I need the Qt::WindowFlags which is an instantiation of the QFlags<T> template, so I added “Flags$” as a valid class pattern:

  opts.allowedClasses = {
      "nv::StringList",
      "QObject",
      "QCoreApplication",
      "QGuiApplication",
      "QApplication",
      "QPaintDevice",
      "QWidget",
      "Flags$",
  }

• But now I have an issue because when implementing the template I actually bind some deleted functions, like:

  // Bind for operator+ (1) with signature: void (Enum) const noexcept
  static auto _bind___add_sig1(lua_State* L) -> int {
  	QString::SectionFlags* self = Luna< QString::SectionFlags >::get(L,1);
  	LUNA_ASSERT(self!=nullptr);
  
  	auto other = (QString::SectionFlag)lua_tointeger(L,2);
  
  	self->operator+(other);
  
  	return 0;
  }

  /

• Let's see how we can fix that… So for now I just updated the collectTemplateParameters function to handle this case specifically:

      -- Check here if we should do some kind of "smart type replacement",
      -- for instance, some times we may receive the template name "QFlags<T>", while
      -- The actual template parameter type name is rather "Enum", thus we need "QFlags<Enum>"
      -- To handle this, we simply extract the template name, and check its the same as our current
      -- context [assuming we only have 1 context],
      -- Then we iterate on each template name and replace it accordingly:
      if #self.templateContexts == 1 then
          local ctx = self.templateContexts[1]
          local tparams = ctx:getTemplateParameters()
  
          if #tparams == 1 then
              -- base template name:
              local idx1 = tname:find("<")
              if idx1 ~= nil then
                  local prefix = tname:sub(1, idx1 - 1)
                  local idx2 = tname:find(">[^>]*$")
                  local tplArg = tname:sub(idx1 + 1, idx2 - 1)
                  local suffix = tname:sub(idx2 + 1)
  
                  -- Check if tplArg is a valid typename (ie. we might see something like "QFlags<int>" for instance)
                  local tpname = tparams[1]:getName()
                  if tplArg ~= tpname and not self:hasType(tplArg) then
                      local tname2 = prefix .. "<" .. tpname .. ">" .. suffix
                      logINFO("=> Auto replacing generic template parameter name: ", tplArg, " => ", tpname)
                      logINFO("=> Type renamed: ", tname, " => ", tname2)
                      tname = tname2
                  end
              end
          end
      end

• … A bit dirty, but we'll see later how to improve on this. Unfortunately with this change, I get more functions properly bound, but I still get the invalid constructors like “QFlags<T>” so there is something else going wrong here, let's see… 🤔 OK ⇒ I simply fixed that by just checking for the template name in the function name when doing the instantiation:

  function Class:instantiateClassFunction(cl, tpfunc)
      if tpfunc == nil then
          return
      end
  
      local funcName = tpfunc:getName()
      logDEBUG("Processing template function: ", funcName)
  
      local tplName = self:getName()
  
      -- Replace the class name if this is a constructor:
      if funcName:startsWith(tplName .. "<") then
          logDEBUG("Detected template constructor: ", funcName)
          funcName = cl:getName()
      end

• And now the build it OK! Great 👍!

• Incredible! Now my simple application works in lua:

  function Class:run()
      logDEBUG("Loading QT bindings...")
      bm:loadBindings("QT", "luaQT")
      logDEBUG("Done loading QT bindings...")
  
      self:init()
      self:runLoop()
      self:uninit()
      logDEBUG("Done running app.")
  end
  
  function Class:init(resizable)
      logDEBUG("Initializing QT app")
      self.app = Qt.QApplication({ "my_app" })
  
      self.win = Qt.QWidget()
      self.win:resize(800, 600)
      self.win:show()
  end
  
  function Class:uninit()
      logDEBUG("Uninitializing QT app")
      self.win = nil
      self.app = nil
  end
  
  -- Implementation of a default run loop in lua:
  function Class:runLoop()
      logDEBUG("Running QT app")
      Qt.QApplication.exec()
  end

• ⇒ I get an empty window displayed, but that's all what I requested for! That's really amazing 😆!

• Let's now start building a main window instead of a simple widget. OK, but then I would like to call setWindowTitle() so I need support for QString: adding that.
• ⇒ Added the QStringConverter class
• Then using that converter for the QString class:

      -- We manually register the types that should be available directly:
      typeManager:addLuaConverter("^std::string$", import("reflection.lua.StringConverter"))
      typeManager:addLuaConverter("^nv::String$", import("reflection.lua.StringConverter"))
      typeManager:addLuaConverter("^CXString$", import("reflection.lua.CXStringConverter"))
      typeManager:addLuaConverter("^QString$", import("reflection.lua.QStringConverter"))
      typeManager:addLuaConverter("^luna::LuaFunction", import("reflection.lua.LuaFunctionConverter"))
      typeManager:addLuaConverter("^nv::RefPtr<.+>$", import("reflection.lua.RefPtrConverter"))
      typeManager:addLuaConverter(".", import("reflection.lua.VectorConverter"))

• Note: I got some strange issue with default values not being captured in some cases, so just displaying a warning, and not using any default when this happens now (to be investigated later?):

  function Class:getNumDefaultArguments()
      local count = 0;
      for k, arg in ipairs(self._arguments) do
          -- If we already have defaults, the remaining args should also have default:
          local hasDef = arg:hasDefault()
          if (count ~= 0 and not hasDef) then
              logWARN("No default value provided for argument ", k, " in sig ", self:getName())
              logWARN("=> Function full name: ", self:getParent():getFullName())
              return 0
          end
          if hasDef then
              count = count + 1
          end
      end
  
      return count
  end

• OK, now I can set the application main window title:

  function Class:init(resizable)
      logDEBUG("Initializing QT app")
      self.app = Qt.QApplication({ "my_app" })
  
      -- self.win = Qt.QWidget()
      self.win = Qt.QMainWindow()
      self.win:setWindowTitle("My test app")
      self.win:resize(800, 600)
      self.win:show()
  end

• So we'll need support for the QIcon class: OK

• Adding a gui utils module in lua:

  local utils = {}
  
  ---@type nvl.Context
  local ctx = import "nvl.Context"
  
  --- Function used to create an icon from a png file name
  ---@param name string Icon file name
  ---@return Qt.QIcon icon The icon object
  function utils.create_icon(name)
      local fname = ctx:getPath(ctx:getAppDir(), "assets", "icons", name .. ".png")
      if not ctx:fileExists(fname) then
          logERROR("Cannot find icon file ", fname)
      end
      return Qt.QIcon(fname)
  end
  
  return utils

• Now adding a simple icon to our app:

  function Class:init(resizable)
      logDEBUG("Initializing QT app")
      self.app = Qt.QApplication({ "my_app" })
  
      -- self.win = Qt.QWidget()
      self.win = Qt.QMainWindow()
      self.win:setWindowTitle("My test app")
      self.win:setWindowIcon(gutils.createIcon("cryptoview_icon"))
      self.win:resize(800, 600)
      self.win:show()
  end

• Again, this is working just fine 👍!

• Next, let's try to add a menu bar with a file menu and an open menu item… this will be a first opportunity to try to handle an event.
• Added the bindings for QMenuBar, QAction, QKeySequence, QFont: compilation still OK
• Adding our menu items:

• Note: I already have a serious issue here due to the fact that addAction is defined in both QMenu and the parent QWidget class: only the QMenu::addAction signatures will be considered in that case! To be fixed later

• But first I'm focusing on the signals and slots system:
  • Read this page: https://doc.qt.io/qt-6/signalsandslots.html
  • Then checking the QObject::connect method defined at: https://doc.qt.io/qt-6/qobject.html#signals
  • ⇒ How could I bind this in lua ? 🤔 hhmmm…

• ⇒ Looking into the MetaObject system (https://doc.qt.io/qt-6/metaobjects.html) as I'm starting to feel this could play a key part to build signals/slots…
• Found this page on the usage of QT without the MOC: https://woboq.com/blog/verdigris-qt-without-moc.html
• So checking the repository for the Verdigris project: https://github.com/woboq/verdigris/tree/master/src

• The thing is, I would still need to get a list of the available signals/slots to be able to use them somehow…
• And now I just found this very interesting article, explaining how I could do this with libclang: https://woboq.com/blog/moc-with-clang.html

• But anyway, let's focus on the barebone requirements for now:
• Say I would like to connect an existing widget signal to a given function in lua, for instance with a QAction, that could be something like that:

  local act = Qt.QAction(gutils.createIcon("folder-open-document"), "Open file...", self.win)
  act:connect("triggered()", function() logDEBUG("In the handler") end)

• To be more precise here I think I would need to pass “2triggered(bool)” as the signal name, but let's focus on the lua function part instead for now.
• How could we make this work ?
• Under the hood we would need to establish an actual QT connection, thus calling QObject::connect(const QObject *sender, const char *signal, const QObject *receiver, const char *method, Qt::ConnectionType type = Qt::AutoConnection)
• So we will need some kind of receiver QObject where we will need to provide a slot function

• So let's say we build a new kind of Object, let's call it QLuaConnector. Then to do a connection we will need an instance of that object.
• Thus we get to the question of the memory allocation/lifespan for that object:
• It makes sense to let lua control the lifespan of that object
• So we could keep that as a lua mapping between the actual QT connection generated with that newly created object, and the object itself, so that we could disconnect/release the connector dynamically if there is a need for this ? 🤔

• ⇒ Let's get to it and try to provide some implementation for this.

• So the first version I created to generated the connection to lua was like this:

  auto _lunarawext_lua_connect(QObject& obj, lua_State* L, const char* signalName,
                               const char* slotName, luna::LuaFunction& func)
      -> int {
  
      auto* dst = new QLuaConnector();
  
      auto con_src = QObject::connect(&obj, signalName, dst, slotName);
      auto* con = new QMetaObject::Connection(con_src);
      Luna<QMetaObject::Connection>::push(L, con, true);
      Luna<QObject>::push(L, dst, true);
  
      return 2;
  }

• ⇒ Actually the LunaFunction should be given as construction parameter to the QLuaConnector here, let's change that: OK, now we assign the function:

  auto _lunarawext_lua_connect(QObject& obj, lua_State* L, const char* signalName,
                               const char* slotName, luna::LuaFunction& func)
      -> int {
  
      auto* dst = new QLuaConnector(func);
  
      auto con_src = QObject::connect(&obj, signalName, dst, slotName);
      auto* con = new QMetaObject::Connection(con_src);
      Luna<QMetaObject::Connection>::push(L, con, true);
      Luna<QObject>::push(L, dst, true);
  
      return 2;
  }

• Next, this lua_connect function, is currently provided as a luna raw extension to be able to return 2 objects, the connection itself, and the QLuaConnector, because the idea it then to use those 2 elements as a (key, value) pair in a lua table to keep a reference on them. yet, this design is confusing a bit the LLS, since we get the definition:

  ---@param signalName string
  ---@param slotName string
  ---@param func function
  ---@return integer
  function Qt.QObject:lua_connect(signalName, slotName, func) end

And on top of that, if we build a customization in the QObject class, then that customization will not propagate to the derived classes on module loading, so finding the function may then take some additional time… But actually this is really something I would like to confirm to be workign as expected in fact. So let's forget about the LLS confusion above for a moment, and see if we could continue the implementation directly in lua from this point…

I thus created the following QT extension module:

logDEBUG("Loading QT extensions...")

local qt_connections = {}

-- helper function used to connect lua functions to QT signals
---@param sigName string The signal name with argument list
---@param func function The lua function to be executed as a callback
---@return QMetaObject.Connection con The resulting connection object
function Qt.QObject:connect(sigName, func)
    -- Extract the list of arguments from the signal name:
    local idx = sigName:find("%(")
    local argList = sigName:sub(idx + 1, -1)
    logDEBUG("List of argument names: '" .. argList .. "'")
    local slotName = "1handle(" .. argList .. ")"
    local con, obj = self:lua_connect("2" .. sigName, slotName, func)

    -- Store the connection/luaconnector in a tracking table:
    qt_connections[con] = obj

    -- return the connection:
    return con --[[@as QMetaObject.Connection]]
end

And then we try to use that new “connect” function from our QAction object:

    local menu = mbar:addMenu("&File") --[[@as Qt.QWidget]]
    local act = Qt.QAction(gutils.createIcon("folder-open-document"), "Open file...", self.win)
    act:connect("triggered(bool)", function() logDEBUG("Hello world!") end)

    -- act.triggered.connect(lambda: self.ctx.handle("open_fci_l1c_files", self))
    -- menu:addAction(act)
    -- act:connect()
    Qt.QWidget.addAction(menu, act)

And… that doesn't work at all 😭 as the function is considered to be missing:

2023-03-07 16:55:44.886653 [DEBUG] Done loading QT bindings...
2023-03-07 16:55:44.886656 [DEBUG] Initializing QT app
2023-03-07 16:55:44.900757 [FATAL] Error in lua app:
app.QTApp:40: attempt to call method 'connect' (a nil value)
stack traceback:

That's not so good…🥴 I'm pretty this is because this new function is not registered on the derived classes then… let's see… oohhh, also, one thing to take into consideration here is that I was not using the metatable to define the function, that may also explain the error! ⇒ checking this further. Nope, doesn't work

Okay! So a simple solution that will actually work here is instead to call again __luna.copyAllParents(Qt) at the end of the QT extension file to ensure the method is copied as needed! 👍 Not the most elegant path, but I can live with that for now. And surprisingly, I then get the slot function to be executed when I trigger the signal, cool 😎.

Now time to execute the actual lua function provided to the constructor instead, that should be simple enough:

QLuaConnector::QLuaConnector(luna::LuaFunction& func)
    : _func(func.state, func.index, true) {}

void QLuaConnector::handle(bool val) { _func(val); }

And indeed, with that change, we get out lua callback executed just as expected 🥳! All right!

Oki oki, so we make some interesting progress here: we started with the QT6 bindings generation support on linux, and when all the way to an actually working minimal example constructed in lua and we even get the signals/slots mechanism to work without the need to rely on the QT “Meta-Object Compiler” (MOC) [I guess you start to know me: I'm the one dictating to other piece of software how I want to build my project, and not the other way around 🤣].

So this is pretty nice and all, but I already have an idea on how to improve on this: you see, in the act:connect() call above I would really want to discard the need to specify the signal function signature (ie. “triggered(bool)” in this case), and I think I can do that with some limited changes to the NervBind layer We'll handle that in our next session!