NervLand: Restoring NervApp and Lua support

So in this new session, we continue with the restoring of the NervSeed project started in my previous post. Now it's time to inject some lua magic in there 😋! Or well… a little bit at least… if possible… lol.

• First things first: we need to restore the NervApp class, which is a center part of how I'm planning to build my base components.
• As demonstrated already, we add the class from command line:

  $ nvp cmake add class nervland Core app/NervApp

• ⇒ In fact I had to restore quite a lot of classes already to try to put this back on rails, like ResourceManager, LuaManager, LuaScript, LuScriptLoader, etc.
• And on top of that, I have an error when trying to use the clang compiler to link boost libraries on windows:

  lld-link: error: could not open 'libboost_filesystem-clangw14-mt-x64-1_78.lib': no such file or directory
  clang++: error: linker command failed with exit code 1 (use -v to see invocation)
  ninja: build stopped: subcommand failed.

• The library I have is actually called: libboost_filesystem-clang14-mt-x64-1_78-clang.lib, so I need to figure out how to resolve those library names correctly.
• Hmmm, okay, so I think the ending “-clang” in the lib name could come from the buildid command line argument I was using:

          bjam_cmd = [bjam, "--user-config=user-config.jam",
                      "--buildid=clang", "-j", "8", "toolset=clang",
                      "--prefix="+prefix, "--without-mpi", "-sNO_BZIP2=1",
                      "architecture=x86", "variant=release", "link=static", "threading=multi",
                      "address-model=64"]

• And for the “clangw” instead of “clang”: this is soemthing that need to be fixed on the cmake config side: the easiest option I found was to set the definition add_definitions(-DBOOST_LIB_TOOLSET=“clang14”) in our cmake files. But now I'm thinking I should rather rename the library files to have that clangw14 name directly.

• Next issue on our path, the runtime linkage is not correct 😢:

  lld-link: error: /failifmismatch: mismatch detected for 'RuntimeLibrary':
  >>> msvcprt.lib(locale0_implib.obj) has value MD_DynamicRelease
  >>> libboost_filesystem-clang14-mt-x64-1_79.lib(operations.obj) has value MT_StaticRelease
  clang++: error: linker command failed with exit code 1 (use -v to see invocation)
  ninja: build stopped: subcommand failed.

• I think I should try to pass the correct flags to the linker to fix that ?

• [A few hours later…] Feww.. that was tricky! During the investigation on this Runtime mismatch issue I first discovered that we can specify the MSVC runtime we want in Cmake with this kind of entry:

  set(CMAKE_MSVC_RUNTIME_LIBRARY "MultiThreaded$<$<CONFIG:Debug>:Debug>DLL")

• But then I realized I was already using “MultiThreaded DLL” by default when building with Clang on windows. but still the linker was reporting that the boost libraries were using MT_StaticRelease even if I was using runtime-link=shared on the command line.

• ⇒ In the end I ended up rebuilding boost again, but this time manually specifying the compilation flags I was expecting to be set automatically (-D_MT -D_DLL etc in the snippet below):

          with open(self.get_path(build_dir, "user-config.jam"), "w", encoding="utf-8") as file:
              # Note: Should not add the -std=c++11 flag below as this will lead to an error with C files:
              file.write(f"using clang : {ver_major}.{ver_minor} : {comp_path} : ")
              if self.is_windows:
                  file.write("cxxstd=17 ")
                  file.write(f"<ranlib>\"{comp_dir}/llvm-ranlib.exe\" ")
                  file.write(f"<archiver>\"{comp_dir}/llvm-ar.exe\" ")
                  file.write("<cxxflags>\"-D_CRT_SECURE_NO_WARNINGS -D_MT -D_DLL -Xclang --dependent-lib=msvcrt\" ")
                  # file.write(f"<cxxflags>-D_SILENCE_CXX17_OLD_ALLOCATOR_MEMBERS_DEPRECATION_WARNING ")
                  file.write(";\n")
              else:
                  file.write(f"<compileflags>\"{cxxflags} -fPIC\" ")
                  file.write(f"<linkflags>\"{linkflags}\" ;\n")

• And now this is finally building and running with clang on windows! Ouuufff!! 😳

• Also, to find the boost libraries with the clang14 toolset name instead of clangw14 I added the following workaround in my cmake files:

  add_definitions(-DBOOST_ALL_STATIC_LINK)
  add_definitions(-D_CRT_SECURE_NO_WARNINGS)
  
  if(WIN32 AND CMAKE_CXX_COMPILER_ID MATCHES "Clang")
      # MESSAGE(STATUS "Clang version: ${CMAKE_CXX_COMPILER_VERSION}")
      string(REPLACE "." ";" VERSION_LIST ${CMAKE_CXX_COMPILER_VERSION})
      list(GET VERSION_LIST 0 CLANG_MAJOR_VERSION)
      # list(GET VERSION_LIST 1 CLANG_MINOR_VERSION)
      # list(GET VERSION_LIST 2 CLANG_PATCH_VERSION)
  
      add_definitions(-DBOOST_LIB_TOOLSET="clang${CLANG_MAJOR_VERSION}")
  endif()

• Of course, compiling with MSVC is broken now lol, how could it be otherwise ? The issue comes from an intrinsic I'm using in the SpinLock class:

  D:\Projects\NervLand\modules\nvCore\src\base/SpinLock.h(41): error C3861: '__builtin_ia32_pause': identifier not found

  /

• It seems this builtin_ia32_pause is a gcc specific builtin: https://gcc.gnu.org/onlinedocs/gcc-4.9.4/gcc/X86-Built-in-Functions.html
• ⇒ Actually reading a bit more carefully the page on spinlock implementation (https://rigtorp.se/spinlock/) we already have the answer for the MSVC compiler:

  On GCC and clang we can emit the PAUSE instruction using the built-in function \_\_builtin_ia32_pause() and on MSVC using _mm_pause(). Adding this to our spin-wait loop we get:

• ⇒ OK, both MSVC and clang compilation are now working and our current outputs look like this:

  $ nvp nvl
  Hello world!
  [2022-07-23 22:41:48.102] [info] Creating LogManager here.
  [2022-07-23 22:41:48.103] [info] Setting log level to trace...
  [2022-07-23 22:41:48.103] [info] Creating LogManager: 42
  [2022-07-23 22:41:48.103] [info] Creating NervApp...
  [2022-07-23 22:41:48.103] [info] Creating NervApp object.
  [2022-07-23 22:41:48.103] [trace] Creating MemoryManager object.
  [2022-07-23 22:41:48.103] [info] TODO: should install debug handlers here.
  [2022-07-23 22:41:48.103] [debug] Creating new pool %d of %d blocks with blocksize=%d
  [2022-07-23 22:41:48.103] [info] Creating EventHandler object.
  [2022-07-23 22:41:48.103] [debug] Creating AppComponent ResourceManager
  [2022-07-23 22:41:48.103] [debug] Creating new pool %d of %d blocks with blocksize=%d
  [2022-07-23 22:41:48.103] [debug] Creating ResourceManager object.
  [2022-07-23 22:41:48.103] [trace] Creating ResourceLoader object.
  [2022-07-23 22:41:48.103] [trace] Adding resource search path: 'D:/Projects/assets/lua/'
  [2022-07-23 22:41:48.103] [debug] Creating new pool %d of %d blocks with blocksize=%d
  [2022-07-23 22:41:48.103] [debug] Creating LuaManager object.
  [2022-07-23 22:41:48.103] [info] Destroying NervApp...
  [2022-07-23 22:41:48.103] [debug] Uninitializing NervApp object.
  [2022-07-23 22:41:48.103] [debug] Uninitializing LuaManager...
  [2022-07-23 22:41:48.103] [debug] Releasing components...
  [2022-07-23 22:41:48.103] [trace] Uninitializing component: ResourceManager
  [2022-07-23 22:41:48.103] [debug] Deleting ResourceManager object.
  [2022-07-23 22:41:48.103] [trace] Deleting ResourceLoader object.
  [2022-07-23 22:41:48.103] [debug] Deleting AppComponent object ResourceManager
  [2022-07-23 22:41:48.103] [info] Deleting EventHandler object.
  [2022-07-23 22:41:48.103] [debug] Unloading dynamic libraries...
  [2022-07-23 22:41:48.103] [debug] Destroying LuaManager...
  [2022-07-23 22:41:48.103] [debug] Deleting LuaManager object.
  [2022-07-23 22:41:48.103] [debug] Deleting NervApp object.
  [2022-07-23 22:41:48.103] [info] Destroying memory manager.
  [2022-07-23 22:41:48.103] [trace] Deleting MemoryManager object.
  Deleted LogManager object.
  Looking for memory leaks...
  No memory leak detected.
  Exiting.

• Fixing the new memory pool creation log message: OK
• Using the lua manual to call lua directly without sol2: https://www.lua.org/manual/5.1/manual.html

• To extend the support for lua scripts we need to load the Core module bindings first: let's see how we can write these…
  • Adding the support for DynamicLibrary: OK
• Now restoring support for Luna: that one is a large chunk. But this time I don't want to start with SOL2 embedded in the application at all, so more work to do to refactor the initial bindings.
• Okay hmmm… taking a step back here: generating bindings manually for lua is possible sure, but that would mean a lot of additional work for me, so… that's not really the path I should take I think: I should rather accept to use SOL2 to generate the bindings, just not use it intrusively in the Lua management system.
• Also, the nvCore library will contain a lot of bindable content eventually, end those bindings should be generated with the NervLuna system, so the bindings I'm going to generate for nvCore here should be kept minimal.

• While building my initial bindings with SOL for the LuaManager functions I the realized that I could not convert const char* automatically to nv::String with SOL, that's a pain, but it also got me thinking, and now I have updated the LuaManager to store the internal maps with StringID keys instead of String:

      // mapping of Lua script files: //
      using ScriptMap = Map<StringID, RefPtr<LuaScript>>;

• There is still the risk of hash collision here but I think this is acceptable and should also improve the performances if I keep using this design.

• Another thing I just noticed while loading my lua script from disk with the LuaManager is that I get this kind of pattern:

  [2022-07-29 08:16:23.198] [debug] Loading resource from file: D:/Projects/NervLand/dist/assets/lua/setup.lua
  [2022-07-29 08:16:23.198] [debug] Using default root allocator to allocate/free 5713 bytes.
  [2022-07-29 08:16:23.198] [debug] Using default root allocator to allocate/free 5713 bytes.
  [2022-07-29 08:16:23.198] [debug] Using default root allocator to allocate/free 5713 bytes.
  
  [2022-07-29 08:16:23.198] [debug] Loading resource from file: D:/Projects/NervLand/dist/assets/lua/base/utils.lua
  [2022-07-29 08:16:23.198] [debug] Using default root allocator to allocate/free 5793 bytes.
  [2022-07-29 08:16:23.198] [debug] Using default root allocator to allocate/free 5793 bytes.
  [2022-07-29 08:16:23.198] [debug] Using default root allocator to allocate/free 5793 bytes.
  
  [2022-07-29 08:16:23.199] [debug] Loading resource from file: D:/Projects/NervLand/dist/assets/lua/external/serpent.lua
  [2022-07-29 08:16:23.199] [debug] Using default root allocator to allocate/free 8689 bytes.
  [2022-07-29 08:16:23.199] [debug] Using default root allocator to allocate/free 8689 bytes.
  [2022-07-29 08:16:23.199] [debug] Using default root allocator to allocate/free 8689 bytes.

• ⇒ To me this seems to be an indication that I'm allocating a String once, and then copying it… Maybe an std::move could be used here instead 🤔 ? Checking.
• Ahh! I knew this was a bit serious: I generic template with forward the rvalues correctly with this:

  template <typename T, class... Args>
  auto create_ref_object(Args&&... args) -> RefPtr<T> {
      // cf.
      // https://stackoverflow.com/questions/2821223/how-would-one-call-stdforward-on-all-arguments-in-a-variadic-function
      return MemoryManager::get_root_allocator().create<T>(
          std::forward<Args>(args)...);
  }

• But then my mistake was that I was not using the proper forwarding in the create function:

      template <class T, class... Args> auto create(Args&&... args) -> RefPtr<T> {
          // Allocate the memory
          void* ptr = allocate(sizeof(T));
          CHECK(ptr != nullptr, "Cannot allocate new object.");
  
          // Call the class constructor:
          RefPtr<T> obj(new (ptr) T(args...));
  
          // Assign this allocator to the object:
          obj->set_allocator(this);
  
          // Return the object we just created:
          return obj;
      }

• And in fact I also had to add the std::move() call in the LuaScriptLoader itself:

  auto LuaScriptLoader::load_resource(const char* fullpath) -> RefPtr<RefObject> {
      CHECK(file_exists(fullpath), "The file {} doesn't exist.", fullpath);
  
      // We read the content of the file:
      String content = read_file(fullpath);
  
      logDEBUG("LuaScript source data ptr: {}", (const void*)content.data());
  
      // Create a new LuaScript from that content:
      auto res = create_ref_object<LuaScript>(std::move(content));
      return res;
  }

• OK: issue fixed. And in the process, another thing I noticed is that I may of may not end up using the default root allocator depending on the size of the script to be abllocated Which means that small scripts are placed in the pool allocator instead: that's nice!

• Next critical step: rebuilding the lua Clang module, but with Clang 14 now instead of version 12 (I think ?): this might be tricky lol.

• To get the updated compilation details we use the llvm-config tool:

  llvm-config --cxxflags
  -ID:\Projects\NervProj\libraries\windows_msvc\LLVM-14.0.6\include -std:c++14   /EHsc /GR -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_NONSTDC_NO_WARNINGS -D_SCL_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_WARNINGS -DUNICODE -D_UNICODE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS

• llvm-config --ldflags
  -LIBPATH:D:\Projects\NervProj\libraries\windows_msvc\LLVM-14.0.6\lib

• llvm-config --system-libs
  psapi.lib shell32.lib ole32.lib uuid.lib advapi32.lib zlibstatic.lib

• llvm-config --libs
  LLVMWindowsManifest.lib LLVMXRay.lib LLVMLibDriver.lib LLVMDlltoolDriver.lib LLVMCoverage.lib LLVMLineEditor.lib LLVMX86TargetMCA.lib LLVMX86Disassembler.lib LLVMX86AsmParser.lib LLVMX86CodeGen.lib LLVMX86Desc.lib LLVMX86Info.lib LLVMOrcJIT.lib LLVMMCJIT.lib LLVMJITLink.lib LLVMInterpreter.lib LLVMExecutionEngine.lib LLVMRuntimeDyld.lib LLVMOrcTargetProcess.lib LLVMOrcShared.lib LLVMDWP.lib LLVMSymbolize.lib LLVMDebugInfoPDB.lib LLVMDebugInfoGSYM.lib LLVMOption.lib LLVMObjectYAML.lib LLVMMCA.lib LLVMMCDisassembler.lib LLVMLTO.lib LLVMCFGuard.lib LLVMFrontendOpenACC.lib LLVMExtensions.lib Polly.lib PollyISL.lib LLVMPasses.lib LLVMObjCARCOpts.lib LLVMCoroutines.lib LLVMipo.lib LLVMInstrumentation.lib LLVMVectorize.lib LLVMLinker.lib LLVMFrontendOpenMP.lib LLVMDWARFLinker.lib LLVMGlobalISel.lib LLVMMIRParser.lib LLVMAsmPrinter.lib LLVMDebugInfoMSF.lib LLVMSelectionDAG.lib LLVMCodeGen.lib LLVMIRReader.lib LLVMAsmParser.lib LLVMInterfaceStub.lib LLVMFileCheck.lib LLVMFuzzMutate.lib LLVMTarget.lib LLVMScalarOpts.lib LLVMInstCombine.lib LLVMAggressiveInstCombine.lib LLVMTransformUtils.lib LLVMBitWriter.lib LLVMAnalysis.lib LLVMProfileData.lib LLVMDebugInfoDWARF.lib LLVMObject.lib LLVMTextAPI.lib LLVMMCParser.lib LLVMMC.lib LLVMDebugInfoCodeView.lib LLVMBitReader.lib LLVMCore.lib LLVMRemarks.lib LLVMBitstreamReader.lib LLVMBinaryFormat.lib LLVMTableGen.lib LLVMSupport.lib LLVMDemangle.lib

• But now i'm wondering where I took the sources for my Clang lua module from 🤔: it seems these should be the sources for the LLVM-C module, so I need to dig in the LLVM sources for an update I guess ? ⇒OK The project I'm looking for is clang/tools/libclang: So let's see what kind of updates I have in there…

• Hmmm but ideally I should also build LLVM with LLVM itself… and this requires building libiconv first… let's see. Oh oh… troubles ahead 😬 ⇒ Let's just use the same prebuilt package as for the MSVC compiler.

• Arrf, and I also need libxml2 built statically, which of course is not working: I only get the shared library version even if I explicitly disabled shared. Ah, in fact it's the static library which is built, but with clang there is no “s” suffix appended:

  if(MSVC)
  	if(BUILD_SHARED_LIBS)
  		set_target_properties(
  			LibXml2
  			PROPERTIES
  			DEBUG_POSTFIX d
  		)
  	else()
  		set_target_properties(
  			LibXml2
  			PROPERTIES
  			DEBUG_POSTFIX sd
  			MINSIZEREL_POSTFIX s
  			RELEASE_POSTFIX s
  			RELWITHDEBINFO_POSTFIX s
  		)
  	endif()
  endif()

• ⇒ So let's add that suffix manually for now.

• Next I got an error from cmake not being able to figure out the hostr triple to be used, so I'm manually specifying that for now:

          # When compiling with clang on windows we should specify the Host triple:
          if self.is_windows and self.compiler.is_clang():
              flags += ["-DLLVM_HOST_TRIPLE=x86_64-pc-win32"]

  • Note: I found this page on the LLVM host triple: https://reviews.llvm.org/D22370 ⇒ I might have to patch something in the cmake build process ? 🤔

• Finally, to avoid a lot of warnings in the LLVm construction I also used (cf. https://github.com/alanxz/rabbitmq-c/issues/291):

  self.append_cxxflag("-std=gnu99")

• Hmm, I still have the warnings, and I'm ending with this error:

  cmd.exe /C "cd . && D:\Projects\NervProj\libraries\windows_msvc\LLVM-14.0.6\bin\clang++.exe -fuse-ld=lld-link -nostartfiles -nostdlib -DLIBXML_STATIC -ID:/Projects/NervProj/libraries/windows_clang/libiconv-1.16/include -ID:/Projects/NervProj/libraries/windows_clang/libxml2-2.9.13/include/libxml2 -Werror=date-time -Werror=unguarded-availability-new -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3 -DNDEBUG -D_DLL -D_MT -Xclang --dependent-lib=msvcrt  -LD:/Projects/NervProj/libraries/windows_clang/libxml2-2.9.13/lib -llibxml2s -LD:/Projects/NervProj/libraries/windows_clang/libiconv-1.16/lib -llibiconvStatic -lWs2_32   -Wl,--gc-sections D:/Projects/NervProj/libraries/build/LLVM-14.0.6/build/tools/clang/tools/libclang/libclang.def -shared -o bin\clang.dll  -Xlinker /MANIFEST:EMBED -Xlinker /implib:lib\clang.lib -Xlinker /pdb:bin\clang.pdb -Xlinker /version:14.0 @CMakeFiles\libclang.rsp  && cd ."
  
  
  lld-link: warning: ignoring unknown argument '--gc-sections'
  
  
  lld-link: error: D:/Projects/NervProj/libraries/build/LLVM-14.0.6/build/tools/clang/tools/libclang/libclang.def: unknown file type
  
  
  clang++: error: linker command failed with exit code 1 (use -v to see invocation)

• Also removed pedantic with a patch when building with clang on windows to finally get ride of the language extensions warning.

• Eventually I reached another sneaky error:

  'cmd.exe' n'est pas reconnu en tant que commande interne

  • ⇒ Could be that, in the case of the clang compiler I'm not setting up the PATH correctly to contain C:\Windows\system32 ?
  • And indeed, this is what I have as env by default for clang:

            if self.is_clang():
                env = {}
                env["PATH"] = self.get_cxx_dir()
    
                # inc_dir = f"{self.root_dir}/include/c++/v1"
    
                env["CC"] = self.get_cc_path()
                env["CXX"] = self.get_cxx_path()
    
                # Do not use fPIC on windows:
                # fpic = " -fPIC" if self.is_linux else ""
                # env['CXXFLAGS'] = f"-I{inc_dir} {self.cxxflags}{fpic}"
                # env['CFLAGS'] = f"-I{inc_dir} -w{fpic}"
    
                env["LD_LIBRARY_PATH"] = f"{self.libs_path}"
    
                # If we are on windows, we also need the library path from the MSVC compiler:
                if self.is_windows:
                    bman = self.ctx.get_component("builder")
                    msvc_comp = bman.get_compiler("msvc")
                    msvc_env = msvc_comp.get_env()
                    # logger.info("MSVC compiler env: %s", self.pretty_print(msvc_env))
                    env = self.prepend_env_list(msvc_env["LIB"], env, "LIB")
                self.comp_env = env

  • ⇒ Fixing that with:

                drive = os.getenv("HOMEDRIVE")
                assert drive is not None, "Invalid HOMEDRIVE variable."
                orig_env["PATH"] = f"{drive}\\Windows\\System32;{drive}\\Windows"

• Now back from vacations so trying to build from sources again:

  nvp nvl_build

• But this is failing with the missing LuaJIT dependency on ragnarok:

  2022/08/25 08:24:26 [nvp.core.build_manager] INFO: Running automatic setup for LuaJIT
  Traceback (most recent call last):
    File "D:\Projects\NervProj\nvp\core\cmake_manager.py", line 647, in <module>
      comp.run()
    File "D:\Projects\NervProj\nvp\nvp_component.py", line 93, in run
      res = self.process_command(cmd)
    File "D:\Projects\NervProj\nvp\nvp_component.py", line 83, in process_command
      return self.process_cmd_path(self.ctx.get_command_path())
    File "D:\Projects\NervProj\nvp\core\cmake_manager.py", line 39, in process_cmd_path
      self.build_projects(bprints, dest_dir, rebuild=rebuild)
    File "D:\Projects\NervProj\nvp\core\cmake_manager.py", line 510, in build_projects
      self.build_project(proj_name, install_dir, rebuild)
    File "D:\Projects\NervProj\nvp\core\cmake_manager.py", line 549, in build_project
      var_val = bman.get_library_root_dir(lib_name)
    File "D:\Projects\NervProj\nvp\core\build_manager.py", line 185, in get_library_root_dir
      assert self.dir_exists(dep_dir), f"Library folder {dep_dir} doesn't exist yet."
  AssertionError: Library folder D:\Projects\NervProj\libraries\windows_clang\LuaJIT-2.1 doesn't exist yet.
  2022/08/25 08:24:26 [nvp.nvp_object] ERROR: Subprocess terminated with error code 1 (cmd=['D:\\Projects\\NervProj\\.pyenvs\\min_env\\python.exe', 'D:\\Projects\\NervProj/nvp/core/cmake_manager.py', 'build', 'nervland'])
  2022/08/25 08:24:26 [nvp.components.runner] ERROR: Error occured in script command:
  cmd=['D:\\Projects\\NervProj\\.pyenvs\\min_env\\python.exe', 'D:\\Projects\\NervProj/nvp/core/cmake_manager.py', 'build', 'nervland']
  cwd=D:\Projects\NervLand
  return code=1
  lastest outputs:
      return self.process_cmd_path(self.ctx.get_command_path())
    File "D:\Projects\NervProj\nvp\core\cmake_manager.py", line 39, in process_cmd_path
      self.build_projects(bprints, dest_dir, rebuild=rebuild)
    File "D:\Projects\NervProj\nvp\core\cmake_manager.py", line 510, in build_projects
      self.build_project(proj_name, install_dir, rebuild)
    File "D:\Projects\NervProj\nvp\core\cmake_manager.py", line 549, in build_project
      var_val = bman.get_library_root_dir(lib_name)
    File "D:\Projects\NervProj\nvp\core\build_manager.py", line 185, in get_library_root_dir
      assert self.dir_exists(dep_dir), f"Library folder {dep_dir} doesn't exist yet."
  AssertionError: Library folder D:\Projects\NervProj\libraries\windows_clang\LuaJIT-2.1 doesn't exist yet.

• Checking the dependency… ⇒ OK: now ensuring we only use lower case names in dependency list in check_libraries():

      def check_libraries(self, dep_list, rebuild=False, preview=False, append=False, keep_build=False):
          """Build all the libraries for NervProj."""
  
          # Iterate on each dependency:
          logger.debug("Checking libraries:")
          alldeps = self.config["libraries"]
  
          # Ensure we use only lower case for dependency names:
          dep_list = [dname.lower() for dname in dep_list]

• The build of the luaClang module currently produces a lot of errors such as these:

  clang++: error: no such file or directory: '/Gw'
  clang++: error: no such file or directory: '/MD'
  clang++: error: no such file or directory: '/O2'
  clang++: error: no such file or directory: '/Ob2'

• This is expected since we are using the Clang compiler here and not MSVC. Let's see how we can fix this.

• Side note: Also noticed in the process that setting notify: false for a script using the nvp utility is not working (which is logical since we are calling another subscript in the process). So, can I do something about this ? ⇒ OK: Implemented script inheritance.

• Next fixing the error with the toString(10) calls now using the construct:

          arg.getAsIntegral().toString(buf);
          return cxstring::createDup(buf.data());

• now facing an error with this:

  D:/Projects/NervLand/modules/nvCore/src\lua/sol/sol.hpp:12297:28: error: no matching function for call to object of type 'std::less<>'
                                          return stack::push(L, op(detail::deref(l), detail::deref(r)));
                                                                ^~
  D:/Projects/NervLand/modules/nvCore/src\lua/sol/sol.hpp:23333:26: note: in instantiation of function template specialization 'sol::detail::comparsion_operator_wrap<nv::RefPtr<nv::CursorSet>, std::less<>>' requested here
                                                  lua_CFunction f = &comparsion_operator_wrap<T, std::less<>>;

• ⇒ not quite sure yet what this means and how to fix it 🤔… ⇒ Commenting this for the moment.

• Next error is on:

  lld-link: error: undefined symbol: __std_init_once_link_alternate_names_and_abort
  >>> referenced by LLVMX86CodeGen.lib(X86TargetMachine.cpp.obj):(void __cdecl llvm::initializeX86ExecutionDomainFixPass(class llvm::PassRegistry &))
  >>> referenced by LLVMX86CodeGen.lib(X86TargetMachine.cpp.obj):(int `void __cdecl llvm::initializeX86ExecutionDomainFixPass(class initializeX86ExecutionDomainFixPass::PassRegistry &)'::`1'::dtor$7)
  >>> referenced by LLVMCodeGen.lib(MachineModuleInfo.cpp.obj):(public: __cdecl llvm::MachineModuleInfoWrapperPass::MachineModuleInfoWrapperPass(class llvm::LLVMTargetMachine const *))
  >>> referenced 550 more times
  clang++: error: linker command failed with exit code 1 (use -v to see invocation)
  ninja: build stopped: subcommand failed.

• ⇒ OK: turns out that building on SATURN works for the luaClang module: so it seems this could be due to an MSVC version mismatch on ragnarok (?) ⇒ trying to upgrade.
  • ⇒ Upgrading to msvc-14.33.31629 on ragnarok fixed the link issue there too.

• As I'm setting up a new development environment on my Ragnarok computer I also need to setup the visual studo code settings properly in the NervLand workspace file adding those entries:

    "settings": {
      "clangd.path": "D:/Projects/NervProj/libraries/windows_msvc/LLVM-14.0.6/bin/clangd.exe",
      "cmake.buildDirectory": "${workspaceFolder}/.cache/cmake"
    }

• Then I also installed the following VSCode extensions:
  • clangd
  • C/C++ Extension Pack
  • cmake-format
  • EditorConfig for VSCode
  • Pylance
  • Python
  • reStructured Text
  • reStructured Text Syntax Highlighting
  • Rewrap
  • Prettier - Code Formatter

• Now back to the CursorSet bindings issue in clang_bindings.cpp:

• Compiling most of the CursorSet bindings except for the Factory constructor works fine:

      SOL_BEGIN_REF_CLASS(clang, "CursorSet", CursorSet)
      SOL_CLASS_FUNC(contains);
      SOL_CLASS_FUNC(insert);
      SOL_CLASS_FUNC(dispose);
      // SOL_CALL_FACTORIES([]() { return create_ref_object<class_t>(); });
      SOL_END_CLASS();

• I also tried a similar change in nerv_bindings.cpp to bind the LuaScript class and could reproduce the same error:

      SOL_BEGIN_CLASS(space, "LuaScript", LuaScript)
      SOL_CALL_FACTORIES([]() { return create_ref_object<class_t>("hello"); });
      SOL_END_CLASS()

• ⇒ So it seems this is an error related to the current version of SOL that I'm using… Let's try to use the previous version from our NervSeed project to check: hmmmm, we still have the same error and this is making everything even worst (more warnings and errors ?)

• So let's check if there is an update for SOL… ⇒ Using latest release available for SOL (version 3.3.0) doesn't help 😞 So could this be an issue specific to the clang compiler on windows ? Could be worth it to try compilation on linux: Arrf, there is still a long way to go to get the compilation on linux to work.

• For now, investigating further on those std::less<> and std::equal_to<> classes, I think the issue might be because of the deref() function itself which should return the pointers for RefPtr containers:

  		template <typename T>
  		inline decltype(auto) deref(T&& item) {
  			using Tu = meta::unqualified_t<T>;
  			if constexpr (meta::is_pointer_like_v<Tu>) {
  				return *std::forward<T>(item);
  			}
  			else {
  				return std::forward<T>(item);
  			}
  		}

• ⇒ Let's see if we can specialize that function: Nope, doesn't seem to help/work.

• Arrff, I think I might finally have something here reading more carefully the error message:

  D:\Softs\VisualStudio\VS2022\VC\Tools\MSVC\14.33.31629\include\xstddef:218:31: note: candidate template ignored: substitution failure [with _Ty1 = nv::LuaScript &, _Ty2 = nv::LuaScript &]: invalid operands to binary expression ('nv::LuaScript' and 'nv::LuaScript')
      _NODISCARD constexpr auto operator()(_Ty1&& _Left, _Ty2&& _Right) const

  • ⇒ Might be that our dereferencing operator should return an “&&” value instead of just “&”.

• Ahh, finally found it: this was occuring because I wasn't declaring the “RefPtr” as a smart pointer container for sol, the error is fixed if I add the required specialization:

  namespace sol {
  template <typename T> struct unique_usertype_traits;
  
  template <typename T> struct unique_usertype_traits<nv::RefPtr<T>> {
      typedef T type;
      typedef nv::RefPtr<T> actual_type;
      static const bool value = true;
  
      static bool is_null(const actual_type& value) { return value == nullptr; }
  
      static type* get(const actual_type& p) { return p.get(); }
  };
  
  } // namespace sol
  

• Next logical step would be to try to display a window now, so I restored the support for the base Window and WindowManager implementation,

• Then I need to restore the SDL specific versions of those components: OK

• And register the correct Window Manager when initializing a simple lua app in the NervSeed launcher…

• But first things first, let's revisit a bit the Lua application framework I built initially in my NervSeed project because this is all a bit rusty so I need to refresh my memory

• Hmmm, okay, so, the point is, before I can start building an SDL app from lua, I actually need the bindings generated for the window/view classes :-S So back to NervBind already lol ⇒ So let's just start a new post for that one as I feel this could mean “a lot of additional fun” before I can reach the current goal 🤣