blog:2022:1004_nervluna_restoring_automatic_binding

NervLuna: Restoring automatic lua bindings generation

, , , , , , ,

Continuing here with the restoring process for my NervLand project: as mentioned at the end of my last post, I now need to retore my “NervBind” application to generate some lua bindings automagically: let's get started.

Preparing run to build the nvCore library bindings

  • From my assets/lua/run.lua script file I see that I have an application shortcut name “bind3” that should trigger the build of the bindings for nvCore.
  • ⇒ Let's see what the corresponding config file looks like (Note: I'm going to store the config directly as modules/lua_bindings/Core/nervbind.lua)
  • So I had to restore a lot of different lua classes to prepare for the bindings generation system, and this includes the bind.BindingManager class, which itself, expect the DynamicLibrary class to be bound already hmmm… Do I really need that ? ⇒ Yeahh, the only concern of the LuaManager class is really to load “bindings” once the corresponding bindings registration function is made available, but before that we need the mechanism to load dynamic libraries as a separated features anyway. So let's also provide the DynamicLibrary bindings as part of the low level bindings…
  • Update: stupid me 😅: actually this is already readily available within the NervApp bindings with the load_library(),unload_library() and unload_all_libraries(): so let's use those methods instead.
  • Arrfff, and of course I get a crash just when I load the library:
    • It's not due to my clang_bindings.cpp file (apparently)
    • My next idea is, this could be due to the linking of some of the LLVM/Clang libraries ⇒ let's try to prune those lists. ⇒ Nope, same error
    • Also tried to rebuild everything with msvc compiler ⇒ again, same error 😲!
    • Let's try to load a simple (empty) luaCore.dll instead just to be sure…
    • Oh… I actually get the same error just load my minimal luaCore.dll module 😁 so that doesn't seem to be related to the clang bindings themself (thanks god for that…)
    • So what do we have here ? 🤔
    • aarrrhh… it seems we actually do not even reach the NervApp::load_library function ⇒ So issue seems to be on the SOL bindings layer since I'm trying to pass an “nv::String” object as argument here ⇒ let's turn that into a simple const char* instead.
    • ⇒ Cool! This was it, now I can load the module correctly.
  • Current status: Now building a simple “empty” luaCore module with the command: 
    $ nvp nvl bind_core
  • No error for now in the binding generation process, now let's try to build the resulting library.
  • ⇒ Building gave me some errors with clang, but nothing critical. Only important change is the line #pragma clang diagnostic ignored “-Wundefined-var-template” in the register_package.cpp to avoid too many warnings on template forward declarations.
  • Then, the first non-trivial error I get when uncommenting “core_common.h” is as follow: 
    [2022-09-19 21:08:11.463] [debug] Binding for nv::operator*
[2022-09-19 21:08:11.465] [debug] Binding for std::operator<<
[2022-09-19 21:08:11.465] [critical] Error in lua app:
[string "bind.FunctionWriter"]:287: attempt to index local 'rtype' (a nil value)
stack traceback:
        [string "bind.FunctionWriter"]:287: in function 'writeFunctionOverload'
        [string "bind.FunctionWriter"]:190: in function 'writeFunctionBinds'
        [string "bind.FunctionWriter"]:57: in function 'writeFunctionBindings'
        [string "bind.GlobalFunctionsWriter"]:27: in function 'writeContent'
        [string "bind.GlobalFunctionsWriter"]:6: in function '__init'
        [string "setup.lua"]:160: in function <[string "setup.lua"]:152>
        [string "app.NervBind"]:243: in function 'writeBindings'
        [string "app.NervBind"]:199: in function 'run'
        [string "app.AppBase"]:23: in function '__init'
        [string "app.NervBind"]:12: in function '__init'
        [string "setup.lua"]:160: in function 'app'
        [string "run.lua"]:65: in function <[string "run.lua"]:63>
        [C]: in function 'xpcall'
  • I tried to check the validity of the return type when building a signature: 
      -- Return type cannot be nil:
  CHECK(rtype ~= nil, "Invalid return type for signature: ", signame)
  sig:setReturnType(rtype)
  • But that doesn't seem to be the issue… investigating further. ⇒ okay so it seems this error was related to the fact that I was not including the MSVS include folder as a valid include path.
  • Next issue now: 
    [2022-09-19 21:29:59.527] [debug] Type kind: Unexposed

[2022-09-19 21:29:59.527] [debug] Converting type name 'conditional<is_reference_v<_Ty>, type-parameter-0-0, const typename remove_reference<type-parameter-0-0>::type>::type' to template contextual name: conditional<is_reference_v<${}${_Ty}${}>,${} $${}{_Ty}${}, const typename remove_reference<${}${_Ty}${}>:${}:type>${}::type@std::_Atomic_storage with 2 template parameters.

[2022-09-19 21:29:59.527] [debug] Creating template type conditional<is_reference_v<${}${_Ty}${}>,${} $${}{_Ty}${}, const typename remove_reference<${}${_Ty}${}>:${}:type>${}::type@std::_Atomic_storage with 2 template parameters

[2022-09-19 21:29:59.528] [debug] TypeManager: Registering type {
  "conditional<is_reference_v<${}${_Ty}${}>,${} $${}{_Ty}${}, const typename remove_reference<${}${_Ty}${}>:${}:type>${}::type@std::_Atomic_storage"
}

[2022-09-19 21:29:59.529] [debug] Type already registered for 'conditional<is_reference_v<${}${_Ty}${}>,${} $${}{_Ty}${}, const typename remove_reference<${}${_Ty}${}>:${}:type>${}::type@std::_Atomic_storage'

[2022-09-19 21:29:59.529] [debug] Default value cursor is defined at: D:\Softs\VisualStudio2022CE\VC\Tools\MSVC\14.32.31326\include\atomic:491:60:19868

[2022-09-19 21:29:59.529] [critical] [FATAL] Invalid name for cursor of kind OverloadedDeclRef
stack traceback:
	[string "setup.lua"]:203: in function 'THROW'
	[string "setup.lua"]:198: in function 'CHECK'
	[string "bind.Clang"]:75: in function 'getFullName'
	[string "bind.ClangParser"]:1060: in function 'resolveArgument'
	[string "bind.ClangParser"]:1195: in function 'parseFunction'
	[string "bind.ClangParser"]:1431: in function <[string "bind.ClangParser"]:1355>
	[C]: in function 'visitChildren'
	[string "bind.ClangParser"]:1355: in function 'parseClassTemplate'
	[string "bind.ClangParser"]:1567: in function <[string "bind.ClangParser"]:1524>
	[C]: in function 'visitChildren'
	[string "bind.ClangParser"]:1524: in function 'parseNamespace'
	...
	[string "bind.ClangParser"]:1524: in function 'parseNamespace'
	[string "bind.ClangParser"]:1602: in function 'parseFile'
	[string "app.NervBind"]:169: in function 'parseInputs'
	[string "app.NervBind"]:189: in function 'run'
	[string "app.AppBase"]:23: in function '__init'
	[string "app.NervBind"]:12: in function '__init'
	[string "setup.lua"]:160: in function 'app'
	[string "run.lua"]:65: in function <[string "run.lua"]:63>
	[C]: in function 'xpcall'
	[string "run.lua"]:68: in main chunk (at D:/Projects/NervLand/sources/nvCore/src/lua/nerv_bindings.cpp:27)

[2022-09-19 21:29:59.531] [error] Error in cursorVisitor: C++ exception
  • Note: those outputs above are just crazy… lol I really wonder were I could go with ths mess 😆.
  • Continuing on my path I got a sudden crash when registering a class for “type_info”, and with more outputs I see that I have: 
    [2022-09-20 12:08:41.558] [debug] Registering type for class name type_info
[2022-09-20 12:08:41.559] [debug] TypeManager: Registering type {
  "type_info"
}
[2022-09-20 12:08:41.559] [debug] Done registering class type_info
[2022-09-20 12:08:41.559] [debug] Parsing type_info
[2022-09-20 12:08:41.559] [debug] Visiting children of type_info
  • So, something gong wrong when visiting the children… let's clarify that.
  • Okay, so it seems we reach a first child, but that cursor is somehow invalid, and calling isLocationParseable(cursor) on it will crash ?
  • ⇒ So there was indeed a problem with the invalid locations which I fixed with adding an initial check: 
        SOL_CUSTOM_FUNC(getSpelling) = [](class_t& obj) {
        if (clang_equalLocations(obj, clang_getNullLocation()) != 0) {
            // This is an invalid location:
            std::string filestr("");
            U32 line = 0;
            U32 column = 0;
            U32 offset = 0;
            return std::tuple(filestr, line, column, offset);
        }

        CXFile file;
        U32 line;
        U32 column;
        U32 offset;
        std::cout << "=>getSpellingLocation" << std::endl;
        clang_getSpellingLocation(obj, &file, &line, &column, &offset);
        // We try to retrieve the real file name first if possible:
        std::cout << "=>clang_File_tryGetRealPathName" << std::endl;
        CXString filename = clang_File_tryGetRealPathName(file);
        std::cout << "=>clang_getCString" << std::endl;
        std::string filestr(clang_getCString(filename));
        clang_disposeString(filename);

        if (filestr.empty()) {
            std::cout << "=>clang_getFileName" << std::endl;
            filename = clang_getFileName(file);
            std::cout << "=>clang_getCString" << std::endl;
            filestr = clang_getCString(filename);
            clang_disposeString(filename);
        }

        std::cout << "=>return tuple" << std::endl;
        return std::tuple(filestr, line, column, offset);
    };
  • but now it's time to clean a bit this mess, and figure out how to avoid navigating in the full list of available classes/functions in the whole C++ context: as this is taking far too long (I had to kill the binding generation process… !)
  • Next issue I notice is this: 
    [2022-09-20 18:15:08.698] [debug] Adding name std::locale::facet * to type {
  "std::locale::facet"
}
  • ⇒ We should not have a “base type” also sharing the base type pointer as name, right ? ⇒ OK, fixed: it seems I was mixing pointers and references in my parsing process at that location: 
        while tname:endsWith("%s*&") do
      local n = t:getName().."&"
      local newType = typeManager:getType(n) 
      if not newType then
        logDEBUG("Type inference: ", "Auto generating reference type: ", n)
        newType = Type(n)
        newType:setBaseType(t)
        newType:addReference()
      end
      t = newType
      
      tname = tname:gsub("%s*&$","")
    end
  • I also fixed the registration of the same name again and again for a given type adding and using the hasNameRaw(name) method below: 
    function Class:hasNameRaw(name)
  for _,n in ipairs(self._names) do
    if n == name then
      return true
    end
  end

  return false
end
  • Finally I got a small error in the bindings writing part when trying to process const char*& type, fixed there: 
      if self:isChar() then
    if self:isConst() and self:getPointerCount()==1 and not self:isReference() then
      -- This is a char pointer, so we expect to get a string:
      return import("reflection.lua.ConstCharPtrConverter")
    elseif self:getPointerCount()==0 then
      return import("reflection.lua.CharConverter")
    end
  end
  • ⇒ And now the bindings generation is finally working just fine! Except that it's generating bindings for a lot of std classes that I really don't want to have, so I need to ignore these.
  • I thus updated the luaCore nervbind.lua config file to only keep the classes/functions in the nv namespace, and only select a few simple classes for a start using this filtering: 
    local allowedClasses = {
    "^nv::Vec",
    "^nv::RefObject",
    "^nv::AppComponent"
}

local ignoreClasses = function(ent)
    local name = ent:getFullName()
    if name == "void" then
        -- Keep the class.
        return
    end

    -- for _,p in ipairs(ignoredEntities) do 
    --     if name:find(p) then
    --         ent:setIgnored(true)
    --         logDEBUG("=> Ignoring entity ", name)
    --         return
    --     end
    -- end

    for _,p in ipairs(allowedClasses) do 
        if name:find(p) then
            return
        end
    end
    
    -- Ignore this entity:
    ent:setIgnored(true)
    -- logDEBUG("=> Ignoring entity ", name)
    return

end

Building the nvCore library bindings

  • Now time to try to build…
  • ⇒ Fixed a small syntax issue replacing luna_dumpStack() with luna::dumpStack()
  • Then handling the invalid binding generation for operator“”: 
    D:/Projects/NervLand/sources/lua_bindings/Core/src/luna/register_functions.cpp:112:31: error: expected ';' after top level declarator
static bool _check_nv_operator""_sid_sig1(lua_State* L) {
                              ^
                              ;
1 error generated.
  • Then added the nvCore library as dependency: 
    cfg.cmakeConfig = {
    includeDirs = {"../include" },
    libDirs = {},
    libs = {"nvCore"}
}
  • And then… it's building successfully! 😲 That's almost unbelievable…
  • Next I added the headers for the SDLWindow / SDLWindowManager bindings: generation went OK, but then I have a problem with rvalue handling for the Traits parameter in the SDLWindow constructor: 
    D:/Projects/NervLand/sources/lua_bindings/Core/src/luna/bind_nv_SDLWindow.cpp:241:13: error: no matching constructor for initialization of 'nv::SDLWindow'
        return new nv::SDLWindow(*traits);
                   ^             ~~~~~~~
D:/Projects/NervLand/sources/nvCore/src\view/SDLWindow.h:13:24: note: candidate constructor not viable: no known conversion from 'nv::Window::Traits' to 'const nv::SDLWindow' for 1st argument
    NV_DECLARE_NO_COPY(SDLWindow)
  • Problem is, rvalue handling doesn't work so well from my perspective with lua objects on the stack, so for now let's just use a good old const reference parameter instead.

Loading the luaCore bindings

  • Back to our minimal SDL app in lua, which I can start with the command: 
    $ nvp nvl sdlapp
  • I'm now trying to load th bindings from the luaCore.dll file, but this is failing with this message: 
    [2022-09-21 21:03:15.123] [debug] Loading bindings for Core with libname luaCore
[2022-09-21 21:03:15.123] [debug] Loading dynamic library: luaCore.dll
[2022-09-21 21:03:15.123] [debug] NervApp: load_library...
[2022-09-21 21:03:15.123] [debug] lib file: modules/luaCore.dll
[2022-09-21 21:03:15.140] [debug] Opened DynamicLibrary modules/luaCore.dll
[2022-09-21 21:03:15.140] [critical] [FATAL] No lua bindings could be loaded for Core
stack traceback:
        [string "setup.lua"]:203: in function 'THROW'
        [string "setup.lua"]:198: in function 'CHECK'
        [string "base.BindingsManager"]:43: in function 'loadBindings'
        [string "app.SDLApp"]:11: in function 'init'
  • ⇒ In fact, this is fully expected because we are providing only a regular luaopen entry point in the dll for now, and instead we should register our binding function which could then be called on different lua states.
  • Checking the NervSeed project, it seems I had already implemented some support for this (ie. loading the bindings in a SOL state object), let's find this back and adapt. ⇒ OK, what I need is simply this in the config (after some small changes): 
    cfg.moduleName = "luaCore"
cfg.bindingName = "Core"

-- define asLuaModule to create a regular lua module:
-- cfg.moduleName = "luaCore"
-- cfg.asLuaModule = true
  • Next I'm on the “title” field from Window::Traits: 
    [2022-09-22 12:21:55.154] [critical] Error in lua app:
[string "app.SDLApp"]:22: Cannot find field 'title' in class 'nv.Window.Traits'
stack traceback:
        [C]: in function '__newindex'
        [string "app.SDLApp"]:22: in function 'init'
  • ⇒ Currently that field is not available because I'm ignoring the nv::String class: let's add support for that one.
  • Also manually added lua converter for nv::String: 
      -- We manually register the types that should be available directly:
  -- typeManager:addType("std::string", Type("std::string", "std::string"))
  typeManager:addLuaConverter("std::string", import("reflection.lua.StringConverter"))
  typeManager:addLuaConverter("nv::String", import("reflection.lua.StringConverter"))
  typeManager:addLuaConverter("luna::LuaFunction", import("reflection.lua.LuaFunctionConverter"))
  • OK, next thing I need are the bindings for the SystemTime class: Done.
  • ⇒ And cool, now I can display a empty window 👍! But there are still some issues: most important one is that I'm not using the RefPtr container for my RefObjects, this must be fixed now.

Using the RefPtr smart pointer in NervLuna

  • Actually I just found in the old nervSeed code base that this seems to be directly related to the luna_provider specialization.
  • And I added the specialization for RefObject as follow: 
    namespace luna {

template <>
struct luna_provider<nv::RefObject>
{
    typedef nv::RefPtr<nv::RefObject> container_t;

    static nv::RefObject *get(const container_t &cont)
    {
        return cont.get();
    };

    static void set(container_t &cont, nv::RefObject *ptr)
    {
        cont = ptr;
    };

    static nv::RefObject *release(container_t &cont)
    {
        return cont.release();
    };

    static void destruct(container_t &cont)
    {
        cont.reset();
    };
};

};
  • ⇒ let's add this back directly in core_lua.h…
  • Update: after some more thinking, I now think it makes more sense to place this in the Core/include/compile_context.h file: 
    #ifndef COMPILE_CONTEXT_
#define COMPILE_CONTEXT_

// Write here the content that should be provided during compilation.
#include <core_common.h>

// Add support for RefPtr container usage for RefObjects:
namespace luna {

template <> struct luna_provider<nv::RefObject> {
    using container_t = nv::RefPtr<nv::RefObject>;

    static auto get(const container_t& cont) -> nv::RefObject* {
        return cont.get();
    };

    static void set(container_t& cont, nv::RefObject* ptr) { cont = ptr; };

    static auto release(container_t& cont) -> nv::RefObject* {
        return cont.release();
    };

    static void destruct(container_t& cont) { cont.reset(); };
};

}; // namespace luna

#endif
    /
  • Yet, it seems this is not enough yet as I don't get any reference counting working 🤔… could be this was still work in progress last time I stopped ? Investigating further…
  • Okay, so this was tricky, but I eventually got it to work with the following additional template entries: 
    namespace luna {

// template <typename T, uint64_t U> class luna_provider;

template <typename T>
struct luna_prov_base<T, typename std::enable_if<
                             std::is_base_of<nv::RefObject, T>::value>::type> {
    using type = nv::RefObject;
};

template <typename T>
struct luna_prov_base<T, typename std::enable_if<
                             !std::is_base_of<nv::RefObject, T>::value>::type> {
    using type = T;
};

template <> struct luna_provider<nv::RefObject> {
    using container_t = nv::RefPtr<nv::RefObject>;

    static auto get(const container_t& cont) -> nv::RefObject* {
        return cont.get();
    };

    static void set(container_t& cont, nv::RefObject* ptr) {
        logINFO("Acquiring RefObject at {}", (const void*)ptr);
        cont = ptr;
    };

    static auto release(container_t& cont) -> nv::RefObject* {
        logINFO("Releasing RefObject at {}", (const void*)cont.get());
        return cont.release();
    };

    static void destruct(container_t& cont) {
        logINFO("Resetting RefObject at {}", (const void*)cont.get());
        cont.reset();
    };
};

}; // namespace luna
  • But I'm not really satisfied with this: because I have to define the behavior for non RefObject based classes… but what if I have multiple different smart base objects ? ⇒ Would be far better to have a default behavior embedded in the base framework for types that could not be specialized further… let's see if I can improve on this.
  • okay… so, I'm really confused with all those templates inside templates inside templates now, and I don't seem to get this to work in a generic way, but never mind: I can use the NervLuna framework to figure out what is the base class for each class I'm going to bind and then declare a dedicated specialization as this: 
    template <> struct luna_base<nv::SDLWindowManager> { using type = nv::RefObject; };
  • Then I will use that specialization to figure out what is the provider I should use for that type of class, et na! :-)
  • Here is the lua writer implementation for the new bind_bases.h file I now need: 
    function Class:writeBindProviderBases(classes)
  self:writeLine[[#ifndef BIND_BASES_
#define BIND_BASES_

#include <lua/luna.h>
namespace luna {

]]

  cfg = luna:getConfig()
  cprovs = cfg.custom_providers or {}

  local check_provider = function(cl)
    bnames = cl:getRootBaseNames()

    for _,bname in ipairs(bnames) do
      for _,cprov in ipairs(cprovs) do
        if bname == cprov then
          -- logINFO("Writting custom provider for ", cl:getFullName())
          self:writeSubLine("template <> struct luna_base<${1}> { using type = ${2}; };", cl:getFullName(), cprov)
          return
        end
      end
    end
  end

  for _,cl in ipairs(classes) do
    -- Get the root base of this class:
    check_provider(cl)
    -- logINFO("Got root base names: ", bnames)
  end
  self:newLine()
  self:writeLine("}\n")
  self:writeLine("#endif\n")

  self:writeOutputFile("include/bind_bases.h")
  self:reset()
end
  • And here is an example of a resulting bind_bases file: 
    #ifndef BIND_BASES_
#define BIND_BASES_

#include <lua/luna.h>
namespace luna {


template <> struct luna_base<nv::RefObject> { using type = nv::RefObject; };
template <> struct luna_base<nv::AppComponent> { using type = nv::RefObject; };
template <> struct luna_base<nv::EventHandler> { using type = nv::RefObject; };
template <> struct luna_base<nv::Window> { using type = nv::RefObject; };
template <> struct luna_base<nv::SDLWindow> { using type = nv::RefObject; };
template <> struct luna_base<nv::WindowManager> { using type = nv::RefObject; };
template <> struct luna_base<nv::SDLWindowManager> { using type = nv::RefObject; };

}

#endif
  • ⇒ Feeewwww: and with this, I finally get the RefObject container to be used as expected and now my minimal SDLApp will run without a final memory corruption error, all good 👍!
  • Note: now I also need to “declare” the base classes for which I want to implement a custom provider in the nervbind.lua config file: 
    -- custom providers in this module:
cfg.custom_providers = {"nv::RefObject"}
  • Oh, and did I mention I also added an initial brush version for lua in my dokuwiki syntaxhighlight4 plugin ? ⇒ that's what we have in use on this page now! 😎

Building initial vulkan skeleton app

  • Now that the mininal SDLApp is starting, it's time to move to the next step: building a minimal vulkan app!
  • ⇒ looking into app/VulkanApp.lua as starting point.
  • We will have to load a luaVulkan module here: 
    function Class:init()
    logDEBUG("Loading Vulkan bindings...")
    bm:loadBindings("Vulkan", "luaVulkan")

    -- Create the Library:
    local vk = nv.vulkan
    local lib = vk.Library.instance();
  • Starting to build the nvVulkan module now, But first thing is: we need the vulkan dependency: let's see where we can find that.
  • ⇒ Downloading the latest SDK version from https://vulkan.lunarg.com/
    • That will be version 1.3.224.1
    • Downloading for windows and linux
    • Installed Vulkan 1.3.244.1 on windows
    • Extracted the include and lib folders from there to create our devel package Vulkan-1.3.224.1 (same for msvc and clang)
    • Adding Vulkan as an available NervProj library: OK
    • And finally we add the VULKAN_DIR dependency into the NervLand project: OK
  • Next we try a minimal build: OK, nvVulkan base module ready,
  • Now, trying to prepare the initial luaVulkan module… This went fine, but finally I have a linkage issue as follow: 
    lld-link: error: undefined symbol: public: static unsigned __int64 const luna::LunaTraits<class nv::STLAllocator<char, class nv::DefaultPoolAllocator>>::id
>>> referenced by sources/lua_bindings/Vulkan/src/CMakeFiles/luaVulkan.dir/luna/register_package.cpp.obj:(public: static void __cdecl luna::Luna<class nv::STLAllocator<char, class nv::DefaultPoolAllocator>>::Register(struct lua_State *))
>>> referenced by sources/lua_bindings/Vulkan/src/CMakeFiles/luaVulkan.dir/luna/register_package.cpp.obj:(<auto> __cdecl luna::luna_toUserData<class nv::STLAllocator<char, class nv::DefaultPoolAllocator>>(struct lua_State *, int))
>>> referenced by sources/lua_bindings/Vulkan/src/CMakeFiles/luaVulkan.dir/luna/register_package.cpp.obj:(public: static void __cdecl luna::Luna<class nv::STLAllocator<char, class nv::DefaultPoolAllocator>>::push(struct lua_State *, class nv::STLAllocator<char, class nv::DefaultPoolAllocator> const *, bool))

lld-link: error: undefined symbol: public: static char const *const luna::LunaTraits<class nv::STLAllocator<char, class nv::DefaultPoolAllocator>>::fullName
>>> referenced by sources/lua_bindings/Vulkan/src/CMakeFiles/luaVulkan.dir/luna/register_package.cpp.obj:(public: static void __cdecl luna::Luna<class nv::STLAllocator<char, class nv::DefaultPoolAllocator>>::Register(struct lua_State *))
>>> referenced by sources/lua_bindings/Vulkan/src/CMakeFiles/luaVulkan.dir/luna/register_package.cpp.obj:(public: static void __cdecl luna::Luna<class nv::STLAllocator<char, class nv::DefaultPoolAllocator>>::Register(struct lua_State *))
>>> referenced by sources/lua_bindings/Vulkan/src/CMakeFiles/luaVulkan.dir/luna/register_package.cpp.obj:(public: static int __cdecl luna::Luna<class nv::STLAllocator<char, class nv::DefaultPoolAllocator>>::set_index(struct lua_State *))
>>> referenced 1 more times

lld-link: error: undefined symbol: public: static char const *const luna::LunaTraits<class nv::STLAllocator<char, class nv::DefaultPoolAllocator>>::className
>>> referenced by sources/lua_bindings/Vulkan/src/CMakeFiles/luaVulkan.dir/luna/register_package.cpp.obj:(public: static void __cdecl luna::Luna<class nv::STLAllocator<char, class nv::DefaultPoolAllocator>>::Register(struct lua_State *))
>>> referenced by sources/lua_bindings/Vulkan/src/CMakeFiles/luaVulkan.dir/luna/register_package.cpp.obj:(public: static void __cdecl luna::Luna<class nv::STLAllocator<char, class nv::DefaultPoolAllocator>>::push(struct lua_State *, class nv::STLAllocator<char, class nv::DefaultPoolAllocator> const *, bool))
>>> referenced by sources/lua_bindings/Vulkan/src/CMakeFiles/luaVulkan.dir/luna/register_package.cpp.obj:(public: static void __cdecl luna::Luna<class nv::STLAllocator<char, class nv::DefaultPoolAllocator>>::push(struct lua_State *, class nv::STLAllocator<char, class nv::DefaultPoolAllocator> const *, bool))
  • Point is, currently the binding system will re-bind some of the classes that are already provided by the luaCore module: 
    	Luna< nv::QueueInfo >::Register(L);
	Luna< nv::QueueFamilyInfo >::Register(L);
	Luna< nv::Vector<float> >::Register(L);
	Luna< nv::VulkanMemoryRange >::Register(L);
	Luna< nv::VulkanWindowParameters >::Register(L);
	Luna< nv::STLAllocator<char> >::Register(L);
	Luna< nv::String >::Register(L);
	Luna< nv::DefaultPoolAllocator >::Register(L);
  • ⇒ I should provide a mechanism to avoid that: already bound classes should be listed in some input file: let's implement that.
  • Note: nv::STLAllocator<char> was already found while building luaCore, but I ignored that class on purpose: should do the same here (it comes from the nv::String definition by the way)
  • Basically/ to get a class from the state we use Luna<class_t>::get(): that method will use the LunaTraits<class_t>::id and also access the provider in use fro class_t so we need those 2 elements available even if we are not binding class_t to be able to get() it.
  • To send a value on the stack we use Luna<class_t>::push(), which itself will use traits_t::className, traits_t::namespaces, traits_t::id, traits_t::baseIDs
  • So we need those elements separated and available globally, with specializations such as the following: 
    template<> struct LunaTraits< nv::WindowManager > 
{
  using root_t = nv::RefObject;

  constexpr const char className[] = "WindowManager";
  constexpr const char fullName[] = "nv.WindowManager";
  constexpr const char* namespaces[] = {"nv",0};
  constexpr const char* parents[] = {"nv.AppComponent",0};
  constexpr LunaID id = LUNA_SID("nv::WindowManager");
  constexpr LunaID baseIDs[] = {LUNA_SID("nv::AppComponent"),LUNA_SID("nv::EventHandler"),LUNA_SID("nv::RefObject"),0};
}
  • ⇒ So now I think I should:
    • Remove the luna_base template
    • Separate the LunaTraits in two parts (the traits above and the actual methods in a separate template)
    • OK: Now I have all the LunaTraits<T> specialization written in the common include/${module_name}_classes.h file, and I keep the methods/fields descriped in a LunaBehavior<t> class.

Setting up support to list already bound elements

  • When building the bindings for a new module such as luaVulkan we will inevitably eventually have functions/classes using already bound elements such as nv::RefObject or nv::String. In that case, the corresponding new function or class should not be ignored, but we should also not write any binding for already bound elements.
  • So each module will now write a manifest.lua file listing all the elements bound in that module: we can then collect those manifest files when writing a new module to get the full list of elements already bound that we may rely on.
  • While I'm at it, I'm also going to introduce the support to generate the bindings for multiple modules with a single command: instead of providing a config file on the command line, I could provide another lua file (“all_modules.lua”) containing a list of config files: 
    local path = import "pl.path"

local bind_dir = path.abspath(root_path.."../sources/lua_bindings")

-- Return the list of configs that should be built in order:
return {
    path.join(bind_dir,"Core/nervbind.lua"),
    path.join(bind_dir,"Vulkan/nervbind.lua"),
}
  • Note: to be able to generate bindings for multiple modules in a single run with our all_modules.lua we have to reset the state of the following elements:
    • The LunaManager
    • The TypeManager
    • and the ClangParser

Some more ordering in the register_pacakge.cpp file

  • Recently I introduced the support to sort the classes/functions/enums alphabethically before writing the bindings to file, but this is still not quite perfect in the register_packages.cpp file where we have to “push/pop” namespaces before registering the classes:
  • In that specific cases sorting the namespaces in use should take priority: let's implement that ⇒ here is the updated code to support this: 
      -- Before registering the classes we sort the namespace stacks:
  local nsStacks = {}
  local nsNames = Set()

  for _,cl in ipairs(classes) do
    stack = self:getNamespaceStack(cl)
    stackName = table.concat(stack,".")
    nsStacks[stackName] = nsStacks[stackName] or {}
    nsNames:insert(stackName)
    table.insert(nsStacks[stackName], cl)
  end
  
  nsNames = nsNames:getElements()
  table.sort(nsNames)

  -- Then we should register all the classes:
  for _,nsName in ipairs(nsNames) do
    stack = nsStacks[nsName]
    for _, cl in ipairs(stack) do
      self:updateNamespaceStack(cl)
      self:writeSubLine("Luna< ${1} >::Register(L);", cl:getFullName())
    end
  end
  • Note: I also updated the ordering of the copyParents() calls in the process, with: 
      -- Instead of the previous implementation above we can use the list of namespace names already available:
  -- but we add "_G" to it first:
  table.insert(nsNames, "_G")
  for _, name in ipairs(nsNames) do
    self:writeSubLine("luna_copyParents(L, \"${1}\");", name)
  end

Continuing with the vulkan bindings

  • Now that we have an initial (rather empty) binding module for vulkan, lets add our specific classes and the corresponding bindings in parallel.
  • In the VulkanApp.lua file, it seems the first thing we will need in the “vulkan library”: 
        logDEBUG("Loading Vulkan bindings...")
    bm:loadBindings("Vulkan", "luaVulkan")

    -- Create the Library:
    local vk = nv.vulkan
    local lib = vk.Library.instance();
  • ⇒ let's add that first piece.
  • OK: I have my initial VulkanLibrary class and generated the bindings for it.
  • Now trying to load the class in lua: this works (after some tweaking of the bindings of course), but what I realize here already is that my logger configuration doesn't pass the DLL boundaries (ie. DEBUG messages from nvCore are visible but not those from nvVulkan) ⇒ I really need to restore my LogManager class, maybe using the fmt library for formatting ?

Adding bindings for instance extension names

  • In my previous implementation using the SOL library to generate the bindings I could write ordinary lua tables in the bindings as such: 
        auto exts = vulkan["Extensions"].get_or_create<sol::table>();

	exts["VK_KHR_SURFACE"] = VK_KHR_SURFACE_EXTENSION_NAME;
	exts["VK_KHR_WIN32_SURFACE"] = VK_KHR_WIN32_SURFACE_EXTENSION_NAME;
	exts["VK_EXT_DEBUG_UTILS"] = VK_EXT_DEBUG_UTILS_EXTENSION_NAME;
    exts["VK_KHR_SWAPCHAIN"] = VK_KHR_SWAPCHAIN_EXTENSION_NAME;
  • That's pretty convinient, but unfortunately I don't see how to provide that our of the box in NervLuna 🤔… except if I start pushing custom lua code directly in the bindings maybe ?… But that doesn't sound clean enough… 😖
  • But anyway, I quickly realized I could still handle that in NervLuna with static fields in a struct as follow: 
    namespace nvk {

struct InstanceExtensions {
    InstanceExtensions() = delete;
    static constexpr const char* VK_KHR_SURFACE = VK_KHR_SURFACE_EXTENSION_NAME;
}

} // namespace nvk
    /
  • ⇒ So for now, this will be good enough 😁!

Restoring LogManager encapsulation

  • As mentioned above, I seem to have a problem with the spdlog config not cross my DLL boundaries: I need to sort this out… as I'm basically lost without my precious debug outputs configured properly 😅
  • ⇒ I think what I really need to do here is to use the fmt library directly instead of spdlog, so let's build it:
  • So cloning the repository https://github.com/fmtlib/fmt
  • Then adding the library config in Nervproj: 
      - name: fmt
    url: git@github.com:roche-emmanuel/fmt.git
    version: 9.1.1
  • And adding the builder itself: 
    """This module provide the builder for the fmt library."""

import logging

from nvp.core.build_manager import BuildManager
from nvp.nvp_builder import NVPBuilder

logger = logging.getLogger(__name__)


def register_builder(bman: BuildManager):
    """Register the build function"""

    bman.register_builder("fmt", FmtBuilder(bman))


class FmtBuilder(NVPBuilder):
    """Fmt builder class."""

    def build_on_windows(self, build_dir, prefix, _desc):
        """Build method for Fmt on windows"""

        flags = ["-S", ".", "-B", "build", "-DBUILD_SHARED_LIBS=FALSE", "-DCMAKE_POSITION_INDEPENDENT_CODE=TRUE"]
        self.run_cmake(build_dir, prefix, flags=flags)
        sub_dir = self.get_path(build_dir, "build")
        self.run_ninja(sub_dir)

    def build_on_linux(self, build_dir, prefix, desc):
        """Build method for Fmt on linux"""

        flags = ["-S", ".", "-B", "build", "-DBUILD_SHARED_LIBS=FALSE", "-DCMAKE_POSITION_INDEPENDENT_CODE=TRUE"]
        self.run_cmake(build_dir, prefix, flags=flags)
        sub_dir = self.get_path(build_dir, "build")
        self.run_ninja(sub_dir)
  • Next building the library in static mode with msvc: (cf. https://fmt.dev/latest/usage.html): 
    $ nvp build libs fmt
  • And we do the same with clang: 
    $ nvp build lib fmt -c clang
  • ⇒ OK no problem with the build.
  • Now adding this instead of spdlog in nvCore as an internal library only: [crunching.. crunching… crunching…] And now this is finally OK: I'm back with a working LogManager component and I can continue with the Vulkan implementation.

Additional Vulkan bindings

  • On the vulkna side of things, the next stop is around those lines: 
        local exts = nvk.Extensions;
    local layers = {"VK_LAYER_LUNARG_standard_validation"}
  • ⇒ I don't have that “Extensions”, instead above, I introduced the InstanceExtensions class with static fields. So updating accordingly…
  • And next we are about to create a vulkan instance: 
        local inst = nvk.Instance({exts.VK_KHR_SURFACE, exts.VK_KHR_WIN32_SURFACE, exts.VK_EXT_DEBUG_UTILS}, layers, false)
    logDEBUG("Number of physical vulkan devices: ", inst:getNumPhysicalDevices())
  • ⇒ So adding the VulkanInstance class now. OK
  • Also adding the VulkanPhysicalDevice in the process: OK
  • But, arrggff… now I need support for a custom constructor method in lua for the VulkanInstance to support StringList arguments lol… That might be a bit tricky to implement.
  • ⇒ I've been re-reading my existing code on this aspect, but I have to admit I barely remember what I was doing to add extensions to my code 😅. But it seems there is something to do with LuaFunction elements, __lunaext__ prefix or __lunarawext__ prefix… So let's see if I can figure out how to use these again.
  • Okay, so my idea is to build an extension function to create a VulkanInstance from lua table as follow: 
    inline auto __lunarawext__VulkanInstance(VulkanInstance& /*inst*/, lua_State* L)
    -> int {
    // Construct a new instance from tables on stack:
    int luatop = lua_gettop(L);
    if (luatop < 2)
        return 0;

    // Get the extensions:
    int num = (int)lua_objlen(L, 1);
    nv::StringList exts;
    for (int i = 0; i < num; ++i) {
        // Get from the table at index 1:
        lua_rawgeti(L, 1, i + 1);
        // Read the string from the stack:
        // We can use a simple tostring call here as we don't need the string
        // length for simple extension names:
        const char* str = lua_tostring(L, -1);
        logDEBUG("Reading from lua extension name: {}", str);
        exts.push_back(str);
        // pop the string value:
        lua_pop(L, 1);
    }

    nv::StringList layers;
    for (int i = 0; i < num; ++i) {
        // Get from the table at index 2:
        lua_rawgeti(L, 2, i + 1);
        // Read the string from the stack:
        // We can use a simple tostring call here as we don't need the string
        // length for simple extension names:
        const char* str = lua_tostring(L, -1);
        logDEBUG("Reading from lua layer name: {}", str);
        layers.push_back(str);
        // pop the string value:
        lua_pop(L, 1);
    }

    // VulkanInstance(const nv::StringList& extensions,
    //             const nv::StringList& layers,
    //             bool useDebugMessenger = false);
}
  • Except that I want this function to be used as a constructor: ie, I should not provide any existing “VulkanInstance” reference during the call…I'm wondering if this will work 🤔… Quick answer: it can't work really: even if the bindings are done right, when it's time for Luna to call this function it will of course try to provide an instance reference as first argument. Crap, I need to think harder about this and find a solution.
  • Oh, and placing this lunarawext function in compile_context.h only doesn't seem to be enough: it's not considered for the bindings generation here. So I think I need to declare it in the bind_context.h or shared_context.h too.
  • Okay, so for now instead of going into the troubles of adding support to write constructor extensions, I'm simply going to pretend this is a new create_instance method I'm adding on the VulkanLibrary class instead. I know… lazy implementation like that sounds dangerous… but I feel really lazy/tired/unmotivated right now.
  • Arrfff what next ? ⇒ Seems my function is ignored for now: 
    Ignoring non DLL imported signature: nvk::VulkanLibrary::create_instance [auto (nvk::VulkanLibrary &, lua_State *) -> int]
  • After fixing that, it seems my function is still “non-public” too: 
    Ignoring non-public function nvk::%%__lunarawext__%%create_instance
  • Okay: so I eventually figured out I needed to check for a canonical lua state argument when checking the bindability of a signature: 
    function Class:isLuaConvertible()
  for k,arg in ipairs(self._arguments) do
    -- Canonical lua state is convertible to lua:
    if arg:getType():getLuaConverter() == nil and not arg:isCanonicalLuaState() then
      return false, arg:getType():getName()
    end
  end
end
  • And now, let's also change the prefix for those extensions to use a single underscore instead of 2 as clang tidy doesn't like them: OK
  • Next I had to properly restore the support for the void type, registering that type by default as I'm not registering it in the nvCore bindings anymore: 
        // Also prepare here the registration of the void type:
    luna::pushGlobalNamespace(L);

    luna::Luna<void>::Register(L);

    luna::popNamespace(L);
  • And then we continue with more vulkan functions: namely, we need the createPresentationSurface method now. So we need the VulkanSurface class.

Additional handling of smart pointer containers

  • Hmmm, okay, so I have a bit of a problem with my create_presentation_surface() method as it has the following signature: 
        auto create_presentation_surface(const VulkanWindowParameters& params)
        -> nv::RefPtr<VulkanSurface>;
  • ⇒ The NervBind system is confused because the return type nv::RefPtr<VulkanSurface> is considered unbindable: too bad 😆
  • But really, this should not matter and instead, we should be able to create a nv::RefPtr<VulkanSurface>, and then just use the .get() method when pushing that object on the lua stack… let's see…
  • ⇒ I need to deal with this in the Type:getLuaConverter(): Done !
  • I have now added a dedicated RefPtrConverter class in NervLuna that will handle extracting/injecting RefPtr object on the Lua stack. Base implementation was adapted from the default ClassConverter version: 
    local Class = createClass{name="RefPtrConverter", bases="reflection.LuaConverter"}

-- local luna = import "bind.LunaManager"

-- This converter is used for parameters/results that are raw class objects.
function Class:__init()
  Class.super.__init(self)
end

function Class:writeTypeCheck(buf, idx, atype, hasDef)
  local cond = ""
  if hasDef then
    -- Conditional check:
    cond = ("luatop>=%d && "):format(idx)
  end

  -- Once we have the type we can retrieve the target class from it:
  local tgtCl = atype:getTarget()
  
  -- Extract the real target name:
  local refptr_name = tgtCl:getFullName()
  local cl_name = refptr_name:sub(12, -2)
  local luna = import "bind.LunaManager"

  -- #104: If the type is a pointer, then we should also accept a nil value as input here:
  local allowNull = atype:getPointerCount()>0 and "true" or "false"
  buf:writeSubLine("if( ${1}!luna_isInstanceOf(L,${2},${3},${4}) ) return false;", cond, idx, luna:getSID(cl_name), allowNull)
end

function Class:writeGetter(buf, idx, arg, isField)
  -- The arg value we received should be an instance of the Argument class
  -- So from there we can retrieve: the target type, argument name, default value, etc.
  
  -- if we have the given lue entry then we can retrieve it
  -- Otherwise we use the default value:
  local atype = arg:getType()
  local typeName = atype:getUnqualifiedName()
  local argName = arg:getName()
  local defVal = nil
  if not isField then
    defVal = arg:getDefault()
  end

  -- Extract the real target name:
  local tgtCl = atype:getTarget()
  local refptr_name = tgtCl:getFullName()
  local cl_name = refptr_name:sub(12, -2)

  -- Below we only add the star prefix if we are not trying to use a pointer:
  local pcount = atype:getPointerCount()
  local prefix = (pcount==2 and "&") or (pcount==1 and "") or (pcount==0 and "*")
  local allowNull = pcount>0 and "true" or "false"

  if defVal ~= nil then
    -- We have a default value, so we check the luatop:
    buf:writeSubLine("${1} ${2} = luatop>=${3}? Luna< ${5} >::get(L,${3},${4}) : nullptr;", typeName, argName, idx, allowNull, cl_name)
    return ("luatop>=%d ? %s%s.get() : %s"):format(idx, prefix, argName, defVal)
  else
    -- No default value, we must retrieve it from the lua stack:
    buf:writeSubLine("${1} ${2} = Luna< ${5} >::get(L,${3},${4});", typeName, argName, idx, allowNull, cl_name);
    if isField then
      local contName = arg:isStatic() and arg:getNamespaceName().."::" or "self->"
      buf:writeSubLine("${3}${1} = ${2}${1}.get();", argName, prefix, contName);
    end

    return prefix..argName
  end
end

function Class:writeSetter(buf, varname, type, prefix)
  -- If we have  resulting "instance" we should not just push it on the stack, because the object
  -- would then get out of scope.
  -- So we have to create a copy object on the heap in this case.
  -- So we expect to have a copy constructor ?
  
  -- Update: if the type is a reference, then we can push the pointer to the object directly:

  local typeName = type:getUnqualifiedName();

    -- Extract the real target name:
    local tgtCl = type:getTarget()
    local refptr_name = tgtCl:getFullName()
    local cl_name = refptr_name:sub(12, -2)

  if type:isReference() then
    -- Push the reference directly, and do not own the memory of that object:
    buf:writeSubLine("Luna< ${1} >::push(L, ${2}.get(), true);", cl_name, prefix..varname)
  elseif type:isPointer() then
    -- Push the pointer directly, and do not own the memory of that object:
    buf:writeSubLine("Luna< ${1} >::push(L, ${3}${2}->get(), true);", cl_name, prefix..varname,type:getPointerCount()==2 and "*" or "")
  else
    -- Raw object access:
    -- #104: if we are writting a field getter, then we expect to get a pointer on the field itself,
    -- not a new copy of that field (and in that case we do not own the pointer)
    if prefix~="" then
      buf:writeSubLine("Luna< ${1} >::push(L, ${3}${2}.get(), true);", cl_name, varname, prefix)
    else
      -- buf:writeSubLine("${1}* ${2}_ptr = new ${1}(${3});", typeName, varname, varname)
      -- buf:writeSubLine("Luna< ${1} >::push(L, ${2}_ptr, true);", typeName, varname)
      buf:writeSubLine("Luna< ${1} >::push(L, ${2}.get(), true);", cl_name, varname)
    end
  end
  
end

function Class:isClassSupported(tgt)
  fname = tgt:getFullName()

  -- "nv::RefPtr<"
  clname = fname:sub(12, -2)

  -- logDEBUG("RefPtrConverter: checking support for ", fname, " => clname: '", clname, "'")
  -- check if that class is available:
  local tm = import "reflection.TypeManager"
  cltype = tm:getType(clname)
  if cltype ~= nil then
    logDEBUG("RefPtrConverter: Found type for class ", clname, " in ", fname)
    return true
  end

  return false
end


return Class()

Optimizing enum value names

  • Moving forward with the VulkanApp support, I decided the bind directly the VkSurfaceCapabilitiesKHR struct to not make my life to complex… and in this process, I had to get NervLuna to parse the full Vulkan headers (and it's not crashing 😲!), and as a result this is producing a giant list of awful vulkna enums such as this: 
    	// enum VkAccelerationStructureMemoryRequirementsTypeNV
	lua_newtable(L);
	lua_pushnumber(L, VK_ACCELERATION_STRUCTURE_MEMORY_REQUIREMENTS_TYPE_OBJECT_NV); lua_setfield(L,-2,"VK_ACCELERATION_STRUCTURE_MEMORY_REQUIREMENTS_TYPE_OBJECT_NV");
	lua_pushnumber(L, VK_ACCELERATION_STRUCTURE_MEMORY_REQUIREMENTS_TYPE_BUILD_SCRATCH_NV); lua_setfield(L,-2,"VK_ACCELERATION_STRUCTURE_MEMORY_REQUIREMENTS_TYPE_BUILD_SCRATCH_NV");
	lua_pushnumber(L, VK_ACCELERATION_STRUCTURE_MEMORY_REQUIREMENTS_TYPE_UPDATE_SCRATCH_NV); lua_setfield(L,-2,"VK_ACCELERATION_STRUCTURE_MEMORY_REQUIREMENTS_TYPE_UPDATE_SCRATCH_NV");
	lua_pushnumber(L, VK_ACCELERATION_STRUCTURE_MEMORY_REQUIREMENTS_TYPE_MAX_ENUM_NV); lua_setfield(L,-2,"VK_ACCELERATION_STRUCTURE_MEMORY_REQUIREMENTS_TYPE_MAX_ENUM_NV");
	lua_setfield(L,-2,"VkAccelerationStructureMemoryRequirementsTypeNV");

	// enum VkAccelerationStructureMotionInstanceTypeNV
	lua_newtable(L);
	lua_pushnumber(L, VK_ACCELERATION_STRUCTURE_MOTION_INSTANCE_TYPE_STATIC_NV); lua_setfield(L,-2,"VK_ACCELERATION_STRUCTURE_MOTION_INSTANCE_TYPE_STATIC_NV");
	lua_pushnumber(L, VK_ACCELERATION_STRUCTURE_MOTION_INSTANCE_TYPE_MATRIX_MOTION_NV); lua_setfield(L,-2,"VK_ACCELERATION_STRUCTURE_MOTION_INSTANCE_TYPE_MATRIX_MOTION_NV");
	lua_pushnumber(L, VK_ACCELERATION_STRUCTURE_MOTION_INSTANCE_TYPE_SRT_MOTION_NV); lua_setfield(L,-2,"VK_ACCELERATION_STRUCTURE_MOTION_INSTANCE_TYPE_SRT_MOTION_NV");
	lua_pushnumber(L, VK_ACCELERATION_STRUCTURE_MOTION_INSTANCE_TYPE_MAX_ENUM_NV); lua_setfield(L,-2,"VK_ACCELERATION_STRUCTURE_MOTION_INSTANCE_TYPE_MAX_ENUM_NV");
	lua_setfield(L,-2,"VkAccelerationStructureMotionInstanceTypeNV");
=> So here I'm now thinking I should probably do something to reduce the length of the name for the **values** to be retrieved in each enum: I think I could for instance remove the common part in all values like "VK_ACCELERATION_STRUCTURE_MEMORY_REQUIREMENTS_TYPE_" and "VK_ACCELERATION_STRUCTURE_MOTION_INSTANCE_TYPE_" above => let's handle that!
  • And here the resulting function to handle that: 
    function Class:getCommonPrefixLength(constants)
  -- If we only have 1 element we do not search for a common prefix here:
  if #constants <= 1 then
    return 0
  end

  ref_name = constants[1].value

  prefix = ref_name

  local is_common = function(prefix)
    for _, obj in ipairs(constants) do
      if not obj.name:startsWith(prefix) then
        return false
      end
    end

    return true
  end

  while not is_common(prefix) do
    prefix = prefix:sub(1,-2)
    if prefix == "" then
      return 0
    end
  end

  -- Return the length of the prefix:
  -- logDEBUG("Found common enum prefix: ", prefix)
  return #prefix
end
  • And this works just as desired 👍: 
    	// enum VkAccelerationStructureCompatibilityKHR
	lua_newtable(L);
	lua_pushnumber(L, VK_ACCELERATION_STRUCTURE_COMPATIBILITY_COMPATIBLE_KHR); lua_setfield(L,-2,"COMPATIBLE_KHR");
	lua_pushnumber(L, VK_ACCELERATION_STRUCTURE_COMPATIBILITY_INCOMPATIBLE_KHR); lua_setfield(L,-2,"INCOMPATIBLE_KHR");
	lua_pushnumber(L, VK_ACCELERATION_STRUCTURE_COMPATIBILITY_MAX_ENUM_KHR); lua_setfield(L,-2,"MAX_ENUM_KHR");
	lua_setfield(L,-2,"VkAccelerationStructureCompatibilityKHR");

	// enum VkAccelerationStructureCreateFlagBitsKHR
	lua_newtable(L);
	lua_pushnumber(L, VK_ACCELERATION_STRUCTURE_CREATE_DEVICE_ADDRESS_CAPTURE_REPLAY_BIT_KHR); lua_setfield(L,-2,"DEVICE_ADDRESS_CAPTURE_REPLAY_BIT_KHR");
	lua_pushnumber(L, VK_ACCELERATION_STRUCTURE_CREATE_MOTION_BIT_NV); lua_setfield(L,-2,"MOTION_BIT_NV");
	lua_pushnumber(L, VK_ACCELERATION_STRUCTURE_CREATE_FLAG_BITS_MAX_ENUM_KHR); lua_setfield(L,-2,"FLAG_BITS_MAX_ENUM_KHR");
	lua_setfield(L,-2,"VkAccelerationStructureCreateFlagBitsKHR");

Some initial vector handling

  • Next step now: I need some kind of handling for vectors of objects, like with this function: 
        void get_surface_formats(VulkanSurface* surface,
                             VkSurfaceFormatList& formats) const;
  • I could write an extension to handle that but it would be far better if I could instead handle Vectors programmatically, so let's investigate a bit in that direction.
  • OK: so I'm now checking for nv::Vector containers directly in the ClassWriter, and injecting the relevant functions manually as needed: 
    function Class:injectVectorOf(cl, el_type)
  tm = import "reflection.TypeManager"

  local create_func = function(name, signame, ret_type, args)
    local func = cl:getOrCreateFunction(name)
    local sig = func:createSignature(signame)
    sig:setVisibility(Vis.PUBLIC)
    sig:setImplicit(true)
    sig:setDllImported(true)
    sig:setDefined(true)
    
    if ret_type ~= nil then
      sig:setReturnType(ret_type)
    end

    for _,arg in ipairs(args or {}) do
      sig:addArgument(Argument(arg[1], arg[2], arg[3]))
    end
  end

  local el_name = el_type:getName()
  local u32_t = tm:getType("nv::U32")
  local void_t = tm:getType("void")
  local bool_t = tm:getType("bool")
  local el_ref_t = tm:getType(el_name.."&")
  if el_ref_t == nil then
    local el_t = tm:getType(el_name)
    el_ref_t = Type(el_name.."&", el_t:getTarget(), nil)
    el_ref_t:addReference()
    tm:addType(el_ref_t)
  end

  -- Inject the functions:
  create_func(cl:getName(), "void ()")
  create_func("size", "nv::U32 ()", u32_t)
  create_func("at", el_name.."& (nv::U32)", el_ref_t, {{"n", u32_t}})
  -- create_func("operator[]", el_name.."& (nv::U32)", el_ref_t, {{"n", u32_t}})
  create_func("push_back", "void ("..el_name.."&)", void_t, {{"val", el_ref_t}})
  create_func("pop_back", "void ()", void_t)
  create_func("assign", "void (nv::U32, "..el_name.."&)", void_t, {{"n", u32_t},{"val", el_ref_t}})
  create_func("back", el_name.."& ()", el_ref_t)
  create_func("front", el_name.."& ()", el_ref_t)
  create_func("empty", "bool ()", bool_t)
  create_func("clear", "void ()", void_t)
  create_func("resize", "void (nv::U32)", void_t, {{"n", u32_t}})
end

Stopping here for this post

  • And now I need to get back to my enum bindings in fact: as these seem to be considered as non defined classes for now.
  • But what I really need above all is to stop this post and start a new one as this is definitely too long now 🤣.
  • blog/2022/1004_nervluna_restoring_automatic_binding.txt
  • Last modified: 2022/10/04 19:27
  • by 127.0.0.1