====== NervLuna: A first practical usage case - Part 1 ====== {{tag>dev nervluna bindings}} Hi readers! So here we are back on our NervLuna project for lua bindings generation... Or at least, that's what I "kind of" remember it was about ? I mean, all this stuff seems so far away already lol. But anyway, lately I've been working on lua bindings generation for one of the other professional projects I'm working on (ie. the [[public:work:simcore:simcore|SimCore project]]). I had to get something very quickly on rails, so I immediately turned to the [[https://github.com/ThePhD/sol2|Sol3 library]] of course. And oh boy... (don't get me wrong, I really like the sol3 library, the syntax is cool, you have all the needed features and all in there, but...) this experiment definetely reminded me how complex it was to use that library properly and why I decided to start working on the nervluna project in the first place! So here I am again: back on NervLuna, and I want to try to use it to generate the bindings I need in SimCore while those bindings are still not too big. So let's get started! ====== ====== ===== Back to the basis ===== Hmmm, let's be honest, I haven't been working on this project for ages... and given my really poor memory I hardly remember anything about it ah ah ah LOL. The first question is thus: "Where should I start from ?" => Let me re-read the first article on this project to see if this helps refreshing my mind a bit. Okay, so first thing I'm starting to remember here is that I was using lua to run the binding generation, given some configuration file, soo let's see if I can find again an example of how this was run in my sources... [🔎//searching...//] OK! So now I have something else: just found in my "dist" folder that I have a collection of lua files, and among those files I have definition of "lua apps" entry points! And a very tiny tiny voice in my head is telling me that the lua app I've need using to generate bindings is this **NervBind.lua** app. {{ blog:2021:0306:lua_apps_listing.png?350 }} So now I just need to find where I'm using this "NervBind" script filename... [🔎//searching again with visual studio code search feature... but finding nothing this time//] Hmmm, this would have been too easy of course. So it seems I need to put some more thinking here. Oh! One more step forward now: I was checking the binary folders in my "dist" folder and realized there was no executable file (and not even a luajit.exe), so I tried to rebuild all my code from sources, with one of my commands that I still remember [surprising isn't it ?]: nv_build msvc64 This is producing a great lot of errors lol. Still, eventually it will install a few files: Install the project... -- Install configuration: "Release" -- Up-to-date: D:/Projects/NervSeed/dist/bin/msvc64/./cef_process.exe -- Up-to-date: D:/Projects/NervSeed/dist/bin/msvc64/./cef_process.pdb -- Up-to-date: D:/Projects/NervSeed/dist/bin/msvc64/./cef_process.map -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvCore.dll -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvCore.pdb -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvCore.map -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvCEF.dll -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvCEF.pdb -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvCEF.map -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvLand.dll -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvLand.pdb -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvLand.map -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvView.dll -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvView.pdb -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvView.map CMake Error at sources/nvVulkan/src/cmake_install.cmake:41 (file): file INSTALL cannot find "D:/Projects/NervSeed/build/msvc64/sources/nvVulkan/src/nvVulkan.dll": No error. Call Stack (most recent call first): sources/nvVulkan/cmake_install.cmake:42 (include) sources/cmake_install.cmake:49 (include) cmake_install.cmake:42 (include) ... But still no executable generated here! Yet this made me realize that the executable I'm after is my **NervSeed** app! And that's most probably the one I'm using to load/execute lua apps too. So my first step here should be to rebuild NervSeed correctly: let's try that. ===== Rebuilding NervSeed application and modules ===== Getting the NervSeed sources to compile was actually pretty easy: I just had to disable all the non compiling modules and stuff: * **nvVulkan**: will not compile because I didn't even install the vulkan runtime yet on my computer, so this is just normal I guess. * **nvOCCT**: I had some serious problems/errors while quickly trying to build the openCASCADE dependency, so I simply bypassed that for now, and of course this module will not compile anymore for now. * **lua_bindings**: I had a lot of errors in the compilation here too so just starting with this disabled first. So basically I have the CmakeList content for my sources folder: ADD_SUBDIRECTORY(soil2) ADD_SUBDIRECTORY(libcef_dll) ADD_SUBDIRECTORY(cef_process) ADD_SUBDIRECTORY(nvCore) ADD_SUBDIRECTORY(nvCEF) ADD_SUBDIRECTORY(nvLand) # ADD_SUBDIRECTORY(nvPy) ADD_SUBDIRECTORY(nvView) # ADD_SUBDIRECTORY(nvVulkan) ADD_SUBDIRECTORY(nvDX12) ADD_SUBDIRECTORY(nvGL) # ADD_SUBDIRECTORY(nvOCCT) ADD_SUBDIRECTORY(nvLLVM) # ADD_SUBDIRECTORY(NervSeed0) ADD_SUBDIRECTORY(NervSeed) ADD_SUBDIRECTORY(llvm_syms) # ADD_SUBDIRECTORY(lua_bindings) ADD_SUBDIRECTORY(lua_lfs) # if(${FLAVOR} STREQUAL "WIN64") # ADD_SUBDIRECTORY(nvUnity) # ADD_SUBDIRECTORY(nvUnityProvider) # ADD_SUBDIRECTORY(scUnityGateway) # ENDIF() Then from the tests/ folder I also had to disable my **test_nvVulkan** and **test_vulkan_win** tests obviously. => And then I got NervSeed rebuilt as desired! Yeepeee! ===== Relearning how to use NervSeed ===== Checking the code of the NervSeed app, the "main()" function is actually very minimal and it will delegate everything directly to a Lua environment from what I see [//Yes I know... it's strange to speak this way of your own code lol//], disregarding the error/exception handling parts we only have this: LogManager::instance().setNotifyLevel(LogManager::DEBUG0); { // First we retrieve the NervApp instance: auto& app = NervApp::instance(); // First we retrieve the LuaManager: auto& lm = LuaManager::instance(); // We assign the command line arguments to the NervApp: { auto& state = lm.getMainState(); auto t = state["arg"].get_or_create(); for(U32 i=0; i So here we only collect all the provided arguments into the lua "arg" table, and then we call the "run.lua" script. Cool, this one, I already know where to find: in dist/assets/lua :-). Let's have a look: logDEBUG3("Lua: this is the main run script.") -- logDEBUG("Lua: the provided arguments are: '",arg,"'") -- logDEBUG("Num arg: ", #arg) -- for i=0,#arg do -- logDEBUG("arg ",i,": ", arg[i]) -- end -- The app name should always be the first argument we provide: if #arg < 1 then logERROR("No command provided.") return end -- Otherwise we first check if the first argument correspond to the valid app name: local cmd = arg[1] table.remove(arg,1) -- Prepare a command name mapping local commandMap = { vkapp = "app.VulkanApp", sdlapp = "app.SDLApp", bind = "app.NervBind", bind1 = {"app.NervBind", "-c", "config/bind_test.lua"}, bind2 = {"app.NervBind", "-c", "config/bind_vulkan.lua"}, bind3 = {"app.NervBind", "-c", "config/bind_core.lua"}, -- bind3d = {"app.NervBind", "-c", "config/bind_core.lua", "--debug"}, parse1 = {"app.NervBind", "-p", "sources/lua_bindings/test1/interface/bind_test1.h"}, parse1b = {"app.NervBind", "-p", "sources/lua_bindings/test1/interface/bind_test2.h"}, parse1c = {"app.NervBind", "-p", "sources/lua_bindings/test1/interface/bind_test3.h"}, parse1d = {"app.NervBind", "-p", "sources/lua_bindings/test1/interface/bind_test4.h"}, parse1e = {"app.NervBind", "-p", "sources/lua_bindings/test1/interface/bind_test5.h"}, parse2 = {"app.NervBind", "-p", "sources/lua_bindings/wip_Vulkan/interface/vk_layer.h"}, tests = "app.Telescope", bind0 = "app.NervBind_v0", tsgen = "app.TSDclGen", test1 = "test.Test1" } -- Check if we have a mapping for that command name: local rep = commandMap[cmd] if type(rep) == "string" then cmd = rep elseif type(rep) == "table" then cmd = rep[1] local newargs = {} for i=2,#rep do table.insert(newargs, rep[i]) end -- Add the remaining command line args: for i=1,#arg do table.insert(newargs, arg[i]) end -- Replace the command line args: arg = newargs end if hasScript(cmd) then -- We run that script providing it all the input args except thi script name itself: local app = import(cmd) app(arg) return end logERROR("Unknown script: ", cmd) And now, this is all starting to fall back in place: I now remember this command mapping I was using to avoid having to type long command lines all the time! So here for instance, I should be able to call `NervSeed bind1` and this should automatically call the "NervBind" app giving it the config file "config/bind_test.lua" On top of that, now I'm also remembering I had a dedicated shell script **nv_seed** to call NervSeed from any location with additional setups, and here it is: nv_seed() { local flavor="linux64" local suffix="" local pname=`nv_os` case "${pname}" in cygwin|wsl) flavor="msvc64" suffix=".exe" ;; *) export LD_LIBRARY_PATH="`nv_get_project_dir`/tools/linux/${__nv_tool_gcc}/lib64" ;; esac export VK_INSTANCE_LAYERS="VK_LAYER_LUNARG_standard_validation" local tapp="`nv_get_dist_dir`/bin/$flavor/NervSeed$suffix" $tapp "$@" # unset LD_LIBRARY_PATH } => So now trying the command: **nv_seed bind1** [//crossing fingers...//] Arff, maybe this was a little bit too pretencious afterall: {{ blog:2021:0306:missing_lua_core_dll.png?1080 }} Remember that I disabled the building of all the lua modules above, but the luaCode.dll is one of then, and this is definitely required in NervSeed, so we need to build at least that one => fixing this an rebuilding: **OK** And now that I'm thinking about it, I'm pretty sure I will need at least also the Clang lua module: since we will use clang to parse the input source files... but let's predend I didn't mention that yet. Trying the command **nv_seed bind1** again... and sure enough we have an error on luaClang: {{ blog:2021:0306:missing_lua_clang_dll.png?1080 }} So I activated the build of the luaClang module and after that, it worked! Great!: $ nv_seed bind1 [Debug] Loading dynamic library: luaClang.dll [Debug] Parsed args: { config = "config/bind_test.lua" } [Debug] Loading config file: config/bind_test.lua [Debug] Using include path: D:\Projects\NervSeed\sources\lua_bindings\test1\interface [Debug] Found 26 existing files. [Debug] Using input file: D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h [Debug] Using input file: D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test2.h [Debug] Using input file: D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test3.h [Debug] Using input file: D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test4.h [Debug] Using input file: D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test5.h [Debug] Writing file D:\Projects\NervSeed\sources\lua_bindings\test1\luna_bindings.h [Debug] No change in file D:\Projects\NervSeed\sources\lua_bindings\test1\luna_bindings.h [Debug] Command line args: { "-x", "c++", "-std=c++17", "-ID:\\Projects\\NervSeed\\sources\\lua_bindings\\test1\\include", "-ID:\\Projects\\NervSeed\\sources\\lua_bindings\\test1\\interface" } [Debug] Parsing file: D:\Projects\NervSeed\sources\lua_bindings\test1\luna_bindings.h [Info] Num diagnostics for file D:\Projects\NervSeed\sources\lua_bindings\test1\luna_bindings.h: 3 [Info] - diag 1: D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:185:31: warning: direct base 'nvt::OtherClass' is inaccessible due to ambiguity: class nvt::ClassC -> class nvt::ClassB -> class nvt::OtherClass class nvt::ClassC -> class nvt::OtherClass [-Winaccessible-base] [Info] - diag 2: D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:232:15: warning: anonymous non-C-compatible type given name for linkage purposes by t ypedef declaration; add a tag name here [-Wnon-c-typedef-for-linkage] [Info] - diag 3: D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:291:15: warning: anonymous non-C-compatible type given name for linkage purposes by t ypedef declaration; add a tag name here [-Wnon-c-typedef-for-linkage] [Debug] Processing: 'my_enum' kind: 'EnumDecl [Debug] Creating enum ::my_enum [Debug] Adding enum constant: my_enum::ENUM_VALUE1 [Debug] Adding enum constant: my_enum::ENUM_VALUE2 [Debug] Adding enum constant: my_enum::ENUM_VALUE3 (... much more outputs here ... ) [Debug] Writing file D:\Projects\NervSeed\sources\lua_bindings\test1\src/luna/bind_std_string.cpp [Debug] No change in file D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_std_string.cpp [Debug] Writing class binding for std::basic_ostream> [Debug] Preprocessing function list... [Debug] Keeping 0 on 0 [Debug] Writing function type checks... [Debug] Writing function bindings... [Debug] Done writing function bindings. [Debug] Preprocessing field list... [Debug] Keeping 0/0 fields. [Debug] Writing fields type checks... [Debug] Writing fields bindings... [Debug] Done writing fields bindings. [Debug] Preprocessing field list... [Debug] Keeping 0/0 fields. [Debug] Writing fields type checks... [Debug] Writing fields bindings... [Debug] Done writing fields bindings. [Debug] Writing file D:\Projects\NervSeed\sources\lua_bindings\test1\src/luna/bind_std_basic_ostream_char_std_char_traits_char.cpp [Debug] Updating content of D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_std_basic_ostream_char_std_char_traits_char.cpp [Debug] Writing Package bindings... [Debug] Writing file D:\Projects\NervSeed\sources\lua_bindings\test1\src/luna/register_package.cpp [Debug] Updating content of D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\register_package.cpp [Debug] Done writing bindings. [Debug] Searching for obsolete files... [Debug] Removing obsolete/unused file D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_std_ostream.cpp [Debug] Done searching for obsolete files. [Debug] Uninitializing Clang parser. [Debug] Disposing cidx [Debug] Done uninitializing Clang parser. Deleted LogManager object. Looking for memory leaks... No memory leak detected. Exiting. ===== Investigating cursor visitor exception ===== The previous call of 'nv_seed bind1' seems to have been mostly working fine, but checking the debug outputs more carefully, I eventually noticed this exception somewhere in the middle of the processing: [Warning] No class template registered for 'MyTemplateTest<1>' creating default place holder class in namespace 'nvt' [Debug] Creating class nvt::MyTemplateTest<1> [Debug] Creating concrete class nvt::MyTemplateTest<1> for initial type name: nvt::MyTemplateTest [Error] Error in cursorVisitor: [string "bind.ClangParser"]:366: attempt to index local 't' (a nil value) stack traceback: [string "bind.ClangParser"]:366: in function 'parseTemplateType' [string "bind.ClangParser"]:1121: in function 'parseClass' [string "bind.ClangParser"]:1381: in function <[string "bind.ClangParser"]:1352> [C]: in function 'visitChildren' [string "bind.ClangParser"]:1352: in function 'parseNamespace' [string "bind.ClangParser"]:1379: in function <[string "bind.ClangParser"]:1352> [C]: in function 'visitChildren' [string "bind.ClangParser"]:1352: in function 'parseNamespace' [string "bind.ClangParser"]:1418: in function 'parseFile' [string "app.NervBind"]:137: in function 'parseInputs' [string "app.NervBind"]:154: in function 'run' [string "app.AppBase"]:23: in function '__init' [string "app.NervBind"]:12: in function '__init' [string "setup.lua"]:155: in function 'app' [string "run.lua"]:63: in main chunk [Debug] clang done parsing file: D:\Projects\NervSeed\sources\lua_bindings\test1\luna_bindings.h [Debug] Resolving target types... So this error occurs in parseTemplateType(), but from what I see that would be because the type parameter "t" that we receive in that function call is already nil: and this should come from the parseClass calls just before, so let's check that one. arrf, and indeed in the parseClass() function we have this: logDEBUG("Creating template specialization: ", tname) t = self:parseTemplateType(nil, cxtype, tname) CHECK(t, "Cannot parse template type: ",tname) => So indeed the first parameter (ie. "t") is nil in that case :-S. And right now I'm starting to have an hard time understanding what this all means exactly lol. Anyway, just noticed that at other locations I will create "Type" object when required, so let's do that here too: -- @luna2: If no type is provided already then we create one here with tname: if t == nil then t = Type(tname) end => And with that change the binding code generation seems to work fine without errors! **OK** ===== Invalid anonymous union names issue ===== Before trying to build our "newly" generated bindings for this test1 project, I thought I should at least have a quick look at the generated code... just in case... And of course, here is what I found in the first file I opened: // Typecheck for (anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) (1) with signature: void () static bool _check__anonymous_union_at_D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5_sig1(lua_State* L) { int luatop = lua_gettop(L); if( luatop!=0 ) return false; return true; } // Function bindings // Bind for (anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) constructor (1) with signature: void () static nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)* _bind__anonymous_union_at_D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5_sig1(lua_State* L) { auto res = luna_callDefaultConstructor(L); if(res == nullptr) luaL_error(L, "Cannot default construct instance of class nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)."); return res; } ... 🤢... Yeahh... I always feel the same with those kind of errors: "How on earth could this happen, in something that was working OK so far and that I didn't touch for months ?!". But eventually you'll realize it is just **always** like this, "bugs" just appear out of nowhere! 🥳 Anyway, let's now try to investigate where this could come from. Point is, I'm pretty sure I had something in place already to handle (ie. rename) some "anonymous things", but maybe this is currently not applied on unions ? So the problem seems to come from this code section (inside my StrutB structure definition): union { int intValue; bool boolValue; } my_anon_union; And now I'm thinking this union finding part could be something new that was not happening at all before. Time to gather some debug inputs from clang on our little bind_test1.h file with: nv_seed parse1 | tee parse1.log So this gives us the following outputs: [Debug] level 3: '' of kind 'UnionDecl', type: 'nvt::StructB::(anonymous union at sources/lua_bindings/test1/interface/bind_test1.h:217:5)' WARN: clang_getTypeFullSpelling: no decl found for int [Debug] level 4: 'intValue' of kind 'FieldDecl', type: 'int' WARN: clang_getTypeFullSpelling: no decl found for bool [Debug] level 4: 'boolValue' of kind 'FieldDecl', type: 'bool' [Debug] level 3: 'my_anon_union' of kind 'FieldDecl', type: 'nvt::StructB::(anonymous union at sources/lua_bindings/test1/interface/bind_test1.h:217:5)' [Debug] level 4: '' of kind 'UnionDecl', type: 'nvt::StructB::(anonymous union at sources/lua_bindings/test1/interface/bind_test1.h:217:5)' WARN: clang_getTypeFullSpelling: no decl found for int [Debug] level 5: 'intValue' of kind 'FieldDecl', type: 'int' WARN: clang_getTypeFullSpelling: no decl found for bool [Debug] level 5: 'boolValue' of kind 'FieldDecl', type: 'bool' Right now in ClangParser, when I find a "UnionDecl", I simply write a new class for it apparently: elseif ckind == clang.CursorKind.ClassDecl or ckind == clang.CursorKind.StructDecl or ckind == clang.CursorKind.UnionDecl then obj = self:parseClass(cur, scope, cname) This can make sense to some extents because we have "fields" defined in this union, but then if the union is anonymous (and that's most cases I believe ?) it doesn't make sense to write class bindings for it. So I need to deal with that first point. => So as a first step I decided I should introduce a new entity type for "UNION", and create a "Union" class as well as an "UnionHolder" as below: local Class = createClass{name="Union", bases={"reflection.Scope", "reflection.FieldHolder"}} local luna = import "bind.LunaManager" function Class:__init(name, parent) Class.__initBases(self, name, parent) self:setEntityType(self.EntityType.UNION) end function Class:isAnonymous() return self._name:startsWith(luna.anonymous_union_prefix) end function Class:isBindable() return not self:isIgnored() end return Class local Class = createClass{name="UnionHolder", bases="base.Object"} local Vis = import "reflection.Visibility" function Class:__init(name, parent) Class.super.__init(self) self._children.unions = {} end function Class:getOrCreateUnion(name) CHECK(name~="", "Invalid union name.") for _,obj in ipairs(self._children.unions) do if obj:getName() == name then return obj end end -- We should create this child namespace: logDEBUG("Creating enum ", self:getFullName(), "::", name) local Union = import "reflection.Union" local obj = Union(name, self) table.insert(self._children.unions, obj) return obj end -- Retrieve all the public unions function Class:getPublicUnions(tt) tt = tt or {} for _,el in ipairs(self._children.unions) do if el:getVisibility() == Vis.PUBLIC then table.insert(tt, el) end end return tt end return Class The Union class is also a "FieldHolder" so that I can then add fields like "int intValue" or "bool boolValue" inside the union definition. Then I simply added the UnionHolder base for my "Class" and "Namesapce" classes. And I updated the handling of Unions declarations to be similar to what I was doing for enums: elseif ckind == clang.CursorKind.UnionDecl then -- @luna3: We can also have anonymous unions in fact: if cname=="" then cname = luna:getNextName(luna.anonymous_union_prefix) logDEBUG("Registering anonymous union '", cname, "' with type ", typename) end -- We add that enum to the current scope: obj = scope:getOrCreateUnion(cname) self:parseUnion(cur, obj) And I built the required dedicated function to parse unions of course: function Class:parseUnion(cursor, union) if union:isParsed() then logDEBUG2("Union ", union:getFullName()," is already parsed.") return; end if not cursor:isDefinition() then -- Nothing more to do here, we won't get any definition from this cursor. return end -- Keep going, parsing the class: union:setParsed(true) -- We retrieve all the values inside that enum: cursor:visitChildren(function(cur, parent) local ckind = cur:getKind() CHECK(ckind == clang.CursorKind.FieldDecl, "Unexpect cursor type in union: ", cursor:getKindSpelling()) local cname = cur:getSpelling() obj = union:getOrCreateField(cname) self:parseField(cur, obj) return clang.VisitResult.Continue end) end ... So now let's see how lucky/unlucky I can be with those changes... (=> building again with "nv_seed bind1 | tee bind1.log") And okay, now the processing gives us: [Debug] Registering anonymous union 'luna_anonymous_union_2' with type nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) [Debug] Creating enum nvt::StructB::luna_anonymous_union_2 [Debug] Creating field nvt::StructB::luna_anonymous_union_2::intValue WARN: clang_getTypeFullSpelling: no decl found for int [Debug] Creating field nvt::StructB::luna_anonymous_union_2::boolValue WARN: clang_getTypeFullSpelling: no decl found for bool [Debug] Creating field nvt::StructB::my_anon_union [Debug] Resolving type nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) [Debug] Type kind: Elaborated [Debug] Adding type with class target name: nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5), (canonical type: nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)) [Debug] TypeManager: Registering type { "nvt::StructB::(anonymous union at D:\\Projects\\NervSeed\\sources\\lua_bindings\\test1\\interface\\bind_test1.h:217:5)" } But I still have (some of) those crazy bindings generated ggrrrrrr!! (yet the "typecheck/function bindings" previously reported are now gone): nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)* LunaTraits< nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) >::construct(lua_State* L) { luaL_error(L, "No public constructor available for class nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)"); return nullptr; } void LunaTraits< nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) >::destruct(nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)* obj) { THROW_MSG("No public destructor available for class nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)"); } The actual problem comes from the type resolution at the end of the binding generation process: [Debug] Resolving target types... [Debug] Resolving target for type: nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) [Debug] Couldn't resolve type target, using target name instead: nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) [Debug] Creating class nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) So this is now given us anything meaningfull (but that's expected), and then we still write a class binding for that type: [Debug] Writing class binding for nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) That part we should really avoid somehow: if we have some kind of "anonymous type" then we should not try to write a class binding for it at all. => So now I have updated my "final type resolution" process to catch those kind of invalid anonymous names and avoid creating default class binding for them: if tgtType then local tname = type:getName() logDEBUG("Resolving target for type: ", tname) count = count + 1 local cl = luna:getClass(tgtType) if not cl then tgtName = tgtType:getFullName() logDEBUG("Couldn't resolve type target, using target name instead: ", tgtName) -- Once we have a type name then we split it in parts: local parts = utils.splitEntityName(tgtName) local ns = luna:getRootNamespace() local nparts = #parts for i=1,nparts-1 do if ns:hasClass(parts[i]) then ns = ns:getOrCreateClass(parts[i]) else ns = ns:getOrCreateNamespace(parts[i]) end end -- Then we create our target class: local clname = parts[nparts] -- @luna3: Check if that class name only contains valid characters, otherwise we should ignore it: if self:isValidTypeName(clname) then cl = ns:getOrCreateClass(clname) else logDEBUG("Ignoring unresolvable/invalid type with resulting name: ", clname) end end type:setTarget(cl) end With the helper function: function Class:isValidTypeName(tname) -- Chekc if the provided typename is valid: if tname:find("anonymous enum at") then logDEBUG("Detected anonymous enum type name in ", tname) return false elseif tname:find("anonymous union at") then logDEBUG("Detected anonymous union type name in ", tname) return false end return true end And finally those invalid binding names are gone! Ouuff! : [Debug] Resolving target types... [Debug] Resolving target for type: nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) [Debug] Couldn't resolve type target, using target name instead: nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) [Debug] Detected anonymous union type name in (anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) [Debug] Ignoring unresolvable/invalid type with resulting name: (anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) [Debug] Done resolving 1 target types. Note that here we only really handled "avoiding" writing garbage when unions are found, but in the end, we still have no **support at all** to generate usable bindings for union variables. This is some I will have to deal with eventually. ===== _bind_MyTemplateTest compilation error ===== And now of course you guess it, since the generated binding files were looking OK I moved to the compilation stage of the test1 module. And I got errors... 😭 The first one being: [ 90%] Building CXX object sources/lua_bindings/test1/src/CMakeFiles/luaTest1.dir/luna/bind_nvt_MyTemplateTest_1.cpp.obj bind_nvt_MyStringTest.cpp [ 90%] Building CXX object sources/lua_bindings/test1/src/CMakeFiles/luaTest1.dir/luna/bind_nvt_MyVectorNameTest.cpp.obj bind_nvt_MyTemplateTest_1.cpp D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_nvt_MyTemplateTest_1.cpp(49): error C2065: '_bind_MyTemplateTest' : identificateur non déclaré bind_nvt_MyVectorNameTest.cpp jom: D:\Projects\NervSeed\build\msvc64\sources\lua_bindings\test1\src\CMakeFiles\luaTest1.dir\build.make [sources\lua_bindings\test1\src\CMakeFiles\luaTest1.dir\luna\bind_nvt_My TemplateTest_1.cpp.obj] Error 2 Having a look at that binding file we have indeed: // Overall bind for MyTemplateTest<1> static nvt::MyTemplateTest<1>* _bind_MyTemplateTest_1(lua_State* L) { if(_check_MyTemplateTest_1_sig1(L)) return _bind_MyTemplateTest_1_sig1(L); logERROR("Current lua stack: "<, cannot match any of the 1 signature(s):\n sig1: void ()"); return nullptr; } nvt::MyTemplateTest<1>* LunaTraits< nvt::MyTemplateTest<1> >::construct(lua_State* L) { return _bind_MyTemplateTest<1>(L); } => So it's cool: this one sounds simple to fix: we are already generating a construction function name correctly (on line 2 above), but then when we call it on line 11 it seems I'm not sanitizing the function name correctly. Let's fix that: 🆗! ===== bind_nvt_TplBool.cpp compilation error ===== Next compilation errors on the list are: bind_nvt_TplBool.cpp D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_nvt_TplBool.cpp(23): error C2039: 'TplBool' : n'est pas membre de 'nvt' D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test5.h(9): note: voir la déclaration de 'nvt' D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_nvt_TplBool.cpp(23): error C2143: erreur de syntaxe : absence de ';' avant '*' D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_nvt_TplBool.cpp(23): error C4430: spécificateur de type manquant - int est pris en compte par défaut. Remarque : C+ + ne prend pas en charge int par défaut D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_nvt_TplBool.cpp(23): error C2923: 'nv::LunaTraits' : 'nv::TplBool' n'est pas un argument de type modèle valide pour le paramètre 'T' D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_nvt_TplBool.cpp(23): note: voir la déclaration de 'nv::TplBool' D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_nvt_TplBool.cpp(23): error C2955: 'nv::LunaTraits' : l'utilisation de classe modèle requiert une liste d'arguments modèle D:\Projects\NervSeed\sources\nvCore\include\lua/luna.h(157): note: voir la déclaration de 'nv::LunaTraits' D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_nvt_TplBool.cpp(23): error C2761: 'T *nv::LunaTraits::construct(lua_State *)' : la redéclaration du membre n'est pas autorisée Now, looking at the binding file, the content looks OK. Ahh! but now I get it: the TplBool (and TplInt) types are defined in bind_test2.h: typedef MyTemplateClass TplInt; typedef MyTemplateClass TplBool; But the "bind_test2.h" was not included anywhere in the actuall build process since I was only using this global header: #ifndef BIND_COMMON_ #define BIND_COMMON_ #if defined(_MSC_VER) || defined(__CYGWIN__) || defined(__MINGW32__) || defined( __BCPLUSPLUS__) || defined( __MWERKS__) # if defined( BIND_LIBRARY_STATIC ) # define BIND_EXPORT # elif defined( BIND_LIBRARY ) # define BIND_EXPORT __declspec(dllexport) # else # define BIND_EXPORT __declspec(dllimport) # endif #else # define BIND_EXPORT #endif //#pragma warning( disable: 4251 ) #include #include #include #include #include #include #include #endif So just adding "#include " there, and then we were good 🤟! ===== std::_Simple_types link error ===== So, after the compilation stage, more errors at link stage! mouaahh ahhh ahh... This is insane... That's what I have now: [100%] Linking CXX shared library luaTest1.dll LINK: command "D:\Softs\VisualStudio2017CE\VC\Tools\MSVC\14.16.27023\bin\Hostx64\x64\link.exe /nologo @CMakeFiles\luaTest1.dir\objects1.rsp /out:luaTest1.dll /implib:luaTest1.li b /pdb:D:\Projects\NervSeed\build\msvc64\sources\lua_bindings\test1\src\luaTest1.pdb /dll /version:0.0 /DEBUG /OPT:REF,NOICF /MAP /MAPINFO:EXPORTS -LIBPATH:D:\Projects\NervSeed\ scripts\..\deps\msvc64\boost_1_68_0\lib -LIBPATH:D:\Projects\NervSeed\scripts\..\deps\msvc64\LuaJIT-2.0.5\lib ..\..\..\nvCore\src\nvCore.lib ..\..\..\..\..\..\scripts\..\deps\ms vc64\LuaJIT-2.0.5\lib\lua51.lib opengl32.lib winmm.lib Imm32.lib version.lib ..\..\..\..\..\..\scripts\..\deps\msvc64\LuaJIT-2.0.5\lib\lua51.lib ..\..\..\soil2\soil2.lib kernel3 2.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST /MANIFESTFILE:luaTest1.dll.manifest" failed (exit code 11 69) with the following output: bind_std_Simple_types_double.cpp.obj : error LNK2005: "public: static struct std::_Simple_types * __cdecl nv::LunaTraits >::construct(s truct lua_State *)" (?construct@?$LunaTraits@U?$_Simple_types@N@std@@@nv@@SAPEAU?$_Simple_types@N@std@@PEAUlua_State@@@Z) déjà défini(e) dans bind_class_1.cpp.obj bind_std_Simple_types_double.cpp.obj : error LNK2005: "public: static void __cdecl nv::LunaTraits >::destruct(struct std::_Simple_types *)" (?destruct@?$LunaTraits@U?$_Simple_types@N@std@@@nv@@SAXPEAU?$_Simple_types@N@std@@@Z) déjà défini(e) dans bind_class_1.cpp.obj bind_std_allocator_double.cpp.obj : error LNK2005: "public: static class std::allocator * __cdecl nv::LunaTraits >::construct(struct lua_Sta te *)" (?construct@?$LunaTraits@V?$allocator@N@std@@@nv@@SAPEAV?$allocator@N@std@@PEAUlua_State@@@Z) déjà défini(e) dans bind_std_Rebind_alloc_t_std_allocator_double_double.cpp. obj bind_std_allocator_double.cpp.obj : error LNK2005: "public: static void __cdecl nv::LunaTraits >::destruct(class std::allocator *)" (?destru ct@?$LunaTraits@V?$allocator@N@std@@@nv@@SAXPEAV?$allocator@N@std@@@Z) déjà défini(e) dans bind_std_Rebind_alloc_t_std_allocator_double_double.cpp.obj luaTest1.dll : fatal error LNK1169: un ou plusieurs symboles définis à différentes reprises ont été rencontrés jom: D:\Projects\NervSeed\build\msvc64\sources\lua_bindings\test1\src\CMakeFiles\luaTest1.dir\build.make [sources\lua_bindings\test1\src\luaTest1.dll] Error 2 jom: D:\Projects\NervSeed\build\msvc64\CMakeFiles\Makefile2 [sources\lua_bindings\test1\src\CMakeFiles\luaTest1.dir\all] Error 1 jom: D:\Projects\NervSeed\build\msvc64\Makefile [all] Error 1 And well... hmmm... this is pretty crazy stuff at this level now: in my "bind_class_1.cpp" file I'm actually generating bindings for "std::conditional_t<1,std::_Simple_types,std::_Vec_iter_types>" But in fact, this type ends up being the same thing as just "std::_Simple_types" 🥴 Oh my... It would be very nice if we could figure this out automatically with clang, but I'm not sure this is something that can be done easily (or at all in fact ?), and I really don't need to push it that far for my initial needs here, so I'm jsut giving it a quick try here and otherwise, will keep that for later. Hmmm... interesting... here is what I have in my bind1.log file: [Debug] Creating class std::conditional_t<1,std::_Simple_types,std::_Vec_iter_types> [Debug] Creating concrete class std::conditional_t<1,std::_Simple_types,std::_Vec_iter_types> for initial type name: std::_Simple_types [Debug] Adding name std::conditional_t<1,std::_Simple_types,std::_Vec_iter_types> to type { "std::_Simple_types" } [Debug] Type already registered for 'std::_Simple_types' => So it seems I can actually already figure out that those are the same type somehow [and that's absolutely amazing if you ask me🤩!] So from there, I should really be able to avoid this error then ?! Okay, so I think I see it now: it all comes down to those lines: [Debug] Creating concrete class std::_Simple_types for initial type name: std::_Simple_types [Debug] Creating concrete class std::conditional_t<1,std::_Simple_types,std::_Vec_iter_types> for initial type name: std::_Simple_types So this means I'm creating 2 classes for a single type here: instead of always creating "concrete classes" in that case I should rather check if the type already has a target class assigned to it first. **OK**! Now I get those debug outputs: [Warning] No class template registered for 'conditional_t<1,std::_Simple_types,std::_Vec_iter_types>' creating default place holder class in namespace 'std' [Debug] Detected a type already registered for type name: std::_Simple_types adding alternative name: std::conditional_t<1,std::_Simple_types,std::_Vec_iter_types> [Debug] Adding name std::conditional_t<1,std::_Simple_types,std::_Vec_iter_types> to type { "std::_Simple_types" } [Debug] Type already registered for 'std::_Simple_types' And I have no more "bind_class_1.cpp" file at all ;-) Great! ===== Back to TplBool/TplInt compilation errors 😱 ===== It turns out that the "bind_common.h" file where I placed my "bind_test2.h" header is in fact autogenerated in the process! so my entry disappeared again, because the system considered there was no class defined in that file that was binded, to it was not necessary to include it... God damn it! Give me a break... 😫 Continuing on this path I had to also provide the binding generator with the location of the MSVC include folder to support really finding all the required include files when we automatically build concrete classes below: -- We may proceed to create the class: cl = tplNS:getOrCreateClass(clname) CHECK(cl~=nil, "Invalid class generated from template.") logDEBUG("Creating concrete class ", fname, " for initial type name: ", tname) -- @luna3: We should also set the header file for the class here using the cursor from the cxtype: cl:setHeaderFile(self:getHeaderFile(cxtype:getTypeDeclaration())) -- We assign this class as target: t:setTarget(cl) But then, this means that clang could find even more stuff to play with, and so I get errors on the type "WChar" which I didn't consider at all so far... So, I just added an entry to support that type: elseif tkind == tk.WChar then logDEBUG("Found wchar type: ", tname) local t2 = typeManager:getType("wchar_t") if t2 then logDEBUG("Using already registered type for 'wchat_t'") t = t2 t:addName(tname) end [Oh man... I have a feeling this will mean even more trouble... :-S ] Same thing then required for the NullPtr type: elseif tkind == tk.NullPtr then logDEBUG("Found nullptr type: ", tname) local t2 = typeManager:getType("nullptr_t") if t2 then logDEBUG("Using already registered type for 'nullptr_t'") t = t2 t:addName(tname) end And the "Auto" type... :-S And then I got "unions" defined in unions... :-S, so for course, also not really handled so far: [FATAL] [FATAL] Unexpect cursor type in union: UnionDecl stack traceback: [string "setup.lua"]:215: in function 'THROW' [string "setup.lua"]:210: in function 'CHECK' [string "bind.ClangParser"]:200: in function <[string "bind.ClangParser"]:198> [C]: in function 'visitChildren' [string "bind.ClangParser"]:198: in function 'parseUnion' [string "bind.ClangParser"]:1307: in function <[string "bind.ClangParser"]:1265> [C]: in function 'visitChildren' And then I got even more crashes along that path.. So not definitely don't want to go there for now! => Let's just add our missing header file manually lol. Or at least, ignore those include paths from clang side of things. => So I updated my LunaManager class to avoid parsing files further if they are part of the "ignored include paths" while still generating the header file correctly (hopefully): function Class:isLocationParseable(file) if self:isIgnoredHeaderFile(file) then return false end if self:isHeaderFile(file) or self:isInputFile(file) then return true end return false end function Class:isIgnoredHeaderFile(file) file = path.normpath(file) for _,dir in ipairs(self._config.ignoredIncludePaths or {}) do if file:startsWith(path.normpath(dir)) then return true end end return false end And I specified the MSVC include folder as ignored include folder in the binding config file: local msvcIncludeDir = "D:/Softs/VisualStudio2019CE/VC/Tools/MSVC/14.27.29110/include" cfg.ignoredIncludePaths = {msvcIncludeDir} With those changes the autogenerated bind_common.h file looks like this: #ifndef BIND_COMMON_ #define BIND_COMMON_ #if defined(_MSC_VER) || defined(__CYGWIN__) || defined(__MINGW32__) || defined( __BCPLUSPLUS__) || defined( __MWERKS__) # if defined( BIND_LIBRARY_STATIC ) # define BIND_EXPORT # elif defined( BIND_LIBRARY ) # define BIND_EXPORT __declspec(dllexport) # else # define BIND_EXPORT __declspec(dllimport) # endif #else # define BIND_EXPORT #endif //#pragma warning( disable: 4251 ) #include #include #include #include #include #include #include #include #include #include #include #include #include #include #endif Let's try to build once more with those changes... And **bingo!** Finally, this is compiling fine! Feeeeeewwww! So with this refreshing stage completed, now I can finally think about trying to build the bindings I'm after for SimCore lol. But given how long this article already is I think I should just keep that for a **part 2** blog post, and we should call it a day here. See you next time ✌!