NervLuna: A first practical usage case - Part 1
Hi readers! So here we are back on our NervLuna project for lua bindings generation… Or at least, that's what I “kind of” remember it was about ? I mean, all this stuff seems so far away already lol. But anyway, lately I've been working on lua bindings generation for one of the other professional projects I'm working on (ie. the SimCore project).
I had to get something very quickly on rails, so I immediately turned to the Sol3 library of course. And oh boy… (don't get me wrong, I really like the sol3 library, the syntax is cool, you have all the needed features and all in there, but…) this experiment definetely reminded me how complex it was to use that library properly and why I decided to start working on the nervluna project in the first place!
So here I am again: back on NervLuna, and I want to try to use it to generate the bindings I need in SimCore while those bindings are still not too big. So let's get started!
Back to the basis
Hmmm, let's be honest, I haven't been working on this project for ages… and given my really poor memory I hardly remember anything about it ah ah ah 
. The first question is thus: “Where should I start from ?” ⇒ Let me re-read the first article on this project to see if this helps refreshing my mind a bit.
Okay, so first thing I'm starting to remember here is that I was using lua to run the binding generation, given some configuration file, soo let's see if I can find again an example of how this was run in my sources… [🔎searching…] OK! So now I have something else: just found in my “dist” folder that I have a collection of lua files, and among those files I have definition of “lua apps” entry points! And a very tiny tiny voice in my head is telling me that the lua app I've need using to generate bindings is this NervBind.lua app.
So now I just need to find where I'm using this “NervBind” script filename… [🔎searching again with visual studio code search feature… but finding nothing this time] Hmmm, this would have been too easy of course. So it seems I need to put some more thinking here.
Oh! One more step forward now: I was checking the binary folders in my “dist” folder and realized there was no executable file (and not even a luajit.exe), so I tried to rebuild all my code from sources, with one of my commands that I still remember [surprising isn't it ?]:
nv_build msvc64
This is producing a great lot of errors lol. Still, eventually it will install a few files:
Install the project... -- Install configuration: "Release" -- Up-to-date: D:/Projects/NervSeed/dist/bin/msvc64/./cef_process.exe -- Up-to-date: D:/Projects/NervSeed/dist/bin/msvc64/./cef_process.pdb -- Up-to-date: D:/Projects/NervSeed/dist/bin/msvc64/./cef_process.map -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvCore.dll -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvCore.pdb -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvCore.map -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvCEF.dll -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvCEF.pdb -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvCEF.map -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvLand.dll -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvLand.pdb -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvLand.map -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvView.dll -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvView.pdb -- Installing: D:/Projects/NervSeed/dist/bin/msvc64/./nvView.map CMake Error at sources/nvVulkan/src/cmake_install.cmake:41 (file): file INSTALL cannot find "D:/Projects/NervSeed/build/msvc64/sources/nvVulkan/src/nvVulkan.dll": No error. Call Stack (most recent call first): sources/nvVulkan/cmake_install.cmake:42 (include) sources/cmake_install.cmake:49 (include) cmake_install.cmake:42 (include)
… But still no executable generated here! Yet this made me realize that the executable I'm after is my NervSeed app! And that's most probably the one I'm using to load/execute lua apps too. So my first step here should be to rebuild NervSeed correctly: let's try that.
Rebuilding NervSeed application and modules
Getting the NervSeed sources to compile was actually pretty easy: I just had to disable all the non compiling modules and stuff:
- nvVulkan: will not compile because I didn't even install the vulkan runtime yet on my computer, so this is just normal I guess.
- nvOCCT: I had some serious problems/errors while quickly trying to build the openCASCADE dependency, so I simply bypassed that for now, and of course this module will not compile anymore for now.
- lua_bindings: I had a lot of errors in the compilation here too so just starting with this disabled first.
So basically I have the CmakeList content for my sources folder:
ADD_SUBDIRECTORY(soil2) ADD_SUBDIRECTORY(libcef_dll) ADD_SUBDIRECTORY(cef_process) ADD_SUBDIRECTORY(nvCore) ADD_SUBDIRECTORY(nvCEF) ADD_SUBDIRECTORY(nvLand) # ADD_SUBDIRECTORY(nvPy) ADD_SUBDIRECTORY(nvView) # ADD_SUBDIRECTORY(nvVulkan) ADD_SUBDIRECTORY(nvDX12) ADD_SUBDIRECTORY(nvGL) # ADD_SUBDIRECTORY(nvOCCT) ADD_SUBDIRECTORY(nvLLVM) # ADD_SUBDIRECTORY(NervSeed0) ADD_SUBDIRECTORY(NervSeed) ADD_SUBDIRECTORY(llvm_syms) # ADD_SUBDIRECTORY(lua_bindings) ADD_SUBDIRECTORY(lua_lfs) # if(${FLAVOR} STREQUAL "WIN64") # ADD_SUBDIRECTORY(nvUnity) # ADD_SUBDIRECTORY(nvUnityProvider) # ADD_SUBDIRECTORY(scUnityGateway) # ENDIF()
Then from the tests/ folder I also had to disable my test_nvVulkan and test_vulkan_win tests obviously.
⇒ And then I got NervSeed rebuilt as desired! Yeepeee!
Relearning how to use NervSeed
Checking the code of the NervSeed app, the “main()” function is actually very minimal and it will delegate everything directly to a Lua environment from what I see [Yes I know… it's strange to speak this way of your own code lol], disregarding the error/exception handling parts we only have this:
LogManager::instance().setNotifyLevel(LogManager::DEBUG0);
{
// First we retrieve the NervApp instance:
auto& app = NervApp::instance();
// First we retrieve the LuaManager:
auto& lm = LuaManager::instance();
// We assign the command line arguments to the NervApp:
{
auto& state = lm.getMainState();
auto t = state["arg"].get_or_create<sol::table>();
for(U32 i=0; i<argc; ++i) {
t.set(i, argv[i]);
}
}
// std::cout << "Running app with main thread "<<std::this_thread::get_id()<<std::endl;
lm.executeFile("run.lua");
}
// Finally we destroy the app:
// std::cout<<"Destroying NervApp"<<std::endl;
NervApp::destroy();
logDEBUG2("Destroying memory manager.");
MemoryManager::destroy();
std::cout<<"Exiting."<<std::endl;
return 0;
So here we only collect all the provided arguments into the lua “arg” table, and then we call the “run.lua” script. Cool, this one, I already know where to find: in dist/assets/lua 
. Let's have a look:
logDEBUG3("Lua: this is the main run script.")
-- logDEBUG("Lua: the provided arguments are: '",arg,"'")
-- logDEBUG("Num arg: ", #arg)
-- for i=0,#arg do
-- logDEBUG("arg ",i,": ", arg[i])
-- end
-- The app name should always be the first argument we provide:
if #arg < 1 then
logERROR("No command provided.")
return
end
-- Otherwise we first check if the first argument correspond to the valid app name:
local cmd = arg[1]
table.remove(arg,1)
-- Prepare a command name mapping
local commandMap = {
vkapp = "app.VulkanApp",
sdlapp = "app.SDLApp",
bind = "app.NervBind",
bind1 = {"app.NervBind", "-c", "config/bind_test.lua"},
bind2 = {"app.NervBind", "-c", "config/bind_vulkan.lua"},
bind3 = {"app.NervBind", "-c", "config/bind_core.lua"},
-- bind3d = {"app.NervBind", "-c", "config/bind_core.lua", "--debug"},
parse1 = {"app.NervBind", "-p", "sources/lua_bindings/test1/interface/bind_test1.h"},
parse1b = {"app.NervBind", "-p", "sources/lua_bindings/test1/interface/bind_test2.h"},
parse1c = {"app.NervBind", "-p", "sources/lua_bindings/test1/interface/bind_test3.h"},
parse1d = {"app.NervBind", "-p", "sources/lua_bindings/test1/interface/bind_test4.h"},
parse1e = {"app.NervBind", "-p", "sources/lua_bindings/test1/interface/bind_test5.h"},
parse2 = {"app.NervBind", "-p", "sources/lua_bindings/wip_Vulkan/interface/vk_layer.h"},
tests = "app.Telescope",
bind0 = "app.NervBind_v0",
tsgen = "app.TSDclGen",
test1 = "test.Test1"
}
-- Check if we have a mapping for that command name:
local rep = commandMap[cmd]
if type(rep) == "string" then
cmd = rep
elseif type(rep) == "table" then
cmd = rep[1]
local newargs = {}
for i=2,#rep do
table.insert(newargs, rep[i])
end
-- Add the remaining command line args:
for i=1,#arg do
table.insert(newargs, arg[i])
end
-- Replace the command line args:
arg = newargs
end
if hasScript(cmd) then
-- We run that script providing it all the input args except thi script name itself:
local app = import(cmd)
app(arg)
return
end
logERROR("Unknown script: ", cmd)
</sxhjs>
And now, this is all starting to fall back in place: I now remember this command mapping I was using to avoid having to type long command lines all the time! So here for instance, I should be able to call `NervSeed bind1` and this should automatically call the "NervBind" app giving it the config file "config/bind_test.lua"
On top of that, now I'm also remembering I had a dedicated shell script **nv_seed** to call NervSeed from any location with additional setups, and here it is: <sxh bash; highlight: []>nv_seed()
{
local flavor="linux64"
local suffix=""
local pname=`nv_os`
case "${pname}" in
cygwin|wsl)
flavor="msvc64"
suffix=".exe"
;;
*)
export LD_LIBRARY_PATH="`nv_get_project_dir`/tools/linux/${__nv_tool_gcc}/lib64"
;;
esac
export VK_INSTANCE_LAYERS="VK_LAYER_LUNARG_standard_validation"
local tapp="`nv_get_dist_dir`/bin/$flavor/NervSeed$suffix"
$tapp "$@"
# unset LD_LIBRARY_PATH
}
⇒ So now trying the command: nv_seed bind1 [crossing fingers…] Arff, maybe this was a little bit too pretencious afterall:
Remember that I disabled the building of all the lua modules above, but the luaCode.dll is one of then, and this is definitely required in NervSeed, so we need to build at least that one ⇒ fixing this an rebuilding: OK
Trying the command nv_seed bind1 again… and sure enough we have an error on luaClang:
So I activated the build of the luaClang module and after that, it worked! Great!:
$ nv_seed bind1
[Debug] Loading dynamic library: luaClang.dll
[Debug] Parsed args: {
config = "config/bind_test.lua"
}
[Debug] Loading config file: config/bind_test.lua
[Debug] Using include path: D:\Projects\NervSeed\sources\lua_bindings\test1\interface
[Debug] Found 26 existing files.
[Debug] Using input file: D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h
[Debug] Using input file: D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test2.h
[Debug] Using input file: D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test3.h
[Debug] Using input file: D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test4.h
[Debug] Using input file: D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test5.h
[Debug] Writing file D:\Projects\NervSeed\sources\lua_bindings\test1\luna_bindings.h
[Debug] No change in file D:\Projects\NervSeed\sources\lua_bindings\test1\luna_bindings.h
[Debug] Command line args: {
"-x",
"c++",
"-std=c++17",
"-ID:\\Projects\\NervSeed\\sources\\lua_bindings\\test1\\include",
"-ID:\\Projects\\NervSeed\\sources\\lua_bindings\\test1\\interface"
}
[Debug] Parsing file: D:\Projects\NervSeed\sources\lua_bindings\test1\luna_bindings.h
[Info] Num diagnostics for file D:\Projects\NervSeed\sources\lua_bindings\test1\luna_bindings.h: 3
[Info] - diag 1: D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:185:31: warning: direct base 'nvt::OtherClass' is inaccessible due to ambiguity:
class nvt::ClassC -> class nvt::ClassB -> class nvt::OtherClass
class nvt::ClassC -> class nvt::OtherClass [-Winaccessible-base]
[Info] - diag 2: D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:232:15: warning: anonymous non-C-compatible type given name for linkage purposes by t
ypedef declaration; add a tag name here [-Wnon-c-typedef-for-linkage]
[Info] - diag 3: D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:291:15: warning: anonymous non-C-compatible type given name for linkage purposes by t
ypedef declaration; add a tag name here [-Wnon-c-typedef-for-linkage]
[Debug] Processing: 'my_enum' kind: 'EnumDecl
[Debug] Creating enum ::my_enum
[Debug] Adding enum constant: my_enum::ENUM_VALUE1
[Debug] Adding enum constant: my_enum::ENUM_VALUE2
[Debug] Adding enum constant: my_enum::ENUM_VALUE3
(... much more outputs here ... )
[Debug] Writing file D:\Projects\NervSeed\sources\lua_bindings\test1\src/luna/bind_std_string.cpp
[Debug] No change in file D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_std_string.cpp
[Debug] Writing class binding for std::basic_ostream<char,std::char_traits<char>>
[Debug] Preprocessing function list...
[Debug] Keeping 0 on 0
[Debug] Writing function type checks...
[Debug] Writing function bindings...
[Debug] Done writing function bindings.
[Debug] Preprocessing field list...
[Debug] Keeping 0/0 fields.
[Debug] Writing fields type checks...
[Debug] Writing fields bindings...
[Debug] Done writing fields bindings.
[Debug] Preprocessing field list...
[Debug] Keeping 0/0 fields.
[Debug] Writing fields type checks...
[Debug] Writing fields bindings...
[Debug] Done writing fields bindings.
[Debug] Writing file D:\Projects\NervSeed\sources\lua_bindings\test1\src/luna/bind_std_basic_ostream_char_std_char_traits_char.cpp
[Debug] Updating content of D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_std_basic_ostream_char_std_char_traits_char.cpp
[Debug] Writing Package bindings...
[Debug] Writing file D:\Projects\NervSeed\sources\lua_bindings\test1\src/luna/register_package.cpp
[Debug] Updating content of D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\register_package.cpp
[Debug] Done writing bindings.
[Debug] Searching for obsolete files...
[Debug] Removing obsolete/unused file D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_std_ostream.cpp
[Debug] Done searching for obsolete files.
[Debug] Uninitializing Clang parser.
[Debug] Disposing cidx
[Debug] Done uninitializing Clang parser.
Deleted LogManager object.
Looking for memory leaks...
No memory leak detected.
Exiting.
Investigating cursor visitor exception
The previous call of 'nv_seed bind1' seems to have been mostly working fine, but checking the debug outputs more carefully, I eventually noticed this exception somewhere in the middle of the processing:
[Warning] No class template registered for 'MyTemplateTest<1>' creating default place holder class in namespace 'nvt'
[Debug] Creating class nvt::MyTemplateTest<1>
[Debug] Creating concrete class nvt::MyTemplateTest<1> for initial type name: nvt::MyTemplateTest<true>
[Error] Error in cursorVisitor: [string "bind.ClangParser"]:366: attempt to index local 't' (a nil value)
stack traceback:
[string "bind.ClangParser"]:366: in function 'parseTemplateType'
[string "bind.ClangParser"]:1121: in function 'parseClass'
[string "bind.ClangParser"]:1381: in function <[string "bind.ClangParser"]:1352>
[C]: in function 'visitChildren'
[string "bind.ClangParser"]:1352: in function 'parseNamespace'
[string "bind.ClangParser"]:1379: in function <[string "bind.ClangParser"]:1352>
[C]: in function 'visitChildren'
[string "bind.ClangParser"]:1352: in function 'parseNamespace'
[string "bind.ClangParser"]:1418: in function 'parseFile'
[string "app.NervBind"]:137: in function 'parseInputs'
[string "app.NervBind"]:154: in function 'run'
[string "app.AppBase"]:23: in function '__init'
[string "app.NervBind"]:12: in function '__init'
[string "setup.lua"]:155: in function 'app'
[string "run.lua"]:63: in main chunk
[Debug] clang done parsing file: D:\Projects\NervSeed\sources\lua_bindings\test1\luna_bindings.h
[Debug] Resolving target types...
So this error occurs in parseTemplateType(), but from what I see that would be because the type parameter “t” that we receive in that function call is already nil: and this should come from the parseClass calls just before, so let's check that one.
arrf, and indeed in the parseClass() function we have this:
logDEBUG("Creating template specialization: ", tname)
t = self:parseTemplateType(nil, cxtype, tname)
CHECK(t, "Cannot parse template type: ",tname)</sxhjs>
=> So indeed the first parameter (ie. "t") is nil in that case :-S. And right now I'm starting to have an hard time understanding what this all means exactly lol.
Anyway, just noticed that at other locations I will create "Type" object when required, so let's do that here too: <sxhjs lua> -- @luna2: If no type is provided already then we create one here with tname:
if t == nil then
t = Type(tname)
end</sxhjs>
=> And with that change the binding code generation seems to work fine without errors! **OK**
===== Invalid anonymous union names issue =====
Before trying to build our "newly" generated bindings for this test1 project, I thought I should at least have a quick look at the generated code... just in case... And of course, here is what I found in the first file I opened: <sxh cpp; highlight: []>// Typecheck for (anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) (1) with signature: void ()
static bool _check__anonymous_union_at_D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5_sig1(lua_State* L) {
int luatop = lua_gettop(L);
if( luatop!=0 ) return false;
return true;
}
// Function bindings
// Bind for (anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) constructor (1) with signature: void ()
static nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)* _bind__anonymous_union_at_D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5_sig1(lua_State* L) {
auto res = luna_callDefaultConstructor<nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)>(L);
if(res == nullptr) luaL_error(L, "Cannot default construct instance of class nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5).");
return res;
}
… 🤢… Yeahh… I always feel the same with those kind of errors: “How on earth could this happen, in something that was working OK so far and that I didn't touch for months ?!”. But eventually you'll realize it is just always like this, “bugs” just appear out of nowhere! 🥳
Anyway, let's now try to investigate where this could come from. Point is, I'm pretty sure I had something in place already to handle (ie. rename) some “anonymous things”, but maybe this is currently not applied on unions ?
So the problem seems to come from this code section (inside my StrutB structure definition):
union {
int intValue;
bool boolValue;
} my_anon_union;
And now I'm thinking this union finding part could be something new that was not happening at all before. Time to gather some debug inputs from clang on our little bind_test1.h file with:
nv_seed parse1 | tee parse1.log
So this gives us the following outputs:
[Debug] level 3: '' of kind 'UnionDecl', type: 'nvt::StructB::(anonymous union at sources/lua_bindings/test1/interface/bind_test1.h:217:5)' WARN: clang_getTypeFullSpelling: no decl found for int [Debug] level 4: 'intValue' of kind 'FieldDecl', type: 'int' WARN: clang_getTypeFullSpelling: no decl found for bool [Debug] level 4: 'boolValue' of kind 'FieldDecl', type: 'bool' [Debug] level 3: 'my_anon_union' of kind 'FieldDecl', type: 'nvt::StructB::(anonymous union at sources/lua_bindings/test1/interface/bind_test1.h:217:5)' [Debug] level 4: '' of kind 'UnionDecl', type: 'nvt::StructB::(anonymous union at sources/lua_bindings/test1/interface/bind_test1.h:217:5)' WARN: clang_getTypeFullSpelling: no decl found for int [Debug] level 5: 'intValue' of kind 'FieldDecl', type: 'int' WARN: clang_getTypeFullSpelling: no decl found for bool [Debug] level 5: 'boolValue' of kind 'FieldDecl', type: 'bool'
Right now in ClangParser, when I find a “UnionDecl”, I simply write a new class for it apparently:
elseif ckind == clang.CursorKind.ClassDecl or ckind == clang.CursorKind.StructDecl or ckind == clang.CursorKind.UnionDecl then
obj = self:parseClass(cur, scope, cname)</sxhjs>
This can make sense to some extents because we have "fields" defined in this union, but then if the union is anonymous (and that's most cases I believe ?) it doesn't make sense to write class bindings for it. So I need to deal with that first point.
=> So as a first step I decided I should introduce a new entity type for "UNION", and create a "Union" class as well as an "UnionHolder" as below:
<sxhjs lua>local Class = createClass{name="Union", bases={"reflection.Scope", "reflection.FieldHolder"}}
local luna = import "bind.LunaManager"
function Class:__init(name, parent)
Class.__initBases(self, name, parent)
self:setEntityType(self.EntityType.UNION)
end
function Class:isAnonymous()
return self._name:startsWith(luna.anonymous_union_prefix)
end
function Class:isBindable()
return not self:isIgnored()
end
return Class</sxhjs>
<sxhjs lua>local Class = createClass{name="UnionHolder", bases="base.Object"}
local Vis = import "reflection.Visibility"
function Class:__init(name, parent)
Class.super.__init(self)
self._children.unions = {}
end
function Class:getOrCreateUnion(name)
CHECK(name~="", "Invalid union name.")
for _,obj in ipairs(self._children.unions) do
if obj:getName() == name then
return obj
end
end
-- We should create this child namespace:
logDEBUG("Creating enum ", self:getFullName(), "::", name)
local Union = import "reflection.Union"
local obj = Union(name, self)
table.insert(self._children.unions, obj)
return obj
end
-- Retrieve all the public unions
function Class:getPublicUnions(tt)
tt = tt or {}
for _,el in ipairs(self._children.unions) do
if el:getVisibility() == Vis.PUBLIC then
table.insert(tt, el)
end
end
return tt
end
return Class
</sxhjs>
<note important>The Union class is also a "FieldHolder" so that I can then add fields like "int intValue" or "bool boolValue" inside the union definition.</note>
Then I simply added the UnionHolder base for my "Class" and "Namesapce" classes. And I updated the handling of Unions declarations to be similar to what I was doing for enums: <sxhjs lua> elseif ckind == clang.CursorKind.UnionDecl then
-- @luna3: We can also have anonymous unions in fact:
if cname=="" then
cname = luna:getNextName(luna.anonymous_union_prefix)
logDEBUG("Registering anonymous union '", cname, "' with type ", typename)
end
-- We add that enum to the current scope:
obj = scope:getOrCreateUnion(cname)
self:parseUnion(cur, obj)</sxhjs>
And I built the required dedicated function to parse unions of course: <sxhjs lua>function Class:parseUnion(cursor, union)
if union:isParsed() then
logDEBUG2("Union ", union:getFullName()," is already parsed.")
return;
end
if not cursor:isDefinition() then
-- Nothing more to do here, we won't get any definition from this cursor.
return
end
-- Keep going, parsing the class:
union:setParsed(true)
-- We retrieve all the values inside that enum:
cursor:visitChildren(function(cur, parent)
local ckind = cur:getKind()
CHECK(ckind == clang.CursorKind.FieldDecl, "Unexpect cursor type in union: ", cursor:getKindSpelling())
local cname = cur:getSpelling()
obj = union:getOrCreateField(cname)
self:parseField(cur, obj)
return clang.VisitResult.Continue
end)
end</sxhjs>
... So now let's see how lucky/unlucky I can be with those changes... (=> building again with "nv_seed bind1 | tee bind1.log") And okay, now the processing gives us: <code>[Debug] Registering anonymous union 'luna_anonymous_union_2' with type nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)
[Debug] Creating enum nvt::StructB::luna_anonymous_union_2
[Debug] Creating field nvt::StructB::luna_anonymous_union_2::intValue
WARN: clang_getTypeFullSpelling: no decl found for int
[Debug] Creating field nvt::StructB::luna_anonymous_union_2::boolValue
WARN: clang_getTypeFullSpelling: no decl found for bool
[Debug] Creating field nvt::StructB::my_anon_union
[Debug] Resolving type nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)
[Debug] Type kind: Elaborated
[Debug] Adding type with class target name: nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5), (canonical type: nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5))
[Debug] TypeManager: Registering type {
"nvt::StructB::(anonymous union at D:\\Projects\\NervSeed\\sources\\lua_bindings\\test1\\interface\\bind_test1.h:217:5)"
}</code>
But I still have (some of) those crazy bindings generated ggrrrrrr!! (yet the "typecheck/function bindings" previously reported are now gone): <sxh cpp; highlight: []>nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)* LunaTraits< nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) >::construct(lua_State* L) {
luaL_error(L, "No public constructor available for class nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)");
return nullptr;
}
void LunaTraits< nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) >::destruct(nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)* obj) {
THROW_MSG("No public destructor available for class nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)");
}
The actual problem comes from the type resolution at the end of the binding generation process:
[Debug] Resolving target types... [Debug] Resolving target for type: nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) [Debug] Couldn't resolve type target, using target name instead: nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5) [Debug] Creating class nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)
So this is now given us anything meaningfull (but that's expected), and then we still write a class binding for that type:
[Debug] Writing class binding for nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)
That part we should really avoid somehow: if we have some kind of “anonymous type” then we should not try to write a class binding for it at all.
⇒ So now I have updated my “final type resolution” process to catch those kind of invalid anonymous names and avoid creating default class binding for them:
if tgtType then
local tname = type:getName()
logDEBUG("Resolving target for type: ", tname)
count = count + 1
local cl = luna:getClass(tgtType)
if not cl then
tgtName = tgtType:getFullName()
logDEBUG("Couldn't resolve type target, using target name instead: ", tgtName)
-- Once we have a type name then we split it in parts:
local parts = utils.splitEntityName(tgtName)
local ns = luna:getRootNamespace()
local nparts = #parts
for i=1,nparts-1 do
if ns:hasClass(parts[i]) then
ns = ns:getOrCreateClass(parts[i])
else
ns = ns:getOrCreateNamespace(parts[i])
end
end
-- Then we create our target class:
local clname = parts[nparts]
-- @luna3: Check if that class name only contains valid characters, otherwise we should ignore it:
if self:isValidTypeName(clname) then
cl = ns:getOrCreateClass(clname)
else
logDEBUG("Ignoring unresolvable/invalid type with resulting name: ", clname)
end
end
type:setTarget(cl)
end</sxhjs>
With the helper function: <sxhjs lua>function Class:isValidTypeName(tname)
-- Chekc if the provided typename is valid:
if tname:find("anonymous enum at") then
logDEBUG("Detected anonymous enum type name in ", tname)
return false
elseif tname:find("anonymous union at") then
logDEBUG("Detected anonymous union type name in ", tname)
return false
end
return true
end</sxhjs>
And finally those invalid binding names are gone! Ouuff! : <code>[Debug] Resolving target types...
[Debug] Resolving target for type: nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)
[Debug] Couldn't resolve type target, using target name instead: nvt::StructB::(anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)
[Debug] Detected anonymous union type name in (anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)
[Debug] Ignoring unresolvable/invalid type with resulting name: (anonymous union at D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test1.h:217:5)
[Debug] Done resolving 1 target types.</code>
<note important>Note that here we only really handled "avoiding" writing garbage when unions are found, but in the end, we still have no **support at all** to generate usable bindings for union variables. This is some I will have to deal with eventually.</note>
===== _bind_MyTemplateTest compilation error =====
And now of course you guess it, since the generated binding files were looking OK I moved to the compilation stage of the test1 module. And I got errors... 😭 The first one being: <code>[ 90%] Building CXX object sources/lua_bindings/test1/src/CMakeFiles/luaTest1.dir/luna/bind_nvt_MyTemplateTest_1.cpp.obj
bind_nvt_MyStringTest.cpp
[ 90%] Building CXX object sources/lua_bindings/test1/src/CMakeFiles/luaTest1.dir/luna/bind_nvt_MyVectorNameTest.cpp.obj
bind_nvt_MyTemplateTest_1.cpp
D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_nvt_MyTemplateTest_1.cpp(49): error C2065: '_bind_MyTemplateTest' : identificateur non déclaré
bind_nvt_MyVectorNameTest.cpp
jom: D:\Projects\NervSeed\build\msvc64\sources\lua_bindings\test1\src\CMakeFiles\luaTest1.dir\build.make [sources\lua_bindings\test1\src\CMakeFiles\luaTest1.dir\luna\bind_nvt_My
TemplateTest_1.cpp.obj] Error 2
</code>
Having a look at that binding file we have indeed: <sxh cpp; highlight: [11]>// Overall bind for MyTemplateTest<1>
static nvt::MyTemplateTest<1>* _bind_MyTemplateTest_1(lua_State* L) {
if(_check_MyTemplateTest_1_sig1(L)) return _bind_MyTemplateTest_1_sig1(L);
logERROR("Current lua stack: "<<luna_dumpStack(L));
luaL_error(L, "Binding error for function MyTemplateTest<1>, cannot match any of the 1 signature(s):\n sig1: void ()");
return nullptr;
}
nvt::MyTemplateTest<1>* LunaTraits< nvt::MyTemplateTest<1> >::construct(lua_State* L) {
return _bind_MyTemplateTest<1>(L);
}
⇒ So it's cool: this one sounds simple to fix: we are already generating a construction function name correctly (on line 2 above), but then when we call it on line 11 it seems I'm not sanitizing the function name correctly. Let's fix that: 🆗!
bind_nvt_TplBool.cpp compilation error
Next compilation errors on the list are:
bind_nvt_TplBool.cpp D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_nvt_TplBool.cpp(23): error C2039: 'TplBool' : n'est pas membre de 'nvt' D:\Projects\NervSeed\sources\lua_bindings\test1\interface\bind_test5.h(9): note: voir la déclaration de 'nvt' D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_nvt_TplBool.cpp(23): error C2143: erreur de syntaxe : absence de ';' avant '*' D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_nvt_TplBool.cpp(23): error C4430: spécificateur de type manquant - int est pris en compte par défaut. Remarque : C+ + ne prend pas en charge int par défaut D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_nvt_TplBool.cpp(23): error C2923: 'nv::LunaTraits' : 'nv::TplBool' n'est pas un argument de type modèle valide pour le paramètre 'T' D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_nvt_TplBool.cpp(23): note: voir la déclaration de 'nv::TplBool' D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_nvt_TplBool.cpp(23): error C2955: 'nv::LunaTraits' : l'utilisation de classe modèle requiert une liste d'arguments modèle D:\Projects\NervSeed\sources\nvCore\include\lua/luna.h(157): note: voir la déclaration de 'nv::LunaTraits' D:\Projects\NervSeed\sources\lua_bindings\test1\src\luna\bind_nvt_TplBool.cpp(23): error C2761: 'T *nv::LunaTraits<T>::construct(lua_State *)' : la redéclaration du membre n'est pas autorisée
Now, looking at the binding file, the content looks OK. Ahh! but now I get it: the TplBool (and TplInt) types are defined in bind_test2.h:
typedef MyTemplateClass<int> TplInt; typedef MyTemplateClass<bool> TplBool;
But the “bind_test2.h” was not included anywhere in the actuall build process since I was only using this global header:
#ifndef BIND_COMMON_ #define BIND_COMMON_ #if defined(_MSC_VER) || defined(__CYGWIN__) || defined(__MINGW32__) || defined( __BCPLUSPLUS__) || defined( __MWERKS__) # if defined( BIND_LIBRARY_STATIC ) # define BIND_EXPORT # elif defined( BIND_LIBRARY ) # define BIND_EXPORT __declspec(dllexport) # else # define BIND_EXPORT __declspec(dllimport) # endif #else # define BIND_EXPORT #endif //#pragma warning( disable: 4251 ) #include <core_lua.h> #include <lua/luna.h> #include <bind_context.h> #include <bind_test1.h> #include <bind_test3.h> #include <bind_test4.h> #include <bind_test5.h> #endif
So just adding “#include <bind_test2.h>” there, and then we were good 🤟!
std::_Simple_types<double> link error
So, after the compilation stage, more errors at link stage! mouaahh ahhh ahh… This is insane… That's what I have now:
[100%] Linking CXX shared library luaTest1.dll LINK: command "D:\Softs\VisualStudio2017CE\VC\Tools\MSVC\14.16.27023\bin\Hostx64\x64\link.exe /nologo @CMakeFiles\luaTest1.dir\objects1.rsp /out:luaTest1.dll /implib:luaTest1.li b /pdb:D:\Projects\NervSeed\build\msvc64\sources\lua_bindings\test1\src\luaTest1.pdb /dll /version:0.0 /DEBUG /OPT:REF,NOICF /MAP /MAPINFO:EXPORTS -LIBPATH:D:\Projects\NervSeed\ scripts\..\deps\msvc64\boost_1_68_0\lib -LIBPATH:D:\Projects\NervSeed\scripts\..\deps\msvc64\LuaJIT-2.0.5\lib ..\..\..\nvCore\src\nvCore.lib ..\..\..\..\..\..\scripts\..\deps\ms vc64\LuaJIT-2.0.5\lib\lua51.lib opengl32.lib winmm.lib Imm32.lib version.lib ..\..\..\..\..\..\scripts\..\deps\msvc64\LuaJIT-2.0.5\lib\lua51.lib ..\..\..\soil2\soil2.lib kernel3 2.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST /MANIFESTFILE:luaTest1.dll.manifest" failed (exit code 11 69) with the following output: bind_std_Simple_types_double.cpp.obj : error LNK2005: "public: static struct std::_Simple_types<double> * __cdecl nv::LunaTraits<struct std::_Simple_types<double> >::construct(s truct lua_State *)" (?construct@?$LunaTraits@U?$_Simple_types@N@std@@@nv@@SAPEAU?$_Simple_types@N@std@@PEAUlua_State@@@Z) déjà défini(e) dans bind_class_1.cpp.obj bind_std_Simple_types_double.cpp.obj : error LNK2005: "public: static void __cdecl nv::LunaTraits<struct std::_Simple_types<double> >::destruct(struct std::_Simple_types<double> *)" (?destruct@?$LunaTraits@U?$_Simple_types@N@std@@@nv@@SAXPEAU?$_Simple_types@N@std@@@Z) déjà défini(e) dans bind_class_1.cpp.obj bind_std_allocator_double.cpp.obj : error LNK2005: "public: static class std::allocator<double> * __cdecl nv::LunaTraits<class std::allocator<double> >::construct(struct lua_Sta te *)" (?construct@?$LunaTraits@V?$allocator@N@std@@@nv@@SAPEAV?$allocator@N@std@@PEAUlua_State@@@Z) déjà défini(e) dans bind_std_Rebind_alloc_t_std_allocator_double_double.cpp. obj bind_std_allocator_double.cpp.obj : error LNK2005: "public: static void __cdecl nv::LunaTraits<class std::allocator<double> >::destruct(class std::allocator<double> *)" (?destru ct@?$LunaTraits@V?$allocator@N@std@@@nv@@SAXPEAV?$allocator@N@std@@@Z) déjà défini(e) dans bind_std_Rebind_alloc_t_std_allocator_double_double.cpp.obj luaTest1.dll : fatal error LNK1169: un ou plusieurs symboles définis à différentes reprises ont été rencontrés jom: D:\Projects\NervSeed\build\msvc64\sources\lua_bindings\test1\src\CMakeFiles\luaTest1.dir\build.make [sources\lua_bindings\test1\src\luaTest1.dll] Error 2 jom: D:\Projects\NervSeed\build\msvc64\CMakeFiles\Makefile2 [sources\lua_bindings\test1\src\CMakeFiles\luaTest1.dir\all] Error 1 jom: D:\Projects\NervSeed\build\msvc64\Makefile [all] Error 1
And well… hmmm… this is pretty crazy stuff at this level now: in my “bind_class_1.cpp” file I'm actually generating bindings for “std::conditional_t<1,std::_Simple_types<double>,std::_Vec_iter_types<double,size_t,int64_t,double*,const double*,double&,const double&>>”
But in fact, this type ends up being the same thing as just “std::_Simple_types<double>” 🥴 Oh my…
It would be very nice if we could figure this out automatically with clang, but I'm not sure this is something that can be done easily (or at all in fact ?), and I really don't need to push it that far for my initial needs here, so I'm jsut giving it a quick try here and otherwise, will keep that for later.
Hmmm… interesting… here is what I have in my bind1.log file:
[Debug] Creating class std::conditional_t<1,std::_Simple_types<double>,std::_Vec_iter_types<double,size_t,int64_t,double*,const double*,double&,const double&>>
[Debug] Creating concrete class std::conditional_t<1,std::_Simple_types<double>,std::_Vec_iter_types<double,size_t,int64_t,double*,const double*,double&,const double&>> for initial type name: std::_Simple_types<double>
[Debug] Adding name std::conditional_t<1,std::_Simple_types<double>,std::_Vec_iter_types<double,size_t,int64_t,double*,const double*,double&,const double&>> to type {
"std::_Simple_types<double>"
}
[Debug] Type already registered for 'std::_Simple_types<double>'
⇒ So it seems I can actually already figure out that those are the same type somehow [and that's absolutely amazing if you ask me🤩!] So from there, I should really be able to avoid this error then ?!
Okay, so I think I see it now: it all comes down to those lines:
[Debug] Creating concrete class std::_Simple_types<double> for initial type name: std::_Simple_types<double> [Debug] Creating concrete class std::conditional_t<1,std::_Simple_types<double>,std::_Vec_iter_types<double,size_t,int64_t,double*,const double*,double&,const double&>> for initial type name: std::_Simple_types<double>
So this means I'm creating 2 classes for a single type here: instead of always creating “concrete classes” in that case I should rather check if the type already has a target class assigned to it first. OK! Now I get those debug outputs:
[Warning] No class template registered for 'conditional_t<1,std::_Simple_types<double>,std::_Vec_iter_types<double,size_t,int64_t,double*,const double*,double&,const double&>>' creating default place holder class in namespace 'std'
[Debug] Detected a type already registered for type name: std::_Simple_types<double> adding alternative name: std::conditional_t<1,std::_Simple_types<double>,std::_Vec_iter_types<double,size_t,int64_t,double*,const double*,double&,const double&>>
[Debug] Adding name std::conditional_t<1,std::_Simple_types<double>,std::_Vec_iter_types<double,size_t,int64_t,double*,const double*,double&,const double&>> to type {
"std::_Simple_types<double>"
}
[Debug] Type already registered for 'std::_Simple_types<double>'
And I have no more “bind_class_1.cpp” file at all 
Great!
Back to TplBool/TplInt compilation errors 😱
It turns out that the “bind_common.h” file where I placed my “bind_test2.h” header is in fact autogenerated in the process! so my entry disappeared again, because the system considered there was no class defined in that file that was binded, to it was not necessary to include it… God damn it! Give me a break… 😫
Continuing on this path I had to also provide the binding generator with the location of the MSVC include folder to support really finding all the required include files when we automatically build concrete classes below:
-- We may proceed to create the class:
cl = tplNS:getOrCreateClass(clname)
CHECK(cl~=nil, "Invalid class generated from template.")
logDEBUG("Creating concrete class ", fname, " for initial type name: ", tname)
-- @luna3: We should also set the header file for the class here using the cursor from the cxtype:
cl:setHeaderFile(self:getHeaderFile(cxtype:getTypeDeclaration()))
-- We assign this class as target:
t:setTarget(cl)</sxhjs>
But then, this means that clang could find even more stuff to play with, and so I get errors on the type "WChar" which I didn't consider at all so far... So, I just added an entry to support that type: <sxhjs lua> elseif tkind == tk.WChar then
logDEBUG("Found wchar type: ", tname)
local t2 = typeManager:getType("wchar_t")
if t2 then
logDEBUG("Using already registered type for 'wchat_t'")
t = t2
t:addName(tname)
end</sxhjs>
[Oh man... I have a feeling this will mean even more trouble... :-S ]
Same thing then required for the NullPtr type: <sxhjs lua> elseif tkind == tk.NullPtr then
logDEBUG("Found nullptr type: ", tname)
local t2 = typeManager:getType("nullptr_t")
if t2 then
logDEBUG("Using already registered type for 'nullptr_t'")
t = t2
t:addName(tname)
end</sxhjs>
And the "Auto" type... :-S
And then I got "unions" defined in unions... :-S, so for course, also not really handled so far: <code>[FATAL] [FATAL] Unexpect cursor type in union: UnionDecl
stack traceback:
[string "setup.lua"]:215: in function 'THROW'
[string "setup.lua"]:210: in function 'CHECK'
[string "bind.ClangParser"]:200: in function <[string "bind.ClangParser"]:198>
[C]: in function 'visitChildren'
[string "bind.ClangParser"]:198: in function 'parseUnion'
[string "bind.ClangParser"]:1307: in function <[string "bind.ClangParser"]:1265>
[C]: in function 'visitChildren'</code>
And then I got even more crashes along that path.. So not definitely don't want to go there for now!
=> Let's just add our missing header file manually lol. Or at least, ignore those include paths from clang side of things. => So I updated my LunaManager class to avoid parsing files further if they are part of the "ignored include paths" while still generating the header file correctly (hopefully): <sxhjs lua>function Class:isLocationParseable(file)
if self:isIgnoredHeaderFile(file) then
return false
end
if self:isHeaderFile(file) or self:isInputFile(file) then
return true
end
return false
end
function Class:isIgnoredHeaderFile(file)
file = path.normpath(file)
for _,dir in ipairs(self._config.ignoredIncludePaths or {}) do
if file:startsWith(path.normpath(dir)) then
return true
end
end
return false
end</sxhjs>
And I specified the MSVC include folder as ignored include folder in the binding config file: <sxhjs lua>local msvcIncludeDir = "D:/Softs/VisualStudio2019CE/VC/Tools/MSVC/14.27.29110/include"
cfg.ignoredIncludePaths = {msvcIncludeDir}
</sxhjs>
With those changes the autogenerated bind_common.h file looks like this: <sxh cpp; highlight: []>#ifndef BIND_COMMON_
#define BIND_COMMON_
#if defined(_MSC_VER) || defined(__CYGWIN__) || defined(__MINGW32__) || defined( __BCPLUSPLUS__) || defined( __MWERKS__)
# if defined( BIND_LIBRARY_STATIC )
# define BIND_EXPORT
# elif defined( BIND_LIBRARY )
# define BIND_EXPORT __declspec(dllexport)
# else
# define BIND_EXPORT __declspec(dllimport)
# endif
#else
# define BIND_EXPORT
#endif
//#pragma warning( disable: 4251 )
#include <core_lua.h>
#include <lua/luna.h>
#include <bind_context.h>
#include <bind_test1.h>
#include <bind_test2.h>
#include <bind_test3.h>
#include <bind_test4.h>
#include <bind_test5.h>
#include <xstring>
#include <xmemory>
#include <iosfwd>
#include <vector>
#include <initializer_list>
#include <map>
#endif
Let's try to build once more with those changes… And bingo! Finally, this is compiling fine! Feeeeeewwww!
So with this refreshing stage completed, now I can finally think about trying to build the bindings I'm after for SimCore lol. But given how long this article already is I think I should just keep that for a part 2 blog post, and we should call it a day here. See you next time ✌!