====== NervLuna: generating bindings for nvCore - second run ======

{{tag>cpp dev nvluna bindings}}

So here we are again, with yet another dev session to try to get the lua bindings for my nvCore module to finally compile properly. On the table today: abstract classes, unnamed arguments, already defined functions and finally link errors: "Bon appétit!" LOL. But hey, the issues are not going to simply automatically fix themself, so no choice here: we have to get to work ;-)!

====== ======

===== No construction for abstract class =====

As discussed in [[blog:2020:0618_nervluna_operators_handling|my previous article]] the next issue on the list is on the generation of a constructor for abstract class. It turns out this is relatively easy to fix: from libclang we have the function **clang_CXXRecord_isAbstract** which can be used to figure out if a given class/struct cursor is abstract, so I just need to extend a bit the mechanism already in place.

I add in my "Class" class:

<sxhjs lua>
function Class:isAbstract()
  return self._isAbstract
end

function Class:setAbstract(val)
  if val then
    logDEBUG("Detected abstract class: ", self:getFullName())
  end
  self._isAbstract = val
end
</sxhjs>

Then I update that "abstract" flag when parsing a given class. And finally, if I notice a class is abstract when writing its binding file, I just pretend we don't have any public constructor available (and that will do the trick: users won't be able to contruct that class from lua and the generated code will compile):

<sxhjs lua>
  local cons = cl:getConstructor()
  local pubCtorSigs = cons and cons:getPublicSignatures(true) or {}
  if cl:isAbstract() then
    logDEBUG("Disallowing construction of abstract class ", fname)
    pubCtorSigs = {}
  end
</sxhjs>

And finally we also need to prevent the generation of the contructor function binding at the FunctionWriter level with:

<sxhjs lua>
    elseif f:isConstructor() and f:getParent():isAbstract() then
      -- Should not provide any bindings for abstract classes constructor:
      logDEBUG("Ignoring abstract class constructor: ", fname)
      self:writeLine("// NervLuna: Ignoring abstract class constructor '%s'", fname)
</sxhjs>

===== Unnamed arguments issue =====

With the abstract class issue out of the way, the next compilation error I face is in the bindings generated for the matrice classes I have:

<sxh cpp; highlight: [7,9]>
// Bind for makeScale (1) with signature: void (const nv::Vec3f &)
static int _bind_makeScale_sig1(lua_State* L) {
	// When reaching this call, we assume that the type checking is already done.
	nv::Matrixd* self = Luna< nv::Matrixd >::get(L,1);
	ASSERT(self!=nullptr);

	nv::Vec3f*  = Luna< nv::Vec3f >::get(L,2,false);

	self->makeScale(*);

	return 0;
}
</sxh>

And well, that's the thing: there is just no name provided for the arguments in that case, since we have the functions declaration:

<sxh cpp>
  void makeScale( const nv::Vec3f& );
  void makeScale( const Vec3d& );
  void makeScale( value_type, value_type, value_type );

  void makeTranslate( const nv::Vec3f& );
  void makeTranslate( const Vec3d& );
  void makeTranslate( value_type, value_type, value_type );
</sxh>

=> So, when parsing the function arguments, we should provide default names in case none is provided:

<sxhjs lua>  
  for i=0,num-1 do
    local arg = cursor:getArgument(i)
    local argname = arg:getSpelling()
    -- #112: use a default argument name if none is provided.
    if argname == "" then
      argname = "arg"..i
    end
    logDEBUG("Arg ",i," cursor: ", argname,", type: ", arg:getType():getSpelling())
    -- For each argument we assign the current cursor context:
    self:setContextCursor(paramDecls[i+1])
    sig:addArgument(self:resolveArgument(arg))
  end</sxhjs>

And now the results are much better :-) :

<sxh cpp>
// Bind for makeScale (1) with signature: void (const nv::Vec3f &)
static int _bind_makeScale_sig1(lua_State* L) {
	// When reaching this call, we assume that the type checking is already done.
	nv::Matrixd* self = Luna< nv::Matrixd >::get(L,1);
	ASSERT(self!=nullptr);

	nv::Vec3f* arg0 = Luna< nv::Vec3f >::get(L,2,false);

	self->makeScale(*arg0);

	return 0;
}
</sxh>

Good!

===== Function already defined issue =====

And now it seems the last item on the table for this session will be with multiple definitions of the same operator in the global functions list:

<code>W:\Projects\NervSeed\sources\lua_bindings\wip_Core\src\luna\register_functions.cpp(679): error C2084: function 'bool nv::_check_op_lshift_sig1(lua_State *)' already has a body
W:\Projects\NervSeed\sources\lua_bindings\wip_Core\src\luna\register_functions.cpp(28): note: see previous definition of '_check_op_lshift_sig1'
W:\Projects\NervSeed\sources\lua_bindings\wip_Core\src\luna\register_functions.cpp(765): error C2065: '_check_op_lshift_sig1': undeclared identifier
W:\Projects\NervSeed\sources\lua_bindings\wip_Core\src\luna\register_functions.cpp(1944): error C2084: function 'int nv::_bind_op_lshift_sig1(lua_State *)' already has a body
W:\Projects\NervSeed\sources\lua_bindings\wip_Core\src\luna\register_functions.cpp(752): note: see previous definition of '_bind_op_lshift_sig1'
W:\Projects\NervSeed\sources\lua_bindings\wip_Core\src\luna\register_functions.cpp(2028): error C2084: function 'int nv::_bind_op_lshift(lua_State *)' already has a body
W:\Projects\NervSeed\sources\lua_bindings\wip_Core\src\luna\register_functions.cpp(764): note: see previous definition of '_bind_op_lshift'
W:\Projects\NervSeed\sources\lua_bindings\wip_Core\src\luna\register_functions.cpp(2029): error C2065: '_check_op_lshift_sig1': undeclared identifier
W:\Projects\NervSeed\sources\lua_bindings\wip_Core\src\luna\register_functions.cpp(2029): error C2065: '_bind_op_lshift_sig1': undeclared identifier</code>

Checking the generated code here is what I found:

<sxh cpp>
// Typecheck for operator<< (1) with signature: std::ostream &(std::ostream &, const nv::Quat &)
static bool _check_op_lshift_sig1(lua_State* L) {
	int luatop = lua_gettop(L);
	if( luatop!=2 ) return false;

	if( !luna_isInstanceOf(L,1,SID("std::ostream"),false) ) return false;
	if( !luna_isInstanceOf(L,2,SID("nv::Quat"),false) ) return false;
	return true;
}

// Typecheck for operator<< (1) with signature: std::ostream &(std::ostream &, const nv::Matrixd &)
static bool _check_op_lshift_sig1(lua_State* L) {
	int luatop = lua_gettop(L);
	if( luatop!=2 ) return false;

	if( !luna_isInstanceOf(L,1,SID("std::ostream"),false) ) return false;
	if( !luna_isInstanceOf(L,2,SID("nv::Matrixd"),false) ) return false;
	return true;
}

// Typecheck for operator<< (2) with signature: std::ostream &(std::ostream &, const nv::Vec3f &)
static bool _check_op_lshift_sig2(lua_State* L) {
	int luatop = lua_gettop(L);
	if( luatop!=2 ) return false;

	if( !luna_isInstanceOf(L,1,SID("std::ostream"),false) ) return false;
	if( !luna_isInstanceOf(L,2,SID("nv::Vec3f"),false) ) return false;
	return true;
}
</sxh>

So we have 3 different overloads for this %%operator<<%% but it seems the version generated for nv::Quat is put in a "separated bag" compared to the versions for nv::Matrixd and nv::Vec3f... How could that be ?

In my log file, it seems the function "%%operator<<%%" is created only once...

Ohhh okay :-), I see now: in fact I really have 2 different functions here: one is "%%operator<<%%", and the other is "%%std::operator<<%%" 

<note>I just noticed I wasn't defining the function inside the std namespace for the nv::Quat version: in fact I should change that, but that's not the problem here.</note>

=> So I think the solution here would be to also specify the namespace for the functions in this case: **OK** this works!

===== Resolving link errors =====

Okay, so with our latest fix on function binding prefixes, the compilation of the objects itself is successful, but then **link** stage will fail, apparently for 2 different reasons:

  * **A.** On one side, we are trying to use a luna class for void: this may sound weird, but it is actually required in our system: void must be considered as a class on its own for us to be able to retrieve void pointers correctly from the lua stack.
  * **B.** On the other side, it seems we are trying to build bindings for some functions that are indeed defined but are **not** exported from the nvCore module itself. [//That part might be more tricky to handle//]

The fix for point A is simple: when done parsing the input files and just before starting to write the bindings, we manually inject a "void" class in the entity tree with this config entry:

<sxhjs lua>cfg.processEntities = function(root)
    -- We should register the "void" type:
    local cl = root:getOrCreateClass("void")
    cl:setDefined(false)
end</sxhjs>

Concerning the point B, the offending functions in my source module seem to be the following:

<sxh cpp>
namespace nv
{
bool installDebugHandlers();
bool installExitHandlers();

NVCORE_EXPORT void * allocate_block(std::size_t blockSize);
NVCORE_EXPORT void allocate_block(void * p, std::size_t blockSize);
}
</sxh>

And indeed... the 2 first functions are not marked for export, so we should not try to use them, and the 2 next functions are marked for export, but I just realized they are not defined anywhere in the source module! So the link error is completely legit in this case ;-).

So I just commented out the declaration of the two missing functions, yet, the case of the installXXXHandlers is somewhat more interesting: from a binding generation perspective, we could be linking to a static library for instance, and in this case, we don't need to specify anything else and the bindings generation/compilation will work. But if we are linking to a dynamic library [and we are on windows], we need to see the %%__declspec(dllexport)%% attribute to support binding to the function of interest. So, how could we handle this dual setup issue ? [//Let's think a little about it Manu....//]

**Note**: I just noticed the libclang functions **clang_getCursorLinkage** and **clang_getCursorVisibility**: maybe these can provide me with the information I need here ? Let's check that.

=> Nope :-( : unfortunately this doesn't seem to help here: all my global functions have the linkage "External" and visibility "Default", so this will not help.


Okay, so now, it seems I can in fact retrieve the "attribute(dllimport)" cursor when this is applicable from class/struct declaration cursor. But for global functions I don't see that cursor yet, let's investigate this further. Hmmm, in fact that was easy: displaying the cursor children when parsing a function cursor, I can observe that I have such an attribute (when applicable) as a direct child, for instance:

<code>
[Debug] 	      Processing: 'msleep' kind: 'FunctionDecl
[Debug] 	      Creating function nv::msleep
[Debug] 	      level 1: '' of kind 'attribute(dllimport)', type: ''
[Debug] 	      level 1: 'ms' of kind 'ParmDecl', type: 'int'
[Debug] 	      Adding signature void (int) to function msleep
[Debug] 	      Return type: void
[Debug] 	      Resolving type void
</code>

=> So I should retrieve that attribute at that level.


<note>Actually, the dllimport attribute should be specified per signature in this case. Because some signatures might be imported/exported while others are not.</note>

OK! Now I can also tell if a function (or rather a function signature) is marked as "dll imported". Let's now try to use this information to allow/disallow the bindings generation.

So after adding support to mark signatures as "dllimported or not" I quickly realized I would also need an "isDefined" flag for each signature: because when a function is defined while parsing then it means we don't need to import it. (think about inline or any other form of header defined functions).

Yet, I'm now facing a strange issue where some of the functions I want to bind are considered non-public or non-bindable somehow (?):

<code>
[Debug] 	      Ignoring non public/bindable class function: nv::Matrixd::ptr
[Debug] 	      Ignoring non public/bindable class function: nv::Matrixd::getOrtho
[Debug] 	      Ignoring non public/bindable class function: nv::Matrixd::getFrustum
[Debug] 	      Ignoring non public/bindable class function: nv::Matrixd::getPerspective
</code>


Arrff... Stupid me ;-) => This was just a dumb regression error... but now this is fixed. And, surprise !! The lua bindings module for nvCore **is now compiling** ! Yeeppeeeee :-)!

==== Conclusion ====

So now we can certainly call this a day! We fixed a lot of significant problem during this session, and with the successful compilation of the nvCore bindings this work is finally starting to pay off, feeewww!

Of course, there is still a great lot to do: in case you don't remember, we are currently only compiling a very small subset of the nvCore module content [ie. basically, only the stuff that is provided in the main "core_common.h" header file]. I'm sure we will face more problems when we start extending the list of included header files. And even without this: many functions are currently not binded because we are ignoring them [because no proper "lua converter" is provided for the argument/return types].

But this will be for another day. For the moment, let's relax for a while and enjoy the work done so far because I need a break [//... and I don't have any kitkat left...// mouuuahh ahh ahh] [//Okay, I'm out. :-)//]