NervLuna: generating bindings for nvCore - second run

So here we are again, with yet another dev session to try to get the lua bindings for my nvCore module to finally compile properly. On the table today: abstract classes, unnamed arguments, already defined functions and finally link errors: “Bon appétit!” . But hey, the issues are not going to simply automatically fix themself, so no choice here: we have to get to work !

As discussed in my previous article the next issue on the list is on the generation of a constructor for abstract class. It turns out this is relatively easy to fix: from libclang we have the function clang_CXXRecord_isAbstract which can be used to figure out if a given class/struct cursor is abstract, so I just need to extend a bit the mechanism already in place.

I add in my “Class” class:

function Class:isAbstract()
  return self._isAbstract
end

function Class:setAbstract(val)
  if val then
    logDEBUG("Detected abstract class: ", self:getFullName())
  end
  self._isAbstract = val
end
</sxhjs>

Then I update that "abstract" flag when parsing a given class. And finally, if I notice a class is abstract when writing its binding file, I just pretend we don't have any public constructor available (and that will do the trick: users won't be able to contruct that class from lua and the generated code will compile):

<sxhjs lua>
  local cons = cl:getConstructor()
  local pubCtorSigs = cons and cons:getPublicSignatures(true) or {}
  if cl:isAbstract() then
    logDEBUG("Disallowing construction of abstract class ", fname)
    pubCtorSigs = {}
  end
</sxhjs>

And finally we also need to prevent the generation of the contructor function binding at the FunctionWriter level with:

<sxhjs lua>
    elseif f:isConstructor() and f:getParent():isAbstract() then
      -- Should not provide any bindings for abstract classes constructor:
      logDEBUG("Ignoring abstract class constructor: ", fname)
      self:writeLine("// NervLuna: Ignoring abstract class constructor '%s'", fname)
</sxhjs>

===== Unnamed arguments issue =====

With the abstract class issue out of the way, the next compilation error I face is in the bindings generated for the matrice classes I have:

<sxh cpp; highlight: [7,9]>
// Bind for makeScale (1) with signature: void (const nv::Vec3f &)
static int _bind_makeScale_sig1(lua_State* L) {
	// When reaching this call, we assume that the type checking is already done.
	nv::Matrixd* self = Luna< nv::Matrixd >::get(L,1);
	ASSERT(self!=nullptr);

	nv::Vec3f*  = Luna< nv::Vec3f >::get(L,2,false);

	self->makeScale(*);

	return 0;
}

And well, that's the thing: there is just no name provided for the arguments in that case, since we have the functions declaration:

  void makeScale( const nv::Vec3f& );
  void makeScale( const Vec3d& );
  void makeScale( value_type, value_type, value_type );

  void makeTranslate( const nv::Vec3f& );
  void makeTranslate( const Vec3d& );
  void makeTranslate( value_type, value_type, value_type );

⇒ So, when parsing the function arguments, we should provide default names in case none is provided:

  
  for i=0,num-1 do
    local arg = cursor:getArgument(i)
    local argname = arg:getSpelling()
    -- #112: use a default argument name if none is provided.
    if argname == "" then
      argname = "arg"..i
    end
    logDEBUG("Arg ",i," cursor: ", argname,", type: ", arg:getType():getSpelling())
    -- For each argument we assign the current cursor context:
    self:setContextCursor(paramDecls[i+1])
    sig:addArgument(self:resolveArgument(arg))
  end</sxhjs>

And now the results are much better :-) :

<sxh cpp>
// Bind for makeScale (1) with signature: void (const nv::Vec3f &)
static int _bind_makeScale_sig1(lua_State* L) {
	// When reaching this call, we assume that the type checking is already done.
	nv::Matrixd* self = Luna< nv::Matrixd >::get(L,1);
	ASSERT(self!=nullptr);

	nv::Vec3f* arg0 = Luna< nv::Vec3f >::get(L,2,false);

	self->makeScale(*arg0);

	return 0;
}

Good!

And now it seems the last item on the table for this session will be with multiple definitions of the same operator in the global functions list:

W:\Projects\NervSeed\sources\lua_bindings\wip_Core\src\luna\register_functions.cpp(679): error C2084: function 'bool nv::_check_op_lshift_sig1(lua_State *)' already has a body
W:\Projects\NervSeed\sources\lua_bindings\wip_Core\src\luna\register_functions.cpp(28): note: see previous definition of '_check_op_lshift_sig1'
W:\Projects\NervSeed\sources\lua_bindings\wip_Core\src\luna\register_functions.cpp(765): error C2065: '_check_op_lshift_sig1': undeclared identifier
W:\Projects\NervSeed\sources\lua_bindings\wip_Core\src\luna\register_functions.cpp(1944): error C2084: function 'int nv::_bind_op_lshift_sig1(lua_State *)' already has a body
W:\Projects\NervSeed\sources\lua_bindings\wip_Core\src\luna\register_functions.cpp(752): note: see previous definition of '_bind_op_lshift_sig1'
W:\Projects\NervSeed\sources\lua_bindings\wip_Core\src\luna\register_functions.cpp(2028): error C2084: function 'int nv::_bind_op_lshift(lua_State *)' already has a body
W:\Projects\NervSeed\sources\lua_bindings\wip_Core\src\luna\register_functions.cpp(764): note: see previous definition of '_bind_op_lshift'
W:\Projects\NervSeed\sources\lua_bindings\wip_Core\src\luna\register_functions.cpp(2029): error C2065: '_check_op_lshift_sig1': undeclared identifier
W:\Projects\NervSeed\sources\lua_bindings\wip_Core\src\luna\register_functions.cpp(2029): error C2065: '_bind_op_lshift_sig1': undeclared identifier

Checking the generated code here is what I found:

// Typecheck for operator<< (1) with signature: std::ostream &(std::ostream &, const nv::Quat &)
static bool _check_op_lshift_sig1(lua_State* L) {
	int luatop = lua_gettop(L);
	if( luatop!=2 ) return false;

	if( !luna_isInstanceOf(L,1,SID("std::ostream"),false) ) return false;
	if( !luna_isInstanceOf(L,2,SID("nv::Quat"),false) ) return false;
	return true;
}

// Typecheck for operator<< (1) with signature: std::ostream &(std::ostream &, const nv::Matrixd &)
static bool _check_op_lshift_sig1(lua_State* L) {
	int luatop = lua_gettop(L);
	if( luatop!=2 ) return false;

	if( !luna_isInstanceOf(L,1,SID("std::ostream"),false) ) return false;
	if( !luna_isInstanceOf(L,2,SID("nv::Matrixd"),false) ) return false;
	return true;
}

// Typecheck for operator<< (2) with signature: std::ostream &(std::ostream &, const nv::Vec3f &)
static bool _check_op_lshift_sig2(lua_State* L) {
	int luatop = lua_gettop(L);
	if( luatop!=2 ) return false;

	if( !luna_isInstanceOf(L,1,SID("std::ostream"),false) ) return false;
	if( !luna_isInstanceOf(L,2,SID("nv::Vec3f"),false) ) return false;
	return true;
}

So we have 3 different overloads for this operator<< but it seems the version generated for nv::Quat is put in a “separated bag” compared to the versions for nv::Matrixd and nv::Vec3f… How could that be ?

In my log file, it seems the function “operator<<” is created only once…

Ohhh okay , I see now: in fact I really have 2 different functions here: one is “operator<<”, and the other is “std::operator<<”

⇒ So I think the solution here would be to also specify the namespace for the functions in this case: OK this works!

Okay, so with our latest fix on function binding prefixes, the compilation of the objects itself is successful, but then link stage will fail, apparently for 2 different reasons:

• A. On one side, we are trying to use a luna class for void: this may sound weird, but it is actually required in our system: void must be considered as a class on its own for us to be able to retrieve void pointers correctly from the lua stack.
• B. On the other side, it seems we are trying to build bindings for some functions that are indeed defined but are not exported from the nvCore module itself. [That part might be more tricky to handle]

The fix for point A is simple: when done parsing the input files and just before starting to write the bindings, we manually inject a “void” class in the entity tree with this config entry:

cfg.processEntities = function(root)
    -- We should register the "void" type:
    local cl = root:getOrCreateClass("void")
    cl:setDefined(false)
end</sxhjs>

Concerning the point B, the offending functions in my source module seem to be the following:

<sxh cpp>
namespace nv
{
bool installDebugHandlers();
bool installExitHandlers();

NVCORE_EXPORT void * allocate_block(std::size_t blockSize);
NVCORE_EXPORT void allocate_block(void * p, std::size_t blockSize);
}

And indeed… the 2 first functions are not marked for export, so we should not try to use them, and the 2 next functions are marked for export, but I just realized they are not defined anywhere in the source module! So the link error is completely legit in this case .

So I just commented out the declaration of the two missing functions, yet, the case of the installXXXHandlers is somewhat more interesting: from a binding generation perspective, we could be linking to a static library for instance, and in this case, we don't need to specify anything else and the bindings generation/compilation will work. But if we are linking to a dynamic library [and we are on windows], we need to see the __declspec(dllexport) attribute to support binding to the function of interest. So, how could we handle this dual setup issue ? [Let's think a little about it Manu….]

Note: I just noticed the libclang functions clang_getCursorLinkage and clang_getCursorVisibility: maybe these can provide me with the information I need here ? Let's check that.

⇒ Nope : unfortunately this doesn't seem to help here: all my global functions have the linkage “External” and visibility “Default”, so this will not help.

Okay, so now, it seems I can in fact retrieve the “attribute(dllimport)” cursor when this is applicable from class/struct declaration cursor. But for global functions I don't see that cursor yet, let's investigate this further. Hmmm, in fact that was easy: displaying the cursor children when parsing a function cursor, I can observe that I have such an attribute (when applicable) as a direct child, for instance:

[Debug] 	      Processing: 'msleep' kind: 'FunctionDecl
[Debug] 	      Creating function nv::msleep
[Debug] 	      level 1: '' of kind 'attribute(dllimport)', type: ''
[Debug] 	      level 1: 'ms' of kind 'ParmDecl', type: 'int'
[Debug] 	      Adding signature void (int) to function msleep
[Debug] 	      Return type: void
[Debug] 	      Resolving type void

⇒ So I should retrieve that attribute at that level.

OK! Now I can also tell if a function (or rather a function signature) is marked as “dll imported”. Let's now try to use this information to allow/disallow the bindings generation.

So after adding support to mark signatures as “dllimported or not” I quickly realized I would also need an “isDefined” flag for each signature: because when a function is defined while parsing then it means we don't need to import it. (think about inline or any other form of header defined functions).

Yet, I'm now facing a strange issue where some of the functions I want to bind are considered non-public or non-bindable somehow (?):

[Debug] 	      Ignoring non public/bindable class function: nv::Matrixd::ptr
[Debug] 	      Ignoring non public/bindable class function: nv::Matrixd::getOrtho
[Debug] 	      Ignoring non public/bindable class function: nv::Matrixd::getFrustum
[Debug] 	      Ignoring non public/bindable class function: nv::Matrixd::getPerspective

Arrff… Stupid me ⇒ This was just a dumb regression error… but now this is fixed. And, surprise !! The lua bindings module for nvCore is now compiling ! Yeeppeeeee !

So now we can certainly call this a day! We fixed a lot of significant problem during this session, and with the successful compilation of the nvCore bindings this work is finally starting to pay off, feeewww!

Of course, there is still a great lot to do: in case you don't remember, we are currently only compiling a very small subset of the nvCore module content [ie. basically, only the stuff that is provided in the main “core_common.h” header file]. I'm sure we will face more problems when we start extending the list of included header files. And even without this: many functions are currently not binded because we are ignoring them [because no proper “lua converter” is provided for the argument/return types].

But this will be for another day. For the moment, let's relax for a while and enjoy the work done so far because I need a break [… and I don't have any kitkat left… mouuuahh ahh ahh] [Okay, I'm out. ]