blog:2020:0609_nervluna_field_inheritance

Adding support for fields inheritance in NervLuna

As previously discussed, I've been working on my 'NervLuna' C++/lua binding generator project lately. The project can already be used to generate some interesting bindings, but many “base elements” are still missing. One of them is the support for fields inheritance in classes/structures. So, in this post, we will see together how this can be added.

Suppose we have the following C++ classes:

class ParentClass
{
public:
    int my_int_value{5};
    bool my_bool_value{true};
};

class ChildClass : public ParentClass
{
public:
    int my_second_int{6};
    bool my_second_bool{false};
};

Now suppose we generate the bindings for both those classes. If we then create/retrieve an instance of the class ParentClass in lua, then we should be able to access the public fields in there:

describe("fields inheritance", function()

  it("Should support class level field access", function()
    local obj = nvt.ParentClass()

    -- Now we try to access a real field:
    assert_equal(obj.my_int_value, 5)
    assert_equal(obj.my_bool_value, true)

    -- We try changing the fields value:
    obj.public_int = 8

    -- The new value should be assigned:
    assert_equal(obj.public_int, 8)
  end)
end)
</sxhjs>

=> That part is already working just fine, since this is the **regular field access** mechanism: providing access to the fields that are defined in the class itself.

Same results if we try to access the fields **my_second_int** or **my_second_bool** from an instance of **ChildClass**: since those fields are defined in the child class itself, so the following test is already working too:

<sxhjs lua>
  it("Should support class level field access in child", function()
    local obj = nvt.ChildClass()

    -- Now we try to access a real field:
    assert_equal(obj.my_second_int, 6)
    assert_equal(obj.my_second_bool, false)

    -- We try changing the fields value:
    obj.my_second_int = 10

    -- The new value should be assigned:
    assert_equal(obj.my_second_int, 10)
  end)
</sxhjs>

**But** now, we would also expect to be able to read/write the fields **my_int_value** or **my_bool_value** from in instance of the **ChildClass**, as follow:

<sxhjs lua>
  it("Should support access to inherited fields in child", function()
    local obj = nvt.ChildClass()

    -- Now we try to access a real field:
    assert_equal(obj.my_int_value, 5)
    assert_equal(obj.my_bool_value, true)

    -- We try changing the fields value:
    obj.my_int_value = 10

    -- The new value should be assigned:
    assert_equal(obj.my_int_value, 10)
  end)
</sxhjs>

But obviously, this is the part that is not working yet, simply because in the **%%__index%%** / **%%__newindex%%** metamethods of our ChildClass metatable, we are only listing the fields that are defined on the class itself, not in the parent classes:

<sxh cpp>
luna_RegType LunaTraits< nvt::ChildClass >::fields[] = {
	{"my_second_int", &_field_bind_my_second_int},
	{"my_second_bool", &_field_bind_my_second_bool},
	{0,0}
};

So, how should we proceed to fix this issue ?

  • The __index and __newindex metamethods are implemented as C++ function in NervLuna, so we could consider retrieving all the parents classes fields at that level somehow.
  • But I think there is a more elegant solution to this problem: our C++ functions will anyway look for the field access function in a lua table, so we could proceed here exactly as we already do for the class methods themself: On class creation, we could ask Lua to populate the field table of a given class with the fields available in the table of all the parent classes recursively, and that would make our live really easy if it works as expected :-).

So let's update our luna classes init script accordingly…

And… [Stupid me] now I realize, this should actually just work already! Because I'm currently just writing the fields accessor directly in the class “method” table (ie. which is in fact our class metatable itself), and that part is already handled properly by the class init script. I just didn't run the last unit test described above before, but yes, indeed, it's already working as expected ah ah ah LOL

⇒ Conclusion then is: that there is nothing to change on the code at this level.

And what I learnt here is: when I write a unit test that is supposed to fail, I should really check that it is failing before deciding I should write a blog article about it :-)

Anyway, while we are at it, there is another minor issue that we should deal with here: the write access to non-existing fields. Note that the read access is already working as expected (ie. returning nil in case the field name doesn't match anything we know), so the following unit test will pass:

  it("Should support reading non-existing field", function()
    local obj = nvt.ChildClass()

    -- Now we try to read a dummy field, it should be nil:
    assert_equal(obj.my_dummy_field, nil)
  end)
</sxhjs>

**But** it is not the case with the write access, and the following test is currently crashing (when I try to assign the value "4" to the field "my_dummy_field"):

<sxhjs lua>
  it("Should support writing non-existing field", function()
    local obj = nvt.ChildClass()

    obj.my_dummy_field = 4;

    -- Now we try to read back the value:
    assert_equal(obj.my_dummy_field, 4)
  end)
</sxhjs>

Yet it seems I was already taking care of the case when the field name is not found (ie. when the field value we retrieve below is **not** a lightuserdata):

<sxh cpp>
            lua_getfield(L, -1, fname);

            // if this is a field, then we call the field setter:
            if(lua_type(L,-1) == LUA_TLIGHTUSERDATA)
            {
                // The value we have here should be a "field accessor", so we convert it to that type:
                luna_mfp setter = (luna_mfp)lua_topointer(L,-1);
                ASSERT(setter!=nullptr);
                // pop the lightuserdata and metatable
                lua_pop(L, 2);

                // Then we remove the key value:
                lua_remove(L, -2);
                
                // Next we call that setter function:
                // We only have the actual object userdata on stack + new value at this point:
                return setter(L);
            }
            else {
                // Otherwise, we just assign the new value to our object itself:
                // Pop the lightuserdata and metatable:
                lua_pop(L, 2);

                // We have the value/key/object on the stack:
                lua_rawset(L, 1);
            }

So I felt a bit confused, and decided I should add some debug outputs in the else statement to try to clarify what's happening, and thus I got (just before the call to “lua_pop(L,2)”):

[Debug]               Current luna stack is: stack trace: top 5
  1: [object] ptr=000001BAAEC15610, ID=12944093008736571453 (=nvt.ChildClass), ids=[17338622122622231149 (= nvt.ParentClass),], gc=true
  2: [string] = "my_dummy_field"
  3: [number] = "4"
  4: [table] = 000000002EA68C98
  5: [nil]

And now I finally understand what the problem is: I was thinking that my “obj” value in lua was really a table in the end, but it is not the case: it is rather a userdata. And that's why it doesn't make sense for lua to try to rawset a field value on that object.

⇒ So the conclusion here is that this will just not work at all: we cannot write arbitrary (non-existent) field values inside a concrete luna object. I should make this clear in the code with a proper error message now.

Last thing I would like to check/validate in this devel session is the actual support for complex type of fields, so let's extend our initial test classes now:

class ParentClass
{
public:
    int my_int_value{5};
    bool my_bool_value{true};
};

class ChildClass : public ParentClass
{
public:
    int my_second_int{6};
    bool my_second_bool{false};

    ParentClass my_parent_field;
    ParentClass* my_second_parent{nullptr};
    ParentClass* my_third_parent{nullptr};

    ChildClass() {
        my_third_parent = new ParentClass();
        my_third_parent->my_int_value = 7;
    }

    ~ChildClass() {
        delete my_third_parent;
    }
};

And now let's see if we can access the object/pointer fields as one would naturally expect:

  it("Should support access to object field", function()
    local obj = nvt.ChildClass()

    assert_equal(obj.my_parent_field.my_int_value, 5)
    assert_equal(obj.my_parent_field.my_bool_value, true)
  end)
</sxhjs>

=> **OK**, this is working fine. Yet, I checked the field accessor, and I think I have something wrong in there:

<sxh cpp; highlight: [14,15]>
static int _field_bind_my_parent_field(lua_State* L) {
	if(!_field_check_my_parent_field(L)) {
		luaL_error(L, "Invalid stack elements for field my_parent_field access.");
		return 0;
	}
	int luatop = lua_gettop(L);
	nvt::ChildClass* self = Luna< nvt::ChildClass >::get(L,1);
	if(luatop==2) {
		nvt::ParentClass* my_parent_field = Luna< nvt::ParentClass >::get(L,2);
		self->my_parent_field = *my_parent_field;
		return 0;
	}
	else {
		nvt::ParentClass* my_parent_field_ptr = new nvt::ParentClass(self->my_parent_field);
		Luna< nvt::ParentClass >::push(L, my_parent_field_ptr, true);
		return 1;
	}
}

In the else statement of the function just above, we handle the “getter” part of the accessor, and we see here that we are actually creating a new copy instance of the ParentClass object that we then push on the stack: this is incorrect, instead we should really use a pointer on the existing instance. So let's fix that. Done. Now we have the generated code:

static int _field_bind_my_parent_field(lua_State* L) {
	if(!_field_check_my_parent_field(L)) {
		luaL_error(L, "Invalid stack elements for field my_parent_field access.");
		return 0;
	}
	int luatop = lua_gettop(L);
	nvt::ChildClass* self = Luna< nvt::ChildClass >::get(L,1);
	if(luatop==2) {
		nvt::ParentClass* my_parent_field = Luna< nvt::ParentClass >::get(L,2);
		self->my_parent_field = *my_parent_field;
		return 0;
	}
	else {
		Luna< nvt::ParentClass >::push(L, &self->my_parent_field, false);
		return 1;
	}
}

⇒ Much better :-) [and all unit tests are still passed of course.]

Let's continue with the access to the nil pointer now:

  it("Should support access to nil object", function()
    local obj = nvt.ChildClass()

    assert_equal(obj.my_second_parent, nil)

    -- But we should be able to assign another parent class pointer to that field:
    local p = nvt.ParentClass();
    obj.my_second_parent = p;

    p.my_int_value = 6;
    assert_equal(obj.my_second_parent.my_int_value, 6)

    p.my_int_value = 7;
    assert_equal(obj.my_second_parent.my_int_value, 7)
  end)
</sxhjs>

Hmmm... unfortunately, this is failing on the first assertion apparently ("assert_equal(obj.my_second_parent, nil)") since I have the error message: <code>Assert failed: expected userdata: 0x0ffd0f30 to be equal to nil</code>

Let's check the accessor code here again:

<sxh cpp>
static int _field_bind_my_second_parent(lua_State* L) {
	if(!_field_check_my_second_parent(L)) {
		luaL_error(L, "Invalid stack elements for field my_second_parent access.");
		return 0;
	}
	int luatop = lua_gettop(L);
	nvt::ChildClass* self = Luna< nvt::ChildClass >::get(L,1);
	if(luatop==2) {
		nvt::ParentClass* my_second_parent = Luna< nvt::ParentClass >::get(L,2);
		self->my_second_parent = my_second_parent;
		return 0;
	}
	else {
		Luna< nvt::ParentClass >::push(L, self->my_second_parent, false);
		return 1;
	}
}

So we see that we use the Luna::push() function to try to push an instance of our class on the stack. Yet I would really expect that function to push a nil value if the pointer that we receive as input is nullptr, yet this is currently not the case: let's fix that. OK

Now this will bring on the table another consideration: when we try to “get” a luna object from the stack instead (ie. performing the opposite operation) then we should be ready to receive nil values too and convert them dynamically. Let's fix that too. Done

Now let's try the unit test just above once more… All good! :-D

And now jsut one final unit test to confirm we can read existing pointer (but that is sort of already confirmed with the previous test in fact, so not expecting any surprise here):

it("Should support access to pointer object", function()
    local obj = nvt.ChildClass()

    -- Should not be nil on creation:
    assert_not_nil(obj.my_third_parent)

    -- And we should be able to read the internal value:
    assert_equal(obj.my_third_parent.my_int_value, 7)
    
    -- Lets update the value:
    obj.my_third_parent.my_int_value = 10
    assert_equal(obj.my_third_parent.my_int_value, 10)
  end)

And indeed, this is working just fine. All right!

So I think this is good enough for this little development session: in summary, we confirmed that we had support for field inheritance in the binding generation, that non-existent fields could not be written to and would generate a proper error message if the user tries to do that, and also that we could deal with non-trivial types for fields such as object instances or pointer.

Next thing I should consider now is to as support for const field handling: if a field is const, we should not provide a “setter” part in the field accessor function (actually, the code generated with such inputs would probably not compile currently). But this will be for another day!

Meanwhile, happy hacking everyone! ;-)

  • blog/2020/0609_nervluna_field_inheritance.txt
  • Last modified: 2020/07/10 12:11
  • by 127.0.0.1