Optimizing QT6 bindings

So, continuing today from the end of our last session, where I introduced some initial minimal QT6 bindings to be able to display a very simple window with a menu bar, and handling a signal connection in lua. This time, we will try to improve a bit on this initial implementation to try to collect all the QT signals automatically when generating the bindings 👍! Let's see if the idea I have will work 😎

First step I would like to take is directly inspired from this article: https://woboq.com/blog/moc-with-clang.html

⇒ The idea would be to let the clang-c library find annotations where we have the signals (and slots) defined in each class. And for that the idea is to provide a custom definition for the Q_SIGNAL/Q_SLOT macros as follow for instance:

#define Q_SIGNAL  __attribute__((annotate("qt_signal")))
#define Q_SLOT    __attribute__((annotate("qt_slot")))

Let's see if we can already simply detected those annotations in the nervBind parsing…

First I added support to specify additional arguments for clang during the parsing:

    if opts.clangAdditionalArgs then
        table.appendAll(cfg.clangArgs, opts.clangAdditionalArgs)
    end

Next I specify a new definition on the command line… which is actually stupid lol… I already have the “NERVBIND” flag so I can add my definitions right in the binding root file (ie. “bind_context.h”), so let's do that instead:

#ifdef __NERBVBIND__
#define Q_SIGNALS                                                              \
  public                                                                       \
    __attribute__((annotate("qt_signal")))
#endif

OK, so now when parsing a class, let's see if we can detect that annotation: hmmm, nope, not really working. But then I figured out that I need to place my macro in shared_context.h which is included first (well after the empty parse_context.h file), and I simply have to define the “QT_ANNOTATE_ACCESS_SPECIFIER” macro as follow:

#ifdef __NERVBIND__
#define QT_ANNOTATE_ACCESS_SPECIFIER(name) __attribute__((annotate(#name)))
#endif

With that change the AST I can generate parsing the QT headers will look like this:

  - CXXAccessSpecifier "" {}
  - CXXMethod "event" {attribute(dllimport) attribute(override) protected bool QAction::event(QEvent* )}: FunctionProto (bool (QEvent*))
  - CXXConstructor "QAction" {attribute(dllimport) protected void QAction::QAction(QActionPrivate& dd, QObject* parent)}: FunctionProto (void (QActionPrivate&,QObject*))
  - CXXAccessSpecifier "" {}
    - attribute(annotate) "qt_slot" {attribute(annotate)}
  - CXXMethod "trigger" {attribute(dllimport) attribute(annotate) public void QAction::trigger()}: FunctionProto (void ())
  - CXXMethod "hover" {attribute(dllimport) attribute(annotate) public void QAction::hover()}: FunctionProto (void ())
  - CXXMethod "setChecked" {attribute(dllimport) attribute(annotate) public void QAction::setChecked(bool )}: FunctionProto (void (bool))
  - CXXMethod "toggle" {attribute(dllimport) attribute(annotate) public void QAction::toggle()}: FunctionProto (void ())
  - CXXMethod "setEnabled" {attribute(dllimport) attribute(annotate) public void QAction::setEnabled(bool )}: FunctionProto (void (bool))
  - CXXMethod "resetEnabled" {attribute(dllimport) attribute(annotate) public void QAction::resetEnabled()}: FunctionProto (void ())
  - CXXMethod "setDisabled" {attribute(dllimport) attribute(annotate) public void QAction::setDisabled(bool b)}: FunctionProto (void (bool))
  - CXXMethod "setVisible" {attribute(dllimport) attribute(annotate) public void QAction::setVisible(bool )}: FunctionProto (void (bool))
  - CXXAccessSpecifier "" {}
    - attribute(annotate) "qt_signal" {attribute(annotate)}
  - CXXMethod "changed" {attribute(dllimport) attribute(annotate) public void QAction::changed()}: FunctionProto (void ())
  - CXXMethod "enabledChanged" {attribute(dllimport) attribute(annotate) public void QAction::enabledChanged(bool enabled)}: FunctionProto (void (bool))
  - CXXMethod "checkableChanged" {attribute(dllimport) attribute(annotate) public void QAction::checkableChanged(bool checkable)}: FunctionProto (void (bool))
  - CXXMethod "visibleChanged" {attribute(dllimport) attribute(annotate) public void QAction::visibleChanged()}: FunctionProto (void ())
  - CXXMethod "triggered" {attribute(dllimport) attribute(annotate) public void QAction::triggered(bool checked)}: FunctionProto (void (bool))
  - CXXMethod "hovered" {attribute(dllimport) attribute(annotate) public void QAction::hovered()}: FunctionProto (void ())
  - CXXMethod "toggled" {attribute(dllimport) attribute(annotate) public void QAction::toggled(bool )}: FunctionProto (void (bool))
  - CXXAccessSpecifier "" {}
  - CXXConstructor "QAction" {private void QAction::QAction(const QAction& )}: FunctionProto (void (const QAction&))
  - CXXMethod "operator=" {private QAction& QAction::operator=(const QAction& )}: FunctionProto (QAction&(const QAction&))

⇒ How amazing is this ?! 😎 The only thing I really need to do here is to search of a child attribute each time I reach a CXXAccessSpecifier cursor, and with that toggle between “standard”, “signal” or “slot” mode for the following CXXMethods until that mode is changed again. That should do the trick 🤔.

To test this theory I added the following code in my parseClass() function, and this seems to work as expected:

        if ckind == clang.CursorKind.CXXAccessSpecifier then
            -- scope:setCurrentVisibility(cur:getCXXAccessSpecifier())
            -- QT extension: we check here if we have an annotation attribute:
            local child = cur:getFirstChild()
            if child and child:getKindSpelling() == "attribute(annotate)" then
                currentFuncType = child:getSpelling()
                logDEBUG("Entering function type ", currentFuncType)
            elseif currentFuncType then
                logDEBUG("Leaving function type ", currentFuncType)
                currentFuncType = nil
            end

Now, I will also store that “category” information in the created functions inside a given class:

---Assign a categoy to this function
---@param cat string|nil The category
function Class:setCategory(cat)
    self._category = cat
end

---Retrieve the current category from this function
---@return string|nil category this function category or nil
function Class:getCategory()
    return self._category
end

Now we still have a bunch of CXXMethods in each class, but some of them will have the category “qt_signal”, and others “qt_slot”, (and otherwise a nil value by default). What should I do with these ?

For a signal function triggered(bool val) I could maybe generate another function in the bindings, called onTriggered(luna::LuaFunction& func)

We could then provide a custom implementation for that new function that would write something like that:

static auto _bind_onTriggered_sig1(lua_State* L) -> int {
	QAction* self = Luna< QAction >::get(L,1);
	LUNA_ASSERT(self!=nullptr);

	luna::LuaFunction func(L, 2);

  auto* dst = new QLuaConnector(func);

  auto con_src = QObject::connect(self, QAction::triggered, dst, QLuaConnector::handle);
  auto* con = new QMetaObject::Connection(con_src);
  
  // Store the QLuaConnector object in a lua table here with con as key
  // Luna<QObject>::push(L, dst, true);

  Luna<QMetaObject::Connection>::push(L, con, true);
	return 1;
}

Or even better, since we can use a lambda or a functor for the connection here, I could now write something like that (completely avoiding the need for a QLuaConnector object then):

static auto _bind_onTriggered_sig1(lua_State* L) -> int {
	QAction* self = Luna< QAction >::get(L,1);
	LUNA_ASSERT(self!=nullptr);

	luna::LuaFunction func(L, 2);

  func.makeRef();

  auto con_src = QObject::connect(self, QAction::triggered, self, func);
  auto* con = new QMetaObject::Connection(con_src);
  
  Luna<QMetaObject::Connection>::push(L, con, true);
	return 1;
}

⇒ That last option would be very cool as it will be a lot easier to use/maintain and less memory will be used in the process. So let's try to implement that. Yet, one detail I'm thinking about now is that the “function category” value I added above should really be on a given signature only each time: we might have multiple signatures for a function name and only one of them declared as a signal for instance, so let's start with fixing that:

            -- Add the category to the signature:
            if currentFuncCat then
                local ftype = cur:getType()
                local signame = ftype:getSpelling()

                local sig = obj:getSignatureByName(signame)
                CHECK(sig ~= nil, "Invalid signature object")
                ---@cast sig -nil
                sig:setCategory(currentFuncCat)
            end

Next, let's add this makeRef() method in the LuaFunction that we will need above, here is the updated code for that:

    LuaFunction(lua_State* L, int idx, bool asRef = false)
        : state(L), index(idx) {
        LUNA_ASSERT(lua_isfunction(state, index));
        if (asRef) {
            makeRef();
        }
    };

    ~LuaFunction() {
        if (isRef && nv::LuaManager::instance().is_state_opened(state)) {
            // Unregister the ref:
            luaL_unref(state, LUA_REGISTRYINDEX, index);
        }
    };

    void makeRef() {
        // Turn this lua function into a ref if it's not a ref already:
        if (isRef) {
            return;
        }

        isRef = true;
        // Keep a ref on this function:
        lua_pushvalue(state, index);

        // Store the reference on the function and keep its ref index:
        // NOLINTNEXTLINE
        index = luaL_ref(state, LUA_REGISTRYINDEX);

        lua_State* L = state;
        LUNA_ASSERT(index != LUA_NOREF);
    }

Next we need to collect all the signatures marked as “qt_signal” in a post-processing step: And this I think could be enabled from the main config file for the QT bindings 🤔 ?

Okay, so for now I decided to write a simple “Entity processor” as follow:

--- Processor function that will generate QT signals where applicable:
---@param ent reflection.Entity Current entity
---@return boolean continue flag
local processor = function(ent)
    if ent:isFunction() then
        local func = ent --[[@as reflection.Function]]
        if not func:isClassMethod() then
            return false
        end

        -- This is a class method, so we check if one of the signature is a signal:
        local sigs = func:getSignaturesByCategory("qt_signal")
        if #sigs == 0 then
            -- No signal for this method
            return false
        end

        logDEBUG("==> Detected " .. #sigs .. " signal(s) for function " .. func:getFullName())

        -- TODO: Should process the function here.
        return false
    end
    return ent:isNamespace()
end

return processor

And then I simply add this processor at the end of the parsing of the QT input files from the config:

-- Signal generator:
cfgtor:addEntityProcessor(import "bind.processors.gen_qt_signals")

return cfgtor:createConfig("QT", opts)

And this will produce the following outputs when executed (which is exactly what I wanted):

2023-03-08 08:13:48.607968 [DEBUG] clang done parsing file: D:\Projects\NervLand\sources/lua_bindings/QT\luna_bindings.h
2023-03-08 08:13:48.607972 [DEBUG] Resolving target types...
2023-03-08 08:13:48.608175 [DEBUG] Done resolving 0 target types.
2023-03-08 08:13:48.608185 [DEBUG] TypeManager: registered 0 types.
2023-03-08 08:13:48.610431 [DEBUG] ==> Detected 1 signal(s) for function QObject::destroyed
2023-03-08 08:13:48.610448 [DEBUG] ==> Detected 1 signal(s) for function QObject::objectNameChanged
2023-03-08 08:13:48.611214 [DEBUG] ==> Detected 1 signal(s) for function QWidget::windowTitleChanged
2023-03-08 08:13:48.611225 [DEBUG] ==> Detected 1 signal(s) for function QWidget::windowIconChanged
2023-03-08 08:13:48.611228 [DEBUG] ==> Detected 1 signal(s) for function QWidget::windowIconTextChanged
2023-03-08 08:13:48.611232 [DEBUG] ==> Detected 1 signal(s) for function QWidget::customContextMenuRequested
2023-03-08 08:13:48.611347 [DEBUG] ==> Detected 1 signal(s) for function QCoreApplication::aboutToQuit
2023-03-08 08:13:48.611353 [DEBUG] ==> Detected 1 signal(s) for function QCoreApplication::organizationNameChanged
2023-03-08 08:13:48.611356 [DEBUG] ==> Detected 1 signal(s) for function QCoreApplication::organizationDomainChanged

As mentioned above, next we need to manually generate that signal helper function (ie. “onTriggered(bool)” for instance). The updated code for the processor will thus be:

--- Processor function that will generate QT signals where applicable:
---@param ent reflection.Entity Current entity
---@return boolean continue flag
local processor = function(ent)
    if ent:isFunction() then
        local func = ent --[[@as reflection.Function]]
        if not func:isClassMethod() then
            return false
        end

        -- This is a class method, so we check if one of the signature is a signal:
        local sigs = func:getSignaturesByCategory("qt_signal")
        if #sigs == 0 then
            -- No signal for this method
            return false
        end

        logDEBUG("==> Detected " .. #sigs .. " signal(s) for function " .. func:getFullName())

        -- Create the new function the class:
        local cl = func:getParent() --[[@as reflection.Class]]

        local sig_func_name = "on_" .. func:getName()
        -- That function name should not be available already:
        CHECK(cl:hasFunction(sig_func_name) == false, "Signal function name ", sig_func_name, " already exists in ",
            cl:getFullName())

        local signal_func = cl:getOrCreateFunction(sig_func_name)
        local tm = luna.tm

        -- Create the signatures:
        for _, refsig in ipairs(sigs) do
            local sig = signal_func:createSignature(refsig:getName())

            sig:setVisibility(clang.Vis.PUBLIC)
            sig:setDllImported(true)
            sig:setDefined(true)

            sig:setReturnType(tm:getType("void"))

            sig:addArgument(Argument("func", tm:getType("luna::LuaFunction&")))
        end

        return false
    end
    return ent:isNamespace()
end

This processor will indeed the generate the bindings functions properly, for instance:

// Bind for on_enabledChanged (1) with signature: void (bool)
static auto _bind_on_enabledChanged_sig1(lua_State* L) -> int {
	QAction* self = Luna< QAction >::get(L,1);
	LUNA_ASSERT(self!=nullptr);

	luna::LuaFunction func(L, 2);

	self->on_enabledChanged(func);

	return 0;
}

// Overall bind for on_enabledChanged
static auto _bind_on_enabledChanged(lua_State* L) -> int {
	if(_check_on_enabledChanged_sig1(L)) return _bind_on_enabledChanged_sig1(L);

	LUNA_ERROR_MSG("Current lua stack: "<<luna::dumpStack(L));
	// NOLINTNEXTLINE
	luaL_error(L, "Binding error for function on_enabledChanged, cannot match any of the 1 signature(s):\n  sig1: void (bool)");
	return 0;
}

The problem with the code generated above is that the function call self->on_enabledChanged(func); doesn't actually make any sense since that function doesn't really exists 😄.

⇒ What I need here is a new mechanism to inject some custom calling code for those functions, but that should not be too hard to provide in fact, let's see…

So for the next update cycle I used this custom code generation:

            -- We should generate some custom code to be injected for this function signature:
            local code = [[func.makeRef();

    auto con_src = QObject::connect(self, &${func_name}, self, func);
    auto* con = new QMetaObject::Connection(con_src);

    Luna<QMetaObject::Connection>::push(L, con, true);
    return 1;]]

            -- code = code:replaceAll("${sig_func_name}", sig_func_name)
            code = code:replaceAll("${func_name}", func:getFullName())
            code = code:replaceAll("${sig_idx}", idx)

            sig:setCustomCode(code)

Which gives us a nice looking code in the bindings:

static auto _bind_on_toggled_sig1(lua_State* L) -> int {
	QAction* self = Luna< QAction >::get(L,1);
	LUNA_ASSERT(self!=nullptr);

	luna::LuaFunction func(L, 2);

	func.makeRef();

    auto con_src = QObject::connect(self, &QAction::toggled, self, func);
    auto* con = new QMetaObject::Connection(con_src);

    Luna<QMetaObject::Connection>::push(L, con, true);
    return 1;
}

But unfortunately, this will not compile yet since I deleted the copy constructor for LuaFunction 😅, but maybe I can enable that in the end… Hmmm, not quite: would really not be a good idea to have multiple copy of a reference lua function 😵‍💫 so let's try a lambda instead passing the function as reference… arrfff, naaa, that's too much plain, let's try to think a bit harder on the LuaFunction copy constructor instead… [thinking hard…] Eureka! And here we go:

    LuaFunction(const LuaFunction& rhs)
        : state(rhs.state), index(rhs.index), isRef(rhs.isRef) {
        if (rhs.isRef) {
            // We need special care here in case the source function is already
            // a reference: We push the referenced value on the stack:
            lua_rawgeti(state, LUA_REGISTRYINDEX, index);

            // And next we make another ref on it:
            index = luaL_ref(state, LUA_REGISTRYINDEX);

            lua_State* L = state;
            LUNA_ASSERT(index != LUA_NOREF);
        }
    };

⇒ Now let's see if we can compile all this… Oooppss… nope: seems we'll try to push a QPalette at some point on the stack, which we don't support yet ⇒ So yeah, I should probably first check that my signal function is bindable 😅

And I also had to introduce a simple operator() function in LuaFunction taking no argument at all (and thus discarding pushArgs()):

    template <typename... Args> void operator()(const Args&... args) {
        pushSelf();
        pushArgs(args...);
        execute();
    }

    // Simple version with no arguments:
    void operator()() {
        pushSelf();
        execute();
    }

Hmmm, but sig:isBindable() doesn't seem to help filtering out the unsupported classes yet how could that be 🤔? Ohh, of course: that's because I should really execute that entity processor after ignoring the classes/functions/etc:

    cfg.processEntities = function(root, tm)
        -- local cl = root:getOrCreateClass("void")
        -- cl:setDefined(false)
        root:foreachClass(ignoreClasses)

        root:foreachEntity(ignoreFuncs)

        root:foreachEntity(ignoreEnums)

        for _, proc in ipairs(self.entityProcessors) do
            root:foreachEntity(proc)
        end
    end

Arrggg… Still not compiling then: we still have an error when trying to push a QLayoutDirection value on the stack: I suspect this is because this is an enum class ? HHmmm, well, no, not even: it's really just a simple enum, but I need to handle pushing such things then…

⇒ I eventually found a way to handle this, introducing an additional custom LunaPusher specialization (with an additional template parameter in the parent class template):

template <typename T>
struct LunaPusher<T, typename std::enable_if<std::is_enum<T>::value>::type> {
    static void pushValue(lua_State* L, const T& arg) {
        lua_pushinteger(L, (int)arg);
    }
};

And now this is finally compiling! Hhoouurraa 🥳!

Now time to give this a try using this updated code:

    local menu = mbar:addMenu("&File") --[[@as Qt.QWidget]]
    local act = Qt.QAction(gutils.createIcon("folder-open-document"), "Open file...", self.win)
    act:on_triggered(function(val) logDEBUG("Yeeppee! It works! value is: ", val) end)

⇒ And this will indeed works just fine! Yeeppeee 🥳!

So now, we have our QT signals available in lua automatically, and we don't even need the QLuaConnector object class anymore (but I'll just keep it around as reference for a while, in case I need to write some QT class in C++ directly later).

Yet, there is still one annoying issue here that I would like to fix, and that's the issue we currently have with the overloading of a parent class function in a child class, like this is the case for instance with QWidget::addAction vs QMenu::addAction

Hmmmm… well, this is indeed a problem that would deserve to be solved eventually, but in fact here, it seems the QMenu::addAction methods are deprecated ⇒ So I could just as well ignore them in the bindings config!

And with that minimal change, I can now use the correct/desired syntax in the lua file:

    -- Add menu bar:
    local mbar = self.win:menuBar()

    local menu = mbar:addMenu("&File") --[[@as Qt.QMenu]]
    local act = Qt.QAction(gutils.createIcon("folder-open-document"), "Open file...", self.win)
    act:on_triggered(function(val) logDEBUG("Yeeppee! It works! value is: ", val) end)
    menu:addAction(act)

From this point, I'm actually not quite sure what I should really do next: I have many ideas in mind that I would like to investigate, and only so much time to do it

I would like to really consider how to integrate Vulkan inside my QT framework in an optimal way, because in the end, I still want to do most of my “sub-projects” in a 3D environment, but I feel this is going to be another trick step to take so I should keep that for another time.

I would also like to start implementing a stable-diffusion project in QQT but this is again going to be really tricky…

I also have my NervStudio project which I could re-implement here in lua, but again… (guess what?) That will be tricky and will require a working vulkan environment 😅

So I think I should just stop here for the moment and think more about what I should really do as a next step before dying in any further