ThumbGen: Introducing drawsvg support

In this article we continue our journey on how to improve on our youtube thumbnail generation process.

Youtube video for this article available at:

One cool thing I think I could have in the thumbnail generator is the ability to generate either linear or spherical gradients to use then as thumbnail background. So let's try to add that…

But first, some cleanup and updates: I need to improve on the mechanism I used so far to generate the “outlines”: instead of edges detection I should use the same distance to foreground computation technic as for the “contour generation” process used when removing backgrounds:

    def apply_outline(self, sub_img, contour_size, contour_color):
        """Apply the outline"""

        sub_arr = np.array(sub_img)
        mask = sub_arr[:, :, 3]
        col = [np.float32(el) / 255.0 for el in contour_color]

        # Compute the distance to foreground:
        dist = self.compute_distance_to_foreground(mask)

        img_arr = sub_arr.astype(np.float32) / 255.0

        # Prepare the result image:
        res = np.copy(img_arr)

        # Fill with the contour color:
        idx = dist <= contour_size
        alpha = np.array(mask).astype(np.float32) / 255.0

        for i in range(4):
            # Fill with the contour color:
            res[idx, i] = col[i]
            # Re-add the subject on top of contour:
            res[:, :, i] = img_arr[:, :, i] * alpha + res[:, :, i] * (1.0 - alpha)

        sub_arr = (res * 255.0).astype(np.uint8)

        return Image.fromarray(sub_arr)

Actually, when considering this, I thought about SVG, and thus foudn the python package drawsvg which looks great. But this depends on cairo, so let's see how I could build cairo for windows…

⇒ I think I can use the file https://github.com/preshing/cairo-windows/blob/master/build-cairo-windows.sh as a template and from that, integrate the build of pixman/cairo into NervProj (note that I have already build zlib/libpng/freetype anyway)

Starting with a builder for pixman (on windows only for now, linux should be a piece of cake anyway):

    def build_on_windows(self, build_dir, prefix, _desc):
        """Build on windows method"""

        # Only applicable to msvc
        self.check(self.compiler.is_msvc(), "Only available wit MSVC compiler")

        # Reference:
        # cd pixman
        # sed s/-MD/-MT/ Makefile.win32.common > Makefile.win32.common.fixed
        # mv Makefile.win32.common.fixed Makefile.win32.common
        # if [ $MSVC_PLATFORM_NAME = x64 ]; then
        #     # pass -B for switching between x86/x64
        #     make pixman -B -f Makefile.win32 "CFG=release" "MMX=off"
        # else
        #     make pixman -B -f Makefile.win32 "CFG=release"
        # fi
        # cd ..

        # Path the Makefile.win32.common file:
        self.patch_file(self.get_path(build_dir, "Makefile.win32.common"), "MD", "MT")

        # Run the make command:
        flags = ["pixman", "-B", "-f", "Makefile.win32", "CFG=release", "MMX=off"]
        self.exec_make(build_dir, flags)

        # Manually install the files:
        inc_dir = self.get_path(prefix, "include")
        self.make_folder(inc_dir)
        self.copy_file(self.get_path(build_dir, "pixman/pixman-version.h"), self.get_path(inc_dir, "pixman-version.h"))
        self.copy_file(self.get_path(build_dir, "pixman/pixman.h"), self.get_path(inc_dir, "pixman.h"))
        lib_dir = self.get_path(prefix, "lib")
        self.make_folder(lib_dir)
        self.copy_file(self.get_path(build_dir, "pixman/release/pixman-1.lib"), self.get_path(lib_dir, "pixman-1.lib"))

OK! compilation is working fine (after fixing the path to “make” on windows). Now moving to cairo itself!

For the cairo package, we actually have a tar.xz source package, and extraction failed on windows of course, so it was time to upgrade my unzip_package method (need to do import tarfile obviously):

    def unzip_package(self, src_pkg_path, dest_dir, target_name=None):
        """Unzip a package"""

        # check if this is a tar.xz archive:
        if src_pkg_path.endswith(".tar.xz"):
            # cmd = ["tar", "-xvJf", src_pkg_path, "-C", dest_dir]
            with tarfile.open(src_pkg_path, "r:xz") as tar:
                tar.extractall(path=dest_dir)
            logger.info("Done extracting %s.", src_pkg_path)
            return
        elif src_pkg_path.endswith(".tar.gz") or src_pkg_path.endswith(".tgz"):
            # cmd = ["tar", "-xvzf", src_pkg_path, "-C", dest_dir]
            with tarfile.open(src_pkg_path, "r:gz") as tar:
                tar.extractall(path=dest_dir)
            logger.info("Done extracting %s.", src_pkg_path)
            return
        elif src_pkg_path.endswith(".7z.exe"):
            if target_name is None:
                target_name = self.remove_file_extension(os.path.basename(src_pkg_path))
            cmd = [self.get_unzip_path(), "x", "-o" + dest_dir + "/" + target_name, src_pkg_path]
        else:
            cmd = [self.get_unzip_path(), "x", "-o" + dest_dir, src_pkg_path]
        self.execute(cmd, verbose=self.settings["verbose"])

…Hmmm, interesting 🤔: I retrieve the package for cairo-1.17.8 but can't find any kind of makefile/configure file in there 😲 that's weird. let's try version 1.17.2 as used in the reference script. OK that one seems better

Feeww… This was a little bit tricky, but I finally now have cairo built for windows with the following builder function 👍!

    def build_on_windows(self, build_dir, prefix, _desc):
        """Build on windows method"""

        # Only applicable to msvc
        self.check(self.compiler.is_msvc(), "Only available with MSVC compiler")

        # Reference:
        # cd cairo
        # sed 's/-MD/-MT/;s/zdll.lib/zlib.lib/' build/Makefile.win32.common > Makefile.win32.common.fixed
        # mv Makefile.win32.common.fixed build/Makefile.win32.common
        # if [ $USE_FREETYPE -ne 0 ]; then
        #     sed '/^CAIRO_LIBS =/s/$/ $(top_builddir)\/..\/freetype\/freetype.lib/;/^DEFAULT_CFLAGS =/s/$/ -I$(top_srcdir)\/..\/freetype\/include/' build/Makefile.win32.common > Makefile.win32.common.fixed
        # else
        #     sed '/^CAIRO_LIBS =/s/ $(top_builddir)\/..\/freetype\/freetype.lib//;/^DEFAULT_CFLAGS =/s/ -I$(top_srcdir)\/..\/freetype\/include//' build/Makefile.win32.common > Makefile.win32.common.fixed
        # fi
        # mv Makefile.win32.common.fixed build/Makefile.win32.common
        # sed "s/CAIRO_HAS_FT_FONT=./CAIRO_HAS_FT_FONT=$USE_FREETYPE/" build/Makefile.win32.features > Makefile.win32.features.fixed
        # mv Makefile.win32.features.fixed build/Makefile.win32.features
        # # pass -B for switching between x86/x64
        # make -B -f Makefile.win32 cairo "CFG=release"
        # cd ..

        zlib_dir = self.man.get_library_root_dir("zlib").replace("\\", "/")
        png_dir = self.man.get_library_root_dir("libpng").replace("\\", "/")
        pixman_dir = self.man.get_library_root_dir("pixman").replace("\\", "/")
        ft_dir = self.man.get_library_root_dir("freetype").replace("\\", "/")
        brotli_dir = self.man.get_library_root_dir("brotli").replace("\\", "/")
        harfbuzz_dir = self.man.get_library_root_dir("harfbuzz").replace("\\", "/")

        # brotli_lib = "brotlidec.a" if self.compiler.is_emcc() else "brotlidec.lib"
        # harfbuzz_lib = "harfbuzz.a" if self.compiler.is_emcc() else "harfbuzz.lib"
        # png_lib = "libpng16_static.lib" if self.is_windows else "libpng16.a"
        # z_lib = "zlibstatic.lib" if self.is_windows else "libz.a"

        # Path the Makefile.win32.common file:
        ft_libs = f"{ft_dir}/lib/freetype.lib {harfbuzz_dir}/lib/harfbuzz.lib"
        ft_libs += f" {brotli_dir}/lib/brotlidec.lib {brotli_dir}/lib/brotlicommon.lib"

        self.multi_patch_file(
            self.get_path(build_dir, "build/Makefile.win32.common"),
            # ("-MD", "-MT"),
            ("-I$(ZLIB_PATH)/", f"-I{zlib_dir}/include -I{ft_dir}/include/freetype2"),
            ("$(ZLIB_PATH)/zdll.lib", f"{zlib_dir}/lib/zlibstatic.lib {ft_libs}"),
            ("-I$(LIBPNG_PATH)/", f"-I{png_dir}/include"),
            ("$(LIBPNG_PATH)/libpng.lib", f"{png_dir}/lib/libpng16_static.lib"),
            ("-I$(PIXMAN_PATH)/pixman/", f"-I{pixman_dir}/include"),
            ("$(PIXMAN_PATH)/pixman/$(CFG)/pixman-1.lib", f"{pixman_dir}/lib/pixman-1.lib"),
            ("@mkdir -p $(CFG)/`dirname $<`", ""),
            # ("CFG_LDFLAGS :=", "CFG_LDFLAGS := "),
        )
        #  /NODEFAULTLIB:MSVCRT /NODEFAULTLIB:libucrt
        self.patch_file(
            self.get_path(build_dir, "build/Makefile.win32.features"), "CAIRO_HAS_FT_FONT=0", "CAIRO_HAS_FT_FONT=1"
        )
        self.patch_file(self.get_path(build_dir, "build/Makefile.win32.features-h"), '"', "")
        self.patch_file(
            self.get_path(build_dir, "src/Makefile.win32"),
            '@for x in $(enabled_cairo_headers); do echo "	src/$$x"; done',
            "",
        )

        # Manually prepare the build folders:
        self.make_folder(self.get_path(build_dir, "src/release/win32"))

        # Run the make command:
        flags = ["-B", "-f", "Makefile.win32", "cairo", "CFG=release"]
        self.exec_make(build_dir, flags)

        # Manually install the files:
        headers = [
            "cairo-features.h",
            "cairo.h",
            "cairo-deprecated.h",
            "cairo-win32.h",
            "cairo-script.h",
            "cairo-ps.h",
            "cairo-pdf.h",
            "cairo-svg.h",
            "cairo-ft.h",
        ]

        self.install_files(".", r"cairo-version\.h$", "include")
        self.install_files("src", r"\.h$", "include", included=headers)
        self.install_files("src/release", r"cairo\.dll$", "bin")
        self.install_files("src/release", r"cairo\.lib$", "lib")
        self.install_files("src/release", r"cairo-static\.lib$", "lib")

First let's update the python environment to include this package:

  media_env:
    inherit: default_env
    packages:
      - moviepy
      - Pillow
      - ffmpeg-python
      - opencv-python
      - rembg[gpu]
      - scipy
      - drawsvg[all]

Next we add a simple script command to test drawsvg:

  test-drawsvg:
    notify: false
    custom_python_env: media_env
    cmd: ${PYTHON} ${PROJECT_ROOT_DIR}/nvp/media/thumb_generator.py drawsvg-test
    python_path: ["${NVP_ROOT_DIR}"]

And here is the test function implementation (directly taken from an example on the page https://pypi.org/project/drawsvg/#full-feature-install):

    def drawsvg_test(self):
        """Test function for drawsvg"""
        logger.info("Should perform drawsvg test here.")

        d = draw.Drawing(
            400,
            200,
            origin="center",
            animation_config=draw.types.SyncedAnimationConfig(
                # Animation configuration
                duration=8,  # Seconds
                show_playback_progress=True,
                show_playback_controls=True,
            ),
        )
        d.append(draw.Rectangle(-200, -100, 400, 200, fill="#eee"))  # Background
        d.append(draw.Circle(0, 0, 40, fill="green"))  # Center circle

        # Animation
        circle = draw.Circle(0, 0, 0, fill="gray")  # Moving circle
        circle.add_key_frame(0, cx=-100, cy=0, r=0)
        circle.add_key_frame(2, cx=0, cy=-100, r=40)
        circle.add_key_frame(4, cx=100, cy=0, r=0)
        circle.add_key_frame(6, cx=0, cy=100, r=40)
        circle.add_key_frame(8, cx=-100, cy=0, r=0)
        d.append(circle)
        r = draw.Rectangle(0, 0, 0, 0, fill="silver")  # Moving square
        r.add_key_frame(0, x=-100, y=0, width=0, height=0)
        r.add_key_frame(2, x=0 - 20, y=-100 - 20, width=40, height=40)
        r.add_key_frame(4, x=100, y=0, width=0, height=0)
        r.add_key_frame(6, x=0 - 20, y=100 - 20, width=40, height=40)
        r.add_key_frame(8, x=-100, y=0, width=0, height=0)
        d.append(r)

        # Changing text
        draw.native_animation.animate_text_sequence(
            d, [0, 2, 4, 6], ["0", "1", "2", "3"], 30, 0, 1, fill="yellow", center=True
        )

        # Save as a standalone animated SVG or HTML
        # d.save_svg('playback-controls.svg')
        # d.save_html('playback-controls.html')

        # Display in Jupyter notebook
        # d.display_image()  # Display SVG as an image (will not be interactive)
        # d.display_iframe()  # Display as interactive SVG (alternative)
        # d.as_gif('orbit.gif', fps=10)  # Render as a GIF image, optionally save to file
        d.as_mp4("orbit.mp4", fps=60, verbose=True)  # Render as an MP4 video, optionally save to file
        # d.as_spritesheet('orbit-spritesheet.png', row_length=10, fps=3)  # Render as a spritesheet
        # d.display_inline()  # Display as interactive SVG
        logger.info("Generation done.")

        return True

⇒ As expected, when I try to run that, I get an error (expected because I'm not providing cairo.dll yet in the environment):

  File "D:\Projects\NervProj\.pyenvs\media_env\lib\site-packages\drawsvg\jupyter.py", line 12, in rasterize
    return raster.Raster.from_svg(self.svg)
  File "D:\Projects\NervProj\.pyenvs\media_env\lib\site-packages\drawsvg\raster.py", line 51, in from_svg
    cairosvg = delay_import_cairo()
  File "D:\Projects\NervProj\.pyenvs\media_env\lib\site-packages\drawsvg\raster.py", line 11, in delay_import_cairo
    raise ImportError(
ImportError: Failed to load CairoSVG. drawSvg will be unable to output PNG or other raster image formats. See https://github.com/cduck/drawsvg#full-feature-install for more details.

Now let's try to provide the module:

  media_env:
    inherit: default_env
    packages:
      - moviepy
      - Pillow
      - ffmpeg-python
      - opencv-python
      - rembg[gpu]
      - scipy
      - drawsvg[all]
    additional_modules:
      cairo.dll: http://files.nervtech.org/nvp_packages/modules/cairo.dll

And now trying again the test function… Ohh my god, this just worked 😲! I got a nice orbit.mp4 file generated, I can't believe it 🤣:

Now it's time to start playing a little bit with drawsvg eh eh eh 😆.

⇒ Reference documentation pages:

• https://github.com/cduck/drawsvg/blob/master/docs/index.md
• https://github.com/cduck/drawsvg/blob/master/README.md