blog:2022:0828_video_stabilization [NervTech's Wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== NervHome: Videos stabilization utility ======

{{tag>dev python nervhome video ffmpeg}}

So in this article, we will document the setup of the video stabilization utility in the NervProj framework (inside the NervHome project). This utility will be based on the former version written in bash scripts and converted into a modern python application in the process.

====== ======

===== FFmpeg setup =====

  * Currently, the command I'm using to "convert video" files is as follow: <sxh bash; highlight: []>$ nv_admin convert_movies --stabilize --gopro3p --normaudio</sxh>
  * This command will run the following python code: <sxh python; highlight: []>    if cmd == "convert_movies":
        CHECK(len(params)<=3,"Unexpected number of arguments: %s" % args)
        stabilize=False
        # Should default to auto correction here:
        correctLens="auto"
        normAudio=False
        if "--stabilize" in params:
            logDEBUG("Using stabilization.")
            stabilize=True
        if "--gopro3p" in params:
            logDEBUG("Applying lens correction for gopro3+")
            correctLens="gopro3p"
        if "--no-lens-correction" in params:
            logDEBUG("Disabling lens correction")
            correctLens="none"
        if "--normaudio" in params:
            logDEBUG("Applying audio normalization.");
            normAudio=True

        # Before doing any video conversion we should concat the gopro files (by default):
        nvConcatGoproFiles(os.getcwd())

        conv = nvVideoConverter(os.getcwd(), stabilize=stabilize, correctLens=correctLens, normAudio=normAudio)
        conv.convertMovies()
        return</sxh>

  * => Thus I should use the **nvVideoConverter** as an initial reference.
  * Yet, the first thing we need is the ffmpeg tool with the VideStab plugin (and support for x264/x265)
  * Downloading windows version from: https://www.gyan.dev/ffmpeg/builds/
  * Downloading linux version from: https://johnvansickle.com/ffmpeg/

  * Installing the ffmpeg tool with the command: <sxh bash; highlight: []>$ nvp tools install</sxh>

  * => Now we have ffmpeg version 2022.8.25 ready on windows and version 2022.07.22 on linux: let's try to use them now.
  
===== Building the VideoConverter class =====

  * I built the VideoConverter using the old implementation as mentioned above.

  * Official ffmpeg filters documentation: http://ffmpeg.org/ffmpeg-filters.html

  * Added the following script definition: <sxh javascript; highlight: []>    "vconv": {
      "custom_python_env": "media_env",
      "cmd": "${PYTHON} ${PROJECT_ROOT_DIR}/nvh/media/video_converter.py",
      "python_path": ["${PROJECT_ROOT_DIR}", "${NVP_ROOT_DIR}"]
    },</sxh>

  * So now we can convert video files with a command such as: <sxh bash; highlight: []>nvp vconv convert -s -n</sxh>

  * => This will process all the valid video files found in the current working directory.

===== Sharpening filter =====

  * In the conversion process, I also added support for a **-p** flag to add some sharpening post process on the image: this is improving the quality of the image a little after the stabilization processing, but it's also increasing the size of the files significantly, and thus, I'm not using this for my default video processing pipeline.

<note>If really needed I could still increase the resolution of my videos (with some AI processing stage 😄 ?) and apply the sharpening afterwards anyway</note>

===== Displaying videos side by side =====

  * I found this page on displaying 2 half videos side by side: https://stackoverflow.com/questions/35349935/ffmpeg-crop-with-side-by-side-merge
    => This could be usefull to validate the video processing.

  * Adding the command **hsxs** to display half videos side by side: <sxh python; highlight: []>    def hstack_files(self, inputs):
        """Generate hstack video from a list of input video files"""
        tools: ToolsManager = self.get_component("tools")

        ffmpeg = tools.get_tool_path("ffmpeg")

        # For now assume we only have 2 files:
        self.check(len(inputs) == 2, "Invalid number of input files.")

        # We add support here for lens correction:
        base_cmd = [ffmpeg, "-threads", "8"]
        for fname in inputs:
            base_cmd += ["-i", fname]

        base_cmd += [
            "-filter_complex",
            "[0:v]crop=iw/2:ih:0:0[left]; [1:v]crop=iw/2:ih:ow:0[right]; [left][right]hstack",
        ]

        # Ouput file name:
        out_file = Path(inputs[0])
        out_file = out_file.with_suffix(".hstack" + out_file.suffix)
        base_cmd += [out_file]

        logger.info("Executing command: %s", base_cmd)
        res, rcode, outs = self.execute(base_cmd)

        if not res:
            logger.error("hstack video generation failed with return code %d:\n%s", rcode, outs)</sxh>

  * And now I can generate side by side half videos with a simple command such as: <sxh bash; highlight: []>$ nvp vconv hsxs DSCF0066.MOV DSCF0066.x265.mkv</sxh>