NervHome: Videos stabilization utility

So in this article, we will document the setup of the video stabilization utility in the NervProj framework (inside the NervHome project). This utility will be based on the former version written in bash scripts and converted into a modern python application in the process.

• Currently, the command I'm using to “convert video” files is as follow:

  $ nv_admin convert_movies --stabilize --gopro3p --normaudio

• This command will run the following python code:

      if cmd == "convert_movies":
          CHECK(len(params)<=3,"Unexpected number of arguments: %s" % args)
          stabilize=False
          # Should default to auto correction here:
          correctLens="auto"
          normAudio=False
          if "--stabilize" in params:
              logDEBUG("Using stabilization.")
              stabilize=True
          if "--gopro3p" in params:
              logDEBUG("Applying lens correction for gopro3+")
              correctLens="gopro3p"
          if "--no-lens-correction" in params:
              logDEBUG("Disabling lens correction")
              correctLens="none"
          if "--normaudio" in params:
              logDEBUG("Applying audio normalization.");
              normAudio=True
  
          # Before doing any video conversion we should concat the gopro files (by default):
          nvConcatGoproFiles(os.getcwd())
  
          conv = nvVideoConverter(os.getcwd(), stabilize=stabilize, correctLens=correctLens, normAudio=normAudio)
          conv.convertMovies()
          return

• ⇒ Thus I should use the nvVideoConverter as an initial reference.
• Yet, the first thing we need is the ffmpeg tool with the VideStab plugin (and support for x264/x265)
• Downloading windows version from: https://www.gyan.dev/ffmpeg/builds/
• Downloading linux version from: https://johnvansickle.com/ffmpeg/

• Installing the ffmpeg tool with the command:

  $ nvp tools install

• ⇒ Now we have ffmpeg version 2022.8.25 ready on windows and version 2022.07.22 on linux: let's try to use them now.

• I built the VideoConverter using the old implementation as mentioned above.

• Official ffmpeg filters documentation: http://ffmpeg.org/ffmpeg-filters.html

• Added the following script definition:

      "vconv": {
        "custom_python_env": "media_env",
        "cmd": "${PYTHON} ${PROJECT_ROOT_DIR}/nvh/media/video_converter.py",
        "python_path": ["${PROJECT_ROOT_DIR}", "${NVP_ROOT_DIR}"]
      },

• So now we can convert video files with a command such as:

  nvp vconv convert -s -n

• ⇒ This will process all the valid video files found in the current working directory.

• In the conversion process, I also added support for a -p flag to add some sharpening post process on the image: this is improving the quality of the image a little after the stabilization processing, but it's also increasing the size of the files significantly, and thus, I'm not using this for my default video processing pipeline.

• I found this page on displaying 2 half videos side by side: https://stackoverflow.com/questions/35349935/ffmpeg-crop-with-side-by-side-merge

⇒ This could be usefull to validate the video processing.

• Adding the command hsxs to display half videos side by side:

      def hstack_files(self, inputs):
          """Generate hstack video from a list of input video files"""
          tools: ToolsManager = self.get_component("tools")
  
          ffmpeg = tools.get_tool_path("ffmpeg")
  
          # For now assume we only have 2 files:
          self.check(len(inputs) == 2, "Invalid number of input files.")
  
          # We add support here for lens correction:
          base_cmd = [ffmpeg, "-threads", "8"]
          for fname in inputs:
              base_cmd += ["-i", fname]
  
          base_cmd += [
              "-filter_complex",
              "[0:v]crop=iw/2:ih:0:0[left]; [1:v]crop=iw/2:ih:ow:0[right]; [left][right]hstack",
          ]
  
          # Ouput file name:
          out_file = Path(inputs[0])
          out_file = out_file.with_suffix(".hstack" + out_file.suffix)
          base_cmd += [out_file]
  
          logger.info("Executing command: %s", base_cmd)
          res, rcode, outs = self.execute(base_cmd)
  
          if not res:
              logger.error("hstack video generation failed with return code %d:\n%s", rcode, outs)

• And now I can generate side by side half videos with a simple command such as:

  $ nvp vconv hsxs DSCF0066.MOV DSCF0066.x265.mkv