/dpinterview

AV (Audio-Video) processing pipeline, used to generate QC Reports for the AV files.

Primary LanguagePythonApache License 2.0Apache-2.0

AV Pipeline

This is an AV (Audio-Video) processing pipeline, used to generate QC Reports for the AV files. Designed to support Zoom recordings, the pipeline can be extended to support other AV files as well.

Architecture

alt text

Features

  • Supports decrytion of encrypted AV files
    • Uses cryptease to decrypt the encrypted AV files
  • Extracts Metadata from the AV files using ffprobe
  • Detects if videos have black-bars using opencv
  • Split Interviews into Left and Right Streams using ffmpeg
    • Assumes that the video has a 2 Column layout
    • Each column is a different speaker (Interviewer and Subject)
  • Extracts facial features using OpenFace
  • Generates a QC Report for the AV files
    • The QC Report includes the following:
      • Metadata
      • Facial Features
      • Openface Face in Frame
      • Openface Confidence

Tech Stack

  • Python
  • OpenCV
  • FFmpeg
  • OpenFace
  • Postgres (Database)