ffmpeg-templates

Installation
Usage
Getting Started
- Template Syntax
- Typescript Interface
Documentation

Installation

Requires ffmpeg and deno >= 1.5.0

deno install --allow-read --allow-run --unstable -f https://raw.githubusercontent.com/andykais/ffmpeg-templates/main/ffmpeg-templates.ts

Usage

ffmpeg-templates v0.2.0

Usage: ffmpeg-templates <template_filepath> [<output_folder>] [options]

ARGS:
  <template_filepath>                       Path to a YAML or JSON template file which defines the structure of
                                            the outputted video

  <output_folder>                           The folder in which the output and generated assets will be saved to.
                                            When not specified, a folder will be created adjacent to the template.

OPTIONS:
  --preview                                 Instead of outputting the whole video, output a single frame as a jpg.
                                            Use this flag to set up your layouts and iterate quickly.

  --open                                    Open the outputted file after it is rendered.

  --watch                                   Run continously when the template file changes. This is most useful
                                            in tandem with --preview.

  --develop                                 Alias for running "--watch --preview --open"

  --quiet                                   Do not print a progress bar

  --debug                                   Write debug information to a file

  --help                                    Print this message.

Getting Started

# create a video from a template
ffmpeg-templates template.yml output.mp4

# render a single frame from the output at the specified timestamp
# and continuously re-render when the template file is changed
ffmpeg-templates template.yml output.jpg --render-sample-frame 00:00:03 --watch

Template Syntax

A video project is defined using a template file like this example one below. Think of this like the video editor GUI. Full documentation exists here

clips:
  # specify clips in an array, the only field necessary is 'file'
  - file: './some-neato-video.mp4'
  
  - file: './another-clip.mp4'
    layout: # use the layout field to position and size the clip in the output
      width: '50%' # lots of fields can accept percentages of the total size of the output
      x:
        offset: '12px' # regular pixel inputs are also accepted most places
        align: 'right' # snap this clip to the righthand side
    crop: # sometimes you may want to crop a clip, this is also optional
      left: '10%'
      
  - file: './something-really-long.mp4'
    id: 'BACKGROUND_VIDEO' # you can specify an id for the timeline below
    speed: '50%' # slow down or speed up a video
    trim: { end: 'fit' } # this video is too long, so lets make sure it is trimmed to the length of the next-longest video

# by default, all clips are started at the same time, but you can use the timeline to change up that order.
# Lets start one video in the background, and then play two other clips on top of it, one after the other.
timeline:
  00:00:00:
    - ['BACKGROUND_VIDEO']
    - [CLIP_0, CLIP_1]

Javascript Interface

import { render_video, render_sample_frame } from 'https://raw.githubusercontent.com/andykais/ffmpeg-templates/main/lib/mod.ts'


const template = { clips: [{ file: './input.mp4' }] }
const output_folder = 'output'
const options = { cwd: '~/Projects' }
await render_video(template, output_filepath, options)

Motivation

So why does this exist? There are countless video editors out there, and this little command line program cant possibly match their feature set, so why use this?

In the end, it comes down to your opinions on GUI programs. I believe that there is a lot less interesting things that can be done with a traditional video editor. This isn't to say this little program can do more, but it does open the door for reusability, and automation in ways that a GUI never could. For instance, if I wanted to truly make a template where one video is swapped out for another, its a single line change in the template file. Doing the same thing inside a GUI program is not nearly as trivial. It would mean opening a project file, massaging the new clip to the same shape, size and length as an old clip, and placing it again.

ffmpeg-templates is really just nicer syntax on top of what the ffmpeg program already offers, but it offers an easy to pick up syntax and schema. Everything that this program can do, is defined in a single schema file. No complicated tutorials, no hidden settings in a application preferences. Its just a bare bones video editor.

Roadmap

(please ignore this mess)

  2s [----------------------------------------------] 0.0%error: Uncaught (in promise) Busy: Resource is unavailable because it is in use by a promise
    at processResponse (deno:core/core.js:223:11)
    at Object.jsonOpSync (deno:core/core.js:246:12)
    at Object.consoleSize (deno:runtime/js/40_tty.js:7:17)
    at progress_callback (ffmpeg-templates.ts:60:30)
    at copied_options.progress_callback (ffmpeg-templates.ts:96:54)
    at ffmpeg (mod.ts:508:9)
    at async render (mod.ts:645:3)
    at async render_sample_frame (mod.ts:659:10)
    at async try_render_video (ffmpeg-templates.ts:103:9)

make YAMLErrors recoverable
make background color a parameter
add border param
add transitions
- cross fade
- screen wipe?
crop width/height percentage instead of left vs right?
input error on crop <= 0
make framerate configurable framerate: { fps: 60, smooth: true }
cache some of the font asset creation work
add -pix_fmt yuv420p to get better compatability
trim.stop_total where stop is performed against total time.
when showing preview, hint with the next keyframe (not really helpful actually)
cancel previous render when file is changed
improve preview time by seeking inputs right to the desired frame. Unsure if there are implications, but it should work!
if we run into more memory topping out issues, we can try out segment muxer (or possibly do it by hand)
reverse the stacking order. Lower things in the timeline appear on top of other things. It makes more sense
some kind of audio cues in the preview window. Its impossible to line up audio only clips with the preview system right now
secondary command for building music videos? General workflow would be:

ffmpeg-music-video <audio_file>
> "Tap [spacebar] whenever you hear a beat that you want a clip to start at. Ready? Y/n"
> ...[space]..[space]....[space]...etc
> "3 marker(s) recorded for clips. Preview markers, reset markers, or add clips? Pp/Rr/Cc"
> "Input a folder containing clips. Clips will be added in alphabetical order:" <clip_folder>
> "Clips and timeline have been recorded to template.yml. Render video now? Y/n"

audio visualizations to help with previews? https://www.youtube.com/watch?v=M58rc7cxl_s
template.layers or template.stack or template.order to separate the vertical ordering from the timeline? Not necessary, but possibly more clear
add CI testing (install ffmpeg via apt, or use a cache: https://stackoverflow.com/questions/59269850/caching-apt-packages-in-github-actions-workflow)