GEGL video editing system

gcut is a video editing engine for GEGL. It permits re-arranging bits of video with clean cuts and crossfades, as well as applying filtering chains to individual clips or as global filters on top of the edited video. At the moment gcut is primarily a data-model and file format, it can be used to programatically generate video that are then rendered. There is also an experimental user interface that can be used to augment and at some point replace understanding gcuts text file syntax.

Features

  • single-track editing, with cross fades

  • keeps source audio when editing

  • tuning of in/out points.

  • video audio correctly cut

  • shuffling of clips

  • background render processes rendering target-resolution cached frames.

  • proxy media support, permits editing with scaled down files for interactive speeds when editing workstation cannot deal with fullhd / 4k source footage at interactive speeds.

  • ui preview renderer using proxies - in separate thread for ui drawing/interaction; to keep it from blocking interaction.

  • drag and drop of media from desktop file managers

  • creating and editing per clip op-chains

  • interactive tuning scalar/string/boolean properties of ops

  • animating scalar properties of ops (only linear interpolation/key-framing for now)

  • timestamped auto-save

See [gcut.h](https://git.gnome.org/browse/gegl/tree/gcut/gcut.h) for more up-to-date current plans/todos/fixme.

Example output

The GEGL video from Libre Graphics Meeting 2016 in London, https://www.youtube.com/watch?v=GJJPgLGrSgc was made from raw footage using gcut, the default testproject of gcut which is in this repo as default.edl produces the following video when rendered the first time (TODO: update this video with newer render): https://www.youtube.com/watch?v=n91QbTMawuc

Graphical User Interface

The UI is written using microraptor gui, using lightweight CSS styling for text and cairo for drawing the interface direct from data structures.

The current UI is the minimal amount of UI needed for keyboard centered editing with some mouse based scrubbing and positioning. The available keyboard actions are different for the first and last frames of a clip compared with the mid-clip frames, when jumping between clips with up/down arrows, one jumps between the first frames of clips. The filter editing UI is not yet fully usful for more than visualizing data created in a text-editor on the project file.

The basic view, with the F1 keyboard shortcut help overlaid over the video.

gcut with help

In this screenshot, showing the purely synthetic used gegl operations default project of gcut. Showing visualization of keyframed parameters and permit setting them through sliders. Slanted clip transitions indicates cross-fades. Note that the current UI is the first attempt at a direct mapping of the file format.

more complex ui

The amount of code, written in C is about equally divided 50/50 between the core rendering logic in C and the UI, both about 3000 lines. All the actual work and flexibility is provided by GEGL itself, new operations created for one application become automatically available in possibly all of GIMP and gnome-photos as they are added to GEGL the system. Two ops in particular in GEGL which trace their history back to a pre-GEGL 2004era video editor called bauxite, [gegl:ff-load.h](http://gegl.org/operations/gegl-ff-load.html) and [gegl:ff-save.h](http://gegl.org/operations/gegl-ff-save.html) which provide the ability to load, decode, endcode and save GEGL buffers a video frames, and associated audio - with playback and seeking oriented caching mechanisms. It could be possible to add alternatives to these operations using for instance gstreamer, and the rest of gcut would remain unchanged.

It is planned to, eventually, rewrite the UI part from C to lua, even if even this C ui is very young, it will however use the same toolkit, cairo and microraptor gui, aiming for shorter interaction cycles during development and less fragile code and opening up for easier outside contributions, as well as learning from mistakes and short comings in the UI prototype proof of concept written in C, with mrg and cairo, anything that can be drawn can be made interactive, thus at least editing animation curves could be made a lot more visual and interactive.

File format

An example gcut edl file is as follows:

output-path=result.mp4
A.mp4 200 341
A.mp4 514 732
B.mp4 45 123

If this was stored in a file, test.edl we can run:

$ gcut test.edl render

And gcut will put video and audio content belonging to times from frame no 200 to frame no 341, followed by frames from not 514 to 732 subsequently followed with frames 45-123 from another file B.mp4

if you just run:

$ gcut test.edl

gcut will launch in UI mode, videos can be added by drag and drop from file manager if starting out from scratch.

when quitting gcut will have overwritten the original file with something like the following:

output-path=example-output.mp4
video-width=1920
video-height=1080
fps=59.940060
frame-scale=1.293607
t0=0.000000
frame-no=311
selection-start=216
selection-end=216
A.mp4 200 341
A.mp4 514 732
B.mp4 45 123

The output settings for video-width, video-height and fps have been detected from the first video clip - gcut works well if all clips have the same fps.

After each clip a gegl image processing chain following the format documented at http://gegl.org/gegl-chain.html

One can for instance write:

A.mp4 200 341 -- gaussian-blur std-dev-x=0.1rel std-dev-y=0.1rel gegl:threshold value=0.3

To blur and threshold A.mp4, this feature can be used for using arbitrary GEGL pipelines with interpolated parameters as filters on a video clip. The suffix rel used in the gaussian blur is dependant on the height of the video - this permits the pipeline to be used for proxies as well as for full size video.

Values can also be animated by supplying them inside inside curly brackets, containg keyframe=value pairs in a clip local interpolated time space {0=3.0 3=0.2 10=}. The format for the animated properties are likely to change as the current place-holder linear only format is supplanted.

Entries without a path, but only a filtering part like:

— 10.5s 20s invert-gamma

would insert a chain of invert-gamma from 10.5s until 20s. These entries need to be placed at the beginning of the file. (later, these filters will inherit the position of the insertion cursor of the timeline.)

caching architecture

The data storage model is central to gcuts architecture the user sees as a project file contains the complete description of how to generate a video for a sequence from source assets. The rest of objects used during processing can be regenerated deterministically under the same hashed file names.

This file is broken down into a set of global assignments of key/values, and lines describing clips with path - and filter overlays, with in/out points and optional associated GEGL filter stacks.

gcut keeps cached data in the .gcut subdir in the same directory as the loaded gcut project file, All the projects in a folder share the same .cache directory. The cache data is separated in subdirs for ease of development and debugging.

.gcut/cache/ contains the rendered frames - stored in files that are a hash of a string containing , source clip/frame no and filter chain at given frame. Thus making returns to previous settings reuse previous values. This folder can be removed but all cached frames are lost - gcut should have a feature to remove all non-cached frames.

.gcut/history/ contains undo snapshots of files being edited (backups from frequent auto-save)

.gcut/proxy/ contains scaled down to for quick preview/editing video files

.gcut/thumb/ contains thumb tracks for video clips - thumb tracks are images to show in the clips in the timeline, the thumb tracks are created using iconographer from the proxy videos.

when the UI is running the following threads and processes exist:

gcut project.edl mrg ui thread (cairo + gtk/raw fb) GEGL renderer/evaluation thread

gcut project.edl cache 0 4 background frame cache renderer processes

gcut project.edl cache 1 4 if frameno % 4 == 1 then this one considers

gcut project.edl cache 2 4 it its responsibility to render frameno, the

gcut project.edl cache 3 4 count of such processes is set to the number of cores/processors available.

The background renderer processes are stopped when playback is initiated, as well as every 60 seconds, when a new set of caches (restarts to handle both project file changes and possible memory leaks.)