fritzthecat-blog: Video Cut Automation with ffmpeg

When you cut videos, you normally use a graphical user-interface that facilitates loading of video files, cutting and arranging them onto a timeline, where you also can put transitions between the clips. Finally you "export" the timeline to a result video. I used the free OpenShot tool for a long time now, but when you have a 30-minutes video arranged on the timeline, it takes up to 10 seconds until a cut is actually performed (4 CPUs, 8 GB memory). That is a confusing (did it crash?) and time-consuming work. So why not automate it?

There is an open-source command line tool called ffmpeg, developed since 2000, that provides video conversion, editing, and lots more. It made a major quality jump in 2014 when over 1000 bugs were fixed. OpenShot uses it, like many other multimedia tools do. I tried to utilize it for my video cut automation, and it worked, so here is what I have to contribute.

Mind that I cover MP4 only, not MOV, WMV, AVI or other video formats.

ffmpeg Commands

You can install ffmpeg on LINUX by entering

sudo apt-get install ffmpeg

in a terminal window.

Cuts

Following would cut losslessly from second 5 to the end of video input.mp4 into a new video output.mp4:

ffmpeg -i input.mp4 -ss 0:5 -c copy output.mp4

Next would cut losslessly from hour 1 minute 2 second 20 to hour 2 minute 3 second 30:

ffmpeg -i input.mp4 -ss 1:2:20 -to 2:3:30 -c copy output.mp4

Following would cut from minute 3 second 30 to minute 4 second 40 and convert losslessly to the "transport stream" (ts) format. Such files can not be displayed, but videos with different technical backgrounds can be combined when they are in this neutral format:

ffmpeg -i input.mp4 -ss 3:30 -to 4:40 -c copy -bsf:v h264_mp4toannexb -f mpegts output.mpegts

Yes, it's a cryptic world:-)

Mind that when I say "losslessly" I don't mean without quality loss. Due to precise "output seeking" (-ss AFTER -i) it could happen that you have frozen or black images on start or end of the clip, because of missing key-frames at the cut position.
Inprecise "input seeking" on the other hand (-ss BEFORE -i) would always cut at key-frames, but you lose time precision, in range of 1 or more seconds. This is a broadly discussed topic on the internet.

After some bad experiences with non-appearing title videos when converting to MPEG/TS-format (transport stream) I would actually recommend to use the inaccurate "input seeking":

ffmpeg -ss 5 -i input.mp4 -c copy output.mp4   # removes the leading 4-5 seconds

However you use it, ffmpeg is a technical approach, and you will have to find out a little about video technology.

Joins

Following command would losslessly concatenate the videos named in filelist.txt, in case they all come from the same camera (same codecs, time-bases, ...):

filelist.txt

file GOPR1486.MP4
file GOPR1487.MP4
file GOPR1488.MP4

ffmpeg -f concat -i filelist.txt -c copy output.mp4

The next command would losslessly concatenate the videos when they have been generated as mpegts (transport stream) because they came from different cameras:

filelist.txt

file GOPR1486.mpegts
file GOPR1487.mpegts
file GOPR1488.mpegts

ffmpeg -f concat -i filelist.txt -c copy -bsf:a aac_adtstoasc -brand avc1 -f 3gp output.mp4

These 5 commands should be enough to facilitate automation of a video cutting-plan.

Use Case

My way to produce a video:

Write a cutting-plan while watching the video clips in some good video player (not OpenShot), resulting in a list of video file names, each with 1-n interesting time sections

Cut out the interesting time sections, following the cutting plan

Arrange the cuts onto a timeline, putting transitions between them

Prepend a title

Extras: add text to some scenes, maybe music, do some fade-in and fade-out, ....

Watch the result, do corrections

Export it, i.e. the application mixes together a video from the timeline

Mostly I have just videos from my action camera, so I rarely need conversions to combine clips with different codecs. I copy all videos from the camera into one directory that I name according to the event and its date.

Analysis

Step 1 is manual work, but can result in an machine-readable list of cuts to do.

Step 2 can be done losslessly by ffmpeg.

Step 3 can be done by ffmpeg, lossless, except transitions, this goes into pixels and would require re-encoding.

All other steps can be done by OpenShot, and this doesn't take too much time (select all loaded videos, click context-menu "Add to Timeline"). As support for controlling the the cutting-plan, the below script joinVideos.sh can produce a draft video (without title and transitions) very quickly.

Cutting Plan

Here is an example cutting-plan that I used for development. It shows all the possibilities you have to specify video cuts. Concept was to allow free text everywhere, and make video names and start/end times recognizable easily.

cuts.txt

GOPR1486.MP4
0:0:3.123 - 0:0:8.234  Full ffmpeg time strings, hours:minutes:seconds.millis
0:22 - end             A keyword expressing "until the end"

Following video will be copied as a whole:
GOPR1487.MP4
all

GOPR1488.mp4
0:20 0:25        You don't need to put the "-" in between start and end

If you indent a video, it will be ignored, but be sure to also indent start/end times:
  20200905_133128.mp4
  0:0 - 0:26

This cutting-plan specifies four videos: GOPR1486.MP4, GOPR1487.MP4, GOPR1488.mp4, 20200905_133128.mp4.
GOPR1486.MP4 has a cut from second 3.123 to second 8.234, and from second 22 to the video's end.
The entire GOPR1487.MP4 will be taken.
GOPR1488.mp4 specifies a cut from second 20 to 25. The optional "-" was omitted here.
20200905_133128.mp4 and its time specification will be ignored due to leading spaces.

File and Directory Structure

I always call my cutting-plan cuts.txt and place it in the same directory as the videos to cut. The video-cuts produced will be placed into a cuts sub-directory there. I let the script overwrite any existing directory. Here is an example directory structure for video cutting like done by the script below. The video directory testvideos contains videos to cut (maybe some more than just these), and a text file called cuts.txt.

testvideos

20200905_133128.mp4

GOPR1486.MP4

GOPR1487.MP4

GOPR1488.mp4

cuts.txt

cuts

001_GOPR1486_CUT.MP4

002_GOPR1486_CUT.MP4

003_GOPR1487_CUT.MP4

004_GOPR1488_CUT.MP4

The clips sub-directory inside testvideos was produced by running the cut-script (see below) with the example cuts.txt cutting-plan mentioned above. You see that two clips have been extracted from GOPR1486.MP4, and 20200905_133128.mp4 has been ignored.

Implementation

After some experiments I decided for an awk script, embedded into a UNIX shell script (awk is available with CYGWIN shell also for WINDOWS). awk is ideal for processing non-complex and line-oriented text like a cutting-plan.

→ All scripts below expect the directory containing the video files as first parameter.

cutVideos.sh

This UNIX shell script with nested awk script performs the cuts specified in cutting-plan cuts.txt:

#######################################################
# Carries out a cutting plan containing several videos,
# each video can have a list of cuts.
#######################################################

which ffmpeg >/dev/null || {
	echo "You must have ffmpeg installed to cut videos with this script!" >&2
	exit 1
}

[ -z "$1" ] && {
	echo "SYNTAX: $0 videoDir/[cuts.txt] [-mpegts]" >&2
	echo "	All files.MP4 must be in same directory as cuts.txt is." >&2
	echo "	cuts.txt contains video names and start - end times of cuts below it." >&2
	echo "	Example:" >&2
	echo "		GOPR0123.MP4" >&2
	echo "		7 - 11" >&2
	echo "		0:59 - 1:2:34.5" >&2
	echo "		1:30 - end" >&2
	echo "		GOPR0456.MP4" >&2
	echo "		all" >&2
	echo "	Lines starting with spaces will be ignored." >&2
	echo "	'End' (end of video) and 'All' (whole video) are case-insensitive keywords." >&2
	echo "	Use -mpegts option when videos have different codecs, but mind that results are not .MP4 then, they are to be joined by joinVideos.sh!" >&2
	exit 2
}

if [ -d $1 ]	# assume default cuts.txt
then
	videoDir=$1
	cuttingPlan=cuts.txt
elif [ -f $1 ]	# explicitly named cutting plan
then
	videoDir=`dirname \$1`
	cuttingPlan=`basename \$1`
else
	echo "Could not find directory or file $1" >&2
	exit 3
fi

if [ "$2" = "-mpegts" ]	# write cuts in transport stream format for concat.sh
then
	mpegts=true	# this is needed when videos come from different sources
else
	mpegts=false
fi

cd $videoDir || exit 4

cutsDir=cuts

# remove all former clips
echo "Removing old cuts-directory $cutsDir in `pwd` ..." >&2
rm -rf $cutsDir || exit 5
mkdir $cutsDir || exit 5

# read the cutting plan and execute it
awk '
	BEGIN	{
		IGNORECASE = 1	# make all pattern matching case-insensitive
		toMpegTs = ("'$mpegts'" == "true")	# mpegts comes from underlying shell
		cutsDir = "'$cutsDir'"
		fileNr = 0
		
		if (exists("TITLE.MP4"))
			if (toMpegTs)	{	# must convert to transport-stream format
				videoFile = "TITLE.MP4"
				addCommand("0:0", "end")
			}
			else	# no need to cut title
				system("cp TITLE.MP4 " cutsDir "/000_TITLE_CUT.MP4")
	}
	
	function nextClipFile(fileNr)	{	# build the name "001_GOPR01234_CUT.MP4"
		videoClipFile = toupper(videoFile)
		sub(/\.MP4$/, "", videoClipFile)	# remove extension
		sortNumber = sprintf("%03i", fileNr)	# zero-padded sort number
		videoClipFile = sortNumber "_" videoClipFile "_CUT.MP4" (toMpegTs ? "TS" : "")
		
		return cutsDir "/" videoClipFile	# cutsDir comes from underlying shell
	}
	
	function addCommand(fromTime, toTime)	{
		if (toTime ~ /^end/)	 {
			checkWithinVideo(calculateSeconds(fromTime))
			toTime = ""		# take all until end
			duration = ""
		}
		else	{
			durationSeconds = checkFromTo(fromTime, toTime)	# checks both for correctness and existence
			toTime = "-to " toTime
			duration = "-t " durationSeconds
		}
		
		mpegts = toMpegTs ? " -bsf:v h264_mp4toannexb -f mpegts " : ""
		
		fileNr++
		
		# output seeking by decoding, slow, fails with MPEGTS
		# commands[fileNr] = "ffmpeg -v error -y -i " videoFile " -ss " fromTime " " toTime " -c copy -avoid_negative_ts 1 " mpegts nextClipFile()
		
		# input seeking by keyframes, fast, not precise, works with MPEGTS
		commands[fileNr] = "ffmpeg -v error -y -ss " fromTime " " duration " -i " videoFile " -c copy -avoid_negative_ts 1 " mpegts nextClipFile(fileNr)
	}
	
	function executeCommand(command)	{
		print command
		exitCode = system(command)
		if (exitCode != 0)
			executionError("Command failed with exit " exitCode ": " command, exitCode)
	}
	
	function checkFromTo(from, to)	{
		fromSeconds = calculateSeconds(from)
		toSeconds = calculateSeconds(to)
		
		if (fromSeconds < 0)
			error("Begin-time " fromSeconds " is negative: " $0, 7)
			
		if (fromSeconds >= toSeconds)
			error("Begin-time " from " is greater or equal end-time " to, 7)
		
		checkWithinVideo(toSeconds)
		return toSeconds - fromSeconds
	}
	
	function checkWithinVideo(timeInSeconds)	{
		if (timeInSeconds > 0)	{	# no need to check zero
			durationCommand = "ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 " videoFile
			durationCommand | getline videoSeconds
			close(durationCommand)
			if ( ! videoSeconds )
				error("Could not read video seconds from " videoFile)
			
			print "Checking if " videoFile " having " videoSeconds " seconds contains " timeInSeconds >"/dev/stderr"
			if (timeInSeconds >= videoSeconds)
				error("Time " timeInSeconds " is out of video bounds (" videoSeconds " seconds): " $0, 7)
		}
		return videoSeconds
	}
	
	function calculateSeconds(time)	{
		numberOfTimeParts = split(time, timeParts, ":")
		resultTime = 0
		for (i in timeParts)	{
			timePart = timeParts[i]
			if (timePart !~ /^[0-9\.]+$/)
				error("Invalid time part: " timePart, 7)
			
			resultTime += timePart
			if (i < numberOfTimeParts)
				resultTime *= 60
		}
		return resultTime
	}
	
	function error(message, exitCode)	{
		print "ERROR in '$cuttingPlan' at line " NR ": " message >"/dev/stderr"
		quit(exitCode)
	}
	
	function executionError(message, exitCode)	{
		print "EXECUTION ERROR: " message >"/dev/stderr"
		quit(exitCode)
	}
	
	function quit(exitCode)	{
		errNo = exitCode	# make END do nothing
		exit exitCode
	}
	
	function exists(filePath)	{
		return system("test -f " filePath) == 0
	}
	
	
	/^[a-zA-Z0-9_\-]+\.MP4[ \t]*$/	{	# next video file
		videoFile = $1
		if ( ! exists(videoFile) )
			error("file does not exist or is empty: " videoFile, 8)
	}
	
	/^[0-9]+:?[0-9\.]*[ \t]/	{	# next start - end times of a clip to extract
		if (videoFile)	# there was a video file name before
			addCommand($1, ($2 == "-" ? $3 : $2))
		else
			error("Found start - end time without video file: " $0, 9)
	}
	
	/^all/	{	# copy the whole video as clip
		addCommand("0", "end")
	}
	
	END	{
		if ( ! errNo )	# error() would set this
			if ( ! fileNr )
				error("No video cuts were found in '$cuttingPlan' !", 9)
			else
				for (c in commands)	# execute all collected commands
					executeCommand(commands[c])
	}
' $cuttingPlan || exit $?

echo "Generated cut videos in `pwd`/$cutsDir"

It would take too much space explaining the whole script. (Essentially there is a lot of code here that just checks and documents usage.) Here are the most important things:

If you have several cutting plans, or it is not called cuts.txt, you can append the plan name to the videoDir given as first parameter to the script

The script will replace any existing cuts sub-directory (where the clips are to be placed)

Resulting clips will be named uppercase with a leading sort-order number

Video file names must comply to this regular expression:
/^[a-zA-Z0-9_\-]+\.MP4[ \t]*$/ (letters, digits, underscore, minus, nothing than spaces at line end),
that means you can not have free text on the line where the video name is

The script will terminate with error when
- a video doesn't exist or is empty
- a cut's begin time is after or equal to the end time
- a cut time is not within the video's duration

The ffmpeg command is built together in function addCommand()

When a file named TITLE.MP4 exists in the directory of the videos to cut, the script will copy it into the cuts sub-directory and give it the name 000_TITLE_CUT.MP4, to be the first clip of the result video assembled by joinVideos.sh (clips start with "001_").

joinVideos.sh

This second script is to join video cuts. It builds on some naming conventions created by cutVideos.sh.

#######################################################
# Joins together video clips created by cuts.sh.
#######################################################

checkCompatiblity="false"  # "true" when using different cameras!

[ -z "$1" ] && {
	echo "SYNTAX: $0 videoDir" >&2
	echo "	Joins all _CUT.MP4 videos in given videoDir/cuts." >&2
	echo "	videoDir: directory of original videos where 'cuts' directory is below." >&2
	exit 1
}

[ -d "$1" ] || {
	echo "Given videoDir is not a directory: $1" >&2
}

cd $1/cuts || exit 2	# "cuts" is a naming convention used in cutVideos.sh

allCutsVideo=$1/`basename \$1`.MP4
echo "Creating $allCutsVideo ..." >&2

concatFile=concat.txt
rm -f $concatFile

formatProperties()	{	# $1 = property name, $2 = video file path
	ffprobe -v error -show_entries format=$1 -of default=noprint_wrappers=1 $2 | sort | uniq | sed 's/^/format /'
}

streamProperties()	{	# $1 = property name, $2 = stream name, $3 = video file path
	ffprobe -v error -select_streams $2 -show_entries stream=$1 -of default=noprint_wrappers=1 $3 | sort | uniq | sed 's/^/'$2' /'
}

cleanup()	{
	rm -f ffprobe1.check ffprobe2.check $concatFile
}

error()	{	# message, exitcode
	echo $1 >&2
	cleanup
	exit $2
}

cleanup

# remove preceding result video
[ -f $allCutsVideo ] && {
	echo "Removing existing join-video $allCutsVideo ..." >&2
	rm -f $allCutsVideo || exit 4
}

for videoClip in `ls -1 *_CUT.MP4* | sort`	# naming convention used in cutVideos.sh
do
	case $videoClip in
		*.MP4TS)
			[ "$mpegts" = "false" ] && error "Can not mix .MP4 and .MP4TS files: $videoClip" 6
			mpegts=true
			;;
		*.MP4)
			[ "$mpegts" = "true" ] && error "Can not mix .MP4TS and .MP4 files: $videoClip" 7
			mpegts=false
			;;
	esac
	
	[ "$mpegts" = "false" -a "$checkCompatiblity" = "true" ] &&	{	# check if different video properties
		echo "Compatibility check: $videoClip ..."
	
		if [ -f ffprobe1.check ]
		then
			checkfile=ffprobe2.check
		else
			checkfile=ffprobe1.check
			firstClip=$videoClip
		fi
		
		formatProperties format_name $videoClip >$checkfile
		streamProperties codec_name,profile,time_base,pix_fmt,r_frame_rate,width,height v:0 $videoClip >>$checkfile
		streamProperties codec_name,sample_rate,channels a:0 $videoClip >>$checkfile
		
		[ -f ffprobe2.check ] && 	{	# being at second file
			diff ffprobe1.check ffprobe2.check || error "The video $firstClip (left) seems not to be combinable with $videoClip (right)!" 8
		}
	}
	
	echo "file $videoClip" >>$concatFile
done

[ -f $concatFile ] || error "No *_CUT.MP4* files found in given directory: $1" 9
cat $concatFile

# concatenate the videos
[ "$mpegts" = "true" ] && fromMpegTs="-bsf:a aac_adtstoasc -brand avc1 -f 3gp"
echo "Concatenating videos ..."

ffmpeg -v error -y -f concat -i $concatFile -c copy $fromMpegTs $allCutsVideo

cleanup
echo "Generated $allCutsVideo"

Important to know:

Clips are expected to be named *_CUT.MP4* (uppercase)

The script will terminate with error when it did not find videos in the given directory

The script checks whether the videos can be combined, i.e. have same codecs and time-bases, except when the files have a .MP4TS extension (are in mpegts format). You should remove this check in case it takes too much performance!

Videos in "transport stream" format will be recognized automatically by their .MP4TS extension

If you mix .MP4 and .MP4TS (mpegts) videos, the script will terminate with error before generating any result video

The name of the result video will be [videoDirectory].MP4 and it will reside in videoDirectory, any existing file will be overwritten without warning!

listVideos.sh

To be complete, here is a script that will provide you a cutting-plan skeleton.

#######################################################
# Lists video file names into cutting plan cuts.txt.
#######################################################

[ -z "$1" ] && {
	echo "SYNTAX: $0 videoDir" >&2
	echo "	Lists video files.MP4 into cutting plan cuts.txt." >&2
	exit 1
}

cd $1 || exit 1

cutsText=cuts.txt
[ -f $cutsText ] &&	{	# do not overwrite manually edited one
	echo "ERROR: $1/$cutsText already exists!" >&2
	exit 1
}

ls -1 *.MP4 *.mp4 2>/dev/null | sort >$cutsText

echo "Generated $cutsText video list in `pwd`"

This is useful because it sorts the videos, and there could be many. Mind that long GOPRO videos are split into 4 GB parts, and the names of the split parts do not comply to the alfanumeric sort order of the normal videos, so you need to place them manually. Still the most work will be filling that generated cutting plan with cuts.

Conclusion

Hope this will help you to reduce computer time and give you more time for recording videos - have fun!

fritzthecat-blog

Blog-Archiv

Samstag, 12. September 2020

Video Cut Automation with ffmpeg