Combine Scroll Text Videos with ffmpeg

In my latest Blog articles I presented shell scripts to scale an image to a certain size, and to scroll a multiline text over a background image. Now let's combine these two scripts and generate a video from several text files with associated background images.

As said, you need ffmpeg and ImageMagick installed on your machine.

Preparations

Text files and background images must be associated by following naming convention:

1. A text file must have the extension ".txt"
2. The associated image must have the extension ".png" (can be changed in script line 6)
3. The intended sequence of the files must be expressed by alfabetical sort order of the files (so best prepend "99_" before any file name and change 99 to the intended order number)
4. The base name of the text file (without extension) must be the same as that of the image,
   thus a text file named 01_example.txt will be associated with the image file 01_example.png.

Here is an example for a sequence of 3 video clips with different background images, to be combined into one result video ALLCUTS.MP4:

01_putin-demagogy.png
02_kiev-destruction.png
03_bombed-flat.png
01_putin-demagogy.txt
02_kiev-destruction.txt
03_bombed-flat.txt

Script

• joinScrollTextVideos.sh

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62

################################################# # create and join videos where texts scroll over images ################################################# allCutsVideo=ALLCUTS.MP4 imageExtension=png currentDir=`pwd` cd `dirname \$0` # change to where this script resides PATH=$PATH:`pwd` # take sibling scripts into path cd $currentDir syntax() { echo "SYNTAX: $0 directory [widthxheight]" >&2 echo " Creates and joins videos where texts scroll over images." >&2 echo " Text- and image-files have same name but different extensions: .txt -> .$imageExtension" >&2 echo " directory: the folder where text and image files are" >&2 echo " widthxheight: image target dimension, default is 1920x1080" >&2 echo " CAUTION: file names must not contain spaces or any kind of quotes!" >&2 exit 1 } [ -d "$1" ] || syntax widthxheight=$2 cd $1 concatFile=concat.txt rm -f $concatFile 2>/dev/null for textFile in `ls -1 *.txt | sort` do echo "Checking $textFile for associated image ...." >&2 baseName=`basename "\$textFile" .txt` imageFile="$baseName.$imageExtension" [ -f "$imageFile" ] && { # when there is an associated image for text echo "Found $imageFile ...." >&2 scaledImageFile="scaled_$imageFile" imageToSize.sh "$imageFile" "$scaledImageFile" $widthxheight || exit 2 videoFile="$baseName.MP4" scrollTextOverImage.sh "$scaledImageFile" "$textFile" "$videoFile" || exit 3 rm -f "$scaledImageFile" echo "file $videoFile" >>$concatFile } done [ -f $concatFile ] || { echo "Found no associated .txt and .$imageExtension files in `pwd`" >&2 exit 4 } echo "Joining videos ...." >&2 ffmpeg -v error -y -f concat -i $concatFile -c copy $allCutsVideo rm -f $concatFile echo "Done." >&2

Line 5 defines the name of the resulting video.

Line 6 defines the extension of image files. Change it to "jpeg"or any other extension of your images that ImageMagick supports.

Lines 8 to 11 serve to take sibling scripts into the execution path. Make sure that the scripts imageToSize.sh and scrollTextOverImage.sh are beside joinScrollTextVideos.sh (see my recent articles for their source code).

Line 13 to 21 provides syntax documentation for erroneous calls.

Lines 23 to 29 pick up the call parameters and change to the given directory. The concatFile is needed for ffmpeg, the script removes it in case it exists.

Lines 31 to 50 loop over the sorted list of .txt files in the working directory. First the base name association is done by removing the .txt extension (line 34) and building the according image file name. If that image file exists (line 37), the script will generate a temporary 1920x1080 image from it (line 41) and combine that with the text file (line 44). The resulting video clip is appended to the concatFile as preparation for the final ffmpeg call to join all clips.

Lines 52 to 54 check whether there were videos created and exits when not.

Line 58 finally joins all video clips to one result video called ALLCUTS.MP4, here the concatFile is passed to ffmpeg. The concatFile gets removed afterwards.

Resume

This script lets produce videos quite quickly. Just write your texts, give the files an alfbetical order, choose images and give them the same base name as the text files. Everything else will be done by ImageMagick and ffmpeg.

Scroll Text Over Image with ffmpeg

Here is how to create a video that scrolls a multiline text over an image. You need to have the open source tool ffmpeg installed on your machine to do that. On WINDOWS, you also need some UNIX shell execution environment like CygWin.

Text and Image

Create a text file where you put your text into. All line breaks will be preserved. You should restrict your line length to about 36 characters, so that the text fits into the video's pixel width. You can also control this with the font size, but mind that you need a really big font to be readable on mobiles, too.

Then find an image that fits to your text.

Video by Script

Here is a script that creates a video from your text and image files, calculating the video's length from the number of text lines:

• scrollTextOverImage.sh

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41

################################################### # Creates a video where text scrolls over an image. ################################################### syntax() { echo "SYNTAX: $0 imageFile textFile videoFile" >&2 echo " Scrolls text over image, result is a video." >&2 exit 1 } imageFile="$1" textFile="$2" videoFile="$3" [ -f "$imageFile" -a -f "$textFile" ] || syntax [ -z "$videoFile" ] && syntax # calculate duration of video by number of text lines numberOfLines=`wc -l <"\$textFile"` videoSeconds=`awk 'BEGIN { print 12 + ('\$numberOfLines' + 1) * 1.5 }'` echo "Generating $videoFile that scrolls $textFile over $imageFile, lasting $videoSeconds seconds ...." >&2 # escape percent sign, it would make ffmpeg fail rm -f replaced.txt sed 's/%/\\%/g' <$textFile >replaced.txt ffmpeg -y -v error -loop 1 \ -i "$imageFile" \ -vf drawtext="\ fontsize = 70:\ fontcolor = white:\ borderw = 7:\ bordercolor = black:\ line_spacing = 60:\ textfile = \'replaced.txt\':\ x = (w - text_w) / 2:\ y = h - 80 * t" \ -t $videoSeconds "$videoFile" rm -f replaced.txt

Lines 5 to 16 define the call syntax of the script, pick up the incoming parameters and check them. Line 15 exits with syntax output when one of image or text are not existing files, line 16 when the target video file name was not given.

Line 19 finds out the number of lines inside the text file. This is the base for calculating the video length in seconds, done on line 20. Lines * 5 / 2 is equal to a multiplication with 2.5, expr does integer operations only. Asterisk '*' is a meta-character for the UNIX shell, so we need to escape it by a backslash '\'. Mind that this hardcoded factor 5 / 2 is bound to the scroll speed, and the video dimension 1920 x 1080!

So, where is the scroll speed? Lines 28 to 39 are the execution of ffmpeg. The scroll speed is the "80" on line 38. If you change this to 120, it will scroll faster, but mind that then the video will be longer than the scrolling lasts.

Following is about what happens starting from line 28. The trailing backslashes are useful to bring this very long command line into a readable shape.

The -y option on line 28 will overwrite any existing target file without getting interactive. The -v error option reduces control output to error messages.

Line 29 defines the input image via the -i option. The "quotes" are there for file names that contain spaces.

The -vf option on line 30 starts a filter-operation with name "drawtext". Its parameterization continues down to line 34, using colon ':' separators.

Line 31 gives you the text's font size. Here you can make the letters bigger and smaller. Mind that this will affect the video's length.

Lines 32 to 34 define letters to be white, with a black outline, so that it is readable over both dark and bright background images.

Line spacing on line 35 gives the distance between lines. Also this will affect the video's length.

Line 36 contains the name of the text file. It is enclosed in 'single quotes' to enable file names that contain spaces. The quotes are escaped by backslashes to hide it from the shell, else the $textFile shell variable there would not be substituted.

Lines 37 and 38 define the coordinates for the text.
The horizontal x coordinate on line 37 is defined to center the text (zero is left). The variables w and text_w are provided by ffmpeg, giving the width of the video, and the ready-calculated maximum width of the text.
Line 38 is the vertical scroll effect in y direction (zero is bottom). The variables h and t are provided by ffmpeg, giving the height and the current timestamp of the video. That means the longer the video lasts, the higher the text will move. 80 is the (hardcoded) scroll speed factor.

Line 39 finally declares the length of the generated video via the -t option, and names the output file as last command argument.

Call this script with image, text file and output video file as command line arguments. It will generate a video that scrolls the given text over the given image.

Resume

This script is just an approximation, because it contains hardcoded values. More care should be taken to calculate parameters for the ffmpeg call. Number of text lines, font size, line spacing, video heigth, scroll speed, all of these should go into the calculation of the video length.

In my next Blog I will introduce a script that can create a video out of a sequence of several background images with associated text files.

Scale Images to Same Size with ImageMagick

Scaling a number of images to same size means:

1. specifying a dimension like 1920x1080 pixels (currently the standard video dimension)
2. sizing all images inside a directory to that size
3. while keeping their aspect ratio

This is possible only by adding edges when some width or height is too small to fit the aspect ratio. In other words, every circle in an image will still be a circle afterwards (not an ellipse), but it may have colored edges on top and bottom, or at left and right, symmetrically.

Here is how to do that.

Prerequisites

You must have a UNIX shell script execution environment. Under WINDOWS you can install CygWin. Under LINUX it's normally already integrated into the operating system.

Install ImageMagick on your machine. It is available for many platforms.

Script Fragment

Put following shell script into the directory where your images are, and run it. The script assumes the images are *.png, adjust line 1 when not. Further it sizes to 1920x1080 pixels (a standard dimension), adjust line 8 when you want a different size. You can change the color of the added edges through the "-background colorOfYourChoice" option.

extension=png

for imageFile in *.$extension
do
    sized=`basename "\$imageFile" .\$extension`_sized.$extension
    echo "Scaling $imageFile to $sized ..." >&2
	
    convert "$imageFile" -resize 1920x1080 -gravity center -background black -extent 1920x1080 "$sized"
done

echo "Done." >&2

After execution of the script you will have additional images in the directory, carrying the "_sized" postfix in their names. These are the result images.

Reusable Script

Here is a shell script that sizes one image to an optionally given dimension. You define the path of the target file as command line argument, and, optionally, also the target dimension. The script will check its parameters and output its syntax when called incorrectly:

• imageToSize.sh

####################################################
# Scales an image to a fixed size using ImageMagick.
####################################################

defaultDimension=1920x1080	# standard video dimension
edgeColor=black

syntax()	{
	echo "SYNTAX: $0 sourceImageFile targetImageFile [widthxheight]" >&2
	echo "	Scales an image to width x height, keeping aspect ratio, adding edges." >&2
	echo "	sourceImageFile: the image to scale" >&2
	echo "	targetImageFile: the result image file" >&2
	echo "	widthxheight: optional target dimension, default is $defaultDimension" >&2
	exit 1
}

[ -f "$1" -a -n "$2" ] || syntax
sourceImageFile="$1"
targetImageFile="$2"
dimension=${3:-$defaultDimension}	

echo "Scaling to $dimension from $sourceImageFile to $targetImageFile ..." >&2

# call ImageMagick
convert "$sourceImageFile" \
	-resize $dimension \
	-gravity center \
	-background $edgeColor \
	-extent $dimension \
	"$targetImageFile"

Lots of lines around just one command line that does the real work. But that's how sustainable software is written.

Resume

How long have I been looking for that technique. How short is this solution. Thank you ImageMagick!

Have Source Code in Scrum Reviews

It is a widespread opinion that source code makes no sense in Scrum review meetings. A number of circumstances makes me believe that this is wrong. I think it is important to also look at source code when holding a Scrum review.

What Matters

You might have heard this saying:

"Doesn't matter how it is called as long as it works"

This is a quite destructive statement. It enables developers to solve problems in any way that comes to their mind, even if standardized solutions are available. Imagine a developer that creates a dedicated sort algorithm instead of implementing the Comparator interface. Don't believe they won't do such, they will, because → no one wants to see source code!

When it doesn't matter how source code looks, temporary workers will hack down any problem in record time. You lose code reusability, readability, maintainability. To be clear: this is not about code formatting (spaces, tabs, line lengths, ...), this is about writing code for humans, not machines.

Names matter, readability matters, reusability matters!

All of that can be assured only in review meetings: package names, class names, field names, method names, parameter names, even algorithms and design-patterns must be accurate and understandable. Don't allow acronyms and abbreviations unless they are really common.

Common Language Sense

Common Sense is at the center of Scrum. A good team is one that has a common sense over all project aspects. Source code is a project aspect, moreover it is the actual asset of software production.

Pull-request code-reviews are not enough, such is done by just two developers. The whole Scrum team must see the source code. Criticism and discussion must be possible. Developers must not hide behind their reputation as experts, they must present, explain, and convince the team. (Could require management preparation to avoid formal disputes about indentations.)

It is not acceptable that one developer works in object-oriented style, one in functional style, and the third one just reached structured programming level. They must agree on a common way to implement things, even if this takes time.

Polyglot environments allowing different programming languages are not helpful either, they much more promote the "Doesn't matter" attitude. Freedom of expression is not useful in software development.

If your software product is implemented in English, Hindi, Chinese and Spanish, you will not be able to maintain your source code without the authors. At most you could allow one language for technical solutions (English) and one for business solutions (native), but no more.

All of this must be verified in reviews with the whole team!

Read It Like a Book

Robert C. Martin stated this, referring to Grady Booch:

Clean code reads like well-written prose

Developers are responsible for simple and readable source code within their programming paradigm, be it object-oriented, functional, or structured. And, as stated above, you should not allow to mix all programing paradigms. Thus all developers must understand what their colleagues implement!

How could this be achieved when not in Scrum reviews before the whole team? When it is not readable, not understandable, not arguable, decline the source code, have it rewritten!

Framework Libraries

If the project goal is a framework library, having the source code in review meetings is an absolute must, because the library API will be the result of the project.

Management

Source code quality will affect software maintenance, which makes up 70% of all production efforts. Unfortunately quality assurance tools like Sonar will detect only a certain kind of code duplications, surely no logic duplications. And they can not verify the accuracy of class-, method- and field-names.

When the project management accepts sayings like "Doesn't matter how it is called as long as it works", it will also state "Source code doesn't matter" itself, and this is nothing more than a commitment to throwaway code. When you know how expensive these things are, you may also suspect what gets thrown away here.

Maybe the problem aren't the developers alone? How about demanding sustainability from the management?

Resume

Source code is what we live on in the long term, it's the actual asset of software. Would you dare to trade things without being assured of their sustainability? When no, demand to see source code in Scrum reviews.

Generic Java Template Driven Comparator

Sometimes we need to sort a list of menu-items after a list of label-strings. That means, the template list contains elements of another data-type than the list to be sorted after the template list.

Example

Here is a simplified MenuItem class:

public class MenuItem
{
    public final String label;
    
    public MenuItem(String label) {
        this.label = Objects.requireNonNull(label);
    }

    @Override
    public String toString() {
        return label;
    }
}

And here is a template list that contains String objects instead of MenuItem:

    public static final String CUT = "Cut";
    public static final String COPY = "Copy";
    public static final String PASTE = "Paste";

    final List<Object> sorted = List.of(
        CUT, 
        COPY, 
        PASTE);

Can we sort a list of MenuItem after that list of String?

Generic Comparator

Yes we can ;-)
We need a generic extension of TemplateDrivenComparator (see my latest article for that class):

/**
 * Serves to sort lists where the template-list
 * has a different type than the list to sort.
 */
public abstract class GenericTemplateDrivenComparator extends TemplateDrivenComparator<Object>
{
    @Override
    public int compare(Object o1, Object o2) {
        return super.compare(convert(o1), convert(o2));
    }

    protected abstract Object convert(Object object);
}

This comparator uses the generic <Object> as data-type for list elements. It then overrides the compare() responsibility to convert each of the given parameters to a possibly different object before calling super.compare(). That way any MenuItem parameter can be converted to a String, which is necessary for super.compare() to work correctly.

Here is an example application:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35

public class Main { public static final String CUT = "Cut"; public static final String COPY = "Copy"; public static final String PASTE = "Paste"; public static void main(String[] args) { final List<MenuItem> sequence = new ArrayList<>(); sequence.add(new MenuItem(COPY)); sequence.add(new MenuItem(PASTE)); sequence.add(new MenuItem(CUT)); System.out.println(sequence); final Comparator<Object> comparator = new GenericTemplateDrivenComparator() { private final List<Object> sorted = List.of( CUT, COPY, PASTE); @Override protected List<Object> template() { return sorted; } @Override protected Object convert(Object object) { return (object instanceof MenuItem) ? ((MenuItem) object).label : object; } }; Collections.sort(sequence, comparator); System.out.println(sequence); } }

Lines 3 - 5 define the labels. The main() procedure on line 7 builds a sequence of menu-items from that labels, which is to be sorted. It prints out the menu-item list on line 12.

Lines 14 - 30 define an anonymous comparator class that is made up to sort menu-items after a template list of strings. We see the template list on line 16 - 20, returned on line 24.

The convert() function on ine 27 - 29 uses the instanceof operator to find out of which data-type the given parameter is. In case it is a MenuItem, it returns its label, else it returns the template String directly.

Line 32 finally sorts the list of menu-items (physically).
Output of the application is:

[Copy, Paste, Cut]
[Cut, Copy, Paste]

Resume

Unfortunately I did not find a way how to implement this without using the instanceof operator. The Visitor pattern should be helpful in such cases, but ...

Extending Java Template Driven Comparator

Recently I introduced a comparator that can sort after a given template list. In this article I want to show how you can extend a base comparator to cover a bigger set of sortable elements without repeating the base sort order.

For comfortably editing a template sort order and mixing new elements into it, I will introduce a custom list called MergeList. This class allows to add elements relatively to a given anchor element.

Example Application

The use case covered here is sorting menu items. The base item list contains just "Cut", "Copy" and "Paste", the extension merges "Open", "Clear", "Insert" and "Delete" into it in a way that demonstrates the capacities of the MergeList API.

Following application shows how you can programmatically extend a base sort order without repeating it, by referring to existing list elements when adding new ones. This makes the implementation stable and independent from changes in the base sort order. A runtime-exception would be thrown when an anchor item is no more in the base list, but if you keep all items in constants and remove the according constant when an item is no more used, the compiler will detect such a problem.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59

import java.util.*; public class Main { public static final String CUT = "Cut"; public static final String COPY = "Copy"; public static final String PASTE = "Paste"; public static final String INSERT = "Insert"; public static final String DELETE = "Delete"; public static final String CLEAR = "Clear"; public static final String OPEN = "Open"; public static class BaseComparator extends TemplateDrivenComparator<String> { private final List<String> sorted = List.of( CUT, COPY, PASTE); @Override protected List<String> template() { return sorted; } }; public static class ExtendingComparator extends BaseComparator { private final List<String> sorted = MergeList.of(super.template()) .before(CUT).add(INSERT) .after(CUT).add(DELETE) .after(PASTE).addAll(List.of(CLEAR, OPEN)); @Override protected List<String> template() { return sorted; } }; public static void main(String[] args) { final List<String> sequence = new ArrayList<>(); sequence.add(OPEN); sequence.add(COPY); sequence.add(PASTE); sequence.add(CUT); sequence.add(INSERT); sequence.add(DELETE); sequence.add(CLEAR); System.out.println(sequence); final Comparator<String> comparator = new ExtendingComparator(); Collections.sort(sequence, comparator); System.out.println(sequence); } }

Output of this application is:

[Open, Copy, Paste, Cut, Insert, Delete, Clear]
[Insert, Cut, Delete, Copy, Paste, Clear, Open]

On top of the example class, from line 5 to 12, there are constants describing menu items. They are used in both comparators and the application main() on line 43. To see source code of the here used TemplateDrivenComparator please refer to my recent article.

The base comparator starts on line 14, the extended comparator on line 28. If I would use BaseComparator, then "Insert", "Delete", "Clear" and "Open" would be appended to the end of the item list in a random order. But if I use ExtendingComparator, the order as built from line 31 to 34 would show up, as to be seen in the output above.

Item merging happens on line 31. First, the template() list of items gets fetched from super-class. The MergeList.of() call creates a new list that can be edited comfortably (source code see next chapter).
I tell the new list to add "Insert" before "Cut" on line 32.
On line 33 I tell it to add "Delete" after "Cut".
The expression on line 34 adds "Clear" and "Open" after "Paste".

Why don't I simply use addAll() for appending "Clear" and "Open" to end? Because I want "Clear" and "Open" always being after "Paste", not necessarily at end of list. The base list (containing "Paste") may change in future in a way that "Paste" is followed by other items.
→ We should always be as explicit as possible!

Line 44 to 57 perform the use case. A list of menu items is built in a chaotic order, and printed to stdout. On line 54 the comparator is constructed, on line 56 it is applied to the list that is printed on line 57.

Merge List

Here is the source code for the MergeList used in example above. It features an intuitive API that lets you add before or after a given anchor item, adding either a single item or a list of new items.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73

import java.util.*; /** * A List that can be edited comfortably. */ public class MergeList<T> extends ArrayList<T> { /** Builds an editable MergeList from given List. */ public static <E> MergeList<E> of(List<E> elements) { final MergeList<E> result = new MergeList<>(); result.addAll(elements); return result; } /** * @param anchor required, the element at the insertion position. * @return inserter that will insert before given anchor. */ public Inserter before(T anchor) { return new Inserter(anchor, true); } /** * @param anchor required, the element at the insertion position. * @return inserter that will insert after given anchor. */ public Inserter after(T anchor) { return new Inserter(anchor, false); } public class Inserter { private final T anchor; private final boolean before; private Inserter(T anchor, boolean before) { this.anchor = Objects.requireNonNull(anchor); this.before = before; } public MergeList<T> add(T element) { return addAt(element, anchor, before); } public MergeList<T> addAll(List<T> elements) { return addAllAt(elements, anchor, before); } } private MergeList<T> addAt(T element, T anchor, boolean before) { final int index = assertAnchorIndex(anchor); super.add(before ? index : index + 1, element); return this; } private MergeList<T> addAllAt(List<T> elements, T anchor, boolean before) { final int index = assertAnchorIndex(anchor); super.addAll(before ? index : index + 1, elements); return this; } private int assertAnchorIndex(T anchor) { if (anchor == null) throw new IllegalArgumentException("Anchor can not be null!"); final int index = indexOf(anchor); if (index < 0) throw new IndexOutOfBoundsException("Index not found: "+anchor); return index; } }

This list extends the Java ArrayList and adds facilities to add items at specific positions.

A static factory for such lists is implemented on line 9.

Line 19 exposes a method that lets refer to an anchor item for subsequently adding items before it. Line 27 is the same but for subsequently adding after an anchor item. Calling these methods you get back an Inserter, that means the list is not altered yet.

The Inserter on line 32 does the list modification. It exposes just addAt() and addAllAt() methods, that means I must insert something when I receive an Inserter. Afterwards I get back the MergeList and can continue referring to other anchor items.

As the Inserter is a non-static inner class, it can call private methods of the MergeList instance. The real work is done in addAt() on line 51 and addAllAt() on line 57. The assertAnchorIndex() method on line 65 makes sure that a given anchor exists. It throws an exception when not, else it returns its index.

Resume

I hope I could give hints how to reuse sort orders of template lists. This is a frequent problem to be found in many applications.

You don't want to implement a sort algorithm, you want to implement a comparator and use Collections.sort(list, comparator), because that is much easier than re-inventing the wheel. The Java runtime library will provide the fastest available sort algorithm behind Collections.sort().