Videotized on LINUX

"We live in the YouTube ages" I read on a forum recently. True. Nothing without videos today. Tutorials, presentations, music, sports, gaming, everything has been videotized. Photos were yesterday. We like to watch moving pictures.

Using Action Cameras

So I started with one of these matchbox-sized action cameras.
One switch on front left, for turning on and off, one switch on top left, for immediate filming (without preceding switching on), one switch on left side, for activating the setup, in particular the Wi-Fi network. On back a touch-screen for maintaining camera and clips. Micro-USB plug for loading the battery, a microphone, or connecting to the computer. A Micro-HDMI port. A rechargeable battery and 64 GB Micro-SD card should give 1 1/2 hours of filming. Produces h.264 MP4 files.
Lots of accessory available to attach it to all kinds of helmets, bicycles, tripods, sticks. Expensive, of course. Never seen a camera stick for 46 Euros.

If you have a smartphone, you can download an app to remotely control the camera. This is necessary when the camera is on the helmet and you can not verify what the camera is filming. If you activate the camera's Wi-Fi, configure it to contact the app, and then run the app on the phone, it should find the camera, and you can connect after inputting the camera's password and some number proposed by the camera. After that you can fully control the camera on the phone, you see what it is filming, you can switch it on and off, you can configure it, you can watch and delete its videos.

First thing that struck me is the big amount of data that little thing produces. A Gigabyte for 5 minutes!
Second thing was the bad quality of my hiking video. Shaky shaky!
Third thing was the fact that I filmed too much and wanted to cut that thing down. When I shoot photos, I always dismiss 50 % of them. Why should it be different with video? Thus the whole thing is not worth much without editing software.

Video on LINUX

Plugged the camera to my Ubuntu LINUX via USB, turned it on (it seems to be important to first plug the USB and then turn on the camera). The LINUX USB auto-detection showed me a file system. Had to copy the files to my hard-disk to be able to play the videos.

Playing videos on LINUX seemed to be no problem, double click brought up a video player that did what was expected.

Editing videos seemed to be another story. If you know Gimp (GNU Image Manipulation Program), you know what graphical complexity means. Big big menus full of strange commands.

I looked for video software and came across this Blog article (link), written by a video professional. Was my guide to the following. I did not yet try out Natron, but I tried out Avidemux, Blender and OpenShot. To anticipate my result, I work with OpenShot. Blender is for professionals who know all the terms and acronyms of the video effect world. Avidemux seems to be a little out-of-date, and, like Blender, is more for experts than for normal people like me. By the way, OpenShot uses Blender executables to do the video work :-)

Popular Use Cases

What we want to do with our video is

1. create a title, preferably at start, with a transition into the video
2. create a trailer, preferably at end, with a transition from the video into the trailer
3. concatenate several videos to a big one
4. remove parts of a video that are not needed, have been done twice, were not intended, ...
5. use music (an audio clip) instead of the original sound
6. insert photos to be shown between clips
7. fade clips (and audio pieces) into each other
8. rotate sloped clips, it is hard to always keep the camera horizontal
9. remove that "shaky shaky" from out hiking- or cycling-video (this software is called stabilizer, it is not standard, and it is related to a workflow where you need to mark the region that should be stable)
10. .... (lots of things I do not know yet)

Editing Workflow

Generally, with any video cutter, you load video-, image- and audio-files. Then you drag them onto the timeline, which in fact is the video (movie). We could regard it to be the "movie maker workbench".

You can have several tracks on the timeline, arranged below each other. Normally only the topmost clip is shown. Thus you can show additional pictures while the original video and audio is running. You also can arrange clips on different tracks to overlap each other, using transitions, that means one clip fades out while the next one fades in.

To remove the unwanted pieces from a video clip, you need to cut it into pieces. Then you can remove the unwanted pieces.

To combine several clips to a movie, you can concatenate them on the timeline. Hopefully the software supports automatic adjustment of the concatenated clips, else you will be confronted with inputting frame numbers.

Finally you generate a video from your timeline. This is a long running task which might take one hour for a video of 30 minutes. In Blender this is called "Animate", in Avidemux "Create Job", in OpenShot "Export".

General Video Editor Weaknesses

All applications I looked at had the same usability problems.

• They do not report whether they are working, or they are ready to take the next input. When you set a cut, this takes up to ten seconds. During that time you could do other inputs, but the UI would not respond to them. This raises the impression that the software is crashed / frozen. I would like to call this "responsiveness", but this term currently has been occupied by web development (designating flexible layouts).


• We would like to do our computer work, in any application, by using ADD, DELETE, SELECT, CUT, COPY, PASTE (DRAG and DROP, or MOVE, is just another style to do CUT, COPY, PASTE). None of the tools I've been looking at had something like CUT, COPY, PASTE. Moreover in some, ADD and DELETE were buried under tons of other menu items at places where you would not expect them. We expect these actions to be in the "Edit" menu. SELECT is done by mouse click or mouse drag, the selected region or item must be visually marked, and subsequent actions must refer to that selection.


• ESCAPE has no effect. They do not act upon ESCAPE key to abort the current action, instead they finish it. ESCAPE is often pressed while dragging the mouse. It is a common user convenience to cancel anything when ESCAPE was pressed.


• Expert addiction. We would like to have menus that support our use cases. Instead we are confronted with terms we never heard of, big lists of formats we need half a day to understand, techniques are offered but it is hard to find out how they can play our use cases.


• Crashes. Memory must be observed and managed. These are not Java applications, they contain explicitly implemented memory logic. At least shut down softly to save the state of work.

Oh my, this looks like a general computer user petition.
The video editor area exposes the gap between users and expert technicians in an exemplary way.

OpenShot

Now this is my hope for the future. I configured it to auto-save every minute (unfortunately NOT default) to not lose my work by crashes. Within a short time I could cut my videos and the result was lossless and nice. Added titles, generated transitions between clips, set cuts with 'c' or toolbar button, all without crashes. Only when I rotated a clip and then dared to do several clicks (without response) it said Goodbye.

Caution: You need patience when working with OpenShot (or any video software that does not report long lasting background work). Such a tool has to read Gigabytes of data to provide a video clip for cutting. As soon as you load a video with OpenShot you will have to wait, when setting a cut you will have to wait, when you want to preview your work you will have to wait, even when skipping backward the built-in video player it is slow. But OpenShot rarely dies.

Here are some screenshots to get an impression.

User Interface

This is the UI, top left the files in use, right the video preview, on bottom the timeline with clips. When you create a title, it manifests as image file.

Fading

This shows how to fade clips. You right-click the mouse on the clip you want to fade, choose "Properties" from the popup menu, a dialog opens, change to the "Video" tab and click the checkboxes for fading in and/or out.

Rotation

One of my clips was sloped, the horizon was not horizontal. I applied an effect which is called "Fixed Rotation". After testing around with these values it did what I expected. The preview to the left seems to not work.

Of course the edges of the video were also sloped then, so I added a "Mask" effect.

This is the "Mask" effect configuration for my slight rotation. To find the correct values you need to experiment.

Don't forget to choose your target format when generating (exporting) the final video.

Blender

Written in Python. To get near to video cutting with Blender, you first need to configure it to be a video editor (basically it is a 3D animation software). So I watched some tutorials and tried to learn quickly.

The menus (there are lots of menus) are so big and confusing that it is even hard to find out how to remove a clip from a timeline (it is called "Erase").

The user interface is far from being intuitive. To select a clip you must use the right mouse button. Before you can drag a clip to another location you must press 'g' and then click into the clip. When you then release the mouse button, it is still dragging. Blender calls clips "strips".

All in all I managed to cut down my video, and replace the audio track by some nice music, but when I generated the result video I was confronted with a merciless big list of output formats and options I could not understand. I tried the defaults and out came (after a long time) a video that had lost its quality (sharpness). Did not want to publish such.

Don't use the 'c' key for cutting, in Blender it is 'k', or Shift 'K' (soft or hard cut). Once, I was happily cutting, suddenly the thing did not set cuts any more. Just showed me frame numbers. Was this because I pressed ESCAPE before?
Had lots of such pitfalls and strange occurrences, which surely all could be easily explained by one of the experts. That was my impression: it is for experts only. So, let's send all our videos to the experts for cutting. How poor an expert world would be! But it's open source, so why do I complain !-(

Avidemux

Got the following error when loading my h.264 MP4 video:

Spent some time with Avidemux to find out how to generate the result video: you must create a "job" to do such. The output seemed to have no quality loss, but maybe this was just because I did not configure it :-)

---

JS Visibility Detection

It is not easy for a JavaScript to find out if a certain HTML element is currently visible for the user, meaning it is in the viewport of the browser window. See the discussion on stackoverflow for the great variety of answers to this question.

After reading and trying out some of these answers I decided to create my own solution of this problem. In this Blog I will present that implementation, done in pure JavaScript (no jQuery; but mind that I do not care for browsers that go their own way; Webkit browsers will work with my code).

Why would we need such functionality? Go to one of the new HTML-5 sites and watch the background picture changing when you scroll down. This is done by tracking the visibility of HTML elements. As soon as a certain element gets into view, the background image is exchanged.

Element Rectangle

In modern browsers we have an HTML element function called getBoundingClientRect(). This returns a rectangle with top, left, bottom, right (and even width and height), which is relative to the browser's viewport. The viewport is that part of the HTML page which is currently visible according to the user's scroll-position.

var element = document.getElementById("some-id");
var viewPortRelativeLocation = element.getBoundingClientRect();

To find out whether a rectangle is visible or not we need to translate this rectangle to page-absolute coordinates. Then we could check if this absolute rectangle overlaps the current browser viewport.

Browser Viewport Rectangle

Here is a function that delivers the browser's viewport in absolute coordinates (absolute means relative to the browser's client area, excluding scroll bars).

    var browserViewPort = function() {
      var rectangle = {
        left: window.pageXOffset || document.documentElement.scrollLeft,
        top: window.pageYOffset || document.documentElement.scrollTop,
        width: Math.max(document.documentElement.clientWidth, window.innerWidth || 0),
        height: Math.max(document.documentElement.clientHeight, window.innerHeight || 0)
      };
      rectangle.right = rectangle.left + rectangle.width;
      rectangle.bottom = rectangle.top + rectangle.height;
      return rectangle;
    };

This implementation is a concession to currently still sticky browser differences. I believe that in future document.documentElement.scrollLeft and document.documentElement.clientWidth will be the survivors. Always use a DRY implementation for such, then you can adapt it at any time to new browser standards.

Now to translate the element rectangle to absolute coordinates, we just need to add the browser rectangle to it.

    var getAbsoluteRectangle = function(browserRect, elementRect) {
      return {
        left: elementRect.left + browserRect.left,
        top: elementRect.top + browserRect.top,
        right: elementRect.right + browserRect.left,
        bottom: elementRect.bottom + browserRect.top
      };
    };

Intersecting Rectangles

Seems that we are almost done, but complexity is awaiting behind the corner. I considered writing a separate Blog about intersecting segments, with discussion of all variants I found about it. But to be short I simply will provide an implementation for it.

Intersecting means that the rectangles somehow overlap.

    var intersect = function(r1, r2) {
      return r1.left < r2.right &&
             r1.right > r2.left &&
             r1.top < r2.bottom &&
             r1.bottom > r2.top;
    };

This surely can not be easily understood, but it works. I will deduce this later.

Detecting Visibility

Another complexity is awaiting. An element could be nested within a scrolling container, like

<div style="height: 20em; overflow: auto;">
  ......
</div>

So we need to first check if the element is visible within its parent, and further going from parent to parent until, excluding, the document's body. Only then we can check if the element's rectangle intersects the browser's viewport.

    /** @return true when given element is at least partially visible, else false. */
    var isVisible = function(element) {
      var browser = browserViewPort();
      var rect = getAbsoluteRectangle(browser, element.getBoundingClientRect());
      
      /* could be in a scroll container, so check all parents */
      var parent = element.parentNode;
      while (parent && parent != document.body) {
        var parentRectangle = getAbsoluteRectangle(browser, parent.getBoundingClientRect());
        if ( ! intersect(parentRectangle, rect) || ! intersect(parentRectangle, browser))
          return false;
          
        parent = parent.parentNode;
      }
      
      return intersect(browser, rect);
    };

First the browser's viewport is fetched, needed for translating rectangles to absolute coordinates. Then the element's absolute rectangle is calculated. Now a loop across all parents is started, whenever the element's rectangle does not intersect the parent, or the parent does not intersect the browser's viewport, false is returned. When this all was true, also the element's rectangle is checked to intersect the browser's viewport.

Receiving Scroll Events

Now to perform the trick with changing background images we need to receive scroll events from the browser. Then we can use the function isVisible(element) to find out if a certain element is (at least partially) visible.

Receiving the main scrollbar movements is easy, we just need to add "scroll" listeners to the global JS window object. But for the nested scroll panes we need to know each of them to add a scroll listeners. You can use following code to install them.

    var initializeTracking = function(eventCallback, scrollPanes) {
      window.addEventListener("scroll", eventCallback);
      window.addEventListener("resize", eventCallback);
      
      if (scrollPanes)
        for (var i = 0; i < scrollPanes.length; i++)
          scrollPanes[i].addEventListener("scroll", eventCallback);
    };

This accepts at least one, optionally two parameters. The first is your callback function, called any time some scrollbar is moved. The second can be an array of nested scroll panes to watch.

Test Suite

This is a solution that complies with following test scenarios ...

1. element can be both smaller or bigger than the browser's viewport
2. element is ...
   • ... initially visible, but can be scrolled out of view
   • ... initially invisible, but can be scrolled into view
3. element is nested in a nested scrollable container, that itself is nested in either another container or the page

... and use cases ...

1. user scrolls the page with the main browser scrollbar
2. user scrolls the nested scrollable container, and then scrolls the page with the main browser scrollbar
3. user resizes the browser window in a way that makes elements invisible
4. user resizes a resizable element (CSS resize: both;) in a way that makes nested elements invisible

---

Have a look at my test page and you will understand what I mean. Try to carry out all described use cases with that page. On bottom of that page you find the current JS source code, and you can try out if this also works with the browser of your choice.

LINUX Root Password Confusion

From time to time it is recommendable to change your password. When doing so, you meet the problem of sitting before some application that requires a password, thinking "Did I already change the password here too?".

Surely you know the situation. I met it when trying to upgrade my netbook's Lubuntu LINUX. The update-dialog was asking me for a password. Because the dialog did not proceed from receiving my new password, I decided that it is time to change the root password to my new password on this machine, canceled the software update and entered the following in a terminal window (the '$' is the UNIX prompt):

$ su
Password: oldPassword
$ passwd
Enter new UNIX password: newPassword
Retype new UNIX password: newPassword
passwd: password updated successfully

Starting the software updater then showed the same as before: the password was not accepted. I rebooted the machine to clear all "shadows" of the old password, but that did not help.

Then I searched the Internet for a way to change the root password on LINUX. I found some confusing discussions that Ubuntu has no root password any more, or that root access is done via sudo and the user-password is sufficient for this. But what about the software updater then?

I also tried this:

$ sudo passwd root
[sudo] password for xxx: oldPassword
Enter new UNIX password: newPassword
Retype new UNIX password: newPassword
passwd: password updated successfully

Did not help. The software updater denied the new password. I evaluated this also by calling the menu item "System" - "Network", there you have a button "Unlock" to get root access to network resources. Also "Synaptic" requires root permissions, and it rejected the new password, too.

Now I logged out and tried to log-in as root, to find out what the new root password has become. And it turned out to be the new one, so at least one of the methods above worked, most likely both!

---

This was the moment when I understood that the password that I have to enter for the software updater, or sudo or su, is not the root password. It is my normal user-password. I hadn't yet changed my user password on that computer.

$ passwd
Changing password for xxx.
(current) UNIX password: oldPassword
Enter new UNIX password: newPassword
Retype new UNIX password: newPassword

After that, the software updater accepted the new password and started to work.

---

That means I had changed the root password as normal user, using my user-password, not the root-password.
Normally you can not interact with a UNIX system without logging in, using your user-password. So why does the terminal window ask me for a password that I already had entered when logging into the system?
And why does it ask me for my own password instead of the root password when I want to get root by sudo or su? And why is my normal password sufficient to change the root password? Couldn't I change then anybodies password on this system?

Yes, because I am in the "sudo" group that has administrative permissions. Discussion and answers to some of these questions can be found here:

• https://help.ubuntu.com/community/RootSudo

Ubuntu has become a cooperative system where users are expected to behave properly.

After reading this I did

$ sudo passwd -dl root

to disable the root account again.