Failing The Wrong Way

A lot of people love gmail because it filters out all of their spam. “I never see any spam!” they say, proudly. But the problem is that gmail achieves this by being way too aggressive about classifying things as spam, and the result is that it loses a lot of legitimate emails, too.

So the user is left with one of three options:

  1. Have things go wrong when they miss important emails.
  2. Check their spam folder once a day or so to make sure they don’t miss any important email.
  3. Don’t use email for anything important.

Option #1 is terrible and option #3 is just another way of saying that gmail is a bad email client. But the funny thing about option #2 is that the user is actually reading more spam than I am with my spam filter configuration that allows all of the important email through and only a few spams. I never have to check my spam folder, which means seeing 0-4 spams a day in my regular inbox is reading through way less spam than if I had to check my spam folder.

This relates to the concept in engineering of “which way do you want to fail?” It’s almost never the case that one can do something perfectly—getting absolutely every classification of email right. And every system is going to have a bias—would you rather when it fails your spam filter tends to mis-classify legit email as spam or spam as legit email?

The problem with focusing too much on getting the system perfect is that one can too easily forget that it won’t be perfect anyway, and then one won’t think about how it will fail when it does. A better engineered system puts some thought into figuring out the systemic biases and tweaking them to do the least harm, while also trying to get as close to perfect as is practical without changing the general target of how failure will happen.

Because failing in the wrong direction can be worse than useless. It can be actively harmful.

(The same principle applies to social engineering, by the way.)

Getting Cinelerra to work with a Webcam Video

This is just a quick technical post which I’m leaving more for anyone googling around trying to figure out what was wrong and how to solve it. It’s going to be pretty much nothing like my normal posts.

This is all taking place on Ubuntu 16.04 using a Logitech C920, Cheese to record the webcam video, and Cinelerra compiled from the official upstream (though the same thing happened in cinelerra-cv). I recorded the video in 720p even though the camera supports higher because it saves disk space and my intro and outro are in 720p. Plus it’s just me talking, not something interesting to look at, so I don’t think that the extra disk space, upload time, etc. is worth the trouble, especially since most people stream 720p not 1080p (I believe; I haven’t looked up official youtube statistics if there are any). What Cheese produces—I’d say “by default” but about the only option you get is the recording resolution—is a .webm file, which is a matroska container with VP8 as the video codec and vorbis as the audio codec.

Initially this looked like it worked well as it imported into cinelerra. But then I ran into the first problem: for some reason the video was marked as having a frame rate of 250.0, not 25.0. This caused cinelerra to play it at 10x fast forward. That was fixed easily enough by right-clicking on the media clip and choosing info and setting the framerate to 25. But then I hit the bigger problem which took me quite a while to solve: my voice sounded like the chipmunk effect had been applied to it. Also it was out of sync and then did weird, repeaty things when it ran out but there was still video to go. Of course that sounds like an audio-rate mismatch, but it wasn’t that. I triple-checked that one. Finally I noticed that there was a single channel in the audio stream. Well, actually, I kind if figured that out when inserting the video in cinelerra only pasted one track and I had to duplicate it, but I didn’t notice that for some reason cinelerra thought that the video had two audio channels. I’m not sure if this is a limitation in cinelerra or what, but the number of audio channels was not configurable. So I used ffmpeg to convert the video to stereo with 25fps, and it worked perfectly in cinelerra. For reference, here’s the command like I used:

ffmpeg -i ~/Videos/Webcam/2016-12-14-173128.webm -s hd720 -r 25 -ac 2 -strict -2 output.mp4

I converted to mp4 for an inconsequential reason, I think it would have worked as well to convert to VP8. Also you may notice I scaled to 720p, because I did actually record the original in 1080p and I wanted it to make my intro and outro video segments which as I said above were in 1080p. I believe this (which leaves the video unchanged) would work too:

ffmpeg -i ~/Videos/Webcam/2016-12-14-173128.webm -r 25 -ac 2 -c:v copy -c:a vorbis -strict -2 test.webm

A quick test showed that video to be slightly out of sync, so I’m not sure what that’s about, but something like it would probably work.

That being said, I’m switching to guvcview. It’s got far more options, can pull video at the full 30fps the camera can deliver it (using the camera’s built-in h.264 compression), and can record in stereo audio and video marked at the correct framerate. If I could just figure out some way of getting it to record the h.264 coming off of the camera rather than re-encoding it, that would be even better, but it’s not the end of the world to re-encode it. My computer is far more than fast enough to re-encode in h264 or vp8.

Good luck.