Experiment: Circular Quay Timelapse (14 Aug 15)

Around three weeks ago, on a Friday, I was feeling a little bored at home, so I decided to take advantage of the good weather and head out to do some experimental time-lapse photography. After all, I did have a “barely used” Sevenoak SK-EBH01 to play with, and I have been working around a basic Lightroom + ffmpeg + Handbrake workflow as in my previous timelapses. A timelapse thrives on movement and busyness, so where else would be better than to try it at Circular Quay, the central “ferry terminal” of Sydney. Of course, there is always more to learn, so I suppose I should share some of my experiences.

Shooting

The process of shooting a timelapse is usually rather straightforward. I carry out my Nikon D3200, with my lens of choice, a fully charged EN-EL14a “high capacity” battery, a blank 128Gb SDXC card, an intervalometer, a tripod and the rotating head. Once on site, take a walk around, scope out the surroundings, and decide on a good location, set-up and let it go.

This time around, I wanted to try something a little different – all of my time lapses to date were all done using the Samyang 8mm f/3.5 CS fisheye lens, so I wanted to see what it would be like doing it with a rectilinear lens instead. As a result, this time around, I used my Tamron 17-50mm f/2.8 lens, at 17mm instead.

Another difference was that I wanted to capture a full 360 degree time lapse (as I hadn’t done that yet with this rotating head) and exceed that if possible. I wanted to see how practical it was to “baby” the unit, set it to 180 degrees over 1 hour, and click it on start (as the unit has only a limited selection of rotation times and angles) once it finishes, so I can capture more action. In all, I was expecting to be there for about two and a half hours – making this the longest time lapse I’ve captured so far.

The idea to take some time to film a timelapse actually hit me on Monday, when I visited Milsons Point, only to find that the vantage point was spoilt by fencing and signage which precluded a nice wide field of view. From there, it seemed that Circular Quay was the best side to be on. When I got to Circular Quay, I decided to survey the MCA side of the foreshore, walking all the way to the bridge itself. Unfortunately, at the most advantageous positions, it seems the fencing was even taller and more insurmountable.

In the end, I settled on a position at the end of the public walkway near the Overseas Passenger Terminal, directly opposite the Opera House. Being right at the end of the unfenced area would keep my camera away from fences and people, make it more obvious to people, and give me the widest field of view. I stood around, waiting considerately for fellow tourists and photographers to finish up their shots, before hopping in. After a few trial shots, I pressed the start button on the rotating head and intervalometer, and settled in for a long wait – entertaining myself on my smartphone and having a chat on the phone as well.

Unfortunately, when you choose a good photography position, others want in too, and some people with more guts than me were willing to risk walking on a thin ledge to clamber in front of the camera to get a shot. I would have preferred if they were a little more considerate of my setup, because upon reviewing the data, it seems that someone had bumped the tripod early on, spoiling most of the two and a half hour shoot. How very annoying, but that’s a very “tourist” thing to do, I suppose.

Aside from that, you also get hassled. I thought I knew better, so I tend to choose outdoor locations, in public areas, where there is generally no rules against photography. Unfortunately that wasn’t enough. A random person approached me in a confrontational tone, demanding to know if they had been photographed and requesting deletion of their image. I was caught off guard – since it had never happened to me before, and I generally do my best to be considerate and “obvious” in my shooting. After confirming their location and the fact they would have been photographed, I calmly explained that it was a time-lapse project that had taken two hours to compile at the stage the person had intruded into my shot and that deletion during a shoot is technically difficult to achieve. The person then changed tact slightly, begrudgingly agreeing that it was too much of a demand that I spoil my work, but at the same time alluding that I was likely to be a pedophile photographing children at a beach and complaining that I needed a sign or something obvious as a tripod, with a flashing rotator unit and a “noisy” shutter wasn’t enough. In the name of harmony, I agreed to consider this in the future … but it is unlikely to have solved anything as they would just complain that the signs were too small, blown away, or they just plain “didn’t notice it” and somehow that would be my fault.

Once the set had been finished, the D3200 impressively took a total of over 5740 (!!) RAW shots (just 95 shy of filling the card) on just one EN-EL14a battery and one 128Gb SDXC card. I had a large haul of data to carry home.

The Rules

One of the outcomes of the little scuffle was to actually find out what the rules surrounding photography around Australia is. As it turns out, as much as I thought I actually knew the rules, there are always surprises.

As far as things go, in Australia, people can have no expectation of privacy in a public place, and photography is generally of no issue provided it is done in a public area without prying into areas where reasonable expectations of privacy can be had (i.e. through windows). Complying with requests for deletion are generally not required. It seems that things only get problematic where there are prominently identifiable people and commercial interests are involved, where commercial interests mean the usage of a photograph to promote a product, or where there are photographs of copyright-protected monuments, although these are not always clearly denoted.

Of course, there are always misunderstandings in regards to public areas, with many public areas turning out to be not public areas at all. This includes supermarkets, shopping centres, markets, train stations and interestingly, also includes exceptions for the Sydney Harbour Foreshore Area, Olympic Park and Luna Park which I wasn’t aware of. In the case of Circular Quay, it is part of the Sydney Harbour Foreshore rules, which seems to require the application for a permit for shooting for commercial purposes. As I wasn’t shooting for commercial purposes, I wasn’t covered by this, and looking at the permit application process, they were not going to bother issuing a permit for those with a crew under 10 anyway.

In all, it seems that I was clear of any wrong-doing from what I can tell, but it is disappointing to note that there are murky rules surrounding photography that need to be checked thoroughly, especially when travelling in unfamiliar areas. Sometimes it can feel like an attack on the art and hobby of photography, and it can make it much more tiresome exercise. It seems that many times, the rules are written but not strictly enforced or enforced inconsistently or when it’s convenient. I’ll have to keep that in mind in the future – after all, a tripod is generally a trouble magnet, but it’s an essential part of making a timelapse.

Processing

As with most of my other time lapses, the processing workflow involved downloading all the images and processing them first through Lightroom, to maximise the dynamic range, crop and resize, lens correct and remove chromatic aberration. As this was a set of about 110Gb of 24.2MP RAW images, it took more than 24 hours to actually export them to TIFF, with the right sequenced file names and watermarks embedded. I then used another image management program to convert them to BMP so that ffmpeg could assemble it into an .avi, losslessly compressed in FFV1, to then be pulled into Handbrake for the final encode using x264.

However, this time, I decided we would take on an additional challenge. Seeing as one person objected to being in the time-lapse, I decided to do some work to remove that person. This involved looking for the frames containing that person (localized in time) and then trying to “punch” them out by using earlier frames to compensate for missing data. I decided to do this in Photoshop, primarily using the frame just prior to intrusion, and the frame just past the person leaving the frame.

ghost-person

Sadly, things didn’t turn out as good as I expected – and partly because of my amateurish Photoshop skills. Namely, the first issue is that of object alignment – manual alignment can cause jittering, and as my rotating time-lapse system wasn’t rotating around the hyperfocal point, you get parallax errors as well between shots. The second was slight variations in exposure, which results in a “ghost” person appearing, which is probably due to changes in the sun. Another issue is that the sun moves, so the shadows might not align, and that in the areas where the person was “punched out”, the background remains static, which is relatively obvious in a time-lapse video scenario.

It also was quite time consuming. In all, I find it very hard to believe the person was genuine about their desire not to be photographed, as the person remained in the frame for a full 40-shots, representing 80 seconds of real time, shooting the area with their camera phone, including my set-up. If you’re afraid you might have been captured, and it was your primary concern, you would take steps to remove yourself from the frame immediately especially after seeing the equipment take a shot of you. In the hours it took me to punch her out, I felt a sense of betrayal, and even though I didn’t have to remove the person, their removal turns them into a “ghost feature” as well.

The final “raw” product of the basic workflow looks as following (4k resolution available):

The Issue of Flicker

In the past, the assembled timelapses have had imperfections, but I never really worried. This timelapse, more than the others, had quite a few imperfections – and not just getting the tripod bumped. Watching it carefully, especially towards the end, the sky can be seen to “strobe” quite violently. This fast-changing brightness is flicker, and despite setting manual focus, a fixed shutter speed and aperture, can come about because of:

  • Mechanical shutter instability – the timing of the shutter may vary especially when the shutter ages, or due to changes in environmental and operating conditions.
  • Mechanical aperture diaphragm instability – mechanical tolerances may mean that electronically actuated diaphragms might not settle in the right size every time.

Because it is intrinsically mechanically related issue, it seems eliminating it entirely in shooting is not necessarily possible. However, my other time lapses to date have had less of an issue, because it used the Samyang 8mm fisheye with a manual aperture ring, which eliminates one of the issues. The other thing seems to be that the flicker gets worse towards the end – this may be attributed to the shutter in my D3200 ageing (it’s over 88,000 actuations), warming up in the sun, or more likely due to shutter variation when the battery is low. So maybe it’s a good idea to shoot less, and not to shoot till the battery dies.

But of course, all is not lost. Some help might be found in post-processing. A few paid-for products are available, with some of them working at the RAW metadata level. Others are available for free, but most of them were designed for timelapses where we have a fixed point of view, by mathematically looking at pixel values. This wouldn’t work very well in a rotating timelapse. Could there be approaches which work well with rotating timelapses?

Another approach is to process the video frames themselves. This isn’t an optimal solution, as a lot of metadata is lost, and further “tweaking” of the limited 8-bits per channel result is unlikely to result in a high quality result. However, for beginners like myself, we always gravitate towards the free options first. It never hurts to try.

Veterans of video editing on the Windows platform will be well aware of Avery Lee’s Virtualdub. This is a basic but powerful video editing software for manipulating AVI files, and can work with a variety of .vdf plugin files for filtering and effects. It has many different options, which are useful for those looking to maximise quality in their editing workflow. Of the plugins available for Virtualdub which can be used for correcting flicker are Donald Graft’s Deflicker and MSU Deflicker.

Where are Donald Graft’s VirtualDub Filters?

A while back, Donald Graft of http://neuron2.net was put in charge of cataloging Virtualdub filters, and even to this date, his site is still mentioned by Virtualdub.org as being the main source of filters. If you try to visit it today, you will see that it’s been taken over by a domain parking mob – no longer existing.

dg-domainpark

It seems many people have had difficulty finding the filters as well, with many requests for working download links.

Luckily, as most of the tilers were relatively small ZIP files, and the site was mostly static, the US Archive’s Wayback Machine has a working mirror of virutally all of the filters that were hosted there. You can find the page, but the sidebar navigation is broken in the copy, so here is a direct link to the archived Filters page.

A VfW Hurdle

Utilizing my regular ffmpeg workflow, but instead using .bmp files converted from .tiffs, I now render to FFV1 encoded lossless files (i.e. -c:v ffv1) for further processing, only departing from FFV1 in the final encode stage rather than work with JPEGs as MJPEG. This has the advantage of allowing you to perform multiple passes of modifications maintaining the highest fidelity possible – especially since the images were exported from Lightroom in TIFF format.

Sadly, it seems that the VfW compatible FFV1 decoder in ffdshow just doesn’t work with the latest FFV1 encodings. As a result, when in Virtualdub, you end up seeing an error (VideoSourceAVI error: The operation is not supported. (error code -1)).

virtualdub-not-supported

As it turns out, you have to encode as a “level 1” FFV1 stream for it to work with ffdshow. This can be achieved by changing how you use ffmpeg, by adding the -level 1 argument after declaring the video codec like as follows:

ffmpeg -i input.avi -c:v ffv1 -level 1 -c:a copy output.avi

Of course, this can be done to an existing FFV1 AVI file which was encoded with the latest version in a lossless fashion. The resulting file now loads and processes properly with Virtualdub.

Donald Graft’s Deflicker

Installing Virtualdub filters is as simple as unzipping the .vdf file into the plugins directory inside your Virtualdub folder. Then it’s a case of loading Virtualdub, loading your source file, configuring your filter chain, output options and then saving the output.

This deflicker filter seems to be designed for filtering flicker from recorded film projections which is periodic, and some of the options are likely to cause degradation in the sharpness of the output. Donald Graft’s Deflicker filer doesn’t have many settings to configure, however, as it utilizes a windowing algorithm, it should be able to cope with non-periodic flicker as well.

donald-graft-deflicker

In my case, I increased the Window size, while disabling Softening and Scene change detection. The output looks like this:

The effects are subtle and not immediately obvious – so I decided to make a side-by-side comparison video with the original source RAW footage on the left, and the deflickered output on the right. This was done by using ffmpeg, again, with a filter complex defined as follows (split into multiple lines for legibility):

ffmpeg -i left.avi -i right.avi -filter_complex 
"nullsrc=size=3840x2160:r=30 [bg]; 
[0:v] setpts=PTS-STARTPTS,crop=w=1920:h=2160:x=0:y=0 [lt]; 
[1:v] setpts=PTS-STARTPTS,crop=w=1920:h=2160:x=1920:y=0 [rt]; 
[bg][lt] overlay=shortest=1:format=rgb [bgl]; 
[bgl][rt] overlay=shortest=1:x=1920:format=rgb" 
-c:v ffv1 -c:a copy output.avi

Sadly, I don’t think the deflickered footage is preferable. While it hasn’t produced significantly adverse output, it hasn’t been able to remove the flicker and instead introduces new artifacts where intentional changes in brightness due to people moving in and out and the changing scene causes the image brightness to “pump” similar to how a microphone AGC distorts the audio signal in the presence of fast changing volume levels. Some of the early subtle flicker was removed, although the strong strobing towards the end still persists.

MSU Deflicker

Another commonly used flicker filter is from MSU.

MSU Graphics&Media Lab, Video Group, MSU filters and codecs
http://www.compression.ru/video/

This filter is a little more advanced and claims to be better at retaining intentional changes in brightness from the source footage. I have decided to disable blending, to prevent loss of sharpness.

msu-deflicker-dialogue

The result of MSU deflicker is a little different. I had initial problems getting 4k footage to process correctly with strange patterns appearing in the output. After a restart, it seemed to work correctly, but the output contrast can be seen to be compromised already.

A side by side comparison can be seen below.

Sadly, I can’t say I prefer this version over the original either – the loss of contrast detracts from the aesthetics of the original, and the flicker is still present in some areas, with the “AGC” like effect being visible especially where people are involved. As expected, a post-processing filter isn’t exactly an oracle – it doesn’t know what is intentional changes in brightness, and what isn’t, so motion timelapse deflickering in post processing is a difficult challenge which may never be perfected. As a result, it is preferable to reduce any flickering in the capture process and live with whatever the output may be.

Conclusion

This exercise was a rather interesting expedition in itself, and resulted in several key takeaways:

  • Check the rules before you photograph, if you can actually understand them, and then realize that many of the rules aren’t actively enforced unless you’re posing problems.
  • Some people will try to hassle you just because they can, and sadly, there’s not much we can do about them. But avoiding a protracted confrontation is always desirable, so if it’s not too much, you should just give into their demands if it’s not too much.
  • Flicker from mechanical sources in shooting time-lapses with DSLRs seems to be unavoidable, but can be minimised especially with manual lenses with fixed apertures. It seems also desirable not to let your battery run down to empty, especially if your camera’s shutter performs inconsistently under those conditions. The battery did last a surprisingly long time though …
  • Perhaps shooting with a shutterless ILC camera is a better idea.
  • Donald Graft’s filters are still available from the archives kept by archive.org.
  • Deflicker filters are not oracles that can separate intentional flicker from unintentional flicker, and neither can they correct for more severe non-periodic flicker, and instead can produce their own artifacts. Whether this is preferable or not will depend on the scene being shot.
  • Rotating time lapses pose additional challenges as it can introduce intentional flicker into the scene.
  • ffmpeg produces FFV1 encoded .avi files with a newer encoding that ffdshow’s VfW filter cannot understand – so in case of problems opening FFV1 files in Virtualdub, it may be needed to transcode from FFV1 to FFV1 (level 1).

Every day is another learning experience.

Posted in Photography | Tagged , , , , | Leave a comment

Surprise Package: The MagPi Arrives, by Air, in Hard Copy!

magpi-envelopeIt’s not often, but sometimes things just turn up at your doorstep when you least expect it. Earlier this week, I received this mysterious envelope all the way from element14’s UK office. I wonder what it is …

As you can see, I wasted no time in finding out – tearing the envelope apart before I had even scanned it in.

As it turns out, it is a hard-copy edition of the official Raspberry Pi magazine, cunningly named, The MagPi.

magpi-cover

For those who don’t already know, The MagPi is the one and only official Raspberry Pi magazine. Released on a monthly basis since May 2012, it is a sort of “community” fan-zine turned semi-professional. Unlike traditional magazines, The MagPi started life in electronic form, licensed under Creative Commons (BY-SA-NC 3.0), meaning that the contents of the magazine are essentially “free” to read and reproduce.

From there, Issue 36 (August 2015) marks a milestone for The MagPi and The Raspberry Pi Foundation, as it is the first copy to go out in hard copy print form. The hard copy magazines have an issue price of GBP 5.99, and is available in newsagents and WH Smiths in the UK only, with expansion to US soon. Subscriptions are also being sold at this time, for six months or 1 year, costing GBP 55 for UK, GBP 80 for EU and GBP 90 for “rest of the world” for a one year subscription. Funds raised will go towards supporting the foundation in its goals.

20150819-2037-4776

It was definitely quite exciting to have a hard copy in my hand, knowing that very few others on this side of the world would have the luxury of feeling a nice matte and selective-gloss cover and being able to flip through the pages.

In all, I found The MagPi to contain a good assortment of projects and be quite readable, with different levels of finish and difficulty, and a good way to inspire yourself and educate yourself about the possibilities already existing when it comes to using the Raspberry Pi. Not every project will appeal to every reader, but within its 100 pages, it’s not a “lightweight” by any measure. It doesn’t leave the newcomers out in the cold, with tutorials and FAQ sections, as well as “features” which cover some of the more mundane topics. There are also reviews of hardware, which are always interesting to see, as well as relevant advertising (although many of them UK based) which shows you just what sort of whacky expansion “hats” are being produced, many I wasn’t even aware of. It is quite a substantial effort, and the print itself is of very good quality.

It’s worth a read if you have some down-time, run out of ideas entirely or you’re just interested in learning more about the Raspberry Pi. Best of all, you can always read it free – you can’t beat the low, low price of free! Of course, if you would like to support the foundation, it’s always a good idea to subscribe.

rear-ad-funny

The back cover ad did give me a little bit of a laugh though. I’m sure they mean 192kHz, not 192Mhz, otherwise these boards would make a pretty good oscilloscope …

Conclusion

I congratulate The Raspberry Pi Foundation and The MagPi for reaching the milestone of releasing their first print edition (#36) – it really shows just how popular and useful the Raspberry Pi has become to “everyday” people and engineers alike. I really hope this works out well for them, and I think the hard copy will bring awareness to a whole different audience of people – even if just to get people talking about it. I will definitely treasure this physical copy – thanks element14!

Posted in Computing, Opinion, Raspberry Pi | Tagged , , | 1 Comment

Tech Flashback: Fellowes Neato 2000 CD Labeler Kit

It’s the year 2000 and CD recording at home is a new development. Discs are expensive, and individually wrapped in retail packaging. These discs will have branding all over their faces and inserts. Printable discs didn’t really exist yet in the retail market, with thermal printable discs for replicators with expensive thermal printers and ribbons involved. Inkjet disc printing, let alone affordable disc printers, were still many years to come. But what if you wanted to make a small run of discs for a semi-professional purpose? It would not be economical to visit a replicator to do it, and it would take a while for them to get the job done. Wouldn’t it be nice to have a way to do it yourself at home?

The Fellowes Neato 2000 System

The solution, of course, was an “adhesive label” style system, branded as Neato 2000 and sold by Fellowes (an office supplies company). Kits were widely available, and sold even in department stores so that your “average joe” could now enjoy professional (ahem) results. I had never used one of these kits in my life, mainly because I never really needed for my discs to “look good”, and I had been warned off of them because of potential issues of unbalance, added weight to discs, labels falling off and data layer damage.

However, since one of these kits showed up, donated to me by a friend, I decided to see what was in one of them.

20150816-1359-4762

As you can see, the box itself is very colourful, and tries to make the whole process seem relatively straightforward, while advertising the availability of not just a label for the disc, but also inserts for the jewel case. As expected, the disc in the illustration is a period correct Mitsui Gold CD-R – one of the most highly regarded blank discs of the era. The package is a kit, containing the software itself called MediaFACE, the applicator itself, and a set of paper stocks in photo-quality gloss and matte finishes for inkjet and (some) laser printer use.

Interestingly, Neato is still around today, and their MediaFACE software is now up to Version 5 (for purchase or free with label order) and also available as a web Flash-based application with free demo online.

20150816-1400-4765

20150816-1402-4769The rear of the box elaborates on the possibilities, and tries to sell other labels which are available and compatible with the software. The item itself is Order Code 99960. Software requirements are PC  or Mac, with Windows 3.x, 95 and 98 supported. I’m sure rivals, such as Avery, would have had their own complete solution as well.

A pressed CD is included with the software. This was written back when 8.3-filenames were normal, and the “high resolution” graphics were all 150dpi. Yeah, high-res!

20150816-1401-4767

The applicator itself consists of a spindle-like shaped device with a spring-raised centre that supports the hub ring of the CD. A label is printed, and peeled so that the sticky side is placed facing up on the applicator. A disc is then placed on the applicator, with the top facing the sticky label, and then the central hub is pushed down to “drop” the disc onto the label to apply it.

20150816-1400-4766

A very primitive system, but I suppose if the labels are made with a good tolerance, the centering can be kept to a certain amount of error. It still doesn’t solve the issue of the added weight of the label causing strain on the drive, and the label itself potentially stressing the reflective coating on the disc.

Included in the package is a set of label stock, containing a few sheets of each type.

A4CD2lbl A4CD2lbl-HG

Each sheet has a header with the title of the label type, with the second line containing the template file name, in this case A4CD2lbl.NTT. In this case, the left sheet is a matte type label, and the right is a gloss type label – the gloss type can only be used in inkjet printers, although the documentation does claim that the best results are had with inkjet printing likely because any “bending” of the label causes it to detach from the sheet, might cause the toner to lift from the paper and the heat may affect the adhesive. Two discs labels are printed on a single sheet, which could be a slight inconvenience – and of course, printer calibration is mandatory if you want to have your labels very accurately placed.

Interestingly, the paper stock itself has MediaFACEII on the header, which is newer than the software supplied, so this paper stock may have come from a re-order packet.

A4JCtray A4JCtray-HG

Of course, the label is not the only thing that can be printed – the packet also consists of tray liner templates. This allows you to print the rear insert on pre-perforated stock, tear the edges away, and then fit it to your standard jewel case.

A4JCbook-A A4JCbook-B A4JCbook-HG-A A4JCbook-HG-B

All of the previous stock was single sided, except this one – the Jewel case insert. The design of this was also perforated for easy separating after printing, and allows you to print a “folded” card with front and back, and insides. It’s pretty close to what you can get professionally, although to make the final “quality” result requires that the user prints it with a high quality printer and uses high quality fonts and graphics. Most inkjet printers at the time hadn’t even reached “photo” quality yet … unfortunately.

The CD containing the software can be installed by running SETUP.EXE. Interestingly, it succeeded under a Windows 7 x64 machine, with no trouble at all. I could even run the software. Wow.

mainui

about-mediafaceexample-label

I checked the software “about” for more information, and confirmed that this was an early version of their software. A very nice surprise to see that it works! I then started a basic label project for the above sheets, using some of the included stock images, and proceeded to print it to my local PDF printer. It worked! What a surprise.

As to making and mounting a label … I don’t think that’s necessary. The procedure is pretty much obvious.

Conclusion

If you wanted labels, you really had to pay for the privilege. Buying such a kit with a few sheets wasn’t that cheap from memory, and when you stuffed up or ran out, you’d have to spend quite a bit more buying additional label stock. For a while, it was cheaper than some other alternatives, like early inkjet printing and thermal printing since those solutions required expensive cartridges and ribbons and meant that you had to buy more expensive printable media. As photorealistic printers became more common, printed label quality was better, but they still couldn’t reach the “full coverage” edge-to-edge coverage you would get with full-surface printable discs.

It was especially surprising to see that the software, owing to its very basic features, installs and runs without modification on a modern Windows 7 x64 machine, and can still print labels to modern printers. That being said, optical discs are fading from relevance, direct inkjet printing is now affordable and comes bundled with their own software and templates, inkjet printable discs are widely available in many finishes, and is the preferred technology for labelling discs today without the risk that the label could damage the data layer, unbalance the disc, add too much weight to the disc or peel off the disc in use.

Posted in Computing | Tagged , , , | Leave a comment